I've noticed that if you create e.g. 100k files to a directory and then
delete the files, the directory entry still seems to take a couple of
megabytes. Later whenever accessing the (almost empty) directory it can
take a few seconds to load it into cache.

Is there a way to shrink the directory somehow without having to rmdir()
it? Would be nice if kernel did it automatically, but I could live with
a manual userspace syscall/tool as well.