[RFC PATCH] Limit directory child dentry retention

[Ian Kent <raven@themaw.net>]

Hi all,

Please forgive the long description but this seems to be a long standing
problem so I'd like to offer a suggestion for improvenment (but probably
not a complete solution).

I have seen a problem where, in one case, a directory has around 12M
children with about 80% of them negative. Then an fsnotify function is
called that needs to traverse the entire list of directory children
while holding the inode i_lock with obvious consequences.

Some time ago commit 681ce8623567 ("vfs: Delete the associated dentry
when deleting a file") was merged to try and solve excessive accumulation
of negative dentries. It was later Reverted in Commit 4a4be1ad3a6e due to
performance regressions. In addition commit 172e422ffea2 ("fsnotify: clear
PARENT_WATCHED flags lazily" was suggested as a fix but one of the reports
we have triggeres the problem via fsnotify_add_mark_locked() which still
traverses the entire child list after commit 172e422ffea2 is applied.

Having worked though the above commits (and the revert) it occured to me
that a similarly simple approach would be to only limit directory dentry
retention when some highwater level was reached. A kind of keep a bunch
of negative dentries to try not to interfere with the dcache but discard
them on last dput if there are so many child dentries that the benifit
of caching them would likely be negated. TBH I don't know what that
highwater value should be so that's one thing to be worked out.

I don't have a reproducer but I presume the people I've included on the
cc list may have some tests. TBH I don't see how Commit 681ce8623567
caused a regression since that should have required a re-create (probably
many) for the same file to cause it. Nevertheless the patch here is yet
another a very simple approach to helping with the stale directory dentry
accumulation problem we see all too often.

Thoughts and comments please.

Ian Kent (1):
  vfs: limit directory child dentry retention

 Documentation/admin-guide/sysctl/fs.rst |  7 +++++++
 fs/dcache.c                             | 28 +++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

-- 
2.53.0