[PATCH RFC 00/12] vfs, nfs: concurrent create and unlink in one directory

[Jonathan Flynn <jonathan.flynn@hammerspace.com>]

Hi Neil,

Thank you for this series; it is what makes these patches possible.  I
had them on your v3 and have rebased them onto v4.  They extend it so
that processes on one NFSv4 client can create and remove files in one
directory concurrently.

  operations per second   parent exclusive   parent shared   ratio
  creates                            2,616          29,939   11.4x
  removals                           2,157          32,360   15.0x

One client, 96 processes in one directory, each creating files (open
O_CREAT, a small write, close) for a set time, then removing them all;
NFSv4.2 to a pNFS metadata server that is not knfsd; your v4, the 7.4
VFS changes it builds on, the three silly-delete fixes and this series
on a 7.3-based kernel, among other patches.  Each row switched only its
own path, by a test-only module parameter not in this series, with the
other path on; one run each.  The creates row comes from 300 s of
creates.  The removals row comes from 120 s of creates, and each of its
runs removed about 3.7 million files; the removal rates vary with that
number, so it is given.  With the parent held exclusive, operations go
one server round trip at a time.  2/12 quotes an earlier measurement,
on a 7.1.13-based kernel.

How it works.  2/12 lets a filesystem have the parent locked shared for
O_CREAT (a flag in a new ->iop_flags field), and 6/12 for unlink(2)
(->unlink_shared() and ->unlink_shared_done()).  The dentry then
excludes the name: for a create, its in-lookup state, all
nfs_atomic_open() needs since your 3/9; for a removal, an NFS mark that
lookups, opens and creates of the name wait on until
->unlink_shared_done(), which the VFS calls after the fsnotify delete
event, if any.  What NFS cannot order that way, such as an open file,
it hands back to ->unlink() under the exclusive lock; where the server
reports itself case-insensitive or not case-preserving, NFS sets
SB_I_NAMES_COLLIDE (3/12) and both keep the exclusive lock.  Both paths
call d_alloc_parallel() under the shared lock, as lookup does.  When
your directory-locking changes lift d_alloc_parallel() above i_rwsem,
these calls have to move with lookup's; the exclusion does not change,
and the opt-ins go once the VFS stops taking the parent lock (2/12,
6/12).

Patch flow:
  prereq   1  store re-read server capabilities in one step, for 3
  create   2  VFS opt-in to take the parent lock shared for O_CREAT
           3  NFS sets SB_I_NAMES_COLLIDE where names can collide
           4  NFSv4 directories opt in for O_CREAT
  remove   5  factor vfs_unlink() into a helper, no functional change
           6  VFS opt-in: ->unlink_shared(), ->unlink_shared_done()
           7  a second ->d_fsdata mark, for removals under a shared lock
           8  keep a dentry carrying that mark hashed
           9  no new dentries from a listing that a removal overlapped
          10  the removal method, handing back what it cannot order
          11  look a name up again once its removal has unhashed it
          12  NFSv4 directories opt in for unlink

Three fixes for the silly-delete hand-off, which your 2/9 makes
reachable, are posted separately, in reply to 2/9.

Tested on the same kernel over NFSv4.2, with the new paths on and off:
removal races (with listings, and with tasks killed); O_EXCL creates
racing unlink, with a kernel-side check that no revalidation trusted a
removed entry; inotify event order; two clients; and an overlayfs mount
whose lower file another client renamed away and back.  All passed,
with no kernel warning or hung task, except the two-client test's check
that nothing was left behind: on this kernel and on one with an earlier
revision of this series, it found one silly-renamed file in three of
four runs, with the new paths off as well as on, so they are not the
cause.  Each time, the file left behind was not the one that had been
silly-renamed: the silly-rename's RENAME, which goes by name, had moved
another file of that name onto the hidden name.  A PROVE_LOCKING and
KASAN build ran the removal, killed-task and O_EXCL tests with no
report.  A case-insensitive server, where two spellings of a name never
made two files, was tested on that earlier revision only; that code has
not changed since.  Not run: fstests, knfsd as the server, NFSv4.0 or
4.1, directory delegations (the server grants none), CONFIG_DEBUG_VFS, a
weakly ordered CPU.

One question: are the opt-ins in 2/12 and 6/12 acceptable as interim
interfaces until the directory-locking changes land?

The series applies to 83f32c773255 in Linus's tree on top of your v4
and the three silly-delete fixes.  fs/ compiles there without warnings
at every commit, with and without CONFIG_DEBUG_VFS and lockdep; I have
run it only on the 7.3-based kernel above.

Tools: these patches, their changelogs, the race tests and this letter
were written with Claude Code, using Anthropic's Claude Opus 5 and
Claude Opus 5.5 models, at my direction over many sessions: I chose
among the designs it proposed and directed each revision.  It found, by
reading the code, the bug that 1/12 fixes: while the client reads a
server's capabilities again, a capability the server still offers can
look missing.  The three silly-delete fixes, posted separately, say
which of their problems it found and how they were tested.

Thanks,
Jonathan

Jonathan Flynn (12):
  nfs: update a server's capabilities in one step
  vfs: allow the parent lock to be shared for O_CREAT
  nfs: tell the VFS when two names can be one entry on the server
  nfs: concurrent O_CREAT in one NFSv4 directory
  vfs: factor file removal into a helper, no functional change
  vfs: let a filesystem remove files with the parent directory locked
    shared
  nfs: add a second mark for a removal made with the parent locked
    shared
  nfs: keep the dentry of a shared-parent removal hashed
  nfs: do not cache a name from a listing that a removal overlapped
  nfs: add the removal method for a parent directory locked shared
  nfs: look a name up again once its removal has unhashed it
  nfs: remove files with the parent directory locked shared on NFSv4

 .../filesystems/directory-locking.rst         |  60 +++
 Documentation/filesystems/locking.rst         | 259 ++++++++++++-
 Documentation/filesystems/porting.rst         |  35 ++
 Documentation/filesystems/vfs.rst             |  25 ++
 fs/namei.c                                    | 360 +++++++++++++++++-
 fs/nfs/client.c                               |   1 +
 fs/nfs/dir.c                                  | 300 ++++++++++++++-
 fs/nfs/inode.c                                |   2 +
 fs/nfs/internal.h                             | 211 ++++++++++
 fs/nfs/nfs4file.c                             |   9 +-
 fs/nfs/nfs4proc.c                             |  35 +-
 fs/nfs/nfstrace.h                             |   2 +
 fs/nfs/super.c                                |   2 +
 fs/nfs/unlink.c                               |  16 +-
 include/linux/fs.h                            |  72 ++++
 include/linux/fs/super_types.h                |   1 +
 include/linux/nfs_fs.h                        |  28 ++
 17 files changed, 1361 insertions(+), 57 deletions(-)


base-commit: 83f32c773255720a661aa45d83e36278135f23f5
prerequisite-patch-id: 3405b4d6ab8b0bf24ba63a898083d617deec4f24
prerequisite-patch-id: 096657d6d80d7ca533ccb574d2af65611ede4af2
prerequisite-patch-id: 1a2319c14739918aef86116647fea28ae85ad4df
prerequisite-patch-id: 725cb18c3936d50141ef27dafc159bf50b64a5bd
prerequisite-patch-id: b39cdd352b97e2cd247311fd5db620dbff227bd7
prerequisite-patch-id: f9f3f0e945cadf96113d4ac9ab8f7b0c207af4e5
prerequisite-patch-id: 4c09a15390bb556d79bea1ee733f52a58bec80ef
prerequisite-patch-id: 7c026717a97b2c86a169539822f5d14650f68f8a
prerequisite-patch-id: a702d54495d0f3d7f88b82411d884dcf5aacf9a0
prerequisite-patch-id: a7e7263e8916612f3e4b767946b5d9259029f50e
prerequisite-patch-id: a8e5414df9d9912dcdf9bb1dbb2811ce326a1e1f
prerequisite-patch-id: e8050b998d5790f3bb4311bc65011735ed7d8b2f
-- 
2.43.0