[PATCH v2 0/8] nfsd: reduce the scope of the nfsd_mutex

[PATCH v2 0/8] nfsd: reduce the scope of the nfsd_mutex

[Jeff Layton]

This version fixes a couple of minor issues that Sashiko flagged, and
also implements a suggestion by Neil to use guard() with the new
filecache mutex.

Original cover letter follows:

-------------------8<--------------------

Currently much of nfsd's administration is serialized under the global
nfsd_mutex, even though most of the objects managed under it are
segregated by net namespace.

This patchset does a couple of small cleanups and then adds a new
per-net mutex and moves all of the appropriate per-net data structures
to be protected by it instead of the global mutex. The filecache is
given its own mutex, and the global mutex is renamed and left for
managing just the few remaning global objects.

I also had an LLM cook up some selftests so we can hopefully keep all of
this working properly in the future.

Please consider for v7.4.

Thanks,
Jeff

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Changes in v2:
- Fix potential deadlock in cross-namespace isolation selftest
- Drop binaries from selftest patches
- Use guard() in nfsd_file_cache_init()
- Link to v1: https://lore.kernel.org/r/20260921-nfsd-per-net-mutex-v1-0-4a7287f6504c@kernel.org

---
Jeff Layton (8):
      nfsd: clear NFSD_NET_UP before dropping the generic resource reference
      nfsd: make max_blksize a per-namespace setting
      nfsd: move the control plane to a per-namespace mutex
      nfsd: rename nfsd_mutex to nfsd_global_mutex
      nfsd: give the open file cache its own mutex
      selftests/nfsd: factor the netlink plumbing into a shared header
      selftests/nfsd: add cross-namespace isolation tests
      selftests/nfsd: add a cross-namespace control-plane soak

 fs/nfsd/export.c                                   |  22 +-
 fs/nfsd/filecache.c                                |  26 +-
 fs/nfsd/netns.h                                    |  16 +
 fs/nfsd/nfs4proc.c                                 |   4 +-
 fs/nfsd/nfs4state.c                                |   8 +-
 fs/nfsd/nfsctl.c                                   | 129 ++---
 fs/nfsd/nfsd.h                                     |   4 +-
 fs/nfsd/nfssvc.c                                   | 146 ++++--
 tools/testing/selftests/nfsd/.gitignore            |   2 +
 tools/testing/selftests/nfsd/Makefile              |   2 +
 tools/testing/selftests/nfsd/config                |   6 +
 tools/testing/selftests/nfsd/nfsd_netlink.h        | 319 ++++++++++++
 .../testing/selftests/nfsd/nfsd_netlink_listener.c | 301 +----------
 .../testing/selftests/nfsd/nfsd_netns_isolation.c  | 468 +++++++++++++++++
 tools/testing/selftests/nfsd/nfsd_netns_stress.c   | 572 +++++++++++++++++++++
 tools/testing/selftests/nfsd/settings              |   2 +-
 16 files changed, 1592 insertions(+), 435 deletions(-)
---
base-commit: aeaddc6a3711283c09a9183068897d4fe1fb464f
change-id: 20260918-nfsd-per-net-mutex-8655764940ac

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>

[PATCH v2 1/8] nfsd: clear NFSD_NET_UP before dropping the generic resource reference

[Jeff Layton]

nfsd_shutdown_net() leaves NFSD_NET_UP set across nfsd_shutdown_generic(),
so during that window the bit claims the host-wide file cache and NFSv4
tables are up when the last namespace may already have torn them down.
Nothing observes it today only because one global mutex covers both.

Fold the test and clear into test_and_clear_bit() so the invariant
"NFSD_NET_UP implies the generic resources are up" holds on its own.
No functional change.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/nfssvc.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index cbc989238710..d75fe1523436 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -433,9 +433,13 @@ static void nfsd_shutdown_net(struct net *net)
 
 	percpu_ref_exit(&nn->nfsd_net_ref);
 
-	if (test_bit(NFSD_NET_UP, &nn->flags))
+	/*
+	 * Clear NFSD_NET_UP before dropping this namespace's reference on
+	 * the generic (host-wide) resources, so that the bit never claims
+	 * they are available once they may already be gone.
+	 */
+	if (test_and_clear_bit(NFSD_NET_UP, &nn->flags))
 		nfsd_shutdown_generic();
-	clear_bit(NFSD_NET_UP, &nn->flags);
 }
 
 static DEFINE_SPINLOCK(nfsd_notifier_lock);

-- 
2.55.0

[PATCH v2 2/8] nfsd: make max_blksize a per-namespace setting

[Jeff Layton]

nfsd_max_blksize is module-scope, but /proc/fs/nfsd/max_block_size is a
per-netns file. A container writing it therefore changes the payload size
that every other namespace's next server start will use.

Move it into struct nfsd_net as ->max_blksize. The lazy
nfsd_get_default_max_blksize() fill-in now happens per namespace on first
nfsd_create_serv().

User-visible change: max_block_size no longer leaks across namespaces.

Fixes: 11f779421a39 ("nfsd: containerize NFSd filesystem")
Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/netns.h  | 6 ++++++
 fs/nfsd/nfsctl.c | 8 +++-----
 fs/nfsd/nfsd.h   | 2 --
 fs/nfsd/nfssvc.c | 6 +++---
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
index 0ce7da20aba3..374ce83e2ba0 100644
--- a/fs/nfsd/netns.h
+++ b/fs/nfsd/netns.h
@@ -154,6 +154,12 @@ struct nfsd_net {
 	 */
 	unsigned int min_threads;
 
+	/*
+	 * Maximum size of an NFS READ or WRITE payload.  Zero until the
+	 * first server start in this namespace picks a default.
+	 */
+	unsigned int max_blksize;
+
 	u32 clientid_base;
 	u32 clientid_counter;
 	u32 clverifier_counter;
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index f32311f2d7cf..330d0f12e199 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -885,8 +885,6 @@ static ssize_t write_ports(struct file *file, char *buf, size_t size)
 }
 
 
-int nfsd_max_blksize;
-
 /*
  * write_maxblksize - Set or report the current NFS blksize
  *
@@ -931,12 +929,12 @@ static ssize_t write_maxblksize(struct file *file, char *buf, size_t size)
 			mutex_unlock(&nfsd_mutex);
 			return -EBUSY;
 		}
-		nfsd_max_blksize = bsize;
+		nn->max_blksize = bsize;
 		mutex_unlock(&nfsd_mutex);
 	}
 
-	return scnprintf(buf, SIMPLE_TRANSACTION_LIMIT, "%d\n",
-							nfsd_max_blksize);
+	return scnprintf(buf, SIMPLE_TRANSACTION_LIMIT, "%u\n",
+							nn->max_blksize);
 }
 
 #ifdef CONFIG_NFSD_V4
diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
index d1e413d21e76..0864d6d564e2 100644
--- a/fs/nfsd/nfsd.h
+++ b/fs/nfsd/nfsd.h
@@ -143,8 +143,6 @@ enum {
 extern u64 nfsd_io_cache_read __read_mostly;
 extern u64 nfsd_io_cache_write __read_mostly;
 
-extern int nfsd_max_blksize;
-
 bool nfsd_v4client(struct svc_rqst *rqstp);
 
 /*
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index d75fe1523436..77e1e6ba686d 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -630,12 +630,12 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
 	init_completion(&nn->nfsd_net_free_done);
 	init_completion(&nn->nfsd_net_confirm_done);
 
-	if (nfsd_max_blksize == 0)
-		nfsd_max_blksize = nfsd_get_default_max_blksize();
+	if (nn->max_blksize == 0)
+		nn->max_blksize = nfsd_get_default_max_blksize();
 	nfsd_reset_versions(nn);
 	serv = svc_create_pooled(nfsd_programs, ARRAY_SIZE(nfsd_programs),
 				 &nn->nfsd_svcstats,
-				 nfsd_max_blksize, nfsd);
+				 nn->max_blksize, nfsd);
 	if (serv == NULL) {
 		percpu_ref_exit(&nn->nfsd_net_ref);
 		return -ENOMEM;

-- 
2.55.0

[PATCH v2 3/8] nfsd: move the control plane to a per-namespace mutex

[Jeff Layton]

nfsd_mutex serializes the entire NFSD control plane across every network
namespace. The nfsd genl family sets .parallel_ops, so it is the only
serialization there: one container starting nfsd, or one long
RPC_STATUS_GET dump, stalls every other namespace's admin operations. The
nfsd threads suffer too -- the dynamic-thread autoscaler trylocks the same
mutex on every -ETIMEDOUT and -EBUSY, and a failed trylock skips the
spawn/reap entirely.

Almost none of what the mutex covers is actually shared. Add
nn->nfsd_mutex for the per-namespace control plane:

  - nn->nfsd_serv and the svc_serv members hanging off it (->sv_permsocks,
    ->sv_temp_socks, per-pool thread counts)
  - NFSD_NET_UP / NFSD_NET_LOCKD_UP
  - the settables that may only change while the server is down
    (->nfsd_versions, ->nfsd4_lease, ->nfsd4_grace, ->max_blksize, ...)
  - nn->svc_export_cache / nn->svc_expkey_cache liveness
  - nn->conf_id_hashtbl liveness, for the state-revoke walks

The global nfsd_mutex keeps only what is genuinely host-wide: the
nfsd_users refcount and the resources it brings up (open file cache, NFSv4
global tables), and the address-notifier registration.
nfsd_startup_generic()/nfsd_shutdown_generic() and the notifier
register/unregister now take it internally, so per-net callers never see
it. nfsd_file_cache_purge() likewise takes it itself, which lets
expkey_flush() drop its hand-rolled lock.

write_recoverydir() still takes the global mutex, but that now serializes
writers only. The startup readers of user_recovery_dirname --
nfsd4_init_recdir() and check_for_legacy_methods() -- run under
nn->nfsd_mutex alone, so a write from one namespace can tear the string
under another namespace's startup. Worst case is a bogus path and a
spurious startup error; the buffer is always NUL-terminated in bounds, so
there is nothing to overrun. Left alone deliberately: that global already
had lock-free readers in nfsd4_cltrack_legacy_{topdir,recdir}(), and legacy
client tracking is deprecated and effectively init-netns-only -- its
usermodehelper upcall always runs in the init mount namespace, so the
stored path can only ever name an init-ns path.

Lock ordering is nn->nfsd_mutex outside the global nfsd_mutex; nothing
takes two namespaces' nfsd_mutexes. struct svc_info already indirects
through a mutex pointer, so pool_stats needs only to be pointed at the new
lock.

The notifier refcount becomes a plain int now that it is genuinely
mutex-guarded -- an atomic was never enough to serialize the
register/unregister against the count. nfsd_create_serv() now takes that
reference before publishing nn->nfsd_serv: namespaces are no longer
serialized against each other here, so ordering it the other way would let
the count dip to zero while another namespace's serv is already visible.

Note: the RPC thread pool needs no global lock here. svc_pool_map has its
own svc_pool_map_mutex and per-pool thread counts live in the per-net
svc_serv.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/export.c                                   |  22 ++--
 fs/nfsd/filecache.c                                |   5 +-
 fs/nfsd/netns.h                                    |  10 ++
 fs/nfsd/nfs4proc.c                                 |   4 +-
 fs/nfsd/nfs4state.c                                |   8 +-
 fs/nfsd/nfsctl.c                                   | 117 ++++++++++---------
 fs/nfsd/nfssvc.c                                   | 129 ++++++++++++++-------
 .../testing/selftests/nfsd/nfsd_netlink_listener.c |   6 +-
 8 files changed, 180 insertions(+), 121 deletions(-)

diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index e5a0f1ababe6..265ea8fd31c6 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -248,13 +248,7 @@ static struct cache_head *expkey_alloc(void)
 
 static void expkey_flush(void)
 {
-	/*
-	 * Take the nfsd_mutex here to ensure that the file cache is not
-	 * destroyed while we're in the middle of flushing.
-	 */
-	mutex_lock(&nfsd_mutex);
 	nfsd_file_cache_purge(current->nsproxy->net_ns);
-	mutex_unlock(&nfsd_mutex);
 }
 
 static int expkey_notify(struct cache_detail *cd, struct cache_head *h)
@@ -346,7 +340,7 @@ int nfsd_nl_expkey_get_reqs_dumpit(struct sk_buff *skb,
 
 	nn = net_generic(sock_net(skb->sk), nfsd_net_id);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	cd = nn->svc_expkey_cache;
 	if (!cd) {
@@ -425,7 +419,7 @@ int nfsd_nl_expkey_get_reqs_dumpit(struct sk_buff *skb,
 	kfree(seqnos);
 	kfree(items);
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return ret;
 }
 
@@ -560,7 +554,7 @@ int nfsd_nl_expkey_set_reqs_doit(struct sk_buff *skb,
 
 	nn = net_generic(genl_info_net(info), nfsd_net_id);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	cd = nn->svc_expkey_cache;
 	if (!cd) {
@@ -576,7 +570,7 @@ int nfsd_nl_expkey_set_reqs_doit(struct sk_buff *skb,
 	}
 
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return ret;
 }
 
@@ -673,7 +667,7 @@ int nfsd_nl_svc_export_get_reqs_dumpit(struct sk_buff *skb,
 
 	nn = net_generic(sock_net(skb->sk), nfsd_net_id);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	cd = nn->svc_export_cache;
 	if (!cd) {
@@ -757,7 +751,7 @@ int nfsd_nl_svc_export_get_reqs_dumpit(struct sk_buff *skb,
 	kfree(seqnos);
 	kfree(items);
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return ret;
 }
 
@@ -1056,7 +1050,7 @@ int nfsd_nl_svc_export_set_reqs_doit(struct sk_buff *skb,
 
 	nn = net_generic(genl_info_net(info), nfsd_net_id);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	cd = nn->svc_export_cache;
 	if (!cd) {
@@ -1072,7 +1066,7 @@ int nfsd_nl_svc_export_set_reqs_doit(struct sk_buff *skb,
 	}
 
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return ret;
 }
 
diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c
index 17a94e6fcb15..79b9e8c92e70 100644
--- a/fs/nfsd/filecache.c
+++ b/fs/nfsd/filecache.c
@@ -1011,13 +1011,16 @@ nfsd_file_cache_start_net(struct net *net)
  * nfsd_file_cache_purge - Remove all cache items associated with @net
  * @net: target net namespace
  *
+ * Takes nfsd_mutex so the cache cannot be torn down underneath the
+ * walk.  Callers must not already hold it.
  */
 void
 nfsd_file_cache_purge(struct net *net)
 {
-	lockdep_assert_held(&nfsd_mutex);
+	mutex_lock(&nfsd_mutex);
 	if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1)
 		__nfsd_file_cache_purge(net);
+	mutex_unlock(&nfsd_mutex);
 }
 
 void
diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
index 374ce83e2ba0..35199c17f8d1 100644
--- a/fs/nfsd/netns.h
+++ b/fs/nfsd/netns.h
@@ -164,6 +164,16 @@ struct nfsd_net {
 	u32 clientid_counter;
 	u32 clverifier_counter;
 
+	/*
+	 * Serializes this namespace's control plane: ->nfsd_serv and the
+	 * svc_serv members that hang off it (->sv_permsocks,
+	 * ->sv_temp_socks, thread counts), the NFSD_NET_* flags, and the
+	 * settables above that may only change while the server is down.
+	 *
+	 * Nests outside the global nfsd_mutex.
+	 */
+	struct mutex nfsd_mutex;
+
 	struct svc_info nfsd_info;
 #define nfsd_serv nfsd_info.serv
 
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 3a82af381a8d..7df60abfbff1 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1703,7 +1703,7 @@ static bool nfsd4_copy_on_sb(const struct nfsd4_copy *copy,
  * @net: net namespace containing the copy operations
  * @sb: targeted superblock
  *
- * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ * Context: Caller must hold nn->nfsd_mutex with NFSD_NET_UP set.  Outside
  *          that window nn->conf_id_hashtbl is unallocated or freed,
  *          so the walk would dereference a NULL or dangling pointer.
  */
@@ -1715,7 +1715,7 @@ void nfsd4_cancel_copy_by_sb(struct net *net, struct super_block *sb)
 	unsigned int idhashval;
 	LIST_HEAD(to_cancel);
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 	spin_lock(&nn->client_lock);
 	for (idhashval = 0; idhashval < CLIENT_HASH_SIZE; idhashval++) {
 		struct list_head *head = &nn->conf_id_hashtbl[idhashval];
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 1de6c6d757c3..0f9340eb281e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -2102,7 +2102,7 @@ static void revoke_one_stid(struct nfsd_net *nn, struct nfs4_client *clp,
  * The clients which own the states will subsequently be notified that the
  * states have been "admin-revoked".
  *
- * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ * Context: Caller must hold nn->nfsd_mutex with NFSD_NET_UP set.  Outside
  *          that window nn->conf_id_hashtbl is unallocated or freed,
  *          so the walk would dereference a NULL or dangling pointer.
  */
@@ -2111,7 +2111,7 @@ void nfsd4_revoke_states(struct nfsd_net *nn, struct super_block *sb)
 	unsigned int idhashval;
 	unsigned int sc_types;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	sc_types = SC_TYPE_OPEN | SC_TYPE_LOCK | SC_TYPE_DELEG | SC_TYPE_LAYOUT;
 
@@ -2190,7 +2190,7 @@ static struct nfs4_stid *find_one_export_stid(struct nfs4_client *clp,
  * Userspace (exportfs -u) sends this after removing the last client
  * for a path, enabling the underlying filesystem to be unmounted.
  *
- * Context: Caller must hold nfsd_mutex with NFSD_NET_UP set.  Outside
+ * Context: Caller must hold nn->nfsd_mutex with NFSD_NET_UP set.  Outside
  *          that window nn->conf_id_hashtbl is unallocated or freed,
  *          so the walk would dereference a NULL or dangling pointer.
  */
@@ -2199,7 +2199,7 @@ void nfsd4_revoke_export_states(struct nfsd_net *nn, const struct path *path)
 	unsigned int idhashval;
 	unsigned int sc_types;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	sc_types = SC_TYPE_OPEN | SC_TYPE_LOCK | SC_TYPE_DELEG | SC_TYPE_LAYOUT;
 
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 330d0f12e199..5ae33c21cf71 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -302,15 +302,15 @@ static ssize_t write_unlock_fs(struct file *file, char *buf, size_t size)
 	 * 3.  Is that directory the root of an exported file system?
 	 */
 	error = nlmsvc_unlock_all_by_sb(path.dentry->d_sb);
-	mutex_lock(&nfsd_mutex);
 	nn = net_generic(netns(file), nfsd_net_id);
+	mutex_lock(&nn->nfsd_mutex);
 	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd4_cancel_copy_by_sb(netns(file), path.dentry->d_sb);
 		nfsd4_revoke_states(nn, path.dentry->d_sb);
 	} else {
 		error = -EINVAL;
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	path_put(&path);
 	return error;
@@ -436,12 +436,12 @@ static ssize_t write_threads(struct file *file, char *buf, size_t size)
 		if (newthreads < 0)
 			return -EINVAL;
 		trace_nfsd_ctl_threads(net, newthreads);
-		mutex_lock(&nfsd_mutex);
+		mutex_lock(&nn->nfsd_mutex);
 		if (newthreads > 0 || nn->nfsd_serv != NULL)
 			rv = nfsd_svc(1, &newthreads, net, file->f_cred, NULL);
 		else
 			rv = 0;
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		if (rv < 0)
 			return rv;
 	} else
@@ -484,8 +484,9 @@ static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)
 	int npools;
 	int *nthreads;
 	struct net *net = netns(file);
+	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	npools = nfsd_nrpools(net);
 	if (npools == 0) {
 		/*
@@ -493,7 +494,7 @@ static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)
 		 * writing to the threads file but NOT the pool_threads
 		 * file, sorry.  Report zero threads.
 		 */
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		strcpy(buf, "0\n");
 		return strlen(buf);
 	}
@@ -547,7 +548,7 @@ static ssize_t write_pool_threads(struct file *file, char *buf, size_t size)
 	rv = mesg - buf;
 out_free:
 	kfree(nthreads);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 
@@ -710,11 +711,12 @@ static ssize_t __write_versions(struct file *file, char *buf, size_t size)
  */
 static ssize_t write_versions(struct file *file, char *buf, size_t size)
 {
+	struct nfsd_net *nn = net_generic(netns(file), nfsd_net_id);
 	ssize_t rv;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	rv = __write_versions(file, buf, size);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 
@@ -876,11 +878,12 @@ static ssize_t __write_ports(struct file *file, char *buf, size_t size,
  */
 static ssize_t write_ports(struct file *file, char *buf, size_t size)
 {
+	struct nfsd_net *nn = net_generic(netns(file), nfsd_net_id);
 	ssize_t rv;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	rv = __write_ports(file, buf, size, netns(file));
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 
@@ -924,13 +927,13 @@ static ssize_t write_maxblksize(struct file *file, char *buf, size_t size)
 		bsize = max_t(int, bsize, 1024);
 		bsize = min_t(int, bsize, NFSSVC_MAXBLKSIZE);
 		bsize &= ~(1024-1);
-		mutex_lock(&nfsd_mutex);
+		mutex_lock(&nn->nfsd_mutex);
 		if (nn->nfsd_serv) {
-			mutex_unlock(&nfsd_mutex);
+			mutex_unlock(&nn->nfsd_mutex);
 			return -EBUSY;
 		}
 		nn->max_blksize = bsize;
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 	}
 
 	return scnprintf(buf, SIMPLE_TRANSACTION_LIMIT, "%u\n",
@@ -979,9 +982,9 @@ static ssize_t nfsd4_write_time(struct file *file, char *buf, size_t size,
 {
 	ssize_t rv;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	rv = __nfsd4_write_time(file, buf, size, time, nn);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 
@@ -1084,9 +1087,15 @@ static ssize_t write_recoverydir(struct file *file, char *buf, size_t size)
 	ssize_t rv;
 	struct nfsd_net *nn = net_generic(netns(file), nfsd_net_id);
 
+	/*
+	 * nn->nfsd_mutex guards the nn->nfsd_serv check; the recovery
+	 * dirname itself is still shared between namespaces.
+	 */
+	mutex_lock(&nn->nfsd_mutex);
 	mutex_lock(&nfsd_mutex);
 	rv = __write_recoverydir(file, buf, size, nn);
 	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 #endif
@@ -1530,9 +1539,9 @@ int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
 	int i, ret, rqstp_index = 0;
 	struct nfsd_net *nn;
 
-	mutex_lock(&nfsd_mutex);
-
 	nn = net_generic(sock_net(skb->sk), nfsd_net_id);
+
+	mutex_lock(&nn->nfsd_mutex);
 	if (!nn->nfsd_serv) {
 		ret = -ENODEV;
 		goto out_unlock;
@@ -1649,7 +1658,7 @@ int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
 out:
 	rcu_read_unlock();
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	return ret;
 }
@@ -1659,7 +1668,7 @@ int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
  * @attr: nlattr NFSD_A_SERVER_FH_KEY
  * @nn: nfsd_net
  *
- * Callers should hold nfsd_mutex, returns 0 on success or negative errno.
+ * Callers should hold nn->nfsd_mutex, returns 0 on success or negative errno.
  * Callers must ensure the server is shut down (sv_nrthreads == 0),
  * userspace documentation asserts the key may only be set when the server
  * is not running.
@@ -1714,7 +1723,7 @@ int nfsd_nl_threads_set_doit(struct sk_buff *skb, struct genl_info *info)
 				 GENL_HDRLEN, rem)
 		nrpools++;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	nthreads = kzalloc_objs(int, nrpools);
 	if (!nthreads) {
@@ -1779,7 +1788,7 @@ int nfsd_nl_threads_set_doit(struct sk_buff *skb, struct genl_info *info)
 	if (ret > 0)
 		ret = 0;
 out_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	kfree(nthreads);
 	return ret;
 }
@@ -1808,7 +1817,7 @@ int nfsd_nl_threads_get_doit(struct sk_buff *skb, struct genl_info *info)
 		goto err_free_msg;
 	}
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	err = nla_put_u32(skb, NFSD_A_SERVER_GRACETIME,
 			  nn->nfsd4_grace) ||
@@ -1838,14 +1847,14 @@ int nfsd_nl_threads_get_doit(struct sk_buff *skb, struct genl_info *info)
 			goto err_unlock;
 	}
 
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	genlmsg_end(skb, hdr);
 
 	return genlmsg_reply(skb, info);
 
 err_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 err_free_msg:
 	nlmsg_free(skb);
 
@@ -1868,11 +1877,11 @@ int nfsd_nl_version_set_doit(struct sk_buff *skb, struct genl_info *info)
 	if (GENL_REQ_ATTR_CHECK(info, NFSD_A_SERVER_PROTO_VERSION))
 		return -EINVAL;
 
-	mutex_lock(&nfsd_mutex);
-
 	nn = net_generic(genl_info_net(info), nfsd_net_id);
+
+	mutex_lock(&nn->nfsd_mutex);
 	if (nn->nfsd_serv) {
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		return -EBUSY;
 	}
 
@@ -1915,7 +1924,7 @@ int nfsd_nl_version_set_doit(struct sk_buff *skb, struct genl_info *info)
 		}
 	}
 
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	return 0;
 }
@@ -1943,9 +1952,9 @@ int nfsd_nl_version_get_doit(struct sk_buff *skb, struct genl_info *info)
 		goto err_free_msg;
 	}
 
-	mutex_lock(&nfsd_mutex);
 	nn = net_generic(genl_info_net(info), nfsd_net_id);
 
+	mutex_lock(&nn->nfsd_mutex);
 	for (i = 2; i <= 4; i++) {
 		int j;
 
@@ -1987,13 +1996,13 @@ int nfsd_nl_version_get_doit(struct sk_buff *skb, struct genl_info *info)
 		}
 	}
 
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	genlmsg_end(skb, hdr);
 
 	return genlmsg_reply(skb, info);
 
 err_nfsd_unlock:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 err_free_msg:
 	nlmsg_free(skb);
 
@@ -2004,7 +2013,7 @@ int nfsd_nl_version_get_doit(struct sk_buff *skb, struct genl_info *info)
  * Transport classes NFSD knows how to instantiate. Vetting the name here
  * keeps a bogus string from reaching svc_xprt_create_from_sa(), where an
  * unknown name triggers a request_module("svc%s", name) upcall under
- * nfsd_mutex.
+ * nn->nfsd_mutex.
  */
 static bool nfsd_nl_transport_supported(const char *name)
 {
@@ -2085,14 +2094,15 @@ static int nfsd_nl_validate_listeners(struct genl_info *info)
 	return count;
 }
 
-static size_t nfsd_nl_listener_set_msgsize(struct svc_serv *serv)
+static size_t nfsd_nl_listener_set_msgsize(struct nfsd_net *nn,
+					   struct svc_serv *serv)
 {
 	size_t size = GENL_HDRLEN +		    /* genlmsg_iput() */
 		      nla_total_size(0);	    /* userspace-rpcbind */
 	struct svc_xprt *xprt;
 	unsigned int p;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	for (p = 0; p < serv->sv_nprogs; p++)
 		size += serv->sv_programs[p].pg_nvers *
@@ -2118,15 +2128,16 @@ static struct sk_buff *
 nfsd_nl_listener_set_msg(struct genl_info *info, struct net *net,
 			 struct svc_serv *serv)
 {
+	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct svc_xprt *xprt;
 	struct sk_buff *skb;
 	unsigned int p, i;
 	void *hdr;
 	int err;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
-	skb = genlmsg_new(nfsd_nl_listener_set_msgsize(serv), GFP_KERNEL);
+	skb = genlmsg_new(nfsd_nl_listener_set_msgsize(nn, serv), GFP_KERNEL);
 	if (!skb)
 		return ERR_PTR(-ENOMEM);
 
@@ -2244,10 +2255,10 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
 
 	userspace_rpcbind = nla_get_flag(info->attrs[NFSD_A_SERVER_SOCK_USERSPACE_RPCBIND]);
 
-	mutex_lock(&nfsd_mutex);
-
 	nn = net_generic(net, nfsd_net_id);
 
+	mutex_lock(&nn->nfsd_mutex);
+
 	/*
 	 * An empty list destroys the serv, and nfsd_destroy_serv() drops
 	 * whatever svc_bind() took either way, so teardown is not an
@@ -2258,13 +2269,13 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
 	    nn->nfsd_serv->sv_no_rpcbind != userspace_rpcbind) {
 		NL_SET_ERR_MSG(info->extack,
 			       "cannot change rpcbind ownership while a server exists");
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		return -EBUSY;
 	}
 
 	err = nfsd_create_serv(net, userspace_rpcbind);
 	if (err) {
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		return err;
 	}
 
@@ -2433,7 +2444,7 @@ int nfsd_nl_listener_set_doit(struct sk_buff *skb, struct genl_info *info)
 		nfsd_destroy_serv(net);
 
 out_unlock_mtx:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	/* rskb is only built once err is known to be zero. */
 	if (rskb)
@@ -2467,9 +2478,9 @@ int nfsd_nl_listener_get_doit(struct sk_buff *skb, struct genl_info *info)
 		goto err_free_msg;
 	}
 
-	mutex_lock(&nfsd_mutex);
 	nn = net_generic(genl_info_net(info), nfsd_net_id);
 
+	mutex_lock(&nn->nfsd_mutex);
 	/* no nfs server? Just send empty socket list */
 	if (!nn->nfsd_serv)
 		goto out_unlock_mtx;
@@ -2498,14 +2509,14 @@ int nfsd_nl_listener_get_doit(struct sk_buff *skb, struct genl_info *info)
 	}
 	spin_unlock_bh(&serv->sv_lock);
 out_unlock_mtx:
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	genlmsg_end(skb, hdr);
 
 	return genlmsg_reply(skb, info);
 
 err_serv_unlock:
 	spin_unlock_bh(&serv->sv_lock);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 err_free_msg:
 	nlmsg_free(skb);
 
@@ -2589,7 +2600,7 @@ int nfsd_nl_cache_flush_doit(struct sk_buff *skb, struct genl_info *info)
 	if (info->attrs[NFSD_A_CACHE_FLUSH_MASK])
 		mask = nla_get_u32(info->attrs[NFSD_A_CACHE_FLUSH_MASK]);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 
 	if ((mask & NFSD_CACHE_TYPE_SVC_EXPORT) &&
 	    nn->svc_export_cache)
@@ -2599,7 +2610,7 @@ int nfsd_nl_cache_flush_doit(struct sk_buff *skb, struct genl_info *info)
 	    nn->svc_expkey_cache)
 		cache_purge(nn->svc_expkey_cache);
 
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	return 0;
 }
@@ -2947,14 +2958,14 @@ int nfsd_nl_unlock_filesystem_doit(struct sk_buff *skb,
 
 	error = nlmsvc_unlock_all_by_sb(path.dentry->d_sb);
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd4_cancel_copy_by_sb(net, path.dentry->d_sb);
 		nfsd4_revoke_states(nn, path.dentry->d_sb);
 	} else {
 		error = -EINVAL;
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	path_put(&path);
 	return error;
@@ -2994,13 +3005,13 @@ int nfsd_nl_unlock_export_doit(struct sk_buff *skb, struct genl_info *info)
 	if (error)
 		return error;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	if (test_bit(NFSD_NET_UP, &nn->flags)) {
 		nfsd_file_close_export(net, &path);
 		nfsd4_revoke_export_states(nn, &path);
 	} else
 		error = -EINVAL;
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 
 	path_put(&path);
 	return error;
@@ -3057,7 +3068,8 @@ static __net_init int nfsd_net_init(struct net *net)
 		nn->nfsd_versions[i] = nfsd_support_version(i);
 	for (i = 0; i < sizeof(nn->nfsd4_minorversions); i++)
 		nn->nfsd4_minorversions[i] = nfsd_support_version(4);
-	nn->nfsd_info.mutex = &nfsd_mutex;
+	mutex_init(&nn->nfsd_mutex);
+	nn->nfsd_info.mutex = &nn->nfsd_mutex;
 	nn->nfsd_serv = NULL;
 	nfsd4_init_leases_net(nn);
 	get_random_bytes(&nn->siphash_key, sizeof(nn->siphash_key));
@@ -3121,6 +3133,7 @@ static __net_exit void nfsd_net_exit(struct net *net)
 	percpu_counter_destroy_many(nn->counter, NFSD_STATS_COUNTERS_NUM);
 	nfsd_idmap_shutdown(net);
 	nfsd_export_shutdown(net);
+	mutex_destroy(&nn->nfsd_mutex);
 }
 
 static struct pernet_operations nfsd_net_ops = {
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 77e1e6ba686d..3d47e5c86bd5 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -55,16 +55,22 @@ static __be32			nfsd_init_request(struct svc_rqst *,
 						struct svc_process_info *);
 
 /*
- * nfsd_mutex protects nn->nfsd_serv -- both the pointer itself and some members
- * of the svc_serv struct such as ->sv_temp_socks and ->sv_permsocks.
+ * NFSD's control plane is serialized by two mutexes.
  *
- * Finally, the nfsd_mutex also protects some of the global variables that are
- * accessed when nfsd starts and that are settable via the write_* routines in
- * nfsctl.c. In particular:
+ * Nearly everything is per-namespace and belongs to nn->nfsd_mutex: the
+ * nn->nfsd_serv pointer and the svc_serv members that hang off it
+ * (->sv_permsocks, ->sv_temp_socks, per-pool thread counts), the
+ * NFSD_NET_* flags, and the nfsd_net settables that may only change while
+ * that namespace's server is down (->nfsd_versions, ->nfsd4_lease,
+ * ->nfsd4_grace, ->max_blksize, ...).
  *
- *	user_recovery_dirname
- *	user_lease_time
- *	nfsd_versions
+ * The global nfsd_mutex covers only what is genuinely shared between
+ * namespaces: the nfsd_users refcount and the host-wide resources it
+ * brings up and tears down (the open file cache and the NFSv4 global
+ * tables), the address-notifier registration, and user_recovery_dirname.
+ *
+ * Lock ordering is nn->nfsd_mutex outside the global nfsd_mutex.  Nothing
+ * takes two namespaces' nfsd_mutexes.
  */
 DEFINE_MUTEX(nfsd_mutex);
 
@@ -251,20 +257,23 @@ int nfsd_nrthreads(struct net *net)
 	int rv = 0;
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 
-	/* nfsd_mutex keeps nn->nfsd_serv valid across the read. */
-	mutex_lock(&nfsd_mutex);
+	/* nn->nfsd_mutex keeps nn->nfsd_serv valid across the read. */
+	mutex_lock(&nn->nfsd_mutex);
 	if (nn->nfsd_serv)
 		rv = svc_serv_maxthreads(nn->nfsd_serv);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
 
+/* Number of namespaces holding the host-wide resources up */
 static int nfsd_users = 0;
 
-static int nfsd_startup_generic(void)
+static int __nfsd_startup_generic(void)
 {
 	int ret;
 
+	lockdep_assert_held(&nfsd_mutex);
+
 	if (nfsd_users++)
 		return 0;
 
@@ -284,13 +293,24 @@ static int nfsd_startup_generic(void)
 	return ret;
 }
 
-static void nfsd_shutdown_generic(void)
+static int nfsd_startup_generic(void)
 {
-	if (--nfsd_users)
-		return;
+	int ret;
 
-	nfs4_state_shutdown();
-	nfsd_file_cache_shutdown();
+	mutex_lock(&nfsd_mutex);
+	ret = __nfsd_startup_generic();
+	mutex_unlock(&nfsd_mutex);
+	return ret;
+}
+
+static void nfsd_shutdown_generic(void)
+{
+	mutex_lock(&nfsd_mutex);
+	if (!--nfsd_users) {
+		nfs4_state_shutdown();
+		nfsd_file_cache_shutdown();
+	}
+	mutex_unlock(&nfsd_mutex);
 }
 
 static bool nfsd_needs_lockd(struct nfsd_net *nn)
@@ -505,8 +525,32 @@ static struct notifier_block nfsd_inet6addr_notifier = {
 };
 #endif
 
-/* Only used under nfsd_mutex, so this atomic may be overkill: */
-static atomic_t nfsd_notifier_refcount = ATOMIC_INIT(0);
+/* Number of namespaces with a serv, guarded by nfsd_mutex */
+static int nfsd_notifier_users;
+
+static void nfsd_register_notifiers(void)
+{
+	mutex_lock(&nfsd_mutex);
+	if (!nfsd_notifier_users++) {
+		register_inetaddr_notifier(&nfsd_inetaddr_notifier);
+#if IS_ENABLED(CONFIG_IPV6)
+		register_inet6addr_notifier(&nfsd_inet6addr_notifier);
+#endif
+	}
+	mutex_unlock(&nfsd_mutex);
+}
+
+static void nfsd_unregister_notifiers(void)
+{
+	mutex_lock(&nfsd_mutex);
+	if (!--nfsd_notifier_users) {
+		unregister_inetaddr_notifier(&nfsd_inetaddr_notifier);
+#if IS_ENABLED(CONFIG_IPV6)
+		unregister_inet6addr_notifier(&nfsd_inet6addr_notifier);
+#endif
+	}
+	mutex_unlock(&nfsd_mutex);
+}
 
 /**
  * nfsd_destroy_serv - tear down NFSD's svc_serv for a namespace
@@ -517,19 +561,13 @@ void nfsd_destroy_serv(struct net *net)
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct svc_serv *serv = nn->nfsd_serv;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	spin_lock(&nfsd_notifier_lock);
 	nn->nfsd_serv = NULL;
 	spin_unlock(&nfsd_notifier_lock);
 
-	/* check if the notifier still has clients */
-	if (atomic_dec_return(&nfsd_notifier_refcount) == 0) {
-		unregister_inetaddr_notifier(&nfsd_inetaddr_notifier);
-#if IS_ENABLED(CONFIG_IPV6)
-		unregister_inet6addr_notifier(&nfsd_inet6addr_notifier);
-#endif
-	}
+	nfsd_unregister_notifiers();
 
 	/*
 	 * write_ports can create the server without actually starting
@@ -586,17 +624,17 @@ void nfsd_shutdown_threads(struct net *net)
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct svc_serv *serv;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nn->nfsd_mutex);
 	serv = nn->nfsd_serv;
 	if (serv == NULL) {
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 		return;
 	}
 
 	/* Kill outstanding nfsd threads */
 	svc_set_num_threads(serv, 0, 0);
 	nfsd_destroy_serv(net);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nn->nfsd_mutex);
 }
 
 struct svc_rqst *nfsd_current_rqst(void)
@@ -619,7 +657,7 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct svc_serv *serv;
 
-	WARN_ON(!mutex_is_locked(&nfsd_mutex));
+	WARN_ON(!mutex_is_locked(&nn->nfsd_mutex));
 	if (nn->nfsd_serv)
 		return 0;
 
@@ -650,17 +688,18 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
 		percpu_ref_exit(&nn->nfsd_net_ref);
 		return error;
 	}
+	/*
+	 * Register before publishing nn->nfsd_serv.  Namespaces are only
+	 * serialized against each other by nfsd_mutex here, so
+	 * taking the reference first is what guarantees a visible
+	 * nn->nfsd_serv never coincides with an unregistered notifier.
+	 */
+	nfsd_register_notifiers();
+
 	spin_lock(&nfsd_notifier_lock);
 	nn->nfsd_serv = serv;
 	spin_unlock(&nfsd_notifier_lock);
 
-	/* check if the notifier is already set */
-	if (atomic_inc_return(&nfsd_notifier_refcount) == 1) {
-		register_inetaddr_notifier(&nfsd_inetaddr_notifier);
-#if IS_ENABLED(CONFIG_IPV6)
-		register_inet6addr_notifier(&nfsd_inet6addr_notifier);
-#endif
-	}
 	nfsd_reset_write_verifier(nn);
 	return 0;
 }
@@ -707,7 +746,7 @@ int nfsd_set_nrthreads(int n, int *nthreads, struct net *net)
 	int err = 0;
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	if (nn->nfsd_serv == NULL || n <= 0)
 		return 0;
@@ -777,7 +816,7 @@ nfsd_svc(int n, int *nthreads, struct net *net, const struct cred *cred, const c
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct svc_serv *serv;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nn->nfsd_mutex);
 
 	dprintk("nfsd: creating service\n");
 
@@ -955,13 +994,13 @@ nfsd(void *vrqstp)
 		switch (svc_recv(rqstp, 5 * HZ)) {
 		case -ETIMEDOUT:
 			/* No work arrived within the timeout window */
-			if (mutex_trylock(&nfsd_mutex)) {
+			if (mutex_trylock(&nn->nfsd_mutex)) {
 				if (pool->sp_nrthreads > pool->sp_nrthrmin) {
 					trace_nfsd_dynthread_kill(net, pool);
 					set_bit(RQ_VICTIM, &rqstp->rq_flags);
 					have_mutex = true;
 				} else {
-					mutex_unlock(&nfsd_mutex);
+					mutex_unlock(&nn->nfsd_mutex);
 				}
 			} else {
 				trace_nfsd_dynthread_trylock_fail(net, pool);
@@ -970,7 +1009,7 @@ nfsd(void *vrqstp)
 		case -EBUSY:
 			/* No idle threads; consider spawning another */
 			if (pool->sp_nrthreads < pool->sp_nrthrmax) {
-				if (mutex_trylock(&nfsd_mutex)) {
+				if (mutex_trylock(&nn->nfsd_mutex)) {
 					if (pool->sp_nrthreads < pool->sp_nrthrmax) {
 						int ret;
 
@@ -980,7 +1019,7 @@ nfsd(void *vrqstp)
 							pr_notice_ratelimited("%s: unable to spawn new thread: %d\n",
 									      __func__, ret);
 					}
-					mutex_unlock(&nfsd_mutex);
+					mutex_unlock(&nn->nfsd_mutex);
 				} else {
 					trace_nfsd_dynthread_trylock_fail(net, pool);
 				}
@@ -998,7 +1037,7 @@ nfsd(void *vrqstp)
 	/* Release the thread */
 	svc_exit_thread(rqstp);
 	if (have_mutex)
-		mutex_unlock(&nfsd_mutex);
+		mutex_unlock(&nn->nfsd_mutex);
 	return 0;
 }
 
diff --git a/tools/testing/selftests/nfsd/nfsd_netlink_listener.c b/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
index bef7e8b1ee71..1294057b6f62 100644
--- a/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
+++ b/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
@@ -5,7 +5,7 @@
  *
  * Three groups:
  *   validation  - malformed/abusive LISTENER_SET requests are rejected by
- *                 nfsd_nl_validate_listeners(), before nfsd_mutex is taken.
+ *                 nfsd_nl_validate_listeners(), before nn->nfsd_mutex is taken.
  *   functional  - create/add/remove listeners and verify LISTENER_GET
  *                 reflects the set (round-trip of transport + addr:port).
  *   semantics   - once threads are running (THREADS_SET) a listener change
@@ -949,8 +949,8 @@ TEST_F(nfsd_listener, val_missing_transport)
 }
 
 /*
- * A name matching no transport class must be refused before nfsd_mutex is
- * taken, so it never reaches svc_xprt_create_from_sa() and its
+ * A name matching no transport class must be refused before nn->nfsd_mutex
+ * is taken, so it never reaches svc_xprt_create_from_sa() and its
  * request_module("svc%s", name) upcall.
  *
  * The errno cannot show that -- svc_xprt_create_from_sa() returns

-- 
2.55.0

[PATCH v2 4/8] nfsd: rename nfsd_mutex to nfsd_global_mutex

[Jeff Layton]

With nn->nfsd_mutex now carrying the per-namespace control plane, give the
remaining module-wide mutex a name that says which one a call site means.
Mechanical; no functional change.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/filecache.c | 14 +++++++-------
 fs/nfsd/netns.h     |  2 +-
 fs/nfsd/nfsctl.c    |  4 ++--
 fs/nfsd/nfsd.h      |  2 +-
 fs/nfsd/nfssvc.c    | 34 +++++++++++++++++-----------------
 5 files changed, 28 insertions(+), 28 deletions(-)

diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c
index 79b9e8c92e70..501772bc8f8d 100644
--- a/fs/nfsd/filecache.c
+++ b/fs/nfsd/filecache.c
@@ -869,7 +869,7 @@ nfsd_file_cache_init(void)
 {
 	int ret;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nfsd_global_mutex);
 	if (test_and_set_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1)
 		return 0;
 
@@ -1011,16 +1011,16 @@ nfsd_file_cache_start_net(struct net *net)
  * nfsd_file_cache_purge - Remove all cache items associated with @net
  * @net: target net namespace
  *
- * Takes nfsd_mutex so the cache cannot be torn down underneath the
+ * Takes nfsd_global_mutex so the cache cannot be torn down underneath the
  * walk.  Callers must not already hold it.
  */
 void
 nfsd_file_cache_purge(struct net *net)
 {
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1)
 		__nfsd_file_cache_purge(net);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 }
 
 void
@@ -1045,7 +1045,7 @@ nfsd_file_cache_shutdown(void)
 {
 	int i;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nfsd_global_mutex);
 	if (test_and_clear_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 0)
 		return;
 
@@ -1477,7 +1477,7 @@ int nfsd_file_cache_stats_show(struct seq_file *m, void *v)
 	unsigned long lru = 0, total_age = 0;
 
 	/* Serialize with server shutdown */
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1) {
 		struct bucket_table *tbl;
 		struct rhashtable *ht;
@@ -1491,7 +1491,7 @@ int nfsd_file_cache_stats_show(struct seq_file *m, void *v)
 		buckets = tbl->size;
 		rcu_read_unlock();
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 
 	for_each_possible_cpu(i) {
 		hits += per_cpu(nfsd_file_cache_hits, i);
diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
index 35199c17f8d1..866d5f64641f 100644
--- a/fs/nfsd/netns.h
+++ b/fs/nfsd/netns.h
@@ -170,7 +170,7 @@ struct nfsd_net {
 	 * ->sv_temp_socks, thread counts), the NFSD_NET_* flags, and the
 	 * settables above that may only change while the server is down.
 	 *
-	 * Nests outside the global nfsd_mutex.
+	 * Nests outside nfsd_global_mutex.
 	 */
 	struct mutex nfsd_mutex;
 
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index 5ae33c21cf71..e2455a549e25 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
@@ -1092,9 +1092,9 @@ static ssize_t write_recoverydir(struct file *file, char *buf, size_t size)
 	 * dirname itself is still shared between namespaces.
 	 */
 	mutex_lock(&nn->nfsd_mutex);
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	rv = __write_recoverydir(file, buf, size, nn);
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 	mutex_unlock(&nn->nfsd_mutex);
 	return rv;
 }
diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
index 0864d6d564e2..d824d2c39029 100644
--- a/fs/nfsd/nfsd.h
+++ b/fs/nfsd/nfsd.h
@@ -46,7 +46,7 @@ enum {
 
 extern struct svc_program	nfsd_programs[];
 extern const struct svc_version	nfsd_version2, nfsd_version3, nfsd_version4;
-extern struct mutex		nfsd_mutex;
+extern struct mutex		nfsd_global_mutex;
 extern atomic_t			nfsd_th_cnt;		/* number of available threads */
 
 extern const struct seq_operations nfs_exports_op;
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 3d47e5c86bd5..5cb92c3829f9 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -64,15 +64,15 @@ static __be32			nfsd_init_request(struct svc_rqst *,
  * that namespace's server is down (->nfsd_versions, ->nfsd4_lease,
  * ->nfsd4_grace, ->max_blksize, ...).
  *
- * The global nfsd_mutex covers only what is genuinely shared between
+ * nfsd_global_mutex covers only what is genuinely shared between
  * namespaces: the nfsd_users refcount and the host-wide resources it
  * brings up and tears down (the open file cache and the NFSv4 global
  * tables), the address-notifier registration, and user_recovery_dirname.
  *
- * Lock ordering is nn->nfsd_mutex outside the global nfsd_mutex.  Nothing
- * takes two namespaces' nfsd_mutexes.
+ * Lock ordering is nn->nfsd_mutex outside nfsd_global_mutex.  Nothing takes
+ * two namespaces' nfsd_mutexes.
  */
-DEFINE_MUTEX(nfsd_mutex);
+DEFINE_MUTEX(nfsd_global_mutex);
 
 #if IS_ENABLED(CONFIG_NFS_LOCALIO)
 static const struct svc_version *localio_versions[] = {
@@ -272,7 +272,7 @@ static int __nfsd_startup_generic(void)
 {
 	int ret;
 
-	lockdep_assert_held(&nfsd_mutex);
+	lockdep_assert_held(&nfsd_global_mutex);
 
 	if (nfsd_users++)
 		return 0;
@@ -297,20 +297,20 @@ static int nfsd_startup_generic(void)
 {
 	int ret;
 
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	ret = __nfsd_startup_generic();
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 	return ret;
 }
 
 static void nfsd_shutdown_generic(void)
 {
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	if (!--nfsd_users) {
 		nfs4_state_shutdown();
 		nfsd_file_cache_shutdown();
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 }
 
 static bool nfsd_needs_lockd(struct nfsd_net *nn)
@@ -525,31 +525,31 @@ static struct notifier_block nfsd_inet6addr_notifier = {
 };
 #endif
 
-/* Number of namespaces with a serv, guarded by nfsd_mutex */
+/* Number of namespaces with a serv, guarded by nfsd_global_mutex */
 static int nfsd_notifier_users;
 
 static void nfsd_register_notifiers(void)
 {
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	if (!nfsd_notifier_users++) {
 		register_inetaddr_notifier(&nfsd_inetaddr_notifier);
 #if IS_ENABLED(CONFIG_IPV6)
 		register_inet6addr_notifier(&nfsd_inet6addr_notifier);
 #endif
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 }
 
 static void nfsd_unregister_notifiers(void)
 {
-	mutex_lock(&nfsd_mutex);
+	mutex_lock(&nfsd_global_mutex);
 	if (!--nfsd_notifier_users) {
 		unregister_inetaddr_notifier(&nfsd_inetaddr_notifier);
 #if IS_ENABLED(CONFIG_IPV6)
 		unregister_inet6addr_notifier(&nfsd_inet6addr_notifier);
 #endif
 	}
-	mutex_unlock(&nfsd_mutex);
+	mutex_unlock(&nfsd_global_mutex);
 }
 
 /**
@@ -690,9 +690,9 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
 	}
 	/*
 	 * Register before publishing nn->nfsd_serv.  Namespaces are only
-	 * serialized against each other by nfsd_mutex here, so
-	 * taking the reference first is what guarantees a visible
-	 * nn->nfsd_serv never coincides with an unregistered notifier.
+	 * serialized against each other by nfsd_global_mutex here, so taking
+	 * the reference first is what guarantees a visible nn->nfsd_serv
+	 * never coincides with an unregistered notifier.
 	 */
 	nfsd_register_notifiers();
 

-- 
2.55.0

[PATCH v2 5/8] nfsd: give the open file cache its own mutex

[Jeff Layton]

nfsd_global_mutex still serializes file cache flushes against server
start/stop in unrelated namespaces. nfsd_file_cache_purge() is reached from
expkey_flush() on every "exportfs -f", which makes it the only frequent
taker of the global lock -- and it has nothing to do with nfsd_users or the
NFSv4 global tables it would be waiting on.

Move NFSD_FILE_CACHE_UP and the objects it covers (rhltable, LRU, slabs,
shrinker, fsnotify groups) onto a nfsd_file_cache_mutex private to
filecache.c. nfsd_file_cache_init() and nfsd_file_cache_shutdown() take it
themselves rather than asserting the caller holds the global one, so
nfssvc.c no longer needs to know how the cache locks itself.

nfsd_global_mutex is left holding only nfsd_users, the notifier count and
user_recovery_dirname, all of which are touched solely on server
start/stop.

Ordering is nn->nfsd_mutex outside nfsd_global_mutex outside
nfsd_file_cache_mutex. The cache mutex is reached three ways, all
consistent with that: from nfsd_startup_generic()/nfsd_shutdown_generic()
under the global mutex, from nfsd_shutdown_net() under nn->nfsd_mutex, and
bare from expkey_flush() and the filecache stats file. Nothing called with
the cache mutex held takes either of the others.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/filecache.c | 29 ++++++++++++++++++++---------
 fs/nfsd/nfssvc.c    |  9 +++++----
 2 files changed, 25 insertions(+), 13 deletions(-)

diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c
index 501772bc8f8d..42bd8c7859cf 100644
--- a/fs/nfsd/filecache.c
+++ b/fs/nfsd/filecache.c
@@ -67,6 +67,17 @@
  */
 static DEFINE_SPINLOCK(nfsd_gc_lock);
 
+/*
+ * Guards NFSD_FILE_CACHE_UP and the host-wide objects it covers: the
+ * rhltable, the LRU, the slabs, the shrinker and the fsnotify groups.
+ * Held across cache bring-up and teardown, and by readers and walkers
+ * that need the cache to stay up for the duration (->cache_purge, the
+ * stats file).
+ *
+ * Nests inside nfsd_global_mutex and inside nn->nfsd_mutex.
+ */
+static DEFINE_MUTEX(nfsd_file_cache_mutex);
+
 static DEFINE_PER_CPU(unsigned long, nfsd_file_cache_hits);
 static DEFINE_PER_CPU(unsigned long, nfsd_file_acquisitions);
 static DEFINE_PER_CPU(unsigned long, nfsd_file_allocations);
@@ -869,7 +880,7 @@ nfsd_file_cache_init(void)
 {
 	int ret;
 
-	lockdep_assert_held(&nfsd_global_mutex);
+	guard(mutex)(&nfsd_file_cache_mutex);
 	if (test_and_set_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1)
 		return 0;
 
@@ -1011,16 +1022,16 @@ nfsd_file_cache_start_net(struct net *net)
  * nfsd_file_cache_purge - Remove all cache items associated with @net
  * @net: target net namespace
  *
- * Takes nfsd_global_mutex so the cache cannot be torn down underneath the
- * walk.  Callers must not already hold it.
+ * Takes nfsd_file_cache_mutex so the cache cannot be torn down underneath
+ * the walk.  Callers must not already hold it.
  */
 void
 nfsd_file_cache_purge(struct net *net)
 {
-	mutex_lock(&nfsd_global_mutex);
+	mutex_lock(&nfsd_file_cache_mutex);
 	if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1)
 		__nfsd_file_cache_purge(net);
-	mutex_unlock(&nfsd_global_mutex);
+	mutex_unlock(&nfsd_file_cache_mutex);
 }
 
 void
@@ -1045,7 +1056,7 @@ nfsd_file_cache_shutdown(void)
 {
 	int i;
 
-	lockdep_assert_held(&nfsd_global_mutex);
+	guard(mutex)(&nfsd_file_cache_mutex);
 	if (test_and_clear_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 0)
 		return;
 
@@ -1476,8 +1487,8 @@ int nfsd_file_cache_stats_show(struct seq_file *m, void *v)
 	unsigned int i, count = 0, buckets = 0;
 	unsigned long lru = 0, total_age = 0;
 
-	/* Serialize with server shutdown */
-	mutex_lock(&nfsd_global_mutex);
+	/* Serialize with cache teardown */
+	mutex_lock(&nfsd_file_cache_mutex);
 	if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1) {
 		struct bucket_table *tbl;
 		struct rhashtable *ht;
@@ -1491,7 +1502,7 @@ int nfsd_file_cache_stats_show(struct seq_file *m, void *v)
 		buckets = tbl->size;
 		rcu_read_unlock();
 	}
-	mutex_unlock(&nfsd_global_mutex);
+	mutex_unlock(&nfsd_file_cache_mutex);
 
 	for_each_possible_cpu(i) {
 		hits += per_cpu(nfsd_file_cache_hits, i);
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 5cb92c3829f9..b2c215775e08 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -66,11 +66,12 @@ static __be32			nfsd_init_request(struct svc_rqst *,
  *
  * nfsd_global_mutex covers only what is genuinely shared between
  * namespaces: the nfsd_users refcount and the host-wide resources it
- * brings up and tears down (the open file cache and the NFSv4 global
- * tables), the address-notifier registration, and user_recovery_dirname.
+ * brings up and tears down (the NFSv4 global tables, and the open file
+ * cache -- which guards its own internals with nfsd_file_cache_mutex),
+ * the address-notifier registration, and user_recovery_dirname.
  *
- * Lock ordering is nn->nfsd_mutex outside nfsd_global_mutex.  Nothing takes
- * two namespaces' nfsd_mutexes.
+ * Lock ordering is nn->nfsd_mutex outside nfsd_global_mutex outside
+ * nfsd_file_cache_mutex.  Nothing takes two namespaces' nfsd_mutexes.
  */
 DEFINE_MUTEX(nfsd_global_mutex);
 

-- 
2.55.0

[PATCH v2 6/8] selftests/nfsd: factor the netlink plumbing into a shared header

[Jeff Layton]

nfsd_netlink_listener.c carries a small generic-netlink client -- socket
setup, attribute packing, extack parsing, family resolution -- plus the
LISTENER_SET/THREADS_SET/VERSION_SET wrappers. Other nfsd tests want the
same thing.

Move it to nfsd_netlink.h as static inline helpers, so each test binary
gets its own copy and there is nothing to link. Code is unchanged; only
the three continuation lines that the static inline prefix pushed out of
alignment were reflowed.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 tools/testing/selftests/nfsd/nfsd_netlink.h        | 319 +++++++++++++++++++++
 .../testing/selftests/nfsd/nfsd_netlink_listener.c | 295 +------------------
 2 files changed, 322 insertions(+), 292 deletions(-)

diff --git a/tools/testing/selftests/nfsd/nfsd_netlink.h b/tools/testing/selftests/nfsd/nfsd_netlink.h
new file mode 100644
index 000000000000..c66a980bfc01
--- /dev/null
+++ b/tools/testing/selftests/nfsd/nfsd_netlink.h
@@ -0,0 +1,319 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Shared generic-netlink plumbing for the NFSD selftests.
+ *
+ * Header-only: every helper is static inline, so each test binary gets its
+ * own copy and there is nothing extra to link. nfsd_family must be set by
+ * calling genl_resolve_nfsd() before any of the request helpers are used.
+ */
+#ifndef __SELFTESTS_NFSD_NETLINK_H__
+#define __SELFTESTS_NFSD_NETLINK_H__
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <netinet/in.h>
+#include <linux/netlink.h>
+#include <linux/genetlink.h>
+#include <linux/nfsd_netlink.h>
+
+#define NLA_ALIGN4(len)			(((len) + 3) & ~3)
+#define RECV_TIMEO_SEC			30
+
+static int nfsd_family = -1;		/* set per-test in FIXTURE_SETUP */
+
+/* Extack message from the last genl_request(); empty if there was none. */
+static char last_extack[128];
+
+static inline void die(const char *msg)
+{
+	perror(msg);
+	exit(1);
+}
+
+/* ------------------- minimal generic-netlink plumbing ------------------- */
+
+static inline int genl_open(void)
+{
+	struct sockaddr_nl sa = { .nl_family = AF_NETLINK };
+	struct timeval tv = { .tv_sec = RECV_TIMEO_SEC };
+	int fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
+	int on = 1;
+
+	if (fd < 0)
+		die("socket(NETLINK_GENERIC)");
+	if (bind(fd, (void *)&sa, sizeof(sa)) < 0)
+		die("bind(netlink)");
+	setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
+	/*
+	 * Ask for extack, and cap the ack so the request is not echoed back:
+	 * the TLVs then always follow the fixed part of the error message.
+	 */
+	setsockopt(fd, SOL_NETLINK, NETLINK_EXT_ACK, &on, sizeof(on));
+	setsockopt(fd, SOL_NETLINK, NETLINK_CAP_ACK, &on, sizeof(on));
+	return fd;
+}
+
+/* Stash the extack message of an ack, if it carries one. */
+static inline void parse_extack(const char *rbuf)
+{
+	const struct nlmsghdr *nlh = (const void *)rbuf;
+	const struct nlattr *na;
+	int off, left;
+
+	last_extack[0] = '\0';
+	if (nlh->nlmsg_type != NLMSG_ERROR ||
+	    !(nlh->nlmsg_flags & NLM_F_ACK_TLVS))
+		return;
+
+	off = NLMSG_HDRLEN + NLMSG_ALIGN(sizeof(struct nlmsgerr));
+	left = nlh->nlmsg_len - off;
+	na = (const void *)(rbuf + off);
+
+	while (left >= (int)NLA_HDRLEN) {
+		if ((na->nla_type & NLA_TYPE_MASK) == NLMSGERR_ATTR_MSG) {
+			strncpy(last_extack, (const char *)na + NLA_HDRLEN,
+				sizeof(last_extack) - 1);
+			last_extack[sizeof(last_extack) - 1] = '\0';
+			return;
+		}
+		left -= NLA_ALIGN4(na->nla_len);
+		na = (const void *)((const char *)na + NLA_ALIGN4(na->nla_len));
+	}
+}
+
+/* Append an attribute at @off; return the new (aligned) offset. */
+static inline int put_attr(char *buf, int off, uint16_t type,
+			   const void *data, int len)
+{
+	struct nlattr *na = (void *)(buf + off);
+
+	na->nla_type = type;
+	na->nla_len = NLA_HDRLEN + len;
+	if (len)
+		memcpy(buf + off + NLA_HDRLEN, data, len);
+	return off + NLA_ALIGN4(NLA_HDRLEN + len);
+}
+
+/* Build a genl message header into @buf; return the offset past it. */
+static inline int genl_hdr(char *buf, uint16_t type, uint16_t flags, uint8_t cmd)
+{
+	struct nlmsghdr *nlh = (void *)buf;
+	struct genlmsghdr *gnl = (void *)(buf + NLMSG_HDRLEN);
+
+	memset(buf, 0, NLMSG_HDRLEN + GENL_HDRLEN);
+	nlh->nlmsg_type = type;
+	nlh->nlmsg_flags = flags;
+	nlh->nlmsg_seq = 1;
+	gnl->cmd = cmd;
+	gnl->version = 1;
+	return NLMSG_HDRLEN + GENL_HDRLEN;
+}
+
+/* Send an nfsd command with an ACK; return the ACK errno (<= 0). */
+static inline int genl_request(uint8_t cmd, const char *attrs, int attrs_len)
+{
+	char buf[1 << 20], rbuf[4096];
+	struct nlmsghdr *nlh = (void *)buf;
+	int fd = genl_open();
+	int off, n, ret;
+
+	off = genl_hdr(buf, nfsd_family, NLM_F_REQUEST | NLM_F_ACK, cmd);
+	if (attrs_len) {
+		memcpy(buf + off, attrs, attrs_len);
+		off += attrs_len;
+	}
+	nlh->nlmsg_len = off;
+
+	if (send(fd, buf, off, 0) < 0)
+		die("send(genl)");
+
+	last_extack[0] = '\0';
+	n = recv(fd, rbuf, sizeof(rbuf), 0);
+	if (n < 0) {
+		ret = (errno == EAGAIN || errno == EWOULDBLOCK) ? -ETIMEDOUT : -errno;
+	} else if (((struct nlmsghdr *)rbuf)->nlmsg_type == NLMSG_ERROR) {
+		ret = ((struct nlmsgerr *)NLMSG_DATA(rbuf))->error;
+		parse_extack(rbuf);
+	} else {
+		ret = 0;
+	}
+	close(fd);
+	return ret;
+}
+
+/*
+ * Send a command with attributes and return the full reply message; -errno
+ * on failure. NLM_F_ACK is left off: the kernel reports an error either way,
+ * so the first message back is the reply whenever there is one.
+ */
+static inline int genl_request_reply_attrs(uint8_t cmd, const char *attrs,
+					   int attrs_len, char *rbuf,
+					   size_t rlen)
+{
+	char buf[1 << 20];
+	struct nlmsghdr *nlh = (void *)buf;
+	int fd = genl_open();
+	int off, n, ret;
+
+	off = genl_hdr(buf, nfsd_family, NLM_F_REQUEST, cmd);
+	if (attrs_len) {
+		memcpy(buf + off, attrs, attrs_len);
+		off += attrs_len;
+	}
+	nlh->nlmsg_len = off;
+
+	if (send(fd, buf, off, 0) < 0)
+		die("send(genl reply)");
+
+	n = recv(fd, rbuf, rlen, 0);
+	if (n < 0)
+		ret = (errno == EAGAIN || errno == EWOULDBLOCK) ? -ETIMEDOUT : -errno;
+	else if (((struct nlmsghdr *)rbuf)->nlmsg_type == NLMSG_ERROR)
+		ret = ((struct nlmsgerr *)NLMSG_DATA(rbuf))->error;
+	else
+		ret = n;
+	close(fd);
+	return ret;
+}
+
+static inline int genl_request_reply(uint8_t cmd, char *rbuf, size_t rlen)
+{
+	return genl_request_reply_attrs(cmd, NULL, 0, rbuf, rlen);
+}
+
+/* Resolve the "nfsd" genl family id; -1 if not registered. */
+static inline int genl_resolve_nfsd(void)
+{
+	char buf[1024], rbuf[4096];
+	struct nlmsghdr *nlh = (void *)buf;
+	struct nlmsghdr *rh = (void *)rbuf;
+	struct nlattr *na;
+	int fd, off, left, id = -1;
+
+	fd = genl_open();
+	off = genl_hdr(buf, GENL_ID_CTRL, NLM_F_REQUEST, CTRL_CMD_GETFAMILY);
+	off = put_attr(buf, off, CTRL_ATTR_FAMILY_NAME,
+		       NFSD_FAMILY_NAME, sizeof(NFSD_FAMILY_NAME));
+	nlh->nlmsg_len = off;
+
+	if (send(fd, buf, off, 0) < 0)
+		die("send(GETFAMILY)");
+	if (recv(fd, rbuf, sizeof(rbuf), 0) < 0)
+		die("recv(GETFAMILY)");
+	close(fd);
+
+	if (rh->nlmsg_type == NLMSG_ERROR)
+		return -1;
+
+	na = (void *)((char *)NLMSG_DATA(rh) + GENL_HDRLEN);
+	left = rh->nlmsg_len - NLMSG_HDRLEN - GENL_HDRLEN;
+	while (left >= (int)NLA_HDRLEN) {
+		if (na->nla_type == CTRL_ATTR_FAMILY_ID) {
+			id = *(uint16_t *)((char *)na + NLA_HDRLEN);
+			break;
+		}
+		left -= NLA_ALIGN4(na->nla_len);
+		na = (void *)((char *)na + NLA_ALIGN4(na->nla_len));
+	}
+	return id;
+}
+
+/* ------------------- listener request builders ------------------- */
+
+/* Fine-grained control for negative tests: any field can be omitted/malformed. */
+struct raw_listener {
+	const char *xprt;	/* NULL -> omit NFSD_A_SOCK_TRANSPORT_NAME */
+	int emit_addr;		/* 0 -> omit NFSD_A_SOCK_ADDR */
+	const void *addr;
+	int addr_len;		/* bytes to emit for NFSD_A_SOCK_ADDR */
+};
+
+static inline int put_raw_listener(char *buf, int off, const struct raw_listener *r)
+{
+	struct nlattr *nest = (void *)(buf + off);
+	int inner = off + NLA_HDRLEN;
+
+	if (r->emit_addr)
+		inner = put_attr(buf, inner, NFSD_A_SOCK_ADDR, r->addr, r->addr_len);
+	if (r->xprt)
+		inner = put_attr(buf, inner, NFSD_A_SOCK_TRANSPORT_NAME,
+				 r->xprt, strlen(r->xprt) + 1);
+	nest->nla_type = NFSD_A_SERVER_SOCK_ADDR | NLA_F_NESTED;
+	nest->nla_len = inner - off;
+	return off + NLA_ALIGN4(nest->nla_len);
+}
+
+/* Well-formed loopback listener for @family (AF_INET or AF_INET6). */
+static inline int put_listener_af(char *buf, int off, const char *xprt,
+				  int family, uint16_t port)
+{
+	struct sockaddr_storage ss = {0};
+	struct raw_listener r = { .xprt = xprt, .emit_addr = 1, .addr = &ss };
+
+	if (family == AF_INET6) {
+		struct sockaddr_in6 *s6 = (void *)&ss;
+
+		s6->sin6_family = AF_INET6;
+		s6->sin6_port = htons(port);
+		s6->sin6_addr = in6addr_loopback;
+		r.addr_len = sizeof(*s6);
+	} else {
+		struct sockaddr_in *s4 = (void *)&ss;
+
+		s4->sin_family = AF_INET;
+		s4->sin_port = htons(port);
+		s4->sin_addr.s_addr = htonl(INADDR_LOOPBACK);
+		r.addr_len = sizeof(*s4);
+	}
+	return put_raw_listener(buf, off, &r);
+}
+
+static inline int put_listener(char *buf, int off, const char *xprt, uint16_t port)
+{
+	return put_listener_af(buf, off, xprt, AF_INET, port);
+}
+
+static inline int listener_set(const char *attrs, int len)
+{
+	return genl_request(NFSD_CMD_LISTENER_SET, attrs, len);
+}
+
+/*
+ * Enable exactly one NFS version in this netns. NFSD_CMD_VERSION_SET clears
+ * every version first, so one nest is enough to leave the server v4-only.
+ * It refuses once a serv exists, so call it before any listener.
+ */
+static inline int version_set_only(uint32_t major, uint32_t minor)
+{
+	char attrs[64];
+	struct nlattr *nest = (void *)attrs;
+	int inner = NLA_HDRLEN;
+
+	inner = put_attr(attrs, inner, NFSD_A_VERSION_MAJOR,
+			 &major, sizeof(major));
+	inner = put_attr(attrs, inner, NFSD_A_VERSION_MINOR,
+			 &minor, sizeof(minor));
+	inner = put_attr(attrs, inner, NFSD_A_VERSION_ENABLED, NULL, 0);
+	nest->nla_type = NFSD_A_SERVER_PROTO_VERSION | NLA_F_NESTED;
+	nest->nla_len = inner;
+
+	return genl_request(NFSD_CMD_VERSION_SET, attrs, NLA_ALIGN4(inner));
+}
+
+/* Start (@n > 0) or stop (@n == 0) nfsd threads in this netns. */
+static inline int threads_set(int n)
+{
+	char attrs[64];
+	uint32_t v = n;
+	int off = put_attr(attrs, 0, NFSD_A_SERVER_THREADS, &v, sizeof(v));
+
+	return genl_request(NFSD_CMD_THREADS_SET, attrs, off);
+}
+
+#endif /* __SELFTESTS_NFSD_NETLINK_H__ */
diff --git a/tools/testing/selftests/nfsd/nfsd_netlink_listener.c b/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
index 1294057b6f62..f6f8aa1e1cfe 100644
--- a/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
+++ b/tools/testing/selftests/nfsd/nfsd_netlink_listener.c
@@ -46,267 +46,13 @@
 #include <linux/nfsd_netlink.h>
 
 #include "../kselftest_harness.h"
+#include "nfsd_netlink.h"
 
-#define NFS_PROGRAM			100003
-#define NFS_ACL_PROGRAM			100227
-
-#define NLA_ALIGN4(len)			(((len) + 3) & ~3)
 #define TEST_PORT			20049
 #define MAX_LISTENERS			8
-#define RECV_TIMEO_SEC			30
-
-static int nfsd_family = -1;		/* set per-test in FIXTURE_SETUP */
-
-/* Extack message from the last genl_request(); empty if there was none. */
-static char last_extack[128];
-
-static void die(const char *msg)
-{
-	perror(msg);
-	exit(1);
-}
-
-/* ------------------- minimal generic-netlink plumbing ------------------- */
-
-static int genl_open(void)
-{
-	struct sockaddr_nl sa = { .nl_family = AF_NETLINK };
-	struct timeval tv = { .tv_sec = RECV_TIMEO_SEC };
-	int fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
-	int on = 1;
-
-	if (fd < 0)
-		die("socket(NETLINK_GENERIC)");
-	if (bind(fd, (void *)&sa, sizeof(sa)) < 0)
-		die("bind(netlink)");
-	setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
-	/*
-	 * Ask for extack, and cap the ack so the request is not echoed back:
-	 * the TLVs then always follow the fixed part of the error message.
-	 */
-	setsockopt(fd, SOL_NETLINK, NETLINK_EXT_ACK, &on, sizeof(on));
-	setsockopt(fd, SOL_NETLINK, NETLINK_CAP_ACK, &on, sizeof(on));
-	return fd;
-}
-
-/* Stash the extack message of an ack, if it carries one. */
-static void parse_extack(const char *rbuf)
-{
-	const struct nlmsghdr *nlh = (const void *)rbuf;
-	const struct nlattr *na;
-	int off, left;
-
-	last_extack[0] = '\0';
-	if (nlh->nlmsg_type != NLMSG_ERROR ||
-	    !(nlh->nlmsg_flags & NLM_F_ACK_TLVS))
-		return;
-
-	off = NLMSG_HDRLEN + NLMSG_ALIGN(sizeof(struct nlmsgerr));
-	left = nlh->nlmsg_len - off;
-	na = (const void *)(rbuf + off);
-
-	while (left >= (int)NLA_HDRLEN) {
-		if ((na->nla_type & NLA_TYPE_MASK) == NLMSGERR_ATTR_MSG) {
-			strncpy(last_extack, (const char *)na + NLA_HDRLEN,
-				sizeof(last_extack) - 1);
-			last_extack[sizeof(last_extack) - 1] = '\0';
-			return;
-		}
-		left -= NLA_ALIGN4(na->nla_len);
-		na = (const void *)((const char *)na + NLA_ALIGN4(na->nla_len));
-	}
-}
-
-/* Append an attribute at @off; return the new (aligned) offset. */
-static int put_attr(char *buf, int off, uint16_t type,
-		    const void *data, int len)
-{
-	struct nlattr *na = (void *)(buf + off);
-
-	na->nla_type = type;
-	na->nla_len = NLA_HDRLEN + len;
-	if (len)
-		memcpy(buf + off + NLA_HDRLEN, data, len);
-	return off + NLA_ALIGN4(NLA_HDRLEN + len);
-}
-
-/* Build a genl message header into @buf; return the offset past it. */
-static int genl_hdr(char *buf, uint16_t type, uint16_t flags, uint8_t cmd)
-{
-	struct nlmsghdr *nlh = (void *)buf;
-	struct genlmsghdr *gnl = (void *)(buf + NLMSG_HDRLEN);
-
-	memset(buf, 0, NLMSG_HDRLEN + GENL_HDRLEN);
-	nlh->nlmsg_type = type;
-	nlh->nlmsg_flags = flags;
-	nlh->nlmsg_seq = 1;
-	gnl->cmd = cmd;
-	gnl->version = 1;
-	return NLMSG_HDRLEN + GENL_HDRLEN;
-}
-
-/* Send an nfsd command with an ACK; return the ACK errno (<= 0). */
-static int genl_request(uint8_t cmd, const char *attrs, int attrs_len)
-{
-	char buf[1 << 20], rbuf[4096];
-	struct nlmsghdr *nlh = (void *)buf;
-	int fd = genl_open();
-	int off, n, ret;
-
-	off = genl_hdr(buf, nfsd_family, NLM_F_REQUEST | NLM_F_ACK, cmd);
-	if (attrs_len) {
-		memcpy(buf + off, attrs, attrs_len);
-		off += attrs_len;
-	}
-	nlh->nlmsg_len = off;
-
-	if (send(fd, buf, off, 0) < 0)
-		die("send(genl)");
-
-	last_extack[0] = '\0';
-	n = recv(fd, rbuf, sizeof(rbuf), 0);
-	if (n < 0) {
-		ret = (errno == EAGAIN || errno == EWOULDBLOCK) ? -ETIMEDOUT : -errno;
-	} else if (((struct nlmsghdr *)rbuf)->nlmsg_type == NLMSG_ERROR) {
-		ret = ((struct nlmsgerr *)NLMSG_DATA(rbuf))->error;
-		parse_extack(rbuf);
-	} else {
-		ret = 0;
-	}
-	close(fd);
-	return ret;
-}
-
-/*
- * Send a command with attributes and return the full reply message; -errno
- * on failure. NLM_F_ACK is left off: the kernel reports an error either way,
- * so the first message back is the reply whenever there is one.
- */
-static int genl_request_reply_attrs(uint8_t cmd, const char *attrs,
-				    int attrs_len, char *rbuf, size_t rlen)
-{
-	char buf[1 << 20];
-	struct nlmsghdr *nlh = (void *)buf;
-	int fd = genl_open();
-	int off, n, ret;
-
-	off = genl_hdr(buf, nfsd_family, NLM_F_REQUEST, cmd);
-	if (attrs_len) {
-		memcpy(buf + off, attrs, attrs_len);
-		off += attrs_len;
-	}
-	nlh->nlmsg_len = off;
 
-	if (send(fd, buf, off, 0) < 0)
-		die("send(genl reply)");
-
-	n = recv(fd, rbuf, rlen, 0);
-	if (n < 0)
-		ret = (errno == EAGAIN || errno == EWOULDBLOCK) ? -ETIMEDOUT : -errno;
-	else if (((struct nlmsghdr *)rbuf)->nlmsg_type == NLMSG_ERROR)
-		ret = ((struct nlmsgerr *)NLMSG_DATA(rbuf))->error;
-	else
-		ret = n;
-	close(fd);
-	return ret;
-}
-
-static int genl_request_reply(uint8_t cmd, char *rbuf, size_t rlen)
-{
-	return genl_request_reply_attrs(cmd, NULL, 0, rbuf, rlen);
-}
-
-/* Resolve the "nfsd" genl family id; -1 if not registered. */
-static int genl_resolve_nfsd(void)
-{
-	char buf[1024], rbuf[4096];
-	struct nlmsghdr *nlh = (void *)buf;
-	struct nlmsghdr *rh = (void *)rbuf;
-	struct nlattr *na;
-	int fd, off, left, id = -1;
-
-	fd = genl_open();
-	off = genl_hdr(buf, GENL_ID_CTRL, NLM_F_REQUEST, CTRL_CMD_GETFAMILY);
-	off = put_attr(buf, off, CTRL_ATTR_FAMILY_NAME,
-		       NFSD_FAMILY_NAME, sizeof(NFSD_FAMILY_NAME));
-	nlh->nlmsg_len = off;
-
-	if (send(fd, buf, off, 0) < 0)
-		die("send(GETFAMILY)");
-	if (recv(fd, rbuf, sizeof(rbuf), 0) < 0)
-		die("recv(GETFAMILY)");
-	close(fd);
-
-	if (rh->nlmsg_type == NLMSG_ERROR)
-		return -1;
-
-	na = (void *)((char *)NLMSG_DATA(rh) + GENL_HDRLEN);
-	left = rh->nlmsg_len - NLMSG_HDRLEN - GENL_HDRLEN;
-	while (left >= (int)NLA_HDRLEN) {
-		if (na->nla_type == CTRL_ATTR_FAMILY_ID) {
-			id = *(uint16_t *)((char *)na + NLA_HDRLEN);
-			break;
-		}
-		left -= NLA_ALIGN4(na->nla_len);
-		na = (void *)((char *)na + NLA_ALIGN4(na->nla_len));
-	}
-	return id;
-}
-
-/* ------------------- listener request builders ------------------- */
-
-/* Fine-grained control for negative tests: any field can be omitted/malformed. */
-struct raw_listener {
-	const char *xprt;	/* NULL -> omit NFSD_A_SOCK_TRANSPORT_NAME */
-	int emit_addr;		/* 0 -> omit NFSD_A_SOCK_ADDR */
-	const void *addr;
-	int addr_len;		/* bytes to emit for NFSD_A_SOCK_ADDR */
-};
-
-static int put_raw_listener(char *buf, int off, const struct raw_listener *r)
-{
-	struct nlattr *nest = (void *)(buf + off);
-	int inner = off + NLA_HDRLEN;
-
-	if (r->emit_addr)
-		inner = put_attr(buf, inner, NFSD_A_SOCK_ADDR, r->addr, r->addr_len);
-	if (r->xprt)
-		inner = put_attr(buf, inner, NFSD_A_SOCK_TRANSPORT_NAME,
-				 r->xprt, strlen(r->xprt) + 1);
-	nest->nla_type = NFSD_A_SERVER_SOCK_ADDR | NLA_F_NESTED;
-	nest->nla_len = inner - off;
-	return off + NLA_ALIGN4(nest->nla_len);
-}
-
-/* Well-formed loopback listener for @family (AF_INET or AF_INET6). */
-static int put_listener_af(char *buf, int off, const char *xprt, int family,
-			   uint16_t port)
-{
-	struct sockaddr_storage ss = {0};
-	struct raw_listener r = { .xprt = xprt, .emit_addr = 1, .addr = &ss };
-
-	if (family == AF_INET6) {
-		struct sockaddr_in6 *s6 = (void *)&ss;
-
-		s6->sin6_family = AF_INET6;
-		s6->sin6_port = htons(port);
-		s6->sin6_addr = in6addr_loopback;
-		r.addr_len = sizeof(*s6);
-	} else {
-		struct sockaddr_in *s4 = (void *)&ss;
-
-		s4->sin_family = AF_INET;
-		s4->sin_port = htons(port);
-		s4->sin_addr.s_addr = htonl(INADDR_LOOPBACK);
-		r.addr_len = sizeof(*s4);
-	}
-	return put_raw_listener(buf, off, &r);
-}
-
-static int put_listener(char *buf, int off, const char *xprt, uint16_t port)
-{
-	return put_listener_af(buf, off, xprt, AF_INET, port);
-}
+#define NFS_PROGRAM			100003
+#define NFS_ACL_PROGRAM			100227
 
 /* ------------------- LISTENER_GET parsing ------------------- */
 
@@ -374,32 +120,6 @@ static int parse_listener_get(const char *rbuf, int len,
 
 /* ------------------- convenience wrappers ------------------- */
 
-static int listener_set(const char *attrs, int len)
-{
-	return genl_request(NFSD_CMD_LISTENER_SET, attrs, len);
-}
-
-/*
- * Enable exactly one NFS version in this netns. NFSD_CMD_VERSION_SET clears
- * every version first, so one nest is enough to leave the server v4-only.
- * It refuses once a serv exists, so call it before any listener.
- */
-static int version_set_only(uint32_t major, uint32_t minor)
-{
-	char attrs[64];
-	struct nlattr *nest = (void *)attrs;
-	int inner = NLA_HDRLEN;
-
-	inner = put_attr(attrs, inner, NFSD_A_VERSION_MAJOR,
-			 &major, sizeof(major));
-	inner = put_attr(attrs, inner, NFSD_A_VERSION_MINOR,
-			 &minor, sizeof(minor));
-	inner = put_attr(attrs, inner, NFSD_A_VERSION_ENABLED, NULL, 0);
-	nest->nla_type = NFSD_A_SERVER_PROTO_VERSION | NLA_F_NESTED;
-	nest->nla_len = inner;
-
-	return genl_request(NFSD_CMD_VERSION_SET, attrs, NLA_ALIGN4(inner));
-}
 
 /* ------------------- userspace-rpcbind ------------------- */
 
@@ -552,15 +272,6 @@ static struct listener_ent *find_listener(struct listener_ent *e, int n,
 	return NULL;
 }
 
-/* Start (@n > 0) or stop (@n == 0) nfsd threads in this netns. */
-static int threads_set(int n)
-{
-	char attrs[64];
-	uint32_t v = n;
-	int off = put_attr(attrs, 0, NFSD_A_SERVER_THREADS, &v, sizeof(v));
-
-	return genl_request(NFSD_CMD_THREADS_SET, attrs, off);
-}
 
 /* ------------------- per-netns local rpcbind stub ------------------- */
 

-- 
2.55.0

[PATCH v2 7/8] selftests/nfsd: add cross-namespace isolation tests

[Jeff Layton]

Check that one namespace's NFSD settings and running server stay out of
another's, now that the control plane is serialized per namespace.

  max_blksize_is_per_netns
	max_block_size is reachable only through a per-netns nfsd
	filesystem but used to live in a module-wide variable. Read it in
	a throwaway namespace, change it in a second, read it again in a
	third; the two reads must agree. No value is hardcoded -- the
	default is derived from the size of memory, so the test picks a
	target that differs from whatever this machine reports.

  *_busy_is_per_netns, pool_stats_readable_with_foreign_server,
  listener_get_does_not_show_foreign_listeners
	max_block_size, VERSION_SET and nfsv4leasetime all refuse with
	-EBUSY once that namespace has a serv. A peer namespace holds a
	server up while the test pokes its own, which must not be
	affected. LISTENER_GET must not report the peer's listener.

  expkey_flush_with_foreign_server
	writing /proc/net/rpc/nfsd.fh/flush is the one userspace path that
	reaches nfsd_file_cache_purge(), and so the one that takes the
	file cache lock with nothing else held.

Servers are created with NFSD_A_SERVER_SOCK_USERSPACE_RPCBIND so nothing
calls out to rpcbind, and restricted to NFSv4.1 so nfsd_needs_lockd()
stays false. Each namespace mounts its own nfsd filesystem on a private
tmpfs; the module creates /proc/fs/nfs, not /proc/fs/nfsd, so there is no
mountpoint to borrow.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 tools/testing/selftests/nfsd/.gitignore            |   1 +
 tools/testing/selftests/nfsd/Makefile              |   1 +
 .../testing/selftests/nfsd/nfsd_netns_isolation.c  | 468 +++++++++++++++++++++
 tools/testing/selftests/nfsd/settings              |   2 +-
 4 files changed, 471 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nfsd/.gitignore b/tools/testing/selftests/nfsd/.gitignore
index 19e6dec04d8e..0304b80eb844 100644
--- a/tools/testing/selftests/nfsd/.gitignore
+++ b/tools/testing/selftests/nfsd/.gitignore
@@ -1 +1,2 @@
 nfsd_netlink_listener
+nfsd_netns_isolation
diff --git a/tools/testing/selftests/nfsd/Makefile b/tools/testing/selftests/nfsd/Makefile
index 15ac65549d25..2b7c44c3fb00 100644
--- a/tools/testing/selftests/nfsd/Makefile
+++ b/tools/testing/selftests/nfsd/Makefile
@@ -2,5 +2,6 @@
 CFLAGS += $(KHDR_INCLUDES) -Wall
 
 TEST_GEN_PROGS := nfsd_netlink_listener
+TEST_GEN_PROGS += nfsd_netns_isolation
 
 include ../lib.mk
diff --git a/tools/testing/selftests/nfsd/nfsd_netns_isolation.c b/tools/testing/selftests/nfsd/nfsd_netns_isolation.c
new file mode 100644
index 000000000000..b39ea6a908fd
--- /dev/null
+++ b/tools/testing/selftests/nfsd/nfsd_netns_isolation.c
@@ -0,0 +1,468 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Namespace-isolation tests for the NFSD control plane.
+ *
+ * NFSD's per-namespace settings used to sit behind one module-wide mutex,
+ * and one of them -- the maximum READ/WRITE payload -- was a module-wide
+ * variable reachable through a per-netns file. These tests pin down the
+ * boundary: what one namespace does to its own server must not be visible
+ * to, or block, another.
+ *
+ * Every namespace here gets a private net + mount namespace, a tmpfs on
+ * /mnt so nothing escapes, and its own nfsd filesystem mounted on
+ * /mnt/nfsd. The module creates /proc/fs/nfs, not /proc/fs/nfsd, so the
+ * mount has to be made by hand; it is also what ties a running server to
+ * this test, since nfsd_umount() stops the threads.
+ *
+ * Servers are created with NFSD_A_SERVER_SOCK_USERSPACE_RPCBIND so nothing
+ * ever calls out to rpcbind, and restricted to NFSv4.1 so that
+ * nfsd_needs_lockd() stays false and no lockd instance is started.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mount.h>
+#include <sys/prctl.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <net/if.h>
+#include <netinet/in.h>
+
+#include "../kselftest_harness.h"
+#include "nfsd_netlink.h"
+
+#define NFSD_MNT		"/mnt/nfsd"
+#define TEST_PORT		20049
+
+/* netns_enter() could not build a usable namespace; not a test failure. */
+#define NETNS_NO_SETUP		INT_MIN
+
+/*
+ * Build a private net + mount namespace with an nfsd filesystem on
+ * /mnt/nfsd and loopback up. Returns 0, or NETNS_NO_SETUP when the
+ * environment will not allow it.
+ */
+static int netns_enter(void)
+{
+	struct ifreq ifr = {0};
+	int s;
+
+	if (unshare(CLONE_NEWNET | CLONE_NEWNS) < 0)
+		return NETNS_NO_SETUP;
+	if (mount("", "/", NULL, MS_REC | MS_PRIVATE, NULL) < 0)
+		return NETNS_NO_SETUP;
+
+	/*
+	 * Everything below is created inside this mount namespace only, so
+	 * the mkdir cannot leave anything behind on the host.
+	 */
+	if (mount("tmpfs", "/mnt", "tmpfs", 0, NULL) < 0)
+		return NETNS_NO_SETUP;
+	if (mkdir(NFSD_MNT, 0755) < 0)
+		return NETNS_NO_SETUP;
+	if (mount("nfsd", NFSD_MNT, "nfsd", 0, NULL) < 0)
+		return NETNS_NO_SETUP;
+
+	s = socket(AF_INET, SOCK_DGRAM, 0);
+	if (s < 0)
+		return NETNS_NO_SETUP;
+	strcpy(ifr.ifr_name, "lo");
+	if (ioctl(s, SIOCGIFFLAGS, &ifr) < 0) {
+		close(s);
+		return NETNS_NO_SETUP;
+	}
+	ifr.ifr_flags |= IFF_UP | IFF_RUNNING;
+	if (ioctl(s, SIOCSIFFLAGS, &ifr) < 0) {
+		close(s);
+		return NETNS_NO_SETUP;
+	}
+	close(s);
+
+	nfsd_family = genl_resolve_nfsd();
+	if (nfsd_family < 0)
+		return NETNS_NO_SETUP;
+	return 0;
+}
+
+/* ------------------- nfsdfs file access ------------------- */
+
+static int nfsd_file_read(const char *name, char *buf, size_t len)
+{
+	char path[128];
+	int fd, n;
+
+	snprintf(path, sizeof(path), "%s/%s", NFSD_MNT, name);
+	fd = open(path, O_RDONLY);
+	if (fd < 0)
+		return -errno;
+	n = read(fd, buf, len - 1);
+	close(fd);
+	if (n < 0)
+		return -errno;
+	buf[n] = '\0';
+	return n;
+}
+
+static int nfsd_file_read_int(const char *name)
+{
+	char buf[64];
+	int n = nfsd_file_read(name, buf, sizeof(buf));
+
+	if (n < 0)
+		return n;
+	return atoi(buf);
+}
+
+static int nfsd_file_write_int(const char *name, int val)
+{
+	char path[128], buf[64];
+	int fd, n, len;
+
+	snprintf(path, sizeof(path), "%s/%s", NFSD_MNT, name);
+	len = snprintf(buf, sizeof(buf), "%d\n", val);
+	fd = open(path, O_WRONLY);
+	if (fd < 0)
+		return -errno;
+	n = write(fd, buf, len);
+	close(fd);
+	return n < 0 ? -errno : 0;
+}
+
+/* ------------------- server lifecycle ------------------- */
+
+/*
+ * Bring up a v4.1-only server on a loopback listener, owning rpcbind
+ * registration in userspace so the kernel never issues an rpcbind call.
+ */
+static int server_start(uint16_t port, int nthreads)
+{
+	char attrs[128];
+	int off;
+	int ret;
+
+	ret = version_set_only(4, 1);
+	if (ret)
+		return ret;
+
+	off = put_listener(attrs, 0, "tcp", port);
+	off = put_attr(attrs, off, NFSD_A_SERVER_SOCK_USERSPACE_RPCBIND,
+		       NULL, 0);
+	ret = listener_set(attrs, off);
+	if (ret)
+		return ret;
+
+	return threads_set(nthreads);
+}
+
+static void server_stop(void)
+{
+	threads_set(0);
+	listener_set(NULL, 0);
+}
+
+/* ------------------- run a callback in a fresh namespace ------------------- */
+
+/*
+ * Fork a child into its own namespace, run @fn there and hand back what it
+ * returned. Used for the checks that only need one namespace at a time.
+ */
+static int netns_run(int (*fn)(long), long arg)
+{
+	int p[2], ret = -EIO;
+	pid_t pid;
+
+	if (pipe(p) < 0)
+		return -errno;
+
+	pid = fork();
+	if (pid < 0) {
+		close(p[0]);
+		close(p[1]);
+		return -errno;
+	}
+	if (pid == 0) {
+		int r = netns_enter();
+
+		if (r == 0)
+			r = fn(arg);
+		if (write(p[1], &r, sizeof(r)) != sizeof(r))
+			_exit(1);
+		_exit(0);
+	}
+
+	close(p[1]);
+	if (read(p[0], &ret, sizeof(ret)) != sizeof(ret))
+		ret = -EIO;
+	close(p[0]);
+	waitpid(pid, NULL, 0);
+	return ret;
+}
+
+/* ------------------- a peer namespace held open ------------------- */
+
+/*
+ * A second namespace running a server for as long as the test needs it.
+ * The peer reports readiness on a pipe and waits for a byte before tearing
+ * down, so the test can be sure the server is up while it pokes its own
+ * namespace.
+ */
+struct peer {
+	pid_t pid;
+	int wake;		/* write here to let the peer exit */
+	int ready;		/* peer writes its status here */
+};
+
+static int peer_start(struct peer *pr, uint16_t port)
+{
+	int wake[2], ready[2];
+	pid_t ppid = getpid();
+	char status;
+
+	if (pipe(wake) < 0)
+		return -errno;
+	if (pipe(ready) < 0) {
+		close(wake[0]);
+		close(wake[1]);
+		return -errno;
+	}
+
+	/* peer_stop() handles a dead peer itself; do not die of SIGPIPE first. */
+	signal(SIGPIPE, SIG_IGN);
+
+	pr->pid = fork();
+	if (pr->pid < 0) {
+		close(wake[0]); close(wake[1]);
+		close(ready[0]); close(ready[1]);
+		return -errno;
+	}
+	if (pr->pid == 0) {
+		char c;
+		int r;
+
+		/* Hold no writer of our own, so read() below sees EOF. */
+		close(wake[1]);
+		close(ready[0]);
+		/*
+		 * A parent that dies without reaching peer_stop() would leave
+		 * this namespace and its server pinned by a process blocked
+		 * forever in read().
+		 */
+		prctl(PR_SET_PDEATHSIG, SIGKILL);
+		if (getppid() != ppid)
+			_exit(1);
+
+		r = netns_enter();
+		if (r == 0)
+			r = server_start(port, 1);
+		status = r == NETNS_NO_SETUP ? 'S' : (r ? 'E' : 'R');
+		if (write(ready[1], &status, 1) != 1)
+			_exit(1);
+		/* Hold the namespace open until the test is done with it. */
+		if (read(wake[0], &c, 1) == 1 && status == 'R')
+			server_stop();
+		_exit(0);
+	}
+
+	close(wake[0]);
+	close(ready[1]);
+	pr->wake = wake[1];
+	pr->ready = ready[0];
+
+	if (read(pr->ready, &status, 1) != 1)
+		status = 'E';
+	if (status == 'S')
+		return NETNS_NO_SETUP;
+	return status == 'R' ? 0 : -EIO;
+}
+
+static void peer_stop(struct peer *pr)
+{
+	char c = 'x';
+
+	if (pr->pid <= 0)
+		return;
+	if (write(pr->wake, &c, 1) != 1)
+		kill(pr->pid, SIGKILL);
+	close(pr->wake);
+	close(pr->ready);
+	waitpid(pr->pid, NULL, 0);
+	pr->pid = 0;
+}
+
+/* ===================== max_block_size is per-namespace ===================== */
+
+static int read_max_blksize(long unused)
+{
+	(void)unused;
+	return nfsd_file_read_int("max_block_size");
+}
+
+static int write_max_blksize(long val)
+{
+	int ret = nfsd_file_write_int("max_block_size", (int)val);
+
+	if (ret)
+		return ret;
+	/* Report what stuck, so the caller knows the write was accepted. */
+	return nfsd_file_read_int("max_block_size");
+}
+
+/*
+ * max_block_size is reachable only through a per-netns nfsd filesystem, so
+ * a write in one namespace must not be visible in another. Read it in a
+ * throwaway namespace, change it in a second, then read it again in a
+ * third: the two reads have to agree.
+ *
+ * No value is hardcoded. The default is derived from the size of memory,
+ * so the test picks a target that differs from whatever this machine uses.
+ */
+TEST(max_blksize_is_per_netns)
+{
+	int before, after, wrote, target;
+
+	before = netns_run(read_max_blksize, 0);
+	if (before == NETNS_NO_SETUP)
+		SKIP(return, "cannot set up a private nfsd namespace");
+	ASSERT_GE(before, 0);
+
+	/* Any legal value that is not the one this machine already reports. */
+	target = (before == 262144) ? 131072 : 262144;
+
+	wrote = netns_run(write_max_blksize, target);
+	ASSERT_EQ(target, wrote)
+		TH_LOG("second namespace did not accept max_block_size=%d",
+		       target);
+
+	after = netns_run(read_max_blksize, 0);
+	ASSERT_GE(after, 0);
+	EXPECT_EQ(before, after)
+		TH_LOG("max_block_size leaked between namespaces: %d -> %d",
+		       before, after);
+}
+
+/* ===================== a busy namespace does not busy others ===================== */
+
+FIXTURE(nfsd_peer) {
+	struct peer pr;
+};
+
+FIXTURE_SETUP(nfsd_peer)
+{
+	int ret;
+
+	if (geteuid() != 0)
+		SKIP(return, "must be run as root");
+
+	/* The peer namespace comes first; it must not be ours. */
+	memset(&self->pr, 0, sizeof(self->pr));
+	ret = peer_start(&self->pr, TEST_PORT);
+	if (ret == NETNS_NO_SETUP)
+		SKIP(return, "cannot set up a private nfsd namespace");
+	if (ret)
+		SKIP(return, "peer namespace could not start a server: %d", ret);
+
+	/* Now put this process in a namespace of its own, with no server. */
+	if (netns_enter() != 0)
+		SKIP(return, "cannot set up a private nfsd namespace");
+}
+
+FIXTURE_TEARDOWN(nfsd_peer)
+{
+	if (nfsd_family >= 0)
+		server_stop();
+	peer_stop(&self->pr);
+}
+
+/*
+ * write_maxblksize() refuses with -EBUSY while that namespace has a serv.
+ * The check is on nn->nfsd_serv, so a server belonging to someone else must
+ * not trip it.
+ */
+TEST_F(nfsd_peer, maxblksize_busy_is_per_netns)
+{
+	int cur = nfsd_file_read_int("max_block_size");
+
+	ASSERT_GE(cur, 0);
+	EXPECT_EQ(0, nfsd_file_write_int("max_block_size",
+					 cur == 262144 ? 131072 : 262144))
+		TH_LOG("max_block_size refused while another netns has a server");
+}
+
+/*
+ * NFSD_CMD_VERSION_SET returns -EBUSY once the namespace has a serv. A
+ * server in the peer namespace must not reach us.
+ */
+TEST_F(nfsd_peer, version_set_busy_is_per_netns)
+{
+	EXPECT_EQ(0, version_set_only(4, 1))
+		TH_LOG("VERSION_SET refused while another netns has a server");
+}
+
+/*
+ * The grace and lease times are per-namespace too, and gated on the same
+ * nn->nfsd_serv check.
+ */
+TEST_F(nfsd_peer, leasetime_busy_is_per_netns)
+{
+	EXPECT_EQ(0, nfsd_file_write_int("nfsv4leasetime", 60))
+		TH_LOG("nfsv4leasetime refused while another netns has a server");
+}
+
+/*
+ * pool_stats runs its seq_file under the mutex that also guards the serv.
+ * Reading it here must not be affected by the peer's server, and must
+ * report this namespace, which has no threads at all.
+ */
+TEST_F(nfsd_peer, pool_stats_readable_with_foreign_server)
+{
+	char buf[4096];
+	int n = nfsd_file_read("pool_stats", buf, sizeof(buf));
+
+	ASSERT_GE(n, 0)
+		TH_LOG("pool_stats unreadable: %s", strerror(-n));
+	EXPECT_NE(NULL, strstr(buf, "packets-arrived"));
+}
+
+/*
+ * LISTENER_GET reports this namespace's listeners. With a server running
+ * next door and none here, the list must come back empty rather than
+ * showing the peer's.
+ */
+TEST_F(nfsd_peer, listener_get_does_not_show_foreign_listeners)
+{
+	char rbuf[8192];
+	int n = genl_request_reply(NFSD_CMD_LISTENER_GET, rbuf, sizeof(rbuf));
+
+	ASSERT_GT(n, 0);
+	EXPECT_EQ(NULL, memmem(rbuf, n, "tcp", 4))
+		TH_LOG("LISTENER_GET leaked a listener from another netns");
+}
+
+/*
+ * The file cache is one host-wide object, but the flush that reaches it is
+ * driven from a per-netns file. Doing it here while the peer has a server
+ * up must be harmless -- this is the path that takes the cache lock with
+ * nothing else held.
+ */
+TEST_F(nfsd_peer, expkey_flush_with_foreign_server)
+{
+	int fd = open("/proc/net/rpc/nfsd.fh/flush", O_WRONLY);
+
+	if (fd < 0)
+		SKIP(return, "no /proc/net/rpc/nfsd.fh/flush: %s",
+		     strerror(errno));
+	EXPECT_EQ(2, write(fd, "1\n", 2));
+	close(fd);
+}
+
+TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/nfsd/settings b/tools/testing/selftests/nfsd/settings
index 6091b45d226b..694d70710ff0 100644
--- a/tools/testing/selftests/nfsd/settings
+++ b/tools/testing/selftests/nfsd/settings
@@ -1 +1 @@
-timeout=120
+timeout=300

-- 
2.55.0

[PATCH v2 8/8] selftests/nfsd: add a cross-namespace control-plane soak

[Jeff Layton]

Drive the NFSD control plane from many namespaces at once, to exercise
every edge of the lock nesting:

	nn->nfsd_mutex -> nfsd_global_mutex -> nfsd_file_cache_mutex

Each worker owns a namespace and churns between "no serv" and "serv with
threads", mixing in the operations that cross into the host-wide locks:

  start/stop a server	nn -> global (nfsd_users 0->1->0, notifiers)
			   -> cache (cache init/shutdown)
  empty LISTENER_SET	creates and destroys a serv in one call, so the
			   host-wide refcount goes 0->1->0 on its own
  nfsd.fh/flush		the cache lock with nothing else held
  read filecache	likewise
  read pool_stats	nn, reached through svc_info.mutex

Random churn alone is a weak race finder, so the run ends with
synchronized rounds where every namespace attempts the host-wide 0->1
transition at the same instant. That window only opened up once the
per-namespace lock stopped serializing namespaces against each other.

The test asserts little by itself; the result that matters is that the
kernel did not warn. The taint word is sampled before and after and a
newly set TAINT_WARN fails the run, which catches lockdep splats,
WARN_ON()s and refcount saturation alike. Add PROVE_LOCKING and friends
to the config, and log a note if lockdep looks absent.

Two things can silently gut the coverage, so both are reported rather
than left to look like a pass:

  - without lockdep there is very little for the kernel to complain
    about;
  - a server already running outside these namespaces pins the
    host-wide refcount above zero for the whole run, so the 0->1
    transition the synchronized rounds are built around never happens.

Each worker leaves only NFSv4.1 enabled. A fresh namespace has v3 on,
which makes nfsd_needs_lockd() true, and the lockd that comes up then
waits out an RPC timeout against an rpcbind that is not there on every
single server start.

Every namespace binds the same port; they are isolated, so that must
work, and a bind failure is a louder signal than a silent pass. Sized
from nproc, capped at 8 namespaces and 5s of churn by default, and
tunable with NFSD_STRESS_WORKERS and NFSD_STRESS_SECS. The barrier waits
with a deadline so a worker that dies cannot wedge the run, and the test
carries an explicit timeout because the harness otherwise caps it at
TEST_TIMEOUT_DEFAULT.

Assisted-by: LLM
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 tools/testing/selftests/nfsd/.gitignore          |   1 +
 tools/testing/selftests/nfsd/Makefile            |   1 +
 tools/testing/selftests/nfsd/config              |   6 +
 tools/testing/selftests/nfsd/nfsd_netns_stress.c | 572 +++++++++++++++++++++++
 tools/testing/selftests/nfsd/settings            |   2 +-
 5 files changed, 581 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nfsd/.gitignore b/tools/testing/selftests/nfsd/.gitignore
index 0304b80eb844..2347491c634d 100644
--- a/tools/testing/selftests/nfsd/.gitignore
+++ b/tools/testing/selftests/nfsd/.gitignore
@@ -1,2 +1,3 @@
 nfsd_netlink_listener
 nfsd_netns_isolation
+nfsd_netns_stress
diff --git a/tools/testing/selftests/nfsd/Makefile b/tools/testing/selftests/nfsd/Makefile
index 2b7c44c3fb00..b29bf642c0ad 100644
--- a/tools/testing/selftests/nfsd/Makefile
+++ b/tools/testing/selftests/nfsd/Makefile
@@ -3,5 +3,6 @@ CFLAGS += $(KHDR_INCLUDES) -Wall
 
 TEST_GEN_PROGS := nfsd_netlink_listener
 TEST_GEN_PROGS += nfsd_netns_isolation
+TEST_GEN_PROGS += nfsd_netns_stress
 
 include ../lib.mk
diff --git a/tools/testing/selftests/nfsd/config b/tools/testing/selftests/nfsd/config
index 0eef03af3503..c6407806b58b 100644
--- a/tools/testing/selftests/nfsd/config
+++ b/tools/testing/selftests/nfsd/config
@@ -12,3 +12,9 @@ CONFIG_INOTIFY_USER=y
 CONFIG_SUNRPC=y
 CONFIG_NFSD=y
 CONFIG_NFSD_V4=y
+
+# nfsd_netns_stress leans on lockdep to find anything; without these the
+# soak still runs but proves very little.
+CONFIG_PROVE_LOCKING=y
+CONFIG_DEBUG_MUTEXES=y
+CONFIG_DEBUG_ATOMIC_SLEEP=y
diff --git a/tools/testing/selftests/nfsd/nfsd_netns_stress.c b/tools/testing/selftests/nfsd/nfsd_netns_stress.c
new file mode 100644
index 000000000000..7ca278a5d4a0
--- /dev/null
+++ b/tools/testing/selftests/nfsd/nfsd_netns_stress.c
@@ -0,0 +1,572 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Concurrency soak for the NFSD control plane across network namespaces.
+ *
+ * NFSD's control plane is serialized by three locks, nested in this order:
+ *
+ *	nn->nfsd_mutex  ->  nfsd_global_mutex  ->  nfsd_file_cache_mutex
+ *
+ * Most of what used to be one module-wide mutex is now per-namespace, so
+ * namespaces run their control planes in parallel and only meet on the
+ * host-wide bits: the refcount that brings the open file cache and the
+ * NFSv4 global tables up and down, and the address-notifier registration.
+ *
+ * This test exists to drive every one of those edges at once from many
+ * namespaces. It asserts little by itself -- the point is to give lockdep
+ * something to work with, so it is close to worthless without
+ * CONFIG_PROVE_LOCKING=y. What it does check is that the kernel did not
+ * warn: the taint word is sampled before and after, and a newly set
+ * TAINT_WARN fails the run. That catches lockdep splats, WARN_ON()s and
+ * refcount saturation alike.
+ *
+ * Each worker drives these, which between them cover every edge:
+ *
+ *   start a server	nn -> global (nfsd_users 0->1, notifiers 0->1)
+ *			   -> cache (nfsd_file_cache_init)
+ *   stop a server	nn -> global (1->0) -> cache (cache_shutdown)
+ *   empty LISTENER_SET	creates and destroys a serv in one call, so the
+ *			   host-wide refcount goes 0->1->0 on its own
+ *   nfsd.fh/flush	the cache lock with nothing else held
+ *   read filecache	likewise
+ *   read pool_stats	nn, reached through svc_info.mutex
+ *
+ * Random churn alone is a weak race finder, so the run ends with
+ * synchronized rounds where every worker attempts the host-wide 0->1
+ * transition at the same instant. That is the window that only opened up
+ * once the per-namespace lock stopped serializing namespaces against each
+ * other.
+ *
+ * Tunable with NFSD_STRESS_WORKERS and NFSD_STRESS_SECS.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/mount.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <sys/wait.h>
+#include <net/if.h>
+#include <netinet/in.h>
+
+#include "../kselftest_harness.h"
+#include "nfsd_netlink.h"
+
+#define NFSD_MNT		"/mnt/nfsd"
+#define EXPKEY_FLUSH		"/proc/net/rpc/nfsd.fh/flush"
+
+/*
+ * Every worker binds the same port. They are in different namespaces, so
+ * that has to work; if isolation ever breaks, the second bind fails loudly
+ * rather than the test quietly passing.
+ */
+#define STRESS_PORT		20049
+
+#define DEFAULT_WORKERS		8
+#define DEFAULT_SECS		5
+#define MAX_WORKERS		64
+#define SYNC_ROUNDS		20
+#define BARRIER_TIMEOUT_SEC	30
+
+/*
+ * Generous: the harness caps a test at TEST_TIMEOUT_DEFAULT otherwise, and
+ * the churn duration is tunable.
+ */
+#define SOAK_TIMEOUT_SEC	600
+
+#define TAINT_WARN_BIT		(1UL << 9)
+
+/* Worker exit codes. */
+#define WORKER_OK		0
+#define WORKER_FAIL		1
+#define WORKER_NO_SETUP		2
+
+/* ------------------- cross-process barrier ------------------- */
+
+struct barrier {
+	unsigned int n;
+	unsigned int count;
+	unsigned int generation;
+	unsigned int aborted;	/* latched: a rendezvous was never completed */
+};
+
+/*
+ * Spin with a deadline rather than blocking, so a worker that dies cannot
+ * wedge the run.
+ *
+ * The first waiter to give up latches ->aborted, which kills the barrier
+ * for good: every later call returns at once instead of waiting out its
+ * own deadline. Without that latch a single missing worker costs
+ * BARRIER_TIMEOUT_SEC on every remaining rendezvous -- three per round,
+ * SYNC_ROUNDS rounds -- which runs into tens of minutes before the
+ * harness timeout fires.
+ *
+ * A giving-up waiter leaves its ->count increment behind. That is fine:
+ * once ->aborted is set nothing reads ->count again.
+ *
+ * Returns false if the rendezvous did not happen.
+ */
+static bool barrier_wait(struct barrier *b)
+{
+	unsigned int gen = __atomic_load_n(&b->generation, __ATOMIC_ACQUIRE);
+	time_t deadline = time(NULL) + BARRIER_TIMEOUT_SEC;
+
+	if (__atomic_load_n(&b->aborted, __ATOMIC_ACQUIRE))
+		return false;
+
+	if (__atomic_add_fetch(&b->count, 1, __ATOMIC_ACQ_REL) == b->n) {
+		__atomic_store_n(&b->count, 0, __ATOMIC_RELEASE);
+		__atomic_add_fetch(&b->generation, 1, __ATOMIC_ACQ_REL);
+		return true;
+	}
+	while (__atomic_load_n(&b->generation, __ATOMIC_ACQUIRE) == gen) {
+		if (__atomic_load_n(&b->aborted, __ATOMIC_ACQUIRE))
+			return false;
+		if (time(NULL) > deadline) {
+			__atomic_store_n(&b->aborted, 1, __ATOMIC_RELEASE);
+			return false;
+		}
+		sched_yield();
+	}
+	return true;
+}
+
+/* ------------------- namespace setup ------------------- */
+
+static int netns_enter(void)
+{
+	struct ifreq ifr = {0};
+	int s;
+
+	if (unshare(CLONE_NEWNET | CLONE_NEWNS) < 0)
+		return -errno;
+	if (mount("", "/", NULL, MS_REC | MS_PRIVATE, NULL) < 0)
+		return -errno;
+	if (mount("tmpfs", "/mnt", "tmpfs", 0, NULL) < 0)
+		return -errno;
+	if (mkdir(NFSD_MNT, 0755) < 0)
+		return -errno;
+	if (mount("nfsd", NFSD_MNT, "nfsd", 0, NULL) < 0)
+		return -errno;
+
+	s = socket(AF_INET, SOCK_DGRAM, 0);
+	if (s < 0)
+		return -errno;
+	strcpy(ifr.ifr_name, "lo");
+	if (ioctl(s, SIOCGIFFLAGS, &ifr) < 0) {
+		close(s);
+		return -errno;
+	}
+	ifr.ifr_flags |= IFF_UP | IFF_RUNNING;
+	if (ioctl(s, SIOCSIFFLAGS, &ifr) < 0) {
+		close(s);
+		return -errno;
+	}
+	close(s);
+
+	nfsd_family = genl_resolve_nfsd();
+	if (nfsd_family < 0)
+		return -ENOENT;
+	return 0;
+}
+
+/* ------------------- the operations ------------------- */
+
+static int read_nfsd_file(const char *name)
+{
+	char path[128], buf[4096];
+	int fd, n;
+
+	snprintf(path, sizeof(path), "%s/%s", NFSD_MNT, name);
+	fd = open(path, O_RDONLY);
+	if (fd < 0)
+		return -errno;
+	do {
+		n = read(fd, buf, sizeof(buf));
+	} while (n > 0);
+	close(fd);
+	return n < 0 ? -errno : 0;
+}
+
+/*
+ * The only userspace trigger that reaches nfsd_file_cache_purge(), and so
+ * the only one that takes the file cache lock with no other nfsd lock
+ * held. NFSD_CMD_CACHE_FLUSH does not get there: cache_purge() never calls
+ * the cache_detail's ->flush hook.
+ */
+static int expkey_flush(void)
+{
+	int fd = open(EXPKEY_FLUSH, O_WRONLY);
+	int n;
+
+	if (fd < 0)
+		return -errno;
+	n = write(fd, "1\n", 2);
+	close(fd);
+	return n < 0 ? -errno : 0;
+}
+
+/* Create a serv and tear it straight back down inside one call. */
+static int serv_cycle(void)
+{
+	return listener_set(NULL, 0);
+}
+
+static int server_up(int nthreads)
+{
+	char attrs[128];
+	int off, ret;
+
+	off = put_listener(attrs, 0, "tcp", STRESS_PORT);
+	off = put_attr(attrs, off, NFSD_A_SERVER_SOCK_USERSPACE_RPCBIND,
+		       NULL, 0);
+	ret = listener_set(attrs, off);
+	if (ret)
+		return ret;
+	return threads_set(nthreads);
+}
+
+/* threads_set(0) drops the last thread, which destroys the serv with it. */
+static int server_down(void)
+{
+	return threads_set(0);
+}
+
+/* ------------------- the worker ------------------- */
+
+struct worker_err {
+	const char *op;
+	int err;
+};
+
+static struct worker_err worker_fault;
+
+static bool fail(const char *op, int err)
+{
+	if (err == 0)
+		return false;
+	worker_fault.op = op;
+	worker_fault.err = err;
+	return true;
+}
+
+/*
+ * Random churn between "no serv" and "serv with threads", with the
+ * lock-crossing reads and flushes mixed in at both ends. Every call here
+ * is one that must succeed in the state it is issued from, so any error is
+ * a real failure rather than an expected race.
+ */
+static bool worker_churn(time_t deadline)
+{
+	bool up = false;
+
+	while (time(NULL) < deadline) {
+		int r = random() % 8;
+
+		if (!up) {
+			switch (r) {
+			case 0:
+			case 1:
+				if (fail("serv_cycle", serv_cycle()))
+					return false;
+				break;
+			case 2:
+				if (fail("version_set", version_set_only(4, 1)))
+					return false;
+				break;
+			case 3:
+				if (fail("flush", expkey_flush()))
+					return false;
+				break;
+			case 4:
+				if (fail("filecache", read_nfsd_file("filecache")))
+					return false;
+				break;
+			case 5:
+				if (fail("pool_stats", read_nfsd_file("pool_stats")))
+					return false;
+				break;
+			default:
+				if (fail("server_up", server_up(1 + random() % 3)))
+					return false;
+				up = true;
+				break;
+			}
+		} else {
+			switch (r) {
+			case 0:
+				if (fail("flush", expkey_flush()))
+					return false;
+				break;
+			case 1:
+				if (fail("filecache", read_nfsd_file("filecache")))
+					return false;
+				break;
+			case 2:
+				if (fail("pool_stats", read_nfsd_file("pool_stats")))
+					return false;
+				break;
+			case 3:
+				if (fail("threads_set", threads_set(1 + random() % 3)))
+					return false;
+				break;
+			default:
+				if (fail("server_down", server_down()))
+					return false;
+				up = false;
+				break;
+			}
+		}
+	}
+
+	if (up && fail("server_down", server_down()))
+		return false;
+	return true;
+}
+
+/*
+ * Every worker arrives at the barrier with no serv, so the host-wide
+ * refcount is at zero, then they all try to take it to one together. The
+ * second barrier lines up the drop back to zero the same way.
+ */
+static bool worker_sync_rounds(struct barrier *b)
+{
+	int i;
+
+	for (i = 0; i < SYNC_ROUNDS; i++) {
+		/*
+		 * A failed rendezvous means a peer is gone, which its own
+		 * exit status already reports. Stop the rounds rather than
+		 * stalling on every remaining barrier.
+		 */
+		if (!barrier_wait(b))
+			return true;
+		if (fail("sync server_up", server_up(1)))
+			return false;
+
+		if (!barrier_wait(b))
+			return true;
+		if (fail("sync flush", expkey_flush()))
+			return false;
+
+		if (!barrier_wait(b))
+			return true;
+		if (fail("sync server_down", server_down()))
+			return false;
+	}
+	return true;
+}
+
+static int worker(int idx, struct barrier *b, unsigned int secs)
+{
+	int ret = netns_enter();
+
+	if (ret) {
+		/*
+		 * Report setup trouble rather than a failure: a restricted
+		 * environment is not a kernel bug.
+		 */
+		fprintf(stderr, "netns %d: setup: %s\n", idx, strerror(-ret));
+		return WORKER_NO_SETUP;
+	}
+
+	/*
+	 * Leave only NFSv4.1 enabled. A fresh namespace has v3 on, which
+	 * makes nfsd_needs_lockd() true, and the lockd that comes up then
+	 * tries to reach an rpcbind that is not there -- so every server
+	 * start waits out an RPC timeout and floods the log. The version
+	 * set sticks across serv teardown, so once is enough.
+	 */
+	ret = version_set_only(4, 1);
+	if (ret) {
+		fprintf(stderr, "netns %d: version_set: %s\n", idx,
+			strerror(-ret));
+		return WORKER_FAIL;
+	}
+
+	srandom(getpid() ^ (unsigned int)time(NULL));
+
+	if (!worker_churn(time(NULL) + secs))
+		goto fault;
+	if (!worker_sync_rounds(b))
+		goto fault;
+	return WORKER_OK;
+
+fault:
+	fprintf(stderr, "netns %d: %s: %s\n", idx, worker_fault.op,
+		strerror(-worker_fault.err));
+	server_down();
+	return WORKER_FAIL;
+}
+
+/*
+ * Threads running outside the test's namespaces. Any at all pin the
+ * host-wide refcount above zero for the whole run, so the 0->1 transition
+ * the synchronized rounds are built around never happens.
+ */
+static int nfsd_threads_here(void)
+{
+	char buf[32] = "";
+	int fd = open("/proc/fs/nfsd/threads", O_RDONLY);
+	int n = 0;
+
+	if (fd < 0)
+		return 0;
+	if (read(fd, buf, sizeof(buf) - 1) > 0)
+		n = atoi(buf);
+	close(fd);
+	return n;
+}
+
+/* ------------------- taint ------------------- */
+
+static unsigned long read_taint(void)
+{
+	unsigned long v = 0;
+	FILE *f = fopen("/proc/sys/kernel/tainted", "r");
+
+	if (!f)
+		return 0;
+	if (fscanf(f, "%lu", &v) != 1)
+		v = 0;
+	fclose(f);
+	return v;
+}
+
+/* ------------------- the test ------------------- */
+
+static unsigned int env_uint(const char *name, unsigned int def, unsigned int max)
+{
+	const char *s = getenv(name);
+	unsigned long v;
+
+	if (!s || !*s)
+		return def;
+	v = strtoul(s, NULL, 0);
+	if (v == 0 || v > max)
+		return def;
+	return (unsigned int)v;
+}
+
+FIXTURE(soak) {
+	int unused;
+};
+
+FIXTURE_SETUP(soak) { }
+FIXTURE_TEARDOWN(soak) { }
+
+TEST_F_TIMEOUT(soak, netns_control_plane_soak, SOAK_TIMEOUT_SEC)
+{
+	unsigned long taint_before, taint_after;
+	unsigned int workers, secs;
+	pid_t pid[MAX_WORKERS];
+	int failed = 0, skipped = 0;
+	unsigned int aborted;
+	struct barrier *b;
+	unsigned int i;
+	long ncpu;
+
+	if (geteuid() != 0)
+		SKIP(return, "must be run as root");
+
+	ncpu = sysconf(_SC_NPROCESSORS_ONLN);
+	if (ncpu < 2)
+		ncpu = 2;
+	workers = env_uint("NFSD_STRESS_WORKERS",
+			   ncpu < DEFAULT_WORKERS ? (unsigned int)ncpu
+						  : DEFAULT_WORKERS,
+			   MAX_WORKERS);
+	secs = env_uint("NFSD_STRESS_SECS", DEFAULT_SECS, 3600);
+
+	b = mmap(NULL, sizeof(*b), PROT_READ | PROT_WRITE,
+		 MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	ASSERT_NE(MAP_FAILED, b);
+	b->n = workers;
+	b->count = 0;
+	b->generation = 0;
+	b->aborted = 0;
+
+	TH_LOG("%u namespaces, %us of churn then %d synchronized rounds",
+	       workers, secs, SYNC_ROUNDS);
+
+	/*
+	 * A server already running outside these namespaces holds the
+	 * host-wide refcount above zero for the whole run, so the 0->1
+	 * transition the synchronized rounds are built around never
+	 * happens. Worth saying out loud rather than reporting coverage
+	 * that was not there.
+	 */
+	if (nfsd_threads_here() > 0)
+		TH_LOG("note: nfsd runs outside these namespaces; refcount never hits 0");
+
+	taint_before = read_taint();
+
+	for (i = 0; i < workers; i++) {
+		pid[i] = fork();
+		ASSERT_GE(pid[i], 0);
+		if (pid[i] == 0)
+			_exit(worker(i, b, secs));
+	}
+
+	for (i = 0; i < workers; i++) {
+		int status = 0;
+
+		waitpid(pid[i], &status, 0);
+		if (!WIFEXITED(status)) {
+			failed++;
+			continue;
+		}
+		if (WEXITSTATUS(status) == WORKER_FAIL)
+			failed++;
+		else if (WEXITSTATUS(status) == WORKER_NO_SETUP)
+			skipped++;
+	}
+
+	taint_after = read_taint();
+	aborted = __atomic_load_n(&b->aborted, __ATOMIC_ACQUIRE);
+	munmap(b, sizeof(*b));
+
+	if (skipped == (int)workers)
+		SKIP(return, "no worker could set up a private nfsd namespace");
+
+	EXPECT_EQ(0, failed)
+		TH_LOG("%d of %u namespaces reported an error", failed, workers);
+
+	/*
+	 * A namespace that never reached the rounds -- one that could not
+	 * set up, say -- leaves the rest with nobody to meet. Say so: the
+	 * synchronized part did not run to completion.
+	 */
+	if (aborted)
+		TH_LOG("note: synchronized rounds cut short; a namespace did not arrive");
+
+	/*
+	 * The real result. Without CONFIG_PROVE_LOCKING there is very little
+	 * here for the kernel to complain about, so say so rather than
+	 * letting a quiet pass look like coverage.
+	 */
+	EXPECT_EQ(0, (taint_after & ~taint_before) & TAINT_WARN_BIT)
+		TH_LOG("kernel warned during the run (taint %#lx -> %#lx); check dmesg",
+		       taint_before, taint_after);
+
+	/*
+	 * lockdep_proc_init() puts these in procfs, not debugfs, and
+	 * lockdep_chains is the one that appears only with
+	 * CONFIG_PROVE_LOCKING -- the part that validates ordering rather
+	 * than merely tracking. Root-only, but so is this test.
+	 */
+	if (access("/proc/lockdep_chains", R_OK) != 0)
+		TH_LOG("note: CONFIG_PROVE_LOCKING looks absent; this proves little");
+}
+
+TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/nfsd/settings b/tools/testing/selftests/nfsd/settings
index 694d70710ff0..a62d2fa1275c 100644
--- a/tools/testing/selftests/nfsd/settings
+++ b/tools/testing/selftests/nfsd/settings
@@ -1 +1 @@
-timeout=300
+timeout=600

-- 
2.55.0

Re: [PATCH v2 0/8] nfsd: reduce the scope of the nfsd_mutex

[Chuck Lever]

On Tue, 22 Sep 2026 07:34:01 -0400, Jeff Layton wrote:
> Currently much of nfsd's administration is serialized under the global
> nfsd_mutex, even though most of the objects managed under it are
> segregated by net namespace.
> 
> This patchset does a couple of small cleanups and then adds a new
> per-net mutex and moves all of the appropriate per-net data structures
> to be protected by it instead of the global mutex. The filecache is
> given its own mutex, and the global mutex is renamed and left for
> managing just the few remaning global objects.
> 
> [...]

Applied to nfsd-testing, thanks!

[1/8] nfsd: clear NFSD_NET_UP before dropping the generic resource reference
      commit: 77881893d52fc4ac737bd55727eca54cb95d6002
[2/8] nfsd: make max_blksize a per-namespace setting
      commit: 4a29b69535eb2778255634b795ad9c026269a95f
[3/8] nfsd: move the control plane to a per-namespace mutex
      commit: 88815584b68611c3eeb6f536b8dc1f4a2274c129
[4/8] nfsd: rename nfsd_mutex to nfsd_global_mutex
      commit: 828e48566b8145969f93df212628349e4d2849d0
[5/8] nfsd: give the open file cache its own mutex
      commit: 4ad8170fd7287fbf7adc493c785a284fe213065f
[6/8] selftests/nfsd: factor the netlink plumbing into a shared header
      commit: e99f6a7157acd18cda500c2bcbb17761ecfede3f
[7/8] selftests/nfsd: add cross-namespace isolation tests
      commit: 20801469b68a678a06b51592a1f90cb018983cf9
[8/8] selftests/nfsd: add a cross-namespace control-plane soak
      commit: a55e35036b7e984e601264565c767a23582c2175

--
Chuck Lever