* Re: [PATCH] nfsd: shut down running servers in nfsd_net_exit()
2026-10-26 23:56 [PATCH] nfsd: shut down running servers in nfsd_net_exit() Shuangpeng Bai
@ 2026-10-27 10:48 ` Jeff Layton
2026-10-29 18:04 ` Shuangpeng
2026-10-29 18:10 ` [PATCH v2] nfsd: pin the module while a server exists Shuangpeng Bai
1 sibling, 1 reply; 4+ messages in thread
From: Jeff Layton @ 2026-10-27 10:48 UTC (permalink / raw)
To: Shuangpeng Bai, Chuck Lever
Cc: NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey, linux-nfs,
linux-kernel, stable
On Mon, 2026-10-26 at 19:56 -0400, Shuangpeng Bai wrote:
> Since commit 924f4fb003ba ("NFSD: convert write_threads to netlink
> command") the server can be started over netlink without mounting the
> nfsd filesystem. But that mount was the only thing pinning the module
> while threads run (commit f49169c97fce ("NFSD: Remove
> svc_serv_ops::svo_module") removed the per-thread reference), and
> nfsd_umount() is the only caller of nfsd_shutdown_threads().
>
> So when the server has been started via netlink, a plain "rmmod nfsd"
> succeeds while the threads are still running. exit_nfsd() then frees
> struct nfsd_net in unregister_pernet_subsys(), destroys the DRC slab
> and finally unmaps the module text underneath the running threads:
>
> BUG: KASAN: slab-use-after-free in nfsd_file_net_dispose (fs/nfsd/filecache.c:483) nfsd
> Read of size 8 at addr ffff888109fe92d0 by task nfsd/300
> Call Trace:
> nfsd_file_net_dispose (fs/nfsd/filecache.c:483) nfsd
> nfsd (fs/nfsd/nfssvc.c:957) nfsd
> kthread (kernel/kthread.c:436)
> ret_from_fork (arch/x86/kernel/process.c:158)
> Freed by task 311:
> kfree (mm/slub.c:6802)
> ops_undo_list (net/core/net_namespace.c:257)
> unregister_pernet_operations (net/core/net_namespace.c:1412)
> unregister_pernet_subsys (net/core/net_namespace.c:1459)
> cleanup_module nfsd
> __se_sys_delete_module (kernel/module/main.c:804)
>
> followed by a general protection fault in nfsd_cache_lookup() as soon
> as a thread allocates from the destroyed drc_slab.
>
> Shut down a server that is still running for a net namespace in
> nfsd_net_exit(), which is invoked for every net namespace when the
> module is unloaded. nfsd_shutdown_threads() is a no-op when no server
> exists, so nothing changes for the common case. As the laundromat
> work is queued on laundry_wq until the server is shut down, destroy
> that workqueue after unregister_pernet_subsys().
>
> To reproduce: start the server with nfsdctl (or any other netlink
> client) without mounting the nfsd filesystem, then run "rmmod nfsd".
>
> Fixes: 924f4fb003ba ("NFSD: convert write_threads to netlink command")
> Cc: stable@vger.kernel.org # v6.10+
> Signed-off-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
> ---
> Tested on top of 7.3-rc7 with KASAN and with PROVE_LOCKING: with the
> server started over netlink and an NFSv3 request in flight, "rmmod nfsd"
> now waits for the request to complete and unloads cleanly; starting and
> stopping the server via netlink and via the nfsd filesystem still works.
>
> An alternative would be to take a module reference while a server
> exists so that rmmod fails with EBUSY, as it does while the nfsd
> filesystem is mounted. Stopping the server in nfsd_net_exit() seemed
> simpler and mirrors what nfsd_umount() does, but I am happy to rework
> it if the other behaviour is preferred.
> fs/nfsd/nfsctl.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> index f32311f2d7cf..0a9679114b58 100644
> --- a/fs/nfsd/nfsctl.c
> +++ b/fs/nfsd/nfsctl.c
> @@ -3113,6 +3113,7 @@ static __net_exit void nfsd_net_exit(struct net *net)
> {
> struct nfsd_net *nn = net_generic(net, nfsd_net_id);
>
> + nfsd_shutdown_threads(net);
> kfree_sensitive(nn->fh_key);
> nfsd_net_cb_shutdown(nn);
> nfsd_proc_stat_shutdown(net);
> @@ -3200,9 +3201,9 @@ static void __exit exit_nfsd(void)
> remove_proc_entry("fs/nfs", NULL);
> genl_unregister_family(&nfsd_nl_family);
> unregister_filesystem(&nfsd_fs_type);
> - nfsd4_destroy_laundry_wq();
> unregister_cld_notifier();
> unregister_pernet_subsys(&nfsd_net_ops);
> + nfsd4_destroy_laundry_wq();
> nfsd_drc_slab_free();
> nfsd_lockd_shutdown();
> nfsd4_exit_pnfs();
I think I'm OK with either behavior (shutting down threads on rmmod or
taking a reference), but sashiko has some comments on this patch that
should be addressed:
https://sashiko.dev/#/patchset/20261026235626.649844-1-shuangpeng.kernel%40gmail.com
--
Jeff Layton <jlayton@kernel.org>
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v2] nfsd: pin the module while a server exists
2026-10-26 23:56 [PATCH] nfsd: shut down running servers in nfsd_net_exit() Shuangpeng Bai
2026-10-27 10:48 ` Jeff Layton
@ 2026-10-29 18:10 ` Shuangpeng Bai
1 sibling, 0 replies; 4+ messages in thread
From: Shuangpeng Bai @ 2026-10-29 18:10 UTC (permalink / raw)
To: Chuck Lever, Jeff Layton
Cc: NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey,
Lorenzo Bianconi, linux-nfs, linux-kernel, Shuangpeng Bai,
stable
Since commit 924f4fb003ba ("NFSD: convert write_threads to netlink
command") the server can be started over netlink without mounting the
nfsd filesystem. But that mount was the only thing pinning the module
while a server exists (commit f49169c97fce ("NFSD: Remove
svc_serv_ops::svo_module") removed the per-thread reference).
So when the server has been started via netlink, a plain "rmmod nfsd"
succeeds while the threads are still running. exit_nfsd() then frees
struct nfsd_net in unregister_pernet_subsys(), destroys the DRC slab
and finally unmaps the module text underneath the running threads:
BUG: KASAN: slab-use-after-free in nfsd_file_net_dispose (fs/nfsd/filecache.c:483) nfsd
Read of size 8 at addr ffff888109fe92d0 by task nfsd/300
Call Trace:
nfsd_file_net_dispose (fs/nfsd/filecache.c:483) nfsd
nfsd (fs/nfsd/nfssvc.c:957) nfsd
kthread (kernel/kthread.c:436)
ret_from_fork (arch/x86/kernel/process.c:158)
Freed by task 311:
kfree (mm/slub.c:6802)
ops_undo_list (net/core/net_namespace.c:257)
unregister_pernet_operations (net/core/net_namespace.c:1412)
unregister_pernet_subsys (net/core/net_namespace.c:1459)
cleanup_module nfsd
__se_sys_delete_module (kernel/module/main.c:804)
followed by a general protection fault in nfsd_cache_lookup() as soon
as a thread allocates from the destroyed drc_slab.
Take a module reference in nfsd_create_serv() and drop it in
nfsd_destroy_serv(), so that "rmmod nfsd" fails with EBUSY while a
server exists, just as it does while the nfsd filesystem is mounted.
Use try_module_get() rather than __module_get(): a netlink command
runs without a reference on the nfsd module, and exit_nfsd() only
waits for it in genl_unregister_family(). A THREADS_SET or
LISTENER_SET request that is already running when "rmmod nfsd" starts
would otherwise still create a server after module unload has begun.
With try_module_get() that request fails with ENODEV instead.
Pinning the module also keeps the assumption behind the
__module_get() calls in fs/locks.c (via lm_mod_owner) and in
nfs_common/nfslocalio.c valid: they rely on an active server keeping
the module alive.
To reproduce: start the server with nfsdctl (or any other netlink
client) without mounting the nfsd filesystem, then run "rmmod nfsd".
Fixes: 924f4fb003ba ("NFSD: convert write_threads to netlink command")
Cc: stable@vger.kernel.org # v6.10+
Signed-off-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
---
Changes since v1:
- Pin the module while a server exists instead of shutting the server
down in nfsd_net_exit(). As the sashiko review pointed out, shutting
it down from exit_nfsd() still lets every __module_get(THIS_MODULE)
taken during module exit (fs/locks.c via lm_mod_owner, nfslocalio)
outlive free_module(). With the reference held, exit_nfsd() never
runs while a server exists, so the laundry_wq / cld notifier ordering
changes from v1 are not needed either.
- Take the reference with try_module_get(). A netlink command holds no
reference on nfsd, so with __module_get() a LISTENER_SET/THREADS_SET
that was already running when "rmmod nfsd" started could still bring
up a server, and its threads then ran on the freed module text.
- Not changed: an exiting nfsd thread still executes a few instructions
of module code after svc_stop_kthreads() has been woken up. That
window exists on the umount path as well and is independent of this
fix.
v1: https://lore.kernel.org/linux-nfs/20261026235626.649844-1-shuangpeng.kernel@gmail.com/
Tested on top of 7.3-rc7 with KASAN: with the server started over
netlink, "rmmod nfsd" now fails with EBUSY, also while a request is in
flight and with only a listener configured; after setting the thread
count to 0 the module unloads cleanly; starting and stopping the server
through the nfsd filesystem is unchanged. With a delay injected at the
top of nfsd_create_serv() to hold a netlink start open across "rmmod
nfsd", the start now fails with ENODEV and the module unloads cleanly
(with __module_get() the threads came up and oopsed in freed module
text).
fs/nfsd/nfssvc.c | 20 +++++++++++++++++---
1 file changed, 17 insertions(+), 3 deletions(-)
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index 067543da3ca0..6ae040c50987 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
@@ -579,6 +579,7 @@ void nfsd_destroy_serv(struct net *net)
svc_xprt_destroy_all(serv, net, true);
nfsd_shutdown_net(net);
svc_destroy(&serv);
+ module_put(THIS_MODULE);
}
void nfsd_reset_versions(struct nfsd_net *nn)
@@ -662,10 +663,18 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
if (nn->nfsd_serv)
return 0;
+ /*
+ * Keep the module loaded while the server exists, as a mounted
+ * nfsd filesystem does. A netlink caller holds no module
+ * reference, so this fails if module unload has already begun.
+ */
+ if (!try_module_get(THIS_MODULE))
+ return -ENODEV;
+
error = percpu_ref_init(&nn->nfsd_net_ref, nfsd_net_free,
0, GFP_KERNEL);
if (error)
- return error;
+ goto out_put;
init_completion(&nn->nfsd_net_free_done);
init_completion(&nn->nfsd_net_confirm_done);
@@ -677,7 +686,8 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
nn->max_blksize, nfsd);
if (serv == NULL) {
percpu_ref_exit(&nn->nfsd_net_ref);
- return -ENOMEM;
+ error = -ENOMEM;
+ goto out_put;
}
serv->sv_tls = true;
@@ -688,7 +698,7 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
if (error < 0) {
svc_destroy(&serv);
percpu_ref_exit(&nn->nfsd_net_ref);
- return error;
+ goto out_put;
}
/*
* Register before publishing nn->nfsd_serv. Namespaces are only
@@ -704,6 +714,10 @@ int nfsd_create_serv(struct net *net, bool no_rpcbind)
nfsd_reset_write_verifier(nn);
return 0;
+
+out_put:
+ module_put(THIS_MODULE);
+ return error;
}
int nfsd_nrpools(struct net *net)
--
2.43.0
^ permalink raw reply [flat|nested] 4+ messages in thread