* [PATCH 0/2] nfsd: callback channel fixes
@ 2026-05-26 16:38 Jeff Layton
2026-05-26 16:38 ` [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper Jeff Layton
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Jeff Layton @ 2026-05-26 16:38 UTC (permalink / raw)
To: Chuck Lever, NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey
Cc: linux-nfs, linux-kernel, Jeff Layton
Some fixes for the callback channel machinery. These problems were
mostly found via agentic scans.
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Jeff Layton (2):
nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper
nfsd: clear CALLBACK_RUNNING on failed delegation recall queue
fs/nfsd/nfs4state.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
---
base-commit: 97bac3c7a039675d7ae71fbdf3a7c39e840339b6
change-id: 20260526-cb_recall_any_callback_running_stuck-7f3808033028
Best regards,
--
Jeff Layton <jlayton@kernel.org>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper
2026-05-26 16:38 [PATCH 0/2] nfsd: callback channel fixes Jeff Layton
@ 2026-05-26 16:38 ` Jeff Layton
2026-05-26 16:38 ` [PATCH 2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue Jeff Layton
2026-05-26 18:39 ` [PATCH 0/2] nfsd: callback channel fixes Chuck Lever
2 siblings, 0 replies; 4+ messages in thread
From: Jeff Layton @ 2026-05-26 16:38 UTC (permalink / raw)
To: Chuck Lever, NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey
Cc: linux-nfs, linux-kernel, Jeff Layton
deleg_reaper() sets NFSD4_CALLBACK_RUNNING before checking the
5-second rate limit and cl_cb_state gates. When either gate fires
the loop continues without queuing callback work, so the bit's only
clear site in nfsd41_destroy_cb() is never reached and RECALL_ANY
dispatch is permanently disabled for the affected client.
Move the test_and_set_bit() below both non-queueing gates so the
bit is taken only when nfsd4_run_cb() will be called.
Fixes: 424dd3df1f99 ("nfsd: eliminate cl_ra_cblist and NFSD4_CLIENT_CB_RECALL_ANY")
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
fs/nfsd/nfs4state.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index d5cbf626ab9b..57b99d1e74a6 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -7478,12 +7478,12 @@ deleg_reaper(struct nfsd_net *nn)
continue;
if (atomic_read(&clp->cl_delegs_in_recall))
continue;
- if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
- continue;
if (ktime_get_boottime_seconds() - clp->cl_ra_time < 5)
continue;
if (clp->cl_cb_state != NFSD4_CB_UP)
continue;
+ if (test_and_set_bit(NFSD4_CALLBACK_RUNNING, &clp->cl_ra->ra_cb.cb_flags))
+ continue;
/* release in nfsd4_cb_recall_any_release */
kref_get(&clp->cl_nfsdfs.cl_ref);
--
2.54.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue
2026-05-26 16:38 [PATCH 0/2] nfsd: callback channel fixes Jeff Layton
2026-05-26 16:38 ` [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper Jeff Layton
@ 2026-05-26 16:38 ` Jeff Layton
2026-05-26 18:39 ` [PATCH 0/2] nfsd: callback channel fixes Chuck Lever
2 siblings, 0 replies; 4+ messages in thread
From: Jeff Layton @ 2026-05-26 16:38 UTC (permalink / raw)
To: Chuck Lever, NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey
Cc: linux-nfs, linux-kernel, Jeff Layton
nfsd_break_one_deleg() sets NFSD4_CALLBACK_RUNNING via test_and_set_bit
at entry to serialize recall work, then calls nfsd4_run_cb() to queue
the recall. When the queue attempt fails the refcount bump is undone,
but the RUNNING bit is left set. The only site that clears the bit is
nfsd41_destroy_cb() (fs/nfsd/nfs4callback.c), which runs from the
workqueue and is therefore unreachable when nothing was queued.
The bit becomes a permanent latch on dp->dl_recall.cb_flags: every
subsequent break_lease() on the same delegation hits the early-return
guard in nfsd_break_one_deleg() and silently skips the recall, so the
delegation is never broken and the conflicting open or lock stalls.
Fix by clearing NFSD4_CALLBACK_RUNNING on the !queued branch alongside
the refcount_dec.
Fixes: 1054e8ffc5c4 ("nfsd: prevent callback tasks running concurrently")
Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
fs/nfsd/nfs4state.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 57b99d1e74a6..0b4d7afc42c6 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -3479,8 +3479,10 @@ static void nfsd_break_one_deleg(struct nfs4_delegation *dp)
refcount_inc(&dp->dl_stid.sc_count);
queued = nfsd4_run_cb(&dp->dl_recall);
WARN_ON_ONCE(!queued);
- if (!queued)
+ if (!queued) {
refcount_dec(&dp->dl_stid.sc_count);
+ clear_bit(NFSD4_CALLBACK_RUNNING, &dp->dl_recall.cb_flags);
+ }
}
static bool
--
2.54.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 0/2] nfsd: callback channel fixes
2026-05-26 16:38 [PATCH 0/2] nfsd: callback channel fixes Jeff Layton
2026-05-26 16:38 ` [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper Jeff Layton
2026-05-26 16:38 ` [PATCH 2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue Jeff Layton
@ 2026-05-26 18:39 ` Chuck Lever
2 siblings, 0 replies; 4+ messages in thread
From: Chuck Lever @ 2026-05-26 18:39 UTC (permalink / raw)
To: NeilBrown, Olga Kornievskaia, Dai Ngo, Tom Talpey, Jeff Layton
Cc: Chuck Lever, linux-nfs, linux-kernel
From: Chuck Lever <chuck.lever@oracle.com>
On Tue, 26 May 2026 12:38:44 -0400, Jeff Layton wrote:
> Some fixes for the callback channel machinery. These problems were
> mostly found via agentic scans.
Applied to nfsd-testing, thanks!
[1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper
commit: 02b9bb6ab68a77b1112d48411e4e08308a077ea0
[2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue
commit: e02bd0e5a983b4d617dd2222c81c7f057132c211
--
Chuck Lever <chuck.lever@oracle.com>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-05-26 18:39 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-05-26 16:38 [PATCH 0/2] nfsd: callback channel fixes Jeff Layton
2026-05-26 16:38 ` [PATCH 1/2] nfsd: defer setting NFSD4_CALLBACK_RUNNING in deleg_reaper Jeff Layton
2026-05-26 16:38 ` [PATCH 2/2] nfsd: clear CALLBACK_RUNNING on failed delegation recall queue Jeff Layton
2026-05-26 18:39 ` [PATCH 0/2] nfsd: callback channel fixes Chuck Lever
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®