From: Chuck Lever <cel@kernel.org>
To: Trond Myklebust <trondmy@kernel.org>, Anna Schumaker <anna@kernel.org>
Cc: Jeff Layton <jlayton@kernel.org>, NeilBrown <neil@brown.name>,
Tim Menninger <tmenninger@purestorage.com>,
linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org,
Chuck Lever <cel@kernel.org>
Subject: [PATCH v3 0/4] Reduce lock contention in the NFS client
Date: Tue, 15 Sep 2026 10:20:40 -0400 [thread overview]
Message-ID: <20260915-performance-v3-0-ae26d460bfd3@kernel.org> (raw)
Under a 4KB NFSv3 workload on 100GbE RDMA, roughly 150 RPC worker
threads drive the client, and lock contention dominates its CPU
profile: up to 53% of non-idle cycles are spent in
native_queued_spin_lock_slowpath.
Three locks account for that: reserve_lock on every XID
allocation, queue_lock on every submit and completion, and the
unbound worker pool lock on every enqueue and dequeue for rpciod,
nfsiod, and xprtiod.
This series addresses the first two. v2 also moved the three
workqueues to the WQ_AFFN_SMT scope. Tim Menninger reported an
intermittent throughput regression on a 96-CPU two-socket
NFS/RDMA client and bisected it to the rpciod scope change [1].
The cause is still being worked out in that thread, so the scope
patches are withdrawn until it is understood. Until then, WQ_SYSFS
on all three workqueues lets an administrator set the scope from
user space.
[1] https://lore.kernel.org/linux-nfs/20260902204048.4100864-1-tmenninger@everpuredata.com/
---
Changes in v3:
- Drop the WQ_AFFN_SMT scope patches (Tim Menninger's regression report)
- Rebase on v7.3-rc2
- Link to v2: https://patch.msgid.link/20260902-performance-v2-0-b71c0c082f9d@kernel.org
Changes in v2:
- Fix send bvec use-after-free in xprt_request_dequeue_xprt() (sashiko)
- Replace the three workqueue exports with workqueue_set_affn_scope()
- Split the WQ_SYSFS patch into SUNRPC and NFS patches
- Drop v1 patch 2: async completions in the submitter can deadlock
- Correct the pool cost and SMT group wording in the scope patches
- Link to v1: https://patch.msgid.link/20260831-performance-v1-0-8d9fd9b67f96@kernel.org
---
Chuck Lever (4):
SUNRPC: Use atomic_t for XID allocation
SUNRPC: Split recv_lock out of xprt->queue_lock
SUNRPC: Set WQ_SYSFS on rpciod and xprtiod
NFS: Set WQ_SYSFS on nfsiod
fs/nfs/inode.c | 3 +-
include/linux/sunrpc/xprt.h | 8 ++-
net/sunrpc/sched.c | 5 +-
net/sunrpc/svcsock.c | 6 +--
net/sunrpc/xprt.c | 85 ++++++++++++++++++------------
net/sunrpc/xprtrdma/rpc_rdma.c | 14 ++---
net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 8 +--
net/sunrpc/xprtsock.c | 18 +++----
8 files changed, 85 insertions(+), 62 deletions(-)
---
base-commit: df2908090cda368b01ff43709f51890076c56157
change-id: 20260831-performance-e465e621c1c0
Best regards,
--
Chuck Lever
next reply other threads:[~2026-09-15 14:20 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-15 14:20 Chuck Lever [this message]
2026-09-15 14:20 ` [PATCH v3 1/4] SUNRPC: Use atomic_t for XID allocation Chuck Lever
2026-09-15 14:20 ` [PATCH v3 2/4] SUNRPC: Split recv_lock out of xprt->queue_lock Chuck Lever
2026-09-15 14:20 ` [PATCH v3 3/4] SUNRPC: Set WQ_SYSFS on rpciod and xprtiod Chuck Lever
2026-09-15 14:20 ` [PATCH v3 4/4] NFS: Set WQ_SYSFS on nfsiod Chuck Lever
2026-09-15 14:49 ` [PATCH v3 0/4] Reduce lock contention in the NFS client Jeff Layton
2026-09-15 18:29 ` Tim Menninger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260915-performance-v3-0-ae26d460bfd3@kernel.org \
--to=cel@kernel.org \
--cc=anna@kernel.org \
--cc=jlayton@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=neil@brown.name \
--cc=tmenninger@purestorage.com \
--cc=trondmy@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®