mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Chuck Lever <cel@kernel.org>
To: Trond Myklebust <trondmy@kernel.org>, Anna Schumaker <anna@kernel.org>
Cc: Jeff Layton <jlayton@kernel.org>, NeilBrown <neil@brown.name>,
	 Tim Menninger <tmenninger@purestorage.com>,
	linux-nfs@vger.kernel.org,  linux-kernel@vger.kernel.org,
	Chuck Lever <cel@kernel.org>
Subject: [PATCH v3 0/4] Reduce lock contention in the NFS client
Date: Tue, 15 Sep 2026 10:20:40 -0400	[thread overview]
Message-ID: <20260915-performance-v3-0-ae26d460bfd3@kernel.org> (raw)

Under a 4KB NFSv3 workload on 100GbE RDMA, roughly 150 RPC worker
threads drive the client, and lock contention dominates its CPU
profile: up to 53% of non-idle cycles are spent in
native_queued_spin_lock_slowpath.

Three locks account for that: reserve_lock on every XID
allocation, queue_lock on every submit and completion, and the
unbound worker pool lock on every enqueue and dequeue for rpciod,
nfsiod, and xprtiod.

This series addresses the first two. v2 also moved the three
workqueues to the WQ_AFFN_SMT scope. Tim Menninger reported an
intermittent throughput regression on a 96-CPU two-socket
NFS/RDMA client and bisected it to the rpciod scope change [1].
The cause is still being worked out in that thread, so the scope
patches are withdrawn until it is understood. Until then, WQ_SYSFS
on all three workqueues lets an administrator set the scope from
user space.

[1] https://lore.kernel.org/linux-nfs/20260902204048.4100864-1-tmenninger@everpuredata.com/

---
Changes in v3:
- Drop the WQ_AFFN_SMT scope patches (Tim Menninger's regression report)
- Rebase on v7.3-rc2
- Link to v2: https://patch.msgid.link/20260902-performance-v2-0-b71c0c082f9d@kernel.org

Changes in v2:
- Fix send bvec use-after-free in xprt_request_dequeue_xprt() (sashiko)
- Replace the three workqueue exports with workqueue_set_affn_scope()
- Split the WQ_SYSFS patch into SUNRPC and NFS patches
- Drop v1 patch 2: async completions in the submitter can deadlock
- Correct the pool cost and SMT group wording in the scope patches
- Link to v1: https://patch.msgid.link/20260831-performance-v1-0-8d9fd9b67f96@kernel.org

---
Chuck Lever (4):
      SUNRPC: Use atomic_t for XID allocation
      SUNRPC: Split recv_lock out of xprt->queue_lock
      SUNRPC: Set WQ_SYSFS on rpciod and xprtiod
      NFS: Set WQ_SYSFS on nfsiod

 fs/nfs/inode.c                             |  3 +-
 include/linux/sunrpc/xprt.h                |  8 ++-
 net/sunrpc/sched.c                         |  5 +-
 net/sunrpc/svcsock.c                       |  6 +--
 net/sunrpc/xprt.c                          | 85 ++++++++++++++++++------------
 net/sunrpc/xprtrdma/rpc_rdma.c             | 14 ++---
 net/sunrpc/xprtrdma/svc_rdma_backchannel.c |  8 +--
 net/sunrpc/xprtsock.c                      | 18 +++----
 8 files changed, 85 insertions(+), 62 deletions(-)
---
base-commit: df2908090cda368b01ff43709f51890076c56157
change-id: 20260831-performance-e465e621c1c0

Best regards,
--  
Chuck Lever


             reply	other threads:[~2026-09-15 14:20 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-15 14:20 Chuck Lever [this message]
2026-09-15 14:20 ` [PATCH v3 1/4] SUNRPC: Use atomic_t for XID allocation Chuck Lever
2026-09-15 14:20 ` [PATCH v3 2/4] SUNRPC: Split recv_lock out of xprt->queue_lock Chuck Lever
2026-09-15 14:20 ` [PATCH v3 3/4] SUNRPC: Set WQ_SYSFS on rpciod and xprtiod Chuck Lever
2026-09-15 14:20 ` [PATCH v3 4/4] NFS: Set WQ_SYSFS on nfsiod Chuck Lever
2026-09-15 14:49 ` [PATCH v3 0/4] Reduce lock contention in the NFS client Jeff Layton
2026-09-15 18:29 ` Tim Menninger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260915-performance-v3-0-ae26d460bfd3@kernel.org \
    --to=cel@kernel.org \
    --cc=anna@kernel.org \
    --cc=jlayton@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=neil@brown.name \
    --cc=tmenninger@purestorage.com \
    --cc=trondmy@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®