From: Kunwu Chan <kunwu.chan@gmail.com>
To: paulmck@kernel.org, corbet@lwn.net, mingo@redhat.com,
frederic@kernel.org, neeraj.upadhyay@kernel.org,
josh@joshtriplett.org, urezki@gmail.com, dave@stgolabs.net,
lianux.mm@gmail.com
Cc: stern@rowland.harvard.edu, parri.andrea@gmail.com,
will@kernel.org, peterz@infradead.org, boqun@kernel.org,
npiggin@gmail.com, dhowells@redhat.com, j.alglave@ucl.ac.uk,
luc.maranget@inria.fr, akiyks@gmail.com, dlustig@nvidia.com,
joelagnelf@nvidia.com, skhan@linuxfoundation.org,
rdunlap@infradead.org, longman@redhat.com, rostedt@goodmis.org,
mathieu.desnoyers@efficios.com, jiangshanlai@gmail.com,
qiang.zhang@linux.dev, kunwu.chan@gmail.com,
brads@mainlining.org, linux-kernel@vger.kernel.org,
linux-arch@vger.kernel.org, lkmm@lists.linux.dev,
linux-doc@vger.kernel.org, rcu@vger.kernel.org,
linux-kselftest@vger.kernel.org
Subject: [PATCH RFC v2 00/15] hazptr: batch synchronize operations through a shared scan
Date: Sat, 3 Oct 2026 01:08:32 +0800 [thread overview]
Message-ID: <20261002170847.3653663-1-kunwu.chan@gmail.com> (raw)
This series extends Paul McKenney's v3 hazptr implementation [1]
with a shared scan path for concurrent hazptr_synchronize() callers,
and adapts the lockdep dynamic-key hashlist use case from Boqun
Feng's 2025 shazptr series [2] to the current hazptr API.
The series also adds rcuscale support and torture coverage for
the hazptr implementation.
The lockdep conversion replaces the expedited RCU wait in
lockdep_unregister_key() with hazptr_synchronize(). This limits
the wait to hazard pointers protecting the target hash bucket
instead of waiting for a system-wide expedited RCU grace period.
[1] https://lore.kernel.org/all/20260919000056.3132131-26-paulmck@kernel.org/
[2] https://lore.kernel.org/lkml/20250625031101.12555-1-boqun.feng@gmail.com/
Performance data
================
All measurements are on an ARM64 KVM guest with
HAZPTR_SCALE_NR_OBJS=8 and round-robin updater selection; unless
noted otherwise, nreaders=0. Scale tests run with
PROVE_LOCKING=n.
rcuscale synchronize latency (96 CPUs, nw=1):
nreaders=0 nreaders=1 nreaders=4 nreaders=96
scale avg avg avg avg
--------------------------------------------------------------------
hazptr 46 us 47 us 102 us 106 ms
RCU 8.3 ms 8.5 ms 16.6 ms 54.4 ms
SRCU 8.2 ms 8.0 ms 11.7 ms 11.8 ms
Hazptr single-writer latency remains below 110 us with up to
four readers on this 96-CPU guest, but rises to 106 ms when
all 96 CPUs hold hazard pointers.
Under 16 concurrent synchronize callers (nw=16), per-writer
latency at four CPU counts:
hazptr nw=16 RCU nw=16 SRCU nw=16
CPUs avg avg avg
-----------------------------------------------------------
24 8.0 ms 13.6 ms 9.2 ms
96 8.0 ms 13.3 ms 8.6 ms
128 7.9 ms 13.0 ms 11.7 ms
256 8.0 ms 20.8 ms 9.6 ms
Hazptr remains around 8.0 ms across these CPU counts, with the
scan-kthread retry interval contributing to the latency.
RCU rises to ~21 ms at 256 CPUs while SRCU stays around 9-12 ms.
Reader-side overhead (refscale, 96 readers on the 96-CPU guest,
3 runs each):
hazptr 25.7 ns/op
RCU 96.7 ns/op
SRCU 134.6 ns/op
lockdep workload -- tc qdisc mq x100, ARM64 KVM, PROVE_LOCKING=y,
96 background hazptr readers, function-call IPIs per 100 ops:
CPUs hazptr exp RCU reduction
------------------------------------------------------------
24 196 207 5%
96 189 265 29%
128 193 266 27%
256 222 384 42%
Hazptr issues fewer function-call IPIs at each CPU count, with
the difference reaching 42% at 256 CPUs.
The hazptr.sh test suite passes, including the lockdep scenarios
and the 8-to-256 CPU sweep, with no lockdep warnings, deadlocks,
or crashes.
Additional x86 server testing with Lian Wang is planned(maybe after
LPC).
Changes since RFC/WIP
=====================
- RFC/WIP: https://lore.kernel.org/all/20260922070950.4173245-1-kunwu.chan@gmail.com/
- Split the original 4-patch RFC/WIP into smaller commits covering
shared scanning, correctness, API support, lockdep, scaling, and
torture testing.
- Incorporated Boqun Feng's review feedback: use a Bloom filter to
avoid per-waiter allocation, add scoped_guard() support, and add
a debug option to force the hazptr acquire slow path.
- Fixed scan ordering around backup-slot promotion by scanning all
per-CPU slots before the overflow lists, with a separate
overflow-list phase.
- Simplified the scan cycle to flip first and drain only the old
wildcard generation, with herd7-verified LKMM tests for both the
in-flight and resolved publication cases.
- Extended rcuscale and hazptrtorture coverage, added a selftest
script for the torture configurations, and fixed the
hazptr_release() kernel-doc.
Kunwu Chan (15):
hazptr: add shared scan kthread
hazptr: use Bloom filter for shared scan waiters
hazptr: scan all per-CPU slots before overflow lists
hazptr: add scoped_guard() support
hazptr: add debug option to force the acquire slow path
hazptr: elide redundant first drain pass
Documentation/litmus-tests: add hazptr wildcard-flip escape test
locking/lockdep: use hazptr to wait for dynamic key lookups
rcuscale: add hazptr scale type
hazptr: fix kernel-doc of hazptr_release()
Documentation/litmus-tests: add hazptr acquire-before-scan test
hazptrtorture: add slowpath and lockdep scenarios
hazptrtorture: add READERS4 and READERS0 torture configs
hazptrtorture: add 128- and 256-CPU configs
selftests/rcutorture: add hazptr torture test script
Documentation/litmus-tests/README | 13 +
.../hazptr/hazptr-acquire-before-scan.litmus | 45 +++
.../hazptr/hazptr-wildcard-flip-escape.litmus | 46 +++
include/linux/hazptr.h | 56 ++-
kernel/hazptr.c | 336 +++++++++++++++++-
kernel/locking/lockdep.c | 25 +-
kernel/rcu/Kconfig.debug | 10 +
kernel/rcu/hazptrtorture.c | 57 ++-
kernel/rcu/rcuscale.c | 70 +++-
.../selftests/rcutorture/bin/hazptr.sh | 146 ++++++++
.../rcutorture/configs/hazptr/CFLIST | 6 +
.../rcutorture/configs/hazptr/CPU128 | 16 +
.../rcutorture/configs/hazptr/CPU128.boot | 1 +
.../rcutorture/configs/hazptr/CPU256 | 16 +
.../rcutorture/configs/hazptr/CPU256.boot | 1 +
.../rcutorture/configs/hazptr/LOCKDEP | 17 +
.../rcutorture/configs/hazptr/LOCKDEP.boot | 1 +
.../rcutorture/configs/hazptr/READERS0 | 16 +
.../rcutorture/configs/hazptr/READERS0.boot | 2 +
.../rcutorture/configs/hazptr/READERS4 | 16 +
.../rcutorture/configs/hazptr/READERS4.boot | 2 +
.../rcutorture/configs/hazptr/SLOWPATH | 16 +
22 files changed, 881 insertions(+), 33 deletions(-)
create mode 100644 Documentation/litmus-tests/hazptr/hazptr-acquire-before-scan.litmus
create mode 100644 Documentation/litmus-tests/hazptr/hazptr-wildcard-flip-escape.litmus
create mode 100755 tools/testing/selftests/rcutorture/bin/hazptr.sh
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CPU128
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CPU128.boot
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CPU256
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/CPU256.boot
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/LOCKDEP
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/LOCKDEP.boot
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/READERS0
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/READERS0.boot
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/READERS4
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/READERS4.boot
create mode 100644 tools/testing/selftests/rcutorture/configs/hazptr/SLOWPATH
--
2.43.0
next reply other threads:[~2026-10-02 17:09 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-02 17:08 Kunwu Chan [this message]
2026-10-02 17:08 ` [PATCH RFC v2 01/15] hazptr: add shared scan kthread Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 02/15] hazptr: use Bloom filter for shared scan waiters Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 03/15] hazptr: scan all per-CPU slots before overflow lists Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 04/15] hazptr: add scoped_guard() support Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 05/15] hazptr: add debug option to force the acquire slow path Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 06/15] hazptr: elide redundant first drain pass Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 07/15] Documentation/litmus-tests: add hazptr wildcard-flip escape test Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 08/15] locking/lockdep: use hazptr to wait for dynamic key lookups Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 09/15] rcuscale: add hazptr scale type Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 10/15] hazptr: fix kernel-doc of hazptr_release() Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 11/15] Documentation/litmus-tests: add hazptr acquire-before-scan test Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 12/15] hazptrtorture: add slowpath and lockdep scenarios Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 13/15] hazptrtorture: add READERS4 and READERS0 torture configs Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 14/15] hazptrtorture: add 128- and 256-CPU configs Kunwu Chan
2026-10-02 17:08 ` [PATCH RFC v2 15/15] selftests/rcutorture: add hazptr torture test script Kunwu Chan
2026-10-02 17:12 ` [PATCH RFC v2 00/15] hazptr: batch synchronize operations through a shared scan Bradley Morgan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261002170847.3653663-1-kunwu.chan@gmail.com \
--to=kunwu.chan@gmail.com \
--cc=akiyks@gmail.com \
--cc=boqun@kernel.org \
--cc=brads@mainlining.org \
--cc=corbet@lwn.net \
--cc=dave@stgolabs.net \
--cc=dhowells@redhat.com \
--cc=dlustig@nvidia.com \
--cc=frederic@kernel.org \
--cc=j.alglave@ucl.ac.uk \
--cc=jiangshanlai@gmail.com \
--cc=joelagnelf@nvidia.com \
--cc=josh@joshtriplett.org \
--cc=lianux.mm@gmail.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=lkmm@lists.linux.dev \
--cc=longman@redhat.com \
--cc=luc.maranget@inria.fr \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@redhat.com \
--cc=neeraj.upadhyay@kernel.org \
--cc=npiggin@gmail.com \
--cc=parri.andrea@gmail.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=qiang.zhang@linux.dev \
--cc=rcu@vger.kernel.org \
--cc=rdunlap@infradead.org \
--cc=rostedt@goodmis.org \
--cc=skhan@linuxfoundation.org \
--cc=stern@rowland.harvard.edu \
--cc=urezki@gmail.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®