* [PATCH 1/8] rcutorture: Enable RCU_BOOST where supported
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 2/8] rcutorture: Disable RCU priority boost testing on non-TREE03 scenarios Paul E. McKenney
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Paul E. McKenney, Andrew Morton
Currently, RCU_BOOST depends on RCU_EXPERT, and is therefore often
disabled in stress-test such as those run by syzbot. This means that if
syzbot runs a test that overloads CPUs on kernels built with PREEMPT_RCU,
RCU CPU stall warnings are not unexpected. What happens is that a
low-priority task in an RCU read-side critical section is preempted,
cannot be priority boosted, and stays that way for more than 21 seconds.
This commit therefore drops RCU_EXPERT from the RCU_BOOST "depends on"
line and removes the prompt, thus directly driving RCU_BOOST from the
kernel configuration so as to enable it anywhere it is supported and
might help.
This commit is also marked experimental because neither Ubuntu, Fedora,
nor CentOS enable RCU_BOOST. This suggests less testing of RCU priority
boosting than one might wish.
Reported-by: Andrew Morton <akpm@linux-foundation.org>
Closes: https://lore.kernel.org/all/20260805122952.6ae38af69a0457779cb44359@linux-foundation.org/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/rcu/Kconfig | 10 ++++------
tools/testing/selftests/rcutorture/configs/rcu/NOCB01 | 1 -
tools/testing/selftests/rcutorture/configs/rcu/NOCB02 | 1 -
tools/testing/selftests/rcutorture/configs/rcu/TREE01 | 1 -
tools/testing/selftests/rcutorture/configs/rcu/TREE02 | 1 -
tools/testing/selftests/rcutorture/configs/rcu/TREE08 | 1 -
.../testing/selftests/rcutorture/configs/rcuscale/TINY | 1 -
.../selftests/rcutorture/configs/rcuscale/TRACE01 | 1 -
.../testing/selftests/rcutorture/configs/rcuscale/TREE | 1 -
.../selftests/rcutorture/configs/rcuscale/TREE54 | 1 -
.../selftests/rcutorture/configs/refscale/NOPREEMPT | 1 -
.../selftests/rcutorture/configs/refscale/PREEMPT | 1 -
.../testing/selftests/rcutorture/configs/refscale/TINY | 1 -
13 files changed, 4 insertions(+), 18 deletions(-)
diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig
index bc9483c0b5d84..7b605fa1f576e 100644
--- a/kernel/rcu/Kconfig
+++ b/kernel/rcu/Kconfig
@@ -237,17 +237,15 @@ config RCU_FANOUT_LEAF
Take the default if unsure.
config RCU_BOOST
- bool "Enable RCU priority boosting"
- depends on (RT_MUTEXES && PREEMPT_RCU && RCU_EXPERT) || PREEMPT_RT
- default y if PREEMPT_RT
+ bool
+ default y if (RT_MUTEXES && PREEMPT_RCU) || PREEMPT_RT
help
This option boosts the priority of preempted RCU readers that
block the current preemptible RCU grace period for too long.
This option also prevents heavy loads from blocking RCU
- callback invocation.
+ callback invocation. It is now automatically enabled in
+ any kernel that can benefit from it and that can support it.
- Say Y here if you are working with real-time apps or heavy loads
- Say N here if you are unsure.
config RCU_BOOST_DELAY
int "Milliseconds to delay boosting after RCU grace-period start"
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/NOCB01 b/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
index bbe6d28210ab5..91ca7e8b5725f 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
@@ -14,7 +14,6 @@ CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=y
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/NOCB02 b/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
index 4c2b8cd6d8fdb..ed3f3f7b29e73 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
+++ b/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
@@ -14,7 +14,6 @@ CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=y
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
index 54b1600c7eb5f..be5bf69a1b60f 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
@@ -10,6 +10,5 @@ CONFIG_RCU_TRACE=y
CONFIG_HOTPLUG_CPU=y
CONFIG_RCU_NOCB_CPU=y
CONFIG_DEBUG_LOCK_ALLOC=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE02 b/tools/testing/selftests/rcutorture/configs/rcu/TREE02
index 2871ee599891d..36b9228ae7032 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE02
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE02
@@ -13,7 +13,6 @@ CONFIG_RCU_FANOUT_LEAF=3
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_RCU_EXPERT=y
CONFIG_DEBUG_OBJECTS=y
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE08 b/tools/testing/selftests/rcutorture/configs/rcu/TREE08
index 8b561355b9ef2..ad7d7f873e1eb 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE08
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE08
@@ -13,7 +13,6 @@ CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=y
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TINY b/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
index c1a7d38823fba..f611b69f11895 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
@@ -11,7 +11,6 @@ CONFIG_NO_HZ_FULL=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01 b/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
index f3f1368b83869..69f5764159df0 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
@@ -11,7 +11,6 @@ CONFIG_NO_HZ_FULL=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
index b10706fd03a45..5122cf08d5db2 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
@@ -12,7 +12,6 @@ CONFIG_HIBERNATION=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54 b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
index 9f83e53727969..08d30edb3b99c 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
@@ -15,7 +15,6 @@ CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT b/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
index 1f215e17cbae2..3e7d7793acc73 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
+++ b/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
@@ -14,7 +14,6 @@ CONFIG_HIBERNATION=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_KPROBES=n
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT b/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
index 52e3ef6740567..feed29cf043f6 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
+++ b/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
@@ -12,6 +12,5 @@ CONFIG_HIBERNATION=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/TINY b/tools/testing/selftests/rcutorture/configs/refscale/TINY
index 9a11b578ff344..b67ea7a9128b1 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/TINY
+++ b/tools/testing/selftests/rcutorture/configs/refscale/TINY
@@ -14,7 +14,6 @@ CONFIG_HIBERNATION=n
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
CONFIG_KPROBES=n
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 2/8] rcutorture: Disable RCU priority boost testing on non-TREE03 scenarios
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
2026-11-09 22:19 ` [PATCH 1/8] rcutorture: Enable RCU_BOOST where supported Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 3/8] Documentation/litmus-tests: Add SRCU fastpath anchor-before-scan test Paul E. McKenney
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Paul E. McKenney
The TREE03 scenario has been configured to permit testing of RCU
priority boosting, including setting RCU kthead priorities to 2.
But now that RCU priority boosting is default for preemptible kernels,
it is now automatically tested for TREE01, TREE02, TREE08, and TREE09,
none of which have RCU kthread priorities set up for such testing.
One approach would be to set up these scenarios' kthread priorities,
but this would fail to test common-case kernels having default priorities.
This commit therefore disables the testing of RCU priority boosting
except on scenario TREE03, which is set up properly for this purpose,
though rather unrealistically.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot | 1 +
tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot | 1 +
tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot | 1 +
tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot | 1 +
4 files changed, 4 insertions(+)
create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
index 1cc5b47dde282..73d51d869fd5e 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
@@ -6,3 +6,4 @@ rcu_nocbs=0-1,3-7
rcutorture.nocbs_nthreads=8
rcutorture.nocbs_toggle=1000
rcutorture.fwd_progress=0
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
index dd914fa8f690b..f2ca5d812f3f5 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
@@ -1 +1,2 @@
rcutorture.fwd_progress=2
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
index 94d38445d3937..41c3264f8e8cc 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
@@ -1,3 +1,4 @@
rcupdate.rcu_self_test=1
rcutree.rcu_fanout_exact=1
rcu_nocbs=all
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot
new file mode 100644
index 0000000000000..1f52e460e14a3
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot
@@ -0,0 +1 @@
+rcutorture.test_boost=0
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 3/8] Documentation/litmus-tests: Add SRCU fastpath anchor-before-scan test
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
2026-11-09 22:19 ` [PATCH 1/8] rcutorture: Enable RCU_BOOST where supported Paul E. McKenney
2026-11-09 22:19 ` [PATCH 2/8] rcutorture: Disable RCU priority boost testing on non-TREE03 scenarios Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 4/8] Documentation/litmus-tests: Add SRCU fastpath scan-before-anchor test Paul E. McKenney
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Kunwu Chan, Paul E . McKenney
From: Kunwu Chan <kunwu.chan@gmail.com>
synchronize_srcu_atomic() may end its grace period immediately when
its scan of the per-CPU lock counters finds no readers. Correctness
requires the grace-period anchor written by srcu_gp_start() to precede
the smp_mb() ordering the lock scan. This ordering ensures that any
reader whose lock increment is missed by the scan cannot have
incremented its lock counter before the grace-period anchor, and
therefore cannot be a pre-existing reader of this grace period.
This litmus test models the key ordering between the grace-period
anchor and the lock counter scan, where "seq" models the
grace-period anchor in ->srcu_gp_seq and "ctr" models the per-CPU
->srcu_ctrs[].srcu_locks counter. P0 writes the anchor before the
smp_mb() and the lock scan. P1 models the reader-side counter
increment. P2 models an observer that sees the reader's increment
before seeing the anchor.
The outcome is forbidden by LKMM, and herd7 reports "Never". See
SRCU-fastpath-scan-before-anchor.litmus for the reversed ordering,
which permits this outcome.
Tested with herd7 7.58 using linux-kernel.cfg.
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../SRCU-fastpath-anchor-before-scan.litmus | 55 +++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus
diff --git a/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus b/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus
new file mode 100644
index 0000000000000..8028f2ade733d
--- /dev/null
+++ b/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus
@@ -0,0 +1,55 @@
+C SRCU-fastpath-anchor-before-scan
+
+(*
+ * Result: Never
+ *
+ * The synchronize_srcu_atomic() fastpath may end its grace period
+ * immediately when its scan of the per-CPU lock counters finds no
+ * readers. Correctness requires the grace-period anchor written by
+ * srcu_gp_start() to precede the smp_mb() ordering the lock scan.
+ * This ordering ensures that any reader whose lock increment is missed
+ * by the scan cannot have incremented its lock counter before the
+ * grace-period anchor, and therefore cannot be a pre-existing reader
+ * of this grace period.
+ *
+ * This litmus test models the key ordering between the grace-period
+ * anchor and the lock counter scan, where "seq" models the
+ * grace-period anchor in ->srcu_gp_seq and "ctr" models the per-CPU
+ * ->srcu_ctrs[].srcu_locks counter. P0 writes the anchor before the
+ * smp_mb() and the lock scan. P1 models the reader-side counter
+ * increment. P2 models an observer that sees the reader's increment
+ * before seeing the anchor.
+ *
+ * The outcome is forbidden by LKMM, and herd7 reports "Never". See
+ * SRCU-fastpath-scan-before-anchor.litmus for the reversed ordering,
+ * which permits this outcome.
+ *)
+
+{}
+
+P0(int *seq, int *ctr)
+{
+ int r2;
+
+ WRITE_ONCE(*seq, 1);
+ smp_mb();
+ r2 = READ_ONCE(*ctr);
+}
+
+P1(int *ctr)
+{
+ WRITE_ONCE(*ctr, 1);
+}
+
+P2(int *seq, int *ctr)
+{
+ int r3;
+ int r4;
+
+ r3 = READ_ONCE(*ctr);
+ smp_mb();
+ r4 = READ_ONCE(*seq);
+}
+
+filter (0:r2 = 0)
+exists (2:r3 = 1 /\ 2:r4 = 0)
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 4/8] Documentation/litmus-tests: Add SRCU fastpath scan-before-anchor test
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
` (2 preceding siblings ...)
2026-11-09 22:19 ` [PATCH 3/8] Documentation/litmus-tests: Add SRCU fastpath anchor-before-scan test Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 5/8] rcuscale: Add srcua scale type Paul E. McKenney
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Kunwu Chan, Paul E . McKenney
From: Kunwu Chan <kunwu.chan@gmail.com>
If the synchronize_srcu_atomic() fastpath instead places its lock scan
before the grace-period anchor, the scan can miss a reader whose
increment was already visible before the anchor. That reader already
existed when the grace period started, so completing the grace period
without waiting for it would violate the SRCU grace-period guarantee.
This litmus test models the reversed ordering, with the lock scan placed
before the grace-period anchor. "seq" models the grace-period anchor in
->srcu_gp_seq and "ctr" models the per-CPU ->srcu_ctrs[].srcu_locks
counter. P0 scans the lock counter before writing the anchor, with an
smp_mb() between them. P1 models the reader-side counter increment.
P2 models an observer that sees the reader's increment before seeing
the anchor.
The same outcome is allowed with this ordering, and herd7 reports
"Sometimes". The litmus-tests README is also updated to describe both
SRCU fastpath tests.
Tested with herd7 7.58 using linux-kernel.cfg.
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
Documentation/litmus-tests/README | 19 +++++++
.../SRCU-fastpath-scan-before-anchor.litmus | 52 +++++++++++++++++++
2 files changed, 71 insertions(+)
create mode 100644 Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus
diff --git a/Documentation/litmus-tests/README b/Documentation/litmus-tests/README
index 6c666f3422ea3..4d4ec9c6f2cc7 100644
--- a/Documentation/litmus-tests/README
+++ b/Documentation/litmus-tests/README
@@ -78,3 +78,22 @@ RCU+sync+read.litmus
RCU+sync+free.litmus
Both the above litmus tests demonstrate the RCU grace period guarantee
that an RCU read-side critical section can never span a grace period.
+
+SRCU (/srcu directory)
+----------------------
+
+SRCU-fastpath-anchor-before-scan.litmus
+ This models the synchronize_srcu_atomic() fastpath with the
+ grace-period anchor ordered before the lock-counter scan. This
+ ordering prevents readers that existed before the grace period
+ from being missed by the scan. See
+ SRCU-fastpath-scan-before-anchor.litmus for the reversed
+ ordering.
+
+SRCU-fastpath-scan-before-anchor.litmus
+ This models the synchronize_srcu_atomic() fastpath with the
+ lock-counter scan ordered before the grace-period anchor. This
+ permits the scan to miss readers that existed before the grace
+ period, violating the SRCU grace-period guarantee. See
+ SRCU-fastpath-anchor-before-scan.litmus for the opposite
+ ordering.
diff --git a/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus b/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus
new file mode 100644
index 0000000000000..7df0641b5470f
--- /dev/null
+++ b/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus
@@ -0,0 +1,52 @@
+C SRCU-fastpath-scan-before-anchor
+
+(*
+ * Result: Sometimes
+ *
+ * If the synchronize_srcu_atomic() fastpath instead places its lock
+ * scan before the grace-period anchor, the scan can miss a reader whose
+ * increment was already visible before the anchor. That reader already
+ * existed when the grace period started, so completing the grace period
+ * without waiting for it would violate the SRCU grace-period guarantee.
+ *
+ * This litmus test models the reversed ordering, with the lock scan
+ * placed before the grace-period anchor. "seq" models the grace-period
+ * anchor in ->srcu_gp_seq and "ctr" models the per-CPU
+ * ->srcu_ctrs[].srcu_locks counter. P0 scans the lock counter before
+ * writing the anchor, with an smp_mb() between them. P1 models the
+ * reader-side counter increment. P2 models an observer that sees the
+ * reader's increment before seeing the anchor.
+ *
+ * The same outcome is allowed with this ordering, and herd7 reports
+ * "Sometimes". See SRCU-fastpath-anchor-before-scan.litmus for the
+ * opposite ordering, which forbids this outcome.
+ *)
+
+{}
+
+P0(int *seq, int *ctr)
+{
+ int r2;
+
+ r2 = READ_ONCE(*ctr);
+ smp_mb();
+ WRITE_ONCE(*seq, 1);
+}
+
+P1(int *ctr)
+{
+ WRITE_ONCE(*ctr, 1);
+}
+
+P2(int *seq, int *ctr)
+{
+ int r3;
+ int r4;
+
+ r3 = READ_ONCE(*ctr);
+ smp_mb();
+ r4 = READ_ONCE(*seq);
+}
+
+filter (0:r2 = 0)
+exists (2:r3 = 1 /\ 2:r4 = 0)
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 5/8] rcuscale: Add srcua scale type
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
` (3 preceding siblings ...)
2026-11-09 22:19 ` [PATCH 4/8] Documentation/litmus-tests: Add SRCU fastpath scan-before-anchor test Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 6/8] rcuscale: Consolidate per-writer arrays into struct writer_state Paul E. McKenney
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Kunwu Chan, Paul E . McKenney
From: Kunwu Chan <kunwu.chan@gmail.com>
Add support for scaling atomic SRCU grace periods using
srcu_read_lock_atomic()/srcu_read_unlock_atomic() and
synchronize_srcu_atomic().
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/rcu/rcuscale.c | 41 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 40 insertions(+), 1 deletion(-)
diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
index 1097ec15879cb..b172882824aca 100644
--- a/kernel/rcu/rcuscale.c
+++ b/kernel/rcu/rcuscale.c
@@ -293,6 +293,44 @@ static struct rcu_scale_ops srcud_ops = {
.name = "srcud"
};
+static struct srcu_struct srcua;
+
+static int srcu_scale_atomic_read_lock(void)
+{
+ return srcu_read_lock_atomic(srcu_ctlp);
+}
+
+static void srcu_scale_atomic_read_unlock(int idx)
+{
+ srcu_read_unlock_atomic(srcu_ctlp, idx);
+}
+
+static void srcu_scale_atomic_synchronize(void)
+{
+ synchronize_srcu_atomic(srcu_ctlp);
+}
+
+static void srcu_atomic_scale_init(void)
+{
+ srcu_ctlp = &srcua;
+ init_srcu_struct_atomic(srcu_ctlp);
+}
+
+static struct rcu_scale_ops srcua_ops = {
+ .ptype = SRCU_FLAVOR,
+ .init = srcu_atomic_scale_init,
+ .cleanup = srcu_sync_scale_cleanup,
+ .readlock = srcu_scale_atomic_read_lock,
+ .readunlock = srcu_scale_atomic_read_unlock,
+ .get_gp_seq = srcu_scale_completed,
+ .gp_diff = rcu_seq_diff,
+ .exp_completed = srcu_scale_completed,
+ .sync = srcu_scale_atomic_synchronize,
+ .exp_sync = srcu_scale_atomic_synchronize,
+ .stats = srcu_scale_stats,
+ .name = "srcua"
+};
+
#ifdef CONFIG_TASKS_RCU
/*
@@ -1110,7 +1148,8 @@ rcu_scale_init(void)
long i;
long j;
static struct rcu_scale_ops *scale_ops[] = {
- &rcu_ops, &srcu_ops, &srcud_ops, TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
+ &rcu_ops, &srcu_ops, &srcua_ops, &srcud_ops,
+ TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
};
if (!torture_init_begin(scale_type, verbose))
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 6/8] rcuscale: Consolidate per-writer arrays into struct writer_state
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
` (4 preceding siblings ...)
2026-11-09 22:19 ` [PATCH 5/8] rcuscale: Add srcua scale type Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 7/8] rcu: Print CPU that preempted stalled reader wants to run on Paul E. McKenney
2026-11-09 22:19 ` [PATCH 8/8] doc: Document additional RCU task-stall dump information Paul E. McKenney
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, rostedt, Joel Fernandes,
Bradley Morgan, Davidlohr Bueso, Paul E . McKenney
From: Joel Fernandes <joelagnelf@nvidia.com>
The writer kthreads keep their state in five parallel arrays indexed by
writer number: writer_tasks, writer_durations, writer_n_durations,
writer_done and writer_freelists. Each must be allocated, NULL-checked,
freed and cleared separately, and the cleanup path needs several
guards to cope with any one of them being missing.
Collapse them into a single array of struct writer_state, holding the
task pointer, the duration buffer, the measurement count, the done flag
and the existing struct writer_freelist.
This reduces initialization to one allocation and one out-of-memory
check, and cleanup to a single kfree(). As a side effect, this also
fixes a leak: cleanup only ran its frees under "if (writer_tasks)", so
if the writer_tasks allocation failed while a later allocation (for
example writer_durations or writer_n_durations) still succeeded, those
already-allocated arrays were never freed. With a single allocation,
that partial-allocation state can no longer occur.
Assisted-by: Codex
Fixes: 8704baab9bc8 ("rcutorture: Add RCU grace-period performance tests")
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Tested-by: Bradley Morgan <brads@mainlining.org> # Power10
Acked-by: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/rcu/rcuscale.c | 129 ++++++++++++++++++------------------------
1 file changed, 56 insertions(+), 73 deletions(-)
diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
index b172882824aca..5659eaebc5c0d 100644
--- a/kernel/rcu/rcuscale.c
+++ b/kernel/rcu/rcuscale.c
@@ -115,17 +115,21 @@ struct writer_freelist {
struct writer_mblock *ws_mblocks;
};
+struct writer_state {
+ struct writer_freelist wst_fl;
+ struct task_struct *wst_task;
+ u64 *wst_durations;
+ int wst_n_durations;
+ bool wst_done;
+};
+
static int nrealreaders;
static int nrealwriters;
static int nrealexp;
-static struct task_struct **writer_tasks;
static struct task_struct **reader_tasks;
static struct task_struct **exp_tasks;
-static u64 **writer_durations;
-static bool *writer_done;
-static struct writer_freelist *writer_freelists;
-static int *writer_n_durations;
+static struct writer_state *writer_states;
static atomic_t n_rcu_scale_reader_started;
static atomic_t n_rcu_scale_writer_started;
static atomic_t n_rcu_scale_writer_finished;
@@ -542,9 +546,9 @@ static struct writer_mblock *rcu_scale_alloc(long me)
struct writer_freelist *wflp;
struct writer_mblock *wmbp;
- if (WARN_ON_ONCE(!writer_freelists))
+ if (WARN_ON_ONCE(!writer_states))
return NULL;
- wflp = &writer_freelists[me];
+ wflp = &writer_states[me].wst_fl;
if (llist_empty(&wflp->ws_lhp)) {
// ->ws_lhp is private to its rcu_scale_writer task.
wmbp = container_of(llist_del_all(&wflp->ws_lhg), struct writer_mblock, wmb_node);
@@ -598,8 +602,9 @@ rcu_scale_writer(void *arg)
u64 t;
DEFINE_TORTURE_RANDOM(tr);
u64 *wdp;
- u64 *wdpp = writer_durations[me];
- struct writer_freelist *wflp = &writer_freelists[me];
+ struct writer_state *wsp = &writer_states[me];
+ u64 *wdpp = wsp->wst_durations;
+ struct writer_freelist *wflp = &wsp->wst_fl;
struct writer_mblock *wmbp = NULL;
VERBOSE_SCALEOUT_STRING("rcu_scale_writer task started");
@@ -669,7 +674,7 @@ rcu_scale_writer(void *arg)
started = true;
if (!done && i >= MIN_MEAS && time_after(jiffies, jdone)) {
done = true;
- WRITE_ONCE(writer_done[me], true);
+ WRITE_ONCE(wsp->wst_done, true);
sched_set_normal(current, 0);
pr_alert("%s%s rcu_scale_writer %ld has %d measurements\n",
scale_type, SCALE_FLAG, me, MIN_MEAS);
@@ -687,7 +692,7 @@ rcu_scale_writer(void *arg)
cur_ops->get_gp_seq();
}
if (shutdown_secs) {
- writer_tasks[me] = NULL;
+ wsp->wst_task = NULL;
smp_mb(); /* Assign before wake. */
rcu_scale_cleanup();
kernel_power_off();
@@ -703,10 +708,10 @@ rcu_scale_writer(void *arg)
if (!atomic_xchg(&dumped, 1)) {
for (i = 0; i < nrealwriters; i++) {
- if (writer_done[i])
+ if (writer_states[i].wst_done)
continue;
pr_info("%s: Task %ld flags writer %d:\n", __func__, me, i);
- sched_show_task(writer_tasks[i]);
+ sched_show_task(writer_states[i].wst_task);
}
if (cur_ops->stats)
cur_ops->stats();
@@ -714,7 +719,7 @@ rcu_scale_writer(void *arg)
}
if (!selfreport && time_after(jiffies, jdone + HZ * (70 + me))) {
pr_info("%s: Writer %ld self-report: started %d done %d/%d->%d i %d jdone %lu.\n",
- __func__, me, started, done, writer_done[me], atomic_read(&n_rcu_scale_writer_finished), i, jiffies - jdone);
+ __func__, me, started, done, wsp->wst_done, atomic_read(&n_rcu_scale_writer_finished), i, jiffies - jdone);
selfreport = true;
}
if (gp_succeeded && started && !alldone && i < MAX_MEAS - 1)
@@ -725,7 +730,7 @@ rcu_scale_writer(void *arg)
rcu_scale_free(wmbp);
cur_ops->gp_barrier();
}
- writer_n_durations[me] = i_max + 1;
+ wsp->wst_n_durations = i_max + 1;
torture_kthread_stopping("rcu_scale_writer");
return 0;
}
@@ -1069,13 +1074,11 @@ rcu_scale_cleanup(void)
reader_tasks = NULL;
}
- if (writer_tasks) {
+ if (writer_states) {
for (i = 0; i < nrealwriters; i++) {
torture_stop_kthread(rcu_scale_writer,
- writer_tasks[i]);
- if (!writer_n_durations)
- continue;
- j = writer_n_durations[i];
+ writer_states[i].wst_task);
+ j = writer_states[i].wst_n_durations;
pr_alert("%s%s writer %d gps: %d\n",
scale_type, SCALE_FLAG, i, j);
ngps += j;
@@ -1089,14 +1092,12 @@ rcu_scale_cleanup(void)
rcuscale_seq_diff(b_rcu_gp_test_finished,
b_rcu_gp_test_started));
for (i = 0; i < nrealwriters; i++) {
- if (!writer_durations)
- break;
- if (!writer_n_durations)
- continue;
- wdpp = writer_durations[i];
+ struct writer_state *wsp = &writer_states[i];
+
+ wdpp = wsp->wst_durations;
if (!wdpp)
continue;
- for (j = 0; j < writer_n_durations[i]; j++) {
+ for (j = 0; j < wsp->wst_n_durations; j++) {
wdp = &wdpp[j];
pr_alert("%s%s %4d writer-duration: %5d %llu\n",
scale_type, SCALE_FLAG,
@@ -1104,34 +1105,23 @@ rcu_scale_cleanup(void)
if (j % 100 == 0)
schedule_timeout_uninterruptible(1);
}
- kfree(writer_durations[i]);
- if (writer_freelists) {
+ kfree(wdpp);
+ if (wsp->wst_fl.ws_mblocks) {
int ctr = 0;
struct llist_node *llnp;
- struct writer_freelist *wflp = &writer_freelists[i];
-
- if (wflp->ws_mblocks) {
- llist_for_each(llnp, wflp->ws_lhg.first)
- ctr++;
- llist_for_each(llnp, wflp->ws_lhp.first)
- ctr++;
- WARN_ONCE(ctr != gp_async_max,
- "%s: ctr = %d gp_async_max = %d\n",
- __func__, ctr, gp_async_max);
- kfree(wflp->ws_mblocks);
- }
+
+ llist_for_each(llnp, wsp->wst_fl.ws_lhg.first)
+ ctr++;
+ llist_for_each(llnp, wsp->wst_fl.ws_lhp.first)
+ ctr++;
+ WARN_ONCE(ctr != gp_async_max,
+ "%s: ctr = %d gp_async_max = %d\n",
+ __func__, ctr, gp_async_max);
+ kfree(wsp->wst_fl.ws_mblocks);
}
}
- kfree(writer_tasks);
- writer_tasks = NULL;
- kfree(writer_durations);
- writer_durations = NULL;
- kfree(writer_n_durations);
- writer_n_durations = NULL;
- kfree(writer_done);
- writer_done = NULL;
- kfree(writer_freelists);
- writer_freelists = NULL;
+ kfree(writer_states);
+ writer_states = NULL;
}
/* Do torture-type-specific cleanup operations. */
@@ -1226,37 +1216,30 @@ rcu_scale_init(void)
goto unwind;
}
}
- writer_tasks = kzalloc_objs(writer_tasks[0], nrealwriters);
- writer_durations = kcalloc(nrealwriters, sizeof(*writer_durations), GFP_KERNEL);
- writer_n_durations = kzalloc_objs(*writer_n_durations, nrealwriters);
- writer_done = kzalloc_objs(writer_done[0], nrealwriters);
- if (gp_async) {
- if (gp_async_max <= 0) {
- pr_warn("%s: gp_async_max = %d must be greater than zero.\n",
- __func__, gp_async_max);
- WARN_ON_ONCE(IS_BUILTIN(CONFIG_RCU_TORTURE_TEST));
- firsterr = -EINVAL;
- goto unwind;
- }
- writer_freelists = kzalloc_objs(writer_freelists[0],
- nrealwriters);
+ writer_states = kzalloc_objs(writer_states[0], nrealwriters);
+ if (gp_async && gp_async_max <= 0) {
+ pr_warn("%s: gp_async_max = %d must be greater than zero.\n",
+ __func__, gp_async_max);
+ WARN_ON_ONCE(IS_BUILTIN(CONFIG_RCU_TORTURE_TEST));
+ firsterr = -EINVAL;
+ goto unwind;
}
- if (!writer_tasks || !writer_durations || !writer_n_durations || !writer_done ||
- (gp_async && !writer_freelists)) {
+ if (!writer_states) {
SCALEOUT_ERRSTRING("out of memory");
firsterr = -ENOMEM;
goto unwind;
}
for (i = 0; i < nrealwriters; i++) {
- writer_durations[i] =
- kcalloc(MAX_MEAS, sizeof(*writer_durations[i]),
- GFP_KERNEL);
- if (!writer_durations[i]) {
+ struct writer_state *wsp = &writer_states[i];
+
+ wsp->wst_durations = kcalloc(MAX_MEAS, sizeof(*wsp->wst_durations),
+ GFP_KERNEL);
+ if (!wsp->wst_durations) {
firsterr = -ENOMEM;
goto unwind;
}
- if (writer_freelists) {
- struct writer_freelist *wflp = &writer_freelists[i];
+ if (gp_async) {
+ struct writer_freelist *wflp = &wsp->wst_fl;
init_llist_head(&wflp->ws_lhg);
init_llist_head(&wflp->ws_lhp);
@@ -1274,7 +1257,7 @@ rcu_scale_init(void)
}
}
firsterr = torture_create_kthread(rcu_scale_writer, (void *)i,
- writer_tasks[i]);
+ wsp->wst_task);
if (torture_init_error(firsterr))
goto unwind;
}
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 7/8] rcu: Print CPU that preempted stalled reader wants to run on
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
` (5 preceding siblings ...)
2026-11-09 22:19 ` [PATCH 6/8] rcuscale: Consolidate per-writer arrays into struct writer_state Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
2026-11-09 22:19 ` [PATCH 8/8] doc: Document additional RCU task-stall dump information Paul E. McKenney
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Paul E. McKenney, K Prateek Nayak
This commit defines a cpu_curr_snapshot_rcu() function that invokes
rcu_dereference_all(cpu_curr()) to determine which task is currently
running on the specified CPU. The RCU CPU stall warning then uses this
new function in order to print the task that has preempted the stalled
task (the stalled task being the one that is in an RCU read-side critical
section, thus preventing the current RCU grace period from completing).
This modified stall-warning message also includes the CPU, along with
an exclamation mark ("!") after the CPU number if that CPU is currently
offline.
This is a potential step towards allowing people (or LLMs, for that
matter) locate any tasks that might be preventing the preempted RCU
reader task from running.
Adding a get_task_struct() for the currently running task would be a
bug because we are not in a position to prevent that task from exiting.
Yes, yes, because interrupts are disabled throughout, the corresponding
task_struct structure cannot be freed, but following sequence of events
is all too possible:
o The check_slow_task() function places a pointer to the task_struct
structure in its local variable "th".
o The corresponding task exits, reducing its task_struct structure's
->usage to zero. The task_struct structure is therefore passed
to call_rcu().
o The check_slow_task() function invokes get_task_struct(th),
setting th->usage to one.
o The rcu_print_task_stall() function invokes put_task_struct(),
again reducing ->usage to zero, and again passing the task
struct to call_rcu(). This results in either double-call_rcu()
or use-after-free, depending on when the RCU grace period for
the first call_rcu() completes. Both outcomes are bad.
This commit therefore relies solely on disabled interrupts acting as
an implicit RCU read-side critical section that prevents all of the
task_struct structures from being freed, thus avoiding the need for
get_task_struct().
The original get_task_struct() calls in rcu_print_task_stall() on the
tasks on the ->blkd_tasks list are safe because those tasks cannot exit
without first acquiring the rcu_node ->lock that this code already holds
(because these tasks must remove themselves from the ->blkd_tasks list).
But it is better to be consistent and rely on RCU in both cases.
These pre-existing get_task_struct() and put_task_struct() calls are
therefore removed.
[ paulmck: Apply feedback from K Prateek Nayak and Peter Zijlstra. ]
[ paulmck: Apply feedback from Steven Rostedt. ]
Link: https://lore.kernel.org/all/a7a21bf3-a7d5-45bb-be4f-0119fe53fa20@paulmck-laptop/
Link: https://lore.kernel.org/all/e8b6b372-d16b-4939-aa7a-755b7149e0fb@paulmck-laptop/
Link: https://lore.kernel.org/all/22d0d56c-6cef-47d7-a923-1d888e224b58@paulmck-laptop/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
include/linux/sched.h | 2 ++
kernel/rcu/tree_stall.h | 33 +++++++++++++++++++++++++++------
kernel/sched/core.c | 13 +++++++++++++
3 files changed, 42 insertions(+), 6 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 1c93da75226e3..ba7c7bb13b6e9 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2527,3 +2527,5 @@ extern void migrate_enable(void);
DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
#endif
+
+struct task_struct *cpu_curr_snapshot_rcu(int cpu);
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index 803a56af3259f..08699652f51c2 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -309,6 +309,9 @@ struct rcu_stall_chk_rdr {
bool on_blkd_list;
bool rcu_rdr_running;
int rcu_rdr_boosted;
+ int rcu_rdr_cpu;
+ int rcu_rdr_cpu_offline;
+ struct task_struct *rcu_rdr_cpu_task;
};
/*
@@ -317,8 +320,10 @@ struct rcu_stall_chk_rdr {
*/
static int check_slow_task(struct task_struct *t, void *arg)
{
+ int cpu;
struct rcu_node *rnp;
struct rcu_stall_chk_rdr *rscrp = arg;
+ struct task_struct *th;
if (task_curr(t))
return -EBUSY; // It is running, so decline to inspect it.
@@ -327,6 +332,13 @@ static int check_slow_task(struct task_struct *t, void *arg)
rscrp->on_blkd_list = !list_empty(&t->rcu_node_entry);
rscrp->rcu_rdr_running = task_curr(t);
rscrp->rcu_rdr_boosted = 0;
+ rscrp->rcu_rdr_cpu = cpu = task_cpu(t);
+ rscrp->rcu_rdr_cpu_offline = cpu_is_offline(cpu);
+ lockdep_assert_irqs_disabled();
+ rscrp->rcu_rdr_cpu_task = th = cpu_curr_snapshot_rcu(cpu);
+ // There is no need for get_task_struct(t) because we are in
+ // an implicit RCU read-side critical section due to interrupts
+ // being disabled.
if (rscrp->on_blkd_list) {
rnp = READ_ONCE(t->rcu_blocked_node);
raw_spin_lock_rcu_node(rnp); /* irqs already disabled. */
@@ -352,6 +364,7 @@ static int rcu_print_task_stall(struct rcu_node *rnp, unsigned long flags)
int ndetected = 0;
struct rcu_stall_chk_rdr rscr;
struct task_struct *t;
+ struct task_struct *tc;
struct task_struct *ts[8];
lockdep_assert_irqs_disabled();
@@ -364,7 +377,9 @@ static int rcu_print_task_stall(struct rcu_node *rnp, unsigned long flags)
t = list_entry(rnp->gp_tasks->prev,
struct task_struct, rcu_node_entry);
list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) {
- get_task_struct(t);
+ // There is no need for get_task_struct(t) because we
+ // are in an implicit RCU read-side critical section due
+ // to interrupts being disabled.
ts[i++] = t;
if (i >= ARRAY_SIZE(ts))
break;
@@ -372,19 +387,25 @@ static int rcu_print_task_stall(struct rcu_node *rnp, unsigned long flags)
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
while (i) {
t = ts[--i];
- if (task_call_func(t, check_slow_task, &rscr))
+ if (task_call_func(t, check_slow_task, &rscr)) {
pr_cont(" P%d", t->pid);
- else
- pr_cont(" P%d/%d:%c%c%c%c%c%c",
+ } else {
+ tc = rscr.rcu_rdr_cpu_task;
+ pr_cont(" P%d/%d:%c%c%c%c%c%c CPU %d%s %s (%d)%s",
t->pid, rscr.nesting,
".b"[rscr.rs.b.blocked],
".q"[rscr.rs.b.need_qs],
".e"[rscr.rs.b.exp_hint],
".l"[rscr.on_blkd_list],
".R"[rscr.rcu_rdr_running],
- ".B?"[rscr.rcu_rdr_boosted]);
+ ".B?"[rscr.rcu_rdr_boosted],
+ rscr.rcu_rdr_cpu,
+ rscr.rcu_rdr_cpu_offline ? "!" : "",
+ tc ? tc->comm : "???",
+ tc ? tc->pid : -1,
+ tc ? (is_idle_task(tc) ? " (idle!)" : "") : " (no task)");
+ }
lockdep_assert_irqs_disabled();
- put_task_struct(t);
ndetected++;
}
pr_cont("\n");
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b5b205b1a6e22..98b45b2557e92 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4605,6 +4605,19 @@ struct task_struct *cpu_curr_snapshot(int cpu)
return t;
}
+/**
+ * cpu_curr_snapshot_rcu - Return the task currently running on CPU
+ * @cpu: CPU to query
+ *
+ * The caller must be within an RCU read-side critical section.
+ *
+ * Return: Pointer to task_struct of task running on @cpu.
+ */
+struct task_struct *cpu_curr_snapshot_rcu(int cpu)
+{
+ return rcu_dereference_all(cpu_curr(cpu));
+}
+
/**
* wake_up_process - Wake up a specific process
* @p: The process to be woken up.
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 8/8] doc: Document additional RCU task-stall dump information
2026-11-09 22:19 [PATCH 0/8] Miscellaneous RCU updates for v7.5 Paul E. McKenney
` (6 preceding siblings ...)
2026-11-09 22:19 ` [PATCH 7/8] rcu: Print CPU that preempted stalled reader wants to run on Paul E. McKenney
@ 2026-11-09 22:19 ` Paul E. McKenney
7 siblings, 0 replies; 9+ messages in thread
From: Paul E. McKenney @ 2026-11-09 22:19 UTC (permalink / raw)
To: rcu; +Cc: linux-kernel, kernel-team, rostedt, Paul E. McKenney
This commit adds explanation for the added task-stall dumps, which can
be as follows:
Tasks blocked on level-1 rcu_node (CPUs 0-8): P96/1:b..l.B CPU 5 swapper/5 (0) (idle!)
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
Documentation/RCU/stallwarn.rst | 71 +++++++++++++++++++++++++++++++--
1 file changed, 68 insertions(+), 3 deletions(-)
diff --git a/Documentation/RCU/stallwarn.rst b/Documentation/RCU/stallwarn.rst
index a986358a5c279..b888e13c02a49 100644
--- a/Documentation/RCU/stallwarn.rst
+++ b/Documentation/RCU/stallwarn.rst
@@ -234,6 +234,7 @@ CPU is stalling, it will print a message similar to the following::
INFO: rcu_sched detected stalls on CPUs/tasks:
2-...: (3 GPs behind) idle=06c/0/0 softirq=1453/1455 fqs=0
16-...: (0 ticks this GP) idle=81c/0/0 softirq=764/764 fqs=0
+ Tasks blocked on level-1 rcu_node (CPUs 0-8): P96/1:b..l.B CPU 5 swapper/5 (0) (idle!)
(detected by 32, t=2603 jiffies, g=7075, q=625)
This message indicates that CPU 32 detected that CPUs 2 and 16 were both
@@ -278,6 +279,70 @@ detection passes that the grace-period kthread has made across this
CPU since the last time that this CPU noted the beginning of a grace
period.
+The "Tasks blocked" line lists the tasks blocked on each leaf ``rcu_node``
+structure. The "(CPUs 0-8)" indicates that this particular leaf
+``rcu_node`` structure handles the nine CPUs numbered from 0 to 8,
+which indicates an odd but perfectly legal state of affairs ("0-15"
+is typical). The "P96" indicates that the task with PID 96 is blocking
+the current grace period.
+
+Certain race conditions can prevent the remaining information on this
+line from appearing, most typically that task P96 is currently running on
+some other CPU. The "/1" indicates that this task was preempted within
+its top-level RCU read-side critical section. The ":b..l.B" indicates
+task state as follows, with "." indicating that the corresponding state
+is not in effect:
+
+b (blocked)
+ This task has been preempted (blocked) at least once during its
+ current RCU read-side critical section.
+
+q (needs quiescent state)
+ This task has been flagged as running on a CPU from which RCU needs
+ a quiescent state.
+
+e (expedited hint)
+ An expedited RCU grace period is interested in this task.
+
+l (on ->blkd_tasks list)
+ This task is currently on a list of blocked (preempted) tasks that
+ either are or might some day hold up an RCU grace period. Note that
+ if the task has not yet been preempted, this task will not be on any
+ of these lists, which means that RCU will still be waiting on the
+ CPU the task is running on, not yet the task itself.
+
+R (Running)
+ This task is running (as opposed to just runnable, that is, preempted).
+ This is unusual, but could happen if the task was preempted and
+ started running again before this RCU CPU stall warning.
+
+B (Boosted)
+ This task is currently priority boosted. There is a narrow window
+ during which the rtmutex has been created with this task as owner,
+ but the rcub task has not yet attempted to acquire that rtmutex.
+ During this narrow window, the task would be reported as being boosted,
+ but would not yet actually be boosted.
+
+The "CPU 5" indicates that the task is in CPU 5’s runqueue and that
+this CPU is online. If the CPU were offline, this would instead read
+"CPU 5!". The "swapper/5" is the running task’s ->comm string.
+If there was somehow no task running on CPU 5, this would instead read
+"???". The "(0)" indicates that the running task’s PID is zero.
+Which is a problem. If there was somehow no task running on CPU 5,
+this would instead read "(-1)".
+
+The "(idle!)" indicates that the task currently running on CPU 5 is its
+idle task, which is not supposed to happen, other than during small
+windows of time where one CPU wakes up a task that is to run on an
+idle CPU. (The rescheduling IPI does after all take time to reach the
+idle CPU!) If there was somehow no task running on CPU 5, this would
+instead read "(no task)". Otherwise, nothing would be printed.
+
+If there was more than one task queued on this ``rcu_node`` structure,
+for example, suppose that task P192 was also blocking the current RCU
+grace period. In that case, the line would contine with P192 followed
+optionally by the rest of the per-task information.
+
The "detected by" line indicates which CPU detected the stall (in this
case, CPU 32), how many jiffies have elapsed since the start of the grace
period (in this case 2603), the grace-period sequence number (7075), and
@@ -307,7 +372,7 @@ The "23807" indicates that it has been more than 23 thousand jiffies
since the grace-period kthread ran. The "jiffies_till_next_fqs"
indicates how frequently that kthread should run, giving the number
of jiffies between force-quiescent-state scans, in this case three,
-which is way less than 23807. Finally, the root rcu_node structure's
+which is way less than 23807. Finally, the root ``rcu_node`` structure's
->qsmask field is printed, which will normally be zero.
If the relevant grace-period kthread has been unable to run prior to
@@ -382,10 +447,10 @@ jiffies. The number following the "s:" indicates that the expedited
grace-period sequence counter is 73. The fact that this last value is
odd indicates that an expedited grace period is in flight. The number
following "root:" is a bitmask that indicates which children of the root
-rcu_node structure correspond to CPUs and/or tasks that are blocking the
+``rcu_node`` structure correspond to CPUs and/or tasks that are blocking the
current expedited grace period. If the tree had more than one level,
additional hex numbers would be printed for the states of the other
-rcu_node structures in the tree.
+``rcu_node`` structures in the tree.
As with normal grace periods, PREEMPT_RCU builds can be stalled by
tasks as well as by CPUs, and that the tasks will be indicated by PID,
--
2.40.1
^ permalink raw reply [flat|nested] 9+ messages in thread