* [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
@ 2026-11-16 19:44 Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
Hi,
This is v2 of the series that rewrites rcutorture's RCU priority-boost test
around a CPU-hungry reader, so that it neither depends on disabling RT
throttling nor infers boost failure from a grace period merely taking too
long. The approach is unchanged, see the v1 cover letter for the full
description:
v1: https://lore.kernel.org/all/20261111213826.2035045-1-joelagnelf@nvidia.com/
Changes since v1 (all from Sebastian's review, thanks!):
- 1/7: Use kthread_run_on_cpu() for the reader thread. Range-check
test_boost_reader_ms in rcu_torture_init(), clamping it to between
1 ms and test_boost_duration. Use div64_ul() when printing the
reader's CPU time.
- 2/7: Replaced by Sebastian's patch [1], which also removes the
sysctl_sched_rt_runtime declaration from rcu.h.
- 3/7: Drop the sched/sysctl.h include that crept in through a bad fold,
and also remove rcu_check_boost_fail(), which has no callers left.
- 4/7-7/7: Unchanged, other than the rebase.
- Rebased onto the current rcu/next.
Testing: kvm.sh with --duration 30 and --cpus set to all CPUs, on x86 (32
CPUs) and arm64 (80 Neoverse-N1 cores). Boost failures / boost intervals:
Scenario x86 arm64
TREE03 0 / 220 0 / 217
BUSTED-BOOST 247 / 252 250 / 251 (expected, boosting disabled)
[1] https://lore.kernel.org/all/20261113-sched_dl-v3-3-a76ffa1cdeee@linutronix.de/
Joel Fernandes (6):
rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
rcutorture: Remove unused check_boost_failed hook
doc: Document rcutorture.test_boost_reader_ms
rcu: Add rcutree.boost_disable to disable priority boosting at runtime
doc: Document rcutree.boost_disable
rcutorture: Make BUSTED-BOOST disable RCU priority boosting
Sebastian Andrzej Siewior (1):
rcutorture: Don't try to disable to RT throttling.
.../admin-guide/kernel-parameters.txt | 14 ++
kernel/rcu/rcu.h | 4 -
kernel/rcu/rcutorture.c | 197 +++++++++---------
kernel/rcu/tree_plugin.h | 5 +-
kernel/rcu/tree_stall.h | 57 -----
.../rcutorture/configs/rcu/BUSTED-BOOST.boot | 1 +
6 files changed, 120 insertions(+), 158 deletions(-)
base-commit: e97b821f72876c4760bddc79f0858155e012da8b
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
The old boost test relies on RT throttling being disabled and on grace
periods taking too long, which is fragile. Instead, each interval one
CPU runs a normal-priority reader that spins inside an RCU read-side
critical section until it has consumed test_boost_reader_ms of CPU time
(measured via sum_exec_runtime, so time spent preempted does not count),
while a SCHED_FIFO hog on the same CPU tries to starve it. With RT
throttling enabled, an unboosted reader gets only about 5% of the CPU,
far less than it needs within test_boost_duration. If the reader
finishes, boosting worked, otherwise it failed.
Only one CPU is tested per interval because RCU boosts the readers of an
rcu_node one at a time, so a second slow reader sharing the rcu_node
would wait for the first one's whole CPU burn and falsely fail.
The reader publishes completion before leaving its critical section.
Otherwise the deboost on rcu_read_unlock() lets the hog preempt the
reader before it can set its flag, which falsely fails the test.
Testing methodology:
The test must pass when boosting works and fail when it does not, so
both directions were exercised. All runs used kvm.sh with
--duration 10 --bootargs "rcutorture.test_boost=2" (which also forces
the test on in scenarios whose .boot file disables it).
Success testing: with the series applied and no other changes, run
TREE01 TREE02 TREE03 TREE08 TREE09 BUSTED-BOOST for 10 minutes each.
Every scenario ended "SUCCESS" with rtbf (boost failures) of 0 in 59 to
83 boost intervals. (BUSTED-BOOST can no longer fail by construction
because CONFIG_RCU_BOOST is now enabled automatically for preemptible
RCU.)
Failure testing: break boosting on purpose by making the early-return
check in rcu_initiate_boost() (kernel/rcu/tree_plugin.h) always true:
- if (!rnp->boost_kthread_task ||
+ if (1 || !rnp->boost_kthread_task ||
then run TREE03 and TREE02 for 10 minutes each. Both ended "FAILURE",
with 58 of 72 (TREE03) and 62 of 72 (TREE02) intervals reporting:
rcu_torture_boost boosting failed
Boost inversion: reader got only 203 of 2000 ms CPU in 4003 jiffies
The 203 ms is the CPU time that RT throttling leaves a SCHED_NORMAL task
over test_boost_duration, confirming that the reader finishes only when
boosted. The remaining intervals were discarded because of hotplug or
stutter. Setting CONFIG_RCU_BOOST_DELAY=3000 also produces failures,
roughly half of the intervals.
Pitfall found while testing: a first version set the reader's finished
flag after rcu_read_unlock(), which failed about half of the intervals
in TREE01 (readers on CPUs brought online by its nr_cpus=17 hotplug)
even with boosting working, hence the ordering above.
test_boost_reader_ms should be well below test_boost_duration minus
RCU_BOOST_DELAY and slack: a boosted reader needs about
RCU_BOOST_DELAY + test_boost_reader_ms / 0.95 to finish.
test_boost_reader_ms is clamped to between 1 ms and the length of the
test window, test_boost_duration.
[ jfern: Apply feedback from Sebastian to use kthread_run_on_cpu() ]
[ jfern: Apply feedback from Sebastian to range-check reader_ms ]
[ jfern: Apply feedback from Sebastian to use div64_ul() ]
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
kernel/rcu/rcutorture.c | 184 ++++++++++++++++++++++++----------------
1 file changed, 110 insertions(+), 74 deletions(-)
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 924705b7759d..a81855efbf1f 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -149,6 +149,7 @@ torture_param(int, test_boost, 1, "Test RCU prio boost: 0=no, 1=maybe, 2=yes.");
torture_param(int, test_boost_duration, 4, "Duration of each boost test, seconds.");
torture_param(int, test_boost_holdoff, 0, "Holdoff time from rcutorture start, seconds.");
torture_param(int, test_boost_interval, 7, "Interval between boost tests, seconds.");
+torture_param(int, test_boost_reader_ms, 2000, "Boost-test reader CPU time, ms.");
torture_param(int, test_nmis, 0, "End-test NMI tests, 0 to disable.");
torture_param(bool, test_no_idle_hz, true, "Test support for tickless idle CPUs");
torture_param(int, test_srcu_lockdep, 0, "Test specified SRCU deadlock scenario.");
@@ -333,6 +334,7 @@ static bool shutdown_time_arrived(void)
}
static unsigned long boost_starttime; /* jiffies of next boost test start. */
+static unsigned long boost_claimed; /* ->boost_starttime of claimed test. */
static DEFINE_MUTEX(boost_mutex); /* protect setting boost_starttime */
/* and boost task create/destroy. */
static atomic_t barrier_cbs_count; /* Barrier callbacks registered. */
@@ -1334,10 +1336,17 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
}
/*
- * RCU torture priority-boost testing. Runs one real-time thread per
- * CPU for moderate bursts, repeatedly starting grace periods and waiting
- * for them to complete. If a given grace period takes too long, we assume
- * that priority inversion has occurred.
+ * RCU torture priority-boost testing. Each test interval, one CPU
+ * runs a normal-priority reader that spins inside an RCU read-side
+ * critical section until it has consumed test_boost_reader_ms of CPU
+ * time, while a SCHED_FIFO hog on the same CPU tries to starve it. RT
+ * throttling is left enabled, so an unboosted reader gets only a few
+ * percent of the CPU, far too little to finish within test_boost_duration.
+ * If the reader finishes anyway, boosting lifted it above the hog.
+ *
+ * Only one CPU is tested per interval because RCU boosts the readers
+ * of a given rcu_node one at a time, so a second slow reader sharing the
+ * rcu_node would wait for the first one's whole CPU burn.
*/
static int old_rt_runtime = -1;
@@ -1366,61 +1375,111 @@ static void rcu_torture_enable_rt_throttle(void)
old_rt_runtime = -1;
}
-static bool rcu_torture_boost_failed(unsigned long gp_state, unsigned long *start)
+struct rcu_torture_boost_reader {
+ struct completion done;
+ bool in_cs; // Reader is inside its RCU read-side critical section.
+ bool finished; // Reader consumed all of its CPU time.
+ bool abort; // Hog asks the reader to give up.
+ u64 ran_ns; // CPU time consumed so far.
+};
+
+// Reader CPU time so far, retrying to avoid torn 64-bit loads on 32-bit.
+static u64 rcu_torture_boost_runtime(void)
+{
+ u64 a, b;
+
+ do {
+ a = READ_ONCE(current->se.sum_exec_runtime);
+ b = READ_ONCE(current->se.sum_exec_runtime);
+ } while (a != b);
+ return a;
+}
+
+static int rcu_torture_boost_reader(void *arg)
+{
+ struct rcu_torture_boost_reader *br = arg;
+ u64 goal = (u64)test_boost_reader_ms * NSEC_PER_MSEC;
+ u64 ran;
+ u64 start;
+ int idx;
+
+ idx = cur_ops->readlock();
+ start = rcu_torture_boost_runtime();
+ WRITE_ONCE(br->in_cs, true);
+ do {
+ cpu_relax();
+ ran = rcu_torture_boost_runtime() - start;
+ WRITE_ONCE(br->ran_ns, ran);
+ } while (ran < goal && !READ_ONCE(br->abort) && !torture_must_stop());
+ // Publish before unlocking: deboosting lets the hog preempt us at once.
+ WRITE_ONCE(br->finished, ran >= goal);
+ cur_ops->readunlock(idx);
+ kthread_complete_and_exit(&br->done, 0);
+}
+
+// Run one boost test on this CPU.
+static void rcu_torture_boost_once(void)
{
- int cpu;
static int dbg_done;
- unsigned long end = jiffies;
- bool gp_done;
- unsigned long j;
- static unsigned long last_persist;
- unsigned long lp;
- unsigned long mininterval = test_boost_duration * HZ - HZ / 2;
+ struct rcu_torture_boost_reader br = { };
+ unsigned long endtime;
+ unsigned long gp_state = 0;
+ unsigned long start;
+ struct task_struct *t;
+ bool stuttered = false;
- if (end - *start > mininterval) {
- // Recheck after checking time to avoid false positives.
- smp_mb(); // Time check before grace-period check.
- if (cur_ops->poll_gp_state(gp_state))
- return false; // passed, though perhaps just barely
- if (cur_ops->check_boost_failed && !cur_ops->check_boost_failed(gp_state, &cpu)) {
- // At most one persisted message per boost test.
- j = jiffies;
- lp = READ_ONCE(last_persist);
- if (time_after(j, lp + mininterval) &&
- cmpxchg(&last_persist, lp, j) == lp) {
- if (cpu < 0)
- pr_info("Boost inversion persisted: QS from all CPUs\n");
- else
- pr_info("Boost inversion persisted: No QS from CPU %d\n", cpu);
- }
- return false; // passed on a technicality
+ init_completion(&br.done);
+ t = kthread_run_on_cpu(rcu_torture_boost_reader, &br, raw_smp_processor_id(),
+ "rcu_torture_boost_rdr_%u");
+ if (IS_ERR(t)) {
+ n_rcu_torture_boost_ktrerror++;
+ return;
+ }
+
+ // Wait for the reader to enter its critical section, then start
+ // a grace period that it will block.
+ start = jiffies;
+ while (!READ_ONCE(br.in_cs) && !torture_must_stop() &&
+ time_before(jiffies, start + HZ))
+ schedule_timeout_uninterruptible(1);
+ if (!READ_ONCE(br.in_cs))
+ goto out;
+ gp_state = cur_ops->start_gp_poll();
+
+ // Hog the CPU until the reader finishes or the interval ends.
+ start = jiffies;
+ endtime = start + test_boost_duration * HZ;
+ while (!READ_ONCE(br.finished) && time_before(jiffies, endtime) &&
+ !torture_must_stop()) {
+ if (stutter_wait("rcu_torture_boost")) {
+ sched_set_fifo_low(current);
+ stuttered = true;
+ break;
}
+ cpu_relax();
+ }
+
+ smp_mb(); // Time check before reader check.
+ if (!READ_ONCE(br.finished) && !stuttered && !torture_must_stop()) {
VERBOSE_TOROUT_STRING("rcu_torture_boost boosting failed");
n_rcu_torture_boost_failure++;
+ pr_info("Boost inversion: reader got only %llu of %d ms CPU in %lu jiffies, GP %s.\n",
+ div64_ul(READ_ONCE(br.ran_ns), NSEC_PER_MSEC), test_boost_reader_ms,
+ jiffies - start,
+ cur_ops->poll_gp_state(gp_state) ? "ended" : "still pending");
if (!xchg(&dbg_done, 1) && cur_ops->gp_kthread_dbg) {
pr_info("Boost inversion thread ->rt_priority %u gp_state %lu jiffies %lu\n",
- current->rt_priority, gp_state, end - *start);
+ current->rt_priority, gp_state, jiffies - start);
cur_ops->gp_kthread_dbg();
- // Recheck after print to flag grace period ending during splat.
- gp_done = cur_ops->poll_gp_state(gp_state);
- pr_info("Boost inversion: GP %lu %s.\n", gp_state,
- gp_done ? "ended already" : "still pending");
-
}
-
- return true; // failed
- } else if (cur_ops->check_boost_failed && !cur_ops->check_boost_failed(gp_state, NULL)) {
- *start = jiffies;
}
-
- return false; // passed
+out:
+ WRITE_ONCE(br.abort, true);
+ wait_for_completion(&br.done);
}
static int rcu_torture_boost(void *arg)
{
- unsigned long endtime;
- unsigned long gp_state;
- unsigned long gp_state_time;
unsigned long oldstarttime;
unsigned long booststarttime = get_torture_init_jiffies() + test_boost_holdoff * HZ;
@@ -1441,9 +1500,6 @@ static int rcu_torture_boost(void *arg)
/* Each pass through the following loop does one boost-test cycle. */
do {
- bool failed = false; // Test failed already in this test interval
- bool gp_initiated = false;
-
if (kthread_should_stop())
goto checkwait;
@@ -1457,33 +1513,9 @@ static int rcu_torture_boost(void *arg)
goto checkwait;
}
- // Do one boost-test interval.
- endtime = oldstarttime + test_boost_duration * HZ;
- while (time_before(jiffies, endtime)) {
- // Has current GP gone too long?
- if (gp_initiated && !failed && !cur_ops->poll_gp_state(gp_state))
- failed = rcu_torture_boost_failed(gp_state, &gp_state_time);
- // If we don't have a grace period in flight, start one.
- if (!gp_initiated || cur_ops->poll_gp_state(gp_state)) {
- gp_state = cur_ops->start_gp_poll();
- gp_initiated = true;
- gp_state_time = jiffies;
- }
- if (stutter_wait("rcu_torture_boost")) {
- sched_set_fifo_low(current);
- // If the grace period already ended,
- // we don't know when that happened, so
- // start over.
- if (cur_ops->poll_gp_state(gp_state))
- gp_initiated = false;
- }
- if (torture_must_stop())
- goto checkwait;
- }
-
- // In case the grace period extended beyond the end of the loop.
- if (gp_initiated && !failed && !cur_ops->poll_gp_state(gp_state))
- rcu_torture_boost_failed(gp_state, &gp_state_time);
+ // Do one boost-test interval, if no other CPU claimed it.
+ if (xchg(&boost_claimed, oldstarttime) != oldstarttime)
+ rcu_torture_boost_once();
/*
* Set the start time of the next test interval.
@@ -5151,6 +5183,10 @@ rcu_torture_init(void)
test_boost_interval = 1;
if (test_boost_duration < 2)
test_boost_duration = 2;
+ if (test_boost_reader_ms < 1)
+ test_boost_reader_ms = 1;
+ if (test_boost_reader_ms > test_boost_duration * MSEC_PER_SEC)
+ test_boost_reader_ms = test_boost_duration * MSEC_PER_SEC;
if (rcu_torture_can_boost()) {
boost_starttime = jiffies + test_boost_interval * HZ;
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling.
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Commit 450efca7182a5 ("rcutorture: Disable RT throttling for boost
tests") added the ability to disable RT throttling while rcutorture was
run.
Since v6.12 the DL fair_server takes control of throttling and this
interface no longer serves this purpose. That sysctl control still
exists but serves a different cause.
There is no in-kernel interface to provide this feature, one needs to
echo 0 to each /sys/kernel/debug/sched/fair_server/cpu*/runtime file.
Since sysctl_sched_rt_runtime does not do what it once did, remove it
from rcutorture.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ jfern: Rebase onto the rewritten priority-boost test ]
[ jfern: Remove the now-unneeded sched/sysctl.h include ]
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
kernel/rcu/rcu.h | 2 --
kernel/rcu/rcutorture.c | 29 -----------------------------
2 files changed, 31 deletions(-)
diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 7cae7e64e96d..9f7e60e69275 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -69,8 +69,6 @@
/* A complete grace period count */
#define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1)
-extern int sysctl_sched_rt_runtime;
-
/*
* Return the counter portion of a sequence number previously returned
* by rcu_seq_snap() or rcu_seq_current().
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index a81855efbf1f..4ec2c6e1d3ba 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -43,7 +43,6 @@
#include <linux/torture.h>
#include <linux/vmalloc.h>
#include <linux/sched/debug.h>
-#include <linux/sched/sysctl.h>
#include <linux/oom.h>
#include <linux/tick.h>
#include <linux/rcupdate_trace.h>
@@ -1349,32 +1348,6 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
* rcu_node would wait for the first one's whole CPU burn.
*/
-static int old_rt_runtime = -1;
-
-static void rcu_torture_disable_rt_throttle(void)
-{
- /*
- * Disable RT throttling so that rcutorture's boost threads don't get
- * throttled. Only possible if rcutorture is built-in otherwise the
- * user should manually do this by setting the sched_rt_period_us and
- * sched_rt_runtime sysctls.
- */
- if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime != -1)
- return;
-
- old_rt_runtime = sysctl_sched_rt_runtime;
- sysctl_sched_rt_runtime = -1;
-}
-
-static void rcu_torture_enable_rt_throttle(void)
-{
- if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime == -1)
- return;
-
- sysctl_sched_rt_runtime = old_rt_runtime;
- old_rt_runtime = -1;
-}
-
struct rcu_torture_boost_reader {
struct completion done;
bool in_cs; // Reader is inside its RCU read-side critical section.
@@ -3422,7 +3395,6 @@ static int rcutorture_booster_cleanup(unsigned int cpu)
mutex_lock(&boost_mutex);
t = boost_tasks[cpu];
boost_tasks[cpu] = NULL;
- rcu_torture_enable_rt_throttle();
mutex_unlock(&boost_mutex);
/* This must be outside of the mutex, otherwise deadlock! */
@@ -3460,7 +3432,6 @@ static int rcutorture_booster_init(unsigned int cpu)
/* Don't allow time recalculation while creating a new task. */
mutex_lock(&boost_mutex);
- rcu_torture_disable_rt_throttle();
VERBOSE_TOROUT_STRING("Creating rcu_torture_boost task");
boost_tasks[cpu] = kthread_run_on_cpu(rcu_torture_boost, NULL,
cpu, "rcu_torture_boost_%u");
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
Nothing in rcutorture calls ->check_boost_failed() now that boost
failure is decided by whether the reader finished, so drop the hook
from struct rcu_torture_ops and its initializer.
That also leaves rcu_check_boost_fail() without any callers, so remove
it as well, along with its prototype and its CONFIG_TINY_RCU stub.
[ jfern: Apply feedback from Sebastian to drop stray sysctl.h include ]
[ jfern: Apply feedback from Sebastian to remove rcu_check_boost_fail() ]
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
kernel/rcu/rcu.h | 2 --
kernel/rcu/rcutorture.c | 2 --
kernel/rcu/tree_stall.h | 57 -----------------------------------------
3 files changed, 61 deletions(-)
diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 9f7e60e69275..93542e30b7f4 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -652,7 +652,6 @@ static inline bool rcu_watching_zero_in_eqs(int cpu, int *vp) { return false; }
static inline unsigned long rcu_get_gp_seq(void) { return 0; }
static inline unsigned long rcu_exp_batches_completed(void) { return 0; }
static inline void rcu_force_quiescent_state(void) { }
-static inline bool rcu_check_boost_fail(unsigned long gp_state, int *cpup) { return true; }
static inline void show_rcu_gp_kthreads(void) { }
static inline int rcu_get_gp_kthreads_prio(void) { return 0; }
static inline void rcu_fwd_progress_check(unsigned long j) { }
@@ -662,7 +661,6 @@ static inline void rcu_gp_slow_unregister(atomic_t *rgssp) { }
bool rcu_watching_zero_in_eqs(int cpu, int *vp);
unsigned long rcu_get_gp_seq(void);
unsigned long rcu_exp_batches_completed(void);
-bool rcu_check_boost_fail(unsigned long gp_state, int *cpup);
void show_rcu_gp_kthreads(void);
int rcu_get_gp_kthreads_prio(void);
void rcu_fwd_progress_check(unsigned long j);
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 4ec2c6e1d3ba..ff674ace50a6 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -425,7 +425,6 @@ struct rcu_torture_ops {
void (*fqs)(void);
void (*stats)(void);
void (*gp_kthread_dbg)(void);
- bool (*check_boost_failed)(unsigned long gp_state, int *cpup);
int (*stall_dur)(void);
void (*get_gp_data)(int *flags, unsigned long *gp_seq);
void (*gp_slow_register)(atomic_t *rgssp);
@@ -637,7 +636,6 @@ static struct rcu_torture_ops rcu_ops = {
.cb_barrier = rcu_barrier,
.fqs = rcu_force_quiescent_state,
.gp_kthread_dbg = show_rcu_gp_kthreads,
- .check_boost_failed = rcu_check_boost_fail,
.stall_dur = rcu_jiffies_till_stall_check,
.get_gp_data = rcutorture_get_gp_data,
.gp_slow_register = rcu_gp_slow_register,
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index 08699652f51c..e0607f423564 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -922,63 +922,6 @@ static void check_cpu_stall(struct rcu_data *rdp)
// RCU forward-progress mechanisms, including for callback invocation.
-/*
- * Check to see if a failure to end RCU priority inversion was due to
- * a CPU not passing through a quiescent state. When this happens, there
- * is nothing that RCU priority boosting can do to help, so we shouldn't
- * count this as an RCU priority boosting failure. A return of true says
- * RCU priority boosting is to blame, and false says otherwise. If false
- * is returned, the first of the CPUs to blame is stored through cpup.
- * If there was no CPU blocking the current grace period, but also nothing
- * in need of being boosted, *cpup is set to -1. This can happen in case
- * of vCPU preemption while the last CPU is reporting its quiscent state,
- * for example.
- *
- * If cpup is NULL, then a lockless quick check is carried out, suitable
- * for high-rate usage. On the other hand, if cpup is non-NULL, each
- * rcu_node structure's ->lock is acquired, ruling out high-rate usage.
- */
-bool rcu_check_boost_fail(unsigned long gp_state, int *cpup)
-{
- bool atb = false;
- int cpu;
- unsigned long flags;
- struct rcu_node *rnp;
-
- rcu_for_each_leaf_node(rnp) {
- if (!cpup) {
- if (data_race(READ_ONCE(rnp->qsmask))) {
- return false;
- } else {
- if (READ_ONCE(rnp->gp_tasks))
- atb = true;
- continue;
- }
- }
- *cpup = -1;
- raw_spin_lock_irqsave_rcu_node(rnp, flags);
- if (rnp->gp_tasks)
- atb = true;
- if (!rnp->qsmask) {
- // No CPUs without quiescent states for this rnp.
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- continue;
- }
- // Find the first holdout CPU.
- for_each_leaf_node_possible_cpu(rnp, cpu) {
- if (rnp->qsmask & (1UL << (cpu - rnp->grplo))) {
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- *cpup = cpu;
- return false;
- }
- }
- raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
- }
- // Can't blame CPUs, so must blame RCU priority boosting.
- return atb;
-}
-EXPORT_SYMBOL_GPL(rcu_check_boost_fail);
-
static noinline_for_stack void show_rcu_state(void)
{
unsigned long j;
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
` (2 preceding siblings ...)
2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
Add the new priority-boost reader CPU-time parameter to
kernel-parameters.txt.
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 176648941aec..ac97cbd98b46 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6283,6 +6283,13 @@ Kernel parameters
rcutorture.test_boost_interval= [KNL]
Interval (s) between each boost test.
+ rcutorture.test_boost_reader_ms= [KNL]
+ CPU time (ms) that each boost-test reader must
+ consume within the test_boost_duration interval.
+ The reader runs in an RCU read-side critical section
+ while a real-time CPU hog tries to starve it, so it
+ can finish in time only if it is priority boosted.
+
rcutorture.test_no_idle_hz= [KNL]
Test RCU's dyntick-idle handling. See also the
rcutorture.shuffle_interval parameter.
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
` (3 preceding siblings ...)
2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
CONFIG_RCU_BOOST is now enabled automatically for preemptible RCU and
cannot be turned off, so rcutorture's BUSTED-BOOST scenario, which is
meant to show that rcutorture detects broken priority boosting, can no
longer break it. Add a boot parameter that makes rcu_initiate_boost()
do nothing, giving that scenario a way to break boosting.
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
kernel/rcu/tree_plugin.h | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index ae5d21bae7bc..04a58dd003f4 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1270,6 +1270,9 @@ static int rcu_boost_kthread(void *arg)
return 0;
}
+static bool boost_disable;
+module_param(boost_disable, bool, 0444);
+
/*
* Check to see if it is time to start boosting RCU readers that are
* blocking the current grace period, and, if so, tell the per-rcu_node
@@ -1284,7 +1287,7 @@ static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
__releases(rnp->lock)
{
raw_lockdep_assert_held_rcu_node(rnp);
- if (!rnp->boost_kthread_task ||
+ if (boost_disable || !rnp->boost_kthread_task ||
(!rcu_preempt_blocked_readers_cgp(rnp) && !rnp->exp_tasks)) {
raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
return;
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 6/7] doc: Document rcutree.boost_disable
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
` (4 preceding siblings ...)
2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
Add the new rcutree.boost_disable parameter to kernel-parameters.txt.
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ac97cbd98b46..8f91499b3488 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5651,6 +5651,13 @@ Kernel parameters
Set maximum number of finished RCU callbacks to
process in one batch.
+ rcutree.boost_disable= [KNL]
+ Disable RCU priority boosting at runtime, even in
+ kernels built with CONFIG_RCU_BOOST=y. This is
+ for testing the boost tests, for example by the
+ rcutorture BUSTED-BOOST scenario, and should not
+ be used otherwise.
+
rcutree.csd_lock_suppress_rcu_stall= [KNL]
Do only a one-line RCU CPU stall warning when
there is an ongoing too-long CSD-lock wait.
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
` (5 preceding siblings ...)
2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
To: paulmck, rcu, linux-kernel
Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
linux-kselftest
BUSTED-BOOST is supposed to fail because priority boosting is broken,
but boosting is now always built in for preemptible RCU, so the
scenario passed. Boot it with rcutree.boost_disable=1 so that it again
exercises rcutorture's ability to detect missing priority boosting.
Testing: kvm.sh --configs BUSTED-BOOST --duration 10 now ends FAILURE,
with 81 of 84 boost intervals reporting:
rcu_torture_boost boosting failed
Boost inversion: reader got only 203 of 2000 ms CPU in 4003 jiffies
Without the parameter the same scenario passed with no boost failures.
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot b/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
index 84f6bb98ce99..5aa92357958a 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
@@ -6,3 +6,4 @@ rcutree.gp_cleanup_delay=3
rcutree.kthread_prio=2
threadirqs
rcutree.use_softirq=0
+rcutree.boost_disable=1
--
2.54.0
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-11-16 19:44 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®