mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
@ 2026-11-16 19:44 Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
                   ` (6 more replies)
  0 siblings, 7 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

Hi,

This is v2 of the series that rewrites rcutorture's RCU priority-boost test
around a CPU-hungry reader, so that it neither depends on disabling RT
throttling nor infers boost failure from a grace period merely taking too
long.  The approach is unchanged, see the v1 cover letter for the full
description:

  v1: https://lore.kernel.org/all/20261111213826.2035045-1-joelagnelf@nvidia.com/

Changes since v1 (all from Sebastian's review, thanks!):

 - 1/7: Use kthread_run_on_cpu() for the reader thread.  Range-check
   test_boost_reader_ms in rcu_torture_init(), clamping it to between
   1 ms and test_boost_duration.  Use div64_ul() when printing the
   reader's CPU time.
 - 2/7: Replaced by Sebastian's patch [1], which also removes the
   sysctl_sched_rt_runtime declaration from rcu.h. 
 - 3/7: Drop the sched/sysctl.h include that crept in through a bad fold,
   and also remove rcu_check_boost_fail(), which has no callers left.
 - 4/7-7/7: Unchanged, other than the rebase.
 - Rebased onto the current rcu/next.

Testing: kvm.sh with --duration 30 and --cpus set to all CPUs, on x86 (32
CPUs) and arm64 (80 Neoverse-N1 cores).  Boost failures / boost intervals:

  Scenario      x86           arm64
  TREE03        0 / 220       0 / 217
  BUSTED-BOOST  247 / 252     250 / 251    (expected, boosting disabled)

[1] https://lore.kernel.org/all/20261113-sched_dl-v3-3-a76ffa1cdeee@linutronix.de/

Joel Fernandes (6):
  rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
  rcutorture: Remove unused check_boost_failed hook
  doc: Document rcutorture.test_boost_reader_ms
  rcu: Add rcutree.boost_disable to disable priority boosting at runtime
  doc: Document rcutree.boost_disable
  rcutorture: Make BUSTED-BOOST disable RCU priority boosting

Sebastian Andrzej Siewior (1):
  rcutorture: Don't try to disable to RT throttling.

 .../admin-guide/kernel-parameters.txt         |  14 ++
 kernel/rcu/rcu.h                              |   4 -
 kernel/rcu/rcutorture.c                       | 197 +++++++++---------
 kernel/rcu/tree_plugin.h                      |   5 +-
 kernel/rcu/tree_stall.h                       |  57 -----
 .../rcutorture/configs/rcu/BUSTED-BOOST.boot  |   1 +
 6 files changed, 120 insertions(+), 158 deletions(-)


base-commit: e97b821f72876c4760bddc79f0858155e012da8b
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 1/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

The old boost test relies on RT throttling being disabled and on grace
periods taking too long, which is fragile.  Instead, each interval one
CPU runs a normal-priority reader that spins inside an RCU read-side
critical section until it has consumed test_boost_reader_ms of CPU time
(measured via sum_exec_runtime, so time spent preempted does not count),
while a SCHED_FIFO hog on the same CPU tries to starve it.  With RT
throttling enabled, an unboosted reader gets only about 5% of the CPU,
far less than it needs within test_boost_duration.  If the reader
finishes, boosting worked, otherwise it failed.

Only one CPU is tested per interval because RCU boosts the readers of an
rcu_node one at a time, so a second slow reader sharing the rcu_node
would wait for the first one's whole CPU burn and falsely fail.

The reader publishes completion before leaving its critical section.
Otherwise the deboost on rcu_read_unlock() lets the hog preempt the
reader before it can set its flag, which falsely fails the test.

Testing methodology:

The test must pass when boosting works and fail when it does not, so
both directions were exercised.  All runs used kvm.sh with
--duration 10 --bootargs "rcutorture.test_boost=2" (which also forces
the test on in scenarios whose .boot file disables it).

Success testing: with the series applied and no other changes, run
TREE01 TREE02 TREE03 TREE08 TREE09 BUSTED-BOOST for 10 minutes each.
Every scenario ended "SUCCESS" with rtbf (boost failures) of 0 in 59 to
83 boost intervals.  (BUSTED-BOOST can no longer fail by construction
because CONFIG_RCU_BOOST is now enabled automatically for preemptible
RCU.)

Failure testing: break boosting on purpose by making the early-return
check in rcu_initiate_boost() (kernel/rcu/tree_plugin.h) always true:

	-	if (!rnp->boost_kthread_task ||
	+	if (1 || !rnp->boost_kthread_task ||

then run TREE03 and TREE02 for 10 minutes each.  Both ended "FAILURE",
with 58 of 72 (TREE03) and 62 of 72 (TREE02) intervals reporting:

	rcu_torture_boost boosting failed
	Boost inversion: reader got only 203 of 2000 ms CPU in 4003 jiffies

The 203 ms is the CPU time that RT throttling leaves a SCHED_NORMAL task
over test_boost_duration, confirming that the reader finishes only when
boosted.  The remaining intervals were discarded because of hotplug or
stutter.  Setting CONFIG_RCU_BOOST_DELAY=3000 also produces failures,
roughly half of the intervals.

Pitfall found while testing: a first version set the reader's finished
flag after rcu_read_unlock(), which failed about half of the intervals
in TREE01 (readers on CPUs brought online by its nr_cpus=17 hotplug)
even with boosting working, hence the ordering above.

test_boost_reader_ms should be well below test_boost_duration minus
RCU_BOOST_DELAY and slack: a boosted reader needs about
RCU_BOOST_DELAY + test_boost_reader_ms / 0.95 to finish.

test_boost_reader_ms is clamped to between 1 ms and the length of the
test window, test_boost_duration.

[ jfern: Apply feedback from Sebastian to use kthread_run_on_cpu() ]
[ jfern: Apply feedback from Sebastian to range-check reader_ms ]
[ jfern: Apply feedback from Sebastian to use div64_ul() ]

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcutorture.c | 184 ++++++++++++++++++++++++----------------
 1 file changed, 110 insertions(+), 74 deletions(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 924705b7759d..a81855efbf1f 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -149,6 +149,7 @@ torture_param(int, test_boost, 1, "Test RCU prio boost: 0=no, 1=maybe, 2=yes.");
 torture_param(int, test_boost_duration, 4, "Duration of each boost test, seconds.");
 torture_param(int, test_boost_holdoff, 0, "Holdoff time from rcutorture start, seconds.");
 torture_param(int, test_boost_interval, 7, "Interval between boost tests, seconds.");
+torture_param(int, test_boost_reader_ms, 2000, "Boost-test reader CPU time, ms.");
 torture_param(int, test_nmis, 0, "End-test NMI tests, 0 to disable.");
 torture_param(bool, test_no_idle_hz, true, "Test support for tickless idle CPUs");
 torture_param(int, test_srcu_lockdep, 0, "Test specified SRCU deadlock scenario.");
@@ -333,6 +334,7 @@ static bool shutdown_time_arrived(void)
 }
 
 static unsigned long boost_starttime;	/* jiffies of next boost test start. */
+static unsigned long boost_claimed;	/* ->boost_starttime of claimed test. */
 static DEFINE_MUTEX(boost_mutex);	/* protect setting boost_starttime */
 					/*  and boost task create/destroy. */
 static atomic_t barrier_cbs_count;	/* Barrier callbacks registered. */
@@ -1334,10 +1336,17 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
 }
 
 /*
- * RCU torture priority-boost testing.  Runs one real-time thread per
- * CPU for moderate bursts, repeatedly starting grace periods and waiting
- * for them to complete.  If a given grace period takes too long, we assume
- * that priority inversion has occurred.
+ * RCU torture priority-boost testing.  Each test interval, one CPU
+ * runs a normal-priority reader that spins inside an RCU read-side
+ * critical section until it has consumed test_boost_reader_ms of CPU
+ * time, while a SCHED_FIFO hog on the same CPU tries to starve it.  RT
+ * throttling is left enabled, so an unboosted reader gets only a few
+ * percent of the CPU, far too little to finish within test_boost_duration.
+ * If the reader finishes anyway, boosting lifted it above the hog.
+ *
+ * Only one CPU is tested per interval because RCU boosts the readers
+ * of a given rcu_node one at a time, so a second slow reader sharing the
+ * rcu_node would wait for the first one's whole CPU burn.
  */
 
 static int old_rt_runtime = -1;
@@ -1366,61 +1375,111 @@ static void rcu_torture_enable_rt_throttle(void)
 	old_rt_runtime = -1;
 }
 
-static bool rcu_torture_boost_failed(unsigned long gp_state, unsigned long *start)
+struct rcu_torture_boost_reader {
+	struct completion done;
+	bool in_cs;	// Reader is inside its RCU read-side critical section.
+	bool finished;	// Reader consumed all of its CPU time.
+	bool abort;	// Hog asks the reader to give up.
+	u64 ran_ns;	// CPU time consumed so far.
+};
+
+// Reader CPU time so far, retrying to avoid torn 64-bit loads on 32-bit.
+static u64 rcu_torture_boost_runtime(void)
+{
+	u64 a, b;
+
+	do {
+		a = READ_ONCE(current->se.sum_exec_runtime);
+		b = READ_ONCE(current->se.sum_exec_runtime);
+	} while (a != b);
+	return a;
+}
+
+static int rcu_torture_boost_reader(void *arg)
+{
+	struct rcu_torture_boost_reader *br = arg;
+	u64 goal = (u64)test_boost_reader_ms * NSEC_PER_MSEC;
+	u64 ran;
+	u64 start;
+	int idx;
+
+	idx = cur_ops->readlock();
+	start = rcu_torture_boost_runtime();
+	WRITE_ONCE(br->in_cs, true);
+	do {
+		cpu_relax();
+		ran = rcu_torture_boost_runtime() - start;
+		WRITE_ONCE(br->ran_ns, ran);
+	} while (ran < goal && !READ_ONCE(br->abort) && !torture_must_stop());
+	// Publish before unlocking: deboosting lets the hog preempt us at once.
+	WRITE_ONCE(br->finished, ran >= goal);
+	cur_ops->readunlock(idx);
+	kthread_complete_and_exit(&br->done, 0);
+}
+
+// Run one boost test on this CPU.
+static void rcu_torture_boost_once(void)
 {
-	int cpu;
 	static int dbg_done;
-	unsigned long end = jiffies;
-	bool gp_done;
-	unsigned long j;
-	static unsigned long last_persist;
-	unsigned long lp;
-	unsigned long mininterval = test_boost_duration * HZ - HZ / 2;
+	struct rcu_torture_boost_reader br = { };
+	unsigned long endtime;
+	unsigned long gp_state = 0;
+	unsigned long start;
+	struct task_struct *t;
+	bool stuttered = false;
 
-	if (end - *start > mininterval) {
-		// Recheck after checking time to avoid false positives.
-		smp_mb(); // Time check before grace-period check.
-		if (cur_ops->poll_gp_state(gp_state))
-			return false; // passed, though perhaps just barely
-		if (cur_ops->check_boost_failed && !cur_ops->check_boost_failed(gp_state, &cpu)) {
-			// At most one persisted message per boost test.
-			j = jiffies;
-			lp = READ_ONCE(last_persist);
-			if (time_after(j, lp + mininterval) &&
-			    cmpxchg(&last_persist, lp, j) == lp) {
-				if (cpu < 0)
-					pr_info("Boost inversion persisted: QS from all CPUs\n");
-				else
-					pr_info("Boost inversion persisted: No QS from CPU %d\n", cpu);
-			}
-			return false; // passed on a technicality
+	init_completion(&br.done);
+	t = kthread_run_on_cpu(rcu_torture_boost_reader, &br, raw_smp_processor_id(),
+			       "rcu_torture_boost_rdr_%u");
+	if (IS_ERR(t)) {
+		n_rcu_torture_boost_ktrerror++;
+		return;
+	}
+
+	// Wait for the reader to enter its critical section, then start
+	// a grace period that it will block.
+	start = jiffies;
+	while (!READ_ONCE(br.in_cs) && !torture_must_stop() &&
+	       time_before(jiffies, start + HZ))
+		schedule_timeout_uninterruptible(1);
+	if (!READ_ONCE(br.in_cs))
+		goto out;
+	gp_state = cur_ops->start_gp_poll();
+
+	// Hog the CPU until the reader finishes or the interval ends.
+	start = jiffies;
+	endtime = start + test_boost_duration * HZ;
+	while (!READ_ONCE(br.finished) && time_before(jiffies, endtime) &&
+	       !torture_must_stop()) {
+		if (stutter_wait("rcu_torture_boost")) {
+			sched_set_fifo_low(current);
+			stuttered = true;
+			break;
 		}
+		cpu_relax();
+	}
+
+	smp_mb(); // Time check before reader check.
+	if (!READ_ONCE(br.finished) && !stuttered && !torture_must_stop()) {
 		VERBOSE_TOROUT_STRING("rcu_torture_boost boosting failed");
 		n_rcu_torture_boost_failure++;
+		pr_info("Boost inversion: reader got only %llu of %d ms CPU in %lu jiffies, GP %s.\n",
+			div64_ul(READ_ONCE(br.ran_ns), NSEC_PER_MSEC), test_boost_reader_ms,
+			jiffies - start,
+			cur_ops->poll_gp_state(gp_state) ? "ended" : "still pending");
 		if (!xchg(&dbg_done, 1) && cur_ops->gp_kthread_dbg) {
 			pr_info("Boost inversion thread ->rt_priority %u gp_state %lu jiffies %lu\n",
-				current->rt_priority, gp_state, end - *start);
+				current->rt_priority, gp_state, jiffies - start);
 			cur_ops->gp_kthread_dbg();
-			// Recheck after print to flag grace period ending during splat.
-			gp_done = cur_ops->poll_gp_state(gp_state);
-			pr_info("Boost inversion: GP %lu %s.\n", gp_state,
-				gp_done ? "ended already" : "still pending");
-
 		}
-
-		return true; // failed
-	} else if (cur_ops->check_boost_failed && !cur_ops->check_boost_failed(gp_state, NULL)) {
-		*start = jiffies;
 	}
-
-	return false; // passed
+out:
+	WRITE_ONCE(br.abort, true);
+	wait_for_completion(&br.done);
 }
 
 static int rcu_torture_boost(void *arg)
 {
-	unsigned long endtime;
-	unsigned long gp_state;
-	unsigned long gp_state_time;
 	unsigned long oldstarttime;
 	unsigned long booststarttime = get_torture_init_jiffies() + test_boost_holdoff * HZ;
 
@@ -1441,9 +1500,6 @@ static int rcu_torture_boost(void *arg)
 
 	/* Each pass through the following loop does one boost-test cycle. */
 	do {
-		bool failed = false; // Test failed already in this test interval
-		bool gp_initiated = false;
-
 		if (kthread_should_stop())
 			goto checkwait;
 
@@ -1457,33 +1513,9 @@ static int rcu_torture_boost(void *arg)
 				goto checkwait;
 		}
 
-		// Do one boost-test interval.
-		endtime = oldstarttime + test_boost_duration * HZ;
-		while (time_before(jiffies, endtime)) {
-			// Has current GP gone too long?
-			if (gp_initiated && !failed && !cur_ops->poll_gp_state(gp_state))
-				failed = rcu_torture_boost_failed(gp_state, &gp_state_time);
-			// If we don't have a grace period in flight, start one.
-			if (!gp_initiated || cur_ops->poll_gp_state(gp_state)) {
-				gp_state = cur_ops->start_gp_poll();
-				gp_initiated = true;
-				gp_state_time = jiffies;
-			}
-			if (stutter_wait("rcu_torture_boost")) {
-				sched_set_fifo_low(current);
-				// If the grace period already ended,
-				// we don't know when that happened, so
-				// start over.
-				if (cur_ops->poll_gp_state(gp_state))
-					gp_initiated = false;
-			}
-			if (torture_must_stop())
-				goto checkwait;
-		}
-
-		// In case the grace period extended beyond the end of the loop.
-		if (gp_initiated && !failed && !cur_ops->poll_gp_state(gp_state))
-			rcu_torture_boost_failed(gp_state, &gp_state_time);
+		// Do one boost-test interval, if no other CPU claimed it.
+		if (xchg(&boost_claimed, oldstarttime) != oldstarttime)
+			rcu_torture_boost_once();
 
 		/*
 		 * Set the start time of the next test interval.
@@ -5151,6 +5183,10 @@ rcu_torture_init(void)
 		test_boost_interval = 1;
 	if (test_boost_duration < 2)
 		test_boost_duration = 2;
+	if (test_boost_reader_ms < 1)
+		test_boost_reader_ms = 1;
+	if (test_boost_reader_ms > test_boost_duration * MSEC_PER_SEC)
+		test_boost_reader_ms = test_boost_duration * MSEC_PER_SEC;
 	if (rcu_torture_can_boost()) {
 
 		boost_starttime = jiffies + test_boost_interval * HZ;
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling.
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Commit 450efca7182a5 ("rcutorture: Disable RT throttling for boost
tests") added the ability to disable RT throttling while rcutorture was
run.

Since v6.12 the DL fair_server takes control of throttling and this
interface no longer serves this purpose. That sysctl control still
exists but serves a different cause.

There is no in-kernel interface to provide this feature, one needs to
echo 0 to each /sys/kernel/debug/sched/fair_server/cpu*/runtime file.

Since sysctl_sched_rt_runtime does not do what it once did, remove it
from rcutorture.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ jfern: Rebase onto the rewritten priority-boost test ]
[ jfern: Remove the now-unneeded sched/sysctl.h include ]
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcu.h        |  2 --
 kernel/rcu/rcutorture.c | 29 -----------------------------
 2 files changed, 31 deletions(-)

diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 7cae7e64e96d..9f7e60e69275 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -69,8 +69,6 @@
 /* A complete grace period count */
 #define RCU_SEQ_GP (RCU_SEQ_STATE_MASK + 1)
 
-extern int sysctl_sched_rt_runtime;
-
 /*
  * Return the counter portion of a sequence number previously returned
  * by rcu_seq_snap() or rcu_seq_current().
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index a81855efbf1f..4ec2c6e1d3ba 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -43,7 +43,6 @@
 #include <linux/torture.h>
 #include <linux/vmalloc.h>
 #include <linux/sched/debug.h>
-#include <linux/sched/sysctl.h>
 #include <linux/oom.h>
 #include <linux/tick.h>
 #include <linux/rcupdate_trace.h>
@@ -1349,32 +1348,6 @@ static unsigned long rcutorture_seq_diff(unsigned long new, unsigned long old)
  * rcu_node would wait for the first one's whole CPU burn.
  */
 
-static int old_rt_runtime = -1;
-
-static void rcu_torture_disable_rt_throttle(void)
-{
-	/*
-	 * Disable RT throttling so that rcutorture's boost threads don't get
-	 * throttled. Only possible if rcutorture is built-in otherwise the
-	 * user should manually do this by setting the sched_rt_period_us and
-	 * sched_rt_runtime sysctls.
-	 */
-	if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime != -1)
-		return;
-
-	old_rt_runtime = sysctl_sched_rt_runtime;
-	sysctl_sched_rt_runtime = -1;
-}
-
-static void rcu_torture_enable_rt_throttle(void)
-{
-	if (!IS_BUILTIN(CONFIG_RCU_TORTURE_TEST) || old_rt_runtime == -1)
-		return;
-
-	sysctl_sched_rt_runtime = old_rt_runtime;
-	old_rt_runtime = -1;
-}
-
 struct rcu_torture_boost_reader {
 	struct completion done;
 	bool in_cs;	// Reader is inside its RCU read-side critical section.
@@ -3422,7 +3395,6 @@ static int rcutorture_booster_cleanup(unsigned int cpu)
 	mutex_lock(&boost_mutex);
 	t = boost_tasks[cpu];
 	boost_tasks[cpu] = NULL;
-	rcu_torture_enable_rt_throttle();
 	mutex_unlock(&boost_mutex);
 
 	/* This must be outside of the mutex, otherwise deadlock! */
@@ -3460,7 +3432,6 @@ static int rcutorture_booster_init(unsigned int cpu)
 
 	/* Don't allow time recalculation while creating a new task. */
 	mutex_lock(&boost_mutex);
-	rcu_torture_disable_rt_throttle();
 	VERBOSE_TOROUT_STRING("Creating rcu_torture_boost task");
 	boost_tasks[cpu] = kthread_run_on_cpu(rcu_torture_boost, NULL,
 					      cpu, "rcu_torture_boost_%u");
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

Nothing in rcutorture calls ->check_boost_failed() now that boost
failure is decided by whether the reader finished, so drop the hook
from struct rcu_torture_ops and its initializer.

That also leaves rcu_check_boost_fail() without any callers, so remove
it as well, along with its prototype and its CONFIG_TINY_RCU stub.

[ jfern: Apply feedback from Sebastian to drop stray sysctl.h include ]
[ jfern: Apply feedback from Sebastian to remove rcu_check_boost_fail() ]

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcu.h        |  2 --
 kernel/rcu/rcutorture.c |  2 --
 kernel/rcu/tree_stall.h | 57 -----------------------------------------
 3 files changed, 61 deletions(-)

diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
index 9f7e60e69275..93542e30b7f4 100644
--- a/kernel/rcu/rcu.h
+++ b/kernel/rcu/rcu.h
@@ -652,7 +652,6 @@ static inline bool rcu_watching_zero_in_eqs(int cpu, int *vp) { return false; }
 static inline unsigned long rcu_get_gp_seq(void) { return 0; }
 static inline unsigned long rcu_exp_batches_completed(void) { return 0; }
 static inline void rcu_force_quiescent_state(void) { }
-static inline bool rcu_check_boost_fail(unsigned long gp_state, int *cpup) { return true; }
 static inline void show_rcu_gp_kthreads(void) { }
 static inline int rcu_get_gp_kthreads_prio(void) { return 0; }
 static inline void rcu_fwd_progress_check(unsigned long j) { }
@@ -662,7 +661,6 @@ static inline void rcu_gp_slow_unregister(atomic_t *rgssp) { }
 bool rcu_watching_zero_in_eqs(int cpu, int *vp);
 unsigned long rcu_get_gp_seq(void);
 unsigned long rcu_exp_batches_completed(void);
-bool rcu_check_boost_fail(unsigned long gp_state, int *cpup);
 void show_rcu_gp_kthreads(void);
 int rcu_get_gp_kthreads_prio(void);
 void rcu_fwd_progress_check(unsigned long j);
diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 4ec2c6e1d3ba..ff674ace50a6 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -425,7 +425,6 @@ struct rcu_torture_ops {
 	void (*fqs)(void);
 	void (*stats)(void);
 	void (*gp_kthread_dbg)(void);
-	bool (*check_boost_failed)(unsigned long gp_state, int *cpup);
 	int (*stall_dur)(void);
 	void (*get_gp_data)(int *flags, unsigned long *gp_seq);
 	void (*gp_slow_register)(atomic_t *rgssp);
@@ -637,7 +636,6 @@ static struct rcu_torture_ops rcu_ops = {
 	.cb_barrier		= rcu_barrier,
 	.fqs			= rcu_force_quiescent_state,
 	.gp_kthread_dbg		= show_rcu_gp_kthreads,
-	.check_boost_failed	= rcu_check_boost_fail,
 	.stall_dur		= rcu_jiffies_till_stall_check,
 	.get_gp_data		= rcutorture_get_gp_data,
 	.gp_slow_register	= rcu_gp_slow_register,
diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
index 08699652f51c..e0607f423564 100644
--- a/kernel/rcu/tree_stall.h
+++ b/kernel/rcu/tree_stall.h
@@ -922,63 +922,6 @@ static void check_cpu_stall(struct rcu_data *rdp)
 // RCU forward-progress mechanisms, including for callback invocation.
 
 
-/*
- * Check to see if a failure to end RCU priority inversion was due to
- * a CPU not passing through a quiescent state.  When this happens, there
- * is nothing that RCU priority boosting can do to help, so we shouldn't
- * count this as an RCU priority boosting failure.  A return of true says
- * RCU priority boosting is to blame, and false says otherwise.  If false
- * is returned, the first of the CPUs to blame is stored through cpup.
- * If there was no CPU blocking the current grace period, but also nothing
- * in need of being boosted, *cpup is set to -1.  This can happen in case
- * of vCPU preemption while the last CPU is reporting its quiscent state,
- * for example.
- *
- * If cpup is NULL, then a lockless quick check is carried out, suitable
- * for high-rate usage.  On the other hand, if cpup is non-NULL, each
- * rcu_node structure's ->lock is acquired, ruling out high-rate usage.
- */
-bool rcu_check_boost_fail(unsigned long gp_state, int *cpup)
-{
-	bool atb = false;
-	int cpu;
-	unsigned long flags;
-	struct rcu_node *rnp;
-
-	rcu_for_each_leaf_node(rnp) {
-		if (!cpup) {
-			if (data_race(READ_ONCE(rnp->qsmask))) {
-				return false;
-			} else {
-				if (READ_ONCE(rnp->gp_tasks))
-					atb = true;
-				continue;
-			}
-		}
-		*cpup = -1;
-		raw_spin_lock_irqsave_rcu_node(rnp, flags);
-		if (rnp->gp_tasks)
-			atb = true;
-		if (!rnp->qsmask) {
-			// No CPUs without quiescent states for this rnp.
-			raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
-			continue;
-		}
-		// Find the first holdout CPU.
-		for_each_leaf_node_possible_cpu(rnp, cpu) {
-			if (rnp->qsmask & (1UL << (cpu - rnp->grplo))) {
-				raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
-				*cpup = cpu;
-				return false;
-			}
-		}
-		raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
-	}
-	// Can't blame CPUs, so must blame RCU priority boosting.
-	return atb;
-}
-EXPORT_SYMBOL_GPL(rcu_check_boost_fail);
-
 static noinline_for_stack void show_rcu_state(void)
 {
 	unsigned long j;
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
                   ` (2 preceding siblings ...)
  2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

Add the new priority-boost reader CPU-time parameter to
kernel-parameters.txt.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 176648941aec..ac97cbd98b46 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6283,6 +6283,13 @@ Kernel parameters
 	rcutorture.test_boost_interval= [KNL]
 			Interval (s) between each boost test.
 
+	rcutorture.test_boost_reader_ms= [KNL]
+			CPU time (ms) that each boost-test reader must
+			consume within the test_boost_duration interval.
+			The reader runs in an RCU read-side critical section
+			while a real-time CPU hog tries to starve it, so it
+			can finish in time only if it is priority boosted.
+
 	rcutorture.test_no_idle_hz= [KNL]
 			Test RCU's dyntick-idle handling.  See also the
 			rcutorture.shuffle_interval parameter.
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
                   ` (3 preceding siblings ...)
  2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

CONFIG_RCU_BOOST is now enabled automatically for preemptible RCU and
cannot be turned off, so rcutorture's BUSTED-BOOST scenario, which is
meant to show that rcutorture detects broken priority boosting, can no
longer break it.  Add a boot parameter that makes rcu_initiate_boost()
do nothing, giving that scenario a way to break boosting.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/tree_plugin.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index ae5d21bae7bc..04a58dd003f4 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -1270,6 +1270,9 @@ static int rcu_boost_kthread(void *arg)
 	return 0;
 }
 
+static bool boost_disable;
+module_param(boost_disable, bool, 0444);
+
 /*
  * Check to see if it is time to start boosting RCU readers that are
  * blocking the current grace period, and, if so, tell the per-rcu_node
@@ -1284,7 +1287,7 @@ static void rcu_initiate_boost(struct rcu_node *rnp, unsigned long flags)
 	__releases(rnp->lock)
 {
 	raw_lockdep_assert_held_rcu_node(rnp);
-	if (!rnp->boost_kthread_task ||
+	if (boost_disable || !rnp->boost_kthread_task ||
 	    (!rcu_preempt_blocked_readers_cgp(rnp) && !rnp->exp_tasks)) {
 		raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
 		return;
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 6/7] doc: Document rcutree.boost_disable
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
                   ` (4 preceding siblings ...)
  2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

Add the new rcutree.boost_disable parameter to kernel-parameters.txt.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index ac97cbd98b46..8f91499b3488 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5651,6 +5651,13 @@ Kernel parameters
 			Set maximum number of finished RCU callbacks to
 			process in one batch.
 
+	rcutree.boost_disable=	[KNL]
+			Disable RCU priority boosting at runtime, even in
+			kernels built with CONFIG_RCU_BOOST=y.  This is
+			for testing the boost tests, for example by the
+			rcutorture BUSTED-BOOST scenario, and should not
+			be used otherwise.
+
 	rcutree.csd_lock_suppress_rcu_stall=	[KNL]
 			Do only a one-line RCU CPU stall warning when
 			there is an ongoing too-long CSD-lock wait.
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting
  2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
                   ` (5 preceding siblings ...)
  2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
@ 2026-11-16 19:44 ` Joel Fernandes
  6 siblings, 0 replies; 8+ messages in thread
From: Joel Fernandes @ 2026-11-16 19:44 UTC (permalink / raw)
  To: paulmck, rcu, linux-kernel
  Cc: Sebastian Andrzej Siewior, Frederic Weisbecker, Neeraj Upadhyay,
	Boqun Feng, Joel Fernandes, Josh Triplett, Uladzislau Rezki,
	Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
	Davidlohr Bueso, Jonathan Corbet, Shuah Khan, linux-doc,
	linux-kselftest

BUSTED-BOOST is supposed to fail because priority boosting is broken,
but boosting is now always built in for preemptible RCU, so the
scenario passed.  Boot it with rcutree.boost_disable=1 so that it again
exercises rcutorture's ability to detect missing priority boosting.

Testing: kvm.sh --configs BUSTED-BOOST --duration 10 now ends FAILURE,
with 81 of 84 boost intervals reporting:

	rcu_torture_boost boosting failed
	Boost inversion: reader got only 203 of 2000 ms CPU in 4003 jiffies

Without the parameter the same scenario passed with no boost failures.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot b/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
index 84f6bb98ce99..5aa92357958a 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/BUSTED-BOOST.boot
@@ -6,3 +6,4 @@ rcutree.gp_cleanup_delay=3
 rcutree.kthread_prio=2
 threadirqs
 rcutree.use_softirq=0
+rcutree.boost_disable=1
-- 
2.54.0


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-16 19:44 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 19:44 [PATCH v2 0/7] rcutorture: Rewrite RCU priority-boost test around a CPU-hungry reader Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 1/7] " Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 2/7] rcutorture: Don't try to disable to RT throttling Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 3/7] rcutorture: Remove unused check_boost_failed hook Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 4/7] doc: Document rcutorture.test_boost_reader_ms Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 5/7] rcu: Add rcutree.boost_disable to disable priority boosting at runtime Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 6/7] doc: Document rcutree.boost_disable Joel Fernandes
2026-11-16 19:44 ` [PATCH v2 7/7] rcutorture: Make BUSTED-BOOST disable RCU priority boosting Joel Fernandes

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®