[PATCH for-v7.5 00/14] RCU tests patches for v7.5

[PATCH for-v7.5 00/14] RCU tests patches for v7.5

[Joel Fernandes]

This series has the RCU tests patches for the v7.5 merge window.

Fu Wenxin (1):
  rcutorture: Fix busted_srcud crash on module unload

Joel Fernandes (5):
  kasan: Avoid an unrelated RCU warning in the callback test
  rcuscale: Consolidate per-writer arrays into struct writer_state
  scftorture: Reject empty workloads before starting helpers
  scftorture: Protect the initial affinity check from hotplug
  torture: Forgive SMT-control failures in CPU-hotplug testing

Kunwu Chan (3):
  Documentation/litmus-tests: Add SRCU fastpath anchor-before-scan test
  Documentation/litmus-tests: Add SRCU fastpath scan-before-anchor test
  rcuscale: Add srcua scale type

Paul E. McKenney (2):
  rcutorture: Enable RCU_BOOST where supported
  rcutorture: Disable RCU priority boost testing on non-TREE03 scenarios

Zqiang (3):
  rcutorture: Fix suspicious rcu_dereference_check() usage for
    trivial-preempt
  rcutorture: Make tasks-tracing support duplicate call testing
  rcutorture: Avoid create fwd_prog_tasks for tasks-rude and srcu-atomic

-- 
2.54.0

[PATCH for-v7.5 01/14] Documentation/litmus-tests: Add SRCU fastpath anchor-before-scan test

[Joel Fernandes]

From: Kunwu Chan <kunwu.chan@gmail.com>

synchronize_srcu_atomic() may end its grace period immediately when
its scan of the per-CPU lock counters finds no readers.  Correctness
requires the grace-period anchor written by srcu_gp_start() to precede
the smp_mb() ordering the lock scan.  This ordering ensures that any
reader whose lock increment is missed by the scan cannot have
incremented its lock counter before the grace-period anchor, and
therefore cannot be a pre-existing reader of this grace period.

This litmus test models the key ordering between the grace-period
anchor and the lock counter scan, where "seq" models the
grace-period anchor in ->srcu_gp_seq and "ctr" models the per-CPU
->srcu_ctrs[].srcu_locks counter.  P0 writes the anchor before the
smp_mb() and the lock scan.  P1 models the reader-side counter
increment.  P2 models an observer that sees the reader's increment
before seeing the anchor.

The outcome is forbidden by LKMM, and herd7 reports "Never".  See
SRCU-fastpath-scan-before-anchor.litmus for the reversed ordering,
which permits this outcome.

Tested with herd7 7.58 using linux-kernel.cfg.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 .../SRCU-fastpath-anchor-before-scan.litmus   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100644 Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus

diff --git a/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus b/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus
new file mode 100644
index 000000000000..8028f2ade733
--- /dev/null
+++ b/Documentation/litmus-tests/srcu/SRCU-fastpath-anchor-before-scan.litmus
@@ -0,0 +1,55 @@
+C SRCU-fastpath-anchor-before-scan
+
+(*
+ * Result: Never
+ *
+ * The synchronize_srcu_atomic() fastpath may end its grace period
+ * immediately when its scan of the per-CPU lock counters finds no
+ * readers.  Correctness requires the grace-period anchor written by
+ * srcu_gp_start() to precede the smp_mb() ordering the lock scan.
+ * This ordering ensures that any reader whose lock increment is missed
+ * by the scan cannot have incremented its lock counter before the
+ * grace-period anchor, and therefore cannot be a pre-existing reader
+ * of this grace period.
+ *
+ * This litmus test models the key ordering between the grace-period
+ * anchor and the lock counter scan, where "seq" models the
+ * grace-period anchor in ->srcu_gp_seq and "ctr" models the per-CPU
+ * ->srcu_ctrs[].srcu_locks counter.  P0 writes the anchor before the
+ * smp_mb() and the lock scan.  P1 models the reader-side counter
+ * increment.  P2 models an observer that sees the reader's increment
+ * before seeing the anchor.
+ *
+ * The outcome is forbidden by LKMM, and herd7 reports "Never".  See
+ * SRCU-fastpath-scan-before-anchor.litmus for the reversed ordering,
+ * which permits this outcome.
+ *)
+
+{}
+
+P0(int *seq, int *ctr)
+{
+	int r2;
+
+	WRITE_ONCE(*seq, 1);
+	smp_mb();
+	r2 = READ_ONCE(*ctr);
+}
+
+P1(int *ctr)
+{
+	WRITE_ONCE(*ctr, 1);
+}
+
+P2(int *seq, int *ctr)
+{
+	int r3;
+	int r4;
+
+	r3 = READ_ONCE(*ctr);
+	smp_mb();
+	r4 = READ_ONCE(*seq);
+}
+
+filter (0:r2 = 0)
+exists (2:r3 = 1 /\ 2:r4 = 0)
-- 
2.54.0

[PATCH for-v7.5 02/14] Documentation/litmus-tests: Add SRCU fastpath scan-before-anchor test

[Joel Fernandes]

From: Kunwu Chan <kunwu.chan@gmail.com>

If the synchronize_srcu_atomic() fastpath instead places its lock scan
before the grace-period anchor, the scan can miss a reader whose
increment was already visible before the anchor.  That reader already
existed when the grace period started, so completing the grace period
without waiting for it would violate the SRCU grace-period guarantee.

This litmus test models the reversed ordering, with the lock scan placed
before the grace-period anchor.  "seq" models the grace-period anchor in
->srcu_gp_seq and "ctr" models the per-CPU ->srcu_ctrs[].srcu_locks
counter.  P0 scans the lock counter before writing the anchor, with an
smp_mb() between them.  P1 models the reader-side counter increment.
P2 models an observer that sees the reader's increment before seeing
the anchor.

The same outcome is allowed with this ordering, and herd7 reports
"Sometimes".  The litmus-tests README is also updated to describe both
SRCU fastpath tests.

Tested with herd7 7.58 using linux-kernel.cfg.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 Documentation/litmus-tests/README             | 19 +++++++
 .../SRCU-fastpath-scan-before-anchor.litmus   | 52 +++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100644 Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus

diff --git a/Documentation/litmus-tests/README b/Documentation/litmus-tests/README
index 6c666f3422ea..4d4ec9c6f2cc 100644
--- a/Documentation/litmus-tests/README
+++ b/Documentation/litmus-tests/README
@@ -78,3 +78,22 @@ RCU+sync+read.litmus
 RCU+sync+free.litmus
     Both the above litmus tests demonstrate the RCU grace period guarantee
     that an RCU read-side critical section can never span a grace period.
+
+SRCU (/srcu directory)
+----------------------
+
+SRCU-fastpath-anchor-before-scan.litmus
+    This models the synchronize_srcu_atomic() fastpath with the
+    grace-period anchor ordered before the lock-counter scan.  This
+    ordering prevents readers that existed before the grace period
+    from being missed by the scan.  See
+    SRCU-fastpath-scan-before-anchor.litmus for the reversed
+    ordering.
+
+SRCU-fastpath-scan-before-anchor.litmus
+    This models the synchronize_srcu_atomic() fastpath with the
+    lock-counter scan ordered before the grace-period anchor.  This
+    permits the scan to miss readers that existed before the grace
+    period, violating the SRCU grace-period guarantee.  See
+    SRCU-fastpath-anchor-before-scan.litmus for the opposite
+    ordering.
diff --git a/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus b/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus
new file mode 100644
index 000000000000..7df0641b5470
--- /dev/null
+++ b/Documentation/litmus-tests/srcu/SRCU-fastpath-scan-before-anchor.litmus
@@ -0,0 +1,52 @@
+C SRCU-fastpath-scan-before-anchor
+
+(*
+ * Result: Sometimes
+ *
+ * If the synchronize_srcu_atomic() fastpath instead places its lock
+ * scan before the grace-period anchor, the scan can miss a reader whose
+ * increment was already visible before the anchor.  That reader already
+ * existed when the grace period started, so completing the grace period
+ * without waiting for it would violate the SRCU grace-period guarantee.
+ *
+ * This litmus test models the reversed ordering, with the lock scan
+ * placed before the grace-period anchor.  "seq" models the grace-period
+ * anchor in ->srcu_gp_seq and "ctr" models the per-CPU
+ * ->srcu_ctrs[].srcu_locks counter.  P0 scans the lock counter before
+ * writing the anchor, with an smp_mb() between them.  P1 models the
+ * reader-side counter increment.  P2 models an observer that sees the
+ * reader's increment before seeing the anchor.
+ *
+ * The same outcome is allowed with this ordering, and herd7 reports
+ * "Sometimes".  See SRCU-fastpath-anchor-before-scan.litmus for the
+ * opposite ordering, which forbids this outcome.
+ *)
+
+{}
+
+P0(int *seq, int *ctr)
+{
+	int r2;
+
+	r2 = READ_ONCE(*ctr);
+	smp_mb();
+	WRITE_ONCE(*seq, 1);
+}
+
+P1(int *ctr)
+{
+	WRITE_ONCE(*ctr, 1);
+}
+
+P2(int *seq, int *ctr)
+{
+	int r3;
+	int r4;
+
+	r3 = READ_ONCE(*ctr);
+	smp_mb();
+	r4 = READ_ONCE(*seq);
+}
+
+filter (0:r2 = 0)
+exists (2:r3 = 1 /\ 2:r4 = 0)
-- 
2.54.0

[PATCH for-v7.5 03/14] kasan: Avoid an unrelated RCU warning in the callback test

[Joel Fernandes]

rcu_uaf() passes a false protection condition to
rcu_dereference_protected(), which produces a suspicious-RCU-usage
warning before the intended KASAN failure occurs.

The pointer refers to a fresh allocation that has not been published to
readers. State that protection condition explicitly, leaving the intended
tests intact.

Test:
Built with CONFIG_KASAN_KUNIT_TEST=m
Booted Qemu guest (8 CPUs), ran modprobe kasan_test filtered to kasan.*rcu_uaf*

Signed-off-by: Joel Fernandes <jfern@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
---
 mm/kasan/kasan_test_c.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/kasan/kasan_test_c.c b/mm/kasan/kasan_test_c.c
index b9e167ed5be3..4e23765d722c 100644
--- a/mm/kasan/kasan_test_c.c
+++ b/mm/kasan/kasan_test_c.c
@@ -967,8 +967,9 @@ static void rcu_uaf(struct kunit *test)
 	ptr = kmalloc_obj(struct kasan_rcu_info);
 	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr);
 
+	/* The freshly allocated object has not been published to readers. */
 	global_rcu_ptr = rcu_dereference_protected(
-				(struct kasan_rcu_info __rcu *)ptr, NULL);
+				(struct kasan_rcu_info __rcu *)ptr, true);
 
 	KUNIT_EXPECT_KASAN_FAIL_READ(test,
 			call_rcu(&global_rcu_ptr->rcu, rcu_uaf_reclaim);
-- 
2.54.0

[PATCH for-v7.5 04/14] rcutorture: Fix busted_srcud crash on module unload

[Joel Fernandes]

From: Fu Wenxin <fuwenxin@linux.alibaba.com>

Commit e4ed20c1603c ("rcutorture: Exercise DEFINE_STATIC_SRCU_FAST() and
init_srcu_struct_fast()") split SRCU torture initialization into two
helpers: srcu_torture_init(), which now only points srcu_ctlp at the
static srcu_ctl/srcu_ctlf/srcu_ctlfud structures, and the new
srcud_torture_init(), which calls init_srcu_struct(&srcu_ctld) for the
dynamically allocated srcu_ctld.

That commit updated srcud_ops to use srcud_torture_init(), but left
busted_srcud_ops pairing .init = srcu_torture_init (static srcu_ctl) with
.cleanup = srcu_torture_cleanup (which unconditionally calls
cleanup_srcu_struct(&srcu_ctld)). As a result srcu_ctld is never
initialized when running torture_type=busted_srcud, yet it is torn down
at module unload.

cleanup_srcu_struct() then dereferences ssp->srcu_sup, which is still
NULL for the zero-initialized srcu_ctld, and faults on the ->lock field
accessed by raw_spin_lock_irq_rcu_node():

  Unable to handle kernel paging request at virtual address dfff80000000001b
  ...
  el1_abort -> do_translation_fault -> do_bad_area -> die_kernel_fault

(The fault address is the KASAN shadow of NULL + offsetof(srcu_usage,
lock) == 0xd8.)

This is trivially reproducible:

  modprobe rcutorture torture_type=busted_srcud
  sleep 45
  rmmod rcutorture    # panic here

Fix it by making busted_srcud_ops initialize the same srcu_ctld that its
cleanup handler tears down, matching srcud_ops.

Fixes: e4ed20c1603c ("rcutorture: Exercise DEFINE_STATIC_SRCU_FAST() and init_srcu_struct_fast()")
Signed-off-by: Fu Wenxin <fuwenxin@linux.alibaba.com>
Reviewed-by: Joel Fernandes <jfern@kernel.org>
Link: https://lore.kernel.org/r/20261019023644.2750352-1-fuwenxin@linux.alibaba.com
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcutorture.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 93ca7f06371e..e88573b9b7c6 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1055,7 +1055,7 @@ static struct rcu_torture_ops srcud_ops = {
 /* As above, but broken due to inappropriate reader extension. */
 static struct rcu_torture_ops busted_srcud_ops = {
 	.ttype		= SRCU_FLAVOR,
-	.init		= srcu_torture_init,
+	.init		= srcud_torture_init,
 	.cleanup	= srcu_torture_cleanup,
 	.readlock	= srcu_torture_read_lock,
 	.read_delay	= rcu_read_delay,
-- 
2.54.0

[PATCH for-v7.5 05/14] rcutorture: Fix suspicious rcu_dereference_check() usage for trivial-preempt

[Joel Fernandes]

From: Zqiang <qiang.zhang@linux.dev>

Running the trivial-preempt flavor triggers a lockdep "suspicious RCU
usage" warning from rcu_torture_one_read_start():

WARNING: suspicious RCU usage
kernel/rcu/rcutorture.c:2581 suspicious rcu_dereference_check() usage!
...
lockdep_rcu_suspicious
rcu_torture_one_read_start
rcu_torture_one_read
rcu_torture_reader

The trivial-preempt flavor .readlock_held is set to torture_readlock_not_held(),
which check RCU-bh/RCU-sched critical sections. however, it's read lock
just increments current->rcu_trivial_preempt_nesting, it's not also RCU
read critical sections. the rcu_dereference_check() condition is therefore
false, and lockdep report warnings. this commit therefore drop .readlock_held
callback to avoid this lockdep warnings.

Fixes: c6f4e552e1ea ("rcutorture: Add a textbook-style trivial preemptible RCU")
Signed-off-by: Zqiang <qiang.zhang@linux.dev>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Link: https://lore.kernel.org/r/20261020094907.15781-1-qiang.zhang@linux.dev
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcutorture.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index e88573b9b7c6..d60d17d14f1c 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1164,7 +1164,6 @@ static struct rcu_torture_ops trivial_preempt_ops = {
 	.readlock	= rcu_torture_read_lock_trivial_preempt,
 	.read_delay	= rcu_read_delay,  // just reuse rcu's version.
 	.readunlock	= rcu_torture_read_unlock_trivial_preempt,
-	.readlock_held	= torture_readlock_not_held,
 	.get_gp_seq	= rcu_no_completed,
 	.sync		= synchronize_rcu_trivial_preempt,
 	.exp_sync	= synchronize_rcu_trivial_preempt,
-- 
2.54.0

[PATCH for-v7.5 06/14] rcutorture: Make tasks-tracing support duplicate call testing

[Joel Fernandes]

From: Zqiang <qiang.zhang@linux.dev>

Currently, Tasks-tracing has been reimplemented by SRCU-fast, it supports
the detection of duplicate calls with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y.
This commit therefore set tasks_tracing_ops.debug_objects to 1, enabling
rcutorture's duplicate-call test for the tasks-tracing flavor.

[Joel: On this branch, object_debug=1 already invokes the duplicate-call
test. This records the flavor's duplicate-detection capability rather
than adding a runtime test gate.]

Signed-off-by: Zqiang <qiang.zhang@linux.dev>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Link: https://lore.kernel.org/r/20261023094039.29696-1-qiang.zhang@linux.dev
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcutorture.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index d60d17d14f1c..cbe38aa7ee68 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1313,6 +1313,7 @@ static struct rcu_torture_ops tasks_tracing_ops = {
 	.irq_capable	= 1,
 	.slow_gps	= 1,
 	.nmi_capable	= 1,
+	.debug_objects	= 1,
 	.name		= "tasks-tracing"
 };
 
-- 
2.54.0

[PATCH for-v7.5 07/14] rcutorture: Avoid create fwd_prog_tasks for tasks-rude and srcu-atomic

[Joel Fernandes]

From: Zqiang <qiang.zhang@linux.dev>

When running the tasks-rude and srcu-atomic tests, the fwd_prog_tasks
kthread is created because the default value of fwd_progress is 1.
however, the two RCU flavors not support ->call():

1. tasks-rude: commit b428a9de9bba ("rcutorture: Stop testing RCU Tasks
   Rude asynchronous APIs") removed .call and .cb_barrier from tasks-rude,
   but left the .cbflood_max = 50000.

2. srcu-atomic: srcu_torture_init_forbidden_apis() sets ->call to NULL
   for SRCU_READ_FLAVOR_ATOMIC runs, but leaves ->cbflood_max untouched.

In both cases, the non-zero ->cbflood_max fools the check above into
creating the fwd_prog_tasks kthreads, even though those kthreads cannot
possibly carry out any forward-progress test:

1. The rcu_torture_fwd_prog_cr() returns immediately due to cur_ops->call
   is NULL:

[   68.022674] rcu_torture_fwd_prog_cr: No ->call(), so no forward-progress test 0
[  129.462046] rcu_torture_fwd_prog_cr: No ->call(), so no forward-progress test 0
[  190.902123] rcu_torture_fwd_prog_cr: No ->call(), so no forward-progress test 0
[  252.343809] rcu_torture_fwd_prog_cr: No ->call(), so no forward-progress test 0

2. The rcu_torture_fwd_prog_nr() is never even invoked, because its caller
   requires cur_ops->stall_dur, and neither tasks-rude nor srcu-atomic
   provide ->stall_dur (only rcu_ops does).

Therefore, creating fwd_prog_tasks kthreads is pointless, this commit adds
->call and ->cb_barrier to the forward-progress check, rejecting flavors
that lack the cbflood_max set and a positive ->stall_dur.

simple verification output:

[    3.062959][    T1] srcu-torture: rcu_torture_fwd_prog_init: Disabled, unsupported by RCU flavor under test
[    2.922323][    T1] tasks-rude-torture: rcu_torture_fwd_prog_init: Disabled, unsupported by RCU flavor under test

Suggested-by: Joel Fernandes <jfern@kernel.org>
Signed-off-by: Zqiang <qiang.zhang@linux.dev>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Link: https://lore.kernel.org/r/20261023093716.25717-1-qiang.zhang@linux.dev
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcutorture.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index cbe38aa7ee68..924705b7759d 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -4007,9 +4007,8 @@ static int __init rcu_torture_fwd_prog_init(void)
 	} else if (fwd_progress < 0) {
 		fwd_progress = nr_cpu_ids;
 	}
-	if ((!cur_ops->sync && !cur_ops->call) ||
-	    (!cur_ops->cbflood_max && (!cur_ops->stall_dur || cur_ops->stall_dur() <= 0)) ||
-	    cur_ops == &rcu_busted_ops) {
+	if (!cur_ops->sync || ((!cur_ops->call || !cur_ops->cb_barrier || !cur_ops->cbflood_max) &&
+	    (!cur_ops->stall_dur || cur_ops->stall_dur() <= 0))) {
 		VERBOSE_TOROUT_STRING("rcu_torture_fwd_prog_init: Disabled, unsupported by RCU flavor under test");
 		fwd_progress = 0;
 		return 0;
-- 
2.54.0

[PATCH for-v7.5 08/14] rcuscale: Consolidate per-writer arrays into struct writer_state

[Joel Fernandes]

The writer kthreads keep their state in five parallel arrays indexed by
writer number: writer_tasks, writer_durations, writer_n_durations,
writer_done and writer_freelists.  Each must be allocated, NULL-checked,
freed and cleared separately, and the cleanup path needs several
guards to cope with any one of them being missing.

Collapse them into a single array of struct writer_state, holding the
task pointer, the duration buffer, the measurement count, the done flag
and the existing struct writer_freelist.

This reduces initialization to one allocation and one out-of-memory
check, and cleanup to a single kfree().  As a side effect, this also
fixes a leak: cleanup only ran its frees under "if (writer_tasks)", so
if the writer_tasks allocation failed while a later allocation (for
example writer_durations or writer_n_durations) still succeeded, those
already-allocated arrays were never freed.  With a single allocation,
that partial-allocation state can no longer occur.

Assisted-by: Codex
Fixes: 8704baab9bc8 ("rcutorture: Add RCU grace-period performance tests")
Signed-off-by: Joel Fernandes <jfern@kernel.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Tested-by: Bradley Morgan <brads@mainlining.org> # Power10
Acked-by: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/rcuscale.c | 125 ++++++++++++++++++------------------------
 1 file changed, 54 insertions(+), 71 deletions(-)

diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
index 38c75c37a07c..41abf05a14d9 100644
--- a/kernel/rcu/rcuscale.c
+++ b/kernel/rcu/rcuscale.c
@@ -115,17 +115,21 @@ struct writer_freelist {
 	struct writer_mblock *ws_mblocks;
 };
 
+struct writer_state {
+	struct writer_freelist wst_fl;
+	struct task_struct *wst_task;
+	u64 *wst_durations;
+	int wst_n_durations;
+	bool wst_done;
+};
+
 static int nrealreaders;
 static int nrealwriters;
 static int nrealexp;
-static struct task_struct **writer_tasks;
 static struct task_struct **reader_tasks;
 static struct task_struct **exp_tasks;
 
-static u64 **writer_durations;
-static bool *writer_done;
-static struct writer_freelist *writer_freelists;
-static int *writer_n_durations;
+static struct writer_state *writer_states;
 static atomic_t n_rcu_scale_reader_started;
 static atomic_t n_rcu_scale_writer_started;
 static atomic_t n_rcu_scale_writer_finished;
@@ -504,9 +508,9 @@ static struct writer_mblock *rcu_scale_alloc(long me)
 	struct writer_freelist *wflp;
 	struct writer_mblock *wmbp;
 
-	if (WARN_ON_ONCE(!writer_freelists))
+	if (WARN_ON_ONCE(!writer_states))
 		return NULL;
-	wflp = &writer_freelists[me];
+	wflp = &writer_states[me].wst_fl;
 	if (llist_empty(&wflp->ws_lhp)) {
 		// ->ws_lhp is private to its rcu_scale_writer task.
 		wmbp = container_of(llist_del_all(&wflp->ws_lhg), struct writer_mblock, wmb_node);
@@ -560,8 +564,9 @@ rcu_scale_writer(void *arg)
 	u64 t;
 	DEFINE_TORTURE_RANDOM(tr);
 	u64 *wdp;
-	u64 *wdpp = writer_durations[me];
-	struct writer_freelist *wflp = &writer_freelists[me];
+	struct writer_state *wsp = &writer_states[me];
+	u64 *wdpp = wsp->wst_durations;
+	struct writer_freelist *wflp = &wsp->wst_fl;
 	struct writer_mblock *wmbp = NULL;
 
 	VERBOSE_SCALEOUT_STRING("rcu_scale_writer task started");
@@ -631,7 +636,7 @@ rcu_scale_writer(void *arg)
 			started = true;
 		if (!done && i >= MIN_MEAS && time_after(jiffies, jdone)) {
 			done = true;
-			WRITE_ONCE(writer_done[me], true);
+			WRITE_ONCE(wsp->wst_done, true);
 			sched_set_normal(current, 0);
 			pr_alert("%s%s rcu_scale_writer %ld has %d measurements\n",
 				 scale_type, SCALE_FLAG, me, MIN_MEAS);
@@ -649,7 +654,7 @@ rcu_scale_writer(void *arg)
 						cur_ops->get_gp_seq();
 				}
 				if (shutdown_secs) {
-					writer_tasks[me] = NULL;
+					wsp->wst_task = NULL;
 					smp_mb(); /* Assign before wake. */
 					rcu_scale_cleanup();
 					kernel_power_off();
@@ -665,10 +670,10 @@ rcu_scale_writer(void *arg)
 
 			if (!atomic_xchg(&dumped, 1)) {
 				for (i = 0; i < nrealwriters; i++) {
-					if (writer_done[i])
+					if (writer_states[i].wst_done)
 						continue;
 					pr_info("%s: Task %ld flags writer %d:\n", __func__, me, i);
-					sched_show_task(writer_tasks[i]);
+					sched_show_task(writer_states[i].wst_task);
 				}
 				if (cur_ops->stats)
 					cur_ops->stats();
@@ -676,7 +681,7 @@ rcu_scale_writer(void *arg)
 		}
 		if (!selfreport && time_after(jiffies, jdone + HZ * (70 + me))) {
 			pr_info("%s: Writer %ld self-report: started %d done %d/%d->%d i %d jdone %lu.\n",
-				__func__, me, started, done, writer_done[me], atomic_read(&n_rcu_scale_writer_finished), i, jiffies - jdone);
+				__func__, me, started, done, wsp->wst_done, atomic_read(&n_rcu_scale_writer_finished), i, jiffies - jdone);
 			selfreport = true;
 		}
 		if (gp_succeeded && started && !alldone && i < MAX_MEAS - 1)
@@ -687,7 +692,7 @@ rcu_scale_writer(void *arg)
 		rcu_scale_free(wmbp);
 		cur_ops->gp_barrier();
 	}
-	writer_n_durations[me] = i_max + 1;
+	wsp->wst_n_durations = i_max + 1;
 	torture_kthread_stopping("rcu_scale_writer");
 	return 0;
 }
@@ -1031,13 +1036,11 @@ rcu_scale_cleanup(void)
 		reader_tasks = NULL;
 	}
 
-	if (writer_tasks) {
+	if (writer_states) {
 		for (i = 0; i < nrealwriters; i++) {
 			torture_stop_kthread(rcu_scale_writer,
-					     writer_tasks[i]);
-			if (!writer_n_durations)
-				continue;
-			j = writer_n_durations[i];
+					     writer_states[i].wst_task);
+			j = writer_states[i].wst_n_durations;
 			pr_alert("%s%s writer %d gps: %d\n",
 				 scale_type, SCALE_FLAG, i, j);
 			ngps += j;
@@ -1051,14 +1054,12 @@ rcu_scale_cleanup(void)
 			 rcuscale_seq_diff(b_rcu_gp_test_finished,
 					   b_rcu_gp_test_started));
 		for (i = 0; i < nrealwriters; i++) {
-			if (!writer_durations)
-				break;
-			if (!writer_n_durations)
-				continue;
-			wdpp = writer_durations[i];
+			struct writer_state *wsp = &writer_states[i];
+
+			wdpp = wsp->wst_durations;
 			if (!wdpp)
 				continue;
-			for (j = 0; j < writer_n_durations[i]; j++) {
+			for (j = 0; j < wsp->wst_n_durations; j++) {
 				wdp = &wdpp[j];
 				pr_alert("%s%s %4d writer-duration: %5d %llu\n",
 					scale_type, SCALE_FLAG,
@@ -1066,34 +1067,23 @@ rcu_scale_cleanup(void)
 				if (j % 100 == 0)
 					schedule_timeout_uninterruptible(1);
 			}
-			kfree(writer_durations[i]);
-			if (writer_freelists) {
+			kfree(wdpp);
+			if (wsp->wst_fl.ws_mblocks) {
 				int ctr = 0;
 				struct llist_node *llnp;
-				struct writer_freelist *wflp = &writer_freelists[i];
 
-				if (wflp->ws_mblocks) {
-					llist_for_each(llnp, wflp->ws_lhg.first)
-						ctr++;
-					llist_for_each(llnp, wflp->ws_lhp.first)
-						ctr++;
-					WARN_ONCE(ctr != gp_async_max,
-						  "%s: ctr = %d gp_async_max = %d\n",
-						  __func__, ctr, gp_async_max);
-					kfree(wflp->ws_mblocks);
-				}
+				llist_for_each(llnp, wsp->wst_fl.ws_lhg.first)
+					ctr++;
+				llist_for_each(llnp, wsp->wst_fl.ws_lhp.first)
+					ctr++;
+				WARN_ONCE(ctr != gp_async_max,
+					  "%s: ctr = %d gp_async_max = %d\n",
+					  __func__, ctr, gp_async_max);
+				kfree(wsp->wst_fl.ws_mblocks);
 			}
 		}
-		kfree(writer_tasks);
-		writer_tasks = NULL;
-		kfree(writer_durations);
-		writer_durations = NULL;
-		kfree(writer_n_durations);
-		writer_n_durations = NULL;
-		kfree(writer_done);
-		writer_done = NULL;
-		kfree(writer_freelists);
-		writer_freelists = NULL;
+		kfree(writer_states);
+		writer_states = NULL;
 	}
 
 	/* Do torture-type-specific cleanup operations.  */
@@ -1187,36 +1177,29 @@ rcu_scale_init(void)
 				goto unwind;
 		}
 	}
-	writer_tasks = kzalloc_objs(writer_tasks[0], nrealwriters);
-	writer_durations = kzalloc_objs(*writer_durations, nrealwriters);
-	writer_n_durations = kzalloc_objs(*writer_n_durations, nrealwriters);
-	writer_done = kzalloc_objs(writer_done[0], nrealwriters);
-	if (gp_async) {
-		if (gp_async_max <= 0) {
-			pr_warn("%s: gp_async_max = %d must be greater than zero.\n",
-				__func__, gp_async_max);
-			WARN_ON_ONCE(IS_BUILTIN(CONFIG_RCU_TORTURE_TEST));
-			firsterr = -EINVAL;
-			goto unwind;
-		}
-		writer_freelists = kzalloc_objs(writer_freelists[0],
-						nrealwriters);
+	writer_states = kzalloc_objs(writer_states[0], nrealwriters);
+	if (gp_async && gp_async_max <= 0) {
+		pr_warn("%s: gp_async_max = %d must be greater than zero.\n",
+			__func__, gp_async_max);
+		WARN_ON_ONCE(IS_BUILTIN(CONFIG_RCU_TORTURE_TEST));
+		firsterr = -EINVAL;
+		goto unwind;
 	}
-	if (!writer_tasks || !writer_durations || !writer_n_durations || !writer_done ||
-	    (gp_async && !writer_freelists)) {
+	if (!writer_states) {
 		SCALEOUT_ERRSTRING("out of memory");
 		firsterr = -ENOMEM;
 		goto unwind;
 	}
 	for (i = 0; i < nrealwriters; i++) {
-		writer_durations[i] =
-			kzalloc_objs(*writer_durations[i], MAX_MEAS);
-		if (!writer_durations[i]) {
+		struct writer_state *wsp = &writer_states[i];
+
+		wsp->wst_durations = kzalloc_objs(*wsp->wst_durations, MAX_MEAS);
+		if (!wsp->wst_durations) {
 			firsterr = -ENOMEM;
 			goto unwind;
 		}
-		if (writer_freelists) {
-			struct writer_freelist *wflp = &writer_freelists[i];
+		if (gp_async) {
+			struct writer_freelist *wflp = &wsp->wst_fl;
 
 			init_llist_head(&wflp->ws_lhg);
 			init_llist_head(&wflp->ws_lhp);
@@ -1234,7 +1217,7 @@ rcu_scale_init(void)
 			}
 		}
 		firsterr = torture_create_kthread(rcu_scale_writer, (void *)i,
-						  writer_tasks[i]);
+						  wsp->wst_task);
 		if (torture_init_error(firsterr))
 			goto unwind;
 	}
-- 
2.54.0

[PATCH for-v7.5 09/14] rcuscale: Add srcua scale type

[Joel Fernandes]

From: Kunwu Chan <kunwu.chan@gmail.com>

Add support for scaling atomic SRCU grace periods using
srcu_read_lock_atomic()/srcu_read_unlock_atomic() and
synchronize_srcu_atomic().

Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/rcuscale.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

diff --git a/kernel/rcu/rcuscale.c b/kernel/rcu/rcuscale.c
index 41abf05a14d9..687249d686a5 100644
--- a/kernel/rcu/rcuscale.c
+++ b/kernel/rcu/rcuscale.c
@@ -297,6 +297,44 @@ static struct rcu_scale_ops srcud_ops = {
 	.name		= "srcud"
 };
 
+static struct srcu_struct srcua;
+
+static int srcu_scale_atomic_read_lock(void)
+{
+	return srcu_read_lock_atomic(srcu_ctlp);
+}
+
+static void srcu_scale_atomic_read_unlock(int idx)
+{
+	srcu_read_unlock_atomic(srcu_ctlp, idx);
+}
+
+static void srcu_scale_atomic_synchronize(void)
+{
+	synchronize_srcu_atomic(srcu_ctlp);
+}
+
+static void srcu_atomic_scale_init(void)
+{
+	srcu_ctlp = &srcua;
+	init_srcu_struct_atomic(srcu_ctlp);
+}
+
+static struct rcu_scale_ops srcua_ops = {
+	.ptype		= SRCU_FLAVOR,
+	.init		= srcu_atomic_scale_init,
+	.cleanup	= srcu_sync_scale_cleanup,
+	.readlock	= srcu_scale_atomic_read_lock,
+	.readunlock	= srcu_scale_atomic_read_unlock,
+	.get_gp_seq	= srcu_scale_completed,
+	.gp_diff	= rcu_seq_diff,
+	.exp_completed	= srcu_scale_completed,
+	.sync		= srcu_scale_atomic_synchronize,
+	.exp_sync	= srcu_scale_atomic_synchronize,
+	.stats		= srcu_scale_stats,
+	.name		= "srcua"
+};
+
 #ifdef CONFIG_TASKS_RCU
 
 /*
@@ -1100,7 +1138,8 @@ rcu_scale_init(void)
 	long i;
 	long j;
 	static struct rcu_scale_ops *scale_ops[] = {
-		&rcu_ops, &srcu_ops, &srcud_ops, TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
+		&rcu_ops, &srcu_ops, &srcua_ops, &srcud_ops,
+		TASKS_OPS TASKS_RUDE_OPS TASKS_TRACING_OPS
 	};
 
 	if (!torture_init_begin(scale_type, verbose))
-- 
2.54.0

[PATCH for-v7.5 10/14] scftorture: Reject empty workloads before starting helpers

[Joel Fernandes]

A module loaded with nthreads=0 starts its statistics task, but cleanup
skips stopping that task. Unloading the module then leaves it executing
freed module text. Reject a zero thread count before starting helpers;
negative counts still select the online CPU count.

Also, when built as a module with reschedule-only, this leaves the
selector empty because resched_cpu() is unavailable to modules. It warns
while selecting an operation, invokes no callbacks, and can nevertheless
report SUCCESS.  Reject an empty effective selector after filtering
unsupported operations.

Both failures were reproduced in a guest without the patches.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/scftorture.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/scftorture.c b/kernel/scftorture.c
index 603b2dba1536..ff9a8271a464 100644
--- a/kernel/scftorture.c
+++ b/kernel/scftorture.c
@@ -588,6 +588,12 @@ static int __init scf_torture_init(void)
 
 	scftorture_print_module_parms("Start of test");
 
+	if (!nthreads) {
+		SCFTORTOUT_ERRSTRING("nthreads must be nonzero");
+		firsterr = -EINVAL;
+		goto unwind;
+	}
+
 	if (weight_resched <= 0 &&
 	    weight_single <= 0 && weight_single_rpc <= 0 && weight_single_wait <= 0 &&
 	    weight_many <= 0 && weight_many_wait <= 0 &&
@@ -636,6 +642,11 @@ static int __init scf_torture_init(void)
 	scf_sel_add(weight_many_wait1, SCF_PRIM_MANY, true);
 	scf_sel_add(weight_all1, SCF_PRIM_ALL, false);
 	scf_sel_add(weight_all_wait1, SCF_PRIM_ALL, true);
+	if (!scf_sel_array_len) {
+		SCFTORTOUT_ERRSTRING("no supported operations selected");
+		firsterr = -EINVAL;
+		goto unwind;
+	}
 	scf_sel_dump();
 
 	if (onoff_interval > 0) {
-- 
2.54.0

[PATCH for-v7.5 11/14] scftorture: Protect the initial affinity check from hotplug

[Joel Fernandes]

An invoker binds to its target CPU, sleeps for the holdoff interval, and
then warns if it is running elsewhere. CPU hotplug can offline the target
during that sleep and legitimately move the invoker. A built-in test with
matching worker and hotplug holdoffs reproduces this warning.

Check the initial binding while holding cpus_read_lock(), before sleeping.
If the target is already offline, let the existing offline/rebind path
handle it instead of warning about an impossible initial binding. Release
the read lock before the holdoff and test loop so hotplug remains tested.

Signed-off-by: Joel Fernandes <jfern@kernel.org>

Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/scftorture.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/kernel/scftorture.c b/kernel/scftorture.c
index ff9a8271a464..64ee2c5cf51c 100644
--- a/kernel/scftorture.c
+++ b/kernel/scftorture.c
@@ -479,19 +479,24 @@ static int scftorture_invoker(void *arg)
 
 	VERBOSE_SCFTORTOUT("scftorture_invoker %d: task started", scfp->cpu);
 	cpu = scfp->cpu % nr_cpu_ids;
-	WARN_ON_ONCE(set_cpus_allowed_ptr(current, cpumask_of(cpu)));
+	cpus_read_lock();
+	if (cpu_online(cpu)) {
+		WARN_ON_ONCE(set_cpus_allowed_ptr(current, cpumask_of(cpu)));
+		/* Check affinity before hotplug can move this task away. */
+		curcpu = raw_smp_processor_id();
+		WARN_ONCE(curcpu != cpu,
+			  "%s: Wanted CPU %d, running on %d, nr_cpu_ids = %d\n",
+			  __func__, scfp->cpu, curcpu, nr_cpu_ids);
+	} else {
+		was_offline = true;
+	}
+	cpus_read_unlock();
 	set_user_nice(current, MAX_NICE);
 	if (holdoff)
 		schedule_timeout_interruptible(holdoff * HZ);
 
 	VERBOSE_SCFTORTOUT("scftorture_invoker %d: Waiting for all SCF torturers from cpu %d", scfp->cpu, raw_smp_processor_id());
 
-	// Make sure that the CPU is affinitized appropriately during testing.
-	curcpu = raw_smp_processor_id();
-	WARN_ONCE(curcpu != cpu,
-		  "%s: Wanted CPU %d, running on %d, nr_cpu_ids = %d\n",
-		  __func__, scfp->cpu, curcpu, nr_cpu_ids);
-
 	if (atomic_dec_return(&n_started))
 		while (atomic_read_acquire(&n_started)) {
 			if (torture_must_stop()) {
-- 
2.54.0

[PATCH for-v7.5 12/14] rcutorture: Enable RCU_BOOST where supported

[Joel Fernandes]

From: "Paul E. McKenney" <paulmck@kernel.org>

Currently, RCU_BOOST depends on RCU_EXPERT, and is therefore often
disabled in stress-test such as those run by syzbot.  This means that if
syzbot runs a test that overloads CPUs on kernels built with PREEMPT_RCU,
RCU CPU stall warnings are not unexpected.  What happens is that a
low-priority task in an RCU read-side critical section is preempted,
cannot be priority boosted, and stays that way for more than 21 seconds.

This commit therefore drops RCU_EXPERT from the RCU_BOOST "depends on"
line and removes the prompt, thus directly driving RCU_BOOST from the
kernel configuration so as to enable it anywhere it is supported and
might help.

This commit is also marked experimental because neither Ubuntu, Fedora,
nor CentOS enable RCU_BOOST.  This suggests less testing of RCU priority
boosting than one might wish.

Reported-by: Andrew Morton <akpm@linux-foundation.org>
Closes: https://lore.kernel.org/all/20260805122952.6ae38af69a0457779cb44359@linux-foundation.org/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/rcu/Kconfig                                     | 10 ++++------
 tools/testing/selftests/rcutorture/configs/rcu/NOCB01  |  1 -
 tools/testing/selftests/rcutorture/configs/rcu/NOCB02  |  1 -
 tools/testing/selftests/rcutorture/configs/rcu/TREE01  |  1 -
 tools/testing/selftests/rcutorture/configs/rcu/TREE02  |  1 -
 tools/testing/selftests/rcutorture/configs/rcu/TREE08  |  1 -
 .../testing/selftests/rcutorture/configs/rcuscale/TINY |  1 -
 .../selftests/rcutorture/configs/rcuscale/TRACE01      |  1 -
 .../testing/selftests/rcutorture/configs/rcuscale/TREE |  1 -
 .../selftests/rcutorture/configs/rcuscale/TREE54       |  1 -
 .../selftests/rcutorture/configs/refscale/NOPREEMPT    |  1 -
 .../selftests/rcutorture/configs/refscale/PREEMPT      |  1 -
 .../testing/selftests/rcutorture/configs/refscale/TINY |  1 -
 13 files changed, 4 insertions(+), 18 deletions(-)

diff --git a/kernel/rcu/Kconfig b/kernel/rcu/Kconfig
index b6d3597758fc..2d0132e0a90f 100644
--- a/kernel/rcu/Kconfig
+++ b/kernel/rcu/Kconfig
@@ -238,17 +238,15 @@ config RCU_FANOUT_LEAF
 	  Take the default if unsure.
 
 config RCU_BOOST
-	bool "Enable RCU priority boosting"
-	depends on (RT_MUTEXES && PREEMPT_RCU && RCU_EXPERT) || PREEMPT_RT
-	default y if PREEMPT_RT
+	bool
+	default y if (RT_MUTEXES && PREEMPT_RCU) || PREEMPT_RT
 	help
 	  This option boosts the priority of preempted RCU readers that
 	  block the current preemptible RCU grace period for too long.
 	  This option also prevents heavy loads from blocking RCU
-	  callback invocation.
+	  callback invocation.  It is now automatically enabled in
+	  any kernel that can benefit from it and that can support it.
 
-	  Say Y here if you are working with real-time apps or heavy loads
-	  Say N here if you are unsure.
 
 config RCU_BOOST_DELAY
 	int "Milliseconds to delay boosting after RCU grace-period start"
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/NOCB01 b/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
index bbe6d28210ab..91ca7e8b5725 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/NOCB01
@@ -14,7 +14,6 @@ CONFIG_RCU_FANOUT_LEAF=2
 CONFIG_RCU_NOCB_CPU=y
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/NOCB02 b/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
index 4c2b8cd6d8fd..ed3f3f7b29e7 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
+++ b/tools/testing/selftests/rcutorture/configs/rcu/NOCB02
@@ -14,7 +14,6 @@ CONFIG_RCU_FANOUT_LEAF=2
 CONFIG_RCU_NOCB_CPU=y
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
index 54b1600c7eb5..be5bf69a1b60 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
@@ -10,6 +10,5 @@ CONFIG_RCU_TRACE=y
 CONFIG_HOTPLUG_CPU=y
 CONFIG_RCU_NOCB_CPU=y
 CONFIG_DEBUG_LOCK_ALLOC=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE02 b/tools/testing/selftests/rcutorture/configs/rcu/TREE02
index 2871ee599891..36b9228ae703 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE02
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE02
@@ -13,7 +13,6 @@ CONFIG_RCU_FANOUT_LEAF=3
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=y
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_RCU_EXPERT=y
 CONFIG_DEBUG_OBJECTS=y
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE08 b/tools/testing/selftests/rcutorture/configs/rcu/TREE08
index 8b561355b9ef..ad7d7f873e1e 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE08
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE08
@@ -13,7 +13,6 @@ CONFIG_RCU_FANOUT_LEAF=2
 CONFIG_RCU_NOCB_CPU=y
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_EQS_DEBUG=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TINY b/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
index c1a7d38823fb..f611b69f1189 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TINY
@@ -11,7 +11,6 @@ CONFIG_NO_HZ_FULL=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01 b/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
index f3f1368b8386..69f5764159df 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TRACE01
@@ -11,7 +11,6 @@ CONFIG_NO_HZ_FULL=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
index b10706fd03a4..5122cf08d5db 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE
@@ -12,7 +12,6 @@ CONFIG_HIBERNATION=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54 b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
index 9f83e5372796..08d30edb3b99 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/TREE54
@@ -15,7 +15,6 @@ CONFIG_RCU_FANOUT_LEAF=2
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_RCU_TRACE=y
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT b/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
index 1f215e17cbae..3e7d7793acc7 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
+++ b/tools/testing/selftests/rcutorture/configs/refscale/NOPREEMPT
@@ -14,7 +14,6 @@ CONFIG_HIBERNATION=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_KPROBES=n
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT b/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
index 52e3ef674056..feed29cf043f 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
+++ b/tools/testing/selftests/rcutorture/configs/refscale/PREEMPT
@@ -12,6 +12,5 @@ CONFIG_HIBERNATION=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/TINY b/tools/testing/selftests/rcutorture/configs/refscale/TINY
index 9a11b578ff34..b67ea7a9128b 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/TINY
+++ b/tools/testing/selftests/rcutorture/configs/refscale/TINY
@@ -14,7 +14,6 @@ CONFIG_HIBERNATION=n
 CONFIG_RCU_NOCB_CPU=n
 CONFIG_DEBUG_LOCK_ALLOC=n
 CONFIG_PROVE_LOCKING=n
-CONFIG_RCU_BOOST=n
 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
 CONFIG_RCU_EXPERT=y
 CONFIG_KPROBES=n
-- 
2.54.0

[PATCH for-v7.5 13/14] rcutorture: Disable RCU priority boost testing on non-TREE03 scenarios

[Joel Fernandes]

From: "Paul E. McKenney" <paulmck@kernel.org>

The TREE03 scenario has been configured to permit testing of RCU
priority boosting, including setting RCU kthead priorities to 2.
But now that RCU priority boosting is default for preemptible kernels,
it is now automatically tested for TREE01, TREE02, TREE08, and TREE09,
none of which have RCU kthread priorities set up for such testing.
One approach would be to set up these scenarios' kthread priorities,
but this would fail to test common-case kernels having default priorities.

This commit therefore disables the testing of RCU priority boosting
except on scenario TREE03, which is set up properly for this purpose,
though rather unrealistically.

Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot | 1 +
 tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot | 1 +
 tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot | 1 +
 tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot | 1 +
 4 files changed, 4 insertions(+)
 create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot

diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
index 1cc5b47dde28..73d51d869fd5 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
@@ -6,3 +6,4 @@ rcu_nocbs=0-1,3-7
 rcutorture.nocbs_nthreads=8
 rcutorture.nocbs_toggle=1000
 rcutorture.fwd_progress=0
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
index dd914fa8f690..f2ca5d812f3f 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE02.boot
@@ -1 +1,2 @@
 rcutorture.fwd_progress=2
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
index 94d38445d393..41c3264f8e8c 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
@@ -1,3 +1,4 @@
 rcupdate.rcu_self_test=1
 rcutree.rcu_fanout_exact=1
 rcu_nocbs=all
+rcutorture.test_boost=0
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot b/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot
new file mode 100644
index 000000000000..1f52e460e14a
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE09.boot
@@ -0,0 +1 @@
+rcutorture.test_boost=0
-- 
2.54.0

[PATCH for-v7.5 14/14] torture: Forgive SMT-control failures in CPU-hotplug testing

[Joel Fernandes]

The onoff task ends rcutorture with "End of test: RCU_HOTPLUG" if any
CPU-hotplug attempt fails, other than -EBUSY during boot.  Toggling
/sys/devices/system/cpu/smt/control while onoff runs causes two failures
that are not the test's fault.  cpu_up() returns -EPERM for an SMT thread
that SMT control keeps offline, and device_online() and device_offline()
return 1 when the toggle already did the operation.

Repro: in a 16-vCPU guest (8 cores, 2 threads), run
"modprobe rcutorture onoff_interval=1 gp_exp=0" while looping
"echo off/on > smt/control" for 120 seconds.  This ended in RCU_HOTPLUG,
with no reader errors.  With this change it ends in SUCCESS.

Forgive these like boot-time -EBUSY, by not counting the attempt.

Signed-off-by: Joel Fernandes <jfern@kernel.org>
---
 kernel/torture.c | 40 ++++++++++++++++++++++++++++------------
 1 file changed, 28 insertions(+), 12 deletions(-)

diff --git a/kernel/torture.c b/kernel/torture.c
index 8c4e6b2fe8ba..b860c99e9ca7 100644
--- a/kernel/torture.c
+++ b/kernel/torture.c
@@ -194,6 +194,24 @@ int torture_num_online_cpus(void)
 }
 EXPORT_SYMBOL_GPL(torture_num_online_cpus);
 
+/*
+ * If the CPU-hotplug failure is not the torture test's fault, return a
+ * string saying why it is forgiven.  Otherwise, return NULL.
+ */
+static const char *torture_onoff_forgive(int ret)
+{
+	// PCI probe frequently disables hotplug during boot.
+	if (!rcu_inkernel_boot_has_ended() && ret == -EBUSY)
+		return " (-EBUSY forgiven during boot)";
+	// Hotplug is forbidden, for example by SMT control.
+	if (ret == -EPERM)
+		return " (-EPERM forgiven, hotplug forbidden)";
+	// Something else, such as SMT control, already did the operation.
+	if (ret == 1)
+		return " (already in requested state, forgiven)";
+	return NULL;
+}
+
 /*
  * Attempt to take a CPU offline.  Return false if the CPU is already
  * offline or if it is not subject to CPU-hotplug operations.  The
@@ -204,7 +222,7 @@ bool torture_offline(int cpu, long *n_offl_attempts, long *n_offl_successes,
 {
 	unsigned long delta;
 	int ret;
-	char *s;
+	const char *s;
 	unsigned long starttime;
 
 	if (!cpu_online(cpu) || !cpu_is_hotpluggable(cpu))
@@ -220,12 +238,11 @@ bool torture_offline(int cpu, long *n_offl_attempts, long *n_offl_successes,
 	(*n_offl_attempts)++;
 	ret = remove_cpu(cpu);
 	if (ret) {
-		s = "";
-		if (!rcu_inkernel_boot_has_ended() && ret == -EBUSY) {
-			// PCI probe frequently disables hotplug during boot.
+		s = torture_onoff_forgive(ret);
+		if (s)
 			(*n_offl_attempts)--;
-			s = " (-EBUSY forgiven during boot)";
-		}
+		else
+			s = "";
 		if (verbose)
 			pr_alert("%s" TORTURE_FLAG
 				 "torture_onoff task: offline %d failed%s: errno %d\n",
@@ -266,7 +283,7 @@ bool torture_online(int cpu, long *n_onl_attempts, long *n_onl_successes,
 {
 	unsigned long delta;
 	int ret;
-	char *s;
+	const char *s;
 	unsigned long starttime;
 
 	if (cpu_online(cpu) || !cpu_is_hotpluggable(cpu))
@@ -280,12 +297,11 @@ bool torture_online(int cpu, long *n_onl_attempts, long *n_onl_successes,
 	(*n_onl_attempts)++;
 	ret = add_cpu(cpu);
 	if (ret) {
-		s = "";
-		if (!rcu_inkernel_boot_has_ended() && ret == -EBUSY) {
-			// PCI probe frequently disables hotplug during boot.
+		s = torture_onoff_forgive(ret);
+		if (s)
 			(*n_onl_attempts)--;
-			s = " (-EBUSY forgiven during boot)";
-		}
+		else
+			s = "";
 		if (verbose)
 			pr_alert("%s" TORTURE_FLAG
 				 "torture_onoff task: online %d failed%s: errno %d\n",
-- 
2.54.0