[PATCH 0/3] Torture-test updates for v7.4

[PATCH 0/3] Torture-test updates for v7.4

[Paul E. McKenney]

Hello!

This series adds some torture-test updates:

1.	Fix divide-by-zero with fwd_progress_div=1, courtesy of Kunwu
	Chan.

2.	Synchronously wait for all rcu_torture_irq() callbacks to
	complete, courtesy of Zqiang.

3.	Allow specifying alternative ssh command to kvm-remote.sh.

						Thanx, Paul

------------------------------------------------------------------------

 b/kernel/rcu/rcutorture.c                              |    2 -
 b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh |   22 ++++++++++++-----
 kernel/rcu/rcutorture.c                                |    2 +
 3 files changed, 19 insertions(+), 7 deletions(-)

[PATCH 1/3] rcutorture: Fix divide-by-zero with fwd_progress_div=1

[Paul E. McKenney]

From: Kunwu Chan <kunwu.chan@gmail.com>

When fwd_progress_div=1, the forward-progress test computes:
  sd4 = (sd + div - 1) / div = sd
  dur = sd4 + torture_random(&trs) % (sd - sd4) = sd4 + % 0

The modulo operation with a zero divisor triggers an integer
division by zero (undefined behavior at the C level, #DE trap on x86),
causing a kernel Oops and panic. On x86_64, this manifests as:

  rcu_torture_fwd_prog_nr: Starting forward-progress test 0
  Oops: divide error: 0000 [#1] SMP PTI
  RIP: 0010:rcu_torture_fwd_prog+0x90b/0x1160
  R12: 0000000000000000

The existing guard only handles non-positive values. However,
fwd_progress_div=1 also makes the random range empty because
sd4 == sd.

Change the guard to reject values below 2. The forward-progress test
only reaches this calculation when stall_dur() is positive, so
sd = stall_dur() + 1 >= 2. For fwd_progress_div >= 2, sd4 < sd,
ensuring that sd - sd4 is at least 1.

Keep the existing fallback to the default value of 4 for invalid
values.

Verified with QEMU/KVM: a 138-second run with fwd_progress_div=1
completed 81 forward-progress test cycles without a crash.

Fixes: 1b27291b1ea4f ("rcutorture: Add forward-progress tests for RCU grace periods")
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/rcutorture.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 182d47975efd..79807475b672 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -4024,7 +4024,7 @@ static int __init rcu_torture_fwd_prog_init(void)
 	}
 	if (fwd_progress_holdoff <= 0)
 		fwd_progress_holdoff = 1;
-	if (fwd_progress_div <= 0)
+	if (fwd_progress_div < 2)
 		fwd_progress_div = 4;
 	rfp = kzalloc_objs(*rfp, fwd_progress);
 	fwd_prog_tasks = kzalloc_objs(*fwd_prog_tasks, fwd_progress);
-- 
2.40.1

[PATCH 2/3] rcutorture: Synchronously wait for all rcu_torture_irq() callbacks to complete

[Paul E. McKenney]

From: Zqiang <qiang.zhang@linux.dev>

The rcu_torture_reader() drives RCU readers from interrupt context via
smp_call_function_single(cpu, rcu_torture_irq, NULL, 0) with wait=0, to
runs rcu_torture_irq() on a remote CPU. this is async, nothing waits for
the remote handler to run.

On shutdown, torture_stop_kthread() only waits for each reader kthread to
return, and the reader's timer_delete_sync() only drains its timer. Neither
waits for a rcu_torture_irq() which still pending or executing on a remote
CPU, so it can run after all readers have exited and rcu_torture_cleanup()
has already advanced.

1. rcu_torture_irq() may issue cur_ops->call(rhp, rcu_torture_timer_cb)
   after cur_ops->cb_barrier() has been waiting for all outstanding
   callbacks complete. once the module is unloaded, fires into freed
   module text, a use-after-free happen.

2. rcu_torture_irq() may still be inside rcu_torture_one_read(), holding
   a read-side critical section, when cur_ops->cleanup() tears the flavor
   down (e.g. cleanup_srcu_struct()), triggering an active-reader warning
   or use-after-free of the torn-down structure.

This commit therefore issue a kick_all_cpus_sync() after all readers
kthread have returned and before cur_ops->cb_barrier(), synchronous IPI
round trip to every CPU guarantees that every rcu_torture_irq() which
previously issued by any reader has completed.

Signed-off-by: Zqiang <qiang.zhang@linux.dev>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 kernel/rcu/rcutorture.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 79807475b672..51ed35f5aab1 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -4491,6 +4491,8 @@ rcu_torture_cleanup(void)
 		for (i = 0; i < nrealreaders; i++)
 			torture_stop_kthread(rcu_torture_reader,
 					     reader_tasks[i]);
+		if (irqreader && cur_ops->irq_capable)
+			kick_all_cpus_sync();
 		kfree(reader_tasks);
 		reader_tasks = NULL;
 	}
-- 
2.40.1

[PATCH 3/3] torture: Allow specifying alternative ssh command to kvm-remote.sh

[Paul E. McKenney]

Some environments require use of alternative commands to access the
test hosts.  This commit therefore adds a KVM_REMOTE_SSH environment
variable for this purpose.  If this variable is unset, ssh is used.
Any alternative ssh command must support the usual ssh arguments,
including the command to be executed remotely.  In some cases, you may
need a wrapper script to make the alternative ssh-like command look
enough like ssh to satisfy kvm-remote.sh.

Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 .../selftests/rcutorture/bin/kvm-remote.sh    | 22 ++++++++++++++-----
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
index 48a8052d5dae..a8397f86e49d 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-remote.sh
@@ -6,6 +6,11 @@
 # Usage: kvm-remote.sh "systems" [ <kvm.sh args> ]
 #	 kvm-remote.sh "systems" /path/to/old/run [ <kvm-again.sh args> ]
 #
+# The caller may set the KVM_REMOTE_SSH environment in order to specify
+# an alternative ssh command, which is necessary in some environments
+# for authentication purposes.  This alternative ssn command must support
+# ssh's usual arguments.
+#
 # Copyright (C) 2021 Facebook, Inc.
 #
 # Authors: Paul E. McKenney <paulmck@kernel.org>
@@ -13,6 +18,11 @@
 scriptname=$0
 args="$*"
 
+if test -z "${KVM_REMOTE_SSH}"
+then
+	KVM_REMOTE_SSH=ssh; export KVM_REMOTE_SSH
+fi
+
 if ! test -d tools/testing/selftests/rcutorture/bin
 then
 	echo $scriptname must be run from top-level directory of kernel source tree.
@@ -137,7 +147,7 @@ chmod +x $T/bin/kvm-remote-*.sh
 # Check first to avoid the need for cleanup for system-name typos
 for i in $systems
 do
-	ssh -o BatchMode=yes $i getconf _NPROCESSORS_ONLN > $T/ssh.stdout 2> $T/ssh.stderr
+	${KVM_REMOTE_SSH} -o BatchMode=yes $i getconf _NPROCESSORS_ONLN > $T/ssh.stdout 2> $T/ssh.stderr
 	ret=$?
 	if test "$ret" -ne 0
 	then
@@ -158,14 +168,14 @@ echo Build-products tarball: `du -h $T/binres.tgz` | tee -a "$oldrun/remote-log"
 for i in $systems
 do
 	echo Downloading tarball to $i `date` | tee -a "$oldrun/remote-log"
-	cat $T/binres.tgz | ssh -o BatchMode=yes $i "cd /tmp; tar -xzf -"
+	cat $T/binres.tgz | ${KVM_REMOTE_SSH} -o BatchMode=yes $i "cd /tmp; tar -xzf -"
 	ret=$?
 	tries=0
 	while test "$ret" -ne 0
 	do
 		echo Unable to download $T/binres.tgz to system $i, waiting and then retrying.  $tries prior retries. | tee -a "$oldrun/remote-log"
 		sleep 60
-		cat $T/binres.tgz | ssh -o BatchMode=yes $i "cd /tmp; tar -xzf -"
+		cat $T/binres.tgz | ${KVM_REMOTE_SSH} -o BatchMode=yes $i "cd /tmp; tar -xzf -"
 		ret=$?
 		if test "$ret" -ne 0
 		then
@@ -191,7 +201,7 @@ checkremotefile () {
 
 	while :
 	do
-		ssh -o BatchMode=yes $1 "test -f \"$2\""
+		${KVM_REMOTE_SSH} -o BatchMode=yes $1 "test -f \"$2\""
 		ret=$?
 		if test "$ret" -eq 255
 		then
@@ -239,7 +249,7 @@ startbatches () {
 		then
 			continue # System still running last test, skip.
 		fi
-		ssh -o BatchMode=yes "$i" "cd \"$resdir/$ds\"; touch remote.run; PATH=\"$T/bin:$PATH\" nohup kvm-remote-$curbatch.sh > kvm-remote-$curbatch.sh.out 2>&1 &" 1>&2
+		${KVM_REMOTE_SSH} -o BatchMode=yes "$i" "cd \"$resdir/$ds\"; touch remote.run; PATH=\"$T/bin:$PATH\" nohup kvm-remote-$curbatch.sh > kvm-remote-$curbatch.sh.out 2>&1 &" 1>&2
 		ret=$?
 		if test "$ret" -ne 0
 		then
@@ -281,7 +291,7 @@ do
 		if test "$ret" -eq 1
 		then
 			echo " ---" Collecting results from $i `date` | tee -a "$oldrun/remote-log"
-			( cd "$oldrun"; ssh -o BatchMode=yes $i "cd $rundir; tar -czf - kvm-remote-*.sh.out */console.log */kvm-test-1-run*.sh.out */qemu[_-]pid */qemu-retval */qemu-affinity; rm -rf $T > /dev/null 2>&1" | tar -xzf - )
+			( cd "$oldrun"; ${KVM_REMOTE_SSH} -o BatchMode=yes $i "cd $rundir; tar -czf - kvm-remote-*.sh.out */console.log */kvm-test-1-run*.sh.out */qemu[_-]pid */qemu-retval */qemu-affinity; rm -rf $T > /dev/null 2>&1" | tar -xzf - )
 			break;
 		fi
 		if test "$ret" -eq 255
-- 
2.40.1