From: Kunwu Chan <kunwu.chan@gmail.com>
To: dave@stgolabs.net, paulmck@kernel.org, josh@joshtriplett.org,
frederic@kernel.org, neeraj.upadhyay@kernel.org,
joelagnelf@nvidia.com, boqun@kernel.org, urezki@gmail.com
Cc: rostedt@goodmis.org, mathieu.desnoyers@efficios.com,
jiangshanlai@gmail.com, qiang.zhang@linux.dev,
linux-kernel@vger.kernel.org, rcu@vger.kernel.org,
Kunwu Chan <kunwu.chan@gmail.com>,
Kunwu Chan <chentao@kylinos.cn>
Subject: [PATCH v2 6/7] rcutorture: Add kasan-aux.sh aux-stack checker
Date: Wed, 18 Nov 2026 16:51:46 +0800 [thread overview]
Message-ID: <20261118085147.1668598-7-kunwu.chan@gmail.com> (raw)
In-Reply-To: <20261118085147.1668598-1-kunwu.chan@gmail.com>
kasan-aux.sh runs dedicated aux-stack tests with kvm.sh --kasan,
existing scenarios, and the rcutorture.kasan_debug=1 and
kasan_multi_shot boot parameters. It then verifies the recorded
caller in each intentional report's auxiliary stack, retaining the
report body and using kvm-recheck.sh for the ordinary checks.
Flavors without callback operations are expected to skip the
self-test. Saved results can be rechecked with --check using
their batches list and QEMU commands.
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Suggested-by: Joel Fernandes <joelagnelf@nvidia.com>
Cc: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <chentao@kylinos.cn>
---
Documentation/RCU/torture.rst | 32 +++
.../rcutorture/bin/kasan-aux-check.sh | 158 ++++++++++++
.../selftests/rcutorture/bin/kasan-aux.sh | 228 ++++++++++++++++++
3 files changed, 418 insertions(+)
create mode 100755 tools/testing/selftests/rcutorture/bin/kasan-aux-check.sh
create mode 100755 tools/testing/selftests/rcutorture/bin/kasan-aux.sh
diff --git a/Documentation/RCU/torture.rst b/Documentation/RCU/torture.rst
index 1ad5cc7938113..15a430f5111f4 100644
--- a/Documentation/RCU/torture.rst
+++ b/Documentation/RCU/torture.rst
@@ -212,6 +212,38 @@ Note that --gdb limits you to one scenario per kvm.sh run and requires
that you have another window open from which to run ``gdb`` as instructed
by the script.
+The callback enqueue-stack self-test intentionally triggers a KASAN
+use-after-free report. Run it separately from ordinary KASAN bug hunts::
+
+ tools/testing/selftests/rcutorture/bin/kasan-aux.sh
+
+This uses ``kvm.sh --kasan``, existing scenarios, and the
+``rcutorture.kasan_debug=1`` and ``kasan_multi_shot`` boot parameters.
+The default 30-second runs cover Tree RCU, RCU Tasks, RCU Tasks Trace,
+Tree SRCU, and, except on arm64, Tiny RCU and Tiny SRCU. Use ``--configs``
+and ``--duration`` to override the scenario list and duration.
+
+These scenarios cover the callback enqueue implementations that record
+KASAN auxiliary stacks, rather than all RCU configuration combinations.
+Flavors with no callback operations, such as Tasks Rude, are expected
+to skip the self-test, and the checker requires their skip message and
+a successful ordinary run. Any other skip fails the check, even if
+the ordinary torture run succeeds.
+
+The script requires exactly one intentional report per supported scenario, with
+``rcu_test_kasan()`` in its "Last potentially related work creation"
+stack, followed by successful completion of the same torture flavor.
+Only the verified report's diagnostic markers are filtered; the body
+is retained for the existing checks, including the configuration,
+build, and console checks that ``kvm-recheck.sh`` runs. The original
+logs and the underlying ``kvm.sh`` runtime-error status are preserved.
+Saved results can be rechecked with ``kasan-aux.sh --check
+results-directory``, using the saved ``batches`` file and QEMU commands.
+As with the intentional warnings tested by ``srcu_lockdep.sh``, ordinary
+``kvm-recheck.sh`` and ``kvm-find-errors.sh`` still report the raw
+diagnostic; use the dedicated check to distinguish the expected report
+from failures.
+
Kernel boot arguments can also be supplied, for example, to control
rcutorture's module parameters. For example, to test a change to RCU's
CPU stall-warning code, use "--bootargs 'rcutorture.stall_cpu=30'".
diff --git a/tools/testing/selftests/rcutorture/bin/kasan-aux-check.sh b/tools/testing/selftests/rcutorture/bin/kasan-aux-check.sh
new file mode 100755
index 0000000000000..7a6f425975207
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/kasan-aux-check.sh
@@ -0,0 +1,158 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Verify the intentional rcutorture KASAN report and remove its diagnostics.
+# Flavors without callback operations must instead show the expected skip.
+# Leave all other console output for the usual torture error checks.
+#
+# Authors: Kunwu Chan <kunwu.chan@gmail.com>
+#
+# Usage: kasan-aux-check.sh console.log flavor
+
+if test "$#" -ne 2 || ! test -r "$1"
+then
+ echo "Usage: $0 console.log flavor" >&2
+ exit 1
+fi
+
+awk -v flavor="$2" '
+BEGIN {
+ timestamp = "[[:space:]]*[0-9]+\\.[0-9]+";
+ caller = "[[:space:]]*[CT][[:space:]]*[0-9]+";
+ log_prefix = "^[[:space:]]*\\[(" timestamp "|" caller ")\\][[:space:]]*";
+ frame_prefix = "^(\\[<[[:xdigit:]]+>\\][[:space:]]*)?\\??[[:space:]]*";
+ frame_suffix = "([.][[:alnum:]_.]+)?\\+0x[[:xdigit:]]+/0x[[:xdigit:]]+([[:space:]]|$)";
+ test_name = "rcutorture: WARN: KASAN aux-stack call_" flavor "()";
+ success = flavor "-torture:--- End of test: SUCCESS: ";
+ skip_name = "rcutorture: " flavor " has no callback/barrier ops, skipping KASAN test";
+ skip_mode = flavor ~ /^(tasks-rude|busted|trivial)$/;
+}
+
+function fail(message)
+{
+ print "kasan-aux-check.sh: " message " at line " NR > "/dev/stderr";
+ failed = 1;
+ exit 1;
+}
+
+# Strip printk timestamps/caller IDs for matching, retaining the original line.
+function message(line)
+{
+ sub(/\r$/, "", line);
+ sub(/^<[0-9]+>/, "", line);
+ while (sub(log_prefix, "", line))
+ ;
+ sub(/^[[:space:]]+/, "", line);
+ sub(/[[:space:]]+$/, "", line);
+ return line;
+}
+
+function frame(line, symbol)
+{
+ return line ~ (frame_prefix symbol frame_suffix);
+}
+
+{
+ line = message($0);
+ if (skip_mode) {
+ if (line == skip_name) {
+ if (++skips != 1 || succeeded)
+ fail("unexpected or late skip message");
+ } else if (line ~ /skipping KASAN test|KASAN aux-stack test skipped|not testing KASAN aux stack/)
+ fail("unexpected skip message");
+ if (line ~ /End of test: SUCCESS:/) {
+ if (index(line, success) != 1 || ++succeeded != 1 || skips != 1)
+ fail("unexpected SUCCESS marker");
+ } else if (line ~ /End of test:/)
+ fail("unexpected test result");
+ if (line ~ /^rcutorture: WARN: KASAN aux-stack/ || line ~ /BUG: KASAN:/)
+ fail("unexpected aux-stack test output");
+ print;
+ next;
+ }
+ if (line ~ /skipping KASAN test|KASAN aux-stack test skipped|not testing KASAN aux stack/)
+ fail("KASAN test was skipped");
+ if (line ~ /^rcutorture: WARN: KASAN aux-stack call_[-[:alnum:]_]+\(\) test starting\.$/) {
+ if (line != test_name " test starting." ||
+ ++started != 1 || reports || completed || succeeded)
+ fail("unexpected test start");
+ next;
+ }
+ if (line ~ /^rcutorture: WARN: KASAN aux-stack call_[-[:alnum:]_]+\(\) test complete\.$/) {
+ if (line != test_name " test complete." ||
+ ++completed != 1 || started != 1 || reports != 1 || in_report)
+ fail("unexpected test completion");
+ next;
+ }
+ if (line ~ /End of test: SUCCESS:/) {
+ if (index(line, success) != 1 ||
+ ++succeeded != 1 || completed != 1 || in_report)
+ fail("unexpected SUCCESS marker");
+ }
+ else if (line ~ /End of test:/)
+ fail("unexpected test result");
+ if (line ~ /BUG: KASAN:/) {
+ if (line !~ ("^BUG: KASAN: slab-use-after-free in rcu_test_kasan" frame_suffix))
+ fail("unexpected KASAN report");
+ if (in_report || !separator || started != 1 || completed || ++reports != 1)
+ fail("unexpected or unbounded KASAN report");
+ report_separator = message(separator);
+ printf "%s", before_report;
+ before_report = "";
+ separator = "";
+ in_report = 1;
+ next;
+ }
+ if (in_report) {
+ if (line == report_separator) {
+ if (reads != 1 || traces != 1 || aux_stacks != 1 || !test_frame)
+ fail("incomplete KASAN report or missing caller in auxiliary stack");
+ in_report = 0;
+ next;
+ }
+ if (line ~ /^(Read|Write) of size /) {
+ if (line !~ /^Read of size 1 at addr / || ++reads != 1)
+ fail("unexpected KASAN access");
+ }
+ if (line == "Last potentially related work creation:") {
+ if (++aux_stacks != 1)
+ fail("duplicate auxiliary stack");
+ in_aux = 1;
+ }
+ else if (in_aux) {
+ if (!frame(line, "[[:alnum:]_]+"))
+ in_aux = 0;
+ else if (frame(line, "rcu_test_kasan"))
+ test_frame = 1;
+ }
+ # Suppress only the main trace title in the expected report.
+ if (line ~ /^Call [Tt]race:$/ && ++traces == 1)
+ next;
+ print;
+ next;
+ }
+ if (started && !reports && line ~ /^=+$/) {
+ if (separator) {
+ print separator;
+ printf "%s", before_report;
+ }
+ separator = $0;
+ before_report = "";
+ next;
+ }
+ if (separator) {
+ before_report = before_report $0 "\n";
+ next;
+ }
+ print;
+}
+
+END {
+ if (failed)
+ exit 1;
+ if (skip_mode) {
+ if (skips != 1 || succeeded != 1)
+ fail("missing skip message or SUCCESS marker");
+ } else if (in_report || started != 1 || completed != 1 || succeeded != 1 || reports != 1)
+ fail("missing test markers or truncated KASAN report");
+}' < "$1" || exit 1
diff --git a/tools/testing/selftests/rcutorture/bin/kasan-aux.sh b/tools/testing/selftests/rcutorture/bin/kasan-aux.sh
new file mode 100755
index 0000000000000..ae7896658e94d
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/kasan-aux.sh
@@ -0,0 +1,228 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Run and check the intentional KASAN callback enqueue-stack tests.
+#
+# Authors: Kunwu Chan <kunwu.chan@gmail.com>
+
+scriptname=$0
+RCUTORTURE="$(pwd)/tools/testing/selftests/rcutorture"; export RCUTORTURE
+PATH=${RCUTORTURE}/bin:$PATH; export PATH
+. functions.sh
+
+usage () {
+ echo "Usage: $scriptname [ --datestamp string ] [ --duration time ]"
+ echo " [ --configs \"existing scenario list\" ]"
+ echo " $scriptname --check results-directory"
+ exit 1
+}
+
+ds="$(date +%Y.%m.%d-%H.%M.%S)-kasan-aux"
+duration=30s
+configs="TREE03 TASKS01 TRACE01 SRCU-P"
+if test "$(uname -m)" != aarch64
+then
+ configs="$configs TINY01 SRCU-T"
+fi
+checkdir=
+while test $# -gt 0
+do
+ case "$1" in
+ --check)
+ checkarg --check "(results directory)" "$#" "$2" '.' '^--'
+ checkdir=$2
+ shift
+ ;;
+ --configs)
+ checkarg --configs "(scenario list)" "$#" "$2" '^[^/.a-z]\+$' '^--'
+ configs=$2
+ shift
+ ;;
+ --datestamp)
+ checkarg --datestamp "(relative pathname)" "$#" "$2" '^[a-zA-Z0-9._/-]*$' '^--'
+ ds=$2
+ shift
+ ;;
+ --duration)
+ checkarg --duration "(time)" "$#" "$2" '^[0-9][0-9]*[smhd]\?$' '^--'
+ duration=$2
+ shift
+ ;;
+ "")
+ ;;
+ *)
+ usage
+ ;;
+ esac
+ shift
+done
+
+T="$(mktemp -d "${TMPDIR-/tmp}/kasan-aux.sh.XXXXXX")" || exit 1
+trap 'rm -rf "$T"' EXIT
+nsupported=0
+
+# Keep raw logs intact and let the ordinary recheck handle the verified output.
+check_results () {
+ local rundir config dir copy flavor f configs
+ local work="$T/results"
+
+ rundir=$(cd "$1" && pwd) || return 1
+ if ! test -s "$rundir/batches" || ! test -r "$rundir/log"
+ then
+ echo "$rundir: missing batches or log"
+ return 1
+ fi
+ configs=$(awk '!/^#/ { print $2 }' "$rundir/batches")
+ if test -z "$configs"
+ then
+ echo "$rundir: empty scenario list"
+ return 1
+ fi
+ mkdir "$work" || return 1
+ ln -s "$rundir/log" "$work/log" || return 1
+ echo rcu > "$work/torture_suite"
+ if test -f "$rundir/re-run"
+ then
+ ln -s "$rundir/re-run" "$work/re-run" || return 1
+ fi
+ for config in $configs
+ do
+ dir="$rundir/$config"
+ copy="$work/$config"
+ mkdir "$copy" || return 1
+ # Link inputs only, so recheck cannot overwrite saved diagnostics.
+ for f in .config ConfigFragment ConfigFragment.input Make.defconfig.out Make.out qemu-cmd qemu-retval
+ do
+ if ! test -r "$dir/$f"
+ then
+ echo "$config: missing $f"
+ return 1
+ fi
+ ln -s "$dir/$f" "$copy/$f" || return 1
+ done
+ for f in Make.oldconfig.err Warnings vmlinux vmlinux.xz
+ do
+ ln -s "$dir/$f" "$copy/$f" || return 1
+ done
+ if ! grep -q '^CONFIG_KASAN_GENERIC=y$' "$dir/.config" ||
+ test "$(cat "$dir/qemu-retval")" != 0
+ then
+ echo "$config: Generic KASAN required and QEMU must exit normally"
+ return 1
+ fi
+ # kvm.sh saves a single QEMU command with a double-quoted -append string.
+ # Read it once, retaining the last flavor parameter, as the kernel does.
+ if ! flavor=$(awk '
+ BEGIN { flavor = "rcu"; }
+ /^[[:space:]]*(#|$)/ { next; }
+ {
+ if (++commands != 1 ||
+ !match($0, /(^|[[:space:]])-append[[:space:]]+"/))
+ exit 1;
+ args = substr($0, RSTART + RLENGTH);
+ if (!sub(/"[^"]*$/, "", args))
+ exit 1;
+ # Only the plain kvm.sh-generated format is supported.
+ if (args ~ /\\/)
+ exit 1;
+ # Keep quoted kernel parameter values together.
+ while (args ~ /[^[:space:]]/) {
+ sub(/^[[:space:]]+/, "", args);
+ if (!match(args, /^([^"[:space:]]|"[^"]*")+/))
+ exit 1;
+ word = substr(args, 1, RLENGTH);
+ args = substr(args, RLENGTH + 1);
+ if (args !~ /^([[:space:]]|$)/)
+ exit 1;
+ whole_quote = sub(/^"/, "", word);
+ equals = index(word, "=");
+ param = equals ? substr(word, 1, equals - 1) : word;
+ value = equals ? substr(word, equals + 1) : "";
+ value_quote = sub(/^"/, "", value);
+ if (whole_quote || value_quote) {
+ if (equals)
+ sub(/"$/, "", value);
+ else
+ sub(/"$/, "", param);
+ }
+ if (param == "--" && !equals)
+ break;
+ gsub(/-/, "_", param);
+ if (param == "rcutorture.torture_type")
+ flavor = value;
+ if (param == "kasan_multi_shot")
+ multi_shot = 1;
+ }
+ valid = 1;
+ }
+ END {
+ if (!valid || commands != 1 || !multi_shot ||
+ flavor !~ /^(rcu|srcu|srcud|tasks|tasks-tracing|tasks-rude|busted|trivial)$/)
+ exit 1;
+ print flavor;
+ }' "$dir/qemu-cmd")
+ then
+ echo "$config: invalid saved boot arguments or missing kasan_multi_shot"
+ return 1
+ fi
+ case "$flavor" in
+ rcu|srcu|srcud|tasks|tasks-tracing)
+ nsupported=$((nsupported+1))
+ ;;
+ esac
+ if ! TMPDIR="$T" kasan-aux-check.sh "$dir/console.log" "$flavor" > "$copy/console.log"
+ then
+ return 1
+ fi
+ # kvm-recheck.sh does not propagate parse-build.sh exit status.
+ TMPDIR="$T" parse-build.sh "$dir/Make.out" "$config" || return 1
+ done
+ if ! TMPDIR="$T" kvm-recheck.sh "$work" 2> "$T/recheck.err" || test -s "$T/recheck.err"
+ then
+ cat "$T/recheck.err"
+ return 1
+ fi
+ echo "Summary: KASAN aux-stack checks passed"
+}
+
+if test -n "$checkdir"
+then
+ check_results "$checkdir"
+ exit $?
+fi
+
+rundir="$RCUTORTURE/res/$ds"
+if test -e "$rundir"
+then
+ echo "Results directory already exists: $rundir"
+ exit 1
+fi
+kvm.sh --allcpus --kasan --kconfig "CONFIG_KASAN_GENERIC=y" \
+ --configs "$configs" --duration "$duration" --trust-make \
+ --bootargs "rcutorture.kasan_debug=1 rcutorture.object_debug=0 kasan_multi_shot" \
+ --datestamp "$ds"
+ret=$?
+err=0
+if ! check_results "$rundir" > "$T/check.out" 2>&1
+then
+ err=1
+fi
+# Scenarios with callback operations produce the intentional reports, so
+# kvm.sh must report them as runtime errors. Skip-only selections exit 0.
+if test "$nsupported" -gt 0
+then
+ expected=3
+else
+ expected=0
+fi
+if test "$ret" -ne "$expected"
+then
+ echo "Expected kvm.sh exit status $expected, got $ret"
+ err=1
+fi
+cat "$T/check.out"
+if test -d "$rundir"
+then
+ cat "$T/check.out" >> "$rundir/log"
+fi
+exit "$err"
--
2.43.0
next prev parent reply other threads:[~2026-11-18 8:52 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 8:51 [PATCH v2 0/7] rcu: Record callback enqueue stacks and add torture tests Kunwu Chan
2026-11-18 8:51 ` [PATCH v2 1/7] rcu-tasks: Record callback enqueue stack Kunwu Chan
2026-11-18 8:51 ` [PATCH v2 2/7] rcutorture: Require debug_objects flag for debug-objects test Kunwu Chan
2026-11-18 8:51 ` [PATCH v2 3/7] rcutorture: Test duplicate call_rcu_tasks() callback enqueue Kunwu Chan
2026-11-18 8:51 ` [PATCH v2 4/7] rcutorture: Test duplicate call_rcu_tasks_trace() " Kunwu Chan
2026-11-18 8:51 ` [PATCH v2 5/7] rcutorture: Add KASAN aux-stack self-test Kunwu Chan
2026-11-18 8:51 ` Kunwu Chan [this message]
2026-11-18 8:51 ` [PATCH v2 7/7] rcutorture: Run KASAN aux-stack checks from torture.sh Kunwu Chan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118085147.1668598-7-kunwu.chan@gmail.com \
--to=kunwu.chan@gmail.com \
--cc=boqun@kernel.org \
--cc=chentao@kylinos.cn \
--cc=dave@stgolabs.net \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joelagnelf@nvidia.com \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=neeraj.upadhyay@kernel.org \
--cc=paulmck@kernel.org \
--cc=qiang.zhang@linux.dev \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®