mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run
@ 2026-03-07 22:07 Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 1/9] ktest: Avoid undef warning when WARNINGS_FILE is unset Ricardo B. Marlière
                   ` (9 more replies)
  0 siblings, 10 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:07 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

Hi Steven,

Please consider the following ktest patches.

They fix a few edge cases that I found while using ktest with a small
virtme-ng-based harness [1], and the last two add small features:
PRE_KTEST_DIE and a new --dry-run 

Thanks!
-	Ricardo.

[1]: https://github.com/rbmarliere/k-lab

Assisted-by: codex 5.3
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
Ricardo B. Marlière (9):
      ktest: Avoid undef warning when WARNINGS_FILE is unset
      ktest: Resolve LOG_FILE in test option context
      ktest: Treat undefined self-reference as empty
      ktest: Honor empty per-test option overrides
      ktest: Run commands through list-form shell open
      ktest: Stop dropping console output during power-cycle reboot
      ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
      ktest: Run POST_KTEST hooks on failure and cancellation
      ktest: Add a --dry-run mode

 tools/testing/ktest/ktest.pl    | 158 +++++++++++++++++++++++++++++-----------
 tools/testing/ktest/sample.conf |   6 ++
 2 files changed, 122 insertions(+), 42 deletions(-)
---
base-commit: 4ae12d8bd9a830799db335ee661d6cbc6597f838
change-id: 20260307-ktest-fixes-0ff398d65521

Best regards,
-- 
Ricardo B. Marlière <rbm@suse.com>


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 1/9] ktest: Avoid undef warning when WARNINGS_FILE is unset
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
@ 2026-03-07 22:07 ` Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 2/9] ktest: Resolve LOG_FILE in test option context Ricardo B. Marlière
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:07 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

check_buildlog() probes $warnings_file with -f even when WARNINGS_FILE is
not configured. Perl warns about the uninitialized value and adds noise to
the test log, which can hide the output we actually care about.

Check that WARNINGS_FILE is defined before testing whether the file exists.

Fixes: 4283b169abfb ("ktest: Add make_warnings_file and process full warnings")
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 001c4df9f7df..f48ee64c69da 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -2508,7 +2508,7 @@ sub check_buildlog {
     my $save_no_reboot = $no_reboot;
     $no_reboot = 1;
 
-    if (-f $warnings_file) {
+    if (defined($warnings_file) && -f $warnings_file) {
 	open(IN, $warnings_file) or
 	    dodie "Error opening $warnings_file";
 

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 2/9] ktest: Resolve LOG_FILE in test option context
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 1/9] ktest: Avoid undef warning when WARNINGS_FILE is unset Ricardo B. Marlière
@ 2026-03-07 22:07 ` Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 3/9] ktest: Treat undefined self-reference as empty Ricardo B. Marlière
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:07 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

LOG_FILE is expanded immediately after the config file is parsed with
eval_option(..., -1). That uses the default context, not the same option
resolution path used for tests. If LOG_FILE depends on options that are
finalized per test, it can be resolved from stale values before the first
test starts.

Resolve LOG_FILE through set_test_option("LOG_FILE", 1) instead so it uses
the same expansion rules as the rest of the test options.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index f48ee64c69da..b7a1c8c617e0 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -4391,7 +4391,7 @@ EOF
 read_config $ktest_config;
 
 if (defined($opt{"LOG_FILE"})) {
-    $opt{"LOG_FILE"} = eval_option("LOG_FILE", $opt{"LOG_FILE"}, -1);
+    $opt{"LOG_FILE"} = set_test_option("LOG_FILE", 1);
 }
 
 # Append any configs entered in manually to the config file.

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 3/9] ktest: Treat undefined self-reference as empty
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 1/9] ktest: Avoid undef warning when WARNINGS_FILE is unset Ricardo B. Marlière
  2026-03-07 22:07 ` [PATCH 2/9] ktest: Resolve LOG_FILE in test option context Ricardo B. Marlière
@ 2026-03-07 22:07 ` Ricardo B. Marlière
  2026-03-09 14:40   ` Steven Rostedt
  2026-03-07 22:07 ` [PATCH 4/9] ktest: Honor empty per-test option overrides Ricardo B. Marlière
                   ` (6 subsequent siblings)
  9 siblings, 1 reply; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:07 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

Config variables are expanded when they are assigned. A first-time append
such as:

  VAR := ${VAR} foo

leaves the literal ${VAR} in the stored value because VAR has not been
defined yet. Later expansions then carry the self-reference forward instead
of behaving like an empty prefix.

Drop an unescaped self-reference when the variable has no current value,
and trim the outer whitespace left behind. Keep escaped \${VAR} references
unchanged so literal text still works.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b7a1c8c617e0..b8fcdabffffe 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -910,6 +910,14 @@ sub set_variable {
     if (defined($command_tmp_vars{$lvalue})) {
 	return;
     }
+
+    # If a variable is undefined, treat an unescaped self-reference as empty.
+    if (!defined($variable{$lvalue})) {
+	$rvalue =~ s/(?<!\\)\$\{\Q$lvalue\E\}//g;
+	$rvalue =~ s/^\s+//;
+	$rvalue =~ s/\s+$//;
+    }
+
     if ($rvalue =~ /^\s*$/) {
 	delete $variable{$lvalue};
     } else {

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 4/9] ktest: Honor empty per-test option overrides
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (2 preceding siblings ...)
  2026-03-07 22:07 ` [PATCH 3/9] ktest: Treat undefined self-reference as empty Ricardo B. Marlière
@ 2026-03-07 22:07 ` Ricardo B. Marlière
  2026-03-07 22:08 ` [PATCH 5/9] ktest: Run commands through list-form shell open Ricardo B. Marlière
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:07 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

A per-test override can clear an inherited default option by assigning an
empty value, but __set_test_option() still used option_defined() to decide
whether a per-test key existed. That turned an empty per-test assignment
back into "fall back to the default", so tests still could not clear
inherited settings.

For example:

  DEFAULTS
  (...)
  LOG_FILE = /tmp/ktest-empty-override.log
  CLEAR_LOG = 1
  ADD_CONFIG = /tmp/.config

  TEST_START
  TEST_TYPE = build
  BUILD_TYPE = nobuild
  ADD_CONFIG =

This would run the test with ADD_CONFIG[1] = /tmp/.config

Fix by checking whether the per-test key exists before falling back. If it
does exist but is empty, treat it as unset for that test and stop the
fallback chain there.

Fixes: 22c37a9ac49d ("ktest: Allow tests to undefine default options")
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b8fcdabffffe..42bc505e14cb 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -4191,7 +4191,8 @@ sub __set_test_option {
 
     my $option = "$name\[$i\]";
 
-    if (option_defined($option)) {
+    if (exists($opt{$option})) {
+	return undef if (!option_defined($option));
 	return $opt{$option};
     }
 
@@ -4199,7 +4200,8 @@ sub __set_test_option {
 	if ($i >= $test &&
 	    $i < $test + $repeat_tests{$test}) {
 	    $option = "$name\[$test\]";
-	    if (option_defined($option)) {
+	    if (exists($opt{$option})) {
+		return undef if (!option_defined($option));
 		return $opt{$option};
 	    }
 	}

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 5/9] ktest: Run commands through list-form shell open
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (3 preceding siblings ...)
  2026-03-07 22:07 ` [PATCH 4/9] ktest: Honor empty per-test option overrides Ricardo B. Marlière
@ 2026-03-07 22:08 ` Ricardo B. Marlière
  2026-03-07 22:08 ` [PATCH 6/9] ktest: Stop dropping console output during power-cycle reboot Ricardo B. Marlière
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:08 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

run_command() currently uses string-form open():

  open(CMD, "$command 2>&1 |")

That delegates parsing to the shell but also mixes the stderr redirection
into the command string. Switch to list-form open() with an explicit sh -c
wrapper so shell syntax errors are captured in the same output stream as
command output. Otherwise, important errors can not be retrieved from the
ktest LOG_FILE.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 42bc505e14cb..9d6f50045dbd 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1921,7 +1921,10 @@ sub run_command {
     doprint("$command ... ");
     $start_time = time;
 
-    $pid = open(CMD, "$command 2>&1 |") or
+    $pid = open(CMD, "-|",
+		"sh", "-c",
+		'command=$1; shift; exec 2>&1; eval "$command"',
+		"sh", $command) or
 	(fail "unable to exec $command" and return 0);
 
     if (defined($opt{"LOG_FILE"})) {

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 6/9] ktest: Stop dropping console output during power-cycle reboot
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (4 preceding siblings ...)
  2026-03-07 22:08 ` [PATCH 5/9] ktest: Run commands through list-form shell open Ricardo B. Marlière
@ 2026-03-07 22:08 ` Ricardo B. Marlière
  2026-03-07 22:08 ` [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures Ricardo B. Marlière
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:08 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

The POWER_CYCLE fallback added to reboot() flushes monitor output at the
wrong time. In the untimed reboot path, flushing immediately after
start_monitor() can consume the first output from the new boot before
monitor() begins reading it. In the timed path, flushing after POWER_CYCLE
can eat the "Linux version" banner or REBOOT_SUCCESS_LINE from the new
kernel.

That makes ktest miss the boot it is waiting for and can trigger an
unnecessary second power cycle.

Start the monitor before POWER_CYCLE so the reference counting stays
balanced, but only flush when reboot() was asked to wait for a timed
reboot. Perform that flush before issuing POWER_CYCLE so it drains stale
output from the old kernel instead of consuming the next boot.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 9d6f50045dbd..bd2e2311884c 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1499,12 +1499,13 @@ sub reboot {
     }
 
     if ($powercycle) {
-	run_command "$power_cycle";
-
 	start_monitor;
-	# flush out current monitor
-	# May contain the reboot success line
-	wait_for_monitor 1;
+	if (defined($time)) {
+		# Flush stale console output from the old kernel before power-cycling.
+		wait_for_monitor 1;
+	}
+
+	run_command "$power_cycle";
 
     } else {
 	# Make sure everything has been written to disk

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (5 preceding siblings ...)
  2026-03-07 22:08 ` [PATCH 6/9] ktest: Stop dropping console output during power-cycle reboot Ricardo B. Marlière
@ 2026-03-07 22:08 ` Ricardo B. Marlière
  2026-03-09 15:00   ` Steven Rostedt
  2026-03-07 22:08 ` [PATCH 8/9] ktest: Run POST_KTEST hooks on failure and cancellation Ricardo B. Marlière
                   ` (2 subsequent siblings)
  9 siblings, 1 reply; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:08 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

PRE_KTEST runs before the first test, but its return status is currently
ignored. A failing setup hook can leave the rest of the run executing in a
partially initialized environment.

Add PRE_KTEST_DIE so PRE_KTEST can fail the run in the same way
PRE_BUILD_DIE and PRE_TEST_DIE already can. Keep the default behavior
unchanged when the new option is not set.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl    | 8 +++++++-
 tools/testing/ktest/sample.conf | 6 ++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index bd2e2311884c..b018b937e028 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -101,6 +101,7 @@ my $build_type;
 my $build_options;
 my $final_post_ktest;
 my $pre_ktest;
+my $pre_ktest_die;
 my $post_ktest;
 my $pre_test;
 my $pre_test_die;
@@ -283,6 +284,7 @@ my %option_map = (
     "BUILD_DIR"			=> \$builddir,
     "TEST_TYPE"			=> \$test_type,
     "PRE_KTEST"			=> \$pre_ktest,
+    "PRE_KTEST_DIE"		=> \$pre_ktest_die,
     "POST_KTEST"		=> \$post_ktest,
     "PRE_TEST"			=> \$pre_test,
     "PRE_TEST_DIE"		=> \$pre_test_die,
@@ -4506,7 +4508,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     if ($i == 1) {
 	if (defined($pre_ktest)) {
 	    doprint "\n";
-	    run_command $pre_ktest;
+	    my $ret = run_command $pre_ktest;
+	    if (!$ret && defined($pre_ktest_die) &&
+		$pre_ktest_die) {
+		dodie "failed to pre_ktest\n";
+	    }
 	}
 	if ($email_when_started) {
 	    my $name = get_test_name;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 9c4c449a8f3e..b6e439ef511b 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -494,6 +494,12 @@
 #
 # default (undefined)
 #PRE_KTEST = ${SSH} ~/set_up_test
+#
+# To specify if the test should fail if PRE_KTEST fails,
+# PRE_KTEST_DIE needs to be set to 1. Otherwise the PRE_KTEST
+# result is ignored.
+# (default 0)
+#PRE_KTEST_DIE = 1
 
 # If you want to execute some command after all the tests have
 # completed, you can set this option. Note, it can be set as a

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 8/9] ktest: Run POST_KTEST hooks on failure and cancellation
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (6 preceding siblings ...)
  2026-03-07 22:08 ` [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures Ricardo B. Marlière
@ 2026-03-07 22:08 ` Ricardo B. Marlière
  2026-03-07 22:08 ` [PATCH 9/9] ktest: Add a --dry-run mode Ricardo B. Marlière
  2026-03-09 14:21 ` [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Steven Rostedt
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:08 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

PRE_KTEST can be useful for setting up the environment and POST_KTEST to
tear it down, however POST_KTEST only runs on the normal end-of-run path.
It is skipped when ktest exits through dodie() or cancel_test(). final
cleanup hooks are skipped.

Factor the final hook execution into run_post_ktest(), call it from the
normal exit path and from the early exit paths, and guard it so the hook
runs at most once.

Fixes: 921ed4c7208e ("ktest: Add PRE/POST_KTEST and TEST options")
Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index b018b937e028..8962857ce4a6 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -100,6 +100,7 @@ my $test_type;
 my $build_type;
 my $build_options;
 my $final_post_ktest;
+my $post_ktest_done = 0;
 my $pre_ktest;
 my $pre_ktest_die;
 my $post_ktest;
@@ -1586,6 +1587,24 @@ sub get_test_name() {
     return $name;
 }
 
+sub run_post_ktest {
+    my $cmd;
+
+    return if ($post_ktest_done);
+
+    if (defined($final_post_ktest)) {
+	$cmd = $final_post_ktest;
+    } elsif (defined($post_ktest)) {
+	$cmd = $post_ktest;
+    } else {
+	return;
+    }
+
+    my $cp_post_ktest = eval_kernel_version($cmd);
+    run_command $cp_post_ktest;
+    $post_ktest_done = 1;
+}
+
 sub dodie {
     # avoid recursion
     return if ($in_die);
@@ -1645,6 +1664,7 @@ sub dodie {
     if (defined($post_test)) {
 	run_command $post_test;
     }
+    run_post_ktest;
 
     die @_, "\n";
 }
@@ -4314,6 +4334,7 @@ sub cancel_test {
 	send_email("KTEST: Your [$name] test was cancelled",
 	    "Your test started at $script_start_time was cancelled: sig int");
     }
+    run_post_ktest;
     die "\nCaught Sig Int, test interrupted: $!\n"
 }
 
@@ -4679,11 +4700,7 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     success $i;
 }
 
-if (defined($final_post_ktest)) {
-
-    my $cp_final_post_ktest = eval_kernel_version $final_post_ktest;
-    run_command $cp_final_post_ktest;
-}
+run_post_ktest;
 
 if ($opt{"POWEROFF_ON_SUCCESS"}) {
     halt;

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH 9/9] ktest: Add a --dry-run mode
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (7 preceding siblings ...)
  2026-03-07 22:08 ` [PATCH 8/9] ktest: Run POST_KTEST hooks on failure and cancellation Ricardo B. Marlière
@ 2026-03-07 22:08 ` Ricardo B. Marlière
  2026-03-09 14:21 ` [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Steven Rostedt
  9 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-07 22:08 UTC (permalink / raw)
  To: Steven Rostedt, John Hawley
  Cc: Andrea Righi, Marcos Paulo de Souza, Matthieu Baerts,
	Fernando Fernandez Mancera, Pedro Falcato, linux-kernel,
	Steven Rostedt, Ricardo B. Marlière

When working on a ktest configuration, it is often useful to inspect the
final option values after includes, defaults, per-test overrides, and
variable expansion have been applied, without actually starting a test run.

Add a --dry-run option that reads the configuration, prints the test
preamble using resolved option values, and exits before opening LOG_FILE or
executing any test logic.

This is useful for debugging ktest configurations and for scripts that need
to validate the final resolved settings without triggering side effects.

Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
---
 tools/testing/ktest/ktest.pl | 89 +++++++++++++++++++++++++++++++-------------
 1 file changed, 63 insertions(+), 26 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 8962857ce4a6..de99b82d16ad 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -85,6 +85,7 @@ my %default = (
 );
 
 my $test_log_start = 0;
+my $dry_run = 0;
 
 my $ktest_config = "ktest.conf";
 my $version;
@@ -587,7 +588,7 @@ sub end_monitor;
 sub wait_for_monitor;
 
 sub _logit {
-    if (defined($opt{"LOG_FILE"})) {
+    if (defined($opt{"LOG_FILE"}) && defined(fileno(LOG))) {
 	print LOG @_;
     }
 }
@@ -1365,6 +1366,9 @@ sub read_config {
 	    print "$option\n";
 	}
 	print "Set IGNORE_UNUSED = 1 to have ktest ignore unused variables\n";
+	if ($dry_run) {
+	    return;
+	}
 	if (!read_yn "Do you want to continue?") {
 	    exit -1;
 	}
@@ -4249,6 +4253,53 @@ sub set_test_option {
     return eval_option($name, $option, $i);
 }
 
+sub print_test_preamble {
+    my ($resolved) = @_;
+
+    doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
+
+    for (my $i = 0, my $repeat = 1; $i <= $opt{"NUM_TESTS"}; $i += $repeat) {
+
+	if (!$i) {
+	    doprint "DEFAULT OPTIONS:\n";
+	} else {
+	    doprint "\nTEST $i OPTIONS";
+	    if (defined($repeat_tests{$i})) {
+		$repeat = $repeat_tests{$i};
+		doprint " ITERATE $repeat";
+	    }
+	    doprint "\n";
+	}
+
+	foreach my $option (sort keys %opt) {
+	    my $value;
+
+	    if ($option =~ /\[(\d+)\]$/) {
+		next if ($i != $1);
+
+		if ($resolved) {
+		    my $name = $option;
+		    $name =~ s/\[\d+\]$//;
+		    $value = set_test_option($name, $i);
+		} else {
+		    $value = $opt{$option};
+		}
+	    } else {
+		next if ($i);
+
+		if ($resolved) {
+		    $value = set_test_option($option, 0);
+		} else {
+		    $value = $opt{$option};
+		}
+	    }
+
+	    $value = "" if (!defined($value));
+	    doprint "$option = $value\n";
+	}
+    }
+}
+
 sub find_mailer {
     my ($mailer) = @_;
 
@@ -4348,6 +4399,8 @@ ktest.pl version: $VERSION
                     Sets global BUILD_NOCLEAN to 1
                 -D TEST_TYPE[2]=build
                     Sets TEST_TYPE of test 2 to "build"
+       --dry-run
+                Print resolved test options and exit without running tests.
 
 	        It can also override all temp variables.
                  -D USE_TEMP_DIR:=1
@@ -4379,6 +4432,9 @@ while ( $#ARGV >= 0 ) {
 	} else {
 	    $command_vars[$#command_vars + 1] = $val;
 	}
+    } elsif ( $ARGV[0] eq "--dry-run" ) {
+	$dry_run = 1;
+	shift;
     } elsif ( $ARGV[0] eq "-h" ) {
 	die_usage;
     } else {
@@ -4427,6 +4483,11 @@ EOF
 }
 read_config $ktest_config;
 
+if ($dry_run) {
+    print_test_preamble 1;
+    exit 0;
+}
+
 if (defined($opt{"LOG_FILE"})) {
     $opt{"LOG_FILE"} = set_test_option("LOG_FILE", 1);
 }
@@ -4458,31 +4519,7 @@ if (defined($opt{"LOG_FILE"})) {
     LOG->autoflush(1);
 }
 
-doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
-
-for (my $i = 0, my $repeat = 1; $i <= $opt{"NUM_TESTS"}; $i += $repeat) {
-
-    if (!$i) {
-	doprint "DEFAULT OPTIONS:\n";
-    } else {
-	doprint "\nTEST $i OPTIONS";
-	if (defined($repeat_tests{$i})) {
-	    $repeat = $repeat_tests{$i};
-	    doprint " ITERATE $repeat";
-	}
-	doprint "\n";
-    }
-
-    foreach my $option (sort keys %opt) {
-	if ($option =~ /\[(\d+)\]$/) {
-	    next if ($i != $1);
-	} else {
-	    next if ($i);
-	}
-
-	doprint "$option = $opt{$option}\n";
-    }
-}
+print_test_preamble 0;
 
 $SIG{INT} = qw(cancel_test);
 

-- 
2.53.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run
  2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
                   ` (8 preceding siblings ...)
  2026-03-07 22:08 ` [PATCH 9/9] ktest: Add a --dry-run mode Ricardo B. Marlière
@ 2026-03-09 14:21 ` Steven Rostedt
  9 siblings, 0 replies; 14+ messages in thread
From: Steven Rostedt @ 2026-03-09 14:21 UTC (permalink / raw)
  To: Ricardo B. Marlière
  Cc: John Hawley, Andrea Righi, Marcos Paulo de Souza,
	Matthieu Baerts, Fernando Fernandez Mancera, Pedro Falcato,
	linux-kernel, Steven Rostedt

On Sat, 07 Mar 2026 19:07:55 -0300
Ricardo B. Marlière <rbm@suse.com> wrote:

> Hi Steven,

Hi Ricardo,

> 
> Please consider the following ktest patches.
> 
> They fix a few edge cases that I found while using ktest with a small
> virtme-ng-based harness [1], and the last two add small features:
> PRE_KTEST_DIE and a new --dry-run 

Thanks, I'll do a review and apply it to my local tree. As I use ktest
daily, I always run updates locally for a while before pushing it up, as my
normal day to day usage can find subtle bugs.

-- Steve

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 3/9] ktest: Treat undefined self-reference as empty
  2026-03-07 22:07 ` [PATCH 3/9] ktest: Treat undefined self-reference as empty Ricardo B. Marlière
@ 2026-03-09 14:40   ` Steven Rostedt
  0 siblings, 0 replies; 14+ messages in thread
From: Steven Rostedt @ 2026-03-09 14:40 UTC (permalink / raw)
  To: Ricardo B. Marlière
  Cc: John Hawley, Andrea Righi, Marcos Paulo de Souza,
	Matthieu Baerts, Fernando Fernandez Mancera, Pedro Falcato,
	linux-kernel, Steven Rostedt

On Sat, 07 Mar 2026 19:07:58 -0300
Ricardo B. Marlière <rbm@suse.com> wrote:

> Config variables are expanded when they are assigned. A first-time append
> such as:
> 
>   VAR := ${VAR} foo
> 
> leaves the literal ${VAR} in the stored value because VAR has not been
> defined yet. Later expansions then carry the self-reference forward instead
> of behaving like an empty prefix.
> 
> Drop an unescaped self-reference when the variable has no current value,
> and trim the outer whitespace left behind. Keep escaped \${VAR} references
> unchanged so literal text still works.
> 
> Signed-off-by: Ricardo B. Marlière <rbm@suse.com>
> ---
>  tools/testing/ktest/ktest.pl | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
> index b7a1c8c617e0..b8fcdabffffe 100755
> --- a/tools/testing/ktest/ktest.pl
> +++ b/tools/testing/ktest/ktest.pl
> @@ -910,6 +910,14 @@ sub set_variable {
>      if (defined($command_tmp_vars{$lvalue})) {
>  	return;
>      }
> +
> +    # If a variable is undefined, treat an unescaped self-reference as empty.
> +    if (!defined($variable{$lvalue})) {
> +	$rvalue =~ s/(?<!\\)\$\{\Q$lvalue\E\}//g;

So today I learned about "negative look behind" "?<!" ;-)

-- Steve

> +	$rvalue =~ s/^\s+//;
> +	$rvalue =~ s/\s+$//;
> +    }
> +
>      if ($rvalue =~ /^\s*$/) {
>  	delete $variable{$lvalue};
>      } else {
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
  2026-03-07 22:08 ` [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures Ricardo B. Marlière
@ 2026-03-09 15:00   ` Steven Rostedt
  2026-03-10 10:25     ` Ricardo B. Marlière
  0 siblings, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2026-03-09 15:00 UTC (permalink / raw)
  To: Ricardo B. Marlière
  Cc: John Hawley, Andrea Righi, Marcos Paulo de Souza,
	Matthieu Baerts, Fernando Fernandez Mancera, Pedro Falcato,
	linux-kernel, Steven Rostedt

On Sat, 07 Mar 2026 19:08:02 -0300
Ricardo B. Marlière <rbm@suse.com> wrote:

> PRE_KTEST runs before the first test, but its return status is currently
> ignored. A failing setup hook can leave the rest of the run executing in a
> partially initialized environment.
> 
> Add PRE_KTEST_DIE so PRE_KTEST can fail the run in the same way
> PRE_BUILD_DIE and PRE_TEST_DIE already can. Keep the default behavior
> unchanged when the new option is not set.

Thank you for keeping the previous behavior. My PRE_KTEST usually apply
patches that don't always apply, and yes, I depend on it "not dying" on
failure. Which is probably why that was the way it worked, as the reason I
created that option was to apply patches before testing.

-- Steve

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures
  2026-03-09 15:00   ` Steven Rostedt
@ 2026-03-10 10:25     ` Ricardo B. Marlière
  0 siblings, 0 replies; 14+ messages in thread
From: Ricardo B. Marlière @ 2026-03-10 10:25 UTC (permalink / raw)
  To: Steven Rostedt, Ricardo B. Marlière
  Cc: John Hawley, Andrea Righi, Marcos Paulo de Souza,
	Matthieu Baerts, Fernando Fernandez Mancera, Pedro Falcato,
	linux-kernel, Steven Rostedt

On Mon Mar 9, 2026 at 12:00 PM -03, Steven Rostedt wrote:
> On Sat, 07 Mar 2026 19:08:02 -0300
> Ricardo B. Marlière <rbm@suse.com> wrote:
>
>> PRE_KTEST runs before the first test, but its return status is currently
>> ignored. A failing setup hook can leave the rest of the run executing in a
>> partially initialized environment.
>> 
>> Add PRE_KTEST_DIE so PRE_KTEST can fail the run in the same way
>> PRE_BUILD_DIE and PRE_TEST_DIE already can. Keep the default behavior
>> unchanged when the new option is not set.
>
> Thank you for keeping the previous behavior. My PRE_KTEST usually apply
> patches that don't always apply, and yes, I depend on it "not dying" on
> failure. Which is probably why that was the way it worked, as the reason I
> created that option was to apply patches before testing.

I figured this behavior had to be there for a reason :)

Thanks for reviewing/testing btw !

>
> -- Steve


^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2026-03-10 10:25 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-03-07 22:07 [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Ricardo B. Marlière
2026-03-07 22:07 ` [PATCH 1/9] ktest: Avoid undef warning when WARNINGS_FILE is unset Ricardo B. Marlière
2026-03-07 22:07 ` [PATCH 2/9] ktest: Resolve LOG_FILE in test option context Ricardo B. Marlière
2026-03-07 22:07 ` [PATCH 3/9] ktest: Treat undefined self-reference as empty Ricardo B. Marlière
2026-03-09 14:40   ` Steven Rostedt
2026-03-07 22:07 ` [PATCH 4/9] ktest: Honor empty per-test option overrides Ricardo B. Marlière
2026-03-07 22:08 ` [PATCH 5/9] ktest: Run commands through list-form shell open Ricardo B. Marlière
2026-03-07 22:08 ` [PATCH 6/9] ktest: Stop dropping console output during power-cycle reboot Ricardo B. Marlière
2026-03-07 22:08 ` [PATCH 7/9] ktest: Add PRE_KTEST_DIE for PRE_KTEST failures Ricardo B. Marlière
2026-03-09 15:00   ` Steven Rostedt
2026-03-10 10:25     ` Ricardo B. Marlière
2026-03-07 22:08 ` [PATCH 8/9] ktest: Run POST_KTEST hooks on failure and cancellation Ricardo B. Marlière
2026-03-07 22:08 ` [PATCH 9/9] ktest: Add a --dry-run mode Ricardo B. Marlière
2026-03-09 14:21 ` [PATCH 0/9] tools/testing/ktest: Fixes and a new --dry-run Steven Rostedt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

Powered by JetHome