mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH 0/3] selftests: add option to print the time spent per test
@ 2026-11-18 13:40 Jori Koolstra (CIQ)
  2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
  To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière
  Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)

For CI/CD purposes it is useful to know how much time each selftest
took to complete, for instance, to know which suites to run for a
quick-fail diagnostic, or what percentage of tests you can expect to
complete on a paricular --override-timeout value.

Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
prints a KTAP diagnostic line with the elapsed wall-clock time just
before the result line of each test that is actually run, e.g.:

      # selftests: timers: nanosleep
      # time: 12.34s
      ok 3 selftests: timers: nanosleep

The other two patches clean up the use of 'local' in runner.sh and fix a
bug when using --netns in run_kselftest.sh. Strictly these could be sent
as separate patches, but "selftests: add option to print the time spent
per test" builds on these, and it would otherwise require needless
rebasing for two small fixes.

Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
Jori Koolstra (CIQ) (3):
      selftests: use local variables consistently in runner.sh
      selftests: pass the timeout override to tests run in a netns
      selftests: add option to print the time spent per test

 tools/testing/selftests/kselftest/runner.sh | 50 ++++++++++++++++++++++-------
 tools/testing/selftests/run_kselftest.sh    |  8 +++++
 2 files changed, 47 insertions(+), 11 deletions(-)
---
base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
change-id: 20261117-kselftest-run-timer-df3ac531652e

Best regards,
-- 
Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 1/3] selftests: use local variables consistently in runner.sh
  2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
  2026-11-18 16:13   ` Shuah Khan
  2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
  To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière
  Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)

run_one() and run_many() assign their working variables without
declaring them local, so they leak into the global namespace of
whichever shell sources runner.sh, i.e. run_kselftest.sh or the
RUN_TESTS recipe in lib.mk.

Declare these variables local so that local is used consistently
throughout the file and the sourcing shell's namespace is not polluted.
This does not add a new requirement on the shell: local is already used
in run_many(), in_netns() and run_in_netns().

Also, obey the convention to use upper case names for variables that are
constant once assigned.

Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
 tools/testing/selftests/kselftest/runner.sh | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 45fb8ec38493..9b1dad5441ea 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -58,13 +58,18 @@ tap_timeout()
 
 run_one()
 {
+	local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
+	local line field value interpreter stdbuf cmd rc
+
 	DIR="$1"
 	TEST="$2"
 
 	BASENAME_TEST=$(basename $TEST)
 
 	# Reset any "settings"-file variables.
-	export kselftest_timeout="$kselftest_default_timeout"
+	local kselftest_timeout="$kselftest_default_timeout"
+	# Make kselftest_timeout variables available to tests
+	export kselftest_timeout
 
 	# Safe default if tr not available
 	kselftest_cmd_args_ref="KSELFTEST_ARGS"
@@ -91,8 +96,8 @@ run_one()
 	fi
 
 	# Load per-test-directory kselftest "settings" file.
-	settings="$BASE_DIR/$DIR/settings"
-	if [ -r "$settings" ] ; then
+	SETTINGS="$BASE_DIR/$DIR/settings"
+	if [ -r "$SETTINGS" ] ; then
 		while read line ; do
 			# Skip comments.
 			if echo "$line" | grep -q '^#'; then
@@ -100,8 +105,8 @@ run_one()
 			fi
 			field=$(echo "$line" | cut -d= -f1)
 			value=$(echo "$line" | cut -d= -f2-)
-			eval "kselftest_$field"="$value"
-		done < "$settings"
+			eval "local kselftest_$field"="$value"
+		done < "$SETTINGS"
 	fi
 
 	# Command line timeout overrides the settings file
@@ -164,8 +169,8 @@ run_one()
 
 in_netns()
 {
-	local name=$1
-	ip netns exec $name bash <<-EOF
+	local NAME=$1
+	ip netns exec $NAME bash <<-EOF
 		BASE_DIR=$BASE_DIR
 		source $BASE_DIR/kselftest/runner.sh
 		logfile=$logfile
@@ -175,9 +180,9 @@ in_netns()
 
 run_in_netns()
 {
-	local tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
-	local netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
-	local rc
+	local tmplog netns rc
+	tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
+	netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
 
 	ip netns add $netns
 	if [ $? -ne 0 ]; then
@@ -198,8 +203,10 @@ run_in_netns()
 
 run_many()
 {
+	local DIR TEST BASENAME_TEST
+	local pids pid rc
+
 	DIR="${PWD#${BASE_DIR}/}"
-	local rc
 	pids=
 
 	for TEST in "$@"; do

-- 
2.56.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 2/3] selftests: pass the timeout override to tests run in a netns
  2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
  2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
  2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
  2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan
  3 siblings, 0 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
  To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière
  Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)

When run_kselftest.sh is invoked with -n, each test is run by in_netns()
in a new bash process, which sources runner.sh again and then calls
run_one(). Sourcing runner.sh resets its global settings, which is why
in_netns() explicitly passes on logfile. kselftest_override_timeout is
not passed on, so -o/--override-timeout is silently ignored for tests
that are run in a network namespace and they use the timeout from their
settings file or the default instead.

Pass kselftest_override_timeout to the netns shell as well.

Fixes: 9d0b4ad82d61 ("kselftest/runner.sh: add netns support")
Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
 tools/testing/selftests/kselftest/runner.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 9b1dad5441ea..06748af6fcf7 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -174,6 +174,7 @@ in_netns()
 		BASE_DIR=$BASE_DIR
 		source $BASE_DIR/kselftest/runner.sh
 		logfile=$logfile
+		kselftest_override_timeout=$kselftest_override_timeout
 		run_one $DIR $TEST
 	EOF
 }

-- 
2.56.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 3/3] selftests: add option to print the time spent per test
  2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
  2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
  2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
  2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan
  3 siblings, 0 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
  To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière
  Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)

There is currently no way to see how long each test took when running
kselftests through run_kselftest.sh. This information is useful for
CI/CD systems.

Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
prints a KTAP diagnostic line with the elapsed wall-clock time just
before the result line of each test that is actually run, e.g.:

  # selftests: timers: nanosleep
  # time: 12.34s
  ok 3 selftests: timers: nanosleep

Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
 tools/testing/selftests/kselftest/runner.sh | 22 +++++++++++++++++++++-
 tools/testing/selftests/run_kselftest.sh    |  8 ++++++++
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 06748af6fcf7..60e3bba518ed 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -16,6 +16,7 @@ export timeout_rc=124
 export logfile=/dev/stdout
 export per_test_logging=
 export per_test_log_dir=/tmp
+export per_test_timing=
 export RUN_IN_NETNS=
 
 # Defaults for "settings" file fields:
@@ -56,10 +57,20 @@ tap_timeout()
 	fi
 }
 
+# Print the system uptime in hundredths of a second.
+uptime_cs()
+{
+	local up idle
+
+	read up idle < /proc/uptime
+	# Prefix the fraction with 1 so that a leading 0 isn't parsed as octal.
+	echo $((${up%.*} * 100 + 1${up#*.} - 100))
+}
+
 run_one()
 {
 	local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
-	local line field value interpreter stdbuf cmd rc
+	local line field value interpreter stdbuf cmd rc start elapsed
 
 	DIR="$1"
 	TEST="$2"
@@ -145,10 +156,18 @@ run_one()
 			fi
 		fi
 		cd `dirname $TEST` > /dev/null
+		if [ -n "$per_test_timing" ]; then
+			start=$(uptime_cs)
+		fi
 		(((( tap_timeout "$cmd" 2>&1; echo $? >&3) |
 			tap_prefix >&4) 3>&1) |
 			(read xs; exit $xs)) 4>>"$logfile"
 		rc=$?
+		if [ -n "$per_test_timing" ]; then
+			elapsed=$(($(uptime_cs) - start))
+			ktap_print_msg "$(printf 'time: %d.%02ds' \
+				$((elapsed / 100)) $((elapsed % 100)))"
+		fi
 		case "$rc" in
 		"$KSFT_PASS")
 			ktap_test_pass "$TEST_HDR_MSG";;
@@ -175,6 +194,7 @@ in_netns()
 		source $BASE_DIR/kselftest/runner.sh
 		logfile=$logfile
 		kselftest_override_timeout=$kselftest_override_timeout
+		per_test_timing=$per_test_timing
 		run_one $DIR $TEST
 	EOF
 }
diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
index e11b0e93ba41..0f2a8c619a9c 100755
--- a/tools/testing/selftests/run_kselftest.sh
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -32,6 +32,7 @@ Usage: $0 [OPTIONS]
   -n | --netns			Run each test in namespace
   -h | --help			Show this usage info
   -o | --override-timeout	Number of seconds after which we timeout
+  -T | --timed			Print the time spent on each test
 EOF
 	exit $1
 }
@@ -101,6 +102,13 @@ while true; do
 		-o | --override-timeout)
 			kselftest_override_timeout="$2"
 			shift 2 ;;
+		-T | --timed)
+			if [ ! -r /proc/uptime ]; then
+				echo "Timing tests requires /proc/uptime" >&2
+				exit 1
+			fi
+			per_test_timing=1
+			shift ;;
 		-h | --help)
 			usage 0 ;;
 		"")

-- 
2.56.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] selftests: use local variables consistently in runner.sh
  2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
@ 2026-11-18 16:13   ` Shuah Khan
  0 siblings, 0 replies; 6+ messages in thread
From: Shuah Khan @ 2026-11-18 16:13 UTC (permalink / raw)
  To: Jori Koolstra (CIQ),
	Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière
  Cc: linux-kselftest, linux-kernel, Shuah Khan

On 11/18/26 06:40, Jori Koolstra (CIQ) wrote:
> run_one() and run_many() assign their working variables without
> declaring them local, so they leak into the global namespace of
> whichever shell sources runner.sh, i.e. run_kselftest.sh or the
> RUN_TESTS recipe in lib.mk.
> 

Hmm. so if a timeout value of say 60 seconds comes from a test
settings file, does it persist? Can you see that if you run a
few tests using TARGETS variable?

> Declare these variables local so that local is used consistently
> throughout the file and the sourcing shell's namespace is not polluted.
> This does not add a new requirement on the shell: local is already used
> in run_many(), in_netns() and run_in_netns().

Makes sense.

> 
> Also, obey the convention to use upper case names for variables that are
> constant once assigned.

Can you elaborate on this convention?

I don't want to see these changes combined. We don't combine
cleanup and convention changes with real changes.

> 
> Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
> ---
>   tools/testing/selftests/kselftest/runner.sh | 29 ++++++++++++++++++-----------
>   1 file changed, 18 insertions(+), 11 deletions(-)
> 
> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> index 45fb8ec38493..9b1dad5441ea 100644
> --- a/tools/testing/selftests/kselftest/runner.sh
> +++ b/tools/testing/selftests/kselftest/runner.sh
> @@ -58,13 +58,18 @@ tap_timeout()
>   
>   run_one()
>   {
> +	local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
> +	local line field value interpreter stdbuf cmd rc
> +
>   	DIR="$1"
>   	TEST="$2"
>   
>   	BASENAME_TEST=$(basename $TEST)
>   
>   	# Reset any "settings"-file variables.
> -	export kselftest_timeout="$kselftest_default_timeout"
> +	local kselftest_timeout="$kselftest_default_timeout"
> +	# Make kselftest_timeout variables available to tests
> +	export kselftest_timeout
>   
>   	# Safe default if tr not available
>   	kselftest_cmd_args_ref="KSELFTEST_ARGS"
> @@ -91,8 +96,8 @@ run_one()
>   	fi
>   
>   	# Load per-test-directory kselftest "settings" file.
> -	settings="$BASE_DIR/$DIR/settings"
> -	if [ -r "$settings" ] ; then
> +	SETTINGS="$BASE_DIR/$DIR/settings"
> +	if [ -r "$SETTINGS" ] ; then
>   		while read line ; do
>   			# Skip comments.
>   			if echo "$line" | grep -q '^#'; then
> @@ -100,8 +105,8 @@ run_one()
>   			fi
>   			field=$(echo "$line" | cut -d= -f1)
>   			value=$(echo "$line" | cut -d= -f2-)
> -			eval "kselftest_$field"="$value"
> -		done < "$settings"
> +			eval "local kselftest_$field"="$value"
> +		done < "$SETTINGS"
>   	fi
>   
>   	# Command line timeout overrides the settings file
> @@ -164,8 +169,8 @@ run_one()
>   
>   in_netns()
>   {
> -	local name=$1
> -	ip netns exec $name bash <<-EOF
> +	local NAME=$1
> +	ip netns exec $NAME bash <<-EOF
>   		BASE_DIR=$BASE_DIR
>   		source $BASE_DIR/kselftest/runner.sh
>   		logfile=$logfile
> @@ -175,9 +180,9 @@ in_netns()
>   
>   run_in_netns()
>   {
> -	local tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
> -	local netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
> -	local rc
> +	local tmplog netns rc
> +	tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
> +	netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
>   
>   	ip netns add $netns
>   	if [ $? -ne 0 ]; then
> @@ -198,8 +203,10 @@ run_in_netns()
>   
>   run_many()
>   {
> +	local DIR TEST BASENAME_TEST
> +	local pids pid rc
> +
>   	DIR="${PWD#${BASE_DIR}/}"
> -	local rc
>   	pids=
>   
>   	for TEST in "$@"; do
> 

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 0/3] selftests: add option to print the time spent per test
  2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
                   ` (2 preceding siblings ...)
  2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
@ 2026-11-18 16:15 ` Shuah Khan
  3 siblings, 0 replies; 6+ messages in thread
From: Shuah Khan @ 2026-11-18 16:15 UTC (permalink / raw)
  To: Jori Koolstra (CIQ),
	Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
	Ricardo B. Marlière, Shuah Khan
  Cc: linux-kselftest, linux-kernel

On 11/18/26 06:40, Jori Koolstra (CIQ) wrote:
> For CI/CD purposes it is useful to know how much time each selftest
> took to complete, for instance, to know which suites to run for a
> quick-fail diagnostic, or what percentage of tests you can expect to
> complete on a paricular --override-timeout value.
> 
> Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
> prints a KTAP diagnostic line with the elapsed wall-clock time just
> before the result line of each test that is actually run, e.g.:
> 
>        # selftests: timers: nanosleep
>        # time: 12.34s
>        ok 3 selftests: timers: nanosleep
> 
> The other two patches clean up the use of 'local' in runner.sh and fix a
> bug when using --netns in run_kselftest.sh. Strictly these could be sent
> as separate patches, but "selftests: add option to print the time spent
> per test" builds on these, and it would otherwise require needless
> rebasing for two small fixes.
> 
> Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
> ---
> Jori Koolstra (CIQ) (3):
>        selftests: use local variables consistently in runner.sh

Please spilt the convention related changes and real changes.

>        selftests: pass the timeout override to tests run in a netns
>        selftests: add option to print the time spent per test

These two look good to me. I will pull them all as a series once you
fix the problem with patch 1/3

> 
>   tools/testing/selftests/kselftest/runner.sh | 50 ++++++++++++++++++++++-------
>   tools/testing/selftests/run_kselftest.sh    |  8 +++++
>   2 files changed, 47 insertions(+), 11 deletions(-)
> ---
> base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
> change-id: 20261117-kselftest-run-timer-df3ac531652e
> 
> Best regards,

thanks,
-- Shuah

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-11-18 16:15 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
2026-11-18 16:13   ` Shuah Khan
2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®