* [PATCH 0/3] selftests: add option to print the time spent per test
@ 2026-11-18 13:40 Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière
Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)
For CI/CD purposes it is useful to know how much time each selftest
took to complete, for instance, to know which suites to run for a
quick-fail diagnostic, or what percentage of tests you can expect to
complete on a paricular --override-timeout value.
Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
prints a KTAP diagnostic line with the elapsed wall-clock time just
before the result line of each test that is actually run, e.g.:
# selftests: timers: nanosleep
# time: 12.34s
ok 3 selftests: timers: nanosleep
The other two patches clean up the use of 'local' in runner.sh and fix a
bug when using --netns in run_kselftest.sh. Strictly these could be sent
as separate patches, but "selftests: add option to print the time spent
per test" builds on these, and it would otherwise require needless
rebasing for two small fixes.
Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
Jori Koolstra (CIQ) (3):
selftests: use local variables consistently in runner.sh
selftests: pass the timeout override to tests run in a netns
selftests: add option to print the time spent per test
tools/testing/selftests/kselftest/runner.sh | 50 ++++++++++++++++++++++-------
tools/testing/selftests/run_kselftest.sh | 8 +++++
2 files changed, 47 insertions(+), 11 deletions(-)
---
base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
change-id: 20261117-kselftest-run-timer-df3ac531652e
Best regards,
--
Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/3] selftests: use local variables consistently in runner.sh
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
2026-11-18 16:13 ` Shuah Khan
2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
` (2 subsequent siblings)
3 siblings, 1 reply; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière
Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)
run_one() and run_many() assign their working variables without
declaring them local, so they leak into the global namespace of
whichever shell sources runner.sh, i.e. run_kselftest.sh or the
RUN_TESTS recipe in lib.mk.
Declare these variables local so that local is used consistently
throughout the file and the sourcing shell's namespace is not polluted.
This does not add a new requirement on the shell: local is already used
in run_many(), in_netns() and run_in_netns().
Also, obey the convention to use upper case names for variables that are
constant once assigned.
Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
tools/testing/selftests/kselftest/runner.sh | 29 ++++++++++++++++++-----------
1 file changed, 18 insertions(+), 11 deletions(-)
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 45fb8ec38493..9b1dad5441ea 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -58,13 +58,18 @@ tap_timeout()
run_one()
{
+ local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
+ local line field value interpreter stdbuf cmd rc
+
DIR="$1"
TEST="$2"
BASENAME_TEST=$(basename $TEST)
# Reset any "settings"-file variables.
- export kselftest_timeout="$kselftest_default_timeout"
+ local kselftest_timeout="$kselftest_default_timeout"
+ # Make kselftest_timeout variables available to tests
+ export kselftest_timeout
# Safe default if tr not available
kselftest_cmd_args_ref="KSELFTEST_ARGS"
@@ -91,8 +96,8 @@ run_one()
fi
# Load per-test-directory kselftest "settings" file.
- settings="$BASE_DIR/$DIR/settings"
- if [ -r "$settings" ] ; then
+ SETTINGS="$BASE_DIR/$DIR/settings"
+ if [ -r "$SETTINGS" ] ; then
while read line ; do
# Skip comments.
if echo "$line" | grep -q '^#'; then
@@ -100,8 +105,8 @@ run_one()
fi
field=$(echo "$line" | cut -d= -f1)
value=$(echo "$line" | cut -d= -f2-)
- eval "kselftest_$field"="$value"
- done < "$settings"
+ eval "local kselftest_$field"="$value"
+ done < "$SETTINGS"
fi
# Command line timeout overrides the settings file
@@ -164,8 +169,8 @@ run_one()
in_netns()
{
- local name=$1
- ip netns exec $name bash <<-EOF
+ local NAME=$1
+ ip netns exec $NAME bash <<-EOF
BASE_DIR=$BASE_DIR
source $BASE_DIR/kselftest/runner.sh
logfile=$logfile
@@ -175,9 +180,9 @@ in_netns()
run_in_netns()
{
- local tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
- local netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
- local rc
+ local tmplog netns rc
+ tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
+ netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
ip netns add $netns
if [ $? -ne 0 ]; then
@@ -198,8 +203,10 @@ run_in_netns()
run_many()
{
+ local DIR TEST BASENAME_TEST
+ local pids pid rc
+
DIR="${PWD#${BASE_DIR}/}"
- local rc
pids=
for TEST in "$@"; do
--
2.56.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/3] selftests: pass the timeout override to tests run in a netns
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan
3 siblings, 0 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière
Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)
When run_kselftest.sh is invoked with -n, each test is run by in_netns()
in a new bash process, which sources runner.sh again and then calls
run_one(). Sourcing runner.sh resets its global settings, which is why
in_netns() explicitly passes on logfile. kselftest_override_timeout is
not passed on, so -o/--override-timeout is silently ignored for tests
that are run in a network namespace and they use the timeout from their
settings file or the default instead.
Pass kselftest_override_timeout to the netns shell as well.
Fixes: 9d0b4ad82d61 ("kselftest/runner.sh: add netns support")
Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
tools/testing/selftests/kselftest/runner.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 9b1dad5441ea..06748af6fcf7 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -174,6 +174,7 @@ in_netns()
BASE_DIR=$BASE_DIR
source $BASE_DIR/kselftest/runner.sh
logfile=$logfile
+ kselftest_override_timeout=$kselftest_override_timeout
run_one $DIR $TEST
EOF
}
--
2.56.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 3/3] selftests: add option to print the time spent per test
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
@ 2026-11-18 13:40 ` Jori Koolstra (CIQ)
2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan
3 siblings, 0 replies; 6+ messages in thread
From: Jori Koolstra (CIQ) @ 2026-11-18 13:40 UTC (permalink / raw)
To: Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière
Cc: linux-kselftest, linux-kernel, Jori Koolstra (CIQ)
There is currently no way to see how long each test took when running
kselftests through run_kselftest.sh. This information is useful for
CI/CD systems.
Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
prints a KTAP diagnostic line with the elapsed wall-clock time just
before the result line of each test that is actually run, e.g.:
# selftests: timers: nanosleep
# time: 12.34s
ok 3 selftests: timers: nanosleep
Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
---
tools/testing/selftests/kselftest/runner.sh | 22 +++++++++++++++++++++-
tools/testing/selftests/run_kselftest.sh | 8 ++++++++
2 files changed, 29 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 06748af6fcf7..60e3bba518ed 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -16,6 +16,7 @@ export timeout_rc=124
export logfile=/dev/stdout
export per_test_logging=
export per_test_log_dir=/tmp
+export per_test_timing=
export RUN_IN_NETNS=
# Defaults for "settings" file fields:
@@ -56,10 +57,20 @@ tap_timeout()
fi
}
+# Print the system uptime in hundredths of a second.
+uptime_cs()
+{
+ local up idle
+
+ read up idle < /proc/uptime
+ # Prefix the fraction with 1 so that a leading 0 isn't parsed as octal.
+ echo $((${up%.*} * 100 + 1${up#*.} - 100))
+}
+
run_one()
{
local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
- local line field value interpreter stdbuf cmd rc
+ local line field value interpreter stdbuf cmd rc start elapsed
DIR="$1"
TEST="$2"
@@ -145,10 +156,18 @@ run_one()
fi
fi
cd `dirname $TEST` > /dev/null
+ if [ -n "$per_test_timing" ]; then
+ start=$(uptime_cs)
+ fi
(((( tap_timeout "$cmd" 2>&1; echo $? >&3) |
tap_prefix >&4) 3>&1) |
(read xs; exit $xs)) 4>>"$logfile"
rc=$?
+ if [ -n "$per_test_timing" ]; then
+ elapsed=$(($(uptime_cs) - start))
+ ktap_print_msg "$(printf 'time: %d.%02ds' \
+ $((elapsed / 100)) $((elapsed % 100)))"
+ fi
case "$rc" in
"$KSFT_PASS")
ktap_test_pass "$TEST_HDR_MSG";;
@@ -175,6 +194,7 @@ in_netns()
source $BASE_DIR/kselftest/runner.sh
logfile=$logfile
kselftest_override_timeout=$kselftest_override_timeout
+ per_test_timing=$per_test_timing
run_one $DIR $TEST
EOF
}
diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
index e11b0e93ba41..0f2a8c619a9c 100755
--- a/tools/testing/selftests/run_kselftest.sh
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -32,6 +32,7 @@ Usage: $0 [OPTIONS]
-n | --netns Run each test in namespace
-h | --help Show this usage info
-o | --override-timeout Number of seconds after which we timeout
+ -T | --timed Print the time spent on each test
EOF
exit $1
}
@@ -101,6 +102,13 @@ while true; do
-o | --override-timeout)
kselftest_override_timeout="$2"
shift 2 ;;
+ -T | --timed)
+ if [ ! -r /proc/uptime ]; then
+ echo "Timing tests requires /proc/uptime" >&2
+ exit 1
+ fi
+ per_test_timing=1
+ shift ;;
-h | --help)
usage 0 ;;
"")
--
2.56.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/3] selftests: use local variables consistently in runner.sh
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
@ 2026-11-18 16:13 ` Shuah Khan
0 siblings, 0 replies; 6+ messages in thread
From: Shuah Khan @ 2026-11-18 16:13 UTC (permalink / raw)
To: Jori Koolstra (CIQ),
Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière
Cc: linux-kselftest, linux-kernel, Shuah Khan
On 11/18/26 06:40, Jori Koolstra (CIQ) wrote:
> run_one() and run_many() assign their working variables without
> declaring them local, so they leak into the global namespace of
> whichever shell sources runner.sh, i.e. run_kselftest.sh or the
> RUN_TESTS recipe in lib.mk.
>
Hmm. so if a timeout value of say 60 seconds comes from a test
settings file, does it persist? Can you see that if you run a
few tests using TARGETS variable?
> Declare these variables local so that local is used consistently
> throughout the file and the sourcing shell's namespace is not polluted.
> This does not add a new requirement on the shell: local is already used
> in run_many(), in_netns() and run_in_netns().
Makes sense.
>
> Also, obey the convention to use upper case names for variables that are
> constant once assigned.
Can you elaborate on this convention?
I don't want to see these changes combined. We don't combine
cleanup and convention changes with real changes.
>
> Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
> ---
> tools/testing/selftests/kselftest/runner.sh | 29 ++++++++++++++++++-----------
> 1 file changed, 18 insertions(+), 11 deletions(-)
>
> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> index 45fb8ec38493..9b1dad5441ea 100644
> --- a/tools/testing/selftests/kselftest/runner.sh
> +++ b/tools/testing/selftests/kselftest/runner.sh
> @@ -58,13 +58,18 @@ tap_timeout()
>
> run_one()
> {
> + local DIR TEST BASENAME_TEST BASENAME_SANITIZED SETTINGS TEST_HDR_MSG
> + local line field value interpreter stdbuf cmd rc
> +
> DIR="$1"
> TEST="$2"
>
> BASENAME_TEST=$(basename $TEST)
>
> # Reset any "settings"-file variables.
> - export kselftest_timeout="$kselftest_default_timeout"
> + local kselftest_timeout="$kselftest_default_timeout"
> + # Make kselftest_timeout variables available to tests
> + export kselftest_timeout
>
> # Safe default if tr not available
> kselftest_cmd_args_ref="KSELFTEST_ARGS"
> @@ -91,8 +96,8 @@ run_one()
> fi
>
> # Load per-test-directory kselftest "settings" file.
> - settings="$BASE_DIR/$DIR/settings"
> - if [ -r "$settings" ] ; then
> + SETTINGS="$BASE_DIR/$DIR/settings"
> + if [ -r "$SETTINGS" ] ; then
> while read line ; do
> # Skip comments.
> if echo "$line" | grep -q '^#'; then
> @@ -100,8 +105,8 @@ run_one()
> fi
> field=$(echo "$line" | cut -d= -f1)
> value=$(echo "$line" | cut -d= -f2-)
> - eval "kselftest_$field"="$value"
> - done < "$settings"
> + eval "local kselftest_$field"="$value"
> + done < "$SETTINGS"
> fi
>
> # Command line timeout overrides the settings file
> @@ -164,8 +169,8 @@ run_one()
>
> in_netns()
> {
> - local name=$1
> - ip netns exec $name bash <<-EOF
> + local NAME=$1
> + ip netns exec $NAME bash <<-EOF
> BASE_DIR=$BASE_DIR
> source $BASE_DIR/kselftest/runner.sh
> logfile=$logfile
> @@ -175,9 +180,9 @@ in_netns()
>
> run_in_netns()
> {
> - local tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
> - local netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
> - local rc
> + local tmplog netns rc
> + tmplog="/tmp/$(mktemp -u ${BASENAME_TEST}-XXXXXX)"
> + netns=$(mktemp -u ${BASENAME_TEST}-XXXXXX)
>
> ip netns add $netns
> if [ $? -ne 0 ]; then
> @@ -198,8 +203,10 @@ run_in_netns()
>
> run_many()
> {
> + local DIR TEST BASENAME_TEST
> + local pids pid rc
> +
> DIR="${PWD#${BASE_DIR}/}"
> - local rc
> pids=
>
> for TEST in "$@"; do
>
thanks,
-- Shuah
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 0/3] selftests: add option to print the time spent per test
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
` (2 preceding siblings ...)
2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
@ 2026-11-18 16:15 ` Shuah Khan
3 siblings, 0 replies; 6+ messages in thread
From: Shuah Khan @ 2026-11-18 16:15 UTC (permalink / raw)
To: Jori Koolstra (CIQ),
Shuah Khan, David Ahern, David S. Miller, Hangbin Liu,
Ricardo B. Marlière, Shuah Khan
Cc: linux-kselftest, linux-kernel
On 11/18/26 06:40, Jori Koolstra (CIQ) wrote:
> For CI/CD purposes it is useful to know how much time each selftest
> took to complete, for instance, to know which suites to run for a
> quick-fail diagnostic, or what percentage of tests you can expect to
> complete on a paricular --override-timeout value.
>
> Add a -T/--timed option to run_kselftest.sh. When it is set, run_one()
> prints a KTAP diagnostic line with the elapsed wall-clock time just
> before the result line of each test that is actually run, e.g.:
>
> # selftests: timers: nanosleep
> # time: 12.34s
> ok 3 selftests: timers: nanosleep
>
> The other two patches clean up the use of 'local' in runner.sh and fix a
> bug when using --netns in run_kselftest.sh. Strictly these could be sent
> as separate patches, but "selftests: add option to print the time spent
> per test" builds on these, and it would otherwise require needless
> rebasing for two small fixes.
>
> Signed-off-by: Jori Koolstra (CIQ) <jkoolstra@xs4all.nl>
> ---
> Jori Koolstra (CIQ) (3):
> selftests: use local variables consistently in runner.sh
Please spilt the convention related changes and real changes.
> selftests: pass the timeout override to tests run in a netns
> selftests: add option to print the time spent per test
These two look good to me. I will pull them all as a series once you
fix the problem with patch 1/3
>
> tools/testing/selftests/kselftest/runner.sh | 50 ++++++++++++++++++++++-------
> tools/testing/selftests/run_kselftest.sh | 8 +++++
> 2 files changed, 47 insertions(+), 11 deletions(-)
> ---
> base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4
> change-id: 20261117-kselftest-run-timer-df3ac531652e
>
> Best regards,
thanks,
-- Shuah
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-18 16:15 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 13:40 [PATCH 0/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 1/3] selftests: use local variables consistently in runner.sh Jori Koolstra (CIQ)
2026-11-18 16:13 ` Shuah Khan
2026-11-18 13:40 ` [PATCH 2/3] selftests: pass the timeout override to tests run in a netns Jori Koolstra (CIQ)
2026-11-18 13:40 ` [PATCH 3/3] selftests: add option to print the time spent per test Jori Koolstra (CIQ)
2026-11-18 16:15 ` [PATCH 0/3] " Shuah Khan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®