[PATCH 0/2] rtla: Improve test coverage with output checking and actions

[PATCH 0/2] rtla: Improve test coverage with output checking and actions

[Tomas Glozar]

Recently added actions on threshold/end and checking output with "grep" in tests
enable more accurate testing.

Implement two additions: grep filters for base tests, and a new tests, using
action on threshold to verify if priority of timerlat thread is set correctly.

Tomas Glozar (2):
  rtla/tests: Add grep checks for base test cases
  rtla/tests: Test timerlat -P option using actions

 tools/tracing/rtla/tests/engine.sh                |  2 +-
 tools/tracing/rtla/tests/hwnoise.t                | 11 ++++++-----
 tools/tracing/rtla/tests/osnoise.t                |  6 +++---
 .../tracing/rtla/tests/scripts/check-priority.sh  |  8 ++++++++
 tools/tracing/rtla/tests/timerlat.t               | 15 +++++++++------
 5 files changed, 27 insertions(+), 15 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-priority.sh

-- 
2.49.0

[PATCH 1/2] rtla/tests: Add grep checks for base test cases

[Tomas Glozar]

Checking for patterns in rtla output with grep was added to test rtla
actions. Add grep checks also for base tests where applicable.

Also fix trace event histogram trigger check to use the correct syntax
for the command-line option so that the test passes with the grep check.

Signed-off-by: Tomas Glozar <tglozar@redhat.com>
---
 tools/tracing/rtla/tests/hwnoise.t  | 11 ++++++-----
 tools/tracing/rtla/tests/osnoise.t  |  6 +++---
 tools/tracing/rtla/tests/timerlat.t | 12 +++++++-----
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/tools/tracing/rtla/tests/hwnoise.t b/tools/tracing/rtla/tests/hwnoise.t
index 448877564b8d..23ce250a6852 100644
--- a/tools/tracing/rtla/tests/hwnoise.t
+++ b/tools/tracing/rtla/tests/hwnoise.t
@@ -6,16 +6,17 @@ test_begin
 set_timeout 2m
 
 check "verify help page" \
-	"hwnoise --help"
+	"hwnoise --help" 0 "summary of hardware-related noise"
 check "detect noise higher than one microsecond" \
-	"hwnoise -c 0 -T 1 -d 5s -q"
+	"hwnoise -c 0 -T 1 -d 5s -q" 0
 check "set the automatic trace mode" \
-	"hwnoise -a 5 -d 10s" 2
+	"hwnoise -a 5 -d 10s" 2 "osnoise hit stop tracing"
 check "set scheduling param to the osnoise tracer threads" \
 	"hwnoise -P F:1 -c 0 -r 900000 -d 10s -q"
 check "stop the trace if a single sample is higher than 1 us" \
-	"hwnoise -s 1 -T 1 -t -d 10s" 2
+	"hwnoise -s 1 -T 1 -t -d 10s" 2 "Saving trace to osnoise_trace.txt"
 check "enable a trace event trigger" \
-	"hwnoise -t -e osnoise:irq_noise trigger=\"hist:key=desc,duration:sort=desc,duration:vals=hitcount\" -d 10s"
+	"hwnoise -t -e osnoise:irq_noise --trigger=\"hist:key=desc,duration:sort=desc,duration:vals=hitcount\" -d 10s" \
+	0 "Saving event osnoise:irq_noise hist to osnoise_irq_noise_hist.txt"
 
 test_end
diff --git a/tools/tracing/rtla/tests/osnoise.t b/tools/tracing/rtla/tests/osnoise.t
index 6a4dfa31dc55..7574ec6a5a53 100644
--- a/tools/tracing/rtla/tests/osnoise.t
+++ b/tools/tracing/rtla/tests/osnoise.t
@@ -6,13 +6,13 @@ test_begin
 set_timeout 2m
 
 check "verify help page" \
-	"osnoise --help"
+	"osnoise --help" 0 "osnoise version"
 check "verify the --priority/-P param" \
 	"osnoise top -P F:1 -c 0 -r 900000 -d 10s -q"
 check "verify the --stop/-s param" \
-	"osnoise top -s 30 -T 1 -t" 2
+	"osnoise top -s 30 -T 1" 2 "osnoise hit stop tracing"
 check "verify the  --trace param" \
-	"osnoise hist -s 30 -T 1 -t" 2
+	"osnoise hist -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
 check "verify the --entries/-E param" \
 	"osnoise hist -P F:1 -c 0 -r 900000 -d 10s -b 10 -E 25"
 
diff --git a/tools/tracing/rtla/tests/timerlat.t b/tools/tracing/rtla/tests/timerlat.t
index 2d59ee199c4d..db263dc90a2d 100644
--- a/tools/tracing/rtla/tests/timerlat.t
+++ b/tools/tracing/rtla/tests/timerlat.t
@@ -21,15 +21,17 @@ export RTLA_NO_BPF=$option
 
 # Basic tests
 check "verify help page" \
-	"timerlat --help"
+	"timerlat --help" 0 "timerlat version"
 check "verify -s/--stack" \
-	"timerlat top -s 3 -T 10 -t" 2
+	"timerlat top -s 3 -T 10 -t" 2 "Blocking thread stack trace"
 check "verify -P/--priority" \
 	"timerlat top -P F:1 -c 0 -d 10s -q"
 check "test in nanoseconds" \
-	"timerlat top -i 2 -c 0 -n -d 10s" 2
+	"timerlat top -i 2 -c 0 -n -d 10s" 2 "ns"
 check "set the automatic trace mode" \
-	"timerlat top -a 5 --dump-tasks" 2
+	"timerlat top -a 5" 2 "analyzing it"
+check "dump tasks" \
+	"timerlat top -a 5 --dump-tasks" 2 "Printing CPU tasks"
 check "print the auto-analysis if hits the stop tracing condition" \
 	"timerlat top --aa-only 5" 2
 check "disable auto-analysis" \
@@ -37,7 +39,7 @@ check "disable auto-analysis" \
 check "verify -c/--cpus" \
 	"timerlat hist -c 0 -d 10s"
 check "hist test in nanoseconds" \
-	"timerlat hist -i 2 -c 0 -n -d 10s" 2
+	"timerlat hist -i 2 -c 0 -n -d 10s" 2 "ns"
 
 # Actions tests
 check "trace output through -t" \
-- 
2.49.0

[PATCH 2/2] rtla/tests: Test timerlat -P option using actions

[Tomas Glozar]

The -P option is used to set priority of osnoise and timerlat threads.

Extend the test for -P with --on-threshold calling a script that looks
for running timerlat threads and checks if their priority is set
correctly.

As --on-threshold is only supported by timerlat at the moment, this is
only implemented there so far.

Signed-off-by: Tomas Glozar <tglozar@redhat.com>
---
 tools/tracing/rtla/tests/engine.sh                 | 2 +-
 tools/tracing/rtla/tests/scripts/check-priority.sh | 8 ++++++++
 tools/tracing/rtla/tests/timerlat.t                | 3 ++-
 3 files changed, 11 insertions(+), 2 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-priority.sh

diff --git a/tools/tracing/rtla/tests/engine.sh b/tools/tracing/rtla/tests/engine.sh
index 64c5be4313de..a97d644ead99 100644
--- a/tools/tracing/rtla/tests/engine.sh
+++ b/tools/tracing/rtla/tests/engine.sh
@@ -69,7 +69,7 @@ check() {
 			# Add rtla output and exit code as comments in case of failure
 			echo "$result" | col -b | while read line; do echo "# $line"; done
 			printf "#\n# exit code %s\n" $exitcode
-			[ -n "$expected_output" ] && \
+			[ -n "$expected_output" ] && [ $grep_result -ne 0 ] && \
 				printf "# Output match failed: \"%s\"\n" "$expected_output"
 		fi
 	fi
diff --git a/tools/tracing/rtla/tests/scripts/check-priority.sh b/tools/tracing/rtla/tests/scripts/check-priority.sh
new file mode 100755
index 000000000000..79b702a34a96
--- /dev/null
+++ b/tools/tracing/rtla/tests/scripts/check-priority.sh
@@ -0,0 +1,8 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+pids="$(pgrep ^$1)" || exit 1
+for pid in $pids
+do
+  chrt -p $pid | cut -d ':' -f 2 | head -n1 | grep "^ $2\$" >/dev/null
+  chrt -p $pid | cut -d ':' -f 2 | tail -n1 | grep "^ $3\$" >/dev/null
+done && echo "Priorities are set correctly"
diff --git a/tools/tracing/rtla/tests/timerlat.t b/tools/tracing/rtla/tests/timerlat.t
index db263dc90a2d..c71aed5534bf 100644
--- a/tools/tracing/rtla/tests/timerlat.t
+++ b/tools/tracing/rtla/tests/timerlat.t
@@ -25,7 +25,8 @@ check "verify help page" \
 check "verify -s/--stack" \
 	"timerlat top -s 3 -T 10 -t" 2 "Blocking thread stack trace"
 check "verify -P/--priority" \
-	"timerlat top -P F:1 -c 0 -d 10s -q"
+	"timerlat top -P F:1 -c 0 -d 10s -q -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh timerlatu/ SCHED_FIFO 1\"" \
+	2 "Priorities are set correctly"
 check "test in nanoseconds" \
 	"timerlat top -i 2 -c 0 -n -d 10s" 2 "ns"
 check "set the automatic trace mode" \
-- 
2.49.0