mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	 Arnaldo Carvalho de Melo <acme@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>,
	 Ian Rogers <irogers@google.com>,
	Adrian Hunter <adrian.hunter@intel.com>,
	 James Clark <james.clark@linaro.org>,
	linux-kernel@vger.kernel.org,  linux-perf-users@vger.kernel.org
Subject: [RFC PATCH v1 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs
Date: Fri, 22 May 2026 15:33:42 -0700	[thread overview]
Message-ID: <20260522223342.2393553-15-irogers@google.com> (raw)
In-Reply-To: <20260522223342.2393553-1-irogers@google.com>

This patch updates tools/perf/tests/shell/stat_metrics_values.sh to run
the metrics value validation linter twice: once for the legacy path,
and once under the --new print flag using the newly introduced python
validator `-new` argument:

  $PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir ... -new

This mathematically guarantees that calculated Intel PMU metric values
staged inside the decoupled streaming JSON callbacks are parsed and
validated cleanly against the formal rules engine.

Signed-off-by: Ian Rogers <irogers@google.com>

TAG=agy
CONV=7c33d6a0-70a2-454a-82f7-4de2101a5b9c
---
 tools/perf/tests/shell/stat_metrics_values.sh | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/perf/tests/shell/stat_metrics_values.sh b/tools/perf/tests/shell/stat_metrics_values.sh
index 30566f0b5427..02c0bcf6e078 100755
--- a/tools/perf/tests/shell/stat_metrics_values.sh
+++ b/tools/perf/tests/shell/stat_metrics_values.sh
@@ -22,9 +22,20 @@ for cputype in /sys/bus/event_source/devices/cpu_*; do
 	$PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir -wl "${workload}" \
 		-cputype "${cputype}"
 	ret=$?
-	rm -rf $tmpdir
 	if [ $ret -ne 0 ]; then
 		echo "Metric validation return with errors. Please check metrics reported with errors."
+		rm -rf $tmpdir
+		exit $ret
+	fi
+
+	echo "Testing metrics for: $cputype (New API)"
+	$PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir -wl "${workload}" \
+		-cputype "${cputype}" -new
+	ret=$?
+	rm -rf $tmpdir
+	if [ $ret -ne 0 ]; then
+		echo "Metric validation return with errors (New API). Please check metrics reported with errors."
+		exit $ret
 	fi
 done
 exit $ret
-- 
2.54.0.794.g4f17f83d09-goog


  parent reply	other threads:[~2026-05-22 22:34 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-22 22:33 [RFC PATCH v1 00/14] perf stat: Decouple and modularize metrics/events output printing API Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 01/14] perf stat: Introduce core generic print traversal engine and header stubs Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 02/14] perf stat: Implement standard console (STD) formatting callbacks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 03/14] perf stat: Extend STD output linter to test basic New API checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 04/14] perf stat: Extend STD output linter to test core aggregation checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 05/14] perf stat: Extend STD output linter to test advanced PMU checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 06/14] perf stat: Extend STD output linter to test metric-only checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 07/14] perf stat: Implement CSV formatting callbacks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 08/14] perf stat: Extend CSV output linter to test core aggregation checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 09/14] perf stat: Extend CSV output linter to test advanced PMU and metric-only checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 10/14] perf stat: Implement streaming JSON formatting callbacks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 11/14] perf stat: Extend JSON output linter to test core aggregation checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 12/14] perf stat: Extend JSON output linter to test advanced PMU and metric-only checks Ian Rogers
2026-05-22 22:33 ` [RFC PATCH v1 13/14] perf stat: Add --new support to PMU metrics Python validator Ian Rogers
2026-05-22 22:33 ` Ian Rogers [this message]
2026-05-25 23:18 ` [RFC PATCH v2 00/14] perf stat: Decouple and modularize metrics/events output printing API Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 01/14] perf stat: Introduce core generic print traversal engine and header stubs Ian Rogers
2026-05-25 23:38     ` Arnaldo Carvalho de Melo
2026-05-25 23:48       ` Ian Rogers
2026-05-26  0:20         ` Arnaldo Carvalho de Melo
2026-05-25 23:18   ` [RFC PATCH v2 02/14] perf stat: Implement standard console (STD) formatting callbacks Ian Rogers
2026-05-25 23:49     ` Arnaldo Carvalho de Melo
2026-05-26  0:09       ` Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 03/14] perf stat: Extend STD output linter to test basic New API checks Ian Rogers
2026-05-25 23:39     ` Arnaldo Carvalho de Melo
2026-05-25 23:18   ` [RFC PATCH v2 04/14] perf stat: Extend STD output linter to test core aggregation checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 05/14] perf stat: Extend STD output linter to test advanced PMU checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 06/14] perf stat: Extend STD output linter to test metric-only checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 07/14] perf stat: Implement CSV formatting callbacks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 08/14] perf stat: Extend CSV output linter to test core aggregation checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 09/14] perf stat: Extend CSV output linter to test advanced PMU and metric-only checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 10/14] perf stat: Implement streaming JSON formatting callbacks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 11/14] perf stat: Extend JSON output linter to test core aggregation checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 12/14] perf stat: Extend JSON output linter to test advanced PMU and metric-only checks Ian Rogers
2026-05-25 23:18   ` [RFC PATCH v2 13/14] perf stat: Add --new support to PMU metrics Python validator Ian Rogers
2026-05-25 23:19   ` [RFC PATCH v2 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs Ian Rogers
2026-06-05 18:02   ` [RFC PATCH v2 00/14] perf stat: Decouple and modularize metrics/events output printing API Chun-Tse Shao
2026-07-16  4:32   ` [PATCH v3 00/14] perf stat: Decouple printing API and introduce streaming zero-allocation printers Ian Rogers
2026-07-16  4:32     ` [PATCH v3 01/14] perf stat: Introduce core generic print traversal engine and header stubs Ian Rogers
2026-07-16  4:32     ` [PATCH v3 02/14] perf stat: Implement standard console (STD) formatting callbacks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 03/14] perf stat: Extend STD output linter to test basic New API checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 04/14] perf stat: Extend STD output linter to test core aggregation checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 05/14] perf stat: Extend STD output linter to test advanced PMU checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 06/14] perf stat: Extend STD output linter to test metric-only checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 07/14] perf stat: Implement CSV formatting callbacks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 08/14] perf stat: Extend CSV output linter to test core aggregation checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 09/14] perf stat: Extend CSV output linter to test advanced PMU and metric-only checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 10/14] perf stat: Implement streaming JSON formatting callbacks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 11/14] perf stat: Extend JSON output linter to test core aggregation checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 12/14] perf stat: Extend JSON output linter to test advanced PMU and metric-only checks Ian Rogers
2026-07-16  4:32     ` [PATCH v3 13/14] perf stat: Add --new support to PMU metrics Python validator Ian Rogers
2026-07-16  4:32     ` [PATCH v3 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs Ian Rogers
2026-07-16  7:02     ` [PATCH v4 00/14] perf stat: Decouple and modularize display formatting Ian Rogers
2026-07-16  7:02       ` [PATCH v4 01/14] perf stat: Introduce core generic print traversal engine and header stubs Ian Rogers
2026-07-16 22:22         ` Namhyung Kim
2026-07-16  7:02       ` [PATCH v4 02/14] perf stat: Implement standard console (STD) formatting callbacks Ian Rogers
2026-07-16 22:30         ` Namhyung Kim
2026-07-16  7:02       ` [PATCH v4 03/14] perf stat: Extend STD output linter to test basic New API checks Ian Rogers
2026-07-16  7:02       ` [PATCH v4 04/14] perf stat: Extend STD output linter to test core aggregation checks Ian Rogers
2026-07-16  7:02       ` [PATCH v4 05/14] perf stat: Extend STD output linter to test advanced PMU checks Ian Rogers
2026-07-16  7:02       ` [PATCH v4 06/14] perf stat: Extend STD output linter to test metric-only checks Ian Rogers
2026-07-16 22:32         ` Namhyung Kim
2026-07-16  7:02       ` [PATCH v4 07/14] perf stat: Implement CSV formatting callbacks Ian Rogers
2026-07-16 22:35         ` Namhyung Kim
2026-07-16  7:02       ` [PATCH v4 08/14] perf stat: Extend CSV output linter to test core aggregation checks Ian Rogers
2026-07-16  7:02       ` [PATCH v4 09/14] perf stat: Extend CSV output linter to test advanced PMU and metric-only checks Ian Rogers
2026-07-16  7:02       ` [PATCH v4 10/14] perf stat: Implement streaming JSON formatting callbacks Ian Rogers
2026-07-16 22:43         ` Namhyung Kim
2026-07-16  7:03       ` [PATCH v4 11/14] perf stat: Extend JSON output linter to test core aggregation checks Ian Rogers
2026-07-16  7:03       ` [PATCH v4 12/14] perf stat: Extend JSON output linter to test advanced PMU and metric-only checks Ian Rogers
2026-07-16  7:03       ` [PATCH v4 13/14] perf stat: Add --new support to PMU metrics Python validator Ian Rogers
2026-07-16  7:03       ` [PATCH v4 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs Ian Rogers
2026-07-16 22:48       ` [PATCH v4 00/14] perf stat: Decouple and modularize display formatting Namhyung Kim
2026-07-17  0:23         ` Ian Rogers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260522223342.2393553-15-irogers@google.com \
    --to=irogers@google.com \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox