mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: James Clark <james.clark@linaro.org>
To: TAKANO Mitsuhiro <takano32@gmail.com>
Cc: Ian Rogers <irogers@google.com>,
	John Garry <john.garry@linux.dev>,
	linux-arm-kernel@lists.infradead.org,
	Ilkka Koskinen <ilkka@os.amperecomputing.com>,
	linux-kernel@vger.kernel.org, Will Deacon <will@kernel.org>,
	Mike Leach <mike.leach@arm.com>, Leo Yan <leo.yan@linux.dev>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>,
	Adrian Hunter <adrian.hunter@intel.com>,
	linux-perf-users@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>
Subject: Re: [PATCH v11 2/2] perf jevents: Fail the build on a key the tables may not carry
Date: Fri, 20 Nov 2026 10:06:56 +0000	[thread overview]
Message-ID: <4c59af55-230f-4fd5-980e-91b953a1709a@linaro.org> (raw)
In-Reply-To: <20261120060931.711483-3-takano32@gmail.com>



On 20/11/2026 06:09, TAKANO Mitsuhiro wrote:
> jevents.py takes the fields it wants out of each JSON object by name and
> ignores the rest, so a key that is spelled differently is dropped without
> a word and the object is built without whatever that field held.
> 
> Compare the keys of each object with _valid_keys, the keys a table may
> carry, and fail on anything else.  The error is one line naming the file
> and the key:
> 
>    jevents.py: <path>: unknown key(s): ['BriefDescript6ion']
> 
> Written with Hermes Agent, an LLM assistant; the author reviewed it and
> ran the generator on every arch directory, with and without a key the
> tables may not carry.
> 
> Signed-off-by: TAKANO Mitsuhiro <takano32@gmail.com>
> ---
>   tools/perf/pmu-events/jevents.py          | 70 ++++++++++++++++++-----
>   tools/perf/tests/shell/jevents_bad_key.sh | 43 ++++++++++++++
>   2 files changed, 98 insertions(+), 15 deletions(-)
>   create mode 100755 tools/perf/tests/shell/jevents_bad_key.sh
> 
> diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
> index 080e04b51b8c..65e51dafd42b 100755
> --- a/tools/perf/pmu-events/jevents.py
> +++ b/tools/perf/pmu-events/jevents.py
> @@ -63,6 +63,27 @@ _json_metric_attributes = [
>   # Attributes that are bools or enum int values, encoded as '0', '1',...
>   _json_enum_attributes = ['aggr_mode', 'deprecated', 'event_grouping', 'perpkg',
>                            'default_show_events']
> +# Keys a table under arch/ may carry.  jevents.py takes the fields it wants by
> +# name and drops the rest without a word, so a key that is spelled differently
> +# is only visible as a field that went missing; a table carrying anything else
> +# is an error.  The list holds the keys jevents.py reads, plus the ones the
> +# tables carry and it does not read: Counter, CountersNumFixed and
> +# CountersNumGeneric (read by nothing) and Experimental (read by metric.py).
> +_valid_keys: Set[str] = {
> +    'AggregationMode', 'AnyThread', 'ArchStdEvent', 'BriefDescription',
> +    'Compat', 'ConfigCode', 'Counter', 'CounterMask',
> +    'CountersNumFixed', 'CountersNumGeneric', 'Data_LA', 'DefaultMetricgroupName',
> +    'DefaultShowEvents', 'Deprecated', 'EdgeDetect', 'EnAllCores',
> +    'EnAllSlices', 'Errata', 'EventCode', 'EventidCode',
> +    'EventName', 'Experimental', 'ExtSel', 'FCMask',
> +    'Filter', 'Invert', 'LegacyCacheCode', 'LegacyConfigCode',
> +    'MetricConstraint', 'MetricExpr', 'MetricGroup', 'MetricgroupNoGroup',
> +    'MetricName', 'MetricThreshold', 'MSRIndex', 'MSRValue',
> +    'NodeType', 'PEBS', 'PerPkg', 'PortMask',
> +    'PublicDescription', 'RdWrMask', 'RetirementLatencyMax', 'RetirementLatencyMean',
> +    'RetirementLatencyMin', 'SampleAfterValue', 'ScaleUnit', 'SliceId',
> +    'ThreadMask', 'UMask', 'Unit',
> +}
>   
>   def file_name_to_table_name(prefix: str, parents: Sequence[str],
>                               dirname: str) -> str:
> @@ -196,6 +217,9 @@ class JsonEvent:
>     def __init__(self, jd: dict):
>       """Constructor passed the dictionary of parsed json values."""
>   
> +    if extra := set(jd) - _valid_keys:
> +      raise ValueError(f'unknown key(s): {sorted(extra)}')
> +

All you need is this raise, you don't need any of the other try catch 
changes. It already prints the filename, a little bit of stack and then 
the missing key as the last line:

   Exception processing pmu-events/arch/x86/amdzen1/core.json

   ...

   ValueError: Unknown keys ['EvendtName'] ...

It's perfectly legible as an error as it is and not worth the churn and 
extra noise in the code.

>       def llx(x: int) -> str:
>         """Convert an int to a string similar to a printf modifier of %#llx."""
>         return str(x) if x >= 0 and x < 10 else hex(x)
> @@ -459,6 +483,8 @@ def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]:
>     try:
>       with open(path, encoding='utf-8') as f:
>         events = json.load(f, object_hook=JsonEvent)
> +  except ValueError as exc:
> +    raise ValueError(f'{path}: {exc}') from exc
>     except BaseException:
>       print(f"Exception processing {path}")
>       raise
> @@ -492,6 +518,8 @@ def preprocess_arch_std_files(archpath: str) -> None:
>             _arch_std_events[event.name.lower()] = event
>           if event.metric_name:
>             _arch_std_events[event.metric_name.lower()] = event
> +    except ValueError:
> +        raise
>       except Exception as e:
>           raise RuntimeError(f'Failure processing \'{item.name}\' in \'{archpath}\'') from e
>   
> @@ -643,7 +671,10 @@ def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
>     assert _bcs is not None
>     if item.name.endswith('metricgroups.json'):
>       with open(item.path, encoding='utf-8') as f:
> -      metricgroup_descriptions = json.load(f)
> +      try:
> +        metricgroup_descriptions = json.load(f)
> +      except ValueError as exc:
> +        raise ValueError(f'{item.path}: {exc}') from exc
>       for mgroup in metricgroup_descriptions:
>         assert len(mgroup) > 1, parents
>         description = f"{metricgroup_descriptions[mgroup]}\\000"
> @@ -1473,6 +1504,8 @@ def main() -> None:
>               continue
>         try:
>           action(parents, item)
> +      except ValueError:
> +        raise
>         except Exception as e:
>           raise RuntimeError(f'Action failure for \'{item.name}\' in {parents}') from e
>         if item.is_dir():
> @@ -1536,20 +1569,27 @@ struct pmu_table_entry {
>       if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'):
>         tasks.append((item.path, get_topic(item.name)))
>   
> -  for arch in archs:
> -    arch_path = f'{_args.starting_dir}/{arch}'
> -    preprocess_arch_std_files(arch_path)
> -    ftw(arch_path, [], collect_json)
> -
> -  with concurrent.futures.ProcessPoolExecutor(max_workers=_parallel_worker_count(len(tasks)),
> -                                              initializer=_init_worker,
> -                                              initargs=(_arch_std_events,)) as executor:
> -    for path, topic, events in executor.map(_parallel_read_json_events, tasks):
> -      _json_cache[(path, topic)] = events
> -
> -  for arch in archs:
> -    arch_path = f'{_args.starting_dir}/{arch}'
> -    ftw(arch_path, [], preprocess_one_file)
> +  try:
> +    for arch in archs:
> +      arch_path = f'{_args.starting_dir}/{arch}'
> +      preprocess_arch_std_files(arch_path)
> +      ftw(arch_path, [], collect_json)
> +
> +    with concurrent.futures.ProcessPoolExecutor(max_workers=_parallel_worker_count(len(tasks)),
> +                                                initializer=_init_worker,
> +                                                initargs=(_arch_std_events,)) as executor:
> +      for path, topic, events in executor.map(_parallel_read_json_events, tasks):
> +        _json_cache[(path, topic)] = events
> +
> +    for arch in archs:
> +      arch_path = f'{_args.starting_dir}/{arch}'
> +      ftw(arch_path, [], preprocess_one_file)
> +  except ValueError as exc:
> +    # A worker's traceback is three levels of ProcessPoolExecutor noise around
> +    # the one line that matters, so say it here and stop.  make deletes the
> +    # target it failed to build (.DELETE_ON_ERROR), so the next run reads the
> +    # JSON again rather than a half-written table.
> +    sys.exit(f'jevents.py: {exc}')
>   
>     assert _bcs is not None
>     _bcs.compute()
> diff --git a/tools/perf/tests/shell/jevents_bad_key.sh b/tools/perf/tests/shell/jevents_bad_key.sh
> new file mode 100755
> index 000000000000..7848e60b8b4a
> --- /dev/null
> +++ b/tools/perf/tests/shell/jevents_bad_key.sh
> @@ -0,0 +1,43 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +# Check jevents.py reports unknown keys as errors (no_mlock)
> +#
> +# A key that is spelled differently is dropped without a word, so jevents.py
> +# compares the keys of each object in a table against the keys it reads and
> +# fails on anything else.
> +
> +jevents=$(dirname "$0")/../../pmu-events/jevents.py
> +python=${PYTHON:-python3}
> +
> +# An installed perf has the test but not the generator, and a python that is
> +# not there or too old to run it is a skip, not a failure of the check.
> +[ -r "${jevents}" ] || exit 2
> +[ "$("${python}" -c 'import sys; print(sys.version_info >= (3, 9))' 2>/dev/null)" = True ] \
> +  || exit 2
> +
> +tmp=$(mktemp -d) || exit 2
> +trap 'rm -rf "${tmp}"' EXIT HUP INT TERM
> +tables="${tmp}/arch/x86/model"
> +mkdir -p "${tables}" "${tmp}/arch/common"
> +printf '[]\n' > "${tmp}/arch/common/empty.json"
> +printf 'Family-model,Version,Filename,EventType\nGenuineIntel-6-97,v1.0,model,core\n' \
> +  > "${tmp}/arch/x86/mapfile.csv"
> +printf '[{"EventName": "a", "NotAKey": "x"}]\n' > "${tables}/events.json"
> +
> +if "${python}" -B "${jevents}" x86 all "${tmp}/arch" \
> +    "${tmp}/pmu-events.c" "${tmp}/pmu-events-string.c" 2> "${tmp}/err"; then
> +  echo "jevents.py accepted a key it does not read" >&2
> +  exit 1
> +fi
> +grep -qF -- "NotAKey" "${tmp}/err" || {
> +  echo "jevents.py failed without naming the key" >&2
> +  cat "${tmp}/err" >&2
> +  exit 1
> +}
> +grep -qF -- "${tables}/events.json" "${tmp}/err" || {
> +  echo "jevents.py failed without naming the file" >&2
> +  cat "${tmp}/err" >&2
> +  exit 1
> +}
> +
> +exit 0


  reply	other threads:[~2026-11-20 10:06 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 22:22 [PATCH 0/3] perf jevents: Report the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-10 22:22 ` [PATCH 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-10 22:22 ` [PATCH 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-10 22:22 ` [PATCH 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  2:34 ` [PATCH v2 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  2:34   ` [PATCH v2 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  2:34   ` [PATCH v2 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  2:34   ` [PATCH v2 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  3:43   ` [PATCH v3 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  3:43     ` [PATCH v3 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  3:43     ` [PATCH v3 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  3:43     ` [PATCH v3 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11  4:10     ` [PATCH v4 0/3] perf jevents: Report " TAKANO Mitsuhiro
2026-11-11  4:10       ` [PATCH v4 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11  4:10       ` [PATCH v4 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11  4:10       ` [PATCH v4 3/3] perf jevents: Check the JSON keys that jevents.py never reads TAKANO Mitsuhiro
2026-11-11 13:26       ` [PATCH v5 0/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-11 13:26         ` [PATCH v5 1/3] perf jevents: Name the metric when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11 13:26         ` [PATCH v5 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11 13:26         ` [PATCH v5 3/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-11 15:09         ` [PATCH v6 0/3] " TAKANO Mitsuhiro
2026-11-11 15:09           ` [PATCH v6 1/3] perf jevents: Name the metric and the file when a MetricExpr is missing TAKANO Mitsuhiro
2026-11-11 15:09           ` [PATCH v6 2/3] perf jevents: Make the model argument optional TAKANO Mitsuhiro
2026-11-11 15:09           ` [PATCH v6 3/3] perf jevents: Check the JSON keys a table may carry TAKANO Mitsuhiro
2026-11-12 16:28           ` [PATCH v7 0/1] perf jevents: Check the keys of the JSON tables TAKANO Mitsuhiro
2026-11-12 16:28             ` [PATCH v7 1/1] " TAKANO Mitsuhiro
2026-11-12 17:32             ` [PATCH v8 0/1] " TAKANO Mitsuhiro
2026-11-12 17:32               ` [PATCH v8 1/1] " TAKANO Mitsuhiro
2026-11-12 22:25               ` [PATCH v9 0/1] " TAKANO Mitsuhiro
2026-11-12 22:25                 ` [PATCH v9 1/1] " TAKANO Mitsuhiro
2026-11-13  3:29                 ` [PATCH v10 0/1] " TAKANO Mitsuhiro
2026-11-13  3:29                   ` [PATCH v10 1/1] " TAKANO Mitsuhiro
2026-11-19 14:20                     ` James Clark
2026-11-20  6:09                   ` [PATCH v11 0/2] perf jevents: Fail the build on a key the tables may not carry TAKANO Mitsuhiro
2026-11-20  6:09                     ` [PATCH v11 1/2] perf vendor events: Fix misspelled keys in the event tables TAKANO Mitsuhiro
2026-11-20  6:09                     ` [PATCH v11 2/2] perf jevents: Fail the build on a key the tables may not carry TAKANO Mitsuhiro
2026-11-20 10:06                       ` James Clark [this message]
2026-11-20 13:42                     ` [PATCH v12 0/2] " TAKANO Mitsuhiro
2026-11-20 13:42                       ` [PATCH v12 1/2] perf vendor events: Fix misspelled keys in the event tables TAKANO Mitsuhiro
2026-11-20 13:42                       ` [PATCH v12 2/2] perf jevents: Fail the build on a key the tables may not carry TAKANO Mitsuhiro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4c59af55-230f-4fd5-980e-91b953a1709a@linaro.org \
    --to=james.clark@linaro.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=ilkka@os.amperecomputing.com \
    --cc=irogers@google.com \
    --cc=john.garry@linux.dev \
    --cc=jolsa@kernel.org \
    --cc=leo.yan@linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mike.leach@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=takano32@gmail.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®