[PATCH v4 00/21] perf test attr: Report the failures again and cope with more than one core PMU

[Arnaldo Carvalho de Melo <acme@kernel.org>]

From: Arnaldo Carvalho de Melo <acme@redhat.com>

Hello all,

perf test 82 stopped reporting failures when it got its shell wrapper
(8519e4f44c2af722), so expectations that no longer match perf went
unverified.  The prerequisite fixes are upstream now, so they are not
repeated here.

This series updates those expectations and makes the test cope with
machines with no core PMU, or with more than one.  Asking for one event
on the latter gives one event per core PMU.  The expectations ignore
what that adds; the commit messages say what and why.

The tests that map ring buffers take turns on the per user mlock budget
(kernel.perf_event_mlock_kb) through a robust, process-shared mutex.
The sysctl is left alone.  A test killed while holding the budget does
not leave the next one waiting.

Tested: 'perf test -vv attribute' Ok on an Amlogic A311D (aarch64, as
root), an Intel i7-14700K (hybrid, two core PMUs), x86_64 (AMD Ryzen 9
9950X3D, kernel 7.2.10-200.fc44) and riscv64 (Orange Pi RV).  On
riscv64 the stat tests run and the record tests report unsupp, as
sampling is not available there (no Sscofpmf).

Best regards,

- Arnaldo

What changed from v3:

  - sashiko-bot's v3 review: the python binding no longer leaks the
    parse error entries it prints when parsing fails.
  - sashiko-bot's v3 review: attr.py no longer compares an event against
    itself unrecorded, and masks both sides with both events' types.
  - sashiko-bot's v3 review: evsel__open_cpu() no longer leaks the fds
    of an attempt abandoned to disable missing features.
  - sashiko-bot's v3 review: <stdio.h> included where FILE and
    snprintf() are used, in builtin-test.c and tests-scripts.c.

What changed from v2:

  - sashiko-bot's v2 review: a one-component kernel_since or
    kernel_until (f.i. kernel_until = 6) no longer crashes attr.py.
  - sashiko-bot's v2 review: mutex_trylock() and cond_wait() now take
    over a robust mutex left behind by a dead owner, as mutex_lock().

What changed from v1:

  - Namhyung's python fix and Michael Petlan's two fixes are upstream
    now, so they are dropped from the series.
  - Namhyung's v1 report (test-record-branch-filter-any_call): probe
    events are no longer dumped, and the core PMU fallout is masked.
  - New, not from review: the 'requires = core-pmu' skip, the skip
    reasons and the mlock budget part.
  - New, not from review: the events of the other core PMUs are left
    out, when the expectations name a core PMU.

 tools/build/Makefile.feature                  |   1 +
 tools/build/feature/Makefile                  |   4 +
 tools/build/feature/test-all.c                |   5 +
 .../build/feature/test-pthread-mutex-robust.c |  18 +
 tools/perf/Documentation/perf-test.txt        |  36 +-
 tools/perf/Makefile.config                    |   4 +
 tools/perf/tests/backward-ring-buffer.c       |   2 +-
 tools/perf/tests/builtin-test.c               | 174 ++++++-
 tools/perf/tests/code-reading.c               |   2 +-
 tools/perf/tests/keep-tracking.c              |   2 +-
 tools/perf/tests/mmap-basic.c                 |   1 +
 tools/perf/tests/openat-syscall-tp-fields.c   |   1 +
 tools/perf/tests/perf-record.c                |   1 +
 tools/perf/tests/perf-time-to-tsc.c           |   1 +
 tools/perf/tests/shell/attr/README            |  10 +-
 tools/perf/tests/shell/attr/base-stat-group   |  45 ++
 .../tests/shell/attr/test-record-graph-dwarf  |   2 +-
 .../shell/attr/test-record-group-sampling     |   5 +-
 .../shell/attr/test-record-group-sampling1    |   5 +-
 .../shell/attr/test-record-group-sampling2    |   5 +-
 .../perf/tests/shell/attr/test-record-group1  |   3 +-
 .../perf/tests/shell/attr/test-record-group2  |   3 +-
 .../perf/tests/shell/attr/test-record-group3  |   3 +-
 tools/perf/tests/shell/attr/test-stat-C0      |   1 +
 tools/perf/tests/shell/attr/test-stat-basic   |   1 +
 tools/perf/tests/shell/attr/test-stat-default |  91 ++--
 .../tests/shell/attr/test-stat-detailed-1     |  99 ++--
 .../tests/shell/attr/test-stat-detailed-2     | 111 +++--
 .../tests/shell/attr/test-stat-detailed-3     | 115 +++--
 tools/perf/tests/shell/attr/test-stat-group1  |   3 +-
 .../tests/shell/attr/test-stat-no-inherit     |   1 +
 tools/perf/tests/shell/drm_pmu.sh             |   2 +-
 tools/perf/tests/shell/ftrace.sh              |   2 +-
 tools/perf/tests/shell/kallsyms.sh            |   2 +-
 tools/perf/tests/shell/lib/attr.py            | 461 ++++++++++++++++--
 tools/perf/tests/shell/list.sh                |   2 +-
 tools/perf/tests/shell/perf_sched_stats.sh    |   2 +-
 tools/perf/tests/shell/probe_vfs_getname.sh   |   2 +-
 tools/perf/tests/shell/python-use.sh          |   2 +-
 tools/perf/tests/shell/schedstat_snapshots.sh |   2 +-
 tools/perf/tests/shell/stat+csv_output.sh     |   2 +-
 tools/perf/tests/shell/stat+csv_summary.sh    |   2 +-
 .../tests/shell/stat+event_uniquifying.sh     |   2 +-
 tools/perf/tests/shell/stat+json_output.sh    |   2 +-
 tools/perf/tests/shell/stat+shadow_stat.sh    |   2 +-
 tools/perf/tests/shell/stat+std_output.sh     |   2 +-
 tools/perf/tests/shell/stat.sh                |   2 +-
 .../perf/tests/shell/stat_all_metricgroups.sh |   2 +-
 tools/perf/tests/shell/stat_all_metrics.sh    |   2 +-
 tools/perf/tests/shell/stat_all_pfm.sh        |   2 +-
 .../tests/shell/stat_bpf_counters_cgrp.sh     |   2 +-
 tools/perf/tests/shell/stat_metrics_cgrp.sh   |   2 +-
 tools/perf/tests/shell/stat_metrics_values.sh |   2 +-
 tools/perf/tests/shell/test_ilist_python.sh   |   2 +-
 .../perf/tests/shell/test_stat_cpi_python.sh  |   2 +-
 .../tests/shell/test_test_junit_output.sh     |   5 +-
 .../shell/test_uprobe_from_different_cu.sh    |   2 +-
 tools/perf/tests/sw-clock.c                   |   2 +-
 tools/perf/tests/task-exit.c                  |   1 +
 tools/perf/tests/tests-scripts.c              |  85 +++-
 tools/perf/tests/tests.h                      |  17 +
 tools/perf/util/evsel.c                       |  42 +-
 tools/perf/util/evsel.h                       |   1 +
 tools/perf/util/mutex.c                       | 101 +++-
 tools/perf/util/mutex.h                       |   6 +
 tools/perf/util/pfm.c                         |   1 +
 tools/perf/util/print-events.c                |   1 +
 tools/perf/util/python.c                      |   2 +
 68 files changed, 1256 insertions(+), 272 deletions(-)

base-commit: cb9dba6f4764fc68ab75c36e332887332339f9c3
v1-head: b1300831ecc4503d
v2-head: da166ef7a002
v3-head: c5f0897aceb01743