Re: [PATCH v1] perf test stat: Avoid hybrid assumption when virtualized

[James Clark <james.clark@linaro.org>]

On 11/12/2024 5:58 pm, Ian Rogers wrote:
> On Wed, Dec 11, 2024 at 1:50 AM James Clark <james.clark@linaro.org> wrote:
>>
>> On 11/12/2024 6:10 am, Ian Rogers wrote:
>>> The cycles event will fallback to task-clock in the hybrid test when
>>> running virtualized. Change the test to not fail for this.
>>>
>>> Fixes: a6b8bb2addd0 ("perf test: Add a test for default perf stat command")
>>> Signed-off-by: Ian Rogers <irogers@google.com>
>>> ---
>>>    tools/perf/tests/shell/stat.sh | 10 +++++++---
>>>    1 file changed, 7 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tools/perf/tests/shell/stat.sh b/tools/perf/tests/shell/stat.sh
>>> index 5a2ca2bcf94d..60cea07350e1 100755
>>> --- a/tools/perf/tests/shell/stat.sh
>>> +++ b/tools/perf/tests/shell/stat.sh
>>> @@ -165,9 +165,13 @@ test_hybrid() {
>>>
>>>      if [ "$pmus" -ne "$cycles_events" ]
>>>      then
>>> -    echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
>>> -    err=1
>>> -    return
>>> +     # If virtualized the software task-clock event will be used.
>>> +     if ! perf stat -- true 2>&1 | grep -q "task-clock"
>>> +     then
>>> +       echo "hybrid test [Found $pmus PMUs but $cycles_events cycles events. Failed]"
>>> +       err=1
>>> +       return
>>> +     fi
>>>      fi
>>>      echo "hybrid test [Success]"
>>>    }
>>
>> Hi Ian,
>>
>> Isn't the distinction between task-clock and cpu-clock whether the event
>> is per-cpu or not?
>>
>> $ perf stat -C 1 -- true 2>&1 | grep cpu-clock
>>                 1.49 msec cpu-clock       #    0.917 CPUs utilized
>>
>> $ perf stat -- true 2>&1 | grep task-clock
>>                 0.30 msec task-clock      #    0.366 CPUs utilized
>>
>> The test uses per-task mode so this change makes it always pass, even
>> when the number of cycles events doesn't match the PMUs.
> 
> So I'm confused by the test, but it has caused a passing test to be
> broken for me when I run virtualized. The test is checking a cycles
> event is opened on each hybrid PMU, but this is conflated with
> checking perf stat's "default" output. The cycles event will fall back
> to task-clock in per-task mode but we also open a task-clock in
> default mode. Should:
> ```
> if [ "$pmus" -ne "$cycles_events" ]
> ```
> be something like this then:
> ```
> # The expectation is that default output will have a cycles events on
> each hybrid
> # PMU, but in situations with no cycles PMU events, like
> virtualized,this can fall
> # back to task-clock and so the end count may be 0. Fail if neither
> condition holds.
> if [ "$pmus" -ne "$cycles_events" ] && [ "$pmus" -ne "0" ]

Yes it might make sense to skip this part if pmus == 0 then. Namhyung 
suggested to always assume 1 PMU to cover platforms that don't publish 
any PMU with the cpus file, but I suppose that breaks in this case.

If you still add the fallback check for task-clock it covers hybrid, 1 
PMU and virtualized and we could even rename the test to 
test_default_stat_2.

> ```
> 
> Thanks,
> Ian