Re: [PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[Jens Remus <jremus@linux.ibm.com>]

Hi Ian!

On 8/14/2026 7:43 PM, Ian Rogers wrote:
> On Tue, Aug 11, 2026 at 2:53 AM Jens Remus <jremus@linux.ibm.com> wrote:
>> On 8/8/2026 6:13 AM, Ian Rogers wrote:
>>> On Fri, Aug 7, 2026 at 3:49 AM Jens Remus <jremus@linux.ibm.com> wrote:
>>>>
>>>> On s390 the kernel uses s390 back chain instead of frame pointers for
>>>> stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add
>>>> USER_STACKTRACE support").  This is because frame pointers on s390
>>>> cannot be used for stack tracing. [1]
>>>>
>>>> This requires user space to maintain a s390 back chain.  For instance
>>>> user space to be built with compiler option '-mbackchain' (instead of
>>>> '-fno-omit-frame-pointer' used on other architectures, which should
>>>> better not be used on s390 [1]).
>>>>
>>>> Only few distributions and users built user space with '-mbackchain'.
>>>> Therefore '--call-graph fp' may not produce the expected results.
>>>>
>>>> Commit ca76fb67ebdd ("perf evlist: Improve default event for s390")
>>>> added a warning for s390 that wrongly claimed that "Framepointer
>>>> unwinding lacks kernel support".  Change the warning to hint at using
>>>> '--call-graph dwarf' if user space does not maintain a s390 back chain.
>>>>
>>>> Note that '--call-graph fp' may also be useful for other applications,
>>>> such as OpenJDK maintaining a s390 back chain (does not require JVM
>>>> option '-XX:+PreserveFramePointer' on s390):
>>>>
>>>>   $ perf record --call-graph fp ... -- \
>>>>     java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...
>>>>
>>>> [1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,
>>>>      https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
>>>>
>>>> Fixes: ca76fb67ebdd ("perf evlist: Improve default event for s390")
>>>> Signed-off-by: Jens Remus <jremus@linux.ibm.com>
>>>> ---
>>>>  tools/perf/util/evsel.c | 3 ++-
>>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
>>>> index ea9fa04429f0..c4d67d1b35f3 100644
>>>> --- a/tools/perf/util/evsel.c
>>>> +++ b/tools/perf/util/evsel.c
>>>> @@ -1080,7 +1080,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
>>>>
>>>>         if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
>>>>                 pr_warning_once(
>>>> -                       "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
>>>> +                       "Use '--call-graph dwarf' if user space does not maintain a s390 back chain "
>>>> +                       "(e.g. is not built with '-mbackchain').\n");
>>>
>>> Thanks Jens. The text was based on:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/kernel/perf_cpum_sf.c#n853
>>> ```
>>> static int cpumsf_pmu_event_init(struct perf_event *event)
>>> ...
>>>  /* No support for callchain, stacks and registers */
>>> if (has_branch_stack(event) || is_callchain_event(event))
>>> return -EOPNOTSUPP;
>>> ```
>>
>> According to Thomas, the s390 HW event 'cycles' does not support
>> callchains.  This is because on s390 it provides aggregated historical
>> data, making it impossible to associate a callchain to the sampled data.
>> Your referenced cpumsf_pmu_event_init() code applies specifically to the
>> CPU Measurement Sampling Facility (CPUMSF) PMU and therefore correctly
>> rejects callchains for those HW events.
>>
>> On s390, callchains are therefore only supported with SW events.
>>
>> Since your commit ca76fb67ebdd ("perf evlist: Improve default event for
>> s390") callchains on s390 default to (1) the SW event 'cpu-clock' (or
>> 'task-clock') and (2) 'dwarf'.  The kernel on s390 also supports 'fp'
>> callchains using s390 back chain since commit aa44433ac4ee ("s390: add
>> USER_STACKTRACE support").
>>
>> Therefore the current warning on s390 for 'fp' is misleading.  Given
>> that on s390 a user must explicitly select 'fp', and that on other
>> architectures there is no warning about the requirement for user space
>> to maintain frame pointers (e.g. if they were not built with
>> '-fno-omit-frame-pointer' and '-mno-omoit-leaf-frame-pointer') it might
>> be preferable to remove the warning altogether.
>>
>> We could document in the perf man pages that 'fp' on s390 relies on the
>> s390 back chain rather than frame pointers.
>>
>> What do you think?
> 
> So firstly, sorry for the misleading message and thanks for trying to
> fix it! Also, sorry for the delay in responding and dealing with some
> hospital things. Man page documentation sounds good to me, and having
> a good warning also sounds good. Since this is for s390 you guys are
> much smarter about what to do than I am. Since the function generating
> the warning has an evsel, we can get the PMU from the evsel and check
> things like:
> ```
> if (evsel->pmu && perf_pmu__is_software(evsel->pmu))
> ```
> By which I mean we can provide different warnings for hardware and
> software events. That said, as a hardware event will fail in the
> perf_event_open I'm not sure it is a useful distinction.

Thanks for the hint!  After discussion with Thomas I have opted to
remove the warning to use 'dwarf' instead of 'fp' as it could misguide
users to assume 'fp' is inferior in general.

>>> The backchain option is never tested by perf or apparently in the
>>> kernel, but this warning is supposed to pre-warn about "not supported"
>>> being returned when frame pointer unwinding is requested. Of course,
>>> lacking an s390 I've never tested this. I can imagine the warning
>>> being overly broad, but my understanding is cpum_sf is the only PMU
>>> capable of sampling on s390.
>>>
>>> Perhaps what is needed is additional text after the "Use '--call-graph
>>> dwarf'". The existing initial text at least appears to match the
>>> kernel code. I'm not clear how the lack of back chains would be
>>> reported as an error code, and this could introduce confusion as we
>>> have lots of stack, chain and branch related terms in the perf
>>> codebase.
>>
>> This is analogous to frame pointers on other architectures: the absence
>> of a maintained s390 back chain is not reported as an error.  There is
>> currently no mean to determine from and ELF binary whether it was built
>> to maintain a s390 back chain (i.e. no ELF attribute/flag).
> 
> Thanks for the clarification! I think given this I'm happy to add my tag:
> 
> Reviewed-by: Ian Rogers <irogers@google.com>
> 
> Do you want the maintainers to move forward with this change, or would
> you prefer to add something to the man pages, etc. ?

I have sent a v2.

Regards,
Jens
-- 
Jens Remus
Linux on Z Development (D3303)
jremus@de.ibm.com / jremus@linux.ibm.com

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/