[PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[Jens Remus]

On s390 the kernel uses s390 back chain instead of frame pointers for
stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add
USER_STACKTRACE support").  This is because frame pointers on s390
cannot be used for stack tracing. [1]

This requires user space to maintain a s390 back chain.  For instance
user space to be built with compiler option '-mbackchain' (instead of
'-fno-omit-frame-pointer' used on other architectures, which should
better not be used on s390 [1]).

Only few distributions and users built user space with '-mbackchain'.
Therefore '--call-graph fp' may not produce the expected results.

Commit ca76fb67ebdd ("perf evlist: Improve default event for s390")
added a warning for s390 that wrongly claimed that "Framepointer
unwinding lacks kernel support".  Change the warning to hint at using
'--call-graph dwarf' if user space does not maintain a s390 back chain.

Note that '--call-graph fp' may also be useful for other applications,
such as OpenJDK maintaining a s390 back chain (does not require JVM
option '-XX:+PreserveFramePointer' on s390):

  $ perf record --call-graph fp ... -- \
    java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...

[1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,
     https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/

Fixes: ca76fb67ebdd ("perf evlist: Improve default event for s390")
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
---
 tools/perf/util/evsel.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index ea9fa04429f0..c4d67d1b35f3 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -1080,7 +1080,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
 
 	if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
 		pr_warning_once(
-			"Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
+			"Use '--call-graph dwarf' if user space does not maintain a s390 back chain "
+			"(e.g. is not built with '-mbackchain').\n");
 	}
 
 	evsel__set_sample_bit(evsel, CALLCHAIN);

base-commit: f9a2394a23482bfd330911e9c8295b71724feacd
-- 
2.53.0

Re: [PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[Ian Rogers]

On Fri, Aug 7, 2026 at 3:49 AM Jens Remus <jremus@linux.ibm.com> wrote:
>
> On s390 the kernel uses s390 back chain instead of frame pointers for
> stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add
> USER_STACKTRACE support").  This is because frame pointers on s390
> cannot be used for stack tracing. [1]
>
> This requires user space to maintain a s390 back chain.  For instance
> user space to be built with compiler option '-mbackchain' (instead of
> '-fno-omit-frame-pointer' used on other architectures, which should
> better not be used on s390 [1]).
>
> Only few distributions and users built user space with '-mbackchain'.
> Therefore '--call-graph fp' may not produce the expected results.
>
> Commit ca76fb67ebdd ("perf evlist: Improve default event for s390")
> added a warning for s390 that wrongly claimed that "Framepointer
> unwinding lacks kernel support".  Change the warning to hint at using
> '--call-graph dwarf' if user space does not maintain a s390 back chain.
>
> Note that '--call-graph fp' may also be useful for other applications,
> such as OpenJDK maintaining a s390 back chain (does not require JVM
> option '-XX:+PreserveFramePointer' on s390):
>
>   $ perf record --call-graph fp ... -- \
>     java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...
>
> [1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,
>      https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
>
> Fixes: ca76fb67ebdd ("perf evlist: Improve default event for s390")
> Signed-off-by: Jens Remus <jremus@linux.ibm.com>
> ---
>  tools/perf/util/evsel.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> index ea9fa04429f0..c4d67d1b35f3 100644
> --- a/tools/perf/util/evsel.c
> +++ b/tools/perf/util/evsel.c
> @@ -1080,7 +1080,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
>
>         if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
>                 pr_warning_once(
> -                       "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
> +                       "Use '--call-graph dwarf' if user space does not maintain a s390 back chain "
> +                       "(e.g. is not built with '-mbackchain').\n");

Thanks Jens. The text was based on:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/kernel/perf_cpum_sf.c#n853
```
static int cpumsf_pmu_event_init(struct perf_event *event)
...
 /* No support for callchain, stacks and registers */
if (has_branch_stack(event) || is_callchain_event(event))
return -EOPNOTSUPP;
```
The backchain option is never tested by perf or apparently in the
kernel, but this warning is supposed to pre-warn about "not supported"
being returned when frame pointer unwinding is requested. Of course,
lacking an s390 I've never tested this. I can imagine the warning
being overly broad, but my understanding is cpum_sf is the only PMU
capable of sampling on s390.

Perhaps what is needed is additional text after the "Use '--call-graph
dwarf'". The existing initial text at least appears to match the
kernel code. I'm not clear how the lack of back chains would be
reported as an error code, and this could introduce confusion as we
have lots of stack, chain and branch related terms in the perf
codebase.

Thanks,
Ian

>         }
>
>         evsel__set_sample_bit(evsel, CALLCHAIN);
>
> base-commit: f9a2394a23482bfd330911e9c8295b71724feacd
> --
> 2.53.0
>

Re: [PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[Jens Remus]

Hi Ian!

On 8/8/2026 6:13 AM, Ian Rogers wrote:
> On Fri, Aug 7, 2026 at 3:49 AM Jens Remus <jremus@linux.ibm.com> wrote:
>>
>> On s390 the kernel uses s390 back chain instead of frame pointers for
>> stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add
>> USER_STACKTRACE support").  This is because frame pointers on s390
>> cannot be used for stack tracing. [1]
>>
>> This requires user space to maintain a s390 back chain.  For instance
>> user space to be built with compiler option '-mbackchain' (instead of
>> '-fno-omit-frame-pointer' used on other architectures, which should
>> better not be used on s390 [1]).
>>
>> Only few distributions and users built user space with '-mbackchain'.
>> Therefore '--call-graph fp' may not produce the expected results.
>>
>> Commit ca76fb67ebdd ("perf evlist: Improve default event for s390")
>> added a warning for s390 that wrongly claimed that "Framepointer
>> unwinding lacks kernel support".  Change the warning to hint at using
>> '--call-graph dwarf' if user space does not maintain a s390 back chain.
>>
>> Note that '--call-graph fp' may also be useful for other applications,
>> such as OpenJDK maintaining a s390 back chain (does not require JVM
>> option '-XX:+PreserveFramePointer' on s390):
>>
>>   $ perf record --call-graph fp ... -- \
>>     java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...
>>
>> [1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,
>>      https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
>>
>> Fixes: ca76fb67ebdd ("perf evlist: Improve default event for s390")
>> Signed-off-by: Jens Remus <jremus@linux.ibm.com>
>> ---
>>  tools/perf/util/evsel.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
>> index ea9fa04429f0..c4d67d1b35f3 100644
>> --- a/tools/perf/util/evsel.c
>> +++ b/tools/perf/util/evsel.c
>> @@ -1080,7 +1080,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
>>
>>         if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
>>                 pr_warning_once(
>> -                       "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
>> +                       "Use '--call-graph dwarf' if user space does not maintain a s390 back chain "
>> +                       "(e.g. is not built with '-mbackchain').\n");
> 
> Thanks Jens. The text was based on:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/kernel/perf_cpum_sf.c#n853
> ```
> static int cpumsf_pmu_event_init(struct perf_event *event)
> ...
>  /* No support for callchain, stacks and registers */
> if (has_branch_stack(event) || is_callchain_event(event))
> return -EOPNOTSUPP;
> ```

According to Thomas, the s390 HW event 'cycles' does not support
callchains.  This is because on s390 it provides aggregated historical
data, making it impossible to associate a callchain to the sampled data.
Your referenced cpumsf_pmu_event_init() code applies specifically to the
CPU Measurement Sampling Facility (CPUMSF) PMU and therefore correctly
rejects callchains for those HW events.

On s390, callchains are therefore only supported with SW events.

Since your commit ca76fb67ebdd ("perf evlist: Improve default event for
s390") callchains on s390 default to (1) the SW event 'cpu-clock' (or
'task-clock') and (2) 'dwarf'.  The kernel on s390 also supports 'fp'
callchains using s390 back chain since commit aa44433ac4ee ("s390: add
USER_STACKTRACE support").

Therefore the current warning on s390 for 'fp' is misleading.  Given
that on s390 a user must explicitly select 'fp', and that on other
architectures there is no warning about the requirement for user space
to maintain frame pointers (e.g. if they were not built with
'-fno-omit-frame-pointer' and '-mno-omoit-leaf-frame-pointer') it might
be preferable to remove the warning altogether.

We could document in the perf man pages that 'fp' on s390 relies on the
s390 back chain rather than frame pointers.

What do you think?

> The backchain option is never tested by perf or apparently in the
> kernel, but this warning is supposed to pre-warn about "not supported"
> being returned when frame pointer unwinding is requested. Of course,
> lacking an s390 I've never tested this. I can imagine the warning
> being overly broad, but my understanding is cpum_sf is the only PMU
> capable of sampling on s390.
> 
> Perhaps what is needed is additional text after the "Use '--call-graph
> dwarf'". The existing initial text at least appears to match the
> kernel code. I'm not clear how the lack of back chains would be
> reported as an error code, and this could introduce confusion as we
> have lots of stack, chain and branch related terms in the perf
> codebase.

This is analogous to frame pointers on other architectures: the absence
of a maintained s390 back chain is not reported as an error.  There is
currently no mean to determine from and ELF binary whether it was built
to maintain a s390 back chain (i.e. no ELF attribute/flag).

>>         }
>>
>>         evsel__set_sample_bit(evsel, CALLCHAIN);
>>
>> base-commit: f9a2394a23482bfd330911e9c8295b71724feacd
>> --
>> 2.53.0
Regards,
Jens
-- 
Jens Remus
Linux on Z Development (D3303)
jremus@de.ibm.com / jremus@linux.ibm.com

IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
IBM Data Privacy Statement: https://www.ibm.com/privacy/

Re: [PATCH] perf evsel: Improve frame pointer unwinding warning for s390

[Ian Rogers]

On Tue, Aug 11, 2026 at 2:53 AM Jens Remus <jremus@linux.ibm.com> wrote:
>
> Hi Ian!
>
> On 8/8/2026 6:13 AM, Ian Rogers wrote:
> > On Fri, Aug 7, 2026 at 3:49 AM Jens Remus <jremus@linux.ibm.com> wrote:
> >>
> >> On s390 the kernel uses s390 back chain instead of frame pointers for
> >> stack tracing of user space since v6.7 commit aa44433ac4ee ("s390: add
> >> USER_STACKTRACE support").  This is because frame pointers on s390
> >> cannot be used for stack tracing. [1]
> >>
> >> This requires user space to maintain a s390 back chain.  For instance
> >> user space to be built with compiler option '-mbackchain' (instead of
> >> '-fno-omit-frame-pointer' used on other architectures, which should
> >> better not be used on s390 [1]).
> >>
> >> Only few distributions and users built user space with '-mbackchain'.
> >> Therefore '--call-graph fp' may not produce the expected results.
> >>
> >> Commit ca76fb67ebdd ("perf evlist: Improve default event for s390")
> >> added a warning for s390 that wrongly claimed that "Framepointer
> >> unwinding lacks kernel support".  Change the warning to hint at using
> >> '--call-graph dwarf' if user space does not maintain a s390 back chain.
> >>
> >> Note that '--call-graph fp' may also be useful for other applications,
> >> such as OpenJDK maintaining a s390 back chain (does not require JVM
> >> option '-XX:+PreserveFramePointer' on s390):
> >>
> >>   $ perf record --call-graph fp ... -- \
> >>     java -XX:+UnlockDiagnosticVMOptions -XX:+DumpPerfMapAtExit ...
> >>
> >> [1]: s390: Stack tracing using Frame Pointer, Back Chain, and SFrame,
> >>      https://conf.gnu-tools-cauldron.org/opo25/talk/Y3CVHY/
> >>
> >> Fixes: ca76fb67ebdd ("perf evlist: Improve default event for s390")
> >> Signed-off-by: Jens Remus <jremus@linux.ibm.com>
> >> ---
> >>  tools/perf/util/evsel.c | 3 ++-
> >>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> >> index ea9fa04429f0..c4d67d1b35f3 100644
> >> --- a/tools/perf/util/evsel.c
> >> +++ b/tools/perf/util/evsel.c
> >> @@ -1080,7 +1080,8 @@ static void __evsel__config_callchain(struct evsel *evsel, const struct record_o
> >>
> >>         if (EM_HOST == EM_S390 && param->record_mode == CALLCHAIN_FP) {
> >>                 pr_warning_once(
> >> -                       "Framepointer unwinding lacks kernel support. Use '--call-graph dwarf'\n");
> >> +                       "Use '--call-graph dwarf' if user space does not maintain a s390 back chain "
> >> +                       "(e.g. is not built with '-mbackchain').\n");
> >
> > Thanks Jens. The text was based on:
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/s390/kernel/perf_cpum_sf.c#n853
> > ```
> > static int cpumsf_pmu_event_init(struct perf_event *event)
> > ...
> >  /* No support for callchain, stacks and registers */
> > if (has_branch_stack(event) || is_callchain_event(event))
> > return -EOPNOTSUPP;
> > ```
>
> According to Thomas, the s390 HW event 'cycles' does not support
> callchains.  This is because on s390 it provides aggregated historical
> data, making it impossible to associate a callchain to the sampled data.
> Your referenced cpumsf_pmu_event_init() code applies specifically to the
> CPU Measurement Sampling Facility (CPUMSF) PMU and therefore correctly
> rejects callchains for those HW events.
>
> On s390, callchains are therefore only supported with SW events.
>
> Since your commit ca76fb67ebdd ("perf evlist: Improve default event for
> s390") callchains on s390 default to (1) the SW event 'cpu-clock' (or
> 'task-clock') and (2) 'dwarf'.  The kernel on s390 also supports 'fp'
> callchains using s390 back chain since commit aa44433ac4ee ("s390: add
> USER_STACKTRACE support").
>
> Therefore the current warning on s390 for 'fp' is misleading.  Given
> that on s390 a user must explicitly select 'fp', and that on other
> architectures there is no warning about the requirement for user space
> to maintain frame pointers (e.g. if they were not built with
> '-fno-omit-frame-pointer' and '-mno-omoit-leaf-frame-pointer') it might
> be preferable to remove the warning altogether.
>
> We could document in the perf man pages that 'fp' on s390 relies on the
> s390 back chain rather than frame pointers.
>
> What do you think?

So firstly, sorry for the misleading message and thanks for trying to
fix it! Also, sorry for the delay in responding and dealing with some
hospital things. Man page documentation sounds good to me, and having
a good warning also sounds good. Since this is for s390 you guys are
much smarter about what to do than I am. Since the function generating
the warning has an evsel, we can get the PMU from the evsel and check
things like:
```
if (evsel->pmu && perf_pmu__is_software(evsel->pmu))
```
By which I mean we can provide different warnings for hardware and
software events. That said, as a hardware event will fail in the
perf_event_open I'm not sure it is a useful distinction.

> > The backchain option is never tested by perf or apparently in the
> > kernel, but this warning is supposed to pre-warn about "not supported"
> > being returned when frame pointer unwinding is requested. Of course,
> > lacking an s390 I've never tested this. I can imagine the warning
> > being overly broad, but my understanding is cpum_sf is the only PMU
> > capable of sampling on s390.
> >
> > Perhaps what is needed is additional text after the "Use '--call-graph
> > dwarf'". The existing initial text at least appears to match the
> > kernel code. I'm not clear how the lack of back chains would be
> > reported as an error code, and this could introduce confusion as we
> > have lots of stack, chain and branch related terms in the perf
> > codebase.
>
> This is analogous to frame pointers on other architectures: the absence
> of a maintained s390 back chain is not reported as an error.  There is
> currently no mean to determine from and ELF binary whether it was built
> to maintain a s390 back chain (i.e. no ELF attribute/flag).

Thanks for the clarification! I think given this I'm happy to add my tag:

Reviewed-by: Ian Rogers <irogers@google.com>

Do you want the maintainers to move forward with this change, or would
you prefer to add something to the man pages, etc. ?

Thanks,
Ian

> >>         }
> >>
> >>         evsel__set_sample_bit(evsel, CALLCHAIN);
> >>
> >> base-commit: f9a2394a23482bfd330911e9c8295b71724feacd
> >> --
> >> 2.53.0
> Regards,
> Jens
> --
> Jens Remus
> Linux on Z Development (D3303)
> jremus@de.ibm.com / jremus@linux.ibm.com
>
> IBM Deutschland Research & Development GmbH; Vorsitzender des Aufsichtsrats: Wolfgang Wendt; Geschäftsführung: David Faller; Sitz der Gesellschaft: Ehningen; Registergericht: Amtsgericht Stuttgart, HRB 243294
> IBM Data Privacy Statement: https://www.ibm.com/privacy/
>