[PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[Puranjay Mohan]

On CPUs with X86_FEATURE_AMD_LBR_PMC_FREEZE the LBR is always frozen on
PMI, so the branch stack ends before the IP and callchain, which are
taken when the NMI is delivered. IBS rejects branch-stack events, so
there is no precise alternative.

The freeze only keeps the PMI handler's branches out of the stack, which
matters only when branches into the kernel are recorded; without it the
stack fills with the handler's own branches. A user-only filter does not
record them, and parts without the feature are never frozen (see commit
598c2fafc06f ("perf/x86/amd/lbr: Use freeze based on availability")).

Freeze unless the value written to LBR_SELECT, which is shared by all
events on the CPU, suppresses branches into the kernel.

With perf record -e ex_ret_brn_tkn:u -j any,u on a Zen 5 part running
CPython, the function of the sampled IP differs from the function of
LBR[0].to in 84.3% of samples with the freeze and in 0.1% without it.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
 arch/x86/events/amd/lbr.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index a55646fcb8465..7ec50380ac81d 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -392,7 +392,7 @@ void amd_pmu_lbr_sched_task(struct perf_event_pmu_context *pmu_ctx,
 void amd_pmu_lbr_enable_all(void)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	u64 lbr_select, dbg_ctl, dbg_extn_cfg;
+	u64 lbr_select = 0, dbg_ctl, dbg_extn_cfg;
 
 	if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
 		return;
@@ -405,7 +405,12 @@ void amd_pmu_lbr_enable_all(void)
 
 	if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
 		rdmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
-		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
+		/* Freeze unless the filter suppresses branches into the kernel */
+		if (lbr_select & LBR_KERNEL)
+			dbg_ctl &= ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
+		else
+			dbg_ctl |= DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
+		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
 	}
 
 	rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);

base-commit: 0f443787a7f6ae907975742680aac2600483eaf1
-- 
2.53.0-Meta

Re: [PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[Sandipan Das]

On 23-10-2026 18:11, Puranjay Mohan wrote:
> On CPUs with X86_FEATURE_AMD_LBR_PMC_FREEZE the LBR is always frozen on
> PMI, so the branch stack ends before the IP and callchain, which are
> taken when the NMI is delivered. IBS rejects branch-stack events, so
> there is no precise alternative.
> 
> The freeze only keeps the PMI handler's branches out of the stack, which
> matters only when branches into the kernel are recorded; without it the
> stack fills with the handler's own branches. A user-only filter does not
> record them, and parts without the feature are never frozen (see commit
> 598c2fafc06f ("perf/x86/amd/lbr: Use freeze based on availability")).
> 
> Freeze unless the value written to LBR_SELECT, which is shared by all
> events on the CPU, suppresses branches into the kernel.
> 
> With perf record -e ex_ret_brn_tkn:u -j any,u on a Zen 5 part running
> CPython, the function of the sampled IP differs from the function of
> LBR[0].to in 84.3% of samples with the freeze and in 0.1% without it.
> 

I think the reason behind this observation is skid. Without freeze, LBR
keeps recording new branches until the PMI arrives and causes a privilege
level change. With freeze, it stops at the exact point where a PMC
overflows.

The sites where the overflows occur are the potential hotspots. So if
skid is large enough and the intent of PGO workflows is to always capture
the last branch right before the overflow/hotspot, then having freeze
enabled may be necessary.

> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
>  arch/x86/events/amd/lbr.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
> index a55646fcb8465..7ec50380ac81d 100644
> --- a/arch/x86/events/amd/lbr.c
> +++ b/arch/x86/events/amd/lbr.c
> @@ -392,7 +392,7 @@ void amd_pmu_lbr_sched_task(struct perf_event_pmu_context *pmu_ctx,
>  void amd_pmu_lbr_enable_all(void)
>  {
>  	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
> -	u64 lbr_select, dbg_ctl, dbg_extn_cfg;
> +	u64 lbr_select = 0, dbg_ctl, dbg_extn_cfg;
>  
>  	if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
>  		return;
> @@ -405,7 +405,12 @@ void amd_pmu_lbr_enable_all(void)
>  
>  	if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
>  		rdmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
> -		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
> +		/* Freeze unless the filter suppresses branches into the kernel */
> +		if (lbr_select & LBR_KERNEL)
> +			dbg_ctl &= ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
> +		else
> +			dbg_ctl |= DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
> +		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>  	}
>  
>  	rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
> 
> base-commit: 0f443787a7f6ae907975742680aac2600483eaf1

Re: [PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[Andrii Nakryiko]

On Fri, Oct 23, 2026 at 5:42 AM Puranjay Mohan <puranjay@kernel.org> wrote:
>
> On CPUs with X86_FEATURE_AMD_LBR_PMC_FREEZE the LBR is always frozen on
> PMI, so the branch stack ends before the IP and callchain, which are
> taken when the NMI is delivered. IBS rejects branch-stack events, so
> there is no precise alternative.
>
> The freeze only keeps the PMI handler's branches out of the stack, which
> matters only when branches into the kernel are recorded; without it the
> stack fills with the handler's own branches. A user-only filter does not
> record them, and parts without the feature are never frozen (see commit
> 598c2fafc06f ("perf/x86/amd/lbr: Use freeze based on availability")).
>
> Freeze unless the value written to LBR_SELECT, which is shared by all
> events on the CPU, suppresses branches into the kernel.
>
> With perf record -e ex_ret_brn_tkn:u -j any,u on a Zen 5 part running
> CPython, the function of the sampled IP differs from the function of
> LBR[0].to in 84.3% of samples with the freeze and in 0.1% without it.
>
> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
> ---
>  arch/x86/events/amd/lbr.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
> index a55646fcb8465..7ec50380ac81d 100644
> --- a/arch/x86/events/amd/lbr.c
> +++ b/arch/x86/events/amd/lbr.c
> @@ -392,7 +392,7 @@ void amd_pmu_lbr_sched_task(struct perf_event_pmu_context *pmu_ctx,
>  void amd_pmu_lbr_enable_all(void)
>  {
>         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
> -       u64 lbr_select, dbg_ctl, dbg_extn_cfg;
> +       u64 lbr_select = 0, dbg_ctl, dbg_extn_cfg;
>
>         if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
>                 return;
> @@ -405,7 +405,12 @@ void amd_pmu_lbr_enable_all(void)
>
>         if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
>                 rdmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
> -               wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
> +               /* Freeze unless the filter suppresses branches into the kernel */
> +               if (lbr_select & LBR_KERNEL)

On Intel CPU side, we reset this FREEZE bit in
__intel_pmu_lbr_enable() if is_lbr_call_stack_bit_set(), with the
comment "LBR callstack does not work well with FREEZE_LBRS_ON_PMI.".
So this is not *exactly* what Intel side of things is doing, and I'm
not sure I fully understand the differences, so hopefully someone more
knowledgeable can help.

But what Puranjay is trying to do here is to at least make a
user-mode-only LBR and IP + callstack to be coherent (for PGO data
collection, which we then use for user space app compilation), by
disabling freezing for LBR.

It doesn't seem like there is anything we can do on AMD hardware for
when we want kernel-side LBR and precise IP to be in sync,
unfortunately, but at least for user-space only case it would allow us
to not plain disable freezing altogether at kernel boot time.

Does this approach make sense?


> +                       dbg_ctl &= ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
> +               else
> +                       dbg_ctl |= DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
> +               wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>         }
>
>         rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
>
> base-commit: 0f443787a7f6ae907975742680aac2600483eaf1
> --
> 2.53.0-Meta
>

Re: [PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[Sandipan Das]

On 30-10-2026 01:42, Andrii Nakryiko wrote:
> On Fri, Oct 23, 2026 at 5:42 AM Puranjay Mohan <puranjay@kernel.org> wrote:
>>
>> On CPUs with X86_FEATURE_AMD_LBR_PMC_FREEZE the LBR is always frozen on
>> PMI, so the branch stack ends before the IP and callchain, which are
>> taken when the NMI is delivered. IBS rejects branch-stack events, so
>> there is no precise alternative.
>>
>> The freeze only keeps the PMI handler's branches out of the stack, which
>> matters only when branches into the kernel are recorded; without it the
>> stack fills with the handler's own branches. A user-only filter does not
>> record them, and parts without the feature are never frozen (see commit
>> 598c2fafc06f ("perf/x86/amd/lbr: Use freeze based on availability")).
>>
>> Freeze unless the value written to LBR_SELECT, which is shared by all
>> events on the CPU, suppresses branches into the kernel.
>>
>> With perf record -e ex_ret_brn_tkn:u -j any,u on a Zen 5 part running
>> CPython, the function of the sampled IP differs from the function of
>> LBR[0].to in 84.3% of samples with the freeze and in 0.1% without it.
>>
>> Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
>> ---
>>  arch/x86/events/amd/lbr.c | 9 +++++++--
>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
>> index a55646fcb8465..7ec50380ac81d 100644
>> --- a/arch/x86/events/amd/lbr.c
>> +++ b/arch/x86/events/amd/lbr.c
>> @@ -392,7 +392,7 @@ void amd_pmu_lbr_sched_task(struct perf_event_pmu_context *pmu_ctx,
>>  void amd_pmu_lbr_enable_all(void)
>>  {
>>         struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
>> -       u64 lbr_select, dbg_ctl, dbg_extn_cfg;
>> +       u64 lbr_select = 0, dbg_ctl, dbg_extn_cfg;
>>
>>         if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
>>                 return;
>> @@ -405,7 +405,12 @@ void amd_pmu_lbr_enable_all(void)
>>
>>         if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
>>                 rdmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>> -               wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
>> +               /* Freeze unless the filter suppresses branches into the kernel */
>> +               if (lbr_select & LBR_KERNEL)
> 
> On Intel CPU side, we reset this FREEZE bit in
> __intel_pmu_lbr_enable() if is_lbr_call_stack_bit_set(), with the
> comment "LBR callstack does not work well with FREEZE_LBRS_ON_PMI.".
> So this is not *exactly* what Intel side of things is doing, and I'm
> not sure I fully understand the differences, so hopefully someone more
> knowledgeable can help.
> 
> But what Puranjay is trying to do here is to at least make a
> user-mode-only LBR and IP + callstack to be coherent (for PGO data
> collection, which we then use for user space app compilation), by
> disabling freezing for LBR.
> 
> It doesn't seem like there is anything we can do on AMD hardware for
> when we want kernel-side LBR and precise IP to be in sync,
> unfortunately, but at least for user-space only case it would allow us
> to not plain disable freezing altogether at kernel boot time.
> 
> Does this approach make sense?
> 

Yes, the intent to keep the LBR and IP coherent is clear. However, my
concern is that this may end up dropping the branches preceding the
hotspots since a CPU might keep executing in userspace until the PMI
handler runs. I'm not sure about the effect that this is going to have
on PGO but what if we rely on a mechanism (UAPI + tool flags) to let
users explicitly request freeze?

If there is an use-case where freeze helps, then the facility can be
explicitly requested.

> 
>> +                       dbg_ctl &= ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
>> +               else
>> +                       dbg_ctl |= DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
>> +               wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>>         }
>>
>>         rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
>>
>> base-commit: 0f443787a7f6ae907975742680aac2600483eaf1
>> --
>> 2.53.0-Meta
>>