[PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[PATCH] perf/x86/amd/lbr: Freeze LBR on PMI only when kernel branches are recorded

[Puranjay Mohan]

On CPUs with X86_FEATURE_AMD_LBR_PMC_FREEZE the LBR is always frozen on
PMI, so the branch stack ends before the IP and callchain, which are
taken when the NMI is delivered. IBS rejects branch-stack events, so
there is no precise alternative.

The freeze only keeps the PMI handler's branches out of the stack, which
matters only when branches into the kernel are recorded; without it the
stack fills with the handler's own branches. A user-only filter does not
record them, and parts without the feature are never frozen (see commit
598c2fafc06f ("perf/x86/amd/lbr: Use freeze based on availability")).

Freeze unless the value written to LBR_SELECT, which is shared by all
events on the CPU, suppresses branches into the kernel.

With perf record -e ex_ret_brn_tkn:u -j any,u on a Zen 5 part running
CPython, the function of the sampled IP differs from the function of
LBR[0].to in 84.3% of samples with the freeze and in 0.1% without it.

Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
---
 arch/x86/events/amd/lbr.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
index a55646fcb8465..7ec50380ac81d 100644
--- a/arch/x86/events/amd/lbr.c
+++ b/arch/x86/events/amd/lbr.c
@@ -392,7 +392,7 @@ void amd_pmu_lbr_sched_task(struct perf_event_pmu_context *pmu_ctx,
 void amd_pmu_lbr_enable_all(void)
 {
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
-	u64 lbr_select, dbg_ctl, dbg_extn_cfg;
+	u64 lbr_select = 0, dbg_ctl, dbg_extn_cfg;
 
 	if (!cpuc->lbr_users || !x86_pmu.lbr_nr)
 		return;
@@ -405,7 +405,12 @@ void amd_pmu_lbr_enable_all(void)
 
 	if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
 		rdmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
-		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
+		/* Freeze unless the filter suppresses branches into the kernel */
+		if (lbr_select & LBR_KERNEL)
+			dbg_ctl &= ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
+		else
+			dbg_ctl |= DEBUGCTLMSR_FREEZE_LBRS_ON_PMI;
+		wrmsrq(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
 	}
 
 	rdmsrq(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);

base-commit: 0f443787a7f6ae907975742680aac2600483eaf1
-- 
2.53.0-Meta