[Patch v5 16/18] perf/x86/intel: Refill LBR stack for LBR counter events in PMI handler

[Dapeng Mi <dapeng1.mi@linux.intel.com>]

When handling an LBR counter event, the raw LBR counters in the reserved
field of each LBR entry are cleared after the stack is processed and
reported to userspace. This prevents raw counter data from leaking.

However, if multiple events overflow in the same PMI, an LBR counter
event may not be the first one processed. In that case, the raw LBR
counters may already have been cleared by the previous LBR event, so the
later LBR counter event cannot read the correct values.

Refill the LBR stack before processing an LBR counter event when multiple
events are handled in a single PMI. This restores the correct raw LBR
counter data for the event in this corner case.

This situation is rare, the performance impact from extra refill should
be negligible.

Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 33744916196b ("perf/x86/intel: Support branch counters logging")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/core.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index e1834b329e37..9267f2d7d21b 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -3864,6 +3864,7 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 {
 	struct perf_sample_data data;
 	struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
+	bool lbr_counter_cleared = false;
 	bool lbr_need_update = false;
 	int handled = 0;
 	int bit;
@@ -4025,12 +4026,20 @@ static int handle_pmi_common(struct pt_regs *regs, u64 status)
 		perf_sample_data_init(&data, 0, last_period);
 
 		if (has_branch_stack(event)) {
-			if (lbr_need_update) {
+			/*
+			 * The raw LBR counters stored in the reserved field of
+			 * the LBR entries were cleared during processing of the
+			 * previous LBR event, so refresh the LBR stack again if
+			 * this event needs the LBR counters.
+			 */
+			if (lbr_need_update ||
+			    (lbr_counter_cleared && is_branch_counters_group(event))) {
 				intel_pmu_lbr_read(&cpuc->lbr_stack,
 					cpuc->lbr_entries, cpuc->lbr_counters);
 				lbr_need_update = false;
 			}
 			intel_pmu_lbr_save_brstack(&data, cpuc, event);
+			lbr_counter_cleared = true;
 		}
 
 		x86_pmu_clear_perf_regs(regs);
-- 
2.34.1