[PATCH 08/15] perf/x86/intel: Fix stale PEBS count without counter-group support

[Dapeng Mi <dapeng1.mi@linux.intel.com>]

On platforms without PEBS counter-group support (for example, Sapphire
Rapids), PEBS group read samples can report mismatched counts between the
precise event and its sibling, e.g.,

$ perf record -e \
 '{cpu/instructions,period=100000/pu,cpu/instructions/u}:S' -- sleep 1

The invalid event counts are seen,

1st PEBS record:
.... group nr 2
..... id 0000000000006189, value 0000000000000000, lost 0
..... id 0000000000006269, value 00000000000186c1, lost 0

2nd PEBS record:
.... group nr 2
..... id 0000000000006189, value 00000000000186a0, lost 0
..... id 0000000000006269, value 0000000000030d6b, lost 0

In PEBS records, the first event count can lag behind the second event by
about one sample period, even though both counts should be close or equal.

The root cause is ordering in __intel_pmu_pebs_last_event(): PEBS event
counts are updated after perf_event_output()/perf_event_overflow(). As a
result, perf_output_read_group() snapshots stale PEBS counts.

Move the PEBS count update before
perf_event_output()/perf_event_overflow(), matching the non-PEBS ordering
in handle_pmi_common(), so group reads report correct PEBS event counts.

Fixes: 3c00ed344cef ("perf/x86/intel/ds: Factor out functions for PEBS records processing")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
 arch/x86/events/intel/ds.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c
index f68391d60b2d..1a32f4168a84 100644
--- a/arch/x86/events/intel/ds.c
+++ b/arch/x86/events/intel/ds.c
@@ -2956,22 +2956,6 @@ __intel_pmu_pebs_last_event(struct perf_event *event,
 		setup_sample(event, iregs, at, data, regs);
 	}
 
-	if (iregs == &dummy_iregs) {
-		/*
-		 * The PEBS records may be drained in the non-overflow context,
-		 * e.g., large PEBS + context switch. Perf should treat the
-		 * last record the same as other PEBS records, and doesn't
-		 * invoke the generic overflow handler.
-		 */
-		perf_event_output(event, data, regs);
-	} else {
-		/*
-		 * All but the last records are processed.
-		 * The last one is left to be able to call the overflow handler.
-		 */
-		perf_event_overflow(event, data, regs);
-	}
-
 	if (hwc->flags & PERF_X86_EVENT_AUTO_RELOAD) {
 		if (is_pebs_counter_event_group(event)) {
 			if (corrupted) {
@@ -3011,6 +2995,23 @@ __intel_pmu_pebs_last_event(struct perf_event *event,
 		else
 			intel_pmu_save_and_restart(event);
 	}
+
+	if (iregs == &dummy_iregs) {
+		/*
+		 * The PEBS records may be drained in the non-overflow context,
+		 * e.g., large PEBS + context switch. Perf should treat the
+		 * last record the same as other PEBS records, and doesn't
+		 * invoke the generic overflow handler.
+		 */
+		perf_event_output(event, data, regs);
+	} else {
+		/*
+		 * All but the last records are processed.
+		 * The last one is left to be able to call the overflow handler.
+		 */
+		perf_event_overflow(event, data, regs);
+	}
+
 }
 
 static DEFINE_PER_CPU(struct x86_perf_regs, x86_pebs_regs);
-- 
2.34.1