* [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it
@ 2026-11-19 21:10 Usama Arif
2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw)
To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin,
jolsa, irogers, adrian.hunter, james.clark, linux-perf-users,
linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel,
shakeel.butt, kernel-team
Cc: Usama Arif
do_perf_sw_event() builds samples for software events (context switches,
page faults, ...) in an on-stack struct perf_sample_data. This series
fixes two bugs around that struct and then stops zero-initializing it,
a 256-byte memset that every context switch pays once a context-switch
counter is open anywhere.
Patch 1 initializes the struct for each matching event. Today a sampling
event inherits the sample ID, timestamp, pid/tid and callchain prepared
for the event before it, and sometimes a record size smaller than what
is then written, overrunning the records that follow in its ring buffer.
Two concurrent "perf record -e page-faults,context-switches -c 1 -a"
sessions make the older one fail. An unprivileged user's page-fault
event can do this to root's CPU-wide session. Broken since v6.3.
Patch 2 stops BPF programs from reading an unset data->addr through
bpf_perf_event_data::addr. On INIT_STACK_NONE kernels this returned
leftover stack, mostly kernel pointers, on almost all context-switch
and cpu-clock samples in my tests. INIT_STACK_ALL_ZERO mostly hides it.
It affects all event types, needs CAP_BPF and CAP_PERFMON, and has been
there since v6.1.
Patch 3 marks the struct __uninitialized. On every path from do_perf_sw_event(),
each field is written before it is read, counting events never read the
struct, and it is never copied as a whole, so its padding never leaves
the kernel. A privileged BPF program using bpf_cast_to_kern_ctx() can
still read unprepared fields, and would see stack contents instead of 0,
as it already does on INIT_STACK_NONE kernels.
Patches 1-2 are for perf/urgent and stable, patch 3 for perf/core.
Performance: with sched_yield() ping-pong on one CPU, software counters
add on average 54 ns to a ~627 ns context switch, and patch 3 brings that
to 44 ns (10 rounds, spec_rstack_overflow=off).
Usama Arif (3):
perf/core: Fix perf_sample_data reuse across software events
perf/core: Don't let BPF programs read an uninitialized sample address
perf/core: Don't zero perf_sample_data in do_perf_sw_event()
kernel/events/core.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
--
2.53.1-Meta
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events 2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif @ 2026-11-19 21:10 ` Usama Arif 2026-11-19 21:10 ` [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address Usama Arif 2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif 2 siblings, 0 replies; 8+ messages in thread From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw) To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team Cc: Usama Arif, stable do_perf_sw_event() passes one on-stack struct perf_sample_data to every matching event. perf_prepare_sample() fills in the fields an event samples, marks them in data->sample_flags and skips fields that are already marked, and those marks survive from one event to the next. So a sampling event later in the list emits the sample ID, timestamp, pid/tid and callchain prepared for an earlier one. Its record size is computed from the earlier event's dynamic fields as well and can come out smaller than what is written, in which case the kernel writes past its ring-buffer reservation over the records that follow. Two "perf record -e page-faults,context-switches -c 1 -a" sessions running at the same time hit this: the one started first gets the other's sample IDs and fails with "Bad address". Commit 1d1bfe30dad5 ("perf/core: Fix perf_sample_data not properly initialized for different swevents in perf_tp_event()") fixed the same problem for tracepoints. Do the same here: initialize the struct for each matching event. While at it, drop the unused data argument of perf_swevent_match(). Fixes: bb447c27a467 ("perf/core: Set data->sample_flags in perf_prepare_sample()") Cc: stable@vger.kernel.org Signed-off-by: Usama Arif <usama.arif@linux.dev> --- kernel/events/core.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 687bd398275ba3..1c1ffd103c74a0 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -11142,7 +11142,6 @@ int perf_exclude_event(struct perf_event *event, struct pt_regs *regs) static int perf_swevent_match(struct perf_event *event, enum perf_type_id type, u32 event_id, - struct perf_sample_data *data, struct pt_regs *regs) { if (event->attr.type != type) @@ -11207,11 +11206,10 @@ find_swevent_head(struct swevent_htable *swhash, struct perf_event *event) } static void do_perf_sw_event(enum perf_type_id type, u32 event_id, - u64 nr, - struct perf_sample_data *data, - struct pt_regs *regs) + u64 nr, struct pt_regs *regs, u64 addr) { struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); + struct perf_sample_data data; struct perf_event *event; struct hlist_head *head; @@ -11221,8 +11219,10 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, goto end; hlist_for_each_entry_rcu(event, head, hlist_entry) { - if (perf_swevent_match(event, type, event_id, data, regs)) - perf_swevent_event(event, nr, data, regs); + if (perf_swevent_match(event, type, event_id, regs)) { + perf_sample_data_init(&data, addr, 0); + perf_swevent_event(event, nr, &data, regs); + } } end: rcu_read_unlock(); @@ -11243,13 +11243,10 @@ void perf_swevent_put_recursion_context(int rctx) void ___perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { - struct perf_sample_data data; - if (WARN_ON_ONCE(!regs)) return; - perf_sample_data_init(&data, addr, 0); - do_perf_sw_event(PERF_TYPE_SOFTWARE, event_id, nr, &data, regs); + do_perf_sw_event(PERF_TYPE_SOFTWARE, event_id, nr, regs, addr); } void __perf_sw_event(u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) -- 2.53.1-Meta ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address 2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif 2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif @ 2026-11-19 21:10 ` Usama Arif 2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif 2 siblings, 0 replies; 8+ messages in thread From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw) To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team Cc: Usama Arif, stable A BPF program attached to a perf event gets a struct bpf_perf_event_data, and reading its addr field turns into a load from the perf_sample_data the sample is being assembled in. The verifier allows that load no matter which fields the event samples. perf_sample_data_init() used to store the address unconditionally, so the program read 0 when there was none. To touch fewer cachelines, it now stores the address only when it is non-zero, and perf_prepare_sample() fills it in only when the event samples it (PERF_SAMPLE_ADDR, PHYS_ADDR or DATA_PAGE_SIZE). For all other events, including context switches, cpu-clock and hardware counters like cycles, the program reads whatever was left in that stack slot, often a kernel pointer. Kernels built with CONFIG_INIT_STACK_ALL_ZERO, the default when the compiler supports it, mostly hide this by clearing the struct on function entry. With INIT_STACK_ALL_PATTERN the program reads the fill pattern instead. Clear the address before running the program if nobody filled it in, the same way the SIGTRAP path already checks PERF_SAMPLE_ADDR before using it. Events without a BPF program are not affected. Fixes: 7b0846301531 ("perf: Use sample_flags for addr") Cc: stable@vger.kernel.org Signed-off-by: Usama Arif <usama.arif@linux.dev> --- kernel/events/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/events/core.c b/kernel/events/core.c index 1c1ffd103c74a0..57ea7e3c01a025 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10819,6 +10819,9 @@ static int bpf_overflow_handler(struct perf_event *event, prog = READ_ONCE(event->prog); if (prog) { perf_prepare_sample(data, event, regs); + /* bpf_perf_event_data::addr is readable without PERF_SAMPLE_ADDR */ + if (!(data->sample_flags & PERF_SAMPLE_ADDR)) + data->addr = 0; ret = bpf_prog_run(prog, &ctx); } rcu_read_unlock(); -- 2.53.1-Meta ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() 2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif 2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif 2026-11-19 21:10 ` [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address Usama Arif @ 2026-11-19 21:10 ` Usama Arif 2026-11-20 9:11 ` Peter Zijlstra 2 siblings, 1 reply; 8+ messages in thread From: Usama Arif @ 2026-11-19 21:10 UTC (permalink / raw) To: peterz, mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team Cc: Usama Arif Kernels built with CONFIG_INIT_STACK_ALL_ZERO, the default when the compiler supports it, clear every local variable on function entry. For the 256-byte struct perf_sample_data in do_perf_sw_event() that is a memset, and it runs on every entry, before the list of events is even looked up. do_perf_sw_event() is entered for a software event, e.g. a context switch, as soon as one event of that kind is open anywhere in the system. Counting events add to event->count and never read the struct. For every matching event, perf_sample_data_init() resets sample_flags, period and dyn_size and stores the address if there is one; for a sampling event, perf_prepare_sample() then fills in what its sample_type asks for. A single context-switch counter, e.g. one kept open by a monitoring daemon, is enough for every context switch on the machine to pay for the clearing. On hosts running many small VMs, ___perf_sw_event() is 0.3% of all kernel cycles, and the memset about a seventh of that. Mark the struct __uninitialized, which is what kernels without stack auto-initialization get anyway. Because its contents are copied into a ring buffer that userspace maps, every field that is read must have been written first. That holds for perf_output_sample(), and with the previous patch also for what a BPF program can read through bpf_perf_event_data. In a KVM guest on AMD Zen 5 with software counters open on every CPU, this cuts the time they add to a context switch from 54 ns to 44 ns (sched_yield() between two threads on one CPU, with spec_rstack_overflow=off). Signed-off-by: Usama Arif <usama.arif@linux.dev> --- kernel/events/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 57ea7e3c01a025..fa10ad7cfd99af 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, u64 nr, struct pt_regs *regs, u64 addr) { struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct perf_event *event; struct hlist_head *head; -- 2.53.1-Meta ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() 2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif @ 2026-11-20 9:11 ` Peter Zijlstra 2026-11-20 9:12 ` Peter Zijlstra 2026-11-20 20:02 ` Usama Arif 0 siblings, 2 replies; 8+ messages in thread From: Peter Zijlstra @ 2026-11-20 9:11 UTC (permalink / raw) To: Usama Arif Cc: mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team On Thu, Nov 19, 2026 at 01:10:46PM -0800, Usama Arif wrote: > kernel/events/core.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/kernel/events/core.c b/kernel/events/core.c > index 57ea7e3c01a025..fa10ad7cfd99af 100644 > --- a/kernel/events/core.c > +++ b/kernel/events/core.c > @@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, > u64 nr, struct pt_regs *regs, u64 addr) > { > struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct perf_event *event; > struct hlist_head *head; > I like this one, didn't know about the __uninitialized thing. But I think we wants to be more consistent with this. A little git-grep seems to suggest we ought to do the same elsewhere. Shall I fold the little something below in this patch? --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -9123,7 +9123,7 @@ perf_event_read_event(struct perf_event struct task_struct *task) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct perf_read_event read_event = { .pid = perf_event_pid(event, task), .tid = perf_event_tid(event, task), @@ -9390,7 +9390,7 @@ static void perf_event_task_output(struc { struct perf_task_event *task_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct task_struct *task = task_event->task; int ret; @@ -9534,7 +9534,7 @@ static void perf_event_comm_output(struc { struct perf_comm_event *comm_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; if (!perf_event_comm_match(event)) @@ -9625,7 +9625,7 @@ static void perf_event_namespaces_output { struct perf_namespaces_event *namespaces_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; if (!perf_event_namespaces_match(event)) @@ -9748,7 +9748,7 @@ static void perf_event_cgroup_output(str { struct perf_cgroup_event *cgroup_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; u16 size = sizeof(cgroup_event->event_id) + cgroup_event->path_size; @@ -9859,7 +9859,7 @@ static void perf_event_mmap_output(struc { struct perf_mmap_event *mmap_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int size = sizeof(mmap_event->event_id) + mmap_event->file_size; u32 type = PERF_RECORD_MMAP; u16 misc = PERF_RECORD_MISC_USER; @@ -10155,7 +10155,7 @@ void perf_event_aux_event(struct perf_ev unsigned long size, u64 flags) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct perf_aux_event { struct perf_event_header header; u64 offset; @@ -10188,7 +10188,7 @@ void perf_event_aux_event(struct perf_ev void perf_log_lost_samples(struct perf_event *event, u64 lost) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; struct { @@ -10240,7 +10240,7 @@ static void perf_event_switch_output(str { struct perf_switch_event *se = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; u32 type; u16 misc; u16 size; @@ -10307,7 +10307,7 @@ static void perf_event_switch(struct tas static void perf_log_throttle(struct perf_event *event, int enable) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; struct { @@ -10363,7 +10363,7 @@ static void perf_event_ksymbol_output(st { struct perf_ksymbol_event *ksymbol_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; u16 size = sizeof(ksymbol_event->event_id) + ksymbol_event->name_len; @@ -10450,7 +10450,7 @@ static void perf_event_bpf_output(struct { struct perf_bpf_event *bpf_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; int ret; if (!perf_event_bpf_match(event)) @@ -10540,7 +10540,7 @@ static void perf_callchain_deferred_outp { struct perf_callchain_deferred_event *deferred_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; u64 nr = deferred_event->trace->nr; u16 size, id_size, max_payload; int ret; @@ -10615,7 +10615,7 @@ static void perf_event_text_poke_output( { struct perf_text_poke_event *text_poke_event = data; struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; u16 size = sizeof(text_poke_event->event_id) + text_poke_event->tot + text_poke_event->pad; u64 padding = 0; int ret; @@ -10686,7 +10686,7 @@ EXPORT_SYMBOL_GPL(perf_event_itrace_star static void perf_log_itrace_start(struct perf_event *event) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct perf_aux_event { struct perf_event_header header; u32 pid; @@ -10721,7 +10721,7 @@ static void perf_log_itrace_start(struct void perf_report_aux_output_id(struct perf_event *event, u64 hw_id) { struct perf_output_handle handle; - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct perf_aux_event { struct perf_event_header header; u64 hw_id; @@ -11603,7 +11603,7 @@ void perf_tp_event(u16 event_type, u64 c struct pt_regs *regs, struct hlist_head *head, int rctx, struct task_struct *task) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct perf_event *event; /* @@ -11949,7 +11949,7 @@ void perf_event_free_bpf_prog(struct per #ifdef CONFIG_HAVE_HW_BREAKPOINT void perf_bp_event(struct perf_event *bp, void *data) { - struct perf_sample_data sample; + struct perf_sample_data sample __uninitialized; struct pt_regs *regs = data; /* @@ -12403,7 +12403,7 @@ static int perf_event_set_filter(struct static enum hrtimer_restart perf_swevent_hrtimer(struct hrtimer *hrtimer) { enum hrtimer_restart ret = HRTIMER_RESTART; - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct pt_regs *regs; struct perf_event *event; u64 period; ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() 2026-11-20 9:11 ` Peter Zijlstra @ 2026-11-20 9:12 ` Peter Zijlstra 2026-11-20 20:05 ` Usama Arif 2026-11-20 20:02 ` Usama Arif 1 sibling, 1 reply; 8+ messages in thread From: Peter Zijlstra @ 2026-11-20 9:12 UTC (permalink / raw) To: Usama Arif Cc: mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team On Fri, Nov 20, 2026 at 10:11:51AM +0100, Peter Zijlstra wrote: > On Thu, Nov 19, 2026 at 01:10:46PM -0800, Usama Arif wrote: > > > kernel/events/core.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/kernel/events/core.c b/kernel/events/core.c > > index 57ea7e3c01a025..fa10ad7cfd99af 100644 > > --- a/kernel/events/core.c > > +++ b/kernel/events/core.c > > @@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, > > u64 nr, struct pt_regs *regs, u64 addr) > > { > > struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); > > - struct perf_sample_data data; > > + struct perf_sample_data data __uninitialized; > > struct perf_event *event; > > struct hlist_head *head; > > > > I like this one, didn't know about the __uninitialized thing. But I > think we wants to be more consistent with this. > > A little git-grep seems to suggest we ought to do the same elsewhere. > Shall I fold the little something below in this patch? > And this should be a separate patch again I think. --- diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index 49b6b8fce566..db6c87552594 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -958,7 +958,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); static atomic64_t status_warned = ATOMIC64_INIT(0); u64 reserved, status, mask, new_bits, prev_bits; - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct hw_perf_event *hwc; struct perf_event *event; int handled = 0, idx; diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 96c96369480a..cf0eafd67430 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1483,7 +1483,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) struct cpu_perf_ibs *pcpu = this_cpu_ptr(perf_ibs->pcpu); struct perf_event *event = pcpu->event; struct hw_perf_event *hwc; - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct perf_raw_record raw; struct pt_regs regs; struct perf_ibs_data ibs_data; diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 4e344fb1ae3e..5f8bfc65b256 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2157,7 +2157,7 @@ void x86_pmu_update_perf_regs(struct perf_event *event, int x86_pmu_handle_irq(struct pt_regs *regs) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct cpu_hw_events *cpuc; struct perf_event *event; int idx, handled = 0; diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 0a34d674df59..5d84c18096a9 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3819,7 +3819,7 @@ static int x86_pmu_handle_guest_pebs(struct pt_regs *regs, static int handle_pmi_common(struct pt_regs *regs, u64 status) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); int bit; int handled = 0; diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index 8444670cee4a..73f6fd66856f 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1184,7 +1184,7 @@ int intel_pmu_drain_bts_buffer(void) struct bts_record *at, *base, *top; struct perf_output_handle handle; struct perf_event_header header; - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; unsigned long skip = 0; struct pt_regs regs; @@ -1265,7 +1265,7 @@ int intel_pmu_drain_bts_buffer(void) void intel_pmu_drain_pebs_buffer(void) { struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; WARN_ON_ONCE(cpuc->enabled); diff --git a/arch/x86/events/intel/knc.c b/arch/x86/events/intel/knc.c index e887adc108ac..aa38b0ac3c7b 100644 --- a/arch/x86/events/intel/knc.c +++ b/arch/x86/events/intel/knc.c @@ -213,7 +213,7 @@ static inline void knc_pmu_ack_status(u64 ack) static int knc_pmu_handle_irq(struct pt_regs *regs) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct cpu_hw_events *cpuc; int handled = 0; int bit, loops; diff --git a/arch/x86/events/intel/p4.c b/arch/x86/events/intel/p4.c index 5368dc31787c..4ac429e55935 100644 --- a/arch/x86/events/intel/p4.c +++ b/arch/x86/events/intel/p4.c @@ -1033,7 +1033,7 @@ static int p4_pmu_set_period(struct perf_event *event) static int p4_pmu_handle_irq(struct pt_regs *regs) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct cpu_hw_events *cpuc; struct perf_event *event; struct hw_perf_event *hwc; diff --git a/arch/x86/events/zhaoxin/core.c b/arch/x86/events/zhaoxin/core.c index e506f677db57..fc98a7fdd3bc 100644 --- a/arch/x86/events/zhaoxin/core.c +++ b/arch/x86/events/zhaoxin/core.c @@ -355,7 +355,7 @@ static void zhaoxin_pmu_enable_event(struct perf_event *event) */ static int zhaoxin_pmu_handle_irq(struct pt_regs *regs) { - struct perf_sample_data data; + struct perf_sample_data data __uninitialized; struct cpu_hw_events *cpuc; int handled = 0; u64 status; diff --git a/arch/x86/include/asm/stop_machine.h b/arch/x86/include/asm/stop_machine.h deleted file mode 100644 index e69de29bb2d1..000000000000 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() 2026-11-20 9:12 ` Peter Zijlstra @ 2026-11-20 20:05 ` Usama Arif 0 siblings, 0 replies; 8+ messages in thread From: Usama Arif @ 2026-11-20 20:05 UTC (permalink / raw) To: Peter Zijlstra Cc: mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team On 20/11/2026 01:12, Peter Zijlstra wrote: > On Fri, Nov 20, 2026 at 10:11:51AM +0100, Peter Zijlstra wrote: >> On Thu, Nov 19, 2026 at 01:10:46PM -0800, Usama Arif wrote: >> >>> kernel/events/core.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/kernel/events/core.c b/kernel/events/core.c >>> index 57ea7e3c01a025..fa10ad7cfd99af 100644 >>> --- a/kernel/events/core.c >>> +++ b/kernel/events/core.c >>> @@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, >>> u64 nr, struct pt_regs *regs, u64 addr) >>> { >>> struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); >>> - struct perf_sample_data data; >>> + struct perf_sample_data data __uninitialized; >>> struct perf_event *event; >>> struct hlist_head *head; >>> >> >> I like this one, didn't know about the __uninitialized thing. But I >> think we wants to be more consistent with this. >> >> A little git-grep seems to suggest we ought to do the same elsewhere. >> Shall I fold the little something below in this patch? >> > > And this should be a separate patch again I think. Agreed, but IBS is not safe yet. For PERF_SAMPLE_WEIGHT_STRUCT, perf_ibs_parse_ld_st_data() writes only var1_dw and var2_w but sets PERF_SAMPLE_WEIGHT_TYPE, so perf_output_sample() copies var3_w from whatever is on the stack. INIT_STACK_ALL_ZERO hides that today, and INIT_STACK_NONE kernels have leaked those 2 bytes since 6b2ae4952ef8. With __uninitialized in perf_ibs_handle_irq() every kernel would. We need the below: commit e662f8539ca00096db3ee4d570efe97767427fbe Author: Usama Arif <usama.arif@linux.dev> Date: Fri Nov 20 09:04:13 2026 -0800 perf/x86/amd/ibs: Don't leak stack contents through PERF_SAMPLE_WEIGHT_STRUCT For a load that missed the data cache, perf_ibs_parse_ld_st_data() reports the miss latency as the sample weight and marks PERF_SAMPLE_WEIGHT_TYPE as done, so perf_prepare_sample() leaves the field alone. With PERF_SAMPLE_WEIGHT_STRUCT it writes only var1_dw and var2_w, but perf_output_sample() copies all 8 bytes of the union into the ring buffer. var3_w therefore carries whatever was left in the on-stack struct perf_sample_data in perf_ibs_handle_irq(). CONFIG_INIT_STACK_ALL_ZERO hides this by clearing the struct on entry. Without it, every such sample hands 2 bytes of kernel stack to user space. "perf mem record" asks for PERF_SAMPLE_WEIGHT_STRUCT on x86, and since v6.14 swfilt lets an unprivileged user sample their own tasks with IBS. Write var3_w as well. Fixes: 6b2ae4952ef8 ("perf/x86/amd: Support PERF_SAMPLE_{WEIGHT|WEIGHT_STRUCT}") Cc: stable@vger.kernel.org Signed-off-by: Usama Arif <usama.arif@linux.dev> diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index 66e67e27224f18..9aa47751b37f6c 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1376,6 +1376,7 @@ static void perf_ibs_parse_ld_st_data(__u64 sample_type, if (sample_type & PERF_SAMPLE_WEIGHT_STRUCT) { data->weight.var1_dw = op_data3.dc_miss_lat; data->weight.var2_w = op_data.tag_to_ret_ctr; + data->weight.var3_w = 0; } else if (sample_type & PERF_SAMPLE_WEIGHT) { data->weight.full = op_data3.dc_miss_lat; } The other sites look fine to me, including arch-PEBS: the drain skips records whose first fragment has no basic group, so var3_w is always written there. > > --- > diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c > index 49b6b8fce566..db6c87552594 100644 > --- a/arch/x86/events/amd/core.c > +++ b/arch/x86/events/amd/core.c > @@ -958,7 +958,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) > struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); > static atomic64_t status_warned = ATOMIC64_INIT(0); > u64 reserved, status, mask, new_bits, prev_bits; > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct hw_perf_event *hwc; > struct perf_event *event; > int handled = 0, idx; > diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c > index 96c96369480a..cf0eafd67430 100644 > --- a/arch/x86/events/amd/ibs.c > +++ b/arch/x86/events/amd/ibs.c > @@ -1483,7 +1483,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_ibs, struct pt_regs *iregs) > struct cpu_perf_ibs *pcpu = this_cpu_ptr(perf_ibs->pcpu); > struct perf_event *event = pcpu->event; > struct hw_perf_event *hwc; > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct perf_raw_record raw; > struct pt_regs regs; > struct perf_ibs_data ibs_data; > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c > index 4e344fb1ae3e..5f8bfc65b256 100644 > --- a/arch/x86/events/core.c > +++ b/arch/x86/events/core.c > @@ -2157,7 +2157,7 @@ void x86_pmu_update_perf_regs(struct perf_event *event, > > int x86_pmu_handle_irq(struct pt_regs *regs) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct cpu_hw_events *cpuc; > struct perf_event *event; > int idx, handled = 0; > diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c > index 0a34d674df59..5d84c18096a9 100644 > --- a/arch/x86/events/intel/core.c > +++ b/arch/x86/events/intel/core.c > @@ -3819,7 +3819,7 @@ static int x86_pmu_handle_guest_pebs(struct pt_regs *regs, > > static int handle_pmi_common(struct pt_regs *regs, u64 status) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); > int bit; > int handled = 0; > diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c > index 8444670cee4a..73f6fd66856f 100644 > --- a/arch/x86/events/intel/ds.c > +++ b/arch/x86/events/intel/ds.c > @@ -1184,7 +1184,7 @@ int intel_pmu_drain_bts_buffer(void) > struct bts_record *at, *base, *top; > struct perf_output_handle handle; > struct perf_event_header header; > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > unsigned long skip = 0; > struct pt_regs regs; > > @@ -1265,7 +1265,7 @@ int intel_pmu_drain_bts_buffer(void) > void intel_pmu_drain_pebs_buffer(void) > { > struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > > WARN_ON_ONCE(cpuc->enabled); > > diff --git a/arch/x86/events/intel/knc.c b/arch/x86/events/intel/knc.c > index e887adc108ac..aa38b0ac3c7b 100644 > --- a/arch/x86/events/intel/knc.c > +++ b/arch/x86/events/intel/knc.c > @@ -213,7 +213,7 @@ static inline void knc_pmu_ack_status(u64 ack) > > static int knc_pmu_handle_irq(struct pt_regs *regs) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct cpu_hw_events *cpuc; > int handled = 0; > int bit, loops; > diff --git a/arch/x86/events/intel/p4.c b/arch/x86/events/intel/p4.c > index 5368dc31787c..4ac429e55935 100644 > --- a/arch/x86/events/intel/p4.c > +++ b/arch/x86/events/intel/p4.c > @@ -1033,7 +1033,7 @@ static int p4_pmu_set_period(struct perf_event *event) > > static int p4_pmu_handle_irq(struct pt_regs *regs) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct cpu_hw_events *cpuc; > struct perf_event *event; > struct hw_perf_event *hwc; > diff --git a/arch/x86/events/zhaoxin/core.c b/arch/x86/events/zhaoxin/core.c > index e506f677db57..fc98a7fdd3bc 100644 > --- a/arch/x86/events/zhaoxin/core.c > +++ b/arch/x86/events/zhaoxin/core.c > @@ -355,7 +355,7 @@ static void zhaoxin_pmu_enable_event(struct perf_event *event) > */ > static int zhaoxin_pmu_handle_irq(struct pt_regs *regs) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct cpu_hw_events *cpuc; > int handled = 0; > u64 status; > diff --git a/arch/x86/include/asm/stop_machine.h b/arch/x86/include/asm/stop_machine.h > deleted file mode 100644 > index e69de29bb2d1..000000000000 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() 2026-11-20 9:11 ` Peter Zijlstra 2026-11-20 9:12 ` Peter Zijlstra @ 2026-11-20 20:02 ` Usama Arif 1 sibling, 0 replies; 8+ messages in thread From: Usama Arif @ 2026-11-20 20:02 UTC (permalink / raw) To: Peter Zijlstra Cc: mingo, acme, namhyung, mark.rutland, alexander.shishkin, jolsa, irogers, adrian.hunter, james.clark, linux-perf-users, linux-kernel, bpf, rkeilty, atish.patra, dgreid, riel, shakeel.butt, kernel-team On 20/11/2026 01:11, Peter Zijlstra wrote: > On Thu, Nov 19, 2026 at 01:10:46PM -0800, Usama Arif wrote: > >> kernel/events/core.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/kernel/events/core.c b/kernel/events/core.c >> index 57ea7e3c01a025..fa10ad7cfd99af 100644 >> --- a/kernel/events/core.c >> +++ b/kernel/events/core.c >> @@ -11212,7 +11212,7 @@ static void do_perf_sw_event(enum perf_type_id type, u32 event_id, >> u64 nr, struct pt_regs *regs, u64 addr) >> { >> struct swevent_htable *swhash = this_cpu_ptr(&swevent_htable); >> - struct perf_sample_data data; >> + struct perf_sample_data data __uninitialized; >> struct perf_event *event; >> struct hlist_head *head; >> > > I like this one, didn't know about the __uninitialized thing. But I > think we wants to be more consistent with this. > > A little git-grep seems to suggest we ought to do the same elsewhere. > Shall I fold the little something below in this patch? > Thanks! I went through the below. perf_tp_event(), perf_bp_event() and perf_swevent_hrtimer() call perf_sample_data_init() for each event, like do_perf_sw_event() does. The side-band emitters only use the sample ID fields, which perf_event_header__init() writes and perf_event__output_id_sample() reads under the same sample_id_all check. So they are all fine to do. But they do need the fix in patch 2 to work. There is another fix that I found looking at your next reply. Would you like me to send all the fixes and you can then send the below patch (and the next one?) > > --- a/kernel/events/core.c > +++ b/kernel/events/core.c > @@ -9123,7 +9123,7 @@ perf_event_read_event(struct perf_event > struct task_struct *task) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct perf_read_event read_event = { > .pid = perf_event_pid(event, task), > .tid = perf_event_tid(event, task), > @@ -9390,7 +9390,7 @@ static void perf_event_task_output(struc > { > struct perf_task_event *task_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct task_struct *task = task_event->task; > int ret; > > @@ -9534,7 +9534,7 @@ static void perf_event_comm_output(struc > { > struct perf_comm_event *comm_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > > if (!perf_event_comm_match(event)) > @@ -9625,7 +9625,7 @@ static void perf_event_namespaces_output > { > struct perf_namespaces_event *namespaces_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > > if (!perf_event_namespaces_match(event)) > @@ -9748,7 +9748,7 @@ static void perf_event_cgroup_output(str > { > struct perf_cgroup_event *cgroup_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > u16 size = sizeof(cgroup_event->event_id) + cgroup_event->path_size; > > @@ -9859,7 +9859,7 @@ static void perf_event_mmap_output(struc > { > struct perf_mmap_event *mmap_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int size = sizeof(mmap_event->event_id) + mmap_event->file_size; > u32 type = PERF_RECORD_MMAP; > u16 misc = PERF_RECORD_MISC_USER; > @@ -10155,7 +10155,7 @@ void perf_event_aux_event(struct perf_ev > unsigned long size, u64 flags) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct perf_aux_event { > struct perf_event_header header; > u64 offset; > @@ -10188,7 +10188,7 @@ void perf_event_aux_event(struct perf_ev > void perf_log_lost_samples(struct perf_event *event, u64 lost) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > > struct { > @@ -10240,7 +10240,7 @@ static void perf_event_switch_output(str > { > struct perf_switch_event *se = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > u32 type; > u16 misc; > u16 size; > @@ -10307,7 +10307,7 @@ static void perf_event_switch(struct tas > static void perf_log_throttle(struct perf_event *event, int enable) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > > struct { > @@ -10363,7 +10363,7 @@ static void perf_event_ksymbol_output(st > { > struct perf_ksymbol_event *ksymbol_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > u16 size = sizeof(ksymbol_event->event_id) + ksymbol_event->name_len; > > @@ -10450,7 +10450,7 @@ static void perf_event_bpf_output(struct > { > struct perf_bpf_event *bpf_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > int ret; > > if (!perf_event_bpf_match(event)) > @@ -10540,7 +10540,7 @@ static void perf_callchain_deferred_outp > { > struct perf_callchain_deferred_event *deferred_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > u64 nr = deferred_event->trace->nr; > u16 size, id_size, max_payload; > int ret; > @@ -10615,7 +10615,7 @@ static void perf_event_text_poke_output( > { > struct perf_text_poke_event *text_poke_event = data; > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > u16 size = sizeof(text_poke_event->event_id) + text_poke_event->tot + text_poke_event->pad; > u64 padding = 0; > int ret; > @@ -10686,7 +10686,7 @@ EXPORT_SYMBOL_GPL(perf_event_itrace_star > static void perf_log_itrace_start(struct perf_event *event) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct perf_aux_event { > struct perf_event_header header; > u32 pid; > @@ -10721,7 +10721,7 @@ static void perf_log_itrace_start(struct > void perf_report_aux_output_id(struct perf_event *event, u64 hw_id) > { > struct perf_output_handle handle; > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct perf_aux_event { > struct perf_event_header header; > u64 hw_id; > @@ -11603,7 +11603,7 @@ void perf_tp_event(u16 event_type, u64 c > struct pt_regs *regs, struct hlist_head *head, int rctx, > struct task_struct *task) > { > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct perf_event *event; > > /* > @@ -11949,7 +11949,7 @@ void perf_event_free_bpf_prog(struct per > #ifdef CONFIG_HAVE_HW_BREAKPOINT > void perf_bp_event(struct perf_event *bp, void *data) > { > - struct perf_sample_data sample; > + struct perf_sample_data sample __uninitialized; > struct pt_regs *regs = data; > > /* > @@ -12403,7 +12403,7 @@ static int perf_event_set_filter(struct > static enum hrtimer_restart perf_swevent_hrtimer(struct hrtimer *hrtimer) > { > enum hrtimer_restart ret = HRTIMER_RESTART; > - struct perf_sample_data data; > + struct perf_sample_data data __uninitialized; > struct pt_regs *regs; > struct perf_event *event; > u64 period; ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-11-20 20:05 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2026-11-19 21:10 [PATCH 0/3] perf/core: Fix software event sample data, then stop zeroing it Usama Arif 2026-11-19 21:10 ` [PATCH 1/3] perf/core: Fix perf_sample_data reuse across software events Usama Arif 2026-11-19 21:10 ` [PATCH 2/3] perf/core: Don't let BPF programs read an uninitialized sample address Usama Arif 2026-11-19 21:10 ` [PATCH 3/3] perf/core: Don't zero perf_sample_data in do_perf_sw_event() Usama Arif 2026-11-20 9:11 ` Peter Zijlstra 2026-11-20 9:12 ` Peter Zijlstra 2026-11-20 20:05 ` Usama Arif 2026-11-20 20:02 ` Usama Arif
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®