From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
James Clark <james.clark@linaro.org>
Cc: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <martin.lau@linux.dev>,
Eduard Zingerman <eddyz87@gmail.com>, Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
Leo Yan <leo.yan@linux.dev>, Dmitry Vyukov <dvyukov@google.com>,
"Steinar H . Gunderson" <sesse@google.com>,
Stephen Brennan <stephen.s.brennan@oracle.com>,
Thomas Falcon <thomas.falcon@intel.com>,
linux-perf-users@vger.kernel.org, bpf@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: [PATCH v3 01/19] perf/core: Protect per-CPU callchain buffer across bpf_overflow_handler()
Date: Sun, 15 Nov 2026 23:43:56 -0800 [thread overview]
Message-ID: <20261116074414.3174737-2-irogers@google.com> (raw)
In-Reply-To: <20261116074414.3174737-1-irogers@google.com>
In bpf_overflow_handler(), perf_prepare_sample() runs before bpf_prog_run()
and saves data->callchain pointing into the current CPU's callchain buffer
for the active interrupt_context_level(), after get_perf_callchain() has
already released its callchain_recursion slot.
If the attached BPF_PROG_TYPE_PERF_EVENT program invokes a helper at the
same interrupt_context_level() that calls get_callchain_entry() (such as
bpf_perf_event_output() on a PERF_COUNT_SW_BPF_OUTPUT event configured with
PERF_SAMPLE_CALLCHAIN and a larger stack, or bpf_get_task_stack()), it
re-acquires the same per-CPU callchain buffer and overwrites
data->callchain->nr in place. When bpf_prog_run() returns non-zero and
__perf_event_output() emits the outer event's sample, perf_output_begin()
reserves only the smaller dyn_size computed earlier while
perf_output_sample() copies the overwritten larger data->callchain->nr
entries, overflowing the ring-buffer reservation.
Hold the callchain recursion slot via get_callchain_entry() across
bpf_prog_run() when PERF_SAMPLE_CALLCHAIN has been prepared so nested
callers at the same context level cannot overwrite the active per-CPU
callchain buffer.
Fixes: 0eed28220598 ("perf/core: Call perf_prepare_sample() before running BPF")
Signed-off-by: Ian Rogers <irogers@google.com>
---
kernel/events/core.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 614f9f0e6500..fd9f654de269 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -10818,8 +10818,15 @@ static int bpf_overflow_handler(struct perf_event *event,
rcu_read_lock();
prog = READ_ONCE(event->prog);
if (prog) {
+ struct perf_callchain_entry *callchain = NULL;
+ int callchain_rctx;
+
perf_prepare_sample(data, event, regs);
+ if (data->sample_flags & PERF_SAMPLE_CALLCHAIN)
+ callchain = get_callchain_entry(&callchain_rctx);
ret = bpf_prog_run(prog, &ctx);
+ if (callchain)
+ put_callchain_entry(callchain_rctx);
}
rcu_read_unlock();
out:
--
2.56.1.703.gf848ed807c-goog
next prev parent reply other threads:[~2026-11-16 7:45 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 7:43 [PATCH v3 00/19] perf/core, perf/tools: Add PERF_SAMPLE_BUILD_ID_OFFSET support Ian Rogers
2026-11-16 7:43 ` Ian Rogers [this message]
2026-11-16 7:43 ` [PATCH v3 02/19] perf/uapi: Factor perf_build_id out into its own top-level struct Ian Rogers
2026-11-16 7:43 ` [PATCH v3 03/19] perf/uapi: Add BUILD_ID_OFFSET sample types to UAPI Ian Rogers
2026-11-16 7:43 ` [PATCH v3 04/19] bpf: Export stack_map_get_build_id_offset() for perf events Ian Rogers
2026-11-16 7:44 ` [PATCH v3 05/19] perf/core: Implement BUILD_ID_OFFSET and CALLCHAIN_BUILD_ID_OFFSET sample types Ian Rogers
2026-11-16 7:44 ` [PATCH v3 06/19] perf/core: Support deferred unwinding for BUILD_ID_OFFSET " Ian Rogers
2026-11-16 7:44 ` [PATCH v3 07/19] perf inject: Set ordered_events and ordering_requires_timestamps for --aslr Ian Rogers
2026-11-16 7:44 ` [PATCH v3 08/19] perf session: Avoid double byte-swapping re-parsed deferred samples Ian Rogers
2026-11-16 7:44 ` [PATCH v3 09/19] perf inject: Fix PERF_SAMPLE_READ layout and bounds checks in aslr_tool__process_sample() Ian Rogers
2026-11-16 7:44 ` [PATCH v3 10/19] tools headers UAPI: Sync linux/perf_event.h for struct perf_build_id Ian Rogers
2026-11-16 7:44 ` [PATCH v3 11/19] tools headers UAPI: Sync linux/perf_event.h for BUILD_ID_OFFSET Ian Rogers
2026-11-16 7:44 ` [PATCH v3 12/19] perf: Refactor thread map and symbol APIs to take perf_sample Ian Rogers
2026-11-16 7:44 ` [PATCH v3 13/19] perf maps: Add dsos__find_by_build_id() and maps__find_by_build_id() Ian Rogers
2026-11-16 7:44 ` [PATCH v3 14/19] perf evsel: Parse, synthesize, and dump BUILD_ID_OFFSET sample types Ian Rogers
2026-11-16 7:44 ` [PATCH v3 15/19] perf symbol: Resolve symbols and callchains using build ID and offset Ian Rogers
2026-11-16 7:44 ` [PATCH v3 16/19] perf inject: Add --sample-buildids to convert samples to build ID + offset Ian Rogers
2026-11-16 7:44 ` [PATCH v3 17/19] perf record: Add --buildid-offset option Ian Rogers
2026-11-16 7:44 ` [PATCH v3 18/19] perf test: Add BUILD_ID_OFFSET C unit tests Ian Rogers
2026-11-16 7:44 ` [PATCH v3 19/19] perf test: Add inject --sample-buildids and record --buildid-offset shell tests Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261116074414.3174737-2-irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=dvyukov@google.com \
--cc=eddyz87@gmail.com \
--cc=ihor.solodrai@linux.dev \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=leo.yan@linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=martin.lau@linux.dev \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=sesse@google.com \
--cc=song@kernel.org \
--cc=stephen.s.brennan@oracle.com \
--cc=thomas.falcon@intel.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®