From: Mike Bazov via B4 Relay <devnull+mike.bazov.gmail.com@kernel.org>
To: Song Liu <song@kernel.org>, Jiri Olsa <jolsa@kernel.org>,
Ihor Solodrai <ihor.solodrai@linux.dev>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Yonghong Song <yonghong.song@linux.dev>,
Emil Tsalapatis <emil@etsalapatis.com>,
KP Singh <kpsingh@kernel.org>,
Matt Bobrowski <matt@bobrowski.net>,
Steven Rostedt <rostedt@goodmis.org>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-trace-kernel@vger.kernel.org,
Mike Bazov <mike.bazov@gmail.com>
Subject: [PATCH bpf-next v3 0/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers
Date: Fri, 20 Nov 2026 12:59:10 +0200 [thread overview]
Message-ID: <20261120-bpf-raw-tp-regs-preempt-v3-0-ec109009a5d2@gmail.com> (raw)
bpf_perf_event_output(), bpf_get_stackid() and bpf_get_stack() of raw
tracepoint and tracing programs share three per-CPU pt_regs slots,
picked by a nest counter. These programs can be preempted, and tasks
then leave the slots out of order, so a sample or stack trace of one
task gets the registers of another. Patch 3 fills in a pt_regs on the
stack of each helper instead and removes the slots.
The slots also limited how deep these helpers nest. Two things need a
limit of their own without them, and both can already be hit today
without raw tracepoint programs:
- bpf_perf_event_output() WARNs when entered a fourth time on a CPU.
- Build ID collection in bpf_get_stack()/bpf_get_stackid() and
bpf_find_vma() take mmap_read_trylock(), which fires the mmap_lock
tracepoints. Programs on them that call these helpers nest into each
other as deep as there are programs. Patch 2 limits it to three
levels.
Results on bpf-next 0fb18ce8a2e9, x86-64, debug config, CONFIG_PREEMPT=y,
preempt=full. "base" is that tree, "v2" adds only patch 3, "v3" all
three.
The slot race, 5 runs of 10s each. Two raw tracepoint programs on
sys_enter call the helper, two threads make syscalls on CPU 0 and a
SCHED_FIFO thread preempts them every 20us. For bpf_perf_event_output()
the reproducer reads the return address of each program's helper call
from its JIT image and checks the IP of every sample; bpf_get_stack()
and bpf_get_stackid() get an empty kernel stack instead, because the
unwinder looks for the other task's sp on the current stack:
base v3
bpf_perf_event_output, wrong IP 7-18 0 per program
bpf_get_stack, empty stacks 362-535 0
bpf_get_stackid, empty stacks 168-364 0
The bpf_get_stackid() case with bpftrace, whose kstack calls it; @a[]
and @b[] count the empty stacks:
bpftrace -e '
rawtracepoint:sys_enter /comm == "dd"/ { @a[kstack(1)] = count(); }
rawtracepoint:sys_enter /comm == "dd"/ { @b[kstack(1)] = count(); }' &
taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
taskset -c 0 dd if=/dev/zero of=/dev/null bs=1 &
taskset -c 0 chrt -f 1 perl -e 'select(undef, undef, undef, 2e-5) while 1' &
Nesting. For the WARN, a raw tracepoint or kprobe program calls
bpf_perf_event_output(), and three fentry programs on
perf_event_output() call it too. For mmap_read_trylock(), six programs
on mmap_lock_acquire_returned call bpf_get_stack() with build IDs, or
bpf_find_vma(), while the process does 1000 mmap()/munmap():
base v2 v3
WARN, started from raw tracepoint no yes no
WARN, started from kprobe yes - no
bpf_get_stack, build IDs: depth 4 6 4
calls 1.5M 5.9M 1.5M
bpf_find_vma: depth 6 6 4
calls 5.9M 5.9M 1.5M
Depth counts the programs on the stack; at 4, the last one gets -EBUSY
or the fallback to IPs. The kprobe case wasn't run on v2, which doesn't
change that path.
---
Changes in v3:
- New patch 1: don't WARN when bpf_perf_event_output() nests too deep
(reported by the BPF CI bot).
- New patch 2: limit nesting of mmap_read_trylock() in BPF helpers
(per Alexei's suggestion).
- Patch 3: correct the commit message, bpf_perf_event_output() keeps
its own nesting limit.
- Link to v2: https://lore.kernel.org/r/20261119-bpf-raw-tp-regs-preempt-v2-1-e1e1b90857d6@gmail.com
Changes in v2:
- Use a pt_regs on the stack of each helper and remove the per-CPU
slots, instead of disabling preemption while a slot is in use
(per Andrii's suggestion).
- Link to v1: https://lore.kernel.org/r/20261116-bpf-raw-tp-regs-preempt-v1-1-d01b6dd8573c@gmail.com
---
Mike Bazov (3):
bpf: Don't WARN when bpf_perf_event_output() nests too deep
bpf: Limit nesting of mmap_read_trylock() in BPF helpers
bpf: Use an on-stack pt_regs in the raw tracepoint helpers
kernel/bpf/mmap_unlock_work.h | 16 +++++++--
kernel/trace/bpf_trace.c | 80 +++++++------------------------------------
2 files changed, 26 insertions(+), 70 deletions(-)
---
base-commit: 8835113107b2cf75df33a71e265cdeec5d5d13e0
change-id: 20261116-bpf-raw-tp-regs-preempt-1f7019989ab9
Best regards,
--
Mike Bazov <mike.bazov@gmail.com>
next reply other threads:[~2026-11-20 10:59 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 10:59 Mike Bazov via B4 Relay [this message]
2026-11-20 10:59 ` [PATCH bpf-next v3 1/3] bpf: Don't WARN when bpf_perf_event_output() nests too deep Mike Bazov via B4 Relay
2026-11-20 10:59 ` [PATCH bpf-next v3 2/3] bpf: Limit nesting of mmap_read_trylock() in BPF helpers Mike Bazov via B4 Relay
2026-11-20 10:59 ` [PATCH bpf-next v3 3/3] bpf: Use an on-stack pt_regs in the raw tracepoint helpers Mike Bazov via B4 Relay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120-bpf-raw-tp-regs-preempt-v3-0-ec109009a5d2@gmail.com \
--to=devnull+mike.bazov.gmail.com@kernel.org \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=eddyz87@gmail.com \
--cc=emil@etsalapatis.com \
--cc=ihor.solodrai@linux.dev \
--cc=jolsa@kernel.org \
--cc=kpsingh@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=martin.lau@linux.dev \
--cc=mathieu.desnoyers@efficios.com \
--cc=matt@bobrowski.net \
--cc=memxor@gmail.com \
--cc=mhiramat@kernel.org \
--cc=mike.bazov@gmail.com \
--cc=rostedt@goodmis.org \
--cc=song@kernel.org \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®