* [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator
@ 2026-11-20 20:33 Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 1/8] bpf: user_ringbuf: Handle position wrap Tamir Duberstein
` (7 more replies)
0 siblings, 8 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein, Sashiko
The ring buffer API exposes records only through callbacks. Callers that
need to stop at an operation-specific condition have no non-error way to
end consumption early.
Andrea Righi proposed using positive callback returns to request early
termination. Andrii Nakryiko noted that positive values are already
ignored and changing that behavior could break users, and suggested a
new API:
https://lore.kernel.org/bpf/20240310154726.734289-1-andrea.righi@canonical.com/
Discussion of overwrite-mode consumption identified an iterator as a
better zero-copy interface than callbacks:
https://lore.kernel.org/bpf/CAEf4Bzaq5drHWChXoRBnrmkb6reAsSVj8r=uByFSup31FMA7hw@mail.gmail.com/
Add a caller-owned iterator for consuming one ring buffer map directly.
It returns one borrowed record at a time, allowing ordinary loop
control. Each next call ends the previous borrow and lets the caller
choose whether to publish consumed records or defer publication for
batching. Existing callback-based consumption uses the same internal
iterator core while preserving callback return semantics and per-record
consumer position updates. Callback-based consumption and public
iteration of overwrite maps remain unsupported.
This v3 targets bpf-next/for-next. The following patches from v2 are
omitted after rebasing:
- 1/8, "libbpf: ringbuf: Honor zero consume bounds", was applied as
40edacd47993.
- 2/8, "libbpf: ringbuf: Prevent NULL callback crash", was applied as
c0628a0659b6.
- 3/8, "libbpf: ringbuf: Reject overwrite callback use", was applied as
f756eedb24d4.
- 4/8, "libbpf: ringbuf: Handle position counter wrap", is superseded by
Israel Téllez García's independently developed fix fdd4fad0bbbd.
- 5/8, "bpf: ringbuf: Handle pending position wrap", is superseded by
Israel's independently developed fix 6ff5b56a50c5.
Patches 1-3 resubmit v2 patches 6-8: "bpf: user_ringbuf: Handle position
wrap", "libbpf: ringbuf: Use compiler atomics", and "libbpf: ringbuf:
Prevent missed wakeups". They fix native-width wrap in the kernel's
user-ring consumer and a missed-notification race in libbpf. The atomic
conversion provides the portable full barrier needed by the wakeup fix.
Patches 4-5 add the caller-owned iterator and simplify its traversal.
The iterator preserves the wakeup handshake when destroyed before
exhaustion. Patches 6-8 contain the selftest refactoring, wakeup
regression coverage, callback-semantics checks, and iterator tests.
Selftests follow at the end, as requested during review.
Andrew Werner found the counter-wrap and missed-wakeup failures while
implementing Aya's ring buffer reader. Aya's original implementation
contains the equality reasoning and edge-triggered regression test:
https://github.com/aya-rs/aya/commit/e2cf734490bc188bcedb1eac92d23d81123e42cd
Aya later fixed the consumer ordering with an explicit fence:
https://github.com/aya-rs/aya/commit/7277a57ea8cdb74918d3096a4b22b6d814481973
This reroll continues the ring buffer fixes series and brings the
callback-free API into the same submission. The original callback API
proposal is a related submission:
https://lore.kernel.org/bpf/20260611-bpf-ringbuf-callback-v1-1-c32b6cb6ce40@kernel.org/
Review requesting that callback validation and the new API be discussed
together:
https://lore.kernel.org/bpf/CAADnVQLuNSE5AM6Cqzs4yDFrhTyD=j-y5dfya3dLwk3si9v26w@mail.gmail.com/
Earlier versions were developed with Codex (GPT-5.5). Codex (GPT-6)
assisted with combining the series, addressing review, and revising
the iterator, tests, and changelogs.
The libbpf build and BPF selftests have not been run.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
Changes in v3:
- Combine the remaining fixes with a new caller-owned iterator and its
tests, targeting bpf-next/for-next.
- Drop v2 patches 1-3, which were applied, and v2 patches 4-5, whose
issues were independently fixed, as listed above.
- Preserve -E2BIG for oversized user-ring samples when validating the
advertised window.
- Move the fixes' selftests to separate commits at the end.
- Simplify the wakeup fix's bound and error exits while retaining its
fenced busy retry.
- Keep the compiler atomic stores on single lines and follow the BPF
multiline comment style.
- Link to v2:
https://lore.kernel.org/bpf/20260618-bpf-ringbuf-fixes-v2-0-33fde039ddf3@kernel.org/
Changes in v2:
- Fix additional native 32-bit wrap paths found by Sashiko review.
- Explain modular position comparisons and leave mixed-bitness ABI work
separate.
- Preserve the wakeup handshake across bounded and callback returns.
- Follow BPF comment style and simplify wakeup test cleanup.
- Link to v1: https://patch.msgid.link/20260613-bpf-ringbuf-fixes-v1-0-e623481cb724@kernel.org
---
Tamir Duberstein (8):
bpf: user_ringbuf: Handle position wrap
libbpf: ringbuf: Use compiler atomics
libbpf: ringbuf: Prevent missed wakeups
libbpf: ringbuf: Add a caller-owned iterator
libbpf: ringbuf: Simplify iterator implementation
selftests/bpf: Refactor lifetimes and test wakeups
selftests/bpf: Verify ringbuf callback semantics
selftests/bpf: Test caller-owned ring iterators
kernel/bpf/ringbuf.c | 11 +-
tools/lib/bpf/libbpf.h | 90 +++
tools/lib/bpf/libbpf.map | 3 +
tools/lib/bpf/ringbuf.c | 187 ++++--
tools/testing/selftests/bpf/prog_tests/ringbuf.c | 660 ++++++++++++++++-----
tools/testing/selftests/bpf/progs/test_ringbuf_n.c | 9 +-
6 files changed, 757 insertions(+), 203 deletions(-)
---
base-commit: 82cc2b4383e3edfa1026c8612fb4e90a9329ba11
base-tree: git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
change-id: 20260613-bpf-ringbuf-fixes-e9a8b3c6125b
Best regards,
--
Tamir Duberstein <tamird@kernel.org>
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 1/8] bpf: user_ringbuf: Handle position wrap
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics Tamir Duberstein
` (6 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein, Sashiko
User ring buffer positions are unsigned long counters, but
__bpf_user_ringbuf_peek() widens them to u64 before comparing and
subtracting them. On 32-bit systems, producer_pos wrapping below
consumer_pos therefore appears to move backwards and permanently stalls
the ring.
Keep the positions word-sized and derive the available data with
word-sized subtraction so the arithmetic wraps with the counters. Treat
a zero span as empty and reuse the distance for the advertised-window
check. Validate sample sizes before rejecting a window larger than the
ring, preserving the documented -E2BIG result for oversized samples.
Fixes: 205715673844 ("bpf: Add bpf_user_ringbuf_drain() helper")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/bpf/20260614020552.022A11F000E9@smtp.kernel.org/
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
kernel/bpf/ringbuf.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
index 447863ece26a..5494c0977153 100644
--- a/kernel/bpf/ringbuf.c
+++ b/kernel/bpf/ringbuf.c
@@ -747,9 +747,9 @@ const struct bpf_func_proto bpf_ringbuf_discard_dynptr_proto = {
static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *size)
{
+ unsigned long avail, cons_pos, prod_pos;
int err;
u32 hdr_len, sample_len, total_len, flags, *hdr;
- u64 cons_pos, prod_pos;
/* Synchronizes with smp_store_release() in user-space producer. */
prod_pos = smp_load_acquire(&rb->producer_pos);
@@ -758,7 +758,8 @@ static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *s
/* Synchronizes with smp_store_release() in __bpf_user_ringbuf_sample_release() */
cons_pos = smp_load_acquire(&rb->consumer_pos);
- if (cons_pos >= prod_pos)
+ avail = prod_pos - cons_pos;
+ if (!avail)
return -ENODATA;
hdr = (u32 *)((uintptr_t)rb->data + (uintptr_t)(cons_pos & rb->mask));
@@ -769,7 +770,7 @@ static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *s
total_len = round_up(sample_len + BPF_RINGBUF_HDR_SZ, 8);
/* The sample must fit within the region advertised by the producer position. */
- if (total_len > prod_pos - cons_pos)
+ if (total_len > avail)
return -EINVAL;
/* The sample must fit within the data region of the ring buffer. */
@@ -781,6 +782,10 @@ static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *s
if (err)
return -E2BIG;
+ /* The advertised window must also fit within the ring buffer. */
+ if (avail > ringbuf_total_data_sz(rb))
+ return -EINVAL;
+
if (flags & BPF_RINGBUF_DISCARD_BIT) {
/* If the discard bit is set, the sample should be skipped.
*
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 1/8] bpf: user_ringbuf: Handle position wrap Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 20:33 ` [PATCH bpf-next v3 3/8] libbpf: ringbuf: Prevent missed wakeups Tamir Duberstein
` (5 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
Consumer-side ring buffer code uses architecture-specific smp_* helpers
for shared memory accesses.
Use compiler atomics for shared-memory accesses so the wakeup protocol
can use portable compiler fences with the same atomic interface. Acquire
and release ordering is unchanged.
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/lib/bpf/ringbuf.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index 78fcb3df243f..db67eb98f3e0 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -270,13 +270,13 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
if (n == 0)
return 0;
- cons_pos = smp_load_acquire(r->consumer_pos);
+ cons_pos = __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
do {
got_new_data = false;
- prod_pos = smp_load_acquire(r->producer_pos);
+ prod_pos = __atomic_load_n(r->producer_pos, __ATOMIC_ACQUIRE);
while (prod_pos - cons_pos > 0) {
len_ptr = r->data + (cons_pos & r->mask);
- len = smp_load_acquire(len_ptr);
+ len = __atomic_load_n(len_ptr, __ATOMIC_ACQUIRE);
/* sample not committed yet, bail out for now */
if (len & BPF_RINGBUF_BUSY_BIT)
@@ -290,14 +290,13 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
err = r->sample_cb(r->ctx, sample, len);
if (err < 0) {
/* update consumer pos and bail out */
- smp_store_release(r->consumer_pos,
- cons_pos);
+ __atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
return err;
}
cnt++;
}
- smp_store_release(r->consumer_pos, cons_pos);
+ __atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
if (cnt >= n)
goto done;
@@ -412,8 +411,8 @@ struct ring *ring_buffer__ring(struct ring_buffer *rb, unsigned int idx)
unsigned long ring__consumer_pos(const struct ring *r)
{
- /* Synchronizes with smp_store_release() in ringbuf_process_ring(). */
- return smp_load_acquire(r->consumer_pos);
+ /* Synchronizes with the release store in ringbuf_process_ring(). */
+ return __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
}
unsigned long ring__producer_pos(const struct ring *r)
@@ -421,7 +420,7 @@ unsigned long ring__producer_pos(const struct ring *r)
/* Synchronizes with smp_store_release() in __bpf_ringbuf_reserve() in
* the kernel.
*/
- return smp_load_acquire(r->producer_pos);
+ return __atomic_load_n(r->producer_pos, __ATOMIC_ACQUIRE);
}
size_t ring__avail_data_size(const struct ring *r)
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 3/8] libbpf: ringbuf: Prevent missed wakeups
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 1/8] bpf: user_ringbuf: Handle position wrap Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator Tamir Duberstein
` (4 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein, Sashiko
After consuming the last visible record, ringbuf_process_ring()
publishes the consumer position and checks the producer position. These
operations lack a full StoreLoad barrier. A producer can therefore
commit a new record but read the old consumer position while the
consumer reads the old producer position. The producer sends no
notification and the consumer waits despite a queued record.
Insert a full barrier between publishing a consumer position and the
next producer position load. When a record bound or callback ends the
current invocation first, execute the barrier before returning so the
load in a later invocation completes the same handshake.
After publication, retry a busy record once behind the barrier before
returning. Otherwise a producer could commit it and skip a wakeup based
on the old consumer position.
Document that bounded consumers and callbacks that terminate consumption
must drain before waiting again.
Fixes: bf99c936f947 ("libbpf: Add BPF ring buffer support")
Reported-by: Andrew Werner <awerner32@gmail.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/bpf/20260614015716.945AF1F000E9@smtp.kernel.org/
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/lib/bpf/libbpf.h | 23 +++++++++++++++++++++++
tools/lib/bpf/ringbuf.c | 18 ++++++++++++++----
2 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 49f1120f4021..856650d58578 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -1542,6 +1542,11 @@ struct ring;
struct user_ring_buffer;
/* Callback-based consumption is unsupported for BPF_F_RB_OVERWRITE maps. */
+/*
+ * A negative return stops consumption; non-negative values continue. Stopping
+ * can leave records queued without a new readiness notification. Before
+ * waiting for readiness again, consume until no records remain.
+ */
typedef int (*ring_buffer_sample_fn)(void *ctx, void *data, size_t size);
struct ring_buffer_opts {
@@ -1558,6 +1563,20 @@ LIBBPF_API int ring_buffer__add(struct ring_buffer *rb, int map_fd,
ring_buffer_sample_fn sample_cb, void *ctx);
LIBBPF_API int ring_buffer__poll(struct ring_buffer *rb, int timeout_ms);
LIBBPF_API int ring_buffer__consume(struct ring_buffer *rb);
+
+/**
+ * @brief **ring_buffer__consume_n()** consumes up to a requested number of
+ * records from a ring buffer manager without event polling.
+ *
+ * @param rb A ring buffer manager object.
+ * @param n Maximum number of records to consume.
+ * @return The number of records consumed, or a negative error code on failure.
+ *
+ * Reaching the requested bound does not establish that every ring is empty.
+ * Records can remain queued without a new readiness notification. Before
+ * calling ring_buffer__poll() or waiting on ring_buffer__epoll_fd(), call
+ * ring_buffer__consume() until it returns 0.
+ */
LIBBPF_API int ring_buffer__consume_n(struct ring_buffer *rb, size_t n);
LIBBPF_API int ring_buffer__epoll_fd(const struct ring_buffer *rb);
@@ -1640,6 +1659,10 @@ LIBBPF_API int ring__consume(struct ring *r);
* @param r A ringbuffer object.
* @param n Maximum number of records to consume.
* @return The number of records consumed, or a negative error code on failure.
+ *
+ * Reaching the requested bound does not establish that the ring is empty.
+ * Records can remain queued without a new readiness notification. Before
+ * waiting on ring__map_fd(), call ring__consume() until it returns 0.
*/
LIBBPF_API int ring__consume_n(struct ring *r, size_t n);
diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index db67eb98f3e0..7bf92311da68 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -271,16 +271,16 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
return 0;
cons_pos = __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
- do {
+ for (;;) {
got_new_data = false;
prod_pos = __atomic_load_n(r->producer_pos, __ATOMIC_ACQUIRE);
while (prod_pos - cons_pos > 0) {
len_ptr = r->data + (cons_pos & r->mask);
len = __atomic_load_n(len_ptr, __ATOMIC_ACQUIRE);
- /* sample not committed yet, bail out for now */
+ /* Retry a busy record once after publishing prior records. */
if (len & BPF_RINGBUF_BUSY_BIT)
- goto done;
+ break;
got_new_data = true;
cons_pos += roundup_len(len);
@@ -291,6 +291,7 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
if (err < 0) {
/* update consumer pos and bail out */
__atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
return err;
}
cnt++;
@@ -301,8 +302,17 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
if (cnt >= n)
goto done;
}
- } while (got_new_data);
+ if (!got_new_data)
+ return cnt;
+
+ /*
+ * Order the published consumer position before the next
+ * producer-position load, whether below or in a later invocation.
+ */
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
+ }
done:
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
return cnt;
}
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
` (2 preceding siblings ...)
2026-11-20 20:33 ` [PATCH bpf-next v3 3/8] libbpf: ringbuf: Prevent missed wakeups Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 20:33 ` [PATCH bpf-next v3 5/8] libbpf: ringbuf: Simplify iterator implementation Tamir Duberstein
` (3 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
Ring buffer records are exposed only through callbacks. Callers that
need to stop at an operation-specific condition have no non-error way to
end consumption early.
Add a caller-owned iterator that returns borrowed records one at a time.
Each next call ends the borrow and can publish consumed records before
inspecting another one, allowing callers to batch consumer updates.
Implement callback consumption on the same internal iterator core so
both paths share the record traversal and publication protocol. Allow
iterator-only rings to omit the setup callback.
Preserve callback validation, so callback-based consumption of an
iterator-only ring still returns -EINVAL. Keep the consumer-publication
barrier on early destruction, including bounded and callback-error
returns.
Keep overwrite maps out of the public iterator; handling a stale
consumer position requires a separate resynchronization protocol.
Require iterator users to observe exhaustion before waiting for another
readiness notification.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/lib/bpf/libbpf.h | 67 ++++++++++++++++++
tools/lib/bpf/libbpf.map | 3 +
tools/lib/bpf/ringbuf.c | 172 +++++++++++++++++++++++++++++++++++++----------
3 files changed, 207 insertions(+), 35 deletions(-)
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 856650d58578..cc9e141ffd0a 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -1541,6 +1541,11 @@ struct ring_buffer;
struct ring;
struct user_ring_buffer;
+/* Opaque caller-owned iterator state. */
+struct ring_buffer_iter {
+ __u64 __opaque[8];
+};
+
/* Callback-based consumption is unsupported for BPF_F_RB_OVERWRITE maps. */
/*
* A negative return stops consumption; non-negative values continue. Stopping
@@ -1594,6 +1599,68 @@ LIBBPF_API int ring_buffer__epoll_fd(const struct ring_buffer *rb);
LIBBPF_API struct ring *ring_buffer__ring(struct ring_buffer *rb,
unsigned int idx);
+/**
+ * @brief **ring_buffer_iter_new()** initializes a caller-owned iterator for a
+ * ring buffer map.
+ *
+ * @param it Pointer to the iterator to initialize.
+ * @param r Ring buffer map to iterate. Its ring buffer manager must remain
+ * valid until the iterator is destroyed.
+ * @return 0 on success; a negative error code on failure.
+ *
+ * Every successful call must be paired with ring_buffer_iter_destroy() before
+ * the iterator storage is reused or the ring buffer manager is freed. An
+ * active iterator must not be copied. On failure, the iterator is unchanged.
+ *
+ * A ring consumed only through iterators may be registered with a NULL sample
+ * callback. A sample callback is required to consume records through
+ * callback-based APIs.
+ *
+ * Only one iterator or callback-based consumer may operate on a ring buffer
+ * map at a time. BPF_F_RB_OVERWRITE ring buffer maps are not supported.
+ */
+LIBBPF_API int ring_buffer_iter_new(struct ring_buffer_iter *it,
+ struct ring *r);
+
+/**
+ * @brief **ring_buffer_iter_next()** returns the next available record.
+ *
+ * @param it Pointer to an initialized ring buffer iterator.
+ * @param size Optional pointer that receives the record size.
+ * @param sync Publish consumed records before advancing.
+ * @return A read-only pointer to the record data, or NULL if the iterator is
+ * exhausted.
+ *
+ * The returned record's data becomes invalid on the next call to
+ * ring_buffer_iter_next() or ring_buffer_iter_destroy(), regardless of sync.
+ * When sync is true, this call publishes records returned by earlier calls
+ * before inspecting another one. Otherwise publication may be deferred until
+ * the iterator refreshes the producer position, observes a busy record, or
+ * is destroyed. Once this function returns NULL, it continues returning NULL.
+ * Create a new iterator to check the ring again.
+ *
+ * Only a NULL return establishes a notification-safe observation: after
+ * publishing the consumer position, the iterator observed no committed record.
+ * Before waiting for ring buffer readiness, drive an iterator to NULL. If an
+ * iterator was destroyed early, create another iterator and drive it to NULL
+ * before waiting.
+ */
+LIBBPF_API const void *ring_buffer_iter_next(struct ring_buffer_iter *it,
+ size_t *size, bool sync);
+
+/**
+ * @brief **ring_buffer_iter_destroy()** destroys a ring buffer iterator.
+ *
+ * @param it Pointer to a zero-initialized iterator or one initialized by
+ * ring_buffer_iter_new().
+ *
+ * Any record returned by the previous ring_buffer_iter_next() call is consumed.
+ * Destroying an iterator before ring_buffer_iter_next() returns NULL publishes
+ * the consumed position, but records can remain queued without a new readiness
+ * notification.
+ */
+LIBBPF_API void ring_buffer_iter_destroy(struct ring_buffer_iter *it);
+
/**
* @brief **ring__consumer_pos()** returns the current consumer position in the
* given ringbuffer.
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index a45666f546fa..5ab5a46fedb1 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -479,4 +479,7 @@ LIBBPF_1.9.0 {
bpf_program__load;
bpf_program__load_strategy;
bpf_program__set_load_strategy;
+ ring_buffer_iter_destroy;
+ ring_buffer_iter_new;
+ ring_buffer_iter_next;
} LIBBPF_1.8.0;
diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index 7bf92311da68..2e281e2fa573 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -255,64 +255,166 @@ static int ringbuf_validate_callbacks(const struct ring_buffer *rb)
return 0;
}
-static int64_t ringbuf_process_ring(struct ring *r, size_t n)
+/*
+ * The private state aliases the public iterator's opaque __u64 storage.
+ * Allow that access without violating strict aliasing rules.
+ */
+struct ring_buffer_iter_state {
+ struct ring *ring;
+ unsigned long cons_pos;
+ unsigned long prod_pos;
+ bool consumer_dirty;
+ bool needs_wakeup;
+} __attribute__((__may_alias__));
+
+_Static_assert(sizeof(struct ring_buffer_iter_state) <=
+ sizeof(struct ring_buffer_iter),
+ "ring buffer iterator state exceeds opaque storage");
+_Static_assert(__alignof__(struct ring_buffer_iter_state) <=
+ __alignof__(struct ring_buffer_iter),
+ "ring buffer iterator is under-aligned");
+
+static void ringbuf_iter_publish(struct ring_buffer_iter_state *state)
{
- int *len_ptr, len, err;
- /* 64-bit to avoid overflow in case of extreme application behavior */
- int64_t cnt = 0;
+ if (!state->consumer_dirty)
+ return;
+
+ /* Release consumed records to the producer. */
+ __atomic_store_n(state->ring->consumer_pos, state->cons_pos,
+ __ATOMIC_RELEASE);
+ state->consumer_dirty = false;
+ state->needs_wakeup = true;
+}
+
+static void ringbuf_iter_init(struct ring_buffer_iter *it, struct ring *r)
+{
+ struct ring_buffer_iter_state *state = (void *)it;
+
+ memset(it, 0, sizeof(*it));
+ state->ring = r;
+ state->cons_pos = __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
+ state->prod_pos = state->cons_pos;
+}
+
+int ring_buffer_iter_new(struct ring_buffer_iter *it, struct ring *r)
+{
+ if (r->overwrite)
+ return libbpf_err(-EOPNOTSUPP);
+
+ ringbuf_iter_init(it, r);
+ return 0;
+}
+
+static const void *ringbuf_iter_next(struct ring_buffer_iter *it, size_t *size,
+ bool sync)
+{
+ struct ring_buffer_iter_state *state = (void *)it;
+ struct ring *r = state->ring;
+ int *len_ptr, len;
unsigned long cons_pos, prod_pos;
bool got_new_data;
void *sample;
- err = ringbuf_validate(r);
- if (err)
- return err;
- if (n == 0)
- return 0;
+ if (!r)
+ return NULL;
+ if (sync)
+ ringbuf_iter_publish(state);
+ cons_pos = state->cons_pos;
+ prod_pos = state->prod_pos;
- cons_pos = __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
- for (;;) {
+ do {
got_new_data = false;
- prod_pos = __atomic_load_n(r->producer_pos, __ATOMIC_ACQUIRE);
- while (prod_pos - cons_pos > 0) {
+ if (cons_pos == prod_pos) {
+ ringbuf_iter_publish(state);
+ if (state->needs_wakeup) {
+ /*
+ * Ensure either this sees a new record or its producer sees
+ * the updated consumer position and sends a notification.
+ */
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
+ state->needs_wakeup = false;
+ }
+ prod_pos = __atomic_load_n(r->producer_pos,
+ __ATOMIC_ACQUIRE);
+ state->prod_pos = prod_pos;
+ }
+
+ while (cons_pos != prod_pos) {
len_ptr = r->data + (cons_pos & r->mask);
len = __atomic_load_n(len_ptr, __ATOMIC_ACQUIRE);
/* Retry a busy record once after publishing prior records. */
- if (len & BPF_RINGBUF_BUSY_BIT)
+ if (len & BPF_RINGBUF_BUSY_BIT) {
+ ringbuf_iter_publish(state);
+ got_new_data = state->needs_wakeup;
+ if (got_new_data) {
+ /* Order the consumer update before retrying the header. */
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
+ state->needs_wakeup = false;
+ }
break;
+ }
got_new_data = true;
cons_pos += roundup_len(len);
+ state->cons_pos = cons_pos;
+ state->consumer_dirty = true;
if ((len & BPF_RINGBUF_DISCARD_BIT) == 0) {
sample = (void *)len_ptr + BPF_RINGBUF_HDR_SZ;
- err = r->sample_cb(r->ctx, sample, len);
- if (err < 0) {
- /* update consumer pos and bail out */
- __atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
- __atomic_thread_fence(__ATOMIC_SEQ_CST);
- return err;
- }
- cnt++;
+ if (size)
+ *size = len;
+ return sample;
}
- __atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
-
- if (cnt >= n)
- goto done;
+ if (sync)
+ ringbuf_iter_publish(state);
}
- if (!got_new_data)
- return cnt;
+ } while (got_new_data);
- /*
- * Order the published consumer position before the next
- * producer-position load, whether below or in a later invocation.
- */
+ state->ring = NULL;
+ return NULL;
+}
+
+const void *ring_buffer_iter_next(struct ring_buffer_iter *it, size_t *size,
+ bool sync)
+{
+ return ringbuf_iter_next(it, size, sync);
+}
+
+void ring_buffer_iter_destroy(struct ring_buffer_iter *it)
+{
+ struct ring_buffer_iter_state *state = (void *)it;
+
+ ringbuf_iter_publish(state);
+ /* Order publication before a producer load in a later invocation. */
+ if (state->needs_wakeup)
__atomic_thread_fence(__ATOMIC_SEQ_CST);
+ memset(it, 0, sizeof(*it));
+}
+
+static int64_t ringbuf_process_ring(struct ring *r, size_t n)
+{
+ struct ring_buffer_iter it
+ __attribute__((cleanup(ring_buffer_iter_destroy))) = {};
+ const void *sample;
+ size_t size;
+ int err = ringbuf_validate(r);
+ /* 64-bit to avoid overflow in case of extreme application behavior */
+ int64_t cnt = 0;
+
+ if (err)
+ return err;
+ if (n == 0)
+ return 0;
+
+ ringbuf_iter_init(&it, r);
+ while (cnt < n && (sample = ringbuf_iter_next(&it, &size, true))) {
+ err = r->sample_cb(r->ctx, (void *)sample, size);
+ if (err < 0)
+ return err;
+ cnt++;
}
-done:
- __atomic_thread_fence(__ATOMIC_SEQ_CST);
return cnt;
}
@@ -421,7 +523,7 @@ struct ring *ring_buffer__ring(struct ring_buffer *rb, unsigned int idx)
unsigned long ring__consumer_pos(const struct ring *r)
{
- /* Synchronizes with the release store in ringbuf_process_ring(). */
+ /* Synchronizes with the iterator's release store. */
return __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
}
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 5/8] libbpf: ringbuf: Simplify iterator implementation
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
` (3 preceding siblings ...)
2026-11-20 20:33 ` [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups Tamir Duberstein
` (2 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
The iterator core retained ringbuf_process_ring()'s nested loop shape to
make the functional conversion easy to compare. This leaves mirrored
position variables and a got_new_data flag to coordinate the two loops.
Express the same protocol as a single loop over the iterator state. Keep
consumer publication, producer rechecks, and the single busy-record
retry unchanged.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/lib/bpf/ringbuf.c | 68 ++++++++++++++++++++++---------------------------
1 file changed, 30 insertions(+), 38 deletions(-)
diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
index 2e281e2fa573..9da54e6b5acd 100644
--- a/tools/lib/bpf/ringbuf.c
+++ b/tools/lib/bpf/ringbuf.c
@@ -311,20 +311,14 @@ static const void *ringbuf_iter_next(struct ring_buffer_iter *it, size_t *size,
struct ring_buffer_iter_state *state = (void *)it;
struct ring *r = state->ring;
int *len_ptr, len;
- unsigned long cons_pos, prod_pos;
- bool got_new_data;
- void *sample;
if (!r)
return NULL;
if (sync)
ringbuf_iter_publish(state);
- cons_pos = state->cons_pos;
- prod_pos = state->prod_pos;
- do {
- got_new_data = false;
- if (cons_pos == prod_pos) {
+ for (;;) {
+ if (state->cons_pos == state->prod_pos) {
ringbuf_iter_publish(state);
if (state->needs_wakeup) {
/*
@@ -334,46 +328,44 @@ static const void *ringbuf_iter_next(struct ring_buffer_iter *it, size_t *size,
__atomic_thread_fence(__ATOMIC_SEQ_CST);
state->needs_wakeup = false;
}
- prod_pos = __atomic_load_n(r->producer_pos,
- __ATOMIC_ACQUIRE);
- state->prod_pos = prod_pos;
+ state->prod_pos = __atomic_load_n(r->producer_pos,
+ __ATOMIC_ACQUIRE);
}
- while (cons_pos != prod_pos) {
- len_ptr = r->data + (cons_pos & r->mask);
- len = __atomic_load_n(len_ptr, __ATOMIC_ACQUIRE);
-
- /* Retry a busy record once after publishing prior records. */
- if (len & BPF_RINGBUF_BUSY_BIT) {
- ringbuf_iter_publish(state);
- got_new_data = state->needs_wakeup;
- if (got_new_data) {
- /* Order the consumer update before retrying the header. */
- __atomic_thread_fence(__ATOMIC_SEQ_CST);
- state->needs_wakeup = false;
- }
- break;
- }
+ if (state->cons_pos == state->prod_pos) {
+ state->ring = NULL;
+ return NULL;
+ }
- got_new_data = true;
- cons_pos += roundup_len(len);
- state->cons_pos = cons_pos;
- state->consumer_dirty = true;
+ len_ptr = r->data + (state->cons_pos & r->mask);
+ len = __atomic_load_n(len_ptr, __ATOMIC_ACQUIRE);
- if ((len & BPF_RINGBUF_DISCARD_BIT) == 0) {
- sample = (void *)len_ptr + BPF_RINGBUF_HDR_SZ;
- if (size)
- *size = len;
- return sample;
+ /* Publish prior records and retry a record that is still busy. */
+ if (len & BPF_RINGBUF_BUSY_BIT) {
+ ringbuf_iter_publish(state);
+ if (!state->needs_wakeup) {
+ state->ring = NULL;
+ return NULL;
}
+ /* Order the consumer update before retrying the header. */
+ __atomic_thread_fence(__ATOMIC_SEQ_CST);
+ state->needs_wakeup = false;
+ continue;
+ }
+ state->cons_pos += roundup_len(len);
+ state->consumer_dirty = true;
+
+ if (len & BPF_RINGBUF_DISCARD_BIT) {
if (sync)
ringbuf_iter_publish(state);
+ continue;
}
- } while (got_new_data);
- state->ring = NULL;
- return NULL;
+ if (size)
+ *size = len;
+ return (void *)len_ptr + BPF_RINGBUF_HDR_SZ;
+ }
}
const void *ring_buffer_iter_next(struct ring_buffer_iter *it, size_t *size,
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
` (4 preceding siblings ...)
2026-11-20 20:33 ` [PATCH bpf-next v3 5/8] libbpf: ringbuf: Simplify iterator implementation Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 20:33 ` [PATCH bpf-next v3 7/8] selftests/bpf: Verify ringbuf callback semantics Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 8/8] selftests/bpf: Test caller-owned ring iterators Tamir Duberstein
7 siblings, 1 reply; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
Use cleanup attributes for skeletons, managers, and file descriptors in
the ring buffer tests. Pass skeletons and managers explicitly, and stop
or cancel worker threads before joining them. Preserve the existing
zero-bound, NULL-callback, and overwrite-map regression cases.
Add edge-triggered readiness coverage that consumes one record per call
to exercise the wakeup handshake across bounded returns.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/testing/selftests/bpf/prog_tests/ringbuf.c | 465 +++++++++++++++--------
1 file changed, 310 insertions(+), 155 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf.c b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
index 29be2476c478..633e10aa5537 100644
--- a/tools/testing/selftests/bpf/prog_tests/ringbuf.c
+++ b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
@@ -30,8 +30,6 @@ struct sample {
char comm[16];
};
-static int sample_cnt;
-
static void atomic_inc(int *cnt)
{
__atomic_add_fetch(cnt, 1, __ATOMIC_SEQ_CST);
@@ -44,9 +42,10 @@ static int atomic_xchg(int *cnt, int val)
static int process_sample(void *ctx, void *data, size_t len)
{
+ int *sample_cnt = ctx;
struct sample *s = data;
- atomic_inc(&sample_cnt);
+ atomic_inc(sample_cnt);
switch (s->seq) {
case 0:
@@ -63,11 +62,76 @@ static int process_sample(void *ctx, void *data, size_t len)
}
}
-static struct test_ringbuf_map_key_lskel *skel_map_key;
-static struct test_ringbuf_lskel *skel;
-static struct ring_buffer *ringbuf;
+static void cleanup_ring_buffer(struct ring_buffer **rb)
+{
+ if (*rb)
+ ring_buffer__free(*rb);
+}
+
+static void cleanup_test_ringbuf_lskel(struct test_ringbuf_lskel **skel)
+{
+ if (*skel)
+ test_ringbuf_lskel__destroy(*skel);
+}
+
+static void cleanup_fd(int *fd)
+{
+ if (*fd >= 0)
+ close(*fd);
+}
+
+static void cleanup_test_ringbuf_n_lskel(struct test_ringbuf_n_lskel **skel)
+{
+ if (*skel)
+ test_ringbuf_n_lskel__destroy(*skel);
+}
+
+static void
+cleanup_test_ringbuf_map_key_lskel(struct test_ringbuf_map_key_lskel **skel)
+{
+ if (*skel)
+ test_ringbuf_map_key_lskel__destroy(*skel);
+}
+
+static void
+cleanup_test_ringbuf_write_lskel(struct test_ringbuf_write_lskel **skel)
+{
+ if (*skel)
+ test_ringbuf_write_lskel__destroy(*skel);
+}
+
+static void
+cleanup_test_ringbuf_overwrite_lskel(struct test_ringbuf_overwrite_lskel **skel)
+{
+ if (*skel)
+ test_ringbuf_overwrite_lskel__destroy(*skel);
+}
+
+struct poll_ctx {
+ struct ring_buffer *ringbuf;
+ long timeout;
+};
+
+struct thread_guard {
+ pthread_t thread;
+ bool *stop;
+ bool joined;
+};
+
+static void cleanup_thread(struct thread_guard *thread)
+{
+ if (thread->joined)
+ return;
+
+ if (thread->stop)
+ __atomic_store_n(thread->stop, true, __ATOMIC_RELAXED);
+ else
+ pthread_cancel(thread->thread);
+ pthread_join(thread->thread, NULL);
+ thread->joined = true;
+}
-static void trigger_samples()
+static void trigger_samples(struct test_ringbuf_lskel *skel)
{
skel->bss->dropped = 0;
skel->bss->total = 0;
@@ -80,21 +144,25 @@ static void trigger_samples()
syscall(__NR_getpgid);
}
-static void *poll_thread(void *input)
+static void *poll_thread(void *arg)
{
- long timeout = (long)input;
+ struct poll_ctx *ctx = arg;
- return (void *)(long)ring_buffer__poll(ringbuf, timeout);
+ return (void *)(long)ring_buffer__poll(ctx->ringbuf, ctx->timeout);
}
static void ringbuf_write_subtest(void)
{
- struct test_ringbuf_write_lskel *skel;
+ struct test_ringbuf_write_lskel *skel
+ __attribute__((cleanup(cleanup_test_ringbuf_write_lskel))) =
+ test_ringbuf_write_lskel__open();
int page_size = getpagesize();
size_t *mmap_ptr;
int err, rb_fd;
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) = NULL;
+ int sample_cnt = 0;
- skel = test_ringbuf_write_lskel__open();
if (!ASSERT_OK_PTR(skel, "skel_open"))
return;
@@ -102,25 +170,25 @@ static void ringbuf_write_subtest(void)
err = test_ringbuf_write_lskel__load(skel);
if (!ASSERT_OK(err, "skel_load"))
- goto cleanup;
+ return;
rb_fd = skel->maps.ringbuf.map_fd;
mmap_ptr = mmap(NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, rb_fd, 0);
if (!ASSERT_OK_PTR(mmap_ptr, "rw_cons_pos"))
- goto cleanup;
+ return;
*mmap_ptr = 0x30000;
ASSERT_OK(munmap(mmap_ptr, page_size), "unmap_rw");
skel->bss->pid = getpid();
- ringbuf = ring_buffer__new(rb_fd, process_sample, NULL, NULL);
+ ringbuf = ring_buffer__new(rb_fd, process_sample, &sample_cnt, NULL);
if (!ASSERT_OK_PTR(ringbuf, "ringbuf_new"))
- goto cleanup;
+ return;
err = test_ringbuf_write_lskel__attach(skel);
if (!ASSERT_OK(err, "skel_attach"))
- goto cleanup_ringbuf;
+ return;
skel->bss->discarded = 0;
skel->bss->passed = 0;
@@ -131,12 +199,6 @@ static void ringbuf_write_subtest(void)
ASSERT_EQ(skel->bss->discarded, 2, "discarded");
ASSERT_EQ(skel->bss->passed, 0, "passed");
-
- test_ringbuf_write_lskel__detach(skel);
-cleanup_ringbuf:
- ring_buffer__free(ringbuf);
-cleanup:
- test_ringbuf_write_lskel__destroy(skel);
}
static void ringbuf_subtest(void)
@@ -150,8 +212,13 @@ static void ringbuf_subtest(void)
struct ring *ring;
int map_fd;
unsigned long avail_data, ring_size, cons_pos, prod_pos;
+ struct test_ringbuf_lskel *skel
+ __attribute__((cleanup(cleanup_test_ringbuf_lskel))) =
+ test_ringbuf_lskel__open();
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) = NULL;
+ int sample_cnt = 0;
- skel = test_ringbuf_lskel__open();
if (CHECK(!skel, "skel_open", "skeleton open failed\n"))
return;
@@ -159,7 +226,7 @@ static void ringbuf_subtest(void)
err = test_ringbuf_lskel__load(skel);
if (CHECK(err != 0, "skel_load", "skeleton load failed\n"))
- goto cleanup;
+ return;
rb_fd = skel->maps.ringbuf.map_fd;
/* good read/write cons_pos */
@@ -167,7 +234,7 @@ static void ringbuf_subtest(void)
ASSERT_OK_PTR(mmap_ptr, "rw_cons_pos");
tmp_ptr = mremap(mmap_ptr, page_size, 2 * page_size, MREMAP_MAYMOVE);
if (!ASSERT_ERR_PTR(tmp_ptr, "rw_extend"))
- goto cleanup;
+ return;
ASSERT_ERR(mprotect(mmap_ptr, page_size, PROT_EXEC), "exec_cons_pos_protect");
ASSERT_OK(munmap(mmap_ptr, page_size), "unmap_rw");
@@ -190,7 +257,7 @@ static void ringbuf_subtest(void)
/* good read-only pages */
mmap_ptr = mmap(NULL, 4 * page_size, PROT_READ, MAP_SHARED, rb_fd, 0);
if (!ASSERT_OK_PTR(mmap_ptr, "ro_prod_pos"))
- goto cleanup;
+ return;
ASSERT_ERR(mprotect(mmap_ptr, 4 * page_size, PROT_WRITE), "write_protect");
ASSERT_ERR(mprotect(mmap_ptr, 4 * page_size, PROT_EXEC), "exec_protect");
@@ -200,7 +267,7 @@ static void ringbuf_subtest(void)
/* good read-only pages with initial offset */
mmap_ptr = mmap(NULL, page_size, PROT_READ, MAP_SHARED, rb_fd, page_size);
if (!ASSERT_OK_PTR(mmap_ptr, "ro_prod_pos"))
- goto cleanup;
+ return;
ASSERT_ERR(mprotect(mmap_ptr, page_size, PROT_WRITE), "write_protect");
ASSERT_ERR(mprotect(mmap_ptr, page_size, PROT_EXEC), "exec_protect");
@@ -210,20 +277,20 @@ static void ringbuf_subtest(void)
/* only trigger BPF program for current process */
skel->bss->pid = getpid();
- ringbuf = ring_buffer__new(skel->maps.ringbuf.map_fd,
- process_sample, NULL, NULL);
+ ringbuf = ring_buffer__new(skel->maps.ringbuf.map_fd, process_sample,
+ &sample_cnt, NULL);
if (CHECK(!ringbuf, "ringbuf_create", "failed to create ringbuf\n"))
- goto cleanup;
+ return;
err = test_ringbuf_lskel__attach(skel);
if (CHECK(err, "skel_attach", "skeleton attachment failed: %d\n", err))
- goto cleanup;
+ return;
- trigger_samples();
+ trigger_samples(skel);
ring = ring_buffer__ring(ringbuf, 0);
if (!ASSERT_OK_PTR(ring, "ring_buffer__ring_idx_0"))
- goto cleanup;
+ return;
map_fd = ring__map_fd(ring);
ASSERT_EQ(map_fd, skel->maps.ringbuf.map_fd, "ring_map_fd");
@@ -259,14 +326,14 @@ static void ringbuf_subtest(void)
/* -EDONE is used as an indicator that we are done */
if (CHECK(err != -EDONE, "err_done", "done err: %d\n", err))
- goto cleanup;
+ return;
cnt = atomic_xchg(&sample_cnt, 0);
CHECK(cnt != 2, "cnt", "exp %d samples, got %d\n", 2, cnt);
/* we expect extra polling to return nothing */
err = ring_buffer__poll(ringbuf, 0);
if (CHECK(err != 0, "extra_samples", "poll result: %d\n", err))
- goto cleanup;
+ return;
cnt = atomic_xchg(&sample_cnt, 0);
CHECK(cnt != 0, "cnt", "exp %d samples, got %d\n", 0, cnt);
@@ -278,7 +345,7 @@ static void ringbuf_subtest(void)
1L, skel->bss->discarded);
/* now validate consumer position is updated and returned */
- trigger_samples();
+ trigger_samples(skel);
CHECK(skel->bss->cons_pos != 3 * rec_sz,
"err_cons_pos", "exp %ld, got %ld\n",
3L * rec_sz, skel->bss->cons_pos);
@@ -288,24 +355,35 @@ static void ringbuf_subtest(void)
CHECK(cnt != 2, "cnt", "exp %d samples, got %d\n", 2, cnt);
/* start poll in background w/ long timeout */
- err = pthread_create(&thread, NULL, poll_thread, (void *)(long)10000);
+ struct poll_ctx poll_ctx = {
+ .ringbuf = ringbuf,
+ .timeout = 10000,
+ };
+ err = pthread_create(&thread, NULL, poll_thread, &poll_ctx);
if (CHECK(err, "bg_poll", "pthread_create failed: %d\n", err))
- goto cleanup;
+ return;
+ struct thread_guard thread_guard
+ __attribute__((cleanup(cleanup_thread))) = {
+ .thread = thread,
+ .joined = false,
+ };
/* turn off notifications now */
skel->bss->flags = BPF_RB_NO_WAKEUP;
/* give background thread a bit of a time */
usleep(50000);
- trigger_samples();
+ trigger_samples(skel);
/* sleeping arbitrarily is bad, but no better way to know that
* epoll_wait() **DID NOT** unblock in background thread
*/
usleep(50000);
/* background poll should still be blocked */
err = pthread_tryjoin_np(thread, (void **)&bg_ret);
+ if (!err)
+ thread_guard.joined = true;
if (CHECK(err != EBUSY, "try_join", "err %d\n", err))
- goto cleanup;
+ return;
/* BPF side did everything right */
CHECK(skel->bss->dropped != 0, "err_dropped", "exp %ld, got %ld\n",
@@ -323,12 +401,14 @@ static void ringbuf_subtest(void)
/* produce new samples, no notification should be triggered, because
* consumer is now behind
*/
- trigger_samples();
+ trigger_samples(skel);
/* background poll should still be blocked */
err = pthread_tryjoin_np(thread, (void **)&bg_ret);
+ if (!err)
+ thread_guard.joined = true;
if (CHECK(err != EBUSY, "try_join", "err %d\n", err))
- goto cleanup;
+ return;
/* still no samples, because consumer is behind */
cnt = atomic_xchg(&sample_cnt, 0);
@@ -348,11 +428,13 @@ static void ringbuf_subtest(void)
/* now we should get a pending notification */
usleep(50000);
err = pthread_tryjoin_np(thread, (void **)&bg_ret);
+ if (!err)
+ thread_guard.joined = true;
if (CHECK(err, "join_bg", "err %d\n", err))
- goto cleanup;
+ return;
if (CHECK(bg_ret <= 0, "bg_ret", "epoll_wait result: %ld", bg_ret))
- goto cleanup;
+ return;
/* due to timing variations, there could still be non-notified
* samples, so consume them here to collect all the samples
@@ -375,11 +457,6 @@ static void ringbuf_subtest(void)
2L, skel->bss->total);
CHECK(skel->bss->discarded != 1, "err_discarded", "exp %ld, got %ld\n",
1L, skel->bss->discarded);
-
- test_ringbuf_lskel__detach(skel);
-cleanup:
- ring_buffer__free(ringbuf);
- test_ringbuf_lskel__destroy(skel);
}
/*
@@ -408,14 +485,10 @@ static int process_noop_sample(void *ctx, void *data, size_t len)
static void ringbuf_null_cb_subtest(void)
{
- struct test_ringbuf_n_lskel *skel_n;
- struct ring_buffer *ringbuf = NULL;
- struct ring *ring;
- unsigned long consumer_pos;
- int no_cb_map_fd = -1;
int err;
-
- skel_n = test_ringbuf_n_lskel__open();
+ struct test_ringbuf_n_lskel *skel_n
+ __attribute__((cleanup(cleanup_test_ringbuf_n_lskel))) =
+ test_ringbuf_n_lskel__open();
if (!ASSERT_OK_PTR(skel_n, "test_ringbuf_n_lskel__open"))
return;
@@ -425,80 +498,170 @@ static void ringbuf_null_cb_subtest(void)
err = test_ringbuf_n_lskel__load(skel_n);
if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
- goto cleanup;
+ return;
err = test_ringbuf_n_lskel__attach(skel_n);
if (!ASSERT_OK(err, "test_ringbuf_n_lskel__attach"))
- goto cleanup;
+ return;
syscall(__NR_getpgid);
- no_cb_map_fd = bpf_map_create(BPF_MAP_TYPE_RINGBUF, NULL, 0, 0,
- getpagesize(), NULL);
+ __attribute__((cleanup(cleanup_fd))) int no_cb_map_fd = bpf_map_create(
+ BPF_MAP_TYPE_RINGBUF, NULL, 0, 0, getpagesize(), NULL);
if (!ASSERT_OK_FD(no_cb_map_fd, "bpf_map_create"))
- goto cleanup;
+ return;
/* Manager APIs must validate all rings before consuming any of them. */
- ringbuf = ring_buffer__new(skel_n->maps.ringbuf.map_fd,
- process_noop_sample, NULL, NULL);
- if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
- goto cleanup_fd;
+ {
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) =
+ ring_buffer__new(skel_n->maps.ringbuf.map_fd,
+ process_noop_sample, NULL,
+ NULL);
+ if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
+ return;
+
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
+
+ if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+ return;
+
+ err = ring_buffer__add(ringbuf, no_cb_map_fd, NULL, NULL);
+ if (!ASSERT_OK(err, "ring_buffer__add_no_cb"))
+ return;
+
+ unsigned long consumer_pos = ring__consumer_pos(ring);
+
+ ASSERT_GT(ring__producer_pos(ring), consumer_pos,
+ "producer_pos_mixed_cb");
+
+ err = ring_buffer__consume_n(ringbuf, 0);
+ ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_mixed_cb");
+ err = ring_buffer__consume(ringbuf);
+ ASSERT_EQ(err, -EINVAL, "ringbuf_consume_mixed_cb");
+ err = ring_buffer__poll(ringbuf, 0);
+ ASSERT_EQ(err, -EINVAL, "ringbuf_poll_mixed_cb");
+ ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_mixed_cb");
+ }
- ring = ring_buffer__ring(ringbuf, 0);
- if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
- goto cleanup_ringbuf;
+ /* Direct consumption must reject a ring without a callback. */
+ {
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) =
+ ring_buffer__new(skel_n->maps.ringbuf.map_fd,
+ NULL, NULL, NULL);
+ if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new_no_cb"))
+ return;
+
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
+
+ if (!ASSERT_OK_PTR(ring, "ring_buffer__ring_no_cb"))
+ return;
+ unsigned long consumer_pos = ring__consumer_pos(ring);
+
+ err = ring_buffer__consume_n(ringbuf, 0);
+ ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_no_cb");
+ err = ring__consume_n(ring, 0);
+ ASSERT_EQ(err, -EINVAL, "ring_consume_zero_no_cb");
+ err = ring__consume(ring);
+ ASSERT_EQ(err, -EINVAL, "ring_consume_no_cb");
+ ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_no_cb");
+ }
+}
- err = ring_buffer__add(ringbuf, no_cb_map_fd, NULL, NULL);
- if (!ASSERT_OK(err, "ring_buffer__add_no_cb"))
- goto cleanup_ringbuf;
+#define N_WAKEUP_SAMPLES 20000
- consumer_pos = ring__consumer_pos(ring);
- ASSERT_GT(ring__producer_pos(ring), consumer_pos,
- "producer_pos_mixed_cb");
+struct wakeup_ctx {
+ bool stop;
+};
- err = ring_buffer__consume_n(ringbuf, 0);
- ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_mixed_cb");
- err = ring_buffer__consume(ringbuf);
- ASSERT_EQ(err, -EINVAL, "ringbuf_consume_mixed_cb");
- err = ring_buffer__poll(ringbuf, 0);
- ASSERT_EQ(err, -EINVAL, "ringbuf_poll_mixed_cb");
- ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
- "consumer_pos_mixed_cb");
+static void *wakeup_producer(void *arg)
+{
+ struct wakeup_ctx *ctx = arg;
- ring_buffer__free(ringbuf);
- ringbuf =
- ring_buffer__new(skel_n->maps.ringbuf.map_fd, NULL, NULL, NULL);
- if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new_no_cb"))
- goto cleanup_fd;
+ while (!__atomic_load_n(&ctx->stop, __ATOMIC_RELAXED))
+ syscall(__NR_getpgid);
+ return NULL;
+}
- ring = ring_buffer__ring(ringbuf, 0);
- if (!ASSERT_OK_PTR(ring, "ring_buffer__ring_no_cb"))
- goto cleanup_ringbuf;
- consumer_pos = ring__consumer_pos(ring);
+static void ringbuf_wakeup_subtest(void)
+{
+ int err;
+ struct test_ringbuf_n_lskel *skel_n
+ __attribute__((cleanup(cleanup_test_ringbuf_n_lskel))) =
+ test_ringbuf_n_lskel__open();
+ if (!ASSERT_OK_PTR(skel_n, "test_ringbuf_n_lskel__open"))
+ return;
- err = ring_buffer__consume_n(ringbuf, 0);
- ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_no_cb");
- err = ring__consume_n(ring, 0);
- ASSERT_EQ(err, -EINVAL, "ring_consume_zero_no_cb");
- err = ring__consume(ring);
- ASSERT_EQ(err, -EINVAL, "ring_consume_no_cb");
- ASSERT_EQ(ring__consumer_pos(ring), consumer_pos, "consumer_pos_no_cb");
-
-cleanup_ringbuf:
- ring_buffer__free(ringbuf);
-cleanup_fd:
- close(no_cb_map_fd);
-cleanup:
- test_ringbuf_n_lskel__destroy(skel_n);
+ skel_n->maps.ringbuf.max_entries = getpagesize();
+ skel_n->bss->pid = getpid();
+ skel_n->bss->value = SAMPLE_VALUE;
+
+ err = test_ringbuf_n_lskel__load(skel_n);
+ if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
+ return;
+
+ err = test_ringbuf_n_lskel__attach(skel_n);
+ if (!ASSERT_OK(err, "test_ringbuf_n_lskel__attach"))
+ return;
+
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) =
+ ring_buffer__new(skel_n->maps.ringbuf.map_fd,
+ process_noop_sample, NULL, NULL);
+ if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
+ return;
+
+ struct epoll_event event = {
+ .events = EPOLLIN | EPOLLET,
+ };
+ int epoll_fd __attribute__((cleanup(cleanup_fd))) =
+ epoll_create1(EPOLL_CLOEXEC);
+ if (!ASSERT_OK_FD(epoll_fd, "epoll_create1"))
+ return;
+
+ err = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, skel_n->maps.ringbuf.map_fd,
+ &event);
+ if (!ASSERT_OK(err, "epoll_ctl"))
+ return;
+
+ pthread_t producer;
+ struct wakeup_ctx ctx = {};
+
+ err = pthread_create(&producer, NULL, wakeup_producer, &ctx);
+ if (!ASSERT_OK(err, "pthread_create"))
+ return;
+ struct thread_guard producer_guard
+ __attribute__((cleanup(cleanup_thread))) = {
+ .thread = producer,
+ .stop = &ctx.stop,
+ .joined = false,
+ };
+
+ int total = 0;
+
+ while (total < N_WAKEUP_SAMPLES) {
+ err = epoll_wait(epoll_fd, &event, 1, 1000);
+ if (!ASSERT_EQ(err, 1, "epoll_wait"))
+ return;
+ while ((err = ring_buffer__consume_n(ringbuf, 1)) > 0)
+ total += err;
+ if (!ASSERT_OK(err, "ring_buffer__consume_n"))
+ return;
+ }
}
static void ringbuf_n_subtest(void)
{
- struct test_ringbuf_n_lskel *skel_n;
- struct ring *ring;
+ struct test_ringbuf_n_lskel *skel_n
+ __attribute__((cleanup(cleanup_test_ringbuf_n_lskel))) =
+ test_ringbuf_n_lskel__open();
int err, i;
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) = NULL;
- skel_n = test_ringbuf_n_lskel__open();
if (!ASSERT_OK_PTR(skel_n, "test_ringbuf_n_lskel__open"))
return;
@@ -507,49 +670,46 @@ static void ringbuf_n_subtest(void)
err = test_ringbuf_n_lskel__load(skel_n);
if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
- goto cleanup;
+ return;
ringbuf = ring_buffer__new(skel_n->maps.ringbuf.map_fd,
process_n_sample, NULL, NULL);
if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
- goto cleanup;
+ return;
err = test_ringbuf_n_lskel__attach(skel_n);
if (!ASSERT_OK(err, "test_ringbuf_n_lskel__attach"))
- goto cleanup_ringbuf;
+ return;
/* Produce N_TOT_SAMPLES samples in the ring buffer by calling getpid() */
skel_n->bss->value = SAMPLE_VALUE;
for (i = 0; i < N_TOT_SAMPLES; i++)
syscall(__NR_getpgid);
- ring = ring_buffer__ring(ringbuf, 0);
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
+
if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
- goto cleanup_ringbuf;
+ return;
err = ring_buffer__consume_n(ringbuf, 0);
if (!ASSERT_EQ(err, 0, "ringbuf_consume_zero"))
- goto cleanup_ringbuf;
+ return;
err = ring__consume_n(ring, 0);
if (!ASSERT_EQ(err, 0, "ring_consume_zero"))
- goto cleanup_ringbuf;
+ return;
/* Consume all samples from the ring buffer in batches of N_SAMPLES */
for (i = 0; i < N_TOT_SAMPLES; i += err) {
err = ring_buffer__consume_n(ringbuf, N_SAMPLES);
if (!ASSERT_EQ(err, N_SAMPLES, "rb_consume"))
- goto cleanup_ringbuf;
+ return;
}
-
-cleanup_ringbuf:
- ring_buffer__free(ringbuf);
-cleanup:
- test_ringbuf_n_lskel__destroy(skel_n);
}
static int process_map_key_sample(void *ctx, void *data, size_t len)
{
+ struct test_ringbuf_map_key_lskel *skel_map_key = ctx;
struct sample *s;
int err, val;
@@ -570,8 +730,12 @@ static int process_map_key_sample(void *ctx, void *data, size_t len)
static void ringbuf_map_key_subtest(void)
{
int err;
+ struct test_ringbuf_map_key_lskel *skel_map_key
+ __attribute__((cleanup(cleanup_test_ringbuf_map_key_lskel))) =
+ test_ringbuf_map_key_lskel__open();
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) = NULL;
- skel_map_key = test_ringbuf_map_key_lskel__open();
if (!ASSERT_OK_PTR(skel_map_key, "test_ringbuf_map_key_lskel__open"))
return;
@@ -580,47 +744,44 @@ static void ringbuf_map_key_subtest(void)
err = test_ringbuf_map_key_lskel__load(skel_map_key);
if (!ASSERT_OK(err, "test_ringbuf_map_key_lskel__load"))
- goto cleanup;
+ return;
ringbuf = ring_buffer__new(skel_map_key->maps.ringbuf.map_fd,
- process_map_key_sample, NULL, NULL);
+ process_map_key_sample, skel_map_key, NULL);
if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
- goto cleanup;
+ return;
err = test_ringbuf_map_key_lskel__attach(skel_map_key);
if (!ASSERT_OK(err, "test_ringbuf_map_key_lskel__attach"))
- goto cleanup_ringbuf;
+ return;
syscall(__NR_getpgid);
ASSERT_EQ(skel_map_key->bss->seq, 1, "skel_map_key->bss->seq");
err = ring_buffer__poll(ringbuf, -1);
ASSERT_EQ(err, -EDONE, "ring_buffer__poll");
-
-cleanup_ringbuf:
- ring_buffer__free(ringbuf);
-cleanup:
- test_ringbuf_map_key_lskel__destroy(skel_map_key);
}
static void ringbuf_overwrite_callback_subtest(void)
{
- LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_RB_OVERWRITE);
- struct ring_buffer *ringbuf;
- struct ring *ring;
- int map_fd, err;
+ int err;
- map_fd = bpf_map_create(BPF_MAP_TYPE_RINGBUF, NULL, 0, 0, getpagesize(),
- &opts);
+ LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_RB_OVERWRITE);
+ __attribute__((cleanup(cleanup_fd))) int map_fd = bpf_map_create(
+ BPF_MAP_TYPE_RINGBUF, NULL, 0, 0, getpagesize(), &opts);
if (!ASSERT_OK_FD(map_fd, "bpf_map_create"))
return;
- ringbuf = ring_buffer__new(map_fd, process_noop_sample, NULL, NULL);
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) =
+ ring_buffer__new(map_fd, process_noop_sample, NULL,
+ NULL);
if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
- goto cleanup_fd;
+ return;
+
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
- ring = ring_buffer__ring(ringbuf, 0);
if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
- goto cleanup_ringbuf;
+ return;
err = ring_buffer__consume_n(ringbuf, 0);
ASSERT_EQ(err, -EOPNOTSUPP, "ringbuf_consume_zero");
@@ -632,22 +793,18 @@ static void ringbuf_overwrite_callback_subtest(void)
ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume_zero");
err = ring__consume(ring);
ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume");
-
-cleanup_ringbuf:
- ring_buffer__free(ringbuf);
-cleanup_fd:
- close(map_fd);
}
static void ringbuf_overwrite_mode_subtest(void)
{
unsigned long size, len1, len2, len3, len4, len5;
unsigned long expect_avail_data, expect_prod_pos, expect_over_pos;
- struct test_ringbuf_overwrite_lskel *skel;
+ struct test_ringbuf_overwrite_lskel *skel
+ __attribute__((cleanup(cleanup_test_ringbuf_overwrite_lskel))) =
+ test_ringbuf_overwrite_lskel__open();
int page_size = getpagesize();
int err;
- skel = test_ringbuf_overwrite_lskel__open();
if (!ASSERT_OK_PTR(skel, "skel_open"))
return;
@@ -669,11 +826,11 @@ static void ringbuf_overwrite_mode_subtest(void)
err = test_ringbuf_overwrite_lskel__load(skel);
if (!ASSERT_OK(err, "skel_load"))
- goto cleanup;
+ return;
err = test_ringbuf_overwrite_lskel__attach(skel);
if (!ASSERT_OK(err, "skel_attach"))
- goto cleanup;
+ return;
syscall(__NR_getpgid);
@@ -695,10 +852,6 @@ static void ringbuf_overwrite_mode_subtest(void)
expect_over_pos = len1 + BPF_RINGBUF_HDR_SZ;
ASSERT_EQ(skel->bss->over_pos, expect_over_pos, "check_over_pos");
-
- test_ringbuf_overwrite_lskel__detach(skel);
-cleanup:
- test_ringbuf_overwrite_lskel__destroy(skel);
}
void test_ringbuf(void)
@@ -709,6 +862,8 @@ void test_ringbuf(void)
ringbuf_n_subtest();
if (test__start_subtest("ringbuf_null_cb"))
ringbuf_null_cb_subtest();
+ if (test__start_subtest("ringbuf_wakeup"))
+ ringbuf_wakeup_subtest();
if (test__start_subtest("ringbuf_map_key"))
ringbuf_map_key_subtest();
if (test__start_subtest("ringbuf_write"))
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 7/8] selftests/bpf: Verify ringbuf callback semantics
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
` (5 preceding siblings ...)
2026-11-20 20:33 ` [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 8/8] selftests/bpf: Test caller-owned ring iterators Tamir Duberstein
7 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
Callback consumption ignores positive return values and publishes each
record's consumer position before invoking the next callback. Both
behaviors must survive changes to the consumption core.
Record these behaviors in ringbuf_n. Use an explicitly padded userspace
sample and a fixed-width BPF global value to match the BPF target's
64-bit layout on 32-bit systems.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/testing/selftests/bpf/prog_tests/ringbuf.c | 34 +++++++++++++++++++---
tools/testing/selftests/bpf/progs/test_ringbuf_n.c | 2 +-
2 files changed, 31 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf.c b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
index 633e10aa5537..cfc357dfd1e9 100644
--- a/tools/testing/selftests/bpf/prog_tests/ringbuf.c
+++ b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
@@ -469,13 +469,35 @@ static void ringbuf_subtest(void)
/* Sample value to verify the callback validity */
#define SAMPLE_VALUE 42L
+/* Match the BPF target's 64-bit long and padding. */
+struct sample_n {
+ int pid;
+ int pad;
+ __s64 value;
+ char comm[16];
+};
+
+struct process_n_ctx {
+ struct ring *ring;
+ unsigned long consumer_pos;
+ int callback_count;
+};
+
static int process_n_sample(void *ctx, void *data, size_t len)
{
- struct sample *s = data;
+ struct process_n_ctx *callback_ctx = ctx;
+ struct sample_n *s = data;
+ unsigned long consumer_pos;
ASSERT_EQ(s->value, SAMPLE_VALUE, "sample_value");
-
- return 0;
+ consumer_pos = ring__consumer_pos(callback_ctx->ring);
+ if (callback_ctx->callback_count)
+ ASSERT_GT(consumer_pos, callback_ctx->consumer_pos,
+ "callback_consumer_pos");
+ callback_ctx->consumer_pos = consumer_pos;
+ callback_ctx->callback_count++;
+
+ return 1;
}
static int process_noop_sample(void *ctx, void *data, size_t len)
@@ -672,8 +694,9 @@ static void ringbuf_n_subtest(void)
if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
return;
+ struct process_n_ctx callback_ctx = {};
ringbuf = ring_buffer__new(skel_n->maps.ringbuf.map_fd,
- process_n_sample, NULL, NULL);
+ process_n_sample, &callback_ctx, NULL);
if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
return;
@@ -690,6 +713,7 @@ static void ringbuf_n_subtest(void)
if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
return;
+ callback_ctx.ring = ring;
err = ring_buffer__consume_n(ringbuf, 0);
if (!ASSERT_EQ(err, 0, "ringbuf_consume_zero"))
@@ -705,6 +729,8 @@ static void ringbuf_n_subtest(void)
if (!ASSERT_EQ(err, N_SAMPLES, "rb_consume"))
return;
}
+ ASSERT_EQ(callback_ctx.callback_count, N_TOT_SAMPLES,
+ "positive_callback");
}
static int process_map_key_sample(void *ctx, void *data, size_t len)
diff --git a/tools/testing/selftests/bpf/progs/test_ringbuf_n.c b/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
index 8669eb42dbe0..a95d4f597175 100644
--- a/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
+++ b/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
@@ -22,7 +22,7 @@ struct {
} ringbuf SEC(".maps");
int pid = 0;
-long value = 0;
+__s64 value = 0;
SEC("fentry/" SYS_PREFIX "sys_getpgid")
int test_ringbuf_n(void *ctx)
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH bpf-next v3 8/8] selftests/bpf: Test caller-owned ring iterators
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
` (6 preceding siblings ...)
2026-11-20 20:33 ` [PATCH bpf-next v3 7/8] selftests/bpf: Verify ringbuf callback semantics Tamir Duberstein
@ 2026-11-20 20:33 ` Tamir Duberstein
7 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 20:33 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Martin KaFai Lau,
Song Liu, Yonghong Song, Jiri Olsa, Emil Tsalapatis, Shuah Khan,
Andrea Righi, Xu Kuohai, Andrea Righi, Bing-Jhong Billy Jheng,
David Vernet, Ihor Solodrai
Cc: bpf, linux-kselftest, linux-kernel, Andrew Werner, Zvi Effron,
Andrii Nakryiko, Tamir Duberstein
Verify borrowed-record lifetime, exhaustion, destruction, discard
handling, batched publication, and callback-free iteration. Exercise
edge-triggered epoll while draining through one iterator and while
destroying and recreating the iterator after each record. Observe
exhaustion before waiting for another notification.
Allow the BPF producer to discard a record between returned records, so
the batching test also checks deferred publication across discards.
Signed-off-by: Tamir Duberstein <tamird@kernel.org>
---
tools/testing/selftests/bpf/prog_tests/ringbuf.c | 167 +++++++++++++++++++++
tools/testing/selftests/bpf/progs/test_ringbuf_n.c | 7 +-
2 files changed, 173 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf.c b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
index cfc357dfd1e9..3e5bb241e799 100644
--- a/tools/testing/selftests/bpf/prog_tests/ringbuf.c
+++ b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
@@ -635,6 +635,10 @@ static void ringbuf_wakeup_subtest(void)
process_noop_sample, NULL, NULL);
if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
return;
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
+
+ if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+ return;
struct epoll_event event = {
.events = EPOLLIN | EPOLLET,
@@ -673,6 +677,41 @@ static void ringbuf_wakeup_subtest(void)
if (!ASSERT_OK(err, "ring_buffer__consume_n"))
return;
}
+
+ total = 0;
+ while (total < N_WAKEUP_SAMPLES) {
+ err = epoll_wait(epoll_fd, &event, 1, 1000);
+ if (!ASSERT_EQ(err, 1, "iterator_epoll_wait"))
+ return;
+
+ struct ring_buffer_iter iter
+ __attribute__((cleanup(ring_buffer_iter_destroy))) = {};
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new"))
+ return;
+ while (ring_buffer_iter_next(&iter, NULL, false))
+ total++;
+ }
+
+ /* Release each record through destruction before creating a new iterator. */
+ total = 0;
+ while (total < N_WAKEUP_SAMPLES) {
+ err = epoll_wait(epoll_fd, &event, 1, 1000);
+ if (!ASSERT_EQ(err, 1, "iterator_release_epoll_wait"))
+ return;
+
+ for (;;) {
+ struct ring_buffer_iter iter __attribute__((
+ cleanup(ring_buffer_iter_destroy))) = {};
+
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new"))
+ return;
+ if (!ring_buffer_iter_next(&iter, NULL, false))
+ break;
+ total++;
+ }
+ }
}
static void ringbuf_n_subtest(void)
@@ -733,6 +772,127 @@ static void ringbuf_n_subtest(void)
"positive_callback");
}
+static void ringbuf_iterator_subtest(void)
+{
+ struct test_ringbuf_n_lskel *skel_n
+ __attribute__((cleanup(cleanup_test_ringbuf_n_lskel))) =
+ test_ringbuf_n_lskel__open();
+ const struct sample_n *sample;
+ size_t sample_size;
+ int err, i;
+
+ if (!ASSERT_OK_PTR(skel_n, "test_ringbuf_n_lskel__open"))
+ return;
+
+ skel_n->maps.ringbuf.max_entries = getpagesize();
+ skel_n->bss->pid = getpid();
+ skel_n->bss->value = SAMPLE_VALUE;
+
+ err = test_ringbuf_n_lskel__load(skel_n);
+ if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
+ return;
+
+ err = test_ringbuf_n_lskel__attach(skel_n);
+ if (!ASSERT_OK(err, "test_ringbuf_n_lskel__attach"))
+ return;
+
+ struct ring_buffer *ringbuf
+ __attribute__((cleanup(cleanup_ring_buffer))) =
+ ring_buffer__new(skel_n->maps.ringbuf.map_fd, NULL,
+ NULL, NULL);
+ if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
+ return;
+
+ struct ring *ring = ring_buffer__ring(ringbuf, 0);
+
+ if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+ return;
+
+ struct ring_buffer_iter iter
+ __attribute__((cleanup(ring_buffer_iter_destroy))) = {};
+
+ for (i = 0; i < 3; i++)
+ syscall(__NR_getpgid);
+
+ unsigned long consumer_pos = ring__consumer_pos(ring);
+
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new"))
+ return;
+ sample = ring_buffer_iter_next(&iter, &sample_size, true);
+ if (!ASSERT_OK_PTR(sample, "ring_buffer_iter_next"))
+ return;
+ ASSERT_EQ(sample_size, sizeof(*sample), "sample_size");
+ ASSERT_EQ(sample->value, SAMPLE_VALUE, "sample_value");
+ ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_iter_held");
+
+ ring_buffer_iter_destroy(&iter);
+ ASSERT_GT(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_released");
+
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new_remaining"))
+ return;
+ consumer_pos = ring__consumer_pos(ring);
+ for (i = 0; (sample = ring_buffer_iter_next(&iter, &sample_size, true));
+ i++) {
+ ASSERT_EQ(sample_size, sizeof(*sample),
+ "remaining_sample_size");
+ ASSERT_EQ(sample->value, SAMPLE_VALUE,
+ "remaining_sample_value");
+ if (i)
+ ASSERT_GT(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_released_next");
+ consumer_pos = ring__consumer_pos(ring);
+ }
+ ASSERT_EQ(i, 2, "remaining_samples");
+ ASSERT_GT(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_released_null");
+
+ syscall(__NR_getpgid);
+ sample = ring_buffer_iter_next(&iter, &sample_size, true);
+ ASSERT_NULL(sample, "exhausted_iterator");
+ ring_buffer_iter_destroy(&iter);
+
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new_again"))
+ return;
+ sample = ring_buffer_iter_next(&iter, &sample_size, true);
+ if (!ASSERT_OK_PTR(sample, "ring_buffer_iter_next_again"))
+ return;
+ ASSERT_EQ(sample->value, SAMPLE_VALUE, "new_sample_value");
+ ring_buffer_iter_destroy(&iter);
+
+ for (i = 0; i < 4; i++) {
+ skel_n->bss->discard = i == 1;
+ syscall(__NR_getpgid);
+ }
+ skel_n->bss->discard = false;
+ consumer_pos = ring__consumer_pos(ring);
+ err = ring_buffer_iter_new(&iter, ring);
+ if (!ASSERT_OK(err, "ring_buffer_iter_new_batch"))
+ return;
+ for (i = 0; i < 2; i++) {
+ sample = ring_buffer_iter_next(&iter, &sample_size, false);
+ if (!ASSERT_OK_PTR(sample, "batch_sample"))
+ return;
+ ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
+ "consumer_pos_deferred");
+ }
+ sample = ring_buffer_iter_next(&iter, &sample_size, true);
+ if (!ASSERT_OK_PTR(sample, "batch_sync_sample"))
+ return;
+ ASSERT_EQ(ring__consumer_pos(ring),
+ ring__producer_pos(ring) - sizeof(*sample) -
+ BPF_RINGBUF_HDR_SZ,
+ "consumer_pos_batch_published");
+ sample = ring_buffer_iter_next(&iter, &sample_size, false);
+ ASSERT_NULL(sample, "batch_exhausted");
+ ASSERT_EQ(ring__consumer_pos(ring), ring__producer_pos(ring),
+ "consumer_pos_batch_exhausted");
+}
+
static int process_map_key_sample(void *ctx, void *data, size_t len)
{
struct test_ringbuf_map_key_lskel *skel_map_key = ctx;
@@ -809,6 +969,9 @@ static void ringbuf_overwrite_callback_subtest(void)
if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
return;
+ struct ring_buffer_iter iter
+ __attribute__((cleanup(ring_buffer_iter_destroy))) = {};
+
err = ring_buffer__consume_n(ringbuf, 0);
ASSERT_EQ(err, -EOPNOTSUPP, "ringbuf_consume_zero");
err = ring_buffer__consume(ringbuf);
@@ -819,6 +982,8 @@ static void ringbuf_overwrite_callback_subtest(void)
ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume_zero");
err = ring__consume(ring);
ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume");
+ err = ring_buffer_iter_new(&iter, ring);
+ ASSERT_EQ(err, -EOPNOTSUPP, "ring_buffer_iter_new");
}
static void ringbuf_overwrite_mode_subtest(void)
@@ -886,6 +1051,8 @@ void test_ringbuf(void)
ringbuf_subtest();
if (test__start_subtest("ringbuf_n"))
ringbuf_n_subtest();
+ if (test__start_subtest("ringbuf_iterator"))
+ ringbuf_iterator_subtest();
if (test__start_subtest("ringbuf_null_cb"))
ringbuf_null_cb_subtest();
if (test__start_subtest("ringbuf_wakeup"))
diff --git a/tools/testing/selftests/bpf/progs/test_ringbuf_n.c b/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
index a95d4f597175..1727a2c32812 100644
--- a/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
+++ b/tools/testing/selftests/bpf/progs/test_ringbuf_n.c
@@ -3,6 +3,7 @@
#include <linux/bpf.h>
#include <sched.h>
+#include <stdbool.h>
#include <unistd.h>
#include <bpf/bpf_helpers.h>
#include "bpf_misc.h"
@@ -23,6 +24,7 @@ struct {
int pid = 0;
__s64 value = 0;
+bool discard = false;
SEC("fentry/" SYS_PREFIX "sys_getpgid")
int test_ringbuf_n(void *ctx)
@@ -41,7 +43,10 @@ int test_ringbuf_n(void *ctx)
sample->value = value;
bpf_get_current_comm(sample->comm, sizeof(sample->comm));
- bpf_ringbuf_submit(sample, 0);
+ if (discard)
+ bpf_ringbuf_discard(sample, 0);
+ else
+ bpf_ringbuf_submit(sample, 0);
return 0;
}
--
2.56.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator
2026-11-20 20:33 ` [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator Tamir Duberstein
@ 2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 1 reply; 15+ messages in thread
From: bot+bpf-ci @ 2026-11-20 21:34 UTC (permalink / raw)
To: tamird, ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai,
andrea.righi, billy, void, ihor.solodrai
Cc: bpf, linux-kselftest, linux-kernel, awerner32, zeffron, andriin,
tamird, ast, andrii, daniel, martin.lau, eddyz87, yonghong.song,
mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 1929 bytes --]
> libbpf: ringbuf: Add a caller-owned iterator
TL;DR: The new ring_buffer_iter_new/next/destroy exports in libbpf.map may not
follow libbpf's double-underscore naming convention; they become ABI once
released, so should they be renamed?
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
[ ... ]
> @@ -479,4 +479,7 @@ LIBBPF_1.9.0 {
> bpf_program__load;
> bpf_program__load_strategy;
> bpf_program__set_load_strategy;
> + ring_buffer_iter_destroy;
> + ring_buffer_iter_new;
> + ring_buffer_iter_next;
> } LIBBPF_1.8.0;
Should these three names follow the libbpf naming convention for object
methods?
Documentation/bpf/libbpf/libbpf_naming_convention.rst says functions that
work with an object are built from the object name, a double underscore and
a part describing the purpose, as in bpf_object__open and ring_buffer__new.
The other ring buffer exports all use the separator:
ring_buffer__new, ring_buffer__poll, ring_buffer__consume,
ring_buffer__consume_n, ring_buffer__ring
ring__consume, ring__consume_n, ring__consumer_pos, ring__producer_pos,
ring__avail_data_size, ring__size, ring__map_fd
user_ring_buffer__new, user_ring_buffer__reserve, user_ring_buffer__submit
ring_buffer_iter_new(), ring_buffer_iter_next() and ring_buffer_iter_destroy()
look like the only ring buffer symbols without it, and the iterator is
created from a struct ring * while the existing per-ring operations use the
ring__ prefix.
Once a libbpf release ships these names in libbpf.map they become ABI.
Would ring_buffer_iter__new()/__next()/__destroy() or ring__iter_*() be a
better fit?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44815895103
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics
2026-11-20 20:33 ` [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics Tamir Duberstein
@ 2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 1 reply; 15+ messages in thread
From: bot+bpf-ci @ 2026-11-20 21:34 UTC (permalink / raw)
To: tamird, ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai,
andrea.righi, billy, void, ihor.solodrai
Cc: bpf, linux-kselftest, linux-kernel, awerner32, zeffron, andriin,
tamird, ast, andrii, daniel, martin.lau, eddyz87, yonghong.song,
mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 3232 bytes --]
> libbpf: ringbuf: Use compiler atomics
TL;DR: On arm64 builds targeting RCpc (ldapr), this patch may lose stlr->ldar
ordering, allowing a missed ring buffer wakeup until the later fence patch;
the changelog's "unchanged" claim looks wrong, which matters for bisection.
> diff --git a/tools/lib/bpf/ringbuf.c b/tools/lib/bpf/ringbuf.c
> --- a/tools/lib/bpf/ringbuf.c
> +++ b/tools/lib/bpf/ringbuf.c
[ ... ]
> @@ -270,13 +270,13 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
> if (n == 0)
> return 0;
>
> - cons_pos = smp_load_acquire(r->consumer_pos);
> + cons_pos = __atomic_load_n(r->consumer_pos, __ATOMIC_ACQUIRE);
> do {
> got_new_data = false;
> - prod_pos = smp_load_acquire(r->producer_pos);
> + prod_pos = __atomic_load_n(r->producer_pos, __ATOMIC_ACQUIRE);
[ ... ]
> @@ -290,14 +290,13 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
> cnt++;
> }
>
> - smp_store_release(r->consumer_pos, cons_pos);
> + __atomic_store_n(r->consumer_pos, cons_pos, __ATOMIC_RELEASE);
>
> if (cnt >= n)
> goto done;
The commit message says "Acquire and release ordering is unchanged". Is
that true on arm64?
The old tools helpers in tools/arch/arm64/include/asm/barrier.h are inline
asm, so smp_store_release() emits stlr and smp_load_acquire() emits ldar.
ldar is RCsc, so a store-release followed by a load-acquire cannot be
reordered.
With __atomic_load_n(..., __ATOMIC_ACQUIRE) the compiler may emit ldapr
(RCpc) when targeting FEAT_LRCPC, for example with -march=armv8.3-a or
later, or -mcpu for cores such as Neoverse N1. libbpf accepts these
through EXTRA_CFLAGS.
With RCpc, the release store of consumer_pos can then be reordered with
the following acquire load of producer_pos, or of the record header in
ringbuf_process_ring().
The consumer publishes cons_pos == X and then reads the old producer_pos
(or sees a BUSY header at X) and returns. Meanwhile the kernel producer
runs bpf_ringbuf_commit():
xchg(&hdr->len, new_len);
smp_load_acquire(&rb->consumer_pos);
It reads the old consumer position, decides cons_pos != rec_pos and skips
irq_work_queue(). The consumer then waits without a notification even
though a record is queued.
Before this commit the stlr->ldar ordering ruled that interleaving out on
arm64. After this commit it is possible on RCpc-targeted builds. On
x86_64, ppc, riscv and the generic fallback the window already existed, so
the new exposure is limited to arm64.
This is fixed later in the series by commit 2010ab86a493 ("libbpf: ringbuf:
Prevent missed wakeups"), which adds __atomic_thread_fence(__ATOMIC_SEQ_CST)
after each consumer_pos publication and before the next producer_pos load.
The arm64 exposure therefore exists only at this commit, which matters for
bisection.
Could the fence patch come first, or could this patch carry the fence?
Alternatively, could the changelog note that arm64 loses the stlr->ldar
(RCsc) ordering until the next patch?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44815895103
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups
2026-11-20 20:33 ` [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups Tamir Duberstein
@ 2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 1 reply; 15+ messages in thread
From: bot+bpf-ci @ 2026-11-20 21:34 UTC (permalink / raw)
To: tamird, ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai,
andrea.righi, billy, void, ihor.solodrai
Cc: bpf, linux-kselftest, linux-kernel, awerner32, zeffron, andriin,
tamird, ast, andrii, daniel, martin.lau, eddyz87, yonghong.song,
mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2206 bytes --]
> selftests/bpf: Refactor lifetimes and test wakeups
TL;DR:
- Commit message says what changed but not what was wrong with the old
lifetimes; maybe split the refactor from the new wakeup subtest
- Test nit: repeated tryjoin plus joined=true pairs could go in one helper
> selftests/bpf: Refactor lifetimes and test wakeups
>
> Use cleanup attributes for skeletons, managers, and file descriptors in
> the ring buffer tests. Pass skeletons and managers explicitly, and stop
> or cancel worker threads before joining them. Preserve the existing
> zero-bound, NULL-callback, and overwrite-map regression cases.
>
> Add edge-triggered readiness coverage that consumes one record per call
> to exercise the wakeup handshake across bounded returns.
>
> Signed-off-by: Tamir Duberstein <tamird@kernel.org>
This isn't a bug, but the changelog lists what was changed without saying
what was wrong with the old code.
Could it say what problem the lifetime rework addresses, for example the
poll thread outliving the ring buffer when ringbuf_subtest() returns early?
Also, could the lifetime refactor and the new ringbuf_wakeup subtest go in
separate patches?
[ ... ]
> @@ -323,12 +401,14 @@ static void ringbuf_subtest(void)
[ ... ]
> /* background poll should still be blocked */
> err = pthread_tryjoin_np(thread, (void **)&bg_ret);
> + if (!err)
> + thread_guard.joined = true;
> if (CHECK(err != EBUSY, "try_join", "err %d\n", err))
> - goto cleanup;
> + return;
This isn't a bug, but the same two lines, "if (!err) thread_guard.joined =
true;", follow each of the three pthread_tryjoin_np() calls in
ringbuf_subtest().
cleanup_thread() is only correct if every join site updates joined by hand,
and a later tryjoin without it would make the cleanup call pthread_join()
on an already joined thread.
Could the pthread_tryjoin_np() plus joined = true pairs go through one small
helper that takes the thread_guard, so the state is updated in one place?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44815895103
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics
2026-11-20 21:34 ` bot+bpf-ci
@ 2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 22:38 UTC (permalink / raw)
To: bot+bpf-ci
Cc: ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai, billy, void,
ihor.solodrai, bpf, linux-kselftest, linux-kernel, awerner32,
andriin, martin.lau, mason
> Could the fence patch come first, or could this patch carry the fence?
Combined in v4, with the arm64 RCpc ordering dependency
explained in the commit message.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator
2026-11-20 21:34 ` bot+bpf-ci
@ 2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 22:38 UTC (permalink / raw)
To: bot+bpf-ci
Cc: ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai, billy, void,
ihor.solodrai, bpf, linux-kselftest, linux-kernel, awerner32,
andriin, martin.lau, mason
> Should these three names follow the libbpf naming convention for object
> methods?
Renamed to ring_buffer_iter__new(), __next(), and __destroy() in v4.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups
2026-11-20 21:34 ` bot+bpf-ci
@ 2026-11-20 22:38 ` Tamir Duberstein
0 siblings, 0 replies; 15+ messages in thread
From: Tamir Duberstein @ 2026-11-20 22:38 UTC (permalink / raw)
To: bot+bpf-ci
Cc: ast, daniel, andrii, eddyz87, memxor, martin.lau, song,
yonghong.song, jolsa, emil, shuah, arighi, xukuohai, billy, void,
ihor.solodrai, bpf, linux-kselftest, linux-kernel, awerner32,
andriin, martin.lau, mason
> Could it say what problem the lifetime rework addresses,
Split in v4. The lifetime fix explains how an early return
could free the ring buffer while its polling thread was still running.
> [...]
> so the state is updated in one place?
Added try_join_thread() to update ownership after a successful try-join.
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2026-11-20 23:23 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-20 20:33 [PATCH bpf-next v3 0/8] bpf: Fix ring buffers and add a libbpf iterator Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 1/8] bpf: user_ringbuf: Handle position wrap Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 2/8] libbpf: ringbuf: Use compiler atomics Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 3/8] libbpf: ringbuf: Prevent missed wakeups Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 4/8] libbpf: ringbuf: Add a caller-owned iterator Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 5/8] libbpf: ringbuf: Simplify iterator implementation Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 6/8] selftests/bpf: Refactor lifetimes and test wakeups Tamir Duberstein
2026-11-20 21:34 ` bot+bpf-ci
2026-11-20 22:38 ` Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 7/8] selftests/bpf: Verify ringbuf callback semantics Tamir Duberstein
2026-11-20 20:33 ` [PATCH bpf-next v3 8/8] selftests/bpf: Test caller-owned ring iterators Tamir Duberstein
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®