[PATCH 0/5] perf/core: add AUX buffer kernel-consumer API

[PATCH 0/5] perf/core: add AUX buffer kernel-consumer API

[Kunwu Chan]

ARM SPE and other tracing PMUs write records directly to perf AUX
buffers.  Unlike ordinary sampling PMUs, they do not necessarily
generate a perf_event_overflow() callback for each record.  A kernel
consumer therefore needs to own and drain an AUX buffer rather than
rely on the overflow path.

Userspace obtains AUX storage through perf_event_open() and mmap().
Kernel consumers created through perf_event_create_kernel_counter()
currently have no interface to allocate and manage AUX buffers.

This series adds the minimal perf-core interfaces needed for that use
case.  It was previously posted as RFC [1]; this version incorporates
feedback from that posting and is intended for review toward merging.
Patches 1-2 add the AUX owner interface, lifecycle helpers, and
cursor/copy operations without exposing perf's internal page layout.
Patches 3-5 exercise the new interfaces with KUnit, a userspace
regression test, and a selftest script.  The first consumer is a DAMON
ARM SPE backend, but this series is independent of DAMON and adds no
consumer-specific policy to perf.

The API follows the existing non-overwrite AUX ring-buffer protocol.
Setup and release serialise buffer lifetime transitions through
event->mmap_mutex.  Cursor reads/writes use the same ordering rules as
the userspace mmap consumer, and copy validates ranges in the absolute
cursor domain before applying the ring mask.

Testing
=======

The series builds cleanly at each step.

arm64 (Kunpeng 920, 256 CPUs, SPEv1.1), Image built from this series
and booted on the hardware:

KUnit: pass:39 fail:0 skip:0.  Userspace regression (arm_spe_0): full
pass; the producer head advances with live SPE traffic.  Selftest
script: 92 passed, 0 failed, 2 skipped (both environment skips).
perf record with arm_spe_0 captures data; dmesg stays clean.

x86_64 (vmlinux build, qemu/kvm): KUnit pass:39 fail:0 skip:0, also
under KASAN + lockdep (PROVE_LOCKING) and 15 repeated boots with no
splat.  Userspace regression (Intel PT): full pass.  The selftest
script also runs from installed selftest locations.

Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>

Changes since RFC [1]:

- Serialize AUX teardown under event->mmap_mutex and fix lifetime
  handling around ring_buffer_attach().
- Improve KUnit cleanup, concurrent-release coverage, and buffer-full
  assertions.
- Extend userspace and selftest coverage, including environment
  handling fixes and installed selftest support.
- Clarify AUX lifecycle, ordering, and cursor validation
  documentation.

[1] https://lore.kernel.org/linux-perf-users/20260814144927.489172-1-kunwu.chan@linux.dev/

Kunwu Chan (3):
  perf/core: add AUX buffer ownership for kernel events
  perf/core: add AUX ring accessors for kernel consumers
  selftests/perf_events: add AUX kernel API selftest script

Lian Wang (2):
  perf/core: add KUnit tests for AUX kernel-consumer API
  selftests/perf_events: add userspace AUX regression test

 .../userspace-api/perf_ring_buffer.rst        |   62 +-
 include/linux/perf_event.h                    |   18 +
 kernel/events/Makefile                        |    1 +
 kernel/events/aux_kernel_test.c               | 1152 +++++++++++++++++
 kernel/events/core.c                          |  164 +++
 kernel/events/internal.h                      |    6 +
 kernel/events/ring_buffer.c                   |  211 ++-
 lib/Kconfig.debug                             |   14 +
 tools/testing/selftests/perf_events/Makefile  |    2 +
 .../selftests/perf_events/aux_kernel.sh       |  631 +++++++++
 .../perf_events/aux_kernel_usermode.c         |  627 +++++++++
 tools/testing/selftests/perf_events/config    |    2 +
 12 files changed, 2885 insertions(+), 5 deletions(-)
 create mode 100644 kernel/events/aux_kernel_test.c
 create mode 100755 tools/testing/selftests/perf_events/aux_kernel.sh
 create mode 100644 tools/testing/selftests/perf_events/aux_kernel_usermode.c


base-commit: 6350de8671b94afb7691d110f63bcda42f658a69
-- 
2.43.0

[PATCH 1/5] perf/core: add AUX buffer ownership for kernel events

[Kunwu Chan]

Add an in-kernel AUX owner reference and setup/release helpers for
kernel-created perf events that need an AUX buffer without a userspace
mmap.  The setup path validates that the event is a kernel event with
no parent, rejects non-power-of-two page counts and negative watermark
values, allocates the perf buffer and AUX pages, records the kernel
owner, and attaches the buffer under the same lock.

perf_event_release_aux() stops AUX writers, frees AUX storage, and
detaches the buffer, in that order, matching the AUX teardown ordering
of perf_mmap_close(): perf_pmu_output_stop() walks event->rb->event_list
and rb_free_aux() must run while the buffer is still referenced by the
event.  The single event->rb reference is dropped by
ring_buffer_attach(event, NULL) itself, so the release path does not
put it again.

Both functions run under event->mmap_mutex, the lock that already
serialises ring-buffer attach/detach transitions for an event:
perf_mmap(), perf_mmap_close(), _perf_event_set_output() and
_free_event() all hold it around ring_buffer_attach().  Concurrent
perf_event_release_aux() callers therefore go through the same
serialisation point, and a second release call, or a release of a
buffer this API does not own, is a no-op.

Keep aux_mmap_count dedicated to userspace mappings.
perf_aux_output_begin() accepts a writer while either a userspace or kernel
owner remains, preserving the existing teardown ordering.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 include/linux/perf_event.h  |  11 +++
 kernel/events/core.c        | 164 ++++++++++++++++++++++++++++++++++++
 kernel/events/internal.h    |   6 ++
 kernel/events/ring_buffer.c |  11 ++-
 4 files changed, 188 insertions(+), 4 deletions(-)

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 7797ce207555..78fd2ed11fcf 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1255,6 +1255,17 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr,
 				 perf_overflow_handler_t callback,
 				 void *context);
 
+/*
+ * AUX ring-buffer support for kernel-created events (no user mmap).
+ * perf_event_setup_aux() allocates the buffer the PMU writes into via
+ * perf_aux_output_begin()/perf_aux_output_end(); the paired
+ * perf_event_release_aux() must be called before
+ * perf_event_release_kernel().
+ */
+extern int perf_event_setup_aux(struct perf_event *event, int nr_pages,
+				long watermark);
+extern void perf_event_release_aux(struct perf_event *event);
+
 extern void perf_pmu_migrate_context(struct pmu *pmu,
 				     int src_cpu, int dst_cpu);
 extern int perf_event_read_local(struct perf_event *event, u64 *value,
diff --git a/kernel/events/core.c b/kernel/events/core.c
index de05df65ab3d..966e74645cc9 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7139,6 +7139,170 @@ static void perf_mmap_close(struct vm_area_struct *vma)
 	ring_buffer_put(rb); /* could be last */
 }
 
+/*
+ * perf_event_setup_aux()/perf_event_release_aux() - AUX buffer support for
+ * kernel-created perf events (perf_event_create_kernel_counter()).
+ *
+ * perf_mmap()/perf_mmap_close() build and tear down AUX buffers for
+ * user-space events; kernel consumers (e.g. DAMON's ARM SPE backend) have no
+ * mmap, so they need this symmetric pair to get a buffer the PMU can write
+ * into via perf_aux_output_begin()/perf_aux_output_end().
+ *
+ * event->mmap_mutex is not an mmap-specific lock here: it is the lock that
+ * already serialises ring-buffer attach/detach transitions for an event
+ * (perf_mmap(), perf_mmap_close(), _perf_event_set_output() and
+ * _free_event() all hold it around ring_buffer_attach()).  Both functions
+ * below use it so that the attach and detach they perform are atomic with
+ * respect to those paths and to each other.
+ *
+ * The buffer holds an AUX owner reference (aux_kernel_count = 1) so
+ * perf_aux_output_begin() admits writers and so it is torn down here
+ * rather than by perf_mmap_close().  perf_event_release_aux() also detaches
+ * the event's normal ring-buffer reference before the event is released.
+ */
+
+/**
+ * perf_event_setup_aux() - Allocate an AUX ring buffer for an event.
+ * @event:	Kernel-created perf event (must not have an rb yet).
+ * @nr_pages:	AUX buffer size in pages; power of two, >= 1.
+ * @watermark:	AUX watermark; 0 selects the perf default (half the buffer).
+ *
+ * The buffer is non-overwrite streaming (RING_BUFFER_WRITABLE): the PMU
+ * pauses when the buffer is full until the consumer advances the tail.
+ * Must be called before the event is enabled.  The PMU's ->setup_aux()
+ * callback (e.g. arm_spe_pmu_setup_aux) validates the size and maps the
+ * pages for hardware writes.  The ring buffer is allocated with zero
+ * data pages, so PERF_RECORD_AUX events are not recorded; this is
+ * intentional for kernel consumers that drain trace data directly.
+ * Returns 0 on success, -errno otherwise.
+ */
+int perf_event_setup_aux(struct perf_event *event, int nr_pages,
+			 long watermark)
+{
+	struct perf_buffer *rb;
+	int ret;
+
+	if (!is_kernel_event(event) || event->parent ||
+	    !is_power_of_2(nr_pages) || watermark < 0)
+		return -EINVAL;
+
+	mutex_lock(&event->mmap_mutex);
+	if (event->rb) {
+		ret = -EBUSY;
+		goto out_unlock;
+	}
+
+	rb = rb_alloc(0, 0, event->cpu, 0);
+	if (!rb) {
+		ret = -ENOMEM;
+		goto out_unlock;
+	}
+
+	/* AUX area sits right after the user (control) page. */
+	ret = rb_alloc_aux(rb, event, 1, nr_pages, watermark,
+			   RING_BUFFER_WRITABLE);
+	if (ret)
+		goto err_put;
+
+	/*
+	 * No user-space mmap exists for this buffer; record an in-kernel
+	 * AUX owner (aux_kernel_count) instead so perf_aux_output_begin()
+	 * admits writers and perf_event_release_aux() performs the teardown
+	 * that perf_mmap_close() would otherwise do.
+	 */
+	refcount_set(&rb->aux_kernel_count, 1);
+
+	/* Transfer the allocation reference to the event. */
+	ring_buffer_attach(event, rb);
+	mutex_unlock(&event->mmap_mutex);
+
+	return 0;
+
+err_put:
+	ring_buffer_put(rb);
+out_unlock:
+	mutex_unlock(&event->mmap_mutex);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_event_setup_aux);
+
+/**
+ * perf_event_release_aux() - Tear down an AUX buffer for an event.
+ * @event:	Event with an AUX buffer allocated by perf_event_setup_aux().
+ *
+ * Stops any active AUX writers, frees the AUX pages, and detaches the ring
+ * buffer from the event.  Must be called before perf_event_release_kernel().
+ * Safe to call with a missing rb (no-op); a second call is also a no-op.
+ * A ring buffer not allocated by perf_event_setup_aux() is left attached.
+ *
+ * The whole teardown runs under event->mmap_mutex, the lock that already
+ * serialises ring-buffer attach/detach transitions for this event
+ * (perf_mmap(), perf_mmap_close() and _perf_event_set_output() hold it
+ * around ring_buffer_attach()): concurrent callers on the same event
+ * serialise here, so exactly one of them drops the single
+ * aux_kernel_count owner reference and performs the teardown, and the
+ * others then observe event->rb == NULL and return.  Holding it also
+ * keeps event->rb valid across perf_pmu_output_stop(), which walks
+ * event->rb->event_list, until the detach is done.
+ *
+ * The ordering matches the AUX teardown of perf_mmap_close(): the PMU
+ * output is stopped and the AUX pages freed before the buffer is
+ * detached from the event, because perf_pmu_output_stop() walks
+ * event->rb->event_list and rb_free_aux() must run while the buffer is
+ * still referenced by the event.
+ *
+ * The caller must quiesce AUX consumers before releasing the buffer;
+ * active AUX references at release time indicate a violated lifetime
+ * contract (see the aux_refcount WARN below).
+ */
+void perf_event_release_aux(struct perf_event *event)
+{
+	struct perf_buffer *rb;
+
+	if (!is_kernel_event(event) || event->parent)
+		return;
+
+	mutex_lock(&event->mmap_mutex);
+
+	rb = event->rb;
+	if (!rb)
+		goto out_unlock;
+
+	/* Only tear down a ring buffer that this API owns. */
+	if (!rb_has_aux(rb) || !refcount_read(&rb->aux_kernel_count))
+		goto out_unlock;
+
+	/*
+	 * Drop the kernel AUX-owner reference; writers are refused once it
+	 * reaches zero (perf_aux_output_begin()).  Since concurrent
+	 * releases serialise on mmap_mutex and setup rejects a second
+	 * buffer with -EBUSY, the count can only be 1 here.
+	 */
+	WARN_ON_ONCE(!refcount_dec_and_test(&rb->aux_kernel_count));
+
+	/*
+	 * Stop all AUX events writing to this buffer so the pages can be
+	 * freed; the aux_mutex serialisation matches perf_mmap_close().
+	 */
+	mutex_lock(&rb->aux_mutex);
+	perf_pmu_output_stop(event);
+	rb_free_aux(rb);
+	WARN_ON_ONCE(refcount_read(&rb->aux_refcount));
+	mutex_unlock(&rb->aux_mutex);
+
+	/*
+	 * Detach the buffer from the event.  ring_buffer_attach(event, NULL)
+	 * drops the event->rb reference itself (the allocation reference
+	 * transferred by perf_event_setup_aux()), so no extra
+	 * ring_buffer_put() is needed here.
+	 */
+	ring_buffer_attach(event, NULL);
+
+out_unlock:
+	mutex_unlock(&event->mmap_mutex);
+}
+EXPORT_SYMBOL_GPL(perf_event_release_aux);
+
 static vm_fault_t perf_mmap_pfn_mkwrite(struct vm_fault *vmf)
 {
 	/* The first page is the user control page, others are read-only. */
diff --git a/kernel/events/internal.h b/kernel/events/internal.h
index c03c4f2eea57..760b7659ad87 100644
--- a/kernel/events/internal.h
+++ b/kernel/events/internal.h
@@ -48,6 +48,12 @@ struct perf_buffer {
 	int				aux_nr_pages;
 	int				aux_overwrite;
 	refcount_t			aux_mmap_count;
+	/*
+	 * In-kernel AUX owner reference, set by perf_event_setup_aux():
+	 * admits writers and ties the AUX teardown to
+	 * perf_event_release_aux() when no userspace mmap holds the buffer.
+	 */
+	refcount_t			aux_kernel_count;
 	unsigned long			aux_mmap_locked;
 	void				(*free_aux)(void *);
 	refcount_t			aux_refcount;
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index 1b1ffe0533e5..eae28e42346c 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -395,14 +395,17 @@ void *perf_aux_output_begin(struct perf_output_handle *handle,
 		goto err;
 
 	/*
-	 * If aux_mmap_count is zero, the aux buffer is in perf_mmap_close(),
-	 * about to get freed, so we leave immediately.
+	 * If no AUX owner remains, the buffer is in perf_mmap_close() or
+	 * perf_event_release_aux(), about to get freed, so we leave
+	 * immediately.  aux_mmap_count tracks user-space mmap owners;
+	 * aux_kernel_count tracks in-kernel owners (perf_event_setup_aux()).
 	 *
-	 * Checking rb::aux_mmap_count and rb::refcount has to be done in
+	 * Checking the AUX owner counts and rb::refcount has to be done in
 	 * the same order, see perf_mmap_close. Otherwise we end up freeing
 	 * aux pages in this path, which is a bug, because in_atomic().
 	 */
-	if (!refcount_read(&rb->aux_mmap_count))
+	if (!refcount_read(&rb->aux_mmap_count) &&
+	    !refcount_read(&rb->aux_kernel_count))
 		goto err;
 
 	if (!refcount_inc_not_zero(&rb->aux_refcount))
-- 
2.43.0

[PATCH 2/5] perf/core: add AUX ring accessors for kernel consumers

[Kunwu Chan]

Add kernel-consumer accessors for the published AUX producer head, the
consumer tail, and copying a possibly wrapped AUX interval without exposing
perf's internal page array.

Use the same memory ordering as mmap consumers: order data reads after the
published head with smp_rmb() and order completed data reads before
advancing the tail with smp_mb().  Validate cursor distances in the
absolute cursor domain before applying the ring mask, so rewound,
already-consumed, and future windows are rejected even across unsigned
cursor wrap.

Hold an AUX reference while copying so storage cannot disappear under the
consumer.  Restrict the helpers to buffers owned by the kernel AUX setup
API.  Document the API in
Documentation/userspace-api/perf_ring_buffer.rst.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 .../userspace-api/perf_ring_buffer.rst        |  62 +++++-
 include/linux/perf_event.h                    |   7 +
 kernel/events/ring_buffer.c                   | 202 +++++++++++++++++-
 3 files changed, 269 insertions(+), 2 deletions(-)

diff --git a/Documentation/userspace-api/perf_ring_buffer.rst b/Documentation/userspace-api/perf_ring_buffer.rst
index dc71544532ce..206c57ee6c35 100644
--- a/Documentation/userspace-api/perf_ring_buffer.rst
+++ b/Documentation/userspace-api/perf_ring_buffer.rst
@@ -26,6 +26,7 @@ Perf ring buffer
     3.1  The relationship between AUX and regular ring buffers
     3.2  AUX events
     3.3  Snapshot mode
+    3.4  Kernel-consumer AUX buffer access
 
 
 1. Introduction
@@ -827,4 +828,63 @@ mode.
          |                 AUX Ring buffer 3     | <- aux_head
          +---------------------------------------+
 
-                Figure 9. Snapshot with system wide mode
+                 Figure 9. Snapshot with system wide mode
+
+3.4 Kernel-consumer AUX buffer access
+-------------------------------------
+
+The AUX ring buffer is normally consumed from user space via mmap()
+on the perf event fd.  Some tracing PMUs (e.g. ARM SPE) write trace
+records directly to the AUX buffer without generating a
+perf_event_overflow() callback for each record.  A kernel consumer
+therefore needs to own and drain the AUX buffer itself rather than
+rely on the overflow path.
+
+The perf core provides five exported functions for in-kernel
+consumers that do not have a user-space mmap:
+
+  - ``perf_event_setup_aux(event, nr_pages, watermark)`` — allocate
+    an AUX ring buffer for a kernel-created perf event.  The event
+    must be created by ``perf_event_create_kernel_counter()``; it
+    must not have a parent or an existing ring buffer.  ``nr_pages``
+    must be a power of two and ``watermark`` must be non-negative
+    (0 selects half the buffer).
+
+  - ``perf_event_release_aux(event)`` — tear down the AUX buffer
+    allocated by ``perf_event_setup_aux()``.  Must be called before
+    ``perf_event_release_kernel()``.  Safe to call on an event that
+    never had an AUX buffer, and on one whose buffer was already
+    released (no-op).  A ring buffer not owned by the kernel AUX API
+    is left attached.  The whole teardown is serialised on the
+    event's mmap mutex, so concurrent callers on the same event are
+    safe: one performs the teardown and the others return.
+
+  - ``perf_event_aux_head(event)`` — read the published producer
+    head.  Returns 0 if the event has no ring buffer.
+
+  - ``perf_event_aux_tail_set(event, tail)`` — advance the consumer
+    tail.  The new tail is an absolute cursor: it is accepted only
+    within the current produced window, not ahead of the published
+    producer cursor and no more than one buffer size behind it; an
+    out-of-range tail is rejected with ``-EINVAL``.
+
+  - ``perf_event_aux_copy(event, from, to, buf)`` — copy a possibly
+    wrapped AUX interval into a linear buffer.  The ``from``/``to``
+    cursors are absolute and must lie between the consumer cursor
+    (``tail``) and the published producer cursor (``head``).
+
+The owner reference is tracked by ``aux_kernel_count`` on the
+``perf_buffer``, separate from the userspace ``aux_mmap_count``.
+``perf_aux_output_begin()`` admits a writer while either owner
+count is non-zero, so a kernel consumer and a userspace consumer
+on different events for the same PMU do not interfere.
+
+A userspace mmap and a kernel ``perf_event_setup_aux()`` on the
+*same* event cannot coexist: the second call finds ``event->rb``
+already set and returns ``-EBUSY``.
+
+Memory ordering follows the same protocol as the userspace AUX
+mmap consumer: ``perf_event_aux_head()`` uses ``smp_rmb()`` to pair
+with the producer's data-write barrier before publishing ``aux_head``,
+and ``perf_event_aux_tail_set()`` uses ``smp_mb()`` to order prior
+data reads before advancing ``aux_tail``.
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 78fd2ed11fcf..0d96145f75ed 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1266,6 +1266,13 @@ extern int perf_event_setup_aux(struct perf_event *event, int nr_pages,
 				long watermark);
 extern void perf_event_release_aux(struct perf_event *event);
 
+/* AUX ring accessors for kernel consumers (no user-space mmap). */
+extern unsigned long perf_event_aux_head(struct perf_event *event);
+extern int perf_event_aux_tail_set(struct perf_event *event,
+				   unsigned long tail);
+extern long perf_event_aux_copy(struct perf_event *event, unsigned long from,
+				unsigned long to, void *buf);
+
 extern void perf_pmu_migrate_context(struct pmu *pmu,
 				     int src_cpu, int dst_cpu);
 extern int perf_event_read_local(struct perf_event *event, u64 *value,
diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
index eae28e42346c..2c6f260fcf91 100644
--- a/kernel/events/ring_buffer.c
+++ b/kernel/events/ring_buffer.c
@@ -398,7 +398,8 @@ void *perf_aux_output_begin(struct perf_output_handle *handle,
 	 * If no AUX owner remains, the buffer is in perf_mmap_close() or
 	 * perf_event_release_aux(), about to get freed, so we leave
 	 * immediately.  aux_mmap_count tracks user-space mmap owners;
-	 * aux_kernel_count tracks in-kernel owners (perf_event_setup_aux()).
+	 * aux_kernel_count tracks the in-kernel AUX owner
+	 * (perf_event_setup_aux()).
 	 *
 	 * Checking the AUX owner counts and rb::refcount has to be done in
 	 * the same order, see perf_mmap_close. Otherwise we end up freeing
@@ -585,6 +586,205 @@ void *perf_get_aux(struct perf_output_handle *handle)
 }
 EXPORT_SYMBOL_GPL(perf_get_aux);
 
+/*
+ * perf_event_aux_head()/perf_event_aux_tail_set()/perf_event_aux_copy() -
+ * AUX ring accessors for kernel consumers (e.g. DAMON's ARM SPE backend).
+ *
+ * The write cursor rb->aux_head is maintained by the PMU driver via
+ * perf_aux_output_end(), which also publishes it to user_page->aux_head;
+ * the consumer cursor lives in user_page->aux_tail (same absolute cursor
+ * domain).  The kernel consumer has no mmap, so these are the counterpart
+ * of the user-space mmap protocol.
+ *
+ * As in the user-space protocol, data visibility is the producer's duty:
+ * the PMU driver must make AUX data visible before calling
+ * perf_aux_output_end(), which then publishes the new head.  Kernel
+ * consumers use the same read- and full-barrier ordering as mmap consumers.
+ */
+
+static bool rb_has_kernel_aux(struct perf_buffer *rb)
+{
+	return rb_has_aux(rb) && refcount_read(&rb->aux_kernel_count);
+}
+
+/**
+ * perf_event_aux_head() - Return the event's absolute AUX write cursor.
+ * @event:	Event with an AUX buffer (perf_event_setup_aux()).
+ *
+ * Returns the current rb->aux_head, or 0 if the event has no ring buffer.
+ */
+unsigned long perf_event_aux_head(struct perf_event *event)
+{
+	struct perf_buffer *rb = ring_buffer_get(event);
+	unsigned long head = 0;
+
+	if (rb && rb_has_kernel_aux(rb)) {
+		head = READ_ONCE(rb->user_page->aux_head);
+		/* Pairs with the producer's AUX-data write barrier. */
+		smp_rmb();
+	}
+	if (rb)
+		ring_buffer_put(rb);
+	return head;
+}
+EXPORT_SYMBOL_GPL(perf_event_aux_head);
+
+/**
+ * perf_event_aux_tail_set() - Advance the event's AUX consumer cursor.
+ * @event:	Event with an AUX buffer.
+ * @tail:	New absolute consumer cursor.
+ *
+ * Frees the consumed space so perf_aux_output_begin() can compute space
+ * again for the PMU writer.  @tail is an absolute cursor: it is accepted
+ * only within the current produced window, i.e. not ahead of the
+ * published producer cursor and no more than one buffer size behind it;
+ * a tail outside that window would corrupt the free-space computation
+ * in perf_aux_output_begin() and let the producer overwrite unconsumed
+ * data.
+ *
+ * If a non-overwrite buffer becomes full, the producer may be stopped;
+ * advancing the tail alone does not resume it, and the consumer is
+ * responsible for re-enabling the event if needed (as user-space AUX
+ * consumers do).
+ *
+ * Returns 0 on success, -ENOENT if the event has no ring buffer, -EINVAL
+ * on an out-of-range tail.
+ */
+int perf_event_aux_tail_set(struct perf_event *event, unsigned long tail)
+{
+	struct perf_buffer *rb = ring_buffer_get(event);
+	unsigned long advance, aux_size, head, old_tail;
+	int ret = -EINVAL;
+
+	if (!rb)
+		return -ENOENT;
+
+	if (!rb_has_kernel_aux(rb)) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	aux_size = (unsigned long)rb->aux_nr_pages << PAGE_SHIFT;
+	old_tail = READ_ONCE(rb->user_page->aux_tail);
+	/*
+	 * Pairs with the producer's AUX-data write barrier in
+	 * perf_aux_output_end() before it publishes aux_head.
+	 */
+	smp_rmb();
+	head = READ_ONCE(rb->user_page->aux_head);
+	advance = tail - old_tail;
+
+	/* Modular distances keep the check valid when a cursor wraps. */
+	if (head - old_tail <= aux_size && advance <= head - old_tail) {
+		/* Order all prior AUX data reads before releasing the space. */
+		smp_mb();
+		WRITE_ONCE(rb->user_page->aux_tail, tail);
+		ret = 0;
+	}
+
+out:
+	ring_buffer_put(rb);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_event_aux_tail_set);
+
+/**
+ * perf_event_aux_copy() - Copy an AUX window into a linear buffer.
+ * @event:	Event with an AUX buffer.
+ * @from:	Absolute start cursor (inclusive).
+ * @to:		Absolute end cursor (exclusive).
+ * @buf:	Destination; must hold (to - from) bytes.
+ *
+ * Handles wrap-around within the ring.  Returns the number of bytes
+ * copied, or -errno.  The requested window is validated in the
+ * absolute cursor domain -- between the consumer cursor and the
+ * published producer cursor, and not larger than the ring -- before
+ * the cursors are converted into ring offsets.
+ */
+long perf_event_aux_copy(struct perf_event *event, unsigned long from,
+			 unsigned long to, void *buf)
+{
+	struct perf_buffer *rb = ring_buffer_get(event);
+	unsigned long aux_size, available, head, len, start, tail, tocopy;
+	long ret;
+
+	if (!rb)
+		return -ENOENT;
+
+	if (!rb_has_kernel_aux(rb)) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	/*
+	 * The AUX pages have a lifetime of their own, governed by
+	 * aux_refcount (see rb_alloc_aux): producers and consumers can both
+	 * hold references, and a concurrent perf_event_release_aux() drops
+	 * the owner's.  Take one for the duration of the copy.
+	 */
+	if (!refcount_inc_not_zero(&rb->aux_refcount)) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	if (!buf) {
+		ret = -EINVAL;
+		goto out_aux;
+	}
+
+	aux_size = (unsigned long)rb->aux_nr_pages << PAGE_SHIFT;
+	tail = READ_ONCE(rb->user_page->aux_tail);
+	head = READ_ONCE(rb->user_page->aux_head);
+	/* Pairs with the producer's AUX-data write barrier. */
+	smp_rmb();
+	available = head - tail;
+	start = from - tail;
+	len = to - from;
+
+	/*
+	 * Validate modular distances before masking: the requested window
+	 * must be wholly contained between the consumer cursor and the
+	 * published producer cursor.  This rejects rewound, already
+	 * consumed, and future cursors while remaining correct across
+	 * unsigned cursor wrap.
+	 */
+	if (available > aux_size || start > available ||
+	    len > available - start) {
+		ret = -EINVAL;
+		goto out_aux;
+	}
+	if (!len) {
+		ret = 0;
+		goto out_aux;
+	}
+
+	from &= aux_size - 1;
+	to &= aux_size - 1;
+	ret = 0;
+
+	do {
+		tocopy = PAGE_SIZE - offset_in_page(from);
+		if (to > from)
+			tocopy = min(tocopy, to - from);
+		if (!tocopy)
+			break;
+
+		memcpy(buf + ret, rb->aux_pages[from >> PAGE_SHIFT] +
+		       offset_in_page(from), tocopy);
+
+		ret += tocopy;
+		from += tocopy;
+		from &= aux_size - 1;
+	} while (to != from);
+
+out_aux:
+	rb_free_aux(rb);
+out:
+	ring_buffer_put(rb);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_event_aux_copy);
+
 /*
  * Copy out AUX data from an AUX handle.
  */
-- 
2.43.0

[PATCH 3/5] perf/core: add KUnit tests for AUX kernel-consumer API

[Kunwu Chan]

From: Lian Wang <lianux.mm@gmail.com>

Add a KUnit test suite (perf_aux_kernel) for the perf_event_setup_aux(),
perf_event_release_aux(), perf_event_aux_head(), perf_event_aux_tail_set()
and perf_event_aux_copy() functions.

A self-contained dummy AUX PMU is registered in suite_init so that
every test is deterministic and needs no hardware.  The dummy
setup_aux pre-fills all pages with a recognisable pattern (0xAB);
aux_test_produce() advances the published head via
perf_aux_output_begin/end, and copy tests verify the pattern.

39 test cases cover setup/release lifecycle, writer admission and
blocking, head/tail/copy accessor contract validation (including
wrap-around, boundary cursors, and data correctness), buffer-full
behaviour, negative watermark rejection, independence of two
events on the same PMU, user/kernel coexistence simulation, and
concurrent release on the same event.

Every test event is released through a kunit_add_action() cleanup, so
a test aborted by a KUNIT_ASSERT_*() (which skips the remaining
manual cleanup) still releases its events.  The concurrent-release
test waits on a struct completion instead of a lockless
wait_event() condition, and does not kthread_stop() workers that exit
on their own.  The buffer-full test asserts that perf_aux_output_begin()
refuses a full non-overwrite buffer.

Gated by CONFIG_PERF_AUX_KERNEL_KUNIT_TEST.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
---
 kernel/events/Makefile          |    1 +
 kernel/events/aux_kernel_test.c | 1152 +++++++++++++++++++++++++++++++
 lib/Kconfig.debug               |   14 +
 3 files changed, 1167 insertions(+)
 create mode 100644 kernel/events/aux_kernel_test.c

diff --git a/kernel/events/Makefile b/kernel/events/Makefile
index 91a62f566743..592604db2c76 100644
--- a/kernel/events/Makefile
+++ b/kernel/events/Makefile
@@ -3,4 +3,5 @@ obj-y := core.o ring_buffer.o callchain.o
 
 obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
 obj-$(CONFIG_HW_BREAKPOINT_KUNIT_TEST) += hw_breakpoint_test.o
+obj-$(CONFIG_PERF_AUX_KERNEL_KUNIT_TEST) += aux_kernel_test.o
 obj-$(CONFIG_UPROBES) += uprobes.o
diff --git a/kernel/events/aux_kernel_test.c b/kernel/events/aux_kernel_test.c
new file mode 100644
index 000000000000..fa481b8015db
--- /dev/null
+++ b/kernel/events/aux_kernel_test.c
@@ -0,0 +1,1152 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit tests for the perf AUX kernel-consumer API.
+ *
+ * Tests the exported functions:
+ *   - perf_event_setup_aux() / perf_event_release_aux()
+ *   - perf_event_aux_head() / perf_event_aux_tail_set() / perf_event_aux_copy()
+ *
+ * A self-contained dummy AUX PMU is registered in suite_init so that
+ * every test is deterministic and requires no hardware.  The dummy
+ * setup_aux pre-fills all pages with a recognisable pattern (0xAB);
+ * aux_test_produce() advances the published head via
+ * perf_aux_output_begin/end, and copy tests verify the pattern.
+ *
+ * Every event created by aux_test_create_event() is released through a
+ * kunit_add_action() cleanup, so a test aborted by a KUNIT_ASSERT_*()
+ * (which skips the remaining manual cleanup) still releases its events.
+ *
+ * Run via debugfs:
+ *   echo run > /sys/kernel/debug/kunit/perf_aux_kernel/run
+ *   cat /sys/kernel/debug/kunit/perf_aux_kernel/results
+ */
+
+#include <kunit/test.h>
+#include <linux/completion.h>
+#include <linux/kthread.h>
+#include <linux/perf_event.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+#include <linux/string.h>
+#include "internal.h"
+
+/* ---- Dummy AUX PMU ---- */
+
+static struct pmu dummy_aux_pmu;
+static int dummy_aux_pmu_type = -1;
+
+static int dummy_aux_event_init(struct perf_event *event)
+{
+	return 0;
+}
+
+static void dummy_aux_event_read(struct perf_event *event) { }
+
+static int dummy_aux_event_add(struct perf_event *event, int mode)
+{
+	return 0;
+}
+
+static void dummy_aux_event_del(struct perf_event *event, int mode) { }
+
+static void dummy_aux_event_start(struct perf_event *event, int mode) { }
+
+static void dummy_aux_event_stop(struct perf_event *event, int mode) { }
+
+static void *dummy_aux_setup_aux(struct perf_event *event, void **pages,
+				 int nr_pages, bool overwrite)
+{
+	int i;
+
+	for (i = 0; i < nr_pages; i++)
+		memset(pages[i], 0xAB, PAGE_SIZE);
+
+	return pages;
+}
+
+static void dummy_aux_free_aux(void *priv)
+{
+	/* aux_priv is the pages array; __rb_free_aux frees it via kfree. */
+}
+
+static int dummy_aux_pmu_register(void)
+{
+	int ret;
+
+	if (dummy_aux_pmu_type >= 0)
+		return 0;
+
+	memset(&dummy_aux_pmu, 0, sizeof(dummy_aux_pmu));
+	dummy_aux_pmu.event_init = dummy_aux_event_init;
+	dummy_aux_pmu.add = dummy_aux_event_add;
+	dummy_aux_pmu.del = dummy_aux_event_del;
+	dummy_aux_pmu.start = dummy_aux_event_start;
+	dummy_aux_pmu.stop = dummy_aux_event_stop;
+	dummy_aux_pmu.read = dummy_aux_event_read;
+	dummy_aux_pmu.setup_aux = dummy_aux_setup_aux;
+	dummy_aux_pmu.free_aux = dummy_aux_free_aux;
+	dummy_aux_pmu.capabilities = PERF_PMU_CAP_ITRACE;
+	dummy_aux_pmu.task_ctx_nr = perf_sw_context;
+
+	ret = perf_pmu_register(&dummy_aux_pmu, "dummy_aux", -1);
+	if (ret)
+		return ret;
+
+	dummy_aux_pmu_type = dummy_aux_pmu.type;
+	return 0;
+}
+
+static void dummy_aux_pmu_unregister(void)
+{
+	if (dummy_aux_pmu_type >= 0)
+		perf_pmu_unregister(&dummy_aux_pmu);
+	dummy_aux_pmu_type = -1;
+}
+
+static int suite_init(struct kunit_suite *suite)
+{
+	return dummy_aux_pmu_register();
+}
+
+static void suite_exit(struct kunit_suite *suite)
+{
+	dummy_aux_pmu_unregister();
+}
+
+/* ---- Helpers ---- */
+
+/*
+ * Deferred cleanup for the kernel events created by the tests.
+ *
+ * Registering this via kunit_add_action() makes every test abort-safe:
+ * KUNIT_ASSERT_* macros (and kunit_skip()) terminate the test by
+ * throwing, which skips the remaining manual cleanup, but KUnit still
+ * runs registered actions on the aborted test's exit.  A test that
+ * already called perf_event_release_aux() itself is not affected: a
+ * second release is a no-op, and the LIFO ordering of actions handles
+ * tests that create more than one event.
+ */
+static void aux_test_event_release(void *p)
+{
+	struct perf_event *event = p;
+
+	perf_event_release_aux(event);
+	perf_event_release_kernel(event);
+}
+
+static struct perf_event *aux_test_create_event(struct kunit *test)
+{
+	struct perf_event_attr attr = {};
+	struct perf_event *event;
+
+	attr.type = dummy_aux_pmu_type;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+
+	event = perf_event_create_kernel_counter(&attr, raw_smp_processor_id(),
+						 NULL, NULL, NULL);
+	if (IS_ERR(event))
+		return event;
+
+	if (kunit_add_action_or_reset(test, aux_test_event_release, event)) {
+		perf_event_release_kernel(event);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	return event;
+}
+
+/*
+ * Advance the AUX head by @bytes via perf_aux_output_begin/end.
+ * The pages are pre-filled with 0xAB in dummy_aux_setup_aux, so
+ * the data is already present; this function only publishes it.
+ */
+static unsigned long aux_test_produce(struct perf_event *event, unsigned long bytes)
+{
+	struct perf_output_handle handle;
+	unsigned long head, written = 0;
+
+	while (written < bytes) {
+		unsigned long chunk = min_t(unsigned long, bytes - written, 1024);
+
+		if (!perf_aux_output_begin(&handle, event))
+			break;
+		chunk = min_t(unsigned long, chunk, handle.size);
+		perf_aux_output_end(&handle, chunk);
+		written += chunk;
+	}
+
+	head = perf_event_aux_head(event);
+	return head;
+}
+
+/* ---- Patch-1: Setup / release tests ---- */
+
+static void test_setup_rejects_non_power_of_two(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 3, 0);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	ret = perf_event_setup_aux(event, 0, 0);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	ret = perf_event_setup_aux(event, 6, 0);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+}
+
+static void test_setup_accepts_power_of_two(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int i, ret;
+	int pages[] = { 1, 2, 4, 8, 16, 32, 64 };
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	for (i = 0; i < ARRAY_SIZE(pages); i++) {
+		ret = perf_event_setup_aux(event, pages[i], 0);
+		KUNIT_EXPECT_EQ_MSG(test, 0, ret,
+				    "setup_aux(%d pages) expected 0, got %d",
+				    pages[i], ret);
+		if (ret == 0)
+			perf_event_release_aux(event);
+	}
+}
+
+static void test_setup_rejects_double(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, -EBUSY, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_setup_parent_event_rejected(struct kunit *test)
+{
+	struct perf_event *parent = aux_test_create_event(test);
+	struct perf_event *child = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, parent);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, child);
+
+	child->parent = parent;
+
+	ret = perf_event_setup_aux(child, 1, 0);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	/* Restore before the deferred cleanup releases the events. */
+	child->parent = NULL;
+}
+
+static void test_writer_admitted_after_setup(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	perf_event_release_aux(event);
+}
+
+static void test_writer_blocked_after_release(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	struct perf_output_handle handle;
+	void *addr;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+
+	addr = perf_aux_output_begin(&handle, event);
+	KUNIT_EXPECT_NULL(test, addr);
+}
+
+static void test_release_noop_without_rb(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	perf_event_release_aux(event);
+	KUNIT_EXPECT_TRUE(test, true);
+}
+
+static void test_setup_release_roundtrip(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+	KUNIT_EXPECT_TRUE(test, true);
+}
+
+static void test_multi_setup_release_cycle(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int i, ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	for (i = 0; i < 3; i++) {
+		ret = perf_event_setup_aux(event, 1, 0);
+		KUNIT_ASSERT_EQ_MSG(test, 0, ret,
+				    "cycle %d: setup_aux failed", i);
+		perf_event_release_aux(event);
+	}
+}
+
+static void test_double_release_clean(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+	perf_event_release_aux(event);
+	KUNIT_EXPECT_TRUE(test, true);
+}
+
+/* ---- Patch-2: Accessor tests ---- */
+
+static void test_head_initial_zero(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = perf_event_aux_head(event);
+	KUNIT_EXPECT_EQ(test, 0, head);
+
+	perf_event_release_aux(event);
+}
+
+static void test_head_advances_after_produce(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_full_window(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = PAGE_SIZE;
+	void *buf;
+	unsigned long head;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, buf_size / 2);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	buf = kunit_kmalloc(test, head, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	copied = perf_event_aux_copy(event, 0, head, buf);
+	KUNIT_EXPECT_EQ_MSG(test, (long)head, copied,
+			    "copy(0, %lu) returned %ld, expected %lu",
+			    head, copied, head);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_wrap(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = 2 * PAGE_SIZE;
+	void *buf;
+	unsigned long head, tail;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 2, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	/* Fill the buffer to near-full */
+	head = aux_test_produce(event, buf_size);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	/* Consume half to free space */
+	tail = head / 2;
+	ret = perf_event_aux_tail_set(event, tail);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	/* Produce more — head now wraps past buf_size */
+	head = aux_test_produce(event, buf_size);
+	KUNIT_EXPECT_GT(test, head, buf_size);
+
+	/* Copy a window that spans the ring wrap point */
+	buf = kunit_kmalloc(test, buf_size, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	copied = perf_event_aux_copy(event, tail, head, buf);
+	KUNIT_EXPECT_GT(test, copied, 0);
+	KUNIT_EXPECT_LE(test, copied, (long)buf_size);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_zero_len(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	void *buf;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	buf = kunit_kmalloc(test, 1, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	copied = perf_event_aux_copy(event, 0, 0, buf);
+	KUNIT_EXPECT_EQ(test, 0, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_rejects_rewound(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = PAGE_SIZE;
+	void *buf;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	aux_test_produce(event, buf_size / 2);
+
+	buf = kunit_kmalloc(test, 64, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	/* Reversed cursor range. */
+	copied = perf_event_aux_copy(event, 100, 50, buf);
+	KUNIT_EXPECT_EQ(test, -EINVAL, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_rejects_future(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	void *buf;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	buf = kunit_kmalloc(test, 64, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	/* to > head: future cursor */
+	copied = perf_event_aux_copy(event, head, head + 64, buf);
+	KUNIT_EXPECT_EQ(test, -EINVAL, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_rejects_oversized(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = PAGE_SIZE;
+	void *buf;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	aux_test_produce(event, 512);
+
+	buf = kunit_kmalloc(test, PAGE_SIZE, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	/* Window extends past the published producer cursor. */
+	copied = perf_event_aux_copy(event, 0, 2 * buf_size, buf);
+	KUNIT_EXPECT_EQ(test, -EINVAL, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_rejects_null_buf(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	copied = perf_event_aux_copy(event, 0, 0, NULL);
+	KUNIT_EXPECT_EQ(test, -EINVAL, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_no_rb_enoent(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	void *buf;
+	long ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	buf = kunit_kmalloc(test, 64, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	ret = perf_event_aux_copy(event, 0, 64, buf);
+	KUNIT_EXPECT_EQ(test, -ENOENT, ret);
+}
+
+static void test_tail_set_valid_frees_space(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	ret = perf_event_aux_tail_set(event, head / 2);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_tail_set_rejects_future(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	/* tail=1 when head=0: future */
+	ret = perf_event_aux_tail_set(event, 1);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_tail_set_accepts_within_ring_window(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = PAGE_SIZE;
+	unsigned long head, tail;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	/* Produce enough data so head - tail <= aux_size for a large tail */
+	head = aux_test_produce(event, buf_size / 2);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	/*
+	 * Set tail within the current [old_tail, head] window.
+	 */
+	tail = head / 4;
+	ret = perf_event_aux_tail_set(event, tail);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_tail_set_rejects_behind_ring(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	const unsigned long buf_size = PAGE_SIZE;
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, buf_size / 2);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	/*
+	 * Cursor outside the current producer window.  The requested
+	 * tail is not between head - aux_size and head, so the
+	 * modular-distance check must reject it.
+	 */
+	ret = perf_event_aux_tail_set(event, head + buf_size + 1);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_tail_set_no_rb_enoent(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_aux_tail_set(event, 0);
+	KUNIT_EXPECT_EQ(test, -ENOENT, ret);
+}
+
+static void test_head_no_rb_zero(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	head = perf_event_aux_head(event);
+	KUNIT_EXPECT_EQ(test, 0, head);
+}
+
+/* ---- Additional boundary and correctness tests ---- */
+
+static void test_setup_rejects_negative_watermark(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, -1);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+
+	ret = perf_event_setup_aux(event, 1, -4096);
+	KUNIT_EXPECT_EQ(test, -EINVAL, ret);
+}
+
+static void test_tail_set_noop(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	ret = perf_event_aux_tail_set(event, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_tail_set_consume_all(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	unsigned long head;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	ret = perf_event_aux_tail_set(event, head);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_data_correctness(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	void *buf;
+	unsigned long head;
+	long copied;
+	int ret, i;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	buf = kunit_kmalloc(test, head, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	copied = perf_event_aux_copy(event, 0, head, buf);
+	KUNIT_EXPECT_EQ(test, (long)head, copied);
+
+	for (i = 0; i < copied; i++) {
+		KUNIT_EXPECT_EQ_MSG(test, 0xAB, ((u8 *)buf)[i],
+				    "data mismatch at offset %d", i);
+		if (((u8 *)buf)[i] != 0xAB)
+			break;
+	}
+
+	perf_event_release_aux(event);
+}
+
+static void test_multi_produce_copy_cycle(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	void *buf;
+	unsigned long head, tail = 0;
+	long copied;
+	int ret, cycle;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	buf = kunit_kmalloc(test, PAGE_SIZE, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	for (cycle = 0; cycle < 3; cycle++) {
+		head = aux_test_produce(event, 256);
+		KUNIT_EXPECT_GT(test, head, tail);
+
+		copied = perf_event_aux_copy(event, tail, head, buf);
+		KUNIT_EXPECT_EQ(test, (long)(head - tail), copied);
+
+		ret = perf_event_aux_tail_set(event, head);
+		KUNIT_EXPECT_EQ(test, 0, ret);
+		tail = head;
+	}
+
+	perf_event_release_aux(event);
+}
+
+static void test_copy_rejects_already_consumed(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	void *buf;
+	unsigned long head, tail;
+	long copied;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, 512);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	tail = head / 2;
+	ret = perf_event_aux_tail_set(event, tail);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	buf = kunit_kmalloc(test, 64, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+
+	copied = perf_event_aux_copy(event, tail - 1, tail, buf);
+	KUNIT_EXPECT_EQ(test, -EINVAL, copied);
+
+	perf_event_release_aux(event);
+}
+
+static void test_output_end_zero_size(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	struct perf_output_handle handle;
+	void *addr;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	addr = perf_aux_output_begin(&handle, event);
+	KUNIT_ASSERT_NOT_NULL(test, addr);
+
+	perf_aux_output_end(&handle, 0);
+	KUNIT_EXPECT_EQ(test, 0, perf_event_aux_head(event));
+
+	perf_event_release_aux(event);
+}
+
+static void test_buffer_full_stops_producer(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	struct perf_output_handle handle;
+	const unsigned long buf_size = PAGE_SIZE;
+	unsigned long head;
+	void *addr;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head = aux_test_produce(event, buf_size);
+	KUNIT_EXPECT_GT(test, head, 0);
+
+	addr = perf_aux_output_begin(&handle, event);
+	/*
+	 * The buffer is full and no consumer has freed space, so a new
+	 * non-overwrite transaction must be refused outright, not handed a
+	 * zero-size window.
+	 */
+	KUNIT_EXPECT_NULL_MSG(test, addr,
+			      "begin() must refuse a full non-overwrite buffer");
+	if (addr) {
+		/* Unexpected: keep the AUX refcount balanced. */
+		KUNIT_EXPECT_EQ(test, 0, handle.size);
+		perf_aux_output_end(&handle, 0);
+	}
+
+	perf_event_release_aux(event);
+}
+
+static void test_setup_with_explicit_watermark(struct kunit *test)
+{
+	struct perf_event *event = aux_test_create_event(test);
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	ret = perf_event_setup_aux(event, 2, 4096);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+	perf_event_release_aux(event);
+
+	ret = perf_event_setup_aux(event, 4, 8192);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+	perf_event_release_aux(event);
+}
+
+static void test_two_events_independent(struct kunit *test)
+{
+	struct perf_event *ev_a = aux_test_create_event(test);
+	struct perf_event *ev_b = aux_test_create_event(test);
+	unsigned long head_a, head_b;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_a);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_b);
+
+	ret = perf_event_setup_aux(ev_a, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	ret = perf_event_setup_aux(ev_b, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head_a = aux_test_produce(ev_a, 512);
+	head_b = perf_event_aux_head(ev_b);
+
+	KUNIT_EXPECT_GT(test, head_a, 0);
+	KUNIT_EXPECT_EQ(test, 0, head_b);
+
+	head_b = aux_test_produce(ev_b, 256);
+	KUNIT_EXPECT_GT(test, head_b, 0);
+	KUNIT_EXPECT_EQ(test, head_a, perf_event_aux_head(ev_a));
+
+	perf_event_release_aux(ev_a);
+	perf_event_release_aux(ev_b);
+}
+
+static void test_release_one_does_not_affect_other(struct kunit *test)
+{
+	struct perf_event *ev_a = aux_test_create_event(test);
+	struct perf_event *ev_b = aux_test_create_event(test);
+	unsigned long head_b;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_a);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_b);
+
+	ret = perf_event_setup_aux(ev_a, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	ret = perf_event_setup_aux(ev_b, 1, 0);
+	KUNIT_EXPECT_EQ(test, 0, ret);
+
+	head_b = aux_test_produce(ev_b, 512);
+	KUNIT_EXPECT_GT(test, head_b, 0);
+
+	perf_event_release_aux(ev_a);
+
+	KUNIT_EXPECT_EQ(test, head_b, perf_event_aux_head(ev_b));
+	KUNIT_EXPECT_GT(test, aux_test_produce(ev_b, 256), head_b);
+
+	perf_event_release_aux(ev_b);
+}
+
+/*
+ * Test: kernel consumer and simulated userspace consumer coexist on
+ * the same PMU without interfering.
+ *
+ * Event A uses aux_kernel_count (via perf_event_setup_aux).
+ * Event B swaps its refcounts to aux_mmap_count=1 to simulate a
+ * userspace mmap'd consumer.  Both rings on the same dummy PMU.
+ *
+ *   - produce on A → B's head is unaffected
+ *   - produce on B → A's head is unaffected
+ *   - release A (kernel) → B still produces
+ */
+static void test_user_kernel_coexistence(struct kunit *test)
+{
+	struct perf_event *ev_a = aux_test_create_event(test);
+	struct perf_event *ev_b = aux_test_create_event(test);
+	struct perf_output_handle handle_a, handle_b;
+	unsigned long a_head_before, b_head_before;
+	void *addr;
+	int ret;
+
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_a);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ev_b);
+
+	/* Event A: kernel consumer */
+	ret = perf_event_setup_aux(ev_a, 1, 0);
+	KUNIT_ASSERT_EQ(test, 0, ret);
+
+	/* Event B: start with kernel setup, then simulate userspace.
+	 * ASSERT: the refcount_set() below dereferences ev_b->rb.
+	 */
+	ret = perf_event_setup_aux(ev_b, 1, 0);
+	KUNIT_ASSERT_EQ(test, 0, ret);
+
+	/* Simulate userspace ownership: swap refcounts on B's rb.
+	 * perf_mmap() would set aux_mmap_count=1 for a userspace event.
+	 */
+	refcount_set(&ev_b->rb->aux_mmap_count, 1);
+	refcount_set(&ev_b->rb->aux_kernel_count, 0);
+
+	/* Produce on A — B's head stays 0 */
+	addr = perf_aux_output_begin(&handle_a, ev_a);
+	KUNIT_ASSERT_NOT_NULL(test, addr);
+	perf_aux_output_end(&handle_a, 512);
+	KUNIT_EXPECT_GT(test, ev_a->rb->aux_head, 0);
+	KUNIT_EXPECT_EQ(test, 0, ev_b->rb->aux_head);
+
+	/* Produce on B — A's head unchanged */
+	a_head_before = ev_a->rb->aux_head;
+	addr = perf_aux_output_begin(&handle_b, ev_b);
+	KUNIT_ASSERT_NOT_NULL(test, addr);
+	perf_aux_output_end(&handle_b, 256);
+	KUNIT_EXPECT_GT(test, ev_b->rb->aux_head, 0);
+	KUNIT_EXPECT_EQ(test, a_head_before, ev_a->rb->aux_head);
+
+	/* Release A (kernel) — B still produces */
+	b_head_before = ev_b->rb->aux_head;
+	perf_event_release_aux(ev_a);
+	addr = perf_aux_output_begin(&handle_b, ev_b);
+	KUNIT_ASSERT_NOT_NULL(test, addr);
+	perf_aux_output_end(&handle_b, 128);
+	KUNIT_EXPECT_GT(test, ev_b->rb->aux_head, b_head_before);
+
+	/* Cleanup B: restore kernel ownership so the deferred release
+	 * path (and the explicit one below) tears the buffer down.
+	 */
+	refcount_set(&ev_b->rb->aux_kernel_count, 1);
+	refcount_set(&ev_b->rb->aux_mmap_count, 0);
+	perf_event_release_aux(ev_b);
+}
+
+/*
+ * Test: two kthreads concurrently call perf_event_release_aux() on the
+ * same event.  perf_event_release_aux() serialises the whole teardown on
+ * event->mmap_mutex, so exactly one caller drops aux_kernel_count and
+ * performs the teardown while the other observes event->rb == NULL and
+ * returns as a no-op.  The test verifies the race is handled without a
+ * refcount underflow warning, crash, or leak.
+ */
+struct concurrent_release_ctx {
+	struct perf_event *event;
+	struct completion done;
+};
+
+static int concurrent_release_thread(void *data)
+{
+	struct concurrent_release_ctx *ctx = data;
+
+	perf_event_release_aux(ctx->event);
+	complete(&ctx->done);
+
+	return 0;
+}
+
+static void test_concurrent_release(struct kunit *test)
+{
+	struct concurrent_release_ctx ctx;
+	struct task_struct *threads[2];
+	struct perf_event *event;
+	int created = 0, i;
+
+	event = aux_test_create_event(test);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, event);
+
+	KUNIT_ASSERT_EQ(test, 0, perf_event_setup_aux(event, 1, 0));
+
+	ctx.event = event;
+	init_completion(&ctx.done);
+
+	for (i = 0; i < ARRAY_SIZE(threads); i++) {
+		threads[i] = kthread_run(concurrent_release_thread, &ctx,
+					 "aux_crel_%d", i);
+		if (IS_ERR(threads[i])) {
+			threads[i] = NULL;
+			break;
+		}
+		created++;
+	}
+
+	/*
+	 * Wait for every thread that was actually created before returning:
+	 * ctx lives on this stack, and a struct completion is the right
+	 * wait/notify primitive for an on-stack waiter (unlike a lockless
+	 * wait_event() condition, which the worker could satisfy via
+	 * atomic_dec_and_test() before it gets to wake_up()).
+	 *
+	 * No kthread_stop() here: the workers do not check
+	 * kthread_should_stop() and exit on their own after complete(), and
+	 * stopping an already-exited kthread would touch freed memory.
+	 */
+	for (i = 0; i < created; i++)
+		wait_for_completion(&ctx.done);
+
+	if (created != ARRAY_SIZE(threads))
+		kunit_warn(test, "only %d/%zu release threads created\n",
+			   created, ARRAY_SIZE(threads));
+
+	/*
+	 * If no worker could run, tear the buffer down here; if one did,
+	 * this is the documented no-op second release.  The deferred
+	 * cleanup action releases the event itself.
+	 */
+	perf_event_release_aux(event);
+}
+
+static struct kunit_case aux_kernel_test_cases[] = {
+	KUNIT_CASE(test_setup_rejects_non_power_of_two),
+	KUNIT_CASE(test_setup_accepts_power_of_two),
+	KUNIT_CASE(test_setup_rejects_double),
+	KUNIT_CASE(test_setup_parent_event_rejected),
+	KUNIT_CASE(test_setup_rejects_negative_watermark),
+	KUNIT_CASE(test_setup_with_explicit_watermark),
+	KUNIT_CASE(test_writer_admitted_after_setup),
+	KUNIT_CASE(test_writer_blocked_after_release),
+	KUNIT_CASE(test_release_noop_without_rb),
+	KUNIT_CASE(test_setup_release_roundtrip),
+	KUNIT_CASE(test_multi_setup_release_cycle),
+	KUNIT_CASE(test_double_release_clean),
+	KUNIT_CASE(test_two_events_independent),
+	KUNIT_CASE(test_release_one_does_not_affect_other),
+	KUNIT_CASE(test_user_kernel_coexistence),
+	KUNIT_CASE(test_concurrent_release),
+	KUNIT_CASE(test_head_initial_zero),
+	KUNIT_CASE(test_head_advances_after_produce),
+	KUNIT_CASE(test_head_no_rb_zero),
+	KUNIT_CASE(test_copy_full_window),
+	KUNIT_CASE(test_copy_wrap),
+	KUNIT_CASE(test_copy_zero_len),
+	KUNIT_CASE(test_copy_data_correctness),
+	KUNIT_CASE(test_copy_rejects_rewound),
+	KUNIT_CASE(test_copy_rejects_future),
+	KUNIT_CASE(test_copy_rejects_oversized),
+	KUNIT_CASE(test_copy_rejects_null_buf),
+	KUNIT_CASE(test_copy_rejects_already_consumed),
+	KUNIT_CASE(test_copy_no_rb_enoent),
+	KUNIT_CASE(test_tail_set_valid_frees_space),
+	KUNIT_CASE(test_tail_set_rejects_future),
+	KUNIT_CASE(test_tail_set_accepts_within_ring_window),
+	KUNIT_CASE(test_tail_set_rejects_behind_ring),
+	KUNIT_CASE(test_tail_set_noop),
+	KUNIT_CASE(test_tail_set_consume_all),
+	KUNIT_CASE(test_tail_set_no_rb_enoent),
+	KUNIT_CASE(test_multi_produce_copy_cycle),
+	KUNIT_CASE(test_output_end_zero_size),
+	KUNIT_CASE(test_buffer_full_stops_producer),
+	{},
+};
+
+static struct kunit_suite aux_kernel_test_suite = {
+	.name = "perf_aux_kernel",
+	.suite_init = suite_init,
+	.suite_exit = suite_exit,
+	.test_cases = aux_kernel_test_cases,
+};
+
+kunit_test_suite(aux_kernel_test_suite);
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 134b15a44625..7db7eb07c2f2 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -3073,6 +3073,20 @@ config HW_BREAKPOINT_KUNIT_TEST
 
 	  If unsure, say N.
 
+config PERF_AUX_KERNEL_KUNIT_TEST
+	bool "Test perf AUX kernel-consumer API" if !KUNIT_ALL_TESTS
+	depends on PERF_EVENTS=y
+	depends on KUNIT=y
+	default KUNIT_ALL_TESTS
+	help
+	  Tests for the perf AUX kernel-consumer API functions:
+	  perf_event_setup_aux(), perf_event_release_aux(),
+	  perf_event_aux_head(), perf_event_aux_tail_set(), and
+	  perf_event_aux_copy().  Uses a dummy AUX PMU and does
+	  not require AUX-capable hardware.
+
+	  If unsure, say N.
+
 source "lib/crypto/tests/Kconfig"
 
 config SIPHASH_KUNIT_TEST
-- 
2.43.0

[PATCH 4/5] selftests/perf_events: add userspace AUX regression test

[Kunwu Chan]

From: Lian Wang <lianux.mm@gmail.com>

Add a standalone C program that opens a perf event, mmaps the metadata
and data ring, sets up the AUX area via a second mmap, enables tracing,
reads aux_head from the mmap page, and tears down the mapping.  This
verifies the kernel-side aux_kernel_count changes do not break the
existing userspace AUX mmap protocol.

Environment-dependent outcomes are reported as SKIP rather than PASS:
an AUX area mmap failure (PMU does not support userspace AUX), a PMU
that allows only one open event (e.g. ARM SPE), or a 100 ms window in
which no AUX data was generated are SKIP rather than PASS.  The
two-event independence test follows the full dual-mmap AUX
protocol for both
events so both aux_mmap_count holders really exist, and the
mmap-page-fields test checks the kernel-managed fields
(data_offset/data_size) and the read-back of the user-published
aux_offset/aux_size rather than expecting the kernel to set them.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
---
 tools/testing/selftests/perf_events/Makefile  |   1 +
 .../perf_events/aux_kernel_usermode.c         | 627 ++++++++++++++++++
 tools/testing/selftests/perf_events/config    |   2 +
 3 files changed, 630 insertions(+)
 create mode 100644 tools/testing/selftests/perf_events/aux_kernel_usermode.c

diff --git a/tools/testing/selftests/perf_events/Makefile b/tools/testing/selftests/perf_events/Makefile
index 2e5d85770dfe..fbafbd43f097 100644
--- a/tools/testing/selftests/perf_events/Makefile
+++ b/tools/testing/selftests/perf_events/Makefile
@@ -3,4 +3,5 @@ CFLAGS += -Wl,-no-as-needed -Wall $(KHDR_INCLUDES)
 LDFLAGS += -lpthread
 
 TEST_GEN_PROGS := sigtrap_threads remove_on_exec watermark_signal mmap
+TEST_GEN_PROGS += aux_kernel_usermode
 include ../lib.mk
diff --git a/tools/testing/selftests/perf_events/aux_kernel_usermode.c b/tools/testing/selftests/perf_events/aux_kernel_usermode.c
new file mode 100644
index 000000000000..5c2da4bac074
--- /dev/null
+++ b/tools/testing/selftests/perf_events/aux_kernel_usermode.c
@@ -0,0 +1,627 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Userspace AUX regression test for the perf AUX kernel-consumer API.
+ *
+ * Verifies that the kernel-side AUX changes (aux_kernel_count,
+ * perf_event_setup_aux, etc.) do NOT break the existing userspace
+ * AUX mmap protocol:
+ *   - perf_event_open() with AUX still works
+ *   - mmap AUX buffer still works
+ *   - aux_head / aux_tail from the mmap page are still correct
+ *   - aux_mmap_count is independent of aux_kernel_count
+ *   - Userspace and kernel AUX owners can coexist
+ *
+ * Build:  $(CC) -o aux_kernel_usermode aux_kernel_usermode.c
+ * Usage:  sudo ./aux_kernel_usermode [pmu_name]
+ *
+ * Environment-dependent outcomes are reported as SKIP, not PASS: an
+ * unsupported PMU, a PMU that only allows one open event at a time, or
+ * a system configuration that rejects the event means the protocol was
+ * not exercised, so it must not inflate the pass count.
+ */
+
+#include <errno.h>
+#include <fcntl.h>
+#include <linux/perf_event.h>
+#include <signal.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/syscall.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#define PASS(fmt, ...) printf("  [PASS] " fmt "\n", ##__VA_ARGS__)
+#define FAIL(fmt, ...) do { \
+	printf("  [FAIL] " fmt "\n", ##__VA_ARGS__); \
+	failed = 1; \
+} while (0)
+#define SKIP(fmt, ...) do { \
+	printf("  [SKIP] " fmt "\n", ##__VA_ARGS__); \
+	skipped = 1; \
+} while (0)
+
+static int failed;
+static int skipped;
+
+static long perf_event_open(struct perf_event_attr *attr, pid_t pid,
+			    int cpu, int group_fd, unsigned long flags)
+{
+	return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
+}
+
+/*
+ * Find an AUX PMU type by scanning sysfs PMU directories.
+ */
+static int find_aux_pmu_type(void)
+{
+	char path[256], type_path[261];
+	FILE *f;
+	struct stat st;
+	int type;
+
+	/* Try common AUX PMU names first */
+	static const char * const known[] = {
+		"arm_spe_0", "arm_spe_1", "arm_spe_2",
+		"intel_pt", "intel_pt0",
+		NULL
+	};
+
+	for (int i = 0; known[i]; i++) {
+		snprintf(path, sizeof(path),
+			 "/sys/bus/event_source/devices/%s", known[i]);
+		if (stat(path, &st) == 0 && S_ISDIR(st.st_mode)) {
+			snprintf(type_path, sizeof(type_path),
+				 "%s/type", path);
+			f = fopen(type_path, "r");
+			if (f) {
+				if (fscanf(f, "%d", &type) == 1) {
+					fclose(f);
+					printf("  Found AUX PMU: %s (type=%d)\n",
+					       known[i], type);
+					return type;
+				}
+				fclose(f);
+			}
+		}
+	}
+
+	return -1;
+}
+
+/*
+ * Test 1: Basic perf_event_open() with software event.
+ * This verifies the perf core is not broken at all.
+ */
+static int test_basic_sw_event(void)
+{
+	struct perf_event_attr attr = {};
+	int fd;
+
+	attr.type = PERF_TYPE_SOFTWARE;
+	attr.config = PERF_COUNT_SW_CPU_CLOCK;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+
+	fd = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd < 0) {
+		FAIL("basic SW event: perf_event_open failed (%s)",
+		     strerror(errno));
+		return 1;
+	}
+
+	if (ioctl(fd, PERF_EVENT_IOC_ENABLE, 0) < 0) {
+		FAIL("basic SW event: enable failed (%s)", strerror(errno));
+		close(fd);
+		return 1;
+	}
+	usleep(10000);
+	if (ioctl(fd, PERF_EVENT_IOC_DISABLE, 0) < 0) {
+		FAIL("basic SW event: disable failed (%s)", strerror(errno));
+		close(fd);
+		return 1;
+	}
+
+	close(fd);
+	PASS("basic SW event: open/enable/disable/close");
+	return 0;
+}
+
+/*
+ * Test 2: Userspace AUX mmap full lifecycle.
+ * Opens an AUX event, mmaps the metadata/data ring, sets up the AUX
+ * area via a second mmap, enables, reads aux_head, disables, unmaps,
+ * closes.  This is the critical regression test.
+ *
+ * If the kernel rejects the AUX area mmap, the test is SKIP (not all
+ * PMUs support userspace AUX mmap).
+ */
+static int test_aux_mmap(int pmu_type)
+{
+	struct perf_event_attr attr = {};
+	struct perf_event_mmap_page *mp;
+	void *aux_base;
+	unsigned long aux_size, aux_offset, mmap_size;
+	int fd, ret;
+
+	attr.type = pmu_type;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+	attr.sample_period = 256;
+
+	/* ARM SPE requires period mode (freq=0) */
+	attr.freq = 0;
+
+	fd = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd < 0) {
+		FAIL("AUX mmap: perf_event_open failed (%s)", strerror(errno));
+		return 1;
+	}
+
+	/*
+	 * Step 1: mmap the metadata page and data ring at offset 0.
+	 * The kernel creates the ring buffer and returns the metadata page.
+	 */
+	aux_size = 1UL << 20; /* 1 MiB */
+	mmap_size = aux_size + getpagesize();
+	mp = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
+		  MAP_SHARED, fd, 0);
+	if (mp == MAP_FAILED) {
+		FAIL("AUX mmap: metadata mmap failed (%s)", strerror(errno));
+		close(fd);
+		return 1;
+	}
+
+	/*
+	 * Step 2: set aux_offset and aux_size in the metadata page.
+	 * The AUX area starts after the metadata and data pages.
+	 * The kernel reads these values when the AUX area is mmap'd.
+	 */
+	aux_offset = mmap_size;
+	mp->aux_offset = aux_offset;
+	mp->aux_size = aux_size;
+
+	/*
+	 * Step 3: mmap the AUX area at aux_offset.
+	 * The kernel reads aux_offset from the metadata page, verifies
+	 * the mmap offset matches, and allocates the AUX buffer.
+	 */
+	aux_base = mmap(NULL, aux_size, PROT_READ, MAP_SHARED, fd,
+			aux_offset);
+	if (aux_base == MAP_FAILED) {
+		printf("  [SKIP] AUX mmap: AUX area mmap failed (%s)\n",
+		       strerror(errno));
+		munmap(mp, mmap_size);
+		close(fd);
+		return 0;
+	}
+	PASS("AUX mmap: metadata mmap OK, AUX area mmap OK");
+
+	/* Enable the event */
+	ret = ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+	if (ret < 0) {
+		FAIL("AUX mmap: enable failed (%s)", strerror(errno));
+		munmap(aux_base, aux_size);
+		munmap(mp, mmap_size);
+		close(fd);
+		return 1;
+	}
+
+	/* Run a small workload to generate some AUX data */
+	usleep(100000); /* 100ms */
+
+	/* Disable the event */
+	ret = ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
+	if (ret < 0) {
+		FAIL("AUX mmap: disable failed (%s)", strerror(errno));
+		munmap(aux_base, aux_size);
+		munmap(mp, mmap_size);
+		close(fd);
+		return 1;
+	}
+
+	/* Unmap AUX area */
+	ret = munmap(aux_base, aux_size);
+	if (ret < 0) {
+		FAIL("AUX mmap: AUX munmap failed (%s)", strerror(errno));
+		munmap(mp, mmap_size);
+		close(fd);
+		return 1;
+	}
+
+	/* Unmap metadata */
+	ret = munmap(mp, mmap_size);
+	if (ret < 0) {
+		FAIL("AUX mmap: metadata munmap failed (%s)", strerror(errno));
+		close(fd);
+		return 1;
+	}
+	PASS("AUX mmap: full lifecycle PASS");
+	close(fd);
+	return 0;
+}
+
+/*
+ * Map the data ring (offset 0) of @fd and then, following the AUX
+ * protocol, publish aux_offset/aux_size in the metadata page and map
+ * the AUX area itself.  Returns 0 on success, 1 on failure, and -1
+ * when the kernel refused the AUX mmap (caller should SKIP, since not
+ * every PMU supports userspace AUX).
+ */
+static int mmap_event_aux(int fd, unsigned long aux_size,
+			  struct perf_event_mmap_page **mpp, void **auxp,
+			  unsigned long *mmap_sizep)
+{
+	struct perf_event_mmap_page *mp;
+	void *aux_base;
+	unsigned long mmap_size, aux_offset;
+
+	mmap_size = aux_size + getpagesize();
+	mp = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
+		  MAP_SHARED, fd, 0);
+	if (mp == MAP_FAILED) {
+		FAIL("mmap of metadata/data ring failed (%s)", strerror(errno));
+		return 1;
+	}
+
+	/* The AUX area follows the metadata and data pages. */
+	aux_offset = mmap_size;
+	mp->aux_offset = aux_offset;
+	mp->aux_size = aux_size;
+
+	aux_base = mmap(NULL, aux_size, PROT_READ, MAP_SHARED, fd, aux_offset);
+	if (aux_base == MAP_FAILED) {
+		munmap(mp, mmap_size);
+		return -1;
+	}
+
+	*mpp = mp;
+	*auxp = aux_base;
+	*mmap_sizep = mmap_size;
+	return 0;
+}
+
+/*
+ * Test 3: Verify that aux_mmap_count is independent of aux_kernel_count.
+ * Open two events for the same AUX PMU and give each its own AUX area
+ * through the full dual-mmap protocol, so both aux_mmap_count holders
+ * exist at the same time.  Verifies both events can hold AUX buffers
+ * simultaneously.
+ *
+ * If the PMU allows only one open event (e.g. ARM SPE) the second open
+ * is a SKIP, not a PASS: the coexistence was not exercised.
+ */
+static int test_aux_mmap_independence(int pmu_type)
+{
+	struct perf_event_attr attr = {};
+	struct perf_event_mmap_page *mp1, *mp2;
+	unsigned long aux_size, mmap_size1, mmap_size2;
+	void *aux1, *aux2;
+	int fd1, fd2, ret;
+
+	aux_size = 1UL << 16; /* 64 KiB */
+
+	attr.type = pmu_type;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+	attr.sample_period = 256;
+	attr.freq = 0;
+
+	fd1 = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd1 < 0) {
+		FAIL("AUX independence: first event open failed (%s)",
+		     strerror(errno));
+		return 1;
+	}
+
+	ret = mmap_event_aux(fd1, aux_size, &mp1, &aux1, &mmap_size1);
+	if (ret < 0) {
+		SKIP("AUX independence: first AUX area mmap failed (%s)",
+		     strerror(errno));
+		close(fd1);
+		return 0;
+	}
+	if (ret > 0) {
+		close(fd1);
+		return 1;
+	}
+	PASS("AUX independence: first event has an AUX buffer");
+
+	/* Open a second event for the same PMU */
+	fd2 = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd2 < 0) {
+		/* Some PMUs (e.g. ARM SPE) only allow one event at a time */
+		SKIP("AUX independence: second event open (%s)",
+		     strerror(errno));
+		munmap(aux1, aux_size);
+		munmap(mp1, mmap_size1);
+		close(fd1);
+		return 0;
+	}
+
+	ret = mmap_event_aux(fd2, aux_size, &mp2, &aux2, &mmap_size2);
+	if (ret < 0) {
+		SKIP("AUX independence: second AUX area mmap failed (%s)",
+		     strerror(errno));
+		munmap(aux1, aux_size);
+		munmap(mp1, mmap_size1);
+		close(fd1);
+		close(fd2);
+		return 0;
+	}
+	if (ret > 0) {
+		munmap(aux1, aux_size);
+		munmap(mp1, mmap_size1);
+		close(fd1);
+		close(fd2);
+		return 1;
+	}
+	PASS("AUX independence: second event has an AUX buffer");
+
+	/* Enable both */
+	if (ioctl(fd1, PERF_EVENT_IOC_ENABLE, 0) < 0 ||
+	    ioctl(fd2, PERF_EVENT_IOC_ENABLE, 0) < 0) {
+		FAIL("AUX independence: enable failed (%s)", strerror(errno));
+		munmap(aux2, aux_size); munmap(mp2, mmap_size2);
+		munmap(aux1, aux_size); munmap(mp1, mmap_size1);
+		close(fd1); close(fd2);
+		return 1;
+	}
+	usleep(50000);
+	if (ioctl(fd1, PERF_EVENT_IOC_DISABLE, 0) < 0 ||
+	    ioctl(fd2, PERF_EVENT_IOC_DISABLE, 0) < 0) {
+		FAIL("AUX independence: disable failed (%s)", strerror(errno));
+		munmap(aux2, aux_size); munmap(mp2, mmap_size2);
+		munmap(aux1, aux_size); munmap(mp1, mmap_size1);
+		close(fd1); close(fd2);
+		return 1;
+	}
+
+	PASS("AUX independence: two events ran simultaneously");
+
+	/* Both metadata pages must remain distinct and readable */
+	PASS("AUX independence: event1 head=%llu event2 head=%llu",
+	     (unsigned long long)__atomic_load_n(&mp1->aux_head, __ATOMIC_RELAXED),
+	     (unsigned long long)__atomic_load_n(&mp2->aux_head, __ATOMIC_RELAXED));
+
+	munmap(aux1, aux_size);
+	munmap(mp1, mmap_size1);
+	munmap(aux2, aux_size);
+	munmap(mp2, mmap_size2);
+	close(fd1);
+	close(fd2);
+	PASS("AUX independence: cleanup OK");
+	return 0;
+}
+
+/*
+ * Test 4: Verify that the mmap control page fields are correct.
+ *
+ * version, compat_version, data_offset and data_size are managed by
+ * the kernel; aux_offset and aux_size are published by userspace (they
+ * are the request for the AUX mapping), so the test checks the
+ * kernel-managed fields and the read-back of the user-published ones
+ * instead of expecting the kernel to fill aux_offset in.
+ */
+static int test_mmap_page_fields(int pmu_type)
+{
+	struct perf_event_attr attr = {};
+	struct perf_event_mmap_page *mp;
+	unsigned long total_size;
+	int fd;
+
+	total_size = 1UL << 18; /* 256 KiB */
+
+	attr.type = pmu_type;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+	attr.sample_period = 256;
+	attr.freq = 0;
+
+	fd = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd < 0) {
+		FAIL("mmap fields: perf_event_open failed (%s)", strerror(errno));
+		return 1;
+	}
+
+	mp = mmap(NULL, total_size + getpagesize(), PROT_READ | PROT_WRITE,
+		  MAP_SHARED, fd, 0);
+	if (mp == MAP_FAILED) {
+		FAIL("mmap fields: mmap failed (%s)", strerror(errno));
+		close(fd);
+		return 1;
+	}
+
+	/* Kernel-managed fields */
+	if (mp->version != 1 && mp->version != 0) {
+		FAIL("mmap fields: version=%u (expected 0 or 1)", mp->version);
+		munmap(mp, total_size + getpagesize());
+		close(fd);
+		return 1;
+	}
+	PASS("mmap fields: version=%u", mp->version);
+
+	if (mp->compat_version != 0) {
+		FAIL("mmap fields: compat_version=%u (expected 0)",
+		     mp->compat_version);
+		munmap(mp, total_size + getpagesize());
+		close(fd);
+		return 1;
+	}
+	PASS("mmap fields: compat_version=%u", mp->compat_version);
+
+	if (mp->data_offset == 0 || mp->data_size == 0) {
+		FAIL("mmap fields: data_offset=%llu data_size=%llu (no data ring)",
+		     (unsigned long long)mp->data_offset,
+		     (unsigned long long)mp->data_size);
+		munmap(mp, total_size + getpagesize());
+		close(fd);
+		return 1;
+	}
+	PASS("mmap fields: data_offset=%llu data_size=%llu",
+	     (unsigned long long)mp->data_offset,
+	     (unsigned long long)mp->data_size);
+
+	/* User-published AUX request fields must round-trip */
+	mp->aux_offset = total_size + getpagesize();
+	mp->aux_size = 1UL << 16;
+	if (mp->aux_offset != total_size + getpagesize() || mp->aux_size != 1UL << 16) {
+		FAIL("mmap fields: aux_offset/aux_size write not visible");
+		munmap(mp, total_size + getpagesize());
+		close(fd);
+		return 1;
+	}
+	PASS("mmap fields: aux_offset/aux_size writable and readable");
+
+	munmap(mp, total_size + getpagesize());
+	close(fd);
+	PASS("mmap fields: all fields correct");
+	return 0;
+}
+
+/*
+ * Test 5: Verify that aux_head does not go backwards while the event
+ * runs.  An unchanged head means the PMU produced no AUX data for this
+ * workload, which is a SKIP (the monotonicity was not exercised), not a
+ * PASS; a head that moved backwards is a FAIL, since a u64 cursor
+ * cannot realistically wrap in 100 ms.
+ */
+static int test_aux_head_monotonic(int pmu_type)
+{
+	struct perf_event_attr attr = {};
+	struct perf_event_mmap_page *mp;
+	unsigned long aux_size, mmap_size, head_before, head_after;
+	void *aux_base;
+	int fd, ret;
+
+	aux_size = 1UL << 16;
+	mmap_size = aux_size + getpagesize();
+
+	attr.type = pmu_type;
+	attr.size = sizeof(attr);
+	attr.disabled = 1;
+	attr.sample_period = 256;
+	attr.freq = 0;
+
+	fd = perf_event_open(&attr, 0, -1, -1, 0);
+	if (fd < 0) {
+		/* Some PMUs may reject the default config */
+		SKIP("AUX head monotonic: PMU rejected config (%s)",
+		     strerror(errno));
+		return 0;
+	}
+
+	ret = mmap_event_aux(fd, aux_size, &mp, &aux_base, &mmap_size);
+	if (ret < 0) {
+		SKIP("AUX head monotonic: AUX area mmap failed (%s)",
+		     strerror(errno));
+		close(fd);
+		return 0;
+	}
+	if (ret > 0) {
+		close(fd);
+		return 1;
+	}
+
+	ret = ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+	if (ret < 0) {
+		SKIP("AUX head monotonic: enable failed (%s)", strerror(errno));
+		munmap(aux_base, aux_size);
+		munmap(mp, mmap_size);
+		close(fd);
+		return 0;
+	}
+
+	/* Read initial head */
+	head_before = __atomic_load_n(&mp->aux_head, __ATOMIC_RELAXED);
+
+	usleep(100000); /* 100ms */
+
+	head_after = __atomic_load_n(&mp->aux_head, __ATOMIC_RELAXED);
+
+	ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
+
+	if (head_after > head_before) {
+		PASS("AUX head monotonic: before=%lu after=%lu (advanced)",
+		     head_before, head_after);
+	} else if (head_after == head_before) {
+		SKIP("AUX head monotonic: no AUX data generated in 100ms");
+	} else {
+		FAIL("AUX head monotonic: backwards (before=%lu after=%lu)",
+		     head_before, head_after);
+	}
+
+	munmap(aux_base, aux_size);
+	munmap(mp, mmap_size);
+	close(fd);
+	return 0;
+}
+
+int main(void)
+{
+	int pmu_type;
+
+	if (geteuid() != 0) {
+		fprintf(stderr, "Run as root\n");
+		return 4; /* skip */
+	}
+
+	printf("=== Perf AUX Kernel API: Userspace Regression Test ===\n\n");
+
+	/* Test 1: Basic SW event */
+	printf("--- 1. Basic Software Event ---\n");
+	test_basic_sw_event();
+
+	/* Find an AUX-capable PMU */
+	pmu_type = find_aux_pmu_type();
+	if (pmu_type < 0) {
+		printf("--- 2-5. AUX Tests ---\n");
+		printf("  [SKIP] No AUX-capable PMU found\n");
+	} else {
+		/* Test 2: AUX mmap lifecycle */
+		printf("\n--- 2. AUX Mmap Lifecycle ---\n");
+		test_aux_mmap(pmu_type);
+
+		/* Test 3: AUX mmap independence */
+		printf("\n--- 3. AUX Mmap Independence ---\n");
+		test_aux_mmap_independence(pmu_type);
+
+		/* Test 4: mmap page fields */
+		printf("\n--- 4. Mmap Page Fields ---\n");
+		test_mmap_page_fields(pmu_type);
+
+		/* Test 5: AUX head monotonic */
+		printf("\n--- 5. AUX Head Monotonic ---\n");
+		test_aux_head_monotonic(pmu_type);
+	}
+
+	/* Test 6: Verify /proc/sys/kernel/perf_event_paranoid */
+	printf("\n--- 6. Perf Event Paranoid ---\n");
+	{
+		FILE *f = fopen("/proc/sys/kernel/perf_event_paranoid", "r");
+		int paranoid;
+
+		if (f) {
+			if (fscanf(f, "%d", &paranoid) == 1) {
+				printf("  perf_event_paranoid=%d\n", paranoid);
+				if (paranoid > 2)
+					FAIL("perf_event_paranoid=%d may block AUX",
+					     paranoid);
+				else
+					PASS("perf_event_paranoid=%d (OK)", paranoid);
+			}
+			fclose(f);
+		} else {
+			printf("  [SKIP] Cannot read perf_event_paranoid\n");
+		}
+	}
+
+	printf("\n=== Result: %s ===\n", failed ? "FAIL" : "PASS");
+	if (skipped)
+		printf("=== Note: some tests were SKIPped (environment)\n");
+	return failed ? 1 : 0;
+}
diff --git a/tools/testing/selftests/perf_events/config b/tools/testing/selftests/perf_events/config
index ba58ff2203e4..2508f331cfe0 100644
--- a/tools/testing/selftests/perf_events/config
+++ b/tools/testing/selftests/perf_events/config
@@ -1 +1,3 @@
 CONFIG_PERF_EVENTS=y
+CONFIG_PERF_AUX_KERNEL_KUNIT_TEST=y
+CONFIG_KUNIT_DEBUGFS=y
-- 
2.43.0

[PATCH 5/5] selftests/perf_events: add AUX kernel API selftest script

[Kunwu Chan]

Add a shell script that runs the perf_aux_kernel KUnit suite through
debugfs, builds and runs the userspace regression program, and
performs static-analysis checks on the source tree (memory barriers,
refcount types, boundary validation, EXPORT_SYMBOL_GPL coverage).

When an AUX-capable PMU is present, the script also runs a live
perf record and checks dmesg for errors.

The script works both in-tree and from an installed selftest location:
the source-level static checks are SKIPped when no kernel source tree
is found instead of failing, the installed userspace binary is run
as-is when its source is absent, and runtime rebuilds honour $CC
instead of hardcoding gcc.
Usage:  sudo ./aux_kernel.sh [--pmu arm_spe_0]

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 tools/testing/selftests/perf_events/Makefile  |   1 +
 .../selftests/perf_events/aux_kernel.sh       | 631 ++++++++++++++++++
 2 files changed, 632 insertions(+)
 create mode 100755 tools/testing/selftests/perf_events/aux_kernel.sh

diff --git a/tools/testing/selftests/perf_events/Makefile b/tools/testing/selftests/perf_events/Makefile
index fbafbd43f097..9b82c7ee1238 100644
--- a/tools/testing/selftests/perf_events/Makefile
+++ b/tools/testing/selftests/perf_events/Makefile
@@ -4,4 +4,5 @@ LDFLAGS += -lpthread
 
 TEST_GEN_PROGS := sigtrap_threads remove_on_exec watermark_signal mmap
 TEST_GEN_PROGS += aux_kernel_usermode
+TEST_PROGS += aux_kernel.sh
 include ../lib.mk
diff --git a/tools/testing/selftests/perf_events/aux_kernel.sh b/tools/testing/selftests/perf_events/aux_kernel.sh
new file mode 100755
index 000000000000..2271f105f55b
--- /dev/null
+++ b/tools/testing/selftests/perf_events/aux_kernel.sh
@@ -0,0 +1,631 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Perf AUX Kernel-Consumer API - Comprehensive Selftest
+#
+# Validates the perf-core AUX kernel-consumer API added by the
+# perf_aux_support patch series:
+#   - perf_event_setup_aux() / perf_event_release_aux()
+#   - perf_event_aux_head() / perf_event_aux_tail_set() / perf_event_aux_copy()
+#
+# Test layers:
+#   1. Build-time checks (Kconfig, Makefile, symbol exports, source)
+#   2. KUnit suite (API contract: error paths, lifecycle, validation)
+#   3. Userspace AUX regression (mmap AUX still works, aux_mmap_count
+#      independent, userspace+kernel coexistence)
+#   4. Hardware AUX live test (if ARM SPE or Intel PT is available)
+#   5. DAMON integration verification
+#   6. Memory ordering/barrier static analysis
+#   7. Boundary validation static analysis
+#   8. Concurrency safety checks
+#   9. User/kernel AUX isolation checks
+#
+# Usage:  sudo ./aux_kernel.sh [--pmu arm_spe_0|intel_pt]
+#
+# The script also works from an installed selftest location (make
+# install): when no kernel source tree is found, the source-level
+# static checks (sections 1 partial, 6-9) are SKIPped rather than
+# failed, and the prebuilt aux_kernel_usermode binary is used as-is.
+# Source-only rebuilds honour $CC (and $CROSS_COMPILE) instead of
+# hardcoding gcc.
+#
+# Requirements:
+#   - CONFIG_PERF_EVENTS=y
+#   - CONFIG_PERF_AUX_KERNEL_KUNIT_TEST=y and CONFIG_KUNIT_DEBUGFS=y
+#   - Root privileges for most sections
+
+set -e
+PASSED=0; FAILED=0; SKIPPED=0
+pass() { echo "  [PASS] $1"; PASSED=$((PASSED + 1)); }
+fail() { echo "  [FAIL] $1"; FAILED=$((FAILED + 1)); }
+skip() { echo "  [SKIP] $*"; SKIPPED=$((SKIPPED + 1)); }
+
+# mktemp(1) creates an unpredictable, collision-free directory, so a
+# local user cannot pre-create a symlink at the expected path and have
+# this (possibly root-run) script overwrite a file of their choosing.
+RESULTS_DIR=$(mktemp -d /tmp/perf_aux_kernel_test.XXXXXX)
+exec > >(tee "$RESULTS_DIR/output.log") 2>&1
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+ROOT="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel 2>/dev/null || true)"
+if [[ -z "$ROOT" ]]; then
+	ROOT="/usr/src/linux"
+fi
+
+# Source-level checks need the kernel tree; installed runs have none.
+SRC_EVENTS="$ROOT/kernel/events"
+if [[ -f "$SRC_EVENTS/core.c" ]]; then
+	HAVE_SRC=1
+else
+	HAVE_SRC=0
+	ROOT=""
+fi
+
+# Compiler for the runtime rebuild of the userspace program; the
+# kselftest build system exports CC (with CROSS_COMPILE prefixed).
+CC="${CC:-gcc}"
+
+PMU_ARG="${1:-}"
+if [[ "$PMU_ARG" == "--pmu" ]]; then
+	PMU_NAME="${2:-arm_spe_0}"
+elif [[ -n "$PMU_ARG" ]]; then
+	PMU_NAME="$PMU_ARG"
+else
+	PMU_NAME="arm_spe_0"
+fi
+
+echo "=========================================="
+echo " Perf AUX Kernel-Consumer API Test"
+echo "=========================================="
+echo "Kernel tree:  $ROOT"
+echo "Results dir:  $RESULTS_DIR"
+echo "Target PMU:   $PMU_NAME"
+echo "Script dir:   $SCRIPT_DIR"
+echo ""
+
+# ---- Section 1: Build-time checks ----
+echo "--- 1. Build-time Configuration ---"
+
+# Check that the KUnit test is built
+KUNIT_SUITE="/sys/kernel/debug/kunit/perf_aux_kernel"
+if [[ -d "$KUNIT_SUITE" ]]; then
+	pass "KUnit suite 'perf_aux_kernel' is registered"
+else
+	skip "KUnit suite 'perf_aux_kernel' not registered" \
+		"(CONFIG_PERF_AUX_KERNEL_KUNIT_TEST not set)"
+fi
+
+# Check that required symbols are exported
+SYMBOLS_FILE="/proc/kallsyms"
+if [[ -r "$SYMBOLS_FILE" ]]; then
+	for sym in perf_event_setup_aux perf_event_release_aux \
+		perf_event_aux_head perf_event_aux_tail_set perf_event_aux_copy; do
+		if grep -q "$sym" "$SYMBOLS_FILE" 2>/dev/null; then
+			pass "Symbol exported: $sym"
+		else
+			fail "Symbol not exported: $sym (patch not applied?)"
+		fi
+	done
+else
+	skip "Symbol check" "/proc/kallsyms not readable"
+fi
+
+# Check that the source files exist (in-tree runs only; installed
+# environments have no kernel source, which is a skip, not a failure)
+SRC_EVENTS="$ROOT/kernel/events"
+if [[ "$HAVE_SRC" -eq 1 ]]; then
+	for f in core.c ring_buffer.c internal.h; do
+		if [[ -f "$SRC_EVENTS/$f" ]]; then
+			pass "Source file: kernel/events/$f"
+		else
+			fail "Source file: kernel/events/$f missing"
+		fi
+	done
+
+	# Check that the KUnit test source exists
+	if [[ -f "$SRC_EVENTS/aux_kernel_test.c" ]]; then
+		pass "Source file: kernel/events/aux_kernel_test.c"
+	else
+		skip "Source file: kernel/events/aux_kernel_test.c not found"
+	fi
+else
+	skip "Source tree" "kernel source not available (installed run)"
+fi
+
+# Check that aux_kernel_count is in internal.h
+if [[ "$HAVE_SRC" -eq 1 ]] && grep -q 'aux_kernel_count' "$SRC_EVENTS/internal.h" 2>/dev/null; then
+	pass "struct perf_buffer: aux_kernel_count field present"
+elif [[ "$HAVE_SRC" -eq 1 ]]; then
+	fail "struct perf_buffer: aux_kernel_count field missing"
+fi
+
+# Check that aux_kernel_count is used in perf_aux_output_begin()
+if [[ "$HAVE_SRC" -eq 1 ]]; then
+	if grep -q 'aux_kernel_count' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null; then
+		pass "ring_buffer.c: aux_kernel_count guard in perf_aux_output_begin()"
+	else
+		fail "ring_buffer.c: aux_kernel_count guard missing in perf_aux_output_begin()"
+	fi
+fi
+
+# Check that the new API functions are in core.c / ring_buffer.c
+if [[ "$HAVE_SRC" -eq 1 ]]; then
+	for func in perf_event_setup_aux perf_event_release_aux; do
+		if grep -q "int $func\|void $func" "$SRC_EVENTS/core.c" 2>/dev/null; then
+			pass "core.c: $func() defined"
+		else
+			fail "core.c: $func() not found"
+		fi
+	done
+	for func in perf_event_aux_head perf_event_aux_tail_set perf_event_aux_copy; do
+		if grep -q "$func" "$SRC_EVENTS/ring_buffer.c" 2>/dev/null; then
+			pass "ring_buffer.c: $func() defined"
+		else
+			fail "ring_buffer.c: $func() not found"
+		fi
+	done
+fi
+
+# Check that EXPORT_SYMBOL_GPL is present
+if [[ "$HAVE_SRC" -eq 1 ]]; then
+	for func in perf_event_setup_aux perf_event_release_aux \
+		perf_event_aux_head perf_event_aux_tail_set perf_event_aux_copy; do
+		if grep -q "EXPORT_SYMBOL_GPL($func)" "$SRC_EVENTS/core.c" \
+			"$SRC_EVENTS/ring_buffer.c" 2>/dev/null; then
+			pass "EXPORT_SYMBOL_GPL: $func"
+		else
+			fail "EXPORT_SYMBOL_GPL: $func not found"
+		fi
+	done
+fi
+
+# ---- Section 2: KUnit API contract tests ----
+echo ""
+echo "--- 2. KUnit API Contract Tests ---"
+
+KUNIT_DIR="/sys/kernel/debug/kunit/perf_aux_kernel"
+if [[ -d "$KUNIT_DIR" ]]; then
+	if [[ -f "$KUNIT_DIR/run" ]]; then
+		echo run > "$KUNIT_DIR/run" 2>/dev/null || true
+	fi
+	RES="$KUNIT_DIR/results"
+	if [[ -f "$RES" ]]; then
+		cp "$RES" "$RESULTS_DIR/kunit-results.log"
+
+		if grep -Eq "^[[:space:]]*not ok" "$RES"; then
+			fail "KUnit API contract suite has failing cases"
+			grep -E "^[[:space:]]*not ok" "$RES" | sed 's/^/    /'
+		elif grep -Eq "^[[:space:]]*ok " "$RES"; then
+			pass "KUnit API contract suite: all cases pass"
+		else
+			fail "KUnit API contract suite produced no completed cases"
+		fi
+		echo "    $(grep '# Totals:' "$RES" | tail -1)"
+
+		# Spot-check all 39 test cases (deterministic, dummy PMU)
+		for c in "test_setup_rejects_non_power_of_two" \
+			"test_setup_accepts_power_of_two" \
+			"test_setup_rejects_double" \
+			"test_setup_parent_event_rejected" \
+			"test_setup_rejects_negative_watermark" \
+			"test_setup_with_explicit_watermark" \
+			"test_writer_admitted_after_setup" \
+			"test_writer_blocked_after_release" \
+			"test_release_noop_without_rb" \
+			"test_setup_release_roundtrip" \
+			"test_multi_setup_release_cycle" \
+			"test_double_release_clean" \
+			"test_two_events_independent" \
+			"test_release_one_does_not_affect_other" \
+			"test_user_kernel_coexistence" \
+			"test_concurrent_release" \
+			"test_head_initial_zero" \
+			"test_head_advances_after_produce" \
+			"test_head_no_rb_zero" \
+			"test_copy_full_window" \
+			"test_copy_wrap" \
+			"test_copy_zero_len" \
+			"test_copy_data_correctness" \
+			"test_copy_rejects_rewound" \
+			"test_copy_rejects_future" \
+			"test_copy_rejects_oversized" \
+			"test_copy_rejects_null_buf" \
+			"test_copy_rejects_already_consumed" \
+			"test_copy_no_rb_enoent" \
+			"test_tail_set_valid_frees_space" \
+			"test_tail_set_rejects_future" \
+			"test_tail_set_accepts_within_ring_window" \
+			"test_tail_set_rejects_behind_ring" \
+			"test_tail_set_noop" \
+			"test_tail_set_consume_all" \
+			"test_tail_set_no_rb_enoent" \
+			"test_multi_produce_copy_cycle" \
+			"test_output_end_zero_size" \
+			"test_buffer_full_stops_producer"; do
+			if grep -Eq "^[[:space:]]*ok .*$c" "$RES"; then
+				pass "KUnit case: $c"
+			else
+				fail "KUnit case: $c"
+			fi
+		done
+	else
+		skip "KUnit results" "no results file (suite did not run)"
+	fi
+else
+	skip "KUnit API contract" "CONFIG_PERF_AUX_KERNEL_KUNIT_TEST" \
+		"or CONFIG_KUNIT_DEBUGFS missing"
+fi
+
+# ---- Section 3: Userspace AUX regression (C program) ----
+echo ""
+echo "--- 3. Userspace AUX Regression (api-level) ---"
+
+# Run the C regression test program.
+#
+# Prefer an existing (installed or prebuilt) binary; make install
+# installs TEST_GEN_PROGS binaries but not their sources, so requiring
+# the source would skip the test exactly in CI pipelines that install
+# selftests.  Rebuild only in-tree, when the source is newer than the
+# binary or the binary is missing, and honour $CC then.
+UMODE_SRC="$SCRIPT_DIR/aux_kernel_usermode.c"
+UMODE_BIN="$SCRIPT_DIR/aux_kernel_usermode"
+
+if [[ -x "$UMODE_BIN" && ( ! -f "$UMODE_SRC" || ! "$UMODE_SRC" -nt "$UMODE_BIN" ) ]]; then
+	:
+elif [[ -f "$UMODE_SRC" ]]; then
+	echo "  Building aux_kernel_usermode with ${CC}..."
+	if "$CC" -o "$UMODE_BIN" "$UMODE_SRC" -Wall -Wextra -O2; then
+		:
+	else
+		echo "  $CC build failed"
+		UMODE_BIN=""
+	fi
+else
+	UMODE_BIN=""
+fi
+
+if [[ -n "$UMODE_BIN" && -x "$UMODE_BIN" ]]; then
+	if "$UMODE_BIN" "$PMU_NAME" 2>&1 | tee "$RESULTS_DIR/usermode.log" | \
+		grep -c '\[PASS\]' > /dev/null; then
+		# Count pass/fail from the output
+		UMODE_PASS=$(grep -c '\[PASS\]' "$RESULTS_DIR/usermode.log" || true)
+		UMODE_FAIL=$(grep -c '\[FAIL\]' "$RESULTS_DIR/usermode.log" || true)
+		UMODE_SKIP=$(grep -c '\[SKIP\]' "$RESULTS_DIR/usermode.log" || true)
+		pass "Userspace API regression: $UMODE_PASS passed," \
+			" $UMODE_FAIL failed, $UMODE_SKIP skipped"
+		if [[ "$UMODE_FAIL" -gt 0 ]]; then
+			fail "Userspace API regression: $UMODE_FAIL failures"
+		fi
+	else
+		fail "Userspace API regression: C program failed"
+	fi
+else
+	skip "Userspace API regression" "binary or source not available"
+fi
+
+# Check that aux_mmap_count refcount is not affected by aux_kernel_count
+# (static analysis: verify the two refcounts are separate in internal.h)
+if [[ "$HAVE_SRC" -eq 1 ]]; then
+	if grep -q 'aux_mmap_count' "$SRC_EVENTS/internal.h" 2>/dev/null && \
+	   grep -q 'aux_kernel_count' "$SRC_EVENTS/internal.h" 2>/dev/null; then
+		pass "Refcount separation: aux_mmap_count and aux_kernel_count are distinct"
+	else
+		fail "Refcount separation: aux_mmap_count or aux_kernel_count missing"
+	fi
+fi
+
+# ---- Section 4: Hardware AUX live test ----
+echo ""
+echo "--- 4. Hardware AUX Live Test ---"
+
+PMU_DIR="/sys/bus/event_source/devices/$PMU_NAME"
+PERF_BIN=$(which perf 2>/dev/null || true)
+if [[ -d "$PMU_DIR" ]]; then
+	PMU_TYPE=$(cat "$PMU_DIR/type" 2>/dev/null || true)
+	if [[ -n "$PMU_TYPE" ]]; then
+		pass "AUX PMU $PMU_NAME present (type=$PMU_TYPE)"
+
+		if [[ -f "$PMU_DIR/caps/aux_output" ]]; then
+			pass "PMU $PMU_NAME: aux_output capability present"
+		else
+			skip "PMU $PMU_NAME: aux_output capability not found"
+		fi
+
+		if [[ -n "$PERF_BIN" ]]; then
+			DMESG_BEFORE=$(dmesg 2>/dev/null | wc -l)
+			if "$PERF_BIN" record -e "$PMU_NAME/period=100000/" \
+				-o "$RESULTS_DIR/perf_hw.data" -- sleep 0.5 2>/dev/null; then
+				pass "Hardware AUX: perf record with $PMU_NAME succeeded"
+			else
+				fail "Hardware AUX: perf record with $PMU_NAME failed"
+			fi
+
+			DMESG_AFTER="$RESULTS_DIR/dmesg-hardware.log"
+			dmesg 2>/dev/null > "$DMESG_AFTER" || true
+			DMESG_LINES_AFTER=$(wc -l < "$DMESG_AFTER")
+			if [[ "$DMESG_LINES_AFTER" -ge "$DMESG_BEFORE" ]]; then
+				NEW_DMESG=$(tail -n "+$((DMESG_BEFORE + 1))" "$DMESG_AFTER")
+			else
+				NEW_DMESG=$(cat "$DMESG_AFTER")
+			fi
+			FAIL_LINES=$(printf '%s\n' "$NEW_DMESG" | \
+				grep -Ei "perf.*(fail|warn|error).*aux|"\
+"WARNING:|BUG:|Oops:|lockdep" || true)
+			if [[ -n "$FAIL_LINES" ]]; then
+				fail "dmesg: errors during hardware AUX test"
+				echo "$FAIL_LINES" | sed 's/^/    /'
+			else
+				pass "dmesg: clean during hardware AUX test"
+			fi
+		fi
+	else
+		skip "Hardware AUX" "PMU type not readable"
+	fi
+else
+	skip "Hardware AUX" "PMU $PMU_NAME not available"
+fi
+
+# ---- Section 5: DAMON integration check ----
+echo ""
+echo "--- 5. DAMON Integration Verification ---"
+
+DAMON_ADMIN="/sys/kernel/mm/damon/admin"
+if [[ -d "$DAMON_ADMIN" ]]; then
+	pass "DAMON sysfs interface available"
+
+	if [[ -d "$DAMON_ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events" ]] || \
+	   [[ -d "/sys/kernel/debug/damon/perf_stats" ]]; then
+		pass "DAMON perf observe infrastructure detected"
+	else
+		skip "DAMON perf observe" "debugfs or sysfs interface not available"
+	fi
+
+	DAMON_TEST_DIR="$ROOT/tools/testing/selftests/damon"
+	if [[ -d "$DAMON_TEST_DIR" ]]; then
+		pass "DAMON selftest directory present"
+	else
+		skip "DAMON selftest directory" "not found at $DAMON_TEST_DIR"
+	fi
+else
+	skip "DAMON integration" "DAMON admin interface not available"
+fi
+
+# ---- Section 6: Memory ordering and concurrency checks ----
+echo ""
+echo "--- 6. Memory Ordering & Concurrency ---"
+
+if [[ "$HAVE_SRC" -ne 1 ]]; then
+	skip "Memory ordering checks" "kernel source not available (installed run)"
+else
+
+# Verify smp_rmb() in perf_event_aux_head()
+if grep -q 'smp_rmb' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null; then
+	pass "perf_event_aux_head(): smp_rmb() barrier present"
+else
+	fail "perf_event_aux_head(): smp_rmb() barrier missing"
+fi
+
+# Verify smp_mb() in perf_event_aux_tail_set()
+if grep -A30 'perf_event_aux_tail_set' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'smp_mb'; then
+	pass "perf_event_aux_tail_set(): smp_mb() barrier present"
+else
+	fail "perf_event_aux_tail_set(): smp_mb() barrier missing"
+fi
+
+# Verify aux_refcount protection in perf_event_aux_copy()
+if grep -A80 'perf_event_aux_copy' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'aux_refcount'; then
+	pass "perf_event_aux_copy(): aux_refcount protection present"
+else
+	fail "perf_event_aux_copy(): aux_refcount protection missing"
+fi
+
+# Verify mmap_mutex serialization in setup/release
+if grep -A30 'perf_event_setup_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'mmap_mutex'; then
+	pass "perf_event_setup_aux(): mmap_mutex serialization present"
+else
+	fail "perf_event_setup_aux(): mmap_mutex serialization missing"
+fi
+if grep -A30 'perf_event_release_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'mmap_mutex'; then
+	pass "perf_event_release_aux(): mmap_mutex serialization present"
+else
+	fail "perf_event_release_aux(): mmap_mutex serialization missing"
+fi
+
+# Verify aux_mutex protection in release
+if grep -A60 'perf_event_release_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'aux_mutex'; then
+	pass "perf_event_release_aux(): aux_mutex protection present"
+else
+	fail "perf_event_release_aux(): aux_mutex protection missing"
+fi
+
+fi
+
+# ---- Section 7: Boundary validation checks ----
+echo ""
+echo "--- 7. API Contract: Boundary Validation ---"
+
+if [[ "$HAVE_SRC" -ne 1 ]]; then
+	skip "Boundary validation checks" "kernel source not available (installed run)"
+else
+
+# perf_event_setup_aux() rejects non-power-of-2
+if grep -q 'is_power_of_2' "$SRC_EVENTS/core.c" 2>/dev/null; then
+	pass "perf_event_setup_aux(): is_power_of_2() check present"
+else
+	fail "perf_event_setup_aux(): is_power_of_2() check missing"
+fi
+
+# perf_event_setup_aux() checks is_kernel_event()
+if grep -q 'is_kernel_event' "$SRC_EVENTS/core.c" 2>/dev/null; then
+	pass "perf_event_setup_aux(): is_kernel_event() check present"
+else
+	fail "perf_event_setup_aux(): is_kernel_event() check missing"
+fi
+
+# perf_event_setup_aux() checks event->parent
+if grep -A20 'perf_event_setup_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'event->parent'; then
+	pass "perf_event_setup_aux(): event->parent check present"
+else
+	fail "perf_event_setup_aux(): event->parent check missing"
+fi
+
+# perf_event_aux_copy() validates window size
+if grep -A80 'perf_event_aux_copy' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'available.*aux_size'; then
+	pass "perf_event_aux_copy(): window bounds check present"
+else
+	fail "perf_event_aux_copy(): window bounds check missing"
+fi
+
+# rb_has_kernel_aux() helper
+if grep -q 'rb_has_kernel_aux' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null; then
+	pass "ring_buffer.c: rb_has_kernel_aux() helper defined"
+else
+	fail "ring_buffer.c: rb_has_kernel_aux() helper missing"
+fi
+
+# perf_event_aux_tail_set() validates advance <= head - old_tail
+if grep -A20 'perf_event_aux_tail_set' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'advance.*head.*old_tail'; then
+	pass "perf_event_aux_tail_set(): advance bounds check present"
+else
+	fail "perf_event_aux_tail_set(): advance bounds check missing"
+fi
+
+# perf_event_aux_copy() validates from/to in [tail, head]
+if grep -A80 'perf_event_aux_copy' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'start.*available\|len.*available.*start'; then
+	pass "perf_event_aux_copy(): from/to bounds check present"
+else
+	fail "perf_event_aux_copy(): from/to bounds check missing"
+fi
+
+fi
+
+# ---- Section 8: Concurrency safety checks ----
+echo ""
+echo "--- 8. Concurrency Safety ---"
+
+if [[ "$HAVE_SRC" -ne 1 ]]; then
+	skip "Concurrency safety checks" "kernel source not available (installed run)"
+else
+
+# Check that aux_kernel_count uses refcount_t (atomic)
+if grep -q 'refcount_t.*aux_kernel_count' "$SRC_EVENTS/internal.h" 2>/dev/null; then
+	pass "aux_kernel_count: refcount_t (atomic) type"
+else
+	fail "aux_kernel_count: not refcount_t"
+fi
+
+# Check that release drops the kernel-owner refcount under mmap_mutex
+# (serialising concurrent perf_event_release_aux() callers)
+if grep -A60 'perf_event_release_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'refcount_dec_and_test(&rb->aux_kernel_count)'; then
+	pass "aux_kernel_count: owner refcount dropped in release"
+else
+	fail "aux_kernel_count: owner refcount not dropped in release"
+fi
+
+# Check that aux_refcount is incremented in copy
+if grep -A80 'perf_event_aux_copy' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'refcount_inc_not_zero.*aux_refcount'; then
+	pass "perf_event_aux_copy(): aux_refcount_inc_not_zero() present"
+else
+	fail "perf_event_aux_copy(): aux_refcount_inc_not_zero() missing"
+fi
+
+# Check that rb_free_aux is called at end of copy (release refcount)
+if grep -A80 'perf_event_aux_copy' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'rb_free_aux'; then
+	pass "perf_event_aux_copy(): rb_free_aux() at exit (balanced refcount)"
+else
+	fail "perf_event_aux_copy(): rb_free_aux() missing at exit"
+fi
+
+# Check that is_kernel_event guard is on both setup and release
+if grep -A20 'perf_event_setup_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'is_kernel_event'; then
+	pass "setup side: is_kernel_event() guard present"
+else
+	fail "setup side: is_kernel_event() guard missing"
+fi
+if grep -A20 'perf_event_release_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'is_kernel_event'; then
+	pass "release side: is_kernel_event() guard present"
+else
+	fail "release side: is_kernel_event() guard missing"
+fi
+
+# Check that event->parent guard is on both setup and release
+if grep -A20 'perf_event_setup_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'event->parent'; then
+	pass "setup side: event->parent guard present"
+else
+	fail "setup side: event->parent guard missing"
+fi
+if grep -A20 'perf_event_release_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'event->parent'; then
+	pass "release side: event->parent guard present"
+else
+	fail "release side: event->parent guard missing"
+fi
+
+fi
+
+# ---- Section 9: User/kernel AUX isolation ----
+echo ""
+echo "--- 9. User/Kernel AUX Isolation ---"
+
+if [[ "$HAVE_SRC" -ne 1 ]]; then
+	skip "User/kernel isolation checks" "kernel source not available (installed run)"
+else
+
+if grep -A30 'perf_event_setup_aux' "$SRC_EVENTS/core.c" 2>/dev/null | \
+	grep -q 'EBUSY'; then
+	pass "setup_aux: -EBUSY when event->rb exists (user+kernel exclusion)"
+else
+	fail "setup_aux: missing -EBUSY guard"
+fi
+
+if grep -q 'aux_mmap_count' "$SRC_EVENTS/internal.h" 2>/dev/null && \
+   grep -q 'aux_kernel_count' "$SRC_EVENTS/internal.h" 2>/dev/null; then
+	pass "aux_mmap_count and aux_kernel_count are distinct fields"
+else
+	fail "aux_mmap_count or aux_kernel_count missing"
+fi
+
+if grep -B2 -A2 'aux_mmap_count' "$SRC_EVENTS/ring_buffer.c" 2>/dev/null | \
+	grep -q 'aux_kernel_count'; then
+	pass "perf_aux_output_begin: checks both owner counts"
+else
+	fail "perf_aux_output_begin: does not check both counts"
+fi
+
+if grep -q 'aux_kernel_count' "$SRC_EVENTS/core.c" 2>/dev/null; then
+	pass "release_aux: checks aux_kernel_count ownership"
+else
+	fail "release_aux: missing aux_kernel_count check"
+fi
+
+fi
+
+# ---- Summary ----
+echo ""
+echo "=========================================="
+echo " SUMMARY: $PASSED passed, $FAILED failed, $SKIPPED skipped"
+echo "=========================================="
+echo "Results saved to: $RESULTS_DIR"
+
+if [[ "$FAILED" -gt 0 ]]; then
+	echo "Overall: FAIL"
+	exit 1
+else
+	echo "Overall: PASS"
+	exit 0
+fi
-- 
2.43.0