mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader
@ 2026-11-16 17:56 Kyuwon Cho
  0 siblings, 0 replies; only message in thread
From: Kyuwon Cho @ 2026-11-16 17:56 UTC (permalink / raw)
  To: Steven Rostedt, Masami Hiramatsu
  Cc: Vincent Donnefort, Mathieu Desnoyers, linux-trace-kernel, linux-kernel

[-- Attachment #1: Type: text/plain, Size: 5785 bytes --]

Hi Steve, Masami, Vincent,

While testing v7.4-rc3 with an LLM-guided test generator that we are
developing as a research prototype, we found that reading a CPU's
trace_pipe while another task consumes the same CPU buffer through the
mmap interface (TRACE_MMAP_IOCTL_GET_READER) makes trace_pipe print
entries that have already been overwritten. With KASAN this ends in an
out-of-bounds read in the event output code:

  BUG: KASAN: use-after-free in string (lib/vsprintf.c:650 lib/vsprintf.c:732)
  Read of size 1 at addr ff11000011e2d8ed by task test/89
  Call Trace:
   <TASK>
   string (lib/vsprintf.c:650 lib/vsprintf.c:732)
   vsnprintf (lib/vsprintf.c:2948)
   seq_buf_vprintf (lib/seq_buf.c:74)
   trace_seq_vprintf (kernel/trace/trace_seq.c:175)
   trace_event_printf (kernel/trace/trace_output.c:360)
   trace_raw_output_workqueue_rescued (include/trace/events/workqueue.h:210)
   print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
   tracing_read_pipe (kernel/trace/trace.c:5475)
   vfs_read (fs/read_write.c:552)
   ksys_read (fs/read_write.c:696)
   do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
   </TASK>
  The buggy address belongs to the physical page:
  page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x11e2d
  page_type: f0(buddy)

and the type check in trace_assign_type() fires as well:

  WARNING: kernel/trace/trace_output.c:1798 at trace_print_print+0xeb/0x130, CPU#1: test/89
  Call Trace:
   <TASK>
   print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
   tracing_read_pipe (kernel/trace/trace.c:5475)
   vfs_read (fs/read_write.c:552)
   ksys_read (fs/read_write.c:696)
   do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
   </TASK>

The stale entry carries another event's type, so that event's output
function decodes trace_marker data as its fields, including __data_loc
offsets and lengths that point outside the ring buffer page. Other runs
reported slab-out-of-bounds and global-out-of-bounds reads from the same
path; the ring buffer pages themselves are not freed.

tracing_buffers_ioctl() calls ring_buffer_map_get_reader() without
trace_access_lock():

	if (cmd == TRACE_MMAP_IOCTL_GET_READER) {
		...
		return ring_buffer_map_get_reader(iter->array_buffer->buffer,
						  iter->cpu_file);
	}

The other consuming readers of a CPU buffer, tracing_read_pipe(),
tracing_buffers_read() and tracing_buffers_splice_read(), all take
trace_access_lock(iter->cpu_file). tracing_read_pipe() peeks an event,
prints it with reader_lock dropped and only then consumes it, so a
GET_READER in between can hand the reader page back to the writers. This
has been the case since the ioctl was added in cf9f0f7c4c5b ("tracing:
Allow user-space mapping of the ring-buffer").

Taking trace_access_lock(iter->cpu_file) around
ring_buffer_map_get_reader() in tracing_buffers_ioctl(), as the other
readers do, makes the problem go away here: the reproducer below ran
three times without a report, and the ring-buffer map_test selftest
still passes.

The reproducer needs root, for tracefs. One thread writes to trace_marker
on CPU 0; on CPU 1, one thread loops on TRACE_MMAP_IOCTL_GET_READER on
the mmapped per_cpu/cpu0/trace_pipe_raw and another reads
per_cpu/cpu0/trace_pipe. On an x86_64 QEMU guest with 2 vCPUs, KASAN
and lockdep (gcc 13.3, config attached), the WARNING appears within
seconds in every run and KASAN reports in 2 of 3 runs. Neither reader
alone triggers it.

Checked against v7.4-rc3 (aebb72744764); tracing_buffers_ioctl() is the
same in trace/for-next. We found no earlier report on syzbot or on
linux-trace-kernel.

// Run as root with tracefs available and at least 2 CPUs.
// CPU 0: write to trace_marker. CPU 1: one thread polls
// TRACE_MMAP_IOCTL_GET_READER on the mmapped per_cpu/cpu0/trace_pipe_raw,
// another reads per_cpu/cpu0/trace_pipe.
#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <unistd.h>

#define T "/sys/kernel/tracing"
#define TRACE_MMAP_IOCTL_GET_READER _IO('R', 0x20)

static volatile int stop;

static void pin(int cpu)
{
	cpu_set_t set;

	CPU_ZERO(&set);
	CPU_SET(cpu, &set);
	sched_setaffinity(0, sizeof(set), &set);
}

static void put(const char *file, const char *val)
{
	int fd = open(file, O_WRONLY | O_TRUNC);

	if (fd >= 0) {
		write(fd, val, strlen(val));
		close(fd);
	}
}

static void *writer(void *arg)
{
	char buf[1000];
	int fd = open(T "/trace_marker", O_WRONLY);

	pin(0);
	memset(buf, 'a', sizeof(buf));
	for (unsigned long i = 0; !stop; i++)
		write(fd, buf, 1 + i % sizeof(buf));
	return NULL;
}

static void *mapper(void *arg)
{
	int fd = open(T "/per_cpu/cpu0/trace_pipe_raw", O_RDONLY | O_NONBLOCK);

	pin(1);
	if (mmap(NULL, getpagesize(), PROT_READ, MAP_SHARED, fd, 0) == MAP_FAILED) {
		perror("mmap");
		return NULL;
	}
	while (!stop)
		ioctl(fd, TRACE_MMAP_IOCTL_GET_READER);
	return NULL;
}

static void *cooked(void *arg)
{
	char buf[4096];
	int fd = open(T "/per_cpu/cpu0/trace_pipe", O_RDONLY | O_NONBLOCK);

	pin(1);
	while (!stop)
		read(fd, buf, sizeof(buf));
	return NULL;
}

int main(void)
{
	pthread_t th[3];

	mount("tracefs", T, "tracefs", 0, NULL);
	put(T "/buffer_size_kb", "16");
	put(T "/tracing_on", "1");
	pthread_create(&th[0], NULL, writer, NULL);
	pthread_create(&th[1], NULL, mapper, NULL);
	pthread_create(&th[2], NULL, cooked, NULL);
	sleep(60);
	stop = 1;
	for (int i = 0; i < 3; i++)
		pthread_join(th[i], NULL);
	return 0;
}

Thanks,
Kyuwon Cho

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36886 bytes --]

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2026-11-16 17:56 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 17:56 [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader Kyuwon Cho

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®