[PATCH v5 0/4] tracing: add ring-buffer memory usage statistics

[PATCH v5 0/4] tracing: add ring-buffer memory usage statistics

[Xiang Gao]

Report the memory consumed by the tracing ring buffers, rather than the
usable data capacity exposed by buffer_size_kb. Android low-memory
diagnostics need this to attribute the memory used by tracing when
calculating lost RAM.

The buffers can be spread across the global trace array, dynamically
created instances, and snapshot buffers. Userspace currently has to
discover and sum every instance, and snapshot memory is not exposed by
the per-instance totals.

This series adds:

  /sys/kernel/tracing/trace_stats/memory_usage_kb

reporting:

  global:
  instances:
  snapshots:
  persistents:
  remotes:
  total_system:
  total_carveout:

global, instances, and snapshots come from the page allocator.
persistents use reserved memory. remotes are not yet reported and are
always zero. total_system is the page-allocator memory, while
total_carveout is reserved memory that was never in the free pool. A
per-CPU view is provided under trace_stats/per_cpu/cpuN/.

The memory source is split into system and carveout only for now. A
finer-grained attribute (host/guest/reserved/device) can be added when
remotes are enumerated, since guest and device memory do not exist in
the current accounting.

Changes since v4:
- Categorize the report by buffer type and memory source.
- Make ring_buffer_memory_size() generic so it returns a size for any
  buffer type instead of zero for remote buffers.
- Skip the backup instance, whose vmalloc memory is already accounted in
  VmallocUsed.
- Rebase on v7.4-rc1.
- Split the documentation into its own patch.

Xiang Gao (4):
  ring-buffer: add ring_buffer_memory_size()
  tracing: add aggregate buffer memory usage statistics
  tracing: add per-CPU buffer memory usage statistics
  Documentation: document trace_stats memory_usage_kb

 Documentation/trace/ftrace.rst |  22 ++++
 include/linux/ring_buffer.h    |   1 +
 kernel/trace/ring_buffer.c     |  37 +++++++
 kernel/trace/trace.c           | 188 +++++++++++++++++++++++++++++++++
 4 files changed, 248 insertions(+)

-- 
2.34.1

[PATCH v5 1/4] ring-buffer: add ring_buffer_memory_size()

[Xiang Gao]

Add a helper to report the page memory consumed by a ring buffer for a
given CPU. It accounts for the data sub-buffers, the reader page, the
cached read page, and the mmap metadata page. Unlike
ring_buffer_capacity(), which reports the usable data capacity, this
reports the full pages allocated to the buffer.

The function is generic and returns a size for any buffer type, so
callers that need the memory footprint of remote or persistent buffers
can use it as well.

Signed-off-by: Xiang Gao <gaoxiang17@xiaomi.com>
---
 include/linux/ring_buffer.h |  1 +
 kernel/trace/ring_buffer.c  | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index 649b2276b557..e83e94623e76 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -174,6 +174,7 @@ int ring_buffer_iter_empty(struct ring_buffer_iter *iter);
 bool ring_buffer_iter_dropped(struct ring_buffer_iter *iter);
 
 unsigned long ring_buffer_capacity(struct trace_buffer *buffer, int cpu);
+unsigned long ring_buffer_memory_size(struct trace_buffer *buffer, int cpu);
 unsigned long ring_buffer_max_event_size(struct trace_buffer *buffer);
 
 void ring_buffer_reset_cpu(struct trace_buffer *buffer, int cpu);
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 1fd28a72e6ee..0396ec671998 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -6566,6 +6566,43 @@ unsigned long ring_buffer_capacity(struct trace_buffer *buffer, int cpu)
 }
 EXPORT_SYMBOL_GPL(ring_buffer_capacity);
 
+/**
+ * ring_buffer_memory_size - return the memory used by the buffer (in bytes)
+ * @buffer: The ring buffer.
+ * @cpu: The CPU to get ring buffer memory from.
+ *
+ * Returns the page memory consumed by @cpu, including the data sub-buffers,
+ * the reader page, the cached read page, and the mmap metadata page. Unlike
+ * ring_buffer_capacity(), which reports the usable data capacity, this
+ * accounts for the full pages allocated to the buffer.
+ */
+unsigned long ring_buffer_memory_size(struct trace_buffer *buffer, int cpu)
+{
+	struct ring_buffer_per_cpu *cpu_buffer;
+	unsigned long subbuf_size;
+	unsigned long size;
+
+	if (!cpumask_test_cpu(cpu, buffer->cpumask))
+		return 0;
+
+	cpu_buffer = buffer->buffers[cpu];
+	subbuf_size = PAGE_SIZE << READ_ONCE(buffer->subbuf_order);
+
+	/* Data sub-buffers plus the reader page. */
+	size = (READ_ONCE(cpu_buffer->nr_pages) + 1) * subbuf_size;
+
+	/* The cached read page, if present, is a full sub-buffer page. */
+	if (READ_ONCE(cpu_buffer->free_page.data))
+		size += subbuf_size;
+
+	/* The mmap metadata page is a single system page. */
+	if (READ_ONCE(cpu_buffer->meta_page))
+		size += PAGE_SIZE;
+
+	return size;
+}
+EXPORT_SYMBOL_GPL(ring_buffer_memory_size);
+
 /**
  * ring_buffer_max_event_size - return the max data size of an event
  * @buffer: The ring buffer.
-- 
2.34.1

[PATCH v5 2/4] tracing: add aggregate buffer memory usage statistics

[Xiang Gao]

Add a trace_stats directory with memory_usage_kb reporting the page
memory consumed by the tracing ring buffers, categorized by buffer type
and memory source:

  global:
  instances:
  snapshots:
  persistents:
  remotes:
  total_system:
  total_carveout:

The global, instances, and snapshots values come from the page
allocator. Persistent buffers use reserved memory and are reported under
persistents, while the backup instance uses vmalloc and is skipped since
it is already accounted in VmallocUsed. Remote buffers are not yet
enumerated and report zero.

total_system is the page-allocator memory that contributes to lost RAM,
while total_carveout is reserved memory that was never in the free pool.

Signed-off-by: Xiang Gao <gaoxiang17@xiaomi.com>
---
 kernel/trace/trace.c | 115 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index e8c4efa8107b..6036e368fe10 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -5797,6 +5797,107 @@ tracing_total_capacity_read(struct file *filp, char __user *ubuf, size_t cnt, lo
 	return simple_read_from_buffer(ubuf, cnt, ppos, buf, r);
 }
 
+struct trace_mem_stats {
+	unsigned long	global;
+	unsigned long	instances;
+	unsigned long	snapshots;
+	unsigned long	persistents;
+	unsigned long	remotes;
+	unsigned long	total_system;
+	unsigned long	total_carveout;
+};
+
+static void
+trace_array_buffer_memory(struct trace_array *tr, int cpu,
+			  struct trace_mem_stats *stats)
+{
+	if (tr->range_addr_start) {
+		/*
+		 * Persistent buffers use reserved memory (carveout). The
+		 * backup instance uses vmalloc and is already accounted in
+		 * VmallocUsed, so skip it.
+		 */
+		if (!(tr->flags & TRACE_ARRAY_FL_VMALLOC))
+			stats->persistents +=
+				ring_buffer_memory_size(tr->array_buffer.buffer, cpu);
+	} else if (tr->flags & TRACE_ARRAY_FL_GLOBAL) {
+		stats->global +=
+			ring_buffer_memory_size(tr->array_buffer.buffer, cpu);
+	} else {
+		stats->instances +=
+			ring_buffer_memory_size(tr->array_buffer.buffer, cpu);
+	}
+
+#ifdef CONFIG_TRACER_SNAPSHOT
+	if (tr->snapshot_buffer.buffer)
+		stats->snapshots +=
+			ring_buffer_memory_size(tr->snapshot_buffer.buffer, cpu);
+#endif
+}
+
+static struct trace_mem_stats trace_buffers_memory(void)
+{
+	struct trace_mem_stats stats = {};
+	struct trace_array *tr;
+	int cpu;
+
+	guard(mutex)(&trace_types_lock);
+
+	list_for_each_entry(tr, &ftrace_trace_arrays, list) {
+		for_each_tracing_cpu(cpu)
+			trace_array_buffer_memory(tr, cpu, &stats);
+	}
+
+	/*
+	 * temp_buffer is allocated in tracer_alloc_buffers() and is never
+	 * attached to a trace array. It temporarily holds event data for
+	 * triggers when tracing is off. Account for its pages too.
+	 */
+	if (temp_buffer) {
+		for_each_tracing_cpu(cpu)
+			stats.global += ring_buffer_memory_size(temp_buffer, cpu);
+	}
+
+	stats.total_system = stats.global + stats.instances +
+			     stats.snapshots + stats.remotes;
+	stats.total_carveout = stats.persistents;
+
+	return stats;
+}
+
+static int trace_mem_show(struct seq_file *m, void *v)
+{
+	struct trace_mem_stats stats = trace_buffers_memory();
+
+	seq_printf(m, "global: %lu\n", stats.global >> 10);
+	seq_printf(m, "instances: %lu\n", stats.instances >> 10);
+	seq_printf(m, "snapshots: %lu\n", stats.snapshots >> 10);
+	seq_printf(m, "persistents: %lu\n", stats.persistents >> 10);
+	seq_printf(m, "remotes: %lu\n", stats.remotes >> 10);
+	seq_printf(m, "total_system: %lu\n", stats.total_system >> 10);
+	seq_printf(m, "total_carveout: %lu\n", stats.total_carveout >> 10);
+
+	return 0;
+}
+
+static int trace_mem_open(struct inode *inode, struct file *file)
+{
+	int ret;
+
+	ret = tracing_check_open_get_tr(NULL);
+	if (ret)
+		return ret;
+
+	return single_open(file, trace_mem_show, inode->i_private);
+}
+
+static const struct file_operations trace_mem_fops = {
+	.open		= trace_mem_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
 #define LAST_BOOT_HEADER ((void *)1)
 
 static void *l_next(struct seq_file *m, void *v, loff_t *pos)
@@ -9313,6 +9414,18 @@ static struct notifier_block trace_module_nb = {
 };
 #endif /* CONFIG_MODULES */
 
+static __init void init_trace_stats_tracefs(void)
+{
+	struct dentry *stats_dir;
+
+	stats_dir = tracefs_create_dir("trace_stats", NULL);
+	if (!stats_dir)
+		return;
+
+	trace_create_file("memory_usage_kb", TRACE_MODE_READ, stats_dir,
+			  NULL, &trace_mem_fops);
+}
+
 static __init void tracer_init_tracefs_work_func(struct work_struct *work)
 {
 	/*
@@ -9327,6 +9440,8 @@ static __init void tracer_init_tracefs_work_func(struct work_struct *work)
 	init_tracer_tracefs(&global_trace, NULL);
 	ftrace_init_tracefs_toplevel(&global_trace, NULL);
 
+	init_trace_stats_tracefs();
+
 	trace_create_file("tracing_thresh", TRACE_MODE_WRITE, NULL,
 			&global_trace, &tracing_thresh_fops);
 
-- 
2.34.1

[PATCH v5 3/4] tracing: add per-CPU buffer memory usage statistics

[Xiang Gao]

Provide a per-CPU view of the tracing ring-buffer memory usage under
trace_stats/per_cpu/cpuN/memory_usage_kb, using the same categorized
format as the aggregate file. Each file reports only the buffers
belonging to that CPU.

Signed-off-by: Xiang Gao <gaoxiang17@xiaomi.com>
---
 kernel/trace/trace.c | 75 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 74 insertions(+), 1 deletion(-)

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 6036e368fe10..e7e23ff38105 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -5898,6 +5898,60 @@ static const struct file_operations trace_mem_fops = {
 	.release	= single_release,
 };
 
+static struct trace_mem_stats trace_buffers_memory_cpu(int cpu)
+{
+	struct trace_mem_stats stats = {};
+	struct trace_array *tr;
+
+	guard(mutex)(&trace_types_lock);
+
+	list_for_each_entry(tr, &ftrace_trace_arrays, list)
+		trace_array_buffer_memory(tr, cpu, &stats);
+
+	/* Account for the bootstrapping temp_buffer as well. */
+	if (temp_buffer)
+		stats.global += ring_buffer_memory_size(temp_buffer, cpu);
+
+	stats.total_system = stats.global + stats.instances +
+			     stats.snapshots + stats.remotes;
+	stats.total_carveout = stats.persistents;
+
+	return stats;
+}
+
+static int trace_mem_per_cpu_show(struct seq_file *m, void *v)
+{
+	struct trace_mem_stats stats = trace_buffers_memory_cpu((long)m->private);
+
+	seq_printf(m, "global: %lu\n", stats.global >> 10);
+	seq_printf(m, "instances: %lu\n", stats.instances >> 10);
+	seq_printf(m, "snapshots: %lu\n", stats.snapshots >> 10);
+	seq_printf(m, "persistents: %lu\n", stats.persistents >> 10);
+	seq_printf(m, "remotes: %lu\n", stats.remotes >> 10);
+	seq_printf(m, "total_system: %lu\n", stats.total_system >> 10);
+	seq_printf(m, "total_carveout: %lu\n", stats.total_carveout >> 10);
+
+	return 0;
+}
+
+static int trace_mem_per_cpu_open(struct inode *inode, struct file *file)
+{
+	int ret;
+
+	ret = tracing_check_open_get_tr(NULL);
+	if (ret)
+		return ret;
+
+	return single_open(file, trace_mem_per_cpu_show, inode->i_private);
+}
+
+static const struct file_operations trace_mem_per_cpu_fops = {
+	.open		= trace_mem_per_cpu_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
 #define LAST_BOOT_HEADER ((void *)1)
 
 static void *l_next(struct seq_file *m, void *v, loff_t *pos)
@@ -9416,7 +9470,9 @@ static struct notifier_block trace_module_nb = {
 
 static __init void init_trace_stats_tracefs(void)
 {
-	struct dentry *stats_dir;
+	struct dentry *stats_dir, *per_cpu_dir, *cpu_dir;
+	char cpu_dir_name[30];
+	int cpu;
 
 	stats_dir = tracefs_create_dir("trace_stats", NULL);
 	if (!stats_dir)
@@ -9424,6 +9480,23 @@ static __init void init_trace_stats_tracefs(void)
 
 	trace_create_file("memory_usage_kb", TRACE_MODE_READ, stats_dir,
 			  NULL, &trace_mem_fops);
+
+	per_cpu_dir = tracefs_create_dir("per_cpu", stats_dir);
+	if (!per_cpu_dir)
+		return;
+
+	for_each_tracing_cpu(cpu) {
+		snprintf(cpu_dir_name, 30, "cpu%d", cpu);
+		cpu_dir = tracefs_create_dir(cpu_dir_name, per_cpu_dir);
+		if (!cpu_dir) {
+			pr_warn("Could not create tracefs '%s' entry\n",
+				cpu_dir_name);
+			continue;
+		}
+
+		trace_create_file("memory_usage_kb", TRACE_MODE_READ, cpu_dir,
+				  (void *)(long)cpu, &trace_mem_per_cpu_fops);
+	}
 }
 
 static __init void tracer_init_tracefs_work_func(struct work_struct *work)
-- 
2.34.1

[PATCH v5 4/4] Documentation: document trace_stats memory_usage_kb

[Xiang Gao]

Document the trace_stats/memory_usage_kb file, which reports the tracing
ring-buffer memory usage categorized by buffer type and memory source.

Signed-off-by: Xiang Gao <gaoxiang17@xiaomi.com>
---
 Documentation/trace/ftrace.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst
index 9423722848c6..d7af1e37e6b1 100644
--- a/Documentation/trace/ftrace.rst
+++ b/Documentation/trace/ftrace.rst
@@ -218,6 +218,28 @@ of ftrace. Here is a list of some of the key files:
 
 	This displays the total combined size of all the trace buffers.
 
+  trace_stats/memory_usage_kb:
+
+	This reports the memory consumed by the ring buffers, as opposed to
+	the usable data capacity shown by buffer_size_kb. The value is
+	categorized by buffer type and memory source::
+
+	    global:
+	    instances:
+	    snapshots:
+	    persistents:
+	    remotes:
+	    total_system:
+	    total_carveout:
+
+	global, instances, and snapshots come from the page allocator.
+	persistents use reserved memory. remotes are not yet reported and
+	are always zero. total_system is the page-allocator memory, while
+	total_carveout is reserved memory that was never in the free pool.
+
+	The same file is provided under trace_stats/per_cpu/cpuN/, reporting
+	only the buffers belonging to that CPU.
+
   buffer_subbuf_size_kb:
 
 	This sets or displays the sub buffer size. The ring buffer is broken up
-- 
2.34.1