[RFC] bpf: account ring buffer backing pages separately from Lost RAM

[RFC] bpf: account ring buffer backing pages separately from Lost RAM

[Xiang Gao]

Hi,

I would like to discuss accounting BPF ring buffer backing pages in
system-wide memory reports.

BPF ring buffers allocate their data and metadata as order-0 pages directly
from the buddy allocator, and then map those pages with vmap().

Because vmap() maps caller-owned pages, these backing pages are not counted
by VmallocUsed. They are also not slab pages. As a result, most BPF ring
buffer memory is not represented by an existing named /proc/meminfo category
and appears as Lost RAM in Android memory reports.

We measured this on an Android 6.18 kernel.

Test case:

  32 BPF ring buffer maps
  16 MiB data area per map
  512 MiB total data area

Observed changes:

  Lost RAM:       approximately +529 MiB
  VmallocUsed:      approximately +2 MiB
  Slab:        approximately unchanged

After destroying all maps, the values returned close to baseline.

The question is whether the kernel should expose the unique physical backing
pages of live BPF ring buffers through a dedicated global counter and a
/proc/meminfo entry, for example:

  BpfRingbuf: <value in kB>

The proposed counter would include:

  * ring buffer data pages;
  * metadata pages;
  * consumer and producer position pages.

It would exclude:

  * the second virtual mapping of data pages;
  * the pages[] pointer array;
  * map metadata allocations;
  * vmap page tables.

The goal is to account for the currently unclassified direct backing pages.
Slab- and vmalloc-backed auxiliary allocations are already represented by
existing memory categories and should not be counted again.

A possible implementation is an NR_BPF_RINGBUF vmstat counter maintained by
the ring buffer allocation and free paths, with the aggregate exposed through
/proc/meminfo.

Questions:

1. Is a dedicated BPF ring buffer counter appropriate?
2. Should this be represented as an NR_* vmstat counter?
3. Is /proc/meminfo an acceptable interface for this information?
4. Is counting only unique physical backing pages the correct accounting unit?

Thanks,

Xiang Gao

Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM

[David Hildenbrand (Arm)]

On 8/15/26 11:18, Xiang Gao wrote:
> Hi,

Hi,

> 
> I would like to discuss accounting BPF ring buffer backing pages in
> system-wide memory reports.
> 
> BPF ring buffers allocate their data and metadata as order-0 pages directly
> from the buddy allocator, and then map those pages with vmap().

I assume there is a reason the slab isn't used, right? Are these pages mapped
into user space such that page->mapcount would get used?

Can you point me at relevant code?

> 
> Because vmap() maps caller-owned pages, these backing pages are not counted
> by VmallocUsed. They are also not slab pages. As a result, most BPF ring
> buffer memory is not represented by an existing named /proc/meminfo category
> and appears as Lost RAM in Android memory reports.
> 
> We measured this on an Android 6.18 kernel.
> 
> Test case:
> 
>   32 BPF ring buffer maps
>   16 MiB data area per map
>   512 MiB total data area
> 
> Observed changes:
> 
>   Lost RAM:       approximately +529 MiB
>   VmallocUsed:      approximately +2 MiB
>   Slab:        approximately unchanged
> 
> After destroying all maps, the values returned close to baseline.
> 
> The question is whether the kernel should expose the unique physical backing
> pages of live BPF ring buffers through a dedicated global counter and a
> /proc/meminfo entry, for example:
> 
>   BpfRingbuf: <value in kB>

This looks a bit too specific for my taste. And I think we should try to no
inflate these statistics here too much.

> 
> The proposed counter would include:
> 
>   * ring buffer data pages;
>   * metadata pages;
>   * consumer and producer position pages.
> 
> It would exclude:
> 
>   * the second virtual mapping of data pages;
>   * the pages[] pointer array;
>   * map metadata allocations;
>   * vmap page tables.
> 
> The goal is to account for the currently unclassified direct backing pages.
> Slab- and vmalloc-backed auxiliary allocations are already represented by
> existing memory categories and should not be counted again.
> 
> A possible implementation is an NR_BPF_RINGBUF vmstat counter maintained by
> the ring buffer allocation and free paths, with the aggregate exposed through
> /proc/meminfo.
> 
> Questions:
> 
> 1. Is a dedicated BPF ring buffer counter appropriate?

I don't think so.

See [1] where we just had the same discussion for tracing buffers. For them,
Steve [2] had an idea on how to expose them more fine-grained and tracing specific.

[1] https://lore.kernel.org/r/20260810094025.136705-1-gaoxiang17@xiaomi.com
[2] https://lore.kernel.org/r/20260810105710.6ee5e493@gandalf.local.home

> 2. Should this be represented as an NR_* vmstat counter?

I don't think so.

> 3. Is /proc/meminfo an acceptable interface for this information?

Again, I don't think so. "Lost RAM" really is just "excessive memory allocated
by some other subsystem".

I agree that some users might want to figure out what is consuming that much
memory, but I don't think growing /proc/meminfo in that way is really what we want.

> 4. Is counting only unique physical backing pages the correct accounting unit?

I'd assume the "It would exclude" part above should not be accounted there, if
that's what you mean.

-- 
Cheers,

David

Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM

[Andrii Nakryiko]

On Mon, Aug 17, 2026 at 11:10 AM David Hildenbrand (Arm)
<david@kernel.org> wrote:
>
> On 8/15/26 11:18, Xiang Gao wrote:
> > Hi,
>
> Hi,
>
> >
> > I would like to discuss accounting BPF ring buffer backing pages in
> > system-wide memory reports.
> >
> > BPF ring buffers allocate their data and metadata as order-0 pages directly
> > from the buddy allocator, and then map those pages with vmap().
>
> I assume there is a reason the slab isn't used, right? Are these pages mapped
> into user space such that page->mapcount would get used?
>
> Can you point me at relevant code?

See code in [0]. And yes, these pages are meant to be mapped into user space.

  [0] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/kernel/bpf/ringbuf.c#n93

>
> >
> > Because vmap() maps caller-owned pages, these backing pages are not counted
> > by VmallocUsed. They are also not slab pages. As a result, most BPF ring
> > buffer memory is not represented by an existing named /proc/meminfo category
> > and appears as Lost RAM in Android memory reports.
> >
> > We measured this on an Android 6.18 kernel.
> >
> > Test case:
> >
> >   32 BPF ring buffer maps
> >   16 MiB data area per map
> >   512 MiB total data area
> >
> > Observed changes:
> >
> >   Lost RAM:       approximately +529 MiB
> >   VmallocUsed:      approximately +2 MiB
> >   Slab:        approximately unchanged
> >
> > After destroying all maps, the values returned close to baseline.
> >
> > The question is whether the kernel should expose the unique physical backing
> > pages of live BPF ring buffers through a dedicated global counter and a
> > /proc/meminfo entry, for example:
> >
> >   BpfRingbuf: <value in kB>
>
> This looks a bit too specific for my taste. And I think we should try to no
> inflate these statistics here too much.
>

+1, way too specific

> >
> > The proposed counter would include:
> >
> >   * ring buffer data pages;
> >   * metadata pages;
> >   * consumer and producer position pages.
> >
> > It would exclude:
> >
> >   * the second virtual mapping of data pages;
> >   * the pages[] pointer array;
> >   * map metadata allocations;
> >   * vmap page tables.
> >
> > The goal is to account for the currently unclassified direct backing pages.
> > Slab- and vmalloc-backed auxiliary allocations are already represented by
> > existing memory categories and should not be counted again.
> >
> > A possible implementation is an NR_BPF_RINGBUF vmstat counter maintained by
> > the ring buffer allocation and free paths, with the aggregate exposed through
> > /proc/meminfo.
> >
> > Questions:
> >
> > 1. Is a dedicated BPF ring buffer counter appropriate?
>
> I don't think so.
>
> See [1] where we just had the same discussion for tracing buffers. For them,
> Steve [2] had an idea on how to expose them more fine-grained and tracing specific.
>
> [1] https://lore.kernel.org/r/20260810094025.136705-1-gaoxiang17@xiaomi.com
> [2] https://lore.kernel.org/r/20260810105710.6ee5e493@gandalf.local.home
>

We already report per-BPF ringbuf memory usage either through bpf()
syscall or map's fdinfo. E.g., with `sudo bpftool map show` you'll
see"

1455733: ringbuf  name event_ringbuf  flags 0x0
        key 0B  value 0B  max_entries 262144  memlock 275776B
        btf_id 2193434
        pids tcpeventd(2549812)

where memlock is how much memory is allocated for the ringbuf data area.

> > 2. Should this be represented as an NR_* vmstat counter?
>
> I don't think so.
>
> > 3. Is /proc/meminfo an acceptable interface for this information?
>
> Again, I don't think so. "Lost RAM" really is just "excessive memory allocated
> by some other subsystem".
>
> I agree that some users might want to figure out what is consuming that much
> memory, but I don't think growing /proc/meminfo in that way is really what we want.
>
> > 4. Is counting only unique physical backing pages the correct accounting unit?
>
> I'd assume the "It would exclude" part above should not be accounted there, if
> that's what you mean.
>
> --
> Cheers,
>
> David

Re: [External Mail]Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM

[Andrii Nakryiko]

On Tue, Aug 18, 2026 at 6:46 AM 高翔 <gaoxiang17@xiaomi.com> wrote:
>
> Thanks for the pointer. Understood — no new NR_* counter or
> /proc/meminfo entry.
>
>
> The remaining question is on the consumer side: Android's Lost RAM
> accounting would need to enumerate all live BPF ringbuf maps
> (BPF_MAP_GET_NEXT_ID) and read each map's fdinfo memlock to sum them.
>

For BPF ringbufs specifically, you should be fine just iterating all
map with BPF_MAP_GET_NEXT_ID, getting its FD with
BPF_BTF_GET_FD_BY_ID, and then passing that fd to
BPF_OBJ_GET_INFO_BY_FD to get map's size.

>
> Is per-map fdinfo enumeration the intended way for userspace to get the
> aggregate, or is there a more efficient BPF-specific aggregate interface
> that I'm missing?
>



>
> ________________________________
> 发件人: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> 发送时间: 2026年8月18日 2:22:07
> 收件人: David Hildenbrand (Arm)
> 抄送: Xiang Gao; Andrii Nakryiko; Alexei Starovoitov; Daniel Borkmann; Andrew Morton; 印闯; bpf@vger.kernel.org; linux-mm@kvack.org; linux-fsdevel@vger.kernel.org; linux-kernel@vger.kernel.org; 高翔; Lorenzo Stoakes (Arm); Steven Rostedt
> 主题: [External Mail]Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM
>
> [外部邮件] 此邮件来源于小米公司外部，请谨慎处理。若对邮件安全性存疑，请将邮件转发给misec@xiaomi.com进行反馈
>
> On Mon, Aug 17, 2026 at 11:10 AM David Hildenbrand (Arm)
> <david@kernel.org> wrote:
> >
> > On 8/15/26 11:18, Xiang Gao wrote:
> > > Hi,
> >
> > Hi,
> >
> > >
> > > I would like to discuss accounting BPF ring buffer backing pages in
> > > system-wide memory reports.
> > >
> > > BPF ring buffers allocate their data and metadata as order-0 pages directly
> > > from the buddy allocator, and then map those pages with vmap().
> >
> > I assume there is a reason the slab isn't used, right? Are these pages mapped
> > into user space such that page->mapcount would get used?
> >
> > Can you point me at relevant code?
>
> See code in [0]. And yes, these pages are meant to be mapped into user space.
>
>   [0] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/kernel/bpf/ringbuf.c#n93
>
> >
> > >
> > > Because vmap() maps caller-owned pages, these backing pages are not counted
> > > by VmallocUsed. They are also not slab pages. As a result, most BPF ring
> > > buffer memory is not represented by an existing named /proc/meminfo category
> > > and appears as Lost RAM in Android memory reports.
> > >
> > > We measured this on an Android 6.18 kernel.
> > >
> > > Test case:
> > >
> > >   32 BPF ring buffer maps
> > >   16 MiB data area per map
> > >   512 MiB total data area
> > >
> > > Observed changes:
> > >
> > >   Lost RAM:       approximately +529 MiB
> > >   VmallocUsed:      approximately +2 MiB
> > >   Slab:        approximately unchanged
> > >
> > > After destroying all maps, the values returned close to baseline.
> > >
> > > The question is whether the kernel should expose the unique physical backing
> > > pages of live BPF ring buffers through a dedicated global counter and a
> > > /proc/meminfo entry, for example:
> > >
> > >   BpfRingbuf: <value in kB>
> >
> > This looks a bit too specific for my taste. And I think we should try to no
> > inflate these statistics here too much.
> >
>
> +1, way too specific
>
> > >
> > > The proposed counter would include:
> > >
> > >   * ring buffer data pages;
> > >   * metadata pages;
> > >   * consumer and producer position pages.
> > >
> > > It would exclude:
> > >
> > >   * the second virtual mapping of data pages;
> > >   * the pages[] pointer array;
> > >   * map metadata allocations;
> > >   * vmap page tables.
> > >
> > > The goal is to account for the currently unclassified direct backing pages.
> > > Slab- and vmalloc-backed auxiliary allocations are already represented by
> > > existing memory categories and should not be counted again.
> > >
> > > A possible implementation is an NR_BPF_RINGBUF vmstat counter maintained by
> > > the ring buffer allocation and free paths, with the aggregate exposed through
> > > /proc/meminfo.
> > >
> > > Questions:
> > >
> > > 1. Is a dedicated BPF ring buffer counter appropriate?
> >
> > I don't think so.
> >
> > See [1] where we just had the same discussion for tracing buffers. For them,
> > Steve [2] had an idea on how to expose them more fine-grained and tracing specific.
> >
> > [1] https://lore.kernel.org/r/20260810094025.136705-1-gaoxiang17@xiaomi.com
> > [2] https://lore.kernel.org/r/20260810105710.6ee5e493@gandalf.local.home
> >
>
> We already report per-BPF ringbuf memory usage either through bpf()
> syscall or map's fdinfo. E.g., with `sudo bpftool map show` you'll
> see"
>
> 1455733: ringbuf  name event_ringbuf  flags 0x0
>         key 0B  value 0B  max_entries 262144  memlock 275776B
>         btf_id 2193434
>         pids tcpeventd(2549812)
>
> where memlock is how much memory is allocated for the ringbuf data area.
>
> > > 2. Should this be represented as an NR_* vmstat counter?
> >
> > I don't think so.
> >
> > > 3. Is /proc/meminfo an acceptable interface for this information?
> >
> > Again, I don't think so. "Lost RAM" really is just "excessive memory allocated
> > by some other subsystem".
> >
> > I agree that some users might want to figure out what is consuming that much
> > memory, but I don't think growing /proc/meminfo in that way is really what we want.
> >
> > > 4. Is counting only unique physical backing pages the correct accounting unit?
> >
> > I'd assume the "It would exclude" part above should not be accounted there, if
> > that's what you mean.
> >
> > --
> > Cheers,
> >
> > David

Re: [External Mail]Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM

[Isaac Manjarres]

On Wed, Aug 19, 2026 at 10:24:28AM -0700, Andrii Nakryiko wrote:
> On Tue, Aug 18, 2026 at 6:46 AM 高翔 <gaoxiang17@xiaomi.com> wrote:
> >
> > Thanks for the pointer. Understood — no new NR_* counter or
> > /proc/meminfo entry.
> >
> >
> > The remaining question is on the consumer side: Android's Lost RAM
> > accounting would need to enumerate all live BPF ringbuf maps
> > (BPF_MAP_GET_NEXT_ID) and read each map's fdinfo memlock to sum them.
> >
> 
> For BPF ringbufs specifically, you should be fine just iterating all
> map with BPF_MAP_GET_NEXT_ID, getting its FD with
> BPF_BTF_GET_FD_BY_ID, and then passing that fd to
> BPF_OBJ_GET_INFO_BY_FD to get map's size.
> 
Hi Andrii,

Thanks for the suggestion on this! I did want to express a couple of
concerns with this:

Scalability

I counted the number of maps on one of our devices, and there are 112
maps, meaning that there will be between 224-336 syscalls with this
approach. eBPF is becoming more popular, so I'm concerned about how well
this will scale, if we have to invoke 2-3 syscalls per map.

I had a test program that implemented your suggestion, and it took about
2 ms to identify 39/112 ringbufs. As the number of maps in the system
grows, I'm concerned that the latency associated with computing the
memory usage from ringbufs will become even more expensive. This is
something we had an issue with before on Android, where we had to
iterate through various sysfs files to gather wakeupsource metrics [1].

To improve on this, I was wondering if we could expose the ringbuf
memory usage and potentially other bpf stats through bpffs
(/sys/fs/bpf/stats)? This counter could be a lightweight counter that is
incremented/decremented on ringbuf allocation/freeing so that when it is
read, there aren't any expensive computations.

For this specific metric, we could just use a counter to track how much
memory is being used by ringbufs and have userspace read that. That also
brings me to my next point.

Correctness

The max_entries value is the size of the data in the ringbufs.
However, it doesn't capture the 3 metadata pages associated with each
ringbuf, which leaves a gap of ~468 KB, and that gap can keep growing
as the number of ringbufs increases. It's important to have as much
information as to where memory is being allocated to, as there are
devices with as little as 2 GB of memory that we need to be able to
profile memory usage with.

I think exposing the sum of ringbuf data + metadata pages through the
node I proposed earlier would help achieve this.

[1] https://lore.kernel.org/all/20260511174559.659782-1-wusamuel@google.com/

Thanks,
Isaac

Re: [External Mail]Re: [RFC] bpf: account ring buffer backing pages separately from Lost RAM

[Andrii Nakryiko]

On Fri, Sep 11, 2026 at 4:24 PM Isaac Manjarres
<isaacmanjarres@google.com> wrote:
>
> On Wed, Aug 19, 2026 at 10:24:28AM -0700, Andrii Nakryiko wrote:
> > On Tue, Aug 18, 2026 at 6:46 AM 高翔 <gaoxiang17@xiaomi.com> wrote:
> > >
> > > Thanks for the pointer. Understood — no new NR_* counter or
> > > /proc/meminfo entry.
> > >
> > >
> > > The remaining question is on the consumer side: Android's Lost RAM
> > > accounting would need to enumerate all live BPF ringbuf maps
> > > (BPF_MAP_GET_NEXT_ID) and read each map's fdinfo memlock to sum them.
> > >
> >
> > For BPF ringbufs specifically, you should be fine just iterating all
> > map with BPF_MAP_GET_NEXT_ID, getting its FD with
> > BPF_BTF_GET_FD_BY_ID, and then passing that fd to
> > BPF_OBJ_GET_INFO_BY_FD to get map's size.
> >
> Hi Andrii,
>
> Thanks for the suggestion on this! I did want to express a couple of
> concerns with this:
>
> Scalability
>
> I counted the number of maps on one of our devices, and there are 112
> maps, meaning that there will be between 224-336 syscalls with this
> approach. eBPF is becoming more popular, so I'm concerned about how well
> this will scale, if we have to invoke 2-3 syscalls per map.
>
> I had a test program that implemented your suggestion, and it took about
> 2 ms to identify 39/112 ringbufs. As the number of maps in the system
> grows, I'm concerned that the latency associated with computing the
> memory usage from ringbufs will become even more expensive. This is
> something we had an issue with before on Android, where we had to
> iterate through various sysfs files to gather wakeupsource metrics [1].
>
> To improve on this, I was wondering if we could expose the ringbuf
> memory usage and potentially other bpf stats through bpffs
> (/sys/fs/bpf/stats)? This counter could be a lightweight counter that is
> incremented/decremented on ringbuf allocation/freeing so that when it is
> read, there aren't any expensive computations.
>
> For this specific metric, we could just use a counter to track how much
> memory is being used by ringbufs and have userspace read that. That also
> brings me to my next point.
>

I just don't see a good enough reason to single out ringbuf maps
specifically. other map types also use memory, why would they be
excluded?

If you are worried about too many syscalls, look into map iterator
program types (grep for SEC("iter/bpf_map") in selftests). That will
be super fast and way more generic than what you propose. You can ping
such program in bpffs and that will be you custom /sys/fs/bpf/stats
implementation that you have full control and customizability of

> Correctness
>
> The max_entries value is the size of the data in the ringbufs.
> However, it doesn't capture the 3 metadata pages associated with each
> ringbuf, which leaves a gap of ~468 KB, and that gap can keep growing
> as the number of ringbufs increases. It's important to have as much
> information as to where memory is being allocated to, as there are
> devices with as little as 2 GB of memory that we need to be able to
> profile memory usage with.
>
> I think exposing the sum of ringbuf data + metadata pages through the
> node I proposed earlier would help achieve this.
>
> [1] https://lore.kernel.org/all/20260511174559.659782-1-wusamuel@google.com/
>
> Thanks,
> Isaac