Re: [PATCH bpf-next v3 0/5] bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

[Song Liu <song@kernel.org>]

On Fri, Jul 17, 2026 at 2:29 AM Mike Rapoport <rppt@kernel.org> wrote:
[...]
> > >
> > > "It works, don't touch"? ;-)
> > >
> > > We can add another layer for sub-page allocations to execmem.
> >
> > Sub-page allocation is not a hard requirement here. Using 4kB for
> > each small BPF program isn't too bad. We added bpf_prog_pack to
> > avoid fragmentation of direct map page table entry (caused by W^X
> > requirement). If execmem allocator reserves large enough ROX
> > memory (with PMD page table entries) and reuses them properly,
> > we shouldn't see page table fragmentation getting worse over time.
> > Then, we can use 4kB granularity allocation for BPF programs. (I am
> > not sure about 64kB pages..).
>
> execmem uses PMD_SIZE pages for ROX cache. Since right now it's only
> supported on x86, the issue with oversized large page with 64k base pages
> didn't come up yet.

With 64kB page size, PMD_SIZE is probably too big for execmem.

> > > Since BPF is the only user the easiest would be just to move prog_pack
> > > logic from BPF to execmem and call it a day.
> >
> > If we move to bigger page sizes, say 64kB, there will be other
> > users that would benefit from sub page allocation, right?
>
> Maybe modules could, don't know TBH.
> There is another caveat with sub-page allocations: they must preserve ROX
> and no subsystem except BPF can deal with ROX-only allocations and writable
> copies.

I think ftrace and kprobe could be good users?

> > > Another option is to add a slab-like layer for sub-page allocations to
> > > execmem. This is more complex but it would allow to get rid of the rigid
> > > BPF_PROG_CHUNK_SIZE.
> > >
> > > Maybe it would be also possible to teach SLUB to use execmem_alloc()
> > > instead of alloc_pages() but that's surely the most far fetched one :)
> >
> > I was thinking some rb-tree algorithm might be useful here,
> > something similar to vmap.
>
> Could be that or maple_tree especially considering there's an active
> discussion about converting vmalloc to maple_tree as well:
>
> https://lore.kernel.org/linux-mm/20260613-vmalloc_maple-v1-0-0aa740bb944b@oss.qualcomm.com/
>
> But the choice of allocation algorithm is anyway secondary to the decision
> whether execmem should have the sub-page allocator.
>
> If you say that a page per BPF program is fine, than maybe we can just rip
> off bpf_prog_pack_alloc().
> If BPF still needs to maintain the ability to allocate in smaller chunks,
> then maybe the best way is to keep bpf_prog_pack_alloc() as is because I
> don't foresee other users for small page allocations any time soon.
>
> > > And since we are talking about bpf_prog_pack_alloc(), why
> > > BPF_PROG_PACK_SIZE accounts for num_possible_nodes():
> > >
> > > #define BPF_PROG_PACK_SIZE (SZ_2M * num_possible_nodes())
> > >
> > > Is it an elaborate choice or it was picked to work around older
> > > vmalloc_huge() limitations?
> >
> > It is a bit complicated. The goal is to get PMDs for prog_pack.
> > We can adjust this if vmalloc_huge() changes after that.
>
> Since commit c82be0be9576 ("mm: vmalloc: don't account for number of nodes
> for HUGE_VMAP allocations") vmalloc_huge() is fine with PMD_SIZE regardless
> of number of nodes.

Before execmem can handle sub page allocations, how about we
send allocations that are bigger than page size directly to execmem,
and let bpf_prog_pack handle sub page allocations. Then,
BPF_PROG_PACK_SIZE will be PAGE_SIZE. This should be a net
win for x86_64. Other archs will be the same. WDYT?

Thanks,
Song