Re: [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues

["Qi Zheng" <zhengqi.arch@bytedance.com>]

Hi David,

On 11/12/26 6:05 AM, David Hildenbrand (Arm) wrote:
> On 11/7/26 11:30, Qi Zheng wrote:
>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>
>> Hi all,
>>
>> This series fixes three CMA-related issues in HugeTLB reservation,
>> long-term pinning and allocation:
>>
> 
> Can I hand you over another CMA-related hugetlb problem?
> 
> On arm64, we can end up allocating 1GiB and 32MiB hugetlb folios from the cma
> region.
> 
> Doing that dynamically is not a problem. Doing it during boot doesn't quite work.
> 
> cma_reserve_early() does not handle any kind of alignment.
> 
> So if you allocate a 32M folio followed by a 1G folio, the 1G folio will
> suddenly start at offset 32M, not being naturally aligned in physical address space.

Yes.

> 
> Likely cma_reserve_early() must be taught to align (and maybe consume an order
> instead of a size), similar to how the dynamic cma allocations work.

However, simply adding alignment would leave gaps that the
current activation code cannot distinguish from actual reservations.

The early allocation path would become:

hugetlb_cma_alloc_bootmem()
   -> cma_reserve_early()
        pfn = ALIGN(cmr->early_pfn, nr_pages);
        cmr->early_pfn = pfn + nr_pages;
        cma->available_count -= nr_pages;

For a CMA range starting at 4 GiB, reserving 32 MiB followed by 1 GiB
would produce:

4 GiB       4 GiB + 32 MiB             5 GiB             6 GiB
| 32 MiB used |------ 992 MiB gap ------| 1 GiB used       |
                                                         ^
                                                     early_pfn

Later, cma_activate_area() marks the entire prefix below early_pfn as
allocated, including the gap:

count = early_pfn[r] - cmr->base_pfn;
bitmap_count = cma_bitmap_pages_to_bits(cma, count);
bitmap_set(cmr->bitmap, 0, bitmap_count);

It only initializes and frees pageblocks after that cursor:

for (pfn = early_pfn[r]; pfn < cmr->base_pfn + cmr->count;
      pfn += pageblock_nr_pages)
	init_cma_reserved_pageblock(pfn_to_page(pfn));

The gap therefore remains reserved and marked allocated. Releasing the
two HugeTLB folios only clears their own ranges through
cma_release_frozen(); neither release covers the gap. Those 992 MiB
would be permanently lost from usable CMA capacity.

So we would also need to track the gaps, clear their bitmap bits during
activation, and initialize them as free CMA pageblocks.

> 

Another possibility would be to give 32 MiB pages their own pool,
similar to the separate PMD-sized pool proposed in the other series. On
arm64 with 4 KiB base pages, 2 MiB, 32 MiB and 1 GiB allocations would
then use separate pools.

This could work during early boot too: hugetlb_cma_reserve() establishes
the CMA areas and their ranges before hugetlb_bootmem_alloc() runs.
Although the bitmaps are allocated later, each range already has its
own early_pfn. With naturally aligned ranges and one allocation size
per pool, the cursors would stay aligned without creating gaps.

What do you think?

Thanks,
Qi