[PATCH 0/4] mm: hugetlb: fix some CMA-related issues

[PATCH 0/4] mm: hugetlb: fix some CMA-related issues

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Hi all,

This series fixes four CMA-related issues in HugeTLB reservation and
allocation:

 - Avoid ineffective CMA reservations on 32-bit x86 kernels running on
   CPUs that advertise 1 GiB page support.
 - Honor PF_MEMALLOC_PIN to avoid allocating CMA-backed folios during
   long-term pinning.
 - Skip CMA allocation when the caller disallows direct reclaim.
 - Enforce cpuset restrictions on surplus allocations, preserving memory
   policy semantics and handling concurrent cpuset changes without
   discarding folios that remain eligible.

Thanks,
Qi

Qi Zheng (4):
  x86: hugetlb: disable default CMA reservations on 32-bit kernel
  mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN
  mm: hugetlb: skip CMA when direct reclaim is disallowed
  mm: hugetlb: respect cpuset restrictions for surplus allocations

 arch/x86/mm/hugetlbpage.c |  2 +-
 mm/hugetlb.c              | 38 +++++++++++++++++++++++++++++++-------
 mm/hugetlb_cma.c          |  5 ++++-
 3 files changed, 36 insertions(+), 9 deletions(-)


base-commit: 2637bdec0535a2044cd830eba9a79363651d9043
-- 
2.54.0

[PATCH 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.
arch_hugetlb_cma_order() checks only this feature and returns a nonzero
order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
serve the intended gigantic HugeTLB allocations.

On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
warning that hugetlb_cma is attempting to reserve a non-gigantic order.

Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
order to avoid these ineffective reservations.

Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
Cc: stable@vger.kernel.org
Cc: x86@kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 arch/x86/mm/hugetlbpage.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index 3b26621c91286..70726a99ede7b 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
 
 unsigned int __init arch_hugetlb_cma_order(void)
 {
-	if (boot_cpu_has(X86_FEATURE_GBPAGES))
+	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
 		return PUD_SHIFT - PAGE_SHIFT;
 
 	return 0;
-- 
2.54.0

[PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

With gigantic overcommit enabled, a fault during long-term pinning
can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.

The folio must then be migrated before it can be pinned. However,
fresh gigantic migration targets are not supported, so pinning fails
when the HugeTLB pool has no suitable target, even if the original
fault could have allocated pinnable memory outside CMA.

Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
try the non-CMA fallback, or fail when allocation is restricted to CMA.

Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb_cma.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index 95fd2d190f0d2..dde1c866406fb 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	if (!hugetlb_cma_size)
 		return NULL;
 
+	if (current->flags & PF_MEMALLOC_PIN)
+		return NULL;
+
 retry_cpuset:
 	if (!nodemask) {
 		cpuset_mems_cookie = read_mems_allowed_begin();
-- 
2.54.0

[PATCH 3/4] mm: hugetlb: skip CMA when direct reclaim is disallowed

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Switching HugeTLB CMA allocation from cma_alloc_folio() to
cma_alloc_frozen_compound() stopped forwarding the caller's GFP mask.
The new helper uses GFP_KERNEL, re-enabling direct reclaim even when
the caller explicitly disabled it.

The first MPOL_PREFERRED_MANY allocation attempt clears
__GFP_DIRECT_RECLAIM to try the preferred nodes before falling back to
other nodes. For gigantic surplus pages, the CMA path can therefore
enter direct reclaim while allocating migration targets during this
attempt.

Skip HugeTLB CMA allocation when direct reclaim is disallowed.

Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb_cma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index dde1c866406fb..90f8d718a1f16 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -59,7 +59,7 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	const nodemask_t *nmask;
 	unsigned int cpuset_mems_cookie;
 
-	if (!hugetlb_cma_size)
+	if (!hugetlb_cma_size || !gfpflags_allow_blocking(gfp_mask))
 		return NULL;
 
 	if (current->flags & PF_MEMALLOC_PIN)
-- 
2.54.0

[PATCH 4/4] mm: hugetlb: respect cpuset restrictions for surplus allocations

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Gigantic surplus allocations can escape the faulting task's cpuset.
CMA and the contiguous allocator trust the supplied nodemask without
checking the current cpuset.

For example, a shared HugeTLB VMA has MPOL_BIND={0,1}, and T is a
non-leader thread. With surplus allocation enabled and no eligible free
HugeTLB folio, the following sequence can occur:

  CPU0 (cgroup v1 migration)       CPU1 (thread T)
  -------------------------       ---------------
  cpuset_attach_task(..., T)
    T->mems_allowed = {0}
    /* Non-leader: skip
       mpol_rebind_mm(). */
                                  /* Fault; VMA policy is still {0,1}. */
                                  policy_nodemask()
                                    /* {0,1} intersects {0}. */
                                    nodemask = {0,1}
                                  hugetlb_cma_alloc_frozen_folio()
                                    /* node0 fails; node1 succeeds. */

The 1 GiB folio is allocated on node1 despite T being restricted to node0.

Intersect the interpreted policy with the current cpuset before surplus
allocation, preserving MPOL_BIND and MPOL_PREFERRED_MANY fallback rules.
Apply the restriction at the policy-aware entry point to preserve
per-node pool management semantics.

If the cpuset changes during allocation, rebuild the mask and revalidate
any allocated folio against the current constraints. Keep it if it is
still eligible: freeing an HVO-optimized folio can defer returning its
range to CMA, so discarding a usable folio can cause a needless allocation
failure. Only release the folio and allocate again if it is no longer
eligible.

Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Cc: stable@vger.kernel.org
Reported-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb.c | 38 +++++++++++++++++++++++++++++++-------
 1 file changed, 31 insertions(+), 7 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ce9c8e57b8e5c..6fd6e58ace869 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2181,22 +2181,46 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
 		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
 {
 	struct folio *folio = NULL;
-	nodemask_t *nodemask = mpoli->nodemask;
+	nodemask_t alloc_nodemask;
+	unsigned int cpuset_mems_cookie;
+
+retry:
+	cpuset_mems_cookie = read_mems_allowed_begin();
+	alloc_nodemask = cpuset_current_mems_allowed;
+	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
+	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
+
+	if (folio) {
+		if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
+			goto retry;
+
+		if (node_isset(folio_nid(folio), alloc_nodemask))
+			return folio;
+
+		/* Balance free_huge_folio()'s decrement before caller accounting */
+		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
+		free_huge_folio(folio);
+		folio = NULL;
+	}
 
 	if (mpoli->mode == MPOL_PREFERRED_MANY) {
 		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
 
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
 		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
-						    nodemask);
+						    &alloc_nodemask);
 
-		/* Fallback to all nodes if page==NULL */
-		nodemask = NULL;
+		/* Fallback to all nodes allowed by the current cpuset */
+		alloc_nodemask = cpuset_current_mems_allowed;
 	}
 
-	if (!folio) {
+	if (!folio)
 		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
-						    nodemask);
-	}
+						    &alloc_nodemask);
+
+	if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
+		goto retry;
 
 	return folio;
 }
-- 
2.54.0

Re: [PATCH 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.

I don't think anyone would do that. That said, it is genuinely an issue.

> arch_hugetlb_cma_order() checks only this feature and returns a nonzero
> order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
> Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
> serve the intended gigantic HugeTLB allocations.
> 
> On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
> equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
> VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
> warning that hugetlb_cma is attempting to reserve a non-gigantic order.
> 
> Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
> order to avoid these ineffective reservations.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Cc: x86@kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> With gigantic overcommit enabled, a fault during long-term pinning
> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
> 
> The folio must then be migrated before it can be pinned. However,
> fresh gigantic migration targets are not supported, so pinning fails
> when the HugeTLB pool has no suitable target, even if the original
> fault could have allocated pinnable memory outside CMA.
> 
> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
> try the non-CMA fallback, or fail when allocation is restricted to CMA.
> 
> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

LGTM.

Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> With gigantic overcommit enabled, a fault during long-term pinning
> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
> 
> The folio must then be migrated before it can be pinned. However,
> fresh gigantic migration targets are not supported, so pinning fails
> when the HugeTLB pool has no suitable target, even if the original
> fault could have allocated pinnable memory outside CMA.
> 
> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
> try the non-CMA fallback, or fail when allocation is restricted to CMA.

Wait... I think my Acked-by got ahead of me here. :)

There seems to be a rather odd distinction that GUP is missing.

The usual reason long-term GUP must migrate a page out of a
MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
normal page allocator.  The corresponding CMA bitmap range is still
available, so a future CMA allocation relies on being able to migrate
that page away.  A long-term pin would prevent that.

However, a HugeTLB folio returned by
hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
allocation, not merely a movable page that happens to reside in a
MIGRATE_CMA pageblock.  Its range has already been marked allocated in
the hugetlb CMA bitmap.

Therefore, CMA cannot allocate that range again until the HugeTLB folio
is freed anyway.  Long-term pinning the folio does not seem to make the
CMA area any less available than keeping the HugeTLB folio mapped for
the same amount of time.

folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
not distinguish these two cases.  In that light, rejecting
PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
around an overly broad GUP classification.  It also means that
hugetlb_cma_only users will fail the fault, while other users may consume
a gigantic contiguous range outside the dedicated hugetlb CMA area even
when that area has free space.

Would it make more sense to fix the long-term GUP check instead, so that
an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?

Thanks,
Muchun

> 
> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
> ---
> mm/hugetlb_cma.c | 3 +++
> 1 file changed, 3 insertions(+)
> 
> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
> index 95fd2d190f0d2..dde1c866406fb 100644
> --- a/mm/hugetlb_cma.c
> +++ b/mm/hugetlb_cma.c
> @@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
> 	if (!hugetlb_cma_size)
> 		return NULL;
> 
> +	if (current->flags & PF_MEMALLOC_PIN)
> +		return NULL;
> +
> retry_cpuset:
> 	if (!nodemask) {
> 		cpuset_mems_cookie = read_mems_allowed_begin();
> -- 
> 2.54.0
>