[PATCH 0/4] mm: hugetlb: fix some CMA-related issues

[PATCH 0/4] mm: hugetlb: fix some CMA-related issues

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Hi all,

This series fixes four CMA-related issues in HugeTLB reservation and
allocation:

 - Avoid ineffective CMA reservations on 32-bit x86 kernels running on
   CPUs that advertise 1 GiB page support.
 - Honor PF_MEMALLOC_PIN to avoid allocating CMA-backed folios during
   long-term pinning.
 - Skip CMA allocation when the caller disallows direct reclaim.
 - Enforce cpuset restrictions on surplus allocations, preserving memory
   policy semantics and handling concurrent cpuset changes without
   discarding folios that remain eligible.

Thanks,
Qi

Qi Zheng (4):
  x86: hugetlb: disable default CMA reservations on 32-bit kernel
  mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN
  mm: hugetlb: skip CMA when direct reclaim is disallowed
  mm: hugetlb: respect cpuset restrictions for surplus allocations

 arch/x86/mm/hugetlbpage.c |  2 +-
 mm/hugetlb.c              | 38 +++++++++++++++++++++++++++++++-------
 mm/hugetlb_cma.c          |  5 ++++-
 3 files changed, 36 insertions(+), 9 deletions(-)


base-commit: 2637bdec0535a2044cd830eba9a79363651d9043
-- 
2.54.0

[PATCH 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.
arch_hugetlb_cma_order() checks only this feature and returns a nonzero
order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
serve the intended gigantic HugeTLB allocations.

On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
warning that hugetlb_cma is attempting to reserve a non-gigantic order.

Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
order to avoid these ineffective reservations.

Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
Cc: stable@vger.kernel.org
Cc: x86@kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 arch/x86/mm/hugetlbpage.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index 3b26621c91286..70726a99ede7b 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
 
 unsigned int __init arch_hugetlb_cma_order(void)
 {
-	if (boot_cpu_has(X86_FEATURE_GBPAGES))
+	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
 		return PUD_SHIFT - PAGE_SHIFT;
 
 	return 0;
-- 
2.54.0

Re: [PATCH 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.

I don't think anyone would do that. That said, it is genuinely an issue.

> arch_hugetlb_cma_order() checks only this feature and returns a nonzero
> order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
> Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
> serve the intended gigantic HugeTLB allocations.
> 
> On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
> equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
> VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
> warning that hugetlb_cma is attempting to reserve a non-gigantic order.
> 
> Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
> order to avoid these ineffective reservations.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Cc: x86@kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Jose A. Perez de Azpillaga]

On 11/3/26 9:01 AM, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.

weird. anyway, LGTM:

Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

> arch_hugetlb_cma_order() checks only this feature and returns a nonzero
> order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
> Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
> serve the intended gigantic HugeTLB allocations.
> 
> On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
> equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
> VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
> warning that hugetlb_cma is attempting to reserve a non-gigantic order.
> 
> Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
> order to avoid these ineffective reservations.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Cc: x86@kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
> ---
>  arch/x86/mm/hugetlbpage.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
> index 3b26621c91286..70726a99ede7b 100644
> --- a/arch/x86/mm/hugetlbpage.c
> +++ b/arch/x86/mm/hugetlbpage.c
> @@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
>  
>  unsigned int __init arch_hugetlb_cma_order(void)
>  {
> -	if (boot_cpu_has(X86_FEATURE_GBPAGES))
> +	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
>  		return PUD_SHIFT - PAGE_SHIFT;
>  
>  	return 0;


-- 
cheers,
jose a. p-a

[PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

With gigantic overcommit enabled, a fault during long-term pinning
can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.

The folio must then be migrated before it can be pinned. However,
fresh gigantic migration targets are not supported, so pinning fails
when the HugeTLB pool has no suitable target, even if the original
fault could have allocated pinnable memory outside CMA.

Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
try the non-CMA fallback, or fail when allocation is restricted to CMA.

Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb_cma.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index 95fd2d190f0d2..dde1c866406fb 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	if (!hugetlb_cma_size)
 		return NULL;
 
+	if (current->flags & PF_MEMALLOC_PIN)
+		return NULL;
+
 retry_cpuset:
 	if (!nodemask) {
 		cpuset_mems_cookie = read_mems_allowed_begin();
-- 
2.54.0

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> With gigantic overcommit enabled, a fault during long-term pinning
> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
> 
> The folio must then be migrated before it can be pinned. However,
> fresh gigantic migration targets are not supported, so pinning fails
> when the HugeTLB pool has no suitable target, even if the original
> fault could have allocated pinnable memory outside CMA.
> 
> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
> try the non-CMA fallback, or fail when allocation is restricted to CMA.
> 
> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

LGTM.

Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Muchun Song]

> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> With gigantic overcommit enabled, a fault during long-term pinning
> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
> 
> The folio must then be migrated before it can be pinned. However,
> fresh gigantic migration targets are not supported, so pinning fails
> when the HugeTLB pool has no suitable target, even if the original
> fault could have allocated pinnable memory outside CMA.
> 
> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
> try the non-CMA fallback, or fail when allocation is restricted to CMA.

Wait... I think my Acked-by got ahead of me here. :)

There seems to be a rather odd distinction that GUP is missing.

The usual reason long-term GUP must migrate a page out of a
MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
normal page allocator.  The corresponding CMA bitmap range is still
available, so a future CMA allocation relies on being able to migrate
that page away.  A long-term pin would prevent that.

However, a HugeTLB folio returned by
hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
allocation, not merely a movable page that happens to reside in a
MIGRATE_CMA pageblock.  Its range has already been marked allocated in
the hugetlb CMA bitmap.

Therefore, CMA cannot allocate that range again until the HugeTLB folio
is freed anyway.  Long-term pinning the folio does not seem to make the
CMA area any less available than keeping the HugeTLB folio mapped for
the same amount of time.

folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
not distinguish these two cases.  In that light, rejecting
PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
around an overly broad GUP classification.  It also means that
hugetlb_cma_only users will fail the fault, while other users may consume
a gigantic contiguous range outside the dedicated hugetlb CMA area even
when that area has free space.

Would it make more sense to fix the long-term GUP check instead, so that
an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?

Thanks,
Muchun

> 
> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
> ---
> mm/hugetlb_cma.c | 3 +++
> 1 file changed, 3 insertions(+)
> 
> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
> index 95fd2d190f0d2..dde1c866406fb 100644
> --- a/mm/hugetlb_cma.c
> +++ b/mm/hugetlb_cma.c
> @@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
> 	if (!hugetlb_cma_size)
> 		return NULL;
> 
> +	if (current->flags & PF_MEMALLOC_PIN)
> +		return NULL;
> +
> retry_cpuset:
> 	if (!nodemask) {
> 		cpuset_mems_cookie = read_mems_allowed_begin();
> -- 
> 2.54.0
> 

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Jose A. Perez de Azpillaga]

On 11/3/26 11:56 AM, Muchun Song wrote:
> 
> 
>> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
>>
>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>
>> With gigantic overcommit enabled, a fault during long-term pinning
>> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
>>
>> The folio must then be migrated before it can be pinned. However,
>> fresh gigantic migration targets are not supported, so pinning fails
>> when the HugeTLB pool has no suitable target, even if the original
>> fault could have allocated pinnable memory outside CMA.
>>
>> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
>> try the non-CMA fallback, or fail when allocation is restricted to CMA.
> 
> Wait... I think my Acked-by got ahead of me here. :)
> 
> There seems to be a rather odd distinction that GUP is missing.
> 
> The usual reason long-term GUP must migrate a page out of a
> MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
> normal page allocator.  The corresponding CMA bitmap range is still
> available, so a future CMA allocation relies on being able to migrate
> that page away.  A long-term pin would prevent that.
> 
> However, a HugeTLB folio returned by
> hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
> allocation, not merely a movable page that happens to reside in a
> MIGRATE_CMA pageblock.  Its range has already been marked allocated in
> the hugetlb CMA bitmap.
> 
> Therefore, CMA cannot allocate that range again until the HugeTLB folio
> is freed anyway.  Long-term pinning the folio does not seem to make the
> CMA area any less available than keeping the HugeTLB folio mapped for
> the same amount of time.
> 
> folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
> not distinguish these two cases.  In that light, rejecting
> PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
> around an overly broad GUP classification.  It also means that
> hugetlb_cma_only users will fail the fault, while other users may consume
> a gigantic contiguous range outside the dedicated hugetlb CMA area even
> when that area has free space.
> 
> Would it make more sense to fix the long-term GUP check instead, so that
> an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
> ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?
> 

makes sense to me. with hugetlb_cma_only=1 a 1G FOLL_LONGTERM pin can't
succeed at all today, even on preallocated pages.


-- 
cheers,
jose a. p-a

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Qi Zheng]

Hi Muchun,

On 11/3/26 6:56 PM, Muchun Song wrote:
> 
> 
>> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
>>
>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>
>> With gigantic overcommit enabled, a fault during long-term pinning
>> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
>>
>> The folio must then be migrated before it can be pinned. However,
>> fresh gigantic migration targets are not supported, so pinning fails
>> when the HugeTLB pool has no suitable target, even if the original
>> fault could have allocated pinnable memory outside CMA.
>>
>> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
>> try the non-CMA fallback, or fail when allocation is restricted to CMA.
> 
> Wait... I think my Acked-by got ahead of me here. :)
> 
> There seems to be a rather odd distinction that GUP is missing.
> 
> The usual reason long-term GUP must migrate a page out of a
> MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
> normal page allocator.  The corresponding CMA bitmap range is still
> available, so a future CMA allocation relies on being able to migrate
> that page away.  A long-term pin would prevent that.
> 
> However, a HugeTLB folio returned by
> hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
> allocation, not merely a movable page that happens to reside in a
> MIGRATE_CMA pageblock.  Its range has already been marked allocated in
> the hugetlb CMA bitmap.
> 
> Therefore, CMA cannot allocate that range again until the HugeTLB folio
> is freed anyway.  Long-term pinning the folio does not seem to make the
> CMA area any less available than keeping the HugeTLB folio mapped for
> the same amount of time.
> 
> folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
> not distinguish these two cases.  In that light, rejecting
> PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
> around an overly broad GUP classification.  It also means that
> hugetlb_cma_only users will fail the fault, while other users may consume
> a gigantic contiguous range outside the dedicated hugetlb CMA area even
> when that area has free space.
> 
> Would it make more sense to fix the long-term GUP check instead, so that
> an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
> ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?

Make sense!

I'll replace this patch with an exception for HPG_cma folios in the
MIGRATE_CMA check. This also lets the HugeTLB allocator select preallocated
CMA folios during long-term pinning. MIGRATE_ISOLATE and ZONE_MOVABLE
remain restricted.

Fresh CMA allocations under PF_MEMALLOC_PIN also need to skip ranges
in ZONE_MOVABLE. Returning -EBUSY here lets CMA try its remaining ranges.

The incremental diff against this patch is below.

diff --git a/include/linux/cma.h b/include/linux/cma.h
index 8555d38a97b1..7fd1b6e65b58 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -21,11 +21,13 @@
  #define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)

  struct cma;
+struct folio;

  extern unsigned long totalcma_pages;
  extern phys_addr_t cma_get_base(const struct cma *cma);
  extern unsigned long cma_get_size(const struct cma *cma);
  extern const char *cma_get_name(const struct cma *cma);
+bool cma_folio_is_pinnable(struct folio *folio);

  extern int __init cma_declare_contiguous_nid(phys_addr_t base,
  			phys_addr_t size, phys_addr_t limit,
diff --git a/include/linux/mm.h b/include/linux/mm.h
index cb993a1d0728..d15810be7d96 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3,6 +3,7 @@
  #define _LINUX_MM_H

  #include <linux/args.h>
+#include <linux/cma.h>
  #include <linux/errno.h>
  #include <linux/mmdebug.h>
  #include <linux/gfp.h>
@@ -2975,14 +2976,16 @@ static inline bool is_zero_folio(const struct 
folio *folio)
  	return is_zero_page(&folio->page);
  }

-/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
+/* Borrowed CMA pages and ZONE_MOVABLE folios cannot be pinned long 
term. */
  #ifdef CONFIG_MIGRATION
  static inline bool folio_is_longterm_pinnable(struct folio *folio)
  {
  #ifdef CONFIG_CMA
  	int mt = folio_migratetype(folio);

-	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
+	if (mt == MIGRATE_ISOLATE)
+		return false;
+	if (mt == MIGRATE_CMA && !cma_folio_is_pinnable(folio))
  		return false;
  #endif
  	/* The zero page can be "pinned" but gets special handling. */
diff --git a/mm/cma.c b/mm/cma.c
index a10ea37a261d..c2814f54f86a 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -27,6 +27,7 @@
  #include <linux/log2.h>
  #include <linux/cma.h>
  #include <linux/highmem.h>
+#include <linux/hugetlb.h>
  #include <linux/io.h>
  #include <linux/kmemleak.h>
  #include <trace/events/cma.h>
@@ -55,6 +56,12 @@ const char *cma_get_name(const struct cma *cma)
  }
  EXPORT_SYMBOL_GPL(cma_get_name);

+bool cma_folio_is_pinnable(struct folio *folio)
+{
+	/* Explicit HugeTLB CMA allocations own their bitmap ranges. */
+	return folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio);
+}
+
  static unsigned long cma_bitmap_aligned_mask(const struct cma *cma,
  					     unsigned int align_order)
  {
@@ -792,6 +799,11 @@ static int cma_range_alloc(struct cma *cma, struct 
cma_memrange *cmr,
  	int ret = -EBUSY;
  	struct page *page = NULL;

+	/* Each range lies within one zone; try other ranges if needed. */
+	if ((current->flags & PF_MEMALLOC_PIN) &&
+	    is_zone_movable_page(pfn_to_page(cmr->base_pfn)))
+		return -EBUSY;
+
  	mask = cma_bitmap_aligned_mask(cma, align);
  	offset = cma_bitmap_aligned_offset(cma, cmr, align);
  	bitmap_maxno = cma_bitmap_maxno(cma, cmr);
diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index dde1c866406f..95fd2d190f0d 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -62,9 +62,6 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int 
order, gfp_t gfp_mask,
  	if (!hugetlb_cma_size)
  		return NULL;

-	if (current->flags & PF_MEMALLOC_PIN)
-		return NULL;
-
  retry_cpuset:
  	if (!nodemask) {
  		cpuset_mems_cookie = read_mems_allowed_begin();

Thanks,
Qi

> 
> Thanks,
> Muchun
> 
>>
>> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
>> ---
>> mm/hugetlb_cma.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
>> index 95fd2d190f0d2..dde1c866406fb 100644
>> --- a/mm/hugetlb_cma.c
>> +++ b/mm/hugetlb_cma.c
>> @@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
>> 	if (!hugetlb_cma_size)
>> 		return NULL;
>>
>> +	if (current->flags & PF_MEMALLOC_PIN)
>> +		return NULL;
>> +
>> retry_cpuset:
>> 	if (!nodemask) {
>> 		cpuset_mems_cookie = read_mems_allowed_begin();
>> -- 
>> 2.54.0
>>
> 

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Muchun Song]

> On Nov 4, 2026, at 11:27, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> Hi Muchun,

Hi,

> 
> On 11/3/26 6:56 PM, Muchun Song wrote:
>>> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
>>> 
>>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>> 
>>> With gigantic overcommit enabled, a fault during long-term pinning
>>> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
>>> 
>>> The folio must then be migrated before it can be pinned. However,
>>> fresh gigantic migration targets are not supported, so pinning fails
>>> when the HugeTLB pool has no suitable target, even if the original
>>> fault could have allocated pinnable memory outside CMA.
>>> 
>>> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
>>> try the non-CMA fallback, or fail when allocation is restricted to CMA.
>> Wait... I think my Acked-by got ahead of me here. :)
>> There seems to be a rather odd distinction that GUP is missing.
>> The usual reason long-term GUP must migrate a page out of a
>> MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
>> normal page allocator.  The corresponding CMA bitmap range is still
>> available, so a future CMA allocation relies on being able to migrate
>> that page away.  A long-term pin would prevent that.
>> However, a HugeTLB folio returned by
>> hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
>> allocation, not merely a movable page that happens to reside in a
>> MIGRATE_CMA pageblock.  Its range has already been marked allocated in
>> the hugetlb CMA bitmap.
>> Therefore, CMA cannot allocate that range again until the HugeTLB folio
>> is freed anyway.  Long-term pinning the folio does not seem to make the
>> CMA area any less available than keeping the HugeTLB folio mapped for
>> the same amount of time.
>> folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
>> not distinguish these two cases.  In that light, rejecting
>> PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
>> around an overly broad GUP classification.  It also means that
>> hugetlb_cma_only users will fail the fault, while other users may consume
>> a gigantic contiguous range outside the dedicated hugetlb CMA area even
>> when that area has free space.
>> Would it make more sense to fix the long-term GUP check instead, so that
>> an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
>> ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?
> 
> Make sense!
> 
> I'll replace this patch with an exception for HPG_cma folios in the
> MIGRATE_CMA check. This also lets the HugeTLB allocator select preallocated
> CMA folios during long-term pinning. MIGRATE_ISOLATE and ZONE_MOVABLE
> remain restricted.
> 
> Fresh CMA allocations under PF_MEMALLOC_PIN also need to skip ranges
> in ZONE_MOVABLE. Returning -EBUSY here lets CMA try its remaining ranges.
> 
> The incremental diff against this patch is below.
> 
> diff --git a/include/linux/cma.h b/include/linux/cma.h
> index 8555d38a97b1..7fd1b6e65b58 100644
> --- a/include/linux/cma.h
> +++ b/include/linux/cma.h
> @@ -21,11 +21,13 @@
> #define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)
> 
> struct cma;
> +struct folio;
> 
> extern unsigned long totalcma_pages;
> extern phys_addr_t cma_get_base(const struct cma *cma);
> extern unsigned long cma_get_size(const struct cma *cma);
> extern const char *cma_get_name(const struct cma *cma);
> +bool cma_folio_is_pinnable(struct folio *folio);
> 
> extern int __init cma_declare_contiguous_nid(phys_addr_t base,
> 			phys_addr_t size, phys_addr_t limit,
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index cb993a1d0728..d15810be7d96 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -3,6 +3,7 @@
> #define _LINUX_MM_H
> 
> #include <linux/args.h>
> +#include <linux/cma.h>
> #include <linux/errno.h>
> #include <linux/mmdebug.h>
> #include <linux/gfp.h>
> @@ -2975,14 +2976,16 @@ static inline bool is_zero_folio(const struct folio *folio)
> 	return is_zero_page(&folio->page);
> }
> 
> -/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
> +/* Borrowed CMA pages and ZONE_MOVABLE folios cannot be pinned long term. */
> #ifdef CONFIG_MIGRATION
> static inline bool folio_is_longterm_pinnable(struct folio *folio)
> {
> #ifdef CONFIG_CMA
> 	int mt = folio_migratetype(folio);
> 
> -	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
> +	if (mt == MIGRATE_ISOLATE)
> +		return false;
> +	if (mt == MIGRATE_CMA && !cma_folio_is_pinnable(folio))
> 		return false;

Actually, we do not need cma_folio_is_pinnable(), does the following work for you?

--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2982,6 +2982,9 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio)
 #ifdef CONFIG_CMA
        int mt = folio_migratetype(folio);

+       if (folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio))
+               return true;
+
        if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
                return false;
 #endif

> #endif
> /* The zero page can be "pinned" but gets special handling. */
> diff --git a/mm/cma.c b/mm/cma.c
> index a10ea37a261d..c2814f54f86a 100644
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -27,6 +27,7 @@
> #include <linux/log2.h>
> #include <linux/cma.h>
> #include <linux/highmem.h>
> +#include <linux/hugetlb.h>
> #include <linux/io.h>
> #include <linux/kmemleak.h>
> #include <trace/events/cma.h>
> @@ -55,6 +56,12 @@ const char *cma_get_name(const struct cma *cma)
> }
> EXPORT_SYMBOL_GPL(cma_get_name);
> 
> +bool cma_folio_is_pinnable(struct folio *folio)
> +{
> +	/* Explicit HugeTLB CMA allocations own their bitmap ranges. */
> +	return folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio);
> +}
> +
> static unsigned long cma_bitmap_aligned_mask(const struct cma *cma,
> 					     unsigned int align_order)
> {
> @@ -792,6 +799,11 @@ static int cma_range_alloc(struct cma *cma, struct cma_memrange *cmr,
> 	int ret = -EBUSY;
> 	struct page *page = NULL;
> 
> + 	/* Each range lies within one zone; try other ranges if needed. */
> +	if ((current->flags & PF_MEMALLOC_PIN) &&
> + 	    is_zone_movable_page(pfn_to_page(cmr->base_pfn)))
> +		return -EBUSY;

I don't know why we need this.

Thanks,
Muchun

> +
> 	mask = cma_bitmap_aligned_mask(cma, align);
> 	offset = cma_bitmap_aligned_offset(cma, cmr, align);
> 	bitmap_maxno = cma_bitmap_maxno(cma, cmr);
> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
> index dde1c866406f..95fd2d190f0d 100644
> --- a/mm/hugetlb_cma.c
> +++ b/mm/hugetlb_cma.c
> @@ -62,9 +62,6 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
> 	if (!hugetlb_cma_size)
> 		return NULL;
> 
> -	if (current->flags & PF_MEMALLOC_PIN)
> -		return NULL;
> -
> retry_cpuset:
> 	if (!nodemask) {
> 		cpuset_mems_cookie = read_mems_allowed_begin();
> 
> Thanks,
> Qi
> 
>> Thanks,
>> Muchun
>>> 
>>> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
>>> ---
>>> mm/hugetlb_cma.c | 3 +++
>>> 1 file changed, 3 insertions(+)
>>> 
>>> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
>>> index 95fd2d190f0d2..dde1c866406fb 100644
>>> --- a/mm/hugetlb_cma.c
>>> +++ b/mm/hugetlb_cma.c
>>> @@ -62,6 +62,9 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
>>> 	if (!hugetlb_cma_size)
>>> 		return NULL;
>>> 
>>> +	if (current->flags & PF_MEMALLOC_PIN)
>>> +		return NULL;
>>> +
>>> retry_cpuset:
>>> 	if (!nodemask) {
>>> 		cpuset_mems_cookie = read_mems_allowed_begin();
>>> -- 
>>> 2.54.0

Re: [PATCH 2/4] mm: hugetlb: skip CMA allocations under PF_MEMALLOC_PIN

[Qi Zheng]

Hi Muchun,

On 11/4/26 3:07 PM, Muchun Song wrote:
> 
> 
>> On Nov 4, 2026, at 11:27, Qi Zheng <qi.zheng@linux.dev> wrote:
>>
>> Hi Muchun,
> 
> Hi,
> 
>>
>> On 11/3/26 6:56 PM, Muchun Song wrote:
>>>> On Nov 3, 2026, at 16:01, Qi Zheng <qi.zheng@linux.dev> wrote:
>>>>
>>>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>>>
>>>> With gigantic overcommit enabled, a fault during long-term pinning
>>>> can allocate a fresh HugeTLB folio from CMA despite PF_MEMALLOC_PIN.
>>>>
>>>> The folio must then be migrated before it can be pinned. However,
>>>> fresh gigantic migration targets are not supported, so pinning fails
>>>> when the HugeTLB pool has no suitable target, even if the original
>>>> fault could have allocated pinnable memory outside CMA.
>>>>
>>>> Honor PF_MEMALLOC_PIN in the HugeTLB CMA allocator so the fault can
>>>> try the non-CMA fallback, or fail when allocation is restricted to CMA.
>>> Wait... I think my Acked-by got ahead of me here. :)
>>> There seems to be a rather odd distinction that GUP is missing.
>>> The usual reason long-term GUP must migrate a page out of a
>>> MIGRATE_CMA pageblock is that the page was only borrowed from CMA by the
>>> normal page allocator.  The corresponding CMA bitmap range is still
>>> available, so a future CMA allocation relies on being able to migrate
>>> that page away.  A long-term pin would prevent that.
>>> However, a HugeTLB folio returned by
>>> hugetlb_cma_alloc_frozen_folio() is different.  It is the actual CMA
>>> allocation, not merely a movable page that happens to reside in a
>>> MIGRATE_CMA pageblock.  Its range has already been marked allocated in
>>> the hugetlb CMA bitmap.
>>> Therefore, CMA cannot allocate that range again until the HugeTLB folio
>>> is freed anyway.  Long-term pinning the folio does not seem to make the
>>> CMA area any less available than keeping the HugeTLB folio mapped for
>>> the same amount of time.
>>> folio_is_longterm_pinnable() currently only sees MIGRATE_CMA and does
>>> not distinguish these two cases.  In that light, rejecting
>>> PF_MEMALLOC_PIN in hugetlb_cma_alloc_frozen_folio() looks like working
>>> around an overly broad GUP classification.  It also means that
>>> hugetlb_cma_only users will fail the fault, while other users may consume
>>> a gigantic contiguous range outside the dedicated hugetlb CMA area even
>>> when that area has free space.
>>> Would it make more sense to fix the long-term GUP check instead, so that
>>> an explicitly CMA-allocated HugeTLB folio is considered pinnable, while
>>> ordinary pages merely borrowed from MIGRATE_CMA pageblocks remain unpinnable?
>>
>> Make sense!
>>
>> I'll replace this patch with an exception for HPG_cma folios in the
>> MIGRATE_CMA check. This also lets the HugeTLB allocator select preallocated
>> CMA folios during long-term pinning. MIGRATE_ISOLATE and ZONE_MOVABLE
>> remain restricted.
>>
>> Fresh CMA allocations under PF_MEMALLOC_PIN also need to skip ranges
>> in ZONE_MOVABLE. Returning -EBUSY here lets CMA try its remaining ranges.
>>
>> The incremental diff against this patch is below.
>>
>> diff --git a/include/linux/cma.h b/include/linux/cma.h
>> index 8555d38a97b1..7fd1b6e65b58 100644
>> --- a/include/linux/cma.h
>> +++ b/include/linux/cma.h
>> @@ -21,11 +21,13 @@
>> #define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)
>>
>> struct cma;
>> +struct folio;
>>
>> extern unsigned long totalcma_pages;
>> extern phys_addr_t cma_get_base(const struct cma *cma);
>> extern unsigned long cma_get_size(const struct cma *cma);
>> extern const char *cma_get_name(const struct cma *cma);
>> +bool cma_folio_is_pinnable(struct folio *folio);
>>
>> extern int __init cma_declare_contiguous_nid(phys_addr_t base,
>> 			phys_addr_t size, phys_addr_t limit,
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index cb993a1d0728..d15810be7d96 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -3,6 +3,7 @@
>> #define _LINUX_MM_H
>>
>> #include <linux/args.h>
>> +#include <linux/cma.h>
>> #include <linux/errno.h>
>> #include <linux/mmdebug.h>
>> #include <linux/gfp.h>
>> @@ -2975,14 +2976,16 @@ static inline bool is_zero_folio(const struct folio *folio)
>> 	return is_zero_page(&folio->page);
>> }
>>
>> -/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
>> +/* Borrowed CMA pages and ZONE_MOVABLE folios cannot be pinned long term. */
>> #ifdef CONFIG_MIGRATION
>> static inline bool folio_is_longterm_pinnable(struct folio *folio)
>> {
>> #ifdef CONFIG_CMA
>> 	int mt = folio_migratetype(folio);
>>
>> -	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
>> +	if (mt == MIGRATE_ISOLATE)
>> +		return false;
>> +	if (mt == MIGRATE_CMA && !cma_folio_is_pinnable(folio))
>> 		return false;
> 
> Actually, we do not need cma_folio_is_pinnable(), does the following work for you?

OK, will do.

> 
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2982,6 +2982,9 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio)
>   #ifdef CONFIG_CMA
>          int mt = folio_migratetype(folio);
> 
> +       if (folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio))
> +               return true;
> +
>          if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
>                  return false;
>   #endif

We should move the implementation of folio_is_longterm_pinnable() to
mm/gup.c to avoid the header dependency and build errors.

> 
>> #endif
>> /* The zero page can be "pinned" but gets special handling. */
>> diff --git a/mm/cma.c b/mm/cma.c
>> index a10ea37a261d..c2814f54f86a 100644
>> --- a/mm/cma.c
>> +++ b/mm/cma.c
>> @@ -27,6 +27,7 @@
>> #include <linux/log2.h>
>> #include <linux/cma.h>
>> #include <linux/highmem.h>
>> +#include <linux/hugetlb.h>
>> #include <linux/io.h>
>> #include <linux/kmemleak.h>
>> #include <trace/events/cma.h>
>> @@ -55,6 +56,12 @@ const char *cma_get_name(const struct cma *cma)
>> }
>> EXPORT_SYMBOL_GPL(cma_get_name);
>>
>> +bool cma_folio_is_pinnable(struct folio *folio)
>> +{
>> +	/* Explicit HugeTLB CMA allocations own their bitmap ranges. */
>> +	return folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio);
>> +}
>> +
>> static unsigned long cma_bitmap_aligned_mask(const struct cma *cma,
>> 					     unsigned int align_order)
>> {
>> @@ -792,6 +799,11 @@ static int cma_range_alloc(struct cma *cma, struct cma_memrange *cmr,
>> 	int ret = -EBUSY;
>> 	struct page *page = NULL;
>>
>> + 	/* Each range lies within one zone; try other ranges if needed. */
>> +	if ((current->flags & PF_MEMALLOC_PIN) &&
>> + 	    is_zone_movable_page(pfn_to_page(cmr->base_pfn)))
>> +		return -EBUSY;
> 
> I don't know why we need this.

I had assumed that ZONE_MOVABLE pages must always remain unpinnable.
But Documentation/admin-guide/mm/memory-hotplug.rst explicitly states:

   - Memory blocks overlapping with CMA areas cannot be offlined, this 
applies to
     memory blocks present during boot only.

So pinning explicitly CMA-allocated HugeTLB folios does not introduce
an additional offlining restriction. The ZONE_MOVABLE check I added
in cma_range_alloc() is unnecessary, and I'll drop it.

Thanks,
Qi

[PATCH 3/4] mm: hugetlb: skip CMA when direct reclaim is disallowed

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Switching HugeTLB CMA allocation from cma_alloc_folio() to
cma_alloc_frozen_compound() stopped forwarding the caller's GFP mask.
The new helper uses GFP_KERNEL, re-enabling direct reclaim even when
the caller explicitly disabled it.

The first MPOL_PREFERRED_MANY allocation attempt clears
__GFP_DIRECT_RECLAIM to try the preferred nodes before falling back to
other nodes. For gigantic surplus pages, the CMA path can therefore
enter direct reclaim while allocating migration targets during this
attempt.

Skip HugeTLB CMA allocation when direct reclaim is disallowed.

Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb_cma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index dde1c866406fb..90f8d718a1f16 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -59,7 +59,7 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	const nodemask_t *nmask;
 	unsigned int cpuset_mems_cookie;
 
-	if (!hugetlb_cma_size)
+	if (!hugetlb_cma_size || !gfpflags_allow_blocking(gfp_mask))
 		return NULL;
 
 	if (current->flags & PF_MEMALLOC_PIN)
-- 
2.54.0

[PATCH 4/4] mm: hugetlb: respect cpuset restrictions for surplus allocations

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Gigantic surplus allocations can escape the faulting task's cpuset.
CMA and the contiguous allocator trust the supplied nodemask without
checking the current cpuset.

For example, a shared HugeTLB VMA has MPOL_BIND={0,1}, and T is a
non-leader thread. With surplus allocation enabled and no eligible free
HugeTLB folio, the following sequence can occur:

  CPU0 (cgroup v1 migration)       CPU1 (thread T)
  -------------------------       ---------------
  cpuset_attach_task(..., T)
    T->mems_allowed = {0}
    /* Non-leader: skip
       mpol_rebind_mm(). */
                                  /* Fault; VMA policy is still {0,1}. */
                                  policy_nodemask()
                                    /* {0,1} intersects {0}. */
                                    nodemask = {0,1}
                                  hugetlb_cma_alloc_frozen_folio()
                                    /* node0 fails; node1 succeeds. */

The 1 GiB folio is allocated on node1 despite T being restricted to node0.

Intersect the interpreted policy with the current cpuset before surplus
allocation, preserving MPOL_BIND and MPOL_PREFERRED_MANY fallback rules.
Apply the restriction at the policy-aware entry point to preserve
per-node pool management semantics.

If the cpuset changes during allocation, rebuild the mask and revalidate
any allocated folio against the current constraints. Keep it if it is
still eligible: freeing an HVO-optimized folio can defer returning its
range to CMA, so discarding a usable folio can cause a needless allocation
failure. Only release the folio and allocate again if it is no longer
eligible.

Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Cc: stable@vger.kernel.org
Reported-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb.c | 38 +++++++++++++++++++++++++++++++-------
 1 file changed, 31 insertions(+), 7 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ce9c8e57b8e5c..6fd6e58ace869 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2181,22 +2181,46 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
 		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
 {
 	struct folio *folio = NULL;
-	nodemask_t *nodemask = mpoli->nodemask;
+	nodemask_t alloc_nodemask;
+	unsigned int cpuset_mems_cookie;
+
+retry:
+	cpuset_mems_cookie = read_mems_allowed_begin();
+	alloc_nodemask = cpuset_current_mems_allowed;
+	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
+	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
+
+	if (folio) {
+		if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
+			goto retry;
+
+		if (node_isset(folio_nid(folio), alloc_nodemask))
+			return folio;
+
+		/* Balance free_huge_folio()'s decrement before caller accounting */
+		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
+		free_huge_folio(folio);
+		folio = NULL;
+	}
 
 	if (mpoli->mode == MPOL_PREFERRED_MANY) {
 		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
 
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
 		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
-						    nodemask);
+						    &alloc_nodemask);
 
-		/* Fallback to all nodes if page==NULL */
-		nodemask = NULL;
+		/* Fallback to all nodes allowed by the current cpuset */
+		alloc_nodemask = cpuset_current_mems_allowed;
 	}
 
-	if (!folio) {
+	if (!folio)
 		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
-						    nodemask);
-	}
+						    &alloc_nodemask);
+
+	if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
+		goto retry;
 
 	return folio;
 }
-- 
2.54.0

Re: [PATCH 4/4] mm: hugetlb: respect cpuset restrictions for surplus allocations

[Jose A. Perez de Azpillaga]

On 11/3/26 9:01 AM, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> Gigantic surplus allocations can escape the faulting task's cpuset.
> CMA and the contiguous allocator trust the supplied nodemask without
> checking the current cpuset.
> 
> For example, a shared HugeTLB VMA has MPOL_BIND={0,1}, and T is a
> non-leader thread. With surplus allocation enabled and no eligible free
> HugeTLB folio, the following sequence can occur:
> 
>   CPU0 (cgroup v1 migration)       CPU1 (thread T)
>   -------------------------       ---------------
>   cpuset_attach_task(..., T)
>     T->mems_allowed = {0}
>     /* Non-leader: skip
>        mpol_rebind_mm(). */
>                                   /* Fault; VMA policy is still {0,1}. */
>                                   policy_nodemask()
>                                     /* {0,1} intersects {0}. */
>                                     nodemask = {0,1}
>                                   hugetlb_cma_alloc_frozen_folio()
>                                     /* node0 fails; node1 succeeds. */
> 
> The 1 GiB folio is allocated on node1 despite T being restricted to node0.
> 
> Intersect the interpreted policy with the current cpuset before surplus
> allocation, preserving MPOL_BIND and MPOL_PREFERRED_MANY fallback rules.
> Apply the restriction at the policy-aware entry point to preserve
> per-node pool management semantics.
> 
> If the cpuset changes during allocation, rebuild the mask and revalidate
> any allocated folio against the current constraints. Keep it if it is
> still eligible: freeing an HVO-optimized folio can defer returning its
> range to CMA, so discarding a usable folio can cause a needless allocation
> failure. Only release the folio and allocate again if it is no longer
> eligible.
> 
> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
> Cc: stable@vger.kernel.org
> Reported-by: Lance Yang <lance.yang@linux.dev>
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
> ---
>  mm/hugetlb.c | 38 +++++++++++++++++++++++++++++++-------
>  1 file changed, 31 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index ce9c8e57b8e5c..6fd6e58ace869 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2181,22 +2181,46 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
>  		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
>  {
>  	struct folio *folio = NULL;
> -	nodemask_t *nodemask = mpoli->nodemask;
> +	nodemask_t alloc_nodemask;
> +	unsigned int cpuset_mems_cookie;
> +
> +retry:
> +	cpuset_mems_cookie = read_mems_allowed_begin();
> +	alloc_nodemask = cpuset_current_mems_allowed;
> +	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
> +	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
> +
> +	if (folio) {
> +		if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
> +			goto retry;
> +
> +		if (node_isset(folio_nid(folio), alloc_nodemask))
> +			return folio;
> +
> +		/* Balance free_huge_folio()'s decrement before caller accounting */
> +		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
> +		free_huge_folio(folio);
> +		folio = NULL;
> +	}

the page allocator only retries when the allocation fails, why does this
need to drop a folio that raced with a cpuset change? retrying only on
!folio would make this whole block go away, with the check at the end as:

        if (!folio &&
            unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
                goto retry;

>  
>  	if (mpoli->mode == MPOL_PREFERRED_MANY) {
>  		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
>  
> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
>  		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
> -						    nodemask);
> +						    &alloc_nodemask);
>  
> -		/* Fallback to all nodes if page==NULL */
> -		nodemask = NULL;
> +		/* Fallback to all nodes allowed by the current cpuset */
> +		alloc_nodemask = cpuset_current_mems_allowed;
>  	}
>  
> -	if (!folio) {
> +	if (!folio)
>  		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
> -						    nodemask);
> -	}
> +						    &alloc_nodemask);
> +
> +	if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
> +		goto retry;
>  
>  	return folio;
>  }


-- 
cheers,
jose a. p-a

Re: [PATCH 4/4] mm: hugetlb: respect cpuset restrictions for surplus allocations

[Qi Zheng]

On 11/3/26 8:21 PM, Jose A. Perez de Azpillaga wrote:
> On 11/3/26 9:01 AM, Qi Zheng wrote:
>> From: Qi Zheng <zhengqi.arch@bytedance.com>
>>
>> Gigantic surplus allocations can escape the faulting task's cpuset.
>> CMA and the contiguous allocator trust the supplied nodemask without
>> checking the current cpuset.
>>
>> For example, a shared HugeTLB VMA has MPOL_BIND={0,1}, and T is a
>> non-leader thread. With surplus allocation enabled and no eligible free
>> HugeTLB folio, the following sequence can occur:
>>
>>    CPU0 (cgroup v1 migration)       CPU1 (thread T)
>>    -------------------------       ---------------
>>    cpuset_attach_task(..., T)
>>      T->mems_allowed = {0}
>>      /* Non-leader: skip
>>         mpol_rebind_mm(). */
>>                                    /* Fault; VMA policy is still {0,1}. */
>>                                    policy_nodemask()
>>                                      /* {0,1} intersects {0}. */
>>                                      nodemask = {0,1}
>>                                    hugetlb_cma_alloc_frozen_folio()
>>                                      /* node0 fails; node1 succeeds. */
>>
>> The 1 GiB folio is allocated on node1 despite T being restricted to node0.
>>
>> Intersect the interpreted policy with the current cpuset before surplus
>> allocation, preserving MPOL_BIND and MPOL_PREFERRED_MANY fallback rules.
>> Apply the restriction at the policy-aware entry point to preserve
>> per-node pool management semantics.
>>
>> If the cpuset changes during allocation, rebuild the mask and revalidate
>> any allocated folio against the current constraints. Keep it if it is
>> still eligible: freeing an HVO-optimized folio can defer returning its
>> range to CMA, so discarding a usable folio can cause a needless allocation
>> failure. Only release the folio and allocate again if it is no longer
>> eligible.
>>
>> Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
>> Cc: stable@vger.kernel.org
>> Reported-by: Lance Yang <lance.yang@linux.dev>
>> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
>> ---
>>   mm/hugetlb.c | 38 +++++++++++++++++++++++++++++++-------
>>   1 file changed, 31 insertions(+), 7 deletions(-)
>>
>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>> index ce9c8e57b8e5c..6fd6e58ace869 100644
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -2181,22 +2181,46 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
>>   		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
>>   {
>>   	struct folio *folio = NULL;
>> -	nodemask_t *nodemask = mpoli->nodemask;
>> +	nodemask_t alloc_nodemask;
>> +	unsigned int cpuset_mems_cookie;
>> +
>> +retry:
>> +	cpuset_mems_cookie = read_mems_allowed_begin();
>> +	alloc_nodemask = cpuset_current_mems_allowed;
>> +	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
>> +	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
>> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
>> +
>> +	if (folio) {
>> +		if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
>> +			goto retry;
>> +
>> +		if (node_isset(folio_nid(folio), alloc_nodemask))
>> +			return folio;
>> +
>> +		/* Balance free_huge_folio()'s decrement before caller accounting */
>> +		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
>> +		free_huge_folio(folio);
>> +		folio = NULL;
>> +	}
> 
> the page allocator only retries when the allocation fails, why does this
> need to drop a folio that raced with a cpuset change? retrying only on
> !folio would make this whole block go away, with the check at the end as:

You're right. I overthought the cpuset race and added unnecessary
revalidation of successful allocations.

> 
>          if (!folio &&
>              unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
>                  goto retry;

I'll follow your suggestion and retry only failed allocations, and
update the commit message accordingly.

Thanks,
Qi

> 
>>   
>>   	if (mpoli->mode == MPOL_PREFERRED_MANY) {
>>   		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
>>   
>> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
>>   		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
>> -						    nodemask);
>> +						    &alloc_nodemask);
>>   
>> -		/* Fallback to all nodes if page==NULL */
>> -		nodemask = NULL;
>> +		/* Fallback to all nodes allowed by the current cpuset */
>> +		alloc_nodemask = cpuset_current_mems_allowed;
>>   	}
>>   
>> -	if (!folio) {
>> +	if (!folio)
>>   		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
>> -						    nodemask);
>> -	}
>> +						    &alloc_nodemask);
>> +
>> +	if (unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
>> +		goto retry;
>>   
>>   	return folio;
>>   }
> 
>