[RFC PATCH 0/6] mm: pass alloc_flags through folio, filemap, and bulk allocators

[RFC PATCH 0/6] mm: pass alloc_flags through folio, filemap, and bulk allocators

[Gregory Price]

The folio, filemap, and bulk allocation interfaces currently hard-code
ALLOC_DEFAULT before reaching the page allocator. Multiple MM series
need to select allocator behavior through these paths, so pulling these
changes out ahead helps both series avoid conflicts.

This six-patch series first separates allocator behavior flags from
the bulk allocator fast-path flags and shares their validation and
preparation. It then passes alloc_flags through the MM-internal folio,
NUMA policy, filemap, and bulk helpers.

The existing exported folio, node, filemap, and bulk interfaces remain
ALLOC_DEFAULT wrappers.

Non-task ALLOC_NOLOCK folio allocations bypass task mempolicy and
cpuset-spread selection because those paths are not safe in all
interrupt contexts.

Existing callers retain their current behavior.

The allocator changes were built for x86-64 with DEBUG_VM,
DEBUG_VM_PGFLAGS, DEBUG_PAGEALLOC, PAGE_OWNER, PAGE_TABLE_CHECK,
PROVE_LOCKING, DEBUG_ATOMIC_SLEEP, and memory allocation profiling enabled.

Testing under virtme-ng/QEMU used a four-CPU guest:

- test_vmalloc vm_map_ram passed 50/50 64-page batches and 100/100 one-page
  fallbacks.
- With page_owner=on, all 25 runs produced the expected partial batch because
  page owner deliberately uses the one-page fallback.
- The alloc_tag ioctl selftest passed 4/4 tests.
- The migration selftest passed 6/6 tests.
- The page-fragment smoke, aligned, and nonaligned tests passed 3/3.
- khugepaged -s 2 passed 22 non-swap tests. Its four swap-dependent tests
  reported failure because the guest had no swap configured.

No BUG, WARNING, lockdep, or atomic-sleep diagnostics were reported.

Cc: Vlastimil Babka <vbabka@kernel.org>

Brendan Jackman (3):
  mm/page_alloc: add an alloc_flags-aware folio allocator
  mm/mempolicy: plumb alloc_flags through folio allocation
  mm/filemap: plumb alloc_flags through folio allocation

Gregory Price (3):
  mm/page_alloc: clarify bulk allocator flag scope
  mm/page_alloc: refactor alloc_flags preparation
  mm/page_alloc: let the bulk allocator carry alloc_flags

 mm/filemap.c    |  45 ++++++++--
 mm/mempolicy.c  |  44 +++++++---
 mm/mempolicy.h  |  28 +++++++
 mm/page_alloc.c | 217 ++++++++++++++++++++++++++++--------------------
 mm/page_alloc.h |  10 +++
 5 files changed, 236 insertions(+), 108 deletions(-)
 create mode 100644 mm/mempolicy.h

-- 
2.55.0

[RFC PATCH 1/6] mm/page_alloc: clarify bulk allocator flag scope

[Gregory Price]

alloc_pages_bulk_noprof() uses its local alloc_flags for fast-path
watermark, cpuset, and CMA decisions. These are distinct from the
allocator behavior flags stored in alloc_context.

Rename the local variable to fastpath_alloc_flags.

No change in functionality.

Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/page_alloc.c | 21 ++++++++++++---------
 1 file changed, 12 insertions(+), 9 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 07e03c7c7771..728dd1a36f0f 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5169,7 +5169,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	struct per_cpu_pages *pcp;
 	struct list_head *pcp_list;
 	struct alloc_context ac;
-	unsigned int alloc_flags = ALLOC_WMARK_LOW;
+	unsigned int fastpath_alloc_flags = ALLOC_WMARK_LOW;
 	int nr_populated = 0, nr_account = 0;
 
 	/*
@@ -5210,7 +5210,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	/* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */
 	gfp &= gfp_allowed_mask;
 	gfp = current_gfp_context(gfp);
-	if (!prepare_alloc_pages(gfp, 0, preferred_nid, nodemask, &ac, &gfp, &alloc_flags))
+	if (!prepare_alloc_pages(gfp, 0, preferred_nid, nodemask, &ac, &gfp,
+				 &fastpath_alloc_flags))
 		goto out;
 
 	/* Find an allowed local zone that meets the low watermark. */
@@ -5218,7 +5219,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	for_next_zone_zonelist_nodemask(zone, z, ac.highest_zoneidx, ac.nodemask) {
 		unsigned long mark;
 
-		if (cpusets_enabled() && (alloc_flags & ALLOC_CPUSET) &&
+		if (cpusets_enabled() && (fastpath_alloc_flags & ALLOC_CPUSET) &&
 		    !__cpuset_zone_allowed(zone, gfp)) {
 			continue;
 		}
@@ -5228,16 +5229,17 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 			goto failed;
 		}
 
-		cond_accept_memory(zone, 0, alloc_flags);
+		cond_accept_memory(zone, 0, fastpath_alloc_flags);
 retry_this_zone:
-		mark = wmark_pages(zone, alloc_flags & ALLOC_WMARK_MASK) + nr_pages - nr_populated;
+		mark = wmark_pages(zone, fastpath_alloc_flags & ALLOC_WMARK_MASK) +
+		       nr_pages - nr_populated;
 		if (zone_watermark_fast(zone, 0,  mark,
 				zonelist_zone_idx(ac.preferred_zoneref),
-				alloc_flags, gfp)) {
+				fastpath_alloc_flags, gfp)) {
 			break;
 		}
 
-		if (cond_accept_memory(zone, 0, alloc_flags))
+		if (cond_accept_memory(zone, 0, fastpath_alloc_flags))
 			goto retry_this_zone;
 
 		/* Try again if zone has deferred pages */
@@ -5269,8 +5271,9 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 			continue;
 		}
 
-		page = __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags,
-								pcp, pcp_list);
+		page = __rmqueue_pcplist(zone, 0, ac.migratetype,
+					 fastpath_alloc_flags, pcp,
+					 pcp_list);
 		if (unlikely(!page)) {
 			/* Try and allocate at least one page */
 			if (!nr_account) {
-- 
2.55.0

[RFC PATCH 2/6] mm/page_alloc: refactor alloc_flags preparation

[Gregory Price]

__alloc_frozen_pages_noprof() validates allocation flags, checks the
order, prepares ALLOC_NOLOCK, and selects its initial watermark.

alloc_pages_bulk_noprof() separately initializes its allocation context
and low-watermark flags.

Move this setup into prepare_alloc_flags() and call it from both paths.
The bulk allocator continues to pass ALLOC_DEFAULT.

No change in functionality.

Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/page_alloc.c | 158 ++++++++++++++++++++++++++----------------------
 1 file changed, 87 insertions(+), 71 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 728dd1a36f0f..b4a021fad183 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5138,6 +5138,84 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,
 	return true;
 }
 
+static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order,
+				       unsigned int alloc_flags)
+{
+	if (alloc_flags & ALLOC_NOLOCK)
+		return pcp_allowed_order(order);
+
+	/*
+	 * There are several places where we assume that the order value is sane
+	 * so bail out early if the request is out of bound.
+	 */
+	return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp));
+}
+
+static inline bool alloc_nolock_allowed(void)
+{
+	if (!can_spin_trylock())
+		return false;
+
+	/* Bailout, since _deferred_grow_zone() needs to take a lock */
+	if (deferred_pages_enabled())
+		return false;
+
+	return true;
+}
+
+/*
+ * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock().
+ *
+ * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
+ * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
+ * is not safe in arbitrary context.
+ *
+ * These two are the conditions for gfpflags_allow_spinning() being true.
+ *
+ * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
+ * to warn. Also warn would trigger printk() which is unsafe from
+ * various contexts. We cannot use printk_deferred_enter() to mitigate,
+ * since the running context is unknown.
+ *
+ * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
+ * is safe in any context. Also zeroing the page is mandatory for
+ * BPF use cases.
+ *
+ * Though __GFP_NOMEMALLOC is not checked in the code path below,
+ * specify it here to highlight that alloc_pages_nolock()
+ * doesn't want to deplete reserves.
+ */
+static const gfp_t gfp_nolock = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC |
+				__GFP_COMP;
+
+static __always_inline bool prepare_alloc_flags(gfp_t *gfp, unsigned int order,
+		unsigned int alloc_flags, unsigned int *prepared_alloc_flags,
+		unsigned int *fastpath_alloc_flags)
+{
+	/* Other flags could be supported later if needed. */
+	if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG)))
+		return false;
+
+	if (!alloc_order_allowed(*gfp, order, alloc_flags))
+		return false;
+
+	*prepared_alloc_flags = alloc_flags;
+	*fastpath_alloc_flags = alloc_flags;
+
+	if (alloc_flags & ALLOC_NOLOCK) {
+		/* Certain other flags could be supported later if needed. */
+		VM_WARN_ON_ONCE(*gfp & ~(__GFP_ACCOUNT | gfp_nolock));
+		if (!alloc_nolock_allowed())
+			return false;
+		*gfp |= gfp_nolock;
+		*fastpath_alloc_flags |= ALLOC_WMARK_MIN;
+	} else {
+		*fastpath_alloc_flags |= ALLOC_WMARK_LOW;
+	}
+
+	return true;
+}
+
 /*
  * __alloc_pages_bulk - Allocate a number of order-0 pages to an array
  * @gfp: GFP flags for the allocation
@@ -5169,7 +5247,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	struct per_cpu_pages *pcp;
 	struct list_head *pcp_list;
 	struct alloc_context ac;
-	unsigned int fastpath_alloc_flags = ALLOC_WMARK_LOW;
+	unsigned int fastpath_alloc_flags;
 	int nr_populated = 0, nr_account = 0;
 
 	/*
@@ -5207,6 +5285,10 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		goto failed;
 #endif
 
+	if (!prepare_alloc_flags(&gfp, 0, ALLOC_DEFAULT, &ac.alloc_flags,
+				 &fastpath_alloc_flags))
+		goto out;
+
 	/* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */
 	gfp &= gfp_allowed_mask;
 	gfp = current_gfp_context(gfp);
@@ -5335,56 +5417,6 @@ void free_pages_bulk(struct page **page_array, unsigned long nr_pages)
 	}
 }
 
-static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order,
-				       unsigned int alloc_flags)
-{
-	if (alloc_flags & ALLOC_NOLOCK)
-		return pcp_allowed_order(order);
-
-	/*
-	 * There are several places where we assume that the order value is sane
-	 * so bail out early if the request is out of bound.
-	 */
-	return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp));
-}
-
-static inline bool alloc_nolock_allowed(void)
-{
-	if (!can_spin_trylock())
-		return false;
-
-	/* Bailout, since _deferred_grow_zone() needs to take a lock */
-	if (deferred_pages_enabled())
-		return false;
-
-	return true;
-}
-
-/*
- * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock().
- *
- * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed.
- * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd
- * is not safe in arbitrary context.
- *
- * These two are the conditions for gfpflags_allow_spinning() being true.
- *
- * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason
- * to warn. Also warn would trigger printk() which is unsafe from
- * various contexts. We cannot use printk_deferred_enter() to mitigate,
- * since the running context is unknown.
- *
- * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below
- * is safe in any context. Also zeroing the page is mandatory for
- * BPF use cases.
- *
- * Though __GFP_NOMEMALLOC is not checked in the code path below,
- * specify it here to highlight that alloc_pages_nolock()
- * doesn't want to deplete reserves.
- */
-static const gfp_t gfp_nolock = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC |
-				__GFP_COMP;
-
 /*
  * This is the 'heart' of the zoned buddy allocator.
  */
@@ -5393,29 +5425,13 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order,
 {
 	struct page *page;
 	gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */
-	struct alloc_context ac = {
-		.alloc_flags = alloc_flags,
-	};
-	unsigned int fastpath_alloc_flags = alloc_flags;
-
-	/* Other flags could be supported later if needed. */
-	if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG)))
-		return NULL;
+	struct alloc_context ac;
+	unsigned int fastpath_alloc_flags;
 
-	if (!alloc_order_allowed(gfp, order, alloc_flags))
+	if (!prepare_alloc_flags(&gfp, order, alloc_flags, &ac.alloc_flags,
+				 &fastpath_alloc_flags))
 		return NULL;
 
-	if (alloc_flags & ALLOC_NOLOCK) {
-		/* Certain other flags could be supported later if needed. */
-		VM_WARN_ON_ONCE(gfp & ~(__GFP_ACCOUNT | gfp_nolock));
-		if (!alloc_nolock_allowed())
-			return NULL;
-		gfp |= gfp_nolock;
-		fastpath_alloc_flags |= ALLOC_WMARK_MIN;
-	} else {
-		fastpath_alloc_flags |= ALLOC_WMARK_LOW;
-	}
-
 	gfp &= gfp_allowed_mask;
 	/*
 	 * Apply scoped allocation constraints. This is mainly about GFP_NOFS
-- 
2.55.0

[RFC PATCH 3/6] mm/page_alloc: add an alloc_flags-aware folio allocator

[Gregory Price]

From: Brendan Jackman <brendan.jackman@google.com>

The low-level page allocator accepts explicit allocator behavior flags,
but the folio helper hard-codes ALLOC_DEFAULT.

Add an MM-internal folio allocator that accepts alloc_flags. Keep the
existing exported folio and node interfaces unchanged as ALLOC_DEFAULT
wrappers.

No functional change intended.

Signed-off-by: Brendan Jackman <brendan.jackman@google.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/page_alloc.c | 14 +++++++++++---
 mm/page_alloc.h |  6 ++++++
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index b4a021fad183..feb9c64efd84 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5510,13 +5510,21 @@ struct page *alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsigned int order
 }
 EXPORT_SYMBOL(alloc_pages_node_noprof);
 
-struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,
-		nodemask_t *nodemask)
+struct folio *__folio_alloc_flags_noprof(gfp_t gfp, unsigned int order,
+		int preferred_nid, nodemask_t *nodemask,
+		unsigned int alloc_flags)
 {
 	struct page *page = __alloc_pages_noprof(gfp | __GFP_COMP, order,
-					preferred_nid, nodemask, ALLOC_DEFAULT);
+					preferred_nid, nodemask, alloc_flags);
 	return page_rmappable_folio(page);
 }
+
+struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order,
+		int preferred_nid, nodemask_t *nodemask)
+{
+	return __folio_alloc_flags_noprof(gfp, order, preferred_nid, nodemask,
+					 ALLOC_DEFAULT);
+}
 EXPORT_SYMBOL(__folio_alloc_noprof);
 
 /*
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index b9259deddb59..986e733b586e 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -258,6 +258,12 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_n
 		nodemask_t *nodemask, unsigned int alloc_flags);
 #define __alloc_pages(...)			alloc_hooks(__alloc_pages_noprof(__VA_ARGS__))
 
+struct folio *__folio_alloc_flags_noprof(gfp_t gfp, unsigned int order,
+		int preferred_nid, nodemask_t *nodemask,
+		unsigned int alloc_flags);
+#define __folio_alloc_flags(...) \
+	alloc_hooks(__folio_alloc_flags_noprof(__VA_ARGS__))
+
 extern void zone_pcp_reset(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
-- 
2.55.0

[RFC PATCH 4/6] mm/mempolicy: plumb alloc_flags through folio allocation

[Gregory Price]

From: Brendan Jackman <brendan.jackman@google.com>

The NUMA policy allocation paths hard-code ALLOC_DEFAULT before reaching
the page allocator.

Add an alloc_flags argument to the MM-internal policy allocator and carry
it through preferred-many, huge-page, and general policy allocation paths.
Keep the public mempolicy folio interface as an ALLOC_DEFAULT wrapper.

Task policy selection is not safe in every interrupt context, so non-task
ALLOC_NOLOCK requests bypass it and use the supplied preferred node
directly.

No existing caller behavior is changed.

Link: https://lore.kernel.org/r/20260726-page_alloc-unmapped-v3-25-6f5729aa9832@google.com
Signed-off-by: Brendan Jackman <brendan.jackman@google.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/mempolicy.c | 44 ++++++++++++++++++++++++++++++++------------
 mm/mempolicy.h | 28 ++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+), 12 deletions(-)
 create mode 100644 mm/mempolicy.h

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 4c0b8ff1a7e6..2c01699081b7 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -120,6 +120,7 @@
 #include <linux/memory.h>
 
 #include "internal.h"
+#include "mempolicy.h"
 #include "page_alloc.h"
 
 /* Internal flags */
@@ -2437,7 +2438,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk,
 }
 
 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
-						int nid, nodemask_t *nodemask)
+						int nid, nodemask_t *nodemask,
+						unsigned int alloc_flags)
 {
 	struct page *page;
 	gfp_t preferred_gfp;
@@ -2451,10 +2453,10 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
 	preferred_gfp = gfp | __GFP_NOWARN;
 	preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
 	page = __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask,
-					   ALLOC_DEFAULT);
+					   alloc_flags);
 	if (!page)
 		page = __alloc_frozen_pages_noprof(gfp, order, nid, NULL,
-						   ALLOC_DEFAULT);
+						   alloc_flags);
 
 	return page;
 }
@@ -2466,11 +2468,13 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order,
  * @pol: Pointer to the NUMA mempolicy.
  * @ilx: Index for interleave mempolicy (also distinguishes alloc_pages()).
  * @nid: Preferred node (usually numa_node_id() but @mpol may override it).
+ * @alloc_flags: ALLOC_* flags.
  *
  * Return: The page on success or NULL if allocation fails.
  */
 static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
-		struct mempolicy *pol, pgoff_t ilx, int nid)
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned int alloc_flags)
 {
 	nodemask_t *nodemask;
 	struct page *page;
@@ -2478,7 +2482,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 	nodemask = policy_nodemask(gfp, pol, ilx, &nid);
 
 	if (pol->mode == MPOL_PREFERRED_MANY)
-		return alloc_pages_preferred_many(gfp, order, nid, nodemask);
+		return alloc_pages_preferred_many(gfp, order, nid, nodemask,
+					  alloc_flags);
 
 	if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
 	    /* filter "hugepage" allocation, unless from alloc_pages() */
@@ -2502,7 +2507,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 			 */
 			page = __alloc_frozen_pages_noprof(
 				gfp | __GFP_THISNODE | __GFP_NORETRY, order,
-				nid, NULL, ALLOC_DEFAULT);
+				nid, NULL, alloc_flags);
 			if (page || !(gfp & __GFP_DIRECT_RECLAIM))
 				return page;
 			/*
@@ -2514,7 +2519,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 		}
 	}
 
-	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEFAULT);
+	page = __alloc_frozen_pages_noprof(gfp, order, nid, nodemask,
+					   alloc_flags);
 
 	if (unlikely(pol->mode == MPOL_INTERLEAVE ||
 		     pol->mode == MPOL_WEIGHTED_INTERLEAVE) && page) {
@@ -2530,11 +2536,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order,
 	return page;
 }
 
-struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
-		struct mempolicy *pol, pgoff_t ilx, int nid)
+struct folio *__folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned int alloc_flags)
 {
-	struct page *page = alloc_pages_mpol(gfp | __GFP_COMP, order, pol,
-			ilx, nid);
+	struct page *page;
+
+	if (unlikely((alloc_flags & ALLOC_NOLOCK) && !in_task()))
+		return __folio_alloc_flags_noprof(gfp, order, nid, NULL,
+						 alloc_flags);
+
+	page = alloc_pages_mpol(gfp | __GFP_COMP, order, pol, ilx, nid,
+				alloc_flags);
 	if (!page)
 		return NULL;
 
@@ -2542,6 +2555,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
 	return page_rmappable_folio(page);
 }
 
+struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid)
+{
+	return __folio_alloc_mpol_noprof(gfp, order, pol, ilx, nid,
+					 ALLOC_DEFAULT);
+}
+
 /**
  * vma_alloc_folio - Allocate a folio for a VMA.
  * @gfp: GFP flags.
@@ -2586,7 +2606,7 @@ struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order)
 		pol = get_task_policy(current);
 
 	return alloc_pages_mpol(gfp, order, pol, NO_INTERLEAVE_INDEX,
-				       numa_node_id());
+			       numa_node_id(), ALLOC_DEFAULT);
 }
 
 /**
diff --git a/mm/mempolicy.h b/mm/mempolicy.h
new file mode 100644
index 000000000000..8d2c07868f6e
--- /dev/null
+++ b/mm/mempolicy.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * mm-internal API for mempolicy.c. Public API lives in
+ * include/linux/mempolicy.h.
+ */
+#ifndef __MM_MEMPOLICY_H
+#define __MM_MEMPOLICY_H
+
+#include <linux/gfp.h>
+#include <linux/mempolicy.h>
+
+#include "page_alloc.h"
+
+#ifdef CONFIG_NUMA
+struct folio *__folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned int alloc_flags);
+#else
+static inline struct folio *__folio_alloc_mpol_noprof(gfp_t gfp,
+		unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid,
+		unsigned int alloc_flags)
+{
+	return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), NULL,
+					 alloc_flags);
+}
+#endif
+
+#endif
-- 
2.55.0

[RFC PATCH 5/6] mm/filemap: plumb alloc_flags through folio allocation

[Gregory Price]

From: Brendan Jackman <jackmanb@google.com>

The filemap folio allocator applies NUMA policy and node selection before
reaching the low-level folio allocator. It currently provides no way for
an MM-internal caller to preserve explicit allocator behavior flags across
that selection.

Add an internal filemap folio allocator that accepts alloc_flags and carries
them through every allocation path.

Keep filemap_alloc_folio_noprof() as an ALLOC_DEFAULT wrapper so existing
callers and the exported interface remain unchanged.

No functional change intended.

Link: https://lore.kernel.org/r/20260726-page_alloc-unmapped-v3-26-6f5729aa9832@google.com
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/filemap.c | 45 +++++++++++++++++++++++++++++++++++++++------
 1 file changed, 39 insertions(+), 6 deletions(-)

diff --git a/mm/filemap.c b/mm/filemap.c
index b74bc1e5015c..3b33cc873b73 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -52,6 +52,8 @@
 
 #include <asm/tlbflush.h>
 #include "internal.h"
+#include "mempolicy.h"
+#include "page_alloc.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/filemap.h>
@@ -995,31 +997,62 @@ int filemap_add_folio(struct address_space *mapping, struct folio *folio,
 EXPORT_SYMBOL_GPL(filemap_add_folio);
 
 #ifdef CONFIG_NUMA
-struct folio *filemap_alloc_folio_noprof(gfp_t gfp, unsigned int order,
-		struct mempolicy *policy)
+static inline struct folio *__filemap_alloc_folio_noprof(gfp_t gfp,
+		unsigned int order, struct mempolicy *policy,
+		unsigned int alloc_flags)
 {
 	int n;
 	struct folio *folio;
 
+	/* Task policy and cpuset spreading are not safe in all IRQ contexts. */
+	if (unlikely((alloc_flags & ALLOC_NOLOCK) && !in_task()))
+		return __folio_alloc_flags_noprof(gfp, order, numa_node_id(),
+						 NULL, alloc_flags);
+
 	if (policy)
-		return folio_alloc_mpol_noprof(gfp, order, policy,
-				NO_INTERLEAVE_INDEX, numa_node_id());
+		return __folio_alloc_mpol_noprof(gfp, order, policy,
+				NO_INTERLEAVE_INDEX, numa_node_id(), alloc_flags);
 
 	if (cpuset_do_page_mem_spread()) {
 		unsigned int cpuset_mems_cookie;
+
 		do {
 			cpuset_mems_cookie = read_mems_allowed_begin();
 			n = cpuset_mem_spread_node();
-			folio = __folio_alloc_node_noprof(gfp, order, n);
+			folio = __folio_alloc_flags_noprof(gfp, order, n, NULL,
+							   alloc_flags);
 		} while (!folio && read_mems_allowed_retry(cpuset_mems_cookie));
 
 		return folio;
 	}
-	return folio_alloc_noprof(gfp, order);
+
+	if (in_interrupt() || (gfp & __GFP_THISNODE))
+		return __folio_alloc_flags_noprof(gfp, order, numa_node_id(),
+						 NULL, alloc_flags);
+
+	return __folio_alloc_mpol_noprof(gfp, order, get_task_policy(current),
+			NO_INTERLEAVE_INDEX, numa_node_id(), alloc_flags);
+}
+
+struct folio *filemap_alloc_folio_noprof(gfp_t gfp, unsigned int order,
+		struct mempolicy *policy)
+{
+	return __filemap_alloc_folio_noprof(gfp, order, policy, ALLOC_DEFAULT);
 }
 EXPORT_SYMBOL(filemap_alloc_folio_noprof);
+#else
+static inline struct folio *__filemap_alloc_folio_noprof(gfp_t gfp,
+		unsigned int order, struct mempolicy *policy,
+		unsigned int alloc_flags)
+{
+	return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), NULL,
+					 alloc_flags);
+}
 #endif
 
+#define __filemap_alloc_folio(...) \
+	alloc_hooks(__filemap_alloc_folio_noprof(__VA_ARGS__))
+
 /*
  * filemap_invalidate_lock_two - lock invalidate_lock for two mappings
  *
-- 
2.55.0

[RFC PATCH 6/6] mm/page_alloc: let the bulk allocator carry alloc_flags

[Gregory Price]

__alloc_pages_noprof() accepts explicit allocator behavior flags, but the
bulk allocator does not. Internal callers therefore cannot select those
behaviors through the bulk interface.

Add __alloc_pages_bulk_noprof() for flag-carrying MM callers. Keep
alloc_pages_bulk_noprof() as a flag-free wrapper passing ALLOC_DEFAULT, so
existing callers and the exported interface remain unchanged.

Preserve the original GFP mask for the single-page fallback so it repeats
flag preparation from the caller inputs, and pass the prepared fast-path
flags to prep_new_page().

Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/page_alloc.c | 28 +++++++++++++++++++---------
 mm/page_alloc.h |  4 ++++
 2 files changed, 23 insertions(+), 9 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index feb9c64efd84..8fa2522ef418 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5219,6 +5219,7 @@ static __always_inline bool prepare_alloc_flags(gfp_t *gfp, unsigned int order,
 /*
  * __alloc_pages_bulk - Allocate a number of order-0 pages to an array
  * @gfp: GFP flags for the allocation
+ * @alloc_flags: Allocation behavior flags
  * @preferred_nid: The preferred NUMA node ID to allocate from
  * @nodemask: Set of nodes to allocate from, may be NULL
  * @nr_pages: The number of pages desired in the array
@@ -5237,15 +5238,16 @@ static __always_inline bool prepare_alloc_flags(gfp_t *gfp, unsigned int order,
  * @page_array were set to %NULL on entry, the slots from 0 to the return value
  * - 1 will be filled.
  */
-unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
-			nodemask_t *nodemask, int nr_pages,
-			struct page **page_array)
+unsigned long __alloc_pages_bulk_noprof(gfp_t gfp, unsigned int alloc_flags,
+		int preferred_nid, nodemask_t *nodemask, int nr_pages,
+		struct page **page_array)
 {
 	struct page *page;
 	struct zone *zone;
 	struct zoneref *z;
 	struct per_cpu_pages *pcp;
 	struct list_head *pcp_list;
+	gfp_t orig_gfp = gfp;
 	struct alloc_context ac;
 	unsigned int fastpath_alloc_flags;
 	int nr_populated = 0, nr_account = 0;
@@ -5285,7 +5287,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		goto failed;
 #endif
 
-	if (!prepare_alloc_flags(&gfp, 0, ALLOC_DEFAULT, &ac.alloc_flags,
+	if (!prepare_alloc_flags(&gfp, 0, alloc_flags, &ac.alloc_flags,
 				 &fastpath_alloc_flags))
 		goto out;
 
@@ -5296,7 +5298,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 				 &fastpath_alloc_flags))
 		goto out;
 
-	/* Find an allowed local zone that meets the low watermark. */
+	/* Find an allowed local zone that meets the required watermark. */
 	z = ac.preferred_zoneref;
 	for_next_zone_zonelist_nodemask(zone, z, ac.highest_zoneidx, ac.nodemask) {
 		unsigned long mark;
@@ -5332,8 +5334,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	}
 
 	/*
-	 * If there are no allowed local zones that meets the watermarks then
-	 * try to allocate a single page and reclaim if necessary.
+	 * If there are no allowed local zones that meet the watermarks, try the
+	 * single-page allocator.
 	 */
 	if (unlikely(!zone))
 		goto failed;
@@ -5366,7 +5368,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		}
 		nr_account++;
 
-		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
+		prep_new_page(page, 0, gfp, fastpath_alloc_flags);
 		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
 		kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM);
 		set_page_refcounted(page);
@@ -5382,11 +5384,19 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	return nr_populated;
 
 failed:
-	page = __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask, ALLOC_DEFAULT);
+	page = __alloc_pages_noprof(orig_gfp, 0, preferred_nid, nodemask,
+				    alloc_flags);
 	if (page)
 		page_array[nr_populated++] = page;
 	goto out;
 }
+
+unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
+		nodemask_t *nodemask, int nr_pages, struct page **page_array)
+{
+	return __alloc_pages_bulk_noprof(gfp, ALLOC_DEFAULT, preferred_nid,
+					 nodemask, nr_pages, page_array);
+}
 EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof);
 
 /*
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index 986e733b586e..a09bfd201c3b 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -264,6 +264,10 @@ struct folio *__folio_alloc_flags_noprof(gfp_t gfp, unsigned int order,
 #define __folio_alloc_flags(...) \
 	alloc_hooks(__folio_alloc_flags_noprof(__VA_ARGS__))
 
+unsigned long __alloc_pages_bulk_noprof(gfp_t gfp, unsigned int alloc_flags,
+		int preferred_nid, nodemask_t *nodemask, int nr_pages,
+		struct page **page_array);
+
 extern void zone_pcp_reset(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
-- 
2.55.0

Re: [RFC PATCH 0/6] mm: pass alloc_flags through folio, filemap, and bulk allocators

[Matthew Wilcox]

On Wed, Sep 23, 2026 at 05:10:34PM -0400, Gregory Price wrote:
> The folio, filemap, and bulk allocation interfaces currently hard-code
> ALLOC_DEFAULT before reaching the page allocator. Multiple MM series
> need to select allocator behavior through these paths, so pulling these
> changes out ahead helps both series avoid conflicts.

I really think this 'expose ALLOC flags' idea is wrong.

Re: [RFC PATCH 0/6] mm: pass alloc_flags through folio, filemap, and bulk allocators

[Gregory Price]

On Wed, Sep 23, 2026 at 10:42:07PM +0100, Matthew Wilcox wrote:
> On Wed, Sep 23, 2026 at 05:10:34PM -0400, Gregory Price wrote:
> > The folio, filemap, and bulk allocation interfaces currently hard-code
> > ALLOC_DEFAULT before reaching the page allocator. Multiple MM series
> > need to select allocator behavior through these paths, so pulling these
> > changes out ahead helps both series avoid conflicts.
> 
> I really think this 'expose ALLOC flags' idea is wrong.

It is mm/ internal only - to be clear.

It's also already been partially exposed, this just brings the rest of
the interfaces with it.

There are use cases.  I initially only wanted to expose a zonelist
selection rather than alloc flags, but given ALLOC_UNMAPPED was
in-flight it seemed better not to propose different solutions when there
are existing possibilities.

I'm open to options.

~Gregory