[PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau (Meta)]

Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
storm in defrag_mode"), direct reclaim and compaction for non-movable
requests under defrag_mode run at pageblock_order, to produce the whole
blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
still use the request order. An order-0 request can therefore retry
indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:

- Reclaim at pageblock_order gives up after one pass as soon as a zone
  looks compaction_ready(), and do_try_to_free_pages() then returns 1
  even though nothing was reclaimed. It returns before the retry that
  would reclaim memory.low-protected cgroups, so when most memory is
  protected, the pass that did run finds next to nothing.

- Compaction at pageblock_order fails or is deferred.

- should_reclaim_retry() takes the reported progress as progress for
  the order-0 request and resets no_progress_loops. The request
  retries.

Order 1-3 requests loop the same way, and should_compact_retry() also
checks their pageblock_order compaction result against the request
order.

On a production host (64G, defrag_mode, memory.low covering most of the
workload), 95% of direct reclaim runs were order-9 runs that returned 1
with nothing reclaimed, at up to 60k runs per second. Across ~200M
should_reclaim_retry() calls in a day, no_progress_loops never left 0.
The spinning allocations were SLUB slab refills for inode and dentry
caches. The time spent registers as memory pressure, and pressure-based
OOM killing takes down both workloads and system services.

For promoted requests:

- Reclaim progress does not reset no_progress_loops, as for costly
  orders.

- should_compact_retry() checks the compaction result at the promoted
  order and does not retry COMPACT_SKIPPED, since the request can fall
  back. The compaction priority floor and the COMPACT_SUCCESS retry
  limit stay those of the request order, so a non-costly request still
  gets its COMPACT_PRIO_SYNC_FULL pass before it falls back.

When a promoted request takes the fallback, reset the retry counters,
so that the fallback attempt gets a full retry budget before the OOM
killer is considered. A request whose reclaim was not promoted spent
its budget on the request itself and keeps its counters, as Usama Arif
pointed out.

__alloc_pages_slowpath() computes the promoted order once per iteration
and passes it to direct reclaim, direct compaction and the two retry
helpers next to the request order, so no callee has to recompute it.
Harry Yoo asked for the two orders to be explicit rather than derived
in each callee.

In a VM reproducer (32G, defrag_mode, inode churn under memory.low):

                                        before    after
    should_reclaim_retry() calls           63M     293k
    peak memory pressure (PSI some avg10)  99%      12%

File creation runs 5.7x faster.

Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v3:
 - reset the retry counters at the fallback only for requests whose
   reclaim was promoted; a movable request already spent its budget on
   itself (Usama Arif)

v2:
 - keep the compaction priority floor and the COMPACT_SUCCESS retry
   limit on the request order, so non-costly requests still escalate to
   SYNC_FULL before falling back (Harry Yoo, Johannes Weiner)
 - compute the promoted order once in __alloc_pages_slowpath() and pass
   it to direct reclaim, direct compaction and the retry helpers, instead
   of each of them deriving it (Harry Yoo)

 mm/page_alloc.c | 126 +++++++++++++++++++++++++++++++-----------------
 1 file changed, 81 insertions(+), 45 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..c0be88d2c3ad 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4127,6 +4127,32 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 	return page;
 }
 
+/*
+ * If fallbacks are not permitted (defrag_mode), we either need to
+ * reclaim space in a block of matching type, or clear out an entire
+ * block to allow __rmqueue_claim() to convert.
+ *
+ * Reclaim by itself is primarily freeing space in movable blocks,
+ * since that's where the LRU pages live. So this works for movable
+ * requests, but not for others.
+ *
+ * For those, promote the order of reclaim and compaction to help make
+ * blocks, instead of spinning in reclaim alone unproductively. Retry
+ * decisions based on the outcome of that work - reclaim progress and
+ * compaction results - must account for the promotion as well, so
+ * __alloc_pages_slowpath() computes the promoted order once and passes
+ * it alongside the request order.
+ */
+static inline unsigned int nofrag_promote_order(unsigned int order,
+						unsigned int alloc_flags,
+						const struct alloc_context *ac)
+{
+	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
+		return max(order, pageblock_order);
+
+	return order;
+}
+
 /*
  * Maximum number of compaction retries with a progress before OOM
  * killer is consider as the only way to move forward.
@@ -4137,8 +4163,9 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 /* Try memory compaction for high-order allocations before reclaim */
 static struct page *
 __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
-		unsigned int alloc_flags, const struct alloc_context *ac,
-		enum compact_priority prio, enum compact_result *compact_result)
+		unsigned int compact_order, unsigned int alloc_flags,
+		const struct alloc_context *ac, enum compact_priority prio,
+		enum compact_result *compact_result)
 {
 	struct page *page = NULL;
 	unsigned long pflags;
@@ -4149,22 +4176,6 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 		.order = order,
 		.page = NULL,
 	};
-	int compact_order = order;
-
-	/*
-	 * If fallbacks are not permitted (defrag_mode), we either
-	 * need to reclaim space in a block of matching type, or clear
-	 * out an entire block to allow __rmqueue_claim() to convert.
-	 *
-	 * Reclaim by itself is primarily freeing space in movable
-	 * blocks, since that's where the LRU pages live. So this
-	 * works for movable requests, but not for others.
-	 *
-	 * For those, promote the order to help make blocks, instead
-	 * of spinning in reclaim alone unproductively.
-	 */
-	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
-		compact_order = max(order, pageblock_order);
 
 	if (!compact_order)
 		return NULL;
@@ -4246,7 +4257,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 
 static inline bool
 should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
-		     int alloc_flags,
+		     int compact_order, int alloc_flags,
 		     enum compact_result compact_result,
 		     enum compact_priority *compact_priority,
 		     int *compaction_retries)
@@ -4257,7 +4268,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 	int retries = *compaction_retries;
 	enum compact_priority priority = *compact_priority;
 
-	if (!order)
+	if (!compact_order)
 		return false;
 
 	if (fatal_signal_pending(current))
@@ -4266,10 +4277,14 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 	/*
 	 * Compaction was skipped due to a lack of free order-0
 	 * migration targets. Continue if reclaim can help.
+	 *
+	 * Promoted requests have exhausted their reclaim retries at
+	 * this point, and they can fall back instead.
 	 */
 	if (compact_result == COMPACT_SKIPPED) {
-		ret = compaction_zonelist_suitable(ac, order, alloc_flags,
-						   gfp_mask);
+		if (compact_order == order)
+			ret = compaction_zonelist_suitable(ac, order, alloc_flags,
+							   gfp_mask);
 		goto out;
 	}
 
@@ -4314,8 +4329,9 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 #else
 static inline struct page *
 __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
-		unsigned int alloc_flags, const struct alloc_context *ac,
-		enum compact_priority prio, enum compact_result *compact_result)
+		unsigned int compact_order, unsigned int alloc_flags,
+		const struct alloc_context *ac, enum compact_priority prio,
+		enum compact_result *compact_result)
 {
 	*compact_result = COMPACT_SKIPPED;
 	return NULL;
@@ -4323,7 +4339,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 
 static inline bool
 should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
-		     int alloc_flags,
+		     int compact_order, int alloc_flags,
 		     enum compact_result compact_result,
 		     enum compact_priority *compact_priority,
 		     int *compaction_retries)
@@ -4462,17 +4478,12 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order,
 /* The really slow allocator path where we enter direct reclaim */
 static inline struct page *
 __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
-		unsigned int alloc_flags, const struct alloc_context *ac,
-		unsigned long *did_some_progress)
+		unsigned int reclaim_order, unsigned int alloc_flags,
+		const struct alloc_context *ac, unsigned long *did_some_progress)
 {
 	struct page *page = NULL;
 	unsigned long pflags;
 	bool drained = false;
-	int reclaim_order = order;
-
-	/* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
-	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
-		reclaim_order = max(order, pageblock_order);
 
 	psi_memstall_enter(&pflags);
 	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
@@ -4638,8 +4649,9 @@ bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
  */
 static inline bool
 should_reclaim_retry(gfp_t gfp_mask, unsigned order,
-		     struct alloc_context *ac, int alloc_flags,
-		     bool did_some_progress, int *no_progress_loops)
+		     unsigned int reclaim_order, struct alloc_context *ac,
+		     int alloc_flags, bool did_some_progress,
+		     int *no_progress_loops)
 {
 	struct zone *zone;
 	struct zoneref *z;
@@ -4648,9 +4660,17 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 	/*
 	 * Costly allocations might have made a progress but this doesn't mean
 	 * their order will become available due to high fragmentation so
-	 * always increment the no progress counter for them
+	 * always increment the no progress counter for them.
+	 *
+	 * The same goes for requests whose reclaim is promoted to make whole
+	 * blocks. At that order, reclaim also reports progress when it backs
+	 * off for compaction without freeing anything.
+	 *
+	 * The watermark check below stays at the request order: it asks
+	 * whether the request itself could succeed after reclaim.
 	 */
-	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
+	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
+	    reclaim_order == order)
 		*no_progress_loops = 0;
 	else
 		(*no_progress_loops)++;
@@ -4790,6 +4810,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
 	struct page *page = NULL;
 	unsigned int alloc_flags;
+	unsigned int reclaim_order;
 	unsigned long did_some_progress;
 	enum compact_priority compact_priority;
 	enum compact_result compact_result;
@@ -4934,17 +4955,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	/* If allocation has taken excessively long, warn about it */
 	check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
 
+	/* The order reclaim and compaction work at, see nofrag_promote_order() */
+	reclaim_order = nofrag_promote_order(order, alloc_flags, ac);
+
 	/* Try direct reclaim and then allocating */
 	if (!compact_first) {
-		page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
-							ac, &did_some_progress);
+		page = __alloc_pages_direct_reclaim(gfp_mask, order, reclaim_order,
+						    alloc_flags, ac,
+						    &did_some_progress);
 		if (page)
 			goto got_pg;
 	}
 
 	/* Try direct compaction and then allocating */
-	page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac,
-					compact_priority, &compact_result);
+	page = __alloc_pages_direct_compact(gfp_mask, order, reclaim_order,
+					    alloc_flags, ac, compact_priority,
+					    &compact_result);
 	if (page)
 		goto got_pg;
 
@@ -4996,8 +5022,9 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	    check_retry_zonelist(zonelist_iter_cookie))
 		goto restart;
 
-	if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
-				 did_some_progress > 0, &no_progress_loops))
+	if (should_reclaim_retry(gfp_mask, order, reclaim_order, ac,
+				 alloc_flags, did_some_progress > 0,
+				 &no_progress_loops))
 		goto retry;
 
 	/*
@@ -5007,14 +5034,23 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	 * of free memory (see __compaction_suitable)
 	 */
 	if (did_some_progress > 0 && can_compact &&
-	    should_compact_retry(gfp_mask, ac, order, alloc_flags,
-				 compact_result, &compact_priority,
+	    should_compact_retry(gfp_mask, ac, order, reclaim_order,
+				 alloc_flags, compact_result, &compact_priority,
 				 &compaction_retries))
 		goto retry;
 
-	/* Reclaim/compaction failed to prevent the fallback */
+	/*
+	 * Reclaim/compaction failed to prevent the fallback. A promoted
+	 * request spent its retry budget on making blocks, not on the
+	 * request itself; give its fallback a fresh one before
+	 * considering OOM.
+	 */
 	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
 		alloc_flags &= ~ALLOC_NOFRAGMENT;
+		if (reclaim_order != order) {
+			no_progress_loops = 0;
+			compaction_retries = 0;
+		}
 		goto retry;
 	}
 
-- 
2.54.0

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Usama Arif]

On Fri,  9 Oct 2026 13:55:02 +0100 "Kiryl Shutsemau (Meta)" <kas@kernel.org> wrote:

> Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
> storm in defrag_mode"), direct reclaim and compaction for non-movable
> requests under defrag_mode run at pageblock_order, to produce the whole
> blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
> still use the request order. An order-0 request can therefore retry
> indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:
> 
> - Reclaim at pageblock_order gives up after one pass as soon as a zone
>   looks compaction_ready(), and do_try_to_free_pages() then returns 1
>   even though nothing was reclaimed. It returns before the retry that
>   would reclaim memory.low-protected cgroups, so when most memory is
>   protected, the pass that did run finds next to nothing.
> 
> - Compaction at pageblock_order fails or is deferred.
> 
> - should_reclaim_retry() takes the reported progress as progress for
>   the order-0 request and resets no_progress_loops. The request
>   retries.
> 
> Order 1-3 requests loop the same way, and should_compact_retry() also
> checks their pageblock_order compaction result against the request
> order.
> 
> On a production host (64G, defrag_mode, memory.low covering most of the
> workload), 95% of direct reclaim runs were order-9 runs that returned 1
> with nothing reclaimed, at up to 60k runs per second. Across ~200M
> should_reclaim_retry() calls in a day, no_progress_loops never left 0.
> The spinning allocations were SLUB slab refills for inode and dentry
> caches. The time spent registers as memory pressure, and pressure-based
> OOM killing takes down both workloads and system services.
> 
> For promoted requests:
> 
> - Reclaim progress does not reset no_progress_loops, as for costly
>   orders.
> 
> - should_compact_retry() checks the compaction result at the promoted
>   order and does not retry COMPACT_SKIPPED, since the request can fall
>   back. The compaction priority floor and the COMPACT_SUCCESS retry
>   limit stay those of the request order, so a non-costly request still
>   gets its COMPACT_PRIO_SYNC_FULL pass before it falls back.
> 
> When a promoted request takes the fallback, reset the retry counters,
> so that the fallback attempt gets a full retry budget before the OOM
> killer is considered. A request whose reclaim was not promoted spent
> its budget on the request itself and keeps its counters, as Usama Arif
> pointed out.
> 
> __alloc_pages_slowpath() computes the promoted order once per iteration
> and passes it to direct reclaim, direct compaction and the two retry
> helpers next to the request order, so no callee has to recompute it.
> Harry Yoo asked for the two orders to be explicit rather than derived
> in each callee.
> 
> In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
> 
>                                         before    after
>     should_reclaim_retry() calls           63M     293k
>     peak memory pressure (PSI some avg10)  99%      12%
> 
> File creation runs 5.7x faster.
> 
> Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>

Acked-by: Usama Arif <usama.arif@linux.dev>

> ---
> v3:
>  - reset the retry counters at the fallback only for requests whose
>    reclaim was promoted; a movable request already spent its budget on
>    itself (Usama Arif)
> 
> v2:
>  - keep the compaction priority floor and the COMPACT_SUCCESS retry
>    limit on the request order, so non-costly requests still escalate to
>    SYNC_FULL before falling back (Harry Yoo, Johannes Weiner)
>  - compute the promoted order once in __alloc_pages_slowpath() and pass
>    it to direct reclaim, direct compaction and the retry helpers, instead
>    of each of them deriving it (Harry Yoo)
> 
>  mm/page_alloc.c | 126 +++++++++++++++++++++++++++++++-----------------
>  1 file changed, 81 insertions(+), 45 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..c0be88d2c3ad 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4127,6 +4127,32 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  	return page;
>  }
>  
> +/*
> + * If fallbacks are not permitted (defrag_mode), we either need to
> + * reclaim space in a block of matching type, or clear out an entire
> + * block to allow __rmqueue_claim() to convert.
> + *
> + * Reclaim by itself is primarily freeing space in movable blocks,
> + * since that's where the LRU pages live. So this works for movable
> + * requests, but not for others.
> + *
> + * For those, promote the order of reclaim and compaction to help make
> + * blocks, instead of spinning in reclaim alone unproductively. Retry
> + * decisions based on the outcome of that work - reclaim progress and
> + * compaction results - must account for the promotion as well, so
> + * __alloc_pages_slowpath() computes the promoted order once and passes
> + * it alongside the request order.
> + */
> +static inline unsigned int nofrag_promote_order(unsigned int order,
> +						unsigned int alloc_flags,
> +						const struct alloc_context *ac)
> +{
> +	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> +		return max(order, pageblock_order);
> +
> +	return order;
> +}
> +
>  /*
>   * Maximum number of compaction retries with a progress before OOM
>   * killer is consider as the only way to move forward.
> @@ -4137,8 +4163,9 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  /* Try memory compaction for high-order allocations before reclaim */
>  static struct page *
>  __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		enum compact_priority prio, enum compact_result *compact_result)
> +		unsigned int compact_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, enum compact_priority prio,
> +		enum compact_result *compact_result)
>  {
>  	struct page *page = NULL;
>  	unsigned long pflags;
> @@ -4149,22 +4176,6 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  		.order = order,
>  		.page = NULL,
>  	};
> -	int compact_order = order;
> -
> -	/*
> -	 * If fallbacks are not permitted (defrag_mode), we either
> -	 * need to reclaim space in a block of matching type, or clear
> -	 * out an entire block to allow __rmqueue_claim() to convert.
> -	 *
> -	 * Reclaim by itself is primarily freeing space in movable
> -	 * blocks, since that's where the LRU pages live. So this
> -	 * works for movable requests, but not for others.
> -	 *
> -	 * For those, promote the order to help make blocks, instead
> -	 * of spinning in reclaim alone unproductively.
> -	 */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		compact_order = max(order, pageblock_order);
>  
>  	if (!compact_order)
>  		return NULL;
> @@ -4246,7 +4257,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  
>  static inline bool
>  should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
> -		     int alloc_flags,
> +		     int compact_order, int alloc_flags,
>  		     enum compact_result compact_result,
>  		     enum compact_priority *compact_priority,
>  		     int *compaction_retries)
> @@ -4257,7 +4268,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	int retries = *compaction_retries;
>  	enum compact_priority priority = *compact_priority;
>  
> -	if (!order)
> +	if (!compact_order)
>  		return false;
>  
>  	if (fatal_signal_pending(current))
> @@ -4266,10 +4277,14 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	/*
>  	 * Compaction was skipped due to a lack of free order-0
>  	 * migration targets. Continue if reclaim can help.
> +	 *
> +	 * Promoted requests have exhausted their reclaim retries at
> +	 * this point, and they can fall back instead.
>  	 */
>  	if (compact_result == COMPACT_SKIPPED) {
> -		ret = compaction_zonelist_suitable(ac, order, alloc_flags,
> -						   gfp_mask);
> +		if (compact_order == order)
> +			ret = compaction_zonelist_suitable(ac, order, alloc_flags,
> +							   gfp_mask);
>  		goto out;
>  	}
>  
> @@ -4314,8 +4329,9 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  #else
>  static inline struct page *
>  __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		enum compact_priority prio, enum compact_result *compact_result)
> +		unsigned int compact_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, enum compact_priority prio,
> +		enum compact_result *compact_result)
>  {
>  	*compact_result = COMPACT_SKIPPED;
>  	return NULL;
> @@ -4323,7 +4339,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  
>  static inline bool
>  should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
> -		     int alloc_flags,
> +		     int compact_order, int alloc_flags,
>  		     enum compact_result compact_result,
>  		     enum compact_priority *compact_priority,
>  		     int *compaction_retries)
> @@ -4462,17 +4478,12 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order,
>  /* The really slow allocator path where we enter direct reclaim */
>  static inline struct page *
>  __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		unsigned long *did_some_progress)
> +		unsigned int reclaim_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, unsigned long *did_some_progress)
>  {
>  	struct page *page = NULL;
>  	unsigned long pflags;
>  	bool drained = false;
> -	int reclaim_order = order;
> -
> -	/* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		reclaim_order = max(order, pageblock_order);
>  
>  	psi_memstall_enter(&pflags);
>  	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
> @@ -4638,8 +4649,9 @@ bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
>   */
>  static inline bool
>  should_reclaim_retry(gfp_t gfp_mask, unsigned order,
> -		     struct alloc_context *ac, int alloc_flags,
> -		     bool did_some_progress, int *no_progress_loops)
> +		     unsigned int reclaim_order, struct alloc_context *ac,
> +		     int alloc_flags, bool did_some_progress,
> +		     int *no_progress_loops)
>  {
>  	struct zone *zone;
>  	struct zoneref *z;
> @@ -4648,9 +4660,17 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
>  	/*
>  	 * Costly allocations might have made a progress but this doesn't mean
>  	 * their order will become available due to high fragmentation so
> -	 * always increment the no progress counter for them
> +	 * always increment the no progress counter for them.
> +	 *
> +	 * The same goes for requests whose reclaim is promoted to make whole
> +	 * blocks. At that order, reclaim also reports progress when it backs
> +	 * off for compaction without freeing anything.
> +	 *
> +	 * The watermark check below stays at the request order: it asks
> +	 * whether the request itself could succeed after reclaim.
>  	 */
> -	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
> +	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
> +	    reclaim_order == order)
>  		*no_progress_loops = 0;
>  	else
>  		(*no_progress_loops)++;
> @@ -4790,6 +4810,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
>  	struct page *page = NULL;
>  	unsigned int alloc_flags;
> +	unsigned int reclaim_order;
>  	unsigned long did_some_progress;
>  	enum compact_priority compact_priority;
>  	enum compact_result compact_result;
> @@ -4934,17 +4955,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	/* If allocation has taken excessively long, warn about it */
>  	check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
>  
> +	/* The order reclaim and compaction work at, see nofrag_promote_order() */
> +	reclaim_order = nofrag_promote_order(order, alloc_flags, ac);
> +
>  	/* Try direct reclaim and then allocating */
>  	if (!compact_first) {
> -		page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
> -							ac, &did_some_progress);
> +		page = __alloc_pages_direct_reclaim(gfp_mask, order, reclaim_order,
> +						    alloc_flags, ac,
> +						    &did_some_progress);
>  		if (page)
>  			goto got_pg;
>  	}
>  
>  	/* Try direct compaction and then allocating */
> -	page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac,
> -					compact_priority, &compact_result);
> +	page = __alloc_pages_direct_compact(gfp_mask, order, reclaim_order,
> +					    alloc_flags, ac, compact_priority,
> +					    &compact_result);
>  	if (page)
>  		goto got_pg;
>  
> @@ -4996,8 +5022,9 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	    check_retry_zonelist(zonelist_iter_cookie))
>  		goto restart;
>  
> -	if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
> -				 did_some_progress > 0, &no_progress_loops))
> +	if (should_reclaim_retry(gfp_mask, order, reclaim_order, ac,
> +				 alloc_flags, did_some_progress > 0,
> +				 &no_progress_loops))
>  		goto retry;
>  
>  	/*
> @@ -5007,14 +5034,23 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	 * of free memory (see __compaction_suitable)
>  	 */
>  	if (did_some_progress > 0 && can_compact &&
> -	    should_compact_retry(gfp_mask, ac, order, alloc_flags,
> -				 compact_result, &compact_priority,
> +	    should_compact_retry(gfp_mask, ac, order, reclaim_order,
> +				 alloc_flags, compact_result, &compact_priority,
>  				 &compaction_retries))
>  		goto retry;
>  
> -	/* Reclaim/compaction failed to prevent the fallback */
> +	/*
> +	 * Reclaim/compaction failed to prevent the fallback. A promoted
> +	 * request spent its retry budget on making blocks, not on the
> +	 * request itself; give its fallback a fresh one before
> +	 * considering OOM.
> +	 */
>  	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
>  		alloc_flags &= ~ALLOC_NOFRAGMENT;
> +		if (reclaim_order != order) {
> +			no_progress_loops = 0;
> +			compaction_retries = 0;
> +		}
>  		goto retry;
>  	}
>  
> -- 
> 2.54.0
> 
> 

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Andrew Morton]

On Fri,  9 Oct 2026 13:55:02 +0100 "Kiryl Shutsemau (Meta)" <kas@kernel.org> wrote:

> Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
> storm in defrag_mode"), direct reclaim and compaction for non-movable
> requests under defrag_mode run at pageblock_order, to produce the whole
> blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
> still use the request order. An order-0 request can therefore retry
> indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:
> 
> ...
> 
> When a promoted request takes the fallback, reset the retry counters,
> so that the fallback attempt gets a full retry budget before the OOM
> killer is considered. A request whose reclaim was not promoted spent
> its budget on the request itself and keeps its counters, as Usama Arif
> pointed out.

Thanks, I updated mm-unstable to this version.

> v3:
>  - reset the retry counters at the fallback only for requests whose
>    reclaim was promoted; a movable request already spent its budget on
>    itself (Usama Arif)

hannes's concerns/questions at
https://lore.kernel.org/20261006081050.GA234057@cmpxchg.org weren't
fully addressed?  I'll add a note and await followup.

Here's how v3 altered mm.git:

 mm/page_alloc.c |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

--- a/mm/page_alloc.c~mm-page_alloc-make-defrag_mode-retries-follow-the-promoted-order-v3
+++ a/mm/page_alloc.c
@@ -5030,14 +5030,17 @@ retry:
 		goto retry;
 
 	/*
-	 * Reclaim/compaction failed to prevent the fallback. The retry
-	 * budget was spent on making blocks, not on the request itself;
-	 * give the fallback a fresh one before considering OOM.
+	 * Reclaim/compaction failed to prevent the fallback. A promoted
+	 * request spent its retry budget on making blocks, not on the
+	 * request itself; give its fallback a fresh one before
+	 * considering OOM.
 	 */
 	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
 		alloc_flags &= ~ALLOC_NOFRAGMENT;
-		no_progress_loops = 0;
-		compaction_retries = 0;
+		if (reclaim_order != order) {
+			no_progress_loops = 0;
+			compaction_retries = 0;
+		}
 		goto retry;
 	}
 
_

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Sat, Oct 10, 2026 at 03:49:23PM -0700, Andrew Morton wrote:
> On Fri,  9 Oct 2026 13:55:02 +0100 "Kiryl Shutsemau (Meta)" <kas@kernel.org> wrote:
> 
> > Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
> > storm in defrag_mode"), direct reclaim and compaction for non-movable
> > requests under defrag_mode run at pageblock_order, to produce the whole
> > blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
> > still use the request order. An order-0 request can therefore retry
> > indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:
> > 
> > ...
> > 
> > When a promoted request takes the fallback, reset the retry counters,
> > so that the fallback attempt gets a full retry budget before the OOM
> > killer is considered. A request whose reclaim was not promoted spent
> > its budget on the request itself and keeps its counters, as Usama Arif
> > pointed out.
> 
> Thanks, I updated mm-unstable to this version.
> 
> > v3:
> >  - reset the retry counters at the fallback only for requests whose
> >    reclaim was promoted; a movable request already spent its budget on
> >    itself (Usama Arif)
> 
> hannes's concerns/questions at
> https://lore.kernel.org/20261006081050.GA234057@cmpxchg.org weren't
> fully addressed?  I'll add a note and await followup.

Hm. I believe they were addressed. v2 changes are based on Harry's and
Johannes feedback.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Vlastimil Babka (SUSE)]

On 10/9/26 14:55, Kiryl Shutsemau (Meta) wrote:
> Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
> storm in defrag_mode"), direct reclaim and compaction for non-movable
> requests under defrag_mode run at pageblock_order, to produce the whole
> blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
> still use the request order. An order-0 request can therefore retry
> indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:
> 
> - Reclaim at pageblock_order gives up after one pass as soon as a zone
>   looks compaction_ready(), and do_try_to_free_pages() then returns 1
>   even though nothing was reclaimed. It returns before the retry that
>   would reclaim memory.low-protected cgroups, so when most memory is
>   protected, the pass that did run finds next to nothing.
> 
> - Compaction at pageblock_order fails or is deferred.
> 
> - should_reclaim_retry() takes the reported progress as progress for
>   the order-0 request and resets no_progress_loops. The request
>   retries.
> 
> Order 1-3 requests loop the same way, and should_compact_retry() also
> checks their pageblock_order compaction result against the request
> order.
> 
> On a production host (64G, defrag_mode, memory.low covering most of the
> workload), 95% of direct reclaim runs were order-9 runs that returned 1
> with nothing reclaimed, at up to 60k runs per second. Across ~200M
> should_reclaim_retry() calls in a day, no_progress_loops never left 0.
> The spinning allocations were SLUB slab refills for inode and dentry
> caches. The time spent registers as memory pressure, and pressure-based
> OOM killing takes down both workloads and system services.
> 
> For promoted requests:
> 
> - Reclaim progress does not reset no_progress_loops, as for costly
>   orders.
> 
> - should_compact_retry() checks the compaction result at the promoted
>   order and does not retry COMPACT_SKIPPED, since the request can fall
>   back. The compaction priority floor and the COMPACT_SUCCESS retry
>   limit stay those of the request order, so a non-costly request still
>   gets its COMPACT_PRIO_SYNC_FULL pass before it falls back.
> 
> When a promoted request takes the fallback, reset the retry counters,
> so that the fallback attempt gets a full retry budget before the OOM
> killer is considered. A request whose reclaim was not promoted spent
> its budget on the request itself and keeps its counters, as Usama Arif
> pointed out.
> 
> __alloc_pages_slowpath() computes the promoted order once per iteration
> and passes it to direct reclaim, direct compaction and the two retry
> helpers next to the request order, so no callee has to recompute it.
> Harry Yoo asked for the two orders to be explicit rather than derived
> in each callee.
> 
> In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
> 
>                                         before    after
>     should_reclaim_retry() calls           63M     293k
>     peak memory pressure (PSI some avg10)  99%      12%
> 
> File creation runs 5.7x faster.
> 
> Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>

LGTM, but I think we could simplify and get rid of all the function
parameter additions by adding a field to alloc_context instead? We could
call it e.g. reclaim_compact_order.

The function nofrag_promote_order() probably doesn't need to exist, given it
has just one caller, could be just open coded.

What could be useful instead is wrapper for comparing order vs
ac->reclaim_compact_order. E.g. something like bool
order_nofrag_promoted(order, ac) ?

Vlastimil

> ---
> v3:
>  - reset the retry counters at the fallback only for requests whose
>    reclaim was promoted; a movable request already spent its budget on
>    itself (Usama Arif)
> 
> v2:
>  - keep the compaction priority floor and the COMPACT_SUCCESS retry
>    limit on the request order, so non-costly requests still escalate to
>    SYNC_FULL before falling back (Harry Yoo, Johannes Weiner)
>  - compute the promoted order once in __alloc_pages_slowpath() and pass
>    it to direct reclaim, direct compaction and the retry helpers, instead
>    of each of them deriving it (Harry Yoo)
> 
>  mm/page_alloc.c | 126 +++++++++++++++++++++++++++++++-----------------
>  1 file changed, 81 insertions(+), 45 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..c0be88d2c3ad 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4127,6 +4127,32 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  	return page;
>  }
>  
> +/*
> + * If fallbacks are not permitted (defrag_mode), we either need to
> + * reclaim space in a block of matching type, or clear out an entire
> + * block to allow __rmqueue_claim() to convert.
> + *
> + * Reclaim by itself is primarily freeing space in movable blocks,
> + * since that's where the LRU pages live. So this works for movable
> + * requests, but not for others.
> + *
> + * For those, promote the order of reclaim and compaction to help make
> + * blocks, instead of spinning in reclaim alone unproductively. Retry
> + * decisions based on the outcome of that work - reclaim progress and
> + * compaction results - must account for the promotion as well, so
> + * __alloc_pages_slowpath() computes the promoted order once and passes
> + * it alongside the request order.
> + */
> +static inline unsigned int nofrag_promote_order(unsigned int order,
> +						unsigned int alloc_flags,
> +						const struct alloc_context *ac)
> +{
> +	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> +		return max(order, pageblock_order);
> +
> +	return order;
> +}
> +
>  /*
>   * Maximum number of compaction retries with a progress before OOM
>   * killer is consider as the only way to move forward.
> @@ -4137,8 +4163,9 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  /* Try memory compaction for high-order allocations before reclaim */
>  static struct page *
>  __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		enum compact_priority prio, enum compact_result *compact_result)
> +		unsigned int compact_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, enum compact_priority prio,
> +		enum compact_result *compact_result)
>  {
>  	struct page *page = NULL;
>  	unsigned long pflags;
> @@ -4149,22 +4176,6 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  		.order = order,
>  		.page = NULL,
>  	};
> -	int compact_order = order;
> -
> -	/*
> -	 * If fallbacks are not permitted (defrag_mode), we either
> -	 * need to reclaim space in a block of matching type, or clear
> -	 * out an entire block to allow __rmqueue_claim() to convert.
> -	 *
> -	 * Reclaim by itself is primarily freeing space in movable
> -	 * blocks, since that's where the LRU pages live. So this
> -	 * works for movable requests, but not for others.
> -	 *
> -	 * For those, promote the order to help make blocks, instead
> -	 * of spinning in reclaim alone unproductively.
> -	 */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		compact_order = max(order, pageblock_order);
>  
>  	if (!compact_order)
>  		return NULL;
> @@ -4246,7 +4257,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  
>  static inline bool
>  should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
> -		     int alloc_flags,
> +		     int compact_order, int alloc_flags,
>  		     enum compact_result compact_result,
>  		     enum compact_priority *compact_priority,
>  		     int *compaction_retries)
> @@ -4257,7 +4268,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	int retries = *compaction_retries;
>  	enum compact_priority priority = *compact_priority;
>  
> -	if (!order)
> +	if (!compact_order)
>  		return false;
>  
>  	if (fatal_signal_pending(current))
> @@ -4266,10 +4277,14 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	/*
>  	 * Compaction was skipped due to a lack of free order-0
>  	 * migration targets. Continue if reclaim can help.
> +	 *
> +	 * Promoted requests have exhausted their reclaim retries at
> +	 * this point, and they can fall back instead.
>  	 */
>  	if (compact_result == COMPACT_SKIPPED) {
> -		ret = compaction_zonelist_suitable(ac, order, alloc_flags,
> -						   gfp_mask);
> +		if (compact_order == order)
> +			ret = compaction_zonelist_suitable(ac, order, alloc_flags,
> +							   gfp_mask);
>  		goto out;
>  	}
>  
> @@ -4314,8 +4329,9 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  #else
>  static inline struct page *
>  __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		enum compact_priority prio, enum compact_result *compact_result)
> +		unsigned int compact_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, enum compact_priority prio,
> +		enum compact_result *compact_result)
>  {
>  	*compact_result = COMPACT_SKIPPED;
>  	return NULL;
> @@ -4323,7 +4339,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  
>  static inline bool
>  should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
> -		     int alloc_flags,
> +		     int compact_order, int alloc_flags,
>  		     enum compact_result compact_result,
>  		     enum compact_priority *compact_priority,
>  		     int *compaction_retries)
> @@ -4462,17 +4478,12 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order,
>  /* The really slow allocator path where we enter direct reclaim */
>  static inline struct page *
>  __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
> -		unsigned int alloc_flags, const struct alloc_context *ac,
> -		unsigned long *did_some_progress)
> +		unsigned int reclaim_order, unsigned int alloc_flags,
> +		const struct alloc_context *ac, unsigned long *did_some_progress)
>  {
>  	struct page *page = NULL;
>  	unsigned long pflags;
>  	bool drained = false;
> -	int reclaim_order = order;
> -
> -	/* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		reclaim_order = max(order, pageblock_order);
>  
>  	psi_memstall_enter(&pflags);
>  	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
> @@ -4638,8 +4649,9 @@ bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
>   */
>  static inline bool
>  should_reclaim_retry(gfp_t gfp_mask, unsigned order,
> -		     struct alloc_context *ac, int alloc_flags,
> -		     bool did_some_progress, int *no_progress_loops)
> +		     unsigned int reclaim_order, struct alloc_context *ac,
> +		     int alloc_flags, bool did_some_progress,
> +		     int *no_progress_loops)
>  {
>  	struct zone *zone;
>  	struct zoneref *z;
> @@ -4648,9 +4660,17 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
>  	/*
>  	 * Costly allocations might have made a progress but this doesn't mean
>  	 * their order will become available due to high fragmentation so
> -	 * always increment the no progress counter for them
> +	 * always increment the no progress counter for them.
> +	 *
> +	 * The same goes for requests whose reclaim is promoted to make whole
> +	 * blocks. At that order, reclaim also reports progress when it backs
> +	 * off for compaction without freeing anything.
> +	 *
> +	 * The watermark check below stays at the request order: it asks
> +	 * whether the request itself could succeed after reclaim.
>  	 */
> -	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
> +	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
> +	    reclaim_order == order)
>  		*no_progress_loops = 0;
>  	else
>  		(*no_progress_loops)++;
> @@ -4790,6 +4810,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER;
>  	struct page *page = NULL;
>  	unsigned int alloc_flags;
> +	unsigned int reclaim_order;
>  	unsigned long did_some_progress;
>  	enum compact_priority compact_priority;
>  	enum compact_result compact_result;
> @@ -4934,17 +4955,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	/* If allocation has taken excessively long, warn about it */
>  	check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
>  
> +	/* The order reclaim and compaction work at, see nofrag_promote_order() */
> +	reclaim_order = nofrag_promote_order(order, alloc_flags, ac);
> +
>  	/* Try direct reclaim and then allocating */
>  	if (!compact_first) {
> -		page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
> -							ac, &did_some_progress);
> +		page = __alloc_pages_direct_reclaim(gfp_mask, order, reclaim_order,
> +						    alloc_flags, ac,
> +						    &did_some_progress);
>  		if (page)
>  			goto got_pg;
>  	}
>  
>  	/* Try direct compaction and then allocating */
> -	page = __alloc_pages_direct_compact(gfp_mask, order, alloc_flags, ac,
> -					compact_priority, &compact_result);
> +	page = __alloc_pages_direct_compact(gfp_mask, order, reclaim_order,
> +					    alloc_flags, ac, compact_priority,
> +					    &compact_result);
>  	if (page)
>  		goto got_pg;
>  
> @@ -4996,8 +5022,9 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	    check_retry_zonelist(zonelist_iter_cookie))
>  		goto restart;
>  
> -	if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
> -				 did_some_progress > 0, &no_progress_loops))
> +	if (should_reclaim_retry(gfp_mask, order, reclaim_order, ac,
> +				 alloc_flags, did_some_progress > 0,
> +				 &no_progress_loops))
>  		goto retry;
>  
>  	/*
> @@ -5007,14 +5034,23 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	 * of free memory (see __compaction_suitable)
>  	 */
>  	if (did_some_progress > 0 && can_compact &&
> -	    should_compact_retry(gfp_mask, ac, order, alloc_flags,
> -				 compact_result, &compact_priority,
> +	    should_compact_retry(gfp_mask, ac, order, reclaim_order,
> +				 alloc_flags, compact_result, &compact_priority,
>  				 &compaction_retries))
>  		goto retry;
>  
> -	/* Reclaim/compaction failed to prevent the fallback */
> +	/*
> +	 * Reclaim/compaction failed to prevent the fallback. A promoted
> +	 * request spent its retry budget on making blocks, not on the
> +	 * request itself; give its fallback a fresh one before
> +	 * considering OOM.
> +	 */
>  	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
>  		alloc_flags &= ~ALLOC_NOFRAGMENT;
> +		if (reclaim_order != order) {
> +			no_progress_loops = 0;
> +			compaction_retries = 0;
> +		}
>  		goto retry;
>  	}
>  

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Andrew Morton]

On Mon, 12 Oct 2026 13:40:36 +0200 "Vlastimil Babka (SUSE)" <vbabka@kernel.org> wrote:

> > In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
> > 
> >                                         before    after
> >     should_reclaim_retry() calls           63M     293k
> >     peak memory pressure (PSI some avg10)  99%      12%
> > 
> > File creation runs 5.7x faster.
> > 
> > Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
> > Cc: stable@vger.kernel.org
> > Assisted-by: LLM
> > Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> 
> LGTM, but I think we could simplify and get rid of all the function
> parameter additions by adding a field to alloc_context instead? We could
> call it e.g. reclaim_compact_order.
> 
> The function nofrag_promote_order() probably doesn't need to exist, given it
> has just one caller, could be just open coded.
> 
> What could be useful instead is wrapper for comparing order vs
> ac->reclaim_compact_order. E.g. something like bool
> order_nofrag_promoted(order, ac) ?

I'm concerned about the sheer size of this patch.  It fixes significant
issues, but it's not going to be a simple or safe backport.

So for now please let's optimize for simplicity and backportability and
defer cleanups for the -rc cycle.

Of course, if the changes you suggest achieve that then go for it.

Perhaps we should defer this change into next -rc?  It has cc:stable so
it'll eventually end up where we want it.

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Mon, Oct 12, 2026 at 07:28:34PM -0700, Andrew Morton wrote:
> On Mon, 12 Oct 2026 13:40:36 +0200 "Vlastimil Babka (SUSE)" <vbabka@kernel.org> wrote:
> 
> > > In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
> > > 
> > >                                         before    after
> > >     should_reclaim_retry() calls           63M     293k
> > >     peak memory pressure (PSI some avg10)  99%      12%
> > > 
> > > File creation runs 5.7x faster.
> > > 
> > > Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
> > > Cc: stable@vger.kernel.org
> > > Assisted-by: LLM
> > > Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> > 
> > LGTM, but I think we could simplify and get rid of all the function
> > parameter additions by adding a field to alloc_context instead? We could
> > call it e.g. reclaim_compact_order.
> > 
> > The function nofrag_promote_order() probably doesn't need to exist, given it
> > has just one caller, could be just open coded.
> > 
> > What could be useful instead is wrapper for comparing order vs
> > ac->reclaim_compact_order. E.g. something like bool
> > order_nofrag_promoted(order, ac) ?
> 
> I'm concerned about the sheer size of this patch.  It fixes significant
> issues, but it's not going to be a simple or safe backport.
> 
> So for now please let's optimize for simplicity and backportability and
> defer cleanups for the -rc cycle.
> 
> Of course, if the changes you suggest achieve that then go for it.

They do. The patch with Vlastimil's suggestion is below. It is shorter.
Behaviour is unchanged; the reproducer gives the same results on both
versions.

Vlastimil, does it look like what you had in mind?

> Perhaps we should defer this change into next -rc?  It has cc:stable so
> it'll eventually end up where we want it.

Do you mean the next -rc cycle? I hoped to get it into 7.4-rc1, as we
agreed earlier.

----------------------------------->8----------------------------------

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
Subject: [PATCH v4] mm: page_alloc: make defrag_mode retries follow the promoted order

Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
storm in defrag_mode"), direct reclaim and compaction for non-movable
requests under defrag_mode run at pageblock_order, to produce the whole
blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
still use the request order. An order-0 request can therefore retry
indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:

- Reclaim at pageblock_order gives up after one pass as soon as a zone
  looks compaction_ready(), and do_try_to_free_pages() then returns 1
  even though nothing was reclaimed. It returns before the retry that
  would reclaim memory.low-protected cgroups, so when most memory is
  protected, the pass that did run finds next to nothing.

- Compaction at pageblock_order fails or is deferred.

- should_reclaim_retry() takes the reported progress as progress for
  the order-0 request and resets no_progress_loops. The request
  retries.

Order 1-3 requests loop the same way, and should_compact_retry() also
checks their pageblock_order compaction result against the request
order.

On a production host (64G, defrag_mode, memory.low covering most of the
workload), 95% of direct reclaim runs were order-9 runs that returned 1
with nothing reclaimed, at up to 60k runs per second. Across ~200M
should_reclaim_retry() calls in a day, no_progress_loops never left 0.
The spinning allocations were SLUB slab refills for inode and dentry
caches. The time spent registers as memory pressure, and pressure-based
OOM killing takes down both workloads and system services.

For promoted requests:

- Reclaim progress does not reset no_progress_loops, as for costly
  orders.

- should_compact_retry() checks the compaction result at the promoted
  order and does not retry COMPACT_SKIPPED, since the request can fall
  back. The compaction priority floor and the COMPACT_SUCCESS retry
  limit stay those of the request order, so a non-costly request still
  gets its COMPACT_PRIO_SYNC_FULL pass before it falls back.

When a promoted request takes the fallback, reset the retry counters,
so that the fallback attempt gets a full retry budget before the OOM
killer is considered. A request whose reclaim was not promoted spent
its budget on the request itself and keeps its counters, as Usama Arif
pointed out.

__alloc_pages_slowpath() computes the promoted order once per iteration
and keeps it in alloc_context for direct reclaim, direct compaction and
the two retry helpers, so no callee has to recompute it. Harry Yoo asked
for the promoted order to be explicit rather than derived in each
callee, and Vlastimil Babka suggested alloc_context over new function
parameters.

In a VM reproducer (32G, defrag_mode, inode churn under memory.low):

                                        before    after
    should_reclaim_retry() calls           63M     293k
    peak memory pressure (PSI some avg10)  99%      12%

File creation runs 5.7x faster.

Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
v4:
 - keep the promoted order in alloc_context instead of passing it as a
   new parameter (Vlastimil Babka)

 mm/page_alloc.c | 65 +++++++++++++++++++++++++++++--------------------
 mm/page_alloc.h |  2 ++
 2 files changed, 41 insertions(+), 26 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..c593913a14f8 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4127,6 +4127,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 	return page;
 }
 
+static inline bool order_nofrag_promoted(unsigned int order,
+					 const struct alloc_context *ac)
+{
+	return ac->reclaim_compact_order != order;
+}
+
 /*
  * Maximum number of compaction retries with a progress before OOM
  * killer is consider as the only way to move forward.
@@ -4149,22 +4155,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
 		.order = order,
 		.page = NULL,
 	};
-	int compact_order = order;
-
-	/*
-	 * If fallbacks are not permitted (defrag_mode), we either
-	 * need to reclaim space in a block of matching type, or clear
-	 * out an entire block to allow __rmqueue_claim() to convert.
-	 *
-	 * Reclaim by itself is primarily freeing space in movable
-	 * blocks, since that's where the LRU pages live. So this
-	 * works for movable requests, but not for others.
-	 *
-	 * For those, promote the order to help make blocks, instead
-	 * of spinning in reclaim alone unproductively.
-	 */
-	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
-		compact_order = max(order, pageblock_order);
+	int compact_order = ac->reclaim_compact_order;
 
 	if (!compact_order)
 		return NULL;
@@ -4257,7 +4248,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 	int retries = *compaction_retries;
 	enum compact_priority priority = *compact_priority;
 
-	if (!order)
+	if (!ac->reclaim_compact_order)
 		return false;
 
 	if (fatal_signal_pending(current))
@@ -4266,10 +4257,12 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 	/*
 	 * Compaction was skipped due to a lack of free order-0
 	 * migration targets. Continue if reclaim can help.
+	 * Promoted requests fall back instead.
 	 */
 	if (compact_result == COMPACT_SKIPPED) {
-		ret = compaction_zonelist_suitable(ac, order, alloc_flags,
-						   gfp_mask);
+		if (!order_nofrag_promoted(order, ac))
+			ret = compaction_zonelist_suitable(ac, order,
+							   alloc_flags, gfp_mask);
 		goto out;
 	}
 
@@ -4468,11 +4461,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
 	struct page *page = NULL;
 	unsigned long pflags;
 	bool drained = false;
-	int reclaim_order = order;
-
-	/* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
-	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
-		reclaim_order = max(order, pageblock_order);
+	int reclaim_order = ac->reclaim_compact_order;
 
 	psi_memstall_enter(&pflags);
 	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
@@ -4648,9 +4637,12 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 	/*
 	 * Costly allocations might have made a progress but this doesn't mean
 	 * their order will become available due to high fragmentation so
-	 * always increment the no progress counter for them
+	 * always increment the no progress counter for them. The same goes for
+	 * promoted requests: their reclaim reports progress when it backs off
+	 * for compaction without freeing anything.
 	 */
-	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
+	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
+	    !order_nofrag_promoted(order, ac))
 		*no_progress_loops = 0;
 	else
 		(*no_progress_loops)++;
@@ -4934,6 +4926,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	/* If allocation has taken excessively long, warn about it */
 	check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
 
+	/*
+	 * If fallbacks are not permitted (defrag_mode), we either
+	 * need to reclaim space in a block of matching type, or clear
+	 * out an entire block to allow __rmqueue_claim() to convert.
+	 *
+	 * Reclaim by itself is primarily freeing space in movable
+	 * blocks, since that's where the LRU pages live. So this
+	 * works for movable requests, but not for others.
+	 *
+	 * For those, promote the order to help make blocks, instead
+	 * of spinning in reclaim alone unproductively.
+	 */
+	ac->reclaim_compact_order = order;
+	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
+		ac->reclaim_compact_order = max(order, pageblock_order);
+
 	/* Try direct reclaim and then allocating */
 	if (!compact_first) {
 		page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
@@ -5015,6 +5023,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	/* Reclaim/compaction failed to prevent the fallback */
 	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
 		alloc_flags &= ~ALLOC_NOFRAGMENT;
+		/* Retries went into making blocks, start the fallback afresh */
+		if (order_nofrag_promoted(order, ac)) {
+			no_progress_loops = 0;
+			compaction_retries = 0;
+		}
 		goto retry;
 	}
 
diff --git a/mm/page_alloc.h b/mm/page_alloc.h
index b9259deddb59..2b1017a91928 100644
--- a/mm/page_alloc.h
+++ b/mm/page_alloc.h
@@ -93,6 +93,8 @@ struct alloc_context {
 	bool spread_dirty_pages;
 	/* Only flags that are global to the whole allocation go here. */
 	unsigned int alloc_flags;
+	/* Set by __alloc_pages_slowpath() before reclaim and compaction */
+	unsigned int reclaim_compact_order;
 };
 
 /*
-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Vlastimil Babka (SUSE)]

On 10/13/26 19:37, Kiryl Shutsemau wrote:
> On Mon, Oct 12, 2026 at 07:28:34PM -0700, Andrew Morton wrote:
>> On Mon, 12 Oct 2026 13:40:36 +0200 "Vlastimil Babka (SUSE)" <vbabka@kernel.org> wrote:
>> 
>> > > In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
>> > > 
>> > >                                         before    after
>> > >     should_reclaim_retry() calls           63M     293k
>> > >     peak memory pressure (PSI some avg10)  99%      12%
>> > > 
>> > > File creation runs 5.7x faster.
>> > > 
>> > > Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
>> > > Cc: stable@vger.kernel.org
>> > > Assisted-by: LLM
>> > > Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
>> > 
>> > LGTM, but I think we could simplify and get rid of all the function
>> > parameter additions by adding a field to alloc_context instead? We could
>> > call it e.g. reclaim_compact_order.
>> > 
>> > The function nofrag_promote_order() probably doesn't need to exist, given it
>> > has just one caller, could be just open coded.
>> > 
>> > What could be useful instead is wrapper for comparing order vs
>> > ac->reclaim_compact_order. E.g. something like bool
>> > order_nofrag_promoted(order, ac) ?
>> 
>> I'm concerned about the sheer size of this patch.  It fixes significant
>> issues, but it's not going to be a simple or safe backport.
>> 
>> So for now please let's optimize for simplicity and backportability and
>> defer cleanups for the -rc cycle.
>> 
>> Of course, if the changes you suggest achieve that then go for it.
> 
> They do. The patch with Vlastimil's suggestion is below. It is shorter.
> Behaviour is unchanged; the reproducer gives the same results on both
> versions.
> 
> Vlastimil, does it look like what you had in mind?

Yeah LGTM.

Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>

> 
>> Perhaps we should defer this change into next -rc?  It has cc:stable so
>> it'll eventually end up where we want it.
> 
> Do you mean the next -rc cycle? I hoped to get it into 7.4-rc1, as we
> agreed earlier.

We're fixing a 7.3 commit that went into the stable trees. Too late to rush
for the 7.3 final though. The impact is mitigated by defrag_mode not being
default, so it's fine to fix it in 7.3.x. Thus 7.4-rc1 would be great, rc2
could also work, but wouldn't delay more. I read the merge window might be 3
weeks this time [1] so perhaps rc1 is comfortable in any case.

[1] https://lwn.net/Articles/1099594

> 
> ----------------------------------->8----------------------------------
> 
> From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
> Subject: [PATCH v4] mm: page_alloc: make defrag_mode retries follow the promoted order
> 
> Since commit 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim
> storm in defrag_mode"), direct reclaim and compaction for non-movable
> requests under defrag_mode run at pageblock_order, to produce the whole
> blocks that ALLOC_NOFRAGMENT needs. The retry decisions that follow
> still use the request order. An order-0 request can therefore retry
> indefinitely without ever reaching the ALLOC_NOFRAGMENT fallback:
> 
> - Reclaim at pageblock_order gives up after one pass as soon as a zone
>   looks compaction_ready(), and do_try_to_free_pages() then returns 1
>   even though nothing was reclaimed. It returns before the retry that
>   would reclaim memory.low-protected cgroups, so when most memory is
>   protected, the pass that did run finds next to nothing.
> 
> - Compaction at pageblock_order fails or is deferred.
> 
> - should_reclaim_retry() takes the reported progress as progress for
>   the order-0 request and resets no_progress_loops. The request
>   retries.
> 
> Order 1-3 requests loop the same way, and should_compact_retry() also
> checks their pageblock_order compaction result against the request
> order.
> 
> On a production host (64G, defrag_mode, memory.low covering most of the
> workload), 95% of direct reclaim runs were order-9 runs that returned 1
> with nothing reclaimed, at up to 60k runs per second. Across ~200M
> should_reclaim_retry() calls in a day, no_progress_loops never left 0.
> The spinning allocations were SLUB slab refills for inode and dentry
> caches. The time spent registers as memory pressure, and pressure-based
> OOM killing takes down both workloads and system services.
> 
> For promoted requests:
> 
> - Reclaim progress does not reset no_progress_loops, as for costly
>   orders.
> 
> - should_compact_retry() checks the compaction result at the promoted
>   order and does not retry COMPACT_SKIPPED, since the request can fall
>   back. The compaction priority floor and the COMPACT_SUCCESS retry
>   limit stay those of the request order, so a non-costly request still
>   gets its COMPACT_PRIO_SYNC_FULL pass before it falls back.
> 
> When a promoted request takes the fallback, reset the retry counters,
> so that the fallback attempt gets a full retry budget before the OOM
> killer is considered. A request whose reclaim was not promoted spent
> its budget on the request itself and keeps its counters, as Usama Arif
> pointed out.
> 
> __alloc_pages_slowpath() computes the promoted order once per iteration
> and keeps it in alloc_context for direct reclaim, direct compaction and
> the two retry helpers, so no callee has to recompute it. Harry Yoo asked
> for the promoted order to be explicit rather than derived in each
> callee, and Vlastimil Babka suggested alloc_context over new function
> parameters.
> 
> In a VM reproducer (32G, defrag_mode, inode churn under memory.low):
> 
>                                         before    after
>     should_reclaim_retry() calls           63M     293k
>     peak memory pressure (PSI some avg10)  99%      12%
> 
> File creation runs 5.7x faster.
> 
> Fixes: 7e8756d7ad22 ("mm: page_alloc: fix non-movable reclaim storm in defrag_mode")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> ---
> v4:
>  - keep the promoted order in alloc_context instead of passing it as a
>    new parameter (Vlastimil Babka)
> 
>  mm/page_alloc.c | 65 +++++++++++++++++++++++++++++--------------------
>  mm/page_alloc.h |  2 ++
>  2 files changed, 41 insertions(+), 26 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..c593913a14f8 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -4127,6 +4127,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  	return page;
>  }
>  
> +static inline bool order_nofrag_promoted(unsigned int order,
> +					 const struct alloc_context *ac)
> +{
> +	return ac->reclaim_compact_order != order;
> +}
> +
>  /*
>   * Maximum number of compaction retries with a progress before OOM
>   * killer is consider as the only way to move forward.
> @@ -4149,22 +4155,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order,
>  		.order = order,
>  		.page = NULL,
>  	};
> -	int compact_order = order;
> -
> -	/*
> -	 * If fallbacks are not permitted (defrag_mode), we either
> -	 * need to reclaim space in a block of matching type, or clear
> -	 * out an entire block to allow __rmqueue_claim() to convert.
> -	 *
> -	 * Reclaim by itself is primarily freeing space in movable
> -	 * blocks, since that's where the LRU pages live. So this
> -	 * works for movable requests, but not for others.
> -	 *
> -	 * For those, promote the order to help make blocks, instead
> -	 * of spinning in reclaim alone unproductively.
> -	 */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		compact_order = max(order, pageblock_order);
> +	int compact_order = ac->reclaim_compact_order;
>  
>  	if (!compact_order)
>  		return NULL;
> @@ -4257,7 +4248,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	int retries = *compaction_retries;
>  	enum compact_priority priority = *compact_priority;
>  
> -	if (!order)
> +	if (!ac->reclaim_compact_order)
>  		return false;
>  
>  	if (fatal_signal_pending(current))
> @@ -4266,10 +4257,12 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
>  	/*
>  	 * Compaction was skipped due to a lack of free order-0
>  	 * migration targets. Continue if reclaim can help.
> +	 * Promoted requests fall back instead.
>  	 */
>  	if (compact_result == COMPACT_SKIPPED) {
> -		ret = compaction_zonelist_suitable(ac, order, alloc_flags,
> -						   gfp_mask);
> +		if (!order_nofrag_promoted(order, ac))
> +			ret = compaction_zonelist_suitable(ac, order,
> +							   alloc_flags, gfp_mask);
>  		goto out;
>  	}
>  
> @@ -4468,11 +4461,7 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
>  	struct page *page = NULL;
>  	unsigned long pflags;
>  	bool drained = false;
> -	int reclaim_order = order;
> -
> -	/* Match the slowpath compaction promotion in __alloc_pages_direct_compact */
> -	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> -		reclaim_order = max(order, pageblock_order);
> +	int reclaim_order = ac->reclaim_compact_order;
>  
>  	psi_memstall_enter(&pflags);
>  	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
> @@ -4648,9 +4637,12 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
>  	/*
>  	 * Costly allocations might have made a progress but this doesn't mean
>  	 * their order will become available due to high fragmentation so
> -	 * always increment the no progress counter for them
> +	 * always increment the no progress counter for them. The same goes for
> +	 * promoted requests: their reclaim reports progress when it backs off
> +	 * for compaction without freeing anything.
>  	 */
> -	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER)
> +	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
> +	    !order_nofrag_promoted(order, ac))
>  		*no_progress_loops = 0;
>  	else
>  		(*no_progress_loops)++;
> @@ -4934,6 +4926,22 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	/* If allocation has taken excessively long, warn about it */
>  	check_alloc_stall_warn(gfp_mask, ac->nodemask, order, alloc_start_time);
>  
> +	/*
> +	 * If fallbacks are not permitted (defrag_mode), we either
> +	 * need to reclaim space in a block of matching type, or clear
> +	 * out an entire block to allow __rmqueue_claim() to convert.
> +	 *
> +	 * Reclaim by itself is primarily freeing space in movable
> +	 * blocks, since that's where the LRU pages live. So this
> +	 * works for movable requests, but not for others.
> +	 *
> +	 * For those, promote the order to help make blocks, instead
> +	 * of spinning in reclaim alone unproductively.
> +	 */
> +	ac->reclaim_compact_order = order;
> +	if ((alloc_flags & ALLOC_NOFRAGMENT) && ac->migratetype != MIGRATE_MOVABLE)
> +		ac->reclaim_compact_order = max(order, pageblock_order);
> +
>  	/* Try direct reclaim and then allocating */
>  	if (!compact_first) {
>  		page = __alloc_pages_direct_reclaim(gfp_mask, order, alloc_flags,
> @@ -5015,6 +5023,11 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
>  	/* Reclaim/compaction failed to prevent the fallback */
>  	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
>  		alloc_flags &= ~ALLOC_NOFRAGMENT;
> +		/* Retries went into making blocks, start the fallback afresh */
> +		if (order_nofrag_promoted(order, ac)) {
> +			no_progress_loops = 0;
> +			compaction_retries = 0;
> +		}
>  		goto retry;
>  	}
>  
> diff --git a/mm/page_alloc.h b/mm/page_alloc.h
> index b9259deddb59..2b1017a91928 100644
> --- a/mm/page_alloc.h
> +++ b/mm/page_alloc.h
> @@ -93,6 +93,8 @@ struct alloc_context {
>  	bool spread_dirty_pages;
>  	/* Only flags that are global to the whole allocation go here. */
>  	unsigned int alloc_flags;
> +	/* Set by __alloc_pages_slowpath() before reclaim and compaction */
> +	unsigned int reclaim_compact_order;
>  };
>  
>  /*

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Johannes Weiner]

On Tue, Oct 13, 2026 at 06:37:20PM +0100, Kiryl Shutsemau wrote:
> @@ -4127,6 +4127,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>  	return page;
>  }
>  
> +static inline bool order_nofrag_promoted(unsigned int order,
> +					 const struct alloc_context *ac)
> +{
> +	return ac->reclaim_compact_order != order;
> +}

order_defrag_promoted?

Otherwise looks good to me!
Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>

And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
but Meta is using the feature right now.

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Vlastimil Babka (SUSE)]

On 10/14/26 22:29, Johannes Weiner wrote:
> On Tue, Oct 13, 2026 at 06:37:20PM +0100, Kiryl Shutsemau wrote:
>> @@ -4127,6 +4127,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
>>  	return page;
>>  }
>>  
>> +static inline bool order_nofrag_promoted(unsigned int order,
>> +					 const struct alloc_context *ac)
>> +{
>> +	return ac->reclaim_compact_order != order;
>> +}
> 
> order_defrag_promoted?

No strong preference.
> Otherwise looks good to me!
> Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>
> 
> And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
> but Meta is using the feature right now.

I thought it was there for a long time but it was only added in 2025 :) At
least the enterprise world is slow to adopt things, AFAICS.

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
> > And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
> > but Meta is using the feature right now.
> 
> I thought it was there for a long time but it was only added in 2025 :) At
> least the enterprise world is slow to adopt things, AFAICS.

I was thinking to propose a patch after merge window to make defrag_mode
default. Do you think it is too early? I don't think it is ever going
to be truly stabilized without wider exposure.

We also discussed with Johannes to make a stricter (defrag_mode=2?) mode
that will not give up on fragmentation ever, but OOM instead.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Vlastimil Babka (SUSE)]

On 10/15/26 14:55, Kiryl Shutsemau wrote:
> On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
>> > And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
>> > but Meta is using the feature right now.
>> 
>> I thought it was there for a long time but it was only added in 2025 :) At
>> least the enterprise world is slow to adopt things, AFAICS.
> 
> I was thinking to propose a patch after merge window to make defrag_mode
> default. Do you think it is too early? I don't think it is ever going
> to be truly stabilized without wider exposure.

We can try it, but preferably with arguments (and best, numbers).

> We also discussed with Johannes to make a stricter (defrag_mode=2?) mode
> that will not give up on fragmentation ever, but OOM instead.

Would you use that mode at Meta? That however most likely wouldn't be
general purpose enough to become default.

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Thu, Oct 15, 2026 at 03:53:30PM +0200, Vlastimil Babka (SUSE) wrote:
> On 10/15/26 14:55, Kiryl Shutsemau wrote:
> > On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
> >> > And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
> >> > but Meta is using the feature right now.
> >> 
> >> I thought it was there for a long time but it was only added in 2025 :) At
> >> least the enterprise world is slow to adopt things, AFAICS.
> > 
> > I was thinking to propose a patch after merge window to make defrag_mode
> > default. Do you think it is too early? I don't think it is ever going
> > to be truly stabilized without wider exposure.
> 
> We can try it, but preferably with arguments (and best, numbers).

Will do.

> > We also discussed with Johannes to make a stricter (defrag_mode=2?) mode
> > that will not give up on fragmentation ever, but OOM instead.
> 
> Would you use that mode at Meta? That however most likely wouldn't be
> general purpose enough to become default.

Yes, we consider it to be preferable to fragmentation. But it requires
experimentation.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Dev Jain]

On 15/10/26 6:25 pm, Kiryl Shutsemau wrote:
> On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
>>> And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
>>> but Meta is using the feature right now.
>>
>> I thought it was there for a long time but it was only added in 2025 :) At
>> least the enterprise world is slow to adopt things, AFAICS.
> 
> I was thinking to propose a patch after merge window to make defrag_mode
> default. Do you think it is too early? I don't think it is ever going
> to be truly stabilized without wider exposure.

AFAIU defrag mode is not going to improve mTHP allocation success.

I ran a synthetic benchmark faulting in lots of 64k mTHP with swap enabled,
defrag mode caused me to enter a warn_alloc (and 8 times more compact_stalls),
which didn't happen without defrag.

So perhaps bundle the default with pageblock_order eagerness etc.


> 
> We also discussed with Johannes to make a stricter (defrag_mode=2?) mode
> that will not give up on fragmentation ever, but OOM instead.



> 

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Mon, Oct 19, 2026 at 12:28:40PM +0530, Dev Jain wrote:
> 
> 
> On 15/10/26 6:25 pm, Kiryl Shutsemau wrote:
> > On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
> >>> And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
> >>> but Meta is using the feature right now.
> >>
> >> I thought it was there for a long time but it was only added in 2025 :) At
> >> least the enterprise world is slow to adopt things, AFAICS.
> > 
> > I was thinking to propose a patch after merge window to make defrag_mode
> > default. Do you think it is too early? I don't think it is ever going
> > to be truly stabilized without wider exposure.
> 
> AFAIU defrag mode is not going to improve mTHP allocation success.
> 
> I ran a synthetic benchmark faulting in lots of 64k mTHP with swap enabled,
> defrag mode caused me to enter a warn_alloc (and 8 times more compact_stalls),
> which didn't happen without defrag.

Could you share the benchmark? I would like to play with it.

Which THP defrag setting, and was the range MADV_HUGEPAGE?

With the default "madvise" and no MADV_HUGEPAGE, a 64k fault cannot
reclaim and does not wake kswapd.

How is your system setup?

> So perhaps bundle the default with pageblock_order eagerness etc.

What do you mean by pageblock_order eagerness?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Dev Jain]

On 21/10/26 3:03 pm, Kiryl Shutsemau wrote:
> On Mon, Oct 19, 2026 at 12:28:40PM +0530, Dev Jain wrote:
>>
>>
>> On 15/10/26 6:25 pm, Kiryl Shutsemau wrote:
>>> On Thu, Oct 15, 2026 at 09:16:36AM +0200, Vlastimil Babka (SUSE) wrote:
>>>>> And agree 7.4-rc1 with CC stable is fine. Sadly, not sure if anyone
>>>>> but Meta is using the feature right now.
>>>>
>>>> I thought it was there for a long time but it was only added in 2025 :) At
>>>> least the enterprise world is slow to adopt things, AFAICS.
>>>
>>> I was thinking to propose a patch after merge window to make defrag_mode
>>> default. Do you think it is too early? I don't think it is ever going
>>> to be truly stabilized without wider exposure.
>>
>> AFAIU defrag mode is not going to improve mTHP allocation success.
>>
>> I ran a synthetic benchmark faulting in lots of 64k mTHP with swap enabled,
>> defrag mode caused me to enter a warn_alloc (and 8 times more compact_stalls),
>> which didn't happen without defrag.
> 
> Could you share the benchmark? I would like to play with it.

Benchmark is given at the last. LLM had written it, I had verified it on
a surface level ...
> 
> Which THP defrag setting, and was the range MADV_HUGEPAGE?
> 
> With the default "madvise" and no MADV_HUGEPAGE, a 64k fault cannot
> reclaim and does not wake kswapd.
> 
> How is your system setup?

This is a VM running on an arm64 machine, memory is 7G, only 64K mTHP
enabled with madvise, number of cpus is 4 with 1 node.
/sys/kernel/mm/transparent_hugepage/defrag is madvise.
> 
>> So perhaps bundle the default with pageblock_order eagerness etc.
> 
> What do you mean by pageblock_order eagerness?

I meant, something like, if the least mTHP order enabled is the pageblock
order (so in most cases that means there is no mTHP, only PMD size
is enabled) only then make defrag mode the default.

Benchmark: (running it with time ./compact-bench 99 64 16 20 0)

#define _GNU_SOURCE

#include <errno.h>
#include <stdbool.h>
#include <pthread.h>
#include <sched.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>

struct vmstat_snapshot {
        unsigned long long stall;
        unsigned long long success;
        unsigned long long fail;
        unsigned long long swapout_pages;
};

struct mthp_snapshot {
        unsigned long long anon_fault_alloc;
        unsigned long long anon_fault_fallback;
};

struct khugepaged_settings {
        unsigned long long defrag;
        unsigned long long pages_to_scan;
        unsigned long long scan_sleep_millisecs;
};

struct worker {
        unsigned int cpu;
        size_t bytes;
        size_t huge_size;
        size_t page_size;
        void *mapping;
        int error;
};

static pthread_barrier_t start_barrier;
static struct khugepaged_settings saved_khugepaged;
static bool restore_khugepaged_at_exit;

static void usage(const char *program)
{
        fprintf(stderr,
                "Usage: %s [fill-pct [huge-kB [threads [allocate-pct "
                "[hold-secs]]]]]\n"
                "Defaults: fill=90 huge-kB=64 threads=online-CPUs "
                "allocate=30 hold=5\n",
                program);
}

static unsigned long parse_unsigned(const char *text, const char *name,
                                    unsigned long minimum,
                                    unsigned long maximum)
{
        char *end;
        unsigned long value;

        errno = 0;
        value = strtoul(text, &end, 0);
        if (errno || *text == '\0' || *end != '\0' ||
            value < minimum || value > maximum) {
                fprintf(stderr, "invalid %s: %s\n", name, text);
                exit(EXIT_FAILURE);
        }

        return value;
}

static unsigned long long mem_available_bytes(void)
{
        char line[256];
        unsigned long long kib;
        FILE *file = fopen("/proc/meminfo", "r");

        if (!file) {
                perror("/proc/meminfo");
                exit(EXIT_FAILURE);
        }

        while (fgets(line, sizeof(line), file)) {
                if (sscanf(line, "MemAvailable: %llu kB", &kib) == 1) {
                        fclose(file);
                        return kib * 1024;
                }
        }

        fclose(file);
        fputs("MemAvailable is missing from /proc/meminfo\n", stderr);
        exit(EXIT_FAILURE);
}

static void *map_aligned(size_t bytes, size_t alignment)
{
        uintptr_t start, aligned;
        size_t prefix, suffix;
        void *mapping;

        if (bytes > SIZE_MAX - alignment) {
                errno = ENOMEM;
                return MAP_FAILED;
        }

        mapping = mmap(NULL, bytes + alignment, PROT_READ | PROT_WRITE,
                       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
        if (mapping == MAP_FAILED)
                return MAP_FAILED;

        start = (uintptr_t)mapping;
        aligned = (start + alignment - 1) & ~(alignment - 1);
        prefix = aligned - start;
        suffix = bytes + alignment - prefix - bytes;

        if (prefix)
                munmap((void *)start, prefix);
        if (suffix)
                munmap((void *)(aligned + bytes), suffix);

        return (void *)aligned;
}

static struct vmstat_snapshot read_vmstat(void)
{
        struct vmstat_snapshot result = { 0 };
        char name[128];
        unsigned long long value;
        FILE *file = fopen("/proc/vmstat", "r");

        if (!file) {
                perror("/proc/vmstat");
                exit(EXIT_FAILURE);
        }

        while (fscanf(file, "%127s %llu", name, &value) == 2) {
                if (!strcmp(name, "compact_stall"))
                        result.stall = value;
                else if (!strcmp(name, "compact_success"))
                        result.success = value;
                else if (!strcmp(name, "compact_fail"))
                        result.fail = value;
                else if (!strcmp(name, "pswpout"))
                        result.swapout_pages = value;
        }

        fclose(file);
        return result;
}

static unsigned long long read_counter(const char *path)
{
        unsigned long long value;
        FILE *file = fopen(path, "r");

        if (!file) {
                perror(path);
                exit(EXIT_FAILURE);
        }
        if (fscanf(file, "%llu", &value) != 1) {
                fprintf(stderr, "failed to read %s\n", path);
                fclose(file);
                exit(EXIT_FAILURE);
        }

        fclose(file);
        return value;
}

static bool write_counter(const char *path, unsigned long long value)
{
        FILE *file = fopen(path, "w");

        if (!file) {
                perror(path);
                return false;
        }
        if (fprintf(file, "%llu\n", value) < 0) {
                fprintf(stderr, "failed to write %s\n", path);
                fclose(file);
                return false;
        }
        if (fclose(file)) {
                fprintf(stderr, "failed to close %s\n", path);
                return false;
        }

        return true;
}

static void restore_khugepaged(void)
{
        static const char *root =
                "/sys/kernel/mm/transparent_hugepage/khugepaged";
        char path[256];

        if (!restore_khugepaged_at_exit)
                return;

        snprintf(path, sizeof(path), "%s/pages_to_scan", root);
        (void)write_counter(path, saved_khugepaged.pages_to_scan);
        snprintf(path, sizeof(path), "%s/scan_sleep_millisecs", root);
        (void)write_counter(path, saved_khugepaged.scan_sleep_millisecs);
        snprintf(path, sizeof(path), "%s/defrag", root);
        (void)write_counter(path, saved_khugepaged.defrag);
}

static void quiesce_khugepaged(void)
{
        static const char *root =
                "/sys/kernel/mm/transparent_hugepage/khugepaged";
        char defrag_path[256];
        char pages_path[256];
        char sleep_path[256];

        snprintf(defrag_path, sizeof(defrag_path), "%s/defrag", root);
        snprintf(pages_path, sizeof(pages_path), "%s/pages_to_scan", root);
        snprintf(sleep_path, sizeof(sleep_path), "%s/scan_sleep_millisecs", root);

        saved_khugepaged.defrag = read_counter(defrag_path);
        saved_khugepaged.pages_to_scan = read_counter(pages_path);
        saved_khugepaged.scan_sleep_millisecs = read_counter(sleep_path);
        restore_khugepaged_at_exit = true;
        if (atexit(restore_khugepaged)) {
                fputs("failed to register khugepaged restoration\n", stderr);
                exit(EXIT_FAILURE);
        }

        if (!write_counter(defrag_path, 0) ||
            !write_counter(pages_path, 1) ||
            !write_counter(sleep_path, 4294967295ULL))
                exit(EXIT_FAILURE);

        puts("khugepaged: defrag=0 pages_to_scan=1 "
             "scan_sleep_millisecs=4294967295");
}

static struct mthp_snapshot read_mthp_stats(unsigned long huge_kib)
{
        static const char *root = "/sys/kernel/mm/transparent_hugepage";
        struct mthp_snapshot result;
        char path[256];

        snprintf(path, sizeof(path),
                 "%s/hugepages-%lukB/stats/anon_fault_alloc", root, huge_kib);
        result.anon_fault_alloc = read_counter(path);
        snprintf(path, sizeof(path),
                 "%s/hugepages-%lukB/stats/anon_fault_fallback", root, huge_kib);
        result.anon_fault_fallback = read_counter(path);

        return result;
}

static void *fault_worker(void *argument)
{
        struct worker *worker = argument;
        cpu_set_t cpuset;
        volatile unsigned char *memory;
        size_t offset;
        int barrier_result;

        CPU_ZERO(&cpuset);
        CPU_SET(worker->cpu, &cpuset);
        (void)pthread_setaffinity_np(pthread_self(), sizeof(cpuset), &cpuset);

        worker->mapping = map_aligned(worker->bytes, worker->huge_size);
        if (worker->mapping == MAP_FAILED) {
                worker->error = errno;
        } else if (madvise(worker->mapping, worker->bytes,
                           MADV_HUGEPAGE) != 0) {
                worker->error = errno;
        }

        barrier_result = pthread_barrier_wait(&start_barrier);
        if (barrier_result != 0 && barrier_result != PTHREAD_BARRIER_SERIAL_THREAD) {
                worker->error = barrier_result;
                return NULL;
        }

        if (worker->error)
                return NULL;

        memory = worker->mapping;
        for (offset = 0; offset < worker->bytes; offset += worker->page_size)
                memory[offset] = 1;

        return NULL;
}

int main(int argc, char **argv)
{
        unsigned long fill_percent = 90;
        unsigned long huge_kib = 64;
        unsigned long thread_count;
        unsigned long allocate_percent = 30;
        unsigned long hold_seconds = 5;
        unsigned long online_cpus;
        unsigned long long available;
        struct vmstat_snapshot before, after;
        struct mthp_snapshot mthp_before, mthp_after;
        struct worker *workers;
        pthread_t *threads;
        volatile unsigned char *pressure;
        size_t pressure_bytes, target_bytes;
        size_t target_units, units_per_thread, extra_units;
        size_t huge_size, page_size, offset;
        unsigned long i;
        int error = 0;

        online_cpus = (unsigned long)sysconf(_SC_NPROCESSORS_ONLN);
        thread_count = online_cpus;

        if (argc > 6) {
                usage(argv[0]);
                return EXIT_FAILURE;
        }
        if (argc > 1)
                fill_percent = parse_unsigned(argv[1], "fill percentage", 1, 1000);
        if (argc > 2)
                huge_kib = parse_unsigned(argv[2], "huge-page size", 4, 1048576);
        if (argc > 3)
                thread_count = parse_unsigned(argv[3], "thread count", 1, 4096);
        if (argc > 4)
                allocate_percent = parse_unsigned(argv[4], "allocation percentage",
                                            1, 1000);
        if (argc > 5)
                hold_seconds = parse_unsigned(argv[5], "hold time", 0, 86400);

        page_size = (size_t)sysconf(_SC_PAGESIZE);
        huge_size = (size_t)huge_kib * 1024;
        if (huge_size < page_size || huge_size % page_size ||
            (huge_size & (huge_size - 1))) {
                fputs("huge-kB must describe a power-of-two page multiple\n", stderr);
                return EXIT_FAILURE;
        }
        quiesce_khugepaged();

        available = mem_available_bytes();
        pressure_bytes = (size_t)(available / 100 * fill_percent);
        pressure_bytes &= ~(huge_size - 1);
        if (!pressure_bytes) {
                fputs("computed pressure mapping size is zero\n", stderr);
                return EXIT_FAILURE;
        }

        printf("available=%llu MiB fill=%zu MiB huge=%zu KiB threads=%lu "
               "allocate=%lu%%\n",
               available >> 20, pressure_bytes >> 20, huge_size >> 10,
               thread_count, allocate_percent);
        printf("phase 1: faulting anonymous pressure mapping\n");
        fflush(stdout);

        pressure = map_aligned(pressure_bytes, huge_size);
        if (pressure == MAP_FAILED) {
                perror("pressure mmap");
                return EXIT_FAILURE;
        }
        for (offset = 0; offset < pressure_bytes; offset += page_size)
                pressure[offset] = 1;

        target_bytes = (size_t)(available / 100 * allocate_percent);
        target_units = target_bytes / huge_size;
        if (target_units < thread_count)
                thread_count = target_units;
        if (!thread_count) {
                fputs("not enough requested memory for one target allocation\n", stderr);
                return EXIT_FAILURE;
        }
        target_bytes = target_units * huge_size;

        workers = calloc(thread_count, sizeof(*workers));
        threads = calloc(thread_count, sizeof(*threads));
        if (!workers || !threads) {
                perror("calloc");
                return EXIT_FAILURE;
        }
        if (pthread_barrier_init(&start_barrier, NULL, thread_count + 1)) {
                perror("pthread_barrier_init");
                return EXIT_FAILURE;
        }

        units_per_thread = target_units / thread_count;
        extra_units = target_units % thread_count;
        for (i = 0; i < thread_count; i++) {
                workers[i].cpu = i % online_cpus;
                workers[i].bytes = (units_per_thread + (i < extra_units)) * huge_size;
                workers[i].huge_size = huge_size;
                workers[i].page_size = page_size;
                if (pthread_create(&threads[i], NULL, fault_worker, &workers[i])) {
                        fprintf(stderr, "pthread_create failed at thread %lu\n", i);
                        return EXIT_FAILURE;
                }
        }

        printf("phase 2: faulting %zu MiB through aligned MADV_HUGEPAGE mappings\n",
               target_bytes >> 20);
        fflush(stdout);
        before = read_vmstat();
        mthp_before = read_mthp_stats(huge_kib);
        pthread_barrier_wait(&start_barrier);
        for (i = 0; i < thread_count; i++) {
                pthread_join(threads[i], NULL);
                if (workers[i].error) {
                        fprintf(stderr, "worker %lu: %s\n", i,
                                strerror(workers[i].error));
                        error = 1;
                }
        }
        after = read_vmstat();
        mthp_after = read_mthp_stats(huge_kib);

        printf("RESULT compact_stall=%llu compact_success=%llu compact_fail=%llu "
               "swapout_pages=%llu "
               "anon_fault_alloc=%llu anon_fault_fallback=%llu\n",
               after.stall - before.stall,
               after.success - before.success,
               after.fail - before.fail,
               after.swapout_pages - before.swapout_pages,
               mthp_after.anon_fault_alloc - mthp_before.anon_fault_alloc,
               mthp_after.anon_fault_fallback -
               mthp_before.anon_fault_fallback);
        printf("holding mappings for %lu seconds\n", hold_seconds);
        fflush(stdout);
        sleep(hold_seconds);

        for (i = 0; i < thread_count; i++) {
                if (workers[i].mapping && workers[i].mapping != MAP_FAILED)
                        munmap(workers[i].mapping, workers[i].bytes);
        }
        munmap((void *)pressure, pressure_bytes);
        pthread_barrier_destroy(&start_barrier);
        free(threads);
        free(workers);

        return error ? EXIT_FAILURE : EXIT_SUCCESS;
}

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Fri, Oct 23, 2026 at 12:09:07PM +0530, Dev Jain wrote:
> >> I ran a synthetic benchmark faulting in lots of 64k mTHP with swap enabled,
> >> defrag mode caused me to enter a warn_alloc (and 8 times more compact_stalls),
> >> which didn't happen without defrag.

Thanks for the benchmark. It reproduces on x86 in the same shape.

I think I found the problem.

Under pressure, the free order-4+ pages left are mostly in partly used
unmovable and reclaimable pageblocks. A movable request with
ALLOC_NOFRAGMENT cannot take them. But compaction_suit_allocation_order()
asks __zone_watermark_ok(), which counts free pages of every
migratetype, and returns COMPACT_SUCCESS without compacting anything.
The allocation then fails, and since compaction "succeeded", it is never
deferred.

With the diff below, compaction behaves the same with and without
defrag_mode. Medians of 15 runs of "compact-bench 99 64 16 20 0":

                             defrag_mode=0  defrag_mode=1  defrag_mode=1
                                                           + diff
    compact_stall                    2135          32476          2290
    compact_fail                      668          32330           606
    64K anon_fault_alloc            20689           6780         11804
    64K anon_fault_fallback           776          14502          9687

64K allocation is still lower with defrag_mode. I will look into this
further.

Could you try the diff on your setup?

diff --git a/mm/compaction.c b/mm/compaction.c
index a049415512c6..09be677fff36 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2365,6 +2365,10 @@ static enum compact_result __compact_finished(struct compact_control *cc)
 			!free_area_empty(area, MIGRATE_CMA))
 			return COMPACT_SUCCESS;
 #endif
+		/* Under ALLOC_NOFRAGMENT it may only claim a whole block */
+		if ((cc->alloc_flags & ALLOC_NOFRAGMENT) && order < pageblock_order)
+			continue;
+
 		/*
 		 * Job done if allocation would steal freepages from
 		 * other migratetype buddy lists.
@@ -2510,6 +2514,43 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 	return false;
 }
 
+/*
+ * The watermark check counts free pages of every migratetype.  Under
+ * ALLOC_NOFRAGMENT, a request below pageblock_order cannot take a page from a
+ * block of another migratetype unless it claims the whole block, so look for
+ * a free page the request can actually use.
+ */
+static bool compaction_free_page_usable(struct zone *zone, unsigned int order,
+					int migratetype, unsigned int alloc_flags)
+{
+	unsigned int o;
+	int mt;
+
+	if (!(alloc_flags & ALLOC_NOFRAGMENT) || order >= pageblock_order)
+		return true;
+
+	for (o = order; o < NR_PAGE_ORDERS; o++) {
+		struct free_area *area = &zone->free_area[o];
+
+		if (!free_area_empty(area, migratetype))
+			return true;
+#ifdef CONFIG_CMA
+		if ((alloc_flags & ALLOC_CMA) &&
+		    !free_area_empty(area, MIGRATE_CMA))
+			return true;
+#endif
+		/* From pageblock_order up, a free block can be claimed */
+		if (o < pageblock_order)
+			continue;
+		for (mt = 0; mt < MIGRATE_PCPTYPES; mt++) {
+			if (!free_area_empty(area, mt))
+				return true;
+		}
+	}
+
+	return false;
+}
+
 /*
  * Should we do compaction for target allocation order.
  * Return COMPACT_SUCCESS if allocation for target order can be already
@@ -2520,7 +2561,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 static enum compact_result
 compaction_suit_allocation_order(struct zone *zone, unsigned int order,
 				 int highest_zoneidx, unsigned int alloc_flags,
-				 bool async, bool kcompactd)
+				 int migratetype, bool async, bool kcompactd)
 {
 	unsigned long free_pages;
 	unsigned long watermark;
@@ -2532,7 +2573,8 @@ compaction_suit_allocation_order(struct zone *zone, unsigned int order,
 
 	watermark = wmark_pages(zone, alloc_flags & ALLOC_WMARK_MASK);
 	if (__zone_watermark_ok(zone, order, watermark, highest_zoneidx,
-				alloc_flags, free_pages))
+				alloc_flags, free_pages) &&
+	    compaction_free_page_usable(zone, order, migratetype, alloc_flags))
 		return COMPACT_SUCCESS;
 
 	/*
@@ -2588,6 +2630,7 @@ compact_zone(struct compact_control *cc, struct capture_control *capc)
 		ret = compaction_suit_allocation_order(cc->zone, cc->order,
 						       cc->highest_zoneidx,
 						       cc->alloc_flags,
+						       cc->migratetype,
 						       cc->mode == MIGRATE_ASYNC,
 						       !cc->direct_compaction);
 		if (ret != COMPACT_CONTINUE)
@@ -3073,7 +3116,7 @@ static bool kcompactd_node_suitable(pg_data_t *pgdat)
 		ret = compaction_suit_allocation_order(zone,
 				pgdat->kcompactd_max_order,
 				highest_zoneidx, alloc_flags,
-				false, true);
+				MIGRATE_MOVABLE, false, true);
 		if (ret == COMPACT_CONTINUE)
 			return true;
 	}
@@ -3116,7 +3159,7 @@ static void kcompactd_do_work(pg_data_t *pgdat)
 
 		ret = compaction_suit_allocation_order(zone,
 				cc.order, zoneid, cc.alloc_flags,
-				false, true);
+				MIGRATE_MOVABLE, false, true);
 		if (ret != COMPACT_CONTINUE)
 			continue;
 
-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Fri, Oct 23, 2026 at 11:59:06AM +0100, Kiryl Shutsemau wrote:
> 64K allocation is still lower with defrag_mode. I will look into this
> further.

Direct reclaim has the same problem.

compaction_ready() and should_continue_reclaim() ask the same watermark
check whether the allocation can already succeed. The free order-4+
pages left in unmovable pageblocks make it say yes, and reclaim stops
early.

I tried to generalize the fix: __zone_watermark_ok() takes the request's
migratetype and, under ALLOC_NOFRAGMENT, does not count other
migratetypes' free pages below pageblock_order. Direct reclaim learns
ALLOC_NOFRAGMENT from the allocation.

Medians of 15 runs of "compact-bench 99 64 16 20 0":

                             defrag_mode=0  defrag_mode=1  defrag_mode=1
                                                           + diff
    compact_stall                    2135          32476          8022
    compact_fail                      668          32330          4880
    64K anon_fault_alloc            20689           6780         19649
    64K anon_fault_fallback           776          14502          1851

The diff below replaces the previous one. It needs cleanup and splitting
before it goes upstream.

Could you give it a try?

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 94f9c3ff5416..f924940b5475 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1629,7 +1629,7 @@ static inline unsigned long pgdat_end_pfn(pg_data_t *pgdat)
 void build_all_zonelists(pg_data_t *pgdat);
 bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 			 int highest_zoneidx, unsigned int alloc_flags,
-			 long free_pages);
+			 long free_pages, int migratetype);
 bool zone_watermark_ok(struct zone *z, unsigned int order,
 		unsigned long mark, int highest_zoneidx,
 		unsigned int alloc_flags);
diff --git a/mm/compaction.c b/mm/compaction.c
index a049415512c6..420ff6c735eb 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2344,7 +2344,8 @@ static enum compact_result __compact_finished(struct compact_control *cc)
 					high_wmark_pages(cc->zone),
 					cc->highest_zoneidx, cc->alloc_flags,
 					zone_page_state(cc->zone,
-							NR_FREE_PAGES_BLOCKS)))
+							NR_FREE_PAGES_BLOCKS),
+					cc->migratetype))
 			return COMPACT_SUCCESS;
 
 		return COMPACT_CONTINUE;
@@ -2365,6 +2366,11 @@ static enum compact_result __compact_finished(struct compact_control *cc)
 			!free_area_empty(area, MIGRATE_CMA))
 			return COMPACT_SUCCESS;
 #endif
+		/* Under ALLOC_NOFRAGMENT it may only claim a whole block */
+		if ((cc->alloc_flags & ALLOC_NOFRAGMENT) &&
+		    order < pageblock_order)
+			continue;
+
 		/*
 		 * Job done if allocation would steal freepages from
 		 * other migratetype buddy lists.
@@ -2425,7 +2431,7 @@ static bool __compaction_suitable(struct zone *zone, int order,
 	if (order > PAGE_ALLOC_COSTLY_ORDER)
 		watermark += low_wmark_pages(zone) - min_wmark_pages(zone);
 	return __zone_watermark_ok(zone, 0, watermark, highest_zoneidx,
-				   ALLOC_CMA, free_pages);
+				   ALLOC_CMA, free_pages, MIGRATE_MOVABLE);
 }
 
 /*
@@ -2520,7 +2526,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order,
 static enum compact_result
 compaction_suit_allocation_order(struct zone *zone, unsigned int order,
 				 int highest_zoneidx, unsigned int alloc_flags,
-				 bool async, bool kcompactd)
+				 int migratetype, bool async, bool kcompactd)
 {
 	unsigned long free_pages;
 	unsigned long watermark;
@@ -2532,7 +2538,7 @@ compaction_suit_allocation_order(struct zone *zone, unsigned int order,
 
 	watermark = wmark_pages(zone, alloc_flags & ALLOC_WMARK_MASK);
 	if (__zone_watermark_ok(zone, order, watermark, highest_zoneidx,
-				alloc_flags, free_pages))
+				alloc_flags, free_pages, migratetype))
 		return COMPACT_SUCCESS;
 
 	/*
@@ -2548,7 +2554,8 @@ compaction_suit_allocation_order(struct zone *zone, unsigned int order,
 	    !(alloc_flags & ALLOC_CMA)) {
 		if (!__zone_watermark_ok(zone, 0, watermark + compact_gap(order),
 					 highest_zoneidx, 0,
-					 zone_page_state(zone, NR_FREE_PAGES)))
+					 zone_page_state(zone, NR_FREE_PAGES),
+					 migratetype))
 			return COMPACT_SKIPPED;
 	}
 
@@ -2588,6 +2595,7 @@ compact_zone(struct compact_control *cc, struct capture_control *capc)
 		ret = compaction_suit_allocation_order(cc->zone, cc->order,
 						       cc->highest_zoneidx,
 						       cc->alloc_flags,
+						       cc->migratetype,
 						       cc->mode == MIGRATE_ASYNC,
 						       !cc->direct_compaction);
 		if (ret != COMPACT_CONTINUE)
@@ -3073,7 +3081,7 @@ static bool kcompactd_node_suitable(pg_data_t *pgdat)
 		ret = compaction_suit_allocation_order(zone,
 				pgdat->kcompactd_max_order,
 				highest_zoneidx, alloc_flags,
-				false, true);
+				MIGRATE_MOVABLE, false, true);
 		if (ret == COMPACT_CONTINUE)
 			return true;
 	}
@@ -3116,7 +3124,7 @@ static void kcompactd_do_work(pg_data_t *pgdat)
 
 		ret = compaction_suit_allocation_order(zone,
 				cc.order, zoneid, cc.alloc_flags,
-				false, true);
+				MIGRATE_MOVABLE, false, true);
 		if (ret != COMPACT_CONTINUE)
 			continue;
 
diff --git a/mm/internal.h b/mm/internal.h
index 38b1165212c9..5a7556999ad2 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -68,7 +68,8 @@ void folio_mark_lazyfree(struct folio *folio);
 /* mm/vmscan.c */
 unsigned long zone_reclaimable_pages(struct zone *zone);
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
-				gfp_t gfp_mask, const nodemask_t *mask);
+				gfp_t gfp_mask, const nodemask_t *mask,
+				bool nofragment);
 unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru,
 			      int zone_idx);
 
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..83a52acf9c9c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3597,7 +3597,7 @@ static inline long __zone_watermark_unusable_free(struct zone *z,
  */
 bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 			 int highest_zoneidx, unsigned int alloc_flags,
-			 long free_pages)
+			 long free_pages, int migratetype)
 {
 	long min = mark;
 	int o;
@@ -3655,6 +3655,14 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 			continue;
 
 		for (mt = 0; mt < MIGRATE_PCPTYPES; mt++) {
+			/*
+			 * Under ALLOC_NOFRAGMENT, a request below
+			 * pageblock_order can take another migratetype's page
+			 * only by claiming the whole block.
+			 */
+			if ((alloc_flags & ALLOC_NOFRAGMENT) &&
+			    mt != migratetype && o < pageblock_order)
+				continue;
 			if (!free_area_empty(area, mt))
 				return true;
 		}
@@ -3676,8 +3684,11 @@ bool __zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 bool zone_watermark_ok(struct zone *z, unsigned int order, unsigned long mark,
 		      int highest_zoneidx, unsigned int alloc_flags)
 {
-	return __zone_watermark_ok(z, order, mark, highest_zoneidx, alloc_flags,
-					zone_page_state(z, NR_FREE_PAGES));
+	/* Without the request's migratetype, count every migratetype's pages */
+	return __zone_watermark_ok(z, order, mark, highest_zoneidx,
+				   alloc_flags & ~ALLOC_NOFRAGMENT,
+				   zone_page_state(z, NR_FREE_PAGES),
+				   MIGRATE_MOVABLE);
 }
 
 static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
@@ -3706,7 +3717,7 @@ static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
 	}
 
 	if (__zone_watermark_ok(z, order, mark, highest_zoneidx, alloc_flags,
-					free_pages))
+				free_pages, gfp_migratetype(gfp_mask)))
 		return true;
 
 	/*
@@ -3719,7 +3730,8 @@ static inline bool zone_watermark_fast(struct zone *z, unsigned int order,
 		&& ((alloc_flags & ALLOC_WMARK_MASK) == WMARK_MIN))) {
 		mark = z->_watermark[WMARK_MIN];
 		return __zone_watermark_ok(z, order, mark, highest_zoneidx,
-					alloc_flags, free_pages);
+					alloc_flags, free_pages,
+					gfp_migratetype(gfp_mask));
 	}
 
 	return false;
@@ -4435,7 +4447,7 @@ static unsigned int check_retry_zonelist(unsigned int seq)
 
 /* Perform direct synchronous page reclaim */
 static unsigned long
-__perform_reclaim(gfp_t gfp_mask, unsigned int order,
+__perform_reclaim(gfp_t gfp_mask, unsigned int order, unsigned int alloc_flags,
 					const struct alloc_context *ac)
 {
 	unsigned int noreclaim_flag;
@@ -4449,7 +4461,8 @@ __perform_reclaim(gfp_t gfp_mask, unsigned int order,
 	noreclaim_flag = memalloc_noreclaim_save();
 
 	progress = try_to_free_pages(ac->zonelist, order, gfp_mask,
-								ac->nodemask);
+				     ac->nodemask,
+				     alloc_flags & ALLOC_NOFRAGMENT);
 
 	memalloc_noreclaim_restore(noreclaim_flag);
 	fs_reclaim_release(gfp_mask);
@@ -4475,7 +4488,8 @@ __alloc_pages_direct_reclaim(gfp_t gfp_mask, unsigned int order,
 		reclaim_order = max(order, pageblock_order);
 
 	psi_memstall_enter(&pflags);
-	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order, ac);
+	*did_some_progress = __perform_reclaim(gfp_mask, reclaim_order,
+					       alloc_flags, ac);
 	if (unlikely(!(*did_some_progress)))
 		goto out;
 
@@ -4682,10 +4696,12 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 
 		/*
 		 * Would the allocation succeed if we reclaimed all
-		 * reclaimable pages?
+		 * reclaimable pages?  Count free pages of every migratetype.
 		 */
 		wmark = __zone_watermark_ok(zone, order, min_wmark,
-				ac->highest_zoneidx, alloc_flags, available);
+				ac->highest_zoneidx,
+				alloc_flags & ~ALLOC_NOFRAGMENT,
+				available, ac->migratetype);
 		trace_reclaim_retry_zone(z, order, reclaimable,
 				available, min_wmark, *no_progress_loops, wmark);
 		if (wmark) {
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f11491ee9ed5..f7c561c40e55 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -142,6 +142,9 @@ struct scan_control {
 	/* One of the zones is ready for compaction */
 	unsigned int compaction_ready:1;
 
+	/* The allocation cannot fall back to other migratetypes' blocks */
+	unsigned int nofragment:1;
+
 	/* There is easily reclaimable cold cache in the current node */
 	unsigned int cache_trim_mode:1;
 
@@ -6098,6 +6101,17 @@ static bool in_reclaim_compaction(struct scan_control *sc)
 	return false;
 }
 
+static bool allocation_can_succeed(struct zone *zone, struct scan_control *sc,
+				   unsigned long mark)
+{
+	unsigned int alloc_flags = sc->nofragment ? ALLOC_NOFRAGMENT : 0;
+	long free_pages = zone_page_state(zone, NR_FREE_PAGES);
+	int migratetype = gfp_migratetype(sc->gfp_mask);
+
+	return __zone_watermark_ok(zone, sc->order, mark, sc->reclaim_idx,
+				   alloc_flags, free_pages, migratetype);
+}
+
 /*
  * Reclaim/compaction is used for high-order allocation requests. It reclaims
  * order-0 pages before compacting the zone. should_continue_reclaim() returns
@@ -6136,8 +6150,7 @@ static inline bool should_continue_reclaim(struct pglist_data *pgdat,
 		unsigned long watermark = min_wmark_pages(zone);
 
 		/* Allocation can already succeed, nothing to do */
-		if (zone_watermark_ok(zone, sc->order, watermark,
-				      sc->reclaim_idx, 0))
+		if (allocation_can_succeed(zone, sc, watermark))
 			return false;
 
 		if (compaction_suitable(zone, sc->order, watermark,
@@ -6362,8 +6375,7 @@ static inline bool compaction_ready(struct zone *zone, struct scan_control *sc)
 		return false;
 
 	/* Allocation can already succeed, nothing to do */
-	if (zone_watermark_ok(zone, sc->order, min_wmark_pages(zone),
-			      sc->reclaim_idx, 0))
+	if (allocation_can_succeed(zone, sc, min_wmark_pages(zone)))
 		return true;
 
 	/*
@@ -6767,7 +6779,8 @@ static bool throttle_direct_reclaim(gfp_t gfp_mask, struct zonelist *zonelist,
 }
 
 unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
-				gfp_t gfp_mask, const nodemask_t *nodemask)
+				gfp_t gfp_mask, const nodemask_t *nodemask,
+				bool nofragment)
 {
 	unsigned long nr_reclaimed;
 	struct scan_control sc = {
@@ -6780,6 +6793,7 @@ unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
 		.may_writepage = 1,
 		.may_unmap = 1,
 		.may_swap = 1,
+		.nofragment = nofragment,
 	};
 
 	/*
@@ -7011,7 +7025,7 @@ static bool pgdat_balanced(pg_data_t *pgdat, int order, int highest_zoneidx)
 			free_pages = zone_page_state_snapshot(zone, item);
 
 		if (__zone_watermark_ok(zone, order, mark, highest_zoneidx,
-					0, free_pages))
+					0, free_pages, MIGRATE_MOVABLE))
 			return true;
 	}
 
-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Fri, Oct 23, 2026 at 01:12:02PM +0100, Kiryl Shutsemau wrote:
> The diff below replaces the previous one. It needs cleanup and splitting
> before it goes upstream.

Proper patchset:

https://git.kernel.org/pub/scm/linux/kernel/git/kas/linux.git page_alloc/nofrag-watermark-v1

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v3] mm: page_alloc: make defrag_mode retries follow the promoted order

[Kiryl Shutsemau]

On Wed, Oct 14, 2026 at 10:29:36PM +0200, Johannes Weiner wrote:
> On Tue, Oct 13, 2026 at 06:37:20PM +0100, Kiryl Shutsemau wrote:
> > @@ -4127,6 +4127,12 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
> >  	return page;
> >  }
> >  
> > +static inline bool order_nofrag_promoted(unsigned int order,
> > +					 const struct alloc_context *ac)
> > +{
> > +	return ac->reclaim_compact_order != order;
> > +}
> 
> order_defrag_promoted?

Works for me.

Andrew, could you fold the fixup below?

---

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index c593913a14f8..440e515be50b 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4127,7 +4127,7 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order,
 	return page;
 }
 
-static inline bool order_nofrag_promoted(unsigned int order,
+static inline bool order_defrag_promoted(unsigned int order,
 					 const struct alloc_context *ac)
 {
 	return ac->reclaim_compact_order != order;
@@ -4260,7 +4260,7 @@ should_compact_retry(gfp_t gfp_mask, struct alloc_context *ac, int order,
 	 * Promoted requests fall back instead.
 	 */
 	if (compact_result == COMPACT_SKIPPED) {
-		if (!order_nofrag_promoted(order, ac))
+		if (!order_defrag_promoted(order, ac))
 			ret = compaction_zonelist_suitable(ac, order,
 							   alloc_flags, gfp_mask);
 		goto out;
@@ -4642,7 +4642,7 @@ should_reclaim_retry(gfp_t gfp_mask, unsigned order,
 	 * for compaction without freeing anything.
 	 */
 	if (did_some_progress && order <= PAGE_ALLOC_COSTLY_ORDER &&
-	    !order_nofrag_promoted(order, ac))
+	    !order_defrag_promoted(order, ac))
 		*no_progress_loops = 0;
 	else
 		(*no_progress_loops)++;
@@ -5024,7 +5024,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
 	if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT)) {
 		alloc_flags &= ~ALLOC_NOFRAGMENT;
 		/* Retries went into making blocks, start the fallback afresh */
-		if (order_nofrag_promoted(order, ac)) {
+		if (order_defrag_promoted(order, ac)) {
 			no_progress_loops = 0;
 			compaction_retries = 0;
 		}
-- 
  Kiryl Shutsemau / Kirill A. Shutemov