[PATCH v2 0/2] mm: refactor zonelist constructors and iterators

[PATCH v2 0/2] mm: refactor zonelist constructors and iterators

[Gregory Price]

find_next_best_node() picks the next-closest node when building a
fallback list, and hardcodes N_MEMORY as the set it picks from.

Refactor it into find_next_best_node_in(), which takes the candidate
set explicitly.

This makes the existing behaviour explicit at both mm/memory-tiers.c
call sites - they select demotion targets in fallback order from
N_MEMORY - and lets callers narrow that set.

Then extract the per-node construction loop out of build_zonelists()
into build_node_zonelist(),i parameterised on the candidate nodemask
and destination zonelist index.

Together these allow a zonelist to be built over a candidate set other
than N_MEMORY, into a zonelist other than FALLBACK, and iterated in
fallback order over a caller-defined subset.

These are prerequisites for generating a private node zonelist (nodes
unreachable by default), but are otherwise general improvements to
the existing interfaces so I'm proposing them separately.

No functional change intended - purely refactor commits.

Verified on x86_64:

  - find_next_best_node_in() compiles to the same 319 bytes as
    find_next_best_node().
  - Both mm/memory-tiers.c callers grow a single instruction - the
    added argument.
  - Boot-time fallback orders are identical on a 4-node guest with an
    asymmetric distance matrix.

Gregory Price (2):
  mm: refactor find_next_best_node to find_next_best_node_in
  mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()

 mm/internal.h     |  6 ++--
 mm/memory-tiers.c |  7 +++--
 mm/page_alloc.c   | 74 +++++++++++++++++++----------------------------
 3 files changed, 38 insertions(+), 49 deletions(-)

---

v2: 
- drop extern (Vlastimil)
- move printing into build_node_zonelist() (Vlastimil)
- get rid of node_order array (Vlastimil)
- drop update_load bool (will add back when relevant)

-- 
2.55.0

[PATCH v2 1/2] mm: refactor find_next_best_node to find_next_best_node_in

[Gregory Price]

find_next_best_node() picks the next-closest node for a fallback list
from the full N_MEMORY set. Refactor it into find_next_best_node_in(),
which takes an explicit candidates nodemask.

This enables building fallback lists with non-N_MEMORY candidates.

No functional change: every caller still selects from N_MEMORY.

Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: Balbir Singh <balbirs@nvidia.com>
Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/internal.h     |  6 ++++--
 mm/memory-tiers.c |  7 ++++---
 mm/page_alloc.c   | 13 ++++++++-----
 3 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index da14c56fb24e..1519dd21a900 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1133,7 +1133,8 @@ extern int node_reclaim_mode;
 
 extern unsigned long node_reclaim(struct pglist_data *pgdat,
 				  gfp_t gfp_mask, unsigned int order);
-extern int find_next_best_node(int node, nodemask_t *used_node_mask);
+int find_next_best_node_in(int node, nodemask_t *used_node_mask,
+		const nodemask_t *candidates);
 #else
 #define node_reclaim_mode 0
 
@@ -1142,7 +1143,8 @@ static inline unsigned long node_reclaim(struct pglist_data *pgdat,
 {
 	return 0;
 }
-static inline int find_next_best_node(int node, nodemask_t *used_node_mask)
+static inline int find_next_best_node_in(int node, nodemask_t *used_node_mask,
+		const nodemask_t *candidates)
 {
 	return NUMA_NO_NODE;
 }
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 54851d8a195b..25e121851b58 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -370,7 +370,7 @@ int next_demotion_node(int node, const nodemask_t *allowed_mask)
 	 * closest demotion target.
 	 */
 	nodes_complement(mask, *allowed_mask);
-	return find_next_best_node(node, &mask);
+	return find_next_best_node_in(node, &mask, &node_states[N_MEMORY]);
 }
 
 static void disable_all_demotion_targets(void)
@@ -450,7 +450,7 @@ static void establish_demotion_targets(void)
 		memtier = list_next_entry(memtier, list);
 		tier_nodes = get_memtier_nodemask(memtier);
 		/*
-		 * find_next_best_node, use 'used' nodemask as a skip list.
+		 * find_next_best_node_in, use 'used' nodemask as a skip list.
 		 * Add all memory nodes except the selected memory tier
 		 * nodelist to skip list so that we find the best node from the
 		 * memtier nodelist.
@@ -463,7 +463,8 @@ static void establish_demotion_targets(void)
 		 * in the preferred mask when allocating pages during demotion.
 		 */
 		do {
-			target = find_next_best_node(node, &tier_nodes);
+			target = find_next_best_node_in(node, &tier_nodes,
+							&node_states[N_MEMORY]);
 			if (target == NUMA_NO_NODE)
 				break;
 
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1cfbd0582a10..7efce139d562 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5793,9 +5793,10 @@ static int numa_zonelist_order_handler(const struct ctl_table *table, int write,
 static int node_load[MAX_NUMNODES];
 
 /**
- * find_next_best_node - find the next node that should appear in a given node's fallback list
+ * find_next_best_node_in - find the next node that should appear in a given node's fallback list
  * @node: node whose fallback list we're appending
  * @used_node_mask: nodemask_t of already used nodes
+ * @candidates: nodemask_t of nodes eligible for selection
  *
  * We use a number of factors to determine which is the next node that should
  * appear on a given node's fallback list.  The node should not have appeared
@@ -5807,7 +5808,8 @@ static int node_load[MAX_NUMNODES];
  *
  * Return: node id of the found node or %NUMA_NO_NODE if no node is found.
  */
-int find_next_best_node(int node, nodemask_t *used_node_mask)
+int find_next_best_node_in(int node, nodemask_t *used_node_mask,
+			   const nodemask_t *candidates)
 {
 	int n, val;
 	int min_val = INT_MAX;
@@ -5817,12 +5819,12 @@ int find_next_best_node(int node, nodemask_t *used_node_mask)
 	 * Use the local node if we haven't already, but for memoryless local
 	 * node, we should skip it and fall back to other nodes.
 	 */
-	if (!node_isset(node, *used_node_mask) && node_state(node, N_MEMORY)) {
+	if (!node_isset(node, *used_node_mask) && node_isset(node, *candidates)) {
 		node_set(node, *used_node_mask);
 		return node;
 	}
 
-	for_each_node_state(n, N_MEMORY) {
+	for_each_node_mask(n, *candidates) {
 
 		/* Don't want a node to appear more than once */
 		if (node_isset(n, *used_node_mask))
@@ -5907,7 +5909,8 @@ static void build_zonelists(pg_data_t *pgdat)
 	prev_node = local_node;
 
 	memset(node_order, 0, sizeof(node_order));
-	while ((node = find_next_best_node(local_node, &used_mask)) >= 0) {
+	while ((node = find_next_best_node_in(local_node, &used_mask,
+					      &node_states[N_MEMORY])) >= 0) {
 		/*
 		 * We don't want to pressure a particular node.
 		 * So adding penalty to the first node in same
-- 
2.55.0

Re: [PATCH v2 1/2] mm: refactor find_next_best_node to find_next_best_node_in

[Zi Yan]

On 11 Sep 2026, at 23:04, Gregory Price wrote:

> find_next_best_node() picks the next-closest node for a fallback list
> from the full N_MEMORY set. Refactor it into find_next_best_node_in(),
> which takes an explicit candidates nodemask.
>
> This enables building fallback lists with non-N_MEMORY candidates.
>
> No functional change: every caller still selects from N_MEMORY.
>
> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> Acked-by: Balbir Singh <balbirs@nvidia.com>
> Signed-off-by: Gregory Price <gourry@gourry.net>
> ---
>  mm/internal.h     |  6 ++++--
>  mm/memory-tiers.c |  7 ++++---
>  mm/page_alloc.c   | 13 ++++++++-----
>  3 files changed, 16 insertions(+), 10 deletions(-)
>
LGTM.

Reviewed-by: Zi Yan <ziy@nvidia.com>

Best Regards,
Yan, Zi

[PATCH v2 2/2] mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()

[Gregory Price]

Extract per-node fallback-list construction into build_node_zonelist().
Build each selected node directly into the destination zonelist so no
intermediate node_order array or node count is needed. Print the fallback
order as each node is added.

This lets us build new zonelists from candidate nodemasks instead of just
the default N_MEMORY node state list.

No functional change: build_zonelists() builds and prints the same FALLBACK
list over N_MEMORY with node_load updates as before.

Signed-off-by: Gregory Price <gourry@gourry.net>
---
 mm/page_alloc.c | 63 ++++++++++++++++++-------------------------------
 1 file changed, 23 insertions(+), 40 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7efce139d562..d1888d5630e0 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5857,31 +5857,6 @@ int find_next_best_node_in(int node, nodemask_t *used_node_mask,
 }
 
 
-/*
- * Build zonelists ordered by node and zones within node.
- * This results in maximum locality--normal zone overflows into local
- * DMA zone, if any--but risks exhausting DMA zone.
- */
-static void build_zonelists_in_node_order(pg_data_t *pgdat, int *node_order,
-		unsigned nr_nodes)
-{
-	struct zoneref *zonerefs;
-	int i;
-
-	zonerefs = pgdat->node_zonelists[ZONELIST_FALLBACK]._zonerefs;
-
-	for (i = 0; i < nr_nodes; i++) {
-		int nr_zones;
-
-		pg_data_t *node = NODE_DATA(node_order[i]);
-
-		nr_zones = build_zonerefs_node(node, zonerefs);
-		zonerefs += nr_zones;
-	}
-	zonerefs->zone = NULL;
-	zonerefs->zone_idx = 0;
-}
-
 /*
  * Build __GFP_THISNODE zonelists
  */
@@ -5897,20 +5872,24 @@ static void build_thisnode_zonelists(pg_data_t *pgdat)
 	zonerefs->zone_idx = 0;
 }
 
-static void build_zonelists(pg_data_t *pgdat)
+/*
+ * Build one zonelist ordered by node and zones within node. This results in
+ * maximum locality--normal zone overflows into local DMA zone, if any--but
+ * risks exhausting DMA zone.
+ */
+static void build_node_zonelist(pg_data_t *pgdat, const nodemask_t *candidates,
+				int zlidx)
 {
-	static int node_order[MAX_NUMNODES];
-	int node, nr_nodes = 0;
+	struct zoneref *zonerefs = pgdat->node_zonelists[zlidx]._zonerefs;
 	nodemask_t used_mask = NODE_MASK_NONE;
-	int local_node, prev_node;
+	int local_node = pgdat->node_id;
+	int prev_node = local_node;
+	int node;
 
-	/* NUMA-aware ordering of nodes */
-	local_node = pgdat->node_id;
-	prev_node = local_node;
+	pr_info("Fallback order for Node %d: ", local_node);
 
-	memset(node_order, 0, sizeof(node_order));
 	while ((node = find_next_best_node_in(local_node, &used_mask,
-					      &node_states[N_MEMORY])) >= 0) {
+					      candidates)) >= 0) {
 		/*
 		 * We don't want to pressure a particular node.
 		 * So adding penalty to the first node in same
@@ -5920,18 +5899,22 @@ static void build_zonelists(pg_data_t *pgdat)
 		    node_distance(local_node, prev_node))
 			node_load[node] += 1;
 
-		node_order[nr_nodes++] = node;
+		zonerefs += build_zonerefs_node(NODE_DATA(node), zonerefs);
+		pr_cont("%d ", node);
 		prev_node = node;
 	}
 
-	build_zonelists_in_node_order(pgdat, node_order, nr_nodes);
-	build_thisnode_zonelists(pgdat);
-	pr_info("Fallback order for Node %d: ", local_node);
-	for (node = 0; node < nr_nodes; node++)
-		pr_cont("%d ", node_order[node]);
+	zonerefs->zone = NULL;
+	zonerefs->zone_idx = 0;
 	pr_cont("\n");
 }
 
+static void build_zonelists(pg_data_t *pgdat)
+{
+	build_node_zonelist(pgdat, &node_states[N_MEMORY], ZONELIST_FALLBACK);
+	build_thisnode_zonelists(pgdat);
+}
+
 #ifdef CONFIG_HAVE_MEMORYLESS_NODES
 /*
  * Return node id of node used for "local" allocations.
-- 
2.55.0

Re: [PATCH v2 2/2] mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()

[Vlastimil Babka (SUSE)]

On 9/12/26 05:04, Gregory Price wrote:
> Extract per-node fallback-list construction into build_node_zonelist().
> Build each selected node directly into the destination zonelist so no
> intermediate node_order array or node count is needed. Print the fallback
> order as each node is added.
> 
> This lets us build new zonelists from candidate nodemasks instead of just
> the default N_MEMORY node state list.
> 
> No functional change: build_zonelists() builds and prints the same FALLBACK
> list over N_MEMORY with node_load updates as before.
> 
> Signed-off-by: Gregory Price <gourry@gourry.net>

Nice, glad this way of refactor was feasible.

Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>

> ---
>  mm/page_alloc.c | 63 ++++++++++++++++++-------------------------------
>  1 file changed, 23 insertions(+), 40 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 7efce139d562..d1888d5630e0 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5857,31 +5857,6 @@ int find_next_best_node_in(int node, nodemask_t *used_node_mask,
>  }
>  
>  
> -/*
> - * Build zonelists ordered by node and zones within node.
> - * This results in maximum locality--normal zone overflows into local
> - * DMA zone, if any--but risks exhausting DMA zone.
> - */
> -static void build_zonelists_in_node_order(pg_data_t *pgdat, int *node_order,
> -		unsigned nr_nodes)
> -{
> -	struct zoneref *zonerefs;
> -	int i;
> -
> -	zonerefs = pgdat->node_zonelists[ZONELIST_FALLBACK]._zonerefs;
> -
> -	for (i = 0; i < nr_nodes; i++) {
> -		int nr_zones;
> -
> -		pg_data_t *node = NODE_DATA(node_order[i]);
> -
> -		nr_zones = build_zonerefs_node(node, zonerefs);
> -		zonerefs += nr_zones;
> -	}
> -	zonerefs->zone = NULL;
> -	zonerefs->zone_idx = 0;
> -}
> -
>  /*
>   * Build __GFP_THISNODE zonelists
>   */
> @@ -5897,20 +5872,24 @@ static void build_thisnode_zonelists(pg_data_t *pgdat)
>  	zonerefs->zone_idx = 0;
>  }
>  
> -static void build_zonelists(pg_data_t *pgdat)
> +/*
> + * Build one zonelist ordered by node and zones within node. This results in
> + * maximum locality--normal zone overflows into local DMA zone, if any--but
> + * risks exhausting DMA zone.
> + */
> +static void build_node_zonelist(pg_data_t *pgdat, const nodemask_t *candidates,
> +				int zlidx)
>  {
> -	static int node_order[MAX_NUMNODES];
> -	int node, nr_nodes = 0;
> +	struct zoneref *zonerefs = pgdat->node_zonelists[zlidx]._zonerefs;
>  	nodemask_t used_mask = NODE_MASK_NONE;
> -	int local_node, prev_node;
> +	int local_node = pgdat->node_id;
> +	int prev_node = local_node;
> +	int node;
>  
> -	/* NUMA-aware ordering of nodes */
> -	local_node = pgdat->node_id;
> -	prev_node = local_node;
> +	pr_info("Fallback order for Node %d: ", local_node);
>  
> -	memset(node_order, 0, sizeof(node_order));
>  	while ((node = find_next_best_node_in(local_node, &used_mask,
> -					      &node_states[N_MEMORY])) >= 0) {
> +					      candidates)) >= 0) {
>  		/*
>  		 * We don't want to pressure a particular node.
>  		 * So adding penalty to the first node in same
> @@ -5920,18 +5899,22 @@ static void build_zonelists(pg_data_t *pgdat)
>  		    node_distance(local_node, prev_node))
>  			node_load[node] += 1;
>  
> -		node_order[nr_nodes++] = node;
> +		zonerefs += build_zonerefs_node(NODE_DATA(node), zonerefs);
> +		pr_cont("%d ", node);
>  		prev_node = node;
>  	}
>  
> -	build_zonelists_in_node_order(pgdat, node_order, nr_nodes);
> -	build_thisnode_zonelists(pgdat);
> -	pr_info("Fallback order for Node %d: ", local_node);
> -	for (node = 0; node < nr_nodes; node++)
> -		pr_cont("%d ", node_order[node]);
> +	zonerefs->zone = NULL;
> +	zonerefs->zone_idx = 0;
>  	pr_cont("\n");
>  }
>  
> +static void build_zonelists(pg_data_t *pgdat)
> +{
> +	build_node_zonelist(pgdat, &node_states[N_MEMORY], ZONELIST_FALLBACK);
> +	build_thisnode_zonelists(pgdat);
> +}
> +
>  #ifdef CONFIG_HAVE_MEMORYLESS_NODES
>  /*
>   * Return node id of node used for "local" allocations.

Re: [PATCH v2 2/2] mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()

[Gregory Price]

On Mon, Sep 14, 2026 at 04:56:30PM +0200, Vlastimil Babka (SUSE) wrote:
> On 9/12/26 05:04, Gregory Price wrote:
> > Extract per-node fallback-list construction into build_node_zonelist().
> > Build each selected node directly into the destination zonelist so no
> > intermediate node_order array or node count is needed. Print the fallback
> > order as each node is added.
> > 
> > This lets us build new zonelists from candidate nodemasks instead of just
> > the default N_MEMORY node state list.
> > 
> > No functional change: build_zonelists() builds and prints the same FALLBACK
> > list over N_MEMORY with node_load updates as before.
> > 
> > Signed-off-by: Gregory Price <gourry@gourry.net>
> 
> Nice, glad this way of refactor was feasible.
> 

Yup, came out much cleaner than i expected.

> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> 

Thank you!

Re: [PATCH v2 0/2] mm: refactor zonelist constructors and iterators

[Zenghui Yu]

On 9/12/26 11:04 AM, Gregory Price wrote:
> find_next_best_node() picks the next-closest node when building a
> fallback list, and hardcodes N_MEMORY as the set it picks from.
> 
> Refactor it into find_next_best_node_in(), which takes the candidate
> set explicitly.
> 
> This makes the existing behaviour explicit at both mm/memory-tiers.c
> call sites - they select demotion targets in fallback order from
> N_MEMORY - and lets callers narrow that set.
> 
> Then extract the per-node construction loop out of build_zonelists()
> into build_node_zonelist(),i parameterised on the candidate nodemask
> and destination zonelist index.
> 
> Together these allow a zonelist to be built over a candidate set other
> than N_MEMORY, into a zonelist other than FALLBACK, and iterated in
> fallback order over a caller-defined subset.
> 
> These are prerequisites for generating a private node zonelist (nodes
> unreachable by default), but are otherwise general improvements to
> the existing interfaces so I'm proposing them separately.
> 
> No functional change intended - purely refactor commits.
> 
> Verified on x86_64:
> 
>   - find_next_best_node_in() compiles to the same 319 bytes as
>     find_next_best_node().
>   - Both mm/memory-tiers.c callers grow a single instruction - the
>     added argument.
>   - Boot-time fallback orders are identical on a 4-node guest with an
>     asymmetric distance matrix.
> 
> Gregory Price (2):
>   mm: refactor find_next_best_node to find_next_best_node_in
>   mm/page_alloc: refactor build_node_zonelist() out of build_zonelists()
> 
>  mm/internal.h     |  6 ++--
>  mm/memory-tiers.c |  7 +++--
>  mm/page_alloc.c   | 74 +++++++++++++++++++----------------------------
>  3 files changed, 38 insertions(+), 49 deletions(-)

Reviewed-by: Zenghui Yu (Huawei) <zenghui.yu@linux.dev>

Thanks,
Zenghui