[PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Eric Dumazet]

syzbot reported a sleeping function called from invalid context splat
in bucket_table_alloc().

When rhashtable_insert_slow() rehashes the table under rcu_read_lock(),
it calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN).
If the bucket table allocation uses vmalloc, __vmalloc_node_range_noprof()
invokes vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with
the passed GFP_ATOMIC flags.

If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
GFP_KERNEL when allocating the temporary weights array, triggering
a might_alloc() splat in atomic/RCU contexts.

Pass the gfp flags (masked with GFP_RECLAIM_MASK to strip page-allocator
zone modifiers like __GFP_HIGHMEM) received by
alloc_pages_bulk_weighted_interleave() to kmalloc() instead of
hardcoding GFP_KERNEL. Since the weights buffer is immediately
initialized in full, kmalloc() is sufficient.

Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")
Reported-by: syzbot+0dbf6d295b3350944f0b@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/lkml/6a88837e.ae6ddae5.3da009.0040.GAE@google.com/T/#u
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
Cc: David Hildenbrand <david@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Ying Huang <ying.huang@linux.alibaba.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: linux-mm@kvack.org
---
 mm/mempolicy.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 501e0b80d7da..1ef50ca37d42 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2688,7 +2688,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
 	prev_node = node;
 
 	/* create a local copy of node weights to operate on outside rcu */
-	weights = kzalloc(nr_node_ids, GFP_KERNEL);
+	weights = kmalloc(nr_node_ids, gfp & GFP_RECLAIM_MASK);
 	if (!weights)
 		return total_allocated;
 

base-commit: 4e69c1856bfd9ffb7e9d335a25842fa211628929
-- 
2.55.0.766.g2966f0265a-goog

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Gregory Price]

On Fri, Aug 21, 2026 at 05:04:07PM +0000, Eric Dumazet wrote:
> syzbot reported a sleeping function called from invalid context splat
> in bucket_table_alloc().
> 
> When rhashtable_insert_slow() rehashes the table under rcu_read_lock(),
> it calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN).
> If the bucket table allocation uses vmalloc, __vmalloc_node_range_noprof()
> invokes vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with
> the passed GFP_ATOMIC flags.
> 
> If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
> alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
> GFP_KERNEL when allocating the temporary weights array, triggering
> a might_alloc() splat in atomic/RCU contexts.
> 
> Pass the gfp flags (masked with GFP_RECLAIM_MASK to strip page-allocator
> zone modifiers like __GFP_HIGHMEM) received by
> alloc_pages_bulk_weighted_interleave() to kmalloc() instead of
> hardcoding GFP_KERNEL. Since the weights buffer is immediately
> initialized in full, kmalloc() is sufficient.
> 
> Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")
> Reported-by: syzbot+0dbf6d295b3350944f0b@syzkaller.appspotmail.com
> Closes: https://lore.kernel.org/lkml/6a88837e.ae6ddae5.3da009.0040.GAE@google.com/T/#u
> Signed-off-by: Eric Dumazet <edumazet@google.com>

Yup, seems a solid oversight on my part.

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

~Gregory

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Andrew Morton]

On Fri, 21 Aug 2026 17:04:07 +0000 Eric Dumazet <edumazet@google.com> wrote:

> syzbot reported a sleeping function called from invalid context splat
> in bucket_table_alloc().

That was quick (7 minutes!).  I was just looking at this.

> When rhashtable_insert_slow() rehashes the table under rcu_read_lock(),
> it calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN).
> If the bucket table allocation uses vmalloc, __vmalloc_node_range_noprof()
> invokes vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with
> the passed GFP_ATOMIC flags.
> 
> If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
> alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
> GFP_KERNEL when allocating the temporary weights array, triggering
> a might_alloc() splat in atomic/RCU contexts.

2 years ago.  Why are we discovering this now?

> Pass the gfp flags (masked with GFP_RECLAIM_MASK to strip page-allocator
> zone modifiers like __GFP_HIGHMEM) received by
> alloc_pages_bulk_weighted_interleave() to kmalloc() instead of
> hardcoding GFP_KERNEL. Since the weights buffer is immediately
> initialized in full, kmalloc() is sufficient.
> 
> Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")

I'll add cc:stable

> --- a/mm/mempolicy.c
> +++ b/mm/mempolicy.c
> @@ -2688,7 +2688,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
>  	prev_node = node;
>  
>  	/* create a local copy of node weights to operate on outside rcu */
> -	weights = kzalloc(nr_node_ids, GFP_KERNEL);
> +	weights = kmalloc(nr_node_ids, gfp & GFP_RECLAIM_MASK);

lgtm, thanks.

I wonder if we *really* need the local copy of state->iw_table. 
Perhaps with appropriate care we can directly use state->iw_table in
here.

How much would it hurt to expand the rcu_read_lock() coverage?

A local array of MAX_NUMNODES bytes isn't attractive - 1k of stack.

A spinlock-protected static array would work, if super-rare slowpath.

>  	if (!weights)
>  		return total_allocated;

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Gregory Price]

On Fri, Aug 21, 2026 at 10:40:43AM -0700, Andrew Morton wrote:
> On Fri, 21 Aug 2026 17:04:07 +0000 Eric Dumazet <edumazet@google.com> wrote:
> 
> I wonder if we *really* need the local copy of state->iw_table. 
> Perhaps with appropriate care we can directly use state->iw_table in
> here.
> 
> How much would it hurt to expand the rcu_read_lock() coverage?
> 
> A local array of MAX_NUMNODES bytes isn't attractive - 1k of stack.
> 
> A spinlock-protected static array would work, if super-rare slowpath.
> 

I would need to go back through the original thread when i authored
this, but I think this is possible.  I will add it to my backlog to
investigate.

Weight updates should be *extraordinarily* rare, unless someone is doing
dynamic runtime twiddling of the sysfs values and thinks that's somehow
a good idea (or expecting them to be timely).

~Gregory

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Eric Dumazet]

On Fri, Aug 21, 2026 at 7:40 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Fri, 21 Aug 2026 17:04:07 +0000 Eric Dumazet <edumazet@google.com> wrote:
>
> > syzbot reported a sleeping function called from invalid context splat
> > in bucket_table_alloc().
>
> That was quick (7 minutes!).  I was just looking at this.

That is because I had the syzbot report in our private queue.

I thought it was a networking bug at first.

When I realized this was an mm bug, I decided to cook the patch,
release the syzbot report, and send the patch :)

>
> > When rhashtable_insert_slow() rehashes the table under rcu_read_lock(),
> > it calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN).
> > If the bucket table allocation uses vmalloc, __vmalloc_node_range_noprof()
> > invokes vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with
> > the passed GFP_ATOMIC flags.
> >
> > If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
> > alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
> > GFP_KERNEL when allocating the temporary weights array, triggering
> > a might_alloc() splat in atomic/RCU contexts.
>
> 2 years ago.  Why are we discovering this now?

syzbot got better in recent weeks.

Google folks can look at go/syzkaller-llm-fuzzing

Thanks.

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[Gregory Price]

On Fri, Aug 21, 2026 at 10:40:43AM -0700, Andrew Morton wrote:
> > --- a/mm/mempolicy.c
> > +++ b/mm/mempolicy.c
> > @@ -2688,7 +2688,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
> >  	prev_node = node;
> >  
> >  	/* create a local copy of node weights to operate on outside rcu */
> > -	weights = kzalloc(nr_node_ids, GFP_KERNEL);
> > +	weights = kmalloc(nr_node_ids, gfp & GFP_RECLAIM_MASK);
> 
> I wonder if we *really* need the local copy of state->iw_table. 
> Perhaps with appropriate care we can directly use state->iw_table in
> here.
> 
> How much would it hurt to expand the rcu_read_lock() coverage?
> 

Ah, the current space we'd expand rcu read lock into is the actual
allocation - which we can't do. Same with the spinlock.

We can clean this up with a refcount + rcu_free and kill the allocation
in the hot path.  Will get something out this week.

~Gregory

[PATCH] mm/mempolicy: refcount the weighted interleave state instead of copying it

[Gregory Price]

alloc_pages_bulk_weighted_interleave() copies iw_table into a scratch
array on every call to get the table outside of RCU.

Refcount the weighted interleave state and cleanup with kfree_rcu().
Refcount and iw_table get their own cachelines to prevent false sharing.

This drops a kzalloc/memcpy/kfree per call and deals with a bug induced
by the scratch array's hardcoded GFP_KERNEL and the partial allocation it
returned when that failed.

Tested in VM (KASAN, PROVE_LOCKING and DEBUG_OBJECTS_RCU_HEAD) with a
udelay() injected between the rcu_dereference() and the refcount_inc
to stress the race. Six concurrent bulk allocators racing four threads
writing the sysfs weights took the retry path 2536 times with no splat,
and the published state was back to a count of one at rest.

Replacing the kfree_rcu() with a bare kfree() in that same test reports
a use-after-free immediately, so the test does exercise what the deferred
free protects.

Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")
Cc: stable@vger.kernel.org
Reported-by: Eric Dumazet <edumazet@google.com>
Link: https://lore.kernel.org/all/20260821170407.3721004-1-edumazet@google.com/
Reported-by: syzbot+0dbf6d295b3350944f0b@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/lkml/6a88837e.ae6ddae5.3da009.0040.GAE@google.com/T/#u
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Assisted-by: Claude:claude-opus-5
Signed-off-by: Gregory Price (Meta) <gourry@gourry.net>
---

Hi Andrew - please consider this instead.

 mm/mempolicy.c | 76 ++++++++++++++++++++++++++------------------------
 1 file changed, 39 insertions(+), 37 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 0e5175f1c767..4a3722c63b31 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -112,6 +112,7 @@
 #include <linux/printk.h>
 #include <linux/leafops.h>
 #include <linux/gcd.h>
+#include <linux/refcount.h>
 
 #include <asm/tlbflush.h>
 #include <asm/tlb.h>
@@ -156,7 +157,9 @@ static const int weightiness = 32;
  */
 struct weighted_interleave_state {
 	bool mode_auto;
-	u8 iw_table[];
+	refcount_t refcnt;
+	struct rcu_head rcu;
+	u8 iw_table[] ____cacheline_aligned_in_smp;
 };
 static struct weighted_interleave_state __rcu *wi_state;
 static unsigned int *node_bw_table;
@@ -167,6 +170,27 @@ static unsigned int *node_bw_table;
  */
 static DEFINE_MUTEX(wi_state_lock);
 
+/* Allow sleeping readers to pin the state to avoid taking copies */
+static struct weighted_interleave_state *wi_state_get(void)
+{
+	struct weighted_interleave_state *state;
+
+	rcu_read_lock();
+	while ((state = rcu_dereference(wi_state))) {
+		if (refcount_inc_not_zero(&state->refcnt))
+			break;
+	}
+	rcu_read_unlock();
+
+	return state;
+}
+
+static void wi_state_put(struct weighted_interleave_state *state)
+{
+	if (state && refcount_dec_and_test(&state->refcnt))
+		kfree_rcu(state, rcu);
+}
+
 static u8 get_il_weight(int node)
 {
 	struct weighted_interleave_state *state;
@@ -235,6 +259,7 @@ int mempolicy_set_node_perf(unsigned int node, struct access_coordinate *coords)
 		return -ENOMEM;
 	}
 	new_wi_state->mode_auto = true;
+	refcount_set(&new_wi_state->refcnt, 1);
 	for (i = 0; i < nr_node_ids; i++)
 		new_wi_state->iw_table[i] = 1;
 
@@ -265,10 +290,7 @@ int mempolicy_set_node_perf(unsigned int node, struct access_coordinate *coords)
 	rcu_assign_pointer(wi_state, new_wi_state);
 
 	mutex_unlock(&wi_state_lock);
-	if (old_wi_state) {
-		synchronize_rcu();
-		kfree(old_wi_state);
-	}
+	wi_state_put(old_wi_state);
 out:
 	kfree(old_bw);
 	return 0;
@@ -2632,7 +2654,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
 	unsigned long nr_allocated = 0;
 	unsigned long rounds;
 	unsigned long node_pages, delta;
-	u8 *weights, weight;
+	u8 *table, weight;
 	unsigned int weight_total = 0;
 	unsigned long rem_pages = nr_pages;
 	nodemask_t nodes;
@@ -2676,25 +2698,12 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
 	me->il_weight = 0;
 	prev_node = node;
 
-	/* create a local copy of node weights to operate on outside rcu */
-	weights = kzalloc(nr_node_ids, GFP_KERNEL);
-	if (!weights)
-		return total_allocated;
-
-	rcu_read_lock();
-	state = rcu_dereference(wi_state);
-	if (state) {
-		memcpy(weights, state->iw_table, nr_node_ids * sizeof(u8));
-		rcu_read_unlock();
-	} else {
-		rcu_read_unlock();
-		for (i = 0; i < nr_node_ids; i++)
-			weights[i] = 1;
-	}
+	state = wi_state_get();
+	table = state ? state->iw_table : NULL;
 
 	/* calculate total, detect system default usage */
 	for_each_node_mask(node, nodes)
-		weight_total += weights[node];
+		weight_total += table ? table[node] : 1;
 
 	/*
 	 * Calculate rounds/partial rounds to minimize __alloc_pages_bulk calls.
@@ -2706,10 +2715,10 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
 	rounds = rem_pages / weight_total;
 	delta = rem_pages % weight_total;
 	resume_node = next_node_in(prev_node, nodes);
-	resume_weight = weights[resume_node];
+	resume_weight = table ? table[resume_node] : 1;
 	for (i = 0; i < nnodes; i++) {
 		node = next_node_in(prev_node, nodes);
-		weight = weights[node];
+		weight = table ? table[node] : 1;
 		node_pages = weight * rounds;
 		/* If a delta exists, add this node's portion of the delta */
 		if (delta > weight) {
@@ -2735,7 +2744,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
 	}
 	me->il_prev = resume_node;
 	me->il_weight = resume_weight;
-	kfree(weights);
+	wi_state_put(state);
 	return total_allocated;
 }
 
@@ -3644,6 +3653,7 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 	new_wi_state = kzalloc_flex(*new_wi_state, iw_table, nr_node_ids);
 	if (!new_wi_state)
 		return -ENOMEM;
+	refcount_set(&new_wi_state->refcnt, 1);
 
 	mutex_lock(&wi_state_lock);
 	old_wi_state = rcu_dereference_protected(wi_state,
@@ -3660,10 +3670,7 @@ static ssize_t node_store(struct kobject *kobj, struct kobj_attribute *attr,
 
 	rcu_assign_pointer(wi_state, new_wi_state);
 	mutex_unlock(&wi_state_lock);
-	if (old_wi_state) {
-		synchronize_rcu();
-		kfree(old_wi_state);
-	}
+	wi_state_put(old_wi_state);
 	return count;
 }
 
@@ -3696,6 +3703,7 @@ static ssize_t weighted_interleave_auto_store(struct kobject *kobj,
 	new_wi_state = kzalloc_flex(*new_wi_state, iw_table, nr_node_ids);
 	if (!new_wi_state)
 		return -ENOMEM;
+	refcount_set(&new_wi_state->refcnt, 1);
 	for (i = 0; i < nr_node_ids; i++)
 		new_wi_state->iw_table[i] = 1;
 
@@ -3728,10 +3736,7 @@ static ssize_t weighted_interleave_auto_store(struct kobject *kobj,
 update_wi_state:
 	rcu_assign_pointer(wi_state, new_wi_state);
 	mutex_unlock(&wi_state_lock);
-	if (old_wi_state) {
-		synchronize_rcu();
-		kfree(old_wi_state);
-	}
+	wi_state_put(old_wi_state);
 	return count;
 }
 
@@ -3775,10 +3780,7 @@ static void wi_state_free(void)
 	rcu_assign_pointer(wi_state, NULL);
 	mutex_unlock(&wi_state_lock);
 
-	if (old_wi_state) {
-		synchronize_rcu();
-		kfree(old_wi_state);
-	}
+	wi_state_put(old_wi_state);
 }
 
 static struct kobj_attribute wi_auto_attr =
-- 
2.55.0

Re: [PATCH] mm/mempolicy: refcount the weighted interleave state instead of copying it

[Matthew Wilcox]

On Sun, Aug 23, 2026 at 10:41:17PM -0400, Gregory Price wrote:
> @@ -2632,7 +2654,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
>  	unsigned long nr_allocated = 0;
>  	unsigned long rounds;
>  	unsigned long node_pages, delta;
> -	u8 *weights, weight;
> +	u8 *table, weight;
>  	unsigned int weight_total = 0;
>  	unsigned long rem_pages = nr_pages;
>  	nodemask_t nodes;
> @@ -2676,25 +2698,12 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
>  	me->il_weight = 0;
>  	prev_node = node;
>  
> -	/* create a local copy of node weights to operate on outside rcu */
> -	weights = kzalloc(nr_node_ids, GFP_KERNEL);
> -	if (!weights)
> -		return total_allocated;
> -
> -	rcu_read_lock();
> -	state = rcu_dereference(wi_state);
> -	if (state) {
> -		memcpy(weights, state->iw_table, nr_node_ids * sizeof(u8));
> -		rcu_read_unlock();
> -	} else {
> -		rcu_read_unlock();
> -		for (i = 0; i < nr_node_ids; i++)
> -			weights[i] = 1;
> -	}
> +	state = wi_state_get();
> +	table = state ? state->iw_table : NULL;
>  
>  	/* calculate total, detect system default usage */
>  	for_each_node_mask(node, nodes)
> -		weight_total += weights[node];
> +		weight_total += table ? table[node] : 1;
>  
>  	/*
>  	 * Calculate rounds/partial rounds to minimize __alloc_pages_bulk calls.
> @@ -2706,10 +2715,10 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
>  	rounds = rem_pages / weight_total;
>  	delta = rem_pages % weight_total;
>  	resume_node = next_node_in(prev_node, nodes);
> -	resume_weight = weights[resume_node];
> +	resume_weight = table ? table[resume_node] : 1;
>  	for (i = 0; i < nnodes; i++) {
>  		node = next_node_in(prev_node, nodes);
> -		weight = weights[node];
> +		weight = table ? table[node] : 1;
>  		node_pages = weight * rounds;
>  		/* If a delta exists, add this node's portion of the delta */
>  		if (delta > weight) {
> @@ -2735,7 +2744,7 @@ static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,
>  	}
>  	me->il_prev = resume_node;
>  	me->il_weight = resume_weight;
> -	kfree(weights);
> +	wi_state_put(state);
>  	return total_allocated;
>  }
>  

Bit of a shame to take/put a refcount on every memory allocation.  That
seems like it might hurt (even being on a different cacheline).  Would
it make more sense to use SRCU for this?

Re: [PATCH] mm/mempolicy: refcount the weighted interleave state instead of copying it

[Gregory Price]

On Mon, Aug 24, 2026 at 04:06:03AM +0100, Matthew Wilcox wrote:
> On Sun, Aug 23, 2026 at 10:41:17PM -0400, Gregory Price wrote:
> 
> Bit of a shame to take/put a refcount on every memory allocation.  That
> seems like it might hurt (even being on a different cacheline).  Would
> it make more sense to use SRCU for this?

Bulk allocation - from some measurements it's about once per 100-pages
on average.  Looking back at the data I was getting about 41 refs per
4096 pages (16MB chunks).

Have to imagine there's other cache fighting in that stack too. 

I'm less familiar with the rules around SRCU so i shied away from it,
but I will take a look.

~Gregory

Re: [PATCH] mm/mempolicy: refcount the weighted interleave state instead of copying it

[Gregory Price]

On Sun, Aug 23, 2026 at 11:52:55PM -0400, Gregory Price wrote:
> On Mon, Aug 24, 2026 at 04:06:03AM +0100, Matthew Wilcox wrote:
> > On Sun, Aug 23, 2026 at 10:41:17PM -0400, Gregory Price wrote:
> > 
> > Bit of a shame to take/put a refcount on every memory allocation.  That
> > seems like it might hurt (even being on a different cacheline).  Would
> > it make more sense to use SRCU for this?
> 
> Bulk allocation - from some measurements it's about once per 100-pages
> on average.  Looking back at the data I was getting about 41 refs per
> 4096 pages (16MB chunks).
> 
> Have to imagine there's other cache fighting in that stack too. 
> 
> I'm less familiar with the rules around SRCU so i shied away from it,
> but I will take a look.
> 

I will update to the new fast srcu mechanism, but it'll prevent
backporting beyond 6.18.

Best to backport the gfp fix and update to srcu in normal cadence.

Will submit separately.

~Gregory

Re: [PATCH] mm/mempolicy: refcount the weighted interleave state instead of copying it

[Andrew Morton]

On Sun, 23 Aug 2026 22:41:17 -0400 Gregory Price <gourry@gourry.net> wrote:

> alloc_pages_bulk_weighted_interleave() copies iw_table into a scratch
> array on every call to get the table outside of RCU.
> 
> Refcount the weighted interleave state and cleanup with kfree_rcu().
> Refcount and iw_table get their own cachelines to prevent false sharing.
> 
> This drops a kzalloc/memcpy/kfree per call and deals with a bug induced
> by the scratch array's hardcoded GFP_KERNEL and the partial allocation it
> returned when that failed.
> 
> Tested in VM (KASAN, PROVE_LOCKING and DEBUG_OBJECTS_RCU_HEAD) with a
> udelay() injected between the rcu_dereference() and the refcount_inc
> to stress the race. Six concurrent bulk allocators racing four threads
> writing the sysfs weights took the retry path 2536 times with no splat,
> and the published state was back to a count of one at rest.
> 
> Replacing the kfree_rcu() with a bare kfree() in that same test reports
> a use-after-free immediately, so the test does exercise what the deferred
> free protects.
> 
> ...
> 
> Hi Andrew - please consider this instead.

Let's keep Eric's nice simple one-liner
(https://lore.kernel.org/20260821170407.3721004-1-edumazet@google.com)
for a backportable fix.  THen we can work on more sophisticated
approaches in the next -rc cycle.

Re: [PATCH] mm/mempolicy: Fix sleeping allocation in alloc_pages_bulk_weighted_interleave()

[David Hildenbrand (Arm)]

On 8/21/26 19:04, Eric Dumazet wrote:
> syzbot reported a sleeping function called from invalid context splat
> in bucket_table_alloc().
> 
> When rhashtable_insert_slow() rehashes the table under rcu_read_lock(),
> it calls bucket_table_alloc(..., GFP_ATOMIC | __GFP_NOWARN).
> If the bucket table allocation uses vmalloc, __vmalloc_node_range_noprof()
> invokes vm_area_alloc_pages() -> alloc_pages_bulk_mempolicy_noprof() with
> the passed GFP_ATOMIC flags.
> 
> If the current task has an MPOL_WEIGHTED_INTERLEAVE mempolicy,
> alloc_pages_bulk_weighted_interleave() is called and currently hardcodes
> GFP_KERNEL when allocating the temporary weights array, triggering
> a might_alloc() splat in atomic/RCU contexts.
> 
> Pass the gfp flags (masked with GFP_RECLAIM_MASK to strip page-allocator
> zone modifiers like __GFP_HIGHMEM) received by
> alloc_pages_bulk_weighted_interleave() to kmalloc() instead of
> hardcoding GFP_KERNEL. Since the weights buffer is immediately
> initialized in full, kmalloc() is sufficient.
> 
> Fixes: fa3bea4e1f82 ("mm/mempolicy: introduce MPOL_WEIGHTED_INTERLEAVE for weighted interleaving")
> Reported-by: syzbot+0dbf6d295b3350944f0b@syzkaller.appspotmail.com
> Closes: https://lore.kernel.org/lkml/6a88837e.ae6ddae5.3da009.0040.GAE@google.com/T/#u
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David