[PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[Paul Sherman]

On large NUMA systems (e.g. 4-node 128GB), the physical span between
NUMA nodes may exceed 75% of vmalloc space even though the total percpu
allocation is tiny (e.g. ~6MB for 64 CPUs).

The 75% check was designed for pcpu_get_vm_areas() used by dynamic
percpu chunks, where vmalloc congruency is required. The embed
allocator uses the linear map directly for the first chunk -- physical
addresses are accessible as virtual addresses without vmalloc mapping.
The relevant constraint is total percpu size, not physical span.

Replace the unconditional fallback with a total-size check: if the
aggregate percpu allocation fits within 75% of vmalloc space, proceed
with embed regardless of physical NUMA span. Systems where total percpu
size genuinely exceeds the vmalloc bound retain the page allocator
fallback.

Tested on Sophgo SG2042 (64-hart, 4-NUMA, 128GB DDR4, RISC-V Sv39):
  percpu: embed: span 0x17def7a000 > vmalloc 75% but total 0x5c0000
          fits -- linear map used
  percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208

Link: https://lkml.iu.edu/hypermail/linux/kernel/1707.3/00337.html
Cc: Tejun Heo <tj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Dennis Zhou <dennis@kernel.org>
Signed-off-by: Paul Sherman <shermanpauldylan@gmail.com>
---

This fix is inert on systems where the original check never fired;
it only activates when physical span exceeds 75% of vmalloc but total
percpu size does not.

Architecture analysis:

  Architecture   | vmalloc      | phys span    | total percpu | result
  ---------------|--------------|--------------|--------------|--------
  RISC-V Sv39    | ~88 GB       | ~102 GB      | ~6 MB        | fixed
  RISC-V Sv48    | ~88 TB       | ~102 GB      | ~6 MB        | unaffected
  RISC-V Sv57    | ~44 PB       | ~102 GB      | ~6 MB        | unaffected
  ARM64 large    | ~248 TB      | varies       | tiny         | unaffected
  32-bit NUMA    | ~128 MB      | varies       | may exceed   | correct fallback

Tejun Heo noted in 2017 [Link] that the only constraint is vmalloc
size relative to NUMA node distances, and that making vmalloc bigger
would be the best fix. On RISC-V Sv39 with 88GB vmalloc and 102GB
physical NUMA span that is not an option -- but the embed allocator
does not need congruent vmalloc mapping for the first chunk anyway.

 mm/percpu.c | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index b0676b8054ed..72695778872c 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -3069,13 +3069,32 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 
 	/* warn if maximum distance is further than 75% of vmalloc space */
 	if (max_distance > VMALLOC_TOTAL * 3 / 4) {
-		pr_warn("max_distance=0x%lx too large for vmalloc space 0x%lx\n",
+		pr_warn("percpu: embed: max_distance=0x%lx too large for vmalloc space 0x%lx\n",
 				max_distance, VMALLOC_TOTAL);
 #ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
-		/* and fail if we have fallback */
-		rc = -EINVAL;
-		goto out_free_areas;
-#endif
+		/*
+		 * The embed allocator uses the linear map directly for the
+		 * first chunk -- physical addresses are accessible as virtual
+		 * addresses without vmalloc mapping. The 75% check was designed
+		 * for pcpu_get_vm_areas() (dynamic chunks) where vmalloc
+		 * congruency is required. On large NUMA systems, physical span
+		 * between nodes may exceed vmalloc bounds even though total
+		 * percpu size is tiny. Check total size, not physical span.
+		 */
+		{
+			unsigned long total_size = 0;
+			for (group = 0; group < ai->nr_groups; group++)
+				total_size += (unsigned long)ai->unit_size *
+					ai->groups[group].nr_units;
+			if (total_size > VMALLOC_TOTAL * 3 / 4) {
+				rc = -EINVAL;
+				goto out_free_areas;
+			}
+			pr_info("percpu: embed: span 0x%lx > vmalloc 75%%"
+				" but total 0x%lx fits -- linear map used\n",
+				max_distance, total_size);
+		}
+#endif /* CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK */
 	}
 
 	/*
-- 
2.53.0

Re: [PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[Andrew Morton]

On Fri, 17 Jul 2026 16:16:16 -0700 Paul Sherman <shermanpauldylan@gmail.com> wrote:

> On large NUMA systems (e.g. 4-node 128GB), the physical span between
> NUMA nodes may exceed 75% of vmalloc space even though the total percpu
> allocation is tiny (e.g. ~6MB for 64 CPUs).
> 
> The 75% check was designed for pcpu_get_vm_areas() used by dynamic
> percpu chunks, where vmalloc congruency is required. The embed
> allocator uses the linear map directly for the first chunk -- physical
> addresses are accessible as virtual addresses without vmalloc mapping.
> The relevant constraint is total percpu size, not physical span.
> 
> Replace the unconditional fallback with a total-size check: if the
> aggregate percpu allocation fits within 75% of vmalloc space, proceed
> with embed regardless of physical NUMA span. Systems where total percpu
> size genuinely exceeds the vmalloc bound retain the page allocator
> fallback.
> 
> Tested on Sophgo SG2042 (64-hart, 4-NUMA, 128GB DDR4, RISC-V Sv39):
>   percpu: embed: span 0x17def7a000 > vmalloc 75% but total 0x5c0000
>           fits -- linear map used
>   percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208

Nicely presented patch, thanks.

> Link: https://lkml.iu.edu/hypermail/linux/kernel/1707.3/00337.html
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: Dennis Zhou <dennis@kernel.org>
> Signed-off-by: Paul Sherman <shermanpauldylan@gmail.com>
> ---
> 
> This fix is inert on systems where the original check never fired;
> it only activates when physical span exceeds 75% of vmalloc but total
> percpu size does not.
> 
> Architecture analysis:
> 
>   Architecture   | vmalloc      | phys span    | total percpu | result
>   ---------------|--------------|--------------|--------------|--------
>   RISC-V Sv39    | ~88 GB       | ~102 GB      | ~6 MB        | fixed
>   RISC-V Sv48    | ~88 TB       | ~102 GB      | ~6 MB        | unaffected
>   RISC-V Sv57    | ~44 PB       | ~102 GB      | ~6 MB        | unaffected
>   ARM64 large    | ~248 TB      | varies       | tiny         | unaffected
>   32-bit NUMA    | ~128 MB      | varies       | may exceed   | correct fallback
> 
> Tejun Heo noted in 2017 [Link] that the only constraint is vmalloc
> size relative to NUMA node distances, and that making vmalloc bigger
> would be the best fix. On RISC-V Sv39 with 88GB vmalloc and 102GB
> physical NUMA span that is not an option -- but the embed allocator
> does not need congruent vmalloc mapping for the first chunk anyway.

I suggest all the above be moved above the "---" separator.  It's
useful info for the permanent record.  (And replace "[Link]" with "[1]"
in the usual fashion).

>  mm/percpu.c | 29 ++++++++++++++++++++++++-----
>  1 file changed, 24 insertions(+), 5 deletions(-)

AI review asked a question:
	https://sashiko.dev/#/patchset/20260717231616.9126-1-shermanpauldylan@gmail.com

Re: [PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[Paul Sherman]

Andrew,

Thank you. v2 will incorporate your formatting suggestions.

The Sashiko concern is valid and matches a WARN_ON() we observed in our
boot logs during dynamic percpu chunk allocation. It appears related to
the sparse physical group offsets reused after embed initialization, and
I'll address it in v2.

Thanks,
Paul Sherman

On Fri, 17 Jul 2026, Andrew Morton wrote:
> I suggest all the above be moved above the "---" separator.  It's
> useful info for the permanent record.  (And replace "[Link]" with "[1]"
> in the usual fashion).
>
> AI review asked a question:
>         https://sashiko.dev/#/patchset/20260717231616.9126-1-shermanpauldylan@gmail.com