[PATCH 1/2] powerpc/64s/radix: don't dedup vmemmap covered by an existing PMD leaf

[PATCH 1/2] powerpc/64s/radix: don't dedup vmemmap covered by an existing PMD leaf

[Ritesh Harjani (IBM)]

radix__vmemmap_populate() is asked for one 16MB section, which is 16KB of
struct pages. With no altmap it rounds the request down to a PMD leaf.
That leaf describes 2GB of PFNs with 64KB pages (128MB with 4KB pages),
so it also covers sections that are not present yet. It cannot be freed
while any of those sections are still in use.

A devdax namespace that lands in the same window is marked for vmemmap dedup.
vmemmap_populate_compound_pages() treated the leaf as an existing huge mapping
and skipped it. The core then initializes only the head vmemmap page and assumes
the rest is the shared tail. Those remaining tail slots are real struct pages
still holding the page_init_poison() pattern and are not the shared tail pages.
__init_single_page() warns on the zone mismatch and the following pageblock init
faults in __set_pfnblock_flags_mask().

If the section's vmemmap is already a PMD leaf, clear its compound order
and reuse the leaf, so every struct page in the section is initialized.
The shared-tail layout needs base-page PTEs, so it cannot be built on top of
this leaf and dropping the leaf is not an option while the neighbouring range
is still using it. So, fail the compound populate if a leaf is still seen there.

<Call Trace>
============
[    3.467060][    T1] WARNING: mm/mm_init.c:600 at __init_single_page+0xd0/0xf4, CPU#0: swapper/0/1
<...>
[    3.477419][    T1] NIP [c000000000576b30] __init_single_page+0xd0/0xf4
[    3.477827][    T1] LR [c0000000005778b0] __init_zone_device_page+0x30/0xb8
[    3.478444][    T1] Call Trace:
[    3.478764][    T1]  0x10000 (unreliable)
[    3.479874][    T1]  memmap_init_zone_device+0x164/0x430
[    3.480392][    T1]  memremap_pages+0x43c/0x93c
[    3.480836][    T1]  devm_memremap_pages+0x3c/0xa4
[    3.481328][    T1]  dev_dax_probe+0x264/0x3d0
[    3.481817][    T1]  dax_bus_probe+0x140/0x1c8
[    3.482237][    T1]  really_probe+0x108/0x580
[    3.482681][    T1]  __driver_probe_device+0xcc/0x284
[    3.483186][    T1]  driver_probe_device+0x5c/0x108
[    3.483656][    T1]  __device_attach_driver+0x10c/0x1e8
[    3.484134][    T1]  bus_for_each_drv+0xc0/0x13c
<...>
[    3.499972][    T1] BUG: Kernel NULL pointer dereference on read at 0x00001400
[    3.500417][    T1] Faulting instruction address: 0xc0000000005f4ecc
[    3.500417][    T1] Faulting instruction address: 0xc0000000005f4ecc
cpu 0x0: Vector: 300 (Data Access) at [c000000006756b50]
    pc: c0000000005f4ecc: __set_pfnblock_flags_mask.constprop.0+0x34/0x160
    lr: c000000000577f38: memmap_init_zone_device+0x358/0x430
    pid   = 1, comm = swapper/0
enter ? for help
  memmap_init_zone_device+0x358/0x430
  memmap_init_zone_device+0x220/0x430 (unreliable)
  memremap_pages+0x43c/0x93c
  devm_memremap_pages+0x3c/0xa4
  dev_dax_probe+0x264/0x3d0
  dax_bus_probe+0x140/0x1c8
  really_probe+0x108/0x580
<...>

Fixes: bd0862382adb ("mm/sparse-vmemmap: drop Device DAX-specific population path")
Reported-by: Pavithra <pavrampu@linux.ibm.com>
Assisted-by: LLM
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
Fixes tag here is added since the fix is dependent upon the change mentioned in
"Fixes" tag. The Fixes commit-id is from mm-new branch. Although this could be
an older pre-existing issue, but got exposed after this patch series.

 arch/powerpc/mm/book3s64/radix_pgtable.c | 57 +++++++++++++++++++++---
 1 file changed, 50 insertions(+), 7 deletions(-)

diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index 44868b62b14e..abac39d020ab 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -1114,7 +1114,35 @@ static inline pte_t *vmemmap_pte_alloc(pmd_t *pmdp, int node,
 	return pte_offset_kernel(pmdp, address);
 }

+/*
+ * vmemmap is populated one memory section at a time. A section on PowerPC
+ * is 16MB, so backing its struct pages takes a 16K region of vmemmap.
+ * Without an altmap, radix__vmemmap_populate() satisfies such a request by
+ * mapping a whole PMD_SIZE leaf. One leaf holds the struct pages for
+ * PMD_SIZE / sizeof(struct page) PFNs (2GB of memory with 64K pages, 128MB
+ * with 4K pages) i.e. for many sections, including ones that aren't
+ * present yet.
+ * Return true if any part of [start, end) is already covered by such a leaf.
+ */
+static bool __meminit vmemmap_pmd_leaf_mapped(unsigned long start,
+					      unsigned long end)
+{
+	unsigned long addr;

+	for (addr = start; addr < end; addr = pmd_addr_end(addr, end)) {
+		p4d_t *p4d = p4d_offset(pgd_offset_k(addr), addr);
+		pud_t *pud;
+
+		if (p4d_none(READ_ONCE(*p4d)))
+			continue;
+		pud = pud_offset(p4d, addr);
+		if (pud_none(READ_ONCE(*pud)))
+			continue;
+		if (pmd_leaf(READ_ONCE(*pmd_offset(pud, addr))))
+			return true;
+	}
+	return false;
+}

 int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, int node,
 				      struct vmem_altmap *altmap)
@@ -1128,8 +1156,20 @@ int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, in
 	pte_t *pte;
 	unsigned long pfn = page_to_pfn((struct page *)start);

-	if (vmemmap_optimizable_order(pfn_to_section_compound_order(pfn)))
-		return vmemmap_populate_compound_pages(pfn, start, end, node, NULL);
+	if (vmemmap_optimizable_order(pfn_to_section_compound_order(pfn))) {
+		if (!vmemmap_pmd_leaf_mapped(start, end))
+			return vmemmap_populate_compound_pages(pfn, start, end, node, NULL);
+		/*
+		 * A leaf installed for a neighbouring range already backs this
+		 * section with real struct pages and it cannot be freed while
+		 * that range is present, so the shared tail layout would save
+		 * nothing. The core trusts the section's compound order to tell
+		 * which struct pages it must initialize. So, clear it so that
+		 * every struct page of this section is initialized, then reuse
+		 * the leaf like any other section does below.
+		 */
+		section_set_compound_order(__pfn_to_section(pfn), 0);
+	}
 	/*
 	 * If altmap is present, Make sure we align the start vmemmap addr
 	 * to PAGE_SIZE so that we calculate the correct start_pfn in
@@ -1260,11 +1300,14 @@ int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
 		if (!pmd)
 			return -ENOMEM;

-		if (pmd_leaf(READ_ONCE(*pmd))) {
-			/* existing huge mapping. Skip the range */
-			next = pmd_addr_end(addr, end);
-			continue;
-		}
+		/*
+		 * The struct pages behind a leaf are not the head/shared tail
+		 * layout the core expects for this section and the tails
+		 * would be left uninitialized. radix__vmemmap_populate()
+		 * calls this only when it finds vmemmap optimizable order.
+		 */
+		if (WARN_ON_ONCE(pmd_leaf(READ_ONCE(*pmd))))
+			return -EOPNOTSUPP;
 		pte = vmemmap_pte_alloc(pmd, node, addr);
 		if (!pte)
 			return -ENOMEM;
--
2.39.5

[PATCH 2/2] mm/huge_memory: set the mmu_gather page size in zap_huge_pud()

[Ritesh Harjani (IBM)]

Unmapping a device-dax mapping with 1G alignment on powerpc radix kernel
built with CONFIG_DEBUG_VM triggers this warning on every munmap:

    WARNING: mm/mmu_gather.c:176 at __tlb_remove_folio_pages_size+0x38/0x230
    NIP [c00000000063d508] __tlb_remove_folio_pages_size+0x38/0x230
    LR [c0000000006fb688] zap_huge_pud+0x1b8/0x2d0
    Call Trace:
     zap_huge_pud+0x170/0x2d0 (unreliable)
     __zap_vma_range+0x1604/0x1e20
     unmap_vmas+0xa0/0x1f0
     unmap_region+0xb0/0x130
     vms_complete_munmap_vmas+0x120/0x310
     do_vmi_align_munmap+0x1ec/0x240
     __vm_munmap+0x148/0x220
     sys_munmap+0x28/0x40

To trigger this - it is enough to mmap() /dev/dax0.0, touch the first
byte so that the whole 1G is mapped by a single PUD and then munmap()
it.

PowerPC needs MMU_GATHER_PAGE_SIZE in which mmu_gather needs to record
the page size for which tlb flush is needed. zap_huge_pmd(),
zap_pte_range(), change_huge_pud() etc all do this already. However,
since commit:
  dbe54153296d ("mm/huge_memory: add vmf_insert_folio_pud()")

a device-dax PUD maps a real folio and zap_huge_pud() now hands it to
tlb_remove_page_size(tlb, page, HPAGE_PUD_SIZE). At that point
tlb->page_size is still whatever the previous user left behind, which is
0 for the fresh gather set up by munmap() and so the check fires.

Fix this in the same way like how other similar paths does.

Fixes: dbe54153296d0 ("mm/huge_memory: add vmf_insert_folio_pud()")
Reported-by: Pavithra <pavrampu@linux.ibm.com>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
 mm/huge_memory.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index bd44d58e8684..7e589a2fb02d 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3040,6 +3040,8 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
 	spinlock_t *ptl;
 	pud_t orig_pud;

+	tlb_change_page_size(tlb, HPAGE_PUD_SIZE);
+
 	ptl = __pud_trans_huge_lock(pud, vma);
 	if (!ptl)
 		return 0;
--
2.39.5

Re: [PATCH 1/2] powerpc/64s/radix: don't dedup vmemmap covered by an existing PMD leaf

[Muchun Song]

Hi Ritesh,

> On Oct 27, 2026, at 11:22, Ritesh Harjani (IBM) <ritesh.list@gmail.com> wrote:
> 
> radix__vmemmap_populate() is asked for one 16MB section, which is 16KB of
> struct pages. With no altmap it rounds the request down to a PMD leaf.
> That leaf describes 2GB of PFNs with 64KB pages (128MB with 4KB pages),
> so it also covers sections that are not present yet. It cannot be freed
> while any of those sections are still in use.
> 
> A devdax namespace that lands in the same window is marked for vmemmap dedup.
> vmemmap_populate_compound_pages() treated the leaf as an existing huge mapping
> and skipped it. The core then initializes only the head vmemmap page and assumes
> the rest is the shared tail. Those remaining tail slots are real struct pages
> still holding the page_init_poison() pattern and are not the shared tail pages.

Right.

> __init_single_page() warns on the zone mismatch and the following pageblock init
> faults in __set_pfnblock_flags_mask().

Yeah, the new adding warning is useful.

> 
> If the section's vmemmap is already a PMD leaf, clear its compound order

I think it's enough to fix this issue.

> and reuse the leaf, so every struct page in the section is initialized.
> The shared-tail layout needs base-page PTEs, so it cannot be built on top of
> this leaf and dropping the leaf is not an option while the neighbouring range
> is still using it. So, fail the compound populate if a leaf is still seen there.
> 
> <Call Trace>
> ============
> [    3.467060][    T1] WARNING: mm/mm_init.c:600 at __init_single_page+0xd0/0xf4, CPU#0: swapper/0/1
> <...>
> [    3.477419][    T1] NIP [c000000000576b30] __init_single_page+0xd0/0xf4
> [    3.477827][    T1] LR [c0000000005778b0] __init_zone_device_page+0x30/0xb8
> [    3.478444][    T1] Call Trace:
> [    3.478764][    T1]  0x10000 (unreliable)
> [    3.479874][    T1]  memmap_init_zone_device+0x164/0x430
> [    3.480392][    T1]  memremap_pages+0x43c/0x93c
> [    3.480836][    T1]  devm_memremap_pages+0x3c/0xa4
> [    3.481328][    T1]  dev_dax_probe+0x264/0x3d0
> [    3.481817][    T1]  dax_bus_probe+0x140/0x1c8
> [    3.482237][    T1]  really_probe+0x108/0x580
> [    3.482681][    T1]  __driver_probe_device+0xcc/0x284
> [    3.483186][    T1]  driver_probe_device+0x5c/0x108
> [    3.483656][    T1]  __device_attach_driver+0x10c/0x1e8
> [    3.484134][    T1]  bus_for_each_drv+0xc0/0x13c
> <...>
> [    3.499972][    T1] BUG: Kernel NULL pointer dereference on read at 0x00001400
> [    3.500417][    T1] Faulting instruction address: 0xc0000000005f4ecc
> [    3.500417][    T1] Faulting instruction address: 0xc0000000005f4ecc
> cpu 0x0: Vector: 300 (Data Access) at [c000000006756b50]
>    pc: c0000000005f4ecc: __set_pfnblock_flags_mask.constprop.0+0x34/0x160
>    lr: c000000000577f38: memmap_init_zone_device+0x358/0x430
>    pid   = 1, comm = swapper/0
> enter ? for help
>  memmap_init_zone_device+0x358/0x430
>  memmap_init_zone_device+0x220/0x430 (unreliable)
>  memremap_pages+0x43c/0x93c
>  devm_memremap_pages+0x3c/0xa4
>  dev_dax_probe+0x264/0x3d0
>  dax_bus_probe+0x140/0x1c8
>  really_probe+0x108/0x580
> <...>
> 
> Fixes: bd0862382adb ("mm/sparse-vmemmap: drop Device DAX-specific population path")

Actually, the right Fixes is:

Fixes: f2b79c0d7968 ("powerpc/book3s64/radix: add support for vmemmap optimization for radix")

On older kernels, these errors are silent. They trigger a kernel panic when the user
actually uses Device DAX, rather than during the initialization phase. 

So, for older kernels, we should guide them to backport the patch.

> Reported-by: Pavithra <pavrampu@linux.ibm.com>
> Assisted-by: LLM
> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> ---
> Fixes tag here is added since the fix is dependent upon the change mentioned in
> "Fixes" tag. The Fixes commit-id is from mm-new branch. Although this could be
> an older pre-existing issue, but got exposed after this patch series.

I knew this issue, so I have a local commit to fix it as well. I'd like to share
it with you, because it looks a little simple, does it work for you?

powerpc/mm: fix uninitialized ZONE_DEVICE struct pages with PMDs

PowerPC radix vmemmap population can reuse an existing PMD mapping while
populating a device DAX range whose section is marked as HVO-enabled. The
reused mapping is an ordinary vmemmap mapping: the redundant tail struct
pages are not remapped to the shared tail page.

Leaving the section marked as HVO-enabled makes later ZONE_DEVICE
initialization treat the range as optimized and initialize only the struct
pages that would be backed by a real HVO mapping. The remaining tail struct
pages can then be left uninitialized even though they are still used as
metadata for the device DAX compound page.

The Fixes target predates the section-based vmemmap optimization
infrastructure, so older kernels do not have section state that can be
marked non-HVO. Backports to such kernels need to handle the PMD-reuse
case in powerpc radix code directly, for example by initializing the tail
struct pages that core ZONE_DEVICE initialization would otherwise skip.

Mark the section as non-HVO when reusing an existing PMD mapping, so the
section's HVO state matches the actual vmemmap layout and all required
struct pages are initialized.

Fixes: f2b79c0d7968 ("powerpc/book3s64/radix: add support for vmemmap optimization for radix")
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
---
 arch/powerpc/mm/book3s64/radix_pgtable.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c
index 72793ace1fb6..fc3a2d4b422d 100644
--- a/arch/powerpc/mm/book3s64/radix_pgtable.c
+++ b/arch/powerpc/mm/book3s64/radix_pgtable.c
@@ -1200,7 +1200,16 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
 			return -ENOMEM;

 		if (pmd_leaf(READ_ONCE(*pmd))) {
-			/* existing huge mapping. Skip the range */
+			unsigned long addr_pfn = page_to_pfn((struct page *)addr);
+			struct mem_section *ms = __pfn_to_section(addr_pfn);
+
+			/*
+			 * Existing PMD mappings can cover the edges of a valid
+			 * HVO populate range.  They are ordinary vmemmap mappings,
+			 * so the current section must not be treated as HVO-backed
+			 * when its struct pages are initialized later.
+			 */
+			section_set_order(ms, 0);
 			next = pmd_addr_end(addr, end);
 			continue;
 		}