* [PATCH v6 1/4] mm/rmap: duplicate a PMD mapping as one in folio_dup_file_rmap_pmd()
2026-11-19 13:44 [PATCH v6 0/4] mm/huge_memory: copy PMDs at fork like the PTE path does Konstantin Khorenko
@ 2026-11-19 13:44 ` Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 2/4] mm/huge_memory: copy file THPs at fork like copy_present_ptes() does Konstantin Khorenko
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khorenko @ 2026-11-19 13:44 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand (Arm),
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Peter Xu
Cc: Alex Williamson, kvm, Loïc Molinari, Lorenzo Stoakes,
Shuah Khan, linux-kselftest, dri-devel, Madhavan Srinivasan,
Ritesh Harjani (IBM),
linuxppc-dev, linux-mm, linux-kernel, Konstantin Khorenko,
Jason Gunthorpe, stable, Jose A. Perez de Azpillaga
folio_dup_file_rmap_pmd() passes PGTABLE_LEVEL_PTE to
__folio_dup_file_rmap() for HPAGE_PMD_NR pages, so it adds HPAGE_PMD_NR
PTE mappings to the folio instead of one PMD mapping.
The helper has no caller yet, the next patch adds the first one. Pass
PGTABLE_LEVEL_PMD, as folio_try_dup_anon_rmap_pmd() does.
Fixes: d8ef5e311d7b ("mm/rmap: convert page_dup_file_rmap() to folio_dup_file_rmap_[pte|ptes|pmd]()")
Cc: stable@vger.kernel.org
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
include/linux/rmap.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 52fff8dc74213..9f5bfa317a5cf 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -558,7 +558,7 @@ static inline void folio_dup_file_rmap_pmd(struct folio *folio,
struct page *page, struct vm_area_struct *dst_vma)
{
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
- __folio_dup_file_rmap(folio, page, HPAGE_PMD_NR, dst_vma, PGTABLE_LEVEL_PTE);
+ __folio_dup_file_rmap(folio, page, HPAGE_PMD_NR, dst_vma, PGTABLE_LEVEL_PMD);
#else
WARN_ON_ONCE(true);
#endif
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v6 2/4] mm/huge_memory: copy file THPs at fork like copy_present_ptes() does
2026-11-19 13:44 [PATCH v6 0/4] mm/huge_memory: copy PMDs at fork like the PTE path does Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 1/4] mm/rmap: duplicate a PMD mapping as one in folio_dup_file_rmap_pmd() Konstantin Khorenko
@ 2026-11-19 13:44 ` Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 3/4] mm/huge_memory: read a special PMD under the lock in copy_huge_pmd() Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 4/4] selftests/mm: test fork of PMD-mapped shmem THPs Konstantin Khorenko
3 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khorenko @ 2026-11-19 13:44 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand (Arm),
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Peter Xu
Cc: Alex Williamson, kvm, Loïc Molinari, Lorenzo Stoakes,
Shuah Khan, linux-kselftest, dri-devel, Madhavan Srinivasan,
Ritesh Harjani (IBM),
linuxppc-dev, linux-mm, linux-kernel, Konstantin Khorenko,
Jason Gunthorpe, stable, Jose A. Perez de Azpillaga
This fixes no bug: it makes fork copy a PMD the way copy_present_ptes()
copies a PTE. Until now copy_huge_pmd() skipped every folio PMD of a
non-anonymous VMA and left the child to fault them in again, while the
PTEs of the same VMA are copied.
Treat file folios as the PTE path does: take a reference, duplicate the
rmap and account them. fork then pays a reference and an rmap update
per PMD, and saves the child a fault per PMD it touches. The next patch
sends PFN mappings through the same tail.
That covers the VMAs vma_needs_copy() sends here:
- private file mappings with an anon_vma;
- VM_COPY_ON_FORK ones: uffd-wp and uffd-rwp, VM_MIXEDMAP, and
VM_MAYBE_GUARD, which shared shmem, file and DAX mappings get from a
guard region.
A folio over the page cache refcount limit is not copied, as in
copy_present_ptes(): the child faults it in, and fork() fails with
-ENOMEM only in a VM_MIXEDMAP VMA, which may have no fault handler.
Unlike a skipped PTE, the PMD needs no uffd marker: a file PMD never
carries the uffd bit, as protecting it splits it.
The DAX huge zero PMD is copied too, with a reference taken by
mm_get_huge_zero_folio() as for anonymous VMAs. A migration entry of a
file folio is copied and accounted like copy_nonpresent_pte() does, and
made read-only in both processes only in a CoW mapping. A device
private entry gets the same CoW check, as in copy_nonpresent_pte().
The copy then gets the same tail as __copy_present_ptes(): write
protect both sides in a CoW mapping, mark the child's copy clean in a
shared one, and drop the uffd-wp bit where the child is not armed. An
anonymous THP is now write protected only when its PMD is writable, as
a PTE is.
A page table is deposited under the copy where the fault paths deposit
one: in anonymous VMAs, and wherever the architecture needs one.
Cc: stable@vger.kernel.org # needed by the next patch
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
mm/huge_memory.c | 74 ++++++++++++++++++++++++++++++++++++++------------------
1 file changed, 50 insertions(+), 24 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 641e5a3822e3a..277d27a8a4364 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1913,12 +1913,13 @@ static void copy_huge_non_present_pmd(
pmd_t pmd, pgtable_t pgtable)
{
softleaf_t entry = softleaf_from_pmd(pmd);
- struct folio *src_folio;
+ struct folio *src_folio = softleaf_to_folio(entry);
VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(pmd));
- if (softleaf_is_migration_write(entry) ||
- softleaf_is_migration_read_exclusive(entry)) {
+ if ((softleaf_is_migration_write(entry) ||
+ softleaf_is_migration_read_exclusive(entry)) &&
+ vma_is_cow_mapping(src_vma)) {
entry = make_readable_migration_entry(swp_offset(entry));
pmd = softleaf_to_pmd(entry);
if (pmd_swp_soft_dirty(*src_pmd))
@@ -1931,7 +1932,8 @@ static void copy_huge_non_present_pmd(
* For device private entries, since there are no
* read exclusive entries, writable = !readable
*/
- if (softleaf_is_device_private_write(entry)) {
+ if (softleaf_is_device_private_write(entry) &&
+ vma_is_cow_mapping(src_vma)) {
entry = make_readable_device_private_entry(swp_offset(entry));
pmd = softleaf_to_pmd(entry);
@@ -1942,7 +1944,6 @@ static void copy_huge_non_present_pmd(
set_pmd_at(src_mm, addr, src_pmd, pmd);
}
- src_folio = softleaf_to_folio(entry);
VM_WARN_ON(!folio_test_large(src_folio));
folio_get(src_folio);
@@ -1954,9 +1955,11 @@ static void copy_huge_non_present_pmd(
dst_vma, src_vma);
}
- add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR);
- mm_inc_nr_ptes(dst_mm);
- pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable);
+ add_mm_counter(dst_mm, mm_counter(src_folio), HPAGE_PMD_NR);
+ if (pgtable) {
+ mm_inc_nr_ptes(dst_mm);
+ pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable);
+ }
if (!userfaultfd_protected(dst_vma))
pmd = pmd_swp_clear_uffd(pmd);
set_pmd_at(dst_mm, addr, dst_pmd, pmd);
@@ -1972,6 +1975,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
pmd_t pmd;
pgtable_t pgtable = NULL;
int ret = -ENOMEM;
+ bool writable;
pmd = pmdp_get_lockless(src_pmd);
if (unlikely(pmd_present(pmd) && pmd_special(pmd) &&
@@ -1992,13 +1996,11 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
goto set_pmd;
}
- /* Skip if can be re-fill on fault */
- if (!vma_is_anonymous(dst_vma))
- return 0;
-
- pgtable = pte_alloc_one(dst_mm);
- if (unlikely(!pgtable))
- goto out;
+ if (vma_is_anonymous(dst_vma) || arch_needs_pgtable_deposit()) {
+ pgtable = pte_alloc_one(dst_mm);
+ if (unlikely(!pgtable))
+ goto out;
+ }
dst_ptl = pmd_lock(dst_mm, dst_pmd);
src_ptl = pmd_lockptr(src_mm, src_pmd);
@@ -2016,7 +2018,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
}
if (unlikely(!pmd_trans_huge(pmd))) {
- pte_free(dst_mm, pgtable);
+ if (pgtable)
+ pte_free(dst_mm, pgtable);
goto out_unlock;
}
/*
@@ -2031,13 +2034,30 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
* copy. It just takes a reference.
*/
mm_get_huge_zero_folio(dst_mm);
- goto out_zero_page;
+ goto copy_pmd;
}
src_page = pmd_page(pmd);
VM_BUG_ON_PAGE(!PageHead(src_page), src_page);
src_folio = page_folio(src_page);
+ if (!folio_test_anon(src_folio)) {
+ if (unlikely(folio_ref_count_over_limit(src_folio))) {
+ /*
+ * VM_MIXEDMAP may contain driver-inserted pages with no
+ * fault handler to reconstruct the mapping.
+ */
+ ret = (dst_vma->vm_flags & VM_MIXEDMAP) ? -ENOMEM : 0;
+ if (pgtable)
+ pte_free(dst_mm, pgtable);
+ goto out_unlock;
+ }
+ folio_get(src_folio);
+ folio_dup_file_rmap_pmd(src_folio, src_page, dst_vma);
+ add_mm_counter(dst_mm, mm_counter_file(src_folio),
+ HPAGE_PMD_NR);
+ goto copy_pmd;
+ }
folio_get(src_folio);
if (unlikely(folio_try_dup_anon_rmap_pmd(src_folio, src_page, dst_vma, src_vma))) {
/* Page maybe pinned: split and retry the fault on PTEs. */
@@ -2049,19 +2069,25 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
return -EAGAIN;
}
add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR);
-out_zero_page:
- mm_inc_nr_ptes(dst_mm);
- pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable);
+copy_pmd:
+ if (pgtable) {
+ mm_inc_nr_ptes(dst_mm);
+ pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable);
+ }
- /* See __copy_present_ptes(): restore accessible protection. */
+ /* See __copy_present_ptes(). */
+ writable = pmd_write(pmd);
if (!userfaultfd_protected(dst_vma)) {
if (userfaultfd_rwp(src_vma) && pmd_uffd(pmd))
pmd = pmd_modify(pmd, dst_vma->vm_page_prot);
pmd = pmd_clear_uffd(pmd);
}
-
- pmdp_set_wrprotect(src_mm, addr, src_pmd);
- pmd = pmd_wrprotect(pmd);
+ if (vma_is_cow_mapping(src_vma) && writable) {
+ pmdp_set_wrprotect(src_mm, addr, src_pmd);
+ pmd = pmd_wrprotect(pmd);
+ }
+ if (src_vma->vm_flags & VM_SHARED)
+ pmd = pmd_mkclean(pmd);
set_pmd:
pmd = pmd_mkold(pmd);
set_pmd_at(dst_mm, addr, dst_pmd, pmd);
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v6 3/4] mm/huge_memory: read a special PMD under the lock in copy_huge_pmd()
2026-11-19 13:44 [PATCH v6 0/4] mm/huge_memory: copy PMDs at fork like the PTE path does Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 1/4] mm/rmap: duplicate a PMD mapping as one in folio_dup_file_rmap_pmd() Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 2/4] mm/huge_memory: copy file THPs at fork like copy_present_ptes() does Konstantin Khorenko
@ 2026-11-19 13:44 ` Konstantin Khorenko
2026-11-19 13:44 ` [PATCH v6 4/4] selftests/mm: test fork of PMD-mapped shmem THPs Konstantin Khorenko
3 siblings, 0 replies; 6+ messages in thread
From: Konstantin Khorenko @ 2026-11-19 13:44 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand (Arm),
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Peter Xu
Cc: Alex Williamson, kvm, Loïc Molinari, Lorenzo Stoakes,
Shuah Khan, linux-kselftest, dri-devel, Madhavan Srinivasan,
Ritesh Harjani (IBM),
linuxppc-dev, linux-mm, linux-kernel, Konstantin Khorenko,
Jason Gunthorpe, stable, Sashiko, Jose A. Perez de Azpillaga
copy_huge_pmd() reads a special PMD without the page table lock and
copies that value to the child. The mmap write lock held by fork does
not stop unmap_mapping_range(): it zaps under i_mmap_rwsem and the PMD
lock.
The zap visits the parent VMA before the child's, as dup_mmap() links
the child right after the parent in i_mmap. If it clears the parent PMD
after fork has read it, and passes the child before fork copies it, the
child keeps a mapping the driver has revoked, for example:
- vfio-pci: the child reaches the BAR after the device memory was
disabled or the device was reset, instead of getting SIGBUS;
- panthor: the child maps the pages of an evicted buffer object after
the driver has released them.
The same branch deposits no page table under the copy. ppc64 in hash
mode keeps the hash slot array of every huge PMD in that table, so the
child dereferences a NULL table on the first hash fault, in
__hash_page_thp(), and on the zap, in hash__pmdp_huge_get_and_clear().
Take the locks first and let vm_normal_page_pmd() tell a PFN mapping
from a folio under them, as copy_present_ptes() does with
vm_normal_page(). The zap clears the parent PMD under the same lock, so
either fork finds it none and skips it, or the copy is in place before
the zap reaches the child.
The copy of a PFN mapping then goes through the same tail as a folio:
- it gets a page table where the architecture needs one;
- in a shared mapping it is marked clean in the child, as a PTE is;
- in a CoW mapping, which only VM_MIXEDMAP allows, both sides are write
protected instead of a warning being printed.
Fixes: bc02afbd4d73 ("mm/fork: accept huge pfnmap entries")
Fixes: d1503aa9ab80 ("powerpc/64s: Add support for huge pfnmaps")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20261014-b4-prep-vstor-149847-copy_huge_pmd-recheck-special-pmd-v1-1-d225c60bc316@virtuozzo.com?part=1
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Tested-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> #PowerPC
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
mm/huge_memory.c | 24 +++---------------------
1 file changed, 3 insertions(+), 21 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 277d27a8a4364..adf3ca088bf41 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1977,25 +1977,6 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
int ret = -ENOMEM;
bool writable;
- pmd = pmdp_get_lockless(src_pmd);
- if (unlikely(pmd_present(pmd) && pmd_special(pmd) &&
- !is_huge_zero_pmd(pmd))) {
- dst_ptl = pmd_lock(dst_mm, dst_pmd);
- src_ptl = pmd_lockptr(src_mm, src_pmd);
- spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING);
- /*
- * No need to recheck the pmd, it can't change with write
- * mmap lock held here.
- *
- * Meanwhile, making sure it's not a CoW VMA with writable
- * mapping, otherwise it means either the anon page wrongly
- * applied special bit, or we made the PRIVATE mapping be
- * able to wrongly write to the backend MMIO.
- */
- VM_WARN_ON_ONCE(vma_is_cow_mapping(src_vma) && pmd_write(pmd));
- goto set_pmd;
- }
-
if (vma_is_anonymous(dst_vma) || arch_needs_pgtable_deposit()) {
pgtable = pte_alloc_one(dst_mm);
if (unlikely(!pgtable))
@@ -2037,7 +2018,9 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
goto copy_pmd;
}
- src_page = pmd_page(pmd);
+ src_page = vm_normal_page_pmd(src_vma, addr, pmd);
+ if (!src_page)
+ goto copy_pmd;
VM_BUG_ON_PAGE(!PageHead(src_page), src_page);
src_folio = page_folio(src_page);
@@ -2088,7 +2071,6 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
}
if (src_vma->vm_flags & VM_SHARED)
pmd = pmd_mkclean(pmd);
-set_pmd:
pmd = pmd_mkold(pmd);
set_pmd_at(dst_mm, addr, dst_pmd, pmd);
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v6 4/4] selftests/mm: test fork of PMD-mapped shmem THPs
2026-11-19 13:44 [PATCH v6 0/4] mm/huge_memory: copy PMDs at fork like the PTE path does Konstantin Khorenko
` (2 preceding siblings ...)
2026-11-19 13:44 ` [PATCH v6 3/4] mm/huge_memory: read a special PMD under the lock in copy_huge_pmd() Konstantin Khorenko
@ 2026-11-19 13:44 ` Konstantin Khorenko
2026-11-19 18:32 ` Andrew Morton
3 siblings, 1 reply; 6+ messages in thread
From: Konstantin Khorenko @ 2026-11-19 13:44 UTC (permalink / raw)
To: Andrew Morton, David Hildenbrand (Arm),
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Peter Xu
Cc: Alex Williamson, kvm, Loïc Molinari, Lorenzo Stoakes,
Shuah Khan, linux-kselftest, dri-devel, Madhavan Srinivasan,
Ritesh Harjani (IBM),
linuxppc-dev, linux-mm, linux-kernel, Konstantin Khorenko,
Jason Gunthorpe
Fork a process that maps shmem THPs with PMDs, in a VMA whose page
tables fork copies: a private mapping with an anon_vma, and a shared
one with a guard region. The child must find the PMDs before it
touches the mapping, and its exit must leave the parent's mapping and
the system-wide ShmemPmdMapped counter as they were.
With folio_dup_file_rmap_pmd() adding PTE mappings, the child's exit
takes the parent's share off ShmemPmdMapped while the parent still maps
the folios. Unmapping them drops it once more, so every mapping stays
until exit, where the kernel reports "Bad page cache", or hits a
VM_BUG_ON with CONFIG_DEBUG_VM. A kernel that leaves file THPs to a
refault in the child skips the test.
ShmemPmdMapped is system-wide, and reclaim or NUMA balancing can take a
PMD from the parent while the test runs. So the test makes up to three
attempts, each on a fresh file, and fails only if all of them see the
counter drop by exactly the parent's share: the bug does that on every
fork, other processes only by chance. It passes on the first clean
attempt and skips when none is clean.
The test mounts a tmpfs with huge=always over /tmp in a private mount
namespace, so it leaves the THP settings and the host's mounts and
files alone.
Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com>
---
MAINTAINERS | 1 +
tools/testing/selftests/mm/Makefile | 1 +
tools/testing/selftests/mm/fork_file_thp.c | 256 +++++++++++++++++++++++++++++
tools/testing/selftests/mm/run_vmtests.sh | 1 +
4 files changed, 259 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index d738990161bd0..7ade132e3d3ab 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17941,6 +17941,7 @@ F: mm/huge_memory.c
F: mm/khugepaged.c
F: mm/mm_slot.h
F: tools/mm/thpmaps
+F: tools/testing/selftests/mm/fork_file_thp.c
F: tools/testing/selftests/mm/khugepaged.c
F: tools/testing/selftests/mm/split_huge_page_test.c
F: tools/testing/selftests/mm/transhuge-stress.c
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 0ea72e9979347..2489027b2a36b 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -108,6 +108,7 @@ TEST_GEN_FILES += folio_split_race_test
TEST_GEN_FILES += folio_order_check
TEST_GEN_FILES += khugepaged_sync_check
TEST_GEN_FILES += khugepaged_race
+TEST_GEN_FILES += fork_file_thp
TEST_GEN_FILES += soft-dirty
ifeq ($(ARCH),x86_64)
diff --git a/tools/testing/selftests/mm/fork_file_thp.c b/tools/testing/selftests/mm/fork_file_thp.c
new file mode 100644
index 0000000000000..aaecf7ba6f406
--- /dev/null
+++ b/tools/testing/selftests/mm/fork_file_thp.c
@@ -0,0 +1,256 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * fork() of a process that maps shmem THPs with PMDs, in a VMA whose page
+ * tables fork copies: a private mapping with an anon_vma, and a shared
+ * mapping with a guard region.
+ *
+ * The child must get the PMDs without faulting, and its exit must leave the
+ * parent's mapping and the system-wide ShmemPmdMapped counter as they were.
+ * Kernels that leave file THPs to a refault in the child skip the test.
+ */
+#define _GNU_SOURCE
+#include <asm-generic/mman.h> /* Force the import of the tools version. */
+#include <errno.h>
+#include <fcntl.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sched.h>
+#include <sys/mman.h>
+#include <sys/mount.h>
+#include <sys/wait.h>
+
+#include "kselftest.h"
+#include "vm_util.h"
+
+#define NR_PMDS 4
+#define MAX_PMDSIZE (32UL << 20)
+/* Other processes, reclaim and NUMA balancing can disturb an attempt. */
+#define ATTEMPTS 3
+
+static size_t pagesize, pmdsize, size;
+
+static long meminfo_kb(const char *key)
+{
+ char line[256];
+ long kb = -1;
+ FILE *f = fopen("/proc/meminfo", "r");
+
+ if (!f)
+ return -1;
+ while (fgets(line, sizeof(line), f))
+ if (!strncmp(line, key, strlen(key))) {
+ kb = strtol(line + strlen(key), NULL, 10);
+ break;
+ }
+ fclose(f);
+ return kb;
+}
+
+/* Skips the last page, which may be a guard page, and @skip. */
+static bool data_ok(const char *mem, size_t skip)
+{
+ size_t off;
+
+ for (off = 0; off < size - pagesize; off += pagesize)
+ if (off != skip && mem[off] != (char)(off / pmdsize + 1))
+ return false;
+ return true;
+}
+
+/* The last PMD is zapped by the setup, the others must be PMD-mapped. */
+static bool pmds_ok(char *mem)
+{
+ return check_huge_file(mem, size - pmdsize, NR_PMDS - 1, pmdsize);
+}
+
+/* Maps a fresh shmem file of NR_PMDS PMDs, PMD-aligned, until exit. */
+static char *map_file(bool shared)
+{
+ char *area, *buf, *mem;
+ size_t off;
+ int fd;
+
+ fd = open("/tmp/fork_file_thp", O_RDWR | O_CREAT | O_TRUNC, 0600);
+ if (fd < 0)
+ ksft_exit_fail_perror("open");
+ unlink("/tmp/fork_file_thp");
+ buf = malloc(pmdsize);
+ if (!buf)
+ ksft_exit_fail_msg("malloc failed\n");
+ for (off = 0; off < size; off += pmdsize) {
+ memset(buf, off / pmdsize + 1, pmdsize);
+ if (pwrite(fd, buf, pmdsize, off) != (ssize_t)pmdsize)
+ ksft_exit_fail_perror("pwrite");
+ }
+ free(buf);
+
+ area = mmap(NULL, size + pmdsize, PROT_NONE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+ if (area == MAP_FAILED)
+ ksft_exit_fail_perror("mmap");
+ mem = (char *)(((unsigned long)area + pmdsize - 1) & ~(pmdsize - 1));
+ mem = mmap(mem, size, PROT_READ | PROT_WRITE,
+ (shared ? MAP_SHARED : MAP_PRIVATE) | MAP_FIXED, fd, 0);
+ if (mem == MAP_FAILED)
+ ksft_exit_fail_perror("mmap");
+ close(fd);
+ for (off = 0; off < size; off += pmdsize)
+ FORCE_READ(mem[off]);
+ return mem;
+}
+
+/* Child exit codes, apart from what kselftest helpers exit with. */
+enum result { PASS, NO_PMDS = 10, FAIL_DATA, FAIL_COUNTER, FAIL_PARENT,
+ FAIL_KILLED, COUNTER_OFF, CHANGED, LOST_PMDS, NO_THP };
+
+static enum result fork_once(char *mem, bool shared, long *before, long *after)
+{
+ /* The parent alone maps this much with PMDs. */
+ long own = (NR_PMDS - 1) * (pmdsize >> 10);
+ bool parent_pmds;
+ int status;
+ pid_t pid;
+
+ *before = *after = meminfo_kb("ShmemPmdMapped:");
+ if (*before < own)
+ return COUNTER_OFF;
+ fflush(stdout);
+ pid = fork();
+ if (pid < 0)
+ ksft_exit_fail_perror("fork");
+ if (!pid) {
+ /* Look before touching anything: the PMDs must be there. */
+ if (!pmds_ok(mem))
+ _exit(NO_PMDS);
+ if (!data_ok(mem, pmdsize))
+ _exit(FAIL_DATA);
+ mem[pmdsize] = 'c';
+ _exit(PASS);
+ }
+ if (waitpid(pid, &status, 0) != pid)
+ ksft_exit_fail_perror("waitpid");
+ *after = meminfo_kb("ShmemPmdMapped:");
+
+ if (!WIFEXITED(status))
+ return FAIL_KILLED;
+ /* Reclaim or NUMA balancing took a PMD from the parent. */
+ parent_pmds = pmds_ok(mem);
+ if (WEXITSTATUS(status) == NO_PMDS && !parent_pmds)
+ return LOST_PMDS;
+ if (WEXITSTATUS(status) != PASS)
+ return WEXITSTATUS(status);
+ if (mem[pmdsize] != (shared ? 'c' : 2) || !data_ok(mem, pmdsize))
+ return FAIL_PARENT;
+ if (!parent_pmds)
+ return LOST_PMDS;
+ /* The child's exit took the parent's share off the counter. */
+ if (*before - *after == own)
+ return FAIL_COUNTER;
+ if (*after != *before)
+ return CHANGED;
+ return PASS;
+}
+
+static void test_fork(bool shared)
+{
+ const char *desc = shared ? "shared mapping with a guard region" :
+ "private mapping with an anon_vma";
+ long before = 0, after = 0;
+ enum result res = NO_THP;
+ int i, drops = 0;
+ char *mem;
+
+ for (i = 0; i < ATTEMPTS; i++) {
+ mem = map_file(shared);
+ /* Make fork copy the page tables; the last PMD gets zapped. */
+ if (shared) {
+ if (madvise(mem + size - pagesize, pagesize,
+ MADV_GUARD_INSTALL)) {
+ ksft_test_result_skip("%s: MADV_GUARD_INSTALL: %s\n",
+ desc, strerror(errno));
+ return;
+ }
+ } else {
+ mem[size - pagesize] = 'p';
+ }
+
+ res = pmds_ok(mem) ? fork_once(mem, shared, &before, &after) :
+ res == NO_THP ? NO_THP : LOST_PMDS;
+ /* Later children get only their own mapping. */
+ madvise(mem, size, MADV_DONTFORK);
+ if (res == FAIL_COUNTER)
+ drops++;
+ else if (res != CHANGED && res != LOST_PMDS && res != NO_THP)
+ break;
+ }
+ /* Other processes can mimic the bug's drop, but not on every attempt. */
+ if (res == FAIL_COUNTER && drops < ATTEMPTS)
+ res = CHANGED;
+
+ switch (res) {
+ case PASS:
+ ksft_test_result_pass("%s\n", desc);
+ break;
+ case NO_THP:
+ ksft_test_result_skip("%s: shmem is not mapped with PMDs\n",
+ desc);
+ break;
+ case NO_PMDS:
+ ksft_test_result_skip("%s: fork does not copy file THPs\n", desc);
+ break;
+ case COUNTER_OFF:
+ ksft_test_result_skip("%s: ShmemPmdMapped %ld kB is below what this process maps\n",
+ desc, before);
+ break;
+ case CHANGED:
+ ksft_test_result_skip("%s: ShmemPmdMapped keeps changing\n", desc);
+ break;
+ case LOST_PMDS:
+ ksft_test_result_skip("%s: the parent keeps losing its PMDs\n",
+ desc);
+ break;
+ case FAIL_DATA:
+ ksft_test_result_fail("%s: child sees wrong data\n", desc);
+ break;
+ case FAIL_COUNTER:
+ ksft_test_result_fail("%s: ShmemPmdMapped %ld kB before fork, %ld kB after\n",
+ desc, before, after);
+ break;
+ case FAIL_PARENT:
+ ksft_test_result_fail("%s: parent's mapping changed\n", desc);
+ break;
+ case FAIL_KILLED:
+ ksft_test_result_fail("%s: child killed by a signal\n", desc);
+ break;
+ default:
+ ksft_test_result_fail("%s: child error %d\n", desc, res);
+ break;
+ }
+}
+
+int main(void)
+{
+ ksft_print_header();
+ pagesize = getpagesize();
+ pmdsize = read_pmd_pagesize();
+ if (!pmdsize)
+ ksft_exit_skip("THP is not supported\n");
+ if (pmdsize > MAX_PMDSIZE)
+ ksft_exit_skip("PMD size %zu MiB is too large\n", pmdsize >> 20);
+ size = NR_PMDS * pmdsize;
+ if (geteuid())
+ ksft_exit_skip("needs root to mount tmpfs\n");
+ /* The tmpfs over /tmp lives and dies with a private mount namespace. */
+ if (unshare(CLONE_NEWNS) || mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL))
+ ksft_exit_skip("private mount namespace: %s\n", strerror(errno));
+ if (mount("tmpfs", "/tmp", "tmpfs", 0, "huge=always"))
+ ksft_exit_skip("tmpfs with huge=always: %s\n", strerror(errno));
+
+ ksft_set_plan(2);
+ test_fork(false);
+ test_fork(true);
+ ksft_finished();
+}
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index a1b45a3dedaee..790a6b6746465 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -433,6 +433,7 @@ fi
CATEGORY="thp" run_test ./transhuge-stress -d 20
CATEGORY="thp" run_test ./folio_split_race_test
+CATEGORY="thp" run_test ./fork_file_thp
CATEGORY="migration" run_test ./migration
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v6 4/4] selftests/mm: test fork of PMD-mapped shmem THPs
2026-11-19 13:44 ` [PATCH v6 4/4] selftests/mm: test fork of PMD-mapped shmem THPs Konstantin Khorenko
@ 2026-11-19 18:32 ` Andrew Morton
0 siblings, 0 replies; 6+ messages in thread
From: Andrew Morton @ 2026-11-19 18:32 UTC (permalink / raw)
To: Konstantin Khorenko
Cc: David Hildenbrand (Arm),
Zi Yan, Baolin Wang, Liam R. Howlett, Nico Pache, Ryan Roberts,
Dev Jain, Barry Song, Lance Yang, Usama Arif, Kiryl Shutsemau,
Peter Xu, Alex Williamson, kvm, Loïc Molinari,
Lorenzo Stoakes, Shuah Khan, linux-kselftest, dri-devel,
Madhavan Srinivasan, Ritesh Harjani (IBM),
linuxppc-dev, linux-mm, linux-kernel, Jason Gunthorpe,
Yeoreum Yun
On Thu, 19 Nov 2026 14:44:05 +0100 Konstantin Khorenko <khorenko@virtuozzo.com> wrote:
> Fork a process that maps shmem THPs with PMDs, in a VMA whose page
> tables fork copies: a private mapping with an anon_vma, and a shared
> one with a guard region. The child must find the PMDs before it
> touches the mapping, and its exit must leave the parent's mapping and
> the system-wide ShmemPmdMapped counter as they were.
>
> With folio_dup_file_rmap_pmd() adding PTE mappings, the child's exit
> takes the parent's share off ShmemPmdMapped while the parent still maps
> the folios. Unmapping them drops it once more, so every mapping stays
> until exit, where the kernel reports "Bad page cache", or hits a
> VM_BUG_ON with CONFIG_DEBUG_VM. A kernel that leaves file THPs to a
> refault in the child skips the test.
>
> ShmemPmdMapped is system-wide, and reclaim or NUMA balancing can take a
> PMD from the parent while the test runs. So the test makes up to three
> attempts, each on a fresh file, and fails only if all of them see the
> counter drop by exactly the parent's share: the bug does that on every
> fork, other processes only by chance. It passes on the first clean
> attempt and skips when none is clean.
>
> The test mounts a tmpfs with huge=always over /tmp in a private mount
> namespace, so it leaves the THP settings and the host's mounts and
> files alone.
>
> ...
>
> +static void test_fork(bool shared)
> +{
> + const char *desc = shared ? "shared mapping with a guard region" :
> + "private mapping with an anon_vma";
> + long before = 0, after = 0;
> + enum result res = NO_THP;
> + int i, drops = 0;
> + char *mem;
> +
> + for (i = 0; i < ATTEMPTS; i++) {
> + mem = map_file(shared);
> + /* Make fork copy the page tables; the last PMD gets zapped. */
> + if (shared) {
> + if (madvise(mem + size - pagesize, pagesize,
> + MADV_GUARD_INSTALL)) {
> + ksft_test_result_skip("%s: MADV_GUARD_INSTALL: %s\n",
> + desc, strerror(errno));
> + return;
> + }
> + } else {
> + mem[size - pagesize] = 'p';
> + }
> +
> + res = pmds_ok(mem) ? fork_once(mem, shared, &before, &after) :
> + res == NO_THP ? NO_THP : LOST_PMDS;
> + /* Later children get only their own mapping. */
> + madvise(mem, size, MADV_DONTFORK);
autobot want the return value checked:
: If this fails, the next child inherits an earlier test mapping too.
: Its exit can then change ShmemPmdMapped by more than the expected
: amount, causing the test to skip a real failure.
which seems ignorable - MADV_DONTFORK failures are super unlikely at present.
>
> ...
>
> +int main(void)
> +{
> + ksft_print_header();
> + pagesize = getpagesize();
> + pmdsize = read_pmd_pagesize();
This is going to ruin Mark's day again, thanks to "kselftest: mm:
rename read_pmd_pagesize() to pmd_psize()".
I think I'll alter Yeoreum's above patch like this (presently untested):
--- a/tools/testing/selftests/mm/vm_util.h~kselftest-mm-rename-read_pmd_pagesize-to-pmd_psize-fix
+++ a/tools/testing/selftests/mm/vm_util.h
@@ -93,6 +93,8 @@ unsigned long pagemap_get_pfn(int fd, ch
void clear_softdirty(void);
bool check_for_pattern(FILE *fp, const char *pattern, char *buf, size_t len);
uint64_t pmd_psize(void);
+/* temporary cross-tree compatibility thing: */
+#define read_pmd_pagesize() pmd_psize()
unsigned long rss_anon(void);
bool check_huge_anon(void *addr, size_t len, int nr_hpages, uint64_t hpage_size);
bool check_huge_file(void *addr, size_t len, int nr_hpages, uint64_t hpage_size);
_
and we can clean that up later.
^ permalink raw reply [flat|nested] 6+ messages in thread