[PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[Kelvin Zhang]

A PMD-mapped THP can contain subpages with different mapcounts. For
example, after fork(), the child can unmap only part of the THP. The
parent retains its PMD mapping, while the remaining child mappings
are PTE mappings.

pagemap_pmd_range_thp() passes the same initial page to
__folio_page_mapped_exclusively() for every pagemap entry. With
CONFIG_PAGE_MAPCOUNT=y, this applies the first subpage's exclusivity
to all entries, falsely setting or clearing PM_MMAP_EXCLUSIVE for
subpages with different sharing.

Pass page + idx to check the subpage corresponding to each entry, as
was done before converting to __folio_page_mapped_exclusively(). The
index already accounts for reads starting partway through the PMD.
The conservative whole-folio check without CONFIG_PAGE_MAPCOUNT is
unchanged.

Fixes: eb16876971ea ("fs/proc/task_mmu: remove per-page mapcount dependency for PM_MMAP_EXCLUSIVE (CONFIG_NO_PAGE_MAPCOUNT)")
Cc: stable@vger.kernel.org # 6.15+
Signed-off-by: Kelvin Zhang <zhangxp1998@gmail.com>
---
Tested on x86-64 under QEMU/KVM with CONFIG_TRANSPARENT_HUGEPAGE=y
and CONFIG_PAGE_MAPCOUNT=y, using the same test binary before and after
this change. After fork(), the child unmaps the first, second, middle,
or last 4 KiB subpage, while the parent retains a PMD mapping. Both
bulk pagemap reads and reads starting inside the PMD are checked.

Across three repetitions (12 cases), the unmodified kernel fails 24 of
84 assertions; with this change all 84 pass. Fully private and fully
shared PMD controls pass before and after the fix.

 fs/proc/task_mmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index b87d3e86442d..da4319d972e4 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -2038,7 +2038,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigned long addr,
 		pagemap_entry_t pme;
 
 		if (folio && (flags & PM_PRESENT) &&
-		    __folio_page_mapped_exclusively(folio, page))
+		    __folio_page_mapped_exclusively(folio, page + idx))
 			cur_flags |= PM_MMAP_EXCLUSIVE;
 
 		pme = make_pme(frame, cur_flags);

base-commit: 64703f0fee7421f0ddfcbd8ee1bfa53b25ffc4ca
-- 
2.53.0

Re: [PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[David Hildenbrand (Arm)]

On 11/1/26 18:01, Kelvin Zhang wrote:
> A PMD-mapped THP can contain subpages with different mapcounts. For
> example, after fork(), the child can unmap only part of the THP. The
> parent retains its PMD mapping, while the remaining child mappings
> are PTE mappings.
> 
> pagemap_pmd_range_thp() passes the same initial page to
> __folio_page_mapped_exclusively() for every pagemap entry. With
> CONFIG_PAGE_MAPCOUNT=y, this applies the first subpage's exclusivity
> to all entries, falsely setting or clearing PM_MMAP_EXCLUSIVE for
> subpages with different sharing.
> 
> Pass page + idx to check the subpage corresponding to each entry, as
> was done before converting to __folio_page_mapped_exclusively(). The
> index already accounts for reads starting partway through the PMD.
> The conservative whole-folio check without CONFIG_PAGE_MAPCOUNT is
> unchanged.
> 
> Fixes: eb16876971ea ("fs/proc/task_mmu: remove per-page mapcount dependency for PM_MMAP_EXCLUSIVE (CONFIG_NO_PAGE_MAPCOUNT)")
> Cc: stable@vger.kernel.org # 6.15+
> Signed-off-by: Kelvin Zhang <zhangxp1998@gmail.com>
> ---
> Tested on x86-64 under QEMU/KVM with CONFIG_TRANSPARENT_HUGEPAGE=y
> and CONFIG_PAGE_MAPCOUNT=y, using the same test binary before and after
> this change. After fork(), the child unmaps the first, second, middle,
> or last 4 KiB subpage, while the parent retains a PMD mapping. Both
> bulk pagemap reads and reads starting inside the PMD are checked.
> 
> Across three repetitions (12 cases), the unmodified kernel fails 24 of
> 84 assertions; with this change all 84 pass. Fully private and fully
> shared PMD controls pass before and after the fix.
> 

How was this found. Through an LLM or because there was actually a problem in
practice?

Why do you care about this information to be reliable.

If the answer is "I don't care, I don't care, an LLM reported so I blindly
submit a fix" we might just perform the switch to the CONFIG_NO_PAGE_MAPCOUNT
variant directly.


https://lore.kernel.org/linux-mm/20260412-mapcount-v1-4-05e8dfab52e0@kernel.org/

-- 
Cheers,

David

Re: [PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[David Hildenbrand (Arm)]

On 11/1/26 19:07, David Hildenbrand (Arm) wrote:
> On 11/1/26 18:01, Kelvin Zhang wrote:
>> A PMD-mapped THP can contain subpages with different mapcounts. For
>> example, after fork(), the child can unmap only part of the THP. The
>> parent retains its PMD mapping, while the remaining child mappings
>> are PTE mappings.
>>
>> pagemap_pmd_range_thp() passes the same initial page to
>> __folio_page_mapped_exclusively() for every pagemap entry. With
>> CONFIG_PAGE_MAPCOUNT=y, this applies the first subpage's exclusivity
>> to all entries, falsely setting or clearing PM_MMAP_EXCLUSIVE for
>> subpages with different sharing.
>>
>> Pass page + idx to check the subpage corresponding to each entry, as
>> was done before converting to __folio_page_mapped_exclusively(). The
>> index already accounts for reads starting partway through the PMD.
>> The conservative whole-folio check without CONFIG_PAGE_MAPCOUNT is
>> unchanged.
>>
>> Fixes: eb16876971ea ("fs/proc/task_mmu: remove per-page mapcount dependency for PM_MMAP_EXCLUSIVE (CONFIG_NO_PAGE_MAPCOUNT)")
>> Cc: stable@vger.kernel.org # 6.15+
>> Signed-off-by: Kelvin Zhang <zhangxp1998@gmail.com>
>> ---
>> Tested on x86-64 under QEMU/KVM with CONFIG_TRANSPARENT_HUGEPAGE=y
>> and CONFIG_PAGE_MAPCOUNT=y, using the same test binary before and after
>> this change. After fork(), the child unmaps the first, second, middle,
>> or last 4 KiB subpage, while the parent retains a PMD mapping. Both
>> bulk pagemap reads and reads starting inside the PMD are checked.
>>
>> Across three repetitions (12 cases), the unmodified kernel fails 24 of
>> 84 assertions; with this change all 84 pass. Fully private and fully
>> shared PMD controls pass before and after the fix.
>>
> 
> How was this found. Through an LLM or because there was actually a problem in
> practice?
> 
> Why do you care about this information to be reliable.
> 
> If the answer is "I don't care, I don't care, an LLM reported so I blindly
> submit a fix" we might just perform the switch to the CONFIG_NO_PAGE_MAPCOUNT
> variant directly.
> 
> 
> https://lore.kernel.org/linux-mm/20260412-mapcount-v1-4-05e8dfab52e0@kernel.org/
Sent my mail a bit to soon: the reason is that this might be one of the
best-case scenarios: we broke something by accident, nobody cared for various
releases; so now we can just change it.

Thanks for the report!

-- 
Cheers,

David

Re: [PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[Kelvin Zhang]

[-- Attachment #1: Type: text/plain, Size: 1811 bytes --]

Hi David,

Thanks for the clarification.

I don't have a real application depending on precise per-subpage bit 56
values, or a production failure to report. This is a review/test-driven
report, with LLM assistance, rather than something found through a failing
workload. I followed it up with an actual old/new kernel comparison.

So I have no workload-based objection to switching to the conservative
whole-folio behavior you described. The attached kselftest-style reproducer
is just to make the current CONFIG_PAGE_MAPCOUNT=y behavior reproducible,
not to argue that exact subpage reporting must be preserved as an ABI.

The test forks with a PMD-mapped anonymous THP, then has the child unmap
one base page while the parent retains its PMD mapping. On the unpatched
kernel, unmapping subpage 0 makes the parent's bulk pagemap read report
all 512 subpages exclusive; unmapping subpage 1, 256 or 511 reports none
exclusive. With the one-line fix, only the unmapped-in-child subpage is
reported exclusive in each case. The fully private/shared controls pass
on both kernels.

Tested on x86-64 QEMU/KVM with 4KB base pages, 2MB PMD THPs and
CONFIG_PAGE_MAPCOUNT=y, using mm-hotfixes-unstable at 64703f0fee74 and
that same base plus this patch. Three runs of the same binary gave:

  Before: 20 PASS, 8 FAIL per run
  After:  28 PASS, 0 FAIL per run

To build, save the attachment under tools/testing/selftests/mm/ and run
from the kernel root:

  cc -O2 -Wall -Wextra tools/testing/selftests/mm/pagemap_thp_exclusive.c \
     -o /tmp/pagemap_thp_exclusive
  /tmp/pagemap_thp_exclusive

It uses the existing ../kselftest.h and skips if a 2MB PMD THP cannot be
obtained. Its expectations are deliberately limited to the current
CONFIG_PAGE_MAPCOUNT=y implementation, not the conservative variant.

Thanks,
Kelvin

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: pagemap_thp_exclusive.c --]
[-- Type: text/x-c, Size: 5110 bytes --]

// SPDX-License-Identifier: GPL-2.0
/*
 * Diagnostic for the CONFIG_PAGE_MAPCOUNT=y pagemap PMD path, not a generic
 * bit-56 ABI test: other configurations may report folio-wide exclusivity.
 * Requires 4KB base pages and 2MB anonymous PMD THPs.
 *
 * Save under tools/testing/selftests/mm/ and build from the kernel root:
 * cc -O2 -Wall -Wextra tools/testing/selftests/mm/pagemap_thp_exclusive.c \
 *    -o /tmp/pagemap_thp_exclusive
 */
#define _GNU_SOURCE
#include <fcntl.h>
#include <signal.h>
#include <stdint.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
#include "../kselftest.h"

#ifndef MADV_COLLAPSE
#define MADV_COLLAPSE 25
#endif
#define PAGE_BYTES 4096UL
#define THP_BYTES (2UL * 1024 * 1024)
#define NR_PAGES (THP_BYTES / PAGE_BYTES)
#define PRESENT (1ULL << 63)
#define EXCLUSIVE (1ULL << 56)

static void require(int ok, const char *what)
{
	if (!ok)
		ksft_exit_fail_msg("%s failed (errno=%d)\n", what, errno);
}

static void notify(int fd)
{
	require(write(fd, "x", 1) == 1, "pipe write");
}

static void await(int fd)
{
	char c;

	require(read(fd, &c, 1) == 1, "pipe read");
}

static unsigned long anon_huge_bytes(void *addr)
{
	FILE *f = fopen("/proc/self/smaps", "r");
	unsigned long start, end, kb = 0;
	bool selected = false;
	char line[512];

	require(f != NULL, "open smaps");
	while (fgets(line, sizeof(line), f)) {
		if (sscanf(line, "%lx-%lx", &start, &end) == 2)
			selected = (uintptr_t)addr >= start && (uintptr_t)addr < end;
		if (selected && sscanf(line, "AnonHugePages: %lu kB", &kb) == 1)
			break;
	}
	fclose(f);
	return kb * 1024;
}

/* hole == -1: all shared; hole == -2: all exclusive. */
static void check_bulk(int fd, char *addr, int hole, const char *label)
{
	uint64_t entries[NR_PAGES];
	unsigned int i, wrong = 0, exclusive = 0;
	off_t offset = (uintptr_t)addr / PAGE_BYTES * sizeof(uint64_t);

	require(pread(fd, entries, sizeof(entries), offset) == sizeof(entries),
		"read pagemap");
	for (i = 0; i < NR_PAGES; i++) {
		bool want = hole == -2 || hole == (int)i;

		exclusive += !!(entries[i] & EXCLUSIVE);
		wrong += !(entries[i] & PRESENT) ||
			 !!(entries[i] & EXCLUSIVE) != want;
	}
	ksft_test_result(!wrong, "%s: hole=%d exclusive=%u/%lu wrong=%u\n",
			 label, hole, exclusive, NR_PAGES, wrong);
}

static void check_single(int fd, char *addr, unsigned int index, bool exclusive)
{
	uint64_t entry;
	off_t offset = ((uintptr_t)addr / PAGE_BYTES + index) * sizeof(entry);

	require(pread(fd, &entry, sizeof(entry), offset) == sizeof(entry),
		"read single pagemap entry");
	ksft_test_result((entry & PRESENT) && !!(entry & EXCLUSIVE) == exclusive,
			 "single read: subpage=%u expected exclusive=%d\n",
			 index, exclusive);
}

static void run_case(unsigned int hole)
{
	char *reserve, *addr;
	int go[2], done[2], fd, status;
	pid_t child;

	/* Leave PROT_NONE guards on both sides to prevent VMA merging. */
	reserve = mmap(NULL, 3 * THP_BYTES, PROT_NONE,
		       MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
	require(reserve != MAP_FAILED, "reserve address space");
	addr = (char *)(((uintptr_t)reserve + THP_BYTES) & ~(THP_BYTES - 1));
	require(mmap(addr, THP_BYTES, PROT_READ | PROT_WRITE,
		     MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0) == addr,
		"map anonymous memory");
	require(!madvise(addr, THP_BYTES, MADV_HUGEPAGE), "MADV_HUGEPAGE");
	memset(addr, 0x5a, THP_BYTES);
	if (anon_huge_bytes(addr) != THP_BYTES)
		madvise(addr, THP_BYTES, MADV_COLLAPSE);
	if (anon_huge_bytes(addr) != THP_BYTES) {
		for (unsigned int i = 0; i < 7; i++)
			ksft_test_result_skip("hole=%u: no 2MB PMD THP\n", hole);
		munmap(reserve, 3 * THP_BYTES);
		return;
	}
	fd = open("/proc/self/pagemap", O_RDONLY);
	require(fd >= 0, "open pagemap");
	check_bulk(fd, addr, -2, "before fork");
	require(!pipe(go) && !pipe(done), "pipe");
	child = fork();
	require(child >= 0, "fork");
	if (!child) {
		close(go[1]);
		close(done[0]);
		await(go[0]);
		require(!munmap(addr + hole * PAGE_BYTES, PAGE_BYTES), "child munmap");
		notify(done[1]);
		await(go[0]);
		_exit(0);
	}
	close(go[0]);
	close(done[1]);
	check_bulk(fd, addr, -1, "after fork, before child munmap");
	notify(go[1]);
	await(done[0]);
	ksft_test_result(anon_huge_bytes(addr) == THP_BYTES,
			 "hole=%u: parent still PMD mapped\n", hole);
	check_bulk(fd, addr, hole, "after child munmap");
	check_single(fd, addr, hole, true);
	check_single(fd, addr, hole ? 0 : 1, false);
	notify(go[1]);
	require(waitpid(child, &status, 0) == child && WIFEXITED(status) &&
		!WEXITSTATUS(status), "child exit");
	check_bulk(fd, addr, -2, "after child exit");
	close(go[1]);
	close(done[0]);
	close(fd);
	require(!munmap(reserve, 3 * THP_BYTES), "unmap reservation");
}

int main(void)
{
	unsigned int holes[] = { 0, 1, NR_PAGES / 2, NR_PAGES - 1 };

	ksft_print_header();
	if (sysconf(_SC_PAGESIZE) != PAGE_BYTES)
		ksft_exit_skip("This reproducer requires 4KB base pages\n");
	ksft_print_msg("Requires CONFIG_PAGE_MAPCOUNT=y; not a generic ABI test\n");
	ksft_set_plan(7 * ARRAY_SIZE(holes));
	alarm(30);
	for (unsigned int i = 0; i < ARRAY_SIZE(holes); i++)
		run_case(holes[i]);
	ksft_finished();
}

Re: [PATCH] fs/proc/task_mmu: check THP exclusivity for each subpage

[Andrew Morton]

On Sun,  1 Nov 2026 09:01:00 -0800 Kelvin Zhang <zhangxp1998@gmail.com> wrote:

> A PMD-mapped THP can contain subpages with different mapcounts. For
> example, after fork(), the child can unmap only part of the THP. The
> parent retains its PMD mapping, while the remaining child mappings
> are PTE mappings.
> 
> pagemap_pmd_range_thp() passes the same initial page to
> __folio_page_mapped_exclusively() for every pagemap entry. With
> CONFIG_PAGE_MAPCOUNT=y, this applies the first subpage's exclusivity
> to all entries, falsely setting or clearing PM_MMAP_EXCLUSIVE for
> subpages with different sharing.
> 
> Pass page + idx to check the subpage corresponding to each entry, as
> was done before converting to __folio_page_mapped_exclusively(). The
> index already accounts for reads starting partway through the PMD.
> The conservative whole-folio check without CONFIG_PAGE_MAPCOUNT is
> unchanged.

Thanks.

I don't want to pick on you at all, but this is a classic changelog - I
see this pattern so often.

	a) here's what the code does

	b) here's why that's wrong

	c) here's how I fixed it.

But that's missing a very important bullet point:

	a) here's what the code does

	b) here's why that's wrong

	c) here's how this affects users

	d) here's how I fixed it.

see?  This omission is so common!  And I might have sent one or two
emails asking people for the missing info.

For the missing c), how does this sound?

: With transparent huge pages, /proc/PID/pagemap can incorrectly
: report shared pages as exclusive, or vice versa, after fork() and
: partial unmapping.  This can mislead memory-analysis and
: checkpoint/restore tools.  Actual mappings and copy-on-write behavior
: are unaffected.

And now that we have our statement of userspace impact, busy non-mm
downstream readers of this patch can easily see why your proposed
cc:stable is justified.


Also, it would be helpful to show brief before-and-after excerpts from
/proc/PID/pagemap to help others understand the impact of the change. 
But I don't think we need a v2 just for this.