[PATCH v4 1/2] mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs

[PATCH v4 1/2] mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs

[James Houghton]

Userfaultfd minor faults provides userspace with the ability to manually
install PTEs with UFFDIO_CONTINUE. Right now, MADV_COLLAPSE can map
holes in the VMA when a naturally-aligned THP is present. This is not
true for khugepaged collapse: the PTEs will be retracted, but a PMD will
not be installed.

When MADV_COLLAPSE installs a PMD that mapped holes in the VMA,
userspace is likely to expect UFFDIO_CONTINUE to succeed on the
should-be holes. UFFDIO_CONTINUE will fail and return EEXIST.

This is not inherently a problem, as MADV_COLLAPSE is an explicit
userspace action. But, especially because MADV_COLLAPSE can be invoked
by an external process via process_madvise(), a rogue caller could break
a userfaultfd-minor resolver thread.

Userspace cannot generally use MADV_COLLAPSE to resolve userfault minor
faults, as MADV_COLLAPSE will only resolve such faults if a
naturally-aligned THP is present, so this is not a functional
regression for userspace.

The naturally-aligned THP case is the only case where this quirk exists.
Collapsing otherwise requires all PTEs to be present for
userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
correct. This check is essentially bypassed for naturally-aligned THPs.

Suggested-by: Lance Yang <lance.yang@linux.dev>
Tested-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: James Houghton <jthoughton@google.com>
---
Changes since v3:
  - Prevent page table retraction in patch 1. Please see the comment
    next to it. So this patch now has two hunks instead of one.
  - Reworded the comment and combined the userfaultfd checks in patch 1.
    (Thanks David)
  - Simplified the selftest a bit given the behavior change in patch 1.
  - Undid a change in v3's selftest that incorrectly handled the case
    where MADV_COLLAPSE was not supported entirely.
  - Rebased on top of mm-unstable, which includes Kiryl's changes.
v3: https://lore.kernel.org/linux-mm/20260910023411.514987-1-jthoughton@google.com/
v2: https://lore.kernel.org/linux-mm/20260828222640.1638457-1-jthoughton@google.com/
v1: https://lore.kernel.org/linux-mm/20260828005004.2870750-1-jthoughton@google.com/
---
 mm/khugepaged.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 913086eaf17b..438c4af1c058 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1868,10 +1868,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
 		return SCAN_VMA_CHECK;
 
 	/*
-	 * Keep pmd pgtable while the uffd bit is in use; see comment in
-	 * retract_page_tables().
+	 * Don't collapse if there might be PTE markers for userfaultfd-based
+	 * access protection or if collapsing might bypass userfaultfd minor
+	 * faults.
 	 */
-	if (userfaultfd_protected(vma))
+	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
 		return SCAN_PTE_UFFD;
 
 	folio = filemap_lock_folio(vma->vm_file->f_mapping,
@@ -2093,8 +2094,13 @@ static bool file_backed_vma_is_retractable(struct vm_area_struct *vma)
 	 * and cannot be recycled to a shared PMD. Other vmas can still
 	 * have the same file mapped hugely, but skip this one: it will
 	 * always be mapped in small page size for these registrations.
+	 *
+	 * Userfaultfd-minor-registered VMAs should also not be retracted.
+	 * PMDs will not be installed, as doing so can suppress minor faults.
+	 * If retraction were allowed, khugepaged might continually cause
+	 * unnecessary userfaultfd minor faults for already-CONTINUE'd pages.
 	 */
-	if (userfaultfd_protected(vma))
+	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
 		return false;
 
 	/*

base-commit: b2b4b29b76dabdee576eba66953a66ca61c5fca0
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

[PATCH v4 2/2] mm: selftests: Adjust the MADV_COLLAPSE selftests for uffd-minor

[James Houghton]

The behavior of MADV_COLLAPSE with uffd-minor VMAs has changed.
Previously, MADV_COLLAPSE would install PMDs in parts of a VMA where
PTEs may be missing, essentially suppressing userfaultfd minor faults.
Now, MADV_COLLAPSE returns -EINVAL for uffd-minor VMAs. Update the
selftest to demonstrate this new behavior.

The selftest, if left unmodified, will SKIP when it gets EINVAL back
from MADV_COLLAPSE.
---
 tools/testing/selftests/mm/uffd-unit-tests.c | 69 ++++++++++++++++++--
 1 file changed, 62 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c b/tools/testing/selftests/mm/uffd-unit-tests.c
index 580178630ede..ab195c2c44d3 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -518,19 +518,32 @@ static void uffd_wp_fork_pin_with_event_test(uffd_global_test_opts_t *gopts, uff
 	uffd_wp_fork_pin_test_common(gopts, args, true);
 }
 
-static void check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+static int __check_memory_contents(unsigned long offset, unsigned long nr_pages,
+		uffd_global_test_opts_t *gopts, char *p)
 {
 	unsigned long i, j;
 	uint8_t expected_byte;
 
-	for (i = 0; i < gopts->nr_pages; ++i) {
+	if (nr_pages + offset < nr_pages)
+		err("overflow in memory check");
+	if (nr_pages + offset > gopts->nr_pages)
+		err("out of bounds memory check");
+
+	for (i = offset; i < offset + nr_pages; ++i) {
 		expected_byte = ~((uint8_t)(i % ((uint8_t)-1)));
 		for (j = 0; j < gopts->page_size; j++) {
 			uint8_t v = *(uint8_t *)(p + (i * gopts->page_size) + j);
 			if (v != expected_byte)
-				err("unexpected page contents");
+				return 1;
 		}
 	}
+
+	return 0;
+}
+
+static int check_memory_contents(uffd_global_test_opts_t *gopts, char *p)
+{
+	return __check_memory_contents(0, gopts->nr_pages, gopts, p);
 }
 
 static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_collapse, bool test_wp)
@@ -538,8 +551,8 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	unsigned long p;
 	pthread_t uffd_mon;
 	char c = '\0';
-	struct uffd_args args = { 0 };
-	args.gopts = gopts;
+	struct uffd_args args = { .gopts = gopts };
+	bool bad_contents;
 
 	/*
 	 * NOTE: MADV_COLLAPSE is not yet compatible with WP, so testing
@@ -564,20 +577,59 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 	if (pthread_create(&uffd_mon, NULL, uffd_poll_thread, &args))
 		err("uffd_poll_thread create");
 
+	if (test_collapse) {
+		/*
+		 * Read just a single page and try collapsing. The collapse
+		 * should either be rejected or be a no-op.
+		 */
+		if (__check_memory_contents(0, 1, gopts, gopts->area_dst_alias))
+			err("unexpected memory contents before collapse");
+
+		/* MADV_COLLAPSE might return EINVAL for uffd-minor VMAs. */
+		if (madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
+			      MADV_COLLAPSE) && errno != EINVAL)
+			err("MADV_COLLAPSE failed unexpectedly");
+		/*
+		 * If the above collapse mapped pages that were not explicitly
+		 * CONTINUE'd, the below __check_memory_contents() will not
+		 * fault on some pages, resulting in incorrect contents.
+		 * The page we already touched should not be unmapped, so a
+		 * second userfault on it should not be generated.
+		 */
+	}
+
 	/*
 	 * Read each of the pages back using the UFFD-registered mapping. We
 	 * expect that the first time we touch a page, it will result in a minor
 	 * fault. uffd_poll_thread will resolve the fault by bit-flipping the
 	 * page's contents, and then issuing a CONTINUE ioctl.
 	 */
-	check_memory_contents(gopts, gopts->area_dst_alias);
+	bad_contents = !!check_memory_contents(gopts, gopts->area_dst_alias);
 
 	if (write(gopts->pipefd[1], &c, sizeof(c)) != sizeof(c))
 		err("pipe write");
 	if (pthread_join(uffd_mon, NULL))
 		err("join() failed");
 
+	if (bad_contents && test_collapse) {
+		uffd_test_fail("unexpected memory contents after collapse");
+		return;
+	}
+
+	if (bad_contents) {
+		uffd_test_fail("unexpected memory contents");
+		return;
+	}
+
 	if (test_collapse) {
+		/*
+		 * MADV_COLLAPSE will fail unless userfaultfd-minor is
+		 * unregistered.
+		 */
+		if (uffd_unregister(gopts->uffd, gopts->area_dst_alias,
+				    gopts->nr_pages * gopts->page_size))
+			err("uffd_unregister before MADV_COLLAPSE failed");
+
 		if (madvise(gopts->area_dst_alias, gopts->nr_pages * gopts->page_size,
 			    MADV_COLLAPSE)) {
 			/* It's fine to fail for this one... */
@@ -593,7 +645,10 @@ static void uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
 		 * This won't cause uffd-fault - it purely just makes sure there
 		 * was no corruption.
 		 */
-		check_memory_contents(gopts, gopts->area_dst_alias);
+		if (check_memory_contents(gopts, gopts->area_dst_alias)) {
+			uffd_test_fail("unexpected memory contents");
+			return;
+		}
 	}
 
 	if (args.missing_faults != 0 || args.minor_faults != gopts->nr_pages)
-- 
2.56.0.rc1.315.gc6ed9934b7-goog

Re: [PATCH v4 1/2] mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs

[Kiryl Shutsemau]

On Sat, Oct 03, 2026 at 12:18:58AM +0000, James Houghton wrote:
> Userfaultfd minor faults provides userspace with the ability to manually
> install PTEs with UFFDIO_CONTINUE. Right now, MADV_COLLAPSE can map
> holes in the VMA when a naturally-aligned THP is present. This is not
> true for khugepaged collapse: the PTEs will be retracted, but a PMD will
> not be installed.
> 
> When MADV_COLLAPSE installs a PMD that mapped holes in the VMA,
> userspace is likely to expect UFFDIO_CONTINUE to succeed on the
> should-be holes. UFFDIO_CONTINUE will fail and return EEXIST.
> 
> This is not inherently a problem, as MADV_COLLAPSE is an explicit
> userspace action. But, especially because MADV_COLLAPSE can be invoked
> by an external process via process_madvise(), a rogue caller could break
> a userfaultfd-minor resolver thread.
> 
> Userspace cannot generally use MADV_COLLAPSE to resolve userfault minor
> faults, as MADV_COLLAPSE will only resolve such faults if a
> naturally-aligned THP is present, so this is not a functional
> regression for userspace.
> 
> The naturally-aligned THP case is the only case where this quirk exists.
> Collapsing otherwise requires all PTEs to be present for
> userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
> correct. This check is essentially bypassed for naturally-aligned THPs.
> 
> Suggested-by: Lance Yang <lance.yang@linux.dev>
> Tested-by: Lance Yang <lance.yang@linux.dev>
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
> Changes since v3:
>   - Prevent page table retraction in patch 1. Please see the comment
>     next to it. So this patch now has two hunks instead of one.
>   - Reworded the comment and combined the userfaultfd checks in patch 1.
>     (Thanks David)
>   - Simplified the selftest a bit given the behavior change in patch 1.
>   - Undid a change in v3's selftest that incorrectly handled the case
>     where MADV_COLLAPSE was not supported entirely.
>   - Rebased on top of mm-unstable, which includes Kiryl's changes.
> v3: https://lore.kernel.org/linux-mm/20260910023411.514987-1-jthoughton@google.com/
> v2: https://lore.kernel.org/linux-mm/20260828222640.1638457-1-jthoughton@google.com/
> v1: https://lore.kernel.org/linux-mm/20260828005004.2870750-1-jthoughton@google.com/
> ---
>  mm/khugepaged.c | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index 913086eaf17b..438c4af1c058 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -1868,10 +1868,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
>  		return SCAN_VMA_CHECK;
>  
>  	/*
> -	 * Keep pmd pgtable while the uffd bit is in use; see comment in
> -	 * retract_page_tables().
> +	 * Don't collapse if there might be PTE markers for userfaultfd-based
> +	 * access protection or if collapsing might bypass userfaultfd minor
> +	 * faults.
>  	 */
> -	if (userfaultfd_protected(vma))
> +	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
>  		return SCAN_PTE_UFFD;
>  
>  	folio = filemap_lock_folio(vma->vm_file->f_mapping,
> @@ -2093,8 +2094,13 @@ static bool file_backed_vma_is_retractable(struct vm_area_struct *vma)
>  	 * and cannot be recycled to a shared PMD. Other vmas can still
>  	 * have the same file mapped hugely, but skip this one: it will
>  	 * always be mapped in small page size for these registrations.
> +	 *
> +	 * Userfaultfd-minor-registered VMAs should also not be retracted.
> +	 * PMDs will not be installed, as doing so can suppress minor faults.
> +	 * If retraction were allowed, khugepaged might continually cause
> +	 * unnecessary userfaultfd minor faults for already-CONTINUE'd pages.
>  	 */
> -	if (userfaultfd_protected(vma))
> +	if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
>  		return false;
>  
>  	/*


Did you run 2/2 on current mm-unstable?

Here, on e8d0f6a1b2a4 plus this series, minor-collapse fails every run:

  # failed reason: [unexpected memory contents after collapse]
  not ok 1 minor-collapse on shmem

This patch refuses the collapse too late.

collapse_scan_file() has no VMA, so the uffd check in
collapse_max_ptes_none() never applies, and collapse_file() goes ahead:
it try_to_unmap()s the PTEs userspace already CONTINUE'd and replaces
the page cache.

Only then does try_collapse_pte_mapped_thp() return SCAN_PTE_UFFD.

MADV_COLLAPSE fails with -EINVAL, but the page the test read beforehand
is already unmapped, takes a second minor fault, and gets flipped twice.

Also, your 2/2 missing Signed-off-by.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

Re: [PATCH v4 1/2] mm/khugepaged: Never install PMDs in uffd-minor-registered VMAs

[James Houghton]

On Mon, Oct 12, 2026 at 12:00 PM Kiryl Shutsemau <kas@kernel.org> wrote:
>
> On Sat, Oct 03, 2026 at 12:18:58AM +0000, James Houghton wrote:
> > Userfaultfd minor faults provides userspace with the ability to manually
> > install PTEs with UFFDIO_CONTINUE. Right now, MADV_COLLAPSE can map
> > holes in the VMA when a naturally-aligned THP is present. This is not
> > true for khugepaged collapse: the PTEs will be retracted, but a PMD will
> > not be installed.
> >
> > When MADV_COLLAPSE installs a PMD that mapped holes in the VMA,
> > userspace is likely to expect UFFDIO_CONTINUE to succeed on the
> > should-be holes. UFFDIO_CONTINUE will fail and return EEXIST.
> >
> > This is not inherently a problem, as MADV_COLLAPSE is an explicit
> > userspace action. But, especially because MADV_COLLAPSE can be invoked
> > by an external process via process_madvise(), a rogue caller could break
> > a userfaultfd-minor resolver thread.
> >
> > Userspace cannot generally use MADV_COLLAPSE to resolve userfault minor
> > faults, as MADV_COLLAPSE will only resolve such faults if a
> > naturally-aligned THP is present, so this is not a functional
> > regression for userspace.
> >
> > The naturally-aligned THP case is the only case where this quirk exists.
> > Collapsing otherwise requires all PTEs to be present for
> > userfaultfd-registered VMAs (i.e., max none PTEs is 0), which is
> > correct. This check is essentially bypassed for naturally-aligned THPs.
> >
> > Suggested-by: Lance Yang <lance.yang@linux.dev>
> > Tested-by: Lance Yang <lance.yang@linux.dev>
> > Signed-off-by: James Houghton <jthoughton@google.com>
> > ---
> > Changes since v3:
> >   - Prevent page table retraction in patch 1. Please see the comment
> >     next to it. So this patch now has two hunks instead of one.
> >   - Reworded the comment and combined the userfaultfd checks in patch 1.
> >     (Thanks David)
> >   - Simplified the selftest a bit given the behavior change in patch 1.
> >   - Undid a change in v3's selftest that incorrectly handled the case
> >     where MADV_COLLAPSE was not supported entirely.
> >   - Rebased on top of mm-unstable, which includes Kiryl's changes.
> > v3: https://lore.kernel.org/linux-mm/20260910023411.514987-1-jthoughton@google.com/
> > v2: https://lore.kernel.org/linux-mm/20260828222640.1638457-1-jthoughton@google.com/
> > v1: https://lore.kernel.org/linux-mm/20260828005004.2870750-1-jthoughton@google.com/
> > ---
> >  mm/khugepaged.c | 14 ++++++++++----
> >  1 file changed, 10 insertions(+), 4 deletions(-)
> >
> > diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> > index 913086eaf17b..438c4af1c058 100644
> > --- a/mm/khugepaged.c
> > +++ b/mm/khugepaged.c
> > @@ -1868,10 +1868,11 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
> >               return SCAN_VMA_CHECK;
> >
> >       /*
> > -      * Keep pmd pgtable while the uffd bit is in use; see comment in
> > -      * retract_page_tables().
> > +      * Don't collapse if there might be PTE markers for userfaultfd-based
> > +      * access protection or if collapsing might bypass userfaultfd minor
> > +      * faults.
> >        */
> > -     if (userfaultfd_protected(vma))
> > +     if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
> >               return SCAN_PTE_UFFD;
> >
> >       folio = filemap_lock_folio(vma->vm_file->f_mapping,
> > @@ -2093,8 +2094,13 @@ static bool file_backed_vma_is_retractable(struct vm_area_struct *vma)
> >        * and cannot be recycled to a shared PMD. Other vmas can still
> >        * have the same file mapped hugely, but skip this one: it will
> >        * always be mapped in small page size for these registrations.
> > +      *
> > +      * Userfaultfd-minor-registered VMAs should also not be retracted.
> > +      * PMDs will not be installed, as doing so can suppress minor faults.
> > +      * If retraction were allowed, khugepaged might continually cause
> > +      * unnecessary userfaultfd minor faults for already-CONTINUE'd pages.
> >        */
> > -     if (userfaultfd_protected(vma))
> > +     if (userfaultfd_protected(vma) || userfaultfd_minor(vma))
> >               return false;
> >
> >       /*
>
>
> Did you run 2/2 on current mm-unstable?
>
> Here, on e8d0f6a1b2a4 plus this series, minor-collapse fails every run:
>
>   # failed reason: [unexpected memory contents after collapse]
>   not ok 1 minor-collapse on shmem

Oh, I must have missed this when testing the unit test
"simplification" for this v4. :( Thanks for testing it.

It looks like uffd-wp can tolerate the try_to_unmap() because
cond_install_uffd_wp_ptes() will get called in the process.

> This patch refuses the collapse too late.
>
> collapse_scan_file() has no VMA, so the uffd check in
> collapse_max_ptes_none() never applies, and collapse_file() goes ahead:
> it try_to_unmap()s the PTEs userspace already CONTINUE'd and replaces
> the page cache.
>
> Only then does try_collapse_pte_mapped_thp() return SCAN_PTE_UFFD.
>
> MADV_COLLAPSE fails with -EINVAL, but the page the test read beforehand
> is already unmapped, takes a second minor fault, and gets flipped twice.

Thanks for explaining that, I didn't notice the try_to_unmap(). The v3
version of the selftest didn't have this problem because it only
attempted to access each page once.

The purpose of this patch is just to avoid installing PMDs; unmapping
PTEs is acceptable. I think the logic to truly prevent unmapping in
this case probably isn't worth it. Let me know if you disagree.

So I'll just go ahead with fixing the self-test, applying a diff like
this (sorry for the formatting), undoing the "simplification" from
v3->v4:

diff --git a/tools/testing/selftests/mm/uffd-unit-tests.c
b/tools/testing/selftests/mm/uffd-unit-tests.c
index 74ce43769e58..48435faa8bc8 100644
--- a/tools/testing/selftests/mm/uffd-unit-tests.c
+++ b/tools/testing/selftests/mm/uffd-unit-tests.c
@@ -553,6 +553,7 @@ static void
uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
  char c = '\0';
  struct uffd_args args = { .gopts = gopts };
  bool bad_contents;
+ int checked = 0;

  /*
  * NOTE: MADV_COLLAPSE is not yet compatible with WP, so testing
@@ -596,6 +597,7 @@ static void
uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
  * The page we already touched should not be unmapped, so a
  * second userfault on it should not be generated.
  */
+     checked = 1;
  }

  /*
@@ -604,7 +606,8 @@ static void
uffd_minor_test_common(uffd_global_test_opts_t *gopts, bool test_col
  * fault. uffd_poll_thread will resolve the fault by bit-flipping the
  * page's contents, and then issuing a CONTINUE ioctl.
  */
- bad_contents = !!check_memory_contents(gopts, gopts->area_dst_alias);
+ bad_contents = !!__check_memory_contents(checked,
+         opts->nr_pages - checked, gopts, gopts->area_dst_alias);

> Also, your 2/2 missing Signed-off-by.

Thanks, I'll fix that.

I'm actually away right now, so I'll be slow responding to feedback
and can't send a new version until next week.

Thanks for your patience, Kiryl.