mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Harrison Crosse via B4 Relay <devnull+harrison.crosse.dev@kernel.org>
To: David Hildenbrand <david@kernel.org>,
	Jason Gunthorpe <jgg@ziepe.ca>,
	 John Hubbard <jhubbard@nvidia.com>, Peter Xu <peterx@redhat.com>,
	 Andrew Morton <akpm@linux-foundation.org>,
	Lorenzo Stoakes <ljs@kernel.org>,
	 "Liam R. Howlett" <liam@infradead.org>,
	Vlastimil Babka <vbabka@kernel.org>,
	 Mike Rapoport <rppt@kernel.org>,
	Suren Baghdasaryan <surenb@google.com>,
	 Michal Hocko <mhocko@suse.com>, Shuah Khan <shuah@kernel.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	 linux-kselftest@vger.kernel.org,
	Josef Bacik <josef@toxicpanda.com>,
	 Johannes Weiner <hannes@cmpxchg.org>, Jan Kara <jack@suse.cz>,
	 Harrison Crosse <harrison@crosse.dev>,
	stable@vger.kernel.org
Subject: [PATCH 1/2] mm/gup: make mlock() and MAP_POPULATE handle VM_FAULT_RETRY properly
Date: Wed, 18 Nov 2026 16:56:41 -0500	[thread overview]
Message-ID: <20261118-mmpop-populate-v1-1-9c561d178d11@crosse.dev> (raw)
In-Reply-To: <20261118-mmpop-populate-v1-0-9c561d178d11@crosse.dev>

From: Harrison Crosse <harrison@crosse.dev>

If every read of a page of a file mapping fails, mmap(MAP_POPULATE)
and mlock() of that mapping do not return: the task spins in
__mm_populate(), submitting a new read of the page on each iteration,
until it gets a fatal signal. Touching the same page gets SIGBUS, and
madvise(MADV_POPULATE_READ) fails with -EFAULT.

The first fault on the page reaches filemap_fault() with
FAULT_FLAG_ALLOW_RETRY and without FAULT_FLAG_TRIED. filemap_fault()
releases the mmap_lock to read the page and, because it released the
lock, returns VM_FAULT_RETRY before it looks at the result of the read.
It cannot report the error then: a fault that has released the lock
returns VM_FAULT_RETRY or VM_FAULT_COMPLETED, and neither carries an
error. faultin_page() turns VM_FAULT_RETRY into -EBUSY, so
__get_user_pages() stops at that page without an error, and
__mm_populate() calls populate_vma_page_range() again at that address,
again without FOLL_TRIED. The page fault handlers and
__get_user_pages_locked() retry with FAULT_FLAG_TRIED instead, and on
that attempt filemap_fault() reads the page with the lock held and
returns VM_FAULT_SIGBUS if the read fails.

Commit 631426ba1d45 ("mm/madvise: make MADV_POPULATE_(READ|WRITE) handle
VM_FAULT_RETRY properly") fixed the same loop for MADV_POPULATE_* by
calling __get_user_pages_locked(). That does not fit here:
__get_user_pages_locked() retakes the mmap_lock itself and carries on
with the GUP flags it was given, while populate_vma_page_range()
derives those flags from the VMA that __mm_populate() looks up again,
and may skip, after each unlock. Retrying with a NULL locked instead
would drop FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_KILLABLE, so a
userfaultfd range would fail rather than wait, and folio lock waits
would not be killable.

Instead, have __mm_populate() remember the address of the page whose
fault released the mmap_lock. When the next iteration starts at that
address, fault in that page alone with FOLL_TRIED, as
__get_user_pages_locked() does, then continue with the rest of the
range as before. The retry reads the page with the mmap_lock held, as
filemap_fault() did for a page that was not uptodate before v5.1, and
as the other GUP retries do. With a persistent read error the retry
fails with -EFAULT: mlock() returns -ENOMEM
(__mlock_posix_error_return() maps -EFAULT to it), and MAP_POPULATE,
which ignores populate errors, returns the mapping. A fatal signal
still ends the loop, with -EINTR from __get_user_pages() or, if it
interrupts the FOLL_TRIED read, with -EFAULT. The other callers of
populate_vma_page_range() pass a NULL locked, so they cannot get
VM_FAULT_RETRY and are unchanged.

Reproduced with ext4 on dm-linear over a loop device, with the eight
sectors of one page of a file mapped to the dm "error" target, on arm64
(Debian's 6.12.116 and v7.4-rc3) and x86_64 (v7.4-rc3): mmap(MAP_POPULATE)
and mlock() of the file were still running in the kernel after 5 seconds
and ended when sent SIGKILL. On 6.12.116, the block layer tracepoints
showed about 280,000 bios a second being submitted and failed.

The analysis, the patch and this changelog were prepared with the help
of an AI coding assistant (Claude).

Fixes: 6b4c9f446981 ("filemap: drop the mmap_sem for all blocking operations")
Link: https://lore.kernel.org/all/20191105182211.GA33242@cmpxchg.org/
Link: https://lore.kernel.org/all/20240314161300.382526-1-david@redhat.com/
Cc: stable@vger.kernel.org
Signed-off-by: Harrison Crosse <harrison@crosse.dev>
---
Tested on x86_64 v7.4-rc3 under QEMU (TCG): with the error page,
mmap(MAP_POPULATE) returns the mapping and mlock() fails with ENOMEM,
MADV_POPULATE_READ still fails with EFAULT and touching the page still
gets SIGBUS. With reads of that page delayed by 1 s with dm-delay,
SIGKILL during the read ends the task within 2 ms. Without a read
error, MAP_POPULATE, MAP_LOCKED, mlock() and mlockall() map every page,
and the map_populate, mlock2-tests, mlock-random-test, madv_populate,
mremap_dontunmap, on-fault-limit and guard-regions selftests have no
failures, nor does populate_eio from the next patch. On arm64 v7.4-rc3
under QEMU (TCG), populate_eio fails its mmap(MAP_POPULATE) and mlock()
checks without this patch and passes all three with it.

Not tested: the other checks above on arm64, page sizes other than 4K,
transient read errors, userfaultfd ranges, file systems other than ext4
and FUSE, real disks, non-fatal signals, and the rest of the mm
selftests.
---
 mm/gup.c | 36 ++++++++++++++++++++++++++++++++----
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 2b405cce2..e666761ed 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1795,11 +1795,12 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm,
 }
 
 /**
- * populate_vma_page_range() -  populate a range of pages in the vma.
+ * __populate_vma_page_range() -  populate a range of pages in the vma.
  * @vma:   target vma
  * @start: start address
  * @end:   end address
  * @locked: whether the mmap_lock is still held
+ * @tried: whether this retries a fault that released the mmap_lock
  *
  * This takes care of mlocking the pages too if VM_LOCKED is set.
  *
@@ -1814,8 +1815,9 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm,
  * If @locked is non-NULL, it must held for read only and may be
  * released.  If it's released, *@locked will be set to 0.
  */
-long populate_vma_page_range(struct vm_area_struct *vma,
-		unsigned long start, unsigned long end, int *locked)
+static long __populate_vma_page_range(struct vm_area_struct *vma,
+				      unsigned long start, unsigned long end,
+				      int *locked, bool tried)
 {
 	struct mm_struct *mm = vma->vm_mm;
 	unsigned long nr_pages = (end - start) / PAGE_SIZE;
@@ -1860,6 +1862,8 @@ long populate_vma_page_range(struct vm_area_struct *vma,
 
 	if (locked)
 		gup_flags |= FOLL_UNLOCKABLE;
+	if (tried)
+		gup_flags |= FOLL_TRIED;
 
 	/*
 	 * We made sure addr is within a VMA, so the following will
@@ -1871,6 +1875,13 @@ long populate_vma_page_range(struct vm_area_struct *vma,
 	return ret;
 }
 
+long populate_vma_page_range(struct vm_area_struct *vma,
+			     unsigned long start, unsigned long end,
+			     int *locked)
+{
+	return __populate_vma_page_range(vma, start, end, locked, false);
+}
+
 /*
  * faultin_page_range() - populate (prefault) page tables inside the
  *			  given range readable/writable
@@ -1934,7 +1945,9 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
 {
 	struct mm_struct *mm = current->mm;
 	unsigned long end, nstart, nend;
+	unsigned long retry_addr = 0;
 	struct vm_area_struct *vma = NULL;
+	bool retry = false, tried;
 	int locked = 0;
 	long ret = 0;
 
@@ -1967,8 +1980,21 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
 		 * Now fault in a range of pages. populate_vma_page_range()
 		 * double checks the vma flags, so that it won't mlock pages
 		 * if the vma was already munlocked.
+		 *
+		 * filemap_fault() reports a read error only when
+		 * FAULT_FLAG_TRIED is set: on a first attempt it releases the
+		 * mmap_lock for the read and returns VM_FAULT_RETRY whatever
+		 * the result. So when the last call released the lock at
+		 * nstart, retry that page with FOLL_TRIED, and only that page,
+		 * so that later pages still get a first attempt that releases
+		 * the lock for the read. Compare the address because nstart
+		 * moves past that page if it is no longer mapped, or its VMA
+		 * is skipped, when the lock is retaken.
 		 */
-		ret = populate_vma_page_range(vma, nstart, nend, &locked);
+		tried = retry && nstart == retry_addr;
+		ret = __populate_vma_page_range(vma, nstart,
+						tried ? nstart + PAGE_SIZE : nend,
+						&locked, tried);
 		if (ret < 0) {
 			if (ignore_errors) {
 				ret = 0;
@@ -1977,6 +2003,8 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
 			break;
 		}
 		nend = nstart + ret * PAGE_SIZE;
+		retry = !locked;
+		retry_addr = nend;
 		ret = 0;
 	}
 	if (locked)

-- 
2.56.1



  reply	other threads:[~2026-11-18 21:59 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 21:56 [PATCH 0/2] mm/gup: fix mlock() and MAP_POPULATE looping on read errors Harrison Crosse via B4 Relay
2026-11-18 21:56 ` Harrison Crosse via B4 Relay [this message]
2026-11-18 23:25   ` [PATCH 1/2] mm/gup: make mlock() and MAP_POPULATE handle VM_FAULT_RETRY properly David Hildenbrand (Arm)
2026-11-18 23:59     ` Harrison Crosse
2026-11-18 21:56 ` [PATCH 2/2] selftests/mm: add a test for populating a page that cannot be read Harrison Crosse via B4 Relay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118-mmpop-populate-v1-1-9c561d178d11@crosse.dev \
    --to=devnull+harrison.crosse.dev@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=david@kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=harrison@crosse.dev \
    --cc=jack@suse.cz \
    --cc=jgg@ziepe.ca \
    --cc=jhubbard@nvidia.com \
    --cc=josef@toxicpanda.com \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=mhocko@suse.com \
    --cc=peterx@redhat.com \
    --cc=rppt@kernel.org \
    --cc=shuah@kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=surenb@google.com \
    --cc=vbabka@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®