[PATCH 0/3] mm: fixes for THP migration, swap-in and device-private smaps

[PATCH 0/3] mm: fixes for THP migration, swap-in and device-private smaps

[Usama Arif]

This series contains three fixes for existing THP migration, PTE swap-in
and smaps paths. They were extracted from the work on PMD swap entries
for anonymous THPs [1] and can be merged independently. Sending this
separately as requested by Andrew (should have sent them separately
anyways, my bad).

1. Avoid a folio reference leak when concurrent MADV_DONTNEED clears
   or splits a migration PMD before the device migration split helper
   acquires the PMD lock.

2. Prevent PTE swap-in batching from mapping poisoned subpages of a
   large swap-cache folio. Include the subpage scan helper with this
   fix so it can be applied on its own.

3. Account device-private PMDs using mapcounts in smaps, matching the
   PTE path. Shared mappings get proportional PSS, and exclusive
   mappings are counted as private.

[1] PMD-level swap entries for anonymous THPs:
https://lore.kernel.org/r/20261002095503.3585565-1-usama.arif@linux.dev/

Usama Arif (3):
  mm: don't PTE-batch a swap-in over a hardware-poisoned subpage
  fs/proc/task_mmu: don't read young/dirty from a non-present PMD in
    smaps
  fs/proc/task_mmu: account device-private PMDs as present in smaps

 fs/proc/task_mmu.c | 12 ++++++++----
 mm/internal.h      | 15 +++++++++++++++
 mm/memory.c        |  4 ++++
 3 files changed, 27 insertions(+), 4 deletions(-)

-- 
2.53.0-Meta

[PATCH 1/3] mm: don't PTE-batch a swap-in over a hardware-poisoned subpage

[Usama Arif]

do_swap_page() checks PageHWPoison() on the faulting page, but the
large-folio batching path then maps the rest of the folio without looking
at the other subpages. A poisoned subpage that is not the one being
faulted gets mapped anyway, and the fault that would have reported it
never happens.

Skip batching for such a folio and let each page take its own fault,
where the existing check catches it.

Scan the subpages rather than testing the folio-level PG_has_hwpoisoned:
memory_failure() sets PageHWPoison() on the subpage first and only sets
the folio flag much later, and the bail-outs in between - for example
get_hwpoison_page() losing a page lifecycle race - return with the
subpage marked and the folio flag never set.

Fixes: 508758960b8d ("mm: swap: entirely map large folios found in swapcache")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 mm/internal.h | 15 +++++++++++++++
 mm/memory.c   |  4 ++++
 2 files changed, 19 insertions(+)

diff --git a/mm/internal.h b/mm/internal.h
index cd3675c044e0a..e61c02f4019b0 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -28,6 +28,21 @@ struct folio_batch;
 int __copy_remote_mm_str(struct mm_struct *mm, unsigned long addr,
 		void *buf, int len, unsigned int gup_flags);
 
+/*
+ * Check every page of the folio for HWPoison, rather than relying on the
+ * folio-level PG_has_hwpoisoned, which memory_failure() can leave clear.
+ */
+static inline bool folio_has_any_hwpoisoned_page(const struct folio *folio)
+{
+	long nr = folio_nr_pages(folio);
+	long i;
+
+	for (i = 0; i < nr; i++)
+		if (PageHWPoison(folio_page(folio, i)))
+			return true;
+	return false;
+}
+
 /* mm/workingset.c */
 bool workingset_test_recent(void *shadow, bool file, bool *workingset,
 			    bool flush);
diff --git a/mm/memory.c b/mm/memory.c
index d477a4f986bc8..4b7d456332200 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -5158,6 +5158,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		    swap_pte_batch(folio_ptep, nr, folio_pte) != nr)
 			goto check_folio;
 
+		/* Let a poisoned subpage take its own fault. */
+		if (folio_has_any_hwpoisoned_page(folio))
+			goto check_folio;
+
 		page_idx = idx;
 		address = folio_start;
 		ptep = folio_ptep;
-- 
2.53.0-Meta

[PATCH 2/3] fs/proc/task_mmu: don't read young/dirty from a non-present PMD in smaps

[Usama Arif]

smaps_pmd_entry() passes pmd_young(*pmd) and pmd_dirty(*pmd) to
smaps_account() for every PMD, including migration and device-private
entries, where those bits hold part of the swap offset rather than
accessed/dirty state. smaps_pte_entry() only reads them when the entry
is present.

On x86_64 the offset is stored inverted across bits 9-58, which overlap
_PAGE_SAVED_DIRTY, so pmd_dirty() returns garbage: a clean 2MB
device-private mapping is accounted as 2048 kB dirty, and lazyfree
accounting is suppressed. pgtable_64.h also warns that the hardware can
set the accessed and dirty bits on non-present entries by mistake.

Only read the bits when the PMD is present.

Fixes: c94b6923fa0a ("/proc/PID/smaps: Add PMD migration entry parsing")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 fs/proc/task_mmu.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 22361607020d1..c0d391186c2b8 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1019,13 +1019,15 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	struct vm_area_struct *vma = walk->vma;
 	bool locked = !!(vma->vm_flags & VM_LOCKED);
 	struct page *page = NULL;
-	bool present = false;
+	bool present = false, young = false, dirty = false;
 	struct folio *folio;
 
 	if (pmd_none(*pmd))
 		return;
 	if (pmd_present(*pmd)) {
 		page = vm_normal_page_pmd(vma, addr, *pmd);
+		young = pmd_young(*pmd);
+		dirty = pmd_dirty(*pmd);
 		present = true;
 	} else if (unlikely(thp_migration_supported())) {
 		const softleaf_t entry = softleaf_from_pmd(*pmd);
@@ -1045,8 +1047,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	else
 		mss->file_thp += HPAGE_PMD_SIZE;
 
-	smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd),
-		      locked, present);
+	smaps_account(mss, page, true, young, dirty, locked, present);
 }
 #else
 static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
-- 
2.53.0-Meta

[PATCH 3/3] fs/proc/task_mmu: account device-private PMDs as present in smaps

[Usama Arif]

Device-private PMDs represent mapped pages, but smaps_pmd_entry()
passes present=false to smaps_account(). That treats them like migration
entries: mapcounts are ignored, PSS is charged in full, and exclusive
mappings are accounted as shared.

Match smaps_pte_entry() and mark device-private PMDs as present so
smaps_account() uses mapcounts for PSS and private/shared accounting.

Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
Cc: stable@vger.kernel.org
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 fs/proc/task_mmu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index c0d391186c2b8..ad97532fc1184 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1032,8 +1032,11 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	} else if (unlikely(thp_migration_supported())) {
 		const softleaf_t entry = softleaf_from_pmd(*pmd);
 
-		if (softleaf_has_pfn(entry))
+		if (softleaf_has_pfn(entry)) {
+			if (softleaf_is_device_private(entry))
+				present = true;
 			page = softleaf_to_page(entry);
+		}
 	}
 	if (IS_ERR_OR_NULL(page))
 		return;
-- 
2.53.0-Meta

Re: [PATCH 1/3] mm: don't PTE-batch a swap-in over a hardware-poisoned subpage

[Lorenzo Stoakes (ARM)]

On Wed, Nov 11, 2026 at 03:38:29AM -0800, Usama Arif wrote:
> do_swap_page() checks PageHWPoison() on the faulting page, but the
> large-folio batching path then maps the rest of the folio without looking
> at the other subpages. A poisoned subpage that is not the one being
> faulted gets mapped anyway, and the fault that would have reported it
> never happens.
>
> Skip batching for such a folio and let each page take its own fault,
> where the existing check catches it.
>
> Scan the subpages rather than testing the folio-level PG_has_hwpoisoned:
> memory_failure() sets PageHWPoison() on the subpage first and only sets
> the folio flag much later, and the bail-outs in between - for example
> get_hwpoison_page() losing a page lifecycle race - return with the
> subpage marked and the folio flag never set.
>
> Fixes: 508758960b8d ("mm: swap: entirely map large folios found in swapcache")
> Cc: stable@vger.kernel.org
> Signed-off-by: Usama Arif <usama.arif@linux.dev>
> ---
>  mm/internal.h | 15 +++++++++++++++
>  mm/memory.c   |  4 ++++
>  2 files changed, 19 insertions(+)
>
> diff --git a/mm/internal.h b/mm/internal.h
> index cd3675c044e0a..e61c02f4019b0 100644
> --- a/mm/internal.h
> +++ b/mm/internal.h
> @@ -28,6 +28,21 @@ struct folio_batch;
>  int __copy_remote_mm_str(struct mm_struct *mm, unsigned long addr,
>  		void *buf, int len, unsigned int gup_flags);
>
> +/*
> + * Check every page of the folio for HWPoison, rather than relying on the
> + * folio-level PG_has_hwpoisoned, which memory_failure() can leave clear.
> + */
> +static inline bool folio_has_any_hwpoisoned_page(const struct folio *folio)
> +{
> +	long nr = folio_nr_pages(folio);
> +	long i;
> +
> +	for (i = 0; i < nr; i++)
> +		if (PageHWPoison(folio_page(folio, i)))
> +			return true;
> +	return false;
> +}
> +
>  /* mm/workingset.c */
>  bool workingset_test_recent(void *shadow, bool file, bool *workingset,
>  			    bool flush);
> diff --git a/mm/memory.c b/mm/memory.c
> index d477a4f986bc8..4b7d456332200 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -5158,6 +5158,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>  		    swap_pte_batch(folio_ptep, nr, folio_pte) != nr)
>  			goto check_folio;
>
> +		/* Let a poisoned subpage take its own fault. */
> +		if (folio_has_any_hwpoisoned_page(folio))
> +			goto check_folio;
> +

Hm, isn't scanning every single page in the folio potentially quite costly
here?  I thought the point of batching was to improve performance overall?

It seems broken to have stuff potentially interact with hwpoison but then
to only set the folio-level flag much later?

Also why are we swapping out folios containing hwpoison pages? Shouldn't we
migrate away from the hwpoison? :) (I'm not really familiar with the hw
poison stuff)


>  		page_idx = idx;
>  		address = folio_start;
>  		ptep = folio_ptep;
> --
> 2.53.0-Meta
>

--
Cheers, Lorenzo

Re: [PATCH 2/3] fs/proc/task_mmu: don't read young/dirty from a non-present PMD in smaps

[Lorenzo Stoakes (ARM)]

On Wed, Nov 11, 2026 at 03:38:30AM -0800, Usama Arif wrote:
> smaps_pmd_entry() passes pmd_young(*pmd) and pmd_dirty(*pmd) to
> smaps_account() for every PMD, including migration and device-private
> entries, where those bits hold part of the swap offset rather than
> accessed/dirty state. smaps_pte_entry() only reads them when the entry
> is present.

Could say something something softleaf here ;)

>
> On x86_64 the offset is stored inverted across bits 9-58, which overlap
> _PAGE_SAVED_DIRTY, so pmd_dirty() returns garbage: a clean 2MB
> device-private mapping is accounted as 2048 kB dirty, and lazyfree
> accounting is suppressed. pgtable_64.h also warns that the hardware can
> set the accessed and dirty bits on non-present entries by mistake.
>
> Only read the bits when the PMD is present.
>
> Fixes: c94b6923fa0a ("/proc/PID/smaps: Add PMD migration entry parsing")
> Cc: stable@vger.kernel.org
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Damn, yikes. LGTM so:

Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> ---
>  fs/proc/task_mmu.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 22361607020d1..c0d391186c2b8 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -1019,13 +1019,15 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
>  	struct vm_area_struct *vma = walk->vma;
>  	bool locked = !!(vma->vm_flags & VM_LOCKED);
>  	struct page *page = NULL;
> -	bool present = false;
> +	bool present = false, young = false, dirty = false;
>  	struct folio *folio;
>
>  	if (pmd_none(*pmd))
>  		return;
>  	if (pmd_present(*pmd)) {
>  		page = vm_normal_page_pmd(vma, addr, *pmd);
> +		young = pmd_young(*pmd);
> +		dirty = pmd_dirty(*pmd);
>  		present = true;
>  	} else if (unlikely(thp_migration_supported())) {
>  		const softleaf_t entry = softleaf_from_pmd(*pmd);
> @@ -1045,8 +1047,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
>  	else
>  		mss->file_thp += HPAGE_PMD_SIZE;
>
> -	smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd),
> -		      locked, present);
> +	smaps_account(mss, page, true, young, dirty, locked, present);
>  }
>  #else
>  static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
> --
> 2.53.0-Meta
>

--
Cheers, Lorenzo

Re: [PATCH 3/3] fs/proc/task_mmu: account device-private PMDs as present in smaps

[Lorenzo Stoakes (ARM)]

On Wed, Nov 11, 2026 at 03:38:31AM -0800, Usama Arif wrote:
> Device-private PMDs represent mapped pages, but smaps_pmd_entry()
> passes present=false to smaps_account(). That treats them like migration
> entries: mapcounts are ignored, PSS is charged in full, and exclusive
> mappings are accounted as shared.
>
> Match smaps_pte_entry() and mark device-private PMDs as present so
> smaps_account() uses mapcounts for PSS and private/shared accounting.
>
> Fixes: a30b48bf1b24 ("mm/migrate_device: implement THP migration of zone device pages")
> Cc: stable@vger.kernel.org
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

It feels like PMD level device private entries have been the source of a lot of
issues, maybe a good area to audit in general... :)

Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> ---
>  fs/proc/task_mmu.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index c0d391186c2b8..ad97532fc1184 100644
> --- a/fs/proc/task_mmu.c
> +++ b/fs/proc/task_mmu.c
> @@ -1032,8 +1032,11 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
>  	} else if (unlikely(thp_migration_supported())) {
>  		const softleaf_t entry = softleaf_from_pmd(*pmd);
>
> -		if (softleaf_has_pfn(entry))
> +		if (softleaf_has_pfn(entry)) {
> +			if (softleaf_is_device_private(entry))
> +				present = true;
>  			page = softleaf_to_page(entry);
> +		}
>  	}
>  	if (IS_ERR_OR_NULL(page))
>  		return;
> --
> 2.53.0-Meta
>

--
Cheers, Lorenzo