* [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes
@ 2026-11-17 3:40 Baolin Wang
2026-11-17 4:27 ` Andrew Morton
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Baolin Wang @ 2026-11-17 3:40 UTC (permalink / raw)
To: akpm, david, ljs
Cc: riel, liam, vbabka, harry, jannh, lance.yang, qi.zheng, baohua,
baolin.wang, linux-mm, linux-kernel
Similar to commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped
folios"), the '!pmd_present()' check in page_vma_mapped_walk() can also race
with concurrent page table reclamation via zap_empty_pte_table().
page_vma_mapped_walk() is used by many functions including try_to_unmap(),
try_to_migrate(), folio_mkclean(), pfn_mkclean_range(), remove_migration_ptes(),
page_mapped_in_vma(), and mapping_wrprotect_range(). Except for try_to_unmap()
and try_to_migrate(), the other callers do not perform post-walk accounting
checks, so the race has no observable impact on them.
For migration, the race can cause unnecessary retries. For reclaim via
try_to_unmap(), the race can cause failure to reclaim a folio that is in
fact already unmapped (as shown below), which may cause cold/hot inversion
(e.g., in MGLRU) and increase refaults. In extreme cases, failing to
reclaim unmapped large folios could even cause OOM in theory.
CPU 0 (reclaim) CPU 1 (MADV_DONTNEED)
-------------------------------- --------------------------------
folio_lock(F)
try_to_unmap(F, TTU_SYNC)
zap_pte_range(full PMD range)
spin_lock(ptl)
clear PTEs
zap_empty_pte_table()
spin_trylock(pml)
pmd_clear()
spin_unlock(pml)
page_vma_mapped_walk()
observe !pmd_present(pmde)
nr_pages < HPAGE_PMD_NR
skip synchronization
return false
try_to_unmap() returns
folio_mapped(F) == true
nr_unmap_fail += folio_nr_pages(F)
tlb_flush_mmu_tlbonly()
tlb_flush_rmaps()
remove F's rmaps
spin_unlock(ptl)
Commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped folios")
has already moved the PMD clear operation (via zap_empty_pte_table()) to
after tlb_flush_rmaps(). Similarly, page_vma_mapped_walk() also needs to
use sync_with_folio_pmd_zap() to synchronize and avoid memory ordering
issues on weakly ordered CPUs.
Regarding performance, evaluated the impact on try_to_unmap(). Mapped a file
to generate clean large folios, then used MADV_PAGEOUT to reclaim 1G of mapped
clean large folios, repeated 5 times, with no measurable performance impact.
Before: 16.031 ms over 5 loops
After: 16.303 ms over 5 loops
While the race scenario described in the patch may theoretically affect
performance, the actual impact in real workloads is likely limited I think.
Anyway, will use memory barriers to sync for non PMD-sized folios in a
follow-up.
Lance also reported the possible racy check for small folios[1]. However,
whether to set TTU_SYNC for small folios should be done in a separate patch
(this is a pre-existing issue) with proper measurement (though it may be
hard to measure the actual impact).
[1] https://lore.kernel.org/all/0ddaf336-7cb7-4e83-af2c-f60d23f7aafd@linux.dev/
Fixes: 6375e95f381e ("mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED)")
Cc: stable@vger.kernel.org
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
---
Changes from v1:
- Update commit message per David.
- Add reviewed tags from Qi and Lance. Thanks.
---
mm/page_vma_mapped.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index d45d52436f99..a931243ca42b 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -294,10 +294,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
spin_unlock(pvmw->ptl);
pvmw->ptl = NULL;
} else if (!pmd_present(pmde)) {
- if ((pvmw->flags & PVMW_SYNC) &&
- thp_vma_suitable_order(vma, pvmw->address,
- PMD_ORDER) &&
- (pvmw->nr_pages >= HPAGE_PMD_NR))
+ if (pvmw->flags & PVMW_SYNC)
sync_with_folio_pmd_zap(mm, pvmw->pmd);
goto next_pmd;
--
2.47.3
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes
2026-11-17 3:40 [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes Baolin Wang
@ 2026-11-17 4:27 ` Andrew Morton
2026-11-17 9:08 ` David Hildenbrand (Arm)
2026-11-17 4:54 ` Barry Song
2026-11-17 9:09 ` David Hildenbrand
2 siblings, 1 reply; 5+ messages in thread
From: Andrew Morton @ 2026-11-17 4:27 UTC (permalink / raw)
To: Baolin Wang
Cc: david, ljs, riel, liam, vbabka, harry, jannh, lance.yang,
qi.zheng, baohua, linux-mm, linux-kernel
On Tue, 17 Nov 2026 11:40:22 +0800 Baolin Wang <baolin.wang@linux.alibaba.com> wrote:
> Similar to commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped
> folios"), the '!pmd_present()' check in page_vma_mapped_walk() can also race
> with concurrent page table reclamation via zap_empty_pte_table().
>
> page_vma_mapped_walk() is used by many functions including try_to_unmap(),
> try_to_migrate(), folio_mkclean(), pfn_mkclean_range(), remove_migration_ptes(),
> page_mapped_in_vma(), and mapping_wrprotect_range(). Except for try_to_unmap()
> and try_to_migrate(), the other callers do not perform post-walk accounting
> checks, so the race has no observable impact on them.
>
> For migration, the race can cause unnecessary retries. For reclaim via
> try_to_unmap(), the race can cause failure to reclaim a folio that is in
> fact already unmapped (as shown below), which may cause cold/hot inversion
> (e.g., in MGLRU) and increase refaults. In extreme cases, failing to
> reclaim unmapped large folios could even cause OOM in theory.
Thanks, I've updated mm-hotfixes-unstable to this version.
> Changes from v1:
> - Update commit message per David.
> - Add reviewed tags from Qi and Lance. Thanks.
Confirmed - no code changes here,
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes
2026-11-17 4:27 ` Andrew Morton
@ 2026-11-17 9:08 ` David Hildenbrand (Arm)
0 siblings, 0 replies; 5+ messages in thread
From: David Hildenbrand (Arm) @ 2026-11-17 9:08 UTC (permalink / raw)
To: Andrew Morton, Baolin Wang
Cc: ljs, riel, liam, vbabka, harry, jannh, lance.yang, qi.zheng,
baohua, linux-mm, linux-kernel
On 11/17/26 05:27, Andrew Morton wrote:
> On Tue, 17 Nov 2026 11:40:22 +0800 Baolin Wang <baolin.wang@linux.alibaba.com> wrote:
>
>> Similar to commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped
>> folios"), the '!pmd_present()' check in page_vma_mapped_walk() can also race
>> with concurrent page table reclamation via zap_empty_pte_table().
>>
>> page_vma_mapped_walk() is used by many functions including try_to_unmap(),
>> try_to_migrate(), folio_mkclean(), pfn_mkclean_range(), remove_migration_ptes(),
>> page_mapped_in_vma(), and mapping_wrprotect_range(). Except for try_to_unmap()
>> and try_to_migrate(), the other callers do not perform post-walk accounting
>> checks, so the race has no observable impact on them.
>>
>> For migration, the race can cause unnecessary retries. For reclaim via
>> try_to_unmap(), the race can cause failure to reclaim a folio that is in
>> fact already unmapped (as shown below), which may cause cold/hot inversion
>> (e.g., in MGLRU) and increase refaults. In extreme cases, failing to
>> reclaim unmapped large folios could even cause OOM in theory.
>
> Thanks, I've updated mm-hotfixes-unstable to this version.
Please drop it.
I'll take care of it.
It would be good to delay picking stuff that I will pick once ready.
--
Cheers,
David
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes
2026-11-17 3:40 [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes Baolin Wang
2026-11-17 4:27 ` Andrew Morton
@ 2026-11-17 4:54 ` Barry Song
2026-11-17 9:09 ` David Hildenbrand
2 siblings, 0 replies; 5+ messages in thread
From: Barry Song @ 2026-11-17 4:54 UTC (permalink / raw)
To: Baolin Wang
Cc: akpm, david, ljs, riel, liam, vbabka, harry, jannh, lance.yang,
qi.zheng, linux-mm, linux-kernel
On Tue, Nov 17, 2026 at 11:40 AM Baolin Wang
<baolin.wang@linux.alibaba.com> wrote:
>
> Similar to commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped
> folios"), the '!pmd_present()' check in page_vma_mapped_walk() can also race
> with concurrent page table reclamation via zap_empty_pte_table().
>
> page_vma_mapped_walk() is used by many functions including try_to_unmap(),
> try_to_migrate(), folio_mkclean(), pfn_mkclean_range(), remove_migration_ptes(),
> page_mapped_in_vma(), and mapping_wrprotect_range(). Except for try_to_unmap()
> and try_to_migrate(), the other callers do not perform post-walk accounting
> checks, so the race has no observable impact on them.
>
> For migration, the race can cause unnecessary retries. For reclaim via
> try_to_unmap(), the race can cause failure to reclaim a folio that is in
> fact already unmapped (as shown below), which may cause cold/hot inversion
> (e.g., in MGLRU) and increase refaults. In extreme cases, failing to
> reclaim unmapped large folios could even cause OOM in theory.
>
> CPU 0 (reclaim) CPU 1 (MADV_DONTNEED)
> -------------------------------- --------------------------------
> folio_lock(F)
> try_to_unmap(F, TTU_SYNC)
> zap_pte_range(full PMD range)
> spin_lock(ptl)
> clear PTEs
> zap_empty_pte_table()
> spin_trylock(pml)
> pmd_clear()
> spin_unlock(pml)
>
> page_vma_mapped_walk()
> observe !pmd_present(pmde)
> nr_pages < HPAGE_PMD_NR
> skip synchronization
> return false
>
> try_to_unmap() returns
> folio_mapped(F) == true
> nr_unmap_fail += folio_nr_pages(F)
> tlb_flush_mmu_tlbonly()
> tlb_flush_rmaps()
> remove F's rmaps
> spin_unlock(ptl)
>
> Commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped folios")
> has already moved the PMD clear operation (via zap_empty_pte_table()) to
> after tlb_flush_rmaps(). Similarly, page_vma_mapped_walk() also needs to
> use sync_with_folio_pmd_zap() to synchronize and avoid memory ordering
> issues on weakly ordered CPUs.
>
> Regarding performance, evaluated the impact on try_to_unmap(). Mapped a file
> to generate clean large folios, then used MADV_PAGEOUT to reclaim 1G of mapped
> clean large folios, repeated 5 times, with no measurable performance impact.
> Before: 16.031 ms over 5 loops
> After: 16.303 ms over 5 loops
>
> While the race scenario described in the patch may theoretically affect
> performance, the actual impact in real workloads is likely limited I think.
> Anyway, will use memory barriers to sync for non PMD-sized folios in a
> follow-up.
>
> Lance also reported the possible racy check for small folios[1]. However,
> whether to set TTU_SYNC for small folios should be done in a separate patch
> (this is a pre-existing issue) with proper measurement (though it may be
> hard to measure the actual impact).
>
> [1] https://lore.kernel.org/all/0ddaf336-7cb7-4e83-af2c-f60d23f7aafd@linux.dev/
> Fixes: 6375e95f381e ("mm: pgtable: reclaim empty PTE page in madvise(MADV_DONTNEED)")
> Cc: stable@vger.kernel.org
> Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> Acked-by: Qi Zheng <qi.zheng@linux.dev>
> Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Barry Song <baohua@kernel.org>
I agree we should start by respecting PVMW_SYNC in all cases,
although there are still some open questions to address.
Whether we should extend SYNC to more folios is a separate question.
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes
2026-11-17 3:40 [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes Baolin Wang
2026-11-17 4:27 ` Andrew Morton
2026-11-17 4:54 ` Barry Song
@ 2026-11-17 9:09 ` David Hildenbrand
2 siblings, 0 replies; 5+ messages in thread
From: David Hildenbrand @ 2026-11-17 9:09 UTC (permalink / raw)
To: akpm, david, ljs, Baolin Wang
Cc: riel, liam, vbabka, harry, jannh, lance.yang, qi.zheng, baohua,
linux-mm, linux-kernel
On Tue, 17 Nov 2026 11:40:22 +0800, Baolin Wang wrote:
> Similar to commit 243cf1f67b36 ("mm: memory: fix truncation removing mapped
> folios"), the '!pmd_present()' check in page_vma_mapped_walk() can also race
> with concurrent page table reclamation via zap_empty_pte_table().
>
> page_vma_mapped_walk() is used by many functions including try_to_unmap(),
> try_to_migrate(), folio_mkclean(), pfn_mkclean_range(), remove_migration_ptes(),
> page_mapped_in_vma(), and mapping_wrprotect_range(). Except for try_to_unmap()
> and try_to_migrate(), the other callers do not perform post-walk accounting
> checks, so the race has no observable impact on them.
>
> [...]
Applied, thanks!
tree: mm/core.git
branch: core-7.5.misc
Applied patches will appear soon in mm-next, followed by linux-next.
Please report any outstanding bugs that were missed during review, so we
can either drop the patch series or decide how to fix it up.
It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even
though the patches have already been applied. If possible, patch trailers
will be updated.
Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. Further, this is not a guarantee that the
patches will go upstream. If in doubt, please check the listed branch.
[1/1] mm: synchronize with PTE table reclaim for all folio sizes
https://git.kernel.org/mm/core/c/bf4c270e
--
Cheers,
David
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-17 9:09 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 3:40 [PATCH v2] mm: synchronize with PTE table reclaim for all folio sizes Baolin Wang
2026-11-17 4:27 ` Andrew Morton
2026-11-17 9:08 ` David Hildenbrand (Arm)
2026-11-17 4:54 ` Barry Song
2026-11-17 9:09 ` David Hildenbrand
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®