Re: [PATCH] mm: synchronize with PTE table reclaim for all folio sizes

[Baolin Wang <baolin.wang@linux.alibaba.com>]

On 11/6/26 10:33 AM, Lance Yang wrote:
> 
> On Fri, Nov 06, 2026 at 09:57:47AM +0800, Baolin Wang wrote:
>>
>>
>> On 11/5/26 9:22 PM, Lance Yang wrote:
>>>
>>> On Thu, Nov 05, 2026 at 11:06:29AM +0100, David Hildenbrand (Arm) wrote:
>>>> On 11/5/26 10:00, Baolin Wang wrote:
>>>>> CC Barry, who added the TTU_SYNC in shrink_folio_list().
>>>>>
>>>>> On 11/5/26 1:48 PM, Lance Yang wrote:
>>>>>>
>>>>>>
>>>>>> On 2026/11/5 12:54, Lance Yang wrote:
>>>>>>>
>>>>>>>
>>>>>>> Good catch!
>>>>>>>
>>>>>>>
>>>>>>> Right. Whether the folio can be PMD-mapped doesn't tell us whether its
>>>>>>> PTE table can be reclaimed, so those extra checks can skip synchronization
>>>>>>> that the caller explicitly asked for ;)
>>>>>>
>>>>>> Ouch ... one thing I missed ... what about order-0 reclaim?
>>>>>>
>>>>>> shrink_folio_list() only sets TTU_SYNC for large folio, so can't we still
>>>>>> hit the same race there?
>>>>>
>>>>> Thanks Lance. But this matches the purpose of the TTU_SYNC flag, which is used
>>>>> to avoid racy checks with PVMW_SYNC. If no TTU_SYNC flag is set, it means the
>>>>> caller can tolerate some racy checks.
>>>>>
>>>>> For this case, I think we should set TTU_SYNC unconditionally in
>>>>> shrink_folio_list(). Barry, what do you think?
>>>>
>>>> That makes sense. I hope this doesn't throw a big wrench into our reclaim
>>>> performance numbers.
>>>>
>>>> Can someone benchmark this?
>>>>
>>>> But maybe we should really follow-up with the folio-size based optimization in
>>>> sync_with_folio_pmd_zap(), where we only need a memory barrier (and sufficient
>>>> documentation why something that cannot be PMD-mapped can get away cheaper).
>>>
>>> Dug into the history a bit :D
>>>
>>> TTU_SYNC goes back to Hugh's THP splitting fix [1]. A racing zap could
>>> clear the PTE/PMD before updating mapcount, so the walker could skip the
>>> lock and return before the accounting caught up.
>>>
>>> Barry later enabled it for large-folio reclaim [2], to avoid skipping
>>> temporarily cleared PTEs and ending up with a partially unmapped folio.
>>> He mentioned that small folios had the same race, but with just one PTE,
>>> they couldn't be partially unmapped. That's where the large-folio check
>>> in shrink_folio_list() came from.
>>>
>>> Then PTE table reclaim [3] added another way to end up with an empty PMD,
>>> even for a folio that can't be PMD-mapped. The truncation fix [4] moved
>>> the rmap flush before the PMD clear and extended synchronization in
>>> zap_pmd_range() to cover those folios too.
>>>
>>> Looking back at the truncation discussion [5], you also pointed out that
>>> page_vma_mapped_walk() needed checking, and suggested a possible
>>> folio-size based optimization in the helper.
>>
>> Thanks.
> 
> Cheers!
> 
>>> [1] https://lore.kernel.org/all/c1e95853-8bcd-d8fd-55fa-e7f2488e78f@google.com/
>>> [2] https://lore.kernel.org/all/20240306095219.71086-1-21cnbao@gmail.com/
>>> [3] https://lore.kernel.org/all/92aba2b319a734913f18ba41e7d86a265f0b84e2.1733305182.git.zhengqi.arch@bytedance.com/
>>> [4] https://lore.kernel.org/all/14ebbd556cc4941f0ebc43da59fa6a2fdca25dc8.1793148609.git.baolin.wang@linux.alibaba.com/
>>> [5] https://lore.kernel.org/all/08a0907f-8e3a-4071-a2e4-fd1411c446f3@kernel.org/
>>>
>>> So, how about starting with order-0 folios?

.......

>>>
>>> atomic_add_negative() provides full ordering for mapcount updates. With
>>> 243cf1f67b36, rmap removal completes before clearing PMD, including when
>>> another CPU reclaims an empty table under PTE lock. So smp_rmb() is enough
>>> here, without an extra writer-side barrier.
>>
>> I'm afraid the memory barrier here is subtle. __folio_remove_rmap() is
>> complex, and in the CONFIG_NO_PAGE_MAPCOUNT branch,
>> folio_sub_return_large_mapcount() doesn't provide full ordering, so we
>> may still need smp_wmb().
> 
> Yeah, agreed for large folios. In the diff above, only order-0 folios
> use smp_rmb().
> 
> Note that order-0 folios take the fully ordered atomic_add_negative()
> branch and skip the CONFIG_NO_PAGE_MAPCOUNT branch entirely.

Yes.

> static __always_inline void __folio_remove_rmap(struct folio *folio,
> 		struct page *page, int nr_pages, struct vm_area_struct *vma,
> 		enum pgtable_level level)
> {
> 	...
> 	switch (level) {
> 	case PGTABLE_LEVEL_PTE:
> 		if (!folio_test_large(folio)) {
> 			nr = atomic_add_negative(-1, &folio->_mapcount);
> 			break;
> 		}
> 
> 		if (IS_ENABLED(CONFIG_NO_PAGE_MAPCOUNT)) {
> 			nr = folio_sub_return_large_mapcount(folio, nr_pages, vma);
> 			...
> 		}
> 		...
> 	}
> 	...
> }
> 
> And large folios still take the PMD lock :D

IIUC, the memory barrier optimization targets all non-PMD-sized folios 
(including large folios). That is, for PMD-sized large folios we still 
use the original PMD lock, while for other non-PMD-sized folios we can 
use memory barriers to sync.