Re: [BUG] shmem: FALLOC_FL_PUNCH_HOLE vs fault-around race corrupts page cache / rss counters

[Baolin Wang <baolin.wang@linux.alibaba.com>]

On 9/24/26 4:34 PM, Pedro Falcato wrote:
> (please use the email I actually use for work, thanks; not sure how
> you got to that one)
> 
> Hi,
> 
> On Thu, Sep 24, 2026 at 06:16:21AM +0000, Ayush Ranjan wrote:
>> Hi,
>>
>> We are seeing shmem/tmpfs page cache corruption on production hosts
>> running a workload that punches holes in a memfd (hole-punch based
>> memory reclaim) while other threads and forked children fault the
>> same MAP_SHARED mapping. The kernel taints but does not oops:
>>
>>    BUG: Bad page cache in process ...  pfn:...
>>    page dumped because: still mapped when deleted
>>    ...
>>    dentry name(?): "memfd:..."
>>
>> and, more frequently, a paired rss-counter imbalance when the mm is
>> torn down, always exactly one PMD-order folio (512 pages):
>>
>>    BUG: Bad rss-counter state mm:... type:MM_FILEPAGES  val:-512
>>    BUG: Bad rss-counter state mm:... type:MM_SHMEMPAGES val:512
>>
>> Seen on 6.12 and 6.18, x86_64, bare metal and VM, with
>> /sys/kernel/mm/transparent_hugepage/shmem_enabled = always.
>>
>> This looks like the same corruption Gregory reported in March, which
>> as far as I can tell stalled without a fix; that report needed ~100
>> ballooning VMs to reproduce:
> 
> Well, I thought (from the lack of replies) that it was probably a bug on
> their side. Perhaps that's not true :/
> 
>>
>>    https://patchew.org/linux/20260326162611.693539-1-gourry@gourry.net/
>>
>> The reproducer at the end of this mail trips it with a single memfd,
>> no VMs or ballooning, within a few minutes on a large machine, so
>> hopefully it makes the race easier to confirm (and to test a fix
>> against).
>>
>> Here is my best understanding of the race -- corrections welcome:
>>
>> shmem guards faults against an in-progress hole punch with
>> inode->i_private: shmem_fault() -> shmem_falloc_wait() waits while
>> shmem_fallocate(PUNCH_HOLE) holds i_private. But shmem's .map_pages
>> is the generic filemap_map_pages() (shmem_vm_ops /
>> shmem_anon_vm_ops), which does not consult i_private and does not
>> take invalidate_lock, and shmem does not use invalidate_lock to
>> serialize faults against truncation the way regular filesystems do --
>> the i_private + waitq scheme stands in for it, but only shmem_fault()
>> participates in that scheme.
>>
>> So while shmem_fallocate(PUNCH_HOLE) is between
>> unmap_mapping_range() and shmem_truncate_range(), a concurrent
>> fault-around can (re-)install PTEs for folios that are about to be
>> truncated:
>>
>>    - filemap_map_pages() samples mm_counter_file(folio) once per batch
>>      and applies it with add_mm_counter() after mapping; if the
>>      folio's swapbacked state changes while it is concurrently torn
> 
> But that cannot happen? We hold the folio lock in filemap_map_pages().
> The folio (naturally) cannot be torn down while we have the folio lock.
> 
>>      down, the map-time counter (MM_FILEPAGES) and the zap-time
>>      counter (MM_SHMEMPAGES) disagree by exactly one folio -- the
>>      +/-512 imbalance above.
>>
>>    - a folio re-mapped in this window (by fault-around directly, or
>>      via a child VMA whose PTEs copy_page_range() installs after
>>      unmap_mapping_range() has already walked the i_mmap tree -- the
>>      dup_mmap() variant discussed in the earlier thread) can be
>>      deleted from the page cache while still mapped -> "still mapped
>>      when deleted".
> 
> No, I don't think this paragraph is true. Page cache truncation (via
> truncate, or fallocate PUNCH_HOLE) takes the folio lock for each folio
> that is about to be truncated out. Mapping folios takes the folio lock
> as well, except in the fork() case where a myriad of weird interval tree
> + PTE lock interactions make it safe (AIUI).

Agree.

However, I did previously fix a race between filemap_map_pages() and 
truncation that caused incorrect folio mappings, and I believe this race 
also exists in shmem. Ayush, could you check whether that fix is present 
in your kernel?

f58df566524e ("mm: filemap: fix nr_pages calculation overflow in 
filemap_map_pages()")

>> Reproducer
>> ----------
>>
>> The race is on PMD-order folios, so khugepaged needs to scan
>> aggressively (with the default 10s scan interval the punched ranges
>> are not re-collapsed fast enough to reproduce quickly):
>>
>>    echo always > /sys/kernel/mm/transparent_hugepage/shmem_enabled
>>    cd /sys/kernel/mm/transparent_hugepage/khugepaged
>>    echo 1    > scan_sleep_millisecs
>>    echo 4096 > pages_to_scan
>>    echo 511  > max_ptes_none
>>
>>    cc -O2 -pthread -o repro repro_shmem_punch_race.c
>>    for i in $(seq $(( $(nproc) / 3 ))); do ./repro 60 & done; wait
>>    # watch: dmesg -w
>>
>> On a 112-CPU host this trips within ~2-5 minutes; this capture is
>> from 6.12.0-204.92.4.4.3.el9uek.x86_64:
> 
> Awesome that you have a reproducer! Have you reproduced this on a mainline
> kernel? Enterprise kernels are not supported upstream.

I've been trying to reproduce the issue on v7.3.0-rc1 for half an hour 
now with Ayush's reproducer, but haven't been able to trigger it.