[PATCH v6 0/5] mm: batch TLB flushing for dirty folios in vmscan

[Zhang Peng <zippermonkey@icloud.com>]

This series introduces batch TLB flushing optimization for dirty folios
during memory reclaim, aiming to reduce IPI overhead on multi-core systems.

Background
----------
Reclaim calls try_to_unmap_flush_dirty() before writing out each dirty
folio. When writable TLB entries are pending, this triggers a flush.
On systems that use IPIs for TLB shootdowns, frequent flushes can add
significant overhead.

Approach
--------
Collect dirty folios into a folio_batch, releasing their locks while
collecting the batch so swap faults can progress. Before writeout, try
locking each folio again and recheck writeback, mappings and DMA pins.
Keep eligible folios locked, flush pending writable TLB entries once,
then page them out. Process a final partial batch at the end of the scan.

Changes
-------
Patch 1: Extract the folio activation block at activate_locked into
         folio_activate_locked().
Patch 2: Extract the folio-freeing path (buffer release, lazyfree,
         __remove_mapping, folio_batch drain) into folio_try_reclaim_free().
         Group retained folios, deferred frees and the reclaimed count
         in reclaim_folio_state, local to each shrink_folio_list() call.
Patch 3: Extract the pageout() dispatch state machine into folio_try_pageout().
Patch 4: Extract the TTU setup and try_to_unmap() block into folio_try_unmap().
Patch 5: Implement batch TLB flushing logic. Dirty folios are accumulated in
         batches and a single TLB flush is performed for each batch before
         calling pageout.

Testing
-------
The benchmark script uses stress-ng to compare TLB shootdown behavior before and
after this patch. It constrains a stress-ng workload via memcg to force reclaim
through shrink_folio_list(), reporting TLB shootdowns and IPIs.

Core benchmark command: stress-ng --vm 16 --vm-bytes 2G --vm-keep --timeout 60

==========================================================================
                 batch_dirty_tlb_flush Benchmark Results
==========================================================================
  Kernel: 7.0.0-rc1+   CPUs: 16
  MemTotal: 31834M   SwapTotal: 8191M
  memcg limit: 512M   alloc: 2G   workers: 16   duration: 60s
--------------------------------------------------------------------------
Metric                 Before        After             Delta (abs / %)
--------------------------------------------------------------------------
bogo ops/s             28238.63      35833.97          +7595.34 (+26.9%)
TLB shootdowns         55428953      17621697          -37807256 (-68.2%)
Function call IPIs     34073695      14498768          -19574927 (-57.4%)
pgscan_anon (pages)    52856224      60252894          7396670 (+14.0%)
pgsteal_anon (pages)   29004962      34054753          5049791 (+17.4%)
--------------------------------------------------------------------------

Suggested-by: Kairui Song <kasong@tencent.com>
Signed-off-by: Zhang Peng <bruzzhang@tencent.com>
---
Changes in v6:
- Keep the four preparatory patches and the batching patch in one series,
  as requested by Barry.
- Patch 1: Clarify the folio_activate_locked() comment and retain Barry's
  Reviewed-by. Document the BUG-to-WARN change in the commit message.
- Patch 2: Return KEEP, ACTIVATE or DONE explicitly, leaving activation
  visible in the caller as requested by Barry. Group retained folios,
  deferred frees and the reclaimed count in reclaim_folio_state.
- Patch 3: Return explicit pageout outcomes, including separate locked
  and unlocked keep results. Leave activation and the subsequent reclaim
  attempt in the caller. Document lock and ownership contracts and make
  the mapping pointer and saved page count const.
- Patch 4: Retain Barry's Reviewed-by.
- Patch 5: Use the explicit helper results and reclaim_folio_state,
  reducing the batch helper from eight arguments to six in response to
  David's suggestion.
- Patch 5: Replace v5's temporary pointer array with batch reuse. Save
  the original count, reinitialize the batch, and use folio_batch_add()
  to collect folios that pass validation. Document that reinitialization
  preserves the array and additions cannot overwrite unread entries.
- Patch 5: Explain the writeback, mapping and DMA-pin rechecks, including
  why the folio lock alone does not exclude GUP. Rename the batch and
  helper around pageout and remove the redundant next label.
- Link to v5:
  https://patch.msgid.link/20260720-batch-tlb-flush-v5-0-db943a0d0d6b@icloud.com

Changes in v5 (addressing David Hildenbrand's review on v4):
- Patch 1: Drop the nr_pages parameter; use folio_nr_pages() inside
  folio_activate_locked(). Add VM_WARN_ON_ONCE_FOLIO(!folio_test_locked)
  at the top of the function. Turn the VM_BUG_ON_FOLIO(folio_test_active)
  into VM_WARN_ON_ONCE_FOLIO and move it to the top.
- Patch 2: Rename folio_free() to folio_try_reclaim_free() to make the
  reclaim-specific role clear and justify the bool return. Make nr_pages
  const. Add VM_WARN_ON_ONCE_FOLIO(folio_ref_count, folio) at the free_it:
  label. Drop the now-redundant "swapped out as a whole" comment and the
  "else continue" after goto keep_locked.
- Patch 3: Rename pageout_one() to folio_try_pageout() for consistency
  with the other helpers. Fix continuation-line alignment.
- Patch 4: Fix continuation-line alignment.
- Patch 5: Replace the in-place folio_batch_reinit() + walk pattern
  with a plain local array; @fbatch is now only read during the walk
  and cleared once afterwards, so its invariant is never temporarily
  broken. Add a comment documenting why each recheck (writeback, mapped,
  dma_pinned) is required.
- Link to v4: https://lore.kernel.org/r/20260525-batch-tlb-flush-v4-0-83789d6abc00@icloud.com

Changes in v4 (addressing Barry Song's review on v3):
- Drop the "track reclaimed pages in reclaim_stat" patch; keep
  shrink_folio_list() returning nr_reclaimed directly. Avoids touching
  the function signature and its MGLRU evict_folios() and
  reclaim_clean_pages_from_list() callers in this series.
- Rename folio_active_bounce() to folio_activate_locked(). The new name
  reflects the precondition (the folio is locked) that callers care about.
- Split the folio_free()/pageout_one() extraction into two patches;
  make pageout_one() return bool so shrink_folio_list() can see whether
  the folio was reclaimed or kept.
- Move the !folio_mapped() check out of folio_try_unmap() into the
  caller, so folio_try_unmap() is only invoked for mapped folios.
- Link to v3: https://lore.kernel.org/r/20260410-batch-tlb-flush-v3-0-ff0b9d3a351a@icloud.com

Changes in v3:
- Patch 5: Replace folio_test_lru() condition check with
  VM_WARN_ON_FOLIO assertion, as PG_lru should never be set for
  isolated folios
- Patch 5: Add comment explaining folio_batch reuse-in-place
  technique in pageout_batch()
- Patch 5: Rewrite comment above folio_unlock() to explain why the
  folio is unlocked while batching
- Link to v2: https://lore.kernel.org/r/20260326-batch-tlb-flush-v2-0-403e523325c4@icloud.com

Changes in v2:
- Fix incorrect comment about page_ref_freeze
- Add folio_maybe_dma_pinned() check in pageout_batch()
- Link to v1: https://lore.kernel.org/r/20260309-batch-tlb-flush-v1-0-eb8fed7d1a9e@icloud.com

---
Zhang Peng (5):
      mm/vmscan: introduce folio_activate_locked() helper
      mm/vmscan: extract folio reclaim freeing from shrink_folio_list()
      mm/vmscan: extract folio pageout from shrink_folio_list()
      mm/vmscan: extract folio unmap logic into folio_try_unmap()
      mm/vmscan: batch dirty TLB flushes before pageout

 mm/vmscan.c | 510 +++++++++++++++++++++++++++++++++++++++---------------------
 1 file changed, 335 insertions(+), 175 deletions(-)
---
base-commit: 9c449f885765cf8d80b2c60b1971e774bba51a3f
change-id: 20260309-batch-tlb-flush-893f0e56b496

Best regards,
-- 
Zhang Peng <bruzzhang@tencent.com>