[PATCH 0/6] mm: batch rmap walks during large folio migration

[Shivank Garg <shivankg@amd.com>]

Migrating a PTE-mapped large folio currently walks the folio's rmap
one page at a time. It does so twice per migration:
1. try_to_migrate_one(): Replace present PTEs with migration entries
2. remove_migration_pte(): Restore migration entries to working PTEs.

For a folio with N contiguous PTEs, each walk repeats page_vma_mapped_walk()
and the associated operation per subpage of the folio. (256 iterations
for 1M folio).

This series enables both walks to process a batch of consecutive PTEs
that map the same large folio and do some refactoring and clean up on
the way.

Performance:

AMD EPYC ZEN 3, 2-socket system (1 node per socket).
move_pages() migrating 1 GB from node 0 -> node 1, throughput
in GB/s:
		         batch-	        batch-restore +
folio     vanilla        restore        unmap          restore%  restore+unmap%
4K        2.98±0.09      2.95±0.07      3.06±0.05        -1.0%    +2.6%
64K       6.16±0.09      6.24±0.08      7.80±0.08        +1.3%   +26.6%
256K      6.85±0.06      6.85±0.09      9.54±0.19        +0.0%   +39.3%
1M        7.06±0.09      7.17±0.09      9.89±0.06        +1.5%   +40.1%
2M       10.97±0.17     10.67±0.13     10.69±0.15        -2.7%    -2.6%

In plain migration, the folio copy itself dominate the cost, so these rmap walk
savings are only partly visible end-to-end.
When the folio copy is batched-copy and offloaded to DMA engine [1], the copy is
no longer the bottlneck and rmap-walk batching translates into much larger gains:

This series + [1], DMA Offload on PTDMA (DCBM), 16 channels:

		     batch-copy
folio   vanilla      -offload(dcbm)  dcbm+restore   dcbm+restore+unmap
4K      3.36 ± 0.10   4.16 ± 0.18     4.10 ± 0.14    4.12 ± 0.13
64K     6.16 ± 0.47   9.63 ± 0.23    10.00 ± 0.33   14.63 ± 0.14
256K    6.70 ± 0.09  12.98 ± 0.18    13.52 ± 0.15   32.30 ± 0.65    (2.38x)
1M      6.99 ± 0.02  14.32 ± 0.67    14.84 ± 0.17   38.97 ± 1.03    (2.62x)
2M     10.74 ± 0.29  64.25 ± 1.48    65.36 ± 1.00   64.97 ± 1.75

[1] https://lore.kernel.org/linux-mm/20260630-shivank-batch-migrate-offload-v6-0-da95d7e8b8a2@amd.com

Applies cleanly on mm-new 10-07-26 (61cccb8363f).

Signed-off-by: Shivank Garg <shivankg@amd.com>
---
Dev Jain (1):
      mm/rmap: Add batched version of folio_try_share_anon_rmap_pte

Shivank Garg (5):
      mm: factor out generic PTE batch detection from swap_pte_batch()
      mm/migrate: split remove_migration_pte_hugetlb() out of remove_migration_pte()
      mm/migrate: batch the restore-side migration rmap walk
      mm/rmap: split try_to_migrate_hugetlb_one() out of try_to_migrate_one()
      mm/rmap: batch the unmap of large folios in try_to_migrate_one()

 include/linux/rmap.h |  52 +++++--
 mm/internal.h        |  36 +++--
 mm/migrate.c         | 190 ++++++++++++++++-------
 mm/rmap.c            | 427 +++++++++++++++++++++++++++++++++------------------
 4 files changed, 464 insertions(+), 241 deletions(-)
---
base-commit: 61cccb8363fcc282d4ae0555b8739dd227f5ad0b
change-id: 20260701-migrate-rmap-batch-8ecc04de1905

Best regards,
-- 
Shivank Garg <shivankg@amd.com>