[RFC PATCH] mm/rmap: use release ordering for small-folio mapcount removal

[RFC PATCH] mm/rmap: use release ordering for small-folio mapcount removal

[Pranith Kumar]

Removing a PTE mapping must precede lowering its folio's mapcount.
The COW path relies on this order to keep the old page from being
reused while a stale translation can still access it. The mapcount
update needs release ordering for that edge, rather than the full
ordering of atomic_add_negative().

Use atomic_add_negative_release() for non-large folios removed at PTE
level. Leave large-folio operations fully ordered; their path can
queue a partially mapped folio for deferred splitting. Update the
COW ordering comment to describe the required guarantee.

In an arm64 page map/unmap microbenchmark with 20,480 minor
faults, simulated cycles fell by 1.54%.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
---
 mm/memory.c | 4 ++--
 mm/rmap.c   | 3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index b65b56fe9714..9e23c64bc743 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4158,9 +4158,9 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
 			 * threads.
 			 *
 			 * The critical issue is to order this
-			 * folio_remove_rmap_pte() with the ptp_clear_flush
+			 * folio_remove_rmap_pte() with the ptep_clear_flush
 			 * above. Those stores are ordered by (if nothing else,)
-			 * the barrier present in the atomic_add_negative
+			 * the release ordering of the mapcount decrement
 			 * in folio_remove_rmap_pte();
 			 *
 			 * Then the TLB flush in ptep_clear_flush ensures that
diff --git a/mm/rmap.c b/mm/rmap.c
index 1131b76bbbc2..62e623d7563b 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1794,7 +1794,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio,
 	switch (level) {
 	case PGTABLE_LEVEL_PTE:
 		if (!folio_test_large(folio)) {
-			nr = atomic_add_negative(-1, &folio->_mapcount);
+			/* Publish the PTE removal before lowering mapcount. */
+			nr = atomic_add_negative_release(-1, &folio->_mapcount);
 			break;
 		}
 
-- 
2.47.3