mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v5 0/12] mm: fix allocator races with memory hotplug
@ 2026-11-01  2:13 Gregory Price (Meta)
  2026-11-01  2:13 ` [PATCH v5 01/12] mm/page_alloc: reset can_retry_reserves when the slowpath restarts Gregory Price (Meta)
                   ` (12 more replies)
  0 siblings, 13 replies; 14+ messages in thread
From: Gregory Price (Meta) @ 2026-11-01  2:13 UTC (permalink / raw)
  To: linux-mm
  Cc: linux-trace-kernel, linux-kernel, kernel-team, akpm, david, ljs,
	liam, vbabka, rppt, surenb, mhocko, kasong, qi.zheng,
	shakeel.butt, baohua, axelrasmussen, yuanchu, weixugc,
	baoquan.he, baolin.wang, rientjes, rostedt, mhiramat,
	mathieu.desnoyers, muchun.song, osalvador, gourry, ziy,
	joshua.hahnjy, rakie.kim, ying.huang, brendan.jackman, hannes,
	bigeasy, clrkwllms, nathan, ndesaulniers, morbo, justinstitt,
	Bessel.Wang, jiang.liu, mgorman, anshuman.khandual

Memory hotplug rebuilds zonelists in place while allocations walk them,
and offlining an emptied zone frees pagesets the vmstat worker can still
hold.

These eleven fixes cover the walkers that crash, fail, or pick the wrong
zone or node as a result, and a twelfth patch removes a helper that
invites the same bug.

   1. a restarted slowpath loses its retry with reserves
   2. offlining an emptied zone frees pagesets still in use
   3. the reclaim_retry_zone tracepoint rereads a zoneref slot
   4. the zonelist retry is compiled out without MEMORY_HOTREMOVE
   5. zonelist_zone() may load its slot more than once
   6. NULL preferred zone after a shrink rebuild
   7. __next_zones_zonelist() rereads a slot
   8. allocations land outside their zone or nodemask
   9. the hugetlb pool walk takes another node's page
  10. alloc_contig_pages() takes a range outside its filters
  11. mempolicy_slab_node() rereads a slot
  12. zonelist_node_idx() invites the same double read

Patches 1-11 are for stable.  Patch 12 is a cleanup and is not.

Patches 8-10 need patch 4, without which their retry is compiled
out on HOTREMOVE=n kernels.

Patch 8 needs patch 1.

Patches 6, 7 and 11 need patch 5, which makes zonelist_zone() read
the slot once.  Patch 12 needs patches 7 and 11, which remove the
other callers of zonelist_node_idx().

Patches 2 and 3 are patches from anthropic, pulled in from Vlastimil's
series to keep the backport issues self-contained on the series.

Testing
=======

Each race was reproduced on unmodified source in a QEMU guest with three
NUMA nodes and a 512M pc-dimm on node 1, left offline at boot.

A test module widens each window with a kprobe that stalls one CPU inside
it, as an interrupt or a vCPU preemption would.

       base                            this series
patch  outcome                  runs   outcome                  runs
=====  =======================  =====  =======================  =====
 1     NULL returned            3/3    page from reserves       3/3
 2     UAF                      6/6    no UAF                   3/3
 3     NULL dereference         3/3    allocation succeeds      3/3
 4     allocation fails         3/3    waits for the rebuild    3/3
 6     NULL dereference         3/3    ZONE_NORMAL page         3/3
 7     NULL dereference         3/3    ZONE_NORMAL page         3/3
 8     wrong zone or node       18/18  right zone and node      18/18
 9     page from node 1         3/3    page from node 2         3/3
10     1G page in ZONE_MOVABLE  3/3    1G page in ZONE_NORMAL   3/3
11     NULL dereference         4/4    keeps allocating         3/3


Cost (these tests were run on v4, v5 hasn't substantially changed)
====
Kernel instructions per order-0 allocation and free in the QEMU guest,
counted with a task-bound hardware instruction counter around 1M
iterations on a pinned kthread, non-debug kernel, mean of 15 rounds.

                           instructions per alloc+free
                             base   this series
  alloc_pages()             890.5         900.5
  bulk, nr=1, nodemask      865.5         875.5
  bulk, nr=16, per page     477.2         478.1


Bare metal, two-socket Xeon Gold 6138 (Skylake-SP, 80 CPUs).

will-it-scale page_fault1-3: each iteration touches one page, so
the unit is millions of faults per second, summed over all tasks.

Base, millions of faults per second:

  tasks                         1     10     40     80
  page_fault1, processes     0.46   4.16  10.51   9.23
  page_fault1, threads       0.46   3.51   4.35   4.19
  page_fault2, processes     0.43   3.66   9.02   8.33
  page_fault2, threads       0.43   3.48   4.37   4.19
  page_fault3, processes     3.89  14.78   8.10   8.00
  page_fault3, threads       3.87  14.29   6.88   7.96

Change in faults per second with this series, + is faster:

  tasks                         1     10     40     80
  page_fault1, processes    -0.3%  +0.1%  -3.0%  -1.7%
  page_fault1, threads      -0.5%  -1.6%  -0.9%  -0.9%
  page_fault2, processes    +0.0%  +0.9%  -1.4%  -0.5%
  page_fault2, threads      +0.3%  +1.5%  -1.4%  -1.0%
  page_fault3, processes    -1.0%  -3.8%  -0.7%  +1.2%
  page_fault3, threads      -0.9%  -0.7%  +0.8%  -0.2%

The boot-to-boot stddev is 1.0-1.8% at 1 task and up to
15% in 10-80 tasks, so basically this is all within noise.

Changes in v5
=============
- Pulled in patches from Vlastimil's series (patches 2 and 3 here)
- inline functions (Vlastimil)
- Added patch 5: zonelist_zone() reads the slot with READ_ONCE(), as
  recommended by sashiko on v4's mempolicy patch.
  Patches 6, 7 and 11 call it instead of open-coding the load.
- Added patch 12, removing zonelist_node_idx().  Not for stable.
- Patch 1: reworded the changelog (Vlastimil).
- Collected Reviewed-by tags.

Link: https://lore.kernel.org/linux-mm/20261028211930.3504634-1-gourry@gourry.net/
Link: https://lore.kernel.org/linux-mm/20261027-pgalloc-ai-hotfixes-v1-0-9c7d1aca801e@kernel.org/
Link: https://lore.kernel.org/linux-mm/20261014222456.2318284-1-gourry@gourry.net/
Link: https://lore.kernel.org/linux-mm/20261015210539.3243563-1-gourry@gourry.net/
Link: https://lore.kernel.org/linux-mm/20261019172346.3344005-1-gourry@gourry.net/

Gregory Price (Meta) (10):
  mm/page_alloc: reset can_retry_reserves when the slowpath restarts
  mm/page_alloc: detect zonelist rebuilds from memory hot-add
  mm/mmzone: read the zoneref slot once in zonelist_zone()
  mm/page_alloc: snapshot the preferred zone against shrink rebuilds
  mm/mmzone: read each zoneref slot once when filtering by nodemask
  mm/page_alloc: retry allocations that raced with a zonelist rebuild
  mm/hugetlb: retry the pool walk when the zonelist is rebuilt
  mm/page_alloc: retry alloc_contig_pages() when the zonelist is rebuilt
  mm/mempolicy: read the zoneref slot once in mempolicy_slab_node()
  mm/mmzone: remove zonelist_node_idx()

Josef Bacik (2):
  mm/page_alloc: keep the pagesets of an empty zone
  mm, page_alloc: pass zone pointer to trace_reclaim_retry_zone

 include/linux/mmzone.h     |   9 +---
 include/trace/events/oom.h |   8 +--
 mm/hugetlb.c               |  13 +++--
 mm/mempolicy.c             |   6 ++-
 mm/mmzone.c                |  11 ++--
 mm/page_alloc.c            | 107 +++++++++++++++++++++++++++----------
 mm/page_alloc.h            |  19 +++++++
 7 files changed, 123 insertions(+), 50 deletions(-)

-- 
2.56.0


^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2026-11-01 11:00 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-01  2:13 [PATCH v5 0/12] mm: fix allocator races with memory hotplug Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 01/12] mm/page_alloc: reset can_retry_reserves when the slowpath restarts Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 02/12] mm/page_alloc: keep the pagesets of an empty zone Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 03/12] mm, page_alloc: pass zone pointer to trace_reclaim_retry_zone Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 04/12] mm/page_alloc: detect zonelist rebuilds from memory hot-add Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 05/12] mm/mmzone: read the zoneref slot once in zonelist_zone() Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 06/12] mm/page_alloc: snapshot the preferred zone against shrink rebuilds Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 07/12] mm/mmzone: read each zoneref slot once when filtering by nodemask Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 08/12] mm/page_alloc: retry allocations that raced with a zonelist rebuild Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 09/12] mm/hugetlb: retry the pool walk when the zonelist is rebuilt Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 10/12] mm/page_alloc: retry alloc_contig_pages() " Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 11/12] mm/mempolicy: read the zoneref slot once in mempolicy_slab_node() Gregory Price (Meta)
2026-11-01  2:13 ` [PATCH v5 12/12] mm/mmzone: remove zonelist_node_idx() Gregory Price (Meta)
2026-11-01 11:00 ` [PATCH v5 0/12] mm: fix allocator races with memory hotplug Vlastimil Babka (SUSE)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®