mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v3 0/3] mm/hugetlb_vmemmap: Fix HVO races and deadlock
@ 2026-11-14 11:25 Muchun Song
  2026-11-14 11:25 ` [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization Muchun Song
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Muchun Song @ 2026-11-14 11:25 UTC (permalink / raw)
  To: Oscar Salvador, Andrew Morton, Miaohe Lin
  Cc: Muchun Song, David Hildenbrand, Mike Rapoport, Naoya Horiguchi,
	Joao Martins, Lorenzo Stoakes, Sashiko, Jiaqi Yan,
	Jose A . Perez de Azpillaga, linux-mm, linux-kernel, stable,
	Muchun Song

This series fixes three HVO issues reported by Sashiko.

The first patch serializes HWPoison updates and removal against HVO
remapping so raw_hwp_page state cannot be lost or freed while a
replacement vmemmap head page is being installed. The second patch uses
an HVO-private rwsem to prevent deferred PMD splits from racing with PTE
remapping before the TLB flush. The last patch switches the vmemmap walk
to the lockless walker so a PMD split allocation cannot recurse into
init_mm.mmap_lock through reclaim.

v3:
- Serialize unpoison_memory() against HVO head remapping.
- Update the first patch subject and description for the unpoison race.
- Collect Reviewed-by tags from Jiaqi Yan and
  Jose A. Perez de Azpillaga.
- Rebase onto mm/mm-new.

v2: https://lore.kernel.org/20261111093747.50048-1-songmuchun@bytedance.com/
- Add the HWPoison serialization fix reported by Sashiko.
- Use an HVO-private rwsem for deferred PMD split serialization.
- Add the lockless page-table walk fix for the PMD split deadlock.

v1: https://lore.kernel.org/20261110112456.62827-1-songmuchun@bytedance.com/

Muchun Song (3):
  mm/hugetlb_vmemmap: fix HWPoison state races during optimization
  mm/hugetlb_vmemmap: fix PMD split TLB flush race
  mm/hugetlb_vmemmap: fix deadlock during PMD split

 mm/hugetlb_vmemmap.c | 28 +++++++++++++++++++++++-----
 mm/hugetlb_vmemmap.h | 24 ++++++++++++++++++++++++
 mm/memory-failure.c  |  3 +++
 3 files changed, 50 insertions(+), 5 deletions(-)


base-commit: 6ec534d64f85e949a5d58070678dcd7711875a7c
-- 
2.54.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization
  2026-11-14 11:25 [PATCH v3 0/3] mm/hugetlb_vmemmap: Fix HVO races and deadlock Muchun Song
@ 2026-11-14 11:25 ` Muchun Song
  2026-11-16  9:02   ` Oscar Salvador (SUSE)
  2026-11-14 11:25 ` [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race Muchun Song
  2026-11-14 11:25 ` [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split Muchun Song
  2 siblings, 1 reply; 7+ messages in thread
From: Muchun Song @ 2026-11-14 11:25 UTC (permalink / raw)
  To: Oscar Salvador, Andrew Morton, Miaohe Lin
  Cc: Muchun Song, David Hildenbrand, Mike Rapoport, Naoya Horiguchi,
	Joao Martins, Lorenzo Stoakes, Sashiko, Jiaqi Yan,
	Jose A . Perez de Azpillaga, linux-mm, linux-kernel, stable,
	Muchun Song

HVO copies the vmemmap head page before remapping it to a newly allocated
backing page. Memory failure can update the old head page after the copy
but before the remap's TLB invalidation completes:

CPU 0                            CPU 1
init_new_hugetlb_folio()
__hugetlb_vmemmap_optimize_folio()
  vmemmap_optimize_pte()
    copy_page(old_head, new_head)
                                 try_memory_failure_hugetlb()
                                   get_huge_page_for_hwpoison()
                                     hugetlb_update_hwpoison(old_head)
  set_pte_at(..., new_head)
  flush_tlb_all()
free_vmemmap_page_list(old_head)

The raw_hwp_page linked to the old head is then leaked and the folio's
HWPoison state is lost when the obsolete backing page is freed, allowing
the poisoned huge page to be allocated later.

Unpoisoning has a similar race: unpoison_memory() can free the
raw_hwp_page and clear PG_hwpoison from the old head after it is copied,
leaving the new head with stale HWPoison state and a freed pointer.

Serialize HVO remapping against HWPoison updates and removal. This
ensures that an update is either copied to the new vmemmap head page or
applied after HVO installs the new mapping and completes the TLB flush,
and prevents unpoisoning from modifying the old head while HVO installs
its replacement.

Fixes: 11aad2631bf7 ("mm/hugetlb_vmemmap: remap head page to newly allocated page")
Cc: stable@vger.kernel.org
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/bug/linux-18e93f17-529b-4d84-ba44-a41829f4be64
Reviewed-by: Jiaqi Yan <jiaqiyan@google.com>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
---
v3:
- Serialize unpoison_memory() against HVO head remapping (suggested by
  Sashiko)
- Reword the subject and commit message for the extended race coverage
- Collect Reviewed-by from Jiaqi Yan and Jose A. Perez de Azpillaga
---
 mm/hugetlb_vmemmap.c | 10 ++++++++++
 mm/hugetlb_vmemmap.h | 24 ++++++++++++++++++++++++
 mm/memory-failure.c  |  3 +++
 3 files changed, 37 insertions(+)

diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index ba8e11e399b2..79ca68dff635 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -15,6 +15,7 @@
 #include <linux/mmdebug.h>
 #include <linux/pagewalk.h>
 #include <linux/pgalloc.h>
+#include <linux/rwsem.h>
 #include <linux/vmemmap-optimization.h>
 
 #include <asm/tlbflush.h>
@@ -41,6 +42,8 @@ struct vmemmap_remap_walk {
 	unsigned long		flags;
 };
 
+DECLARE_RWSEM(vmemmap_remap_rwsem);
+
 static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,
 			     struct vmemmap_remap_walk *walk)
 {
@@ -333,6 +336,9 @@ static int vmemmap_pteify_folio(const struct folio *folio, unsigned long flags)
  * responsible for freeing the pages left on the @vmemmap_pages list, even on
  * failure.
  *
+ * Note: The caller must hold vmemmap_remap_rwsem. If TLB flushing is deferred,
+ * the lock must remain held until the final TLB flush.
+ *
  * Return: %0 on success, negative error code otherwise.
  */
 static int vmemmap_optimize_folio(const struct folio *folio,
@@ -348,6 +354,8 @@ static int vmemmap_optimize_folio(const struct folio *folio,
 		.flags		= flags,
 	};
 
+	rwsem_assert_held(&vmemmap_remap_rwsem);
+
 	VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio), folio);
 
 	if (!vmemmap_optimizable_order(folio_order(folio)))
@@ -536,6 +544,7 @@ void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *folio)
 {
 	LIST_HEAD(vmemmap_pages);
 
+	guard(rwsem_read)(&vmemmap_remap_rwsem);
 	__hugetlb_vmemmap_optimize_folio(h, folio, &vmemmap_pages, 0);
 	free_vmemmap_page_list(&vmemmap_pages);
 }
@@ -588,6 +597,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 
 	flush_tlb_all();
 
+	guard(rwsem_read)(&vmemmap_remap_rwsem);
 	list_for_each_entry(folio, folio_list, lru) {
 		int ret;
 
diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h
index d057e8b12bf1..8b28063d78cc 100644
--- a/mm/hugetlb_vmemmap.h
+++ b/mm/hugetlb_vmemmap.h
@@ -9,9 +9,12 @@
 #ifndef _LINUX_HUGETLB_VMEMMAP_H
 #define _LINUX_HUGETLB_VMEMMAP_H
 #include <linux/hugetlb.h>
+#include <linux/rwsem.h>
 #include <linux/vmemmap-optimization.h>
 
 #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
+extern struct rw_semaphore vmemmap_remap_rwsem;
+
 int hugetlb_vmemmap_restore_folio(const struct hstate *h, struct folio *folio);
 long hugetlb_vmemmap_restore_folios(const struct hstate *h,
 					struct list_head *folio_list,
@@ -20,6 +23,16 @@ void hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *folio)
 void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list);
 void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsigned int order);
 
+static inline void hugetlb_vmemmap_write_lock(void)
+{
+	down_write(&vmemmap_remap_rwsem);
+}
+
+static inline void hugetlb_vmemmap_write_unlock(void)
+{
+	up_write(&vmemmap_remap_rwsem);
+}
+
 static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h)
 {
 	return pages_per_huge_page(h) * sizeof(struct page);
@@ -59,6 +72,14 @@ static inline void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list
 {
 }
 
+static inline void hugetlb_vmemmap_write_lock(void)
+{
+}
+
+static inline void hugetlb_vmemmap_write_unlock(void)
+{
+}
+
 static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct hstate *h)
 {
 	return 0;
@@ -69,6 +90,9 @@ static inline void hugetlb_vmemmap_optimize_bootmem_page(unsigned long pfn, unsi
 }
 #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */
 
+DEFINE_LOCK_GUARD_0(vmemmap_remap,
+		    hugetlb_vmemmap_write_lock(), hugetlb_vmemmap_write_unlock())
+
 static inline bool hugetlb_vmemmap_optimizable(const struct hstate *h)
 {
 	return hugetlb_vmemmap_optimizable_size(h) != 0;
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 78f0a13349cb..6a7b7be9928d 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -69,6 +69,7 @@
 #include "page_alloc.h"
 #include "internal.h"
 #include "memory-failure.h"
+#include "hugetlb_vmemmap.h"
 
 static int sysctl_memory_failure_early_kill __read_mostly;
 
@@ -2038,6 +2039,7 @@ static int get_huge_page_for_hwpoison(unsigned long pfn, int flags,
 	bool count_increased = false;
 	int ret, rc;
 
+	guard(vmemmap_remap)();
 	spin_lock_irq(&hugetlb_lock);
 	folio = page_folio(page);
 	if (!folio_test_hugetlb(folio)) {
@@ -2812,6 +2814,7 @@ int unpoison_memory(unsigned long pfn)
 
 	ghp = get_hwpoison_page(p, MF_UNPOISON);
 	if (!ghp) {
+		guard(vmemmap_remap)();
 		if (folio_test_hugetlb(folio)) {
 			huge = true;
 			count = folio_free_raw_hwp(folio, false);
-- 
2.54.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race
  2026-11-14 11:25 [PATCH v3 0/3] mm/hugetlb_vmemmap: Fix HVO races and deadlock Muchun Song
  2026-11-14 11:25 ` [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization Muchun Song
@ 2026-11-14 11:25 ` Muchun Song
  2026-11-16 10:08   ` Oscar Salvador (SUSE)
  2026-11-14 11:25 ` [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split Muchun Song
  2 siblings, 1 reply; 7+ messages in thread
From: Muchun Song @ 2026-11-14 11:25 UTC (permalink / raw)
  To: Oscar Salvador, Andrew Morton, Miaohe Lin
  Cc: Muchun Song, David Hildenbrand, Mike Rapoport, Naoya Horiguchi,
	Joao Martins, Lorenzo Stoakes, Sashiko, Jiaqi Yan,
	Jose A . Perez de Azpillaga, linux-mm, linux-kernel, stable,
	Muchun Song

Bulk HVO optimization converts vmemmap PMDs for a list of folios into
PTE tables before issuing a single global TLB flush. After publishing
a PTE table, a concurrent single-folio optimization can observe it and
remap its PTEs before the deferred flush:

CPU0                             CPU1
hugetlb_vmemmap_optimize_folios()
  hugetlb_vmemmap_pteify_folio()
    vmemmap_split_pmd()
      pmd_populate_kernel()
      /* TLB flush deferred */
                                 hugetlb_vmemmap_optimize_folio()
                                   vmemmap_pmd_entry()
                                   /* sees a PTE table */
                                   vmemmap_optimize_pte()
                                     set_pte_at()
  flush_tlb_all()

Modern MMUs permit overlapping TLB entries with different mapping
granularities only when they translate to the same physical address. In
this race, the stale PMD translation still points at the original vmemmap
backing range while the new PTE can point at a shared or replacement
vmemmap page for the same virtual address.

Serialize deferred PMD splitting against PTE remapping with the HVO
rwsem. Hold it for write across the split-only pteify batch and its TLB
flush, then downgrade it before remapping the folios' vmemmap pages.
This guarantees that no PTE remapper can observe a newly split PMD until
the stale PMD translations have been invalidated.

Fixes: f4b7e3efaddb ("hugetlb: batch PMD split for bulk vmemmap dedup")
Cc: stable@vger.kernel.org
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/bug/linux-fbdc3410-a4cf-457e-88e6-684380fbadfe
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
---
v3:
- Collect Reviewed-by from Jose A. Perez de Azpillaga
---
 mm/hugetlb_vmemmap.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index 79ca68dff635..6fba3184356c 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -311,6 +311,9 @@ static inline int vmemmap_restore_range(unsigned long start, unsigned long end,
  * is skipped, the caller must flush TLBs after all pteify operations are complete
  * before remapping the folio's vmemmap pages.
  *
+ * Note: The caller must hold vmemmap_remap_rwsem for write from the first
+ * pteify operation through the final TLB flush.
+ *
  * Return: %0 on success, negative error code otherwise.
  */
 static int vmemmap_pteify_folio(const struct folio *folio, unsigned long flags)
@@ -321,6 +324,8 @@ static int vmemmap_pteify_folio(const struct folio *folio, unsigned long flags)
 		.flags		= flags,
 	};
 
+	rwsem_assert_held_write(&vmemmap_remap_rwsem);
+
 	return vmemmap_remap_range(start, end, &walk);
 }
 
@@ -400,6 +405,7 @@ static int vmemmap_restore_folio(const struct folio *folio, unsigned long flags)
 	if (ret)
 		return ret;
 
+	guard(rwsem_read)(&vmemmap_remap_rwsem);
 	return vmemmap_restore_range(start, end, &vmemmap_pages, flags);
 }
 
@@ -568,6 +574,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 	if (list_empty(folio_list))
 		return;
 
+	down_write(&vmemmap_remap_rwsem);
 	list_for_each_entry(folio, folio_list, lru) {
 		int ret;
 
@@ -592,12 +599,14 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 			break;
 	}
 
-	if (!nr_to_optimize)
+	if (!nr_to_optimize) {
+		up_write(&vmemmap_remap_rwsem);
 		return;
+	}
 
 	flush_tlb_all();
+	downgrade_write(&vmemmap_remap_rwsem);
 
-	guard(rwsem_read)(&vmemmap_remap_rwsem);
 	list_for_each_entry(folio, folio_list, lru) {
 		int ret;
 
@@ -620,6 +629,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
 	}
 
 	flush_tlb_all();
+	up_read(&vmemmap_remap_rwsem);
 	free_vmemmap_page_list(&vmemmap_pages);
 }
 
-- 
2.54.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split
  2026-11-14 11:25 [PATCH v3 0/3] mm/hugetlb_vmemmap: Fix HVO races and deadlock Muchun Song
  2026-11-14 11:25 ` [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization Muchun Song
  2026-11-14 11:25 ` [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race Muchun Song
@ 2026-11-14 11:25 ` Muchun Song
  2026-11-16 10:10   ` Oscar Salvador (SUSE)
  2 siblings, 1 reply; 7+ messages in thread
From: Muchun Song @ 2026-11-14 11:25 UTC (permalink / raw)
  To: Oscar Salvador, Andrew Morton, Miaohe Lin
  Cc: Muchun Song, David Hildenbrand, Mike Rapoport, Naoya Horiguchi,
	Joao Martins, Lorenzo Stoakes, Sashiko, Jiaqi Yan,
	Jose A . Perez de Azpillaga, linux-mm, linux-kernel, stable,
	Muchun Song

vmemmap_remap_range() currently walks vmemmap page tables under
init_mm.mmap_lock. If the walk reaches a leaf PMD, vmemmap_split_pmd()
allocates a PTE table with GFP_KERNEL and can enter direct reclaim:

hugetlb_vmemmap_optimize_folio()
  vmemmap_remap_range()
    mmap_read_lock(&init_mm)  <-- outer read lock
    ...
      vmemmap_split_pmd()
        pte_alloc_one_kernel(&init_mm)  /* GFP_KERNEL */
          <direct reclaim>
            ttm_pool_shrinker_scan()
              ...
                change_page_attr_set_clr()
                  mmap_read_lock(&init_mm)  <-- self-deadlock

Since commit d5d8b8662e6e ("x86/mm/pat: Acquire init_mm read lock on
attribute changes to avoid UAF"), reclaim through TTM can enter CPA,
which also takes init_mm.mmap_lock. If a writer is pending, the nested
read blocks behind it while the outer read cannot be released, causing a
self-deadlock.

HVO walks vmemmap ranges whose intermediate page tables remain present.
vmemmap_split_pmd() revalidates the PMD under init_mm.page_table_lock
before installing the PTE table. Use the lockless page-table walker so the
allocation no longer runs under init_mm.mmap_lock.

Fixes: d5d8b8662e6e ("x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF")
Cc: stable@vger.kernel.org
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/sashiko-outbox-206969@kernel.org/
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
---
v3:
- Collect Reviewed-by from Jose A. Perez de Azpillaga
---
 mm/hugetlb_vmemmap.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index 6fba3184356c..5852b0a1c479 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -161,10 +161,8 @@ static int vmemmap_remap_range(unsigned long start, unsigned long end,
 
 	VM_BUG_ON(!PAGE_ALIGNED(start | end));
 
-	mmap_read_lock(&init_mm);
-	ret = walk_kernel_page_table_range(start, end, &vmemmap_remap_ops,
-				    NULL, walk);
-	mmap_read_unlock(&init_mm);
+	ret = walk_kernel_page_table_range_lockless(start, end, &vmemmap_remap_ops,
+						    NULL, walk);
 	if (ret)
 		return ret;
 
-- 
2.54.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization
  2026-11-14 11:25 ` [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization Muchun Song
@ 2026-11-16  9:02   ` Oscar Salvador (SUSE)
  0 siblings, 0 replies; 7+ messages in thread
From: Oscar Salvador (SUSE) @ 2026-11-16  9:02 UTC (permalink / raw)
  To: Muchun Song
  Cc: Oscar Salvador, Andrew Morton, Miaohe Lin, Muchun Song,
	David Hildenbrand, Mike Rapoport, Naoya Horiguchi, Joao Martins,
	Lorenzo Stoakes, Sashiko, Jiaqi Yan, Jose A . Perez de Azpillaga,
	linux-mm, linux-kernel, stable

On Sat, Nov 14, 2026 at 07:25:21PM +0800, Muchun Song wrote:
> HVO copies the vmemmap head page before remapping it to a newly allocated
> backing page. Memory failure can update the old head page after the copy
> but before the remap's TLB invalidation completes:
> 
> CPU 0                            CPU 1
> init_new_hugetlb_folio()
> __hugetlb_vmemmap_optimize_folio()
>   vmemmap_optimize_pte()
>     copy_page(old_head, new_head)
>                                  try_memory_failure_hugetlb()
>                                    get_huge_page_for_hwpoison()
>                                      hugetlb_update_hwpoison(old_head)
>   set_pte_at(..., new_head)
>   flush_tlb_all()
> free_vmemmap_page_list(old_head)
> 
> The raw_hwp_page linked to the old head is then leaked and the folio's
> HWPoison state is lost when the obsolete backing page is freed, allowing
> the poisoned huge page to be allocated later.
> 
> Unpoisoning has a similar race: unpoison_memory() can free the
> raw_hwp_page and clear PG_hwpoison from the old head after it is copied,
> leaving the new head with stale HWPoison state and a freed pointer.
> 
> Serialize HVO remapping against HWPoison updates and removal. This
> ensures that an update is either copied to the new vmemmap head page or
> applied after HVO installs the new mapping and completes the TLB flush,
> and prevents unpoisoning from modifying the old head while HVO installs
> its replacement.
> 
> Fixes: 11aad2631bf7 ("mm/hugetlb_vmemmap: remap head page to newly allocated page")
> Cc: stable@vger.kernel.org
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://sashiko.dev/#/bug/linux-18e93f17-529b-4d84-ba44-a41829f4be64
> Reviewed-by: Jiaqi Yan <jiaqiyan@google.com>
> Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

Acked-by: Oscar Salvador (SUSE) <osalvador@kernel.org>

 

-- 
Oscar Salvador
SUSE Labs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race
  2026-11-14 11:25 ` [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race Muchun Song
@ 2026-11-16 10:08   ` Oscar Salvador (SUSE)
  0 siblings, 0 replies; 7+ messages in thread
From: Oscar Salvador (SUSE) @ 2026-11-16 10:08 UTC (permalink / raw)
  To: Muchun Song
  Cc: Oscar Salvador, Andrew Morton, Miaohe Lin, Muchun Song,
	David Hildenbrand, Mike Rapoport, Naoya Horiguchi, Joao Martins,
	Lorenzo Stoakes, Sashiko, Jiaqi Yan, Jose A . Perez de Azpillaga,
	linux-mm, linux-kernel, stable

On Sat, Nov 14, 2026 at 07:25:22PM +0800, Muchun Song wrote:
> Bulk HVO optimization converts vmemmap PMDs for a list of folios into
> PTE tables before issuing a single global TLB flush. After publishing
> a PTE table, a concurrent single-folio optimization can observe it and
> remap its PTEs before the deferred flush:
> 
> CPU0                             CPU1
> hugetlb_vmemmap_optimize_folios()
>   hugetlb_vmemmap_pteify_folio()
>     vmemmap_split_pmd()
>       pmd_populate_kernel()
>       /* TLB flush deferred */
>                                  hugetlb_vmemmap_optimize_folio()
>                                    vmemmap_pmd_entry()
>                                    /* sees a PTE table */
>                                    vmemmap_optimize_pte()
>                                      set_pte_at()
>   flush_tlb_all()
> 
> Modern MMUs permit overlapping TLB entries with different mapping
> granularities only when they translate to the same physical address. In
> this race, the stale PMD translation still points at the original vmemmap
> backing range while the new PTE can point at a shared or replacement
> vmemmap page for the same virtual address.
> 
> Serialize deferred PMD splitting against PTE remapping with the HVO
> rwsem. Hold it for write across the split-only pteify batch and its TLB
> flush, then downgrade it before remapping the folios' vmemmap pages.
> This guarantees that no PTE remapper can observe a newly split PMD until
> the stale PMD translations have been invalidated.
> 
> Fixes: f4b7e3efaddb ("hugetlb: batch PMD split for bulk vmemmap dedup")
> Cc: stable@vger.kernel.org
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://sashiko.dev/#/bug/linux-fbdc3410-a4cf-457e-88e6-684380fbadfe
> Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

Acked-by: Oscar Salvador (SUSE) <osalvador@kernel.org>

 

-- 
Oscar Salvador
SUSE Labs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split
  2026-11-14 11:25 ` [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split Muchun Song
@ 2026-11-16 10:10   ` Oscar Salvador (SUSE)
  0 siblings, 0 replies; 7+ messages in thread
From: Oscar Salvador (SUSE) @ 2026-11-16 10:10 UTC (permalink / raw)
  To: Muchun Song
  Cc: Oscar Salvador, Andrew Morton, Miaohe Lin, Muchun Song,
	David Hildenbrand, Mike Rapoport, Naoya Horiguchi, Joao Martins,
	Lorenzo Stoakes, Sashiko, Jiaqi Yan, Jose A . Perez de Azpillaga,
	linux-mm, linux-kernel, stable

On Sat, Nov 14, 2026 at 07:25:23PM +0800, Muchun Song wrote:
> vmemmap_remap_range() currently walks vmemmap page tables under
> init_mm.mmap_lock. If the walk reaches a leaf PMD, vmemmap_split_pmd()
> allocates a PTE table with GFP_KERNEL and can enter direct reclaim:
> 
> hugetlb_vmemmap_optimize_folio()
>   vmemmap_remap_range()
>     mmap_read_lock(&init_mm)  <-- outer read lock
>     ...
>       vmemmap_split_pmd()
>         pte_alloc_one_kernel(&init_mm)  /* GFP_KERNEL */
>           <direct reclaim>
>             ttm_pool_shrinker_scan()
>               ...
>                 change_page_attr_set_clr()
>                   mmap_read_lock(&init_mm)  <-- self-deadlock
> 
> Since commit d5d8b8662e6e ("x86/mm/pat: Acquire init_mm read lock on
> attribute changes to avoid UAF"), reclaim through TTM can enter CPA,
> which also takes init_mm.mmap_lock. If a writer is pending, the nested
> read blocks behind it while the outer read cannot be released, causing a
> self-deadlock.
> 
> HVO walks vmemmap ranges whose intermediate page tables remain present.
> vmemmap_split_pmd() revalidates the PMD under init_mm.page_table_lock
> before installing the PTE table. Use the lockless page-table walker so the
> allocation no longer runs under init_mm.mmap_lock.
> 
> Fixes: d5d8b8662e6e ("x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF")
> Cc: stable@vger.kernel.org
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://lore.kernel.org/sashiko-outbox-206969@kernel.org/
> Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

Acked-by: Oscar Salvador (SUSE) <osalvador@kernel.org>

 

-- 
Oscar Salvador
SUSE Labs

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-11-16 10:10 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-14 11:25 [PATCH v3 0/3] mm/hugetlb_vmemmap: Fix HVO races and deadlock Muchun Song
2026-11-14 11:25 ` [PATCH v3 1/3] mm/hugetlb_vmemmap: fix HWPoison state races during optimization Muchun Song
2026-11-16  9:02   ` Oscar Salvador (SUSE)
2026-11-14 11:25 ` [PATCH v3 2/3] mm/hugetlb_vmemmap: fix PMD split TLB flush race Muchun Song
2026-11-16 10:08   ` Oscar Salvador (SUSE)
2026-11-14 11:25 ` [PATCH v3 3/3] mm/hugetlb_vmemmap: fix deadlock during PMD split Muchun Song
2026-11-16 10:10   ` Oscar Salvador (SUSE)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®