[PATCH v2 0/5] x86/mm/pat: CPA fixes

[PATCH v2 0/5] x86/mm/pat: CPA fixes

[Mike Rapoport]

The first three patches are urgent, the third patch fixes BUG() reported
y several people and it depends on the first two.

There were no bug reports that the last two patches fix because bug
manifestations won't yell at users.

TL;DR version:

There are a couple of CPA fixes floating around:

Denis Lunev fixed races between split and collapse of the large mappings:

https://lore.kernel.org/all/20260715183453.2381141-1-den@openvz.org

Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:

https://lore.kernel.org/all/20260723-series-vmap-race-fix-v6-0-8cc77dcc0018@kernel.org

and an issue with stale page tables in IOMMU:

https://lore.kernel.org/all/20260721-fix-cpa-kernel-pagetables-v2-1-2b255deed710@kernel.org

Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr()
used for the verification of RWX:

https://lore.kernel.org/all/20260715144519.934289-1-rppt@kernel.org

Pedro Falcato closed a race between text poking and collapse of large
pages:

https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse

Some of the fixes got merged into x86 tree, some of them got merged into mm
tree and some are still hanging in the air.

The changes here are collected from all these fixes into a single coherent
set on top of tip/x86/mm:
 
* fix for races between CPA and ptdump causing UAF
* update to the fix of the race between split and collapse of large
  mappings
* fix for races between CPA and vmalloc_to_page() in text poking
* fix for stale page tables in IOMMU
* fix for effective RW computation in lookup_address_in_pgd_attr()

---
v2 changes:
* rebased on the current tip/x86/mm that includes peterz's changes for
  DEBUG_PAGEALLOC
* added fix for CPA vs text poking race

v1: https://patch.msgid.link/20260728-cpa-fixes-v1-0-2ed2352300b3@kernel.org

---
Lorenzo Stoakes (ARM) (3):
      x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF
      x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF
      x86/mm/pat: allocate split page tables as kernel page tables

Mike Rapoport (Microsoft) (1):
      x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Pedro Falcato (1):
      x86/alternative: exclude text poking against change_page_attr()

 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
 arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
 include/linux/mmap_lock.h     |  2 ++
 3 files changed, 83 insertions(+), 19 deletions(-)
---
base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
change-id: 20260727-cpa-fixes-d3c73c075672

--
Sincerely yours,
Mike.

[PATCH v2 1/5] x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF

[Mike Rapoport]

From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>

x86 implements page attribute modification using its Change Page
Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page
attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
fragmentation") ranges of kernel page table entries can be collapsed into
huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed
entries previously pointed to, and it does so without any relevant locks
being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and
this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable
mappings.

However, this is problematic for ptdump as it walks ranges it does not own
and thus runs the risk of a use-after-free on page tables freed underneath
it.

In addition, concurrent CPA collapse operations are possible which can also
cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the
whole operation.

It is safe to acquire a sleeping lock as all the callers invoke
set_memory_rox() from process context and in any case,
change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
mutex, disallowing atomic context here.

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Cc: stable@vger.kernel.org
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Will Deacon <will@kernel.org>
Reviewed-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index d8d057f44417..c18b887ee4c2 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index 04b8f61ece5d..f4ceb968aeb3 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -621,6 +621,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {

-- 
2.53.0

[PATCH v2 2/5] x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF

[Mike Rapoport]

From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>

A previous commit protected us against races between ptdump and CPA
collapse, however one still exists between attribute changes and collapse
as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE
entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]
-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the
PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, we can now
resolve this race by acquiring an mmap read lock on init_mm over
__change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry
write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to
__change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can
trigger reclaim and shrinkers may call into CPA recursively, making
deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in
split_large_page() and track whether this is needed by adding a new
'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established)
revalidates that the page table entry is the same as it was prior to the
locks being dropped and __change_page_attr() repeats the entire page table
walk whenever a split occurs, so concurrent split and collapse are
accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table
allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively
operates upon vmalloc ranges, and on x86 only within the module mapping
space.

This is important, because some callers directly invoke
__change_page_attr_set_clr(), bypassing this lock. However, none of these
operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by
  __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via
  __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default,
  valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with
gratitude.

Link: https://lore.kernel.org/all/20260626163213.2284080-1-den@openvz.org/
Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Cc: stable@vger.kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index c18b887ee4c2..2d04a4bf34aa 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:

-- 
2.53.0

[PATCH v2 3/5] x86/alternative: exclude text poking against change_page_attr()

[Mike Rapoport]

From: Pedro Falcato <pfalcato@suse.de>

From time to time, the following BUG can be observed[0]:

> kernel BUG at arch/x86/kernel/alternative.c:2576!
> Oops: invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
> RIP: 0010:__text_poke+0x2aa/0x450
> Call Trace:
>  <TASK>
>  smp_text_poke_batch_finish+0x2a7/0x320
>  __static_call_transform+0xb7/0x220
>  arch_static_call_transform+0x5b/0xb0
>  __static_call_init+0xe9/0x270
>  static_call_module_notify+0x11f/0x150
>  notifier_call_chain+0x61/0xe0
>  blocking_notifier_call_chain_robust+0x63/0xc0
>  load_module+0x1c92/0x20c0
>  init_module_from_file+0xd8/0x140
>  idempotent_init_module+0x100/0x2f0
>  __x64_sys_finit_module+0x71/0xe0
>  do_syscall_64+0xe1/0x610
>  entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:
	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen
if text poking races with CPA, which can possibly result in the collapsing
of page tables (or breaking of PMD hugepages). It is not a problem for most
users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when
CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc
range, and can call set_memory_*() in parallel on it. This can happen to
race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

Fixes: 64f6a4e10c05 ("x86: re-enable EXECMEM_ROX support")
Reported-by: Jiri Slaby <jirislaby@kernel.org>
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1271202 [0]
Reported-by: Steffen Dirkwinkel <lists@steffen.cc>
Link: https://lore.kernel.org/linux-mm/555ea1d43a12c30a8f1eaf10c899b3790d728f33.camel@dirkwinkel.cc/
Cc: stable@vger.kernel.org
Co-developed-by: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Signed-off-by: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Signed-off-by: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 62936a3bde19..f81d6bc90a6f 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2543,6 +2546,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2560,9 +2595,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));

-- 
2.53.0

[PATCH v2 4/5] x86/mm/pat: allocate split page tables as kernel page tables

[Mike Rapoport]

From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>

When splitting a large page in CPA in __split_large_page() we allocate a
PTE directly without going through the standard page table allocation
routines such as pte_alloc_one_kernel().

This means the page table constructor is never called nor is the page table
marked as a kernel page table.

The former results in the folio associated with the page table not being
marked as a page table (__pagetable_ctor() is never called thus neither is
__folio_set_pgtable()) nor are statistics updated to reflect
it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page
table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
tables") kernel page table freeing has been batched and since the
subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
for kernel address space") IOTLB cache entries for kernel page tables have
been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can
contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor,
and thus requires a page table destructor.

Since we cannot assume one is always present (early allocated direct map
page tables are not marked as such), we conditionally call
pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set,
otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables,
which now includes split page tables, take the correct route through
pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear
in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit
5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
choose this as the Fixes target.

Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
Cc: stable@vger.kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Acked-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 2d04a4bf34aa..fbc418dfc597 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }

-- 
2.53.0

[PATCH v2 5/5] x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

[Mike Rapoport (Microsoft)]

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of
the walked page table levels so that verify_rwx() can detect mappings that
are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the
actual permissions, and verify_rwx() treats every mapping as non-writable
and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to
0 or 1.

Fixes: ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
Cc: stable@vger.kernel.org
Assisted-by: Copilot:claude-opus-4.8
Reviewed-by: Juergen Gross <jgross@suse.com>
Tested-by: syzbot@syzkaller.appspotmail.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index fbc418dfc597..430d0b448371 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }

-- 
2.53.0

Re: [PATCH v2 5/5] x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

[Lorenzo Stoakes (ARM)]

On Thu, Aug 13, 2026 at 12:01:28PM +0300, Mike Rapoport (Microsoft) wrote:
> lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of
> the walked page table levels so that verify_rwx() can detect mappings that
> are both writable and executable.
>
> The RW bits are folded into a bool with
>
> 	rw &= pXd_flags(*pXd) & _PAGE_RW;
>
> but _PAGE_RW is 0x2. So consider the accumulation line:
>
>         rw &= pXd_flags(*pXd) & _PAGE_RW;
>
> where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:
>
>         rw = 0x1 & 0x2
>
> and rw always ends up 0.
>
> This way rw becomes false at the first level walked, regardless of the
> actual permissions, and verify_rwx() treats every mapping as non-writable
> and never reports a W^X violation.
>
> Add double negation to the right side to normalize the _PAGE_RW flag to
> 0 or 1.
>
> Fixes: ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
> Cc: stable@vger.kernel.org
> Assisted-by: Copilot:claude-opus-4.8
> Reviewed-by: Juergen Gross <jgross@suse.com>
> Tested-by: syzbot@syzkaller.appspotmail.com
> Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>

Yikes, good find! LGTM so:

Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

> ---
>  arch/x86/mm/pat/set_memory.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index fbc418dfc597..430d0b448371 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>
>  	*level = PG_LEVEL_512G;
>  	*nx |= pgd_flags(*pgd) & _PAGE_NX;
> -	*rw &= pgd_flags(*pgd) & _PAGE_RW;
> +	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
>
>  	p4d = p4d_offset(pgd, address);
>  	if (p4d_none(*p4d))
> @@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>
>  	*level = PG_LEVEL_1G;
>  	*nx |= p4d_flags(*p4d) & _PAGE_NX;
> -	*rw &= p4d_flags(*p4d) & _PAGE_RW;
> +	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
>
>  	pud = pud_offset(p4d, address);
>  	if (pud_none(*pud))
> @@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>
>  	*level = PG_LEVEL_2M;
>  	*nx |= pud_flags(*pud) & _PAGE_NX;
> -	*rw &= pud_flags(*pud) & _PAGE_RW;
> +	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
>
>  	pmd = pmd_offset(pud, address);
>  	if (pmd_none(*pmd))
> @@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>
>  	*level = PG_LEVEL_4K;
>  	*nx |= pmd_flags(*pmd) & _PAGE_NX;
> -	*rw &= pmd_flags(*pmd) & _PAGE_RW;
> +	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
>
>  	return pte_offset_kernel(pmd, address);
>  }
>
> --
> 2.53.0
>

--
Cheers, Lorenzo

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Nikunj A. Dadhania]

On 8/13/2026 2:31 PM, Mike Rapoport wrote:

> The first three patches are urgent, the third patch fixes BUG() reported
> y several people and it depends on the first two.
> 
> There were no bug reports that the last two patches fix because bug
> manifestations won't yell at users.
> 
> TL;DR version:
> 
> There are a couple of CPA fixes floating around:
> 
> Denis Lunev fixed races between split and collapse of the large mappings:
> 
> https://lore.kernel.org/all/20260715183453.2381141-1-den@openvz.org
> 
> Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:
> 
> https://lore.kernel.org/all/20260723-series-vmap-race-fix-v6-0-8cc77dcc0018@kernel.org
> 
> and an issue with stale page tables in IOMMU:
> 
> https://lore.kernel.org/all/20260721-fix-cpa-kernel-pagetables-v2-1-2b255deed710@kernel.org
> 
> Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr()
> used for the verification of RWX:
> 
> https://lore.kernel.org/all/20260715144519.934289-1-rppt@kernel.org
> 
> Pedro Falcato closed a race between text poking and collapse of large
> pages:
> 
> https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse
> 
> Some of the fixes got merged into x86 tree, some of them got merged into mm
> tree and some are still hanging in the air.
> 
> The changes here are collected from all these fixes into a single coherent
> set on top of tip/x86/mm:
> 
> * fix for races between CPA and ptdump causing UAF
> * update to the fix of the race between split and collapse of large
>   mappings
> * fix for races between CPA and vmalloc_to_page() in text poking
> * fix for stale page tables in IOMMU
> * fix for effective RW computation in lookup_address_in_pgd_attr()
> 
> ---
> v2 changes:
> * rebased on the current tip/x86/mm that includes peterz's changes for
>   DEBUG_PAGEALLOC
> * added fix for CPA vs text poking race

Tested v2 on a 512-guest concurrent boot harness across 200 iterations
with no splats. Both crash signatures reported in [1] are no longer seen.

Tested-by: Nikunj A Dadhania <nikunj@amd.com>

Regards,
Nikunj

[1] https://lore.kernel.org/all/20260812063316.21371-1-nikunj@amd.com/
 
> v1: https://patch.msgid.link/20260728-cpa-fixes-v1-0-2ed2352300b3@kernel.org
> 
> ---
> Lorenzo Stoakes (ARM) (3):
>       x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF
>       x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF
>       x86/mm/pat: allocate split page tables as kernel page tables
> 
> Mike Rapoport (Microsoft) (1):
>       x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()
> 
> Pedro Falcato (1):
>       x86/alternative: exclude text poking against change_page_attr()
> 
>  arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
>  arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
>  include/linux/mmap_lock.h     |  2 ++
>  3 files changed, 83 insertions(+), 19 deletions(-)
> ---
> base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
> change-id: 20260727-cpa-fixes-d3c73c075672
> 
> --
> Sincerely yours,
> Mike.
> 

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Lorenzo Stoakes (ARM)]

On Thu, Aug 13, 2026 at 08:35:21PM +0530, Nikunj A. Dadhania wrote:
>
> Tested v2 on a 512-guest concurrent boot harness across 200 iterations
> with no splats. Both crash signatures reported in [1] are no longer seen.
>
> Tested-by: Nikunj A Dadhania <nikunj@amd.com>

Thanks so much for the testing!

And all credit to Pedro for reporting and contributing the key commit that
resolves this issue :)

>
> Regards,
> Nikunj
>
> [1] https://lore.kernel.org/all/20260812063316.21371-1-nikunj@amd.com/
>
> > v1: https://patch.msgid.link/20260728-cpa-fixes-v1-0-2ed2352300b3@kernel.org
> >
> > ---
> > Lorenzo Stoakes (ARM) (3):
> >       x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF
> >       x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF
> >       x86/mm/pat: allocate split page tables as kernel page tables
> >
> > Mike Rapoport (Microsoft) (1):
> >       x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()
> >
> > Pedro Falcato (1):
> >       x86/alternative: exclude text poking against change_page_attr()
> >
> >  arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
> >  arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
> >  include/linux/mmap_lock.h     |  2 ++
> >  3 files changed, 83 insertions(+), 19 deletions(-)
> > ---
> > base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
> > change-id: 20260727-cpa-fixes-d3c73c075672
> >
> > --
> > Sincerely yours,
> > Mike.
> >
>

--
Cheers, Lorenzo

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Pedro Falcato]

On Thu, Aug 13, 2026 at 04:07:38PM +0100, Lorenzo Stoakes (ARM) wrote:
> On Thu, Aug 13, 2026 at 08:35:21PM +0530, Nikunj A. Dadhania wrote:
> >
> > Tested v2 on a 512-guest concurrent boot harness across 200 iterations
> > with no splats. Both crash signatures reported in [1] are no longer seen.
> >
> > Tested-by: Nikunj A Dadhania <nikunj@amd.com>

Nice!

> 
> Thanks so much for the testing!
> 
> And all credit to Pedro for reporting and contributing the key commit that
> resolves this issue :)

And all credit to you for starting this flurry of fixes and patches that
solve Critical Kernel Issues(tm)! My fix would be nothing without yours!

-- 
Pedro

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Andrew Morton]

On Thu, 13 Aug 2026 12:01:23 +0300 Mike Rapoport <rppt@kernel.org> wrote:

> Some of the fixes got merged into x86 tree, some of them got merged into mm
> tree and some are still hanging in the air.

I'm assuming/hoping that this whole series will be handled by the x86
team.


I have retained three random pat patches in mm.git since July 23:

x86-mm-pat-acquire-init_mm-write-lock-on-collapse-to-avoid-uaf.patch
x86-mm-pat-acquire-init_mm-read-lock-on-attribute-change-to-avoid-uaf.patch
x86-mm-pat-allocate-split-page-tables-as-kernel-page-tables.patch

just to get them some testing exposure.  But they're old.  I've heard
no reports since adding them.

There's nothing more to be learned by keeping these in mm.git so I'll
remove them now.

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Atish Patra]

On 8/13/26 2:01 AM, Mike Rapoport wrote:
> The first three patches are urgent, the third patch fixes BUG() reported
> y several people and it depends on the first two.
>
> There were no bug reports that the last two patches fix because bug
> manifestations won't yell at users.
>
> TL;DR version:
>
> There are a couple of CPA fixes floating around:
>
> Denis Lunev fixed races between split and collapse of the large mappings:
>
> https://lore.kernel.org/all/20260715183453.2381141-1-den@openvz.org
>
> Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:
>
> https://lore.kernel.org/all/20260723-series-vmap-race-fix-v6-0-8cc77dcc0018@kernel.org
>
> and an issue with stale page tables in IOMMU:
>
> https://lore.kernel.org/all/20260721-fix-cpa-kernel-pagetables-v2-1-2b255deed710@kernel.org
>
> Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr()
> used for the verification of RWX:
>
> https://lore.kernel.org/all/20260715144519.934289-1-rppt@kernel.org
>
> Pedro Falcato closed a race between text poking and collapse of large
> pages:
>
> https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse
>
> Some of the fixes got merged into x86 tree, some of them got merged into mm
> tree and some are still hanging in the air.
>
> The changes here are collected from all these fixes into a single coherent
> set on top of tip/x86/mm:
>   
> * fix for races between CPA and ptdump causing UAF
> * update to the fix of the race between split and collapse of large
>    mappings
> * fix for races between CPA and vmalloc_to_page() in text poking
> * fix for stale page tables in IOMMU
> * fix for effective RW computation in lookup_address_in_pgd_attr()
>
> ---
> v2 changes:
> * rebased on the current tip/x86/mm that includes peterz's changes for
>    DEBUG_PAGEALLOC
> * added fix for CPA vs text poking race
>
> v1: https://patch.msgid.link/20260728-cpa-fixes-v1-0-2ed2352300b3@kernel.org
>
> ---
> Lorenzo Stoakes (ARM) (3):
>        x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF
>        x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF
>        x86/mm/pat: allocate split page tables as kernel page tables
>
> Mike Rapoport (Microsoft) (1):
>        x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()
>
> Pedro Falcato (1):
>        x86/alternative: exclude text poking against change_page_attr()
>
>   arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
>   arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
>   include/linux/mmap_lock.h     |  2 ++
>   3 files changed, 83 insertions(+), 19 deletions(-)
> ---
> base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
> change-id: 20260727-cpa-fixes-d3c73c075672

Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
1. mainline (commit: 77ae27fd98f3)
2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)

The Reproducer consisted of
1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules

With the above reproducer, both BUG within seconds in unpatched kernel.
1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).

With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).


Tested-by: Atish Patra<atishp@meta.com>

> --
> Sincerely yours,
> Mike.
>

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Lorenzo Stoakes (ARM)]

On Tue, Aug 25, 2026 at 12:12:32AM -0700, Atish Patra wrote:
> Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
> 1. mainline (commit: 77ae27fd98f3)
> 2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)
>
> The Reproducer consisted of
> 1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
> 2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules
>
> With the above reproducer, both BUG within seconds in unpatched kernel.
> 1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
> 2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).
>
> With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).

Amazing, thanks!

>
>
> Tested-by: Atish Patra<atishp@meta.com>

I guess should be applied to patches 1-3 only strictly? Or perhaps 3/5?

>
> > --
> > Sincerely yours,
> > Mike.
> >

--
Cheers, Lorenzo

Re: [PATCH v2 3/5] x86/alternative: exclude text poking against change_page_attr()

[Jiri Slaby]

On 13. 08. 26, 11:01, Mike Rapoport wrote:
> From: Pedro Falcato <pfalcato@suse.de>
> 
>  From time to time, the following BUG can be observed[0]:
> 
>> kernel BUG at arch/x86/kernel/alternative.c:2576!
>> Oops: invalid opcode: 0000 [#1] SMP NOPTI
>> CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
>> RIP: 0010:__text_poke+0x2aa/0x450
>> Call Trace:
>>   <TASK>
>>   smp_text_poke_batch_finish+0x2a7/0x320
>>   __static_call_transform+0xb7/0x220
>>   arch_static_call_transform+0x5b/0xb0
>>   __static_call_init+0xe9/0x270
>>   static_call_module_notify+0x11f/0x150
>>   notifier_call_chain+0x61/0xe0
>>   blocking_notifier_call_chain_robust+0x63/0xc0
>>   load_module+0x1c92/0x20c0
>>   init_module_from_file+0xd8/0x140
>>   idempotent_init_module+0x100/0x2f0
>>   __x64_sys_finit_module+0x71/0xe0
>>   do_syscall_64+0xe1/0x610
>>   entry_SYSCALL_64_after_hwframe+0x76/0x7e
> 
> which matches the following BUG_ON in alternative.c:
> 	/*
> 	 * If something went wrong, crash and burn since recovery paths are not
> 	 * implemented.
> 	 */
> 	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));
> 
> This can happen if vmalloc_to_page() fails, for any reason. Such can happen
> if text poking races with CPA, which can possibly result in the collapsing
> of page tables (or breaking of PMD hugepages). It is not a problem for most
> users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when
> CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc
> range, and can call set_memory_*() in parallel on it. This can happen to
> race against __text_poke and cause havoc in vmalloc_to_page().
> 
> Fix it by excluding against CPA using the init_mm mmap read lock.
> 
> Fixes: 64f6a4e10c05 ("x86: re-enable EXECMEM_ROX support")
> Reported-by: Jiri Slaby <jirislaby@kernel.org>

FWIW
Tested-by: Jiri Slaby <jirislaby@kernel.org>

We have not seen any BUGs since applied to the SUSE's kernel.
https://bugzilla.suse.com/show_bug.cgi?id=1271202#c26

thanks,
-- 
js
suse labs

Re: [PATCH v2 0/5] x86/mm/pat: CPA fixes

[Atish Patra]

On 8/25/26 12:31 AM, Lorenzo Stoakes (ARM) wrote:
> On Tue, Aug 25, 2026 at 12:12:32AM -0700, Atish Patra wrote:
>> Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
>> 1. mainline (commit: 77ae27fd98f3)
>> 2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)
>>
>> The Reproducer consisted of
>> 1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
>> 2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules
>>
>> With the above reproducer, both BUG within seconds in unpatched kernel.
>> 1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
>> 2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).
>>
>> With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).
> Amazing, thanks!
>
>>
>> Tested-by: Atish Patra<atishp@meta.com>
> I guess should be applied to patches 1-3 only strictly? Or perhaps 3/5?

Yes. 1-3 sounds good.


>>> --
>>> Sincerely yours,
>>> Mike.
>>>
> --
> Cheers, Lorenzo

[tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

[tip-bot2 for Mike Rapoport (Microsoft)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     34cd1c931a365a6b81e00e743f9f260050104ff6
Gitweb:        https://git.kernel.org/tip/34cd1c931a365a6b81e00e743f9f260050104ff6
Author:        Mike Rapoport (Microsoft) <rppt@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Mon, 31 Aug 2026 15:20:52 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of
the walked page table levels so that verify_rwx() can detect mappings that
are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the
actual permissions, and verify_rwx() treats every mapping as non-writable
and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to
0 or 1.

Assisted-by: Copilot:claude-opus-4.8
Fixes: ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Tested-by: syzbot@syzkaller.appspotmail.com
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-5-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 4652487..2266609 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     83a2490f0f1e173bb10f0d076a546cf1cd36f609
Gitweb:        https://git.kernel.org/tip/83a2490f0f1e173bb10f0d076a546cf1cd36f609
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Mon, 31 Aug 2026 15:17:53 -07:00

x86/mm/pat: Allocate split page tables as kernel page tables

PTEs are allocated directly without going through the standard page table
allocation routines such as pte_alloc_one_kernel() when splitting a large
page in CPA in __split_large_page().

This means the page table constructor is never called nor is the page table
marked as a kernel page table.

The former results in the folio associated with the page table not being
marked as a page table (__pagetable_ctor() is never called thus neither is
__folio_set_pgtable()) nor are statistics updated to reflect
it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page
table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
tables") kernel page table freeing has been batched and since the
subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
for kernel address space") IOTLB cache entries for kernel page tables have
been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can
contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor,
and thus requires a page table destructor.

Since page table destructors are not always present (early allocated direct
map page tables are not marked as such), conditionally call
pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set.
Otherwise, free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables,
which now includes split page tables, take the correct route through
pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear
in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit
5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
choose this as the Fixes target.

[ dhansen: changelog tweaks for tip style ]

Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Vishal Moola <vishal.moola@gmail.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-4-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index cb5d6d6..4652487 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }

[tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[tip-bot2 for Pedro Falcato]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883
Author:        Pedro Falcato <pfalcato@suse.de>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

>From time to time, the following BUG can be observed[0]:

> kernel BUG at arch/x86/kernel/alternative.c:2576!
> Oops: invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
> RIP: 0010:__text_poke+0x2aa/0x450
> Call Trace:
>  <TASK>
>  smp_text_poke_batch_finish+0x2a7/0x320
>  __static_call_transform+0xb7/0x220
>  arch_static_call_transform+0x5b/0xb0
>  __static_call_init+0xe9/0x270
>  static_call_module_notify+0x11f/0x150
>  notifier_call_chain+0x61/0xe0
>  blocking_notifier_call_chain_robust+0x63/0xc0
>  load_module+0x1c92/0x20c0
>  init_module_from_file+0xd8/0x140
>  idempotent_init_module+0x100/0x2f0
>  __x64_sys_finit_module+0x71/0xe0
>  do_syscall_64+0xe1/0x610
>  entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:
	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen
if text poking races with CPA, which can possibly result in the collapsing
of page tables (or breaking of PMD hugepages). It is not a problem for most
users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when
CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc
range, and can call set_memory_*() in parallel on it. This can happen to
race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

Co-developed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Fixes: 64f6a4e10c05 ("x86: re-enable EXECMEM_ROX support")
Reported-by: Jiri Slaby <jirislaby@kernel.org>
Reported-by: Steffen Dirkwinkel <lists@steffen.cc>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Tested-by: Jiri Slaby <jirislaby@kernel.org>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1271202 [0]
Link: https://lore.kernel.org/linux-mm/555ea1d43a12c30a8f1eaf10c899b3790d728f33.camel@dirkwinkel.cc/
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-3-39b4ff90f91d@kernel.org
---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 91b1cdd..add62db 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse,
however one still exists between attribute changes and collapse as reported
by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE
entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]
-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the
PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this
race by acquiring an mmap read lock on init_mm over
__change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry
write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to
__change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can
trigger reclaim and shrinkers may call into CPA recursively, making
deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in
split_large_page() and track whether this is needed by adding a new
'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established)
revalidates that the page table entry is the same as it was prior to the
locks being dropped and __change_page_attr() repeats the entire page table
walk whenever a split occurs, so concurrent split and collapse are
accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table
allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively
operates upon vmalloc ranges, and on x86 only within the module mapping
space.

This is important, because some callers directly invoke
__change_page_attr_set_clr(), bypassing this lock. However, none of these
operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by
  __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via
  __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default,
  valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with
gratitude.

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Link: https://lore.kernel.org/all/20260626163213.2284080-1-den@openvz.org/
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-2-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index c38faf3..cb5d6d6 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -49,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
@@ -1227,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2121,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page
Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page
attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
fragmentation") ranges of kernel page table entries can be collapsed into
huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed
entries previously pointed to, and it does so without any relevant locks
being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and
this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable
mappings.

However, this is problematic for ptdump as it walks ranges it does not own
and thus runs the risk of a use-after-free on page tables freed underneath
it.

In addition, concurrent CPA collapse operations are possible which can also
cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the
whole operation.

It is safe to acquire a sleeping lock as all the callers invoke
set_memory_rox() from process context and in any case,
change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
mutex, disallowing atomic context here.

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Will Deacon <will@kernel.org>
Reviewed-by: David Carlier <devnexen@gmail.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-1-39b4ff90f91d@kernel.org
---
 include/linux/mmap_lock.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index bec0eab..b8a13b8 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[Jiri Slaby]

On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:
> The following commit has been merged into the x86/urgent branch of tip:
> 
> Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
> Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
> Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
> AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
> Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00

The committed patch to tip is bogus. It contains only the guard definition.

> x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF
> 
> x86 implements page attribute modification using its Change Page
> Attributes (CPA) mechanism.
> 
> This tracks properties of ranges such as cache mode through x86 page
> attributes, and as part of that logic manipulates kernel page tables.
> 
> Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
> fragmentation") ranges of kernel page table entries can be collapsed into
> huge page table entries as part of this logic.
> 
> As part of this collapse, it frees the page tables which the collapsed
> entries previously pointed to, and it does so without any relevant locks
> being held to preclude concurrent kernel page table walkers.
> 
> The only way this code can be reached is if CPA_COLLAPSE is specified, and
> this is only set in set_memory_rox() via:
> 
> set_memory_rox()
> -> change_page_attr_set_clr()
> -> cpa_flush()
> -> cpa_collapse_large_pages()
> 
> Notable users of this are execmem and bpf when manipulating executable
> mappings.
> 
> However, this is problematic for ptdump as it walks ranges it does not own
> and thus runs the risk of a use-after-free on page tables freed underneath
> it.
> 
> In addition, concurrent CPA collapse operations are possible which can also
> cause races.
> 
> Resolve the issue by acquiring the mmap write lock on init_mm across the
> whole operation.
> 
> It is safe to acquire a sleeping lock as all the callers invoke
> set_memory_rox() from process context and in any case,
> change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
> mutex, disallowing atomic context here.
> 
> Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
> Reviewed-by: Will Deacon <will@kernel.org>
> Reviewed-by: David Carlier <devnexen@gmail.com>
> Tested-by: Atish Patra <atishp@meta.com>
> Tested-by: Nikunj A Dadhania <nikunj@amd.com>
> Cc:stable@vger.kernel.org
> Link: https://patch.msgid.link/20260813-cpa-fixes-v2-1-39b4ff90f91d@kernel.org
> ---
>   include/linux/mmap_lock.h | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> index bec0eab..b8a13b8 100644
> --- a/include/linux/mmap_lock.h
> +++ b/include/linux/mmap_lock.h
> @@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
>   DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
>   	     mmap_read_lock(_T), mmap_read_unlock(_T))
>   DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
> +DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
> +	     mmap_write_lock(_T), mmap_write_unlock(_T))
>   
>   static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
>   {
> 

-- 
js
suse labs

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

[Jiri Slaby]

On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:
> The following commit has been merged into the x86/urgent branch of tip:
> 
> Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
> Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
> Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
> AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
> Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00
> 
> x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF
> 
> A previous commit protected against races between ptdump and CPA collapse,
> however one still exists between attribute changes and collapse as reported
> by Denis V. Lunev (linked).
> 
> When an attribute change arises, a lockless page table walker obtains a PTE
> entry, which is later written to via set_pte_atomic():
> 
> ...
> -> change_page_attr_set_clr()
> -> __change_page_attr_set_clr()
> -> __change_page_attr()
> 	-> _lookup_address_cpa()
> 	-> lookup_address_in_pgd_attr()
> 	-> [ lockless page table walker ]
> -> set_pte_atomic()
> 
> There is nothing preventing a concurrent CPA collapse which can free the
> PTE that was retrieved here, resulting in a use-after-free.
> 
> With the mmap write lock taken on init_mm over CPA collapse, resolve this
> race by acquiring an mmap read lock on init_mm over
> __change_page_attr_set_clr().
> 
> This locks across the whole operation over which the walk and the PTE entry
> write occurs, solving the race.
> 
> It is safe to do this here, as no spinlocks are held upon entry to
> __change_page_attr_set_clr().
> 
> However, the lock must not be held over an allocation, as allocation can
> trigger reclaim and shrinkers may call into CPA recursively, making
> deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).
> 
> A page table is allocated when a huge page needs to be split:
> 
> -> change_page_attr_set_clr()
> -> __change_page_attr_set_clr()
> -> __change_page_attr()
> -> split_large_page()
> [ pagetable_alloc() ]
> -> __split_large_page()
> 
> Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in
> split_large_page() and track whether this is needed by adding a new
> 'init_mm_read_locked' flag to struct cpa_data.
> 
> This is safe as __split_large_page() (called with locks re-established)
> revalidates that the page table entry is the same as it was prior to the
> locks being dropped and __change_page_attr() repeats the entire page table
> walk whenever a split occurs, so concurrent split and collapse are
> accounted for.
> 
> Concurrent ptdump is also safe as the lock is only dropped over page table
> allocation during which time the page table has not yet been modified.
> 
> The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively
> operates upon vmalloc ranges, and on x86 only within the module mapping
> space.
> 
> This is important, because some callers directly invoke
> __change_page_attr_set_clr(), bypassing this lock. However, none of these
> operate within the module mapping space.
> 
> * cpa_process_alias() - a recursive helper called by
>    __change_page_attr_set_clr().
> * __set_memory_enc_pgtable() - operates on the direct mapping and (via
>    __vmbus_establish_gpadl()) the vmalloc mapping space.
> * __set_pages_[n]p() - called by set_direct_map_[invalid, default,
>    valid]_noflush(), __kernel_map_pages() - operates on the direct map.
> * kernel_[un]map_pages_in_pgd() - operates on EFI ranges.
> 
> This work is based upon Denis V. Lunev's excellent analysis of the bug with
> gratitude.
> 
> Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Tested-by: Atish Patra <atishp@meta.com>
> Tested-by: Nikunj A Dadhania <nikunj@amd.com>
> Link: https://lore.kernel.org/all/20260626163213.2284080-1-den@openvz.org/
> Cc:stable@vger.kernel.org
> Link: https://patch.msgid.link/20260813-cpa-fixes-v2-2-39b4ff90f91d@kernel.org
> ---
>   arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
>   1 file changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index c38faf3..cb5d6d6 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -22,6 +22,7 @@
>   #include <linux/cc_platform.h>
>   #include <linux/set_memory.h>
>   #include <linux/memregion.h>
> +#include <linux/cleanup.h>
>   
>   #include <asm/e820/api.h>
>   #include <asm/processor.h>
> @@ -49,7 +50,8 @@ struct cpa_data {
>   	unsigned int	flags;
>   	unsigned int	force_split		: 1,
>   			force_static_prot	: 1,
> -			force_flush_all		: 1;
> +			force_flush_all		: 1,
> +			init_mm_read_locked	: 1;
>   	struct page	**pages;
>   };
>   
> @@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
>   
>   static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
>   
> -static void cpa_collapse_large_pages(struct cpa_data *cpa)
> +static void __cpa_collapse_large_pages(struct cpa_data *cpa)
>   {
>   	unsigned long start, addr, end;
>   	struct ptdesc *ptdesc, *tmp;
> @@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
>   	}
>   }
>   
> +static void cpa_collapse_large_pages(struct cpa_data *cpa)
> +{
> +	/*
> +	 * Take the mmap write lock on init_mm to:
> +	 * - Avoid a use-after-free if raced by ptdump (which takes its own
> +	 *   write lock on init_mm).
> +	 * - Serialise concurrent CPA walkers.
> +	 */
> +	scoped_guard(mmap_write_lock, &init_mm)
> +		__cpa_collapse_large_pages(cpa);
> +}
> +

Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not 
sure how you managed to split/combine these two 8-).

thanks,
-- 
js
suse labs

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[Jiri Slaby]

On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:
> The following commit has been merged into the x86/urgent branch of tip:
> 
> Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
> Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883

Was it dropped in the meantime? Gitweb says:
Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from 
any reference in this repository

$ git rev-parse tip/x86/urgent
477cf5cd1f698053df7426b7b8d9339e85e00946
$ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
$

Despite I managed to download it at some point though:
$ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()

What's going on with tip today?

> Author:        Pedro Falcato <pfalcato@suse.de>
> AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
> Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00
> 
> x86/alternative: Exclude text poking against change_page_attr()
thanks,
-- 
js
suse labs

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[Lorenzo Stoakes (ARM)]

On Tue, Sep 01, 2026 at 08:03:21AM +0200, Jiri Slaby wrote:
> On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:
> > The following commit has been merged into the x86/urgent branch of tip:
> >
> > Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
> > Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
> > Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
> > Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> > CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00
>
> The committed patch to tip is bogus. It contains only the guard definition.

Dave - you've somehow applied this patch completely incorrectly to x86/urgent,
I'm not happy with this going to Linus in this form :/

Now the commit message and the actual patch are completely mismatched.

I'm not sure how tip resolves issues like these but is it possible to
replace this with the actual patch that was submitted please?

Thanks.

>
> > x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF
> >
> > x86 implements page attribute modification using its Change Page
> > Attributes (CPA) mechanism.
> >
> > This tracks properties of ranges such as cache mode through x86 page
> > attributes, and as part of that logic manipulates kernel page tables.
> >
> > Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
> > fragmentation") ranges of kernel page table entries can be collapsed into
> > huge page table entries as part of this logic.
> >
> > As part of this collapse, it frees the page tables which the collapsed
> > entries previously pointed to, and it does so without any relevant locks
> > being held to preclude concurrent kernel page table walkers.
> >
> > The only way this code can be reached is if CPA_COLLAPSE is specified, and
> > this is only set in set_memory_rox() via:
> >
> > set_memory_rox()
> > -> change_page_attr_set_clr()
> > -> cpa_flush()
> > -> cpa_collapse_large_pages()
> >
> > Notable users of this are execmem and bpf when manipulating executable
> > mappings.
> >
> > However, this is problematic for ptdump as it walks ranges it does not own
> > and thus runs the risk of a use-after-free on page tables freed underneath
> > it.
> >
> > In addition, concurrent CPA collapse operations are possible which can also
> > cause races.
> >
> > Resolve the issue by acquiring the mmap write lock on init_mm across the
> > whole operation.
> >
> > It is safe to acquire a sleeping lock as all the callers invoke
> > set_memory_rox() from process context and in any case,
> > change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
> > mutex, disallowing atomic context here.
> >
> > Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> > Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> > Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
> > Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Reviewed-by: Will Deacon <will@kernel.org>
> > Reviewed-by: David Carlier <devnexen@gmail.com>
> > Tested-by: Atish Patra <atishp@meta.com>
> > Tested-by: Nikunj A Dadhania <nikunj@amd.com>

It renders all of these tags completly incorrect too.

> > Cc:stable@vger.kernel.org
> > Link: https://patch.msgid.link/20260813-cpa-fixes-v2-1-39b4ff90f91d@kernel.org
> > ---
> >   include/linux/mmap_lock.h | 2 ++
> >   1 file changed, 2 insertions(+)
> >
> > diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
> > index bec0eab..b8a13b8 100644
> > --- a/include/linux/mmap_lock.h
> > +++ b/include/linux/mmap_lock.h
> > @@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
> >   DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
> >   	     mmap_read_lock(_T), mmap_read_unlock(_T))
> >   DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
> > +DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
> > +	     mmap_write_lock(_T), mmap_write_unlock(_T))

Yeah I meant obviously this isn't what the patch is.

> >   static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
> >   {
> >
>
> --
> js
> suse labs
>

--
Cheers, Lorenzo

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[Lorenzo Stoakes (ARM)]

On Tue, Sep 01, 2026 at 08:16:28AM +0200, Jiri Slaby wrote:
> On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:
> > The following commit has been merged into the x86/urgent branch of tip:
> >
> > Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
> > Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883
>
> Was it dropped in the meantime? Gitweb says:
> Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any
> reference in this repository
>
> $ git rev-parse tip/x86/urgent
> 477cf5cd1f698053df7426b7b8d9339e85e00946
> $ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
> $
>
> Despite I managed to download it at some point though:
> $ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
> 1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()
>
> What's going on with tip today?

Yup, I can't see this commit _at all_ in x86/urgent. Something's very
broken here.

>
> > Author:        Pedro Falcato <pfalcato@suse.de>
> > AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
> > Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> > CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00
> >
> > x86/alternative: Exclude text poking against change_page_attr()
> thanks,
> --
> js
> suse labs

--
Cheers, Lorenzo

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

[Lorenzo Stoakes (ARM)]

Dave - Again this patch has been modified vs. what was submitted and the commit
message, tags, etc. are now completely incorrect.

I'm not happy with this going to Linus as-is, is there a way for you to correct
these please?

On Tue, Sep 01, 2026 at 08:05:07AM +0200, Jiri Slaby wrote:
> On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:
> > The following commit has been merged into the x86/urgent branch of tip:
> >
> > Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
> > Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
> > Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
> > Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> > CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00
> >
> > x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF
> >
> > A previous commit protected against races between ptdump and CPA collapse,
> > however one still exists between attribute changes and collapse as reported
> > by Denis V. Lunev (linked).
> >
> > When an attribute change arises, a lockless page table walker obtains a PTE
> > entry, which is later written to via set_pte_atomic():
> >
> > ...
> > -> change_page_attr_set_clr()
> > -> __change_page_attr_set_clr()
> > -> __change_page_attr()
> > 	-> _lookup_address_cpa()
> > 	-> lookup_address_in_pgd_attr()
> > 	-> [ lockless page table walker ]
> > -> set_pte_atomic()
> >
> > There is nothing preventing a concurrent CPA collapse which can free the
> > PTE that was retrieved here, resulting in a use-after-free.
> >
> > With the mmap write lock taken on init_mm over CPA collapse, resolve this
> > race by acquiring an mmap read lock on init_mm over
> > __change_page_attr_set_clr().
> >
> > This locks across the whole operation over which the walk and the PTE entry
> > write occurs, solving the race.
> >
> > It is safe to do this here, as no spinlocks are held upon entry to
> > __change_page_attr_set_clr().
> >
> > However, the lock must not be held over an allocation, as allocation can
> > trigger reclaim and shrinkers may call into CPA recursively, making
> > deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).
> >
> > A page table is allocated when a huge page needs to be split:
> >
> > -> change_page_attr_set_clr()
> > -> __change_page_attr_set_clr()
> > -> __change_page_attr()
> > -> split_large_page()
> > [ pagetable_alloc() ]
> > -> __split_large_page()
> >
> > Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in
> > split_large_page() and track whether this is needed by adding a new
> > 'init_mm_read_locked' flag to struct cpa_data.
> >
> > This is safe as __split_large_page() (called with locks re-established)
> > revalidates that the page table entry is the same as it was prior to the
> > locks being dropped and __change_page_attr() repeats the entire page table
> > walk whenever a split occurs, so concurrent split and collapse are
> > accounted for.
> >
> > Concurrent ptdump is also safe as the lock is only dropped over page table
> > allocation during which time the page table has not yet been modified.
> >
> > The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively
> > operates upon vmalloc ranges, and on x86 only within the module mapping
> > space.
> >
> > This is important, because some callers directly invoke
> > __change_page_attr_set_clr(), bypassing this lock. However, none of these
> > operate within the module mapping space.
> >
> > * cpa_process_alias() - a recursive helper called by
> >    __change_page_attr_set_clr().
> > * __set_memory_enc_pgtable() - operates on the direct mapping and (via
> >    __vmbus_establish_gpadl()) the vmalloc mapping space.
> > * __set_pages_[n]p() - called by set_direct_map_[invalid, default,
> >    valid]_noflush(), __kernel_map_pages() - operates on the direct map.
> > * kernel_[un]map_pages_in_pgd() - operates on EFI ranges.
> >
> > This work is based upon Denis V. Lunev's excellent analysis of the bug with
> > gratitude.
> >
> > Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
> > Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> > Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> > Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> > Tested-by: Atish Patra <atishp@meta.com>
> > Tested-by: Nikunj A Dadhania <nikunj@amd.com>
> > Link: https://lore.kernel.org/all/20260626163213.2284080-1-den@openvz.org/
> > Cc:stable@vger.kernel.org
> > Link: https://patch.msgid.link/20260813-cpa-fixes-v2-2-39b4ff90f91d@kernel.org
> > ---
> >   arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
> >   1 file changed, 25 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> > index c38faf3..cb5d6d6 100644
> > --- a/arch/x86/mm/pat/set_memory.c
> > +++ b/arch/x86/mm/pat/set_memory.c
> > @@ -22,6 +22,7 @@
> >   #include <linux/cc_platform.h>
> >   #include <linux/set_memory.h>
> >   #include <linux/memregion.h>
> > +#include <linux/cleanup.h>
> >   #include <asm/e820/api.h>
> >   #include <asm/processor.h>
> > @@ -49,7 +50,8 @@ struct cpa_data {
> >   	unsigned int	flags;
> >   	unsigned int	force_split		: 1,
> >   			force_static_prot	: 1,
> > -			force_flush_all		: 1;
> > +			force_flush_all		: 1,
> > +			init_mm_read_locked	: 1;
> >   	struct page	**pages;
> >   };
> > @@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
> >   static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
> > -static void cpa_collapse_large_pages(struct cpa_data *cpa)
> > +static void __cpa_collapse_large_pages(struct cpa_data *cpa)
> >   {
> >   	unsigned long start, addr, end;
> >   	struct ptdesc *ptdesc, *tmp;
> > @@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
> >   	}
> >   }
> > +static void cpa_collapse_large_pages(struct cpa_data *cpa)
> > +{
> > +	/*
> > +	 * Take the mmap write lock on init_mm to:
> > +	 * - Avoid a use-after-free if raced by ptdump (which takes its own
> > +	 *   write lock on init_mm).
> > +	 * - Serialise concurrent CPA walkers.
> > +	 */
> > +	scoped_guard(mmap_write_lock, &init_mm)
> > +		__cpa_collapse_large_pages(cpa);
> > +}
> > +
>
> Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not sure
> how you managed to split/combine these two 8-).

Yeah this is completely incorrect, now the commit message, tags, etc. are all
wrong.

>
> thanks,
> --
> js
> suse labs
>

--
Cheers, Lorenzo

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[Jiri Slaby]

On 01. 09. 26, 9:18, Lorenzo Stoakes (ARM) wrote:
> On Tue, Sep 01, 2026 at 08:16:28AM +0200, Jiri Slaby wrote:
>> On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:
>>> The following commit has been merged into the x86/urgent branch of tip:
>>>
>>> Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
>>> Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883
>>
>> Was it dropped in the meantime? Gitweb says:
>> Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any
>> reference in this repository
>>
>> $ git rev-parse tip/x86/urgent
>> 477cf5cd1f698053df7426b7b8d9339e85e00946
>> $ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
>> $
>>
>> Despite I managed to download it at some point though:
>> $ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
>> 1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()
>>
>> What's going on with tip today?
> 
> Yup, I can't see this commit _at all_ in x86/urgent. Something's very
> broken here.

FTR, I dug out the logs:
  + 096edb090797...4a0e6473921a master     -> tip/master  (forced update)
  + 096edb090797...4a0e6473921a tip/urgent -> tip/tip/urgent  (forced 
update)
  + 34cd1c931a36...477cf5cd1f69 x86/urgent -> tip/x86/urgent  (forced 
update)

The original x86/urgent's HEAD (34cd1c931a36) indeed contains the commit 
(1c4d1c070d7).

>>> Author:        Pedro Falcato <pfalcato@suse.de>
>>> AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
>>> Committer:     Dave Hansen <dave.hansen@linux.intel.com>
>>> CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00
>>>
>>> x86/alternative: Exclude text poking against change_page_attr()
>> thanks,
>> --
>> js
>> suse labs
> 
> --
> Cheers, Lorenzo


-- 
js
suse labs

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[Lorenzo Stoakes (ARM)]

On Tue, Sep 01, 2026 at 09:22:09AM +0200, Jiri Slaby wrote:
> On 01. 09. 26, 9:18, Lorenzo Stoakes (ARM) wrote:
> > On Tue, Sep 01, 2026 at 08:16:28AM +0200, Jiri Slaby wrote:
> > > On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:
> > > > The following commit has been merged into the x86/urgent branch of tip:
> > > >
> > > > Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
> > > > Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883
> > >
> > > Was it dropped in the meantime? Gitweb says:
> > > Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any
> > > reference in this repository
> > >
> > > $ git rev-parse tip/x86/urgent
> > > 477cf5cd1f698053df7426b7b8d9339e85e00946
> > > $ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
> > > $
> > >
> > > Despite I managed to download it at some point though:
> > > $ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
> > > 1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()
> > >
> > > What's going on with tip today?
> >
> > Yup, I can't see this commit _at all_ in x86/urgent. Something's very
> > broken here.
>
> FTR, I dug out the logs:
>  + 096edb090797...4a0e6473921a master     -> tip/master  (forced update)
>  + 096edb090797...4a0e6473921a tip/urgent -> tip/tip/urgent  (forced update)
>  + 34cd1c931a36...477cf5cd1f69 x86/urgent -> tip/x86/urgent  (forced update)
>
> The original x86/urgent's HEAD (34cd1c931a36) indeed contains the commit
> (1c4d1c070d7).

I mean 'forced updated' gives me hope at least that these patches can be rebased
back to sanity :)

>
> > > > Author:        Pedro Falcato <pfalcato@suse.de>
> > > > AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
> > > > Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> > > > CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00
> > > >
> > > > x86/alternative: Exclude text poking against change_page_attr()
> > > thanks,
> > > --
> > > js
> > > suse labs
> >
> > --
> > Cheers, Lorenzo
>
>
> --
> js
> suse labs

--
Cheers, Lorenzo

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

[Dave Hansen]

On 8/31/26 23:05, Jiri Slaby wrote:
> Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not
> sure how you managed to split/combine these two 😎.

Hurrying at the end of the day and being a bonehead will do it.
I'm going to back them out from x86/urgent and take a fresh go at it
today, post-coffee.

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[Dave Hansen]

On 9/1/26 00:10, Lorenzo Stoakes (ARM) wrote:
> Dave - you've somehow applied this patch completely incorrectly to x86/urgent,
> I'm not happy with this going to Linus in this form :/

Me neither, btw.

So, here it is applied:

> https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1

The diffstat exactly matches the mbox that I sucked in. I'm going to
launch some tests overnight.

The only things I really munged were a little merge issue with patch 1,
a SoB ordering issue in patch 3, and a few scattered rewordings in the
commit messages.

Not pushed to x86/urgent, yet.

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[Lorenzo Stoakes (ARM)]

On Tue, Sep 01, 2026 at 04:36:38PM -0700, Dave Hansen wrote:
> On 9/1/26 00:10, Lorenzo Stoakes (ARM) wrote:
> > Dave - you've somehow applied this patch completely incorrectly to x86/urgent,
> > I'm not happy with this going to Linus in this form :/
>
> Me neither, btw.

Yeah, sorry to complain so loudly, but just obviously want to make sure these
are sorted correctly!

>
> So, here it is applied:
>
> > https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1
>
> The diffstat exactly matches the mbox that I sucked in. I'm going to
> launch some tests overnight.

Thanks!

I had a look through and all LGTM :)

>
> The only things I really munged were a little merge issue with patch 1,
> a SoB ordering issue in patch 3, and a few scattered rewordings in the
> commit messages.

That's all good of course!

>
> Not pushed to x86/urgent, yet.

Ack, understood that you want to beat them about a bit with some tests, let me
know if there's anything else you need on these!

--
Cheers, Lorenzo

[tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

[tip-bot2 for Mike Rapoport (Microsoft)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     453e7859443446b837d905d6f2983a76c867247d
Gitweb:        https://git.kernel.org/tip/453e7859443446b837d905d6f2983a76c867247d
Author:        Mike Rapoport (Microsoft) <rppt@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Tue, 01 Sep 2026 15:03:18 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of
the walked page table levels so that verify_rwx() can detect mappings that
are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the
actual permissions, and verify_rwx() treats every mapping as non-writable
and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to
0 or 1.

Assisted-by: Copilot:claude-opus-4.8
Fixes: ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Tested-by: syzbot@syzkaller.appspotmail.com
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-5-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 4652487..2266609 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     0e33126d5def407397deaf617559a1a2a7f4b1ae
Gitweb:        https://git.kernel.org/tip/0e33126d5def407397deaf617559a1a2a7f4b1ae
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Tue, 01 Sep 2026 14:58:43 -07:00

x86/mm/pat: Allocate split page tables as kernel page tables

A PTE is allocated directly without going through the standard page table
allocation routines (such as pte_alloc_one_kernel()) when the CPA code
splits a large page (__split_large_page()).

This means the page table constructor is never called nor is the page table
marked as a kernel page table.

The former results in the folio associated with the page table not being
marked as a page table (__pagetable_ctor() is never called thus neither is
__folio_set_pgtable()) nor are statistics updated to reflect
it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page
table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page
tables") kernel page table freeing has been batched and since the
subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries
for kernel address space") IOTLB cache entries for kernel page tables have
been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can
contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor,
and thus requires a page table destructor.

Destructors are not always present, like for early allocated direct map
page tables). Conditionally call pagetable_dtor_free() if the PG_table
folio flag for the ptdesc is set, otherwise we free the page table via
pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables,
which now includes split page tables, take the correct route through
pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear
in nr_page_table_pages in /proc/vmstat (as do other kernel page tables
allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit
5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so
choose this as the Fixes target.

[ dhansen: rephrase in imperative mood ]

Fixes: 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Vishal Moola <vishal.moola@gmail.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-4-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index cb5d6d6..4652487 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }

[tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

[tip-bot2 for Pedro Falcato]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     e679ba0983757e9567aeda97cc35c99241d420ee
Gitweb:        https://git.kernel.org/tip/e679ba0983757e9567aeda97cc35c99241d420ee
Author:        Pedro Falcato <pfalcato@suse.de>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Tue, 01 Sep 2026 14:54:28 -07:00

x86/alternative: Exclude text poking against change_page_attr()

>From time to time, the following BUG can be observed[0]:

> kernel BUG at arch/x86/kernel/alternative.c:2576!
> Oops: invalid opcode: 0000 [#1] SMP NOPTI
> CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
> RIP: 0010:__text_poke+0x2aa/0x450
> Call Trace:
>  <TASK>
>  smp_text_poke_batch_finish+0x2a7/0x320
>  __static_call_transform+0xb7/0x220
>  arch_static_call_transform+0x5b/0xb0
>  __static_call_init+0xe9/0x270
>  static_call_module_notify+0x11f/0x150
>  notifier_call_chain+0x61/0xe0
>  blocking_notifier_call_chain_robust+0x63/0xc0
>  load_module+0x1c92/0x20c0
>  init_module_from_file+0xd8/0x140
>  idempotent_init_module+0x100/0x2f0
>  __x64_sys_finit_module+0x71/0xe0
>  do_syscall_64+0xe1/0x610
>  entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:
	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen
if text poking races with CPA, which can possibly result in the collapsing
of page tables (or breaking of PMD hugepages). It is not a problem for most
users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when
CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc
range, and can call set_memory_*() in parallel on it. This can happen to
race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

[ dhansen: Fix up SoB ordering. The actual code flow here was:
	   Pedro=>Lorenzo=>Mike=>Me which is reflected in the SoB chain
	   now. I *believe* Mike simply picked up Lorenzo's update to
	   Pedro's post from the Link: ]

Fixes: 64f6a4e10c05 ("x86: re-enable EXECMEM_ROX support")
Reported-by: Jiri Slaby <jirislaby@kernel.org>
Reported-by: Steffen Dirkwinkel <lists@steffen.cc>
Signed-off-by: Pedro Falcato <pfalcato@suse.de>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Co-developed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Tested-by: Jiri Slaby <jirislaby@kernel.org>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1271202 [0]
Link: https://lore.kernel.org/linux-mm/555ea1d43a12c30a8f1eaf10c899b3790d728f33.camel@dirkwinkel.cc/
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-3-39b4ff90f91d@kernel.org
---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index 91b1cdd..add62db 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     884801a901bd340c44a202e2ef5c29b48e3a4d93
Gitweb:        https://git.kernel.org/tip/884801a901bd340c44a202e2ef5c29b48e3a4d93
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Tue, 01 Sep 2026 14:35:22 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse,
however one still exists between attribute changes and collapse as reported
by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE
entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]
-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the
PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this
race by acquiring an mmap read lock on init_mm over
__change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry
write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to
__change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can
trigger reclaim and shrinkers may call into CPA recursively, making
deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in
split_large_page() and track whether this is needed by adding a new
'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established)
revalidates that the page table entry is the same as it was prior to the
locks being dropped and __change_page_attr() repeats the entire page table
walk whenever a split occurs, so concurrent split and collapse are
accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table
allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively
operates upon vmalloc ranges, and on x86 only within the module mapping
space.

This is important, because some callers directly invoke
__change_page_attr_set_clr(), bypassing this lock. However, none of these
operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by
  __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via
  __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default,
  valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with
gratitude.

[ dhansen: move to imperative voice in changelog ]

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Link: https://lore.kernel.org/all/20260626163213.2284080-1-den@openvz.org/
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-2-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 4abddd7..cb5d6d6 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

[tip-bot2 for Lorenzo Stoakes (ARM)]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     4cfad2657c7c87b1172a9c343b74cf59b3769873
Gitweb:        https://git.kernel.org/tip/4cfad2657c7c87b1172a9c343b74cf59b3769873
Author:        Lorenzo Stoakes (ARM) <ljs@kernel.org>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <dave.hansen@linux.intel.com>
CommitterDate: Tue, 01 Sep 2026 14:31:49 -07:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page
Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page
attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after
fragmentation") ranges of kernel page table entries can be collapsed into
huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed
entries previously pointed to, and it does so without any relevant locks
being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and
this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable
mappings.

However, this is problematic for ptdump as it walks ranges it does not own
and thus runs the risk of a use-after-free on page tables freed underneath
it.

In addition, concurrent CPA collapse operations are possible which can also
cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the
whole operation.

It is safe to acquire a sleeping lock as all the callers invoke
set_memory_rox() from process context and in any case,
change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a
mutex, disallowing atomic context here.

Fixes: 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Will Deacon <will@kernel.org>
Reviewed-by: David Carlier <devnexen@gmail.com>
Tested-by: Atish Patra <atishp@meta.com>
Tested-by: Nikunj A Dadhania <nikunj@amd.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260813-cpa-fixes-v2-1-39b4ff90f91d@kernel.org
---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index c38faf3..4abddd7 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h
index bec0eab..b8a13b8 100644
--- a/include/linux/mmap_lock.h
+++ b/include/linux/mmap_lock.h
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

[Nathan Chancellor]

Hi folks,

On Wed, Sep 02, 2026 at 06:33:31PM -0000, tip-bot2 for Mike Rapoport (Microsoft) wrote:
> The following commit has been merged into the x86/urgent branch of tip:
> 
> Commit-ID:     453e7859443446b837d905d6f2983a76c867247d
> Gitweb:        https://git.kernel.org/tip/453e7859443446b837d905d6f2983a76c867247d
> Author:        Mike Rapoport (Microsoft) <rppt@kernel.org>
> AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
> Committer:     Dave Hansen <dave.hansen@linux.intel.com>
> CommitterDate: Tue, 01 Sep 2026 15:03:18 -07:00
> 
> x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()
> 
> lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of
> the walked page table levels so that verify_rwx() can detect mappings that
> are both writable and executable.
> 
> The RW bits are folded into a bool with
> 
> 	rw &= pXd_flags(*pXd) & _PAGE_RW;
> 
> but _PAGE_RW is 0x2. So consider the accumulation line:
> 
>         rw &= pXd_flags(*pXd) & _PAGE_RW;
> 
> where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:
> 
>         rw = 0x1 & 0x2
> 
> and rw always ends up 0.
> 
> This way rw becomes false at the first level walked, regardless of the
> actual permissions, and verify_rwx() treats every mapping as non-writable
> and never reports a W^X violation.
> 
> Add double negation to the right side to normalize the _PAGE_RW flag to
> 0 or 1.
> 
> Assisted-by: Copilot:claude-opus-4.8
> Fixes: ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
> Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Reviewed-by: Juergen Gross <jgross@suse.com>
> Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> Tested-by: syzbot@syzkaller.appspotmail.com
> Tested-by: Atish Patra <atishp@meta.com>
> Tested-by: Nikunj A Dadhania <nikunj@amd.com>
> Cc:stable@vger.kernel.org
> Link: https://patch.msgid.link/20260813-cpa-fixes-v2-5-39b4ff90f91d@kernel.org
> ---
>  arch/x86/mm/pat/set_memory.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
> index 4652487..2266609 100644
> --- a/arch/x86/mm/pat/set_memory.c
> +++ b/arch/x86/mm/pat/set_memory.c
> @@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>  
>  	*level = PG_LEVEL_512G;
>  	*nx |= pgd_flags(*pgd) & _PAGE_NX;
> -	*rw &= pgd_flags(*pgd) & _PAGE_RW;
> +	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
>  
>  	p4d = p4d_offset(pgd, address);
>  	if (p4d_none(*p4d))
> @@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>  
>  	*level = PG_LEVEL_1G;
>  	*nx |= p4d_flags(*p4d) & _PAGE_NX;
> -	*rw &= p4d_flags(*p4d) & _PAGE_RW;
> +	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
>  
>  	pud = pud_offset(p4d, address);
>  	if (pud_none(*pud))
> @@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>  
>  	*level = PG_LEVEL_2M;
>  	*nx |= pud_flags(*pud) & _PAGE_NX;
> -	*rw &= pud_flags(*pud) & _PAGE_RW;
> +	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
>  
>  	pmd = pmd_offset(pud, address);
>  	if (pmd_none(*pmd))
> @@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
>  
>  	*level = PG_LEVEL_4K;
>  	*nx |= pmd_flags(*pmd) & _PAGE_NX;
> -	*rw &= pmd_flags(*pmd) & _PAGE_RW;
> +	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
>  
>  	return pte_offset_kernel(pmd, address);
>  }

I just bisected the following warning on a couple of my test machines to
commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in
lookup_address_in_pgd_attr()") in next-20260904.

  [Sep 4 21:36] CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0400000 - 0xffffffffc0400fff PFN 100e00
  [  +0.000001] WARNING: arch/x86/mm/pat/set_memory.c:722 at __change_page_attr_set_clr+0xde7/0x1290, CPU#0: swapper/0/0
  [  +0.000005] Modules linked in:
  [  +0.000002] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.3.0-rc1-debug-00006-g453e78594434 #1 PREEMPT(full)  2950d432dd3910251071a66f3134fe0875432786
  [  +0.000001] Hardware name: ASUS System Product Name/PRIME Z590M-PLUS, BIOS 1801 12/26/2022
  [  +0.000001] RIP: 0010:__change_page_attr_set_clr+0xdff/0x1290
  [  +0.000002] Code: 80 7c 24 42 00 0f 85 3a 04 00 00 48 8d 3d 19 8d 79 02 49 89 d9 4c 89 e1 4c 89 d2 4c 89 f6 4d 8d 84 24 ff 0f 00 00 4c 89 14 24 <67> 48 0f b9 3a 4c 8b 14 24 48 8b 0d 81 44 bf 01 41 f6 c2 01 0f 85
  [  +0.000001] RSP: 0000:ffffffff87003c60 EFLAGS: 00010246
  [  +0.000002] RAX: 0000000000000002 RBX: 0000000000100e00 RCX: ffffffffc0400000
  [  +0.000000] RDX: 0000000000000123 RSI: 8000000000000123 RDI: ffffffff872e50c0
  [  +0.000001] RBP: 8000000100e00123 R08: ffffffffc0400fff R09: 0000000000100e00
  [  +0.000001] R10: 0000000000000123 R11: 0000000000000001 R12: ffffffffc0400000
  [  +0.000000] R13: 0000000100e00123 R14: 8000000000000123 R15: ffffffff87003d58
  [  +0.000001] FS:  0000000000000000(0000) GS:ffff8ad1777a7000(0000) knlGS:0000000000000000
  [  +0.000001] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  [  +0.000001] CR2: ffff8ad0a4201000 CR3: 00000007e3022001 CR4: 0000000000770ef0
  [  +0.000001] PKRU: 55555554
  [  +0.000001] Call Trace:
  [  +0.000001]  <TASK>
  [  +0.000002]  ? _vm_unmap_aliases+0x219/0x280
  [  +0.000003]  change_page_attr_set_clr+0x161/0x250
  [  +0.000002]  ? events_sysfs_show+0x5d/0x80
  [  +0.000002]  set_memory_x+0x39/0x50
  [  +0.000002]  apply_retpolines+0x656/0x6d0
  [  +0.000002]  ? events_sysfs_show+0x5d/0x80
  [  +0.000002]  ? events_sysfs_show+0x6c/0x80
  [  +0.000001]  ? events_sysfs_show+0x62/0x80
  [  +0.000001]  alternative_instructions+0x3c/0xd0
  [  +0.000003]  arch_cpu_finalize_init+0x130/0x190
  [  +0.000003]  start_kernel+0x97d/0xa10
  [  +0.000002]  x86_64_start_reservations+0x24/0x30
  [  +0.000002]  x86_64_start_kernel+0xda/0xe0
  [  +0.000002]  common_startup_64+0x13e/0x151
  [  +0.000003]  </TASK>
  [  +0.000001] ---[ end trace 0000000000000000 ]---

  # bad: [af5f12805e5cefa4fe68d6127c7e1fb78cd5535c] Add linux-next specific files for 20260904
  # good: [421066905cbceca1f78cba5f7d92b4980317ab2b] Merge tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
  git bisect start 'af5f12805e5cefa4fe68d6127c7e1fb78cd5535c' '421066905cbceca1f78cba5f7d92b4980317ab2b'
  # bad: [7e9b8247e9129fa3803020d3262e2384a0cfc29a] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/hid/hid.git
  git bisect bad 7e9b8247e9129fa3803020d3262e2384a0cfc29a
  # bad: [c4faef7e83f9c601c1264c0fad20af0e00b5a52b] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git
  git bisect bad c4faef7e83f9c601c1264c0fad20af0e00b5a52b
  # good: [14bd0d1671d1655d33419b60c9c476d0590636dd] Merge branch 'drm-fixes' of https://gitlab.freedesktop.org/drm/kernel.git
  git bisect good 14bd0d1671d1655d33419b60c9c476d0590636dd
  # good: [8d6fa663f11a859304187a70c569ec0617aa2461] Merge https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-unstable into for-next
  git bisect good 8d6fa663f11a859304187a70c569ec0617aa2461
  # bad: [276cefa1ab31a5d4fe5e4446f5e504499cc03320] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/mm/linux.git
  git bisect bad 276cefa1ab31a5d4fe5e4446f5e504499cc03320
  # good: [aad09a60e0012e85b03540a12e1fd1a61646712b] Merge branch 'rust-fixes' of https://github.com/Rust-for-Linux/linux.git
  git bisect good aad09a60e0012e85b03540a12e1fd1a61646712b
  # bad: [586e57cb2f695099a15e72029804a0592e6ad566] Merge branch into tip/master: 'x86/urgent'
  git bisect bad 586e57cb2f695099a15e72029804a0592e6ad566
  # good: [223f1750ff141a7e59b2724b645f348ffdab12e9] Merge branch into tip/master: 'sched/urgent'
  git bisect good 223f1750ff141a7e59b2724b645f348ffdab12e9
  # good: [e679ba0983757e9567aeda97cc35c99241d420ee] x86/alternative: Exclude text poking against change_page_attr()
  git bisect good e679ba0983757e9567aeda97cc35c99241d420ee
  # bad: [453e7859443446b837d905d6f2983a76c867247d] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()
  git bisect bad 453e7859443446b837d905d6f2983a76c867247d
  # good: [0e33126d5def407397deaf617559a1a2a7f4b1ae] x86/mm/pat: Allocate split page tables as kernel page tables
  git bisect good 0e33126d5def407397deaf617559a1a2a7f4b1ae
  # first 'bad' commit: [453e7859443446b837d905d6f2983a76c867247d] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

If there is any information I can provide to help debug this, I am happy
to provide it.

-- 
Cheers,
Nathan