* [PATCH v2 1/5] riscv: mm: Extract helper mark_new_valid_map()
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
@ 2026-03-03 5:29 ` Vivian Wang
2026-03-03 5:29 ` [PATCH v2 2/5] riscv: kfence: Call mark_new_valid_map() for kfence_unprotect() Vivian Wang
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Vivian Wang @ 2026-03-03 5:29 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui
Cc: linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, stable,
Vivian Wang
In preparation of a future patch using the same mechanism for
non-vmalloc addresses, extract the mark_new_valid_map() helper from
flush_cache_vmap().
No functional change intended.
Cc: <stable@vger.kernel.org>
Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
---
arch/riscv/include/asm/cacheflush.h | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h
index 0092513c3376..b1a2ac665792 100644
--- a/arch/riscv/include/asm/cacheflush.h
+++ b/arch/riscv/include/asm/cacheflush.h
@@ -43,20 +43,23 @@ do { \
#ifdef CONFIG_64BIT
extern u64 new_vmalloc[NR_CPUS / sizeof(u64) + 1];
extern char _end[];
+static inline void mark_new_valid_map(void)
+{
+ int i;
+
+ /*
+ * We don't care if concurrently a cpu resets this value since
+ * the only place this can happen is in handle_exception() where
+ * an sfence.vma is emitted.
+ */
+ for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i)
+ new_vmalloc[i] = -1ULL;
+}
#define flush_cache_vmap flush_cache_vmap
static inline void flush_cache_vmap(unsigned long start, unsigned long end)
{
- if (is_vmalloc_or_module_addr((void *)start)) {
- int i;
-
- /*
- * We don't care if concurrently a cpu resets this value since
- * the only place this can happen is in handle_exception() where
- * an sfence.vma is emitted.
- */
- for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i)
- new_vmalloc[i] = -1ULL;
- }
+ if (is_vmalloc_or_module_addr((void *)start))
+ mark_new_valid_map();
}
#define flush_cache_vmap_early(start, end) local_flush_tlb_kernel_range(start, end)
#endif
--
2.53.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 2/5] riscv: kfence: Call mark_new_valid_map() for kfence_unprotect()
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
2026-03-03 5:29 ` [PATCH v2 1/5] riscv: mm: Extract helper mark_new_valid_map() Vivian Wang
@ 2026-03-03 5:29 ` Vivian Wang
2026-03-03 5:29 ` [PATCH v2 3/5] riscv: mm: Rename new_vmalloc into new_valid_map_cpus Vivian Wang
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Vivian Wang @ 2026-03-03 5:29 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui
Cc: linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, stable,
Yanko Kaneti, Vivian Wang
In kfence_protect_page(), which kfence_unprotect() calls, we cannot send
IPIs to other CPUs to ask them to flush TLB. This may lead to those CPUs
spuriously faulting on a recently allocated kfence object despite it
being valid, leading to false positive use-after-free reports.
Fix this by calling mark_new_valid_map() so that the page fault handling
code path notices the spurious fault and flushes TLB then retries the
access.
Update the comment in handle_exception to indicate that
new_valid_map_cpus_check also handles kfence_unprotect() spurious
faults.
Note that kfence_protect() has the same stale TLB entries problem, but
that leads to false negatives, which is fine with kfence.
Cc: <stable@vger.kernel.org>
Reported-by: Yanko Kaneti <yaneti@declera.com>
Fixes: b3431a8bb336 ("riscv: Fix IPIs usage in kfence_protect_page()")
Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
---
arch/riscv/include/asm/kfence.h | 7 +++++--
arch/riscv/kernel/entry.S | 6 ++++--
2 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/include/asm/kfence.h b/arch/riscv/include/asm/kfence.h
index d08bf7fb3aee..29cb3a6ee113 100644
--- a/arch/riscv/include/asm/kfence.h
+++ b/arch/riscv/include/asm/kfence.h
@@ -6,6 +6,7 @@
#include <linux/kfence.h>
#include <linux/pfn.h>
#include <asm-generic/pgalloc.h>
+#include <asm/cacheflush.h>
#include <asm/pgtable.h>
static inline bool arch_kfence_init_pool(void)
@@ -17,10 +18,12 @@ static inline bool kfence_protect_page(unsigned long addr, bool protect)
{
pte_t *pte = virt_to_kpte(addr);
- if (protect)
+ if (protect) {
set_pte(pte, __pte(pte_val(ptep_get(pte)) & ~_PAGE_PRESENT));
- else
+ } else {
set_pte(pte, __pte(pte_val(ptep_get(pte)) | _PAGE_PRESENT));
+ mark_new_valid_map();
+ }
preempt_disable();
local_flush_tlb_kernel_range(addr, addr + PAGE_SIZE);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 60eb221296a6..ced7a2b160ce 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -136,8 +136,10 @@ SYM_CODE_START(handle_exception)
#ifdef CONFIG_64BIT
/*
- * The RISC-V kernel does not eagerly emit a sfence.vma after each
- * new vmalloc mapping, which may result in exceptions:
+ * The RISC-V kernel does not flush TLBs on all CPUS after each new
+ * vmalloc mapping or kfence_unprotect(), which may result in
+ * exceptions:
+ *
* - if the uarch caches invalid entries, the new mapping would not be
* observed by the page table walker and an invalidation is needed.
* - if the uarch does not cache invalid entries, a reordered access
--
2.53.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 3/5] riscv: mm: Rename new_vmalloc into new_valid_map_cpus
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
2026-03-03 5:29 ` [PATCH v2 1/5] riscv: mm: Extract helper mark_new_valid_map() Vivian Wang
2026-03-03 5:29 ` [PATCH v2 2/5] riscv: kfence: Call mark_new_valid_map() for kfence_unprotect() Vivian Wang
@ 2026-03-03 5:29 ` Vivian Wang
2026-03-03 5:29 ` [PATCH v2 4/5] riscv: mm: Use the bitmap API for new_valid_map_cpus Vivian Wang
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Vivian Wang @ 2026-03-03 5:29 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui
Cc: linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, Vivian Wang
Since this mechanism is now used for the kfence pool, which comes from
the linear mapping and not vmalloc, rename new_vmalloc into
new_valid_map_cpus to avoid misleading readers.
No functional change intended.
Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
---
arch/riscv/include/asm/cacheflush.h | 6 +++---
arch/riscv/kernel/entry.S | 38 ++++++++++++++++++-------------------
arch/riscv/mm/init.c | 2 +-
3 files changed, 23 insertions(+), 23 deletions(-)
diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h
index b1a2ac665792..8c7a0ef2635a 100644
--- a/arch/riscv/include/asm/cacheflush.h
+++ b/arch/riscv/include/asm/cacheflush.h
@@ -41,7 +41,7 @@ do { \
} while (0)
#ifdef CONFIG_64BIT
-extern u64 new_vmalloc[NR_CPUS / sizeof(u64) + 1];
+extern u64 new_valid_map_cpus[NR_CPUS / sizeof(u64) + 1];
extern char _end[];
static inline void mark_new_valid_map(void)
{
@@ -52,8 +52,8 @@ static inline void mark_new_valid_map(void)
* the only place this can happen is in handle_exception() where
* an sfence.vma is emitted.
*/
- for (i = 0; i < ARRAY_SIZE(new_vmalloc); ++i)
- new_vmalloc[i] = -1ULL;
+ for (i = 0; i < ARRAY_SIZE(new_valid_map_cpus); ++i)
+ new_valid_map_cpus[i] = -1ULL;
}
#define flush_cache_vmap flush_cache_vmap
static inline void flush_cache_vmap(unsigned long start, unsigned long end)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index ced7a2b160ce..9c6acfd09141 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -20,44 +20,44 @@
.section .irqentry.text, "ax"
-.macro new_vmalloc_check
+.macro new_valid_map_cpus_check
REG_S a0, TASK_TI_A0(tp)
csrr a0, CSR_CAUSE
/* Exclude IRQs */
- blt a0, zero, .Lnew_vmalloc_restore_context_a0
+ blt a0, zero, .Lnew_valid_map_cpus_restore_context_a0
REG_S a1, TASK_TI_A1(tp)
- /* Only check new_vmalloc if we are in page/protection fault */
+ /* Only check new_valid_map_cpus if we are in page/protection fault */
li a1, EXC_LOAD_PAGE_FAULT
- beq a0, a1, .Lnew_vmalloc_kernel_address
+ beq a0, a1, .Lnew_valid_map_cpus_kernel_address
li a1, EXC_STORE_PAGE_FAULT
- beq a0, a1, .Lnew_vmalloc_kernel_address
+ beq a0, a1, .Lnew_valid_map_cpus_kernel_address
li a1, EXC_INST_PAGE_FAULT
- bne a0, a1, .Lnew_vmalloc_restore_context_a1
+ bne a0, a1, .Lnew_valid_map_cpus_restore_context_a1
-.Lnew_vmalloc_kernel_address:
+.Lnew_valid_map_cpus_kernel_address:
/* Is it a kernel address? */
csrr a0, CSR_TVAL
- bge a0, zero, .Lnew_vmalloc_restore_context_a1
+ bge a0, zero, .Lnew_valid_map_cpus_restore_context_a1
/* Check if a new vmalloc mapping appeared that could explain the trap */
REG_S a2, TASK_TI_A2(tp)
/*
* Computes:
- * a0 = &new_vmalloc[BIT_WORD(cpu)]
+ * a0 = &new_valid_map_cpus[BIT_WORD(cpu)]
* a1 = BIT_MASK(cpu)
*/
lw a2, TASK_TI_CPU(tp)
/*
- * Compute the new_vmalloc element position:
+ * Compute the new_valid_map_cpus element position:
* (cpu / 64) * 8 = (cpu >> 6) << 3
*/
srli a1, a2, 6
slli a1, a1, 3
- la a0, new_vmalloc
+ la a0, new_valid_map_cpus
add a0, a0, a1
/*
- * Compute the bit position in the new_vmalloc element:
+ * Compute the bit position in the new_valid_map_cpus element:
* bit_pos = cpu % 64 = cpu - (cpu / 64) * 64 = cpu - (cpu >> 6) << 6
* = cpu - ((cpu >> 6) << 3) << 3
*/
@@ -67,12 +67,12 @@
li a2, 1
sll a1, a2, a1
- /* Check the value of new_vmalloc for this cpu */
+ /* Check the value of new_valid_map_cpus for this cpu */
REG_L a2, 0(a0)
and a2, a2, a1
- beq a2, zero, .Lnew_vmalloc_restore_context
+ beq a2, zero, .Lnew_valid_map_cpus_restore_context
- /* Atomically reset the current cpu bit in new_vmalloc */
+ /* Atomically reset the current cpu bit in new_valid_map_cpus */
amoxor.d a0, a1, (a0)
/* Only emit a sfence.vma if the uarch caches invalid entries */
@@ -84,11 +84,11 @@
csrw CSR_SCRATCH, x0
sret
-.Lnew_vmalloc_restore_context:
+.Lnew_valid_map_cpus_restore_context:
REG_L a2, TASK_TI_A2(tp)
-.Lnew_vmalloc_restore_context_a1:
+.Lnew_valid_map_cpus_restore_context_a1:
REG_L a1, TASK_TI_A1(tp)
-.Lnew_vmalloc_restore_context_a0:
+.Lnew_valid_map_cpus_restore_context_a0:
REG_L a0, TASK_TI_A0(tp)
.endm
@@ -146,7 +146,7 @@ SYM_CODE_START(handle_exception)
* could "miss" the new mapping and traps: in that case, we only need
* to retry the access, no sfence.vma is required.
*/
- new_vmalloc_check
+ new_valid_map_cpus_check
#endif
REG_S sp, TASK_TI_KERNEL_SP(tp)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 811e03786c56..9922c22a2a5f 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -37,7 +37,7 @@
#include "../kernel/head.h"
-u64 new_vmalloc[NR_CPUS / sizeof(u64) + 1];
+u64 new_valid_map_cpus[NR_CPUS / sizeof(u64) + 1];
struct kernel_mapping kernel_map __ro_after_init;
EXPORT_SYMBOL(kernel_map);
--
2.53.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 4/5] riscv: mm: Use the bitmap API for new_valid_map_cpus
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
` (2 preceding siblings ...)
2026-03-03 5:29 ` [PATCH v2 3/5] riscv: mm: Rename new_vmalloc into new_valid_map_cpus Vivian Wang
@ 2026-03-03 5:29 ` Vivian Wang
2026-03-03 5:29 ` [PATCH v2 5/5] riscv: mm: Unconditionally sfence.vma for spurious fault Vivian Wang
2026-06-07 6:40 ` [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Paul Walmsley
5 siblings, 0 replies; 7+ messages in thread
From: Vivian Wang @ 2026-03-03 5:29 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui
Cc: linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, Vivian Wang
The bitmap was defined with incorrect size. Fix it by using the proper
bitmap API in C code. The corresponding assembly code is still okay and
remains unchanged.
Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
---
arch/riscv/include/asm/cacheflush.h | 8 +++-----
arch/riscv/mm/init.c | 2 +-
2 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h
index 8c7a0ef2635a..8cfe59483a8f 100644
--- a/arch/riscv/include/asm/cacheflush.h
+++ b/arch/riscv/include/asm/cacheflush.h
@@ -41,19 +41,17 @@ do { \
} while (0)
#ifdef CONFIG_64BIT
-extern u64 new_valid_map_cpus[NR_CPUS / sizeof(u64) + 1];
+/* This is accessed in assembly code. cpumask_var_t would be too complex. */
+extern DECLARE_BITMAP(new_valid_map_cpus, NR_CPUS);
extern char _end[];
static inline void mark_new_valid_map(void)
{
- int i;
-
/*
* We don't care if concurrently a cpu resets this value since
* the only place this can happen is in handle_exception() where
* an sfence.vma is emitted.
*/
- for (i = 0; i < ARRAY_SIZE(new_valid_map_cpus); ++i)
- new_valid_map_cpus[i] = -1ULL;
+ bitmap_fill(new_valid_map_cpus, NR_CPUS);
}
#define flush_cache_vmap flush_cache_vmap
static inline void flush_cache_vmap(unsigned long start, unsigned long end)
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 9922c22a2a5f..a2fc70f72269 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -37,7 +37,7 @@
#include "../kernel/head.h"
-u64 new_valid_map_cpus[NR_CPUS / sizeof(u64) + 1];
+DECLARE_BITMAP(new_valid_map_cpus, NR_CPUS);
struct kernel_mapping kernel_map __ro_after_init;
EXPORT_SYMBOL(kernel_map);
--
2.53.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 5/5] riscv: mm: Unconditionally sfence.vma for spurious fault
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
` (3 preceding siblings ...)
2026-03-03 5:29 ` [PATCH v2 4/5] riscv: mm: Use the bitmap API for new_valid_map_cpus Vivian Wang
@ 2026-03-03 5:29 ` Vivian Wang
2026-06-07 6:40 ` [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Paul Walmsley
5 siblings, 0 replies; 7+ messages in thread
From: Vivian Wang @ 2026-03-03 5:29 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui
Cc: linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, stable,
Vivian Wang
Svvptc does not guarantee that it's safe to just return here. Since we
have already cleared our bit, if, theoretically, the bounded timeframe
for the accessed page to become valid still hasn't happened after sret,
we could fault again and actually crash.
Hopefully, these spurious faults should be rare enough that this is an
acceptable slowdown.
Cc: <stable@vger.kernel.org>
Fixes: 503638e0babf ("riscv: Stop emitting preventive sfence.vma for new vmalloc mappings")
Signed-off-by: Vivian Wang <wangruikang@iscas.ac.cn>
---
arch/riscv/kernel/entry.S | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 9c6acfd09141..34717bd1fa91 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -75,8 +75,11 @@
/* Atomically reset the current cpu bit in new_valid_map_cpus */
amoxor.d a0, a1, (a0)
- /* Only emit a sfence.vma if the uarch caches invalid entries */
- ALTERNATIVE("sfence.vma", "nop", 0, RISCV_ISA_EXT_SVVPTC, 1)
+ /*
+ * A sfence.vma is required here. Even if we had Svvptc, there's no
+ * guarantee that after returning we wouldn't just fault again.
+ */
+ sfence.vma
REG_L a0, TASK_TI_A0(tp)
REG_L a1, TASK_TI_A1(tp)
--
2.53.0
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes
2026-03-03 5:29 [PATCH v2 0/5] riscv: kfence: Handle the spurious fault after kfence_unprotect(), and related fixes Vivian Wang
` (4 preceding siblings ...)
2026-03-03 5:29 ` [PATCH v2 5/5] riscv: mm: Unconditionally sfence.vma for spurious fault Vivian Wang
@ 2026-06-07 6:40 ` Paul Walmsley
5 siblings, 0 replies; 7+ messages in thread
From: Paul Walmsley @ 2026-06-07 6:40 UTC (permalink / raw)
To: Vivian Wang
Cc: Paul Walmsley, Palmer Dabbelt, Alexandre Ghiti,
Alexander Potapenko, Marco Elver, Dmitry Vyukov, Yunhui Cui,
linux-riscv, linux-kernel, kasan-dev, Palmer Dabbelt, stable,
Yanko Kaneti
On Tue, 3 Mar 2026, Vivian Wang wrote:
> kfence_unprotect() on RISC-V doesn't flush TLBs, because we can't send
> IPIs in some contexts where kfence objects are allocated. This leads to
> spurious faults and kfence false positives.
>
> Avoid these spurious faults using the same "new_vmalloc" mechanism,
> which I have renamed new_valid_map_cpus to avoid confusion, since the
> kfence pool comes from the linear mapping, not vmalloc.
>
> Commit b3431a8bb336 ("riscv: Fix IPIs usage in kfence_protect_page()")
> only seemed to consider false negatives, which are indeed tolerable.
> False positives on the other hand are not okay since they waste
> developer time (or just my time somehow?) and spam kmsg making
> diagnosing other problems difficult.
>
> Patch 2 is the implementation to poke (what was called) new_vmalloc upon
> kfence_unprotect(). Patch 1 is some refactoring that patch 2 depends on.
> Patch 3 through 5 are some additional refactoring and minor fixes.
Thanks, queued for v7.2.
- Paul
^ permalink raw reply [flat|nested] 7+ messages in thread