[RFC][PATCH 00/24] x86 Kernel Shadow Stacks

[RFC][PATCH 00/24] x86 Kernel Shadow Stacks

[Dave Hansen]

Shadow stacks are a processor security feature that provides a
parallel, separate, protected stack for storing return addresses
(mostly). There is currently support for them when running userspace,
but not running the kernel.

Shadow stacks are a very powerful control flow protection mechanism.
They represent probably the single most important kernel mitigation in
silicon that Linux is not using.

This series is a proof-of-concept to run the kernel with shadow
stacks. Some of it will hopefully be mergeable in its current form.
Other parts are pure junk and will need to be completely redone.
Note the lack of SoB lines.

The point of the series and of this post is to start the conversation
about what an initial merge might look like. What are the
showstoppers?  Let's say eBPF was not supported, or you could only
use kernel shadow stacks if mitigation=off. Would that be mergeable?

I also owe a big debt to the Xen project and Andrew Cooper for
leading the way on this with their implementation and answering
questions along the way. Thank you Andy!

Let's dive in.

The series is roughly broken up into a few parts. This ordering is
not strict yet and needs refinement.

Patches  1- 7: arch-independent infrastructure to get shadow stacks
	       in to existing thread/task structures
Patches  8-16: Run the kernel single threaded with shadow stacks.
Patches 16-19: Run userspace and context switch between tasks
Patches 20-23: Handle kernel RIP modifications

== Background ==

=== Page Permissions ===

Shadow stacks have special page permissions. They are read-only and
dirty (Write=0, Dirty=1). This means that the normal direct map can
not be used to map them. Either the direct map needs to be modified,
or some alias mapping needs to be created.

=== FRED ===

The kernel often needs to stop normal execution and go off to do
something else like handle an interrupt or exception. Handling these
events often means manipulating the stack. FRED is a new (released
in 2026 CPUs) x86 hardware feature that simplifies event handling,
including some stack manipulation. When in use, FRED replaces the
ancient IDT mechanism. This means that kernel stack manipulation when
using FRED and the IDT look very different.

By "look very different", I mean that a FRED kernel shadow stack
implementation is relatively straightforward and the IDT is a
hellscape of essentially unfixable problems.

== Design ==

There are a few critical parts of the design:
 1. A shadow stack is allocated per-thread. Space is carved out
    in the thread_info structure, adjacent to the normal stack.
    vmap'd stacks are leveraged. They already provide an alias of the
    program stack with 4k pages so it is straightforward to create an
    alias of the 4k shadow stack page with its special permissions in
    the same area.
 2. FRED is a required prerequisite. If someone wants to go back and
    do an IDT implementation, I'm all for it. But FRED is way simpler
    and it is absolutely the place to _start_.
 3. In addition to the per-thread stacks, there are also kernel
    exception stacks. This is also true for shadow stacks. Leverage
    the "CEA" infrastructure to allocate space for the exception
    shadow stacks. (look for "x86/cea" patches).
 4. The kernel shadow stack pointer needs per-thread storage and needs
    to be context-switched.
 5. Any event handling (exceptions or interrupts) that modifies the
    "return to" instruction pointer now needs corresponding changes
    to the shadow stack event frame.

== Warts ==

In order of things I want folks looking at.

 * The shadow stack is not used for the init_task. The most ideal
   thing would be to find a convenient, simple time where the regular
   stack is shallow and it is simple to create a corresponding shadow
   stack. This is not currently done.
 * The context switching assembly is overly complicated and also
   subverts the safety in the architecture-intended
   RSTORSSP/SAVEPREVSSP sequence. This is more nicely fixed once all
   tasks have a shadow stack, including the init task.
 * Locating the FRED shadow stack event frame is ugly ("Stash kernel
   SSP on FRED entry"). It is currently implemented by reading the
   shadow stack pointer during event entry and stashing it on the
   normal stack. Xen seems to accomplish the goal by unwinding the
   stack instead, which has more complicated code, but has adds no
   complexity or overhead to event entry or the entry data structures.
 * The direct map alias of the shadow stack is left in place. This
   means that an attacker can write to the shadow stack without needing
   to leverage one of the very limited WRSS kernel sites. This is a
   big weakness in the security model. But, kernel shadow stacks with
   this weakness are still way better than no kernel shadow stacks.

== Future Work / TODO ==

 * The main thing left is to chew through the list of things in the
   last patch that are incompatible with kernel shadow stacks. There
   are some quite essential things in there.
 * Add a "warn on #CP and continue". Good for debugging. But, bad
   for security. Ideally, a secure kernel would have no easy to
   reach gadgets to disable shadow stacks.
 * Zap the direct map shadow stack alias. Might also require some
   performance optimization work to try and constrain the shadow
   stack allocations to a more modest set of the physical address
   space so the direct map damage is lessened.
 * Work with non-vmap stacks
 * Work with IDT-based systems (without FRED)

== Testing ==

I'm testing this on actual FRED hardware, an off the shelf "Intel(R)
Core(TM) Ultra 5 325" Panther Lake system. So far, I've only run it
under a qemu that has "cet-ss" suport (11.1.2) on a host with
out-of-tree support for CET-SS:

	https://github.com/sohilmeh/linux-dev/

It boots to userspace with 4 CPUs.

This series is based on 7.3-rc6.

 arch/x86/Kconfig                      |  116 ++++++++++++++++++++++++++++++++++
 arch/x86/Kconfig.cpufeatures          |    5 +
 arch/x86/entry/entry_64.S             |   87 +++++++++++++++++++++++++
 arch/x86/entry/entry_64_fred.S        |   22 ++++++
 arch/x86/include/asm/cpu_entry_area.h |   38 +++++++----
 arch/x86/include/asm/cpufeatures.h    |    9 +-
 arch/x86/include/asm/entry-common.h   |    2 
 arch/x86/include/asm/fred.h           |    8 ++
 arch/x86/include/asm/insn-eval.h      |   12 +++
 arch/x86/include/asm/page_64_types.h  |    9 ++
 arch/x86/include/asm/pgtable.h        |    3 
 arch/x86/include/asm/pgtable_types.h  |    2 
 arch/x86/include/asm/processor.h      |    4 +
 arch/x86/include/asm/shstk.h          |    3 
 arch/x86/include/asm/special_insns.h  |    7 ++
 arch/x86/include/asm/thread_info.h    |   13 +++
 arch/x86/kernel/asm-offsets.c         |    3 
 arch/x86/kernel/cet.c                 |   54 +++++++++++++++
 arch/x86/kernel/cpu/common.c          |   14 ++--
 arch/x86/kernel/dumpstack_64.c        |    9 +-
 arch/x86/kernel/fred.c                |   14 ++++
 arch/x86/kernel/process.c             |   35 ++++++++++
 arch/x86/kernel/shstk.c               |   21 ++++++
 arch/x86/mm/cpu_entry_area.c          |   25 ++++---
 arch/x86/mm/extable.c                 |    2 
 include/linux/sched.h                 |   10 ++
 include/linux/sched/task_stack.h      |   28 ++++++--
 include/linux/thread_info.h           |   12 +++
 kernel/fork.c                         |   54 ++++++++++++---
 mm/kasan/report.c                     |    6 -
 30 files changed, 570 insertions(+), 57 deletions(-)

[RFC][PATCH 01/24] kernel/fork: Break out vmap task allocation into helper

[Dave Hansen]

Prepare for kernel shadow stacks. Move the low-level vmalloc call
into its own helper. This will soon have to deal with more things
like appending the shadow stack allocation and setting up the
different permissions for shadow stack pages.

---

 b/kernel/fork.c |   11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff -puN kernel/fork.c~alloc_thread_stack_node-perms kernel/fork.c
--- a/kernel/fork.c~alloc_thread_stack_node-perms	2026-10-21 12:36:32.589870843 -0700
+++ b/kernel/fork.c	2026-10-21 12:36:32.592870848 -0700
@@ -330,6 +330,13 @@ err:
 	return ret;
 }
 
+static void *__vmalloc_thread_stack_node(int node)
+{
+	return __vmalloc_node(THREAD_SIZE, THREAD_ALIGN,
+			      GFP_VMAP_STACK,
+			      node, __builtin_return_address(0));
+}
+
 static int alloc_thread_stack_node(struct task_struct *tsk, int node)
 {
 	struct vm_struct *vm_area;
@@ -356,9 +363,7 @@ static int alloc_thread_stack_node(struc
 		return 0;
 	}
 
-	stack = __vmalloc_node(THREAD_SIZE, THREAD_ALIGN,
-				     GFP_VMAP_STACK,
-				     node, __builtin_return_address(0));
+	stack = __vmalloc_thread_stack_node(node);
 	if (!stack)
 		return -ENOMEM;
 
_

[RFC][PATCH 02/24] kernel/fork: Use sizeof() union instead of TASK_SIZE

[Dave Hansen]

thread_union is sized with TASK_SIZE. Instead of needing the definition
and this code to match, just use the thread_union size. This way, if the
thread union size changes, this code will still be correct.

Foreshadowing: shadow stack support will change the size of thread_union.

---

 b/kernel/fork.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff -puN kernel/fork.c~alloc_thread_stack_node-2 kernel/fork.c
--- a/kernel/fork.c~alloc_thread_stack_node-2	2026-10-21 12:36:33.082871786 -0700
+++ b/kernel/fork.c	2026-10-21 12:36:33.085871792 -0700
@@ -315,7 +315,7 @@ static int memcg_charge_kernel_stack(str
 	int ret;
 	int nr_charged = 0;
 
-	BUG_ON(vm_area->nr_pages != THREAD_SIZE / PAGE_SIZE);
+	BUG_ON(vm_area->nr_pages != sizeof(union thread_union) / PAGE_SIZE);
 
 	for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) {
 		ret = memcg_kmem_charge_page(vm_area->pages[i], GFP_KERNEL, 0);
@@ -332,7 +332,7 @@ err:
 
 static void *__vmalloc_thread_stack_node(int node)
 {
-	return __vmalloc_node(THREAD_SIZE, THREAD_ALIGN,
+	return __vmalloc_node(sizeof(union thread_union), THREAD_ALIGN,
 			      GFP_VMAP_STACK,
 			      node, __builtin_return_address(0));
 }
_

[RFC][PATCH 03/24] kernel/sched: Move thread stack to its own structure

[Dave Hansen]

Right now, the thread stack is directly declared in 'thread_union'.
There is shortly going to be more than one stack. Give all the stacks
their own structure instead of declaring *a* stack directly.

---

 b/include/linux/sched.h |    6 +++++-
 b/mm/kasan/report.c     |    6 +++---
 2 files changed, 8 insertions(+), 4 deletions(-)

diff -puN include/linux/sched.h~alloc_thread_stack_node-3 include/linux/sched.h
--- a/include/linux/sched.h~alloc_thread_stack_node-3	2026-10-21 12:36:33.575872729 -0700
+++ b/include/linux/sched.h	2026-10-21 12:36:33.613872802 -0700
@@ -1986,12 +1986,16 @@ extern void ia64_set_curr_task(int cpu,
 
 void yield(void);
 
+struct thread_stacks {
+	unsigned long stack[THREAD_SIZE/sizeof(long)];
+};
+
 union thread_union {
 	struct task_struct task;
 #ifndef CONFIG_THREAD_INFO_IN_TASK
 	struct thread_info thread_info;
 #endif
-	unsigned long stack[THREAD_SIZE/sizeof(long)];
+	struct thread_stacks stacks;
 };
 
 #ifndef CONFIG_THREAD_INFO_IN_TASK
diff -puN mm/kasan/report.c~alloc_thread_stack_node-3 mm/kasan/report.c
--- a/mm/kasan/report.c~alloc_thread_stack_node-3	2026-10-21 12:36:33.610872797 -0700
+++ b/mm/kasan/report.c	2026-10-21 12:36:33.613872802 -0700
@@ -365,9 +365,9 @@ static inline bool kernel_or_module_addr
 
 static inline bool init_task_stack_addr(const void *addr)
 {
-	return addr >= (void *)&init_thread_union.stack &&
-		(addr <= (void *)&init_thread_union.stack +
-			sizeof(init_thread_union.stack));
+	return addr >= (void *)&init_thread_union.stacks.stack &&
+		(addr <= (void *)&init_thread_union.stacks.stack +
+			sizeof(init_thread_union.stacks.stack));
 }
 
 static void print_address_description(void *addr, u8 tag,
_

[RFC][PATCH 04/24] kernel/fork: Allow arch-specific stack initialization

[Dave Hansen]

Right now, the entire thread_union has uniform permissions. Shadow
stacks obviously do not have the same page permissions as the rest
of the stacks. Add a new arch-specific hook so shadow stack page
permissions can be established.

Note: this is slow. It requires TLB invalidation in the arch code,
which is abhorrently stupid. It will be fixed up in due time.

---

 b/include/linux/thread_info.h |    5 +++++
 b/kernel/fork.c               |   14 +++++++++++---
 2 files changed, 16 insertions(+), 3 deletions(-)

diff -puN include/linux/thread_info.h~generic-arch_init_thread_stacks include/linux/thread_info.h
--- a/include/linux/thread_info.h~generic-arch_init_thread_stacks	2026-10-21 12:36:34.114873761 -0700
+++ b/include/linux/thread_info.h	2026-10-21 12:36:34.177873882 -0700
@@ -228,6 +228,11 @@ static inline int arch_within_stack_fram
 static inline void arch_setup_new_exec(void) { }
 #endif
 
+struct thread_stacks;
+#ifndef arch_init_thread_stacks
+static inline void arch_init_thread_stacks(struct thread_stacks *ts) { }
+#endif
+
 void arch_task_cache_init(void); /* for CONFIG_SH */
 void arch_release_task_struct(struct task_struct *tsk);
 int arch_dup_task_struct(struct task_struct *dst,
diff -puN kernel/fork.c~generic-arch_init_thread_stacks kernel/fork.c
--- a/kernel/fork.c~generic-arch_init_thread_stacks	2026-10-21 12:36:34.175873878 -0700
+++ b/kernel/fork.c	2026-10-21 12:36:34.178873884 -0700
@@ -332,9 +332,17 @@ err:
 
 static void *__vmalloc_thread_stack_node(int node)
 {
-	return __vmalloc_node(sizeof(union thread_union), THREAD_ALIGN,
-			      GFP_VMAP_STACK,
-			      node, __builtin_return_address(0));
+	union thread_union *tu;
+
+	tu = __vmalloc_node(sizeof(union thread_union), THREAD_ALIGN,
+			    GFP_VMAP_STACK,
+			    node, __builtin_return_address(0));
+	if (!tu)
+		return NULL;
+
+	arch_init_thread_stacks(&tu->stacks);
+
+	return tu;
 }
 
 static int alloc_thread_stack_node(struct task_struct *tsk, int node)
_

[RFC][PATCH 05/24] kernel/task: Add shadow stack to thread structure

[Dave Hansen]

Add a shadow stack member to the stack structure. It is zero-sized
for now. Adding it permits new code to reference it, though.

---

 b/include/linux/sched.h |    4 ++++
 1 file changed, 4 insertions(+)

diff -puN include/linux/sched.h~add-shadow_stack-to-thread_stacks include/linux/sched.h
--- a/include/linux/sched.h~add-shadow_stack-to-thread_stacks	2026-10-21 12:36:34.689874862 -0700
+++ b/include/linux/sched.h	2026-10-21 12:36:34.691874865 -0700
@@ -1986,8 +1986,12 @@ extern void ia64_set_curr_task(int cpu,
 
 void yield(void);
 
+#ifndef TASK_SHADOW_STACK_SIZE
+#define TASK_SHADOW_STACK_SIZE	0
+#endif
 struct thread_stacks {
 	unsigned long stack[THREAD_SIZE/sizeof(long)];
+	u8 shadow_stack[TASK_SHADOW_STACK_SIZE];
 };
 
 union thread_union {
_

[RFC][PATCH 06/24] kernel/task: Add "runtime" check for shadow stack support

[Dave Hansen]

Add a generic helper to see whether shadow stacks are supported at runtime.
This is needed to switch shadow stack code on and off at runtime. But, it
also serves to help the compiler optimize shadow stack code out as well.

---

 b/include/linux/thread_info.h |    7 +++++++
 1 file changed, 7 insertions(+)

diff -puN include/linux/thread_info.h~generic-arch-shadow-stacks-enabled include/linux/thread_info.h
--- a/include/linux/thread_info.h~generic-arch-shadow-stacks-enabled	2026-10-21 12:36:35.168875778 -0700
+++ b/include/linux/thread_info.h	2026-10-21 12:36:35.170875782 -0700
@@ -233,6 +233,13 @@ struct thread_stacks;
 static inline void arch_init_thread_stacks(struct thread_stacks *ts) { }
 #endif
 
+#ifndef arch_kernel_shstk_enabled
+static inline bool arch_kernel_shstk_enabled(void)
+{
+	return false;
+}
+#endif
+
 void arch_task_cache_init(void); /* for CONFIG_SH */
 void arch_release_task_struct(struct task_struct *tsk);
 int arch_dup_task_struct(struct task_struct *dst,
_

[RFC][PATCH 07/24] kernel/fork: Clear shadow stacks alongside normal stacks

[Dave Hansen]

The old stack clear does not work for shadow stacks work because the
shadow stack page is not writeable via the vmalloc() mapping. It has
shadow stack permissions after all.

Use the 'vm_struct' for the vmap'ing to find the struct page for
the shadow stack and clear it via the direct map alias.

This should all compile back to the old clear_pages() if shadow
stacks are not supported.

This also only supports a single shadow stack page for now. It's
obviously not rocket science to add a loop, but leave it as simple
as possible for now since x86 has a single shadow stack page.

---

 b/kernel/fork.c |   25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff -puN kernel/fork.c~generic-clear-stack-area kernel/fork.c
--- a/kernel/fork.c~generic-clear-stack-area	2026-10-21 12:36:35.657876714 -0700
+++ b/kernel/fork.c	2026-10-21 12:36:35.660876720 -0700
@@ -345,6 +345,28 @@ static void *__vmalloc_thread_stack_node
 	return tu;
 }
 
+static inline void clear_stack_area(struct vm_struct *vm_area)
+{
+	union thread_union *tu = vm_area->addr;
+
+	int nr_stack_pages = sizeof(tu->stacks.stack)        / PAGE_SIZE;
+	int nr_shstk_pages = sizeof(tu->stacks.shadow_stack) / PAGE_SIZE;
+
+	/* Clear stale pointers from reused stack. */
+	clear_pages(&tu->stacks.stack, nr_stack_pages);
+
+	if (!nr_shstk_pages || !arch_kernel_shstk_enabled())
+		return;
+	/*
+	 * Clear the shadow stack page too, but via its direct map
+	 * alias. Note that the shadow stack is right after the
+	 * regular stack, so the nr_stack_pages'th page is the
+	 * first shadow stack page.
+	 */
+	BUILD_BUG_ON(nr_shstk_pages && nr_shstk_pages > 1);
+	clear_highpage(vm_area->pages[nr_stack_pages]);
+}
+
 static int alloc_thread_stack_node(struct task_struct *tsk, int node)
 {
 	struct vm_struct *vm_area;
@@ -363,8 +385,7 @@ static int alloc_thread_stack_node(struc
 
 		stack = kasan_reset_tag(vm_area->addr);
 
-		/* Clear stale pointers from reused stack. */
-		clear_pages(vm_area->addr, vm_area->nr_pages);
+		clear_stack_area(vm_area);
 
 		tsk->stack_vm_area = vm_area;
 		tsk->stack = stack;
_

[RFC][PATCH 08/24] x86/cpu: Add basic x86 kernel shadow stack infrastructure

[Dave Hansen]

Add a kernel shadow stack CPU feature and Kconfig option. Enable it
to help compile testing during development. This polarity will be
flipped in the eventual merge-ready code. TODO/FIXME.

Add the feature bit next to the userspace one. This requires a
minor amount of reshuffling of software bits.

On the naming, use "KERN" to make for nice natural vertical alignment
with "USER".

The Kconfig dependencies are not "real" yet. They represent the
dependencies present in the coming implementation.

---

 b/arch/x86/Kconfig                   |    9 +++++++++
 b/arch/x86/Kconfig.cpufeatures       |    5 +++++
 b/arch/x86/include/asm/cpufeatures.h |    9 +++++----
 b/arch/x86/kernel/cpu/common.c       |   14 ++++++++++----
 4 files changed, 29 insertions(+), 8 deletions(-)

diff -puN arch/x86/include/asm/cpufeatures.h~x86-kernel-shadow-stacks-base arch/x86/include/asm/cpufeatures.h
--- a/arch/x86/include/asm/cpufeatures.h~x86-kernel-shadow-stacks-base	2026-10-21 12:36:36.156877670 -0700
+++ b/arch/x86/include/asm/cpufeatures.h	2026-10-21 12:36:36.187877729 -0700
@@ -309,10 +309,10 @@
 #define X86_FEATURE_SMBA		(11*32+21) /* Slow Memory Bandwidth Allocation */
 #define X86_FEATURE_BMEC		(11*32+22) /* Bandwidth Monitoring Event Configuration */
 #define X86_FEATURE_USER_SHSTK		(11*32+23) /* "user_shstk" Shadow stack support for user mode applications */
-#define X86_FEATURE_SRSO		(11*32+24) /* AMD BTB untrain RETs */
-#define X86_FEATURE_SRSO_ALIAS		(11*32+25) /* AMD BTB untrain RETs through aliasing */
-#define X86_FEATURE_IBPB_ON_VMEXIT	(11*32+26) /* Issue an IBPB only on VMEXIT */
-#define X86_FEATURE_APIC_MSRS_FENCE	(11*32+27) /* IA32_TSC_DEADLINE and X2APIC MSRs need fencing */
+#define X86_FEATURE_KERN_SHSTK		(11*32+24) /* Kernel Shadow stack support */
+#define X86_FEATURE_SRSO		(11*32+25) /* AMD BTB untrain RETs */
+#define X86_FEATURE_SRSO_ALIAS		(11*32+26) /* AMD BTB untrain RETs through aliasing */
+#define X86_FEATURE_IBPB_ON_VMEXIT	(11*32+27) /* Issue an IBPB only on VMEXIT */
 #define X86_FEATURE_ZEN2		(11*32+28) /* CPU based on Zen2 microarchitecture */
 #define X86_FEATURE_ZEN3		(11*32+29) /* CPU based on Zen3 microarchitecture */
 #define X86_FEATURE_ZEN4		(11*32+30) /* CPU based on Zen4 microarchitecture */
@@ -430,6 +430,7 @@
 #define X86_FEATURE_SUCCOR		(17*32+ 1) /* "succor" Uncorrectable error containment and recovery */
 #define X86_FEATURE_CPPC_PERF_PRIO	(17*32+ 2) /* CPPC Floor Perf support */
 #define X86_FEATURE_SMCA		(17*32+ 3) /* "smca" Scalable MCA */
+#define X86_FEATURE_APIC_MSRS_FENCE	(17*32+ 4) /* IA32_TSC_DEADLINE and X2APIC MSRs need fencing */
 
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */
 #define X86_FEATURE_AVX512_4VNNIW	(18*32+ 2) /* "avx512_4vnniw" AVX-512 Neural Network Instructions */
diff -puN arch/x86/Kconfig~x86-kernel-shadow-stacks-base arch/x86/Kconfig
--- a/arch/x86/Kconfig~x86-kernel-shadow-stacks-base	2026-10-21 12:36:36.166877689 -0700
+++ b/arch/x86/Kconfig	2026-10-21 12:36:36.189877733 -0700
@@ -1865,6 +1865,15 @@ config X86_USER_SHADOW_STACK
 
 	  If unsure, say N.
 
+config X86_KERN_SHADOW_STACK
+	bool "X86 kernel shadow stack"
+	depends on X86_FRED # implies X86_64
+	# Shadow stacks are allocated next to the
+	# regular stack in the VMAP area:
+	depends on VMAP_STACK
+	depends on AS_WRUSS
+	select X86_CET
+
 config INTEL_TDX_HOST
 	bool "Intel Trust Domain Extensions (TDX) host support"
 	depends on CPU_SUP_INTEL
diff -puN arch/x86/Kconfig.cpufeatures~x86-kernel-shadow-stacks-base arch/x86/Kconfig.cpufeatures
--- a/arch/x86/Kconfig.cpufeatures~x86-kernel-shadow-stacks-base	2026-10-21 12:36:36.168877693 -0700
+++ b/arch/x86/Kconfig.cpufeatures	2026-10-21 12:36:36.189877733 -0700
@@ -205,3 +205,8 @@ config X86_DISABLED_FEATURE_SEV_SNP
 config X86_DISABLED_FEATURE_INVLPGB
 	def_bool y
 	depends on !BROADCAST_TLB_FLUSH
+
+config X86_DISABLED_FEATURE_KERN_SHSTK
+        def_bool y
+        depends on !X86_KERN_SHADOW_STACK
+
diff -puN arch/x86/kernel/cpu/common.c~x86-kernel-shadow-stacks-base arch/x86/kernel/cpu/common.c
--- a/arch/x86/kernel/cpu/common.c~x86-kernel-shadow-stacks-base	2026-10-21 12:36:36.184877724 -0700
+++ b/arch/x86/kernel/cpu/common.c	2026-10-21 12:36:36.189877733 -0700
@@ -634,20 +634,26 @@ __noendbr void ibt_restore(u64 save)
 
 static __always_inline void setup_cet(struct cpuinfo_x86 *c)
 {
-	bool user_shstk, kernel_ibt;
+	bool user_shstk = 0;
+	bool kern_shstk = 0;
+	bool kernel_ibt;
 
 	if (!IS_ENABLED(CONFIG_X86_CET))
 		return;
 
 	kernel_ibt = HAS_KERNEL_IBT && cpu_feature_enabled(X86_FEATURE_IBT);
-	user_shstk = cpu_feature_enabled(X86_FEATURE_SHSTK) &&
-		     IS_ENABLED(CONFIG_X86_USER_SHADOW_STACK);
+	if (cpu_feature_enabled(X86_FEATURE_SHSTK)) {
+		user_shstk = IS_ENABLED(CONFIG_X86_USER_SHADOW_STACK);
+		kern_shstk = IS_ENABLED(CONFIG_X86_KERN_SHADOW_STACK);
+	}
 
-	if (!kernel_ibt && !user_shstk)
+	if (!kernel_ibt && !user_shstk && !kern_shstk)
 		return;
 
 	if (user_shstk)
 		set_cpu_cap(c, X86_FEATURE_USER_SHSTK);
+	if (kern_shstk)
+		set_cpu_cap(c, X86_FEATURE_KERN_SHSTK);
 
 	if (kernel_ibt)
 		wrmsrq(MSR_IA32_S_CET, CET_ENDBR_EN);
_

[RFC][PATCH 09/24] x86/cea: Refactor "CEA" macros to deal with multiple stacks

[Dave Hansen]

There are a series of CPU Entry Area (CEA) helper macros to make
it easier to deal with the CEA structure geometry. Right now,
many of them can deal only with the regular stack.

Prepare for shadow stacks by refactoring the macros to take a
(partial) structure member name. For now, that is just "stack".

This is a pure refactoring. No behavioral changes intended.

Nits: CEA_ESTACK_PAGES gets a whitespace change to make it
consistent with the new width of the four related macros above
it. There's also some slight rewrapping of the macros in a place
or two to make them more readable.

---

 b/arch/x86/include/asm/cpu_entry_area.h |   23 ++++++++++++-----------
 b/arch/x86/kernel/dumpstack_64.c        |    9 +++++----
 2 files changed, 17 insertions(+), 15 deletions(-)

diff -puN arch/x86/include/asm/cpu_entry_area.h~cea-estacks-shadows arch/x86/include/asm/cpu_entry_area.h
--- a/arch/x86/include/asm/cpu_entry_area.h~cea-estacks-shadows	2026-10-21 12:36:36.741878790 -0700
+++ b/arch/x86/include/asm/cpu_entry_area.h	2026-10-21 12:36:36.746878799 -0700
@@ -55,19 +55,20 @@ enum exception_stack_ordering {
 	N_EXCEPTION_STACKS
 };
 
-#define CEA_ESTACK_SIZE(st)					\
-	sizeof(((struct cea_exception_stacks *)0)->st## _stack)
+#define CEA_ESTACK_SIZE(st, member)					\
+	sizeof(((struct cea_exception_stacks *)0)->st## _ ##member)
 
-#define CEA_ESTACK_BOT(ceastp, st)				\
-	((unsigned long)&(ceastp)->st## _stack)
+#define CEA_ESTACK_BOT(ceastp, st, member)				\
+	((unsigned long)&(ceastp)->st## _ ##member)
 
-#define CEA_ESTACK_TOP(ceastp, st)				\
-	(CEA_ESTACK_BOT(ceastp, st) + CEA_ESTACK_SIZE(st))
+#define CEA_ESTACK_TOP(ceastp, st, member)				\
+	(CEA_ESTACK_BOT(ceastp, st, member) +				\
+	 CEA_ESTACK_SIZE(st, member))
 
-#define CEA_ESTACK_OFFS(st)					\
-	offsetof(struct cea_exception_stacks, st## _stack)
+#define CEA_ESTACK_OFFS(st, member)					\
+	offsetof(struct cea_exception_stacks, st## _ ##member)
 
-#define CEA_ESTACK_PAGES					\
+#define CEA_ESTACK_PAGES						\
 	(sizeof(struct cea_exception_stacks) / PAGE_SIZE)
 
 #endif
@@ -145,9 +146,9 @@ static __always_inline struct entry_stac
 }
 
 #define __this_cpu_ist_top_va(name)					\
-	CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name)
+	CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name, stack)
 
 #define __this_cpu_ist_bottom_va(name)					\
-	CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name)
+	CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name, stack)
 
 #endif
diff -puN arch/x86/kernel/dumpstack_64.c~cea-estacks-shadows arch/x86/kernel/dumpstack_64.c
--- a/arch/x86/kernel/dumpstack_64.c~cea-estacks-shadows	2026-10-21 12:36:36.743878793 -0700
+++ b/arch/x86/kernel/dumpstack_64.c	2026-10-21 12:36:36.746878799 -0700
@@ -69,10 +69,11 @@ struct estack_pages {
 };
 
 #define EPAGERANGE(st)							\
-	[PFN_DOWN(CEA_ESTACK_OFFS(st)) ...				\
-	 PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = {	\
-		.offs	= CEA_ESTACK_OFFS(st),				\
-		.size	= CEA_ESTACK_SIZE(st),				\
+	[PFN_DOWN(CEA_ESTACK_OFFS(st, stack)) ...			\
+	 PFN_DOWN(CEA_ESTACK_OFFS(st, stack) +				\
+		  CEA_ESTACK_SIZE(st, stack) - 1)] = {			\
+		.offs	= CEA_ESTACK_OFFS(st, stack),			\
+		.size	= CEA_ESTACK_SIZE(st, stack),			\
 		.type	= STACK_TYPE_EXCEPTION + ESTACK_ ##st, }
 
 /*
_

[RFC][PATCH 10/24] x86/cea: Refactor stack mapping macros

[Dave Hansen]

Continuing the trend, the stack mapping macros can currently
only work with the "_stack" structure member. Make them more
flexible by taking a partial structure member name.

Also make the mapping permission explicit. This will allow
shadow stack permission mappings in the near future.

This does make following the macro maze a bit harder. But,
in the end, it's quite readable to have code that looks like
this:

	cea_map_stack(MCE, stack, PAGE_KERNEL);
vs.
	cea_map_stack(MCE, shadow_stack, PAGE_KERNEL_SHSTK);


---

 b/arch/x86/mm/cpu_entry_area.c |   20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff -puN arch/x86/mm/cpu_entry_area.c~cea-estacks-shadows-refactor-0 arch/x86/mm/cpu_entry_area.c
--- a/arch/x86/mm/cpu_entry_area.c~cea-estacks-shadows-refactor-0	2026-10-21 12:36:37.273879808 -0700
+++ b/arch/x86/mm/cpu_entry_area.c	2026-10-21 12:36:37.275879812 -0700
@@ -130,10 +130,10 @@ static void __init percpu_setup_debug_st
 
 #ifdef CONFIG_X86_64
 
-#define cea_map_stack(name) do {					\
-	npages = sizeof(estacks->name## _stack) / PAGE_SIZE;		\
-	cea_map_percpu_pages(cea->estacks.name## _stack,		\
-			estacks->name## _stack, npages, PAGE_KERNEL);	\
+#define cea_map_stack(name, member, perm) do {				\
+	npages = sizeof(estacks->name## _ ##member) / PAGE_SIZE;	\
+	cea_map_percpu_pages(cea->estacks.name## _ ##member,		\
+			estacks->name## _ ##member, npages, perm);	\
 	} while (0)
 
 static void __init percpu_setup_exception_stacks(unsigned int cpu)
@@ -151,15 +151,15 @@ static void __init percpu_setup_exceptio
 	 * by guard pages so each stack must be mapped separately. DB2 is
 	 * not mapped; it just exists to catch triple nesting of #DB.
 	 */
-	cea_map_stack(DF);
-	cea_map_stack(NMI);
-	cea_map_stack(DB);
-	cea_map_stack(MCE);
+	cea_map_stack(DF,  stack, PAGE_KERNEL);
+	cea_map_stack(NMI, stack, PAGE_KERNEL);
+	cea_map_stack(DB,  stack, PAGE_KERNEL);
+	cea_map_stack(MCE, stack, PAGE_KERNEL);
 
 	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) {
 		if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
-			cea_map_stack(VC);
-			cea_map_stack(VC2);
+			cea_map_stack(VC,  stack, PAGE_KERNEL);
+			cea_map_stack(VC2, stack, PAGE_KERNEL);
 		}
 	}
 }
_

[RFC][PATCH 11/24] x86/mm: Add kernel shadow stack permission macros

[Dave Hansen]

Add PAGE_KERNEL* macros for kernel-permission shadow stacks. These allow
shadow stack mappings to be established in a visually similar way to
existing PAGE_KERNEL* users.

---

 b/arch/x86/include/asm/pgtable_types.h |    2 ++
 1 file changed, 2 insertions(+)

diff -puN arch/x86/include/asm/pgtable_types.h~PAGE_KERNEL_SHSTK arch/x86/include/asm/pgtable_types.h
--- a/arch/x86/include/asm/pgtable_types.h~PAGE_KERNEL_SHSTK	2026-10-21 12:36:37.794880805 -0700
+++ b/arch/x86/include/asm/pgtable_types.h	2026-10-21 12:36:37.797880811 -0700
@@ -224,6 +224,7 @@ enum page_cache_mode {
 #define __PAGE_KERNEL_ROX	 (__PP|   0|   0|___A|   0|   0|   0|___G)
 #define __PAGE_KERNEL		 (__PP|__RW|   0|___A|__NX|___D|   0|___G)
 #define __PAGE_KERNEL_EXEC	 (__PP|__RW|   0|___A|   0|___D|   0|___G)
+#define __PAGE_KERNEL_SHSTK	 (__PP|   0|   0|___A|__NX|___D|   0|___G)
 #define __PAGE_KERNEL_NOCACHE	 (__PP|__RW|   0|___A|__NX|___D|   0|___G| __NC)
 #define __PAGE_KERNEL_VVAR	 (__PP|   0|_USR|___A|__NX|   0|   0|___G)
 #define __PAGE_KERNEL_LARGE	 (__PP|__RW|   0|___A|__NX|___D|_PSE|___G)
@@ -249,6 +250,7 @@ enum page_cache_mode {
 #define PAGE_KERNEL_RO		__pgprot_mask(__PAGE_KERNEL_RO         | _ENC)
 #define PAGE_KERNEL_EXEC	__pgprot_mask(__PAGE_KERNEL_EXEC       | _ENC)
 #define PAGE_KERNEL_EXEC_NOENC	__pgprot_mask(__PAGE_KERNEL_EXEC       |    0)
+#define PAGE_KERNEL_SHSTK	__pgprot_mask(__PAGE_KERNEL_SHSTK      | _ENC)
 #define PAGE_KERNEL_ROX		__pgprot_mask(__PAGE_KERNEL_ROX        | _ENC)
 #define PAGE_KERNEL_NOCACHE	__pgprot_mask(__PAGE_KERNEL_NOCACHE    | _ENC)
 #define PAGE_KERNEL_LARGE	__pgprot_mask(__PAGE_KERNEL_LARGE      | _ENC)
_

[RFC][PATCH 12/24] x86/cea: Map and use FRED exception shadow stacks

[Dave Hansen]

FRED hardware automatically switches stacks between levels on different
kinds of kernel entries. Each normal FRED "RSP" stack has a corresponding
"SSP" shadow stack.

Leverage the existing CPU Entry Area infrastructure. Add shadow stacks
for levels 1-3 right next to the existing RSP stacks. Map them with
shadow stack permissions. Last, point the FRED SSP MSRs at the exception
stacks.

Just like the task stacks, do not leave a guard page since regular and
shadow stacks have mutually exclusive permissions and serve as guards
for each other.

Nit: The "return style" of the new cea_map_stack() and wrmsr() code
looks superficially inconsistent. But each site is consistent with the
code around it, just not when looking at the two sites against each
other.

== Future Work ==

The *_stack_guard[] elements could probably be removed now. When in
use, shadow stacks serve as guard pages for the regular stacks. When
not in use, the shadow stack region could remain unmapped and also act
as a guard area. But that would take some more fancy dynamic mapping
work. The existing CEA code is dirt simple and doing anything dynamic
would add complexity. KISS. The downside is wasting three pages per
CPU when shadow stacks support is compiled in but not used at runtime.

---

 b/arch/x86/include/asm/cpu_entry_area.h |   15 +++++++++++++++
 b/arch/x86/kernel/fred.c                |   14 ++++++++++++++
 b/arch/x86/mm/cpu_entry_area.c          |    5 +++++
 3 files changed, 34 insertions(+)

diff -puN arch/x86/include/asm/cpu_entry_area.h~cea-estacks-shadows-1 arch/x86/include/asm/cpu_entry_area.h
--- a/arch/x86/include/asm/cpu_entry_area.h~cea-estacks-shadows-1	2026-10-21 12:36:38.292881759 -0700
+++ b/arch/x86/include/asm/cpu_entry_area.h	2026-10-21 12:36:38.320881812 -0700
@@ -10,6 +10,12 @@
 
 #ifdef CONFIG_X86_64
 
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+#define EXCEPTION_SHADOW_STKSZ	PAGE_SIZE
+#else
+#define EXCEPTION_SHADOW_STKSZ	0
+#endif
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 #define VC_EXCEPTION_STKSZ	EXCEPTION_STKSZ
 #else
@@ -19,10 +25,13 @@
 /* Macro to enforce the same ordering and stack sizes */
 #define ESTACKS_MEMBERS(guardsize, optional_stack_size)		\
 	char	DF_stack_guard[guardsize];			\
+	char	DF_shadow_stack[EXCEPTION_SHADOW_STKSZ];	\
 	char	DF_stack[EXCEPTION_STKSZ];			\
 	char	NMI_stack_guard[guardsize];			\
+	char	NMI_shadow_stack[EXCEPTION_SHADOW_STKSZ];	\
 	char	NMI_stack[EXCEPTION_STKSZ];			\
 	char	DB_stack_guard[guardsize];			\
+	char	DB_shadow_stack[EXCEPTION_SHADOW_STKSZ];	\
 	char	DB_stack[EXCEPTION_STKSZ];			\
 	char	MCE_stack_guard[guardsize];			\
 	char	MCE_stack[EXCEPTION_STKSZ];			\
@@ -151,4 +160,10 @@ static __always_inline struct entry_stac
 #define __this_cpu_ist_bottom_va(name)					\
 	CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name, stack)
 
+#define __this_cpu_shadow_bottom_va(name)				\
+	CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name, shadow_stack)
+
+#define __this_cpu_shadow_top_va(name)					\
+	CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name, shadow_stack)
+
 #endif
diff -puN arch/x86/kernel/fred.c~cea-estacks-shadows-1 arch/x86/kernel/fred.c
--- a/arch/x86/kernel/fred.c~cea-estacks-shadows-1	2026-10-21 12:36:38.294881762 -0700
+++ b/arch/x86/kernel/fred.c	2026-10-21 12:36:38.320881812 -0700
@@ -87,4 +87,18 @@ void cpu_init_fred_rsps(void)
 	wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
 	wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
 	wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
+
+	if (!cpu_has(&boot_cpu_data, X86_FEATURE_SHSTK))
+		return;
+
+	/*
+	 * ...and the FRED shadow stacks when supported by the CPU.
+	 *
+	 * Note that this is strictly checking CPU support, not the
+	 * kernel feature flag: X86_FEATURE_KERN_SHSTK. These MSRs
+	 * may get set and then never used.
+	 */
+	wrmsrq(MSR_IA32_FRED_SSP1, __this_cpu_shadow_top_va(DB));
+	wrmsrq(MSR_IA32_FRED_SSP2, __this_cpu_shadow_top_va(NMI));
+	wrmsrq(MSR_IA32_FRED_SSP3, __this_cpu_shadow_top_va(DF));
 }
diff -puN arch/x86/mm/cpu_entry_area.c~cea-estacks-shadows-1 arch/x86/mm/cpu_entry_area.c
--- a/arch/x86/mm/cpu_entry_area.c~cea-estacks-shadows-1	2026-10-21 12:36:38.318881808 -0700
+++ b/arch/x86/mm/cpu_entry_area.c	2026-10-21 12:36:38.321881814 -0700
@@ -162,6 +162,11 @@ static void __init percpu_setup_exceptio
 			cea_map_stack(VC2, stack, PAGE_KERNEL);
 		}
 	}
+
+	/* Map the per-CPU exception shadow stacks: */
+	cea_map_stack(DB,  shadow_stack, PAGE_KERNEL_SHSTK);
+	cea_map_stack(NMI, shadow_stack, PAGE_KERNEL_SHSTK);
+	cea_map_stack(DF,  shadow_stack, PAGE_KERNEL_SHSTK);
 }
 #else
 static void __init percpu_setup_exception_stacks(unsigned int cpu)
_

[RFC][PATCH 13/24] x86/task: Grow thread_union for shadow stacks

[Dave Hansen]

Define the size of an x86 shadow stack: one page. This will inflate the
virtual size and physical backing of thread_union, wasting one page
per task on non-shadow-stack configurations.

This waste can be cleaned up with the "double vmap()" approach discussed
earlier. For now, please ignore the gluttonous use of DRAM.

---

 b/arch/x86/include/asm/page_64_types.h |    9 +++++++++
 1 file changed, 9 insertions(+)

diff -puN arch/x86/include/asm/page_64_types.h~x86-define-shadow-stack-size arch/x86/include/asm/page_64_types.h
--- a/arch/x86/include/asm/page_64_types.h~x86-define-shadow-stack-size	2026-10-21 12:36:38.870882865 -0700
+++ b/arch/x86/include/asm/page_64_types.h	2026-10-21 12:36:38.872882869 -0700
@@ -15,6 +15,15 @@
 #define THREAD_SIZE_ORDER	(2 + KASAN_STACK_ORDER)
 #define THREAD_SIZE  (PAGE_SIZE << THREAD_SIZE_ORDER)
 
+/*
+ * Grow 'thread_union' to make space for the shadow stack
+ * page. For now, this wastes a page of space per task on
+ * systems that do not support shadow stacks.
+ */
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+#define TASK_SHADOW_STACK_SIZE PAGE_SIZE
+#endif
+
 #define EXCEPTION_STACK_ORDER (1 + KASAN_STACK_ORDER)
 #define EXCEPTION_STKSZ (PAGE_SIZE << EXCEPTION_STACK_ORDER)
 
_

[RFC][PATCH 14/24] x86/mm: Establish shadow stack permissions in thread mapping

[Dave Hansen]

This represents the simplest way to establish a per-task shadow stack
mapping. It is not an efficient way to do it. It costs a TLB flush
each time a stack is allocated which is wicked stupid and slow. But,
it's simple and will function for now. Fix it properly later by doing:

	vm_area = __get_vm_area_node()
	pages = kmalloc(); // the array
	vmap(PAGE_KERNEL) // 16k
	vmap(PAGE_KERNEL_SHSTK) // 4k

That way, there's no TLB flush, and the shadow stack permissions are
set up properly. The shadow stack is allocated in the same way as the
normal stack, and the permissions are set up correctly

== Optimization ==

It might be nice to turn arch_init_thread_stacks() into a static call
to keep the overhead down.

Nit: I don't love the #ifdef in shstk.c, but it's over a tiny area.

---

 b/arch/x86/include/asm/thread_info.h |    7 +++++++
 b/arch/x86/kernel/shstk.c            |   21 +++++++++++++++++++++
 2 files changed, 28 insertions(+)

diff -puN arch/x86/include/asm/thread_info.h~x86-set-shadow-stack-permissions arch/x86/include/asm/thread_info.h
--- a/arch/x86/include/asm/thread_info.h~x86-set-shadow-stack-permissions	2026-10-21 12:36:39.397883874 -0700
+++ b/arch/x86/include/asm/thread_info.h	2026-10-21 12:36:39.409883897 -0700
@@ -226,6 +226,13 @@ static inline int arch_within_stack_fram
 
 extern void arch_setup_new_exec(void);
 #define arch_setup_new_exec arch_setup_new_exec
+
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+struct thread_stacks;
+extern void arch_init_thread_stacks(struct thread_stacks *ts);
+#define arch_init_thread_stacks arch_init_thread_stacks
+#endif
+
 #endif	/* !__ASSEMBLER__ */
 
 #endif /* _ASM_X86_THREAD_INFO_H */
diff -puN arch/x86/kernel/shstk.c~x86-set-shadow-stack-permissions arch/x86/kernel/shstk.c
--- a/arch/x86/kernel/shstk.c~x86-set-shadow-stack-permissions	2026-10-21 12:36:39.406883891 -0700
+++ b/arch/x86/kernel/shstk.c	2026-10-21 12:36:39.409883897 -0700
@@ -630,3 +630,24 @@ bool shstk_is_enabled(void)
 {
 	return features_enabled(ARCH_SHSTK_SHSTK);
 }
+
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+static inline int set_shstk_pte(pte_t *ptep, unsigned long addr, void *data)
+{
+	set_pte(ptep, pte_mkwrite_shstk(pte_wrprotect(ptep_get(ptep))));
+	return 0;
+}
+
+void arch_init_thread_stacks(struct thread_stacks *ts)
+{
+	unsigned long shstk_vaddr = (unsigned long)&ts->shadow_stack;
+	unsigned long shstk_size  = sizeof(ts->shadow_stack);
+
+	if (!cpu_feature_enabled(X86_FEATURE_KERN_SHSTK))
+		return;
+
+	apply_to_existing_page_range(&init_mm, shstk_vaddr, shstk_size,
+				     set_shstk_pte, NULL);
+	flush_tlb_kernel_range(shstk_vaddr, shstk_vaddr + shstk_size);
+}
+#endif
_

[RFC][PATCH 15/24] x86/task: Hook up arch hook to detect shadow stack enabling

[Dave Hansen]

Provide an x86 definition for arch_kernel_shstk_enabled(), used in the
generic code to skip any attempts to clear shadow stack pages.

---

 b/arch/x86/include/asm/thread_info.h |    6 ++++++
 1 file changed, 6 insertions(+)

diff -puN arch/x86/include/asm/thread_info.h~x86-shadow-stacks-enabled arch/x86/include/asm/thread_info.h
--- a/arch/x86/include/asm/thread_info.h~x86-shadow-stacks-enabled	2026-10-21 12:36:39.920884876 -0700
+++ b/arch/x86/include/asm/thread_info.h	2026-10-21 12:36:39.922884879 -0700
@@ -231,6 +231,12 @@ extern void arch_setup_new_exec(void);
 struct thread_stacks;
 extern void arch_init_thread_stacks(struct thread_stacks *ts);
 #define arch_init_thread_stacks arch_init_thread_stacks
+
+static inline bool arch_kernel_shstk_enabled(void)
+{
+	return cpu_feature_enabled(X86_FEATURE_KERN_SHSTK);
+}
+#define arch_kernel_shstk_enabled arch_kernel_shstk_enabled
 #endif
 
 #endif	/* !__ASSEMBLER__ */
_

[RFC][PATCH 16/24] x86/mm: Kill shadow stack page permission warning

[Dave Hansen]

Shadow stack permissions are now normal and valid for kernel mappings.
Kill the warning.

---

 b/arch/x86/include/asm/pgtable.h |    3 ---
 1 file changed, 3 deletions(-)

diff -puN arch/x86/include/asm/pgtable.h~axe-shstk-pte-warning arch/x86/include/asm/pgtable.h
--- a/arch/x86/include/asm/pgtable.h~axe-shstk-pte-warning	2026-10-21 12:36:40.424885841 -0700
+++ b/arch/x86/include/asm/pgtable.h	2026-10-21 12:36:40.427885846 -0700
@@ -730,9 +730,6 @@ static inline pgprotval_t check_pgprot(p
 static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot)
 {
 	phys_addr_t pfn = (phys_addr_t)page_nr << PAGE_SHIFT;
-	/* This bit combination is used to mark shadow stacks */
-	WARN_ON_ONCE((pgprot_val(pgprot) & (_PAGE_DIRTY | _PAGE_RW)) ==
-			_PAGE_DIRTY);
 	pfn ^= protnone_mask(pgprot_val(pgprot));
 	pfn &= PTE_PFN_MASK;
 	return __pte(pfn | check_pgprot(pgprot));
_

[RFC][PATCH 17/24] x86/task: Context switch the kernel shadow stack register value

[Dave Hansen]

This is the spot where things go off the rails. As I look at this, I
think it may be a mistake to permit a shadow-stack kernel to have non
shadow-stack tasks. It would just be too easy to leverage turn the
ability to set kshstk_ssp=0 into a control flow hijack.

This is also *not* the way the architecture intended context switching
to go. Feedback here would be much appreciated.

I'll continue on describing what is here.

The basic problem set:
 1. SSP needs to be context-switched just like RSP:
  1a. The kernel SSP value is discrete from the user SSP value
  1b. The kernel SSP value needs storage space
 2. At *least* the init task does not have a shadow stack mapping. The
    expectation is that there will be other tasks. For instance, a
    "permissive" mode after a shadow stack violation where the kernel
    warns and keeps on running.

== Solution ==

First, establish and maintain kernel SSP storage at:

	thread_struct->kshstk_ssp

A 0 value means that the task does not use kernel shadow stacks.

Second, drive MSR_IA32_S_CET from that value. If it is 0, then
clear CET_SHSTK_EN|CET_WRSS_EN during the context switch. If it
is non-zero set those bits.

Third, save and restore the kernel SSP value. This is substantially
more complicated than RSP switching because SSP is not directly
accessible and needs to be loaded with RSTORSSP, which is picky.
It's not picky for no reason. It is *intentionally* picky to foil
stack pivoting attacks. Before it will write SSP, it requires an
in-memory "restore token" be present on the shadow stack.

Just like the regular stack, this switch *must* be done in
assembly.

== Optimizations ==

    <<<
    These are probably all silly to talk about at this point if a
    over over to the more correct RSTORSSP/SAVEPREVSSP is made.
    >>>

First, TASK_thread_kshstk_ssp() will always be 0 when kernel shadow
stacks are disabled. This could be ALTERNATIVE()'d out to a plain
jump without the MOV and TEST.

Second, the ENABLE/DISABLE macros could avoid the WRMSR when it is
not changing values.

It might be possible to move the assembly logic like the
MSR_IA32_S_CET or restore token calculations out to C.

---

 b/arch/x86/entry/entry_64.S        |   87 +++++++++++++++++++++++++++++++++++++
 b/arch/x86/include/asm/processor.h |    4 +
 b/arch/x86/include/asm/shstk.h     |    3 +
 b/arch/x86/kernel/asm-offsets.c    |    3 +
 4 files changed, 97 insertions(+)

diff -puN arch/x86/entry/entry_64.S~per-task-ssp-storage arch/x86/entry/entry_64.S
--- a/arch/x86/entry/entry_64.S~per-task-ssp-storage	2026-10-21 12:36:40.914886779 -0700
+++ b/arch/x86/entry/entry_64.S	2026-10-21 12:36:40.948886844 -0700
@@ -34,6 +34,7 @@
 #include <asm/percpu.h>
 #include <asm/asm.h>
 #include <asm/smap.h>
+#include <asm/shstk.h>
 #include <asm/pgtable_types.h>
 #include <asm/frame.h>
 #include <asm/trapnr.h>
@@ -169,6 +170,87 @@ SYM_INNER_LABEL(entry_SYSRETQ_end, SYM_L
 	int3
 SYM_CODE_END(entry_SYSCALL_64)
 
+.macro SAVE_KERNEL_SHADOW_STACK
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	/* kshstk_ssp==0 means no kernel shadow stack, skip handling */
+	cmpq	$0, TASK_thread_kshstk_ssp(%rdi)
+	je	.Lprev_no_shstk\@
+	rdsspq	%rax
+	movq	%rax, TASK_thread_kshstk_ssp(%rdi)
+.Lprev_no_shstk\@:
+#endif
+.endm
+
+.macro ENABLE_KERNEL_SHADOW_STACK
+	movl	$MSR_IA32_S_CET, %ecx
+	rdmsr
+	orl	$(CET_SHSTK_EN | CET_WRSS_EN), %eax
+	wrmsr
+.endm
+
+.macro DISABLE_KERNEL_SHADOW_STACK
+	movl	$MSR_IA32_S_CET, %ecx
+	rdmsr
+	andl	$~(CET_SHSTK_EN | CET_WRSS_EN), %eax
+	wrmsr
+.endm
+
+/*
+ * Clobbers rax, rcx, rdx, r8, r9. rdi/rsi must survive for the rest
+ * of __switch_to_asm.
+ *
+ * Dumb and slow. Always does WRMSR, regardless of if changes need to
+ * be made.
+ */
+.macro LOAD_KERNEL_SHADOW_STACK
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	movq	TASK_thread_kshstk_ssp(%rsi), %r8
+	testq	%r8, %r8
+	jz	.Lnext_no_shstk\@
+
+	ENABLE_KERNEL_SHADOW_STACK
+
+	/*
+	 * This should be just a:
+	 *
+	 * 	rstorssp
+	 * 	saveprevssp
+	 *
+	 * sequence. But, currently, the previous task might not
+	 * have a valid SSP, like the init_task. This needs fixing.
+	 * But, for now, just hack a Prev-SSP token in.
+	 *
+	 * The right fix is to give all tasks a valid SSP universally
+	 * and then initialize the token once at task creation time,
+	 * probably near the spot that ret_from_fork_asm is also
+	 * written to the shadow stack.
+	 */
+	leaq	-8(%r8), %r9
+	movq	%r8, %rax
+	/*
+	 * Set type=0. Indicates a "Restore token" to satisfy RSTORSSP:
+	 * Set mode=1. Indicates 64-bit mode.
+	 * */
+	orq	$SHSTK_TOKEN_MODE64, %rax
+	wrssq	%rax, (%r9)
+
+	rstorssp (%r9)
+
+	/*
+	 * Discard the Previous-SSP token left by RSTORSSP.
+	 *
+	 * This will go away once SAVEPREVSSP is being used properly.
+	 */
+	movl	$1, %eax
+	incsspq	%rax
+	jmp	.Lshstk_done\@
+
+.Lnext_no_shstk\@:
+	DISABLE_KERNEL_SHADOW_STACK
+.Lshstk_done\@:
+#endif
+.endm
+
 /*
  * %rdi: prev task
  * %rsi: next task
@@ -191,6 +273,8 @@ SYM_FUNC_START(__switch_to_asm)
 	movq	%rsp, TASK_threadsp(%rdi)
 	movq	TASK_threadsp(%rsi), %rsp
 
+	SAVE_KERNEL_SHADOW_STACK
+
 #ifdef CONFIG_STACKPROTECTOR
 	movq	TASK_stack_canary(%rsi), %rbx
 	movq	%rbx, PER_CPU_VAR(__stack_chk_guard)
@@ -205,6 +289,9 @@ SYM_FUNC_START(__switch_to_asm)
 	 */
 	FILL_RETURN_BUFFER %r12, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_CTXSW
 
+	/* Note that this clobbers tons of stuff: */
+	LOAD_KERNEL_SHADOW_STACK
+
 	/* restore callee-saved registers */
 	popq	%r15
 	popq	%r14
diff -puN arch/x86/include/asm/processor.h~per-task-ssp-storage arch/x86/include/asm/processor.h
--- a/arch/x86/include/asm/processor.h~per-task-ssp-storage	2026-10-21 12:36:40.927886804 -0700
+++ b/arch/x86/include/asm/processor.h	2026-10-21 12:36:40.949886846 -0700
@@ -532,6 +532,10 @@ struct thread_struct {
 
 	struct thread_shstk	shstk;
 #endif
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	/* Kernel shadow stack register: */
+	void *kshstk_ssp;
+#endif
 };
 
 #ifdef CONFIG_X86_DEBUG_FPU
diff -puN arch/x86/include/asm/shstk.h~per-task-ssp-storage arch/x86/include/asm/shstk.h
--- a/arch/x86/include/asm/shstk.h~per-task-ssp-storage	2026-10-21 12:36:40.943886835 -0700
+++ b/arch/x86/include/asm/shstk.h	2026-10-21 12:36:40.949886846 -0700
@@ -2,6 +2,9 @@
 #ifndef _ASM_X86_SHSTK_H
 #define _ASM_X86_SHSTK_H
 
+#define SHSTK_TOKEN_MODE64	BIT_ULL(0) /* Token created in 64-bit mode */
+#define SHSTK_TOKEN_TYPE	BIT_ULL(1) /* Types: Restore: 0, Prev-SSP: 1 */
+
 #ifndef __ASSEMBLER__
 #include <linux/types.h>
 
diff -puN arch/x86/kernel/asm-offsets.c~per-task-ssp-storage arch/x86/kernel/asm-offsets.c
--- a/arch/x86/kernel/asm-offsets.c~per-task-ssp-storage	2026-10-21 12:36:40.945886838 -0700
+++ b/arch/x86/kernel/asm-offsets.c	2026-10-21 12:36:40.949886846 -0700
@@ -43,6 +43,9 @@ static void __used common(void)
 
 	BLANK();
 	OFFSET(TASK_threadsp, task_struct, thread.sp);
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	OFFSET(TASK_thread_kshstk_ssp, task_struct, thread.kshstk_ssp);
+#endif
 #ifdef CONFIG_STACKPROTECTOR
 	OFFSET(TASK_stack_canary, task_struct, stack_canary);
 #endif
_

[RFC][PATCH 18/24] kernel/task: Add a helper to get the thread_union

[Dave Hansen]

There's currently no helper that, given a task, can work back to the
"outer" thread_union. Add one to help locating the shadow stack
mapping location.

---

 b/include/linux/sched/task_stack.h |    5 +++++
 1 file changed, 5 insertions(+)

diff -puN include/linux/sched/task_stack.h~generic-task_thread_union include/linux/sched/task_stack.h
--- a/include/linux/sched/task_stack.h~generic-task_thread_union	2026-10-21 12:36:41.575888044 -0700
+++ b/include/linux/sched/task_stack.h	2026-10-21 12:36:41.577888048 -0700
@@ -23,6 +23,11 @@ static __always_inline void *task_stack_
 	return task->stack;
 }
 
+static inline union thread_union *task_thread_union(struct task_struct *tsk)
+{
+        return (union thread_union *)task_stack_page(tsk);
+}
+
 #define setup_thread_stack(new,old)	do { } while(0)
 
 static __always_inline unsigned long *end_of_stack(const struct task_struct *task)
_

[RFC][PATCH 19/24] x86/process: Initialize cloned task shadow stack

[Dave Hansen]

A child task can obviously not share its shadow stack with its parent.
It needs its own shadow stack.

First, ensure that the child's ->kshstk_ssp is pointed to its own
shadow stack. Second, write a pointer to the shadow stack itself that
corresponds to the RSP value consumed in ret_from_fork_asm.

Note that this implementation uses the direct map alias of the new
task's shadow stack. It would probably be best to use WRSS. But
shadow stacks may not be enabled on the current CPU before the
context switch making WRSS unavailable.

An implementation that universally enabled shadow stacks and made
the ISA available at the first context switch could use WRSS. This
is something else that will be simpler if the init task gets a working
shadow stack.

TODO: I don't like the #ifdef in the .c file, but there needs to
be an #ifdef somewhere for 'kshstk_ssp' since it is not defined when
shadow stacks are disabled.

---

 b/arch/x86/kernel/process.c |   35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff -puN arch/x86/kernel/process.c~enable-task-kssp arch/x86/kernel/process.c
--- a/arch/x86/kernel/process.c~enable-task-kssp	2026-10-21 12:36:42.071888995 -0700
+++ b/arch/x86/kernel/process.c	2026-10-21 12:36:42.074889001 -0700
@@ -167,6 +167,40 @@ __visible void ret_from_fork(struct task
 	syscall_exit_to_user_mode(regs);
 }
 
+/*
+ * The goal here is to build a shadow stack for the new thread. It
+ * just needs to survive the first RET from ret_from_fork_asm().
+ */
+static void prep_kernel_shadow_stack(struct task_struct *tsk)
+{
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	unsigned long *ss_top_slot;
+	u8 *shstk_base_writable;
+	u8 *shstk_base_vmap;
+	u8 *ss_top;
+
+	if (!cpu_feature_enabled(X86_FEATURE_KERN_SHSTK))
+		return;
+
+	/* Start with the shadow stack permission mapping: */
+	shstk_base_vmap = &task_thread_union(tsk)->stacks.shadow_stack[0];
+
+	/* Walk the page tables and find the direct map alias of the sshstk */
+	shstk_base_writable = page_address(vmalloc_to_page(shstk_base_vmap));
+
+	/* Point at the end of the shadow stack: */
+	ss_top = shstk_base_writable + TASK_SHADOW_STACK_SIZE;
+	/* Back up one ulong from the end to write the first return pointer: */
+	ss_top_slot = (unsigned long *)ss_top - 1;
+
+	/* Write the return address for the first RET: */
+	*ss_top_slot = (unsigned long)ret_from_fork_asm;
+
+	// Needs helper?
+	tsk->thread.kshstk_ssp = shstk_base_vmap + TASK_SHADOW_STACK_SIZE - sizeof(unsigned long);
+#endif
+}
+
 int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 {
 	u64 clone_flags = args->flags;
@@ -185,6 +219,7 @@ int copy_thread(struct task_struct *p, c
 	frame->bp = encode_frame_pointer(childregs);
 	frame->ret_addr = (unsigned long) ret_from_fork_asm;
 	p->thread.sp = (unsigned long) fork_frame;
+	prep_kernel_shadow_stack(p);
 	p->thread.io_bitmap = NULL;
 	clear_tsk_thread_flag(p, TIF_IO_BITMAP);
 	p->thread.iopl_warn = 0;
_

[RFC][PATCH 20/24] x86/entry: Stash kernel SSP on FRED entry

[Dave Hansen]

Kernel shadow stacks are intended to preserve control flow integrity
across all kinds of RIP transitions, not just strict CALLs. One of
those transitions is when there is a kernel interrupt or exception
and the kernel stops what it was doing and jumps over to
asm_fred_entrypoint_kernel.

On those jumps, FRED hardware writes a special frame to the shadow
stack. That frame includes the return address of the interrupted
kernel code. When returning from the event via ERETS, the hardware
compares this shadow stack frame with the return address on the
regular stack to ensure that they match.

The kernel munges event return addresses for lots of reasons. But,
with shadow stacks, it also needs to munge the corresponding
shadow stack frame contents.

Make it easy for RIP-munging code to locate the shadow stack frame.
Stash off the shadow stack pointer so it sits adjacent to pt_regs.
Without this, anything wishing to locate the special shadow stack
frame would need to walk up the stack. I'm not opposed to doing
it that way, this just seemed simpler for now.

Note, since RDSSPQ is a NOP with shadow stacks off, this assembly
is safe either way, although it does a unnecessary push/pop.

---

 b/arch/x86/entry/entry_64_fred.S |   22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff -puN arch/x86/entry/entry_64_fred.S~fred-entry-ssp-stash arch/x86/entry/entry_64_fred.S
--- a/arch/x86/entry/entry_64_fred.S~fred-entry-ssp-stash	2026-10-21 12:36:42.552889917 -0700
+++ b/arch/x86/entry/entry_64_fred.S	2026-10-21 12:36:42.555889923 -0700
@@ -16,6 +16,26 @@
 	.code64
 	.section .noinstr.text, "ax"
 
+/*
+ * Kernel-mode events: save the SSP of the shadow stack frame FRED
+ * pushed (old SSP, LIP, CS) just below pt_regs. See struct
+ * fred_kernel_frame.
+ */
+.macro PUSH_KERNEL_EVENT_SSP
+#ifdef CONFIG_X86_USER_SHADOW_STACK
+	xorl	%eax, %eax
+	rdsspq	%rax		/* nop if shadow stacks are off  */
+	pushq	%rax
+	UNWIND_HINT_REGS offset=8
+#endif
+.endm
+
+.macro POP_KERNEL_EVENT_SSP
+#ifdef CONFIG_X86_USER_SHADOW_STACK
+	addq	$8, %rsp
+#endif
+.endm
+
 .macro FRED_ENTER
 	UNWIND_HINT_END_OF_STACK
 	ANNOTATE_NOENDBR
@@ -54,7 +74,9 @@ SYM_CODE_END(asm_fred_entrypoint_user)
 	.org asm_fred_entrypoint_user + 256, 0xcc
 SYM_CODE_START_NOALIGN(asm_fred_entrypoint_kernel)
 	FRED_ENTER
+	PUSH_KERNEL_EVENT_SSP
 	call	fred_entry_from_kernel
+	POP_KERNEL_EVENT_SSP
 	FRED_EXIT
 	ERETS
 SYM_CODE_END(asm_fred_entrypoint_kernel)
_

[RFC][PATCH 21/24] x86/extable: Update shadow stack entry frame during RIP overwrites

[Dave Hansen]

Previous work stashed the entry SSP value on the regular (RSP) stack.
Use that stashed value to locate the FRED entry frame on the shadow
stack. Update the "LIP" value in that frame to match the updated
regs->rip value.

The now-matching values will keep FRED happy during ERETS.

Also include a stub function for non-CET builds.

This "rewrite rip" infrastructure will get used in quite a few other
locations. The extable site is the most critical. Do it first. A
normal kernel won't boot without it because it allows kernel
exceptions to be "stepped over" for "safe" variants like wrmsr_safe().

---

 b/arch/x86/include/asm/insn-eval.h     |   12 +++++++
 b/arch/x86/include/asm/special_insns.h |    7 ++++
 b/arch/x86/kernel/cet.c                |   54 +++++++++++++++++++++++++++++++++
 b/arch/x86/mm/extable.c                |    2 -
 4 files changed, 74 insertions(+), 1 deletion(-)

diff -puN arch/x86/include/asm/insn-eval.h~fred-entry-ssp-muck arch/x86/include/asm/insn-eval.h
--- a/arch/x86/include/asm/insn-eval.h~fred-entry-ssp-muck	2026-10-21 12:36:43.032890836 -0700
+++ b/arch/x86/include/asm/insn-eval.h	2026-10-21 12:36:43.041890854 -0700
@@ -82,4 +82,16 @@ static inline void insn_assign_reg(unsig
 	}
 }
 
+#ifdef CONFIG_X86_CET
+void regs_set_return_ip(struct pt_regs *regs, unsigned long ip);
+#else
+static inline void regs_set_return_ip(struct pt_regs *regs, unsigned long ip)
+{
+	/* This is only for changing kernel return addresses: */
+	WARN_ON_ONCE(user_mode(regs));
+
+	regs->ip = ip;
+}
+#endif
+
 #endif /* _ASM_X86_INSN_EVAL_H */
diff -puN arch/x86/include/asm/special_insns.h~fred-entry-ssp-muck arch/x86/include/asm/special_insns.h
--- a/arch/x86/include/asm/special_insns.h~fred-entry-ssp-muck	2026-10-21 12:36:43.034890840 -0700
+++ b/arch/x86/include/asm/special_insns.h	2026-10-21 12:36:43.042890856 -0700
@@ -303,6 +303,13 @@ static __always_inline void tile_release
 	asm volatile(".byte 0xc4, 0xe2, 0x78, 0x49, 0xc0");
 }
 
+static __always_inline void wrss(u64 val, u64 *addr)
+{
+	asm volatile("wrssq %[val], %[addr]"
+		     : [addr] "=m" (*addr)
+		     : [val] "r" (val));
+}
+
 #endif /* __KERNEL__ */
 
 #endif /* _ASM_X86_SPECIAL_INSNS_H */
diff -puN arch/x86/kernel/cet.c~fred-entry-ssp-muck arch/x86/kernel/cet.c
--- a/arch/x86/kernel/cet.c~fred-entry-ssp-muck	2026-10-21 12:36:43.036890844 -0700
+++ b/arch/x86/kernel/cet.c	2026-10-21 12:36:43.042890856 -0700
@@ -4,6 +4,8 @@
 #include <asm/bugs.h>
 #include <asm/msr.h>
 #include <asm/traps.h>
+#include <asm/insn-eval.h>
+#include <asm/special_insns.h>
 
 enum cp_error_code {
 	CP_EC        = (1 << 15) - 1,
@@ -160,3 +162,55 @@ DEFINE_IDTENTRY_ERRORCODE(exc_control_pr
 			do_unexpected_cp(regs, error_code);
 	}
 }
+
+/*
+ * FRED hardware pushes this frame on to the shadow stack for
+ * kernel=>kernel entry. First (lowest address) in this struct
+ * is the last entry pushed by the hardware.
+ */
+struct fred_shstk_entry_frame {
+	u64 ssef_ssp;
+	u64 ssef_lip;
+	u64 ssef_cs;
+};
+
+/*
+ * The kernel=>kernel entry code pushes SSP on to the entry stack.
+ * This only occurs for kernel=>kernel entry. Not userspace.
+ *
+ * SSP is pushed on to the frame below pt_regs. At the point it is
+ * pushed, it points to a special FRED entry frame.
+ */
+struct fred_kernel_frame {
+	struct fred_shstk_entry_frame *ssef;
+	struct pt_regs regs;
+};
+
+/*
+ * Change the return IP of an event. Use this instead of directly
+ * manipulating regs->ip.
+ *
+ * The hardware saves a special FRED entry frame on to the shadow
+ * stack. It contains the RIP of the interrupted context. ERETS
+ * ensures that this matches regs->ip.
+ *
+ * Update the FRED entry frame in addition to regs->ip.
+ */
+void regs_set_return_ip(struct pt_regs *regs, unsigned long ip)
+{
+	struct fred_shstk_entry_frame *ssef;
+
+	regs->ip = ip;
+
+	if (!cpu_feature_enabled(X86_FEATURE_KERN_SHSTK))
+		return;
+
+	/* Only kernel-mode events save the 'ssef' pointer. */
+	WARN_ON_ONCE(user_mode(regs));
+
+	ssef = container_of(regs, struct fred_kernel_frame, regs)->ssef;
+	if (!ssef)
+		return;
+
+	wrss(ip, &ssef->ssef_lip);
+}
diff -puN arch/x86/mm/extable.c~fred-entry-ssp-muck arch/x86/mm/extable.c
--- a/arch/x86/mm/extable.c~fred-entry-ssp-muck	2026-10-21 12:36:43.039890850 -0700
+++ b/arch/x86/mm/extable.c	2026-10-21 12:36:43.042890856 -0700
@@ -38,7 +38,7 @@ static bool ex_handler_default(const str
 	if (e->data & EX_FLAG_CLEAR_DX)
 		regs->dx = 0;
 
-	regs->ip = ex_fixup_addr(e);
+	regs_set_return_ip(regs, ex_fixup_addr(e));
 	return true;
 }
 
_

[RFC][PATCH 22/24] kernel/task: Add a helper to locate the top of the shadow stack

[Dave Hansen]

Just like the normal RSP stack, the shadow stack starts at the top of the
stack and grows down. Add a helper to calculate the top (exclusive)
address of the shadow stack.

Note that this is generic code without #ifdefs. The shadow_stack member
is present and 0 bytes long on configs without shadow stacks.

---

 b/include/linux/sched/task_stack.h |   23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff -puN include/linux/sched/task_stack.h~generic-shstk-top-helper include/linux/sched/task_stack.h
--- a/include/linux/sched/task_stack.h~generic-shstk-top-helper	2026-10-21 12:36:43.603891930 -0700
+++ b/include/linux/sched/task_stack.h	2026-10-21 12:36:43.606891936 -0700
@@ -23,11 +23,6 @@ static __always_inline void *task_stack_
 	return task->stack;
 }
 
-static inline union thread_union *task_thread_union(struct task_struct *tsk)
-{
-        return (union thread_union *)task_stack_page(tsk);
-}
-
 #define setup_thread_stack(new,old)	do { } while(0)
 
 static __always_inline unsigned long *end_of_stack(const struct task_struct *task)
@@ -119,4 +114,22 @@ static inline int kstack_end(void *addr)
 	return !(((unsigned long)addr+sizeof(void*)-1) & (THREAD_SIZE-sizeof(void*)));
 }
 
+static inline union thread_union *task_thread_union(struct task_struct *tsk)
+{
+        return (union thread_union *)task_stack_page(tsk);
+}
+
+/*
+ * Return the address of the top of the shadow stack for a task.
+ *
+ * Note that this is the address just above the last shadow stack
+ * slot. It is an invalid address and needs to be decremented by
+ * something before actual use.
+ */
+static inline void *task_shstk_top(struct task_struct *tsk)
+{
+        return task_thread_union(tsk)->stacks.shadow_stack +
+                sizeof(task_thread_union(tsk)->stacks.shadow_stack);
+}
+
 #endif /* _LINUX_SCHED_TASK_STACK_H */
_

[RFC][PATCH 23/24] x86/entry: Populate FRED register for user=>kernel transitions

[Dave Hansen]

== Background ==

All the shadow stack work up until now has focused on running inside
the kernel. The work has manipulated the actual shadow stack pointer
(SSP) and prepared FRED hardware to update SSP across kernel=>kernel
event boundaries (the SSP values stashed in FRED_SSP[123]). Note that
the shadow stacks for event handling are fixed for a given CPU and
never change.

Once in place, this lets the kernel move SSP around for normal
CALL/RET operations and also handle FRED-delivered events like
interrupts and faults.

But, this all only covers kernel=>kernel event boundaries like when
the kernel takes an NMI or a page fault.

== Problem ==

There is currently no handling for user=>kernel transitions. Those
events must use the active kernel task's SSP which is *not* constant
on each CPU.

== Solution ==

Point FRED_SSP0 to the active kernel task's SSP before returning to
userspace. This ensures that the FRED hardware will set SSP to the
correct per-task value when doing the user=>kernel transition.

Mirror FRED RSP0 handling for SSP0. Avoid setting FRED_SSP0 during
context switching itself. It is not necessary to have a "good"
FRED_SSP0 value when running the kernel, only userspace.

---

 b/arch/x86/include/asm/entry-common.h |    2 +-
 b/arch/x86/include/asm/fred.h         |    8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff -puN arch/x86/include/asm/entry-common.h~fred_update_ssp0 arch/x86/include/asm/entry-common.h
--- a/arch/x86/include/asm/entry-common.h~fred_update_ssp0	2026-10-21 12:36:44.081892846 -0700
+++ b/arch/x86/include/asm/entry-common.h	2026-10-21 12:36:44.086892856 -0700
@@ -65,7 +65,7 @@ static inline void arch_exit_to_user_mod
 		arch_exit_work(ti_work);
 
 	fred_update_rsp0();
-
+	fred_update_ssp0();
 #ifdef CONFIG_COMPAT
 	/*
 	 * Compat syscalls set TS_COMPAT.  Make sure we clear it before
diff -puN arch/x86/include/asm/fred.h~fred_update_ssp0 arch/x86/include/asm/fred.h
--- a/arch/x86/include/asm/fred.h~fred_update_ssp0	2026-10-21 12:36:44.083892850 -0700
+++ b/arch/x86/include/asm/fred.h	2026-10-21 12:36:44.086892856 -0700
@@ -105,6 +105,13 @@ static __always_inline void fred_update_
 		__this_cpu_write(fred_rsp0, rsp0);
 	}
 }
+static __always_inline void fred_update_ssp0(void)
+{
+#ifdef CONFIG_X86_KERN_SHADOW_STACK
+	if (current->thread.kshstk_ssp)
+		wrmsrns(MSR_IA32_FRED_SSP0, (u64)task_shstk_top(current));
+#endif
+}
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
@@ -112,6 +119,7 @@ static inline void cpu_init_fred_rsps(vo
 static inline void fred_complete_exception_setup(void) { }
 static inline void fred_sync_rsp0(unsigned long rsp0) { }
 static inline void fred_update_rsp0(void) { }
+static inline void fred_update_ssp0(void) { }
 #endif /* CONFIG_X86_FRED */
 #endif /* !__ASSEMBLER__ */
 
_

[RFC][PATCH 24/24] [DO NOT MERGE] x86/shstk: Depend out everything incompatible with kernel shadow stacks

[Dave Hansen]

Note: this is a 100% vibe-coded pile of goo. It will at least present
the rough scale of the work that's left to do though.

===

Kernel shadow stacks make the CPU check every RET against a second,
hardware-maintained stack of return addresses. Anything that "returns"
somewhere other than where the matching CALL came from takes a #CP.
Quite a lot of kernel code does exactly that, on purpose.

Exclude all of it at Kconfig time. This is deliberately the biggest
hammer available. Each dependency is annotated with the specific
construct that breaks so it can be revisited when that code is taught
about shadow stacks:

 * Retpolines, call depth tracking and SRSO Safe-RET all execute a RET
   whose return address was not pushed by the matching CALL.
 * Function graph tracing, rethook/fprobe, livepatch and error injection
   all hijack return addresses or redirect the IP that ftrace_regs_caller
   RETs to.
 * kprobes emulates CALL and RET by moving the return address on the
   regular stack only.
 * BPF trampolines return to the grandparent with SKIP_FRAME and
   bpf_throw() unwinds frames without executing their RETs.
 * kexec, hibernation and S3 resume all RET or jump into saved context
   without S_CET, the SSP MSRs or SSP itself being saved or restored.
 * EFI mixed mode uses LRETQ/LRET, which are shadow stack checked.
 * KVM refuses to load when host S_CET.SH_STK_EN is set and VMX loads
   HOST_SSP=0 on every VM-exit.
 * Xen PV runs the kernel in ring 3 without FRED.
 * kgdb writes regs->ip behind FRED's back.

Also document, in the same place, the features that were examined and
found compatible (rethunk, retbleed untrain, ITS thunks, clear_bhb_loop,
PTI by way of it clearing FRED, IBT, user shadow stacks), and the known
gaps that Kconfig can not express: text_poke_bp() CALL emulation,
direct regs->ip writes, the RSB_CTXSW fill in __switch_to_asm, and EFI
runtime services.

Runtime-selected mitigations (spectre_v2=retpoline on an eIBRS CPU,
retbleed=stuff, RSB filling) can not be caught here and are handled by
a separate runtime check in the CET init code.

---

 b/arch/x86/Kconfig |  107 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)

diff -puN arch/x86/Kconfig~kshstk-kconfig-depend-out-incompatible arch/x86/Kconfig
--- a/arch/x86/Kconfig~kshstk-kconfig-depend-out-incompatible	2026-10-21 12:36:44.598893837 -0700
+++ b/arch/x86/Kconfig	2026-10-21 12:36:44.601893842 -0700
@@ -1865,6 +1865,40 @@ config X86_USER_SHADOW_STACK
 
 	  If unsure, say N.
 
+# Kernel shadow stacks make the CPU check every RET against a second,
+# hardware-maintained stack of return addresses. Anything that
+# "returns" somewhere other than where the matching CALL came from
+# takes a #CP. That rules out a surprising amount of kernel code. The
+# dependencies below exclude all of it at build time. Each one names
+# the construct that breaks so it can be revisited when that code is
+# taught about shadow stacks.
+#
+# Examined and believed compatible, so NOT excluded here:
+#  - MITIGATION_RETHUNK: __x86_return_thunk is a plain RET.
+#  - MITIGATION_UNRET_ENTRY: retbleed_return_thunk is a plain RET and
+#    entry_untrain_ret is a balanced CALL/RET.
+#  - MITIGATION_ITS: the ITS indirect thunks are JMPs and
+#    its_return_thunk is a plain RET. (its=stuff is caught at runtime
+#    via X86_FEATURE_CALL_DEPTH.)
+#  - MITIGATION_SPECTRE_BHI: clear_bhb_loop() is balanced CALL/RET.
+#  - MITIGATION_PAGE_TABLE_ISOLATION: PTI clears X86_FEATURE_FRED at
+#    runtime and kernel shadow stacks require FRED, so the two can
+#    never be active together.
+#  - X86_KERNEL_IBT, X86_USER_SHADOW_STACK: separate CET controls.
+#
+# Known gaps that Kconfig can not express and that the design must
+# still handle:
+#  - text_poke_bp() CALL emulation (int3_emulate_call(), used by
+#    static_call and ftrace call-site patching and by the boot-time
+#    int3_selftest()) pushes the return address only on the regular
+#    stack.
+#  - Any code that writes regs->ip directly instead of going through
+#    regs_set_return_ip() leaves the FRED shadow stack frame stale.
+#  - The RSB_CTXSW RSB fill in __switch_to_asm pushes 32 bogus shadow
+#    stack entries. It is survivable only because SSP is reloaded
+#    with RSTORSSP immediately afterwards.
+#  - EFI runtime services run firmware code that is not shadow stack
+#    aware while kernel shadow stacks are live.
 config X86_KERN_SHADOW_STACK
 	bool "X86 kernel shadow stack"
 	depends on X86_FRED # implies X86_64
@@ -1873,6 +1907,79 @@ config X86_KERN_SHADOW_STACK
 	depends on VMAP_STACK
 	depends on AS_WRUSS
 	select X86_CET
+	# The retpoline sequence is "call; mov %reg,(%rsp); ret": the
+	# CALL pushes a shadow stack entry and the RET then uses a
+	# rewritten return address. Mismatch every time.
+	depends on !MITIGATION_RETPOLINE
+	# The indirect call thunks are retpoline-based (see above) and
+	# call_depth_return_thunk() refills the RSB with 16 CALLs, then
+	# does "add $128,%rsp; ret", leaving 16 stale shadow stack entries.
+	depends on !MITIGATION_CALL_DEPTH_TRACKING
+	# Safe-RET is "call srso_safe_ret" followed by
+	# "lea 8(%rsp),%rsp; ret": the RET consumes the caller's return
+	# address while the shadow stack top holds the CALL's.
+	depends on !MITIGATION_SRSO
+	# Replaces return addresses on the stack with return_to_handler,
+	# and return_to_handler itself returns with a retpoline-style
+	# "call; mov %rdi,(%rsp); ret".
+	depends on !FUNCTION_GRAPH_TRACER
+	# kprobe_emulate_call(), kprobe_emulate_ret() and
+	# kprobe_emulate_call_indirect() emulate CALL and RET by moving
+	# the return address on the regular stack only. KRETPROBES and
+	# OPTPROBES hang off this symbol as well.
+	depends on !KPROBES
+	# arch_rethook_trampoline() is a return address hijack: the
+	# probed function RETs to the trampoline instead of its caller.
+	# FPROBE selects RETHOOK.
+	depends on !RETHOOK
+	depends on !FPROBE
+	# klp_ftrace_handler() redirects execution by changing the
+	# ftrace_regs IP. ftrace_regs_caller writes that IP into the
+	# return slot and RETs to it, but the shadow stack still holds
+	# the original address.
+	depends on !LIVEPATCH
+	# override_function_with_return() points regs->ip at
+	# just_return_func, a bare RET that pops the probed function's
+	# return address without the function's own RET ever running.
+	depends on !FUNCTION_ERROR_INJECTION
+	# BPF trampolines with BPF_TRAMP_F_SKIP_FRAME do "add $8,%rsp; ret"
+	# to return to the grandparent. bpf_throw() unwinds BPF frames by
+	# walking the stack and resuming at the exception callback with
+	# the unwound RSP, skipping the intervening RETs.
+	depends on !BPF_JIT
+	# relocate_kernel() switches CR3 and CR4 with a shadow stack live
+	# and uses "pushq $0 ... ret" as the handoff into purgatory. It
+	# needs S_CET.SH_STK_EN cleared first, which nothing does yet.
+	depends on !KEXEC_CORE
+	# restore_registers() RETs on a stack reloaded from the image
+	# with S_CET, the PLx/FRED_SSPx MSRs and SSP itself never saved or
+	# restored. Same problem for do_suspend_lowlevel()/wakeup_long64(),
+	# which resume with "jmp *%rax" into the middle of the function
+	# and then RET.
+	depends on !HIBERNATION
+	depends on !SUSPEND
+	# __efi64_thunk() drops into 32-bit firmware with LRETQ and comes
+	# back with LRET. Far returns are shadow stack checked and expect
+	# a far CALL frame that was never pushed.
+	depends on !EFI_MIXED
+	# KVM snapshots host S_CET at load, refuses to load if SH_STK_EN
+	# is set (kvm_x86_vendor_init()), and VMX programs HOST_SSP=0 for
+	# every VM-exit. vmenter.S also runs FILL_RETURN_BUFFER before
+	# its RET when RSB_VMEXIT or RSB_VMEXIT_LITE is active.
+	depends on !KVM
+	# PV guests run the kernel in ring 3 without FRED. Kernel shadow
+	# stacks are FRED-only.
+	depends on !XEN_PV
+	# kgdb_arch_set_pc() writes regs->ip directly rather than through
+	# regs_set_return_ip(), so the FRED shadow stack frame goes stale.
+	# The breakpoint and single-step paths have not been audited.
+	depends on !KGDB
+	help
+	  Shadow stacks for the kernel itself. Every CALL pushes the
+	  return address on a second, hardware-protected stack and every
+	  RET checks it. Requires FRED. See the comments above for the
+	  long list of kernel features this is currently incompatible
+	  with.
 
 config INTEL_TDX_HOST
 	bool "Intel Trust Domain Extensions (TDX) host support"
_

Re: [RFC][PATCH 00/24] x86 Kernel Shadow Stacks

[Peter Zijlstra]

On Wed, Oct 21, 2026 at 12:51:20PM -0700, Dave Hansen wrote:
>  * Work with IDT-based systems (without FRED)

This simply cannot work -- IDT relies on that horrific NMI nesting
stuff, and shadow stacks hate on that something fierce.

Re: [RFC][PATCH 12/24] x86/cea: Map and use FRED exception shadow stacks

[Peter Zijlstra]

On Wed, Oct 21, 2026 at 12:51:36PM -0700, Dave Hansen wrote:
> 
> FRED hardware automatically switches stacks between levels on different
> kinds of kernel entries. Each normal FRED "RSP" stack has a corresponding
> "SSP" shadow stack.
> 
> Leverage the existing CPU Entry Area infrastructure. Add shadow stacks
> for levels 1-3 right next to the existing RSP stacks. Map them with
> shadow stack permissions. Last, point the FRED SSP MSRs at the exception
> stacks.
> 
> Just like the task stacks, do not leave a guard page since regular and
> shadow stacks have mutually exclusive permissions and serve as guards
> for each other.
> 
> Nit: The "return style" of the new cea_map_stack() and wrmsr() code
> looks superficially inconsistent. But each site is consistent with the
> code around it, just not when looking at the two sites against each
> other.
> 
> == Future Work ==
> 
> The *_stack_guard[] elements could probably be removed now. When in
> use, shadow stacks serve as guard pages for the regular stacks. When
> not in use, the shadow stack region could remain unmapped and also act
> as a guard area. But that would take some more fancy dynamic mapping
> work. The existing CEA code is dirt simple and doing anything dynamic
> would add complexity. KISS. The downside is wasting three pages per
> CPU when shadow stacks support is compiled in but not used at runtime.

People were trying to *shrink* stacks ;-)