Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Dave Hansen]

So, I'm working on kernel shadow stacks. I have some nice optimization
code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to
userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.

Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very
important way: it is an alias for PL0_SSP, which is managed via
XFEATURE_CET_KERNEL which is in:

	/* Supervisor features which are enabled only in guest FPUs */
	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL

So, I was running along just fine in my nice shadow stack kernel going
in and out of the kernel just fine. But the first time I did an XRSTOR,
the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache
didn't notice and I tried to run userspace with SSP0=0. Much hilarity
ensued on the next kernel entry.

The most adorable part of all this is that I have KVM compiled
completely out on my config. The "only in guest FPUs" code is biting me
on a kernel that can't even run guests.

I think this all started because of 8b05b3c98816 ("x86/fpu/xstate: Add
CET supervisor xfeature support as a guest-only feature"), which seems
like mostly my doing. So I have mostly myself to blame.

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
needed at context switch time, only when running userspace.

I could just aggressively blow away my SSP0 cache any time an XRSTOR
touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly
that XFEATURE_CET_KERNEL is being blown away at _all_ during normal
context switches. Heck, it goes further than that because:

fpregs_restore_userregs() does:

	restore_fpregs_from_fpstate(fpu->fpstate,
				    XFEATURE_MASK_FPSTATE);

Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

So a "restore_userregs" function is blowing away supervisor state.

IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.

I'd like to get to a place where only the KVM code even has
XFEATURE_CET_KERNEL in its RFBM.

Anybody have any bright ideas?

Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Paolo Bonzini]

On 10/23/26 21:03, Dave Hansen wrote:
> So, I'm working on kernel shadow stacks. I have some nice optimization
> code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to
> userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.
> 
> Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very
> important way: it is an alias for PL0_SSP, which is managed via
> XFEATURE_CET_KERNEL which is in:
> 
> 	/* Supervisor features which are enabled only in guest FPUs */
> 	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL
> 
> So, I was running along just fine in my nice shadow stack kernel going
> in and out of the kernel just fine. But the first time I did an XRSTOR,
> the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache
> didn't notice and I tried to run userspace with SSP0=0. Much hilarity
> ensued on the next kernel entry.
> 
> The most adorable part of all this is that I have KVM compiled
> completely out on my config. The "only in guest FPUs" code is biting me
> on a kernel that can't even run guests.

If you want to quickly work around for your !CONFIG_KVM kernel, I would do

#ifdef CONFIG_KVM
/* Supervisor features which are enabled only in guest FPUs */
#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL
#else
#define XFEATURE_MASK_GUEST_SUPERVISOR  0
#endif

and a small adjustment like

         BUILD_BUG_ON((XFEATURE_MASK_USER_SUPPORTED |
                       XFEATURE_MASK_SUPERVISOR_SUPPORTED) &
                      ~XFEATURES_INIT_FPSTATE_HANDLED);

That will get rid of the bit in XSS.

> Any thoughts on what we should do?
> 
> Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
> lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
> needed at context switch time, only when running userspace.

True but, just to understand, what is the performance like if you 
actually use XSAVES/XRSTORS to update SSP0 on context switches instead 
of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations 
as well.

> I could just aggressively blow away my SSP0 cache any time an XRSTOR
> touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly
> that XFEATURE_CET_KERNEL is being blown away at _all_ during normal
> context switches. Heck, it goes further than that because:
> 
> fpregs_restore_userregs() does:
> 
> 	restore_fpregs_from_fpstate(fpu->fpstate,
> 				    XFEATURE_MASK_FPSTATE);
> 
> Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

The actual mask used by restore_fpregs_from_fpstate is tighter though, 
it's "fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE".

Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz 
(sorry too late to think it through :)) that has CET_KERNEL for the 
guest FPU and not for the regular one?

> So a "restore_userregs" function is blowing away supervisor state.
> 
> IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.
> 
> I'd like to get to a place where only the KVM code even has
> XFEATURE_CET_KERNEL in its RFBM.
That I agree, but I think it can be handled in 
{save,restore}_fpregs_to_fpstate().

Paolo

Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Dave Hansen]

On 10/23/26 14:00, Paolo Bonzini wrote:
> On 10/23/26 21:03, Dave Hansen wrote:
...
> If you want to quickly work around for your !CONFIG_KVM kernel, I would do
> 
> #ifdef CONFIG_KVM
> /* Supervisor features which are enabled only in guest FPUs */
> #define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL
> #else
> #define XFEATURE_MASK_GUEST_SUPERVISOR  0
> #endif

Yup, that'd work for sure. Thanks!

>> Any thoughts on what we should do?
>>
>> Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
>> lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
>> needed at context switch time, only when running userspace.
> 
> True but, just to understand, what is the performance like if you
> actually use XSAVES/XRSTORS to update SSP0 on context switches instead
> of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations
> as well.

First, we can use WRMSRNS for this MSR, so it's faster than usual.
Second, on kernel=>kernel context switches that don't go out to
userspace, it doesn't need to be written at all. It's kinda like the
user FPU state. Third, I'm actually reusing the fred_ssp0 cache, so
further reduces the incremental cost. Fourth, it's not _really_
context-switched. There's never a need to read the register. It can
always be conjured out of thin air. It's possible to only write the
register, never read it.

I can benchmark it, but I seriously doubt that an XSAVE/XRSTOR at
context switch of three fields is going to beat an WRMSRNS of one.

>> I could just aggressively blow away my SSP0 cache any time an XRSTOR
>> touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly
>> that XFEATURE_CET_KERNEL is being blown away at _all_ during normal
>> context switches. Heck, it goes further than that because:
>>
>> fpregs_restore_userregs() does:
>>
>>     restore_fpregs_from_fpstate(fpu->fpstate,
>>                     XFEATURE_MASK_FPSTATE);
>>
>> Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.
> 
> The actual mask used by restore_fpregs_from_fpstate is tighter though,
> it's "fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE".
> 
> Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz
> (sorry too late to think it through :)) that has CET_KERNEL for the
> guest FPU and not for the regular one?

Sure I can give that a shot.

There's one minor issue in that we need to init dynamic features which
_aren't_ in fpstate->xfeatures when context switching. There's a little
note about this in restore_fpregs_from_fpstate().

I think what you're getting at is that we need _something_ to say that
KVM fpstates should muck with XFEATURE_CET_KERNEL and task fpstates
should not, even though the task ones have space for XFEATURE_CET_KERNEL.

That's certainly doable.

>> So a "restore_userregs" function is blowing away supervisor state.
>>
>> IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.
>>
>> I'd like to get to a place where only the KVM code even has
>> XFEATURE_CET_KERNEL in its RFBM.
> That I agree, but I think it can be handled in {save,restore}
> _fpregs_to_fpstate().

I think I understand what you're going after at least. I'll poke at it a
bit more.

Thanks for the thoughts!

Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Andrew Cooper]

On 23/10/2026 10:21 pm, Dave Hansen wrote:
> On 10/23/26 14:00, Paolo Bonzini wrote:
>> On 10/23/26 21:03, Dave Hansen wrote:
>>> Any thoughts on what we should do?
>>>
>>> Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
>>> lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
>>> needed at context switch time, only when running userspace.
>> True but, just to understand, what is the performance like if you
>> actually use XSAVES/XRSTORS to update SSP0 on context switches instead
>> of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations
>> as well.
> First, we can use WRMSRNS for this MSR, so it's faster than usual.
> Second, on kernel=>kernel context switches that don't go out to
> userspace, it doesn't need to be written at all. It's kinda like the
> user FPU state. Third, I'm actually reusing the fred_ssp0 cache, so
> further reduces the incremental cost.

XSTATE_CET_S is a weird thing.  It contains MSRs for PL{0..2}_SSP which
are the shstk equivalents of tss->rsp{0..2}.

The Ring1/2 aspects are practically useless; CET-SS was the thing which
finally caused me to put Ring1 PV kernels to rest.  All of
tss->rsp{0..2} are switched together, so I suppose PL{0..2}_SSP were
grouped for the same reason, not that 64bit OSes use task switching.

Windows uses %xmm in the main ABI, so already need to XSAVE/XRSTOR when
switching between kernel threads.  So it's "simple" to add one extra bit
to RFBM while doing this.


VMMs probably want to use XSTATE_CET_S simply to have the guest state
held together.  Even then, it does feel like a waste; PL1/2_SSP are
going to be 0 ~all of the time, while PL0_SSP is nonzero ~all of the
time in a CET-enabled kernel, so you can't even rely on the INIT
optimisation to speed things up.

Linux obviously doesn't want to do an X{SAVE,RSTOR}S between kernel
threads, and isn't using Rings 1/2, so the WRMSRNS approach seems fine. 
And I agree that it only needs to be good at the point of returning to
userspace.


> Fourth, it's not _really_
> context-switched. There's never a need to read the register. It can
> always be conjured out of thin air.

The requirements are different, depending on FRED or not.

Without FRED, PL0_SSP must point to a Supervisor Shadow Stack token. 
IRET marks the token non-busy (squashing any errors), and IDT delivery
checks it can atomically set the token busy, yielding #CP if not.

With FRED, ERETU checks that SSP == SL0_SSP and yields #CP on a
mismatch.  There are no checks on entry.

~Andrew

Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Chao Gao]

>> Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz
>> (sorry too late to think it through :)) that has CET_KERNEL for the
>> guest FPU and not for the regular one?
>
>Sure I can give that a shot.
>
>There's one minor issue in that we need to init dynamic features which
>_aren't_ in fpstate->xfeatures when context switching. There's a little
>note about this in restore_fpregs_from_fpstate().
>
>I think what you're getting at is that we need _something_ to say that
>KVM fpstates should muck with XFEATURE_CET_KERNEL and task fpstates
>should not, even though the task ones have space for XFEATURE_CET_KERNEL.
>
>That's certainly doable.

Sounds like you are talking about something like this, which Sean
suggested in [1]:

diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c
index 1015af1ae562..8e7599e3b923 100644
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -167,6 +167,16 @@ void restore_fpregs_from_fpstate(struct fpstate *fpstate, u64 mask)
                 */
                xfd_update_state(fpstate);
 
+               /*
+                * Leave supervisor CET state as-is when loading host state
+                * (kernel or userspace).  Supervisor CET state is managed via
+                * XSTATE for KVM guests, but the host never consumes said
+                * state (doesn't support supervisor shadow stacks), i.e. it's
+                * safe to keep guest state loaded into hardware.
+                */
+               if (!fpstate->is_guest)
+                       mask &= ~XFEATURE_MASK_CET_KERNEL;
+

We first looked at enabling CET_KERNEL for all tasks, and that version
included this optimization so that restoring a non-KVM fpstate doesn't
touch CET_KERNEL.

We then switched to the guest-only approach to avoid allocating CET_KERNEL
space in non-KVM fpstates. The optimization could still be applied on top
of that, but it was dropped after its benefit was questioned off-list:

"""
1. It hacks in an optimization with conditionals around the edges,
instead of integrating with the way the rest of the FPU works.
2. There is no proven benefit of this narrow optimization and x86
maintainers are already skeptical.
"""

This optimization can be added back if it benefits kernel shadow stacks.
But os_xrstor_supervisor() and os_xrstor_safe() would need the same
handling.

[1]: https://lore.kernel.org/all/ZM1jV3UPL0AMpVDI@google.com/

Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Dave Hansen]

On 10/27/26 07:18, Chao Gao wrote:
> We then switched to the guest-only approach to avoid allocating CET_KERNEL
> space in non-KVM fpstates. The optimization could still be applied on top
> of that, but it was dropped after its benefit was questioned off-list:
> 
> """
> 1. It hacks in an optimization with conditionals around the edges,
> instead of integrating with the way the rest of the FPU works.
> 2. There is no proven benefit of this narrow optimization and x86
> maintainers are already skeptical.
> """
> 
> This optimization can be added back if it benefits kernel shadow stacks.
> But os_xrstor_supervisor() and os_xrstor_safe() would need the same
> handling.

That makes it more functional and less of an optimization. But, no less
of a hack.