[PATCH v3 0/3] x86/fred: Fix SEV-ES/SNP guest boot failures

[PATCH v3 0/3] x86/fred: Fix SEV-ES/SNP guest boot failures

[Nikunj A Dadhania]

SEV-ES and SNP guests fail to boot when FRED is enabled due to missing #VC
exception handling and a CR-pinning issue that prematurely enables FRED before
its MSRs are configured.

Patch 1 fixes the implicit FSGSBASE dependency by moving FSGSBASE enablement
to cpu_init_exception_handling(), ensuring it's enabled before exception
handling infrastructure is set up on both boot and secondary CPUs.

Patch 2 fixes the CR-pinning issue by deferring enforcement during secondary
CPU bringup, preventing FRED from being enabled before it is configured and
able to handle exceptions.

Patch 3 adds #VC exception handling to FRED and fixes early GHCB access to
use boot_ghcb before per-CPU GHCBs are initialized.

These fixes enable SEV-ES/SNP guests to boot successfully with FRED enabled.

Changelog:
v2 -> v3:
* Fix SNP guest boot failure reported by Borislav Petkov - the CR pinning
  disable patch exposed an implicit FSGSBASE dependency that is now fixed
  by moving FSGSBASE enablement to cpu_init_exception_handling() (Boris/Sohil)

v1 -> v2:
https://lore.kernel.org/kvm/20260226092349.803491-1-nikunj@amd.com/
* Instead of moving pr_info(), fix the root cause by disabling CR pinning
  during secondary CPU bringup (Dave Hansen)
* Use unified #VC exception handler in fred_hwexc() (Dave Hansen)
* Collect RoB (Sohil Mehta)

v1: https://lore.kernel.org/kvm/20260205051030.1225975-1-nikunj@amd.com/

Dave Hansen (1):
  x86/cpu: Disable CR pinning during CPU bringup

Nikunj A Dadhania (2):
  x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()
  x86/fred: Fix early boot failures on SEV-ES/SNP guests

 arch/x86/coco/sev/noinstr.c  |  6 ++++++
 arch/x86/entry/entry_fred.c  | 14 ++++++++++++
 arch/x86/kernel/cpu/common.c | 41 +++++++++++++++++++++++++++---------
 3 files changed, 51 insertions(+), 10 deletions(-)


base-commit: 270b06a2452b0a20b149591ee90e22e3d8d55358
-- 
2.48.1

[PATCH v3 1/3] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[Nikunj A Dadhania]

Move FSGSBASE enablement from identify_cpu() to
cpu_init_exception_handling() to ensure it is enabled before any exceptions
can occur on both boot and secondary CPUs.

== Background ==

Exception entry code (paranoid_entry()) uses ALTERNATIVE patching based on
X86_FEATURE_FSGSBASE to decide whether to use RDGSBASE/WRGSBASE
instructions or the slower RDMSR/SWAPGS sequence for saving/restoring
GSBASE.

For boot CPU, ALTERNATIVE patching happens after enabling FSGSBASE in CR4.
When the feature is available, the code is permanently patched to use
RDGSBASE/WRGSBASE, which require CR4.FSGSBASE=1 to execute without
triggering #UD.

== Boot Sequence ==

Boot CPU (with CR pinning enabled):
  trap_init()
    cpu_init()                   <- Uses unpatched code (RDMSR/SWAPGS)
      x2apic_setup()
  ...
  arch_cpu_finalize_init()
    identify_boot_cpu()
      identify_cpu()
        cr4_set_bits(X86_CR4_FSGSBASE)  # Enables the feature
	# This becomes part of cr4_pinned_bits
    ...
    alternative_instructions()   <- Patches code to use RDGSBASE/WRGSBASE

Secondary CPUs (with CR pinning enabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=1
                                    set implicitly via cr4_pinned_bits

    cpu_init()                   <- exceptions work because FSGSBASE is
                                    already enabled

Secondary CPU (with CR pinning disabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=0
    cpu_init()
      x2apic_setup()
        rdmsrq(MSR_IA32_APICBASE)  <- Triggers #VC in SNP guests
          exc_vmm_communication()
            paranoid_entry()       <- Uses RDGSBASE with CR4.FSGSBASE=0
                                      (patched code)
    ...
    ap_starting()
      identify_secondary_cpu()
        identify_cpu()
	  cr4_set_bits(X86_CR4_FSGSBASE)  <- Enables the feature, which is
                                             too late

== CR Pinning ==

Currently, for secondary CPUs, CR4.FSGSBASE is set implicitly through
CR-pinning: the boot CPU sets it during identify_cpu(), it becomes part of
cr4_pinned_bits, and cr4_init() applies those pinned bits to secondary
CPUs. This works but creates an undocumented dependency between cr4_init()
and the pinning mechanism.

== Problem ==

Secondary CPUs boot after alternatives have been applied globally. They
execute already-patched paranoid_entry() code that uses RDGSBASE/WRGSBASE
instructions, which require CR4.FSGSBASE=1. Upcoming changes to CR pinning
behavior will break the implicit dependency, causing secondary CPUs to
generate #UD.

This issue manifests on AMD SEV-SNP guests, where the rdmsrq() in
x2apic_setup() triggers a #VC exception early during cpu_init(). The #VC
handler (exc_vmm_communication()) executes the patched paranoid_entry()
path. Without CR4.FSGSBASE enabled, RDGSBASE instructions trigger #UD.

== Fix ==

Enable FSGSBASE explicitly in cpu_init_exception_handling() before loading
exception handlers. This makes the dependency explicit and ensures both
boot and secondary CPUs have FSGSBASE enabled before paranoid_entry()
executes.

Fixes: c82965f9e530 ("x86/entry/64: Handle FSGSBASE enabled paranoid entry/exit")
Cc: stable@vger.kernel.org
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Reported-by: Borislav Petkov <bp@alien8.de>
Suggested-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
---
 arch/x86/kernel/cpu/common.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index bb937bc4b00f..6778ec5846b6 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2066,12 +2066,6 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	setup_umip(c);
 	setup_lass(c);
 
-	/* Enable FSGSBASE instructions if available. */
-	if (cpu_has(c, X86_FEATURE_FSGSBASE)) {
-		cr4_set_bits(X86_CR4_FSGSBASE);
-		elf_hwcap2 |= HWCAP2_FSGSBASE;
-	}
-
 	/*
 	 * The vendor-specific functions might have changed features.
 	 * Now we do "generic changes."
@@ -2432,6 +2426,18 @@ void cpu_init_exception_handling(bool boot_cpu)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
+	/*
+	 * On CPUs with FSGSBASE support, paranoid_entry() uses
+	 * ALTERNATIVE-patched RDGSBASE/WRGSBASE instructions. Secondary CPUs
+	 * boot after alternatives are patched globally, so early exceptions
+	 * execute patched code that depends on FSGSBASE. Enable the feature
+	 * before any exceptions occur.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FSGSBASE)) {
+		cr4_set_bits(X86_CR4_FSGSBASE);
+		elf_hwcap2 |= HWCAP2_FSGSBASE;
+	}
+
 	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
 		/* The boot CPU has enabled FRED during early boot */
 		if (!boot_cpu)
-- 
2.48.1

Re: [PATCH v3 1/3] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[Sohil Mehta]

Hi Nikunj,

The code changes and the commit message looks fine to me. I would
suggest a minor correction to the code comment and the commit log. I
found it slightly misleading. It can probably be fixed up while applying
as well.

On 3/18/2026 12:56 AM, Nikunj A Dadhania wrote:
> Move FSGSBASE enablement from identify_cpu() to
> cpu_init_exception_handling() to ensure it is enabled before any exceptions
> can occur on both boot and secondary CPUs.
> 

It would be more accurate to say ".. before any exceptions that uses
paranoid_entry() can occur.."

I think early exceptions such #VC can still occur and the
bringup_idt_table is set up to handle those.

> == Background ==

...

> +	/*
> +	 * On CPUs with FSGSBASE support, paranoid_entry() uses
> +	 * ALTERNATIVE-patched RDGSBASE/WRGSBASE instructions. Secondary CPUs
> +	 * boot after alternatives are patched globally, so early exceptions
> +	 * execute patched code that depends on FSGSBASE. Enable the feature
> +	 * before any exceptions occur.


On similar lines. I suggest getting rid of the last line: "Enable the
feature before any exceptions occur." The rest looks fine to me.


> +	 */
> +	if (cpu_feature_enabled(X86_FEATURE_FSGSBASE)) {
> +		cr4_set_bits(X86_CR4_FSGSBASE);
> +		elf_hwcap2 |= HWCAP2_FSGSBASE;
> +	}
> +

Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>

Re: [PATCH v3 1/3] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[Borislav Petkov]

On Wed, Mar 18, 2026 at 06:54:46AM -0700, Sohil Mehta wrote:
> It would be more accurate to say ".. before any exceptions that uses
> paranoid_entry() can occur.."
> 
> I think early exceptions such #VC can still occur and the
> bringup_idt_table is set up to handle those.

I really don't understand what the splitting of hairs is supposed to bring
here?!

The commit message is more than clear and overly detailed - I even thought of
shortening it because it went too long...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v3 1/3] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[Dave Hansen]

On 3/18/26 08:39, Borislav Petkov wrote:
> On Wed, Mar 18, 2026 at 06:54:46AM -0700, Sohil Mehta wrote:
>> It would be more accurate to say ".. before any exceptions that uses
>> paranoid_entry() can occur.."
>>
>> I think early exceptions such #VC can still occur and the
>> bringup_idt_table is set up to handle those.
> I really don't understand what the splitting of hairs is supposed to bring
> here?!
> 
> The commit message is more than clear and overly detailed - I even thought of
> shortening it because it went too long...

Yeah, while Sohil's suggested addition is technically correct, the
suggestion is a _bit_ too detailed for my taste. I'm OK leaving it like
Nikunj's original version.

Re: [PATCH v3 1/3] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[Sohil Mehta]

On 3/18/2026 8:53 AM, Dave Hansen wrote:
> On 3/18/26 08:39, Borislav Petkov wrote:
>> On Wed, Mar 18, 2026 at 06:54:46AM -0700, Sohil Mehta wrote:
>>> It would be more accurate to say ".. before any exceptions that uses
>>> paranoid_entry() can occur.."
>>>
>>> I think early exceptions such #VC can still occur and the
>>> bringup_idt_table is set up to handle those.
>> I really don't understand what the splitting of hairs is supposed to bring
>> here?!
>>
>> The commit message is more than clear and overly detailed - I even thought of
>> shortening it because it went too long...
> 
> Yeah, while Sohil's suggested addition is technically correct, the
> suggestion is a _bit_ too detailed for my taste. I'm OK leaving it like
> Nikunj's original version.


Sorry about the churn.

While reviewing the patch, I was trying to understand the difference
between the early exception handlers and the one setup by
cpu_init_exception_handling() on BSP and APs. It caught my attention
that the code comment said, "Enable the feature (FSGSBASE) before any
exceptions occur."

I got a bit carried away with the technicalities!

[tip: x86/urgent] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[tip-bot2 for Nikunj A Dadhania]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     31e1e8c4eb7ff7fd8cb8a9d0b216c25d18ea37eb
Gitweb:        https://git.kernel.org/tip/31e1e8c4eb7ff7fd8cb8a9d0b216c25d18ea37eb
Author:        Nikunj A Dadhania <nikunj@amd.com>
AuthorDate:    Wed, 18 Mar 2026 07:56:52 
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Wed, 18 Mar 2026 16:40:42 +01:00

x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

Move FSGSBASE enablement from identify_cpu() to cpu_init_exception_handling()
to ensure it is enabled before any exceptions can occur on both boot and
secondary CPUs.

== Background ==

Exception entry code (paranoid_entry()) uses ALTERNATIVE patching based on
X86_FEATURE_FSGSBASE to decide whether to use RDGSBASE/WRGSBASE instructions
or the slower RDMSR/SWAPGS sequence for saving/restoring GSBASE.

On boot CPU, ALTERNATIVE patching happens after enabling FSGSBASE in CR4.
When the feature is available, the code is permanently patched to use
RDGSBASE/WRGSBASE, which require CR4.FSGSBASE=1 to execute without triggering

== Boot Sequence ==

Boot CPU (with CR pinning enabled):
  trap_init()
    cpu_init()                   <- Uses unpatched code (RDMSR/SWAPGS)
      x2apic_setup()
  ...
  arch_cpu_finalize_init()
    identify_boot_cpu()
      identify_cpu()
        cr4_set_bits(X86_CR4_FSGSBASE)  # Enables the feature
	# This becomes part of cr4_pinned_bits
    ...
    alternative_instructions()   <- Patches code to use RDGSBASE/WRGSBASE

Secondary CPUs (with CR pinning enabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=1
                                    set implicitly via cr4_pinned_bits

    cpu_init()                   <- exceptions work because FSGSBASE is
                                    already enabled

Secondary CPU (with CR pinning disabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=0
    cpu_init()
      x2apic_setup()
        rdmsrq(MSR_IA32_APICBASE)  <- Triggers #VC in SNP guests
          exc_vmm_communication()
            paranoid_entry()       <- Uses RDGSBASE with CR4.FSGSBASE=0
                                      (patched code)
    ...
    ap_starting()
      identify_secondary_cpu()
        identify_cpu()
	  cr4_set_bits(X86_CR4_FSGSBASE)  <- Enables the feature, which is
                                             too late

== CR Pinning ==

Currently, for secondary CPUs, CR4.FSGSBASE is set implicitly through
CR-pinning: the boot CPU sets it during identify_cpu(), it becomes part of
cr4_pinned_bits, and cr4_init() applies those pinned bits to secondary CPUs.
This works but creates an undocumented dependency between cr4_init() and the
pinning mechanism.

== Problem ==

Secondary CPUs boot after alternatives have been applied globally. They
execute already-patched paranoid_entry() code that uses RDGSBASE/WRGSBASE
instructions, which require CR4.FSGSBASE=1. Upcoming changes to CR pinning
behavior will break the implicit dependency, causing secondary CPUs to
generate #UD.

This issue manifests itself on AMD SEV-SNP guests, where the rdmsrq() in
x2apic_setup() triggers a #VC exception early during cpu_init(). The #VC
handler (exc_vmm_communication()) executes the patched paranoid_entry() path.
Without CR4.FSGSBASE enabled, RDGSBASE instructions trigger #UD.

== Fix ==

Enable FSGSBASE explicitly in cpu_init_exception_handling() before loading
exception handlers. This makes the dependency explicit and ensures both
boot and secondary CPUs have FSGSBASE enabled before paranoid_entry()
executes.

Fixes: c82965f9e530 ("x86/entry/64: Handle FSGSBASE enabled paranoid entry/exit")
Reported-by: Borislav Petkov <bp@alien8.de>
Suggested-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260318075654.1792916-2-nikunj@amd.com
---
 arch/x86/kernel/cpu/common.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index a8ff437..7840b22 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2050,12 +2050,6 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	setup_umip(c);
 	setup_lass(c);
 
-	/* Enable FSGSBASE instructions if available. */
-	if (cpu_has(c, X86_FEATURE_FSGSBASE)) {
-		cr4_set_bits(X86_CR4_FSGSBASE);
-		elf_hwcap2 |= HWCAP2_FSGSBASE;
-	}
-
 	/*
 	 * The vendor-specific functions might have changed features.
 	 * Now we do "generic changes."
@@ -2416,6 +2410,18 @@ void cpu_init_exception_handling(bool boot_cpu)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
+	/*
+	 * On CPUs with FSGSBASE support, paranoid_entry() uses
+	 * ALTERNATIVE-patched RDGSBASE/WRGSBASE instructions. Secondary CPUs
+	 * boot after alternatives are patched globally, so early exceptions
+	 * execute patched code that depends on FSGSBASE. Enable the feature
+	 * before any exceptions occur.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FSGSBASE)) {
+		cr4_set_bits(X86_CR4_FSGSBASE);
+		elf_hwcap2 |= HWCAP2_FSGSBASE;
+	}
+
 	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
 		/* The boot CPU has enabled FRED during early boot */
 		if (!boot_cpu)

[tip: x86/urgent] x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

[tip-bot2 for Nikunj A Dadhania]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     05243d490bb7852a8acca7b5b5658019c7797a52
Gitweb:        https://git.kernel.org/tip/05243d490bb7852a8acca7b5b5658019c7797a52
Author:        Nikunj A Dadhania <nikunj@amd.com>
AuthorDate:    Wed, 18 Mar 2026 07:56:52 
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Mon, 23 Mar 2026 13:29:50 +01:00

x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()

Move FSGSBASE enablement from identify_cpu() to cpu_init_exception_handling()
to ensure it is enabled before any exceptions can occur on both boot and
secondary CPUs.

== Background ==

Exception entry code (paranoid_entry()) uses ALTERNATIVE patching based on
X86_FEATURE_FSGSBASE to decide whether to use RDGSBASE/WRGSBASE instructions
or the slower RDMSR/SWAPGS sequence for saving/restoring GSBASE.

On boot CPU, ALTERNATIVE patching happens after enabling FSGSBASE in CR4.
When the feature is available, the code is permanently patched to use
RDGSBASE/WRGSBASE, which require CR4.FSGSBASE=1 to execute without triggering

== Boot Sequence ==

Boot CPU (with CR pinning enabled):
  trap_init()
    cpu_init()                   <- Uses unpatched code (RDMSR/SWAPGS)
      x2apic_setup()
  ...
  arch_cpu_finalize_init()
    identify_boot_cpu()
      identify_cpu()
        cr4_set_bits(X86_CR4_FSGSBASE)  # Enables the feature
	# This becomes part of cr4_pinned_bits
    ...
    alternative_instructions()   <- Patches code to use RDGSBASE/WRGSBASE

Secondary CPUs (with CR pinning enabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=1
                                    set implicitly via cr4_pinned_bits

    cpu_init()                   <- exceptions work because FSGSBASE is
                                    already enabled

Secondary CPU (with CR pinning disabled):
  start_secondary()
    cr4_init()                   <- Code already patched, CR4.FSGSBASE=0
    cpu_init()
      x2apic_setup()
        rdmsrq(MSR_IA32_APICBASE)  <- Triggers #VC in SNP guests
          exc_vmm_communication()
            paranoid_entry()       <- Uses RDGSBASE with CR4.FSGSBASE=0
                                      (patched code)
    ...
    ap_starting()
      identify_secondary_cpu()
        identify_cpu()
	  cr4_set_bits(X86_CR4_FSGSBASE)  <- Enables the feature, which is
                                             too late

== CR Pinning ==

Currently, for secondary CPUs, CR4.FSGSBASE is set implicitly through
CR-pinning: the boot CPU sets it during identify_cpu(), it becomes part of
cr4_pinned_bits, and cr4_init() applies those pinned bits to secondary CPUs.
This works but creates an undocumented dependency between cr4_init() and the
pinning mechanism.

== Problem ==

Secondary CPUs boot after alternatives have been applied globally. They
execute already-patched paranoid_entry() code that uses RDGSBASE/WRGSBASE
instructions, which require CR4.FSGSBASE=1. Upcoming changes to CR pinning
behavior will break the implicit dependency, causing secondary CPUs to
generate #UD.

This issue manifests itself on AMD SEV-SNP guests, where the rdmsrq() in
x2apic_setup() triggers a #VC exception early during cpu_init(). The #VC
handler (exc_vmm_communication()) executes the patched paranoid_entry() path.
Without CR4.FSGSBASE enabled, RDGSBASE instructions trigger #UD.

== Fix ==

Enable FSGSBASE explicitly in cpu_init_exception_handling() before loading
exception handlers. This makes the dependency explicit and ensures both
boot and secondary CPUs have FSGSBASE enabled before paranoid_entry()
executes.

Fixes: c82965f9e530 ("x86/entry/64: Handle FSGSBASE enabled paranoid entry/exit")
Reported-by: Borislav Petkov <bp@alien8.de>
Suggested-by: Sohil Mehta <sohil.mehta@intel.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/20260318075654.1792916-2-nikunj@amd.com
---
 arch/x86/kernel/cpu/common.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index a8ff437..7840b22 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2050,12 +2050,6 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	setup_umip(c);
 	setup_lass(c);
 
-	/* Enable FSGSBASE instructions if available. */
-	if (cpu_has(c, X86_FEATURE_FSGSBASE)) {
-		cr4_set_bits(X86_CR4_FSGSBASE);
-		elf_hwcap2 |= HWCAP2_FSGSBASE;
-	}
-
 	/*
 	 * The vendor-specific functions might have changed features.
 	 * Now we do "generic changes."
@@ -2416,6 +2410,18 @@ void cpu_init_exception_handling(bool boot_cpu)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
+	/*
+	 * On CPUs with FSGSBASE support, paranoid_entry() uses
+	 * ALTERNATIVE-patched RDGSBASE/WRGSBASE instructions. Secondary CPUs
+	 * boot after alternatives are patched globally, so early exceptions
+	 * execute patched code that depends on FSGSBASE. Enable the feature
+	 * before any exceptions occur.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FSGSBASE)) {
+		cr4_set_bits(X86_CR4_FSGSBASE);
+		elf_hwcap2 |= HWCAP2_FSGSBASE;
+	}
+
 	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
 		/* The boot CPU has enabled FRED during early boot */
 		if (!boot_cpu)

[PATCH v3 2/3] x86/cpu: Disable CR pinning during CPU bringup

[Nikunj A Dadhania]

From: Dave Hansen <dave.hansen@linux.intel.com>

== CR Pinning Background ==

Modern CPU hardening features like SMAP/SMEP are enabled by flipping
control register (CR) bits. Attackers find these features inconvenient and
often try to disable them.

CR-pinning is a kernel hardening feature that detects when
security-sensitive control bits are flipped off, complains about it, then
turns them back on. The CR-pinning checks are performed in the CR
manipulation helpers.

X86_CR4_FRED controls FRED enabling and is pinned. There is a single,
system-wide static key that controls CR-pinning behavior. The static key is
enabled by the boot CPU after it has established its CR configuration.

The end result is that CR-pinning is not active while initializing the boot
CPU but it is active while bringing up secondary CPUs.

== FRED Background ==

FRED is a new hardware entry/exit feature for the kernel. It is not on by
default and started out as Intel-only. AMD is just adding support now.

FRED has MSRs for configuration and is enabled by the pinned X86_CR4_FRED
bit. It should not be enabled until after MSRs are properly initialized.

== SEV Background ==

AMD SEV-ES and SEV-SNP use #VC (Virtualization Communication) exceptions to
handle operations that require hypervisor assistance. These exceptions
occur during various operations including MMIO access, CPUID instructions,
and certain memory accesses.

Writes to the console can generate #VC.

== Problem ==

CR-pinning implicitly enables FRED on secondary CPUs at a different point
than the boot CPU. This point is *before* the CPU has done an explicit
cr4_set_bits(X86_CR4_FRED) and before the MSRs are initialized. This means
that there is a window where no exceptions can be handled.

For SEV-ES/SNP and TDX guests, any console output during this window
triggers #VC or #VE exceptions that result in triple faults because the
exception handlers rely on FRED MSRs that aren't yet configured.

== Fix ==

Defer CR-pinning enforcement during secondary CPU bringup. This avoids any
implicit CR changes during CPU bringup, ensuring that FRED is not enabled
before it is configured and able to handle a #VC or #VE.

Drop CR4 pinning logic from cr4_init() as it runs only during early
secondary bring up while the CPU is still offline, so CR4 pinning is never
in effect there. Remove the redundant pinned-mask application and add
WARN_ON_ONCE() to detect any future changes that might violate this
assumption.

This also aligns boot and secondary CPU bringup.

Note: FRED is not on by default anywhere so this is not likely to be
causing many problems. The only reason this was noticed was that AMD
started to enable FRED and was turning it on.

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Reported-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
[ Nikunj: Updated SEV background section wording ]
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Cc: stable@vger.kernel.org # 6.9+
---
 arch/x86/kernel/cpu/common.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 6778ec5846b6..b2a4a506eae9 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -453,6 +453,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
+static bool cr_pinning_enabled(void)
+{
+	if (!static_branch_likely(&cr_pinning))
+		return false;
+
+	/*
+	 * Do not enforce pinning during CPU bringup. It might
+	 * turn on features that are not set up yet, like FRED.
+	 */
+	if (!cpu_online(smp_processor_id()))
+		return false;
+
+	return true;
+}
+
 void native_write_cr0(unsigned long val)
 {
 	unsigned long bits_missing = 0;
@@ -460,7 +475,7 @@ void native_write_cr0(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr0": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & X86_CR0_WP) != X86_CR0_WP)) {
 			bits_missing = X86_CR0_WP;
 			val |= bits_missing;
@@ -479,7 +494,7 @@ void __no_profile native_write_cr4(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr4": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & cr4_pinned_mask) != cr4_pinned_bits)) {
 			bits_changed = (val & cr4_pinned_mask) ^ cr4_pinned_bits;
 			val = (val & ~cr4_pinned_mask) | cr4_pinned_bits;
@@ -521,8 +536,8 @@ void cr4_init(void)
 
 	if (boot_cpu_has(X86_FEATURE_PCID))
 		cr4 |= X86_CR4_PCIDE;
-	if (static_branch_likely(&cr_pinning))
-		cr4 = (cr4 & ~cr4_pinned_mask) | cr4_pinned_bits;
+
+	WARN_ON_ONCE(cr_pinning_enabled());
 
 	__write_cr4(cr4);
 
-- 
2.48.1

[tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[tip-bot2 for Dave Hansen]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     cccc0c8ff0a9849378dcbc1d2ee6ca8018740aab
Gitweb:        https://git.kernel.org/tip/cccc0c8ff0a9849378dcbc1d2ee6ca8018740aab
Author:        Dave Hansen <dave.hansen@linux.intel.com>
AuthorDate:    Wed, 18 Mar 2026 07:56:53 
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Wed, 18 Mar 2026 16:40:54 +01:00

x86/cpu: Disable CR pinning during CPU bringup

== CR Pinning Background ==

Modern CPU hardening features like SMAP/SMEP are enabled by flipping control
register (CR) bits. Attackers find these features inconvenient and often try
to disable them.

CR-pinning is a kernel hardening feature that detects when security-sensitive
control bits are flipped off, complains about it, then turns them back on. The
CR-pinning checks are performed in the CR manipulation helpers.

X86_CR4_FRED controls FRED enabling and is pinned. There is a single,
system-wide static key that controls CR-pinning behavior. The static key is
enabled by the boot CPU after it has established its CR configuration.

The end result is that CR-pinning is not active while initializing the boot
CPU but it is active while bringing up secondary CPUs.

== FRED Background ==

FRED is a new hardware entry/exit feature for the kernel. It is not on by
default and started out as Intel-only. AMD is just adding support now.

FRED has MSRs for configuration and is enabled by the pinned X86_CR4_FRED
bit. It should not be enabled until after MSRs are properly initialized.

== SEV Background ==

AMD SEV-ES and SEV-SNP use #VC (Virtualization Communication) exceptions to
handle operations that require hypervisor assistance. These exceptions
occur during various operations including MMIO access, CPUID instructions,
and certain memory accesses.

Writes to the console can generate #VC.

== Problem ==

CR-pinning implicitly enables FRED on secondary CPUs at a different point
than the boot CPU. This point is *before* the CPU has done an explicit
cr4_set_bits(X86_CR4_FRED) and before the MSRs are initialized. This means
that there is a window where no exceptions can be handled.

For SEV-ES/SNP and TDX guests, any console output during this window
triggers #VC or #VE exceptions that result in triple faults because the
exception handlers rely on FRED MSRs that aren't yet configured.

== Fix ==

Defer CR-pinning enforcement during secondary CPU bringup. This avoids any
implicit CR changes during CPU bringup, ensuring that FRED is not enabled
before it is configured and able to handle a #VC or #VE.

Drop CR4 pinning logic from cr4_init() as it runs only during early
secondary bring up while the CPU is still offline, so CR4 pinning is never
in effect there. Remove the redundant pinned-mask application and add
WARN_ON_ONCE() to detect any future changes that might violate this
assumption.

This also aligns boot and secondary CPU bringup.

Note: FRED is not on by default anywhere so this is not likely to be
causing many problems. The only reason this was noticed was that AMD
started to enable FRED and was turning it on.

  [ Nikunj: Updated SEV background section wording ]

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Reported-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Cc: stable@vger.kernel.org # 6.9+
Link: https://patch.msgid.link/20260318075654.1792916-3-nikunj@amd.com
---
 arch/x86/kernel/cpu/common.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 7840b22..dbd7bce 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
+static bool cr_pinning_enabled(void)
+{
+	if (!static_branch_likely(&cr_pinning))
+		return false;
+
+	/*
+	 * Do not enforce pinning during CPU bringup. It might
+	 * turn on features that are not set up yet, like FRED.
+	 */
+	if (!cpu_online(smp_processor_id()))
+		return false;
+
+	return true;
+}
+
 void native_write_cr0(unsigned long val)
 {
 	unsigned long bits_missing = 0;
@@ -444,7 +459,7 @@ void native_write_cr0(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr0": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & X86_CR0_WP) != X86_CR0_WP)) {
 			bits_missing = X86_CR0_WP;
 			val |= bits_missing;
@@ -463,7 +478,7 @@ void __no_profile native_write_cr4(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr4": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & cr4_pinned_mask) != cr4_pinned_bits)) {
 			bits_changed = (val & cr4_pinned_mask) ^ cr4_pinned_bits;
 			val = (val & ~cr4_pinned_mask) | cr4_pinned_bits;
@@ -505,8 +520,8 @@ void cr4_init(void)
 
 	if (boot_cpu_has(X86_FEATURE_PCID))
 		cr4 |= X86_CR4_PCIDE;
-	if (static_branch_likely(&cr_pinning))
-		cr4 = (cr4 & ~cr4_pinned_mask) | cr4_pinned_bits;
+
+	WARN_ON_ONCE(cr_pinning_enabled());
 
 	__write_cr4(cr4);
 

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Peter Zijlstra]

On Wed, Mar 18, 2026 at 06:51:10PM -0000, tip-bot2 for Dave Hansen wrote:
> --- a/arch/x86/kernel/cpu/common.c
> +++ b/arch/x86/kernel/cpu/common.c
> @@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
>  static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
>  static unsigned long cr4_pinned_bits __ro_after_init;
>  
> +static bool cr_pinning_enabled(void)
> +{
> +	if (!static_branch_likely(&cr_pinning))
> +		return false;
> +
> +	/*
> +	 * Do not enforce pinning during CPU bringup. It might
> +	 * turn on features that are not set up yet, like FRED.
> +	 */
> +	if (!cpu_online(smp_processor_id()))
> +		return false;
> +
> +	return true;
> +}

Urgh, so this means all an attack needs to do is disable the online bit
and it gets to poke CR4 bits.

This seems unfortunate.

And sure, randomly clearing the online bit will eventually cause havoc,
but I suspect you still get plenty time until the system goes wobbly.

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Borislav Petkov]

On Wed, Mar 18, 2026 at 09:47:22PM +0100, Peter Zijlstra wrote:
> On Wed, Mar 18, 2026 at 06:51:10PM -0000, tip-bot2 for Dave Hansen wrote:
> > --- a/arch/x86/kernel/cpu/common.c
> > +++ b/arch/x86/kernel/cpu/common.c
> > @@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
> >  static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
> >  static unsigned long cr4_pinned_bits __ro_after_init;
> >  
> > +static bool cr_pinning_enabled(void)
> > +{
> > +	if (!static_branch_likely(&cr_pinning))
> > +		return false;
> > +
> > +	/*
> > +	 * Do not enforce pinning during CPU bringup. It might
> > +	 * turn on features that are not set up yet, like FRED.
> > +	 */
> > +	if (!cpu_online(smp_processor_id()))
> > +		return false;
> > +
> > +	return true;
> > +}
> 
> Urgh, so this means all an attack needs to do is disable the online bit
> and it gets to poke CR4 bits.
> 
> This seems unfortunate.
> 
> And sure, randomly clearing the online bit will eventually cause havoc,
> but I suspect you still get plenty time until the system goes wobbly.

My idea was that this is only temporary and then, ontop, we'll do something
like this:

https://lore.kernel.org/r/cb492a37-3517-4738-b435-73311402e820@intel.com

I.e., you figure out all the CR4 pinned bits on the BSP *once*, cast them in
stone and then replicate them on the APs when they come up.

I.e., you figure everything out the earliest and then no more switching.

Then all that gunk will disappear, hopefully.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Peter Zijlstra]

On Wed, Mar 18, 2026 at 10:08:13PM +0100, Borislav Petkov wrote:
> On Wed, Mar 18, 2026 at 09:47:22PM +0100, Peter Zijlstra wrote:
> > On Wed, Mar 18, 2026 at 06:51:10PM -0000, tip-bot2 for Dave Hansen wrote:
> > > --- a/arch/x86/kernel/cpu/common.c
> > > +++ b/arch/x86/kernel/cpu/common.c
> > > @@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
> > >  static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
> > >  static unsigned long cr4_pinned_bits __ro_after_init;
> > >  
> > > +static bool cr_pinning_enabled(void)
> > > +{
> > > +	if (!static_branch_likely(&cr_pinning))
> > > +		return false;
> > > +
> > > +	/*
> > > +	 * Do not enforce pinning during CPU bringup. It might
> > > +	 * turn on features that are not set up yet, like FRED.
> > > +	 */
> > > +	if (!cpu_online(smp_processor_id()))
> > > +		return false;
> > > +
> > > +	return true;
> > > +}
> > 
> > Urgh, so this means all an attack needs to do is disable the online bit
> > and it gets to poke CR4 bits.
> > 
> > This seems unfortunate.
> > 
> > And sure, randomly clearing the online bit will eventually cause havoc,
> > but I suspect you still get plenty time until the system goes wobbly.
> 
> My idea was that this is only temporary and then, ontop, we'll do something

This isn't temporary, this is marked for infinite backports :/ And it is
really really bad.

> like this:
> 
> https://lore.kernel.org/r/cb492a37-3517-4738-b435-73311402e820@intel.com

I'm not understanding.

> I.e., you figure out all the CR4 pinned bits on the BSP *once*, cast them in
> stone and then replicate them on the APs when they come up.

That's what we do now. Its just that the AP bringup code doesn't seem
capable of dealing with this.

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Borislav Petkov]

On Wed, Mar 18, 2026 at 10:30:29PM +0100, Peter Zijlstra wrote:
> This isn't temporary, this is marked for infinite backports :/ And it is
> really really bad.

Ok, zapping all three. I'll redo the whole thing tomorrow on a clear head and
then we can talk.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Peter Zijlstra]

On Wed, Mar 18, 2026 at 09:47:22PM +0100, Peter Zijlstra wrote:
> On Wed, Mar 18, 2026 at 06:51:10PM -0000, tip-bot2 for Dave Hansen wrote:
> > --- a/arch/x86/kernel/cpu/common.c
> > +++ b/arch/x86/kernel/cpu/common.c
> > @@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
> >  static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
> >  static unsigned long cr4_pinned_bits __ro_after_init;
> >  
> > +static bool cr_pinning_enabled(void)
> > +{
> > +	if (!static_branch_likely(&cr_pinning))
> > +		return false;
> > +
> > +	/*
> > +	 * Do not enforce pinning during CPU bringup. It might
> > +	 * turn on features that are not set up yet, like FRED.
> > +	 */
> > +	if (!cpu_online(smp_processor_id()))
> > +		return false;
> > +
> > +	return true;
> > +}
> 
> Urgh, so this means all an attack needs to do is disable the online bit
> and it gets to poke CR4 bits.
> 
> This seems unfortunate.
> 
> And sure, randomly clearing the online bit will eventually cause havoc,
> but I suspect you still get plenty time until the system goes wobbly.

So what is the problem with removing FRED from cr4_pinned_mask?
Specifically, set it up such that if you 'accidentally' clear that, the
machines insta dies a horrible death.

So currently we setup an IDT and everything, then setup the FRED MSRs,
flip CR4_FRED and call it a day. But we could just explicitly poison all
the IDT stuff to cause tripple faults.

Fixing that up is a much bigger ask of an attacker, no?

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Dave Hansen]

On 3/18/26 14:09, Peter Zijlstra wrote:
> So currently we setup an IDT and everything, then setup the FRED MSRs,
> flip CR4_FRED and call it a day. But we could just explicitly poison all
> the IDT stuff to cause tripple faults.

We already have:

        /* Enable FRED */
        cr4_set_bits(X86_CR4_FRED);
        /* Any further IDT use is a bug */
        idt_invalidate();

which I think means that if you clear X86_CR4_FRED, you triple-fault on
the next reference to the IDT. That's a fate far worse than having the
CR-pinning code silently fix up X86_CR4_FRED.

It's arguable that having X86_CR4_FRED pinned in the first place makes
things less secure if an attacker is thwacking CR4 bits.

Re: [tip: x86/urgent] x86/cpu: Disable CR pinning during CPU bringup

[Peter Zijlstra]

On Wed, Mar 18, 2026 at 09:47:22PM +0100, Peter Zijlstra wrote:
> On Wed, Mar 18, 2026 at 06:51:10PM -0000, tip-bot2 for Dave Hansen wrote:
> > --- a/arch/x86/kernel/cpu/common.c
> > +++ b/arch/x86/kernel/cpu/common.c
> > @@ -437,6 +437,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
> >  static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
> >  static unsigned long cr4_pinned_bits __ro_after_init;
> >  
> > +static bool cr_pinning_enabled(void)
> > +{
> > +	if (!static_branch_likely(&cr_pinning))
> > +		return false;
> > +
> > +	/*
> > +	 * Do not enforce pinning during CPU bringup. It might
> > +	 * turn on features that are not set up yet, like FRED.
> > +	 */
> > +	if (!cpu_online(smp_processor_id()))
> > +		return false;
> > +
> > +	return true;
> > +}
> 
> Urgh, so this means all an attack needs to do is disable the online bit
> and it gets to poke CR4 bits.
> 
> This seems unfortunate.
> 
> And sure, randomly clearing the online bit will eventually cause havoc,
> but I suspect you still get plenty time until the system goes wobbly.

The below tries to explain the CR pinning; and shows how the above
effectively disables the entire scheme since the online bit lives in RW
memory.

That is, the sequence:

  clear online bit
  ROP into 'mov %reg, %CR4'
  (re)set online bit

is fairly trivial, all things considering.

---
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index bb937bc4b00f..994e09d8c2fb 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -450,6 +450,19 @@ late_initcall(cpu_finalize_pre_userspace);
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
 					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED;
+
+/*
+ * The CR pinning protects against ROP on the 'mov %reg, %CRn' instruction(s).
+ * Since you can ROP directly to these instructions (barring shadow stack),
+ * any protection must follow immediately and unconditionally after that.
+ *
+ * Specifically, the CR[04] write functions below will have the value
+ * validation controlled by the @cr_pinning static_branch which is
+ * __ro_after_init, just like the cr4_pinned_bits value.
+ *
+ * Once set, an attacker will have to defeat page-tables to get around these
+ * restrictions. Which is a much bigger ask than 'simple' ROP.
+ */
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 

[PATCH] x86/cpu: Add comment clarifying CRn pinning

[Peter Zijlstra]

Since Boris wanted a nice patch to just press 'apply' on, here goes :-)


---
Subject: x86/cpu: Add comment clarifying CRn pinning
From: Peter Zijlstra <peterz@infradead.org>
Date: Wed, 18 Mar 2026 23:09:39 +0100

To avoid future confusion on the purpose and design of the CRn pinning
code.

Also note that if the attacker controls page-tables, the CRn bits
loose much of the attraction anyway.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 arch/x86/kernel/cpu/common.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -434,6 +434,19 @@ static __always_inline void setup_lass(s
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
 					     X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED;
+
+/*
+ * The CR pinning protects against ROP on the 'mov %reg, %CRn' instruction(s).
+ * Since you can ROP directly to these instructions (barring shadow stack),
+ * any protection must follow immediately and unconditionally after that.
+ *
+ * Specifically, the CR[04] write functions below will have the value
+ * validation controlled by the @cr_pinning static_branch which is
+ * __ro_after_init, just like the cr4_pinned_bits value.
+ *
+ * Once set, an attacker will have to defeat page-tables to get around these
+ * restrictions. Which is a much bigger ask than 'simple' ROP.
+ */
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 

Re: [PATCH] x86/cpu: Add comment clarifying CRn pinning

[Borislav Petkov]

On Fri, Mar 20, 2026 at 10:25:21AM +0100, Peter Zijlstra wrote:
> 
> Since Boris wanted a nice patch to just press 'apply' on, here goes :-)

/me presses that key!

Thanks man!

:-P

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

[tip: x86/urgent] x86/cpu: Add comment clarifying CRn pinning

[tip-bot2 for Peter Zijlstra]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     a3e93cac25316aad03bf561e3c205f4ca0b8f452
Gitweb:        https://git.kernel.org/tip/a3e93cac25316aad03bf561e3c205f4ca0b8f452
Author:        Peter Zijlstra <peterz@infradead.org>
AuthorDate:    Fri, 20 Mar 2026 10:25:21 +01:00
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Mon, 23 Mar 2026 14:25:53 +01:00

x86/cpu: Add comment clarifying CRn pinning

To avoid future confusion on the purpose and design of the CRn pinning code.

Also note that if the attacker controls page-tables, the CRn bits lose much of
the attraction anyway.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260320092521.GG3739106@noisy.programming.kicks-ass.net
---
 arch/x86/kernel/cpu/common.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index c57e897..ec06701 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -434,6 +434,19 @@ static __always_inline void setup_lass(struct cpuinfo_x86 *c)
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
 					     X86_CR4_FSGSBASE | X86_CR4_CET;
+
+/*
+ * The CR pinning protects against ROP on the 'mov %reg, %CRn' instruction(s).
+ * Since you can ROP directly to these instructions (barring shadow stack),
+ * any protection must follow immediately and unconditionally after that.
+ *
+ * Specifically, the CR[04] write functions below will have the value
+ * validation controlled by the @cr_pinning static_branch which is
+ * __ro_after_init, just like the cr4_pinned_bits value.
+ *
+ * Once set, an attacker will have to defeat page-tables to get around these
+ * restrictions. Which is a much bigger ask than 'simple' ROP.
+ */
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 

[PATCH v3 3/3] x86/fred: Fix early boot failures on SEV-ES/SNP guests

[Nikunj A Dadhania]

FRED-enabled SEV-ES and SNP guests fail to boot due to the following issues
in the early boot sequence:

* FRED does not have a #VC exception handler in the dispatch logic

* Early FRED #VC exceptions attempt to use uninitialized per-CPU GHCBs
  instead of boot_ghcb

Add X86_TRAP_VC case to fred_hwexc() with a new exc_vmm_communication()
function that provides the unified entry point FRED requires, dispatching
to existing user/kernel handlers based on privilege level. The function is
already declared via DECLARE_IDTENTRY_VC().

Fix early GHCB access by falling back to boot_ghcb in
__sev_{get,put}_ghcb() when per-CPU GHCBs are not yet initialized.

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Cc: stable@vger.kernel.org # 6.9+
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
---
 arch/x86/coco/sev/noinstr.c |  6 ++++++
 arch/x86/entry/entry_fred.c | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c
index 9d94aca4a698..5afd663a1c21 100644
--- a/arch/x86/coco/sev/noinstr.c
+++ b/arch/x86/coco/sev/noinstr.c
@@ -121,6 +121,9 @@ noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return boot_ghcb;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
@@ -164,6 +167,9 @@ noinstr void __sev_put_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c
index 88c757ac8ccd..fbe2d10dd737 100644
--- a/arch/x86/entry/entry_fred.c
+++ b/arch/x86/entry/entry_fred.c
@@ -177,6 +177,16 @@ static noinstr void fred_extint(struct pt_regs *regs)
 	}
 }
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+noinstr void exc_vmm_communication(struct pt_regs *regs, unsigned long error_code)
+{
+	if (user_mode(regs))
+		return user_exc_vmm_communication(regs, error_code);
+	else
+		return kernel_exc_vmm_communication(regs, error_code);
+}
+#endif
+
 static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 {
 	/* Optimize for #PF. That's the only exception which matters performance wise */
@@ -207,6 +217,10 @@ static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 #ifdef CONFIG_X86_CET
 	case X86_TRAP_CP: return exc_control_protection(regs, error_code);
 #endif
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	case X86_TRAP_VC: return exc_vmm_communication(regs, error_code);
+#endif
+
 	default: return fred_bad_type(regs, error_code);
 	}
 
-- 
2.48.1

[tip: x86/urgent] x86/fred: Fix early boot failures on SEV-ES/SNP guests

[tip-bot2 for Nikunj A Dadhania]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     afc77687e917b386e6878107cf70b43a38d6f209
Gitweb:        https://git.kernel.org/tip/afc77687e917b386e6878107cf70b43a38d6f209
Author:        Nikunj A Dadhania <nikunj@amd.com>
AuthorDate:    Wed, 18 Mar 2026 07:56:54 
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Wed, 18 Mar 2026 16:40:54 +01:00

x86/fred: Fix early boot failures on SEV-ES/SNP guests

FRED-enabled SEV-(ES,SNP) guests fail to boot due to the following issues
in the early boot sequence:

* FRED does not have a #VC exception handler in the dispatch logic

* Early FRED #VC exceptions attempt to use uninitialized per-CPU GHCBs
  instead of boot_ghcb

Add X86_TRAP_VC case to fred_hwexc() with a new exc_vmm_communication()
function that provides the unified entry point FRED requires, dispatching
to existing user/kernel handlers based on privilege level. The function is
already declared via DECLARE_IDTENTRY_VC().

Fix early GHCB access by falling back to boot_ghcb in
__sev_{get,put}_ghcb() when per-CPU GHCBs are not yet initialized.

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: stable@vger.kernel.org # 6.9+
Link: https://patch.msgid.link/20260318075654.1792916-4-nikunj@amd.com
---
 arch/x86/coco/sev/noinstr.c |  6 ++++++
 arch/x86/entry/entry_fred.c | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c
index 9d94aca..5afd663 100644
--- a/arch/x86/coco/sev/noinstr.c
+++ b/arch/x86/coco/sev/noinstr.c
@@ -121,6 +121,9 @@ noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return boot_ghcb;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
@@ -164,6 +167,9 @@ noinstr void __sev_put_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c
index 88c757a..fbe2d10 100644
--- a/arch/x86/entry/entry_fred.c
+++ b/arch/x86/entry/entry_fred.c
@@ -177,6 +177,16 @@ static noinstr void fred_extint(struct pt_regs *regs)
 	}
 }
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+noinstr void exc_vmm_communication(struct pt_regs *regs, unsigned long error_code)
+{
+	if (user_mode(regs))
+		return user_exc_vmm_communication(regs, error_code);
+	else
+		return kernel_exc_vmm_communication(regs, error_code);
+}
+#endif
+
 static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 {
 	/* Optimize for #PF. That's the only exception which matters performance wise */
@@ -207,6 +217,10 @@ static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 #ifdef CONFIG_X86_CET
 	case X86_TRAP_CP: return exc_control_protection(regs, error_code);
 #endif
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	case X86_TRAP_VC: return exc_vmm_communication(regs, error_code);
+#endif
+
 	default: return fred_bad_type(regs, error_code);
 	}
 

[tip: x86/urgent] x86/fred: Fix early boot failures on SEV-ES/SNP guests

[tip-bot2 for Nikunj A Dadhania]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     3645eb7e3915990a149460c151a00894cb586253
Gitweb:        https://git.kernel.org/tip/3645eb7e3915990a149460c151a00894cb586253
Author:        Nikunj A Dadhania <nikunj@amd.com>
AuthorDate:    Wed, 18 Mar 2026 07:56:54 
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Mon, 23 Mar 2026 14:18:18 +01:00

x86/fred: Fix early boot failures on SEV-ES/SNP guests

FRED-enabled SEV-(ES,SNP) guests fail to boot due to the following issues
in the early boot sequence:

* FRED does not have a #VC exception handler in the dispatch logic

* Early FRED #VC exceptions attempt to use uninitialized per-CPU GHCBs
  instead of boot_ghcb

Add X86_TRAP_VC case to fred_hwexc() with a new exc_vmm_communication()
function that provides the unified entry point FRED requires, dispatching
to existing user/kernel handlers based on privilege level. The function is
already declared via DECLARE_IDTENTRY_VC().

Fix early GHCB access by falling back to boot_ghcb in
__sev_{get,put}_ghcb() when per-CPU GHCBs are not yet initialized.

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: <stable@kernel.org>  # 6.12+
Link: https://patch.msgid.link/20260318075654.1792916-4-nikunj@amd.com
---
 arch/x86/coco/sev/noinstr.c |  6 ++++++
 arch/x86/entry/entry_fred.c | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c
index 9d94aca..5afd663 100644
--- a/arch/x86/coco/sev/noinstr.c
+++ b/arch/x86/coco/sev/noinstr.c
@@ -121,6 +121,9 @@ noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return boot_ghcb;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
@@ -164,6 +167,9 @@ noinstr void __sev_put_ghcb(struct ghcb_state *state)
 
 	WARN_ON(!irqs_disabled());
 
+	if (!sev_cfg.ghcbs_initialized)
+		return;
+
 	data = this_cpu_read(runtime_data);
 	ghcb = &data->ghcb_page;
 
diff --git a/arch/x86/entry/entry_fred.c b/arch/x86/entry/entry_fred.c
index 88c757a..fbe2d10 100644
--- a/arch/x86/entry/entry_fred.c
+++ b/arch/x86/entry/entry_fred.c
@@ -177,6 +177,16 @@ static noinstr void fred_extint(struct pt_regs *regs)
 	}
 }
 
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+noinstr void exc_vmm_communication(struct pt_regs *regs, unsigned long error_code)
+{
+	if (user_mode(regs))
+		return user_exc_vmm_communication(regs, error_code);
+	else
+		return kernel_exc_vmm_communication(regs, error_code);
+}
+#endif
+
 static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 {
 	/* Optimize for #PF. That's the only exception which matters performance wise */
@@ -207,6 +217,10 @@ static noinstr void fred_hwexc(struct pt_regs *regs, unsigned long error_code)
 #ifdef CONFIG_X86_CET
 	case X86_TRAP_CP: return exc_control_protection(regs, error_code);
 #endif
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+	case X86_TRAP_VC: return exc_vmm_communication(regs, error_code);
+#endif
+
 	default: return fred_bad_type(regs, error_code);
 	}
 

Re: [PATCH v3 0/3] x86/fred: Fix SEV-ES/SNP guest boot failures

[Borislav Petkov]

On Wed, Mar 18, 2026 at 07:56:51AM +0000, Nikunj A Dadhania wrote:
> Dave Hansen (1):
>   x86/cpu: Disable CR pinning during CPU bringup
> 
> Nikunj A Dadhania (2):
>   x86/cpu: Enable FSGSBASE early in cpu_init_exception_handling()
>   x86/fred: Fix early boot failures on SEV-ES/SNP guests
> 
>  arch/x86/coco/sev/noinstr.c  |  6 ++++++
>  arch/x86/entry/entry_fred.c  | 14 ++++++++++++
>  arch/x86/kernel/cpu/common.c | 41 +++++++++++++++++++++++++++---------
>  3 files changed, 51 insertions(+), 10 deletions(-)

Looks good so far in testing, I'll hammer on it some more and then queue it
later.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette