* [PATCH v7 1/7] x86/cpu, kvm: Add support for cpuid leaf 80000021/EAX (FeatureExt2Eax)
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-16 23:01 ` [PATCH v7 2/7] x86/cpu, kvm: Add the NO_NESTED_DATA_BP feature Kim Phillips
` (6 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
Add the 80000021/EAX leaf that advertises features in later Zen
processors. The majority of the features will be used in the kernel
and thus a separate leaf is appropriate.
Include KVM's reverse_cpuid entry because features are used by VM
guests, too.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/include/asm/cpufeature.h | 7 +++++--
arch/x86/include/asm/cpufeatures.h | 2 +-
arch/x86/include/asm/disabled-features.h | 3 ++-
arch/x86/include/asm/required-features.h | 3 ++-
arch/x86/kernel/cpu/common.c | 3 +++
arch/x86/kvm/reverse_cpuid.h | 1 +
6 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h
index 1a85e1fb0922..ce0c8f7d3218 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -32,6 +32,7 @@ enum cpuid_leafs
CPUID_8000_0007_EBX,
CPUID_7_EDX,
CPUID_8000_001F_EAX,
+ CPUID_8000_0021_EAX,
};
#define X86_CAP_FMT_NUM "%d:%d"
@@ -94,8 +95,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 17, feature_bit) || \
CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 18, feature_bit) || \
CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 19, feature_bit) || \
+ CHECK_BIT_IN_MASK_WORD(REQUIRED_MASK, 20, feature_bit) || \
REQUIRED_MASK_CHECK || \
- BUILD_BUG_ON_ZERO(NCAPINTS != 20))
+ BUILD_BUG_ON_ZERO(NCAPINTS != 21))
#define DISABLED_MASK_BIT_SET(feature_bit) \
( CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 0, feature_bit) || \
@@ -118,8 +120,9 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 17, feature_bit) || \
CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 18, feature_bit) || \
CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 19, feature_bit) || \
+ CHECK_BIT_IN_MASK_WORD(DISABLED_MASK, 20, feature_bit) || \
DISABLED_MASK_CHECK || \
- BUILD_BUG_ON_ZERO(NCAPINTS != 20))
+ BUILD_BUG_ON_ZERO(NCAPINTS != 21))
#define cpu_has(c, bit) \
(__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 : \
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 6cfa7143c316..a84536876794 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -13,7 +13,7 @@
/*
* Defines x86 CPU feature bits
*/
-#define NCAPINTS 20 /* N 32-bit words worth of info */
+#define NCAPINTS 21 /* N 32-bit words worth of info */
#define NBUGINTS 1 /* N 32-bit bug flags */
/*
diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h
index c44b56f7ffba..5dfa4fb76f4b 100644
--- a/arch/x86/include/asm/disabled-features.h
+++ b/arch/x86/include/asm/disabled-features.h
@@ -124,6 +124,7 @@
#define DISABLED_MASK17 0
#define DISABLED_MASK18 0
#define DISABLED_MASK19 0
-#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)
+#define DISABLED_MASK20 0
+#define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21)
#endif /* _ASM_X86_DISABLED_FEATURES_H */
diff --git a/arch/x86/include/asm/required-features.h b/arch/x86/include/asm/required-features.h
index aff774775c67..7ba1726b71c7 100644
--- a/arch/x86/include/asm/required-features.h
+++ b/arch/x86/include/asm/required-features.h
@@ -98,6 +98,7 @@
#define REQUIRED_MASK17 0
#define REQUIRED_MASK18 0
#define REQUIRED_MASK19 0
-#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 20)
+#define REQUIRED_MASK20 0
+#define REQUIRED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 21)
#endif /* _ASM_X86_REQUIRED_FEATURES_H */
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index ce40e7caa555..d762654d16a0 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1093,6 +1093,9 @@ void get_cpu_cap(struct cpuinfo_x86 *c)
if (c->extended_cpuid_level >= 0x8000001f)
c->x86_capability[CPUID_8000_001F_EAX] = cpuid_eax(0x8000001f);
+ if (c->extended_cpuid_level >= 0x80000021)
+ c->x86_capability[CPUID_8000_0021_EAX] = cpuid_eax(0x80000021);
+
init_scattered_cpuid_features(c);
init_speculation_control(c);
diff --git a/arch/x86/kvm/reverse_cpuid.h b/arch/x86/kvm/reverse_cpuid.h
index 042d0aca3c92..81f4e9ce0c77 100644
--- a/arch/x86/kvm/reverse_cpuid.h
+++ b/arch/x86/kvm/reverse_cpuid.h
@@ -68,6 +68,7 @@ static const struct cpuid_reg reverse_cpuid[] = {
[CPUID_12_EAX] = {0x00000012, 0, CPUID_EAX},
[CPUID_8000_001F_EAX] = {0x8000001f, 0, CPUID_EAX},
[CPUID_7_1_EDX] = { 7, 1, CPUID_EDX},
+ [CPUID_8000_0021_EAX] = {0x80000021, 0, CPUID_EAX},
};
/*
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH v7 2/7] x86/cpu, kvm: Add the NO_NESTED_DATA_BP feature
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
2023-01-16 23:01 ` [PATCH v7 1/7] x86/cpu, kvm: Add support for cpuid leaf 80000021/EAX (FeatureExt2Eax) Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-16 23:01 ` [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature Kim Phillips
` (5 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
The "Processor ignores nested data breakpoints" feature was being
open-coded for KVM in __do_cpuid_func(). Add it to its newly added
CPUID leaf 0x80000021 EAX proper, and propagate it in kvm_set_cpu_caps()
instead.
Also drop the bit description comments now it's more self-describing.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 3 +++
arch/x86/kvm/cpuid.c | 8 ++++++--
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index a84536876794..8255b95a7987 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -428,6 +428,9 @@
#define X86_FEATURE_V_TSC_AUX (19*32+ 9) /* "" Virtual TSC_AUX */
#define X86_FEATURE_SME_COHERENT (19*32+10) /* "" AMD hardware-enforced cache coherency */
+/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
+#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" AMD No Nested Data Breakpoints */
+
/*
* BUG word(s)
*/
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 596061c1610e..c9081e3a1b66 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -741,6 +741,10 @@ void kvm_set_cpu_caps(void)
0 /* SME */ | F(SEV) | 0 /* VM_PAGE_FLUSH */ | F(SEV_ES) |
F(SME_COHERENT));
+ kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
+ F(NO_NESTED_DATA_BP)
+ );
+
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
F(ACE2) | F(ACE2_EN) | F(PHE) | F(PHE_EN) |
@@ -1222,9 +1226,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
break;
case 0x80000021:
entry->ebx = entry->ecx = entry->edx = 0;
+ cpuid_entry_override(entry, CPUID_8000_0021_EAX);
/*
* Pass down these bits:
- * EAX 0 NNDBP, Processor ignores nested data breakpoints
* EAX 2 LAS, LFENCE always serializing
* EAX 6 NSCB, Null selector clear base
*
@@ -1235,7 +1239,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
* KVM doesn't support SMM_CTL.
* EAX 9 SMM_CTL MSR is not supported
*/
- entry->eax &= BIT(0) | BIT(2) | BIT(6);
+ entry->eax &= BIT(2) | BIT(6);
entry->eax |= BIT(9);
if (static_cpu_has(X86_FEATURE_LFENCE_RDTSC))
entry->eax |= BIT(2);
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
2023-01-16 23:01 ` [PATCH v7 1/7] x86/cpu, kvm: Add support for cpuid leaf 80000021/EAX (FeatureExt2Eax) Kim Phillips
2023-01-16 23:01 ` [PATCH v7 2/7] x86/cpu, kvm: Add the NO_NESTED_DATA_BP feature Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-17 11:33 ` Borislav Petkov
2023-01-17 21:25 ` [tip: x86/cpu] x86/cpu, kvm: Move X86_FEATURE_LFENCE_RDTSC tip-bot2 for Kim Phillips
2023-01-16 23:01 ` [PATCH v7 4/7] x86/cpu, kvm: Add the Null Selector Clears Base feature Kim Phillips
` (4 subsequent siblings)
7 siblings, 2 replies; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
The LFENCE_RDTSC / LFENCE always serializing feature was a scattered bit
and open-coded for KVM in __do_cpuid_func(). Add it to its newly added
CPUID leaf 0x80000021 EAX proper, and propagate it in kvm_set_cpu_caps()
instead. Drop the bit description comments now it's more self-describing.
Also, in amd_init(), don't bother setting DE_CFG[1] any more if we already
have the X86_FEATURE_LFENCE_RDTSC feature (set by hardware).
Whilst there, switch to using the more efficient cpu_feature_enabled()
instead of static_cpu_has().
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 3 ++-
arch/x86/kernel/cpu/amd.c | 2 +-
arch/x86/kvm/cpuid.c | 9 ++++-----
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 8255b95a7987..b22b2e8fef00 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -97,7 +97,7 @@
#define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspace */
#define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */
#define X86_FEATURE_AMD_LBR_V2 ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */
-#define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" LFENCE synchronizes RDTSC */
+/* FREE, was #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) "" LFENCE synchronizes RDTSC */
#define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */
#define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */
#define X86_FEATURE_ALWAYS ( 3*32+21) /* "" Always-present feature */
@@ -430,6 +430,7 @@
/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" AMD No Nested Data Breakpoints */
+#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
/*
* BUG word(s)
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index f769d6d08b43..208c2ce8598a 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -956,7 +956,7 @@ static void init_amd(struct cpuinfo_x86 *c)
init_amd_cacheinfo(c);
- if (cpu_has(c, X86_FEATURE_XMM2)) {
+ if (!cpu_has(c, X86_FEATURE_LFENCE_RDTSC) && cpu_has(c, X86_FEATURE_XMM2)) {
/*
* Use LFENCE for execution serialization. On families which
* don't have that MSR, LFENCE is already serializing.
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index c9081e3a1b66..d7a13716b7c8 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,8 +742,10 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC)
);
+ if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
+ kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1229,7 +1231,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
/*
* Pass down these bits:
- * EAX 2 LAS, LFENCE always serializing
* EAX 6 NSCB, Null selector clear base
*
* Other defined bits are for MSRs that KVM does not expose:
@@ -1239,10 +1240,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
* KVM doesn't support SMM_CTL.
* EAX 9 SMM_CTL MSR is not supported
*/
- entry->eax &= BIT(2) | BIT(6);
+ entry->eax &= BIT(6);
entry->eax |= BIT(9);
- if (static_cpu_has(X86_FEATURE_LFENCE_RDTSC))
- entry->eax |= BIT(2);
if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
entry->eax |= BIT(6);
break;
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature
2023-01-16 23:01 ` [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature Kim Phillips
@ 2023-01-17 11:33 ` Borislav Petkov
2023-01-17 21:25 ` [tip: x86/cpu] x86/cpu, kvm: Move X86_FEATURE_LFENCE_RDTSC tip-bot2 for Kim Phillips
1 sibling, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-01-17 11:33 UTC (permalink / raw)
To: Kim Phillips
Cc: x86, Boris Ostrovsky, Dave Hansen, H. Peter Anvin, Ingo Molnar,
Joao Martins, Jonathan Corbet, Konrad Rzeszutek Wilk,
Paolo Bonzini, Sean Christopherson, Thomas Gleixner,
David Woodhouse, Greg Kroah-Hartman, Juergen Gross,
Peter Zijlstra, Tony Luck, Tom Lendacky, Alexey Kardashevskiy,
kvm, linux-doc, linux-kernel
On Mon, Jan 16, 2023 at 05:01:55PM -0600, Kim Phillips wrote:
> The LFENCE_RDTSC / LFENCE always serializing feature was a scattered bit
> and open-coded for KVM in __do_cpuid_func(). Add it to its newly added
> CPUID leaf 0x80000021 EAX proper, and propagate it in kvm_set_cpu_caps()
> instead. Drop the bit description comments now it's more self-describing.
>
> Also, in amd_init(), don't bother setting DE_CFG[1] any more if we already
For the future, please use passive voice in your commit message: no "we" or "I",
etc, and describe your changes in imperative mood. Personal pronouns are
ambiguous in text, especially with so many parties/companies/etc developing the
kernel so let's avoid them please.
I'll fix it up now.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread* [tip: x86/cpu] x86/cpu, kvm: Move X86_FEATURE_LFENCE_RDTSC
2023-01-16 23:01 ` [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature Kim Phillips
2023-01-17 11:33 ` Borislav Petkov
@ 2023-01-17 21:25 ` tip-bot2 for Kim Phillips
1 sibling, 0 replies; 18+ messages in thread
From: tip-bot2 for Kim Phillips @ 2023-01-17 21:25 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Kim Phillips, Borislav Petkov (AMD), x86, linux-kernel
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: b594d2036c8440cda5488009d93966ae8a50c3e4
Gitweb: https://git.kernel.org/tip/b594d2036c8440cda5488009d93966ae8a50c3e4
Author: Kim Phillips <kim.phillips@amd.com>
AuthorDate: Mon, 16 Jan 2023 17:01:55 -06:00
Committer: Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 17 Jan 2023 13:00:12 +01:00
x86/cpu, kvm: Move X86_FEATURE_LFENCE_RDTSC
The LFENCE_RDTSC / LFENCE always serializing feature was a scattered bit
and open-coded for KVM in __do_cpuid_func(). Add it to its newly added
CPUID leaf 0x80000021 EAX proper, and propagate it in kvm_set_cpu_caps()
instead. Drop the bit description comments now it's more self-describing.
Also, in amd_init(), don't bother setting DE_CFG[1] any more.
Whilst there, switch to using the more efficient cpu_feature_enabled()
instead of static_cpu_has().
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20230116230159.1511393-4-kim.phillips@amd.com
---
arch/x86/include/asm/cpufeatures.h | 3 ++-
arch/x86/kernel/cpu/amd.c | 2 +-
arch/x86/kvm/cpuid.c | 9 ++++-----
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 1b2d40a..901128e 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -97,7 +97,7 @@
#define X86_FEATURE_SYSENTER32 ( 3*32+15) /* "" sysenter in IA32 userspace */
#define X86_FEATURE_REP_GOOD ( 3*32+16) /* REP microcode works well */
#define X86_FEATURE_AMD_LBR_V2 ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */
-#define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) /* "" LFENCE synchronizes RDTSC */
+/* FREE, was #define X86_FEATURE_LFENCE_RDTSC ( 3*32+18) "" LFENCE synchronizes RDTSC */
#define X86_FEATURE_ACC_POWER ( 3*32+19) /* AMD Accumulated Power Mechanism */
#define X86_FEATURE_NOPL ( 3*32+20) /* The NOPL (0F 1F) instructions */
#define X86_FEATURE_ALWAYS ( 3*32+21) /* "" Always-present feature */
@@ -429,6 +429,7 @@
/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" No Nested Data Breakpoints */
+#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
/*
* BUG word(s)
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index f769d6d..208c2ce 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -956,7 +956,7 @@ static void init_amd(struct cpuinfo_x86 *c)
init_amd_cacheinfo(c);
- if (cpu_has(c, X86_FEATURE_XMM2)) {
+ if (!cpu_has(c, X86_FEATURE_LFENCE_RDTSC) && cpu_has(c, X86_FEATURE_XMM2)) {
/*
* Use LFENCE for execution serialization. On families which
* don't have that MSR, LFENCE is already serializing.
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 69e433e..88c9700 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,8 +742,10 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC)
);
+ if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
+ kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1229,7 +1231,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
/*
* Pass down these bits:
- * EAX 2 LAS, LFENCE always serializing
* EAX 6 NSCB, Null selector clear base
*
* Other defined bits are for MSRs that KVM does not expose:
@@ -1239,10 +1240,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
* KVM doesn't support SMM_CTL.
* EAX 9 SMM_CTL MSR is not supported
*/
- entry->eax &= BIT(2) | BIT(6);
+ entry->eax &= BIT(6);
entry->eax |= BIT(9);
- if (static_cpu_has(X86_FEATURE_LFENCE_RDTSC))
- entry->eax |= BIT(2);
if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
entry->eax |= BIT(6);
break;
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 4/7] x86/cpu, kvm: Add the Null Selector Clears Base feature
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
` (2 preceding siblings ...)
2023-01-16 23:01 ` [PATCH v7 3/7] x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing feature Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-17 21:25 ` [tip: x86/cpu] " tip-bot2 for Kim Phillips
2023-01-16 23:01 ` [PATCH v7 5/7] x86/cpu, kvm: Add the SMM_CTL MSR not present feature Kim Phillips
` (3 subsequent siblings)
7 siblings, 1 reply; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
The Null Selector Clears Base feature was being open-coded for KVM in
__do_cpuid_func(). Add it to its newly added CPUID leaf 0x80000021 EAX
proper, and propagate it in kvm_set_cpu_caps() instead.
Also drop the bit description comments now it's more self-describing.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kvm/cpuid.c | 10 +++-------
2 files changed, 4 insertions(+), 7 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index b22b2e8fef00..ccef41ff718c 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -431,6 +431,7 @@
/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" AMD No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
+#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" AMD Null Selector Clears Base */
/*
* BUG word(s)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index d7a13716b7c8..afa86241f752 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,10 +742,12 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | F(NULL_SEL_CLR_BASE)
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
+ if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
+ kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1230,9 +1232,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
entry->ebx = entry->ecx = entry->edx = 0;
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
/*
- * Pass down these bits:
- * EAX 6 NSCB, Null selector clear base
- *
* Other defined bits are for MSRs that KVM does not expose:
* EAX 3 SPCL, SMM page configuration lock
* EAX 13 PCMSR, Prefetch control MSR
@@ -1240,10 +1239,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
* KVM doesn't support SMM_CTL.
* EAX 9 SMM_CTL MSR is not supported
*/
- entry->eax &= BIT(6);
entry->eax |= BIT(9);
- if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
- entry->eax |= BIT(6);
break;
/*Add support for Centaur's CPUID instruction*/
case 0xC0000000:
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [tip: x86/cpu] x86/cpu, kvm: Add the Null Selector Clears Base feature
2023-01-16 23:01 ` [PATCH v7 4/7] x86/cpu, kvm: Add the Null Selector Clears Base feature Kim Phillips
@ 2023-01-17 21:25 ` tip-bot2 for Kim Phillips
0 siblings, 0 replies; 18+ messages in thread
From: tip-bot2 for Kim Phillips @ 2023-01-17 21:25 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Kim Phillips, Borislav Petkov (AMD), x86, linux-kernel
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: b1366f515fd65fb83518fded6520894efa4e228f
Gitweb: https://git.kernel.org/tip/b1366f515fd65fb83518fded6520894efa4e228f
Author: Kim Phillips <kim.phillips@amd.com>
AuthorDate: Mon, 16 Jan 2023 17:01:56 -06:00
Committer: Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 17 Jan 2023 13:00:12 +01:00
x86/cpu, kvm: Add the Null Selector Clears Base feature
The Null Selector Clears Base feature was being open-coded for KVM in
__do_cpuid_func(). Add it to its newly added CPUID leaf 0x80000021 EAX
proper, and propagate it in kvm_set_cpu_caps() instead.
Also drop the bit description comments now it's more self-describing.
[ bp: Convert test in check_null_seg_clears_base() too. ]
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20230116230159.1511393-5-kim.phillips@amd.com
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kernel/cpu/common.c | 4 +---
arch/x86/kvm/cpuid.c | 10 +++-------
3 files changed, 5 insertions(+), 10 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 901128e..6bed80c 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -430,6 +430,7 @@
/* AMD-defined Extended Feature 2 EAX, CPUID level 0x80000021 (EAX), word 20 */
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
+#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" Null Selector Clears Base */
/*
* BUG word(s)
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index e6f3234..e6bf9b1 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1685,9 +1685,7 @@ void check_null_seg_clears_base(struct cpuinfo_x86 *c)
if (!IS_ENABLED(CONFIG_X86_64))
return;
- /* Zen3 CPUs advertise Null Selector Clears Base in CPUID. */
- if (c->extended_cpuid_level >= 0x80000021 &&
- cpuid_eax(0x80000021) & BIT(6))
+ if (cpu_has(c, X86_FEATURE_NULL_SEL_CLR_BASE))
return;
/*
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 88c9700..04f2f48 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,10 +742,12 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | F(NULL_SEL_CLR_BASE)
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
+ if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
+ kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1230,9 +1232,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
entry->ebx = entry->ecx = entry->edx = 0;
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
/*
- * Pass down these bits:
- * EAX 6 NSCB, Null selector clear base
- *
* Other defined bits are for MSRs that KVM does not expose:
* EAX 3 SPCL, SMM page configuration lock
* EAX 13 PCMSR, Prefetch control MSR
@@ -1240,10 +1239,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
* KVM doesn't support SMM_CTL.
* EAX 9 SMM_CTL MSR is not supported
*/
- entry->eax &= BIT(6);
entry->eax |= BIT(9);
- if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
- entry->eax |= BIT(6);
break;
/*Add support for Centaur's CPUID instruction*/
case 0xC0000000:
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 5/7] x86/cpu, kvm: Add the SMM_CTL MSR not present feature
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
` (3 preceding siblings ...)
2023-01-16 23:01 ` [PATCH v7 4/7] x86/cpu, kvm: Add the Null Selector Clears Base feature Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-17 21:25 ` [tip: x86/cpu] " tip-bot2 for Kim Phillips
2023-01-16 23:01 ` [PATCH v7 6/7] x86/cpu: Support AMD Automatic IBRS Kim Phillips
` (2 subsequent siblings)
7 siblings, 1 reply; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
The SMM_CTL MSR not present feature was being open-coded for KVM in
__do_cpuid_func(). Add it to its newly added CPUID leaf 0x80000021 EAX
proper, and propagate it in kvm_set_cpu_caps() instead.
Also drop the bit description comments now the code is more
self-describing, and retain the SmmPgCfgLock and PrefetchCtlMsr feature
bit comments at the kvm_cpu_cap_mask() callsite.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kvm/cpuid.c | 13 +++----------
2 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index ccef41ff718c..861d312c7955 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -432,6 +432,7 @@
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" AMD No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" AMD Null Selector Clears Base */
+#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* "" AMD SMM_CTL MSR is not present */
/*
* BUG word(s)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index afa86241f752..9ba75ad9d976 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,12 +742,14 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | F(NULL_SEL_CLR_BASE)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | 0 /* SmmPgCfgLock */ |
+ F(NULL_SEL_CLR_BASE) | 0 /* PrefetchCtlMsr */
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE);
+ kvm_cpu_cap_set(X86_FEATURE_NO_SMM_CTL_MSR);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1231,15 +1233,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
case 0x80000021:
entry->ebx = entry->ecx = entry->edx = 0;
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
- /*
- * Other defined bits are for MSRs that KVM does not expose:
- * EAX 3 SPCL, SMM page configuration lock
- * EAX 13 PCMSR, Prefetch control MSR
- *
- * KVM doesn't support SMM_CTL.
- * EAX 9 SMM_CTL MSR is not supported
- */
- entry->eax |= BIT(9);
break;
/*Add support for Centaur's CPUID instruction*/
case 0xC0000000:
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [tip: x86/cpu] x86/cpu, kvm: Add the SMM_CTL MSR not present feature
2023-01-16 23:01 ` [PATCH v7 5/7] x86/cpu, kvm: Add the SMM_CTL MSR not present feature Kim Phillips
@ 2023-01-17 21:25 ` tip-bot2 for Kim Phillips
0 siblings, 0 replies; 18+ messages in thread
From: tip-bot2 for Kim Phillips @ 2023-01-17 21:25 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Kim Phillips, Borislav Petkov (AMD), x86, linux-kernel
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: b7059f21b1c1c4ca5966d16bd3d9fa45cee89a87
Gitweb: https://git.kernel.org/tip/b7059f21b1c1c4ca5966d16bd3d9fa45cee89a87
Author: Kim Phillips <kim.phillips@amd.com>
AuthorDate: Mon, 16 Jan 2023 17:01:57 -06:00
Committer: Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 17 Jan 2023 13:00:12 +01:00
x86/cpu, kvm: Add the SMM_CTL MSR not present feature
The SMM_CTL MSR not present feature was being open-coded for KVM in
__do_cpuid_func(). Add it to its newly added CPUID leaf 0x80000021 EAX
proper, and propagate it in kvm_set_cpu_caps() instead.
Also drop the bit description comments now the code is more
self-describing, and retain the SmmPgCfgLock and PrefetchCtlMsr feature
bit comments at the kvm_cpu_cap_mask() callsite.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20230116230159.1511393-6-kim.phillips@amd.com
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kvm/cpuid.c | 13 +++----------
2 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 6bed80c..86e98bd 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -431,6 +431,7 @@
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" Null Selector Clears Base */
+#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* "" SMM_CTL MSR is not present */
/*
* BUG word(s)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 04f2f48..56f00d9 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -742,12 +742,14 @@ void kvm_set_cpu_caps(void)
F(SME_COHERENT));
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
- F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | F(NULL_SEL_CLR_BASE)
+ F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | 0 /* SmmPgCfgLock */ |
+ F(NULL_SEL_CLR_BASE) | 0 /* PrefetchCtlMsr */
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE);
+ kvm_cpu_cap_set(X86_FEATURE_NO_SMM_CTL_MSR);
kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
@@ -1231,15 +1233,6 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
case 0x80000021:
entry->ebx = entry->ecx = entry->edx = 0;
cpuid_entry_override(entry, CPUID_8000_0021_EAX);
- /*
- * Other defined bits are for MSRs that KVM does not expose:
- * EAX 3 SPCL, SMM page configuration lock
- * EAX 13 PCMSR, Prefetch control MSR
- *
- * KVM doesn't support SMM_CTL.
- * EAX 9 SMM_CTL MSR is not supported
- */
- entry->eax |= BIT(9);
break;
/*Add support for Centaur's CPUID instruction*/
case 0xC0000000:
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 6/7] x86/cpu: Support AMD Automatic IBRS
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
` (4 preceding siblings ...)
2023-01-16 23:01 ` [PATCH v7 5/7] x86/cpu, kvm: Add the SMM_CTL MSR not present feature Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-17 21:25 ` [tip: x86/cpu] " tip-bot2 for Kim Phillips
2023-01-16 23:01 ` [PATCH v7 7/7] x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest Kim Phillips
2023-01-17 11:54 ` [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Borislav Petkov
7 siblings, 1 reply; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
The AMD Zen4 core supports a new feature called Automatic IBRS.
It is a "set-and-forget" feature that means that, like
Intel's Enhanced IBRS, h/w manages its IBRS mitigation
resources automatically across CPL transitions.
The feature is advertised by CPUID_Fn80000021_EAX bit 8 and is
enabled by setting MSR C000_0080 (EFER) bit 21.
Enable Automatic IBRS by default if the CPU feature is present.
It typically provides greater performance over the incumbent
generic retpolines mitigation.
Reuse the SPECTRE_V2_EIBRS spectre_v2_mitigation enum.
AMD Automatic IBRS and Intel Enhanced IBRS have similar
bugs.c enablement. Add NO_EIBRS_PBRSB to cpu_vuln_whitelist,
since AMD Automatic IBRS isn't affected by PBRSB-eIBRS.
The kernel command line option spectre_v2=eibrs is
used to select AMD Automatic IBRS, if available.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
---
Documentation/admin-guide/hw-vuln/spectre.rst | 6 +++---
.../admin-guide/kernel-parameters.txt | 6 +++---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/include/asm/msr-index.h | 2 ++
arch/x86/kernel/cpu/bugs.c | 20 +++++++++++--------
arch/x86/kernel/cpu/common.c | 19 ++++++++++--------
6 files changed, 32 insertions(+), 22 deletions(-)
diff --git a/Documentation/admin-guide/hw-vuln/spectre.rst b/Documentation/admin-guide/hw-vuln/spectre.rst
index c4dcdb3d0d45..3fe6511c5405 100644
--- a/Documentation/admin-guide/hw-vuln/spectre.rst
+++ b/Documentation/admin-guide/hw-vuln/spectre.rst
@@ -610,9 +610,9 @@ kernel command line.
retpoline,generic Retpolines
retpoline,lfence LFENCE; indirect branch
retpoline,amd alias for retpoline,lfence
- eibrs enhanced IBRS
- eibrs,retpoline enhanced IBRS + Retpolines
- eibrs,lfence enhanced IBRS + LFENCE
+ eibrs Enhanced/Auto IBRS
+ eibrs,retpoline Enhanced/Auto IBRS + Retpolines
+ eibrs,lfence Enhanced/Auto IBRS + LFENCE
ibrs use IBRS to protect kernel
Not specifying this option is equivalent to
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 6cfa6e3996cf..839fa0fefb58 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5729,9 +5729,9 @@
retpoline,generic - Retpolines
retpoline,lfence - LFENCE; indirect branch
retpoline,amd - alias for retpoline,lfence
- eibrs - enhanced IBRS
- eibrs,retpoline - enhanced IBRS + Retpolines
- eibrs,lfence - enhanced IBRS + LFENCE
+ eibrs - Enhanced/Auto IBRS
+ eibrs,retpoline - Enhanced/Auto IBRS + Retpolines
+ eibrs,lfence - Enhanced/Auto IBRS + LFENCE
ibrs - use IBRS to protect kernel
Not specifying this option is equivalent to
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 861d312c7955..d5acc4dc5906 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -432,6 +432,7 @@
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" AMD No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" AMD Null Selector Clears Base */
+#define X86_FEATURE_AUTOIBRS (20*32+ 8) /* "" AMD Automatic IBRS */
#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* "" AMD SMM_CTL MSR is not present */
/*
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index a85019756003..cb3d0f6e6ac2 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -25,6 +25,7 @@
#define _EFER_SVME 12 /* Enable virtualization */
#define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */
#define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */
+#define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */
#define EFER_SCE (1<<_EFER_SCE)
#define EFER_LME (1<<_EFER_LME)
@@ -33,6 +34,7 @@
#define EFER_SVME (1<<_EFER_SVME)
#define EFER_LMSLE (1<<_EFER_LMSLE)
#define EFER_FFXSR (1<<_EFER_FFXSR)
+#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS)
/* Intel MSRs. Some also available on other CPUs */
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 4a0add86c182..cf81848b72f4 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1238,9 +1238,9 @@ static const char * const spectre_v2_strings[] = {
[SPECTRE_V2_NONE] = "Vulnerable",
[SPECTRE_V2_RETPOLINE] = "Mitigation: Retpolines",
[SPECTRE_V2_LFENCE] = "Mitigation: LFENCE",
- [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced IBRS",
- [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced IBRS + LFENCE",
- [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced IBRS + Retpolines",
+ [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced / Automatic IBRS",
+ [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced / Automatic IBRS + LFENCE",
+ [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced / Automatic IBRS + Retpolines",
[SPECTRE_V2_IBRS] = "Mitigation: IBRS",
};
@@ -1309,7 +1309,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void)
cmd == SPECTRE_V2_CMD_EIBRS_LFENCE ||
cmd == SPECTRE_V2_CMD_EIBRS_RETPOLINE) &&
!boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) {
- pr_err("%s selected but CPU doesn't have eIBRS. Switching to AUTO select\n",
+ pr_err("%s selected but CPU doesn't have Enhanced or Automatic IBRS. Switching to AUTO select\n",
mitigation_options[i].option);
return SPECTRE_V2_CMD_AUTO;
}
@@ -1495,8 +1495,12 @@ static void __init spectre_v2_select_mitigation(void)
pr_err(SPECTRE_V2_EIBRS_EBPF_MSG);
if (spectre_v2_in_ibrs_mode(mode)) {
- x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
- update_spec_ctrl(x86_spec_ctrl_base);
+ if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
+ msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
+ } else {
+ x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
+ update_spec_ctrl(x86_spec_ctrl_base);
+ }
}
switch (mode) {
@@ -1580,8 +1584,8 @@ static void __init spectre_v2_select_mitigation(void)
/*
* Retpoline protects the kernel, but doesn't protect firmware. IBRS
* and Enhanced IBRS protect firmware too, so enable IBRS around
- * firmware calls only when IBRS / Enhanced IBRS aren't otherwise
- * enabled.
+ * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't
+ * otherwise enabled.
*
* Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because
* the user might select retpoline on the kernel command line and if
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index d762654d16a0..b441758d2680 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1229,8 +1229,8 @@ static const __initconst struct x86_cpu_id cpu_vuln_whitelist[] = {
VULNWL_AMD(0x12, NO_MELTDOWN | NO_SSB | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
/* FAMILY_ANY must be last, otherwise 0x0f - 0x12 matches won't work */
- VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
- VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
+ VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB),
+ VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB),
/* Zhaoxin Family 7 */
VULNWL(CENTAUR, 7, X86_MODEL_ANY, NO_SPECTRE_V2 | NO_SWAPGS | NO_MMIO),
@@ -1341,8 +1341,16 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
!cpu_has(c, X86_FEATURE_AMD_SSB_NO))
setup_force_cpu_bug(X86_BUG_SPEC_STORE_BYPASS);
- if (ia32_cap & ARCH_CAP_IBRS_ALL)
+ /*
+ * AMD's AutoIBRS is equivalent to Intel's eIBRS - use the Intel feature
+ * flag and protect from vendor-specific bugs via the whitelist.
+ */
+ if ((ia32_cap & ARCH_CAP_IBRS_ALL) || cpu_has(c, X86_FEATURE_AUTOIBRS)) {
setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED);
+ if (!cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) &&
+ !(ia32_cap & ARCH_CAP_PBRSB_NO))
+ setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB);
+ }
if (!cpu_matches(cpu_vuln_whitelist, NO_MDS) &&
!(ia32_cap & ARCH_CAP_MDS_NO)) {
@@ -1404,11 +1412,6 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
setup_force_cpu_bug(X86_BUG_RETBLEED);
}
- if (cpu_has(c, X86_FEATURE_IBRS_ENHANCED) &&
- !cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) &&
- !(ia32_cap & ARCH_CAP_PBRSB_NO))
- setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB);
-
if (cpu_matches(cpu_vuln_whitelist, NO_MELTDOWN))
return;
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [tip: x86/cpu] x86/cpu: Support AMD Automatic IBRS
2023-01-16 23:01 ` [PATCH v7 6/7] x86/cpu: Support AMD Automatic IBRS Kim Phillips
@ 2023-01-17 21:25 ` tip-bot2 for Kim Phillips
0 siblings, 0 replies; 18+ messages in thread
From: tip-bot2 for Kim Phillips @ 2023-01-17 21:25 UTC (permalink / raw)
To: linux-tip-commits
Cc: Kim Phillips, Borislav Petkov (AMD), Dave Hansen, x86, linux-kernel
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: 116598cbdba664ba995658582034f966805877b5
Gitweb: https://git.kernel.org/tip/116598cbdba664ba995658582034f966805877b5
Author: Kim Phillips <kim.phillips@amd.com>
AuthorDate: Mon, 16 Jan 2023 17:01:58 -06:00
Committer: Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 17 Jan 2023 13:00:12 +01:00
x86/cpu: Support AMD Automatic IBRS
The AMD Zen4 core supports a new feature called Automatic IBRS.
It is a "set-and-forget" feature that means that, like Intel's Enhanced IBRS,
h/w manages its IBRS mitigation resources automatically across CPL transitions.
The feature is advertised by CPUID_Fn80000021_EAX bit 8 and is enabled by
setting MSR C000_0080 (EFER) bit 21.
Enable Automatic IBRS by default if the CPU feature is present. It typically
provides greater performance over the incumbent generic retpolines mitigation.
Reuse the SPECTRE_V2_EIBRS spectre_v2_mitigation enum. AMD Automatic IBRS and
Intel Enhanced IBRS have similar enablement. Add NO_EIBRS_PBRSB to
cpu_vuln_whitelist, since AMD Automatic IBRS isn't affected by PBRSB-eIBRS.
The kernel command line option spectre_v2=eibrs is used to select AMD Automatic
IBRS, if available.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/r/20230116230159.1511393-7-kim.phillips@amd.com
---
Documentation/admin-guide/hw-vuln/spectre.rst | 6 ++---
Documentation/admin-guide/kernel-parameters.txt | 6 ++---
arch/x86/include/asm/cpufeatures.h | 1 +-
arch/x86/include/asm/msr-index.h | 2 ++-
arch/x86/kernel/cpu/bugs.c | 20 +++++++++-------
arch/x86/kernel/cpu/common.c | 19 ++++++++-------
6 files changed, 32 insertions(+), 22 deletions(-)
diff --git a/Documentation/admin-guide/hw-vuln/spectre.rst b/Documentation/admin-guide/hw-vuln/spectre.rst
index c4dcdb3..3fe6511 100644
--- a/Documentation/admin-guide/hw-vuln/spectre.rst
+++ b/Documentation/admin-guide/hw-vuln/spectre.rst
@@ -610,9 +610,9 @@ kernel command line.
retpoline,generic Retpolines
retpoline,lfence LFENCE; indirect branch
retpoline,amd alias for retpoline,lfence
- eibrs enhanced IBRS
- eibrs,retpoline enhanced IBRS + Retpolines
- eibrs,lfence enhanced IBRS + LFENCE
+ eibrs Enhanced/Auto IBRS
+ eibrs,retpoline Enhanced/Auto IBRS + Retpolines
+ eibrs,lfence Enhanced/Auto IBRS + LFENCE
ibrs use IBRS to protect kernel
Not specifying this option is equivalent to
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 6cfa6e3..839fa0f 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5729,9 +5729,9 @@
retpoline,generic - Retpolines
retpoline,lfence - LFENCE; indirect branch
retpoline,amd - alias for retpoline,lfence
- eibrs - enhanced IBRS
- eibrs,retpoline - enhanced IBRS + Retpolines
- eibrs,lfence - enhanced IBRS + LFENCE
+ eibrs - Enhanced/Auto IBRS
+ eibrs,retpoline - Enhanced/Auto IBRS + Retpolines
+ eibrs,lfence - Enhanced/Auto IBRS + LFENCE
ibrs - use IBRS to protect kernel
Not specifying this option is equivalent to
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 86e98bd..06909dc 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -431,6 +431,7 @@
#define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* "" No Nested Data Breakpoints */
#define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* "" LFENCE always serializing / synchronizes RDTSC */
#define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* "" Null Selector Clears Base */
+#define X86_FEATURE_AUTOIBRS (20*32+ 8) /* "" Automatic IBRS */
#define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* "" SMM_CTL MSR is not present */
/*
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index cb359d6..617b29a 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -25,6 +25,7 @@
#define _EFER_SVME 12 /* Enable virtualization */
#define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */
#define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */
+#define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */
#define EFER_SCE (1<<_EFER_SCE)
#define EFER_LME (1<<_EFER_LME)
@@ -33,6 +34,7 @@
#define EFER_SVME (1<<_EFER_SVME)
#define EFER_LMSLE (1<<_EFER_LMSLE)
#define EFER_FFXSR (1<<_EFER_FFXSR)
+#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS)
/* Intel MSRs. Some also available on other CPUs */
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 5f33704..b41486a 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1238,9 +1238,9 @@ static const char * const spectre_v2_strings[] = {
[SPECTRE_V2_NONE] = "Vulnerable",
[SPECTRE_V2_RETPOLINE] = "Mitigation: Retpolines",
[SPECTRE_V2_LFENCE] = "Mitigation: LFENCE",
- [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced IBRS",
- [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced IBRS + LFENCE",
- [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced IBRS + Retpolines",
+ [SPECTRE_V2_EIBRS] = "Mitigation: Enhanced / Automatic IBRS",
+ [SPECTRE_V2_EIBRS_LFENCE] = "Mitigation: Enhanced / Automatic IBRS + LFENCE",
+ [SPECTRE_V2_EIBRS_RETPOLINE] = "Mitigation: Enhanced / Automatic IBRS + Retpolines",
[SPECTRE_V2_IBRS] = "Mitigation: IBRS",
};
@@ -1309,7 +1309,7 @@ static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void)
cmd == SPECTRE_V2_CMD_EIBRS_LFENCE ||
cmd == SPECTRE_V2_CMD_EIBRS_RETPOLINE) &&
!boot_cpu_has(X86_FEATURE_IBRS_ENHANCED)) {
- pr_err("%s selected but CPU doesn't have eIBRS. Switching to AUTO select\n",
+ pr_err("%s selected but CPU doesn't have Enhanced or Automatic IBRS. Switching to AUTO select\n",
mitigation_options[i].option);
return SPECTRE_V2_CMD_AUTO;
}
@@ -1495,8 +1495,12 @@ static void __init spectre_v2_select_mitigation(void)
pr_err(SPECTRE_V2_EIBRS_EBPF_MSG);
if (spectre_v2_in_ibrs_mode(mode)) {
- x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
- update_spec_ctrl(x86_spec_ctrl_base);
+ if (boot_cpu_has(X86_FEATURE_AUTOIBRS)) {
+ msr_set_bit(MSR_EFER, _EFER_AUTOIBRS);
+ } else {
+ x86_spec_ctrl_base |= SPEC_CTRL_IBRS;
+ update_spec_ctrl(x86_spec_ctrl_base);
+ }
}
switch (mode) {
@@ -1580,8 +1584,8 @@ static void __init spectre_v2_select_mitigation(void)
/*
* Retpoline protects the kernel, but doesn't protect firmware. IBRS
* and Enhanced IBRS protect firmware too, so enable IBRS around
- * firmware calls only when IBRS / Enhanced IBRS aren't otherwise
- * enabled.
+ * firmware calls only when IBRS / Enhanced / Automatic IBRS aren't
+ * otherwise enabled.
*
* Use "mode" to check Enhanced IBRS instead of boot_cpu_has(), because
* the user might select retpoline on the kernel command line and if
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index e6bf9b1..62c73c5 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1229,8 +1229,8 @@ static const __initconst struct x86_cpu_id cpu_vuln_whitelist[] = {
VULNWL_AMD(0x12, NO_MELTDOWN | NO_SSB | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
/* FAMILY_ANY must be last, otherwise 0x0f - 0x12 matches won't work */
- VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
- VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO),
+ VULNWL_AMD(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB),
+ VULNWL_HYGON(X86_FAMILY_ANY, NO_MELTDOWN | NO_L1TF | NO_MDS | NO_SWAPGS | NO_ITLB_MULTIHIT | NO_MMIO | NO_EIBRS_PBRSB),
/* Zhaoxin Family 7 */
VULNWL(CENTAUR, 7, X86_MODEL_ANY, NO_SPECTRE_V2 | NO_SWAPGS | NO_MMIO),
@@ -1341,8 +1341,16 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
!cpu_has(c, X86_FEATURE_AMD_SSB_NO))
setup_force_cpu_bug(X86_BUG_SPEC_STORE_BYPASS);
- if (ia32_cap & ARCH_CAP_IBRS_ALL)
+ /*
+ * AMD's AutoIBRS is equivalent to Intel's eIBRS - use the Intel feature
+ * flag and protect from vendor-specific bugs via the whitelist.
+ */
+ if ((ia32_cap & ARCH_CAP_IBRS_ALL) || cpu_has(c, X86_FEATURE_AUTOIBRS)) {
setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED);
+ if (!cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) &&
+ !(ia32_cap & ARCH_CAP_PBRSB_NO))
+ setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB);
+ }
if (!cpu_matches(cpu_vuln_whitelist, NO_MDS) &&
!(ia32_cap & ARCH_CAP_MDS_NO)) {
@@ -1404,11 +1412,6 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c)
setup_force_cpu_bug(X86_BUG_RETBLEED);
}
- if (cpu_has(c, X86_FEATURE_IBRS_ENHANCED) &&
- !cpu_matches(cpu_vuln_whitelist, NO_EIBRS_PBRSB) &&
- !(ia32_cap & ARCH_CAP_PBRSB_NO))
- setup_force_cpu_bug(X86_BUG_EIBRS_PBRSB);
-
if (cpu_matches(cpu_vuln_whitelist, NO_MELTDOWN))
return;
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 7/7] x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
` (5 preceding siblings ...)
2023-01-16 23:01 ` [PATCH v7 6/7] x86/cpu: Support AMD Automatic IBRS Kim Phillips
@ 2023-01-16 23:01 ` Kim Phillips
2023-01-17 21:25 ` [tip: x86/cpu] " tip-bot2 for Kim Phillips
2023-01-17 11:54 ` [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Borislav Petkov
7 siblings, 1 reply; 18+ messages in thread
From: Kim Phillips @ 2023-01-16 23:01 UTC (permalink / raw)
To: x86
Cc: Kim Phillips, Borislav Petkov, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, kvm, linux-doc, linux-kernel
Add the AMD Automatic IBRS feature bit to those being
propagated to the guest, and enable the guest EFER bit.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
---
arch/x86/kvm/cpuid.c | 2 +-
arch/x86/kvm/svm/svm.c | 3 +++
arch/x86/kvm/x86.c | 3 +++
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 9ba75ad9d976..293ef07b34c3 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -743,7 +743,7 @@ void kvm_set_cpu_caps(void)
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | 0 /* SmmPgCfgLock */ |
- F(NULL_SEL_CLR_BASE) | 0 /* PrefetchCtlMsr */
+ F(NULL_SEL_CLR_BASE) | F(AUTOIBRS) | 0 /* PrefetchCtlMsr */
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 9a194aa1a75a..60c7c880266b 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4969,6 +4969,9 @@ static __init int svm_hardware_setup(void)
tsc_aux_uret_slot = kvm_add_user_return_msr(MSR_TSC_AUX);
+ if (boot_cpu_has(X86_FEATURE_AUTOIBRS))
+ kvm_enable_efer_bits(EFER_AUTOIBRS);
+
/* Check for pause filtering support */
if (!boot_cpu_has(X86_FEATURE_PAUSEFILTER)) {
pause_filter_count = 0;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index da4bbd043a7b..8dd0cb230ef5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1685,6 +1685,9 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data)
static bool __kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer)
{
+ if (efer & EFER_AUTOIBRS && !guest_cpuid_has(vcpu, X86_FEATURE_AUTOIBRS))
+ return false;
+
if (efer & EFER_FFXSR && !guest_cpuid_has(vcpu, X86_FEATURE_FXSR_OPT))
return false;
--
2.34.1
^ permalink raw reply [flat|nested] 18+ messages in thread* [tip: x86/cpu] x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest
2023-01-16 23:01 ` [PATCH v7 7/7] x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest Kim Phillips
@ 2023-01-17 21:25 ` tip-bot2 for Kim Phillips
0 siblings, 0 replies; 18+ messages in thread
From: tip-bot2 for Kim Phillips @ 2023-01-17 21:25 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Kim Phillips, Borislav Petkov (AMD), x86, linux-kernel
The following commit has been merged into the x86/cpu branch of tip:
Commit-ID: eea6c7957ae7fa2bf90e44c91ef013176db69483
Gitweb: https://git.kernel.org/tip/eea6c7957ae7fa2bf90e44c91ef013176db69483
Author: Kim Phillips <kim.phillips@amd.com>
AuthorDate: Mon, 16 Jan 2023 17:01:59 -06:00
Committer: Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 17 Jan 2023 13:00:12 +01:00
x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest
Add the AMD Automatic IBRS feature bit to those being propagated to the guest,
and enable the guest EFER bit.
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20230116230159.1511393-8-kim.phillips@amd.com
---
arch/x86/kvm/cpuid.c | 2 +-
arch/x86/kvm/svm/svm.c | 3 +++
arch/x86/kvm/x86.c | 3 +++
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 56f00d9..6ce0203 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -743,7 +743,7 @@ void kvm_set_cpu_caps(void)
kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | 0 /* SmmPgCfgLock */ |
- F(NULL_SEL_CLR_BASE) | 0 /* PrefetchCtlMsr */
+ F(NULL_SEL_CLR_BASE) | F(AUTOIBRS) | 0 /* PrefetchCtlMsr */
);
if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 9a194aa..60c7c88 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4969,6 +4969,9 @@ static __init int svm_hardware_setup(void)
tsc_aux_uret_slot = kvm_add_user_return_msr(MSR_TSC_AUX);
+ if (boot_cpu_has(X86_FEATURE_AUTOIBRS))
+ kvm_enable_efer_bits(EFER_AUTOIBRS);
+
/* Check for pause filtering support */
if (!boot_cpu_has(X86_FEATURE_PAUSEFILTER)) {
pause_filter_count = 0;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index da4bbd0..8dd0cb2 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1685,6 +1685,9 @@ static int do_get_msr_feature(struct kvm_vcpu *vcpu, unsigned index, u64 *data)
static bool __kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer)
{
+ if (efer & EFER_AUTOIBRS && !guest_cpuid_has(vcpu, X86_FEATURE_AUTOIBRS))
+ return false;
+
if (efer & EFER_FFXSR && !guest_cpuid_has(vcpu, X86_FEATURE_FXSR_OPT))
return false;
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS
2023-01-16 23:01 [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Kim Phillips
` (6 preceding siblings ...)
2023-01-16 23:01 ` [PATCH v7 7/7] x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest Kim Phillips
@ 2023-01-17 11:54 ` Borislav Petkov
2023-01-20 1:40 ` Sean Christopherson
7 siblings, 1 reply; 18+ messages in thread
From: Borislav Petkov @ 2023-01-17 11:54 UTC (permalink / raw)
To: Paolo Bonzini, Sean Christopherson
Cc: kvm, Kim Phillips, x86, Boris Ostrovsky, Dave Hansen,
H. Peter Anvin, Ingo Molnar, Joao Martins, Jonathan Corbet,
Konrad Rzeszutek Wilk, Paolo Bonzini, Sean Christopherson,
Thomas Gleixner, David Woodhouse, Greg Kroah-Hartman,
Juergen Gross, Peter Zijlstra, Tony Luck, Tom Lendacky,
Alexey Kardashevskiy, linux-doc, linux-kernel
On Mon, Jan 16, 2023 at 05:01:52PM -0600, Kim Phillips wrote:
> Kim Phillips (7):
> x86/cpu, kvm: Add support for cpuid leaf 80000021/EAX (FeatureExt2Eax)
> x86/cpu, kvm: Add the NO_NESTED_DATA_BP feature
> x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing
> feature
> x86/cpu, kvm: Add the Null Selector Clears Base feature
> x86/cpu, kvm: Add the SMM_CTL MSR not present feature
> x86/cpu: Support AMD Automatic IBRS
> x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest
>
> Documentation/admin-guide/hw-vuln/spectre.rst | 6 ++--
> .../admin-guide/kernel-parameters.txt | 6 ++--
> arch/x86/include/asm/cpufeature.h | 7 +++--
> arch/x86/include/asm/cpufeatures.h | 11 +++++--
> arch/x86/include/asm/disabled-features.h | 3 +-
> arch/x86/include/asm/msr-index.h | 2 ++
> arch/x86/include/asm/required-features.h | 3 +-
> arch/x86/kernel/cpu/amd.c | 2 +-
> arch/x86/kernel/cpu/bugs.c | 20 ++++++++-----
> arch/x86/kernel/cpu/common.c | 22 +++++++++-----
> arch/x86/kvm/cpuid.c | 30 +++++++------------
> arch/x86/kvm/reverse_cpuid.h | 1 +
> arch/x86/kvm/svm/svm.c | 3 ++
> arch/x86/kvm/x86.c | 3 ++
> 14 files changed, 71 insertions(+), 48 deletions(-)
KVM folks,
I'm going to route this through the tip tree, along with the KVM bits.
Holler if we should do something else to avoid any potential conflicts.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS
2023-01-17 11:54 ` [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS Borislav Petkov
@ 2023-01-20 1:40 ` Sean Christopherson
2023-01-20 11:07 ` Borislav Petkov
0 siblings, 1 reply; 18+ messages in thread
From: Sean Christopherson @ 2023-01-20 1:40 UTC (permalink / raw)
To: Borislav Petkov
Cc: Paolo Bonzini, kvm, Kim Phillips, x86, Boris Ostrovsky,
Dave Hansen, H. Peter Anvin, Ingo Molnar, Joao Martins,
Jonathan Corbet, Konrad Rzeszutek Wilk, Thomas Gleixner,
David Woodhouse, Greg Kroah-Hartman, Juergen Gross,
Peter Zijlstra, Tony Luck, Tom Lendacky, Alexey Kardashevskiy,
linux-doc, linux-kernel
On Tue, Jan 17, 2023, Borislav Petkov wrote:
> On Mon, Jan 16, 2023 at 05:01:52PM -0600, Kim Phillips wrote:
>
> > Kim Phillips (7):
> > x86/cpu, kvm: Add support for cpuid leaf 80000021/EAX (FeatureExt2Eax)
> > x86/cpu, kvm: Add the NO_NESTED_DATA_BP feature
> > x86/cpu, kvm: Move the LFENCE_RDTSC / LFENCE always serializing
> > feature
> > x86/cpu, kvm: Add the Null Selector Clears Base feature
> > x86/cpu, kvm: Add the SMM_CTL MSR not present feature
> > x86/cpu: Support AMD Automatic IBRS
> > x86/cpu, kvm: Propagate the AMD Automatic IBRS feature to the guest
> >
> > Documentation/admin-guide/hw-vuln/spectre.rst | 6 ++--
> > .../admin-guide/kernel-parameters.txt | 6 ++--
> > arch/x86/include/asm/cpufeature.h | 7 +++--
> > arch/x86/include/asm/cpufeatures.h | 11 +++++--
> > arch/x86/include/asm/disabled-features.h | 3 +-
> > arch/x86/include/asm/msr-index.h | 2 ++
> > arch/x86/include/asm/required-features.h | 3 +-
> > arch/x86/kernel/cpu/amd.c | 2 +-
> > arch/x86/kernel/cpu/bugs.c | 20 ++++++++-----
> > arch/x86/kernel/cpu/common.c | 22 +++++++++-----
> > arch/x86/kvm/cpuid.c | 30 +++++++------------
> > arch/x86/kvm/reverse_cpuid.h | 1 +
> > arch/x86/kvm/svm/svm.c | 3 ++
> > arch/x86/kvm/x86.c | 3 ++
> > 14 files changed, 71 insertions(+), 48 deletions(-)
>
> KVM folks,
>
> I'm going to route this through the tip tree, along with the KVM bits.
>
> Holler if we should do something else to avoid any potential conflicts.
Sorry, completely missed this.
There will be a minor conflict in KVM's reverse_cpuid, but it's trivial to resolve.
I don't anticipate any other conflicts, so taking this through tip does seem like
the best option.
If possible, a new version to fix the bisection issues in patches 2 and 3 would
be nice, but again it's not a big deal. The breakage is very, very minor.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 0/7] x86/cpu, kvm: Support AMD Automatic IBRS
2023-01-20 1:40 ` Sean Christopherson
@ 2023-01-20 11:07 ` Borislav Petkov
0 siblings, 0 replies; 18+ messages in thread
From: Borislav Petkov @ 2023-01-20 11:07 UTC (permalink / raw)
To: Sean Christopherson
Cc: Paolo Bonzini, kvm, Kim Phillips, x86, Boris Ostrovsky,
Dave Hansen, H. Peter Anvin, Ingo Molnar, Joao Martins,
Jonathan Corbet, Konrad Rzeszutek Wilk, Thomas Gleixner,
David Woodhouse, Greg Kroah-Hartman, Juergen Gross,
Peter Zijlstra, Tony Luck, Tom Lendacky, Alexey Kardashevskiy,
linux-doc, linux-kernel
On Fri, Jan 20, 2023 at 01:40:13AM +0000, Sean Christopherson wrote:
> Sorry, completely missed this.
Nothing to be sorry for - can't notice everything in the flood. :)
> There will be a minor conflict in KVM's reverse_cpuid, but it's trivial to resolve.
> I don't anticipate any other conflicts, so taking this through tip does seem like
> the best option.
Ok, thx.
> If possible, a new version to fix the bisection issues in patches 2 and 3 would
> be nice, but again it's not a big deal. The breakage is very, very minor.
Yap, I've zapped them and their removal will take a bit to propagate to
linux-next.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 18+ messages in thread