* [RFC PATCH 01/19] KVM: TDX: Add support for find pending IRQ in a protected local APIC
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
@ 2025-02-28 8:50 ` Neeraj Upadhyay
2025-02-28 8:50 ` [RFC PATCH 02/19] KVM: x86: Assume timer IRQ was injected if APIC state is protected Neeraj Upadhyay
` (17 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:50 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Sean Christopherson <seanjc@google.com>
Add flag and hook to KVM's local APIC management to support determining
whether or not a TDX guest as a pending IRQ. For TDX vCPUs, the virtual
APIC page is owned by the TDX module and cannot be accessed by KVM. As a
result, registers that are virtualized by the CPU, e.g. PPR, cannot be
read or written by KVM. To deliver interrupts for TDX guests, KVM must
send an IRQ to the CPU on the posted interrupt notification vector. And
to determine if TDX vCPU has a pending interrupt, KVM must check if there
is an outstanding notification.
Return "no interrupt" in kvm_apic_has_interrupt() if the guest APIC is
protected to short-circuit the various other flows that try to pull an
IRQ out of the vAPIC, the only valid operation is querying _if_ an IRQ is
pending, KVM can't do anything based on _which_ IRQ is pending.
Intentionally omit sanity checks from other flows, e.g. PPR update, so as
not to degrade non-TDX guests with unnecessary checks. A well-behaved KVM
and userspace will never reach those flows for TDX guests, but reaching
them is not fatal if something does go awry.
Note, this doesn't handle interrupts that have been delivered to the vCPU
but not yet recognized by the core, i.e. interrupts that are sitting in
vmcs.GUEST_INTR_STATUS. Querying that state requires a SEAMCALL and will
be supported in a future patch.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
[Neeraj.Upadhyay@amd.com : Pick common ->guest_apic_protected bits]
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
- Not intended for review. Taken as a base patch for this RFC development.
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/kvm/irq.c | 3 +++
arch/x86/kvm/lapic.c | 3 +++
arch/x86/kvm/lapic.h | 2 ++
5 files changed, 10 insertions(+)
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index c35550581da0..5abc048aec07 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -114,6 +114,7 @@ KVM_X86_OP_OPTIONAL(pi_start_assignment)
KVM_X86_OP_OPTIONAL(apicv_pre_state_restore)
KVM_X86_OP_OPTIONAL(apicv_post_state_restore)
KVM_X86_OP_OPTIONAL_RET0(dy_apicv_has_pending_interrupt)
+KVM_X86_OP_OPTIONAL(protected_apic_has_interrupt)
KVM_X86_OP_OPTIONAL(set_hv_timer)
KVM_X86_OP_OPTIONAL(cancel_hv_timer)
KVM_X86_OP(setup_mce)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index f378cd43241c..97e95b88bc6f 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1830,6 +1830,7 @@ struct kvm_x86_ops {
void (*apicv_pre_state_restore)(struct kvm_vcpu *vcpu);
void (*apicv_post_state_restore)(struct kvm_vcpu *vcpu);
bool (*dy_apicv_has_pending_interrupt)(struct kvm_vcpu *vcpu);
+ bool (*protected_apic_has_interrupt)(struct kvm_vcpu *vcpu);
int (*set_hv_timer)(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc,
bool *expired);
diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c
index 63f66c51975a..f0644d0bbe11 100644
--- a/arch/x86/kvm/irq.c
+++ b/arch/x86/kvm/irq.c
@@ -100,6 +100,9 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v)
if (kvm_cpu_has_extint(v))
return 1;
+ if (lapic_in_kernel(v) && v->arch.apic->guest_apic_protected)
+ return static_call(kvm_x86_protected_apic_has_interrupt)(v);
+
return kvm_apic_has_interrupt(v) != -1; /* LAPIC */
}
EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt);
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index a009c94c26c2..8eefbaf4a456 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2966,6 +2966,9 @@ int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu)
if (!kvm_apic_present(vcpu))
return -1;
+ if (apic->guest_apic_protected)
+ return -1;
+
__apic_update_ppr(apic, &ppr);
return apic_has_interrupt_for_ppr(apic, ppr);
}
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 1a8553ebdb42..e33c969439f7 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -65,6 +65,8 @@ struct kvm_lapic {
bool sw_enabled;
bool irr_pending;
bool lvt0_in_nmi_mode;
+ /* Select registers in the vAPIC cannot be read/written. */
+ bool guest_apic_protected;
/* Number of bits set in ISR. */
s16 isr_count;
/* The highest vector set in ISR; if -1 - invalid, must scan ISR. */
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 02/19] KVM: x86: Assume timer IRQ was injected if APIC state is protected
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
2025-02-28 8:50 ` [RFC PATCH 01/19] KVM: TDX: Add support for find pending IRQ in a protected local APIC Neeraj Upadhyay
@ 2025-02-28 8:50 ` Neeraj Upadhyay
2025-02-28 8:50 ` [RFC PATCH 03/19] KVM: x86: Convert guest_apic_protected bool to an enum type Neeraj Upadhyay
` (16 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:50 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Sean Christopherson <seanjc@google.com>
If APIC state is protected, i.e. the vCPU is a TDX guest, assume a timer
IRQ was injected when deciding whether or not to busy wait in the "timer
advanced" path. The "real" vIRR is not readable/writable, so trying to
query for a pending timer IRQ will return garbage.
Note, TDX can scour the PIR if it wants to be more precise and skip the
"wait" call entirely.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
- Not intended for review. Taken as a base patch for this RFC development.
arch/x86/kvm/lapic.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 8eefbaf4a456..65f69537c105 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1797,8 +1797,17 @@ static void apic_update_lvtt(struct kvm_lapic *apic)
static bool lapic_timer_int_injected(struct kvm_vcpu *vcpu)
{
struct kvm_lapic *apic = vcpu->arch.apic;
- u32 reg = kvm_lapic_get_reg(apic, APIC_LVTT);
+ u32 reg;
+ /*
+ * Assume a timer IRQ was "injected" if the APIC is protected. KVM's
+ * copy of the vIRR is bogus, it's the responsibility of the caller to
+ * precisely check whether or not a timer IRQ is pending.
+ */
+ if (apic->guest_apic_protected)
+ return true;
+
+ reg = kvm_lapic_get_reg(apic, APIC_LVTT);
if (kvm_apic_hw_enabled(apic)) {
int vec = reg & APIC_VECTOR_MASK;
void *bitmap = apic->regs + APIC_ISR;
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 03/19] KVM: x86: Convert guest_apic_protected bool to an enum type
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
2025-02-28 8:50 ` [RFC PATCH 01/19] KVM: TDX: Add support for find pending IRQ in a protected local APIC Neeraj Upadhyay
2025-02-28 8:50 ` [RFC PATCH 02/19] KVM: x86: Assume timer IRQ was injected if APIC state is protected Neeraj Upadhyay
@ 2025-02-28 8:50 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 04/19] x86/cpufeatures: Add Secure AVIC CPU Feature Neeraj Upadhyay
` (15 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:50 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Convert guest_apic_protected lapic struct member to an enum..
This allows to categorize guest APIC state protection according
to the KVM interrupt delivery mechanism.
This is used to distinguish between SNP Secure AVIC's interrupt
injection-based interrupt delivery and TDX's posted interrupt
delivery mechanism. Use value 0 to indicate unprotected APIC
so that functions like kvm_cpu_has_interrupt(), which require
KVM to call an arch-specific callback to determine whether
there are any interrupts that need to be delivered to the vCPU,
can still use non-zero guest_apic_protected check. Subsequent
patches for Secure AVIC-specific interrupt injection checks
will need to use specific guest_apic_protected value.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/lapic.h | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index e33c969439f7..c9ef9bce438b 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -55,6 +55,12 @@ struct kvm_timer {
bool hv_timer_in_use;
};
+enum kvm_apic_protection {
+ APIC_STATE_UNPROTECTED,
+ APIC_STATE_PROTECTED_POSTED_INTR,
+ APIC_STATE_PROTECTED_INJECTED_INTR,
+};
+
struct kvm_lapic {
unsigned long base_address;
struct kvm_io_device dev;
@@ -66,7 +72,7 @@ struct kvm_lapic {
bool irr_pending;
bool lvt0_in_nmi_mode;
/* Select registers in the vAPIC cannot be read/written. */
- bool guest_apic_protected;
+ enum kvm_apic_protection guest_apic_protected;
/* Number of bits set in ISR. */
s16 isr_count;
/* The highest vector set in ISR; if -1 - invalid, must scan ISR. */
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 04/19] x86/cpufeatures: Add Secure AVIC CPU Feature
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (2 preceding siblings ...)
2025-02-28 8:50 ` [RFC PATCH 03/19] KVM: x86: Convert guest_apic_protected bool to an enum type Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 05/19] KVM: SVM: Add support for Secure AVIC capability in KVM Neeraj Upadhyay
` (14 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Add CPU feature detection for Secure AVIC. The Secure AVIC feature
provides hardware acceleration for performance sensitive APIC
accesses and support for managing guest owned APIC state for
SEV-SNP guests.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index 2ba5c11941ee..a63ca1b34b3a 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -452,6 +452,7 @@
#define X86_FEATURE_V_TSC_AUX (19*32+ 9) /* Virtual TSC_AUX */
#define X86_FEATURE_SME_COHERENT (19*32+10) /* AMD hardware-enforced cache coherency */
#define X86_FEATURE_DEBUG_SWAP (19*32+14) /* "debug_swap" AMD SEV-ES full debug state swap support */
+#define X86_FEATURE_SECURE_AVIC (19*32+26) /* Secure AVIC */
#define X86_FEATURE_ALLOWED_SEV_FEATURES (19*32+27) /* Allowed SEV Features */
#define X86_FEATURE_SVSM (19*32+28) /* "svsm" SVSM present */
#define X86_FEATURE_HV_INUSE_WR_ALLOWED (19*32+30) /* Allow Write to in-use hypervisor-owned pages */
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 05/19] KVM: SVM: Add support for Secure AVIC capability in KVM
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (3 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 04/19] x86/cpufeatures: Add Secure AVIC CPU Feature Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 06/19] KVM: SVM: Initialize apic protected state for SAVIC guests Neeraj Upadhyay
` (13 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Add support to KVM for determining if a system is capable of
supporting Secure AVIC feature.
Secure AVIC feature support is determined based on:
- secure_avic module parameter is set.
- X86_FEATURE_SECURE_AVIC CPU Feature bit is set.
- SNP feature is supported.
Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/include/asm/svm.h | 1 +
arch/x86/kvm/svm/sev.c | 9 +++++++++
2 files changed, 10 insertions(+)
diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index 9391eca5412e..f81b417fe836 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -292,6 +292,7 @@ static_assert((X2AVIC_MAX_PHYSICAL_ID & AVIC_PHYSICAL_MAX_INDEX_MASK) == X2AVIC_
#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)
#define SVM_SEV_FEAT_ALTERNATE_INJECTION BIT(4)
#define SVM_SEV_FEAT_DEBUG_SWAP BIT(5)
+#define SVM_SEV_FEAT_SECURE_AVIC BIT(16)
#define SVM_SEV_FEAT_ALLOWED_SEV_FEATURES BIT_ULL(63)
#define SVM_SEV_FEAT_INT_INJ_MODES \
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index bf4e85e11a7b..82209cd56ec6 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -58,6 +58,10 @@ static bool sev_es_debug_swap_enabled = true;
module_param_named(debug_swap, sev_es_debug_swap_enabled, bool, 0444);
static u64 sev_supported_vmsa_features;
+/* enable/disable SEV-SNP Secure AVIC support */
+bool sev_snp_savic_enabled = true;
+module_param_named(secure_avic, sev_snp_savic_enabled, bool, 0444);
+
#define AP_RESET_HOLD_NONE 0
#define AP_RESET_HOLD_NAE_EVENT 1
#define AP_RESET_HOLD_MSR_PROTO 2
@@ -2962,6 +2966,8 @@ void __init sev_set_cpu_caps(void)
kvm_cpu_cap_set(X86_FEATURE_SEV_SNP);
kvm_caps.supported_vm_types |= BIT(KVM_X86_SNP_VM);
}
+ if (sev_snp_savic_enabled)
+ kvm_cpu_cap_set(X86_FEATURE_SECURE_AVIC);
}
void __init sev_hardware_setup(void)
@@ -3082,6 +3088,9 @@ void __init sev_hardware_setup(void)
!cpu_feature_enabled(X86_FEATURE_NO_NESTED_DATA_BP))
sev_es_debug_swap_enabled = false;
+ if (!sev_snp_supported || !cpu_feature_enabled(X86_FEATURE_SECURE_AVIC))
+ sev_snp_savic_enabled = false;
+
sev_supported_vmsa_features = 0;
if (sev_es_debug_swap_enabled)
sev_supported_vmsa_features |= SVM_SEV_FEAT_DEBUG_SWAP;
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 06/19] KVM: SVM: Initialize apic protected state for SAVIC guests
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (4 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 05/19] KVM: SVM: Add support for Secure AVIC capability in KVM Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 07/19] KVM: SEV: Do not intercept SECURE_AVIC_CONTROL MSR Neeraj Upadhyay
` (12 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Initialize lapic's guest_apic_protected for Secure AVIC guests.
This is only an initialization commit and actual support for
creating Secure AVIC enabled guests and injecting interrupts
would be added in later commits.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/svm.c | 3 +++
arch/x86/kvm/svm/svm.h | 5 +++++
2 files changed, 8 insertions(+)
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 456d841298f7..d4191c0a0133 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -1442,6 +1442,9 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
if (!vmcb01_page)
goto out;
+ if (sev_savic_active(vcpu->kvm))
+ vcpu->arch.apic->guest_apic_protected = APIC_STATE_PROTECTED_INJECTED_INTR;
+
if (sev_es_guest(vcpu->kvm)) {
/*
* SEV-ES guests require a separate VMSA page used to contain
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 9d7cdb8fbf87..7cde221e477e 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -756,6 +756,10 @@ void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order);
void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end);
int sev_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn);
+static inline bool sev_savic_active(struct kvm *kvm)
+{
+ return to_kvm_sev_info(kvm)->vmsa_features & SVM_SEV_FEAT_SECURE_AVIC;
+}
#else
static inline struct page *snp_safe_alloc_page_node(int node, gfp_t gfp)
{
@@ -786,6 +790,7 @@ static inline int sev_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn)
{
return 0;
}
+static inline bool sev_savic_active(struct kvm *kvm) { return false; }
#endif
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 07/19] KVM: SEV: Do not intercept SECURE_AVIC_CONTROL MSR
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (5 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 06/19] KVM: SVM: Initialize apic protected state for SAVIC guests Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 08/19] KVM: SVM/SEV/X86: Secure AVIC: Add support to inject interrupts Neeraj Upadhyay
` (11 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Kishon Vijay Abraham I <kvijayab@amd.com>
The SECURE_AVIC_CONTROL MSR (0xc0010138) holds the GPA of the APIC
backing page and bitfields to enable Secure AVIC and NMI. This MSR
is populated by the guest and the hypervisor should not intercept it
so that the guest can properly set the MSR. Disable intercepting the
SECURE_AVIC_CONTROL MSR for Secure AVIC enabled guests.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/include/asm/msr-index.h | 2 ++
arch/x86/kvm/svm/sev.c | 3 +++
arch/x86/kvm/svm/svm.c | 1 +
arch/x86/kvm/svm/svm.h | 2 +-
4 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 3ae84c3b8e6d..6fb734228726 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -683,6 +683,8 @@
#define MSR_AMD64_SNP_RESV_BIT 18
#define MSR_AMD64_SNP_RESERVED_MASK GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
+#define MSR_AMD64_SECURE_AVIC_CONTROL 0xc0010138
+
#define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f
#define MSR_AMD64_RMP_BASE 0xc0010132
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 82209cd56ec6..6313679a65b8 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4573,6 +4573,9 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm)
/* Clear intercepts on selected MSRs */
set_msr_interception(vcpu, svm->msrpm, MSR_EFER, 1, 1);
set_msr_interception(vcpu, svm->msrpm, MSR_IA32_CR_PAT, 1, 1);
+
+ if (sev_savic_active(vcpu->kvm))
+ set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_SECURE_AVIC_CONTROL, 1, 1);
}
void sev_init_vmcb(struct vcpu_svm *svm)
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index d4191c0a0133..d00ae58c0b0a 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -110,6 +110,7 @@ static const struct svm_direct_access_msrs {
{ .index = MSR_EFER, .always = false },
{ .index = MSR_IA32_CR_PAT, .always = false },
{ .index = MSR_AMD64_SEV_ES_GHCB, .always = true },
+ { .index = MSR_AMD64_SECURE_AVIC_CONTROL, .always = false },
{ .index = MSR_TSC_AUX, .always = false },
{ .index = X2APIC_MSR(APIC_ID), .always = false },
{ .index = X2APIC_MSR(APIC_LVR), .always = false },
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 7cde221e477e..e855f101e60f 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -44,7 +44,7 @@ static inline struct page *__sme_pa_to_page(unsigned long pa)
#define IOPM_SIZE PAGE_SIZE * 3
#define MSRPM_SIZE PAGE_SIZE * 2
-#define MAX_DIRECT_ACCESS_MSRS 48
+#define MAX_DIRECT_ACCESS_MSRS 49
#define MSRPM_OFFSETS 32
extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly;
extern bool npt_enabled;
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 08/19] KVM: SVM/SEV/X86: Secure AVIC: Add support to inject interrupts
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (6 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 07/19] KVM: SEV: Do not intercept SECURE_AVIC_CONTROL MSR Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 09/19] KVM: SVM/SEV/X86: Secure AVIC: Add hypervisor side IPI Delivery Support Neeraj Upadhyay
` (10 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Add support to inject interrupts from emulated devices/para-virtualized
devices for Secure AVIC guests. Secure AVIC requires RequestedIRR and
UpdateIRR VMCB fields to be set before VMRUN for interrupt injection.
Secure AVIC allows multiple interrupts to be injected at the same time. So
on interrupt injection, the entire contents of APIC_IRR in host APIC
backing page are copied to RequestedIRR. As guest PPR state is not visible
in KVM, all pending interrupts in host APIC_IRR are considered as
injectable. Secure AVIC HW handles re-injection itself so no explicit
handling by KVM is required.
Secure AVIC does not require an IRQ window as hardware manages interrupt
delivery to the guest and can detect if the guest is in a state to accept
interrupts. So, short-circuit interrupt_allowed() and enable_irq_window()
ops for Secure AVIC.
Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/include/asm/svm.h | 8 ++++++--
arch/x86/kvm/irq.c | 3 +++
arch/x86/kvm/lapic.c | 13 +++++++++----
arch/x86/kvm/svm/sev.c | 39 ++++++++++++++++++++++++++++++++++++++
arch/x86/kvm/svm/svm.c | 13 +++++++++++++
arch/x86/kvm/svm/svm.h | 5 ++++-
arch/x86/kvm/x86.c | 12 +++++++++++-
7 files changed, 85 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index f81b417fe836..59253e3b28f3 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -159,9 +159,13 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
u64 avic_physical_id; /* Offset 0xf8 */
u8 reserved_7[8];
u64 vmsa_pa; /* Used for an SEV-ES guest */
- u8 reserved_8[40];
+ u8 reserved_8[36];
+ u8 update_irr; /* Offset 0x134 */
+ u8 reserved_9[3];
u64 allowed_sev_features; /* Offset 0x138 */
- u8 reserved_9[672];
+ u8 reserved_10[16];
+ u32 requested_irr[8]; /* Offset 0x150 */
+ u8 reserved_11[624];
/*
* Offset 0x3e0, 32 bytes reserved
* for use by hypervisor/software.
diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c
index f0644d0bbe11..fbfd897ea412 100644
--- a/arch/x86/kvm/irq.c
+++ b/arch/x86/kvm/irq.c
@@ -47,6 +47,9 @@ static int pending_userspace_extint(struct kvm_vcpu *v)
*/
int kvm_cpu_has_extint(struct kvm_vcpu *v)
{
+ if (v->arch.apic->guest_apic_protected)
+ return 0;
+
/*
* FIXME: interrupt.injected represents an interrupt whose
* side-effects have already been applied (e.g. bit from IRR
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 65f69537c105..7b2ee5263644 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2975,11 +2975,16 @@ int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu)
if (!kvm_apic_present(vcpu))
return -1;
- if (apic->guest_apic_protected)
+ switch (apic->guest_apic_protected) {
+ case APIC_STATE_PROTECTED_POSTED_INTR:
return -1;
-
- __apic_update_ppr(apic, &ppr);
- return apic_has_interrupt_for_ppr(apic, ppr);
+ case APIC_STATE_PROTECTED_INJECTED_INTR:
+ return apic_search_irr(apic);
+ case APIC_STATE_UNPROTECTED:
+ default:
+ __apic_update_ppr(apic, &ppr);
+ return apic_has_interrupt_for_ppr(apic, ppr);
+ }
}
EXPORT_SYMBOL_GPL(kvm_apic_has_interrupt);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 6313679a65b8..080b71ade88d 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -34,6 +34,7 @@
#include "svm_ops.h"
#include "cpuid.h"
#include "trace.h"
+#include "lapic.h"
#define GHCB_VERSION_MAX 2ULL
#define GHCB_VERSION_DEFAULT 2ULL
@@ -4986,3 +4987,41 @@ int sev_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn)
return level;
}
+
+void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected)
+{
+ struct kvm_lapic *apic;
+ bool has_interrupts;
+ int vec, vec_pos;
+ u32 val;
+ int i;
+
+ /* Secure AVIC HW takes care of re-injection */
+ if (reinjected)
+ return;
+
+ apic = svm->vcpu.arch.apic;
+ has_interrupts = false;
+
+ for (i = 0; i < ARRAY_SIZE(svm->vmcb->control.requested_irr); i++) {
+ val = __kvm_lapic_get_reg(apic->regs, APIC_IRR + i * 0x10);
+ if (!val)
+ continue;
+ has_interrupts = true;
+ svm->vmcb->control.requested_irr[i] |= val;
+ do {
+ vec_pos = __ffs(val);
+ vec = (i << 5) + vec_pos;
+ kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR);
+ val = val & ~BIT(vec_pos);
+ } while (val);
+ }
+
+ if (has_interrupts)
+ svm->vmcb->control.update_irr |= BIT(0);
+}
+
+bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu)
+{
+ return kvm_apic_has_interrupt(vcpu) != -1;
+}
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index d00ae58c0b0a..7cfd6e916c74 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -50,6 +50,8 @@
#include "svm.h"
#include "svm_ops.h"
+#include "lapic.h"
+
#include "kvm_onhyperv.h"
#include "svm_onhyperv.h"
@@ -3679,6 +3681,9 @@ static void svm_inject_irq(struct kvm_vcpu *vcpu, bool reinjected)
struct vcpu_svm *svm = to_svm(vcpu);
u32 type;
+ if (sev_savic_active(vcpu->kvm))
+ return sev_savic_set_requested_irr(svm, reinjected);
+
if (vcpu->arch.interrupt.soft) {
if (svm_update_soft_interrupt_rip(vcpu))
return;
@@ -3860,6 +3865,9 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection)
{
struct vcpu_svm *svm = to_svm(vcpu);
+ if (sev_savic_active(vcpu->kvm))
+ return 1;
+
if (svm->nested.nested_run_pending)
return -EBUSY;
@@ -3880,6 +3888,9 @@ static void svm_enable_irq_window(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
+ if (sev_savic_active(vcpu->kvm))
+ return;
+
/*
* In case GIF=0 we can't rely on the CPU to tell us when GIF becomes
* 1, because that's a separate STGI/VMRUN intercept. The next time we
@@ -5092,6 +5103,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
.apicv_post_state_restore = avic_apicv_post_state_restore,
.required_apicv_inhibits = AVIC_REQUIRED_APICV_INHIBITS,
+ .protected_apic_has_interrupt = sev_savic_has_pending_interrupt,
+
.get_exit_info = svm_get_exit_info,
.get_entry_info = svm_get_entry_info,
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index e855f101e60f..f70c161ad352 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -760,6 +760,8 @@ static inline bool sev_savic_active(struct kvm *kvm)
{
return to_kvm_sev_info(kvm)->vmsa_features & SVM_SEV_FEAT_SECURE_AVIC;
}
+void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected);
+bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu);
#else
static inline struct page *snp_safe_alloc_page_node(int node, gfp_t gfp)
{
@@ -791,7 +793,8 @@ static inline int sev_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn)
return 0;
}
static inline bool sev_savic_active(struct kvm *kvm) { return false; }
-
+static inline void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected) {}
+static inline bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu) { return false; }
#endif
/* vmenter.S */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 8e77e61d4fbd..a9bd774baa4e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -10426,7 +10426,17 @@ static int kvm_check_and_inject_events(struct kvm_vcpu *vcpu,
if (r < 0)
goto out;
if (r) {
- int irq = kvm_cpu_get_interrupt(vcpu);
+ int irq;
+
+ /*
+ * Do not ack the interrupt here for APIC_STATE_PROTECTED_INJECTED_INTR.
+ * ->inject_irq reads the APIC_IRR state and clears it.
+ */
+ if (vcpu->arch.apic->guest_apic_protected ==
+ APIC_STATE_PROTECTED_INJECTED_INTR)
+ irq = kvm_apic_has_interrupt(vcpu);
+ else
+ irq = kvm_cpu_get_interrupt(vcpu);
if (!WARN_ON_ONCE(irq == -1)) {
kvm_queue_interrupt(vcpu, irq, false);
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 09/19] KVM: SVM/SEV/X86: Secure AVIC: Add hypervisor side IPI Delivery Support
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (7 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 08/19] KVM: SVM/SEV/X86: Secure AVIC: Add support to inject interrupts Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 10/19] KVM: SVM: Secure AVIC: Do not inject "Exceptions" for Secure AVIC Neeraj Upadhyay
` (9 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Secure AVIC hardware only accelerates self IPIs. For cross-vCPU IPI,
source vCPU updates the APIC_IRR of destination vCPU and then issues
VMGEXIT with type "SVM_EXIT_MSR" to propagate APIC_ICR write to the
hypervisor.
Hypervisor then examines the ICR data and sends doorbell to running vCPUs
using AVIC Doorbell MSR or wakes up a blocking vCPU. Hypervisor then
resumes the vCPU which issued the VMGEXIT.
Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 216 ++++++++++++++++++++++++++++++++++++++++-
arch/x86/kvm/svm/svm.h | 2 +
2 files changed, 217 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 080b71ade88d..d8413c7f4832 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3496,6 +3496,89 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)
struct svm_cpu_data *sd = per_cpu_ptr(&svm_data, cpu);
unsigned int asid = sev_get_asid(svm->vcpu.kvm);
+ /*
+ * It should be safe to clear sev_savic_has_pending_ipi here.
+ *
+ * Following are the scenarios possible:
+ *
+ * Scenario 1: sev_savic_has_pending_ipi is set before hlt exit of the
+ * target vCPU.
+ *
+ * Source vCPU Target vCPU
+ *
+ * 1. Set APIC_IRR of target
+ * vCPU.
+ *
+ * 2. VMGEXIT
+ *
+ * 3. Set ...has_pending_ipi
+ *
+ * savic_handle_icr_write()
+ * ..._has_pending_ipi = true
+ *
+ * 4. avic_ring_doorbell()
+ * - VS -
+ *
+ * 4. VMEXIT
+ *
+ * 5. ..._has_pending_ipi = false
+ *
+ * 6. VM entry
+ *
+ * 7. hlt exit
+ *
+ * In this case, any VM exit taken by target vCPU before hlt exit
+ * clears sev_savic_has_pending_ipi. On hlt exit, idle halt intercept
+ * would find the V_INTR set and skip hlt exit.
+ *
+ * Scenario 2: sev_savic_has_pending_ipi is set when target vCPU
+ * has taken hlt exit.
+ *
+ * Source vCPU Target vCPU
+ *
+ * 1. hlt exit
+ *
+ * 2. Set ...has_pending_ipi
+ * 3. kvm_vcpu_has_events() returns true
+ * and VM is reentered.
+ *
+ * vcpu_block()
+ * kvm_arch_vcpu_runnable()
+ * kvm_vcpu_has_events()
+ * <return true as ..._has_pending_ipi
+ * is set>
+ *
+ * 4. On VM entry, APIC_IRR state is re-evaluated
+ * and V_INTR is set and interrupt is delivered
+ * to vCPU.
+ *
+ *
+ * Scenario 3: sev_savic_has_pending_ipi is set while halt exit is happening:
+ *
+ *
+ * Source vCPU Target vCPU
+ *
+ * 1. hlt
+ * Hardware check V_INTR to determine
+ * if hlt exit need to be taken. No other
+ * exit such as intr exit can be taken
+ * while this sequence is being executed.
+ *
+ * 2. Set APIC_IRR of target vCPU.
+ *
+ * 3. Set ...has_pending_ipi
+ * 4. hlt exit taken.
+ *
+ * 5. ...has_pending_ipi being set is observed
+ * by target vCPU and the vCPU is resumed.
+ *
+ * In this scenario, hardware ensures that target vCPU does not take any exit
+ * between checking V_INTR state and halt exit. So, sev_savic_has_pending_ipi
+ * remains set when vCPU takes hlt exit.
+ */
+ if (READ_ONCE(svm->sev_savic_has_pending_ipi))
+ WRITE_ONCE(svm->sev_savic_has_pending_ipi, false);
+
/* Assign the asid allocated with this SEV guest */
svm->asid = asid;
@@ -4303,6 +4386,129 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
return 0;
}
+static void savic_handle_icr_write(struct kvm_vcpu *kvm_vcpu, u64 icr)
+{
+ struct kvm *kvm = kvm_vcpu->kvm;
+ struct kvm_vcpu *vcpu;
+ u32 icr_low, icr_high;
+ bool in_guest_mode;
+ unsigned long i;
+
+ icr_low = lower_32_bits(icr);
+ icr_high = upper_32_bits(icr);
+
+ /*
+ * TODO: Instead of scanning all the vCPUS, get fastpath working which should
+ * look similar to avic_kick_target_vcpus_fast().
+ */
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!kvm_apic_match_dest(vcpu, kvm_vcpu->arch.apic, icr_low & APIC_SHORT_MASK,
+ icr_high, icr_low & APIC_DEST_MASK))
+ continue;
+
+ /*
+ * Setting sev_savic_has_pending_ipi could result in a spurious
+ * wakeup from hlt (as kvm_cpu_has_interrupt() would return true)
+ * if destination CPU is inside guest and guest does a halt exit
+ * after handling the IPI. sev_savic_has_pending_ipi gets cleared
+ * on vm entry, so there can be at most one spurious wakeup per IPI.
+ * For vcpu->mode == IN_GUEST_MODE, sev_savic_has_pending_ipi need
+ * to be set to handle the case where destination vCPU has taken
+ * halt exit and the source CPU has not observed vcpu->mode !=
+ * IN_GUEST_MODE.
+ */
+ WRITE_ONCE(to_svm(vcpu)->sev_savic_has_pending_ipi, true);
+ /* Order sev_savic_has_pending_ipi write and vcpu->mode read. */
+ smp_mb();
+ /* Pairs with smp_store_release in vcpu_enter_guest. */
+ in_guest_mode = (smp_load_acquire(&vcpu->mode) == IN_GUEST_MODE);
+ if (in_guest_mode) {
+ /*
+ * Signal the doorbell to tell hardware to inject the IRQ.
+ *
+ * If the vCPU exits the guest before the doorbell chimes,
+ * below memory ordering guarantees that destination vCPU
+ * observes sev_savic_has_pending_ipi == true before
+ * blocking.
+ *
+ * Src-CPU Dest-CPU
+ *
+ * savic_handle_icr_write()
+ * sev_savic_has_pending_ipi = true
+ * smp_mb()
+ * smp_load_acquire(&vcpu->mode)
+ *
+ * - VS -
+ * vcpu->mode = OUTSIDE_GUEST_MODE
+ * __kvm_emulate_halt()
+ * kvm_cpu_has_interrupt()
+ * smp_mb()
+ * if (sev_savic_has_pending_ipi)
+ * return true;
+ *
+ * [S1]
+ * sev_savic_has_pending_ipi = true
+ *
+ * SMP_MB
+ *
+ * [L1]
+ * vcpu->mode
+ * [S2]
+ * vcpu->mode = OUTSIDE_GUEST_MODE
+ *
+ *
+ * SMP_MB
+ *
+ * [L2] sev_savic_has_pending_ipi == true
+ *
+ * exists (L1=IN_GUEST_MODE /\ L2=false)
+ *
+ * Above condition does not exit. So, if source CPU observes vcpu->mode
+ * = IN_GUEST_MODE (L1), sev_savic_has_pending_ipi load by destination CPU
+ * (L2) should observe the store (S1) from source CPU.
+ */
+ avic_ring_doorbell(vcpu);
+ } else {
+ /*
+ * Wake the vCPU if it was blocking.
+ *
+ * Memory ordering is provided by smp_mb() in rcuwait_wake_up() on
+ * source CPU and smp_mb() in set_current_state() inside
+ * kvm_vcpu_block() on dest CPU.
+ */
+ kvm_vcpu_kick(vcpu);
+ }
+ }
+}
+
+static bool savic_handle_msr_exit(struct kvm_vcpu *vcpu)
+{
+ u32 msr, reg;
+
+ msr = kvm_rcx_read(vcpu);
+ reg = (msr - APIC_BASE_MSR) << 4;
+
+ switch (reg) {
+ case APIC_ICR:
+ /*
+ * Only APIC_ICR wrmsr requires special handling
+ * for Secure AVIC guests to wake up destination
+ * vCPUs.
+ */
+ if (to_svm(vcpu)->vmcb->control.exit_info_1) {
+ u64 data = kvm_read_edx_eax(vcpu);
+
+ savic_handle_icr_write(vcpu, data);
+ return true;
+ }
+ break;
+ default:
+ break;
+ }
+
+ return false;
+}
+
int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -4445,6 +4651,11 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
control->exit_info_1, control->exit_info_2);
ret = -EINVAL;
break;
+ case SVM_EXIT_MSR:
+ if (sev_savic_active(vcpu->kvm) && savic_handle_msr_exit(vcpu))
+ return 1;
+
+ fallthrough;
default:
ret = svm_invoke_exit_handler(vcpu, exit_code);
}
@@ -5023,5 +5234,8 @@ void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected)
bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu)
{
- return kvm_apic_has_interrupt(vcpu) != -1;
+ /* See memory ordering description in savic_handle_icr_write(). */
+ smp_mb();
+ return READ_ONCE(to_svm(vcpu)->sev_savic_has_pending_ipi) ||
+ kvm_apic_has_interrupt(vcpu) != -1;
}
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index f70c161ad352..62e3581b7d31 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -327,6 +327,8 @@ struct vcpu_svm {
/* Guest GIF value, used when vGIF is not enabled */
bool guest_gif;
+
+ bool sev_savic_has_pending_ipi;
};
struct svm_cpu_data {
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 10/19] KVM: SVM: Secure AVIC: Do not inject "Exceptions" for Secure AVIC
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (8 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 09/19] KVM: SVM/SEV/X86: Secure AVIC: Add hypervisor side IPI Delivery Support Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 11/19] KVM: SVM/SEV: Do not intercept exceptions for Secure AVIC guest Neeraj Upadhyay
` (8 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Kishon Vijay Abraham I <kvijayab@amd.com>
Secure AVIC does not support injecting "Exceptions" from hypervisor.
Return from svm_inject_exception() for Secure AVIC.
HW takes care of delivering exceptions initiated by guest as well as
re-injecting exceptions initiated by guest (in case there's an intercept
before delivering the exceptions). However exceptions cannot be
explicitly injected from Hypervisor when Secure AVIC is enabled.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/svm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 7cfd6e916c74..58733b63bcd7 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -463,6 +463,9 @@ static void svm_inject_exception(struct kvm_vcpu *vcpu)
struct kvm_queued_exception *ex = &vcpu->arch.exception;
struct vcpu_svm *svm = to_svm(vcpu);
+ if (sev_savic_active(vcpu->kvm))
+ return;
+
kvm_deliver_exception_payload(vcpu, ex);
if (kvm_exception_is_soft(ex->vector) &&
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 11/19] KVM: SVM/SEV: Do not intercept exceptions for Secure AVIC guest
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (9 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 10/19] KVM: SVM: Secure AVIC: Do not inject "Exceptions" for Secure AVIC Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 12/19] KVM: SVM/SEV: Secure AVIC: Set VGIF in VMSA area Neeraj Upadhyay
` (7 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Exceptions cannot be explicitly injected from Hypervisor to Secure
AVIC guests. So, clear exceptions intercepts so that all exceptions
are routed directly to such guests.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index d8413c7f4832..5106afc40cc8 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4786,8 +4786,17 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm)
set_msr_interception(vcpu, svm->msrpm, MSR_EFER, 1, 1);
set_msr_interception(vcpu, svm->msrpm, MSR_IA32_CR_PAT, 1, 1);
- if (sev_savic_active(vcpu->kvm))
+ if (sev_savic_active(vcpu->kvm)) {
set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_SECURE_AVIC_CONTROL, 1, 1);
+
+ /* Clear all exception intercepts. */
+ clr_exception_intercept(svm, PF_VECTOR);
+ clr_exception_intercept(svm, UD_VECTOR);
+ clr_exception_intercept(svm, MC_VECTOR);
+ clr_exception_intercept(svm, AC_VECTOR);
+ clr_exception_intercept(svm, DB_VECTOR);
+ clr_exception_intercept(svm, GP_VECTOR);
+ }
}
void sev_init_vmcb(struct vcpu_svm *svm)
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 12/19] KVM: SVM/SEV: Secure AVIC: Set VGIF in VMSA area
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (10 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 11/19] KVM: SVM/SEV: Do not intercept exceptions for Secure AVIC guest Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 13/19] KVM: SVM/SEV: Secure AVIC: Enable NMI support Neeraj Upadhyay
` (6 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Kishon Vijay Abraham I <kvijayab@amd.com>
Secure AVIC requires VGIF in VMSA to be set in order for interrupts to
be delivered to the guest. Usually for enabling GIF, VGIF field (bit 9
of VINTR_CTRL in VMCB) is set. However Secure AVIC ignores the VGIF
field in VMCB and requires VGIF field in VINTR_CTRL of VMSA to be set.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 5106afc40cc8..07a8a0c09382 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -854,6 +854,8 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
save->dr6 = svm->vcpu.arch.dr6;
save->sev_features = sev->vmsa_features;
+ if (sev_savic_active(vcpu->kvm))
+ save->vintr_ctrl |= V_GIF_MASK;
/*
* Skip FPU and AVX setup with KVM_SEV_ES_INIT to avoid
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 13/19] KVM: SVM/SEV: Secure AVIC: Enable NMI support
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (11 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 12/19] KVM: SVM/SEV: Secure AVIC: Set VGIF in VMSA area Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 14/19] KVM: SVM/SEV: Add SVM_VMGEXIT_SECURE_AVIC GHCB protocol event handling Neeraj Upadhyay
` (5 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Kishon Vijay Abraham I <kvijayab@amd.com>
Secure AVIC only allows vNMI flow for injecting NMI interrupts to the
guest. Also, Secure AVIC hardware manages NMI delivery to the guest and
can detect if the guest is in a state to accept NMI. So, update NMI
injection code flow for Secure AVIC to inject V_NMI and allow NMI
injection if there is no V_NMI pending. In addition, Secure AVIC
requires V_NMI_ENABLE in VINTR_CTRL field of VMSA to be set. Set
V_NMI_ENABLE in VINTR_CTRL field of VMSA.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 2 +-
arch/x86/kvm/svm/svm.c | 56 ++++++++++++++++++++++++++----------------
2 files changed, 36 insertions(+), 22 deletions(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 07a8a0c09382..40314c4086c2 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -855,7 +855,7 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
save->sev_features = sev->vmsa_features;
if (sev_savic_active(vcpu->kvm))
- save->vintr_ctrl |= V_GIF_MASK;
+ save->vintr_ctrl |= V_GIF_MASK | V_NMI_ENABLE_MASK;
/*
* Skip FPU and AVX setup with KVM_SEV_ES_INIT to avoid
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 58733b63bcd7..08d5dc55e175 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -3625,27 +3625,6 @@ static void pre_svm_run(struct kvm_vcpu *vcpu)
new_asid(svm, sd);
}
-static void svm_inject_nmi(struct kvm_vcpu *vcpu)
-{
- struct vcpu_svm *svm = to_svm(vcpu);
-
- svm->vmcb->control.event_inj = SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_NMI;
-
- if (svm->nmi_l1_to_l2)
- return;
-
- /*
- * No need to manually track NMI masking when vNMI is enabled, hardware
- * automatically sets V_NMI_BLOCKING_MASK as appropriate, including the
- * case where software directly injects an NMI.
- */
- if (!is_vnmi_enabled(svm)) {
- svm->nmi_masked = true;
- svm_set_iret_intercept(svm);
- }
- ++vcpu->stat.nmi_injections;
-}
-
static bool svm_is_vnmi_pending(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -3679,6 +3658,33 @@ static bool svm_set_vnmi_pending(struct kvm_vcpu *vcpu)
return true;
}
+static void svm_inject_nmi(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ if (sev_savic_active(vcpu->kvm)) {
+ svm_set_vnmi_pending(vcpu);
+ ++vcpu->stat.nmi_injections;
+ return;
+ }
+
+ svm->vmcb->control.event_inj = SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_NMI;
+
+ if (svm->nmi_l1_to_l2)
+ return;
+
+ /*
+ * No need to manually track NMI masking when vNMI is enabled, hardware
+ * automatically sets V_NMI_BLOCKING_MASK as appropriate, including the
+ * case where software directly injects an NMI.
+ */
+ if (!is_vnmi_enabled(svm)) {
+ svm->nmi_masked = true;
+ svm_set_iret_intercept(svm);
+ }
+ ++vcpu->stat.nmi_injections;
+}
+
static void svm_inject_irq(struct kvm_vcpu *vcpu, bool reinjected)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -3826,6 +3832,14 @@ bool svm_nmi_blocked(struct kvm_vcpu *vcpu)
static int svm_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection)
{
struct vcpu_svm *svm = to_svm(vcpu);
+
+ /* Secure AVIC only support V_NMI based NMI injection. */
+ if (sev_savic_active(vcpu->kvm)) {
+ if (svm->vmcb->control.int_ctl & V_NMI_PENDING_MASK)
+ return 0;
+ return 1;
+ }
+
if (svm->nested.nested_run_pending)
return -EBUSY;
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 14/19] KVM: SVM/SEV: Add SVM_VMGEXIT_SECURE_AVIC GHCB protocol event handling
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (12 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 13/19] KVM: SVM/SEV: Secure AVIC: Enable NMI support Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 15/19] KVM: x86: Secure AVIC: Indicate APIC is enabled by guest SW _always_ Neeraj Upadhyay
` (4 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
This GHCB protocol event is used by the guest to notify KVM
of the GPA of the APIC backing page being used by a vCPU. The APIC ID
parameter is used to identify the vCPU to which the backing page
action is related is assigned. An APIC ID value of 0xffff_ffff_ffff_ffff
means that the backing page action is for the vCPU performing the call.
Secure AVIC requires the guest vCPU APIC backing page entry to be
always present in the guest’s Nested Paginge Table (NPT) while the
vCPU is running because some AVIC hardware acceleration sequences
may not be restartable when Secure AVIC is enabled. If an access to
the guest's APIC backing page by Secure AVIC hardware results in a
nested page fault, the BUSY bit in the VMSA is set and subsequent
VMRUN fails with a VMEXIT_BUSY error code. VMEXIT_BUSY is unrecoverable
in this instance and the vCPU cannot be resumed post this event.
Two actions are available to the guest to notify KVM of these pages.
• SVM_VMGEXIT_SAVIC_REGISTER_BACKING_PAGE (0)
A Secure AVIC guest should use this action to inform KVM of the
page-aligned GPA that will be used as the Secure AVIC backing
page for the specified vCPU.
To ensure that the backing page NPT entry is present while vCPU is
running, KVM does a PSMASH for the GPA if the corresponding NPT entry
is of size 2M. Without PSMASH, it is possible for other allocations
to be part of the same 2M page as the APIC backing page and any
modifications (page state change from private to shared) to any one
of those allocations would result in splitting the 2M page to 4K pages.
This would result in zapping the 2M PTE while APIC backing page is
potentially being accessed by Secure AVIC hardware.
Setting a Secure AVIC backing page GPA automatically clears any
currently set Secure AVIC backing page GPA.
• SVM_VMGEXIT_SAVIC_UNREGISTER_BACKING_PAGE (1)
A guest may use this action to inform KVM that the previously set
GPA is no longer being used as the Secure AVIC backing page for
the specified vCPU. This removes the requirement on KVM to ensure
that the specified GPA is always present in the NPT of the guest
while the specified vCPU is running. KVM returns the GPA that was
currently SET or 0 if there was no previously set GPA.
Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Initial GHCB draft spec for the new GHCB event for Secure AVIC is at:
https://lore.kernel.org/linux-coco/3453675d-ca29-4715-9c17-10b56b3af17e@amd.com/T/#u
The GHCB event has been updated to pass the action param
for GPA register/unregister. The new GHCB spec will be published soon.
I will share the link to the updated spec once it is available
publically.
arch/x86/include/uapi/asm/svm.h | 3 ++
arch/x86/kvm/svm/sev.c | 58 +++++++++++++++++++++++++++++++++
arch/x86/kvm/svm/svm.h | 1 +
3 files changed, 62 insertions(+)
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index ec1321248dac..0a1e8687f464 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -117,6 +117,9 @@
#define SVM_VMGEXIT_AP_CREATE 1
#define SVM_VMGEXIT_AP_DESTROY 2
#define SVM_VMGEXIT_SNP_RUN_VMPL 0x80000018
+#define SVM_VMGEXIT_SECURE_AVIC 0x8000001a
+#define SVM_VMGEXIT_SAVIC_REGISTER_BACKING_PAGE 0
+#define SVM_VMGEXIT_SAVIC_UNREGISTER_BACKING_PAGE 1
#define SVM_VMGEXIT_HV_FEATURES 0x8000fffd
#define SVM_VMGEXIT_TERM_REQUEST 0x8000fffe
#define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code) \
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 40314c4086c2..77c1ecebf677 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3400,6 +3400,14 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
!kvm_ghcb_rcx_is_valid(svm))
goto vmgexit_err;
break;
+ case SVM_VMGEXIT_SECURE_AVIC:
+ if (!sev_savic_active(vcpu->kvm) ||
+ !kvm_ghcb_rax_is_valid(svm))
+ goto vmgexit_err;
+ if (svm->vmcb->control.exit_info_1 == SVM_VMGEXIT_SAVIC_REGISTER_BACKING_PAGE)
+ if (!kvm_ghcb_rbx_is_valid(svm))
+ goto vmgexit_err;
+ break;
case SVM_VMGEXIT_MMIO_READ:
case SVM_VMGEXIT_MMIO_WRITE:
if (!kvm_ghcb_sw_scratch_is_valid(svm))
@@ -4511,6 +4519,53 @@ static bool savic_handle_msr_exit(struct kvm_vcpu *vcpu)
return false;
}
+static int sev_handle_savic_vmgexit(struct vcpu_svm *svm)
+{
+ struct kvm_vcpu *vcpu = NULL;
+ u64 apic_id;
+
+ apic_id = kvm_rax_read(&svm->vcpu);
+
+ if (apic_id == -1ULL) {
+ vcpu = &svm->vcpu;
+ } else {
+ vcpu = kvm_get_vcpu_by_id(vcpu->kvm, apic_id);
+ if (!vcpu)
+ goto savic_request_invalid;
+ }
+
+ switch (svm->vmcb->control.exit_info_1) {
+ case SVM_VMGEXIT_SAVIC_REGISTER_BACKING_PAGE:
+ gpa_t gpa;
+
+ gpa = kvm_rbx_read(&svm->vcpu);
+ if (!PAGE_ALIGNED(gpa))
+ goto savic_request_invalid;
+
+ /*
+ * sev_handle_rmp_fault() invocation would result in PSMASH if
+ * NPTE size is 2M.
+ */
+ sev_handle_rmp_fault(vcpu, gpa, 0);
+ to_svm(vcpu)->sev_savic_gpa = gpa;
+ break;
+ case SVM_VMGEXIT_SAVIC_UNREGISTER_BACKING_PAGE:
+ kvm_rbx_write(&svm->vcpu, to_svm(vcpu)->sev_savic_gpa);
+ to_svm(vcpu)->sev_savic_gpa = 0;
+ break;
+ default:
+ goto savic_request_invalid;
+ }
+
+ return 1;
+
+savic_request_invalid:
+ ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 2);
+ ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, GHCB_ERR_INVALID_INPUT);
+
+ return 1;
+}
+
int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -4653,6 +4708,9 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
control->exit_info_1, control->exit_info_2);
ret = -EINVAL;
break;
+ case SVM_VMGEXIT_SECURE_AVIC:
+ ret = sev_handle_savic_vmgexit(svm);
+ break;
case SVM_EXIT_MSR:
if (sev_savic_active(vcpu->kvm) && savic_handle_msr_exit(vcpu))
return 1;
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 62e3581b7d31..be87b9a0284f 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -329,6 +329,7 @@ struct vcpu_svm {
bool guest_gif;
bool sev_savic_has_pending_ipi;
+ gpa_t sev_savic_gpa;
};
struct svm_cpu_data {
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 15/19] KVM: x86: Secure AVIC: Indicate APIC is enabled by guest SW _always_
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (13 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 14/19] KVM: SVM/SEV: Add SVM_VMGEXIT_SECURE_AVIC GHCB protocol event handling Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 16/19] KVM: x86: Secure AVIC: Add IOAPIC EOI support for level interrupts Neeraj Upadhyay
` (3 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
From: Kishon Vijay Abraham I <kvijayab@amd.com>
Guest SW indicates APIC is enabled by writing to APIC_SPIV (
Bit 8: APIC Software Enable). However in the case of Secure AVIC,
APIC_SPIV is not propagated to hypervisor as Secure AVIC HW
itself can detect whether the Guest SW has enabled APIC or not and
handle accordingly.
To handle this case where the HW handles APIC SW Enable in the
hypervisor, always return 'true' from kvm_apic_sw_enabled() if
Secure AVIC is active. This would let hypervisor assume Guest SW
has always enabled APIC and prevent it from taking actions it
usually does when Guest SW has not enabled APIC.
This is especially used when accepting interrupts to be injected to
the Guest and for injecting LAPIC timer interrupts.
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/lapic.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index c9ef9bce438b..a1367689d53c 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -206,6 +206,9 @@ extern struct static_key_false_deferred apic_sw_disabled;
static inline bool kvm_apic_sw_enabled(struct kvm_lapic *apic)
{
+ if (apic->guest_apic_protected == APIC_STATE_PROTECTED_INJECTED_INTR)
+ return true;
+
if (static_branch_unlikely(&apic_sw_disabled.key))
return apic->sw_enabled;
return true;
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 16/19] KVM: x86: Secure AVIC: Add IOAPIC EOI support for level interrupts
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (14 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 15/19] KVM: x86: Secure AVIC: Indicate APIC is enabled by guest SW _always_ Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 17/19] KVM: x86/ioapic: Disable RTC_GSI EOI tracking for protected APIC Neeraj Upadhyay
` (2 subsequent siblings)
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Secure AVIC accelerates EOI writes for edge-triggered interrupts.
For level-triggered interrupts, EOI msr write is forwarded to
hypervisor. Handle APIC_EOI msr write VMGEXIT and propagate EOI
writes to IOAPIC. Current implementation reuses unused host APIC_ISR
regs space to maintain information about active level-triggered
interrupts. As host APIC_TMR state is updated from IOAPIC redirect
entry, host APIC_TMR is used to identify level-triggered IOAPIC
interrupts.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/lapic.h | 5 +++++
arch/x86/kvm/svm/sev.c | 26 ++++++++++++++++++++++++++
2 files changed, 31 insertions(+)
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index a1367689d53c..4e41c7ea4f66 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -159,6 +159,11 @@ static inline void kvm_lapic_clear_vector(int vec, void *bitmap)
clear_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
}
+static inline int kvm_lapic_test_vector(int vec, void *bitmap)
+{
+ return test_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
+}
+
static inline void kvm_lapic_set_vector(int vec, void *bitmap)
{
set_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 77c1ecebf677..a7e916891226 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -4491,9 +4491,27 @@ static void savic_handle_icr_write(struct kvm_vcpu *kvm_vcpu, u64 icr)
}
}
+static int find_highest_isr(struct kvm_lapic *apic)
+{
+ int vec_per_reg = 32;
+ int max_vec = 256;
+ u32 *reg;
+ int vec;
+
+ for (vec = max_vec - 32; vec >= 0; vec -= vec_per_reg) {
+ reg = apic->regs + APIC_ISR + REG_POS(vec);
+ if (*reg)
+ return __fls(*reg) + vec;
+ }
+
+ return -1;
+}
+
static bool savic_handle_msr_exit(struct kvm_vcpu *vcpu)
{
+ struct kvm_lapic *apic;
u32 msr, reg;
+ int vec;
msr = kvm_rcx_read(vcpu);
reg = (msr - APIC_BASE_MSR) << 4;
@@ -4512,6 +4530,12 @@ static bool savic_handle_msr_exit(struct kvm_vcpu *vcpu)
return true;
}
break;
+ case APIC_EOI:
+ apic = vcpu->arch.apic;
+ vec = find_highest_isr(apic);
+ kvm_lapic_clear_vector(vec, apic->regs + APIC_ISR);
+ kvm_apic_set_eoi_accelerated(vcpu, vec);
+ return true;
default:
break;
}
@@ -5294,6 +5318,8 @@ void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected)
vec = (i << 5) + vec_pos;
kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR);
val = val & ~BIT(vec_pos);
+ if (kvm_lapic_test_vector(vec, apic->regs + APIC_TMR))
+ kvm_lapic_set_vector(vec, apic->regs + APIC_ISR);
} while (val);
}
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 17/19] KVM: x86/ioapic: Disable RTC_GSI EOI tracking for protected APIC
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (15 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 16/19] KVM: x86: Secure AVIC: Add IOAPIC EOI support for level interrupts Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 18/19] X86: SVM: Check injected vectors before waiting for timer expiry Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 19/19] KVM: SVM/SEV: Allow creating VMs with Secure AVIC enabled Neeraj Upadhyay
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Disable RTC_GSI pending EOI tracking for guests with protected APIC
state such as Secure AVIC. RTC_GSI is a edge-triggered vector and
Secure AVIC accelerates EOI write for edge-triggered interrupts.
In addition, APIC_ISR state is not visible within KVM and is part
of guest-owned backing page. So, lazy pending EOI checks also cannot
be done. So, disable tracking of pending EOI. This means coalesced
RTC interrupts cannot be detected by userspace for Secure AVIC guests.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/ioapic.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/ioapic.c b/arch/x86/kvm/ioapic.c
index 995eb5054360..7d68e8ef6245 100644
--- a/arch/x86/kvm/ioapic.c
+++ b/arch/x86/kvm/ioapic.c
@@ -113,6 +113,9 @@ static void __rtc_irq_eoi_tracking_restore_one(struct kvm_vcpu *vcpu)
struct dest_map *dest_map = &ioapic->rtc_status.dest_map;
union kvm_ioapic_redirect_entry *e;
+ if (vcpu->arch.apic->guest_apic_protected)
+ return;
+
e = &ioapic->redirtbl[RTC_GSI];
if (!kvm_apic_match_dest(vcpu, NULL, APIC_DEST_NOSHORT,
e->fields.dest_id,
@@ -443,6 +446,7 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
{
union kvm_ioapic_redirect_entry *entry = &ioapic->redirtbl[irq];
struct kvm_lapic_irq irqe;
+ struct kvm_vcpu *vcpu;
int ret;
if (entry->fields.mask ||
@@ -472,7 +476,9 @@ static int ioapic_service(struct kvm_ioapic *ioapic, int irq, bool line_status)
BUG_ON(ioapic->rtc_status.pending_eoi != 0);
ret = kvm_irq_delivery_to_apic(ioapic->kvm, NULL, &irqe,
&ioapic->rtc_status.dest_map);
- ioapic->rtc_status.pending_eoi = (ret < 0 ? 0 : ret);
+ vcpu = kvm_get_vcpu(ioapic->kvm, 0);
+ if (!vcpu->arch.apic->guest_apic_protected)
+ ioapic->rtc_status.pending_eoi = (ret < 0 ? 0 : ret);
} else
ret = kvm_irq_delivery_to_apic(ioapic->kvm, NULL, &irqe, NULL);
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 18/19] X86: SVM: Check injected vectors before waiting for timer expiry
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (16 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 17/19] KVM: x86/ioapic: Disable RTC_GSI EOI tracking for protected APIC Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
2025-02-28 8:51 ` [RFC PATCH 19/19] KVM: SVM/SEV: Allow creating VMs with Secure AVIC enabled Neeraj Upadhyay
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
For Secure AVIC guests, call kvm_wait_lapic_expire() only when
the vector for LVTT is set in requested_IRR.
KVM always assumes a timer IRQ was injected if APIC state is
protected. For Secure AVIC guests, check to-be-injected vectors
in the requested_IRR to avoid unnecessary wait calls.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 8 ++++++++
arch/x86/kvm/svm/svm.c | 3 ++-
arch/x86/kvm/svm/svm.h | 2 ++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index a7e916891226..881311227504 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -5334,3 +5334,11 @@ bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu)
return READ_ONCE(to_svm(vcpu)->sev_savic_has_pending_ipi) ||
kvm_apic_has_interrupt(vcpu) != -1;
}
+
+bool sev_savic_timer_int_injected(struct kvm_vcpu *vcpu)
+{
+ u32 reg = kvm_lapic_get_reg(vcpu->arch.apic, APIC_LVTT);
+ int vec = reg & APIC_VECTOR_MASK;
+
+ return to_svm(vcpu)->vmcb->control.requested_irr[vec / 32] & BIT(vec % 32);
+}
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 08d5dc55e175..1323ec14f76a 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -4292,7 +4292,8 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu,
clgi();
kvm_load_guest_xsave_state(vcpu);
- kvm_wait_lapic_expire(vcpu);
+ if (!sev_savic_active(vcpu->kvm) || sev_savic_timer_int_injected(vcpu))
+ kvm_wait_lapic_expire(vcpu);
/*
* If this vCPU has touched SPEC_CTRL, restore the guest's value if
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index be87b9a0284f..b129ae089186 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -765,6 +765,7 @@ static inline bool sev_savic_active(struct kvm *kvm)
}
void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected);
bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu);
+bool sev_savic_timer_int_injected(struct kvm_vcpu *vcpu);
#else
static inline struct page *snp_safe_alloc_page_node(int node, gfp_t gfp)
{
@@ -798,6 +799,7 @@ static inline int sev_private_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn)
static inline bool sev_savic_active(struct kvm *kvm) { return false; }
static inline void sev_savic_set_requested_irr(struct vcpu_svm *svm, bool reinjected) {}
static inline bool sev_savic_has_pending_interrupt(struct kvm_vcpu *vcpu) { return false; }
+static inline bool sev_savic_timer_int_injected(struct kvm_vcpu *vcpu) { return true; }
#endif
/* vmenter.S */
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread* [RFC PATCH 19/19] KVM: SVM/SEV: Allow creating VMs with Secure AVIC enabled
2025-02-28 8:50 [RFC PATCH 00/19] AMD: Add Secure AVIC KVM Support Neeraj Upadhyay
` (17 preceding siblings ...)
2025-02-28 8:51 ` [RFC PATCH 18/19] X86: SVM: Check injected vectors before waiting for timer expiry Neeraj Upadhyay
@ 2025-02-28 8:51 ` Neeraj Upadhyay
18 siblings, 0 replies; 20+ messages in thread
From: Neeraj Upadhyay @ 2025-02-28 8:51 UTC (permalink / raw)
To: seanjc, pbonzini, kvm
Cc: linux-kernel, bp, tglx, mingo, dave.hansen, Thomas.Lendacky,
nikunj, Santosh.Shukla, Vasant.Hegde, Suravee.Suthikulpanit,
David.Kaplan, x86, hpa, peterz, huibo.wang, naveen.rao,
binbin.wu, isaku.yamahata
Now that all the initialization required to enable secure AVIC is done,
set supported_vmsa_features to allow creating Secure AVIC VMs.
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
arch/x86/kvm/svm/sev.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 881311227504..6b1ce8bc490c 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3100,6 +3100,9 @@ void __init sev_hardware_setup(void)
if (sev_es_enabled && cpu_feature_enabled(X86_FEATURE_ALLOWED_SEV_FEATURES))
sev_supported_vmsa_features |= SVM_SEV_FEAT_ALLOWED_SEV_FEATURES;
+
+ if (sev_snp_savic_enabled)
+ sev_supported_vmsa_features |= SVM_SEV_FEAT_SECURE_AVIC;
}
void sev_hardware_unsetup(void)
--
2.34.1
^ permalink raw reply [flat|nested] 20+ messages in thread