[PATCH 06/16] KVM: x86: Widen the plane vCPU fd ioctl allowlist

[David Woodhouse <dwmw2@infradead.org>]

From: David Woodhouse <dwmw@amazon.co.uk>

A VMM that puts an initial VTL context on a higher plane cannot finish
the job through the plane vCPU's fd. kvm_arch_is_vcpu_plane_ioctl()
refuses the ioctls it needs, while the one capability ioctl it does
admit, KVM_ENABLE_CAP, reaches *every* vCPU capability handler.

 - KVM_GET/SET_MP_STATE: a plane vCPU for an AP starts in
   wait-for-INIT like any non-boot x86 vCPU, and nothing ever sends
   INIT/SIPI to the higher plane of an AP. The VMM has to mark it
   runnable itself.
 - KVM_SET_GUEST_DEBUG: single-step and breakpoints on a non-zero
   plane. The debug exits surface through the plane-0 KVM_RUN loop
   like any other plane exit.
 - KVM_X86_SETUP_MCE: mcg_cap is per vCPU and every twin starts with
   the KVM_MAX_MCE_BANKS default, so programming plane 0 alone leaves
   the planes disagreeing. Hyper-V shares MCG state across VTLs.
 - KVM_ENABLE_CAP: accept only the audited capabilities on a plane fd,
   the SynIC pair and the two CPUID-enforcement toggles, which change
   state that each twin keeps separately. Anything else gets -EINVAL, as
   non-allowlisted ioctls already do. The nested paths behind
   enlightened VMCS and direct TLB flush have never been audited for
   plane_level > 0, and without the filter any future vCPU cap would
   work on planes without anyone having decided that it should.

KVM_SET_GUEST_DEBUG on a higher plane can set KVM_GUESTDBG_BLOCKIRQ
there, but the VM-wide APICv BLOCKIRQ inhibit is recomputed over
kvm_for_each_vcpu(), which walks plane 0 only. The inhibit then never
rises for a higher plane, and a plane-0 update drops it while a higher
plane still wants it. Recompute it over every twin. Nothing reaches
this yet (multi-plane VMs are SEV-SNP only, which inhibits APICv
anyway), but it would bite as soon as planes and APICv could coexist.

api.rst marks the new ioctls, and KVM_ENABLE_CAP, as accepted on all
planes, and lists the capabilities a plane fd accepts.

The KVM_ENABLE_CAP filter was found by a local Sashiko review.

The KVM_X86_SETUP_MCE part is from Mushahid Hussain.

Co-developed-by: Mushahid Hussain <hmushi@amazon.co.uk>
Not-signed-off-by: Mushahid Hussain <hmushi@amazon.co.uk>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 Documentation/virt/kvm/api.rst | 14 ++++++++-----
 arch/x86/kvm/x86.c             | 38 ++++++++++++++++++++++++++++++----
 2 files changed, 43 insertions(+), 9 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 6c701cda61ef..d21378183c51 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -1494,7 +1494,7 @@ documentation when it pops into existence).
 
 :Capability: KVM_CAP_ENABLE_CAP
 :Architectures: mips, ppc, s390, x86, loongarch
-:Type: vcpu ioctl
+:Type: vcpu ioctl (all planes)
 :Parameters: struct kvm_enable_cap (in)
 :Returns: 0 on success; -1 on error
 
@@ -1515,6 +1515,10 @@ do support it, it only works for extensions that are supported for enablement.
 To check if a capability can be enabled, the KVM_CHECK_EXTENSION ioctl should
 be used.
 
+On x86, the vCPU fd of a plane other than plane 0 accepts only
+KVM_CAP_HYPERV_SYNIC, KVM_CAP_HYPERV_SYNIC2, KVM_CAP_HYPERV_ENFORCE_CPUID and
+KVM_CAP_ENFORCE_PV_FEATURE_CPUID; any other capability fails with -EINVAL.
+
 ::
 
   struct kvm_enable_cap {
@@ -1549,7 +1553,7 @@ for vm-wide capabilities.
 
 :Capability: KVM_CAP_MP_STATE
 :Architectures: x86, s390, arm64, riscv, loongarch
-:Type: vcpu ioctl
+:Type: vcpu ioctl (all planes)
 :Parameters: struct kvm_mp_state (out)
 :Returns: 0 on success; -1 on error
 
@@ -1632,7 +1636,7 @@ whether the vcpu is runnable.
 
 :Capability: KVM_CAP_MP_STATE
 :Architectures: x86, s390, arm64, riscv, loongarch
-:Type: vcpu ioctl
+:Type: vcpu ioctl (all planes)
 :Parameters: struct kvm_mp_state (in)
 :Returns: 0 on success; -1 on error
 
@@ -3779,7 +3783,7 @@ handled.
 
 :Capability: KVM_CAP_SET_GUEST_DEBUG
 :Architectures: x86, s390, ppc, arm64
-:Type: vcpu ioctl
+:Type: vcpu ioctl (all planes)
 :Parameters: struct kvm_guest_debug (in)
 :Returns: 0 on success; -1 on error
 
@@ -4623,7 +4627,7 @@ capabilities will have the corresponding bits set.
 
 :Capability: KVM_CAP_MCE
 :Architectures: x86
-:Type: vcpu ioctl
+:Type: vcpu ioctl (all planes)
 :Parameters: u64 mcg_cap (in)
 :Returns: 0 on success,
          -EFAULT if u64 mcg_cap cannot be read,
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 25f574bfbb32..cd0e9b6cf8f7 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3416,6 +3416,25 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
 	if (cap->flags)
 		return -EINVAL;
 
+	/*
+	 * KVM_ENABLE_CAP is accepted on plane vCPU fds, but only the caps
+	 * below have been audited for plane_level > 0.  The nested paths
+	 * behind eVMCS and direct TLB flush have not; refuse those rather
+	 * than implicitly extending every current and future vCPU cap to
+	 * planes.
+	 */
+	if (vcpu->plane_level) {
+		switch (cap->cap) {
+		case KVM_CAP_HYPERV_SYNIC:
+		case KVM_CAP_HYPERV_SYNIC2:
+		case KVM_CAP_HYPERV_ENFORCE_CPUID:
+		case KVM_CAP_ENFORCE_PV_FEATURE_CPUID:
+			break;
+		default:
+			return -EINVAL;
+		}
+	}
+
 	switch (cap->cap) {
 #ifdef CONFIG_KVM_HYPERV
 	case KVM_CAP_HYPERV_SYNIC2:
@@ -3468,6 +3487,7 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)
 {
 	switch (ioctl) {
 	case KVM_ENABLE_CAP:
+	case KVM_SET_GUEST_DEBUG:
 	case KVM_GET_DEBUGREGS:
 	case KVM_SET_DEBUGREGS:
 	case KVM_GET_LAPIC:
@@ -3480,6 +3500,8 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)
 	case KVM_SET_ONE_REG:
 	case KVM_GET_SREGS2:
 	case KVM_SET_SREGS2:
+	case KVM_GET_MP_STATE:
+	case KVM_SET_MP_STATE:
 	case KVM_GET_VCPU_EVENTS:
 	case KVM_SET_VCPU_EVENTS:
 	case KVM_GET_XCRS:
@@ -3488,6 +3510,7 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)
 	case KVM_GET_XSAVE2:
 	case KVM_SET_XSAVE:
 	case KVM_GET_REG_LIST:
+	case KVM_X86_SETUP_MCE:
 		return true;
 	default:
 		return false;
@@ -9355,19 +9378,26 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_task_switch);
 static void kvm_arch_vcpu_guestdbg_update_apicv_inhibit(struct kvm *kvm)
 {
 	bool set = false;
-	struct kvm_vcpu *vcpu;
+	struct kvm_vcpu *vcpu, *v;
 	unsigned long i;
+	int p;
 
 	if (!enable_apicv)
 		return;
 
 	down_write(&kvm->arch.apicv_update_lock);
 
+	/*
+	 * KVM_SET_GUEST_DEBUG is accepted on every plane's vCPU fd, and
+	 * kvm_for_each_vcpu() only walks plane 0: look at every twin.
+	 */
 	kvm_for_each_vcpu(i, vcpu, kvm) {
-		if (vcpu->guest_debug & KVM_GUESTDBG_BLOCKIRQ) {
-			set = true;
-			break;
+		vcpu_for_each_plane(vcpu->common, p, v) {
+			if (v->guest_debug & KVM_GUESTDBG_BLOCKIRQ)
+				set = true;
 		}
+		if (set)
+			break;
 	}
 	__kvm_set_or_clear_apicv_inhibit(kvm, APICV_INHIBIT_REASON_BLOCKIRQ, set);
 	up_write(&kvm->arch.apicv_update_lock);
-- 
2.50.1