[PATCH v3 00/12] KVM: Improve VMware guest support

[PATCH v3 00/12] KVM: Improve VMware guest support

[Zack Rusin]

This series lets userspace VMMs run unmodified VMware guests on KVM;
VMware Workstation uses it to run on top of KVM.  I'm sorry this
version took a lot longer than I'd have liked, but I think it addresses
all the comments and results in a very nice cleanup for KVM in general.

VMware guests talk to the hypervisor through the "backdoor", a set of
about one hundred commands implemented in VMware's userspace.  Guests
reach it through IN/OUT/INS/OUTS on two I/O ports, which VMware allows
from CPL3 regardless of I/O permissions, through RDPMC on three
pseudo-PMCs, and, on newer guests, through VMCALL/VMMCALL with VMware's
magic value in EAX.  KVM already emulates the I/O-port and pseudo-PMC
interfaces, but only behind a module parameter that applies to every VM
on the host.

Patch 1 makes the pseudo-PMCs honor enable_vmware_backdoor.
Patches 2-4 move VMware code to vmware.{c,h} and add CONFIG_KVM_VMWARE.
Patches 5-7 add a per-VM capability, hypercall exits and nested routing.
Patches 8-12 add CPL3 selftest helpers and tests.

Notes:
 - Paolo asked in v1 for someone with the spec to go through the
   commands; Doug did that in the v1 thread and none needs KVM
   (SGDT/SIDT/SLDT/STR are obsolete with UMIP, GET_VCPU_INFO flags are
   constants on KVM, GETTIME returns host time without the TSC,
   STEALCLOCK is ESX-only).
 - The I/O-port and pseudo-PMC interfaces share one flag, as two parts
   of the legacy backdoor.
 - VMware hypercalls are never claimed from L2, so a Hyper-V L2 TLB
   flush that KVM handles in L0 stays a Hyper-V hypercall.
 - SEV: KVM doesn't intercept #GP, so the flag can't bypass I/O
   permissions there.  Protected-state VMs can't enable hypercall exits,
   and KVM doesn't forward hypercalls from protected vCPUs of legacy
   SEV-ES VMs either.
 - tools/include/uapi is left to the perf sync, per its README.
 - Sean, Paolo: the new MAINTAINERS entry lists Doug and me; do you want
   to be listed as for Hyper-V and Xen?

The series is based on kvm-x86/next.

Testing:
 - every patch builds with GCC and Clang, CONFIG_KVM_VMWARE=y and =n
 - KVM selftests and KVM-unit-tests on Intel and AMD, with and without
   the series, enable_vmware_backdoor=0 and =1: no differences besides
   the new tests
 - a legacy KVM_SEV_ES_INIT guest on bare-metal AMD gets no
   KVM_EXIT_VMWARE
 - Windows 11 with VBS runs under VMware Workstation on the series

Changes since v2:
 - New patch 1: the pseudo-PMCs honor enable_vmware_backdoor
 - Split the first patch into code movement, helpers and Kconfig; all
   code in vmware.{c,h}; CONFIG_KVM_VMWARE depends on KVM_X86 (Sean)
 - kvm_is_vmware_*() predicates with the enabled check inside (Sean)
 - One KVM_CAP_X86_VMWARE with flags in the x86 UAPI header instead of
   three capabilities; flags are one-way and only accepted before vCPU
   creation (Xin, Sean); "_L0" dropped from the nested flag (Sean)
 - No stub for the hypercall handler, no #ifdef plus stub (Sean)
 - struct kvm_vmware_exit in the x86 UAPI header (Sean); the exit now
   carries only mode and CPL, registers go through KVM_SYNC_X86_REGS,
   and exception completion is validated
 - Only L1 hypercalls with VMware's magic are claimed
 - Nested SVM fixed: v2's nested_svm_exit_special() hook never fired
 - Selftests: KUT port (Sean), hypercall, Hyper-V coexistence and
   nested routing tests

v2: https://lore.kernel.org/all/20250422161304.579394-1-zack.rusin@broadcom.com
v1: https://lore.kernel.org/all/20241030033514.1728937-1-zack.rusin@broadcom.com

Zack Rusin (12):
  KVM: x86/pmu: Honor enable_vmware_backdoor for VMware pseudo-PMCs
  KVM: x86: Move VMware backdoor support to separate files
  KVM: x86: Encapsulate VMware backdoor checks
  KVM: x86: Add CONFIG_KVM_VMWARE
  KVM: x86: Add a unified VMware capability
  KVM: x86: Forward VMware hypercalls to userspace
  KVM: x86: Route nested VMware backdoors to KVM
  KVM: selftests: Add helpers to run x86 guest code in user mode
  KVM: selftests: Port VMware backdoor coverage
  KVM: selftests: Test VMware hypercall exits
  KVM: selftests: Test VMware hypercall exits with nested Hyper-V
    flushes
  KVM: selftests: Test nested VMware backdoor routing

 Documentation/virt/kvm/api.rst                | 119 +++++-
 MAINTAINERS                                   |  10 +
 arch/x86/include/asm/kvm_host.h               |  11 +
 arch/x86/include/uapi/asm/kvm.h               |  27 ++
 arch/x86/kvm/Kconfig                          |  12 +
 arch/x86/kvm/Makefile                         |   1 +
 arch/x86/kvm/emulate.c                        |   9 +-
 arch/x86/kvm/pmu.c                            |  41 +-
 arch/x86/kvm/pmu.h                            |   5 -
 arch/x86/kvm/svm/svm.c                        |  12 +-
 arch/x86/kvm/vmware.c                         | 201 +++++++++
 arch/x86/kvm/vmware.h                         | 102 +++++
 arch/x86/kvm/vmx/nested.c                     |   5 +
 arch/x86/kvm/vmx/vmx.c                        |   5 +-
 arch/x86/kvm/x86.c                            |  63 ++-
 arch/x86/kvm/x86.h                            |   2 -
 include/uapi/linux/kvm.h                      |   6 +
 tools/testing/selftests/kvm/Makefile.kvm      |   3 +
 .../selftests/kvm/include/x86/hyperv.h        |  21 +
 .../selftests/kvm/include/x86/processor.h     |   7 +
 .../testing/selftests/kvm/lib/x86/processor.c |  87 ++++
 .../testing/selftests/kvm/x86/hyperv_evmcs.c  |  27 +-
 .../selftests/kvm/x86/hyperv_svm_test.c       |  21 +-
 .../selftests/kvm/x86/vmware_backdoor_test.c  | 272 ++++++++++++
 .../selftests/kvm/x86/vmware_hypercall_test.c | 388 ++++++++++++++++++
 .../kvm/x86/vmware_nested_backdoor_test.c     | 223 ++++++++++
 .../selftests/kvm/x86/xen_vmcall_test.c       |  12 +-
 27 files changed, 1587 insertions(+), 105 deletions(-)
 create mode 100644 arch/x86/kvm/vmware.c
 create mode 100644 arch/x86/kvm/vmware.h
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_hypercall_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_nested_backdoor_test.c


base-commit: c37cd5acfc3762e3fec1bf8418dc1471e3d6297d
-- 
2.53.0

[PATCH v3 01/12] KVM: x86/pmu: Honor enable_vmware_backdoor for VMware pseudo-PMCs

[Zack Rusin]

Check enable_vmware_backdoor before kvm_pmu_rdpmc() hands an RDPMC of a
VMware pseudo-PMC to the VMware emulation.

The pseudo-PMCs are part of the VMware backdoor, but only the
emulator's check for RDPMC at CPL3 with CR4.PCE clear honors
enable_vmware_backdoor.  An RDPMC that KVM intercepts at CPL0, or at
CPL3 with CR4.PCE set, returns the host TSC or time to any guest, even
with the backdoor disabled.  Inject #GP instead, as for any other
invalid PMC index.

Fixes: 2d7921c499af ("KVM: x86: Add support for VMware backdoor Pseudo-PMCs")
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index eef2f792dc9e..ca6dfb44949f 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -759,7 +759,7 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 	if (!pmu->version)
 		return 1;
 
-	if (is_vmware_backdoor_pmc(idx))
+	if (enable_vmware_backdoor && is_vmware_backdoor_pmc(idx))
 		return kvm_pmu_rdpmc_vmware(vcpu, idx, data);
 
 	pmc = kvm_pmu_call(rdpmc_ecx_to_pmc)(vcpu, idx, &mask);
-- 
2.53.0

[PATCH v3 02/12] KVM: x86: Move VMware backdoor support to separate files

[Zack Rusin]

Move the enable_vmware_backdoor module parameter, the VMware backdoor
port and pseudo-PMC definitions, and the VMware pseudo-PMC and opcode
helpers out of x86.c, emulate.c and pmu.{c,h} into new vmware.{c,h}
files, and add a MAINTAINERS entry for the new files.

KVM's VMware emulation is spread across the emulator, the PMU code and
x86.c, and both vendor modules read the module parameter directly.  The
following patches make the VMware interfaces a per-VM setting, add a
userspace exit for VMware hypercalls and allow compiling the emulation
out.  Collect the code in one place first, as KVM already does for
Hyper-V and Xen emulation, so that those changes stay local and the
code movement can be reviewed on its own.

No functional change intended.

Link: https://lore.kernel.org/r/aIEgVpjXDR0BXgHq@google.com
Link: https://lore.kernel.org/r/aIEwOToiAkKfQA-4@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 MAINTAINERS            |  9 +++++
 arch/x86/kvm/Makefile  |  3 +-
 arch/x86/kvm/emulate.c |  4 +--
 arch/x86/kvm/pmu.c     | 35 +-----------------
 arch/x86/kvm/pmu.h     |  5 ---
 arch/x86/kvm/svm/svm.c |  1 +
 arch/x86/kvm/vmware.c  | 81 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmware.h  | 22 ++++++++++++
 arch/x86/kvm/vmx/vmx.c |  1 +
 arch/x86/kvm/x86.c     | 36 +------------------
 arch/x86/kvm/x86.h     |  2 --
 11 files changed, 119 insertions(+), 80 deletions(-)
 create mode 100644 arch/x86/kvm/vmware.c
 create mode 100644 arch/x86/kvm/vmware.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 222b7fc2c3aa..017053e8f76e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15042,6 +15042,15 @@ F:	arch/x86/kvm/svm/hyperv.*
 F:	arch/x86/kvm/svm/svm_onhyperv.*
 F:	arch/x86/kvm/vmx/hyperv.*
 
+KVM X86 VMWARE (KVM/VMware)
+M:	Zack Rusin <zack.rusin@broadcom.com>
+M:	Doug Covelli <doug.covelli@broadcom.com>
+L:	kvm@vger.kernel.org
+S:	Supported
+T:	git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
+T:	git https://github.com/kvm-x86/linux.git
+F:	arch/x86/kvm/vmware.*
+
 KVM X86 Xen (KVM/Xen)
 M:	David Woodhouse <dwmw2@infradead.org>
 M:	Paul Durrant <paul@xen.org>
diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index c6bf463f5032..0cb47cfc1162 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -6,7 +6,8 @@ ccflags-$(CONFIG_KVM_WERROR) += -Werror
 include $(srctree)/virt/kvm/Makefile.kvm
 
 kvm-y			+= x86.o emulate.o irq.o lapic.o cpuid.o msrs.o pmu.o regs.o \
-			   mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o
+			   mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o \
+			   vmware.o
 
 kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o
 kvm-$(CONFIG_KVM_IOAPIC) += i8259.o i8254.o ioapic.o
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index c1b21282187f..a7bcde6f0359 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -33,6 +33,7 @@
 #include "tss.h"
 #include "mmu.h"
 #include "pmu.h"
+#include "vmware.h"
 
 /*
  * Operand types
@@ -2523,9 +2524,6 @@ static bool emulator_bad_iopl(struct x86_emulate_ctxt *ctxt)
 	return ctxt->ops->cpl(ctxt) > iopl;
 }
 
-#define VMWARE_PORT_VMPORT	(0x5658)
-#define VMWARE_PORT_VMRPC	(0x5659)
-
 static bool emulator_io_port_access_allowed(struct x86_emulate_ctxt *ctxt,
 					    u16 port, u16 len)
 {
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index ca6dfb44949f..ae7000c44677 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -23,6 +23,7 @@
 #include "cpuid.h"
 #include "lapic.h"
 #include "pmu.h"
+#include "vmware.h"
 
 /* This is enough to filter the vast majority of currently defined events. */
 #define KVM_PMU_EVENT_FILTER_MAX_EVENTS 300
@@ -716,40 +717,6 @@ int kvm_pmu_check_rdpmc_early(struct kvm_vcpu *vcpu, unsigned int idx)
 	return kvm_pmu_call(check_rdpmc_early)(vcpu, idx);
 }
 
-bool is_vmware_backdoor_pmc(u32 pmc_idx)
-{
-	switch (pmc_idx) {
-	case VMWARE_BACKDOOR_PMC_HOST_TSC:
-	case VMWARE_BACKDOOR_PMC_REAL_TIME:
-	case VMWARE_BACKDOOR_PMC_APPARENT_TIME:
-		return true;
-	}
-	return false;
-}
-
-static int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
-{
-	u64 ctr_val;
-
-	switch (idx) {
-	case VMWARE_BACKDOOR_PMC_HOST_TSC:
-		ctr_val = rdtsc();
-		break;
-	case VMWARE_BACKDOOR_PMC_REAL_TIME:
-		ctr_val = ktime_get_boottime_ns();
-		break;
-	case VMWARE_BACKDOOR_PMC_APPARENT_TIME:
-		ctr_val = ktime_get_boottime_ns() +
-			vcpu->kvm->arch.kvmclock_offset;
-		break;
-	default:
-		return 1;
-	}
-
-	*data = ctr_val;
-	return 0;
-}
-
 int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 090c9bbb74f4..3fc2ce713b6f 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -17,10 +17,6 @@
 #define fixed_ctrl_field(ctrl_reg, idx) \
 	(((ctrl_reg) >> ((idx) * INTEL_FIXED_BITS_STRIDE)) & INTEL_FIXED_BITS_MASK)
 
-#define VMWARE_BACKDOOR_PMC_HOST_TSC		0x10000
-#define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
-#define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
-
 #define KVM_FIXED_PMC_BASE_IDX INTEL_PMC_IDX_FIXED
 
 struct kvm_pmu_ops {
@@ -287,7 +283,6 @@ void kvm_pmu_branch_retired(struct kvm_vcpu *vcpu);
 void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu);
 void kvm_mediated_pmu_put(struct kvm_vcpu *vcpu);
 
-bool is_vmware_backdoor_pmc(u32 pmc_idx);
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu);
 bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu);
 
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 721dc5e7e13a..980841d153f3 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -9,6 +9,7 @@
 #include "smm.h"
 #include "cpuid.h"
 #include "pmu.h"
+#include "vmware.h"
 
 #include <linux/module.h>
 #include <linux/kernel.h>
diff --git a/arch/x86/kvm/vmware.c b/arch/x86/kvm/vmware.c
new file mode 100644
index 000000000000..772f295f8ff5
--- /dev/null
+++ b/arch/x86/kvm/vmware.c
@@ -0,0 +1,81 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * KVM VMware emulation support
+ *
+ * Copyright (C) 2025-2026 Broadcom. All Rights Reserved. The term
+ * "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
+ */
+
+#include <linux/moduleparam.h>
+
+#include "kvm_emulate.h"
+#include "vmware.h"
+
+bool __read_mostly enable_vmware_backdoor;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_vmware_backdoor);
+module_param(enable_vmware_backdoor, bool, 0444);
+
+bool is_vmware_backdoor_pmc(u32 pmc_idx)
+{
+	switch (pmc_idx) {
+	case VMWARE_BACKDOOR_PMC_HOST_TSC:
+	case VMWARE_BACKDOOR_PMC_REAL_TIME:
+	case VMWARE_BACKDOOR_PMC_APPARENT_TIME:
+		return true;
+	}
+	return false;
+}
+
+int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data)
+{
+	u64 ctr_val;
+
+	switch (idx) {
+	case VMWARE_BACKDOOR_PMC_HOST_TSC:
+		ctr_val = rdtsc();
+		break;
+	case VMWARE_BACKDOOR_PMC_REAL_TIME:
+		ctr_val = ktime_get_boottime_ns();
+		break;
+	case VMWARE_BACKDOOR_PMC_APPARENT_TIME:
+		ctr_val = ktime_get_boottime_ns() +
+			vcpu->kvm->arch.kvmclock_offset;
+		break;
+	default:
+		return 1;
+	}
+
+	*data = ctr_val;
+	return 0;
+}
+
+bool is_vmware_backdoor_opcode(struct x86_emulate_ctxt *ctxt)
+{
+	switch (ctxt->opcode_len) {
+	case 1:
+		switch (ctxt->b) {
+		case 0xe4:	/* IN */
+		case 0xe5:
+		case 0xec:
+		case 0xed:
+		case 0xe6:	/* OUT */
+		case 0xe7:
+		case 0xee:
+		case 0xef:
+		case 0x6c:	/* INS */
+		case 0x6d:
+		case 0x6e:	/* OUTS */
+		case 0x6f:
+			return true;
+		}
+		break;
+	case 2:
+		switch (ctxt->b) {
+		case 0x33:	/* RDPMC */
+			return true;
+		}
+		break;
+	}
+
+	return false;
+}
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
new file mode 100644
index 000000000000..debabaff9654
--- /dev/null
+++ b/arch/x86/kvm/vmware.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __KVM_X86_VMWARE_H
+#define __KVM_X86_VMWARE_H
+
+#include <linux/kvm_host.h>
+
+struct x86_emulate_ctxt;
+
+#define VMWARE_BACKDOOR_PMC_HOST_TSC		0x10000
+#define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
+#define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
+
+#define VMWARE_PORT_VMPORT	(0x5658)
+#define VMWARE_PORT_VMRPC	(0x5659)
+
+extern bool enable_vmware_backdoor;
+
+bool is_vmware_backdoor_pmc(u32 pmc_idx);
+int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
+bool is_vmware_backdoor_opcode(struct x86_emulate_ctxt *ctxt);
+
+#endif /* __KVM_X86_VMWARE_H */
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 68fa251dda60..d407eecf1034 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -68,6 +68,7 @@
 #include "trace.h"
 #include "vmcs.h"
 #include "vmcs12.h"
+#include "vmware.h"
 #include "vmx.h"
 #include "x86.h"
 #include "x86_ops.h"
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 2cd9ee316aad..389ae6213c85 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -31,6 +31,7 @@
 #include "pmu.h"
 #include "hyperv.h"
 #include "lapic.h"
+#include "vmware.h"
 #include "xen.h"
 #include "smm.h"
 
@@ -191,10 +192,6 @@ module_param(min_timer_period_us, uint, 0644);
 static u32 __read_mostly tsc_tolerance_ppm = 250;
 module_param(tsc_tolerance_ppm, uint, 0644);
 
-bool __read_mostly enable_vmware_backdoor = false;
-module_param(enable_vmware_backdoor, bool, 0444);
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_vmware_backdoor);
-
 /*
  * Flags to manipulate forced emulation behavior (any non-zero value will
  * enable forced emulation).
@@ -6291,37 +6288,6 @@ static bool kvm_vcpu_check_code_breakpoint(struct kvm_vcpu *vcpu,
 	return false;
 }
 
-static bool is_vmware_backdoor_opcode(struct x86_emulate_ctxt *ctxt)
-{
-	switch (ctxt->opcode_len) {
-	case 1:
-		switch (ctxt->b) {
-		case 0xe4:	/* IN */
-		case 0xe5:
-		case 0xec:
-		case 0xed:
-		case 0xe6:	/* OUT */
-		case 0xe7:
-		case 0xee:
-		case 0xef:
-		case 0x6c:	/* INS */
-		case 0x6d:
-		case 0x6e:	/* OUTS */
-		case 0x6f:
-			return true;
-		}
-		break;
-	case 2:
-		switch (ctxt->b) {
-		case 0x33:	/* RDPMC */
-			return true;
-		}
-		break;
-	}
-
-	return false;
-}
-
 static bool is_soft_int_instruction(struct x86_emulate_ctxt *ctxt,
 				    int emulation_type)
 {
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 43850b524774..4d0a8558d931 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -718,8 +718,6 @@ static inline bool kvm_mpx_supported(void)
 
 extern unsigned int min_timer_period_us;
 
-extern bool enable_vmware_backdoor;
-
 extern int pi_inject_timer;
 
 static inline u64 nsec_to_cycles(struct kvm_vcpu *vcpu, u64 nsec)
-- 
2.53.0

[PATCH v3 03/12] KVM: x86: Encapsulate VMware backdoor checks

[Zack Rusin]

Add kvm_is_vmware_backdoor_enabled(), kvm_is_vmware_io_port(),
kvm_is_vmware_backdoor_pmc() and kvm_is_vmware_backdoor_opcode(), and
use them instead of checking enable_vmware_backdoor and the VMware port
and pseudo-PMC numbers directly.  Make the module parameter and the
pseudo-PMC numbers private to vmware.c, and take the backdoor port
numbers from <asm/vmware.h> instead of keeping a second copy.

The next patches make the backdoor a per-VM setting.  With every caller
asking whether a vCPU accepts a VMware interface, instead of reading a
global, that change stays inside vmware.c.

Check the enabled state inside the port and pseudo-PMC predicates, so
that no caller can treat a port or PMC index as VMware's for a VM that
hasn't enabled the backdoor.

No functional change intended.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/aIEgVpjXDR0BXgHq@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/kvm/emulate.c |  5 +--
 arch/x86/kvm/pmu.c     |  6 +--
 arch/x86/kvm/svm/svm.c |  6 +--
 arch/x86/kvm/vmware.c  | 92 +++++++++++++++++++++++++-----------------
 arch/x86/kvm/vmware.h  | 19 +++------
 arch/x86/kvm/vmx/vmx.c |  4 +-
 arch/x86/kvm/x86.c     |  2 +-
 7 files changed, 72 insertions(+), 62 deletions(-)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index a7bcde6f0359..da626bed93c5 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -2539,8 +2539,7 @@ static bool emulator_io_port_access_allowed(struct x86_emulate_ctxt *ctxt,
 	 * VMware allows access to these ports even if denied
 	 * by TSS I/O permission bitmap. Mimic behavior.
 	 */
-	if (enable_vmware_backdoor &&
-	    ((port == VMWARE_PORT_VMPORT) || (port == VMWARE_PORT_VMRPC)))
+	if (kvm_is_vmware_io_port(ctxt->vcpu, port))
 		return true;
 
 	ops->get_segment(ctxt, &tr, &tr_seg, &base3, VCPU_SREG_TR);
@@ -3908,7 +3907,7 @@ static int check_rdpmc(struct x86_emulate_ctxt *ctxt)
 	 * VMware allows access to these Pseduo-PMCs even when read via RDPMC
 	 * in Ring3 when CR4.PCE=0.
 	 */
-	if (enable_vmware_backdoor && is_vmware_backdoor_pmc(rcx))
+	if (kvm_is_vmware_backdoor_pmc(ctxt->vcpu, rcx))
 		return X86EMUL_CONTINUE;
 
 	/*
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index ae7000c44677..dacd40ca8620 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -726,8 +726,8 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 	if (!pmu->version)
 		return 1;
 
-	if (enable_vmware_backdoor && is_vmware_backdoor_pmc(idx))
-		return kvm_pmu_rdpmc_vmware(vcpu, idx, data);
+	if (kvm_is_vmware_backdoor_pmc(vcpu, idx))
+		return kvm_vmware_pmu_rdpmc(vcpu, idx, data);
 
 	pmc = kvm_pmu_call(rdpmc_ecx_to_pmc)(vcpu, idx, &mask);
 	if (!pmc)
@@ -773,7 +773,7 @@ bool kvm_need_rdpmc_intercept(struct kvm_vcpu *vcpu)
 	 * VMware allows access to these Pseduo-PMCs even when read via RDPMC
 	 * in Ring3 when CR4.PCE=0.
 	 */
-	if (enable_vmware_backdoor)
+	if (kvm_is_vmware_backdoor_enabled(vcpu))
 		return true;
 
 	return kvm_need_any_pmc_intercept(vcpu) ||
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 980841d153f3..a2196d8f60aa 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -237,7 +237,7 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)
 
 			svm_leave_nested(vcpu);
 			/* #GP intercept is still needed for vmware backdoor */
-			if (!enable_vmware_backdoor)
+			if (!kvm_is_vmware_backdoor_enabled(vcpu))
 				clr_exception_intercept(svm, GP_VECTOR);
 
 			/*
@@ -1159,7 +1159,7 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 	 * We intercept those #GP and allow access to them anyway
 	 * as VMware does.
 	 */
-	if (enable_vmware_backdoor)
+	if (kvm_is_vmware_backdoor_enabled(vcpu))
 		set_exception_intercept(svm, GP_VECTOR);
 
 	svm_set_intercept(svm, INTERCEPT_INTR);
@@ -2410,7 +2410,7 @@ static int gp_interception(struct kvm_vcpu *vcpu)
 	 * VMware backdoor emulation on #GP interception only handles
 	 * IN{S}, OUT{S}, and RDPMC, and only for L1.
 	 */
-	if (!enable_vmware_backdoor || is_guest_mode(vcpu))
+	if (!kvm_is_vmware_backdoor_enabled(vcpu) || is_guest_mode(vcpu))
 		goto reinject;
 
 	return kvm_emulate_instruction(vcpu, EMULTYPE_VMWARE_GP | EMULTYPE_NO_DECODE);
diff --git a/arch/x86/kvm/vmware.c b/arch/x86/kvm/vmware.c
index 772f295f8ff5..e1062144b750 100644
--- a/arch/x86/kvm/vmware.c
+++ b/arch/x86/kvm/vmware.c
@@ -8,15 +8,35 @@
 
 #include <linux/moduleparam.h>
 
-#include "kvm_emulate.h"
+#include <asm/vmware.h>
+
 #include "vmware.h"
 
-bool __read_mostly enable_vmware_backdoor;
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(enable_vmware_backdoor);
+static bool __read_mostly enable_vmware_backdoor;
 module_param(enable_vmware_backdoor, bool, 0444);
 
-bool is_vmware_backdoor_pmc(u32 pmc_idx)
+#define VMWARE_BACKDOOR_PMC_HOST_TSC		0x10000
+#define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
+#define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
+
+bool kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
 {
+	return enable_vmware_backdoor;
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_is_vmware_backdoor_enabled);
+
+bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
+{
+	return kvm_is_vmware_backdoor_enabled(vcpu) &&
+	       (port == VMWARE_HYPERVISOR_PORT ||
+		port == VMWARE_HYPERVISOR_PORT_HB);
+}
+
+bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx)
+{
+	if (!kvm_is_vmware_backdoor_enabled(vcpu))
+		return false;
+
 	switch (pmc_idx) {
 	case VMWARE_BACKDOOR_PMC_HOST_TSC:
 	case VMWARE_BACKDOOR_PMC_REAL_TIME:
@@ -26,7 +46,38 @@ bool is_vmware_backdoor_pmc(u32 pmc_idx)
 	return false;
 }
 
-int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data)
+bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode)
+{
+	switch (opcode_len) {
+	case 1:
+		switch (opcode) {
+		case 0xe4:	/* IN */
+		case 0xe5:
+		case 0xec:
+		case 0xed:
+		case 0xe6:	/* OUT */
+		case 0xe7:
+		case 0xee:
+		case 0xef:
+		case 0x6c:	/* INS */
+		case 0x6d:
+		case 0x6e:	/* OUTS */
+		case 0x6f:
+			return true;
+		}
+		break;
+	case 2:
+		switch (opcode) {
+		case 0x33:	/* RDPMC */
+			return true;
+		}
+		break;
+	}
+
+	return false;
+}
+
+int kvm_vmware_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data)
 {
 	u64 ctr_val;
 
@@ -48,34 +99,3 @@ int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data)
 	*data = ctr_val;
 	return 0;
 }
-
-bool is_vmware_backdoor_opcode(struct x86_emulate_ctxt *ctxt)
-{
-	switch (ctxt->opcode_len) {
-	case 1:
-		switch (ctxt->b) {
-		case 0xe4:	/* IN */
-		case 0xe5:
-		case 0xec:
-		case 0xed:
-		case 0xe6:	/* OUT */
-		case 0xe7:
-		case 0xee:
-		case 0xef:
-		case 0x6c:	/* INS */
-		case 0x6d:
-		case 0x6e:	/* OUTS */
-		case 0x6f:
-			return true;
-		}
-		break;
-	case 2:
-		switch (ctxt->b) {
-		case 0x33:	/* RDPMC */
-			return true;
-		}
-		break;
-	}
-
-	return false;
-}
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
index debabaff9654..bf250cc6e232 100644
--- a/arch/x86/kvm/vmware.h
+++ b/arch/x86/kvm/vmware.h
@@ -4,19 +4,10 @@
 
 #include <linux/kvm_host.h>
 
-struct x86_emulate_ctxt;
-
-#define VMWARE_BACKDOOR_PMC_HOST_TSC		0x10000
-#define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
-#define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
-
-#define VMWARE_PORT_VMPORT	(0x5658)
-#define VMWARE_PORT_VMRPC	(0x5659)
-
-extern bool enable_vmware_backdoor;
-
-bool is_vmware_backdoor_pmc(u32 pmc_idx);
-int kvm_pmu_rdpmc_vmware(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
-bool is_vmware_backdoor_opcode(struct x86_emulate_ctxt *ctxt);
+bool kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu);
+bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port);
+bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx);
+bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode);
+int kvm_vmware_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
 
 #endif /* __KVM_X86_VMWARE_H */
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index d407eecf1034..edcc5714cb6f 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -918,7 +918,7 @@ void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu)
 	 * We intercept those #GP and allow access to them anyway
 	 * as VMware does.
 	 */
-	if (enable_vmware_backdoor)
+	if (kvm_is_vmware_backdoor_enabled(vcpu))
 		eb |= (1u << GP_VECTOR);
 	if ((vcpu->guest_debug &
 	     (KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_USE_SW_BP)) ==
@@ -5483,7 +5483,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 		error_code = vmcs_read32(VM_EXIT_INTR_ERROR_CODE);
 
 	if (!vmx->rmode.vm86_active && is_gp_fault(intr_info)) {
-		WARN_ON_ONCE(!enable_vmware_backdoor);
+		WARN_ON_ONCE(!kvm_is_vmware_backdoor_enabled(vcpu));
 
 		/*
 		 * VMware backdoor emulation on #GP interception only handles
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 389ae6213c85..0548e34bf634 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6402,7 +6402,7 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 	}
 
 	if ((emulation_type & EMULTYPE_VMWARE_GP) &&
-	    !is_vmware_backdoor_opcode(ctxt)) {
+	    !kvm_is_vmware_backdoor_opcode(ctxt->opcode_len, ctxt->b)) {
 		kvm_queue_exception_e(vcpu, GP_VECTOR, 0);
 		return 1;
 	}
-- 
2.53.0

[PATCH v3 04/12] KVM: x86: Add CONFIG_KVM_VMWARE

[Zack Rusin]

Add CONFIG_KVM_VMWARE, defaulting to "y", to control whether KVM's
VMware emulation is built.  When it is disabled, vmware.o isn't built,
the VMware predicates are compile-time false, and the
enable_vmware_backdoor module parameter doesn't exist: a
kvm.enable_vmware_backdoor setting is ignored, and modprobe warns about
an unknown parameter.

VMware emulation is only useful to VMware-aware guests and to VMMs that
offer VMware compatibility.  Let kernels that don't need it, e.g. to
reduce the attack surface exposed to guests, leave it out, as they can
already do for Hyper-V and Xen emulation.  Default to "y" so that
existing configurations keep the backdoor and its module parameter.

Provide __always_inline stubs for the predicates that gate VMware
behavior, so that callers need no #ifdefs and the compiler drops the
dead branches.  Leave the operations behind the predicates declared but
undefined when the option is disabled: a call that isn't guarded by a
predicate then fails to link instead of silently doing nothing.

Suggested-by: Sean Christopherson <seanjc@google.com>
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Link: https://lore.kernel.org/r/Zy0__5YB9F5d0eZn@google.com
Link: https://lore.kernel.org/r/e3f943a7-a40a-45cb-b0d9-e3ed58344d8b@redhat.com
Link: https://lore.kernel.org/r/aIEmawm9gLflg8zt@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/kvm/Kconfig  | 12 ++++++++++++
 arch/x86/kvm/Makefile |  4 ++--
 arch/x86/kvm/vmware.h | 31 +++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index 1719059570fb..b34d6d0e30c9 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -212,6 +212,18 @@ config KVM_HYPERV
 
 	  If unsure, say "Y".
 
+config KVM_VMWARE
+	bool "VMware guest support"
+	depends on KVM_X86
+	default y
+	help
+	  Provides KVM support for emulating VMware interfaces. VMware-aware
+	  guests use these interfaces to communicate with the hypervisor.
+	  Userspace hypervisors can also use them to provide VMware
+	  compatibility to their guests.
+
+	  If unsure, say "Y".
+
 config KVM_XEN
 	bool "Support for Xen hypercall interface"
 	depends on KVM_X86
diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index 0cb47cfc1162..794c847ccecc 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -6,13 +6,13 @@ ccflags-$(CONFIG_KVM_WERROR) += -Werror
 include $(srctree)/virt/kvm/Makefile.kvm
 
 kvm-y			+= x86.o emulate.o irq.o lapic.o cpuid.o msrs.o pmu.o regs.o \
-			   mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o \
-			   vmware.o
+			   mtrr.o debugfs.o mmu/mmu.o mmu/page_track.o mmu/spte.o
 
 kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o
 kvm-$(CONFIG_KVM_IOAPIC) += i8259.o i8254.o ioapic.o
 kvm-$(CONFIG_KVM_HYPERV) += hyperv.o
 kvm-$(CONFIG_KVM_XEN)	+= xen.o
+kvm-$(CONFIG_KVM_VMWARE) += vmware.o
 kvm-$(CONFIG_KVM_SMM)	+= smm.o
 
 kvm-intel-y		+= vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
index bf250cc6e232..5951e419e1a7 100644
--- a/arch/x86/kvm/vmware.h
+++ b/arch/x86/kvm/vmware.h
@@ -4,10 +4,41 @@
 
 #include <linux/kvm_host.h>
 
+#ifdef CONFIG_KVM_VMWARE
+
 bool kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu);
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port);
 bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx);
 bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode);
+
+#else /* !CONFIG_KVM_VMWARE */
+
+static __always_inline bool
+kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
+static __always_inline bool
+kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
+{
+	return false;
+}
+
+static __always_inline bool
+kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx)
+{
+	return false;
+}
+
+static __always_inline bool
+kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode)
+{
+	return false;
+}
+
+#endif /* CONFIG_KVM_VMWARE */
+
 int kvm_vmware_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
 
 #endif /* __KVM_X86_VMWARE_H */
-- 
2.53.0

[PATCH v3 05/12] KVM: x86: Add a unified VMware capability

[Zack Rusin]

Add KVM_CAP_X86_VMWARE, a VM capability that takes a mask of VMware
interfaces to enable, with KVM_VMWARE_ENABLE_BACKDOOR enabling the
I/O-port and pseudo-PMC backdoor.  KVM_CHECK_EXTENSION returns the
supported flags.  Track the flags per VM, and keep
enable_vmware_backdoor as the initial value of the backdoor flag.

The module parameter enables the backdoor for every VM on the host,
whichever VMM created it.  A per-VM flag lets VMMs enable it only where
it is wanted, and the module parameter keeps existing VMMs working.

Use one capability, so that the hypercall and nested-backdoor flags
added next share its validation and rules.  Accept flags only before
the first vCPU is created, because vCPU setup configures #GP and RDPMC
interception from them, and don't allow clearing them.  Limit the flags
to bits 0-30, as KVM_CHECK_EXTENSION returns an int.

Enabling the backdoor for an SEV guest still doesn't bypass the guest's
I/O permissions: KVM doesn't intercept #GP for SEV guests.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/c60b32a3-e699-4a8c-8d52-09b34c4d5269@redhat.com
Link: https://lore.kernel.org/r/aAjrOgsooR4RYIJr@google.com
Link: https://lore.kernel.org/r/aAkgV3ja9NbDsrju@google.com
Link: https://lore.kernel.org/r/aIEkrc7Mdf2ia1Mm@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 Documentation/virt/kvm/api.rst  | 26 ++++++++++++++++++++++++++
 arch/x86/include/asm/kvm_host.h | 11 +++++++++++
 arch/x86/include/uapi/asm/kvm.h |  2 ++
 arch/x86/kvm/vmware.c           | 20 +++++++++++++++++---
 arch/x86/kvm/vmware.h           | 31 ++++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c              | 17 +++++++++++++++++
 include/uapi/linux/kvm.h        |  1 +
 7 files changed, 104 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index afd4fcac6ecd..57517ebd4340 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -9185,6 +9185,32 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key
 interpretation are disabled. If cmma has already been enabled or the
 hpage_2g module parameter is not set to 1, -EINVAL is returned.
 
+7.48 KVM_CAP_X86_VMWARE
+-----------------------
+
+:Architectures: x86
+:Target: VM
+:Parameters: args[0] contains a mask of ``KVM_VMWARE_*`` flags
+:Returns: 0 on success, -EINVAL if unsupported or after vCPU creation
+
+``KVM_CHECK_EXTENSION`` returns the mask of supported flags.  Userspace can
+enable one or more supported VMware paravirtual interfaces by setting the
+corresponding bits in ``args[0]``.  Userspace must ignore supported bits that
+it does not recognize.  Flags may be enabled only before creating the first
+vCPU and cannot subsequently be disabled.  Because ``KVM_CHECK_EXTENSION``
+returns an ``int``, VMware flags are limited to bits 0 through 30.  Advertising
+additional flags would require a new query interface.
+
+``KVM_VMWARE_ENABLE_BACKDOOR`` enables KVM's emulation of VMware's legacy I/O
+port and pseudo-PMC backdoor interfaces.  The ``enable_vmware_backdoor``
+module parameter remains a compatibility default that enables this flag for
+all newly created VMs.
+
+For SEV guests, KVM cannot intercept #GP to decode and emulate a backdoor I/O
+instruction.  Enabling this flag therefore does not bypass the guest's I/O
+permission checks.  Privileged backdoor I/O and pseudo-PMC accesses remain
+available.
+
 8. Other capabilities.
 ======================
 
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 3efef3415f8e..4b8b1498be80 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1116,6 +1116,13 @@ struct kvm_xen {
 };
 #endif
 
+#ifdef CONFIG_KVM_VMWARE
+/* VMware emulation context */
+struct kvm_vmware {
+	u64 enabled_caps;
+};
+#endif
+
 enum kvm_irqchip_mode {
 	KVM_IRQCHIP_NONE,
 #ifdef CONFIG_KVM_IOAPIC
@@ -1257,6 +1264,10 @@ struct kvm_arch {
 	struct kvm_hv hyperv;
 #endif
 
+#ifdef CONFIG_KVM_VMWARE
+	struct kvm_vmware vmware;
+#endif
+
 #ifdef CONFIG_KVM_XEN
 	struct kvm_xen xen;
 #endif
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 00ce265391e1..57167683692b 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -965,6 +965,8 @@ struct kvm_hyperv_eventfd {
 /* x86-specific KVM_EXIT_HYPERCALL flags. */
 #define KVM_EXIT_HYPERCALL_LONG_MODE	_BITULL(0)
 
+#define KVM_VMWARE_ENABLE_BACKDOOR	_BITULL(0)
+
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_SW_PROTECTED_VM	1
 #define KVM_X86_SEV_VM		2
diff --git a/arch/x86/kvm/vmware.c b/arch/x86/kvm/vmware.c
index e1062144b750..3136e2971afe 100644
--- a/arch/x86/kvm/vmware.c
+++ b/arch/x86/kvm/vmware.c
@@ -19,11 +19,25 @@ module_param(enable_vmware_backdoor, bool, 0444);
 #define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
 #define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
 
-bool kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
+void kvm_vmware_init_vm(struct kvm *kvm)
 {
-	return enable_vmware_backdoor;
+	if (enable_vmware_backdoor)
+		kvm->arch.vmware.enabled_caps |= KVM_VMWARE_ENABLE_BACKDOOR;
+}
+
+int kvm_vmware_enable_cap(struct kvm *kvm, u64 flags)
+{
+	int r = -EINVAL;
+
+	mutex_lock(&kvm->lock);
+	if (!kvm->created_vcpus) {
+		kvm->arch.vmware.enabled_caps |= flags;
+		r = 0;
+	}
+	mutex_unlock(&kvm->lock);
+
+	return r;
 }
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_is_vmware_backdoor_enabled);
 
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
 {
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
index 5951e419e1a7..c4ad1958a4da 100644
--- a/arch/x86/kvm/vmware.h
+++ b/arch/x86/kvm/vmware.h
@@ -6,13 +6,40 @@
 
 #ifdef CONFIG_KVM_VMWARE
 
-bool kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu);
+static __always_inline bool kvm_is_vmware_supported(void)
+{
+	return true;
+}
+
+static __always_inline bool
+kvm_is_vmware_cap_enabled(struct kvm *kvm, u64 cap)
+{
+	return kvm->arch.vmware.enabled_caps & cap;
+}
+
+static __always_inline bool
+kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
+{
+	return kvm_is_vmware_cap_enabled(vcpu->kvm, KVM_VMWARE_ENABLE_BACKDOOR);
+}
+
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port);
 bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx);
 bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode);
 
 #else /* !CONFIG_KVM_VMWARE */
 
+static __always_inline bool kvm_is_vmware_supported(void)
+{
+	return false;
+}
+
+static __always_inline bool
+kvm_is_vmware_cap_enabled(struct kvm *kvm, u64 cap)
+{
+	return false;
+}
+
 static __always_inline bool
 kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
 {
@@ -39,6 +66,8 @@ kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode)
 
 #endif /* CONFIG_KVM_VMWARE */
 
+void kvm_vmware_init_vm(struct kvm *kvm);
+int kvm_vmware_enable_cap(struct kvm *kvm, u64 flags);
 int kvm_vmware_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
 
 #endif /* __KVM_X86_VMWARE_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0548e34bf634..1425376b4f0e 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -118,6 +118,8 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_host);
 
 #define KVM_CAP_PMU_VALID_MASK KVM_PMU_CAP_DISABLE
 
+#define KVM_CAP_VMWARE_VALID_MASK KVM_VMWARE_ENABLE_BACKDOOR
+
 #define KVM_GUESTDBG_VALID_MASK \
 	(KVM_GUESTDBG_ENABLE | \
 	KVM_GUESTDBG_SINGLESTEP | \
@@ -2428,6 +2430,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_READONLY_MEM:
 		r = kvm ? kvm_arch_has_readonly_mem(kvm) : 1;
 		break;
+	case KVM_CAP_X86_VMWARE:
+		BUILD_BUG_ON(KVM_CAP_VMWARE_VALID_MASK > INT_MAX);
+		if (kvm_is_vmware_supported())
+			r = KVM_CAP_VMWARE_VALID_MASK;
+		break;
 	default:
 		break;
 	}
@@ -4262,6 +4269,14 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_unlock(&kvm->lock);
 		break;
 	}
+	case KVM_CAP_X86_VMWARE:
+		r = -EINVAL;
+		if (!kvm_is_vmware_supported() ||
+		    (cap->args[0] & ~KVM_CAP_VMWARE_VALID_MASK))
+			break;
+
+		r = kvm_vmware_enable_cap(kvm, cap->args[0]);
+		break;
 	default:
 		r = -EINVAL;
 		break;
@@ -9936,6 +9951,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 
 	kvm_apicv_init(kvm);
 	kvm_hv_init_vm(kvm);
+	if (kvm_is_vmware_supported())
+		kvm_vmware_init_vm(kvm);
 	kvm_xen_init_vm(kvm);
 
 	if (ignore_msrs && !report_ignored_msrs) {
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 8dff2fc1972e..e43c15c6e5cc 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1000,6 +1000,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_PPC_COMPAT_CAPS 250
 #define KVM_CAP_ARM_PMU_V3_STRICT 251
 #define KVM_CAP_GUEST_MEMFD_MEMORY_ATTRIBUTES 252
+#define KVM_CAP_X86_VMWARE 253
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
-- 
2.53.0

[PATCH v3 06/12] KVM: x86: Forward VMware hypercalls to userspace

[Zack Rusin]

Add KVM_VMWARE_ENABLE_HYPERCALL.  When it is enabled, exit to userspace
with KVM_EXIT_VMWARE when an L1 vCPU executes VMCALL or VMMCALL with
VMware's magic value in EAX, and let userspace complete the call by
updating the guest's registers or by having KVM inject #UD, #GP or #PF.

VMware guests reach the hypervisor through the "backdoor": either the
I/O-port interface that KVM already emulates or, on newer guests,
VMCALL or VMMCALL, with the same commands and register conventions.
VMware's userspace implements the roughly one hundred commands, and
reviewing the command list on the mailing list found none that needs
to be handled in KVM (see the Links), so leave them in userspace.

Only claim calls that carry the magic value, so that the same VM can
still use Xen, Hyper-V or KVM hypercalls.  Never claim a call from L2:
the only L2 hypercall that L0 handles itself is Hyper-V's L2 TLB flush,
which doesn't use RAX as an input.

Pass operands and results through KVM_SYNC_X86_REGS instead of defining
a second register ABI; KVM_EXIT_HYPERCALL returns a single value in
RAX, while backdoor commands use RAX through RBP.  Report the CPL and
leave the privilege policy to userspace, as VMware Tools call the
backdoor from user mode.  Instead of completing the call, userspace can
have KVM inject a validated #UD, #GP or #PF at the hypercall, and may
update the registers as well to keep the partial progress of a
high-bandwidth transfer.

Don't forward hypercalls of vCPUs whose state is protected, as KVM
can't access their registers: reject the flag for VMs with protected
state, omit it from their VM-scoped KVM_CHECK_EXTENSION result, and
ignore it once a vCPU of a VM set up with the legacy KVM_SEV_ES_INIT
becomes protected.

Link: https://lore.kernel.org/r/c60b32a3-e699-4a8c-8d52-09b34c4d5269@redhat.com
Link: https://lore.kernel.org/r/CADH9ctD1uf_yBA3NXNQu7TJa_TPhLRN=0YZ3j2gGhgmaFRdCFg@mail.gmail.com
Link: https://lore.kernel.org/r/CADH9ctBivnvP1tNcatLKzd8EDz8Oo6X65660j8ccxYzk3aFzCA@mail.gmail.com
Link: https://lore.kernel.org/r/aIEmawm9gLflg8zt@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 Documentation/virt/kvm/api.rst  | 84 ++++++++++++++++++++++++++++++++-
 arch/x86/include/uapi/asm/kvm.h | 24 ++++++++++
 arch/x86/kvm/vmware.c           | 78 ++++++++++++++++++++++++++++++
 arch/x86/kvm/vmware.h           |  7 +++
 arch/x86/kvm/x86.c              | 11 ++++-
 include/uapi/linux/kvm.h        |  5 ++
 6 files changed, 206 insertions(+), 3 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 57517ebd4340..e2f0e329f748 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6963,7 +6963,7 @@ to the byte array.
 .. note::
 
       For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN,
-      KVM_EXIT_EPR, KVM_EXIT_HYPERCALL, KVM_EXIT_TDX,
+      KVM_EXIT_EPR, KVM_EXIT_HYPERCALL, KVM_EXIT_TDX, KVM_EXIT_VMWARE,
       KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR the corresponding
       operations are complete (and guest state is consistent) only after userspace
       has re-entered the kernel with KVM_RUN.  The kernel side will first finish
@@ -7017,6 +7017,73 @@ Definition of ``flags``:
 At the point of exit, PC points to the instruction immediately following
 the trapping instruction.
 
+::
+
+		/* KVM_EXIT_VMWARE */
+		struct kvm_vmware_exit {
+			__u32 type;
+			__u32 pad1;
+			union {
+				struct {
+					__u32 longmode;
+					__u32 cpl;
+					struct {
+						__u32 inject;
+						__u32 pad2;
+						__u32 vector;
+						__u32 error_code;
+						__u64 address;
+					} exception;
+				} hcall;
+			};
+		} vmware;
+
+KVM_EXIT_VMWARE is returned when userspace has enabled VMware hypercall
+forwarding and the guest executes a VMCALL or VMMCALL with VMware's magic
+value in RAX.  ``type`` is ``KVM_EXIT_VMWARE_HCALL``.  ``longmode`` reports
+the calling convention and ``cpl`` reports the privilege level.  Userspace
+must set ``pad1`` and ``exception.pad2`` to zero when re-entering KVM.
+
+The VMware exit takes precedence over Xen, Hyper-V, and KVM hypercalls only
+when the low 32 bits of RAX contain VMware's magic value.  Non-matching calls
+continue through the normal hypercall dispatch.  Xen and KVM hypercall numbers
+never match the magic value, but a 64-bit Hyper-V hypercall does not use RAX
+as an input, so one issued with the magic value left in RAX is reported as
+KVM_EXIT_VMWARE.  KVM reports CPL but does not enforce a VMware hypercall
+privilege policy; that policy belongs to userspace.
+
+KVM never reports an L2 hypercall through KVM_EXIT_VMWARE.  VMX reflects an
+L2 VMCALL to L1 and SVM injects #UD for an L2 VMMCALL that L1 does not
+intercept, except that both handle a supported Hyper-V nested TLB flush
+hypercall in L0, as a Hyper-V hypercall even if RAX holds VMware's magic
+value.  VMware hypercall exits cannot be enabled for a VM with protected vCPU
+state, and KVM does not forward hypercalls from a vCPU whose state is
+protected, e.g. in a VM initialized with the legacy KVM_SEV_ES_INIT.
+
+Hypercall operands are available through ``KVM_SYNC_X86_REGS``.  Userspace
+must request synchronized REGS through ``kvm_valid_regs`` on every KVM_RUN
+that can produce this exit; otherwise the synchronized state is stale.  To
+complete the hypercall successfully, userspace may update all GPRs and RFLAGS,
+set ``kvm_dirty_regs`` accordingly, and re-enter with KVM_RUN.  KVM applies
+the synchronized registers using KVM_SET_REGS semantics and then advances
+past the hypercall instruction.  Userspace must leave synchronized RIP at the
+hypercall instruction so that KVM advances it exactly once.  RFLAGS is
+accepted with normal KVM_SET_REGS ownership; KVM forces X86_EFLAGS_FIXED but
+does not apply an exit-specific writable-bit mask.
+
+Userspace may instead set ``exception.inject`` to one and request #UD, #GP,
+or #PF.  #UD requires a zero error code and address.  #GP accepts an error
+code verbatim and requires a zero address.  #PF also accepts the error code
+verbatim and uses ``address`` as CR2.  Userspace may dirty synchronized REGS
+when requesting an exception, e.g. to preserve partial progress from a
+high-bandwidth backdoor transfer.  KVM applies the synchronized registers
+before queuing the exception.  Userspace must leave synchronized RIP at the
+hypercall instruction so that the exception is delivered at the faulting
+instruction.  Invalid completion data makes KVM_RUN fail with -EINVAL and
+leaves the instruction incomplete.  The completion callback is consumed, so
+another KVM_RUN re-executes the instruction and produces a fresh
+KVM_EXIT_VMWARE.
+
 ::
 
 		/* KVM_EXIT_TPR_ACCESS */
@@ -9211,6 +9278,21 @@ instruction.  Enabling this flag therefore does not bypass the guest's I/O
 permission checks.  Privileged backdoor I/O and pseudo-PMC accesses remain
 available.
 
+``KVM_VMWARE_ENABLE_HYPERCALL`` forwards VMCALL and VMMCALL instructions to
+userspace as ``KVM_EXIT_VMWARE``.  The architecture-specific ``vmware``
+member of ``struct kvm_run`` reports the guest's calling convention and CPL.
+Operands and successful register and RFLAGS outputs are transferred through
+``KVM_SYNC_X86_REGS`` as described in chapter 5.  The flag cannot be enabled
+for a VM with protected vCPU state, and a VM-fd ``KVM_CHECK_EXTENSION`` omits
+the flag for such a VM.  In a VM initialized with the legacy
+``KVM_SEV_ES_INIT``, the flag can be enabled but has no effect on a vCPU once
+its state is protected.
+
+Userspace can instead request that the hypercall instruction inject an
+exception by setting ``vmware.hcall.exception.inject`` and filling in the
+validated vector, error code, and address fields.  When an exception is
+requested, KVM does not advance past the hypercall instruction.
+
 8. Other capabilities.
 ======================
 
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 57167683692b..3b9f53bce2d9 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -53,6 +53,7 @@
 #define __KVM_HAVE_DEBUGREGS
 #define __KVM_HAVE_XSAVE
 #define __KVM_HAVE_XCRS
+#define __KVM_HAVE_VMWARE_EXIT
 
 /* Architectural interrupt line count. */
 #define KVM_NR_INTERRUPTS 256
@@ -966,6 +967,29 @@ struct kvm_hyperv_eventfd {
 #define KVM_EXIT_HYPERCALL_LONG_MODE	_BITULL(0)
 
 #define KVM_VMWARE_ENABLE_BACKDOOR	_BITULL(0)
+#define KVM_VMWARE_ENABLE_HYPERCALL	_BITULL(1)
+
+struct kvm_vmware_exit {
+#define KVM_EXIT_VMWARE_HCALL	1
+	__u32 type;
+	__u32 pad1;
+	union {
+		struct {
+			/* out */
+			__u32 longmode;
+			__u32 cpl;
+
+			/* in */
+			struct {
+				__u32 inject;
+				__u32 pad2;
+				__u32 vector;
+				__u32 error_code;
+				__u64 address;
+			} exception;
+		} hcall;
+	};
+};
 
 #define KVM_X86_DEFAULT_VM	0
 #define KVM_X86_SW_PROTECTED_VM	1
diff --git a/arch/x86/kvm/vmware.c b/arch/x86/kvm/vmware.c
index 3136e2971afe..8407dccf81fd 100644
--- a/arch/x86/kvm/vmware.c
+++ b/arch/x86/kvm/vmware.c
@@ -11,6 +11,7 @@
 #include <asm/vmware.h>
 
 #include "vmware.h"
+#include "x86.h"
 
 static bool __read_mostly enable_vmware_backdoor;
 module_param(enable_vmware_backdoor, bool, 0444);
@@ -29,6 +30,10 @@ int kvm_vmware_enable_cap(struct kvm *kvm, u64 flags)
 {
 	int r = -EINVAL;
 
+	if (kvm->arch.has_protected_state &&
+	    (flags & KVM_VMWARE_ENABLE_HYPERCALL))
+		return -EINVAL;
+
 	mutex_lock(&kvm->lock);
 	if (!kvm->created_vcpus) {
 		kvm->arch.vmware.enabled_caps |= flags;
@@ -39,6 +44,79 @@ int kvm_vmware_enable_cap(struct kvm *kvm, u64 flags)
 	return r;
 }
 
+static int complete_vmware_hypercall_exit(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vmware_exit *vmware = &vcpu->run->vmware;
+	u32 pad1 = READ_ONCE(vmware->pad1);
+	u32 inject = READ_ONCE(vmware->hcall.exception.inject);
+	u32 pad2 = READ_ONCE(vmware->hcall.exception.pad2);
+	u32 vector = READ_ONCE(vmware->hcall.exception.vector);
+	u32 error_code = READ_ONCE(vmware->hcall.exception.error_code);
+	u64 address = READ_ONCE(vmware->hcall.exception.address);
+
+	if (pad1 || pad2 || inject > 1)
+		return -EINVAL;
+
+	if (!inject)
+		return kvm_skip_emulated_instruction(vcpu);
+
+	switch (vector) {
+	case UD_VECTOR:
+		if (error_code || address)
+			return -EINVAL;
+		kvm_queue_exception(vcpu, UD_VECTOR);
+		break;
+	case GP_VECTOR:
+		if (address)
+			return -EINVAL;
+		kvm_queue_exception_e(vcpu, GP_VECTOR, error_code);
+		break;
+	case PF_VECTOR: {
+		struct x86_exception fault = {
+			.vector = PF_VECTOR,
+			.error_code_valid = true,
+			.error_code = error_code,
+			.address = address,
+		};
+
+		kvm_inject_page_fault(vcpu, &fault, false);
+		break;
+	}
+	default:
+		return -EINVAL;
+	}
+
+	/* Retry the instruction so the exception is delivered at its RIP. */
+	return 1;
+}
+
+bool kvm_is_vmware_hypercall(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Never claim a hypercall from L2: the only one that L0 handles is
+	 * Hyper-V's L2 TLB flush.  The exit also needs the vCPU's registers,
+	 * which KVM can't access once the vCPU's state is protected.
+	 */
+	return kvm_is_vmware_cap_enabled(vcpu->kvm,
+					 KVM_VMWARE_ENABLE_HYPERCALL) &&
+	       !is_guest_mode(vcpu) && !vcpu->arch.guest_state_protected &&
+	       lower_32_bits(kvm_rax_read_raw(vcpu)) == VMWARE_HYPERVISOR_MAGIC;
+}
+
+int kvm_vmware_hypercall(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vmware_exit *vmware = &vcpu->run->vmware;
+
+	memset(vmware, 0, sizeof(*vmware));
+	vcpu->run->exit_reason = KVM_EXIT_VMWARE;
+	vmware->type = KVM_EXIT_VMWARE_HCALL;
+	vmware->hcall.longmode = is_64_bit_hypercall(vcpu);
+	vmware->hcall.cpl = kvm_x86_call(get_cpl)(vcpu);
+
+	vcpu->arch.complete_userspace_io = complete_vmware_hypercall_exit;
+	return 0;
+}
+
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
 {
 	return kvm_is_vmware_backdoor_enabled(vcpu) &&
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
index c4ad1958a4da..de1dc8971ec6 100644
--- a/arch/x86/kvm/vmware.h
+++ b/arch/x86/kvm/vmware.h
@@ -23,6 +23,7 @@ kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
 	return kvm_is_vmware_cap_enabled(vcpu->kvm, KVM_VMWARE_ENABLE_BACKDOOR);
 }
 
+bool kvm_is_vmware_hypercall(struct kvm_vcpu *vcpu);
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port);
 bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx);
 bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode);
@@ -46,6 +47,11 @@ kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
 	return false;
 }
 
+static __always_inline bool kvm_is_vmware_hypercall(struct kvm_vcpu *vcpu)
+{
+	return false;
+}
+
 static __always_inline bool
 kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
 {
@@ -68,6 +74,7 @@ kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode)
 
 void kvm_vmware_init_vm(struct kvm *kvm);
 int kvm_vmware_enable_cap(struct kvm *kvm, u64 flags);
+int kvm_vmware_hypercall(struct kvm_vcpu *vcpu);
 int kvm_vmware_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned int idx, u64 *data);
 
 #endif /* __KVM_X86_VMWARE_H */
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 1425376b4f0e..bd4142763f10 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -118,7 +118,8 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_host);
 
 #define KVM_CAP_PMU_VALID_MASK KVM_PMU_CAP_DISABLE
 
-#define KVM_CAP_VMWARE_VALID_MASK KVM_VMWARE_ENABLE_BACKDOOR
+#define KVM_CAP_VMWARE_VALID_MASK (KVM_VMWARE_ENABLE_BACKDOOR | \
+				   KVM_VMWARE_ENABLE_HYPERCALL)
 
 #define KVM_GUESTDBG_VALID_MASK \
 	(KVM_GUESTDBG_ENABLE | \
@@ -2432,8 +2433,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		break;
 	case KVM_CAP_X86_VMWARE:
 		BUILD_BUG_ON(KVM_CAP_VMWARE_VALID_MASK > INT_MAX);
-		if (kvm_is_vmware_supported())
+		if (kvm_is_vmware_supported()) {
 			r = KVM_CAP_VMWARE_VALID_MASK;
+			if (kvm && kvm->arch.has_protected_state)
+				r &= ~KVM_VMWARE_ENABLE_HYPERCALL;
+		}
 		break;
 	default:
 		break;
@@ -7469,6 +7473,9 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(____kvm_emulate_hypercall);
 
 int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 {
+	if (kvm_is_vmware_hypercall(vcpu))
+		return kvm_vmware_hypercall(vcpu);
+
 	if (kvm_xen_hypercall_enabled(vcpu->kvm))
 		return kvm_xen_hypercall(vcpu);
 
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e43c15c6e5cc..c01bb6a612fe 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -193,6 +193,7 @@ struct kvm_exit_snp_req_certs {
 #define KVM_EXIT_ARM_SEA          41
 #define KVM_EXIT_ARM_LDST64B      42
 #define KVM_EXIT_SNP_REQ_CERTS    43
+#define KVM_EXIT_VMWARE           44
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -497,6 +498,10 @@ struct kvm_run {
 		} arm_sea;
 		/* KVM_EXIT_SNP_REQ_CERTS */
 		struct kvm_exit_snp_req_certs snp_req_certs;
+#ifdef __KVM_HAVE_VMWARE_EXIT
+		/* KVM_EXIT_VMWARE */
+		struct kvm_vmware_exit vmware;
+#endif
 		/* Fix the size of the union. */
 		char padding[256];
 	};
-- 
2.53.0

[PATCH v3 07/12] KVM: x86: Route nested VMware backdoors to KVM

[Zack Rusin]

Add KVM_VMWARE_ENABLE_NESTED_BACKDOOR.  When it and the backdoor flag
are both enabled, handle the #GP from a CPL3 VMware backdoor I/O access
in L2 in KVM, as for L1, even if L1 intercepts #GP.  Treat an access as
a backdoor access only if EAX holds VMware's magic value and DX selects
one of the two backdoor ports; L1 keeps receiving every other #GP.

VMware Tools use the I/O-port backdoor from user mode and rely on the
hypervisor ignoring the I/O permission checks for the backdoor ports.
When Hyper-V runs in the guest, e.g. for Windows virtualization-based
security (VBS), Windows and the Tools run as L2.  The access then
raises #GP and normal nested routing delivers it to Hyper-V, which
doesn't implement the backdoor.

Make the routing opt-in, because a nested hypervisor that implements
the backdoor itself, such as ESXi, has to keep receiving its guests'
accesses.  Cover only the I/O-port backdoor; L2 VMCALL and VMMCALL
handling doesn't change.

On VMX, claim the exit in nested_vmx_l0_wants_exit().  On SVM, a #GP
that L0 intercepts always reaches gp_interception() first, which
reinjects any #GP that KVM doesn't handle so that it reaches L1 if L1
intercepts #GP; check the filter there.

Link: https://lore.kernel.org/r/CADH9ctAGt_VriKA7Ch1L9U+xud-6M54GzaPOM_2sSA780TpAYw@mail.gmail.com
Link: https://lore.kernel.org/r/CABgObfb3Ttfg6H+_RpNQGSYKw9BLEwx3+EysXdL-wbpd1pkGHQ@mail.gmail.com
Link: https://lore.kernel.org/r/aAjrOgsooR4RYIJr@google.com
Link: https://lore.kernel.org/r/aIEnm7Y1VN-PCfF8@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 Documentation/virt/kvm/api.rst  |  9 +++++++++
 arch/x86/include/uapi/asm/kvm.h |  1 +
 arch/x86/kvm/svm/svm.c          |  7 +++++--
 arch/x86/kvm/vmware.c           |  8 ++++++++
 arch/x86/kvm/vmware.h           | 22 ++++++++++++++++++++++
 arch/x86/kvm/vmx/nested.c       |  5 +++++
 arch/x86/kvm/x86.c              |  3 ++-
 7 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index e2f0e329f748..14ce68600551 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -9293,6 +9293,15 @@ exception by setting ``vmware.hcall.exception.inject`` and filling in the
 validated vector, error code, and address fields.  When an exception is
 requested, KVM does not advance past the hypercall instruction.
 
+``KVM_VMWARE_ENABLE_NESTED_BACKDOOR`` makes KVM handle VMware backdoor I/O
+from a nested guest itself, even when the nested hypervisor intercepts #GP.
+KVM claims only a #GP raised at CPL3 while EAX holds VMware's magic value and
+DX holds one of the two backdoor ports; the nested hypervisor still receives
+every other #GP, and VMCALL/VMMCALL from the nested guest are not affected.
+This is needed, for example, for a Windows guest with Hyper-V VBS enabled:
+Hyper-V does not implement the VMware backdoor used by VMware Tools.  This
+flag has an effect only when ``KVM_VMWARE_ENABLE_BACKDOOR`` is also enabled.
+
 8. Other capabilities.
 ======================
 
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 3b9f53bce2d9..18c2d908eaa2 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -968,6 +968,7 @@ struct kvm_hyperv_eventfd {
 
 #define KVM_VMWARE_ENABLE_BACKDOOR	_BITULL(0)
 #define KVM_VMWARE_ENABLE_HYPERCALL	_BITULL(1)
+#define KVM_VMWARE_ENABLE_NESTED_BACKDOOR _BITULL(2)
 
 struct kvm_vmware_exit {
 #define KVM_EXIT_VMWARE_HCALL	1
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index a2196d8f60aa..bae7065fb360 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -2408,9 +2408,12 @@ static int gp_interception(struct kvm_vcpu *vcpu)
 
 	/*
 	 * VMware backdoor emulation on #GP interception only handles
-	 * IN{S}, OUT{S}, and RDPMC, and only for L1.
+	 * IN{S}, OUT{S}, and RDPMC.  Handle L2 accesses only when userspace
+	 * explicitly requests that L0 own the nested VMware backdoor.
 	 */
-	if (!kvm_is_vmware_backdoor_enabled(vcpu) || is_guest_mode(vcpu))
+	if (!kvm_is_vmware_backdoor_enabled(vcpu) ||
+	    (is_guest_mode(vcpu) &&
+	     !kvm_is_vmware_nested_backdoor(vcpu, svm_get_cpl(vcpu))))
 		goto reinject;
 
 	return kvm_emulate_instruction(vcpu, EMULTYPE_VMWARE_GP | EMULTYPE_NO_DECODE);
diff --git a/arch/x86/kvm/vmware.c b/arch/x86/kvm/vmware.c
index 8407dccf81fd..066f033e7ea3 100644
--- a/arch/x86/kvm/vmware.c
+++ b/arch/x86/kvm/vmware.c
@@ -117,6 +117,14 @@ int kvm_vmware_hypercall(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+bool kvm_is_vmware_nested_backdoor(struct kvm_vcpu *vcpu, u32 cpl)
+{
+	return kvm_is_vmware_nested_backdoor_enabled(vcpu->kvm) && cpl == 3 &&
+	       lower_32_bits(kvm_rax_read_raw(vcpu)) == VMWARE_HYPERVISOR_MAGIC &&
+	       kvm_is_vmware_io_port(vcpu, kvm_rdx_read_raw(vcpu));
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_is_vmware_nested_backdoor);
+
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
 {
 	return kvm_is_vmware_backdoor_enabled(vcpu) &&
diff --git a/arch/x86/kvm/vmware.h b/arch/x86/kvm/vmware.h
index de1dc8971ec6..e2ba0b8b4400 100644
--- a/arch/x86/kvm/vmware.h
+++ b/arch/x86/kvm/vmware.h
@@ -24,9 +24,19 @@ kvm_is_vmware_backdoor_enabled(struct kvm_vcpu *vcpu)
 }
 
 bool kvm_is_vmware_hypercall(struct kvm_vcpu *vcpu);
+
+static __always_inline bool
+kvm_is_vmware_nested_backdoor_enabled(struct kvm *kvm)
+{
+	return kvm_is_vmware_cap_enabled(kvm, KVM_VMWARE_ENABLE_BACKDOOR) &&
+	       kvm_is_vmware_cap_enabled(kvm,
+					 KVM_VMWARE_ENABLE_NESTED_BACKDOOR);
+}
+
 bool kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port);
 bool kvm_is_vmware_backdoor_pmc(struct kvm_vcpu *vcpu, u32 pmc_idx);
 bool kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode);
+bool kvm_is_vmware_nested_backdoor(struct kvm_vcpu *vcpu, u32 cpl);
 
 #else /* !CONFIG_KVM_VMWARE */
 
@@ -52,6 +62,12 @@ static __always_inline bool kvm_is_vmware_hypercall(struct kvm_vcpu *vcpu)
 	return false;
 }
 
+static __always_inline bool
+kvm_is_vmware_nested_backdoor_enabled(struct kvm *kvm)
+{
+	return false;
+}
+
 static __always_inline bool
 kvm_is_vmware_io_port(struct kvm_vcpu *vcpu, u16 port)
 {
@@ -70,6 +86,12 @@ kvm_is_vmware_backdoor_opcode(u8 opcode_len, u8 opcode)
 	return false;
 }
 
+static __always_inline bool
+kvm_is_vmware_nested_backdoor(struct kvm_vcpu *vcpu, u32 cpl)
+{
+	return false;
+}
+
 #endif /* CONFIG_KVM_VMWARE */
 
 void kvm_vmware_init_vm(struct kvm *kvm);
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 1a98f28651f7..2d1297036564 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -18,6 +18,7 @@
 #include "posted_intr.h"
 #include "sgx.h"
 #include "trace.h"
+#include "vmware.h"
 #include "vmx.h"
 #include "smm.h"
 #include "x86_ops.h"
@@ -6490,6 +6491,10 @@ static bool nested_vmx_l0_wants_exit(struct kvm_vcpu *vcpu,
 			return true;
 		else if (is_ve_fault(intr_info))
 			return true;
+		else if (is_gp_fault(intr_info) &&
+			 kvm_is_vmware_nested_backdoor(vcpu,
+						       vmx_get_cpl(vcpu)))
+			return true;
 		return false;
 	case EXIT_REASON_EXTERNAL_INTERRUPT:
 		return true;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bd4142763f10..6b6ed02d6063 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -119,7 +119,8 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_host);
 #define KVM_CAP_PMU_VALID_MASK KVM_PMU_CAP_DISABLE
 
 #define KVM_CAP_VMWARE_VALID_MASK (KVM_VMWARE_ENABLE_BACKDOOR | \
-				   KVM_VMWARE_ENABLE_HYPERCALL)
+				   KVM_VMWARE_ENABLE_HYPERCALL | \
+				   KVM_VMWARE_ENABLE_NESTED_BACKDOOR)
 
 #define KVM_GUESTDBG_VALID_MASK \
 	(KVM_GUESTDBG_ENABLE | \
-- 
2.53.0

[PATCH v3 08/12] KVM: selftests: Add helpers to run x86 guest code in user mode

[Zack Rusin]

Add vm_init_user_mode() and vcpu_enter_user_mode().  The former installs
flat 64-bit user code and data segments, makes the guest memory mapped
so far accessible to user mode, and gives the TSS a kernel stack and no
I/O permission bitmap; the latter switches a vCPU to CPL3.

The selftest infrastructure only sets up CPL0, but testing the VMware
backdoor requires guest code at CPL3, where the I/O permission checks
and CR4.PCE apply.  Provide the setup once instead of open coding it in
each test.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 .../selftests/kvm/include/x86/processor.h     |  7 ++
 .../testing/selftests/kvm/lib/x86/processor.c | 87 +++++++++++++++++++
 2 files changed, 94 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h
index 3e4351ccb359..f7fd4aafccae 100644
--- a/tools/testing/selftests/kvm/include/x86/processor.h
+++ b/tools/testing/selftests/kvm/include/x86/processor.h
@@ -1310,6 +1310,13 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector,
 
 gva_t vm_alloc_stack(struct kvm_vm *vm, int nr_pages);
 
+/* Flat 64-bit code and data segments for user mode, see vm_init_user_mode(). */
+#define USER_CS_SELECTOR	0x28
+#define USER_DS_SELECTOR	0x30
+
+void vm_init_user_mode(struct kvm_vm *vm);
+void vcpu_enter_user_mode(struct kvm_vcpu *vcpu);
+
 /*
  * Exception fixup morphs #DE to an arbitrary magic vector so that '0' can be
  * used to signal "no expcetion".
diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c
index ee8a1715d844..6099d386abe9 100644
--- a/tools/testing/selftests/kvm/lib/x86/processor.c
+++ b/tools/testing/selftests/kvm/lib/x86/processor.c
@@ -768,6 +768,93 @@ void vm_install_exception_handler(struct kvm_vm *vm, int vector,
 	handlers[vector] = (gva_t)handler;
 }
 
+struct tss64 {
+	u32 reserved0;
+	u64 rsp0;
+	u64 rsp1;
+	u64 rsp2;
+	u64 reserved1;
+	u64 ist[7];
+	u64 reserved2;
+	u16 reserved3;
+	u16 iomap_base;
+} __packed;
+
+static void kvm_seg_set_user_code_64bit(struct kvm_segment *segp)
+{
+	kvm_seg_set_kernel_code_64bit(segp);
+	segp->selector = USER_CS_SELECTOR | 3;
+	segp->dpl = 3;
+}
+
+static void kvm_seg_set_user_data_64bit(struct kvm_segment *segp)
+{
+	kvm_seg_set_kernel_data_64bit(segp);
+	segp->selector = USER_DS_SELECTOR | 3;
+	segp->dpl = 3;
+}
+
+static void virt_set_user_pte(struct kvm_vm *vm, gva_t gva)
+{
+	struct kvm_mmu *mmu = &vm->mmu;
+	u64 *pte = &mmu->pgd;
+	int level;
+
+	for (level = mmu->pgtable_levels; level >= PG_LEVEL_4K; level--) {
+		pte = virt_get_pte(vm, mmu, pte, gva, level);
+		TEST_ASSERT(is_present_pte(mmu, pte),
+			    "PTE not PRESENT at level %d for gva: 0x%lx",
+			    level, gva);
+		*pte |= PTE_USER_MASK(mmu);
+		if (is_huge_pte(mmu, pte))
+			break;
+	}
+}
+
+/*
+ * Prepare @vm for running guest code at CPL3: install flat user-mode code and
+ * data segments, make all memory that is currently mapped accessible to user
+ * mode, and give the TSS a kernel stack so that exceptions from user mode can
+ * be delivered.  The TSS has no I/O permission bitmap, i.e. user mode can't
+ * access I/O ports unless IOPL allows it.  Call this after allocating all
+ * memory, vCPUs included, that user mode needs.  All vCPUs share the kernel
+ * stack.
+ */
+void vm_init_user_mode(struct kvm_vm *vm)
+{
+	struct tss64 *tss = addr_gva2hva(vm, vm->arch.tss);
+	sparsebit_idx_t begin, end, page;
+	struct kvm_segment seg;
+
+	kvm_seg_set_user_code_64bit(&seg);
+	kvm_seg_fill_gdt_64bit(vm, &seg);
+	kvm_seg_set_user_data_64bit(&seg);
+	kvm_seg_fill_gdt_64bit(vm, &seg);
+
+	tss->rsp0 = vm_alloc_stack(vm, 1);
+	tss->iomap_base = sizeof(*tss);
+
+	sparsebit_for_each_set_range(vm->vpages_mapped, begin, end) {
+		for (page = begin; page <= end; page++)
+			virt_set_user_pte(vm, page << vm->page_shift);
+	}
+}
+
+/* Run @vcpu at CPL3 using the segments installed by vm_init_user_mode(). */
+void vcpu_enter_user_mode(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sregs sregs;
+
+	vcpu_sregs_get(vcpu, &sregs);
+	kvm_seg_set_user_code_64bit(&sregs.cs);
+	kvm_seg_set_user_data_64bit(&sregs.ds);
+	sregs.es = sregs.ds;
+	sregs.fs = sregs.ds;
+	sregs.gs = sregs.ds;
+	sregs.ss = sregs.ds;
+	vcpu_sregs_set(vcpu, &sregs);
+}
+
 void assert_on_unhandled_exception(struct kvm_vcpu *vcpu)
 {
 	struct ucall uc;
-- 
2.53.0

[PATCH v3 09/12] KVM: selftests: Port VMware backdoor coverage

[Zack Rusin]

Add vmware_backdoor_test, adapted from KVM-unit-tests' vmware_backdoors
test.  From CPL3, with CR4.PCE clear and no I/O permission bitmap in
the TSS, read the VMware pseudo-PMCs with RDPMC and issue backdoor
commands on the backdoor port with IN.  Verify that the pseudo-PMC
reads succeed and the port accesses exit to userspace only when the
backdoor is enabled and raise #GP otherwise, and that other PMC indices
and ports always raise #GP.  Repeat the pseudo-PMC checks at CPL3 with
CR4.PCE set and at CPL0, where RDPMC doesn't fault before KVM handles
it.  Skip the pseudo-PMC checks if KVM doesn't virtualize a PMU, as KVM
then injects #GP for every RDPMC.

Run the checks for a VM that relies on the enable_vmware_backdoor
default and for a VM that enables KVM_VMWARE_ENABLE_BACKDOOR, and
verify that KVM_CAP_X86_VMWARE rejects every flag it doesn't report.
KVM-unit-tests can only cover the module parameter, which applies to
every VM, while the selftest also covers the per-VM capability.

Suggested-by: Sean Christopherson <seanjc@google.com>
Link: https://lore.kernel.org/r/aIEoNpe-LFoH6LBm@google.com
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 MAINTAINERS                                   |   1 +
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/vmware_backdoor_test.c  | 272 ++++++++++++++++++
 3 files changed, 274 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_backdoor_test.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 017053e8f76e..39fa67a180f5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15050,6 +15050,7 @@ S:	Supported
 T:	git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
 T:	git https://github.com/kvm-x86/linux.git
 F:	arch/x86/kvm/vmware.*
+F:	tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
 
 KVM X86 Xen (KVM/Xen)
 M:	David Woodhouse <dwmw2@infradead.org>
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 95867977beb3..0ef08d7783ae 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -140,6 +140,7 @@ TEST_GEN_PROGS_x86 += x86/sync_regs_test
 TEST_GEN_PROGS_x86 += x86/ucna_injection_test
 TEST_GEN_PROGS_x86 += x86/userspace_io_test
 TEST_GEN_PROGS_x86 += x86/userspace_msr_exit_test
+TEST_GEN_PROGS_x86 += x86/vmware_backdoor_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_access_test
 TEST_GEN_PROGS_x86 += x86/vmx_apicv_updates_test
 TEST_GEN_PROGS_x86 += x86/vmx_exception_with_invalid_guest_state
diff --git a/tools/testing/selftests/kvm/x86/vmware_backdoor_test.c b/tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
new file mode 100644
index 000000000000..e4e87d2ce3c3
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
@@ -0,0 +1,272 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025-2026 Broadcom. All Rights Reserved. The term
+ * "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
+ *
+ * Adapted from kvm-unit-tests' x86/vmware_backdoors.c.
+ */
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define VMWARE_BACKDOOR_PMC_HOST_TSC		0x10000
+#define VMWARE_BACKDOOR_PMC_REAL_TIME		0x10001
+#define VMWARE_BACKDOOR_PMC_APPARENT_TIME	0x10002
+
+#define VMWARE_BACKDOOR_PORT	0x5658
+#define VMWARE_MAGIC		0x564d5868
+#define VMPORT_CMD_GETVERSION	0x0a
+#define VMPORT_CMD_ILLEGAL	0xfff
+#define VMPORT_DEFAULT_RETVAL	0xdeadbeef
+#define RANDOM_IO_PORT		0x1234
+#define RANDOM_PMC		0xfff
+
+struct insn_result {
+	u64 value;
+	u8 vector;
+};
+
+struct backdoor_results {
+	struct insn_result pmc[4];
+	struct insn_result port[3];
+};
+
+enum backdoor_mode {
+	USER_MODE,		/* CPL3, CR4.PCE clear */
+	USER_MODE_PCE,		/* CPL3, CR4.PCE set */
+	KERNEL_MODE,		/* CPL0 */
+};
+
+static struct backdoor_results results;
+static bool expect_backdoor;
+static bool test_pmcs;
+
+static struct insn_result guest_rdpmc(u32 pmc)
+{
+	struct insn_result result;
+	u32 eax;
+	u32 edx;
+	u64 error_code;
+
+	asm volatile(KVM_ASM_SAFE("rdpmc")
+		     : "=a"(eax), "=d"(edx),
+		       KVM_ASM_SAFE_OUTPUTS(result.vector, error_code)
+		     : "c"(pmc)
+		     : KVM_ASM_SAFE_CLOBBERS);
+	result.value = ((u64)edx << 32) | eax;
+	return result;
+}
+
+static struct insn_result guest_inl(u16 port, u32 command)
+{
+	struct insn_result result;
+	unsigned long rax = VMPORT_DEFAULT_RETVAL;
+	unsigned long rbx = VMPORT_DEFAULT_RETVAL;
+	unsigned long rcx = command;
+	unsigned long rdx = port;
+	u64 error_code;
+
+	asm volatile(KVM_ASM_SAFE("inl %%dx, %%eax")
+		     : "+a"(rax), "+b"(rbx), "+c"(rcx), "+d"(rdx),
+		       KVM_ASM_SAFE_OUTPUTS(result.vector, error_code)
+		     :
+		     : KVM_ASM_SAFE_CLOBBERS);
+	result.value = rbx;
+	return result;
+}
+
+static void guest_read_pmcs(void)
+{
+	results.pmc[0] = guest_rdpmc(VMWARE_BACKDOOR_PMC_HOST_TSC);
+	results.pmc[1] = guest_rdpmc(VMWARE_BACKDOOR_PMC_REAL_TIME);
+	results.pmc[2] = guest_rdpmc(VMWARE_BACKDOOR_PMC_APPARENT_TIME);
+	results.pmc[3] = guest_rdpmc(RANDOM_PMC);
+}
+
+static void guest_check_pmcs(void)
+{
+	int i;
+
+	for (i = 0; i < 3; i++) {
+		GUEST_ASSERT_EQ(results.pmc[i].vector,
+				expect_backdoor ? 0 : GP_VECTOR);
+		if (expect_backdoor)
+			GUEST_ASSERT_NE(results.pmc[i].value, 0);
+	}
+	GUEST_ASSERT_EQ(results.pmc[3].vector, GP_VECTOR);
+}
+
+static void guest_done_handler(struct ex_regs *regs)
+{
+	if (test_pmcs)
+		guest_check_pmcs();
+
+	GUEST_ASSERT_EQ(results.port[0].vector,
+			expect_backdoor ? 0 : GP_VECTOR);
+	GUEST_ASSERT_EQ(results.port[1].vector,
+			expect_backdoor ? 0 : GP_VECTOR);
+	if (expect_backdoor)
+		GUEST_ASSERT_EQ(results.port[0].value, VMWARE_MAGIC);
+	GUEST_ASSERT_EQ(results.port[2].vector, GP_VECTOR);
+
+	GUEST_DONE();
+}
+
+static void guest_user_code(void)
+{
+	if (test_pmcs)
+		guest_read_pmcs();
+
+	results.port[0] = guest_inl(VMWARE_BACKDOOR_PORT,
+				    VMPORT_CMD_GETVERSION);
+	results.port[1] = guest_inl(VMWARE_BACKDOOR_PORT,
+				    VMPORT_CMD_ILLEGAL);
+	results.port[2] = guest_inl(RANDOM_IO_PORT, VMPORT_CMD_ILLEGAL);
+
+	asm volatile("int3");
+}
+
+/* At CPL0, only the pseudo-PMCs depend on the backdoor. */
+static void guest_kernel_code(void)
+{
+	guest_read_pmcs();
+	guest_check_pmcs();
+	GUEST_DONE();
+}
+
+static void handle_vmware_port(struct kvm_vcpu *vcpu)
+{
+	struct kvm_run *run = vcpu->run;
+	struct kvm_regs regs;
+	u32 *data;
+
+	TEST_ASSERT_EQ(run->io.direction, KVM_EXIT_IO_IN);
+	TEST_ASSERT_EQ(run->io.size, 4);
+	TEST_ASSERT_EQ(run->io.count, 1);
+	TEST_ASSERT_EQ(run->io.port, VMWARE_BACKDOOR_PORT);
+
+	data = (void *)run + run->io.data_offset;
+	*data = 0;
+
+	vcpu_regs_get(vcpu, &regs);
+	regs.rbx = VMWARE_MAGIC;
+	vcpu_regs_set(vcpu, &regs);
+}
+
+static void test_unsupported_flags(u64 supported)
+{
+	struct kvm_vm *vm = vm_create_barebones();
+	int bit, ret;
+
+	for (bit = 0; bit < 64; bit++) {
+		if (supported & BIT_ULL(bit))
+			continue;
+
+		errno = 0;
+		ret = __vm_enable_cap(vm, KVM_CAP_X86_VMWARE, BIT_ULL(bit));
+		TEST_ASSERT(ret == -1 && errno == EINVAL,
+			    "Unsupported VMware flag %d: ret=%d errno=%d",
+			    bit, ret, errno);
+	}
+	kvm_vm_free(vm);
+}
+
+static void run_backdoor_test(bool enable_cap, bool expected,
+			      enum backdoor_mode mode)
+{
+	struct kvm_sregs sregs;
+	struct idt_entry *idt;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int port_exits = 0;
+
+	expect_backdoor = expected;
+	memset(&results, 0, sizeof(results));
+
+	vm = vm_create(1);
+	sync_global_to_guest(vm, expect_backdoor);
+	sync_global_to_guest(vm, test_pmcs);
+	sync_global_to_guest(vm, results);
+	if (enable_cap)
+		vm_enable_cap(vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_BACKDOOR);
+	if (mode == KERNEL_MODE) {
+		vcpu = vm_vcpu_add(vm, 0, guest_kernel_code);
+	} else {
+		vcpu = vm_vcpu_add(vm, 0, guest_user_code);
+
+		idt = addr_gva2hva(vm, vm->arch.idt);
+		idt[BP_VECTOR].dpl = 3;
+		vm_install_exception_handler(vm, BP_VECTOR, guest_done_handler);
+		vm_init_user_mode(vm);
+		vcpu_enter_user_mode(vcpu);
+	}
+
+	if (mode == USER_MODE_PCE) {
+		vcpu_sregs_get(vcpu, &sregs);
+		sregs.cr4 |= X86_CR4_PCE;
+		vcpu_sregs_set(vcpu, &sregs);
+	}
+
+	for (;;) {
+		struct kvm_run *run = vcpu->run;
+		struct ucall uc;
+
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		if (run->io.port == VMWARE_BACKDOOR_PORT) {
+			TEST_ASSERT(expected,
+				    "Disabled VMware backdoor reached userspace");
+			handle_vmware_port(vcpu);
+			port_exits++;
+			continue;
+		}
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			break;
+		case UCALL_DONE:
+			TEST_ASSERT_EQ(port_exits,
+				       expected && mode != KERNEL_MODE ? 2 : 0);
+			kvm_vm_free(vm);
+			return;
+		default:
+			TEST_FAIL("Unexpected ucall 0x%lx", uc.cmd);
+		}
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	unsigned int supported = kvm_check_cap(KVM_CAP_X86_VMWARE);
+	enum backdoor_mode mode;
+	bool module_default;
+
+	TEST_REQUIRE(supported & KVM_VMWARE_ENABLE_BACKDOOR);
+
+	test_unsupported_flags(supported);
+
+	/* KVM serves the pseudo-PMCs only if it virtualizes a PMU. */
+	test_pmcs = kvm_is_pmu_enabled();
+	if (!test_pmcs)
+		ksft_print_msg("vPMU disabled, skipping pseudo-PMC checks\n");
+
+	module_default = get_kvm_param_bool("enable_vmware_backdoor");
+	run_backdoor_test(false, module_default, USER_MODE);
+	run_backdoor_test(true, true, USER_MODE);
+
+	/*
+	 * RDPMC at CPL0, or at CPL3 with CR4.PCE set, doesn't raise the #GP
+	 * that the emulator handles, so check the pseudo-PMCs there as well.
+	 */
+	if (test_pmcs) {
+		for (mode = USER_MODE_PCE; mode <= KERNEL_MODE; mode++) {
+			run_backdoor_test(false, module_default, mode);
+			run_backdoor_test(true, true, mode);
+		}
+	}
+	return 0;
+}
-- 
2.53.0

[PATCH v3 10/12] KVM: selftests: Test VMware hypercall exits

[Zack Rusin]

Add vmware_hypercall_test to cover KVM_VMWARE_ENABLE_HYPERCALL and
KVM_EXIT_VMWARE, and enable VMware hypercall exits in xen_vmcall_test.

Check the exit's calling convention and CPL reporting at CPL0 and CPL3
and in 64-bit and compatibility mode, that all GPR inputs are visible
in full width through KVM_SYNC_X86_REGS, and that synchronized GPR and
RFLAGS outputs complete the call.  Check that #UD, #GP and #PF
completions keep RIP at the hypercall, deliver the requested error code
and CR2, and can also return modified registers.  Check that invalid
completions fail with -EINVAL and that the call is retried afterwards,
that the flags can't be enabled after vCPU creation, that hypercalls
without the magic value still reach KVM, and that protected VM types
neither advertise nor accept the flag.

xen_vmcall_test already issues Xen and Hyper-V hypercalls from one
guest; enabling VMware hypercall exits there verifies that the magic
value filter leaves both alone.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 MAINTAINERS                                   |   1 +
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/vmware_hypercall_test.c | 388 ++++++++++++++++++
 .../selftests/kvm/x86/xen_vmcall_test.c       |  12 +-
 4 files changed, 401 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_hypercall_test.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 39fa67a180f5..38a735c9221e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15051,6 +15051,7 @@ T:	git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
 T:	git https://github.com/kvm-x86/linux.git
 F:	arch/x86/kvm/vmware.*
 F:	tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
+F:	tools/testing/selftests/kvm/x86/vmware_hypercall_test.c
 
 KVM X86 Xen (KVM/Xen)
 M:	David Woodhouse <dwmw2@infradead.org>
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 0ef08d7783ae..fe7ba905facd 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -141,6 +141,7 @@ TEST_GEN_PROGS_x86 += x86/ucna_injection_test
 TEST_GEN_PROGS_x86 += x86/userspace_io_test
 TEST_GEN_PROGS_x86 += x86/userspace_msr_exit_test
 TEST_GEN_PROGS_x86 += x86/vmware_backdoor_test
+TEST_GEN_PROGS_x86 += x86/vmware_hypercall_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_access_test
 TEST_GEN_PROGS_x86 += x86/vmx_apicv_updates_test
 TEST_GEN_PROGS_x86 += x86/vmx_exception_with_invalid_guest_state
diff --git a/tools/testing/selftests/kvm/x86/vmware_hypercall_test.c b/tools/testing/selftests/kvm/x86/vmware_hypercall_test.c
new file mode 100644
index 000000000000..d882d3661022
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmware_hypercall_test.c
@@ -0,0 +1,388 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025-2026 Broadcom. All Rights Reserved. The term
+ * "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
+ */
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define HYPERCALL_INSN_SIZE	3
+#define VMWARE_MAGIC		0x564d5868
+#define ARG_RAX			(0xdeadbeef00000000ULL | VMWARE_MAGIC)
+#define ARGVALUE(x)		(0xdeadbeef5a5a0000ULL + (x))
+#define OUTVALUE(x)		(0x1234500000000000ULL + (x))
+#define FAULT_ADDRESS		0xdeadbeefcafef000ULL
+#define FAULT_ERROR_CODE	0x5a
+
+static const u8 vmx_vmcall[HYPERCALL_INSN_SIZE] = { 0x0f, 0x01, 0xc1 };
+static const u8 svm_vmmcall[HYPERCALL_INSN_SIZE] = { 0x0f, 0x01, 0xd9 };
+
+struct vmware_test {
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	gva_t insn_gva;
+};
+
+static void unused_guest_code(void)
+{
+	GUEST_DONE();
+}
+
+static const u8 *native_hypercall_insn(void)
+{
+	if (host_cpu_is_intel)
+		return vmx_vmcall;
+	if (host_cpu_is_amd_compatible)
+		return svm_vmmcall;
+
+	TEST_FAIL("VMware hypercall test requires an Intel or AMD-compatible CPU");
+}
+
+static void set_compat_mode(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sregs sregs;
+
+	vcpu_sregs_get(vcpu, &sregs);
+	sregs.cs.l = 0;
+	sregs.cs.db = 1;
+	vcpu_sregs_set(vcpu, &sregs);
+}
+
+static struct vmware_test create_test(bool cpl3, bool compat, u64 rax)
+{
+	struct vmware_test test;
+	struct kvm_regs regs;
+	u8 *insn;
+
+	test.vm = vm_create(1);
+	TEST_ASSERT(vm_check_cap(test.vm, KVM_CAP_X86_VMWARE) &
+		    KVM_VMWARE_ENABLE_HYPERCALL,
+		    "Ordinary VM does not advertise VMware hypercall exits");
+	vm_enable_cap(test.vm, KVM_CAP_X86_VMWARE,
+		      KVM_VMWARE_ENABLE_HYPERCALL);
+	test.vcpu = vm_vcpu_add(test.vm, 0, unused_guest_code);
+	test.insn_gva = vm_alloc_page(test.vm);
+
+	insn = addr_gva2hva(test.vm, test.insn_gva);
+	memcpy(insn, native_hypercall_insn(), HYPERCALL_INSN_SIZE);
+	insn[HYPERCALL_INSN_SIZE] = 0xf4; /* HLT */
+
+	vcpu_regs_get(test.vcpu, &regs);
+	regs.rax = rax;
+	regs.rbx = ARGVALUE(2);
+	regs.rcx = ARGVALUE(3);
+	regs.rdx = ARGVALUE(4);
+	regs.rsi = ARGVALUE(5);
+	regs.rdi = ARGVALUE(6);
+	regs.rbp = ARGVALUE(7);
+	regs.rip = test.insn_gva;
+	vcpu_regs_set(test.vcpu, &regs);
+
+	if (cpl3) {
+		vm_init_user_mode(test.vm);
+		vcpu_enter_user_mode(test.vcpu);
+	}
+	if (compat)
+		set_compat_mode(test.vcpu);
+
+	return test;
+}
+
+static void destroy_test(struct vmware_test *test)
+{
+	kvm_vm_free(test->vm);
+}
+
+static void assert_hypercall_state(struct vmware_test *test, bool cpl3,
+				   bool compat)
+{
+	struct kvm_vmware_exit *vmware = &test->vcpu->run->vmware;
+	struct kvm_regs *regs = &test->vcpu->run->s.regs.regs;
+
+	TEST_ASSERT_EQ(vmware->type, KVM_EXIT_VMWARE_HCALL);
+	TEST_ASSERT_EQ(vmware->hcall.longmode, !compat);
+	TEST_ASSERT_EQ(vmware->hcall.cpl, cpl3 ? 3 : 0);
+	TEST_ASSERT_EQ(regs->rax, ARG_RAX);
+	TEST_ASSERT_EQ(regs->rbx, ARGVALUE(2));
+	TEST_ASSERT_EQ(regs->rcx, ARGVALUE(3));
+	TEST_ASSERT_EQ(regs->rdx, ARGVALUE(4));
+	TEST_ASSERT_EQ(regs->rsi, ARGVALUE(5));
+	TEST_ASSERT_EQ(regs->rdi, ARGVALUE(6));
+	TEST_ASSERT_EQ(regs->rbp, ARGVALUE(7));
+}
+
+static void run_to_vmware_exit(struct vmware_test *test, bool cpl3,
+			       bool compat)
+{
+	test->vcpu->run->kvm_valid_regs = KVM_SYNC_X86_REGS;
+	vcpu_run(test->vcpu);
+	TEST_ASSERT_KVM_EXIT_REASON(test->vcpu, KVM_EXIT_VMWARE);
+	TEST_ASSERT(test->vcpu->run->kvm_valid_regs & KVM_SYNC_X86_REGS,
+		    "KVM did not return synchronized GPRs");
+	assert_hypercall_state(test, cpl3, compat);
+}
+
+static void compare_regs(const struct kvm_regs *actual,
+			 const struct kvm_regs *expected)
+{
+	TEST_ASSERT_EQ(actual->rax, expected->rax);
+	TEST_ASSERT_EQ(actual->rbx, expected->rbx);
+	TEST_ASSERT_EQ(actual->rcx, expected->rcx);
+	TEST_ASSERT_EQ(actual->rdx, expected->rdx);
+	TEST_ASSERT_EQ(actual->rsi, expected->rsi);
+	TEST_ASSERT_EQ(actual->rdi, expected->rdi);
+	TEST_ASSERT_EQ(actual->rsp, expected->rsp);
+	TEST_ASSERT_EQ(actual->rbp, expected->rbp);
+	TEST_ASSERT_EQ(actual->r8, expected->r8);
+	TEST_ASSERT_EQ(actual->r9, expected->r9);
+	TEST_ASSERT_EQ(actual->r10, expected->r10);
+	TEST_ASSERT_EQ(actual->r11, expected->r11);
+	TEST_ASSERT_EQ(actual->r12, expected->r12);
+	TEST_ASSERT_EQ(actual->r13, expected->r13);
+	TEST_ASSERT_EQ(actual->r14, expected->r14);
+	TEST_ASSERT_EQ(actual->r15, expected->r15);
+	TEST_ASSERT_EQ(actual->rip, expected->rip);
+	TEST_ASSERT_EQ(actual->rflags, expected->rflags);
+}
+
+static void test_synchronized_output(bool compat)
+{
+	struct vmware_test test = create_test(false, compat, ARG_RAX);
+	struct kvm_run *run = test.vcpu->run;
+	struct kvm_regs actual;
+	struct kvm_regs expected;
+	int ret;
+
+	run_to_vmware_exit(&test, false, compat);
+	expected = run->s.regs.regs;
+	expected.rax = OUTVALUE(1);
+	expected.rbx = OUTVALUE(2);
+	expected.rcx = OUTVALUE(3);
+	expected.rdx = OUTVALUE(4);
+	expected.rsi = OUTVALUE(5);
+	expected.rdi = OUTVALUE(6);
+	expected.rsp = OUTVALUE(7);
+	expected.rbp = OUTVALUE(8);
+	expected.r8 = OUTVALUE(9);
+	expected.r9 = OUTVALUE(10);
+	expected.r10 = OUTVALUE(11);
+	expected.r11 = OUTVALUE(12);
+	expected.r12 = OUTVALUE(13);
+	expected.r13 = OUTVALUE(14);
+	expected.r14 = OUTVALUE(15);
+	expected.r15 = OUTVALUE(16);
+	expected.rflags = X86_EFLAGS_CF | X86_EFLAGS_ZF;
+	run->s.regs.regs = expected;
+	run->kvm_dirty_regs = KVM_SYNC_X86_REGS;
+	run->immediate_exit = 1;
+
+	errno = 0;
+	ret = __vcpu_run(test.vcpu);
+	run->immediate_exit = 0;
+	TEST_ASSERT(ret == -1 && errno == EINTR,
+		    "Completion with immediate_exit failed: ret=%d errno=%d",
+		    ret, errno);
+
+	expected.rip += HYPERCALL_INSN_SIZE;
+	expected.rflags |= X86_EFLAGS_FIXED;
+	vcpu_regs_get(test.vcpu, &actual);
+	compare_regs(&actual, &expected);
+	destroy_test(&test);
+}
+
+static void test_valid_exception(u32 vector, u32 error_code, u64 address,
+				 bool dirty_regs)
+{
+	struct vmware_test test = create_test(false, false, ARG_RAX);
+	struct kvm_run *run = test.vcpu->run;
+	struct kvm_vcpu_events events;
+	struct kvm_sregs sregs;
+	struct kvm_regs expected;
+	struct kvm_regs regs;
+	u64 fault_rip;
+	int ret;
+
+	run_to_vmware_exit(&test, false, false);
+	expected = run->s.regs.regs;
+	fault_rip = run->s.regs.regs.rip;
+	run->vmware.hcall.exception.inject = 1;
+	run->vmware.hcall.exception.vector = vector;
+	run->vmware.hcall.exception.error_code = error_code;
+	run->vmware.hcall.exception.address = address;
+	if (dirty_regs) {
+		expected.rax = OUTVALUE(1);
+		expected.rcx = OUTVALUE(3);
+		expected.rdx = OUTVALUE(4);
+		expected.rsi = OUTVALUE(5);
+		expected.rdi = OUTVALUE(6);
+		expected.rbp = OUTVALUE(8);
+		expected.rflags = X86_EFLAGS_CF | X86_EFLAGS_ZF |
+				  X86_EFLAGS_FIXED;
+		run->s.regs.regs = expected;
+		run->kvm_dirty_regs = KVM_SYNC_X86_REGS;
+	} else {
+		run->kvm_dirty_regs = 0;
+	}
+	run->immediate_exit = 1;
+
+	errno = 0;
+	ret = __vcpu_run(test.vcpu);
+	run->immediate_exit = 0;
+	TEST_ASSERT(ret == -1 && errno == EINTR,
+		    "Exception completion failed: ret=%d errno=%d", ret, errno);
+
+	vcpu_regs_get(test.vcpu, &regs);
+	compare_regs(&regs, &expected);
+	TEST_ASSERT_EQ(regs.rip, fault_rip);
+	/*
+	 * KVM_GET_VCPU_EVENTS delivers a pending exception payload when the
+	 * exception-payload capability is disabled.  Read CR2 afterwards.
+	 */
+	vcpu_events_get(test.vcpu, &events);
+	TEST_ASSERT(events.exception.pending || events.exception.injected,
+		    "Exception %u was not queued", vector);
+	TEST_ASSERT_EQ(events.exception.nr, vector);
+	TEST_ASSERT_EQ(events.exception.has_error_code, vector != UD_VECTOR);
+	if (vector != UD_VECTOR)
+		TEST_ASSERT_EQ(events.exception.error_code, error_code);
+	if (vector == PF_VECTOR) {
+		vcpu_sregs_get(test.vcpu, &sregs);
+		TEST_ASSERT_EQ(sregs.cr2, address);
+	}
+	destroy_test(&test);
+}
+
+static void test_invalid_completion(u32 inject, u32 vector, u32 error_code,
+				    u64 address, u32 pad1, u32 pad2)
+{
+	struct vmware_test test = create_test(false, false, ARG_RAX);
+	struct kvm_run *run = test.vcpu->run;
+	u64 fault_rip;
+	int ret;
+
+	run_to_vmware_exit(&test, false, false);
+	fault_rip = run->s.regs.regs.rip;
+	run->vmware.pad1 = pad1;
+	run->vmware.hcall.exception.inject = inject;
+	run->vmware.hcall.exception.pad2 = pad2;
+	run->vmware.hcall.exception.vector = vector;
+	run->vmware.hcall.exception.error_code = error_code;
+	run->vmware.hcall.exception.address = address;
+	run->kvm_dirty_regs = 0;
+
+	errno = 0;
+	ret = _vcpu_run(test.vcpu);
+	TEST_ASSERT(ret == -1 && errno == EINVAL,
+		    "Invalid completion was accepted: ret=%d errno=%d", ret, errno);
+
+	run->vmware.pad1 = 0;
+	run->vmware.hcall.exception.pad2 = 0;
+	run->exit_reason = KVM_EXIT_UNKNOWN;
+	run_to_vmware_exit(&test, false, false);
+	TEST_ASSERT_EQ(run->s.regs.regs.rip, fault_rip);
+	destroy_test(&test);
+}
+
+static void test_non_vmware_hypercall(void)
+{
+	struct vmware_test test = create_test(false, false,
+					      KVM_HC_MAP_GPA_RANGE);
+	struct kvm_regs regs;
+
+	vm_enable_cap(test.vm, KVM_CAP_EXIT_HYPERCALL,
+		      BIT(KVM_HC_MAP_GPA_RANGE));
+	vcpu_regs_get(test.vcpu, &regs);
+	regs.rbx = PAGE_SIZE;
+	regs.rcx = 1;
+	regs.rdx = 0;
+	vcpu_regs_set(test.vcpu, &regs);
+
+	vcpu_run(test.vcpu);
+	TEST_ASSERT_KVM_EXIT_REASON(test.vcpu, KVM_EXIT_HYPERCALL);
+	TEST_ASSERT_EQ(test.vcpu->run->hypercall.nr, KVM_HC_MAP_GPA_RANGE);
+	destroy_test(&test);
+}
+
+static void test_protected_rejection(void)
+{
+	struct kvm_vm *vm;
+	unsigned int supported = kvm_check_cap(KVM_CAP_VM_TYPES);
+	int vmware_caps;
+	u32 type;
+	int ret;
+
+	if (supported & BIT(KVM_X86_SEV_ES_VM)) {
+		type = KVM_X86_SEV_ES_VM;
+	} else if (supported & BIT(KVM_X86_SNP_VM)) {
+		type = KVM_X86_SNP_VM;
+	} else if (supported & BIT(KVM_X86_TDX_VM)) {
+		type = KVM_X86_TDX_VM;
+	} else {
+		ksft_print_msg("No protected x86 VM type, skipping rejection test\n");
+		return;
+	}
+
+	vm = vm_create_barebones_type(type);
+	vmware_caps = vm_check_cap(vm, KVM_CAP_X86_VMWARE);
+	TEST_ASSERT(!(vmware_caps & KVM_VMWARE_ENABLE_HYPERCALL),
+		    "Protected VM advertises VMware hypercall exits");
+	TEST_ASSERT(vmware_caps & KVM_VMWARE_ENABLE_BACKDOOR,
+		    "Protected VM does not advertise the VMware backdoor");
+	errno = 0;
+	ret = __vm_enable_cap(vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_HYPERCALL);
+	TEST_ASSERT(ret == -1 && errno == EINVAL,
+		    "Protected VM accepted VMware hypercall exits");
+	vm_enable_cap(vm, KVM_CAP_X86_VMWARE, KVM_VMWARE_ENABLE_BACKDOOR);
+	kvm_vm_free(vm);
+}
+
+int main(int argc, char *argv[])
+{
+	struct vmware_test test;
+	unsigned int supported;
+	int ret;
+
+	supported = kvm_check_cap(KVM_CAP_X86_VMWARE);
+	TEST_REQUIRE(supported & KVM_VMWARE_ENABLE_HYPERCALL);
+
+	test = create_test(false, false, ARG_RAX);
+	errno = 0;
+	ret = __vm_enable_cap(test.vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_BACKDOOR);
+	TEST_ASSERT(ret == -1 && errno == EINVAL,
+		    "Enabling VMware flags after vCPU creation should fail");
+	destroy_test(&test);
+
+	test_synchronized_output(false);
+	test_synchronized_output(true);
+
+	test = create_test(true, false, ARG_RAX);
+	run_to_vmware_exit(&test, true, false);
+	destroy_test(&test);
+
+	test = create_test(false, true, ARG_RAX);
+	run_to_vmware_exit(&test, false, true);
+	destroy_test(&test);
+
+	test_valid_exception(UD_VECTOR, 0, 0, false);
+	test_valid_exception(GP_VECTOR, FAULT_ERROR_CODE, 0, false);
+	test_valid_exception(PF_VECTOR, FAULT_ERROR_CODE, FAULT_ADDRESS, true);
+
+	test_invalid_completion(2, UD_VECTOR, 0, 0, 0, 0);
+	test_invalid_completion(1, DE_VECTOR, 0, 0, 0, 0);
+	test_invalid_completion(1, NMI_VECTOR, 0, 0, 0, 0);
+	test_invalid_completion(1, DF_VECTOR, 0, 0, 0, 0);
+	test_invalid_completion(1, 32, 0, 0, 0, 0);
+	test_invalid_completion(1, UD_VECTOR, 1, 0, 0, 0);
+	test_invalid_completion(1, UD_VECTOR, 0, FAULT_ADDRESS, 0, 0);
+	test_invalid_completion(1, GP_VECTOR, 0, FAULT_ADDRESS, 0, 0);
+	test_invalid_completion(0, 0, 0, 0, 1, 0);
+	test_invalid_completion(0, 0, 0, 0, 0, 1);
+
+	test_non_vmware_hypercall();
+	test_protected_rejection();
+	return 0;
+}
diff --git a/tools/testing/selftests/kvm/x86/xen_vmcall_test.c b/tools/testing/selftests/kvm/x86/xen_vmcall_test.c
index 2585087cdf5c..096de77884a8 100644
--- a/tools/testing/selftests/kvm/x86/xen_vmcall_test.c
+++ b/tools/testing/selftests/kvm/x86/xen_vmcall_test.c
@@ -81,6 +81,7 @@ static void guest_code(void)
 
 int main(int argc, char *argv[])
 {
+	unsigned int vmware_caps;
 	unsigned int xen_caps;
 	struct kvm_vcpu *vcpu;
 	struct kvm_vm *vm;
@@ -88,7 +89,16 @@ int main(int argc, char *argv[])
 	xen_caps = kvm_check_cap(KVM_CAP_XEN_HVM);
 	TEST_REQUIRE(xen_caps & KVM_XEN_HVM_CONFIG_INTERCEPT_HCALL);
 
-	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm = vm_create(1);
+	vmware_caps = kvm_check_cap(KVM_CAP_X86_VMWARE);
+	if (vmware_caps & KVM_VMWARE_ENABLE_HYPERCALL) {
+		vm_enable_cap(vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_HYPERCALL);
+	} else {
+		ksft_print_msg("VMware hypercall exits unavailable; "
+			       "skipping coexistence coverage\n");
+	}
+	vcpu = vm_vcpu_add(vm, 0, guest_code);
 	vcpu_set_hv_cpuid(vcpu);
 
 	struct kvm_xen_hvm_config hvmc = {
-- 
2.53.0

[PATCH v3 11/12] KVM: selftests: Test VMware hypercall exits with nested Hyper-V flushes

[Zack Rusin]

Enable VMware hypercall exits in hyperv_evmcs and hyperv_svm_test when
KVM supports them, and issue the first L2 TLB flush hypercall with
VMware's magic value in RAX.

When L1 enables it, L0 handles Hyper-V's L2 TLB flush hypercall itself,
through the same dispatch that forwards VMware hypercalls.  The 64-bit
Hyper-V calling convention doesn't use RAX as an input, so a flush with
VMware's magic value left in RAX must still be handled as a Hyper-V
hypercall instead of exiting to userspace.

Add hyperv_hypercall_rax() to issue a Hyper-V hypercall with a given
value in RAX, and enable the VMware flag again when hyperv_evmcs
restores its state into a new VM.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 .../selftests/kvm/include/x86/hyperv.h        | 21 +++++++++++++++
 .../testing/selftests/kvm/x86/hyperv_evmcs.c  | 27 +++++++++++++++----
 .../selftests/kvm/x86/hyperv_svm_test.c       | 21 ++++++++++-----
 3 files changed, 58 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/x86/hyperv.h b/tools/testing/selftests/kvm/include/x86/hyperv.h
index 78003f5a22f3..06d14be3c4ee 100644
--- a/tools/testing/selftests/kvm/include/x86/hyperv.h
+++ b/tools/testing/selftests/kvm/include/x86/hyperv.h
@@ -286,6 +286,27 @@ static inline void hyperv_hypercall(u64 control, gva_t input_address,
 	GUEST_ASSERT((hv_status & 0xffff) == 0);
 }
 
+/*
+ * Issue a Hyper-V hypercall with @rax in RAX, which the 64-bit calling
+ * convention doesn't use as an input, and assert that it succeeded.
+ */
+static inline void hyperv_hypercall_rax(u64 control, gva_t input_address,
+					gva_t output_address, u64 rax)
+{
+	u64 error_code;
+	u8 vector;
+
+	asm volatile("mov %[output_address], %%r8\n\t"
+		     KVM_ASM_SAFE("vmcall")
+		     : "+a" (rax), "+c" (control), "+d" (input_address),
+		       KVM_ASM_SAFE_OUTPUTS(vector, error_code)
+		     : [output_address] "r"(output_address)
+		     : "cc", "memory", "r8", KVM_ASM_SAFE_CLOBBERS);
+
+	GUEST_ASSERT(!vector);
+	GUEST_ASSERT((rax & 0xffff) == 0);
+}
+
 /* Write 'Fast' hypercall input 'data' to the first 'n_sse_regs' SSE regs */
 static inline void hyperv_write_xmm_input(void *data, int n_sse_regs)
 {
diff --git a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c b/tools/testing/selftests/kvm/x86/hyperv_evmcs.c
index c81f2dd4f2a1..cdd4e523c62a 100644
--- a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c
+++ b/tools/testing/selftests/kvm/x86/hyperv_evmcs.c
@@ -18,6 +18,8 @@
 #include "hyperv.h"
 #include "vmx.h"
 
+#define VMWARE_HYPERVISOR_MAGIC	0x564d5868
+
 static int ud_count;
 
 static void guest_ud_handler(struct ex_regs *regs)
@@ -63,9 +65,13 @@ void l2_guest_code(void)
 	vmcall();
 	rdmsr_from_l2(MSR_GS_BASE); /* intercepted */
 
-	/* L2 TLB flush tests */
-	hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0,
-			 HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS);
+	/*
+	 * L2 TLB flush tests.  Leave VMware's magic value in RAX to check that
+	 * VMware hypercall exits don't claim the flush that L0 handles.
+	 */
+	hyperv_hypercall_rax(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0,
+			     HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS,
+			     VMWARE_HYPERVISOR_MAGIC);
 	rdmsr_from_l2(MSR_FS_BASE);
 	/*
 	 * Note: hypercall status (RAX) is not preserved correctly by L1 after
@@ -209,6 +215,13 @@ void inject_nmi(struct kvm_vcpu *vcpu)
 	vcpu_events_set(vcpu, &events);
 }
 
+static void vm_enable_vmware_hypercall(struct kvm_vm *vm)
+{
+	if (kvm_check_cap(KVM_CAP_X86_VMWARE) & KVM_VMWARE_ENABLE_HYPERCALL)
+		vm_enable_cap(vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_HYPERCALL);
+}
+
 static struct kvm_vcpu *save_restore_vm(struct kvm_vm *vm,
 					struct kvm_vcpu *vcpu)
 {
@@ -222,7 +235,9 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *vm,
 	kvm_vm_release(vm);
 
 	/* Restore state in a new VM.  */
-	vcpu = vm_recreate_with_one_vcpu(vm);
+	kvm_vm_restart(vm);
+	vm_enable_vmware_hypercall(vm);
+	vcpu = vm_vcpu_recreate(vm, 0);
 	vcpu_set_hv_cpuid(vcpu);
 	vcpu_enable_evmcs(vcpu);
 	vcpu_load_state(vcpu, state);
@@ -251,7 +266,9 @@ int main(int argc, char *argv[])
 	TEST_REQUIRE(kvm_has_cap(KVM_CAP_HYPERV_ENLIGHTENED_VMCS));
 	TEST_REQUIRE(kvm_hv_cpu_has(HV_X64_NESTED_DIRECT_FLUSH));
 
-	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm = vm_create(1);
+	vm_enable_vmware_hypercall(vm);
+	vcpu = vm_vcpu_add(vm, 0, guest_code);
 
 	hcall_page = vm_alloc_pages(vm, 1);
 	memset(addr_gva2hva(vm, hcall_page), 0x0,  getpagesize());
diff --git a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c b/tools/testing/selftests/kvm/x86/hyperv_svm_test.c
index 3a3dc31072e8..fe0762ae8260 100644
--- a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c
+++ b/tools/testing/selftests/kvm/x86/hyperv_svm_test.c
@@ -18,6 +18,8 @@
 #include "svm_util.h"
 #include "hyperv.h"
 
+#define VMWARE_HYPERVISOR_MAGIC	0x564d5868
+
 /* Exit to L1 from L2 with RDMSR instruction */
 static inline void rdmsr_from_l2(u32 msr)
 {
@@ -44,11 +46,14 @@ void l2_guest_code(void)
 
 	GUEST_SYNC(5);
 
-	/* L2 TLB flush tests */
-	hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE |
-			 HV_HYPERCALL_FAST_BIT, 0x0,
-			 HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |
-			 HV_FLUSH_ALL_PROCESSORS);
+	/*
+	 * L2 TLB flush tests.  Leave VMware's magic value in RAX to check that
+	 * VMware hypercall exits don't claim the flush that L0 handles.
+	 */
+	hyperv_hypercall_rax(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE |
+			     HV_HYPERCALL_FAST_BIT, 0x0,
+			     HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |
+			     HV_FLUSH_ALL_PROCESSORS, VMWARE_HYPERVISOR_MAGIC);
 	rdmsr_from_l2(MSR_FS_BASE);
 	/*
 	 * Note: hypercall status (RAX) is not preserved correctly by L1 after
@@ -156,7 +161,11 @@ int main(int argc, char *argv[])
 	TEST_REQUIRE(kvm_hv_cpu_has(HV_X64_NESTED_DIRECT_FLUSH));
 
 	/* Create VM */
-	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm = vm_create(1);
+	if (kvm_check_cap(KVM_CAP_X86_VMWARE) & KVM_VMWARE_ENABLE_HYPERCALL)
+		vm_enable_cap(vm, KVM_CAP_X86_VMWARE,
+			      KVM_VMWARE_ENABLE_HYPERCALL);
+	vcpu = vm_vcpu_add(vm, 0, guest_code);
 	vcpu_set_hv_cpuid(vcpu);
 	vcpu_alloc_svm(vm, &nested_gva);
 	vcpu_alloc_hyperv_test_pages(vm, &hv_pages_gva);
-- 
2.53.0

[PATCH v3 12/12] KVM: selftests: Test nested VMware backdoor routing

[Zack Rusin]

Add vmware_nested_backdoor_test.  With KVM_VMWARE_ENABLE_BACKDOOR and
KVM_VMWARE_ENABLE_NESTED_BACKDOOR enabled, run L2 under an L1 that
intercepts #GP, and verify that a VMware backdoor access from CPL3 in
L2 is handled by L0 and exits to userspace.  Verify that an access with
the wrong magic value, one to the first port above the two backdoor
ports, and one from CPL2 still cause a #GP exit to L1, as does a valid
access without the nested flag.  With only the nested flag, expect L0
to handle the access only if enable_vmware_backdoor is set.

This is the situation the flag exists for: L1 receives the #GP for the
backdoor access but can't service it, so only the narrow set of
backdoor accesses may be taken away from L1.  Exercise VMX and SVM
separately, because KVM routes nested exceptions to L1 through
different paths on the two vendors.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 MAINTAINERS                                   |   3 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../kvm/x86/vmware_nested_backdoor_test.c     | 223 ++++++++++++++++++
 3 files changed, 225 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/vmware_nested_backdoor_test.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 38a735c9221e..3118792a22df 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15050,8 +15050,7 @@ S:	Supported
 T:	git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
 T:	git https://github.com/kvm-x86/linux.git
 F:	arch/x86/kvm/vmware.*
-F:	tools/testing/selftests/kvm/x86/vmware_backdoor_test.c
-F:	tools/testing/selftests/kvm/x86/vmware_hypercall_test.c
+F:	tools/testing/selftests/kvm/x86/vmware_*.c
 
 KVM X86 Xen (KVM/Xen)
 M:	David Woodhouse <dwmw2@infradead.org>
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index fe7ba905facd..6dd1f336da99 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -142,6 +142,7 @@ TEST_GEN_PROGS_x86 += x86/userspace_io_test
 TEST_GEN_PROGS_x86 += x86/userspace_msr_exit_test
 TEST_GEN_PROGS_x86 += x86/vmware_backdoor_test
 TEST_GEN_PROGS_x86 += x86/vmware_hypercall_test
+TEST_GEN_PROGS_x86 += x86/vmware_nested_backdoor_test
 TEST_GEN_PROGS_x86 += x86/vmx_apic_access_test
 TEST_GEN_PROGS_x86 += x86/vmx_apicv_updates_test
 TEST_GEN_PROGS_x86 += x86/vmx_exception_with_invalid_guest_state
diff --git a/tools/testing/selftests/kvm/x86/vmware_nested_backdoor_test.c b/tools/testing/selftests/kvm/x86/vmware_nested_backdoor_test.c
new file mode 100644
index 000000000000..3d12be7906e0
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86/vmware_nested_backdoor_test.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025-2026 Broadcom. All Rights Reserved. The term
+ * "Broadcom" refers to Broadcom Inc. and/or its subsidiaries.
+ */
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "svm_util.h"
+#include "test_util.h"
+#include "vmx.h"
+
+#define VMWARE_BACKDOOR_PORT	0x5658
+#define VMWARE_MAGIC		0x564d5868
+#define VMPORT_CMD_GETVERSION	0x0a
+
+#define NESTED_BACKDOOR_FLAGS	(KVM_VMWARE_ENABLE_BACKDOOR | \
+				 KVM_VMWARE_ENABLE_NESTED_BACKDOOR)
+
+struct nested_backdoor_case {
+	u64 flags;
+	u64 rax;
+	u64 rdx;
+	u32 cpl;
+	bool expect_l0;
+};
+
+static struct nested_backdoor_case guest_case;
+
+static void guest_done_handler(struct ex_regs *regs)
+{
+	GUEST_DONE();
+}
+
+static void l2_guest_code(void)
+{
+	unsigned long rax = guest_case.rax;
+	unsigned long rbx = 0;
+	unsigned long rcx = VMPORT_CMD_GETVERSION;
+	unsigned long rdx = guest_case.rdx;
+
+	asm volatile("inl %%dx, %%eax"
+		     : "+a"(rax), "+b"(rbx), "+c"(rcx), "+d"(rdx));
+	GUEST_ASSERT(guest_case.expect_l0);
+	asm volatile("int3");
+}
+
+static void l1_svm_code(struct svm_test_data *svm)
+{
+	struct vmcb *vmcb = svm->vmcb;
+
+	generic_svm_setup(svm, l2_guest_code);
+	vmcb->control.intercept_exceptions |= BIT_ULL(GP_VECTOR);
+
+	vmcb->save.cpl = guest_case.cpl;
+	vmcb->save.cs.selector = USER_CS_SELECTOR | guest_case.cpl;
+	vmcb->save.cs.attrib &= ~SVM_SELECTOR_DPL_MASK;
+	vmcb->save.cs.attrib |= guest_case.cpl << SVM_SELECTOR_DPL_SHIFT;
+	vmcb->save.ss.selector = USER_DS_SELECTOR | guest_case.cpl;
+	vmcb->save.ss.attrib &= ~SVM_SELECTOR_DPL_MASK;
+	vmcb->save.ss.attrib |= guest_case.cpl << SVM_SELECTOR_DPL_SHIFT;
+	vmcb->save.ds.selector = USER_DS_SELECTOR | guest_case.cpl;
+	vmcb->save.ds.attrib &= ~SVM_SELECTOR_DPL_MASK;
+	vmcb->save.ds.attrib |= guest_case.cpl << SVM_SELECTOR_DPL_SHIFT;
+	vmcb->save.es.selector = USER_DS_SELECTOR | guest_case.cpl;
+	vmcb->save.es.attrib &= ~SVM_SELECTOR_DPL_MASK;
+	vmcb->save.es.attrib |= guest_case.cpl << SVM_SELECTOR_DPL_SHIFT;
+
+	run_guest(vmcb, svm->vmcb_gpa);
+	if (!guest_case.expect_l0) {
+		GUEST_ASSERT_EQ(vmcb->control.exit_code,
+				SVM_EXIT_EXCP_BASE + GP_VECTOR);
+		GUEST_DONE();
+	}
+	GUEST_FAIL("L1 received nested SVM exit 0x%lx",
+		   vmcb->control.exit_code);
+}
+
+static void l1_vmx_code(struct vmx_pages *vmx)
+{
+	prepare_for_vmx_operation(vmx);
+	load_vmcs(vmx);
+	prepare_vmcs(vmx, l2_guest_code);
+
+	vmwrite(EXCEPTION_BITMAP, BIT_ULL(GP_VECTOR));
+	vmwrite(GUEST_CS_SELECTOR, USER_CS_SELECTOR | guest_case.cpl);
+	vmwrite(GUEST_CS_AR_BYTES, 0xa09b | guest_case.cpl << 5);
+	vmwrite(GUEST_SS_SELECTOR, USER_DS_SELECTOR | guest_case.cpl);
+	vmwrite(GUEST_SS_AR_BYTES, 0xc093 | guest_case.cpl << 5);
+	vmwrite(GUEST_DS_SELECTOR, USER_DS_SELECTOR | guest_case.cpl);
+	vmwrite(GUEST_DS_AR_BYTES, 0xc093 | guest_case.cpl << 5);
+	vmwrite(GUEST_ES_SELECTOR, USER_DS_SELECTOR | guest_case.cpl);
+	vmwrite(GUEST_ES_AR_BYTES, 0xc093 | guest_case.cpl << 5);
+
+	vmlaunch();
+	if (!guest_case.expect_l0) {
+		GUEST_ASSERT_EQ(vmread(VM_EXIT_REASON) & 0xffff,
+				EXIT_REASON_EXCEPTION_NMI);
+		GUEST_ASSERT_EQ(vmread(VM_EXIT_INTR_INFO) & 0xff,
+				GP_VECTOR);
+		GUEST_DONE();
+	}
+	GUEST_FAIL("L1 received nested VMX exit 0x%lx",
+		   vmread(VM_EXIT_REASON));
+}
+
+static void __attribute__((__flatten__)) l1_guest_code(void *test_data)
+{
+	if (this_cpu_has(X86_FEATURE_SVM))
+		l1_svm_code(test_data);
+	else
+		l1_vmx_code(test_data);
+}
+
+static void setup_user_environment(struct kvm_vm *vm)
+{
+	struct idt_entry *idt = addr_gva2hva(vm, vm->arch.idt);
+
+	idt[BP_VECTOR].dpl = 3;
+	vm_install_exception_handler(vm, BP_VECTOR, guest_done_handler);
+	vm_init_user_mode(vm);
+}
+
+static void handle_vmware_port(struct kvm_vcpu *vcpu)
+{
+	struct kvm_run *run = vcpu->run;
+	struct kvm_regs regs;
+	u32 *data;
+
+	TEST_ASSERT_EQ(run->io.direction, KVM_EXIT_IO_IN);
+	TEST_ASSERT_EQ(run->io.size, 4);
+	TEST_ASSERT_EQ(run->io.count, 1);
+	TEST_ASSERT_EQ(run->io.port, VMWARE_BACKDOOR_PORT);
+
+	data = (void *)run + run->io.data_offset;
+	*data = 0;
+
+	vcpu_regs_get(vcpu, &regs);
+	regs.rbx = VMWARE_MAGIC;
+	vcpu_regs_set(vcpu, &regs);
+}
+
+static void run_test(const struct nested_backdoor_case *test_case)
+{
+	gva_t nested_test_data_gva;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int port_exits = 0;
+
+	vm = vm_create(1);
+	guest_case = *test_case;
+	sync_global_to_guest(vm, guest_case);
+	vm_enable_cap(vm, KVM_CAP_X86_VMWARE, test_case->flags);
+	vcpu = vm_vcpu_add(vm, 0, l1_guest_code);
+
+	if (kvm_cpu_has(X86_FEATURE_SVM))
+		vcpu_alloc_svm(vm, &nested_test_data_gva);
+	else
+		vcpu_alloc_vmx(vm, &nested_test_data_gva);
+	vcpu_args_set(vcpu, 1, nested_test_data_gva);
+	setup_user_environment(vm);
+
+	for (;;) {
+		struct kvm_run *run = vcpu->run;
+		struct ucall uc;
+
+		vcpu_run(vcpu);
+		TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+		if (run->io.port == VMWARE_BACKDOOR_PORT) {
+			handle_vmware_port(vcpu);
+			port_exits++;
+			continue;
+		}
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			break;
+		case UCALL_DONE:
+			TEST_ASSERT_EQ(port_exits, test_case->expect_l0);
+			kvm_vm_free(vm);
+			return;
+		default:
+			TEST_FAIL("Unexpected I/O exit on port 0x%x (ucall 0x%lx)",
+				  run->io.port, uc.cmd);
+		}
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	bool module_default = get_kvm_param_bool("enable_vmware_backdoor");
+	const struct nested_backdoor_case test_cases[] = {
+		{ NESTED_BACKDOOR_FLAGS,
+		  VMWARE_MAGIC, VMWARE_BACKDOOR_PORT, 3, true },
+		{ NESTED_BACKDOOR_FLAGS,
+		  0, VMWARE_BACKDOOR_PORT, 3, false },
+		{ NESTED_BACKDOOR_FLAGS,
+		  VMWARE_MAGIC, VMWARE_BACKDOOR_PORT + 2, 3, false },
+		{ NESTED_BACKDOOR_FLAGS,
+		  VMWARE_MAGIC, VMWARE_BACKDOOR_PORT, 2, false },
+		/* L1 keeps the #GP without the nested flag... */
+		{ KVM_VMWARE_ENABLE_BACKDOOR,
+		  VMWARE_MAGIC, VMWARE_BACKDOOR_PORT, 3, false },
+		/* ...and the nested flag needs the backdoor to be enabled. */
+		{ KVM_VMWARE_ENABLE_NESTED_BACKDOOR,
+		  VMWARE_MAGIC, VMWARE_BACKDOOR_PORT, 3, module_default },
+	};
+	unsigned int supported;
+	int i;
+
+	supported = kvm_check_cap(KVM_CAP_X86_VMWARE);
+	TEST_REQUIRE((supported & NESTED_BACKDOOR_FLAGS) ==
+		     NESTED_BACKDOOR_FLAGS);
+	TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM) ||
+		     kvm_cpu_has(X86_FEATURE_VMX));
+
+	for (i = 0; i < ARRAY_SIZE(test_cases); i++)
+		run_test(&test_cases[i]);
+
+	return 0;
+}
-- 
2.53.0