[PATCH v5 0/6] KVM: x86: Add LASS virtualization support

[PATCH v5 0/6] KVM: x86: Add LASS virtualization support

[Sohil Mehta]

Linear Address Space Separation (LASS) is a security feature that blocks
accesses across the user/kernel boundary based on bit 63 of the linear
address alone, before any page walk is performed. Host support for LASS
has been merged [1][2]. This series adds the KVM virtualization support.

Patches
-------
I am posting a refreshed version because kvm-x86/next has advanced and some
patches needed to be rebased. v5 contains only a couple of minor changes.
Overall, the series is ready for maintainer review (and potential inclusion
in the next cycle).

Changes in v5
-------------
 - Rebased the patches onto kvm-x86/next (kvm-x86-next-2026.10.13)
 - Added a function comment to vmx_is_lass_violation() (patch 4)
 - Dropped the LASS x86 selftest (already posted separately [3])
 - Avoided the LASS specific KVM CR4 write test (patch 6)
 - Replaced Binbin's SoBs with Reviewed-bys
 - Picked up review and test tags

v4: https://lore.kernel.org/all/20260806011536.4172258-1-sohil.mehta@intel.com/

Background
----------
The host support series [4] covers the motivation, base enforcement
mechanism, kernel toggling of RFLAGS and CR4.LASS, and the userspace
exception notifications. Here's a brief summary of the SDM bits [5] that
affects KVM support.

When LASS is enabled, the CPU applies a violation check using bit 63 to
every access to a linear address prior to page walks. A user-mode access
to a supervisor address, or a supervisor-mode access to a user address,
typically raises #GP (or #SS in rare cases) instead of a #PF that
SMAP/SMEP alone would produce. LASS takes effect only in IA-32e mode.

Enforcement for supervisor-mode data accesses additionally requires SMAP
to be enabled, and is suppressed for explicit accesses when RFLAGS.AC=1.
Linear addresses used for TLB invalidation (INVLPG, INVPCID, INVVPID)
are not subject to LASS. Unlike canonicality checks, LASS only applies
to code fetches and not branch targets.

Note, LASS is now part of the SDM instead of the ISE. There are minor
changes to the wording but nothing substantial. It also includes a
clarification that the relative ordering of LASS and canonicality checks
is not defined and cannot be determined by software.

KVM support
-----------
Hardware enforces LASS on guest code that runs natively, so KVM mostly needs
to let the guest turn it on. The exception is instruction emulation, when KVM
must apply the same LASS violation checks, so that emulated accesses behave
the same way.

For enabling, KVM advertises LASS to userspace, and lets the guest set
CR4.LASS, including in nested VMX, when LASS is enumerated in its CPUID.

Patch 1-4: Enhance the emulator to handle LASS violation checks.
Patch   5: Guest CPUID and CR4 handling. Expose LASS to userspace.
Patch   6: Extend set_sregs_test selftest for LASS.

Testing
-------
1. Basic enumeration and enabling in guest and nested environment.
2. LASS enforcement tests (userspace + LKDTM + test kernel module)
 - Userspace access to kernel address (read, write, instr fetch)
 - Kernel access to user address (read, write, instr fetch)
 - RFLAGS.AC=1 suppression (read, write)
 - FEP tests for the above cases

The x86 userspace selftest was posted separately and is under review for x86
inclusion [3].

KVM selftests and kvm-unit-tests run guest code in the lower half at CPL0, so
enabling CR4.LASS makes the next instruction fetch a violation and
triple-faults the guest. I used a set of LLM generated kvm-unit-tests [6] that
overcome the limitation. Planning to clean them up and post them separately.

Kishen Maloor and Farrah Chen tested v4 on Sierra Forest and Clearwater
Forest, using the QEMU support [7] to expose LASS to the guest.

Links
-----
[1]: https://lore.kernel.org/lkml/20251201231537.736899-1-dave.hansen@linux.intel.com/
[2]: https://lore.kernel.org/lkml/20260413154235.1543087-1-dave.hansen@linux.intel.com/
[3]: https://lore.kernel.org/lkml/20261009020025.4175228-1-sohil.mehta@intel.com/
[4]: https://lore.kernel.org/all/20251118182911.2983253-1-sohil.mehta@intel.com/
[5]: "Linear-Address Pre-Processing", Intel SDM (June 2026), Vol 3, Chapter 4.
[6]: https://github.com/sohilmeh/kvm-unit-tests/tree/lass-llm
[7]: https://lore.kernel.org/all/20260826035734.114685-1-kishen.maloor@intel.com/

Previous versions
-----------------
v4: https://lore.kernel.org/all/20260806011536.4172258-1-sohil.mehta@intel.com/
v3: https://lore.kernel.org/lkml/20230913124227.12574-1-binbin.wu@linux.intel.com/
v2: https://lore.kernel.org/all/20230718131844.5706-1-guang.zeng@intel.com/
v1: https://lore.kernel.org/all/20230601142309.6307-1-guang.zeng@intel.com/
v0: https://lore.kernel.org/lkml/20230420133724.11398-1-guang.zeng@intel.com/


Binbin Wu (1):
  KVM: x86: Add an emulator flag to differentiate branch targets from
    fetches

Sohil Mehta (2):
  KVM: x86: Use linear_read_system() to read the TSS I/O bitmap
  KVM: selftests: Extend set_sregs_test to cover LASS

Zeng Guang (3):
  KVM: x86: Add LASS violation checks during instruction emulation
  KVM: VMX: Implement LASS violation check
  KVM: x86: Virtualize LASS and advertise support to userspace

 arch/x86/include/asm/kvm-x86-ops.h            |  1 +
 arch/x86/include/asm/kvm_host.h               |  2 +
 arch/x86/kvm/cpuid.c                          |  1 +
 arch/x86/kvm/emulate.c                        | 28 ++++++--
 arch/x86/kvm/kvm_emulate.h                    |  4 +-
 arch/x86/kvm/regs.h                           |  4 +-
 arch/x86/kvm/vmx/main.c                       |  1 +
 arch/x86/kvm/vmx/nested.c                     | 11 +--
 arch/x86/kvm/vmx/sgx.c                        |  3 +-
 arch/x86/kvm/vmx/vmx.c                        | 68 ++++++++++++++++++-
 arch/x86/kvm/vmx/vmx.h                        |  3 +
 arch/x86/kvm/x86.c                            |  9 ++-
 .../selftests/kvm/include/x86/processor.h     |  2 +
 .../selftests/kvm/x86/set_sregs_test.c        |  3 +
 14 files changed, 125 insertions(+), 15 deletions(-)


base-commit: dea4f6380f4ec96e8f3b0aaf2343764bd0d25ef4
-- 
2.43.0

[PATCH v5 1/6] KVM: x86: Add an emulator flag to differentiate branch targets from fetches

[Sohil Mehta]

From: Binbin Wu <binbin.wu@linux.intel.com>

Add a new emulator flag, X86EMUL_F_BRANCH, and use it instead of
X86EMUL_F_FETCH in assign_eip() to distinguish between instruction fetch
and branch target computation for features that handle them differently.

For example, Linear Address Space Separation (LASS) applies to code
fetches but not branch target calculations. A LASS violation occurs only
when the branch target is used to fetch an instruction.

X86EMUL_F_BRANCH is still a code access, so add it to the code segment
readability check in __linearize() along with the Linear Address Masking
(LAM) untagging exemption in vmx_get_untagged_addr().

For now, X86EMUL_F_BRANCH and X86EMUL_F_FETCH are identical as far as
KVM is concerned. No functional change intended.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
---
v5:
 - Picked up Tested-by tags

v4:
 - Added LAM untagging exemption
 - Reworded the commit message
---
 arch/x86/kvm/emulate.c     | 5 +++--
 arch/x86/kvm/kvm_emulate.h | 1 +
 arch/x86/kvm/vmx/vmx.c     | 3 ++-
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index c1b21282187f..15872e722af8 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -654,7 +654,8 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 		    (flags & X86EMUL_F_WRITE))
 			goto bad;
 		/* unreadable code segment */
-		if (!(flags & X86EMUL_F_FETCH) && (desc.type & 8) && !(desc.type & 2))
+		if (!(flags & (X86EMUL_F_FETCH | X86EMUL_F_BRANCH)) &&
+		    (desc.type & 8) && !(desc.type & 2))
 			goto bad;
 		lim = desc_limit_scaled(&desc);
 		if (!(desc.type & 8) && (desc.type & 4)) {
@@ -705,7 +706,7 @@ static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst)
 	if (ctxt->op_bytes != sizeof(unsigned long))
 		addr.ea = dst & ((1UL << (ctxt->op_bytes << 3)) - 1);
 	rc = __linearize(ctxt, addr, &max_size, 1, ctxt->mode, &linear,
-			 X86EMUL_F_FETCH);
+			 X86EMUL_F_BRANCH);
 	if (rc == X86EMUL_CONTINUE)
 		ctxt->_eip = addr.ea;
 	return rc;
diff --git a/arch/x86/kvm/kvm_emulate.h b/arch/x86/kvm/kvm_emulate.h
index 10886bea82c9..0623637f923b 100644
--- a/arch/x86/kvm/kvm_emulate.h
+++ b/arch/x86/kvm/kvm_emulate.h
@@ -105,6 +105,7 @@ struct x86_instruction_info {
 #define X86EMUL_F_INVLPG		BIT(3)
 #define X86EMUL_F_MSR			BIT(4)
 #define X86EMUL_F_DT_LOAD		BIT(5)
+#define X86EMUL_F_BRANCH		BIT(6)
 
 struct x86_emulate_ops {
 	void (*vm_bugged)(struct x86_emulate_ctxt *ctxt);
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index b341af4c6057..04a9acf4a39a 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -8436,7 +8436,8 @@ gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags
 	int lam_bit;
 	unsigned long cr3_bits;
 
-	if (flags & (X86EMUL_F_FETCH | X86EMUL_F_IMPLICIT | X86EMUL_F_INVLPG))
+	if (flags & (X86EMUL_F_FETCH | X86EMUL_F_BRANCH | X86EMUL_F_IMPLICIT |
+		     X86EMUL_F_INVLPG))
 		return gva;
 
 	if (!is_64_bit_mode(vcpu))
-- 
2.43.0

[PATCH v5 2/6] KVM: x86: Use linear_read_system() to read the TSS I/O bitmap

[Sohil Mehta]

TSS I/O permission bitmap reads are implicit supervisor accesses which
are subject to Linear Address Space Separation (LASS) enforcement.
Though highly unlikely, if a guest configures a TSS base in the user
half, hardware would raise a #GP on access when LASS is enabled.

Currently, the emulator reads the I/O permission bitmap from the TSS by
calling read_std() directly which is inconsistent with other implicit
accesses in the emulator such as IDT reads, GDT/LDT reads and TSS reads
during task switch.

An upcoming change will add a check to linear_read_system() to catch
LASS violations. For consistency as well as to keep LASS enforcement
centralized, switch both I/O bitmap reads to linear_read_system().

Note, emulator_io_port_access_allowed() doesn't propagate faults, so
even though linear_read_system() will set the exception details they
will be ignored.

While at it, fix an off-by-one in the I/O bitmap bounds check to account
for the 2-byte read and ensure both bytes are within the TSS limit. The
SDM mandates a trailing 0xFF byte after the bitmap so any out-of-bounds
access would be all 1s (denying access). Make the change primarily to
ensure hardware fidelity. A correctly configured OS will not run into
this issue.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
---
v5:
 - Picked up Tested-by tags

v4:
- New patch

There could be a pre-existing issue here. It is unlikely that any OS
demand-pages the I/O bitmap portion of the TSS. But if it does, the #PF
details could get lost and the guest would get a #GP instead of a
restartable #PF. Propagating the #PF to the callers is a larger change
that is beyond the scope of this series.
---
 arch/x86/kvm/emulate.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 15872e722af8..fbf47f5ae298 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -2555,12 +2555,12 @@ static bool emulator_io_port_access_allowed(struct x86_emulate_ctxt *ctxt,
 #ifdef CONFIG_X86_64
 	base |= ((u64)base3) << 32;
 #endif
-	r = ops->read_std(ctxt, base + 102, &io_bitmap_ptr, 2, NULL, true);
+	r = linear_read_system(ctxt, base + 102, &io_bitmap_ptr, 2);
 	if (r != X86EMUL_CONTINUE)
 		return false;
-	if (io_bitmap_ptr + port/8 > desc_limit_scaled(&tr_seg))
+	if (io_bitmap_ptr + port/8 + 1 > desc_limit_scaled(&tr_seg))
 		return false;
-	r = ops->read_std(ctxt, base + io_bitmap_ptr + port/8, &perm, 2, NULL, true);
+	r = linear_read_system(ctxt, base + io_bitmap_ptr + port/8, &perm, 2);
 	if (r != X86EMUL_CONTINUE)
 		return false;
 	if ((perm >> bit_idx) & mask)
-- 
2.43.0

[PATCH v5 3/6] KVM: x86: Add LASS violation checks during instruction emulation

[Sohil Mehta]

From: Zeng Guang <guang.zeng@intel.com>

When Linear Address Space Separation (LASS) is enabled, the processor
applies a LASS violation check on every access to a linear address. To
align with hardware behavior, KVM needs to perform the same check during
instruction emulation before consulting the page tables.

Add a new callback to x86_emulate_ops to let the emulator query whether
an access would trigger a LASS violation. The callback takes the linear
address and the size that describe the memory access, plus a set of
flags that convey the type of access.

Add the LASS violation check to __linearize() so that every explicit
guest memory access is validated along with the other linear address
checks. The SDM (June 2026), Vol3, Chapter 4, specifically states that
there is no relative ordering between the canonicality check and the
LASS violation check. Also, there is no prioritization specified between
the faults generated by alignment checks and LASS violations.

Implicit supervisor accesses bypass __linearize(). So, add LASS checks
for those in linear_read_system() and linear_write_system() and tag the
access as implicit.

For now, emulator_is_lass_violation() is a no-op. Later, it will be
wired up to the VMX implementation.

Signed-off-by: Zeng Guang <guang.zeng@intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
---
v5:
 - Replaced Binbin's SoB with a Reviewed-by
 - Picked up Tested-by tags

v4:
 - Switched to using gva_t for the address parameter
 - Clarified LASS and canonicality checks and alignment checks ordering
 - Reworded the commit message
 - Use kvm_x86_call() instead of static_call()

Note, internal AI review warned about missing canonicality checks during
implicit accesses. A valid guest kernel shouldn't trigger canonicality
faults. But for correctness, such checks could be considered. Though,
that is beyond the scope of this series.
---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/emulate.c             | 17 +++++++++++++++++
 arch/x86/kvm/kvm_emulate.h         |  3 ++-
 arch/x86/kvm/x86.c                 |  7 +++++++
 5 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index 48fa973b98d5..e16ea56e5911 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -147,6 +147,7 @@ KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP_OPTIONAL(get_untagged_addr)
+KVM_X86_OP_OPTIONAL_RET0(is_lass_violation)
 KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)
 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
 KVM_X86_OP_OPTIONAL_RET0(gmem_make_private)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e2eef7057bd3..1099ec2801db 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1743,6 +1743,8 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags);
+	bool (*is_lass_violation)(struct kvm_vcpu *vcpu, gva_t gva,
+				  unsigned int size, unsigned int flags);
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
 #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CONVERT
 	int (*gmem_make_private)(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index fbf47f5ae298..e071b1de1dac 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -675,6 +675,16 @@ static __always_inline int __linearize(struct x86_emulate_ctxt *ctxt,
 		}
 		break;
 	}
+
+	/*
+	 * LASS and canonicality checks generate the same fault and their
+	 * relative order is not architecturally defined. Also, the SDM
+	 * doesn't prioritize LASS violations against the misalignment #GP,
+	 * so any order is fine.
+	 */
+	if (ctxt->ops->is_lass_violation(ctxt, la, size, flags))
+		goto bad;
+
 	if (la & (insn_alignment(ctxt, size) - 1))
 		return emulate_gp(ctxt, 0);
 	return X86EMUL_CONTINUE;
@@ -781,6 +791,9 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctxt, int rel)
 static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear,
 			      void *data, unsigned size)
 {
+	if (ctxt->ops->is_lass_violation(ctxt, linear, size, X86EMUL_F_IMPLICIT))
+		return emulate_gp(ctxt, 0);
+
 	return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
@@ -788,6 +801,10 @@ static int linear_write_system(struct x86_emulate_ctxt *ctxt,
 			       ulong linear, void *data,
 			       unsigned int size)
 {
+	if (ctxt->ops->is_lass_violation(ctxt, linear, size,
+					 X86EMUL_F_IMPLICIT | X86EMUL_F_WRITE))
+		return emulate_gp(ctxt, 0);
+
 	return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, true);
 }
 
diff --git a/arch/x86/kvm/kvm_emulate.h b/arch/x86/kvm/kvm_emulate.h
index 0623637f923b..c4bd50a28e8a 100644
--- a/arch/x86/kvm/kvm_emulate.h
+++ b/arch/x86/kvm/kvm_emulate.h
@@ -249,7 +249,8 @@ struct x86_emulate_ops {
 
 	gva_t (*get_untagged_addr)(struct x86_emulate_ctxt *ctxt, gva_t addr,
 				   unsigned int flags);
-
+	bool (*is_lass_violation)(struct x86_emulate_ctxt *ctxt, gva_t addr,
+				  unsigned int size, unsigned int flags);
 	bool (*is_canonical_addr)(struct x86_emulate_ctxt *ctxt, gva_t addr,
 				  unsigned int flags);
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 32563a91fba9..bd69934cafd3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5843,6 +5843,12 @@ static gva_t emulator_get_untagged_addr(struct x86_emulate_ctxt *ctxt,
 					       addr, flags);
 }
 
+static bool emulator_is_lass_violation(struct x86_emulate_ctxt *ctxt, gva_t addr,
+				       unsigned int size, unsigned int flags)
+{
+	return kvm_x86_call(is_lass_violation)(emul_to_vcpu(ctxt), addr, size, flags);
+}
+
 static bool emulator_is_canonical_addr(struct x86_emulate_ctxt *ctxt,
 				       gva_t addr, unsigned int flags)
 {
@@ -5902,6 +5908,7 @@ static const struct x86_emulate_ops emulate_ops = {
 	.get_xcr             = emulator_get_xcr,
 	.set_xcr             = emulator_set_xcr,
 	.get_untagged_addr   = emulator_get_untagged_addr,
+	.is_lass_violation   = emulator_is_lass_violation,
 	.is_canonical_addr   = emulator_is_canonical_addr,
 	.page_address_valid  = emulator_page_address_valid,
 };
-- 
2.43.0

[PATCH v5 4/6] KVM: VMX: Implement LASS violation check

[Sohil Mehta]

From: Zeng Guang <guang.zeng@intel.com>

Add a VMX implementation of the is_lass_violation() hook to let KVM
detect Linear Address Space Separation (LASS) violations on linear
addresses generated during emulation. LASS uses bit 63 of the linear
address to determine which half of the address space is being targeted,
and reports a violation when that half doesn't match the current
privilege level.

Note, LASS takes effect only in IA-32e mode; it is ignored in legacy
mode. LASS enforcement for supervisor-mode data accesses additionally
requires SMAP to be enabled, and is suppressed for explicit accesses
when RFLAGS.AC=1.

Enforce LASS violations on emulated instruction fetches and data
accesses, including implicit supervisor accesses, so that the mode-based
protections are applied before paging. Also enforce LASS on the linear
addresses consumed by emulated VMX and SGX ENCLS instructions.

Linear addresses used for TLB invalidation (INVLPG, INVPCID, and
INVVPID) and branch targets are not subject to LASS enforcement.

Signed-off-by: Zeng Guang <guang.zeng@intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chang S. Bae <chang.seok.bae@intel.com>
---
v5:
 - Added a function comment describing the basic flow
 - Replaced Binbin's SoB with a Reviewed-by
 - Picked up review and test tags

v4:
 - Switch to using gva_t for the address argument
 - Split patch diff and reworded the commit message

Note, Sashiko flagged a missing CR4.SMEP check for supervisor fetches
and suggested is_64_bit_mode() instead of is_long_mode(). Both are false
positives. LASS enforces the equivalent of SMEP regardless of CR4.SMEP,
and it applies throughout IA-32e mode, including compatibility mode.
---
 arch/x86/kvm/vmx/main.c   |  1 +
 arch/x86/kvm/vmx/nested.c | 11 ++++---
 arch/x86/kvm/vmx/sgx.c    |  3 +-
 arch/x86/kvm/vmx/vmx.c    | 64 +++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.h    |  3 ++
 arch/x86/kvm/x86.c        |  2 +-
 6 files changed, 77 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index d75170a6199e..0696ba3b77fe 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -1144,6 +1144,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector,
 
 	.get_untagged_addr = vmx_get_untagged_addr,
+	.is_lass_violation = vmx_is_lass_violation,
 
 	.mem_enc_ioctl = vt_op_tdx_only(mem_enc_ioctl),
 	.vcpu_mem_enc_ioctl = vt_op_tdx_only(vcpu_mem_enc_ioctl),
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 9b0bfa2f854c..1794f25e0724 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5309,11 +5309,12 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsigned long exit_qualification,
 			*ret = off;
 
 		*ret = vmx_get_untagged_addr(vcpu, *ret, 0);
-		/* Long mode: #GP(0)/#SS(0) if the memory address is in a
-		 * non-canonical form. This is the only check on the memory
-		 * destination for long mode!
+		/*
+		 * Long mode: #GP(0)/#SS(0) if the memory address is in a
+		 * non-canonical form, or if the access violates LASS.
 		 */
-		exn = is_noncanonical_address(*ret, vcpu, 0);
+		exn = is_noncanonical_address(*ret, vcpu, 0) ||
+		      vmx_is_lass_violation(vcpu, *ret, len, 0);
 	} else {
 		/*
 		 * When not in long mode, the virtual/linear address is
@@ -6128,7 +6129,7 @@ static int handle_invvpid(struct kvm_vcpu *vcpu)
 	if (type != VMX_VPID_EXTENT_ALL_CONTEXT && !operand.vpid)
 		return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
 
-	/* LAM doesn't apply to addresses that are inputs to TLB invalidation. */
+	/* LAM and LASS don't apply to addresses that are inputs to TLB invalidation. */
 	if (type == VMX_VPID_EXTENT_INDIVIDUAL_ADDR &&
 	    is_noncanonical_invlpg_address(operand.gla, vcpu))
 		return nested_vmx_fail(vcpu, VMXERR_INVALID_OPERAND_TO_INVEPT_INVVPID);
diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c
index 771c75a58343..4ac305ed6dea 100644
--- a/arch/x86/kvm/vmx/sgx.c
+++ b/arch/x86/kvm/vmx/sgx.c
@@ -39,7 +39,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsigned long offset,
 		fault = true;
 	} else if (likely(is_64_bit_mode(vcpu))) {
 		*gva = vmx_get_untagged_addr(vcpu, *gva, 0);
-		fault = is_noncanonical_address(*gva, vcpu, 0);
+		fault = is_noncanonical_address(*gva, vcpu, 0) ||
+			vmx_is_lass_violation(vcpu, *gva, size, 0);
 	} else {
 		*gva &= 0xffffffff;
 		fault = (s.unusable) ||
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 04a9acf4a39a..6245de123ec3 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -8469,6 +8469,70 @@ gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags
 	return (sign_extend64(gva, lam_bit) & ~BIT_ULL(63)) | (gva & BIT_ULL(63));
 }
 
+/*
+ * Check if an access violates LASS purely based on bit 63 of the linear
+ * address and whether the access targets the "wrong" half of the address
+ * space. The basic flow is straightforward:
+ *
+ *      if (exempt from LASS)
+ *              no violation;
+ *      else if (user access)
+ *              violation when address is in the supervisor half;
+ *      else
+ *              violation when address is in the user half;
+ *
+ * but the actual behavior is far more subtle due to the many exceptions and
+ * edge cases. For example, the implementation depends on which accesses are
+ * exempt, what constitutes a user access, and how supervisor data accesses
+ * differ from instruction fetches.
+ */
+bool vmx_is_lass_violation(struct kvm_vcpu *vcpu, gva_t gva,
+			   unsigned int size, unsigned int flags)
+{
+	const bool is_supervisor_address = !!(gva & BIT_ULL(63));
+	const bool implicit_supervisor = !!(flags & X86EMUL_F_IMPLICIT);
+	const bool fetch = !!(flags & X86EMUL_F_FETCH);
+
+	if (!kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS) || !is_long_mode(vcpu))
+		return false;
+
+	/*
+	 * INVLPG isn't subject to LASS, e.g. to allow invalidating userspace
+	 * addresses without toggling RFLAGS.AC.  Branch targets aren't
+	 * subject to LASS in order to simplify far control transfers (the
+	 * subsequent fetch will enforce LASS as appropriate).
+	 */
+	if (flags & (X86EMUL_F_BRANCH | X86EMUL_F_INVLPG))
+		return false;
+
+	if (!implicit_supervisor && vmx_get_cpl(vcpu) == 3)
+		return is_supervisor_address;
+
+	/*
+	 * LASS enforcement for supervisor-mode data accesses depends on SMAP
+	 * being enabled, and like SMAP ignores explicit accesses if RFLAGS.AC=1.
+	 */
+	if (!fetch) {
+		if (!kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP))
+			return false;
+
+		if (!implicit_supervisor && (kvm_get_rflags(vcpu) & X86_EFLAGS_AC))
+			return false;
+	}
+
+	/*
+	 * The entire access must be in the appropriate address space.  Note,
+	 * if LAM is supported, @gva has already been untagged, so barring a
+	 * massive architecture change to expand the canonical address range,
+	 * it's impossible for a user access to straddle user and supervisor
+	 * address spaces.
+	 */
+	if (size && !((gva + size - 1) & BIT_ULL(63)))
+		return true;
+
+	return !is_supervisor_address;
+}
+
 static unsigned int vmx_handle_intel_pt_intr(void)
 {
 	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h
index 6c9dd5b2365d..48ae3b213f97 100644
--- a/arch/x86/kvm/vmx/vmx.h
+++ b/arch/x86/kvm/vmx/vmx.h
@@ -353,6 +353,9 @@ u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu);
 
 gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags);
 
+bool vmx_is_lass_violation(struct kvm_vcpu *vcpu, gva_t gva,
+			   unsigned int size, unsigned int flags);
+
 void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu, bool enable);
 
 u64 vmx_get_supported_debugctl(struct kvm_vcpu *vcpu, bool host_initiated);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bd69934cafd3..ba94ade695fc 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -10857,7 +10857,7 @@ int kvm_handle_invpcid(struct kvm_vcpu *vcpu, unsigned long type, gva_t gva)
 	switch (type) {
 	case INVPCID_TYPE_INDIV_ADDR:
 		/*
-		 * LAM doesn't apply to addresses that are inputs to TLB
+		 * LAM and LASS don't apply to addresses that are inputs to TLB
 		 * invalidation.
 		 */
 		if ((!pcid_enabled && (operand.pcid != 0)) ||
-- 
2.43.0

[PATCH v5 5/6] KVM: x86: Virtualize LASS and advertise support to userspace

[Sohil Mehta]

From: Zeng Guang <guang.zeng@intel.com>

Virtualize Linear Address Space Separation (LASS) by letting the guest
enable CR4.LASS[bit 27]. Allow the guest to set CR4.LASS only if LASS is
enumerated in the guest's CPUID. Set CR4.LASS in the emulated
IA32_VMX_CR4_FIXED1 MSR so that LASS can be enabled in nested VMX
operation as well.

Keep CR4.LASS KVM-owned instead of making it guest-owned. LASS is
expected to be enabled once per vCPU at boot and rarely toggled at
runtime.

LASS is enumerated by CPUID.(EAX=07H,ECX=1):EAX.LASS[bit 6] and only
works in IA-32e mode. Advertise LASS to userspace on 64-bit kernels and
only when it is supported by the host. Notably, it will not be exposed
to userspace if the deprecated vsyscall=emulate is set on the kernel
command line.

Signed-off-by: Zeng Guang <guang.zeng@intel.com>
Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
---
v5:
 - Replaced Binbin's SoB with a Reviewed-by
 - Picked up Tested-by tags

v4:
 - Split patch diffs and reworded the commit message
 - Use X86_64_F() instead of F()
---
 arch/x86/kvm/cpuid.c   | 1 +
 arch/x86/kvm/regs.h    | 4 +++-
 arch/x86/kvm/vmx/vmx.c | 1 +
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index adafd6fa2c54..3941f5ce9425 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1016,6 +1016,7 @@ void kvm_initialize_cpu_caps(void)
 		F(SM4),
 		F(AVX_VNNI),
 		F(AVX512_BF16),
+		X86_64_F(LASS),
 		F(CMPCCXADD),
 		F(FZRM),
 		F(FSRS),
diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h
index 97c17b120a57..a1ac69ac20f5 100644
--- a/arch/x86/kvm/regs.h
+++ b/arch/x86/kvm/regs.h
@@ -28,7 +28,7 @@ static_assert(!(KVM_POSSIBLE_CR0_GUEST_BITS & X86_CR0_PDPTR_BITS));
 			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
 			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
 			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \
-			  | X86_CR4_LAM_SUP | X86_CR4_CET))
+			  | X86_CR4_LAM_SUP | X86_CR4_CET | X86_CR4_LASS))
 
 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
@@ -404,6 +404,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 		__reserved_bits |= X86_CR4_PCIDE;       \
 	if (!__cpu_has(__c, X86_FEATURE_LAM))           \
 		__reserved_bits |= X86_CR4_LAM_SUP;     \
+	if (!__cpu_has(__c, X86_FEATURE_LASS))          \
+		__reserved_bits |= X86_CR4_LASS;        \
 	if (!__cpu_has(__c, X86_FEATURE_SHSTK) &&       \
 	    !__cpu_has(__c, X86_FEATURE_IBT))           \
 		__reserved_bits |= X86_CR4_CET;         \
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 6245de123ec3..273c09f15fa3 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -7751,6 +7751,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
 	cr4_fixed1_update(X86_CR4_LAM_SUP,    eax, feature_bit(LAM));
+	cr4_fixed1_update(X86_CR4_LASS,       eax, feature_bit(LASS));
 
 #undef cr4_fixed1_update
 }
-- 
2.43.0

[PATCH v5 6/6] KVM: selftests: Extend set_sregs_test to cover LASS

[Sohil Mehta]

Extend set_sregs_test to cover CR4.LASS and verify that KVM_SET_SREGS
accepts CR4.LASS when LASS is supported, and rejects it, without
modifying sregs, when it isn't.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Kishen Maloor <kishen.maloor@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
---
v5:
 - Avoided the CR4 write test specific to LASS.
 - Picked up review and test tags

v4:
 - New patch
---
 tools/testing/selftests/kvm/include/x86/processor.h | 2 ++
 tools/testing/selftests/kvm/x86/set_sregs_test.c    | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h
index 5898a277a2e0..606ee9fe0ddb 100644
--- a/tools/testing/selftests/kvm/include/x86/processor.h
+++ b/tools/testing/selftests/kvm/include/x86/processor.h
@@ -81,6 +81,7 @@ const char *ex_str(int vector);
 #define X86_CR4_SMEP		(1ul << 20)
 #define X86_CR4_SMAP		(1ul << 21)
 #define X86_CR4_PKE		(1ul << 22)
+#define X86_CR4_LASS		(1ul << 27)
 
 struct xstate_header {
 	u64				xstate_bv;
@@ -197,6 +198,7 @@ struct kvm_x86_cpu_feature {
 #define	X86_FEATURE_SPEC_CTRL		KVM_X86_CPU_FEATURE(0x7, 0, EDX, 26)
 #define	X86_FEATURE_ARCH_CAPABILITIES	KVM_X86_CPU_FEATURE(0x7, 0, EDX, 29)
 #define	X86_FEATURE_PKS			KVM_X86_CPU_FEATURE(0x7, 0, ECX, 31)
+#define	X86_FEATURE_LASS		KVM_X86_CPU_FEATURE(0x7, 1, EAX, 6)
 #define	X86_FEATURE_XTILECFG		KVM_X86_CPU_FEATURE(0xD, 0, EAX, 17)
 #define	X86_FEATURE_XTILEDATA		KVM_X86_CPU_FEATURE(0xD, 0, EAX, 18)
 #define	X86_FEATURE_XSAVES		KVM_X86_CPU_FEATURE(0xD, 1, EAX, 3)
diff --git a/tools/testing/selftests/kvm/x86/set_sregs_test.c b/tools/testing/selftests/kvm/x86/set_sregs_test.c
index 5afc4187f5eb..f30dc61fe00f 100644
--- a/tools/testing/selftests/kvm/x86/set_sregs_test.c
+++ b/tools/testing/selftests/kvm/x86/set_sregs_test.c
@@ -72,6 +72,8 @@ static u64 calc_supported_cr4_feature_bits(void)
 		cr4 |= X86_CR4_SMAP;
 	if (kvm_cpu_has(X86_FEATURE_PKU))
 		cr4 |= X86_CR4_PKE;
+	if (kvm_cpu_has(X86_FEATURE_LASS))
+		cr4 |= X86_CR4_LASS;
 
 	return cr4;
 }
@@ -128,6 +130,7 @@ static void test_cr_bits(struct kvm_vcpu *vcpu, u64 cr4)
 	TEST_INVALID_SREG_BIT(vcpu, cr4, sregs, X86_CR4_SMEP);
 	TEST_INVALID_SREG_BIT(vcpu, cr4, sregs, X86_CR4_SMAP);
 	TEST_INVALID_SREG_BIT(vcpu, cr4, sregs, X86_CR4_PKE);
+	TEST_INVALID_SREG_BIT(vcpu, cr4, sregs, X86_CR4_LASS);
 
 	for (i = 32; i < 64; i++)
 		TEST_INVALID_SREG_BIT(vcpu, cr0, sregs, BIT(i));
-- 
2.43.0