[PATCH v2 00/19] KVM: arm64: selftests: Tests for protected VMs

[PATCH v2 00/19] KVM: arm64: selftests: Tests for protected VMs

[Fuad Tabba]

Hi folks,

Changes since v1 [1]:
  - New patch 1, which hides generic PtrAuth from a protected VM
    without PtrAuth. The pkvm_id_regs test checks for it.
  - New pkvm_sve test, and MOPS, PIE, POE and FPMR checks in
    pkvm_id_regs.
  - Use the selftest syscall wrappers and vm_check_cap().
  - Marked pkvm_hvc() as called from the guest. (Joey)
  - Collected Reviewed-by from Joey on patches 3, 6 and 7. (thanks!)

This series adds selftests for protected VMs. A test skips on a host
without pKVM, and so does any case where the host doesn't have the
required feature, e.g., PtrAuth, SVE or hugetlb pages.

A couple of points:

- Patch 1 is a fix, included here so that pkvm_id_regs can check the
  correct behaviour. Without it, generic PtrAuth is advertised to a
  protected VM without PtrAuth.

- Each area gets its own test rather than cases folded into existing
  tests such as psci_test, hypercalls, set_id_regs, etc. A protected
  guest requires its own boot path, and has to share the ucall pool
  before its first ucall. On top of that, a protected guest gets the
  opposite result for many of those tests' checks.

- Cases for a feature a protected VM may not have follow what the VM
  is given, so they pass with or without the series that adds it [2],
  and run once it lands. The debug cases follow
  ID_AA64DFR0_EL1.DebugVer, the SVE cases KVM_CAP_ARM_SVE on a
  protected VM's fd, and the MOPS, PIE, POE and FPMR checks their ID
  register fields.

The series is structured as follows:

  01:     Hide generic PtrAuth from protected VMs without PtrAuth.
  02:     external_aborts: an abort injected before a vCPU's first run.
  03-05:  Generic selftest changes: creating a protected VM, the pKVM
          function IDs and the ucall pool's address.
  06-07:  Protected-VM library: boot trampoline and helpers.
  08-18:  The tests, one per area.
  19:     MOPS, PIE, POE and FPMR checks in pkvm_id_regs.

Based on Linux 7.4-rc1 (6628e7d4c6b25; also applies to 7.4-rc2 7475ea8b58616).

Cheers,
/fuad

[1] https://lore.kernel.org/all/20261104084842.132944-1-fuad.tabba@linux.dev/
[2] https://lore.kernel.org/all/20261109083205.806619-1-fuad.tabba@linux.dev/

Fuad Tabba (18):
  KVM: arm64: Hide generic ptrauth from protected VMs without ptrauth
  KVM: arm64: selftests: Inject an abort before a vCPU's first run
  KVM: selftests: Allow arm64 tests to create protected VMs
  tools: arm-smccc: Add the pKVM memory sharing function IDs
  KVM: selftests: Export the ucall pool's guest physical address
  KVM: arm64: selftests: Add protected-VM library helpers
  KVM: arm64: selftests: Add pkvm_host_abi host-interface test
  KVM: arm64: selftests: Add pkvm_psci multi-vCPU PSCI test
  KVM: arm64: selftests: Add pkvm_mem_share hypercall test
  KVM: arm64: selftests: Add pkvm_smccc hypercall test
  KVM: arm64: selftests: Add pkvm_id_regs register and trap test
  KVM: arm64: selftests: Add pkvm_vcpu_regs register-state test
  KVM: arm64: selftests: Add pkvm_irq interrupt test
  KVM: arm64: selftests: Add pkvm_isolation host-access test
  KVM: arm64: selftests: Add pkvm_lifecycle teardown test
  KVM: arm64: selftests: Add pkvm_exceptions debug and SError test
  KVM: arm64: selftests: Add pkvm_sve vector-length test
  KVM: arm64: selftests: Check MOPS, PIE, POE and FPMR in a protected VM

Will Deacon (1):
  KVM: arm64: selftests: Add protected-VM boot trampoline

 arch/arm64/kvm/hyp/nvhe/sys_regs.c            |   4 +-
 tools/include/linux/arm-smccc.h               |  22 +
 tools/testing/selftests/kvm/Makefile.kvm      |  13 +
 .../selftests/kvm/arm64/external_aborts.c     |  21 +
 .../selftests/kvm/arm64/pkvm_exceptions.c     | 261 ++++++++
 .../selftests/kvm/arm64/pkvm_host_abi.c       | 435 ++++++++++++
 .../selftests/kvm/arm64/pkvm_id_regs.c        | 394 +++++++++++
 tools/testing/selftests/kvm/arm64/pkvm_irq.c  | 268 ++++++++
 .../selftests/kvm/arm64/pkvm_isolation.c      | 225 +++++++
 .../selftests/kvm/arm64/pkvm_lifecycle.c      | 110 +++
 .../selftests/kvm/arm64/pkvm_mem_share.c      | 394 +++++++++++
 tools/testing/selftests/kvm/arm64/pkvm_psci.c | 632 ++++++++++++++++++
 .../testing/selftests/kvm/arm64/pkvm_smccc.c  | 117 ++++
 tools/testing/selftests/kvm/arm64/pkvm_sve.c  | 357 ++++++++++
 .../selftests/kvm/arm64/pkvm_vcpu_regs.c      | 218 ++++++
 .../selftests/kvm/include/arm64/pkvm.h        |  75 +++
 .../selftests/kvm/include/ucall_common.h      |   1 +
 tools/testing/selftests/kvm/lib/arm64/pkvm.c  | 255 +++++++
 .../selftests/kvm/lib/arm64/pvm-entry.S       |  57 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    |   5 +-
 .../testing/selftests/kvm/lib/ucall_common.c  |   9 +-
 21 files changed, 3868 insertions(+), 5 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_exceptions.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_host_abi.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_irq.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_isolation.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_lifecycle.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_mem_share.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_psci.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_smccc.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_sve.c
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_vcpu_regs.c
 create mode 100644 tools/testing/selftests/kvm/include/arm64/pkvm.h
 create mode 100644 tools/testing/selftests/kvm/lib/arm64/pkvm.c
 create mode 100644 tools/testing/selftests/kvm/lib/arm64/pvm-entry.S

-- 
2.47.3

[PATCH v2 01/19] KVM: arm64: Hide generic ptrauth from protected VMs without ptrauth

[Fuad Tabba]

A protected VM without the ptrauth vCPU features shouldn't have pointer
authentication in its ID registers. Commit 3d7ff00700d1 ("KVM: arm64:
Rework specifying restricted features for protected VMs") gated APA,
API, APA3 and GPA3 on vm_has_ptrauth(), but left GPA and GPI capped
at IMP. So generic authentication is advertised to a protected guest
without ptrauth, and PACGA UNDEFs.

Gate GPA and GPI on vm_has_ptrauth() too.

Fixes: 3d7ff00700d1 ("KVM: arm64: Rework specifying restricted features for protected VMs")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/hyp/nvhe/sys_regs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/sys_regs.c b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
index 32e82653ebf7e..e5502d5651b05 100644
--- a/arch/arm64/kvm/hyp/nvhe/sys_regs.c
+++ b/arch/arm64/kvm/hyp/nvhe/sys_regs.c
@@ -157,8 +157,8 @@ static const struct pvm_ftr_bits pvmid_aa64isar1[] = {
 	MAX_FEAT(ID_AA64ISAR1_EL1, JSCVT, IMP),
 	MAX_FEAT(ID_AA64ISAR1_EL1, FCMA, IMP),
 	MAX_FEAT(ID_AA64ISAR1_EL1, LRCPC, LRCPC3),
-	MAX_FEAT(ID_AA64ISAR1_EL1, GPA, IMP),
-	MAX_FEAT(ID_AA64ISAR1_EL1, GPI, IMP),
+	MAX_FEAT_FUNC(ID_AA64ISAR1_EL1, GPA, IMP, vm_has_ptrauth),
+	MAX_FEAT_FUNC(ID_AA64ISAR1_EL1, GPI, IMP, vm_has_ptrauth),
 	MAX_FEAT(ID_AA64ISAR1_EL1, FRINTTS, IMP),
 	MAX_FEAT(ID_AA64ISAR1_EL1, SB, IMP),
 	MAX_FEAT(ID_AA64ISAR1_EL1, SPECRES, COSP_RCTX),
-- 
2.47.3

[PATCH v2 02/19] KVM: arm64: selftests: Inject an abort before a vCPU's first run

[Fuad Tabba]

KVM commits an external abort injected with KVM_SET_VCPU_EVENTS
immediately, even before the vCPU's first run. For a normal VM on a pKVM
host that path panicked the hypervisor until commit a2bb5f5c67a95
("KVM: arm64: Pin the host vCPU before adjusting its PC under pKVM"). No
test injects before the first run.

Add one, checking that the guest takes the abort at its entry point.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 .../selftests/kvm/arm64/external_aborts.c     | 21 +++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/tools/testing/selftests/kvm/arm64/external_aborts.c b/tools/testing/selftests/kvm/arm64/external_aborts.c
index 1a858d0d82968..45c9300b2ca0e 100644
--- a/tools/testing/selftests/kvm/arm64/external_aborts.c
+++ b/tools/testing/selftests/kvm/arm64/external_aborts.c
@@ -113,6 +113,26 @@ static void vcpu_run_expect_sync(struct kvm_vcpu *vcpu)
 	__vcpu_run_expect(vcpu, UCALL_SYNC);
 }
 
+static void test_sea_before_run_guest(void)
+{
+	GUEST_FAIL("Guest ran before taking the injected abort");
+}
+
+/* Test that an abort injected before the vCPU's first run is taken at its entry point. */
+static void test_sea_before_run(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm = vm_create_with_dabt_handler(&vcpu, test_sea_before_run_guest,
+							expect_sea_handler);
+
+	expected_abort_pc = (u64)test_sea_before_run_guest;
+	sync_global_to_guest(vm, expected_abort_pc);
+
+	vcpu_inject_sea(vcpu);
+	vcpu_run_expect_done(vcpu);
+	kvm_vm_free(vm);
+}
+
 extern char test_mmio_abort_insn;
 
 static noinline void test_mmio_abort_guest(void)
@@ -420,6 +440,7 @@ static void test_serror_amo(void)
 
 int main(void)
 {
+	test_sea_before_run();
 	test_mmio_abort();
 	test_mmio_nisv();
 	test_mmio_nisv_abort();
-- 
2.47.3

[PATCH v2 03/19] KVM: selftests: Allow arm64 tests to create protected VMs

[Fuad Tabba]

On arm64, the VM type passed to KVM_CREATE_VM carries the IPA size,
which the selftest library fills in. The library rejects any type a test
provides in vm_shape.type. A protected VM is created by setting
KVM_VM_TYPE_ARM_PROTECTED in that type, so a test can't create one.

Accept KVM_VM_TYPE_ARM_PROTECTED from the test, and add the IPA size to
it as for any other VM.

Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/lib/kvm_util.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index d42e201eba763..96c1e70dba3d3 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -400,9 +400,10 @@ struct kvm_vm *____vm_create(struct vm_shape shape)
 	}
 
 #ifdef __aarch64__
-	TEST_ASSERT(!vm->type, "ARM doesn't support test-provided types");
+	TEST_ASSERT(!(vm->type & ~KVM_VM_TYPE_ARM_PROTECTED),
+		    "ARM supports only KVM_VM_TYPE_ARM_PROTECTED as a test-provided type");
 	if (vm->pa_bits != 40)
-		vm->type = KVM_VM_TYPE_ARM_IPA_SIZE(vm->pa_bits);
+		vm->type |= KVM_VM_TYPE_ARM_IPA_SIZE(vm->pa_bits);
 #endif
 
 	vm_open(vm);
-- 
2.47.3

[PATCH v2 04/19] tools: arm-smccc: Add the pKVM memory sharing function IDs

[Fuad Tabba]

The tools copy of arm-smccc.h lacks the pKVM guest hypercalls that the
protected-VM selftests make.

The kernel's arm-smccc.h defines the vendor-hyp UID with UUID_INIT(),
which requires linux/uuid.h, and tools doesn't have one. So rather than
sync the whole header, copy the function numbers and IDs of HYP_MEMINFO,
MEM_SHARE and MEM_UNSHARE from include/linux/arm-smccc.h.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/include/linux/arm-smccc.h | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tools/include/linux/arm-smccc.h b/tools/include/linux/arm-smccc.h
index 63ce9bebccd32..8c8f4ad639018 100644
--- a/tools/include/linux/arm-smccc.h
+++ b/tools/include/linux/arm-smccc.h
@@ -111,6 +111,10 @@
 /* KVM "vendor specific" services */
 #define ARM_SMCCC_KVM_FUNC_FEATURES		0
 #define ARM_SMCCC_KVM_FUNC_PTP			1
+/* Start of pKVM hypercall range */
+#define ARM_SMCCC_KVM_FUNC_HYP_MEMINFO		2
+#define ARM_SMCCC_KVM_FUNC_MEM_SHARE		3
+#define ARM_SMCCC_KVM_FUNC_MEM_UNSHARE		4
 #define ARM_SMCCC_KVM_FUNC_FEATURES_2		127
 #define ARM_SMCCC_KVM_NUM_FUNCS			128
 
@@ -133,6 +137,24 @@
 			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
 			   ARM_SMCCC_KVM_FUNC_PTP)
 
+#define ARM_SMCCC_VENDOR_HYP_KVM_HYP_MEMINFO_FUNC_ID			\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_64,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_KVM_FUNC_HYP_MEMINFO)
+
+#define ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID			\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_64,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_KVM_FUNC_MEM_SHARE)
+
+#define ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID			\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL,				\
+			   ARM_SMCCC_SMC_64,				\
+			   ARM_SMCCC_OWNER_VENDOR_HYP,			\
+			   ARM_SMCCC_KVM_FUNC_MEM_UNSHARE)
+
 /* ptp_kvm counter type ID */
 #define KVM_PTP_VIRT_COUNTER			0
 #define KVM_PTP_PHYS_COUNTER			1
-- 
2.47.3

[PATCH v2 05/19] KVM: selftests: Export the ucall pool's guest physical address

[Fuad Tabba]

A protected guest's memory is private until the guest itself shares it,
so the guest has to share the ucall pool before its first ucall. The
host can't do that for it. Only the guest's copy of ucall_pool holds the
pool's address.

Add get_ucall_pool_gpa(), which reads that copy, so that the host can
pass the address to the guest.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/include/ucall_common.h | 1 +
 tools/testing/selftests/kvm/lib/ucall_common.c     | 9 ++++++++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/include/ucall_common.h b/tools/testing/selftests/kvm/include/ucall_common.h
index cbdcb0a50c4f9..3a6cdf9aa0464 100644
--- a/tools/testing/selftests/kvm/include/ucall_common.h
+++ b/tools/testing/selftests/kvm/include/ucall_common.h
@@ -41,6 +41,7 @@ __printf(5, 6) void ucall_assert(u64 cmd, const char *exp,
 u64 get_ucall(struct kvm_vcpu *vcpu, struct ucall *uc);
 void ucall_init(struct kvm_vm *vm, gpa_t mmio_gpa);
 int ucall_nr_pages_required(u64 page_size);
+gpa_t get_ucall_pool_gpa(struct kvm_vm *vm);
 
 /*
  * Perform userspace call without any associated data.  This bare call avoids
diff --git a/tools/testing/selftests/kvm/lib/ucall_common.c b/tools/testing/selftests/kvm/lib/ucall_common.c
index 029ce21f9f2fa..aa1f0f92c6d1d 100644
--- a/tools/testing/selftests/kvm/lib/ucall_common.c
+++ b/tools/testing/selftests/kvm/lib/ucall_common.c
@@ -21,10 +21,17 @@ int ucall_nr_pages_required(u64 page_size)
 
 /*
  * ucall_pool holds per-VM values (global data is duplicated by each VM), it
- * must not be accessed from host code.
+ * must not be accessed from host code. The host reads only a VM's copy.
  */
 static struct ucall_header *ucall_pool;
 
+gpa_t get_ucall_pool_gpa(struct kvm_vm *vm)
+{
+	struct ucall_header **pool = addr_gva2hva(vm, (gva_t)&ucall_pool);
+
+	return addr_gva2gpa(vm, (gva_t)*pool);
+}
+
 void ucall_init(struct kvm_vm *vm, gpa_t mmio_gpa)
 {
 	struct ucall_header *hdr;
-- 
2.47.3

[PATCH v2 06/19] KVM: arm64: selftests: Add protected-VM boot trampoline

[Fuad Tabba]

From: Will Deacon <will@kernel.org>

EL2 takes only PC and X0 from the host when it boots a protected vCPU,
and resets everything else. So the registers the selftest library sets
before the first run never reach the guest.

Boot a protected guest through a trampoline that restores X0-X7, seven
EL1 system registers and the stack pointer from a block X0 points to,
then enables the MMU and branches to the guest's entry point.

Signed-off-by: Will Deacon <will@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../selftests/kvm/include/arm64/pkvm.h        | 49 ++++++++++++++++
 .../selftests/kvm/lib/arm64/pvm-entry.S       | 57 +++++++++++++++++++
 3 files changed, 107 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/arm64/pkvm.h
 create mode 100644 tools/testing/selftests/kvm/lib/arm64/pvm-entry.S

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 17a4256e90819..c254e82f9a786 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -35,6 +35,7 @@ LIBKVM_arm64 += lib/arm64/gic_v3.c
 LIBKVM_arm64 += lib/arm64/gic_v3_its.c
 LIBKVM_arm64 += lib/arm64/handlers.S
 LIBKVM_arm64 += lib/arm64/processor.c
+LIBKVM_arm64 += lib/arm64/pvm-entry.S
 LIBKVM_arm64 += lib/arm64/spinlock.c
 LIBKVM_arm64 += lib/arm64/ucall.c
 LIBKVM_arm64 += lib/arm64/vgic.c
diff --git a/tools/testing/selftests/kvm/include/arm64/pkvm.h b/tools/testing/selftests/kvm/include/arm64/pkvm.h
new file mode 100644
index 0000000000000..29600342eca13
--- /dev/null
+++ b/tools/testing/selftests/kvm/include/arm64/pkvm.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2025 Google LLC
+ * Author: Will Deacon <will@kernel.org>
+ */
+
+#ifndef SELFTEST_KVM_ARM64_PKVM_H
+#define SELFTEST_KVM_ARM64_PKVM_H
+
+#ifndef __ASSEMBLER__
+/*
+ * Register state the boot trampoline (pvm-entry.S) restores. Only PC and X0
+ * survive to a protected guest's first run, so X0 points at this block.
+ */
+struct pvm_boot_args {
+	u64	jump_tgt;
+
+	u64	regs[8];
+	u64	cpacr_el1;
+	u64	sctlr_el1;
+	u64	tcr_el1;
+	u64	mair_el1;
+	u64	ttbr0_el1;
+	u64	vbar_el1;
+
+	u64	tpidr_el1;
+	u64	sp_el1;
+};
+#endif
+
+#define PVM_BOOT_ARGS_OFF_JUMP_TGT	0
+#define PVM_BOOT_ARGS_OFF_X0		(PVM_BOOT_ARGS_OFF_JUMP_TGT + 8)
+#define PVM_BOOT_ARGS_OFF_X1		(PVM_BOOT_ARGS_OFF_X0 + 8)
+#define PVM_BOOT_ARGS_OFF_X2		(PVM_BOOT_ARGS_OFF_X1 + 8)
+#define PVM_BOOT_ARGS_OFF_X3		(PVM_BOOT_ARGS_OFF_X2 + 8)
+#define PVM_BOOT_ARGS_OFF_X4		(PVM_BOOT_ARGS_OFF_X3 + 8)
+#define PVM_BOOT_ARGS_OFF_X5		(PVM_BOOT_ARGS_OFF_X4 + 8)
+#define PVM_BOOT_ARGS_OFF_X6		(PVM_BOOT_ARGS_OFF_X5 + 8)
+#define PVM_BOOT_ARGS_OFF_X7		(PVM_BOOT_ARGS_OFF_X6 + 8)
+#define PVM_BOOT_ARGS_OFF_CPACR_EL1	(PVM_BOOT_ARGS_OFF_X7 + 8)
+#define PVM_BOOT_ARGS_OFF_SCTLR_EL1	(PVM_BOOT_ARGS_OFF_CPACR_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_TCR_EL1	(PVM_BOOT_ARGS_OFF_SCTLR_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_MAIR_EL1	(PVM_BOOT_ARGS_OFF_TCR_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_TTBR0_EL1	(PVM_BOOT_ARGS_OFF_MAIR_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_VBAR_EL1	(PVM_BOOT_ARGS_OFF_TTBR0_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_TPIDR_EL1	(PVM_BOOT_ARGS_OFF_VBAR_EL1 + 8)
+#define PVM_BOOT_ARGS_OFF_SP_EL1	(PVM_BOOT_ARGS_OFF_TPIDR_EL1 + 8)
+
+#endif /* SELFTEST_KVM_ARM64_PKVM_H */
diff --git a/tools/testing/selftests/kvm/lib/arm64/pvm-entry.S b/tools/testing/selftests/kvm/lib/arm64/pvm-entry.S
new file mode 100644
index 0000000000000..d00abe03f8f84
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/arm64/pvm-entry.S
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2025 Google LLC
+ * Author: Will Deacon <will@kernel.org>
+ */
+
+#include "pkvm.h"
+
+	.text
+	.balign 16
+	.globl	pvm_entry_start
+pvm_entry_start:
+	mov	x30, x0
+
+	ldr	x0, [x30, #PVM_BOOT_ARGS_OFF_X0]
+	ldr	x1, [x30, #PVM_BOOT_ARGS_OFF_X1]
+	ldr	x2, [x30, #PVM_BOOT_ARGS_OFF_X2]
+	ldr	x3, [x30, #PVM_BOOT_ARGS_OFF_X3]
+	ldr	x4, [x30, #PVM_BOOT_ARGS_OFF_X4]
+	ldr	x5, [x30, #PVM_BOOT_ARGS_OFF_X5]
+	ldr	x6, [x30, #PVM_BOOT_ARGS_OFF_X6]
+	ldr	x7, [x30, #PVM_BOOT_ARGS_OFF_X7]
+
+	ldr	x8, [x30, #PVM_BOOT_ARGS_OFF_JUMP_TGT]
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_CPACR_EL1]
+	msr	cpacr_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_TCR_EL1]
+	msr	tcr_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_MAIR_EL1]
+	msr	mair_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_TTBR0_EL1]
+	msr	ttbr0_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_VBAR_EL1]
+	msr	vbar_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_TPIDR_EL1]
+	msr	tpidr_el1, x9
+
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_SP_EL1]
+	mov	sp, x9
+
+	/* No stale TLB entries, and the writes above in effect, before the MMU is on. */
+	tlbi	vmalle1
+	dsb	nsh
+	isb
+	ldr	x9, [x30, #PVM_BOOT_ARGS_OFF_SCTLR_EL1]
+	msr	sctlr_el1, x9
+	isb
+
+	br	x8
+	.globl	pvm_entry_end
+pvm_entry_end:
-- 
2.47.3

[PATCH v2 07/19] KVM: arm64: selftests: Add protected-VM library helpers

[Fuad Tabba]

A protected-VM test has to create the VM with the protected type and
PSCI 0.2, boot each vCPU through the trampoline, and have the guest
share the ucall pool before its first ucall.

Add lib/arm64/pkvm.c to do this. It also has a run loop, an SMCCC call
for the guest and a reader of the locked-memory count. A secondary
runs once with immediate_exit set, since a vCPU is a valid CPU_ON
target only after its first KVM_RUN. No capability advertises pKVM,
so pkvm_supported() tries to create a protected VM. The host filters
capabilities per VM, so pkvm_has_cap() checks one on a protected
VM's fd.

Co-developed-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/include/arm64/pkvm.h        |  26 ++
 tools/testing/selftests/kvm/lib/arm64/pkvm.c  | 255 ++++++++++++++++++
 3 files changed, 282 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/lib/arm64/pkvm.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index c254e82f9a786..e173a9adad3f2 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -34,6 +34,7 @@ LIBKVM_arm64 += lib/arm64/gic.c
 LIBKVM_arm64 += lib/arm64/gic_v3.c
 LIBKVM_arm64 += lib/arm64/gic_v3_its.c
 LIBKVM_arm64 += lib/arm64/handlers.S
+LIBKVM_arm64 += lib/arm64/pkvm.c
 LIBKVM_arm64 += lib/arm64/processor.c
 LIBKVM_arm64 += lib/arm64/pvm-entry.S
 LIBKVM_arm64 += lib/arm64/spinlock.c
diff --git a/tools/testing/selftests/kvm/include/arm64/pkvm.h b/tools/testing/selftests/kvm/include/arm64/pkvm.h
index 29600342eca13..899be3d731b5a 100644
--- a/tools/testing/selftests/kvm/include/arm64/pkvm.h
+++ b/tools/testing/selftests/kvm/include/arm64/pkvm.h
@@ -8,6 +8,9 @@
 #define SELFTEST_KVM_ARM64_PKVM_H
 
 #ifndef __ASSEMBLER__
+#include "kvm_util.h"
+#include "ucall_common.h"
+
 /*
  * Register state the boot trampoline (pvm-entry.S) restores. Only PC and X0
  * survive to a protected guest's first run, so X0 points at this block.
@@ -26,6 +29,29 @@ struct pvm_boot_args {
 	u64	tpidr_el1;
 	u64	sp_el1;
 };
+
+/* Memslot of the trampoline's page. Test memslots start above it. */
+#define PKVM_IDMAP_MEMSLOT	1
+
+bool pkvm_supported(void);
+bool pkvm_has_cap(long cap);
+struct kvm_vm *pkvm_vm_create(unsigned int nr_vcpus, u32 features, struct kvm_vcpu *vcpus[]);
+struct kvm_vm *pkvm_vm_create_with_one_vcpu(struct kvm_vcpu **vcpu, void *guest_code);
+
+/* Call after all register setup: later register writes don't reach the guest. */
+void pkvm_vcpu_setup_boot(struct kvm_vcpu *vcpu, void *guest_code);
+
+/* Call before the primary's first run. */
+gpa_t pkvm_vcpu_setup_boot_secondary(struct kvm_vcpu *vcpu, void *guest_code);
+gpa_t pkvm_idmap_gpa(struct kvm_vm *vm);
+
+void pkvm_run_loop(struct kvm_vcpu *vcpu, void (*on_sync)(struct kvm_vcpu *vcpu, struct ucall *uc));
+size_t pkvm_host_locked_vm(void);
+
+/* Called from the guest. */
+unsigned long pkvm_hvc(u32 func, u64 arg0, u64 arg1, u64 arg2);
+
+void pkvm_guest_init(void);
 #endif
 
 #define PVM_BOOT_ARGS_OFF_JUMP_TGT	0
diff --git a/tools/testing/selftests/kvm/lib/arm64/pkvm.c b/tools/testing/selftests/kvm/lib/arm64/pkvm.c
new file mode 100644
index 0000000000000..5119817f8b13b
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/arm64/pkvm.c
@@ -0,0 +1,255 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Protected-VM support for the arm64 KVM selftest library.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ * Author: Will Deacon <will@kernel.org>
+ */
+
+#include <errno.h>
+
+#include <linux/arm-smccc.h>
+#include <linux/kernel.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "ucall_common.h"
+
+/* Set in the guest's copy only, by pkvm_vm_create(). */
+static gpa_t pkvm_ucall_pool_gpa;
+static size_t pkvm_ucall_pool_size;
+
+extern char pvm_entry_start[], pvm_entry_end[];
+
+#define ASSERT_BOOT_ARGS_OFF(field, off)				\
+	static_assert(offsetof(struct pvm_boot_args, field) == PVM_BOOT_ARGS_OFF_##off, #off)
+
+ASSERT_BOOT_ARGS_OFF(jump_tgt, JUMP_TGT);
+ASSERT_BOOT_ARGS_OFF(regs[0], X0);
+ASSERT_BOOT_ARGS_OFF(regs[1], X1);
+ASSERT_BOOT_ARGS_OFF(regs[2], X2);
+ASSERT_BOOT_ARGS_OFF(regs[3], X3);
+ASSERT_BOOT_ARGS_OFF(regs[4], X4);
+ASSERT_BOOT_ARGS_OFF(regs[5], X5);
+ASSERT_BOOT_ARGS_OFF(regs[6], X6);
+ASSERT_BOOT_ARGS_OFF(regs[7], X7);
+ASSERT_BOOT_ARGS_OFF(cpacr_el1, CPACR_EL1);
+ASSERT_BOOT_ARGS_OFF(sctlr_el1, SCTLR_EL1);
+ASSERT_BOOT_ARGS_OFF(tcr_el1, TCR_EL1);
+ASSERT_BOOT_ARGS_OFF(mair_el1, MAIR_EL1);
+ASSERT_BOOT_ARGS_OFF(ttbr0_el1, TTBR0_EL1);
+ASSERT_BOOT_ARGS_OFF(vbar_el1, VBAR_EL1);
+ASSERT_BOOT_ARGS_OFF(tpidr_el1, TPIDR_EL1);
+ASSERT_BOOT_ARGS_OFF(sp_el1, SP_EL1);
+
+unsigned long pkvm_hvc(u32 func, u64 arg0, u64 arg1, u64 arg2)
+{
+	struct arm_smccc_res res;
+
+	smccc_hvc(func, arg0, arg1, arg2, 0, 0, 0, 0, &res);
+	return res.a0;
+}
+
+/* Shares whole granules, so pages next to the pool are shared with it. */
+void pkvm_guest_init(void)
+{
+	u64 granule = pkvm_hvc(ARM_SMCCC_VENDOR_HYP_KVM_HYP_MEMINFO_FUNC_ID, 0, 0, 0);
+	gpa_t end = pkvm_ucall_pool_gpa + pkvm_ucall_pool_size;
+	gpa_t gpa;
+
+	for (gpa = pkvm_ucall_pool_gpa & ~(granule - 1); gpa < end; gpa += granule)
+		GUEST_ASSERT_EQ(pkvm_hvc(ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID, gpa, 0, 0), 0);
+}
+
+bool pkvm_supported(void)
+{
+	int kvm_fd, vm_fd;
+
+	kvm_fd = open_kvm_dev_path_or_exit();
+	vm_fd = __kvm_ioctl(kvm_fd, KVM_CREATE_VM, (void *)KVM_VM_TYPE_ARM_PROTECTED);
+
+	if (vm_fd >= 0)
+		close(vm_fd);
+	close(kvm_fd);
+
+	return vm_fd >= 0;
+}
+
+/* The host filters capabilities per VM, so ask a protected VM. */
+bool pkvm_has_cap(long cap)
+{
+	struct kvm_vm *vm = vm_create_barebones_type(KVM_VM_TYPE_ARM_PROTECTED);
+	bool ret = vm_check_cap(vm, cap);
+
+	kvm_vm_free(vm);
+	return ret;
+}
+
+/*
+ * The trampoline turns the MMU on, so it runs from an identity-mapped page:
+ * the first guest page of a host page at the top of the IPA space.
+ */
+gpa_t pkvm_idmap_gpa(struct kvm_vm *vm)
+{
+	unsigned int npages = vm_adjust_num_guest_pages(vm->mode, 1);
+
+	return (vm_compute_max_gfn(vm) + 1 - npages) * vm->page_size;
+}
+
+struct kvm_vm *pkvm_vm_create(unsigned int nr_vcpus, u32 features, struct kvm_vcpu *vcpus[])
+{
+	struct vm_shape shape = VM_SHAPE_DEFAULT;
+	struct kvm_vm *vm;
+	gpa_t idmap_gpa;
+	unsigned int i;
+
+	TEST_ASSERT(pvm_entry_end - pvm_entry_start +
+		    nr_vcpus * sizeof(struct pvm_boot_args) <= MIN_PAGE_SIZE,
+		    "trampoline and %u boot-args blocks overflow the idmap page", nr_vcpus);
+
+	shape.type = KVM_VM_TYPE_ARM_PROTECTED;
+	vm = __vm_create(shape, nr_vcpus, 0);
+
+	for (i = 0; i < nr_vcpus; i++) {
+		struct kvm_vcpu_init init;
+
+		kvm_get_default_vcpu_target(vm, &init);
+		init.features[0] |= features | BIT(KVM_ARM_VCPU_PSCI_0_2);
+		if (i > 0)
+			init.features[0] |= BIT(KVM_ARM_VCPU_POWER_OFF);
+
+		vcpus[i] = aarch64_vcpu_add(vm, i, &init, NULL);
+	}
+	kvm_arch_vm_finalize_vcpus(vm);
+
+	idmap_gpa = pkvm_idmap_gpa(vm);
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, idmap_gpa, PKVM_IDMAP_MEMSLOT,
+				    vm_adjust_num_guest_pages(vm->mode, 1), 0);
+	memcpy(addr_gpa2hva(vm, idmap_gpa), pvm_entry_start, pvm_entry_end - pvm_entry_start);
+	virt_map(vm, idmap_gpa, idmap_gpa, 1);
+
+	vm_init_descriptor_tables(vm);
+	for (i = 0; i < nr_vcpus; i++) {
+		vcpu_set_reg(vcpus[i], ARM64_CORE_REG(regs.pc), idmap_gpa);
+		vcpu_init_descriptor_tables(vcpus[i]);
+	}
+
+	write_guest_global(vm, pkvm_ucall_pool_gpa, get_ucall_pool_gpa(vm));
+	write_guest_global(vm, pkvm_ucall_pool_size,
+			   ucall_nr_pages_required(vm->page_size) * vm->page_size);
+
+	return vm;
+}
+
+/* Every block must be written before the primary's first run, when the host donates the page. */
+static gpa_t pkvm_boot_args_gpa(struct kvm_vcpu *vcpu)
+{
+	struct kvm_vm *vm = vcpu->vm;
+
+	return pkvm_idmap_gpa(vm) + vm->page_size - (vcpu->id + 1) * sizeof(struct pvm_boot_args);
+}
+
+static void pkvm_snapshot_boot_args(struct kvm_vcpu *vcpu, void *guest_code)
+{
+	struct pvm_boot_args *args = addr_gpa2hva(vcpu->vm, pkvm_boot_args_gpa(vcpu));
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(args->regs); i++)
+		args->regs[i] = vcpu_get_reg(vcpu, ARM64_CORE_REG(regs.regs[i]));
+
+	args->jump_tgt  = (u64)guest_code;
+	args->cpacr_el1 = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_CPACR_EL1));
+	args->sctlr_el1 = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL1));
+	args->tcr_el1   = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TCR_EL1));
+	args->mair_el1  = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MAIR_EL1));
+	args->ttbr0_el1 = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TTBR0_EL1));
+	args->vbar_el1  = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_VBAR_EL1));
+	args->tpidr_el1 = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_TPIDR_EL1));
+	args->sp_el1    = vcpu_get_reg(vcpu, ARM64_CORE_REG(sp_el1));
+}
+
+void pkvm_vcpu_setup_boot(struct kvm_vcpu *vcpu, void *guest_code)
+{
+	pkvm_snapshot_boot_args(vcpu, guest_code);
+	vcpu_set_reg(vcpu, ARM64_CORE_REG(regs.regs[0]), pkvm_boot_args_gpa(vcpu));
+}
+
+gpa_t pkvm_vcpu_setup_boot_secondary(struct kvm_vcpu *vcpu, void *guest_code)
+{
+	int ret;
+
+	pkvm_snapshot_boot_args(vcpu, guest_code);
+
+	/*
+	 * A guest's CPU_ON requires the target's hyp vCPU, which KVM_RUN creates.
+	 * With immediate_exit set, KVM_RUN returns before the vCPU runs.
+	 */
+	vcpu->run->immediate_exit = 1;
+	ret = __vcpu_run(vcpu);
+	vcpu->run->immediate_exit = 0;
+	TEST_ASSERT(ret == -1 && errno == EINTR,
+		    "secondary hyp-vCPU registration: rc %d errno %d", ret, errno);
+
+	return pkvm_boot_args_gpa(vcpu);
+}
+
+struct kvm_vm *pkvm_vm_create_with_one_vcpu(struct kvm_vcpu **vcpu, void *guest_code)
+{
+	struct kvm_vm *vm = pkvm_vm_create(1, 0, vcpu);
+
+	pkvm_vcpu_setup_boot(*vcpu, guest_code);
+	return vm;
+}
+
+void pkvm_run_loop(struct kvm_vcpu *vcpu, void (*on_sync)(struct kvm_vcpu *vcpu, struct ucall *uc))
+{
+	bool done = false;
+	struct ucall uc;
+
+	while (!done) {
+		u64 cmd;
+
+		vcpu_run(vcpu);
+
+		switch (cmd = get_ucall(vcpu, &uc)) {
+		case UCALL_SYNC:
+			if (on_sync)
+				on_sync(vcpu, &uc);
+			break;
+		case UCALL_DONE:
+			done = true;
+			break;
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			break;
+		case UCALL_PRINTF:
+			REPORT_GUEST_PRINTF(uc);
+			break;
+		default:
+			TEST_FAIL("Unexpected guest exit: ucall %lu, arg %lu", cmd, uc.args[1]);
+		}
+	}
+}
+
+/* Bytes the current process has locked, per RLIMIT_MEMLOCK accounting. */
+size_t pkvm_host_locked_vm(void)
+{
+	size_t size = 0, locked_kb = 0;
+	char *line = NULL;
+	FILE *f;
+
+	f = fopen("/proc/self/status", "r");
+	TEST_ASSERT(f, "fopen /proc/self/status");
+
+	while (getline(&line, &size, f) > 0) {
+		if (sscanf(line, "VmLck:\t%zu kB", &locked_kb) == 1)
+			break;
+	}
+
+	free(line);
+	fclose(f);
+
+	return locked_kb << 10;
+}
-- 
2.47.3

[PATCH v2 08/19] KVM: arm64: selftests: Add pkvm_host_abi host-interface test

[Fuad Tabba]

The host isn't trusted with a protected guest's state, so once a
protected vCPU has run, KVM rejects the ioctls that would read or change
that state. The host can still inject a virtual SError and read back
whether one is pending. The VM gets only the capabilities and vCPU
features pKVM supports.

Check what the host's KVM interface allows and rejects for a protected
VM, before and after the first run, and that the host accounts the VM's
memory as locked until the VM is freed.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_host_abi.c       | 435 ++++++++++++++++++
 2 files changed, 436 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_host_abi.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index e173a9adad3f2..42e530e0e573b 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -190,6 +190,7 @@ TEST_GEN_PROGS_arm64 += arm64/nv_pre_fault_memory_test
 TEST_GEN_PROGS_arm64 += arm64/external_aborts
 TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
+TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_host_abi.c b/tools/testing/selftests/kvm/arm64/pkvm_host_abi.c
new file mode 100644
index 0000000000000..4da73585e146a
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_host_abi.c
@@ -0,0 +1,435 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_host_abi - what the host's KVM interface allows a protected VM.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <errno.h>
+#include <sys/mman.h>
+
+#include <linux/bits.h>
+#include <linux/psci.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define TEST_MEMSLOT	(PKVM_IDMAP_MEMSLOT + 1)
+#define TEST_GPA	BIT_ULL(30)
+#define TEST_PAGES	16
+
+/* Every probe sets PSCI_0_2, so that VCPU_INIT fails only on the feature under test. */
+#define PVM_INIT_FEATURES	BIT(KVM_ARM_VCPU_PSCI_0_2)
+
+#define CAP(c)	{ KVM_CAP_##c, "KVM_CAP_" #c }
+
+/* Caps a protected VM has on any pKVM host. */
+static const struct {
+	long cap;
+	const char *name;
+} allowed_caps[] = {
+	CAP(ONE_REG),
+	CAP(ARM_PSCI),
+	CAP(ARM_PSCI_0_2),
+	CAP(NR_VCPUS),
+	CAP(MAX_VCPUS),
+	CAP(MAX_VCPU_ID),
+	CAP(ARM_VM_IPA_SIZE),
+	CAP(MP_STATE),
+	CAP(VCPU_EVENTS),
+	CAP(VCPU_ATTRIBUTES),
+	CAP(IMMEDIATE_EXIT),
+	CAP(IOEVENTFD),
+	CAP(IRQFD_RESAMPLE),
+	CAP(ARM_IRQ_LINE_LAYOUT_2),
+};
+
+/* Set in the guest's copy by the host before the first run. */
+static u64 test_size;
+
+static void guest_finalize(void)
+{
+	pkvm_guest_init();
+	GUEST_DONE();
+}
+
+/* One write per MIN_PAGE_SIZE faults in every page of the test memslot. */
+static void guest_touch_memory(void)
+{
+	volatile u64 *end = (u64 *)(TEST_GPA + test_size);
+	volatile u64 *p;
+
+	pkvm_guest_init();
+
+	for (p = (u64 *)TEST_GPA; p < end; p += MIN_PAGE_SIZE / sizeof(*p))
+		*p = 1;
+
+	GUEST_DONE();
+}
+
+/* A protected VM with one vCPU and no VCPU_INIT yet. */
+static struct kvm_vcpu *create_uninitialized_vcpu(struct kvm_vm **vm_out)
+{
+	struct vm_shape shape = VM_SHAPE_DEFAULT;
+
+	shape.type = KVM_VM_TYPE_ARM_PROTECTED;
+	*vm_out = __vm_create(shape, 1, 0);
+	return __vm_vcpu_add(*vm_out, 0);
+}
+
+/* The errno of KVM_ARM_VCPU_INIT with @features on a protected vCPU, or 0. */
+static int vcpu_init_errno(u32 features)
+{
+	struct kvm_vcpu_init init;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int err;
+
+	vcpu = create_uninitialized_vcpu(&vm);
+	kvm_get_default_vcpu_target(vm, &init);
+	init.features[0] |= features;
+
+	err = __vcpu_ioctl(vcpu, KVM_ARM_VCPU_INIT, &init) ? errno : 0;
+
+	kvm_vm_free(vm);
+	return err;
+}
+
+static void assert_feature_accepted(u32 feature, const char *name)
+{
+	int err = vcpu_init_errno(PVM_INIT_FEATURES | feature);
+
+	TEST_ASSERT(!err, "%s on a protected vCPU: errno %d", name, err);
+	ksft_test_result_pass("%s accepted at VCPU_INIT\n", name);
+}
+
+static void assert_feature_rejected(u32 feature, const char *name)
+{
+	int err = vcpu_init_errno(PVM_INIT_FEATURES | feature);
+
+	TEST_ASSERT(err == EINVAL, "%s on a protected vCPU: errno %d", name, err);
+	ksft_test_result_pass("%s rejected at VCPU_INIT\n", name);
+}
+
+static void test_vcpu_init_conditional_features(void)
+{
+	int err = vcpu_init_errno(0);
+
+	TEST_ASSERT(err == EINVAL, "no PSCI_0_2 on a protected vCPU: errno %d", err);
+	assert_feature_accepted(BIT(KVM_ARM_VCPU_PSCI_0_2), "PSCI_0_2");
+
+	if (kvm_has_cap(KVM_CAP_ARM_PMU_V3))
+		assert_feature_rejected(BIT(KVM_ARM_VCPU_PMU_V3), "PMU_V3");
+	else
+		ksft_test_result_skip("PMU_V3 rejected at VCPU_INIT: PMUv3 unavailable\n");
+
+	/* A protected VM may have SVE exactly when its VM fd reports KVM_CAP_ARM_SVE. */
+	if (pkvm_has_cap(KVM_CAP_ARM_SVE))
+		assert_feature_accepted(BIT(KVM_ARM_VCPU_SVE), "SVE");
+	else if (kvm_has_cap(KVM_CAP_ARM_SVE))
+		assert_feature_rejected(BIT(KVM_ARM_VCPU_SVE), "SVE");
+	else
+		ksft_test_result_skip("SVE at VCPU_INIT: SVE unavailable\n");
+
+	if (kvm_has_cap(KVM_CAP_ARM_PTRAUTH_ADDRESS) && kvm_has_cap(KVM_CAP_ARM_PTRAUTH_GENERIC))
+		assert_feature_accepted(BIT(KVM_ARM_VCPU_PTRAUTH_ADDRESS) |
+					BIT(KVM_ARM_VCPU_PTRAUTH_GENERIC), "PtrAuth");
+	else
+		ksft_test_result_skip("PtrAuth accepted at VCPU_INIT: PtrAuth unavailable\n");
+}
+
+/* KVM_ARM_VCPU_INIT can be repeated until the vCPU's first run. */
+static void test_vcpu_init_repeated(void)
+{
+	struct kvm_vcpu_init init;
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vcpu = create_uninitialized_vcpu(&vm);
+	kvm_get_default_vcpu_target(vm, &init);
+	init.features[0] |= PVM_INIT_FEATURES;
+	vcpu_ioctl(vcpu, KVM_ARM_VCPU_INIT, &init);
+	vcpu_ioctl(vcpu, KVM_ARM_VCPU_INIT, &init);
+	kvm_vm_free(vm);
+
+	ksft_test_result_pass("VCPU_INIT repeated before the first run\n");
+}
+
+static void assert_ext(struct kvm_vm *vm, long ext, bool want_nonzero, const char *name)
+{
+	int r = vm_check_cap(vm, ext);
+
+	if (want_nonzero)
+		TEST_ASSERT(r > 0, "%s: expected nonzero, got %d", name, r);
+	else
+		TEST_ASSERT(r == 0, "%s: expected 0, got %d", name, r);
+}
+
+static void test_caps(void)
+{
+	struct kvm_vm *vm = vm_create_barebones_type(KVM_VM_TYPE_ARM_PROTECTED);
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(allowed_caps); i++) {
+		assert_ext(vm, allowed_caps[i].cap, true, allowed_caps[i].name);
+		ksft_test_result_pass("%s allowed\n", allowed_caps[i].name);
+	}
+
+	if (kvm_has_cap(KVM_CAP_ARM_PTRAUTH_ADDRESS) && kvm_has_cap(KVM_CAP_ARM_PTRAUTH_GENERIC)) {
+		assert_ext(vm, KVM_CAP_ARM_PTRAUTH_ADDRESS, true, "KVM_CAP_ARM_PTRAUTH_ADDRESS");
+		assert_ext(vm, KVM_CAP_ARM_PTRAUTH_GENERIC, true, "KVM_CAP_ARM_PTRAUTH_GENERIC");
+		ksft_test_result_pass("KVM_CAP_ARM_PTRAUTH_* allowed\n");
+	} else {
+		ksft_test_result_skip("KVM_CAP_ARM_PTRAUTH_* allowed: PtrAuth unavailable\n");
+	}
+
+	if (kvm_has_cap(KVM_CAP_ARM_PMU_V3)) {
+		assert_ext(vm, KVM_CAP_ARM_PMU_V3, false, "KVM_CAP_ARM_PMU_V3");
+		ksft_test_result_pass("KVM_CAP_ARM_PMU_V3 blocked\n");
+	} else {
+		ksft_test_result_skip("KVM_CAP_ARM_PMU_V3 blocked: PMUv3 unavailable\n");
+	}
+
+	assert_ext(vm, KVM_CAP_READONLY_MEM, false, "KVM_CAP_READONLY_MEM");
+	ksft_test_result_pass("KVM_CAP_READONLY_MEM blocked\n");
+
+	assert_ext(vm, KVM_CAP_SET_GUEST_DEBUG, false, "KVM_CAP_SET_GUEST_DEBUG");
+	ksft_test_result_pass("KVM_CAP_SET_GUEST_DEBUG blocked\n");
+
+	assert_ext(vm, KVM_CAP_ARM_INJECT_EXT_DABT, false, "KVM_CAP_ARM_INJECT_EXT_DABT");
+	ksft_test_result_pass("KVM_CAP_ARM_INJECT_EXT_DABT blocked\n");
+
+	/* Blocked explicitly, and reads 0 anyway on a host without MTE. */
+	assert_ext(vm, KVM_CAP_ARM_MTE, false, "KVM_CAP_ARM_MTE");
+	ksft_test_result_pass("KVM_CAP_ARM_MTE blocked\n");
+
+	kvm_vm_free(vm);
+}
+
+/* VM ioctls whose capability a protected VM doesn't get fail with EINVAL. */
+static void test_vm_ioctls_rejected(void)
+{
+	struct kvm_vm *vm = vm_create_barebones_type(KVM_VM_TYPE_ARM_PROTECTED);
+	struct kvm_arm_counter_offset offset = {};
+	struct reg_mask_range range = {};
+	int ret;
+
+	ret = __kvm_has_device_attr(vm->fd, KVM_ARM_VM_SMCCC_CTRL, KVM_ARM_VM_SMCCC_FILTER);
+	TEST_ASSERT(ret == -1 && errno == EINVAL, "SMCCC_CTRL: rc %d errno %d", ret, errno);
+	ret = __vm_ioctl(vm, KVM_ARM_SET_COUNTER_OFFSET, &offset);
+	TEST_ASSERT(ret == -1 && errno == EINVAL, "SET_COUNTER_OFFSET: rc %d errno %d", ret, errno);
+	ret = __vm_ioctl(vm, KVM_ARM_GET_REG_WRITABLE_MASKS, &range);
+	TEST_ASSERT(ret == -1 && errno == EINVAL, "GET_REG_WRITABLE_MASKS: rc %d errno %d",
+		    ret, errno);
+
+	kvm_vm_free(vm);
+	ksft_test_result_pass("VM ioctls without their capability rejected\n");
+}
+
+/* The host allows every ONE_REG until a protected vCPU's first KVM_RUN. */
+static void test_one_reg_gate(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	u64 val;
+	int ret;
+
+	vm = pkvm_vm_create(1, 0, &vcpu);
+
+	/*
+	 * pkvm_vcpu_setup_boot() snapshots regs[0..7] and SCTLR_EL1 for the
+	 * trampoline, so write back SCTLR_EL1 unchanged and probe regs[10].
+	 */
+	ret = __vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL1), &val);
+	TEST_ASSERT_EQ(ret, 0);
+	ret = __vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL1), val);
+	TEST_ASSERT_EQ(ret, 0);
+	ret = __vcpu_get_reg(vcpu, ARM64_CORE_REG(regs.pc), &val);
+	TEST_ASSERT_EQ(ret, 0);
+	ret = __vcpu_set_reg(vcpu, ARM64_CORE_REG(regs.regs[10]), 0x1234);
+	TEST_ASSERT_EQ(ret, 0);
+
+	pkvm_vcpu_setup_boot(vcpu, guest_finalize);
+	pkvm_run_loop(vcpu, NULL);
+
+	ret = __vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL1), 0);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "non-CORE post-run: rc %d errno %d", ret, errno);
+	ret = __vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_SCTLR_EL1), &val);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "non-CORE get post-run: rc %d errno %d", ret, errno);
+
+	ret = __vcpu_get_reg(vcpu, ARM64_CORE_REG(regs.pc), &val);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "CORE pc post-run: rc %d errno %d", ret, errno);
+	ret = __vcpu_set_reg(vcpu, ARM64_CORE_REG(regs.regs[4]), 0);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "CORE regs[4] post-run: rc %d errno %d", ret, errno);
+	ksft_test_result_pass("ONE_REG gate\n");
+
+	kvm_vm_free(vm);
+}
+
+static void test_vcpu_ioctl_gates(void)
+{
+	struct kvm_guest_debug dbg = { .control = KVM_GUESTDBG_ENABLE };
+	struct kvm_vcpu_events events = {};
+	struct kvm_vcpu_init init;
+	struct kvm_vcpu *vcpu;
+	u64 ipa = TEST_GPA;
+	struct kvm_vm *vm;
+	int ret;
+
+	vm = pkvm_vm_create(1, 0, &vcpu);
+
+	ret = __vcpu_ioctl(vcpu, KVM_SET_GUEST_DEBUG, &dbg);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "pre-run SET_GUEST_DEBUG: rc %d errno %d",
+		    ret, errno);
+	/* EL2 implements PSCI 1.1, so the host can't set an older version. */
+	ret = __vcpu_set_reg(vcpu, KVM_REG_ARM_PSCI_VERSION, PSCI_VERSION(1, 0));
+	TEST_ASSERT(ret == -1 && errno == EINVAL, "PSCI_VERSION 1.0: rc %d errno %d", ret, errno);
+	vcpu_set_reg(vcpu, KVM_REG_ARM_PSCI_VERSION, PSCI_VERSION(1, 1));
+	ksft_test_result_pass("pre-run: SET_GUEST_DEBUG refused, PSCI_VERSION 1.1 at least\n");
+
+	pkvm_vcpu_setup_boot(vcpu, guest_finalize);
+	pkvm_run_loop(vcpu, NULL);
+
+	ret = __vcpu_ioctl(vcpu, KVM_SET_GUEST_DEBUG, &dbg);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "post-run SET_GUEST_DEBUG: rc %d errno %d",
+		    ret, errno);
+	kvm_get_default_vcpu_target(vm, &init);
+	init.features[0] |= PVM_INIT_FEATURES;
+	ret = __vcpu_ioctl(vcpu, KVM_ARM_VCPU_INIT, &init);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "post-run VCPU_INIT: rc %d errno %d", ret, errno);
+
+	events.exception.ext_dabt_pending = 1;
+	ret = __vcpu_ioctl(vcpu, KVM_SET_VCPU_EVENTS, &events);
+	TEST_ASSERT(ret == -1 && errno == EPERM, "ext_dabt_pending: rc %d errno %d", ret, errno);
+	events.exception.ext_dabt_pending = 0;
+	events.exception.serror_pending = 1;
+	vcpu_events_set(vcpu, &events);
+	memset(&events, 0, sizeof(events));
+	vcpu_events_get(vcpu, &events);
+	TEST_ASSERT(events.exception.serror_pending, "post-run SError not read back");
+
+	ret = __vcpu_has_device_attr(vcpu, KVM_ARM_VCPU_PVTIME_CTRL, KVM_ARM_VCPU_PVTIME_IPA);
+	TEST_ASSERT(ret == -1 && errno == ENXIO, "HAS PVTIME_IPA: rc %d errno %d", ret, errno);
+	ret = __vcpu_device_attr_get(vcpu, KVM_ARM_VCPU_PVTIME_CTRL, KVM_ARM_VCPU_PVTIME_IPA, &ipa);
+	TEST_ASSERT(ret == -1 && errno == ENXIO, "GET PVTIME_IPA: rc %d errno %d", ret, errno);
+	ret = __vcpu_device_attr_set(vcpu, KVM_ARM_VCPU_PVTIME_CTRL, KVM_ARM_VCPU_PVTIME_IPA, &ipa);
+	TEST_ASSERT(ret == -1 && errno == ENXIO, "SET PVTIME_IPA: rc %d errno %d", ret, errno);
+
+	kvm_vm_free(vm);
+	ksft_test_result_pass("post-run: debug, VCPU_INIT, abort injection and PVTIME refused\n");
+}
+
+/* GET_REG_LIST is gated on VCPU_INIT and SVE finalization, not on the vCPU having run. */
+static void test_get_reg_list_gate(void)
+{
+	struct kvm_reg_list reg_list = { .n = 0 };
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int ret;
+
+	vcpu = create_uninitialized_vcpu(&vm);
+	ret = __vcpu_ioctl(vcpu, KVM_GET_REG_LIST, &reg_list);
+	TEST_ASSERT(ret == -1 && errno == ENOEXEC, "uninitialized: rc %d errno %d", ret, errno);
+	kvm_vm_free(vm);
+
+	vm = pkvm_vm_create(1, 0, &vcpu);
+	free(vcpu_get_reg_list(vcpu));
+	kvm_vm_free(vm);
+	ksft_test_result_pass("GET_REG_LIST gate\n");
+
+	if (pkvm_has_cap(KVM_CAP_ARM_SVE)) {
+		vm = pkvm_vm_create(1, BIT(KVM_ARM_VCPU_SVE), &vcpu);
+		ret = __vcpu_ioctl(vcpu, KVM_GET_REG_LIST, &reg_list);
+		TEST_ASSERT(ret == -1 && errno == EPERM,
+			    "SVE not finalized: rc %d errno %d", ret, errno);
+		kvm_vm_free(vm);
+		ksft_test_result_pass("GET_REG_LIST on an unfinalized SVE vCPU\n");
+	} else {
+		ksft_test_result_skip("GET_REG_LIST on an unfinalized SVE vCPU: %s\n",
+				      "SVE unavailable to a protected VM");
+	}
+}
+
+static void test_memslot_flags_rejected(void)
+{
+	size_t size = getpagesize();
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	void *hva;
+	int ret;
+
+	vm = pkvm_vm_create(1, 0, &vcpu);
+
+	hva = kvm_mmap(size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1);
+
+	ret = __vm_set_user_memory_region2(vm, TEST_MEMSLOT, KVM_MEM_READONLY, TEST_GPA,
+					   size, hva, -1, 0);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "read-only memslot on a protected VM: rc %d errno %d", ret, errno);
+	ksft_test_result_pass("read-only memslot rejected at creation\n");
+
+	ret = __vm_set_user_memory_region2(vm, TEST_MEMSLOT, KVM_MEM_LOG_DIRTY_PAGES, TEST_GPA,
+					   size, hva, -1, 0);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "dirty-logging memslot on a protected VM: rc %d errno %d", ret, errno);
+
+	vm_set_user_memory_region2(vm, TEST_MEMSLOT, 0, TEST_GPA, size, hva, -1, 0);
+	ret = __vm_set_user_memory_region2(vm, TEST_MEMSLOT, KVM_MEM_LOG_DIRTY_PAGES, TEST_GPA,
+					   size, hva, -1, 0);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "dirty logging enabled on a protected VM: rc %d errno %d", ret, errno);
+	ksft_test_result_pass("dirty logging rejected at creation and when enabled\n");
+
+	kvm_munmap(hva, size);
+	kvm_vm_free(vm);
+}
+
+/* The host pins and accounts a protected VM's memory until the VM is freed. */
+static void test_memory_locked(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	size_t locked;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_touch_memory);
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, TEST_GPA, TEST_MEMSLOT,
+				    TEST_PAGES, 0);
+	virt_map(vm, TEST_GPA, TEST_GPA, TEST_PAGES);
+	write_guest_global(vm, test_size, TEST_PAGES * vm->page_size);
+
+	pkvm_run_loop(vcpu, NULL);
+	locked = pkvm_host_locked_vm();
+	TEST_ASSERT(locked > TEST_PAGES * vm->page_size,
+		    "guest memory not locked while the VM is alive: %zu bytes", locked);
+
+	kvm_vm_free(vm);
+	TEST_ASSERT_EQ(pkvm_host_locked_vm(), 0);
+
+	ksft_test_result_pass("memory locked while running\n");
+}
+
+int main(void)
+{
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(ARRAY_SIZE(allowed_caps) + 21);
+
+	assert_feature_rejected(BIT(KVM_ARM_VCPU_EL1_32BIT), "EL1_32BIT");
+	test_vcpu_init_conditional_features();
+	test_vcpu_init_repeated();
+	test_caps();
+	test_vm_ioctls_rejected();
+	test_one_reg_gate();
+	test_vcpu_ioctl_gates();
+	test_get_reg_list_gate();
+	test_memslot_flags_rejected();
+	test_memory_locked();
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 09/19] KVM: arm64: selftests: Add pkvm_psci multi-vCPU PSCI test

[Fuad Tabba]

EL2 handles a protected guest's CPU_ON, CPU_OFF, PSCI_VERSION and
PSCI_FEATURES, and forwards the rest to the host. So only the guest
can power on its vCPUs.

Test each PSCI call, and that the host can't make a secondary runnable
without a CPU_ON from the guest. Run the secondaries on their own
threads to race CPU_ON against CPU_OFF and against another CPU_ON. A
CPU_ON of a vCPU that has called CPU_OFF returns ALREADY_ON until
the host powers it off.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 tools/testing/selftests/kvm/arm64/pkvm_psci.c | 632 ++++++++++++++++++
 2 files changed, 633 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_psci.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 42e530e0e573b..aa256fb66d226 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -191,6 +191,7 @@ TEST_GEN_PROGS_arm64 += arm64/external_aborts
 TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
+TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_psci.c b/tools/testing/selftests/kvm/arm64/pkvm_psci.c
new file mode 100644
index 0000000000000..07a50af9c9ad9
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_psci.c
@@ -0,0 +1,632 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_psci - PSCI in a protected VM, including CPU_ON racing CPU_OFF and CPU_ON.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <pthread.h>
+
+#include <linux/bitfield.h>
+#include <linux/psci.h>
+#include <asm/barrier.h>
+#include <asm/cputype.h>
+
+#include "arch_timer.h"
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define MAX_VCPUS	3
+
+/* A well-formed MPIDR with no matching vCPU. */
+#define BOGUS_MPIDR	0xf
+
+/* The primary's GUEST_SYNC: the host runs the secondary, then resumes the primary. */
+#define SYNC_RUN_SECONDARY	1
+
+#define STRESS_BOOTS	50
+#define ALT_BOOTS	50
+#define HANDSHAKE_CYCLES	50
+#define RACE_ROUNDS	10
+
+#define WAIT_MS		30000
+
+/* An identity-mapped page for the boot context the primary writes. */
+#define ALT_MEMSLOT	(PKVM_IDMAP_MEMSLOT + 1)
+#define ALT_CTX		BIT_ULL(30)
+
+/* Set in the guest's copy by the host before the first run. */
+static u64 boot_entry;
+static u64 vcpu_mpidr[MAX_VCPUS];
+static u64 vcpu_ctx[MAX_VCPUS];
+
+static u32 boots;
+static u64 expected_ctx;
+static u64 booted_ctx;
+static u32 go_off;
+
+static u32 race_ready;
+static u32 race_go;
+static u32 race_done;
+static unsigned long race_ret[2];
+
+#define wait_until(cond)							\
+do {										\
+	u64 __end = timer_get_cntct(VIRTUAL) + msec_to_cycles(WAIT_MS);	\
+										\
+	while (!(cond))								\
+		__GUEST_ASSERT(timer_get_cntct(VIRTUAL) < __end,		\
+			       "timed out waiting for %s", #cond);		\
+} while (0)
+
+static unsigned long __psci_cpu_on(u64 mpidr, u64 entry, u64 ctx)
+{
+	return pkvm_hvc(PSCI_0_2_FN64_CPU_ON, mpidr, entry, ctx);
+}
+
+static unsigned long psci_cpu_on(int id)
+{
+	return __psci_cpu_on(vcpu_mpidr[id], boot_entry, vcpu_ctx[id]);
+}
+
+static unsigned long psci_affinity_info(int id)
+{
+	return pkvm_hvc(PSCI_0_2_FN64_AFFINITY_INFO, vcpu_mpidr[id], 0, 0);
+}
+
+static void psci_cpu_off(void)
+{
+	pkvm_hvc(PSCI_0_2_FN_CPU_OFF, 0, 0, 0);
+	GUEST_ASSERT(0);
+}
+
+static void guest_primary(void)
+{
+	pkvm_guest_init();
+
+	GUEST_ASSERT_EQ(pkvm_hvc(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0), PSCI_VERSION(1, 1));
+	GUEST_ASSERT_EQ(pkvm_hvc(PSCI_1_0_FN_PSCI_FEATURES, PSCI_0_2_FN64_CPU_ON, 0, 0),
+			PSCI_RET_SUCCESS);
+	GUEST_ASSERT_EQ(pkvm_hvc(PSCI_1_0_FN_PSCI_FEATURES, PSCI_0_2_FN_MIGRATE, 0, 0),
+			PSCI_RET_NOT_SUPPORTED);
+
+	GUEST_ASSERT_EQ(psci_affinity_info(1), PSCI_0_2_AFFINITY_LEVEL_OFF);
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_SUCCESS);
+
+	/* The host handles AFFINITY_INFO, and returns ON for a CPU_ON target. */
+	GUEST_ASSERT_EQ(psci_affinity_info(1), PSCI_0_2_AFFINITY_LEVEL_ON);
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_ON_PENDING);
+
+	GUEST_SYNC(SYNC_RUN_SECONDARY);
+
+	GUEST_ASSERT_EQ(psci_affinity_info(1), PSCI_0_2_AFFINITY_LEVEL_ON);
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_ALREADY_ON);
+
+	GUEST_ASSERT_EQ(__psci_cpu_on(BOGUS_MPIDR, boot_entry, 0), PSCI_RET_INVALID_PARAMS);
+
+	GUEST_DONE();
+}
+
+/* The primary's pkvm_guest_init() set up ucall for the whole VM. */
+static void guest_secondary(void)
+{
+	GUEST_DONE();
+}
+
+static void guest_cpu_on_unregistered(void)
+{
+	pkvm_guest_init();
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_INVALID_PARAMS);
+	GUEST_DONE();
+}
+
+static void guest_boot_and_done(void)
+{
+	pkvm_guest_init();
+	GUEST_DONE();
+}
+
+static void guest_system_off(void)
+{
+	pkvm_hvc(PSCI_0_2_FN_SYSTEM_OFF, 0, 0, 0);
+	GUEST_ASSERT(0);
+}
+
+static void guest_system_reset(void)
+{
+	pkvm_hvc(PSCI_0_2_FN_SYSTEM_RESET, 0, 0, 0);
+	GUEST_ASSERT(0);
+}
+
+static void guest_system_reset2_smc32(void)
+{
+	pkvm_hvc(PSCI_1_1_FN_SYSTEM_RESET2, PSCI_1_1_RESET_TYPE_SYSTEM_WARM_RESET, 0, 0);
+	GUEST_ASSERT(0);
+}
+
+static void guest_system_reset2_smc64(void)
+{
+	pkvm_hvc(PSCI_1_1_FN64_SYSTEM_RESET2, PSCI_1_1_RESET_TYPE_SYSTEM_WARM_RESET, 0, 0);
+	GUEST_ASSERT(0);
+}
+
+static void guest_stress_primary(void)
+{
+	unsigned long successes = 0;
+
+	pkvm_guest_init();
+	while (READ_ONCE(boots) < STRESS_BOOTS) {
+		unsigned long ret = psci_cpu_on(1);
+
+		__GUEST_ASSERT(ret == PSCI_RET_SUCCESS || ret == PSCI_RET_ALREADY_ON ||
+			       ret == PSCI_RET_ON_PENDING, "CPU_ON 0x%lx", ret);
+		successes += ret == PSCI_RET_SUCCESS;
+	}
+
+	/*
+	 * SUCCESS means the secondary ran. The last boot has completed and the
+	 * secondary stays on after it, so no CPU_ON and no boot is in flight.
+	 */
+	GUEST_ASSERT_EQ(successes, STRESS_BOOTS);
+	GUEST_DONE();
+}
+
+static void guest_stress_secondary(void)
+{
+	u32 k = READ_ONCE(boots) + 1;
+
+	WRITE_ONCE(boots, k);
+	if (k == STRESS_BOOTS)
+		GUEST_DONE();
+	psci_cpu_off();
+}
+
+static void alt_boot(u64 ctx)
+{
+	u32 k = READ_ONCE(boots) + 1;
+
+	GUEST_ASSERT_EQ(ctx, READ_ONCE(expected_ctx));
+	WRITE_ONCE(booted_ctx, ctx);
+	/* Publishes booted_ctx, and the primary changes expected_ctx only once it sees this. */
+	smp_store_release(&boots, k);
+	if (k == ALT_BOOTS)
+		GUEST_DONE();
+	psci_cpu_off();
+}
+
+static void guest_alt_secondary_host_ctx(void)
+{
+	alt_boot(vcpu_ctx[1]);
+}
+
+static void guest_alt_secondary_guest_ctx(void)
+{
+	alt_boot(ALT_CTX);
+}
+
+static void clean_to_poc(void *p, size_t size)
+{
+	u64 line = 4UL << SYS_FIELD_GET(CTR_EL0, DminLine, read_sysreg(ctr_el0));
+	u64 addr;
+
+	for (addr = (u64)p & ~(line - 1); addr < (u64)p + size; addr += line)
+		asm volatile("dc cvac, %0" : : "r"(addr) : "memory");
+	dsb(sy);
+}
+
+static void guest_alt_primary(void)
+{
+	struct pvm_boot_args *alt = (struct pvm_boot_args *)ALT_CTX;
+	bool guest_ctx = true;
+	unsigned long ret;
+	u64 ctx;
+	u32 k;
+
+	pkvm_guest_init();
+
+	/* The trampoline reads the context with the MMU off. */
+	*alt = *(struct pvm_boot_args *)vcpu_ctx[1];
+	alt->jump_tgt = (u64)guest_alt_secondary_guest_ctx;
+	clean_to_poc(alt, sizeof(*alt));
+
+	for (k = 1; k <= ALT_BOOTS; k++) {
+		/*
+		 * CPU_ON returns ALREADY_ON until the host has powered the secondary off.
+		 * Each attempt passes the other context, so a boot from the previous attempt's
+		 * context fails alt_boot()'s check.
+		 */
+		do {
+			guest_ctx = !guest_ctx;
+			ctx = guest_ctx ? ALT_CTX : vcpu_ctx[1];
+			WRITE_ONCE(expected_ctx, ctx);
+			ret = __psci_cpu_on(vcpu_mpidr[1], boot_entry, ctx);
+			__GUEST_ASSERT(ret == PSCI_RET_SUCCESS || ret == PSCI_RET_ALREADY_ON,
+				       "CPU_ON 0x%lx", ret);
+		} while (ret != PSCI_RET_SUCCESS);
+
+		/* Only the primary powers the secondary on, so no other boot is in flight. */
+		wait_until(smp_load_acquire(&boots) >= k);
+		GUEST_ASSERT_EQ(READ_ONCE(boots), k);
+		GUEST_ASSERT_EQ(READ_ONCE(booted_ctx), ctx);
+	}
+
+	GUEST_DONE();
+}
+
+static void guest_handshake_secondary(void)
+{
+	u32 k = READ_ONCE(boots) + 1;
+
+	WRITE_ONCE(boots, k);
+	if (k > HANDSHAKE_CYCLES)
+		GUEST_DONE();
+	wait_until(READ_ONCE(go_off) == k);
+	psci_cpu_off();
+}
+
+/* The secondary is ON until it has called CPU_OFF. */
+static bool is_off(int id)
+{
+	unsigned long ret = psci_affinity_info(id);
+
+	__GUEST_ASSERT(ret == PSCI_0_2_AFFINITY_LEVEL_ON || ret == PSCI_0_2_AFFINITY_LEVEL_OFF,
+		       "AFFINITY_INFO 0x%lx", ret);
+	return ret == PSCI_0_2_AFFINITY_LEVEL_OFF;
+}
+
+static void guest_handshake_primary(void)
+{
+	u32 k;
+
+	pkvm_guest_init();
+
+	for (k = 1; k <= HANDSHAKE_CYCLES; k++) {
+		GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_SUCCESS);
+
+		/* The secondary has booted and waits for go_off, so it is still ON. */
+		wait_until(READ_ONCE(boots) >= k);
+		GUEST_ASSERT_EQ(psci_affinity_info(1), PSCI_0_2_AFFINITY_LEVEL_ON);
+
+		WRITE_ONCE(go_off, k);
+
+		/* It is OFF and only the primary powers it on, so no boot is in flight. */
+		wait_until(is_off(1));
+		GUEST_ASSERT_EQ(READ_ONCE(boots), k);
+	}
+
+	/* One more boot lets the secondary finish instead of powering off. */
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_SUCCESS);
+	GUEST_DONE();
+}
+
+static void guest_race_caller(void)
+{
+	WRITE_ONCE(race_ready, 1);
+	wait_until(READ_ONCE(race_go));
+	race_ret[1] = psci_cpu_on(2);
+	/* Publishes race_ret[1] to the primary. */
+	smp_store_release(&race_done, 1);
+	GUEST_DONE();
+}
+
+/* No CPU_OFF: a target that powered off would let the loser win too. */
+static void guest_race_target(void)
+{
+	WRITE_ONCE(boots, READ_ONCE(boots) + 1);
+	GUEST_DONE();
+}
+
+static void guest_race_primary(void)
+{
+	unsigned long loser;
+	int winner;
+
+	pkvm_guest_init();
+
+	/* vCPU 1 has never been on, and the primary is its only caller. */
+	GUEST_ASSERT_EQ(psci_cpu_on(1), PSCI_RET_SUCCESS);
+	wait_until(READ_ONCE(race_ready));
+
+	WRITE_ONCE(race_go, 1);
+	race_ret[0] = psci_cpu_on(2);
+
+	/* Both callers have returned, and vCPU 2 has booted and stays on. */
+	wait_until(smp_load_acquire(&race_done));
+	wait_until(READ_ONCE(boots));
+	GUEST_ASSERT_EQ(READ_ONCE(boots), 1);
+
+	winner = race_ret[0] == PSCI_RET_SUCCESS ? 0 : 1;
+	loser = race_ret[!winner];
+	GUEST_ASSERT_EQ(race_ret[winner], PSCI_RET_SUCCESS);
+	__GUEST_ASSERT(loser == PSCI_RET_ON_PENDING || loser == PSCI_RET_ALREADY_ON,
+		       "losing CPU_ON 0x%lx", loser);
+
+	GUEST_SYNC(loser);
+	GUEST_DONE();
+}
+
+static struct kvm_vcpu *secondary_vcpu;
+static u64 sync_arg;
+
+static void run_secondary_on_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	if (uc->args[1] == SYNC_RUN_SECONDARY)
+		pkvm_run_loop(secondary_vcpu, NULL);
+}
+
+static void wire_target(struct kvm_vm *vm, struct kvm_vcpu *vcpu)
+{
+	u64 mpidr = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MPIDR_EL1));
+
+	write_guest_global(vm, vcpu_mpidr[vcpu->id], mpidr & MPIDR_HWID_BITMASK);
+	write_guest_global(vm, boot_entry, pkvm_idmap_gpa(vm));
+}
+
+static void wire_secondary(struct kvm_vm *vm, struct kvm_vcpu *vcpu, void *guest_code)
+{
+	gpa_t ctx;
+
+	wire_target(vm, vcpu);
+	ctx = pkvm_vcpu_setup_boot_secondary(vcpu, guest_code);
+	write_guest_global(vm, vcpu_ctx[vcpu->id], ctx);
+}
+
+static void test_protocol(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+	secondary_vcpu = vcpus[1];
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_primary);
+	wire_secondary(vm, vcpus[1], guest_secondary);
+
+	pkvm_run_loop(vcpus[0], run_secondary_on_sync);
+
+	kvm_vm_free(vm);
+}
+
+/* vCPU 1 exists but never runs, so EL2 never creates its hyp vCPU. */
+static void test_cpu_on_unregistered(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_cpu_on_unregistered);
+	wire_target(vm, vcpus[1]);
+
+	pkvm_run_loop(vcpus[0], NULL);
+
+	kvm_vm_free(vm);
+}
+
+static void assert_system_event(void *guest_code, u32 type, u64 flags)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_run *run;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(1, 0, &vcpu);
+	pkvm_vcpu_setup_boot(vcpu, guest_code);
+
+	run = vcpu->run;
+	vcpu_run(vcpu);
+
+	TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_SYSTEM_EVENT);
+	TEST_ASSERT_EQ(run->system_event.type, type);
+	TEST_ASSERT_EQ(run->system_event.data[0], flags);
+
+	kvm_vm_free(vm);
+}
+
+/* Only one vCPU may be RUNNABLE when EL2 creates its hyp vCPU: the primary. */
+static void assert_second_runnable_rejected(struct kvm_vcpu *vcpu)
+{
+	int ret = _vcpu_run(vcpu);
+
+	TEST_ASSERT(ret == -1 && errno == EINVAL,
+		    "second RUNNABLE vCPU: expected -1/EINVAL, got %d/%d", ret, errno);
+}
+
+static void test_forged_runnable(void)
+{
+	struct kvm_mp_state mps = { .mp_state = KVM_MP_STATE_RUNNABLE };
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+	pkvm_vcpu_setup_boot(vcpus[0], guest_boot_and_done);
+	pkvm_run_loop(vcpus[0], NULL);
+
+	vcpu_mp_state_set(vcpus[1], &mps);
+	assert_second_runnable_rejected(vcpus[1]);
+
+	kvm_vm_free(vm);
+}
+
+static void test_second_primary(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vcpu_init init;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+	pkvm_vcpu_setup_boot(vcpus[0], guest_boot_and_done);
+	pkvm_run_loop(vcpus[0], NULL);
+
+	/* Re-INIT without POWER_OFF also leaves the vCPU RUNNABLE. */
+	kvm_get_default_vcpu_target(vm, &init);
+	init.features[0] |= BIT(KVM_ARM_VCPU_PSCI_0_2);
+	vcpu_ioctl(vcpus[1], KVM_ARM_VCPU_INIT, &init);
+	assert_second_runnable_rejected(vcpus[1]);
+
+	kvm_vm_free(vm);
+}
+
+/* Once EL2 holds a secondary's hyp vCPU off, the host rejects powering it on. */
+static void test_run_gate_off(void)
+{
+	struct kvm_mp_state mps = { .mp_state = KVM_MP_STATE_RUNNABLE };
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+	int ret;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+
+	pkvm_vcpu_setup_boot_secondary(vcpus[1], guest_boot_and_done);
+
+	ret = __vcpu_ioctl(vcpus[1], KVM_SET_MP_STATE, &mps);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "forged RUNNABLE: expected -1/EPERM, got %d/%d", ret, errno);
+	mps.mp_state = KVM_MP_STATE_SUSPENDED;
+	ret = __vcpu_ioctl(vcpus[1], KVM_SET_MP_STATE, &mps);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "forged SUSPENDED: expected -1/EPERM, got %d/%d", ret, errno);
+
+	kvm_vm_free(vm);
+}
+
+static void *vcpu_thread(void *vcpu)
+{
+	pkvm_run_loop(vcpu, NULL);
+	return NULL;
+}
+
+static void record_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	sync_arg = uc->args[1];
+}
+
+/* The secondary runs on its own thread, so a CPU_ON can race its CPU_OFF. */
+static void run_threaded(struct kvm_vcpu *vcpus[])
+{
+	pthread_t thread;
+
+	pthread_create(&thread, NULL, vcpu_thread, vcpus[1]);
+	pkvm_run_loop(vcpus[0], record_sync);
+	pthread_join(thread, NULL);
+}
+
+static void test_stress(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_stress_primary);
+	wire_secondary(vm, vcpus[1], guest_stress_secondary);
+
+	run_threaded(vcpus);
+
+	kvm_vm_free(vm);
+}
+
+static void test_alternating_ctx(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, ALT_CTX, ALT_MEMSLOT,
+				    vm_adjust_num_guest_pages(vm->mode, 1), 0);
+	virt_map(vm, ALT_CTX, ALT_CTX, 1);
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_alt_primary);
+	wire_secondary(vm, vcpus[1], guest_alt_secondary_host_ctx);
+
+	run_threaded(vcpus);
+
+	kvm_vm_free(vm);
+}
+
+static void test_handshake(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_handshake_primary);
+	wire_secondary(vm, vcpus[1], guest_handshake_secondary);
+
+	run_threaded(vcpus);
+
+	kvm_vm_free(vm);
+}
+
+/* Returns the number of rounds whose loser got ON_PENDING: its CPU_ON overlapped the winner's. */
+static int test_concurrent_cpu_on(void)
+{
+	int i, on_pending = 0;
+
+	for (i = 0; i < RACE_ROUNDS; i++) {
+		struct kvm_vcpu *vcpus[3];
+		pthread_t target;
+		struct kvm_vm *vm;
+
+		vm = pkvm_vm_create(3, 0, vcpus);
+
+		pkvm_vcpu_setup_boot(vcpus[0], guest_race_primary);
+		wire_secondary(vm, vcpus[1], guest_race_caller);
+		wire_secondary(vm, vcpus[2], guest_race_target);
+
+		pthread_create(&target, NULL, vcpu_thread, vcpus[2]);
+		run_threaded(vcpus);
+		pthread_join(target, NULL);
+
+		kvm_vm_free(vm);
+
+		on_pending += sync_arg == PSCI_RET_ON_PENDING;
+	}
+
+	return on_pending;
+}
+
+int main(void)
+{
+	int on_pending;
+
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(13);
+
+	test_protocol();
+	ksft_test_result_pass("PSCI protocol and CPU_ON boot\n");
+	test_cpu_on_unregistered();
+	ksft_test_result_pass("CPU_ON of a vCPU with no hyp vCPU\n");
+	assert_system_event(guest_system_off, KVM_SYSTEM_EVENT_SHUTDOWN, 0);
+	ksft_test_result_pass("SYSTEM_OFF\n");
+	assert_system_event(guest_system_reset, KVM_SYSTEM_EVENT_RESET, 0);
+	ksft_test_result_pass("SYSTEM_RESET\n");
+	assert_system_event(guest_system_reset2_smc32, KVM_SYSTEM_EVENT_RESET,
+			    KVM_SYSTEM_EVENT_RESET_FLAG_PSCI_RESET2);
+	ksft_test_result_pass("SYSTEM_RESET2 (SMC32)\n");
+	assert_system_event(guest_system_reset2_smc64, KVM_SYSTEM_EVENT_RESET,
+			    KVM_SYSTEM_EVENT_RESET_FLAG_PSCI_RESET2);
+	ksft_test_result_pass("SYSTEM_RESET2 (SMC64)\n");
+	test_forged_runnable();
+	ksft_test_result_pass("forged RUNNABLE rejected at hyp vCPU creation\n");
+	test_second_primary();
+	ksft_test_result_pass("second RUNNABLE vCPU rejected\n");
+	test_run_gate_off();
+	ksft_test_result_pass("forged RUNNABLE and SUSPENDED of an OFF vCPU rejected\n");
+	test_stress();
+	ksft_test_result_pass("CPU_ON/CPU_OFF race: %d boots\n", STRESS_BOOTS);
+	test_alternating_ctx();
+	ksft_test_result_pass("CPU_ON alternating context: %d boots\n", ALT_BOOTS);
+	test_handshake();
+	ksft_test_result_pass("CPU_OFF/CPU_ON handshake: %d cycles\n", HANDSHAKE_CYCLES);
+	on_pending = test_concurrent_cpu_on();
+	ksft_test_result_pass("concurrent CPU_ON: one winner in %d rounds, %d ON_PENDING losers\n",
+			      RACE_ROUNDS, on_pending);
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 10/19] KVM: arm64: selftests: Add pkvm_mem_share hypercall test

[Fuad Tabba]

A protected guest gives the host access to one of its pages with
MEM_SHARE and takes it back with MEM_UNSHARE. The page is a host page,
whose size HYP_MEMINFO returns. At teardown EL2 zeroes every page the
guest hasn't left shared.

Exercise the three calls, the host's access to shared and private pages,
and the zeroing. Also check that KVM_RUN fails with EIO on
hugetlb-backed memory, that sharing an IPA outside every memslot makes
the guest take an external abort, and that faults each requiring new
stage-2 tables succeed.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_mem_share.c      | 394 ++++++++++++++++++
 2 files changed, 395 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_mem_share.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index aa256fb66d226..88c1159cb3bf1 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -191,6 +191,7 @@ TEST_GEN_PROGS_arm64 += arm64/external_aborts
 TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
+TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_mem_share.c b/tools/testing/selftests/kvm/arm64/pkvm_mem_share.c
new file mode 100644
index 0000000000000..9a4e96fc8d046
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_mem_share.c
@@ -0,0 +1,394 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_mem_share - MEM_SHARE, MEM_UNSHARE and HYP_MEMINFO, and EL2 zeroing a
+ * protected VM's private pages at teardown.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <errno.h>
+#include <setjmp.h>
+#include <signal.h>
+#include <stdio.h>
+
+#include <linux/arm-smccc.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/*
+ * A page here is a host page, the granule EL2 shares in. Set in the guest's
+ * copy by the host before the first run.
+ */
+static u64 granule;
+
+#define GUEST_MEMSLOT	(PKVM_IDMAP_MEMSLOT + 1)
+#define GPA_BASE	BIT_ULL(30)
+#define GVA_BASE	GPA_BASE
+#define GPA_PAGES	16
+#define GPA_SIZE	((size_t)GPA_PAGES * granule)
+
+/* Each case uses its own pages of the guest memslot. */
+#define P_SHARE			0
+#define P_SHARED_AT_TEARDOWN	1
+
+/* Recognizable value the host must still see in a page that isn't zeroed. */
+#define DIRTY_VAL		0x0badf00ddeadbeefUL
+
+/* An aligned IPA outside every memslot. */
+#define OUTSIDE_IPA		BIT_ULL(31)
+
+/* What one last-level stage-2 table maps: 2M on a 4K host. */
+#define BLOCK_SIZE		(granule / 8 * granule)
+
+/* One page in each block of a memslot centered on 512G. */
+#define PRESSURE_MEMSLOT	(GUEST_MEMSLOT + 1)
+#define PRESSURE_PAGES		64
+#define PRESSURE_SIZE		((u64)PRESSURE_PAGES * BLOCK_SIZE)
+#define PRESSURE_GPA		(BIT_ULL(39) - PRESSURE_SIZE / 2)
+#define PRESSURE_GVA		BIT_ULL(32)
+
+/* Commands that act on a page take its index. */
+enum guest_cmd {
+	CMD_HOST_INC_SHARED = 1,
+	CMD_HOST_ACCESS_PRIVATE,
+};
+
+#define HOST_CMD(cmd, idx)	GUEST_SYNC_ARGS(cmd, idx, 0, 0, 0)
+
+static u64 page_ipa(int idx)
+{
+	return GPA_BASE + (u64)idx * granule;
+}
+
+static volatile u64 *page_va(int idx)
+{
+	return (volatile u64 *)(GVA_BASE + (u64)idx * granule);
+}
+
+static unsigned long hvc_share(u64 ipa, u64 arg2, u64 arg3)
+{
+	return pkvm_hvc(ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID, ipa, arg2, arg3);
+}
+
+static unsigned long hvc_unshare(u64 ipa, u64 arg2, u64 arg3)
+{
+	return pkvm_hvc(ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID, ipa, arg2, arg3);
+}
+
+static void test_meminfo(void)
+{
+	u32 meminfo = ARM_SMCCC_VENDOR_HYP_KVM_HYP_MEMINFO_FUNC_ID;
+
+	GUEST_ASSERT_EQ(pkvm_hvc(meminfo, 0, 0, 0), granule);
+
+	GUEST_ASSERT_EQ(pkvm_hvc(meminfo, 1, 0, 0), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(pkvm_hvc(meminfo, 0, 1, 0), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(pkvm_hvc(meminfo, 0, 0, 1), SMCCC_RET_INVALID_PARAMETER);
+}
+
+static void test_arg_validation(void)
+{
+	u64 ipa = page_ipa(P_SHARE);
+
+	/* Non-zero arg2 or arg3, and an unaligned IPA, are rejected before any page walk. */
+	GUEST_ASSERT_EQ(hvc_share(ipa, 1, 0), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(hvc_share(ipa, 0, 1), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(hvc_share(ipa + 0x80, 0, 0), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(hvc_unshare(ipa, 1, 0), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(hvc_unshare(ipa, 0, 1), SMCCC_RET_INVALID_PARAMETER);
+	GUEST_ASSERT_EQ(hvc_unshare(ipa + 0x80, 0, 0), SMCCC_RET_INVALID_PARAMETER);
+}
+
+static void test_share_unshare(void)
+{
+	volatile u64 *val = page_va(P_SHARE);
+	u64 ipa = page_ipa(P_SHARE);
+
+	*val = 42;
+
+	HOST_CMD(CMD_HOST_ACCESS_PRIVATE, P_SHARE);
+	GUEST_ASSERT_EQ(*val, 42);
+
+	GUEST_ASSERT_EQ(hvc_share(ipa, 0, 0), SMCCC_RET_SUCCESS);
+	GUEST_ASSERT_EQ(hvc_share(ipa, 0, 0), SMCCC_RET_INVALID_PARAMETER);
+
+	HOST_CMD(CMD_HOST_INC_SHARED, P_SHARE);
+	GUEST_ASSERT_EQ(*val, 43);
+
+	GUEST_ASSERT_EQ(hvc_unshare(ipa, 0, 0), SMCCC_RET_SUCCESS);
+	GUEST_ASSERT_EQ(hvc_unshare(ipa, 0, 0), SMCCC_RET_INVALID_PARAMETER);
+
+	HOST_CMD(CMD_HOST_ACCESS_PRIVATE, P_SHARE);
+	GUEST_ASSERT_EQ(*val, 43);
+}
+
+/* Dirty every page, so that a page that reads 0 after teardown is one EL2 zeroed. */
+static void guest_dirty_region(void)
+{
+	volatile u64 *p = (volatile u64 *)GVA_BASE;
+	size_t i;
+
+	for (i = 0; i < GPA_SIZE / sizeof(*p); i++)
+		p[i] = DIRTY_VAL;
+}
+
+static void guest_code(void)
+{
+	pkvm_guest_init();
+
+	test_meminfo();
+	test_arg_validation();
+	test_share_unshare();
+
+	guest_dirty_region();
+
+	/* Teardown doesn't zero a page the guest left shared. */
+	GUEST_ASSERT_EQ(hvc_share(page_ipa(P_SHARED_AT_TEARDOWN), 0, 0), SMCCC_RET_SUCCESS);
+
+	GUEST_DONE();
+}
+
+static void expect_sea_handler(struct ex_regs *regs)
+{
+	u64 esr = read_sysreg(esr_el1);
+
+	GUEST_ASSERT_EQ(ESR_ELx_EC(esr), ESR_ELx_EC_DABT_CUR);
+	GUEST_ASSERT_EQ(esr & ESR_ELx_FSC_TYPE, ESR_ELx_FSC_EXTABT);
+
+	GUEST_DONE();
+}
+
+static void guest_share_outside(void)
+{
+	pkvm_guest_init();
+
+	hvc_share(OUTSIDE_IPA, 0, 0);
+	GUEST_FAIL("MEM_SHARE of an IPA outside every memslot returned to the guest");
+}
+
+/*
+ * Each fault requires a new stage-2 last-level table, and the first one above
+ * 512G a new table at every level below the root.
+ */
+static void guest_memcache_pressure(void)
+{
+	volatile u64 *p = (volatile u64 *)PRESSURE_GVA;
+	int i;
+
+	pkvm_guest_init();
+
+	for (i = 0; i < PRESSURE_PAGES; i++)
+		p[i * granule / sizeof(*p)] = i + 1;
+	for (i = 0; i < PRESSURE_PAGES; i++)
+		GUEST_ASSERT_EQ(p[i * granule / sizeof(*p)], i + 1);
+
+	GUEST_DONE();
+}
+
+static void guest_touch_one_page(void)
+{
+	pkvm_guest_init();
+
+	*page_va(0) = 1;
+
+	GUEST_DONE();
+}
+
+static sigjmp_buf sigsegv_jmpbuf;
+static volatile void *sigsegv_expected;
+
+/* The test maps the guest memslot itself, so it can read the memory after the VM is freed. */
+static u64 *guest_mem;
+
+static u64 *host_page(int idx)
+{
+	return guest_mem + (size_t)idx * granule / sizeof(*guest_mem);
+}
+
+static void sigsegv_handler(int sig, siginfo_t *si, void *ctx)
+{
+	TEST_ASSERT(si->si_addr == sigsegv_expected,
+		    "SIGSEGV at %p, expected %p", si->si_addr, sigsegv_expected);
+	siglongjmp(sigsegv_jmpbuf, 1);
+}
+
+static void host_access_private(int idx)
+{
+	struct sigaction old, sa = {
+		.sa_sigaction	= sigsegv_handler,
+		.sa_flags	= SA_SIGINFO,
+	};
+	volatile u64 *p = host_page(idx);
+	bool faulted = false;
+
+	sigsegv_expected = (void *)p;
+	sigaction(SIGSEGV, &sa, &old);
+	if (sigsetjmp(sigsegv_jmpbuf, 1) == 0)
+		(*p)++;
+	else
+		faulted = true;
+	sigaction(SIGSEGV, &old, NULL);
+
+	TEST_ASSERT(faulted, "Host accessed private guest memory without faulting");
+}
+
+static void on_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	u64 *p = host_page(uc->args[2]);
+
+	switch (uc->args[1]) {
+	case CMD_HOST_INC_SHARED:
+		(*p)++;
+		break;
+	case CMD_HOST_ACCESS_PRIVATE:
+		host_access_private(uc->args[2]);
+		break;
+	default:
+		TEST_FAIL("Unexpected guest command %lu", uc->args[1]);
+	}
+}
+
+static void assert_page(int idx, u64 val)
+{
+	u64 *p = host_page(idx);
+	size_t i;
+
+	for (i = 0; i < granule / sizeof(*p); i++)
+		TEST_ASSERT(p[i] == val, "Guest page %d holds 0x%lx, not 0x%lx, after teardown",
+			    idx, p[i], val);
+}
+
+static void test_run(void)
+{
+	const char *name = "share/unshare, and teardown zeroing";
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int i;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_code);
+	guest_mem = kvm_mmap(GPA_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1);
+	vm_set_user_memory_region2(vm, GUEST_MEMSLOT, 0, GPA_BASE, GPA_SIZE, guest_mem, -1, 0);
+	virt_map(vm, GVA_BASE, GPA_BASE, GPA_SIZE / vm->page_size);
+	write_guest_global(vm, granule, granule);
+
+	pkvm_run_loop(vcpu, on_sync);
+	kvm_vm_free(vm);
+
+	for (i = 0; i < GPA_PAGES; i++)
+		assert_page(i, i == P_SHARED_AT_TEARDOWN ? DIRTY_VAL : 0);
+	kvm_munmap(guest_mem, GPA_SIZE);
+
+	ksft_test_result_pass("%s\n", name);
+}
+
+/*
+ * EL2 turns the share into a stage-2 translation fault on the IPA with ISV clear.
+ * The host finds no memslot for the IPA and injects an external abort.
+ */
+static void test_share_outside_memslot(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_share_outside);
+	vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, ESR_ELx_EC_DABT_CUR, expect_sea_handler);
+
+	pkvm_run_loop(vcpu, NULL);
+	kvm_vm_free(vm);
+
+	ksft_test_result_pass("share of an IPA outside every memslot takes an external abort\n");
+}
+
+static void test_memcache_pressure(void)
+{
+	const char *name = "faults that each require new stage-2 tables";
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	void *mem;
+	int i;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_memcache_pressure);
+
+	/* 32G on a 64K host, of which the guest touches one page per block. */
+	mem = kvm_mmap(PRESSURE_SIZE, PROT_READ | PROT_WRITE,
+		       MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1);
+	vm_set_user_memory_region2(vm, PRESSURE_MEMSLOT, 0, PRESSURE_GPA, PRESSURE_SIZE,
+				   mem, -1, 0);
+	for (i = 0; i < PRESSURE_PAGES; i++)
+		virt_pg_map(vm, PRESSURE_GVA + i * granule, PRESSURE_GPA + i * BLOCK_SIZE);
+	write_guest_global(vm, granule, granule);
+
+	pkvm_run_loop(vcpu, NULL);
+	kvm_vm_free(vm);
+	kvm_munmap(mem, PRESSURE_SIZE);
+
+	ksft_test_result_pass("%s\n", name);
+}
+
+static unsigned long hugetlb_free_pages(void)
+{
+	unsigned long nr = 0;
+	char line[64];
+	FILE *f;
+
+	f = fopen("/proc/meminfo", "r");
+	TEST_ASSERT(f, "fopen /proc/meminfo");
+
+	while (fgets(line, sizeof(line), f)) {
+		if (sscanf(line, "HugePages_Free: %lu", &nr) == 1)
+			break;
+	}
+	fclose(f);
+
+	return nr;
+}
+
+/* Hugetlb pages aren't swap-backed, so the first fault on them fails. */
+static void test_hugetlb_refused(const char *name)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	u64 npages;
+	int ret;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_touch_one_page);
+	npages = get_backing_src_pagesz(VM_MEM_SRC_ANONYMOUS_HUGETLB) / vm->page_size;
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS_HUGETLB, GPA_BASE, GUEST_MEMSLOT,
+				    npages, 0);
+	virt_map(vm, GVA_BASE, GPA_BASE, 1);
+
+	ret = _vcpu_run(vcpu);
+	TEST_ASSERT(ret == -1 && errno == EIO,
+		    "hugetlb-backed memslot: KVM_RUN rc %d errno %d", ret, errno);
+
+	kvm_vm_free(vm);
+
+	ksft_test_result_pass("%s\n", name);
+}
+
+int main(void)
+{
+	const char *hugetlb = "hugetlb memory rejected for a protected VM";
+
+	TEST_REQUIRE(pkvm_supported());
+	granule = getpagesize();
+
+	ksft_print_header();
+	ksft_set_plan(4);
+
+	test_run();
+
+	if (hugetlb_free_pages())
+		test_hugetlb_refused(hugetlb);
+	else
+		ksft_test_result_skip("%s: no hugetlb pages free\n", hugetlb);
+
+	test_share_outside_memslot();
+	test_memcache_pressure();
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 11/19] KVM: arm64: selftests: Add pkvm_smccc hypercall test

[Fuad Tabba]

EL2 handles a protected guest's SMCCC calls other than PSCI rather than
forwarding them to the host. The vendor-hyp feature bitmap advertises
only FEATURES and the three memory calls.

Check SMCCC_VERSION, the vendor-hyp UID and feature bitmap, that the
calls EL2 doesn't implement (FF-A, TRNG, an unknown function) return
NOT_SUPPORTED, and that an SMC makes the guest take an UNDEF.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/arm64/pkvm_smccc.c  | 117 ++++++++++++++++++
 2 files changed, 118 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_smccc.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 88c1159cb3bf1..6bb436bd36205 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -193,6 +193,7 @@ TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
+TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_smccc.c b/tools/testing/selftests/kvm/arm64/pkvm_smccc.c
new file mode 100644
index 0000000000000..64bfa12b26824
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_smccc.c
@@ -0,0 +1,117 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_smccc - the SMCCC calls EL2 handles for a protected VM, and its SMC.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <linux/arm-smccc.h>
+#include <linux/bits.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/* KVM UID 28b46fb6-2ec5-11e9-a9ca-4b564d003a74, as four SMCCC result regs. */
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0	0xb66fb428U
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1	0xe911c52eU
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2	0x564bcaa9U
+#define ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3	0x743a004dU
+
+/* The vendor-hyp feature bitmap EL2 advertises to protected guests. */
+#define KVM_FEATURES (					\
+	BIT(ARM_SMCCC_KVM_FUNC_FEATURES) |		\
+	BIT(ARM_SMCCC_KVM_FUNC_HYP_MEMINFO) |		\
+	BIT(ARM_SMCCC_KVM_FUNC_MEM_SHARE) |		\
+	BIT(ARM_SMCCC_KVM_FUNC_MEM_UNSHARE))
+
+#define FFA_VERSION_FUNC_ID						\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,	\
+			   ARM_SMCCC_OWNER_STANDARD, 0x63)
+
+/* An unallocated vendor-hyp function id. */
+#define UNKNOWN_FUNC_ID							\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_64,	\
+			   ARM_SMCCC_OWNER_VENDOR_HYP, 30)
+
+extern unsigned char smc_insn;
+
+static volatile u64 undef_pc;
+
+static void guest_code(void)
+{
+	struct arm_smccc_res res;
+	unsigned long ret;
+
+	pkvm_guest_init();
+
+	/* The vendor-hyp calls require SMCCC 1.1. */
+	ret = pkvm_hvc(ARM_SMCCC_VERSION_FUNC_ID, 0, 0, 0);
+	__GUEST_ASSERT((s32)ret >= ARM_SMCCC_VERSION_1_1, "SMCCC_VERSION 0x%lx", ret);
+
+	smccc_hvc(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, 0, 0, 0, 0, 0, 0, 0, &res);
+	GUEST_ASSERT_EQ(res.a0, ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0);
+	GUEST_ASSERT_EQ(res.a1, ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1);
+	GUEST_ASSERT_EQ(res.a2, ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2);
+	GUEST_ASSERT_EQ(res.a3, ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3);
+
+	ret = pkvm_hvc(ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID, 0, 0, 0);
+	GUEST_ASSERT_EQ(ret, KVM_FEATURES);
+
+	/* EL2 returns NOT_SUPPORTED for a call it doesn't implement, rather than forward it. */
+	GUEST_ASSERT_EQ(pkvm_hvc(FFA_VERSION_FUNC_ID, 0, 0, 0), SMCCC_RET_NOT_SUPPORTED);
+	GUEST_ASSERT_EQ(pkvm_hvc(ARM_SMCCC_TRNG_VERSION, 0, 0, 0), SMCCC_RET_NOT_SUPPORTED);
+	GUEST_ASSERT_EQ(pkvm_hvc(UNKNOWN_FUNC_ID, 0, 0, 0), SMCCC_RET_NOT_SUPPORTED);
+
+	GUEST_DONE();
+}
+
+static void undef_handler(struct ex_regs *regs)
+{
+	undef_pc = regs->pc;
+	regs->pc += 4;
+}
+
+/* EL2 injects an UNDEF for a protected guest's SMC. */
+static void guest_smc(void)
+{
+	pkvm_guest_init();
+
+	{
+		register u64 x0 asm("x0") = ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID;
+
+		asm volatile("smc_insn: smc #0" : "+r"(x0) : : "memory");
+	}
+	GUEST_ASSERT_EQ(undef_pc, (u64)&smc_insn);
+	GUEST_DONE();
+}
+
+static void run_guest(void *guest_code, bool smc)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_code);
+	if (smc)
+		vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, ESR_ELx_EC_UNKNOWN, undef_handler);
+	pkvm_run_loop(vcpu, NULL);
+	kvm_vm_free(vm);
+}
+
+int main(void)
+{
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(2);
+
+	run_guest(guest_code, false);
+	ksft_test_result_pass("SMCCC surface\n");
+
+	run_guest(guest_smc, true);
+	ksft_test_result_pass("SMC takes an UNDEF\n");
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 12/19] KVM: arm64: selftests: Add pkvm_id_regs register and trap test

[Fuad Tabba]

pKVM gives a protected guest its own view of the ID registers,
hiding the features pKVM doesn't support. Their registers UNDEF,
except some hidden debug registers, which are RAZ/WI.

Test that view and those UNDEFs from the guest, and the
registers EL2 emulates with fixed values. The debug checks follow
ID_AA64DFR0_EL1.DebugVer. The PtrAuth and SVE cases run when a
protected VM's fd reports the capability, and the SVE case also
compares ID_AA64ZFR0_EL1 with what host userspace reads.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_id_regs.c        | 324 ++++++++++++++++++
 2 files changed, 325 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_id_regs.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 6bb436bd36205..8855c9abb2fd0 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -191,6 +191,7 @@ TEST_GEN_PROGS_arm64 += arm64/external_aborts
 TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
+TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c b/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
new file mode 100644
index 0000000000000..93ed557a30bc6
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
@@ -0,0 +1,324 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_id_regs - the ID registers a protected VM sees, and which of its
+ * register accesses UNDEF.
+ *
+ * Copyright (C) 2025 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+
+#include "gic_v3.h"
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/* vCPU 0's MPIDR: Aff0=0, bit31=1, MT=0. */
+#define MPIDR_VCPU0	0x80000000UL
+
+/* Hidden fields with a non-zero Not-Implemented encoding. */
+#define PVM_ID_AA64DFR0_NI	(					\
+	SYS_FIELD_PREP_ENUM(ID_AA64DFR0_EL1, DoubleLock, NI) |		\
+	SYS_FIELD_PREP_ENUM(ID_AA64DFR0_EL1, MTPMU, NI))
+#define PVM_ID_AA64MMFR4_NI	SYS_FIELD_PREP_ENUM(ID_AA64MMFR4_EL1, E2H0, NI)
+
+/* What self-hosted debug exposes: DebugVer and the breakpoint, watchpoint and CTX counts. */
+#define PVM_ID_AA64DFR0_DEBUG	(ID_AA64DFR0_EL1_DebugVer_MASK | ID_AA64DFR0_EL1_BRPs_MASK | \
+				 ID_AA64DFR0_EL1_WRPs_MASK | ID_AA64DFR0_EL1_CTX_CMPs_MASK)
+
+/* Breakpoint and watchpoint 0 controls with E clear: every byte, EL1, loads and stores. */
+#define DBGBCR_VAL	((0xfUL << 5) | (1UL << 1))
+#define DBGWCR_VAL	((0xffUL << 5) | (3UL << 3) | (1UL << 1))
+
+extern unsigned char brk_insn;
+
+static volatile bool undef_seen;
+static volatile u64 brk_pc;
+
+static void undef_handler(struct ex_regs *regs)
+{
+	undef_seen = true;
+	regs->pc += 4;
+}
+
+static void brk_handler(struct ex_regs *regs)
+{
+	brk_pc = regs->pc;
+	regs->pc += 4;
+}
+
+/* Access @expr must make the guest take an UNDEF (EC_UNKNOWN). */
+#define GUEST_ASSERT_UNDEF(expr) do {	\
+	undef_seen = false;		\
+	expr;				\
+	GUEST_ASSERT(undef_seen);	\
+} while (0)
+
+/* @reg reads back @pattern with no UNDEF. */
+#define GUEST_ASSERT_RW(reg, pattern) do {		\
+	u64 __saved = read_sysreg_s(reg);		\
+	undef_seen = false;				\
+	write_sysreg_s(pattern, reg);			\
+	GUEST_ASSERT_EQ(read_sysreg_s(reg), pattern);	\
+	write_sysreg_s(__saved, reg);			\
+	GUEST_ASSERT(!undef_seen);			\
+} while (0)
+
+static void assert_aarch32_id_raz(void)
+{
+	/* AArch32 is unsupported for protected guests: its ID block reads 0. */
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_PFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_PFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_DFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR2_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR3_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR2_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR3_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR4_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR5_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR4_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_ISAR6_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_MVFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_MVFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_MVFR2_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(sys_reg(3, 0, 0, 3, 3)), 0);	/* unallocated */
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_PFR2_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_DFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_MMFR5_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(sys_reg(3, 0, 0, 3, 7)), 0);	/* unallocated */
+}
+
+/* A zero DebugVer means EL2 hides self-hosted debug from the guest. */
+static bool debug_exposed(void)
+{
+	return SYS_FIELD_GET(ID_AA64DFR0_EL1, DebugVer, read_sysreg_s(SYS_ID_AA64DFR0_EL1));
+}
+
+static void assert_id_regs(void)
+{
+	u64 pfr0 = read_sysreg_s(SYS_ID_AA64PFR0_EL1);
+	u64 dfr0 = read_sysreg_s(SYS_ID_AA64DFR0_EL1);
+
+	if (debug_exposed()) {
+		u64 debugver = SYS_FIELD_GET(ID_AA64DFR0_EL1, DebugVer, dfr0);
+		u64 brps = SYS_FIELD_GET(ID_AA64DFR0_EL1, BRPs, dfr0);
+
+		/* Exposed debug: at most v8.8, and at least two breakpoints and two watchpoints. */
+		GUEST_ASSERT_EQ(dfr0 & ~PVM_ID_AA64DFR0_DEBUG, PVM_ID_AA64DFR0_NI);
+		GUEST_ASSERT(debugver >= ID_AA64DFR0_EL1_DebugVer_IMP);
+		GUEST_ASSERT(debugver <= ID_AA64DFR0_EL1_DebugVer_V8P8);
+		GUEST_ASSERT(brps >= 1);
+		GUEST_ASSERT(SYS_FIELD_GET(ID_AA64DFR0_EL1, WRPs, dfr0) >= 1);
+		GUEST_ASSERT(SYS_FIELD_GET(ID_AA64DFR0_EL1, CTX_CMPs, dfr0) <= brps);
+	} else {
+		/* Debug and PMU are hidden: DFR0 has a fixed value, no breakpoints. */
+		GUEST_ASSERT_EQ(dfr0, PVM_ID_AA64DFR0_NI);
+	}
+
+	/* With no vCPU features, SVE, ZFR0 and PtrAuth read 0. */
+	GUEST_ASSERT_EQ(SYS_FIELD_GET(ID_AA64PFR0_EL1, SVE, pfr0), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ZFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ISAR1_EL1) &
+			(ID_AA64ISAR1_EL1_APA_MASK | ID_AA64ISAR1_EL1_API_MASK |
+			 ID_AA64ISAR1_EL1_GPA_MASK | ID_AA64ISAR1_EL1_GPI_MASK), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ISAR2_EL1) &
+			(ID_AA64ISAR2_EL1_APA3_MASK | ID_AA64ISAR2_EL1_GPA3_MASK), 0);
+
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64DFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64AFR0_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64AFR1_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64SMFR0_EL1), 0);
+	GUEST_ASSERT_EQ(SYS_FIELD_GET(ID_AA64PFR1_EL1, MTE, read_sysreg_s(SYS_ID_AA64PFR1_EL1)), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64MMFR4_EL1), PVM_ID_AA64MMFR4_NI);
+
+	assert_aarch32_id_raz();
+}
+
+static void assert_debug_raz_wi(void)
+{
+	/* Breakpoint/watchpoint 0, MDSCR and OS-lock regs are RAZ/WI, with no UNDEF. */
+	undef_seen = false;
+	write_sysreg_s(~0UL, SYS_DBGBVRn_EL1(0));
+	write_sysreg_s(~0UL, SYS_DBGBCRn_EL1(0));
+	write_sysreg_s(~0UL, SYS_DBGWVRn_EL1(0));
+	write_sysreg_s(~0UL, SYS_DBGWCRn_EL1(0));
+	write_sysreg_s(~0UL, SYS_MDSCR_EL1);
+	write_sysreg_s(~0UL, SYS_OSLAR_EL1);
+	write_sysreg_s(~0UL, SYS_OSDLR_EL1);
+	isb();
+
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_DBGBVRn_EL1(0)), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_DBGBCRn_EL1(0)), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_DBGWVRn_EL1(0)), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_DBGWCRn_EL1(0)), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_MDSCR_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_OSLSR_EL1), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_OSDLR_EL1), 0);
+	GUEST_ASSERT(!undef_seen);
+
+	/* Breakpoint/watchpoint indices above 0 are not implemented. */
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_DBGBVRn_EL1(1)));
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_DBGWVRn_EL1(1)));
+}
+
+static void assert_debug_regs(void)
+{
+	u64 addr = (u64)&brk_insn;
+
+	/* Breakpoint/watchpoint 0 hold what the guest writes. */
+	GUEST_ASSERT_RW(SYS_DBGBVRn_EL1(0), addr);
+	GUEST_ASSERT_RW(SYS_DBGBCRn_EL1(0), DBGBCR_VAL);
+	GUEST_ASSERT_RW(SYS_DBGWVRn_EL1(0), addr & ~7UL);
+	GUEST_ASSERT_RW(SYS_DBGWCRn_EL1(0), DBGWCR_VAL);
+
+	/*
+	 * MDSCR doesn't UNDEF. The OS lock is implemented. The OS double lock isn't,
+	 * so OSDLR is RAZ/WI.
+	 */
+	undef_seen = false;
+	write_sysreg_s(read_sysreg_s(SYS_MDSCR_EL1), SYS_MDSCR_EL1);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_OSLSR_EL1) & OSLSR_EL1_OSLM_MASK,
+			OSLSR_EL1_OSLM_IMPLEMENTED);
+	write_sysreg_s(~0UL, SYS_OSDLR_EL1);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_OSDLR_EL1), 0);
+	GUEST_ASSERT(!undef_seen);
+}
+
+static void assert_undef_regs(void)
+{
+	/* The PMU is hidden, and its registers UNDEF. */
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_PMCR_EL0));
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_PMCNTENSET_EL0));
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_PMUSERENR_EL0));
+
+	/* Limited-ordering region registers UNDEF. */
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_LORID_EL1));
+
+	/* AMU and MTE are hidden, and so are the PtrAuth keys without the PtrAuth vCPU features. */
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_AMCNTENSET0_EL0));
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_GCR_EL1));
+	GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_APIAKEYLO_EL1));
+
+	/*
+	 * Nor is SVE, so an SVE instruction UNDEFs even with CPACR_EL1.ZEN set. On a CPU
+	 * with SME but no SVE, SMEN gates it instead, so set that too.
+	 */
+	write_sysreg(read_sysreg(cpacr_el1) | CPACR_EL1_ZEN_EL1EN | CPACR_EL1_SMEN_EL1EN,
+		     cpacr_el1);
+	isb();
+	GUEST_ASSERT_UNDEF(asm volatile(".arch_extension sve\n rdvl x0, #1" : : : "x0"));
+}
+
+static void assert_host_handled(void)
+{
+	/* Host-handled registers reach the guest with no UNDEF. */
+	undef_seen = false;
+	(void)read_sysreg_s(SYS_CTR_EL0);
+	(void)read_sysreg_s(SYS_CLIDR_EL1);
+	GUEST_ASSERT(!undef_seen);
+}
+
+static void guest_baseline(void)
+{
+	pkvm_guest_init();
+
+	assert_id_regs();
+	if (debug_exposed())
+		assert_debug_regs();
+	else
+		assert_debug_raz_wi();
+	assert_undef_regs();
+	assert_host_handled();
+
+	asm volatile("brk_insn: brk #0");
+	GUEST_ASSERT_EQ(brk_pc, (u64)&brk_insn);
+
+	/* EL2 emulates ICC_SRE_EL1 with a fixed value. */
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ICC_SRE_EL1),
+			ICC_SRE_EL1_DIB | ICC_SRE_EL1_DFB | ICC_SRE_EL1_SRE);
+
+	GUEST_ASSERT_EQ(read_sysreg(mpidr_el1), MPIDR_VCPU0);
+
+	GUEST_DONE();
+}
+
+/* The host implements one of QARMA5, an IMP DEF algorithm or QARMA3: any may appear. */
+static void guest_ptrauth(void)
+{
+	pkvm_guest_init();
+	GUEST_ASSERT((read_sysreg_s(SYS_ID_AA64ISAR1_EL1) &
+		      (ID_AA64ISAR1_EL1_APA_MASK | ID_AA64ISAR1_EL1_API_MASK)) |
+		     (read_sysreg_s(SYS_ID_AA64ISAR2_EL1) &
+		      (ID_AA64ISAR2_EL1_APA3_MASK | ID_AA64ISAR2_EL1_GPA3_MASK)));
+	GUEST_DONE();
+}
+
+/* Set in the guest's copy by the host: ID_AA64ZFR0_EL1 as host userspace reads it. */
+static u64 host_zfr0;
+
+static void guest_sve(void)
+{
+	pkvm_guest_init();
+	GUEST_ASSERT_EQ(SYS_FIELD_GET(ID_AA64PFR0_EL1, SVE, read_sysreg_s(SYS_ID_AA64PFR0_EL1)),
+			ID_AA64PFR0_EL1_SVE_IMP);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ZFR0_EL1), host_zfr0);
+	GUEST_DONE();
+}
+
+static void run_guest(u32 features, void *guest_code)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(1, features, &vcpu);
+	if (features & BIT(KVM_ARM_VCPU_SVE)) {
+		int feature = KVM_ARM_VCPU_SVE;
+
+		vcpu_ioctl(vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
+		host_zfr0 = read_sysreg_s(SYS_ID_AA64ZFR0_EL1);
+		sync_global_to_guest(vm, host_zfr0);
+	}
+	vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, ESR_ELx_EC_UNKNOWN, undef_handler);
+	vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, ESR_ELx_EC_BRK64, brk_handler);
+
+	pkvm_vcpu_setup_boot(vcpu, guest_code);
+	pkvm_run_loop(vcpu, NULL);
+	kvm_vm_free(vm);
+}
+
+int main(void)
+{
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(3);
+
+	run_guest(0, guest_baseline);
+	ksft_test_result_pass("baseline ID view and UNDEF matrix\n");
+
+	if (pkvm_has_cap(KVM_CAP_ARM_SVE)) {
+		run_guest(BIT(KVM_ARM_VCPU_SVE), guest_sve);
+		ksft_test_result_pass("SVE ID delta\n");
+	} else {
+		ksft_test_result_skip("SVE ID delta: SVE unavailable to a protected VM\n");
+	}
+
+	if (pkvm_has_cap(KVM_CAP_ARM_PTRAUTH_ADDRESS) &&
+	    pkvm_has_cap(KVM_CAP_ARM_PTRAUTH_GENERIC)) {
+		run_guest(BIT(KVM_ARM_VCPU_PTRAUTH_ADDRESS) |
+			  BIT(KVM_ARM_VCPU_PTRAUTH_GENERIC), guest_ptrauth);
+		ksft_test_result_pass("PtrAuth ID delta\n");
+	} else {
+		ksft_test_result_skip("PtrAuth ID delta: PtrAuth unavailable\n");
+	}
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 13/19] KVM: arm64: selftests: Add pkvm_vcpu_regs register-state test

[Fuad Tabba]

A protected vCPU's registers are held at EL2, which handles some exits
itself and returns others to the host. The host can move the vCPU's
thread to another CPU at any exit. The guest's registers have to keep
their values through all of that.

Verify that x19-x28 and v0-v31 keep the values the guest loaded across
each kind of exit, and across a host CPU migration at every MMIO exit.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_vcpu_regs.c      | 218 ++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_vcpu_regs.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 8855c9abb2fd0..1e40e793aa6f3 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -195,6 +195,7 @@ TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
+TEST_GEN_PROGS_arm64 += arm64/pkvm_vcpu_regs
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
 TEST_GEN_PROGS_arm64 += arm64/set_id_regs
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_vcpu_regs.c b/tools/testing/selftests/kvm/arm64/pkvm_vcpu_regs.c
new file mode 100644
index 0000000000000..41023b62800aa
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_vcpu_regs.c
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_vcpu_regs - a protected guest's registers across every kind of exit.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <sched.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/* Accesses to this IPA, outside every memslot, exit to the VMM as MMIO. */
+#define MMIO_IPA	BIT_ULL(31)
+#define MMIO_READ_VAL	0x0123456789abcdefUL
+
+/* One new host page per stage-2 fault. */
+#define FAULT_MEMSLOT	(PKVM_IDMAP_MEMSLOT + 1)
+#define FAULT_GPA	BIT_ULL(30)
+
+#define NR_EXITS	64
+
+enum exit_kind {
+	EXIT_MMIO_WRITE,
+	EXIT_MMIO_READ,
+	EXIT_HVC,
+	EXIT_SYSREG_EL2,
+	EXIT_SYSREG_HOST,
+	EXIT_S2_FAULT,
+	NR_EXIT_KINDS,
+};
+
+static const char * const exit_kind_names[] = {
+	[EXIT_MMIO_WRITE]	= "MMIO writes to the VMM",
+	[EXIT_MMIO_READ]	= "MMIO reads from the VMM",
+	[EXIT_HVC]		= "hypercalls EL2 handles",
+	[EXIT_SYSREG_EL2]	= "sysreg traps EL2 handles",
+	[EXIT_SYSREG_HOST]	= "sysreg traps the host handles",
+	[EXIT_S2_FAULT]		= "stage-2 faults on new memory",
+};
+
+/* x19-x28, v0-v31, then what the exit left in x0. */
+struct reg_state {
+	u64 x[10];
+	u64 v[32][2];
+	u64 x0;
+};
+
+#define REG_WORDS	(offsetof(struct reg_state, x0) / sizeof(u64))
+
+static_assert(offsetof(struct reg_state, v) == 80, "v0 offset");
+static_assert(offsetof(struct reg_state, x0) == 592, "x0 offset");
+
+#define X_REGS	"19, 20, 21, 22, 23, 24, 25, 26, 27, 28"
+#define V_REGS	"0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, "		\
+		"16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31"
+
+/* @op x19-x28 and v0-v31 at their struct reg_state offsets from @buf. */
+#define REGS_OP(op, buf)								\
+	".irp num, " X_REGS "\n" op "	x\\num, [%[" buf "], #8 * (\\num - 19)]\n.endr\n"	\
+	".irp num, " V_REGS "\n" op "	q\\num, [%[" buf "], #80 + 16 * \\num]\n.endr\n"
+
+/* Load the registers, take the exit @insn causes, and store them again. */
+#define REGS_ACROSS(insn, addr)								\
+({											\
+	asm volatile(REGS_OP("ldr", "in") insn "\n" REGS_OP("str", "out")		\
+		     "str	x0, [%[out], #592]\n"						\
+		     : : [in] "r"(&regs_in), [out] "r"(&regs_out), [arg] "r"(addr)		\
+		     : "x0", "x1", "x2", "x3",						\
+		       "x19", "x20", "x21", "x22", "x23", "x24", "x25", "x26", "x27", "x28",	\
+		       "v0", "v1", "v2", "v3", "v4", "v5", "v6", "v7",			\
+		       "v8", "v9", "v10", "v11", "v12", "v13", "v14", "v15",		\
+		       "v16", "v17", "v18", "v19", "v20", "v21", "v22", "v23",		\
+		       "v24", "v25", "v26", "v27", "v28", "v29", "v30", "v31",		\
+		       "memory");							\
+})
+
+/* Set in the guest's copy by the host before the first run. */
+static int exit_kind;
+static u64 host_page_size;
+
+static struct reg_state regs_in, regs_out;
+
+static void guest_regs_across_exits(void)
+{
+	u64 *in = (u64 *)&regs_in, *out = (u64 *)&regs_out;
+	int i, j;
+
+	pkvm_guest_init();
+
+	for (i = 0; i < NR_EXITS; i++) {
+		/* Different in every word and on every exit. */
+		for (j = 0; j < REG_WORDS; j++)
+			in[j] = 0x0101010101010101UL * (j + 1) ^ (u64)i << 32;
+
+		switch (exit_kind) {
+		case EXIT_MMIO_WRITE:
+			REGS_ACROSS("str	x0, [%[arg]]", MMIO_IPA);
+			break;
+		case EXIT_MMIO_READ:
+			REGS_ACROSS("ldr	x0, [%[arg]]", MMIO_IPA);
+			GUEST_ASSERT_EQ(regs_out.x0, MMIO_READ_VAL);
+			break;
+		case EXIT_HVC:
+			REGS_ACROSS("mov	w0, #0x80000000\n hvc	#0", 0);
+			break;
+		case EXIT_SYSREG_EL2:
+			REGS_ACROSS("mrs	x0, id_aa64pfr0_el1", 0);
+			break;
+		case EXIT_SYSREG_HOST:
+			REGS_ACROSS("mrs	x0, clidr_el1", 0);
+			break;
+		case EXIT_S2_FAULT:
+			REGS_ACROSS("str	x0, [%[arg]]", FAULT_GPA + i * host_page_size);
+			break;
+		}
+
+		for (j = 0; j < REG_WORDS; j++)
+			__GUEST_ASSERT(out[j] == in[j], "exit %d: %s%d is 0x%lx, not 0x%lx", i,
+				       j < 10 ? "x" : "v", j < 10 ? 19 + j : (j - 10) / 2,
+				       out[j], in[j]);
+	}
+
+	GUEST_DONE();
+}
+
+static cpu_set_t host_cpus;
+
+/* Move this thread, and with it the vCPU's next run, to another of its CPUs. */
+static void migrate_thread(void)
+{
+	int cpu = sched_getcpu();
+
+	do
+		cpu = (cpu + 1) % CPU_SETSIZE;
+	while (!CPU_ISSET(cpu, &host_cpus));
+
+	pin_self_to_cpu(cpu);
+	TEST_ASSERT_EQ(sched_getcpu(), cpu);
+}
+
+static void run_regs_across_exits(struct kvm_vcpu *vcpu, bool migrate)
+{
+	struct kvm_run *run = vcpu->run;
+	u64 data = MMIO_READ_VAL;
+	struct ucall uc;
+
+	for (;;) {
+		vcpu_run(vcpu);
+
+		if (run->exit_reason == KVM_EXIT_MMIO && run->mmio.phys_addr == MMIO_IPA) {
+			if (!run->mmio.is_write)
+				memcpy(run->mmio.data, &data, sizeof(data));
+			if (migrate)
+				migrate_thread();
+			continue;
+		}
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_DONE:
+			return;
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			break;
+		default:
+			TEST_FAIL("Unexpected exit: %s", exit_reason_str(run->exit_reason));
+		}
+	}
+}
+
+/* With @migrate, the vCPU moves to another host CPU at every MMIO exit. */
+static void test_regs_across_exits(enum exit_kind kind, bool migrate)
+{
+	struct kvm_vcpu *vcpu;
+	unsigned int npages;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_regs_across_exits);
+	npages = NR_EXITS * getpagesize() / vm->page_size;
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, FAULT_GPA, FAULT_MEMSLOT, npages, 0);
+	virt_map(vm, FAULT_GPA, FAULT_GPA, npages);
+	virt_map(vm, MMIO_IPA, MMIO_IPA, 1);
+	write_guest_global(vm, exit_kind, kind);
+	write_guest_global(vm, host_page_size, getpagesize());
+
+	run_regs_across_exits(vcpu, migrate);
+
+	kvm_vm_free(vm);
+}
+
+int main(void)
+{
+	int kind;
+
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(1 + NR_EXIT_KINDS);
+
+	for (kind = 0; kind < NR_EXIT_KINDS; kind++) {
+		test_regs_across_exits(kind, false);
+		ksft_test_result_pass("x19-x28 and v0-v31 across %s\n", exit_kind_names[kind]);
+	}
+
+	kvm_sched_getaffinity(0, sizeof(host_cpus), &host_cpus);
+	if (CPU_COUNT(&host_cpus) < 2) {
+		ksft_test_result_skip("x19-x28 and v0-v31 across host CPU migrations: one CPU\n");
+	} else {
+		test_regs_across_exits(EXIT_MMIO_WRITE, true);
+		kvm_sched_setaffinity(0, sizeof(host_cpus), &host_cpus);
+		ksft_test_result_pass("x19-x28 and v0-v31 across host CPU migrations\n");
+	}
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 14/19] KVM: arm64: selftests: Add pkvm_irq interrupt test

[Fuad Tabba]

The host emulates a protected guest's vGIC, so EL2 copies the list
registers to and from the host each time the host runs the vCPU. EL2
also keeps its own copy of the guest's virtual timer state, which the
host can't change.

Check that the virtual timer's PPI arrives exactly once, with CVAL and
CTL kept across exits, that SGIs pass between two vCPUs, and that the
timer wakes a vCPU in WFI.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm     |   1 +
 tools/testing/selftests/kvm/arm64/pkvm_irq.c | 268 +++++++++++++++++++
 2 files changed, 269 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_irq.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 1e40e793aa6f3..13998964facd5 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -192,6 +192,7 @@ TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
+TEST_GEN_PROGS_arm64 += arm64/pkvm_irq
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_irq.c b/tools/testing/selftests/kvm/arm64/pkvm_irq.c
new file mode 100644
index 0000000000000..f92fa6f0de8de
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_irq.c
@@ -0,0 +1,268 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_irq - the virtual timer's PPI and SGIs in a protected VM.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <pthread.h>
+
+#include <linux/bitfield.h>
+#include <linux/psci.h>
+#include <asm/cputype.h>
+
+#include "arch_timer.h"
+#include "gic.h"
+#include "gic_v3.h"
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define TIMER_MS	10
+#define WAIT_MS		30000
+#define NR_EXITS	16
+#define SGI_ROUNDS	16
+
+#define SGI_PING	1
+#define SGI_PONG	2
+
+/* Set in the guest's copy by the host before the first run. */
+static u32 vtimer_irq;
+static u64 boot_entry;
+static u64 vcpu_mpidr[2];
+static u64 vcpu_ctx[2];
+
+static u32 irqs[2];
+static u32 last_intid[2];
+static u32 ready;
+
+#define wait_until(cond)							\
+do {										\
+	u64 __end = timer_get_cntct(VIRTUAL) + msec_to_cycles(WAIT_MS);	\
+										\
+	while (!(cond))								\
+		__GUEST_ASSERT(timer_get_cntct(VIRTUAL) < __end,		\
+			       "timed out waiting for %s", #cond);		\
+} while (0)
+
+static void guest_irq_handler(struct ex_regs *regs)
+{
+	u32 intid = gic_get_and_ack_irq();
+	u32 cpu = guest_get_vcpuid();
+
+	if (intid == IAR_SPURIOUS)
+		return;
+
+	if (intid == vtimer_irq) {
+		__GUEST_ASSERT(timer_get_ctl(VIRTUAL) & CTL_ISTATUS, "CNTV_CTL 0x%x",
+			       timer_get_ctl(VIRTUAL));
+		GUEST_ASSERT(timer_get_cntct(VIRTUAL) >= timer_get_cval(VIRTUAL));
+		/* Mask the timer before the EOI, or its level-sensitive PPI is taken again. */
+		timer_set_ctl(VIRTUAL, CTL_IMASK);
+	}
+
+	WRITE_ONCE(last_intid[cpu], intid);
+	gic_set_eoi(intid);
+	WRITE_ONCE(irqs[cpu], irqs[cpu] + 1);
+}
+
+static void guest_irq_init(unsigned int nr_cpus)
+{
+	local_irq_disable();
+	pkvm_guest_init();
+	gic_init(GIC_V3, nr_cpus);
+}
+
+static u64 start_vtimer(void)
+{
+	u64 cval = timer_get_cntct(VIRTUAL) + msec_to_cycles(TIMER_MS);
+
+	gic_irq_enable(vtimer_irq);
+	timer_set_cval(VIRTUAL, cval);
+	timer_set_ctl(VIRTUAL, CTL_ENABLE);
+	return cval;
+}
+
+static void guest_vtimer(void)
+{
+	u64 cval;
+	int i;
+
+	guest_irq_init(1);
+	cval = start_vtimer();
+
+	/* Exits before the timer fires, then with its interrupt pending but masked. */
+	for (i = 0; i < NR_EXITS; i++)
+		GUEST_SYNC(i);
+	wait_until(timer_get_ctl(VIRTUAL) & CTL_ISTATUS);
+	for (i = 0; i < NR_EXITS; i++)
+		GUEST_SYNC(i);
+
+	GUEST_ASSERT_EQ(timer_get_cval(VIRTUAL), cval);
+	GUEST_ASSERT_EQ(timer_get_ctl(VIRTUAL), CTL_ENABLE | CTL_ISTATUS);
+	GUEST_ASSERT_EQ(READ_ONCE(irqs[0]), 0);
+
+	local_irq_enable();
+	wait_until(READ_ONCE(irqs[0]));
+
+	/* The EOI must reach the host, or it presents the interrupt again. */
+	for (i = 0; i < NR_EXITS; i++)
+		GUEST_SYNC(i);
+	local_irq_disable();
+
+	GUEST_ASSERT_EQ(READ_ONCE(irqs[0]), 1);
+	GUEST_ASSERT_EQ(READ_ONCE(last_intid[0]), vtimer_irq);
+	GUEST_DONE();
+}
+
+static void send_sgi(int id, u32 intid)
+{
+	u64 mpidr = vcpu_mpidr[id];
+
+	/* The target reads this vCPU's counters once it has taken the SGI. */
+	dsb(ishst);
+	write_sysreg_s(FIELD_PREP(ICC_SGI1R_SGI_ID_MASK, intid) |
+		       FIELD_PREP(ICC_SGI1R_AFFINITY_1_MASK, MPIDR_AFFINITY_LEVEL(mpidr, 1)) |
+		       FIELD_PREP(ICC_SGI1R_TARGET_LIST_MASK, BIT(MPIDR_AFFINITY_LEVEL(mpidr, 0))),
+		       SYS_ICC_SGI1R_EL1);
+	isb();
+}
+
+static void guest_sgi_primary(void)
+{
+	u32 k;
+
+	guest_irq_init(2);
+	gic_irq_enable(SGI_PONG);
+
+	GUEST_ASSERT_EQ(pkvm_hvc(PSCI_0_2_FN64_CPU_ON, vcpu_mpidr[1], boot_entry, vcpu_ctx[1]),
+			PSCI_RET_SUCCESS);
+	wait_until(READ_ONCE(ready));
+
+	local_irq_enable();
+	for (k = 1; k <= SGI_ROUNDS; k++) {
+		send_sgi(1, SGI_PING);
+
+		/* vCPU 1 sends PONG only once it has taken PING, so neither is in flight. */
+		wait_until(READ_ONCE(irqs[0]) >= k);
+		GUEST_ASSERT_EQ(READ_ONCE(irqs[0]), k);
+		GUEST_ASSERT_EQ(READ_ONCE(irqs[1]), k);
+		GUEST_ASSERT_EQ(READ_ONCE(last_intid[0]), SGI_PONG);
+		GUEST_ASSERT_EQ(READ_ONCE(last_intid[1]), SGI_PING);
+	}
+	local_irq_disable();
+
+	GUEST_DONE();
+}
+
+/* The primary's guest_irq_init() set up ucall and the distributor. */
+static void guest_sgi_secondary(void)
+{
+	u32 k;
+
+	gic_init(GIC_V3, 2);
+	gic_irq_enable(SGI_PING);
+	local_irq_enable();
+	WRITE_ONCE(ready, 1);
+
+	for (k = 1; k <= SGI_ROUNDS; k++) {
+		wait_until(READ_ONCE(irqs[1]) >= k);
+		send_sgi(0, SGI_PONG);
+	}
+	local_irq_disable();
+
+	GUEST_DONE();
+}
+
+static void guest_wfi(void)
+{
+	guest_irq_init(1);
+	start_vtimer();
+
+	/* WFI can complete before the timer fires, so wait in it until the handler has run. */
+	while (!READ_ONCE(irqs[0])) {
+		wfi();
+		local_irq_enable();
+		isb(); /* handle IRQ */
+		local_irq_disable();
+	}
+
+	GUEST_ASSERT_EQ(READ_ONCE(irqs[0]), 1);
+	GUEST_ASSERT_EQ(READ_ONCE(last_intid[0]), vtimer_irq);
+	GUEST_DONE();
+}
+
+static void irq_vm_setup(struct kvm_vm *vm, struct kvm_vcpu *vcpu)
+{
+	vm_install_exception_handler(vm, VECTOR_IRQ_CURRENT, guest_irq_handler);
+	write_guest_global(vm, vtimer_irq, vcpu_get_vtimer_irq(vcpu));
+}
+
+static void test_one_vcpu(void *guest_code)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_code);
+	irq_vm_setup(vm, vcpu);
+
+	pkvm_run_loop(vcpu, NULL);
+
+	kvm_vm_free(vm);
+}
+
+static void wire_mpidr(struct kvm_vm *vm, struct kvm_vcpu *vcpu)
+{
+	u64 mpidr = vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_MPIDR_EL1));
+
+	write_guest_global(vm, vcpu_mpidr[vcpu->id], mpidr & MPIDR_HWID_BITMASK);
+}
+
+static void *vcpu_thread(void *vcpu)
+{
+	pkvm_run_loop(vcpu, NULL);
+	return NULL;
+}
+
+static void test_sgi(void)
+{
+	struct kvm_vcpu *vcpus[2];
+	pthread_t thread;
+	struct kvm_vm *vm;
+	gpa_t ctx;
+
+	vm = pkvm_vm_create(2, 0, vcpus);
+	irq_vm_setup(vm, vcpus[0]);
+
+	pkvm_vcpu_setup_boot(vcpus[0], guest_sgi_primary);
+	wire_mpidr(vm, vcpus[0]);
+	wire_mpidr(vm, vcpus[1]);
+	write_guest_global(vm, boot_entry, pkvm_idmap_gpa(vm));
+	ctx = pkvm_vcpu_setup_boot_secondary(vcpus[1], guest_sgi_secondary);
+	write_guest_global(vm, vcpu_ctx[1], ctx);
+
+	pthread_create(&thread, NULL, vcpu_thread, vcpus[1]);
+	pkvm_run_loop(vcpus[0], NULL);
+	pthread_join(thread, NULL);
+
+	kvm_vm_free(vm);
+}
+
+int main(void)
+{
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(3);
+
+	test_one_vcpu(guest_vtimer);
+	ksft_test_result_pass("virtual timer PPI across exits\n");
+	test_sgi();
+	ksft_test_result_pass("SGIs between two vCPUs: %d rounds\n", SGI_ROUNDS);
+	test_one_vcpu(guest_wfi);
+	ksft_test_result_pass("the virtual timer wakes a vCPU waiting in WFI\n");
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 15/19] KVM: arm64: selftests: Add pkvm_isolation host-access test

[Fuad Tabba]

The host keeps its userspace mapping of a protected guest's memory, but
can't reach the data through it. The host kernel's copy_to_user() and
copy_from_user() on that mapping fail. If the host kernel pins a page
and accesses it through its own mapping, as process_vm_readv() does,
EL2 zeroes and reclaims the page, and the guest loses it.

Test each of these, that MADV_DONTNEED and munmap() leave the guest's
pages pinned and accounted, and that a normal VM's fault on them fails.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_isolation.c      | 225 ++++++++++++++++++
 2 files changed, 226 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_isolation.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 13998964facd5..68bdd01f62270 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -193,6 +193,7 @@ TEST_GEN_PROGS_arm64 += arm64/page_fault_test
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
 TEST_GEN_PROGS_arm64 += arm64/pkvm_irq
+TEST_GEN_PROGS_arm64 += arm64/pkvm_isolation
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_isolation.c b/tools/testing/selftests/kvm/arm64/pkvm_isolation.c
new file mode 100644
index 0000000000000..f083d0434238a
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_isolation.c
@@ -0,0 +1,225 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_isolation - what the host can and can't do with a protected VM's memory.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <errno.h>
+#include <sys/mman.h>
+#include <sys/uio.h>
+#include <unistd.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define DATA_MEMSLOT	(PKVM_IDMAP_MEMSLOT + 1)
+#define DATA_GPA	BIT_ULL(30)
+#define DATA_PAGES	16
+#define DATA_SIZE	((size_t)DATA_PAGES * MIN_PAGE_SIZE)
+
+#define GUEST_VAL	0x0badf00ddeadbeefUL
+#define HOST_VAL	0x0123456789abcdefUL
+
+static volatile u64 *page_va(int idx)
+{
+	return (volatile u64 *)(DATA_GPA + (u64)idx * MIN_PAGE_SIZE);
+}
+
+/* The host acts on the guest's memory at the sync, then the guest checks it. */
+static void guest_code(void)
+{
+	int i;
+
+	pkvm_guest_init();
+
+	for (i = 0; i < DATA_PAGES; i++)
+		*page_va(i) = GUEST_VAL + i;
+
+	GUEST_SYNC(0);
+
+	for (i = 0; i < DATA_PAGES; i++)
+		GUEST_ASSERT_EQ(*page_va(i), GUEST_VAL + i);
+
+	GUEST_DONE();
+}
+
+static void guest_np_read(void)
+{
+	*page_va(0);
+	GUEST_DONE();
+}
+
+/* The host kernel's copy to or from private memory faults, and the uaccess fixup fails it. */
+static void host_write_from_private(void *hva)
+{
+	ssize_t ret;
+	int fds[2];
+
+	TEST_ASSERT(!pipe(fds), "pipe: errno %d", errno);
+
+	ret = write(fds[1], hva, MIN_PAGE_SIZE);
+	TEST_ASSERT(ret == -1 && errno == EFAULT,
+		    "write() from private memory: rc %zd errno %d", ret, errno);
+
+	close(fds[0]);
+	close(fds[1]);
+}
+
+static void host_read_into_private(void *hva)
+{
+	u64 val = HOST_VAL;
+	ssize_t ret;
+	int fds[2];
+
+	TEST_ASSERT(!pipe(fds), "pipe: errno %d", errno);
+	TEST_ASSERT_EQ(write(fds[1], &val, sizeof(val)), sizeof(val));
+
+	ret = read(fds[0], hva, sizeof(val));
+	TEST_ASSERT(ret == -1 && errno == EFAULT,
+		    "read() into private memory: rc %zd errno %d", ret, errno);
+
+	close(fds[0]);
+	close(fds[1]);
+}
+
+/* The host's long-term pins keep the guest's pages and their locked-memory accounting. */
+static void host_dontneed(void *hva)
+{
+	size_t locked = pkvm_host_locked_vm();
+
+	kvm_madvise(hva, DATA_SIZE, MADV_DONTNEED);
+	TEST_ASSERT_EQ(pkvm_host_locked_vm(), locked);
+
+	/* The host's mapping now has a new page, which the guest doesn't see. */
+	TEST_ASSERT_EQ(*(u64 *)hva, 0);
+	*(u64 *)hva = HOST_VAL;
+}
+
+static void host_munmap(void *hva)
+{
+	size_t locked = pkvm_host_locked_vm();
+
+	kvm_munmap(hva, DATA_SIZE);
+	TEST_ASSERT_EQ(pkvm_host_locked_vm(), locked);
+}
+
+/* A normal VM's fault on private memory fails, and the protected guest keeps its data. */
+static void host_np_vm_on_private(void *hva)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	int ret;
+
+	vm = vm_create_with_one_vcpu(&vcpu, guest_np_read);
+	vm_set_user_memory_region(vm, DATA_MEMSLOT, 0, DATA_GPA, getpagesize(), hva);
+	virt_map(vm, DATA_GPA, DATA_GPA, 1);
+
+	ret = _vcpu_run(vcpu);
+	TEST_ASSERT(ret == -1 && errno == EPERM,
+		    "normal VM's fault on private memory: rc %d errno %d", ret, errno);
+
+	kvm_vm_free(vm);
+}
+
+static void (*host_action)(void *hva);
+
+static void on_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	host_action(addr_gpa2hva(vcpu->vm, DATA_GPA));
+}
+
+static struct kvm_vm *vm_create_with_data(struct kvm_vcpu **vcpu)
+{
+	struct kvm_vm *vm = pkvm_vm_create_with_one_vcpu(vcpu, guest_code);
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, DATA_GPA, DATA_MEMSLOT,
+				    DATA_PAGES, 0);
+	virt_map(vm, DATA_GPA, DATA_GPA, DATA_PAGES);
+	return vm;
+}
+
+static void test_host_access(void (*action)(void *hva), const char *name)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = vm_create_with_data(&vcpu);
+
+	host_action = action;
+	pkvm_run_loop(vcpu, on_sync);
+
+	kvm_vm_free(vm);
+	TEST_ASSERT_EQ(pkvm_host_locked_vm(), 0);
+
+	ksft_test_result_pass("%s\n", name);
+}
+
+/*
+ * The host never reads the guest's data: a kernel copy from private memory
+ * reads zeros, and the guest's next access to the page fails.
+ */
+static void test_process_vm_readv(void)
+{
+	u64 buf[2] = { HOST_VAL, HOST_VAL };
+	struct iovec local = { .iov_base = buf, .iov_len = sizeof(buf) };
+	struct kvm_vcpu *vcpu;
+	struct iovec remote;
+	struct kvm_vm *vm;
+	struct ucall uc;
+	ssize_t ret;
+	int i;
+
+	vm = vm_create_with_data(&vcpu);
+	vcpu_run(vcpu);
+	TEST_ASSERT_EQ(get_ucall(vcpu, &uc), UCALL_SYNC);
+
+	remote = (struct iovec){ .iov_base = addr_gpa2hva(vm, DATA_GPA), .iov_len = sizeof(buf) };
+	ret = process_vm_readv(getpid(), &local, 1, &remote, 1, 0);
+	TEST_ASSERT_EQ(ret, sizeof(buf));
+	for (i = 0; i < ARRAY_SIZE(buf); i++)
+		TEST_ASSERT(!buf[i], "process_vm_readv() of private memory read 0x%lx", buf[i]);
+
+	/* The guest's next access is its read of that page. */
+	ret = _vcpu_run(vcpu);
+	TEST_ASSERT(ret == -1 && errno == EFAULT,
+		    "guest access after process_vm_readv(): rc %zd errno %d", ret, errno);
+
+	kvm_vm_free(vm);
+	TEST_ASSERT_EQ(pkvm_host_locked_vm(), 0);
+
+	ksft_test_result_pass("process_vm_readv() of private memory reads no guest data\n");
+}
+
+static const struct {
+	void (*action)(void *hva);
+	const char *name;
+} host_accesses[] = {
+	{ host_write_from_private, "write() from private memory fails with EFAULT" },
+	{ host_read_into_private, "read() into private memory fails with EFAULT" },
+	{ host_dontneed, "MADV_DONTNEED of private memory leaves the guest's data" },
+	{ host_munmap, "munmap() of private memory leaves the guest's data" },
+};
+
+int main(void)
+{
+	const char *np_vm = "a normal VM's fault on private memory fails with EPERM";
+	int i;
+
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(ARRAY_SIZE(host_accesses) + 2);
+
+	for (i = 0; i < ARRAY_SIZE(host_accesses); i++)
+		test_host_access(host_accesses[i].action, host_accesses[i].name);
+
+	test_process_vm_readv();
+
+	test_host_access(host_np_vm_on_private, np_vm);
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 16/19] KVM: arm64: selftests: Add pkvm_lifecycle teardown test

[Fuad Tabba]

Each protected VM holds an entry in EL2's table of KVM_MAX_PVMS (255)
VMs until teardown has reclaimed its pages, so if teardown leaks
entries, KVM_CREATE_VM eventually fails.

Create, run and free twice that many VMs, four alive at once and freed
in random order. Check that each VM's memory survives the others'
teardown, and that freeing a VM drops the process's locked memory by
what its runs added.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_lifecycle.c      | 110 ++++++++++++++++++
 2 files changed, 111 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_lifecycle.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 68bdd01f62270..214f38e3f67a7 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -194,6 +194,7 @@ TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
 TEST_GEN_PROGS_arm64 += arm64/pkvm_irq
 TEST_GEN_PROGS_arm64 += arm64/pkvm_isolation
+TEST_GEN_PROGS_arm64 += arm64/pkvm_lifecycle
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_lifecycle.c b/tools/testing/selftests/kvm/arm64/pkvm_lifecycle.c
new file mode 100644
index 0000000000000..0ee475799abc5
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_lifecycle.c
@@ -0,0 +1,110 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_lifecycle - more protected VMs than EL2's VM table holds, freed out of order.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <stdlib.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/* EL2's VM table has KVM_MAX_PVMS entries, and a VM holds one until it's freed. */
+#define VM_TABLE_SIZE	255
+/* Enough that leaking the entry of one VM in two fills the table. */
+#define NR_VMS		(2 * VM_TABLE_SIZE)
+#define NR_ALIVE	4
+
+struct pvm {
+	struct kvm_vm *vm;
+	struct kvm_vcpu *vcpu;
+	u64 tag;
+	size_t locked;
+};
+
+/* Set in the guest's copy by the host before the first run. */
+static u64 tag;
+
+static void guest_code(void)
+{
+	pkvm_guest_init();
+	for (;;)
+		GUEST_SYNC(READ_ONCE(tag));
+}
+
+/* The tag comes from the guest's own memory, so the VM must survive the others' teardown. */
+static void run_vm(struct pvm *p)
+{
+	size_t locked = pkvm_host_locked_vm();
+	struct ucall uc;
+	u64 cmd;
+
+	vcpu_run(p->vcpu);
+
+	switch (cmd = get_ucall(p->vcpu, &uc)) {
+	case UCALL_SYNC:
+		break;
+	case UCALL_ABORT:
+		REPORT_GUEST_ASSERT(uc);
+		break;
+	default:
+		TEST_FAIL("Unexpected guest exit: ucall %lu", cmd);
+	}
+	TEST_ASSERT_EQ(uc.args[1], p->tag);
+
+	p->locked += pkvm_host_locked_vm() - locked;
+}
+
+static void create_vm(struct pvm *p, u64 tag_val)
+{
+	p->vm = pkvm_vm_create_with_one_vcpu(&p->vcpu, guest_code);
+	p->tag = tag_val;
+	p->locked = 0;
+	write_guest_global(p->vm, tag, tag_val);
+
+	run_vm(p);
+	TEST_ASSERT(p->locked, "VM %lu pinned no memory", tag_val);
+}
+
+static void free_vm(struct pvm *p)
+{
+	size_t locked = pkvm_host_locked_vm();
+
+	kvm_vm_free(p->vm);
+	TEST_ASSERT_EQ(locked - pkvm_host_locked_vm(), p->locked);
+}
+
+int main(void)
+{
+	struct pvm vms[NR_ALIVE];
+	int i, j;
+
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(1);
+
+	for (i = 0; i < NR_ALIVE; i++)
+		create_vm(&vms[i], i + 1);
+
+	for (; i < NR_VMS; i++) {
+		struct pvm *victim = &vms[random() % NR_ALIVE];
+
+		for (j = 0; j < NR_ALIVE; j++)
+			run_vm(&vms[j]);
+
+		free_vm(victim);
+		create_vm(victim, i + 1);
+	}
+
+	for (j = 0; j < NR_ALIVE; j++)
+		free_vm(&vms[j]);
+
+	ksft_test_result_pass("%d VMs, %d alive at once, freed in random order\n",
+			      NR_VMS, NR_ALIVE);
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 17/19] KVM: arm64: selftests: Add pkvm_exceptions debug and SError test

[Fuad Tabba]

pKVM can hide self-hosted debug from a protected guest, which then reads
ID_AA64DFR0_EL1.DebugVer as 0. A virtual SError the host injects has to
be delivered by EL2, since the host can't see whether the guest has it
masked.

Verify that the guest's own handler takes a hardware breakpoint, a
watchpoint and a single step, and that an injected SError is pending in
ISR_EL1.A while PSTATE.A masks it and is taken once unmasked. The debug
cases skip while ID_AA64DFR0_EL1.DebugVer is 0.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/arm64/pkvm_exceptions.c     | 261 ++++++++++++++++++
 2 files changed, 262 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_exceptions.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 214f38e3f67a7..1045136aacc16 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -190,6 +190,7 @@ TEST_GEN_PROGS_arm64 += arm64/nv_pre_fault_memory_test
 TEST_GEN_PROGS_arm64 += arm64/external_aborts
 TEST_GEN_PROGS_arm64 += arm64/mmio_sign_ext
 TEST_GEN_PROGS_arm64 += arm64/page_fault_test
+TEST_GEN_PROGS_arm64 += arm64/pkvm_exceptions
 TEST_GEN_PROGS_arm64 += arm64/pkvm_host_abi
 TEST_GEN_PROGS_arm64 += arm64/pkvm_id_regs
 TEST_GEN_PROGS_arm64 += arm64/pkvm_irq
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_exceptions.c b/tools/testing/selftests/kvm/arm64/pkvm_exceptions.c
new file mode 100644
index 0000000000000..f7f9b666dd089
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_exceptions.c
@@ -0,0 +1,261 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_exceptions - debug exceptions and a virtual SError in a protected VM.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+/* Debug register programming bits, as debug-exceptions.c has them. */
+#define DBGBCR_LEN8	(0xff << 5)
+#define DBGBCR_EXEC	(0x0 << 3)
+#define DBGBCR_EL1	(0x1 << 1)
+#define DBGBCR_E	(0x1 << 0)
+
+#define DBGWCR_LEN8	(0xff << 5)
+#define DBGWCR_RD	(0x1 << 3)
+#define DBGWCR_WR	(0x2 << 3)
+#define DBGWCR_EL1	(0x1 << 1)
+#define DBGWCR_E	(0x1 << 0)
+
+#define SPSR_SS		(1 << 21)
+
+#define NR_STEPS	3
+
+enum { CMD_INJECT_SERROR, CMD_CHECK_SERROR };
+
+extern unsigned char hw_bp, ss_start;
+
+static volatile u64 ex_pc, ex_addr;
+static volatile u64 ss_addr[NR_STEPS + 1];
+static volatile int ss_idx;
+static volatile char write_data;
+static volatile bool serror_taken;
+
+static bool debug_exposed;
+
+/* Masks debug on return so the breakpointed instruction runs. */
+static void hw_bp_handler(struct ex_regs *regs)
+{
+	ex_pc = regs->pc;
+	regs->pstate |= PSR_D_BIT;
+}
+
+/* Records FAR, the accessed address, and masks debug on return so the access completes. */
+static void wp_handler(struct ex_regs *regs)
+{
+	ex_pc = regs->pc;
+	ex_addr = read_sysreg(far_el1);
+	regs->pstate |= PSR_D_BIT;
+}
+
+/* Records the stepped PC and sets SPSR.SS so that the next instruction is stepped too. */
+static void ss_handler(struct ex_regs *regs)
+{
+	__GUEST_ASSERT(ss_idx <= NR_STEPS, "step %d past the stepped range", ss_idx);
+	ss_addr[ss_idx++] = regs->pc;
+	regs->pstate |= SPSR_SS;
+}
+
+static void reset_debug_state(void)
+{
+	asm volatile("msr daifset, #8");
+
+	/* Clear the OS lock and double lock so debug exceptions are permitted. */
+	write_sysreg(0, osdlr_el1);
+	write_sysreg(0, oslar_el1);
+	isb();
+
+	write_sysreg(0, mdscr_el1);
+	write_sysreg(0, dbgbcr0_el1);
+	write_sysreg(0, dbgbvr0_el1);
+	write_sysreg(0, dbgwcr0_el1);
+	write_sysreg(0, dbgwvr0_el1);
+	isb();
+
+	ex_pc = 0;
+	ex_addr = 0;
+}
+
+/* A zero DebugVer means EL2 hides self-hosted debug, so the case has nothing to test. */
+static bool guest_debug_exposed(void)
+{
+	bool exposed = SYS_FIELD_GET(ID_AA64DFR0_EL1, DebugVer, read_sysreg(id_aa64dfr0_el1));
+
+	GUEST_SYNC(exposed);
+	return exposed;
+}
+
+static void enable_monitor_debug(void)
+{
+	asm volatile("msr daifclr, #8");
+	write_sysreg(read_sysreg(mdscr_el1) | MDSCR_EL1_KDE | MDSCR_EL1_MDE, mdscr_el1);
+	isb();
+}
+
+static void guest_hw_breakpoint(void)
+{
+	pkvm_guest_init();
+	if (!guest_debug_exposed())
+		GUEST_DONE();
+
+	reset_debug_state();
+	write_sysreg(DBGBCR_LEN8 | DBGBCR_EXEC | DBGBCR_EL1 | DBGBCR_E, dbgbcr0_el1);
+	write_sysreg((u64)&hw_bp, dbgbvr0_el1);
+	enable_monitor_debug();
+	asm volatile("hw_bp: nop");
+	GUEST_ASSERT_EQ(ex_pc, (u64)&hw_bp);
+
+	reset_debug_state();
+	GUEST_DONE();
+}
+
+static void guest_watchpoint(void)
+{
+	pkvm_guest_init();
+	if (!guest_debug_exposed())
+		GUEST_DONE();
+
+	reset_debug_state();
+	write_sysreg(DBGWCR_LEN8 | DBGWCR_RD | DBGWCR_WR | DBGWCR_EL1 | DBGWCR_E, dbgwcr0_el1);
+	/*
+	 * BAS = 0xff watches all eight bytes only from a doubleword-aligned DBGWVR.
+	 * FAR still holds the byte's address.
+	 */
+	write_sysreg((u64)&write_data & ~7UL, dbgwvr0_el1);
+	enable_monitor_debug();
+	write_data = 'x';
+	GUEST_ASSERT_EQ(write_data, 'x');
+	GUEST_ASSERT_EQ(ex_addr, (u64)&write_data);
+
+	reset_debug_state();
+	GUEST_DONE();
+}
+
+static void guest_single_step(void)
+{
+	pkvm_guest_init();
+	if (!guest_debug_exposed())
+		GUEST_DONE();
+
+	reset_debug_state();
+	write_sysreg(read_sysreg(mdscr_el1) | MDSCR_EL1_KDE | MDSCR_EL1_SS, mdscr_el1);
+	isb();
+	ss_idx = 0;
+	/* Unmasking makes the step active-pending, so the first one is taken at ss_start. */
+	asm volatile("msr daifclr, #8\n"
+		     "ss_start:\n"
+		     "mrs x0, esr_el1\n"
+		     "add x0, x0, #1\n"
+		     "msr daifset, #8\n"
+		     : : : "x0");
+	GUEST_ASSERT_EQ(ss_addr[0], (u64)&ss_start);
+	GUEST_ASSERT_EQ(ss_addr[1], (u64)&ss_start + 4);
+	GUEST_ASSERT_EQ(ss_addr[2], (u64)&ss_start + 8);
+
+	reset_debug_state();
+	GUEST_DONE();
+}
+
+static void serror_handler(struct ex_regs *regs)
+{
+	GUEST_ASSERT_EQ(ESR_ELx_EC(read_sysreg(esr_el1)), ESR_ELx_EC_SERROR);
+	serror_taken = true;
+}
+
+/*
+ * The guest sees the injected SError in ISR_EL1.A while PSTATE.A masks it, and
+ * takes it on unmasking. EL2 has to deliver it through HCR_EL2.VSE, because the
+ * host can't read a protected guest's PSTATE.A for serror_is_masked().
+ */
+static void guest_serror(void)
+{
+	pkvm_guest_init();
+
+	GUEST_SYNC(CMD_INJECT_SERROR);
+	GUEST_ASSERT(read_sysreg(isr_el1) & ISR_EL1_A);
+	GUEST_ASSERT(!serror_taken);
+
+	local_serror_enable();
+	isb();
+	local_serror_disable();
+
+	GUEST_ASSERT(serror_taken);
+	GUEST_SYNC(CMD_CHECK_SERROR);
+	GUEST_DONE();
+}
+
+static void serror_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	struct kvm_vcpu_events events = {};
+
+	switch (uc->args[1]) {
+	case CMD_INJECT_SERROR:
+		events.exception.serror_pending = true;
+		vcpu_events_set(vcpu, &events);
+		memset(&events, 0, sizeof(events));
+		vcpu_events_get(vcpu, &events);
+		TEST_ASSERT(events.exception.serror_pending, "vSError not pending after injection");
+		break;
+	case CMD_CHECK_SERROR:
+		vcpu_events_get(vcpu, &events);
+		TEST_ASSERT(!events.exception.serror_pending, "vSError still pending once taken");
+		break;
+	}
+}
+
+static void test_serror(void)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_serror);
+	vm_install_exception_handler(vm, VECTOR_ERROR_CURRENT, serror_handler);
+	pkvm_run_loop(vcpu, serror_sync);
+	kvm_vm_free(vm);
+}
+
+static void debug_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	debug_exposed = uc->args[1];
+}
+
+static void run_debug_guest(void *guest_code, int ec, handler_fn handler, const char *name)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm_install_sync_handler(vm, VECTOR_SYNC_CURRENT, ec, handler);
+	pkvm_run_loop(vcpu, debug_sync);
+	kvm_vm_free(vm);
+
+	if (debug_exposed)
+		ksft_test_result_pass("%s\n", name);
+	else
+		ksft_test_result_skip("%s: self-hosted debug not exposed to protected guests\n",
+				      name);
+}
+
+int main(void)
+{
+	TEST_REQUIRE(pkvm_supported());
+
+	ksft_print_header();
+	ksft_set_plan(4);
+
+	run_debug_guest(guest_hw_breakpoint, ESR_ELx_EC_BREAKPT_CUR, hw_bp_handler,
+			"hardware breakpoint");
+	run_debug_guest(guest_watchpoint, ESR_ELx_EC_WATCHPT_CUR, wp_handler, "watchpoint");
+	run_debug_guest(guest_single_step, ESR_ELx_EC_SOFTSTP_CUR, ss_handler, "single step");
+
+	test_serror();
+	ksft_test_result_pass("virtual SError taken on unmasking\n");
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 18/19] KVM: arm64: selftests: Add pkvm_sve vector-length test

[Fuad Tabba]

EL2 saves and restores a protected guest's SVE registers at the vector
length the host set before the first run. On every exit, it puts the
host's own vector length back.

Run a protected guest at each vector length KVM supports, and check
that it reads that length with RDVL, and that its predicates and FFR
survive exits. At the largest length, check Z, P and FFR across moves to
another host CPU. Check the host thread's vector length after every
exit. The cases skip when a protected VM can't have SVE.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 tools/testing/selftests/kvm/Makefile.kvm     |   1 +
 tools/testing/selftests/kvm/arm64/pkvm_sve.c | 357 +++++++++++++++++++
 2 files changed, 358 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/arm64/pkvm_sve.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 1045136aacc16..b25ed1834f197 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -199,6 +199,7 @@ TEST_GEN_PROGS_arm64 += arm64/pkvm_lifecycle
 TEST_GEN_PROGS_arm64 += arm64/pkvm_mem_share
 TEST_GEN_PROGS_arm64 += arm64/pkvm_psci
 TEST_GEN_PROGS_arm64 += arm64/pkvm_smccc
+TEST_GEN_PROGS_arm64 += arm64/pkvm_sve
 TEST_GEN_PROGS_arm64 += arm64/pkvm_vcpu_regs
 TEST_GEN_PROGS_arm64 += arm64/psci_test
 TEST_GEN_PROGS_arm64 += arm64/sea_to_user
diff --git a/tools/testing/selftests/kvm/arm64/pkvm_sve.c b/tools/testing/selftests/kvm/arm64/pkvm_sve.c
new file mode 100644
index 0000000000000..e34780aea4855
--- /dev/null
+++ b/tools/testing/selftests/kvm/arm64/pkvm_sve.c
@@ -0,0 +1,357 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * pkvm_sve - SVE in a protected VM: at each vector length KVM supports,
+ * RDVL and the guest's predicate and FFR state across exits, then its Z, P
+ * and FFR state across moves to another host CPU, and the host's vector
+ * length after each protected KVM_RUN.
+ *
+ * Copyright (C) 2026 Google LLC
+ * Author: Fuad Tabba <fuad.tabba@linux.dev>
+ */
+
+#include <errno.h>
+#include <sched.h>
+
+#include "kvm_util.h"
+#include "pkvm.h"
+#include "processor.h"
+#include "test_util.h"
+
+#define VQS_WORDS	KVM_ARM64_SVE_VLS_WORDS
+
+/* Accesses to this unbacked IPA exit to the VMM as MMIO. */
+#define MMIO_IPA	BIT_ULL(31)
+
+#define NR_MIGRATIONS	32
+
+/* The architecture's largest VL, 2048 bits. */
+#define VL_ARCH_MAX	__sve_vl_from_vq(16)
+
+/* VL in bytes, set in the guest's copy by the host. */
+static u64 guest_expected_vl;
+
+/* The host thread's VL before any KVM_RUN. */
+static unsigned long host_vl;
+
+/* Z0-Z31, P0-P15 and FFR at the guest's VL, as LDR and STR (vector) lay them out. */
+static u8 z_in[32 * VL_ARCH_MAX], z_out[32 * VL_ARCH_MAX];
+static u8 p_in[16 * VL_ARCH_MAX / 8], p_out[16 * VL_ARCH_MAX / 8];
+static u8 ffr_in[VL_ARCH_MAX / 8], ffr_out[VL_ARCH_MAX / 8];
+
+static cpu_set_t host_cpus;
+
+static bool vq_present(const u64 *vqs, unsigned int vq)
+{
+	unsigned int i = (vq - KVM_ARM64_SVE_VQ_MIN) / 64;
+	unsigned int b = (vq - KVM_ARM64_SVE_VQ_MIN) % 64;
+
+	return vqs[i] & BIT_ULL(b);
+}
+
+static void vq_set(u64 *vqs, unsigned int vq)
+{
+	unsigned int i = (vq - KVM_ARM64_SVE_VQ_MIN) / 64;
+	unsigned int b = (vq - KVM_ARM64_SVE_VQ_MIN) % 64;
+
+	vqs[i] |= BIT_ULL(b);
+}
+
+/* Enable SVE and FP at EL1 and request the largest VL, returning the one granted. */
+static u64 guest_enable_sve(void)
+{
+	u64 vl;
+
+	write_sysreg(read_sysreg(cpacr_el1) | CPACR_EL1_ZEN_EL1EN | CPACR_EL1_FPEN_EL1EN,
+		     cpacr_el1);
+	isb();
+
+	write_sysreg_s(0xf, SYS_ZCR_EL1);
+	isb();
+	asm volatile(".arch_extension sve\n rdvl %0, #1" : "=r"(vl));
+
+	return vl;
+}
+
+/*
+ * The predicates and FFR are VL-sized and, unlike Z, untouched by compiler or
+ * ucall SIMD, so a save or restore at the wrong VL shows up here.
+ */
+static void guest_sve_vl(void)
+{
+	u64 p_ok, np_ok, ffr_ok;
+	int i;
+
+	pkvm_guest_init();
+	GUEST_ASSERT_EQ(guest_enable_sve(), guest_expected_vl);
+
+	asm volatile(".arch_extension sve\n"
+		     "ptrue	p0.b\n"		/* all-true, spans the VL */
+		     "pfalse	p4.b\n"		/* all-false pattern */
+		     "setffr\n"			/* FFR all-true */
+		     ::: "p0", "p4", "memory");
+
+	/* Each ucall is a full exit and entry, saving and restoring the guest SVE state. */
+	for (i = 0; i < 3; i++)
+		GUEST_SYNC(i);
+
+	asm volatile(".arch_extension sve\n"
+		     "cntp	%[p], p0, p0.b\n"	/* == VL bytes if the all-true mask held */
+		     "cntp	%[np], p0, p4.b\n"	/* == 0 if the all-false mask held */
+		     "rdffr	p1.b\n"
+		     "cntp	%[f], p0, p1.b\n"	/* == VL bytes if FFR held */
+		     : [p] "=r"(p_ok), [np] "=r"(np_ok), [f] "=r"(ffr_ok)
+		     :: "p0", "p1");
+
+	GUEST_ASSERT_EQ(p_ok, guest_expected_vl);
+	GUEST_ASSERT_EQ(np_ok, 0);
+	GUEST_ASSERT_EQ(ffr_ok, guest_expected_vl);
+
+	GUEST_DONE();
+}
+
+#define P_REGS	"0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15"
+#define Z_REGS	P_REGS ", 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31"
+
+/* @op every Z or P register at its index times its size from @buf. */
+#define Z_OP(op, buf)	\
+	".irp num, " Z_REGS "\n" op "	z\\num, [%[" buf "], #\\num, mul vl]\n.endr\n"
+#define P_OP(op, buf)	\
+	".irp num, " P_REGS "\n" op "	p\\num, [%[" buf "], #\\num, mul vl]\n.endr\n"
+
+/* A predicate is VL / 8 bytes. */
+static void fill_sve_in(int iter, u64 vl)
+{
+	u64 *z = (u64 *)z_in, *p = (u64 *)p_in;
+	int i;
+
+	/* Different in every word and on every exit. */
+	for (i = 0; i < 32 * vl / sizeof(u64); i++)
+		z[i] = 0x0101010101010101UL * (i + 1) ^ (u64)iter << 40;
+	for (i = 0; i < 16 * vl / 8 / sizeof(u64); i++)
+		p[i] = ~z[i];
+
+	/* WRFFR requires a run of active elements from the first, or FFR is UNPREDICTABLE. */
+	memset(ffr_in, 0, sizeof(ffr_in));
+	for (i = 0; i <= iter % vl; i++)
+		ffr_in[i / 8] |= BIT(i % 8);
+}
+
+static void check_sve_out(int iter, u64 vl)
+{
+	const u64 *z = (u64 *)z_out, *want = (u64 *)z_in;
+	int i;
+
+	for (i = 0; i < 32 * vl / sizeof(u64); i++)
+		__GUEST_ASSERT(z[i] == want[i], "migration %d: z%lu is 0x%lx, not 0x%lx",
+			       iter, i / (vl / sizeof(u64)), z[i], want[i]);
+	for (i = 0; i < 16 * vl / 8; i++)
+		__GUEST_ASSERT(p_out[i] == p_in[i], "migration %d: p%lu is 0x%x, not 0x%x",
+			       iter, i / (vl / 8), p_out[i], p_in[i]);
+	for (i = 0; i < vl / 8; i++)
+		__GUEST_ASSERT(ffr_out[i] == ffr_in[i], "migration %d: FFR is 0x%x, not 0x%x",
+			       iter, ffr_out[i], ffr_in[i]);
+}
+
+static void guest_sve_migrate(void)
+{
+	u64 vl;
+	int i;
+
+	pkvm_guest_init();
+	vl = guest_enable_sve();
+	GUEST_ASSERT_EQ(vl, guest_expected_vl);
+
+	for (i = 0; i < NR_MIGRATIONS; i++) {
+		fill_sve_in(i, vl);
+
+		asm volatile(".arch_extension sve\n"
+			     "ldr	p0, [%[ffr_in]]\n"
+			     "wrffr	p0.b\n"
+			     P_OP("ldr", "p_in") Z_OP("ldr", "z_in")
+			     "str	x0, [%[mmio]]\n"
+			     Z_OP("str", "z_out") P_OP("str", "p_out")
+			     "rdffr	p0.b\n"
+			     "str	p0, [%[ffr_out]]\n"
+			     : : [z_in] "r"(z_in), [z_out] "r"(z_out), [p_in] "r"(p_in),
+			       [p_out] "r"(p_out), [ffr_in] "r"(ffr_in), [ffr_out] "r"(ffr_out),
+			       [mmio] "r"(MMIO_IPA)
+			     : "v0", "v1", "v2", "v3", "v4", "v5", "v6", "v7",
+			       "v8", "v9", "v10", "v11", "v12", "v13", "v14", "v15",
+			       "v16", "v17", "v18", "v19", "v20", "v21", "v22", "v23",
+			       "v24", "v25", "v26", "v27", "v28", "v29", "v30", "v31",
+			       "p0", "p1", "p2", "p3", "p4", "p5", "p6", "p7",
+			       "p8", "p9", "p10", "p11", "p12", "p13", "p14", "p15",
+			       "memory");
+
+		check_sve_out(i, vl);
+	}
+
+	GUEST_DONE();
+}
+
+/* The host thread's VL is unchanged by a protected run. */
+static void assert_host_vl(void)
+{
+	unsigned long vl;
+
+	asm volatile(".arch_extension sve\n rdvl %0, #1" : "=r"(vl));
+	TEST_ASSERT_EQ(vl, host_vl);
+}
+
+static void on_sync(struct kvm_vcpu *vcpu, struct ucall *uc)
+{
+	assert_host_vl();
+}
+
+static void set_vls_and_finalize(struct kvm_vcpu *vcpu, u64 *vqs)
+{
+	struct kvm_one_reg reg = {
+		.id = KVM_REG_ARM64_SVE_VLS,
+		.addr = (u64)vqs,
+	};
+	int feature = KVM_ARM_VCPU_SVE;
+
+	vcpu_ioctl(vcpu, KVM_SET_ONE_REG, &reg);
+	vcpu_ioctl(vcpu, KVM_ARM_VCPU_FINALIZE, &feature);
+}
+
+static void probe_available_vqs(u64 *vqs)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+
+	vm = pkvm_vm_create(1, BIT(KVM_ARM_VCPU_SVE), &vcpu);
+	TEST_ASSERT(!__vcpu_get_reg(vcpu, KVM_REG_ARM64_SVE_VLS, vqs),
+		    "KVM_GET_ONE_REG(SVE_VLS): errno %d", errno);
+	kvm_vm_free(vm);
+}
+
+/* Narrow the VM to max VQ @target and run the guest at that VL. */
+static void test_vl(unsigned int target, const u64 *avail)
+{
+	u64 vqs[VQS_WORDS] = {};
+	struct kvm_vcpu *vcpu;
+	struct kvm_vm *vm;
+	unsigned int vq;
+
+	/* The bitmap must match the host-available VQs up to the maximum. */
+	for (vq = KVM_ARM64_SVE_VQ_MIN; vq <= target; vq++)
+		if (vq_present(avail, vq))
+			vq_set(vqs, vq);
+
+	vm = pkvm_vm_create(1, BIT(KVM_ARM_VCPU_SVE), &vcpu);
+	set_vls_and_finalize(vcpu, vqs);
+	write_guest_global(vm, guest_expected_vl, __sve_vl_from_vq(target));
+	pkvm_vcpu_setup_boot(vcpu, guest_sve_vl);
+
+	pkvm_run_loop(vcpu, on_sync);
+	assert_host_vl();
+
+	kvm_vm_free(vm);
+}
+
+/* Move this thread, and with it the vCPU's next run, to another of its CPUs. */
+static void migrate_thread(void)
+{
+	int cpu = sched_getcpu();
+
+	do
+		cpu = (cpu + 1) % CPU_SETSIZE;
+	while (!CPU_ISSET(cpu, &host_cpus));
+
+	pin_self_to_cpu(cpu);
+	TEST_ASSERT_EQ(sched_getcpu(), cpu);
+}
+
+/* Run the VM at the largest VQ in @avail, moving it to another CPU at every MMIO exit. */
+static void test_migrate(unsigned int max_vq, u64 *avail)
+{
+	struct kvm_vcpu *vcpu;
+	struct kvm_run *run;
+	struct kvm_vm *vm;
+	struct ucall uc;
+
+	vm = pkvm_vm_create(1, BIT(KVM_ARM_VCPU_SVE), &vcpu);
+	set_vls_and_finalize(vcpu, avail);
+	write_guest_global(vm, guest_expected_vl, __sve_vl_from_vq(max_vq));
+	pkvm_vcpu_setup_boot(vcpu, guest_sve_migrate);
+	virt_map(vm, MMIO_IPA, MMIO_IPA, 1);
+
+	run = vcpu->run;
+	for (;;) {
+		vcpu_run(vcpu);
+		assert_host_vl();
+
+		if (run->exit_reason == KVM_EXIT_MMIO && run->mmio.phys_addr == MMIO_IPA) {
+			migrate_thread();
+			continue;
+		}
+
+		switch (get_ucall(vcpu, &uc)) {
+		case UCALL_DONE:
+			kvm_vm_free(vm);
+			return;
+		case UCALL_ABORT:
+			REPORT_GUEST_ASSERT(uc);
+			break;
+		default:
+			TEST_FAIL("Unexpected exit: %s", exit_reason_str(run->exit_reason));
+		}
+	}
+}
+
+int main(void)
+{
+	unsigned int vq, max_vq = 0, nr_vqs = 0;
+	u64 avail[VQS_WORDS] = {};
+	bool sve;
+
+	TEST_REQUIRE(pkvm_supported());
+
+	sve = pkvm_has_cap(KVM_CAP_ARM_SVE);
+	if (sve) {
+		probe_available_vqs(avail);
+		for (vq = KVM_ARM64_SVE_VQ_MIN; vq <= KVM_ARM64_SVE_VQ_MAX; vq++) {
+			if (!vq_present(avail, vq))
+				continue;
+			max_vq = vq;
+			nr_vqs++;
+		}
+		TEST_ASSERT(nr_vqs, "no SVE vector lengths advertised");
+	}
+
+	ksft_print_header();
+	ksft_set_plan(2 + (sve ? nr_vqs : 1));
+
+	if (!sve) {
+		const char *why = "SVE unavailable to a protected VM";
+
+		ksft_test_result_skip("vector lengths: %s\n", why);
+		ksft_test_result_skip("Z, P and FFR across host CPU migrations: %s\n", why);
+		ksft_test_result_skip("host VL preserved: %s\n", why);
+		ksft_finished();
+	}
+
+	/* The cap implies host SVE, so the host thread can read its own VL. */
+	asm volatile(".arch_extension sve\n rdvl %0, #1" : "=r"(host_vl));
+
+	for (vq = KVM_ARM64_SVE_VQ_MIN; vq <= KVM_ARM64_SVE_VQ_MAX; vq++) {
+		if (!vq_present(avail, vq))
+			continue;
+		test_vl(vq, avail);
+		ksft_test_result_pass("VQ %u: RDVL, predicates and FFR across exits\n", vq);
+	}
+
+	kvm_sched_getaffinity(0, sizeof(host_cpus), &host_cpus);
+	if (CPU_COUNT(&host_cpus) < 2) {
+		ksft_test_result_skip("Z, P and FFR across host CPU migrations: one CPU\n");
+	} else {
+		test_migrate(max_vq, avail);
+		kvm_sched_setaffinity(0, sizeof(host_cpus), &host_cpus);
+		ksft_test_result_pass("VQ %u: Z, P and FFR across host CPU migrations\n", max_vq);
+	}
+
+	ksft_test_result_pass("host VL preserved across protected runs\n");
+
+	ksft_finished();
+}
-- 
2.47.3

[PATCH v2 19/19] KVM: arm64: selftests: Check MOPS, PIE, POE and FPMR in a protected VM

[Fuad Tabba]

For each of MOPS, S1PIE, S1POE and FPMR that a protected VM's ID
registers advertise, EL2 has to set up the HCRX_EL2 enables, the
fine-grained traps and the context switch to match.

Check from the guest that each feature works when its ID field is set
and UNDEFs when it is clear. Working means the registers read back what
the guest writes, and a SETP/SETM/SETE fill completes. At v7.4-rc1 pKVM
advertises none of the four, so only the UNDEF cases run. Also check
that ID_AA64PFR2_EL1, ID_AA64ISAR3_EL1 and ID_AA64MMFR3_EL1 expose
nothing beyond the supported features, and that WFxT and the 128-bit
System register features stay hidden.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 .../selftests/kvm/arm64/pkvm_id_regs.c        | 70 +++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c b/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
index 93ed557a30bc6..7a8367af3e009 100644
--- a/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
+++ b/tools/testing/selftests/kvm/arm64/pkvm_id_regs.c
@@ -33,6 +33,11 @@
 #define DBGBCR_VAL	((0xfUL << 5) | (1UL << 1))
 #define DBGWCR_VAL	((0xffUL << 5) | (3UL << 3) | (1UL << 1))
 
+/* Allocated encodings only: a reserved one reads back UNKNOWN. */
+#define PIR_PATTERN	0xeca87653210eca87UL
+#define POR_PATTERN	0x76543210UL
+#define FPMR_PATTERN	0x2a5a55c000UL
+
 extern unsigned char brk_insn;
 
 static volatile bool undef_seen;
@@ -132,6 +137,20 @@ static void assert_id_regs(void)
 	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ISAR2_EL1) &
 			(ID_AA64ISAR2_EL1_APA3_MASK | ID_AA64ISAR2_EL1_GPA3_MASK), 0);
 
+	/* WFxT and the 128-bit System register and instruction features are hidden. */
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ISAR2_EL1) &
+			(ID_AA64ISAR2_EL1_WFxT_MASK | ID_AA64ISAR2_EL1_SYSREG_128_MASK |
+			 ID_AA64ISAR2_EL1_SYSINSTR_128_MASK), 0);
+
+	/* Of these registers, only the fields of the features pKVM supports can be set. */
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64PFR2_EL1) & ~ID_AA64PFR2_EL1_FPMR_MASK, 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64ISAR3_EL1) &
+			~(ID_AA64ISAR3_EL1_FPRCVT_MASK | ID_AA64ISAR3_EL1_LSFE_MASK |
+			  ID_AA64ISAR3_EL1_FAMINMAX_MASK | ID_AA64ISAR3_EL1_LSUI_MASK), 0);
+	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64MMFR3_EL1) &
+			~(ID_AA64MMFR3_EL1_TCRX_MASK | ID_AA64MMFR3_EL1_SCTLRX_MASK |
+			  ID_AA64MMFR3_EL1_S1PIE_MASK | ID_AA64MMFR3_EL1_S1POE_MASK), 0);
+
 	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64DFR1_EL1), 0);
 	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64AFR0_EL1), 0);
 	GUEST_ASSERT_EQ(read_sysreg_s(SYS_ID_AA64AFR1_EL1), 0);
@@ -217,6 +236,56 @@ static void assert_undef_regs(void)
 	GUEST_ASSERT_UNDEF(asm volatile(".arch_extension sve\n rdvl x0, #1" : : : "x0"));
 }
 
+/* setp [x0]!, x1!, x2; setm; sete */
+static void mops_set(void *dst, u64 len, u64 val)
+{
+	register void *d asm("x0") = dst;
+	register u64 n asm("x1") = len;
+	register u64 v asm("x2") = val;
+
+	asm volatile(".inst 0x19c20420\n\t.inst 0x19c24420\n\t.inst 0x19c28420"
+		     : "+r"(d), "+r"(n) : "r"(v) : "cc", "memory");
+}
+
+/* A feature's registers and instructions work when it's advertised and UNDEF when hidden. */
+static void assert_feature_access(void)
+{
+	u64 mmfr3 = read_sysreg_s(SYS_ID_AA64MMFR3_EL1);
+	u64 buf[4] = {};
+	int i;
+
+	if (SYS_FIELD_GET(ID_AA64MMFR3_EL1, S1PIE, mmfr3)) {
+		GUEST_ASSERT_RW(SYS_PIR_EL1, PIR_PATTERN);
+		GUEST_ASSERT_RW(SYS_PIRE0_EL1, PIR_PATTERN);
+	} else {
+		GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_PIR_EL1));
+		GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_PIRE0_EL1));
+	}
+
+	if (SYS_FIELD_GET(ID_AA64MMFR3_EL1, S1POE, mmfr3)) {
+		GUEST_ASSERT_RW(SYS_POR_EL1, POR_PATTERN);
+		GUEST_ASSERT_RW(SYS_POR_EL0, POR_PATTERN);
+	} else {
+		GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_POR_EL1));
+		GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_POR_EL0));
+	}
+
+	if (SYS_FIELD_GET(ID_AA64PFR2_EL1, FPMR, read_sysreg_s(SYS_ID_AA64PFR2_EL1)))
+		GUEST_ASSERT_RW(SYS_FPMR, FPMR_PATTERN);
+	else
+		GUEST_ASSERT_UNDEF((void)read_sysreg_s(SYS_FPMR));
+
+	if (SYS_FIELD_GET(ID_AA64ISAR2_EL1, MOPS, read_sysreg_s(SYS_ID_AA64ISAR2_EL1))) {
+		undef_seen = false;
+		mops_set(buf, sizeof(buf), 0x5a);
+		GUEST_ASSERT(!undef_seen);
+		for (i = 0; i < ARRAY_SIZE(buf); i++)
+			GUEST_ASSERT_EQ(buf[i], 0x5a5a5a5a5a5a5a5aUL);
+	} else {
+		GUEST_ASSERT_UNDEF(mops_set(buf, sizeof(buf), 0x5a));
+	}
+}
+
 static void assert_host_handled(void)
 {
 	/* Host-handled registers reach the guest with no UNDEF. */
@@ -236,6 +305,7 @@ static void guest_baseline(void)
 	else
 		assert_debug_raz_wi();
 	assert_undef_regs();
+	assert_feature_access();
 	assert_host_handled();
 
 	asm volatile("brk_insn: brk #0");
-- 
2.47.3