[PATCH v2 0/3] KVM: arm64: Honour KVM_VM_TYPE_ARM_IPA_SIZE under pKVM

[PATCH v2 0/3] KVM: arm64: Honour KVM_VM_TYPE_ARM_IPA_SIZE under pKVM

[Fuad Tabba]

Hi folks,

Changes since v1 [1]:
  - Dropped v1's kvm_get_ipa_max() patch. EL2 checks the requested IPA
    size against a copy of the host's limit taken at init.
  - Added patch 2, which aligns the PGD of EL2's selftest VM.
  - Rebased on 7.4-rc1.

pKVM ignores KVM_VM_TYPE_ARM_IPA_SIZE: every VM gets the host's IPA
limit, and an out-of-range request is accepted. So the vgic_init
selftest fails on any pKVM host whose limit exceeds 40 bits.

With patch 3, a VM gets the IPA size its VMM requested, and EL2 rejects
one the host itself would reject.

Patch 1 rejects a PGD donation that isn't aligned to its size, which
crashes EL2 today. It's first because patch 3 lets the host choose
that size. Patch 2 aligns the PGD of EL2's own selftest VM under
CONFIG_NVHE_EL2_DEBUG.

A VMM that requests less than the host limit now gets that size. One
passing the default type gets 40 bits. On a host whose limit is below
40 bits the default type is rejected, as it is without pKVM. kvmtool
and QEMU request the size their memory map spans, and crosvm the host
limit, so none of them places anything beyond its request.

On QEMU, with 7.4-rc1 as a 64K/PA_BITS_48 pKVM host on a 52-bit part,
KVM_RUN fails on a 52-bit VM's first access to a memslot above 2^48.
With patch 3 the guest writes through it. The same kernel, with
CONFIG_NVHE_EL2_DEBUG, panics at pKVM init on cortex-a57, which patch 2
fixes.

On an M4, whose limit is 40 bits, pKVM now rejects a 44-bit request, as
the M4 does without pKVM. A protected kvmtool guest there boots with the
32 bits it requests.

Based on Linux 7.4-rc1 (6628e7d4c6b25).

Cheers,
/fuad

[1] https://lore.kernel.org/all/20260917091826.2639205-1-fuad.tabba@linux.dev/

Fuad Tabba (2):
  KVM: arm64: Align the pKVM selftest VM's stage-2 PGD
  KVM: arm64: Honour the requested IPA size under pKVM

Quentin Perret (1):
  KVM: arm64: Check PGD alignment when creating a pVM

 arch/arm64/include/asm/kvm_hyp.h      |  1 +
 arch/arm64/kvm/arm.c                  |  1 +
 arch/arm64/kvm/hyp/nvhe/mem_protect.c |  2 +-
 arch/arm64/kvm/hyp/nvhe/pkvm.c        | 34 +++++++++++++++++++++------
 arch/arm64/kvm/mmu.c                  |  4 +---
 5 files changed, 31 insertions(+), 11 deletions(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.47.3

[PATCH v2 1/3] KVM: arm64: Check PGD alignment when creating a pVM

[Fuad Tabba]

From: Quentin Perret <qperret@google.com>

Martijn reported a hypervisor crash when providing pKVM with an
undersized PGD allocation. Indeed, although the size of the PGD
allocation at EL2 is not under host control, a smaller host-side
allocation can lead to providing pKVM with a misaligned PGD, which will
cause the guest stage-2 init to fail in a bad way. Specifically,
guest_s2_zalloc_pages_exact() expects a successful allocation from
hyp_alloc_pages(), which can only happen if the pool has been pre-filled
with a physically aligned high-order page.

In order to guarantee allocation success in this path, check the
host-provided PGD alignment early on.

Fixes: 60dfe093ec13 ("KVM: arm64: Instantiate guest stage-2 page-tables at EL2")
Reported-by: Martijn Bogaard <martijnbogaard@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
[Fuad: Returned -EINVAL and moved the check ahead of hyp_alloc(), with
 the other input checks in __pkvm_init_vm(). Cast pgd_hva for its __kern
 type, updated the comment, and pointed Fixes: at the commit that
 initialises the guest's pool from the PGD.]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/hyp/nvhe/mem_protect.c | 2 +-
 arch/arm64/kvm/hyp/nvhe/pkvm.c        | 8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
index d4c332a918035..5b624b1ab32e8 100644
--- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c
+++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c
@@ -186,7 +186,7 @@ static void *guest_s2_zalloc_pages_exact(size_t size)
 {
 	void *addr = hyp_alloc_pages(&current_vm->pool, get_order(size));
 
-	WARN_ON(size != (PAGE_SIZE << get_order(size)));
+	WARN_ON(!addr || size != (PAGE_SIZE << get_order(size)));
 	hyp_split_page(hyp_virt_to_page(addr));
 
 	return addr;
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index fc0475f0877d1..03e530e15b4a2 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -892,8 +892,8 @@ void teardown_selftest_vm(void)
  *
  * host_kvm: A pointer to the host's struct kvm.
  * pgd_hva: The host va of the area being donated for the stage-2 PGD for
- *	    the VM. Must be page aligned. Its size is implied by the VM's
- *	    VTCR.
+ *	    the VM. Must be aligned to its size, which is implied by the
+ *	    VM's VTCR.
  *
  * Return 0 success, negative error code on failure.
  */
@@ -924,6 +924,10 @@ int __pkvm_init_vm(struct kvm *host_kvm, void __kern *pgd_hva)
 
 	vm_size = pkvm_get_hyp_vm_size(nr_vcpus);
 	pgd_size = kvm_pgtable_stage2_pgd_size(host_mmu.arch.mmu.vtcr);
+	if (!IS_ALIGNED((unsigned long)pgd_hva, pgd_size)) {
+		ret = -EINVAL;
+		goto err_unpin_kvm;
+	}
 
 	hyp_vm = hyp_alloc(vm_size);
 	if (!hyp_vm) {
-- 
2.47.3

[PATCH v2 2/3] KVM: arm64: Align the pKVM selftest VM's stage-2 PGD

[Fuad Tabba]

With CONFIG_NVHE_EL2_DEBUG, pKVM init sets up a selftest VM whose PGD is
the start of its 32-page block. That block is the first allocation out
of the hyp reserve, which falls back to page alignment when its
PMD-aligned allocation fails (512M on a 64K host).

A PGD of concatenated tables must be naturally aligned, or the VM's pool
never coalesces it into a block hyp_alloc_pages() can return. So when
the reserve isn't aligned to the PGD's size, the allocation in
guest_s2_zalloc_pages_exact() fails and pKVM init panics. A 64K host
with a 44-bit PA has a four-page PGD, and on a 1G QEMU VM the reserve
landed at 0x7fae0000.

Take the aligned pages inside the block instead. The pages either side
of it go to the memcache and the pool. The PGD is at most 16 pages, so
it always fits.

Fixes: 48d5645072930 ("KVM: arm64: Extend pKVM selftest for np-guests")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/hyp/nvhe/pkvm.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 03e530e15b4a2..820baf9ff74a2 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -847,10 +847,14 @@ struct pkvm_hyp_vcpu *init_selftest_vm(void *virt)
 {
 	struct hyp_page *p = hyp_virt_to_page(virt);
 	unsigned long min_pages, seeded = 0;
+	void *pgd;
 	int i;
 
 	selftest_vm.kvm.arch.mmu.vtcr = host_mmu.arch.mmu.vtcr;
-	WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, virt));
+
+	/* A concatenated PGD must be naturally aligned for the pool to coalesce it. */
+	pgd = PTR_ALIGN(virt, kvm_pgtable_stage2_pgd_size(selftest_vm.kvm.arch.mmu.vtcr));
+	WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, pgd));
 
 	/*
 	 * Mirror pkvm_refill_memcache() for the share/donate pre-checks;
-- 
2.47.3

[PATCH v2 3/3] KVM: arm64: Honour the requested IPA size under pKVM

[Fuad Tabba]

pKVM gives every VM the host's IPA limit, whatever size the VMM
requested with KVM_VM_TYPE_ARM_IPA_SIZE, so the vgic_init selftest fails
on any pKVM host whose limit exceeds 40 bits: the addresses it expects a
40-bit VM to reject are in range. EL2 also sizes the guest's stage 2
from the host stage 2's VTCR, which on a 64K kernel built for 48-bit PAs
on a 52-bit part is 48 bits where the limit is 52. KVM_RUN then returns
-ERANGE on the guest's first access to a memslot above 2^48 that the
host accepted.

Have EL2 take the IPA size from the VM's VTCR, the one the host sized
the donated PGD from, and drop the host-side override, so the request
is checked and honoured as on any other host. A VMM passing the default
type then gets 40 bits, and -EINVAL on a host whose limit is below 40,
as without pKVM. EL2 rejects a size below 32 bits or beyond the host's
IPA limit, which kvm_hyp_init_symbols() copies to EL2 before the host
drops its privileges.

Fixes: 60dfe093ec13 ("KVM: arm64: Instantiate guest stage-2 page-tables at EL2")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_hyp.h |  1 +
 arch/arm64/kvm/arm.c             |  1 +
 arch/arm64/kvm/hyp/nvhe/pkvm.c   | 20 ++++++++++++++++----
 arch/arm64/kvm/mmu.c             |  4 +---
 4 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index b0e42be3d9eef..8cffdf614af46 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -154,6 +154,7 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val);
 
 extern unsigned long kvm_nvhe_sym(__icache_flags);
 extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits);
+extern u32 kvm_nvhe_sym(kvm_ipa_limit);
 extern enum mitigation_state kvm_nvhe_sym(spectre_v2_state);
 extern enum mitigation_state kvm_nvhe_sym(spectre_v4_state);
 extern enum mitigation_state kvm_nvhe_sym(spectre_bhb_state);
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 157bf4b812020..81262193ffaca 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -2836,6 +2836,7 @@ static void kvm_hyp_init_symbols(void)
 	kvm_nvhe_sym(id_aa64smfr0_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64SMFR0_EL1);
 	kvm_nvhe_sym(__icache_flags) = __icache_flags;
 	kvm_nvhe_sym(kvm_arm_vmid_bits) = kvm_arm_vmid_bits;
+	kvm_nvhe_sym(kvm_ipa_limit) = get_kvm_ipa_limit();
 
 	kvm_nvhe_sym(spectre_v2_state) = arm64_get_spectre_v2_state();
 	kvm_nvhe_sym(spectre_v4_state) = arm64_get_spectre_v4_state();
diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c
index 820baf9ff74a2..0cbe15b15d08b 100644
--- a/arch/arm64/kvm/hyp/nvhe/pkvm.c
+++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c
@@ -30,6 +30,8 @@ unsigned long __icache_flags;
 /* Used by kvm_get_vttbr(). */
 unsigned int kvm_arm_vmid_bits;
 
+u32 kvm_ipa_limit;
+
 unsigned int kvm_host_sve_max_vl;
 
 /*
@@ -505,7 +507,8 @@ static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_vcpus[],
 }
 
 static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
-			     unsigned int nr_vcpus, pkvm_handle_t handle)
+			     unsigned int nr_vcpus, pkvm_handle_t handle,
+			     u64 vtcr)
 {
 	struct kvm_s2_mmu *mmu = &hyp_vm->kvm.arch.mmu;
 	int idx = vm_handle_to_idx(handle);
@@ -525,7 +528,7 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp_vm,
 	/* VMID 0 is reserved for the host */
 	atomic64_set(&mmu->vmid.id, idx + 1);
 
-	mmu->vtcr = host_mmu.arch.mmu.vtcr;
+	mmu->vtcr = vtcr;
 	mmu->arch = &hyp_vm->kvm.arch;
 	mmu->pgt = &hyp_vm->pgt;
 }
@@ -908,6 +911,8 @@ int __pkvm_init_vm(struct kvm *host_kvm, void __kern *pgd_hva)
 	unsigned int nr_vcpus;
 	pkvm_handle_t handle;
 	void *pgd = NULL;
+	u32 phys_shift;
+	u64 vtcr;
 	int ret;
 
 	ret = hyp_pin_shared_mem(host_kvm, host_kvm + 1);
@@ -926,8 +931,15 @@ int __pkvm_init_vm(struct kvm *host_kvm, void __kern *pgd_hva)
 		goto err_unpin_kvm;
 	}
 
+	phys_shift = VTCR_EL2_IPA(READ_ONCE(host_kvm->arch.mmu.vtcr));
+	if (phys_shift < ARM64_MIN_PARANGE_BITS || phys_shift > kvm_ipa_limit) {
+		ret = -EINVAL;
+		goto err_unpin_kvm;
+	}
+
+	vtcr = kvm_get_vtcr(id_aa64mmfr0_el1_sys_val, id_aa64mmfr1_el1_sys_val, phys_shift);
 	vm_size = pkvm_get_hyp_vm_size(nr_vcpus);
-	pgd_size = kvm_pgtable_stage2_pgd_size(host_mmu.arch.mmu.vtcr);
+	pgd_size = kvm_pgtable_stage2_pgd_size(vtcr);
 	if (!IS_ALIGNED((unsigned long)pgd_hva, pgd_size)) {
 		ret = -EINVAL;
 		goto err_unpin_kvm;
@@ -945,7 +957,7 @@ int __pkvm_init_vm(struct kvm *host_kvm, void __kern *pgd_hva)
 	if (!pgd)
 		goto err_free_hyp_vm;
 
-	init_pkvm_hyp_vm(host_kvm, hyp_vm, nr_vcpus, handle);
+	init_pkvm_hyp_vm(host_kvm, hyp_vm, nr_vcpus, handle, vtcr);
 
 	ret = kvm_guest_prepare_stage2(hyp_vm, pgd);
 	if (ret)
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1db10d8d6db73..861d10983d0d0 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -972,9 +972,7 @@ static int kvm_init_ipa_range(struct kvm_s2_mmu *mmu, unsigned long type)
 	u32 phys_shift;
 
 	phys_shift = KVM_VM_TYPE_ARM_IPA_SIZE(type);
-	if (is_protected_kvm_enabled()) {
-		phys_shift = kvm_ipa_limit;
-	} else if (phys_shift) {
+	if (phys_shift) {
 		if (phys_shift > kvm_ipa_limit ||
 		    phys_shift < ARM64_MIN_PARANGE_BITS)
 			return -EINVAL;
-- 
2.47.3