[PATCH] KVM: arm64: Fix host stage-2 finalization

[PATCH] KVM: arm64: Fix host stage-2 finalization

[Quentin Perret]

We currently walk the hypervisor stage-1 page-table towards the end of
hyp init in nVHE protected mode and adjust the host page ownership
attributes in its stage-2 in order to get a consistent state from both
point of views. The walk is done on the entire hyp VA space, and expects
to only ever find page-level mappings. While this expectation is
reasonable in the half of hyp VA space that maps memory with a fixed
offset (see the loop in pkvm_create_mappings_locked()), it can be
incorrect in the other half where nothing prevents the usage of block
mappings. For instance, on systems where memory is physically aligned at
an address that happens to maps to a PMD aligned VA in the hyp_vmemmap,
kvm_pgtable_hyp_map() will install block mappings when backing the
hyp_vmemmap, which will later cause finalize_host_mappings() to fail.
Furthermore, it should be noted that all pages backing the hyp_vmemmap
are also mapped in the 'fixed offset range' of the hypervisor, which
implies that finalize_host_mappings() will walk both aliases and update
the host stage-2 attributes twice. The order in which this happens is
unpredictable, though, since the hyp VA layout is highly dependent on
the position of the idmap page, hence resulting in a fragile mess at
best.

In order to fix all of this, let's restrict the finalization walk to
only cover memory regions in the 'fixed-offset range' of the hyp VA
space and nothing else. This not only fixes a correctness issue, but
will also result in a slighlty faster hyp initialization overall.

Fixes: 2c50166c62ba ("KVM: arm64: Mark host bss and rodata section as shared")
Signed-off-by: Quentin Perret <qperret@google.com>
---
 arch/arm64/kvm/hyp/nvhe/setup.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c
index 862c7b514e20..578f71798c2e 100644
--- a/arch/arm64/kvm/hyp/nvhe/setup.c
+++ b/arch/arm64/kvm/hyp/nvhe/setup.c
@@ -178,7 +178,7 @@ static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level,
 
 	phys = kvm_pte_to_phys(pte);
 	if (!addr_is_memory(phys))
-		return 0;
+		return -EINVAL;
 
 	/*
 	 * Adjust the host stage-2 mappings to match the ownership attributes
@@ -207,8 +207,18 @@ static int finalize_host_mappings(void)
 		.cb	= finalize_host_mappings_walker,
 		.flags	= KVM_PGTABLE_WALK_LEAF,
 	};
+	int i, ret;
+
+	for (i = 0; i < hyp_memblock_nr; i++) {
+		struct memblock_region *reg = &hyp_memory[i];
+		u64 start = (u64)hyp_phys_to_virt(reg->base);
+
+		ret = kvm_pgtable_walk(&pkvm_pgtable, start, reg->size, &walker);
+		if (ret)
+			return ret;
+	}
 
-	return kvm_pgtable_walk(&pkvm_pgtable, 0, BIT(pkvm_pgtable.ia_bits), &walker);
+	return 0;
 }
 
 void __noreturn __pkvm_init_finalise(void)
-- 
2.34.0.rc0.344.g81b53c2807-goog

Re: [PATCH] KVM: arm64: Fix host stage-2 finalization

[Marc Zyngier]

On Mon, 8 Nov 2021 15:46:32 +0000, Quentin Perret wrote:
> We currently walk the hypervisor stage-1 page-table towards the end of
> hyp init in nVHE protected mode and adjust the host page ownership
> attributes in its stage-2 in order to get a consistent state from both
> point of views. The walk is done on the entire hyp VA space, and expects
> to only ever find page-level mappings. While this expectation is
> reasonable in the half of hyp VA space that maps memory with a fixed
> offset (see the loop in pkvm_create_mappings_locked()), it can be
> incorrect in the other half where nothing prevents the usage of block
> mappings. For instance, on systems where memory is physically aligned at
> an address that happens to maps to a PMD aligned VA in the hyp_vmemmap,
> kvm_pgtable_hyp_map() will install block mappings when backing the
> hyp_vmemmap, which will later cause finalize_host_mappings() to fail.
> Furthermore, it should be noted that all pages backing the hyp_vmemmap
> are also mapped in the 'fixed offset range' of the hypervisor, which
> implies that finalize_host_mappings() will walk both aliases and update
> the host stage-2 attributes twice. The order in which this happens is
> unpredictable, though, since the hyp VA layout is highly dependent on
> the position of the idmap page, hence resulting in a fragile mess at
> best.
> 
> [...]

Applied to next, thanks!

[1/1] KVM: arm64: Fix host stage-2 finalization
      commit: 50a8d3315960c74095c59e204db44abd937d4b5d

Cheers,

	M.
-- 
Without deviation from the norm, progress is not possible.