[PATCH] KVM: arm64: nv: Invalidate whole L1 blocks on TLBI RIPAS2E1

[PATCH] KVM: arm64: nv: Invalidate whole L1 blocks on TLBI RIPAS2E1

[Dan Carpenter]

The shadow stage-2 can map one of the L1's stage-2 blocks with several
smaller leaves.  This happens when the host backs the memory with
smaller pages, when the memslot is dirty logging, or when the L1 block
size falls between two host block sizes.

When the L1 invalidates part of such a block, handle_ripas2e1is() only
unmaps the requested range, so the shadow leaves for the rest of the
block stay valid.  A TLBI must remove every TLB entry for a block it
touches, including the parts outside the range.  The L1 does not need
to issue another TLBI before reusing the rest of the block, so the L2
can still reach those pages through the stale shadow entries.

Use compute_tlb_inval_range() in s2_mmu_unmap_range() to find the L1
block size, the same as the IPAS2E1 path does, and widen the range to
whole L1 blocks.  The other caller, handle_vmalls12e1is(), unmaps the
whole IPA space, so the widening does not change anything there.

This only affects nested virtualization: the host must be booted with
kvm-arm.mode=nested and the VM must be given a virtual EL2.  In such a
VM it breaks the isolation that the L1 hypervisor provides.  An L2 can
keep reading and writing pages that the L1 has taken back and reused,
either for itself or for another L2.  The host, other VMs and VMs
without nested virtualization are not affected, because the shadow
stage-2 only maps the L1 VM's own memory.

Fixes: 5d476ca57d7d ("KVM: arm64: nv: Add handling of range-based TLBI operations")
Cc: stable@vger.kernel.org
Signed-off-by: Dan Carpenter <dan@cryspen.com>
---
This is part of our formal analysis work at Pereq.  I'm not super
familiar with this code.  Claude wrote a self test that I used to
test, but it's crap and not worth preserving.

I also considered widening the range in handle_ripas2e1is().  We would
need to use the largest block size in that scenario and it seemed like
it might hurt performance.

 arch/arm64/kvm/sys_regs.c | 13 ++++++++++++++--
 1 file changed, 12 insertions(+), 1 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 3d3f4eb82fbd..e1a687dfeea7 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -4116,6 +4116,17 @@ union tlbi_info {
 static void s2_mmu_unmap_range(struct kvm_s2_mmu *mmu,
 			       const union tlbi_info *info)
 {
+	u64 start = info->range.start;
+	u64 end = start + info->range.size;
+
+	/*
+	 * An L1 stage-2 block may be held in the shadow S2 as several smaller
+	 * leaves, and R_TVTYQ requires the TLBI to invalidate all of them, so
+	 * widen each end of the range to the L1 block containing it.
+	 */
+	start = ALIGN_DOWN(start, compute_tlb_inval_range(mmu, start >> 12));
+	end = ALIGN(end, compute_tlb_inval_range(mmu, (end - 1) >> 12));
+
 	/*
 	 * The unmap operation is allowed to drop the MMU lock and block, which
 	 * means that @mmu could be used for a different context than the one
@@ -4139,7 +4150,7 @@ static void s2_mmu_unmap_range(struct kvm_s2_mmu *mmu,
 	 * the L1 needs to put its stage-2 in a consistent state before doing
 	 * the TLBI.
 	 */
-	kvm_stage2_unmap_range(mmu, info->range.start, info->range.size, true);
+	kvm_stage2_unmap_range(mmu, start, end - start, true);
 }
 
 static bool handle_vmalls12e1is(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
-- 
2.53.0