[PATCH 16/16] KVM: SVM: Make KVM_REQ_VMSA_PAGE_RELOAD per plane

[David Woodhouse <dwmw2@infradead.org>]

From: David Woodhouse <dwmw@amazon.co.uk>

An SNP guest with multiple VMPLs gives each plane's vCPU its own VMSA,
but KVM_REQ_VMSA_PAGE_RELOAD still treats the vCPU as having one. When
guest_memfd invalidates the page behind a higher VMPL's VMSA,
sev_gmem_invalidate_range() looks only at the plane-0 vCPUs and never
asks the twin to reload. And when a twin's own reload races with an
invalidation and re-raises the request, it lands in the shared common
requests and is consumed by whichever twin runs first. That reloads
*its* VMSA and leaves the target twin entering the guest with a stale
vmsa_pa.

Mark the request KVM_REQUEST_PER_PLANE, as
KVM_REQ_UPDATE_PROTECTED_GUEST_STATE already is, so it stays with the
twin it was raised on, and have the invalidation walk every twin of
each vCPU and target the one whose VMSA GPA falls in the range. The
kick still reaches whichever twin is running, because mode is shared
in the common.

UNTESTED on SNP: build-tested only, as I have no multi-VMPL SNP host.

Fixes the VMSA reload for higher VMPLs in "kvm: sev: Allow for VMPL
level specification in AP create".

Fixes: https://lore.kernel.org/all/20260608144252.351443-58-joro@8bytes.org/
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 arch/x86/include/asm/kvm_host.h |  3 ++-
 arch/x86/kvm/svm/sev.c          | 21 +++++++++++++++------
 2 files changed, 17 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c7471699e2af..a4912735ad3d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -129,7 +129,8 @@
 #define KVM_REQ_HV_TLB_FLUSH \
 	KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
 #define KVM_REQ_VMSA_PAGE_RELOAD \
-	KVM_ARCH_REQ_FLAGS(33, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)
+	KVM_ARCH_REQ_FLAGS(33, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | \
+			       KVM_REQUEST_PER_PLANE)
 #define KVM_REQ_UPDATE_PROTECTED_GUEST_STATE \
 	KVM_ARCH_REQ_FLAGS(34, KVM_REQUEST_WAIT | KVM_REQUEST_PER_PLANE)
 
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 32a4ba4ffdbb..c8a9f808dcc4 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -5682,7 +5682,13 @@ void sev_gmem_invalidate_range(struct kvm *kvm, struct kvm_gfn_range *range)
 		return;
 
 	kvm_for_each_vcpu(i, vcpu, kvm) {
+		struct kvm_vcpu *twin;
+		int p;
+
 		/*
+		 * Every plane's vCPU has its own VMSA, so check each twin and
+		 * target the request at the twin whose VMSA it is.
+		 *
 		 * Read snp_guest_vmsa_gpa without taking the vCPU's VMSA mutex
 		 * (or its generic mutex) as mmu_lock is held, i.e. this task
 		 * can't sleep.  The VMSA is invalidated outside of mmu_lock,
@@ -5691,12 +5697,15 @@ void sev_gmem_invalidate_range(struct kvm *kvm, struct kvm_gfn_range *range)
 		 * positive is benign, as a spurious request simply forces the
 		 * vCPU to re-establish its VMSA.
 		 */
-		gpa_t gpa = READ_ONCE(to_svm(vcpu)->sev_es.snp_guest_vmsa_gpa);
-
-		if (VALID_PAGE(gpa) &&
-		    gpa_to_gfn(gpa) >= range->start &&
-		    gpa_to_gfn(gpa) < range->end)
-			kvm_make_request_and_kick(KVM_REQ_VMSA_PAGE_RELOAD, vcpu);
+		vcpu_for_each_plane(vcpu->common, p, twin) {
+			gpa_t gpa = READ_ONCE(to_svm(twin)->sev_es.snp_guest_vmsa_gpa);
+
+			if (VALID_PAGE(gpa) &&
+			    gpa_to_gfn(gpa) >= range->start &&
+			    gpa_to_gfn(gpa) < range->end)
+				kvm_make_request_and_kick(KVM_REQ_VMSA_PAGE_RELOAD,
+							  twin);
+		}
 	}
 }
 
-- 
2.50.1