From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E38237F8C3 for ; Wed, 26 Aug 2026 09:02:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.198 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787734930; cv=none; b=nFyoYHZpWAbE1911qxGi1hQnkAD5mh0jcAVRus8BfIE7fHXKXb5NqvxiMKicQS7bO1p5z3JGEJK9VAW21WgfvM0+V5PDWS3p+BfXs8WK2Hkc4N2gB92AzoVBWQUfjk8Lb/Ih0kXDEo2fq54fkDo7/yQ7nT8566a3fvZY5YcPyiY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787734930; c=relaxed/simple; bh=M+i2SxvXQ8DSphNQ8ShtoJPE8ttDtKFUtUCnzxjFNwY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IgHM6uAKCy8Pz8g4o2E+lGd/4SvK7gs0qrZgMtCWH0hh3nnlB6aMjJP37i7NsIzCOMPoCZbc4Fyn3yQN56RmBx28IL/RCSuBubc9PxZs2+aKNoqndE5OClyHJCKSuMWYVEmSuFValvcphajFO/ZXNvDbuAMnOoc3H9ZRnet6uAk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ackerleytng.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ki6qHcy5; arc=none smtp.client-ip=209.85.215.198 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ackerleytng.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ki6qHcy5" Received: by mail-pg1-f198.google.com with SMTP id 41be03b00d2f7-cc1c5810451so701009a12.1 for ; Wed, 26 Aug 2026 02:02:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787734927; x=1788339727; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=EuS9O11vr193qDv9HWb04MHk5U9f4uvrgo/7abqfaWM=; b=ki6qHcy5sv+9OlMiVd5y7G1srwUw08k3MDmIca3r9dtpgS4QAGWdb4zN/Xr1StaHR/ ccLMYIwmH2jVp7dQoieMQHkb1WDkdhkM4Rt8tRALcKWtkSwz+wzBCFhhLIwykLfy+iu1 oxZYtIDVIGPzzjCmtXTnAoFx2ZxiIrmZ06OtRTxzz7iJYnae5eZtOXd8jmFykYJ6Q5In tfvIUoWzOEBgr0HJ1kedNNb43zqvhpVqHkk/weu1uyXzqdnsjq2BCGKmwP9bB1UVttbO FmwYcVllqWe29ZFUfzg/hNAaYpu8CalDkn9Q8nQTUQjogSlT+h0uIG6KJTnd8DdIwaeF 8MVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787734927; x=1788339727; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=EuS9O11vr193qDv9HWb04MHk5U9f4uvrgo/7abqfaWM=; b=PR7AJoQ6m4gsW+8VOvBZQWqoMS0qWY8PHT96YVv3/oAco91EqhipJTRkudaG8oEoIS nfPdH8dFCyyLV+E4eVXlnQfnCYt0uQxrlOgT0gPpVRCARqDwOIF+Q1qxwPNbY0H7Pb6/ vhSLv8vy+03DN6VtuQuRYSk6BlBCueo68RdFlBl33gVz+ffiCEpjQqeKYw1wrgxDKnGv fXGaG4uk1oGrb0xxU/Yr+lgBAdbDWlHgWlB/i9IcPAcgsKKQMV2bzPmI8mjnJbM7w6gL Ka+tKSQ2cav28FlSZoRldKRwPolB+82LeDaNaErHZ3W9Kg1fuYjnLPvYT4MZr0D6JdzT xIqA== X-Forwarded-Encrypted: i=1; AHgh+RpUHUoa7mOK4FxhhRfpHALrAGpXABmsb29AU5X4gN1QGJXQymAIgFQ/zmzhm0KkujJQXTD+VJw/kgArwms=@vger.kernel.org X-Gm-Message-State: AFuF++labrQ+cWmHOx39oBVkCeuHEMgEUxejzkWSvVncoUzGyehDNmW0 aBM/rkRTQAJfq5XtPNN0AYYnIzPYidHOVh5RprT2Fe9QpE1b2qJWN6wVsMj7iMCXoStCzlYy0fE u+DlCu83DDY2tutGxi2zqFhQWgg== X-Received: from plbmz15.prod.google.com ([2002:a17:903:350f:b0:2cc:6063:a658]) (user=ackerleytng job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:e70e:b0:38e:524:8797 with SMTP id 98e67ed59e1d1-3966d4503ccmr11876759a91.13.1787734925365; Wed, 26 Aug 2026 02:02:05 -0700 (PDT) Date: Wed, 26 Aug 2026 09:01:47 +0000 In-Reply-To: <20260826-gmem-no-return-page-v4-0-3bb9c1ddb4e3@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260826-gmem-no-return-page-v4-0-3bb9c1ddb4e3@google.com> X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Developer-Signature: v=1; a=ed25519-sha256; t=1787734920; l=3858; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=ifBAzXs8dlmanG/v8Acwb5QccjLKb55JHzRk/esPfm4=; b=kJMdcnrib7/3q/fmH14PIC5RislnUwwDuforS4mPsCFG6U4RlmpkwZgDuAO6oLG+VTfDnsS8p VAKOV4a6zA9BcErbmRbTF6QWV2iI5gPuSe8RTZJyLl6TjYx7ftseQ1s X-Mailer: b4 0.16.0 Message-ID: <20260826-gmem-no-return-page-v4-2-3bb9c1ddb4e3@google.com> Subject: [PATCH v4 2/5] KVM: SEV: Drop page refcount early during RMP fault handling From: Ackerley Tng To: Sean Christopherson , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Ashish Kalra , Michael Roth , Brijesh Singh , Marc Zyngier , Oliver Upton , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , David Hildenbrand , Yan Zhao , "Edgecombe, Rick P" , Vishal Annapurve , Fuad Tabba Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, Ackerley Tng Content-Type: text/plain; charset="utf-8" From: Sean Christopherson Rework KVM's handling of RMP faults to rely on MMU invalidation logic for safety, instead of the current approach of holding onto a folio reference until the RMP operations are complete. I.e. drop the reference gifted by guest_memfd immediately after getting the PFN, and instead do RMP updates under mmu_lock, after checking for relevant MMU invalidations. This will allow dropping guest_memfd's reference gifting entirely, which is ideally how KVM would operate for all "follow PFN" operations (GUP has many more complications, which is why KVM holds a reference across page faults *on top* of the standard MMU invalidation logic). Signed-off-by: Sean Christopherson Reviewed-by: Michael Roth Co-developed-by: Ackerley Tng Signed-off-by: Ackerley Tng --- arch/x86/kvm/svm/sev.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index e198469eb074c..0d027cff734cf 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -5019,6 +5019,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) struct kvm_memory_slot *slot; struct kvm *kvm = vcpu->kvm; int order, rmp_level, ret; + unsigned long mmu_seq; struct page *page; bool assigned; kvm_pfn_t pfn; @@ -5046,18 +5047,22 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) return; } + mmu_seq = kvm->mmu_invalidate_seq; + smp_rmb(); + ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order); if (ret) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n", gpa); return; } + kvm_release_page_unused(page); ret = snp_lookup_rmpentry(pfn, &assigned, &rmp_level); if (ret || !assigned) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no assigned RMP entry found for GPA 0x%llx PFN 0x%llx error %d\n", gpa, pfn, ret); - goto out_no_trace; + return; } /* @@ -5085,27 +5090,31 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) if (rmp_level == PG_LEVEL_4K) goto out; - ret = snp_rmptable_psmash(pfn); - if (ret) { - /* - * Look it up again. If it's 4K now then the PSMASH may have - * raced with another process and the issue has already resolved - * itself. If it's not assigned, then this must have raced with - * another process that made this page shared. - */ - if (!snp_lookup_rmpentry(pfn, &assigned, &rmp_level) && - ((assigned && rmp_level == PG_LEVEL_4K) || !assigned)) + scoped_guard(read_lock, &kvm->mmu_lock) { + if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) goto out; - pr_warn_ratelimited("SEV: Unable to split RMP entry for GPA 0x%llx PFN 0x%llx ret %d\n", - gpa, pfn, ret); + ret = snp_rmptable_psmash(pfn); + if (ret) { + /* + * Look it up again. If it's 4K now then the PSMASH may + * have raced with another process and the issue has + * already resolved itself. If it's not assigned, then + * this must have raced with another process that made + * this page shared. + */ + if (!snp_lookup_rmpentry(pfn, &assigned, &rmp_level) && + ((assigned && rmp_level == PG_LEVEL_4K) || !assigned)) + goto out; + + pr_warn_ratelimited("SEV: Unable to split RMP entry for GPA 0x%llx PFN 0x%llx ret %d\n", + gpa, pfn, ret); + } } kvm_zap_gfn_range(kvm, gfn, gfn + PTRS_PER_PMD); out: trace_kvm_rmp_fault(vcpu, gpa, pfn, error_code, rmp_level, ret); -out_no_trace: - kvm_release_page_unused(page); } static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) -- 2.55.0.887.g758fc8c411-goog