From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 676FB23B61B; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781829108; cv=none; b=TNy4O14qX7YEKMCUBLWN1wIQTKoOo3pM3+OMXQS813NTXgNuh9VR9szFJndMmy9y4BZJ5FrY4IXg/7z3q7zMHXrmaJ7zU6yZy64vuuL6e4ZBzy8iRMj6lRvtVEPpBXen2ni38h0tGysd+AHoti5VIkMwP49hCNe65zSjW8nc4AA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781829108; c=relaxed/simple; bh=EoGYSuCoSjGpdWmEhpLLk1Atpm/fDiS350q2C1aR+IY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=o2W2mNjNP9taDtN0eBH/U8lf3SO9TxgFIlzJowBUZMdp+vMF5to3f36uWzstfiqZRAxS9mN855UgUlK3aq9AmaTsdSWCRFEVvtaR+o2+opnz+wqwJjmLEEQHmHDkFM6TFlnqrqcb8UzTqWt9j8wEyLF5iKY9ZcV98H0gbKvLVpc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Qowp0iEB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Qowp0iEB" Received: by smtp.kernel.org (Postfix) with ESMTPS id 2AA48C2BCFC; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1781829108; bh=EoGYSuCoSjGpdWmEhpLLk1Atpm/fDiS350q2C1aR+IY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Qowp0iEBsQtvtPoIV/xqmwGnms1M0zWgjxe7RH4s0pTlzwKkCqEHyLqWyS3eRQ7JD m+Av7ydkK7BGXnjCxzPZNtlpy8RG0zddCXbmZJDslDPDFumyfVU0SPDYK3VEudL3nI 6bKrJnFAzCZWMyIAyCZSlqd5swg4iAg7D9onJXldl9LWZga4KATsOCtU1FsU3WiAhv TgEMSjgr5M/5pMBJvWD/mIkQQn8g4/H3R3lGPaDnkQI6/hP+EBi3+FKr+x/v3UNWSu V5az3SM7OjJDyyl9WjsXGD3espHBPSYcrvHB4I5l7zCO2Lp9QEwouwicSGhmerhd8R X5ArvEkpIJ6Gw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1911BCDB461; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Thu, 18 Jun 2026 17:31:52 -0700 Subject: [PATCH v8 15/46] KVM: guest_memfd: Call arch invalidate hooks on conversion Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260618-gmem-inplace-conversion-v8-15-9d2959357853@google.com> References: <20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com> In-Reply-To: <20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, tabba@google.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1781829104; l=2768; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=E0npEy6B1rbQQ4tFaJTBsuBD+qzlEIeINKdXRgy0S54=; b=8FxZDoe7Vx/8EB7MoL7nMRqVXQZWT4PVJS7nEbmyHF20sL4YMgZoBkDxeN/BlJRpM2huh9oKR 50v8fs38ztoAcPA7hx9HxbXWF/KgYJi6oT2NZf30ZeCl1HNXByz/yA4 X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng When memory in guest_memfd is converted from private to shared, the platform-specific state associated with the guest-private pages must be invalidated or cleaned up. Iterate over the folios in the affected range and call the kvm_arch_gmem_invalidate() hook for each PFN range. This allows architectures to perform necessary teardown, such as updating hardware metadata or encryption states, before the pages are transitioned to the shared state. Invoke this helper after indicating to KVM's mmu code that an invalidation is in progress to stop in-flight page faults from succeeding. Reviewed-by: Fuad Tabba Signed-off-by: Ackerley Tng --- virt/kvm/guest_memfd.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 433f79047b9d1..3c94442bc8131 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -607,6 +607,42 @@ static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start, return safe; } +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE +static void kvm_gmem_invalidate(struct inode *inode, pgoff_t start, pgoff_t end) +{ + struct folio_batch fbatch; + pgoff_t next = start; + int i; + + folio_batch_init(&fbatch); + while (filemap_get_folios(inode->i_mapping, &next, end - 1, &fbatch)) { + for (i = 0; i < folio_batch_count(&fbatch); ++i) { + struct folio *folio = fbatch.folios[i]; + pgoff_t start_index, end_index; + kvm_pfn_t start_pfn, end_pfn; + + start_index = max(start, folio->index); + end_index = min(end, folio_next_index(folio)); + /* + * end_index is either in folio or points to + * the first page of the next folio. Hence, + * all pages in range [start_index, end_index) + * are contiguous. + */ + start_pfn = folio_file_pfn(folio, start_index); + end_pfn = start_pfn + end_index - start_index; + + kvm_arch_gmem_invalidate(start_pfn, end_pfn); + } + + folio_batch_release(&fbatch); + cond_resched(); + } +} +#else +static void kvm_gmem_invalidate(struct inode *inode, pgoff_t start, pgoff_t end) {} +#endif + static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, size_t nr_pages, uint64_t attrs, pgoff_t *err_index) @@ -647,7 +683,12 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, */ kvm_gmem_invalidate_start(inode, start, end); + + if (!to_private) + kvm_gmem_invalidate(inode, start, end); + mas_store_prealloc(&mas, xa_mk_value(attrs)); + kvm_gmem_invalidate_end(inode, start, end); out: filemap_invalidate_unlock(mapping); -- 2.55.0.rc0.738.g0c8ab3ebcc-goog