From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C09ED25A642; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781829108; cv=none; b=rUZxNXwnps5zFcl4+cq3q9p+Acy+W1lpxc3d+bcMMDJqCRPb+TVTXKh3sbSJdMBvKtdf7vuDmBki8H0f0UCbB5tBNIoyFlq2wyZ9arOrt50jbGLMRnr9J1ed5UUmjJn7vA6vvSE5oxtnBEXFPJTwCKWOOdBftN4MSqJw95UcNPI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781829108; c=relaxed/simple; bh=NuW8Yph2xxSUg+BxjzkwpGfcorlKR26w4MsHwkJqK4A=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tC9SYhZRPasOUbqFNd8HJtgIy7Md/9dWNXt/9/eNz8PEUkfXa0HMWt9fSiy7XNRUsht6jBuLFkKbtt42SKMNP3n0wqr31cqOdV3mL/wYz6NACFUgu7+jLcjBmshLfIenm/iWI8i/Lnl8/BZVayK8cxxRwMAyCVQEl6OiCCdDLdI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fd3QXQO3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fd3QXQO3" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9A8B6C4AF1B; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1781829108; bh=NuW8Yph2xxSUg+BxjzkwpGfcorlKR26w4MsHwkJqK4A=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=fd3QXQO3t7yCmzAzEcKMWitUdh/fdw/mslbuUmjI5yXCdAHOzCvya77nZEpcfvZ8k kUxO65PdWslQyBeLBMWPyRvEBwhxyIYxpz/GiRIc76SpzoRFI9CDeaVUhM0MjacFvY ayuWxnOwDYdkQeN7I3h0+lLTFctxIJzf7gOXsCrW5RQEGGKUb5p5lRGUPRe7NFvGEC AGoVwlWSmr8q8NCKaBVjugLHCYv684ky/Iyh44nMEaKdlqPyM01Tz+ktetn49vqlw+ fbWUrvQ+ppS7jjvWnEVWnsWflkDZJMRFOrID65jVI0mUVVoEwebReaS16NnKzhHWE2 k5y36p6Dc6GPA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E351CD98FC; Fri, 19 Jun 2026 00:31:48 +0000 (UTC) From: Ackerley Tng via B4 Relay Date: Thu, 18 Jun 2026 17:31:55 -0700 Subject: [PATCH v8 18/46] KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion safety check Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260618-gmem-inplace-conversion-v8-18-9d2959357853@google.com> References: <20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com> In-Reply-To: <20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com> To: aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, tabba@google.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, suzuki.poulose@arm.com, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , Vlastimil Babka , Baoquan He Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, Ackerley Tng X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1781829104; l=3092; i=ackerleytng@google.com; s=20260225; h=from:subject:message-id; bh=/O+oiH2JXm7A6g3+sauBBielmQgPvvVOEru8T+k3ktE=; b=lPbIyUOgWSuySRYSO0/c0omWv+bzADKlEWA9gmzmv96f4S30la/SRUoDkGlcCtD9VQdWpWUNh CS4q5+4AXTEAevp5x4mMAFNIn59TOdGkn3wVrNpYfvD9sy2/BHoKY/x X-Developer-Key: i=ackerleytng@google.com; a=ed25519; pk=sAZDYXdm6Iz8FHitpHeFlCMXwabodTm7p8/3/8xUxuU= X-Endpoint-Received: by B4 Relay for ackerleytng@google.com/20260225 with auth_id=649 X-Original-From: Ackerley Tng Reply-To: ackerleytng@google.com From: Ackerley Tng When checking if a guest_memfd folio is safe for conversion, its refcount is examined. A folio may be present in a per-CPU lru_add fbatch, which temporarily increases its refcount. This can lead to a false positive, incorrectly indicating that the folio is in use and preventing the conversion, even if it is otherwise safe. The conversion process might not be on the same CPU that holds the folio in its fbatch, making a simple per-CPU check insufficient. To address this, drain all CPUs' lru_add fbatches if an unexpectedly high refcount is encountered during the safety check. This is performed at most once per conversion request. Draining only if the folio in question may be lru cached. guest_memfd folios are unevictable, so they can only reside in the lru_add fbatch. If the folio's refcount is still unsafe after draining, then the conversion is truly deemed unsafe. Reviewed-by: Fuad Tabba Signed-off-by: Ackerley Tng --- mm/swap.c | 2 ++ virt/kvm/guest_memfd.c | 18 ++++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 5cc44f0de9877..3134d9d3d7c30 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -37,6 +37,7 @@ #include #include #include +#include #include "internal.h" @@ -904,6 +905,7 @@ void lru_add_drain_all(void) lru_add_drain(); } #endif /* CONFIG_SMP */ +EXPORT_SYMBOL_FOR_KVM(lru_add_drain_all); atomic_t lru_disable_count = ATOMIC_INIT(0); diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index cec8fa26ece17..d163559da0235 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -8,6 +8,7 @@ #include #include #include +#include #include "kvm_mm.h" @@ -597,6 +598,7 @@ static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start, const int filemap_get_folios_refcount = 1; pgoff_t last = start + nr_pages - 1; struct folio_batch fbatch; + bool lru_drained = false; bool safe = true; pgoff_t next; int i; @@ -606,12 +608,20 @@ static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start, next = start; while (safe && filemap_get_folios(mapping, &next, last, &fbatch)) { - for (i = 0; i < folio_batch_count(&fbatch); ++i) { + for (i = 0; i < folio_batch_count(&fbatch);) { struct folio *folio = fbatch.folios[i]; - if (folio_ref_count(folio) != - folio_nr_pages(folio) + filemap_get_folios_refcount) { - safe = false; + safe = (folio_ref_count(folio) == + folio_nr_pages(folio) + + filemap_get_folios_refcount); + + if (safe) { + ++i; + } else if (folio_may_be_lru_cached(folio) && + !lru_drained) { + lru_add_drain_all(); + lru_drained = true; + } else { *err_index = max(start, folio->index); break; } -- 2.55.0.rc0.738.g0c8ab3ebcc-goog