From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1792425025; b=JhBowMAZo12J700lWEWj+fxXcpHZmx3xRsHuYmhLkB9bhMRhYYnHFomrRL5/XcNlyUWU +sLI6TQAlO3pV4f0NWiwlXDZpRqfHEPl3OOfRw2nNIEiTPehg1K+aIYFqtKSPYz+VUkSY lQ5LUBwkpZ8YFRYtZWy5EjLM48ANueuxStRlx+BDp5ae6GjumI5f5FoXCRjy0dR1ePEFm ikNxakF3+Rx6tf8tUaNAAjOc/ALuJB4dlKPW7aiQ5D/zDAFRDLIfR3yBuaC7m1VdOpJ2R XwArsE7cqKH4rpQ6tR2s8h+3SOJyGvXf8IS21Xh5c5meDvFwjm8bdmqpBGlfsyeTo+Q== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1792425025; h=Received:DKIM-Signature:Received:Received:X-ME-Sender:X-ME-Received: X-ME-Proxy-Cause:X-ME-Proxy:Feedback-ID:Received:From:To:Cc:Subject: Date:Message-ID:X-Mailer:In-Reply-To:References:MIME-Version: Content-Transfer-Encoding; bh=+5Nfzrde1GD9FfRh74qOmGM0CUmDocdXRs1qdcRag3k=; b=Vwt9VifGF1Kj+7/GNqbp8aZfzoc511kIJrYJXabHvL/rWR/oWFPTdFtDuKx2knTHd07B ioycfiIHZwJLcVjGyX946CCrZv3bhPwNZhyeHjV7hBskIeCTLtuB278iJjWYVdUC8knRE mri02qQ0CzWKSZFPJ+ELIAptaQmLMvyfw5xdd7iqNuF4qXGOwU2bAsbxnQVxkuio7Bsep YUMkIpP+EnRY0nuoE7FQw2BaclXF0P2oDXpMuH+EPcgxRHc5MlpjuW8Dg1YqcoV+uGnXQ /QhpLgCmB1NfScCIbDDwqrfrN1p/Lbu+C0tew6eG72w+V7mdJlB7ge2+FIZo9W7F3SA== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=ljvEE+u3; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43766607D342; Mon, 19 Oct 2026 15:50:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=ljvEE+u3 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55D361F000C6; Mon, 19 Oct 2026 15:50:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1792425025; bh=+5Nfzrde1GD9FfRh74qOmGM0CUmDocdXRs1qdcRag3k=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ljvEE+u3oeDjYDCjGhqAkTws6/4QJfEktpl7VIAzkm1wbv/mLvVPT7PJyAWLDq5Gt Dx8v1Zw9QfsqO7syFOlR4E6YWIoNdoSWU0CEKKtHX4PPP55zLI04atrDRCVajHT+CL U0tJRg4MDCiT9DcKIT5hlRuTHsgHPMa0pqQwpU+1ieemLEpvbjGXkbCBJOcquo+RKj M1G4CbV319nSBW1W/1kGoRI19m1eGKJlY09uVX5xEgZejFwiN3BOoyLHr6DfkdGrIn XvWXI1/CNypi4ePfPPquLRjRyIpeqEG4vTh1CMZE0hu6JLrzZL7dJJkvNsKD1AFPgK icz1lv5eA/kIw== Received: from phl-compute-05.internal (phl-compute-05.internal [10.202.2.45]) by mailfauth.ams.internal (Postfix) with ESMTP id 2DA0E1980047; Mon, 19 Oct 2026 11:50:17 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Mon, 19 Oct 2026 11:50:20 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTE+kjgISKw/D5xbtovMvE/5+MXw9OAGtDBfqR9Zk5dZNOqhNWGTcEGO5Lmn7qsaMJ P+2+K5eT4SfShSSEvxMo9+0NhMtUrbV4W6Uy0A3XpuEB672w8089GbqFaxEF0wWgdpl5Mt ZGUplA1E9XDXzNgkO2MrFgMfcNS4u61SieDRP0v46gjH0l2eZ2DL9zme+jYfCIiYlLA1YJ nvbn1HssXPAJFHKdL+OCiEp78jaTWZC2fJHjA4vCnfghQrymIXyUGAHupLMJJEm2bjt0nv kEsuBoXYnW/I4u0Q3SZjAxl07Y4ERxrcHOAeD/N6wYko4+jO11wXgs98kOmjLgJ2rMOJIZ dfeJ0xQ74xoR6DfShFlSEAqdM3aQWhYGU9iaTUYTePGvrkWjscCqx8LuiCBIrLxzWO5esr XIWYeR8T4YsFXzTV8r5ByFLLce3aWQVDs/AH8HBc2svQkWdXOwpi0Xzyr1iXxxy3YDQxuu d+vqpBfcAJrom7RgY/UM1LcB98A5Ic6xZOKqm1cV0yUmWcjUHVykTDmFvPPPI9Ab4nRoMb UX70np05MwHA6FfDOUprjmrpo5WFE5+GZaMR+0EHllstbAYlSw/gD9O0Orj1fDNJWYImOg jGxc6OHbvGHcn3reOVmwkwpuisL9nJUWe/8+U8Oz7SBLCzpL0Si1/Pkv3cNQ X-ME-Proxy: Feedback-ID: i10464835:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 19 Oct 2026 11:50:15 -0400 (EDT) From: "Kiryl Shutsemau (Meta)" To: Andrew Morton , Linus Torvalds , David Hildenbrand , Matthew Wilcox , Vlastimil Babka , Lorenzo Stoakes , Jan Kara , Hugh Dickins , Christian Brauner , Alexander Viro Cc: Miklos Szeredi , Miaohe Lin , Naoya Horiguchi , Oscar Salvador , "Liam R . Howlett" , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Kairui Song , Qi Zheng , Shakeel Butt , Axel Rasmussen , Yuanchu Xie , Wei Xu , Baoquan He , "Paul E . McKenney" , Pedro Falcato , Harry Yoo , Luis Chamberlain , Ilya Gladyshev , Jaegeuk Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, fuse-devel@lists.linux.dev, kernel-team@meta.com, "Kiryl Shutsemau (Meta)" Subject: [RFC PATCH 09/18] mm: have folio_mark_accessed() return whether the folio is hot Date: Mon, 19 Oct 2026 16:49:02 +0100 Message-ID: <20261019154913.1765974-10-kas@kernel.org> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20261019154913.1765974-1-kas@kernel.org> References: <20261019154913.1765974-1-kas@kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Linus Torvalds Each call of folio_mark_accessed() moves a folio one step up what the LRU records of its accesses. Once the folio is at the top, the call leaves it as it was: the folio is as hot as the LRU can tell. Return whether the call found the folio there. On the classic LRU that is active, referenced and not idle. On the multi-gen LRU it is PG_referenced, all the refs bits and PG_workingset, which lru_gen_inc_refs() now reports. An unevictable folio counts as hot once it is referenced, and on the multi-gen LRU from the first call: it stays in memory in any case. A dropbehind folio is not hot: it is dropped after the read. No caller uses the value yet. Preparation for reading the folios that it calls hot without a folio reference. Assisted-by: LLM Signed-off-by: Linus Torvalds [kas: shorter changelog] Signed-off-by: Kiryl Shutsemau (Meta) --- include/linux/swap.h | 2 +- mm/folio.c | 46 ++++++++++++++++++++++++++++++-------------- 2 files changed, 33 insertions(+), 15 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 5658a1634b85..67aa5271ae6c 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -295,7 +295,7 @@ extern unsigned long totalreserve_pages; /* linux/mm/folio.c */ void folio_add_lru(struct folio *folio); -void folio_mark_accessed(struct folio *folio); +bool folio_mark_accessed(struct folio *folio); void lru_add_drain_all(void); enum lru_cache_drained { diff --git a/mm/folio.c b/mm/folio.c index 50a6dbe55998..6144b9cf301b 100644 --- a/mm/folio.c +++ b/mm/folio.c @@ -352,28 +352,34 @@ static void __lru_cache_activate_folio(struct folio *folio) #ifdef CONFIG_LRU_GEN -static void lru_gen_inc_refs(struct folio *folio) +/* + * Return true if @folio is hot: unevictable, or its flags already held as + * many accesses as they can. + */ +static bool lru_gen_inc_refs(struct folio *folio) { unsigned long new_flags, old_flags = READ_ONCE(folio->flags.f); if (folio_test_unevictable(folio)) - return; + return true; /* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced(folio)) { set_mask_bits(&folio->flags.f, LRU_REFS_MASK, BIT(PG_referenced)); - return; + return false; } do { if ((old_flags & LRU_REFS_MASK) == LRU_REFS_MASK) { - if (!folio_test_workingset(folio)) - folio_set_workingset(folio); - return; + if (folio_test_workingset(folio)) + return true; + folio_set_workingset(folio); + return false; } new_flags = old_flags + BIT(LRU_REFS_PGOFF); } while (!try_cmpxchg(&folio->flags.f, &old_flags, new_flags)); + return false; } static bool lru_gen_clear_refs(struct folio *folio) @@ -396,8 +402,9 @@ static bool lru_gen_clear_refs(struct folio *folio) #else /* !CONFIG_LRU_GEN */ -static void lru_gen_inc_refs(struct folio *folio) +static bool lru_gen_inc_refs(struct folio *folio) { + return false; } static bool lru_gen_clear_refs(struct folio *folio) @@ -419,15 +426,20 @@ static bool lru_gen_clear_refs(struct folio *folio) * * When a newly allocated folio is not yet visible, so safe for non-atomic ops, * __folio_set_referenced() may be substituted for folio_mark_accessed(). + * + * Return: true if @folio is hot: it was already as accessed as the LRU + * records. On the classic LRU that is active or unevictable, referenced + * and not idle; on the multi-gen LRU, unevictable, or PG_referenced, all + * the refs bits and PG_workingset set. False for a dropbehind folio. */ -void folio_mark_accessed(struct folio *folio) +bool folio_mark_accessed(struct folio *folio) { + bool hot = false; + if (folio_test_dropbehind(folio)) - return; - if (lru_gen_enabled()) { - lru_gen_inc_refs(folio); - return; - } + return false; + if (lru_gen_enabled()) + return lru_gen_inc_refs(folio); if (!folio_test_referenced(folio)) { folio_set_referenced(folio); @@ -437,6 +449,7 @@ void folio_mark_accessed(struct folio *folio) * this list is never rotated or maintained, so marking an * unevictable page accessed has no effect. */ + hot = true; } else if (!folio_test_active(folio)) { /* * If the folio is on the LRU, queue it for activation via @@ -450,9 +463,14 @@ void folio_mark_accessed(struct folio *folio) __lru_cache_activate_folio(folio); folio_clear_referenced(folio); workingset_activation(folio); + } else { + hot = true; } - if (folio_test_idle(folio)) + if (folio_test_idle(folio)) { folio_clear_idle(folio); + hot = false; + } + return hot; } EXPORT_SYMBOL(folio_mark_accessed); -- 2.54.0