From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=pass header.oldest-pass=0 smtp.remote-ip=100.103.45.18 ARC-Seal: i=2; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=pass; t=1795035834; b=HBmMUYHeV5D0g4XChD/pNygVFDAriSVHnxZDW3FhD/mBGk4jweSsrMhVcgY1A7uoXU9J zX1OS8XHSSNhEcraLkHWGdq5CTdJRd+7wc9m5nZlmc+UJmASr00AosGn4BV/0FVVO4VUU 8YEB0y51CJCkEqJF4mST0aXwBTGaaJm1NGJFWABoe7D++gPb/Rzv/py01i3wHEqxN4rpy cjfY6kUC5qmdf5mJjhr/qiP/SdYyAhL/YdnrG7f3zczhX0egCKUatp1k2RHtYQW1heqwk cXoZAAt3yvk0uPCiFfMhErmXDdV6bszkDcNJvTyY0yaVqMbbjzGAPdYN/K7iesl53eA== ARC-Message-Signature: i=2; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795035834; h=Received:DKIM-Signature:Received:From:Date:Subject:MIME-Version: Content-Type:Message-Id:References:In-Reply-To:To:Cc:X-Mailer: X-Developer-Signature:X-Developer-Key:X-Endpoint-Received: X-Original-From:Reply-To:Content-Transfer-Encoding; bh=IzsbJHeLt6AX6DClQU1Tp3h6tzemqX+s4gHhI4j9FYs=; b=Cm6xrjV+3sf08BRqVMKb8N2IRr25tO02tbOb0BnrdF7LZ2zCt90kJE0wqyGM8YGixvk6 z6IIqYmR8uwI24L91v4HSmVUeqbJKTKbF6P3xHNH6my5o5enwR82dSchwoNeKGgOIFC7z G+BJnegyZzo4iBYqBTABnEMRse7JfAe/Q8oS14D1l+kpS6sUgxwTLOv5K1YAi1yccc7mJ PppV7Ue06q2BSK275lvykDEmy7iStqisgqylUMm3Lqp9uaZwhT+VYlBcLC8/gu26M8B8X mFWSGmCM0Ps9uBPtSlRxq+55OF2v7aJ42ATqA0FBNIb0ZNhBBJZd8bbntovHXSA7MGw== ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=fTwF4mZ2; arc=pass header.oldest-pass=0 smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20AFF60A261F for ; Wed, 18 Nov 2026 21:03:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=fTwF4mZ2 Authentication-Results: smtp.kernel.org; arc=none smtp.remote-ip=100.80.115.77 ARC-Seal: i=1; d=kernel.org; s=arc20260519; a=rsa-sha256; cv=none; t=1795035834; b=TgiaFScJYbX4l67tOy0J5jTUoiHkElwP9NRG7nnVsdIWeIFFDb94ect94+iv7akRBT2O 7FROT/8mle8ttJGH/tMGfZHUfb/6ohx1LYXFFlITr711g9O4HqxtC19PcsfvyzzQ9Efhd Gb78LR/k9BgzKAO5S09Cn2u91t6kCkR3jNPL4XgrM5nDky+cryKLoK7pf7N6TqeZs5tBl rsmXkiyAlrH7PrBz+V1FGHTxzweHyLsk/rvXXyXfYA+HuyJMAA/+Uha/awPtZfL1DUN/D iwBsAvnWImKQQ2pdPBmf2T5IkU4Igi0UgGGwQU37g4afud01X0bhveyMwLnzyIrwMCw== ARC-Message-Signature: i=1; d=kernel.org; s=arc20260519; a=rsa-sha256; c=relaxed/relaxed; t=1795035834; h=DKIM-Signature:Received:From:Date:Subject:MIME-Version:Content-Type: Message-Id:References:In-Reply-To:To:Cc:X-Mailer: X-Developer-Signature:X-Developer-Key:X-Endpoint-Received: X-Original-From:Reply-To:Content-Transfer-Encoding; bh=IzsbJHeLt6AX6DClQU1Tp3h6tzemqX+s4gHhI4j9FYs=; b=DWCXpQrw4BtbU2urQJJtVrSdAHxS0MmyDC1FXHyZehbz2cPo/3Ym5xlimlHBlN1qA11v ghYRjAvKNFlE+WeFH43sUxxRJBxm7lUAhOFfCSGeOzHPTv/TIVHzrn7hVNa5toaz9zhoG bDnuVZihlthmY+k3aNi2faFNTtvWfR0c5AdDRmDaaPjepTlizvjc1ZQVG/2jF7Fq8KKE+ H9tkMjSOYvX/efYMvfvXVbefK2y2B1Q41i3tBnkmT7rQeiGWAbFh+lihL8ixc9OhpkZU0 GC5h8BRKvR245SifKb6vmoRrC13n8Lp3H21UeukwdI4z93QaPLjEXmkdEMMCWn9Cx7g== ARC-Authentication-Results: i=1; smtp.kernel.org; arc=none smtp.remote-ip=100.80.115.77 Received: from lkml.kernel.org (nexus.subspace.kernel.org [100.80.115.77]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.kernel.org (Postfix) with ESMTPS id 353EE1F000D1; Wed, 18 Nov 2026 21:03:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1795035833; bh=IzsbJHeLt6AX6DClQU1Tp3h6tzemqX+s4gHhI4j9FYs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To; b=fTwF4mZ2ntiCM6HOgmmENBnhFJG4Ogpk4lkeCNkSqSS6Jpm8UZXsPzLBfaDqSWuKg jLSshVGFyBF4UpBHrmTOUf39tmQ7SWnqHRlTw1cXrXMaTsuAUMvQugfusd/xg0XF6+ U4+5bsKZkZ4dwC/VMtwK+VbBrLDUGuwXxpeScPgIgv4yQyrBKF6UDz+lu2eAFuZygW 7w1iu2SdoUq+3IxGlF0JT7/Y6kdoAoI4T9HUr2Iod786SbKYmuoLyHhrKfsUEcMcLR UL+TwgvMwXPLbAf5XctNndKpA9dmJmBk7bWwOtP3okwkBa3N2G1UeoCpgW9ar5utqc ww4btBDWZc+zw== Received: from nexus.subspace.kernel.org (localhost [IPv6:::1]) by lkml.kernel.org (Postfix) with ESMTP id CCE944AFA0; Wed, 18 Nov 2026 21:03:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Thu, 19 Nov 2026 05:03:39 +0800 Subject: [PATCH v5 07/17] mm/mglru: make folio lru referenced times count a generic API X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Message-Id: <20261119-mglru-fg-v5-7-9eee2fe97a2d@tencent.com> References: <20261119-mglru-fg-v5-0-9eee2fe97a2d@tencent.com> In-Reply-To: <20261119-mglru-fg-v5-0-9eee2fe97a2d@tencent.com> To: linux-mm@kvack.org Cc: Andrew Morton , Johannes Weiner , Muchun Song , Qi Zheng , Ying Huang , Chris Li , Baoquan He , Nico Pache , Usama Arif , Michal Hocko , Roman Gushchin , Shakeel Butt , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Vlastimil Babka , Suren Baghdasaryan , Nhat Pham , Youngjun Park , Zi Yan , Gregory Price , "Matthew Wilcox (Oracle)" , Baolin Wang , Ryan Roberts , Dev Jain , Lance Yang , Hugh Dickins , SJ Park , David Rientjes , Yu Zhao , Vernon Yang , Zicheng Wang , Chen Ridong , Tal Zussman , Kairui Song , Xiang Liu , Ehab Ababneh , Zhao Zhenzhuo , Lian Wang , Joanne Koong , Kunwu Chan , Hui Zhu , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, damon@lists.linux.dev, linux-doc@vger.kernel.org, Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1795035820; l=4693; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=koGM/cEkqmlw+8OmW+1cKVMTHOPOsjOLqgFFy8X/KuU=; b=zi/6EnbKEEehIkojCaP08SItBx3AmxZ9iynviSfz0axtK8BbU4T2ebhake7ZZUYYYMxUobit/ dhlpf518quqDlSdQTWZXqY7Y4QUCWJ5nRk1PPMq8DvUs3x0SaQPIdbl X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com Content-Transfer-Encoding: 7bit From: Kairui Song Pure code shuffle, no behavior change. Make the folio LRU refs helpers available regardless of CONFIG_LRU_GEN. Without MGLRU the extra bits are never set, so they are inert. Move the lru_gen_* helpers into the CONFIG_LRU_GEN section. This prepares for unifying the API for checking folio referenced and workingset status. folio_migrate_lru_refs() now migrates the complete refs count, including PG_workingset (bit 1 of the encoding), bitwise identical to the direct copy dropped from folio_migrate_flags(). lru_set_refs_flags() skips the nonexistent mask bits when LRU_REFS_WIDTH collapses to 0, preserving the two independent flags for !CONFIG_LRU_GEN builds. Tested-by: Zhao Zhenzhuo Signed-off-by: Kairui Song --- include/linux/mm_inline.h | 55 ++++++++++++++++++++++++----------------------- mm/migrate.c | 2 -- 2 files changed, 28 insertions(+), 29 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f38dad1198fc..180db43c3a85 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -184,6 +184,12 @@ static inline int lru_get_gen_flags(unsigned long flags) return gen; } +static inline int folio_lru_gen(const struct folio *folio) +{ + return lru_get_gen_flags(READ_ONCE(*const_folio_flags(folio, 0))); +} +#endif + /** * lru_set_refs_flags - Set the LRU referenced count to folio flags. * @flags: pointer to the folio flags @@ -206,7 +212,8 @@ static inline void lru_set_refs_flags(unsigned long *flags, unsigned int refs) *flags |= BIT(PG_referenced); if (refs & BIT(1)) *flags |= BIT(PG_workingset); - *flags |= ((unsigned long)refs >> 2) << LRU_REFS_PGOFF; + if (LRU_REFS_WIDTH) + *flags |= ((unsigned long)refs >> 2) << LRU_REFS_PGOFF; } /** @@ -252,13 +259,31 @@ static inline void folio_set_lru_refs(struct folio *folio, unsigned int refs) } while (!try_cmpxchg(folio_flags(folio, 0), &old_flags, new_flags)); } +#ifdef CONFIG_LRU_GEN void folio_inc_lru_refs(struct folio *folio, unsigned int flags); +#else +static inline void folio_inc_lru_refs(struct folio *folio, unsigned int flags) +{ + /* Should not be called with !CONFIG_LRU_GEN */ + WARN_ON_ONCE(1); +} +#endif -static inline int folio_lru_gen(const struct folio *folio) +/** + * folio_migrate_lru_refs - copy the reference state to a new folio + * @new: the destination folio + * @old: the source folio + * + * Transfer the reference state to @new during migration: the MGLRU + * refs count, or PG_referenced and PG_workingset (bits 0-1 of the + * encoding) for the active/inactive LRU. + */ +static inline void folio_migrate_lru_refs(struct folio *new, const struct folio *old) { - return lru_get_gen_flags(READ_ONCE(*const_folio_flags(folio, 0))); + folio_set_lru_refs(new, folio_lru_refs(old)); } +#ifdef CONFIG_LRU_GEN static inline bool lru_gen_is_old(struct lruvec *lruvec, int type, int gen) { unsigned long min_seq = READ_ONCE(lruvec->lrugen.min_seq[type]); @@ -424,20 +449,6 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, return true; } - -/** - * folio_migrate_lru_refs - copy the reference state to a new folio - * @new: the destination folio - * @old: the source folio - * - * Transfer the reference state to @new during migration: the MGLRU - * refs count, including PG_referenced, or just PG_referenced for the - * active/inactive LRU. - */ -static inline void folio_migrate_lru_refs(struct folio *new, const struct folio *old) -{ - folio_set_lru_refs(new, folio_lru_refs(old)); -} #else /* !CONFIG_LRU_GEN */ static inline bool lru_gen_enabled(void) @@ -470,16 +481,6 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, { return false; } - -static inline void folio_inc_lru_refs(struct folio *folio, unsigned int flags) -{ -} - -static inline void folio_migrate_lru_refs(struct folio *new, const struct folio *old) -{ - if (folio_test_referenced(old)) - folio_set_referenced(new); -} #endif /* CONFIG_LRU_GEN */ static __always_inline diff --git a/mm/migrate.c b/mm/migrate.c index 28a8d7734735..c8c49357df87 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -796,8 +796,6 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_set_active(newfolio); } else if (folio_test_clear_unevictable(folio)) folio_set_unevictable(newfolio); - if (folio_test_workingset(folio)) - folio_set_workingset(newfolio); if (folio_test_checked(folio)) folio_set_checked(newfolio); /* -- 2.56.1