From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A61356E07F; Thu, 15 Oct 2026 18:15:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792088156; cv=none; b=nVzHSwFpWhZufAeO2+tJuounWz1p+50DOiLWhG0hrvB7ST/XVijDhH0NyaqBFLaPqFPdc9bfqcxRQ5QENRxmqTHsoIiQDWBnJN96AwdjzGwYIQ4XmraqA8c4J8iQHeIBcipao8g9CULuvY3EGJBfRIRewF0jlJJP/FDLA1CjK10= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792088156; c=relaxed/simple; bh=aXanTrv/q1uzORNQ0D7X/HNSQWLDCgGjHvyMIPDoJMQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=H+aIeQDj8NovytR2a3j4Vsek1ai1+fDqgfY98m1vzm68j4alXc9CjMzdubY2bjf2IYrVA/KPFejgHJ6SGdzmRhDrnmWrQo7tAQ7545qtNxOVJ4l+q+yMFVlAj96OVeL4R5KFPAMUEZ3ZODy4c+ZM+mSJHaLrIOBmN6JtlL0pkIY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O0c+xTaO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O0c+xTaO" Received: by smtp.kernel.org (Postfix) with ESMTPS id 93E98C4AF48; Thu, 15 Oct 2026 18:15:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1792088155; bh=aXanTrv/q1uzORNQ0D7X/HNSQWLDCgGjHvyMIPDoJMQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=O0c+xTaOezeSAIpT8j/CsXqvpM9BMqBpSJtUVXUk8NK9s390HXMkwjqEQCckxETGq TFbxK366KbU8AmdBLEuVJ8VIPTLQvOxx5bbxpTlPUGzII5KO9UEffCYZ2TrXDuhTu3 Myoaf3hCv5wruPYq+iFS7Of07Iv/R3AEtpqgjsKVM834ufUH2vVF55DVQ/Yy9vXYaH N2A8pEWKL3oPgIRgh4pGBRGwTr9KOKs3FPyfistQGo9yvB4QKprmulZv4FbBnhNQ8t jR64YoreNIle3uwxSQ3Foyufx4Dd2EaqkJCRE5N5317FOuXiaaoBjRtoxrnnWYOAy1 pfavLGAA+JSwA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79BD1CCA464; Thu, 15 Oct 2026 18:15:55 +0000 (UTC) From: Kairui Song via B4 Relay Date: Fri, 16 Oct 2026 02:15:55 +0800 Subject: [PATCH v4 06/17] mm/mglru: move add/del LRU size accounting out of lru_gen_update_size() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261016-mglru-fg-v4-6-2106750c0bfa@tencent.com> References: <20261016-mglru-fg-v4-0-2106750c0bfa@tencent.com> In-Reply-To: <20261016-mglru-fg-v4-0-2106750c0bfa@tencent.com> To: linux-mm@kvack.org, "Liam R. Howlett" , SJ Park Cc: Andrew Morton , Johannes Weiner , Muchun Song , Qi Zheng , Ying Huang , Chris Li , Baoquan He , Nico Pache , Usama Arif , Michal Hocko , Roman Gushchin , Shakeel Butt , David Hildenbrand , Lorenzo Stoakes , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Vlastimil Babka , Suren Baghdasaryan , Kemeng Shi , Nhat Pham , Youngjun Park , Zi Yan , Gregory Price , "Matthew Wilcox (Oracle)" , Baolin Wang , Ryan Roberts , Dev Jain , Lance Yang , Hugh Dickins , David Rientjes , Yu Zhao , Vernon Yang , Zicheng Wang , Chen Ridong , Tal Zussman , Kairui Song , Xiang Liu , Ehab Ababneh , zhaozhengzhuo , Lian Wang , Kunwu Chan , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, damon@lists.linux.dev, Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1792088152; l=3201; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=R/1YuK+DuWVnDJLZi3UVamo0/HdrkSNNWuBiwaeVr3w=; b=2XLndrLsif/WtrelNCXSnsafTuNoASKjH8a8Rr0KhO6jbM9NO93sfCEV4tVmH61jPs7137Vv2 m5xLqJfCA4/BdxGQmIqtwu86gcbOIJVwUzpuYsoIQ563tkjn20h4A8R X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Pure code shuffle, no behavior change. lru_gen_update_size() now only updates the per-generation counters plus the promotion move; the addition and deletion active/inactive accounting move to lru_gen_add_folio() and lru_gen_del_folio(), which know the generation and can apply the active-window test directly. The unevictable case is untouched: unevictable folios never enter lru_gen_add_folio() and keep being accounted by lruvec_add_folio() as before. Tested-by: zhaozhengzhuo Signed-off-by: Kairui Song --- include/linux/mm_inline.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index cfcbe4a9aa85..65415eeac802 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -313,21 +313,9 @@ static inline void lru_gen_update_size(struct lruvec *lruvec, struct folio *foli if (new_gen >= 0) atomic_long_add(delta, &lrugen->nr_pages[new_gen][type][zone]); - /* addition */ - if (old_gen < 0) { - if (lru_gen_is_active(lruvec, new_gen)) - lru += LRU_ACTIVE; - __update_lru_size(lruvec, lru, zone, delta); + /* return now if not a promotion */ + if (old_gen < 0 || new_gen < 0) return; - } - - /* deletion */ - if (new_gen < 0) { - if (lru_gen_is_active(lruvec, old_gen)) - lru += LRU_ACTIVE; - __update_lru_size(lruvec, lru, zone, -delta); - return; - } /* promotion */ if (!lru_gen_is_active(lruvec, old_gen) && lru_gen_is_active(lruvec, new_gen)) { @@ -381,6 +369,8 @@ static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, int gen = folio_lru_gen(folio); int type = folio_is_file_lru(folio); int zone = folio_zonenum(folio); + int delta = folio_nr_pages(folio); + enum lru_list lru = type * LRU_INACTIVE_FILE; struct lru_gen_folio *lrugen = &lruvec->lrugen; BUILD_BUG_ON(BIT(LRU_GEN_WIDTH - 1) != MAX_NR_GENS); @@ -396,6 +386,10 @@ static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *folio, set_mask_bits(folio_flags(folio, 0), LRU_GEN_MASK | BIT(PG_active), flags); lru_gen_update_size(lruvec, folio, -1, gen); + if (lru_gen_is_active(lruvec, gen)) + lru += LRU_ACTIVE; + __update_lru_size(lruvec, lru, zone, delta); + /* for folio_rotate_reclaimable() */ if (reclaiming) list_add_tail(&folio->lru, &lrugen->folios[gen][type][zone]); @@ -409,6 +403,9 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, { unsigned long flags; int gen = folio_lru_gen(folio); + int zone = folio_zonenum(folio); + int delta = folio_nr_pages(folio); + enum lru_list lru = folio_is_file_lru(folio) * LRU_INACTIVE_FILE; if (gen < 0) return false; @@ -422,6 +419,9 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, gen = ((flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; lru_gen_update_size(lruvec, folio, gen, -1); + if (lru_gen_is_active(lruvec, gen)) + lru += LRU_ACTIVE; + __update_lru_size(lruvec, lru, zone, -delta); list_del(&folio->lru); return true; -- 2.56.0