From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mta1.migadu.com (out-92.mta1.migadu.com [95.215.58.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C8F53F3276 for ; Mon, 24 Aug 2026 09:17:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.92 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787563031; cv=none; b=TDnkwUrA6ryDv/kzpNm574R+WTUcttiLg9mdD9l/K7K5b9nKVDyNsyRBpOAIVLFLRn4wgvEBlY8WV7WQKlFJwYQ02BI0Lsud8Tc6KVKWj8r7U+y2E9H4yy2WJJVvuSdNJOU7YFwjqKRprBTvDtpRusz/ABuw5Aau1M1APNsfHDU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787563031; c=relaxed/simple; bh=O/9kZBkuJKeSt0mvaZpQcJ4Mx4IyhlVLvwU3R2sLsUw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=ok3B2PcPSyqFEiSlx/eEiXaca0Y9smaQ3ymbhAUqoFJlRVqahH4atZyD2EO+dB534olCVolOvi41QTptvaRBXrQ2eAfEcvyNX4DbKjiip5f6OhSsDzYTvwq3HAsQKo7DohAuE8AxL7WxJYoTJGGKCvHmXuCaw3ztzpDb9Eowtpw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=lYrHWXUs; arc=none smtp.client-ip=95.215.58.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="lYrHWXUs" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=O/9kZBkuJKeSt0mvaZpQcJ4Mx4IyhlVLvwU3R2sLsUw=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1787563027; v=1; x=1788167827; b=lYrHWXUsbJLYgQaupe9hjNej2voPhzHCBLPxt6H0Z9l4J3rzmbTbF041GrMdQRsGFk2aH+tI L2LoI2Ix0BqoGpU7hoKA5LM5SBTsMlIjOR+RIQwuE+uCkrs/e74iW7n4Iack98zfvTxMLROeMiI 3113e+xcdWDJeum9B+uxEO7E= X-Envelope-To: linux-kernel@vger.kernel.org Received: from teawater-KVM-Virtual-Machine (39.156.73.13) by smtp.migadu.com with ESMTPS id e1b4ba6028b59e6b; Mon, 24 Aug 2026 09:17:07 +0000 X-Mizu-Trace-ID: e1b4ba6028b59e6b X-Migadu-Flow: FLOW_OUT From: "Hui Zhu" To: Andrew Morton , Kairui Song , Qi Zheng , Shakeel Butt , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Johannes Weiner , David Hildenbrand , Michal Hocko , Lorenzo Stoakes , Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Hui Zhu Subject: [PATCH] mm: workingset: account MGLRU pages in count_shadow_nodes() Date: Mon, 24 Aug 2026 17:16:30 +0800 Message-ID: <20260824091631.836473-1-hui.zhu@linux.dev> X-Mailer: git-send-email 2.53.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Hui Zhu Commit 7404bd37cfbe ("mm: workingset: use lruvec_lru_size() to get the number of lru pages") switched count_shadow_nodes() to lruvec_lru_size(). With CONFIG_MEMCG enabled, lruvec_lru_size() reads mz->lru_zone_size, which is only maintained through mem_cgroup_update_lru_size(). MGLRU never goes there: all of its accounting (lru_gen_update_size(), reset_batch_size(), inc_max_seq() and __lru_gen_reparent_memcg()) uses __update_lru_size(), which skips the memcg array, and the classic lruvec_add/del_folio() paths return early once lru_gen_add/del_folio() succeeded. The four evictable LRU lists are therefore always seen as empty when MGLRU is on, and the shadow node budget (pages >> 3) is based on slab and unevictable pages only. For a memcg holding 1 GiB of page cache and 64 MiB of slab the budget drops from ~35k nodes to ~2k, so the workingset shadow shrinker reclaims eviction tokens almost as fast as they are created. Refaults then find live pages instead of shadow entries, lru_gen_refault() cannot restore the workingset state of hot pages, and thrashing protection is lost. This hits every memcg reclaim and, since the root memcg is iterated as well, global reclaim too. Fix this by adding the per-generation counters of an enabled multi-gen LRU to the budget. lrugen->nr_pages[] is eventually consistent and may go transiently negative while batched updates are pending, so clamp the sum at zero. Keep lruvec_lru_size() as it still accounts for the unevictable pages. Fixes: 7404bd37cfbe ("mm: workingset: use lruvec_lru_size() to get the number of lru pages") Signed-off-by: Hui Zhu --- mm/workingset.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/mm/workingset.c b/mm/workingset.c index f351798e723a..da20034d28fc 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -653,6 +653,37 @@ void workingset_update_node(struct xa_node *node) } } +#ifdef CONFIG_LRU_GEN +/* + * The per-generation counters are eventually consistent and may go + * transiently negative while batched updates are pending, so clamp + * the sum at zero. + */ +static unsigned long count_lru_gen_pages(struct lruvec *lruvec) +{ + struct lru_gen_folio *lrugen = &lruvec->lrugen; + long nr_pages = 0; + int gen, type, zone; + + if (!lrugen->enabled) + return 0; + + for (gen = 0; gen < MAX_NR_GENS; gen++) { + for (type = 0; type < ANON_AND_FILE; type++) { + long *cnt = lrugen->nr_pages[gen][type]; + + for (zone = 0; zone < MAX_NR_ZONES; zone++) + nr_pages += READ_ONCE(cnt[zone]); + } + } + + if (nr_pages <= 0) + return 0; + + return nr_pages; +} +#endif + static unsigned long count_shadow_nodes(struct shrinker *shrinker, struct shrink_control *sc) { @@ -697,6 +728,15 @@ static unsigned long count_shadow_nodes(struct shrinker *shrinker, for (pages = 0, i = 0; i < NR_LRU_LISTS; i++) pages += lruvec_lru_size(lruvec, i, MAX_NR_ZONES - 1); +#ifdef CONFIG_LRU_GEN + /* + * MGLRU accounts for the evictable pages in + * lrugen->nr_pages[] instead of mz->lru_zone_size, which + * lruvec_lru_size() reads, so the loop above misses them. + */ + pages += count_lru_gen_pages(lruvec); +#endif + pages += lruvec_page_state_local( lruvec, NR_SLAB_RECLAIMABLE_B) >> PAGE_SHIFT; pages += lruvec_page_state_local( -- 2.53.0