From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=209.85.167.200 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794009553; b=36/eFBvI9dS+Q3g6PK0X31+8XXHTpaeHRTZ6CeOTrZ56vsaB8/mDkRfFp2xWBEZQVloL h30ISeglmHsb6SqLsSZyVG2ewUMFa24xmbJ+UiD3GJrzxnQKmVvQe4nf+cpNd79ia/Yb+ mG77GLHxS7PT9CYUKlDfITGe+foTKeqez4XmleW4K6P/5shHBynwmECGp05n64BJzvrr0 52ql/s+qtyuLA3miwF5o3QQ7b9BtGAdhVtFY2rU+d+s4gFQ+uPjuid60FhSaIaP4I6yAH /D1/p6tmKg8zYrHlk1CnKqfiHnE9fOVvjnBV09Xi4oB98zW4Dm4SwbcZnn8W2kuAWUw== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794009553; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Received:Date:Mime-Version: X-Mailer:Message-ID:Subject:From:To:Cc:Content-Type; bh=QLxBnr2dWVF7awuaywn040Wg1Wa6I8q0xUXMqEWtl88=; b=Hfct7WjfP5HCVhtB9ZSCwKCrvb2lwlLQjN5h7lQG2cme2XJpXzAwSPwo0x9XFunu63VV E2NvNkPUwHNb/XAKM+FkNXMe9+jfpMi1QOnHRvrNcDg+8xtj1vvn7VXgNuO+dspgHDurm YwgHlLU2MfrNVtKRFIzMcJgLZvwpo7dSVNlJAQ+A7AY7z73ihIQndZPua/MkkE/+2mUt8 PtJ/oBc3jZP2LG1GkBg8VsA/JMqQWMVTERBr28YfBbawSAQmHh0l+D6wmg46EdibCWbwl aHwrYjFuurpMYu6+Yiz1GwM8iL76YbPYFX0MxZ7HLJuFByT9dQ3Eoes3Yxp7E9Ws+hQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=google.com; spf=pass smtp.mailfrom=flex--tjmercier.bounces.google.com; dkim=pass header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=OMpqWCP3; arc=none smtp.remote-ip=209.85.167.200 Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9EC0606A32C for ; Fri, 6 Nov 2026 23:59:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org D9EC0606A32C Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--tjmercier.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=OMpqWCP3 Received: by mail-oi1-f200.google.com with SMTP id 5614622812f47-567e742f481so725103b6e.3 for ; Fri, 06 Nov 2026 15:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20260707; t=1794009552; x=1794614352; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:mime-version:date:from :to:cc:subject:date:message-id:reply-to:content-type; bh=QLxBnr2dWVF7awuaywn040Wg1Wa6I8q0xUXMqEWtl88=; b=OMpqWCP3w2TNWkCpaGDqlfhPWO7GAkSXve+GiumCIEy+wpd8n9eBF0ZdfrQjNkuexF gRixkS6t0FqUbM2FyUoEePBUWNxCS7oL6wnv48is0SPAG3aDwgoou0I36MZSMari3V7x rtRpPyXLBYy2mXYNv3VZuhjEiywNom6v5ZaIhrUPAl4FgYnZT4SZItLe+vkIxhvxkKYM CZn/Pghml7RHbdGLmq6P93YVS57kGfAIZiZ7JCuP7rXpdstHK7IxhJ15p6FTSbLHwkjv uMFnW9vCglhhTTWakcpZM0gfC7pPCUhMScxetQKFDT7LIkkruatKFs3c4y24gQjktgwL nYAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794009552; x=1794614352; h=content-type:cc:to:from:subject:message-id:mime-version:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=QLxBnr2dWVF7awuaywn040Wg1Wa6I8q0xUXMqEWtl88=; b=Xr/nfz0WGADz26l+OgcJAH2zD0Jak+Gd6T1hZ1A3snjEr3c1JqyurJOfoZOrJryw8g 7sslcshvGuzKEvHZVP5OfDQ0OkJRnBCf7zf/e6kiwyyTUpDEbswyaEn2tt30LhMQyBi/ OnNorHlduL6hvvV0BiKho4SkGk+kQq8o8M4ofcUdkKK0QAGRpBXgSIL+IKSlg82nJxot 5FU5R8Mz+IGrBrFo56IbWhYpc5+dBEqks3BVRUYDkAFmOKq9HaUdbV+AaAmIay+ghaJl 4woH3F60IW54jWpkIotUmCnlMtG9bxVj/ZPszw1j/g2ZQFb231kQCMVezmndI+mFW5fm VAXA== X-Forwarded-Encrypted: i=1; AKwUvBxSwroH0nVAPK+wToCu0hEz4L12iEpeLFf4Es2Hln6ITgijWoC0YYZzvGlHK2PJO5p1BJyN3ijVH08Bxa0=@vger.kernel.org X-Gm-Message-State: APuU4Z/dyGhulYnXAJa9dHF/G+U5Lfkzo7LBQSfNoKx9FqeNbMY02J7j pz5/c6psvNbpCt0dmr2Sg6FOutYlqOilvfnxk+7IqAXrLVmo1DztPTaIHbvOymxOupvT0tUEF5T qLAnI1ZQjSaTErCkvYA== X-Received: from iokk25-n1.prod.google.com ([2002:a05:6602:8319:10b0:9c4:de2b:c40f]) (user=tjmercier job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6808:1985:b0:55c:8ee:a9c4 with SMTP id 5614622812f47-567e115e036mr3038381b6e.43.1794009551333; Fri, 06 Nov 2026 15:59:11 -0800 (PST) Date: Fri, 6 Nov 2026 15:59:04 -0800 X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 X-Mailer: git-send-email 2.56.0.455.g6c5a503b38-goog Message-ID: <20261106235904.1092712-1-tjmercier@google.com> Subject: [PATCH] mm: workingset: compare shadow nodes node-wide without kmem accounting From: "T.J. Mercier" To: Andrew Morton , Johannes Weiner Cc: Shakeel Butt , Michal Hocko , Roman Gushchin , Muchun Song , Qi Zheng , Dave Chinner , "=?UTF-8?q?Michal=20Koutn=C3=BD?=" , David Hildenbrand , Lorenzo Stoakes , Kairui Song , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Baoquan He , Baolin Wang , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, "T.J. Mercier" , Jaewon Kim , Jaeseon Sim , Youngchan Kim , Dmitry Skiba Content-Type: text/plain; charset="UTF-8" count_shadow_nodes() limits the number of xarray nodes holding shadow entries to 1/8th of the pages that could be using them. For a memcg, it counts the LRU and slab pages local to that memcg. With kmem accounting disabled (cgroup.memory=nokmem), shadow nodes are not charged to any memcg and __list_lru_init() makes the shadow_nodes list_lru non-memcg-aware, so all shadow nodes of a NUMA node are on a single list. shrinker_alloc() falls back to a non-memcg-aware shadow node shrinker, which shrink_slab() only invokes for the root memcg. count_shadow_nodes() therefore compares every shadow node on the NUMA node against the pages local to the root memcg. When most tasks run in child memcgs, the root memcg only holds a small fraction of all pages, so the shadow node limit derived from the root memcg page count is far below the number of shadow nodes on the NUMA node. Since the shrinker has seeks == 0, every root shrink_slab() call trims about half of the "excess" shadow nodes. Most refault information is thrown away, so refaults of recently evicted file folios are not detected. Refaulted folios lose PG_workingset and the workingset_refault and workingset_restore stats drop. In testing on devices with kmem accounting disabled, moving tasks from the root memcg into per-application memcgs dropped workingset_nodes from ~37,000 to ~900 and raised workingset_nodereclaim 4.6x while pgsteal_file rose by ~12%. When kmem accounting is disabled, compare the shadow nodes against the whole NUMA node, as is done without memcg. Behavior with kmem accounting enabled is unchanged. Cc: Jaewon Kim Tested-by: Jaeseon Sim # Galaxy A18 LTE Tested-by: Youngchan Kim # Galaxy S27 Tested-by: Dmitry Skiba # Pixel 10 Signed-off-by: T.J. Mercier --- mm/workingset.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index 1504f91cdca5..87d5e55dd684 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -682,7 +682,16 @@ static unsigned long count_shadow_nodes(struct shrinker *shrinker, * PAGE_SIZE / xa_nodes / node_entries * 8 / PAGE_SIZE */ #ifdef CONFIG_MEMCG - if (sc->memcg) { + /* + * Without kmem accounting the shadow_nodes list_lru is not + * memcg-aware. So the count above covers the whole NUMA node, and the + * shrinker only runs for the root memcg, whose local pages can be a + * small fraction of the node's when most tasks run in child memcgs. + * Compare the shadow node count against the memcg page count only when + * the shadow_nodes list_lru is memcg-aware. Otherwise use the NUMA node + * page count as is done without memcg. + */ + if (sc->memcg && shadow_nodes.memcg_aware) { struct lruvec *lruvec; int i; -- 2.56.0.455.g6c5a503b38-goog