From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50FBF45516D for ; Fri, 7 Aug 2026 20:21:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786134083; cv=none; b=lsbJklmUoE92HYE1+ddmP07TMg4xjNGfSL+BEtxs6e5KiSQUNQNF25nlnwpYyqDhDiLF2+VOQII53gEs9PbYvut/6XTJPeEXh6Y7kqDcDUaPAjICJhjS9unzclKTQOFFk6s01RdK9uHqH3QvIs4Iq+RswBHmfoFIEcKge5swKfw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786134083; c=relaxed/simple; bh=4YHmAs4imI5gKBDc4Wehbn5cIF5tjkqbuwyC0X3KRSI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iPp87rsZuIzA2A8aBPj55wZ2XYahxPedsFoG70XhXTZ5DctpMUy03Z0UemQazNwQZRxrlklhjOh4XeLavt/brwKLYQLyotAwdYiwufj2rbNbC9McVed9HmiiwGaP0cv6lOgCAq3la9EFPPGgmiOWksNk9nVEBARH6ejKtCCiah8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aJaHmT5o; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aJaHmT5o" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-6ae5baaef5dso1914978eaf.1 for ; Fri, 07 Aug 2026 13:21:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1786134080; x=1786738880; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7/o7PyJxYr/auJmws9zryGpCICrELhYgOPRDbTKuSBc=; b=aJaHmT5oyPYDad1NKHaiW0BNFW2HugGkOZeSpMyYtJcyrz8nLE1vCoKTznIdZxMOvB IQQ09H8e404BOub7VB+I13fSbKSpz/AZ1APhY+eYPW5ElLF6PYoC3O1jaKcnA8D9ffs/ lpYGA6WXdRgwoBRgQxa/vYKbPUcPTOiXtmle4UssWrHguNwpJyeqPqFFWPnAMqpbgXP8 u+wabKckWHK1lj1MZ2LsULNxenxpZp7G6yJ41nke79Z9qOVITz7uPZOhSKHZIAB8eZ/X hpQSNPlGD5Xw3azQ+oueJfiAh6EEol4mWa5cG3RABNOXWqYwcJpWymBCnmhat76uPqGw UISQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786134080; x=1786738880; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7/o7PyJxYr/auJmws9zryGpCICrELhYgOPRDbTKuSBc=; b=EcvTu3aLwhCnf6+hEwLriXtXkPybBMUErOW0plDmXAgvrDMf/WbHgpncQKJoQzWuQf wNAGL1VqLM4MMSxIvn3tKOptu7X5R96aSmjWrncmL/ZZMWGmVuYIz7nY1UGEBuanklPu lZ/cP8/mmrJ3QtaGL/dc+pEsOgupBPBbplkYWV95kMfhcrigmpdo32UCuZoMfAi9v+BH KqwxnmHHyysLwhvwud0Dk7dob19cpjniGQJwmdbbxgIZRclqgCjAC++2Ch1+NbT3XQUz PV0oghbu6sh0ZdVGsGgAS96/sf6LAseACi/aK8Dt9/2VBfLr3jyYsOxULtqeKadyeT+1 pE2Q== X-Forwarded-Encrypted: i=1; AHgh+Rpy31JNh+xo69UPM7XWD+Idn5o2iTalm7ZTyLXT497BnedZWUsvi7WiN+PXOHvtdMveJzfh0mOsk9RqebU=@vger.kernel.org X-Gm-Message-State: AOJu0Yz0idhzpgMOXOwPu36xdkBwNemR8fsrlZ7Z+oppLZSLoLAI88j2 PzTsoYmCimlfsDCmcn227C6EatZJrdKcYr6usw+EUfTW5m90YDBN9swg X-Gm-Gg: AR+sD106IPMW+1UsDL9MVKQ0qIkxNwJrhvA6FDK7jlFTid4NjgtgXhbNvQjmPtaiWpp RPUkMIRiRqhtxX0yl84/wGN8pmCErVNrd/J8M7ZhTZ8MrQMAgQSB99Oz+EJL/IRRxXdl2YvR/He O8VP5G7ufOstvZUO99dALxa7lufSaewpqmKIVjTueHzmVXB+GYbYcYsI+BsWnj7+oNbFAnthWBZ 1t1H7Dz65Mkd4SnR2gJE4PoT/GXq9nchBrj76jUjLiOGtuGGWdA4fehAKuyOzbjpazrXZK6WdTv +hzyvEo8Yo+TvGR3TybHmJ4vZ8s4bVvAjeHn/4Gz7X2L9Bj9a8nxlPnjehm7R/ZPMZhRRhvNn9m H1YwjxVqEYUrQPcYtUaM81m3buwDWwJJX5DVIzUpmaKlIxAhM3YXHQvVcKOYYojN9HpnC854334 hOpycYDvjOQAGY0BmVtSLViRG0PHAbC+0ZpMNL1xBNf5k3jy43QsG4AaNRwAP2aaDopKtYcnUkv MgqdqdsxEnaKjtGGJg= X-Received: by 2002:a05:6820:8c7:b0:6a3:6f5d:4d6c with SMTP id 006d021491bc7-6b044e7bbdfmr825692eaf.6.1786134080128; Fri, 07 Aug 2026 13:21:20 -0700 (PDT) Received: from localhost ([2a03:2880:10ff:12::]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-459f1a9d9b9sm2606796fac.7.2026.08.07.13.21.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Aug 2026 13:21:19 -0700 (PDT) From: Joshua Hahn To: Johannes Weiner , Gregory Price Cc: Alistair Popple , Andrew Morton , Axel Rasmussen , Barry Song , Ben Segall , Brendan Jackman , Byungchul Park , David Hildenbrand , David Rientjes , Dietmar Eggemann , "Harry Yoo (Oracle)" , Ingo Molnar , Juri Lelli , K Prateek Nayak , Kairui Song , "Liam R. Howlett" , Lorenzo Stoakes , Matthew Brost , Mel Gorman , Michal Hocko , Michal Hocko , Mike Rapoport , Muchun Song , Peter Zijlstra , Qi Zheng , Rakie Kim , Roman Gushchin , Shakeel Butt , Steven Rostedt , Suren Baghdasaryan , "T.J. Mercier" , Valentin Schneider , Vincent Guittot , Vlastimil Babka , Wei Xu , Ying Huang , Yosry Ahmed , Yuanchu Xie , Zi Yan , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@meta.com Subject: [RFC PATCH v3 13/14] mm/memcontrol, sched/numa: Gate NUMA promotions into memcg tiers Date: Fri, 7 Aug 2026 13:20:56 -0700 Message-ID: <20260807202059.2620949-14-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260807202059.2620949-1-joshua.hahnjy@gmail.com> References: <20260807202059.2620949-1-joshua.hahnjy@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Memory promotions that go through should_numa_migrate_memory determine if a promotion should be ratelimited / throttled by checking how much headroom there is in the destination node. If there is enough headroom, there is no reason to be throttling promotions. On tiered systems, however, a promotion may trigger reclaim on a node that has plenty of promotion headroom since the memcg tier may be at the limit. For these allocations, we should make sure that memcg tier fullness is also considered when determining whether a promotion should be able to go through without getting limited. Add an additional condition to check before letting a promotion candidate go through un-ratelimited, by checking if the memcg tier is already at its limit. No-op unless the system has tiered memcg limits enabled. Signed-off-by: Joshua Hahn --- include/linux/memcontrol.h | 7 +++++++ kernel/sched/fair.c | 3 ++- mm/memcontrol.c | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+), 1 deletion(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9c2f11191a499..a7c366b431a0e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -654,6 +654,8 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *target, page_counter_read(&memcg->memory); } +bool mem_cgroup_tier_over_limit(struct folio *folio, int dst_nid); + int __mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp); /** @@ -1172,6 +1174,11 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *target, return false; } +static inline bool mem_cgroup_tier_over_limit(struct folio *folio, int dst_nid) +{ + return false; +} + static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp) { diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d78467ec6ee13..397b0f3e67f5f 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2697,7 +2697,8 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, long nr = folio_nr_pages(folio); pgdat = NODE_DATA(dst_nid); - if (pgdat_free_space_enough(pgdat)) { + if (pgdat_free_space_enough(pgdat) && + !mem_cgroup_tier_over_limit(folio, dst_nid)) { /* workload changed, reset hot threshold */ pgdat->nbp_threshold = 0; mod_node_page_state(pgdat, PGPROMOTE_CANDIDATE_NRL, nr); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4dce7c6fefd98..05611a01aa082 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2590,6 +2590,41 @@ static u64 swap_find_max_overage(struct mem_cgroup *memcg) return max_overage; } +bool mem_cgroup_tier_over_limit(struct folio *folio, int dst_nid) +{ + struct mem_cgroup *memcg; + int dst_slot; + + if (!mem_cgroup_tiered_limits()) + return false; + + dst_slot = nid_tier_slot(dst_nid); + if (nid_tier_slot(folio_nid(folio)) == dst_slot) + return false; + + guard(rcu)(); + memcg = folio_memcg(folio); + if (!memcg || mem_cgroup_is_root(memcg)) + return false; + + do { + struct page_counter *tier_counter; + unsigned long limit; + + tier_counter = mem_cgroup_tier_counter(memcg, dst_slot); + if (!tier_counter) + continue; + + limit = min(READ_ONCE(tier_counter->max), + READ_ONCE(tier_counter->high)); + if (page_counter_read(tier_counter) > limit) + return true; + } while ((memcg = parent_mem_cgroup(memcg)) && + !mem_cgroup_is_root(memcg)); + + return false; +} + /* * Get the number of jiffies that we should penalise a mischievous cgroup which * is exceeding its memory.high by checking both it and its ancestors. -- 2.53.0-Meta