From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oa1-f53.google.com (mail-oa1-f53.google.com [209.85.160.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B7C93FE662 for ; Tue, 24 Mar 2026 14:58:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774364301; cv=none; b=D1K6RfQSE52a6zEoBxAKpTew6SeQBYQXlFpBuHGWt4j3gOROIG+bO4L2JWdbC/V61eSB2chGNqfBBORmJBndmo74wOMmTsP1hlJftm9t14RLlaqtuE8oUZmRiMyjxAWhEJeJzLPagLoL12kiE1M/rJ2ksa3Qzvl0aqps2UnqVZk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774364301; c=relaxed/simple; bh=UmL+1K91qgh5SgxUxjDJBvS9D7yGeWFwhKJrrOnDlSE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=khHD21x5vviudp1rWUXlwMp8PC5EL5fpgmmxXdqVJ4H3eO13+v3S25c6FZ3SY1aoNTjJuH2JSqHQs67YJyai9txW8Snw7PfvDYAlGDMCd13+f5Sqmbv8YfaYxRBbtovY8PyQscXK8+6mhebs4Pel7ZmC1QthRiO5kyDAdkclmkY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gCmGxzMp; arc=none smtp.client-ip=209.85.160.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gCmGxzMp" Received: by mail-oa1-f53.google.com with SMTP id 586e51a60fabf-41708f6aa5fso3062349fac.2 for ; Tue, 24 Mar 2026 07:58:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774364299; x=1774969099; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NiY1YRqWKN2c0v1X9CNmB9tc2X+pcdgcgZJabn7k3xw=; b=gCmGxzMpPKHYcr7Y24z9dK3Xor3T4cRN61VB2yLj3bbCnTuhSrzRHeHypnL3vIANoa IjdyLUczaaxocXJY1hV8qb2pZrN1YJEUeM0E+JhCIhbX6K/1R9pODF41iNFnOUIdFtdt /Eik9IqBJENG5/NciuwfEaNEBRtyiO4eQX+LjUnA3Rut5anN+GziW6tDne/SitRCnfyt yzcTjJ2jVmcy+PjyALrRT6gQxbPTiAxipYQDdNae+oZAIJRWkx5ObKDFH/2enW1Jm5m1 tAdrIjjfuEdDQms4u91aIxhgemMzlSb7Q732UN5NKnuZDnwK2mtO4y4uywB4LtfKQt7x ZqYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774364299; x=1774969099; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=NiY1YRqWKN2c0v1X9CNmB9tc2X+pcdgcgZJabn7k3xw=; b=SwJoTlNxiJCC0hlUynD+m4LyPsqBsCfta+Lslx9WIYKjR1m7FwIwXWiCiOyc5sLr2C uJso094P34ynK3ZB/QXVQmCVAqRyxPg7xzj9SzBQVV1otAmQX4FpvwX67Iv3bmi3IfQU XD+05FvsCZ8nUNnE5zqNqzW7cHUrHWjxp6ljab4v+0IgfDbguNMDOcC1WbEMEet70yge yP5ik6ZiGUsV2lC/JkLq9Mshpt0rYKuzzgS/ILAcfxe6u7Oy34ri0t21IeKh4hkqXuQq 1ovlUmHSs2sDH3cEok7ImzjcxJIHAjea2NlXiiY2BIuVC0aoI/sS5etuQAvWLtzNJ6bC mFqA== X-Forwarded-Encrypted: i=1; AJvYcCXGmEzZPAIhCMdlWbckfeJnKK2KiLcEZJlz7d3ilV+FlU9+rkgWDfw3ZVeSI8vT9UkHjl97xViXhMqCdss=@vger.kernel.org X-Gm-Message-State: AOJu0YxIDGkJZ+kWcZUEBpbzW4HD0NNZAJnvyQ5uFIJzsOv1DcR38HNX J6uIeuGFPvJHSkJu0fD0FaLRvPrO3jxIeN7gvpTuZnPeujUR5megm3cU X-Gm-Gg: ATEYQzyCLTisQOVrXntsZuaL3kDWNouqwNYssycAqbSOI9Wv4SPI3lThN2P+GKWDr8X 0DYsI7DJzPJWNWgyaDGJYzoqn0UkEv3Lr9NQPTc9Fod9ILff5EdPv+5/rBzb5KWKtjC86ZlGDPT sXjAzl763WChQ/9HQc430NlP8Be8/7C4t9+ceFF6HFtXuryQGrZwwgoOgsFnZndBctP11JFly2i VVqtoS4Mu9Kn9Srl/KDZvsYtxR4oVC48R/8tOUjVfZSTkRaOToY1IVcq8SQL5ePl3CrWokqwk9G kNvlIu+gdCGOdiPN1/yRlgdqlmwqZPHgEQp4RRfAFAd9U1aU3ZPtm8ijbVeQSWra0vg0PkYRlIk rQcGtZOLDy9zeP0hszgt4M/JTNLGI7uy/dLh2DuowXY+6xruuZteA3MI4XGet7WSQg+zbAtaoLY H+M0rq6oAQQQ13uHMca6ECdA== X-Received: by 2002:a05:6870:c248:b0:409:6862:aba5 with SMTP id 586e51a60fabf-41c11157d40mr10061336fac.25.1774364298770; Tue, 24 Mar 2026 07:58:18 -0700 (PDT) Received: from localhost ([2a03:2880:10ff:5e::]) by smtp.gmail.com with ESMTPSA id 586e51a60fabf-41c14ddbcb3sm13662511fac.14.2026.03.24.07.58.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Mar 2026 07:58:18 -0700 (PDT) From: Joshua Hahn To: Donet Tom Cc: Gregory Price , Johannes Weiner , Kaiyang Zhao , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Waiman Long , Chen Ridong , Tejun Heo , Michal Koutny , Axel Rasmussen , Yuanchu Xie , Wei Xu , Qi Zheng , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: Re: [RFC PATCH 0/6] mm/memcontrol: Make memcg limits tier-aware Date: Tue, 24 Mar 2026 07:58:16 -0700 Message-ID: <20260324145816.3939303-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <13eb0f7a-95bc-4337-9d38-a06db0700777@linux.ibm.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit On Tue, 24 Mar 2026 16:00:34 +0530 Donet Tom wrote: > Hi Josua > > On 2/24/26 4:08 AM, Joshua Hahn wrote: > > Memory cgroups provide an interface that allow multiple workloads on a > > host to co-exist, and establish both weak and strong memory isolation > > guarantees. For large servers and small embedded systems alike, memcgs > > provide an effective way to provide a baseline quality of service for > > protected workloads. > > > > This works, because for the most part, all memory is equal (except for > > zram / zswap). Restricting a cgroup's memory footprint restricts how > > much it can hurt other workloads competing for memory. Likewise, setting > > memory.low or memory.min limits can provide weak and strong guarantees > > to the performance of a cgroup. > > > > However, on systems with tiered memory (e.g. CXL / compressed memory), > > the quality of service guarantees that memcg limits enforced become less > > effective, as memcg has no awareness of the physical location of its > > charged memory. In other words, a workload that is well-behaved within > > its memcg limits may still be hurting the performance of other > > well-behaving workloads on the system by hogging more than its > > "fair share" of toptier memory. > > > > Introduce tier-aware memcg limits, which scale memory.low/high to > > reflect the ratio of toptier:total memory the cgroup has access. > > > > Take the following scenario as an example: > > On a host with 3:1 toptier:lowtier, say 150G toptier, and 50Glowtier, > > setting a cgroup's limits to: > > memory.min: 15G > > memory.low: 20G > > memory.high: 40G > > memory.max: 50G > > > > Will be enforced at the toptier as: > > memory.min: 15G > > memory.toptier_low: 15G (20 * 150/200) > > memory.toptier_high: 30G (40 * 150/200) > > memory.max: 50G > > Hello Donet, Thank you for reviewing the series! I hope you are doing well. > Currently, the high and low thresholds are adjusted based on the ratio > of top-tier to total memory. One concern I see is that if the working > set size exceeds the top-tier high threshold, it could lead to frequent > demotions and promotions. Instead, would it make sense to introduce a > tunable knob to configure the top-tier high threshold? Yes, this is true. It is also a concern that I have, and I think that adding a tunable knob could be helpful. The other side of the question is whether there are too many tunables for the users already, with min / low / high / max. I'm hoping to get a consensus for this at LSFMMBPF, I hope we can talk about it there! The other way to approach this is to throttle promotions and demotions when workloads are thrashing. Personally I prefer this decision, although it isn't mutually exclusive to adding more knobs. > Another concern is that if the lower-tier memory size is very large, the > cgroup may end up getting only a small portion of higher-tier memory. I think the issue you mentioned above is a bigger problem. If the lower tier memory is large and the toptier memory is small, then it makes toptier memory an even more constrained resource, so splitting it fairly among the cgroups becomes an even bigger issue. Remember, we're limiting workloads' toptier memory usage because other workloads have to use it; if we let a cgroup use more toptier memory, it has to come from another cgroup's share. Thanks again. Please let me know if you have any other concerns, I'm excited to talk about this more as well! Joshua