From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=91.218.175.240 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794393532; b=BTP6f+4la9mZuQYWyfk4jx5yBXm9+85YoAlqhn8cXLFCTxNV9aJprzuNFSNcu38PodsL IVjvRVuH7+gq8qokb8ZPYC9jyWE7pbMFAKK/Hatr3DAPuBHQdPmoWJssVT4exj+WJD/pa RyXr3VJN5S0w8Tsxm8BIjoZfA7wQSA8mXxzHez3IWNNKdVxxq1tF9CxY3WG/0/AmKJYZm Ij4TFw2SxNkO8ipxhbHYSwElFBwTLq9WX6mZtCM7PdnVV/IxY2I+/7Pbp0IQXfomoGlco 9IaHf1yuu8i5HS6yPWvFkOREWGkrVQO42oQKc2/10bLMLeI0+/Jm4xb2TO5P1ZV3IVg== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794393532; h=DMARC-Filter:X-Envelope-To:DKIM-Signature:X-Envelope-To:Received: X-Mizu-Trace-ID:X-Migadu-Flow:From:To:Cc:Subject:Date:Message-ID: X-Mailer:MIME-Version:Content-Transfer-Encoding; bh=xDU+VUQuiVqA+errFgpqHKql5l47PsMQESWCLQkI4gc=; b=lhfoz9kRSk7qB8Vcaj6XQxM1Myj/YT/U1Tuvl6IoJrCHmo83YM9UhkWBWCxwX50l3sd2 n5vYtNKVAf1A/TeozI08fGzj3BK4EJlMDMt8TOjMUjV/U0pS3ppbHTtikEMs/sgOE6BBW xKJrCf+U4a7MuvxrBjLqMPrt/JryW/JNjtwDiE4MD78wrZ2O1HtsySj8kSB6480pPUtJL tpQ5ZDbXXqbYacysttQ0a8Pl7XppKuapgE38wzUuJcO747BJULc4F/3Pz5viA91a09ekr Rpg6mTmTdRfXaJ/FQNnJ0IS0dMPSwyFCYGQMzTH3WT98Yd8uYUAPPy1riA9zn0038KQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=bjvfla5w; arc=none smtp.remote-ip=91.218.175.240 Received: from mta0.migadu.com (out-240.mta0.migadu.com [91.218.175.240]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69480606A327 for ; Wed, 11 Nov 2026 10:38:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 69480606A327 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key, unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=bjvfla5w X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=Pgs9Rl2H9Mrm9v98ZZj54dqFp+ymM4kJacjilJCmcW0=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1794393528; v=1; x=1794998328; b=bjvfla5wO8wtHYY4gSPf478Ha6EcgBkojYdgSSSWnEhPH2JZVAQ33yT1o3xNJVDqUw7YvvH0 3EFHYlF/woFlHpDarnibVMC3p+oYEJxF6nosE0wGj9PNePOfrbD+l/9/tfEvKHC0UVKE/X/jmiB gJmazx4HcncKYSXjLw154HKc= X-Envelope-To: linux-kernel@vger.kernel.org Received: by mta12.migadu.com with ESMTPS id 272d3a1a2ff3a6d3; Wed, 11 Nov 2026 10:38:46 +0000 X-Mizu-Trace-ID: 272d3a1a2ff3a6d3 X-Migadu-Flow: FLOW_OUT From: Hui Zhu To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Kumar Kartikeya Dwivedi , Song Liu , Yonghong Song , Jiri Olsa , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , JP Kobryn , Andrew Morton , Shuah Khan , davem@davemloft.net, Jakub Kicinski , Jesper Dangaard Brouer , Stanislav Fomichev , KP Singh , Tao Chen , Mykyta Yatsenko , Leon Hwang , Anton Protopopov , Amery Hung , Tobias Klauser , Eyal Birger , Rong Tao , Hao Luo , Peter Zijlstra , Miguel Ojeda , Nathan Chancellor , Kees Cook , Tejun Heo , Jeff Xu , mkoutny@suse.com, Jan Hendrik Farr , Christian Brauner , Randy Dunlap , Brian Gerst , Masahiro Yamada , Willem de Bruijn , Jason Xing , Paul Chaignon , Lance Yang , Jiayuan Chen , Emil Tsalapatis , Ihor Solodrai , Barry Song , Geliang Tang , linux-kernel@vger.kernel.org, bpf@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: Hui Zhu Subject: [RFC PATCH v2 0/2] memcg_ext: dynamic memory.low protection requests Date: Wed, 11 Nov 2026 18:38:14 +0800 Message-ID: X-Mailer: git-send-email 2.43.0 X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Hui Zhu This is v2 of the RFC "memcg_ext: dynamic memory.low and memory.min protection" [1]. It sits on the memcg_ext base [2], which adds bpf_memcg_ops - a struct_ops type attaching memory controller policy to a cgroup - and lets a BPF policy take part in memory.low protection: not by deciding whether reclaim skips the cgroup, which is what the RFC did and what the review correctly rejected, but by requesting protection and going through the same arbitration as a write to memory.low. Two things changed since the RFC, and both are answers to the review. THE CEILING As posted, a policy could make reclaim skip a cgroup whose usage was above what its parent affords. That defeats the cap effective_protection() applies at subsequent levels precisely to keep delegation safe: a delegated tenant can attach a policy to its own cgroup, so it could grant itself protection its parent never gave. That was a defect in the interface, not a property to defend. In v2 the hook no longer decides anything; it requests. The request is collected before the effective value is derived, and what a cgroup claims becomes the larger of its memory.low and the request - capped by the parent's effective protection and shared out with the siblings by the same code, on the same inputs, as a declared value. A request can move protection around inside the parent's budget; it cannot lift the subtree above what the parent affords. Anything a policy can request, a write to memory.low can request too. What is left to the policy is when and on what evidence the request is made: every pass, at the point where the effective value is about to be derived, rather than between passes. min_policy is dropped rather than converted: this round concentrates on memory.low, and the interface goes from two members to one. A memory.min counterpart can be revisited later. The rework also removes the memcontrol.h change entirely: mem_cgroup_below_low() and mem_cgroup_below_min() keep their signatures and the series no longer touches the protection checks. One cost is stated here rather than left to be found: the callback now runs for every memcg on every pass - with MGLRU that includes lru_gen_age_node() walking every memcg on the node - where the RFC asked only about the cgroups reclaim had found unprotected. With nothing attached, a call is a page counter read, a static key check and a comparison against the stored request. Measured under the reclaim storm of the demonstration below: ~56k hook calls in a 30s window, 52ns average per call, 2.9ms of in-hook CPU in total - 0.01% of one core. THE MEASUREMENT The review asked why the same cannot be achieved by changing memory.low from userspace - the same question Michal asked on the v7 thread [3] in May. This time the answer is a measurement rather than an argument. The claim under test is a window, not a throughput. Direct reclaim runs in the allocating task's own context, and shrink_node_memcgs() examines memcgs one at a time, calling mem_cgroup_calculate_protection() per memcg. An agent cannot do anything for the memcgs a pass has already examined, and for the ones it has not reached, a concurrent write to memory.low is picked up only once the agent has been woken, scheduled, read memory.stat or memory.pressure, decided and written - in a burst that window holds at least one full pass, which is where the working set goes. After the rework the hook sits at the point where the request is collected, so it is asked on that pass. Method. Three implementations run the same integer policy on the same evidence and differ only in where the decision runs: U userspace agent: preads the victim's memory.current on a 5ms timer and pwrites the sample to memory.low. Made as strong as I can: pre-opened fds, no allocation in the loop, and its own cgroup with memory.min so it survives the pressure (disclosed). A PSI-poll trigger variant exists and was measured too. B1 a bpf_wq callback on a 5ms timer reads the same counter through bpf_mem_cgroup_usage() and stores the verdict; the low_request hook executes it. The control for "the action runs in the kernel". B2 the low_request hook decides from ctx->usage - the value the kernel read for this very pass - on every protection pass. The only arm that claims something userspace cannot do. The shared evidence is the fall rate of the victim's memory.current: reclaim uncharges it synchronously in-pass, so a falling usage means "the victim is being stolen from" with no statistics-flush dependency. The textbook channels were tried and measured blind: workingset refaults need the shadow entry to survive eviction (96M stolen from the victim counted as 2 refault pages under the filler's shrinker churn), and pgsteal is only readable from flushed counters (a whole burst window passed with the in-hook arm seeing zero movement while the victim lost 19k pages). The threshold is a rate - pages per second, not a per-sample fall - so the ~10x spread in sampling frequency between the arms does not bias the comparison. Workload. Under T (memory.max=160M, memory.low=64M, swap off): a victim A (32M file working set re-read on a 500ms pace, hooks attached), a filler B (112M of file cache rewritten continuously - a renewable flow of cold pages that keeps satisfying T's reclaim while A is protected), and a one-shot 64M anonymous burst C that is unreclaimable without swap and raises A's share of the scan. Metric: A's pgsteal over a 30s window. Protocol: 15 rounds, arms in rotating order, a sanity gate every round that verifies the protection chain before any round counts, cache dropped between runs, every knob identical across arms and rounds; then a sweep of U's trigger interval (1/5/20/100ms timer and PSI poll, five repetitions per point, reported as medians). QEMU VM, 8 GiB, no swap, booted with psi=1. Results (medians [IQR], 15 rounds per arm): stolen from A B2 452 [339-755] U 3723 [2825-4730] B1 3960 [3211-4853] - B2's median is 8.2x below U's and 8.8x below B1's, and the IQRs do not overlap (Mann-Whitney p < 1e-4). In 14 of 15 rounds B2's worst (968 pages) stayed below both other arms' best round (2051 / 2615). - The tails overlap only where the kernel itself broke protection: B2's outlier round logged 920 MEMCG_LOW overrides, and B1's own outlier - 9303 pages, the worst single round in the dataset - logged 352. That is memory.low's documented soft ceiling: reclaim legally overrides it when T's max cannot be satisfied from unprotected supply. It applies to every arm and to any policy alike; memory.min is the hard guarantee and is out of scope here. - Overhead: B2's entire in-hook cost is 2.9ms per 30s window (~56k calls at 52ns average, 0.01% of one core); U's agent costs 140ms for ~6.7k decisions and 187 memory.low writes; B1's hook is 1.7ms plus ~7k wq timer callbacks whose CPU is not in that figure. - U and B1 are statistically indistinguishable. Moving the action into the kernel buys nothing by itself; what makes the difference is deciding at the collection point. - The sweep answers "poll faster". U at 1ms leaks a median 1308 pages at 374ms of agent CPU per window; at 5ms, 5330 at 138ms; at 20ms, 7746 at 34ms; at 100ms it engages in only 2 of 5 windows and by then the drain has happened. B2 beats U's best point (452 vs 1308) at 1/129 of that point's CPU. - The PSI-poll trigger never fired in any window - a publishable negative result, not a bug: the victim stalls 0.3-0.6 ms/s, an order of magnitude below the 10 ms/s trigger, while the filler stalls 39-44 ms/s. The cgroup being stolen from is the last place PSI-on-the-victim can see it. - Time to first protection is statistically identical across the arms (medians 2.1-2.8s): under this pacing it measures when a drain episode happens, not how fast an arm reacts inside one, so I do not rank arms by it. In short: the same policy, decided at the collection point instead of out of band, cut the victim's losses ~8x at 2% of the userspace agent's CPU, and beat even the agent's most aggressive polling point while costing 129x less CPU. The scripts and the protocol description are in [4]; the raw CSV rows and per-run logs behind the numbers above are at [5]. THE PATCHES 1/2 memcg_ext: arbitrate low protection requests like memory.low struct page_counter grows low_req; low_setting() becomes max(low, low_req) everywhere protection is derived; and the bpf_memcg_ops callback moves from mem_cgroup_below_low() to mem_cgroup_calculate_protection(), so the request is collected before elow is computed. The request is recomputed every pass and reported as none whenever no program can run, so detaching the last program drops what it asked for on the next pass - nothing to clean up. 2/2 selftests/bpf: Add bpf_memcg_ops protection test A three-level tree, a policy on the grandchild requesting its current usage, memory.low on the middle cgroup as the budget, and reclaim from the top. Three budgets, same request: with 0 the effective value stays 0; with 2M an 8M request comes back capped to 2M and reclaim still goes into the victim; with 16M it is granted in full, reclaim skips the victim and memory.events low moves. The ctx contents are checked along the way. [1] https://lore.kernel.org/all/cover.1793086858.git.zhuhui@kylinos.cn/ [2] https://lore.kernel.org/all/20260921192559.2619635-1-shakeel.butt@linux.dev/ [3] https://lore.kernel.org/all/cover.1779760876.git.zhuhui@kylinos.cn/ [4] https://github.com/teawater/memcg-async-reclaim/tree/memcg_ext_20261109 [5] https://github.com/teawater/memcg-async-reclaim/tree/memcg_ext_20261109/log/run-20261111-071332 Hui Zhu (2): memcg_ext: arbitrate low protection requests like memory.low selftests/bpf: Add bpf_memcg_ops protection test include/linux/bpf_memcontrol.h | 97 +++++- include/linux/page_counter.h | 8 + mm/bpf_memcontrol.c | 155 ++++++--- mm/memcontrol.c | 30 ++ mm/page_counter.c | 51 ++- .../bpf/prog_tests/memcg_ops_protection.c | 297 ++++++++++++++++++ .../bpf/progs/memcg_ops_protection.c | 82 +++++ 7 files changed, 678 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/memcg_ops_protection.c create mode 100644 tools/testing/selftests/bpf/progs/memcg_ops_protection.c -- 2.43.0