Re: [PATCH 0/3] mm/mempolicy: add per-cgroup weighted interleave controls

["Li Zhe" <lizhe.67@bytedance.com>]

On 11/4/26 2:42 PM, Gregory Price wrote:
> On Wed, Nov 04, 2026 at 10:44:46AM +0800, Li Zhe wrote:
>> Does that distinction make the cgroup-scoped override acceptable? If not,
>> could you suggest a better way to support this kind of administrative
>> weight override without adding new mempolicy syscalls?
>>
> I'm a little confused why cgroup-anything is needed here when you could
> just override the weight table / interleave functions.
>
> I don't particularly like the idea of bpf hooks for a single mempolicy
> mode as opposed to something more generally useful - and I question
> how useful task-mempolicies are in general for tiered memory systems
> (they also affect kernel allocation placement, which is not great).
>
> You have many patch proposals recently trying to adjust the mempolicy
> components and I think it might be helpful if we could chat a bit
> about what big problems you're actually trying to solve.
>
> I get that you want better control over task/task-groups, i think that
> can be useful - but I also know that demotion immediately mucks all
> this placement logic up (and we can't check vma / task policies at
> demotion time) so I question how realisitic the value of these
> extensions will be.


Thanks for the feedback.

Our target environment is a class of machines with both DRAM and CXL
memory. On these machines, different services run together in different
cgroups. For us, CXL memory has two main values: it increases the total
memory capacity of the machine, and it lowers the memory cost for
workloads that need a large resident memory footprint. DRAM is the
faster but also more expensive tier, so the practical question becomes:
how much DRAM does each workload really need, and how much of the rest
can be placed on CXL memory?

 From our earlier testing, different workloads have different suitable
DRAM/CXL ratios. By "suitable", I do not mean maintaining a fixed ratio
for its own sake. I mean that, after using some CXL memory, the
workload's important metrics, such as throughput, average latency, tail
latency, or other SLOs, are still within an acceptable range.

So the core problem I am trying to solve is: on a mixed DRAM/CXL memory
machine, each workload in each cgroup should be able to run with its own
suitable DRAM/CXL ratio, so that it can stay near its own
performance/cost balance point.

This is why a single system-wide weighted interleave table is not enough
for our use case. Two co-located services may have completely different
suitable DRAM/CXL ratios. The cgroup is also the more natural management
unit for us, because resource isolation, charging, SLO monitoring, and
userspace control logic are all organized by cgroup, not by individual
task.

I understand that Joshua's approach can solve part of this problem.
However, based on the current discussion, that direction still seems to
need more discussion before it can move forward upstream.

Of course, initial allocation is not the whole story. Demotion and
promotion will change the actual page placement over time, so I do not
expect weighted interleave alone to provide long-term placement control.
For long-term adjustment, using a userspace controller with
memory.reclaim is a more natural and convenient way to adjust placement
over time.

The value of weighted interleave here is in the initial allocation
phase. First, for bandwidth-sensitive workloads, it allows DRAM and CXL
bandwidth to be used together from startup, instead of concentrating
allocations on one tier first and relying on later reclaim or demotion to
reshape the distribution. Second, it gives the container scheduler a
more deterministic view of memory capacity, so the scheduler can place
workloads based on the total memory capacity provided by both DRAM and
CXL, rather than treating only DRAM as the schedulable capacity.

In the long term, we want to maintain the target ratio through a dynamic
control loop. Memory tiering promotes hot pages back to DRAM. A
userspace controller observes each cgroup's per-tier memory footprint
and workload metrics, decides whether the current ratio has deviated
from the target, and then uses memory.reclaim to apply reclaim pressure
to the source tier that should be relieved [1].

The target ratio is not meant to be a strict hard limit. The userspace
controller can adjust the policy based on workload metrics: when cost is
the priority, it can keep more cold memory on CXL; when performance or
SLO pressure becomes high, it can temporarily relax the ratio and allow
the workload to use more DRAM. In other words, I am not trying to make
demotion reinterpret task policy or VMA policy. I am trying to adjust
the actual resident distribution at cgroup granularity through a
userspace feedback loop.

This is also the background for my other patchsets. The memory.reclaim
source selector is intended to give the userspace controller a more
precise actuator [1]. In our testing, this is an important capability for
building the feedback loop.

The sched/numa memory tiering patchset came from issues we found while
testing this model [2]. Its goal is to make tiering promotion behavior
adapt better to different scenarios, and to avoid tiering-only feedback
being misused by the normal AutoNUMA task placement heuristic.

I am not attached to the exact API shape in this RFC. If the
per-cgroup weighted interleave BPF kfuncs are too narrow, or if you
think there is a more general control plane, or a better control
granularity, I am happy to adjust the direction.

[1] 
https://lore.kernel.org/all/20261102070440.16262-1-lizhe.67@bytedance.com/
[2] 
https://lore.kernel.org/all/20261028041253.74839-1-lizhe.67@bytedance.com/

Thanks,
Zhe

>
> ~Gregory