Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Ben,

On 9/21/26 9:53 AM, Ben Horgan wrote:
> On 21/09/2026 17:06, Reinette Chatre wrote:
>> On 9/21/26 3:22 AM, Ben Horgan wrote:
>>> On 18/09/2026 23:47, Reinette Chatre wrote:

...>>>>
>>>> Are you instead saying that resctrl should expose a node scoped control as that of a L3 scope control?
>>>
>>> The problem is that this already happens, for MPAM, in the degenerate case. This is the consequence
>>> of allowing memory MSC to pretend to be L3 scoped MSC and so represented by the MB resource. This
>>> only happens when there is a single L3 and a single NUMA node with no caches in between so that the
>>> traffic at either end of the path is effectively equivalent and the scope for both is global as you
>>> can't turn off the last CPU or the only NUMA node. In want of a better term, let's call this "L3 is
>>> NUMA". These shenanigans are what lead to the complexity of having more that one MPAM class backing
>>> a resource which we discuss at [1]. The need for the association is that, where possible, the memory
>>> allocation and memory bandwidth counters are exposed to the user using the existing resctrl
>>> interface and, as such, existing software will continue to work.
>>>
>>> What I was suggesting, is that in "native" mode we do away with any pretending. That is, memory MSC
>>> always back resources with NODE scope even in the degenerate "L3 is NUMA" case. One benefit of this
>>> is that the resctrl hierarchy can be more correct. I was about to write that a second benefit was
>>> that if there is a L3 cache MPAM class and a memory MPAM class which both provide memory allocation
>>> they could both be displayed in resctrl, MB and MB_NODE. The same consideration applies for memory
>>> bandwidth counters. However, as the MPAM driver in the "L3 is NUMA" case, always chooses the L3 MPAM
>>> class to back MB over other classes, the MPAM driver can just be extended to back MB_NODE with the
>>> memory class that was not chosen. In this way it will works the same way as in the non-degenerate
>>> cases and the same mechanism is in place for memory bandwidth counters.
>>>
>>> Hence, the only benefit would be a slightly closer description of the hardware in resctrl in the "L3
>>> is NUMA" case. In performing this exercise of laying out my reasons for allowing the native/legacy
>>> control to move things across resources  I see it is unnecessary and so the native/legacy switch
>>> should, as per your proposal, *not* move controls between resources. Thank you for your patience.
>>>
>>> [1] https://lore.kernel.org/linux-arm-kernel/0b0eef2d-ce3b-48c8-af08-5889e7396e4f@arm.com/
>>
>> If I understand correctly an "L3 is NUMA" would be presented to user space as:
>>
>> info/
>> ├── MB/
>> │   └── schemata/
>> │       ├── MB/
>> │       │   └── MB_L3_MAX/
>> │       └── MB_L3_MIN/
>> └── MB_NODE/
>>     └── schemata/
>>         ├── MB_NODE_MAX/
>>         └── MB_NODE_MIN/
>>
>> with the "MB" resource backed by the "L3 cache MPAM class" and the "MB_NODE" resource backed
>> by the "memory MPAM class". Is this getting close?
> 
> In the case when there is an L3 cache MPAM class and a memory MPAM class both support memory
> bandwidth allocation then yes. However, I don't know of systems like this today and this isn't
> dependent on the "L3 is NUMA". If there are multiple L3 and multiple NUMA nodes you would still get
> this structure. It just relies on both the L3 cache MPAM class and the memory MPAM class supporting
> bandwidth allocation (mbw_max/mbw_min). The "L3 is NUMA" is all about pretending a memory MPAM class
> is a L3 cache MPAM class so that it fits into the existing expectations of the resctrl interface.

I am missing something

From earlier description ("... the complexity of having more that one MPAM class backing a resource
which we discuss at [1]... ") I understood that "L3 is NUMA" scenario has two classes available and
currently just uses one to back the MB resource. Further, that the new interface makes it possible
for "the MPAM driver can just be extended to back MB_NODE with the memory class that was not chosen".

Above seems to state that such systems that have two classes do not exist?

My concern is the impact on the user interface. When MPAM "stops pretending", could a system that
exposes an MB resource before the new interface boot into new kernel with the new interface and no
longer have an MB resource but instead just have an MB_NODE resource?

>> It sounds to me as though both resources would be able to concurrently support monitoring so
>> resctrl can be expected to expose NODE scoped memory bandwidth monitoring events retrieved from
>> the "memory MPAM class" and L3 scoped memory bandwidth monitoring events retrieved from the
>> "L3 cache MPAM class"?
> 
> Yes. Unless we are in a "L3 is NUMA" setup to expose memory bandwidth counters from a memory MPAM
> class they would need to have node scope and they could co-exist with other memory bandwidth
> counters at L3 scope.

It is starting to sound more and more that the new interfaces may result in user interface changes
on MPAM systems?

Apologies but I do not think the MPAM plan is clear to me.
>> So far I've viewed support for monitoring events at different scope as "after multiple controls"
>> work that the "CPU-less NUMA nodes" depends on but looks like it is needed sooner.
> 
> Why do you think it needs to be sooner rather than just a next step? Is there some contradiction or
> incompatibility I'm missing. I thought that the monitors would just remain unexposed to resctrl
> until there is an interface.

No, you are not missing anything. My comment was based on my realization that the monitoring scope
changes are needed by more systems than I originally thought.

Reinette