Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Ben,

On 7/15/26 1:34 AM, Ben Horgan wrote:
> On 7/14/26 23:06, Reinette Chatre wrote:
>> On 7/10/26 1:59 PM, Fenghua Yu wrote:
>>> On 6/25/26 08:43, Reinette Chatre wrote:
>>>> On 6/24/26 6:26 PM, Fenghua Yu wrote:
>>>>> On 6/24/26 15:22, Reinette Chatre wrote:
>>>>>> On 6/24/26 12:08 PM, Fenghua Yu wrote:
>>>>>>> On 5/29/26 11:06, Reinette Chatre wrote:
>>>>>>>
>>>>>>> As Shaopen and Ben mentioned earlier, we are working on two MPAM
>>>>>>> features that may need to change schemata interface. The CPU-less
>>>>>>> feature was discussed on LPC (although the interfaces will be
>>>>>>> slightly different from the LPC).
>>>>>>
>>>>>> I know. Here is where I tried to engage with you on needed interfaces after LPC:
>>>>>> https://lore.kernel.org/lkml/fb1e2686-237b-4536-acd6-15159abafcba@intel.com/
>>>>>
>>>>> MPAM ACPI defines MSC (Memory System Control) is defined in one of two ways (not both) on one platform:
>>>>> 1. L3 and memory together on each processor MSC
>>>>> 2. L3 in processor MSC and memory control/monitoring in different memory MSCs.
>>>
>>> Ben said there is type 3 platform:
>>> 3. L3 cache and memory bandwidth in processor MSCs and memory bandwidth in different memory MSCs.
>>>
>>>>>
>>>>> On type 1 platform, schemata is legacy:
>>>>> MB:1=100;2=100  <-- cache id 1 and 2 as domain id
>>>>>
>>>>> On type 2 platform, I will not reuse "MB:" name. Instead, define new resource name "MBN:" for numa node and schemata is:
>>>>> MBN:0=100;1=100;2=100;10=100;18=100;26=100 <-- numa id 0, 1, 2, 10, 18,
>>>>>                             26 as domain id
>>>>> On type 2 platform, there won't be "MB:" line. Numa 0 and 1
>>>>> are for mbm allocation on socket 0 and 1. 2,10, 18 and 26 are for GPU
>>>>> memory nodes allocation.
>>>
>>> On type 3 platform, there could be "MB:" line for L3 cache and "MB_NODE:" for numa node. Example schemata is:
>>>
>>>      MB:1=100;2=100                 <-- cache id 1 and 2 as domain id
>>> MB_NODE:0=100;1=100;2=100;10=100;18=100;26=100 <-- numa id 0, 1, 2, 10,
>>>                                                    18, 26 as domain id
>>>>
>>>> (to help make things explicit I will refer to what you call "MBN" as "MB_NODE" to make it
>>>> explicit that it is memory bandwidth allocation at node scope)
>>>>
>>>> I am trying to consider how this can be accomplished while also considering all the other
>>>> new hardware features that resctrl need to support. Consider, for example, AMD's "Global
>>>> MBA" (https://lore.kernel.org/lkml/cover.1776980182.git.babu.moger@amd.com/) that throttles
>>>> memory bandwidth at L3 scope but the user configures allocations at NODE scope. At this time
>>>> the plan is to support this with a second control associated with the MB resource that can
>>>> allocate memory bandwidth at node scope. See
>>>> https://lore.kernel.org/lkml/430ffb48-29f4-44d9-9164-9f8b743b2739@amd.com/
>>>>
>>>> If resctrl creates a new resource for node scoped memory bandwidth allocations to support these
>>>> "type 2" systems then that will result in an inconsistent interface between architectures that
>>>> we should avoid.
>>>>
>>>> Have you been listening in on the discussions surrounding emulated controls? Considering that,
>>>> would it be possible to support the "MB" control on a type "2" system but have it be backed by
>>>> (emulated by) the underlying "MB_NODE" control?
>>>>
>>>> resctrl could expose both controls on these "type 2" systems but make it clear that "MB"
>>>> is emulated by "MB_NODE". For example:
>>>>
>>>> info/
>>>> └── MB/
>>>>      └── resource_schemata/
>>>>          └── MB/
>>>>              └── MB_NODE/
>>>>
>>>> User will see both controls in schemata file but when changes are made to "MB" control it
>>>> will show in the "MB_NODE" control and vice-versa. User could also disable the "MB" control
>>>> that will establish familiarity with the interface at which point resctrl can drop the
>>>> "MB" control from the schemata file on these "type 2" systems.
>>>>
>>>> Having the MB resource available with an MB control will keep resctrl backward compatible
>>>> if there are any tools that expect that. If backward compatibility is not of concern then
>>>> resctrl could initialize with the emulated control disabled by default. See discussion at
>>>> https://lore.kernel.org/lkml/5e575bc2-e67f-4696-9332-33c54023c057@intel.com/
>>>> that describes a new resctrl capability in support of RISC-V and RDT.
>>>> With this resctrl could initialize with:
>>>>
>>>> info/
>>>> └── MB/
>>>>      └── resource_schemata/
>>>>          ├── MB/
>>>>          │   ├── MB_NODE/
>>>>          │   │   └── status:enabled
>>>>          │   └── status:disabled
>>>>          └── mode:legacy [native]
>>>>
>>>> With above a "type 2" system will boot with its schemata file just containing the "MB_NODE"
>>>> control while info/MB describes the memory bandwidth resource.
>>>>
>>>
>>> On type 3 machine, schemata has both MB in legacy mode with cache id as domain id and MB_NODE with numa id as domain id.
>>>
>>> Is this directory OK?
>>>
>>>  info/
>>>  └── MB/
>>>       └── resource_schemata/
>>>           ├── MB/
>>>           │   ├── MB_NODE/
>>>           │   │   └── status:disabled
>>>           │   └── status:enabled
>>>           ├── MB_NODE/
>>>           └── mode:node
>>>
>>> 1. MB and MB_NODE are shown in parallel in inf/MB/resource_schemata/
>>
>> I do not think there is a need to expose an emulated MB_NODE control if the actual MB_NODE
>> hardware control exists. 
>>
>>> 2. mode is set as "node" meaning "MB" is for L3 and "MB_NODE" is for numa node
>>
>> I assume you mean "scope" instead of "mode"? (more below)
>>
>>> 3. Emulation "MB_NODE" is disabled (or should the "MB_NODE" sub-dir be invisible?)
>>
>> Right, I do not think emulation is needed here. No need to make it invisible since it should not exist.
>>
>> From what I understand these "type 3" machines could be simplified to:
>>
>> info/
>> └── MB/
>>     └── resource_schemata/
>>         ├── MB/
>>         │   └── scope:L3
>>         └── MB_NODE/
>>             └── scope:NODE
>>
>> Beyond this I believe that MPAM currently emulates the MB control with its "MB_MAX" control and users may want
>> to make bandwidth allocations at the fine granularity that it supports. Taking this into account the interface
>> may end up looking like:
>>
>> info/
>> └── MB/
>>     └── resource_schemata/
>>         ├── MB/
>>         │   ├── MB_MAX/
>>         │   │   └── scope:L3
>>         │   └── scope:L3
>>         └── MB_NODE/
>>             └── scope:NODE
>>
>> A system like above will thus have three schemata file entries:
>> MB
>> MB_MAX
>> MB_NODE
>>
>> Three schemata file entries would be unnecessary for users familiar with the finer granularity MB_MAX control
>> so that is where the "mode" file can be used to disable the legacy MB control to just expose MB_MAX and MB_NODE
>> on these systems.
>>> Would that work for these systems?
> 
> I'm not clear on the behaviour when the schemata file contains both MB and MB_MAX entries. I assume
> we should expect that the user might read back the whole file, modify what they expect to change and
> then write everything back. If legacy software does this and wants to change the MB schemata then
> the MB and MB_MAX values would be incompatible. Will we just ignore values that haven't changed and
> fail the write if two dependent schemata are changed at the same time?

An alternative would be to follow Dave Martin's suggestion to prefix emulated controls in schemata file
with "#":
https://lore.kernel.org/lkml/aNv53UmFGDBL0z3O@e133380.arm.com/
This was one element of original discussion that fizzled a bit since it was not obvious that user
space would behave like this. Consider, for example, from Documentation/filesystems/resctrl.rst:

	Reading/writing the schemata file
	---------------------------------
	... When writing you only need to specify those values which you wish to change. ..


> Alternatively, could the "mode" file be used to switch between "MB" and "MB_MAX" and the two never
> need be shown at the same time. The user opts in to using the new interface, "MB_MAX" by setting
> "mode" and can just toggle back if they want to use "MB" directly again.
Interesting. So far the "mode" options have been "legacy" and "native" where "legacy" would show the
legacy as well as emulated controls in the schemata file and "native" will only show the new controls.
resctrl could make the default of "legacy" mean that *only* the legacy control is shown without insight
into the controls it is being emulated with. The only insight to this would continue to be via the
hierarchy in the info directory, which would show relationship but not the actual control values.
Do you think there could be a need for users (excluding validation?) that may want to see the underlying
control values used to emulate a legacy control?

I assume for backward compatibility that "legacy" would remain the default so a possible inconvenience
here would be that users familiar with the new controls would forever need to switch the mode before being
able to use them.

Having this clear distinction could simplify the implementation.

Reinette