Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Ben Horgan <ben.horgan@arm.com>]

Hi Reinette,

On 7/16/26 17:04, Reinette Chatre wrote:
> Hi Ben, Chenyu, and Tony,
> 
> On 7/16/26 7:59 AM, Ben Horgan wrote:
>> On 7/15/26 16:41, Reinette Chatre wrote:
>>> On 7/15/26 1:34 AM, Ben Horgan wrote:
>>>> On 7/14/26 23:06, Reinette Chatre wrote:
> 
> ...
> 
>>>>> Beyond this I believe that MPAM currently emulates the MB control with its "MB_MAX" control and users may want
>>>>> to make bandwidth allocations at the fine granularity that it supports. Taking this into account the interface
>>>>> may end up looking like:
>>>>>
>>>>> info/
>>>>> └── MB/
>>>>>     └── resource_schemata/
>>>>>         ├── MB/
>>>>>         │   ├── MB_MAX/
>>>>>         │   │   └── scope:L3
>>>>>         │   └── scope:L3
>>>>>         └── MB_NODE/
>>>>>             └── scope:NODE
>>>>>
>>>>> A system like above will thus have three schemata file entries:
>>>>> MB
>>>>> MB_MAX
>>>>> MB_NODE
>>>>>
>>>>> Three schemata file entries would be unnecessary for users familiar with the finer granularity MB_MAX control
>>>>> so that is where the "mode" file can be used to disable the legacy MB control to just expose MB_MAX and MB_NODE
>>>>> on these systems.
>>>>>> Would that work for these systems?
>>>>
>>>> I'm not clear on the behaviour when the schemata file contains both MB and MB_MAX entries. I assume
>>>> we should expect that the user might read back the whole file, modify what they expect to change and
>>>> then write everything back. If legacy software does this and wants to change the MB schemata then
>>>> the MB and MB_MAX values would be incompatible. Will we just ignore values that haven't changed and
>>>> fail the write if two dependent schemata are changed at the same time?
>>>
>>> An alternative would be to follow Dave Martin's suggestion to prefix emulated controls in schemata file
>>> with "#":
>>> https://lore.kernel.org/lkml/aNv53UmFGDBL0z3O@e133380.arm.com/
>>
>> I was aware of this but I had (mis)thought that the "mode" file was the alternative to this that was
>> being taken forward.
>>
>>> This was one element of original discussion that fizzled a bit since it was not obvious that user
>>> space would behave like this. Consider, for example, from Documentation/filesystems/resctrl.rst:
>>>
>>> 	Reading/writing the schemata file
>>> 	---------------------------------
>>> 	... When writing you only need to specify those values which you wish to change. ..
>>
>> Yeah, I don't know but it does seem like something a user could reasonably expect to work.
> 
> ok. 
> 
>>>> Alternatively, could the "mode" file be used to switch between "MB" and "MB_MAX" and the two never
>>>> need be shown at the same time. The user opts in to using the new interface, "MB_MAX" by setting
>>>> "mode" and can just toggle back if they want to use "MB" directly again.
>>> Interesting. So far the "mode" options have been "legacy" and "native" where "legacy" would show the
>>> legacy as well as emulated controls in the schemata file and "native" will only show the new controls.
>>> resctrl could make the default of "legacy" mean that *only* the legacy control is shown without insight
>>> into the controls it is being emulated with. The only insight to this would continue to be via the
>>> hierarchy in the info directory, which would show relationship but not the actual control values.
>>> Do you think there could be a need for users (excluding validation?) that may want to see the underlying
>>> control values used to emulate a legacy control?
>>
>> They may want to see the underlying values to be able to migrate there legacy configuration to the
>> native configuration but they can just write their legacy configuration and then toggle the mode to
>> see the values in the native mode.
>>
>>>
>>> I assume for backward compatibility that "legacy" would remain the default so a possible inconvenience
>>> here would be that users familiar with the new controls would forever need to switch the mode before being
>>> able to use them.
>>
>> This does also make the new controls slightly harder to discover.
>>
> What if resctrl combines the two suggestions? Specifically, for backward compatibility the mode will be
> "legacy" and if there are emulated controls then resctrl displays them in schemata file with "#" prefix
> but not(*) support any changes from user space to the underlying hardware controls. This will make new controls
> easier to discover and let user space see the underlying values, but not break a user space that
> may read schemata file, change a few values, and write entire file back.

My initial impression is that this would work but is probably unnecessary.

> 
> How the "enabled" vs "disabled" state of a control works with this needs some confirmation since region-aware
> MBA support has this extra caveat of supporting MSR and ACPI interfaces which results in the relationship
> between "mode" and "status" of underlying controls not being consistent. This may be ok but please consider
> example below.
> 
> Thinking through this with examples as I understand MPAM and RDT region aware so far. Could this work for
> MPAM and region aware MBA?
> 
> (*) Should resctrl allow user space to change underlying control value of an emulated control in "legacy" mode?

I don't think this causes problems for MPAM but for RDT region aware controls couldn't you end up
with a control state that isn't reachable just by configuring the legacy schema.

> 
> MPAM
> ====
> 1) System default
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_MAX/
>         │   │   └── status:enabled
>         │   └── status:enabled
>         └── mode:[legacy] native
> 
> schemata file
> -------------
> MB:...
> # MB_MAX:...
> 
> - In above, user space attempting to change MB_MAX via schemata file with or without "#" prefix will have no effect.
>   TBD: there should not be obstacles to resctrl supporting user space changes by removing the "#" prefix.

Perhaps a different prefix for read only lines as opposed to commented lines.

Thanks,

Ben

> - MB_MAX control values would show the current active control values of MB_MAX control used to emulate the MB control
> 
> 2) User space switches to fine grained controls
> 
> # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_MAX/
>         │   │   └── status:enabled
>         │   └── status:disabled
>         └── mode:legacy [native]
> 
> schemata file
> -------------
> MB_MAX:...
> 
> - Legacy control not displayed at all
> - no # prefix
> 
> RDT region aware when using MSR interface for legacy controls
> =============================================================
> 1) System default
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_REGION0_MAX/
>         │   │   └── status:disabled
>         │   ├── MB_REGION1_MAX/
>         │   │   └── status:disabled
>         │   ├── MB_REGION2_MAX/
>         │   │   └── status:disabled
>         │   ├── MB_REGION3_MAX/
>         │   │   └── status:disabled
>         │   └── status:enabled
>         └── mode:[legacy] native
> 
> schemata file
> -------------
> MB:...
> 
> 2) User space switches to fine grained controls
> 
> # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_REGION0_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION1_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION2_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION3_MAX/
>         │   │   └── status:enabled
>         │   └── status:disabled
>         └── mode:legacy [native]
> 
> schemata file
> -------------
> MB_REGION0_MAX:...
> MB_REGION1_MAX:...
> MB_REGION2_MAX:...
> MB_REGION3_MAX:...
> 
> 
> RDT region aware when using ACPI interface for legacy controls
> =============================================================
> 1) System default
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_REGION0_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION1_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION2_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION3_MAX/
>         │   │   └── status:enabled
>         │   └── status:enabled
>         └── mode:[legacy] native
> 
> schemata file
> -------------
> MB:...
> # MB_REGION0_MAX:...
> # MB_REGION1_MAX:...
> # MB_REGION2_MAX:...
> # MB_REGION3_MAX:...
> 
> 2) User space switches to fine grained controls
> 
> # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode
> 
> info hierarchy
> --------------
> info/
> └── MB/
>     └── schemata/
>         ├── MB/
>         │   ├── MB_REGION0_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION1_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION2_MAX/
>         │   │   └── status:enabled
>         │   ├── MB_REGION3_MAX/
>         │   │   └── status:enabled
>         │   └── status:disabled
>         └── mode:legacy [native]
> 
> schemata file
> -------------
> MB_REGION0_MAX:...
> MB_REGION1_MAX:...
> MB_REGION2_MAX:...
> MB_REGION3_MAX:...
> 
> Reinette