Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Ben Horgan <ben.horgan@arm.com>]

Hi Reinette,

On 7/16/26 18:07, Reinette Chatre wrote:
> Hi Ben,
> 
> On 7/16/26 9:44 AM, Ben Horgan wrote:
>> Hi Reinette,
>>
>> On 7/16/26 17:04, Reinette Chatre wrote:
>>> Hi Ben, Chenyu, and Tony,
>>>
>>> On 7/16/26 7:59 AM, Ben Horgan wrote:
>>>> On 7/15/26 16:41, Reinette Chatre wrote:
>>>>> On 7/15/26 1:34 AM, Ben Horgan wrote:
>>>>>> On 7/14/26 23:06, Reinette Chatre wrote:
>>>
>>> ...
>>>
>>>>>>> Beyond this I believe that MPAM currently emulates the MB control with its "MB_MAX" control and users may want
>>>>>>> to make bandwidth allocations at the fine granularity that it supports. Taking this into account the interface
>>>>>>> may end up looking like:
>>>>>>>
>>>>>>> info/
>>>>>>> └── MB/
>>>>>>>     └── resource_schemata/
>>>>>>>         ├── MB/
>>>>>>>         │   ├── MB_MAX/
>>>>>>>         │   │   └── scope:L3
>>>>>>>         │   └── scope:L3
>>>>>>>         └── MB_NODE/
>>>>>>>             └── scope:NODE
>>>>>>>
>>>>>>> A system like above will thus have three schemata file entries:
>>>>>>> MB
>>>>>>> MB_MAX
>>>>>>> MB_NODE
>>>>>>>
>>>>>>> Three schemata file entries would be unnecessary for users familiar with the finer granularity MB_MAX control
>>>>>>> so that is where the "mode" file can be used to disable the legacy MB control to just expose MB_MAX and MB_NODE
>>>>>>> on these systems.
>>>>>>>> Would that work for these systems?
>>>>>>
>>>>>> I'm not clear on the behaviour when the schemata file contains both MB and MB_MAX entries. I assume
>>>>>> we should expect that the user might read back the whole file, modify what they expect to change and
>>>>>> then write everything back. If legacy software does this and wants to change the MB schemata then
>>>>>> the MB and MB_MAX values would be incompatible. Will we just ignore values that haven't changed and
>>>>>> fail the write if two dependent schemata are changed at the same time?
>>>>>
>>>>> An alternative would be to follow Dave Martin's suggestion to prefix emulated controls in schemata file
>>>>> with "#":
>>>>> https://lore.kernel.org/lkml/aNv53UmFGDBL0z3O@e133380.arm.com/
>>>>
>>>> I was aware of this but I had (mis)thought that the "mode" file was the alternative to this that was
>>>> being taken forward.
>>>>
>>>>> This was one element of original discussion that fizzled a bit since it was not obvious that user
>>>>> space would behave like this. Consider, for example, from Documentation/filesystems/resctrl.rst:
>>>>>
>>>>> 	Reading/writing the schemata file
>>>>> 	---------------------------------
>>>>> 	... When writing you only need to specify those values which you wish to change. ..
>>>>
>>>> Yeah, I don't know but it does seem like something a user could reasonably expect to work.
>>>
>>> ok. 
>>>
>>>>>> Alternatively, could the "mode" file be used to switch between "MB" and "MB_MAX" and the two never
>>>>>> need be shown at the same time. The user opts in to using the new interface, "MB_MAX" by setting
>>>>>> "mode" and can just toggle back if they want to use "MB" directly again.
>>>>> Interesting. So far the "mode" options have been "legacy" and "native" where "legacy" would show the
>>>>> legacy as well as emulated controls in the schemata file and "native" will only show the new controls.
>>>>> resctrl could make the default of "legacy" mean that *only* the legacy control is shown without insight
>>>>> into the controls it is being emulated with. The only insight to this would continue to be via the
>>>>> hierarchy in the info directory, which would show relationship but not the actual control values.
>>>>> Do you think there could be a need for users (excluding validation?) that may want to see the underlying
>>>>> control values used to emulate a legacy control?
>>>>
>>>> They may want to see the underlying values to be able to migrate there legacy configuration to the
>>>> native configuration but they can just write their legacy configuration and then toggle the mode to
>>>> see the values in the native mode.
>>>>
>>>>>
>>>>> I assume for backward compatibility that "legacy" would remain the default so a possible inconvenience
>>>>> here would be that users familiar with the new controls would forever need to switch the mode before being
>>>>> able to use them.
>>>>
>>>> This does also make the new controls slightly harder to discover.
>>>>
>>> What if resctrl combines the two suggestions? Specifically, for backward compatibility the mode will be
>>> "legacy" and if there are emulated controls then resctrl displays them in schemata file with "#" prefix
>>> but not(*) support any changes from user space to the underlying hardware controls. This will make new controls
>>> easier to discover and let user space see the underlying values, but not break a user space that
>>> may read schemata file, change a few values, and write entire file back.
>>
>> My initial impression is that this would work but is probably unnecessary.
> 
> ok. My goal was to present ideas to address the issues raised so far. Ideally this would result in discussion of
> pros/cons. If you find this unnecessary, could you please expand with some insight into which parts you find
> unnecessary or how you would prefer this solved? I am finding it difficult to interpret above response.

Sorry, yes. I was a bit lazy in my reply yesterday. If the mode is easy and non-destructive to
change then doesn't the user get the same information with just the burden of toggling the mode and
then reading the schemata a second time. This does rely on the user being aware of the new behaviour
though.

If the emulation isn't a, one to one, bijection then is changing mode expected to be destructive?
More on this below.

> 
>>> How the "enabled" vs "disabled" state of a control works with this needs some confirmation since region-aware
>>> MBA support has this extra caveat of supporting MSR and ACPI interfaces which results in the relationship
>>> between "mode" and "status" of underlying controls not being consistent. This may be ok but please consider
>>> example below.
>>>
>>> Thinking through this with examples as I understand MPAM and RDT region aware so far. Could this work for
>>> MPAM and region aware MBA?
>>>
>>> (*) Should resctrl allow user space to change underlying control value of an emulated control in "legacy" mode?
>>
>> I don't think this causes problems for MPAM but for RDT region aware controls couldn't you end up
>> with a control state that isn't reachable just by configuring the legacy schema.
> 
> Could you please highlight the scenario you refer to?

My expectation here was that for any value of MB, X, the value for each of the regions would be the
same, Y. (Not sure if this is actually the case.)

MB: X
# MB_REGION0_MAX:Y
# MB_REGION1_MAX:Y
# MB_REGION2_MAX:Y
# MB_REGION3_MAX:Y

So, if one of the regions was to be changed individually then it would be in a state not reachable
by just changing the legacy control, MB. Potentially this complicated switching mode as well as
uncommenting and writing a schemata.

I was thinking that in legacy mode that the region values would always be kept the same but perhaps
the legacy (MB) value could just be the 'best estimate' and the native (MB_REGION0) values show the
full story. 'best estimate' could be difficult to choose but allows switching mode to be
non-destructive.

For MPAM, at least until Fenghua sent his series today on MBA control emulation [1], I was expecting
that MPAM would only use emulation for exposing finer grade controls to the user and not allowing
different underlying hardware to be used for the emulation. To me, it seems reasonable that a new
control would involve a new schema but I do appreciate the benefit of being able to continue to use
the old interface without stopping a new interface being used. The danger is that there are
unexpected user visible side effects, e.g. domain lifetime, and that control behaviour is different
from the user expectations.

[1] https://lore.kernel.org/linux-arm-kernel/cover.1784217438.git.fenghuay@nvidia.com/

>>
>>>
>>> MPAM
>>> ====
>>> 1) System default
>>>
>>> info hierarchy
>>> --------------
>>> info/
>>> └── MB/
>>>     └── schemata/
>>>         ├── MB/
>>>         │   ├── MB_MAX/
>>>         │   │   └── status:enabled
>>>         │   └── status:enabled
>>>         └── mode:[legacy] native
>>>
>>> schemata file
>>> -------------
>>> MB:...
>>> # MB_MAX:...
>>>
>>> - In above, user space attempting to change MB_MAX via schemata file with or without "#" prefix will have no effect.
>>>   TBD: there should not be obstacles to resctrl supporting user space changes by removing the "#" prefix.
>>
>> Perhaps a different prefix for read only lines as opposed to commented lines.
> 
> Which prefix do you have in mind?

I was avoiding that.. but perhaps for read only lines perhaps they could be in brackets as it hints
at it being an extra explanation.

(MB_MAX:...)


Thanks,

Ben

> 
> Reinette
> 
>