From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EDB7343030C for ; Thu, 16 Jul 2026 16:44:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784220280; cv=none; b=pfk4ictsAr26Av9RYYuyY2MAdEKX8ZAc/vwZMRaO01T504m6SWjLND/6UtNbHV9wRSX4ALkwxuvI0lZsClKiFYtS0ZjIGXEj1fhQt21MpFMfN6L7zZZedDVqixl+sM9W0JcdopJIYWZF2MMaPDkKfoWJUnb6pL6+MbdC7e+wiBE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784220280; c=relaxed/simple; bh=le6ABtjUnHVVonR1npjSyuQKJrUY04pFTvyKBGqITF0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=rH7+xXjhKocxADA4GRVqi8lRl8vX2tJQ8EZHMmswktwxJhurC2g0iiW3X660Y+iCQ6yCa6i5kSmUIz99Rdoio1Va5vi0lMEDRZp5Q4chQN3Yd7mldO7XPTBKseqsXdbeNVinrmkKMJXxqIfj6xhTl9jD5ElgF+c/p6uqq7nraaY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=iCOYSaNv; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="iCOYSaNv" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 387A91476; Thu, 16 Jul 2026 09:44:26 -0700 (PDT) Received: from [10.2.212.8] (e134344.arm.com [10.2.212.8]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 88AD63F7B4; Thu, 16 Jul 2026 09:44:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1784220270; bh=le6ABtjUnHVVonR1npjSyuQKJrUY04pFTvyKBGqITF0=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=iCOYSaNvZq1pGyo5hD6CYUHDhReZ/cdLPNybb9A7E/qk0BuBr/y+WZyKFbRcGmhNy pBSmZKHPfZfm62lYchIqLaaTMMDQcmzHrna8fAVN+A20pio3YvPZzN5BW0+rl2iPkR LCvPgsWhJS9fQYqtZJdKvP8gXrX86Pw7j0sNIkPM= Message-ID: <749bd904-523d-4e9d-8493-0e8cfd79949e@arm.com> Date: Thu, 16 Jul 2026 17:44:27 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Thunderbird Daily Subject: Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept To: Reinette Chatre , Fenghua Yu , Tony Luck , James Morse , Dave Martin , Babu Moger , Drew Fustini , Chen Yu Cc: Borislav Petkov , Thomas Gleixner , Dave Hansen , Peter Newman , "x86@kernel.org" , "linux-kernel@vger.kernel.org" References: <5ee87762-1898-4b62-94da-85b3e9917ecc@intel.com> <62701203-c4a3-4ec2-a9af-602e1fc15863@nvidia.com> <8f9f78dd-e3f5-4b35-bc72-0eb5dafdcedf@nvidia.com> <36163a81-9737-49e3-93ef-6c392f7272f0@intel.com> <0fc6df54-26c7-43fa-948a-528cd94937f1@arm.com> <9049378c-699a-4155-b1e4-737a1d7265d5@intel.com> <57740b97-80ee-4632-bca3-dc43cd7776c2@arm.com> <44f26cd4-be79-476e-b002-7ccfb7705179@intel.com> Content-Language: en-US From: Ben Horgan In-Reply-To: <44f26cd4-be79-476e-b002-7ccfb7705179@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi Reinette, On 7/16/26 17:04, Reinette Chatre wrote: > Hi Ben, Chenyu, and Tony, > > On 7/16/26 7:59 AM, Ben Horgan wrote: >> On 7/15/26 16:41, Reinette Chatre wrote: >>> On 7/15/26 1:34 AM, Ben Horgan wrote: >>>> On 7/14/26 23:06, Reinette Chatre wrote: > > ... > >>>>> Beyond this I believe that MPAM currently emulates the MB control with its "MB_MAX" control and users may want >>>>> to make bandwidth allocations at the fine granularity that it supports. Taking this into account the interface >>>>> may end up looking like: >>>>> >>>>> info/ >>>>> └── MB/ >>>>> └── resource_schemata/ >>>>> ├── MB/ >>>>> │   ├── MB_MAX/ >>>>> │   │   └── scope:L3 >>>>> │   └── scope:L3 >>>>> └── MB_NODE/ >>>>> └── scope:NODE >>>>> >>>>> A system like above will thus have three schemata file entries: >>>>> MB >>>>> MB_MAX >>>>> MB_NODE >>>>> >>>>> Three schemata file entries would be unnecessary for users familiar with the finer granularity MB_MAX control >>>>> so that is where the "mode" file can be used to disable the legacy MB control to just expose MB_MAX and MB_NODE >>>>> on these systems. >>>>>> Would that work for these systems? >>>> >>>> I'm not clear on the behaviour when the schemata file contains both MB and MB_MAX entries. I assume >>>> we should expect that the user might read back the whole file, modify what they expect to change and >>>> then write everything back. If legacy software does this and wants to change the MB schemata then >>>> the MB and MB_MAX values would be incompatible. Will we just ignore values that haven't changed and >>>> fail the write if two dependent schemata are changed at the same time? >>> >>> An alternative would be to follow Dave Martin's suggestion to prefix emulated controls in schemata file >>> with "#": >>> https://lore.kernel.org/lkml/aNv53UmFGDBL0z3O@e133380.arm.com/ >> >> I was aware of this but I had (mis)thought that the "mode" file was the alternative to this that was >> being taken forward. >> >>> This was one element of original discussion that fizzled a bit since it was not obvious that user >>> space would behave like this. Consider, for example, from Documentation/filesystems/resctrl.rst: >>> >>> Reading/writing the schemata file >>> --------------------------------- >>> ... When writing you only need to specify those values which you wish to change. .. >> >> Yeah, I don't know but it does seem like something a user could reasonably expect to work. > > ok. > >>>> Alternatively, could the "mode" file be used to switch between "MB" and "MB_MAX" and the two never >>>> need be shown at the same time. The user opts in to using the new interface, "MB_MAX" by setting >>>> "mode" and can just toggle back if they want to use "MB" directly again. >>> Interesting. So far the "mode" options have been "legacy" and "native" where "legacy" would show the >>> legacy as well as emulated controls in the schemata file and "native" will only show the new controls. >>> resctrl could make the default of "legacy" mean that *only* the legacy control is shown without insight >>> into the controls it is being emulated with. The only insight to this would continue to be via the >>> hierarchy in the info directory, which would show relationship but not the actual control values. >>> Do you think there could be a need for users (excluding validation?) that may want to see the underlying >>> control values used to emulate a legacy control? >> >> They may want to see the underlying values to be able to migrate there legacy configuration to the >> native configuration but they can just write their legacy configuration and then toggle the mode to >> see the values in the native mode. >> >>> >>> I assume for backward compatibility that "legacy" would remain the default so a possible inconvenience >>> here would be that users familiar with the new controls would forever need to switch the mode before being >>> able to use them. >> >> This does also make the new controls slightly harder to discover. >> > What if resctrl combines the two suggestions? Specifically, for backward compatibility the mode will be > "legacy" and if there are emulated controls then resctrl displays them in schemata file with "#" prefix > but not(*) support any changes from user space to the underlying hardware controls. This will make new controls > easier to discover and let user space see the underlying values, but not break a user space that > may read schemata file, change a few values, and write entire file back. My initial impression is that this would work but is probably unnecessary. > > How the "enabled" vs "disabled" state of a control works with this needs some confirmation since region-aware > MBA support has this extra caveat of supporting MSR and ACPI interfaces which results in the relationship > between "mode" and "status" of underlying controls not being consistent. This may be ok but please consider > example below. > > Thinking through this with examples as I understand MPAM and RDT region aware so far. Could this work for > MPAM and region aware MBA? > > (*) Should resctrl allow user space to change underlying control value of an emulated control in "legacy" mode? I don't think this causes problems for MPAM but for RDT region aware controls couldn't you end up with a control state that isn't reachable just by configuring the legacy schema. > > MPAM > ==== > 1) System default > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │   ├── MB_MAX/ > │   │   └── status:enabled > │   └── status:enabled > └── mode:[legacy] native > > schemata file > ------------- > MB:... > # MB_MAX:... > > - In above, user space attempting to change MB_MAX via schemata file with or without "#" prefix will have no effect. > TBD: there should not be obstacles to resctrl supporting user space changes by removing the "#" prefix. Perhaps a different prefix for read only lines as opposed to commented lines. Thanks, Ben > - MB_MAX control values would show the current active control values of MB_MAX control used to emulate the MB control > > 2) User space switches to fine grained controls > > # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │ ├── MB_MAX/ > │ │ └── status:enabled > │ └── status:disabled > └── mode:legacy [native] > > schemata file > ------------- > MB_MAX:... > > - Legacy control not displayed at all > - no # prefix > > RDT region aware when using MSR interface for legacy controls > ============================================================= > 1) System default > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │   ├── MB_REGION0_MAX/ > │   │   └── status:disabled > │   ├── MB_REGION1_MAX/ > │   │   └── status:disabled > │   ├── MB_REGION2_MAX/ > │   │   └── status:disabled > │   ├── MB_REGION3_MAX/ > │   │   └── status:disabled > │   └── status:enabled > └── mode:[legacy] native > > schemata file > ------------- > MB:... > > 2) User space switches to fine grained controls > > # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │ ├── MB_REGION0_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION1_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION2_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION3_MAX/ > │ │ └── status:enabled > │ └── status:disabled > └── mode:legacy [native] > > schemata file > ------------- > MB_REGION0_MAX:... > MB_REGION1_MAX:... > MB_REGION2_MAX:... > MB_REGION3_MAX:... > > > RDT region aware when using ACPI interface for legacy controls > ============================================================= > 1) System default > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │ ├── MB_REGION0_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION1_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION2_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION3_MAX/ > │ │ └── status:enabled > │ └── status:enabled > └── mode:[legacy] native > > schemata file > ------------- > MB:... > # MB_REGION0_MAX:... > # MB_REGION1_MAX:... > # MB_REGION2_MAX:... > # MB_REGION3_MAX:... > > 2) User space switches to fine grained controls > > # echo "native" > /sys/fs/resctrl/info/MB/schemata/mode > > info hierarchy > -------------- > info/ > └── MB/ > └── schemata/ > ├── MB/ > │ ├── MB_REGION0_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION1_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION2_MAX/ > │ │ └── status:enabled > │ ├── MB_REGION3_MAX/ > │ │ └── status:enabled > │ └── status:disabled > └── mode:legacy [native] > > schemata file > ------------- > MB_REGION0_MAX:... > MB_REGION1_MAX:... > MB_REGION2_MAX:... > MB_REGION3_MAX:... > > Reinette