From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D59A4395DB8 for ; Wed, 7 Jan 2026 15:53:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767801218; cv=none; b=RxF4OSFu+/SxG81nYsUbo+9709+41jQk5C8o9kL04L55avLD/njKEX7qR17+yx6q5UYGbbjochMwz6z+2MRaD/1vKmOw1Jvz+/WumvxADr6L5RS4MBwspXRR4BLVQGrPho5aRrPho1W7u/uim+aTzkWuprPKHD/nFKTVUjM3/oE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767801218; c=relaxed/simple; bh=y/dlD3w3UDHtkfFyenDAmeW5wFTBsHVhM8LszroeKw0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tA/coppXT4GceCvWwD7Q7QWKO3Ta8QTBQDOg/Gfe48jFl8MFmob0Bz6jNX/DlLe/1SCmrnfYJDY5uCNzpRYYvdXhT2lLsCPdRTO8PvPNCeWHvUiCnAYeV1bwDCOO8vVffJJoqElG+MX+IB/QlfXuvWAy6NEvc2AS8+2YgmpnQMQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4EEA3497; Wed, 7 Jan 2026 07:53:16 -0800 (PST) Received: from e133380.arm.com (e133380.arm.com [10.1.197.67]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 079953F6A8; Wed, 7 Jan 2026 07:53:20 -0800 (PST) Date: Wed, 7 Jan 2026 15:53:15 +0000 From: Dave Martin To: "Chen, Yu C" Cc: Reinette Chatre , Babu Moger , Fenghua Yu , Tony Luck , James Morse , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Jonathan Corbet , x86@kernel.org, linux-kernel@vger.kernel.org, fustini@kernel.org Subject: Re: [RFC] fs/resctrl: Generic schema description Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Hi, On Fri, Dec 26, 2025 at 06:38:52PM +0800, Chen, Yu C wrote: > Hi Reinette and all, > > On 12/17/2025 6:26 AM, Reinette Chatre wrote: > > Hi Babu and Fenghua, > > > > Could you please consider how the new AMD and MPAM features [2] may benefit > > from the new interfaces proposed here? More below ... > > > > On 10/24/25 4:12 AM, Dave Martin wrote: > > [snip] > > > > > One thing I was pondering is that resctrl currently uses L3 interchangeably > > as a scope and a resource but if instead that is separated then it should be > > easier to support interactions with resource at a different scope. > > > > I am concerned that, for example, support for Global Memory Bandwidth Allocation > > (GMBA) is planned to be done with a new resource. resctrl already has a > > "memory bandwidth allocation" resource and introducing a new resource to essentially > > manage the same resource, but at a different scope, sounds like a risk of fragmentation > > and duplication to me. > > > > What if the "resource control" instead gains a new property, for example, "scope" that > > essentially communicates to user space what a domain ID in the schemata file means. > > > > It is not clear to me what a "domain ID" of GMBA means so I will use the MPAM CPU-less > > MBM as example that I expect will build on SMBA that supports CXL.mem. Consider, an interface > > like below: > > > > info > > └── SMBA > > └── resource_schemata > > ├── SMBA > > │   ├── max > > │   ├── min > > │   ├── resolution > > │   ├── scale > > │   ├── scope <== contains "L3" I guess we already have this confusion about domain IDs with monitoring domains not necessarily being the same as control domains. (The generic schema description does not try to address monitoring domains, but the concept is still valid...) "scope" seems a resaonable name. What values would be expected here for the pre-existing schemata? I'm thinking "L2" for L2_foo schemata "L3" for L3_foo "L3" for MB (at least for the old MB schema) Is it worth splitting out the level as a separate value? e.g., scope = "cache" level = 3 Not all scopes will need a "level" parameter. (This may not be sufficient for the region-aware case that Chenyu outlines below.) > > │   ├── tolerance > > │   ├── type > > │   └── unit > > └── SMBA_NODE > > ├── max > > ├── min > > ├── resolution > > ├── scale > > ├── scope <== contains "NODE" > > Would it be more user-friendly to explicitly show "node0, node1, ..." > rather than "NODE"? After all, we can already infer the "NODE" type from > the schemata name "SMBA_NODE". I think that having an explicit declaration of the scope is probably useful even for things that are included in the schema name. Part of the reason for describing the schema explicitly is because inferring everything from the name does not feel scalable as we add more different schemata and resource types. Having said that, the schema names should still provide a good clue as to what the schema represents. I'm not sure that we should simply list possible domain IDs here: For MPAM, the domain IDs can be huge, random-looking numbers that do not necessarily start from 0 (as currently implemented in the MPAM driver). In any case, we need not just names for the individual domain IDs, but an idea of what they represent. Maybe we could stick with opaque "scope" names as in Reinette's proposal, and solve the problem of enumating the domain IDs separately. For the commonly-used scopes, we probably don't need to bother, since the enumeration is available elsewhere: * for NUMA nodes and cache IDs, /sys/devices/system/node/node* (or /sys/devices/system/node/possible) ? * for cache IDs at level , the set of values present in all the files /sys/devices/system/cpu/cpu*/cache/index/id ? > > ├── tolerance > > ├── type > > └── unit > > > > With an interface like above there is a single resource and allocating it at a different > > scope is just another control. This correlates to how other parts of resctrl is managed. > > For example, it can become explicit that the monitor groups' mon_data directory contains > > sub-directories organized by scope. For example: > > > > mon_data > > ├── mon_L3_00 <== monitoring data at scope L3 > > │   ├── llc_occupancy > > │   ├── mbm_local_bytes > > │   └── mbm_total_bytes > > ├── mon_L3_01 <== monitoring data at scope L3 > > │ ├── llc_occupancy > > │ ├── mbm_local_bytes > > │ └── mbm_total_bytes > > ├── mon_NODE_00 <== monitoring data at scope NODE > > Does this mean the domain ID is "0", which corresponds to node0? > This seems to align with the presentation Fenghua's presentation at LPC, > where he mentioned that for CPU-less resctrl, the domain ID changes > from an L3 ID to a node ID. In an ideal world, we would have a generic description for the monitors. Coming up with a "scope" concept that works for monitoring domains feels like something we should aim for, even if we don't yet describe this explicitly for monitors. Then, we could say that mon_L3_00 has scope = "cache" level = 3 domain = 0 (assuming that the monitoring domain really does align with the cache control domain). > > > │ └── mbm_total_bytes > > └── mon_NODE_01 <== monitoring data at scope NODE > > └── mbm_total_bytes > > > > Please let me take this chance to elaborate on region-aware RDT > in more detail. I am wondering if the interface could be further > extended to support this feature. > > A "region" can be defined as a set of physical addresses that > belong to the same memory tier. The region ID is per socket > (i.e., unique within a single socket). Suppose we have a 2-socket > platform as follows: > > > S0: 1LM Direct DDR ==> NUMA node 0 > CXL HDM (Tier2) ==> NUMA node 2 > S1: 1LM Direct DDR ==> NUMA node 1 > CXL HDM (Tier2) ==> NUMA node 3 > > region0 on S0 is node0, region1 on S0 is node2, > region0 on S1 is node1, region1 on S1 is node3. > > Let us assume that each socket has 2 LLC domains. > For example, S0 has LLC domain0 and LLC domain1, > S1 has LLC domain2 and LLC domain3. > > We propose the following schemata: > __ > for example, > MB_REGION1_OPT:0=511;1=510;2=509;3=508 > it means, for LLC domain0 on S0, the throttle > level for node2(because region1 on S0 is node2) > is 511. For LLC domain2 on S1, the throttle > level for node3(because region1 is node2 on > S1 is node3) is 509. > > Users could query the exact definition of REGION1 > by checking the info directory. > > info > └── MB > └── resource_schemata > ├── MB_REGION1_OPT > │ ├── max > │ ├── min > │ ├── resolution > │ ├── scale > │ ├── scope <== "0=node2;1=node3" (node2 on S0, node3 on S1) > │ ├── tolerance > │ ├── type > │ └── unit > > > thanks, > Chenyu Hmmm, that's interesting. If there is a grouping on NUMA nodes, is that advertised anywhere in sysfs already? Ideally, there would already be a definition of what "region 0" is in terms of the NUMA topology, and we could just refer to it. Cheers ---Dave