From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6BA3C3BC668 for ; Tue, 4 Aug 2026 09:11:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785834687; cv=none; b=IE1evOEj7ejjvdj8KDppvuDAkTTONZWiQZLYNoEO6i7g1lvCNEtrOKcGzRasTK0EzfP3GKBLWWlP621A9fFxnbhkqOK5MXYVplEYXili+nitTTDhhRkaYdFxrLPzLJdrIQu1tQbi6M/0YjXmBTgOXCRtnwLt7t5WTnd3upbdGEc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785834687; c=relaxed/simple; bh=PcLlt2znbAymIvRO6SAF2fGZhTwLIFqQoaJVTa+t1P8=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=CS+UajoecV/LxXylU5OODpuhKcPXEdyKZoHS0H5ZfdfyuI5QAkUh52P3bkmoyioOactDFDfG2Os9OfO+iR5mcsPRJU6+r0vwqSF75V6EiBm83SFG5H6/FMyAStNB5xuBrW2Nd33PD5Qpv1ZwDpKnn0Xdc/I31zDmkS3xAJSAYsQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=sKX918HC; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="sKX918HC" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 754F61476; Tue, 4 Aug 2026 02:11:20 -0700 (PDT) Received: from [10.2.212.8] (e134344.arm.com [10.2.212.8]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9C38D3F66F; Tue, 4 Aug 2026 02:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1785834684; bh=PcLlt2znbAymIvRO6SAF2fGZhTwLIFqQoaJVTa+t1P8=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=sKX918HC+lM2w8qQt2cZ/qUqd3wWDQNYNoAH94XWYe5rDR65NwhC+ll8lAtrpabBA S/5d3PPj/zxBciuOSpMir5LbNbFBiyMQ3yjz947QkLKy+KQQJISDQRqII997APZrw2 /DfIjm+N+mMszjh/sDv43QP4JAQ0u2G9BerPK7uI= Message-ID: <618ba724-d734-43d3-b90e-11ba7f31c2b6@arm.com> Date: Tue, 4 Aug 2026 10:11:21 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Thunderbird Daily Subject: Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept To: Babu Moger , Reinette Chatre , Fenghua Yu , Tony Luck , James Morse , Dave Martin , Drew Fustini , Chen Yu Cc: Borislav Petkov , Thomas Gleixner , Dave Hansen , Peter Newman , "x86@kernel.org" , "linux-kernel@vger.kernel.org" References: <62701203-c4a3-4ec2-a9af-602e1fc15863@nvidia.com> <8f9f78dd-e3f5-4b35-bc72-0eb5dafdcedf@nvidia.com> <36163a81-9737-49e3-93ef-6c392f7272f0@intel.com> <0fc6df54-26c7-43fa-948a-528cd94937f1@arm.com> <9049378c-699a-4155-b1e4-737a1d7265d5@intel.com> <57740b97-80ee-4632-bca3-dc43cd7776c2@arm.com> <44f26cd4-be79-476e-b002-7ccfb7705179@intel.com> <749bd904-523d-4e9d-8493-0e8cfd79949e@arm.com> <9db33feb-cf04-420c-a99a-e31e4b8e4954@arm.com> <8fd6caed-820f-457a-a1ef-a0a006fa52aa@intel.com> <4ef15dde-2fbb-4763-93b6-4333b02d6859@arm.com> <7b751c28-2f04-42b7-b957-af6447e7f824@intel.com> <34b95afb-8b60-4680-9ad1-90c5b24e8fb7@arm.com> <08f016bc-2ba6-439e-bb3e-20061166402c@intel.com> <21e614b8-50fa-49e4-87c3-e6bdb4e83ab1@amd.com> <653c9a0c-c665-4016-90b2-3f55e06050b3@amd.com> Content-Language: en-US From: Ben Horgan In-Reply-To: <653c9a0c-c665-4016-90b2-3f55e06050b3@amd.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi Babu, On 7/22/26 18:02, Babu Moger wrote: > Hi Ben, > > On 7/22/26 05:47, Ben Horgan wrote: >> Hi Babu, >> >> On 7/21/26 21:02, Babu Moger wrote: >>> Hi Ben/Reinette, >>> >>> On 7/21/26 12:30, Reinette Chatre wrote: >>>> Hi Ben, >>>> >>>> On 7/21/26 6:23 AM, Ben Horgan wrote: >>>>> On 7/20/26 23:54, Reinette Chatre wrote: >>>>>> On 7/20/26 6:30 AM, Ben Horgan wrote: >>>> ...> >>>>> The former, info/ contains a directory for each allocation scope of each resource. > >>>>>> >>>>>> The other x86 feature to consider is AMD's upcoming "Global" MBA/SMBA that exposes memory >>>>>> bandwidth allocation >>>>>> in "groups of L3" that I understand could usually be mapped to NODE scope (but it remains >>>>>> controlled at L3 scope), >>>>>> except for one configuration where it is "SYSTEM"(?) scope. >>>>>> Ref.: https://lore.kernel.org/ >>>>>> lkml/8f77f498b1c77fa8fd8f5d5687f03ae598068544.1776980182.git.babu.moger@amd.com/ >>>>> >>>>> Hmmm, I'm not sure that the scope can be considered to be NODE scope for GMBA. To me it seems >>>>> to be >>>>> accidental that it maps to the NUMA node but really the scope is just a grouping of L3 instances. >>>>> For a control to NUMA scope I would expect the resctrl domains to go offline and online in sync >>>>> with >>>>> the NUMA nodes. For GMBA it looks like it would just going offline/online based on whether any of >>>>> the CPUs and so L3 instances in the group are online. Am I correct here? >>>>> >>>>> Assuming the domains are on L3 groups rather than NUMA also changes which end of the link the >>>>> traffic is regulated and so how cross-NUMA traffic behaves differently. If the domain is an L3 >>>>> group >>>>> then a task running on a CPU affine to that L3 group won't be throttled unless that particular >>>>> domain is throttled but with NUMA node domains it may be throttled if it has traffic going to that >>>>> domain. >>>> >>>> I'll defer to Babu for accurate answers about this hardware capability. >>>> >>> >>> To me, Global MBA should be considered a NODE-scoped resource. In some configurations it may appear >>> as SYSTEM-scoped, but that is effectively equivalent to a single-node encompassing the entire >>> system. In such cases, there is only one schemata entry controlling the whole system. >>> >>> Yes, multiple L3 instances are grouped together to form a NODE. Internally, programming is still >>> performed at the L3 level, but that implementation detail can be hidden from users and does not need >>> to be exposed through the interface. >> >> We seem to have two things that can both, somewhat reasonably, be called NODE scope in the resctrl >> user interface but the behaviour required for an MPAM system and an AMD system appears different >> from the point of view of lifecycle of the resctrl domain. >> >> For MPAM NUMA scope the MSC instance (MPAM hardware interface) is at the memory controller and so >> goes on and offline based on whether the NUMA node is offline or online. For AMD NUMA scope it looks >> to me that the lifecycle of the resctrl domains would be tied to the CPUs associated with the NUMA >> node. To me it does seem odd that a control with a domain associated with an offline NUMA node can >> continue to throttle (cross-NUMA) traffic. >> >> Is there any GLBE Control Domain ID or similar that is exposed to the user, e.g. is sysfs, or is >> this just implicitly the NUMA id? >> > Yes, the GLBE Control Domain ID is exposed to the user. It is essentially equivalent to the NUMA ID. What's the on/off lifecycle of these nodes? Does it follow the NUMA lifecycle as managed by the NUMA node notifiers documented in Documentation/core-api/memory-hotplug.rst or is it just linked to the cpu hotplug as is done currently for the resctrl cache based domains. Thanks, Ben > > Thanks > Babu