From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DE970363C60 for ; Fri, 5 Jun 2026 16:37:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780677465; cv=none; b=U2ROaDuCJj1II7SAXs+Si0YXL9TazRTuHgyP5IQ19Ce44XVR5fuBhu+ORlFeK5kQaVFMwzVrnIkST65zDpyUylGkjWMiRvnk2omALJVBzpgcOPyVXd9JlIdVThNcSVRkL9htE2YBBqUq9jagHphZpukBokmX+tAXEbj9FOwdxtw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780677465; c=relaxed/simple; bh=P+QXfB5RTp0zx5edx0enhFcFe1vsiZjNYGi7Pgb5KJw=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=I88xM1mG8zf0jExoovQs5Zoxc5/5ja00LZVDyxSMeBVMSiBNMzsvgFMuH0s7pXfS2q6mYtj/NJ2QOHYGX+UcvSs4lPBtaDdQudydUj3D9VbsYkb5k3yuvo9gFseXHeqjlUyQK31u8RnoMeCDO2oGg1OVHSjCFGN0mu3NzbE4awU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=jfr6Aezw; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="jfr6Aezw" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 43A981CDD; Fri, 5 Jun 2026 09:37:34 -0700 (PDT) Received: from [10.1.196.46] (e134344.arm.com [10.1.196.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F02833F7D8; Fri, 5 Jun 2026 09:37:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1780677459; bh=P+QXfB5RTp0zx5edx0enhFcFe1vsiZjNYGi7Pgb5KJw=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=jfr6Aezw0GI1TlsAosjVRH9tIhiXJjLOmDID5copf0rKhzBwV+8qzczsjR8IzTJM6 L/Lc73l3AshRBBIKPpKMIKlCzGIAHfnL7gD/2buqxiqcQRC4M4+cUOG4u5Vqc6kfW2 s+lpAUc608WbP2718qrvdKGCRuNgs/MFoukI5xVI= Message-ID: <3ef279c9-c4b3-48cb-8235-da8a871b54f9@arm.com> Date: Fri, 5 Jun 2026 17:37:33 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Thunderbird Daily Subject: Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept To: Reinette Chatre , Tony Luck , James Morse , Dave Martin , Babu Moger , Drew Fustini , Fenghua Yu , Chen Yu Cc: Borislav Petkov , Thomas Gleixner , Dave Hansen , Peter Newman , "x86@kernel.org" , "linux-kernel@vger.kernel.org" References: <29c95b69-e1a4-46b1-ab8b-45c09308b924@arm.com> <7f2d0cc3-266c-4fa3-a9ba-7352325c76d1@intel.com> Content-Language: en-US From: Ben Horgan In-Reply-To: <7f2d0cc3-266c-4fa3-a9ba-7352325c76d1@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi Reinette, On 6/5/26 16:39, Reinette Chatre wrote: > Hi Ben, > > On 6/5/26 7:53 AM, Ben Horgan wrote: >> On 6/4/26 18:43, Reinette Chatre wrote: >>> On 6/3/26 8:15 AM, Ben Horgan wrote: >>>> On 5/29/26 19:06, Reinette Chatre wrote: > > ... > >>> >>>> I plumbed in support for the MB_MIN resource schema which also works under light >>>> testing. The only fs resctrl code change I needed was: >>>> >>>> --- a/include/linux/resctrl.h >>>> +++ b/include/linux/resctrl.h >>>> @@ -483,6 +483,9 @@ static inline u32 resctrl_get_default_ctrlval(struct >>>> resctrl_ctrl *ctrl) >>>> case RESCTRL_CTRL_BITMAP: >>>> return BIT_MASK(ctrl->cache.cbm_len) - 1; >>>> case RESCTRL_CTRL_SCALAR: >>>> + if (ctrl->name == RESCTRL_CTRL_NAME_MIN) >>>> + return ctrl->membw.min_bw; >>>> + >>>> return ctrl->membw.max_bw; >>>> } >>>> >>>> >>>> At least on MPAM systems, we use a default of 0 for minimum bandwidth controls >>>> as the maximum bandwidth controls only take effect if their value is higher than >>>> the minimum bandwidth value. I have specialised this on the ctrl->name which >>>> breaks your ctrl->type based classification but that's fixable by just adding a >>>> default field to membw. >>> >>> This I am not sure about. In my understanding a typical "default" value means >>> "no throttling" and, at least on Intel, this default hardware state has been >>> summarized as "min" == "max" == "optimal". >> >> Ok, this sounds odd to me but that is probably because I don't know what Intel >> systems do. On MPAM systems a MIN control is a boost rather than a throttling >> control. Although, you can always think of that as throttling the traffic with >> the other PARTIDs. >> >>> >>> Are you saying that on MPAM systems if "min" == "max" then max bandwidth controls >>> do not take effect? Could you please elaborate what happens if "min" == "max"? >> >> Table 5-4 from section 5.2.8 of the IHI0099B.b shows the interaction between the >> min and maximum controls. >> >> If used bandwidth is The preference is Description >> Below the minimum High Only high requests compete with this >> request. >> Above the minimum: >> Below the maximum Medium High requests are serviced first then >> >> this request competes with other >> medium requests. >> >> Above the maximum, Low Requests are not serviced if any high >> when HARDLIM is 0 or medium requests are available. >> >> Above the maximum, None Requests are not serviced >> when HARDLIM is 1 >> >> So if we keep the minimum and the maximum controls values always the same then >> all traffic will be given "high" preference until the target bandwidth is >> reached. For some MPAM systems it is recommended to set the minimum value as 5% >> less than the maximum value to get a reliable target bandwidth. As 5% seems >> implementation specific and some systems don't have min controls it seemed >> better to just match the MB control with a maximum bandwidth control and let the >> user have freedom to choose the minimum bandwidth control when MB_MIN support is >> added. >> >> If a default for the minimum of the maximum possible bandwidth is used (100%) >> then any change of the maximum won't have any effect as it's always less than >> minimum (if that's unchanged) and so all traffic is high preference. I now see >> from your reply below that you are planning on not allowing this kind of >> configuration. >> >> If the minimum always tracks the maximum then we lose the distinction between >> medium and high preference traffic and so to reserve some high preference >> bandwidth for one control group we'd have to change the configuration in the >> other controls groups so that they're bandwidth preference is medium (minimum >> value at 0). > > I do not think we are talking about the same thing here. I am *not* saying > that minimum and maximum controls should always be the same. > > The discussion is about a proposed change to resctrl_get_default_ctrlval(). resctrl > uses this function in two places: > - When creating a new resource group: > The intention here is that when user space creates a new resource group it should > be created with maximum allocations possible. For MBA this means "unthrottled". I would contend that for minimum controls that a policy of 'maximum allocation possible' isn't a useful default. I try and explain a bit more below. > After creating the resource group user space can adjust allocations to match > workload requirements. > - When unmounting the resctrl fs. > The intention here is that all controls are set to unthrottled to stop any possible > impact to system when user space stops using resctrl. > > resctrl_get_default_ctrlval() is thus intended to support an unthrottled baseline from > where user space can make configuration changes as supported by hardware and required > by workloads. The baseline that I see makes most sense for a minimum control is to have the default as 0. This just means that there is no "guaranteed"/high preference bandwidth reserved for the control group. I would say this still unthrottled but just not giving a boost. With this default the user can use MB (backed by max bandwidth) without having to know about MB_MIN (keeping it constant). If the default is 100% for min bandwidth then the user needs to know to set MB_MIN to be able to use MB. Having a default of 100% for max bandwidth, correspondingly means a user can change MB_MIN and see guaranteed bandwidth effects without having to know about MB/MB_MAX. Does this make sense? > > I see that the MPAM driver internally uses resctrl_get_default_ctrlval() in a couple > of places and I am not considering this usage here. If internally MPAM has other > usages for this function where it does not mean "unthrottled" then perhaps > it would be better to create a new function that matches the usage? I don't think the internal usage makes a difference here. One process thing I was wondering about so that I know how to structure my patches. In the series you have a few patches which touch all architectures; these have the prefix mpam,x86,fs/resctrl. Is this how you would like cross architectures patches to look like or is it just for convenience in the rfc and a patch per-architecture is preferable? Thanks, Ben