From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D80B2701CB for ; Tue, 14 Jul 2026 22:06:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=192.198.163.16 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784066770; cv=fail; b=eaBw2bfgLJmEM/zGtqs+sbuRy72POGhOdIguKfCnHOGPit6iqCH+C3jqAhOSN29SL1XVPeduGk45ifTCZQB7rvXawawFZpiVi+jN62Pli/gXq7v/GVn2tM18SJFTSxQ6DUIVigEqGJAFWJfbtiO0Q6IggHrdwyfnDcx3x9uozuo= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784066770; c=relaxed/simple; bh=VCBF+D+hK0lzPhGIRj28iSz9d0QliKnQfUxe0E3MEa4=; h=Message-ID:Date:Subject:To:CC:References:From:In-Reply-To: Content-Type:MIME-Version; b=UE7+XnSdw+QdKR3HMr0morzrOBFmD8rFzWW5fhEOHAS0w1/zNBl5OBRHVGklHZ20BG07v0X3LHnZKfqOzITZsDV001RxANGJyj/a4toHIG7TGggnFjh+mPtdb9+uhhbZaZe8tfaEiphL2Qdf368wXRl0iyFyIum9soaXhoGpAvI= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Yv+WZ7FT; arc=fail smtp.client-ip=192.198.163.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Yv+WZ7FT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1784066768; x=1815602768; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=VCBF+D+hK0lzPhGIRj28iSz9d0QliKnQfUxe0E3MEa4=; b=Yv+WZ7FT+bgpVIpA+TPtSGWCGxYRVRCZ8e4a9bHmCtdv7ncIqrNVehcl CJc+9Am6KBY1LILEJ+IpwbHNYsaBA6qG0oV9F+qUH5KxqC9TIFlR8A2oy ymCfN44hZJnq5u6nR82o229iUWrxCQvvBGoF4WAbXBxKLhqOiU3qni9SS 71iSM8Rfjl5AkEIuUS+b2wczIoxq2ivhajc7nHV/d77maxx/Ptel31T9v 2hkJ6wjxPSkwLV6vq5BiYdljjyp8YVf/ETuJ/Jj5GratjV7DF0CCI1cxU lteTIC6vhM4IzF51PQbJUKIfAM6jO6cvv+m5p6X56P0D04wllZEnkn58j Q==; X-CSE-ConnectionGUID: Snxaw+59S7epagvf/1PJog== X-CSE-MsgGUID: yg17MhwjT/WeyxV05hVEzQ== X-IronPort-AV: E=McAfee;i="6800,10657,11847"; a="72224679" X-IronPort-AV: E=Sophos;i="6.25,164,1779174000"; d="scan'208";a="72224679" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2026 15:06:07 -0700 X-CSE-ConnectionGUID: TdZTPEgiTYG6XKpxOPKZ4A== X-CSE-MsgGUID: FrsCrsCEQV6Hdfs93sA9Tg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,164,1779174000"; d="scan'208";a="251562906" Received: from orsmsx901.amr.corp.intel.com ([10.22.229.23]) by fmviesa006.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2026 15:06:07 -0700 Received: from ORSMSX902.amr.corp.intel.com (10.22.229.24) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.43; Tue, 14 Jul 2026 15:06:06 -0700 Received: from ORSEDG903.ED.cps.intel.com (10.7.248.13) by ORSMSX902.amr.corp.intel.com (10.22.229.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.43 via Frontend Transport; Tue, 14 Jul 2026 15:06:06 -0700 Received: from BN1PR04CU002.outbound.protection.outlook.com (52.101.56.1) by edgegateway.intel.com (134.134.137.113) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.43; Tue, 14 Jul 2026 15:06:06 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nD3H7UOY2da+teLxSFWJfAgynKoqvmVsfIHTOWI+5zIfmx2vQ/kUxmgjCl1FWO29xfuaVbino8/Qr6QihfiYFqV+0Q1n8u3FarGSbsvn4YviFFUhfTvf1NXCpVsv1QAnjLzU7oWRiaNgZcQWt03VQ/tjJKTT/2rqBAo2m4j6344qOxLoiEP55iVOvoBQycBqSusHtgJJtnHPe2b5zy0Ys7qfpWNRMeouOXqvCpQan26iBEJQLfLQXv7d1ANkZ+2NIQAC5jTuTrdV/si43N7J+Tx9O+QG8mx7RUzY+ymfbED3LzwLOT1fUU2lktWgZjTxIpXXJphQm7CifxcS3TJ8kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f3+kSZHCwSrzj0mgT362qJDBB2SW2ox+N1sMrhFervA=; b=FOW3aBE7H8XUgVa1pyiI5AjpxqNhCYXy0IkWldbVpqcSYSgqL9My5WoGa5GNumy0zK4YRWAxe55v4YnoWOU5RQo5P5e4rhQ8FUFM414D/ElUR5qW/5BU8RsaK1xOKeHpRChbyIi1z5fxwGtJYik9x8SyU2E7yDhuc+F0+3rO3lD1EotvAM+F/y7pJzpMhULD9MdjiLVhRI32HIgxMsvtCS93UiQ/uyV+SJmy2wbxsOndxRtxgz2FOpyaTaGzVzphlfmhF1jRUl070TQ2EqqsRjPceue8ldAdt2w1lmsv1PCoZTziZjANlvCbpV5W9LfEuijGaw+AU9RY+ZU7x3VLDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from SJ2PR11MB8370.namprd11.prod.outlook.com (2603:10b6:a03:540::20) by MW4PR11MB6810.namprd11.prod.outlook.com (2603:10b6:303:207::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.202.19; Tue, 14 Jul 2026 22:06:03 +0000 Received: from SJ2PR11MB8370.namprd11.prod.outlook.com ([fe80::b6cf:ce77:3cdf:7cc]) by SJ2PR11MB8370.namprd11.prod.outlook.com ([fe80::b6cf:ce77:3cdf:7cc%5]) with mapi id 15.21.0202.014; Tue, 14 Jul 2026 22:06:03 +0000 Message-ID: <36163a81-9737-49e3-93ef-6c392f7272f0@intel.com> Date: Tue, 14 Jul 2026 15:06:01 -0700 User-Agent: Mozilla Thunderbird Subject: Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept To: Fenghua Yu , Tony Luck , "Ben Horgan" , James Morse , Dave Martin , Babu Moger , Drew Fustini , Chen Yu CC: Borislav Petkov , Thomas Gleixner , "Dave Hansen" , Peter Newman , "x86@kernel.org" , "linux-kernel@vger.kernel.org" References: <5ee87762-1898-4b62-94da-85b3e9917ecc@intel.com> <62701203-c4a3-4ec2-a9af-602e1fc15863@nvidia.com> <8f9f78dd-e3f5-4b35-bc72-0eb5dafdcedf@nvidia.com> From: Reinette Chatre Content-Language: en-US In-Reply-To: <8f9f78dd-e3f5-4b35-bc72-0eb5dafdcedf@nvidia.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-ClientProxiedBy: MW4PR04CA0389.namprd04.prod.outlook.com (2603:10b6:303:81::34) To SJ2PR11MB8370.namprd11.prod.outlook.com (2603:10b6:a03:540::20) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ2PR11MB8370:EE_|MW4PR11MB6810:EE_ X-MS-Office365-Filtering-Correlation-Id: f5593f41-9751-4781-7d72-08dee1f418a0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|366016|1800799024|376014|7416014|3023799007|6133799003|56012099006|5023799004|11063799006|4143699003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: BLjJhXwulhhOFSFUzlqFG3UAUtOW+3Uc23W5Pr08PJn52NLT9CfjwNDRf/ahUMbQsQ85Mgor7Da+CT3YK07wLSNBYEuRSmJiRxlKBehhYaP5myQmXDZayGutVsg0pfPdM0siBqwGLkmnQ1lmtg+rRawKuh13rEJCHh1ejH34IQm7V+oL4MK37CVIPTt0YhzaRZpvuWt9/2gXfswMv9Rz1TTZUyuvW7mEEhsR6LKB1GJPq5HTdHI0A4+XO1rbcMVj0od78dlzLYzRVvKq8/DBYg2Bx4lJXHdTDGo8/Z1HcycnUJaWj8lW5gnt51Bktsd4dgd5ienJeTpMlUzV+O+72QN/vpfb/fscp/EikDE6ow+PagFyB7WggwQdle/UZDV6fdbpi1950O7sQZ2m455f2y/qXq9MLofZkaLalQXigHw99SlFSyT/peM7lHpdGtSLWvUlXGyw802Ovr78pIu1qU8gFbLWt39rhuUCfrquHlalU0gmL62yPR68gkiGjNkEVjjWwYfPI1BaGv8jk3EY9WuSCsA/tZU/eTdl4lHNlNVm11GYMlfewLZ/ko3zuXlud8QpDB9d17m+lOxNQgsUuKCqvtfgB6kkhjvpfN18oBQN8tmKmNwoaSS57blSsyGo X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ2PR11MB8370.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(366016)(1800799024)(376014)(7416014)(3023799007)(6133799003)(56012099006)(5023799004)(11063799006)(4143699003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OE8vd3I4MGpBeWRKRDNsMHQrV3ZRbEd6TFBtbWJFMDBuWDFmTjFELzlvWm91?= =?utf-8?B?bjFUc3hyYkwzRFo4ZzdHZEUzc1ZtM1p3bXhSUU5GVjdHUWxTakMwRDk4em9P?= =?utf-8?B?cVRkUlBHSy9iTlNQUjc4aXJuaEdwbjRDYjFBL3RLTHVZYjdHYzhXcDkyc09o?= =?utf-8?B?Smg2VWFibU55dm5hV3R0OTkycENRTVZrNDZBcmhTcnBtYXFtaVFJdGYxc2M4?= =?utf-8?B?d1VLU2NLVXJyWWNlTFFCWEZhYk9NckVFakFQZTJCaXZKSmZYbksvY2NkRmgv?= =?utf-8?B?VVd5Z0s1OEdyWWFNR0EwN2NGL1EzT3RjdmhjYi9FVXZFdCtIdzVlRFQ3bGF2?= =?utf-8?B?V3VjQ09yT2NRZGlSR00vNUhBNUFxOEYrRjI2MHdnMU8rdG1sb05oL2svOVBs?= =?utf-8?B?QjFDQTl2Q21KVnNOSU4raTBYbVhDUTV3b0U4eS9uK20vT0hITGdEUno0bnh1?= =?utf-8?B?YndIOHZENkl1ODBYZk5ZTHVIMjQyanNLVW1TK2UxbmlJbkVyeHRIRHFKUDh4?= =?utf-8?B?ZklRZHFKOU5NakVVY1J3ZEt0TythVnR4Ny9XMVZCN05nMkU5Mnh0bnp4UkVM?= =?utf-8?B?MG9GRytldGErdUpqU3B6alB5NXg1WjNNTERSeHdoYzlqU3NmVHkxdENkSDZs?= =?utf-8?B?TDlDN21XanJVaXJlRFJxS0VJUks0WDhEYmN6TmZWcUNzN3RFV0lNSG1wWXR6?= =?utf-8?B?S28wZkI5blhRNFcvZ0xUbjVhNUY5dWg2ZWsxZE5OU1dxNmpRVTdBMVJSa29u?= =?utf-8?B?eTlNL2hCVlFHOTBIRUp5V3BHbzFCN1d5SndIeUZnMXdIUnZSOERQQkkrdzZr?= =?utf-8?B?b05sQTNVamdIMVJVM0lxdk83RTFFZ0VkMXlTdU1TdS9LVzdwN21SSklUWHgy?= =?utf-8?B?aFF2T0I3Z1RTcllaakhQQldyTGFDY1ZPRTF5NTlmTVhJRU81bUIrNlRHT0Nw?= =?utf-8?B?bE9HVTBHd2YybXdycm85SktDSkR6WG1nNUNNTHc2VVpGNUltRW9iNFdzTFFD?= =?utf-8?B?R0pTZTcvRUdIYkxjeS9HdjNHWVpnMnRncTJ1eFlkOGlwZmxFaTN5a3A5Y0NC?= =?utf-8?B?ZEV0SHl2Rk0yWVNkZG5aRHVOanRoNVcrZmdmKy95Q0puNlFRZDJocWtNTjZp?= =?utf-8?B?Y3llTVNsTW9oRktHb0FVTGJsL0lsbkFiUnprVUVEamNMSFUvaVdZS3dWZ1N4?= =?utf-8?B?QWhDbDIyc0VPcDU2WU92NUs4WUpIcWNYWUlXWC8vV2ZPK2xxT1FNT3JYbnpU?= =?utf-8?B?RmI2QnhPUG1oRnBQcVJ1SDlhdldjam5RTFN6NjQxL2xjb2dWYkh4VHJyL2da?= =?utf-8?B?VFJEUU1zZkE1eGZkYVpwek9GSHZacTBqb3VCM3RvenpZb2F5RFBiRWh2azJY?= =?utf-8?B?YjNDWGQzRDg2YXNmbHM3UmJ4WjhFMHYzV0FlSVZxZGFUWFdaMSs0WHQzdVFN?= =?utf-8?B?MXRibCt1YUNqTlJUaEl1OXo2aSt1ellMMUlpelFWRHJSM3k2b2l0YXlBRjJr?= =?utf-8?B?VXFQMzYzb0JCbHpoampmVXhmTHdYSHFrVkFHczNGMm45RWp3YklPM05wS250?= =?utf-8?B?UnhjdzVVMEE5M0cvUEhwVSszajk0bm1iYVlWSmZua0pqTzNrZnhpWWpYcUF3?= =?utf-8?B?eGk0M0x6Mldza0dPcmUyMXAyNTcwaW14UGYvbVdvdm1OWGtsUElhNjN5d3Rh?= =?utf-8?B?RlJNVGZmb3dSQnQ2MnBSa0ZDNVhVZHV3Vm5pdlB1b0JtZ3JwaE9yMFlSR1Ur?= =?utf-8?B?eVVKS2Ywa2xIanZKYnorb1lJZk9teXJmV1N3VWZJSVQ4Zm4zMkNpNndzQUZI?= =?utf-8?B?V2YyN2VYcTEyaDJUYlU0V0V0UFVneW5sVVhvdzV1M3dwd2U4NGhXaXhFOFFG?= =?utf-8?B?ek92OHV5dXdObjRGcFYzcjZMcVp1d090UU5DTnh5UGljVVFIeEZ6YUtnelh4?= =?utf-8?B?cXN2czBVYUNJNzhvRnZReUZnR21vYWpmNGo4aHVyUXdaTzNGK255bkJPM005?= =?utf-8?B?dkpydVBQZmNaRStVSUJ6ZjJTWjJlOUFiME12OGxUV0c3Q0t6aFRGQVhmTnRq?= =?utf-8?B?OFd0Z0VRelJCcjFhUXluOTZ5b1Jvb0FrVTlxNk9GMGs5MVMyQTczUDFDUGli?= =?utf-8?B?bXRtT1Zac0hUM2R2dUt5NDhTWXhvbktsdVMwWmphblVtSzd0TzB0N0x2Qmd6?= =?utf-8?B?RXF0aEJReUp2bS8wa2lLaDA3bUZnYmJQRDUzekJCN21DTzNEdENXNFVJTUp3?= =?utf-8?B?dkZCN1JCY1JkUlA4aXhXTGxqbEhTMHdyRnlYK1V4c0ZVdDNzVTRnaU9zUW9X?= =?utf-8?B?TDk3UnE4ZXBua0tTMERBYmRTOW10R2x1dDFEWndhK3J1QWtJSEkyR3l2SFh4?= =?utf-8?Q?q/s9d6SkSdjcF4lk=3D?= X-Exchange-RoutingPolicyChecked: NC8J/WsWMBLnKVGySsxqpcuy5/b9Wlm+IVCV1kq7zu8etx94/KJ+lY2KGqoA2yYROUs283YcT4/f0/Yw4+sAEfrQFCkrcB+zAF4iRf4dAsd56JywkLlu5P9LOLwbzLfMiTonxZVwz/gtnWUvLbcGakcOVr1llmpJ8AQSoIFtGhYYTa7N6vwsrs1FybXpazIWkWcscnne6saWOeXRFNBsVx+ujyaiCsC5G6CSc6VzzOYshfDjfxhCWxbTBIWwMsr3BaLn7iA0mKhF6M8LDCk4yWOq7yXs4eLYz5kepc4teD5Jq+3agnd/Tpx01wPVS1UFZy5SnNJ9BZpsd7wxrGMkCQ== X-MS-Exchange-CrossTenant-Network-Message-Id: f5593f41-9751-4781-7d72-08dee1f418a0 X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB8370.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2026 22:06:03.5686 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0WeWgyLnV949Xj+puew4hj8Anf5YN4Lpk+xUjGcZyX6lDOc7K24xaoZ+VUJfoIMR0nZ6T1D45gQq/JDyhKqu+5DMW9wkJJf/ncBt4pk1gWA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6810 X-OriginatorOrg: intel.com Hi Fenghua, On 7/10/26 1:59 PM, Fenghua Yu wrote: > On 6/25/26 08:43, Reinette Chatre wrote: >> On 6/24/26 6:26 PM, Fenghua Yu wrote: >>> On 6/24/26 15:22, Reinette Chatre wrote: >>>> On 6/24/26 12:08 PM, Fenghua Yu wrote: >>>>> On 5/29/26 11:06, Reinette Chatre wrote: >>>>> >>>>> As Shaopen and Ben mentioned earlier, we are working on two MPAM >>>>> features that may need to change schemata interface. The CPU-less >>>>> feature was discussed on LPC (although the interfaces will be >>>>> slightly different from the LPC). >>>> >>>> I know. Here is where I tried to engage with you on needed interfaces after LPC: >>>> https://lore.kernel.org/lkml/fb1e2686-237b-4536-acd6-15159abafcba@intel.com/ >>> >>> MPAM ACPI defines MSC (Memory System Control) is defined in one of two ways (not both) on one platform: >>> 1. L3 and memory together on each processor MSC >>> 2. L3 in processor MSC and memory control/monitoring in different memory MSCs. > > Ben said there is type 3 platform: > 3. L3 cache and memory bandwidth in processor MSCs and memory bandwidth in different memory MSCs. > >>> >>> On type 1 platform, schemata is legacy: >>> MB:1=100;2=100  <-- cache id 1 and 2 as domain id >>> >>> On type 2 platform, I will not reuse "MB:" name. Instead, define new resource name "MBN:" for numa node and schemata is: >>> MBN:0=100;1=100;2=100;10=100;18=100;26=100 <-- numa id 0, 1, 2, 10, 18, >>>                             26 as domain id >>> On type 2 platform, there won't be "MB:" line. Numa 0 and 1 >>> are for mbm allocation on socket 0 and 1. 2,10, 18 and 26 are for GPU >>> memory nodes allocation. > > On type 3 platform, there could be "MB:" line for L3 cache and "MB_NODE:" for numa node. Example schemata is: > >      MB:1=100;2=100                 <-- cache id 1 and 2 as domain id > MB_NODE:0=100;1=100;2=100;10=100;18=100;26=100 <-- numa id 0, 1, 2, 10, >                                                    18, 26 as domain id >> >> (to help make things explicit I will refer to what you call "MBN" as "MB_NODE" to make it >> explicit that it is memory bandwidth allocation at node scope) >> >> I am trying to consider how this can be accomplished while also considering all the other >> new hardware features that resctrl need to support. Consider, for example, AMD's "Global >> MBA" (https://lore.kernel.org/lkml/cover.1776980182.git.babu.moger@amd.com/) that throttles >> memory bandwidth at L3 scope but the user configures allocations at NODE scope. At this time >> the plan is to support this with a second control associated with the MB resource that can >> allocate memory bandwidth at node scope. See >> https://lore.kernel.org/lkml/430ffb48-29f4-44d9-9164-9f8b743b2739@amd.com/ >> >> If resctrl creates a new resource for node scoped memory bandwidth allocations to support these >> "type 2" systems then that will result in an inconsistent interface between architectures that >> we should avoid. >> >> Have you been listening in on the discussions surrounding emulated controls? Considering that, >> would it be possible to support the "MB" control on a type "2" system but have it be backed by >> (emulated by) the underlying "MB_NODE" control? >> >> resctrl could expose both controls on these "type 2" systems but make it clear that "MB" >> is emulated by "MB_NODE". For example: >> >> info/ >> └── MB/ >>      └── resource_schemata/ >>          └── MB/ >>              └── MB_NODE/ >> >> User will see both controls in schemata file but when changes are made to "MB" control it >> will show in the "MB_NODE" control and vice-versa. User could also disable the "MB" control >> that will establish familiarity with the interface at which point resctrl can drop the >> "MB" control from the schemata file on these "type 2" systems. >> >> Having the MB resource available with an MB control will keep resctrl backward compatible >> if there are any tools that expect that. If backward compatibility is not of concern then >> resctrl could initialize with the emulated control disabled by default. See discussion at >> https://lore.kernel.org/lkml/5e575bc2-e67f-4696-9332-33c54023c057@intel.com/ >> that describes a new resctrl capability in support of RISC-V and RDT. >> With this resctrl could initialize with: >> >> info/ >> └── MB/ >>      └── resource_schemata/ >>          ├── MB/ >>          │   ├── MB_NODE/ >>          │   │   └── status:enabled >>          │   └── status:disabled >>          └── mode:legacy [native] >> >> With above a "type 2" system will boot with its schemata file just containing the "MB_NODE" >> control while info/MB describes the memory bandwidth resource. >> > > On type 3 machine, schemata has both MB in legacy mode with cache id as domain id and MB_NODE with numa id as domain id. > > Is this directory OK? > >  info/ >  └── MB/ >       └── resource_schemata/ >           ├── MB/ >           │   ├── MB_NODE/ >           │   │   └── status:disabled >           │   └── status:enabled >           ├── MB_NODE/ >           └── mode:node > > 1. MB and MB_NODE are shown in parallel in inf/MB/resource_schemata/ I do not think there is a need to expose an emulated MB_NODE control if the actual MB_NODE hardware control exists. > 2. mode is set as "node" meaning "MB" is for L3 and "MB_NODE" is for numa node I assume you mean "scope" instead of "mode"? (more below) > 3. Emulation "MB_NODE" is disabled (or should the "MB_NODE" sub-dir be invisible?) Right, I do not think emulation is needed here. No need to make it invisible since it should not exist. >From what I understand these "type 3" machines could be simplified to: info/ └── MB/ └── resource_schemata/ ├── MB/ │   └── scope:L3 └── MB_NODE/ └── scope:NODE Beyond this I believe that MPAM currently emulates the MB control with its "MB_MAX" control and users may want to make bandwidth allocations at the fine granularity that it supports. Taking this into account the interface may end up looking like: info/ └── MB/ └── resource_schemata/ ├── MB/ │   ├── MB_MAX/ │   │   └── scope:L3 │   └── scope:L3 └── MB_NODE/ └── scope:NODE A system like above will thus have three schemata file entries: MB MB_MAX MB_NODE Three schemata file entries would be unnecessary for users familiar with the finer granularity MB_MAX control so that is where the "mode" file can be used to disable the legacy MB control to just expose MB_MAX and MB_NODE on these systems. Would that work for these systems? ... >>>>> There is another MPAM feature called MBW Max hardlimit which sets >>>>> "MB:" allocation as hardlimit (i.e. MBW throttling percentage must >>>>> be satisfied) per domain. Adding a new "MB_HLIM:" line in schemata. >>>>> It's 1:1 mapped to "MB:" to control hardlimit of MB throttling >>>>> percentage on each domain. By default hardlimit is off (0) and can >>>>> be turned on to set MBW Max hardlimit on a domain. >>>> >>>> ack. This sounds like a new control associated with the MB resource. >>>> This is a boolean control as Dave highlighted in previous discussion so >>>> resctrl would need to know its properties. >>>> See https://lore.kernel.org/lkml/aO0Oazuxt54hQFbx@e133380.arm.com/ >>>> >>> >>> Right. ("MB_HLIM" name may be adjusted accordingly when "MB_MAX" is available.) >>> >>>>> For exmple: >>>>> MB_HLIM: 0=0;1=0;2=1;10=0;18=0;26=0 >>>>> MB:0=100;1=100;2=80;10=100;18=100;26=100 >>>>> >>>>> On GPU memory numa node 2: cannot use more than 80% of total max mbw even if there is still idle mem bandwidth on this node). >>>>> >>>>> MBW allocations on all other domains are soft limited, meaning MBW can be used more than specified if mem is idle. >>>>> >>>> >>>> ack. >>>> >>>>>>            L3:0=fff;1=fff >>>>>> # echo 'MB_MIN:0=50' > schemata >>>>>> # cat schemata >>>>>>            MB_MAX:0=100;1=100 >>>>>>            MB_MIN:0=50;1=100 >>>>>>            MB:0=100;1=100 >>>>>>            L3:0=fff;1=fff >>>>>> >>>>>> Writing to the dummy control will call a dummy callback that just prints to the >>>>>> kernel log: >>>>>> "resctrl: Updata temporary MIN control on domain 0 with user value 50" >>>>>> >>>>>> >>>>>> Example output of info/MB/: >>>>>> /sys/fs/resctrl/info/MB/thread_throttle_mode:max >>>>>> /sys/fs/resctrl/info/MB/num_closids:15 >>>>>> /sys/fs/resctrl/info/MB/delay_linear:1 >>>>>> /sys/fs/resctrl/info/MB/min_bandwidth:10 >>>>> >>>>> Add two new MB info RO files: >>>>> 1. /sys/fs/resctrl/info/MB/domain_id >>>>> It shows "numa" for using numa id in "MB:" or "cache" for using legacy cache id. >>>> >>>> This proposal introduces a *global* property to the MB *resource*? It does not seem as though >>>> this takes into account *anything* about how resctrl can support new hardware that has been >>>> discussed before, during, or after LPC. You have not participated in these discussions and >>>> now make an orthogonal proposal that does not take into account *any* of the requirements >>>> that we have been struggling with for months. >>>> >>>> Why should this proposal be taken seriously? In your absence folks have been trying to >>>> accommodate how these upcoming products and be supported and the "scope" file associated with >>>> a control is intended to communicate to user space how the domain ID should be interpreted. >>>> >>>> Why are you proposing something entirely different here without even acknowledging current >>>> approach and explaining why it does not work for you? >>>> >>> >>> So can I change this part to adding the following files in info dirctory? >>> >>> 1. For numa memory bw allocation (MBN): >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/ >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/resolution:100 >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/tolerance:5 >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/type:scalar >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/min:10 >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/scale:1 >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/scope:NUMA >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/unit:all >>> /sys/fs/resctrl/info/MBN/resource_schemata/MBN/max:100 >> >> This is not just about adding files to the info directory. The files, directories, their relationships, >> and content have meaning. All I see from these proposals is an attempt to slap some new files into >> resctrl without any consideration to present consistent interface to users and without consideration of >> other architectures that need to be supported by resctrl. >> >> resctrl needs to provide a generic and consistent interface to user space irrespective of the >> underlying architecture. Architectures cannot just slap some new files for their convenience. >> >>> >>>>> 2. /sys/fs/resctrl/info/MB/max_lim >>>>> It shows number 0-3 for MPAM MBW max limit behaviors: 0 for supporting both softlimit and hardlimit, etc. >>>> >>>> Again this adds another *global* property to the MB resource but then above you >>>> describe the new "MB_HLIM" schemata file entry that implies that it is a new control >>>> for the MB resource. Having it be a new control for the MB resource matches earlier >>>> discussions. To support this I thus expect it to be exposed as a new control with >>>> potentially a new type if any of the existing planned types do not suffice. >>>> >>> >>> How about adding these MB_HLIM dir and files in info? >>> >>> /sys/fs/resctrl/info/MB_HLIM/resource_schemata/MB_HLIM/type: boolean >>> /sys/fs/resctrl/info/MB_HLIM/resource_schemata/MB_HLIM/max_lim: 0 >> >> This presents "MB_HLIM" as a *resource* to user space. It is not a resource >> but a *control* of a resource, no? I thus expect it to instead look something like >> below that makes it clear that MB_HARDMAX is a control of the MB resource. >> >> info >> └── MB >>      └── resource_schemata >>          ├── MB >>          └── MB_HARDMAX > > Yes, this makes sense. I have changed to this hierarchy. Thank you very much for considering this approach. Reinette