From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011043.outbound.protection.outlook.com [52.101.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62F6A1A3029 for ; Wed, 28 Jan 2026 04:08:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.43 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769573340; cv=fail; b=h/xgiHNq3MIo70J1BhONSo/lx13bXWmXxgbuuZx6V005xne6wa7G+F6hw/eU/KTBnIn6DjytPoTB6awhknvU37yBUxe6lfE/tsj4hjkoi+oxbQo44+K/S08KDVqHaMV4ZU9w7h/xTURxZyv4tpqUad+WmzdC5miTmEIT/uTdoUE= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769573340; c=relaxed/simple; bh=MBxxIKhq7pK4v0ITcBAsNFuVxNiPC6oUXTm+MNJq+MQ=; h=Message-ID:Date:MIME-Version:Subject:From:To:CC:References: In-Reply-To:Content-Type; b=tR4O+RPTuyVvGXedyDyhXvwW8TxrLRkSd73b+dhPbw/rTxfW/hfVbaUdv3mmy1gTV5f8mIVY1bCh5ahL+TZIRKnWd35UO5onusMXoQmVvrnAv6uW9rx1kNNWEtrY5UJkqouUlbp2pTKwvkP37YJW5PnzvYWu8KkWYXl+8ct54Mo= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Fi/sogjk; arc=fail smtp.client-ip=52.101.52.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Fi/sogjk" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sXDtK73XAKzEwIohF7rLMOMaQ60lsXnh+cbKXbCiJ0W9ZuyuiUg8YR3Tw8O22Nz0rlK/oW4jdFGMDlnLEqAC0PXoK1SRcW752L7UE8hImDUffxAMbmFZRaKUtQTP0zPB3zGmSMGbt+HKPJT+dJ/Ccrnek15b9u1KZ2g1RqXrSUpdb22JbsEF/+Yp7rmuJKleiLf4Q1XrAbfGmKyeNN3fsA9pTv3Apza9MXGRKnmHX+AS53FbQ2NxjqWJf2fNh3/6FvKWvWwW0JgK46VUjv3q+IYcwNpXUtAHe+w5Flsyxd+jfXg6l6AKAQHR352+5rgQ0cRhteLw9SFqff+DGrL7kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KiJlXFnR9cC43jlkshVagTR2rAeJLddT5MK8fh6OKpk=; b=C5g6voZ3QUKHw3mpiqe4A7l01ChlRmHCnFvJFE4GepNYMIC0HEXcy+K8szl6YzVHiYOcVw9lfe9dBdhvXH4qgjus9NZD0U9XKEzsipprjTxJOadqEsmYz5PEbk69/46ps+ridx4331E/EvPtAfHEhV60konq05FmvipZayHsA+xF4qr0t4FoboUCprbFbPr9Rmt5HIjw0t+lhWrV6T3RwvrLKfGcRbfS0wlmKcvOzL+HP2DFb2vuwOxNEzQswyZ7hPkAZqcXARXOfN5TkOwMdNDWHVgY5fY+x35Ww4J/YV3pH1uhV0VEVaJLkRCIs7HyJ+InJod7c+zxwF68bcGIGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=infradead.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KiJlXFnR9cC43jlkshVagTR2rAeJLddT5MK8fh6OKpk=; b=Fi/sogjkB6bQ2LWKze2sEAcIq4nSG8Y5FhYI7/Ba+Z2LSnwMPIIHH9+yfD83o/E4+OuqJ9XWS6++30M9MNmC/XYuzAxQH7Pl7fsghffr5FVqYs2LkWOIdlWCwuZW2iv6Ce2p2CQU5Oo9i56/UojLJE7kJesQR0nzMQP/b7dMBvE= Received: from DM6PR17CA0021.namprd17.prod.outlook.com (2603:10b6:5:1b3::34) by SA5PPF7F0CA3746.namprd12.prod.outlook.com (2603:10b6:80f:fc04::8d1) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9564.8; Wed, 28 Jan 2026 04:08:52 +0000 Received: from CY4PEPF0000EE3B.namprd03.prod.outlook.com (2603:10b6:5:1b3:cafe::5a) by DM6PR17CA0021.outlook.office365.com (2603:10b6:5:1b3::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9564.7 via Frontend Transport; Wed, 28 Jan 2026 04:08:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by CY4PEPF0000EE3B.mail.protection.outlook.com (10.167.242.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9564.3 via Frontend Transport; Wed, 28 Jan 2026 04:08:52 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.17; Tue, 27 Jan 2026 22:08:51 -0600 Received: from satlexmb07.amd.com (10.181.42.216) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 27 Jan 2026 22:08:36 -0600 Received: from [10.136.38.16] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Tue, 27 Jan 2026 20:08:32 -0800 Message-ID: <396c640b-81bb-4be4-860d-7ab3ff667795@amd.com> Date: Wed, 28 Jan 2026 09:38:31 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 4/4] sched/fair: Proportional newidle balance From: K Prateek Nayak To: Peter Zijlstra CC: Mario Roy , Chris Mason , "Joseph Salisbury" , Adam Li , Hazem Mohamed Abuelfotoh , Josh Don , , , , , , , , , References: <20251107160645.929564468@infradead.org> <20251107161739.770122091@infradead.org> <8760001e-0274-454c-a4e4-1f38a9695b88@gmail.com> <20260123105046.GM171111@noisy.programming.kicks-ass.net> <20260123110306.GA217302@noisy.programming.kicks-ass.net> <52b5b07d-89ee-4266-a229-ce9a8c6f8cb1@amd.com> Content-Language: en-US In-Reply-To: <52b5b07d-89ee-4266-a229-ce9a8c6f8cb1@amd.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Received-SPF: None (SATLEXMB04.amd.com: kprateek.nayak@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3B:EE_|SA5PPF7F0CA3746:EE_ X-MS-Office365-Filtering-Correlation-Id: e7f1b67b-0b87-4a8d-d8b4-08de5e22f25e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|1800799024|36860700013|30052699003|19052099003|13003099007; X-Microsoft-Antispam-Message-Info: =?utf-8?B?azlSTUJVOU8vTkQwTkZOaUMxZHRTa25pa3NUVEpVVk8vVXQ0MDhFaTM0VXY0?= =?utf-8?B?Z21VZjMxbW10ZGlpR0t6ek5SY2dzNTlDZkhGYXoxTjIxMlROWjNVT1VFTE5X?= =?utf-8?B?dTRTaVRhMmM1dUlyUjB5YzI1ZTR6SVFRT3AzZTBxWnBHRTNYcGZHZ012RGlW?= =?utf-8?B?bFBFVGZvc1BXZlJ6bG95UkxRampvZ2IzWGg5d1djMmhsRytPQXN4bXhQalNh?= =?utf-8?B?MFlhaENHMzE1RHRlK05BRENFNkx6bGk0RXQrYldQbnhxR1JlNVpWOTJqVkM0?= =?utf-8?B?eEZYcXUyemdEVGV6aks1YkczS0FXaWV1aEF5Zm9HWjZjV1FJU1BKRGV4YVNG?= =?utf-8?B?b3VFRTlZWXZpMG1ZYnI0eFkwZ3dTWlR0WFhmekMxUjVmaDREVTFoOFNIczFY?= =?utf-8?B?WEJEZ2F3K2hJem4zVDBRR1hpTXQxQlZJNkJ2R3IydnhNTjF4a3B0eW5wTVQ2?= =?utf-8?B?UmQwUVUyak8xcEFzbUNBenZhVG5kREdGQ1c2eHRMa0pvdUlXVFNJbHJZdTlY?= =?utf-8?B?WVQ3UnRTdDZoZC9mTDYvdWNCZ2F2WkJabW53VW9BNThOekw3Y2tuV1J3TElS?= =?utf-8?B?YStXU2o1V09hSkY3SnBpWHRQYXFmeFFDTWpVVnAwZmY0ZWk0NW45RnJuazRa?= =?utf-8?B?TjBUUFNxMjhON3QvQmtTclRwU2FIaXZDNGRZbUxuL21KMXpJZlZKZEVhTXNq?= =?utf-8?B?UnF1RVJ6dXJDdEdMOVFZTTVwRk9tS2I2bmxHMEpGd2haYnNSTVJtRUUyNC9j?= =?utf-8?B?NXp1RWtmZ3hDc1RLZUIwSWVORlJ4UGxpU2x4WVJheUdFbGEyby9FZU5nM0dL?= =?utf-8?B?RHRCNzNROWc2TkZPQU16NG1jeE82M1N3WHhLdDJSWkJ4Y0JrOWpLaEZvUHNJ?= =?utf-8?B?RitQQWN1S3RZdzJJM3BjMU9XdDU2R1Nkc0pkb2dmZUk0ZGRUcVRxQ2lHZDVw?= =?utf-8?B?Y0F3VGYrOFV2ZStkMmE4ZFBQdEptSXNFRGhCdkFMYmVKSUZ3UDVDbW9kS2hv?= =?utf-8?B?dFdyZDhYNDZONVc5a2RlMkhtdHBENjlaZW14OXFtVk5RMFRZQlN0ME5PVFNY?= =?utf-8?B?QldsVnJKekdubVNKKzVFcGxDTkNvekpXcnRzSk1ud3dZVnptWHEySVJvT2xV?= =?utf-8?B?djNsdXQ1WFRlTTBuMm5pQmc0ZTMzbTNvZThBNEdycTJKNThraGlYTmg5cUk2?= =?utf-8?B?ZEtETEpKU3VYV0Vwc0hwWDVMaTBvU053Yi8vbWhLVjJYMWFPZmhLbUNhUWlC?= =?utf-8?B?dkFnb1JLeTNIdFd0TGRGWllrbHFpbjhobE9HNmJwdXMzZE44aWpWNUNqM3hV?= =?utf-8?B?eDM2Uy9EMW5tT3A4d29DdkRocnduWGxscjJaSm5UMG15NGM5RTBHYXFodUhn?= =?utf-8?B?SUJoTjNjdXNmQ0VFdE9HdTJjYnR4NkJLeHU5dkRqanMwbEZYd0plWk5nZEQ4?= =?utf-8?B?SVVWOFllTG5raGJZYXBYWXV5VzJSNURFR1FqRkRFeldaS2RhME9EM2RDN2hn?= =?utf-8?B?VnJXZC91dEgrRHVqUmtZN3NHSEMreldLbTlYREtMVnhWdnAzWEtRck5LK2Qy?= =?utf-8?B?ZEd5NWoybUxwclk0YXRMNzNZdXE3Qk14c3hMTUcyZDVHZUV5SU1PMzY4RGlm?= =?utf-8?B?cmY3ZTlmZlJ6S3BkdWdoNHVLeHUzbzhsWUFMTy9WZnZtalpaQ1RqV1EvWno1?= =?utf-8?B?WW5VaTdCNHVjRnl1R1NnT2p6citpZkR2dVYwUDA1TG1UL29pYnlNR2xESUp3?= =?utf-8?B?cyt2N1l2MXpEaklkQW14U0ttUnJOdTdWSGtTLzF5WGY1eHY0aHMyamdFWU54?= =?utf-8?B?TlN3RWcyZE03VVErL3g2ZEtTSUVXYnYwdUNra2NLbkpVNm1yc0RVczB2U3d5?= =?utf-8?B?V28ySUphSllabVk3WmtkMUE5OWdPaEQ5ajRBelpxcFdSSVJBa1VlV014S2M1?= =?utf-8?B?MU94bFkzWnZNN01TM3pRVWp4Zk5OR1NNcHd1M2o2VHFCVmpQd2NEMVUyajVu?= =?utf-8?B?amFubHBaeEdJU253b3QrOUNSbEdudjdTN1NFOGlLYTUycWVIN25KdGUxOHgv?= =?utf-8?B?WXFhWk16QmlUeVh2V2U2T3Fzc0loakY4TnZMNzJrbVNlSk1lV2d0d3VYL1hi?= =?utf-8?B?ZUN2R0NzTE9ySDdVUnVNY2F2ckhnMFVoejhUdkI2UkVYWVVhS1YzQ2x0SWNL?= =?utf-8?B?R2hPOThUU2pic1JqbXR3YTlDdGV3VE13aFlaZlZRbWdRVGxHSE9Gc2F2d2Vk?= =?utf-8?Q?qjjCFFdEMPRPlc6VKHMtFVi7CewvpkMtZXcvfAVFTw=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(1800799024)(36860700013)(30052699003)(19052099003)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2026 04:08:52.0511 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7f1b67b-0b87-4a8d-d8b4-08de5e22f25e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3B.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA5PPF7F0CA3746 On 1/23/2026 5:54 PM, K Prateek Nayak wrote: > Hello Peter, > > On 1/23/2026 4:33 PM, Peter Zijlstra wrote: >> On Fri, Jan 23, 2026 at 11:50:46AM +0100, Peter Zijlstra wrote: >>> On Sun, Jan 18, 2026 at 03:46:22PM -0500, Mario Roy wrote: >>>> The patch "Proportional newidle balance" introduced a regression >>>> with Linux 6.12.65 and 6.18.5. There is noticeable regression with >>>> easyWave testing. [1] >>>> >>>> The CPU is AMD Threadripper 9960X CPU (24/48). I followed the source >>>> to install easyWave [2]. That is fetching the two tar.gz archives. >>> >>> What is the actual configuration of that chip? Is it like 3*8 or 4*6 >>> (CCX wise). A quick google couldn't find me the answer :/ >> >> Obviously I found it right after sending this. It's a 4x6 config. >> Meaning it needs newidle to balance between those 4 domains. >> >> Pratheek -- are you guys still considering that SIS_NODE thing? That >> worked really well for workstation chips, but there were some issues on >> Epyc or so. > > SIS_NODE was really turned out to be a trade-off between search > time vs search opportunity, especially when the system was heavily > overloaded. > > Let me rebase those old patches and give it a spin over the weekend > on a couple of those large machines (128C/256T and 192C/384T per > socket) to see the damage. I'll update here by Tuesday or post out > a series if I see the situation having changed on the recent > kernels - some benchmarks had a completely different bottleneck > there when we looked closer last. So these are the results on tip:sched/core merged onto tip:sched/urgent with SIS_NODE and SIS_NODE + SIS_UTIL [1] on a 512 CPUs machine with (2 sockets x 16 CCXs (LLCs) x 8C/16T Zen4c cores): tl;dr (*) Consistent regressions, even with SIS_UTIL bailout on higher domain; Benchmark are mainly measuring tail-latency or have a thundering heard behavior that SIS_UTIL uwith default imbalance_pct isn't able to fully adjust to. (#) Data has run-to-run variance but is still worse on average. Note: Although "new-schbench-wakeup-latency" shows regression, the baseline is few "us" and a couple more "us" addition appears as a ~ 20%-30% regression. I'm still fighting dependency hell to get some of the longer running benchmarks running on this system but I expect a few pct regressions like last time [2]. System: - 2 x 128C/256T Zen4c system with 16CCXs per socket - Boost on - C2 disabled - Each socket is a NUMA node Kernels: tip: tip:sched/core at commit 377521af0341 ("sched: remove task_struct->faults_disabled_mapping") merged onto tip:sched/urgent at commit 15257cc2f905 ("sched/fair: Revert force wakeup preemption") sis_node: tip + sis_node patch + cpumask_and() moved to after SIS_UTIL bailout [3] sis_node: Tree from [1] based on tip:sched/core merged onto tip:sched/urgent Full results: ================================================================== Test : hackbench Units : Normalized time in seconds Interpretation: Lower is better Statistic : AMean ================================================================== Case: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1-groups 1.00 [ -0.00](11.61) 0.76 [ 24.30]( 4.43) 0.76 [ 24.05]( 2.93) 2-groups 1.00 [ -0.00]( 9.73) 0.86 [ 14.22](17.59) 0.80 [ 19.85](15.31) 4-groups 1.00 [ -0.00]( 5.88) 0.78 [ 21.87](11.93) 0.78 [ 21.64](14.33) 8-groups 1.00 [ -0.00]( 2.93) 0.92 [ 8.44]( 3.99) 0.92 [ 7.79]( 4.04) 16-groups 1.00 [ -0.00]( 1.77) 0.90 [ 10.47]( 5.61) 0.94 [ 5.92]( 5.65) ================================================================== Test : tbench Units : Normalized throughput Interpretation: Higher is better Statistic : AMean ================================================================== Clients: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1 1.00 [ 0.00]( 0.20) 1.00 [ -0.07]( 0.16) 1.01 [ 0.53]( 0.23) 2 1.00 [ 0.00]( 0.35) 1.00 [ -0.03]( 0.58) 1.00 [ 0.12]( 0.20) 4 1.00 [ 0.00]( 0.09) 1.01 [ 0.60]( 0.60) 1.00 [ 0.16]( 0.15) 8 1.00 [ 0.00]( 0.63) 1.00 [ -0.35]( 0.53) 1.00 [ 0.26]( 0.19) 16 1.00 [ 0.00]( 0.97) 1.00 [ 0.33]( 0.30) 1.01 [ 1.16]( 0.50) 32 1.00 [ 0.00]( 0.98) 1.02 [ 1.54]( 0.91) 1.01 [ 1.10]( 0.26) 64 1.00 [ 0.00]( 3.45) 1.02 [ 1.88]( 0.48) 1.02 [ 1.78]( 1.29) 128 1.00 [ 0.00]( 2.49) 1.00 [ -0.01]( 1.63) 0.99 [ -0.68]( 1.88) 256 1.00 [ 0.00]( 0.57) 1.01 [ 0.73]( 0.45) 1.01 [ 0.92]( 0.35) 512 1.00 [ 0.00]( 3.92) 0.51 [-48.55]( 0.11) 0.80 [-19.59]( 6.31) (*) 1024 1.00 [ 0.00]( 0.10) 0.98 [ -2.11]( 0.09) 0.97 [ -3.29]( 0.28) 2048 1.00 [ 0.00]( 0.09) 0.98 [ -2.08]( 0.28) 0.99 [ -0.75]( 0.48) ================================================================== Test : stream-10 Units : Normalized Bandwidth, MB/s Interpretation: Higher is better Statistic : HMean ================================================================== Test: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) Copy 1.00 [ 0.00]( 0.31) 0.99 [ -0.70]( 0.57) 1.00 [ -0.09]( 1.44) Scale 1.00 [ 0.00]( 0.38) 0.99 [ -1.00]( 0.49) 1.00 [ 0.32]( 1.41) Add 1.00 [ 0.00]( 0.31) 0.99 [ -0.95]( 0.63) 1.00 [ 0.43]( 1.16) Triad 1.00 [ 0.00]( 0.18) 0.99 [ -0.84]( 0.68) 1.00 [ 0.16]( 1.12) ================================================================== Test : stream-100 Units : Normalized Bandwidth, MB/s Interpretation: Higher is better Statistic : HMean ================================================================== Test: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) Copy 1.00 [ 0.00]( 1.46) 1.00 [ 0.39]( 1.57) 1.01 [ 0.82]( 0.52) Scale 1.00 [ 0.00]( 1.45) 1.00 [ 0.49]( 1.37) 1.01 [ 1.20]( 0.55) Add 1.00 [ 0.00]( 1.09) 1.00 [ 0.31]( 0.94) 1.01 [ 0.79]( 0.35) Triad 1.00 [ 0.00]( 1.06) 1.00 [ 0.22]( 1.02) 1.01 [ 0.56]( 0.19) ================================================================== Test : netperf Units : Normalized Througput Interpretation: Higher is better Statistic : AMean ================================================================== Clients: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1-clients 1.00 [ 0.00]( 0.27) 0.99 [ -0.82]( 0.26) 0.99 [ -0.78]( 0.16) 2-clients 1.00 [ 0.00]( 0.28) 0.99 [ -0.87]( 0.19) 1.00 [ -0.17]( 0.67) 4-clients 1.00 [ 0.00]( 0.38) 1.00 [ -0.47]( 0.33) 0.99 [ -0.53]( 0.31) 8-clients 1.00 [ 0.00]( 0.34) 0.99 [ -0.55]( 0.18) 1.00 [ -0.33]( 0.24) 16-clients 1.00 [ 0.00]( 0.30) 1.00 [ -0.39]( 0.23) 1.00 [ -0.19]( 0.26) 32-clients 1.00 [ 0.00]( 0.43) 1.00 [ -0.40]( 0.57) 1.00 [ -0.24]( 0.68) 64-clients 1.00 [ 0.00]( 0.82) 1.00 [ -0.12]( 0.45) 1.00 [ -0.14]( 0.70) 128-clients 1.00 [ 0.00]( 1.21) 1.00 [ 0.10]( 1.28) 1.00 [ 0.08]( 1.19) 256-clients 1.00 [ 0.00]( 1.38) 1.01 [ 0.65]( 0.89) 1.00 [ 0.34]( 0.89) 512-clients 1.00 [ 0.00]( 8.76) 0.47 [-52.76]( 1.64) 0.77 [-23.10](10.06) (*) 768-clients 1.00 [ 0.00](34.29) 0.83 [-16.89](30.45) 0.98 [ -2.16](36.19) 1024-clients 1.00 [ 0.00](47.96) 0.91 [ -9.29](36.02) 0.98 [ -1.93](46.36) ================================================================== Test : schbench Units : Normalized 99th percentile latency in us Interpretation: Lower is better Statistic : Median ================================================================== #workers: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1 1.00 [ -0.00](14.20) 1.72 [-72.00](15.01) 0.88 [ 12.00]( 4.55) 2 1.00 [ -0.00]( 1.68) 1.09 [ -8.82]( 6.96) 0.97 [ 2.94]( 9.90) 4 1.00 [ -0.00]( 4.45) 1.18 [-17.65]( 5.29) 1.03 [ -2.94]( 3.24) 8 1.00 [ -0.00]( 2.44) 1.12 [-12.20]( 4.35) 1.02 [ -2.44]( 2.38) 16 1.00 [ -0.00]( 0.00) 1.04 [ -3.64]( 1.75) 0.98 [ 1.82]( 1.85) 32 1.00 [ -0.00]( 2.87) 1.03 [ -2.53]( 2.80) 0.99 [ 1.27]( 1.47) 64 1.00 [ -0.00]( 3.17) 1.02 [ -1.57]( 5.72) 0.98 [ 2.36]( 2.30) 128 1.00 [ -0.00]( 2.95) 1.01 [ -1.35]( 3.03) 1.00 [ -0.00]( 1.13) 256 1.00 [ -0.00]( 1.17) 0.99 [ 1.23]( 1.75) 0.99 [ 1.43]( 1.56) 512 1.00 [ -0.00]( 4.54) 1.14 [-13.60]( 2.41) 0.97 [ 2.50]( 0.42) 768 1.00 [ -0.00]( 2.24) 1.27 [-27.44]( 3.18) 1.12 [-11.54]( 5.64) (*) 1024 1.00 [ -0.00]( 0.28) 1.14 [-14.20]( 0.56) 1.13 [-13.00]( 1.01) (*) ================================================================== Test : new-schbench-requests-per-second Units : Normalized Requests per second Interpretation: Higher is better Statistic : Median ================================================================== #workers: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.15) 2 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.15) 4 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.00) 1.00 [ 0.29]( 0.15) 8 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.00) 1.00 [ 0.29]( 0.00) 16 1.00 [ 0.00]( 0.15) 1.00 [ -0.29]( 0.15) 1.00 [ 0.00]( 0.00) 32 1.00 [ 0.00]( 0.15) 1.00 [ -0.29]( 0.00) 1.00 [ 0.00]( 0.15) 64 1.00 [ 0.00]( 0.00) 1.00 [ 0.00]( 0.00) 1.00 [ 0.29]( 0.00) 128 1.00 [ 0.00]( 0.27) 1.00 [ 0.00](18.48) 0.65 [-34.50](24.12) (#) 256 1.00 [ 0.00]( 0.00) 0.99 [ -0.58]( 0.00) 0.99 [ -0.58]( 0.00) 512 1.00 [ 0.00]( 1.05) 1.00 [ 0.00]( 0.20) 1.00 [ 0.39]( 0.87) 768 1.00 [ 0.00]( 0.95) 0.98 [ -1.88]( 0.93) 0.99 [ -0.71]( 0.53) 1024 1.00 [ 0.00]( 0.49) 0.99 [ -0.81]( 0.57) 1.00 [ 0.00]( 0.74) ================================================================== Test : new-schbench-wakeup-latency Units : Normalized 99th percentile latency in us Interpretation: Lower is better Statistic : Median ================================================================== #workers: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1 1.00 [ -0.00]( 6.74) 2.38 [-137.50](29.34) 1.75 [-75.00]( 9.53) 2 1.00 [ -0.00](12.06) 1.27 [-27.27]( 9.53) 1.36 [-36.36]( 6.59) 4 1.00 [ -0.00](11.71) 1.33 [-33.33]( 3.30) 1.33 [-33.33]( 3.16) 8 1.00 [ -0.00]( 0.00) 1.27 [-27.27](12.69) 1.09 [ -9.09]( 4.43) 16 1.00 [ -0.00]( 4.84) 1.09 [ -9.09]( 4.43) 1.18 [-18.18](10.79) 32 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1.10 [-10.00]( 4.56) 64 1.00 [ -0.00](13.22) 1.00 [ -0.00]( 5.00) 1.00 [ -0.00]( 9.68) 128 1.00 [ -0.00]( 8.13) 1.00 [ -0.00]( 8.85) 1.18 [-18.18](13.76) 256 1.00 [ -0.00]( 2.97) 1.02 [ -1.94]( 3.80) 1.08 [ -7.77]( 7.13) 512 1.00 [ -0.00]( 1.25) 1.00 [ 0.37]( 0.68) 1.00 [ -0.37]( 1.81) 768 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1024 1.00 [ -0.00]( 0.63) 1.00 [ -0.11]( 4.06) 1.00 [ -0.11]( 3.13) ================================================================== Test : new-schbench-request-latency Units : Normalized 99th percentile latency in us Interpretation: Lower is better Statistic : Median ================================================================== #workers: tip[pct imp](CV) sis-node[pct imp](CV) sis-node-w-sis-util[pct imp](CV) 1 1.00 [ -0.00]( 0.14) 1.00 [ -0.26]( 0.14) 1.00 [ -0.00]( 0.14) 2 1.00 [ -0.00]( 0.14) 1.00 [ -0.26]( 0.00) 1.00 [ -0.00]( 0.14) 4 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1.00 [ 0.26]( 0.14) 8 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1.00 [ 0.26]( 0.14) 16 1.00 [ -0.00]( 0.00) 1.00 [ -0.00]( 0.00) 1.01 [ -0.53]( 1.18) 32 1.00 [ -0.00]( 0.54) 1.01 [ -1.05]( 0.59) 0.99 [ 0.53]( 0.27) 64 1.00 [ -0.00]( 0.00) 1.00 [ 0.26]( 1.08) 1.00 [ 0.26](31.75) 128 1.00 [ -0.00]( 0.61) 1.00 [ -0.00]( 4.19) 1.10 [-10.22]( 4.79) (#) 256 1.00 [ -0.00]( 0.43) 1.01 [ -1.39]( 0.74) 1.02 [ -1.63]( 0.66) 512 1.00 [ -0.00]( 3.32) 1.00 [ 0.23]( 1.62) 1.04 [ -3.72]( 3.79) 768 1.00 [ -0.00]( 0.88) 0.95 [ 4.52]( 0.63) 0.98 [ 1.94]( 0.54) 1024 1.00 [ -0.00]( 1.01) 0.98 [ 1.54]( 0.91) 1.00 [ 0.17]( 0.31) Let me go play around with imbalance_pct for SIS_UITL at PKG/NODE domain to see if there is a sweet spot that keeps everything happy while things are happier on average. I doubt if Meta's workload will be happy with more aggressive SIS_UTIL limits since data from David's SHARED_RUNQ series [4] showed that specific workload requires aggressive search + aggressive newidle balance. References: [1] https://github.com/kudureranganath/linux/commits/kudure/sched/sis_node/ [2] https://lore.kernel.org/all/3de5c24f-6437-f21b-ed61-76b86a199e8c@amd.com/ [3] https://github.com/kudureranganath/linux/commit/7639cf7632853b91e6a5b449eee08d3399b10d31 [4] https://lore.kernel.org/lkml/20230809221218.163894-1-void@manifault.com/ -- Thanks and Regards, Prateek