From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013029.outbound.protection.outlook.com [40.93.201.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA3CE253932 for ; Thu, 19 Mar 2026 08:45:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.29 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773909947; cv=fail; b=cxvQ1nSK8nQUe8O502joDiJh5Kwa676WWNsTJfDRINjcffrLQ89HcP0ieUO4qrT+gCuNVwz62jNShZk+beZKQ1LkwRbNSNE0+7+RKoxZ6OuyrsygFGkeUSiC91mRF0LYgcSM544Qyy9oPHy3qqTAb3EnaFl0aO3us8J8o53Ppz8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773909947; c=relaxed/simple; bh=yM8jWXzlqLOMkD4BYqWDr6fFSYWAHqS5Pxehd3/x2Bs=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=lopdlL3H54kE+jp4oyAmEfibvplQgVSpxSCASgSjvOid0cJKn33bwMKklIExdrJR4JwgAVYapJgO27PoF7HNu1FWE8JL+rEifN6RHRn1U93zVEQX8wQsFGsHda0D/grb6AwW20ww/1TUwXxgMh4mkiFSZveH0KVaa+I82ylml4o= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=bZyfeRQS; arc=fail smtp.client-ip=40.93.201.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="bZyfeRQS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=i614ZwSA/uA1BPYt9CTW3u4wnQtCTrbnl8qv8etBbwbgzI0dPqsIV/veAMqri2mqDZr5T9MiR7L/AZt6B/cLztHOQL0ybjOA/Y3eAKvv0Jicc1dF6ybKJXisp5NFHrX++9uqVFvq7y7UHNI1UbLtltG6qlPbY0vxjC3Pi+cznXC05uLj94N6Qo1l0LxYH1nRpTfnlYv5VPnzHOrx0sidIEmYa09aDp/EuOhjUrt0fcmnt9XIjuOFLfva4XNkbCpSlJqzerDoaEUihfUK4jWbvrB3YHBnsZPVKHmciACN5t+F/78E1dFd+rZjx4HhLb8KUT8yxqNJAyOp5DCtp2hFcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wXMJvrC8EfPrW3tXW21tWRtH/G0eaZjtnFKIeka6UTc=; b=ARUyYwGGmJxVKqtJGfspWW3aw3oTrAmPoszBuIF5cX8OrdnDYOrN/YdWnGvbHCglujtBKomu2eEX2L57HWrQLw2+4o8BRJn+1sMiFmjZiKCiVS4aHqfZQD1H2AvFnFbNFtaJBBX9GPdJSbQ4zyGHlnLIvN8tae7sF2pGna2j2MVLuw/9PXoIYwzDhCVIRQ6ndJEidfyOdG97RXAI9mB6z8cG865BSVHgtRGa72enk3zcAW5kUd1CBURW3QaJi4PDAQvKJweVanT1mLVr3o1KBWMWXssu6HjeaTDNNLIApr8s5qWDlUy1MYTScN2ck1aw7l1L8DUUrgfa8xHO+Wb5Ww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wXMJvrC8EfPrW3tXW21tWRtH/G0eaZjtnFKIeka6UTc=; b=bZyfeRQSsRNbUy//l7vyOxoP0z/HUbPBzZ0nDJ1BlwvOgIHSVpbTokvVQPdX1a0786DfU8mA6XHJPvcOGkhASh22yRP2nMI5qYfLQTCveZipgqUP5nMgMcPbZzIuY/PyaShPK3QnQFk67gqYTQSE5j8JWqfa++RdZkNBp44lJHV1amSplr3vg9pgxpLK60mX7L31FFRBOOs2MJLL0Tu78gMDtamjbX9SC4CKKjS3KgoHJZAzA5U14QalCoCXXKzFqaZxGjcgmZp1YJFrw2RdedfK4N6GJZpPGiEdg+rC8fnXYsZxveO+4WYqoaLpCGUSqmwIQO95g/z6SEMKdnNd3w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by LV9PR12MB9760.namprd12.prod.outlook.com (2603:10b6:408:2f0::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19; Thu, 19 Mar 2026 08:45:43 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9723.010; Thu, 19 Mar 2026 08:45:37 +0000 Date: Thu, 19 Mar 2026 09:45:34 +0100 From: Andrea Righi To: Vincent Guittot Cc: Christian Loehle , Ingo Molnar , Peter Zijlstra , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Joel Fernandes , linux-kernel@vger.kernel.org Subject: Re: [PATCH] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection Message-ID: References: <20260318092214.130908-1-arighi@nvidia.com> <4830a5aa-0682-4501-af92-8a2e7858b1d3@arm.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MI2P293CA0010.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:45::20) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|LV9PR12MB9760:EE_ X-MS-Office365-Filtering-Correlation-Id: 10fca7f7-3077-46f9-dd8c-08de8593e488 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: ow4XHlWGQ20eS1aBm96DtqNlwsVoLtdm7Zot8xtvdZ3jKLuD+/hT1pZu778wUrYMXo36wsajzG0nw44gdp+rtvwGa5nvjZvOxp0M9gcDHIwIgN2jbhg2zl9k9oqjirYKCe/hgXc5o5Y4nI2bex9k09UycU9F6klPUzO8bvzLaqNyHP6gjJKL+/MICkmyYVWvn04eZUUQkjOBX5uyRMAgM2+Le38PbaplkcDyXz9UcYdwgK5XZonxNe8lbdKEZCjRqBmvbp2+Abva1pUbpFcAHPY6CAcm9gv7J/yLKZP/cIZBQFCoN9uvI+L+luxdQkOPaE0nPRfD80JsCNW+Sjd2cHdK30xA8Qv0NOtEcXRu8wnlN72R5sl0/ofqB0GY9xZU9N98Jb2Qd3Vek5udSPsqEFojOYmMMmOBg4SnGIXH0XDPgw78OzS9oKxFkwxjFsW4ADgCVVFx2ozRhpIIJdqE0goDRJuow5o6tBvFdWVTxVz0HUTSB72gWQUsEwI3a1RM8+CM8uhnOS4w8CeykB669CAsTKWYQXPqjXNLAla87Slnfy3Lafcnxwwm85VBNGktxzAAbSG4fctCMcIdGaLanzrQe7gYJMmcpo65XO6j8Bjt1b3GiMXMoUoRzQRL27KbmIm1C9fPtBWn1aQsOT2H8GA3gt89ED1oLhcDEQi36BnosnS2wi983zqsQukvV0E4jnuz9JWKSOL+CtUcAQyZj/uDhtxE/L5qroxVL/uBMvk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?atcD6gpp86XwIMQYgbuEtm+DZJmX//wXLObnApY5GL6Xndb3vXd/vkFDdIC3?= =?us-ascii?Q?cbivSt786+7Mwu9KMrVHJNaHNWCbvR7b9nSIz3euzcSm6XNklII6QPj9LtSK?= =?us-ascii?Q?DkfRLbIIK6cxsqXsihVaZATuSeva54T+UOQyxxDVb+M3UxPrDPxPvF+Cblvq?= =?us-ascii?Q?IpkAUUPjakVM4nWzvAH7kdel6/kUbbFQFBQL7o1VYAC7trFtI4zq0FJ/VS/z?= =?us-ascii?Q?JU4YponZTEZiAHkirIa8XabKvo3J19B9vU4h6jBPCNQOwGFhiVDo5XlFsA9D?= =?us-ascii?Q?UQzuCJGJQOMZJKqnitIeAmzN/VGUldlaiT5PqOLiq5eu7v+b8k9g5IE82cXu?= =?us-ascii?Q?TwFloOBHxghJBcIkcMoaJLr2dySBl7qZ4FIc9TPad32TdJexcLrZL8H1gn91?= =?us-ascii?Q?zp9AdZ4ia8j9Gv3zXSZacBYrP10TFK55oNOmqMQORxSgn0ptQOcAoNbU9Lw1?= =?us-ascii?Q?XXOQ6nAXuJZvAtlXP3VEBLM9WUcXEjO3B5KOPykbyDDJz0k9khVDra7O38lM?= =?us-ascii?Q?h0yQDF1BjxNYqZc5nBdQSksT5utAKhkLYjI1kpZfd8iAc2yNIKYGG7JfI/Hn?= =?us-ascii?Q?RLQ2kpMlI5bxSyIQUWhlG+L4eOgTFV6Bqw0emnDGfmxsACeZiqjIy1nmxko5?= =?us-ascii?Q?mb50ExcCFBOA7W7sJzap4bJQ9KGz/my4vQUQz7Opw8Zw8F04huf85NJuHD3C?= =?us-ascii?Q?xPaPD+xfk/Lk0wknUVKM/4fhn4f4AX9hxzgKO0x5H54mKNk2VkJJvXBUaofd?= =?us-ascii?Q?+tmaKLDBPlOe+KtPxL5RqSlbQjhzhCojB2+3SHvX7VTGbEmmJtdEHQazyTg7?= =?us-ascii?Q?fLsMu7crNRZVCNibPEufHLj6Y/0QMVgnLxOHo7rWZjSJY/xbkv0/7j/E/HA6?= =?us-ascii?Q?6JU9E8KCsdO7h0mRnYPUHD7jT9FlEJnriY2cz6cn3ep5y48IttYq7Bda9K17?= =?us-ascii?Q?q16FErzanANSYO2rXA0D1esAVikzJtwrmlXUCAafaMp4SW/0P+bBqfUsAuwt?= =?us-ascii?Q?fcmYZBRlqELr9cFi+wh54/YiIecvYqt9gFiEVmBFp73egh4pZbY4dH0PPsaE?= =?us-ascii?Q?XhRtlGnxYTnImd9qTYPo6gHfmCdj6uLVhGpiKiEmTqo/leUAYvAfpclD7vUg?= =?us-ascii?Q?WBd4D7tbKZYuVvaIuKbNjaC8DIlrD1vu0FdonlT3/uOZXdF5rOBnFvWDJ176?= =?us-ascii?Q?EsFD1mj+bJ84xvtM95tFjJPNi5rCJQO7+hSjp7w5tiKmPSvDGqcaO7nuXxIb?= =?us-ascii?Q?UlkMcwEWvoHgeL8CWdEemlE36pGYmAX9dhPfLpQls9g1IUaUqW+NcJLXy2OS?= =?us-ascii?Q?0q8j4XoSiiYAAdfMwITfSabF7sMnIhJmOODVIgMFI/d1+QEfbItTTIjhxwDK?= =?us-ascii?Q?0KDSVqWeAVFJWqIMwEFKkYtvMvNWiy2K35oynPymqQ0ZDVmNI1JEMmfH/aap?= =?us-ascii?Q?yI6Rq8lI01sVWHbOA4/45wmkL0MlUsnHdizNe3cN/s65ZrecV8OJYwoAsueQ?= =?us-ascii?Q?fl3gwbZvk77MGGw1JiJYP3wFcutCHD0mrC8dVqJakv8py2Zjfr4iOVI8MMj2?= =?us-ascii?Q?wXru+mEFLWoebi2BauCJgFHGrfi3RwQ++5PM7hT7O2ACotY9dmqlhtkuzdGg?= =?us-ascii?Q?FV67n9livWdTy2QXTvwqrr/fdLAaG/YcQZJAdGzCfYs5K+gXIsuO8wuXTzZn?= =?us-ascii?Q?6UbFautHDAfYaOmyUO5XHbJzNxz57r/y9g2aCXJSrCglW4H5?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 10fca7f7-3077-46f9-dd8c-08de8593e488 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2026 08:45:37.5754 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MVQS5HL13RzzqhFAjv/Zzx1LYThQWI09bzAB4pEFY1H1klHfRKPShrxi+vmIFSqzkIPPIb/1LliyfUuEdpXeZQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV9PR12MB9760 Hi Vincent, On Thu, Mar 19, 2026 at 08:20:27AM +0100, Vincent Guittot wrote: > On Wed, 18 Mar 2026 at 18:09, Andrea Righi wrote: > > > > Hi Christian, > > > > On Wed, Mar 18, 2026 at 03:43:26PM +0000, Christian Loehle wrote: > > > On 3/18/26 10:31, Andrea Righi wrote: > > > > Hi Vincent, > > > > > > > > On Wed, Mar 18, 2026 at 10:41:15AM +0100, Vincent Guittot wrote: > > > >> On Wed, 18 Mar 2026 at 10:22, Andrea Righi wrote: > > > >>> > > > >>> On systems with asymmetric CPU capacity (e.g., ACPI/CPPC reporting > > > >>> different per-core frequencies), the wakeup path uses > > > >>> select_idle_capacity() and prioritizes idle CPUs with higher capacity > > > >>> for better task placement. However, when those CPUs belong to SMT cores, > > > >> > > > >> Interesting, which kind of system has both SMT and SD_ASYM_CPUCAPACITY > > > >> ? I thought both were never set simultaneously and SD_ASYM_PACKING was > > > >> used for system involving SMT like x86 > > > > > > > > It's an NVIDIA platform (not publicly available yet), where the firmware > > > > exposes different CPU capacities and has SMT enabled, so both > > > > SD_ASYM_CPUCAPACITY and SMT are present. I'm not sure whether the final > > > > firmware release will keep this exact configuration (there's a good chance > > > > it will), so I'm targeting it to be prepared. > > > > > > > > > Andrea, > > > that makes me think, I've played with a nvidia grace available to me recently, > > > which sets slightly different CPPC highest_perf values (~2%) which automatically > > > will set SD_ASYM_CPUCAPACITY and run the entire capacity-aware scheduling > > > machinery for really almost negligible capacity differences, where it's > > > questionable how sensible that is. > > > > That looks like the same system that I've been working with. I agree that > > treating small CPPC differences as full asymmetry can be a bit overkill. > > > > I've been experimenting with flattening the capacities (to force the > > "regular" idle CPU selection policy), which performs better than the > > current asym-capacity CPU selection. However, adding the SMT awareness to > > the asym-capacity, seems to give a consistent +2-3% (same set of > > CPU-intensive benchmarks) compared to flatening alone, which is not bad. > > Do you mean that this patch is +2% > vs plain SMP > than current asym > cpucapacity implementation ? Yes, that's correct. More exactly: speedup % ------------------------------+------------ current asym CPU capacity | - equal CPU capacity | +13.6% SMT-aware asym CPU capacity | +15.0% Thanks, -Andrea