From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010040.outbound.protection.outlook.com [52.101.56.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A6293F8807 for ; Wed, 1 Apr 2026 12:42:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.40 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775047360; cv=fail; b=cZqolO8Xd4qwdkWqPDKadyDgbMR040J4fD+SpfNO6sWYQ7EwzOsOAuoVhqRNugmWGGJamnETvcmNFgyMKlVH+ni6OIgkKRnvezi+8gvyrD/X3mxfduChg8kn+30wYSIrZeWOGISKZybvW4vn71Jz+K1vVsCpTLUmM7EjDfBq+h8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775047360; c=relaxed/simple; bh=mbAKcf63rvGH1GWkDR1e2v0wHFcphhLqtImO8WWbZLk=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=MsEPipA57PaYWAmzBXHiCYeXf9Te+GT7GTapnP0NcpN1oAMWn1sJZym7LLQzqBPQijyBiKL7lxXvuV8yDHUuavLdxtHB/SOHCkBOxB6qaeV0F8fBOdLRhhRIDsvJFyBdJ13TJ5+qni5HuKtjaxKF4Dlye8RZR7eBsR06qwqxRxQ= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=lKcK5uNn; arc=fail smtp.client-ip=52.101.56.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="lKcK5uNn" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MAlNgwbdPGVwHaqSK2Vk5sd8qCo3q8MSoGt/69YdyoZnuCGdbX99m1O3/xj+c6pZOn1y3qZoQlAQvTUKWoIubY5OCaTdCe7VqgkHBql9uELBb69dFV3RaGTAqqU1Qj49lKQHOChe9mIjpnjNSuUJAalsW2DagvnvCCH1qQg6YkEGxP5/R5mU6JiyzRcxmm6eSA/EATFrSjeNmRGtHC4doukhuK2gdZV1ZHAhC8MnelXgz4AYR9scLJ1eomQHAFk3lOrvTN6oOeNKfREb+ZmA2KDHGVEoTxYoRRESvJ5j9yFGWDjCF0pVmliujJ33dVFeyi34e8EowFlijQ86+ebi0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9dwVWozL0tj4IWr+axLlm/w2oIsBE2Bk2O3hGKCV9o4=; b=L3z1wgHfV3daeMYSvxk/8QjB/X5A/a+Qv+xCKQDYSxMjbn/jaS8YejWEHjxRAaHIZUBj8EqRKJrKtXHG70ZlP4Gocv5ozyI3mD1hIZDnSzD09TWKKR/bjyijmxII/4rxGt5atEYd/XDYF7Fimlba3voDmULzKgDXsuoUMykOEu8j17ESPlin2O+FE1JQkJVqDU4cME1tD60QJDf17Loph0BciisbhFdtmt2443o+egXIoQ4ySVFFArfP7SN5qLNLTReojs4uByv2GW/6td86W+KjJHeJv9azr6v0ZdHKz1eR/JW+3Oxuvi8TQ6JGez4UjRtF/jwRywStzTWZe1kAEA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9dwVWozL0tj4IWr+axLlm/w2oIsBE2Bk2O3hGKCV9o4=; b=lKcK5uNnVlIPaoBrOV9BcJj+yRV7+lchazKWrVlsrCqTOeSq4j/SZPhUD775wZk2o08tkZYPA6Bvj23lCGFu2hdR9FOYTJY36xhfMj8EFMjDqxOkmNBvONS2u2WyK+fvGUxUMmeULyxmG+8dUgA01eArIb5ixAy+HbVRzjjHtmNAN+S+rjd1otNMJRgH4kuwncGa9sn3I7Cs3so/VUSh/LUForoslQEMCUalbFD5cRB2/3JKDPmIQGJu19roFJZ1r/9MVUx6wuXrkxLZztUFa82xqppylCBZsKJPga8aAwHMHavS5QNLQFVBLb3/69FjXiz/qEanCigjaevY4vdOdA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DS4PR12MB999078.namprd12.prod.outlook.com (2603:10b6:8:2fd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.18; Wed, 1 Apr 2026 12:42:34 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9769.016; Wed, 1 Apr 2026 12:42:33 +0000 Date: Wed, 1 Apr 2026 14:42:17 +0200 From: Andrea Righi To: Vincent Guittot Cc: Dietmar Eggemann , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/4] sched/fair: SMT-aware asymmetric CPU capacity Message-ID: References: <20260326151211.1862600-1-arighi@nvidia.com> <193f1cd1-ced3-4b37-83af-ea43a7e5e3d0@arm.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: ZR0P278CA0218.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:6a::8) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DS4PR12MB999078:EE_ X-MS-Office365-Filtering-Correlation-Id: dd652111-8b01-4294-80b1-08de8fec2523 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: h90vBGP7/Z1Y5JNc0+Grdk3GCozaF/FA+mFY/avMElE9H7gvLMcF2FO8NquMibHwjUn+hVpObmDlqFEXu34NI+JF2zUhbUYUagCMX4K9KHvFSlFHcLnxfpIyI3M6bKqh17ENqbdgtE/JuFrS1fVSAZMy0791Lz8wKFyFlW+t6pIW80vQzj7ApEl7EfwhkNW+eXMauDLSpVVy8/T6AWhzXCjvrI+9He9MtD6BCJbDBEbmsDSaxwjqry++t1N4oCEvkic94FXrbAkBjb3TYWbFPR5412LsSGmtlFvYFLkkbnYPsxr0mko3kQUdeoTv/wAOR6ij8CkM7G+Gj2JNCS6W/JM5yj6hCS/fQQyBYZpDM2UIVCa5RAGfXM1O5+yHEqs8Yio3neAejRjIojT6zxqOmb+37yziYMT6tPA6F1odIIBbONJospSxtSOHGvVACutr1JwkYJ7CMogc7l+bmarrA9WF+38/PhtlgJBILiBBnBn7fCUdG5zKsLnOphL4bu8ABlpOnFh+13FHPYxtS37Sf+Cr5uAQQnN0n78GsQOLun5XVtDsTgOILJ02prLnuTqM7zJhbE3dj4SICt3tZ4npLXEzVrGMZceJb7RPmHghctotcK50znDbztd3coMmB7XtLjEnDXMOAbZ1O1go4KahUNJ5hGVE9ccI1qdHES4N7hDGFkbzh28SYh5GTUfjx9gimYzkvGsLZHklraX8t/2fwRV9JecKQiDqss6x6vHNvqQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?hkPa79X6f/bWRQala4kIcWBt5XB3/mOG6AGcW6mlO8sZBo8BXfkQ973HdHPG?= =?us-ascii?Q?3c9qMYLurY+n2ho4PJKT0YH2Laq5Bb1aYwrWxhTymAEJVBb3y4A8o2vnWob5?= =?us-ascii?Q?OwbzoD4NKy9AAbD2PC36I/mMgmGDGsvps2xOU4BhbYwWFrKGFuf7wTwfwMMf?= =?us-ascii?Q?aQ8uU80qfVcKPOf3t88R1i8594on1atZQxz9i2LChOTJ/bFYQ+OO1Twvau5C?= =?us-ascii?Q?HHpXfvIqlsP09mMY4LZRwNuz1LVXEwSQZmaCb+kaKNXclH9j0PTiASAaYZWI?= =?us-ascii?Q?I5T/voZw2BSIU9JxOO9/rR3Kq+SPtGs83ap+QnNQSDflKHvl+raz72KxDA2S?= =?us-ascii?Q?odJyWaJFwMs48XZ6mrHW/rbdkoiBWHy+bgqteRNBhYhA9x4ayGoiDBeYF2sl?= =?us-ascii?Q?m2wzf6NhtMRsUC5/pLLZ2BIZrZVkefL5XJp+t4VtFBzu2C32fPUgEXotv1hL?= =?us-ascii?Q?VaNJZBqlpcBVGuwGwdPvYNu8mEQ4TDTs4iTCvJRnnuEPdnCvUvWwlABY4/3y?= =?us-ascii?Q?QzL3ylI0qLQ0oM4Vtieq9WIeRLjW8lDq3/7vOSyj/gnayST2AaRVYRTIn32h?= =?us-ascii?Q?+hsTvqtKJ7Etg1rPNGOaTLJ+KV7p78+J64rjcrLB79Fgb3snn1WUctdtqp52?= =?us-ascii?Q?VapshNgS7IT6ADOW29DufUfzVMC6rvNv9yRtGuTQ9tflv3GYrG6y1JO5rO5E?= =?us-ascii?Q?9ssppRvxAnue0WgwTp7eZCAB8ZEQ9vg0nLYjrDvK5R8RyycVqEWqJuWCxyLE?= =?us-ascii?Q?7VzNod60lDkMF4ioE3V7YMrq4QOTtJVCBVoApB1QkanssZjPiDJCVFX1/kmV?= =?us-ascii?Q?dIhfQdoH+KQZevyuJ1LYje/PU0O14Fr/FrDV8qgE5sQ5QsUXQPdGgMQoy/Os?= =?us-ascii?Q?eK6s/JOtVM49dF7JHC5APIdgig9ZgJn4M8zNfa0UeqFwH8s5b92JIvXTmPsd?= =?us-ascii?Q?p1LmM/do3cnbnlSPlPmvsXMQiXbQqtwumkkie84QhRXkQ4VBkz3CXPaM4Rlz?= =?us-ascii?Q?wJBxF60yEwKcohoEgrv/eWXx40zOpdgBAVMALg2/NBJ++xJhx11hzQqMor+7?= =?us-ascii?Q?ISVua00rcWBFGkhFb/UISjregoiXhYtOBcivGGVXluDV3UNDJgBTd/LoDg2k?= =?us-ascii?Q?xTYsTR8ge0ouEZozYxhl957LnFc1aW8SIhKrwrdUGWGAlXPwUMdaIXxghy/P?= =?us-ascii?Q?vOEVmpZiE8/VjhCuWX5EbmXPfFjKTZf2Kub3VYfWXGfqq2a07ZU+zVw5s8j/?= =?us-ascii?Q?usbMI15yZKl6wOuNBuYx2KYOLEGiha/Vx6TzGmkhbHU/ShEAh+7RNxoFu/Aq?= =?us-ascii?Q?pbILWX9mRhuptbZJro1Ps+w2dtr4H7dvBl6tt3idzFcebvgHmlZNz39lRgvX?= =?us-ascii?Q?rZoKC840nGHFDlFb/e/E7vfNtVKLTpjQ+lj7HYR/OxbsFvJDaq3wJhOQ+p9B?= =?us-ascii?Q?UDqdVbr1wm2YAK7FFMdmC7uevdzLSA4kjTD5JshlnwMhDJHSD2H10JilR+4P?= =?us-ascii?Q?JhTnk7DrQWwlZFnfakCXyVZcADOx00MdiNXuFJfjq8TBWK6JonYLqi+6E/5O?= =?us-ascii?Q?D+IMRMreb5gC4CiERNoXIU4WlSCVo3CgYcYuDDI8eOuqwbbtWC88C4m7NgRV?= =?us-ascii?Q?atSdFUl8Gk4EPuQyoCqqU0WU8tVoFfhvMgZBAGxvB2tyZ07GxmgUkbzl+FXj?= =?us-ascii?Q?UnpE/Gs4NpWS4xyslhalPR7tj/BN1BKRI3YrjuXjGnwrhwCFKRh6QNhhxyaD?= =?us-ascii?Q?XIhmsTLPcQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd652111-8b01-4294-80b1-08de8fec2523 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2026 12:42:33.4845 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uvUNKg49JLdk3Pbpun4081RrO940HbU7cXlMb6wOdWrMu2OnLdPXTCVscb/NB57ohQLvXnheqyiMDlTD/b9VFQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR12MB999078 On Wed, Apr 01, 2026 at 02:08:27PM +0200, Vincent Guittot wrote: > On Wed, 1 Apr 2026 at 13:57, Dietmar Eggemann wrote: > > > > On 31.03.26 11:04, Andrea Righi wrote: > > > Hi Dietmar, > > > > > > On Tue, Mar 31, 2026 at 12:30:55AM +0200, Dietmar Eggemann wrote: > > >> Hi Andrea, > > >> > > >> On 26.03.26 16:02, Andrea Righi wrote: > > > > [...] > > > > >> So does (2) with NO_SIS_UTIL performs worse than (1) with your smt > > >> related add-ons in sic()? > > > > > > Thanks for running these experiments and sharing the data, this is very > > > useful! > > > > > > I did a quick test on Vera using the NVBLAS benchmark, comparing NO > > > ASYM_CPUCAPACITY with and without SIS_UTIL, but the difference seems to be > > > within error range. I'll also run DCPerf MediaWiki with all the different > > > > I'm not familiar with the NVBLAS benchmark. Does it drive your system > > into 'sd->shared->nr_idle_scan = 0' state? It's something internally unfortunately... it's just running a single CPU-intensive task for each SMT core (in practice half of the CPUs tasks). I don't think we're hitting sd->shared->nr_idle_scan == 0 in this case. > > > > We just have to understand where this benefit of using sic() instead of > > sis() is coming from. I'm doubtful that this is the best_cpu thing after > > if (!choose_idle_cpu(cpu, p)) in sic()'s for_each_cpu_wrap(cpu, cpus, > > target) loop given that the CPU capacity diffs are so small. > > > > > configurations to see if I get similar results. > > > > > > More in general, I agree that for small capacity differences (e.g., within > > > ~5%) the benefits of using ASYM_CPUCAPACITY is questionable. And I'm also > > > fine to go back to the idea of grouping together CPUS within the 5% > > > capacity window, if we think it's a safer approach (results in your case > > > are quite evident, and BTW, that means we also shouldn't have > > > ASYM_CPU_CAPACITY on Grace, so in theory the 5% threshold should also > > > improve performance on Grace, that doesn't have SMT). > > > > There shouldn't be so many machines with these binning-introduced small > > CPU capacity diffs out there? In fact, I only know about your Grace > > (!smt) and Vera (smt) machines. > > In any case it's always better to add the support than enabling asym_packing > > > > > > That said, I still think there's value in adding SMT awareness to > > > select_idle_capacity(). Even if we decide to avoid ASYM_CPUCAPACITY for > > > small capacity deltas, we should ensure that the behavior remains > > > reasonable if both features are enabled, for any reason. Right now, there > > > are cases where the current behavior leads to significant performance > > > degradation (~2x), so having a mechanism to prevent clearly suboptimal task > > > placement still seems worthwhile. Essentially, what I'm saying is that one > > > thing doesn't exclude the other. > > > > IMHO, in case we would know where this improvement is coming from using > > sic() instead of default sis() (which already as smt support) then > > maybe, it's a lot of extra code at the end ... And mobile big.LITTLE > > (with larger CPU capacity diffs) doesn't have smt. > > The last proposal based on prateek proposal in sic() doesn't seems that large Exactly, I was referring just to that patch, which would solve the big part of the performance issue. We can ignore the ILB part for now. Thanks, -Andrea