From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010038.outbound.protection.outlook.com [52.101.193.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FDD13DCDAE for ; Tue, 19 May 2026 07:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.38 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779176853; cv=fail; b=ed7bSN/o4gzryn9ww5XKvq56o+ixZDNcUYTU5mHVr+aQYtk2WNfMuWFVZFpTLbHkCmCkGwIaODHaJWjom9CCrQdis4DsE2xYwx91QLkVD8SSsQoF7w42++DYntAOEYY3fqpbF7RwkMRdzQ/3vr9QZO6jNqfFMd9rcNT/TYlO56g= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779176853; c=relaxed/simple; bh=0xg+daBi/sd0wGWFXxdEve9NwNu7QBGu3eTvUs72CxA=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=WNhxNc/HgQcUAdIBkKVgIFmDZk4naL+Klc98Hfh5RZ4FXkJBiPgwvgXT9u9v44QErjJiSyGDkUdcLb/pdgWszey5A0uhizZQyCrkMhTm/q3zBIrFkYA1wgoT1SR2o9Xie0yXMHP5siEzSBi24ag6Ff7BQtCppzOK+mzG4hpbJ8E= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=pVDue690; arc=fail smtp.client-ip=52.101.193.38 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="pVDue690" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=n3doNCT8WouZR4vDHiSslgaSoRfJuk+hscVyoDz70wArqIQQ8pC35lCucwTh9o2SI2xLM5TWi2dGFmV1gBE18fyyea7esnHqmd9HTV9vvozFqJPHqrYgmVfj7e11VkN8q3JermLnBp1LgDOZXxiT7VsNlzo7LdJSfTk3LoL9EpYXLGeTM+wG3LDYavuUqVqcL8HwGkvdBjiNab42JYP/WmPt3FUGMMun9za0j9RXMeHXS5uu063Wn+HZf/OqXNEpvu0QgaI8Qv9IpnWKxtWpsrE4qEkkiwricJgOYSSOyZInU6BVlYVi7FAbZgF9IrMZJZDOYTgMI0IFPvLv6/Ahvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=40mAXbe28zbm9hCtehteZMUPV5/8GXG0VqLozBveQKo=; b=NYnRlbqz5V8NXIIyepqDxvV9Y8ZIWqYbk0Srxq4A0QTFIM8/t31aZyoLuCPLcFhHoPWOR4qcWQSH0pkYpfHn/aVRGacOIkIj57RyB5yEX9ONHhNk3S9DyLdqlcyatyq3Cp/uCUc/5U5Gzw+6NXKA8OggkWKCanJwlcyVzWIpwxKlBGFNlJKV4vaxnYOO7hHlbDWhVZFcBBsBH4ktv0PAgMXe7j9WarPauveE174RUn73xlZFG8RuJOLPorpuGUKZAW62q8YrxwXlqi5f0euLuf1IuiaZouhKJKVrpjvxWwe8rcc2Uk5ZDj0BoVQ99fr42VRR0pxeSCyYlosc0+AbWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=nvidia.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=40mAXbe28zbm9hCtehteZMUPV5/8GXG0VqLozBveQKo=; b=pVDue690Vq7qi8hW7N91VTDzPcgax+D+Tp8tTB7MQed6r4EKhUp3NIz2OUPX6FpUw7yu6Nd/nGFFFlMUiOMkKtlXPH9NyciurJ6CTVRd5388sAYW/jBlUXt2E2kYlWtjjr8mmUs5TTI9+BRBC0Pq8l8V0/+qACaxeKZjzUwk4Vc= Received: from SN7PR04CA0120.namprd04.prod.outlook.com (2603:10b6:806:122::35) by IA1PR12MB6652.namprd12.prod.outlook.com (2603:10b6:208:38a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.21; Tue, 19 May 2026 07:47:28 +0000 Received: from SA2PEPF0000150B.namprd04.prod.outlook.com (2603:10b6:806:122:cafe::85) by SN7PR04CA0120.outlook.office365.com (2603:10b6:806:122::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.48.14 via Frontend Transport; Tue, 19 May 2026 07:47:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by SA2PEPF0000150B.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.11 via Frontend Transport; Tue, 19 May 2026 07:47:28 +0000 Received: from satlexmb10.amd.com (10.181.42.219) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 19 May 2026 02:47:27 -0500 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb10.amd.com (10.181.42.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 19 May 2026 02:47:27 -0500 Received: from [10.136.36.105] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Tue, 19 May 2026 02:47:21 -0500 Message-ID: <55196e3b-ba1e-42c9-b80b-5c91306df452@amd.com> Date: Tue, 19 May 2026 13:17:20 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity To: Andrea Righi CC: Peter Zijlstra , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , "Mel Gorman" , Valentin Schneider , "Christian Loehle" , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , , , References: <20260516055850.1345932-1-arighi@nvidia.com> <20260518205859.GY3126523@noisy.programming.kicks-ass.net> <81583f79-facf-473e-b756-31af724245bc@amd.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF0000150B:EE_|IA1PR12MB6652:EE_ X-MS-Office365-Filtering-Correlation-Id: 1d019e75-59bd-4692-d544-08deb57adff2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|36860700016|1800799024|82310400026|11063799003|56012099003|22082099003|18002099003|4143699003; X-Microsoft-Antispam-Message-Info: qnae261EPjjFS80X9B2HKtdMU7vuTw49m42V+EE7Mm/u5e5uWLwWlkuC/orNy0IUP9uEokqUUu/6hB2qPqfd+uJVyDzY/XPOJ7pNqu24I79ZEVcpXF+o7FF3Bd8BNapzYnnQBfGjK0+Ne8EcbE+zFC9+4sySMVqPHnONZjktqIfApso4h/+oKCr/mAvNqze5FQ9/vVCBhMxs1w+6Rwk3eYyAZ7a+bgUDms2/Bidje2TpzNGvJKodXuqyKc8Bd1LE0wPn6T31DxAEJuqADpfKGEY3McFu4kgXzfwHbt9w/S9lcBUbdmFYoGWSRqAnopMGuJ94Cj6YC/pXw3nGrwXHjOC9qBrtjasK6i0aAho7RBDrOhi0A5mvw+EbU+8BH0UGtqwhmcho5k4FXLCc3Yt92GD0zQU+lbD/jHM0VJGAvVVAUckrf6RM+h+2/fzUYW7MUDNCald7kHSbcTI6v/n3LuHO7ZOCI+WI3q5HtFHohiGr1RPX8xlss9IMumt+D1+Iz90dE9XykX66/Y6bocSW6ap+rV0qNUUmSwfR9hhfJkwhtycTzMJsgEw25RLOCIN2EqNRvoesMmmSOVCMZqZ/Bd88+tr2wF5Ki5VW/g413E436TKuI7geTKuIY5iruKZB3SxunGlJD50y1OXJDhlJTqql3dC4cGd0s3feQDX1gcMZh08cvuWjkO12jem8F76uIAN1FHqHrKha71o8Oj7ZXTymsIBmkyQr+ZoL6azB9X8= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(376014)(36860700016)(1800799024)(82310400026)(11063799003)(56012099003)(22082099003)(18002099003)(4143699003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: e+zOluQ8fxSpSC3Ywx3WPqTnbpOfMSyw4g3NqKlIQ39RDjmMc+xNmpV4Yu0zNwYjQMabUHVpVS68s6u0dacdFYF3f+Dh1DVaqdNu0Mk/zMdmQjj3g32Ou4Lw41iXwZlxQ25JuQVTecKZl0EgXot+a1My+jnuOnCqupgvmG/nSq1uDBJ6AQ1z9C4c9RnIvCRNhXhq1lI55jy+mEaqEhfX+nI9wJww4i+wbIyMBjmcjhjw2IA7xp2h7NPaEqkbQNqK8SAO4vGfX7Xef/tHVFu2Z9Ay7OnCJzrDpJgmM432Rrp9wGzLb+hWxq3I6zU6IHjLuMQefW7SA1sC5RdoTKoMOVTmf8o+hMKjX3hcya16QVmWP4qEBtFhturtR6LhWvCdGuYzq2Uo4UAD4J+S3/5lal/CLxLdjcrQIb+zaLofWcItkNQnUptsT1AYIDN7xW+z X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2026 07:47:28.0331 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1d019e75-59bd-4692-d544-08deb57adff2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF0000150B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6652 Hello Andrea, Thank you for taking a look at the diff! On 5/19/2026 12:13 PM, Andrea Righi wrote: > Hi Prateek, > > On Tue, May 19, 2026 at 11:22:32AM +0530, K Prateek Nayak wrote: >> Hello Peter, Andrea, >> >> On 5/19/2026 2:28 AM, Peter Zijlstra wrote: >>> @@@ -2775,20 -3049,16 +3107,15 @@@ build_sched_domains(const struct cpumas >>> if (!sd) >>> continue; >>> >>> + if (has_asym) >>> - asym_claimed = claim_asym_sched_domain_shared(&d, i); >>> ++ claim_asym_sched_domain_shared(&d, i); >>> + >>> /* First, find the topmost SD_SHARE_LLC domain */ >>> while (sd->parent && (sd->parent->flags & SD_SHARE_LLC)) >>> sd = sd->parent; >>> >>> if (sd->flags & SD_SHARE_LLC) { >>> - /* >>> - * Initialize the sd->shared for SD_SHARE_LLC unless >>> - * the asym path above already claimed it. >>> - */ >>> - if (!asym_claimed) >>> - init_sched_domain_shared(&d, sd); >>> - int sd_id = cpumask_first(sched_domain_span(sd)); >>> - >>> - sd->shared = *per_cpu_ptr(d.sds, sd_id); >>> - atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); >>> - atomic_inc(&sd->shared->ref); >>> ++ init_sched_domain_shared(&d, sd); >> >> This will run into a small problem with "nr_idle_scan" if >> cpumask_first(sched_domain_span(sd)) is the same for both sd_asym and >> sd_llc. > > Ah, good catch! When cpumask_first(asym_span) == cpumask_first(llc_span) > (big.LITTLE typical case), both sd_asym->shared and sd_llc->shared would alias > to d->sds[0]. > >> >> Load balancer at different domains will populate "nr_idle_scan" with >> different values and they alias to same ->shared if one isn't >> degenerated and I believe there is at least one way to hit the WARN_ON() >> from cpu_attach_domain() if the SD_ASYM_CPUCAPACITY_FULL comes before >> the last SD_SHARE_LLC domain and the latter is degenerated. >> >> How about this: >> >> (On top of queue:sched/core; Lightly tested on !ASYM_CPUCAPACITY system) >> >> diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h >> index fe09d3268bc9..1d2c98dca211 100644 >> --- a/include/linux/sched/topology.h >> +++ b/include/linux/sched/topology.h >> @@ -67,7 +67,15 @@ struct sched_domain_shared { >> atomic_t ref; >> atomic_t nr_busy_cpus; >> int has_idle_cores; >> - int nr_idle_scan; >> + union { >> + int nr_idle_scan; >> + /* >> + * Used during allocation to claim the >> + * sched_domain_shared object at >> + * multiple levels. > > I think between build and the first LB tick, readers of nr_idle_scan may observe > leftover SD_* flags in nr_idle_scan. This shouldn't be a problem and should > self-heal soon, but maybe it's worth a comment? Something like: > > * Note: between build and the first periodic LB tick, which > * rewrites the union via update_idle_cpu_scan(), readers of > * nr_idle_scan may observe the transient SD_* flag value as > * the scan bound. The flag bits are small positive integers, > * so the effect is just a slightly relaxed scan bound for one > * window and self-heals on the first tick. Ack! We start with 0 today which isn't representative of the system state either and depend on the eventual correctness to fix the value after a hotplug / cpuset. I can fold in the note and resend it as a formal patch. Peter, would you prefer a formal patch or would you like to do this (or something similar) as a part of the conflict resolution itself? >> + BUG_ON(!sd->shared); > > Unreachable in practice, but should we have a WARN_ON_ONCE() + > bail/early-return? In this way we'd fall back to using LLC's shared for > sd_balance_shared, which seems nicer than a BUG_ON(). Ack! We can just use the last CPU's "sds" if we don't end up finding a free one as a backup. I just had the BUG_ON() to easily spot my VM crashing ;-) -- Thanks and Regards, Prateek