From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SJ2PR03CU001.outbound.protection.outlook.com (mail-westusazon11012037.outbound.protection.outlook.com [52.101.43.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D518A351C14 for ; Tue, 19 May 2026 05:52:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.43.37 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779169969; cv=fail; b=XiqBJRjQz9L/sJ4bmpd7bncvljcQ15YIX4IGJg8WirUuetKr7F7Yl9GvQxFmnAs6GOKYC2w91cjOHPI460b49CAWfOdLK+3R8aYm6/+z+B3u118F6EwX9WUw4elzhzVO3gfIGwcmHZUuApZRdgByReEZgs5FFexMPruiYLk1xOE= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779169969; c=relaxed/simple; bh=VMbUvP1SuGqCjkXbxHQuhtGS+zWSB6BwkSpIQzKGD0E=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=hZzOlhBG0tbGyc0WpZldUGQmSrlLxAMBvFMS3bjb1nAFL6bI7UowgCs6B/O4h48+D31fIdWK0QKows1XDhLCDdghOM0jAoh5ubhgEjmerMWz3+FPPEiMo/07sftwknOStqXz5gwWrRLFRl8wXVDpm+NSupCUFd4Hcdzj9xTsd/0= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=MWJZPvZP; arc=fail smtp.client-ip=52.101.43.37 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="MWJZPvZP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BW/4l8UFNKIHw9dCvhc5D69QHINui5r/AyGzsRUmYjwhiXhJroJE0wpZWY5AdDSR8iMsfwL2ffIPWv+LthxYeoFzHGDtr8zFtyyJrP4+X13ue2WaG045nCUjKa/MWURGSeUvduMTmtbyUm8TLHxbF9NWqJt7H0vheNdZFxFaVZaeLOXX9SOVLbAK54cmsIyVg1W2Esq8iKLO3KBWSCWtLxO6DvG9MEjrAhyGIRdZ7MfRr5TaYCauUc6neWcpqhYGWH8/p+w1GlgkAc1hP9BEnOGvHfNx3mDRQNv0ymuvKUGQBIkd7Qe8Er0B9Bce2M4wtxUylGVkXCjhT5KdO14f5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/con46yF381jVIvGdETEpGgqbd/fsB+t1HHaOXsmbp0=; b=oB87UeqOZfT3Oixd7+ssV782kby/CEXH7HBh8LxTD2NRai7GZVH2NFAt/3ZOL2la6mfNd7wvLdO/BIPEr1dlLbZqMyiOKMnujLmFdwLYywpagD/RIAklUyw0oZXZOCMpQk0HifGZBdO6R8jvA/N9Uro5OtZm0OqpssWK2pIbkEGAJ9s8XMmS5P3eOSH9v4SKtpRZM9o5D7y94IINMIgeS30/nERo/hKCxSuDmnwnH5d9yTP2TCWmVTYumkBVjj4d/ks7WLX/7jeDlok2OnJCK9mbxyz1ZkmRYrwRm6Kj9IE1eKF+DVAgwGgZ+NJY/VsYpTcP4pHP/tTP9jZ+fY81yA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=infradead.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/con46yF381jVIvGdETEpGgqbd/fsB+t1HHaOXsmbp0=; b=MWJZPvZP61LCI0IlMslK6vtkf2gJkpoBqWCRO3w2lqigkZtLKPh9151rsZ4qLphheLL4JASK3oU5pDYTKckV0O5RuXgyN5/EW6WNBBf2ERhv+VIzUIY0bu2q4Z8rUWbDqwQXNKUGa2WUufUn1dlhZWe2y0POgVFmJmyfFfaLuks= Received: from BY5PR16CA0007.namprd16.prod.outlook.com (2603:10b6:a03:1a0::20) by DS0PR12MB7969.namprd12.prod.outlook.com (2603:10b6:8:146::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.24; Tue, 19 May 2026 05:52:41 +0000 Received: from BY1PEPF0001AE16.namprd04.prod.outlook.com (2603:10b6:a03:1a0:cafe::9d) by BY5PR16CA0007.outlook.office365.com (2603:10b6:a03:1a0::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.25.24 via Frontend Transport; Tue, 19 May 2026 05:52:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by BY1PEPF0001AE16.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.48.11 via Frontend Transport; Tue, 19 May 2026 05:52:40 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.41; Tue, 19 May 2026 00:52:40 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Tue, 19 May 2026 00:52:39 -0500 Received: from [10.136.36.105] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Tue, 19 May 2026 00:52:34 -0500 Message-ID: <81583f79-facf-473e-b756-31af724245bc@amd.com> Date: Tue, 19 May 2026 11:22:32 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity To: Peter Zijlstra , Andrea Righi CC: Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , , , References: <20260516055850.1345932-1-arighi@nvidia.com> <20260518205859.GY3126523@noisy.programming.kicks-ass.net> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260518205859.GY3126523@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Received-SPF: None (SATLEXMB03.amd.com: kprateek.nayak@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY1PEPF0001AE16:EE_|DS0PR12MB7969:EE_ X-MS-Office365-Filtering-Correlation-Id: b24b73bb-2ab6-4297-2191-08deb56ad6e3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|36860700016|376014|7416014|22082099003|18002099003|56012099003|4143699003|11063799003; X-Microsoft-Antispam-Message-Info: 0SXFFXNHufdY0KUvVpCQT+C3QOxQ/4ITnX5ppYYv0gSZlq1k0jTgME5+Q4IHkyBFjZahXE0D2IbJB21qfco2lR/1gk3+pGdgmBvU0qYYDyWaxyZ5q0dkTVkbPiwus3E+oeAMNHiMfvObe9qjBIyLYH6Z0XiYmGK36Va0Io8zgWX9X25Pwj/5P9R4swl2/gJNM4Gj9L7eLPSuas+RRGYjyWOlHrS9852sT4YpTahWoDCKndXhdEqP0m2FzZw2hJ9a8Sm6zMFC0W6CKzyNipRnxTreDE1SdrIjjBib+e6aIVk6uV83Bb1B16ssOUTJUSX4IEmdh5Jwwk0hL3NVdk+B53+gVf0eAa2DIto3QQw330el+7PhUksZVBRzqU6ZhRd/5+U4j8/fCNtA3YPpUsOr9tVsfa8HECNkE3eLQegRosJPkUutYWTRTn9o9XBZV+2xq+DzwpwsHnwSe/yqxwI2PPMPSynimbbnU5eg0F9sfYYnFttXrF3uk9X6PxPwMYb3LPRhPvTxB+43u+aZVgd9kwPUnAA8zbZZHgPDF905M/sPT1DVJbXvNJAj5q1ULuoA6o4QPuyXo7FwaY9dUTH1Zpeo6kNGv518PxI8Lhhw8kAN3jxLsY88P41ujnNVuuLcTy4DzMU4wVjPDERxD/1bsbUxEPgimXZTtXsc8ElkrvyZ69GTH86uGPF4dA4itm+keLmDPIUYpnU1BulvHjgo6U6Lkt8q4tzPezaZaqN/l98= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(36860700016)(376014)(7416014)(22082099003)(18002099003)(56012099003)(4143699003)(11063799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cjITB0H+e4e2qzw8n6K/dF0r740vkAa/dXL55URrHscTtfKDpeSXQ/eog/O0XdwfN/ElC/WaujHECB9m6u7ZJrDQTgr9t0bEnP35ZlwNb59AiPtsp93yvy/yxMoAfx/zI6pWUOU8oRZ9mqgBxOzn6tlyfAgr2Qoc8MYhpKkr9p2c5fH2mWCt3SGVEksYnpWOCg4DNy6kSZQ9sTkh+cT63kXG2JLjy1m0Xy8UT2hocukWos0ONzWHZtW9+VhSZ9WJ5d7Ugl91Nw8F3Z2c38HOwMRq9tSvyzYVElELBwmWX0BQ9/Jpj9Z/znFsSp6/mhbTyuMKKSOrzbwRd7mBrs1GYpx7Vyz+uRSemhlcoS5KoX4Arle4s5ror0SimiWVn1BUTr9huVdI+plvuggLrV5WNMsyEB7PkIs3LcAv9mivyKmjGiSy5y5J6i7oPIAtyhTO X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2026 05:52:40.8208 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b24b73bb-2ab6-4297-2191-08deb56ad6e3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BY1PEPF0001AE16.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7969 Hello Peter, Andrea, On 5/19/2026 2:28 AM, Peter Zijlstra wrote: > @@@ -2775,20 -3049,16 +3107,15 @@@ build_sched_domains(const struct cpumas > if (!sd) > continue; > > + if (has_asym) > - asym_claimed = claim_asym_sched_domain_shared(&d, i); > ++ claim_asym_sched_domain_shared(&d, i); > + > /* First, find the topmost SD_SHARE_LLC domain */ > while (sd->parent && (sd->parent->flags & SD_SHARE_LLC)) > sd = sd->parent; > > if (sd->flags & SD_SHARE_LLC) { > - /* > - * Initialize the sd->shared for SD_SHARE_LLC unless > - * the asym path above already claimed it. > - */ > - if (!asym_claimed) > - init_sched_domain_shared(&d, sd); > - int sd_id = cpumask_first(sched_domain_span(sd)); > - > - sd->shared = *per_cpu_ptr(d.sds, sd_id); > - atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); > - atomic_inc(&sd->shared->ref); > ++ init_sched_domain_shared(&d, sd); This will run into a small problem with "nr_idle_scan" if cpumask_first(sched_domain_span(sd)) is the same for both sd_asym and sd_llc. Load balancer at different domains will populate "nr_idle_scan" with different values and they alias to same ->shared if one isn't degenerated and I believe there is at least one way to hit the WARN_ON() from cpu_attach_domain() if the SD_ASYM_CPUCAPACITY_FULL comes before the last SD_SHARE_LLC domain and the latter is degenerated. How about this: (On top of queue:sched/core; Lightly tested on !ASYM_CPUCAPACITY system) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index fe09d3268bc9..1d2c98dca211 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -67,7 +67,15 @@ struct sched_domain_shared { atomic_t ref; atomic_t nr_busy_cpus; int has_idle_cores; - int nr_idle_scan; + union { + int nr_idle_scan; + /* + * Used during allocation to claim the + * sched_domain_shared object at + * multiple levels. + */ + int alloc_flags; + }; #ifdef CONFIG_SCHED_CACHE unsigned long util_avg; unsigned long capacity; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index dbfd9657f897..9ebd14652e9d 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -623,6 +623,12 @@ static void free_sched_groups(struct sched_group *sg, int free_sgc) } while (sg != first); } +static void free_sched_domain_shared(struct sched_domain_shared *sds) +{ + if (sds && atomic_dec_and_test(&sds->ref)) + kfree(sds); +} + static void destroy_sched_domain(struct sched_domain *sd) { /* @@ -631,9 +637,7 @@ static void destroy_sched_domain(struct sched_domain *sd) * dropping group/capacity references, freeing where none remain. */ free_sched_groups(sd->groups, 1); - - if (sd->shared && atomic_dec_and_test(&sd->shared->ref)) - kfree(sd->shared); + free_sched_domain_shared(sd->shared); #ifdef CONFIG_SCHED_CACHE /* only the bottom sd has llc_counts array */ @@ -755,7 +759,14 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu) /* Pick reference to parent->shared. */ if (parent->shared) { - WARN_ON_ONCE(tmp->shared); + /* + * It is safe to free a sd->shared that + * has not been published yet. If a + * sd->shared was published, the refcount + * will end up being non-zero and it will + * not be freed here. + */ + free_sched_domain_shared(tmp->shared); tmp->shared = parent->shared; parent->shared = NULL; } @@ -2916,11 +2927,40 @@ static void adjust_numa_imbalance(struct sched_domain *sd_llc) } } -static void init_sched_domain_shared(struct s_data *d, struct sched_domain *sd) +static void +init_sched_domain_shared(struct s_data *d, struct sched_domain *sd, int flags) { - int sd_id = cpumask_first(sched_domain_span(sd)); + int cpu; + + /* + * Multiple domains can try to claim a shared object like + * SD_ASYM_CPUCAPACITY and SD_SHARE_LLC which can alias to + * same cpumask_first(sched_domain_span(sd)) CPU and can + * cause "nr_idle_scan" to be populated incorrectly during + * load balncing. + * + * Find the first CPU in sched_domain_span(sd) with an + * unclaimed domain (!alloc_flags) or where the alloc_flag + * matches the requested flag (SD_* flag) + */ + for_each_cpu(cpu, sched_domain_span(sd)) { + struct sched_domain_shared *sds = *per_cpu_ptr(d->sds, cpu); + + /* + * If the domain only has single CPU, allow temporary overlap + * in allocation since the domains will be degenerated anyways. + */ + if (!sds->alloc_flags || + sd->span_weight == 1 || + sds->alloc_flags == flags) { + sds->alloc_flags = flags; + sd->shared = sds; + break; + } + } + + BUG_ON(!sd->shared); - sd->shared = *per_cpu_ptr(d->sds, sd_id); /* * nr_busy_cpus is consumed only by the NOHZ kick path via * sd_balance_shared; on the asym-capacity path it is initialized but @@ -2960,7 +3000,7 @@ static bool claim_asym_sched_domain_shared(struct s_data *d, int cpu) if (!sd_asym || (sd_asym->flags & SD_NUMA)) return false; - init_sched_domain_shared(d, sd_asym); + init_sched_domain_shared(d, sd_asym, SD_ASYM_CPUCAPACITY); return true; } @@ -3115,7 +3155,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att sd = sd->parent; if (sd->flags & SD_SHARE_LLC) { - init_sched_domain_shared(&d, sd); + init_sched_domain_shared(&d, sd, SD_SHARE_LLC); /* * In presence of higher domains, adjust the -- Thanks and Regards, Prateek