From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010042.outbound.protection.outlook.com [52.101.61.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BF3632B12F for ; Tue, 19 May 2026 11:27:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.42 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779190050; cv=fail; b=HS0FLmAJSUNrILQRoeBzRNR/5/DxJEle/Kkyx/0tpPZLHpLV9hZZ2U03Kwz31T2fI2aguankaV6EDhgJl2z7jPU4edlgwlUtwmgzYRF8O/73ONVJnNwLPhogW8fhEP8TBG/LeHlJTMNG0OkZXZBnmhy2DmxFRCHv5Smj+QPk7bc= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779190050; c=relaxed/simple; bh=TZqNULumPeNs/+qyZbtFNwzVKM3PeTryxesxcCxbx9w=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=KTYsD8Skfzsg6s08Cn+/xwn/hK3W0NvbUHeJIjAXX4G41pBUHIVTT7JX12zWSGib7AmD8DdDHV111fOtpbudEKYw7KY9njv/ek/i0MR4lgB6PQu80Ml8bJrIEik1EEQJM+cRlH2BlxKpkEwuMhnTmGuFfYSAsrzWl05P2PG12J4= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=eAo3V78D; arc=fail smtp.client-ip=52.101.61.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="eAo3V78D" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SuF8ERSbn4NwznsuBdYVB6Em8/zEU/vKcqSt4+RLRS2rncqF1HsmRBo7EmuHhU2Ydjqwrzx2zxwqwH74wcPhn8+1mAXvfKArM3VUUjCqFL+GxgWsty9HNVAgQz7PJ4NHEn5+Wq6Pk2cjKONDaQrGqNIxDqCr6xudspxhmkbvHrNge0YgYdWhjiYBaz57L5OytrObVGPcB02bFR/Xr+WJBnm57R+OuLSG+DYiAHpJNBbijugwjhi0d0FEjL1MSse8MXU+fw+5pMn3JGIlvfpjKa2LatUUIYrk2YqijgY/1jtih0lgiq97oooFrXD9CliNoe2fRVnP59nZR8JnUGlApQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xbIHe9IL1zjwt3VGT+NouRT45zpe2HaRM+I7xj0IiuU=; b=YBfYJlVPbT9o7RPhUJo69+zOlsgYaQSHtQcWsQSPSM3LNB+WbPx0Eyvf+ekUvOMJyILnvmiCjqLGyJyo+SOtXaRVD856aRl5IJYZ9/jhBejI9yHnOgo/ZUA6PPpO3VKQV20lYGL8G4d7OXxmcpbNUt83cnynFF25FcXrLWx2QbX50UDiILR5b0pdopwjnO0PLB0OhHaIiRxr0JVYRGDnJDwscHSXBlwWoTiP0D3V26js8m48GYAtxBychH/oe6eqQizZKwnpQo6armMPUu1XnB/b2IcjAiV7hcfGuWqWtgW8RWkJRfFU3GipQJfmYuqV2jq4EeyqKjK85VaNKhdhkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=infradead.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xbIHe9IL1zjwt3VGT+NouRT45zpe2HaRM+I7xj0IiuU=; b=eAo3V78D0IFp8QBOAh/S/CEQPJklfZNe+C8MGK80lQQZGaaYKnTG5STJ2qjhbkTkR/A9NTq1vCYeCF/1RebJKu2GBKt7Q3Kp2MGPWyQsXPO29gmDhruShU5Rv2i5jKtOp0El61XWk+eRzVQsaT4Bjo41YmKgytFRJt9CtGz5GAI= Received: from PH7P221CA0084.NAMP221.PROD.OUTLOOK.COM (2603:10b6:510:328::35) by IA0PPFC855560D7.namprd12.prod.outlook.com (2603:10b6:20f:fc04::be4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.25; Tue, 19 May 2026 11:27:24 +0000 Received: from MW1PEPF0001615D.namprd21.prod.outlook.com (2603:10b6:510:328:cafe::d4) by PH7P221CA0084.outlook.office365.com (2603:10b6:510:328::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.48.14 via Frontend Transport; Tue, 19 May 2026 11:27:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by MW1PEPF0001615D.mail.protection.outlook.com (10.167.249.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.71.0 via Frontend Transport; Tue, 19 May 2026 11:27:23 +0000 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 19 May 2026 06:27:23 -0500 Received: from [10.252.91.158] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Tue, 19 May 2026 06:27:17 -0500 Message-ID: <0d7d6bb0-cdc4-433b-8810-bcba490aea60@amd.com> Date: Tue, 19 May 2026 16:57:11 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity To: Peter Zijlstra CC: Andrea Righi , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , , , References: <20260516055850.1345932-1-arighi@nvidia.com> <20260518205859.GY3126523@noisy.programming.kicks-ass.net> <81583f79-facf-473e-b756-31af724245bc@amd.com> <20260519084634.GA3126523@noisy.programming.kicks-ass.net> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260519084634.GA3126523@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW1PEPF0001615D:EE_|IA0PPFC855560D7:EE_ X-MS-Office365-Filtering-Correlation-Id: 9e9393ca-4994-4af1-c07c-08deb5999945 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|7416014|36860700016|22082099003|18002099003|56012099003|11063799006|4143699003; X-Microsoft-Antispam-Message-Info: yC4W+pggdktEAcoLT0FJUGz0MEdFpUhxJ3mB0+rdRma/rLT3GtiZeBL3mZ8FQfP/Zy/yDypjQgS8Q4lH04L8Q2BIwnewryidHptFbOTgClDCum0gvTCia888tuXIwlCSIbnN7XgxfBv+XZbCg2jswq9gfa/h6BnPBJOylLIsjOSKeTFvrZ7oOfads1+IjwV0ESvyaMi/DJGb74QH+ixxCyGyNFV44STK1kJMha+4T+OKIdzf/WPCywdASWb9odAn7dU3s9lkrEDC3Kzk/CwDmK+2/iXOULb3LeSOgkUFQKELmyeE8AmNvA8IWEHRkXtVEpyjY7/PCmC7YQCkxSvN3O2aD2aKGYCc3zUDaGMm+J0Kaexv9+IgUUUThPipBGbQj0BeGkWBdb6sqNZl+i/WcSS6VPE3+4cfEF8FnLgcxH2QuskPSJXpeWeRkOBLp88EIQurmlMFeP558uZxch+ABwP381PibrZCkSbLk0aVPes6WZI6RCdNZxKRfvWtVrOShysR9SukLzAYy1nNqa7h4GFntr8T4/2Df/jIttYMR0e2q7cro9ike6XfzpDdWZXQ4riqA3Aou40Q7KCwpZ7ENTXI+wLHP2NZCg99Rd/ZlDmBxBIOJk3qMel0VbFB17XlyoTPI3B87YCtVpv8SLzgT61jpJSGlBk+d24AIPKShQdTbvhnrb18K+gvE8O/Og43F9mdATrfDKaoCZ/Ln+6zYTTDhm0gknUsFBqsP9oReSc= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(7416014)(36860700016)(22082099003)(18002099003)(56012099003)(11063799006)(4143699003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: iTmcAqTry4GDbF1XQQ6bpCZP1YoRdBS/gLBN7bUmEsJ9YcA9Z0WcL7ShIL0sdUO66fBpaBRiub1aqRzFdS1K5xYaH6Hed7bF6xPaRdEQ6YtnZJ43boyje0k6EE/uB1kzm/idFe3K9U9aNL/ur1FgdXAP5KwdprHnKv0W1O4JBUlfQez5Zi4nw32G3E3OFji8PqS0fBS7Q4Z3hxDbrsHvvjcEo31oPXBGiauCFxenYvC7F/FIlEgolNvMY6fXGmmt+8wZGV8d+rSt6jk1X4BQSFyeLd7K2RgyTCVPCCeC33mFqtbPXyY8137csM6IbzZo7htAqnXfEpyUXDrXPU4QnxR3zXs12mIC+6t7aPtn1Lw28vHU0JXWCW5d42bVLI+6Nz8wKtussU8g8U6LK7NzV+Vhic2csvXh4iP7xgPZYMAnT/FzSWwNXnU33O8ASXit X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2026 11:27:23.7965 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9e9393ca-4994-4af1-c07c-08deb5999945 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MW1PEPF0001615D.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PPFC855560D7 Hello Peter, On 5/19/2026 2:16 PM, Peter Zijlstra wrote: > Shrikanth noted I had an old version of his SMT ifdeffery patches on, so > I need to rebuild that tree (and the merge) anyway. > > Do you want me to munge this in, or keep it on top as a fixie? Feel free to munge it but if you want to retain some context, here is the full patch with suggestions from Andrea incorporated in: (Based on top of 5162728eecc2 ("Merge branch 'sched/cache'")) --- >From b0a8ad4b225820c2369f45242517c1c06bac1826 Mon Sep 17 00:00:00 2001 From: K Prateek Nayak Date: Tue, 19 May 2026 05:14:23 +0000 Subject: [PATCH] sched/topology: Allow multiple domains to claim sched_domain_shared Recent optimizations of sd->shared assignment moved to allocating a single instance of per-CPU sched_domain_shared objects per s_data. Recent optimizations to select_idle_capacity() moved the sd->shared assignments to "sd_asym" domain when ASYM_CPUCAPACITY is detected but cache-aware scheduling mandates the presence of "sd_llc_shared" to compute and cache per-LLC statistics. Use an "alloc_flags" union in sched_domain_shared to claim a sched_domain_shared object per sched_domain. Allocation starts searching for an available / matching sched_domain_shared instance from the first CPU of sched_domain_span(sd) (sd can be sd_llc, or sd_asym). If the shared object is claimed by another domain, the instance corresponding to next CPU in the domain span is explored until a matching / available instance is found. In case of a single CPU in sched_domain_span(), the domain will be degenerated and a temporary overlap of ->shared objects across different domains is acceptable. "alloc_flags" forms a union with "nr_idle_scan" and the stale flags are left as is when the sd->shared is published. The expectation is for the first load balancing instance to correct the value just like the current behavior, except the initial value is no longer 0. Originally-by: Peter Zijlstra Tested-by: Andrea Righi Signed-off-by: K Prateek Nayak --- include/linux/sched/topology.h | 16 ++++++++- kernel/sched/topology.c | 63 +++++++++++++++++++++++++++++----- 2 files changed, 69 insertions(+), 10 deletions(-) diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h index fe09d3268bc9..b5d9d7c2b8ad 100644 --- a/include/linux/sched/topology.h +++ b/include/linux/sched/topology.h @@ -67,7 +67,21 @@ struct sched_domain_shared { atomic_t ref; atomic_t nr_busy_cpus; int has_idle_cores; - int nr_idle_scan; + union { + int nr_idle_scan; + /* + * Used during allocation to claim the sched_domain_shared + * object at multiple levels. + * + * Note: between build and the first periodic LB tick, which + * rewrites the union via update_idle_cpu_scan(), readers of + * nr_idle_scan may observe the transient SD_* flag value as + * the scan bound. The flag bits are small positive integers, + * so the effect is just a slightly relaxed scan bound for one + * window and self-heals on the first tick. + */ + int alloc_flags; + }; #ifdef CONFIG_SCHED_CACHE unsigned long util_avg; unsigned long capacity; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index dbfd9657f897..df2ceb54c970 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -623,6 +623,12 @@ static void free_sched_groups(struct sched_group *sg, int free_sgc) } while (sg != first); } +static void free_sched_domain_shared(struct sched_domain_shared *sds) +{ + if (sds && atomic_dec_and_test(&sds->ref)) + kfree(sds); +} + static void destroy_sched_domain(struct sched_domain *sd) { /* @@ -631,9 +637,7 @@ static void destroy_sched_domain(struct sched_domain *sd) * dropping group/capacity references, freeing where none remain. */ free_sched_groups(sd->groups, 1); - - if (sd->shared && atomic_dec_and_test(&sd->shared->ref)) - kfree(sd->shared); + free_sched_domain_shared(sd->shared); #ifdef CONFIG_SCHED_CACHE /* only the bottom sd has llc_counts array */ @@ -755,7 +759,14 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu) /* Pick reference to parent->shared. */ if (parent->shared) { - WARN_ON_ONCE(tmp->shared); + /* + * It is safe to free a sd->shared that + * has not been published yet. If a + * sd->shared was published, the refcount + * will end up being non-zero and it will + * not be freed here. + */ + free_sched_domain_shared(tmp->shared); tmp->shared = parent->shared; parent->shared = NULL; } @@ -2916,11 +2927,45 @@ static void adjust_numa_imbalance(struct sched_domain *sd_llc) } } -static void init_sched_domain_shared(struct s_data *d, struct sched_domain *sd) +static void +init_sched_domain_shared(struct s_data *d, struct sched_domain *sd, int flags) { - int sd_id = cpumask_first(sched_domain_span(sd)); + struct sched_domain_shared *sds = NULL; + int cpu; + + /* + * Multiple domains can try to claim a shared object like + * SD_ASYM_CPUCAPACITY and SD_SHARE_LLC which can alias to + * same cpumask_first(sched_domain_span(sd)) CPU and can + * cause "nr_idle_scan" to be populated incorrectly during + * load balancing. + * + * Find the first CPU in sched_domain_span(sd) with an + * unclaimed domain (!alloc_flags) or where the alloc_flag + * matches the requested flag (SD_* flag) + * + * If the domain only has single CPU, allow temporary overlap + * in allocation since the domains will be degenerated later. + */ + for_each_cpu(cpu, sched_domain_span(sd)) { + sds = *per_cpu_ptr(d->sds, cpu); + + if (!sds->alloc_flags || + sd->span_weight == 1 || + sds->alloc_flags == flags) { + sds->alloc_flags = flags; + sd->shared = sds; + break; + } + } + + /* + * Use the sd_shared corresponding to the last + * CPU in the span if none are avaialable. + */ + if (WARN_ON_ONCE(!sd->shared)) + sd->shared = sds; - sd->shared = *per_cpu_ptr(d->sds, sd_id); /* * nr_busy_cpus is consumed only by the NOHZ kick path via * sd_balance_shared; on the asym-capacity path it is initialized but @@ -2960,7 +3005,7 @@ static bool claim_asym_sched_domain_shared(struct s_data *d, int cpu) if (!sd_asym || (sd_asym->flags & SD_NUMA)) return false; - init_sched_domain_shared(d, sd_asym); + init_sched_domain_shared(d, sd_asym, SD_ASYM_CPUCAPACITY); return true; } @@ -3115,7 +3160,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att sd = sd->parent; if (sd->flags & SD_SHARE_LLC) { - init_sched_domain_shared(&d, sd); + init_sched_domain_shared(&d, sd, SD_SHARE_LLC); /* * In presence of higher domains, adjust the -- Thanks and Regards, Prateek