From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BN8PR05CU002.outbound.protection.outlook.com (mail-eastus2azon11011001.outbound.protection.outlook.com [52.101.57.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1655F3BED24 for ; Thu, 2 Apr 2026 11:06:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.57.1 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775127972; cv=fail; b=HS3WA7wHWVP3xYqvnXeOo2PH2x3xgJ+Bvfcr1CAm8lwuQ2Lrr8XHEGZmv4liEGKVO44rQswUUktB+Xh7L7ruiq51/CnrSvsen8+TJqkOhybhWhPuuUVfFmnJne1bcsmgdXzuxD06IySBgHcA9nOywBDTV46NnVlCnhXhZJtMU3Y= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775127972; c=relaxed/simple; bh=3l1d27rqbJUlZNxukxPxBSlqp8x7Zd+XxNMdlYNKvOw=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=K6CWpGdAdKwyJPRb+sIOtC2OqISuypd6TFUYZ7pFngHJP9BnIpF8PLTMgHhsQ+kUqlBBGTzOcne9X4/MB3CqAy/0Vert39E/ujrAYvBdF3QFAoQI1HZjJz0mSReSZnTejOaLxylu1Mj0v5kwSUiXqPNyQGMWx50bdyRmQ9IclyM= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=oMJ+Qogw; arc=fail smtp.client-ip=52.101.57.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="oMJ+Qogw" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sf+vXOc2fimgvVSChEi7gO+ZypJ+OZq+S8lfBwKOux2IagJfIZDx3nD1ZcOScuGv5NghjDnXDwVJA6iqzEaJVpo6KlxiSKW5yqNiN2ZFsdFfbGoHdxhSlQ75YmKHbr4+F+LQBtLmolSQ3UGtnlGBMJMAQO9dwQCOQ9mgZREA9DK+8/ifHWrBSr4eG/1OpJiC+Ea9buTGT8J6cnJUoS3b2PoEvp4CTzfp782KgST7XqiC9BQ4IK+2EyzR//+wI/+tcfrdidbFYX5Q1+1/Dvcs1+g/AfgS7YvAkUl9yNE/tJcYt+B9+i1vpWRCZE+0Y6+hJVlDFdYxdfBBXGtTUC80Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ii31NXgTXuV/d6Tdm5zxWSI+nGBiIPl/d4iyBmVc5eo=; b=Ohv9NmMHIaloBDA6TaOLb/+soRykjlzxJtDdZVX+yCy9cZm91hayYuzBOhW4FkG7LY6vS7+YkWy5zzP9vR1lW42Ae64Vi8FVB1hmEQx5SOUMMgDerefajUR+h2QlX5AAta9HVs4JqdT0cldD6JsOcIaM47tlp/i2kPsVeHlh0d+k/8ILwwEGMdoekX2lbLty8HJB7C5E2ctg+FOHutzeXtz/ljk4gjrlXEQ7XmE1xidVK/yuq3edgo0x3cJvsLN8GVFFsyl+Fvp73awHyA2O7/gsXJyolhIUDHqou6yes8iD/l1TtggbrDpWXoTNo472ngiaTLU0YYpVizYTLUeAAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=infradead.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ii31NXgTXuV/d6Tdm5zxWSI+nGBiIPl/d4iyBmVc5eo=; b=oMJ+QogwAwd/4CkZWnNi3D9z+CPwGsX0EU7j75P3ESZzKxxlwWCYtvBdb/BxxlGFQ8jH7H+tSAlAk5d75VPTzKiZpfJUWUqdKXP6N5iLkPGF634KYEvkEn95i3Is9J/DYVHJckW1epofZuasD1XPxyjKIRU+07y6uIzHSNC28xY= Received: from SJ0PR13CA0168.namprd13.prod.outlook.com (2603:10b6:a03:2c7::23) by LV2PR12MB999097.namprd12.prod.outlook.com (2603:10b6:408:353::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.18; Thu, 2 Apr 2026 11:06:06 +0000 Received: from SJ1PEPF00001CE9.namprd03.prod.outlook.com (2603:10b6:a03:2c7:cafe::80) by SJ0PR13CA0168.outlook.office365.com (2603:10b6:a03:2c7::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9769.17 via Frontend Transport; Thu, 2 Apr 2026 11:06:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by SJ1PEPF00001CE9.mail.protection.outlook.com (10.167.242.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9745.21 via Frontend Transport; Thu, 2 Apr 2026 11:06:06 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Thu, 2 Apr 2026 06:06:05 -0500 Received: from [10.136.42.52] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Thu, 2 Apr 2026 06:06:03 -0500 Message-ID: <93d7eb33-c3a5-4498-bc26-57806b73d9e0@amd.com> Date: Thu, 2 Apr 2026 16:36:02 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 4/4] sched/rt: Split cpupri_vec->cpumask to per NUMA node to reduce contention To: Peter Zijlstra CC: "Chen, Yu C" , Tim Chen , Pan Deng , , , References: <20260320124003.GU3738786@noisy.programming.kicks-ass.net> <63a095f02428700a7ff2623b8ea81e524a406834.camel@linux.intel.com> <20260324120008.GB3738010@noisy.programming.kicks-ass.net> <138c3f9d-309f-41e6-aa72-a3f6bd713bf0@intel.com> <22072ef8-5aec-49ac-9cc4-8a80bec14261@amd.com> <64649c85-29ab-4f70-a0c4-3c83cbdae2fc@intel.com> <20260402105530.GA3738786@noisy.programming.kicks-ass.net> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260402105530.GA3738786@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF00001CE9:EE_|LV2PR12MB999097:EE_ X-MS-Office365-Filtering-Correlation-Id: 63b93e24-d477-44f8-357e-08de90a7d64c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|1800799024|13003099007|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: B0W+pJEJhiatQSu1AH1YZqHbZN7PMrP/cPG53krUueENorNqg4VwAyStZDGha6bzPMkDIFdb9W6UZYMgtWaV4zKzIaqgPB4ZQhRnxtxloA2V4kw1z1oJI6IyUvR1n5Cf8oz4F1Xft+v/Ub260OWt6wQzMgyc0RzrLQ/6o3sIy0Cd4k7MzG0wE7KNLxtqHfOH5Jd+t0sd/BV1pxx0AWk7zCTHi6QFSh9Lw3YohwLpaBcHownuiF5p7L1vtZaDbOp74I6UFW6IjPCKUrIs1Wx1y8wbVtM8FMgMzZx7SETPdZUJBLzttsTemRfjmezGACi/iGRujQYzLileiS9on1QxvkkTboUINM9J5U414XORsh6Azjyqhh2iDjjWoww/DIRXGjbPxPlhA6K/tlplZNvXDCKBVzGloqDvkE8R6/mn2xeY00p/C78BjvHbhcKbpZEr0AcNo41HjC1qhggXmzJGjMk+bdJJpVYv/NYCg0KcTQSwPYBv9ZrSdo5gZ75jcnwrWU8gdePtSd1fgjNPyrMLC6p8Uw+XKMfDcvqickyiBipoJxpiuWzFnDqwgwbG24su8cTjyAFsWQpiZ6ktXQO61XxrZY2lmN0Gv28iIGEiwQYvaMZRLWTywFBnO+++p4Sg6d+G2ZxFlQjjTJP5Jsu6XkvgECrJqAvI2CvEn4nfSc6+655c42fv3c5ezggJx076HbrG+d9VgirWURM8JnOBbSHFoRngvQZ9gH1dvDSayw2VE4eSo6YR5CNU8nD56FNLn7KIVgyfIfAiSHgYINNGrQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(1800799024)(13003099007)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: oCppT70yB4Uq1AfNO1ZskGu+/6/V0+hgXT9XhASu9I8Ws9gEmSCTw0Qp5NOHPSLNdnwYOBojM34H6pBEI0pHbxKUsX5D2BsdnxdSOoVs/25rbk+NJiTIxkkswDSOOXsO5G+eFmlLsoEDMsaw+QxK/eYs1KtgzVf+QCHYQ3xqncHlXPShLNh3pSPCBe2zC3aK6jT///mJ0itlFyWZvQGTkRIC8erWoPR0n25qVgOTabXoSdkc3rtKzhHXV6O5LeWE39lcq+YCp9gakPyC5jEmDS2ZfWtLX+pebP/FPsUjLN07QY6rwCUSFZtEs05g7ZeYzFVbOMfKduy2jTNDzz2aP+2xeGePhHPqkuuKDdz3KiIWsiSmy/tVbIv4rBK0rAE5il426+5vagdz3jvlaI1uMq+gxEmLyEKi2KJy12haBFPIHhMmfyeydE20YI6gOck8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Apr 2026 11:06:06.1761 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 63b93e24-d477-44f8-357e-08de90a7d64c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF00001CE9.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB999097 Hello Peter, On 4/2/2026 4:25 PM, Peter Zijlstra wrote: > On Thu, Apr 02, 2026 at 10:11:11AM +0530, K Prateek Nayak wrote: > >> It is still not super clear to me how the logic deals with more than >> 128CPUs in a DIE domain because that'll need more than the u64 but >> sbm_find_next_bit() simply does: >> >> tmp = leaf->bitmap & mask; /* All are u64 */ >> >> expecting just the u64 bitmap to represent all the CPUs in the leaf. >> >> If we have, say 256 CPUs per DIE, we get shift(7) and arch_sbm_mask >> as 7f (127) which allows a leaf to more than 64 CPUs but we are >> using the "u64 bitmap" directly and not: >> >> find_next_bit(bitmap, arch_sbm_mask) >> >> Am I missing something here? > > Nope. That logic just isn't there, that was left as an exercise to the > reader :-) Ack! Let me go fiddle with that. > > For AMD in particular it would be good to have one leaf per CCD, but > since CCD are not enumerated in your topology (they really should be), I > didn't do that. We got the extended topology leaf 0x80000026 since 4th Generation EPYC and we (well Thomas) added the parser support in v6.10 [1] so we can discover the CCD boundary using that now ;-) https://lore.kernel.org/all/20240314050432.1710-1-kprateek.nayak@amd.com/ > > Now, I seem to remember we had this discussion in the past some time, > and you had some hacks available. That, I believe, was for the NPS boundaries that we don't expose in NPS1 but CCX should be good enough. > > Anyway, the whole premise was to have one leaf/cacheline per cache, such > that high frequency atomic ops set/clear bit, don't bounce the line > around. > > I took the nohz bitmap, because it was relatively simple and is known to > suffer from contention under certain workloads. Ack! It would be better to tie it to the TOPO_TILE_DOMAIN then which maps to the "CCX" on AMD and is the LLC boundary. CCD is just a cluster of CCX that is nearby - mostly the dense core offerings enumerate 2CCX per CCD. -- Thanks and Regards, Prateek