From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CEC528468E for ; Tue, 31 Mar 2026 05:38:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=198.175.65.10 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774935497; cv=fail; b=pWXvzvOJIHLHtrBpDIwFXg508DYow00CPbCLdrs/zmBw2/rL6lhBtg9f4cUZX9BWgapDyQhNBOLykYOLtipCxSbgt+oOjwnHJnNf7V8kzLe+9D2ayurc1YMTFmF3I9J4onSbgW70Atz/GwUKgLZuqPjd3vazk936nYMKn5Lc+pg= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774935497; c=relaxed/simple; bh=QXuyj/wQn8uVXZUVii8BbYGdLX7AVS7l5JTltg91QTw=; h=Message-ID:Date:Subject:To:CC:References:From:In-Reply-To: Content-Type:MIME-Version; b=kEDdF8MYR3wTV5tuOYzdAw4aXNI8svZwDeWR05bbmuAe3v9DXz+nBthm18lL6NwQdsSat2XMh5CZhiT9M69p73c0Hpjq0oKOsWmGcb2WJlfzPA5kgYmCIt/0frmXHkBAQxPm1ImQ3V3qi3n0QePbsPA6t/8cmOnI3kAnO/ZhejY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bJdAhnLP; arc=fail smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bJdAhnLP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774935497; x=1806471497; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=QXuyj/wQn8uVXZUVii8BbYGdLX7AVS7l5JTltg91QTw=; b=bJdAhnLPpFLTKOMr6QOfG+6HuoILvgKxZZCIzwCGmaVq66uuZTCY6qD1 fHMV2D8As+v3U262jcVnWOLBoWnZ8galGLknTacyAnj76drJsqf+05o+M oYY6fP3Faj4HE0/ocPnm8EvBtRPixF6m5XAc+QegjrASBmC+lSOBFE3gS /iwsEl/O1+ITvh6/FCRCKGnrjGzqownhMSPed+FdfA5JCZN0jsCcs19OL emw1z8uFBF6AoinHmoPrUdqKWbQuKiH3KZW48O9zz7S2DY+7gw9EwRJ1S nLDXXl5BRmpzm3VY36Ys9wl7YwOVXHz9WntKwR9FAwnqgbSyJIwWAwEZl g==; X-CSE-ConnectionGUID: V7cB+kA3RqmUgtFUwUCvhg== X-CSE-MsgGUID: MQv/MDVjS7K6hyozqGsuLw== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93319847" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93319847" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 22:38:16 -0700 X-CSE-ConnectionGUID: Gk/VDnwOTlKXmMRi74HZkA== X-CSE-MsgGUID: QQxH67h+R/iAO8nnC4f6iA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="223378245" Received: from orsmsx903.amr.corp.intel.com ([10.22.229.25]) by fmviesa008.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 22:37:49 -0700 Received: from ORSMSX901.amr.corp.intel.com (10.22.229.23) by ORSMSX903.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Mon, 30 Mar 2026 22:37:49 -0700 Received: from ORSEDG902.ED.cps.intel.com (10.7.248.12) by ORSMSX901.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37 via Frontend Transport; Mon, 30 Mar 2026 22:37:49 -0700 Received: from SN4PR2101CU001.outbound.protection.outlook.com (40.93.195.63) by edgegateway.intel.com (134.134.137.112) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Mon, 30 Mar 2026 22:37:48 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CYT73fk3dMMo7h6GFGIYUO3yxQ3kFH8sRnms4Ph0EN9uTjveVJlKzdUm3n5QxUuWEoo5/2RDfycOPosDdCab1ZdbADzcAupbK3UsthXvjjZsXhSQvjS3//hiiSCgMkooiMM2yj6orzLF7Z7sBIPFXG5RzMhOAv+30LOJIVqyNTXgdvqn94nqUaXEurb44R8f9/m4EJHr5sduKJYeMSR2+jVLqDAD2d1iUorjjwHvml4SzipBuPdotDFMihcY1ZL2kH23a0JtKfku6eEsuzFxFG4OjlbHZ2gH/e3UYDGi0Up7F2VadzJe++r/PyIPS1m3jwesoYS3yxP9B61k6hwlbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8JGL5Jw/HTQDpdx92DabPK1VtpMyIkUmNlYI9ptH0L8=; b=oT+FbzEh/vRzg/r6XUTMCbYCgKPxh3eRy8twOf8yarJJAkHBRC2lJz+osPYyEsF5NPx4Jkj34npPNHUzZSbRzaOUsaB7q2nhhT/d6a+S6RcP+0U6weVaPxOLgaWqbzo7sX94Uj0bZwVkJdKu+CxCo3wLDdLgEQqAEL2F86rPjUnSJ+2gtc7sF0kuGl0+htDA5f7OfE8LI80ykXLgnHXG/O/DqXfVfh+7U9hPpFhFSx92C3PggyDnSGtQ8AeFkqj9FaXTuiQMoS6cF3Y0Ot2P6/91QD1dUbzfOa6BNB5B6MkK+kiyWTkhSR9x3AufTy6qUzbMngWOWvEQsK/exnPwkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) by PH0PR11MB7660.namprd11.prod.outlook.com (2603:10b6:510:26f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.15; Tue, 31 Mar 2026 05:37:46 +0000 Received: from DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765]) by DM4PR11MB6020.namprd11.prod.outlook.com ([fe80::3058:1480:e4ac:5765%6]) with mapi id 15.20.9769.014; Tue, 31 Mar 2026 05:37:46 +0000 Message-ID: <138c3f9d-309f-41e6-aa72-a3f6bd713bf0@intel.com> Date: Tue, 31 Mar 2026 13:37:39 +0800 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 4/4] sched/rt: Split cpupri_vec->cpumask to per NUMA node to reduce contention To: Peter Zijlstra , Tim Chen CC: Pan Deng , , , , References: <20260320124003.GU3738786@noisy.programming.kicks-ass.net> <63a095f02428700a7ff2623b8ea81e524a406834.camel@linux.intel.com> <20260324120008.GB3738010@noisy.programming.kicks-ass.net> Content-Language: en-US From: "Chen, Yu C" In-Reply-To: <20260324120008.GB3738010@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: TPYP295CA0021.TWNP295.PROD.OUTLOOK.COM (2603:1096:7d0:a::11) To DM4PR11MB6020.namprd11.prod.outlook.com (2603:10b6:8:61::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR11MB6020:EE_|PH0PR11MB7660:EE_ X-MS-Office365-Filtering-Correlation-Id: 8735d35a-de09-4152-717e-08de8ee7a369 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: 0EQrpOCjW47rY3FHpbLm4XMhaoNZhcHjkWgQZasiPf98FIjaLs3vg3GdJQ4/684zy0AQcM3qRrQ2TP84zn08WV5HHImGGilpEncfoYSaE+Ce3w6KnH+7SACLhkTKY5V27y/MDFrZWK2WimP4r9sBPX4nbRYyWUHy9GoZ3QSE2Rgto1xuzWKmq3Lgyukh7ugiHLPp1ESBpV+Qz2RgFgiSNGUwrpv/4ixN+05NAwByFTkKPnmWVraQUsmrcDKMokmUmJP49dQizzv+moIT2CfrebKTI6vSVupx44be+SkWrs/6IWkkZ/gPiWsmm428zedjzD6iGyFmapa4dt8JGcasivGtwc3GXjZSNf2GNy6laxd5qcR9F6QMdgDoHsJP+VQOhCI/5rZxP6Qp2qdnSw05HT6gPUtpKrA42qPLeQ29PVggogS3cjTmjlADYZ7DG91v0WO26yNpzFAl+f5L2MkdHQk37Ze2dStcU810ixioXDi+q7fikUJs0pm+KVPLyrdAEOYkPpSW883k7MREVF7Tivr6ERcOVlIzpLp+bsVOrJ4+8ddhYv0SCU+SzeCtlX0zCi+6zQ4mjHxPewz/AXc6KRs1+tJVr47yp20gZmeGs7cd6yMPuXnJwvTZNKa2Pbb4op2FJYGstDApebdbVK42aSly5QkAz1Qvy0r6Z+lsZdSCAU+ZagUTYbrzs6N361n8hv8RVFx3XCCEd4TDQjHuHmgo/QbTnLHoiTynrU0bZr4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR11MB6020.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?STlnbEFSVnFTVXVRMnBJbnBOM1E5ZUhCYjFKeS9nRDhYTXVMTlB0dzhDTDJz?= =?utf-8?B?cnhHaGdLUzhRVFNqRWZGRkFYUUxHd05xSG1iTjE4d2t3b214eFIrU2FhK2g0?= =?utf-8?B?RGdrRExWcUxHcFhHQUppekZUVmNPRktJWGZub3lHcVJMVDRIWnlnSHI3aFZy?= =?utf-8?B?TTRveS9wTFVyVEJ2bXRBcjZUamcvZFZ2TFZoQTdUSFpRMUpueXlubzExZHB3?= =?utf-8?B?dDZjbm9NblpXbWl6N0FVWHFOamRwZG9RdmZzOEFnemR6Yk43bU1Wb01yNVdI?= =?utf-8?B?N0F3cUtTUHJldUFIWHJlbCs4VEdwTk54VzJtRXlhMjVuMWlJTlk5M2x1RXNy?= =?utf-8?B?Q2hRZTdrRUZWcEtzbXF3TGliQ2t5Vm5wbTBWaktSeVphdXVicTE1WFRQT1hu?= =?utf-8?B?L1ROWnc5UmNLa3hDWnVyYzNHaWVQaEtnUURxUzY4SldTYTMvQkhFdFN5Ty9N?= =?utf-8?B?VUdIQVEyUnQvZTIrUGl3eWNtM3h5U3pkWGMzcHZ0QUN5ZnFJKzdXcXVoeGFy?= =?utf-8?B?eHFkcENKRnhDaDV6bTRtcXAxT1JhWXlMOTIzaWpYdXNYZDNNYThmOGxtbWdZ?= =?utf-8?B?S1VPTno5SXJ6anBDWmdGRVJ1VE9NZGdpZjV3TkZiQ1dtZkZhUURpbVJ1SXZZ?= =?utf-8?B?ZStGb3ZOdjljRDF0SDRvc0E5OTg5T2pJeE5KcENNWUpDR1RMZHNhSWxRNWpu?= =?utf-8?B?SFpaQ29YcE9FUzFFVC9qQjZhZjlwQTRveVNhK0hOcEFKcGFlSGJ1SzczMjIy?= =?utf-8?B?T1RXYUFjaTNWcGRsblYyMW1nQ3JseW1PdWpkME1Zc003aFRKb3p6OVRlT0Vi?= =?utf-8?B?dVJGVEkyREYyc2FKSWdYMDFMcE1zTTg4TUZITE1rK2NhY0FtZnNxKzRCRis0?= =?utf-8?B?MXpxYW9zYTc5MnhIa3lQU2tCVUVPN1ovM0VNM0o1Rld5ajlxcGFOYkdWMjZJ?= =?utf-8?B?VXBPTUUrVnV1QXRzeUxWb0hBcHBncVFSWTJZc2hCU2N2djNvWmdsQWxJSmJD?= =?utf-8?B?WFV1T3hqK2c3cElaUVhiNjBvYm9tZnEzVDYwYTdoVFlzL1FjWEorUnQ5WXdG?= =?utf-8?B?QUFmeHV1Y1B2cXo4RCtYMHgrclZyUUVabEJpL05PRmhOSCtYNDhVa2VqaHNn?= =?utf-8?B?WU83SDJpdWpVRGoxZDZ4cU01VzRSQWkzaUhFMHVlVGR4NHRlWExLWDJsNWZm?= =?utf-8?B?akhLMUJRMEUyUzJZVTViNndkRWNxb1FHSE1OMDZNcGM4SkNkYnJkQjBqNndK?= =?utf-8?B?T3JleWZVOWJVcHZTSEZQRWM2SkpCM2Q3K011WjJjcUdlaWltWDR4Wm44NDk5?= =?utf-8?B?cVgvK09rQlBYdEF5N2puR0lGUVNveTI4QWN0MzYwVmxLeG9GakVhaWVaaGo3?= =?utf-8?B?UXVnWU04cXNuMUJ2dUhTUDlDWFlreTFaUEVHMWQ2ZC9VYnNZVGw0SHVySTFU?= =?utf-8?B?Uy8weXNLWDExNGEveTNMdXZEdTFJZzNGQXVlajdhS1VqTzd3R05MaEZZUTIx?= =?utf-8?B?ZHc5TFNtd0pyVFNvcGhPa0JVMXBvUnpmLzFRR0tnanVFcmJ4RWZ6V1Z0V0p2?= =?utf-8?B?aWpXNk9QVE1EamQ2MFJRckV1bVc0eGdIdERTTnZpcWt4b3ZHaE1vYnEwQVRK?= =?utf-8?B?eGFJRjFnaHNWYVFNaDVBQmRiNk5ZTVhVVGhWK0dJa0RvK0dvMmFXb2RoQWxp?= =?utf-8?B?cDZPQXErTmUvcEZZZmZVVzBCTGNQZGFUTktWbzM3VXJGQmZxR0F0cUtFR21y?= =?utf-8?B?WHdJcWNXdExNbHNOMVVyQzRGVDRpQkRHZ0ZoZjR4WGRPU08zNGZVWGo5VDNz?= =?utf-8?B?MjFjTjVBeEYwZXdEUitGYzZ6VVlmSG9CN0srL2l6MWE1SHRrampvUU1GMWcv?= =?utf-8?B?ekxOc1VWZGdCaXBKYi9LdGVmMFg4RmRsZE5WZjltUklmY0hYc1Z2aDdqZzlv?= =?utf-8?B?eGxDUU0zTFN2aE1GcW1MMmxGVGgrcXFLOHZHQjVSQmZTaFF6RjRRUjVFRFoy?= =?utf-8?B?dmJ1MldLaTRUYzZpM1BkOHk2SFZXWnpCM1JMdFJpWU5ILzJ1ejJoVzdUaTND?= =?utf-8?B?RlpPTThKcUExbTRyZGtpNkd4UWVwR3d5TmV1MnoyRHVlNTQ2dUVrMW1LZ0do?= =?utf-8?B?KzFxNkd2U0pvQjBLc0dVdFllbmhDYTBraFpFL2t4ZlF5WHY1V0xQRzc3dDRl?= =?utf-8?B?VXg3Umx6aWFvSjYzeU5MZ2V0NWIxT0ZCd3NUREhlcVVEeXp2akZmK1FmVlNq?= =?utf-8?B?ZVBJQTh6a3Iyc3RsNEFnUDdlU1NxeDhURFdpRkp2K1ZhVm54TzhVYkduNFI4?= =?utf-8?B?SFJqMExNcEt4ZzJHaGVkVGJCRGRWTFMxRGJWOHVXSTkyaEtEdUNPUT09?= X-Exchange-RoutingPolicyChecked: Hu1OxeBia2z2rsvmWPw1SnyBe2lDTkz/e/zSGS6+/Qu6sx1YpMizsKaiMVXscmzt/Hr+Te3UCCevX3UpGAM7+yOp9GqWWr3p5OOh2pi2GpHp3kF4wsSgPDDWRImKya1YoSOgeip2j3m4/Rh0URMp/LLco8J11Mmuf9HcVRRnZbF5RbZKtW5fK0y5DT7mccCjZ241hoKUeBFxrp+/bJU0tyUFyYVXYO5APz6NXtELLltYRgcmlYPHNcuoTm1NL4YUWUJkpFozscqgI5JPKm1RLQbF+eGy9txnY/g7yXt6kabf5KokwFnDTJFnZEnqLBaH7EAp4Dn8CMTY8ddkw++cww== X-MS-Exchange-CrossTenant-Network-Message-Id: 8735d35a-de09-4152-717e-08de8ee7a369 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6020.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Mar 2026 05:37:46.4539 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GR3PwdAkU1UCPPFN1K7JQh0Anr6S5NeRmwEPJZ67Su0nyIuoqLjpWwyb8KABo0Zdykdv+c/Bb5Xw/f7vP14STA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7660 X-OriginatorOrg: intel.com On 3/24/2026 8:00 PM, Peter Zijlstra wrote: > On Mon, Mar 23, 2026 at 11:45:01AM -0700, Tim Chen wrote: >> On Fri, 2026-03-20 at 13:40 +0100, Peter Zijlstra wrote: >>> On Mon, Jul 21, 2025 at 02:10:26PM +0800, Pan Deng wrote: >>> >>>> This change splits `cpupri_vec->cpumask` into per-NUMA-node data to >>>> mitigate false sharing. >>> >>> So I really do think we need something here. We're running into the >>> whole cpumask contention thing on a semi regular basis. >>> [ ... ] > + > +unsigned int sbm_find_next_bit(struct sbm *sbm, int start) > +{ > + struct sbm_leaf *leaf = (void *)sbm; > + struct sbm_root *root = (void *)sbm; > + int nr = start >> arch_sbm_shift; > + int bit = start & arch_sbm_mask; > + unsigned long tmp, mask = (~0UL) << bit; > + if (sbm->type == st_root) { > + for (; nr < arch_sbm_leafs; nr++, mask = ~0UL) { > + leaf = root->leafs[nr]; > + tmp = leaf->bitmap & mask; > + if (!tmp) > + continue; I suppose this should be if (tmp) break; otherwise [ 40.071616] watchdog: BUG: soft lockup - CPU#0 stuck for 30s! [swapper/0:0] [ 40.071616] Modules linked in: [ 40.071616] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.0.0-rc5-sbm-+ #16 PREEMPT(full) [ 40.071616] RIP: 0010:sbm_find_next_bit+0x2a/0xa0 > + } > + } else { > + tmp = leaf->bitmap & mask; > + } > + if (!tmp) > + return -1; > + return (nr << arch_sbm_shift) | __ffs(tmp); > +} update of the test: With above change, I did a simple hackbench test on a system with multiple LLCs within 1 node, so the benefit is significant(+12%~+30%) when system is under-loaded, while some regression when overloaded(-10%)(need to figure out) thanks, Chenyu