From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010026.outbound.protection.outlook.com [40.93.198.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 912B23195E0 for ; Wed, 25 Feb 2026 08:51:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.26 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772009511; cv=fail; b=kLRao62WM+h0zNCj5gKX9tM3YDO/s2UEfIwyLqPbVR48RnRmkaAgbEXMRlwnqh3l67pblUnvCTSTIDcsF5umdXANHuNGOZvHssY6Qfqh9Zc8kn0QSEX+RBMMu3dLGQFme8yrQGCoRfo8tBS+F5G6o4Y9OSv8sDyVy9vh85DCm+w= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772009511; c=relaxed/simple; bh=tqlnuXxvpKHu89YqKHmG+jrZAvHHMWNX86yskC7nF8E=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=RPSO2uU2kYOP/8cTWqfQaZ9ZDxri5T5O4HsOjFOCAJtjh71teqNY4YaJDbnNwIkVN5Y3yyeQeGeh/B3qPsrJESmIbRaso0/Soaxq19WI1T5ve6LUOBEmxtakviai96UqzPvHzeF0cDSCzANCyTPBoruoH4Itv8bYJUHmgalenO0= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=vehXUuZp; arc=fail smtp.client-ip=40.93.198.26 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="vehXUuZp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xnekPiIq2eASpKYrQVmoedYT4CxS+e+pdohn18cOkfZVQVO4VWXkSnCXj3GckY3PXa6Pw5kQ50j5zDihd9L78uZnbrqQyKBLdDyOcnhj9956KHvl0B2fRxuB52nA9TtetRTp49c4SVhdyXRRLCvLaszw5Vf/Zqo8OGSZnjVYMesmSCdrB+eZDvU5vxMS+/dNpK+E/ptPP5DRKUuBFC295kM8aO/tlBuY9JY7uwPxR3yZEvDp9B7l5w/w9zcslAGP/1FbBL2O8WlQW7JyM9Z72CyDQ0fHs8RjRA2MYeUEmcwwn8K+8joZ1kQzfFvMC+cfq7y1qckLQEEMv48sFuQRBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wFiwJLQqM7zuQNi6DNmTJ1MbxHx1f58fRjvJd0vHUa8=; b=BAaU3Hu4wrniXxM196eH/2gxHVWo2SO/GeXxgHzpeBHaHRQImhTdsewlyw/vz2RNoUye2BC88SYlfSUgEuyZNgs0qiok1rX36yY8S4FQ3yV3SMTGjh0AWGOq4Weic8/hUodwWlN+ASubQLtmFZ9DOL3Kq+PYGdFs2lli/fMicHdQ1xSb8UdewMEhoNK2EMRwEPfZjZoSZYqNevgv7IqPtpV4mZ5PLWeNqFN/UoZZ0cpd76amX8EuCFRJUhKEtYEyaB5lFkvfw+pD7xCHaTQzd83LDautdjW/JJuBT6fP3UjUT+bxUamv7RDKzmFOmPM+Q8ep1t3Sq4U7OOJ8eAT97A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linutronix.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wFiwJLQqM7zuQNi6DNmTJ1MbxHx1f58fRjvJd0vHUa8=; b=vehXUuZpI1qjUd7gxO6iO+ebRcSuJ03HsOOUarXWRIRrFIC/wokOJhMsbARMpOSjqkledtFd93FNUEpq7VwkO9LQg/Xg6sdsy6txYgJQsiy/QIfqoaoBAHngHHScY195bjpOQgv5gl1UVm07z+YlKX/gABj+CONx+/izZa3Q4P4= Received: from CH5P222CA0007.NAMP222.PROD.OUTLOOK.COM (2603:10b6:610:1ee::19) by DM4PR12MB7573.namprd12.prod.outlook.com (2603:10b6:8:10f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9632.22; Wed, 25 Feb 2026 08:51:42 +0000 Received: from CH3PEPF00000010.namprd04.prod.outlook.com (2603:10b6:610:1ee:cafe::fc) by CH5P222CA0007.outlook.office365.com (2603:10b6:610:1ee::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9632.23 via Frontend Transport; Wed, 25 Feb 2026 08:51:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by CH3PEPF00000010.mail.protection.outlook.com (10.167.244.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9632.12 via Frontend Transport; Wed, 25 Feb 2026 08:51:42 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 25 Feb 2026 02:51:42 -0600 Received: from [10.85.46.153] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Wed, 25 Feb 2026 02:51:39 -0600 Message-ID: <99a81417-5dbf-4c29-8dac-a9ed17ee24d6@amd.com> Date: Wed, 25 Feb 2026 14:21:33 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH] futex: Dynamically allocate futex_queues depending on nr_node_ids To: Sebastian Andrzej Siewior CC: Thomas Gleixner , Ingo Molnar , , Peter Zijlstra , Darren Hart , Davidlohr Bueso , =?UTF-8?Q?Andr=C3=A9_Almeida?= References: <20260128101358.20954-1-kprateek.nayak@amd.com> <20260224111342.-9hafXl5@linutronix.de> <20260225073939.HLbwiSs5@linutronix.de> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260225073939.HLbwiSs5@linutronix.de> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PEPF00000010:EE_|DM4PR12MB7573:EE_ X-MS-Office365-Filtering-Correlation-Id: dad21c8f-a233-4f05-0a97-08de744b1908 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|36860700013|13003099007; X-Microsoft-Antispam-Message-Info: =?utf-8?B?akkvMzlEdmg4azBiYWZkT1A0VUFCRjNIZ0l3WmpaOTJTY2RLTzRlZ2ZmK3BR?= =?utf-8?B?Ni8rSHptTWs5QmFRbG9EOW9EcmszQlN6bHBoS3VYUWtOY3RRR0hsa3cxM1Ra?= =?utf-8?B?bWhUdUJSRmR2dk5LbHBCSWNBclhYR2lsLzl0MVQvT2ZvNXF6V1dibzVudUc5?= =?utf-8?B?Ny96N1Rzbm8vd0U5SmtXd0FKNU12NVIwM0pRRlJ5eFVQMitZdFQwYlNXMndi?= =?utf-8?B?WU1MRXdTaFU2aTNKNk9TN2phSVVrdVNWUVZjenJ2OW4wTjZSN3ZFeGdvUTRt?= =?utf-8?B?YUVEb3FKQ1pTNDczVUhnMU5BWVMxNFZJNzhrZkhkc3doTTV5Z2tCRlNEejdN?= =?utf-8?B?S1dPQWE0T3hXNEFONlBEUkp4T25oKzBjN2xaODZ4clFQWnVlM0lmRW5EbGdZ?= =?utf-8?B?dFVZMzg0Ujk1YjU4N1ZVTkxXMzd6VG5YNCs5R3BXaEhORS9wWk9ZZHR5RUEz?= =?utf-8?B?MHBHK3lhd1ZPNC8wUWd2NU5jNHQ4MzFrdkxjY1FRdXZLenFMYzVGWWY1Nktw?= =?utf-8?B?aHpSK01oSC8vVUZQSzF4OXBoeDE1bUhGdW9zL3RWNlNoR0V2NUJTeDcwZUdH?= =?utf-8?B?dVdsUG1KTzYraVJJQmppNTF1RUN2U1A2K1VoV2NyYlVzcDV1ZWZ3QTRqRUh3?= =?utf-8?B?aWtMTVhTT2plTGJhblFtdjE1MzdkYVBqd0dvYWgvNW9McEwrSThCODFCWEdk?= =?utf-8?B?NEI1SGh1WWdGNzdkTkVibys1N3ZRVmYzRytpSlU3enVyMXNUbVkrajVGb3Y2?= =?utf-8?B?eURtRUJzRDJxemtOdzBUYzJSM0FrYS9CUWJ1ejYyekM4K1FTUjQySnlrUUR4?= =?utf-8?B?L2dIM3V4TnRjeURhdzBManYxakRHeTczTlJUSmRPM09MeXFHMVBidUJVL2Mr?= =?utf-8?B?ZlhOYVB6NC9kU0hKWUFjZUtFd2x3cCtMNXF3dnFXbm0xczhuNUpqaDRqODdn?= =?utf-8?B?L1RDMEhmdTd5dllRbHpnRU9Udm54MkFkMzVsNlpvTGFjMWo5NStDbU1DQm9R?= =?utf-8?B?UXMzbDgxTHkvdjhjSWtVRnJLZGZtSFlndkhaOXRJRy9lZXk3SHN3SnpOelN5?= =?utf-8?B?aXBIeHAwRCtVd1I4Z0RCNTA0OUNLUGo1My9yN284NUdIVzdZTlAzS0puSE5i?= =?utf-8?B?dlVHa0IwamIwTVRkMzBPVG9ZcFVjOENnRmd0emVXUVM5TWw3SE5LalROYXhU?= =?utf-8?B?Z3ZZcDByeDZtVXI1VTA5WjhianZxMUsrYjIvVmp4dkZ0NVpLUituQnd4TGhx?= =?utf-8?B?QlJ2b0NhcDM5NWF1NC9UbHViTlFIRHR0K25MS1ZoLzZqUnFBcjJLeG5mN2Ir?= =?utf-8?B?aGJFaGVPQUpJOUw2dzNXVnZoS1lBa0pMQU1UcjJRSzVuZlBFZlQvNWRVL2lE?= =?utf-8?B?bWFMdm0rTnJEaWtSa2xsL05IY3NvbVZaY3pNTUNkdTVjalcyQURKQi9ncldy?= =?utf-8?B?c2l5Q3orSFFBdUhibWJwbUhGTEkzTURLZGhPd25YWUZocm1YellnaG5lb1pD?= =?utf-8?B?a05MUVptU2xKUWE4Q3ZLdmpOZnZ6R2dJbkkwd2tDSFpZa0E4YW4yYmt3VWdy?= =?utf-8?B?NjVxb0dmS2dKOHhMTjljTktST281LzAzdHhMUllSZkpoOVRMN2NlRndFSW5K?= =?utf-8?B?WUNBQnBsOXlXQjBEWkRGZU5YWmVoTDBLUXZPbS9IaUFkTkI3bGpTWGNKbHJ4?= =?utf-8?B?UXFrMndRUmtjRDIrQkI2MUFrcTFDVzI4aVRFUkpWampMNDczclJnK2orZ0VX?= =?utf-8?B?TnFreUVnS213UkxJOHdhbzJGb3hSd0RPbjZQRWppS2tiNGlZV3pDOTFHK3hX?= =?utf-8?B?NEs2YzIwNmFXeFBmb2twQ0JNWmU3cUhrQTFtaUFoR0ltN3R4Zi9tLzJIL1I2?= =?utf-8?B?aXU1Smt4Ym5jVnM5emJ1UDAwM1FOOWVUdE5tZWZWMWozaytBaVQzTy9Lcmc4?= =?utf-8?B?bnVOYUJaUTBlK2hJdWFYWitmN2Mvb2JjNGs5RXo0U1V0SkQwd1ZJR2Q2U3Jl?= =?utf-8?B?Y3dLT3loMUZvbXo0cjZlczYvTkk0WWFxODdDUUh3WjRPcEMxRkc4dGdOQkd5?= =?utf-8?B?eWF1VURERVNwYzlQOUV1OVg5UFB0UE9HRi9oaEN1OFE3aDRmYXZMSzFOSFRx?= =?utf-8?B?bCtTU1VPNjNnNXlpSVF2aVd3MkF3Rm9WcERFK1lOQ0k3bHEzbTZybFIwMjU2?= =?utf-8?B?cEp1cjFVdjZNRnRyNnU1NzlBaFVqMFYxL1ZvMFptRkh6WlJIUnV6bDY4N0Ja?= =?utf-8?Q?JeY690S5/OIFIbzqi9KpGMWDAM+hINiDnhAqxe7wkM=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013)(13003099007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8/nCaoru77KmJyll8GZI+KsvsfiTHOwRtrA4uXG6qtpS5ijrkzyMxd4V2yMclciWmGfvForBw+PGgSF5AFn4PNB2m9QRfsFyv3jA6QIrL/2/UUHvLgwXx4E8/H2JhXvx4LVB5bBKXLBS+YJNrb4jjXK+3KqpeYos3j2bCJicXGy1W0/bg0rg1P00HCosT7U7FRJChqzYXsjneRh8KfyXv+sdsw/AfFnTyV+sGWaxH4tPXnKJnpw1wbzmple3rUa609ru7TXbmrgDxiQYauu/m+XbdO4DAK/daN2siUj8mixwXeQkbi8rZ5qgala7CKd6wz23TWCqoaGM5z8n0SCNDBX5yel8ezMwh6l32h/MR9SQT9BqwzELPP2+TDbfNnf2MZhkabHYuLnPOOAHinbZy2cpGanvnRjbZlOV1wkatQuy6lvrdbVMkxlIDUxMdUUA X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2026 08:51:42.3974 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dad21c8f-a233-4f05-0a97-08de744b1908 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH3PEPF00000010.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7573 On 2/25/2026 1:09 PM, Sebastian Andrzej Siewior wrote: > On 2026-02-25 09:06:08 [+0530], K Prateek Nayak wrote: >> Hello Sebastian, > Hi Prateek, > >> On 2/24/2026 4:43 PM, Sebastian Andrzej Siewior wrote: >>> My question was initially is 1024 for max-nodes something that people >>> really use. It was introduced as of >>> https://lore.kernel.org/all/alpine.DEB.2.00.1003101537330.30724@chino.kir.corp.google.com/ >>> >>> but it looks odd. It might just one or two machines which are left :) >> >> I have it on good faith that some EPYC user on distro kernels turn on >> "L3 as NUMA" option which currently results into 32 NUMA nodes on our >> largest configuration. >> >> Adding a little bit more margin for CXL nodes should make even >> CONFIG_NODES_SHIFT=6 pretty sane default for most real-word configs. >> I don't think we can go more than 10 or so CXL nodes considering the >> number of PCIe lanes unless there are more creative ways to attach >> tiered memory that appear as a NUMA node. >> >> I'm not sure if Intel has similar crazy combination but NODES_SHIFT=6 >> can accommodate (16 socket * SNC-3) + up to 16 CXL nodes so it should >> be fine for most distro users too? > > Okay. According to Kconfig, this is the default for X86_64. The 10 gets > set by MAXSMP. This option raises the NR_CPUS_DEFAULT to 8192. That > might the overkill. What would be a sane value for NR_CPUS_DEFAULT? I would have thought a quarter of that would be plenty but looking at the footnote in [1] that says "16 socket GNR system" and the fact that GNR can feature up to 256 threads per socket - that could theoretically put such systems at that NR_CPUS_DEFAULT limit - I don't know if it is practically possible. [1] https://lore.kernel.org/lkml/aYPjOgiO_XsFWnWu@hpe.com/ Still, I doubt such setup would practically cross more than 64 nodes. Why was this selected as the default for MAXSMP? It came from [2] but I'm not really able to understand why other than this line in Mike's response: "MAXSMP" represents what's really usable so we just set it to the max of range to test for scalability? Seems little impractical for real-world cases but on the flip side if we don't sit it, some bits might not get enough testing? [2] https://lore.kernel.org/lkml/20080326014137.934171000@polaris-admin.engr.sgi.com/ >I don't have anything that exceeds 3 digits but I also don't have anything > with more than 4 nodes ;) And mine tops out at 32 nodes ;-) -- Thanks and Regards, Prateek