From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010069.outbound.protection.outlook.com [52.101.193.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1801C823DD for ; Fri, 27 Feb 2026 08:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.69 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772182072; cv=fail; b=hpZao7G+8Mt5r2b0ngOVDBwOXHN4aVyPDmGSzfwgh+Je2hijsMMRvJyrPtAvI/NML5l8SEEswU1TzSynfpITcuHko3u4GjukOnj6thLl5/zL7+XetT03EYc/bbS6r8SX4MPTSyo6MigTeVfxpM7UdL2Uk48JcD2hTAP+duEYPEQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772182072; c=relaxed/simple; bh=7TWDNsCui5DgGnxVYBRvxdE6Xlf5xNwz+pau9LINnMA=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=snK9y4AMsZO/5VA9a39yPAyomPLy9YWsiCzCpwaUQ4sy3WIuFH1wMZSJ5ij/bn/k0VcRHj7PFwTltvyzT77Q2FAK+FvSU0cOcLzh+X2TZOAa6L7+fc7fYySxt1RuqRA+nZoJDZdHttd1ac7Lkizv8ihkJKCmCdqoOWUFoJn4CtQ= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=YWGIDJFh; arc=fail smtp.client-ip=52.101.193.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="YWGIDJFh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wF5DfqPkwxN052K7zkmJ37UDZukymevBumy333kN10mOQSS5k7+7TT3kwEWJVmJjXxbhwepJGEntMvh3YuRRsEd3D18z4VUY3uRvUinriIa2dJKsPvwZB+w3AucmXrd6xZFPJ83Uhtyz119zrt01NGtuToKigOpTK7Bu5zFvOxBcwBfEwp/FFxgiWvO39O4U4JaDOuT3kvGm8InweFgQAUSqjqRkyXQAsWEVpICnNlBAZ8dRDGd0Yw7nf2wuWbnhV//lSgUgETLKyQONrTwlYHDNesMRSgQHS+tEwI/b1npmFR5Yq8GCx4455joj4As6JdEVJhfdTzpQdMlx4jUfQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=heSK9mY7lqWfH3SAqv1sQbW+nVa85329TUiNuT6KeBA=; b=q5fooGT8GqTdveBzd0+EoSZLA+eipi13/jYTEeVVUhhPNDd1+JX3ODBMwpv1cwjJHhzGGWUU4GxXFMZ4kgMIC3szvNpWV2Mn/p+PDcIqca4yJ0mhnX4LBNEtmHfi8K1jhraM9CeFSeyra476J4Q3sMRPn6pbTWg14V3hhEkk/r9UQldLgIXyBT9Vhx7XB0Y3lM+iE70gaEPGlyH0gVGnwNXAJEd/kaY1HRFSBzRfgzXPP0cY0aOTK4jxnvGjKQ/z18rV55+eXJQYI+9oXqlqWWLjivT5DaLQqbMdBM6z4Te7c/SaVJ9sXTxCsoeb3V/FcI8r22nFE+q+4qU8/LK6+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linutronix.de smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=heSK9mY7lqWfH3SAqv1sQbW+nVa85329TUiNuT6KeBA=; b=YWGIDJFhdRpEx3BLQdH38DUbIeibbxx/7ygZgx8CVTaJ17KKojCLSID/2ddBijYbdyVsBr2sK8uAT68+IQsP00Omzwd3JO2A8HhTfVLcaxN3hcH8li7A7d0cGXu7p64NVPUphJI6Iux90PFMpbr02JUrzXE1sqgkA8exQkOSl9A= Received: from SJ0PR13CA0066.namprd13.prod.outlook.com (2603:10b6:a03:2c4::11) by DM4PR12MB6544.namprd12.prod.outlook.com (2603:10b6:8:8d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9654.14; Fri, 27 Feb 2026 08:47:40 +0000 Received: from BY1PEPF0001AE17.namprd04.prod.outlook.com (2603:10b6:a03:2c4:cafe::88) by SJ0PR13CA0066.outlook.office365.com (2603:10b6:a03:2c4::11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9632.22 via Frontend Transport; Fri, 27 Feb 2026 08:47:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by BY1PEPF0001AE17.mail.protection.outlook.com (10.167.242.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9654.16 via Frontend Transport; Fri, 27 Feb 2026 08:47:40 +0000 Received: from satlexmb07.amd.com (10.181.42.216) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Fri, 27 Feb 2026 02:47:39 -0600 Received: from [10.136.47.108] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Fri, 27 Feb 2026 02:47:37 -0600 Message-ID: <789f0aeb-6c46-4218-8780-742fa7ff8f8d@amd.com> Date: Fri, 27 Feb 2026 14:17:31 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH] futex: Dynamically allocate futex_queues depending on nr_node_ids To: Sebastian Andrzej Siewior CC: Thomas Gleixner , Ingo Molnar , , Peter Zijlstra , Darren Hart , Davidlohr Bueso , =?UTF-8?Q?Andr=C3=A9_Almeida?= References: <20260128101358.20954-1-kprateek.nayak@amd.com> <20260224111342.-9hafXl5@linutronix.de> <20260225073939.HLbwiSs5@linutronix.de> <99a81417-5dbf-4c29-8dac-a9ed17ee24d6@amd.com> <20260225092213.oRHJyLG5@linutronix.de> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <20260225092213.oRHJyLG5@linutronix.de> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY1PEPF0001AE17:EE_|DM4PR12MB6544:EE_ X-MS-Office365-Filtering-Correlation-Id: ca303fd1-ce50-4965-bda8-08de75dcdd90 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026|13003099007; X-Microsoft-Antispam-Message-Info: h1ZwthNR276j92S7V23R3NtJ6iZj9X8LdZbTqiKzA3fkWZW3LrtcpTAUWf6Qj5hknF8njBo70rNK8OvbNse92UrBvkW3I2Lk3gSzpuwF9rNh6tfy8+VYpkAG96IgrsacvPy+mY2NdFZVO/QkmhMCi4kWaEd6IuTWNPj/lladz5pnN84pRCyEPUAJB8fVLEiHV2veGIm/p0D/Tml9GC69TJ4tyJFrzgW42kt3X0H+OfJb++0uX8LxfOFHXsdyKFEtuWDFcKpYfIG8wLHTc26Lu79Wg22BXVsNL/8o737cOZNdjr1PmOWuHYrJDw1uWm1vV+SWm6vcJcICwDx4N3VgzC9NWruHv5oBPeYzA0ieQTxM4J++C0X97BUhGlMaaORXSl5UjnolHOC3vhUOXBVoGD1QJDZl7u92XRPUSEoOwtRze3mmw6X48RTNdmgmGmtHiYqCg22y3BM1BYx6HVvEdcrOW+STOiCrNN7GOpyVIttopH+Q+nAN42UFRfCA5Lgog0d/JmYd6wicOLAAQ97E7uZc6zh/oqza7HQrzHWmAzHHZYTiVZBlsu5IieV8xiUeS0YPvcKMRwr7HhxJ5vHmZ4cVWzuJR0W9XnEiR8XRy/4qABrM6v0AGAH6hbFuJq4o7r2AxX03qpXrpTB56w/De5S82xPm9sydADv3TEqcvOc31PHpyQDoZWwkZHD42wUDyAFbS/Tpcx4d8lrrcHe+X+36afKPH0MNbiiobUqu9x9iBbV/Fng+B4rp/tovDdUwPNNExHhiAcgzh1p+FdED8w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026)(13003099007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: y61h/j0xWs4Aw0bjhtXlV4nfzg64/3zQWl414vcWdJ6EbUrxhtqI0DtsIEwLbTYBrSBlrp9upr2R/S0c5GsDyheixiPIyzCdPtFkF2EKD0PIOGRDlcnlpI5Mz8cOiOlv0lsD/supr5R7/oMkqrfMWK/NvdtzYuzY70i7TKA4TrdqwYKi7NJxXCRsH543llVy0owL8v73mktyFfgtSIqGhiiy/Up2dsrv3GeHfhbQmpayAwhdqpEAuvynU5+45B238Rtg/2nutc9/vrRLKXFsB6Psa4KAN2WMpZdmFkTMr/KYlmh3NrfThtcy2PYKzHIDy2uKOjie6+Ap2qEBQJjHFyy9X48a8jhcC9txHj6dJlMbCwBxqNyCq+IZk9QcyFZpsr2ngdjla8J8/DABZ9+17VjNZNRC2tqsxdO7LPziTpvTyCzHwDj6sHRDctLXiMhm X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2026 08:47:40.2775 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ca303fd1-ce50-4965-bda8-08de75dcdd90 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BY1PEPF0001AE17.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6544 Hey Sebastian, Sorry for the delay! On 2/25/2026 2:52 PM, Sebastian Andrzej Siewior wrote: > On 2026-02-25 14:21:33 [+0530], K Prateek Nayak wrote: > Hi Prateek, > >> I would have thought a quarter of that would be plenty but looking at >> the footnote in [1] that says "16 socket GNR system" and the fact that >> GNR can feature up to 256 threads per socket - that could theoretically >> put such systems at that NR_CPUS_DEFAULT limit - I don't know if it is >> practically possible. >> >> [1] https://lore.kernel.org/lkml/aYPjOgiO_XsFWnWu@hpe.com/ >> >> Still, I doubt such setup would practically cross more than 64 nodes. > > I am still trying to figure out if this is practical or some drunk guys > saying "you know what would be fun?" > >> Why was this selected as the default for MAXSMP? It came from [2] but >> I'm not really able to understand why other than this line in Mike's >> response: >> >> "MAXSMP" represents what's really usable >> >> so we just set it to the max of range to test for scalability? Seems >> little impractical for real-world cases but on the flip side if we >> don't sit it, some bits might not get enough testing? > > Sounds like it. What would be sane default upper limit then? Something > like 1024 CPUs? 2048? Or even more than that? I feel the current default for NR_CPUS can be be retained as is just to be on the safer side. Turns out QEMU allows for a ridiculous amount of vCPUs per guest and I've found enough evidence of extremely large guests running oversubscribed that sometimes run distro kernels :-( > > I would try to use this and convince Debian to drop MAXSMP and then > lower NODES_SHIFT to default 6. I would need a default for > NR_CPUS_DEFAULT without having people complaining about missing CPUs. > Maybe we could get a sane default setting in kernel without testing > limits. *Theoretically* with SNC-3 and 16 sockets + CXL we can get close to the !MAXSMP limits for NODES_SHIFT (6) so perhaps we should drop it down a couple of notch from 10 as far as defaults are concerned to 8 - that should give us ample room for a long time in my opinion. Folks who are doing *insane* NUMA emulation can perhaps explain the use case or resort to building a kernel with a non-default NODES_SHIFT. > > Also probably will compile two kernels to see how much memory this safes > in total since there should be other data structures depending on max > CPUs/ NODEs. To keep the configs as close as possible, I had to resort to selecting CONFIG_CPUMASK_OFFSTACK for !MAXSMP. Following was bloat-o-meter output with the reduced NODES_SHIFT on kernels built with very close to Ubuntu distro config: o NODES_SHIFT=8 : Total: Before=33017117, After=32109495, chg -2.75% o NODES_SHIFT=6 : Total: Before=33017117, After=31930101, chg -3.29% o NODES_SHIFT=6; NR_CPUS=4k : Total: Before=33017117, After=31196664, chg -5.51% o NODES_SHIFT=6; NR_CPUS=2k : Total: Before=33017117, After=30829862, chg -6.62% That last couple configs adds ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP. If I remove that dependency , I don't really see any change to the bloat-o-meter results so I don't think it makes much of a difference. Runtime memory consumption difference are within the noise range for me - I really couldn't see anything meaningful difference (or even a trend with multiple runs) between the extreme configs after boot. I haven't done any meaningful longer testing to pot anything. I'll let you decide what is a good trade off between space saving and future headaches :-) -- Thanks and Regards, Prateek