From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F95341C2E8 for ; Fri, 27 Feb 2026 14:42:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772203331; cv=none; b=GbFje6HeLFeoqqRTgyrhmSkubRrRlSNxgD7V/+L5EIuP0vId/RIMzg2G53fR8ekuUquBDBJ6lIvrtX0nChlSZDIebU6JfI8INhqTK+zfQ7lWwnaKXThAfwSNXLpJoe9VAc1mqHZMZcnxDYrbYPWJgZ0Prx6TZ73QuCNgN0Aj7u8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772203331; c=relaxed/simple; bh=7VDf2Zi8vlQ97ZxP1hilISJuI1NhS2k7wtmfjEr7eR0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jCZfTwiADpfrO1sBhGWV5C39ZuYg0IZDAq1PTm0fIxNnsikjb6Xc29FUJ0FBySi4MQyjkTf54iBASGTam3CCOGi1TI5RO9F9COFhQblcaCahS6vvAQ3CNTky7c9nU/xRXbVG0pPjqxtPjwIFSLsLjQC+Oizsz9fl444us7x9HRk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=bVcA6OKC; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bVcA6OKC" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=P9ha9pyhuvlMDEQBo9VvHrzpJH+jzWZgc4AniATzZXw=; b=bVcA6OKCQBbMVil+m4oJK2PEA7 6T9Kxt3rLg8Z3i8hrY7hzlwqh2BT+IpKOM09zLC+OP2GNNRE+IUL2JEZx20r1Is6nXi4BkYa/wt9n xe2lR1XlrVoKdBC3m3DyzcWnfx7JiaWFKvPaWcD56Q+qePFY0dc602CLmpDliZH6ak2JonyL44inN bde34ti7L5tVZXk0G+gID8W8IZVZyHXzdJEMXq5OjyFIPn9bmRagLZEu7qiUuBbXSaq6Z6LNCmlci lwI4fDQvoD85GwBEDmfyJicEmuMCbpVhph8o5SRx0utLgVnvVd66VyBzZA0z6aHeK5Yd35DNMOcy/ WH9esCzA==; Received: from 2001-1c00-8d85-5700-266e-96ff-fe07-7dcc.cable.dynamic.v6.ziggo.nl ([2001:1c00:8d85:5700:266e:96ff:fe07:7dcc] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1vvz2a-0000000DGaK-0kiV; Fri, 27 Feb 2026 14:42:04 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 31C3E30066A; Fri, 27 Feb 2026 15:42:03 +0100 (CET) Date: Fri, 27 Feb 2026 15:42:03 +0100 From: Peter Zijlstra To: K Prateek Nayak Cc: Thomas Gleixner , Ingo Molnar , Sebastian Andrzej Siewior , linux-kernel@vger.kernel.org, Darren Hart , Davidlohr Bueso , =?iso-8859-1?Q?Andr=E9?= Almeida Subject: Re: [RFC PATCH] futex: Dynamically allocate futex_queues depending on nr_node_ids Message-ID: <20260227144203.GJ1282955@noisy.programming.kicks-ass.net> References: <20260128101358.20954-1-kprateek.nayak@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260128101358.20954-1-kprateek.nayak@amd.com> On Wed, Jan 28, 2026 at 10:13:58AM +0000, K Prateek Nayak wrote: > CONFIG_NODES_SHIFT (which influences MAX_NUMNODES) is often configured > generously by distros while the actual number of possible NUMA nodes on > most systems is often quite conservative. > > Instead of reserving MAX_NUMNODES worth of space for futex_queues, > dynamically allocate it based on "nr_node_ids" at the time of > futex_init(). > > "nr_node_ids" at the time of futex_init() is cached as "nr_futex_queues" > to compensate for the extra dereference necessary to access the elements > of futex_queues which ends up in a different cacheline now. > > Running 5 runs of perf bench futex showed no measurable impact for any > variants on a dual socket 3rd generation AMD EPYC system (2 x 64C/128T): > > variant locking/futex base + patch %diff > futex/hash 1220783.2 1333296.2 (9%) > futex/wake 0.71186 0.72584 (2%) > futex/wake-parallel 0.00624 0.00664 (6%) > futex/requeue 0.25088 0.26102 (4%) > futex/lock-pi 57.6 57.8 (0%) > > Note: futex/hash had noticeable run to run variance on test machine. > > "nr_node_ids" can rarely be larger than num_possible_nodes() but the > additional space allows for simpler handling of node index in presence > of sparse node_possible_map. > > Reported-by: Sebastian Andrzej Siewior > Signed-off-by: K Prateek Nayak > --- > Sebastian, > > Does this work for your concerns with the large "MAX_NUMNODES" values on > most distros? It does put the "queues" into a separate cacheline from > the __futex_data. > > The other option is to dynamically allocate the entire __futex_data as: > > struct { > unsigned long hashmask; > unsigned int hashshift; > unsigned int nr_queues; > struct futex_hash_bucket *queues[] __counted_by(nr_queues); > } *__futex_data __ro_after_init; > > with a variable length "queues" at the end if we want to ensure > everything ends up in the same cacheline but all the __futex_data > member access would then be pointer dereferencing which might not be > ideal. > > Thoughts? Both will result in at least one extra deref/cacheline for each futex op, no?