From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6B2533C535 for ; Thu, 5 Feb 2026 23:29:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770334170; cv=none; b=iRosPLXGW2+xJHFojmw3EqQI32VeucI1QIlGiEawdvQpifbE532pio73zCG+lTvqKFX9MU1nzFhFzjFy6dH6AdDNBxdn2UVFL1L8wyySsPZgb0/yc733ZBbkiw6Yn/6XL+Zy34vVpC0zhJh1q6JYI3bDYWHgjnvrFyDVewmOSjI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770334170; c=relaxed/simple; bh=N2cC3Zchal1rqpxkBwoNHbYNbbf4dVR+COt1oYV0yF0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=nRQeONRjeVd2DpOFD190N1aQP4BiLuFVKrc2Qp2w7mgo261nWpY9WTGJEDMXYUjtlYtR3XgW577X060cvCHFsOsb5c5f4+5pyVIRjCe6Za2nXQf+PSBv4GBVJOq7hgb95Sfg6zANuZqbds7BddZe+blIJMUyR8SdpbPcxoRVZgU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gIjGijXD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gIjGijXD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 05FACC4CEF7; Thu, 5 Feb 2026 23:29:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1770334170; bh=N2cC3Zchal1rqpxkBwoNHbYNbbf4dVR+COt1oYV0yF0=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=gIjGijXDhT/xQIjL2nE0O58zYTFQaO8/pliyDk8apFFMzOVHDwnMp1ZrIE5R2vlBo 0vzG1VVsmorILcrd9XNDeQDC9kWlJsTX3SoYMQB6GIbNNM4aE1QhE7KG9GAR8qaPCh X1dUsc9RzHArWm5wnP9b5YFdLN3dySxM1CPh3UlpLlnnPYK2NKA21wCr/XtraocX4a XHtoaWBuW0Y/05TLrawXf0lcpeT5e2YmQqThS9Mxwg6KTLblPJvx6MVpeawWMW8oiT Gt942xOv2IOAE6pznWrK+XFkYW6VbWid9Ep1YuaGPcFvgdP1Ke0XaQC5cMLsEK6jWQ KibNwJnaib1Xg== Message-ID: Date: Thu, 5 Feb 2026 18:29:28 -0500 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] workqueue: Automatic affinity scope fallback for single-pod topologies To: Tejun Heo Cc: jiangshanlai@gmail.com, linux-kernel@vger.kernel.org, Chuck Lever References: <20260205024912.6753-1-cel@kernel.org> Content-Language: en-US From: Chuck Lever Organization: kernel.org In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 2/5/26 5:10 PM, Tejun Heo wrote: > Hello, Chuck. > > On Wed, Feb 04, 2026 at 09:49:11PM -0500, Chuck Lever wrote: >> +static bool __init cpus_share_cluster(int cpu0, int cpu1) >> +{ >> + return cpumask_test_cpu(cpu0, topology_cluster_cpumask(cpu1)); >> +} > > Cluster boundary == core boundary for a lot of CPUs. I don't think this is > going to work. Fair enough; WQ_AFFN_CLUSTER is not a reliable intermediate level. On x86 cpu_clustergroup_mask() returns cpu_l2c_shared_mask(), which is per-core on many chips. The arm64 cpu_clustergroup_mask() has a similar collapse: when cluster_sibling spans the coregroup, it falls back to SMT siblings. And the generic fallback in topology.h is cpumask_of(cpu). I was hoping it would be a proper intermediate sharding scope. > Here are a couple options:> > - Introduce an affinity level which splits CACHE according to some > adjustable heuristics. > > - Make the NFS workqueue default to WQ_AFFN_CORE (or maybe switch based on > some heuristics) or switch to a per-cpu workqueue. The issue I see is that the contention isn't confined to a single workqueue. In the NFS-over-RDMA I/O paths, at least four unbound workqueues are in the hot path: - rpciod (WQ_UNBOUND) in net/sunrpc/sched.c -- core RPC task wake-up on every completion - xprtiod (WQ_UNBOUND) in net/sunrpc/xprt.c -- transport cleanup and receive processing - nfsiod (WQ_UNBOUND) in fs/nfs/inode.c -- direct write and local I/O completion - svcrdma_wq (WQ_UNBOUND) in svc_rdma.c -- send context and write info release on every RDMA completion These span three subsystems and maintainers. Other RDMA ULPs (iSER, SRP target, kSMBd) have their own unbound workqueues with the same exposure. Tuning each one individually is fragile, and any new WQ_UNBOUND workqueue added to these paths inherits the degenerate default behavior. Even on platforms with a large core-to-pod ratio, pool lock contention is going to be a significant problem when WQ pools are shared by more than a handful of cores. I don't have access to the kind of hardware needed to deeply test sharding ideas, so I'll drop this patch for now and simply set boot command line options on all my systems. -- Chuck Lever