From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6BF9731064B for ; Wed, 24 Jun 2026 11:48:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782301683; cv=none; b=CnCOMm0AD16fWmXIlsIbXM5b9ReCJTtcBMWlI6zt7XiEsVPS4wBHKBky6jmDeu0mgoBX9unLkdr1bjxwNU9/AwNtgh+5cskflkJr0TErK1xsKE4S4E+zmHCIZrPFlJgpljXSzNn9XR5xvHkZQCek8EhfRQP00Z5x4kts/JZun10= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782301683; c=relaxed/simple; bh=okx5uRiid5ZmFdb1U6ImZ8a9mpHfgDnle80J7gJZAl8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V2nsjrhQP6AcJlPkX6fLJoHfPSOHnw3LnfqwvpfrRM8nzAbXffpFEBgB8l0QBF/ORy8MjOX1bCdS/NNhrcerCUSKVwarp4Uwxwl9JOu32S1wSnGmEpoW0V5pP9+Q6SMOzfnWCE26IIWFo6CpOTe2/BJeH2dRrNJQvwM2vdHuMm0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=goK008jf; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="goK008jf" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=SztcDVig7NStIWoVhjFKSfytwulz0PeIVPkWf121ws8=; b=goK008jfjQG/uP7w2B6VvBd8yH 1BAehOajDVYfnrBOsgdCjG/Ss86FoZgwFZ+KPFSFm7MHD+d3EU3oWxeS14kb46I8nUL351tT+W4cq /RpOO/6Wv3pP1AfvRsCPAiNyeRRoebC9SjPXiDsFyWul+e/4D2LWzQv1dacUVWZhTQ5nLQKce+cnX /cIjMCCQvwCGsG7JteHDNEGSQXmSE+9yZ8f+JcU9Sg68N5vmU1rmJRycZwmCK2U9DUpN89PKSlEgM ickxpUHAZHyforT9qsqZi6DDjYKk8uq8wxK/hIVXuYd/aTYH4MWTC5DUffqldtV51hKWny23DvHe4 Sj31fPUQ==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wcM5I-002Oxl-2F; Wed, 24 Jun 2026 11:48:00 +0000 From: Breno Leitao Date: Wed, 24 Jun 2026 04:47:40 -0700 Subject: [PATCH v4 2/3] workqueue: defer the worker wakeup outside pool->lock in __queue_work() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260624-fastwake-v4-2-7b6d7b494a44@debian.org> References: <20260624-fastwake-v4-0-7b6d7b494a44@debian.org> In-Reply-To: <20260624-fastwake-v4-0-7b6d7b494a44@debian.org> To: Tejun Heo , Lai Jiangshan , bigeasy@linutronix.de Cc: linux-kernel@vger.kernel.org, marco.crivellari@suse.com, frederic@kernel.org, bigeasy@linutronix.de, Hillf Danton , Breno Leitao , kernel-team@meta.com, kmagar@redhat.com, psuriset@redhat.com, david.dai@linux.dev X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1705; i=leitao@debian.org; h=from:subject:message-id; bh=okx5uRiid5ZmFdb1U6ImZ8a9mpHfgDnle80J7gJZAl8=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqO8PkAdmGQY33BR8aXkMuziB5adwVuUMcWDyPi rvfIEyCMXSJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCajvD5AAKCRA1o5Of/Hh3 bXa8EACQ50Jkm5GaeVehFULWbao+rPPgnhzgDoL1Tx85QWnsr7SFzXvBPFh1BPf6L6FprbTKPr4 M/rxrwjH43bz+SH1XK+azOkLgKTHmecyKfW2t+NSuFzMrdZRfF22/WWXVYSwpmQXTusg3s2buDM Wd5opO+/emQLdOSu8jLCdS9x+zi9VlCJRG+5dkjglwP0y9cEKNCMqjj+HXYApUZD1kek7b16bqH MTzfSMBxW0F0pdQ4NBiwSU/2uVoQW0iOiwofELoumvq464x9hqzf+GDRi8tJoCtLECJgclCGAqI /dVTTn8fH6oJwwIby62PHd5cSzoVKgsYJuxRkJqpSehmaXeWnMcox33xy6pg3HUqFkIWkte6qvP LDQ0EALQmVdFZDgduKLqi9NcPy3RH4GmFx+WcwmgdB7zx9t2DkQ6zxBRa2azCu8YSIdESDWePK4 ousjVBFsQ+Gvm8w5bZPWQwm6AxOtVW48spGIuYlnsbsr8+L1GTn7gctp1zsPuOpMUErh+y6gwMV O8ay2kpQevjK+iDpzwDtQMEx0zzTkPinn0OMm3rvmnQFEQNk0WyFJmXqAlCcSDQJRRhFSRLrvIH 05WBCHhCoKZgwWf6WivFIO0hhBC6KR74fkXuKBlzetyp8lBugDqUdeHzcCPngs1levWKmDtBeit NXWpdQYZXg8BKsw== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao __queue_work() is the enqueue hot path: it inserts the work item and calls kick_pool() while holding pool->lock. kick_pool() ends in a wakeup, which takes the target task's rq->lock, so rq->lock nests under pool->lock on every enqueue that wakes a worker on a contended unbound pool. Use kick_pool_pick() to select and claim the worker under pool->lock, queue it on an on-stack wake_q, and issue the wakeup with wake_up_q() right after dropping the lock. Worker selection, wake_cpu setup and claiming the worker off pool->idle_list still happen under the lock; only the rq->lock acquisition moves out. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index fd3b5bc78df9e..972f783f98281 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -2299,6 +2299,7 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, { struct pool_workqueue *pwq; struct worker_pool *last_pool, *pool; + DEFINE_WAKE_Q(wakeq); unsigned int work_flags; unsigned int req_cpu = cpu; @@ -2421,7 +2422,7 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, trace_workqueue_activate_work(work); insert_work(pwq, work, &pool->worklist, work_flags); - kick_pool(pool); + kick_pool_pick(pool, &wakeq); } else { work_flags |= WORK_STRUCT_INACTIVE; insert_work(pwq, work, &pwq->inactive_works, work_flags); @@ -2429,6 +2430,7 @@ static void __queue_work(int cpu, struct workqueue_struct *wq, out: raw_spin_unlock(&pool->lock); + wake_up_q(&wakeq); rcu_read_unlock(); } -- 2.53.0-Meta