From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from stravinsky.debian.org (stravinsky.debian.org [82.195.75.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7145C2EC09F for ; Wed, 24 Jun 2026 11:48:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=82.195.75.108 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782301687; cv=none; b=Uvd19ftRUcAMwBqunfHoLhrRP/YcSNL96AWtAX4dgx8HIvpdzYTWSIZWmGo0k+tTEy0uPFqi3UvhwCfEKDJpH9MgznWQOS70tlKH2vCgxUSLO1ZIR9Lh9OvSxElz1u767bpiFYC1dDibG/tNMaVUGKuzWlYRhMXt7hfLpZ1pny0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782301687; c=relaxed/simple; bh=s1T3k5TUsWavtaRcz8VFJEoQsFelm4PYCakdszfMph8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FGx86yZWSA1RyDuWYOj1WrrcIqYv/t3wPGCra8qxzzBk4p3VLcRxrIVrolxrc+ya6qu15NHtejFnpWdX6I8p2hHXQps6YtKk/2a9Mvat3WhyregMidrVOF9hcoduBoqVcEZOlT3Fy5xy0Zgbe76PBDuR0VIcOfkI5pg716W2HGs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org; spf=pass smtp.mailfrom=debian.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b=i1Y/Rm7P; arc=none smtp.client-ip=82.195.75.108 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=debian.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=debian.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=debian.org header.i=@debian.org header.b="i1Y/Rm7P" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description; bh=bsxGkGI1r/+lazbX5WmPogoZo+gehkA6kggiQWzW9Uw=; b=i1Y/Rm7PBU4JsrkP73albP0sL6 f5YaKVQ7MojtXzfN6H/coV42TdFaOhk+8zJuPSaxpSVPwAjXFO9jxW/52rlVluBI38cgf74lKbvfl ae0315E5Js60F4ex5n39wegJsOj+P52NrJTdUXbpcenVnPP+jNql+ZacdYxX3K4qTTi/soVe9xb73 XT6U+9UfShY240hHj7qJQohF6kgDd267Chejhz/DDl0oIczBKQwlvwWDN9XHkbd7Y7jotKWDW8iwj jCc+PZ8OTkmV0ULrRbfX5BWftfrEixkwN6IfkzJfXFDi1b4sS23htTz6EeP3ouGoGmDn6gVDWg1vX VJj1r/Mw==; Received: from authenticated-user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from ) id 1wcM5M-002Oxt-2N; Wed, 24 Jun 2026 11:48:04 +0000 From: Breno Leitao Date: Wed, 24 Jun 2026 04:47:41 -0700 Subject: [PATCH v4 3/3] workqueue: defer the worker wakeup outside pool->lock in process_one_work() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260624-fastwake-v4-3-7b6d7b494a44@debian.org> References: <20260624-fastwake-v4-0-7b6d7b494a44@debian.org> In-Reply-To: <20260624-fastwake-v4-0-7b6d7b494a44@debian.org> To: Tejun Heo , Lai Jiangshan , bigeasy@linutronix.de Cc: linux-kernel@vger.kernel.org, marco.crivellari@suse.com, frederic@kernel.org, bigeasy@linutronix.de, Hillf Danton , Breno Leitao , kernel-team@meta.com, kmagar@redhat.com, psuriset@redhat.com, david.dai@linux.dev X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1575; i=leitao@debian.org; h=from:subject:message-id; bh=s1T3k5TUsWavtaRcz8VFJEoQsFelm4PYCakdszfMph8=; b=owEBbQKS/ZANAwAIATWjk5/8eHdtAcsmYgBqO8PkMwGB9XNWmdf6sI1sgBYC4CO6FTeoGG425 riDzmR4heWJAjMEAAEIAB0WIQSshTmm6PRnAspKQ5s1o5Of/Hh3bQUCajvD5AAKCRA1o5Of/Hh3 bVPUD/9gExubBwQwRR51J3GDt06ZLLLCIdv7Qo3wvkRyf/TTdYAYUWdpex3DJ9IzzBzNO4a7DyI qzGAnazOTrepvMb9vf+DQ8c2s3CAFZpwOCaVutOzkAFKubpYLgBJZt2Nt5/Mjlx4mYyVg+b5N0x muoaasGy0TmaaaGsSmflBHL9t5TUh7vvlTEQoLpwQX5JyEhg6KydCliRRsBzUoggIq97kLyklEL PoufrsoYCu2lJUKSVoNC9s9oiuMAaraykDMSVislyRKmPWMNf624D/zwU/SEg95eI8ZUViDbU8J myfKjFXzobfjLVuOfljWSBaUE3ApDpKBBPQ5NCOeIk66j1kOnBFOZ7hGJvL7XYUGQYYT+lPtWrL 8eblgYm/KK6NzSIRWbQPm4JSvF2DZOJCul5Sg7QEGNPoCWUIOFfjMjWNXNynaHoCgY/6Opwzg+u N+9Va/gwg7l9nNAlqLJFSEbjR+T5svM6B+31d/ekl45/nyN4uVOiHtDVQvR58A1/5gADEYX398a Y6eyvmW3y7b7a226ss92OBG16VvFWCIzzN+nTLDBizrwQ22nGHxw7H6xvH7TKZmVUib36I6zcBe tttxrPx1M3SGkuhcmBLGBJXBTwWbp6BQXv1YpuAPXPtMQKHyPEXx3/26iXhG+ne7XFl4nNmw/ld Me1ZbIVKpL558Wg== X-Developer-Key: i=leitao@debian.org; a=openpgp; fpr=AC8539A6E8F46702CA4A439B35A3939FFC78776D X-Debian-User: leitao process_one_work() kicks the pool to chain execution of the remaining work items on WORKER_NOT_RUNNING pools (the UNBOUND and CPU_INTENSIVE ones), calling kick_pool() while holding pool->lock. As in the enqueue path, the wakeup pulls the target rq->lock in under pool->lock. Use kick_pool_pick() to select and claim the worker under pool->lock and issue the wakeup with wake_up_q() after the lock is dropped. Signed-off-by: Breno Leitao --- kernel/workqueue.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 972f783f98281..ab62af99852ce 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -3251,6 +3251,7 @@ __acquires(&pool->lock) { struct pool_workqueue *pwq = get_work_pwq(work); struct worker_pool *pool = worker->pool; + DEFINE_WAKE_Q(wakeq); unsigned long work_data; int lockdep_start_depth, rcu_start_depth; bool bh_draining = pool->flags & POOL_BH_DRAINING; @@ -3305,7 +3306,7 @@ __acquires(&pool->lock) * chain execution of the pending work items for WORKER_NOT_RUNNING * workers such as the UNBOUND and CPU_INTENSIVE ones. */ - kick_pool(pool); + kick_pool_pick(pool, &wakeq); /* * Record the last pool and clear PENDING which should be the last @@ -3317,6 +3318,7 @@ __acquires(&pool->lock) pwq->stats[PWQ_STAT_STARTED]++; raw_spin_unlock_irq(&pool->lock); + wake_up_q(&wakeq); rcu_start_depth = rcu_preempt_depth(); lockdep_start_depth = lockdep_depth(current); -- 2.53.0-Meta