[PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[Håkon Bugge]

Adding a missing EXPORT_SYMBOL_GPL() in workqueue.c, as the
workqueue_set_min_active() function is needed in the next commit.

Fixes: 8f172181f24b ("workqueue: Implement workqueue_set_min_active()")
Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>

---

v1 -> v2:
      * Added commit message
      * Added Fixes: tag
---
 kernel/workqueue.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 78068ae8f28a6..0d439df28452b 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -6183,6 +6183,7 @@ void workqueue_set_min_active(struct workqueue_struct *wq, int min_active)
 	wq_adjust_max_active(wq);
 	mutex_unlock(&wq->mutex);
 }
+EXPORT_SYMBOL_GPL(workqueue_set_min_active);
 
 /**
  * current_work - retrieve %current task's work struct
-- 
2.43.5

[PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Håkon Bugge]

When running with N online CPUs, where N is fairly large, let's say
512, a deadlock may happen in test_cycle_work() when running with
N + 1 kernel threads. The reason is that the min_active is too small
in order to let N + 1 worker threads run concurrently. The following
is my analyzes. The test sets up a circular dependency with ww_mutexes
and completions:

	work[1]   completes signal[0]
	work[2]   completes signal[1]
	...
	work[511] completes signal[510]
	work[512] would complete signal[511]
	work[0]   completes signal[512]

Therefore:

	work[0..510] -> blocked in ww_mutex_lock(b_mutex)
	work[511]    -> blocked in wait_for_completion(signal[511])
	work[512]    -> inactive; test_cycle_work() has not started

Because the last worker thread has not started, worker 511 hangs
forever in wait_for_completion().

This bug produces the following splats (slightly edited for better
brevity). This for the worker threads hung in ww_mutex_lock():

 INFO: task kworker/u2066:1:13658 blocked for more than 123 seconds.
 Workqueue: test-ww_mutex test_cycle_work [test_ww_mutex]
 Call Trace:
  __schedule+0x28e/0x670
  schedule+0x27/0xa0
  schedule_preempt_disabled+0x15/0x30
  __ww_mutex_lock.constprop.0+0x841/0xe00
  test_cycle_work+0x9d/0x150 [test_ww_mutex]
  process_one_work+0x196/0x370
  worker_thread+0x1af/0x320
  kthread+0xe3/0x120
  ret_from_fork+0x19e/0x260
  ret_from_fork_asm+0x1a/0x30

And this for the single worker thread 511, hung in
wait_for_completion():

 Workqueue: test-ww_mutex test_cycle_work [test_ww_mutex]
 Call Trace:
  __schedule+0x28e/0x670
  schedule+0x27/0xa0
  schedule_timeout+0xac/0xf0
  __wait_for_common+0x97/0x1b0
  test_cycle_work+0x80/0x160 [test_ww_mutex]
  process_one_work+0x196/0x370
  worker_thread+0x1af/0x320
  kthread+0xe3/0x120
  ret_from_fork+0x19e/0x260
  ret_from_fork_asm+0x1a/0x30

We fix this by adjusting the wq's min_active parameter.  When
num_online_cpus() has been sampled in run_tests(), we adjust the
{min,max}_active values of the wq, to make sure both are set to ncpus
+ 1 in order to avoid the above deadlock.

Note that this fix is invariant to num_online_cpus() changing after it
has been sampled, because the RC here is the number of runnable worker
threads vs. threads created, not per se the number of online CPUs.

Also, in order to avoid exceeding WQ_MAX_ACTIVE, we create cycle_ncpus
and clamp it. We do not want to change ncpus for the other tests,
not affected by this bug.

Fixes: d1b42b800e5d ("locking/ww_mutex: Add kselftests for resolving ww_mutex cyclic deadlocks")
Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>

---

v1 -> v2:
      * Added Bradley's r-b
      * Reworded comment in run_tests()
---
 kernel/locking/test-ww_mutex.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/kernel/locking/test-ww_mutex.c b/kernel/locking/test-ww_mutex.c
index 47e016a4f4fea..e11f6b869b4d4 100644
--- a/kernel/locking/test-ww_mutex.c
+++ b/kernel/locking/test-ww_mutex.c
@@ -676,6 +676,7 @@ static int stress(struct ww_class *class, int nlocks, int nthreads, unsigned int
 static int run_tests(struct ww_class *class)
 {
 	int ncpus = num_online_cpus();
+	int cycle_ncpus = min_t(int, ncpus, WQ_MAX_ACTIVE - 1);
 	int ret, i;
 
 	ret = test_mutex(class);
@@ -696,7 +697,17 @@ static int run_tests(struct ww_class *class)
 			return ret;
 	}
 
-	ret = test_cycle(class, ncpus);
+	/*
+	 * test_cycle_work() has a linear dependency which requires
+	 * all kernel threads to be run-able at once. With N CPUs and
+	 * N + 1 worker threads, deadlock may happen. Hence, adjust
+	 * min_active. Raise max first, min_active is clamped to it.
+	 * Cap N so that N + 1 doesn't exceed WQ_MAX_ACTIVE.
+	 */
+	workqueue_set_max_active(wq, cycle_ncpus + 1);
+	workqueue_set_min_active(wq, cycle_ncpus + 1);
+
+	ret = test_cycle(class, cycle_ncpus);
 	if (ret)
 		return ret;
 
-- 
2.43.5

Re: [PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[Bradley Morgan]

On 10 August 2026 17:34:30 BST, "Håkon Bugge" <haakon.bugge@oracle.com>
wrote:
>Adding a missing EXPORT_SYMBOL_GPL() in workqueue.c, as the
>workqueue_set_min_active() function is needed in the next commit.
>
>Fixes: 8f172181f24b ("workqueue: Implement workqueue_set_min_active()

Ok.

Reviewed-by: Bradley Morgan <include@grrlz.net>

>Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
>
>---
>
>v1 -> v2:
>      * Added commit message
>      * Added Fixes: tag
>---
> kernel/workqueue.c | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/kernel/workqueue.c b/kernel/workqueue.c
>index 78068ae8f28a6..0d439df28452b 100644
>--- a/kernel/workqueue.c
>+++ b/kernel/workqueue.c
>@@ -6183,6 +6183,7 @@ void workqueue_set_min_active(struct workqueue_struct *wq, int min_active)
> 	wq_adjust_max_active(wq);
> 	mutex_unlock(&wq->mutex);
> }
>+EXPORT_SYMBOL_GPL(workqueue_set_min_active);
> 
> /**
>  * current_work - retrieve %current task's work struct
>


Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Bradley Morgan]

On 10 August 2026 17:34:31 BST, "Håkon Bugge" <haakon.bugge@oracle.com>
wrote:
>When running with N online CPUs, where N is fairly large, let's say
>512, a deadlock may happen in test_cycle_work() when running with
>N + 1 kernel threads. The reason is that the min_active is too small
>in order to let N + 1 worker threads run concurrently. The following
>is my analyzes. The test sets up a circular dependency with ww_mutexes
>and completions:
>
>	work[1]   completes signal[0]
>	work[2]   completes signal[1]
>	...
>	work[511] completes signal[510]
>	work[512] would complete signal[511]
>	work[0]   completes signal[512]
>
>Therefore:
>
>	work[0..510] -> blocked in ww_mutex_lock(b_mutex)
>	work[511]    -> blocked in wait_for_completion(signal[511])
>	work[512]    -> inactive; test_cycle_work() has not started
>
>Because the last worker thread has not started, worker 511 hangs
>forever in wait_for_completion().
>
>This bug produces the following splats (slightly edited for better
>brevity). This for the worker threads hung in ww_mutex_lock():
>
> INFO: task kworker/u2066:1:13658 blocked for more than 123 seconds.
> Workqueue: test-ww_mutex test_cycle_work [test_ww_mutex]
> Call Trace:
>  __schedule+0x28e/0x670
>  schedule+0x27/0xa0
>  schedule_preempt_disabled+0x15/0x30
>  __ww_mutex_lock.constprop.0+0x841/0xe00
>  test_cycle_work+0x9d/0x150 [test_ww_mutex]
>  process_one_work+0x196/0x370
>  worker_thread+0x1af/0x320
>  kthread+0xe3/0x120
>  ret_from_fork+0x19e/0x260
>  ret_from_fork_asm+0x1a/0x30
>
>And this for the single worker thread 511, hung in
>wait_for_completion():
>
> Workqueue: test-ww_mutex test_cycle_work [test_ww_mutex]
> Call Trace:
>  __schedule+0x28e/0x670
>  schedule+0x27/0xa0
>  schedule_timeout+0xac/0xf0
>  __wait_for_common+0x97/0x1b0
>  test_cycle_work+0x80/0x160 [test_ww_mutex]
>  process_one_work+0x196/0x370
>  worker_thread+0x1af/0x320
>  kthread+0xe3/0x120
>  ret_from_fork+0x19e/0x260
>  ret_from_fork_asm+0x1a/0x30
>
>We fix this by adjusting the wq's min_active parameter.  When
>num_online_cpus() has been sampled in run_tests(), we adjust the
>{min,max}_active values of the wq, to make sure both are set to ncpus
>+ 1 in order to avoid the above deadlock.
>
>Note that this fix is invariant to num_online_cpus() changing after it
>has been sampled, because the RC here is the number of runnable worker
>threads vs. threads created, not per se the number of online CPUs.
>
>Also, in order to avoid exceeding WQ_MAX_ACTIVE, we create cycle_ncpus
>and clamp it. We do not want to change ncpus for the other tests,
>not affected by this bug.
>
>Fixes: d1b42b800e5d ("locking/ww_mutex: Add kselftests for resolving ww_mutex cyclic deadlocks")
>Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
>Reviewed-by: Bradley Morgan <include@grrlz.net>

Shouldn't a rb be on top of a sob?

>
>---
>
>v1 -> v2:
>      * Added Bradley's r-b
>      * Reworded comment in run_tests()
>---
> kernel/locking/test-ww_mutex.c | 13 ++++++++++++-
> 1 file changed, 12 insertions(+), 1 deletion(-)
>
>diff --git a/kernel/locking/test-ww_mutex.c b/kernel/locking/test-ww_mutex.c
>index 47e016a4f4fea..e11f6b869b4d4 100644
>--- a/kernel/locking/test-ww_mutex.c
>+++ b/kernel/locking/test-ww_mutex.c
>@@ -676,6 +676,7 @@ static int stress(struct ww_class *class, int nlocks, int nthreads, unsigned int
> static int run_tests(struct ww_class *class)
> {
> 	int ncpus = num_online_cpus();
>+	int cycle_ncpus = min_t(int, ncpus, WQ_MAX_ACTIVE - 1);
> 	int ret, i;
> 
> 	ret = test_mutex(class);
>@@ -696,7 +697,17 @@ static int run_tests(struct ww_class *class)
> 			return ret;
> 	}
> 
>-	ret = test_cycle(class, ncpus);
>+	/*
>+	 * test_cycle_work() has a linear dependency which requires
>+	 * all kernel threads to be run-able at once. With N CPUs and
>+	 * N + 1 worker threads, deadlock may happen. Hence, adjust
>+	 * min_active. Raise max first, min_active is clamped to it.
>+	 * Cap N so that N + 1 doesn't exceed WQ_MAX_ACTIVE.
>+	 */
>+	workqueue_set_max_active(wq, cycle_ncpus + 1);
>+	workqueue_set_min_active(wq, cycle_ncpus + 1);
>+
>+	ret = test_cycle(class, cycle_ncpus);
> 	if (ret)
> 		return ret;
> 
>

Thanks!

Re: [PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[Tejun Heo]

On Mon, Aug 10, 2026 at 06:34:30PM +0200, Håkon Bugge wrote:
> Adding a missing EXPORT_SYMBOL_GPL() in workqueue.c, as the
> workqueue_set_min_active() function is needed in the next commit.

Can you just spell out who's gonna use it? I'm only cc'd on this one and
patch descs should be self-contained - it can reference future changes but
it should still be at least somewhat specific.

Thanks.

-- 
tejun

Re: [PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[Bradley Morgan]

On 10 August 2026 20:00:22 BST, Tejun Heo <tj@kernel.org> wrote:
>On Mon, Aug 10, 2026 at 06:34:30PM +0200, Håkon Bugge wrote:
>> Adding a missing EXPORT_SYMBOL_GPL() in workqueue.c, as the
>> workqueue_set_min_active() function is needed in the next commit.
>
>Can you just spell out who's gonna use it? I'm only cc'd on this one and
>patch descs should be self-contained - it can reference future changes but
>it should still be at least somewhat specific.
>

Mmm, shouldn't be complicated, perhaps

Export workqueue_set_min_active() so that tests/loadable modules,
e.g: test-ww_mutex can adjust workqueue concurrency levels to
avoid deadlocks during cyclic test runs

Again, feel free to bikeshed.




>Thanks.
>
>

Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Haakon Bugge]

> On 10 Aug 2026, at 18:45, Bradley Morgan <include@grrlz.net> wrote:
> 
> [snip]

> >Fixes: d1b42b800e5d ("locking/ww_mutex: Add kselftests for resolving ww_mutex cyclic deadlocks")
> >Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
> >Reviewed-by: Bradley Morgan <include@grrlz.net>
> 
> Shouldn't a rb be on top of a sob?

Not to my knowledge. Do you have a reference? Looking at recent
commits, I see a great variability as to your Q. E.g., see this
commit:

a50026bdb867 ("iov_iter: get rid of 'copy_mc' flag")


Thxs, Håkon

Re: [PATCH v2 1/2] workqueue: Add missing EXPORT_SYMBOL_GPL for workqueue_set_min_active

[Haakon Bugge]

> On 10 Aug 2026, at 21:05, Bradley Morgan <include@grrlz.net> wrote:
> On 10 August 2026 20:00:22 BST, Tejun Heo <tj@kernel.org> wrote:
> >On Mon, Aug 10, 2026 at 06:34:30PM +0200, Håkon Bugge wrote:
> >> Adding a missing EXPORT_SYMBOL_GPL() in workqueue.c, as the
> >> workqueue_set_min_active() function is needed in the next commit.
> >
> >Can you just spell out who's gonna use it? I'm only cc'd on this one and
> >patch descs should be self-contained - it can reference future changes but
> >it should still be at least somewhat specific.
> 
Hi Tejun,

Let me more specific in my v3 and also explicitly CC you on the commit
using workqueue_set_min_active().

> Mmm, shouldn't be complicated, perhaps
> 
> Export workqueue_set_min_active() so that tests/loadable modules,
> e.g: test-ww_mutex can adjust workqueue concurrency levels to
> avoid deadlocks during cyclic test runs

Bradley, thanks for the suggestion. Will incorporate it in my v3.

Again, feel free to bikeshed.

As always ;-)


Thxs, Håkon

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Bradley Morgan]

On 11 August 2026 16:32:40 BST, Haakon Bugge <haakon.bugge@oracle.com>
wrote:
>
>
>> On 10 Aug 2026, at 18:45, Bradley Morgan <include@grrlz.net> wrote:
>> 
>> [snip]
>
>> >Fixes: d1b42b800e5d ("locking/ww_mutex: Add kselftests for resolving
>ww_mutex cyclic deadlocks")
>> >Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
>> >Reviewed-by: Bradley Morgan <include@grrlz.net>
>> 
>> Shouldn't a rb be on top of a sob?
>
>Not to my knowledge. Do you have a reference? Looking at recent
>commits, I see a great variability as to your Q. E.g., see this
>commit:
>
>a50026bdb867 ("iov_iter: get rid of 'copy_mc' flag")
>
>
>Thxs, Håkon
>
>

usually from my very small experience, I tend to merge patches into my
-next tree, a private one I use to test and stuff, and using B4, RBs tend to be on top of SOBs.

Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Bradley Morgan]

On 11 August 2026 16:39:58 BST, Bradley Morgan <include@grrlz.net> wrote:
>On 11 August 2026 16:32:40 BST, Haakon Bugge <haakon.bugge@oracle.com>
>wrote:
>>
>>
>>> On 10 Aug 2026, at 18:45, Bradley Morgan <include@grrlz.net> wrote:
>>> 
>>> [snip]
>>
>>> >Fixes: d1b42b800e5d ("locking/ww_mutex: Add kselftests for resolving
>>ww_mutex cyclic deadlocks")
>>> >Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
>>> >Reviewed-by: Bradley Morgan <include@grrlz.net>
>>> 
>>> Shouldn't a rb be on top of a sob?
>>
>>Not to my knowledge. Do you have a reference? Looking at recent
>>commits, I see a great variability as to your Q. E.g., see this
>>commit:
>>
>>a50026bdb867 ("iov_iter: get rid of 'copy_mc' flag")
>>
>>
>>Thxs, Håkon
>>
>>
>
>usually from my very small experience, I tend to merge patches into my
>-next tree, a private one I use to test and stuff, and using B4, RBs tend to be on top of SOBs.
>
>Thanks!


Btw:

<chris@chris-wilson.co.uk>: Host or domain name not found. Name service
error  for name=chris-wilson.co.uk type=AAAA: Host not found

His email seems to bounce? Remove him from cc

Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Haakon Bugge]

> On 11 Aug 2026, at 17:41, Bradley Morgan <include@grrlz.net> wrote:
> On 11 August 2026 16:39:58 BST, Bradley Morgan <include@grrlz.net> wrote:
> >On 11 August 2026 16:32:40 BST, Haakon Bugge <haakon.bugge@oracle.com>

[snip]

> Btw:
> 
> <chris@chris-wilson.co.uk>: Host or domain name not found. Name service
> error for name=chris-wilson.co.uk type=AAAA: Host not found
> 
> His email seems to bounce? Remove him from cc

I know. I do not explicitly CC him, but his address comes from
the MAINTAINERS file. He has to be removed from that.


Thxs, Håkon

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Bradley Morgan]

On 11 August 2026 16:50:02 BST, Haakon Bugge <haakon.bugge@oracle.com>
wrote:
>
>
>> On 11 Aug 2026, at 17:41, Bradley Morgan <include@grrlz.net> wrote:
>> On 11 August 2026 16:39:58 BST, Bradley Morgan <include@grrlz.net>
>wrote:
>> >On 11 August 2026 16:32:40 BST, Haakon Bugge <haakon.bugge@oracle.com>
>
>[snip]
>
>> Btw:
>> 
>> <chris@chris-wilson.co.uk>: Host or domain name not found. Name service
>> error for name=chris-wilson.co.uk type=AAAA: Host not found
>> 
>> His email seems to bounce? Remove him from cc
>
>I know. I do not explicitly CC him, but his address comes from
>the MAINTAINERS file. He has to be removed from that.
>

urgh, I'll do a patch on that..


>
>Thxs, Håkon
>
>
>

Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Bradley Morgan]

On 11 August 2026 16:50:39 BST, Bradley Morgan <include@grrlz.net> wrote:
>On 11 August 2026 16:50:02 BST, Haakon Bugge <haakon.bugge@oracle.com>
>wrote:
>>
>>
>>> On 11 Aug 2026, at 17:41, Bradley Morgan <include@grrlz.net> wrote:
>>> On 11 August 2026 16:39:58 BST, Bradley Morgan <include@grrlz.net>
>>wrote:
>>> >On 11 August 2026 16:32:40 BST, Haakon Bugge <haakon.bugge@oracle.com>
>>
>>[snip]
>>
>>> Btw:
>>> 
>>> <chris@chris-wilson.co.uk>: Host or domain name not found. Name service
>>> error for name=chris-wilson.co.uk type=AAAA: Host not found
>>> 
>>> His email seems to bounce? Remove him from cc
>>
>>I know. I do not explicitly CC him, but his address comes from
>>the MAINTAINERS file. He has to be removed from that.
>>
>
>urgh, I'll do a patch on that..
>
>
>>
>>Thxs, Håkon
>>
>>
>>
>
>Thanks!

WORKQUEUE
M:      Tejun Heo <tj@kernel.org>
R:      Lai Jiangshan <jiangshanlai@gmail.com>
S:      Maintained
T:      git git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git
F:      Documentation/core-api/workqueue.rst
F:      include/linux/workqueue.h
F:      kernel/workqueue.c
F:      kernel/workqueue_internal.h


actually. Linux-next, maybe ./get-maintainer.pl catches it?
Thanks!

Re: [PATCH v2 2/2] test-ww_mutex: Fix deadlock in test_cycle_work

[Haakon Bugge]

> On 11 Aug 2026, at 17:53, Bradley Morgan <include@grrlz.net> wrote:
> 
> WORKQUEUE
> M: Tejun Heo <tj@kernel.org>
> R: Lai Jiangshan <jiangshanlai@gmail.com>
> S: Maintained
> T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git
> F: Documentation/core-api/workqueue.rst
> F: include/linux/workqueue.h
> F: kernel/workqueue.c
> F: kernel/workqueue_internal.h

I have to correct myself. Chris is added, because of the
Fixes: d1b42b800e5d tag.


Thxs, Håkon