Re: Subject: Warning in workqueue.c

[Tejun Heo <tj@kernel.org>]

Hey, Peter.

On Fri, Feb 14, 2014 at 05:09:23PM +0100, Peter Zijlstra wrote:
> > Ingo, Peter, Jason is reporting workqueue triggering warning because a
> > worker is running on the wrong CPU, which is relatively reliably
> > reproducible with the above workload on s390. 
> 
> Wasn't that a feature of workqueues? You know we've had arguments about
> that behaviour -- I'm strongly in favour of flushing and killing workers
> on unplug, but you let them run on the wrong cpu.
> 
> So strongly in fact, I'd call the current behaviour quite insane and
> broken :-)

Hey, we now even keep normal kthreads across cpu down/ups. :)

> Yeah, just calling schedule() won't fix placement, you need to actually
> block and wake-up. But given you've called things like
> set_cpus_allowed_ptr() and such to set the mask back to 5..

Hmmm... I see.  It's kinda weird that the code has been there for so
long and this is the first time it's getting reported.

> You can try something like the below which makes it slightly more
> aggressive about moving tasks about.
> 
> > Any ideas?
> 
> Not really; s390 doesn't have NUMA, so all those changes are out.
> 
> ---
>  kernel/sched/core.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index fb9764fbc537..20bd4de44bb3 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4504,7 +4504,8 @@ int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask)
>  
>  	rq = task_rq_lock(p, &flags);
>  
> -	if (cpumask_equal(&p->cpus_allowed, new_mask))
> +	if (cpumask_equal(&p->cpus_allowed, new_mask) && 
> +	    cpumask_test_cpu(rq->cpu, &p->cpus_allowed))
>  		goto out;
>  
>  	if (!cpumask_intersects(new_mask, cpu_active_mask)) {

Hmmm... weird, p's rq shouldn't have changed without its cpus_allowed
busted.  Anyways, let's wait for Jason's test results and see whether
this is a regression at all.

Thanks.

-- 
tejun