Re: [RFC PATCH(Experimental) 2/4] Revert changes to workqueue.c

[Srivatsa Vaddagiri <vatsa@in.ibm.com>]

On Wed, Feb 14, 2007 at 11:09:04PM +0300, Oleg Nesterov wrote:
> What else you don't like? Why do you want to remove cwq_should_stop() and
> restore an ugly (ugly for workqueue.c) kthread_stop/kthread_should_stop() ?

What is ugly abt kthread_stop in workqueue.c?

I feel it is nice if the cleanup is synchronous i.e when cpu_down() is
complete, all the dead cpu's worker threads would have terminated.
Otherwise we expose races between CPU_UP_PREPARE/kthread_create and the
(old) thread exiting.

> We can restore take_over_works(), although I don't see why this is needed.
> But cwq_should_stop() will just work regardless, why do you want to add
> this "wait_to_die" ... well, hack :)

wait_to_die is not a new "hack"! Its already used in several other
places ..

> > -static DEFINE_MUTEX(workqueue_mutex);
> > +static DEFINE_SPINLOCK(workqueue_lock);
> 
> No. We can't do this. see below.

Ok ..

> >  struct workqueue_struct *__create_workqueue(const char *name,
> >  					    int singlethread, int freezeable)
> >  {
> > @@ -798,17 +756,20 @@ struct workqueue_struct *__create_workqu
> >  		INIT_LIST_HEAD(&wq->list);
> >  		cwq = init_cpu_workqueue(wq, singlethread_cpu);
> >  		err = create_workqueue_thread(cwq, singlethread_cpu);
> > +		if (!err)
> > +			wake_up_process(cwq->thread);
> >  	} else {
> > -		mutex_lock(&workqueue_mutex);
> > +		spin_lock(&workqueue_lock);
> >  		list_add(&wq->list, &workqueues);
> > -
> > -		for_each_possible_cpu(cpu) {
> > +		spin_unlock(&workqueue_lock);
> > +		for_each_online_cpu(cpu) {
> >  			cwq = init_cpu_workqueue(wq, cpu);
> > -			if (err || !(cpu_online(cpu) || cpu == embryonic_cpu))
> > -				continue;
> >  			err = create_workqueue_thread(cwq, cpu);
> > +			if (err)
> > +				break;
> 
> No, we can't break. We are going to execute destroy_workqueue(), it will
> iterate over all cwqs.

and try to kthread_stop() uninitialized cwq->thread?

How abt retaining the break above but setting cwq->thread = NULL in
create_workqueue_thread in failure case?

> > +static void take_over_work(struct workqueue_struct *wq, unsigned int cpu)
> > +{

<snip>

> > +}
> 
> I think this is unneeded complication, but ok, should work.

This is required if we want to stop per-cpu threads synchronously.

> > +	case CPU_DEAD:
> > +		list_for_each_entry(wq, &workqueues, list)
> > +			take_over_work(wq, hotcpu);
> > +		break;
> > +
> > +	case CPU_DEAD_KILL_THREADS:
> > +		list_for_each_entry(wq, &workqueues, list)
> > +			cleanup_workqueue_thread(wq, hotcpu);
> >  	}
> 
> Both CPU_UP_CANCELED and CPU_DEAD_KILL_THREADS runs after thaw_processes(),
> this means that workqueue_cpu_callback() is racy wrt create/destroy workqueue,
> we should take the mutex, and it can't be spinlock_t.

Ok yes ..thanks for pointing out!

-- 
Regards,
vatsa