Re: [PATCH 1/2] workqueues: make get_online_cpus() useable for work->func()

[Heiko Carstens <heiko.carstens@de.ibm.com>]

On Sun, Jun 29, 2008 at 08:51:31PM +0400, Oleg Nesterov wrote:
> workqueue_cpu_callback(CPU_DEAD) flushes cwq->thread under
> cpu_maps_update_begin(). This means that the multithreaded workqueues can't
> use get_online_cpus() due to the possible deadlock, very bad and very old
> problem.
> 
> Introduce the new state, CPU_POST_DEAD, which is called after
> cpu_hotplug_done() but before cpu_maps_update_done().
> 
> Change workqueue_cpu_callback() to use CPU_POST_DEAD instead of CPU_DEAD.
> This means that create/destroy functions can't rely on get_online_cpus()
> any longer and should take cpu_add_remove_lock instead.

Ah, nice!

> --- 26-rc2/kernel/cpu.c~WQ_4_GET_ONLINE_CPUS	2008-05-18 15:44:18.000000000 +0400
> +++ 26-rc2/kernel/cpu.c	2008-06-29 20:03:19.000000000 +0400
> @@ -261,6 +261,11 @@ out_allowed:
>  	set_cpus_allowed_ptr(current, &old_allowed);
>  out_release:
>  	cpu_hotplug_done();
> +	if (!err) {

This should be (!err && !cpu_online(cpu)), no?

This is because it might be that __stop_machine_run() succeeded, but
take_cpu_down() failed and therefore our cpu is still online.

Which in turn is not good when doing this:

> +		if (raw_notifier_call_chain(&cpu_chain, CPU_POST_DEAD | mod,
> +					    hcpu) == NOTIFY_BAD)
> +			BUG();
> +	}