Re: [PATCH 1/4] sched: consult online mask instead of active in select_fallback_rq()

[Peter Zijlstra <peterz@infradead.org>]

On Thu, 2010-05-13 at 12:48 +0200, Tejun Heo wrote:
> If called after sched_class chooses a CPU which isn't in a task's
> cpus_allowed mask, select_fallback_rq() can end up migrating a task
> which is bound to an !active but online cpu to an active cpu.  This is
> dangerous because active is cleared before CPU_DOWN_PREPARE is called
> and subsystems expect affinities of kthreads and other tasks to be
> maintained till their CPU_DOWN_PREPARE callbacks are complete.

So 6ad4c188 (sched: Fix balance vs hotplug race) moved it that early
because it was done too late.

Could we not instead do it explicitly after CPU_DOWN_PREPARE? It would
of course mean removing the partition_sched_domain() and
generate_sched_domains() calls from these callbacks and doing it
explicitly.

So we need to do it before we take the CPU down, but I think we can do
it after DOWN_PREPARE.

> Consult cpu_online_mask instead.
> 
> This problem is triggered by cmwq.  During CPU_DOWN_PREPARE, hotplug
> callback creates the trustee kthread and kthread_bind()s it to the
> target cpu, and the trustee is expected to run on that cpu.

It doesn't explain wth a trustee kthread is, which pretty much renders
the whole paragraph useless.