[PATCH 1/2] cpuset: fix cpuset_cpus_allowed_fallback() don't update tsk->rt.nr_cpus_allowed

[KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>]

The rule is, we have to update tsk->rt.nr_cpus_allowed too if we change
tsk->cpus_allowed. Otherwise RT scheduler may confuse.

This patch fixes it.

btw, system_state checking is very important. current boot sequence is (1) smp_init
(ie secondary cpus up and created cpu bound kthreads). (2) sched_init_smp().
Then following bad scenario can be happen,

(1) cpuup call notifier(CPU_UP_PREPARE)
(2) A cpu notifier consumer create FIFO kthread
(3) It call kthread_bind()
   ... but, now secondary cpu haven't ONLINE
(3) schedule() makes fallback and cpuset_cpus_allowed_fallback
    change task->cpus_allowed
(4) find_lowest_rq() touch local_cpu_mask if task->rt.nr_cpus_allowed != 1,
    but it haven't been initialized.

RCU folks plan to introduce such FIFO kthread and our testing hitted the
above issue. Then this patch also protect it.

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Ingo Molnar <mingo@elte.hu>
---
 include/linux/cpuset.h |    1 +
 kernel/cpuset.c        |    1 +
 kernel/sched.c         |    4 ++++
 3 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
index f20eb8f..42dcbdc 100644
--- a/include/linux/cpuset.h
+++ b/include/linux/cpuset.h
@@ -147,6 +147,7 @@ static inline void cpuset_cpus_allowed(struct task_struct *p,
 static inline int cpuset_cpus_allowed_fallback(struct task_struct *p)
 {
 	cpumask_copy(&p->cpus_allowed, cpu_possible_mask);
+	p->rt.nr_cpus_allowed = cpumask_weight(&p->cpus_allowed);
 	return cpumask_any(cpu_active_mask);
 }
 
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 1ceeb04..6e5bbe8 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -2220,6 +2220,7 @@ int cpuset_cpus_allowed_fallback(struct task_struct *tsk)
 		cpumask_copy(&tsk->cpus_allowed, cpu_possible_mask);
 		cpu = cpumask_any(cpu_active_mask);
 	}
+	tsk->rt.nr_cpus_allowed = cpumask_weight(&tsk->cpus_allowed);
 
 	return cpu;
 }
diff --git a/kernel/sched.c b/kernel/sched.c
index fd4625f..bfcd219 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -2352,6 +2352,10 @@ static int select_fallback_rq(int cpu, struct task_struct *p)
 	if (dest_cpu < nr_cpu_ids)
 		return dest_cpu;
 
+	/* Don't worry. It's temporary mismatch. */
+	if (system_state < SYSTEM_RUNNING)
+		return cpu;
+
 	/* No more Mr. Nice Guy. */
 	dest_cpu = cpuset_cpus_allowed_fallback(p);
 	/*
-- 
1.7.3.1