Re: current linux-2.6.git: cpusets completely broken

["Dmitry Adamushko" <dmitry.adamushko@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2877 bytes --]

2008/7/11 Vegard Nossum <vegard.nossum@gmail.com>:
> On Fri, Jul 11, 2008 at 9:36 PM, Paul Menage <menage@google.com> wrote:
>> On Fri, Jul 11, 2008 at 12:07 PM, Vegard Nossum <vegard.nossum@gmail.com> wrote:
>>>
>>> The result of having CPUSETS enabled as above is a 100% reproducible
>>> BUG on the very first cpu hot-unplug:
>>>
>>> ------------[ cut here ]------------
>>> kernel BUG at xxx/linux-2.6/kernel/sched.c:5859!
>>
>> That doesn't quite match up with any BUG in 2.6.26-rc9 - what tree is
>> this last crash based on?
>
> latest mainline. Commit e5a5816f7875207cb0a0a7032e39a4686c5e10a4.
>
> Is this one:
>
> /* called under rq->lock with disabled interrupts */
> static void migrate_dead(unsigned int dead_cpu, struct task_struct *p)
> {
>        struct rq *rq = cpu_rq(dead_cpu);
>
>        /* Must be exiting, otherwise would be on tasklist. */
>        BUG_ON(!p->exit_state);
>
>>> Also, this is on the latest linux-2.6.git! Since we're so close to
>>> release, maybe cpusets should simply be marked BROKEN for now? (Unless
>>> we can fix it, of course. The alternative is to apply Miao Xie's
>>> workaround patch temporarily.)
>>
>> If we were going to mark anything as broken, wouldn't cpu-hotplug be
>> the more appropriate victim? I suspect that there are more systems
>> using cpusets in production environments than using cpu hotplug. But
>> as you say, fixing it sounds better.
>
> I'm sorry for the harsh characterization and suggestion; please accept
> my apology. It was purely a result of my excitement at having made
> some progress in this case.
>
> But I have more good news; reverting this:
>
> commit f18f982abf183e91f435990d337164c7a43d1e6d
> Author: Max Krasnyansky <maxk@qualcomm.com>
> Date:   Thu May 29 11:17:01 2008 -0700
>
>    sched: CPU hotplug events must not destroy scheduler domains created by the
> cpusets

Does the patch below help?

(non-white-space-damaged version is attached)

diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 9fceb97..ae61dc9 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -1912,11 +1912,21 @@ static void common_cpu_mem_hotplug_unplug(void)
 static int cpuset_handle_cpuhp(struct notifier_block *unused_nb,
                                unsigned long phase, void *unused_cpu)
 {
-       if (phase == CPU_DYING || phase == CPU_DYING_FROZEN)
+       swicth (phase) {
+       case CPU_UP_CANCELED:
+       case CPU_UP_CANCELED_FROZEN:
+       case CPU_DOWN_FAILED:
+       case CPU_DOWN_FAILED_FROZEN:
+       case CPU_ONLINE:
+       case CPU_ONLINE_FROZEN:
+       case CPU_DEAD:
+       case CPU_DEAD_FROZEN:
+               common_cpu_mem_hotplug_unplug();
+               break;
+       default:
                return NOTIFY_DONE;

-       common_cpu_mem_hotplug_unplug();
-       return 0;
+       return NOTIFY_OK;
 }

 #ifdef CONFIG_MEMORY_HOTPLU



-- 
Best regards,
Dmitry Adamushko

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 003-fix-cpuset.patch --]
[-- Type: text/x-diff; name=003-fix-cpuset.patch, Size: 760 bytes --]

diff --git a/kernel/cpuset.c b/kernel/cpuset.c
index 9fceb97..ae61dc9 100644
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -1912,11 +1912,21 @@ static void common_cpu_mem_hotplug_unplug(void)
 static int cpuset_handle_cpuhp(struct notifier_block *unused_nb,
 				unsigned long phase, void *unused_cpu)
 {
-	if (phase == CPU_DYING || phase == CPU_DYING_FROZEN)
+	swicth (phase) {
+	case CPU_UP_CANCELED:
+	case CPU_UP_CANCELED_FROZEN:
+	case CPU_DOWN_FAILED:
+	case CPU_DOWN_FAILED_FROZEN:
+	case CPU_ONLINE:
+	case CPU_ONLINE_FROZEN:
+	case CPU_DEAD:
+	case CPU_DEAD_FROZEN:
+		common_cpu_mem_hotplug_unplug();
+		break;
+	default:
 		return NOTIFY_DONE;
 
-	common_cpu_mem_hotplug_unplug();
-	return 0;
+	return NOTIFY_OK;
 }
 
 #ifdef CONFIG_MEMORY_HOTPLUG