Re: [PATCH v2] keep rd->online and cpu_online_map in sync

[Gautham R Shenoy <ego@in.ibm.com>]

On Mon, Mar 10, 2008 at 06:34:56PM -0700, Suresh Siddha wrote:
> On Mon, Mar 10, 2008 at 04:36:13PM -0700, Andrew Morton wrote:
> > > --- a/kernel/sched.c
> > > +++ b/kernel/sched.c
> > > @@ -5881,7 +5881,8 @@ migration_call(struct notifier_block *nfb, unsigned long action, void *hcpu)
> > >  		spin_unlock_irq(&rq->lock);
> > >  		break;
> > >  
> > > -	case CPU_DOWN_PREPARE:
> > > +	case CPU_DYING:
> > > +	case CPU_DYING_FROZEN:
> > >  		/* Update our root-domain */
> > >  		rq = cpu_rq(cpu);
> > >  		spin_lock_irqsave(&rq->lock, flags);
> > 
> > Does this make
> > cpu-hotplug-register-update_sched_domains-notifier-with-higher-prio.patch
> > (below) obsolete,
> 
> Yes. I would like to Ack for the
> keep-rd-online-and-cpu_online_map-in-sync.patch, as it is more cleaner.
> 
> 
> > or do we want both?  
> 
> No. I don't think so. Gautham, do you agree?

Yup I agree! Greg's patch is better since i clears the
bit *just* before we take the cpu down. That way we won't mess with the 
ordering. 

But I would still prefer that we update the sched_domains early on 
after cpu-hotplug rather than doing it after the subsystems have run 
their notifiers.

Will send a seperate patch for that one.
> 
> thanks,
> suresh
> 
> > 
> > --- a/kernel/sched.c~cpu-hotplug-register-update_sched_domains-notifier-with-higher-prio
> > +++ a/kernel/sched.c
> > @@ -7096,8 +7096,16 @@ void __init sched_init_smp(void)
> >  	if (cpus_empty(non_isolated_cpus))
> >  		cpu_set(smp_processor_id(), non_isolated_cpus);
> >  	put_online_cpus();
> > -	/* XXX: Theoretical race here - CPU may be hotplugged now */
> > -	hotcpu_notifier(update_sched_domains, 0);
> > +	/*
> > +	 * XXX: Theoretical race here - CPU may be hotplugged now
> > +	 *
> > +	 * We register the notifier with priority 11, which means that
> > +	 * update_sched_domains() will be called just before migration_call().
> > +	 *
> > +	 * This is necessary to ensure that the rt wake up logic works fine
> > +	 * and the rq->rd->online_map remains in sync with the cpu_online_map.
> > +	 */
> > +	hotcpu_notifier(update_sched_domains, 11);
> >  
> >  	/* Move init over to a non-isolated CPU */
> >  	if (set_cpus_allowed(current, non_isolated_cpus) < 0)
> > _
> > 

-- 
Thanks and Regards
gautham