Re: sched_mc_power_savings broken with CGROUPS+CPUSETS

[Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>]

* Max Krasnyansky <maxk@qualcomm.com> [2008-08-29 13:17:02]:

> Vaidyanathan Srinivasan wrote:
> > Hi,
> > 
> > sched_mc_power_savings seems to be broken with CGROUPS+CPUSETS.
> > When CONFIG_CPUSETS=y the attached BUG_ON() is being hit.
> > 
> > I added a BUG_ON to check if SD_POWERSAVINGS_BALANCE is set at
> > SD_LV_CPU whenever sched_mc_power_savings is set.
> > 
> > This BUG is hit when config CONFIG_CPUSETS (depends on CONFIG_CGROUPS)
> > is just compiled in while this is never hit when they are compiled
> > out.  The fact that SD_POWERSAVINGS_BALANCE being cleared even when
> > sched_mc_power_savings = 1 completely breaks the
> > sched_mc_power_savings heuristics.
> > 
> > To recreate the problem,
> > Have sched_mc power savings enabled CONFIG_SCHED_MC=y
> > Add this BUG_ON()
> > 
> > echo 1 > /sys/devices/system/cpu/sched_mc_power_savings
> > 
> > Try these these on a multi core x86 box.
> > 
> > sched_mc_power_savings seems to be broken from 2.6.26-rc1, but
> > I do not have a confirmation that the root cause is same in all
> > successive versions. sched_mc_power_savings works perfect in
> > 2.6.25.
> > 
> > Please help me root cause the issue.  Please point me to changes that
> > may potential cause this bug.
> 
> That's my fault. I redid domain rebuild/hotplug handling awhile ago and missed
> the fact that partition_sched_domains() is trying to avoid unnecessary domain
> rebuilds. Primary issue at the time was circular locking issues and the the
> testing that I was doing then was exercising a bunch of different scenarios
> (ie cpu up/down, cpuset create/destroy, mcpowersave on/off) and I did not
> notice that mcpowersave did not actually trigger domain rebuilds.
> 
> Anyway, I sent you guys a patch that should fix this issue.
> Please confirm.

Hi Max,

Thank you very much for identifying the source of the problem.  I will
test your patch and report.

My BUG_ON() may still fail during the window of change.  I will watch
out for that.

--Vaidy