Re: sched_mc_power_savings broken with CGROUPS+CPUSETS

[Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>]

* Peter Zijlstra <peterz@infradead.org> [2008-08-29 15:23:57]:

> On Fri, 2008-08-29 at 18:45 +0530, Vaidyanathan Srinivasan wrote:
> > Hi,
> > 
> > sched_mc_power_savings seems to be broken with CGROUPS+CPUSETS.
> > When CONFIG_CPUSETS=y the attached BUG_ON() is being hit.
> > 
> > I added a BUG_ON to check if SD_POWERSAVINGS_BALANCE is set at
> > SD_LV_CPU whenever sched_mc_power_savings is set.
> > 
> > This BUG is hit when config CONFIG_CPUSETS (depends on CONFIG_CGROUPS)
> > is just compiled in while this is never hit when they are compiled
> > out.  The fact that SD_POWERSAVINGS_BALANCE being cleared even when
> > sched_mc_power_savings = 1 completely breaks the
> > sched_mc_power_savings heuristics.
> > 
> > To recreate the problem, 
> > Have sched_mc power savings enabled CONFIG_SCHED_MC=y
> > Add this BUG_ON()
> > 
> > echo 1 > /sys/devices/system/cpu/sched_mc_power_savings
> > 
> > Try these these on a multi core x86 box.
> > 
> > sched_mc_power_savings seems to be broken from 2.6.26-rc1, but
> > I do not have a confirmation that the root cause is same in all
> > successive versions. sched_mc_power_savings works perfect in 
> > 2.6.25.
> > 
> > Please help me root cause the issue.  Please point me to changes that
> > may potential cause this bug.
> 
> I'm still greatly mistified by all that power savings code.
> 
> Its hard to read and utterly hard to comprehend - I've been about to rip
> the whole stuff out on several occasions. But so far tried to carefully
> thread around it maintaining its operation even though not fully
> understood.
> 
> Someone with clue - preferably the authors of the code in question -
> should enlighten us with a patch that adds some comments as to the
> intent of said lines of code.

I will try to add my notes around it.  I have been playing around with
sched_mc to make it work for different workloads.

--Vaidy