Re: [patchlet] sched: fix rt throttle runtime borrowing

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Tue, 2011-03-08 at 14:27 +0100, Mike Galbraith wrote:

> > Also, how much of a problem is it really? When I start a FIFO spinner on
> > my machine I can still ssh in and kill the thing.
> 
> It's a problem if you have one box.  Also, try starting a hefty load
> then having an rt task go nuts.  Nothing good happens here. 

Right, so I think we're not aggressive enough to migrate tasks away from
very small cpu_power CPUs, trapping tasks on such CPUs.

Of course, this is no help for pinned tasks.. but then you get what you
asked for isn't it ;-)

> > Not allowing 100% FIFO usage on SMP is going to make it very very hard
> > to implement any kind of fifo-cgroup stuff.
> 
> The only thing I care much about is the default setup.  The safety net
> should work, otherwise it's a waste.

Right, but how much trouble can be avoided by making the sched_fair
load-balancer migrate tasks away from very small cpu_power CPUs?

It won't avoid actual deadlocks when someone tries to wait for workqueue
broadcasts and the like, but how much of that is actually happening?

> Maybe only doing the borrow thing when there are active RT groups is the
> right thing to do.  (minus knob) 

Thing is the whole borrowing needs to go, Dario and me finally came up
with a 'sane' way to implement fifo-cgroups, but that does include
explicitly allowing starving CPUs.

Not allowing that very quickly degenerates into massive trouble like
gang-scheduling or bouncing tasks around like mad and generally messing
up the 'load-balancer'.