Re: [PATCH] sched: make update_sd_pick_busiest return true on a busier sd

[Michael Neuling <mikey@neuling.org>]

On Fri, 2014-07-25 at 17:27 +0200, Peter Zijlstra wrote:
> On Tue, Jul 22, 2014 at 02:45:59PM -0400, Rik van Riel wrote:
> > Currently update_sd_pick_busiest only returns true when an sd
> > is overloaded, or for SD_ASYM_PACKING when a domain is busier
> > than average and a higher numbered domain than the target.
> > 
> > This breaks load balancing between domains that are not overloaded,
> > in the !SD_ASYM_PACKING case. This patch makes update_sd_pick_busiest
> > return true when the busiest sd yet is encountered.
> > 
> > On a 4 node system, this seems to result in the load balancer finally
> > putting 1 thread of a 4 thread test run of "perf bench numa mem" on
> > each node, where before the load was generally not spread across all
> > nodes.
> 
> So for !ASYM the code is effectively:
> 
> 	return sgs->avg_load > sds->busiest_stat.avg_load;
> 
> I'd like to at least add a clause that makes overloaded groups
> prioritized over !overloaded groups.
> 
> Also, like we found earlier, calculate_imbalance() relies on the
> sum_nr_running > group_capacity_factor thing, which you've just
> 'wrecked', so we'd need an update to that part too.
> 
> > Behaviour for SD_ASYM_PACKING does not seem to match the comment,
> > in that groups with below average load average are ignored, but I
> > have no hardware to test that so I have left the behaviour of that
> > code unchanged.
> 
> Mikey, does that stuff work as expected?

Sorry for the slow response.  v1 and v2 both pass testing on POWER7.  So
FWIW...

Acked-By: Michael Neuling <mikey@neuling.org>

Mikey