Re: [RFC PATCH] sched: Pass affine target cpu into wake_affine

[Mike Galbraith <efault@gmx.de>]

On Tue, 2010-01-05 at 04:44 +0100, Mike Galbraith wrote:
> On Tue, 2010-01-05 at 10:48 +0800, Lin Ming wrote:
> > On Mon, 2010-01-04 at 17:03 +0800, Lin Ming wrote:
> > > commit a03ecf08d7bbdd979d81163ea13d194fe21ad339
> > > Author: Lin Ming <ming.m.lin@intel.com>
> > > Date:   Mon Jan 4 14:14:50 2010 +0800
> > > 
> > >     sched: Pass affine target cpu into wake_affine
> > >     
> > >     Since commit a1f84a3(sched: Check for an idle shared cache in select_task_rq_fair()),
> > >     the affine target maybe adjusted to any idle cpu in cache sharing domains
> > >     instead of current cpu.
> > >     But wake_affine still use current cpu to calculate load which is wrong.
> > >     
> > >     This patch passes affine cpu into wake_affine.
> > >     
> > >     Signed-off-by: Lin Ming <ming.m.lin@intel.com>
> > 
> > Mike,
> > 
> > Any comment of this patch?
> 
> The patch definitely looks like the right thing to do, but when I tried
> this, it didn't work out well.  Since I can't seem to recall precise
> details, I'll let my box either remind me or give it's ack.

Unfortunately, box reminded me.  mysql+oltp peak throughput with
nr_clients == nr_cpus

tip   37012.34
tip+  33025.83
          .892

We really only want to check for shared cache on ramp-up and/or longish
intermission.  Once there's enough work to go around, interleaving is a
big problem for these synchronous tasks.  Doing the silly thing gets us
the ramp-up gain without too much pain, though there is definitely pain
for very fast switchers.

Looking always costs you a cache miss, not looking costs you throughput
on ramp/intermission.  Damned if you do, damned if you don't.

	-Mike