Re: bisected: futex regression >= 3.14 - was - Slowdown due to threads bouncing between HT cores

[Mike Galbraith <umgwanakikbuti@gmail.com>]

On Sun, 2014-10-26 at 14:16 +0100, Mike Galbraith wrote: 
> On Sun, 2014-10-26 at 11:39 +0100, Steinar H. Gunderson wrote: 
> > On Fri, Oct 24, 2014 at 06:38:41PM +0200, Mike Galbraith wrote:
> > >>> Whew, good, futex.c is hard.  Heads up chess guys <punt>. 
> > >> I wonder whether the barrier fix which got into 3.17 late fixes that
> > >> issue as well.
> > > Yes, it did.
> > 
> > This is only about the lockup, right, not that the threads bounce around a
> > lot and make things slower?
> 
> Yes.  I couldn't reproduce your results, on my 28 core+ht box, there was
> little difference between pinned/unpinned.  Maybe the box was too quite,
> not enough random activity to motivate select_idle_sibling().

Can you try the below?

---
 kernel/sched/fair.c |    6 ++++++
 1 file changed, 6 insertions(+)

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4477,6 +4477,12 @@ static int select_idle_sibling(struct ta
 		return i;
 
 	/*
+	 * Don't bounce hefty tasks about, they may be hitting contention.
+	 */
+	if (p->se.load.weight - (p->se.load.weight/4) < p->se.avg.load_avg_contrib)
+		return target;
+
+	/*
 	 * Otherwise, iterate the domains and find an elegible idle cpu.
 	 */
 	sd = rcu_dereference(per_cpu(sd_llc, target));