From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753948AbZJ2JO7 (ORCPT ); Thu, 29 Oct 2009 05:14:59 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753006AbZJ2JO6 (ORCPT ); Thu, 29 Oct 2009 05:14:58 -0400 Received: from mail.gmx.net ([213.165.64.20]:39271 "HELO mail.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1752728AbZJ2JO5 (ORCPT ); Thu, 29 Oct 2009 05:14:57 -0400 X-Authenticated: #14349625 X-Provags-ID: V01U2FsdGVkX1+xHU9MEH6x7fT9lKjNpVpuTSL93jp59gMlpEXJfl O3+vFTpE3e+6zo Subject: [patch] Re: hackbench regression with kernel 2.6.32-rc1 From: Mike Galbraith To: "Zhang, Yanmin" Cc: Peter Zijlstra , LKML , Ingo Molnar In-Reply-To: <1256797608.16282.28.camel@ymzhang> References: <1255079943.25078.23.camel@ymzhang> <1255084986.8802.46.camel@laptop> <1255331120.3684.43.camel@ymzhang> <1255357264.10420.15.camel@twins> <1255403522.3684.57.camel@ymzhang> <1255691192.7029.13.camel@marge.simson.net> <1256630584.16282.13.camel@ymzhang> <1256654547.17752.13.camel@marge.simson.net> <1256722141.16282.20.camel@ymzhang> <1256739779.7503.59.camel@marge.simson.net> <1256777448.16282.21.camel@ymzhang> <1256795190.7048.63.camel@marge.simson.net> <1256797608.16282.28.camel@ymzhang> Content-Type: text/plain; charset="UTF-8" Date: Thu, 29 Oct 2009 10:14:51 +0100 Message-Id: <1256807691.7158.54.camel@marge.simson.net> Mime-Version: 1.0 X-Mailer: Evolution 2.24.1.1 Content-Transfer-Encoding: 8bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.46 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2009-10-29 at 14:26 +0800, Zhang, Yanmin wrote: > On Thu, 2009-10-29 at 06:46 +0100, Mike Galbraith wrote: > > SD_PREFER_LOCAL is still on in rc1 though (double checks;), so you'll go > > through the power saving code until you reach a domain containing both > > waker's cpu and wakee's previous cpu even if that code already found > > that a higher domain wasn't overloaded. Looks to me like that block > > wants a want_sd && qualifier. > > > > Even it you turn SD_PREFER_LOCAL off, you can still hit the overhead if > > SD_POWERSAVINGS_BALANCE is set, so I'd make sure both are off and see if > > that's the source (likely, since the rest is already off). > Yes. SD_POWERSAVINGS_BALANCE is disabled by default. I applied Peter's patch which > turning SD_PREFER_LOCAL off for MC and cpu domain and it doesn't help. > perf counter shows select_task_rq_fair still consumes about 5% cpu time. Eventually, > I found for_each_cpu in for_each_domain consumes the 5% cpu time, because Peter's > patch doesn't turn off SD_PREFER_LOCAL for node domain. > I turned it off for node domain against the latest tips tree and tbench regression > disappears on a Nehalem machine and becomes about 2% on another one. > > Can we turn it off for node domain by default? If it's hurting fast path overhead to the tune of an order of magnitude, I guess there's no choice but to either fix it or turn it off. Since SD_BALANCE_WAKE is off globally, I don't see any point in keeping SD_PREFER_LOCAL at any level. (That said, what we need is for this to not be so expensive that we can't afford it in the fast path). sched: Disable SD_PREFER_LOCAL at node level. Yanmin Zhang reported that SD_PREFER_LOCAL induces an order of magnitude increase in select_task_rq_fair() overhead while running heavy wakeup benchmarks (tbench and vmark). Since SD_BALANCE_WAKE is off at node level, turn SD_PREFER_LOCAL off as well pending further investigation. Signed-off-by: Mike Galbraith Cc: Ingo Molnar Cc: Peter Zijlstra Reported-by: Zhang, Yanmin LKML-Reference: diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index d823c24..40e37b1 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -143,7 +143,7 @@ extern unsigned long node_remap_size[]; | 1*SD_BALANCE_FORK \ | 0*SD_BALANCE_WAKE \ | 1*SD_WAKE_AFFINE \ - | 1*SD_PREFER_LOCAL \ + | 0*SD_PREFER_LOCAL \ | 0*SD_SHARE_CPUPOWER \ | 0*SD_POWERSAVINGS_BALANCE \ | 0*SD_SHARE_PKG_RESOURCES \