From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753019AbbC3MC2 (ORCPT ); Mon, 30 Mar 2015 08:02:28 -0400 Received: from foss.arm.com ([217.140.101.70]:60749 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752945AbbC3MCZ (ORCPT ); Mon, 30 Mar 2015 08:02:25 -0400 Date: Mon, 30 Mar 2015 13:03:03 +0100 From: Morten Rasmussen To: Peter Zijlstra Cc: Preeti U Murthy , "mingo@kernel.org" , "riel@redhat.com" , "daniel.lezcano@linaro.org" , "vincent.guittot@linaro.org" , "srikar@linux.vnet.ibm.com" , "pjt@google.com" , "benh@kernel.crashing.org" , "efault@gmx.de" , "linux-kernel@vger.kernel.org" , "iamjoonsoo.kim@lge.com" , "svaidy@linux.vnet.ibm.com" , "tim.c.chen@linux.intel.com" , "jason.low2@hp.com" Subject: Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs Message-ID: <20150330120302.GT18994@e105550-lin.cambridge.arm.com> References: <20150326130014.21532.17158.stgit@preeti.in.ibm.com> <20150327143839.GO18994@e105550-lin.cambridge.arm.com> <55158966.4050300@linux.vnet.ibm.com> <20150327175651.GR18994@e105550-lin.cambridge.arm.com> <20150330110632.GT23123@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150330110632.GT23123@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: > On Fri, Mar 27, 2015 at 05:56:51PM +0000, Morten Rasmussen wrote: > > > I agree that it is hard to predict how many additional cpus you need, > > but I don't think you necessarily need that information as long as you > > start by filling up the cpu that was kicked to do the > > nohz_idle_balance() first. > > > Reducing unnecessary wakeups is quite important for energy consumption > > and something a lot of effort is put into. You really don't want to wake > > up another cluster/package unnecessarily just because there was only one > > nohz-idle cpu left in the previous one which could have handled the > > additional load. It gets even worse if the other cluster is less > > energy-efficient (big.LITTLE). > > So the only way to get tasks to cross your cluster is by balancing that > domain. At this point we'll compute sg stats for either group > (=cluster). > > The only thing we need to ensure is that it doesn't view the small > cluster as overloaded (as long as it really isn't of course), as long as > its not viewed as overloaded it will not pull tasks from it into the big > cluster, no matter how many ILBs we run before the ILB duty cpu's > rebalance_domains() call. > > I'm really not seeing the problem here. I see. The group_classify() should take care of it in all cases of balancing across clusters. You would be iterating over all cpus in the other cluster running rebalance_domains() if the balancer cpu happens to be the last one in the little cluster though. However, within the cluster (in case you have 2 or more nohz-idle cpus) you still take a double hit. No?