mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Christoph Lameter <clameter@sgi.com>
To: "Chen, Kenneth W" <kenneth.w.chen@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>,
	"Siddha, Suresh B" <suresh.b.siddha@intel.com>,
	akpm@osdl.org, mm-commits@vger.kernel.org,
	nickpiggin@yahoo.com.au, linux-kernel@vger.kernel.org
Subject: RE: + sched-use-tasklet-to-call-balancing.patch added to -mm tree
Date: Fri, 10 Nov 2006 10:50:13 -0800 (PST)	[thread overview]
Message-ID: <Pine.LNX.4.64.0611101027100.25459@schroedinger.engr.sgi.com> (raw)
In-Reply-To: <000001c70490$01cea4b0$8bc8180a@amr.corp.intel.com>

On Thu, 9 Nov 2006, Chen, Kenneth W wrote:

> All results are within noise range.  The global tasklet does a fairly 
> good job especially on context switch intensive workload like aim7, 
> volanomark, tbench etc.  Note all machines are non-numa platform.
 
> Base on the data, I think we should make the load balance tasklet one per numa
> node instead of one per CPU.

I have done some testing on NUMA with 8p / 4n and 256 p / 128n which seems 
to indicate that this is doing very well. Having a global tasklet avoids 
concurrent load balancing from multiple nodes which smoothes out the load 
balancing load over all nodes in a NUMA system. It fixes the issues that 
we saw with contention during concurrent load balancing.

On a 8p system:

I) Percent of ticks where load balancing was found to be required

II) Percent of ticks where we attempted load balancing
   but we found that we need to try again due to load balancing
   in progress elsewhere (This increases (I) since we found that
   load balancing was required but we decided to defer. Tasklet
   was not invoked).

    	I)	II) 
Boot:	70%	~1%
AIM7:	30%	2%
Idle:	50%	 <0.5%

256p:
	I)	II)
Boot:	80%	30%
AIM7:	90%	30%
Idle: 	95%	30%

So on larger systems we have more attempts to concurrently execute the 
tasklet which also inflates I). I would expect that rate to be raised 
further on even larger systems until we likely reach a point of continuous 
load balancing somehwere in the system at 1024p (will take some time to to 
such a system). It is likely prefereable at that scale to still not have 
concurrency. Concurrency may mean that we concurrently balance huge sched 
domains which causes long delays on multiple processor and may cause 
contention between those load balancing threads that attempt to move tasks 
from the same busy processor.

Suresh noted at the Intel OS forum yesterday that we may be able to avoid 
running load balancing of the larger sched domains from all processors. 
That would reduce the overhead of scheduling and reduce the percentage of 
time that load blancing is active even on very large systems.



  reply	other threads:[~2006-11-10 18:50 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <200611032205.kA3M5wmJ003178@shell0.pdx.osdl.net>
2006-11-07  7:32 ` Ingo Molnar
2006-11-07 17:44   ` Christoph Lameter
2006-11-07 17:31     ` Siddha, Suresh B
2006-11-07 17:55       ` Christoph Lameter
2006-11-07 17:50         ` Siddha, Suresh B
2006-11-07 18:18           ` Christoph Lameter
2006-11-07 19:17           ` Christoph Lameter
2006-11-07 20:31             ` Ingo Molnar
2006-11-07 20:59               ` Christoph Lameter
2006-11-07 21:35                 ` Chen, Kenneth W
2006-11-07 21:49                   ` Christoph Lameter
2006-11-10  6:18                     ` Chen, Kenneth W
2006-11-10 18:50                       ` Christoph Lameter [this message]
2006-11-10 21:38                         ` Ingo Molnar
2006-11-11  1:01                           ` Christoph Lameter
2006-11-11  2:05                             ` Chen, Kenneth W
2006-11-11  2:51                               ` Christoph Lameter
2006-11-13  4:03                                 ` Chen, Kenneth W
2006-11-13  5:44                                   ` Christoph Lameter
2006-11-13  6:40                                     ` Chen, Kenneth W
2006-11-14  1:06                                       ` [patch] sched domain: move sched group allocations to percpu area Siddha, Suresh B
2006-11-14  8:23                                         ` Ingo Molnar
2006-11-14 16:57                                         ` Christoph Lameter
2006-11-10 21:42                         ` + sched-use-tasklet-to-call-balancing.patch added to -mm tree Ingo Molnar
2006-11-10 22:32                         ` Siddha, Suresh B
2006-11-11  1:14                           ` Christoph Lameter
2006-11-07 19:22           ` Christoph Lameter
2006-11-07 20:29         ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Pine.LNX.4.64.0611101027100.25459@schroedinger.engr.sgi.com \
    --to=clameter@sgi.com \
    --cc=akpm@osdl.org \
    --cc=kenneth.w.chen@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=mm-commits@vger.kernel.org \
    --cc=nickpiggin@yahoo.com.au \
    --cc=suresh.b.siddha@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®