From: Christoph Lameter <clameter@sgi.com>
To: "Chen, Kenneth W" <kenneth.w.chen@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>,
"Siddha, Suresh B" <suresh.b.siddha@intel.com>,
akpm@osdl.org, mm-commits@vger.kernel.org,
nickpiggin@yahoo.com.au, linux-kernel@vger.kernel.org
Subject: RE: + sched-use-tasklet-to-call-balancing.patch added to -mm tree
Date: Fri, 10 Nov 2006 10:50:13 -0800 (PST) [thread overview]
Message-ID: <Pine.LNX.4.64.0611101027100.25459@schroedinger.engr.sgi.com> (raw)
In-Reply-To: <000001c70490$01cea4b0$8bc8180a@amr.corp.intel.com>
On Thu, 9 Nov 2006, Chen, Kenneth W wrote:
> All results are within noise range. The global tasklet does a fairly
> good job especially on context switch intensive workload like aim7,
> volanomark, tbench etc. Note all machines are non-numa platform.
> Base on the data, I think we should make the load balance tasklet one per numa
> node instead of one per CPU.
I have done some testing on NUMA with 8p / 4n and 256 p / 128n which seems
to indicate that this is doing very well. Having a global tasklet avoids
concurrent load balancing from multiple nodes which smoothes out the load
balancing load over all nodes in a NUMA system. It fixes the issues that
we saw with contention during concurrent load balancing.
On a 8p system:
I) Percent of ticks where load balancing was found to be required
II) Percent of ticks where we attempted load balancing
but we found that we need to try again due to load balancing
in progress elsewhere (This increases (I) since we found that
load balancing was required but we decided to defer. Tasklet
was not invoked).
I) II)
Boot: 70% ~1%
AIM7: 30% 2%
Idle: 50% <0.5%
256p:
I) II)
Boot: 80% 30%
AIM7: 90% 30%
Idle: 95% 30%
So on larger systems we have more attempts to concurrently execute the
tasklet which also inflates I). I would expect that rate to be raised
further on even larger systems until we likely reach a point of continuous
load balancing somehwere in the system at 1024p (will take some time to to
such a system). It is likely prefereable at that scale to still not have
concurrency. Concurrency may mean that we concurrently balance huge sched
domains which causes long delays on multiple processor and may cause
contention between those load balancing threads that attempt to move tasks
from the same busy processor.
Suresh noted at the Intel OS forum yesterday that we may be able to avoid
running load balancing of the larger sched domains from all processors.
That would reduce the overhead of scheduling and reduce the percentage of
time that load blancing is active even on very large systems.
next prev parent reply other threads:[~2006-11-10 18:50 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <200611032205.kA3M5wmJ003178@shell0.pdx.osdl.net>
2006-11-07 7:32 ` Ingo Molnar
2006-11-07 17:44 ` Christoph Lameter
2006-11-07 17:31 ` Siddha, Suresh B
2006-11-07 17:55 ` Christoph Lameter
2006-11-07 17:50 ` Siddha, Suresh B
2006-11-07 18:18 ` Christoph Lameter
2006-11-07 19:17 ` Christoph Lameter
2006-11-07 20:31 ` Ingo Molnar
2006-11-07 20:59 ` Christoph Lameter
2006-11-07 21:35 ` Chen, Kenneth W
2006-11-07 21:49 ` Christoph Lameter
2006-11-10 6:18 ` Chen, Kenneth W
2006-11-10 18:50 ` Christoph Lameter [this message]
2006-11-10 21:38 ` Ingo Molnar
2006-11-11 1:01 ` Christoph Lameter
2006-11-11 2:05 ` Chen, Kenneth W
2006-11-11 2:51 ` Christoph Lameter
2006-11-13 4:03 ` Chen, Kenneth W
2006-11-13 5:44 ` Christoph Lameter
2006-11-13 6:40 ` Chen, Kenneth W
2006-11-14 1:06 ` [patch] sched domain: move sched group allocations to percpu area Siddha, Suresh B
2006-11-14 8:23 ` Ingo Molnar
2006-11-14 16:57 ` Christoph Lameter
2006-11-10 21:42 ` + sched-use-tasklet-to-call-balancing.patch added to -mm tree Ingo Molnar
2006-11-10 22:32 ` Siddha, Suresh B
2006-11-11 1:14 ` Christoph Lameter
2006-11-07 19:22 ` Christoph Lameter
2006-11-07 20:29 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.64.0611101027100.25459@schroedinger.engr.sgi.com \
--to=clameter@sgi.com \
--cc=akpm@osdl.org \
--cc=kenneth.w.chen@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=mm-commits@vger.kernel.org \
--cc=nickpiggin@yahoo.com.au \
--cc=suresh.b.siddha@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®