mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Salman Qazi <sqazi@google.com>
To: svaidy@linux.vnet.ibm.com
Cc: Arjan van de Ven <arjan@infradead.org>,
	linux-kernel@vger.kernel.org,
	linux-pm@lists.linux-foundation.org,
	Andrew Morton <akpm@linux-foundation.org>,
	Michael Rubin <mrubin@google.com>,
	Taliver Heath <taliver@google.com>
Subject: Re: RFC: A proposal for power capping through forced idle in the  Linux Kernel
Date: Tue, 15 Dec 2009 13:00:21 -0800	[thread overview]
Message-ID: <4352991a0912151300x142f9233ua75c2264f9e0bc79@mail.gmail.com> (raw)
In-Reply-To: <20091215115052.GB878@dirshya.in.ibm.com>

On Tue, Dec 15, 2009 at 3:50 AM, Vaidyanathan Srinivasan
<svaidy@linux.vnet.ibm.com> wrote:
> * Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com> [2009-12-15 15:59:09]:
>
>> * Salman Qazi <sqazi@google.com> [2009-12-14 16:36:20]:
>>
>> > On Mon, Dec 14, 2009 at 4:19 PM, Arjan van de Ven <arjan@infradead.org> wrote:
>> > > On Mon, 14 Dec 2009 15:11:47 -0800
>> > > Salman Qazi <sqazi@google.com> wrote:
>> > >
>> > >
>> > > I like the general idea, I have one request (that I didn't see quite in
>> > > your explanation): Please make sure that all cpus in the system do
>> > > their idle injection at the same time, so that memory can go into power
>> > > saving mode as well during this time etc etc...
>> > >
>>
>> The value of the overall idea is well understood but the
>> implementation and benefits in terms of power savings was the major
>> point of discussion earlier.
>>
>> > With the current interface, the forced idle percentages on the CPUs
>> > are controlled independently.  There's a trade-off here.  If we inject
>> > idle cycles on all the CPU at the same time, our machine
>> > responsiveness also degrades: essentially every CPU becomes equally
>> > bad for an interactive task to run on.  Our aim at the moment is to
>> > try to concentrate the idle cycles on a small set of CPUs, to strive
>> > to leave some CPUs where interactive tasks can run unhindered.  But,
>> > given a different workload and goals the correct policy may be
>> > different.
>> >
>> > Simultaneously idling multiple "cores" becomes necessary in the SMT
>> > case: as there is no point in idling a single thread, while the other
>> > thread is running full tilt.  So, in such a case it is necessary to
>> > idle all the threads making up the physical core.  This feature has
>> > not been implemented yet.
>> >
>> > I think the best approach may be to provide a way to specify the
>> > policy from the user space.  Basically let the user decide at what
>> > level of CPU hierarchy the forced idle percentages are specified.
>> > Then, in the levels below, we simply inject at the same time.
>>
>> Synchronising the idle times across multiple cores and also selecting
>> sibling threads belonging to the same core is important.  The current
>> ACPI forced idle driver can inject idle time but not synchronized
>> across multiple cores.
>>
>> Allowing the scheduler load balancer to avoid using a part of the
>> sched domain tree will allow easy grouping of sibling threads and
>> sibling cores if that saves more power.
>>
>> However as Arjan mentioned, new architectures have significant power
>> savings at full system idle where memory power is reduced.  Injecting
>> idle time in any of the core will actually increase the utilisation on
>> the other cores (unless the system is full loaded) and reduce the full
>> system idle time opportunity.  Basically injecting idle time on some
>> of the cores in the system goes against the race-to-idle policy
>> thereby decreasing overall system operating efficiency.
>>
>> Can you please clarify the following questions:
>>
>> * What is the typical duration of idle time injected?
>>         - 10s of milli seconds?  CPUs are expected to goto lowest
>>           power idle state within this time?
>>
>> * You mentioned that natural idle time in the system is taken into
>>   account before injecting forced idle time, which is a good feature
>>   to have.
>>         - In most workloads, as the utilisation drops, all the cpus
>>           have similar idle times.  This is favourable for exploiting
>>           memory power saving.
>>         - Now when more idle time need to be inserted, is it
>>           uniformly spread across all CPUs?
>
> * How is the fairness issue in the scheduler handled?  Inserting idle
>  time may affect interactivity and fairness badly.

As mentioned in the design, we have two features to make this work.
First, we have "Eager Injection" phase, where we do not let any batch
tasks run but permit interactive tasks to run.  This phase lasts until
we are either sure that we have enough idle cycles (in which case
everyone is free to run) or we are sure that we have to spend the rest
of the interval injecting.  This latter scenario is called the lazy
injection phase.

Second, we have "power capping priority", a per-cgroup value which
determines the order in which the "blame" is assigned for the injected
cycles.  For the purposes of scheduling decisions, we pretend that the
lowest power capping priority job was running when we were injecting
idle cycles.  If the lowest priority job did not deserve sufficient
run time in the period in question, then we move to the next higher
priority job and so on.  Thus, we penalize the jobs in the power
capping priority order for the time spent injecting idle cycles.  This
allows us to make sure that important jobs get to use the available
power, and the less important jobs are the first to suffer.

>
> --Vaidy
>

  reply	other threads:[~2009-12-15 21:00 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-12-14 23:11 Salman Qazi
2009-12-14 23:21 ` Andi Kleen
2009-12-14 23:51   ` tytso
2009-12-15  0:42     ` Salman Qazi
2009-12-22 19:48     ` Peter Zijlstra
2009-12-15  0:19 ` Arjan van de Ven
2009-12-15  0:36   ` Salman Qazi
2009-12-15  1:06     ` Arjan van de Ven
2009-12-15 20:15       ` Salman Qazi
2009-12-17 11:01         ` Arjan van de Ven
2009-12-15 10:29     ` Vaidyanathan Srinivasan
2009-12-15 11:50       ` Vaidyanathan Srinivasan
2009-12-15 21:00         ` Salman Qazi [this message]
2009-12-15 20:50       ` Salman Qazi
2009-12-22 19:48   ` Peter Zijlstra
2009-12-22 19:57     ` Arjan van de Ven
2009-12-18 17:04 ` Pavel Machek
2009-12-22 21:10   ` Salman Qazi
2009-12-23  9:49     ` Pavel Machek
2009-12-21  8:57 ` Pavel Machek
2009-12-22 21:15   ` Salman Qazi
2009-12-23  9:52     ` Pavel Machek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4352991a0912151300x142f9233ua75c2264f9e0bc79@mail.gmail.com \
    --to=sqazi@google.com \
    --cc=akpm@linux-foundation.org \
    --cc=arjan@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pm@lists.linux-foundation.org \
    --cc=mrubin@google.com \
    --cc=svaidy@linux.vnet.ibm.com \
    --cc=taliver@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®