From: Steven Sistare <steven.sistare@oracle.com>
To: Mike Galbraith <efault@gmx.de>,
Rohit Jain <rohit.k.jain@oracle.com>,
linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, mingo@redhat.com, joelaf@google.com,
jbacik@fb.com, riel@redhat.com, juri.lelli@redhat.com,
dhaval.giani@oracle.com
Subject: Re: [RFC 1/2] sched: reduce migration cost between faster caches for idle_balance
Date: Thu, 15 Feb 2018 13:21:46 -0500 [thread overview]
Message-ID: <e9591680-4278-5344-6876-d16e8f9c53c4@oracle.com> (raw)
In-Reply-To: <1518718056.13961.23.camel@gmx.de>
On 2/15/2018 1:07 PM, Mike Galbraith wrote:
> On Thu, 2018-02-15 at 11:35 -0500, Steven Sistare wrote:
>> On 2/10/2018 1:37 AM, Mike Galbraith wrote:
>>> On Fri, 2018-02-09 at 11:08 -0500, Steven Sistare wrote:
>>>>>> @@ -8804,7 +8803,8 @@ static int idle_balance(struct rq *this_rq, struct rq_flags *rf)
>>>>>> if (!(sd->flags & SD_LOAD_BALANCE))
>>>>>> continue;
>>>>>>
>>>>>> - if (this_rq->avg_idle < curr_cost + sd->max_newidle_lb_cost) {
>>>>>> + if (this_rq->avg_idle < curr_cost + sd->max_newidle_lb_cost +
>>>>>> + sd->sched_migration_cost) {
>>>>>> update_next_balance(sd, &next_balance);
>>>>>> break;
>>>>>> }
>>>>>
>>>>> Ditto.
>>>>
>>>> The old code did not migrate if the expected costs exceeded the expected idle
>>>> time. The new code just adds the sd-specific penalty (essentially loss of cache
>>>> footprint) to the costs. The for_each_domain loop visit smallest to largest
>>>> sd's, hence visiting smallest to largest migration costs (though the tunables do
>>>> not enforce an ordering), and bails at the first sd where the total cost is a lose.
>>>
>>> Hrm..
>>>
>>> You're now adding a hypothetical cost to the measured cost of running
>>> the LB machinery, which implies that the measurement is insufficient,
>>> but you still don't say why it is insufficient. What happens if you
>>> don't do that? I ask, because when I removed the...
>>>
>>> this_rq->avg_idle < sysctl_sched_migration_cost
>>>
>>> ...bits to check removal effect for Peter, the original reason for it
>>> being added did not re-materialize, making me wonder why you need to
>>> make this cutoff more aggressive.
>>
>> The current code with sysctl_sched_migration_cost discourages migration
>> too much, per our test results.
>
> That's why I asked you what happens if you only whack the _apparently_
> (but maybe not) obsolete old throttle, it appeared likely that your win
> came from allowing a bit more migration than the simple throttle
> allowed, which if true, would obviate the need for anything more.
>
>> Can you provide more details on the sysbench oltp test that motivated you
>> to add sysctl_sched_migration_cost to idle_balance, so Rohit can re-test it?
>
> The problem at that time was the cycle overhead of entering that LB
> path at high frequency. Dirt simple.
I get that. I meant please provide details on test parameters and config if
you remember them.
- Steve
next prev parent reply other threads:[~2018-02-15 18:22 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-08 22:19 [RFC 0/2] sched: Make idle_balance smarter about topology Rohit Jain
2018-02-08 22:19 ` [RFC 1/2] sched: reduce migration cost between faster caches for idle_balance Rohit Jain
2018-02-09 3:42 ` Mike Galbraith
2018-02-09 16:08 ` Steven Sistare
2018-02-10 6:37 ` Mike Galbraith
2018-02-15 16:35 ` Steven Sistare
2018-02-15 18:07 ` Mike Galbraith
2018-02-15 18:21 ` Steven Sistare [this message]
2018-02-15 18:39 ` Mike Galbraith
2018-02-15 18:07 ` Rohit Jain
2018-02-16 4:53 ` Mike Galbraith
2018-02-08 22:19 ` [RFC 2/2] Introduce sysctl(s) for the migration costs Rohit Jain
2018-02-09 3:54 ` Mike Galbraith
2018-02-09 16:10 ` Steven Sistare
2018-02-09 17:08 ` Mike Galbraith
2018-02-09 17:33 ` Steven Sistare
2018-02-09 17:50 ` Mike Galbraith
2018-02-12 15:28 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e9591680-4278-5344-6876-d16e8f9c53c4@oracle.com \
--to=steven.sistare@oracle.com \
--cc=dhaval.giani@oracle.com \
--cc=efault@gmx.de \
--cc=jbacik@fb.com \
--cc=joelaf@google.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=rohit.k.jain@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®