From: "tip-bot2 for Josh Don" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Josh Don <joshdon@google.com>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: sched/core] sched: Allow newidle balancing to bail out of load_balance
Date: Mon, 13 Jun 2022 08:43:24 -0000 [thread overview]
Message-ID: <165510980461.4207.16837948600111903560.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20220609025515.2086253-1-joshdon@google.com>
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 792b9f65a568f48c50b3175536db9cde5a1edcc0
Gitweb: https://git.kernel.org/tip/792b9f65a568f48c50b3175536db9cde5a1edcc0
Author: Josh Don <joshdon@google.com>
AuthorDate: Wed, 08 Jun 2022 19:55:15 -07:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Mon, 13 Jun 2022 10:30:01 +02:00
sched: Allow newidle balancing to bail out of load_balance
While doing newidle load balancing, it is possible for new tasks to
arrive, such as with pending wakeups. newidle_balance() already accounts
for this by exiting the sched_domain load_balance() iteration if it
detects these cases. This is very important for minimizing wakeup
latency.
However, if we are already in load_balance(), we may stay there for a
while before returning back to newidle_balance(). This is most
exacerbated if we enter a 'goto redo' loop in the LBF_ALL_PINNED case. A
very straightforward workaround to this is to adjust should_we_balance()
to bail out if we're doing a CPU_NEWLY_IDLE balance and new tasks are
detected.
This was tested with the following reproduction:
- two threads that take turns sleeping and waking each other up are
affined to two cores
- a large number of threads with 100% utilization are pinned to all
other cores
Without this patch, wakeup latency was ~120us for the pair of threads,
almost entirely spent in load_balance(). With this patch, wakeup latency
is ~6us.
Signed-off-by: Josh Don <joshdon@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20220609025515.2086253-1-joshdon@google.com
---
kernel/sched/fair.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7d8ef01..8bed757 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9824,9 +9824,15 @@ static int should_we_balance(struct lb_env *env)
/*
* In the newly idle case, we will allow all the CPUs
* to do the newly idle load balance.
+ *
+ * However, we bail out if we already have tasks or a wakeup pending,
+ * to optimize wakeup latency.
*/
- if (env->idle == CPU_NEWLY_IDLE)
+ if (env->idle == CPU_NEWLY_IDLE) {
+ if (env->dst_rq->nr_running > 0 || env->dst_rq->ttwu_pending)
+ return 0;
return 1;
+ }
/* Try to find first idle CPU */
for_each_cpu_and(cpu, group_balance_mask(sg), env->cpus) {
prev parent reply other threads:[~2022-06-13 8:43 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-09 2:55 [PATCH] sched: allow " Josh Don
2022-06-09 13:41 ` Vincent Guittot
2022-06-09 19:40 ` Josh Don
2022-06-10 8:10 ` Vincent Guittot
2022-06-10 18:46 ` Josh Don
2022-06-13 8:43 ` tip-bot2 for Josh Don [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=165510980461.4207.16837948600111903560.tip-bot2@tip-bot2 \
--to=tip-bot2@linutronix.de \
--cc=joshdon@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®