From: Peter Zijlstra <peterz@infradead.org>
To: tglx@linutronix.de, mingo@kernel.org
Cc: linux-kernel@vger.kernel.org, bigeasy@linutronix.de,
qais.yousef@arm.com, swood@redhat.com, peterz@infradead.org,
valentin.schneider@arm.com, juri.lelli@redhat.com,
vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
bristot@redhat.com, vincent.donnefort@arm.com, tj@kernel.org
Subject: [PATCH -v2 07/17] sched: Fix hotplug vs CPU bandwidth control
Date: Mon, 05 Oct 2020 16:57:24 +0200 [thread overview]
Message-ID: <20201005150921.661842442@infradead.org> (raw)
In-Reply-To: <20201005145717.346020688@infradead.org>
Since we now migrate tasks away before DYING, we should also move
bandwidth unthrottle, otherwise we can gain tasks from unthrottle
after we expect all tasks to be gone already.
Also; it looks like the RT balancers don't respect cpu_active() and
instead rely on rq->online in part, complete this. This too requires
we do set_rq_offline() earlier to match the cpu_active() semantics.
(The bigger patch is to convert RT to cpu_active() entirely)
Since set_rq_online() is called from sched_cpu_activate(), place
set_rq_offline() in sched_cpu_deactivate().
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/sched/core.c | 14 ++++++++++----
kernel/sched/deadline.c | 5 +----
kernel/sched/rt.c | 5 +----
3 files changed, 12 insertions(+), 12 deletions(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6979,6 +6979,8 @@ int sched_cpu_activate(unsigned int cpu)
int sched_cpu_deactivate(unsigned int cpu)
{
+ struct rq *rq = cpu_rq(cpu);
+ struct rq_flags rf;
int ret;
set_cpu_active(cpu, false);
@@ -6993,6 +6995,14 @@ int sched_cpu_deactivate(unsigned int cp
balance_push_set(cpu, true);
+ rq_lock_irqsave(rq, &rf);
+ if (rq->rd) {
+ update_rq_clock();
+ BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span));
+ set_rq_offline(rq);
+ }
+ rq_unlock_irqrestore(rq, &rf);
+
#ifdef CONFIG_SCHED_SMT
/*
* When going down, decrement the number of cores with SMT present.
@@ -7074,10 +7084,6 @@ int sched_cpu_dying(unsigned int cpu)
sched_tick_stop(cpu);
rq_lock_irqsave(rq, &rf);
- if (rq->rd) {
- BUG_ON(!cpumask_test_cpu(cpu, rq->rd->span));
- set_rq_offline(rq);
- }
BUG_ON(rq->nr_running != 1);
rq_unlock_irqrestore(rq, &rf);
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -543,7 +543,7 @@ static int push_dl_task(struct rq *rq);
static inline bool need_pull_dl_task(struct rq *rq, struct task_struct *prev)
{
- return dl_task(prev);
+ return rq->online && dl_task(prev);
}
static DEFINE_PER_CPU(struct callback_head, dl_push_head);
@@ -2326,9 +2326,6 @@ static void rq_online_dl(struct rq *rq)
/* Assumes rq->lock is held */
static void rq_offline_dl(struct rq *rq)
{
- if (rq->dl.overloaded)
- dl_clear_overload(rq);
-
cpudl_clear(&rq->rd->cpudl, rq->cpu);
cpudl_clear_freecpu(&rq->rd->cpudl, rq->cpu);
}
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -265,7 +265,7 @@ static void pull_rt_task(struct rq *this
static inline bool need_pull_rt_task(struct rq *rq, struct task_struct *prev)
{
/* Try to pull RT tasks here if we lower this rq's prio */
- return rq->rt.highest_prio.curr > prev->prio;
+ return rq->online && rq->rt.highest_prio.curr > prev->prio;
}
static inline int rt_overloaded(struct rq *rq)
@@ -2245,9 +2245,6 @@ static void rq_online_rt(struct rq *rq)
/* Assumes rq->lock is held */
static void rq_offline_rt(struct rq *rq)
{
- if (rq->rt.overloaded)
- rt_clear_overload(rq);
-
__disable_runtime(rq);
cpupri_set(&rq->rd->cpupri, rq->cpu, CPUPRI_INVALID);
next prev parent reply other threads:[~2020-10-05 15:10 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-05 14:57 [PATCH -v2 00/17] sched: Migrate disable support Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 01/17] stop_machine: Add function and caller debug info Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 02/17] sched: Fix balance_callback() Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 03/17] sched/hotplug: Ensure only per-cpu kthreads run during hotplug Peter Zijlstra
2020-10-06 7:25 ` Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 04/17] sched/core: Wait for tasks being pushed away on hotplug Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 05/17] workqueue: Manually break affinity " Peter Zijlstra
2020-10-05 16:31 ` Tejun Heo
2020-10-05 14:57 ` [PATCH -v2 06/17] sched/hotplug: Consolidate task migration on CPU unplug Peter Zijlstra
2020-10-05 14:57 ` Peter Zijlstra [this message]
2020-10-06 12:46 ` [PATCH -v2 07/17] sched: Fix hotplug vs CPU bandwidth control Vincent Guittot
2020-10-06 13:33 ` Peter Zijlstra
2020-10-09 20:41 ` Dietmar Eggemann
2020-10-12 12:52 ` Peter Zijlstra
2020-10-12 13:18 ` Peter Zijlstra
2020-10-12 14:14 ` Dietmar Eggemann
2020-10-05 14:57 ` [PATCH -v2 08/17] sched: Massage set_cpus_allowed() Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 09/17] sched: Add migrate_disable() Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 10/17] sched: Fix migrate_disable() vs set_cpus_allowed_ptr() Peter Zijlstra
[not found] ` <CH2PR14MB41833F828B4D3BA5A7B6CE7B9A0B0@CH2PR14MB4183.namprd14.prod.outlook.com>
2020-10-12 11:36 ` Peter Zijlstra
[not found] ` <CH2PR14MB4183BF26F02A4AD4F45CADA59A070@CH2PR14MB4183.namprd14.prod.outlook.com>
2020-10-13 13:20 ` Valentin Schneider
2020-10-05 14:57 ` [PATCH -v2 11/17] sched/core: Make migrate disable and CPU hotplug cooperative Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 12/17] sched,rt: Use cpumask_any*_distribute() Peter Zijlstra
2020-10-06 14:09 ` Juri Lelli
2020-10-06 14:20 ` Peter Zijlstra
2020-10-06 15:55 ` Qais Yousef
2020-10-07 7:22 ` Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 13/17] sched,rt: Use the full cpumask for balancing Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 14/17] sched, lockdep: Annotate ->pi_lock recursion Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 15/17] sched: Fix migrate_disable() vs rt/dl balancing Peter Zijlstra
2020-10-06 7:59 ` Peter Zijlstra
2020-10-06 13:44 ` Steven Rostedt
2020-10-06 13:50 ` Peter Zijlstra
2020-10-06 11:20 ` Valentin Schneider
2020-10-06 13:48 ` Peter Zijlstra
2020-10-06 14:37 ` Juri Lelli
2020-10-06 14:48 ` Peter Zijlstra
2020-10-07 5:29 ` Juri Lelli
2020-10-06 16:19 ` Valentin Schneider
2020-10-07 19:22 ` Valentin Schneider
2020-10-07 21:08 ` Peter Zijlstra
2020-10-07 22:32 ` Valentin Schneider
2020-10-08 10:48 ` Sebastian Andrzej Siewior
2020-10-12 9:56 ` Dietmar Eggemann
2020-10-12 11:28 ` Peter Zijlstra
2020-10-12 12:22 ` Dietmar Eggemann
2020-10-05 14:57 ` [PATCH -v2 16/17] sched/proc: Print accurate cpumask vs migrate_disable() Peter Zijlstra
2020-10-05 14:57 ` [PATCH -v2 17/17] sched: Add migrate_disable() tracepoints Peter Zijlstra
2020-10-13 14:01 ` [PATCH 1/2] sched: Deny self-issued __set_cpus_allowed_ptr() when migrate_disable() Valentin Schneider
2020-10-13 14:15 ` Sebastian Andrzej Siewior
2020-10-13 14:21 ` Peter Zijlstra
2020-10-15 9:14 ` Valentin Schneider
2020-10-13 14:22 ` Valentin Schneider
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
2020-10-13 14:01 ` [PATCH 2/2] sched: Comment affine_move_task() Valentin Schneider
2020-11-11 8:23 ` [tip: sched/core] " tip-bot2 for Valentin Schneider
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201005150921.661842442@infradead.org \
--to=peterz@infradead.org \
--cc=bigeasy@linutronix.de \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=qais.yousef@arm.com \
--cc=rostedt@goodmis.org \
--cc=swood@redhat.com \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=valentin.schneider@arm.com \
--cc=vincent.donnefort@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®