From: Waiman Long <llong@redhat.com>
To: Juri Lelli <juri.lelli@redhat.com>, Tejun Heo <tj@kernel.org>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Koutny <mkoutny@suse.com>, Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>
Cc: Qais Yousef <qyousef@layalina.io>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
"Joel Fernandes (Google)" <joel@joelfernandes.org>,
Suleiman Souhlal <suleiman@google.com>,
Aashish Sharma <shraash@google.com>,
Shin Kawamura <kawasin@google.com>,
Vineeth Remanan Pillai <vineeth@bitbyteword.org>,
linux-kernel@vger.kernel.org, cgroups@vger.kernel.org
Subject: Re: [PATCH 2/2] sched/deadline: Correctly account for allocated bandwidth during hotplug
Date: Wed, 13 Nov 2024 11:06:24 -0500 [thread overview]
Message-ID: <8e55c640-c931-4b9c-a501-c5b0a654a420@redhat.com> (raw)
In-Reply-To: <20241113125724.450249-3-juri.lelli@redhat.com>
On 11/13/24 7:57 AM, Juri Lelli wrote:
> For hotplug operations, DEADLINE needs to check that there is still enough
> bandwidth left after removing the CPU that is going offline. We however
> fail to do so currently.
>
> Restore the correct behavior by restructuring dl_bw_manage() a bit, so
> that overflow conditions (not enough bandwidth left) are properly
> checked. Also account for dl_server bandwidth, i.e. discount such
> bandwidht in the calculation since NORMAL tasks will be anyway moved
> away from the CPU as a result of the hotplug operation.
>
> Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
> ---
> kernel/sched/core.c | 2 +-
> kernel/sched/deadline.c | 33 ++++++++++++++++++++++++---------
> kernel/sched/sched.h | 2 +-
> 3 files changed, 26 insertions(+), 11 deletions(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 43e453ab7e20..d1049e784510 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -8057,7 +8057,7 @@ static void cpuset_cpu_active(void)
> static int cpuset_cpu_inactive(unsigned int cpu)
> {
> if (!cpuhp_tasks_frozen) {
> - int ret = dl_bw_check_overflow(cpu);
> + int ret = dl_bw_deactivate(cpu);
>
> if (ret)
> return ret;
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index e53208a50279..609685c5df05 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -3467,29 +3467,31 @@ int dl_cpuset_cpumask_can_shrink(const struct cpumask *cur,
> }
>
> enum dl_bw_request {
> - dl_bw_req_check_overflow = 0,
> + dl_bw_req_deactivate = 0,
> dl_bw_req_alloc,
> dl_bw_req_free
> };
>
> static int dl_bw_manage(enum dl_bw_request req, int cpu, u64 dl_bw)
> {
> - unsigned long flags;
> + unsigned long flags, cap;
> struct dl_bw *dl_b;
> bool overflow = 0;
> + u64 fair_server_bw = 0;
>
> rcu_read_lock_sched();
> dl_b = dl_bw_of(cpu);
> raw_spin_lock_irqsave(&dl_b->lock, flags);
>
> - if (req == dl_bw_req_free) {
> + cap = dl_bw_capacity(cpu);
> + switch (req) {
> + case dl_bw_req_free:
> __dl_sub(dl_b, dl_bw, dl_bw_cpus(cpu));
> - } else {
> - unsigned long cap = dl_bw_capacity(cpu);
> -
> + break;
> + case dl_bw_req_alloc:
> overflow = __dl_overflow(dl_b, cap, 0, dl_bw);
>
> - if (req == dl_bw_req_alloc && !overflow) {
> + if (!overflow) {
> /*
> * We reserve space in the destination
> * root_domain, as we can't fail after this point.
> @@ -3498,6 +3500,19 @@ static int dl_bw_manage(enum dl_bw_request req, int cpu, u64 dl_bw)
> */
> __dl_add(dl_b, dl_bw, dl_bw_cpus(cpu));
> }
> + break;
> + case dl_bw_req_deactivate:
> + /*
> + * cpu is going offline and NORMAL tasks will be moved away
> + * from it. We can thus discount dl_server bandwidth
> + * contribution as it won't need to be servicing tasks after
> + * the cpu is off.
> + */
> + if (cpu_rq(cpu)->fair_server.dl_server)
> + fair_server_bw = cpu_rq(cpu)->fair_server.dl_bw;
> +
> + overflow = __dl_overflow(dl_b, cap, fair_server_bw, 0);
> + break;
This part can still cause a failure in one of test cases in my cpuset
partition test script. In this particular case, the CPU to be offlined
is an isolated CPU with scheduling disabled. As a result, total_bw is 0
and the __dl_overflow() test failed. Is there a way to skip the
__dl_overflow() test for isolated CPUs? Can we use a null total_bw as a
proxy for that?
Thanks,
Longman
next prev parent reply other threads:[~2024-11-13 16:06 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-13 12:57 [PATCH 0/2] Fix DEADLINE bandwidth accounting in root domain changes and hotplug Juri Lelli
2024-11-13 12:57 ` [PATCH 1/2] sched/deadline: Restore dl_server bandwidth on non-destructive root domain changes Juri Lelli
2024-11-13 13:43 ` Phil Auld
2024-11-13 14:57 ` Juri Lelli
2024-11-13 16:00 ` Waiman Long
2024-11-17 0:54 ` Joel Fernandes
2024-11-13 12:57 ` [PATCH 2/2] sched/deadline: Correctly account for allocated bandwidth during hotplug Juri Lelli
2024-11-13 13:49 ` Phil Auld
2024-11-13 14:58 ` Juri Lelli
2024-11-13 16:22 ` Phil Auld
2024-11-13 16:06 ` Waiman Long [this message]
2024-11-13 16:40 ` Juri Lelli
2024-11-13 16:42 ` Waiman Long
2024-11-13 16:50 ` Waiman Long
2024-11-13 18:11 ` Juri Lelli
2024-11-13 18:19 ` Waiman Long
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8e55c640-c931-4b9c-a501-c5b0a654a420@redhat.com \
--to=llong@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=joel@joelfernandes.org \
--cc=juri.lelli@redhat.com \
--cc=kawasin@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=peterz@infradead.org \
--cc=qyousef@layalina.io \
--cc=rostedt@goodmis.org \
--cc=shraash@google.com \
--cc=suleiman@google.com \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vineeth@bitbyteword.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®