From: sashiko-bot@kernel.org
To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
Cc: linux-rt-devel@lists.linux.dev, tj@kernel.org,
hannes@cmpxchg.org, mkoutny@suse.com,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v7 28/40] sched/deadline: Keep a task in the root domain that counts its bandwidth
Date: Tue, 17 Nov 2026 15:32:59 +0000 [thread overview]
Message-ID: <sashiko-outbox-219085@kernel.org> (raw)
In-Reply-To: <2c1a48adf25ac3528a1f88632bda53826db5399e.1794922146.git.mathieu.desnoyers@efficios.com>
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Medium] sched/deadline: missing wake_cpu update in
__set_cpus_allowed_ptr_locked()
- [Medium] sched/deadline: wrong root domain in
dl_task_offline_migration() on rebuild
- [Medium] sched/deadline: missing READ_ONCE() on dl_bw_cpu in
dl_task_rebuild_cpu()
--
commit fecf231094845bbb7da45c72106b5c90fffb123d
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
sched/deadline: Keep a task in the root domain that counts its bandwidth
This commit adds dl_task_bw_span() and updates CPU selection paths for
SCHED_DEADLINE tasks to prefer CPUs in the root domain that accounts for
the task's bandwidth.
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index e9f80407f75be..dc270a23179d5 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3264,6 +3264,23 @@ static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
> dest_cpu = ctx->dl_bw_cpu;
> if (!task_on_rq_queued(p))
> p->wake_cpu = dest_cpu;
> + } else {
> + const struct cpumask *span = dl_task_bw_span(p);
> + unsigned int cpu = nr_cpu_ids;
> +
> + /*
> + * No room was made for @p in another root domain. Keep @p
> + * in the root domain that counts its bandwidth, if the new
> + * mask holds one of its CPUs.
> + */
> + if (span && !cpumask_test_cpu(dest_cpu, span))
> + cpu = cpumask_first_and_and(cpu_valid_mask,
> + ctx->new_mask, span);
> + if (cpu < nr_cpu_ids) {
> + dest_cpu = cpu;
> + if (!task_on_rq_queued(p))
> + p->wake_cpu = dest_cpu;
> + }
> }
[Severity: Medium]
Does this leave p->wake_cpu stale when cpumask_any_and_distribute()
already picked a dest_cpu inside span?
When cpumask_test_cpu(dest_cpu, span) is true, cpu stays at nr_cpu_ids and
p->wake_cpu is not updated for a sleeping task. Because affine_move_task()
does not move sleeping tasks, p->wake_cpu remains on the old CPU (such as
an offlining housekeeping CPU during cpuset_hotplug_update_tasks()). If p
then wakes up during the subsequent sched domain rebuild while
dl_task_bw_span(p) returns NULL, select_task_rq() and select_fallback_rq()
can place p on an isolated CPU in def_root_domain and move its bandwidth
without admission control.
Updating p->wake_cpu whenever span is non-NULL and dest_cpu is in span
(and !task_on_rq_queued(p)) would avoid this.
[ ... ]
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index 6c6a4d9cd0548..e5f77d3c5fa03 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
[ ... ]
> @@ -901,18 +903,36 @@ static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
>
> static struct rq *dl_task_offline_migration(struct rq *rq, struct task_struct *p)
> {
> + const struct cpumask *span;
> struct rq *later_rq = NULL;
> struct dl_bw *dl_b;
>
> later_rq = find_lock_later_rq(p, rq);
> + /*
> + * find_lock_later_rq() looks in the root domain of @rq. Once @rq is
> + * offline, a rebuild attaches it to def_root_domain. That root domain
> + * may not count @p. Do not take @p out of the root domain that counts
> + * it while @p has an active CPU there.
> + */
> + span = dl_task_bw_span(p);
> + if (later_rq && span && !cpumask_test_cpu(later_rq->cpu, span) &&
> + cpumask_first_and_and(cpu_active_mask, p->cpus_ptr, span) < nr_cpu_ids) {
> + double_unlock_balance(rq, later_rq);
> + later_rq = NULL;
> + }
> if (!later_rq) {
> int cpu;
>
> /*
> * If we cannot preempt any rq, fall back to pick any
> - * online CPU:
> + * online CPU. Try the root domain that counts @p first.
> */
> - cpu = cpumask_any_and(cpu_active_mask, p->cpus_ptr);
> + cpu = nr_cpu_ids;
> + if (span)
> + cpu = cpumask_first_and_and(cpu_active_mask,
> + p->cpus_ptr, span);
> + if (cpu >= nr_cpu_ids)
> + cpu = cpumask_any_and(cpu_active_mask, p->cpus_ptr);
[Severity: Medium]
Can dl_task_offline_migration() still move a throttled task and its
bandwidth into def_root_domain when dl_task_timer() fires during a sched
domain rebuild?
CPU0 CPU1 (<hardirq>)
sched_cpu_deactivate(H)
sched_set_rq_offline(rq_H)
cpuset_cpu_inactive(H)
partition_sched_domains_locked()
dl_bw_rebuild_begin()
detach_destroy_domains()
cpu_attach_domain(def_rd, H)
dl_task_timer(p)
dl_task_offline_migration(rq_H, p)
find_lock_later_rq() -> iso_rq
dl_task_bw_span(p) -> NULL
p->dl.dl_bw_cpu = iso_rq->cpu
dl_rebuild_rd_accounting()
dl_add_task_root_domain(p)
dl_task_rebuild_cpu(p) -> iso_rq->cpu
While dl_bw_rebuild_seq is odd, rq_H is already attached to
def_root_domain, so find_lock_later_rq() searches def_root_domain (or
cpumask_any_and() picks the first active CPU in p->cpus_ptr, which can be
an isolated CPU). Because dl_task_bw_span(p) returns NULL during the
rebuild, the check above does not reject the isolated later_rq.
Unlike migrate_task_rq_dl() and task_woken_dl(), which skip moving
bandwidth while dl_bw_rebuilding() is true, dl_task_offline_migration()
then unconditionally subtracts p->dl.dl_bw from src_rq->rd->dl_bw (which
can underflow total_bw before dl_clear_root_domains() runs), adds it to
def_root_domain, and overwrites p->dl.dl_bw_cpu with the isolated CPU.
Both the rebuild and later wakeups then treat def_root_domain as the root
domain that counts p.
[ ... ]
> @@ -3987,17 +4075,37 @@ static unsigned int dl_task_rebuild_cpu(struct task_struct *p)
> */
> cpumask_and(msk, &p->cpus_mask, cpu_active_mask);
> cpu = task_cpu(p);
> - if (cpumask_test_cpu(cpu, msk) ||
> - (cpu == recorded && cpu_active(cpu) && task_on_rq_queued(p)))
> + if (cpu == recorded && cpu_active(cpu) && task_on_rq_queued(p))
> return cpu;
> if (cpumask_empty(msk)) {
> if (recorded >= 0 && cpu_active(recorded))
> return recorded;
> dl_get_task_effective_cpus(p, msk);
> cpumask_and(msk, msk, cpu_active_mask);
> - } else if (recorded >= 0 && cpumask_test_cpu(recorded, msk)) {
> - return recorded;
> }
> + /*
> + * Keep the bandwidth where it was counted: with the housekeeping
> + * CPUs, or with the CPUs that isolcpus="domain" isolates. The root
> + * domains are new, and the CPU recorded can be offline. So prefer a
> + * CPU of the mask of the same kind as the CPU recorded: housekeeping
> + * or not.
> + */
> + if (recorded >= 0) {
> + const struct cpumask *hk = housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT);
> + bool rec_hk = cpumask_test_cpu(recorded, hk);
> + unsigned int same;
> +
> + if (cpumask_test_cpu(cpu, msk) && cpumask_test_cpu(cpu, hk) == rec_hk)
> + return cpu;
> + if (cpumask_test_cpu(recorded, msk))
> + return recorded;
> + for_each_cpu(same, msk) {
> + if (cpumask_test_cpu(same, hk) == rec_hk)
> + return same;
> + }
> + }
[Severity: Medium]
Should p->dl.dl_bw_cpu be read with READ_ONCE() when initializing recorded
in dl_task_rebuild_cpu()?
dl_add_task_root_domain() calls dl_task_rebuild_cpu(p) while holding only
p->pi_lock, before acquiring dl_departed_lock. Concurrently, a dying
deadline task in its final __schedule() holds rq->lock and
dl_departed_lock (not p->pi_lock) and can write p->dl.dl_bw_cpu = -1 via
task_non_contending() -> dl_task_release_bw(). Because recorded is loaded
with a plain read and used after calling housekeeping_cpumask(), the
compiler is free to reload p->dl.dl_bw_cpu after the recorded >= 0 check
and pass -1 to cpumask_test_cpu().
--
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1794922146.git.mathieu.desnoyers@efficios.com?part=28
next prev parent reply other threads:[~2026-11-17 15:33 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 14:10 [PATCH v7 00/40] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 01/40] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 02/40] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 03/40] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 04/40] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 05/40] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-17 14:38 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 06/40] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 07/40] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 08/40] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 09/40] rtmutex: Continue the chain walk when a waiter's deadline donor changes Mathieu Desnoyers
2026-11-17 14:50 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 10/40] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 11/40] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 12/40] sched/deadline: Replenish a throttled task boosted while asleep Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 13/40] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 14/40] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 15/40] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 16/40] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 17/40] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 18/40] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-17 15:14 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 19/40] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 20/40] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 21/40] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 22/40] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 23/40] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-17 15:18 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 24/40] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 25/40] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 26/40] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-17 15:31 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 27/40] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-17 15:29 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 28/40] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-17 15:32 ` sashiko-bot [this message]
2026-11-17 14:10 ` [PATCH v7 29/40] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-17 15:41 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 30/40] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 31/40] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 32/40] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 33/40] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-17 15:46 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 34/40] sched/deadline: Cap the runtime left of a constrained task at dl_runtime Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 35/40] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 36/40] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-17 15:45 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 37/40] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 38/40] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-17 15:23 ` sashiko-bot
2026-11-17 14:11 ` [PATCH v7 39/40] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-17 14:11 ` [PATCH v7 40/40] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-219085@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=mathieu.desnoyers@efficios.com \
--cc=mkoutny@suse.com \
--cc=sashiko-reviews@lists.linux.dev \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®