From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: juri.lelli@redhat.com, peterz@infradead.org, mingo@redhat.com,
vincent.guittot@linaro.org
Cc: linux-kernel@vger.kernel.org,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
tglx@kernel.org, will@kernel.org, boqun@kernel.org,
dietmar.eggemann@arm.com, rostedt@goodmis.org,
bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
kprateek.nayak@amd.com, luca.abeni@santannapisa.it,
mathieu.poirier@linaro.org, longman@redhat.com,
ridong.chen@linux.dev, tj@kernel.org, hannes@cmpxchg.org,
mkoutny@suse.com, shuah@kernel.org, bigeasy@linutronix.de,
clrkwllms@kernel.org, parri.andrea@gmail.com,
m.szyprowski@samsung.com, kayracizmeci@gmail.com,
weihengqiuu@gmail.com, cgroups@vger.kernel.org,
linux-kselftest@vger.kernel.org, linux-rt-devel@lists.linux.dev,
stable@vger.kernel.org
Subject: [PATCH v7 29/40] sched/deadline: Keep the last CPU a task has in its root domain
Date: Tue, 17 Nov 2026 09:10:50 -0500 [thread overview]
Message-ID: <a075dad17300a5b31dc3d99bef81fb9dadd6cbcd.1794922146.git.mathieu.desnoyers@efficios.com> (raw)
In-Reply-To: <cover.1794922146.git.mathieu.desnoyers@efficios.com>
dl_bw_deactivate() refuses to take a CPU offline when the rest of its
root domain cannot hold the deadline bandwidth counted there. It also
keeps the last CPU of a root domain that counts a deadline task. That
test comes from commit d4742f6ed7ea ("sched/deadline: Correctly account
for allocated bandwidth during hotplug").
That test looks at the root domain, not at the mask of each task. With
isolcpus="domain", a cpuset that is not a partition root can hold one
housekeeping CPU and some isolated CPUs. A deadline task in it can be
counted in the housekeeping root domain. When that housekeeping CPU
goes offline, the root domain keeps its other CPUs, so the offline is
accepted. But the task has only isolated CPUs left. It runs in the
default root domain, which did not admit it. Its bandwidth follows with
no admission test, and the default root domain can exceed its limit.
Refuse the offline with EBUSY in that case. Walk the deadline tasks
counted in the root domain of the CPU. Refuse if one of them has CPUs
left to run on, but none in that root domain. Skip the walk if no task
holds a bandwidth. A task whose mask holds no other CPU gets the CPUs of
an ancestor of its cpuset: add cpuset_cpus_allowed_without() to tell
which. Like __set_cpus_allowed_ptr(), it keeps them within the affinity
set by the user, if they intersect. Skip a dead task: it runs no more.
Do not refuse during a suspend. It takes the CPUs offline with the
tasks frozen, and the resume puts them back. Do not refuse without
admission control either: there is no limit to exceed.
An administrator who needs the CPU offline can first give the cpuset of
the task another CPU of that root domain. The offline is then accepted,
and the task wakes up on that CPU: the hotplug_keeps_root_domain case
tests this. The administrator can also turn admission control off, by
writing -1 to /proc/sys/kernel/sched_rt_runtime_us.
The walk cannot miss a task. sched_setattr() only makes a deadline task
of a task whose mask holds its whole root domain. sched_setaffinity()
only gives a deadline task such a mask. A mask also gets smaller by a
cpuset change or a cpuset move, and both exclude CPU hotplug.
For user-space, the refused offline is a write of 0 to
/sys/devices/system/cpu/cpuN/online that fails with EBUSY. An offline
that leaves each deadline task a CPU in its root domain is accepted as
before.
The case hotplug_last_refused of deadline_bw_test, added later in this
series, tests this. Without this patch the offline is accepted. The
cases hotplug_user_mask_refused and hotplug_dead_accepted test the
affinity set by the user and a dead task.
Fixes: d4742f6ed7ea ("sched/deadline: Correctly account for allocated bandwidth during hotplug")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: stable@vger.kernel.org
---
include/linux/cpuset.h | 9 +++++++
kernel/cgroup/cpuset.c | 26 ++++++++++++++++++
kernel/sched/deadline.c | 60 ++++++++++++++++++++++++++++++++++++++++-
3 files changed, 94 insertions(+), 1 deletion(-)
diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
index 5d0c01a25354..68849539cc8b 100644
--- a/include/linux/cpuset.h
+++ b/include/linux/cpuset.h
@@ -79,6 +79,8 @@ extern void lockdep_assert_cpuset_lock_held(void);
extern void cpuset_cpus_allowed_locked(struct task_struct *p, struct cpumask *mask);
extern void cpuset_cpus_allowed(struct task_struct *p, struct cpumask *mask);
extern bool cpuset_cpus_allowed_fallback(struct task_struct *p);
+extern void cpuset_cpus_allowed_without(struct task_struct *p, int cpu,
+ struct cpumask *mask);
extern int cpuset_num_cpus(struct cgroup *cgroup);
extern nodemask_t cpuset_mems_allowed(struct task_struct *p);
#define cpuset_current_mems_allowed (current->mems_allowed)
@@ -214,6 +216,13 @@ static inline bool cpuset_cpus_allowed_fallback(struct task_struct *p)
return false;
}
+static inline void cpuset_cpus_allowed_without(struct task_struct *p, int cpu,
+ struct cpumask *mask)
+{
+ cpumask_and(mask, task_cpu_possible_mask(p), cpu_active_mask);
+ cpumask_clear_cpu(cpu, mask);
+}
+
static inline int cpuset_num_cpus(struct cgroup *cgroup)
{
return num_online_cpus();
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 7bdb9e21a7ed..3ccd61a37d30 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -4312,6 +4312,32 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask)
spin_unlock_irqrestore(&callback_lock, flags);
}
+/*
+ * Set @pmask to the active CPUs that the cpuset of @tsk gives it once @cpu
+ * is offline: those of the nearest ancestor that keeps one. Leave it empty
+ * if there is none. As cpuset_update_tasks_cpumask() does, keep the CPUs
+ * possible for @tsk. As __set_cpus_allowed_ptr() does, keep those of the
+ * affinity the user set, if one is left. The caller holds the CPU hotplug
+ * lock for writing, which keeps the cpusets and their tasks as they are.
+ * It holds the pi_lock of @tsk, which keeps its user affinity.
+ */
+void cpuset_cpus_allowed_without(struct task_struct *tsk, int cpu,
+ struct cpumask *pmask)
+{
+ struct cpuset *cs;
+
+ guard(rcu)();
+ for (cs = task_cs(tsk); cs; cs = parent_cs(cs)) {
+ cpumask_and(pmask, cs->effective_cpus, cpu_active_mask);
+ cpumask_clear_cpu(cpu, pmask);
+ if (!cpumask_empty(pmask))
+ break;
+ }
+ cpumask_and(pmask, pmask, task_cpu_possible_mask(tsk));
+ if (tsk->user_cpus_ptr && cpumask_intersects(pmask, tsk->user_cpus_ptr))
+ cpumask_and(pmask, pmask, tsk->user_cpus_ptr);
+}
+
/**
* cpuset_cpus_allowed_fallback - final fallback before complete catastrophe.
* @tsk: pointer to task_struct with which the scheduler is struggling
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index e5f77d3c5fa0..47264640de14 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -5028,9 +5028,67 @@ static int dl_bw_manage(enum dl_bw_request req, int cpu, u64 dl_bw)
return overflow ? -EBUSY : 0;
}
+/*
+ * Return true if @p has CPUs to run on once @cpu is offline, but none in
+ * the root domain of @cpu, which counts its bandwidth. A dead task runs no
+ * more. It can wait for its 0-lag time, or for its parent to reap it.
+ * Return false for it.
+ */
+static bool dl_task_leaves_span(struct task_struct *p, int cpu,
+ const struct cpumask *span)
+{
+ struct cpumask *msk;
+
+ guard(raw_spinlock_irqsave)(&p->pi_lock);
+ if (READ_ONCE(p->__state) == TASK_DEAD || dl_task_bw_span(p) != span)
+ return false;
+ msk = this_cpu_cpumask_var_ptr(local_cpu_mask_dl);
+ /* A task in a migrate_disable() section has a cpus_ptr of one CPU. */
+ cpumask_and(msk, &p->cpus_mask, cpu_active_mask);
+ cpumask_clear_cpu(cpu, msk);
+ /* With no CPU left, the cpuset gives @p the CPUs of an ancestor. */
+ if (cpumask_empty(msk))
+ cpuset_cpus_allowed_without(p, cpu, msk);
+ return !cpumask_empty(msk) && !cpumask_intersects(msk, span);
+}
+
+/*
+ * Return true if @cpu is the last CPU that a deadline task can run on in
+ * the root domain that counts its bandwidth. With @cpu offline, the task
+ * would run in another root domain, which did not admit it. The mask of
+ * a task can hold CPUs of two root domains: with isolcpus="domain", or on
+ * cgroup v1 with sched_load_balance off.
+ *
+ * A suspend takes the CPUs offline with the tasks frozen, and the resume
+ * puts them back. Without admission control, there is no limit to exceed.
+ * Return false in both cases.
+ */
+static bool dl_cpu_last_of_task(int cpu)
+{
+ const struct cpumask *span;
+ struct task_struct *g, *p;
+
+ if (cpuhp_tasks_frozen || !dl_bandwidth_enabled())
+ return false;
+ /* No task holds a bandwidth: do not walk the task list. */
+ if (!atomic_read(&dl_nr_bw_tasks))
+ return false;
+ guard(rcu)();
+ span = cpu_rq(cpu)->rd->span;
+ for_each_process_thread(g, p) {
+ if (task_has_dl_policy(p) && dl_task_leaves_span(p, cpu, span))
+ return true;
+ }
+ return false;
+}
+
int dl_bw_deactivate(int cpu)
{
- return dl_bw_manage(dl_bw_req_deactivate, cpu, 0);
+ int ret = dl_bw_manage(dl_bw_req_deactivate, cpu, 0);
+
+ if (!ret && dl_cpu_last_of_task(cpu))
+ ret = -EBUSY;
+ return ret;
}
int dl_bw_alloc(int cpu, u64 dl_bw)
--
2.47.3
next prev parent reply other threads:[~2026-11-17 14:11 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 14:10 [PATCH v7 00/40] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 01/40] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 02/40] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 03/40] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 04/40] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 05/40] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-17 14:38 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 06/40] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 07/40] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 08/40] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 09/40] rtmutex: Continue the chain walk when a waiter's deadline donor changes Mathieu Desnoyers
2026-11-17 14:50 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 10/40] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 11/40] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 12/40] sched/deadline: Replenish a throttled task boosted while asleep Mathieu Desnoyers
2026-11-17 14:55 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 13/40] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 14/40] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 15/40] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 16/40] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 17/40] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 18/40] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-17 15:14 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 19/40] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 20/40] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 21/40] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 22/40] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 23/40] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-17 15:18 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 24/40] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 25/40] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 26/40] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-17 15:31 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 27/40] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-17 15:29 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 28/40] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-17 15:32 ` sashiko-bot
2026-11-17 14:10 ` Mathieu Desnoyers [this message]
2026-11-17 15:41 ` [PATCH v7 29/40] sched/deadline: Keep the last CPU a task has in its root domain sashiko-bot
2026-11-17 14:10 ` [PATCH v7 30/40] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 31/40] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 32/40] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 33/40] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-17 15:46 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 34/40] sched/deadline: Cap the runtime left of a constrained task at dl_runtime Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 35/40] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 36/40] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-17 15:45 ` sashiko-bot
2026-11-17 14:10 ` [PATCH v7 37/40] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-17 14:10 ` [PATCH v7 38/40] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-17 15:23 ` sashiko-bot
2026-11-17 14:11 ` [PATCH v7 39/40] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-17 14:11 ` [PATCH v7 40/40] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a075dad17300a5b31dc3d99bef81fb9dadd6cbcd.1794922146.git.mathieu.desnoyers@efficios.com \
--to=mathieu.desnoyers@efficios.com \
--cc=bigeasy@linutronix.de \
--cc=boqun@kernel.org \
--cc=bsegall@google.com \
--cc=cgroups@vger.kernel.org \
--cc=clrkwllms@kernel.org \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=kayracizmeci@gmail.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-rt-devel@lists.linux.dev \
--cc=longman@redhat.com \
--cc=luca.abeni@santannapisa.it \
--cc=m.szyprowski@samsung.com \
--cc=mathieu.poirier@linaro.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mkoutny@suse.com \
--cc=parri.andrea@gmail.com \
--cc=peterz@infradead.org \
--cc=ridong.chen@linux.dev \
--cc=rostedt@goodmis.org \
--cc=shuah@kernel.org \
--cc=stable@vger.kernel.org \
--cc=tglx@kernel.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=weihengqiuu@gmail.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®