mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: juri.lelli@redhat.com, peterz@infradead.org, mingo@redhat.com,
	vincent.guittot@linaro.org
Cc: linux-kernel@vger.kernel.org,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	tglx@kernel.org, will@kernel.org, boqun@kernel.org,
	dietmar.eggemann@arm.com, rostedt@goodmis.org,
	bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
	kprateek.nayak@amd.com, luca.abeni@santannapisa.it,
	mathieu.poirier@linaro.org, longman@redhat.com,
	ridong.chen@linux.dev, tj@kernel.org, hannes@cmpxchg.org,
	mkoutny@suse.com, shuah@kernel.org, bigeasy@linutronix.de,
	clrkwllms@kernel.org, parri.andrea@gmail.com,
	m.szyprowski@samsung.com, kayracizmeci@gmail.com,
	weihengqiuu@gmail.com, cgroups@vger.kernel.org,
	linux-kselftest@vger.kernel.org, linux-rt-devel@lists.linux.dev,
	stable@vger.kernel.org
Subject: [PATCH v6 21/37] sched: Make deadline policy and affinity changes wait for a cpuset attach
Date: Mon, 16 Nov 2026 14:05:33 -0500	[thread overview]
Message-ID: <20261116190549.230119-22-mathieu.desnoyers@efficios.com> (raw)
In-Reply-To: <20261116190549.230119-1-mathieu.desnoyers@efficios.com>

__sched_setscheduler() takes cpuset_mutex for a change that involves
SCHED_DEADLINE. Commit 710da3c8ea7d ("sched/core: Prevent race condition
between cpuset and __sched_setscheduler()") added that lock. It keeps
the cpuset code from changing the root domains while the deadline
admission test runs.

But a cpuset attach drops cpuset_mutex between cpuset_can_attach() and
cpuset_attach(). A sched_setattr() can run in between. Take a process
in the root cgroup. It has deadline threads, and a thread T that is not
a deadline task. The process is moved into a partition root P:

  echo PID > P/cgroup.procs              sched_setattr(T, SCHED_DEADLINE)

  cpuset_can_attach()
    lock cpuset_mutex
    reserve room in P for the deadline
      threads: T is not one of them
    unlock cpuset_mutex
                                         lock cpuset_mutex
                                         admit T in the root domain it
                                           is still in
                                         unlock cpuset_mutex
  cpuset_attach()
    lock cpuset_mutex
    move T to P, where nothing is
      reserved for it
    unlock cpuset_mutex

The attach still moves the bandwidth of T. It is freed in the root
domain T left, and P has no room reserved to count it. No root domain
counts it any more, and P can admit deadline tasks beyond its capacity.

In testing, P was a one-CPU partition root, which admits 95%. The
process had two sleeping deadline threads of 30% each, and
sched_setattr() made T a deadline task of 40%. The move and the
sched_setattr() both succeeded, with no room in P for the three. With
threads of 20% and a T of 30%, P then admitted five more tasks of 10%,
where it had room for two.

Fix this by taking cpuset_mutex in __sched_setscheduler() only when no
attach is in progress. cpuset_partition_write() and
cpuset_hotplug_update_tasks() already wait for an attach that way. A
deadline policy change then comes before the attach, and is part of its
reservation, or after it.

With this patch, and no room in P for the three, the two never both
succeed. If the sched_setattr() comes first, the move is refused with
EBUSY. If it comes after, T has its new CPUs and has not run on them.
The call is then refused with EPERM, as for any sleeping task moved to
another partition.

sched_setaffinity() on a deadline task can run in between as well. A
later patch of this series reserves the bandwidth of the task on one CPU
of its new affinity, and records that CPU with the task. It is
"cgroup/cpuset: Reserve the bandwidth in the root domain a task moves
to". A new affinity in between could leave that CPU out. cpuset_attach()
would then move no bandwidth, and the task would stay counted in the
root domain it left. So sched_setaffinity() waits the same way when the
task is a deadline task.

sched_setaffinity() tests the policy with no lock, before it decides to
wait. A sched_setattr() can make the task a deadline task right after
that test. So test the policy again under the runqueue lock, where the
mask changes. If the task is a deadline task there and the caller did
not wait, the call starts over, and waits. The new SCA_DL_NOWAIT flag
asks for that test.

For user-space, these calls now wait until a cpuset attach in progress
ends:

 - sched_setattr() and sched_setscheduler() on a deadline task
 - sched_setattr() that makes a task a deadline task
 - sched_setaffinity() on a deadline task

The attach can be the move of any process. The wait is not
interruptible: a signal does not end it. Outside that window, the
results of these calls are unchanged.

Three cases of deadline_bw_test, added later in this series, test this.
move_became_deadline and move_became_deadline_no_room fail without this
patch. move_affinity_changed fails once the bandwidth is reserved on
one CPU, if sched_setaffinity() does not wait.

A fourth case, move_affinity_became_deadline, covers the second test of
the policy. Its window is a few instructions wide. A delay added after
the first test, for testing only, widens it. The case then fails without
the second test in 3 guests of 3. It passes with it.

Fixes: 710da3c8ea7d ("sched/core: Prevent race condition between cpuset and __sched_setscheduler()")
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: stable@vger.kernel.org
---
 include/linux/cpuset.h  |  2 ++
 kernel/cgroup/cpuset.c  | 11 +++++++++++
 kernel/sched/core.c     | 10 ++++++++++
 kernel/sched/sched.h    |  2 ++
 kernel/sched/syscalls.c | 40 ++++++++++++++++++++++++++++++++--------
 5 files changed, 57 insertions(+), 8 deletions(-)

diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
index db1cc2fc0324..5d0c01a25354 100644
--- a/include/linux/cpuset.h
+++ b/include/linux/cpuset.h
@@ -73,6 +73,7 @@ extern void cpuset_init_smp(void);
 extern void cpuset_force_rebuild(void);
 extern void cpuset_update_active_cpus(void);
 extern void cpuset_lock(void);
+extern void cpuset_lock_no_attach(void);
 extern void cpuset_unlock(void);
 extern void lockdep_assert_cpuset_lock_held(void);
 extern void cpuset_cpus_allowed_locked(struct task_struct *p, struct cpumask *mask);
@@ -192,6 +193,7 @@ static inline void cpuset_update_active_cpus(void)
 }
 
 static inline void cpuset_lock(void) { }
+static inline void cpuset_lock_no_attach(void) { }
 static inline void cpuset_unlock(void) { }
 static inline void lockdep_assert_cpuset_lock_held(void) { }
 
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 51b6be460794..ce9c57617f49 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -378,6 +378,17 @@ static void wait_attach_done_lock(void)
 	}
 }
 
+/**
+ * cpuset_lock_no_attach - Acquire cpuset_mutex with no task attach in progress
+ *
+ * A task attach drops cpuset_mutex between cpuset_can_attach() and
+ * cpuset_attach(). Use this to keep a change out of that window.
+ */
+void cpuset_lock_no_attach(void)
+{
+	wait_attach_done_lock();
+}
+
 static inline void check_insane_mems_config(nodemask_t *nodes)
 {
 	if (!cpusets_insane_config() &&
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2a964286a7cd..52c291de384e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3195,6 +3195,16 @@ static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
 		goto out;
 	}
 
+	/*
+	 * sched_setaffinity() tested the policy with no lock, and did not
+	 * wait for a cpuset attach in progress. @p became a deadline task
+	 * since: let the caller wait and try again.
+	 */
+	if ((ctx->flags & SCA_DL_NOWAIT) && task_has_dl_policy(p)) {
+		ret = -EAGAIN;
+		goto out;
+	}
+
 	/*
 	 * Check under the same lock that __sched_setscheduler() holds for
 	 * its admission test, so that p cannot become SCHED_DEADLINE
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 9e3ca69abc36..2087ad70c56e 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2985,6 +2985,8 @@ extern struct task_struct *pick_task_idle(struct rq *rq, struct rq_flags *rf);
 #define SCA_DL_CHECK		0x10
 /* The caller made room for the deadline bandwidth in the new root domain. */
 #define SCA_DL_BW_RESERVED	0x20
+/* The caller did not wait for a cpuset attach: -EAGAIN for a deadline task. */
+#define SCA_DL_NOWAIT		0x40
 
 extern void update_group_capacity(struct sched_domain *sd, int cpu);
 
diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c
index 4f47b17151ed..70b364b4f82d 100644
--- a/kernel/sched/syscalls.c
+++ b/kernel/sched/syscalls.c
@@ -564,10 +564,15 @@ int __sched_setscheduler(struct task_struct *p,
 	 * information. Do not take cpuset_mutex when pi is unset: the
 	 * caller is normalize_rt_tasks(), which holds tasklist_lock and
 	 * cannot sleep.
+	 *
+	 * Wait for a task attach in progress. cpuset_can_attach() reserved
+	 * the bandwidth of the deadline tasks it saw, and cpuset_attach()
+	 * moves them. A task that enters or leaves the class in between
+	 * would not match that reservation.
 	 */
 	if (pi && (dl_policy(policy) || dl_policy(p->policy))) {
 		cpuset_locked = true;
-		cpuset_lock();
+		cpuset_lock_no_attach();
 		dl_accounting_lock();
 	}
 
@@ -1289,6 +1294,7 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
 	struct affinity_context ac;
 	struct cpumask *user_mask;
 	int retval;
+	bool dl;
 
 	CLASS(find_get_task, p)(pid);
 	if (!p)
@@ -1318,13 +1324,31 @@ long sched_setaffinity(pid_t pid, const struct cpumask *in_mask)
 		return -ENOMEM;
 	}
 
-	ac = (struct affinity_context){
-		.new_mask  = in_mask,
-		.user_mask = user_mask,
-		.flags     = SCA_USER,
-	};
-
-	retval = __sched_setaffinity(p, &ac);
+	/*
+	 * Wait for a cpuset attach in progress before changing the affinity
+	 * of a deadline task. cpuset_can_attach() reserved the bandwidth of
+	 * the task for the CPUs it moves to, and cpuset_attach() moves it
+	 * there. A new affinity in between could change those CPUs.
+	 *
+	 * The policy is tested here with no lock. A task that becomes a
+	 * deadline task after this test is found under its runqueue lock,
+	 * with SCA_DL_NOWAIT. Wait and try again then.
+	 */
+	for (;;) {
+		dl = task_has_dl_policy(p);
+		ac = (struct affinity_context){
+			.new_mask  = in_mask,
+			.user_mask = user_mask,
+			.flags     = SCA_USER | (dl ? 0 : SCA_DL_NOWAIT),
+		};
+		if (dl)
+			cpuset_lock_no_attach();
+		retval = __sched_setaffinity(p, &ac);
+		if (dl)
+			cpuset_unlock();
+		if (dl || retval != -EAGAIN)
+			break;
+	}
 	kfree(ac.user_mask);
 
 	return retval;
-- 
2.47.3


  parent reply	other threads:[~2026-11-16 19:06 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 19:05 [PATCH v6 00/37] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 01/37] sched: Refuse SCHED_FLAG_KEEP_PARAMS that would change the policy or the class Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 02/37] sched/deadline: Do not admit bandwidth for parameters that are kept Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 03/37] sched: Let a task boosted by a deadline waiter change its policy Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 04/37] sched/deadline: Do not divide by zero active CPUs of a root domain Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 05/37] sched/deadline: Refuse dl_server debugfs writes while the rq is offline Mathieu Desnoyers
2026-11-16 19:30   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 06/37] sched: Take cpuset_mutex for a deadline policy change only with pi Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 07/37] sched/deadline: Do not lose a throttled task switched out and back Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 08/37] sched/deadline: Do not replenish a boost from cleared parameters Mathieu Desnoyers
2026-11-16 19:42   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 09/37] sched/deadline: Reset the deadline and the runtime with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 10/37] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-16 19:38   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 11/37] sched/deadline: Release the bandwidth of a task that leaves the class asleep Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 12/37] sched/deadline: Reject SCHED_DEADLINE on exited tasks Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 13/37] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 14/37] sched/deadline: Fix this_bw when a task is changed during a remote wakeup Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 15/37] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 16/37] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-16 19:44   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 17/37] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 18/37] sched/deadline: Check SCHED_DEADLINE affinity under the rq lock Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 19/37] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 20/37] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-16 19:05 ` Mathieu Desnoyers [this message]
2026-11-16 19:31   ` [PATCH v6 21/37] sched: Make deadline policy and affinity changes wait for a cpuset attach sashiko-bot
2026-11-16 19:05 ` [PATCH v6 22/37] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 23/37] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-16 19:56   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 24/37] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-16 20:01   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 25/37] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-16 19:47   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 26/37] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-16 19:50   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 27/37] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-16 19:51   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 28/37] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 29/37] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 30/37] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 31/37] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-16 20:12   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 32/37] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-16 20:08   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 33/37] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-16 20:04   ` sashiko-bot
2026-11-16 19:05 ` [PATCH v6 34/37] sched/debug: Show the deadline bandwidth of each runqueue Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 35/37] selftests/sched: Add a test of SCHED_DEADLINE policy round trips Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 36/37] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-16 19:05 ` [PATCH v6 37/37] selftests/sched: Add a test of SCHED_DEADLINE bandwidth accounting Mathieu Desnoyers
2026-11-16 19:49   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116190549.230119-22-mathieu.desnoyers@efficios.com \
    --to=mathieu.desnoyers@efficios.com \
    --cc=bigeasy@linutronix.de \
    --cc=boqun@kernel.org \
    --cc=bsegall@google.com \
    --cc=cgroups@vger.kernel.org \
    --cc=clrkwllms@kernel.org \
    --cc=dietmar.eggemann@arm.com \
    --cc=hannes@cmpxchg.org \
    --cc=juri.lelli@redhat.com \
    --cc=kayracizmeci@gmail.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=longman@redhat.com \
    --cc=luca.abeni@santannapisa.it \
    --cc=m.szyprowski@samsung.com \
    --cc=mathieu.poirier@linaro.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=mkoutny@suse.com \
    --cc=parri.andrea@gmail.com \
    --cc=peterz@infradead.org \
    --cc=ridong.chen@linux.dev \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=tglx@kernel.org \
    --cc=tj@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    --cc=weihengqiuu@gmail.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®