mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: sashiko-bot@kernel.org
To: "Mathieu Desnoyers" <mathieu.desnoyers@efficios.com>
Cc: linux-rt-devel@lists.linux.dev, tj@kernel.org,
	hannes@cmpxchg.org, mkoutny@suse.com,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v8 16/33] sched/deadline: Keep the last CPU a task has in its root domain
Date: Thu, 19 Nov 2026 20:48:06 +0000	[thread overview]
Message-ID: <sashiko-outbox-224050@kernel.org> (raw)
In-Reply-To: <6e555ba3ad1013206d77ded129a127dadfd3b983.1795116791.git.mathieu.desnoyers@efficios.com>

Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [Medium] cgroup/cpuset: empty mask in cpuset_cpus_allowed_without() on
  top_cpuset loss
- [Medium] cgroup/cpuset: cpuset_cpus_allowed_without() walks past empty
  valid partition
- [Medium] sched/deadline: missed root domain move when offline
  invalidates PRS_ISOLATED
--

commit 6a0cfaf09c2412ad93623e8c7467bb4c5d10dd61
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

sched/deadline: Keep the last CPU a task has in its root domain

This commit refuses CPU offline in dl_bw_deactivate() with -EBUSY when
removing the CPU would leave a deadline task with runnable CPUs only
outside the root domain that admitted its bandwidth, or when invalidating
a cpuset partition would move a deadline task to another root domain.

> diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> index 7bdb9e21a7edb..5299652d47b6e 100644
> --- a/kernel/cgroup/cpuset.c
> +++ b/kernel/cgroup/cpuset.c

[ ... ]

> @@ -4312,6 +4334,234 @@ void cpuset_cpus_allowed(struct task_struct *tsk, struct cpumask *pmask)

[ ... ]

> +bool cpuset_cpus_allowed_without(struct task_struct *tsk, int cpu,
> +				 struct cpumask *pmask)
> +{
> +	const struct cpumask *possible = task_cpu_possible_mask(tsk);
> +	struct cpuset *cs = task_cs(tsk), *from;
> +
> +	guard(rcu)();
> +	if (!is_in_v2_mode()) {
> +		/*
> +		 * cgroup v1 moves the tasks of a cpuset left with no CPU to
> +		 * the nearest ancestor that keeps one.
> +		 */
> +		for (from = cs; from; from = parent_cs(from)) {
> +			cpumask_and(pmask, from->effective_cpus, cpu_active_mask);
> +			cpumask_clear_cpu(cpu, pmask);
> +			if (cpumask_intersects(pmask, possible))
> +				break;
> +		}
> +	} else {
> +		/*
> +		 * cgroup v2 works out the CPUs of each cpuset from the top
> +		 * down. A cpuset keeps those of its parent that cpuset.cpus
> +		 * holds, or takes all of them if there are none. Start from
> +		 * the nearest partition root that keeps an active CPU, or from
> +		 * the top cpuset. A partition root left with no CPU of its own
> +		 * can take those of a child partition.
> +		 */
> +		for (from = cs; ; from = parent_cs(from)) {
> +			if (from != &top_cpuset && !is_partition_valid(from))
> +				continue;
> +			cpumask_and(pmask, from->effective_cpus, cpu_active_mask);
> +			cpumask_clear_cpu(cpu, pmask);
> +			if (cpumask_empty(pmask) && from != &top_cpuset &&
> +			    !partition_child_lost(from, cpu, pmask))
> +				cpumask_clear(pmask);

[Severity: Medium]
Does cpuset_cpus_allowed_without() leave pmask empty when from reaches
top_cpuset and @cpu was the last active CPU in top_cpuset.effective_cpus?

When top_cpuset has no active CPUs left outside subpartitions_cpus,
cpuset_handle_hotplug() clears subpartitions_cpus and reclaims all
remaining active CPUs into top_cpuset.effective_cpus:

kernel/cgroup/cpuset.c:cpuset_handle_hotplug() {
    ...
		if (!cpumask_empty(subpartitions_cpus)) {
			if (cpumask_subset(&new_cpus, subpartitions_cpus)) {
				cpumask_clear(subpartitions_cpus);
			} else {
				cpumask_andnot(&new_cpus, &new_cpus,
					       subpartitions_cpus);
			}
		}
		cpumask_copy(top_cpuset.effective_cpus, &new_cpus);
    ...
}

cpuset_offline_moves_dl() skips non-partition descendants whose enclosing
partition is top_cpuset (a == &top_cpuset), leaving their deadline tasks
to dl_task_leaves_span().  Because pmask stays empty at top_cpuset here,
cpuset_cpus_allowed_without() skips effective_cpus_below() and
tsk->user_cpus_ptr filtering and returns false.  dl_task_leaves_span()
then takes the select_fallback_rq() branch and accepts the offline
whenever the task's current root domain has another active CPU (for
example, a boot-isolated CPU in def_root_domain), even if the CPUs
reclaimed from subpartitions_cpus put the task's cpuset in a different
root domain without admission control.

> +			if (!cpumask_empty(pmask) || from == &top_cpuset)
> +				break;
> +		}
> +		effective_cpus_below(cs, from, pmask);

[Severity: Medium]
Can this loop walk past an unpopulated valid parent partition root that
stays valid with an empty effective_cpus mask during hotplug?

Suppose an unpopulated outer partition p_outer delegates all of its
active effective_xcpus to two child partitions: p_inner1 on @cpu (an
isolated partition sharing def_root_domain with a boot-isolated CPU in
top_cpuset) and p_inner2 on another active CPU.  During hotplug,
cpuset_handle_hotplug() visits descendants in pre-order: when it visits
p_outer, p_inner1 is still a valid partition root, so
partition_is_populated(p_outer, NULL) is false, p_inner2 is not
invalidated, and p_outer stays valid with an empty effective_cpus mask.
When cpuset_hotplug_update_tasks() next visits p_inner1, p_inner1 is
invalidated and inherits p_outer's empty effective_cpus mask.

Here, when from walks up from p_inner1 to p_outer, pmask is empty and
partition_child_lost(p_outer, cpu, pmask) returns NULL because
partition_populated_offline(p_outer) is false.  Instead of stopping at
p_outer with an empty mask, the loop continues up to top_cpuset and
returns true with top_cpuset's CPUs intersected with tsk->user_cpus_ptr.
If tsk->user_cpus_ptr restricts the task in p_inner1 to @cpu and the
boot-isolated CPU in top_cpuset, dl_task_leaves_span() sees the
boot-isolated CPU in msk and accepts the offline, whereas actual hotplug
gives p_inner1 an empty effective_cpus mask and dl_task_rebuild_cpu()
falls back to guarantee_active_cpus() (ignoring user_cpus_ptr) and moves
the task's bandwidth to a housekeeping CPU in top_cpuset.

[ ... ]

> +bool cpuset_offline_moves_dl(int cpu)
> +{
> +	struct cgroup_subsys_state *pos_css;
> +	struct cpuset *cp, *a, *p = &top_cpuset, *lost = NULL;
> +	bool found = false;
> +
> +	if (!is_in_v2_mode())
> +		return false;
> +	rcu_read_lock();
> +	cpuset_for_each_descendant_pre(cp, pos_css, &top_cpuset) {
> +		if (cp != &top_cpuset && is_partition_valid(cp) &&
> +		    cpumask_test_cpu(cpu, cp->effective_cpus)) {
> +			p = cp;
> +			break;
> +		}
> +	}
> +	if (p != &top_cpuset) {
> +		lost = partition_child_lost(p, cpu, &dl_offline_cpus);
> +	} else {
> +		cpumask_and(&dl_offline_cpus, top_cpuset.effective_cpus, cpu_active_mask);
> +		cpumask_clear_cpu(cpu, &dl_offline_cpus);
> +		if (cpumask_empty(&dl_offline_cpus) && !cpumask_empty(subpartitions_cpus))
> +			lost = &top_cpuset;
> +	}
> +	if (!lost) {
> +		rcu_read_unlock();
> +		return false;
> +	}
> +	cpuset_for_each_descendant_pre(cp, pos_css, lost) {
> +		if (cp == &top_cpuset)
> +			continue;
> +		/* The partition that gives @cp its CPUs. */
> +		for (a = cp; a != lost && !is_partition_valid(a); a = parent_cs(a))
> +			;
> +		if (a == &top_cpuset ||
> +		    (a->partition_root_state == PRS_ISOLATED &&
> +		     p->partition_root_state == PRS_ISOLATED) ||
> +		    (a == lost && a->partition_root_state != PRS_ISOLATED &&
> +		     p->partition_root_state != PRS_ISOLATED))
> +			continue;
> +		if (!css_tryget_online(&cp->css))
> +			continue;
> +		rcu_read_unlock();
> +		found = dl_task_in_cpuset(cp);
> +		rcu_read_lock();
> +		css_put(&cp->css);
> +		if (found)
> +			break;
> +	}
> +	rcu_read_unlock();
> +	return found;
> +}
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index c33e73e920ebb..eead45a0c1191 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -5066,9 +5066,79 @@ static int dl_bw_manage(enum dl_bw_request req, int cpu, u64 dl_bw)
>  	return overflow ? -EBUSY : 0;
>  }
>  
> +/*
> + * Return true if @p has CPUs to run on once @cpu is offline, but none in
> + * the root domain of @cpu, which counts its bandwidth. A dead task runs no
> + * more. It can wait for its 0-lag time, or for its parent to reap it.
> + * Return false for it.
> + */
> +static bool dl_task_leaves_span(struct task_struct *p, int cpu,
> +				const struct cpumask *span)
> +{
> +	struct cpumask *msk;
> +
> +	guard(raw_spinlock_irqsave)(&p->pi_lock);
> +	if (READ_ONCE(p->__state) == TASK_DEAD || dl_task_bw_span(p) != span)
> +		return false;
> +	msk = this_cpu_cpumask_var_ptr(local_cpu_mask_dl);
> +	/* A task in a migrate_disable() section has a cpus_ptr of one CPU. */
> +	cpumask_and(msk, &p->cpus_mask, cpu_active_mask);
> +	cpumask_clear_cpu(cpu, msk);
> +	/* With no CPU left, ask which CPUs the cpuset gives @p. */
> +	if (cpumask_empty(msk) && !cpuset_cpus_allowed_without(p, cpu, msk)) {
> +		/*
> +		 * select_fallback_rq() picks the CPU. It gives @p the CPUs of
> +		 * @span if one of them is active and @p can run on all of them.
> +		 */
> +		if (cpumask_any_and_but(span, cpu_active_mask, cpu) < nr_cpu_ids &&
> +		    cpumask_subset(span, task_cpu_possible_mask(p)))
> +			return false;
> +	}
> +	return !cpumask_empty(msk) && !cpumask_intersects(msk, span);
> +}

[Severity: Medium]
Can an offline that invalidates an isolated partition (PRS_ISOLATED) still
move a deadline task from def_root_domain to a housekeeping root domain
without being refused?

Suppose CPU 1 is isolated at boot via isolcpus=domain,1, while CPU 0 and
CPU 2 are in housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT).  top_cpuset has
effective_cpus = {1, 2}, and a child PRS_ISOLATED partition P has
effective_cpus = {0} with a deadline task recorded on CPU 0 and counted in
def_root_domain (whose pre-offline span is {0, 1}).  When CPU 0 is taken
offline:

1. cpuset_offline_moves_dl(0) finds p = P, calls partition_child_lost()
   which returns NULL because P has no active effective_xcpus left, and
   returns false.

2. dl_task_leaves_span() gets msk = {1, 2} from top_cpuset via
   cpuset_cpus_allowed_without(), sees that msk intersects the pre-offline
   span {0, 1} at CPU 1, and returns false.

3. Once CPU 0 is offline, P is invalidated into top_cpuset and the task
   gets cpus_mask = {1, 2}.  During dl_rebuild_rd_accounting(),
   dl_task_rebuild_cpu() matches the recorded CPU against
   housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT):

kernel/sched/deadline.c:dl_task_rebuild_cpu() {
    ...
	if (recorded >= 0) {
		const struct cpumask *hk = housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT);
		bool rec_hk = cpumask_test_cpu(recorded, hk);
		unsigned int same;
    ...
		for_each_cpu(same, msk) {
			if (cpumask_test_cpu(same, hk) == rec_hk)
				return same;
		}
	}
    ...
}

Because CPU 0 was isolated by PRS_ISOLATED rather than at boot, rec_hk is
true, so dl_task_rebuild_cpu() picks housekeeping CPU 2 instead of
boot-isolated CPU 1 and moves the task's bandwidth into top_cpuset's
housekeeping root domain without admission control.

A similar move happens if top_cpuset holds only boot-isolated CPU 0 (with
a deadline task in top_cpuset) and an empty PRS_ISOLATED partition holds
housekeeping CPU 1: offlining CPU 0 invalidates the PRS_ISOLATED
partition so CPU 1 leaves def_root_domain to form top_cpuset's
housekeeping root domain, while dl_task_leaves_span() still tests against
the pre-offline def_root_domain span {0, 1}.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1795116790.git.mathieu.desnoyers@efficios.com?part=16

  parent reply	other threads:[~2026-11-19 20:48 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 20:15 [PATCH v8 00/33] sched/deadline: bandwidth accounting fixes and selftests Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 01/33] sched/deadline: Keep departing bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 02/33] sched/deadline: Rebuild the root domains' accounting from the task list Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 03/33] sched/deadline: Record the root domain holding a task's bandwidth Mathieu Desnoyers
2026-11-19 20:51   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 04/33] sched/deadline: Leave a dying task out of a rebuild of the accounting Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 05/33] selftests/sched: Test SCHED_DEADLINE bandwidth across a root domain rebuild Mathieu Desnoyers
2026-11-19 20:28   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 06/33] selftests/sched: Test the root domain rebuild in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 07/33] sched/deadline: Move a task's bandwidth only where room was made Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 08/33] cgroup/cpuset: Wait for an attach before rebuilding the sched domains Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 09/33] sched: Make deadline policy and affinity changes wait for a cpuset attach Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 10/33] cgroup/cpuset: Give back the bandwidth an attach did not move Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 11/33] cgroup/cpuset: Reserve the bandwidth in the root domain a task moves to Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 12/33] selftests/sched: Test cpuset moves in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 13/33] sched/deadline: Make a rebuild count a task in the root domain it runs in Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 14/33] sched/deadline: Move a task's bandwidth to " Mathieu Desnoyers
2026-11-19 20:53   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 15/33] sched/deadline: Keep a task in the root domain that counts its bandwidth Mathieu Desnoyers
2026-11-19 20:55   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 16/33] sched/deadline: Keep the last CPU a task has in its root domain Mathieu Desnoyers
2026-11-19 20:22   ` Mathieu Desnoyers
2026-11-19 20:48   ` sashiko-bot [this message]
2026-11-19 20:15 ` [PATCH v8 17/33] cgroup/cpuset: Refuse to take a deadline task out of " Mathieu Desnoyers
2026-11-19 20:43   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 18/33] selftests/sched: Test the root domain of a task in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:27   ` sashiko-bot
2026-11-19 20:15 ` [PATCH v8 19/33] sched/deadline: Count only active CPUs when spreading extra_bw Mathieu Desnoyers
2026-11-19 20:15 ` [PATCH v8 20/33] rtmutex: Continue the chain walk when a waiter's deadline donor changes Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 21/33] rtmutex: Continue the walk when the owner's top waiter changes lock Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 22/33] selftests/sched: Test the rtmutex chain walk in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 23/33] sched/deadline: Reset the donor's runtime at a deboost in the class Mathieu Desnoyers
2026-11-19 20:53   ` sashiko-bot
2026-11-19 20:16 ` [PATCH v8 24/33] sched/deadline: Replenish a throttled task boosted while asleep Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 25/33] sched/deadline: Count the bandwidth a boosted task is given Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 26/33] sched/deadline: Start a period for a task that enters the class blocked Mathieu Desnoyers
2026-11-19 20:57   ` sashiko-bot
2026-11-19 20:16 ` [PATCH v8 27/33] sched/deadline: Start a period at a deboost out of the class Mathieu Desnoyers
2026-11-19 20:47   ` sashiko-bot
2026-11-19 20:16 ` [PATCH v8 28/33] selftests/sched: Test deadline boosts in deadline_bw_test Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 29/33] sched/deadline: Cancel the replenishment timer with the parameters Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 30/33] sched/deadline: Cap the runtime left of a constrained task at dl_runtime Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 31/33] sched/deadline: Check the runtime of a task that comes back runnable Mathieu Desnoyers
2026-11-19 20:16 ` [PATCH v8 32/33] sched/deadline: Check the runtime of a task whose parameters change Mathieu Desnoyers
2026-11-19 20:59   ` sashiko-bot
2026-11-19 20:16 ` [PATCH v8 33/33] selftests/sched: Test parameter changes in deadline_bw_test Mathieu Desnoyers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=sashiko-outbox-224050@kernel.org \
    --to=sashiko-bot@kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-devel@lists.linux.dev \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mkoutny@suse.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®