Re: [PATCH 1/4] sched/deadline: Fix stale root domain in cpuset bandwidth transfer

[Marek Szyprowski <m.szyprowski@samsung.com>]

On 03.11.2026 12:26, Juri Lelli wrote:
> dl_task_needs_bw_move() and set_cpus_allowed_dl() use task_rq(p)->rd
> to determine a task's current root domain. For a sleeping task, this
> is stale: it points to the rq of the CPU the task last ran on, which
> may belong to a different root domain after a cpuset move.
>
> This causes incorrect bandwidth accounting when a sleeping DL task is
> moved between exclusive cpuset partitions and back:
>
>   1. Move root->partition: task_rq(p)->rd = root, destination in
>      partition -> different -> bandwidth transferred correctly.
>   2. Move partition->root: task_rq(p)->rd is still root (task never
>      ran) -> looks like same domain -> no transfer. Partition keeps
>      stale bandwidth, root domain is short.
>
> The root domain then over-admits, and when the task exits total_bw
> underflows.
>
> Introduce dl_task_root_domain_cpu() helper that uses the task's
> cpus_mask (which still reflects the old/source cpuset at call time)
> to find the root domain where the bandwidth is actually accounted,
> instead of relying on the potentially stale task_rq(p)->rd.
>
> Use cpus_mask rather than cpus_ptr, as the latter may be temporarily
> narrowed to a single CPU by migrate_disable().
>
> Reported-by: Weiheng Qiu <weihengqiuu@gmail.com>
> Closes: https://lore.kernel.org/all/14508cb2-805f-41ac-9f6b-bfd7c3d7cb1c@gmail.com/
> Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
This patch landed recently in linux-next as commit b4a61fdc19c5
("sched/deadline: Fix stale root domain in cpuset bandwidth transfer").
I found that it triggers kernel BUG during CPU hot-plug tests on some
of my tests boards. Reverting it on top of next-20261104 (together with
dependent 41a7968022ca3) fixes the issue.

Here is the log from Amlogic Meson G12B based Odroid-N2 board
(arch/arm64/boot/dts/amlogic/meson-g12b-odroid-n2.dts):


root@target:~# for i in /sys/devices/system/cpu/cpu[1-9]; do echo 0 >$i/online; done; sleep 4; for i in /sys/devices/system/cpu/cpu[1-9]; do echo 1 >$i/online; done

psci: CPU1 killed (polled 0 ms)
psci: CPU2 killed (polled 0 ms)
psci: CPU3 killed (polled 0 ms)
psci: CPU4 killed (polled 0 ms)
psci: CPU5 killed (polled 0 ms)
CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
arch_timer: CPU2: Trapping CNTVCT access
CPU2: Booted secondary processor 0x0000000100 [0x410fd092]
cpufreq: cpufreq_policy_online: CPU2: Running at unlisted initial frequency: 999999 kHz, changing to: 1000000 kHz
------------[ cut here ]------------
kernel BUG at kernel/sched/deadline.c:3340!
Internal error: Oops - BUG: 00000000f2000800 [#1]  SMP
Modules linked in: ...
CPU: 2 UID: 0 PID: 1403 Comm: sugov:2 Tainted: G         C          7.4.0-rc1+ #16963 PREEMPT(full)
Tainted: [C]=CRAP
Hardware name: Hardkernel ODROID-N2 (DT)
pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : dl_task_root_domain_cpu+0x44/0x48
lr : dl_task_root_domain_cpu+0x28/0x48
sp : ffff8000853dbd00
x29: ffff8000853dbd00 x28: 0000000000000000 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: ffff0000f4768f40
x23: 0000000000000000 x22: ffff8000853dbdc8 x21: ffff0000f4776480
x20: ffff8000853dbe10 x19: ffff800082e2f000 x18: 00000000ffffffff
x17: 616974696e692064 x16: 657473696c6e7520 x15: ffff800083b338c0
x14: 0000000000000001 x13: 0000000000000000 x12: 0000000000000001
x11: 000000000000000a x10: 0000000000000000 x9 : 000000000000008e
x8 : ffff0000f47770c0 x7 : ffff00000f039db0 x6 : 0000000000000040
x5 : 0000000000000003 x4 : 0000000000000001 x3 : 0000000000000000
x2 : 0000000000000006 x1 : 0000000000000006 x0 : 0000000000000006
Call trace:
 dl_task_root_domain_cpu+0x44/0x48 (P)
 dl_task_needs_bw_move.part.0+0x18/0x60
 set_cpus_allowed_dl+0x2c/0xd8
 __set_cpus_allowed_ptr_locked+0x154/0x1f0
 __set_cpus_allowed_ptr+0x68/0xc0
 set_cpus_allowed_ptr+0x34/0x60
 kthread_affine_node+0xe4/0x10c
 kthread+0x124/0x128
 ret_from_fork+0x10/0x20
Code: f9400bf3 a8c27bfd d50323bf d65f03c0 (d4210000)
---[ end trace 0000000000000000 ]---
note: sugov:2[1403] exited with irqs disabled



> ---
>  kernel/sched/deadline.c | 31 ++++++++++++++++++++++++-------
>  1 file changed, 24 insertions(+), 7 deletions(-)
>
> diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
> index c0ebdcde5fe5d..68019dafbaf18 100644
> --- a/kernel/sched/deadline.c
> +++ b/kernel/sched/deadline.c
> @@ -3324,14 +3324,28 @@ static void task_woken_dl(struct rq *rq, struct task_struct *p)
>  	}
>  }
>  
> +/*
> + * Find the root domain where a DL task's bandwidth is currently
> + * accounted. For a sleeping task that was moved between cpusets
> + * without running, task_rq(p)->rd may be stale. Use the task's
> + * current cpus_mask (not yet updated to the new mask) to find the
> + * actual root domain. Use cpus_mask rather than cpus_ptr, which
> + * may be temporarily narrowed by migrate_disable().
> + */
> +static unsigned int dl_task_root_domain_cpu(struct task_struct *p)
> +{
> +	unsigned int cpu;
> +
> +	cpu = cpumask_first_and(&p->cpus_mask, cpu_active_mask);
> +	BUG_ON(cpu >= nr_cpu_ids);
> +	return cpu;
> +}
> +
>  static void set_cpus_allowed_dl(struct task_struct *p,
>  				struct affinity_context *ctx)
>  {
> -	struct rq *rq;
> -
>  	WARN_ON_ONCE(!dl_task(p));
>  
> -	rq = task_rq(p);
>  	/*
>  	 * Migrating a SCHED_DEADLINE task between exclusive
>  	 * cpusets (different root_domains) entails a bandwidth
> @@ -3339,16 +3353,16 @@ static void set_cpus_allowed_dl(struct task_struct *p,
>  	 * domain (see cpuset_can_attach()).
>  	 */
>  	if (dl_task_needs_bw_move(p, ctx->new_mask)) {
> -		struct dl_bw *src_dl_b;
> +		unsigned int src_cpu = dl_task_root_domain_cpu(p);
> +		struct dl_bw *src_dl_b = &cpu_rq(src_cpu)->rd->dl_bw;
>  
> -		src_dl_b = dl_bw_of(cpu_of(rq));
>  		/*
>  		 * We now free resources of the root_domain we are migrating
>  		 * off. In the worst case, sched_setattr() may temporary fail
>  		 * until we complete the update.
>  		 */
>  		raw_spin_lock(&src_dl_b->lock);
> -		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(task_cpu(p)));
> +		__dl_sub(src_dl_b, p->dl.dl_bw, dl_bw_cpus(src_cpu));
>  		raw_spin_unlock(&src_dl_b->lock);
>  	}
>  
> @@ -3358,10 +3372,13 @@ static void set_cpus_allowed_dl(struct task_struct *p,
>  bool dl_task_needs_bw_move(struct task_struct *p,
>  			   const struct cpumask *new_mask)
>  {
> +	unsigned int cpu;
> +
>  	if (!dl_task(p))
>  		return false;
>  
> -	return !cpumask_intersects(task_rq(p)->rd->span, new_mask);
> +	cpu = dl_task_root_domain_cpu(p);
> +	return !cpumask_intersects(cpu_rq(cpu)->rd->span, new_mask);
>  }
>  
>  /* Assumes rq->lock is held */
>
Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland