Re: [PATCH v5 18/29] sched/deadline: Move a task's bandwidth only where room was made

[Waiman Long <longman@redhat.com>]

On 11/13/26 8:51 AM, Mathieu Desnoyers wrote:
> Hi Waiman,
>
> On 2026-11-12 23:37, Waiman Long wrote:
>> For cpuset v2, it is not possible for a cpuset to span two root domains.
>> I believe it is not possible for v1 too, though I am not 100% sure.
>> Please let me know if you are aware of a way to do that.
> It happens with isolcpus="domain". The CPUs it isolates stay in the
> effective CPUs of the top cpuset, and of any cpuset that is not a
> partition root. They are in the default root domain. The other CPUs
> are in another root domain.
>
> The message should have named isolcpus. The planned next version does.
Right, top_cpuset is the exception here. Boot isolated CPUs are not 
allowed in a root partition. They are only allowed in isolated 
partitions. So root partition other than the top one cannot span more 
than one root domain.
>
> This is on tip sched/urgent (v7.4-rc1 based), in a 5-CPU guest booted
> with isolcpus=domain,1,2. A is a plain cgroup v2 cpuset:
>
>    # cat /sys/devices/system/cpu/isolated
>    1-2
>    # echo 0-4 > A/cpuset.cpus
>    # cat A/cpuset.cpus.effective
>    0-4
>    # cat A/cpuset.cpus.partition
>    member
>
> Two tasks in A call sched_setattr(SCHED_DEADLINE). Task 95 is pinned to
> CPUs 0,3,4 and asks for 10%. Task 98 is pinned to CPUs 1,2 and asks for
> 20%. Both are admitted. /sys/kernel/debug/sched/debug gives total_bw of
> the root domain of each CPU. "Before" is the fair servers alone:
>
>    CPU     before    with the two tasks
>    0,3,4   157284    262141   (+10%)
>    1,2     104856    314571   (+20%)
>
> So the two tasks are in the same cpuset and are counted in two root
> domains. Then task 95 is given CPU 1 alone:
>
>    # taskset -p 2 95
>    taskset: can't set pid 95's affinity: Device or resource busy
>    # echo -1 > /proc/sys/kernel/sched_rt_runtime_us
>    # taskset -p 2 95
>    pid 95's new affinity mask: 2
>
> After that, total_bw is 157284 for CPUs 0,3,4 and still 314571 for
> CPUs 1,2. The 10% of task 95 is counted nowhere. That is the case the
> patch fixes.
>
> I did not test cgroup v1.
>
> The planned next version has a new cpuset patch for the same
> configuration. A deadline task in such a cpuset can also change root
> domain when its cpuset loses CPUs. The patch refuses that:
> - a write to cpuset.cpus fails with EBUSY if it leaves a deadline task
>    no CPU in the root domain that counts it;
> - cpuset makes a partition root invalid if it would do the same.

We already have pending cpuset patch that will disallow taking the last 
housekeeping CPU from the top_cpuset partition.  The partition will be 
disabled in that patch if users attempt to do so. So it is effectively 
changing the root domain to that of the parent cpuset.

As said above, a partition below the top_cpuset cannot span more than 
one root domain. Are you referring to the case that shrinking the cpuset 
CPU count to reduce the total DL bandwidth will fail with EBUSY when it 
is less than what the existing DL tasks require?

Thanks,
Longman
>
> My reply to Juri on patch 20 describes it with two related patches:
> https://lore.kernel.org/all/e21aaf40-b929-444c-b707-3200e21d7301@efficios.com/
>
> I would like your opinion on these two refusals before I post the
> patch.
>
> Thanks,
>
> Mathieu