Re: [PATCH v5 18/29] sched/deadline: Move a task's bandwidth only where room was made

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

Hi Waiman,

On 2026-11-12 23:37, Waiman Long wrote:
> For cpuset v2, it is not possible for a cpuset to span two root domains.
> I believe it is not possible for v1 too, though I am not 100% sure.
> Please let me know if you are aware of a way to do that.

It happens with isolcpus="domain". The CPUs it isolates stay in the
effective CPUs of the top cpuset, and of any cpuset that is not a
partition root. They are in the default root domain. The other CPUs
are in another root domain.

The message should have named isolcpus. The planned next version does.

This is on tip sched/urgent (v7.4-rc1 based), in a 5-CPU guest booted
with isolcpus=domain,1,2. A is a plain cgroup v2 cpuset:

  # cat /sys/devices/system/cpu/isolated
  1-2
  # echo 0-4 > A/cpuset.cpus
  # cat A/cpuset.cpus.effective
  0-4
  # cat A/cpuset.cpus.partition
  member

Two tasks in A call sched_setattr(SCHED_DEADLINE). Task 95 is pinned to
CPUs 0,3,4 and asks for 10%. Task 98 is pinned to CPUs 1,2 and asks for
20%. Both are admitted. /sys/kernel/debug/sched/debug gives total_bw of
the root domain of each CPU. "Before" is the fair servers alone:

  CPU     before    with the two tasks
  0,3,4   157284    262141   (+10%)
  1,2     104856    314571   (+20%)

So the two tasks are in the same cpuset and are counted in two root
domains. Then task 95 is given CPU 1 alone:

  # taskset -p 2 95
  taskset: can't set pid 95's affinity: Device or resource busy
  # echo -1 > /proc/sys/kernel/sched_rt_runtime_us
  # taskset -p 2 95
  pid 95's new affinity mask: 2

After that, total_bw is 157284 for CPUs 0,3,4 and still 314571 for
CPUs 1,2. The 10% of task 95 is counted nowhere. That is the case the
patch fixes.

I did not test cgroup v1.

The planned next version has a new cpuset patch for the same
configuration. A deadline task in such a cpuset can also change root
domain when its cpuset loses CPUs. The patch refuses that:
- a write to cpuset.cpus fails with EBUSY if it leaves a deadline task
  no CPU in the root domain that counts it;
- cpuset makes a partition root invalid if it would do the same.

My reply to Juri on patch 20 describes it with two related patches:
https://lore.kernel.org/all/e21aaf40-b929-444c-b707-3200e21d7301@efficios.com/

I would like your opinion on these two refusals before I post the
patch.

Thanks,

Mathieu