Re: [BUG] cpuset: WARNING in cpuset_update_sd_hk_unlock() when isolated partitions take every housekeeping CPU

[Waiman Long <longman@redhat.com>]

On 11/8/26 9:57 AM, Mathieu Desnoyers wrote:
> Hi,
>
> With isolcpus=domain, cpuset accepts isolated partitions that leave no
> housekeeping CPU. housekeeping_update() then refuses the new mask, and
> cpuset_update_sd_hk_unlock() warns.
>
> I found this while testing a fix for another cpuset problem [1]. Its
> sequence goes through that problem first, so I ran it only on a kernel
> with that fix.
>
> Trigger
> =======
>
> Boot 5 CPUs with isolcpus=domain,0,1. CPUs 2 to 4 are then the
> housekeeping CPUs. With cgroup v2:
>
>    cd /sys/fs/cgroup
>    echo +cpuset > cgroup.subtree_control
>    mkdir P Q
>    echo 4 > P/cpuset.cpus
>    echo isolated > P/cpuset.cpus.partition
>    echo 2-3 > Q/cpuset.cpus
>    echo root > Q/cpuset.cpus.partition
>    echo isolated > Q/cpuset.cpus.partition    # WARNING
>
> The last write is accepted. Q reads back as isolated.
>
>    WARNING: kernel/cgroup/cpuset.c:1481 at cpuset_update_sd_hk_unlock+0x79/0x80, CPU#1: k2/93
>    CPU: 1 UID: 0 PID: 93 Comm: k2 Not tainted 7.4.0-rc1-00268-ge99d0e004b05 #108 PREEMPT(lazy)
>    RAX: 00000000ffffffea
>    Call Trace:
>     cpuset_partition_write+0x9e/0xe0
>     kernfs_fop_write_iter+0x14d/0x210
>     vfs_write+0x252/0x550
>     ksys_write+0x6c/0xe0
>     do_syscall_64+0x109/0x5d0
>
> RAX is -EINVAL, the return of housekeeping_update().
>
> Analysis
> ========
>
>  From reading tip sched/urgent (006991aa8997):
> - Making Q isolated would remove CPUs 2 and 3 from HK_TYPE_DOMAIN.
>    CPUs 0 and 1 are isolated at boot, and P holds CPU 4. So no
>    housekeeping CPU would remain.
> - housekeeping_update() refuses that mask with -EINVAL. Commit
>    03ff73510169 ("cpuset: Update HK_TYPE_DOMAIN cpumask from cpuset")
>    calls it under WARN_ON_ONCE() in cpuset_update_sd_hk_unlock().
> - prstate_housekeeping_conflict() refuses such a partition only with
>    nohz_full, since commit 103b08709e8a ("cgroup/cpuset: Fail if
>    isolated and nohz_full don't leave any housekeeping").
>
> So cpuset and the housekeeping code disagree on what is valid. The
> write returns 0, and the isolated CPUs of the top cpuset read 0-4. But
> by reading, HK_TYPE_DOMAIN keeps its old mask.
>
> Should cpuset refuse an isolated partition that leaves no
> HK_TYPE_DOMAIN housekeeping CPU, as it does for nohz_full? I did not
> write a patch, since you may prefer another place for the check.

As said in my previous email, cpuset will ensure that there will be at 
least one online housekeeping CPU left for generating sched domain for 
the top_cpuset.

Cheers,
Longman

>
> The analysis was written by an LLM coding assistant under my direction
> and review.
>
> Thanks,
>
> Mathieu
>
> [1] The empty sched domain report, sent along with this one.