From: Waiman Long <longman@redhat.com>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Juri Lelli <juri.lelli@redhat.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Michal Hocko <mhocko@suse.com>, Tejun Heo <tj@kernel.org>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
Valentin Schneider <vschneid@redhat.com>,
K Prateek Nayak <kprateek.nayak@amd.com>,
linux-kernel@vger.kernel.org
Subject: Re: [BUG] sched: BUG() in select_fallback_rq() at boot with isolcpus=domain on the boot CPU and CONFIG_CPUSETS=n
Date: Fri, 20 Nov 2026 12:31:45 -0500 [thread overview]
Message-ID: <d09eae60-9fb5-4e10-86c4-50914207c355@redhat.com> (raw)
In-Reply-To: <5a16dda3-f128-4109-a52b-f51e01309b35@redhat.com>
On 11/20/26 11:52 AM, Waiman Long wrote:
> On 11/19/26 11:03 AM, Frederic Weisbecker wrote:
>> Le Wed, Nov 18, 2026 at 10:55:35AM -0500, Waiman Long a écrit :
>>> On 11/18/26 9:49 AM, Frederic Weisbecker wrote:
>>>> Le Tue, Nov 17, 2026 at 10:13:02AM -0500, Waiman Long a écrit :
>>>>> On 11/17/26 9:48 AM, Mathieu Desnoyers wrote:
>>>>>> Hi,
>>>>>>
>>>>>> With CONFIG_CPUSETS=n, a kernel booted with isolcpus=domain,0
>>>>>> hits the
>>>>>> BUG() in select_fallback_rq() during boot. CPU 0 is the boot CPU. I
>>>>>> found it while testing a SCHED_DEADLINE series [1]. I then
>>>>>> reproduced
>>>>>> it on v7.4-rc2 alone. tip/master of 2026-11-17 and next-20261116
>>>>>> have
>>>>>> the same code. A proposed fix follows the scissors line.
>>>>>>
>>>>>> Trigger
>>>>>> =======
>>>>>>
>>>>>> Build x86-64 with CONFIG_CPUSETS=n. Boot a VM of 4 CPUs with
>>>>>> isolcpus=domain,0 or isolcpus=domain,0,1. The kernel panics before
>>>>>> init runs, in every boot.
>>>>>>
>>>>>> These boot fine:
>>>>>> - the same kernel with isolcpus=domain,1,2, or without isolcpus;
>>>>>> - a kernel with CONFIG_CPUSETS=y, with isolcpus=domain,0,1.
>>>>> I am now thinking about disallowing the boot CPU from being domain
>>>>> isolated.
>>>>> That will include using isolcpus=domain boot time isolation
>>>>> parameter as
>>>>> well as in setting up isolated cpuset partition. Will that be a
>>>>> simpler
>>>>> solution than fixing all these places where isolating the boot CPU
>>>>> will
>>>>> cause various problems that need to be fixed?
>>>> You can try that, hopefully nobody relies on that. nohz_full
>>>> supports the
>>>> boot CPU to be isolated but it's a heresy.
>>> The upcoming runtime modification of nohz_full CPU will not support
>>> the boot
>>> CPU
>> Agreed.
>>
>>> at least for x86, as the boot CPU cannot be made offline.
>> We should forbid that on all archs.
>>
>>> I would not
>>> recommend putting the boot CPU into nohz_full boot parameter, though
>>> we are
>>> not going to forbid it.
>> It's already supported and unfortunately we can't remove that as that
>> would
>> break kernel ABI.
>
> I just found that that CPU 0 will be automatically cleared from the
> nohz_full list in tick_nohz_init() for CONFIG_PM_SLEEP_SMP &&
> !CONFIG_PM_SLEEP_SMP_NONZERO_CPU kernels. IOW, that means most kernel
> will do CPU 0 cleariing. Although we don't explicitly say that CPU 0
> cannot be a nohz_full CPU, most kernels will not allow that.
Thinking about it, the clearing is done on the tick_nohz_full_mask which
is used by the timekeeping code and in
/sys/devices/system/cpu/nohz_full. So from a user point of view CPU 0
isn't in the nohz_full list. However, the HK_TYPE_KERNEL_NOISE cpumask
will also have CPU 0 missing. This is inconsistent. Should we fix that
by setting the bit 0 of HK_TYPE_KERNEL_NOISE cpumask when it is cleared
from the tick_nohz_full_mask?
Regards,
Longman
prev parent reply other threads:[~2026-11-20 17:31 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 14:48 Mathieu Desnoyers
2026-11-17 15:02 ` Peter Zijlstra
2026-11-18 17:35 ` Mathieu Desnoyers
2026-11-17 15:13 ` Waiman Long
2026-11-18 14:49 ` Frederic Weisbecker
2026-11-18 15:55 ` Waiman Long
2026-11-19 16:03 ` Frederic Weisbecker
2026-11-20 16:52 ` Waiman Long
2026-11-20 17:31 ` Waiman Long [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d09eae60-9fb5-4e10-86c4-50914207c355@redhat.com \
--to=longman@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=frederic@kernel.org \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®