From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1C1028469B for ; Sun, 22 Mar 2026 22:33:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774218821; cv=none; b=Nk7dgcMU3Bx8LzYZbudDdcX6PB1ec2BPjLL1Yv3evfDIxBsvJ//p5z2HTveBGJcMfiYgzXiGXvkTnhSKwBLSlZ2b0Kz5sygfdxtYjena7ZFSr+uKBGRrwBXUewDI4P9vAExbg276cy62ZZNQZEGzUIR8lCNXT7LcEpkA4KNRa/w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774218821; c=relaxed/simple; bh=miKFP2shdHF5cyEypJcB04TjvKCBOJizz7LbDlr45LE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=A0zGCsAZ1G5d4LPGGoREYBkjfP+15QfXyzSAE7qPTaKcoSzH/3c+Z6DxGjEr1KjYaKc08/XqL8e6iQyJfqDgqgnLOnIaYieqSyI+kPOXp5Mo8Ay2LIyMD9dKK14N2sXZ+AJj+Dm08OWY1JCj/ruZSJ5h7SCcFRdTK0XqN4SKRNM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=W1MQ+JT4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="W1MQ+JT4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E92D8C19424; Sun, 22 Mar 2026 22:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1774218821; bh=miKFP2shdHF5cyEypJcB04TjvKCBOJizz7LbDlr45LE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=W1MQ+JT430VI7LaulnNeaZIz09o3prE8RwWy3PAROLKn4/pN6977NCBgJFNe2upqm 00xa24Anp9xP7jbQO5GOdGxKyyh/JptTrV/y+NB6AqwkgoiDL4cqsrJX9FWCZ/rx/4 1nr7/r7N629Ej/pWGPgP9NAV4tf66O9ger1xSf5MCSzsXzCav30xKfA7vAqJTW1pB2 bCSe7faA7bT2z3U4Wj8/DmehFzSX6qnAtHFdd99kmFqfJhShDtc4YkEsm2Wowtii0m J/5yIzcv1x6a4WIW5ELX8w7t90VAoxtBqW/wUF+7qS5WRaz+QZ+j5NxbsuGBkWFxEN lPxrBolc5YaJw== Date: Sun, 22 Mar 2026 23:33:38 +0100 From: Frederic Weisbecker To: sheviks Cc: Waiman Long , linux-kernel@vger.kernel.org Subject: Re: [QUESTION/REGRESSION] Unbound kthreads scheduled on nohz_full CPUs after commit 041ee6f3727a Message-ID: References: <98b598f7-cbc6-42bd-bd01-d077549628fd@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Le Mon, Mar 23, 2026 at 01:48:07AM +0800, sheviks a écrit : > Hi Frederic, Waiman and maintainers, > > A quick follow-up on my previous report. After leaving the system idle > for a longer period, I made an observation that pinpoints the issue > more precisely. > > The cgroup v2 dynamic isolation does eventually work. I noticed that > the unbound kthreads are eventually migrated to the housekeeping CPU > (CPU 0), but only after they wake up from sleep and enter the running > state. This lazy migration highlights why commit 041ee6f3727a is > causing issues for setups using nohz_full= without isolcpus=: > > 1. At boot time, because isolcpus= is absent, the HK_TYPE_DOMAIN mask > includes the nohz_full CPUs. > > 2. When unbound kthreads are initially spawned or have their affinity > set, the new logic relies solely on HK_TYPE_DOMAIN. Consequently, they > are placed on the nohz_full CPUs and immediately go to sleep there. > > 3. They remain "trapped" on the isolated CPUs until a wake-up event > finally forces the scheduler to migrate them according to the updated > cgroup affinity. > > This brings the focus back to HK_TYPE_KTHREAD vs HK_TYPE_DOMAIN. While > HK_TYPE_DOMAIN might default to all CPUs without isolcpus=, > HK_TYPE_KTHREAD correctly excludes the nohz_full CPUs from the very > beginning. > > Is this "spawn on nohz_full and wait for wake-up to migrate" behavior > intended? Yes. The affinity is applied right after the first wake-up of the kthread. And this wake-up is supposed to happen right after the kthread creation because the only purpose of this first wake-up is to allow for calling kthread_park(), kthread_bind() or kthread_affine_preferred() between kthread_create() and wake_up_process(). So I'm wondering why you're facing such an issue. Because by the time you create cgroup isolated partitions, all kthreads should have performed their first wake-up already. Which kthread did you observe after cgroup setting that didn't perform its first wake-up? > To prevent these sleeping kthreads from polluting isolated > CPUs before cgroups can intervene, should the initial affinity check > still consider HK_TYPE_KTHREAD alongside or instead of HK_TYPE_DOMAIN? Well, domain isolation wants unbound kthreads to move away. And nohz_full doesn't make sense without domain isolation. So we should focus on making things work for HK_TYPE_DOMAIN. Thanks. -- Frederic Weisbecker SUSE Labs