From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8327137F013 for ; Mon, 23 Mar 2026 21:31:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774301519; cv=none; b=Zda0y/C93s6MxB1Gp63UnHV0vFane8wp0JYpe2UxGYR8PUiIDUxAEJ22b0SFCPWFcY2Cauhp3VuyxIUt3SlhO93ehtP+BeJ41DfhnpF/HAKGUv/97F/QkDbWpJrVOxucOwgmU5Qqzj9+yA6wZpv+w6Y0QPhfC1T1o5WY22JaWbA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774301519; c=relaxed/simple; bh=rCTYKmeWPO8/lRYYOrE45zaNLuxINPTEpj7GuEPfjjc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OqH76WI+t+RxV4XRw2oF1PmOEyxzX3vnjf0YBDe62/8j/3ARl0nParBnLZLOFIN47mz56NwpllLNp4xTxt8k4vHehE989mYaV2dceytX9UXuifpkapP6a1RKe/2ElJpWvCOjx1tthmPIQTAnzUHcv26xYXcnLZUl1K902/LB5Gw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RbBpWmLD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RbBpWmLD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CF18CC4CEF7; Mon, 23 Mar 2026 21:31:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1774301519; bh=rCTYKmeWPO8/lRYYOrE45zaNLuxINPTEpj7GuEPfjjc=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RbBpWmLDQGMCHtpOVXoObX43C3t710uKvzsqtk33xBiQYE/9joltJWIq0SQZNwa+O O6qoFCg8Bi6e3o6/ZBWSCNsTHtD2K7Qv6J/hyHS6J5MKcTOPXah2xrLoJk/nzFd2TM Y/PaVfi097oKdV75oaJXBUr913qxe++3Xj/kA1WhkIEe+D3+76rkAeydos7VyHNb98 GSwpLwFC1Ssn2/nhblN9bTDlK+C3mXX3gewxnE5zlgKZ+ZxVSHpEZung4CPXcmgYzq CfdqCRxTxv1DXpeBxSEK1hMYhFqEpc2184HAaH2IjSj0KmjXPdx9ihOwMQgBwaVudL hZr5FU03xOIQA== Date: Mon, 23 Mar 2026 22:31:56 +0100 From: Frederic Weisbecker To: sheviks Cc: Waiman Long , linux-kernel@vger.kernel.org Subject: Re: [QUESTION/REGRESSION] Unbound kthreads scheduled on nohz_full CPUs after commit 041ee6f3727a Message-ID: References: <98b598f7-cbc6-42bd-bd01-d077549628fd@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Le Mon, Mar 23, 2026 at 10:00:32AM +0800, sheviks a écrit : > Frederic Weisbecker 於 2026年3月23日週一 上午6:33寫道： > > Yes. The affinity is applied right after the first wake-up of the kthread. > > And this wake-up is supposed to happen right after the kthread creation > > because the only purpose of this first wake-up is to allow for calling > > kthread_park(), kthread_bind() or kthread_affine_preferred() between > > kthread_create() and wake_up_process(). > > > > So I'm wondering why you're facing such an issue. Because by the time > > you create cgroup isolated partitions, all kthreads should have performed > > their first wake-up already. > > > > Which kthread did you observe after cgroup setting that didn't perform its > > first wake-up? > > > > Thank you for the response. I have performed a more detailed > observation to track the migration behavior of these kthreads over > time. > > Setup and Procedure: > Booted with: nohz_full=1-7 rcu_nocbs=1-7 irqaffinity=0 (No isolcpus). > Monitored kthreads on CPUs 1-7 using: ps -eLo cpuid,comm | grep -e > COMM -e "^ *[1-7] " | grep -ve "/[1-7]$" -e "kworker/[1-7]:" -e nvme0q > > Initially, there were 30 kthreads residing on CPUs 1-7 right after boot. > Manually created /sys/fs/cgroup/isolated1.slice and configured > cpuset.cpus.exclusive and cpuset.cpus.partition=isolated. > > Migration Timeline: > Within 1 minute: rcuog/0 and rcuog/4 migrated back to CPU 0. > At 2 minutes: khungtaskd and jbd2/zram0-8 migrated. > At 8 minutes: kthreadd migrated. > At 17 minutes: pr/legacy migrated. The count dropped to 24 kthreads. > After 9 hours: 24 kthreads remain on CPUs 1-7. > > The 24 kthreads remaining on CPUs 1-7 after 9 hours: > CPUID COMMAND > 1 card0-crtc3 > 1 ksmd > 1 scsi_eh_4 > 1 scsi_eh_9 > 3 pool_workqueue_release > 4 card0-crtc0 > 4 kdevtmpfs > 4 rcu_exp_gp_kthread_worker > 4 scsi_eh_5 > 5 oom_reaper > 5 rcub/0 > 5 scsi_eh_0 > 5 scsi_eh_1 > 5 scsi_eh_2 > 5 scsi_eh_8 > 6 kswapd0 > 6 psimon > 6 scsi_eh_6 > 6 watchdogd > 7 card0-crtc1 > 7 card0-crtc2 > 7 psimon > 7 scsi_eh_3 > 7 scsi_eh_7 "ps -o cpuid" tells where the task is currently running or, if sleeping, where it ran last. The cpuids that belong to isolated CPUs you're observing on some kthreads are there because those tasks have slept the whole time since the cpuset isolated partition was created. Yet they have been correctly migrated to CPU 0 and that will be displayed on "ps -o cpuid" the next time those kthreads are woken up. Use taskset for more accurate information as to where a task is allowed to run. Thanks. -- Frederic Weisbecker SUSE Labs