From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDADA3803FC for ; Tue, 13 Jan 2026 11:18:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768303114; cv=none; b=k+mEpjG7r4hVcQPRIRHjT5W9bAiFz7IWYZpwGSljhj7XsOemGg6ZCi1r+xCSMB+1zynq2ou9rxWaaKGnnVaqpWTGIdMRF8E6WG2dQivJv9U5Hl3mERarRgzRoRRfSurz3Sgvia9CQfWmcpY8iDUu8N05Nuvhrn0M5BtsFdWqW9M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768303114; c=relaxed/simple; bh=8CYkRyipAPbDobAjbgCGenbbJohY+DJ16z5WAClHhog=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=WcI2CqzHiKkmtNgJ4Y6+U4geu0RpmvSTHRfFt0lChMCcdOJSMcnDXD8tMyOfWiL5BRkigYbcm2YMosNGXnKoBt4sGmtr5j3rMn2gxAgQv9ZMzz364Q46InzAhorFqCA49PHPuhRgc2Ji2mztDqEm+wYfW6vusbKssvheRYIevnw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ilvzuZMk; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ilvzuZMk" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60D2elYe021077; Tue, 13 Jan 2026 11:18:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=psHlaj gNgFceIeplemgjcq6yQr0cs4/ResoGcYA23FU=; b=ilvzuZMkALFfm3C+etJaaU pS9+5GQJY14dGqYmJqLG96jjj9VLqPg/J6kuN5E1paKMhXZd0UwJd2fJjij5Iu8A x/VzW6irVBAhzGzBZi8DqbVdXbAVNPRqvlpNJnRxg5b/z69dJLHV4RRwJLOsmvzw FFWn8GrCZYUQyOGd0up6CnsSFIKgUVffLNVrPcPm1tFjYAEZCc6pbwf9yXrfKnnH yfWtsRGBju0gZynOCS+VCo1sEKzc7wA7rdTkGAkB909S6M9J/hF7CCkg2eQWfr7S Wd0EJ/eczPb9rtNDnrEb3PIMoyLmTcRoijI7rBsyk/FutI6Q+InqvuR1N7kTRPAw == Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bkeepv2xn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jan 2026 11:18:11 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60DAQghh025503; Tue, 13 Jan 2026 11:18:10 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bm23n3p97-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jan 2026 11:18:09 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60DBI8YM41681304 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Jan 2026 11:18:08 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E37D82004E; Tue, 13 Jan 2026 11:18:07 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9863620040; Tue, 13 Jan 2026 11:18:05 +0000 (GMT) Received: from [9.39.17.221] (unknown [9.39.17.221]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 13 Jan 2026 11:18:05 +0000 (GMT) Message-ID: <134bdd32-6ca9-4860-9c3b-786411fadbe9@linux.ibm.com> Date: Tue, 13 Jan 2026 16:48:04 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 1/3] sched/fair: Move checking for nohz cpus after time check To: Vincent Guittot Cc: mingo@kernel.org, peterz@infradead.org, linux-kernel@vger.kernel.org, kprateek.nayak@amd.com, juri.lelli@redhat.com, vschneid@redhat.com, tglx@kernel.org, dietmar.eggemann@arm.com, anna-maria@linutronix.de, frederic@kernel.org, wangyang.guo@intel.com References: <20260112050442.138446-1-sshegde@linux.ibm.com> <20260112050442.138446-2-sshegde@linux.ibm.com> From: Shrikanth Hegde Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=DI6CIiNb c=1 sm=1 tr=0 ts=696629f3 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=sa7JEVSvoXnGnPwfkx8A:9 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: c8OvqjKZhHa7KXURZu6fEWLVRgxXhmwt X-Proofpoint-ORIG-GUID: c8OvqjKZhHa7KXURZu6fEWLVRgxXhmwt X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTEzMDA5NiBTYWx0ZWRfX/dUm+6WHsJjD KoTps0LNj0TcKJcIXv96sZGvhahauUxEf+baxw8/ezvkTgAEF29Jj9KJlqhBieILLE3ffRkrhul KdcqjzSf33kw1V0c96kmbJ6VFSuzPq9j7/LSBB65qlo6Fwa0+A6ktV1Tp1qOouOKfDRF1fK68YP vWILen7567TmaHSrNeYlf3+6coEH+T61PhZvH/spddb1RLWlJ3bzRAP/bNhB1KQoFhQPnHlyP1U 4gGOfiLQ9Cw76o7FgFwJJDVy94sgCB//zscOnlnnxO6AUZ9XP8kZT+R9CDj38V5hswPGMmf35Ba rF3SCwz+yL1g/YHvOZ41GWkl5FdxExNfhOEbCzBSp2UMV3P4huCEJmUizYwdaJJCf9tnO8mMTWv XRXGxN55rBWFPIYMdZLA74+SoCAJ+/56eKG9SaweGWM1WzFl568iEfg7Pozmij+P0Vy4i0mJWwT jt6zefsH5+GQwM18h3w== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-13_02,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 phishscore=0 impostorscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601130096 On 1/13/26 3:51 PM, Vincent Guittot wrote: > On Tue, 13 Jan 2026 at 10:23, Shrikanth Hegde wrote: >> >> >> >> On 1/13/26 2:37 PM, Vincent Guittot wrote: >>> On Mon, 12 Jan 2026 at 06:05, Shrikanth Hegde wrote: >>>> >>>> NOHZ idle load balancer is kicked off only after time check. So move >>>> the atomic read after the time check to access it only when needed. >>>> >>>> When there are no idle CPUs(100% busy), even if the flag gets set to >>>> NOHZ_STATS_KICK | NOHZ_NEXT_KICK, find_new_ilb will fail and >>>> there will be no NOHZ idle balance. The current behaviour is retained. >>>> >>>> Note: This patch doesn't solve any cacheline overheads. No improvement >>>> in performance apart from saving a few cycles of atomic_read. >>> >>> But won't these cycles be then wasted by calling needlessly kick_ilb >>> >> >> when there are nohz cpus, i.e nohz.nr_cpus > 0, there is no change in codeflow. >> >> Only when system is 100%(which is expected to be rare), nohz.nr_cpus == 0, >> then it is expected that has_blocked_load = 0. So flags shouldn't be set. > > The way we are setting/clearing has_blocked_load vs > nr_cpus/idle_cpus_mask implies that it's possible to get > has_blocked_load == 1 but nr_cpus == 0 although it's a corner case and > not a default behavior > > No CPUs are idle: nr_cpus == 0 > > CPU 0 enters idle > - inc nr_cpus and set idle_cpus_mask > - set nohz.has_blocked > > CPU0 wakes up > > Tick fires on CPU0 > - dec nr_cpus and clear idle_cpus_mask > - nohz.has_blocked == 1, most probably now > nohz.next_blocked, if > now < nohz.next_balance, we skip the test of nr_cpus and we call > kick_ilb() but nr_cpus == 0 and idle_cpus_mask is empty > >> Note we are still doing a return if nohz.nr_cpus == 0. So kick_ilb shouldn't be >> called. > > The return can be skipped by if (time_before(now, nohz.next_balance)) goto out > Assuming HZ=1000, I see LOAD_AVG_PERIOD = 32, whereas next_balance is usually 60. so it is a really narrow window of 28 ticks and system being close to 100% busy. >> >> Do you see any path still calling kick_ilb un-necessarily? > > Yes but at the same time it's clearly not the main case > > So i assume we can do this patch considering the common case? If not, let me know, I can drop it.