From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 510B3316918 for ; Mon, 5 Jan 2026 10:40:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767609623; cv=none; b=HzHcTotWEpDSH7jbdihM2csoD7DQSj0Azobdk0M1XjbGqRfn4YWE5lhjYCdZov7JuWnKKtgzatBWnNCmHpP6DINRMXbmdw2VIh5OtrMkT3luVhIQtiPDQAHmO1zu/b87PT4bTToMZJXTM9wEE/NQqtWFgwEBQ/Ve6K/FTCS1wXo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767609623; c=relaxed/simple; bh=qvVU4a47hFpKwBKfKOAKkCJtebxHiNEqZE0zeMYR+MM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=gyY11FVkS3koXPYAWw6IRQg0GvtHGKimaxR2EUTyDgrlK04a1C05QAtixldGwokyqxJJoGEM2OIK04fpDdI07g1dDRigfMZQyt7OG11oWsSm4nLr/nuR1Fmaogc6EIg/l8A0o6LwUkOikiGWcl9xnXTQy3Pez9oOU2Q51fBeKe8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FLlCg/0L; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FLlCg/0L" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 604HNQfC028326; Mon, 5 Jan 2026 10:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=HUjDtC 8T6eK9MKiwKmpftjsu72nsceVK3d49OgkEnpg=; b=FLlCg/0Lz2o0+6OQKmrFkw XCFD8k5LUo/A2+J01t+/9Cj1ByBZlalfQJtd7R7GWGSN8YzFDmgWH8PVjVMtQ+KQ SquIOIC/LOxhtCSFn4Mg/3XCvrkvkhX2zA6wdGP7IGNOFNyKfSn1e60stdHkP7Qf fny44VDKHNBPlDf+IrzNcpsTLmCoscHmcS8s7Q3ohrfF7ipKTPV35CjMN1Teaxv6 ZUj4pSi3TZ9PQORbfG8k23Bni+gRXJsh8aRgmoPs8aoqrJFucWRuG5zAsvwq/0Q6 K80uck1NiXBKpXOMWXRKEELKls4GDbPl8S1FCRCrNMPVmYKZJWtkDczB0O84XpnA == Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4betrte4w6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:40:04 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 605ABkcU023511; Mon, 5 Jan 2026 10:40:03 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bg3rm1uew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Jan 2026 10:40:03 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 605Ae1rR36831598 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Jan 2026 10:40:01 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7440D2004E; Mon, 5 Jan 2026 10:40:01 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDBAC20040; Mon, 5 Jan 2026 10:39:58 +0000 (GMT) Received: from [9.39.29.28] (unknown [9.39.29.28]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 5 Jan 2026 10:39:58 +0000 (GMT) Message-ID: <71485a68-c0b0-446a-8326-7c10c583e076@linux.ibm.com> Date: Mon, 5 Jan 2026 16:09:57 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/3] sched/fair: Change likelyhood of nohz.nr_cpus and do stats update if its due To: K Prateek Nayak Cc: linux-kernel@vger.kernel.org, juri.lelli@redhat.com, vschneid@redhat.com, tglx@linutronix.de, dietmar.eggemann@arm.com, anna-maria@linutronix.de, frederic@kernel.org, wangyang.guo@intel.com, mingo@kernel.org, peterz@infradead.org, vincent.guittot@linaro.org References: <20260102124744.360872-1-sshegde@linux.ibm.com> <20260102124744.360872-3-sshegde@linux.ibm.com> <880e3610-dbdb-42a8-9ddb-ab2e7d3cdc1f@linux.ibm.com> Content-Language: en-US From: Shrikanth Hegde In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=aaJsXBot c=1 sm=1 tr=0 ts=695b9504 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=eJTnxkHzwNBm_rMR:21 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=9lgnc0N0y7-2AwFzhlsA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: W604vsDKEZDHHDadeBX04odRyiI6I1XQ X-Proofpoint-ORIG-GUID: W604vsDKEZDHHDadeBX04odRyiI6I1XQ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA1MDA5MSBTYWx0ZWRfXyiS5x1XTCD5Z rd0wDRAz5pNtDTbcq7dJIEsefHT6N2mkII5XSqKIl7Zuw8h+R6jS/q4kx3kaakWT3whrh4/AAIj s8a4QJpCHOTdP8aPRMwyRLN6B2tzmLPojOnhWtBZtW4yemyNPu9T5h2dP2e5as6yt4jNF+7yvn3 uU/dOYe2QgMf32rBX3Hncf6mpexi70GmbYpFOyz59ZEbP+mq1IH/+J6cEChb0dMmBCxIVTx30IU L3Uq1a/8gCXXbFCLbzaBxjG5OVTICl1tcLLatnnCCKRA+N7g6fW0rjRMCjZ7e58mBr6aFuVeFpu 1EntrQ5lg9V+rf6qJUPoroLqeOemtY3l9zYsLo8J1Ou+ldwLCrRrh86NAw6pmS1rZZWZ1xOvVG/ NXmziZiareoaRTPH5OL7R/mbdfRp7rkxAVFp5N3CEqn4ncFuyWYReqE7/Xxl8BwYerEo930P5oI F2gRe3Hboj+xzoJBcaw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-05_01,2025-12-31_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 phishscore=0 spamscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601050091 Hi Prateek, > > On 1/5/2026 10:37 AM, Shrikanth Hegde wrote: >>>> --- a/kernel/sched/fair.c >>>> +++ b/kernel/sched/fair.c >>>> @@ -12456,10 +12456,10 @@ static void nohz_balancer_kick(struct rq *rq) >>>>         /* >>>>        * None are in tickless mode and hence no need for NOHZ idle load >>>> -     * balancing: >>>> +     * balancing, do stats update if its due >>>>        */ >>>> -    if (likely(!atomic_read(&nohz.nr_cpus))) >>>> -        return; >>>> +    if (unlikely(!atomic_read(&nohz.nr_cpus))) >>>> +        goto out; >> >> Did something got edited here? > > Welp! Stray edit. My bad. Reverted back to original diff. > >> >>> Since we are sure that "nohz.nr_cpus" is 0, there is a good chance >>> find_new_ilb() in kick_ilb() will not find any CPU to run balance on, so >>> why not just retain that return? >>> >>> The "flags" can only be set to (NOHZ_NEXT_KICK | NOHZ_STATS_KICK) on >>> this path and kick_ilb() will simply return early without updating >>> "nohz.next_balance" when it doesn't see NOHZ_BALANCE_KICK and fails to >>> find any CPU. Might as well keep the early return. >>> >> >> The only reason why flags would be set is, if nohz.has_blocked_load >> is set and time is after next_blocked. In that case, doing a stats >> based balance will make nohz.has_blocked_load=0 and subsequent invocations >> flags =0 and no load balance will happen if nr_cpus stays 0. >> >> However, if we just, has_blocked_load might remains stale value. >> >> Isn't that the case? > > So cumulatively, including Patch 3, we do: > > flags = 0; > > if (READ_ONCE(nohz.has_blocked_load) && ...) > flags = NOHZ_STATS_KICK; > > if (time_before(now, nohz.next_balance)) > goto out; /* Checks nohz.idle_cpus_mask in find_new_ilb() ... (1) */ > > if (unlikely(cpumask_empty(nohz.idle_cpus_mask))) > goto out; /* Still goes to kick_ilb() ... (2) */ > > ... > > out: > if (READ_ONCE(nohz.needs_update)) > flags |= NOHZ_NEXT_KICK; > > /* assume either NOHZ_STATS_KICK or NOHZ_NEXT_KICK is set */ > kick_ilb() > { > if (flags & NOHZ_BALANCE_KICK) /* Not possible */ > ... > > ilb_cpu = find_new_ilb(); /* Find CPU in nohz.idle_cpus_mask */ > > > If we arrive here from (2), we know "nohz.idle_cpus_mask" was empty a > while back and we've not updated any global "nohz" state. If we don't > find an ilb_cpu, we just do: > > if (ilb_cpu < 0) > return; > > So why not simply return from (2)? > I see, kick_ilb though called will not do a balance since ilb_cpu was not found. I don't want to have that return in between the two out's. How about we do below? When there are no idle CPUs left, both has_blocked_load and needs_update should be reset. no? diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 805b53d9709e..fa0e6065bc9c 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12377,6 +12377,15 @@ static inline int find_new_ilb(void) return ilb_cpu; } + /* There is no idle CPU left. + * reset has_blocked_load and needs_update, such that unless + * some CPU enters idle state, it will not trigger kick_ilb + */ + if (READ_ONCE(nohz.has_blocked_load)) + WRITE_ONCE(nohz.has_blocked_load, 0); + if (READ_ONCE(nohz.needs_update)) + WRITE_ONCE(nohz.needs_update, 0); + return -1; }