From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BF0231717F for ; Tue, 21 Apr 2026 05:07:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776748021; cv=none; b=k05QLkZQ2PHKQv+K7rP46t2LscT+VBwAJoPVwK3d/sReMd2wfKOrkZ/AqdE5OEMyf+Zgh4NQ4brsYXY6n2GyTDiT25fUS8tXZ2NgImdkGuRXUSBWpJqClJbtHg9iRczarw81OD6xxwPFsAZ2cQS4m75xz85AnRBNzBrkU6BtShE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776748021; c=relaxed/simple; bh=LbeUE+E3oYqxXFa/lZKLBs8qMgWamHK3xc0eqgdyzN8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AqxLj7st29sfZwr39uEEbf4HD3mT0sA1Cnz3Eet9j/lnATRyl5h1USv0JPtOML5VSSn/bD1SYoN5GpvsYPbs6sMAgTzbqMB8CpyLCU3yjDfn5xTd9eq8c4i5rTQfIt7D+vTkaGOpRT+DZyiNwSL1PGx1SJ5An6EhneMOnQs7Zok= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=H/wVnT28; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="H/wVnT28" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 63KLuIDa1333537; Tue, 21 Apr 2026 05:06:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2025-04-25; bh=914f6 r/QwsjKarzH+h4sMjm4K7Bye7FlescO+tqnoK8=; b=H/wVnT28tkAfhzuh/2qGO o4fbIZMKo/gHGPtajzYxU9L+mekLEk20WuIfVqnOa86MWdZPuI2p//7RDT0kQohp +dYvcHXfiCtfJQaPqQb4O1PgpUz9mFk5DC82eMNsDwrFC6o5rVX0ITvr8useuKLn KKn1DOm9hcds8ljOAf/h766o8cDy0iogUSdOqBnsHFC4tlU2DU7RWzpvnTQWjsJ+ Ybt0F0ZGPMjjYfPFSXvyQNPTEsT6vuBjPQuf9xWTGIYiozE9nDaWMVvBvfQMDfa6 W5bq2Alr1GBw2xnT6fRp5lWr0vocHVZIM4grp/GNO7kJEeE4cDGcAu7UuO6HKA9i w== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4dm27vvpnk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 21 Apr 2026 05:06:45 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.7/8.18.1.7) with ESMTP id 63L56RIg005544; Tue, 21 Apr 2026 05:06:44 GMT Received: from imran-metabox.au.oracle.com (dhcp-10-191-74-155.vpn.oracle.com [10.191.74.155]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 4dn1afjjd1-3; Tue, 21 Apr 2026 05:06:44 +0000 (GMT) From: Imran Khan To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org Cc: dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/2] sched/fair: distribute nohz ILB work across idle CPUs. Date: Tue, 21 Apr 2026 13:06:22 +0800 Message-Id: <20260421050622.19869-3-imran.f.khan@oracle.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260421050622.19869-1-imran.f.khan@oracle.com> References: <20260421050622.19869-1-imran.f.khan@oracle.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-21_01,2026-04-20_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 lowpriorityscore=0 bulkscore=0 spamscore=0 malwarescore=0 suspectscore=0 phishscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2604070000 definitions=main-2604210048 X-Authority-Analysis: v=2.4 cv=JYCMa0KV c=1 sm=1 tr=0 ts=69e705e5 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=jiCTI4zE5U7BLdzWsZGv:22 a=EIcjfB9IiI4px24ztqRk:22 a=yPCof4ZbAAAA:8 a=Z6aT2bJ7UOw33JPfh9YA:9 X-Proofpoint-ORIG-GUID: kGYD2yXwMApEwaVK_TewJ1WsH8fGYCom X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDIxMDA0OCBTYWx0ZWRfX1V6tPdpEldDZ kWwTzJxnKwHG9rFV7uBdR5EVf4apUQg2LDwXtCorgA9848lwB/KR6vHt7/CWEXRSF5836RrNLz/ yDRsNh/HrOfNqt2huGomgSWS1rFBakQGGUiONPCJ3iZT71aGmoIzSmtLpDCGMh0GiEWmFQwneXM diUGTuhKhY3swLqKinJN8s2rmZKflY8bymkrXVadOOQy4ctPZOGSUhXSJj9znGocumpVJhMAiKe nPFjblVMfZCkllEJYDheMjE1Ti5gMvEDRJwa+8VYRSiIPNrPD01PuA6i+zL+SXQeH60wVcS6GqV wuGT4p9mr2CvcQUI4Jv9BlCm2LOHvnvRvcaYhtR67TiXR8YyFocQuI05PrkBZUA6zx0ZpdTb6AV D5kol9CuIZbVnjFIISCWm8sqPApZIRvwqXzP57pwqeD9whVlz0kGPWhde7WoprXNbYwkovTM3Ho UjF6aA6+bfJ5YQNdlLw== X-Proofpoint-GUID: kGYD2yXwMApEwaVK_TewJ1WsH8fGYCom find_new_ilb() uses for_each_cpu_and() to iterate nohz.idle_cpus_mask from the lowest bit upward, returning the first idle housekeeping CPU it finds. This can (unfairly) select the lowest nohz idle CPU most of the times. Fix this by selecting nohz ILB CPU in a round robin way and thus distributing the nohz ILB work (which can be significant on large scale systems) across all eligible idle CPUs. Signed-off-by: Imran Khan --- kernel/sched/fair.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index bd35275a05b38..93bdb542ff714 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -7213,6 +7213,7 @@ static struct { cpumask_var_t idle_cpus_mask; int has_blocked_load; /* Idle CPUS has blocked load */ int needs_update; /* Newly idle CPUs need their next_balance collated */ + int ilb_cpu_last; /* Last CPU selected for nohz ILB */ unsigned long next_balance; /* in jiffy units */ unsigned long next_blocked; /* Next update of blocked load in jiffies */ } nohz ____cacheline_aligned; @@ -12420,13 +12421,17 @@ static inline int find_new_ilb(void) hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE); - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) { + for_each_cpu_wrap(ilb_cpu, nohz.idle_cpus_mask, nohz.ilb_cpu_last + 1) { + if (!cpumask_test_cpu(ilb_cpu, hk_mask)) + continue; if (ilb_cpu == smp_processor_id()) continue; - if (idle_cpu(ilb_cpu)) + if (idle_cpu(ilb_cpu)) { + nohz.ilb_cpu_last = ilb_cpu; return ilb_cpu; + } } return -1; -- 2.34.1