From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A70C233CEA9 for ; Tue, 7 Apr 2026 19:21:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775589708; cv=none; b=dk2jFmDG1RyZqZnTcLtVwnDz1U5EAxEp9/Thn0/h5/BbAQbF4kGQs2AfdbrKwZpx2shpIN/PR6EeqkaNOt23XHxXrqv4RyQRxnF0UgLdWIfoiWu1EOV5QlDuCmJpBynuAIAUrz6tjRXAgnzXAZPpPpBH29Ra8Tr7AWMwM7jXMCs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775589708; c=relaxed/simple; bh=m7S2I9rKVjRdEaxnrkwb0vV70J5VBV4oktpUOB7SmH4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kGylYpxYr2KDQ8/50H2c/YDzescBRgieM/eObfiWIg+MTUp5+85JknB6UCmnlUq51AJDCXdqvbFe5fSJt8sjEYwF8AKFrxYNiTe5UdbWLKXWgPMlEu85qemAXm8kHMBZH6OK8Y5SLjyB+ze7KSqwCilNSsI+05Q5TQnEon9sZus= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=tB9Y+mbV; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="tB9Y+mbV" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 637EY9e72325939; Tue, 7 Apr 2026 19:21:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=dkN8iedzNu44DV9+2 K0uvztl6e//Dv31TD4FOwHFi5Y=; b=tB9Y+mbV5GNFKIRT8rYrYvSdcFxDNDmNl U2XSGlex0qak93W/gG9QAIK/cpcuGLUK/yhjn+4koHhgETpAVI9sh7WYi/SGlX+O GWcDocL9n0ibN3zv08KjeJXz49SUxdAZ1LsNWy+m8a7BjO9ubyUyrkmuUXmoGmPC MF0E2Dy8viFbhbrD28RsIN8ok+N+N0yL6T3KlJEDEETm3Dkqqj8ih1zMjCchTIMN ympJbcdNIvLcJFg+QiP2KPv8DhTD4YzWcIzBwjBtc4FAg89Xt9QM0HdU0ZlRWTHz DVGotcSn8En2zVtCMbPHxJtQeQU552HLgITZmpvxFeD8Y7cNBJvPg== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4dcn2kccv6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 07 Apr 2026 19:21:30 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 637Itgn9018946; Tue, 7 Apr 2026 19:21:29 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4dcme9cjy0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 07 Apr 2026 19:21:29 +0000 Received: from smtpav05.fra02v.mail.ibm.com (smtpav05.fra02v.mail.ibm.com [10.20.54.104]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 637JLPRK54264154 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 7 Apr 2026 19:21:25 GMT Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7C32A20043; Tue, 7 Apr 2026 19:21:25 +0000 (GMT) Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 27AAA20040; Tue, 7 Apr 2026 19:21:21 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.39.20.194]) by smtpav05.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 7 Apr 2026 19:21:20 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, tglx@linutronix.de, yury.norov@gmail.com, gregkh@linuxfoundation.org Cc: sshegde@linux.ibm.com, pbonzini@redhat.com, seanjc@google.com, kprateek.nayak@amd.com, vschneid@redhat.com, iii@linux.ibm.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, mgorman@suse.de, bsegall@google.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, joelagnelf@nvidia.com Subject: [PATCH v2 15/17] sched/core: Handle steal values and mark CPUs as preferred Date: Wed, 8 Apr 2026 00:49:48 +0530 Message-ID: <20260407191950.643549-16-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260407191950.643549-1-sshegde@linux.ibm.com> References: <20260407191950.643549-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDA3MDE3NSBTYWx0ZWRfXyn67EzmtPpjD LnLTH7o59MQh+ickx7ZM3iGWHRWNwAJM3L4Vz3nmcjiH1OOSCiGdCqAA6o5VYNvHcVh/DYzUApH gNiAm5VG/tcSSXyUwO3rnHrWclURBuuywoHe74/e4FFPH1rkCNqwLSYe7dMZrz2EITCMPOUs3d/ jQ1BI6h9QKgaKAxOfWZMyuLhyj/6ixO8VAgqW/m8Tgln6rSNoEUe3vasr8LwPi/qOQFhmZth6kM LsLnaAEP76ByInsgHF0Bak5LEJBU+BTFTqt6jVup5hPv94mzC6Vq4sVyCzLuHPpAy0lsYW6uyRJ vuwfpMTDiOUuXYbPZHopHpXRFrNRR/m2bSntD7WIllL2P3Cz9A/T021rhrHyo5sUd7K6g/LwPwK L97//0u6yuoFKUSk/d1yshhoPRACtCNgJAe5NZvZfWUH1KrNRx/mM/fNCWGgYE1oJQC3X8YJW2k NwFLtiBW54TR0MPQroA== X-Proofpoint-ORIG-GUID: 3_L5be_2DOcgE0SDwjhnGpp8Drfd3Eef X-Authority-Analysis: v=2.4 cv=e9k2j6p/ c=1 sm=1 tr=0 ts=69d5593a cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=_P_wQ3gu65oW4jFxmyEA:9 X-Proofpoint-GUID: 3ahv-g3OOHnV_5CYACRba9p2pxpCDHOZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-07_04,2026-04-07_05,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 malwarescore=0 suspectscore=0 spamscore=0 bulkscore=0 adultscore=0 priorityscore=1501 phishscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2604010000 definitions=main-2604070175 This is the main periodic work which handles the steal time values. - Compute the steal time by looking CPUTIME_STEAL across all online CPUs - Compute steal ratio. It is multiplied by 100 to handle the fractional values. - If the steal time higher than threshold, reduce the number of preferred CPUs by 1 core. The last core in the intersection of online and preferred CPUs will be marked as non-preferred. Ensure at least one core is left as preferred always. - If the steal time lower than threshold, increase the number of preferred CPUs by 1 core. First online core which is not in cpu_preferred_mask will be marked as preferred. If all cores are aleady set to preferred, bail out. Increase/Decrease may need to modify the splicing across NUMA nodes. It is being kept simple for now. Signed-off-by: Shrikanth Hegde --- kernel/sched/core.c | 52 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1c6fcf1ae4fe..6e2b733adf45 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -11349,15 +11349,65 @@ void sched_init_steal_monitor(void) steal_mon.sampling_period_ms = 1000; /* once per second */ } -/* This is only a skeleton. Subsequent patches introduce more of it */ void sched_steal_detection_work(struct work_struct *work) { struct steal_monitor_t *sm = container_of(work, struct steal_monitor_t, work); + int this_cpu = raw_smp_processor_id(); + u64 delta_steal, delta_ns, steal = 0; + u64 steal_ratio; ktime_t now; + int tmp_cpu; + + for_each_cpu(tmp_cpu, cpu_online_mask) + steal += kcpustat_cpu(tmp_cpu).cpustat[CPUTIME_STEAL]; /* Update the prev_time for next iteration*/ now = ktime_get(); + delta_steal = steal > sm->prev_steal ? steal - sm->prev_steal : 0; + delta_ns = max_t(u64, ktime_to_ns(ktime_sub(now, sm->prev_time)), 1); + sm->prev_time = now; + sm->prev_steal = steal; + +#ifdef CONFIG_SCHED_SMT + /* Multiply by 100 to consider the fractional values of steal time */ + steal_ratio = (delta_steal * 100 * 100) / (delta_ns * num_online_cpus()); + + /* If the steal time values are high, reduce one core from preferred CPUs */ + if (steal_ratio > sm->high_threshold) { + int last_cpu; + + cpumask_and(sm->tmp_mask, cpu_online_mask, cpu_preferred_mask); + last_cpu = cpumask_last(sm->tmp_mask); + + /* + * If the core belongs to the housekeeping CPUs, no action is + * taken. This leaves at least one core preferred always. + * This ensures at least some CPUs are available to run + */ + if (cpumask_equal(cpu_smt_mask(last_cpu), cpu_smt_mask(this_cpu))) + return; + + for_each_cpu(tmp_cpu, cpu_smt_mask(last_cpu)) { + set_cpu_preferred(tmp_cpu, false); + if (tick_nohz_full_cpu(tmp_cpu)) + tick_nohz_dep_set_cpu(tmp_cpu, TICK_DEP_BIT_SCHED); + } + } + + /* If the steal time values are low, increase one core as preferred CPUs */ + if (steal_ratio < sm->low_threshold) { + int first_cpu; + + first_cpu = cpumask_first_andnot(cpu_online_mask, cpu_preferred_mask); + /* All CPUs are preferred. Nothing to increase further */ + if (first_cpu >= nr_cpu_ids) + return; + + for_each_cpu(tmp_cpu, cpu_smt_mask(first_cpu)) + set_cpu_preferred(tmp_cpu, true); + } +#endif } void sched_trigger_steal_computation(int cpu) -- 2.47.3