From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B74D3093B5 for ; Wed, 17 Jun 2026 17:44:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781718293; cv=none; b=tVzloWA2vE08hzKHkK9/gu2UYkXLmdVL8knucUUVWj14hZx6gP3oNdib7xmXxhWBHspmzWD9T7pnLRBCbcHDNdPLj3C72FJmVbc4/x9OOOYzIvDwFFuL2+GrjGg2CzMEBxy6EZflUHqErJNvwo2qlmxYAfMuLQxZ9S7T3OuiYkk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781718293; c=relaxed/simple; bh=jxPntGD1TU4WzyA7JKiiH68VtqQ+4q+0ZaAnmddQEro=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qMZBYvobrn+JwReSLXmpqzbdZqOjFTHiUlFT4e1492YwUOYnGUm1NddD9PoCXzQPFuevXDMfX2GCpOBQEqC6qEf+PUWVTE6Fb/6IAFnqattc/kNg9gu+n1qQ+KGlusqjNcZ0ftAMU8U7PCJCpHTxGUO+Xhib64BsQNeEvrseMvw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=tStnj6Mh; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="tStnj6Mh" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 65HFpg1D744203; Wed, 17 Jun 2026 17:44:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=USXOcsAu8S4GLgRqV ltB+G1olKrFaC84EJ4DfKQSdlQ=; b=tStnj6Mh3HFnX2w8srDI8zCcGLeT8EWqW d569kBlzjcWK3pNL792gLjdGc0mC2ZNvgySiqh+02LsgndbWSLCvkRA4p41knfG4 8XuSeJSVNoUWpZNi/eHzJtl1usmap1fNH1bpvT6LT372M8aABL/JaFjKmEQi38KT /UdMW6G11CvP7PFR88TP0Nnvk6MnJvwjEi1GDg5jLrZjffT8nF+AroDeRJPyKFMa jOSVfh1Dra8XmB9I54YdCbRIwuLKxH1k1BVqIzVAjvORXuTgY7K//lJBCtgmMAME MHhTsbZhLBFV1Sd4hR8SXXWwN8UG5BBFctWATXIZQHjuQbUMWrzTw== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eueqx47cd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 17 Jun 2026 17:44:35 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 65HHYb1l012784; Wed, 17 Jun 2026 17:44:35 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4eudvambk1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 17 Jun 2026 17:44:35 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 65HHiV7u52953486 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 17 Jun 2026 17:44:31 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5CE6820049; Wed, 17 Jun 2026 17:44:31 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4DA8620040; Wed, 17 Jun 2026 17:44:24 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.39.28.77]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 17 Jun 2026 17:44:24 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com Cc: sshegde@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, mgorman@suse.de, bsegall@google.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org Subject: [PATCH v4 16/20] sched/core: Introduce default arch handling code for inc/dec preferred CPUs Date: Wed, 17 Jun 2026 23:11:35 +0530 Message-ID: <20260617174139.155540-17-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260617174139.155540-1-sshegde@linux.ibm.com> References: <20260617174139.155540-1-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNjE3MDE2NiBTYWx0ZWRfX5G56ukJXoJpE TFaC2WYgdEpfyMOchQMhCrgMsOisVy9r++nv5Aopdp4cgw1s5QZN2u/w/T3tmOGp424plK7e14E j7w2WvbhSQTSA2+OF0pKw+v7r4WXiIHQwtgQmrIK8xfxMX62sud5NW8VS4gcNf6GXqQh84wGXD0 2jxPqZW0IOekpi2IFrAYIPDbAPKOcHkmgdhu3mvBxwZ41ub45jQ8y+Yq5hrD6SCxQmITQlro6Rf ihtQ/FbcXNz9bWPk1OgenjS2enu9XfS2kvSKfyW8f3VhN30Pk6xcPqaortVUKlfbB/VpI8/Eca6 i6XR4E1fnO7rPtuNoyv9qG5AzNo44a9psEnDX4QZKZJav9Saki5WdVp7OPEGwpT+Yhl3040GHDC K+EQffWLFGxRK3t+OGKaUt4prPPn0325HZqJh23r7Y41YKHticq2oB+CDriINF9dr78ptFzWIBR wwIj/QryGoPx6KvbGLg== X-Proofpoint-GUID: gvz26piQZar1-SN-jLzFOkTDt1-BhTTB X-Proofpoint-Spam-Info: AW1haW4tMjYwNjE3MDE2NiBTYWx0ZWRfX1DGZlscdJAGE FqN2AxM574XDwh8D45qoAJ3gbVWHQU2Tqh7uXljZF1XxbKJFEO2+k8o4hlE4qD+VVAp6XEKjUXg Gq/VslqerNvReezcdDDx2k/LTf2l/W4= X-Proofpoint-ORIG-GUID: -TGildwPsxrzZsV0Ogw_CDiG37RmxieQ X-Authority-Analysis: v=2.4 cv=auGCzyZV c=1 sm=1 tr=0 ts=6a32dd04 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=FelO9ux0wxsA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=pGLVZ1KIGr3R53PB0VQA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.125,FMLib:17.12.100.49 definitions=2026-06-17_02,2026-06-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 impostorscore=0 spamscore=0 lowpriorityscore=0 adultscore=0 malwarescore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2606170166 Define default handlers for high/low steal time. If arch has better decision logic, may override the default implementation. - If the steal time higher than threshold, reduce the number of preferred CPUs by 1 core. The last core in the intersection of active and preferred CPUs will be marked as non-preferred. Ensure at least one core is left as preferred always. - If the steal time lower than threshold, increase the number of preferred CPUs by 1 core. First active core which is not in cpu_preferred_mask will be marked as preferred. If all cores are already set to preferred, bail out. Increase/Decrease may need to modify the splicing across NUMA nodes. It is being kept simple for now. Signed-off-by: Shrikanth Hegde --- v3->v4: - active instead of online - added comment for enabling tick for nohz_full. include/linux/sched.h | 2 ++ kernel/sched/core.c | 61 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index 5b15353ed7ef..e435f3073ffc 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -2529,5 +2529,7 @@ struct steal_monitor_t { }; extern struct steal_monitor_t steal_mon; +void arch_dec_preferred_cpus(struct steal_monitor_t *sm, u64 steal_ratio); +void arch_inc_preferred_cpus(struct steal_monitor_t *sm, u64 steal_ratio); #endif #endif diff --git a/kernel/sched/core.c b/kernel/sched/core.c index f1a91021e357..c77045055604 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -11400,6 +11400,67 @@ void sched_init_steal_monitor(void) steal_mon.sampling_period_ms = 1000; /* once per second */ } +/* + * Default implementation of decrementing the preferred CPUs based on steal + * time. This is simple logic and decrease the preferred CPUs by 1 core. + * It takes out the last core in the active & preferred. + * + * Ensure at least one housekeeping core is always kept as preferred + * + * Could be overwritten by arch specific handling. + */ +#ifndef arch_dec_preferred_cpus +void arch_dec_preferred_cpus(struct steal_monitor_t *sm, u64 steal_ratio) +{ + int last_cpu, tmp_cpu; + int this_cpu = raw_smp_processor_id(); + + last_cpu = cpumask_last(cpu_preferred_mask); + + /* + * If the core belongs to the housekeeping CPUs, no action is + * taken. This leaves at least one core preferred always. + * This ensures at least some CPUs are available to run + */ + if (cpumask_equal(cpu_smt_mask(last_cpu), cpu_smt_mask(this_cpu))) + return; + + /* + * set tick bit for nohz_full CPU to push the task out. Once the tasks + * are pushed out, bit will be cleared + */ + for_each_cpu_and(tmp_cpu, cpu_smt_mask(last_cpu), cpu_active_mask) { + set_cpu_preferred(tmp_cpu, false); + if (tick_nohz_full_cpu(tmp_cpu)) + tick_nohz_dep_set_cpu(tmp_cpu, TICK_DEP_BIT_SCHED); + } +} +#endif + +/* + * Default implementation of incrementing preferred CPUs based on steal + * time. This is simple logic and increases the preferred CPUs by 1 core. + * It adds the first core in active & !preferred + * + * Nothing to do if active == preferred + * + * Could be overwritten by arch specific handling. + */ +#ifndef arch_inc_preferred_cpus +void arch_inc_preferred_cpus(struct steal_monitor_t *sm, u64 steal_ratio) +{ + int first_cpu, tmp_cpu; + + first_cpu = cpumask_first_andnot(cpu_active_mask, cpu_preferred_mask); + /* All CPUs are preferred. Nothing to increase further */ + if (first_cpu >= nr_cpu_ids) + return; + + for_each_cpu_and(tmp_cpu, cpu_smt_mask(first_cpu), cpu_active_mask) + set_cpu_preferred(tmp_cpu, true); +} +#endif + /* This is only a skeleton. Subsequent patches introduce more of it */ void sched_steal_detection_work(struct work_struct *work) { -- 2.47.3