From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60D3C145FE0 for ; Tue, 7 Apr 2026 19:21:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775589670; cv=none; b=kUiKquGru4fS1hmvgSrzUfO/dg3/eoSE4jpJb1dkZIplsMjnbYsEzuSD6ESn12vLKZ7mw7ZGZ/KO77aRIFIaoiYvcrogE0JAssY2ItULEUIVhwMN2bMNHXoanTs2xqKeE00eP711ReMyziws/P1QNRw91HODOMHwFvlKd6RrgMI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775589670; c=relaxed/simple; bh=lkbdkpp41ihu8FtbIuOZZBS+nzDsWLZzUcFujLx73UI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=M8g6Yhb71eT7ppBs0wLTLwTJGXdrmIdslfHRySjXSXOv4f9PUjLznOIzhZ3i9QRtvUK5lMivB/6Hpp2oZAjB1xJ6LSC6OUY6cHdPoVdkO6KExP4kAnbBdnFODb2brouwNCaHwDEss+JZROuMU7sPnbki4sXPl2u7Mub4x7DKpyA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=TQmoVtvn; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="TQmoVtvn" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 637Ecno92211619; Tue, 7 Apr 2026 19:20:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:message-id:mime-version :subject:to; s=pp1; bh=vxK1fH7USVgZPktmvYSmmFVXpvkMKS6J8LZs2vUjr IY=; b=TQmoVtvn7xpX39rLJmOKthr3pDkQFPhXZkWgORjnQKw3zOF8CJqPU5b9u vAjRSDj1Yo7uwDYjvgafG+rrCv9UTgh7YKeH2eMaw0Q2/8bVHhabbVyYAL5jrKcA VNXozPcag5OnlEyA57cKdVqV8av5nmq0nCgmAi5ubslRXQM//yKNI5f12uyGj2oz kJwJOh+arVE2Rit0NFB6lO401CpsQdIRj1j4PLLaLQt9W2uJTrOlR6Cw/e1pn6Vm cTWo3PDzYYl2/1cjbNoQ6C0DKXwKJeFJLFF0yy4Q2WwLBP2+7vUhmG/YffBtz2uN fQsTNj76wrbTDANAT0cHHYTqC2sRw== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4dcn2hcccf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 07 Apr 2026 19:20:16 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 637IY3Ql013882; Tue, 7 Apr 2026 19:20:16 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4dcmf44js7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 07 Apr 2026 19:20:16 +0000 Received: from smtpav05.fra02v.mail.ibm.com (smtpav05.fra02v.mail.ibm.com [10.20.54.104]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 637JKBr730474728 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 7 Apr 2026 19:20:11 GMT Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A577120043; Tue, 7 Apr 2026 19:20:11 +0000 (GMT) Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10ADD20040; Tue, 7 Apr 2026 19:20:07 +0000 (GMT) Received: from li-7bb28a4c-2dab-11b2-a85c-887b5c60d769.ibm.com.com (unknown [9.39.20.194]) by smtpav05.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 7 Apr 2026 19:20:06 +0000 (GMT) From: Shrikanth Hegde To: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, tglx@linutronix.de, yury.norov@gmail.com, gregkh@linuxfoundation.org Cc: sshegde@linux.ibm.com, pbonzini@redhat.com, seanjc@google.com, kprateek.nayak@amd.com, vschneid@redhat.com, iii@linux.ibm.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, mgorman@suse.de, bsegall@google.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, joelagnelf@nvidia.com Subject: [PATCH v2 00/17] sched/paravirt: Introduce cpu_preferred_mask and steal-driven vCPU backoff Date: Wed, 8 Apr 2026 00:49:33 +0530 Message-ID: <20260407191950.643549-1-sshegde@linux.ibm.com> X-Mailer: git-send-email 2.51.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDA3MDE3NSBTYWx0ZWRfXw5wIZk/vTLNk eJWAopGeh+SQLkC3WYAtbos4gd1G1sYD5+1kBMr/nwAEaZXmEZWUg5TPaQ1hkFhsRlYJ3L4lvk3 cSPa1AzryxOGLn4AVRL78ZhN4tTw2IaeI4lpKp2yy+8NF09I5jHDS++3tyrGUWI6hruXMVqnQaY Q1L3fhTuMrVOZpu74o1zOQLpHWpEW/HwgOriUQt2F1MlMfesY9TKdbSzXbw+ri3hPws2XQQ23ho Ft3YoPWuQORt864os81eSPviJxrAmT7FES7kbYe7F4Zhj+EdDP+jD+D02yZwLedVwyiWS1sCO2Q jeGWFM/c0KKZLKP62MnLy6h2nbzQIJ3OLWx11Yi81DOE1ItBi+OenBewcx0t8ftSFYWuuL+TEHq bRSVfdPRj2QCMyXr7p8IYtOKqRLNrLfzl4wMcZI4506kmLVYb/YVzLZ6QSIhl67vB079P6sEzv5 tCfT1i8bEn7A5X/6INA== X-Proofpoint-GUID: l85K-tnImA7QPLGMJZiuO06cD7xA468Z X-Authority-Analysis: v=2.4 cv=a/wAM0SF c=1 sm=1 tr=0 ts=69d558f1 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=vnREMb7VAAAA:8 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=YDXU0nAqK60JmZ1treQA:9 X-Proofpoint-ORIG-GUID: fN_tgCA-tJuZMOl5ol8XqNc-Kkp8Qg64 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-07_04,2026-04-07_05,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1011 spamscore=0 impostorscore=0 priorityscore=1501 phishscore=0 lowpriorityscore=0 adultscore=0 malwarescore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2604010000 definitions=main-2604070175 In the virtualized environment, often there is vCPU overcommit. i.e. sum of CPUs in all guests(virtual CPU aka vCPU) exceed the underlying physical CPU (managed by host aka pCPU). When many guests ask for CPU at the same time, host/hypervisor would fail to satisfy that ask and has to preempt one vCPU to run another. If the guests co-ordinate and ask for less CPU overall, that reduces the vCPU threads in host, and vCPU preemption goes down. Steal time is an indication of the underlying contention. Based on that, if the guests reduce the vCPU request that proportionally, it would achieve the desired outcome. The added advantage is, it would reduce the lockholder preemption. A vCPU maybe holding a spinlock, but still could get preempted. Such cases will reduce since there is less vCPU preemption and lockholder will run to completion since it would have disabled preemption in the guest. Workload could run with time-slice extention to reduce lockholder preemption for userspace locks, and this could help reduce lockholder preemption even for kernelspace due to vCPU preemption. Currently there is no infra in scheduler which moves away the task from some CPUs without breaking the userspace affinities. CPU hotplug, isolated CPUset would achieve moving the task off some CPUs at runtime, But if some task is affined to specific CPUs, taking those CPUs away results in affinity list being reset. That breaks the user affinities, Since this is driven by scheduler rather than user doing so, can't do that. So need a new infra. It would be better if it is lightweight. Core idea is: - Maintain set of CPUs which can be used by workload. It is denoted as cpu_preferred_mask - Periodically compute the steal time. If steal time is high/low based on the thresholds, either reduce/increase the preferred CPUs. - If a CPU is marked as non-preferred, push the task running on it if possible. - Use this CPU state in wakeup and load balance to ensure tasks run within preferred CPUs. For the host kernel, there is no steal time, so no changes to its preferred CPUs. So series would affect only the guest kernels. Current series implements a simple steal time monitor, which reduces/increases the number of cores by 1 depending on the steal time. It also implements a very simple method to avoid oscillations. If there is need a need for more complex mechanisms for these, then doing them via a steal time governors maybe an idea. One needs to enable the feature STEAL_MONITOR to see the steal time values being processed and preferred CPUs being set correctly. In most of the systems where there is no steal time, preferred CPUs will be same as online CPUs. I will attach the irqbalance patch which detects the changes in this mask and re-adjusts the irq affinities. Series doesn't address when irqbalance=n. Assuming many distros have irqbalance=y by default. Discussion at LPC 2025: https://www.youtube.com/watch?v=sZKpHVUUy1g *** Please provide your suggestions and comments *** ===================================================================== Patch Layout: PATCH 01: Remove stale schedstats. Independent of the series. PATCH 02-04: Introduce cpu_preferred_mask. PATCH 05-09: Make scheduler aware of this mask. PATCH 10: Push the current task in sched_tick if cpu is non-preferred. PATCH 11: Add a new schedstat. PATCH 12: Add a new sched feature: STEAL_MONITOR PATCH 13-17: Periodically calculating steal time and take appropriate action. ====================================================================== Performance Numbers: baseline: tip/master at 8a5f70eb7e4f (Merge branch into tip/master: 'x86/tdx') on PowerPC: powerVM hypervisor: +++++++++ Daytrader +++++++++ It is a database workload which simulates stock live trading. There are two VMs. The same workload is run in both VMs at the same time. VM1 is bigger than VM2. Note: VM1 sees 20% steal time, and VM2 sees 10% steal time with baseline. (with series: STEAL_MONITOR=y and Default debug steal_mon values) On VM1: baseline with_series Throughput 1x 1.3x On VM2: baseline with_series Throughput 1x 1.1x (with series: STEAL_MONITOR=y and Period 100, High 200, Low 100) On VM1: baseline with_series Throughput: 1x 1.45x On VM2: baseline with_series Throughput: 1x 1.13x Verdict: Shows good improvement with default values. Even better when tuned the debug knobs. +++++++++ Hackbench +++++++++ (with series: STEAL_MONITOR=y and Period 100, High 200, Low 100) On VM1: baseline with_series 10 groups 10.3 8.5 30 groups 40.8 25.5 60 groups 77.2 47.8 on VM2: baseline with_series 10 groups 8.4 7.5 30 groups 25.3 19.8 60 groups 41.7 36.3 Verdict: With tuned values, shows very good improvement. ========================================================================== Since v1: - A new name - Preferred CPUs and cpu_preferred_mask I had initially used the name as "Usable CPUs", but this seemed better. I thought of pv_preferred too, but left it as it could be too long. - Arch independent code. Everything happens in scheduler. steal time is generic construct and this would help avoid each architecture doing the same thing more or less. Dropped powerpc code. - Removed hacks around wakeups. Made it as part of available_idle_cpu which take care of many of the wakeup decisions. same for rt code. - Implement a work function to calculate the steal times and enforce the policy decisions. This ensures sched_tick doesn't suffer any major latency. - Steal time computation is gated with sched feature STEAL_MONITOR to avoid any overheads in systems which don't have vCPU overcommit. Feature is disabled by default. - CPU_CAPACITY=1 was not considered since one needs the state of all CPUs which have this special value. Computing that in hotpath is not ideal. - Using cpuset was not considered since it was quite tricky, given there is different versions and cgroups is natively user driven. v1: https://lore.kernel.org/all/20251119124449.1149616-1-sshegde@linux.ibm.com/#t earlier versions: https://lore.kernel.org/all/236f4925-dd3c-41ef-be04-47708c9ce129@linux.ibm.com/ TODO: - Splicing of CPUs across NUMA nodes when CPUs aren't split equally. - irq affinity when irqbalance=n. Not sure if this is worth. - Avoid running any unbound housekeeping work on non-preferred CPUs such as in find_new_ilb. Tried, but showed a little regression in no noise case. So didn't consider. - This currently works for kernel built with CONFIG_SCHED_SMT. Didn't want to sprinkle too many ifdefs there. Not sure if there is any system which needs this feature but !SMT. If so, let me know. Seeing those ifdefs makes me wonder, Maybe we could cleanup CONFIG_SCHED_SMT with cpumask_of(cpu) in case of !SMT? - Performance numbers in KVM with x86, s390. Sorry for sending it this late. This series is the one which is meant for discussion at OSPM 2026. Shrikanth Hegde (17): sched/debug: Remove unused schedstats sched/docs: Document cpu_preferred_mask and Preferred CPU concept cpumask: Introduce cpu_preferred_mask sysfs: Add preferred CPU file sched/core: allow only preferred CPUs in is_cpu_allowed sched/fair: Select preferred CPU at wakeup when possible sched/fair: load balance only among preferred CPUs sched/rt: Select a preferred CPU for wakeup and pulling rt task sched/core: Keep tick on non-preferred CPUs until tasks are out sched/core: Push current task from non preferred CPU sched/debug: Add migration stats due to non preferred CPUs sched/feature: Add STEAL_MONITOR feature sched/core: Introduce a simple steal monitor sched/core: Compute steal values at regular intervals sched/core: Handle steal values and mark CPUs as preferred sched/core: Mark the direction of steal values to avoid oscillations sched/debug: Add debug knobs for steal monitor .../ABI/testing/sysfs-devices-system-cpu | 11 + Documentation/scheduler/sched-arch.rst | 48 ++++ Documentation/scheduler/sched-debug.rst | 27 +++ drivers/base/cpu.c | 12 + include/linux/cpumask.h | 22 ++ include/linux/sched.h | 4 +- kernel/cpu.c | 6 + kernel/sched/core.c | 219 +++++++++++++++++- kernel/sched/cpupri.c | 4 + kernel/sched/debug.c | 10 +- kernel/sched/fair.c | 8 +- kernel/sched/features.h | 3 + kernel/sched/rt.c | 4 + kernel/sched/sched.h | 41 ++++ 14 files changed, 409 insertions(+), 10 deletions(-) -- 2.47.3