Re: [PATCH RFC 0/2] sched: Introduce idle SMT priority for asymmetric capacity systems

[Tim Chen <tim.c.chen@linux.intel.com>]

On Thu, 2026-10-08 at 10:30 +0200, Mete Durlu wrote:
> Summary
> ===========================================================================
> On systems with asymmetric CPU capacities, the scheduler prefers fully idle
> cores over idle SMT siblings of busy cores. This works generally well but
> virtualized platforms where low capacity cores should be avoided are not
> considered. Introduce a new config option and arch hook to prioritize
> SMT utilization.
> 
> Background
> ===========================================================================
> The scheduler has been moving toward better utilization of fully idle cores
> and avoiding SMT penalties. This trend (notably after commit 25a32e400a14
> ("sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection")
> broke the behavior s390 is relying on to concentrate workloads on its
> high-capacity cores. On s390 CPU capacities represent the CPUs runtime
> entitlement assigned by the hypervisor. The idle SMT siblings of busy
> high-capacity cores start to perform better than fully idle low-capacity
> cores as the whole machine(containing the logical partitions) starts
> approaching to a {fully,over}loaded state. Grouping load on high-capacity
> cores keeps shared low-capacity cores(which are shared more aggressively)
> idle longer, reducing noise to neighboring partitions and improving
> overall performance.
> 
> Approach
> ===========================================================================
> This series introduces SCHED_IDLE_SMT_PRIO config option and the
> sched_idle_smt_prio static branch, allowing architectures to treat idle
> SMT siblings of busy cores as equal candidates during asymmetric capacity
> load balancing.
> Static branch checks are placed at paths considering fully idle cores
> over idle SMT threads in presence of asymmetric CPU capacities within
> scheduling groups. Inserted checks mostly override hints for idle core
> selection or cause early exits favoring idle SMT siblings.
> One more static branch check is added to new task path in order to favor
> high capacity SMT siblings during initial task placement, therefore the
> tasks are immediately placed in high capacity SMT siblings instead of
> considering most idle but low capacity cores.
> 
> Architectures opt in by selecting ARCH_SUPPORTS_SCHED_IDLE_SMT_PRIO and
> implementing arch_needs_idle_smt_prio(), which is evaluated during each
> asym_cpu_capacity_scan() to track the state as runtime topology changes.
> 
> The second patch enables the feature for s390 when running on
> hardware-backed topology in an LPAR with vertical polarization active
> and system is approaching to a target state.
> Otherwise the branch stays disabled and the scheduler falls back to the
> standard idle-core preference.
> 
> No functional change on architectures that do not select
> ARCH_SUPPORTS_SCHED_IDLE_SMT_PRIO.
> 
> Performance Results
> ===========================================================================
> Since this change effects the {fully,over}loaded state it is difficult
> to test at the moment. Therefore there are no concrete numbers or
> metrics for now for s390.
> For architectures which do not opt in to this feature, no performance
> change is expected.
> 
> Considerations and Open Questions
> ===========================================================================
> The main goal for this series is adding a simple mechanism for
> architectures to switch between idle core and idle SMT priority while
> keeping the introduced footprint as small as possible. But there are
> some ideas and questions to consider;
> 
> 1. Should arch_needs_idle_smt_prio() hook be removed?

Hi Mete, 

One assumption in the series should be made explicit, and I think
it answers your question 1. 

The idea only works if two CPUs that the guest sees as SMT siblings
really run on the same physical core. "An idle sibling of a busy
high-capacity core" is cheaper than a fully idle low-capacity core
only because it uses a physical core the partition already has. That
holds on an LPAR, where hypervisor dispatches whole cores. It does not
hold in general for a guest whose hypervisor schedules vCPUs one at a
time, such as a typical KVM guest with "threads=2". There the guest's
sibling masks are just a description, and packing onto "siblings"
gives no benefit.

Nothing in the series checks this, and it is not documented:

- In patch 1, the default arch_needs_idle_smt_prio() returns true,
  and the Kconfig help does not mention that siblings must share a
  physical core. Any arch that selects
  ARCH_SUPPORTS_SCHED_IDLE_SMT_PRIO takes this on without being told.

- In patch 2, the hook checks only the topology mode:

> +bool arch_needs_idle_smt_prio(void)
> +{
> +     if (topology_mode != TOPOLOGY_MODE_HW)
> +             return false;
> +     return true;
> +}

Today s390 seems to be safe because sibling masks are
only built in HW mode.  It is the only place where this
precondition can be enforced.

So I would keep the hook. It is the only place where this
precondition can be enforced. Removing it and tying the feature to
"Kconfig + asymmetric capacities" would drop the check entirely.
Concretely:

1) Document the contract in the Kconfig help and above the hook,
   for example: "Only select this if CPUs reported as SMT siblings
   are always dispatched on the same physical core."

2) Make the s390 hook test the conditions from the cover letter,
   something like:

        return machine_is_lpar() &&
               topology_mode == TOPOLOGY_MODE_HW &&
               smp_cpu_mtid;

Tim

>    The architecture hook is there to allow for any sort of logic to
>    dynamically decide when to flip the mechanism, but it can also be
>    removed if everyone agrees that this behaviour is not something that
>    should be dynamically flipped. It can be simply tied to detection of
>    asymmetric capacities and selection of Kconfig option.
> 
> 2. Is there a simpler way to implement this mechanism?
>    The proposed approach is chosen as the other features effecting the
>    scheduler's behavior, use the same method. If there is a more
>    efficient way to implement the same mechanism I'd be glad to use
>    that instead.
> 
> 3. There are no performance measurements *yet*.
>    On s390 the ideal conditions for this mechanism to be beneficial
>    usually surface when the whole system is fully loaded and resource
>    sharing between the logical partitions starts to get expensive.
>    Creating such an environment requires time, therefore no benchmark
>    results are available yet.
> 
> ---
> base-commit: 587858367581b9c55c3690f4e63382ad622719d4
> 
> Mete Durlu (2):
>       kernel/sched: Introduce idle SMT priority
>       s390/topology: Enable SCHED_IDLE_SMT_PRIO
> 
>  arch/Kconfig                     | 14 +++++++++
>  arch/s390/Kconfig                |  1 +
>  arch/s390/include/asm/topology.h |  5 +++
>  arch/s390/kernel/topology.c      |  9 ++++++
>  include/linux/sched/topology.h   |  9 ++++++
>  kernel/sched/fair.c              | 67 +++++++++++++++++++++++++++++-----------
>  kernel/sched/sched.h             | 10 ++++++
>  kernel/sched/topology.c          | 17 ++++++++++
>  8 files changed, 114 insertions(+), 18 deletions(-)