Re: [PATCH v4 0/6] sched: Fix cluster scheduling in the presence of asymmetric capacity

[Ricardo Neri <ricardo.neri-calderon@linux.intel.com>]

On Mon, Jun 08, 2026 at 06:37:41PM +0100, Christian Loehle wrote:
> On 6/8/26 13:57, Ricardo Neri wrote:
> > Hi,
> > 
> > This is v4 of the series. The most important change in this version is a
> > pre-work patch to fix a bug that surfaced after the SMT-aware asymmetric
> > CPU capacity patchset from Andrea and Prateek [1] was applied. This led me
> > to do more testing. Please read the changelog for details.
> > 
> > Cluster scheduling aims to maximize performance by spreading load across
> > clusters of CPUs that share mid-level resources [2]. It works well on
> > uniform systems, but it breaks down on topologies with big and small
> > cores arranged in clusters. As a result, it fails on several generations
> > of Intel processors already shipped and upcoming.
> > 
> > Consider the topology below of big (B) cores and clusters of small (s)
> > cores.
> >          ------   ------
> >          | B  |   | B  |   -----------------   -----------------
> >          |    |   |    |   | s | s | s | s |   | s | s | s | s |
> >          ------   ------   -----------------   -----------------
> >          | L2 |   | L2 |   |      L2       |   |       L2      |
> >          -------------------------------------------------------
> >          |                          L3                         |
> >          -------------------------------------------------------
> > 
> > On a partially busy system (one with idle CPUs; busy CPUs have one task
> > each), scheduling for asymmetric capacity ensures that misfit tasks land on
> > the big CPUs. The remaining tasks, misfit or not, run on the small CPUs.
> > When CONFIG_SCHED_CLUSTER is enabled, these remaining tasks are supposed to
> > be evenly spread among the small-CPU clusters. Today, this does not
> > happen.
> > 
> > Several issues in the load balancer prevent a small CPU in one cluster
> > from pulling tasks from another:
> > 
> >  a) update_sd_pick_busiest() may select a fully_busy group with higher
> >     per-CPU capacity as the busiest, preventing a subsequent fully_busy
> >     group of equal capacity from being correctly selected.
> >  b) Misfit-load statistics are used to identify tasks that would benefit
> >     from migrating to bigger CPUs. Accounting misfit load is pointless if
> >     the destination CPU is equally small, and it also blocks balancing
> >     between clusters.
> >  c) Due to b), groups that are truly has_spare or fully_busy get
> >     misclassified as misfit_task. update_sd_pick_busiest() then skips
> >     them, since a small destination CPU cannot help with misfit tasks.
> >  d) Once a busiest group has been identified, sched_balance_find_src_rq()
> >     will refuse to migrate tasks to CPUs of equal capacity, even when
> >     doing so is precisely what is required to balance small-CPU clusters.
> >  e) The SD_PREFER_SIBLING flag is missing from scheduling domains with
> >     asymmetric capacity, preventing the balancer from equalizing load
> >     across sibling small-core clusters.
> > 
> > Together, these issues prevent cluster-level balancing on systems with
> > asymmetric CPU capacity.
> > 
> > This series addresses each problem and restores the intended behavior.
> > Details, rationale, and code changes are explained in each patch.
> > 
> > I tested these patches on Alder Lake, which has both SMT Pcores and
> > clusters of Ecores. I tested with SMT both disabled and enabled. I also
> > tested on Lunar Lake and Panther Lake, which have an Ecore cluster not
> > connected to the L3 cache. I repeated the same experiment with
> > CONFIG_SCHED_CLUSTER disabled. The load balancer behaves as expected.
> > 
> > Link: https://lore.kernel.org/all/20260509180955.1840064-1-arighi@nvidia.com/ [1]
> > Link: https://lore.kernel.org/r/20210924085104.44806-1-21cnbao@gmail.com/ [2]
> > 
> > Changes in v4:
> >  - Patch 1 (pre-work): Fixed a bug that would block load balancing on SMT
> >    cores with more than one busy sibling.
> >  - Patch 2 (pre-work): Fixed a bug that would needlessly update
> >    sg_overloaded.
> >  - Patch 5: Reworked logic using a local variable for improved
> >    readability.
> >  - Added Reviewed-by tags from Chen Yu, Tim, and Vincent. Thanks!
> >  - Link to v3: https://lore.kernel.org/r/20260514-rneri-fix-cas-clusters-v3-0-0037869554bd@linux.intel.com
> > 
> > Changes in v3:
> >  - Patch 3: Reverted the inverted runtime capacity check. The inverted
> >    form resulted in migrations to CPUs of slightly lower capacity. Guarded
> >    the check for architectural capacity with the sched_cluster_active
> >    static key.
> >  - Patch 4: Expanded the patch description to explain the behavior of
> >    overloaded groups and low-capacity clusters with spare capacity.
> >  - Added Reviewed-by tags from Christian. Thanks!
> >  - Link to v2: https://lore.kernel.org/r/20260429-rneri-fix-cas-clusters-v2-0-cd787de35cc6@linux.intel.com
> > 
> > Changes in v2:
> >  - Patch 1: Rewrote patch description for clarity. Added a note
> >    clarifying that SD_ASYM_CPUCAPACITY and SMT are mutually
> >    exclusive. (Tim)
> >  - Patch 2: Fixed a bug where the capacity check inadvertently broke
> >    the mutual exclusion of the sched_reduced_capacity() path. Keep
> >    marking the root domain as overloaded when misfit tasks are present
> >    to allow bigger CPUs to help via newly idle balance. (sashiko)
> >    Fixed the description to state that capacity_greater() looks for
> >    differences of ~5% or more, not 20%. (Christian)
> >  - Patch 3: Use arch_scale_cpu_capacity() instead of capacity_of() to
> >    ignore runtime capacity variability. Inverted the capacity check.
> >    (Christian)
> >  - Patch 4: Reworded the patch description for clarity.
> >  - Link to v1: https://lore.kernel.org/r/20260330-rneri-fix-cas-clusters-v1-0-1e465b6fecb2@linux.intel.com/
> > 
> > ---
> > Ricardo Neri (6):
> >       sched/fair: Do not skip CPUs of similar capacity with busy SMT siblings
> >       sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY
> >       sched/fair: Check CPU capacity before comparing group types during load balance
> >       sched/fair: Skip misfit load accounting when the destination CPU cannot help
> >       sched/fair: Allow load balancing between CPUs of identical capacity
> >       sched/topology: Do not clear SD_PREFER_SIBLING in domains with clusters
> > 
> >  include/linux/sched/sd_flags.h |  3 ++-
> >  kernel/sched/fair.c            | 57 +++++++++++++++++++++++++++++++-----------
> >  kernel/sched/topology.c        | 14 +++++++++--
> >  3 files changed, 56 insertions(+), 18 deletions(-)
> > ---
> > base-commit: 83313bb25a6ace43b0cb5bde881213e6cfb3b046
> > change-id: 20250620-rneri-fix-cas-clusters-bb4287d1e152
> > 
> > Best regards,
> 
> Since I don't really have an arm64 machine that hits the described case just
> right, I tested the series on a synthetic arm64 qemu topology with two
> equal-capacity little clusters and one 1024 cluster.
> 
> The guest was booted with QEMU virt, 8 CPUs and a custom dtb. The resulting
> topology is:
> cluster0: CPUs 0-1, cpu_capacity=446
> cluster1: CPUs 2-3, cpu_capacity=446
> cluster2: CPUs 4-7, cpu_capacity=1024
> The dtb describes the clusters with cpu-map. The test kernel was built with
> CONFIG_SCHED_CLUSTER enabled.
> 
> I used an rt-app workload with 8 (nr_cpus) SCHED_OTHER tasks.
> Each task used the same two phases:
> "pinned": {
>     "loop": 100,
>     "run": 99000,
>     "timer": { "ref": "unique", "period": 100000 },
>     "cpus": [0, 1, 4, 5, 6, 7]
> },
> "open": {
>     "loop": 100,
>     "run": 99000,
>     "timer": { "ref": "unique", "period": 100000 },
>     "cpus": [0, 1, 2, 3, 4, 5, 6, 7]
> }
> 
> The intent is to first force the workload onto cluster0 plus the big cluster,
> leaving the second little cluster unused. Then the affinity mask is opened to
> all CPUs. If load balancing across equal-capacity clusters works, CPUs 2-3
> should receive a meaningful share of the work (instead of only occasional
> migrations).
> 
> I counted rt-app sched_switch events per cluster in the open phase. The pass
> condition was that cluster1_little receives at least 20% of open-phase rt-app
> sched_switch events.
> 
> Results over three runs (for the open phases):
> 
> mainline:
> run0: cluster0 5.7%, cluster1 5.7%, big 88.6%  FAIL
> run1: cluster0 5.5%, cluster1 6.2%, big 88.3%  FAIL
> run2: cluster0 4.3%, cluster1 4.7%, big 91.0%  FAIL
> 
> with this series:
> run0: cluster0 38.6%, cluster1 31.4%, big 30.1%  PASS
> run1: cluster0 33.2%, cluster1 60.6%, big 6.3%   PASS
> run2: cluster0 33.3%, cluster1 60.6%, big 6.1%   PASS
> 
> (The pinned phase behaved as expected in all runs: there were no rt-app
> sched_switch samples on CPUs 2-3 before the affinity mask was opened.)
> 
> For the series (patch 1/6 is a different setup, so maybe except for that)
> Tested-by: Christian Loehle <christian.loehle@arm.com>

Many thanks for your tests! I have two questions: Do you see similar
results of you spawn less tasks than nr_cpus? Perhaps with 6 tasks? They
should continue to be evenly distributed on same-capacity clusters.

Also, are these high-utilization tasks? If yes, the high-capacity cluster
should be fully utilized before any tasks overflow to the lower-capacity
clusters.