From: Chen Yu <chen.yu@linux.dev>
To: Andrea Righi <arighi@nvidia.com>
Cc: Vincent Guittot <vincent.guittot@linaro.org>,
Rik van Riel <riel@surriel.com>,
Usama Arif <usama.arif@linux.dev>,
peterz@infradead.org, mingo@redhat.com, kprateek.nayak@amd.com,
juri.lelli@redhat.com, dietmar.eggemann@arm.com,
rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
vschneid@redhat.com, samuelcrossley@gmail.com,
linux-kernel@vger.kernel.org, stable@vger.kernel.org,
kas@kernel.org, Arnav Aghav <aghav@meta.com>
Subject: Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to select_idle_capacity()"
Date: Mon, 16 Nov 2026 21:00:14 +0800 [thread overview]
Message-ID: <avr-XumgYsrnA6vZ@three-body> (raw)
In-Reply-To: <avglp8nnc4u9OCb5@gpd4>
On Sat, Nov 14, 2026 at 10:11:03AM +0100, Andrea Righi wrote:
> Date: Sat, 14 Nov 2026 10:11:03 +0100
> From: Andrea Righi <arighi@nvidia.com>
> To: Vincent Guittot <vincent.guittot@linaro.org>
> Cc: Rik van Riel <riel@surriel.com>, Usama Arif <usama.arif@linux.dev>,
> peterz@infradead.org, mingo@redhat.com, kprateek.nayak@amd.com,
> juri.lelli@redhat.com, dietmar.eggemann@arm.com, rostedt@goodmis.org,
> bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
> samuelcrossley@gmail.com, linux-kernel@vger.kernel.org,
> stable@vger.kernel.org, kas@kernel.org, Arnav Aghav <aghav@meta.com>
> Subject: Re: [PATCH] Revert "sched/fair: Add SIS_UTIL support to
> select_idle_capacity()"
>
> On Sat, Nov 14, 2026 at 09:18:30AM +0100, Vincent Guittot wrote:
> > On Fri, 13 Nov 2026 at 20:32, Rik van Riel <riel@surriel.com> wrote:
> > >
> > > On Fri, 2026-11-13 at 07:31 -0800, Usama Arif wrote:
> > > >
> > > > On a 72-CPU NVIDIA grace arm64 server, small reported capacity
> > > > differences
> > > > (1013-1024) within the single LLC activate asymmetric selection. Both
> > > > the reference and affected kernels were observed with a zero scan
> > > > hint. The reference kernel ignored it and still searched the domain.
> > > > A latency-sensitive, wakeup-heavy production workload saw its
> > > > reported
> > > > minute p50 latencies average 129 ms on the reference kernel and 223
> > > > ms
> > > > on the affected kernel.
> > > >
> > > > Disabling SIS_UTIL reduced the aggregate involuntary context switch
> > > > rate of worker main threads from 71.6k/s to 13.3k/s; re-enabling it
> > > > raised the rate to 73.7k/s.
> > >
> > > Afaik, the asymmetrical selection is supposed
> > > to solve a very specific problem: when a task
> > > requires X amount of CPU capacity to run, we
> > > try to avoid scheduling it on a CPU that has
> > > only a fraction of X available.
> > >
> > > This creates the issue that a task that appears
> > > to "need" exactly X, because it's been running
> > > on a CPU with full available capacity, ends up
> > > not being scheduled at all when the available
> > > idle core has X-3% available capacity, because
> > > that CPU happens to spend 3% of its time handling
> > > interrupts, or running something else.
> >
> > The task will be scheduled on this idle cpu/core of (X-3% capacity) if
> > no other idle CPU with X capacity is available.
> > The selection in select_idle_capacity is:
> > - Idle core with enough capacity
> > - Idle core with enough capacity but doesn't fullfill uclamp_min
> > - Idle core that doesn't fit
> > - Idle CPU with enough capacity
> > - Idle CPU with enough capacity but doesn't fullfill uclamp_min
> > - Idle CPU that doesn't fit
>
> Right, and when the scan budget is exhausted select_idle_capacity()
> returns the best idle candidate found so far, even if it doesn't fit.
>
> However, in the zero-budget case, we return before examining any
> candidates in the scan domain. Relaxing the capacity-fit rule inside
> that scan wouldn't change the outcome, because we never reach that
> check.
>
> Maybe Rik's suggestion could instead help with the earlier checks in
> select_idle_sibling()? Accepting an idle target, previous or recently
> used CPU whose capacity is "close enough" could avoid reaching the
> zero-budget bailout.
>
> The tradeoff is that we'd accept a potentially weaker CPU immediately,
> without searching for a better one.
>
Do you something similar to cluster wakeup, give the wakeup a second chance to find an idle
CPU in the LLC. I suppose what you suggested is:
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 470ba0dc6719..d1f3b179013e 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9141,6 +9141,24 @@ static inline bool asym_fits_cpu(unsigned long util,
return true;
}
+static int asym_idle_retry(struct task_struct *p, int prev, int target,
+ int recent)
+{
+ if (choose_idle_cpu(target, p))
+ return target;
+
+ if (prev != target && cpus_share_cache(prev, target) &&
+ choose_idle_cpu(prev, p))
+ return prev;
+
+ if (recent != prev && recent != target &&
+ cpus_share_cache(recent, target) &&
+ choose_idle_cpu(recent, p))
+ return recent;
+
+ return target;
+}
+
/*
* Try and locate an idle core/thread in the LLC cache domain.
*/
@@ -9149,7 +9167,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
bool has_idle_core = false;
struct sched_domain *sd;
unsigned long task_util, util_min, util_max;
- int i, recent_used_cpu, prev_aff = -1;
+ int i, recent_used_cpu, recent, prev_aff = -1;
/*
* On asymmetric system, update task utilization because we will check
@@ -9206,6 +9224,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ recent = recent_used_cpu;
p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
@@ -9244,7 +9263,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
target = i;
goto select_smt_priority;
}
- return target;
+ return asym_idle_retry(p, prev, target, recent);
}
}
--
2.43.0
> Thanks,
> -Andrea
>
> >
> > >
> > > In that situation, it will be better to run the
> > > task on the CPU with 97% capacity right now,
> > > than to not run the task until a CPU frees up
> > > later.
> > >
> > > Things like SMT and power-driven hardware
> > > scaling complicate the matter further, because
> > > we simply cannot see the full amount of capacity
> > > that is available on each CPU.
> > >
> > > That makes me wonder if we should simply accept
> > > any CPU that's "close enough to X" when dealing
> > > with asymmetrical placement.
> > >
> > > Say, anything that's above 3/4 of the capacity
> > > of X would be considered good enough to run the task.
> > >
> > > That still prevents us from placing a CPU intensive
> > > task on a little core in a big.LITTLE system, or
> > > from placing a task on a CPU that spends 80% of its
> > > time handling interrupts, but it will allow a task
> > > to run right now, when the capacity imbalance is small.
> > >
> > > Does that make sense, or am I forgetting something
> > > important?
> > >
> > > --
> > > All Rights Reversed.
prev parent reply other threads:[~2026-11-16 12:59 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-13 15:31 Usama Arif
2026-11-13 17:28 ` Andrea Righi
2026-11-13 18:26 ` Usama Arif
2026-11-13 21:32 ` Andrea Righi
2026-11-14 9:06 ` Vincent Guittot
2026-11-13 19:31 ` Rik van Riel
2026-11-14 8:18 ` Vincent Guittot
2026-11-14 9:11 ` Andrea Righi
2026-11-16 13:00 ` Chen Yu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=avr-XumgYsrnA6vZ@three-body \
--to=chen.yu@linux.dev \
--cc=aghav@meta.com \
--cc=arighi@nvidia.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=kas@kernel.org \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=rostedt@goodmis.org \
--cc=samuelcrossley@gmail.com \
--cc=stable@vger.kernel.org \
--cc=usama.arif@linux.dev \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®