Re: [RFC PATCH 0/6] sched/fair: Introduce the SMT BUDDY ASSIST feature

[Chen Yu <yu.c.chen@intel.com>]

Hi Kunyu,

On Thu, Oct 22, 2026 at 06:08:21PM +0800, Li kunyu wrote:
> From: Peter Zijlstra <peterz@infradead.org>
> 
> On Thu, Oct 22, 2026 at 03:40:44PM +0800, Peter Zijlstra wrote:
> > On Tue, Oct 20, 2026 at 04:37:40PM +0800, Li kunyu wrote:
> > > [Problem]
> > > 
> > > It was found that on servers equipped with the Hygon 7447v CPU (Model 7,
> > > 384 logical cores), migrating tasks across LLC scheduling domains during
> > > load balancing caused significant performance degradation:
> > > 
> > > 1. A low L3 cache hitting rate (due to factors such as non-shared L3
> > > caches).
> > > 
> > > 2. The traversal overhead of higher-level scheduling domains (e.g.,
> > > cross-NUMA scheduling domains).
> > > 
> > > Similar issues were also observed on CPUs from other vendors during
> > > testing, though the degree of memory bandwidth increase and other
> > > latency effects varied. Consequently, a series of patches was developed
> > > to address and improve these issues.
> > > 
> > > 
> > > [Solution]
> > > 
> > > This RFC patch series introduces the SMT_BUDDY_ASSIST scheduling
> > > feature. The main work is as follows:
> > > 
> > > 1. Push CFS tasks to SMT sibling CPUs during periodic load balancing.
> > > 
> > > 2. Skip SMT-domain balancing after a successful push and avoid pulling
> > > the pushed task back.
> > > 
> > > 3. Restrict newidle balancing so it does not pull from higher-level
> > > scheduling domains.
> > > 
> > > 4. Compensate for the resulting load imbalance with a burst-task wakeup
> > > path that sends tasks to idle CPUs.
> > 
> > So you're combatting undue LLC migrations with brute force :-( Mostly
> > this looks like something where cache aware scheduling should help, no?
> > That ought to make cross-LLC migrations less attractive.
> > 
> > Now, I realize that cache-aware scheduling as is won't help stream; when
> > build using OpenMP the whole thing is one big process and cache-aware
> > mostly gives up, when build using MPI its a ton of single threads and
> > cache-aware will ignore them.
> > 
> > Still, I think you really should look that way, perhaps investigate if
> > something like:
> > 
> >   https://patch.msgid.link/cover.1787955777.git.tim.c.chen%40linux.intel.com
> > 
> > can help you.
> 
> Thank you for your suggestion. I looked at this patch series and think
> the way it exposes the interface to users ("always" | "advise" |
> "never") is quite good. We will add the related interfaces in a subsequent
> version.
> 
>

If I understand correctly, the goal of your patch set is to inhibit cross-node
task migration so as to keep memory locality.

Patch 4 is the main one, which forbids the newly-idle cross-node balancing. As
a result of Patch 4, more tasks end up on a single node, so the kernel needs to
spread them within that node more aggressively, this is why Patches 1, 2, 3 and
5 were introduced.

Before digging further into the design, I think one question needs to be clarified:
In the scenario you are trying to optimize, do the workload's threads share data
with each other?

1. If the threads share data with each other, then leveraging cache-aware scheduling
   with prctl to put them together in the same LLC/node might be a good choice.
   Furthermore, there is a proposal [1] to enhance prctl to allow the user to specify
   how aggressively the threads should be aggregated, which would work in the "stream"
   case.

2. If the threads do not share data with each other and have a high memory footprint,
   then leaving them where their memory resides would be a good choice. This is what
   NUMA balancing tries to achieve, and it is possible that threads of the same process
   are spread on different nodes.

From the [Problem] section in the cover letter, it seems you are trying to address case 2?
If this is the case, why doesn't NUMA balancing help to inhibit the cross-node task migration?
Or is it because NUMA balancing is disabled on your platform?

Besides, there is a patch set to extend cache-aware scheduling to cover multiple
preferred LLCs and nodes [2], so as to mitigate cross-LLC/cross-node latency as much as
possible, not sure whether this work addresses the issue you mentioned.

[1] https://lore.kernel.org/lkml/tencent_93116D14C771DC8C988C10E3C634BE0CC107@qq.com/
[2] https://lore.kernel.org/lkml/20260827122816.756234-1-wujianyong@hygon.cn/

thanks,
Chenyu

> > I'm not sure why you're doing the SMT push thing; but there's this
> > thing:
> > 
> >   https://patch.msgid.link/20261002154415.2270586-8-vincent.guittot%40linaro.org
> 
> I previously looked at the relevant patches, and my implementation
> approach was slightly different. Here, I mainly focused on quickly
> distributing tasks to the SMT sibling CPUs; during this load-balancing
> cycle I could still pull tasks, achieving faster CPU load balancing
> (mainly for rapid local task dispersal, and it does not substantially
>  conflict with the approach in that patch). In subsequent patches, I
> also made new considerations: checking the cfs.h_nr_runnable count and
> no longer focusing on sleeping tasks.
> 
> Task pushing at the SMT level is part of the overall functionality.
> Combined with the changes in select_task_rq_fair(), it achieves rapid
> overall task dispersal.
> 
> Combined with the aforementioned user interface, this makes it easy to
> enable or disable SMT-level task pushing.
> 
> 
> thinks,
> kunyu