mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: Li kunyu <likunyu@open-hieco.net>
Cc: mingo@redhat.com, juri.lelli@redhat.com,
	vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
	rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
	vschneid@redhat.com, kprateek.nayak@amd.com,
	linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 0/6] sched/fair: Introduce the SMT_BUDDY_ASSIST feature
Date: Thu, 22 Oct 2026 09:40:44 +0200	[thread overview]
Message-ID: <20261022074044.GA4120091@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <20261020083746.37068-1-likunyu@open-hieco.net>

On Tue, Oct 20, 2026 at 04:37:40PM +0800, Li kunyu wrote:
> [Problem]
> 
> It was found that on servers equipped with the Hygon 7447v CPU (Model 7,
> 384 logical cores), migrating tasks across LLC scheduling domains during
> load balancing caused significant performance degradation:
> 
> 1. A low L3 cache hitting rate (due to factors such as non-shared L3
> caches).
> 
> 2. The traversal overhead of higher-level scheduling domains (e.g.,
> cross-NUMA scheduling domains).
> 
> Similar issues were also observed on CPUs from other vendors during
> testing, though the degree of memory bandwidth increase and other
> latency effects varied. Consequently, a series of patches was developed
> to address and improve these issues.
> 
> 
> [Solution]
> 
> This RFC patch series introduces the SMT_BUDDY_ASSIST scheduling
> feature. The main work is as follows:
> 
> 1. Push CFS tasks to SMT sibling CPUs during periodic load balancing.
> 
> 2. Skip SMT-domain balancing after a successful push and avoid pulling
> the pushed task back.
> 
> 3. Restrict newidle balancing so it does not pull from higher-level
> scheduling domains.
> 
> 4. Compensate for the resulting load imbalance with a burst-task wakeup
> path that sends tasks to idle CPUs.

So you're combatting undue LLC migrations with brute force :-( Mostly
this looks like something where cache aware scheduling should help, no?
That ought to make cross-LLC migrations less attractive.

Now, I realize that cache-aware scheduling as is won't help stream; when
build using OpenMP the whole thing is one big process and cache-aware
mostly gives up, when build using MPI its a ton of single threads and
cache-aware will ignore them.

Still, I think you really should look that way, perhaps investigate if
something like:

  https://patch.msgid.link/cover.1787955777.git.tim.c.chen%40linux.intel.com

can help you.




I'm not sure why you're doing the SMT push thing; but there's this
thing:

  https://patch.msgid.link/20261002154415.2270586-8-vincent.guittot%40linaro.org




WARNING: multiple messages have this Message-ID

From: Li kunyu <likunyu@open-hieco.net>
To: peterz@infradead.org, Li kunyu <likunyu@open-hieco.net>
Cc: bsegall@google.com, dietmar.eggemann@arm.com,
	juri.lelli@redhat.com, kprateek.nayak@amd.com,
	linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com,
	rostedt@goodmis.org, vincent.guittot@linaro.org,
	vschneid@redhat.com
Subject: Re: [RFC PATCH 0/6] sched/fair: Introduce the SMT BUDDY ASSIST feature
Date: Thu, 22 Oct 2026 18:08:21 +0800	[thread overview]
Message-ID: <20261022074044.GA4120091@noisy.programming.kicks-ass.net> (raw)
Message-ID: <20261022100821.981VXwAZyKKAsG00eaxoDO9kollDWxuTGq7ALzXMQDc@z> (raw)
In-Reply-To: <20261020083746.37068-1-likunyu@open-hieco.net>

From: Peter Zijlstra <peterz@infradead.org>

On Thu, Oct 22, 2026 at 03:40:44PM +0800, Peter Zijlstra wrote:
> On Tue, Oct 20, 2026 at 04:37:40PM +0800, Li kunyu wrote:
> > [Problem]
> > 
> > It was found that on servers equipped with the Hygon 7447v CPU (Model 7,
> > 384 logical cores), migrating tasks across LLC scheduling domains during
> > load balancing caused significant performance degradation:
> > 
> > 1. A low L3 cache hitting rate (due to factors such as non-shared L3
> > caches).
> > 
> > 2. The traversal overhead of higher-level scheduling domains (e.g.,
> > cross-NUMA scheduling domains).
> > 
> > Similar issues were also observed on CPUs from other vendors during
> > testing, though the degree of memory bandwidth increase and other
> > latency effects varied. Consequently, a series of patches was developed
> > to address and improve these issues.
> > 
> > 
> > [Solution]
> > 
> > This RFC patch series introduces the SMT_BUDDY_ASSIST scheduling
> > feature. The main work is as follows:
> > 
> > 1. Push CFS tasks to SMT sibling CPUs during periodic load balancing.
> > 
> > 2. Skip SMT-domain balancing after a successful push and avoid pulling
> > the pushed task back.
> > 
> > 3. Restrict newidle balancing so it does not pull from higher-level
> > scheduling domains.
> > 
> > 4. Compensate for the resulting load imbalance with a burst-task wakeup
> > path that sends tasks to idle CPUs.
> 
> So you're combatting undue LLC migrations with brute force :-( Mostly
> this looks like something where cache aware scheduling should help, no?
> That ought to make cross-LLC migrations less attractive.
> 
> Now, I realize that cache-aware scheduling as is won't help stream; when
> build using OpenMP the whole thing is one big process and cache-aware
> mostly gives up, when build using MPI its a ton of single threads and
> cache-aware will ignore them.
> 
> Still, I think you really should look that way, perhaps investigate if
> something like:
> 
>   https://patch.msgid.link/cover.1787955777.git.tim.c.chen%40linux.intel.com
> 
> can help you.

Thank you for your suggestion. I looked at this patch series and think
the way it exposes the interface to users ("always" | "advise" |
"never") is quite good. We will add the related interfaces in a subsequent
version.


> I'm not sure why you're doing the SMT push thing; but there's this
> thing:
> 
>   https://patch.msgid.link/20261002154415.2270586-8-vincent.guittot%40linaro.org

I previously looked at the relevant patches, and my implementation
approach was slightly different. Here, I mainly focused on quickly
distributing tasks to the SMT sibling CPUs; during this load-balancing
cycle I could still pull tasks, achieving faster CPU load balancing
(mainly for rapid local task dispersal, and it does not substantially
 conflict with the approach in that patch). In subsequent patches, I
also made new considerations: checking the cfs.h_nr_runnable count and
no longer focusing on sleeping tasks.

Task pushing at the SMT level is part of the overall functionality.
Combined with the changes in select_task_rq_fair(), it achieves rapid
overall task dispersal.

Combined with the aforementioned user interface, this makes it easy to
enable or disable SMT-level task pushing.


thinks,
kunyu

  parent reply	other threads:[~2026-10-22  7:40 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-20  8:37 Li kunyu
2026-10-20  8:37 ` [RFC PATCH 1/6] sched/fair: Add SMT sibling push in periodic load balance Li kunyu
2026-10-21 12:34   ` Peter Zijlstra
2026-10-20  8:37 ` [RFC PATCH 2/6] sched/fair: Add is_smt_buddy_run() to skip SMT domain Li kunyu
2026-10-21 12:35   ` Peter Zijlstra
2026-10-20  8:37 ` [RFC PATCH 3/6] sched/fair: SMT sibling exclusion after successful push Li kunyu
2026-10-21 12:39   ` Peter Zijlstra
2026-10-20  8:37 ` [RFC PATCH 4/6] sched/fair: Restrict newidle balance to LLC domain Li kunyu
2026-10-21 12:42   ` Peter Zijlstra
2026-10-20  8:37 ` [RFC PATCH 5/6] sched/fair: Add task burst threshold for large-scale bursts Li kunyu
2026-10-21 12:54   ` Peter Zijlstra
2026-10-20  8:37 ` [RFC PATCH 6/6] sched/docs: Add documentation for SMT Buddy Assist feature Li kunyu
2026-10-21 12:55   ` Peter Zijlstra
2026-10-21 12:44 ` [RFC PATCH 0/6] sched/fair: Introduce the SMT_BUDDY_ASSIST feature Kayra Cizmeci
2026-10-22  7:40 ` Peter Zijlstra [this message]
2026-10-22 10:08   ` [RFC PATCH 0/6] sched/fair: Introduce the SMT BUDDY ASSIST feature Li kunyu
2026-10-22 16:02     ` Chen Yu
2026-10-22 20:25 ` [RFC PATCH 0/6] sched/fair: Introduce the SMT_BUDDY_ASSIST feature Tim Chen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261022074044.GA4120091@noisy.programming.kicks-ass.net \
    --to=peterz@infradead.org \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=juri.lelli@redhat.com \
    --cc=kprateek.nayak@amd.com \
    --cc=likunyu@open-hieco.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®