Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Andrea Righi <arighi@nvidia.com>]

On Wed, Sep 09, 2026 at 12:06:06PM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 9/9/2026 11:56 AM, Andrea Righi wrote:
> > NVIDIA Olympus implements SMT with two symmetric processing elements (PEs).
> > When only one PE is active, the core operates in single-thread mode and
> > that PE can use the full core resources. When both PEs are active, the core
> > operates in two-thread mode and the PEs share those resources. This
> > behavior is common to SMT implementations, but Olympus is particularly
> > sensitive to brief sibling activations because returning from two-thread
> > mode to single-thread mode after a sibling becomes idle is not immediate.
> > As described by commit 293f9611ae735 ("sched/fair: Prefer fully idle cores
> > for NOHZ balancing"):
> > 
> >   Briefly activating an otherwise idle sibling can reduce the performance
> >   available to the other sibling and this effect does not necessarily end
> >   once the activated sibling becomes idle: after the ILB finishes and its
> >   CPU enters WFI, full single-thread performance is restored only after the
> >   sibling has remained idle for a qualification interval (10 Ki cycles on
> >   the tested Vera system).
> > 
> > That change prevents the NOHZ idle load balancer from unnecessarily waking
> > a sibling of a busy PE. However, ordinary task placement can still select
> > either sibling of an idle core and repeated changes of the active PE can
> > keep Olympus cores in two-thread mode despite little or no useful overlap
> > between the siblings.
> > 
> > This series makes PE0 the preferred sibling of an Olympus core using
> > SD_ASYM_PACKING and teaches the fair scheduler's idle-selection paths to
> > honor asymmetric SMT priority. The scheduler first selects a candidate CPU
> > and core according to its existing placement and capacity rules, then
> > chooses the highest-priority available sibling within that core. The
> > generic scheduler behavior is enabled only when an architecture supplies
> > an SD_ASYM_PACKING SMT domain.
> > 
> > PE0 and PE1 have equal steady-state capacity, the preference does not
> > identify a faster PE. PE0 is used only as a canonical choice when both
> > siblings are available. Consistently selecting the same sibling avoids
> > alternating the active PE across wakeups, lets PE1 remain idle for longer,
> > and allows more cores to remain in, or return to, full-resource
> > single-thread mode.
> > 
> > The series was tested on a two-node Vera system using an 88-thread
> > single-precision GEMM on the 88 physical cores of NUMA node 0.
> > 
> > With the workload allowed to choose either sibling of every core, observed
> > throughput improved from approximately 9.4 TFLOP/s on the baseline kernel
> > to approximately 10.1 TFLOP/s with this series applied. Repeated runs also
> > became more predictable because the workload consistently settled on PE0
> > while PE1 remained quiet.
> > 
> > Changes in v5:
> >  - Remove the redundant olympus_prefer_pe0 state (K Prateek Nayak)
> >  - Link to v4: https://lore.kernel.org/r/20260908082345.103087-1-arighi@nvidia.com
> 
> My replies failed to keep up with your speed on iterations :-)

Haha, sorry about that! :D

> 
> Since this is same as v4 and I had taken it for a spin on a
> x86 and an ARM64 server, feel free to include:
> 
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

And thanks!

-Andrea