[PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Andrea Righi]

NVIDIA Olympus implements SMT with two symmetric processing elements (PEs).
When only one PE is active, the core operates in single-thread mode and
that PE can use the full core resources. When both PEs are active, the core
operates in two-thread mode and the PEs share those resources. This
behavior is common to SMT implementations, but Olympus is particularly
sensitive to brief sibling activations because returning from two-thread
mode to single-thread mode after a sibling becomes idle is not immediate.
As described by commit 293f9611ae735 ("sched/fair: Prefer fully idle cores
for NOHZ balancing"):

  Briefly activating an otherwise idle sibling can reduce the performance
  available to the other sibling and this effect does not necessarily end
  once the activated sibling becomes idle: after the ILB finishes and its
  CPU enters WFI, full single-thread performance is restored only after the
  sibling has remained idle for a qualification interval (10 Ki cycles on
  the tested Vera system).

That change prevents the NOHZ idle load balancer from unnecessarily waking
a sibling of a busy PE. However, ordinary task placement can still select
either sibling of an idle core and repeated changes of the active PE can
keep Olympus cores in two-thread mode despite little or no useful overlap
between the siblings.

This series makes PE0 the preferred sibling of an Olympus core using
SD_ASYM_PACKING and teaches the fair scheduler's idle-selection paths to
honor asymmetric SMT priority. The scheduler first selects a candidate CPU
and core according to its existing placement and capacity rules, then
chooses the highest-priority available sibling within that core. The
generic scheduler behavior is enabled only when an architecture supplies
an SD_ASYM_PACKING SMT domain.

PE0 and PE1 have equal steady-state capacity, the preference does not
identify a faster PE. PE0 is used only as a canonical choice when both
siblings are available. Consistently selecting the same sibling avoids
alternating the active PE across wakeups, lets PE1 remain idle for longer,
and allows more cores to remain in, or return to, full-resource
single-thread mode.

The series was tested on a two-node Vera system using an 88-thread
single-precision GEMM on the 88 physical cores of NUMA node 0.

With the workload allowed to choose either sibling of every core, observed
throughput improved from approximately 9.4 TFLOP/s on the baseline kernel
to approximately 10.1 TFLOP/s with this series applied. Repeated runs also
became more predictable because the workload consistently settled on PE0
while PE1 remained quiet.

Changes in v5:
 - Remove the redundant olympus_prefer_pe0 state (K Prateek Nayak)
 - Link to v4: https://lore.kernel.org/r/20260908082345.103087-1-arighi@nvidia.com

Changes in v4:
 - Honor the SMT sibling priority in the slow path (Srikar Dronamraju)
 - Rename the consolidated helper to select_idle_smt_cpu() (Srikar Dronamraju)
 - Link to v3: https://lore.kernel.org/r/20260907163513.4172411-1-arighi@nvidia.com

Changes in v3:
 - Consolidate the SMT-priority adjustment in select_idle_sibling() after an
   idle candidate has been selected (K Prateek Nayak)
 - Fold the asym SMT checks into select_idle_smt_priority() and scan the
   scheduling-domain span directly (K Prateek Nayak)
 - Link to v2: https://lore.kernel.org/r/20260904091838.3617894-1-arighi@nvidia.com

Changes in v2:
 - Clarify that the generic scheduler change also covers POWER7 (Dietmar
   Eggemann)
 - Simplify sched_smt_asym_prefer() by inspecting the lowest scheduling
   domain directly (Dietmar Eggemann)
 - Link to v1: https://lore.kernel.org/r/20260831181800.1668646-1-arighi@nvidia.com

Andrea Righi (2):
      arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores
      sched/fair: Honor asymmetric SMT priority in idle selection

 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 51 +++++++++++++++++++++++
 kernel/sched/fair.c               | 85 +++++++++++++++++++++++++++++++--------
 kernel/sched/sched.h              |  6 +++
 kernel/sched/topology.c           | 36 +++++++++++++++++
 6 files changed, 163 insertions(+), 17 deletions(-)

[PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
capacity but two different resource modes. One-Thread Active mode gives
one PE the full core, while waking the other PE restores Two-Thread
Active mode and partitions decode, issue, cache, TLB, and vector
resources. Returning to full-resource mode requires the sibling to
remain in WFI for 10 Ki cycles.

Measurements show that pinned workloads perform equally on either PE,
but freely migratable workloads lose substantial throughput when they
alternate between PE identities. Consistently selecting PE0 keeps PE1
idle, avoids repeated SMT repartitioning, and restores
one-thread-per-core performance.

Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This
is independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal
capacity, while physical cores with different maximum frequencies are
handled by a higher scheduling domain.

Firmware currently provides no interface for describing the preferred
SMT sibling. Detect Olympus by MIDR until such an interface is
available.

Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 51 +++++++++++++++++++++++++++++++
 3 files changed, 53 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..e5a7a4b2e3844 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,55 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Firmware does not currently describe this preference, so detect
+ * Olympus by MIDR until a firmware interface is available.
+ */
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	return cpu_smt_flags() | SD_ASYM_PACKING;
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, NULL, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
-- 
2.55.0

[PATCH 2/2] sched/fair: Honor asymmetric SMT priority in idle selection

[Andrea Righi]

POWER7 and NVIDIA Olympus use SD_ASYM_PACKING at the shared-capacity SMT
level to order hardware threads. Idle CPU selection does not consult
that order, so a task can wake on an arbitrary sibling and remain there
until load balancing corrects the placement. On these systems, that
initial choice can prevent the core from entering its preferred
lower-thread resource mode and cause a large and persistent performance
loss.

When idle selection finds an available CPU in an SMT core, choose the
highest-priority available sibling. On SMT2 Olympus this only changes
selection on fully idle cores. A partially idle core has only one
available CPU. On wider SMT systems such as POWER7, it also fills
available siblings in priority order while the core is partially busy.

Apply the preference to idle-core and idle-CPU scans,
asymmetric-capacity scans, target, previous, recently-used CPU fast
paths and the slow path. Inspect the lowest scheduling domain directly,
but require both CPUs to share its span because isolcpus can split
hardware siblings across scheduling domains.

Keep physical-core capacity selection independent from SMT sibling
ordering. SD_ASYM_CPUCAPACITY first selects among cores with different
maximum capacities, then SD_ASYM_PACKING selects the preferred available
sibling inside the chosen core, whose siblings continue to share equal
capacity.

Reviewed-by: Srikar Dronamraju <srikar@linux.ibm.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/fair.c     | 85 ++++++++++++++++++++++++++++++++---------
 kernel/sched/sched.h    |  6 +++
 kernel/sched/topology.c | 36 +++++++++++++++++
 3 files changed, 110 insertions(+), 17 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index b8bd308c2d5b1..37837c36288a0 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8587,6 +8587,35 @@ static inline bool test_idle_cores(int cpu)
 	return false;
 }
 
+/*
+ * Redirect a CPU to a higher-priority available sibling in its SMT domain,
+ * subject to task affinity.
+ */
+static inline int select_idle_smt_cpu(struct task_struct *p, int cpu)
+{
+	struct sched_domain *sd;
+	int best = cpu;
+	int sibling;
+
+	if (!sched_smt_asym_active())
+		return cpu;
+
+	sd = rcu_dereference_all(cpu_rq(cpu)->sd);
+	if (!sd || !(sd->flags & SD_SHARE_CPUCAPACITY) ||
+	    !(sd->flags & SD_ASYM_PACKING))
+		return cpu;
+
+	for_each_cpu_and(sibling, sched_domain_span(sd), p->cpus_ptr) {
+		if (sibling == best || !choose_idle_cpu(sibling, p))
+			continue;
+
+		if (sched_asym_prefer(sibling, best))
+			best = sibling;
+	}
+
+	return best;
+}
+
 /*
  * Scans the local SMT mask to see if the entire core is idle, and records this
  * information in sd_balance_shared->has_idle_cores.
@@ -8971,7 +9000,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 
 	if (choose_idle_cpu(target, p) &&
 	    asym_fits_cpu(task_util, util_min, util_max, target))
-		return target;
+		goto select_smt_priority;
 
 	/*
 	 * If the previous CPU is cache affine and idle, don't be stupid:
@@ -8981,8 +9010,10 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	    asym_fits_cpu(task_util, util_min, util_max, prev)) {
 
 		if (!static_branch_unlikely(&sched_cluster_active) ||
-		    cpus_share_resources(prev, target))
-			return prev;
+		    cpus_share_resources(prev, target)) {
+			target = prev;
+			goto select_smt_priority;
+		}
 
 		prev_aff = prev;
 	}
@@ -9000,7 +9031,8 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	    prev == smp_processor_id() &&
 	    this_rq()->nr_running <= 1 &&
 	    asym_fits_cpu(task_util, util_min, util_max, prev)) {
-		return prev;
+		target = prev;
+		goto select_smt_priority;
 	}
 
 	/* Check a recently used CPU as a potential idle candidate: */
@@ -9014,8 +9046,10 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	    asym_fits_cpu(task_util, util_min, util_max, recent_used_cpu)) {
 
 		if (!static_branch_unlikely(&sched_cluster_active) ||
-		    cpus_share_resources(recent_used_cpu, target))
-			return recent_used_cpu;
+		    cpus_share_resources(recent_used_cpu, target)) {
+			target = recent_used_cpu;
+			goto select_smt_priority;
+		}
 
 	} else {
 		recent_used_cpu = -1;
@@ -9037,7 +9071,11 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 		 */
 		if (sd) {
 			i = select_idle_capacity(p, sd, target);
-			return ((unsigned)i < nr_cpumask_bits) ? i : target;
+			if ((unsigned int)i < nr_cpumask_bits) {
+				target = i;
+				goto select_smt_priority;
+			}
+			return target;
 		}
 	}
 
@@ -9050,14 +9088,18 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 
 		if (!has_idle_core && cpus_share_cache(prev, target)) {
 			i = select_idle_smt(p, sd, prev);
-			if ((unsigned int)i < nr_cpumask_bits)
-				return i;
+			if ((unsigned int)i < nr_cpumask_bits) {
+				target = i;
+				goto select_smt_priority;
+			}
 		}
 	}
 
 	i = select_idle_cpu(p, sd, has_idle_core, target);
-	if ((unsigned)i < nr_cpumask_bits)
-		return i;
+	if ((unsigned int)i < nr_cpumask_bits) {
+		target = i;
+		goto select_smt_priority;
+	}
 
 	/*
 	 * For cluster machines which have lower sharing cache like L2 or
@@ -9065,12 +9107,19 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	 * first. But prev_cpu or recent_used_cpu may also be a good candidate,
 	 * use them if possible when no idle CPU found in select_idle_cpu().
 	 */
-	if ((unsigned int)prev_aff < nr_cpumask_bits)
-		return prev_aff;
-	if ((unsigned int)recent_used_cpu < nr_cpumask_bits)
-		return recent_used_cpu;
+	if ((unsigned int)prev_aff < nr_cpumask_bits) {
+		target = prev_aff;
+		goto select_smt_priority;
+	}
+	if ((unsigned int)recent_used_cpu < nr_cpumask_bits) {
+		target = recent_used_cpu;
+		goto select_smt_priority;
+	}
 
 	return target;
+
+select_smt_priority:
+	return select_idle_smt_cpu(p, target);
 }
 
 /**
@@ -9747,8 +9796,10 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
 	}
 
 	/* Slow path */
-	if (unlikely(sd))
-		return sched_balance_find_dst_cpu(sd, p, cpu, prev_cpu, sd_flag);
+	if (unlikely(sd)) {
+		new_cpu = sched_balance_find_dst_cpu(sd, p, cpu, prev_cpu, sd_flag);
+		return select_idle_smt_cpu(p, new_cpu);
+	}
 
 	/* Fast path */
 	if (wake_flags & WF_TTWU)
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6c3ad70e58b8e..568cb1ed2dd6b 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2240,6 +2240,7 @@ DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
 DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
 
 extern struct static_key_false sched_asym_cpucapacity;
+extern struct static_key_false sched_smt_asym_packing;
 extern struct static_key_false sched_cluster_active;
 
 static __always_inline bool sched_asym_cpucap_active(void)
@@ -2247,6 +2248,11 @@ static __always_inline bool sched_asym_cpucap_active(void)
 	return static_branch_unlikely(&sched_asym_cpucapacity);
 }
 
+static __always_inline bool sched_smt_asym_active(void)
+{
+	return static_branch_unlikely(&sched_smt_asym_packing);
+}
+
 struct sched_group_capacity {
 	atomic_t		ref;
 	/*
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 0248227d983a7..06c40eb5932af 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -683,8 +683,24 @@ DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
 DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
 
 DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
+DEFINE_STATIC_KEY_FALSE(sched_smt_asym_packing);
 DEFINE_STATIC_KEY_FALSE(sched_cluster_active);
 
+static bool has_asym_smt_domain(int cpu)
+{
+	struct sched_domain *sd;
+
+	for_each_domain(cpu, sd) {
+		if (!(sd->flags & SD_SHARE_CPUCAPACITY))
+			break;
+
+		if (sd->flags & SD_ASYM_PACKING)
+			return true;
+	}
+
+	return false;
+}
+
 static void update_top_cache_domain(int cpu)
 {
 	struct sched_domain_shared *sds = NULL;
@@ -3084,6 +3100,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 	struct rq *rq = NULL;
 	int i, ret = -ENOMEM;
 	bool has_asym = false;
+	bool has_asym_smt = false;
 	bool has_cluster = false;
 
 	if (WARN_ON(cpumask_empty(cpu_map)))
@@ -3202,6 +3219,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 
 		cpu_attach_domain(sd, d.rd, i);
 
+		if (has_asym_smt_domain(i))
+			has_asym_smt = true;
+
 		if (lowest_flag_domain(i, SD_CLUSTER))
 			has_cluster = true;
 	}
@@ -3210,6 +3230,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 	if (has_asym)
 		static_branch_inc_cpuslocked(&sched_asym_cpucapacity);
 
+	if (has_asym_smt)
+		static_branch_inc_cpuslocked(&sched_smt_asym_packing);
+
 	if (has_cluster)
 		static_branch_inc_cpuslocked(&sched_cluster_active);
 
@@ -3310,11 +3333,24 @@ int __init sched_init_domains(const struct cpumask *cpu_map)
 static void detach_destroy_domains(const struct cpumask *cpu_map)
 {
 	unsigned int cpu = cpumask_any(cpu_map);
+	bool has_asym_smt = false;
 	int i;
 
+	rcu_read_lock();
+	for_each_cpu(i, cpu_map) {
+		if (has_asym_smt_domain(i)) {
+			has_asym_smt = true;
+			break;
+		}
+	}
+	rcu_read_unlock();
+
 	if (rcu_access_pointer(per_cpu(sd_asym_cpucapacity, cpu)))
 		static_branch_dec_cpuslocked(&sched_asym_cpucapacity);
 
+	if (has_asym_smt)
+		static_branch_dec_cpuslocked(&sched_smt_asym_packing);
+
 	if (static_branch_unlikely(&sched_cluster_active))
 		static_branch_dec_cpuslocked(&sched_cluster_active);
 
-- 
2.55.0

Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[K Prateek Nayak]

Hello Andrea,

On 9/9/2026 11:56 AM, Andrea Righi wrote:
> NVIDIA Olympus implements SMT with two symmetric processing elements (PEs).
> When only one PE is active, the core operates in single-thread mode and
> that PE can use the full core resources. When both PEs are active, the core
> operates in two-thread mode and the PEs share those resources. This
> behavior is common to SMT implementations, but Olympus is particularly
> sensitive to brief sibling activations because returning from two-thread
> mode to single-thread mode after a sibling becomes idle is not immediate.
> As described by commit 293f9611ae735 ("sched/fair: Prefer fully idle cores
> for NOHZ balancing"):
> 
>   Briefly activating an otherwise idle sibling can reduce the performance
>   available to the other sibling and this effect does not necessarily end
>   once the activated sibling becomes idle: after the ILB finishes and its
>   CPU enters WFI, full single-thread performance is restored only after the
>   sibling has remained idle for a qualification interval (10 Ki cycles on
>   the tested Vera system).
> 
> That change prevents the NOHZ idle load balancer from unnecessarily waking
> a sibling of a busy PE. However, ordinary task placement can still select
> either sibling of an idle core and repeated changes of the active PE can
> keep Olympus cores in two-thread mode despite little or no useful overlap
> between the siblings.
> 
> This series makes PE0 the preferred sibling of an Olympus core using
> SD_ASYM_PACKING and teaches the fair scheduler's idle-selection paths to
> honor asymmetric SMT priority. The scheduler first selects a candidate CPU
> and core according to its existing placement and capacity rules, then
> chooses the highest-priority available sibling within that core. The
> generic scheduler behavior is enabled only when an architecture supplies
> an SD_ASYM_PACKING SMT domain.
> 
> PE0 and PE1 have equal steady-state capacity, the preference does not
> identify a faster PE. PE0 is used only as a canonical choice when both
> siblings are available. Consistently selecting the same sibling avoids
> alternating the active PE across wakeups, lets PE1 remain idle for longer,
> and allows more cores to remain in, or return to, full-resource
> single-thread mode.
> 
> The series was tested on a two-node Vera system using an 88-thread
> single-precision GEMM on the 88 physical cores of NUMA node 0.
> 
> With the workload allowed to choose either sibling of every core, observed
> throughput improved from approximately 9.4 TFLOP/s on the baseline kernel
> to approximately 10.1 TFLOP/s with this series applied. Repeated runs also
> became more predictable because the workload consistently settled on PE0
> while PE1 remained quiet.
> 
> Changes in v5:
>  - Remove the redundant olympus_prefer_pe0 state (K Prateek Nayak)
>  - Link to v4: https://lore.kernel.org/r/20260908082345.103087-1-arighi@nvidia.com

My replies failed to keep up with your speed on iterations :-)

Since this is same as v4 and I had taken it for a spin on a
x86 and an ARM64 server, feel free to include:

Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

-- 
Thanks and Regards,
Prateek

Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Andrea Righi]

On Wed, Sep 09, 2026 at 12:06:06PM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 9/9/2026 11:56 AM, Andrea Righi wrote:
> > NVIDIA Olympus implements SMT with two symmetric processing elements (PEs).
> > When only one PE is active, the core operates in single-thread mode and
> > that PE can use the full core resources. When both PEs are active, the core
> > operates in two-thread mode and the PEs share those resources. This
> > behavior is common to SMT implementations, but Olympus is particularly
> > sensitive to brief sibling activations because returning from two-thread
> > mode to single-thread mode after a sibling becomes idle is not immediate.
> > As described by commit 293f9611ae735 ("sched/fair: Prefer fully idle cores
> > for NOHZ balancing"):
> > 
> >   Briefly activating an otherwise idle sibling can reduce the performance
> >   available to the other sibling and this effect does not necessarily end
> >   once the activated sibling becomes idle: after the ILB finishes and its
> >   CPU enters WFI, full single-thread performance is restored only after the
> >   sibling has remained idle for a qualification interval (10 Ki cycles on
> >   the tested Vera system).
> > 
> > That change prevents the NOHZ idle load balancer from unnecessarily waking
> > a sibling of a busy PE. However, ordinary task placement can still select
> > either sibling of an idle core and repeated changes of the active PE can
> > keep Olympus cores in two-thread mode despite little or no useful overlap
> > between the siblings.
> > 
> > This series makes PE0 the preferred sibling of an Olympus core using
> > SD_ASYM_PACKING and teaches the fair scheduler's idle-selection paths to
> > honor asymmetric SMT priority. The scheduler first selects a candidate CPU
> > and core according to its existing placement and capacity rules, then
> > chooses the highest-priority available sibling within that core. The
> > generic scheduler behavior is enabled only when an architecture supplies
> > an SD_ASYM_PACKING SMT domain.
> > 
> > PE0 and PE1 have equal steady-state capacity, the preference does not
> > identify a faster PE. PE0 is used only as a canonical choice when both
> > siblings are available. Consistently selecting the same sibling avoids
> > alternating the active PE across wakeups, lets PE1 remain idle for longer,
> > and allows more cores to remain in, or return to, full-resource
> > single-thread mode.
> > 
> > The series was tested on a two-node Vera system using an 88-thread
> > single-precision GEMM on the 88 physical cores of NUMA node 0.
> > 
> > With the workload allowed to choose either sibling of every core, observed
> > throughput improved from approximately 9.4 TFLOP/s on the baseline kernel
> > to approximately 10.1 TFLOP/s with this series applied. Repeated runs also
> > became more predictable because the workload consistently settled on PE0
> > while PE1 remained quiet.
> > 
> > Changes in v5:
> >  - Remove the redundant olympus_prefer_pe0 state (K Prateek Nayak)
> >  - Link to v4: https://lore.kernel.org/r/20260908082345.103087-1-arighi@nvidia.com
> 
> My replies failed to keep up with your speed on iterations :-)

Haha, sorry about that! :D

> 
> Since this is same as v4 and I had taken it for a spin on a
> x86 and an ARM64 server, feel free to include:
> 
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

And thanks!

-Andrea

Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Peter Zijlstra]

On Wed, Sep 09, 2026 at 08:26:07AM +0200, Andrea Righi wrote:

> Andrea Righi (2):
>       arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores
>       sched/fair: Honor asymmetric SMT priority in idle selection
> 
>  arch/arm64/include/asm/topology.h |  1 +
>  arch/arm64/kernel/smp.c           |  1 +
>  arch/arm64/kernel/topology.c      | 51 +++++++++++++++++++++++
>  kernel/sched/fair.c               | 85 +++++++++++++++++++++++++++++++--------
>  kernel/sched/sched.h              |  6 +++
>  kernel/sched/topology.c           | 36 +++++++++++++++++
>  6 files changed, 163 insertions(+), 17 deletions(-)

I tentatively picked these up, but I need an arm64 ack before I can
merge them, and ideally vingu will bless them too :-)

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Dietmar Eggemann]

On 09.09.26 08:26, Andrea Righi wrote:

[...]

> +void __init arm64_init_sched_topology(void)
> +{
> +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> +		return;
> +
> +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> +		return;
> +
> +	if (!topology_core_has_smt(smp_processor_id()))
> +		return;
> +
> +	set_sched_topology(arm64_asym_smt_topology);
> +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");

I'm not really a big fan of using this arm64 setup for NVIDIA Olympus
alone here.

> +}
> +
> +int arch_asym_cpu_priority(int cpu)
> +{
> +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> +}
> +

This is done since it will only be called for NVIDIA Olympus since it's
the only CPU model enabling this. I think it will work but it's still
ugly :-)

[...]

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Will Deacon]

On Wed, Sep 09, 2026 at 08:26:08AM +0200, Andrea Righi wrote:
> diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> index d28438f8b83f1..e5a7a4b2e3844 100644
> --- a/arch/arm64/kernel/topology.c
> +++ b/arch/arm64/kernel/topology.c
> @@ -19,6 +19,8 @@
>  #include <linux/init.h>
>  #include <linux/percpu.h>
>  #include <linux/sched/isolation.h>
> +#include <linux/sched/topology.h>
> +#include <linux/smp.h>
>  #include <linux/xarray.h>
>  
>  #include <asm/cpu.h>
> @@ -44,6 +46,55 @@
>  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
>  static cpumask_var_t amu_fie_cpus;
>  
> +/*
> + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> + * two-thread active mode, with resources partitioned between the PEs.
> + *
> + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> + * mode. Firmware does not currently describe this preference, so detect
> + * Olympus by MIDR until a firmware interface is available.
> + */

Fix the firmware?

If we add this MIDR kludge, I guarantee that it will never be fixed.

Will

Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Will Deacon]

On Wed, Sep 09, 2026 at 03:14:15PM +0200, Peter Zijlstra wrote:
> On Wed, Sep 09, 2026 at 08:26:07AM +0200, Andrea Righi wrote:
> 
> > Andrea Righi (2):
> >       arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores
> >       sched/fair: Honor asymmetric SMT priority in idle selection
> > 
> >  arch/arm64/include/asm/topology.h |  1 +
> >  arch/arm64/kernel/smp.c           |  1 +
> >  arch/arm64/kernel/topology.c      | 51 +++++++++++++++++++++++
> >  kernel/sched/fair.c               | 85 +++++++++++++++++++++++++++++++--------
> >  kernel/sched/sched.h              |  6 +++
> >  kernel/sched/topology.c           | 36 +++++++++++++++++
> >  6 files changed, 163 insertions(+), 17 deletions(-)
> 
> I tentatively picked these up, but I need an arm64 ack before I can
> merge them, and ideally vingu will bless them too :-)

We've had five versions of this in five minutes, but the arch code is
pretty filthy tbh. Detecting topology based on MIDR is a non-starter,
sorry.

Will

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Will,

On Wed, Sep 09, 2026 at 04:34:18PM +0100, Will Deacon wrote:
> On Wed, Sep 09, 2026 at 08:26:08AM +0200, Andrea Righi wrote:
> > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> > index d28438f8b83f1..e5a7a4b2e3844 100644
> > --- a/arch/arm64/kernel/topology.c
> > +++ b/arch/arm64/kernel/topology.c
> > @@ -19,6 +19,8 @@
> >  #include <linux/init.h>
> >  #include <linux/percpu.h>
> >  #include <linux/sched/isolation.h>
> > +#include <linux/sched/topology.h>
> > +#include <linux/smp.h>
> >  #include <linux/xarray.h>
> >  
> >  #include <asm/cpu.h>
> > @@ -44,6 +46,55 @@
> >  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
> >  static cpumask_var_t amu_fie_cpus;
> >  
> > +/*
> > + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> > + * two-thread active mode, with resources partitioned between the PEs.
> > + *
> > + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> > + * mode. Firmware does not currently describe this preference, so detect
> > + * Olympus by MIDR until a firmware interface is available.
> > + */
> 
> Fix the firmware?
> 
> If we add this MIDR kludge, I guarantee that it will never be fixed.

Understood. The existing PPTT already describes the SMT relationship, but as far
as I can tell ACPI doesn't currently provide a way to describe a preferred SMT
sibling. Both PEs also have equal isolated performance, so representing this
using different _CPC highest_perf values wouldn't accurately describe their
capacity.

I'll drop the MIDR-based enablement and coordinate with the firmware team on a
proper ABI.

In the meantime, for systems with the existing firmware, would an explicit
kernel boot option be acceptable as a fallback? It'd enable SD_ASYM_PACKING at
the SMT level and prefer the primary PE, but only when requested by the user;
the kernel would not infer this policy from the CPU model.

Other ideas?

Thanks,
-Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

On Wed, Sep 09, 2026 at 05:19:08PM +0200, Dietmar Eggemann wrote:
> On 09.09.26 08:26, Andrea Righi wrote:
> 
> [...]
> 
> > +void __init arm64_init_sched_topology(void)
> > +{
> > +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> > +		return;
> > +
> > +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> > +		return;
> > +
> > +	if (!topology_core_has_smt(smp_processor_id()))
> > +		return;
> > +
> > +	set_sched_topology(arm64_asym_smt_topology);
> > +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
> 
> I'm not really a big fan of using this arm64 setup for NVIDIA Olympus
> alone here.
> 
> > +}
> > +
> > +int arch_asym_cpu_priority(int cpu)
> > +{
> > +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> > +}
> > +
> 
> This is done since it will only be called for NVIDIA Olympus since it's
> the only CPU model enabling this. I think it will work but it's still
> ugly :-)
> 
> [...]
> 

Agreed on both points. As mentioned in the other email, I'll coordinate with the
firmware team on a proper ABI for describing the PE0 preference. For the systems
with the current firmware, I don't have many ideas on how to enable this... a
special scheduler facility enabled via kernel boot option? Other ideas?

Thanks,
-Andrea

Re: [PATCH v5 0/2] sched: Enable preferred SMT siblings on NVIDIA Olympus

[Peter Zijlstra]

On Wed, Sep 09, 2026 at 04:36:01PM +0100, Will Deacon wrote:
> On Wed, Sep 09, 2026 at 03:14:15PM +0200, Peter Zijlstra wrote:
> > On Wed, Sep 09, 2026 at 08:26:07AM +0200, Andrea Righi wrote:
> > 
> > > Andrea Righi (2):
> > >       arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores
> > >       sched/fair: Honor asymmetric SMT priority in idle selection
> > > 
> > >  arch/arm64/include/asm/topology.h |  1 +
> > >  arch/arm64/kernel/smp.c           |  1 +
> > >  arch/arm64/kernel/topology.c      | 51 +++++++++++++++++++++++
> > >  kernel/sched/fair.c               | 85 +++++++++++++++++++++++++++++++--------
> > >  kernel/sched/sched.h              |  6 +++
> > >  kernel/sched/topology.c           | 36 +++++++++++++++++
> > >  6 files changed, 163 insertions(+), 17 deletions(-)
> > 
> > I tentatively picked these up, but I need an arm64 ack before I can
> > merge them, and ideally vingu will bless them too :-)
> 
> We've had five versions of this in five minutes, but the arch code is
> pretty filthy tbh. Detecting topology based on MIDR is a non-starter,
> sorry.

Yeah, Andrea is a wee bit fast with re-posting. I'll drop this, no
worries.

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Will Deacon]

On Wed, Sep 09, 2026 at 06:07:04PM +0200, Andrea Righi wrote:
> On Wed, Sep 09, 2026 at 04:34:18PM +0100, Will Deacon wrote:
> > On Wed, Sep 09, 2026 at 08:26:08AM +0200, Andrea Righi wrote:
> > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> > > index d28438f8b83f1..e5a7a4b2e3844 100644
> > > --- a/arch/arm64/kernel/topology.c
> > > +++ b/arch/arm64/kernel/topology.c
> > > @@ -19,6 +19,8 @@
> > >  #include <linux/init.h>
> > >  #include <linux/percpu.h>
> > >  #include <linux/sched/isolation.h>
> > > +#include <linux/sched/topology.h>
> > > +#include <linux/smp.h>
> > >  #include <linux/xarray.h>
> > >  
> > >  #include <asm/cpu.h>
> > > @@ -44,6 +46,55 @@
> > >  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
> > >  static cpumask_var_t amu_fie_cpus;
> > >  
> > > +/*
> > > + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> > > + * two-thread active mode, with resources partitioned between the PEs.
> > > + *
> > > + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> > > + * mode. Firmware does not currently describe this preference, so detect
> > > + * Olympus by MIDR until a firmware interface is available.
> > > + */
> > 
> > Fix the firmware?
> > 
> > If we add this MIDR kludge, I guarantee that it will never be fixed.
> 
> Understood. The existing PPTT already describes the SMT relationship, but as far
> as I can tell ACPI doesn't currently provide a way to describe a preferred SMT
> sibling. Both PEs also have equal isolated performance, so representing this
> using different _CPC highest_perf values wouldn't accurately describe their
> capacity.
> 
> I'll drop the MIDR-based enablement and coordinate with the firmware team on a
> proper ABI.

Thanks, Andrea, I must confess that I wasn't expecting that!

> In the meantime, for systems with the existing firmware, would an explicit
> kernel boot option be acceptable as a fallback? It'd enable SD_ASYM_PACKING at
> the SMT level and prefer the primary PE, but only when requested by the user;
> the kernel would not infer this policy from the CPU model.

I'm ok with that.

Will

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Dietmar Eggemann]

On 09.09.26 18:17, Andrea Righi wrote:
> On Wed, Sep 09, 2026 at 05:19:08PM +0200, Dietmar Eggemann wrote:
>> On 09.09.26 08:26, Andrea Righi wrote:
>>
>> [...]
>>
>>> +void __init arm64_init_sched_topology(void)
>>> +{
>>> +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
>>> +		return;
>>> +
>>> +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
>>> +		return;
>>> +
>>> +	if (!topology_core_has_smt(smp_processor_id()))
>>> +		return;
>>> +
>>> +	set_sched_topology(arm64_asym_smt_topology);
>>> +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
>>
>> I'm not really a big fan of using this arm64 setup for NVIDIA Olympus
>> alone here.
>>
>>> +}
>>> +
>>> +int arch_asym_cpu_priority(int cpu)
>>> +{
>>> +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
>>> +}
>>> +
>>
>> This is done since it will only be called for NVIDIA Olympus since it's
>> the only CPU model enabling this. I think it will work but it's still
>> ugly :-)
>>
>> [...]
>>
> 
> Agreed on both points. As mentioned in the other email, I'll coordinate with the
> firmware team on a proper ABI for describing the PE0 preference. For the systems
> with the current firmware, I don't have many ideas on how to enable this... a
> special scheduler facility enabled via kernel boot option? Other ideas?
With the explicit kernel boot time option discussed in the other thread,
how do you want to code arch_asym_cpu_priority()?

AFAIK, Arm architecture does not require the first/primary HW thread of
an SMT core to have MPIDR_EL1.Aff0 == 0.

The closest I came to you setup in Arm64 is a ThunderX2 SMT-4 machine on
which I can run 'OpenBLAS benchmark/sgemm.goto'. This machine has
symmetric CPU capacity though like Power7.
It has MPIDR_EL1.Aff0 == {0,1,2,3} for HW threads but '== 0' wouldn't
work here.

[...]

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

On Thu, Sep 10, 2026 at 09:53:38AM +0100, Will Deacon wrote:
> On Wed, Sep 09, 2026 at 06:07:04PM +0200, Andrea Righi wrote:
> > On Wed, Sep 09, 2026 at 04:34:18PM +0100, Will Deacon wrote:
> > > On Wed, Sep 09, 2026 at 08:26:08AM +0200, Andrea Righi wrote:
> > > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> > > > index d28438f8b83f1..e5a7a4b2e3844 100644
> > > > --- a/arch/arm64/kernel/topology.c
> > > > +++ b/arch/arm64/kernel/topology.c
> > > > @@ -19,6 +19,8 @@
> > > >  #include <linux/init.h>
> > > >  #include <linux/percpu.h>
> > > >  #include <linux/sched/isolation.h>
> > > > +#include <linux/sched/topology.h>
> > > > +#include <linux/smp.h>
> > > >  #include <linux/xarray.h>
> > > >  
> > > >  #include <asm/cpu.h>
> > > > @@ -44,6 +46,55 @@
> > > >  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
> > > >  static cpumask_var_t amu_fie_cpus;
> > > >  
> > > > +/*
> > > > + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> > > > + * two-thread active mode, with resources partitioned between the PEs.
> > > > + *
> > > > + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> > > > + * mode. Firmware does not currently describe this preference, so detect
> > > > + * Olympus by MIDR until a firmware interface is available.
> > > > + */
> > > 
> > > Fix the firmware?
> > > 
> > > If we add this MIDR kludge, I guarantee that it will never be fixed.
> > 
> > Understood. The existing PPTT already describes the SMT relationship, but as far
> > as I can tell ACPI doesn't currently provide a way to describe a preferred SMT
> > sibling. Both PEs also have equal isolated performance, so representing this
> > using different _CPC highest_perf values wouldn't accurately describe their
> > capacity.
> > 
> > I'll drop the MIDR-based enablement and coordinate with the firmware team on a
> > proper ABI.
> 
> Thanks, Andrea, I must confess that I wasn't expecting that!

I try to be a good citizen every now and then. :)

> 
> > In the meantime, for systems with the existing firmware, would an explicit
> > kernel boot option be acceptable as a fallback? It'd enable SD_ASYM_PACKING at
> > the SMT level and prefer the primary PE, but only when requested by the user;
> > the kernel would not infer this policy from the CPU model.
> 
> I'm ok with that.

Awesome! I'll work through Dietmar's point for this and post a new version
without the MIDR-based detection.

Thanks,
-Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Dietmar,

On Thu, Sep 10, 2026 at 12:11:06PM +0200, Dietmar Eggemann wrote:
> On 09.09.26 18:17, Andrea Righi wrote:
> > On Wed, Sep 09, 2026 at 05:19:08PM +0200, Dietmar Eggemann wrote:
> >> On 09.09.26 08:26, Andrea Righi wrote:
> >>
> >> [...]
> >>
> >>> +void __init arm64_init_sched_topology(void)
> >>> +{
> >>> +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> >>> +		return;
> >>> +
> >>> +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> >>> +		return;
> >>> +
> >>> +	if (!topology_core_has_smt(smp_processor_id()))
> >>> +		return;
> >>> +
> >>> +	set_sched_topology(arm64_asym_smt_topology);
> >>> +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
> >>
> >> I'm not really a big fan of using this arm64 setup for NVIDIA Olympus
> >> alone here.
> >>
> >>> +}
> >>> +
> >>> +int arch_asym_cpu_priority(int cpu)
> >>> +{
> >>> +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> >>> +}
> >>> +
> >>
> >> This is done since it will only be called for NVIDIA Olympus since it's
> >> the only CPU model enabling this. I think it will work but it's still
> >> ugly :-)
> >>
> >> [...]
> >>
> > 
> > Agreed on both points. As mentioned in the other email, I'll coordinate with the
> > firmware team on a proper ABI for describing the PE0 preference. For the systems
> > with the current firmware, I don't have many ideas on how to enable this... a
> > special scheduler facility enabled via kernel boot option? Other ideas?
> With the explicit kernel boot time option discussed in the other thread,
> how do you want to code arch_asym_cpu_priority()?

Good point.

I think we can avoid an arm64 override entirely and rely on the default
implementation:

int __weak arch_asym_cpu_priority(int cpu)
{
	return -cpu;
}

The boot option would only add SD_ASYM_PACKING to the SMT domain. A rough
prototype could look like this in kernel/sched/topology.c:

static bool sched_smt_asym_packing __read_mostly;

static int __init setup_sched_smt_asym_packing(char *str)
{
	sched_smt_asym_packing = true;
	return 1;
}
__setup("sched_smt_asym_packing", setup_sched_smt_asym_packing);

int cpu_smt_flags(void)
{
	int flags = SD_SHARE_CPUCAPACITY | SD_SHARE_LLC;

	if (sched_smt_asym_packing)
		flags |= SD_ASYM_PACKING;

	return flags;
}

This would make the option generic rather than specific to arm64 or Olympus.

> 
> AFAIK, Arm architecture does not require the first/primary HW thread of
> an SMT core to have MPIDR_EL1.Aff0 == 0.

Agreed. Using the default -cpu ordering avoids making any assumptions about
MPIDR and consistently prefers the lowest-numbered logical CPU in each SMT
domain.

> 
> The closest I came to you setup in Arm64 is a ThunderX2 SMT-4 machine on
> which I can run 'OpenBLAS benchmark/sgemm.goto'. This machine has
> symmetric CPU capacity though like Power7.
> It has MPIDR_EL1.Aff0 == {0,1,2,3} for HW threads but '== 0' wouldn't
> work here.
> 
> [...]

Right, -cpu provides a total ordering for all threads, so this should provide
the expected sibling ordering on the ThunderX2 machine as well. If you're
willing to test the next version there, that would be very useful!

Thanks,
-Andrea

Re: [PATCH 2/2] sched/fair: Honor asymmetric SMT priority in idle selection

[Dietmar Eggemann]

On 09.09.26 08:26, Andrea Righi wrote:

[...]

> +static bool has_asym_smt_domain(int cpu)
> +{
> +	struct sched_domain *sd;
> +
> +	for_each_domain(cpu, sd) {
> +		if (!(sd->flags & SD_SHARE_CPUCAPACITY))
> +			break;
> +
> +		if (sd->flags & SD_ASYM_PACKING)
> +			return true;
> +	}
> +
> +	return false;
> +}

Replace for_each_domain() w/ sd = rcu_dereference_all(cpu_rq(cpu)->sd)
here as well?

[...]

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Dietmar Eggemann]

On 11.09.26 11:11, Andrea Righi wrote:
> Hi Dietmar,
> 
> On Thu, Sep 10, 2026 at 12:11:06PM +0200, Dietmar Eggemann wrote:
>> On 09.09.26 18:17, Andrea Righi wrote:
>>> On Wed, Sep 09, 2026 at 05:19:08PM +0200, Dietmar Eggemann wrote:
>>>> On 09.09.26 08:26, Andrea Righi wrote:

[...]
> I think we can avoid an arm64 override entirely and rely on the default
> implementation:
> 
> int __weak arch_asym_cpu_priority(int cpu)
> {
> 	return -cpu;
> }
> 
> The boot option would only add SD_ASYM_PACKING to the SMT domain. A rough
> prototype could look like this in kernel/sched/topology.c:
> 
> static bool sched_smt_asym_packing __read_mostly;
> 
> static int __init setup_sched_smt_asym_packing(char *str)
> {
> 	sched_smt_asym_packing = true;
> 	return 1;
> }
> __setup("sched_smt_asym_packing", setup_sched_smt_asym_packing);
> 
> int cpu_smt_flags(void)
> {
> 	int flags = SD_SHARE_CPUCAPACITY | SD_SHARE_LLC;
> 
> 	if (sched_smt_asym_packing)
> 		flags |= SD_ASYM_PACKING;
> 
> 	return flags;
> }
> 
> This would make the option generic rather than specific to arm64 or Olympus.
Ah, OK, no arm64 private 'struct sched_domain_topology_level
arm64_asym_smt_topology[]' anymore.

[...]

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Prateek,

On Wed, Sep 09, 2026 at 01:39:07AM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 9/8/2026 1:53 PM, Andrea Righi wrote:
> > NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
> > capacity but two different resource modes. One-Thread Active mode gives
> > one PE the full core, while waking the other PE restores Two-Thread
> > Active mode and partitions decode, issue, cache, TLB, and vector
> > resources. Returning to full-resource mode requires the sibling to
> > remain in WFI for 10 Ki cycles.
> > 
> > Measurements show that pinned workloads perform equally on either PE,
> > but freely migratable workloads lose substantial throughput when they
> > alternate between PE identities. Consistently selecting PE0 keeps PE1
> > idle, avoids repeated SMT repartitioning, and restores
> > one-thread-per-core performance.
> > 
> > Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
> > identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This
> > is independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal
> > capacity, while physical cores with different maximum frequencies are
> > handled by a higher scheduling domain.
> > 
> > Firmware currently provides no interface for describing the preferred
> > SMT sibling. Detect Olympus by MIDR until such an interface is
> > available.
> > 
> > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> 
> Feel free to include:
> 
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
> 
> Just one nit below ...
> > +/*
> > + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> > + * two-thread active mode, with resources partitioned between the PEs.
> > + *
> > + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> > + * mode. Firmware does not currently describe this preference, so detect
> > + * Olympus by MIDR until a firmware interface is available.
> > + */
> > +static bool olympus_prefer_pe0 __ro_after_init;
> 
> nit. What is the purpose of "olympus_prefer_pe0"?
> 
> > +
> > +#ifdef CONFIG_SCHED_SMT
> > +static int arm64_smt_flags(void)
> > +{
> > +	int flags = cpu_smt_flags();
> > +
> > +	if (olympus_prefer_pe0)
> > +		flags |= SD_ASYM_PACKING;
> 
> If this flag function is being used, olympus_prefer_pe0 is always true,
> and the scheduler will never call into arch_asym_cpu_priority() without
> SD_ASYM_PACKING being set so olympus_prefer_pe0 is always true there
> too.
> 
> Seems redundant unless you were planning to do set_sched_topology()
> unconditionally for whole of arm64.

Right, there's no plan to install this topology unconditionally.

Since arm64_asym_smt_topology is installed only after detecting an Olympus
system with SMT, the boolean doesn't provide any additional state and can be
removed. I'll do that in the next version.

Thanks!
-Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[K Prateek Nayak]

Hello Andrea,

On 9/8/2026 1:53 PM, Andrea Righi wrote:
> NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
> capacity but two different resource modes. One-Thread Active mode gives
> one PE the full core, while waking the other PE restores Two-Thread
> Active mode and partitions decode, issue, cache, TLB, and vector
> resources. Returning to full-resource mode requires the sibling to
> remain in WFI for 10 Ki cycles.
> 
> Measurements show that pinned workloads perform equally on either PE,
> but freely migratable workloads lose substantial throughput when they
> alternate between PE identities. Consistently selecting PE0 keeps PE1
> idle, avoids repeated SMT repartitioning, and restores
> one-thread-per-core performance.
> 
> Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
> identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This
> is independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal
> capacity, while physical cores with different maximum frequencies are
> handled by a higher scheduling domain.
> 
> Firmware currently provides no interface for describing the preferred
> SMT sibling. Detect Olympus by MIDR until such an interface is
> available.
> 
> Signed-off-by: Andrea Righi <arighi@nvidia.com>

Feel free to include:

Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

Just one nit below ...
> +/*
> + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> + * two-thread active mode, with resources partitioned between the PEs.
> + *
> + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> + * mode. Firmware does not currently describe this preference, so detect
> + * Olympus by MIDR until a firmware interface is available.
> + */
> +static bool olympus_prefer_pe0 __ro_after_init;

nit. What is the purpose of "olympus_prefer_pe0"?

> +
> +#ifdef CONFIG_SCHED_SMT
> +static int arm64_smt_flags(void)
> +{
> +	int flags = cpu_smt_flags();
> +
> +	if (olympus_prefer_pe0)
> +		flags |= SD_ASYM_PACKING;

If this flag function is being used, olympus_prefer_pe0 is always true,
and the scheduler will never call into arch_asym_cpu_priority() without
SD_ASYM_PACKING being set so olympus_prefer_pe0 is always true there
too.

Seems redundant unless you were planning to do set_sched_topology()
unconditionally for whole of arm64.

> +
> +	return flags;
> +}

-- 
Thanks and Regards,
Prateek

[PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
capacity but two different resource modes. One-Thread Active mode gives
one PE the full core, while waking the other PE restores Two-Thread
Active mode and partitions decode, issue, cache, TLB, and vector
resources. Returning to full-resource mode requires the sibling to
remain in WFI for 10 Ki cycles.

Measurements show that pinned workloads perform equally on either PE,
but freely migratable workloads lose substantial throughput when they
alternate between PE identities. Consistently selecting PE0 keeps PE1
idle, avoids repeated SMT repartitioning, and restores
one-thread-per-core performance.

Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This
is independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal
capacity, while physical cores with different maximum frequencies are
handled by a higher scheduling domain.

Firmware currently provides no interface for describing the preferred
SMT sibling. Detect Olympus by MIDR until such an interface is
available.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..0dd9eec1c4946 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,66 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Firmware does not currently describe this preference, so detect
+ * Olympus by MIDR until a firmware interface is available.
+ */
+static bool olympus_prefer_pe0 __ro_after_init;
+
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	int flags = cpu_smt_flags();
+
+	if (olympus_prefer_pe0)
+		flags |= SD_ASYM_PACKING;
+
+	return flags;
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, NULL, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	olympus_prefer_pe0 = true;
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	if (!olympus_prefer_pe0)
+		return 0;
+
+	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
-- 
2.55.0

[PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
capacity but two different resource modes. One-Thread Active mode gives
one PE the full core, while waking the other PE restores Two-Thread
Active mode and partitions decode, issue, cache, TLB, and vector
resources. Returning to full-resource mode requires the sibling to
remain in WFI for 10 Ki cycles.

Measurements show that pinned workloads perform equally on either PE,
but freely migratable workloads lose substantial throughput when they
alternate between PE identities. Consistently selecting PE0 keeps PE1
idle, avoids repeated SMT repartitioning, and restores
one-thread-per-core performance.

Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This
is independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal
capacity, while physical cores with different maximum frequencies are
handled by a higher scheduling domain.

Firmware currently provides no interface for describing the preferred
SMT sibling. Detect Olympus by MIDR until such an interface is
available.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..0dd9eec1c4946 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,66 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Firmware does not currently describe this preference, so detect
+ * Olympus by MIDR until a firmware interface is available.
+ */
+static bool olympus_prefer_pe0 __ro_after_init;
+
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	int flags = cpu_smt_flags();
+
+	if (olympus_prefer_pe0)
+		flags |= SD_ASYM_PACKING;
+
+	return flags;
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, NULL, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	olympus_prefer_pe0 = true;
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	if (!olympus_prefer_pe0)
+		return 0;
+
+	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
-- 
2.55.0

[PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
capacity but two different resource modes. One-Thread Active mode gives
one PE the full core, while waking the other PE restores Two-Thread
Active mode and partitions decode, issue, cache, TLB, and vector
resources. Returning to full-resource mode requires the sibling to
remain in WFI for 10 Ki cycles.

Measurements show that pinned workloads perform equally on either PE,
but freely migratable workloads lose substantial throughput when they
alternate between PE identities. Consistently selecting PE0 keeps PE1
idle, avoids repeated SMT repartitioning, and restores one-thread-per-core
performance.

Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This is
independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal capacity,
while physical cores with different maximum frequencies are handled by
a higher scheduling domain.

Firmware currently provides no interface for describing the preferred
SMT sibling. Detect Olympus by MIDR until such an interface is available.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..0dd9eec1c4946 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,66 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Firmware does not currently describe this preference, so detect
+ * Olympus by MIDR until a firmware interface is available.
+ */
+static bool olympus_prefer_pe0 __ro_after_init;
+
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	int flags = cpu_smt_flags();
+
+	if (olympus_prefer_pe0)
+		flags |= SD_ASYM_PACKING;
+
+	return flags;
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, NULL, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	olympus_prefer_pe0 = true;
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	if (!olympus_prefer_pe0)
+		return 0;
+
+	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
-- 
2.55.0

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Christian,

On Tue, Sep 01, 2026 at 09:32:56AM +0100, Christian Loehle wrote:
> On 8/31/26 22:43, Andrea Righi wrote:
...
> > I can experiment with this combined priority, but I think removing
> > SD_ASYM_CPUCAPACITY is a separate policy change rather than an alternative
> > implementation of this fix.
> 
> Cool thanks, and sorry for curveballing the approach like this, I wish I had
> the platform to test these ideas myself :/ 

No problem, thanks for looking into this! Access to Vera systems is problematic
also on my side, especially for all the time it takes to run all these tests. :)

> 
> > 
> > A static asym-packing priority does not preserve the capacity-aware semantics
> > used for task fitting, uclamp, misfit handling and migration. The current
> > approach keeps those semantics when selecting a physical core, then applies the
> > PE preference only within that core.
> 
> Right, but arguably most of these semantics become questionable as soon as the
> core enters two-thread mode, since the capacity available to each PE then
> depends on the state of its sibling.

Yes, I agree with that the current capacity model doesn't account the capacity
lost when an SMT sibling becomes active. And Olympus makes that limitation
particularly visible.

That said, I think the static CPU capacity is still meaningful when there's no
SMT contention. SD_ASYM_CPUCAPACITY can compare the task's demand against the
standalone capacity of the candidate physical cores and select an appropriate
one. And looking at the results, this appears to be beneficial. Then the
SMT-local asym-packing priority can select the preferred PE within that core,
which helps keep its sibling idle and preserve the uncontended state whenever
possible.

Once every usable physical core already has an active PE, any additional work
invitably introduces SMT contention. At that point, the effective capacity
becomes sibling-state-dependent and neither SD_ASYM_CPUCAPACITY nor a combined
static asym-packing priority can accurately model it.

> 
> Task fitting:
> We consider two tasks with util=400 to fit on two capacity=1000 PEs, even
> though once both PEs are active neither may have anything close to capacity
> 1000 available. In other words, the capacity used for fitting doesn't account
> for the capacity "stolen" by activating the sibling.

Correct, util_fits_cpu() doesn't reduce capacity merely because the sibling is
busy. The scheduler handles this through the SD_SHARE_CPUCAPACITY topology,
idle-core selection and SMT balancing, which tries to move work from a busy SMT
core to an idle core when possible. It doesn't provide a dynamic numerical
capacity for each PE.

But the same limitation remains with the combined asym-packing. Once both
siblings must be used, neither static priority describes how the core resources
are partitioned.

> 
> Uclamp:
> Isn't uclamp, and particularly its bucket implementation, fundamentally a poor
> fit for these platforms in the first place? Even if we tried to represent these
> small capacity differences through uclamp, we'd need something like
> UCLAMP_BUCKETS_COUNT=512 or 1024 to get useful resolution. We currently limit
> it to 20, and for good reason: the overhead.

Yeah, the uclamp buckets are used to aggregate runnable-task clamps on a
runqueue, they don't encode CPU capacity classes. So it's a different story.

> 
> Misfit handling:
> This seems problematic for essentially the same reason as task fitting. A task
> can be classified as fitting while the core is in one-thread mode, then lose a
> substantial fraction of its effective CPU capacity when the sibling becomes
> active, without the static CPU capacity reflecting that change. Conversely,
> migrating it to an otherwise equivalent core and allowing that core to return
> to one-thread mode changes the effective capacity (and therefore utilization)
> again.

Yes, misfit handling compares a task against the CPU's standalone capacity, it
doesn't dynamically reduce that capacity when an SMT sibling becomes busy. That
is true for regular SMT as well, and sibling contention is handled separately by
the SMT balancing logic. The combined asym-packing priority doesn't change this,
it orders CPUs but doesn't make capacity depend on sibling state.

> 
> I'm assuming the CPU_CYCLES counter advancement isn't affected by the
> one-thread/two-thread mode transition?

My understanding is that the core counter continues to advance according to the
PE clock while the PE is active. The mode transition doesn't change the clock
frequency, so the AMU ratio will not reflect the loss of issue/cache/vector
resources.

So, yes, you are right that the existing capacity model does not fully describe
SMT interference. This is probably a broader dynamic-SMT capacity issue and
neither of the policies discussed here models it explicitly.

That said, this series achieves the intended result for the GEMM benchmark (with
similar results observed also for other CPU-intensive workloads): with SMT off,
running one CPU-intensive task per physical core reaches the same ~10 TFLOP/s as
the patched kernel with SMT enabled and the same number of tasks. So the
scheduler now appears to select the preferred PE consistently and avoid the
unwanted resource-mode transitions.

> 
> > 
> > Also, encoding the combined priority alone would not fix the problem addressed
> > by patch 2: the idle-selection paths currently do not consult asymmetric SMT
> > priority. They can still return an arbitrary idle sibling regardless of how
> > arch_asym_cpu_priority() is defined. Patch 2 adds that missing behavior and
> > scopes it to the shared-capacity SMT domain.
> 
> Sure, patch 2 is a different story altogether.

Ok.

> 
> > 
> >> I had suggested this a while ago, did you have a stab at that by any chance,
> >> too?
> > 
> > I tested your CPPC-based asym-packing series, but not this particular
> > combined-priority variant. IIUC the earlier proposal was replacing
> > capacity-aware scheduling for minor physical-core capacity differences, SMT
> > sibling ordering looks like an orthogonal problem.
> > 
> > And at the time, the combined SMT-aware SD_ASYM_CPUCAPACITY approach also gave
> > the best Vera results of the alternatives I tested, which is another reason I
> > kept physical-core capacity selection separate here.
> > 
> >> Am I missing something altogether?
> > 
> > Combining the priorities is a valid experiment, but I'm not sure if it
> > completely solves the problem by itself, I'll give it a try and share the
> > results.
> 
> Thanks again, i'll have a look and give it some more thoughts myself.

Thanks!
-Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Christian Loehle]

On 8/31/26 22:43, Andrea Righi wrote:
> Hi Christian,
> 
> On Mon, Aug 31, 2026 at 10:13:42PM +0100, Christian Loehle wrote:
>> On 8/31/26 19:10, Andrea Righi wrote:
>>> NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
>>> capacity but two different resource modes. One-Thread Active mode gives
>>> one PE the full core, while waking the other PE restores Two-Thread
>>> Active mode and partitions decode, issue, cache, TLB, and vector
>>> resources. Returning to full-resource mode requires the sibling to
>>> remain in WFI for 10 Ki cycles.
>>>
>>> Measurements show that pinned workloads perform equally on either PE,
>>> but freely migratable workloads lose substantial throughput when they
>>> alternate between PE identities. Consistently selecting PE0 keeps PE1
>>> idle, avoids repeated SMT repartitioning, and restores one-thread-per-core
>>> performance.
>>>
>>> Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
>>> identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This is
>>> independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal capacity,
>>> while physical cores with different maximum frequencies are handled by
>>> a higher scheduling domain.
>>
>> But why? This asympacking + CAS interaction is a bit hard to comprehend IMV.
> 
> The two mehcanisms describe different preferences at different scheduling domain
> levels: SD_ASYM_CPUCAPACITY selects among physical cores with different max
> capacities, SD_ASYM_PACKING is set only on the SMT domain and selects the
> canonical PE within the core chose by the existing placement and capacity logic.
> 
>> Why can't we encode both preferences in the asym-packing priority, e.g.
>>
>> priority(cpu) = is_primary(cpu) ? 2 * highest_perf(cpu)
>>                                 : highest_perf(cpu)
>>
>> so that all primary PEs are preferred over all sibling PEs, while still
>> preserving the highest_perf ordering within each group, and do away with
>> SD_ASYM_CPUCAPACITY on Vera altogether?
> 
> I can experiment with this combined priority, but I think removing
> SD_ASYM_CPUCAPACITY is a separate policy change rather than an alternative
> implementation of this fix.

Cool thanks, and sorry for curveballing the approach like this, I wish I had
the platform to test these ideas myself :/ 

> 
> A static asym-packing priority does not preserve the capacity-aware semantics
> used for task fitting, uclamp, misfit handling and migration. The current
> approach keeps those semantics when selecting a physical core, then applies the
> PE preference only within that core.

Right, but arguably most of these semantics become questionable as soon as the
core enters two-thread mode, since the capacity available to each PE then
depends on the state of its sibling.

Task fitting:
We consider two tasks with util=400 to fit on two capacity=1000 PEs, even
though once both PEs are active neither may have anything close to capacity
1000 available. In other words, the capacity used for fitting doesn't account
for the capacity "stolen" by activating the sibling.

Uclamp:
Isn't uclamp, and particularly its bucket implementation, fundamentally a poor
fit for these platforms in the first place? Even if we tried to represent these
small capacity differences through uclamp, we'd need something like
UCLAMP_BUCKETS_COUNT=512 or 1024 to get useful resolution. We currently limit
it to 20, and for good reason: the overhead.

Misfit handling:
This seems problematic for essentially the same reason as task fitting. A task
can be classified as fitting while the core is in one-thread mode, then lose a
substantial fraction of its effective CPU capacity when the sibling becomes
active, without the static CPU capacity reflecting that change. Conversely,
migrating it to an otherwise equivalent core and allowing that core to return
to one-thread mode changes the effective capacity (and therefore utilization)
again.

I'm assuming the CPU_CYCLES counter advancement isn't affected by the
one-thread/two-thread mode transition?

> 
> Also, encoding the combined priority alone would not fix the problem addressed
> by patch 2: the idle-selection paths currently do not consult asymmetric SMT
> priority. They can still return an arbitrary idle sibling regardless of how
> arch_asym_cpu_priority() is defined. Patch 2 adds that missing behavior and
> scopes it to the shared-capacity SMT domain.

Sure, patch 2 is a different story altogether.

> 
>> I had suggested this a while ago, did you have a stab at that by any chance,
>> too?
> 
> I tested your CPPC-based asym-packing series, but not this particular
> combined-priority variant. IIUC the earlier proposal was replacing
> capacity-aware scheduling for minor physical-core capacity differences, SMT
> sibling ordering looks like an orthogonal problem.
> 
> And at the time, the combined SMT-aware SD_ASYM_CPUCAPACITY approach also gave
> the best Vera results of the alternatives I tested, which is another reason I
> kept physical-core capacity selection separate here.
> 
>> Am I missing something altogether?
> 
> Combining the priorities is a valid experiment, but I'm not sure if it
> completely solves the problem by itself, I'll give it a try and share the
> results.

Thanks again, i'll have a look and give it some more thoughts myself.

> 
> Thanks,
> -Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Christian,

On Mon, Aug 31, 2026 at 11:43:54PM +0200, Andrea Righi wrote:
...
> Combining the priorities is a valid experiment, but I'm not sure if it
> completely solves the problem by itself, I'll give it a try and share the
> results.

I did some tests comparing this asym-capacity+asym-packing approach vs the
combined-asym-packing approach (tested patch for the combined-asym-packing
approach is at the end - patch 2 is the same).

Policies tested
---------------

combined-asym-packing:

- combined capacity/PE priority
- SD_ASYM_PACKING at all topology levels
- SD_ASYM_CPUCAPACITY disabled

asym-capacity+smt-asym-packing:

- capacity-aware physical-core selection retained
- PE0 priority applied only in the SMT domain

[ Both policies include the idle-selection fix - patch 2 ]

SGEMM throughput
----------------

The primary result is ten repetitions of the exact 88-thread command provided.

Metric                    combined-asym-packing    asym-capacity+smt-asym-packing    Difference
Average throughput        9.864 +/- 0.180 TFLOP/s  10.094 +/- 0.065 TFLOP/s          +2.34%
Best throughput           10.194 +/- 0.193 TFLOP/s 10.374 +/- 0.065 TFLOP/s          +1.77%
Minimum average run       9.575 TFLOP/s            9.968 TFLOP/s                    +4.10%
Maximum average run       10.057 TFLOP/s           10.204 TFLOP/s                   +1.46%

asym-capacity+smt-asym-packing averages 10.095 TFLOP/s versus 9.920 TFLOP/s, a
1.76% advantage. More importantly, the run-to-run standard deviation drops from
180 to 65 GFLOP/s. combined-asym-packing can reach a good peak, but it does not
sustain it as reliably.

Thread scaling
--------------

Three runs per point, average throughput:

Threads  combined-asym-packing    asym-capacity+smt-asym-packing    Difference
22       3.065 +/- 0.002 TFLOP/s  3.068 +/- 0.002 TFLOP/s          +0.08%
44       5.849 +/- 0.064 TFLOP/s  5.888 +/- 0.022 TFLOP/s          +0.66%
88       9.975 +/- 0.128 TFLOP/s  10.062 +/- 0.107 TFLOP/s         +0.86%
176      10.695 +/- 0.004 TFLOP/s 10.704 +/- 0.012 TFLOP/s         +0.09%

The difference is specifically most visible around the intended
one-thread-per-core operating point. At 176 threads, where both SMT PEs are
used, the policies are effectively tied (as expected).

Cyclic wake-up latency
----------------------

Values are averages of three runs. Percentiles are the mean of each run's
reported percentile.

Condition  Metric  combined-asym-packing  asym-capacity+smt-asym-packing
Idle       median  6.433 us               6.398 us
Idle       p99     12.753 us              12.494 us
Idle       p99.9   18.324 us              19.025 us
Loaded     median  5.114 us               4.500 us
Loaded     p99     12.398 us              12.759 us
Loaded     p99.9   24.240 us              20.954 us

Idle latency is essentially tied. Under concurrent 88-thread SGEMM,
asym-capacity+smt-asym-packing improves median latency by 12% and p99.9 by
13.6%.

The worst observed loaded sample was 579.5 us with combined-asym-packing and
74.5 us with asym-capacity+smt-asym-packing. That is only one outlier and should
not be generalized without longer runs, but it favors
asym-capacity+smt-asym-packing.

Concurrent SGEMM throughput was tied: 9.902 versus 9.899 TFLOP/s.

Scheduler microbenchmarks
-------------------------

Lower is better for these results.

Test                       combined-asym-packing  asym-capacity+smt-asym-packing  Difference
sched pipe, processes      4.478 us/op            4.446 us/op                     -0.7%
sched pipe, threads        3.618 us/op            3.625 us/op                     +0.2%
SMT pair, CPU 0/176        1.815 us/op            1.813 us/op                     tied
Separate cores, CPU 0/1    4.388 us/op            4.311 us/op                     -1.8%
Unrestricted node 0        4.284 us/op            4.385 us/op                     +2.4%

These simple ping-pong results are effectively tied.

combined-asym-packing did materially better in the broader 160-task sched
messaging socket tests:

Test            combined-asym-packing  asym-capacity+smt-asym-packing
Process/socket  0.669 s                1.039 s
Thread/socket   0.653 s                0.989 s
Process/pipe    0.296 s                0.337 s

This suggests that combined asym-packing can help some highly communicating,
oversubscribed workloads by changing how runnable tasks are packed.

Futex
-----

Test             combined-asym-packing  asym-capacity+smt-asym-packing  Difference
Wake one         0.1695 ms              0.1800 ms                       +6.2%
Wake all         0.1734 ms              0.1693 ms                       -2.4%
Parallel wake    0.0430 ms              0.0332 ms                       -22.6%
Hash throughput  4.052 Mops/s           4.064 Mops/s                    +0.3%

Futex hashing is tied. Wake results are mixed, with
asym-capacity+smt-asym-packing notably better in the parallel-waker case.

Conclusion
----------

The combined priority is technically workable, but combined-asym-packing does
more than express the PE0 preference:

- it replaces SD_ASYM_CPUCAPACITY with asym-packing across the Olympus topology.
- It changes placement policy for unrelated multi-core and communication-heavy
  workloads
- It still requires the idle-selection scheduler change
- It produces lower and substantially more variable throughput

The asym-capacity+smt-asym-packing seems to solve the specific spatial-SMT issue
on Vera, retains the existing capacity-aware semantics, provides the best
target-workload result and has no general latency regression in this dataset.

combined-asym-packing patch
---------------------------

The following is the combined-asym-packing policy patch tested in this report.
The idle-selection patch was present in both test kernels and is therefore not
included in this policy delta.

---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 90 +++++++++++++++++++++++++++++++
 include/linux/sched/topology.h    |  2 +
 kernel/sched/topology.c           |  8 +++
 5 files changed, 102 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..42a5c0d4f5d2e 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,94 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Combine that preference with the normalized maximum CPU capacity so
+ * asym-packing also orders physical cores by performance. Firmware does not
+ * currently describe the PE preference, so detect Olympus by MIDR until a
+ * firmware interface is available.
+ */
+static bool olympus_prefer_pe0 __ro_after_init;
+
+static int arm64_asym_packing_flags(void)
+{
+	return olympus_prefer_pe0 ? SD_ASYM_PACKING : 0;
+}
+
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	return cpu_smt_flags() | arm64_asym_packing_flags();
+}
+#endif
+
+#ifdef CONFIG_SCHED_CLUSTER
+static int arm64_cluster_flags(void)
+{
+	return cpu_cluster_flags() | arm64_asym_packing_flags();
+}
+#endif
+
+#ifdef CONFIG_SCHED_MC
+static int arm64_core_flags(void)
+{
+	return cpu_core_flags() | arm64_asym_packing_flags();
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, arm64_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, arm64_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, arm64_asym_packing_flags, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	olympus_prefer_pe0 = true;
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling capacity and PE asym-packing for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	int priority;
+
+	if (!olympus_prefer_pe0)
+		return 0;
+
+	/* cpu_scale preserves the ordering provided by CPPC highest_perf. */
+	priority = topology_get_cpu_scale(cpu);
+	if (MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0)
+		priority *= 2;
+
+	return priority;
+}
+
+bool arch_asym_cpu_capacity_enabled(void)
+{
+	return !olympus_prefer_pe0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h
index b5d9d7c2b8add..dd40b8f466ca0 100644
--- a/include/linux/sched/topology.h
+++ b/include/linux/sched/topology.h
@@ -50,6 +50,8 @@ extern const struct cpumask *tl_mc_mask(struct sched_domain_topology_level *tl,
 extern const struct cpumask *tl_pkg_mask(struct sched_domain_topology_level *tl, int cpu);
 
 extern int arch_asym_cpu_priority(int cpu);
+/* Return false when the architecture represents capacity through packing. */
+bool arch_asym_cpu_capacity_enabled(void);
 
 struct sched_domain_attr {
 	int relax_domain_level;
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 0248227d983a7..8ccc734efd748 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1682,6 +1682,9 @@ asym_cpu_capacity_classify(const struct cpumask *sd_span,
 	struct asym_cap_data *entry;
 	int count = 0, miss = 0;
 
+	if (!arch_asym_cpu_capacity_enabled())
+		return 0;
+
 	/*
 	 * Count how many unique CPU capacities this domain spans across
 	 * (compare sched_domain CPUs mask with ones representing  available
@@ -1709,6 +1712,11 @@ asym_cpu_capacity_classify(const struct cpumask *sd_span,
 
 }
 
+bool __weak arch_asym_cpu_capacity_enabled(void)
+{
+	return true;
+}
+
 static void free_asym_cap_entry(struct rcu_head *head)
 {
 	struct asym_cap_data *entry = container_of(head, struct asym_cap_data, rcu);

Thanks,
-Andrea

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

Hi Christian,

On Mon, Aug 31, 2026 at 10:13:42PM +0100, Christian Loehle wrote:
> On 8/31/26 19:10, Andrea Righi wrote:
> > NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
> > capacity but two different resource modes. One-Thread Active mode gives
> > one PE the full core, while waking the other PE restores Two-Thread
> > Active mode and partitions decode, issue, cache, TLB, and vector
> > resources. Returning to full-resource mode requires the sibling to
> > remain in WFI for 10 Ki cycles.
> > 
> > Measurements show that pinned workloads perform equally on either PE,
> > but freely migratable workloads lose substantial throughput when they
> > alternate between PE identities. Consistently selecting PE0 keeps PE1
> > idle, avoids repeated SMT repartitioning, and restores one-thread-per-core
> > performance.
> > 
> > Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
> > identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This is
> > independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal capacity,
> > while physical cores with different maximum frequencies are handled by
> > a higher scheduling domain.
> 
> But why? This asympacking + CAS interaction is a bit hard to comprehend IMV.

The two mehcanisms describe different preferences at different scheduling domain
levels: SD_ASYM_CPUCAPACITY selects among physical cores with different max
capacities, SD_ASYM_PACKING is set only on the SMT domain and selects the
canonical PE within the core chose by the existing placement and capacity logic.

> Why can't we encode both preferences in the asym-packing priority, e.g.
> 
> priority(cpu) = is_primary(cpu) ? 2 * highest_perf(cpu)
>                                 : highest_perf(cpu)
> 
> so that all primary PEs are preferred over all sibling PEs, while still
> preserving the highest_perf ordering within each group, and do away with
> SD_ASYM_CPUCAPACITY on Vera altogether?

I can experiment with this combined priority, but I think removing
SD_ASYM_CPUCAPACITY is a separate policy change rather than an alternative
implementation of this fix.

A static asym-packing priority does not preserve the capacity-aware semantics
used for task fitting, uclamp, misfit handling and migration. The current
approach keeps those semantics when selecting a physical core, then applies the
PE preference only within that core.

Also, encoding the combined priority alone would not fix the problem addressed
by patch 2: the idle-selection paths currently do not consult asymmetric SMT
priority. They can still return an arbitrary idle sibling regardless of how
arch_asym_cpu_priority() is defined. Patch 2 adds that missing behavior and
scopes it to the shared-capacity SMT domain.

> I had suggested this a while ago, did you have a stab at that by any chance,
> too?

I tested your CPPC-based asym-packing series, but not this particular
combined-priority variant. IIUC the earlier proposal was replacing
capacity-aware scheduling for minor physical-core capacity differences, SMT
sibling ordering looks like an orthogonal problem.

And at the time, the combined SMT-aware SD_ASYM_CPUCAPACITY approach also gave
the best Vera results of the alternatives I tested, which is another reason I
kept physical-core capacity selection separate here.

> Am I missing something altogether?

Combining the priorities is a valid experiment, but I'm not sure if it
completely solves the problem by itself, I'll give it a try and share the
results.

Thanks,
-Andrea

> 
> > 
> > Firmware currently provides no interface for describing the preferred
> > SMT sibling. Detect Olympus by MIDR until such an interface is available.
> > 
> > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> > ---
> >  arch/arm64/include/asm/topology.h |  1 +
> >  arch/arm64/kernel/smp.c           |  1 +
> >  arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
> >  3 files changed, 64 insertions(+)
> > > 
> > diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
> > index b9eaf4ad70850..edc1c59b3448d 100644
> > --- a/arch/arm64/include/asm/topology.h
> > +++ b/arch/arm64/include/asm/topology.h
> > @@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
> >  #include <linux/arch_topology.h>
> >  
> >  void update_freq_counters_refs(void);
> > +void arm64_init_sched_topology(void);
> >  
> >  /* Replace task scheduler's default frequency-invariant accounting */
> >  #define arch_scale_freq_tick topology_scale_freq_tick
> > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> > index a61dc3016a117..0135ac4eea8bd 100644
> > --- a/arch/arm64/kernel/smp.c
> > +++ b/arch/arm64/kernel/smp.c
> > @@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
> >  	hyp_mode_check();
> >  	setup_system_features();
> >  	setup_user_features();
> > +	arm64_init_sched_topology();
> >  	mark_linear_text_alias_ro();
> >  }
> >  
> > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> > index d28438f8b83f1..0dd9eec1c4946 100644
> > --- a/arch/arm64/kernel/topology.c
> > +++ b/arch/arm64/kernel/topology.c
> > @@ -19,6 +19,8 @@
> >  #include <linux/init.h>
> >  #include <linux/percpu.h>
> >  #include <linux/sched/isolation.h>
> > +#include <linux/sched/topology.h>
> > +#include <linux/smp.h>
> >  #include <linux/xarray.h>
> >  
> >  #include <asm/cpu.h>
> > @@ -44,6 +46,66 @@
> >  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
> >  static cpumask_var_t amu_fie_cpus;
> >  
> > +/*
> > + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> > + * two-thread active mode, with resources partitioned between the PEs.
> > + *
> > + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> > + * mode. Firmware does not currently describe this preference, so detect
> > + * Olympus by MIDR until a firmware interface is available.
> > + */
> > +static bool olympus_prefer_pe0 __ro_after_init;
> > +
> > +#ifdef CONFIG_SCHED_SMT
> > +static int arm64_smt_flags(void)
> > +{
> > +	int flags = cpu_smt_flags();
> > +
> > +	if (olympus_prefer_pe0)
> > +		flags |= SD_ASYM_PACKING;
> > +
> > +	return flags;
> > +}
> > +#endif
> > +
> > +static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
> > +#ifdef CONFIG_SCHED_SMT
> > +	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
> > +#endif
> > +#ifdef CONFIG_SCHED_CLUSTER
> > +	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
> > +#endif
> > +#ifdef CONFIG_SCHED_MC
> > +	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
> > +#endif
> > +	SDTL_INIT(tl_pkg_mask, NULL, PKG),
> > +	{ NULL, },
> > +};
> > +
> > +void __init arm64_init_sched_topology(void)
> > +{
> > +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> > +		return;
> > +
> > +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> > +		return;
> > +
> > +	if (!topology_core_has_smt(smp_processor_id()))
> > +		return;
> > +
> > +	olympus_prefer_pe0 = true;
> > +	set_sched_topology(arm64_asym_smt_topology);
> > +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
> > +}
> > +
> > +int arch_asym_cpu_priority(int cpu)
> > +{
> > +	if (!olympus_prefer_pe0)
> > +		return 0;
> > +
> > +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> > +}
> > +
> >  struct amu_cntr_sample {
> >  	u64		arch_const_cycles_prev;
> >  	u64		arch_core_cycles_prev;
> 

Re: [PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Christian Loehle]

On 8/31/26 19:10, Andrea Righi wrote:
> NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
> capacity but two different resource modes. One-Thread Active mode gives
> one PE the full core, while waking the other PE restores Two-Thread
> Active mode and partitions decode, issue, cache, TLB, and vector
> resources. Returning to full-resource mode requires the sibling to
> remain in WFI for 10 Ki cycles.
> 
> Measurements show that pinned workloads perform equally on either PE,
> but freely migratable workloads lose substantial throughput when they
> alternate between PE identities. Consistently selecting PE0 keeps PE1
> idle, avoids repeated SMT repartitioning, and restores one-thread-per-core
> performance.
> 
> Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
> identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This is
> independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal capacity,
> while physical cores with different maximum frequencies are handled by
> a higher scheduling domain.

But why? This asympacking + CAS interaction is a bit hard to comprehend IMV.
Why can't we encode both preferences in the asym-packing priority, e.g.

priority(cpu) = is_primary(cpu) ? 2 * highest_perf(cpu)
                                : highest_perf(cpu)

so that all primary PEs are preferred over all sibling PEs, while still
preserving the highest_perf ordering within each group, and do away with
SD_ASYM_CPUCAPACITY on Vera altogether?
I had suggested this a while ago, did you have a stab at that by any chance,
too?
Am I missing something altogether?

> 
> Firmware currently provides no interface for describing the preferred
> SMT sibling. Detect Olympus by MIDR until such an interface is available.
> 
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> ---
>  arch/arm64/include/asm/topology.h |  1 +
>  arch/arm64/kernel/smp.c           |  1 +
>  arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
>  3 files changed, 64 insertions(+)
> > 
> diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
> index b9eaf4ad70850..edc1c59b3448d 100644
> --- a/arch/arm64/include/asm/topology.h
> +++ b/arch/arm64/include/asm/topology.h
> @@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
>  #include <linux/arch_topology.h>
>  
>  void update_freq_counters_refs(void);
> +void arm64_init_sched_topology(void);
>  
>  /* Replace task scheduler's default frequency-invariant accounting */
>  #define arch_scale_freq_tick topology_scale_freq_tick
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index a61dc3016a117..0135ac4eea8bd 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
>  	hyp_mode_check();
>  	setup_system_features();
>  	setup_user_features();
> +	arm64_init_sched_topology();
>  	mark_linear_text_alias_ro();
>  }
>  
> diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
> index d28438f8b83f1..0dd9eec1c4946 100644
> --- a/arch/arm64/kernel/topology.c
> +++ b/arch/arm64/kernel/topology.c
> @@ -19,6 +19,8 @@
>  #include <linux/init.h>
>  #include <linux/percpu.h>
>  #include <linux/sched/isolation.h>
> +#include <linux/sched/topology.h>
> +#include <linux/smp.h>
>  #include <linux/xarray.h>
>  
>  #include <asm/cpu.h>
> @@ -44,6 +46,66 @@
>  static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
>  static cpumask_var_t amu_fie_cpus;
>  
> +/*
> + * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
> + * two-thread active mode, with resources partitioned between the PEs.
> + *
> + * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
> + * mode. Firmware does not currently describe this preference, so detect
> + * Olympus by MIDR until a firmware interface is available.
> + */
> +static bool olympus_prefer_pe0 __ro_after_init;
> +
> +#ifdef CONFIG_SCHED_SMT
> +static int arm64_smt_flags(void)
> +{
> +	int flags = cpu_smt_flags();
> +
> +	if (olympus_prefer_pe0)
> +		flags |= SD_ASYM_PACKING;
> +
> +	return flags;
> +}
> +#endif
> +
> +static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
> +#ifdef CONFIG_SCHED_SMT
> +	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
> +#endif
> +#ifdef CONFIG_SCHED_CLUSTER
> +	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
> +#endif
> +#ifdef CONFIG_SCHED_MC
> +	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
> +#endif
> +	SDTL_INIT(tl_pkg_mask, NULL, PKG),
> +	{ NULL, },
> +};
> +
> +void __init arm64_init_sched_topology(void)
> +{
> +	if (!IS_ENABLED(CONFIG_SCHED_SMT))
> +		return;
> +
> +	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
> +		return;
> +
> +	if (!topology_core_has_smt(smp_processor_id()))
> +		return;
> +
> +	olympus_prefer_pe0 = true;
> +	set_sched_topology(arm64_asym_smt_topology);
> +	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
> +}
> +
> +int arch_asym_cpu_priority(int cpu)
> +{
> +	if (!olympus_prefer_pe0)
> +		return 0;
> +
> +	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
> +}
> +
>  struct amu_cntr_sample {
>  	u64		arch_const_cycles_prev;
>  	u64		arch_core_cycles_prev;

[PATCH 1/2] arm64: topology: Prefer PE0 on NVIDIA Olympus SMT cores

[Andrea Righi]

NVIDIA Olympus implements spatial SMT with symmetric steady-state PE
capacity but two different resource modes. One-Thread Active mode gives
one PE the full core, while waking the other PE restores Two-Thread
Active mode and partitions decode, issue, cache, TLB, and vector
resources. Returning to full-resource mode requires the sibling to
remain in WFI for 10 Ki cycles.

Measurements show that pinned workloads perform equally on either PE,
but freely migratable workloads lose substantial throughput when they
alternate between PE identities. Consistently selecting PE0 keeps PE1
idle, avoids repeated SMT repartitioning, and restores one-thread-per-core
performance.

Describe this scheduling preference with SD_ASYM_PACKING and give PE0,
identified by MPIDR_EL1.Aff0, the higher arch_asym_cpu_priority(). This is
independent of SD_ASYM_CPUCAPACITY: SMT siblings retain equal capacity,
while physical cores with different maximum frequencies are handled by
a higher scheduling domain.

Firmware currently provides no interface for describing the preferred
SMT sibling. Detect Olympus by MIDR until such an interface is available.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 arch/arm64/include/asm/topology.h |  1 +
 arch/arm64/kernel/smp.c           |  1 +
 arch/arm64/kernel/topology.c      | 62 +++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)

diff --git a/arch/arm64/include/asm/topology.h b/arch/arm64/include/asm/topology.h
index b9eaf4ad70850..edc1c59b3448d 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -18,6 +18,7 @@ int pcibus_to_node(struct pci_bus *bus);
 #include <linux/arch_topology.h>
 
 void update_freq_counters_refs(void);
+void arm64_init_sched_topology(void);
 
 /* Replace task scheduler's default frequency-invariant accounting */
 #define arch_scale_freq_tick topology_scale_freq_tick
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index a61dc3016a117..0135ac4eea8bd 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -443,6 +443,7 @@ void __init smp_cpus_done(unsigned int max_cpus)
 	hyp_mode_check();
 	setup_system_features();
 	setup_user_features();
+	arm64_init_sched_topology();
 	mark_linear_text_alias_ro();
 }
 
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index d28438f8b83f1..0dd9eec1c4946 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -19,6 +19,8 @@
 #include <linux/init.h>
 #include <linux/percpu.h>
 #include <linux/sched/isolation.h>
+#include <linux/sched/topology.h>
+#include <linux/smp.h>
 #include <linux/xarray.h>
 
 #include <asm/cpu.h>
@@ -44,6 +46,66 @@
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, arch_max_freq_scale) =  1UL << (2 * SCHED_CAPACITY_SHIFT);
 static cpumask_var_t amu_fie_cpus;
 
+/*
+ * Switching the active PE on an NVIDIA Olympus SMT core can keep the core in
+ * two-thread active mode, with resources partitioned between the PEs.
+ *
+ * Prefer PE0 so PE1 can remain idle and the core can stay in full-resource
+ * mode. Firmware does not currently describe this preference, so detect
+ * Olympus by MIDR until a firmware interface is available.
+ */
+static bool olympus_prefer_pe0 __ro_after_init;
+
+#ifdef CONFIG_SCHED_SMT
+static int arm64_smt_flags(void)
+{
+	int flags = cpu_smt_flags();
+
+	if (olympus_prefer_pe0)
+		flags |= SD_ASYM_PACKING;
+
+	return flags;
+}
+#endif
+
+static struct sched_domain_topology_level arm64_asym_smt_topology[] = {
+#ifdef CONFIG_SCHED_SMT
+	SDTL_INIT(tl_smt_mask, arm64_smt_flags, SMT),
+#endif
+#ifdef CONFIG_SCHED_CLUSTER
+	SDTL_INIT(tl_cls_mask, cpu_cluster_flags, CLS),
+#endif
+#ifdef CONFIG_SCHED_MC
+	SDTL_INIT(tl_mc_mask, cpu_core_flags, MC),
+#endif
+	SDTL_INIT(tl_pkg_mask, NULL, PKG),
+	{ NULL, },
+};
+
+void __init arm64_init_sched_topology(void)
+{
+	if (!IS_ENABLED(CONFIG_SCHED_SMT))
+		return;
+
+	if ((read_cpuid_id() & MIDR_CPU_MODEL_MASK) != MIDR_NVIDIA_OLYMPUS)
+		return;
+
+	if (!topology_core_has_smt(smp_processor_id()))
+		return;
+
+	olympus_prefer_pe0 = true;
+	set_sched_topology(arm64_asym_smt_topology);
+	pr_info("Enabling PE0 SMT preference for NVIDIA Olympus\n");
+}
+
+int arch_asym_cpu_priority(int cpu)
+{
+	if (!olympus_prefer_pe0)
+		return 0;
+
+	return MPIDR_AFFINITY_LEVEL(cpu_logical_map(cpu), 0) == 0;
+}
+
 struct amu_cntr_sample {
 	u64		arch_const_cycles_prev;
 	u64		arch_core_cycles_prev;
-- 
2.55.0