Re: Cache-aware scheduling does not work well with amd big/little cores

[Chen Yu <yu.c.chen@intel.com>]

On Mon, Sep 14, 2026 at 12:27:53PM +0200, Klaus Kusche wrote:
> 
> I did some very quick tests.
> 
> 1.) /sys/kernel/sched/debug/domains/* does not exist on my system,
> not even with debug_fs on.
> 
> /sys/kernel/debug/x86/sched_itmt_enabled is "Y",
> /sys/kernel/debug/x86/sched_core_priority looks good
> (big cores have values almost twice as high as small cores)
> 
> 2.) The situation with 7.2.5 which seems to include 
> https://lore.kernel.org/lkml/20260825174112.2580942-1-tim.c.chen@linux.intel.com/
> is almost unchanged: Without cache aware scheduling, 
> my build jobs run faster (wallclock time):
> Just 6:16 compared to 6:19 for my kernel build with full LTO,
> but 4:50 compared to 5:20 (???) for my python uv build
> (the python uv build seems to be a very interesting test case?)
> 
> However, with cache sched enabled, in spite of the longer 
> wallclock time, cpu seconds are sometimes a little bit lower.
> 
> 3.) https://lore.kernel.org/lkml/20260810033742.1688718-1-yu.c.chen@intel.com/

This patch inhibits ASYM_PACKING and favors Cache-Aware-Scheduling,
which is the opposite of what your platform expects.

> seems to make things much worse:
> Kernel builds had the LTO step and the CC compressed step
> placed on little cores for significant amounts of time,
> resulting in total build times above 8 minutes.
> Same impression by watching the bar graph for individual cores
> for the uv build.
>

After a second thought, I wonder if we should disable Cache-Aware
Scheduling if ASYM_PACKING is enabled, because the latter would
prefer a higher priority CPU rather than just choosing a random
L3 to aggregate the threads. Would the following patch work
for you? (just compile tested, as I do not have a multi-LLC hybrid
AMD platform for testing) Thanks.

diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 0248227d983a..07bc8a302574 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -684,6 +684,7 @@ DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
 
 DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
 DEFINE_STATIC_KEY_FALSE(sched_cluster_active);
+DEFINE_STATIC_KEY_FALSE(sched_asym_packing_active);
 
 static void update_top_cache_domain(int cpu)
 {
@@ -957,6 +958,11 @@ static void _sched_cache_active_set(void)
 		return;
 	}
 
+	if (static_branch_unlikely(&sched_asym_packing_active)) {
+		static_branch_disable_cpuslocked(&sched_cache_active);
+		return;
+	}
+
 	/*
 	 * user wants it or not ?
 	 * TBD: read before writing the static key.
@@ -3085,6 +3091,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 	int i, ret = -ENOMEM;
 	bool has_asym = false;
 	bool has_cluster = false;
+	bool has_asym_packing = false;
 
 	if (WARN_ON(cpumask_empty(cpu_map)))
 		goto error;
@@ -3204,6 +3211,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 
 		if (lowest_flag_domain(i, SD_CLUSTER))
 			has_cluster = true;
+
+		if (highest_flag_domain(i, SD_ASYM_PACKING))
+			has_asym_packing = true;
 	}
 	rcu_read_unlock();
 
@@ -3213,6 +3223,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
 	if (has_cluster)
 		static_branch_inc_cpuslocked(&sched_cluster_active);
 
+	if (has_asym_packing)
+		static_branch_inc_cpuslocked(&sched_asym_packing_active);
+
 	if (rq && sched_debug_verbose)
 		pr_info("root domain span: %*pbl\n", cpumask_pr_args(cpu_map));
 
@@ -3318,6 +3331,9 @@ static void detach_destroy_domains(const struct cpumask *cpu_map)
 	if (static_branch_unlikely(&sched_cluster_active))
 		static_branch_dec_cpuslocked(&sched_cluster_active);
 
+	if (rcu_access_pointer(per_cpu(sd_asym_packing, cpu)))
+		static_branch_dec_cpuslocked(&sched_asym_packing_active);
+
 	rcu_read_lock();
 	for_each_cpu(i, cpu_map)
 		cpu_attach_domain(NULL, &def_root_domain, i);
-- 
2.43.0