[PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik via B4 Relay]

From: Shubhang Kaushik <shubhang@os.amperecomputing.com>

When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
sibling CPU might migrate away from its previous CPU even if that CPU
is not overutilized. This sacrifices cache locality and introduces
unnecessary migration overhead.

This patch refines the wakeup heuristic in `select_idle_sibling()`. If
EAS is active and the task's previous CPU (`prev`) is not overutilized,
the scheduler will prioritize waking the task on `prev`, avoiding an
unneeded migration and preserving cache-hotness.

---
v2:
- Addressed reviewer comments to handle this special condition
  within the selection logic, prioritizing the
  previous CPU if not overutilized for EAS.
- Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/

Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
---
 kernel/sched/fair.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 	    asym_fits_cpu(task_util, util_min, util_max, target))
 		return target;
 
-	/*
-	 * If the previous CPU is cache affine and idle, don't be stupid:
-	 */
+	/* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
 	if (prev != target && cpus_share_cache(prev, target) &&
 	    (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
 	    asym_fits_cpu(task_util, util_min, util_max, prev)) {
@@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
 		prev_aff = prev;
 	}
 
+	/*
+	 * If the previous CPU is not overutilized, prefer it for cache locality.
+	 * This prevents migration away from a cache-hot CPU that can still
+	 * handle the task without causing an overload.
+	 */
+	if (sched_energy_enabled() && !cpu_overutilized(prev))
+		return prev;
+
 	/*
 	 * Allow a per-cpu kthread to stack with the wakee if the
 	 * kworker thread and the tasks previous CPUs are the same.

---
base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
change-id: 20251030-b4-follow-up-ff03b4533a2d

Best regards,
-- 
Shubhang Kaushik <shubhang@os.amperecomputing.com>

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Christian Loehle]

On 10/30/25 19:19, Shubhang Kaushik via B4 Relay wrote:
> From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> 
> When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> sibling CPU might migrate away from its previous CPU even if that CPU
> is not overutilized. This sacrifices cache locality and introduces
> unnecessary migration overhead.
> 
> This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> EAS is active and the task's previous CPU (`prev`) is not overutilized,
> the scheduler will prioritize waking the task on `prev`, avoiding an
> unneeded migration and preserving cache-hotness.
> 
> ---
> v2:
> - Addressed reviewer comments to handle this special condition
>   within the selection logic, prioritizing the
>   previous CPU if not overutilized for EAS.
> - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
> 
> Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> ---
>  kernel/sched/fair.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>  	    asym_fits_cpu(task_util, util_min, util_max, target))
>  		return target;
>  
> -	/*
> -	 * If the previous CPU is cache affine and idle, don't be stupid:
> -	 */
> +	/* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
>  	if (prev != target && cpus_share_cache(prev, target) &&
>  	    (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
>  	    asym_fits_cpu(task_util, util_min, util_max, prev)) {
> @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>  		prev_aff = prev;
>  	}
>  
> +	/*
> +	 * If the previous CPU is not overutilized, prefer it for cache locality.
> +	 * This prevents migration away from a cache-hot CPU that can still
> +	 * handle the task without causing an overload.
> +	 */
> +	if (sched_energy_enabled() && !cpu_overutilized(prev))
> +		return prev;
> +
>  	/*
>  	 * Allow a per-cpu kthread to stack with the wakee if the
>  	 * kworker thread and the tasks previous CPUs are the same.
> 
> ---
> base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> change-id: 20251030-b4-follow-up-ff03b4533a2d
> 
> Best regards,

So if you're actually targetting EAS I don't get why you would check overutilized (instead
of asym_fits, what about uclamp?) but also, given that many EAS systems have only one common
llc I don't quite get why you would want this anyway.
Do you have a system / workload showing a benefit?
(I find with EAS heavily relying on wakeups, what we do in the slow path isn't that important
for most workloads...)

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik OS]

Yes, I agree that EAS approach is not suitable in this case as they require a heterogenous CPU topology.
The issue is that the existing checks are for a completely idle CPU, whereas `cpu_overutilized` implies
the CPU is busy but not yet overloaded. I ventured into EAS as this `cpu_overutilized` relies on
`sched_energy_enabled()` being active. The point I wanted to convey is that - we still need a `cpu_busy?`
check to make use of the cache locality - for SMP systems. Would appreciate some pointers on the same lines..

Regards,
Shubhang

________________________________________
From: Christian Loehle <christian.loehle@arm.com>
Sent: Friday, October 31, 2025 3:17 AM
To: Shubhang Kaushik OS; Ingo Molnar; Peter Zijlstra; Juri Lelli; Vincent Guittot; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang
Cc: Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

On 10/30/25 19:19, Shubhang Kaushik via B4 Relay wrote:
> From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
>
> When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> sibling CPU might migrate away from its previous CPU even if that CPU
> is not overutilized. This sacrifices cache locality and introduces
> unnecessary migration overhead.
>
> This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> EAS is active and the task's previous CPU (`prev`) is not overutilized,
> the scheduler will prioritize waking the task on `prev`, avoiding an
> unneeded migration and preserving cache-hotness.
>
> ---
> v2:
> - Addressed reviewer comments to handle this special condition
>   within the selection logic, prioritizing the
>   previous CPU if not overutilized for EAS.
> - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
>
> Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> ---
>  kernel/sched/fair.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>           asym_fits_cpu(task_util, util_min, util_max, target))
>               return target;
>
> -     /*
> -      * If the previous CPU is cache affine and idle, don't be stupid:
> -      */
> +     /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
>       if (prev != target && cpus_share_cache(prev, target) &&
>           (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
>           asym_fits_cpu(task_util, util_min, util_max, prev)) {
> @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>               prev_aff = prev;
>       }
>
> +     /*
> +      * If the previous CPU is not overutilized, prefer it for cache locality.
> +      * This prevents migration away from a cache-hot CPU that can still
> +      * handle the task without causing an overload.
> +      */
> +     if (sched_energy_enabled() && !cpu_overutilized(prev))
> +             return prev;
> +
>       /*
>        * Allow a per-cpu kthread to stack with the wakee if the
>        * kworker thread and the tasks previous CPUs are the same.
>
> ---
> base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> change-id: 20251030-b4-follow-up-ff03b4533a2d
>
> Best regards,

So if you're actually targetting EAS I don't get why you would check overutilized (instead
of asym_fits, what about uclamp?) but also, given that many EAS systems have only one common
llc I don't quite get why you would want this anyway.
Do you have a system / workload showing a benefit?
(I find with EAS heavily relying on wakeups, what we do in the slow path isn't that important
for most workloads...)

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Christian Loehle]

On 10/31/25 16:59, Shubhang Kaushik OS wrote:
> Yes, I agree that EAS approach is not suitable in this case as they require a heterogenous CPU topology.
> The issue is that the existing checks are for a completely idle CPU, whereas `cpu_overutilized` implies
> the CPU is busy but not yet overloaded. I ventured into EAS as this `cpu_overutilized` relies on
> `sched_energy_enabled()` being active. The point I wanted to convey is that - we still need a `cpu_busy?`
> check to make use of the cache locality - for SMP systems. Would appreciate some pointers on the same lines..

So the main issue is that with existing code if a CPU is "overloaded" isn't all that well defined.
For EAS we know if !rd->overloaded => all CPUs are !cpu_overutilized(). We could just pick any where
the task still fits (and we do).
For SMP what 'overloaded' will actually mean depends on the rest of the system (or at least domain). 


> Regards,
> Shubhang
> [sip]

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Madadi Vineeth Reddy]

Hi Shubhang,

On 31/10/25 00:49, Shubhang Kaushik wrote:
> When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> sibling CPU might migrate away from its previous CPU even if that CPU
> is not overutilized. This sacrifices cache locality and introduces
> unnecessary migration overhead.
> 
> This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> EAS is active and the task's previous CPU (`prev`) is not overutilized,
> the scheduler will prioritize waking the task on `prev`, avoiding an
> unneeded migration and preserving cache-hotness.
> 
> ---
> v2:
> - Addressed reviewer comments to handle this special condition
>   within the selection logic, prioritizing the
>   previous CPU if not overutilized for EAS.
> - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
> 
> Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> ---
>  kernel/sched/fair.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>  	    asym_fits_cpu(task_util, util_min, util_max, target))
>  		return target;
>  
> -	/*
> -	 * If the previous CPU is cache affine and idle, don't be stupid:
> -	 */
> +	/* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
>  	if (prev != target && cpus_share_cache(prev, target) &&
>  	    (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
>  	    asym_fits_cpu(task_util, util_min, util_max, prev)) {
> @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>  		prev_aff = prev;
>  	}
>  
> +	/*
> +	 * If the previous CPU is not overutilized, prefer it for cache locality.
> +	 * This prevents migration away from a cache-hot CPU that can still
> +	 * handle the task without causing an overload.
> +	 */
> +	if (sched_energy_enabled() && !cpu_overutilized(prev))
> +		return prev;
> +

The above !cpu_overutilized(prev) is placed before recent_used_cpu idle check.
This means if prev is busy (but not overutilized) and recent_used_cpu is completely
idle, the task returns to prev and misses the idle opportunity.

Is cache locality prioritized even over idle CPU availability?
Are there measurements showing this trade-off is worthwhile for real workloads?

Thank you,
Madadi Vineeth Reddy

>  	/*
>  	 * Allow a per-cpu kthread to stack with the wakee if the
>  	 * kworker thread and the tasks previous CPUs are the same.
> 
> ---
> base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> change-id: 20251030-b4-follow-up-ff03b4533a2d
> 
> Best regards,

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Christian Loehle]

On 10/31/25 18:11, Christian Loehle wrote:
> On 10/31/25 16:59, Shubhang Kaushik OS wrote:
>> Yes, I agree that EAS approach is not suitable in this case as they require a heterogenous CPU topology.
>> The issue is that the existing checks are for a completely idle CPU, whereas `cpu_overutilized` implies
>> the CPU is busy but not yet overloaded. I ventured into EAS as this `cpu_overutilized` relies on
>> `sched_energy_enabled()` being active. The point I wanted to convey is that - we still need a `cpu_busy?`
>> check to make use of the cache locality - for SMP systems. Would appreciate some pointers on the same lines..
> 
> So the main issue is that with existing code if a CPU is "overloaded" isn't all that well defined.
> For EAS we know if !rd->overloaded => all CPUs are !cpu_overutilized(). We could just pick any where

rd->overutilized is what I meant of course...
Hopefully the point is still clear, with EAS (and CAS given some limitations) we can answer the "is this
CPU _busy_?" question by just looking at that CPU.

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Vincent Guittot]

On Thu, 30 Oct 2025 at 20:19, Shubhang Kaushik via B4 Relay
<devnull+shubhang.os.amperecomputing.com@kernel.org> wrote:
>
> From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
>
> When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> sibling CPU might migrate away from its previous CPU even if that CPU
> is not overutilized. This sacrifices cache locality and introduces
> unnecessary migration overhead.
>
> This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> EAS is active and the task's previous CPU (`prev`) is not overutilized,
> the scheduler will prioritize waking the task on `prev`, avoiding an
> unneeded migration and preserving cache-hotness.
>
> ---
> v2:
> - Addressed reviewer comments to handle this special condition
>   within the selection logic, prioritizing the
>   previous CPU if not overutilized for EAS.
> - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
>
> Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> ---
>  kernel/sched/fair.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>             asym_fits_cpu(task_util, util_min, util_max, target))
>                 return target;
>
> -       /*
> -        * If the previous CPU is cache affine and idle, don't be stupid:
> -        */
> +       /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
>         if (prev != target && cpus_share_cache(prev, target) &&
>             (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
>             asym_fits_cpu(task_util, util_min, util_max, prev)) {
> @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>                 prev_aff = prev;
>         }
>
> +       /*
> +        * If the previous CPU is not overutilized, prefer it for cache locality.
> +        * This prevents migration away from a cache-hot CPU that can still
> +        * handle the task without causing an overload.
> +        */
> +       if (sched_energy_enabled() && !cpu_overutilized(prev))

From your previous answer on v1, I don't think that you use
heterogeneous system so eas will not be enabled in your case and even
when used find_energy_efficient_cpu() will be called before

select_idle_sibling looks for an idle cpu that shares the cache with
target, Isn't such migration inside the same LLC good in your case ?

Otherwise you might want to check in wake_affine() where we decide
between local cpu and previous cpu which one should be the target.
This can have an impact especially if there are not in the same LLC

> +               return prev;
> +
>         /*
>          * Allow a per-cpu kthread to stack with the wakee if the
>          * kworker thread and the tasks previous CPUs are the same.
>
> ---
> base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> change-id: 20251030-b4-follow-up-ff03b4533a2d
>
> Best regards,
> --
> Shubhang Kaushik <shubhang@os.amperecomputing.com>
>
>

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik OS]

> From your previous answer on v1, I don't think that you use
> heterogeneous system so eas will not be enabled in your case and even
> when used find_energy_efficient_cpu() will be called before

I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.

> Otherwise you might want to check in wake_affine() where we decide
> between local cpu and previous cpu which one should be the target.
> This can have an impact especially if there are not in the same LLC

While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.
We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.

Thanks,
Shubhang Kaushik

________________________________________
From: Vincent Guittot <vincent.guittot@linaro.org>
Sent: Monday, November 3, 2025 1:04 AM
To: Shubhang Kaushik OS
Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

On Thu, 30 Oct 2025 at 20:19, Shubhang Kaushik via B4 Relay
<devnull+shubhang.os.amperecomputing.com@kernel.org> wrote:
>
> From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
>
> When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> sibling CPU might migrate away from its previous CPU even if that CPU
> is not overutilized. This sacrifices cache locality and introduces
> unnecessary migration overhead.
>
> This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> EAS is active and the task's previous CPU (`prev`) is not overutilized,
> the scheduler will prioritize waking the task on `prev`, avoiding an
> unneeded migration and preserving cache-hotness.
>
> ---
> v2:
> - Addressed reviewer comments to handle this special condition
>   within the selection logic, prioritizing the
>   previous CPU if not overutilized for EAS.
> - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
>
> Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> ---
>  kernel/sched/fair.c | 12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>             asym_fits_cpu(task_util, util_min, util_max, target))
>                 return target;
>
> -       /*
> -        * If the previous CPU is cache affine and idle, don't be stupid:
> -        */
> +       /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
>         if (prev != target && cpus_share_cache(prev, target) &&
>             (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
>             asym_fits_cpu(task_util, util_min, util_max, prev)) {
> @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
>                 prev_aff = prev;
>         }
>
> +       /*
> +        * If the previous CPU is not overutilized, prefer it for cache locality.
> +        * This prevents migration away from a cache-hot CPU that can still
> +        * handle the task without causing an overload.
> +        */
> +       if (sched_energy_enabled() && !cpu_overutilized(prev))

From your previous answer on v1, I don't think that you use
heterogeneous system so eas will not be enabled in your case and even
when used find_energy_efficient_cpu() will be called before

select_idle_sibling looks for an idle cpu that shares the cache with
target, Isn't such migration inside the same LLC good in your case ?

Otherwise you might want to check in wake_affine() where we decide
between local cpu and previous cpu which one should be the target.
This can have an impact especially if there are not in the same LLC

> +               return prev;
> +
>         /*
>          * Allow a per-cpu kthread to stack with the wakee if the
>          * kworker thread and the tasks previous CPUs are the same.
>
> ---
> base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> change-id: 20251030-b4-follow-up-ff03b4533a2d
>
> Best regards,
> --
> Shubhang Kaushik <shubhang@os.amperecomputing.com>
>
>

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Dietmar Eggemann]

On 13.11.25 01:26, Shubhang Kaushik OS wrote:
>> From your previous answer on v1, I don't think that you use
>> heterogeneous system so eas will not be enabled in your case and even
>> when used find_energy_efficient_cpu() will be called before
> 
> I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.
> 
>> Otherwise you might want to check in wake_affine() where we decide
>> between local cpu and previous cpu which one should be the target.
>> This can have an impact especially if there are not in the same LLC
> 
> While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.

Which testcases are you running on your Altra box? I assume it's a
single NUMA node (80 CPUs).

For us, 'perf bench sched messaging` w/o CONFIG_SCHED_CLUSTER, so only
PKG SD (i.e. sis() only returns prev or this CPU) gives better results
then w/ CONFIG_SCHED_CLUSTER.

> We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.

IMHO, the scheduler only cares about shared LLC (and shared L2 with
CONFIG_SCHED_CLUSTER). Can you check:

$ cat /sys/devices/system/cpu/cpu0/cache/index*/{type,shared_cpu_map}
Data
Instruction
Unified
Unified                                                 <-- (1)
00000000,00000000,00000000,00000000,00000001
00000000,00000000,00000000,00000000,00000001
00000000,00000000,00000000,00000000,00000001
CPU mask > 00000000,00000000,00000000,00000000,00000001 <-- (1)

Does (1) exists? IMHO it doesn't.

I assume your machine is quite unique here. IIRC, you configure 2 CPUs
groups in your ACPI pptt which then form a 2 CPUs cluster_cpumask and
since your core_mask (in cpu_coregrop_mask()) has only 1 CPU, it gets
set to the cluster_cpumask so at the end you have a 2 CPU MC SD and no
CLS SD plus an 80 CPU PKG SD.

This CLS->MC propagation is somehow important since only then you get a
valid 'sd = rcu_dereference(per_cpu(sd_llc, target))' in sis() so you
not just return target (prev or this CPU).
But I can imagine that your MC cpumask is way too small for the SIS_UTIL
based selection of an idle CPU.

[...]

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Vincent Guittot]

On Thu, 13 Nov 2025 at 01:26, Shubhang Kaushik OS
<Shubhang@os.amperecomputing.com> wrote:
>
> > From your previous answer on v1, I don't think that you use
> > heterogeneous system so eas will not be enabled in your case and even
> > when used find_energy_efficient_cpu() will be called before
>
> I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.
>
> > Otherwise you might want to check in wake_affine() where we decide
> > between local cpu and previous cpu which one should be the target.
> > This can have an impact especially if there are not in the same LLC
>
> While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.
> We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.

So I'm confused about your topology; Could you share your scheduling topology ?

Also I'm not sure what problem you are trying to solve.
select_idle_sibling() is all about finding an idle CPU that shares
cache with target with target being either local cpu or prev cpu.

If target is prev cpu, we select an idle CPU that shares cache with
prev and the cache locality should be preserved (at least at L3 level
or even cluer level if you have one)

If target is local cpu and it shares cache with prev, the result
should be similar as above at L3 level but maybe not at cluster level

If target is local cpu and it doesn't share cache with prev then
select_idle_sibling() is not the right place and you should look at
wake affine to favor prev cpu is some cases to be defined

Thanks,
Vincent

>
> Thanks,
> Shubhang Kaushik
>
> ________________________________________
> From: Vincent Guittot <vincent.guittot@linaro.org>
> Sent: Monday, November 3, 2025 1:04 AM
> To: Shubhang Kaushik OS
> Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup
>
> On Thu, 30 Oct 2025 at 20:19, Shubhang Kaushik via B4 Relay
> <devnull+shubhang.os.amperecomputing.com@kernel.org> wrote:
> >
> > From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> >
> > When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> > sibling CPU might migrate away from its previous CPU even if that CPU
> > is not overutilized. This sacrifices cache locality and introduces
> > unnecessary migration overhead.
> >
> > This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> > EAS is active and the task's previous CPU (`prev`) is not overutilized,
> > the scheduler will prioritize waking the task on `prev`, avoiding an
> > unneeded migration and preserving cache-hotness.
> >
> > ---
> > v2:
> > - Addressed reviewer comments to handle this special condition
> >   within the selection logic, prioritizing the
> >   previous CPU if not overutilized for EAS.
> > - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
> >
> > Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> > ---
> >  kernel/sched/fair.c | 12 +++++++++---
> >  1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> >             asym_fits_cpu(task_util, util_min, util_max, target))
> >                 return target;
> >
> > -       /*
> > -        * If the previous CPU is cache affine and idle, don't be stupid:
> > -        */
> > +       /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
> >         if (prev != target && cpus_share_cache(prev, target) &&
> >             (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
> >             asym_fits_cpu(task_util, util_min, util_max, prev)) {
> > @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> >                 prev_aff = prev;
> >         }
> >
> > +       /*
> > +        * If the previous CPU is not overutilized, prefer it for cache locality.
> > +        * This prevents migration away from a cache-hot CPU that can still
> > +        * handle the task without causing an overload.
> > +        */
> > +       if (sched_energy_enabled() && !cpu_overutilized(prev))
>
> From your previous answer on v1, I don't think that you use
> heterogeneous system so eas will not be enabled in your case and even
> when used find_energy_efficient_cpu() will be called before
>
> select_idle_sibling looks for an idle cpu that shares the cache with
> target, Isn't such migration inside the same LLC good in your case ?
>
> Otherwise you might want to check in wake_affine() where we decide
> between local cpu and previous cpu which one should be the target.
> This can have an impact especially if there are not in the same LLC
>
> > +               return prev;
> > +
> >         /*
> >          * Allow a per-cpu kthread to stack with the wakee if the
> >          * kworker thread and the tasks previous CPUs are the same.
> >
> > ---
> > base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> > change-id: 20251030-b4-follow-up-ff03b4533a2d
> >
> > Best regards,
> > --
> > Shubhang Kaushik <shubhang@os.amperecomputing.com>
> >
> >

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik OS]

Our current kernel has CONFIG_SCHED_CLUSTER enabled. While it shows 2 NUMA nodes, only node0 is the one containing 0-79 cores.

NUMA:
  NUMA node(s):              2
  NUMA node0 CPU(s):         0-79
  NUMA node1 CPU(s):

I run similar perf testcases along with MySQL and AI workloads.

> IMHO, the scheduler only cares about shared LLC (and shared L2 with
> CONFIG_SCHED_CLUSTER). Can you check:

$ cat /sys/devices/system/cpu/cpu0/cache/index*/{type,shared_cpu_map}
Data
Instruction
Unified
0000,00000000,00000001
0000,00000000,00000001
0000,00000000,00000001

The output confirms that the extra Unified cache entry (1) does not exist in our sysfs view.
Correct, this Altra machine only a 2-CPU MC SD, which results in the small MC cpumask.

[
    "cpu78",
    {
        "MC": "['78-79']",
        "PKG": "['0-79']"
    }
][
    "cpu79",
    {
        "MC": "['78-79']",
        "PKG": "['0-79']"
    }
]

Thanks,
Shubhang Kaushik

________________________________________
From: Dietmar Eggemann <dietmar.eggemann@arm.com>
Sent: Thursday, November 13, 2025 6:54 AM
To: Shubhang Kaushik OS; Vincent Guittot
Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

On 13.11.25 01:26, Shubhang Kaushik OS wrote:
>> From your previous answer on v1, I don't think that you use
>> heterogeneous system so eas will not be enabled in your case and even
>> when used find_energy_efficient_cpu() will be called before
>
> I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.
>
>> Otherwise you might want to check in wake_affine() where we decide
>> between local cpu and previous cpu which one should be the target.
>> This can have an impact especially if there are not in the same LLC
>
> While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.

Which testcases are you running on your Altra box? I assume it's a
single NUMA node (80 CPUs).

For us, 'perf bench sched messaging` w/o CONFIG_SCHED_CLUSTER, so only
PKG SD (i.e. sis() only returns prev or this CPU) gives better results
then w/ CONFIG_SCHED_CLUSTER.

> We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.

IMHO, the scheduler only cares about shared LLC (and shared L2 with
CONFIG_SCHED_CLUSTER). Can you check:

$ cat /sys/devices/system/cpu/cpu0/cache/index*/{type,shared_cpu_map}
Data
Instruction
Unified
Unified                                                 <-- (1)
00000000,00000000,00000000,00000000,00000001
00000000,00000000,00000000,00000000,00000001
00000000,00000000,00000000,00000000,00000001
CPU mask > 00000000,00000000,00000000,00000000,00000001 <-- (1)

Does (1) exists? IMHO it doesn't.

I assume your machine is quite unique here. IIRC, you configure 2 CPUs
groups in your ACPI pptt which then form a 2 CPUs cluster_cpumask and
since your core_mask (in cpu_coregrop_mask()) has only 1 CPU, it gets
set to the cluster_cpumask so at the end you have a 2 CPU MC SD and no
CLS SD plus an 80 CPU PKG SD.

This CLS->MC propagation is somehow important since only then you get a
valid 'sd = rcu_dereference(per_cpu(sd_llc, target))' in sis() so you
not just return target (prev or this CPU).
But I can imagine that your MC cpumask is way too small for the SIS_UTIL
based selection of an idle CPU.

[...]

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik OS]

Hi Vincent,

Makes sense, thanks for the clarification. The system I'm using is an Ampere Altra which consists of 2-CPU MC domains (e.g., MC: [78-79]), all within a single PKG domain (PKG: [0-79]). The shared cache is confined to the 2-CPU MC domain and concentrating too much work on that pair can itself become a bottleneck, best not to introduce extra stacking bias within the MC domain.

I agree that `select_idle_sibling()` is not the appropriate place to address the behavior I am investigating. Rather, it might need some refinement in `wake_affine()` while choosing between `this_cpu` and `prev_cpu` across different MC domains.

Thanks,
Shubhang

________________________________________
From: Vincent Guittot <vincent.guittot@linaro.org>
Sent: Friday, November 14, 2025 5:36 AM
To: Shubhang Kaushik OS
Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

On Thu, 13 Nov 2025 at 01:26, Shubhang Kaushik OS
<Shubhang@os.amperecomputing.com> wrote:
>
> > From your previous answer on v1, I don't think that you use
> > heterogeneous system so eas will not be enabled in your case and even
> > when used find_energy_efficient_cpu() will be called before
>
> I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.
>
> > Otherwise you might want to check in wake_affine() where we decide
> > between local cpu and previous cpu which one should be the target.
> > This can have an impact especially if there are not in the same LLC
>
> While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.
> We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.

So I'm confused about your topology; Could you share your scheduling topology ?

Also I'm not sure what problem you are trying to solve.
select_idle_sibling() is all about finding an idle CPU that shares
cache with target with target being either local cpu or prev cpu.

If target is prev cpu, we select an idle CPU that shares cache with
prev and the cache locality should be preserved (at least at L3 level
or even cluer level if you have one)

If target is local cpu and it shares cache with prev, the result
should be similar as above at L3 level but maybe not at cluster level

If target is local cpu and it doesn't share cache with prev then
select_idle_sibling() is not the right place and you should look at
wake affine to favor prev cpu is some cases to be defined

Thanks,
Vincent

>
> Thanks,
> Shubhang Kaushik
>
> ________________________________________
> From: Vincent Guittot <vincent.guittot@linaro.org>
> Sent: Monday, November 3, 2025 1:04 AM
> To: Shubhang Kaushik OS
> Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup
>
> On Thu, 30 Oct 2025 at 20:19, Shubhang Kaushik via B4 Relay
> <devnull+shubhang.os.amperecomputing.com@kernel.org> wrote:
> >
> > From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> >
> > When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> > sibling CPU might migrate away from its previous CPU even if that CPU
> > is not overutilized. This sacrifices cache locality and introduces
> > unnecessary migration overhead.
> >
> > This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> > EAS is active and the task's previous CPU (`prev`) is not overutilized,
> > the scheduler will prioritize waking the task on `prev`, avoiding an
> > unneeded migration and preserving cache-hotness.
> >
> > ---
> > v2:
> > - Addressed reviewer comments to handle this special condition
> >   within the selection logic, prioritizing the
> >   previous CPU if not overutilized for EAS.
> > - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
> >
> > Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> > ---
> >  kernel/sched/fair.c | 12 +++++++++---
> >  1 file changed, 9 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> >             asym_fits_cpu(task_util, util_min, util_max, target))
> >                 return target;
> >
> > -       /*
> > -        * If the previous CPU is cache affine and idle, don't be stupid:
> > -        */
> > +       /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
> >         if (prev != target && cpus_share_cache(prev, target) &&
> >             (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
> >             asym_fits_cpu(task_util, util_min, util_max, prev)) {
> > @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> >                 prev_aff = prev;
> >         }
> >
> > +       /*
> > +        * If the previous CPU is not overutilized, prefer it for cache locality.
> > +        * This prevents migration away from a cache-hot CPU that can still
> > +        * handle the task without causing an overload.
> > +        */
> > +       if (sched_energy_enabled() && !cpu_overutilized(prev))
>
> From your previous answer on v1, I don't think that you use
> heterogeneous system so eas will not be enabled in your case and even
> when used find_energy_efficient_cpu() will be called before
>
> select_idle_sibling looks for an idle cpu that shares the cache with
> target, Isn't such migration inside the same LLC good in your case ?
>
> Otherwise you might want to check in wake_affine() where we decide
> between local cpu and previous cpu which one should be the target.
> This can have an impact especially if there are not in the same LLC
>
> > +               return prev;
> > +
> >         /*
> >          * Allow a per-cpu kthread to stack with the wakee if the
> >          * kworker thread and the tasks previous CPUs are the same.
> >
> > ---
> > base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> > change-id: 20251030-b4-follow-up-ff03b4533a2d
> >
> > Best regards,
> > --
> > Shubhang Kaushik <shubhang@os.amperecomputing.com>
> >
> >

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Vincent Guittot]

On Tue, 18 Nov 2025 at 02:27, Shubhang Kaushik OS
<Shubhang@os.amperecomputing.com> wrote:
>
> Hi Vincent,
>
> Makes sense, thanks for the clarification. The system I'm using is an Ampere Altra which consists of 2-CPU MC domains (e.g., MC: [78-79]), all within a single PKG domain (PKG: [0-79]). The shared cache is confined to the 2-CPU MC domain and concentrating too much work on that pair can itself become a bottleneck, best not to introduce extra stacking bias within the MC domain.

Ok, I thought that you would have a system cache as the last level of
cache for all CPUs which would mean:
CLUSTER:[78-79]
MC:[0-79]

IIUC: With a last level of cache shared between 2 CPUs, you want to
favor prev cpu over local cpu at wakeup because local cpu is most
probably not in the same cache domain as prev and you will not take
advantage of cache hotness

>
> I agree that `select_idle_sibling()` is not the appropriate place to address the behavior I am investigating. Rather, it might need some refinement in `wake_affine()` while choosing between `this_cpu` and `prev_cpu` across different MC domains.
>
> Thanks,
> Shubhang
>
> ________________________________________
> From: Vincent Guittot <vincent.guittot@linaro.org>
> Sent: Friday, November 14, 2025 5:36 AM
> To: Shubhang Kaushik OS
> Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup
>
> On Thu, 13 Nov 2025 at 01:26, Shubhang Kaushik OS
> <Shubhang@os.amperecomputing.com> wrote:
> >
> > > From your previous answer on v1, I don't think that you use
> > > heterogeneous system so eas will not be enabled in your case and even
> > > when used find_energy_efficient_cpu() will be called before
> >
> > I agree that the EAS centric approach in the current patch is misplaced for our homogeneous systems.
> >
> > > Otherwise you might want to check in wake_affine() where we decide
> > > between local cpu and previous cpu which one should be the target.
> > > This can have an impact especially if there are not in the same LLC
> >
> > While wake_affine() modifications seem logical, I see that they cause performance regressions across the board due to the inherent trade-offs in altering that critical initial decision point.
> > We might need to solve the non-idle fallback within `select_idle_sibling` to ring fence the impact for preserving locality effectively.
>
> So I'm confused about your topology; Could you share your scheduling topology ?
>
> Also I'm not sure what problem you are trying to solve.
> select_idle_sibling() is all about finding an idle CPU that shares
> cache with target with target being either local cpu or prev cpu.
>
> If target is prev cpu, we select an idle CPU that shares cache with
> prev and the cache locality should be preserved (at least at L3 level
> or even cluer level if you have one)
>
> If target is local cpu and it shares cache with prev, the result
> should be similar as above at L3 level but maybe not at cluster level
>
> If target is local cpu and it doesn't share cache with prev then
> select_idle_sibling() is not the right place and you should look at
> wake affine to favor prev cpu is some cases to be defined
>
> Thanks,
> Vincent
>
> >
> > Thanks,
> > Shubhang Kaushik
> >
> > ________________________________________
> > From: Vincent Guittot <vincent.guittot@linaro.org>
> > Sent: Monday, November 3, 2025 1:04 AM
> > To: Shubhang Kaushik OS
> > Cc: Ingo Molnar; Peter Zijlstra; Juri Lelli; Dietmar Eggemann; Steven Rostedt; Ben Segall; Mel Gorman; Valentin Schneider; Shubhang Kaushik; Shijie Huang; Frank Wang; Christopher Lameter; Adam Li; linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup
> >
> > On Thu, 30 Oct 2025 at 20:19, Shubhang Kaushik via B4 Relay
> > <devnull+shubhang.os.amperecomputing.com@kernel.org> wrote:
> > >
> > > From: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> > >
> > > When Energy Aware Scheduling (EAS) is enabled, a task waking up on a
> > > sibling CPU might migrate away from its previous CPU even if that CPU
> > > is not overutilized. This sacrifices cache locality and introduces
> > > unnecessary migration overhead.
> > >
> > > This patch refines the wakeup heuristic in `select_idle_sibling()`. If
> > > EAS is active and the task's previous CPU (`prev`) is not overutilized,
> > > the scheduler will prioritize waking the task on `prev`, avoiding an
> > > unneeded migration and preserving cache-hotness.
> > >
> > > ---
> > > v2:
> > > - Addressed reviewer comments to handle this special condition
> > >   within the selection logic, prioritizing the
> > >   previous CPU if not overutilized for EAS.
> > > - Link to v1: https://lore.kernel.org/all/20251017-b4-sched-cfs-refactor-propagate-v1-1-1eb0dc5b19b3@os.amperecomputing.com/
> > >
> > > Signed-off-by: Shubhang Kaushik <shubhang@os.amperecomputing.com>
> > > ---
> > >  kernel/sched/fair.c | 12 +++++++++---
> > >  1 file changed, 9 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index 25970dbbb27959bc130d288d5f80677f75f8db8b..ac94463627778f09522fb5420f67b903a694ad4d 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -7847,9 +7847,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> > >             asym_fits_cpu(task_util, util_min, util_max, target))
> > >                 return target;
> > >
> > > -       /*
> > > -        * If the previous CPU is cache affine and idle, don't be stupid:
> > > -        */
> > > +       /* Reschedule on an idle, cache-sharing sibling to preserve affinity: */
> > >         if (prev != target && cpus_share_cache(prev, target) &&
> > >             (available_idle_cpu(prev) || sched_idle_cpu(prev)) &&
> > >             asym_fits_cpu(task_util, util_min, util_max, prev)) {
> > > @@ -7861,6 +7859,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
> > >                 prev_aff = prev;
> > >         }
> > >
> > > +       /*
> > > +        * If the previous CPU is not overutilized, prefer it for cache locality.
> > > +        * This prevents migration away from a cache-hot CPU that can still
> > > +        * handle the task without causing an overload.
> > > +        */
> > > +       if (sched_energy_enabled() && !cpu_overutilized(prev))
> >
> > From your previous answer on v1, I don't think that you use
> > heterogeneous system so eas will not be enabled in your case and even
> > when used find_energy_efficient_cpu() will be called before
> >
> > select_idle_sibling looks for an idle cpu that shares the cache with
> > target, Isn't such migration inside the same LLC good in your case ?
> >
> > Otherwise you might want to check in wake_affine() where we decide
> > between local cpu and previous cpu which one should be the target.
> > This can have an impact especially if there are not in the same LLC
> >
> > > +               return prev;
> > > +
> > >         /*
> > >          * Allow a per-cpu kthread to stack with the wakee if the
> > >          * kworker thread and the tasks previous CPUs are the same.
> > >
> > > ---
> > > base-commit: e53642b87a4f4b03a8d7e5f8507fc3cd0c595ea6
> > > change-id: 20251030-b4-follow-up-ff03b4533a2d
> > >
> > > Best regards,
> > > --
> > > Shubhang Kaushik <shubhang@os.amperecomputing.com>
> > >
> > >

Re: [PATCH v2] sched/fair: Prefer cache locality for EAS wakeup

[Shubhang Kaushik Prasanna Kumar]

Hi Vineeth,

	> Is cache locality prioritized even over idle CPU availability?

I had assumed that the fast path prioritizes cache locality over idle CPU selection. However, our performance measurements indicate that, in this specific scenario, the trade-off is not worthwhile.
The data shows that prioritizing the busy, `cache-hot` core leads to performance regressions and an overload on wakeup. Which  means an overhead of staying on the busy core outweighs the cost of migrating to an idle core.
In development, I am trying an alternative approach to benefit from cache locality in this path.

Regards,
Shubhang Kaushik