Re: [RFC][PATCH 2/5] sched: Optimize ttwu() / select_task_rq()

[Peter Zijlstra <peterz@infradead.org>]

On Mon, Jun 09, 2025 at 07:01:47AM +0200, Mike Galbraith wrote:

Right; so the problem being that we can race with
migrate_disable_switch().

>  kernel/sched/core.c |    8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4313,7 +4313,10 @@ int try_to_wake_up(struct task_struct *p
>  		    ttwu_queue_wakelist(p, task_cpu(p), wake_flags))
>  			break;
>  
> -		cpu = select_task_rq(p, p->wake_cpu, &wake_flags);
> +		if (is_migration_disabled(p))
> +			cpu = -1;
> +		else
> +			cpu = select_task_rq(p, p->wake_cpu, &wake_flags);
>  
>  		/*
>  		 * If the owning (remote) CPU is still in the middle of schedule() with
> @@ -4326,6 +4329,9 @@ int try_to_wake_up(struct task_struct *p
>  		 */
>  		smp_cond_load_acquire(&p->on_cpu, !VAL);
>  
> +		if (cpu == -1)
> +			cpu = select_task_rq(p, p->wake_cpu, &wake_flags);
> +
>  		if (task_cpu(p) != cpu) {
>  			if (p->in_iowait) {
>  				delayacct_blkio_end(p);
> 

So select_task_rq() already checks is_migration_disabled(); just not
well enough. Also, I'm thinking that if we see migration_disabled, we
don't need to call it a second time, just let it be where it was.

Does something like this help? Specifically, when nr_cpus_allowed == 1
|| is_migration_disabled(), don't change @cpu at all.

--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3593,7 +3593,7 @@ int select_task_rq(struct task_struct *p
 		cpu = p->sched_class->select_task_rq(p, cpu, *wake_flags);
 		*wake_flags |= WF_RQ_SELECTED;
 	} else {
-		cpu = cpumask_any(p->cpus_ptr);
+		cpu = task_cpu(p);
 	}
 
 	/*