Re: [RFC PATCH 1/5] sched/fair: Add push task framework

[K Prateek Nayak <kprateek.nayak@amd.com>]

Hello Peter,

Thank you for the review.

On 4/10/2025 3:30 PM, Peter Zijlstra wrote:
> On Wed, Apr 09, 2025 at 11:15:35AM +0000, K Prateek Nayak wrote:
> 
>> +static void fair_add_pushable_task(struct rq *rq, struct task_struct *p)
>> +{
>> +	if (fair_push_task(p)) {
>> +		plist_del(&p->pushable_tasks, &rq->cfs.pushable_tasks);
>> +		plist_node_init(&p->pushable_tasks, p->prio);
> 
> I gotta aks, why do we care about ordering the push list on p->prio for
> fair?

This bit was carried over from Vincent's series. I assume it was
inspired by the RT, DL's push mechanism. Perhaps fair can get away with
MRU ordering similar to rq->cfs_tasks?

> 
>> +		plist_add(&p->pushable_tasks, &rq->cfs.pushable_tasks);
>> +	}
>> +}
>> +
>>   /*
>>    * select_task_rq_fair: Select target runqueue for the waking task in domains
>>    * that have the relevant SD flag set. In practice, this is SD_BALANCE_WAKE,
> 
>> @@ -8914,6 +8978,12 @@ pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf
>>   		put_prev_entity(cfs_rq, pse);
>>   		set_next_entity(cfs_rq, se);
>>   
>> +		/*
>> +		 * The previous task might be eligible for being pushed on
>> +		 * another cpu if it is still active.
>> +		 */
>> +		fair_add_pushable_task(rq, prev);
>> +
>>   		__set_next_task_fair(rq, p, true);

So __set_next_task_fair() removes the current task from the pushable list
when it is picked ...

>>   	}
>>   
>> @@ -8986,6 +9056,13 @@ static void put_prev_task_fair(struct rq *rq, struct task_struct *prev, struct t
>>   		cfs_rq = cfs_rq_of(se);
>>   		put_prev_entity(cfs_rq, se);
>>   	}
>> +
>> +	/*
>> +	 * The previous task might be eligible for being pushed on another cpu
>> +	 * if it is still active.
>> +	 */
>> +	fair_add_pushable_task(rq, prev);
>> +
>>   }
> 
> These two are tricky; while they will work with a push balance callback,
> they will cause some pain with pulling from the push lists; a-la
> newidle.
> 
> Notably, we must be sure to check ->on_cpu.

... so we should be safe here. Other considerations are covered by
detach_task() but pushable task lists to only contain queued tasks
that can run on more than one CPU and Vincent was cautious enough to
place a bunch of WARN_ON_ONCE() in pick_next_pushable_fair_task() to
catch those cases :)

> 
> Perhaps later patches add this, let me continue reading...

-- 
Thanks and Regards,
Prateek