Re: [RFC PATCH 5/5] sched/fair: Proactive idle balance using push mechanism

[K Prateek Nayak <kprateek.nayak@amd.com>]

On 4/10/2025 3:59 PM, Peter Zijlstra wrote:

[..snip..]

>>   /*
>>    * See if the non running fair tasks on this rq can be sent on other CPUs
>>    * that fits better with their profile.
>>    */
>>   static bool push_fair_task(struct rq *rq)
>>   {
>> +	struct cpumask *cpus = this_cpu_cpumask_var_ptr(load_balance_mask);
>> +	struct task_struct *p = pick_next_pushable_fair_task(rq);
>> +	int cpu, this_cpu = cpu_of(rq);
>> +
>> +	if (!p)
>> +		return false;
>> +
>> +	if (!cpumask_and(cpus, nohz.idle_cpus_mask, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)))
>> +		goto requeue;
> 
> So I think the main goal here should be to get rid of the whole single
> nohz balancing thing.
> 
> This global state/mask has been shown to be a problem over and over again.
> 
> Ideally we keep a nohz idle mask per LLC (right next to the overload
> mask you introduced earlier), along with a bit in the sched_domain tree
> upwards of that to indicate a particular llc/ node / distance-group has
> nohz idle.
> 
> Then if the topmost domain has the bit set it means there are nohz cpus
> to be found, and we can (slowly) iterate the domain tree up from
> overloaded LLC to push tasks around.

I'll to through fair.c to understand all the usecases of
"nohz.idle_cpus_mask" and then start with this bit for v2 to see if that
blows up in some way. I'll be back shortly.

> 
> Anyway, yes, you gotta start somewhere :-)

Thanks a ton for the initial review. I'll go analyze more to see what
bits are making benchmarks go sad.

-- 
Thanks and Regards,
Prateek