Re: [RFC PATCH] sched/fair: Remove sched_idle_cpu() usages in select_task_rq_fair()

[Chengming Zhou <chengming.zhou@linux.dev>]

+cc Josh and Viresh, I forgot to cc you, sorry!

On 2025/8/20 21:53, Christian Loehle wrote:
> On 8/19/25 16:32, Chen, Yu C wrote:
>> On 8/18/2025 9:24 PM, Christian Loehle wrote:
>>> On 8/18/25 13:47, Chengming Zhou wrote:
>>>> These sched_idle_cpu() considerations in select_task_rq_fair() is based
>>>> on an assumption that the wakee task can pick a cpu running sched_idle
>>>> task and preempt it to run, faster than picking an idle cpu to preempt
>>>> the idle task.
>>>>
>>>> This assumption is correct, but it also brings some problems:
>>>>
>>>> 1. work conservation: Often sched_idle tasks are also picking the cpu
>>>> which is already running sched_idle task, instead of utilizing a real
>>>> idle cpu, so work conservation is somewhat broken.
>>>>
>>>> 2. sched_idle group: This sched_idle_cpu() is just not correct with
>>>> sched_idle group running. Look a simple example below.
>>>>
>>>>          root
>>>>      /        \
>>>>      kubepods    system
>>>>      /    \
>>>> burstable    besteffort
>>>>          (cpu.idle == 1)
>>>>
>>>> When a sched_idle cpu is just running tasks from besteffort group,
>>>> sched_idle_cpu() will return true in this case, but this cpu pick
>>>> is bad for wakee task from system group. Because the system group
>>>> has lower weight than kubepods, work conservation is somewhat
>>>> broken too.
>>>>
>>>> In a nutshell, sched_idle_cpu() should consider the wakee task group's
>>>> relationship with sched_idle tasks running on the cpu.
>>>>
>>>> Obviously, it's hard to do so. This patch chooses the simple approach
>>>> to remove all sched_idle_cpu() considerations in select_task_rq_fair()
>>>> to bring back work conservation in these cases.
>>>
>>> OTOH sched_idle_cpu() CPUs are guaranteed to not be in an idle state and
>>> potentially already have DVFS on some higher level...
>>>
>> Is it because the schedutil governor considers the utilization
>> of SCHED_IDLE, thus causing schedutil to request a higher
>> frequency?
> 
> For intel_pstate active (HWP and !HWP) the same issue should persist, no?
> 
>>
>> The commit 3c29e651e16d ("sched/fair: Fall back to sched-idle
>> CPU if an idle CPU isn't found") mentions that choosing a CPU
>> running a SCHED_IDLE task can avoid waking a CPU from a deep
>> sleep state.
>>
>> If this is the case, can we say that if an administrator sets
>> the cpufreq governor to "performance" and disables deep idle
>> states, an idle CPU would be more preferable than a CPU running
>> a SCHED_IDLE task? On the other hand, if
>> per_cpu(cpufreq_update_util_data, cpu) is NULL and only shallow
>> idle states are enabled in idle_get_state(), should we skip
>> SCHED_IDLE to achieve work conservation?
> 
> That's probably getting the most out of it.
> That being said, strictly speaking the SCHED_IDLE CPU and the
> SHALLOW_IDLE CPU may still share a power and thermal budget, which
> may make preempting the sched-idle task on SCHED_IDLE CPU the
> better choice.

I admit these sched_idle_cpu() considerations are good motivation,
at least for !sched_idle tasks, to preempt sched_idle task instead of
waking a real idle cpu.

But it oversimplified the complicated situations we have, such as
the cases above, which make the scheduler's work conservation performance
really bad.

Thanks.