Re: [PATCHv2] sched/fair: Skip SCHED_IDLE rq for SCHED_IDLE task

[Christian Loehle <christian.loehle@arm.com>]

On 2/4/26 07:48, K Prateek Nayak wrote:
> Hello Chris,
> 
> On 2/4/2026 12:19 AM, Christian Loehle wrote:
>> CPUs whose rq only have SCHED_IDLE tasks running are considered to be
>> equivalent to truly idle CPUs during wakeup path. For fork and exec
>> SCHED_IDLE is even preferred.
>> This is based on the assumption that the SCHED_IDLE CPU is not in an
>> idle state and might be in a higher P-state, allowing the task/wakee
>> to run immediately without sharing the rq.
>>
>> However this assumption doesn't hold if the wakee has SCHED_IDLE policy
>> itself, as it will share the rq with existing SCHED_IDLE tasks. In this
>> case, we are better off continuing to look for a truly idle CPU.
>>
>> On a Intel Xeon 2-socket with 64 logical cores in total this yields
>> for kernel compilation using SCHED_IDLE:
>>
>> +---------+----------------------+----------------------+--------+
>> | workers | mainline (seconds)   | patch (seconds)      | delta% |
>> +=========+======================+======================+========+
>> |       1 | 4384.728 ± 21.085    | 3843.250 ± 16.235    | -12.35 |
>> |       2 | 2242.513 ± 2.099     | 1971.696 ± 2.842     | -12.08 |
>> |       4 | 1199.324 ± 1.823     | 1033.744 ± 1.803     | -13.81 |
>> |       8 |  649.083 ± 1.959     |  559.123 ± 4.301     | -13.86 |
>> |      16 |  370.425 ± 0.915     |  325.906 ± 4.623     | -12.02 |
>> |      32 |  234.651 ± 2.255     |  217.266 ± 0.253     |  -7.41 |
>> |      64 |  202.286 ± 1.452     |  197.977 ± 2.275     |  -2.13 |
>> |     128 |  217.092 ± 1.687     |  212.164 ± 1.138     |  -2.27 |
>> +---------+----------------------+----------------------+--------+
> 
> I couldn't spot much difference for kernel compilation on my
> 3rd Generation EPYC system likely due to smaller LLC size. For
> sched-messaging, I found the following interesting trend when
> running with SCHED_IDLE:
> 
>   (Normalized runtime [Var%]; %diff - higher the better)
> 
>                 tip/sched:core           +patch        (%diff)
>     
>      1-group      1.00 [5.00%]        0.88 [10.78%]    11.80%
>      2-group      1.00 [5.15%]        0.93 [26.06%]     6.99%
>      4-group      1.00 [5.48%]        0.89 [11.03%]    11.13%
>      8-group      1.00 [6.62%]        1.21 [12.37%]   -21.30%
>     16-group      1.00 [9.46%]        1.28 [ 9.42%]   -27.59%
> 
> 
> There is a good improvement for lower utilization. Once the
> system is trending towards overutilized but SIS_UTIL cut-off
> is still non-zero, we search a little bit longer for a fully
> idle CPU when the probability for finding one is actually
> low.
> 
> I suppose that scenario is rare where we only have SCHED_IDLE
> tasks that care about throughput on a busy system to
> actually notice this but it was worth pointing out.

If we're unlikely to find a good candidate then doing anything
on wakeup is kind of a waste of time, especially for sched
messaging.
So I guess without $PATCH it will basically always bail out
when looking at the first few CPUs because it sees SCHED_IDLE
sched messaging :)

> 
> Feel free to include:
> 
> Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>

Thanks for testing!