Re: [REGRESSION] [PATCH v3 7/7] sched/eevdf: Move to a single runqueue

[K Prateek Nayak <kprateek.nayak@amd.com>]

On 9/23/2026 10:18 PM, Chen Yu wrote:
> On Wed, Sep 23, 2026 at 07:51:35PM +0530, Aishwarya Rambhadran wrote:
>> Hi Peter,
>>
>> We have observed some schbench p99 request latency regressions while
>> comparing v7.2 against v7.3 cycle on an AWS Graviton3 (m7g.metal)
>> using Fastpath (Linux kernel performance benchmarking tool).
>>
>> The schbench suite as a whole does not show a significant regression,
>> and the results appear to be a mix of improvements and regressions
>> depending on the thread configuration. However, a few configurations
>> show fairly large p99 request latency regressions. For example,
>> comparing v7.2 against v7.3-rc4:
>>
>> message-thread:16, worker-thread:16
>> v7.2 = 662869 us
>> v7.3-rc4 = 969557 us
>> change% = -31.63%
>>
>> message-thread:64, worker-thread:4
>> v7.2 = 730453 us
>> v7.3-rc4 = 942251 us
>> change% = -22.48%
>>
>> message-thread:32, worker-thread:4
>> v7.2 = 74251 us
>> v7.3-rc4 = 56725 us
>> change% = +30.90%
>>
>>
>> So this does not appear to be a general latency degradation; some
>> configs improve while others regress.
>>
>> I also ran a Fastpath automated git bisection for the m 16, t 16 p99
>> request latency regression between v7.2 and v7.3-rc1, which identified
>> this commit as the first bad commit. The original regression was ~30.9%,
>> & the bisection reproducibility check showed ~27.8% regression with 20
>> repeats across 2 boot sessions.
>>
>> This commit looks relevant since it changes EEVDF runqueue organization
>> and is intended, in part, to address latency issues with hierarchical fair/
>> cgroup scheduling. I also noticed that there have been fixes on top of this
>> change, including 68e37487810a ("sched/fair: Fix flat hierarchy"), which
>> fixes vruntime handling and has a Fixes: 85570f10a4c6 tag.
>>
>> Given that the overall schbench result is mostly neutral while some p99
>> configurations move considerably in both directions, I wanted to check
>> whether this could be an expected trade-off from the single-runqueue
>> change, or whether the p99 regressions are worth investigating further.
>>
>> I can run additional tests or provide the full results/bisection data if
>> useful.
>>
>>
> 
> Would disabling WA_WEIGHT help? I happened to find that this workaround works
> for me to restore netperf performance (though it is a throughput score rather
> than a latency one), it seems that task_h_load() becomes much
> larger(also on sched/task_h_load):
> echo NO_WA_WEIGHT > /sys/kernel/debug/sched/features
> https://lore.kernel.org/lkml/aoxah90s0bQ4tcUW@three-body/ 

In addition to Chenyu's suggestion, I would also suggest trying out
different cgroup modes via "/sys/kernel/debug/sched/cgroup_mode" if
NO_WA_WEIGHT doesn't resolve your issue.
(P.S. Peter has a branch with some fixes for the task_h_load() at
https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=sched/task_h_load
in case you can try a custom kernel build on your setup)

In my testing, I did see some regression in schbench with the default
"concur" mode for RPS metric but the "smp" mode which used the older
formula for weight calculation seemed to be running on par with the
hierarchical pick.
https://lore.kernel.org/lkml/65980119-a26a-40d6-bec8-dd597833a6e8@amd.com/

-- 
Thanks and Regards,
Prateek