From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Ryan Roberts <ryan.roberts@arm.com>,
Mel Gorman <mgorman@techsingularity.net>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org,
Aishwarya TCV <Aishwarya.TCV@arm.com>
Subject: Re: [REGRESSION] sched/fair: Reimplement NEXT_BUDDY to align with EEVDF goals
Date: Wed, 7 Jan 2026 16:30:09 +0100 [thread overview]
Message-ID: <051285c7-56c0-455d-ab3e-9959edb5d13f@arm.com> (raw)
In-Reply-To: <c111cc52-4133-4f57-b753-139a5ff2b395@arm.com>
On 05.01.26 12:45, Ryan Roberts wrote:
> On 02/01/2026 15:52, Dietmar Eggemann wrote:
>> On 02.01.26 13:38, Ryan Roberts wrote:
[...]
>>>> All testing is done on AWS Graviton3 (arm64) bare metal systems. (R)/(I) mean
>>>> statistically significant regression/improvement, where "statistically
>>>> significant" means the 95% confidence intervals do not overlap".
>>
>> You mentioned that you reverted this patch 'patch 2/2 'sched/fair:
>> Reimplement NEXT_BUDDY to align with EEVDF goals'.
>>
>> Does this mean NEXT_BUDDY is still enabled, i.e. you haven't reverted
>> patch 1/2 'sched/fair: Enable scheduler feature NEXT_BUDDY' as well?
>
> Yes that's correct; patch 1 is still present. I could revert that as well and rerun if useful?
Well, I assume this would be more valuable. Before this patch-set (e.g
v6.18), NEXT_BUDDY was disabled and this is what people are running.
Now (>= v6.19-rc1) we have NEXT_BUDDY=true (1/2) and 'NEXT_BUDDY aligned
to EEVDF' (2/2). This is what people will run when they switch to v6.19
later.
But patch 2/2 changes more than the 'if (sched_feat(NEXT_BUDDY) ...'
condition. So testing 'w/o 2/2' vs. 'w/ 2/2' and 'NEXT_BUDDY=false'
could be helpful as well.
>> ---
>>
>> Mel mentioned that he tested on a 2-socket machine. So I guess something
>> like my Intel Xeon Silver 4314:
>>
>> cpu0 0 0
>> domain0 SMT 00000001,00000001
>> domain1 MC 55555555,55555555
>> domain2 NUMA ffffffff,ffffffff
>>
>> node distances:
>> node 0 1
>> 0: 10 20
>> 1: 20 10
>>
>> Whereas I assume the Graviton3 has 64 CPUs (cores) flat in a single MC
>> domain? I guess topology has influence in benchmark numbers here as well.
>
> I can't easily enable scheduler debugging right now (which I think is needed to
> get this info directly?). But that's what I'd expect, yes. lscpu confirms there
> is a single NUMA node and topology for cpu0 gives this if it helps:
>
> /sys/devices/system/cpu/cpu0/topology$ grep "" -r .
> ./cluster_cpus:ffffffff,ffffffff
> ./cluster_cpus_list:0-63
> ./physical_package_id:0
> ./core_cpus_list:0
> ./core_siblings:ffffffff,ffffffff
> ./cluster_id:0
> ./core_siblings_list:0-63
> ./package_cpus:ffffffff,ffffffff
> ./package_cpus_list:0-63
[...]
OK, so single (flat) MC domain with 64 CPUs.
>> There was also a lot of improvement on schbench (wakeup latency) on
>> higher percentiles (>= 99.0th) on the 2-socket machine with those 2
>> patches. I guess you haven't seen those on Grav3?
>>
>
> I don't have schbench results for 6.18 but I do have them for 6.19-rc1 and for
> revert-next-buddy. The means have moved a bit but there are only a couple of
> cases that we consisder statistically significant (marked (R)egression /
> (I)mprovement):
>
> +----------------------------+------------------------------------------------------+-------------+-------------------+
> | Benchmark | Result Class | 6-19-0-rc1 | revert-next-buddy |
> +============================+======================================================+=============+===================+
> | schbench/thread-contention | -m 16 -t 1 -r 10 -s 1000, avg_rps (req/sec) | 1263.97 | -6.43% |
> | | -m 16 -t 1 -r 10 -s 1000, req_latency_p99 (usec) | 15088.00 | -0.28% |
> | | -m 16 -t 1 -r 10 -s 1000, wakeup_latency_p99 (usec) | 3.00 | 0.00% |
> | | -m 16 -t 4 -r 10 -s 1000, avg_rps (req/sec) | 6433.07 | -10.99% |
> | | -m 16 -t 4 -r 10 -s 1000, req_latency_p99 (usec) | 15088.00 | -0.39% |
> | | -m 16 -t 4 -r 10 -s 1000, wakeup_latency_p99 (usec) | 4.17 | (R) -16.67% |
> | | -m 16 -t 16 -r 10 -s 1000, avg_rps (req/sec) | 1458.33 | -1.57% |
> | | -m 16 -t 16 -r 10 -s 1000, req_latency_p99 (usec) | 813056.00 | 15.46% |
> | | -m 16 -t 16 -r 10 -s 1000, wakeup_latency_p99 (usec) | 14240.00 | -5.97% |
> | | -m 16 -t 64 -r 10 -s 1000, avg_rps (req/sec) | 434.22 | 3.21% |
> | | -m 16 -t 64 -r 10 -s 1000, req_latency_p99 (usec) | 11354112.00 | 2.92% |
> | | -m 16 -t 64 -r 10 -s 1000, wakeup_latency_p99 (usec) | 63168.00 | -2.87% |
> | | -m 32 -t 1 -r 10 -s 1000, avg_rps (req/sec) | 2828.63 | 2.58% |
> | | -m 32 -t 1 -r 10 -s 1000, req_latency_p99 (usec) | 15088.00 | 0.00% |
> | | -m 32 -t 1 -r 10 -s 1000, wakeup_latency_p99 (usec) | 3.00 | 0.00% |
> | | -m 32 -t 4 -r 10 -s 1000, avg_rps (req/sec) | 3182.15 | 5.18% |
> | | -m 32 -t 4 -r 10 -s 1000, req_latency_p99 (usec) | 116266.67 | 8.22% |
> | | -m 32 -t 4 -r 10 -s 1000, wakeup_latency_p99 (usec) | 6186.67 | (R) -5.34% |
> | | -m 32 -t 16 -r 10 -s 1000, avg_rps (req/sec) | 749.20 | 2.91% |
> | | -m 32 -t 16 -r 10 -s 1000, req_latency_p99 (usec) | 3702784.00 | (I) 13.76% |
> | | -m 32 -t 16 -r 10 -s 1000, wakeup_latency_p99 (usec) | 33514.67 | 0.24% |
> | | -m 32 -t 64 -r 10 -s 1000, avg_rps (req/sec) | 392.23 | 3.42% |
> | | -m 32 -t 64 -r 10 -s 1000, req_latency_p99 (usec) | 16695296.00 | (I) 5.82% |
> | | -m 32 -t 64 -r 10 -s 1000, wakeup_latency_p99 (usec) | 120618.67 | -3.22% |
> | | -m 64 -t 1 -r 10 -s 1000, avg_rps (req/sec) | 5951.15 | 5.02% |
> | | -m 64 -t 1 -r 10 -s 1000, req_latency_p99 (usec) | 15157.33 | 0.42% |
> | | -m 64 -t 1 -r 10 -s 1000, wakeup_latency_p99 (usec) | 3.67 | -4.35% |
> | | -m 64 -t 4 -r 10 -s 1000, avg_rps (req/sec) | 1510.23 | -1.38% |
> | | -m 64 -t 4 -r 10 -s 1000, req_latency_p99 (usec) | 802816.00 | 13.73% |
> | | -m 64 -t 4 -r 10 -s 1000, wakeup_latency_p99 (usec) | 14890.67 | -10.44% |
> | | -m 64 -t 16 -r 10 -s 1000, avg_rps (req/sec) | 458.87 | 4.60% |
> | | -m 64 -t 16 -r 10 -s 1000, req_latency_p99 (usec) | 11348650.67 | (I) 2.67% |
> | | -m 64 -t 16 -r 10 -s 1000, wakeup_latency_p99 (usec) | 63445.33 | (R) -5.48% |
> | | -m 64 -t 64 -r 10 -s 1000, avg_rps (req/sec) | 541.33 | 2.65% |
> | | -m 64 -t 64 -r 10 -s 1000, req_latency_p99 (usec) | 36743850.67 | (I) 10.95% |
> | | -m 64 -t 64 -r 10 -s 1000, wakeup_latency_p99 (usec) | 211370.67 | -1.94% |
> +----------------------------+------------------------------------------------------+-------------+-------------------+
>
> I could get the results for 6.18 if useful, but I think what I have probably
> shows enough of the picture: This patch has not impacted schbench much on
> this HW.
I see. IMHO, task scheduler tests are all about putting the right about
of stress onto the system, not too little and not too much.
I guess, these ~ `-m 64 -t 4` tests should be fine for a 64 CPUs system.
Not sure which parameter set Mel was using on his 2 socket machine. And
I still assume he tested w/o (base) against with these 2 patches.
The other test Mel was using is this modified dbench4 (prefers
throughput (less preemption)). Not sure if this is part of the MmTests
suite?
It would be nice to be able to run the same tests on different machines
(with a parameter set adapted to the number of CPUs), so we have only
the arch and the topology as variables). But there is definitely more
variety (e.g. used filesystem, etc) ... so this is not trivial.
[...]
next prev parent reply other threads:[~2026-01-07 15:30 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-12 12:25 [PATCH 0/2 v5] Reintroduce NEXT_BUDDY for EEVDF Mel Gorman
2025-11-12 12:25 ` [PATCH 1/2] sched/fair: Enable scheduler feature NEXT_BUDDY Mel Gorman
2025-11-14 12:19 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2025-11-17 16:23 ` tip-bot2 for Mel Gorman
[not found] ` <20251112122521.1331238-3-mgorman@techsingularity.net>
2025-11-12 14:48 ` [PATCH 2/2] sched/fair: Reimplement NEXT_BUDDY to align with EEVDF goals Peter Zijlstra
2025-11-13 8:26 ` Madadi Vineeth Reddy
2025-11-13 9:04 ` Mel Gorman
2025-11-14 12:13 ` Peter Zijlstra
2025-11-14 12:19 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2025-11-17 16:23 ` tip-bot2 for Mel Gorman
2025-12-22 10:57 ` [REGRESSION] " Ryan Roberts
2026-01-02 12:38 ` Ryan Roberts
2026-01-02 15:52 ` Dietmar Eggemann
2026-01-05 11:45 ` Ryan Roberts
2026-01-05 14:38 ` Shrikanth Hegde
2026-01-05 16:33 ` Ryan Roberts
2026-01-07 15:30 ` Dietmar Eggemann [this message]
2026-01-08 8:50 ` Mel Gorman
2026-01-08 13:15 ` Ryan Roberts
2026-01-09 10:15 ` Ryan Roberts
2026-01-12 7:47 ` Peter Zijlstra
2026-01-12 8:52 ` Ryan Roberts
2026-01-12 9:57 ` Peter Zijlstra
2026-01-12 10:27 ` Ryan Roberts
2026-01-13 6:31 ` K Prateek Nayak
2026-01-15 10:16 ` Mel Gorman
2026-01-08 10:01 ` [REGRESSION] [PATCH 0/2 v5] Reintroduce NEXT_BUDDY for EEVDF Madadi Vineeth Reddy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=051285c7-56c0-455d-ab3e-9959edb5d13f@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=Aishwarya.TCV@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=peterz@infradead.org \
--cc=ryan.roberts@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®