From: Mike Galbraith <efault@gmx.de>
To: Tom Gebhardt <tomge68@gmail.com>, peterz@infradead.org
Cc: mingo@redhat.com, linux-kernel@vger.kernel.org
Subject: Re: sched/fair: DELAY_DEQUEUE causes ~25% pipe IPC regression on Raspberry Pi 5
Date: Mon, 20 Apr 2026 05:53:37 +0200 [thread overview]
Message-ID: <2877d886d8470f4ea3393e8c2e570a059c2b8b1b.camel@gmx.de> (raw)
In-Reply-To: <CAOX0jgh-BHHZJJ4vB_mFNwaUBqxfjWE0ahfGwBB4oCOfCYaYGg@mail.gmail.com>
Greetings,
On Thu, 2026-04-16 at 14:22 +0200, Tom Gebhardt wrote:
> Hi Peter,
>
> I would like to report a measurable pipe IPC throughput regression introduced by
> commit 152e11f ("sched/fair: Implement delayed dequeue"), first
> present in v6.12-rc1.
>
> This has been independently confirmed on the official Raspberry Pi
> Linux issue tracker
> (raspberrypi/linux #7308), where the RPi kernel team directed the
> issue upstream.
>
>
> Hardware / Software
> -------------------
> - Raspberry Pi 5 Model B, BCM2712 (C1 stepping), 8 GB RAM
> - Raspberry Pi OS Bookworm (arm64)
> - Kernels tested: 6.6.78-v8-16k+ (rpi-6.6.y), 6.12.75+rpt-rpi-2712,
> 6.12.81-v8-16k+ (custom)
> - Benchmark: stress-ng 0.15.06, --pipe 4 --timeout 20s --metrics-brief
>
>
> Observed regression
> -------------------
> Comparing pipe IPC throughput across kernels (overclocked, arm_freq=2800):
>
> Kernel pipe bogo ops/s vs. 6.6
> 6.6.78 2 487 746 100%
> 6.12.75 1 651 427 -34%
> 6.18.21 2 049 701 -18%
>
> This regression pattern is consistent across two separate Raspberry Pi
> 5 units and has
> been independently reproduced by the RPi kernel team with 20-run averages:
> 6.6=2065 Kops/s, 6.12=1662, 6.18=1805, 7.0=1570 (lowest).
Does reproduction require more than the posted stress-ng args?
Raw data from my pi5+bookworm+6.12.75+rpt-rpi-2712 (no overclock)
rpi5:..debug/sched # stress-ng --version
stress-ng, version 0.21.00 (gcc 12.2.0, aarch64 Linux 6.12.75+rpt-rpi-2712) 💻🔥
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info: [2965] setting to a 20 secs run per stressor
stress-ng: info: [2965] dispatching hogs: 4 pipe
stress-ng: metrc: [2965] stressor bogo ops real time usr time sys time bogo ops/s bogo ops/s
stress-ng: metrc: [2965] (secs) (secs) (secs) (real time) (usr+sys time)
stress-ng: metrc: [2965] pipe 57731375 20.00 6.69 73.24 2886616.84 722246.62
stress-ng: info: [2965] skipped: 0
stress-ng: info: [2965] passed: 4: pipe (4)
stress-ng: info: [2965] failed: 0
stress-ng: info: [2965] metrics untrustworthy: 0
stress-ng: info: [2965] successful run completed in 20.00 secs
rpi5:..debug/sched # echo NO_DELAY_DEQUEUE > features
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info: [3048] setting to a 20 secs run per stressor
stress-ng: info: [3048] dispatching hogs: 4 pipe
stress-ng: metrc: [3048] stressor bogo ops real time usr time sys time bogo ops/s bogo ops/s
stress-ng: metrc: [3048] (secs) (secs) (secs) (real time) (usr+sys time)
stress-ng: metrc: [3048] pipe 58814508 20.00 6.78 73.17 2940713.90 735679.81
stress-ng: info: [3048] skipped: 0
stress-ng: info: [3048] passed: 4: pipe (4)
stress-ng: info: [3048] failed: 0
stress-ng: info: [3048] metrics untrustworthy: 0
stress-ng: info: [3048] successful run completed in 20.00 secs
rpi5:..debug/sched # echo DELAY_DEQUEUE > features
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info: [3349] setting to a 20 secs run per stressor
stress-ng: info: [3349] dispatching hogs: 4 pipe
stress-ng: metrc: [3349] stressor bogo ops real time usr time sys time bogo ops/s bogo ops/s
stress-ng: metrc: [3349] (secs) (secs) (secs) (real time) (usr+sys time)
stress-ng: metrc: [3349] pipe 58058262 20.00 6.62 73.32 2902974.63 726327.83
stress-ng: info: [3349] skipped: 0
stress-ng: info: [3349] passed: 4: pipe (4)
stress-ng: info: [3349] failed: 0
stress-ng: info: [3349] metrics untrustworthy: 0
stress-ng: info: [3349] successful run completed in 20.00 secs
next prev parent reply other threads:[~2026-04-20 3:53 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-16 12:22 Tom Gebhardt
2026-04-16 20:04 ` Christian Loehle
2026-04-17 15:42 ` Vincent Guittot
2026-04-18 16:12 ` Tom Gebhardt
2026-04-20 14:27 ` Vincent Guittot
2026-04-20 3:53 ` Mike Galbraith [this message]
2026-04-20 10:59 ` Tom Gebhardt
2026-04-21 3:55 ` Mike Galbraith
2026-04-21 12:46 ` Tom Gebhardt
2026-04-21 13:14 ` Mike Galbraith
2026-04-23 4:51 ` Mike Galbraith
2026-04-23 8:04 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2877d886d8470f4ea3393e8c2e570a059c2b8b1b.camel@gmx.de \
--to=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tomge68@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®