mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mike Galbraith <efault@gmx.de>
To: Tom Gebhardt <tomge68@gmail.com>, peterz@infradead.org
Cc: mingo@redhat.com, linux-kernel@vger.kernel.org
Subject: Re: sched/fair: DELAY_DEQUEUE causes ~25% pipe IPC regression on Raspberry Pi 5
Date: Mon, 20 Apr 2026 05:53:37 +0200	[thread overview]
Message-ID: <2877d886d8470f4ea3393e8c2e570a059c2b8b1b.camel@gmx.de> (raw)
In-Reply-To: <CAOX0jgh-BHHZJJ4vB_mFNwaUBqxfjWE0ahfGwBB4oCOfCYaYGg@mail.gmail.com>

Greetings,

On Thu, 2026-04-16 at 14:22 +0200, Tom Gebhardt wrote:
> Hi Peter,
> 
> I would like to report a measurable pipe IPC throughput regression introduced by
> commit 152e11f ("sched/fair: Implement delayed dequeue"), first
> present in v6.12-rc1.
> 
> This has been independently confirmed on the official Raspberry Pi
> Linux issue tracker
> (raspberrypi/linux #7308), where the RPi kernel team directed the
> issue upstream.
> 
> 
> Hardware / Software
> -------------------
> - Raspberry Pi 5 Model B, BCM2712 (C1 stepping), 8 GB RAM
> - Raspberry Pi OS Bookworm (arm64)
> - Kernels tested: 6.6.78-v8-16k+ (rpi-6.6.y), 6.12.75+rpt-rpi-2712,
> 6.12.81-v8-16k+ (custom)
> - Benchmark: stress-ng 0.15.06, --pipe 4 --timeout 20s --metrics-brief
> 
> 
> Observed regression
> -------------------
> Comparing pipe IPC throughput across kernels (overclocked, arm_freq=2800):
> 
>   Kernel        pipe bogo ops/s   vs. 6.6
>   6.6.78            2 487 746      100%
>   6.12.75           1 651 427      -34%
>   6.18.21           2 049 701      -18%
> 
> This regression pattern is consistent across two separate Raspberry Pi
> 5 units and has
> been independently reproduced by the RPi kernel team with 20-run averages:
> 6.6=2065 Kops/s, 6.12=1662, 6.18=1805, 7.0=1570 (lowest).

Does reproduction require more than the posted stress-ng args?

Raw data from my pi5+bookworm+6.12.75+rpt-rpi-2712 (no overclock)

rpi5:..debug/sched # stress-ng --version
stress-ng, version 0.21.00 (gcc 12.2.0, aarch64 Linux 6.12.75+rpt-rpi-2712) 💻🔥
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [2965] setting to a 20 secs run per stressor
stress-ng: info:  [2965] dispatching hogs: 4 pipe
stress-ng: metrc: [2965] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [2965]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [2965] pipe           57731375     20.00      6.69     73.24   2886616.84      722246.62
stress-ng: info:  [2965] skipped: 0
stress-ng: info:  [2965] passed: 4: pipe (4)
stress-ng: info:  [2965] failed: 0
stress-ng: info:  [2965] metrics untrustworthy: 0
stress-ng: info:  [2965] successful run completed in 20.00 secs
rpi5:..debug/sched # echo NO_DELAY_DEQUEUE > features
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [3048] setting to a 20 secs run per stressor
stress-ng: info:  [3048] dispatching hogs: 4 pipe
stress-ng: metrc: [3048] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [3048]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [3048] pipe           58814508     20.00      6.78     73.17   2940713.90      735679.81
stress-ng: info:  [3048] skipped: 0
stress-ng: info:  [3048] passed: 4: pipe (4)
stress-ng: info:  [3048] failed: 0
stress-ng: info:  [3048] metrics untrustworthy: 0
stress-ng: info:  [3048] successful run completed in 20.00 secs
rpi5:..debug/sched # echo DELAY_DEQUEUE > features
rpi5:..debug/sched # stress-ng --pipe 4 --timeout 20s --metrics-brief
stress-ng: info:  [3349] setting to a 20 secs run per stressor
stress-ng: info:  [3349] dispatching hogs: 4 pipe
stress-ng: metrc: [3349] stressor       bogo ops real time  usr time  sys time   bogo ops/s     bogo ops/s
stress-ng: metrc: [3349]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: metrc: [3349] pipe           58058262     20.00      6.62     73.32   2902974.63      726327.83
stress-ng: info:  [3349] skipped: 0
stress-ng: info:  [3349] passed: 4: pipe (4)
stress-ng: info:  [3349] failed: 0
stress-ng: info:  [3349] metrics untrustworthy: 0
stress-ng: info:  [3349] successful run completed in 20.00 secs

  parent reply	other threads:[~2026-04-20  3:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-16 12:22 Tom Gebhardt
2026-04-16 20:04 ` Christian Loehle
2026-04-17 15:42 ` Vincent Guittot
2026-04-18 16:12   ` Tom Gebhardt
2026-04-20 14:27     ` Vincent Guittot
2026-04-20  3:53 ` Mike Galbraith [this message]
2026-04-20 10:59   ` Tom Gebhardt
2026-04-21  3:55     ` Mike Galbraith
2026-04-21 12:46       ` Tom Gebhardt
2026-04-21 13:14         ` Mike Galbraith
2026-04-23  4:51       ` Mike Galbraith
2026-04-23  8:04         ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2877d886d8470f4ea3393e8c2e570a059c2b8b1b.camel@gmx.de \
    --to=efault@gmx.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=tomge68@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®