mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Chen Yu <yu.c.chen@intel.com>
To: Aishwarya Rambhadran <aishwarya.rambhadran@arm.com>
Cc: Peter Zijlstra <peterz@infradead.org>, <mingo@kernel.org>,
	<longman@redhat.com>, <chenridong@huaweicloud.com>,
	<juri.lelli@redhat.com>, <vincent.guittot@linaro.org>,
	<dietmar.eggemann@arm.com>, <rostedt@goodmis.org>,
	<bsegall@google.com>, <mgorman@suse.de>, <vschneid@redhat.com>,
	<tj@kernel.org>, <hannes@cmpxchg.org>, <mkoutny@suse.com>,
	<cgroups@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<jstultz@google.com>, <kprateek.nayak@amd.com>,
	<qyousef@layalina.io>, Ryan Roberts <ryan.roberts@arm.com>,
	<chen.yu@linux.dev>
Subject: Re: [REGRESSION] [PATCH v3 7/7] sched/eevdf: Move to a single runqueue
Date: Thu, 24 Sep 2026 00:48:10 +0800	[thread overview]
Message-ID: <arQCykkkCnADu5ek@chenyu-dev> (raw)
In-Reply-To: <bd81bbec-ee59-401a-bec6-116196ad3a34@arm.com>

On Wed, Sep 23, 2026 at 07:51:35PM +0530, Aishwarya Rambhadran wrote:
> Hi Peter,
> 
> We have observed some schbench p99 request latency regressions while
> comparing v7.2 against v7.3 cycle on an AWS Graviton3 (m7g.metal)
> using Fastpath (Linux kernel performance benchmarking tool).
> 
> The schbench suite as a whole does not show a significant regression,
> and the results appear to be a mix of improvements and regressions
> depending on the thread configuration. However, a few configurations
> show fairly large p99 request latency regressions. For example,
> comparing v7.2 against v7.3-rc4:
> 
> message-thread:16, worker-thread:16
> v7.2 = 662869 us
> v7.3-rc4 = 969557 us
> change% = -31.63%
> 
> message-thread:64, worker-thread:4
> v7.2 = 730453 us
> v7.3-rc4 = 942251 us
> change% = -22.48%
> 
> message-thread:32, worker-thread:4
> v7.2 = 74251 us
> v7.3-rc4 = 56725 us
> change% = +30.90%
> 
> 
> So this does not appear to be a general latency degradation; some
> configs improve while others regress.
> 
> I also ran a Fastpath automated git bisection for the m 16, t 16 p99
> request latency regression between v7.2 and v7.3-rc1, which identified
> this commit as the first bad commit. The original regression was ~30.9%,
> & the bisection reproducibility check showed ~27.8% regression with 20
> repeats across 2 boot sessions.
> 
> This commit looks relevant since it changes EEVDF runqueue organization
> and is intended, in part, to address latency issues with hierarchical fair/
> cgroup scheduling. I also noticed that there have been fixes on top of this
> change, including 68e37487810a ("sched/fair: Fix flat hierarchy"), which
> fixes vruntime handling and has a Fixes: 85570f10a4c6 tag.
> 
> Given that the overall schbench result is mostly neutral while some p99
> configurations move considerably in both directions, I wanted to check
> whether this could be an expected trade-off from the single-runqueue
> change, or whether the p99 regressions are worth investigating further.
> 
> I can run additional tests or provide the full results/bisection data if
> useful.
> 
>

Would disabling WA_WEIGHT help? I happened to find that this workaround works
for me to restore netperf performance (though it is a throughput score rather
than a latency one), it seems that task_h_load() becomes much
larger(also on sched/task_h_load):
echo NO_WA_WEIGHT > /sys/kernel/debug/sched/features
https://lore.kernel.org/lkml/aoxah90s0bQ4tcUW@three-body/ 

thanks,
Chenyu



  reply	other threads:[~2026-09-23 17:01 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-05 12:40 [PATCH v3 0/7] sched: Flatten the pick Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 1/7] sched/fair: Add cgroup_mode switch Peter Zijlstra
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 2/7] sched/fair: Add cgroup_mode: up Peter Zijlstra
2026-06-05 15:07   ` Peter Zijlstra
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 3/7] sched/fair: Add cgroup_mode: max Peter Zijlstra
2026-06-10 15:09   ` Waiman Long
2026-06-10 15:42     ` Waiman Long
2026-06-11 13:49       ` Peter Zijlstra
2026-06-11 13:47     ` Peter Zijlstra
2026-06-11 20:57       ` Waiman Long
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 4/7] sched/fair: Add cgroup_mode: concur Peter Zijlstra
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 5/7] sched/fair: Add cgroup_mode: tasks Peter Zijlstra
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 6/7] sched/fair: Change the default cgroup_mode to concur Peter Zijlstra
2026-06-30  9:03   ` [tip: sched/core] " tip-bot2 for Peter Zijlstra
2026-06-05 12:40 ` [PATCH v3 7/7] sched/eevdf: Move to a single runqueue Peter Zijlstra
2026-06-20  3:54   ` Chen, Yu C
2026-06-26 11:40     ` Peter Zijlstra
2026-06-29 14:02       ` Vincent Guittot
2026-06-30  9:03     ` [tip: sched/core] sched/fair: Fix overflow in update_tg_cfs_runnable() tip-bot2 for Chen, Yu C
2026-06-30  9:03   ` [tip: sched/core] sched/eevdf: Move to a single runqueue tip-bot2 for Peter Zijlstra (Intel)
2026-09-23 14:21   ` [REGRESSION] [PATCH v3 7/7] " Aishwarya Rambhadran
2026-09-23 16:48     ` Chen Yu [this message]
2026-09-24  7:05       ` K Prateek Nayak
2026-09-24  8:19       ` Mike Galbraith
2026-06-09  5:37 ` [PATCH v3 0/7] sched: Flatten the pick K Prateek Nayak
2026-06-12  2:29 ` Shubhang Kaushik
2026-08-17 16:05 ` Szabina Korbai
2026-08-17 16:35   ` K Prateek Nayak
2026-08-18  9:04     ` Szabina Korbai
2026-08-18  9:16       ` Peter Zijlstra
2026-08-21 10:37         ` Szabina Korbai
2026-08-24 14:51         ` Chen Yu
2026-08-24 13:53           ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=arQCykkkCnADu5ek@chenyu-dev \
    --to=yu.c.chen@intel.com \
    --cc=aishwarya.rambhadran@arm.com \
    --cc=bsegall@google.com \
    --cc=cgroups@vger.kernel.org \
    --cc=chen.yu@linux.dev \
    --cc=chenridong@huaweicloud.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=hannes@cmpxchg.org \
    --cc=jstultz@google.com \
    --cc=juri.lelli@redhat.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=mgorman@suse.de \
    --cc=mingo@kernel.org \
    --cc=mkoutny@suse.com \
    --cc=peterz@infradead.org \
    --cc=qyousef@layalina.io \
    --cc=rostedt@goodmis.org \
    --cc=ryan.roberts@arm.com \
    --cc=tj@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®