* Re: [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp
2026-08-07 20:38 [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp Shubhang Kaushik (Ampere)
@ 2026-08-17 23:15 ` Shubhang
2026-08-18 8:02 ` Vincent Guittot
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Shubhang @ 2026-08-17 23:15 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, John Stultz, Zhan Xusheng,
Christopher Lameter, Shubhang Kaushik
Cc: linux-kernel
Hello all,
Gentle ping for v3. I repeated the Hackbench comparison with multiple runs
per kernel. The reported thread and process cases stayed within about 1.5%
mean delta, with similar baseline run-to-run variation.
Please let me know if there are further concerns with this approach.
Thanks,
Shubhang Kaushik
On Fri, 7 Aug 2026, Shubhang Kaushik (Ampere) wrote:
> Fixes: 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Acked-by: John Stultz <jstultz@google.com>
> Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
> ---
> Temporary tracing under hackbench load confirmed that
> update_rq_avg_idle() can be reached with rq->idle_stamp == 0.
> Hackbench showed no material regression versus v7.2-rc5 mainline.
>
> Related discussion:
> https://lore.kernel.org/r/20260423023322.1293923-1-firelzrd@gmail.com
>
> This is a narrower variant of the earlier proposal. It keeps the
> rq->idle_stamp guard in update_rq_avg_idle(), but intentionally does not
> stamp idle entry from set_next_task_idle(), preserving the existing
> newidle accounting model and avoiding force-idle/proxy-exec accounting
> concerns.
> ---
> Changes in v3:
> - Describe the sched_balance_newidle()/ttwu_pending path as an
> example of entering idle without a valid rq->idle_stamp.
> - Drop unlikely() from the idle_stamp check.
> - Add Acked-by from John Stultz.
>
> Link to v2: https://lore.kernel.org/r/20260806-master-v2-1-e1f3a1a0c903@gentwo.org
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp
2026-08-07 20:38 [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp Shubhang Kaushik (Ampere)
2026-08-17 23:15 ` Shubhang
@ 2026-08-18 8:02 ` Vincent Guittot
2026-08-31 18:17 ` Shubhang
2026-09-02 7:22 ` [tip: sched/urgent] " tip-bot2 for Shubhang Kaushik (Ampere)
3 siblings, 0 replies; 6+ messages in thread
From: Vincent Guittot @ 2026-08-18 8:02 UTC (permalink / raw)
To: Shubhang Kaushik (Ampere)
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, John Stultz, Zhan Xusheng, Christopher Lameter,
Shubhang Kaushik, linux-kernel
On Fri, 7 Aug 2026 at 22:47, Shubhang Kaushik (Ampere) <sh@gentwo.org> wrote:
>
> Commit 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved
> to an idle CPU") moved rq->avg_idle accounting out of the wakeup path and
> into put_prev_task_idle(), so that the idle interval is consumed whenever
> the idle task is switched out.
>
> The wakeup-side accounting that it replaced only updated rq->avg_idle
> when rq->idle_stamp was non-zero. The new helper lost that validity
> check and unconditionally computes:
>
> rq_clock(rq) - rq->idle_stamp
>
> If rq->idle_stamp is zero, this uses rq_clock(rq) as the sample. That is
> not a valid idle duration and can immediately drive rq->avg_idle to its
> clamp.
>
> This can happen when sched_balance_newidle() returns before setting
> rq->idle_stamp, for example when this_rq->ttwu_pending is set. In that
> case the rq can switch to the idle task with idle_stamp still zero and
> leave idle again when the pending wakeup is processed.
>
> Other paths can also switch to the idle task without setting
> rq->idle_stamp via newidle_balance(), for example find_proxy_task() or
> force-idling.
>
> Restore the idle_stamp validity check in update_rq_avg_idle() and skip
> the rq->avg_idle update when there is no measured idle interval.
>
> Fixes: 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Acked-by: John Stultz <jstultz@google.com>
> Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
> ---
> Temporary tracing under hackbench load confirmed that
> update_rq_avg_idle() can be reached with rq->idle_stamp == 0.
> Hackbench showed no material regression versus v7.2-rc5 mainline.
>
> Related discussion:
> https://lore.kernel.org/r/20260423023322.1293923-1-firelzrd@gmail.com
>
> This is a narrower variant of the earlier proposal. It keeps the
> rq->idle_stamp guard in update_rq_avg_idle(), but intentionally does not
> stamp idle entry from set_next_task_idle(), preserving the existing
> newidle accounting model and avoiding force-idle/proxy-exec accounting
> concerns.
> ---
> Changes in v3:
> - Describe the sched_balance_newidle()/ttwu_pending path as an
> example of entering idle without a valid rq->idle_stamp.
> - Drop unlikely() from the idle_stamp check.
> - Add Acked-by from John Stultz.
>
> Link to v2: https://lore.kernel.org/r/20260806-master-v2-1-e1f3a1a0c903@gentwo.org
>
> Changes in v2:
> - Add Reviewed-by from Prateek.
> - Mention find_proxy_task() and force-idling as examples of paths that
> can switch to the idle task without a valid rq->idle_stamp.
> - Cc John Stultz.
>
> Link to v1: https://lore.kernel.org/r/20260728-master-v1-1-f95d9b0147d2@gentwo.org
> ---
> kernel/sched/core.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 96226707c2f6135341aa779b8262f113e103d8ad..68fa724cd668fb6c4cead329d05fa95e2f1ea5db 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3732,11 +3732,17 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
>
> void update_rq_avg_idle(struct rq *rq)
> {
> - u64 delta = rq_clock(rq) - rq->idle_stamp;
> - u64 max = 2*rq->max_idle_balance_cost;
> + u64 idle_stamp = rq->idle_stamp;
> + u64 delta, max;
> +
> + if (!idle_stamp)
> + return;
> +
> + delta = rq_clock(rq) - idle_stamp;
>
> update_avg(&rq->avg_idle, delta);
>
> + max = 2 * rq->max_idle_balance_cost;
> if (rq->avg_idle > max)
> rq->avg_idle = max;
> rq->idle_stamp = 0;
>
> ---
> base-commit: 3f008280327ba5ad132965abab0c7846283cef0c
> change-id: 20260728-master-55cd7cc13290
>
> Best regards,
> --
> Shubhang Kaushik (Ampere) <sh@gentwo.org>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp
2026-08-07 20:38 [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp Shubhang Kaushik (Ampere)
2026-08-17 23:15 ` Shubhang
2026-08-18 8:02 ` Vincent Guittot
@ 2026-08-31 18:17 ` Shubhang
2026-09-01 2:44 ` Zhan Xusheng
2026-09-02 7:22 ` [tip: sched/urgent] " tip-bot2 for Shubhang Kaushik (Ampere)
3 siblings, 1 reply; 6+ messages in thread
From: Shubhang @ 2026-08-31 18:17 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Valentin Schneider, K Prateek Nayak, John Stultz, Zhan Xusheng,
Christopher Lameter, Shubhang Kaushik
Cc: linux-kernel
Hi,
Could you please let me know whether it is suitable for the sched/core
tree for v7.3, or whether further work is needed?
Thanks,
Shubhang Kaushik
On Fri, 7 Aug 2026, Shubhang Kaushik (Ampere) wrote:
> Commit 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved
> to an idle CPU") moved rq->avg_idle accounting out of the wakeup path and
> into put_prev_task_idle(), so that the idle interval is consumed whenever
> the idle task is switched out.
>
> The wakeup-side accounting that it replaced only updated rq->avg_idle
> when rq->idle_stamp was non-zero. The new helper lost that validity
> check and unconditionally computes:
>
> rq_clock(rq) - rq->idle_stamp
>
> If rq->idle_stamp is zero, this uses rq_clock(rq) as the sample. That is
> not a valid idle duration and can immediately drive rq->avg_idle to its
> clamp.
>
> This can happen when sched_balance_newidle() returns before setting
> rq->idle_stamp, for example when this_rq->ttwu_pending is set. In that
> case the rq can switch to the idle task with idle_stamp still zero and
> leave idle again when the pending wakeup is processed.
>
> Other paths can also switch to the idle task without setting
> rq->idle_stamp via newidle_balance(), for example find_proxy_task() or
> force-idling.
>
> Restore the idle_stamp validity check in update_rq_avg_idle() and skip
> the rq->avg_idle update when there is no measured idle interval.
>
> Fixes: 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
> Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Acked-by: John Stultz <jstultz@google.com>
> Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
> ---
> Temporary tracing under hackbench load confirmed that
> update_rq_avg_idle() can be reached with rq->idle_stamp == 0.
> Hackbench showed no material regression versus v7.2-rc5 mainline.
>
> Related discussion:
> https://lore.kernel.org/r/20260423023322.1293923-1-firelzrd@gmail.com
>
> This is a narrower variant of the earlier proposal. It keeps the
> rq->idle_stamp guard in update_rq_avg_idle(), but intentionally does not
> stamp idle entry from set_next_task_idle(), preserving the existing
> newidle accounting model and avoiding force-idle/proxy-exec accounting
> concerns.
> ---
> Changes in v3:
> - Describe the sched_balance_newidle()/ttwu_pending path as an
> example of entering idle without a valid rq->idle_stamp.
> - Drop unlikely() from the idle_stamp check.
> - Add Acked-by from John Stultz.
>
> Link to v2: https://lore.kernel.org/r/20260806-master-v2-1-e1f3a1a0c903@gentwo.org
>
> Changes in v2:
> - Add Reviewed-by from Prateek.
> - Mention find_proxy_task() and force-idling as examples of paths that
> can switch to the idle task without a valid rq->idle_stamp.
> - Cc John Stultz.
>
> Link to v1: https://lore.kernel.org/r/20260728-master-v1-1-f95d9b0147d2@gentwo.org
> ---
> kernel/sched/core.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 96226707c2f6135341aa779b8262f113e103d8ad..68fa724cd668fb6c4cead329d05fa95e2f1ea5db 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3732,11 +3732,17 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
>
> void update_rq_avg_idle(struct rq *rq)
> {
> - u64 delta = rq_clock(rq) - rq->idle_stamp;
> - u64 max = 2*rq->max_idle_balance_cost;
> + u64 idle_stamp = rq->idle_stamp;
> + u64 delta, max;
> +
> + if (!idle_stamp)
> + return;
> +
> + delta = rq_clock(rq) - idle_stamp;
>
> update_avg(&rq->avg_idle, delta);
>
> + max = 2 * rq->max_idle_balance_cost;
> if (rq->avg_idle > max)
> rq->avg_idle = max;
> rq->idle_stamp = 0;
>
> ---
> base-commit: 3f008280327ba5ad132965abab0c7846283cef0c
> change-id: 20260728-master-55cd7cc13290
>
> Best regards,
> --
> Shubhang Kaushik (Ampere) <sh@gentwo.org>
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp
2026-08-31 18:17 ` Shubhang
@ 2026-09-01 2:44 ` Zhan Xusheng
0 siblings, 0 replies; 6+ messages in thread
From: Zhan Xusheng @ 2026-09-01 2:44 UTC (permalink / raw)
To: sh
Cc: mingo, peterz, juri.lelli, vincent.guittot, dietmar.eggemann,
rostedt, bsegall, mgorman, vschneid, kprateek.nayak, jstultz, cl,
shubhang, linux-kernel, zhanxusheng
The diff of 4b603f1551a73 is its own evidence, which may be quicker to
look at than the description:
- if (rq->idle_stamp) {
- u64 delta = rq_clock(rq) - rq->idle_stamp;
+void update_rq_avg_idle(struct rq *rq)
+{
+ u64 delta = rq_clock(rq) - rq->idle_stamp;
The trigger moved on purpose, from the wakeup to idle switch-out. The
guard came off with it, and nothing about that fails to compile.
Two things about it that the changelog leaves narrower than they are.
rq->idle_stamp has exactly one setter, fair.c:14563, and it sits below the
ttwu_pending bail-out at 14555. So the reachable set is not the three
examples but any path into idle that misses that one line, and since
update_rq_avg_idle() zeroes the field on the way out, every such entry
starts from zero again.
The sample is not merely invalid, it saturates on the first hit.
update_avg() adds diff/8, so a rq_clock() of ~1e12 on a few minutes of
uptime moves avg_idle by ~1.25e11, against a clamp of
2*max_idle_balance_cost in the 1e4..1e5 range. Both places that consult
avg_idle to decide whether an idle interval was long enough to be worth
balancing, fair.c:14504 and 14584, then stop holding anything back. It
only ever adds newidle balancing, never removes it, which fits latency
reports rather than wrong results.
I also went looking for a stale stamp rather than a zero one and did not
find one: 14674 clears it when newidle actually pulled a task, so no idle
follows, and the other write is sched_init(). The field is either zero or
this idle period's rq_clock, which is what the restored guard splits on.
Reviewed-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Thanks,
Zhan Xusheng
^ permalink raw reply [flat|nested] 6+ messages in thread
* [tip: sched/urgent] sched/core: Skip rq->avg_idle update without a valid idle_stamp
2026-08-07 20:38 [PATCH v3] sched/core: Skip rq->avg_idle update without a valid idle_stamp Shubhang Kaushik (Ampere)
` (2 preceding siblings ...)
2026-08-31 18:17 ` Shubhang
@ 2026-09-02 7:22 ` tip-bot2 for Shubhang Kaushik (Ampere)
3 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Shubhang Kaushik (Ampere) @ 2026-09-02 7:22 UTC (permalink / raw)
To: linux-tip-commits
Cc: Shubhang Kaushik (Ampere), Peter Zijlstra (Intel),
K Prateek Nayak, Vincent Guittot, John Stultz, x86, linux-kernel
The following commit has been merged into the sched/urgent branch of tip:
Commit-ID: c6dcd97c8be75f052a1ca52cf79b03e7292962f1
Gitweb: https://git.kernel.org/tip/c6dcd97c8be75f052a1ca52cf79b03e7292962f1
Author: Shubhang Kaushik (Ampere) <sh@gentwo.org>
AuthorDate: Fri, 07 Aug 2026 13:38:52 -07:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Wed, 02 Sep 2026 09:17:49 +02:00
sched/core: Skip rq->avg_idle update without a valid idle_stamp
Commit 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved
to an idle CPU") moved rq->avg_idle accounting out of the wakeup path and
into put_prev_task_idle(), so that the idle interval is consumed whenever
the idle task is switched out.
The wakeup-side accounting that it replaced only updated rq->avg_idle
when rq->idle_stamp was non-zero. The new helper lost that validity
check and unconditionally computes:
rq_clock(rq) - rq->idle_stamp
If rq->idle_stamp is zero, this uses rq_clock(rq) as the sample. That is
not a valid idle duration and can immediately drive rq->avg_idle to its
clamp.
This can happen when sched_balance_newidle() returns before setting
rq->idle_stamp, for example when this_rq->ttwu_pending is set. In that
case the rq can switch to the idle task with idle_stamp still zero and
leave idle again when the pending wakeup is processed.
Other paths can also switch to the idle task without setting
rq->idle_stamp via newidle_balance(), for example find_proxy_task() or
force-idling.
Restore the idle_stamp validity check in update_rq_avg_idle() and skip
the rq->avg_idle update when there is no measured idle interval.
Fixes: 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260807-master-v3-1-c328354efed3@gentwo.org
---
kernel/sched/core.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index f782751..7472450 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3742,11 +3742,17 @@ static inline void ttwu_do_wakeup(struct task_struct *p)
void update_rq_avg_idle(struct rq *rq)
{
- u64 delta = rq_clock(rq) - rq->idle_stamp;
- u64 max = 2*rq->max_idle_balance_cost;
+ u64 idle_stamp = rq->idle_stamp;
+ u64 delta, max;
+
+ if (!idle_stamp)
+ return;
+
+ delta = rq_clock(rq) - idle_stamp;
update_avg(&rq->avg_idle, delta);
+ max = 2 * rq->max_idle_balance_cost;
if (rq->avg_idle > max)
rq->avg_idle = max;
rq->idle_stamp = 0;
^ permalink raw reply [flat|nested] 6+ messages in thread