[PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[albin_yang]

From: Wei Yang <albinwyang@tencent.com>

When a task is blocked on a kernel mutex (a proxy-exec donor) and its mutex
owner runs on another CPU, find_proxy_task() migrates the still-blocked donor
to the owner's CPU via proxy_migrate_task(). This happens when the donor is
picked as next while not current, and the owner is on_rq and not delayed. The
migration re-attaches the donor with attach_task() -> activate_task(rq, p,
ENQUEUE_NOCLOCK), i.e. without ENQUEUE_RESTORE, so sched_info_enqueue() arms
last_queued even though the donor is not runnable at all -- a fake enqueue.

The real wakeup does not correct it: the donor is still on_rq and is not
sched_delayed, so ttwu_runnable() never calls enqueue_task(), and last_queued
keeps the migration timestamp. sched_info_arrive() then charges the whole
interval from the migration until the donor actually runs, i.e. the time it
spent blocked on the mutex is counted as time spent waiting for a CPU.

Fix by not re-arming last_queued while the task is still blocked on a mutex.
A delayed dequeue sleeper has no blocked_on, so this is an additional
condition rather than a replacement for the se.sched_delayed check, and the
DELAY_DEQUEUE fix is unaffected.

Measured on a 2 CPU VM: the same ~170 ms block is charged ~167 ms of
run_delay when the mutex owner runs on another CPU, and 0.01 ms with this
patch.

Fixes: b049b81bdff6 ("sched: Handle blocked-waiter migration (and return migration)")
Signed-off-by: Wei Yang <albinwyang@tencent.com>
---

Reproducer notes and full numbers (5 iterations each, run_delay is field 2 of
/proc/<tid>/schedstat). A userspace futex mutex never sets blocked_on, so this
needs a small test module exposing a kernel mutex:

  config                          block      run_delay delta   migrated
  -----------------------------   -------    ---------------   --------
  cross-cpu, unfixed              169.9 ms   167.1 ms          5/5
  cross-cpu, DELAY_DEQUEUE fix    169.9 ms   167.1 ms          5/5
  cross-cpu, with this patch      169.8 ms     0.01 ms         5/5
  same-cpu (control)              167.3 ms     0.00 ms         0/5

Sanity check: with the donor runnable and contending for the CPU, run_delay
still grows by ~39 ms over an 80 ms window (5/5), so the gate suppresses only
the fake enqueue, not real waits.

Applies on top of:
  [PATCH v4] sched/stats: Fix run_delay over-count for migrated sched_delayed tasks
  https://lore.kernel.org/all/20261024042753.2276779-1-albin_yang@163.com/

Kept separate from the DELAY_DEQUEUE fix on purpose: the delayed dequeue
over-count exists from v6.12, while this one needs the proxy exec
blocked-waiter migration, which only landed in v7.1. Coupling them would
either fail to build (v6.12.y..v6.16.y, no proxy exec at all) or add a
dependency on code that cannot trigger (v6.17.y..v7.0.y, proxy exec present
but no blocked-waiter migration). The two are also independent: the
DELAY_DEQUEUE fix alone leaves this case at 167.1 ms.

 kernel/sched/stats.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
index dc626f99ffd9..ef52fbe86bba 100644
--- a/kernel/sched/stats.h
+++ b/kernel/sched/stats.h
@@ -290,7 +290,7 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
  */
 static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
 {
-	if (!t->sched_info.last_queued && !t->se.sched_delayed)
+	if (!t->sched_info.last_queued && !t->se.sched_delayed && !task_is_blocked(t))
 		t->sched_info.last_queued = rq_clock(rq);
 }
 
-- 
2.43.7

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Sun, Oct 25, 2026 at 09:07:40PM +0800, albin_yang@163.com wrote:

Hi Wei,

> From: Wei Yang <albinwyang@tencent.com>
> 
> When a task is blocked on a kernel mutex (a proxy-exec donor) and its mutex
> owner runs on another CPU, find_proxy_task() migrates the still-blocked donor
> to the owner's CPU via proxy_migrate_task(). This happens when the donor is
> picked as next while not current, and the owner is on_rq and not delayed. The
> migration re-attaches the donor with attach_task() -> activate_task(rq, p,
> ENQUEUE_NOCLOCK), i.e. without ENQUEUE_RESTORE, so sched_info_enqueue() arms
> last_queued even though the donor is not runnable at all -- a fake enqueue.
> 
> The real wakeup does not correct it: the donor is still on_rq and is not
> sched_delayed, so ttwu_runnable() never calls enqueue_task(), and last_queued
> keeps the migration timestamp. sched_info_arrive() then charges the whole
> interval from the migration until the donor actually runs, i.e. the time it

Well it normally should be like that but, sched_info_arrive() only gets
called when we *hit* the CPU. The issue is already that we start the
counter when we are not waiting for a CPU. The counter gets
charged on dequeue.

ttwu_runnable() calls proxy_needs_return() in the proxy case, then that
calls dequeue_block_task() with TASK_WAKING, where we call
sched_info_dequeue(). Or Am I missing something?

> spent blocked on the mutex is counted as time spent waiting for a CPU.
> 
> Fix by not re-arming last_queued while the task is still blocked on a mutex.
> A delayed dequeue sleeper has no blocked_on, so this is an additional
> condition rather than a replacement for the se.sched_delayed check, and the
> DELAY_DEQUEUE fix is unaffected.
> 
> Measured on a 2 CPU VM: the same ~170 ms block is charged ~167 ms of
> run_delay when the mutex owner runs on another CPU, and 0.01 ms with this
> patch.
> 
> Fixes: b049b81bdff6 ("sched: Handle blocked-waiter migration (and return migration)")
> Signed-off-by: Wei Yang <albinwyang@tencent.com>
> ---
> 
> Reproducer notes and full numbers (5 iterations each, run_delay is field 2 of
> /proc/<tid>/schedstat). A userspace futex mutex never sets blocked_on, so this
> needs a small test module exposing a kernel mutex:
> 
>   config                          block      run_delay delta   migrated
>   -----------------------------   -------    ---------------   --------
>   cross-cpu, unfixed              169.9 ms   167.1 ms          5/5
>   cross-cpu, DELAY_DEQUEUE fix    169.9 ms   167.1 ms          5/5
>   cross-cpu, with this patch      169.8 ms     0.01 ms         5/5
>   same-cpu (control)              167.3 ms     0.00 ms         0/5
> 
> Sanity check: with the donor runnable and contending for the CPU, run_delay
> still grows by ~39 ms over an 80 ms window (5/5), so the gate suppresses only
> the fake enqueue, not real waits.
> 
> Applies on top of:
>   [PATCH v4] sched/stats: Fix run_delay over-count for migrated sched_delayed tasks
>   https://lore.kernel.org/all/20261024042753.2276779-1-albin_yang@163.com/

This should be a series instead of separate patches if this goes above.

Also AFAICT this could go wrong once John’s series is applied:

https://lore.kernel.org/lkml/20261023220431.1106611-8-jstultz@google.com/

Adding John to CC.

With this patch, now waiters get dragged from their beloved rq's and
gets in to owner's own list. In this case, on_rq can be false, and
ttwu_runnable() could early return without clearing blocked_on.

Waiters normally wake up with their owners, and wait the CPU for them
so normally we should not start the timer. But on a hierarchy like this,

Where A sleeps and holds the mutex M while B wants it, B with
mutex_lock_interruptible(), and when B gets a kill signal
B need to enqueue separately from A, so it starts to wait.
But blocked_on is set, so B's counter never starts.
This only happens when both pathes are applied.

There's a chance I'm getting something wrong tho. :-).

Thanks,

Kayra

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[albin_yang]

Hi Kayra,

Thanks for your review.

> ... sched_info_dequeue() ... Or Am I missing something?

No, you're right: the fake enqueue starts the counter at migration time, and
the bill is settled later in sched_info_dequeue() on the proxy_needs_return()
path (dequeue_block_task() with TASK_WAKING). The ~167 ms I measured is that
delta. I had written sched_info_arrive() charging it directly, which is wrong -
I'll fix the description.

> This should be a series instead of separate patches if this goes above.

Agreed - I'll resend as [PATCH v5 1/2] (DELAY_DEQUEUE fix) + [PATCH v5 2/2]
(this one).

> ... when B gets a kill signal B need to enqueue separately from A, so it
> starts to wait. But blocked_on is set, so B's counter never starts.

A case I hadn't considered, and my sanity check does not cover it: I only
verified that a donor waking normally and then contending for the CPU still
accumulates run_delay. A waiter enqueuing separately from its owner does
genuinely start waiting for a CPU.

So let me ask first: in that path, is blocked_on really still set when B is
enqueued separately? If yes, !task_is_blocked() would under-count B, and the
options as I see them are:

  a) a predicate that is also cleared on that path,
  b) clear blocked_on there (proxy-exec territory - for you/John/Peter),
  c) hold this patch until John's series lands and re-evaluate then,
  d) fix it at the source instead: give proxy_migrate_task() DEQUEUE_SAVE /
     ENQUEUE_RESTORE so the migration never touches sched_info at all. That
     sidesteps the blocked_on question entirely, but it also affects psi, rt,
     dl and sched_ext, so I'd want your/John's input first.

Do you know whether blocked_on is still set there? If it is, I'd still prefer
to take this now, since the under-count only shows up once John's series is in,
and revisit (or adjust) then - unless you'd rather wait.

Adding John to CC.

Thanks,
Wei Yang

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Mon, Oct 26, 2026 at 09:53:51PM +0800, albin_yang@163.com wrote:
> Hi Kayra,
> 
> Thanks for your review.
> 
> > ... sched_info_dequeue() ... Or Am I missing something?
> 
> No, you're right: the fake enqueue starts the counter at migration time, and
> the bill is settled later in sched_info_dequeue() on the proxy_needs_return()
> path (dequeue_block_task() with TASK_WAKING). The ~167 ms I measured is that
> delta. I had written sched_info_arrive() charging it directly, which is wrong -
> I'll fix the description.
> 
> > This should be a series instead of separate patches if this goes above.
> 
> Agreed - I'll resend as [PATCH v5 1/2] (DELAY_DEQUEUE fix) + [PATCH v5 2/2]
> (this one).
> 
> > ... when B gets a kill signal B need to enqueue separately from A, so it
> > starts to wait. But blocked_on is set, so B's counter never starts.
> 
> A case I hadn't considered, and my sanity check does not cover it: I only
> verified that a donor waking normally and then contending for the CPU still
> accumulates run_delay. A waiter enqueuing separately from its owner does
> genuinely start waiting for a CPU.
> 
> So let me ask first: in that path, is blocked_on really still set when B is
> enqueued separately? If yes, !task_is_blocked() would under-count B, and the
> options as I see them are:

AFAICT from the code, yes. BTW, I thought about this and I think a
separate fix would be better clearing blocked_on in this case. Since
the task is really waiting for CPU time. Separate from its owner.

Or am I missing something and this accually does not works?

> 
>   a) a predicate that is also cleared on that path,
>   b) clear blocked_on there (proxy-exec territory - for you/John/Peter),
>   c) hold this patch until John's series lands and re-evaluate then,
>   d) fix it at the source instead: give proxy_migrate_task() DEQUEUE_SAVE /
>      ENQUEUE_RESTORE so the migration never touches sched_info at all. That
>      sidesteps the blocked_on question entirely, but it also affects psi, rt,
>      dl and sched_ext, so I'd want your/John's input first.
> 
> Do you know whether blocked_on is still set there? If it is, I'd still prefer
> to take this now, since the under-count only shows up once John's series is in,
> and revisit (or adjust) then - unless you'd rather wait.

It's up to Peter to take it or not so... IDK.

Thanks,

Kayra

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[K Prateek Nayak]

On 10/25/2026 6:37 PM, albin_yang@163.com wrote:
> diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
> index dc626f99ffd9..ef52fbe86bba 100644
> --- a/kernel/sched/stats.h
> +++ b/kernel/sched/stats.h
> @@ -290,7 +290,7 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
>   */
>  static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
>  {
> -	if (!t->sched_info.last_queued && !t->se.sched_delayed)
> +	if (!t->sched_info.last_queued && !t->se.sched_delayed && !task_is_blocked(t))

I think the sched info bits should be adapted to use the p->is_blocked
indicator instead of adding these endless exceptions.

Could you see if something like below works:

  (Only build tested; Prepared on top of sched-core-2026-10-18)

diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
index ebe0a7765f98..bd8c5cdfd12a 100644
--- a/kernel/sched/stats.h
+++ b/kernel/sched/stats.h
@@ -233,16 +233,13 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
 /*
  * We are interested in knowing how long it was from the *first* time a
  * task was queued to the time that it finally hit a CPU, we call this routine
- * from dequeue_task() to account for possible rq->clock skew across CPUs. The
- * delta taken on each CPU would annul the skew.
+ * from dequeue_task() / __schedule() to account for possible rq->clock skew
+ * across CPUs. The delta taken on each CPU would annul the skew.
  */
-static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
+static inline void __sched_info_dequeue(struct rq *rq, struct task_struct *t)
 {
 	unsigned long long delta = 0;
 
-	if (!t->sched_info.last_queued)
-		return;
-
 	delta = rq_clock(rq) - t->sched_info.last_queued;
 	t->sched_info.last_queued = 0;
 	t->sched_info.run_delay += delta;
@@ -255,6 +252,15 @@ static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
 	rq_sched_info_dequeue(rq, delta);
 }
 
+static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
+{
+	/* blocked tasks are handled at sched_info_depart() */
+	if (t->is_blocked)
+		return;
+
+	__sched_info_dequeue(rq, t);
+}
+
 /*
  * Called when a task finally hits the CPU.  We can now calculate how
  * long it was waiting to run.  We also note when it began so that we
@@ -290,7 +296,7 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
  */
 static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
 {
-	if (!t->sched_info.last_queued)
+	if (t->is_blocked)
 		t->sched_info.last_queued = rq_clock(rq);
 }
 
@@ -310,6 +316,9 @@ static inline void sched_info_depart(struct rq *rq, struct task_struct *t)
 
 	if (task_is_running(t))
 		sched_info_enqueue(rq, t);
+
+	if (t->is_blocked)
+		__sched_info_dequeue(rq, t);
 }
 
 /*
---

It is similar to how we handle PSI signals.

>  		t->sched_info.last_queued = rq_clock(rq);
>  }
>  

-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[albin_yang]

Hi Prateek,

Thanks - using is_blocked as the single indicator is certainly cleaner than
piling up exceptions.

I rebuilt your diff so it applies cleanly and gave it a run. I'm afraid it
does not work as-is in my testing - two problems:

1) sched_info_depart() now calls __sched_info_dequeue(), which no longer has
   the "if (!last_queued) return;" guard. A blocking task has last_queued == 0
   (sched_info_arrive() cleared it at switch-in), so delta becomes
   rq_clock(rq) - 0, i.e. the whole uptime:

     cross-cpu: blocked 169.80 ms, run_delay delta 169047.52 ms
     same-cpu:  blocked 168.70 ms, run_delay delta 185515.00 ms

   The control is affected too, so this is not proxy specific.

2) sched_info_enqueue() now arms only when t->is_blocked, leaving ordinary
   runnable tasks with no counter at all - my sanity check went from +39 ms to
   0.00 ms in every iteration, including the control. I do not think either
   polarity works: is_blocked is still set during the real wakeup enqueue
   (delayed sleeper: inside ttwu_runnable()'s is_blocked branch; donor: cleared
   only later by ttwu_do_wakeup()). So it can gate the charge side, but the
   arming side needs something already cleared at wakeup - task_is_blocked() or
   sched_delayed.

The delayed reproducer shows 0, but that is easy to misread: it measures only a
delta, and the bogus depart charge lands before the window. Its absolute
run_delay is polluted too. (The 0.00 ms sanity figure is from the proxy
reproducer, which carries that check.)

For reference, !sched_delayed && !task_is_blocked() gives 0.01 ms on the proxy
case with the sanity check intact (+39 ms), and 0/20 on the delayed reproducer.

Happy to retest whatever you come up with.

Thanks,
Wei Yang

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[K Prateek Nayak]

Hello Wei,

On 10/26/2026 7:08 PM, albin_yang@163.com wrote:
> Hi Prateek,
> 
> Thanks - using is_blocked as the single indicator is certainly cleaner than
> piling up exceptions.
> 
> I rebuilt your diff so it applies cleanly and gave it a run. I'm afraid it
> does not work as-is in my testing

Thank you for testing the changes!

> two problems:
> 
> 1) sched_info_depart() now calls __sched_info_dequeue(), which no longer has
>    the "if (!last_queued) return;" guard. A blocking task has last_queued == 0
>    (sched_info_arrive() cleared it at switch-in), so delta becomes
>    rq_clock(rq) - 0, i.e. the whole uptime:
> 
>      cross-cpu: blocked 169.80 ms, run_delay delta 169047.52 ms
>      same-cpu:  blocked 168.70 ms, run_delay delta 185515.00 ms
> 
>    The control is affected too, so this is not proxy specific.

Ah! That is an oversight. I didn't realise arrive clears the last_queued
and yes, doing it unconditionally will account the full uptime.

> 
> 2) sched_info_enqueue() now arms only when t->is_blocked, leaving ordinary
>    runnable tasks with no counter at all - my sanity check went from +39 ms to
>    0.00 ms in every iteration, including the control. I do not think either
>    polarity works: is_blocked is still set during the real wakeup enqueue
>    (delayed sleeper: inside ttwu_runnable()'s is_blocked branch; donor: cleared
>    only later by ttwu_do_wakeup()). So it can gate the charge side, but the
>    arming side needs something already cleared at wakeup - task_is_blocked() or
>    sched_delayed.

Ack! I forgot task migrations. So based on a better look, all we have to
do i start accounting when the task becomes runnable again (p->is_blocked
is cleared) and don't account any time when the task is "on_rq" while
being blocked (p->is_blocked && p->on_rq).

> 
> The delayed reproducer shows 0, but that is easy to misread: it measures only a
> delta, and the bogus depart charge lands before the window. Its absolute
> run_delay is polluted too. (The 0.00 ms sanity figure is from the proxy
> reproducer, which carries that check.)
> 
> For reference, !sched_delayed && !task_is_blocked() gives 0.01 ms on the proxy
> case with the sanity check intact (+39 ms), and 0/20 on the delayed reproducer.
> 
> Happy to retest whatever you come up with.

So all things considered, I think the following should work unless I'm
terribly mistaken yet again :-) I've left my reasoning as comments:

diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
index ebe0a7765f98..c4c624deb25b 100644
--- a/kernel/sched/stats.h
+++ b/kernel/sched/stats.h
@@ -239,6 +239,15 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
 static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
 {
 	unsigned long long delta = 0;
+	/*
+	 * If a task is blocked (blocked donor, delayed dequeue task),
+	 * do not account the queued time as runnable. A task that is
+	 * blocking would have already cleared the "last_queued"
+	 * indicator from sched_info_arrive() when it got a chance to
+	 * run on the CPU.
+	 */
+	if (t->is_blocked)
+		t->sched_info.last_queued = 0;
 
 	if (!t->sched_info.last_queued)
 		return;
@@ -290,7 +299,23 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
  */
 static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
 {
-	if (!t->sched_info.last_queued)
+	/*
+	 * "t->is_blocked" is only cleared after the enqueue is done and
+	 * the task is woken up. If the task is blocked, update the
+	 * "last_queued" timestamp to that of last enqueue - if the task
+	 * is still blocked while it leaves the runqueue, this timestamp
+	 * is ignored and is cleared at sched_info_dequeue().
+	 *
+	 * To transition a task back to blocked after wakeup, it has to
+	 * run on the CPU and naturally reach a safe preemption point.
+	 * sched_info_arrive() will do the correct wait time accounting
+	 * based on the last "last_queued".
+	 *
+	 * Any movement of task after being marked not blocked will be
+	 * correctly captured as queued time at enqueue + dequeue
+	 * boundaries when "t->is_blocked" cleared.
+	 */
+	if (!t->sched_info.last_queued || t->is_blocked)
 		t->sched_info.last_queued = rq_clock(rq);
 }
 
---

That "t->is_blocked" check in sched_info_enqueue() may not be
necessary either but I've left it in there for now since I did my
quick testing with it.

-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Tue, Oct 27, 2026 at 09:41:51AM +0530, K Prateek Nayak wrote:
> Hello Wei,
> 
> On 10/26/2026 7:08 PM, albin_yang@163.com wrote:
> > Hi Prateek,
> > 
> > Thanks - using is_blocked as the single indicator is certainly cleaner than
> > piling up exceptions.
> > 
> > I rebuilt your diff so it applies cleanly and gave it a run. I'm afraid it
> > does not work as-is in my testing
> 
> Thank you for testing the changes!
> 
> > two problems:
> > 
> > 1) sched_info_depart() now calls __sched_info_dequeue(), which no longer has
> >    the "if (!last_queued) return;" guard. A blocking task has last_queued == 0
> >    (sched_info_arrive() cleared it at switch-in), so delta becomes
> >    rq_clock(rq) - 0, i.e. the whole uptime:
> > 
> >      cross-cpu: blocked 169.80 ms, run_delay delta 169047.52 ms
> >      same-cpu:  blocked 168.70 ms, run_delay delta 185515.00 ms
> > 
> >    The control is affected too, so this is not proxy specific.
> 
> Ah! That is an oversight. I didn't realise arrive clears the last_queued
> and yes, doing it unconditionally will account the full uptime.
> 
> > 
> > 2) sched_info_enqueue() now arms only when t->is_blocked, leaving ordinary
> >    runnable tasks with no counter at all - my sanity check went from +39 ms to
> >    0.00 ms in every iteration, including the control. I do not think either
> >    polarity works: is_blocked is still set during the real wakeup enqueue
> >    (delayed sleeper: inside ttwu_runnable()'s is_blocked branch; donor: cleared
> >    only later by ttwu_do_wakeup()). So it can gate the charge side, but the
> >    arming side needs something already cleared at wakeup - task_is_blocked() or
> >    sched_delayed.
> 
> Ack! I forgot task migrations. So based on a better look, all we have to
> do i start accounting when the task becomes runnable again (p->is_blocked
> is cleared) and don't account any time when the task is "on_rq" while
> being blocked (p->is_blocked && p->on_rq).
> 
> > 
> > The delayed reproducer shows 0, but that is easy to misread: it measures only a
> > delta, and the bogus depart charge lands before the window. Its absolute
> > run_delay is polluted too. (The 0.00 ms sanity figure is from the proxy
> > reproducer, which carries that check.)
> > 
> > For reference, !sched_delayed && !task_is_blocked() gives 0.01 ms on the proxy
> > case with the sanity check intact (+39 ms), and 0/20 on the delayed reproducer.
> > 
> > Happy to retest whatever you come up with.
> 
> So all things considered, I think the following should work unless I'm
> terribly mistaken yet again :-) I've left my reasoning as comments:
> 
> diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
> index ebe0a7765f98..c4c624deb25b 100644
> --- a/kernel/sched/stats.h
> +++ b/kernel/sched/stats.h
> @@ -239,6 +239,15 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
>  static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
>  {
>  	unsigned long long delta = 0;
> +	/*
> +	 * If a task is blocked (blocked donor, delayed dequeue task),
> +	 * do not account the queued time as runnable. A task that is
> +	 * blocking would have already cleared the "last_queued"
> +	 * indicator from sched_info_arrive() when it got a chance to
> +	 * run on the CPU.
> +	 */
> +	if (t->is_blocked)
> +		t->sched_info.last_queued = 0;
>  
>  	if (!t->sched_info.last_queued)
>  		return;
> @@ -290,7 +299,23 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
>   */
>  static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
>  {
> -	if (!t->sched_info.last_queued)
> +	/*
> +	 * "t->is_blocked" is only cleared after the enqueue is done and
> +	 * the task is woken up. If the task is blocked, update the
> +	 * "last_queued" timestamp to that of last enqueue - if the task
> +	 * is still blocked while it leaves the runqueue, this timestamp
> +	 * is ignored and is cleared at sched_info_dequeue().
> +	 *
> +	 * To transition a task back to blocked after wakeup, it has to
> +	 * run on the CPU and naturally reach a safe preemption point.
> +	 * sched_info_arrive() will do the correct wait time accounting
> +	 * based on the last "last_queued".
> +	 *
> +	 * Any movement of task after being marked not blocked will be
> +	 * correctly captured as queued time at enqueue + dequeue
> +	 * boundaries when "t->is_blocked" cleared.
> +	 */
> +	if (!t->sched_info.last_queued || t->is_blocked)
>  		t->sched_info.last_queued = rq_clock(rq);
>  }
>  
> ---
> 
> That "t->is_blocked" check in sched_info_enqueue() may not be
> necessary either but I've left it in there for now since I did my
> quick testing with it.


Hi Prateek,

IMO too doing this with is_blocked would be cleaner.

But I think this not works on a case where a donor is in a different CPU
from its owner, like:

A holds the mutex M in CPU0, B is the waiter and its on CPU1.

proxy_migrate_task() gets B to CPU0.
After that, A suddenly wants to goto CPU1. And it goes.
Now B goes back to its home CPU next to A.

Now in here, timer starts.

And suddenly A drops the mutex. Now B is not is_blocked.

Because B is back to its home CPU, there is no dequeue.
Even if there was B is not is_blocked so timer wouldn't be reset.

Normally we need to add the time that task waited for itself.
But in this case we add the time task waited for its mutex + for itself.

Or am I missssssing something?


> 
> -- 
> Thanks and Regards,
> Prateek
> 

Thanks,

Kayra

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[K Prateek Nayak]

Hello Kayra,

On 10/27/2026 11:41 AM, Kayra Cizmeci wrote:
>> So all things considered, I think the following should work unless I'm
>> terribly mistaken yet again :-) I've left my reasoning as comments:
>>
>> diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
>> index ebe0a7765f98..c4c624deb25b 100644
>> --- a/kernel/sched/stats.h
>> +++ b/kernel/sched/stats.h
>> @@ -239,6 +239,15 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
>>  static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
>>  {
>>  	unsigned long long delta = 0;
>> +	/*
>> +	 * If a task is blocked (blocked donor, delayed dequeue task),
>> +	 * do not account the queued time as runnable. A task that is
>> +	 * blocking would have already cleared the "last_queued"
>> +	 * indicator from sched_info_arrive() when it got a chance to
>> +	 * run on the CPU.
>> +	 */
>> +	if (t->is_blocked)
>> +		t->sched_info.last_queued = 0;
>>  
>>  	if (!t->sched_info.last_queued)
>>  		return;
>> @@ -290,7 +299,23 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
>>   */
>>  static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
>>  {
>> -	if (!t->sched_info.last_queued)
>> +	/*
>> +	 * "t->is_blocked" is only cleared after the enqueue is done and
>> +	 * the task is woken up. If the task is blocked, update the
>> +	 * "last_queued" timestamp to that of last enqueue - if the task
>> +	 * is still blocked while it leaves the runqueue, this timestamp
>> +	 * is ignored and is cleared at sched_info_dequeue().
>> +	 *
>> +	 * To transition a task back to blocked after wakeup, it has to
>> +	 * run on the CPU and naturally reach a safe preemption point.
>> +	 * sched_info_arrive() will do the correct wait time accounting
>> +	 * based on the last "last_queued".
>> +	 *
>> +	 * Any movement of task after being marked not blocked will be
>> +	 * correctly captured as queued time at enqueue + dequeue
>> +	 * boundaries when "t->is_blocked" cleared.
>> +	 */
>> +	if (!t->sched_info.last_queued || t->is_blocked)
>>  		t->sched_info.last_queued = rq_clock(rq);
>>  }
>>  
>> ---
>>
>> That "t->is_blocked" check in sched_info_enqueue() may not be
>> necessary either but I've left it in there for now since I did my
>> quick testing with it.
> 
> 
> Hi Prateek,
> 
> IMO too doing this with is_blocked would be cleaner.
> 
> But I think this not works on a case where a donor is in a different CPU
> from its owner, like:
> 
> A holds the mutex M in CPU0, B is the waiter and its on CPU1.
> 
> proxy_migrate_task() gets B to CPU0.
>> After that, A suddenly wants to goto CPU1. And it goes.
> Now B goes back to its home CPU next to A.
> 
> Now in here, timer starts.
> 
> And suddenly A drops the mutex. Now B is not is_blocked.
> 
> Because B is back to its home CPU, there is no dequeue.
> Even if there was B is not is_blocked so timer wouldn't be reset.
> 
> Normally we need to add the time that task waited for itself.
> But in this case we add the time task waited for its mutex + for itself.
> 
> Or am I missssssing something?

Indeed. That is possible and can probably be fixed by doing a
sched_info_enqueue() in proxy_needs_return() for
task_cpu(p) == p->wake_cpu case like:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 04b62c16fa8c..3d261a4e5b0b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3838,8 +3838,11 @@ static inline bool proxy_needs_return(struct rq *rq, struct task_struct *p)
 	 * will not apply. But if it did, this check is the safe way around
 	 * and would migrate.
 	 */
-	if (task_cpu(p) == p->wake_cpu)
+	if (task_cpu(p) == p->wake_cpu) {
+		/* Begin wait time accounting before marking !is_blocked. */
+		sched_info_enqueue(rq, p);
 		return false;
+	}
 
 	scoped_guard(raw_spinlock, &p->blocked_lock) {
 		/* Task is waking up; clear any blocked_on relationship */
---

Not sure if this solution is ugly or not for calling
sched_info_enqueue() out of the enqueue path :-)

-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Tue, Oct 27, 2026 at 11:54:35AM +0530, K Prateek Nayak wrote:
> Hello Kayra,
> 
> On 10/27/2026 11:41 AM, Kayra Cizmeci wrote:
> >> So all things considered, I think the following should work unless I'm
> >> terribly mistaken yet again :-) I've left my reasoning as comments:
> >>
> >> diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
> >> index ebe0a7765f98..c4c624deb25b 100644
> >> --- a/kernel/sched/stats.h
> >> +++ b/kernel/sched/stats.h
> >> @@ -239,6 +239,15 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
> >>  static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
> >>  {
> >>  	unsigned long long delta = 0;
> >> +	/*
> >> +	 * If a task is blocked (blocked donor, delayed dequeue task),
> >> +	 * do not account the queued time as runnable. A task that is
> >> +	 * blocking would have already cleared the "last_queued"
> >> +	 * indicator from sched_info_arrive() when it got a chance to
> >> +	 * run on the CPU.
> >> +	 */
> >> +	if (t->is_blocked)
> >> +		t->sched_info.last_queued = 0;
> >>  
> >>  	if (!t->sched_info.last_queued)
> >>  		return;
> >> @@ -290,7 +299,23 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
> >>   */
> >>  static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
> >>  {
> >> -	if (!t->sched_info.last_queued)
> >> +	/*
> >> +	 * "t->is_blocked" is only cleared after the enqueue is done and
> >> +	 * the task is woken up. If the task is blocked, update the
> >> +	 * "last_queued" timestamp to that of last enqueue - if the task
> >> +	 * is still blocked while it leaves the runqueue, this timestamp
> >> +	 * is ignored and is cleared at sched_info_dequeue().
> >> +	 *
> >> +	 * To transition a task back to blocked after wakeup, it has to
> >> +	 * run on the CPU and naturally reach a safe preemption point.
> >> +	 * sched_info_arrive() will do the correct wait time accounting
> >> +	 * based on the last "last_queued".
> >> +	 *
> >> +	 * Any movement of task after being marked not blocked will be
> >> +	 * correctly captured as queued time at enqueue + dequeue
> >> +	 * boundaries when "t->is_blocked" cleared.
> >> +	 */
> >> +	if (!t->sched_info.last_queued || t->is_blocked)
> >>  		t->sched_info.last_queued = rq_clock(rq);
> >>  }
> >>  
> >> ---
> >>
> >> That "t->is_blocked" check in sched_info_enqueue() may not be
> >> necessary either but I've left it in there for now since I did my
> >> quick testing with it.
> > 
> > 
> > Hi Prateek,
> > 
> > IMO too doing this with is_blocked would be cleaner.
> > 
> > But I think this not works on a case where a donor is in a different CPU
> > from its owner, like:
> > 
> > A holds the mutex M in CPU0, B is the waiter and its on CPU1.
> > 
> > proxy_migrate_task() gets B to CPU0.
> >> After that, A suddenly wants to goto CPU1. And it goes.
> > Now B goes back to its home CPU next to A.
> > 
> > Now in here, timer starts.
> > 
> > And suddenly A drops the mutex. Now B is not is_blocked.
> > 
> > Because B is back to its home CPU, there is no dequeue.
> > Even if there was B is not is_blocked so timer wouldn't be reset.
> > 
> > Normally we need to add the time that task waited for itself.
> > But in this case we add the time task waited for its mutex + for itself.
> > 
> > Or am I missssssing something?
> 
> Indeed. That is possible and can probably be fixed by doing a
> sched_info_enqueue() in proxy_needs_return() for
> task_cpu(p) == p->wake_cpu case like:
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 04b62c16fa8c..3d261a4e5b0b 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3838,8 +3838,11 @@ static inline bool proxy_needs_return(struct rq *rq, struct task_struct *p)
>  	 * will not apply. But if it did, this check is the safe way around
>  	 * and would migrate.
>  	 */
> -	if (task_cpu(p) == p->wake_cpu)
> +	if (task_cpu(p) == p->wake_cpu) {
> +		/* Begin wait time accounting before marking !is_blocked. */
> +		sched_info_enqueue(rq, p);
>  		return false;
> +	}
>  
>  	scoped_guard(raw_spinlock, &p->blocked_lock) {
>  		/* Task is waking up; clear any blocked_on relationship */
> ---
> 
> Not sure if this solution is ugly or not for calling
> sched_info_enqueue() out of the enqueue path :-)

I think this fixes the issue. BTW I think this should skip curr, no?
On a case like this:

C holds M1 in CPU0, D waits that mutex in CPU1. A holds mutex M2 in
CPU2. B is the working task on CPU1 but it wants mutex M2 and we enter
__schedule(). B gets is_blocked. D gets chosen. We open the rq lock to
get D to CPU0. While this happens A drops the mutex and wakes up B.
we enter ttwu_runnable() and B is still curr. And we start the timer.

Or am I MISSSING something?




> 
> -- 
> Thanks and Regards,
> Prateek
> 

Thanks,

Kayra

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[K Prateek Nayak]

On 10/27/2026 12:22 PM, Kayra Cizmeci wrote:
> I think this fixes the issue. BTW I think this should skip curr, no?
> On a case like this:
> 
> C holds M1 in CPU0, D waits that mutex in CPU1. A holds mutex M2 in
> CPU2. B is the working task on CPU1 but it wants mutex M2 and we enter
> __schedule(). B gets is_blocked. D gets chosen. We open the rq lock to
> get D to CPU0. While this happens A drops the mutex and wakes up B.
> we enter ttwu_runnable() and B is still curr. And we start the timer.

Good catch that!

> Or am I MISSSING something?

I can't believe something like wait time accounting can get this
complicated! Anyhow, We can perhaps call sched_info_enqueue() in
ttwu_runnable() instead of proxy_needs_return() like:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 04b62c16fa8c..d5758b78b741 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3956,6 +3956,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
 		 * it should preempt the task that is current now.
 		 */
 		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
+		/* Start accounting wait time. */
+		sched_info_enqueue(rq, p);
 	}
 	ttwu_do_wakeup(p);
 	return 1;
---

-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Tue, Oct 27, 2026 at 03:15:40PM +0530, K Prateek Nayak wrote:
> On 10/27/2026 12:22 PM, Kayra Cizmeci wrote:
> > I think this fixes the issue. BTW I think this should skip curr, no?
> > On a case like this:
> > 
> > C holds M1 in CPU0, D waits that mutex in CPU1. A holds mutex M2 in
> > CPU2. B is the working task on CPU1 but it wants mutex M2 and we enter
> > __schedule(). B gets is_blocked. D gets chosen. We open the rq lock to
> > get D to CPU0. While this happens A drops the mutex and wakes up B.
> > we enter ttwu_runnable() and B is still curr. And we start the timer.
> 
> Good catch that!
> 
> > Or am I MISSSING something?
> 
> I can't believe something like wait time accounting can get this
> complicated! Anyhow, We can perhaps call sched_info_enqueue() in

TBH, me neither :-).

> ttwu_runnable() instead of proxy_needs_return() like:

I was thinking about that too! LGTM. FWIW, this depends
on is_blocked check in sched_info_enqueue().

Wei, what do you think? Will this work for you?

> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 04b62c16fa8c..d5758b78b741 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3956,6 +3956,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
>  		 * it should preempt the task that is current now.
>  		 */
>  		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
> +		/* Start accounting wait time. */
> +		sched_info_enqueue(rq, p);
>  	}
>  	ttwu_do_wakeup(p);
>  	return 1;
> ---
> 
> -- 
> Thanks and Regards,
> Prateek
> 

Thanks!

Kayra

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[K Prateek Nayak]

On 10/27/2026 3:23 PM, Kayra Cizmeci wrote:
> Wei, what do you think? Will this work for you?

Full diff for convenience:

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 04b62c16fa8c..4b99ef5b3b2d 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3956,6 +3956,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
 		 * it should preempt the task that is current now.
 		 */
 		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
+		/* Start accounting wait time. */
+		sched_info_enqueue(rq, p);
 	}
 	ttwu_do_wakeup(p);
 	return 1;
diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
index ebe0a7765f98..c4c624deb25b 100644
--- a/kernel/sched/stats.h
+++ b/kernel/sched/stats.h
@@ -239,6 +239,15 @@ static inline void psi_account_irqtime(struct rq *rq, struct task_struct *curr,
 static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
 {
 	unsigned long long delta = 0;
+	/*
+	 * If a task is blocked (blocked donor, delayed dequeue task),
+	 * do not account the queued time as runnable. A task that is
+	 * blocking would have already cleared the "last_queued"
+	 * indicator from sched_info_arrive() when it got a chance to
+	 * run on the CPU.
+	 */
+	if (t->is_blocked)
+		t->sched_info.last_queued = 0;
 
 	if (!t->sched_info.last_queued)
 		return;
@@ -290,7 +299,23 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
  */
 static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
 {
-	if (!t->sched_info.last_queued)
+	/*
+	 * "t->is_blocked" is only cleared after the enqueue is done and
+	 * the task is woken up. If the task is blocked, update the
+	 * "last_queued" timestamp to that of last enqueue - if the task
+	 * is still blocked while it leaves the runqueue, this timestamp
+	 * is ignored and is cleared at sched_info_dequeue().
+	 *
+	 * To transition a task back to blocked after wakeup, it has to
+	 * run on the CPU and naturally reach a safe preemption point.
+	 * sched_info_arrive() will do the correct wait time accounting
+	 * based on the last "last_queued".
+	 *
+	 * Any movement of task after being marked not blocked will be
+	 * correctly captured as queued time at enqueue + dequeue
+	 * boundaries when "t->is_blocked" cleared.
+	 */
+	if (!t->sched_info.last_queued || t->is_blocked)
 		t->sched_info.last_queued = rq_clock(rq);
 }
 
-- 
Thanks and Regards,
Prateek

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 03:15:40PM +0530, K Prateek Nayak wrote:
> On 10/27/2026 12:22 PM, Kayra Cizmeci wrote:
> > I think this fixes the issue. BTW I think this should skip curr, no?
> > On a case like this:
> > 
> > C holds M1 in CPU0, D waits that mutex in CPU1. A holds mutex M2 in
> > CPU2. B is the working task on CPU1 but it wants mutex M2 and we enter
> > __schedule(). B gets is_blocked. D gets chosen. We open the rq lock to
> > get D to CPU0. While this happens A drops the mutex and wakes up B.
> > we enter ttwu_runnable() and B is still curr. And we start the timer.
> 
> Good catch that!
> 
> > Or am I MISSSING something?
> 
> I can't believe something like wait time accounting can get this
> complicated! 

:-)

> Anyhow, We can perhaps call sched_info_enqueue() in
> ttwu_runnable() instead of proxy_needs_return() like:
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 04b62c16fa8c..d5758b78b741 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3956,6 +3956,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
>  		 * it should preempt the task that is current now.
>  		 */
>  		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
> +		/* Start accounting wait time. */
> +		sched_info_enqueue(rq, p);
>  	}
>  	ttwu_do_wakeup(p);
>  	return 1;

So the normal wakeup path goes through enqueue_task() which gets the
sched_info_enqueue() callback there. But yes, ttwu_runnable() does not
go through that, and needs this additional call.

There is one more site in ttwu() that also doesn't go through
enqueue_task() and that is the p==current thing at the top.

Re: [PATCH] sched/stats: Fix run_delay over-count for proxy-migrated blocked donors

[Kayra Cizmeci]

On Tue, Oct 27, 2026 at 01:09:45PM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2026 at 03:15:40PM +0530, K Prateek Nayak wrote:
> > On 10/27/2026 12:22 PM, Kayra Cizmeci wrote:
> > > I think this fixes the issue. BTW I think this should skip curr, no?
> > > On a case like this:
> > > 
> > > C holds M1 in CPU0, D waits that mutex in CPU1. A holds mutex M2 in
> > > CPU2. B is the working task on CPU1 but it wants mutex M2 and we enter
> > > __schedule(). B gets is_blocked. D gets chosen. We open the rq lock to
> > > get D to CPU0. While this happens A drops the mutex and wakes up B.
> > > we enter ttwu_runnable() and B is still curr. And we start the timer.
> > 
> > Good catch that!
> > 
> > > Or am I MISSSING something?
> > 
> > I can't believe something like wait time accounting can get this
> > complicated! 
> 
> :-)
> 
> > Anyhow, We can perhaps call sched_info_enqueue() in
> > ttwu_runnable() instead of proxy_needs_return() like:
> > 
> > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > index 04b62c16fa8c..d5758b78b741 100644
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -3956,6 +3956,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
> >  		 * it should preempt the task that is current now.
> >  		 */
> >  		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
> > +		/* Start accounting wait time. */
> > +		sched_info_enqueue(rq, p);
> >  	}
> >  	ttwu_do_wakeup(p);
> >  	return 1;
> 
> So the normal wakeup path goes through enqueue_task() which gets the
> sched_info_enqueue() callback there. But yes, ttwu_runnable() does not
> go through that, and needs this additional call.
> 
> There is one more site in ttwu() that also doesn't go through
> enqueue_task() and that is the p==current thing at the top.

The counter already measures how much time a task waits for CPU,
curr is already is at the CPU so we don't need a call from there.

Or not?

Thanks,

Kayra