[PATCH v2] sched/stats: Fix run_delay over-count for tasks parked on the runqueue while blocked

[PATCH v2] sched/stats: Fix run_delay over-count for tasks parked on the runqueue while blocked

[albin_yang]

From: Wei Yang <albinwyang@tencent.com>

A task parked on the runqueue while blocked -- a proxy-exec donor migrated
to its mutex owner's CPU, or a DELAY_DEQUEUE sleeper migrated across CPUs --
is re-attached with activate_task(rq, p, ENQUEUE_NOCLOCK), i.e. without
ENQUEUE_RESTORE. That arms sched_info.last_queued although the task is not
waiting for a CPU, and sched_info_dequeue() charges the parked time as
run-queue wait when proxy_needs_return() dequeues the donor (TASK_WAKING)
on wakeup.

Use p->is_blocked as the primary indicator: sched_info_dequeue() drops
last_queued without charging while blocked; sched_info_enqueue() refreshes
it on every enqueue while blocked, so the stamp ends up at the real wakeup;
ttwu_runnable() adds an explicit sched_info_enqueue() call under the
existing !task_on_cpu() guard, for wakeups that have no enqueue of their
own. Delayed sleepers also have is_blocked set, so both cases are covered.

The p == current early return in try_to_wake_up() now documents why it
needs no sched_info_enqueue() call.

Both over-counts disappear with this patch, and run_delay still grows
normally for a runnable task contending for the CPU, so the counter is
alive rather than switched off.

Fixes: b049b81bdff6 ("sched: Handle blocked-waiter migration (and return migration)")
Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Suggested-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Signed-off-by: Wei Yang <albinwyang@tencent.com>
---

v2: switch from the v1 !task_is_blocked() gate to the is_blocked based
approach proposed by Prateek Nayak. Subject widened: v1 only mentioned
proxy-migrated donors, while the same gating fixes both cases.

Measured with two CPUs used (CPU0/CPU1); run_delay is field 2 of
/proc/<tid>/schedstat:

  proxy exec, donor migrated to the owner's CPU (~170 ms block)
    unpatched       167.07 - 167.09 ms   5/5   migrated
    patched           0.00 ms           20/20  migrated
    same-cpu control  0.00 ms            0/5   not migrated

  DELAY_DEQUEUE, sleeper migrated across CPUs (~150 ms sleep)
    unpatched       147.94 - 147.99 ms   5/20
    patched           0.00 ms            0/100

The DELAY_DEQUEUE reproducer self-checks (must reproduce unpatched) and
shows ~40 ms of run_delay growth per round throughout.

v1: https://lore.kernel.org/all/20261025130740.3324391-1-albin_yang@163.com/
Full test results and the discussion of the split:
https://lore.kernel.org/all/20261101143101.353033-1-albin_yang@163.com/

p->is_blocked was added by 4c2a20413d7f ("sched: Add is_blocked task flag")
and is only available from v7.2-rc1. The DELAY_DEQUEUE over-count goes back
to v6.12 (152e11f6df29) and still needs the sched_delayed based variant on
older stable trees; that is a separate, independent fix. A second Fixes:
tag would make the stable backport machinery try this on v6.12.y, where
is_blocked does not exist. Hence only one Fixes: tag above.
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1a088d4576f5..bb7914b713b1 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3962,6 +3962,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
 		 * it should preempt the task that is current now.
 		 */
 		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
+		/* Start accounting wait time. */
+		sched_info_enqueue(rq, p);
 	}
 	ttwu_do_wakeup(p);
 	return 1;
@@ -4357,6 +4359,11 @@ int try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
 		if (!ttwu_state_match(p, state, &success))
 			goto out;
 
+		/*
+		 * No sched_info_enqueue() here: current is running, not
+		 * waiting for a CPU. If preempted, sched_info_switch() ->
+		 * sched_info_depart() arms the wait clock instead.
+		 */
 		trace_sched_waking(p);
 		ttwu_do_wakeup(p);
 		goto out;
diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
index ebe0a7765f98..84d1fb208740 100644
--- a/kernel/sched/stats.h
+++ b/kernel/sched/stats.h
@@ -240,6 +240,9 @@ static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
 {
 	unsigned long long delta = 0;
 
+	if (t->is_blocked)
+		t->sched_info.last_queued = 0;
+
 	if (!t->sched_info.last_queued)
 		return;
 
@@ -290,7 +293,7 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
  */
 static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
 {
-	if (!t->sched_info.last_queued)
+	if (!t->sched_info.last_queued || t->is_blocked)
 		t->sched_info.last_queued = rq_clock(rq);
 }
 
-- 
2.43.7

Re: [PATCH v2] sched/stats: Fix run_delay over-count for tasks parked on the runqueue while blocked

[Kayra Cizmeci]

Hi Wei!

On Sat, Nov 14, 2026 at 05:26:37PM +0800, albin_yang@163.com wrote:
> From: Wei Yang <albinwyang@tencent.com>
> 
> A task parked on the runqueue while blocked -- a proxy-exec donor migrated
> to its mutex owner's CPU, or a DELAY_DEQUEUE sleeper migrated across CPUs --
> is re-attached with activate_task(rq, p, ENQUEUE_NOCLOCK), i.e. without
> ENQUEUE_RESTORE. That arms sched_info.last_queued although the task is not
> waiting for a CPU, and sched_info_dequeue() charges the parked time as
> run-queue wait when proxy_needs_return() dequeues the donor (TASK_WAKING)
> on wakeup.
> 
> Use p->is_blocked as the primary indicator: sched_info_dequeue() drops
> last_queued without charging while blocked; sched_info_enqueue() refreshes
> it on every enqueue while blocked, so the stamp ends up at the real wakeup;
> ttwu_runnable() adds an explicit sched_info_enqueue() call under the
> existing !task_on_cpu() guard, for wakeups that have no enqueue of their
> own. Delayed sleepers also have is_blocked set, so both cases are covered.
> 
> The p == current early return in try_to_wake_up() now documents why it
> needs no sched_info_enqueue() call.
> 
> Both over-counts disappear with this patch, and run_delay still grows
> normally for a runnable task contending for the CPU, so the counter is
> alive rather than switched off.
> 
> Fixes: b049b81bdff6 ("sched: Handle blocked-waiter migration (and return migration)")
> Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com>
> Suggested-by: Kayra Cizmeci <kayracizmeci@gmail.com>
> Signed-off-by: Wei Yang <albinwyang@tencent.com>
> ---

[snip...]

> v1: https://lore.kernel.org/all/20261025130740.3324391-1-albin_yang@163.com/
> Full test results and the discussion of the split:
> https://lore.kernel.org/all/20261101143101.353033-1-albin_yang@163.com/
> 
> p->is_blocked was added by 4c2a20413d7f ("sched: Add is_blocked task flag")
> and is only available from v7.2-rc1. The DELAY_DEQUEUE over-count goes back
> to v6.12 (152e11f6df29) and still needs the sched_delayed based variant on
> older stable trees; that is a separate, independent fix. A second Fixes:
> tag would make the stable backport machinery try this on v6.12.y, where
> is_blocked does not exist. Hence only one Fixes: tag above.
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 1a088d4576f5..bb7914b713b1 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -3962,6 +3962,8 @@ static int ttwu_runnable(struct task_struct *p, int wake_flags)
>  		 * it should preempt the task that is current now.
>  		 */
>  		wakeup_preempt(rq, p, wake_flags | WF_TTWU_RQ);
> +		/* Start accounting wait time. */
> +		sched_info_enqueue(rq, p);
>  	}
>  	ttwu_do_wakeup(p);
>  	return 1;
> @@ -4357,6 +4359,11 @@ int try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
>  		if (!ttwu_state_match(p, state, &success))
>  			goto out;
>  
> +		/*
> +		 * No sched_info_enqueue() here: current is running, not
> +		 * waiting for a CPU. If preempted, sched_info_switch() ->
> +		 * sched_info_depart() arms the wait clock instead.
> +		 */
>  		trace_sched_waking(p);
>  		ttwu_do_wakeup(p);
>  		goto out;
> diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h
> index ebe0a7765f98..84d1fb208740 100644
> --- a/kernel/sched/stats.h
> +++ b/kernel/sched/stats.h
> @@ -240,6 +240,9 @@ static inline void sched_info_dequeue(struct rq *rq, struct task_struct *t)
>  {
>  	unsigned long long delta = 0;
>  
> +	if (t->is_blocked)
> +		t->sched_info.last_queued = 0;
> +
>  	if (!t->sched_info.last_queued)
>  		return;
>  
> @@ -290,7 +293,7 @@ static void sched_info_arrive(struct rq *rq, struct task_struct *t)
>   */
>  static inline void sched_info_enqueue(struct rq *rq, struct task_struct *t)
>  {
> -	if (!t->sched_info.last_queued)
> +	if (!t->sched_info.last_queued || t->is_blocked)
>  		t->sched_info.last_queued = rq_clock(rq);
>  }
>  
> -- 
> 2.43.7
> 

So I see you removed some of the comments from Prateek's diff.
Is there a reason or you just forgot to add them? :-).

The diff is here:
https://lore.kernel.org/all/4c8b0413-2ade-447f-9313-ff51dca7c7c4@amd.com/

And what is your plan for v4?


Cheers,

Kayra