[PATCH v5 0/2] sched: update the rq->avg_idle when a task is moved to an idle CPU

[PATCH v5 0/2] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Huang Shijie]

In the newidle balance, the rq->idle_stamp may set to a non-zero value
if it cannot pull any task.

In the wakeup, it will detect the rq->idle_stamp, and updates
the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
to zero.

Besides the wakeup, current code does not end the CPU idle status
when a task is moved to the idle CPU, such as fork/clone, execve,
or other cases.

This patch set tries to resolve it.

v4--> v5:
    Modify the changelog.

   --v4:https://lkml.org/lkml/2025/11/28/300

v3--> v4:
     Remove the code for delayed task.

   --v3: https://lkml.org/lkml/2025/11/27/456

v2--> v3:
  -- merge patch 3 into patch 2:
      move update_rq_avg_idle() to enqueue_task().

   --v2: https://lkml.org/lkml/2025/11/27/214   

v1--> v2:
  -- Put update_rq_avg_idle() to activate_task()
  -- Add Delay-dequeue task check.	

   --v1: https://lkml.org/lkml/2025/11/24/97


Huang Shijie (2):
  sched/fair: set rq->idle_stamp at the end of the sched_balance_newidle
  sched: update the rq->avg_idle when a task is moved to an idle CPU

 kernel/sched/core.c | 29 +++++++++++++++++------------
 kernel/sched/fair.c | 12 ++++++++----
 2 files changed, 25 insertions(+), 16 deletions(-)

-- 
2.40.1

[PATCH v5 1/2] sched/fair: set rq->idle_stamp at the end of the sched_balance_newidle

[Huang Shijie]

In current newidle balance, the rq->idle_stamp may set to a non-zero value
if it cannot pull any task.

In the wakeup, it will detect the rq->idle_stamp, and updates
the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
to zero.

Besides the wakeup, current code does not end the CPU idle status
when a task is moved to the idle CPU, such as fork/clone, execve,
or other cases.

In order to fix this issue, we want to add a hook(update_rq_avg_idle())
in the enqueue_task(). With this hook, if a task is moved to the idle CPU,
it will update the rq->avg_idle. Unfortunately, this hook is also called
in the newidle balance:
   sched_balance_newidle() --> sched_balance_rq() --> .. --> enqueue_task()

If we still set rq->idle_stamp at the beginning of sched_balance_newidle(),
the rq->avg_idle will not be updated correctly.

In order to make it work correctly, save the idle_stamp at the beginning
of sched_balance_newidle(). If newidle balance cannot pull any task,
set the saved value for rq->idle_stamp. With this method,
the newidle balance still work correctly, and the hook in enqueue_task()
also works correctly.

Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com>
---
 kernel/sched/fair.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 1855975b8248..cfdd22e5dcab 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12865,6 +12865,7 @@ static int sched_balance_newidle(struct rq *this_rq, struct rq_flags *rf)
 	u64 t0, t1, curr_cost = 0;
 	struct sched_domain *sd;
 	int pulled_task = 0;
+	u64 idle_stamp;
 
 	update_misfit_status(NULL, this_rq);
 
@@ -12880,7 +12881,9 @@ static int sched_balance_newidle(struct rq *this_rq, struct rq_flags *rf)
 	 * for CPU_NEWLY_IDLE, such that we measure the this duration
 	 * as idle time.
 	 */
-	this_rq->idle_stamp = rq_clock(this_rq);
+	idle_stamp = rq_clock(this_rq);
+
+	this_rq->idle_stamp = 0;
 
 	/*
 	 * Do not pull tasks towards !active CPUs...
@@ -12992,10 +12995,11 @@ static int sched_balance_newidle(struct rq *this_rq, struct rq_flags *rf)
 	if (time_after(this_rq->next_balance, next_balance))
 		this_rq->next_balance = next_balance;
 
-	if (pulled_task)
-		this_rq->idle_stamp = 0;
-	else
+	if (!pulled_task) {
+		/* Set it here on purpose. */
+		this_rq->idle_stamp = idle_stamp;
 		nohz_newidle_balance(this_rq);
+	}
 
 	rq_repin_lock(this_rq, rf);
 
-- 
2.40.1

[PATCH v5 2/2] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Huang Shijie]

In the newidle balance, the rq->idle_stamp may set to a non-zero value
if it cannot pull any task.

In the wakeup, it will detect the rq->idle_stamp, and updates
the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
to zero.

Besides the wakeup, current code does not end the CPU idle status
when a task is moved to the idle CPU, such as fork/clone, execve,
or other cases. In order to get more accurate rq->avg_idle,
we need to update it at more places(not only the wakeup).

This patch introduces a helper: update_rq_avg_idle().
And uses it in enqueue_task(), so it will update the rq->avg_idle
when a task is moved to an idle CPU at:
   -- wakeup
   -- fork/clone
   -- execve
   -- idle balance
   -- other cases

Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com>
---
 kernel/sched/core.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9f10cfbdc228..2e3c4043de51 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2078,6 +2078,21 @@ unsigned long get_wchan(struct task_struct *p)
 	return ip;
 }
 
+static void update_rq_avg_idle(struct rq *rq)
+{
+	if (rq->idle_stamp) {
+		u64 delta = rq_clock(rq) - rq->idle_stamp;
+		u64 max = 2*rq->max_idle_balance_cost;
+
+		update_avg(&rq->avg_idle, delta);
+
+		if (rq->avg_idle > max)
+			rq->avg_idle = max;
+
+		rq->idle_stamp = 0;
+	}
+}
+
 void enqueue_task(struct rq *rq, struct task_struct *p, int flags)
 {
 	if (!(flags & ENQUEUE_NOCLOCK))
@@ -2100,6 +2115,8 @@ void enqueue_task(struct rq *rq, struct task_struct *p, int flags)
 
 	if (sched_core_enabled(rq))
 		sched_core_enqueue(rq, p);
+
+	update_rq_avg_idle(rq);
 }
 
 /*
@@ -3645,18 +3662,6 @@ ttwu_do_activate(struct rq *rq, struct task_struct *p, int wake_flags,
 		p->sched_class->task_woken(rq, p);
 		rq_repin_lock(rq, rf);
 	}
-
-	if (rq->idle_stamp) {
-		u64 delta = rq_clock(rq) - rq->idle_stamp;
-		u64 max = 2*rq->max_idle_balance_cost;
-
-		update_avg(&rq->avg_idle, delta);
-
-		if (rq->avg_idle > max)
-			rq->avg_idle = max;
-
-		rq->idle_stamp = 0;
-	}
 }
 
 /*
-- 
2.40.1

Re: [PATCH v5 0/2] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Dietmar Eggemann]

On 03.12.25 06:30, Huang Shijie wrote:
> In the newidle balance, the rq->idle_stamp may set to a non-zero value
> if it cannot pull any task.
> 
> In the wakeup, it will detect the rq->idle_stamp, and updates
> the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
> to zero.
> 
> Besides the wakeup, current code does not end the CPU idle status
> when a task is moved to the idle CPU, such as fork/clone, execve,
> or other cases.

Is there a case which is the most important one? (e.g. idle load balancing)?

I still can't see the reason why this code should be changed (was asked
in v4 already).

Do you have a workload on your machine which gives you better scores in
case you set 'rq->idle_stamp = 0' in all those cases?

What is is actual issue in the current code?

Does 'this_rq->avg_idle' get too big so you spend too much useless time
(since pulled_task == 0) in sched_balance_newidle() because the bailout
condition:

sched_balance_newidle()

  ...
  if (!get_rd_overloaded(this_rq->rd) ||
      this_rq->avg_idle < sd->max_newidle_lb_cost) {
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    ...
    goto out;

doesn't trigger anymore?

[...]

Re: [PATCH v5 0/2] sched: update the rq->avg_idle when a task is moved to an idle CPU

[Shijie Huang]

On 04/12/2025 21:09, Dietmar Eggemann wrote:
> On 03.12.25 06:30, Huang Shijie wrote:
>> In the newidle balance, the rq->idle_stamp may set to a non-zero value
>> if it cannot pull any task.
>>
>> In the wakeup, it will detect the rq->idle_stamp, and updates
>> the rq->avg_idle, then ends the CPU idle status by setting rq->idle_stamp
>> to zero.
>>
>> Besides the wakeup, current code does not end the CPU idle status
>> when a task is moved to the idle CPU, such as fork/clone, execve,
>> or other cases.
> Is there a case which is the most important one? (e.g. idle load balancing)?
>
> I still can't see the reason why this code should be changed (was asked
> in v4 already).

I noticed that the rq->idle_stamp is not zero when rq->idle_stamp is set in

begining of sched_balance_newidle(). It is a strange behavior.

then I found a new task(clone/fork, etc) may be move to this rq while 
this rq->idle_stamp

is still not zero. it triggers the second newidle on this CPU.


>
> Do you have a workload on your machine which gives you better scores in
> case you set 'rq->idle_stamp = 0' in all those cases?

I tested the specjbb, the performance does not change.

Btw, the "rq->idle_stamp = 0" is redandent, we can remove it in patch 1.


> What is is actual issue in the current code?

the current code is confusing to me :)

Current  rq->idle_stamp is equal to rq->wakeup_idle_stamp.


>
> Does 'this_rq->avg_idle' get too big so you spend too much useless time
> (since pulled_task == 0) in sched_balance_newidle() because the bailout
> condition:
>
> sched_balance_newidle()
>
>    ...
>    if (!get_rd_overloaded(this_rq->rd) ||
>        this_rq->avg_idle < sd->max_newidle_lb_cost) {
>        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>      ...
>      goto out;
>
> doesn't trigger anymore?

Not this issue.


Thanks

Huang Shijie

>
> [...]