[PATCH v5 0/4] sched: Handle split scheduling and execution contexts in task ticks

[PATCH v5 0/4] sched: Handle split scheduling and execution contexts in task ticks

[Hui Su]

Proxy execution separates the task selected for scheduling from the task
whose execution state advances. This series dispatches scheduler ticks to
both contexts, keeps NUMA and cache work tied to the execution task, and
accounts core-scheduling slice service across same-donor execution-owner
handoffs.

Scheduling-class policy work remains donor-owned, while execution-context
consumers use rq->curr. The core-scheduling fix keeps a per-runqueue donor
service baseline so a synthetic same-donor proxy reselect preserves service
already consumed on behalf of the donor.

Link: https://lore.kernel.org/r/20260909092901.2989564-1-sh_def@163.com

Changes since v4:

- Rework FAIR task-tick handling into one donor-owned block followed by one
  execution-context block, with NUMA and cache work in the latter.
- Keep task_tick_scx() donor-gated pending the separate sched_ext/proxy
  execution integration.
- Preserve the donor core-slice baseline across same-donor synthetic
  reselects, while resetting it for a new donor or explicit set-next
  reactivation.
- Drop the RT watchdog patch from this series; continue that work separately.

Validation:

- x86_64 vmlinux/modules/bzImage builds passed with the main configuration
  and CONFIG_SCHED_PROXY_EXEC=n, CONFIG_SCHED_CORE=n,
  CONFIG_SCHED_CACHE=n, CONFIG_FAIR_GROUP_SCHED=n,
  CONFIG_NUMA_BALANCING=y, and CONFIG_SCHED_CLASS_EXT=y variants.
  sched_ext selftests also built successfully with the SCX-enabled kernel.
- x86_64 QEMU boot smoke and feature-off boot tests: PASS.
- Exact-head two-node NUMA/proxy smoke: PASS.
- Previous validation of the unchanged patch content covers scheduler
  selftests, NO_HZ_FULL proxy smoke, and SMT core-scheduling behavior.
- A focused same-donor CORE-01 test split service across two execution
  owners. Each individual portion stayed below the force-idle threshold,
  while accumulated donor service crossed it with one core_sched_start
  baseline.

No mixed sched_ext/proxy runtime was run because the configurations are
currently mutually exclusive. Runtime validation was performed on x86_64
only; no 32-bit or ARM runtime validation was performed for this revision.

Hui Su (4):
  sched: Dispatch task ticks for donor and execution classes
  sched/numa: Drive NUMA task tick from execution context
  sched/cache: Drive cache task tick from execution context
  sched/core: Fix donor slice accounting under proxy execution

 kernel/sched/core.c      | 22 ++++++++++--
 kernel/sched/deadline.c  | 10 ++++--
 kernel/sched/ext/ext.c   | 22 +++++++++---
 kernel/sched/fair.c      | 77 ++++++++++++++++++++++++----------------
 kernel/sched/idle.c      |  8 ++---
 kernel/sched/rt.c        | 13 ++++---
 kernel/sched/sched.h     |  4 ++-
 kernel/sched/stop_task.c |  5 ++-
 8 files changed, 108 insertions(+), 53 deletions(-)


base-commit: cba2348ab114391f5b1a00fa65c5b739f13f0563
-- 
2.55.0

[PATCH v5 1/4] sched: Dispatch task ticks for donor and execution classes

[Hui Su]

Proxy execution can run a task from one scheduling class on behalf of a
donor from another. Scheduler ticks therefore need to reach both the class
which owns the scheduling context and the class which owns the execution
context.

Remove the task argument from sched_class::task_tick() and let each class
select the state it owns through the runqueue. Add a common task_tick()
dispatcher which calls the donor class first and, when proxy execution
splits the classes, the execution class afterwards.

Calling the donor first preserves the existing runtime-accounting order for
execution-context consumers. Keep the existing class-specific tick behavior
donor-gated in this patch, so the change only introduces the new interface
and dispatch mechanism.

In particular, keep task_tick_scx() explicitly gated on the donor class.
If the callback is reached only because the execution context belongs to
sched_ext while another class supplies the donor, it must not perform SCX
scheduling-context work. This aligns the dispatcher with the ownership
direction used by the pending sched_ext proxy-execution work. Full
sched_ext/proxy-execution integration is not enabled by the current
Kconfig; that accounting conversion remains outside this patch.

Suggested-by: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20260908104407.GD687043@noisy.programming.kicks-ass.net
Link: https://lore.kernel.org/r/aqGa0J1_LM99oDkP@gpd4
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/core.c      | 22 +++++++++++++++++++---
 kernel/sched/deadline.c  | 10 +++++++---
 kernel/sched/ext/ext.c   | 22 +++++++++++++++++-----
 kernel/sched/fair.c      | 25 +++++++++++++++----------
 kernel/sched/idle.c      |  8 ++++----
 kernel/sched/rt.c        | 13 +++++++++----
 kernel/sched/sched.h     |  2 +-
 kernel/sched/stop_task.c |  5 ++---
 8 files changed, 74 insertions(+), 33 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b998ef6b87af..05e599665fdd 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -892,6 +892,22 @@ void update_rq_clock(struct rq *rq)
 	update_rq_clock_task(rq, delta);
 }
 
+/*
+ * Run the scheduling-context class first so its runtime update precedes
+ * execution-context tick work. A different execution class runs second.
+ * Same-class proxy execution gets one callback; ownership-specific work
+ * can select rq->donor or rq->curr as appropriate.
+ */
+static inline void task_tick(struct rq *rq, int queued)
+{
+	const struct sched_class *curr_class = rq->curr->sched_class;
+	const struct sched_class *donor_class = rq->donor->sched_class;
+
+	donor_class->task_tick(rq, queued);
+	if (sched_proxy_exec() && curr_class != donor_class)
+		curr_class->task_tick(rq, queued);
+}
+
 #ifdef CONFIG_SCHED_HRTICK
 /*
  * Use HR-timers to deliver accurate preemption points.
@@ -923,7 +939,7 @@ static enum hrtimer_restart hrtick(struct hrtimer *timer)
 
 	rq_lock(rq, &rf);
 	update_rq_clock(rq);
-	rq->donor->sched_class->task_tick(rq, rq->donor, 1);
+	task_tick(rq, 1);
 	rq_unlock(rq, &rf);
 
 	return HRTIMER_NORESTART;
@@ -5799,7 +5815,7 @@ void sched_tick(void)
 	if (dynamic_preempt_lazy() && tif_test_bit(TIF_NEED_RESCHED_LAZY))
 		resched_curr(rq);
 
-	donor->sched_class->task_tick(rq, donor, 0);
+	task_tick(rq, 0);
 	if (sched_feat(LATENCY_WARN))
 		resched_latency = cpu_resched_latency(rq);
 	calc_global_load_tick(rq);
@@ -5895,7 +5911,7 @@ static void sched_tick_remote(struct work_struct *work)
 				u64 delta = rq_clock_task(rq) - curr->se.exec_start;
 				WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 30);
 			}
-			curr->sched_class->task_tick(rq, curr, 0);
+			task_tick(rq, 0);
 
 			calc_load_nohz_remote(rq);
 		}
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 0663c00c41c0..da7613acab18 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -2873,11 +2873,15 @@ static void put_prev_task_dl(struct rq *rq, struct task_struct *p, struct task_s
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
- * parameters.
+ * and all state must be accessed through @rq.
  */
-static void task_tick_dl(struct rq *rq, struct task_struct *p, int queued)
+static void task_tick_dl(struct rq *rq, int queued)
 {
+	struct task_struct *p = rq->donor;
+
+	if (p->sched_class != &dl_sched_class)
+		return;
+
 	update_curr_dl(rq);
 
 	update_dl_rq_load_avg(rq_clock_pelt(rq), rq, 1);
diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 51de1d8b72a1..8c389852be6e 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -3789,9 +3789,21 @@ void scx_tick(struct rq *rq)
 	update_other_load_avgs(rq);
 }
 
-static void task_tick_scx(struct rq *rq, struct task_struct *curr, int queued)
+static void task_tick_scx(struct rq *rq, int queued)
 {
-	struct scx_sched *sch = scx_task_sched(curr);
+	struct task_struct *donor = rq->donor;
+	struct scx_sched *sch;
+
+	/*
+	 * task_tick() may also invoke this callback as the execution class
+	 * after dispatching the donor class. SCX scheduling state belongs to
+	 * the scheduling context, so there is nothing to do here unless the
+	 * donor itself belongs to sched_ext.
+	 */
+	if (donor->sched_class != &ext_sched_class)
+		return;
+
+	sch = scx_task_sched(donor);
 
 	update_curr_scx(rq);
 
@@ -3800,11 +3812,11 @@ static void task_tick_scx(struct rq *rq, struct task_struct *curr, int queued)
 	 * management.
 	 */
 	if (scx_bypassing(sch, cpu_of(rq)))
-		scx_set_task_slice(curr, 0);
+		scx_set_task_slice(donor, 0);
 	else if (SCX_HAS_OP(sch, tick))
-		SCX_CALL_OP_TASK(sch, tick, rq, curr);
+		SCX_CALL_OP_TASK(sch, tick, rq, donor);
 
-	if (!curr->scx.slice)
+	if (!donor->scx.slice)
 		resched_curr(rq);
 }
 
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index ade1eceb39b8..6f1777799371 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15057,12 +15057,17 @@ static inline void task_tick_core(struct rq *rq, struct task_struct *curr) {}
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
- * parameters.
+ * and all state must be accessed through @rq.
  */
-static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
+static void task_tick_fair(struct rq *rq, int queued)
 {
-	struct sched_entity *se = &curr->se;
+	struct task_struct *donor = rq->donor;
+	struct sched_entity *se;
+
+	if (donor->sched_class != &fair_sched_class)
+		return;
+
+	se = &donor->se;
 
 	if (se->on_rq) {
 		unsigned long weight = NICE_0_LOAD;
@@ -15075,7 +15080,7 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
 			weight = __calc_prop_weight(cfs_rq, se, weight);
 		}
 
-		se = &curr->se;
+		se = &donor->se;
 		reweight_eevdf(cfs_rq, se, weight, se->on_rq);
 	}
 
@@ -15083,14 +15088,14 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
 		return;
 
 	if (static_branch_unlikely(&sched_numa_balancing))
-		task_tick_numa(rq, curr);
+		task_tick_numa(rq, donor);
 
-	task_tick_cache(rq, curr);
+	task_tick_cache(rq, donor);
 
-	update_misfit_status(curr, rq);
-	check_update_overutilized_status(task_rq(curr));
+	update_misfit_status(donor, rq);
+	check_update_overutilized_status(task_rq(donor));
 
-	task_tick_core(rq, curr);
+	task_tick_core(rq, donor);
 }
 
 /*
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index eb73b65ce6c4..c1e597b0912a 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -532,12 +532,12 @@ dequeue_task_idle(struct rq *rq, struct task_struct *p, int flags)
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
- * parameters.
+ * and all state must be accessed through @rq.
  */
-static void task_tick_idle(struct rq *rq, struct task_struct *curr, int queued)
+static void task_tick_idle(struct rq *rq, int queued)
 {
-	update_curr_idle(rq);
+	if (rq->donor->sched_class == &idle_sched_class)
+		update_curr_idle(rq);
 }
 
 static void switching_to_idle(struct rq *rq, struct task_struct *p)
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index 85303add726d..dd058a6ca06b 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -2535,12 +2535,17 @@ static inline void watchdog(struct rq *rq, struct task_struct *p) { }
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
- * parameters.
+ * and all state must be accessed through @rq.
  */
-static void task_tick_rt(struct rq *rq, struct task_struct *p, int queued)
+static void task_tick_rt(struct rq *rq, int queued)
 {
-	struct sched_rt_entity *rt_se = &p->rt;
+	struct task_struct *p = rq->donor;
+	struct sched_rt_entity *rt_se;
+
+	if (p->sched_class != &rt_sched_class)
+		return;
+
+	rt_se = &p->rt;
 
 	update_curr_rt(rq);
 	update_rt_rq_load_avg(rq_clock_pelt(rq), rq, 1);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index e656c7059bf8..6a8deddc725b 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2718,7 +2718,7 @@ struct sched_class {
 	 * sched_tick: rq->lock
 	 * sched_tick_remote: rq->lock
 	 */
-	void (*task_tick)(struct rq *rq, struct task_struct *p, int queued);
+	void (*task_tick)(struct rq *rq, int queued);
 	/*
 	 * sched_cgroup_fork: p->pi_lock
 	 */
diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c
index c909ca0d8c87..87b46fc74f81 100644
--- a/kernel/sched/stop_task.c
+++ b/kernel/sched/stop_task.c
@@ -68,10 +68,9 @@ static void put_prev_task_stop(struct rq *rq, struct task_struct *prev, struct t
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
- * parameters.
+ * and all state must be accessed through @rq.
  */
-static void task_tick_stop(struct rq *rq, struct task_struct *curr, int queued)
+static void task_tick_stop(struct rq *rq, int queued)
 {
 }
 
-- 
2.55.0

Re: [PATCH v5 1/4] sched: Dispatch task ticks for donor and execution classes

[Kayra Cizmeci]

Hello All, :>

> Proxy execution can run a task from one scheduling class on behalf of a
> donor from another. Scheduler ticks therefore need to reach both the class
> which owns the scheduling context and the class which owns the execution
> context.

So, donor and curr's classes may be different and we should call both of their task_ticks.

> Calling the donor first preserves the existing runtime-accounting order for
> execution-context consumers. Keep the existing class-specific tick behavior
> donor-gated in this patch, so the change only introduces the new interface
> and dispatch mechanism.

Hm..

> +/*
> + * Run the scheduling-context class first so its runtime update precedes
> + * execution-context tick work. A different execution class runs second.
> + * Same-class proxy execution gets one callback; ownership-specific work
> + * can select rq->donor or rq->curr as appropriate.
> + */
> +static inline void task_tick(struct rq *rq, int queued)
> +{
> +	const struct sched_class *curr_class = rq->curr->sched_class;
> +	const struct sched_class *donor_class = rq->donor->sched_class;
> +
> +	donor_class->task_tick(rq, queued);
> +	if (sched_proxy_exec() && curr_class != donor_class)
> +		curr_class->task_tick(rq, queued);
> +}

>  #ifdef CONFIG_SCHED_HRTICK
>  /*
>   * Use HR-timers to deliver accurate preemption points.
> @@ -923,7 +939,7 @@ static enum hrtimer_restart hrtick(struct hrtimer *timer)
>  
>  	rq_lock(rq, &rf);
>  	update_rq_clock(rq);
> -	rq->donor->sched_class->task_tick(rq, rq->donor, 1);
> +	task_tick(rq, 1);
>  	rq_unlock(rq, &rf);

So. Let's assemble a senario.

rq->donor's class method will be DL, and rq->curr's will be fair. And the sched_proxy_exec() will be true.

ON THE OLD:
Only DL runs. That's all :>

ON THE NEW:
task_tick firstly runs donor's method. That's DL. Then we goto curr's class which is fair, and run that.
But because of the newly added checks, because of the donor's sched class not equals to the fair's the code just,
doesn't run. 

Wouldn't it be better to merge this patch and it's use case together?

The commit message is confusing too "Scheduler ticks therefore need to reach both the class
which owns the scheduling context and the class which owns the execution
context." But the patch doesn't do this on it's own. 

Thanks, 
Kayra

Re: [PATCH v5 1/4] sched: Dispatch task ticks for donor and execution classes

[Hui Su]

Hi Kayra,

> Wouldn't it be better to merge this patch and it's use case together?
>
> The commit message is confusing too "Scheduler ticks therefore need to
> reach both the class which owns the scheduling context and the class which
> owns the execution context." But the patch doesn't do this on it's own.

You are right that patch 1 alone does not enable useful execution-context
work in the FAIR callback: the callback is still donor-gated there, so a FAIR
callback reached only as the execution class returns without doing FAIR
scheduling work.

That is intentional. I split the series so that patch 1 is a
behavior-preserving conversion of the sched_class::task_tick() interface and
the common dispatcher:

  - patch 1: generic donor/execution callback dispatch
  - patch 2: move NUMA tick work to the execution context
  - patch 3: move cache tick work to the execution context

Keeping patch 1 donor-gated makes the API and dispatcher change independently
reviewable, while patches 2 and 3 introduce the execution-context users.

I agree that the patch 1 commit message is confusing because its opening
paragraph can read as if patch 1 itself already enables useful
execution-context tick work. I will reword it to make the preparatory nature
explicit, for example:

  This patch adds the common donor/execution task_tick() dispatcher and
  updates the callback interface. Class-specific tick work remains
  donor-owned here; subsequent patches use the new dispatch path for
  execution-context NUMA and cache work.

I would prefer to keep patch 1 separate from patch 2 for now. It keeps the
sched_class::task_tick() interface conversion and cross-class dispatcher
separate from the NUMA semantic fix, and keeps the NUMA and cache fixes as
separate changes with their own Fixes: tags. If there is a preference for
introducing the dispatcher together with its first execution-context user,
I can fold patch 2 into patch 1.

Thanks,
Hui

[PATCH v5 2/4] sched/numa: Drive NUMA task tick from execution context

[Hui Su]

Proxy execution separates the scheduling context in rq->donor from the
execution context in rq->curr. The task tick dispatcher can therefore
invoke task_tick_fair() for either side of a cross-class proxy
relationship.

task_tick_numa() operates on state associated with the task actually
executing, including its mm and NUMA work state. Task-level execution
runtime is likewise accounted to rq->curr, and task_tick_numa() uses that
runtime to drive periodic NUMA scanning.

Order task_tick_fair() as one donor-owned block followed by one
execution-context block. Keep the cache tick in the donor block for this
patch; the following patch moves it with the other execution-context work.
Run the NUMA tick only when rq->curr belongs to the fair scheduling class.
The donor class tick runs first, so execution runtime is accounted before
the NUMA tick consumes it.

Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_proxy_task()")
Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Suggested-by: Tim Chen <tim.c.chen@linux.intel.com>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20260909110348.GZ4120091@noisy.programming.kicks-ass.net
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/fair.c | 46 +++++++++++++++++++++++----------------------
 1 file changed, 24 insertions(+), 22 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 6f1777799371..002456cf9f8d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15061,41 +15061,43 @@ static inline void task_tick_core(struct rq *rq, struct task_struct *curr) {}
  */
 static void task_tick_fair(struct rq *rq, int queued)
 {
-	struct task_struct *donor = rq->donor;
-	struct sched_entity *se;
+	struct task_struct *curr = rq->curr, *donor = rq->donor;
 
-	if (donor->sched_class != &fair_sched_class)
-		return;
+	if (donor->sched_class == &fair_sched_class) {
+		struct sched_entity *se = &donor->se;
 
-	se = &donor->se;
+		if (se->on_rq) {
+			unsigned long weight = NICE_0_LOAD;
+			struct cfs_rq *cfs_rq;
 
-	if (se->on_rq) {
-		unsigned long weight = NICE_0_LOAD;
-		struct cfs_rq *cfs_rq;
+			for_each_sched_entity(se) {
+				cfs_rq = cfs_rq_of(se);
+				entity_tick(cfs_rq, se, queued);
 
-		for_each_sched_entity(se) {
-			cfs_rq = cfs_rq_of(se);
-			entity_tick(cfs_rq, se, queued);
+				weight = __calc_prop_weight(cfs_rq, se, weight);
+			}
 
-			weight = __calc_prop_weight(cfs_rq, se, weight);
+			se = &donor->se;
+			reweight_eevdf(cfs_rq, se, weight, se->on_rq);
 		}
 
-		se = &donor->se;
-		reweight_eevdf(cfs_rq, se, weight, se->on_rq);
+		if (!queued) {
+			task_tick_cache(rq, donor);
+			update_misfit_status(donor, rq);
+			check_update_overutilized_status(task_rq(donor));
+
+			task_tick_core(rq, donor);
+		}
 	}
 
 	if (queued)
 		return;
 
-	if (static_branch_unlikely(&sched_numa_balancing))
-		task_tick_numa(rq, donor);
-
-	task_tick_cache(rq, donor);
-
-	update_misfit_status(donor, rq);
-	check_update_overutilized_status(task_rq(donor));
+	/* Update state owned by the execution context. */
+	if (curr->sched_class == &fair_sched_class &&
+	    static_branch_unlikely(&sched_numa_balancing))
+		task_tick_numa(rq, curr);
 
-	task_tick_core(rq, donor);
 }
 
 /*
-- 
2.55.0

[PATCH v5 3/4] sched/cache: Drive cache task tick from execution context

[Hui Su]

Proxy execution separates the scheduling context in rq->donor from the
execution context in rq->curr. Keep the fair task tick organized as one
donor-owned block followed by one execution-context block.

Task cache accounting is performed for rq->curr by account_mm_sched(), and
cache tick handling must operate on the same execution context. With proxy
execution, keeping task_tick_cache() on the donor can leave the executing
task's cache state and epoch stale.

Move task_tick_cache() into the execution-context section alongside the
NUMA tick. The final layout has one donor-owned block containing entity,
misfit, overutilized and core-scheduling work, followed by one
execution-context block containing NUMA and cache work. Preserve the normal
queued/hrtick gating while attributing each operation to the right context.

Fixes: df0d98475954 ("sched/cache: Introduce infrastructure for cache-aware load balancing")
Suggested-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/fair.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 002456cf9f8d..ed08b287016a 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15082,7 +15082,6 @@ static void task_tick_fair(struct rq *rq, int queued)
 		}
 
 		if (!queued) {
-			task_tick_cache(rq, donor);
 			update_misfit_status(donor, rq);
 			check_update_overutilized_status(task_rq(donor));
 
@@ -15094,10 +15093,12 @@ static void task_tick_fair(struct rq *rq, int queued)
 		return;
 
 	/* Update state owned by the execution context. */
-	if (curr->sched_class == &fair_sched_class &&
-	    static_branch_unlikely(&sched_numa_balancing))
-		task_tick_numa(rq, curr);
+	if (curr->sched_class == &fair_sched_class) {
+		if (static_branch_unlikely(&sched_numa_balancing))
+			task_tick_numa(rq, curr);
 
+		task_tick_cache(rq, curr);
+	}
 }
 
 /*
-- 
2.55.0

[PATCH v5 4/4] sched/core: Fix donor slice accounting under proxy execution

[Hui Su]

Core scheduling uses __entity_slice_used() to decide whether the current
scheduling context has consumed enough of its slice to let a force-idled
SMT sibling run.

The check is correctly made against rq->donor, since the slice belongs to
the scheduling context. With proxy execution, however, task runtime is
accounted to rq->curr. The donor's sum_exec_runtime therefore does not
advance while another task executes on its behalf, causing

	se->sum_exec_runtime - se->prev_sum_exec_runtime

to remain near zero and preventing the force-idle reschedule from
triggering.

Using rq->curr is not correct either, as that would compare the execution
task's runtime against its own slice rather than the donor's slice.

Track the donor's task-clock timestamp when it is selected and measure the
elapsed service using exec_start. update_se() advances the donor's
exec_start from rq_clock_task() even under proxy execution, while the
accumulated task runtime itself is charged to rq->curr.

Keep the selection baseline in struct rq: there is only one active donor
per runqueue. Storing it in every sched_entity needlessly increases the
size of the hot sched_entity structure, especially on 32-bit builds.

Proxy execution can switch rq->curr from one execution owner to another
without changing rq->donor. The scheduler performs a synthetic
put_prev_task()/set_next_task() pair for the unchanged donor in that case.
Preserve the baseline for that reselect, or service accumulated before the
owner handoff would be lost. Refresh it when selecting a different donor
or through an explicit set-next reactivation.

Keeping the comparison in the task-clock domain also avoids depending on
the entity's weight. A vruntime delta accumulated across different weights
cannot reliably be compared against a slice converted using only the
current weight.

Only snapshot task entities, as task_tick_core() performs the consumed
slice check on the donor task.

In non-proxy testing, the task-clock predicate matched the existing
sum_exec_runtime predicate across HZ=100/250/1000 and nice -10/0/+10.
Under proxy execution, the donor's sum_exec_runtime delta remained zero
while the task-clock delta advanced and triggered the force-idle
reschedule. A focused same-donor owner-handoff test split the donor's
service across two execution owners. Each owner's individual service stayed
below half of the donor slice, while their accumulated service exceeded the
threshold and triggered the force-idle reschedule. Both phases kept the
same core_sched_start baseline.

Fixes: aa4f74dfd42b ("sched: Fix runtime accounting w/ split exec & sched contexts")
Suggested-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Hui Su <sh_def@163.com>
---
 kernel/sched/fair.c  | 27 ++++++++++++++++++---------
 kernel/sched/sched.h |  2 ++
 2 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index ed08b287016a..9a7be87f1991 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6469,7 +6469,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags)
 }
 
 static void
-set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se)
+set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, bool reset_core_slice)
 {
 	/* 'current' is not kept within the tree. */
 	if (se->on_rq) {
@@ -6502,6 +6502,10 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se)
 	}
 
 	se->prev_sum_exec_runtime = se->sum_exec_runtime;
+#ifdef CONFIG_SCHED_CORE
+	if (reset_core_slice && entity_is_task(se))
+		rq_of(cfs_rq)->core_sched_start = se->exec_start;
+#endif
 }
 
 static bool __dequeue_task(struct rq *rq, struct task_struct *p, int flags);
@@ -14786,16 +14790,16 @@ static void rq_offline_fair(struct rq *rq)
 
 #ifdef CONFIG_SCHED_CORE
 static inline bool
-__entity_slice_used(struct sched_entity *se, int min_nr_tasks)
+__entity_slice_used(struct rq *rq, struct sched_entity *se, int min_nr_tasks)
 {
-	u64 rtime = se->sum_exec_runtime - se->prev_sum_exec_runtime;
-	u64 slice = se->slice;
+	/* exec_start advances with donor service under proxy execution. */
+	u64 rtime = se->exec_start - rq->core_sched_start;
 
-	return (rtime * min_nr_tasks > slice);
+	return (rtime * min_nr_tasks > se->slice);
 }
 
 #define MIN_NR_TASKS_DURING_FORCEIDLE	2
-static inline void task_tick_core(struct rq *rq, struct task_struct *curr)
+static inline void task_tick_core(struct rq *rq, struct task_struct *donor)
 {
 	if (!sched_core_enabled(rq))
 		return;
@@ -14815,7 +14819,7 @@ static inline void task_tick_core(struct rq *rq, struct task_struct *curr)
 	 * if we need to give up the CPU.
 	 */
 	if (rq->core->core_forceidle_count && rq->cfs.h_nr_queued == 1 &&
-	    __entity_slice_used(&curr->se, MIN_NR_TASKS_DURING_FORCEIDLE))
+	    __entity_slice_used(rq, &donor->se, MIN_NR_TASKS_DURING_FORCEIDLE))
 		resched_curr(rq);
 }
 
@@ -15049,7 +15053,7 @@ static int task_is_throttled_fair(struct task_struct *p, int cpu)
 	return throttled_hierarchy(cfs_rq);
 }
 #else /* !CONFIG_SCHED_CORE: */
-static inline void task_tick_core(struct rq *rq, struct task_struct *curr) {}
+static inline void task_tick_core(struct rq *rq, struct task_struct *donor) {}
 #endif /* !CONFIG_SCHED_CORE */
 
 /*
@@ -15257,6 +15261,11 @@ static void set_next_task_fair(struct rq *rq, struct task_struct *p, bool first)
 {
 	struct sched_entity *se = &p->se;
 	bool throttled = false;
+	/*
+	 * Reset for a new donor or reactivation, but preserve a same-donor
+	 * proxy reselect so service before an owner handoff is retained.
+	 */
+	bool reset_core_slice = !first || rq->donor != p;
 	struct cfs_rq *cfs_rq = &rq->cfs;
 	unsigned long weight = NICE_0_LOAD;
 	bool on_rq = se->on_rq;
@@ -15271,7 +15280,7 @@ static void set_next_task_fair(struct rq *rq, struct task_struct *p, bool first)
 
 		if (!IS_ENABLED(CONFIG_FAIR_GROUP_SCHED) ||
 		    !first || !cfs_rq->h_curr)
-			set_next_entity(cfs_rq, se);
+			set_next_entity(cfs_rq, se, reset_core_slice);
 
 		/* ensure bandwidth has been allocated on our new cfs_rq */
 		throttled |= account_cfs_rq_runtime(cfs_rq, 0);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6a8deddc725b..f2f1e3642831 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1382,6 +1382,8 @@ struct rq {
 	unsigned int		core_forceidle_seq;
 	unsigned int		core_forceidle_occupation;
 	u64			core_forceidle_start;
+	/* Task-clock baseline for the current core-scheduling donor slice. */
+	u64			core_sched_start;
 	unsigned int		core_pick_in_flight;
 #endif /* CONFIG_SCHED_CORE */
 
-- 
2.55.0