[PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[Jemmy Wong]

After the split between scheduling and execution context, update_se()
charges the open run interval to rq->curr. The tick and the runtime
sample still treated the donor as the task on CPU. Under proxy
execution the donor is blocked on a kernel mutex and the lock holder
is the task that runs.

The NUMA and cache tick hooks move to rq->curr, and only when that
task is fair, matching account_mm_sched(). task_sched_runtime()
flushes when the sampled task is rq->curr, through the donor class's
update_curr(), because that reads the donor's exec_start and
update_se() then adds the delta to rq->curr.

The single-task CFS bandwidth check is pointed at the donor, matching
sched_fair_update_stop_tick(). Both return before the bandwidth test
unless one task is queued, so the tick decision does not change during
a proxy and the third patch has no Fixes: tag.

Without CONFIG_SCHED_PROXY_EXEC, curr and donor are a union and these
are the same task.

Jemmy Wong (3):
  sched/fair: Drive NUMA and cache tick work from the execution context
  sched/core: Flush pending runtime for the execution context
  sched/core: Check CFS bandwidth against the donor for NOHZ

 kernel/sched/core.c | 18 ++++++++++++------
 kernel/sched/fair.c | 28 ++++++++++++++++++----------
 2 files changed, 30 insertions(+), 16 deletions(-)

--
2.54.0 (Apple Git-157)

[PATCH 1/3] sched/fair: Drive NUMA and cache tick work from the execution context

[Jemmy Wong]

task_tick() is invoked with rq->donor. Under proxy execution the donor
is blocked on a mutex while rq->curr burns the CPU.

update_se() already charges sum_exec_runtime and LLC runtime to
rq->curr. The NUMA and cache tick hooks still used the donor, so
node_stamp + numa_scan_period was never crossed (donor runtime is
frozen) and TWA_RESUME task_work was queued on a blocked task.

Pass rq->curr for those hooks, matching psi_account_irqtime() and
wq_worker_tick(), and rename task_tick_fair()'s argument from curr to
donor so the scheduling context is not mistaken for rq->curr.

The lock holder need not be a fair task, so only run the hooks when
rq->curr is in fair_sched_class. This keeps NUMA scanning off non-fair
tasks and keeps task_tick_cache() consistent with account_mm_sched(),
which already ignores non-fair tasks.

Without CONFIG_SCHED_PROXY_EXEC the two rq members are a union.

Fixes: aa4f74dfd42b ("sched: Fix runtime accounting w/ split exec & sched contexts")
Fixes: df0d98475954 ("sched/cache: Introduce infrastructure for cache-aware load balancing")
Signed-off-by: Jemmy Wong <jemmywong512@gmail.com>
Assisted-by: LLM
---
 kernel/sched/fair.c | 28 ++++++++++++++++++----------
 1 file changed, 18 insertions(+), 10 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 57360f5cdde4..9dbc127a9ee1 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15326,12 +15326,13 @@ static inline void task_tick_core(struct rq *rq, struct task_struct *curr) {}
  *
  * NOTE: This function can be called remotely by the tick offload that
  * goes along full dynticks. Therefore no local assumption can be made
- * and everything must be accessed through the @rq and @curr passed in
+ * and everything must be accessed through the @rq and @donor passed in
  * parameters.
  */
-static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
+static void task_tick_fair(struct rq *rq, struct task_struct *donor, int queued)
 {
-	struct sched_entity *se = &curr->se;
+	struct sched_entity *se = &donor->se;
+	struct task_struct *curr = rq->curr;
 
 	if (se->on_rq) {
 		unsigned long weight = NICE_0_LOAD;
@@ -15344,22 +15345,29 @@ static void task_tick_fair(struct rq *rq, struct task_struct *curr, int queued)
 			weight = __calc_prop_weight(cfs_rq, se, weight);
 		}
 
-		se = &curr->se;
+		se = &donor->se;
 		reweight_eevdf(cfs_rq, se, weight, se->on_rq);
 	}
 
 	if (queued)
 		return;
 
-	if (static_branch_unlikely(&sched_numa_balancing))
-		task_tick_numa(rq, curr);
+	/*
+	 * NUMA and cache work follow the execution context: update_se()
+	 * charges runtime to rq->curr, and TWA_RESUME must run on that
+	 * task. Under proxy-exec rq->curr may be of another class.
+	 */
+	if (curr->sched_class == &fair_sched_class) {
+		if (static_branch_unlikely(&sched_numa_balancing))
+			task_tick_numa(rq, curr);
 
-	task_tick_cache(rq, curr);
+		task_tick_cache(rq, curr);
+	}
 
-	update_misfit_status(curr, rq);
-	check_update_overutilized_status(task_rq(curr));
+	update_misfit_status(donor, rq);
+	check_update_overutilized_status(task_rq(donor));
 
-	task_tick_core(rq, curr);
+	task_tick_core(rq, donor);
 }
 
 /*
-- 
2.54.0 (Apple Git-157)

[PATCH 2/3] sched/core: Flush pending runtime for the execution context

[Jemmy Wong]

task_sched_runtime() reports sum_exec_runtime to getrusage() and
CLOCK_THREAD_CPUTIME_ID. It commits the open run interval first, but
only when @p is the donor, using @p->sched_class->update_curr().

The interval is not added to the donor. The donor class's update_curr()
reads the donor's exec_start, and update_se() charges that delta to
rq->curr. Under proxy execution those tasks differ: the lock holder is
on CPU as rq->curr, and the waiter blocked on the mutex is rq->donor.
A runtime read of the lock holder skipped the flush and dropped the
time since the last update.

Commit the interval when @p is rq->curr, through the donor class's
update_curr(). The delta is still taken from the donor's exec_start
and still applied to @p. Without CONFIG_SCHED_PROXY_EXEC, curr and
donor are a union and therefore the same task, so the condition is
unchanged.

Fixes: aa4f74dfd42b ("sched: Fix runtime accounting w/ split exec & sched contexts")
Signed-off-by: Jemmy Wong <jemmywong512@gmail.com>
Assisted-by: LLM
---
 kernel/sched/core.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1fe40de6ebe3..b2005fafae59 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5715,14 +5715,18 @@ unsigned long long task_sched_runtime(struct task_struct *p)
 
 	rq = task_rq_lock(p, &rf);
 	/*
-	 * Must be ->curr _and_ ->on_rq.  If dequeued, we would
-	 * project cycles that may never be accounted to this
+	 * Must be the execution context _and_ ->on_rq.  If dequeued, we
+	 * would project cycles that may never be accounted to this
 	 * thread, breaking clock_gettime().
+	 *
+	 * Pending time is charged to rq->curr by update_se(), not the
+	 * donor.  Under proxy execution those can differ; flush via the
+	 * donor's update_curr() so the delta lands on p.
 	 */
-	if (task_current_donor(rq, p) && task_on_rq_queued(p)) {
+	if (task_current(rq, p) && task_on_rq_queued(p)) {
 		prefetch_curr_exec_start(p);
 		update_rq_clock(rq);
-		p->sched_class->update_curr(rq);
+		rq->donor->sched_class->update_curr(rq);
 	}
 	ns = p->se.sum_exec_runtime;
 	task_rq_unlock(rq, p, &rf);
-- 
2.54.0 (Apple Git-157)

[PATCH 3/3] sched/core: Check CFS bandwidth against the donor for NOHZ

[Jemmy Wong]

sched_can_stop_tick() tests rq->curr, the execution context, for CFS
bandwidth. Quota is consumed by the donor, the scheduling context:
account_cfs_rq_runtime() charges rq->donor's cfs_rq, and
sched_fair_update_stop_tick() already passes the donor picked in
set_next_task_fair() to cfs_task_bw_constrained().

Test rq->donor in sched_can_stop_tick() as well, so both callers name
the task whose quota is consumed. The tick decision is unchanged:
both paths return before the bandwidth test unless nr_running == 1,
and proxy execution keeps the donor and the lock holder (curr) queued.
Without CONFIG_SCHED_PROXY_EXEC, curr and donor are a union and
therefore the same task.

No functional change.

Signed-off-by: Jemmy Wong <jemmywong512@gmail.com>
Assisted-by: LLM
---
 kernel/sched/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b2005fafae59..7eddbc14ddcb 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1477,9 +1477,11 @@ bool sched_can_stop_tick(struct rq *rq)
 	 * This check prevents clearing the bit if a newly enqueued task here is
 	 * dequeued by migrating while the constrained task continues to run.
 	 * E.g. going from 2->1 without going through pick_next_task().
+	 *
+	 * Test the donor, as sched_fair_update_stop_tick() does.
 	 */
-	if (__need_bw_check(rq, rq->curr)) {
-		if (cfs_task_bw_constrained(rq->curr))
+	if (__need_bw_check(rq, rq->donor)) {
+		if (cfs_task_bw_constrained(rq->donor))
 			return false;
 	}
 
-- 
2.54.0 (Apple Git-157)

Re: [PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[Kayra Cizmeci]

> After the split between scheduling and execution context, update_se()
> charges the open run interval to rq->curr. The tick and the runtime
> sample still treated the donor as the task on CPU. Under proxy
> execution the donor is blocked on a kernel mutex and the lock holder
> is the task that runs.

Please look at here: https://lore.kernel.org/lkml/20260913064722.1534766-1-sh_def@163.com/

I think y'all are trying to fix the same thing, no? (Or atleast similar in most ways.)

Adding Hui and John to CC.

Thanks,
Kayra

Re: [PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[Jemmy Wong]

Hi Kayra,

Yes. My first patch and Hui's v5 2/4 and 3/4 move the NUMA and cache
ticks onto rq->curr when that task is fair. Hui's 1/4 is the right
shape: task_tick() runs the donor class and, when a proxy crosses
classes, the execution class as well. A fair lock holder whose donor
is RT or deadline still gets those hooks. My patch only changes
task_tick_fair(), so that case stays missing. I'll drop it.

The other two are not in that series.

task_sched_runtime() still flushes only when @p is the donor, through
@p->sched_class->update_curr(). update_se() adds the delta to rq->curr,
so a runtime read of the lock holder drops the open interval. The fix
flushes when @p is rq->curr, through the donor class's update_curr().

sched_can_stop_tick() still passes rq->curr to
cfs_task_bw_constrained(). sched_fair_update_stop_tick() already
passes the donor. The third patch makes those callers match. It does
not change the tick while a proxy is active, because both checks
return unless nr_running == 1.

Thanks,
Jemmy


> On Oct 8, 2026, at 11:33 PM, Kayra Cizmeci <kayracizmeci@gmail.com> wrote:
> 
>> After the split between scheduling and execution context, update_se()
>> charges the open run interval to rq->curr. The tick and the runtime
>> sample still treated the donor as the task on CPU. Under proxy
>> execution the donor is blocked on a kernel mutex and the lock holder
>> is the task that runs.
> 
> Please look at here: https://lore.kernel.org/lkml/20260913064722.1534766-1-sh_def@163.com/
> 
> I think y'all are trying to fix the same thing, no? (Or atleast similar in most ways.)
> 
> Adding Hui and John to CC.
> 
> Thanks,
> Kayra

Re: [PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[Hui Su]

> Yes. My first patch and Hui's v5 2/4 and 3/4 move the NUMA and cache
> ticks onto rq->curr when that task is fair. Hui's 1/4 is the right
> shape: task_tick() runs the donor class and, when a proxy crosses
> classes, the execution class as well. A fair lock holder whose donor
> is RT or deadline still gets those hooks. My patch only changes
> task_tick_fair(), so that case stays missing. I'll drop it.

Thanks Jemmy,

Yes, agreed on 1/3.

For 2/3, I think this also overlaps with an earlier standalone patch I
posted on Sep 2:

[PATCH] sched/core: fix task_sched_runtime() for proxy execution
Message-ID: <20260902112539.879979-1-sh_def@163.com>
https://lore.kernel.org/r/20260902112539.879979-1-sh_def@163.com/

It makes the same two changes in task_sched_runtime(): test whether @p is
the execution context with task_current(), while still invoking
update_curr() through rq->donor->sched_class.

For 3/3, Andrea has an earlier patch, first posted on Jul 16, addressing
the same NOHZ CFS bandwidth issue:

sched: Make NOHZ CFS bandwidth checks follow proxy donor
https://lore.kernel.org/r/20260716132229.61603-2-arighi@nvidia.com/

In particular, that patch also removes the nr_running == 1 restriction.
As Andrea points out, a retained proxy donor keeps both the donor and
mutex owner queued, so keeping that restriction would skip the bandwidth
check in the active proxy case.

Thanks,
Hui

Re: [PATCH 0/3] sched: Fix proxy-exec use of curr and donor

[Jemmy Wong]

Hi Hui,

Agreed on 2/3. Your Sep 2 patch makes the same task_sched_runtime()
change: task_current() for the execution context, and update_curr()
through the donor class. I'll drop it.

Agreed on 3/3 as well. Andrea's patch is earlier, and it removes the
nr_running == 1 guard that mine keeps. With that guard the bandwidth
test does not run while a proxy leaves both tasks queued. I'll drop
this one too.

Thanks for the pointers.

Jemmy

> On Oct 9, 2026, at 12:07 PM, Hui Su <sh_def@163.com> wrote:
> 
>> Yes. My first patch and Hui's v5 2/4 and 3/4 move the NUMA and cache
>> ticks onto rq->curr when that task is fair. Hui's 1/4 is the right
>> shape: task_tick() runs the donor class and, when a proxy crosses
>> classes, the execution class as well. A fair lock holder whose donor
>> is RT or deadline still gets those hooks. My patch only changes
>> task_tick_fair(), so that case stays missing. I'll drop it.
> 
> Thanks Jemmy,
> 
> Yes, agreed on 1/3.
> 
> For 2/3, I think this also overlaps with an earlier standalone patch I
> posted on Sep 2:
> 
> [PATCH] sched/core: fix task_sched_runtime() for proxy execution
> Message-ID: <20260902112539.879979-1-sh_def@163.com>
> https://lore.kernel.org/r/20260902112539.879979-1-sh_def@163.com/
> 
> It makes the same two changes in task_sched_runtime(): test whether @p is
> the execution context with task_current(), while still invoking
> update_curr() through rq->donor->sched_class.
> 
> For 3/3, Andrea has an earlier patch, first posted on Jul 16, addressing
> the same NOHZ CFS bandwidth issue:
> 
> sched: Make NOHZ CFS bandwidth checks follow proxy donor
> https://lore.kernel.org/r/20260716132229.61603-2-arighi@nvidia.com/
> 
> In particular, that patch also removes the nr_running == 1 restriction.
> As Andrea points out, a retained proxy donor keeps both the donor and
> mutex owner queued, so keeping that restriction would skip the bandwidth
> check in the active proxy case.
> 
> Thanks,
> Hui
>