[PATCH sched/urgent] sched/core: Don't lower rq->next_class in proxy_reset_donor()

[PATCH sched/urgent] sched/core: Don't lower rq->next_class in proxy_reset_donor()

[Tejun Heo]

A pick that drops the rq lock, sched_balance_newidle() or
do_pick_task_scx(), first lowers rq->next_class to its own class if it's
above. After relocking, it restarts if rq_modified_above() shows a higher
class wakeup. proxy_reset_donor() sets next_class to rq->curr's class
unconditionally and can lower it in that window. With a sched_ext task as
rq->curr:

1. fair's queue is empty and sched_balance_newidle() drops the rq lock with
   next_class at ext from the last pick.
2. An RT wakeup raises next_class to rt and calls resched_curr().
3. rq->donor, which proxy exec migrated to this CPU, wakes up.
   proxy_needs_return() calls proxy_reset_donor(), which sets next_class
   back to ext.
4. rq_modified_above() sees nothing above fair, sched_ext returns a task and
   __schedule() clears need_resched. The RT task runs only at the CPU's next
   schedule.

This needs a sched_ext donor. With a fair donor, fair's queue isn't empty
and newidle doesn't run, and RT and DL donors are picked before fair. Proxy
exec with sched_ext became possible with c6fe97c34a1a ("sched: Allow
enabling proxy exec with sched_ext").

Only raise next_class in proxy_reset_donor(). next_class above rq->curr's
class is what wakeup_preempt() leaves after any higher class wakeup, and the
->wakeup_preempt() methods ignore a donor of another class since
fa02b2868420 ("sched/core: Fix inter-class wakeup_preempt()"). The
resched_curr() in proxy_reset_donor() makes the next pick set next_class.
The raise stays so that rq_modified_above() still catches a reset to a curr
of a higher class.

Fixes: c6fe97c34a1a ("sched: Allow enabling proxy exec with sched_ext")
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 kernel/sched/core.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 04b62c16fa8c..f850f4ae19ea 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3809,7 +3809,12 @@ static inline void proxy_reset_donor(struct rq *rq)
 	WARN_ON_ONCE(rq->donor == rq->curr);
 
 	put_prev_set_next_task(rq, rq->donor, rq->curr);
-	rq->next_class = rq->curr->sched_class;
+	/*
+	 * A pick that dropped the rq lock checks next_class after relocking to
+	 * catch a higher class wakeup, see rq_modified_above(). Don't lower it.
+	 */
+	if (sched_class_above(rq->curr->sched_class, rq->next_class))
+		rq->next_class = rq->curr->sched_class;
 	rq_set_donor(rq, rq->curr);
 	zap_balance_callbacks(rq);
 	resched_curr(rq);
-- 
2.56.0