Re: [PATCH 07/18] sched: Add sched_ext hooks for proxy execution

[Andrea Righi <arighi@nvidia.com>]

Hi Peter,

On Wed, Sep 16, 2026 at 11:00:17AM +0200, Peter Zijlstra wrote:
> On Tue, Sep 15, 2026 at 10:30:38PM +0200, Andrea Righi wrote:
> > On Thu, Sep 10, 2026 at 12:38:18PM +0200, Peter Zijlstra wrote:
> 
> > > > @@ -7264,6 +7264,8 @@ static void __sched notrace __schedule(int sched_mode)
> > > >  			donor->sched_class->put_prev_task(rq, donor, donor);
> > > >  			donor->sched_class->set_next_task(rq, donor, true);
> > > >  		}
> > > > +		scx_proxy_donor_start(rq);
> > > > +		scx_proxy_resolved(rq);
> > > >  	} else {
> > > >  		rq_set_donor(rq, next);
> > > >  	}
> > > 
> > > Can you expand on the need for scx_proxy_resolved() ? I understand the
> > > other two, but this one I'm struggling with a bit.
> > 
> > Yeah and the name is a poor choice, it should renamed
> > scx_proxy_reenqueue_retry() or something similar.
> > 
> > It's the retry point for a task that sched_ext couldn't move while
> > processing a dispatch from a remote DSQ (used later in the series).
> > 
> > When sched_ext consumes a task dispatched to a CPU other than the one
> > whose rq currently owns it, it may find (after locking the task's
> > source rq) that proxy exec has made the task either the physical
> > current task or the active donor. It can't migrate the task in that
> > state, so it parks the task on the source rq's reject DSQ.
> > 
> > If the deferred reject-DSQ drain runs while the task is still current
> > or donating, the task must remain parked. Retrying immediately could
> > spin until the proxy relationship changes. The hook provides the
> > notification that proxy selection has run again, allowing sched_ext to
> > schedule another deferred drain after the current context switch.
> > 
> > I couldn't find an existing event that covers this transition without
> > periodically retrying the reject-DSQ drain. Is there a better place to
> > trigger this retry?
> 
> Ah, so its a little like that problem we had with ->balance() and
> ->pick_task(). Where a task gets taken off the DSQ and moved to the
> local queue, but when not picked, it must be moved back.

Exactly, that is the same general kind of problem.

> 
> In this case, ->donor is visible to ext and all is well, but ->curr is
> not so easy.
> 
> I'm still a little confused though, why not leave blocked tasks on this
> reject queue. If they're needed, the proxy mechanism will move them
> around. They won't actually ever run except through proxy.
> 
> The point where they will become runnable again, is through wakeup. So
> why not delay everything until that point?
> 
> Or am I not understanding ext again?

I think the missing detail is that the task rejected by the remote DSQ transfer
isn't necessarily blocked.

Consider a blocked donor D proxy-executing a runnable EXT mutex owner O on CPU0.
O is rq->curr and D is rq->donor. Since O is executing using D's scheduling
context, O's own EXT scheduling entity may still be on a non-local DSQ.

CPU1 could concurrently consume O from that non-local DSQ and start transferring
it to CPU1. After dropping the DSQ lock and acquiring O's source rq lock, it
discovers that O has become rq->curr on CPU0 and therefore can't be migrated.
O is then parked on CPU0's reject DSQ.

When proxy exec switches away from O, O stops being rq->curr but remains
runnable. There's no wakeup in this case, because O never blocked. If it remains
on the reject DSQ, it's no longer visible to the BPF scheduler and can't
subsequently run using its own scheduling context.

The rq->donor case has a similar issue even though that task is blocked: a
blocked donor can stop being the active donor before the mutex wakes it (for
example, if another scheduling context is selected). The proxy exec mechanism
follows the owner chain after the donor has been selected, it doesn't select
tasks from the internal reject DSQ. Leaving the donor there until wakeup would
prevent it from being selected again for proxy execution in the meantime.

This is why the retry is tied to another proxy selection point, once the task is
no longer rq->curr or rq->donor, it can be re-enqueued to through its BPF
scheduler.

Thanks,
-Andrea