Re: [PATCH 1/2] sched_ext: Add lazy preemption support

[Andrea Righi <arighi@nvidia.com>]

Hi Tejun,

On Sun, Sep 13, 2026 at 06:50:14AM -1000, Tejun Heo wrote:
> Hello, Andrea.
> 
> This mostly looks fine to me, but the kick path changes could be simpler.
> 
> On Fri, Sep 11, 2026 at 09:56:53PM +0200, Andrea Righi wrote:
> > +		if ((sch->ops.flags & SCX_OPS_LAZY_SLICE_EXPIRY) &&
> > +		    !scx_bypassing(sch, cpu_of(rq)))
> > +			resched_curr_lazy(rq);
> 
> Could lazy slice expiry be a per-task flag, with the ops flag providing the
> default? That would let a scheduler choose immediate or lazy expiry for
> individual tasks. BPF should be able to override the default in either
> direction, with bypass still forcing immediate rescheduling.

Yes, that makes sense. We can add a BPF-writable per-task slice-expiry setting
(e.g., p->scx.slice_expires_lazy), initialize it from SCX_OPS_LAZY_SLICE_EXPIRY
immediately before ops.enable() and the BPF scheduler may override it from
ops.enable() or any subsequent callback.

> 
> There's also a NO_HZ_FULL corner case. If a remote target is running with
> SCX_SLICE_INF and its tick stopped, resched_curr_lazy() sends no IPI, and
> clearing the slice doesn't restart the tick. sched_tick_remote() calls
> task_tick_scx() directly, bypassing the lazy-to-immediate promotion in
> sched_tick(). With lazy expiry enabled, it keeps requesting lazy
> rescheduling. That leaves delivery dependent on another interrupt, such as
> the deadline server timer. Both the kick and enqueue paths need to arrange
> progress for a tick-stopped target.

Good point. We can add a common helper for lazy SCX rescheduling, when the
runqueue was allowed to stop its tick, the helper clears SCX_RQ_CAN_STOP_TICK
and update the scheduler tick dependency before calling resched_curr_lazy().
And setting TICK_DEP_BIT_SCHED will restart/kick the tick as needed.

Both the lazy enqueue and lazy kick paths will use this helper after
successfully clearing the current task's slice.

> 
> > +	if (preempt)
> > +		cpumask_clear_cpu(cpu, pcpu->cpus_to_preempt);
> > +	if (preempt_lazy)
> > +		cpumask_clear_cpu(cpu, pcpu->cpus_to_preempt_lazy);
> 
> Why not clear both masks where cpus_to_preempt was previously cleared,
> including the skipped-kick path? There's no need for the additional
> conditions here.

Ack. I'll change this.

> 
> > +	if (unlikely((flags & SCX_KICK_PREEMPT) && (flags & SCX_KICK_PREEMPT_LAZY))) {
> > +		scx_error(sch, "SCX_KICK_PREEMPT and SCX_KICK_PREEMPT_LAZY cannot be combined");
> > +		return;
> > +	}
> > +	if (unlikely((flags & SCX_KICK_PREEMPT_LAZY) && (flags & SCX_KICK_WAIT))) {
> > +		scx_error(sch, "SCX_KICK_PREEMPT_LAZY cannot be used with SCX_KICK_WAIT");
> > +		return;
> > +	}
> 
> Do we need to reject all these combinations? PREEMPT should win over
> PREEMPT_LAZY, as it does when separate calls request both. WAIT can force
> immediate rescheduling. A plain kick combined with lazy preemption should
> still clear the slice and reschedule immediately.

Agreed. We can accept these combinations and apply the following precedence:

 PREEMPT or WAIT > plain kick > PREEMPT_LAZY

A lazy request should still clear the slice when it is combined with an
immediate request. And I'll make the enqueue interface consistent with kick.

> 
> > +		if (!cpumask_test_cpu(cpu, pcpu->cpus_to_preempt))
> > +			cpumask_set_cpu(cpu, pcpu->cpus_to_preempt_lazy);
> 
> [ ... ]
> 
> > +			cpumask_clear_cpu(cpu, pcpu->cpus_to_preempt_lazy);
> 
> Can we just accumulate the requested bits and resolve precedence in
> kick_one_cpu()? That would remove both the guard against cpus_to_preempt and
> the clearing of cpus_to_preempt_lazy.

Yep, makes sense.

Thanks,
-Andrea