[PATCHSET v4 sched_ext/for-6.20] sched_ext: Fix ops.dequeue() semantics

[PATCHSET v4 sched_ext/for-6.20] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

The callback ops.dequeue() is provided to let BPF schedulers observe when a
task leaves the scheduler, either because it is dispatched or due to a task
property change. However, this callback is currently unreliable and not
invoked systematically, which can result in missed ops.dequeue() events.

In particular, once a task is removed from the scheduler (whether for
dispatch or due to a property change) the BPF scheduler loses visibility of
the task and the sched_ext core may not always trigger ops.dequeue().

This breaks accurate accounting (i.e., per-DSQ queued runtime sums) and
prevents reliable tracking of task lifecycle transitions.

This patch set fixes the semantics of ops.dequeue(), by guaranteeing that
each task entering the BPF scheduler's custody triggers exactly one
ops.dequeue() call when it leaves that custody, whether the exit is due to
a dispatch (regular or via a core scheduling pick) or to a scheduling
property change (e.g.  sched_setaffinity(), sched_setscheduler(),
set_user_nice(), NUMA balancing, etc.).

To identify property change dequeues a new ops.dequeue() flag is
introduced: %SCX_DEQ_SCHED_CHANGE.

Together, these changes allow BPF schedulers to reliably track task
ownership and maintain accurate accounting.

= Open issues =

Even though a few refinements are still pending, I'm sending a new
patchset, so we can comment more effectively based on the latest
agreed-upon semantics.

Open issues that still need agreement:

 - Should we trigger ops.dequeue() for tasks dispatched to SCX_DSQ_GLOBAL?
   (do we treat SCX_DSQ_GLOBAL as a local DSQ or a user DSQ?). In the
   current implementation, SCX_DSQ_GLOBAL is treated like a built-in user
   DSQ -> ops.dequeue() invoked for tasks dispatched to SCX_DSQ_GLOBAL.

Changes in v4:
 - Introduce the concept of "BPF scheduler custody"
 - Do not trigger ops.dequeue() for tasks directly dispatched to local DSQs
 - Trigger ops.dequeue() only once; after the task leaves BPF scheduler
   custody, further dequeue events are not reported.
 - Link to v3: https://lore.kernel.org/all/20260126084258.3798129-1-arighi@nvidia.com

Changes in v3:
 - Rename SCX_DEQ_ASYNC to SCX_DEQ_SCHED_CHANGE
 - Handle core-sched dequeues (Kuba)
 - Link to v2: https://lore.kernel.org/all/20260121123118.964704-1-arighi@nvidia.com

Changes in v2:
 - Distinguish between "dispatch" dequeues and "property change" dequeues
   (flag SCX_DEQ_ASYNC)
 - Link to v1: https://lore.kernel.org/all/20251219224450.2537941-1-arighi@nvidia.com

Andrea Righi (2):
      sched_ext: Fix ops.dequeue() semantics
      selftests/sched_ext: Add test to validate ops.dequeue() semantics

 Documentation/scheduler/sched-ext.rst           |  76 ++++++++
 include/linux/sched/ext.h                       |   1 +
 kernel/sched/ext.c                              | 168 ++++++++++++++++-
 kernel/sched/ext_internal.h                     |   7 +
 tools/sched_ext/include/scx/enum_defs.autogen.h |   1 +
 tools/sched_ext/include/scx/enums.autogen.bpf.h |   2 +
 tools/sched_ext/include/scx/enums.autogen.h     |   1 +
 tools/testing/selftests/sched_ext/Makefile      |   1 +
 tools/testing/selftests/sched_ext/dequeue.bpf.c | 234 ++++++++++++++++++++++++
 tools/testing/selftests/sched_ext/dequeue.c     | 198 ++++++++++++++++++++
 10 files changed, 686 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/sched_ext/dequeue.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/dequeue.c

[PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

Currently, ops.dequeue() is only invoked when the sched_ext core knows
that a task resides in BPF-managed data structures, which causes it to
miss scheduling property change events. In addition, ops.dequeue()
callbacks are completely skipped when tasks are dispatched to non-local
DSQs from ops.select_cpu(). As a result, BPF schedulers cannot reliably
track task state.

Fix this by guaranteeing that each task entering the BPF scheduler's
custody triggers exactly one ops.dequeue() call when it leaves that
custody, whether the exit is due to a dispatch (regular or via a core
scheduling pick) or to a scheduling property change (e.g.
sched_setaffinity(), sched_setscheduler(), set_user_nice(), NUMA
balancing, etc.).

BPF scheduler custody concept: a task is considered to be in "BPF
scheduler's custody" when it has been queued in BPF-managed data
structures and the BPF scheduler is responsible for its lifecycle.
Custody ends when the task is dispatched to a local DSQ, selected by
core scheduling, or removed due to a property change.

Tasks directly dispatched to local DSQs (via %SCX_DSQ_LOCAL or
%SCX_DSQ_LOCAL_ON) bypass the BPF scheduler entirely and are not in its
custody. As a result, ops.dequeue() is not invoked for these tasks.

To identify dequeues triggered by scheduling property changes, introduce
the new ops.dequeue() flag %SCX_DEQ_SCHED_CHANGE: when this flag is set,
the dequeue was caused by a scheduling property change.

New ops.dequeue() semantics:
 - ops.dequeue() is invoked exactly once when the task leaves the BPF
   scheduler's custody, in one of the following cases:
   a) regular dispatch: task was dispatched to a non-local DSQ (global
      or user DSQ), ops.dequeue() called without any special flags set
   b) core scheduling dispatch: core-sched picks task before dispatch,
      dequeue called with %SCX_DEQ_CORE_SCHED_EXEC flag set
   c) property change: task properties modified before dispatch,
      dequeue called with %SCX_DEQ_SCHED_CHANGE flag set

This allows BPF schedulers to:
 - reliably track task ownership and lifecycle,
 - maintain accurate accounting of managed tasks,
 - update internal state when tasks change properties.

Cc: Tejun Heo <tj@kernel.org>
Cc: Emil Tsalapatis <emil@etsalapatis.com>
Cc: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 Documentation/scheduler/sched-ext.rst         |  76 ++++++++
 include/linux/sched/ext.h                     |   1 +
 kernel/sched/ext.c                            | 168 +++++++++++++++++-
 kernel/sched/ext_internal.h                   |   7 +
 .../sched_ext/include/scx/enum_defs.autogen.h |   1 +
 .../sched_ext/include/scx/enums.autogen.bpf.h |   2 +
 tools/sched_ext/include/scx/enums.autogen.h   |   1 +
 7 files changed, 253 insertions(+), 3 deletions(-)

diff --git a/Documentation/scheduler/sched-ext.rst b/Documentation/scheduler/sched-ext.rst
index 404fe6126a769..6d9e82e6ca9d4 100644
--- a/Documentation/scheduler/sched-ext.rst
+++ b/Documentation/scheduler/sched-ext.rst
@@ -252,6 +252,80 @@ The following briefly shows how a waking task is scheduled and executed.
 
    * Queue the task on the BPF side.
 
+   **Task State Tracking and ops.dequeue() Semantics**
+
+   Once ``ops.select_cpu()`` or ``ops.enqueue()`` is called, the task may
+   enter the "BPF scheduler's custody" depending on where it's dispatched:
+
+   * **Direct dispatch to local DSQs** (``SCX_DSQ_LOCAL`` or
+     ``SCX_DSQ_LOCAL_ON | cpu``): The task bypasses the BPF scheduler
+     entirely and goes straight to the CPU's local run queue. The task
+     never enters BPF custody, and ``ops.dequeue()`` will not be called.
+
+   * **Dispatch to non-local DSQs** (``SCX_DSQ_GLOBAL`` or custom DSQs):
+     the task enters the BPF scheduler's custody. When the task later
+     leaves BPF custody (dispatched to a local DSQ, picked by core-sched,
+     or dequeued for sleep/property changes), ``ops.dequeue()`` will be
+     called exactly once.
+
+   * **Queued on BPF side**: The task is in BPF data structures and in BPF
+     custody, ``ops.dequeue()`` will be called when it leaves.
+
+   The key principle: **ops.dequeue() is called when a task leaves the BPF
+   scheduler's custody**. A task is in BPF custody if it's on a non-local
+   DSQ or in BPF data structures. Once dispatched to a local DSQ or after
+   ops.dequeue() is called, the task is out of BPF custody and the BPF
+   scheduler no longer needs to track it.
+
+   This works correctly with the ``ops.select_cpu()`` direct dispatch
+   optimization: even though it skips ``ops.enqueue()`` invocation, if the
+   task is dispatched to a non-local DSQ, it enters BPF custody and will
+   get ``ops.dequeue()`` when it leaves. This provides the performance
+   benefit of avoiding the ``ops.enqueue()`` roundtrip while maintaining
+   correct state tracking.
+
+   The dequeue can happen for different reasons, distinguished by flags:
+
+   1. **Regular dispatch workflow**: when the task is dispatched from a
+      non-local DSQ to a local DSQ (leaving BPF custody for execution),
+      ``ops.dequeue()`` is triggered without any special flags.
+
+   2. **Core scheduling pick**: when ``CONFIG_SCHED_CORE`` is enabled and
+      core scheduling picks a task for execution while it's still in BPF
+      custody, ``ops.dequeue()`` is called with the
+      ``SCX_DEQ_CORE_SCHED_EXEC`` flag.
+
+   3. **Scheduling property change**: when a task property changes (via
+      operations like ``sched_setaffinity()``, ``sched_setscheduler()``,
+      priority changes, CPU migrations, etc.) while the task is still in
+      BPF custody, ``ops.dequeue()`` is called with the
+      ``SCX_DEQ_SCHED_CHANGE`` flag set in ``deq_flags``.
+
+   **Important**: Once a task has left BPF custody (dispatched to local
+   DSQ), property changes will not trigger ``ops.dequeue()``, since the
+   task is no longer being managed by the BPF scheduler.
+
+   **Property Change Notifications for Running Tasks**:
+
+   For tasks that have left BPF custody (running or on local DSQs),
+   property changes can be intercepted through the dedicated callbacks:
+
+   * ``ops.set_cpumask()``: Called when a task's CPU affinity changes
+     (e.g., via ``sched_setaffinity()``). This callback is invoked for
+     all tasks regardless of their state or BPF custody.
+
+   * ``ops.set_weight()``: Called when a task's scheduling weight/priority
+     changes (e.g., via ``sched_setscheduler()`` or ``set_user_nice()``).
+     This callback is also invoked for all tasks.
+
+   These callbacks provide complete coverage for property changes,
+   complementing ``ops.dequeue()`` which only applies to tasks in BPF
+   custody.
+
+   BPF schedulers can choose not to implement ``ops.dequeue()`` if they
+   don't need to track these transitions. The sched_ext core will safely
+   handle all dequeue operations regardless.
+
 3. When a CPU is ready to schedule, it first looks at its local DSQ. If
    empty, it then looks at the global DSQ. If there still isn't a task to
    run, ``ops.dispatch()`` is invoked which can use the following two
@@ -319,6 +393,8 @@ by a sched_ext scheduler:
                 /* Any usable CPU becomes available */
 
                 ops.dispatch(); /* Task is moved to a local DSQ */
+
+                ops.dequeue(); /* Exiting BPF scheduler */
             }
             ops.running();      /* Task starts running on its assigned CPU */
             while (task->scx.slice > 0 && task is runnable)
diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h
index bcb962d5ee7d8..0d003d2845393 100644
--- a/include/linux/sched/ext.h
+++ b/include/linux/sched/ext.h
@@ -84,6 +84,7 @@ struct scx_dispatch_q {
 /* scx_entity.flags */
 enum scx_ent_flags {
 	SCX_TASK_QUEUED		= 1 << 0, /* on ext runqueue */
+	SCX_TASK_OPS_ENQUEUED	= 1 << 1, /* under ext scheduler's custody */
 	SCX_TASK_RESET_RUNNABLE_AT = 1 << 2, /* runnable_at should be reset */
 	SCX_TASK_DEQD_FOR_SLEEP	= 1 << 3, /* last dequeue was for SLEEP */
 
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index afe28c04d5aa7..6d6f1253039d8 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -924,6 +924,19 @@ static void touch_core_sched(struct rq *rq, struct task_struct *p)
 #endif
 }
 
+/**
+ * is_local_dsq - Check if a DSQ ID represents a local DSQ
+ * @dsq_id: DSQ ID to check
+ *
+ * Returns true if @dsq_id is a local DSQ, false otherwise. Local DSQs are
+ * per-CPU queues where tasks go directly to execution.
+ */
+static inline bool is_local_dsq(u64 dsq_id)
+{
+	return dsq_id == SCX_DSQ_LOCAL ||
+	       (dsq_id & SCX_DSQ_LOCAL_ON) == SCX_DSQ_LOCAL_ON;
+}
+
 /**
  * touch_core_sched_dispatch - Update core-sched timestamp on dispatch
  * @rq: rq to read clock from, must be locked
@@ -1274,6 +1287,24 @@ static void mark_direct_dispatch(struct scx_sched *sch,
 
 	p->scx.ddsp_dsq_id = dsq_id;
 	p->scx.ddsp_enq_flags = enq_flags;
+
+	/*
+	 * Mark the task as entering BPF scheduler's custody if it's being
+	 * dispatched to a non-local DSQ. This handles the case where
+	 * ops.select_cpu() directly dispatches to a non-local DSQ - even
+	 * though ops.enqueue() won't be called, the task enters BPF
+	 * custody and should get ops.dequeue() when it leaves.
+	 *
+	 * For local DSQs, clear the flag, since the task bypasses the BPF
+	 * scheduler entirely. This also clears any flag that was set by
+	 * do_enqueue_task() before we knew the dispatch destination.
+	 */
+	if (SCX_HAS_OP(sch, dequeue)) {
+		if (!is_local_dsq(dsq_id))
+			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
+		else
+			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+	}
 }
 
 static void direct_dispatch(struct scx_sched *sch, struct task_struct *p,
@@ -1287,6 +1318,40 @@ static void direct_dispatch(struct scx_sched *sch, struct task_struct *p,
 
 	p->scx.ddsp_enq_flags |= enq_flags;
 
+	/*
+	 * The task is about to be dispatched, handle ops.dequeue() based
+	 * on where the task is going.
+	 *
+	 * Key principle: ops.dequeue() is called when a task leaves the
+	 * BPF scheduler's custody. A task is in BPF custody if it's on a
+	 * non-local DSQ or in BPF data structures. Once dispatched to a
+	 * local DSQ, it's out of BPF custody.
+	 *
+	 * Direct dispatch to local DSQs: task never enters BPF scheduler's
+	 * custody, it goes straight to the CPU. Don't call ops.dequeue()
+	 * and clear the flag so future property changes also won't trigger
+	 * it.
+	 *
+	 * Direct dispatch to non-local DSQs: task enters BPF scheduler's
+	 * custody. Mark the task as in BPF custody so that when it's later
+	 * dispatched to a local DSQ or dequeued for property changes,
+	 * ops.dequeue() will be called.
+	 *
+	 * This also handles the ops.select_cpu() direct dispatch to
+	 * non-local DSQs: the shortcut skips ops.enqueue() invocation but
+	 * the task still enters BPF custody if dispatched to a non-local
+	 * DSQ, and thus needs ops.dequeue() when it leaves.
+	 */
+	if (SCX_HAS_OP(sch, dequeue)) {
+		if (!is_local_dsq(dsq->id)) {
+			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
+		} else {
+			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
+				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
+			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+		}
+	}
+
 	/*
 	 * We are in the enqueue path with @rq locked and pinned, and thus can't
 	 * double lock a remote rq and enqueue to its local DSQ. For
@@ -1391,6 +1456,21 @@ static void do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
 	WARN_ON_ONCE(atomic_long_read(&p->scx.ops_state) != SCX_OPSS_NONE);
 	atomic_long_set(&p->scx.ops_state, SCX_OPSS_QUEUEING | qseq);
 
+	/*
+	 * Mark that ops.enqueue() is being called for this task. This
+	 * indicates the task is entering the BPF scheduler's data
+	 * structures (QUEUED state).
+	 *
+	 * However, if the task was already marked as in BPF custody by
+	 * mark_direct_dispatch() (ops.select_cpu() direct dispatch to
+	 * non-local DSQ), don't clear that - keep the flag set so
+	 * ops.dequeue() will be called when appropriate.
+	 *
+	 * Only track this flag if ops.dequeue() is implemented.
+	 */
+	if (SCX_HAS_OP(sch, dequeue))
+		p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
+
 	ddsp_taskp = this_cpu_ptr(&direct_dispatch_task);
 	WARN_ON_ONCE(*ddsp_taskp);
 	*ddsp_taskp = p;
@@ -1523,6 +1603,30 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
 
 	switch (opss & SCX_OPSS_STATE_MASK) {
 	case SCX_OPSS_NONE:
+		/*
+		 * Task is not in BPF data structures (either dispatched to
+		 * a DSQ or running). Only call ops.dequeue() if the task
+		 * is still in BPF scheduler's custody
+		 * (%SCX_TASK_OPS_ENQUEUED is set).
+		 *
+		 * If the task has already been dispatched to a local DSQ
+		 * (left BPF custody), the flag will be clear and we skip
+		 * ops.dequeue()
+		 *
+		 * If this is a property change (not sleep/core-sched) and
+		 * the task is still in BPF custody, set the
+		 * %SCX_DEQ_SCHED_CHANGE flag.
+		 */
+		if (SCX_HAS_OP(sch, dequeue) &&
+		    p->scx.flags & SCX_TASK_OPS_ENQUEUED) {
+			u64 flags = deq_flags;
+
+			if (!(deq_flags & (DEQUEUE_SLEEP | SCX_DEQ_CORE_SCHED_EXEC)))
+				flags |= SCX_DEQ_SCHED_CHANGE;
+
+			SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, flags);
+			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+		}
 		break;
 	case SCX_OPSS_QUEUEING:
 		/*
@@ -1531,9 +1635,24 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
 		 */
 		BUG();
 	case SCX_OPSS_QUEUED:
-		if (SCX_HAS_OP(sch, dequeue))
-			SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq,
-					 p, deq_flags);
+		/*
+		 * Task is still on the BPF scheduler (not dispatched yet).
+		 * Call ops.dequeue() to notify. Add %SCX_DEQ_SCHED_CHANGE
+		 * only for property changes, not for core-sched picks or
+		 * sleep.
+		 *
+		 * Clear the flag after calling ops.dequeue(): the task is
+		 * leaving BPF scheduler's custody.
+		 */
+		if (SCX_HAS_OP(sch, dequeue)) {
+			u64 flags = deq_flags;
+
+			if (!(deq_flags & (DEQUEUE_SLEEP | SCX_DEQ_CORE_SCHED_EXEC)))
+				flags |= SCX_DEQ_SCHED_CHANGE;
+
+			SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, flags);
+			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+		}
 
 		if (atomic_long_try_cmpxchg(&p->scx.ops_state, &opss,
 					    SCX_OPSS_NONE))
@@ -1630,6 +1749,7 @@ static void move_local_task_to_local_dsq(struct task_struct *p, u64 enq_flags,
 					 struct scx_dispatch_q *src_dsq,
 					 struct rq *dst_rq)
 {
+	struct scx_sched *sch = scx_root;
 	struct scx_dispatch_q *dst_dsq = &dst_rq->scx.local_dsq;
 
 	/* @dsq is locked and @p is on @dst_rq */
@@ -1638,6 +1758,15 @@ static void move_local_task_to_local_dsq(struct task_struct *p, u64 enq_flags,
 
 	WARN_ON_ONCE(p->scx.holding_cpu >= 0);
 
+	/*
+	 * Task is moving from a non-local DSQ to a local DSQ. Call
+	 * ops.dequeue() if the task was in BPF custody.
+	 */
+	if (SCX_HAS_OP(sch, dequeue) && (p->scx.flags & SCX_TASK_OPS_ENQUEUED)) {
+		SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, dst_rq, p, 0);
+		p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+	}
+
 	if (enq_flags & (SCX_ENQ_HEAD | SCX_ENQ_PREEMPT))
 		list_add(&p->scx.dsq_list.node, &dst_dsq->list);
 	else
@@ -2107,6 +2236,24 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
 
 	BUG_ON(!(p->scx.flags & SCX_TASK_QUEUED));
 
+	/*
+	 * Direct dispatch to local DSQs: call ops.dequeue() if task was in
+	 * BPF custody, then clear the %SCX_TASK_OPS_ENQUEUED flag.
+	 *
+	 * Dispatch to non-local DSQs: task is in BPF scheduler's custody.
+	 * Mark it so ops.dequeue() will be called when it leaves.
+	 */
+	if (SCX_HAS_OP(sch, dequeue)) {
+		if (!is_local_dsq(dsq_id)) {
+			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
+		} else {
+			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
+				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, task_rq(p), p, 0);
+
+			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+		}
+	}
+
 	dsq = find_dsq_for_dispatch(sch, this_rq(), dsq_id, p);
 
 	if (dsq->id == SCX_DSQ_LOCAL)
@@ -2894,6 +3041,14 @@ static void scx_enable_task(struct task_struct *p)
 
 	lockdep_assert_rq_held(rq);
 
+	/*
+	 * Clear enqueue/dequeue tracking flags when enabling the task.
+	 * This ensures a clean state when the task enters SCX. Only needed
+	 * if ops.dequeue() is implemented.
+	 */
+	if (SCX_HAS_OP(sch, dequeue))
+		p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
+
 	/*
 	 * Set the weight before calling ops.enable() so that the scheduler
 	 * doesn't see a stale value if they inspect the task struct.
@@ -2925,6 +3080,13 @@ static void scx_disable_task(struct task_struct *p)
 	if (SCX_HAS_OP(sch, disable))
 		SCX_CALL_OP_TASK(sch, SCX_KF_REST, disable, rq, p);
 	scx_set_task_state(p, SCX_TASK_READY);
+
+	/*
+	 * Clear enqueue/dequeue tracking flags when disabling the task.
+	 * Only needed if ops.dequeue() is implemented.
+	 */
+	if (SCX_HAS_OP(sch, dequeue))
+		p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
 }
 
 static void scx_exit_task(struct task_struct *p)
diff --git a/kernel/sched/ext_internal.h b/kernel/sched/ext_internal.h
index 386c677e4c9a0..befa9a5d6e53f 100644
--- a/kernel/sched/ext_internal.h
+++ b/kernel/sched/ext_internal.h
@@ -982,6 +982,13 @@ enum scx_deq_flags {
 	 * it hasn't been dispatched yet. Dequeue from the BPF side.
 	 */
 	SCX_DEQ_CORE_SCHED_EXEC	= 1LLU << 32,
+
+	/*
+	 * The task is being dequeued due to a property change (e.g.,
+	 * sched_setaffinity(), sched_setscheduler(), set_user_nice(),
+	 * etc.).
+	 */
+	SCX_DEQ_SCHED_CHANGE	= 1LLU << 33,
 };
 
 enum scx_pick_idle_cpu_flags {
diff --git a/tools/sched_ext/include/scx/enum_defs.autogen.h b/tools/sched_ext/include/scx/enum_defs.autogen.h
index c2c33df9292c2..dcc945304760f 100644
--- a/tools/sched_ext/include/scx/enum_defs.autogen.h
+++ b/tools/sched_ext/include/scx/enum_defs.autogen.h
@@ -21,6 +21,7 @@
 #define HAVE_SCX_CPU_PREEMPT_UNKNOWN
 #define HAVE_SCX_DEQ_SLEEP
 #define HAVE_SCX_DEQ_CORE_SCHED_EXEC
+#define HAVE_SCX_DEQ_SCHED_CHANGE
 #define HAVE_SCX_DSQ_FLAG_BUILTIN
 #define HAVE_SCX_DSQ_FLAG_LOCAL_ON
 #define HAVE_SCX_DSQ_INVALID
diff --git a/tools/sched_ext/include/scx/enums.autogen.bpf.h b/tools/sched_ext/include/scx/enums.autogen.bpf.h
index 2f8002bcc19ad..5da50f9376844 100644
--- a/tools/sched_ext/include/scx/enums.autogen.bpf.h
+++ b/tools/sched_ext/include/scx/enums.autogen.bpf.h
@@ -127,3 +127,5 @@ const volatile u64 __SCX_ENQ_CLEAR_OPSS __weak;
 const volatile u64 __SCX_ENQ_DSQ_PRIQ __weak;
 #define SCX_ENQ_DSQ_PRIQ __SCX_ENQ_DSQ_PRIQ
 
+const volatile u64 __SCX_DEQ_SCHED_CHANGE __weak;
+#define SCX_DEQ_SCHED_CHANGE __SCX_DEQ_SCHED_CHANGE
diff --git a/tools/sched_ext/include/scx/enums.autogen.h b/tools/sched_ext/include/scx/enums.autogen.h
index fedec938584be..fc9a7a4d9dea5 100644
--- a/tools/sched_ext/include/scx/enums.autogen.h
+++ b/tools/sched_ext/include/scx/enums.autogen.h
@@ -46,4 +46,5 @@
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_LAST); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_CLEAR_OPSS); \
 	SCX_ENUM_SET(skel, scx_enq_flags, SCX_ENQ_DSQ_PRIQ); \
+	SCX_ENUM_SET(skel, scx_deq_flags, SCX_DEQ_SCHED_CHANGE); \
 } while (0)
-- 
2.52.0

[PATCH 2/2] selftests/sched_ext: Add test to validate ops.dequeue() semantics

[Andrea Righi]

Add a new kselftest to validate the following scenarios:
 - scenario 0 (Local DSQ): tasks don't increment enqueue counters since
   they never enter BPF scheduler custody (no tracking or validation,
   expects 0 enqueues and 0 dequeues),
 - scenario 1 (User DSQ): full enqueue/dequeue lifecycle tracking with
   state machine validation (expects 1:1 enqueue/dequeue pairing).

The test validates that:
 - local DSQ dispatch don't trigger ops.dequeue(),
 - non-local DSQ dispatch has exact 1:1 ops.enqueue/dequeue() pairing,
 - dispatch dequeues have no flags (normal workflow),
 - property change dequeues have %SCX_DEQ_SCHED_CHANGE flag,
 - no duplicate enqueues or invalid state transitions are happening.

This validates that the new ops.dequeue() semantics work correctly for
all task lifecycle scenarios.

Cc: Tejun Heo <tj@kernel.org>
Cc: Emil Tsalapatis <emil@etsalapatis.com>
Cc: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../testing/selftests/sched_ext/dequeue.bpf.c | 238 ++++++++++++++++++
 tools/testing/selftests/sched_ext/dequeue.c   | 201 +++++++++++++++
 3 files changed, 440 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/dequeue.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/dequeue.c

diff --git a/tools/testing/selftests/sched_ext/Makefile b/tools/testing/selftests/sched_ext/Makefile
index 5fe45f9c5f8fd..764e91edabf93 100644
--- a/tools/testing/selftests/sched_ext/Makefile
+++ b/tools/testing/selftests/sched_ext/Makefile
@@ -161,6 +161,7 @@ all_test_bpfprogs := $(foreach prog,$(wildcard *.bpf.c),$(INCLUDE_DIR)/$(patsubs
 
 auto-test-targets :=			\
 	create_dsq			\
+	dequeue				\
 	enq_last_no_enq_fails		\
 	ddsp_bogus_dsq_fail		\
 	ddsp_vtimelocal_fail		\
diff --git a/tools/testing/selftests/sched_ext/dequeue.bpf.c b/tools/testing/selftests/sched_ext/dequeue.bpf.c
new file mode 100644
index 0000000000000..092956becb554
--- /dev/null
+++ b/tools/testing/selftests/sched_ext/dequeue.bpf.c
@@ -0,0 +1,238 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A scheduler that validates ops.dequeue() is called correctly:
+ * - for tasks on BPF data structures (not yet dispatched)
+ * - for tasks dispatched to non-local DSQs (global or user DSQs)
+ * - That every ops.enqueue() is followed by ops.dequeue() except for tasks
+ *   directly dispatched to local DSQs, which bypass the BPF scheduler
+ *   entirely
+ *
+ * Copyright (c) 2026 NVIDIA Corporation.
+ */
+
+#include <scx/common.bpf.h>
+
+#define SHARED_DSQ	0
+
+char _license[] SEC("license") = "GPL";
+
+UEI_DEFINE(uei);
+
+/*
+ * Counters to track the lifecycle of tasks:
+ * - enqueue_cnt: Number of times ops.enqueue() was called
+ * - dequeue_cnt: Number of times ops.dequeue() was called (any type)
+ * - dispatch_dequeue_cnt: Number of regular dispatch dequeues (no flag)
+ * - change_dequeue_cnt: Number of property change dequeues (%SCX_DEQ_SCHED_CHANGE)
+ */
+u64 enqueue_cnt, dequeue_cnt, dispatch_dequeue_cnt, change_dequeue_cnt;
+
+/*
+ * Test scenarios:
+ * - 0: Dispatch to local DSQ (bypasses BPF scheduler, no dequeue callbacks)
+ * - 1: Dispatch to shared DSQ (enters BPF scheduler, dequeue callbacks expected)
+ */
+u32 test_scenario;
+
+/*
+ * Per-task state to track lifecycle and validate workflow semantics.
+ * State transitions:
+ *   NONE -> ENQUEUED (on enqueue)
+ *   ENQUEUED -> DISPATCHED (on dispatch dequeue)
+ *   DISPATCHED -> NONE (on property change dequeue or re-enqueue)
+ *   ENQUEUED -> NONE (on property change dequeue before dispatch)
+ */
+enum task_state {
+	TASK_NONE = 0,      /* Task is outside scheduler control */
+	TASK_ENQUEUED,      /* ops.enqueue() called, waiting for dequeue */
+	TASK_DISPATCHED,    /* Dispatch dequeue received, can get property change or re-enqueue */
+};
+
+struct task_ctx {
+	enum task_state state; /* Current state in the workflow */
+	u64 enqueue_seq;       /* Sequence number for debugging */
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_TASK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, struct task_ctx);
+} task_ctx_stor SEC(".maps");
+
+static struct task_ctx *try_lookup_task_ctx(struct task_struct *p)
+{
+	return bpf_task_storage_get(&task_ctx_stor, p, 0, 0);
+}
+
+s32 BPF_STRUCT_OPS(dequeue_select_cpu, struct task_struct *p,
+		   s32 prev_cpu, u64 wake_flags)
+{
+	/* Always bounce to ops.enqueue() */
+	return prev_cpu;
+}
+
+void BPF_STRUCT_OPS(dequeue_enqueue, struct task_struct *p, u64 enq_flags)
+{
+	struct task_ctx *tctx;
+
+	tctx = try_lookup_task_ctx(p);
+	if (!tctx)
+		return;
+
+	switch (test_scenario) {
+	case 0:
+		/*
+		 * Scenario 0: Direct dispatch to the local DSQ.
+		 *
+		 * Task bypasses BPF scheduler entirely - no enqueue
+		 * tracking, no dequeue callbacks. Don't increment counters
+		 * or validate state since the task never enters BPF
+		 * scheduler management.
+		 */
+		scx_bpf_dsq_insert(p, SCX_DSQ_LOCAL, SCX_SLICE_DFL, enq_flags);
+		break;
+
+	case 1:
+		/*
+		 * Scenario 1: Dispatch to shared DSQ.
+		 *
+		 * Task enters BPF scheduler management - track
+		 * enqueue/dequeue lifecycle and validate state
+		 * transitions.
+		 */
+		__sync_fetch_and_add(&enqueue_cnt, 1);
+
+		/*
+		 * Validate state transition: enqueue is only valid from
+		 * NONE or DISPATCHED states. Getting enqueue while in
+		 * ENQUEUED state indicates a missing dequeue.
+		 */
+		if (tctx->state == TASK_ENQUEUED)
+			scx_bpf_error("%d (%s): enqueue while in ENQUEUED state seq=%llu",
+				      p->pid, p->comm, tctx->enqueue_seq);
+
+		/* Transition to ENQUEUED state */
+		tctx->state = TASK_ENQUEUED;
+		tctx->enqueue_seq++;
+
+		scx_bpf_dsq_insert(p, SHARED_DSQ, SCX_SLICE_DFL, enq_flags);
+		break;
+	}
+}
+
+void BPF_STRUCT_OPS(dequeue_dequeue, struct task_struct *p, u64 deq_flags)
+{
+	struct task_ctx *tctx;
+
+	__sync_fetch_and_add(&dequeue_cnt, 1);
+
+	tctx = try_lookup_task_ctx(p);
+	if (!tctx)
+		return;
+
+	/*
+	 * For scenario 0 (local DSQ), ops.dequeue() should never be called
+	 * because tasks bypass the BPF scheduler entirely. If we get here,
+	 * it's a kernel bug. We don't track enqueues for scenario 0, so
+	 * tctx->enqueue_seq will be 0.
+	 */
+	if (test_scenario == 0) {
+		scx_bpf_error("%d (%s): dequeue called for local DSQ scenario - kernel bug!",
+			      p->pid, p->comm);
+		return;
+	}
+
+	/*
+	 * Validate state: dequeue should only happen from ENQUEUED or
+	 * DISPATCHED states. Getting dequeue from NONE indicates a bug.
+	 */
+	if (tctx->state == TASK_NONE) {
+		scx_bpf_error("%d (%s): dequeue from NONE state seq=%llu",
+			      p->pid, p->comm, tctx->enqueue_seq);
+		return;
+	}
+
+	if (deq_flags & SCX_DEQ_SCHED_CHANGE) {
+		/*
+		 * Property change interrupting the workflow. Valid from
+		 * both ENQUEUED and DISPATCHED states. Transitions task
+		 * back to NONE state.
+		 */
+		__sync_fetch_and_add(&change_dequeue_cnt, 1);
+
+		/* Validate state transition */
+		if (tctx->state != TASK_ENQUEUED && tctx->state != TASK_DISPATCHED)
+			scx_bpf_error("%d (%s): invalid property change dequeue state=%d seq=%llu",
+				      p->pid, p->comm, tctx->state, tctx->enqueue_seq);
+
+		/* Transition back to NONE: task outside scheduler control */
+		tctx->state = TASK_NONE;
+	} else {
+		/*
+		 * Regular dispatch dequeue: normal workflow step. Valid
+		 * only from ENQUEUED state (after enqueue, before dispatch
+		 * dequeue). Transitions to DISPATCHED state.
+		 */
+		__sync_fetch_and_add(&dispatch_dequeue_cnt, 1);
+
+		/* Dispatch dequeue should not have %SCX_DEQ_SCHED_CHANGE flag */
+		if (deq_flags & SCX_DEQ_SCHED_CHANGE)
+			scx_bpf_error("%d (%s): SCX_DEQ_SCHED_CHANGE in dispatch dequeue seq=%llu",
+				      p->pid, p->comm, tctx->enqueue_seq);
+
+		/* Must be in ENQUEUED state */
+		if (tctx->state != TASK_ENQUEUED)
+			scx_bpf_error("%d (%s): dispatch dequeue from state %d seq=%llu",
+				      p->pid, p->comm, tctx->state, tctx->enqueue_seq);
+
+		/* Transition to DISPATCHED: normal cycle completed dispatch */
+		tctx->state = TASK_DISPATCHED;
+	}
+}
+
+void BPF_STRUCT_OPS(dequeue_dispatch, s32 cpu, struct task_struct *prev)
+{
+	scx_bpf_dsq_move_to_local(SHARED_DSQ);
+}
+
+s32 BPF_STRUCT_OPS(dequeue_init_task, struct task_struct *p,
+		   struct scx_init_task_args *args)
+{
+	struct task_ctx *tctx;
+
+	tctx = bpf_task_storage_get(&task_ctx_stor, p, 0,
+				   BPF_LOCAL_STORAGE_GET_F_CREATE);
+	if (!tctx)
+		return -ENOMEM;
+
+	return 0;
+}
+
+s32 BPF_STRUCT_OPS_SLEEPABLE(dequeue_init)
+{
+	s32 ret;
+
+	ret = scx_bpf_create_dsq(SHARED_DSQ, -1);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+void BPF_STRUCT_OPS(dequeue_exit, struct scx_exit_info *ei)
+{
+	UEI_RECORD(uei, ei);
+}
+
+SEC(".struct_ops.link")
+struct sched_ext_ops dequeue_ops = {
+	.select_cpu		= (void *)dequeue_select_cpu,
+	.enqueue		= (void *)dequeue_enqueue,
+	.dequeue		= (void *)dequeue_dequeue,
+	.dispatch		= (void *)dequeue_dispatch,
+	.init_task		= (void *)dequeue_init_task,
+	.init			= (void *)dequeue_init,
+	.exit			= (void *)dequeue_exit,
+	.name			= "dequeue_test",
+};
diff --git a/tools/testing/selftests/sched_ext/dequeue.c b/tools/testing/selftests/sched_ext/dequeue.c
new file mode 100644
index 0000000000000..805970834612b
--- /dev/null
+++ b/tools/testing/selftests/sched_ext/dequeue.c
@@ -0,0 +1,201 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2025 NVIDIA Corporation.
+ */
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <unistd.h>
+#include <signal.h>
+#include <bpf/bpf.h>
+#include <scx/common.h>
+#include <sys/wait.h>
+#include <sched.h>
+#include <pthread.h>
+#include "scx_test.h"
+#include "dequeue.bpf.skel.h"
+
+#define NUM_WORKERS 8
+
+/*
+ * Worker function that creates enqueue/dequeue events. It alternates
+ * between CPU work, sleeping, and affinity changes to trigger dequeues.
+ */
+static void worker_fn(int id)
+{
+	cpu_set_t cpuset;
+	int i;
+	volatile int sum = 0;
+
+	for (i = 0; i < 1000; i++) {
+		int j;
+
+		/* Do some work to trigger scheduling events */
+		for (j = 0; j < 10000; j++)
+			sum += j;
+
+		/* Change affinity to trigger dequeue */
+		if (i % 10 == 0) {
+			CPU_ZERO(&cpuset);
+			/* Rotate through the first 4 CPUs */
+			CPU_SET(i % 4, &cpuset);
+			sched_setaffinity(0, sizeof(cpuset), &cpuset);
+		}
+
+		/* Do additional work */
+		for (j = 0; j < 10000; j++)
+			sum += j;
+
+		/* Sleep to trigger dequeue */
+		usleep(1000 + (id * 100));
+	}
+
+	exit(0);
+}
+
+static enum scx_test_status run_scenario(struct dequeue *skel, u32 scenario,
+					 const char *scenario_name)
+{
+	struct bpf_link *link;
+	pid_t pids[NUM_WORKERS];
+	int i, status;
+	u64 enq_start, deq_start, dispatch_deq_start, change_deq_start;
+	u64 enq_delta, deq_delta, dispatch_deq_delta, change_deq_delta;
+
+	/* Set the test scenario */
+	skel->bss->test_scenario = scenario;
+
+	/* Record starting counts */
+	enq_start = skel->bss->enqueue_cnt;
+	deq_start = skel->bss->dequeue_cnt;
+	dispatch_deq_start = skel->bss->dispatch_dequeue_cnt;
+	change_deq_start = skel->bss->change_dequeue_cnt;
+
+	link = bpf_map__attach_struct_ops(skel->maps.dequeue_ops);
+	SCX_FAIL_IF(!link, "Failed to attach struct_ops for scenario %s", scenario_name);
+
+	/* Fork worker processes to generate enqueue/dequeue events */
+	for (i = 0; i < NUM_WORKERS; i++) {
+		pids[i] = fork();
+		SCX_FAIL_IF(pids[i] < 0, "Failed to fork worker %d", i);
+
+		if (pids[i] == 0) {
+			worker_fn(i);
+			/* Should not reach here */
+			exit(1);
+		}
+	}
+
+	/* Wait for all workers to complete */
+	for (i = 0; i < NUM_WORKERS; i++) {
+		SCX_FAIL_IF(waitpid(pids[i], &status, 0) != pids[i],
+			    "Failed to wait for worker %d", i);
+		SCX_FAIL_IF(status != 0, "Worker %d exited with status %d", i, status);
+	}
+
+	bpf_link__destroy(link);
+
+	SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_UNREG));
+
+	/* Calculate deltas */
+	enq_delta = skel->bss->enqueue_cnt - enq_start;
+	deq_delta = skel->bss->dequeue_cnt - deq_start;
+	dispatch_deq_delta = skel->bss->dispatch_dequeue_cnt - dispatch_deq_start;
+	change_deq_delta = skel->bss->change_dequeue_cnt - change_deq_start;
+
+	printf("%s:\n", scenario_name);
+	printf("  enqueues: %lu\n", (unsigned long)enq_delta);
+	printf("  dequeues: %lu (dispatch: %lu, property_change: %lu)\n",
+	       (unsigned long)deq_delta,
+	       (unsigned long)dispatch_deq_delta,
+	       (unsigned long)change_deq_delta);
+
+	/*
+	 * Validate enqueue/dequeue lifecycle tracking.
+	 *
+	 * For scenario 0 (Local DSQ), both enqueues and dequeues should be
+	 * 0 because tasks bypass the BPF scheduler entirely: they never
+	 * enter BPF scheduler's custody. For scenario 1 (ser DSQ) , we
+	 * expect both enqueues and dequeues.
+	 *
+	 * The BPF code does strict state machine validation with
+	 * scx_bpf_error() to ensure the workflow semantics are correct. If
+	 * we reach here without errors, the semantics are validated
+	 * correctly.
+	 */
+	if (scenario == 0) {
+		/* Local DSQ: tasks bypass BPF scheduler completely */
+		SCX_EQ(enq_delta, 0);
+		SCX_EQ(deq_delta, 0);
+		SCX_EQ(dispatch_deq_delta, 0);
+		SCX_EQ(change_deq_delta, 0);
+	} else {
+		/* Non-local DSQ: tasks enter BPF scheduler's custody */
+		SCX_GT(enq_delta, 0);
+		SCX_GT(deq_delta, 0);
+		/* Validate 1:1 enqueue/dequeue pairing */
+		SCX_EQ(enq_delta, deq_delta);
+	}
+
+	return SCX_TEST_PASS;
+}
+
+static enum scx_test_status setup(void **ctx)
+{
+	struct dequeue *skel;
+
+	skel = dequeue__open();
+	SCX_FAIL_IF(!skel, "Failed to open skel");
+	SCX_ENUM_INIT(skel);
+	SCX_FAIL_IF(dequeue__load(skel), "Failed to load skel");
+
+	*ctx = skel;
+
+	return SCX_TEST_PASS;
+}
+
+static enum scx_test_status run(void *ctx)
+{
+	struct dequeue *skel = ctx;
+	enum scx_test_status status;
+
+	status = run_scenario(skel, 0, "Local DSQ (direct dispatch)");
+	if (status != SCX_TEST_PASS)
+		return status;
+
+	status = run_scenario(skel, 1, "User DSQ");
+	if (status != SCX_TEST_PASS)
+		return status;
+
+	printf("\n=== Summary ===\n");
+	printf("Total enqueues: %lu\n", (unsigned long)skel->bss->enqueue_cnt);
+	printf("Total dequeues: %lu\n", (unsigned long)skel->bss->dequeue_cnt);
+	printf("  Dispatch dequeues: %lu (no flag, normal workflow)\n",
+	       (unsigned long)skel->bss->dispatch_dequeue_cnt);
+	printf("  Property change dequeues: %lu (SCX_DEQ_SCHED_CHANGE flag)\n",
+	       (unsigned long)skel->bss->change_dequeue_cnt);
+	printf("\nAll scenarios passed - no state machine violations detected\n");
+	printf("-> Validated: Local DSQ dispatch bypasses BPF scheduler (no dequeue callbacks)\n");
+	printf("-> Validated: Non-local DSQ dispatch triggers dequeue callbacks\n");
+	printf("-> Validated: Dispatch dequeues have no flags (normal workflow)\n");
+	printf("-> Validated: Async dequeues have SCX_DEQ_SCHED_CHANGE flag (interruptions)\n");
+	printf("-> Validated: No duplicate enqueues or invalid state transitions\n");
+
+	return SCX_TEST_PASS;
+}
+
+static void cleanup(void *ctx)
+{
+	struct dequeue *skel = ctx;
+
+	dequeue__destroy(skel);
+}
+
+struct scx_test dequeue_test = {
+	.name = "dequeue",
+	.description = "Verify ops.dequeue() semantics for local and non-local DSQ dispatch",
+	.setup = setup,
+	.run = run,
+	.cleanup = cleanup,
+};
+
+REGISTER_SCX_TEST(&dequeue_test)
-- 
2.52.0

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Christian Loehle]

On 2/1/26 09:08, Andrea Righi wrote:
> Currently, ops.dequeue() is only invoked when the sched_ext core knows
> that a task resides in BPF-managed data structures, which causes it to
> miss scheduling property change events. In addition, ops.dequeue()
> callbacks are completely skipped when tasks are dispatched to non-local
> DSQs from ops.select_cpu(). As a result, BPF schedulers cannot reliably
> track task state.
> 
> Fix this by guaranteeing that each task entering the BPF scheduler's
> custody triggers exactly one ops.dequeue() call when it leaves that
> custody, whether the exit is due to a dispatch (regular or via a core
> scheduling pick) or to a scheduling property change (e.g.
> sched_setaffinity(), sched_setscheduler(), set_user_nice(), NUMA
> balancing, etc.).
> 
> BPF scheduler custody concept: a task is considered to be in "BPF
> scheduler's custody" when it has been queued in BPF-managed data
> structures and the BPF scheduler is responsible for its lifecycle.
> Custody ends when the task is dispatched to a local DSQ, selected by
> core scheduling, or removed due to a property change.
> 
> Tasks directly dispatched to local DSQs (via %SCX_DSQ_LOCAL or
> %SCX_DSQ_LOCAL_ON) bypass the BPF scheduler entirely and are not in its
> custody. As a result, ops.dequeue() is not invoked for these tasks.
> 
> To identify dequeues triggered by scheduling property changes, introduce
> the new ops.dequeue() flag %SCX_DEQ_SCHED_CHANGE: when this flag is set,
> the dequeue was caused by a scheduling property change.
> 
> New ops.dequeue() semantics:
>  - ops.dequeue() is invoked exactly once when the task leaves the BPF
>    scheduler's custody, in one of the following cases:
>    a) regular dispatch: task was dispatched to a non-local DSQ (global
>       or user DSQ), ops.dequeue() called without any special flags set
>    b) core scheduling dispatch: core-sched picks task before dispatch,
>       dequeue called with %SCX_DEQ_CORE_SCHED_EXEC flag set
>    c) property change: task properties modified before dispatch,
>       dequeue called with %SCX_DEQ_SCHED_CHANGE flag set
> 
> This allows BPF schedulers to:
>  - reliably track task ownership and lifecycle,
>  - maintain accurate accounting of managed tasks,
>  - update internal state when tasks change properties.
> 

So I have finally gotten around updating scx_storm to the new semantics,
see:
https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics

I don't think the new ops.dequeue() are enough to make inserts to local-on
from anywhere safe, because it's still racing with dequeue from another CPU?

Furthermore I can reproduce the following with this patch applied quite easily
with something like

hackbench -l 1000 & timeout 10 ./build/scheds/c/scx_storm

[   44.356878] sched_ext: BPF scheduler "simple" enabled
[   59.315370] sched_ext: BPF scheduler "simple" disabled (unregistered from user space)
[   85.366747] sched_ext: BPF scheduler "storm" enabled
[   85.371324] ------------[ cut here ]------------
[   85.373370] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#5: gmain/1111
[   85.373392] Modules linked in: qrtr
[   85.380088] ------------[ cut here ]------------
[   85.380719] ------------[ cut here ]------------
[   85.380722] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#10: kworker/u48:1/82
[   85.380728] Modules linked in: qrtr 8021q garp mrp stp llc binfmt_misc sm3_ce r8169 cdns3_pci_wrap nf_tables nfnetlink fuse dm_mod ipv6
[   85.380745] CPU: 10 UID: 0 PID: 82 Comm: kworker/u48:1 Tainted: G S                  6.19.0-rc7-cix-build+ #256 PREEMPT 
[   85.380749] Tainted: [S]=CPU_OUT_OF_SPEC
[   85.380750] Hardware name: Radxa Computer (Shenzhen) Co., Ltd. Radxa Orion O6/Radxa Orion O6, BIOS 1.1.0-1 2025-12-25T02:55:53+00:00
[   85.380754] Workqueue:  0x0 (events_unbound)
[   85.380760] Sched_ext: storm (enabled+all), task: runnable_at=+0ms
[   85.380762] pstate: 634000c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
[   85.380764] pc : update_locked_rq+0x64/0x6c
[   85.380767] lr : update_locked_rq+0x60/0x6c
[   85.380769] sp : ffff8000803a3bd0
[   85.380770] x29: ffff8000803a3bd0 x28: fffffdffbf622dc0 x27: ffff0000911e5040
[   85.380773] x26: 0000000000000000 x25: ffffd204426cad80 x24: ffffd20442ba5bb8
[   85.380776] x23: c00000000000000a x22: 0000000000000000 x21: ffffd20442ba4830
[   85.380778] x20: ffff00009af0b000 x19: ffff0001fef2ed80 x18: 0000000000000000
[   85.380781] x17: 0000000000000000 x16: 0000000000000000 x15: 0000aaaadd996940
[   85.380783] x14: 0000000000000000 x13: 00000000000a0000 x12: 0000000000000000
[   85.380786] x11: 0000000000000040 x10: ffffd204402e7ca0 x9 : ffffd2044324b000
[   85.380788] x8 : ffff0000810e0000 x7 : 0000d00202cc2dc0 x6 : 0000000000000050
[   85.380790] x5 : ffffd204426b5648 x4 : fffffdffbf622dc0 x3 : ffff0000810e0000
[   85.380793] x2 : 0000000000000002 x1 : ffff2dfdbc960000 x0 : 0000000000000000
[   85.380795] Call trace:
[   85.380796]  update_locked_rq+0x64/0x6c (P)
[   85.380799]  flush_dispatch_buf+0x2a8/0x2dc
[   85.380801]  pick_task_scx+0x2b0/0x6d4
[   85.380804]  __schedule+0x62c/0x1060
[   85.380811]  schedule+0x48/0x15c
[   85.380813]  worker_thread+0xdc/0x358
[   85.380824]  kthread+0x134/0x1fc
[   85.380831]  ret_from_fork+0x10/0x20
[   85.380839] irq event stamp: 34386
[   85.380840] hardirqs last  enabled at (34385): [<ffffd20441511408>] _raw_spin_unlock_irq+0x30/0x6c
[   85.380850] hardirqs last disabled at (34386): [<ffffd20441507100>] __schedule+0x510/0x1060
[   85.380852] softirqs last  enabled at (34014): [<ffffd204400c7280>] handle_softirqs+0x514/0x52c
[   85.380865] softirqs last disabled at (34007): [<ffffd204400105c4>] __do_softirq+0x14/0x20
[   85.380867] ---[ end trace 0000000000000000 ]---
[   85.380969] ------------[ cut here ]------------
[   85.380970] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#10: kworker/u48:1/82
[   85.380974] Modules linked in: qrtr 8021q garp mrp stp llc binfmt_misc sm3_ce r8169 cdns3_pci_wrap nf_tables nfnetlink fuse dm_mod ipv6
[   85.380984] CPU: 10 UID: 0 PID: 82 Comm: kworker/u48:1 Tainted: G S      W           6.19.0-rc7-cix-build+ #256 PREEMPT 
[   85.380987] Tainted: [S]=CPU_OUT_OF_SPEC, [W]=WARN
[   85.380988] Hardware name: Radxa Computer (Shenzhen) Co., Ltd. Radxa Orion O6/Radxa Orion O6, BIOS 1.1.0-1 2025-12-25T02:55:53+00:00
[   85.380990] Workqueue:  0x0 (events_unbound)
[   85.380993] Sched_ext: storm (enabled+all), task: runnable_at=+0ms
[   85.380994] pstate: 634000c9 (nZCv daIF +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
[   85.380996] pc : update_locked_rq+0x64/0x6c
[   85.380997] lr : update_locked_rq+0x60/0x6c
[   85.380999] sp : ffff8000803a3bd0
[   85.381000] x29: ffff8000803a3bd0 x28: fffffdffbf622dc0 x27: ffff00009151b580
[   85.381002] x26: 0000000000000000 x25: ffffd204426cad80 x24: ffffd20442ba5bb8
[   85.381005] x23: c00000000000000a x22: 0000000000000000 x21: ffffd20442ba4830
[   85.381007] x20: ffff00009af0b000 x19: ffff0001fef52d80 x18: 0000000000000000
[   85.381009] x17: 0000000000000000 x16: 0000000000000000 x15: 0000aaaae6917960
[   85.381012] x14: 0000000000000000 x13: 00000000000a0000 x12: 0000000000000000
[   85.381014] x11: 0000000000000040 x10: ffffd204402e7ca0 x9 : ffffd2044324b000
[   85.381016] x8 : ffff0000810e0000 x7 : 0000d00202cc2dc0 x6 : 0000000000000050
[   85.381019] x5 : ffffd204426b5648 x4 : fffffdffbf622dc0 x3 : ffff0000810e0000
[   85.381021] x2 : 0000000000000002 x1 : ffff2dfdbc960000 x0 : 0000000000000000
[   85.381023] Call trace:
[   85.381024]  update_locked_rq+0x64/0x6c (P)
[   85.381026]  flush_dispatch_buf+0x2a8/0x2dc
[   85.381028]  pick_task_scx+0x2b0/0x6d4
[   85.381030]  __schedule+0x62c/0x1060
[   85.381032]  schedule+0x48/0x15c
[   85.381034]  worker_thread+0xdc/0x358
[   85.381036]  kthread+0x134/0x1fc
[   85.381039]  ret_from_fork+0x10/0x20
[   85.381041] irq event stamp: 34394
[   85.381042] hardirqs last  enabled at (34393): [<ffffd20441511408>] _raw_spin_unlock_irq+0x30/0x6c
[   85.381044] hardirqs last disabled at (34394): [<ffffd20441507100>] __schedule+0x510/0x1060
[   85.381046] softirqs last  enabled at (34014): [<ffffd204400c7280>] handle_softirqs+0x514/0x52c
[   85.381049] softirqs last disabled at (34007): [<ffffd204400105c4>] __do_softirq+0x14/0x20
[   85.381050] ---[ end trace 0000000000000000 ]---
[   85.381199] ------------[ cut here ]------------
[   85.381201] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#10: kworker/u48:1/82

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

Hi Christian,

On Sun, Feb 01, 2026 at 10:47:22PM +0000, Christian Loehle wrote:
> On 2/1/26 09:08, Andrea Righi wrote:
> > Currently, ops.dequeue() is only invoked when the sched_ext core knows
> > that a task resides in BPF-managed data structures, which causes it to
> > miss scheduling property change events. In addition, ops.dequeue()
> > callbacks are completely skipped when tasks are dispatched to non-local
> > DSQs from ops.select_cpu(). As a result, BPF schedulers cannot reliably
> > track task state.
> > 
> > Fix this by guaranteeing that each task entering the BPF scheduler's
> > custody triggers exactly one ops.dequeue() call when it leaves that
> > custody, whether the exit is due to a dispatch (regular or via a core
> > scheduling pick) or to a scheduling property change (e.g.
> > sched_setaffinity(), sched_setscheduler(), set_user_nice(), NUMA
> > balancing, etc.).
> > 
> > BPF scheduler custody concept: a task is considered to be in "BPF
> > scheduler's custody" when it has been queued in BPF-managed data
> > structures and the BPF scheduler is responsible for its lifecycle.
> > Custody ends when the task is dispatched to a local DSQ, selected by
> > core scheduling, or removed due to a property change.
> > 
> > Tasks directly dispatched to local DSQs (via %SCX_DSQ_LOCAL or
> > %SCX_DSQ_LOCAL_ON) bypass the BPF scheduler entirely and are not in its
> > custody. As a result, ops.dequeue() is not invoked for these tasks.
> > 
> > To identify dequeues triggered by scheduling property changes, introduce
> > the new ops.dequeue() flag %SCX_DEQ_SCHED_CHANGE: when this flag is set,
> > the dequeue was caused by a scheduling property change.
> > 
> > New ops.dequeue() semantics:
> >  - ops.dequeue() is invoked exactly once when the task leaves the BPF
> >    scheduler's custody, in one of the following cases:
> >    a) regular dispatch: task was dispatched to a non-local DSQ (global
> >       or user DSQ), ops.dequeue() called without any special flags set
> >    b) core scheduling dispatch: core-sched picks task before dispatch,
> >       dequeue called with %SCX_DEQ_CORE_SCHED_EXEC flag set
> >    c) property change: task properties modified before dispatch,
> >       dequeue called with %SCX_DEQ_SCHED_CHANGE flag set
> > 
> > This allows BPF schedulers to:
> >  - reliably track task ownership and lifecycle,
> >  - maintain accurate accounting of managed tasks,
> >  - update internal state when tasks change properties.
> > 
> 
> So I have finally gotten around updating scx_storm to the new semantics,
> see:
> https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics
> 
> I don't think the new ops.dequeue() are enough to make inserts to local-on
> from anywhere safe, because it's still racing with dequeue from another CPU?

Yeah, with this patch set BPF schedulers get proper ops.dequeue()
callbacks, but we're not fixing the usage of SCX_DSQ_LOCAL_ON from
ops.dispatch().

When task properties change between scx_bpf_dsq_insert() and the actual
dispatch, task_can_run_on_remote_rq() can still trigger a fatal
scx_error().

The ops.dequeue(SCX_DEQ_SCHED_CHANGE) notifications happens after the
property change, so it can't prevent already-queued dispatches from
failing. The race window is between ops.dispatch() returning and
dispatch_to_local_dsq() executing.

We can address this in a separate patch set. One thing at a time. :)

> 
> Furthermore I can reproduce the following with this patch applied quite easily
> with something like
> 
> hackbench -l 1000 & timeout 10 ./build/scheds/c/scx_storm
> 
> [   44.356878] sched_ext: BPF scheduler "simple" enabled
> [   59.315370] sched_ext: BPF scheduler "simple" disabled (unregistered from user space)
> [   85.366747] sched_ext: BPF scheduler "storm" enabled
> [   85.371324] ------------[ cut here ]------------
> [   85.373370] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#5: gmain/1111

Ah yes! I think I see it, can you try this on top?

Thanks,
-Andrea

 kernel/sched/ext.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 6d6f1253039d8..d8fed4a49195d 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -2248,7 +2248,7 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
 			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
 		} else {
 			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
-				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, task_rq(p), p, 0);
+				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
 
 			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
 		}

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

On Mon, Feb 02, 2026 at 08:45:18AM +0100, Andrea Righi wrote:
...
> > So I have finally gotten around updating scx_storm to the new semantics,
> > see:
> > https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics
> > 
> > I don't think the new ops.dequeue() are enough to make inserts to local-on
> > from anywhere safe, because it's still racing with dequeue from another CPU?
> 
> Yeah, with this patch set BPF schedulers get proper ops.dequeue()
> callbacks, but we're not fixing the usage of SCX_DSQ_LOCAL_ON from
> ops.dispatch().
> 
> When task properties change between scx_bpf_dsq_insert() and the actual
> dispatch, task_can_run_on_remote_rq() can still trigger a fatal
> scx_error().
> 
> The ops.dequeue(SCX_DEQ_SCHED_CHANGE) notifications happens after the
> property change, so it can't prevent already-queued dispatches from
> failing. The race window is between ops.dispatch() returning and
> dispatch_to_local_dsq() executing.
> 
> We can address this in a separate patch set. One thing at a time. :)

Thinking more on this, the problem is that we're passing enforce=true to
task_can_run_on_remote_rq(), triggering a critical failure - scx_error().
There's a logic in task_can_run_on_remote_rq() to fallback to the global
DSQ, that doesn't happen if we pass enforce=true, due to scx_error().

However, instead of the global DSQ fallback, I was wondering if it'd be
better to simply re-enqueue the task - setting SCX_ENQ_REENQ - if the
target local DSQ isn't valid anymore when the dispatch is finalized.

In this way using SCX_DSQ_LOCAL_ON | cpu from ops.dispatch() would simply
trigger a re-enqueue when "cpu" isn't valid anymore (due to concurrent
affinity / migration disabled changes) and the BPF scheduler can handle
that in another ops.enqueue().

What do you think?

Thanks,
-Andrea

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Christian Loehle]

On 2/2/26 09:26, Andrea Righi wrote:
> On Mon, Feb 02, 2026 at 08:45:18AM +0100, Andrea Righi wrote:
> ...
>>> So I have finally gotten around updating scx_storm to the new semantics,
>>> see:
>>> https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics
>>>
>>> I don't think the new ops.dequeue() are enough to make inserts to local-on
>>> from anywhere safe, because it's still racing with dequeue from another CPU?
>>
>> Yeah, with this patch set BPF schedulers get proper ops.dequeue()
>> callbacks, but we're not fixing the usage of SCX_DSQ_LOCAL_ON from
>> ops.dispatch().
>>
>> When task properties change between scx_bpf_dsq_insert() and the actual
>> dispatch, task_can_run_on_remote_rq() can still trigger a fatal
>> scx_error().
>>
>> The ops.dequeue(SCX_DEQ_SCHED_CHANGE) notifications happens after the
>> property change, so it can't prevent already-queued dispatches from
>> failing. The race window is between ops.dispatch() returning and
>> dispatch_to_local_dsq() executing.
>>
>> We can address this in a separate patch set. One thing at a time. :)
> 
> Thinking more on this, the problem is that we're passing enforce=true to
> task_can_run_on_remote_rq(), triggering a critical failure - scx_error().
> There's a logic in task_can_run_on_remote_rq() to fallback to the global
> DSQ, that doesn't happen if we pass enforce=true, due to scx_error().
> 
> However, instead of the global DSQ fallback, I was wondering if it'd be
> better to simply re-enqueue the task - setting SCX_ENQ_REENQ - if the
> target local DSQ isn't valid anymore when the dispatch is finalized.
> 
> In this way using SCX_DSQ_LOCAL_ON | cpu from ops.dispatch() would simply
> trigger a re-enqueue when "cpu" isn't valid anymore (due to concurrent
> affinity / migration disabled changes) and the BPF scheduler can handle
> that in another ops.enqueue().
> 
> What do you think?

I think that's a lot more versatile for the BPF scheduler than using the
global DSQ as fallback in that case, so yeah I'm all for it!

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Christian Loehle]

On 2/2/26 07:45, Andrea Righi wrote:
> Hi Christian,
> 
> On Sun, Feb 01, 2026 at 10:47:22PM +0000, Christian Loehle wrote:
>> On 2/1/26 09:08, Andrea Righi wrote:
>>> Currently, ops.dequeue() is only invoked when the sched_ext core knows
>>> that a task resides in BPF-managed data structures, which causes it to
>>> miss scheduling property change events. In addition, ops.dequeue()
>>> callbacks are completely skipped when tasks are dispatched to non-local
>>> DSQs from ops.select_cpu(). As a result, BPF schedulers cannot reliably
>>> track task state.
>>>
>>> Fix this by guaranteeing that each task entering the BPF scheduler's
>>> custody triggers exactly one ops.dequeue() call when it leaves that
>>> custody, whether the exit is due to a dispatch (regular or via a core
>>> scheduling pick) or to a scheduling property change (e.g.
>>> sched_setaffinity(), sched_setscheduler(), set_user_nice(), NUMA
>>> balancing, etc.).
>>>
>>> BPF scheduler custody concept: a task is considered to be in "BPF
>>> scheduler's custody" when it has been queued in BPF-managed data
>>> structures and the BPF scheduler is responsible for its lifecycle.
>>> Custody ends when the task is dispatched to a local DSQ, selected by
>>> core scheduling, or removed due to a property change.
>>>
>>> Tasks directly dispatched to local DSQs (via %SCX_DSQ_LOCAL or
>>> %SCX_DSQ_LOCAL_ON) bypass the BPF scheduler entirely and are not in its
>>> custody. As a result, ops.dequeue() is not invoked for these tasks.
>>>
>>> To identify dequeues triggered by scheduling property changes, introduce
>>> the new ops.dequeue() flag %SCX_DEQ_SCHED_CHANGE: when this flag is set,
>>> the dequeue was caused by a scheduling property change.
>>>
>>> New ops.dequeue() semantics:
>>>  - ops.dequeue() is invoked exactly once when the task leaves the BPF
>>>    scheduler's custody, in one of the following cases:
>>>    a) regular dispatch: task was dispatched to a non-local DSQ (global
>>>       or user DSQ), ops.dequeue() called without any special flags set
>>>    b) core scheduling dispatch: core-sched picks task before dispatch,
>>>       dequeue called with %SCX_DEQ_CORE_SCHED_EXEC flag set
>>>    c) property change: task properties modified before dispatch,
>>>       dequeue called with %SCX_DEQ_SCHED_CHANGE flag set
>>>
>>> This allows BPF schedulers to:
>>>  - reliably track task ownership and lifecycle,
>>>  - maintain accurate accounting of managed tasks,
>>>  - update internal state when tasks change properties.
>>>
>>
>> So I have finally gotten around updating scx_storm to the new semantics,
>> see:
>> https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics
>>
>> I don't think the new ops.dequeue() are enough to make inserts to local-on
>> from anywhere safe, because it's still racing with dequeue from another CPU?
> 
> Yeah, with this patch set BPF schedulers get proper ops.dequeue()
> callbacks, but we're not fixing the usage of SCX_DSQ_LOCAL_ON from
> ops.dispatch().
> 
> When task properties change between scx_bpf_dsq_insert() and the actual
> dispatch, task_can_run_on_remote_rq() can still trigger a fatal
> scx_error().
> 
> The ops.dequeue(SCX_DEQ_SCHED_CHANGE) notifications happens after the
> property change, so it can't prevent already-queued dispatches from
> failing. The race window is between ops.dispatch() returning and
> dispatch_to_local_dsq() executing.
> 
> We can address this in a separate patch set. One thing at a time. :)
> 
>>
>> Furthermore I can reproduce the following with this patch applied quite easily
>> with something like
>>
>> hackbench -l 1000 & timeout 10 ./build/scheds/c/scx_storm
>>
>> [   44.356878] sched_ext: BPF scheduler "simple" enabled
>> [   59.315370] sched_ext: BPF scheduler "simple" disabled (unregistered from user space)
>> [   85.366747] sched_ext: BPF scheduler "storm" enabled
>> [   85.371324] ------------[ cut here ]------------
>> [   85.373370] WARNING: kernel/sched/sched.h:1571 at update_locked_rq+0x64/0x6c, CPU#5: gmain/1111
> 
> Ah yes! I think I see it, can you try this on top?
> 
> Thanks,
> -Andrea
> 
>  kernel/sched/ext.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 6d6f1253039d8..d8fed4a49195d 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -2248,7 +2248,7 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
>  			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
>  		} else {
>  			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
> -				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, task_rq(p), p, 0);
> +				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
>  
>  			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
>  		}

Yup, that fixes it!

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Kuba Piecuch]

Hi Andrea,

Looks good overall, but we need to settle on the global DSQ semantics, plus
some edge cases that need clearing up.

On Sun Feb 1, 2026 at 9:08 AM UTC, Andrea Righi wrote:
> diff --git a/Documentation/scheduler/sched-ext.rst b/Documentation/scheduler/sched-ext.rst
> index 404fe6126a769..6d9e82e6ca9d4 100644
> --- a/Documentation/scheduler/sched-ext.rst
> +++ b/Documentation/scheduler/sched-ext.rst
> @@ -252,6 +252,80 @@ The following briefly shows how a waking task is scheduled and executed.
>  
>     * Queue the task on the BPF side.
>  
> +   **Task State Tracking and ops.dequeue() Semantics**
> +
> +   Once ``ops.select_cpu()`` or ``ops.enqueue()`` is called, the task may
> +   enter the "BPF scheduler's custody" depending on where it's dispatched:
> +
> +   * **Direct dispatch to local DSQs** (``SCX_DSQ_LOCAL`` or
> +     ``SCX_DSQ_LOCAL_ON | cpu``): The task bypasses the BPF scheduler
> +     entirely and goes straight to the CPU's local run queue. The task
> +     never enters BPF custody, and ``ops.dequeue()`` will not be called.
> +
> +   * **Dispatch to non-local DSQs** (``SCX_DSQ_GLOBAL`` or custom DSQs):
> +     the task enters the BPF scheduler's custody. When the task later
> +     leaves BPF custody (dispatched to a local DSQ, picked by core-sched,
> +     or dequeued for sleep/property changes), ``ops.dequeue()`` will be
> +     called exactly once.
> +
> +   * **Queued on BPF side**: The task is in BPF data structures and in BPF
> +     custody, ``ops.dequeue()`` will be called when it leaves.
> +
> +   The key principle: **ops.dequeue() is called when a task leaves the BPF
> +   scheduler's custody**. A task is in BPF custody if it's on a non-local
> +   DSQ or in BPF data structures. Once dispatched to a local DSQ or after
> +   ops.dequeue() is called, the task is out of BPF custody and the BPF
> +   scheduler no longer needs to track it.
> +
> +   This works correctly with the ``ops.select_cpu()`` direct dispatch
> +   optimization: even though it skips ``ops.enqueue()`` invocation, if the
> +   task is dispatched to a non-local DSQ, it enters BPF custody and will
> +   get ``ops.dequeue()`` when it leaves. This provides the performance
> +   benefit of avoiding the ``ops.enqueue()`` roundtrip while maintaining
> +   correct state tracking.
> +
> +   The dequeue can happen for different reasons, distinguished by flags:
> +
> +   1. **Regular dispatch workflow**: when the task is dispatched from a
> +      non-local DSQ to a local DSQ (leaving BPF custody for execution),
> +      ``ops.dequeue()`` is triggered without any special flags.

Maybe add a note that this can happen asynchronously, without the BPF
scheduler explicitly dispatching the task to a local DSQ, when the task
is on a global DSQ? Or maybe make that case into a separate dequeue reason
with its own flag, e.g. SCX_DEQ_PICKED_FROM_GLOBAL_DSQ?

> diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h
> index bcb962d5ee7d8..0d003d2845393 100644
> --- a/include/linux/sched/ext.h
> +++ b/include/linux/sched/ext.h
> @@ -84,6 +84,7 @@ struct scx_dispatch_q {
>  /* scx_entity.flags */
>  enum scx_ent_flags {
>  	SCX_TASK_QUEUED		= 1 << 0, /* on ext runqueue */
> +	SCX_TASK_OPS_ENQUEUED	= 1 << 1, /* under ext scheduler's custody */

Nit: I think "in BPF scheduler's custody" would be a bit clearer, as
"ext scheduler" could potentially be interpreted to mean SCHED_CLASS_EXT
as a whole.

> @@ -1523,6 +1603,30 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
>  
>  	switch (opss & SCX_OPSS_STATE_MASK) {
>  	case SCX_OPSS_NONE:
> +		/*
> +		 * Task is not in BPF data structures (either dispatched to
> +		 * a DSQ or running). Only call ops.dequeue() if the task
> +		 * is still in BPF scheduler's custody
> +		 * (%SCX_TASK_OPS_ENQUEUED is set).
> +		 *
> +		 * If the task has already been dispatched to a local DSQ
> +		 * (left BPF custody), the flag will be clear and we skip
> +		 * ops.dequeue()
> +		 *
> +		 * If this is a property change (not sleep/core-sched) and
> +		 * the task is still in BPF custody, set the
> +		 * %SCX_DEQ_SCHED_CHANGE flag.
> +		 */
> +		if (SCX_HAS_OP(sch, dequeue) &&
> +		    p->scx.flags & SCX_TASK_OPS_ENQUEUED) {
> +			u64 flags = deq_flags;
> +
> +			if (!(deq_flags & (DEQUEUE_SLEEP | SCX_DEQ_CORE_SCHED_EXEC)))
> +				flags |= SCX_DEQ_SCHED_CHANGE;

I think this logic will result in ops.dequeue(SCHED_CHANGE) being called for
tasks being picked from a global DSQ being migrated from a remote rq to the
local rq, which, while technically correct since the task is migrating rqs,
may be confusing, since it fits two cases in the documentation:

* Since the task is leaving BPF custody for execution, ops.dequeue() should be
  called without any special flags.
* Since the task is being migrated between rqs, ops.dequeue() should be called
  with SCX_DEQ_SCHED_CHANGE.

> +
> +			SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, flags);
> +			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
> +		}
>  		break;
>  	case SCX_OPSS_QUEUEING:
>  		/*

Thanks,
Kuba

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Kuba Piecuch]

Hi Andrea,

On Mon Feb 2, 2026 at 7:45 AM UTC, Andrea Righi wrote:
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 6d6f1253039d8..d8fed4a49195d 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -2248,7 +2248,7 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
>  			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
>  		} else {
>  			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
> -				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, task_rq(p), p, 0);
> +				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
>  
>  			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
>  		}

This looks risky from a locking perspective. Are we relying on
SCX_OPSS_DISPATCHING to protect against racing dequeues? If so, it might
be worth calling out in a comment.

Thanks,
Kuba

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

On Mon, Feb 02, 2026 at 10:02:30AM +0000, Christian Loehle wrote:
> On 2/2/26 09:26, Andrea Righi wrote:
> > On Mon, Feb 02, 2026 at 08:45:18AM +0100, Andrea Righi wrote:
> > ...
> >>> So I have finally gotten around updating scx_storm to the new semantics,
> >>> see:
> >>> https://github.com/cloehle/scx/tree/cloehle/scx-storm-qmap-insert-local-dequeue-semantics
> >>>
> >>> I don't think the new ops.dequeue() are enough to make inserts to local-on
> >>> from anywhere safe, because it's still racing with dequeue from another CPU?
> >>
> >> Yeah, with this patch set BPF schedulers get proper ops.dequeue()
> >> callbacks, but we're not fixing the usage of SCX_DSQ_LOCAL_ON from
> >> ops.dispatch().
> >>
> >> When task properties change between scx_bpf_dsq_insert() and the actual
> >> dispatch, task_can_run_on_remote_rq() can still trigger a fatal
> >> scx_error().
> >>
> >> The ops.dequeue(SCX_DEQ_SCHED_CHANGE) notifications happens after the
> >> property change, so it can't prevent already-queued dispatches from
> >> failing. The race window is between ops.dispatch() returning and
> >> dispatch_to_local_dsq() executing.
> >>
> >> We can address this in a separate patch set. One thing at a time. :)
> > 
> > Thinking more on this, the problem is that we're passing enforce=true to
> > task_can_run_on_remote_rq(), triggering a critical failure - scx_error().
> > There's a logic in task_can_run_on_remote_rq() to fallback to the global
> > DSQ, that doesn't happen if we pass enforce=true, due to scx_error().
> > 
> > However, instead of the global DSQ fallback, I was wondering if it'd be
> > better to simply re-enqueue the task - setting SCX_ENQ_REENQ - if the
> > target local DSQ isn't valid anymore when the dispatch is finalized.
> > 
> > In this way using SCX_DSQ_LOCAL_ON | cpu from ops.dispatch() would simply
> > trigger a re-enqueue when "cpu" isn't valid anymore (due to concurrent
> > affinity / migration disabled changes) and the BPF scheduler can handle
> > that in another ops.enqueue().
> > 
> > What do you think?
> 
> I think that's a lot more versatile for the BPF scheduler than using the
> global DSQ as fallback in that case, so yeah I'm all for it!
> 

Ack, I already have a working patch do to this, I'll post it as a separate
patch set.

Thanks,
-Andrea

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

Hi Kuba,

sorry for the late response.

On Mon, Feb 02, 2026 at 01:59:24PM +0000, Kuba Piecuch wrote:
> Hi Andrea,
> 
> On Mon Feb 2, 2026 at 7:45 AM UTC, Andrea Righi wrote:
> > diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> > index 6d6f1253039d8..d8fed4a49195d 100644
> > --- a/kernel/sched/ext.c
> > +++ b/kernel/sched/ext.c
> > @@ -2248,7 +2248,7 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
> >  			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
> >  		} else {
> >  			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
> > -				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, task_rq(p), p, 0);
> > +				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
> >  
> >  			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
> >  		}
> 
> This looks risky from a locking perspective. Are we relying on
> SCX_OPSS_DISPATCHING to protect against racing dequeues? If so, it might
> be worth calling out in a comment.

You're right, we're relying on SCX_OPSS_DISPATCHING to protect against
racing dequeues and this definitely deserves a comment. How about something
like the following?

Thanks,
-Andrea

---
diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
index 292adf10fee1b..b189339e74101 100644
--- a/kernel/sched/ext.c
+++ b/kernel/sched/ext.c
@@ -2260,6 +2260,15 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
 		if (!is_terminal_dsq(dsq_id)) {
 			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
 		} else {
+			/*
+			 * Locking: we're holding the @rq lock (the
+			 * dispatch CPU's rq), but not necessarily
+			 * task_rq(p), since @p may be from a remote CPU.
+			 *
+			 * This is safe because SCX_OPSS_DISPATCHING state
+			 * prevents racing dequeues, any concurrent
+			 * ops_dequeue() will wait for this state to clear.
+			 */
 			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
 				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);
 

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Kuba Piecuch]

On Wed Feb 4, 2026 at 9:36 AM UTC, Andrea Righi wrote:
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 292adf10fee1b..b189339e74101 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -2260,6 +2260,15 @@ static void finish_dispatch(struct scx_sched *sch, struct rq *rq,
>  		if (!is_terminal_dsq(dsq_id)) {
>  			p->scx.flags |= SCX_TASK_OPS_ENQUEUED;
>  		} else {
> +			/*
> +			 * Locking: we're holding the @rq lock (the
> +			 * dispatch CPU's rq), but not necessarily
> +			 * task_rq(p), since @p may be from a remote CPU.
> +			 *
> +			 * This is safe because SCX_OPSS_DISPATCHING state
> +			 * prevents racing dequeues, any concurrent
> +			 * ops_dequeue() will wait for this state to clear.
> +			 */
>  			if (p->scx.flags & SCX_TASK_OPS_ENQUEUED)
>  				SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, 0);

Looks good, thanks :)

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Andrea Righi]

On Mon, Feb 02, 2026 at 11:56:43AM +0000, Kuba Piecuch wrote:
> Hi Andrea,
> 
> Looks good overall, but we need to settle on the global DSQ semantics, plus
> some edge cases that need clearing up.

On this one I think we settled on the assumption that SCX_DSQ_GLOBAL can be
considered a "terminal DSQ", so we won't trigger ops.dequeue().

> 
> On Sun Feb 1, 2026 at 9:08 AM UTC, Andrea Righi wrote:
> > diff --git a/Documentation/scheduler/sched-ext.rst b/Documentation/scheduler/sched-ext.rst
> > index 404fe6126a769..6d9e82e6ca9d4 100644
> > --- a/Documentation/scheduler/sched-ext.rst
> > +++ b/Documentation/scheduler/sched-ext.rst
> > @@ -252,6 +252,80 @@ The following briefly shows how a waking task is scheduled and executed.
> >  
> >     * Queue the task on the BPF side.
> >  
> > +   **Task State Tracking and ops.dequeue() Semantics**
> > +
> > +   Once ``ops.select_cpu()`` or ``ops.enqueue()`` is called, the task may
> > +   enter the "BPF scheduler's custody" depending on where it's dispatched:
> > +
> > +   * **Direct dispatch to local DSQs** (``SCX_DSQ_LOCAL`` or
> > +     ``SCX_DSQ_LOCAL_ON | cpu``): The task bypasses the BPF scheduler
> > +     entirely and goes straight to the CPU's local run queue. The task
> > +     never enters BPF custody, and ``ops.dequeue()`` will not be called.
> > +
> > +   * **Dispatch to non-local DSQs** (``SCX_DSQ_GLOBAL`` or custom DSQs):
> > +     the task enters the BPF scheduler's custody. When the task later
> > +     leaves BPF custody (dispatched to a local DSQ, picked by core-sched,
> > +     or dequeued for sleep/property changes), ``ops.dequeue()`` will be
> > +     called exactly once.
> > +
> > +   * **Queued on BPF side**: The task is in BPF data structures and in BPF
> > +     custody, ``ops.dequeue()`` will be called when it leaves.
> > +
> > +   The key principle: **ops.dequeue() is called when a task leaves the BPF
> > +   scheduler's custody**. A task is in BPF custody if it's on a non-local
> > +   DSQ or in BPF data structures. Once dispatched to a local DSQ or after
> > +   ops.dequeue() is called, the task is out of BPF custody and the BPF
> > +   scheduler no longer needs to track it.
> > +
> > +   This works correctly with the ``ops.select_cpu()`` direct dispatch
> > +   optimization: even though it skips ``ops.enqueue()`` invocation, if the
> > +   task is dispatched to a non-local DSQ, it enters BPF custody and will
> > +   get ``ops.dequeue()`` when it leaves. This provides the performance
> > +   benefit of avoiding the ``ops.enqueue()`` roundtrip while maintaining
> > +   correct state tracking.
> > +
> > +   The dequeue can happen for different reasons, distinguished by flags:
> > +
> > +   1. **Regular dispatch workflow**: when the task is dispatched from a
> > +      non-local DSQ to a local DSQ (leaving BPF custody for execution),
> > +      ``ops.dequeue()`` is triggered without any special flags.
> 
> Maybe add a note that this can happen asynchronously, without the BPF
> scheduler explicitly dispatching the task to a local DSQ, when the task
> is on a global DSQ? Or maybe make that case into a separate dequeue reason
> with its own flag, e.g. SCX_DEQ_PICKED_FROM_GLOBAL_DSQ?

And I guess we don't need this if we consider SCX_DSQ_GLOBAL as a terminal
DSQ, because we won't trigger ops.dequeue().

> 
> > diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h
> > index bcb962d5ee7d8..0d003d2845393 100644
> > --- a/include/linux/sched/ext.h
> > +++ b/include/linux/sched/ext.h
> > @@ -84,6 +84,7 @@ struct scx_dispatch_q {
> >  /* scx_entity.flags */
> >  enum scx_ent_flags {
> >  	SCX_TASK_QUEUED		= 1 << 0, /* on ext runqueue */
> > +	SCX_TASK_OPS_ENQUEUED	= 1 << 1, /* under ext scheduler's custody */
> 
> Nit: I think "in BPF scheduler's custody" would be a bit clearer, as
> "ext scheduler" could potentially be interpreted to mean SCHED_CLASS_EXT
> as a whole.

Ack. Will change that.

> 
> > @@ -1523,6 +1603,30 @@ static void ops_dequeue(struct rq *rq, struct task_struct *p, u64 deq_flags)
> >  
> >  	switch (opss & SCX_OPSS_STATE_MASK) {
> >  	case SCX_OPSS_NONE:
> > +		/*
> > +		 * Task is not in BPF data structures (either dispatched to
> > +		 * a DSQ or running). Only call ops.dequeue() if the task
> > +		 * is still in BPF scheduler's custody
> > +		 * (%SCX_TASK_OPS_ENQUEUED is set).
> > +		 *
> > +		 * If the task has already been dispatched to a local DSQ
> > +		 * (left BPF custody), the flag will be clear and we skip
> > +		 * ops.dequeue()
> > +		 *
> > +		 * If this is a property change (not sleep/core-sched) and
> > +		 * the task is still in BPF custody, set the
> > +		 * %SCX_DEQ_SCHED_CHANGE flag.
> > +		 */
> > +		if (SCX_HAS_OP(sch, dequeue) &&
> > +		    p->scx.flags & SCX_TASK_OPS_ENQUEUED) {
> > +			u64 flags = deq_flags;
> > +
> > +			if (!(deq_flags & (DEQUEUE_SLEEP | SCX_DEQ_CORE_SCHED_EXEC)))
> > +				flags |= SCX_DEQ_SCHED_CHANGE;
> 
> I think this logic will result in ops.dequeue(SCHED_CHANGE) being called for
> tasks being picked from a global DSQ being migrated from a remote rq to the
> local rq, which, while technically correct since the task is migrating rqs,
> may be confusing, since it fits two cases in the documentation:
> 
> * Since the task is leaving BPF custody for execution, ops.dequeue() should be
>   called without any special flags.
> * Since the task is being migrated between rqs, ops.dequeue() should be called
>   with SCX_DEQ_SCHED_CHANGE.

This also should be fixed with the new logic, because a task disptched to a
global DSQ is considered outside of the BPF scheduler's custody, so
ops.dequeue() is not invoked at all.

I'll post a new patch set later today, so we can better discuss if all
these assumptions have been addressed properly. :)

> 
> > +
> > +			SCX_CALL_OP_TASK(sch, SCX_KF_REST, dequeue, rq, p, flags);
> > +			p->scx.flags &= ~SCX_TASK_OPS_ENQUEUED;
> > +		}
> >  		break;
> >  	case SCX_OPSS_QUEUEING:
> >  		/*
> 
> Thanks,
> Kuba

Thanks,
-Andrea

Re: [PATCH 1/2] sched_ext: Fix ops.dequeue() semantics

[Kuba Piecuch]

On Wed Feb 4, 2026 at 10:11 AM UTC, Andrea Righi wrote:
> On Mon, Feb 02, 2026 at 11:56:43AM +0000, Kuba Piecuch wrote:
>> Hi Andrea,
>> 
>> Looks good overall, but we need to settle on the global DSQ semantics, plus
>> some edge cases that need clearing up.
>
> On this one I think we settled on the assumption that SCX_DSQ_GLOBAL can be
> considered a "terminal DSQ", so we won't trigger ops.dequeue().

Correct, I made this comment before we settled it.

Thanks,
Kuba