[PATCHSET v2 sched_ext/for-7.5] sched_ext: Add NUMA balancing support

[PATCHSET v2 sched_ext/for-7.5] sched_ext: Add NUMA balancing support

[Andrea Righi]

Tasks running under a BPF scheduler are currently invisible to NUMA
balancing: their address space is never scanned for hinting faults, so
they never record NUMA fault statistics and never acquire a preferred
node. A BPF scheduler therefore has no information about where a task's
memory is when deciding where to run it.

This series lets a BPF scheduler opt into the NUMA hinting-fault scan by
implementing a new callback, ops.set_numa_node(). The existing scan is
driven from the sched_ext tick for the tasks of such a scheduler, and
the hinting faults then work for them as they do for fair tasks: the
kernel maintains their NUMA fault statistics and preferred node, and
migrates the memory a task accesses toward the node it runs on. The
tasks themselves are not migrated: their placement is left to the BPF
scheduler.

ops.set_numa_node() reports the task's preferred node: once after
ops.enable(), as ops.set_weight() does for the weight, and then every
time the preference changes. Changes are reported from sched_setnuma(),
under the task's rq lock and before the task is re-enqueued, so the
scheduler can update its placement state before ops.enqueue() sees the
task.

To keep sched core free of class-specific code, sched_setnuma() notifies
the task's class through a new optional sched_class->numa_node_changed()
method, which only sched_ext implements at the moment.

scx_bpf_task_numa_nid() lets a scheduler read the current preference
from any of its other callbacks. For a task on a BPF scheduler that
doesn't implement ops.set_numa_node() it returns NUMA_NO_NODE, as such a
task isn't scanned and its preference would be left over from its
previous scheduling class. The value is advisory; nothing in the kernel
acts on it for these tasks.

Ideally the ultimate goal is to give BPF schedulers both controls that
NUMA balancing combines: moving a task to the node where its memory is
and moving the memory a task uses to the node where the task runs. This
series provides the first: with the preferred node available, a
scheduler can decide where to run the task. A second series, to follow
once this one lands, will provide the second: a kfunc that lets a
scheduler request the node a task's memory should migrate to.

Vladimir Vdovin recently posted an RFC [1] that drives the scan from the
sched_ext tick in the same way and skips the fair-class task migration
for tasks under a BPF scheduler. This series covers what that RFC
proposed and adds the callback and the kfunc on top.

This topic was also discussed at the sched_ext microconference at LPC
2026 [2], in the context of placing accelerator workloads close to their
devices and memory.

Patches 1 and 2 are small preparatory changes to the generic NUMA
balancing code: the first makes the scan available to other scheduling
classes, the second skips the fair-class task migration for tasks owned
by a BPF scheduler and never picks them as swap partners of a migrating
fair task. Patch 3 adds the sched_class method. Apart from that swap
exclusion, which only matters when fair and BPF-scheduled tasks share
CPUs (scx partial mode enabled), none of them changes the behavior of
fair, RT or deadline tasks. Patch 4 adds ops.set_numa_node() and drives
the scan for the schedulers that implement it. Patch 5 adds
scx_bpf_task_numa_nid(), which lets a BPF scheduler read a task's
preferred node from any of its callbacks and returns NUMA_NO_NODE for
the tasks of a scheduler that doesn't implement ops.set_numa_node().
Patch 6 adds a selftest.

Changes in v2:
 - Replace the SCX_OPS_NUMA_BALANCING flag with the ops.set_numa_node()
   callback, which reports preferred node changes as events and enables the
   hinting-fault scan when it is implemented (Tejun Heo)
 - Keep driving the scan from the donor under proxy execution, consistent with
   task_tick_fair() (Vladimir Vdovin)
 - Never pick a task owned by a BPF scheduler as the swap partner of a
   fair task migrated by NUMA balancing
 - Return NUMA_NO_NODE from scx_bpf_task_numa_nid() for tasks whose scheduler
   doesn't implement ops.set_numa_node()
 - Extend the selftest to check that every enabled task gets an initial
   notification before it runs, that changes are reported without duplicates
   and that the notifications agree with scx_bpf_task_numa_nid(), and require
   a preferred node change on a machine with NUMA balancing enabled
 - Link to v1:
   https://lore.kernel.org/r/20261004072901.3579967-1-arighi@nvidia.com

[1] https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
[2] https://lpc.events/event/20/sessions/270/

Git tree:
git://git.kernel.org/pub/scm/linux/kernel/git/arighi/linux.git scx-numa-preferred-node

Andrea Righi (6):
  sched/numa: Let other scheduling classes drive NUMA scanning
  sched/numa: Leave the placement of a BPF-scheduled task to its
    scheduler
  sched/numa: Notify the scheduling class of preferred node changes
  sched_ext: Report preferred NUMA nodes through ops.set_numa_node()
  sched_ext: Add a kfunc to read a task's preferred NUMA node
  selftests/sched_ext: Test preferred NUMA node notifications

 kernel/sched/core.c                           |   7 +-
 kernel/sched/ext/ext.c                        |  88 +++++++
 kernel/sched/ext/internal.h                   |  20 ++
 kernel/sched/fair.c                           |  21 +-
 kernel/sched/sched.h                          |  12 +
 tools/sched_ext/include/scx/compat.bpf.h      |  10 +
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/numa_nid.bpf.c        | 142 +++++++++++
 tools/testing/selftests/sched_ext/numa_nid.c  | 233 ++++++++++++++++++
 9 files changed, 527 insertions(+), 7 deletions(-)
 create mode 100644 tools/testing/selftests/sched_ext/numa_nid.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/numa_nid.c

[PATCH 1/6] sched/numa: Let other scheduling classes drive NUMA scanning

[Andrea Righi]

The periodic scan that installs the NUMA hinting faults is driven by
task_tick_numa(), which is reachable only from task_tick_fair(). What
the scan feeds is class independent, but a task that is not in the fair
class is never scanned, records no faults and never gets a preferred
node.

Make task_tick_numa() available to the other scheduling classes (e.g.,
sched_ext). Declare it in sched.h and move the !CONFIG_NUMA_BALANCING
stub there, so a class that has no interest in NUMA balancing keeps
costing nothing.

No functional change: task_tick_fair() remains the only caller.

Reported-by: Vladimir Vdovin <deliran@verdict.gg>
Link: https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/fair.c  | 6 +-----
 kernel/sched/sched.h | 5 +++++
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 0e279a7768754..36df9a4dde534 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4636,7 +4636,7 @@ void init_numa_balancing(u64 clone_flags, struct task_struct *p)
 /*
  * Drive the periodic memory faults..
  */
-static void task_tick_numa(struct rq *rq, struct task_struct *curr)
+void task_tick_numa(struct rq *rq, struct task_struct *curr)
 {
 	struct callback_head *work = &curr->numa_work;
 	u64 period, now;
@@ -4702,10 +4702,6 @@ static void update_scan_period(struct task_struct *p, int new_cpu)
 
 #else /* !CONFIG_NUMA_BALANCING: */
 
-static void task_tick_numa(struct rq *rq, struct task_struct *curr)
-{
-}
-
 static inline void account_numa_enqueue(struct rq *rq, struct task_struct *p)
 {
 }
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 9e7993256c817..2b05c41e66e29 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2151,6 +2151,7 @@ extern int migrate_task_to(struct task_struct *p, int cpu);
 extern int migrate_swap(struct task_struct *p, struct task_struct *t,
 			int cpu, int scpu);
 extern void init_numa_balancing(u64 clone_flags, struct task_struct *p);
+extern void task_tick_numa(struct rq *rq, struct task_struct *curr);
 
 #else /* !CONFIG_NUMA_BALANCING: */
 
@@ -2159,6 +2160,10 @@ init_numa_balancing(u64 clone_flags, struct task_struct *p)
 {
 }
 
+static inline void task_tick_numa(struct rq *rq, struct task_struct *curr)
+{
+}
+
 #endif /* !CONFIG_NUMA_BALANCING */
 
 int task_llc(const struct task_struct *p);
-- 
2.56.0

[PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Andrea Righi]

A task under a BPF scheduler can take NUMA hinting faults via
task_numa_fault() and then reach numa_migrate_preferred(), which may
migrate the task to its preferred node through task_numa_migrate().

The migration of such a task should always be driven by its BPF
scheduler, which owns the placement of its tasks.

Keep collecting the statistics, which is what makes
p->numa_preferred_nid worth reading, but skip the migration for a task
owned by a BPF scheduler. The scheduler can read the preferred node and
decide for itself whether moving the task is worth the cost.

For the same reason, never pick a task owned by a BPF scheduler as the
swap partner of another task that NUMA balancing migrates, e.g., a fair
task when the BPF scheduler runs in partial mode. Such a task is
treated like a kthread on the destination CPU: the migrating task can
still move there alone, but the BPF-scheduled task is left in place.

Reported-by: Vladimir Vdovin <deliran@verdict.gg>
Link: https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/fair.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 36df9a4dde534..5064c7b707f4d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
 
 	rcu_read_lock();
 	cur = rcu_dereference_all(dst_rq->curr);
+	/*
+	 * Never pick a task under a BPF scheduler as the swap partner: its
+	 * placement is left to that scheduler, see numa_migrate_preferred().
+	 * Moving the faulting task alone to the destination is still allowed.
+	 */
 	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
-		    !cur->mm))
+		    !cur->mm || task_on_scx(cur)))
 		cur = NULL;
 
 	/*
@@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
 	if (task_node(p) == p->numa_preferred_nid)
 		return;
 
+	/*
+	 * A task under a BPF scheduler is placed by that scheduler. Keep the
+	 * statistics coming, which is what makes p->numa_preferred_nid worth
+	 * reading, but leave the placement alone.
+	 */
+	if (task_on_scx(p))
+		return;
+
 	/* Otherwise, try migrate to a CPU on the preferred node */
 	task_numa_migrate(p);
 }
-- 
2.56.0

[PATCH 3/6] sched/numa: Notify the scheduling class of preferred node changes

[Andrea Righi]

sched_setnuma() updates p->numa_preferred_nid under the task's rq lock
and within a sched_change section, but the task's scheduling class has
no way to learn about the new preference other than polling the field.

Add an optional sched_class->numa_node_changed() method, called from
sched_setnuma() when the preferred node actually changes, after the new
value is stored and before the task is re-enqueued. This lets a class
that keeps its own placement state (e.g., sched_ext) react to the change
without sched core having to know about any specific class.

No functional change: no class implements the method yet.

Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/core.c  | 7 ++++++-
 kernel/sched/sched.h | 7 +++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2f46922982f32..542f28fd95603 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -8331,8 +8331,13 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
 void sched_setnuma(struct task_struct *p, int nid)
 {
 	guard(task_rq_lock)(p);
-	scoped_guard (sched_change, p, DEQUEUE_SAVE)
+	scoped_guard (sched_change, p, DEQUEUE_SAVE) {
+		bool changed = p->numa_preferred_nid != nid;
+
 		p->numa_preferred_nid = nid;
+		if (changed && p->sched_class->numa_node_changed)
+			p->sched_class->numa_node_changed(task_rq(p), p, nid);
+	}
 }
 #endif /* CONFIG_NUMA_BALANCING */
 
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 2b05c41e66e29..974f5df919aec 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2814,6 +2814,13 @@ struct sched_class {
 	void (*task_change_group)(struct task_struct *p);
 #endif
 
+#ifdef CONFIG_NUMA_BALANCING
+	/*
+	 * sched_setnuma: task_rq_lock + sched_change
+	 */
+	void (*numa_node_changed)(struct rq *rq, struct task_struct *p, int nid);
+#endif
+
 #ifdef CONFIG_SCHED_CORE
 	/*
 	 * pick_next_task: rq->lock
-- 
2.56.0

[PATCH 4/6] sched_ext: Report preferred NUMA nodes through ops.set_numa_node()

[Andrea Righi]

BPF schedulers need NUMA access information to place tasks close to
their memory, while retaining control over task placement. Deliver the
preferred node through ops.set_numa_node() so schedulers can update
their placement state when the preference changes instead of polling.

Report the current preference after ops.enable(), including
NUMA_NO_NODE when no preference is available. Notify subsequent changes
through sched_class->numa_node_changed(), which sched_setnuma() calls
under the task's rq lock, before re-enqueueing the task. Support both
CPU and CID schedulers. Kernels without NUMA balancing deliver only the
initial NUMA_NO_NODE notification.

Drive NUMA hinting-fault scans from the sched_ext tick when the callback
is implemented and global NUMA balancing is enabled. The faults maintain
NUMA statistics and can migrate accessed memory toward the accessing
CPU, while task placement remains the BPF scheduler's responsibility.

Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/ext/ext.c      | 46 +++++++++++++++++++++++++++++++++++++
 kernel/sched/ext/internal.h | 20 ++++++++++++++++
 2 files changed, 66 insertions(+)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 2a2b56c6074c6..076e3757c0634 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -4379,6 +4379,21 @@ static void task_tick_scx(struct rq *rq, struct task_struct *donor, int queued)
 	else if (SCX_HAS_OP(sch, tick))
 		SCX_CALL_OP_TASK(sch, tick, rq, donor);
 
+	/*
+	 * If requested by the scheduler, drive the NUMA hinting-fault scan as
+	 * task_tick_fair() does. Neither the scan nor the statistics it feeds
+	 * depend on the scheduling class. Where the task then runs is left to
+	 * that scheduler: NUMA balancing neither migrates a task it owns nor
+	 * picks one as a swap partner, see numa_migrate_preferred() and
+	 * task_numa_compare().
+	 *
+	 * The tick that only refreshes an already queued task does not scan,
+	 * matching the @queued check in task_tick_fair().
+	 */
+	if (!queued && SCX_HAS_OP(sch, set_numa_node) &&
+	    static_branch_unlikely(&sched_numa_balancing))
+		task_tick_numa(rq, donor);
+
 	if (!donor->scx.slice) {
 		/* the slice can't be trusted while bypassing */
 		if (READ_ONCE(donor->scx.lazy_resched) &&
@@ -4528,6 +4543,15 @@ static void __scx_enable_task(struct scx_sched *sch, struct task_struct *p)
 
 	if (SCX_HAS_OP(sch, set_weight))
 		SCX_CALL_OP_TASK(sch, set_weight, rq, p, p->scx.weight);
+
+	if (SCX_HAS_OP(sch, set_numa_node)) {
+		s32 nid = NUMA_NO_NODE;
+
+#ifdef CONFIG_NUMA_BALANCING
+		nid = p->numa_preferred_nid;
+#endif
+		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
+	}
 }
 
 void scx_enable_task(struct scx_sched *sch, struct task_struct *p)
@@ -4834,6 +4858,20 @@ void sched_ext_dead(struct task_struct *p)
 	}
 }
 
+#ifdef CONFIG_NUMA_BALANCING
+static void numa_node_changed_scx(struct rq *rq, struct task_struct *p, int nid)
+{
+	struct scx_sched *sch = scx_task_sched(p);
+
+	if (task_dead_and_done(p) ||
+	    scx_get_task_state(p) != SCX_TASK_ENABLED)
+		return;
+
+	if (SCX_HAS_OP(sch, set_numa_node))
+		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
+}
+#endif
+
 static void reweight_task_scx(struct rq *rq, struct task_struct *p,
 			      const struct load_weight *lw)
 {
@@ -5745,6 +5783,10 @@ DEFINE_SCHED_CLASS(ext) = {
 
 	.update_curr		= update_curr_scx,
 
+#ifdef CONFIG_NUMA_BALANCING
+	.numa_node_changed	= numa_node_changed_scx,
+#endif
+
 #ifdef CONFIG_UCLAMP_TASK
 	.uclamp_enabled		= 1,
 #endif
@@ -9069,6 +9111,7 @@ static void sched_ext_ops__quiescent(struct task_struct *p, u64 deq_flags) {}
 static bool sched_ext_ops__yield(struct task_struct *from, struct task_struct *to__nullable) { return false; }
 static bool sched_ext_ops__core_sched_before(struct task_struct *a, struct task_struct *b) { return false; }
 static void sched_ext_ops__set_weight(struct task_struct *p, u32 weight) {}
+static void sched_ext_ops__set_numa_node(struct task_struct *p, s32 nid) {}
 static void sched_ext_ops__set_cpumask(struct task_struct *p, const struct cpumask *mask) {}
 static void sched_ext_ops__update_idle(s32 cpu, bool idle) {}
 static void sched_ext_ops__cpu_acquire(s32 cpu, struct scx_cpu_acquire_args *args) {}
@@ -9111,6 +9154,7 @@ static struct sched_ext_ops __bpf_ops_sched_ext_ops = {
 	.yield			= sched_ext_ops__yield,
 	.core_sched_before	= sched_ext_ops__core_sched_before,
 	.set_weight		= sched_ext_ops__set_weight,
+	.set_numa_node		= sched_ext_ops__set_numa_node,
 	.set_cpumask		= sched_ext_ops__set_cpumask,
 	.update_idle		= sched_ext_ops__update_idle,
 	.cpu_acquire		= sched_ext_ops__cpu_acquire,
@@ -9181,6 +9225,7 @@ static struct sched_ext_ops_cid __bpf_ops_sched_ext_ops_cid = {
 	.yield			= sched_ext_ops__yield,
 	.core_sched_before	= sched_ext_ops__core_sched_before,
 	.set_weight		= sched_ext_ops__set_weight,
+	.set_numa_node		= sched_ext_ops__set_numa_node,
 	.set_cmask		= sched_ext_ops_cid__set_cmask,
 	.update_idle		= sched_ext_ops__update_idle,
 	.init_task		= sched_ext_ops__init_task,
@@ -12026,6 +12071,7 @@ static int __init scx_init(void)
 	CID_OFFSET_MATCH(yield, yield);
 	CID_OFFSET_MATCH(core_sched_before, core_sched_before);
 	CID_OFFSET_MATCH(set_weight, set_weight);
+	CID_OFFSET_MATCH(set_numa_node, set_numa_node);
 	CID_OFFSET_MATCH(update_idle, update_idle);
 	CID_OFFSET_MATCH(init_task, init_task);
 	CID_OFFSET_MATCH(exit_task, exit_task);
diff --git a/kernel/sched/ext/internal.h b/kernel/sched/ext/internal.h
index 80c746054ce7a..a5a8044d3c5a2 100644
--- a/kernel/sched/ext/internal.h
+++ b/kernel/sched/ext/internal.h
@@ -616,6 +616,25 @@ struct sched_ext_ops {
 	 */
 	void (*set_weight)(struct task_struct *p, u32 weight);
 
+	/**
+	 * @set_numa_node: Report a task's preferred NUMA node
+	 * @p: task whose preferred node is being reported
+	 * @nid: preferred node, or NUMA_NO_NODE if there is no preference
+	 *
+	 * Called after ops.enable() to report the current preference and when
+	 * the preference changes. The task's rq lock is held, and updates are
+	 * reported before the task is re-enqueued, if it is runnable.
+	 *
+	 * Implementing this op enables NUMA hinting-fault scans for the
+	 * scheduler's tasks when CONFIG_NUMA_BALANCING and kernel.numa_balancing
+	 * are enabled. The faults maintain NUMA statistics and the preferred
+	 * node, and can migrate accessed memory toward the accessing CPU.
+	 * Task placement remains the BPF scheduler's responsibility; @nid is
+	 * advisory. Without CONFIG_NUMA_BALANCING, only the initial
+	 * NUMA_NO_NODE notification is delivered.
+	 */
+	void (*set_numa_node)(struct task_struct *p, s32 nid);
+
 	/**
 	 * @set_cpumask: Set CPU affinity
 	 * @p: task to set CPU affinity for
@@ -1214,6 +1233,7 @@ struct sched_ext_ops_cid {
 	bool (*core_sched_before)(struct task_struct *a,
 				   struct task_struct *b);
 	void (*set_weight)(struct task_struct *p, u32 weight);
+	void (*set_numa_node)(struct task_struct *p, s32 nid);
 	void (*set_cmask)(struct task_struct *p,
 			   const struct scx_cmask *cmask__arena);
 	void (*update_idle)(s32 cid, bool idle);
-- 
2.56.0

[PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[Andrea Righi]

NUMA balancing derives a preferred node for each task from its hinting
faults, p->numa_preferred_nid, but a BPF scheduler has no reliable way
to read it.

Add scx_bpf_task_numa_nid() to return the task's preferred node, or
NUMA_NO_NODE when the task has none. The value is a hint the scheduler
can weigh against its own placement rules. It is read without the task's
rq lock, so it is a snapshot.

This complements ops.set_numa_node(), which reports changes as they
happen, by letting a scheduler read the current preference from any of
its other callbacks.

Only report the preference of a task on a BPF scheduler if the scheduler
that owns the task implements ops.set_numa_node(), and NUMA_NO_NODE
otherwise: such a task isn't scanned, so its preference would be left
over from its previous scheduling class. Check the owner, not the
caller, as with sub-schedulers they can differ and only the owner
decides whether the task is scanned. A task that isn't on sched_ext
reports its preference as is, which NUMA balancing keeps current for
fair tasks.

Add a compat wrapper that returns NUMA_NO_NODE on kernels without the
kfunc.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 kernel/sched/ext/ext.c                   | 42 ++++++++++++++++++++++++
 tools/sched_ext/include/scx/compat.bpf.h | 10 ++++++
 2 files changed, 52 insertions(+)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 076e3757c0634..fba2c83451ebc 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -11530,6 +11530,47 @@ __bpf_kfunc s32 scx_bpf_task_cid(const struct task_struct *p)
 	return tbl[task_cpu(p)];
 }
 
+/**
+ * scx_bpf_task_numa_nid - Node NUMA balancing would prefer a task to run on
+ * @p: task of interest
+ *
+ * Return p->numa_preferred_nid: the node NUMA balancing has derived from @p's
+ * hinting faults as the best one to run @p on, which is the node with the most
+ * faults, adjusted for CPU-less nodes and for the NUMA group @p may be part of.
+ * Return NUMA_NO_NODE when @p has no preference yet. A task has none until its
+ * address space has been scanned a few times. Disabling kernel.numa_balancing
+ * stops the scan, which leaves the last preference in place.
+ *
+ * A task on a BPF scheduler that doesn't implement ops.set_numa_node() isn't
+ * scanned, so its preference would be left over from its previous scheduling
+ * class. Return NUMA_NO_NODE for such a task. For a task that isn't on
+ * sched_ext, return its preference as is.
+ *
+ * The value is a hint, not an instruction: nothing in the kernel moves a task
+ * under a BPF scheduler to this node and the scheduler is free to weigh it
+ * against its own placement rules. The value is read without @p's rq lock, so
+ * it is a snapshot. In particular, a task that just changed scheduling class
+ * may report a preference from before the change until its new class has
+ * rescanned its address space.
+ */
+__bpf_kfunc s32 scx_bpf_task_numa_nid(const struct task_struct *p)
+{
+#ifdef CONFIG_NUMA_BALANCING
+	if (task_on_scx(p)) {
+		struct scx_sched *sch;
+
+		guard(rcu)();
+		sch = scx_task_sched_rcu(p);
+		if (!sch || !SCX_HAS_OP(sch, set_numa_node))
+			return NUMA_NO_NODE;
+	}
+
+	return READ_ONCE(p->numa_preferred_nid);
+#else
+	return NUMA_NO_NODE;
+#endif
+}
+
 /**
  * scx_bpf_locked_rq - Return the rq currently locked by SCX
  * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs
@@ -11836,6 +11877,7 @@ BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE)
 BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU)
 BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU)
 BTF_ID_FLAGS(func, scx_bpf_task_cid, KF_RCU)
+BTF_ID_FLAGS(func, scx_bpf_task_numa_nid, KF_RCU)
 BTF_ID_FLAGS(func, scx_bpf_locked_rq, KF_IMPLICIT_ARGS | KF_RET_NULL)
 BTF_ID_FLAGS(func, scx_bpf_cpu_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)
 BTF_ID_FLAGS(func, scx_bpf_cid_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)
diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/include/scx/compat.bpf.h
index 3520a749f8ce6..7dc94e54db167 100644
--- a/tools/sched_ext/include/scx/compat.bpf.h
+++ b/tools/sched_ext/include/scx/compat.bpf.h
@@ -478,6 +478,16 @@ static inline bool scx_bpf_task_set_dsq_vtime(struct task_struct *p, u64 vtime)
 	return true;
 }
 
+/* v7.5: Add scx_bpf_task_numa_nid(). */
+s32 scx_bpf_task_numa_nid___new(const struct task_struct *p) __ksym __weak;
+
+static inline s32 scx_bpf_task_numa_nid(const struct task_struct *p)
+{
+	if (bpf_ksym_exists(scx_bpf_task_numa_nid___new))
+		return scx_bpf_task_numa_nid___new(p);
+	return NUMA_NO_NODE;
+}
+
 /* v7.4: Add scx_bpf_task_set_lazy_resched(). */
 bool scx_bpf_task_set_lazy_resched___new(struct task_struct *p, bool lazy) __ksym __weak;
 
-- 
2.56.0

[PATCH 6/6] selftests/sched_ext: Test preferred NUMA node notifications

[Andrea Righi]

Exercise ops.set_numa_node() with a scheduler that tracks enable and
disable in task storage.

Require an initial notification for every enable before the task runs or
is disabled. Validate node IDs and check that notifications match the
preference returned by scx_bpf_task_numa_nid(), with no duplicate change
notifications, and that the kfunc still agrees with the last
notification whenever the task runs.

Run a memory-touching workload to exercise hinting faults. On a machine
with at least two memory nodes and numa_balancing enabled, require a
change notification: run the workload in a new process, which starts
without a preferred node, until one is reported, for up to 30 seconds.
A preferred node only depends on the hinting faults taken, not on pages
being migrated. Elsewhere, report the notification counts without
requiring a change.

Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 tools/testing/selftests/sched_ext/Makefile    |   1 +
 .../selftests/sched_ext/numa_nid.bpf.c        | 142 +++++++++++
 tools/testing/selftests/sched_ext/numa_nid.c  | 233 ++++++++++++++++++
 3 files changed, 376 insertions(+)
 create mode 100644 tools/testing/selftests/sched_ext/numa_nid.bpf.c
 create mode 100644 tools/testing/selftests/sched_ext/numa_nid.c

diff --git a/tools/testing/selftests/sched_ext/Makefile b/tools/testing/selftests/sched_ext/Makefile
index 44083f1c65969..af4cd6c682ea2 100644
--- a/tools/testing/selftests/sched_ext/Makefile
+++ b/tools/testing/selftests/sched_ext/Makefile
@@ -187,6 +187,7 @@ auto-test-targets :=			\
 	non_scx_kfunc_deny		\
 	nohz_tick			\
 	numa				\
+	numa_nid			\
 	allowed_cpus			\
 	peek_dsq			\
 	prog_run			\
diff --git a/tools/testing/selftests/sched_ext/numa_nid.bpf.c b/tools/testing/selftests/sched_ext/numa_nid.bpf.c
new file mode 100644
index 0000000000000..e584112ca957f
--- /dev/null
+++ b/tools/testing/selftests/sched_ext/numa_nid.bpf.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Verify initial preferred-node delivery and subsequent NUMA notifications.
+ *
+ * Copyright (c) 2026 NVIDIA Corporation.
+ */
+
+#include <scx/common.bpf.h>
+
+char _license[] SEC("license") = "GPL";
+
+UEI_DEFINE(uei);
+
+struct task_ctx {
+	bool enabled;
+	bool notified;
+	s32 nid;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_TASK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, struct task_ctx);
+} task_ctx_stor SEC(".maps");
+
+u64 nr_enabled;
+u64 nr_initial;
+u64 nr_updates;
+/* Notifications with and without a preferred node. */
+u64 nr_with_nid;
+u64 nr_without_nid;
+
+s32 BPF_STRUCT_OPS_SLEEPABLE(numa_nid_init_task, struct task_struct *p,
+			     struct scx_init_task_args *args)
+{
+	if (!bpf_task_storage_get(&task_ctx_stor, p, 0,
+				  BPF_LOCAL_STORAGE_GET_F_CREATE))
+		return -ENOMEM;
+	return 0;
+}
+
+static struct task_ctx *task_ctx(struct task_struct *p)
+{
+	struct task_ctx *tctx = bpf_task_storage_get(&task_ctx_stor, p, 0, 0);
+
+	if (!tctx)
+		scx_bpf_error("task %d has no task storage", p->pid);
+	return tctx;
+}
+
+void BPF_STRUCT_OPS(numa_nid_enable, struct task_struct *p)
+{
+	struct task_ctx *tctx = task_ctx(p);
+
+	if (!tctx)
+		return;
+	tctx->enabled = true;
+	tctx->notified = false;
+	__sync_fetch_and_add(&nr_enabled, 1);
+}
+
+void BPF_STRUCT_OPS(numa_nid_set_numa_node, struct task_struct *p, s32 nid)
+{
+	struct task_ctx *tctx = task_ctx(p);
+	s32 task_nid = scx_bpf_task_numa_nid(p);
+
+	if (!tctx)
+		return;
+	if (!tctx->enabled) {
+		scx_bpf_error("task %d notified before enable or after disable", p->pid);
+		return;
+	}
+	if (nid != task_nid) {
+		scx_bpf_error("task %d reported node %d, stored node %d",
+			      p->pid, nid, task_nid);
+		return;
+	}
+	if (tctx->notified) {
+		if (nid == tctx->nid) {
+			scx_bpf_error("task %d repeated node %d", p->pid, nid);
+			return;
+		}
+		__sync_fetch_and_add(&nr_updates, 1);
+	} else {
+		__sync_fetch_and_add(&nr_initial, 1);
+	}
+	tctx->notified = true;
+	tctx->nid = nid;
+
+	if (nid == NUMA_NO_NODE) {
+		__sync_fetch_and_add(&nr_without_nid, 1);
+		return;
+	}
+
+	if (nid < 0 || nid >= scx_bpf_nr_node_ids()) {
+		scx_bpf_error("task %d reported node %d, kernel has %d node ids",
+			      p->pid, nid, scx_bpf_nr_node_ids());
+		return;
+	}
+
+	__sync_fetch_and_add(&nr_with_nid, 1);
+}
+
+void BPF_STRUCT_OPS(numa_nid_running, struct task_struct *p)
+{
+	struct task_ctx *tctx = task_ctx(p);
+
+	if (!tctx)
+		return;
+	if (!tctx->notified)
+		scx_bpf_error("task %d ran before initial node notification", p->pid);
+	if (scx_bpf_task_numa_nid(p) != tctx->nid)
+		scx_bpf_error("task %d kfunc node differs from notification", p->pid);
+}
+
+void BPF_STRUCT_OPS(numa_nid_disable, struct task_struct *p)
+{
+	struct task_ctx *tctx = task_ctx(p);
+
+	if (!tctx)
+		return;
+	if (!tctx->notified)
+		scx_bpf_error("task %d disabled without initial node notification", p->pid);
+	tctx->enabled = false;
+}
+
+void BPF_STRUCT_OPS(numa_nid_exit, struct scx_exit_info *ei)
+{
+	UEI_RECORD(uei, ei);
+}
+
+SEC(".struct_ops.link")
+struct sched_ext_ops numa_nid_ops = {
+	.init_task			= (void *)numa_nid_init_task,
+	.enable				= (void *)numa_nid_enable,
+	.set_numa_node			= (void *)numa_nid_set_numa_node,
+	.running			= (void *)numa_nid_running,
+	.disable			= (void *)numa_nid_disable,
+	.exit				= (void *)numa_nid_exit,
+	.name				= "numa_nid",
+};
diff --git a/tools/testing/selftests/sched_ext/numa_nid.c b/tools/testing/selftests/sched_ext/numa_nid.c
new file mode 100644
index 0000000000000..de540dd13f468
--- /dev/null
+++ b/tools/testing/selftests/sched_ext/numa_nid.c
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2026 NVIDIA Corporation.
+ */
+#define _GNU_SOURCE
+#include <bpf/bpf.h>
+#include <linux/numa.h>
+#include <pthread.h>
+#include <scx/common.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+#include "numa_nid.bpf.skel.h"
+#include "scx_test.h"
+
+#define UPDATE_TIMEOUT		30
+
+#define WORKLOAD_SIZE		(64 << 20)
+#define WORKLOAD_THREADS	2
+#define WORKLOAD_SECONDS	3
+/* Returned by touch_memory() when its buffer can't be allocated. */
+#define WORKLOAD_FAILED		((void *)1)
+
+/*
+ * Touch a private buffer for a while. The hinting-fault scan skips the pages
+ * of a single-threaded process that are already on the node it runs on, so
+ * one such thread alone would never take a hinting fault: run two of them.
+ */
+static void *touch_memory(void *arg)
+{
+	struct timespec start, now;
+	volatile char *buffer;
+	size_t offset;
+
+	buffer = malloc(WORKLOAD_SIZE);
+	if (!buffer)
+		return WORKLOAD_FAILED;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	do {
+		for (offset = 0; offset < WORKLOAD_SIZE; offset += 4096)
+			buffer[offset]++;
+		clock_gettime(CLOCK_MONOTONIC, &now);
+	} while (now.tv_sec - start.tv_sec < WORKLOAD_SECONDS);
+
+	free((void *)buffer);
+	return NULL;
+}
+
+/*
+ * Run touch_memory() in the calling thread and in WORKLOAD_THREADS - 1 new
+ * ones. Return false if a thread can't be created or a buffer allocated.
+ */
+static bool run_workload(void)
+{
+	pthread_t threads[WORKLOAD_THREADS - 1];
+	bool failed = false;
+	void *ret;
+	int i, nr_threads;
+
+	for (nr_threads = 0; nr_threads < WORKLOAD_THREADS - 1; nr_threads++) {
+		if (pthread_create(&threads[nr_threads], NULL, touch_memory, NULL)) {
+			failed = true;
+			break;
+		}
+	}
+	failed |= touch_memory(NULL) == WORKLOAD_FAILED;
+	for (i = 0; i < nr_threads; i++) {
+		pthread_join(threads[i], &ret);
+		failed |= ret == WORKLOAD_FAILED;
+	}
+	return !failed;
+}
+
+/*
+ * The node list is in bitmap list format, e.g. "0-3" or "0,2-3", where a range
+ * always spans at least two nodes, so the second node is either the one after
+ * the first in a range or the next entry in the list.
+ */
+static int second_memory_node(void)
+{
+	FILE *file;
+	int first, second = NUMA_NO_NODE;
+	char sep;
+
+	file = fopen("/sys/devices/system/node/has_memory", "r");
+	if (!file)
+		return NUMA_NO_NODE;
+	if (fscanf(file, "%d%c", &first, &sep) == 2) {
+		if (sep == '-')
+			second = first + 1;
+		else if (sep == ',' && fscanf(file, "%d", &second) != 1)
+			second = NUMA_NO_NODE;
+	}
+	fclose(file);
+	return second;
+}
+
+static bool numa_balancing_enabled(void)
+{
+	FILE *file;
+	int mode = 0;
+
+	file = fopen("/proc/sys/kernel/numa_balancing", "r");
+	if (!file)
+		return false;
+	if (fscanf(file, "%d", &mode) != 1)
+		mode = 0;
+	fclose(file);
+	/* NUMA_BALANCING_NORMAL */
+	return mode & 1;
+}
+
+/*
+ * Run the workload in a new process until the scheduler is notified of a
+ * preferred node change, for up to UPDATE_TIMEOUT seconds. A new thread
+ * inherits the preferred node of the test, which may already match where its
+ * memory is, but a new process starts without one and acquires one from the
+ * hinting faults of the workload, whether or not its pages migrate. Return
+ * false if the workload fails.
+ */
+static bool wait_for_update(struct numa_nid *skel)
+{
+	struct timespec start, now;
+	bool ok = true;
+	pid_t pid;
+
+	pid = fork();
+	if (pid < 0)
+		return false;
+	if (!pid) {
+		while (run_workload())
+			;
+		_exit(1);
+	}
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	do {
+		if (__atomic_load_n(&skel->bss->nr_updates, __ATOMIC_ACQUIRE))
+			break;
+		if (waitpid(pid, NULL, WNOHANG) == pid) {
+			ok = false;
+			pid = 0;
+			break;
+		}
+		usleep(100000);
+		clock_gettime(CLOCK_MONOTONIC, &now);
+	} while (now.tv_sec - start.tv_sec < UPDATE_TIMEOUT);
+
+	if (pid) {
+		kill(pid, SIGKILL);
+		waitpid(pid, NULL, 0);
+	}
+	return ok;
+}
+
+static enum scx_test_status setup(void **ctx)
+{
+	struct numa_nid *skel;
+
+	skel = numa_nid__open();
+	SCX_FAIL_IF(!skel, "Failed to open");
+	SCX_ENUM_INIT(skel);
+	SCX_FAIL_IF(numa_nid__load(skel), "Failed to load skel");
+
+	*ctx = skel;
+
+	return SCX_TEST_PASS;
+}
+
+static enum scx_test_status run(void *ctx)
+{
+	struct numa_nid *skel = ctx;
+	struct bpf_link *link;
+	bool update;
+
+	/*
+	 * A preferred node change can only be expected on a machine with at
+	 * least two memory nodes and numa_balancing enabled.
+	 */
+	update = second_memory_node() >= 0 && numa_balancing_enabled();
+	if (!update)
+		fprintf(stderr, "Skipping the preferred node update check\n");
+
+	link = bpf_map__attach_struct_ops(skel->maps.numa_nid_ops);
+	SCX_FAIL_IF(!link, "Failed to attach scheduler");
+
+	/*
+	 * Give the scan something to work on, so that the tasks of this test
+	 * can acquire a preferred node on a NUMA machine with numa_balancing
+	 * enabled. The scheduler reports an error through UEI if it ever sees
+	 * a node id that is not valid and the counts below show what it saw.
+	 */
+	if (!run_workload() || (update && !wait_for_update(skel))) {
+		SCX_ERR("Failed to run the workload");
+		bpf_link__destroy(link);
+		return SCX_TEST_FAIL;
+	}
+
+	fprintf(stderr, "notifications with a preferred node: %lu, without: %lu\n",
+		(unsigned long)skel->bss->nr_with_nid,
+		(unsigned long)skel->bss->nr_without_nid);
+
+	bpf_link__destroy(link);
+	SCX_EQ(skel->data->uei.kind, EXIT_KIND(SCX_EXIT_UNREG));
+	SCX_GT(skel->bss->nr_enabled, 0);
+	SCX_EQ(skel->bss->nr_initial, skel->bss->nr_enabled);
+	fprintf(stderr, "initial notifications: %lu, updates: %lu\n",
+		(unsigned long)skel->bss->nr_initial,
+		(unsigned long)skel->bss->nr_updates);
+	if (update)
+		SCX_GT(skel->bss->nr_updates, 0);
+
+	return SCX_TEST_PASS;
+}
+
+static void cleanup(void *ctx)
+{
+	struct numa_nid *skel = ctx;
+
+	numa_nid__destroy(skel);
+}
+
+struct scx_test numa_nid = {
+	.name = "numa_nid",
+	.description = "Verify preferred NUMA node notifications to BPF",
+	.setup = setup,
+	.run = run,
+	.cleanup = cleanup,
+};
+REGISTER_SCX_TEST(&numa_nid)
-- 
2.56.0

Re: [PATCH 3/6] sched/numa: Notify the scheduling class of preferred node changes

[K Prateek Nayak]

Hello Andrea,

On 10/27/2026 9:43 AM, Andrea Righi wrote:
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -8331,8 +8331,13 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
>  void sched_setnuma(struct task_struct *p, int nid)
>  {
>  	guard(task_rq_lock)(p);
> -	scoped_guard (sched_change, p, DEQUEUE_SAVE)
> +	scoped_guard (sched_change, p, DEQUEUE_SAVE) {
> +		bool changed = p->numa_preferred_nid != nid;

Don't the callers of sched_setnuma() already check for this?

Only one that doesn't is task_numa_migrate() but
numa_migrate_preferred() only calls it when
"task_node(p) == p->numa_preferred_nid" is false and "!sd "case resets
"p->numa_preferred_nid" to "task_node(p)" when the task cannot be
migrated.

Afaict, "changed" is always true here. Is there a scenario that I might
be missing?

> +
>  		p->numa_preferred_nid = nid;
> +		if (changed && p->sched_class->numa_node_changed)
> +			p->sched_class->numa_node_changed(task_rq(p), p, nid);
> +	}
>  }
>  #endif /* CONFIG_NUMA_BALANCING */

-- 
Thanks and Regards,
Prateek

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[K Prateek Nayak]

Hello Andrea,

On 10/27/2026 9:43 AM, Andrea Righi wrote:
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 36df9a4dde534..5064c7b707f4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
>  
>  	rcu_read_lock();
>  	cur = rcu_dereference_all(dst_rq->curr);
> +	/*
> +	 * Never pick a task under a BPF scheduler as the swap partner: its
> +	 * placement is left to that scheduler, see numa_migrate_preferred().
> +	 * Moving the faulting task alone to the destination is still allowed.
> +	 */
>  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> -		    !cur->mm))
> +		    !cur->mm || task_on_scx(cur)))

So if you are here, it must be from:

  task_numa_migrate()
    task_numa_find_cpu()
      task_numa_compare()

but task_on_scx(p) would skip this and it means a fair task is looking
for a swap candidate when scx running in partial mode right?

At that point, it feels fair class should be well within its rights to
call shots on the swap candidate even if it is an scx task.

If scx has a ops.set_numa() defined, it would have accumulated hinting
faults and the CPU it is running on is a wrong node anyways so the
migration shouldn't be bad in grand scheme of things. If ops.set_numa()
is not defined, then the scheduler doesn't care about NUMA placement
and it should be okay for a fair task to acquire that CPU.

>  		cur = NULL;
>  
>  	/*
> @@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
>  	if (task_node(p) == p->numa_preferred_nid)
>  		return;
>  
> +	/*
> +	 * A task under a BPF scheduler is placed by that scheduler. Keep the
> +	 * statistics coming, which is what makes p->numa_preferred_nid worth
> +	 * reading, but leave the placement alone.
> +	 */
> +	if (task_on_scx(p))
> +		return;
> +

Restricting ext on this path seems fine.

>  	/* Otherwise, try migrate to a CPU on the preferred node */
>  	task_numa_migrate(p);
>  }

-- 
Thanks and Regards,
Prateek

Re: [PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[K Prateek Nayak]

Hello Andrea,

On 10/27/2026 9:43 AM, Andrea Righi wrote:
> +__bpf_kfunc s32 scx_bpf_task_numa_nid(const struct task_struct *p)
> +{
> +#ifdef CONFIG_NUMA_BALANCING
> +	if (task_on_scx(p)) {
> +		struct scx_sched *sch;
> +
> +		guard(rcu)();
> +		sch = scx_task_sched_rcu(p);
> +		if (!sch || !SCX_HAS_OP(sch, set_numa_node))
> +			return NUMA_NO_NODE;
> +	}
> +
> +	return READ_ONCE(p->numa_preferred_nid);

Would the ext scheduler need the raw numa_preferred_nid when
sched_numa_balancing is disabled?

It may just be pointing to some stale value if NUMA balancing
was disabled at runtime right?

> +#else
> +	return NUMA_NO_NODE;
> +#endif
> +}
> +
>  /**
>   * scx_bpf_locked_rq - Return the rq currently locked by SCX
>   * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs
> @@ -11836,6 +11877,7 @@ BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE)
>  BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU)

On a tangential note, scx_bpf_task_running() just does a
rcu_access_pointer() (resolves to a READ_ONCE()) and doesn't take a
rcu_read_lock() so does it really need KF_RCU?

>  BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU)
>  BTF_ID_FLAGS(func, scx_bpf_task_cid, KF_RCU)
> +BTF_ID_FLAGS(func, scx_bpf_task_numa_nid, KF_RCU)
>  BTF_ID_FLAGS(func, scx_bpf_locked_rq, KF_IMPLICIT_ARGS | KF_RET_NULL)
>  BTF_ID_FLAGS(func, scx_bpf_cpu_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)
>  BTF_ID_FLAGS(func, scx_bpf_cid_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)

-- 
Thanks and Regards,
Prateek

Re: [PATCH 3/6] sched/numa: Notify the scheduling class of preferred node changes

[Andrea Righi]

Hi Prateek,

On Tue, Oct 27, 2026 at 11:00:17AM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 10/27/2026 9:43 AM, Andrea Righi wrote:
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -8331,8 +8331,13 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
> >  void sched_setnuma(struct task_struct *p, int nid)
> >  {
> >  	guard(task_rq_lock)(p);
> > -	scoped_guard (sched_change, p, DEQUEUE_SAVE)
> > +	scoped_guard (sched_change, p, DEQUEUE_SAVE) {
> > +		bool changed = p->numa_preferred_nid != nid;
> 
> Don't the callers of sched_setnuma() already check for this?
> 
> Only one that doesn't is task_numa_migrate() but
> numa_migrate_preferred() only calls it when
> "task_node(p) == p->numa_preferred_nid" is false and "!sd "case resets
> "p->numa_preferred_nid" to "task_node(p)" when the task cannot be
> migrated.
> 
> Afaict, "changed" is always true here. Is there a scenario that I might
> be missing?

With the current callers it's always true for the tasks that can reach
the callback.

 added the check just because ->numa_node_changed() is a generic
sched_class callback and its contract is that it's invoked only when the
preferred node actually changes. Checking it in sched_setnuma(), under
the rq lock we're already holding, makes sure that the contract is
honored independently of what the callers of sched_setnuma() are doing,
now and in the future, instead of relying on each of them to check it.

That said, I don't have a strong opinion on that, if we prefer to keep
sched_setnuma() minimal, I can drop the check and document that the
callers must only invoke it on a change.

Thanks,
-Andrea

> 
> > +
> >  		p->numa_preferred_nid = nid;
> > +		if (changed && p->sched_class->numa_node_changed)
> > +			p->sched_class->numa_node_changed(task_rq(p), p, nid);
> > +	}
> >  }
> >  #endif /* CONFIG_NUMA_BALANCING */
> 
> -- 
> Thanks and Regards,
> Prateek
> 

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Andrea Righi]

Hi Prateek,

On Tue, Oct 27, 2026 at 11:15:58AM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 10/27/2026 9:43 AM, Andrea Righi wrote:
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 36df9a4dde534..5064c7b707f4d 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
> >  
> >  	rcu_read_lock();
> >  	cur = rcu_dereference_all(dst_rq->curr);
> > +	/*
> > +	 * Never pick a task under a BPF scheduler as the swap partner: its
> > +	 * placement is left to that scheduler, see numa_migrate_preferred().
> > +	 * Moving the faulting task alone to the destination is still allowed.
> > +	 */
> >  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> > -		    !cur->mm))
> > +		    !cur->mm || task_on_scx(cur)))
> 
> So if you are here, it must be from:
> 
>   task_numa_migrate()
>     task_numa_find_cpu()
>       task_numa_compare()
> 
> but task_on_scx(p) would skip this and it means a fair task is looking
> for a swap candidate when scx running in partial mode right?

Correct.

> 
> At that point, it feels fair class should be well within its rights to
> call shots on the swap candidate even if it is an scx task.
> 
> If scx has a ops.set_numa() defined, it would have accumulated hinting
> faults and the CPU it is running on is a wrong node anyways so the
> migration shouldn't be bad in grand scheme of things. If ops.set_numa()
> is not defined, then the scheduler doesn't care about NUMA placement
> and it should be okay for a fair task to acquire that CPU.

The fair task can still acquire that CPU: with cur = NULL,
task_numa_compare() can still pick the destination CPU for the fair task
alone, and once it's there it'll preempt the sched_ext task, since fair
wins over scx. What the check prevents is only the other half of the
swap, which is fair moving the scx task to the source CPU, overriding
the placement decided by the BPF scheduler (the idea is that fair should
be able to preempt scx, but not override scx's task placement).

And the fact that the scx task is not on its preferred node doesn't
necessarily mean that it's on the wrong node: the preferred node only
tracks memory locality, but the BPF scheduler may have placed the task
there on purpose for other reasons (e.g., close to a device, which is
one of the use cases that motivated this work), using the preferred node
only as a hint.

There's also a more practical problem: the swap decision is based on
data that isn't maintained for sched_ext tasks. task_numa_compare()
does:

	load = task_h_load(env->p) - task_h_load(cur);

and for an scx task se.avg is not updated, so task_h_load(cur) reflects
whatever load the task had when it left fair. Similarly, if the BPF
scheduler doesn't implement ops.set_numa_node(), the task isn't scanned,
so its NUMA fault statistics are stale as well. So fair would be
deciding the swap based on frozen values.

So I think it makes sense to apply the same principle as in
numa_migrate_preferred() also to the other task involved in the swap.

What do you think?

Thanks,
-Andrea

> 
> >  		cur = NULL;
> >  
> >  	/*
> > @@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
> >  	if (task_node(p) == p->numa_preferred_nid)
> >  		return;
> >  
> > +	/*
> > +	 * A task under a BPF scheduler is placed by that scheduler. Keep the
> > +	 * statistics coming, which is what makes p->numa_preferred_nid worth
> > +	 * reading, but leave the placement alone.
> > +	 */
> > +	if (task_on_scx(p))
> > +		return;
> > +
> 
> Restricting ext on this path seems fine.
> 
> >  	/* Otherwise, try migrate to a CPU on the preferred node */
> >  	task_numa_migrate(p);
> >  }
> 
> -- 
> Thanks and Regards,
> Prateek
> 

Re: [PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[Andrea Righi]

Hi Prateek,

On Tue, Oct 27, 2026 at 11:42:22AM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 10/27/2026 9:43 AM, Andrea Righi wrote:
> > +__bpf_kfunc s32 scx_bpf_task_numa_nid(const struct task_struct *p)
> > +{
> > +#ifdef CONFIG_NUMA_BALANCING
> > +	if (task_on_scx(p)) {
> > +		struct scx_sched *sch;
> > +
> > +		guard(rcu)();
> > +		sch = scx_task_sched_rcu(p);
> > +		if (!sch || !SCX_HAS_OP(sch, set_numa_node))
> > +			return NUMA_NO_NODE;
> > +	}
> > +
> > +	return READ_ONCE(p->numa_preferred_nid);
> 
> Would the ext scheduler need the raw numa_preferred_nid when
> sched_numa_balancing is disabled?
> 
> It may just be pointing to some stale value if NUMA balancing
> was disabled at runtime right?

Right, if NUMA balancing is disabled at runtime the value stays at the
last preference.

This is consistent with fair: disabling NUMA balancing at runtime only
flips the sched_numa_balancing static key, p->numa_preferred_nid is
never reset, fair stops acting on it in most places (e.g.,
migrate_degrades_locality()), but not everywhere, find_idlest_group()
still uses it in the slow wakeup path. So a BPF scheduler sees the same
value that fair keeps and can decide whether to use it or not.

A more accurate approach could be to notify the sched classes when
sched_numa_balancing changes (e.g., resetting the preferred node of all
tasks when it's disabled and recomputing it when it's re-enabled), but
that would be a significant rework, involving fair as well, and it's
outside the scope of this series. I'd rather start by making ext
consistent with fair, and then rework both in a more accurate and
complete way as a follow-up, if we want.

> 
> > +#else
> > +	return NUMA_NO_NODE;
> > +#endif
> > +}
> > +
> >  /**
> >   * scx_bpf_locked_rq - Return the rq currently locked by SCX
> >   * @aux: implicit BPF argument to access bpf_prog_aux hidden from BPF progs
> > @@ -11836,6 +11877,7 @@ BTF_ID_FLAGS(func, scx_bpf_put_cpumask, KF_RELEASE)
> >  BTF_ID_FLAGS(func, scx_bpf_task_running, KF_RCU)
> 
> On a tangential note, scx_bpf_task_running() just does a
> rcu_access_pointer() (resolves to a READ_ONCE()) and doesn't take a
> rcu_read_lock() so does it really need KF_RCU?

Ah yes, that's tricky. KF_RCU doesn't mean that the kfunc takes
rcu_read_lock(), it's a verifier flag that allows the kfunc to accept
RCU-protected pointers as arguments, in addition to trusted (refcounted)
ones.

scx_bpf_task_running() still dereferences p (task_rq(p)->donor), so p
needs to be valid for the duration of the call, and KF_RCU is the
weakest guarantee that the verifier can enforce for that. Same for
scx_bpf_task_numa_nid().

> >  BTF_ID_FLAGS(func, scx_bpf_task_cpu, KF_RCU)
> >  BTF_ID_FLAGS(func, scx_bpf_task_cid, KF_RCU)
> > +BTF_ID_FLAGS(func, scx_bpf_task_numa_nid, KF_RCU)
> >  BTF_ID_FLAGS(func, scx_bpf_locked_rq, KF_IMPLICIT_ARGS | KF_RET_NULL)
> >  BTF_ID_FLAGS(func, scx_bpf_cpu_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)
> >  BTF_ID_FLAGS(func, scx_bpf_cid_curr, KF_IMPLICIT_ARGS | KF_RET_NULL | KF_RCU_PROTECTED)

Thanks for taking a look at this series!
-Andrea

Re: [PATCH 3/6] sched/numa: Notify the scheduling class of preferred node changes

[K Prateek Nayak]

Hello Andrea,

>> On 10/27/2026 9:43 AM, Andrea Righi wrote:
>>> --- a/kernel/sched/core.c
>>> +++ b/kernel/sched/core.c
>>> @@ -8331,8 +8331,13 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
>>>  void sched_setnuma(struct task_struct *p, int nid)
>>>  {
>>>  	guard(task_rq_lock)(p);
>>> -	scoped_guard (sched_change, p, DEQUEUE_SAVE)
>>> +	scoped_guard (sched_change, p, DEQUEUE_SAVE) {
>>> +		bool changed = p->numa_preferred_nid != nid;
>>
>> Don't the callers of sched_setnuma() already check for this?
>>
>> Only one that doesn't is task_numa_migrate() but
>> numa_migrate_preferred() only calls it when
>> "task_node(p) == p->numa_preferred_nid" is false and "!sd "case resets
>> "p->numa_preferred_nid" to "task_node(p)" when the task cannot be
>> migrated.
>>
>> Afaict, "changed" is always true here. Is there a scenario that I might
>> be missing?
> 
> With the current callers it's always true for the tasks that can reach
> the callback.
> 
>  added the check just because ->numa_node_changed() is a generic
> sched_class callback and its contract is that it's invoked only when the
> preferred node actually changes. Checking it in sched_setnuma(), under
> the rq lock we're already holding, makes sure that the contract is
> honored independently of what the callers of sched_setnuma() are doing,
> now and in the future, instead of relying on each of them to check it.
> 
> That said, I don't have a strong opinion on that, if we prefer to keep
> sched_setnuma() minimal, I can drop the check and document that the
> callers must only invoke it on a change.

I too don't have a strong opinion but I'm all for smaller diffstat
whenever possible ;-)

Let us wait on Peter in case he would prefer to have the defensive
check. FWIW, I've been running a kernel with a:

  WARN_ON_ONCE(p->numa_preferred_nid == nid)

for a while and I haven't hit any warnings yet from sched_setnuma().

-- 
Thanks and Regards,
Prateek

"Program testing can be used to show the presence of bugs,
 but never to show their absence!" - Edsger W. Dijkstra

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[K Prateek Nayak]

Hello Andrea,

On 10/27/2026 12:13 PM, Andrea Righi wrote:
>> On 10/27/2026 9:43 AM, Andrea Righi wrote:
>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>>> index 36df9a4dde534..5064c7b707f4d 100644
>>> --- a/kernel/sched/fair.c
>>> +++ b/kernel/sched/fair.c
>>> @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
>>>  
>>>  	rcu_read_lock();
>>>  	cur = rcu_dereference_all(dst_rq->curr);
>>> +	/*
>>> +	 * Never pick a task under a BPF scheduler as the swap partner: its
>>> +	 * placement is left to that scheduler, see numa_migrate_preferred().
>>> +	 * Moving the faulting task alone to the destination is still allowed.
>>> +	 */
>>>  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
>>> -		    !cur->mm))
>>> +		    !cur->mm || task_on_scx(cur)))
>>
>> So if you are here, it must be from:
>>
>>   task_numa_migrate()
>>     task_numa_find_cpu()
>>       task_numa_compare()
>>
>> but task_on_scx(p) would skip this and it means a fair task is looking
>> for a swap candidate when scx running in partial mode right?
> 
> Correct.
> 
>>
>> At that point, it feels fair class should be well within its rights to
>> call shots on the swap candidate even if it is an scx task.
>>
>> If scx has a ops.set_numa() defined, it would have accumulated hinting
>> faults and the CPU it is running on is a wrong node anyways so the
>> migration shouldn't be bad in grand scheme of things. If ops.set_numa()
>> is not defined, then the scheduler doesn't care about NUMA placement
>> and it should be okay for a fair task to acquire that CPU.
> 
> The fair task can still acquire that CPU: with cur = NULL,
> task_numa_compare() can still pick the destination CPU for the fair task
> alone, and once it's there it'll preempt the sched_ext task, since fair
> wins over scx. What the check prevents is only the other half of the
> swap, which is fair moving the scx task to the source CPU, overriding
> the placement decided by the BPF scheduler (the idea is that fair should
> be able to preempt scx, but not override scx's task placement).

I guess that is fair - if there is a better target, it should be
preferred. 
> And the fact that the scx task is not on its preferred node doesn't
> necessarily mean that it's on the wrong node: the preferred node only
> tracks memory locality, but the BPF scheduler may have placed the task
> there on purpose for other reasons (e.g., close to a device, which is
> one of the use cases that motivated this work), using the preferred node
> only as a hint.

But is anyone running ext in partial mode expecting good performance
without having moved a ton of tasks to ext beforehand?
> There's also a more practical problem: the swap decision is based on
> data that isn't maintained for sched_ext tasks. task_numa_compare()
> does:
> 
> 	load = task_h_load(env->p) - task_h_load(cur);
> 
> and for an scx task se.avg is not updated, so task_h_load(cur) reflects
> whatever load the task had when it left fair. Similarly, if the BPF
> scheduler doesn't implement ops.set_numa_node(), the task isn't scanned,
> so its NUMA fault statistics are stale as well. So fair would be
> deciding the swap based on frozen values.

I think for fair, since there are no other fair tasks, it is indeed a
CPU without load - much like an idle CPU. NUMA balancing also looks
at load imbalance on the migrate swap path and it wouldn't have seen
the load back then either :-) 
> So I think it makes sense to apply the same principle as in
> numa_migrate_preferred() also to the other task involved in the swap.
> 
> What do you think?

Ack! I think you can keep that check. Let us not disturb a CPU just
because it is preemptible if there is a better option / idle CPUs
around.

-- 
Thanks and Regards,
Prateek

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Andrea Righi]

On Tue, Oct 27, 2026 at 02:30:31PM +0530, K Prateek Nayak wrote:
> Hello Andrea,
> 
> On 10/27/2026 12:13 PM, Andrea Righi wrote:
> >> On 10/27/2026 9:43 AM, Andrea Righi wrote:
> >>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >>> index 36df9a4dde534..5064c7b707f4d 100644
> >>> --- a/kernel/sched/fair.c
> >>> +++ b/kernel/sched/fair.c
> >>> @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
> >>>  
> >>>  	rcu_read_lock();
> >>>  	cur = rcu_dereference_all(dst_rq->curr);
> >>> +	/*
> >>> +	 * Never pick a task under a BPF scheduler as the swap partner: its
> >>> +	 * placement is left to that scheduler, see numa_migrate_preferred().
> >>> +	 * Moving the faulting task alone to the destination is still allowed.
> >>> +	 */
> >>>  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> >>> -		    !cur->mm))
> >>> +		    !cur->mm || task_on_scx(cur)))
> >>
> >> So if you are here, it must be from:
> >>
> >>   task_numa_migrate()
> >>     task_numa_find_cpu()
> >>       task_numa_compare()
> >>
> >> but task_on_scx(p) would skip this and it means a fair task is looking
> >> for a swap candidate when scx running in partial mode right?
> > 
> > Correct.
> > 
> >>
> >> At that point, it feels fair class should be well within its rights to
> >> call shots on the swap candidate even if it is an scx task.
> >>
> >> If scx has a ops.set_numa() defined, it would have accumulated hinting
> >> faults and the CPU it is running on is a wrong node anyways so the
> >> migration shouldn't be bad in grand scheme of things. If ops.set_numa()
> >> is not defined, then the scheduler doesn't care about NUMA placement
> >> and it should be okay for a fair task to acquire that CPU.
> > 
> > The fair task can still acquire that CPU: with cur = NULL,
> > task_numa_compare() can still pick the destination CPU for the fair task
> > alone, and once it's there it'll preempt the sched_ext task, since fair
> > wins over scx. What the check prevents is only the other half of the
> > swap, which is fair moving the scx task to the source CPU, overriding
> > the placement decided by the BPF scheduler (the idea is that fair should
> > be able to preempt scx, but not override scx's task placement).
> 
> I guess that is fair - if there is a better target, it should be
> preferred. 
> > And the fact that the scx task is not on its preferred node doesn't
> > necessarily mean that it's on the wrong node: the preferred node only
> > tracks memory locality, but the BPF scheduler may have placed the task
> > there on purpose for other reasons (e.g., close to a device, which is
> > one of the use cases that motivated this work), using the preferred node
> > only as a hint.
> 
> But is anyone running ext in partial mode expecting good performance
> without having moved a ton of tasks to ext beforehand?

Yeah, partial mode has always been a bit controversial, users of this
mode are mostly those that rely on fair-specific features (e.g., EAS)
but still want to experiment with sched_ext. And to make it work
effectively, the best approach is to statically partition the system, to
avoid mixing fair and ext tasks on the same CPUs.

In that scenario I think it's even more important to not let fair move
scx tasks around. Hopefully, once the ext class is able to provides all
the features that fair provides, we can stop supporting partial mode.

Thanks,
-Andrea

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 05:13:31AM +0100, Andrea Righi wrote:
> A task under a BPF scheduler can take NUMA hinting faults via
> task_numa_fault() and then reach numa_migrate_preferred(), which may
> migrate the task to its preferred node through task_numa_migrate().
> 
> The migration of such a task should always be driven by its BPF
> scheduler, which owns the placement of its tasks.
> 
> Keep collecting the statistics, which is what makes
> p->numa_preferred_nid worth reading, but skip the migration for a task
> owned by a BPF scheduler. The scheduler can read the preferred node and
> decide for itself whether moving the task is worth the cost.
> 
> For the same reason, never pick a task owned by a BPF scheduler as the
> swap partner of another task that NUMA balancing migrates, e.g., a fair
> task when the BPF scheduler runs in partial mode. Such a task is
> treated like a kthread on the destination CPU: the migrating task can
> still move there alone, but the BPF-scheduled task is left in place.
> 
> Reported-by: Vladimir Vdovin <deliran@verdict.gg>
> Link: https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
> Signed-off-by: Andrea Righi <arighi@nvidia.com>
> ---
>  kernel/sched/fair.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 36df9a4dde534..5064c7b707f4d 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
>  
>  	rcu_read_lock();
>  	cur = rcu_dereference_all(dst_rq->curr);
> +	/*
> +	 * Never pick a task under a BPF scheduler as the swap partner: its
> +	 * placement is left to that scheduler, see numa_migrate_preferred().
> +	 * Moving the faulting task alone to the destination is still allowed.
> +	 */
>  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> -		    !cur->mm))
> +		    !cur->mm || task_on_scx(cur)))
>  		cur = NULL;

Rather than check task_on_scx(); your argument with Prateek seems to
highlight that the actual check you're looking for should be something
like: cur->class != &fair_sched_class or somesuch, right?

>  	/*
> @@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
>  	if (task_node(p) == p->numa_preferred_nid)
>  		return;
>  
> +	/*
> +	 * A task under a BPF scheduler is placed by that scheduler. Keep the
> +	 * statistics coming, which is what makes p->numa_preferred_nid worth
> +	 * reading, but leave the placement alone.
> +	 */
> +	if (task_on_scx(p))
> +		return;
> +
>  	/* Otherwise, try migrate to a CPU on the preferred node */
>  	task_numa_migrate(p);
>  }

I am somewhat confused on all this still, probably because I've not
looked at it for more than a couple of minutes at this point :-)

But rather than putting a callback in sched_setnuma() should you not put
a class call here?

That is; have this be:

	p->class->task_numa_migrate(p);

and have fair set task_numa_migrate_fair(), which is the current
function, and then ext can implement the BPF thing.

Anyway, if you want to do this task_on_scx() thing like you did, its in
the wrong spot, that condition right above it is irrelevant if you're
not going to be calling task_numa_migrate(). 

Re: [PATCH 4/6] sched_ext: Report preferred NUMA nodes through ops.set_numa_node()

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 05:13:33AM +0100, Andrea Righi wrote:

> @@ -4528,6 +4543,15 @@ static void __scx_enable_task(struct scx_sched *sch, struct task_struct *p)
>  
>  	if (SCX_HAS_OP(sch, set_weight))
>  		SCX_CALL_OP_TASK(sch, set_weight, rq, p, p->scx.weight);
> +
> +	if (SCX_HAS_OP(sch, set_numa_node)) {
> +		s32 nid = NUMA_NO_NODE;
> +
> +#ifdef CONFIG_NUMA_BALANCING
> +		nid = p->numa_preferred_nid;
> +#endif
> +		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
> +	}
>  }
>  
>  void scx_enable_task(struct scx_sched *sch, struct task_struct *p)
> @@ -4834,6 +4858,20 @@ void sched_ext_dead(struct task_struct *p)
>  	}
>  }
>  
> +#ifdef CONFIG_NUMA_BALANCING
> +static void numa_node_changed_scx(struct rq *rq, struct task_struct *p, int nid)
> +{
> +	struct scx_sched *sch = scx_task_sched(p);
> +
> +	if (task_dead_and_done(p) ||
> +	    scx_get_task_state(p) != SCX_TASK_ENABLED)
> +		return;
> +
> +	if (SCX_HAS_OP(sch, set_numa_node))
> +		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
> +}
> +#endif

I dont get this; why are you calling set_numa_mode() on enable? The
signal was sched_setnuma(), but now you're doing more.

Should not the existing enable method check p->numa_preferred_nid rather
than injecting this weird extra call?

Re: [PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 07:57:36AM +0100, Andrea Righi wrote:
> Hi Prateek,
> 
> On Tue, Oct 27, 2026 at 11:42:22AM +0530, K Prateek Nayak wrote:
> > Hello Andrea,
> > 
> > On 10/27/2026 9:43 AM, Andrea Righi wrote:
> > > +__bpf_kfunc s32 scx_bpf_task_numa_nid(const struct task_struct *p)
> > > +{
> > > +#ifdef CONFIG_NUMA_BALANCING
> > > +	if (task_on_scx(p)) {
> > > +		struct scx_sched *sch;
> > > +
> > > +		guard(rcu)();
> > > +		sch = scx_task_sched_rcu(p);
> > > +		if (!sch || !SCX_HAS_OP(sch, set_numa_node))
> > > +			return NUMA_NO_NODE;
> > > +	}
> > > +
> > > +	return READ_ONCE(p->numa_preferred_nid);
> > 
> > Would the ext scheduler need the raw numa_preferred_nid when
> > sched_numa_balancing is disabled?
> > 
> > It may just be pointing to some stale value if NUMA balancing
> > was disabled at runtime right?
> 
> Right, if NUMA balancing is disabled at runtime the value stays at the
> last preference.
> 
> This is consistent with fair: disabling NUMA balancing at runtime only
> flips the sched_numa_balancing static key, p->numa_preferred_nid is
> never reset, fair stops acting on it in most places (e.g.,
> migrate_degrades_locality()), but not everywhere, find_idlest_group()
> still uses it in the slow wakeup path. So a BPF scheduler sees the same
> value that fair keeps and can decide whether to use it or not.
> 
> A more accurate approach could be to notify the sched classes when
> sched_numa_balancing changes (e.g., resetting the preferred node of all
> tasks when it's disabled and recomputing it when it's re-enabled), but
> that would be a significant rework, involving fair as well, and it's
> outside the scope of this series. I'd rather start by making ext
> consistent with fair, and then rework both in a more accurate and
> complete way as a follow-up, if we want.

But should we not cleanup first and then add shiny on top?

Re: [PATCHSET v2 sched_ext/for-7.5] sched_ext: Add NUMA balancing support

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 05:13:29AM +0100, Andrea Righi wrote:

> Ideally the ultimate goal is to give BPF schedulers both controls that
> NUMA balancing combines: moving a task to the node where its memory is
> and moving the memory a task uses to the node where the task runs. This
> series provides the first: with the preferred node available, a
> scheduler can decide where to run the task. A second series, to follow
> once this one lands, will provide the second: a kfunc that lets a
> scheduler request the node a task's memory should migrate to.

Fair has no real control over the memory migration part either; why
would you want to give BPF that?

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Andrea Righi]

Hi Peter,

On Tue, Oct 27, 2026 at 11:43:17AM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2026 at 05:13:31AM +0100, Andrea Righi wrote:
> > A task under a BPF scheduler can take NUMA hinting faults via
> > task_numa_fault() and then reach numa_migrate_preferred(), which may
> > migrate the task to its preferred node through task_numa_migrate().
> > 
> > The migration of such a task should always be driven by its BPF
> > scheduler, which owns the placement of its tasks.
> > 
> > Keep collecting the statistics, which is what makes
> > p->numa_preferred_nid worth reading, but skip the migration for a task
> > owned by a BPF scheduler. The scheduler can read the preferred node and
> > decide for itself whether moving the task is worth the cost.
> > 
> > For the same reason, never pick a task owned by a BPF scheduler as the
> > swap partner of another task that NUMA balancing migrates, e.g., a fair
> > task when the BPF scheduler runs in partial mode. Such a task is
> > treated like a kthread on the destination CPU: the migrating task can
> > still move there alone, but the BPF-scheduled task is left in place.
> > 
> > Reported-by: Vladimir Vdovin <deliran@verdict.gg>
> > Link: https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
> > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> > ---
> >  kernel/sched/fair.c | 15 ++++++++++++++-
> >  1 file changed, 14 insertions(+), 1 deletion(-)
> > 
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 36df9a4dde534..5064c7b707f4d 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
> >  
> >  	rcu_read_lock();
> >  	cur = rcu_dereference_all(dst_rq->curr);
> > +	/*
> > +	 * Never pick a task under a BPF scheduler as the swap partner: its
> > +	 * placement is left to that scheduler, see numa_migrate_preferred().
> > +	 * Moving the faulting task alone to the destination is still allowed.
> > +	 */
> >  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> > -		    !cur->mm))
> > +		    !cur->mm || task_on_scx(cur)))
> >  		cur = NULL;
> 
> Rather than check task_on_scx(); your argument with Prateek seems to
> highlight that the actual check you're looking for should be something
> like: cur->class != &fair_sched_class or somesuch, right?

Right, the actual condition is "cur is not a fair task". I used
task_on_scx() only to avoid changing the behavior of RT/DL tasks, but
the same arguments apply to them: their se.avg isn't updated either, so
task_h_load(cur) is stale, and their placement is driven by their own
class (push/pull), so fair shouldn't move them around.

So yeah... "task not in the fair class" seems the right thing to do.

> 
> >  	/*
> > @@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
> >  	if (task_node(p) == p->numa_preferred_nid)
> >  		return;
> >  
> > +	/*
> > +	 * A task under a BPF scheduler is placed by that scheduler. Keep the
> > +	 * statistics coming, which is what makes p->numa_preferred_nid worth
> > +	 * reading, but leave the placement alone.
> > +	 */
> > +	if (task_on_scx(p))
> > +		return;
> > +
> >  	/* Otherwise, try migrate to a CPU on the preferred node */
> >  	task_numa_migrate(p);
> >  }
> 
> I am somewhat confused on all this still, probably because I've not
> looked at it for more than a couple of minutes at this point :-)
> 
> But rather than putting a callback in sched_setnuma() should you not put
> a class call here?
> 
> That is; have this be:
> 
> 	p->class->task_numa_migrate(p);
> 
> and have fair set task_numa_migrate_fair(), which is the current
> function, and then ext can implement the BPF thing.

Yeah, that'd be a much cleaner approach than special-casing ext in fair.
But I think it gives a different signal than the sched_setnuma()
callback.

numa_migrate_preferred() is called from task_numa_fault(), in the
context of the faulting task, at most every numa_migrate_retry, and it
only gets to this point when the task isn't on its preferred node. So a
class call here means "this task is misplaced, migrate it if you want",
repeated at each retry, even if the preferred node remains the same.

The sched_setnuma() callback instead reports when the preferred node
changes, which is what a BPF scheduler needs to use the preferred node
as a placement hint.

So I think the two are complementary: the class call would replace the
task_on_scx() check here (ext can just skip the migration, or expose it
to BPF if we think it's useful), while the notification would stay in
sched_setnuma().

One thing to consider: RT/DL tasks would also stop being migrated here
(they can take hinting faults too), which seems more correct honestly:
task_numa_migrate() decides the move based on task_h_load(p), which is
stale for them, and their faults only come from pages scanned by other
fair threads of the same mm, since they don't drive the scanning
themselves.

Would that work for you?

> 
> Anyway, if you want to do this task_on_scx() thing like you did, its in
> the wrong spot, that condition right above it is irrelevant if you're
> not going to be calling task_numa_migrate(). 

Right, I'll move it before that check. It still needs to stay after the
p->numa_migrate_retry update though, otherwise the time_after() check in
task_numa_fault() would be always true for scx tasks and
task_numa_placement() would be called at every hinting fault.

Thanks,
-Andrea

Re: [PATCH 4/6] sched_ext: Report preferred NUMA nodes through ops.set_numa_node()

[Andrea Righi]

On Tue, Oct 27, 2026 at 11:49:01AM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2026 at 05:13:33AM +0100, Andrea Righi wrote:
> 
> > @@ -4528,6 +4543,15 @@ static void __scx_enable_task(struct scx_sched *sch, struct task_struct *p)
> >  
> >  	if (SCX_HAS_OP(sch, set_weight))
> >  		SCX_CALL_OP_TASK(sch, set_weight, rq, p, p->scx.weight);
> > +
> > +	if (SCX_HAS_OP(sch, set_numa_node)) {
> > +		s32 nid = NUMA_NO_NODE;
> > +
> > +#ifdef CONFIG_NUMA_BALANCING
> > +		nid = p->numa_preferred_nid;
> > +#endif
> > +		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
> > +	}
> >  }
> >  
> >  void scx_enable_task(struct scx_sched *sch, struct task_struct *p)
> > @@ -4834,6 +4858,20 @@ void sched_ext_dead(struct task_struct *p)
> >  	}
> >  }
> >  
> > +#ifdef CONFIG_NUMA_BALANCING
> > +static void numa_node_changed_scx(struct rq *rq, struct task_struct *p, int nid)
> > +{
> > +	struct scx_sched *sch = scx_task_sched(p);
> > +
> > +	if (task_dead_and_done(p) ||
> > +	    scx_get_task_state(p) != SCX_TASK_ENABLED)
> > +		return;
> > +
> > +	if (SCX_HAS_OP(sch, set_numa_node))
> > +		SCX_CALL_OP_TASK(sch, set_numa_node, rq, p, nid);
> > +}
> > +#endif
> 
> I dont get this; why are you calling set_numa_mode() on enable? The
> signal was sched_setnuma(), but now you're doing more.
> 
> Should not the existing enable method check p->numa_preferred_nid rather
> than injecting this weird extra call?

This follows what sched_ext already does with the other "set" callbacks:
ops.set_weight() and ops.set_cpumask() are also invoked at enable with
the initial value, right after ops.enable(), so that a BPF scheduler can
track that property in a single callback, instead of reading the initial
value in ops.enable() and then tracking the changes in a separate
callback.

That said, ops.enable() can also read the initial value directly with
scx_bpf_task_numa_nid() (introduced in patch 5), so if we think it's
cleaner I can drop the call at enable and document that ops.enable()
should do that.

Thanks,
-Andrea

Re: [PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[Andrea Righi]

On Tue, Oct 27, 2026 at 11:50:56AM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2026 at 07:57:36AM +0100, Andrea Righi wrote:
> > Hi Prateek,
> > 
> > On Tue, Oct 27, 2026 at 11:42:22AM +0530, K Prateek Nayak wrote:
> > > Hello Andrea,
> > > 
> > > On 10/27/2026 9:43 AM, Andrea Righi wrote:
> > > > +__bpf_kfunc s32 scx_bpf_task_numa_nid(const struct task_struct *p)
> > > > +{
> > > > +#ifdef CONFIG_NUMA_BALANCING
> > > > +	if (task_on_scx(p)) {
> > > > +		struct scx_sched *sch;
> > > > +
> > > > +		guard(rcu)();
> > > > +		sch = scx_task_sched_rcu(p);
> > > > +		if (!sch || !SCX_HAS_OP(sch, set_numa_node))
> > > > +			return NUMA_NO_NODE;
> > > > +	}
> > > > +
> > > > +	return READ_ONCE(p->numa_preferred_nid);
> > > 
> > > Would the ext scheduler need the raw numa_preferred_nid when
> > > sched_numa_balancing is disabled?
> > > 
> > > It may just be pointing to some stale value if NUMA balancing
> > > was disabled at runtime right?
> > 
> > Right, if NUMA balancing is disabled at runtime the value stays at the
> > last preference.
> > 
> > This is consistent with fair: disabling NUMA balancing at runtime only
> > flips the sched_numa_balancing static key, p->numa_preferred_nid is
> > never reset, fair stops acting on it in most places (e.g.,
> > migrate_degrades_locality()), but not everywhere, find_idlest_group()
> > still uses it in the slow wakeup path. So a BPF scheduler sees the same
> > value that fair keeps and can decide whether to use it or not.
> > 
> > A more accurate approach could be to notify the sched classes when
> > sched_numa_balancing changes (e.g., resetting the preferred node of all
> > tasks when it's disabled and recomputing it when it's re-enabled), but
> > that would be a significant rework, involving fair as well, and it's
> > outside the scope of this series. I'd rather start by making ext
> > consistent with fair, and then rework both in a more accurate and
> > complete way as a follow-up, if we want.
> 
> But should we not cleanup first and then add shiny on top?

Fair enough. Looking more closely, I think there're only two places in
fair that still use p->numa_preferred_nid when NUMA balancing is
disabled at runtime: find_idlest_group() and the fbq classification in
the load balancer (via rq->nr_numa_running / nr_preferred_running).
Everything else is already gated by sched_numa_balancing.

Assuming I'm not missing anything, we can probably add a preparatory
patch that gates those two on sched_numa_balancing as well, then make
scx_bpf_task_numa_nid() return NUMA_NO_NODE when NUMA balancing is
disabled. In this way the stale value is never used.

Thanks,
-Andrea

Re: [PATCHSET v2 sched_ext/for-7.5] sched_ext: Add NUMA balancing support

[Andrea Righi]

On Tue, Oct 27, 2026 at 11:52:09AM +0100, Peter Zijlstra wrote:
> On Tue, Oct 27, 2026 at 05:13:29AM +0100, Andrea Righi wrote:
> 
> > Ideally the ultimate goal is to give BPF schedulers both controls that
> > NUMA balancing combines: moving a task to the node where its memory is
> > and moving the memory a task uses to the node where the task runs. This
> > series provides the first: with the preferred node available, a
> > scheduler can decide where to run the task. A second series, to follow
> > once this one lands, will provide the second: a kfunc that lets a
> > scheduler request the node a task's memory should migrate to.
> 
> Fair has no real control over the memory migration part either; why
> would you want to give BPF that?

Yeah, that's a fair point and the wording in the cover letter is a bit
too vague.

The use case behind the second part is accelerator/device locality,
discussed at LPC in the sched_ext MC [1]: with multi-GPU systems the
heaviest user of a task's host memory is often the device, which NUMA
balancing can't see, since hinting faults only track CPU accesses. So
memory follows the CPU even when it should stay close to the device, and
today the only option is to pin everything with numactl and/or disable
NUMA balancing.

But that needs to be discussed separately: as Gregory pointed out at
LPC, NUMA balancing needs some cleanups first, and the right interface
may not be BPF-specific at all (e.g., extending the home node concept of
set_mempolicy_home_node() to NUMA balancing, or a device-aware hint in
mm that both fair and ext could use, or something else).

The first step for now is to, at least, allow sched_ext to see the
preferred node computed by NUMA balancing, which is what this series
does.

[1] https://lpc.events/event/20/sessions/270/attachments/2223/5283/lpc-2026-sched_ext-mc-report.md

Thanks,
-Andrea

Re: [PATCH 2/6] sched/numa: Leave the placement of a BPF-scheduled task to its scheduler

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 12:09:33PM +0100, Andrea Righi wrote:
> Hi Peter,
> 
> On Tue, Oct 27, 2026 at 11:43:17AM +0100, Peter Zijlstra wrote:
> > On Tue, Oct 27, 2026 at 05:13:31AM +0100, Andrea Righi wrote:
> > > A task under a BPF scheduler can take NUMA hinting faults via
> > > task_numa_fault() and then reach numa_migrate_preferred(), which may
> > > migrate the task to its preferred node through task_numa_migrate().
> > > 
> > > The migration of such a task should always be driven by its BPF
> > > scheduler, which owns the placement of its tasks.
> > > 
> > > Keep collecting the statistics, which is what makes
> > > p->numa_preferred_nid worth reading, but skip the migration for a task
> > > owned by a BPF scheduler. The scheduler can read the preferred node and
> > > decide for itself whether moving the task is worth the cost.
> > > 
> > > For the same reason, never pick a task owned by a BPF scheduler as the
> > > swap partner of another task that NUMA balancing migrates, e.g., a fair
> > > task when the BPF scheduler runs in partial mode. Such a task is
> > > treated like a kthread on the destination CPU: the migrating task can
> > > still move there alone, but the BPF-scheduled task is left in place.
> > > 
> > > Reported-by: Vladimir Vdovin <deliran@verdict.gg>
> > > Link: https://lore.kernel.org/r/20261002124559.10367-1-deliran@verdict.gg
> > > Signed-off-by: Andrea Righi <arighi@nvidia.com>
> > > ---
> > >  kernel/sched/fair.c | 15 ++++++++++++++-
> > >  1 file changed, 14 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index 36df9a4dde534..5064c7b707f4d 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -3267,8 +3267,13 @@ static bool task_numa_compare(struct task_numa_env *env,
> > >  
> > >  	rcu_read_lock();
> > >  	cur = rcu_dereference_all(dst_rq->curr);
> > > +	/*
> > > +	 * Never pick a task under a BPF scheduler as the swap partner: its
> > > +	 * placement is left to that scheduler, see numa_migrate_preferred().
> > > +	 * Moving the faulting task alone to the destination is still allowed.
> > > +	 */
> > >  	if (cur && ((cur->flags & (PF_EXITING | PF_KTHREAD)) ||
> > > -		    !cur->mm))
> > > +		    !cur->mm || task_on_scx(cur)))
> > >  		cur = NULL;
> > 
> > Rather than check task_on_scx(); your argument with Prateek seems to
> > highlight that the actual check you're looking for should be something
> > like: cur->class != &fair_sched_class or somesuch, right?
> 
> Right, the actual condition is "cur is not a fair task". I used
> task_on_scx() only to avoid changing the behavior of RT/DL tasks, but
> the same arguments apply to them: their se.avg isn't updated either, so
> task_h_load(cur) is stale, and their placement is driven by their own
> class (push/pull), so fair shouldn't move them around.
> 
> So yeah... "task not in the fair class" seems the right thing to do.
> 
> > 
> > >  	/*
> > > @@ -3645,6 +3650,14 @@ static void numa_migrate_preferred(struct task_struct *p)
> > >  	if (task_node(p) == p->numa_preferred_nid)
> > >  		return;
> > >  
> > > +	/*
> > > +	 * A task under a BPF scheduler is placed by that scheduler. Keep the
> > > +	 * statistics coming, which is what makes p->numa_preferred_nid worth
> > > +	 * reading, but leave the placement alone.
> > > +	 */
> > > +	if (task_on_scx(p))
> > > +		return;
> > > +
> > >  	/* Otherwise, try migrate to a CPU on the preferred node */
> > >  	task_numa_migrate(p);
> > >  }
> > 
> > I am somewhat confused on all this still, probably because I've not
> > looked at it for more than a couple of minutes at this point :-)
> > 
> > But rather than putting a callback in sched_setnuma() should you not put
> > a class call here?
> > 
> > That is; have this be:
> > 
> > 	p->class->task_numa_migrate(p);
> > 
> > and have fair set task_numa_migrate_fair(), which is the current
> > function, and then ext can implement the BPF thing.
> 
> Yeah, that'd be a much cleaner approach than special-casing ext in fair.
> But I think it gives a different signal than the sched_setnuma()
> callback.
> 
> numa_migrate_preferred() is called from task_numa_fault(), in the
> context of the faulting task, at most every numa_migrate_retry, and it
> only gets to this point when the task isn't on its preferred node. So a
> class call here means "this task is misplaced, migrate it if you want",
> repeated at each retry, even if the preferred node remains the same.

This is an important signal; it is fundamental to the whole controller.
Without a force to keep a task on its preferred node, the memory will
keep migrating around for no purpose.

A task really should be moved 'back' to its preferred node it at all
possible, and it remaining to run outside of it had better have a good
reason (like imbalance). Of course, if a task is ran elsewhere, then
eventually memory will migrate to the 'new' place and then a new
preferred node will be selected.

The new place is typically constrained by the numa distance, when its
not possible to run on the preferred node, it tries to run near, which
then limits the possible nodes and will have it converge on a new node.

The recently added cache-aware will help here, since that is a short
term signal to stick to one place, ensuring the memory migration is
also likely to be more stable and the whole thing will converge on a new
node more quickly.

> The sched_setnuma() callback instead reports when the preferred node
> changes, which is what a BPF scheduler needs to use the preferred node
> as a placement hint.
> 
> So I think the two are complementary: the class call would replace the
> task_on_scx() check here (ext can just skip the migration, or expose it
> to BPF if we think it's useful), while the notification would stay in
> sched_setnuma().

I am still not quite sure I understand why sched_ext needs the
sched_setnuma() hint for placement.

> One thing to consider: RT/DL tasks would also stop being migrated here
> (they can take hinting faults too), which seems more correct honestly:
> task_numa_migrate() decides the move based on task_h_load(p), which is
> stale for them, and their faults only come from pages scanned by other
> fair threads of the same mm, since they don't drive the scanning
> themselves.
> 
> Would that work for you?

RT/DL should definitely be ignored. 

> > Anyway, if you want to do this task_on_scx() thing like you did, its in
> > the wrong spot, that condition right above it is irrelevant if you're
> > not going to be calling task_numa_migrate(). 
> 
> Right, I'll move it before that check. It still needs to stay after the
> p->numa_migrate_retry update though, otherwise the time_after() check in
> task_numa_fault() would be always true for scx tasks and
> task_numa_placement() would be called at every hinting fault.

Just so.

Re: [PATCH 5/6] sched_ext: Add a kfunc to read a task's preferred NUMA node

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 12:21:01PM +0100, Andrea Righi wrote:

> > But should we not cleanup first and then add shiny on top?
> 
> Fair enough. Looking more closely, I think there're only two places in
> fair that still use p->numa_preferred_nid when NUMA balancing is
> disabled at runtime: find_idlest_group() and the fbq classification in
> the load balancer (via rq->nr_numa_running / nr_preferred_running).
> Everything else is already gated by sched_numa_balancing.
> 
> Assuming I'm not missing anything, we can probably add a preparatory
> patch that gates those two on sched_numa_balancing as well, then make
> scx_bpf_task_numa_nid() return NUMA_NO_NODE when NUMA balancing is
> disabled. In this way the stale value is never used.

Sounds like a plan :-)

Re: [PATCHSET v2 sched_ext/for-7.5] sched_ext: Add NUMA balancing support

[Peter Zijlstra]

On Tue, Oct 27, 2026 at 12:38:49PM +0100, Andrea Righi wrote:
> On Tue, Oct 27, 2026 at 11:52:09AM +0100, Peter Zijlstra wrote:
> > On Tue, Oct 27, 2026 at 05:13:29AM +0100, Andrea Righi wrote:
> > 
> > > Ideally the ultimate goal is to give BPF schedulers both controls that
> > > NUMA balancing combines: moving a task to the node where its memory is
> > > and moving the memory a task uses to the node where the task runs. This
> > > series provides the first: with the preferred node available, a
> > > scheduler can decide where to run the task. A second series, to follow
> > > once this one lands, will provide the second: a kfunc that lets a
> > > scheduler request the node a task's memory should migrate to.
> > 
> > Fair has no real control over the memory migration part either; why
> > would you want to give BPF that?
> 
> Yeah, that's a fair point and the wording in the cover letter is a bit
> too vague.
> 
> The use case behind the second part is accelerator/device locality,
> discussed at LPC in the sched_ext MC [1]: with multi-GPU systems the
> heaviest user of a task's host memory is often the device, which NUMA
> balancing can't see, since hinting faults only track CPU accesses. So
> memory follows the CPU even when it should stay close to the device, and
> today the only option is to pin everything with numactl and/or disable
> NUMA balancing.
> 
> But that needs to be discussed separately: as Gregory pointed out at
> LPC, NUMA balancing needs some cleanups first, and the right interface
> may not be BPF-specific at all (e.g., extending the home node concept of
> set_mempolicy_home_node() to NUMA balancing, or a device-aware hint in
> mm that both fair and ext could use, or something else).
> 
> The first step for now is to, at least, allow sched_ext to see the
> preferred node computed by NUMA balancing, which is what this series
> does.

Right, I think that came up at OSPM as well; regular numa balancing
would want this as well. This really should not be a bpf thing.