[PATCH] sched_ext: Skip per-CPU data allocation for built-in DSQs

[PATCH] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Qiurong Fang]

From: fangqiurong <fangqiurong@kylinos.cn>

Deferred reenqueue tracking is only supported for user DSQs:
schedule_dsq_reenq() serves the local DSQ through
sch->pcpu->deferred_reenq_local and rejects all other built-in DSQ
ids. Every built-in DSQ still carries nr_cpu_ids * sizeof(struct
scx_dsq_pcpu) bytes of dead per-CPU memory.

Skip the allocation for built-in DSQ ids.

Fixes: 30b0515342db ("sched_ext: Add per-CPU data to DSQs")
Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
---
 kernel/sched/ext/ext.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index c539d15cda63..eaa64b8193e1 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -5048,6 +5048,10 @@ s32 scx_init_dsq(struct scx_dispatch_q *dsq, u64 dsq_id, struct scx_sched *sch)
 	dsq->id = dsq_id;
 	dsq->sched = sch;
 
+	/* Deferred reenqueue tracking is only supported for user DSQs */
+	if (dsq_id & SCX_DSQ_FLAG_BUILTIN)
+		return 0;
+
 	dsq->pcpu = alloc_percpu(struct scx_dsq_pcpu);
 	if (!dsq->pcpu)
 		return -ENOMEM;
@@ -5066,6 +5070,9 @@ static void exit_dsq(struct scx_dispatch_q *dsq)
 {
 	s32 cpu;
 
+	if (!dsq->pcpu)
+		return;
+
 	for_each_possible_cpu(cpu) {
 		struct scx_dsq_pcpu *pcpu = per_cpu_ptr(dsq->pcpu, cpu);
 		struct scx_deferred_reenq_user *dru = &pcpu->deferred_reenq_user;
-- 
2.43.0

Re: [PATCH] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Zhan Xusheng]

From: Zhan Xusheng <zhanxusheng@xiaomi.com>

On Thu, 27 Aug 2026 16:23:29 +0800, Qiurong Fang wrote:
> Every built-in DSQ still carries nr_cpu_ids * sizeof(struct
> scx_dsq_pcpu) bytes of dead per-CPU memory.

It is quadratic rather than linear, which is worth saying.  The number of
built-in DSQs scales with nr_cpu_ids too, because most of them are
initialised inside for_each_possible_cpu():

  ext.c:8681  SCX_DSQ_LOCAL
  ext.c:8683  SCX_DSQ_REJECT   (CONFIG_EXT_SUB_SCHED)
  ext.c:7172  SCX_DSQ_BYPASS
  sub.c:672   SCX_DSQ_RESCUE   (per rq, via scx_rescue_init())

Only SCX_DSQ_GLOBAL is a single instance.  Each of those then allocates
nr_cpu_ids entries, so what you drop is on the order of nr_cpu_ids^2
struct scx_dsq_pcpu, which on a large machine is a very different number
from the one the changelog suggests.

The guard holds.  ext.c:1130 is the only read of dsq->pcpu, and it already
sits under !(dsq->id & SCX_DSQ_FLAG_BUILTIN) at 1127, the complement of
what you skip on; the local DSQ branch above uses sch->pcpu instead.  All
five ids passed to scx_init_dsq() carry the flag (sched/ext.h:58-62), so
nothing is missed either.  exit_dsq() contains nothing but the pcpu walk
and the free, so the early return skips nothing else, and it is needed
rather than tidy: per_cpu_ptr(NULL, cpu) hands the loop an offset pointer
and list_empty() would read it, so relying on free_percpu(NULL) alone
would not do.

Two smaller things.  A Fixes: tag routes this to stable, which is more
than a memory saving asks for unless you see a correctness angle.  And the
mail header says "Qiurong Fang" while the From: and Signed-off-by in the
body say "fangqiurong".

Thanks,
Zhan Xusheng

Re: [PATCH] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Tejun Heo]

Hello,

On Thu, Aug 27, 2026 at 04:23:29PM +0800, Qiurong Fang wrote:
> From: fangqiurong <fangqiurong@kylinos.cn>
> 
> Deferred reenqueue tracking is only supported for user DSQs:
> schedule_dsq_reenq() serves the local DSQ through
> sch->pcpu->deferred_reenq_local and rejects all other built-in DSQ
> ids. Every built-in DSQ still carries nr_cpu_ids * sizeof(struct
> scx_dsq_pcpu) bytes of dead per-CPU memory.
> 
> Skip the allocation for built-in DSQ ids.

Let's rename it to dsq->pcpu_user while at it.

Thanks.

-- 
tejun

[PATCH v2] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Qiurong Fang]

From: fangqiurong <fangqiurong@kylinos.cn>

Deferred reenqueue tracking is only supported for user DSQs:
schedule_dsq_reenq() serves the local DSQ through
sch->pcpu->deferred_reenq_local and rejects all other built-in DSQ
ids. Every DSQ still allocates nr_cpu_ids * sizeof(struct scx_dsq_pcpu)
bytes of per-CPU memory for it.

Skip the allocation for built-in DSQ ids. While at it, rename ->pcpu to
->pcpu_user so that the per-CPU data reads as user-DSQ only.

Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
---
v2: Rename ->pcpu to ->pcpu_user as suggested by Tejun. Describe the
    memory waste as quadratic as pointed out by Zhan Xusheng. Drop the
    Fixes: tag as this is a plain memory saving with no correctness
    impact.
---
 include/linux/sched/ext.h |  2 +-
 kernel/sched/ext/ext.c    | 20 ++++++++++++++------
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h
index 582d7cd4a983..3c793d51c000 100644
--- a/include/linux/sched/ext.h
+++ b/include/linux/sched/ext.h
@@ -91,7 +91,7 @@ struct scx_dispatch_q {
 	struct rhash_head	hash_node;
 	struct llist_node	free_node;
 	struct scx_sched	*sched;
-	struct scx_dsq_pcpu __percpu *pcpu;
+	struct scx_dsq_pcpu __percpu *pcpu_user;
 	struct rcu_head		rcu;
 };
 
diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index c539d15cda63..efb492876cd6 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -1127,7 +1127,8 @@ void schedule_dsq_reenq(struct scx_sched *sch, struct scx_dispatch_q *dsq,
 	} else if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN)) {
 		rq = this_rq();
 
-		struct scx_dsq_pcpu *dsq_pcpu = per_cpu_ptr(dsq->pcpu, cpu_of(rq));
+		struct scx_dsq_pcpu *dsq_pcpu =
+			per_cpu_ptr(dsq->pcpu_user, cpu_of(rq));
 		struct scx_deferred_reenq_user *dru = &dsq_pcpu->deferred_reenq_user;
 
 		/*
@@ -5048,12 +5049,16 @@ s32 scx_init_dsq(struct scx_dispatch_q *dsq, u64 dsq_id, struct scx_sched *sch)
 	dsq->id = dsq_id;
 	dsq->sched = sch;
 
-	dsq->pcpu = alloc_percpu(struct scx_dsq_pcpu);
-	if (!dsq->pcpu)
+	/* Deferred reenqueue tracking is only supported for user DSQs */
+	if (dsq_id & SCX_DSQ_FLAG_BUILTIN)
+		return 0;
+
+	dsq->pcpu_user = alloc_percpu(struct scx_dsq_pcpu);
+	if (!dsq->pcpu_user)
 		return -ENOMEM;
 
 	for_each_possible_cpu(cpu) {
-		struct scx_dsq_pcpu *pcpu = per_cpu_ptr(dsq->pcpu, cpu);
+		struct scx_dsq_pcpu *pcpu = per_cpu_ptr(dsq->pcpu_user, cpu);
 
 		pcpu->dsq = dsq;
 		INIT_LIST_HEAD(&pcpu->deferred_reenq_user.node);
@@ -5066,8 +5071,11 @@ static void exit_dsq(struct scx_dispatch_q *dsq)
 {
 	s32 cpu;
 
+	if (!dsq->pcpu_user)
+		return;
+
 	for_each_possible_cpu(cpu) {
-		struct scx_dsq_pcpu *pcpu = per_cpu_ptr(dsq->pcpu, cpu);
+		struct scx_dsq_pcpu *pcpu = per_cpu_ptr(dsq->pcpu_user, cpu);
 		struct scx_deferred_reenq_user *dru = &pcpu->deferred_reenq_user;
 		struct rq *rq = cpu_rq(cpu);
 
@@ -5081,7 +5089,7 @@ static void exit_dsq(struct scx_dispatch_q *dsq)
 		}
 	}
 
-	free_percpu(dsq->pcpu);
+	free_percpu(dsq->pcpu_user);
 }
 
 static void free_dsq_rcufn(struct rcu_head *rcu)
-- 
2.43.0

Re: [PATCH v2] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Zhan Xusheng]

The v2 note says the waste is described as quadratic, but the changelog
still gives it per DSQ:

> Every DSQ still allocates nr_cpu_ids * sizeof(struct scx_dsq_pcpu)
> bytes of per-CPU memory for it.

Four of the five built-in ids come one per CPU: SCX_DSQ_BYPASS in the
for_each_possible_cpu() at ext.c:7173, SCX_DSQ_LOCAL and SCX_DSQ_REJECT in
the one at 8679, SCX_DSQ_RESCUE per rq in scx_rescue_init().  Only
SCX_DSQ_GLOBAL is per node.  So the total is nr_cpu_ids squared, and a
reader of the changelog as it stands will work out a factor of nr_cpu_ids.
Worth spelling out, since that is where the saving is.

The code is right.  scx_init_dsq() memsets the whole dsq first
(ext.c:5046) and sets lock, list, id and sched before the new early
return, so nothing a built-in id needs is skipped, and exit_dsq()'s
->pcpu_user test is the load-bearing kind: 7101, 5367 and 7334 all call it
with built-in DSQs.  The only reader of the per-CPU data, ext.c:1130, was
already behind the !(dsq->id & SCX_DSQ_FLAG_BUILTIN) test above it.

Reviewed-by: Zhan Xusheng <zhanxusheng@xiaomi.com>

Thanks,
Zhan Xusheng

Re: [PATCH v2] sched_ext: Skip per-CPU data allocation for built-in DSQs

[Tejun Heo]

Applied to sched_ext/for-7.4 with Zhan's Reviewed-by added, the description
reworded to say the waste is quadratic in the CPU count, and the
scx_init_dsq() comment reworded as the local DSQ does track deferred reenqs,
just through sch->pcpu.

Thanks.

--
tejun