From: Peter Zijlstra <peterz@infradead.org>
To: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Chen Yu <yu.c.chen@intel.com>,
mingo@kernel.org, juri.lelli@redhat.com,
dietmar.eggemann@arm.com, rostedt@goodmis.org,
bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
kprateek.nayak@amd.com, tj@kernel.org,
linux-kernel@vger.kernel.org, chen.yu@linux.dev,
tim.c.chen@linux.intel.com
Subject: Re: [PATCH 4/4] sched/fair: Rework/fix task_h_load()
Date: Wed, 2 Sep 2026 10:13:01 +0200 [thread overview]
Message-ID: <20260902081301.GS4120091@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <CAKfTPtBNAGQUQZbaDKYU7+Nsrcnj4M4p=54XkeML7rrLy+x+ZQ@mail.gmail.com>
On Wed, Sep 02, 2026 at 09:55:49AM +0200, Vincent Guittot wrote:
> On Wed, 2 Sept 2026 at 07:39, Chen Yu <yu.c.chen@intel.com> wrote:
> >
> > On Fri, Aug 28, 2026 at 09:41:03AM +0200, Peter Zijlstra wrote:
> >
> > [ ... ]
> >
> > > @@ -15230,6 +15255,9 @@ static void set_next_task_fair(struct rq
> > > weight = __calc_prop_weight(cfs_rq, se, weight);
> > > }
> > >
> > > + for_each_sched_entity_bl(se, cfs_rq)
> > > + update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
> > > +
> >
> > It panics during bootup on a 192 Cores system,
> > [ 9.445362][ T1687] Oops: general protection fault, kernel NULL pointer dereference 0x69: 0000 [#1] SMP NOPTI
> > [ 18.987940][ T1687] CPU: 58 UID: 0 PID: 1687 Comm: systemd-udevd Not tainted 7.3.0-rc1-flat-hload+ #9 PREEMPTLAZY
> > [ 19.022374][ T1687] RIP: 0010:pick_task_fair+0x43/0xd0
> > [ 19.175316][ T1687] <TASK>
> > [ 19.181239][ T1687] __pick_next_task+0x49/0x1b0
> > [ 19.189229][ T1687] __schedule+0x14b/0x6b0
> > [ 19.196711][ T1687] preempt_schedule+0x3a/0x60
> > [ 19.204543][ T1687] preempt_schedule_thunk+0x16/0x40
> > [ 19.212931][ T1687] _raw_spin_unlock_irqrestore+0x2b/0x30
> > [ 19.221833][ T1687] autogroup_move_group+0xc5/0x160
> > [ 19.230124][ T1687] sched_autogroup_create_attach+0xa7/0x180
> > [ 19.239280][ T1687] ksys_setsid+0x12c/0x170
> > [ 19.246753][ T1687] __do_sys_setsid+0xe/0x20
> > [ 19.254292][ T1687] do_syscall_64+0xbc/0x470
> > [ 19.307858][ T1687] entry_SYSCALL_64_after_hwframe+0x76/0x7e
> > [ 19.316405][ T1687] RIP: 0033:0x7f347bfbcb9b
> >
> > It seems that the crash is a read of se->sched_delayed on a NULL se inside
> > pick_next_entity(), so pick_eevdf() return NULL
>
> I faced the same crash while testing
Weirdly that crash didn't show up for me :-(, I had a few others that I
cured.
> >
> > After the following top->down backlink traverse,
> > for_each_sched_entity_bl(se, cfs_rq)
> > update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
> >
> > cfs_rq is not the root->cfs_rq anymore, but a middle cfs_rq(autogroup
> > in above example). Meanwhile rq->cfs.curr remains NULL because the
> > rq->cfs.curr has been dequeued if the task is runnable and queued:
> > if (on_rq)
> > __dequeue_entity(cfs_rq, se)
> >
> > se = &p->se;
> > cfs_rq->curr = se; /*wrong cfs_rq*/
> >
> > Then later _raw_spin_unlock_irqrestore triggers the scheduling
> > it picks from rq->cfs_rq.curr with a NULL tree.
> >
> > Maybe we need to restore the rq->cfs_rq after the
> > for_each_sched_entity_bl()?
> > se = &p->se;
> > cfs_rq = &rq->cfs; <--
>
> Yes, this fixes it for me too
So I had this issue in task_tick_fair(), where
for_each_sched_entity_bl() clobbered cfs_rq, and fixed that by moving
things after reweight_eevdf() (which is what uses cfs_rq).
I at point I did actually look to see if anybody else would suffer that
same problem, but clearly I missed one.
I'm thinking the problem here is set_next_task_fair() ? I think I
misread the:
se = &p->se;
cfs_rq->curr = se;
to reset both se and cfs_rq, but clearly it doesn't. The below should
fix I suppose. Let me go and try and reproduce.
---
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15287,9 +15287,6 @@ static void set_next_task_fair(struct rq
weight = __calc_prop_weight(cfs_rq, se, weight);
}
- for_each_sched_entity_bl(se, cfs_rq)
- update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
-
if (throttled)
task_throttle_setup_work(p);
@@ -15302,6 +15299,9 @@ static void set_next_task_fair(struct rq
set_protect_slice(cfs_rq, se);
}
+ for_each_sched_entity_bl(se, cfs_rq)
+ update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
+
if (task_on_rq_queued(p)) {
/*
* Move the next running task to the front of the list, so our
next prev parent reply other threads:[~2026-09-02 8:13 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-28 7:40 [PATCH 0/4] sched/fair: Rework tash_h_load() Peter Zijlstra
2026-08-28 7:41 ` [PATCH 1/4] sched: Rename/clarify sched_class::task_tick(.queued) argument Peter Zijlstra
2026-08-28 7:41 ` [PATCH 2/4] sched/fair: Fold cfs_rq_of(se) into for_each_sched_entity() Peter Zijlstra
2026-08-28 7:41 ` [PATCH 3/4] sched/fair: Extend for_each_sched_entity() with a back-link Peter Zijlstra
2026-08-28 7:41 ` [PATCH 4/4] sched/fair: Rework/fix task_h_load() Peter Zijlstra
2026-08-31 10:10 ` Vincent Guittot
2026-08-31 10:37 ` Peter Zijlstra
2026-08-31 12:06 ` Vincent Guittot
2026-08-31 13:07 ` Peter Zijlstra
2026-09-02 5:26 ` Chen Yu
2026-09-02 7:55 ` Vincent Guittot
2026-09-02 8:13 ` Peter Zijlstra [this message]
2026-09-02 10:36 ` Peter Zijlstra
2026-09-02 10:39 ` Vincent Guittot
2026-09-02 10:37 ` Vincent Guittot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260902081301.GS4120091@noisy.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bsegall@google.com \
--cc=chen.yu@linux.dev \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tim.c.chen@linux.intel.com \
--cc=tj@kernel.org \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®