mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Peter Zijlstra <peterz@infradead.org>
To: Vincent Guittot <vincent.guittot@linaro.org>
Cc: Chen Yu <yu.c.chen@intel.com>,
	mingo@kernel.org, juri.lelli@redhat.com,
	dietmar.eggemann@arm.com, rostedt@goodmis.org,
	bsegall@google.com, mgorman@suse.de, vschneid@redhat.com,
	kprateek.nayak@amd.com, tj@kernel.org,
	linux-kernel@vger.kernel.org, chen.yu@linux.dev,
	tim.c.chen@linux.intel.com
Subject: Re: [PATCH 4/4] sched/fair: Rework/fix task_h_load()
Date: Wed, 2 Sep 2026 10:13:01 +0200	[thread overview]
Message-ID: <20260902081301.GS4120091@noisy.programming.kicks-ass.net> (raw)
In-Reply-To: <CAKfTPtBNAGQUQZbaDKYU7+Nsrcnj4M4p=54XkeML7rrLy+x+ZQ@mail.gmail.com>

On Wed, Sep 02, 2026 at 09:55:49AM +0200, Vincent Guittot wrote:
> On Wed, 2 Sept 2026 at 07:39, Chen Yu <yu.c.chen@intel.com> wrote:
> >
> > On Fri, Aug 28, 2026 at 09:41:03AM +0200, Peter Zijlstra wrote:
> >
> > [ ... ]
> >
> > > @@ -15230,6 +15255,9 @@ static void set_next_task_fair(struct rq
> > >                       weight = __calc_prop_weight(cfs_rq, se, weight);
> > >       }
> > >
> > > +     for_each_sched_entity_bl(se, cfs_rq)
> > > +             update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
> > > +
> >
> > It panics during bootup on a 192 Cores system,
> > [    9.445362][ T1687] Oops: general protection fault, kernel NULL pointer dereference 0x69: 0000 [#1] SMP NOPTI
> > [   18.987940][ T1687] CPU: 58 UID: 0 PID: 1687 Comm: systemd-udevd Not tainted 7.3.0-rc1-flat-hload+ #9 PREEMPTLAZY
> > [   19.022374][ T1687] RIP: 0010:pick_task_fair+0x43/0xd0
> > [   19.175316][ T1687]  <TASK>
> > [   19.181239][ T1687]  __pick_next_task+0x49/0x1b0
> > [   19.189229][ T1687]  __schedule+0x14b/0x6b0
> > [   19.196711][ T1687]  preempt_schedule+0x3a/0x60
> > [   19.204543][ T1687]  preempt_schedule_thunk+0x16/0x40
> > [   19.212931][ T1687]  _raw_spin_unlock_irqrestore+0x2b/0x30
> > [   19.221833][ T1687]  autogroup_move_group+0xc5/0x160
> > [   19.230124][ T1687]  sched_autogroup_create_attach+0xa7/0x180
> > [   19.239280][ T1687]  ksys_setsid+0x12c/0x170
> > [   19.246753][ T1687]  __do_sys_setsid+0xe/0x20
> > [   19.254292][ T1687]  do_syscall_64+0xbc/0x470
> > [   19.307858][ T1687]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> > [   19.316405][ T1687] RIP: 0033:0x7f347bfbcb9b
> >
> > It seems that the crash is a read of se->sched_delayed on a NULL se inside
> > pick_next_entity(), so pick_eevdf() return NULL
> 
> I faced the same crash while testing

Weirdly that crash didn't show up for me :-(, I had a few others that I
cured.

> >
> > After the following top->down backlink traverse,
> > for_each_sched_entity_bl(se, cfs_rq)
> >     update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
> >
> > cfs_rq is not the root->cfs_rq anymore, but a middle cfs_rq(autogroup
> > in above example). Meanwhile rq->cfs.curr remains NULL because the
> > rq->cfs.curr has been dequeued if the task is runnable and queued:
> > if (on_rq)
> >         __dequeue_entity(cfs_rq, se)
> >
> > se = &p->se;
> > cfs_rq->curr = se; /*wrong cfs_rq*/
> >
> > Then later _raw_spin_unlock_irqrestore triggers the scheduling
> > it picks from rq->cfs_rq.curr with a NULL tree.
> >
> > Maybe we need to restore the rq->cfs_rq after the
> > for_each_sched_entity_bl()?
> > se = &p->se;
> > cfs_rq = &rq->cfs; <--
> 
> Yes, this fixes it for me too

So I had this issue in task_tick_fair(), where
for_each_sched_entity_bl() clobbered cfs_rq, and fixed that by moving
things after reweight_eevdf() (which is what uses cfs_rq).

I at point I did actually look to see if anybody else would suffer that
same problem, but clearly I missed one.

I'm thinking the problem here is set_next_task_fair() ? I think I
misread the:

	se = &p->se;
	cfs_rq->curr = se;

to reset both se and cfs_rq, but clearly it doesn't. The below should
fix I suppose. Let me go and try and reproduce.


---
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -15287,9 +15287,6 @@ static void set_next_task_fair(struct rq
 			weight = __calc_prop_weight(cfs_rq, se, weight);
 	}
 
-	for_each_sched_entity_bl(se, cfs_rq)
-		update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
-
 	if (throttled)
 		task_throttle_setup_work(p);
 
@@ -15302,6 +15299,9 @@ static void set_next_task_fair(struct rq
 			set_protect_slice(cfs_rq, se);
 	}
 
+	for_each_sched_entity_bl(se, cfs_rq)
+		update_cfs_rq_h_load(group_cfs_rq(se), se, cfs_rq);
+
 	if (task_on_rq_queued(p)) {
 		/*
 		 * Move the next running task to the front of the list, so our

  reply	other threads:[~2026-09-02  8:13 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-28  7:40 [PATCH 0/4] sched/fair: Rework tash_h_load() Peter Zijlstra
2026-08-28  7:41 ` [PATCH 1/4] sched: Rename/clarify sched_class::task_tick(.queued) argument Peter Zijlstra
2026-08-28  7:41 ` [PATCH 2/4] sched/fair: Fold cfs_rq_of(se) into for_each_sched_entity() Peter Zijlstra
2026-08-28  7:41 ` [PATCH 3/4] sched/fair: Extend for_each_sched_entity() with a back-link Peter Zijlstra
2026-08-28  7:41 ` [PATCH 4/4] sched/fair: Rework/fix task_h_load() Peter Zijlstra
2026-08-31 10:10   ` Vincent Guittot
2026-08-31 10:37     ` Peter Zijlstra
2026-08-31 12:06       ` Vincent Guittot
2026-08-31 13:07         ` Peter Zijlstra
2026-09-02  5:26   ` Chen Yu
2026-09-02  7:55     ` Vincent Guittot
2026-09-02  8:13       ` Peter Zijlstra [this message]
2026-09-02 10:36         ` Peter Zijlstra
2026-09-02 10:39           ` Vincent Guittot
2026-09-02 10:37         ` Vincent Guittot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260902081301.GS4120091@noisy.programming.kicks-ass.net \
    --to=peterz@infradead.org \
    --cc=bsegall@google.com \
    --cc=chen.yu@linux.dev \
    --cc=dietmar.eggemann@arm.com \
    --cc=juri.lelli@redhat.com \
    --cc=kprateek.nayak@amd.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tim.c.chen@linux.intel.com \
    --cc=tj@kernel.org \
    --cc=vincent.guittot@linaro.org \
    --cc=vschneid@redhat.com \
    --cc=yu.c.chen@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®