From: Peng Wang <peng_wang@linux.alibaba.com>
To: Vincent Guittot <vincent.guittot@linaro.org>
Cc: bsegall@google.com, dietmar.eggemann@arm.com,
juri.lelli@redhat.com, linux-kernel@vger.kernel.org,
mgorman@suse.de, mingo@redhat.com, peterz@infradead.org,
rostedt@goodmis.org, stable@vger.kernel.org,
vdavydov.dev@gmail.com, vschneid@redhat.com
Subject: Re: [PATCH v3] sched/fair: Clear ->h_load_next when unregistering cgroup
Date: Wed, 3 Dec 2025 16:17:15 +0800 [thread overview]
Message-ID: <aS_yC_smGTS-_JbK@U-N9MN20RF-1935.local> (raw)
In-Reply-To: <CAKfTPtBMVkCVCSMN2ztpA=kje-BoNFt=bK44_zdkMeAPfNMdqA@mail.gmail.com>
On Fri, Oct 24, 2025 at 09:52:59AM +0200, Vincent Guittot wrote:
> On Fri, 24 Oct 2025 at 09:24, Peng Wang <peng_wang@linux.alibaba.com> wrote:
> >
> > An invalid pointer dereference bug was reported on arm64 cpu, and has
> > not yet been seen on x86. A partial oops looks like:
> >
> > Call trace:
> > update_cfs_rq_h_load+0x80/0xb0
> > wake_affine+0x158/0x168
> > select_task_rq_fair+0x364/0x3a8
> > try_to_wake_up+0x154/0x648
> > wake_up_q+0x68/0xd0
> > futex_wake_op+0x280/0x4c8
> > do_futex+0x198/0x1c0
> > __arm64_sys_futex+0x11c/0x198
> >
> > Link: https://lore.kernel.org/all/20251013071820.1531295-1-CruzZhao@linux.alibaba.com/
> >
> > We found that the task_group corresponding to the problematic se
> > is not in the parent task_group’s children list, indicating that
> > h_load_next points to an invalid address. Consider the following
> > cgroup and task hierarchy:
> >
> > A
> > / \
> > / \
> > B E
> > / \ |
> > / \ t2
> > C D
> > | |
> > t0 t1
> >
> > Here follows a timing sequence that may be responsible for triggering
> > the problem:
> >
> > CPU X CPU Y CPU Z
> > wakeup t0
> > set list A->B->C
> > traverse A->B->C
> > t0 exits
> > destroy C
> > wakeup t2
> > set list A->E wakeup t1
> > set list A->B->D
> > traverse A->B->C
> > panic
> >
> > CPU Z sets ->h_load_next list to A->B->D, but due to arm64 weaker memory
> > ordering, Y may observe A->B before it sees B->D, then in this time window,
> > it can traverse A->B->C and reach an invalid se.
> >
> > We can avoid stale pointer accesses by clearing ->h_load_next when
> > unregistering cgroup.
> >
> > Suggested-by: Vincent Guittot <vincent.guittot@linaro.org>
> > Fixes: 685207963be9 ("sched: Move h_load calculation to task_h_load()")
> > Cc: <stable@vger.kernel.org>
> > Co-developed-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
> > Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
> > Signed-off-by: Peng Wang <peng_wang@linux.alibaba.com>
>
> Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Gentle Ping
Hi, Peter and Vincent,
after applying this patch, update_cfs_rq_h_load crashing has not occurred for the past two weeks.
The patch has proven to be effective. Would you consider merging it?
>
> > ---
> > kernel/sched/fair.c | 9 +++++++++
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index cee1793e8277..32b466605925 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -13418,6 +13418,8 @@ void unregister_fair_sched_group(struct task_group *tg)
> > struct rq *rq = cpu_rq(cpu);
> >
> > if (se) {
> > + struct cfs_rq *parent_cfs_rq = cfs_rq_of(se);
> > +
> > if (se->sched_delayed) {
> > guard(rq_lock_irqsave)(rq);
> > if (se->sched_delayed) {
> > @@ -13427,6 +13429,13 @@ void unregister_fair_sched_group(struct task_group *tg)
> > list_del_leaf_cfs_rq(cfs_rq);
> > }
> > remove_entity_load_avg(se);
> > +
> > + /*
> > + * Clear parent's h_load_next if it points to the
> > + * sched_entity being freed to avoid stale pointer.
> > + */
> > + if (READ_ONCE(parent_cfs_rq->h_load_next) == se)
> > + WRITE_ONCE(parent_cfs_rq->h_load_next, NULL);
> > }
> >
> > /*
> > --
> > 2.27.0
> >
next prev parent reply other threads:[~2025-12-03 8:17 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-15 12:19 [PATCH] sched/fair: Clear ->h_load_next after hierarchical load Peng Wang
2025-10-15 12:44 ` Peter Zijlstra
2025-10-15 13:14 ` Vincent Guittot
2025-10-15 14:46 ` Peter Zijlstra
2025-10-22 9:00 ` Peng Wang
2025-10-22 12:52 ` Vincent Guittot
2025-10-23 6:28 ` [PATCH v2] sched/fair: Clear ->h_load_next when unregistering cgroup Peng Wang
2025-10-24 7:08 ` Vincent Guittot
2025-10-24 7:23 ` [PATCH v3] " Peng Wang
2025-10-24 7:52 ` Vincent Guittot
2025-12-03 8:17 ` Peng Wang [this message]
2025-10-16 3:06 ` [PATCH] sched/fair: Clear ->h_load_next after hierarchical load Peng Wang
2025-11-17 18:59 ` Krister Johansen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aS_yC_smGTS-_JbK@U-N9MN20RF-1935.local \
--to=peng_wang@linux.alibaba.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=stable@vger.kernel.org \
--cc=vdavydov.dev@gmail.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®