From: Tejun Heo <tj@kernel.org>
To: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
Al Viro <viro@zeniv.linux.org.uk>,
Zefan Li <lizefan.x@bytedance.com>,
Johannes Weiner <hannes@cmpxchg.org>,
stable@vger.kernel.org,
Richard Purdie <richard.purdie@linuxfoundation.org>
Subject: Re: [PATCH] cgroup1: fix leaked context root causing sporadic NULL deref in LTP
Date: Mon, 12 Jul 2021 07:34:18 -1000 [thread overview]
Message-ID: <YOx9GkxAGwAb3Yyr@slm.duckdns.org> (raw)
In-Reply-To: <20210616125157.438837-1-paul.gortmaker@windriver.com>
On Wed, Jun 16, 2021 at 08:51:57AM -0400, Paul Gortmaker wrote:
> Richard reported sporadic (roughly one in 10 or so) null dereferences and
> other strange behaviour for a set of automated LTP tests. Things like:
>
> BUG: kernel NULL pointer dereference, address: 0000000000000008
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> PGD 0 P4D 0
> Oops: 0000 [#1] PREEMPT SMP PTI
> CPU: 0 PID: 1516 Comm: umount Not tainted 5.10.0-yocto-standard #1
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-48-gd9c812dda519-prebuilt.qemu.org 04/01/2014
> RIP: 0010:kernfs_sop_show_path+0x1b/0x60
>
> ...or these others:
>
> RIP: 0010:do_mkdirat+0x6a/0xf0
> RIP: 0010:d_alloc_parallel+0x98/0x510
> RIP: 0010:do_readlinkat+0x86/0x120
>
> There were other less common instances of some kind of a general scribble
> but the common theme was mount and cgroup and a dubious dentry triggering
> the NULL dereference. I was only able to reproduce it under qemu by
> replicating Richard's setup as closely as possible - I never did get it
> to happen on bare metal, even while keeping everything else the same.
>
> In commit 71d883c37e8d ("cgroup_do_mount(): massage calling conventions")
> we see this as a part of the overall change:
>
> --------------
> struct cgroup_subsys *ss;
> - struct dentry *dentry;
>
> [...]
>
> - dentry = cgroup_do_mount(&cgroup_fs_type, fc->sb_flags, root,
> - CGROUP_SUPER_MAGIC, ns);
>
> [...]
>
> - if (percpu_ref_is_dying(&root->cgrp.self.refcnt)) {
> - struct super_block *sb = dentry->d_sb;
> - dput(dentry);
> + ret = cgroup_do_mount(fc, CGROUP_SUPER_MAGIC, ns);
> + if (!ret && percpu_ref_is_dying(&root->cgrp.self.refcnt)) {
> + struct super_block *sb = fc->root->d_sb;
> + dput(fc->root);
> deactivate_locked_super(sb);
> msleep(10);
> return restart_syscall();
> }
> --------------
>
> In changing from the local "*dentry" variable to using fc->root, we now
> export/leave that dentry pointer in the file context after doing the dput()
> in the unlikely "is_dying" case. With LTP doing a crazy amount of back to
> back mount/unmount [testcases/bin/cgroup_regression_5_1.sh] the unlikely
> becomes slightly likely and then bad things happen.
>
> A fix would be to not leave the stale reference in fc->root as follows:
>
> --------------
> dput(fc->root);
> + fc->root = NULL;
> deactivate_locked_super(sb);
> --------------
>
> ...but then we are just open-coding a duplicate of fc_drop_locked() so we
> simply use that instead.
>
> Cc: Al Viro <viro@zeniv.linux.org.uk>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Zefan Li <lizefan.x@bytedance.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: stable@vger.kernel.org # v5.1+
> Reported-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> Fixes: 71d883c37e8d ("cgroup_do_mount(): massage calling conventions")
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
I dropped the ball on this and this didn't get pushed. Re-applied to
for-5.14-fixes. Will send out in a few days.
Thanks.
--
tejun
prev parent reply other threads:[~2021-07-12 17:34 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-16 12:51 Paul Gortmaker
2021-06-16 15:23 ` Tejun Heo
2021-06-30 16:10 ` Mark Brown
2021-06-30 22:31 ` Richard Purdie
2021-07-01 12:11 ` Mark Brown
2021-07-01 12:43 ` Richard Purdie
2021-07-12 17:34 ` Tejun Heo [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YOx9GkxAGwAb3Yyr@slm.duckdns.org \
--to=tj@kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan.x@bytedance.com \
--cc=paul.gortmaker@windriver.com \
--cc=richard.purdie@linuxfoundation.org \
--cc=stable@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®