mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Tejun Heo <tj@kernel.org>
To: Baokun Li <libaokun@linux.alibaba.com>
Cc: linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk,
	brauner@kernel.org, jack@suse.cz, linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Subject: Re: [PATCH v4 1/3] writeback: fix race between cgroup_writeback_umount() and inode_switch_wbs()
Date: Thu, 21 May 2026 07:18:24 -1000	[thread overview]
Message-ID: <ag8-YOCHsfySqcdH@slm.duckdns.org> (raw)
In-Reply-To: <20260521095016.2791354-2-libaokun@linux.alibaba.com>

On Thu, May 21, 2026 at 05:50:14PM +0800, Baokun Li wrote:
> When a container exits, the following BUG_ON() is occasionally triggered:
> 
> ==================================================================
>  VFS: Busy inodes after unmount of sdb (ext4)
>  ------------[ cut here ]------------
>  kernel BUG at fs/super.c:695!
>  CPU: 3 PID: 6 Comm: containerd-shim Tainted: G OE K 6.6 #1
>  pstate: 63400009 (nZCv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--)
>  pc : generic_shutdown_super+0xf0/0x100
>  lr : generic_shutdown_super+0xf0/0x100
>  Call trace:
>   generic_shutdown_super+0xf0/0x100
>   kill_block_super+0x20/0x48
>   ext4_kill_sb+0x28/0x60
>   deactivate_locked_super+0x54/0x130
>   deactivate_super+0x84/0xa0
>   cleanup_mnt+0xa4/0x140
>   __cleanup_mnt+0x18/0x28
>   task_work_run+0x78/0xe0
>   do_notify_resume+0x204/0x240
> ==================================================================
> 
> The root cause is a race between cgroup_writeback_umount() and
> inode_switch_wbs()/cleanup_offline_cgwb(). There is a window between
> inode_prepare_wbs_switch() returning true and the subsequent
> wb_queue_isw() call. Following is the process that triggers the issue:
> 
>       CPU A (umount)           |          CPU B (writeback)
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>                                  inode_switch_wbs/cleanup_offline_cgwb
>                                   atomic_inc(&isw_nr_in_flight)
>                                   inode_prepare_wbs_switch
>                                    -> passes SB_ACTIVE check
>                                    __iget(inode)
>  generic_shutdown_super
>   sb->s_flags &= ~SB_ACTIVE
>   cgroup_writeback_umount(sb)
>    smp_mb()
>    atomic_read(&isw_nr_in_flight)
>    rcu_barrier()
>     -> no pending RCU callbacks
>    flush_workqueue(isw_wq)
>     -> nothing queued, returns
>   evict_inodes(sb)
>    -> Inode skipped as isw still holds a ref.
>   sop->put_super(sb)
>    /* destroys percpu counters */
>   -> VFS: Busy inodes after unmount!
>                                   wb_queue_isw()
>                                    queue_work(isw_wq, ...)
>                                   /* later in work function */
>                                   inode_switch_wbs_work_fn
>                                    process_inode_switch_wbs
>                                     iput() -> evict
>                                      percpu_counter_dec() // UAF!
> 
> Fix this by extending the RCU read-side critical section in
> inode_switch_wbs() and cleanup_offline_cgwb() to cover from
> inode_prepare_wbs_switch() through wb_queue_isw().  Since there is
> no sleep in this window, rcu_read_lock() can be used.  Then add a
> synchronize_rcu() in cgroup_writeback_umount() before the existing
> rcu_barrier(), so that all in-flight switchers that have passed the
> SB_ACTIVE check have completed queue_work() before flush_workqueue()
> is called.
> 
> The existing rcu_barrier() is intentionally retained so this fix can
> be backported unchanged to stable kernels (5.10.y, 6.6.y, ...) that
> still queue switches via queue_rcu_work(). It is a no-op on current
> mainline (since commit e1b849cfa6b6 ("writeback: Avoid contention on
> wb->list_lock when switching inodes")) and is removed in a follow-up
> patch.
> 
> Fixes: a1a0e23e4903 ("writeback: flush inode cgroup wb switches instead of pinning super_block")
> Cc: stable@vger.kernel.org
> Suggested-by: Jan Kara <jack@suse.cz>
> Link: https://lore.kernel.org/all/mxnjq2l6guusfchvauxr3v7c4bwjasybxlleqbbh4efloeqspz@iqylk76ohufz
> Reviewed-by: Jan Kara <jack@suse.cz>
> Signed-off-by: Baokun Li <libaokun@linux.alibaba.com>

Acked-by: Tejun Heo <tj@kernel.org>

Thanks.

-- 
tejun

  reply	other threads:[~2026-05-21 17:18 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-21  9:50 [PATCH v4 0/3] " Baokun Li
2026-05-21  9:50 ` [PATCH v4 1/3] " Baokun Li
2026-05-21 17:18   ` Tejun Heo [this message]
2026-05-21  9:50 ` [PATCH v4 2/3] writeback: drop now-unnecessary rcu_barrier() in cgroup_writeback_umount() Baokun Li
2026-05-21 17:18   ` Tejun Heo
2026-05-21  9:50 ` [PATCH v4 3/3] writeback: use a per-sb counter to drain inode wb switches at umount Baokun Li
2026-05-21 17:40   ` Tejun Heo
2026-05-22 10:07 ` [PATCH v4 0/3] writeback: fix race between cgroup_writeback_umount() and inode_switch_wbs() Christian Brauner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ag8-YOCHsfySqcdH@slm.duckdns.org \
    --to=tj@kernel.org \
    --cc=brauner@kernel.org \
    --cc=jack@suse.cz \
    --cc=libaokun@linux.alibaba.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®