* [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted @ 2017-03-20 22:58 Ming Lin 2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin 0 siblings, 1 reply; 6+ messages in thread From: Ming Lin @ 2017-03-20 22:58 UTC (permalink / raw) To: nbd-general, Josef Bacik, Ratna Manoj Bolla Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang, james.liu, Markus Pargmann Hi all, I run into a BUG_ON(!buffer_mapped(bh)) crash with below script. $ rbd-nbd map mypool/myimg $ mkfs.ext4 /dev/nbd0 $ mount /dev/nbd0 /mnt/ $ rbd-nbd unmap /dev/nbd0 $ umount /mnt [ 1248.870131] kernel BUG at /home/mlin/linux/fs/buffer.c:3103! [ 1248.871214] invalid opcode: 0000 [#1] SMP [ 1248.879468] CPU: 0 PID: 2450 Comm: umount Tainted: G E 4.11.0-rc2+ #2 [ 1248.896579] Call Trace: [ 1248.897056] __sync_dirty_buffer+0x6e/0xe0 [ 1248.897870] ext4_commit_super+0x1eb/0x290 [ext4] [ 1248.898795] ext4_put_super+0x2fa/0x3c0 [ext4] [ 1248.899662] generic_shutdown_super+0x6f/0x100 [ 1248.900525] kill_block_super+0x27/0x70 [ 1248.901257] deactivate_locked_super+0x43/0x70 [ 1248.902112] deactivate_super+0x46/0x60 [ 1248.902869] cleanup_mnt+0x3f/0x80 [ 1248.903526] __cleanup_mnt+0x12/0x20 [ 1248.904218] task_work_run+0x83/0xb0 [ 1248.904941] exit_to_usermode_loop+0x59/0x7b [ 1248.905769] do_syscall_64+0x165/0x180 [ 1248.907603] entry_SYSCALL64_slow_path+0x25/0x25 Last year, Ratna posted a patch to fix it. https://lkml.org/lkml/2016/4/20/257 Ratna's script to reproduce the bug. $ qemu-img create -f qcow2 f.img 1G $ mkfs.ext4 f.img $ qemu-nbd -c /dev/nbd0 f.img $ mount /dev/nbd0 dir $ killall -KILL qemu-nbd $ sleep 1 $ ls dir $ umount dir I ported Rantna's patch to 4.11-rc2 and confirmed that it fixes the crash. Jan Kara had some comments about this bug: http://www.kernelhub.org/?p=2&msg=361407 I hope to fix this bug in the upstream kernel first and then back port it to our production system. Please see "PATCH 1/1" for detail. Thanks, Ming ^ permalink raw reply [flat|nested] 6+ messages in thread
* [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() 2017-03-20 22:58 [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted Ming Lin @ 2017-03-20 22:58 ` Ming Lin 2017-03-22 20:48 ` [Nbd] " Ming Lin 2017-03-23 17:51 ` Josef Bacik 0 siblings, 2 replies; 6+ messages in thread From: Ming Lin @ 2017-03-20 22:58 UTC (permalink / raw) To: nbd-general, Josef Bacik, Ratna Manoj Bolla Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang, james.liu, Markus Pargmann From: Ratna Manoj Bolla <manoj.br@gmail.com> When a filesystem is mounted on a nbd device and on a disconnect, because of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are getting destroyed under mounted filesystem. After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect, followed by a sys_umount(), generic_shutdown_super()->... ->__sync_blockdev()->... -blkdev_writepages()->... ->do_invalidatepage()->... -discard_buffer() is discarding superblock buffer_head assumed to be in mapped state by ext4_commit_super(). [mlin: ported to 4.11-rc2] Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com --- drivers/block/nbd.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index cb4ccfc..a6a3643 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd) static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev) { - bd_set_size(bdev, 0); + if (bdev->bd_openers <= 1) + bd_set_size(bdev, 0); set_capacity(nbd->disk, 0); kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE); @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd) static void nbd_bdev_reset(struct block_device *bdev) { + if (bdev->bd_openers > 1) + return; set_device_ro(bdev, false); bdev->bd_inode->i_size = 0; if (max_part > 0) { @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev) { sock_shutdown(nbd); nbd_clear_que(nbd); - kill_bdev(bdev); + + __invalidate_device(bdev, true); nbd_bdev_reset(bdev); /* * We want to give the run thread a chance to wait for everybody -- 1.8.3.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Nbd] [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() 2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin @ 2017-03-22 20:48 ` Ming Lin 2017-03-22 21:43 ` Josef Bacik 2017-03-23 17:51 ` Josef Bacik 1 sibling, 1 reply; 6+ messages in thread From: Ming Lin @ 2017-03-22 20:48 UTC (permalink / raw) To: Josef Bacik Cc: nbd-general, Ratna Manoj Bolla, lkml, linux-block, jianshu.ljs, LIU, Fei, Markus Pargmann, xiongwei.jiang On Mon, Mar 20, 2017 at 3:58 PM, Ming Lin <mlin@kernel.org> wrote: > From: Ratna Manoj Bolla <manoj.br@gmail.com> > > When a filesystem is mounted on a nbd device and on a disconnect, because > of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are > getting destroyed under mounted filesystem. > > After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect, > followed by a sys_umount(), > generic_shutdown_super()->... > ->__sync_blockdev()->... > -blkdev_writepages()->... > ->do_invalidatepage()->... > -discard_buffer() is discarding superblock buffer_head assumed > to be in mapped state by ext4_commit_super(). > > [mlin: ported to 4.11-rc2] > Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com > --- > drivers/block/nbd.c | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c > index cb4ccfc..a6a3643 100644 > --- a/drivers/block/nbd.c > +++ b/drivers/block/nbd.c > @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd) > > static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev) > { > - bd_set_size(bdev, 0); > + if (bdev->bd_openers <= 1) > + bd_set_size(bdev, 0); > set_capacity(nbd->disk, 0); > kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE); > > @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd) > > static void nbd_bdev_reset(struct block_device *bdev) > { > + if (bdev->bd_openers > 1) > + return; > set_device_ro(bdev, false); > bdev->bd_inode->i_size = 0; > if (max_part > 0) { > @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev) > { > sock_shutdown(nbd); > nbd_clear_que(nbd); > - kill_bdev(bdev); > + > + __invalidate_device(bdev, true); > nbd_bdev_reset(bdev); > /* > * We want to give the run thread a chance to wait for everybody > -- > 1.8.3.1 Hi Josef, Any comments? Thanks, Ming ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Nbd] [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() 2017-03-22 20:48 ` [Nbd] " Ming Lin @ 2017-03-22 21:43 ` Josef Bacik 0 siblings, 0 replies; 6+ messages in thread From: Josef Bacik @ 2017-03-22 21:43 UTC (permalink / raw) To: Ming Lin Cc: nbd-general, Ratna Manoj Bolla, lkml, linux-block, jianshu.ljs, LIU, Fei, Markus Pargmann, xiongwei.jiang Hey sorry I just got back from LSF, I’ll look at this in the morning. Thanks, Josef On 3/22/17, 4:48 PM, "Ming Lin" <mlin@kernel.org> wrote: On Mon, Mar 20, 2017 at 3:58 PM, Ming Lin <mlin@kernel.org> wrote: > From: Ratna Manoj Bolla <manoj.br@gmail.com> > > When a filesystem is mounted on a nbd device and on a disconnect, because > of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are > getting destroyed under mounted filesystem. > > After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect, > followed by a sys_umount(), > generic_shutdown_super()->... > ->__sync_blockdev()->... > -blkdev_writepages()->... > ->do_invalidatepage()->... > -discard_buffer() is discarding superblock buffer_head assumed > to be in mapped state by ext4_commit_super(). > > [mlin: ported to 4.11-rc2] > Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com > --- > drivers/block/nbd.c | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c > index cb4ccfc..a6a3643 100644 > --- a/drivers/block/nbd.c > +++ b/drivers/block/nbd.c > @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd) > > static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev) > { > - bd_set_size(bdev, 0); > + if (bdev->bd_openers <= 1) > + bd_set_size(bdev, 0); > set_capacity(nbd->disk, 0); > kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE); > > @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd) > > static void nbd_bdev_reset(struct block_device *bdev) > { > + if (bdev->bd_openers > 1) > + return; > set_device_ro(bdev, false); > bdev->bd_inode->i_size = 0; > if (max_part > 0) { > @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev) > { > sock_shutdown(nbd); > nbd_clear_que(nbd); > - kill_bdev(bdev); > + > + __invalidate_device(bdev, true); > nbd_bdev_reset(bdev); > /* > * We want to give the run thread a chance to wait for everybody > -- > 1.8.3.1 Hi Josef, Any comments? Thanks, Ming ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() 2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin 2017-03-22 20:48 ` [Nbd] " Ming Lin @ 2017-03-23 17:51 ` Josef Bacik 2017-03-23 20:58 ` Ming Lin 1 sibling, 1 reply; 6+ messages in thread From: Josef Bacik @ 2017-03-23 17:51 UTC (permalink / raw) To: Ming Lin, nbd-general, Ratna Manoj Bolla Cc: linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang, james.liu Yeah I think this is ok, I’ll throw it on my queue for fixes for this cycle. Thanks, Josef On 3/20/17, 6:58 PM, "Ming Lin" <mlin@kernel.org> wrote: From: Ratna Manoj Bolla <manoj.br@gmail.com> When a filesystem is mounted on a nbd device and on a disconnect, because of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are getting destroyed under mounted filesystem. After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect, followed by a sys_umount(), generic_shutdown_super()->... ->__sync_blockdev()->... -blkdev_writepages()->... ->do_invalidatepage()->... -discard_buffer() is discarding superblock buffer_head assumed to be in mapped state by ext4_commit_super(). [mlin: ported to 4.11-rc2] Signed-off-by: Ratna Manoj Bolla <manoj.br@gmail.com --- drivers/block/nbd.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index cb4ccfc..a6a3643 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -125,7 +125,8 @@ static const char *nbdcmd_to_ascii(int cmd) static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev) { - bd_set_size(bdev, 0); + if (bdev->bd_openers <= 1) + bd_set_size(bdev, 0); set_capacity(nbd->disk, 0); kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE); @@ -603,6 +604,8 @@ static void nbd_reset(struct nbd_device *nbd) static void nbd_bdev_reset(struct block_device *bdev) { + if (bdev->bd_openers > 1) + return; set_device_ro(bdev, false); bdev->bd_inode->i_size = 0; if (max_part > 0) { @@ -666,7 +669,8 @@ static int nbd_clear_sock(struct nbd_device *nbd, struct block_device *bdev) { sock_shutdown(nbd); nbd_clear_que(nbd); - kill_bdev(bdev); + + __invalidate_device(bdev, true); nbd_bdev_reset(bdev); /* * We want to give the run thread a chance to wait for everybody -- 1.8.3.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() 2017-03-23 17:51 ` Josef Bacik @ 2017-03-23 20:58 ` Ming Lin 0 siblings, 0 replies; 6+ messages in thread From: Ming Lin @ 2017-03-23 20:58 UTC (permalink / raw) To: Josef Bacik Cc: nbd-general, Ratna Manoj Bolla, linux-block, linux-kernel, jianshu.ljs, xiongwei.jiang, james.liu On Thu, Mar 23, 2017 at 10:51 AM, Josef Bacik <jbacik@fb.com> wrote: > Yeah I think this is ok, I’ll throw it on my queue for fixes for this cycle. Thanks, Great. Thanks. > > Josef ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-03-23 20:59 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-20 22:58 [RFC PATCH 0/1] nbd: fix crash when unmaping nbd device with fs still mounted Ming Lin 2017-03-20 22:58 ` [RFC PATCH 1/1] nbd: replace kill_bdev() with __invalidate_device() Ming Lin 2017-03-22 20:48 ` [Nbd] " Ming Lin 2017-03-22 21:43 ` Josef Bacik 2017-03-23 17:51 ` Josef Bacik 2017-03-23 20:58 ` Ming Lin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
Powered by JetHome