From: Dave Jones <davej@codemonkey.org.uk>
To: Chris Mason <clm@fb.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Jens Axboe <axboe@fb.com>, Andy Lutomirski <luto@amacapital.net>,
Andy Lutomirski <luto@kernel.org>,
Al Viro <viro@zeniv.linux.org.uk>, Josef Bacik <jbacik@fb.com>,
David Sterba <dsterba@suse.com>,
linux-btrfs <linux-btrfs@vger.kernel.org>,
Linux Kernel <linux-kernel@vger.kernel.org>,
Dave Chinner <david@fromorbit.com>
Subject: Re: btrfs btree_ctree_super fault
Date: Sun, 6 Nov 2016 11:55:39 -0500 [thread overview]
Message-ID: <20161106165539.ybwm6rqvzh2k6uja@codemonkey.org.uk> (raw)
In-Reply-To: <20161031194454.GA49877@clm-mbp.thefacebook.com>
<subject changed, hopefully we're done with bio corruption for now>
On Mon, Oct 31, 2016 at 01:44:55PM -0600, Chris Mason wrote:
> On Mon, Oct 31, 2016 at 12:35:16PM -0700, Linus Torvalds wrote:
> >On Mon, Oct 31, 2016 at 11:55 AM, Dave Jones <davej@codemonkey.org.uk> wrote:
> >>
> >> BUG: Bad page state in process kworker/u8:12 pfn:4e0e39
> >> page:ffffea0013838e40 count:0 mapcount:0 mapping:ffff8804a20310e0 index:0x100c
> >> flags: 0x400000000000000c(referenced|uptodate)
> >> page dumped because: non-NULL mapping
> >
> >Hmm. So this seems to be btrfs-specific, right?
> >
> >I searched for all your "non-NULL mapping" cases, and they all seem to
> >have basically the same call trace, with some work thread doing
> >writeback and going through btrfs_writepages().
> >
> >Sounds like it's a race with either fallocate hole-punching or
> >truncate. I'm not seeing it, but I suspect it's btrfs, since DaveJ
> >clearly ran other filesystems too but I am not seeing this backtrace
> >for anything else.
>
> Agreed, I think this is a separate bug, almost certainly btrfs specific.
> I'll work with Dave on a better reproducer.
Still refining my 'capture ftrace when trinity detects taint' feature,
but in the meantime, here's a variant I don't think we've seen before:
general protection fault: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
CPU: 3 PID: 1913 Comm: trinity-c51 Not tainted 4.9.0-rc3-think+ #3
task: ffff880503350040 task.stack: ffffc90000240000
RIP: 0010:[<ffffffffa007c2f6>]
[<ffffffffa007c2f6>] write_ctree_super+0x96/0xb30 [btrfs]
RSP: 0018:ffffc90000243c90 EFLAGS: 00010286
RAX: dae05adadadad000 RBX: 0000000000000000 RCX: 0000000000000002
RDX: ffff8804fdfcc000 RSI: ffff8804edcee313 RDI: ffff8804edcee1c3
RBP: ffffc90000243d00 R08: 0000000000000003 R09: ffff880000000000
R10: 0000000000000001 R11: 0000000000000100 R12: ffff88045151c548
R13: 0000000000000000 R14: ffff8804ee5122a8 R15: ffff8804572267e8
FS: 00007f25c3e0eb40(0000) GS:ffff880507e00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f25c1560d44 CR3: 0000000454e20000 CR4: 00000000001406e0
DR0: 00007fee93506000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
Stack:
0000000000000001
ffff88050227b3f8
ffff8804fff01b28
00000001810b7f35
ffffffffa007c265
0000000000000001
ffffc90000243cb8
000000002c9a8645
ffff8804fff01b28
ffff8804fff01b28
ffff88045151c548
0000000000000000
Call Trace:
[<ffffffffa007c265>] ? write_ctree_super+0x5/0xb30 [btrfs]
[<ffffffffa00d2956>] btrfs_sync_log+0x886/0xa60 [btrfs]
[<ffffffffa009f4f9>] btrfs_sync_file+0x479/0x4d0 [btrfs]
[<ffffffff812789ab>] vfs_fsync_range+0x4b/0xb0
[<ffffffff81260755>] ? __fget_light+0x5/0x60
[<ffffffff81278a6d>] do_fsync+0x3d/0x70
[<ffffffff81278a35>] ? do_fsync+0x5/0x70
[<ffffffff81278d20>] SyS_fsync+0x10/0x20
[<ffffffff81002d81>] do_syscall_64+0x61/0x170
[<ffffffff81894a8b>] entry_SYSCALL64_slow_path+0x25/0x25
Code: c7 48 8b 42 30 4c 8b 08 48 b8 00 00 00 00 00 16 00 00 49 03 81 a0 01 00 00 49 b9 00 00 00 00 00 88 ff ff 48 c1 f8 06 48 c1 e0 0c <4a> 8b 44 08 50 48 39 46 08 0f 84 8d 08 00 00 49 63 c0 48 8d 0c
RIP
[<ffffffffa007c2f6>] write_ctree_super+0x96/0xb30 [btrfs]
RSP <ffffc90000243c90>
All code
========
0: c7 (bad)
1: 48 8b 42 30 mov 0x30(%rdx),%rax
5: 4c 8b 08 mov (%rax),%r9
8: 48 b8 00 00 00 00 00 movabs $0x160000000000,%rax
f: 16 00 00
12: 49 03 81 a0 01 00 00 add 0x1a0(%r9),%rax
19: 49 b9 00 00 00 00 00 movabs $0xffff880000000000,%r9
20: 88 ff ff
23: 48 c1 f8 06 sar $0x6,%rax
27: 48 c1 e0 0c shl $0xc,%rax
2b:* 4a 8b 44 08 50 mov 0x50(%rax,%r9,1),%rax <-- trapping instruction
30: 48 39 46 08 cmp %rax,0x8(%rsi)
34: 0f 84 8d 08 00 00 je 0x8c7
3a: 49 63 c0 movslq %r8d,%rax
3d: 48 rex.W
3e: 8d .byte 0x8d
3f: 0c .byte 0xc
Code starting with the faulting instruction
===========================================
0: 4a 8b 44 08 50 mov 0x50(%rax,%r9,1),%rax
5: 48 39 46 08 cmp %rax,0x8(%rsi)
9: 0f 84 8d 08 00 00 je 0x89c
f: 49 63 c0 movslq %r8d,%rax
12: 48 rex.W
13: 8d .byte 0x8d
14: 0c .byte 0xc
According to objdump -S, it looks like this is an inlined copy of backup_super_roots
root_backup = info->super_for_commit->super_roots + last_backup;
2706: 48 8d b8 2b 0b 00 00 lea 0xb2b(%rax),%rdi
270d: 48 63 c1 movslq %ecx,%rax
2710: 48 8d 34 80 lea (%rax,%rax,4),%rsi
2714: 48 8d 04 b0 lea (%rax,%rsi,4),%rax
2718: 48 8d 34 c7 lea (%rdi,%rax,8),%rsi
btrfs_header_generation(info->tree_root->node))
271c: 48 8b 42 30 mov 0x30(%rdx),%rax
2720: 4c 8b 08 mov (%rax),%r9
2723: 48 b8 00 00 00 00 00 movabs $0x160000000000,%rax
272a: 16 00 00
272d: 49 03 81 a0 01 00 00 add 0x1a0(%r9),%rax
if (btrfs_backup_tree_root_gen(root_backup) ==
2734: 49 b9 00 00 00 00 00 movabs $0xffff880000000000,%r9
273b: 88 ff ff
273e: 48 c1 f8 06 sar $0x6,%rax
2742: 48 c1 e0 0c shl $0xc,%rax
2746: 4a 8b 44 08 50 mov 0x50(%rax,%r9,1),%rax <-- trapping instruction
274b: 48 39 46 08 cmp %rax,0x8(%rsi)
274f: 0f 84 8d 08 00 00 je 2fe2 <write_ctree_super+0x932>
next prev parent reply other threads:[~2016-11-06 16:55 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-11 14:45 btrfs bio linked list corruption Dave Jones
2016-10-11 15:11 ` Al Viro
2016-10-11 15:19 ` Dave Jones
2016-10-11 15:20 ` Chris Mason
2016-10-11 15:49 ` Dave Jones
2016-10-11 15:54 ` Chris Mason
2016-10-11 16:25 ` Dave Jones
2016-10-12 13:47 ` Dave Jones
2016-10-12 14:40 ` Dave Jones
2016-10-12 14:42 ` Chris Mason
2016-10-13 18:16 ` Dave Jones
2016-10-13 21:18 ` Chris Mason
2016-10-13 21:56 ` Dave Jones
2016-10-16 0:42 ` Dave Jones
2016-10-18 1:07 ` Chris Mason
2016-10-18 22:42 ` Dave Jones
2016-10-18 23:12 ` Jens Axboe
2016-10-18 23:31 ` Chris Mason
2016-10-18 23:36 ` Jens Axboe
2016-10-18 23:39 ` Linus Torvalds
2016-10-18 23:42 ` Chris Mason
2016-10-19 0:10 ` Linus Torvalds
2016-10-19 0:19 ` Chris Mason
2016-10-19 0:28 ` Linus Torvalds
2016-10-20 22:48 ` Dave Jones
2016-10-19 1:05 ` Andy Lutomirski
2016-10-20 22:50 ` Dave Jones
2016-10-20 23:01 ` Andy Lutomirski
2016-10-20 23:03 ` Dave Jones
2016-10-20 23:23 ` Andy Lutomirski
2016-10-21 20:02 ` Dave Jones
2016-10-21 20:17 ` Chris Mason
2016-10-21 20:23 ` Dave Jones
2016-10-21 20:38 ` Chris Mason
2016-10-21 20:41 ` Josef Bacik
2016-10-21 21:11 ` Dave Jones
2016-10-22 15:20 ` Dave Jones
2016-10-23 21:32 ` Chris Mason
2016-10-24 4:40 ` Dave Jones
2016-10-24 13:42 ` Chris Mason
2016-10-26 0:27 ` Dave Jones
2016-10-26 1:33 ` Linus Torvalds
2016-10-26 1:39 ` Linus Torvalds
2016-10-26 16:30 ` Dave Jones
2016-10-26 16:48 ` Linus Torvalds
2016-10-26 18:18 ` Dave Jones
2016-10-26 18:42 ` Dave Jones
2016-10-26 19:06 ` Linus Torvalds
2016-10-26 20:00 ` Chris Mason
2016-10-26 21:52 ` Chris Mason
2016-10-26 22:21 ` Linus Torvalds
2016-10-26 22:40 ` Dave Jones
2016-10-26 22:51 ` Linus Torvalds
2016-10-26 22:55 ` Jens Axboe
2016-10-26 22:58 ` Linus Torvalds
2016-10-26 23:03 ` Jens Axboe
2016-10-26 23:07 ` Dave Jones
2016-10-26 23:08 ` Linus Torvalds
2016-10-26 23:20 ` Jens Axboe
2016-10-26 23:38 ` Chris Mason
2016-10-26 23:47 ` Dave Jones
2016-10-27 0:00 ` Jens Axboe
2016-10-27 13:33 ` Chris Mason
2016-10-31 18:55 ` Dave Jones
2016-10-31 19:35 ` Linus Torvalds
2016-10-31 19:44 ` Chris Mason
2016-11-06 16:55 ` Dave Jones [this message]
2016-11-08 14:59 ` btrfs btree_ctree_super fault Dave Jones
2016-11-08 15:08 ` Chris Mason
2016-11-10 14:35 ` Dave Jones
2016-11-10 15:27 ` Chris Mason
2016-11-23 19:34 ` bio linked list corruption Dave Jones
2016-11-23 19:58 ` Dave Jones
2016-12-01 15:32 ` btrfs_destroy_inode warn (outstanding extents) Dave Jones
2016-12-03 16:48 ` Dave Jones
2016-12-07 16:15 ` Dave Jones
2016-12-09 21:12 ` Steven Rostedt
2016-12-04 23:04 ` bio linked list corruption Vegard Nossum
2016-12-05 11:10 ` Vegard Nossum
2016-12-05 17:09 ` Vegard Nossum
2016-12-05 17:21 ` Dave Jones
2016-12-05 17:55 ` Linus Torvalds
2016-12-05 19:11 ` Vegard Nossum
2016-12-05 20:10 ` Linus Torvalds
2016-12-05 20:35 ` Linus Torvalds
2016-12-05 21:33 ` Vegard Nossum
2016-12-06 8:42 ` Vegard Nossum
2016-12-06 8:16 ` Peter Zijlstra
2016-12-06 8:36 ` Ingo Molnar
2016-12-06 16:33 ` Linus Torvalds
2016-12-05 20:10 ` Vegard Nossum
2016-12-05 18:11 ` Andy Lutomirski
2016-12-05 18:25 ` Linus Torvalds
2016-12-05 18:26 ` Vegard Nossum
2016-10-26 23:19 ` Chris Mason
2016-10-26 23:21 ` Jens Axboe
2016-10-27 6:33 ` Christoph Hellwig
2016-10-27 16:34 ` Linus Torvalds
2016-10-27 16:36 ` Jens Axboe
2016-10-26 23:01 ` Dave Jones
2016-10-26 23:05 ` Jens Axboe
2016-10-26 22:52 ` Jens Axboe
2016-10-26 22:07 ` Linus Torvalds
2016-10-26 22:54 ` Chris Mason
2016-10-27 5:41 ` Dave Chinner
2016-10-27 17:23 ` Dave Jones
2016-10-24 20:06 ` Andy Lutomirski
2016-10-24 20:46 ` Linus Torvalds
2016-10-24 21:17 ` Linus Torvalds
2016-10-24 21:50 ` Linus Torvalds
2016-10-24 22:02 ` Chris Mason
2016-10-24 22:42 ` Andy Lutomirski
2016-10-25 0:00 ` Linus Torvalds
2016-10-25 1:09 ` Andy Lutomirski
2016-10-19 17:09 ` Philipp Hahn
2016-10-19 17:43 ` Linus Torvalds
2016-10-20 6:52 ` Ingo Molnar
2016-10-20 7:17 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161106165539.ybwm6rqvzh2k6uja@codemonkey.org.uk \
--to=davej@codemonkey.org.uk \
--cc=axboe@fb.com \
--cc=clm@fb.com \
--cc=david@fromorbit.com \
--cc=dsterba@suse.com \
--cc=jbacik@fb.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=luto@kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®