From: Nikolay Borisov <kernel@kyup.com>
To: "Holger Hoffstätte" <holger.hoffstaette@googlemail.com>,
linux-kernel@vger.kernel.org
Subject: Re: Kernel 4.1.6 Panic due to slab corruption
Date: Mon, 7 Sep 2015 14:30:49 +0300 [thread overview]
Message-ID: <55ED7569.5090906@kyup.com> (raw)
In-Reply-To: <pan.2015.09.07.10.37.07@googlemail.com>
Hi,
If you have the vmlinux image for the kernel you were running at the
time, the crash occured, could you post the output of addr2line -f -e
path/to/vmlinux ffffffff8115bd4d to see if it also fails in
get_freepointer.
Regards,
Nikolay
On 09/07/2015 01:37 PM, Holger Hoffstätte wrote:
> On Mon, 07 Sep 2015 11:41:17 +0300, Nikolay Borisov wrote:
>
>> Hello,
>>
>> On one of our servers I've observed the a kernel pannic
>> happening with the following backtrace:
>>
>> [654405.527070] BUG: unable to handle kernel paging request at 0000000000028001
>> [654405.527076] IP: [<ffffffff81182a59>] kmem_cache_alloc_node+0x99/0x1e0
>> [654405.527085] PGD 14bef58067 PUD 2ab358067 PMD 0
>
> Interesting! I can't offer much help but had a similar panic just the other day
> for no apparent reason while running a bunch of compiles. First time I've seen
> this with 4.1.6:
>
> Sep 5 20:42:02 ragnarok kernel: BUG: unable to handle kernel paging request at ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: IP: [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: PGD 1aa2067 PUD 61f7fd067 PMD 0
> Sep 5 20:42:02 ragnarok kernel: Oops: 0000 [#1] SMP
> Sep 5 20:42:02 ragnarok kernel: Modules linked in: auth_rpcgss oid_registry nfsv4 nfs lockd grace fscache sunrpc autofs4 sch_fq_codel snd_hda_codec_realtek x86_pkg_temp_thermal coretemp snd_hda_codec_generic crc32_pclmul crc32c_intel aesni_intel radeon aes_x86_64 glue_helper snd_hda_codec_hdmi lrw gf128mul ablk_helper cryptd i2c_algo_bit snd_usb_audio uvcvideo snd_hda_intel drm_kms_helper snd_hda_controller snd_hwdep videobuf2_vmalloc snd_usbmidi_lib videobuf2_memops snd_hda_codec videobuf2_core snd_rawmidi i2c_i801 ttm snd_hda_core v4l2_common snd_seq_device videodev snd_pcm usbhid drm snd_timer r8169 snd i2c_core mii soundcore parport_pc parport
> Sep 5 20:42:02 ragnarok kernel: CPU: 0 PID: 32755 Comm: sh Not tainted 4.1.6 #1
> Sep 5 20:42:02 ragnarok kernel: Hardware name: Gigabyte Technology Co., Ltd. P67-DS3-B3/P67-DS3-B3, BIOS F1 05/06/2011
> Sep 5 20:42:02 ragnarok kernel: task: ffff880569712e20 ti: ffff8804e4d90000 task.ti: ffff8804e4d90000
> Sep 5 20:42:02 ragnarok kernel: RIP: 0010:[<ffffffff8115bd4d>] [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: RSP: 0018:ffff8804e4d93d88 EFLAGS: 00010282
> Sep 5 20:42:02 ragnarok kernel: RAX: 0000000000000000 RBX: ffff8805e7eacce0 RCX: 000000000001f7e8
> Sep 5 20:42:02 ragnarok kernel: RDX: 000000000001f7e7 RSI: 00000000000000d0 RDI: 0000000000018c70
> Sep 5 20:42:02 ragnarok kernel: RBP: ffff8804e4d93dc8 R08: ffff88061f418c70 R09: 0000000000000000
> Sep 5 20:42:02 ragnarok kernel: R10: ffffffff81748318 R11: ffffea00139bb500 R12: 00000000000000d0
> Sep 5 20:42:02 ragnarok kernel: R13: ffff880606890600 R14: ffffffff8100d039 R15: ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: FS: 00007f9c1d2f2700(0000) GS:ffff88061f400000(0000) knlGS:0000000000000000
> Sep 5 20:42:02 ragnarok kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740 CR3: 00000005f68ce000 CR4: 00000000000406f0
> Sep 5 20:42:02 ragnarok kernel: Stack:
> Sep 5 20:42:02 ragnarok kernel: 0000000000000000 ffff88061f7e6c00 0000000000000002 ffff8805e7eacce0
> Sep 5 20:42:02 ragnarok kernel: ffff880569712e20 0000000001200011 ffff8805e7eacce0 ffff880569712e20
> Sep 5 20:42:02 ragnarok kernel: ffff8804e4d93de8 ffffffff8100d039 0000000000000000 00007f9c1d2f29d0
> Sep 5 20:42:02 ragnarok kernel: Call Trace:
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8100d039>] arch_dup_task_struct+0x69/0x170
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104de8f>] copy_process.part.8+0x14f/0x1760
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8113909f>] ? handle_mm_fault+0xd0f/0x13a0
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff81171c14>] ? get_empty_filp+0xd4/0x1c0
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105b63f>] ? recalc_sigpending+0x1f/0x60
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f657>] do_fork+0xd7/0x370
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105ed07>] ? sigprocmask+0x57/0x90
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f976>] SyS_clone+0x16/0x20
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff81571d17>] system_call_fastpath+0x12/0x6a
> Sep 5 20:42:02 ragnarok kernel: Code: 65 4c 03 05 ee e3 ea 7e 49 83 78 10 00 4d 8b 38 0f 84 b0 00 00 00 4d 85 ff 0f 84 a7 00 00 00 49 63 45 20 48 8d 4a 01 49 8b 7d 00 <49> 8b 1c 07 4c 89 f8 65 48 0f c7 0f 0f 94 c0 84 c0 74 b9 49 63
> Sep 5 20:42:02 ragnarok kernel: RIP [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: RSP <ffff8804e4d93d88>
> Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: ---[ end trace e4478715791f5752 ]---
> Sep 5 20:42:02 ragnarok kernel: BUG: unable to handle kernel paging request at ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: IP: [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: PGD 1aa2067 PUD 61f7fd067 PMD 0
> Sep 5 20:42:02 ragnarok kernel: Oops: 0000 [#2] SMP
> Sep 5 20:42:02 ragnarok kernel: Modules linked in: auth_rpcgss oid_registry nfsv4 nfs lockd grace fscache sunrpc autofs4 sch_fq_codel snd_hda_codec_realtek x86_pkg_temp_thermal coretemp snd_hda_codec_generic crc32_pclmul crc32c_intel aesni_intel radeon aes_x86_64 glue_helper snd_hda_codec_hdmi lrw gf128mul ablk_helper cryptd i2c_algo_bit snd_usb_audio uvcvideo snd_hda_intel drm_kms_helper snd_hda_controller snd_hwdep videobuf2_vmalloc snd_usbmidi_lib videobuf2_memops snd_hda_codec videobuf2_core snd_rawmidi i2c_i801 ttm snd_hda_core v4l2_common snd_seq_device videodev snd_pcm usbhid drm snd_timer r8169 snd i2c_core mii soundcore parport_pc parport
> Sep 5 20:42:02 ragnarok kernel: CPU: 0 PID: 32550 Comm: sh Tainted: G D 4.1.6 #1
> Sep 5 20:42:02 ragnarok kernel: Hardware name: Gigabyte Technology Co., Ltd. P67-DS3-B3/P67-DS3-B3, BIOS F1 05/06/2011
> Sep 5 20:42:02 ragnarok kernel: task: ffff880602cd1ec0 ti: ffff8805b26ac000 task.ti: ffff8805b26ac000
> Sep 5 20:42:02 ragnarok kernel: RIP: 0010:[<ffffffff8115bd4d>] [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: RSP: 0018:ffff8805b26afd88 EFLAGS: 00010282
> Sep 5 20:42:02 ragnarok kernel: RAX: 0000000000000000 RBX: ffff8805e7ea8f60 RCX: 000000000001f7e8
> Sep 5 20:42:02 ragnarok kernel: RDX: 000000000001f7e7 RSI: 00000000000000d0 RDI: 0000000000018c70
> Sep 5 20:42:02 ragnarok kernel: RBP: ffff8805b26afdc8 R08: ffff88061f418c70 R09: 0000000000000000
> Sep 5 20:42:02 ragnarok kernel: R10: ffffffff81748318 R11: ffffea0015a2ec00 R12: 00000000000000d0
> Sep 5 20:42:02 ragnarok kernel: R13: ffff880606890600 R14: ffffffff8100d039 R15: ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: FS: 00007f9c1d2f2700(0000) GS:ffff88061f400000(0000) knlGS:0000000000000000
> Sep 5 20:42:02 ragnarok kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740 CR3: 00000005e3b76000 CR4: 00000000000406f0
> Sep 5 20:42:02 ragnarok kernel: Stack:
> Sep 5 20:42:02 ragnarok kernel: 0000000000000000 ffff88061f7e6c00 0000000000000002 ffff8805e7ea8f60
> Sep 5 20:42:02 ragnarok kernel: ffff880602cd1ec0 0000000001200011 ffff8805e7ea8f60 ffff880602cd1ec0
> Sep 5 20:42:02 ragnarok kernel: ffff8805b26afde8 ffffffff8100d039 0000000000000000 00007f9c1d2f29d0
> Sep 5 20:42:02 ragnarok kernel: Call Trace:
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8100d039>] arch_dup_task_struct+0x69/0x170
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104de8f>] copy_process.part.8+0x14f/0x1760
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8126a936>] ? security_file_alloc+0x16/0x20
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff81171c14>] ? get_empty_filp+0xd4/0x1c0
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff81185966>] ? __d_instantiate+0x96/0xf0
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff812c6b1a>] ? find_next_zero_bit+0x1a/0x30
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105b63f>] ? recalc_sigpending+0x1f/0x60
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f657>] do_fork+0xd7/0x370
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105ed07>] ? sigprocmask+0x57/0x90
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f976>] SyS_clone+0x16/0x20
> Sep 5 20:42:02 ragnarok kernel: [<ffffffff81571d17>] system_call_fastpath+0x12/0x6a
> Sep 5 20:42:02 ragnarok kernel: Code: 65 4c 03 05 ee e3 ea 7e 49 83 78 10 00 4d 8b 38 0f 84 b0 00 00 00 4d 85 ff 0f 84 a7 00 00 00 49 63 45 20 48 8d 4a 01 49 8b 7d 00 <49> 8b 1c 07 4c 89 f8 65 48 0f c7 0f 0f 94 c0 84 c0 74 b9 49 63
> Sep 5 20:42:02 ragnarok kernel: RIP [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
> Sep 5 20:42:02 ragnarok kernel: RSP <ffff8805b26afd88>
> Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740
> Sep 5 20:42:02 ragnarok kernel: ---[ end trace e4478715791f5753 ]---
>
> ..etc.
>
> I also have all of
>
> CONFIG_SLUB_DEBUG=y
> CONFIG_SLUB=y
> CONFIG_SLUB_CPU_PARTIAL=y
>
> set.
>
> Hope this helps somewhat.
>
> Holger
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
next prev parent reply other threads:[~2015-09-07 11:30 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-07 8:41 Nikolay Borisov
2015-09-07 10:37 ` Holger Hoffstätte
2015-09-07 11:30 ` Nikolay Borisov [this message]
2015-09-07 11:49 ` Holger Hoffstätte
2015-09-07 11:58 ` Holger Hoffstätte
2015-09-07 11:14 ` Nikolay Borisov
2015-09-08 13:58 ` Christoph Lameter
2015-09-08 14:06 ` Nikolay Borisov
2015-09-08 14:27 ` Christoph Lameter
2015-09-08 14:41 ` Nikolay Borisov
2015-09-08 15:15 ` Christoph Lameter
2015-09-09 11:40 ` Nikolay Borisov
2015-09-09 14:01 ` Christoph Lameter
2015-09-09 16:26 ` Vlastimil Babka
2015-09-09 17:58 ` Christoph Lameter
2015-09-10 6:12 ` Nikolay Borisov
2015-09-10 16:30 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55ED7569.5090906@kyup.com \
--to=kernel@kyup.com \
--cc=holger.hoffstaette@googlemail.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome