* kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210
@ 2013-10-20 18:05 Sander Eikelenboom
2013-10-22 15:12 ` Christoph Lameter
0 siblings, 1 reply; 5+ messages in thread
From: Sander Eikelenboom @ 2013-10-20 18:05 UTC (permalink / raw)
To: Christoph Lameter, Pekka Enberg; +Cc: linux-kernel
Hi,
With i 3.12.0-rc6 kernel i encounterd this splat:
[19424.151472] ------------[ cut here ]------------
[19424.151496] kernel BUG at mm/slub.c:3338!
[19424.151506] invalid opcode: 0000 [#1] PREEMPT SMP
[19424.151520] Modules linked in:
[19424.151532] CPU: 1 PID: 2824 Comm: glusterfs Not tainted 3.12.0-rc6-20131020 #1
[19424.151549] task: ffff88001d140000 ti: ffff88001df4c000 task.ti: ffff88001df4c000
[19424.151561] RIP: e030:[<ffffffff8117912c>] [<ffffffff8117912c>] kfree+0x20c/0x210
[19424.151580] RSP: e02b:ffff88001df4dde8 EFLAGS: 00010246
[19424.151589] RAX: 0040000000000000 RBX: ffff88001df4de68 RCX: 0000000000000000
[19424.151600] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff88001df4de68
[19424.151611] RBP: ffff88001df4de28 R08: ffff88001e0f7930 R09: 0000000000000058
[19424.151622] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88001cf4de68
[19424.151633] R13: ffffea000077d340 R14: ffff88001e317a00 R15: 0000000000000058
[19424.151649] FS: 00007f549af8c700(0000) GS:ffff88001fd00000(0000) knlGS:0000000000000000
[19424.151661] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[19424.151671] CR2: 00007f549f026000 CR3: 000000001c891000 CR4: 0000000000000660
[19424.151683] Stack:
[19424.151689] 0000000000020050 0000000000000000 0000000000000000 0000000000000000
[19424.151708] ffff88001cf4de68 0000000000000002 ffff88001e317a00 0000000000000058
[19424.151726] ffff88001df4df18 ffffffff81186b01 ffff88001df4ded8 ffffffff810f0edf
[19424.151744] Call Trace:
[19424.151756] [<ffffffff81186b01>] do_readv_writev+0x111/0x250
[19424.151771] [<ffffffff810f0edf>] ? lock_release_non_nested+0xaf/0x320
[19424.151784] [<ffffffff810f127f>] ? lock_release+0x12f/0x250
[19424.151796] [<ffffffff81186ccd>] vfs_readv+0x2d/0x50
[19424.151807] [<ffffffff81186d44>] SyS_readv+0x54/0xd0
[19424.151820] [<ffffffff81a560b9>] system_call_fastpath+0x16/0x1b
[19424.151830] Code: 89 4d 8b 6d 30 e9 7e fe ff ff 48 8b 4d c0 48 89 da 4c 89 ee 4c 89 e7 e8 54 e4 8c 00 e9 69 ff ff ff e8 19 a2 8d 00 e9 2f ff ff ff <0f> 0b 66 90 55 48 89 e5 48 83 ec 30 48 89 5d d8 4c 89 75 f0 48
[19424.151954] RIP [<ffffffff8117912c>] kfree+0x20c/0x210
[19424.151970] RSP <ffff88001df4dde8>
[19424.151983] ---[ end trace 225a70bf87bd3a82 ]---
--
Sander
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210
2013-10-20 18:05 kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210 Sander Eikelenboom
@ 2013-10-22 15:12 ` Christoph Lameter
2013-10-22 15:29 ` Sander Eikelenboom
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Lameter @ 2013-10-22 15:12 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Pekka Enberg, linux-kernel
On Sun, 20 Oct 2013, Sander Eikelenboom wrote:
> With i 3.12.0-rc6 kernel i encounterd this splat:
Likely some slab object got corrupted.
Please enable debugging by either specifying slub_debug on the kernel
command line or enabling CONFIG_SLUB_DEBUG_ON
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210
2013-10-22 15:12 ` Christoph Lameter
@ 2013-10-22 15:29 ` Sander Eikelenboom
2013-11-09 18:18 ` Sander Eikelenboom
0 siblings, 1 reply; 5+ messages in thread
From: Sander Eikelenboom @ 2013-10-22 15:29 UTC (permalink / raw)
To: Christoph Lameter; +Cc: Pekka Enberg, linux-kernel
Tuesday, October 22, 2013, 5:12:35 PM, you wrote:
> On Sun, 20 Oct 2013, Sander Eikelenboom wrote:
>> With i 3.12.0-rc6 kernel i encounterd this splat:
> Likely some slab object got corrupted.
> Please enable debugging by either specifying slub_debug on the kernel
> command line or enabling CONFIG_SLUB_DEBUG_ON
Well since it seems a "one timer" so i'm not very sure if and when i would encounter it again.
It seems "slub_debug" on the commandline would do the same, so i will run my VM with that for some time, see if triggers again.
--
Sander
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210
2013-10-22 15:29 ` Sander Eikelenboom
@ 2013-11-09 18:18 ` Sander Eikelenboom
2013-11-11 14:57 ` Christoph Lameter
0 siblings, 1 reply; 5+ messages in thread
From: Sander Eikelenboom @ 2013-11-09 18:18 UTC (permalink / raw)
To: Christoph Lameter, Pekka Enberg; +Cc: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]
Hello Sander,
Tuesday, October 22, 2013, 5:29:05 PM, you wrote:
> Tuesday, October 22, 2013, 5:12:35 PM, you wrote:
>> On Sun, 20 Oct 2013, Sander Eikelenboom wrote:
>>> With i 3.12.0-rc6 kernel i encounterd this splat:
>> Likely some slab object got corrupted.
>> Please enable debugging by either specifying slub_debug on the kernel
>> command line or enabling CONFIG_SLUB_DEBUG_ON
> Well since it seems a "one timer" so i'm not very sure if and when i would encounter it again.
> It seems "slub_debug" on the commandline would do the same, so i will run my VM with that for some time, see if triggers again.
> --
> Sander
Something triggered another oops on this VM, although it looks a bit different.
This VM got quite some I/O and network load, it got a lot of large files transferred to Samba and the files are stored on a glusterfs volume (by tcp/ip).
[34856.970314] =============================================================================
[34856.970332] BUG kmalloc-2048 (Not tainted): Poison overwritten
[34856.970339] -----------------------------------------------------------------------------
[34856.970339]
[34856.970350] Disabling lock debugging due to kernel taint
[34856.970357] INFO: 0xffff88001b36099b-0xffff88001b36099b. First byte 0x6a instead of 0x6b
[34856.970371] INFO: Allocated in __alloc_skb+0x82/0x290 age=2 cpu=0 pid=2818
[34856.970381] __slab_alloc.isra.57+0x61e/0x6d6
[34856.970389] __kmalloc_node_track_caller+0xc9/0x1b0
[34856.970396] __kmalloc_reserve.isra.49+0x37/0xa0
[34856.970402] __alloc_skb+0x82/0x290
[34856.970408] sk_stream_alloc_skb+0x3f/0x120
[34856.970414] tcp_sendmsg+0x615/0xe20
[34856.970419] inet_sendmsg+0xa6/0x100
[34856.970426] sock_aio_write+0xc2/0xe0
[34856.970433] do_sync_readv_writev+0x4e/0x80
[34856.970439] do_readv_writev+0xe8/0x270
[34856.970444] vfs_writev+0x30/0x60
[34856.970449] SyS_writev+0x54/0xd0
[34856.970454] system_call_fastpath+0x16/0x1b
[34856.970461] INFO: Freed in skb_free_head+0x64/0x70 age=2 cpu=0 pid=0
[34856.970468] __slab_free+0x30/0x561
[34856.970474] kfree+0x1fd/0x210
[34856.970479] skb_free_head+0x64/0x70
[34856.970484] skb_release_data+0xf3/0x100
[34856.970492] skb_release_all+0x23/0x30
[34856.970497] __kfree_skb+0x11/0xa0
[34856.970503] tcp_ack+0x64a/0xf20
[34856.970509] tcp_rcv_established+0xe2/0x5c0
[34856.970515] tcp_v4_do_rcv+0x14e/0x4a0
[34856.970520] tcp_v4_rcv+0x8f2/0x9c0
[34856.970527] ip_local_deliver_finish+0xd8/0x1d0
[34856.970533] ip_local_deliver+0x34/0x90
[34856.970539] ip_rcv_finish+0x150/0x660
[34856.970544] ip_rcv+0x22b/0x370
[34856.970551] __netif_receive_skb_core+0x6e2/0x8b0
[34856.970557] __netif_receive_skb+0x1c/0x70
[34856.970563] INFO: Slab 0xffffea00006cd800 objects=13 used=13 fp=0x (null) flags=0x40000000004080
[34856.970572] INFO: Object 0xffff88001b360948 @offset=2376 fp=0xffff88001b3637b0
[34856.970572]
The rest is in the attached dmesg.
attached: dmesg-oops-3.12.0-20131104.txt
--
Sander
[-- Attachment #2: dmesg-oops-3.12.0-20131104.txt --]
[-- Type: text/plain, Size: 47098 bytes --]
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.12.0-20131104 (root@serveerstertje) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP PREEMPT Mon Nov 4 09:34:18 CET 2013
[ 0.000000] Command line: root=/dev/xvda1 ro slub_debug
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x000000001fffffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] MPS support code is not built-in.
[ 0.000000] Using acpi=off or acpi=noirq or pci=noacpi may have problem
[ 0.000000] DMI not present or invalid.
[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000000] No AGP bridge found
[ 0.000000] e820: last_pfn = 0x20000 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] Base memory trampoline at [ffff88000009a000] 9a000 size 24576
[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[ 0.000000] [mem 0x00000000-0x000fffff] page 4k
[ 0.000000] init_memory_mapping: [mem 0x1fe00000-0x1fffffff]
[ 0.000000] [mem 0x1fe00000-0x1fffffff] page 4k
[ 0.000000] BRK [0x02d72000, 0x02d72fff] PGTABLE
[ 0.000000] init_memory_mapping: [mem 0x1c000000-0x1fdfffff]
[ 0.000000] [mem 0x1c000000-0x1fdfffff] page 4k
[ 0.000000] BRK [0x02d73000, 0x02d73fff] PGTABLE
[ 0.000000] BRK [0x02d74000, 0x02d74fff] PGTABLE
[ 0.000000] BRK [0x02d75000, 0x02d75fff] PGTABLE
[ 0.000000] BRK [0x02d76000, 0x02d76fff] PGTABLE
[ 0.000000] BRK [0x02d77000, 0x02d77fff] PGTABLE
[ 0.000000] init_memory_mapping: [mem 0x00100000-0x1bffffff]
[ 0.000000] [mem 0x00100000-0x1bffffff] page 4k
[ 0.000000] RAMDISK: [mem 0x03188000-0x03f9cfff]
[ 0.000000] NUMA turned off
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x1fffffff]
[ 0.000000] NODE_DATA [mem 0x1ff1d000-0x1ff27fff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00001000-0x0009ffff]
[ 0.000000] node 0: [mem 0x00100000-0x1fffffff]
[ 0.000000] On node 0 totalpages: 130975
[ 0.000000] DMA zone: 64 pages used for memmap
[ 0.000000] DMA zone: 21 pages reserved
[ 0.000000] DMA zone: 3999 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 1984 pages used for memmap
[ 0.000000] DMA32 zone: 126976 pages, LIFO batch:31
[ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] nr_irqs_gsi: 16
[ 0.000000] e820: [mem 0x20000000-0xffffffff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.4-unstable (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:2 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88001fc00000 s83200 r8192 d23296 u1048576
[ 0.000000] pcpu-alloc: s83200 r8192 d23296 u1048576 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 0 1
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 128906
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/xvda1 ro slub_debug
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Memory: 468620K/523900K available (10729K kernel code, 978K rwdata, 4200K rodata, 1068K init, 9580K bss, 55280K reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.
[ 0.000000] Additional per-CPU info printed with stalls.
[ 0.000000] RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=2.
[ 0.000000] NR_IRQS:4352 nr_irqs:288 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[ 0.000000] ... MAX_LOCKDEP_SUBCLASSES: 8
[ 0.000000] ... MAX_LOCK_DEPTH: 48
[ 0.000000] ... MAX_LOCKDEP_KEYS: 8191
[ 0.000000] ... CLASSHASH_SIZE: 4096
[ 0.000000] ... MAX_LOCKDEP_ENTRIES: 16384
[ 0.000000] ... MAX_LOCKDEP_CHAINS: 32768
[ 0.000000] ... CHAINHASH_SIZE: 16384
[ 0.000000] memory used by lock dependency info: 5855 kB
[ 0.000000] per task-struct memory footprint: 1920 bytes
[ 0.000000] kmemleak: Kernel memory leak detector disabled
[ 0.000000] Xen: using vcpuop timer interface
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] tsc: Detected 3200.180 MHz processor
[ 0.000000] tsc: Marking TSC unstable due to TSCs unsynchronized
[ 0.003333] Calibrating delay loop (skipped), value calculated using timer frequency.. 6402.04 BogoMIPS (lpj=10667266)
[ 0.003333] pid_max: default: 32768 minimum: 301
[ 0.003333] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.003333] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.003333] Mount-cache hash table entries: 256
[ 0.003333] Initializing cgroup subsys freezer
[ 0.003333] Initializing cgroup subsys blkio
[ 0.003333] tseg: 0000000000
[ 0.003333] CPU: Physical Processor ID: 0
[ 0.003333] CPU: Processor Core ID: 1
[ 0.003333] Last level iTLB entries: 4KB 512, 2MB 16, 4MB 8
[ 0.003333] Last level dTLB entries: 4KB 512, 2MB 128, 4MB 64
[ 0.003333] tlb_flushall_shift: 4
[ 0.044828] cpu 0 spinlock event irq 17
[ 0.045830] Performance Events:
[ 0.045848] no APIC, boot with the "lapic" boot parameter to force-enable it.
[ 0.045855] no hardware sampling interrupt available.
[ 0.045880] Broken PMU hardware detected, using software events only.
[ 0.045882] Failed to access perfctr msr (MSR c0010004 is 0)
[ 0.063598] NMI watchdog: disabled (cpu0): hardware events not enabled
[ 0.070288] installing Xen timer for CPU 1
[ 0.070377] cpu 1 spinlock event irq 24
[ 0.070456] SMP alternatives: lockdep: fixing up alternatives
[ 0.070469] SMP alternatives: switching to SMP code
[ 0.112838] Brought up 2 CPUs
[ 0.113810] devtmpfs: initialized
[ 0.115040] xor: measuring software checksum speed
[ 0.144411] prefetch64-sse: 5412.000 MB/sec
[ 0.178307] generic_sse: 5330.400 MB/sec
[ 0.178318] xor: using function: prefetch64-sse (5412.000 MB/sec)
[ 0.178361] xen:grant_table: Grant tables using version 2 layout
[ 0.178420] Grant table initialized
[ 0.178917] NET: Registered protocol family 16
[ 0.183776] PCI: setting up Xen PCI frontend stub
[ 0.183800] PCI: pci_cache_line_size set to 64 bytes
[ 0.320441] bio: create slab <bio-0> at 0
[ 0.376754] raid6: sse2x1 3860 MB/s
[ 0.433368] raid6: sse2x2 5362 MB/s
[ 0.490003] raid6: sse2x4 5857 MB/s
[ 0.490005] raid6: using algorithm sse2x4 (5857 MB/s)
[ 0.490006] raid6: using intx1 recovery algorithm
[ 0.490306] ACPI: Interpreter disabled.
[ 0.490306] xen:balloon: Initialising balloon driver
[ 0.493520] xen_balloon: Initialising balloon driver
[ 0.496781] vgaarb: loaded
[ 0.497942] SCSI subsystem initialized
[ 0.497942] libata version 3.00 loaded.
[ 0.500538] usbcore: registered new interface driver usbfs
[ 0.500538] usbcore: registered new interface driver hub
[ 0.500538] usbcore: registered new device driver usb
[ 0.500538] Linux video capture interface: v2.00
[ 0.503529] pps_core: LinuxPPS API ver. 1 registered
[ 0.503542] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.503612] PTP clock support registered
[ 0.503625] Advanced Linux Sound Architecture Driver Initialized.
[ 0.503648] PCI: System does not support PCI
[ 0.503650] PCI: System does not support PCI
[ 0.506907] Bluetooth: Core ver 2.16
[ 0.506997] NET: Registered protocol family 31
[ 0.506997] Bluetooth: HCI device and connection manager initialized
[ 0.506997] Bluetooth: HCI socket layer initialized
[ 0.506997] Bluetooth: L2CAP socket layer initialized
[ 0.506997] Bluetooth: SCO socket layer initialized
[ 0.507279] Switched to clocksource xen
[ 0.507396] FS-Cache: Loaded
[ 0.507636] pnp: PnP ACPI: disabled
[ 0.539117] NET: Registered protocol family 2
[ 0.539897] TCP established hash table entries: 4096 (order: 4, 65536 bytes)
[ 0.540019] TCP bind hash table entries: 4096 (order: 6, 262144 bytes)
[ 0.540277] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.540362] TCP: reno registered
[ 0.540401] UDP hash table entries: 256 (order: 3, 40960 bytes)
[ 0.540458] UDP-Lite hash table entries: 256 (order: 3, 40960 bytes)
[ 0.540871] NET: Registered protocol family 1
[ 0.540925] PCI: CLS 0 bytes, default 64
[ 0.541330] Trying to unpack rootfs image as initramfs...
[ 0.604671] Freeing initrd memory: 14420K (ffff880003188000 - ffff880003f9d000)
[ 0.810210] DMA-API: preallocated 65536 debug entries
[ 0.810232] DMA-API: debugging enabled by kernel config
[ 0.810899] platform rtc_cmos: registered platform RTC device (no PNP device found)
[ 0.813974] Scanning for low memory corruption every 60 seconds
[ 0.817883] sha1_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817897] sha256_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817902] sha512_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817908] AVX or AES-NI instructions are not detected.
[ 0.817912] AVX instructions are not detected.
[ 0.817916] AVX instructions are not detected.
[ 0.820512] audit: initializing netlink socket (disabled)
[ 0.820601] type=2000 audit(1383778218.007:1): initialized
[ 0.822275] bounce pool size: 64 pages
[ 0.822293] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.878200] VFS: Disk quotas dquot_6.5.2
[ 0.878818] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.897415] NTFS driver 2.1.30 [Flags: R/W].
[ 0.900483] fuse init (API version 7.22)
[ 0.905703] bio: create slab <bio-1> at 1
[ 0.910323] Btrfs loaded
[ 0.914633] GFS2 installed
[ 0.916780] ceph: loaded (mds proto 32)
[ 0.916827] msgmni has been set to 943
[ 0.920847] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[ 0.920891] io scheduler noop registered
[ 0.920896] io scheduler deadline registered
[ 0.921766] io scheduler cfq registered (default)
[ 0.923034] crc32: CRC_LE_BITS = 64, CRC_BE BITS = 64
[ 0.923046] crc32: self tests passed, processed 225944 bytes in 109410 nsec
[ 0.923163] crc32c: CRC_LE_BITS = 64
[ 0.923168] crc32c: self tests passed, processed 225944 bytes in 54579 nsec
[ 0.924335] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.925132] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[ 0.925447] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.925456] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
[ 0.926534] usbcore: registered new interface driver udlfb
[ 0.927691] xen:xen_evtchn: Event-channel device installed
[ 0.932426] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.936695] Linux agpgart interface v0.103
[ 0.938516] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
[ 0.938518] Hangcheck: Using getrawmonotonic().
[ 0.938784] [drm] Initialized drm 1.1.0 20060810
[ 0.939112] [drm] radeon kernel modesetting enabled.
[ 0.957684] brd: module loaded
[ 0.966412] loop: module loaded
[ 0.975940] tun: Universal TUN/TAP device driver, 1.6
[ 0.975955] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[ 0.976391] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[ 0.976403] e1000: Copyright (c) 1999-2006 Intel Corporation.
[ 0.976818] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
[ 0.976831] e1000e: Copyright(c) 1999 - 2013 Intel Corporation.
[ 0.977343] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
[ 0.977353] igb: Copyright (c) 2007-2013 Intel Corporation.
[ 0.977696] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
[ 0.977709] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[ 0.978502] xen_netfront: Initialising Xen virtual ethernet driver
[ 0.983343] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.983353] ehci-pci: EHCI PCI platform driver
[ 0.984249] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.984258] ohci-pci: OHCI PCI platform driver
[ 0.984564] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.986205] usbcore: registered new interface driver usblp
[ 0.986958] usbcore: registered new interface driver usb-storage
[ 0.987932] usbcore: registered new interface driver usbserial
[ 0.988274] usbcore: registered new interface driver cp210x
[ 0.988703] usbserial: USB Serial support registered for cp210x
[ 0.988952] usbcore: registered new interface driver cypress_m8
[ 0.989238] usbserial: USB Serial support registered for DeLorme Earthmate USB
[ 0.989573] usbserial: USB Serial support registered for HID->COM RS232 Adapter
[ 0.989870] usbserial: USB Serial support registered for Nokia CA-42 V2 Adapter
[ 0.990111] usbcore: registered new interface driver mos7720
[ 0.990417] usbserial: USB Serial support registered for Moschip 2 port adapter
[ 0.990654] usbcore: registered new interface driver mos7840
[ 0.993506] usbserial: USB Serial support registered for Moschip 7840/7820 USB Serial Driver
[ 0.994349] i8042: PNP: No PS/2 controller found. Probing ports directly.
[ 1.009583] blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[ 2.002564] i8042: No controller found
[ 2.004200] mousedev: PS/2 mouse device common for all mice
[ 2.066703] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 2.068031] rtc_cmos: probe of rtc_cmos failed with error -38
[ 2.071810] lirc_dev: IR Remote Control driver registered, major 248
[ 2.071854] IR NEC protocol handler initialized
[ 2.071860] IR RC5(x) protocol handler initialized
[ 2.071864] IR RC6 protocol handler initialized
[ 2.071868] IR JVC protocol handler initialized
[ 2.071873] IR Sony protocol handler initialized
[ 2.071877] IR RC5 (streamzap) protocol handler initialized
[ 2.071882] IR SANYO protocol handler initialized
[ 2.071886] IR MCE Keyboard/mouse protocol handler initialized
[ 2.071891] IR LIRC bridge handler initialized
[ 2.071896] cx25821: driver version 0.0.106 loaded
[ 2.072860] usbcore: registered new interface driver pvrusb2
[ 2.072890] pvrusb2: V4L in-tree version:Hauppauge WinTV-PVR-USB2 MPEG2 Encoder/Tuner
[ 2.072897] pvrusb2: Debug mask is 31 (0x1f)
[ 2.073903] sp5100_tco: SP5100/SB800 TCO WatchDog Timer Driver v0.05
[ 2.074490] xen_wdt: Xen WatchDog Timer Driver v0.01
[ 2.075432] xen_wdt: initialized (timeout=60s, nowayout=0)
[ 2.079512] device-mapper: ioctl: 4.26.0-ioctl (2013-08-15) initialised: dm-devel@redhat.com
[ 2.082630] device-mapper: cache-policy-mq: version 1.0.0 loaded
[ 2.082660] device-mapper: cache cleaner: version 1.0.0 loaded
[ 2.082666] Bluetooth: Virtual HCI driver ver 1.3
[ 2.083086] Bluetooth: HCI UART driver ver 2.2
[ 2.083098] Bluetooth: HCI H4 protocol initialized
[ 2.083103] Bluetooth: HCI BCSP protocol initialized
[ 2.083107] Bluetooth: HCILL protocol initialized
[ 2.083111] Bluetooth: HCIATH3K protocol initialized
[ 2.083115] Bluetooth: HCI Three-wire UART (H5) protocol initialized
[ 2.083439] usbcore: registered new interface driver bcm203x
[ 2.084033] usbcore: registered new interface driver bpa10x
[ 2.084374] usbcore: registered new interface driver bfusb
[ 2.084716] usbcore: registered new interface driver btusb
[ 2.085210] usbcore: registered new interface driver ath3k
[ 2.085814] hidraw: raw HID events driver (C) Jiri Kosina
[ 2.088974] usbcore: registered new interface driver usbhid
[ 2.088983] usbhid: USB HID core driver
[ 2.097677] usbcore: registered new interface driver snd-usb-audio
[ 2.097975] usbcore: registered new interface driver snd-ua101
[ 2.098252] usbcore: registered new interface driver snd-usb-usx2y
[ 2.098526] usbcore: registered new interface driver snd-usb-caiaq
[ 2.098797] usbcore: registered new interface driver snd-usb-6fire
[ 2.098882] Netfilter messages via NETLINK v0.30.
[ 2.098913] nfnl_acct: registering with nfnetlink.
[ 2.099088] nf_conntrack version 0.5.0 (3773 buckets, 15092 max)
[ 2.100344] ctnetlink v0.93: registering with nfnetlink.
[ 2.101476] xt_time: kernel timezone is -0000
[ 2.101498] xvda: xvda1 xvda2
[ 2.101509] ip_set: protocol 6
[ 2.101550] IPVS: Registered protocols ()
[ 2.102027] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[ 2.102231] IPVS: Creating netns size=1904 id=0
[ 2.102595] IPVS: ipvs loaded.
[ 2.103069] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 2.103473] TCP: cubic registered
[ 2.103480] NET: Registered protocol family 17
[ 2.104492] Bridge firewalling registered
[ 2.104524] Ebtables v2.0 registered
[ 2.105453] Bluetooth: RFCOMM TTY layer initialized
[ 2.105561] Bluetooth: RFCOMM socket layer initialized
[ 2.105567] Bluetooth: RFCOMM ver 1.11
[ 2.105571] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 2.105576] Bluetooth: BNEP filters: protocol multicast
[ 2.105598] Bluetooth: BNEP socket layer initialized
[ 2.105603] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[ 2.105626] Bluetooth: HIDP socket layer initialized
[ 2.105668] Key type ceph registered
[ 2.107631] libceph: loaded (mon/osd proto 15/24)
[ 2.109119] registered taskstats version 1
[ 2.109623] console [netcon0] enabled
[ 2.109634] netconsole: network logging started
[ 2.109640] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 2.109837] ALSA device list:
[ 2.109844] No soundcards found.
[ 2.111258] Freeing unused kernel memory: 1068K (ffffffff822f6000 - ffffffff82401000)
[ 2.111275] Write protecting the kernel read-only data: 18432k
[ 2.120228] Freeing unused kernel memory: 1548K (ffff880001a7d000 - ffff880001c00000)
[ 2.121537] Freeing unused kernel memory: 1944K (ffff88000201a000 - ffff880002200000)
[ 2.174907] udevd[1207]: starting version 175
[ 4.324325] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
[ 8.933317] udevd[1429]: starting version 175
[ 15.570258] Adding 523260k swap on /dev/xvda2. Priority:-1 extents:1 across:523260k SS
[ 15.637450] EXT4-fs (xvda1): re-mounted. Opts: (null)
[ 15.896610] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro
[34856.970314] =============================================================================
[34856.970332] BUG kmalloc-2048 (Not tainted): Poison overwritten
[34856.970339] -----------------------------------------------------------------------------
[34856.970339]
[34856.970350] Disabling lock debugging due to kernel taint
[34856.970357] INFO: 0xffff88001b36099b-0xffff88001b36099b. First byte 0x6a instead of 0x6b
[34856.970371] INFO: Allocated in __alloc_skb+0x82/0x290 age=2 cpu=0 pid=2818
[34856.970381] __slab_alloc.isra.57+0x61e/0x6d6
[34856.970389] __kmalloc_node_track_caller+0xc9/0x1b0
[34856.970396] __kmalloc_reserve.isra.49+0x37/0xa0
[34856.970402] __alloc_skb+0x82/0x290
[34856.970408] sk_stream_alloc_skb+0x3f/0x120
[34856.970414] tcp_sendmsg+0x615/0xe20
[34856.970419] inet_sendmsg+0xa6/0x100
[34856.970426] sock_aio_write+0xc2/0xe0
[34856.970433] do_sync_readv_writev+0x4e/0x80
[34856.970439] do_readv_writev+0xe8/0x270
[34856.970444] vfs_writev+0x30/0x60
[34856.970449] SyS_writev+0x54/0xd0
[34856.970454] system_call_fastpath+0x16/0x1b
[34856.970461] INFO: Freed in skb_free_head+0x64/0x70 age=2 cpu=0 pid=0
[34856.970468] __slab_free+0x30/0x561
[34856.970474] kfree+0x1fd/0x210
[34856.970479] skb_free_head+0x64/0x70
[34856.970484] skb_release_data+0xf3/0x100
[34856.970492] skb_release_all+0x23/0x30
[34856.970497] __kfree_skb+0x11/0xa0
[34856.970503] tcp_ack+0x64a/0xf20
[34856.970509] tcp_rcv_established+0xe2/0x5c0
[34856.970515] tcp_v4_do_rcv+0x14e/0x4a0
[34856.970520] tcp_v4_rcv+0x8f2/0x9c0
[34856.970527] ip_local_deliver_finish+0xd8/0x1d0
[34856.970533] ip_local_deliver+0x34/0x90
[34856.970539] ip_rcv_finish+0x150/0x660
[34856.970544] ip_rcv+0x22b/0x370
[34856.970551] __netif_receive_skb_core+0x6e2/0x8b0
[34856.970557] __netif_receive_skb+0x1c/0x70
[34856.970563] INFO: Slab 0xffffea00006cd800 objects=13 used=13 fp=0x (null) flags=0x40000000004080
[34856.970572] INFO: Object 0xffff88001b360948 @offset=2376 fp=0xffff88001b3637b0
[34856.970572]
[34856.973176] Bytes b4 ffff88001b360938: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
[34856.973176] Object ffff88001b360948: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360958: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360968: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360978: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360988: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360998: 6b 6b 6b 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkjkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360aa8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ab8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ac8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ad8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ae8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360af8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ba8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360be8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bf8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ca8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ce8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cf8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360da8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360db8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360dc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360dd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360de8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360df8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ea8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360eb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ec8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ed8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ee8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ef8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fa8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fe8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ff8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361008: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361018: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361028: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361038: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361048: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361058: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361068: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361078: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361088: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361098: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361108: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361118: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361128: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361138: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk.
[34856.973176] Redzone ffff88001b361148: bb bb bb bb bb bb bb bb ........
[34856.973176] Padding ffff88001b361288: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
[34856.973176] CPU: 1 PID: 2818 Comm: glusterfs Tainted: G B 3.12.0-20131104 #1
[34856.973176] ffff88001b360948 ffff88001ad218e8 ffffffff81a6a8ba ffff88001eb36b40
[34856.973176] ffff88001e802d80 ffff88001ad21918 ffffffff8117af19 ffff88001b36099c
[34856.973176] ffff88001e802d80 000000000000006b ffff88001b360948 ffff88001ad21968
[34856.973176] Call Trace:
[34856.973176] [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[34856.973176] [<ffffffff8117af19>] print_trailer+0xf9/0x160
[34856.973176] [<ffffffff8117b0b2>] check_bytes_and_report+0xe2/0x120
[34856.973176] [<ffffffff8117b2b7>] check_object+0x1c7/0x240
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff81a66cc2>] alloc_debug_processing+0x153/0x168
[34856.973176] [<ffffffff81a672f5>] __slab_alloc.isra.57+0x61e/0x6d6
[34856.973176] [<ffffffff810ef7bb>] ? trace_hardirqs_on_caller+0xfb/0x240
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff810ef624>] ? mark_held_locks+0x94/0x130
[34856.973176] [<ffffffff8117ef49>] __kmalloc_node_track_caller+0xc9/0x1b0
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff818ff9d7>] __kmalloc_reserve.isra.49+0x37/0xa0
[34856.973176] [<ffffffff818ffa92>] ? __alloc_skb+0x52/0x290
[34856.973176] [<ffffffff818ffac2>] __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff810f372a>] ? lock_release+0x12a/0x250
[34856.973176] [<ffffffff819af1af>] sk_stream_alloc_skb+0x3f/0x120
[34856.973176] [<ffffffff819aff85>] tcp_sendmsg+0x615/0xe20
[34856.973176] [<ffffffff819d7e16>] inet_sendmsg+0xa6/0x100
[34856.973176] [<ffffffff819d7d70>] ? inet_autobind+0x60/0x60
[34856.973176] [<ffffffff818f57b2>] sock_aio_write+0xc2/0xe0
[34856.973176] [<ffffffff811899be>] do_sync_readv_writev+0x4e/0x80
[34856.973176] [<ffffffff8118b0e8>] do_readv_writev+0xe8/0x270
[34856.973176] [<ffffffff810f372a>] ? lock_release+0x12a/0x250
[34856.973176] [<ffffffff8118b2a0>] vfs_writev+0x30/0x60
[34856.973176] [<ffffffff8118b444>] SyS_writev+0x54/0xd0
[34856.973176] [<ffffffff81a76379>] system_call_fastpath+0x16/0x1b
[34856.973176] FIX kmalloc-2048: Restoring 0xffff88001b36099b-0xffff88001b36099b=0x6b
[34856.973176]
[34856.973176] FIX kmalloc-2048: Marking all objects used
[81155.636434] =============================================================================
[81155.636459] BUG skbuff_head_cache (Tainted: G B ): Poison overwritten
[81155.636469] -----------------------------------------------------------------------------
[81155.636469]
[81155.636481] INFO: 0xffff88001e1e7b1b-0xffff88001e1e7b1b. First byte 0x6a instead of 0x6b
[81155.636495] INFO: Allocated in build_skb+0x2f/0x1c0 age=3 cpu=0 pid=0
[81155.636506] __slab_alloc.isra.57+0x61e/0x6d6
[81155.636513] kmem_cache_alloc+0x10a/0x140
[81155.636519] build_skb+0x2f/0x1c0
[81155.636524] __netdev_alloc_skb+0x91/0xf0
[81155.636532] xennet_alloc_rx_buffers+0xb6/0x340
[81155.636538] xennet_poll+0x6c7/0xdb0
[81155.636545] net_rx_action+0x159/0x250
[81155.636551] __do_softirq+0xea/0x1e0
[81155.636557] call_softirq+0x1c/0x30
[81155.636564] do_softirq+0x95/0x100
[81155.636569] irq_exit+0xa2/0xd0
[81155.636576] xen_evtchn_do_upcall+0x2f/0x40
[81155.636582] xen_do_hypervisor_callback+0x1e/0x30
[81155.636595] INFO: Freed in __kfree_skb+0x3a/0xa0 age=2 cpu=0 pid=0
[81155.636603] __slab_free+0x30/0x561
[81155.636608] kmem_cache_free+0x212/0x230
[81155.636614] __kfree_skb+0x3a/0xa0
[81155.636620] tcp_rcv_established+0x33b/0x5c0
[81155.636627] tcp_v4_do_rcv+0x14e/0x4a0
[81155.636633] tcp_v4_rcv+0x8f2/0x9c0
[81155.636639] ip_local_deliver_finish+0xd8/0x1d0
[81155.636647] ip_local_deliver+0x34/0x90
[81155.636653] ip_rcv_finish+0x150/0x660
[81155.636659] ip_rcv+0x22b/0x370
[81155.636664] __netif_receive_skb_core+0x6e2/0x8b0
[81155.636670] __netif_receive_skb+0x1c/0x70
[81155.636676] netif_receive_skb+0x1e/0x100
[81155.636681] xennet_poll+0x8a4/0xdb0
[81155.636687] net_rx_action+0x159/0x250
[81155.636692] __do_softirq+0xea/0x1e0
[81155.636698] INFO: Slab 0xffffea0000787980 objects=14 used=14 fp=0x (null) flags=0x40000000004080
[81155.636707] INFO: Object 0xffff88001e1e7b00 @offset=6912 fp=0xffff88001e1e6b40
[81155.636707]
[81155.636719] Bytes b4 ffff88001e1e7af0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
[81155.636728] Object ffff88001e1e7b00: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636738] Object ffff88001e1e7b10: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6a 6b 6b 6b 6b kkkkkkkkkkkjkkkk
[81155.636748] Object ffff88001e1e7b20: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636757] Object ffff88001e1e7b30: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636766] Object ffff88001e1e7b40: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636775] Object ffff88001e1e7b50: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636785] Object ffff88001e1e7b60: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636794] Object ffff88001e1e7b70: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636803] Object ffff88001e1e7b80: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636812] Object ffff88001e1e7b90: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636821] Object ffff88001e1e7ba0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636830] Object ffff88001e1e7bb0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636839] Object ffff88001e1e7bc0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636848] Object ffff88001e1e7bd0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636857] Object ffff88001e1e7be0: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
[81155.636869] Redzone ffff88001e1e7be8: bb bb bb bb bb bb bb bb ........
[81155.636881] Padding ffff88001e1e7d28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
[81155.636890] Padding ffff88001e1e7d38: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
[81155.636912] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G B 3.12.0-20131104 #1
[81155.636914] ffff88001e1e7b00 ffff88001fc039d0 ffffffff81a6a8ba ffffffff822134e0
[81155.636920] ffff88001e93e6c0 ffff88001fc03a00 ffffffff8117af19 ffff88001e1e7b1c
[81155.636924] ffff88001e93e6c0 000000000000006b ffff88001e1e7b00 ffff88001fc03a50
[81155.636928] Call Trace:
[81155.636930] <IRQ> [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[81155.636936] [<ffffffff8117af19>] print_trailer+0xf9/0x160
[81155.636940] [<ffffffff8117b0b2>] check_bytes_and_report+0xe2/0x120
[81155.636943] [<ffffffff8117b2b7>] check_object+0x1c7/0x240
[81155.636947] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636950] [<ffffffff81a66cc2>] alloc_debug_processing+0x153/0x168
[81155.636954] [<ffffffff81a672f5>] __slab_alloc.isra.57+0x61e/0x6d6
[81155.636957] [<ffffffff819a058d>] ? ip_local_deliver_finish+0x17d/0x1d0
[81155.636961] [<ffffffff819a044c>] ? ip_local_deliver_finish+0x3c/0x1d0
[81155.636964] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636968] [<ffffffff819a0754>] ? ip_local_deliver+0x34/0x90
[81155.636971] [<ffffffff8199ff00>] ? ip_rcv_finish+0x150/0x660
[81155.636975] [<ffffffff8117ce7a>] kmem_cache_alloc+0x10a/0x140
[81155.636978] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636981] [<ffffffff818fe78f>] build_skb+0x2f/0x1c0
[81155.636985] [<ffffffff819004b1>] __netdev_alloc_skb+0x91/0xf0
[81155.636988] [<ffffffff816d50a6>] xennet_alloc_rx_buffers+0xb6/0x340
[81155.636991] [<ffffffff8190d1be>] ? netif_receive_skb+0x1e/0x100
[81155.636995] [<ffffffff816d6fb7>] xennet_poll+0x6c7/0xdb0
[81155.636999] [<ffffffff8190d669>] net_rx_action+0x159/0x250
[81155.637003] [<ffffffff8109fa3a>] __do_softirq+0xea/0x1e0
[81155.637006] [<ffffffff81a777bc>] call_softirq+0x1c/0x30
[81155.637009] [<ffffffff81010dc5>] do_softirq+0x95/0x100
[81155.637013] [<ffffffff8109fce2>] irq_exit+0xa2/0xd0
[81155.637016] [<ffffffff814bb2af>] xen_evtchn_do_upcall+0x2f/0x40
[81155.637019] [<ffffffff81a7781e>] xen_do_hypervisor_callback+0x1e/0x30
[81155.637021] <EOI> [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[81155.637027] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[81155.637031] [<ffffffff81008da0>] ? xen_safe_halt+0x10/0x20
[81155.637034] [<ffffffff810185cd>] ? default_idle+0x1d/0x30
[81155.637038] [<ffffffff81018f0e>] ? arch_cpu_idle+0x2e/0x40
[81155.637042] [<ffffffff810db410>] ? cpu_startup_entry+0x80/0x1d0
[81155.637046] [<ffffffff81a5ebf1>] ? rest_init+0xc1/0xd0
[81155.637049] [<ffffffff81a5eb30>] ? csum_partial_copy_generic+0x170/0x170
[81155.637054] [<ffffffff8230bee6>] ? start_kernel+0x3d3/0x3e0
[81155.637058] [<ffffffff8230b912>] ? repair_env_string+0x5e/0x5e
[81155.637061] [<ffffffff810d89f7>] ? __add_preferred_console.constprop.18+0x87/0xb0
[81155.637065] [<ffffffff8230b5f8>] ? x86_64_start_reservations+0x2a/0x2c
[81155.637069] [<ffffffff8230ecd9>] ? xen_start_kernel+0x563/0x56d
[81155.637072] FIX skbuff_head_cache: Restoring 0xffff88001e1e7b1b-0xffff88001e1e7b1b=0x6b
[81155.637072]
[81155.637084] FIX skbuff_head_cache: Marking all objects used
[81234.562076] BUG: unable to handle kernel paging request at ffffffff80014120
[81234.562131] IP: [<ffffffff80014120>] 0xffffffff8001411f
[81234.562159] PGD 220f067 PUD 2210067 PMD 0
[81234.562186] Oops: 0010 [#1] PREEMPT SMP
[81234.562192] Modules linked in:
[81234.562198] CPU: 1 PID: 7992 Comm: smbd Tainted: G B 3.12.0-20131104 #1
[81234.562206] task: ffff88001d934780 ti: ffff88000eb28000 task.ti: ffff88000eb28000
[81234.562212] RIP: e030:[<ffffffff80014120>] [<ffffffff80014120>] 0xffffffff8001411f
[81234.562219] RSP: e02b:ffff88000eb29e10 EFLAGS: 00010246
[81234.562224] RAX: ffffffff80014120 RBX: fffffffffffffff4 RCX: 0000000000000000
[81234.562230] RDX: 000000000007e000 RSI: 0000000000000000 RDI: 0000000000000000
[81234.562235] RBP: ffff88000eb29e28 R08: 0000000000000022 R09: 00007ffffffff000
[81234.562241] R10: 000000000000007e R11: fffffffffffffff4 R12: 000000000007e000
[81234.562246] R13: 0000000000000022 R14: ffff88001b25cac0 R15: 000000000007e000
[81234.562254] FS: 00007fdc5cc42720(0000) GS:ffff88001fd00000(0000) knlGS:0000000000000000
[81234.562260] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[81234.562265] CR2: ffffffff80014120 CR3: 0000000004d1d000 CR4: 0000000000000660
[81234.562271] Stack:
[81234.562274] ffffffff8115ce9d 0000000000000003 0000000000000000 ffff88000eb29e88
[81234.562283] ffffffff8115fb4a 0000000000000000 0000000000000000 000000000000007e
[81234.562292] 0000000000000000 ffff88000eb29e88 ffff88001b25cb50 0000000000000000
[81234.562300] Call Trace:
[81234.562308] [<ffffffff8115ce9d>] ? get_unmapped_area+0x9d/0x150
[81234.562314] [<ffffffff8115fb4a>] do_mmap_pgoff+0x10a/0x3e0
[81234.562321] [<ffffffff8114ba10>] vm_mmap_pgoff+0x70/0xb0
[81234.562326] [<ffffffff8115e38c>] SyS_mmap_pgoff+0x1cc/0x260
[81234.562334] [<ffffffff814209ee>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[81234.562342] [<ffffffff81013f1d>] SyS_mmap+0x1d/0x20
[81234.562349] [<ffffffff81a76379>] system_call_fastpath+0x16/0x1b
[81234.562354] Code: Bad RIP value.
[81234.562363] RIP [<ffffffff80014120>] 0xffffffff8001411f
[81234.562368] RSP <ffff88000eb29e10>
[81234.562372] CR2: ffffffff80014120
[81234.562379] ---[ end trace 4d2d192c2478dd2a ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210
2013-11-09 18:18 ` Sander Eikelenboom
@ 2013-11-11 14:57 ` Christoph Lameter
0 siblings, 0 replies; 5+ messages in thread
From: Christoph Lameter @ 2013-11-11 14:57 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: Pekka Enberg, linux-kernel, netdev, eric.dumazet
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2466 bytes --]
On Sat, 9 Nov 2013, Sander Eikelenboom wrote:
> This VM got quite some I/O and network load, it got a lot of large files
> transferred to Samba and the files are stored on a glusterfs volume (by
> tcp/ip).
That looks like its related to networking. CCing netdev.
>
>
> [34856.970314] =============================================================================
> [34856.970332] BUG kmalloc-2048 (Not tainted): Poison overwritten
> [34856.970339] -----------------------------------------------------------------------------
> [34856.970339]
> [34856.970350] Disabling lock debugging due to kernel taint
> [34856.970357] INFO: 0xffff88001b36099b-0xffff88001b36099b. First byte 0x6a instead of 0x6b
> [34856.970371] INFO: Allocated in __alloc_skb+0x82/0x290 age=2 cpu=0 pid=2818
> [34856.970381] __slab_alloc.isra.57+0x61e/0x6d6
> [34856.970389] __kmalloc_node_track_caller+0xc9/0x1b0
> [34856.970396] __kmalloc_reserve.isra.49+0x37/0xa0
> [34856.970402] __alloc_skb+0x82/0x290
> [34856.970408] sk_stream_alloc_skb+0x3f/0x120
> [34856.970414] tcp_sendmsg+0x615/0xe20
> [34856.970419] inet_sendmsg+0xa6/0x100
> [34856.970426] sock_aio_write+0xc2/0xe0
> [34856.970433] do_sync_readv_writev+0x4e/0x80
> [34856.970439] do_readv_writev+0xe8/0x270
> [34856.970444] vfs_writev+0x30/0x60
> [34856.970449] SyS_writev+0x54/0xd0
> [34856.970454] system_call_fastpath+0x16/0x1b
> [34856.970461] INFO: Freed in skb_free_head+0x64/0x70 age=2 cpu=0 pid=0
> [34856.970468] __slab_free+0x30/0x561
> [34856.970474] kfree+0x1fd/0x210
> [34856.970479] skb_free_head+0x64/0x70
> [34856.970484] skb_release_data+0xf3/0x100
> [34856.970492] skb_release_all+0x23/0x30
> [34856.970497] __kfree_skb+0x11/0xa0
> [34856.970503] tcp_ack+0x64a/0xf20
> [34856.970509] tcp_rcv_established+0xe2/0x5c0
> [34856.970515] tcp_v4_do_rcv+0x14e/0x4a0
> [34856.970520] tcp_v4_rcv+0x8f2/0x9c0
> [34856.970527] ip_local_deliver_finish+0xd8/0x1d0
> [34856.970533] ip_local_deliver+0x34/0x90
> [34856.970539] ip_rcv_finish+0x150/0x660
> [34856.970544] ip_rcv+0x22b/0x370
> [34856.970551] __netif_receive_skb_core+0x6e2/0x8b0
> [34856.970557] __netif_receive_skb+0x1c/0x70
> [34856.970563] INFO: Slab 0xffffea00006cd800 objects=13 used=13 fp=0x (null) flags=0x40000000004080
> [34856.970572] INFO: Object 0xffff88001b360948 @offset=2376 fp=0xffff88001b3637b0
> [34856.970572]
>
> The rest is in the attached dmesg.
>
> attached: dmesg-oops-3.12.0-20131104.txt
> --
> Sander
[-- Attachment #2: Type: TEXT/PLAIN, Size: 47098 bytes --]
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.12.0-20131104 (root@serveerstertje) (gcc version 4.7.2 (Debian 4.7.2-5) ) #1 SMP PREEMPT Mon Nov 4 09:34:18 CET 2013
[ 0.000000] Command line: root=/dev/xvda1 ro slub_debug
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x000000001fffffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] MPS support code is not built-in.
[ 0.000000] Using acpi=off or acpi=noirq or pci=noacpi may have problem
[ 0.000000] DMI not present or invalid.
[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000000] No AGP bridge found
[ 0.000000] e820: last_pfn = 0x20000 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] Base memory trampoline at [ffff88000009a000] 9a000 size 24576
[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[ 0.000000] [mem 0x00000000-0x000fffff] page 4k
[ 0.000000] init_memory_mapping: [mem 0x1fe00000-0x1fffffff]
[ 0.000000] [mem 0x1fe00000-0x1fffffff] page 4k
[ 0.000000] BRK [0x02d72000, 0x02d72fff] PGTABLE
[ 0.000000] init_memory_mapping: [mem 0x1c000000-0x1fdfffff]
[ 0.000000] [mem 0x1c000000-0x1fdfffff] page 4k
[ 0.000000] BRK [0x02d73000, 0x02d73fff] PGTABLE
[ 0.000000] BRK [0x02d74000, 0x02d74fff] PGTABLE
[ 0.000000] BRK [0x02d75000, 0x02d75fff] PGTABLE
[ 0.000000] BRK [0x02d76000, 0x02d76fff] PGTABLE
[ 0.000000] BRK [0x02d77000, 0x02d77fff] PGTABLE
[ 0.000000] init_memory_mapping: [mem 0x00100000-0x1bffffff]
[ 0.000000] [mem 0x00100000-0x1bffffff] page 4k
[ 0.000000] RAMDISK: [mem 0x03188000-0x03f9cfff]
[ 0.000000] NUMA turned off
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x000000001fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x1fffffff]
[ 0.000000] NODE_DATA [mem 0x1ff1d000-0x1ff27fff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00001000-0x0009ffff]
[ 0.000000] node 0: [mem 0x00100000-0x1fffffff]
[ 0.000000] On node 0 totalpages: 130975
[ 0.000000] DMA zone: 64 pages used for memmap
[ 0.000000] DMA zone: 21 pages reserved
[ 0.000000] DMA zone: 3999 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 1984 pages used for memmap
[ 0.000000] DMA32 zone: 126976 pages, LIFO batch:31
[ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] nr_irqs_gsi: 16
[ 0.000000] e820: [mem 0x20000000-0xffffffff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.4-unstable (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:2 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff88001fc00000 s83200 r8192 d23296 u1048576
[ 0.000000] pcpu-alloc: s83200 r8192 d23296 u1048576 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 0 1
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 128906
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/xvda1 ro slub_debug
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Memory: 468620K/523900K available (10729K kernel code, 978K rwdata, 4200K rodata, 1068K init, 9580K bss, 55280K reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] RCU dyntick-idle grace-period acceleration is enabled.
[ 0.000000] Additional per-CPU info printed with stalls.
[ 0.000000] RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=2.
[ 0.000000] NR_IRQS:4352 nr_irqs:288 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[ 0.000000] ... MAX_LOCKDEP_SUBCLASSES: 8
[ 0.000000] ... MAX_LOCK_DEPTH: 48
[ 0.000000] ... MAX_LOCKDEP_KEYS: 8191
[ 0.000000] ... CLASSHASH_SIZE: 4096
[ 0.000000] ... MAX_LOCKDEP_ENTRIES: 16384
[ 0.000000] ... MAX_LOCKDEP_CHAINS: 32768
[ 0.000000] ... CHAINHASH_SIZE: 16384
[ 0.000000] memory used by lock dependency info: 5855 kB
[ 0.000000] per task-struct memory footprint: 1920 bytes
[ 0.000000] kmemleak: Kernel memory leak detector disabled
[ 0.000000] Xen: using vcpuop timer interface
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] tsc: Detected 3200.180 MHz processor
[ 0.000000] tsc: Marking TSC unstable due to TSCs unsynchronized
[ 0.003333] Calibrating delay loop (skipped), value calculated using timer frequency.. 6402.04 BogoMIPS (lpj=10667266)
[ 0.003333] pid_max: default: 32768 minimum: 301
[ 0.003333] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.003333] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.003333] Mount-cache hash table entries: 256
[ 0.003333] Initializing cgroup subsys freezer
[ 0.003333] Initializing cgroup subsys blkio
[ 0.003333] tseg: 0000000000
[ 0.003333] CPU: Physical Processor ID: 0
[ 0.003333] CPU: Processor Core ID: 1
[ 0.003333] Last level iTLB entries: 4KB 512, 2MB 16, 4MB 8
[ 0.003333] Last level dTLB entries: 4KB 512, 2MB 128, 4MB 64
[ 0.003333] tlb_flushall_shift: 4
[ 0.044828] cpu 0 spinlock event irq 17
[ 0.045830] Performance Events:
[ 0.045848] no APIC, boot with the "lapic" boot parameter to force-enable it.
[ 0.045855] no hardware sampling interrupt available.
[ 0.045880] Broken PMU hardware detected, using software events only.
[ 0.045882] Failed to access perfctr msr (MSR c0010004 is 0)
[ 0.063598] NMI watchdog: disabled (cpu0): hardware events not enabled
[ 0.070288] installing Xen timer for CPU 1
[ 0.070377] cpu 1 spinlock event irq 24
[ 0.070456] SMP alternatives: lockdep: fixing up alternatives
[ 0.070469] SMP alternatives: switching to SMP code
[ 0.112838] Brought up 2 CPUs
[ 0.113810] devtmpfs: initialized
[ 0.115040] xor: measuring software checksum speed
[ 0.144411] prefetch64-sse: 5412.000 MB/sec
[ 0.178307] generic_sse: 5330.400 MB/sec
[ 0.178318] xor: using function: prefetch64-sse (5412.000 MB/sec)
[ 0.178361] xen:grant_table: Grant tables using version 2 layout
[ 0.178420] Grant table initialized
[ 0.178917] NET: Registered protocol family 16
[ 0.183776] PCI: setting up Xen PCI frontend stub
[ 0.183800] PCI: pci_cache_line_size set to 64 bytes
[ 0.320441] bio: create slab <bio-0> at 0
[ 0.376754] raid6: sse2x1 3860 MB/s
[ 0.433368] raid6: sse2x2 5362 MB/s
[ 0.490003] raid6: sse2x4 5857 MB/s
[ 0.490005] raid6: using algorithm sse2x4 (5857 MB/s)
[ 0.490006] raid6: using intx1 recovery algorithm
[ 0.490306] ACPI: Interpreter disabled.
[ 0.490306] xen:balloon: Initialising balloon driver
[ 0.493520] xen_balloon: Initialising balloon driver
[ 0.496781] vgaarb: loaded
[ 0.497942] SCSI subsystem initialized
[ 0.497942] libata version 3.00 loaded.
[ 0.500538] usbcore: registered new interface driver usbfs
[ 0.500538] usbcore: registered new interface driver hub
[ 0.500538] usbcore: registered new device driver usb
[ 0.500538] Linux video capture interface: v2.00
[ 0.503529] pps_core: LinuxPPS API ver. 1 registered
[ 0.503542] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.503612] PTP clock support registered
[ 0.503625] Advanced Linux Sound Architecture Driver Initialized.
[ 0.503648] PCI: System does not support PCI
[ 0.503650] PCI: System does not support PCI
[ 0.506907] Bluetooth: Core ver 2.16
[ 0.506997] NET: Registered protocol family 31
[ 0.506997] Bluetooth: HCI device and connection manager initialized
[ 0.506997] Bluetooth: HCI socket layer initialized
[ 0.506997] Bluetooth: L2CAP socket layer initialized
[ 0.506997] Bluetooth: SCO socket layer initialized
[ 0.507279] Switched to clocksource xen
[ 0.507396] FS-Cache: Loaded
[ 0.507636] pnp: PnP ACPI: disabled
[ 0.539117] NET: Registered protocol family 2
[ 0.539897] TCP established hash table entries: 4096 (order: 4, 65536 bytes)
[ 0.540019] TCP bind hash table entries: 4096 (order: 6, 262144 bytes)
[ 0.540277] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.540362] TCP: reno registered
[ 0.540401] UDP hash table entries: 256 (order: 3, 40960 bytes)
[ 0.540458] UDP-Lite hash table entries: 256 (order: 3, 40960 bytes)
[ 0.540871] NET: Registered protocol family 1
[ 0.540925] PCI: CLS 0 bytes, default 64
[ 0.541330] Trying to unpack rootfs image as initramfs...
[ 0.604671] Freeing initrd memory: 14420K (ffff880003188000 - ffff880003f9d000)
[ 0.810210] DMA-API: preallocated 65536 debug entries
[ 0.810232] DMA-API: debugging enabled by kernel config
[ 0.810899] platform rtc_cmos: registered platform RTC device (no PNP device found)
[ 0.813974] Scanning for low memory corruption every 60 seconds
[ 0.817883] sha1_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817897] sha256_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817902] sha512_ssse3: Neither AVX nor SSSE3 is available/usable.
[ 0.817908] AVX or AES-NI instructions are not detected.
[ 0.817912] AVX instructions are not detected.
[ 0.817916] AVX instructions are not detected.
[ 0.820512] audit: initializing netlink socket (disabled)
[ 0.820601] type=2000 audit(1383778218.007:1): initialized
[ 0.822275] bounce pool size: 64 pages
[ 0.822293] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.878200] VFS: Disk quotas dquot_6.5.2
[ 0.878818] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.897415] NTFS driver 2.1.30 [Flags: R/W].
[ 0.900483] fuse init (API version 7.22)
[ 0.905703] bio: create slab <bio-1> at 1
[ 0.910323] Btrfs loaded
[ 0.914633] GFS2 installed
[ 0.916780] ceph: loaded (mds proto 32)
[ 0.916827] msgmni has been set to 943
[ 0.920847] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[ 0.920891] io scheduler noop registered
[ 0.920896] io scheduler deadline registered
[ 0.921766] io scheduler cfq registered (default)
[ 0.923034] crc32: CRC_LE_BITS = 64, CRC_BE BITS = 64
[ 0.923046] crc32: self tests passed, processed 225944 bytes in 109410 nsec
[ 0.923163] crc32c: CRC_LE_BITS = 64
[ 0.923168] crc32c: self tests passed, processed 225944 bytes in 54579 nsec
[ 0.924335] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.925132] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[ 0.925447] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.925456] acpiphp_ibm: ibm_acpiphp_init: acpi_walk_namespace failed
[ 0.926534] usbcore: registered new interface driver udlfb
[ 0.927691] xen:xen_evtchn: Event-channel device installed
[ 0.932426] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.936695] Linux agpgart interface v0.103
[ 0.938516] Hangcheck: starting hangcheck timer 0.9.1 (tick is 180 seconds, margin is 60 seconds).
[ 0.938518] Hangcheck: Using getrawmonotonic().
[ 0.938784] [drm] Initialized drm 1.1.0 20060810
[ 0.939112] [drm] radeon kernel modesetting enabled.
[ 0.957684] brd: module loaded
[ 0.966412] loop: module loaded
[ 0.975940] tun: Universal TUN/TAP device driver, 1.6
[ 0.975955] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[ 0.976391] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[ 0.976403] e1000: Copyright (c) 1999-2006 Intel Corporation.
[ 0.976818] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
[ 0.976831] e1000e: Copyright(c) 1999 - 2013 Intel Corporation.
[ 0.977343] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
[ 0.977353] igb: Copyright (c) 2007-2013 Intel Corporation.
[ 0.977696] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
[ 0.977709] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[ 0.978502] xen_netfront: Initialising Xen virtual ethernet driver
[ 0.983343] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.983353] ehci-pci: EHCI PCI platform driver
[ 0.984249] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.984258] ohci-pci: OHCI PCI platform driver
[ 0.984564] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.986205] usbcore: registered new interface driver usblp
[ 0.986958] usbcore: registered new interface driver usb-storage
[ 0.987932] usbcore: registered new interface driver usbserial
[ 0.988274] usbcore: registered new interface driver cp210x
[ 0.988703] usbserial: USB Serial support registered for cp210x
[ 0.988952] usbcore: registered new interface driver cypress_m8
[ 0.989238] usbserial: USB Serial support registered for DeLorme Earthmate USB
[ 0.989573] usbserial: USB Serial support registered for HID->COM RS232 Adapter
[ 0.989870] usbserial: USB Serial support registered for Nokia CA-42 V2 Adapter
[ 0.990111] usbcore: registered new interface driver mos7720
[ 0.990417] usbserial: USB Serial support registered for Moschip 2 port adapter
[ 0.990654] usbcore: registered new interface driver mos7840
[ 0.993506] usbserial: USB Serial support registered for Moschip 7840/7820 USB Serial Driver
[ 0.994349] i8042: PNP: No PS/2 controller found. Probing ports directly.
[ 1.009583] blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[ 2.002564] i8042: No controller found
[ 2.004200] mousedev: PS/2 mouse device common for all mice
[ 2.066703] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 2.068031] rtc_cmos: probe of rtc_cmos failed with error -38
[ 2.071810] lirc_dev: IR Remote Control driver registered, major 248
[ 2.071854] IR NEC protocol handler initialized
[ 2.071860] IR RC5(x) protocol handler initialized
[ 2.071864] IR RC6 protocol handler initialized
[ 2.071868] IR JVC protocol handler initialized
[ 2.071873] IR Sony protocol handler initialized
[ 2.071877] IR RC5 (streamzap) protocol handler initialized
[ 2.071882] IR SANYO protocol handler initialized
[ 2.071886] IR MCE Keyboard/mouse protocol handler initialized
[ 2.071891] IR LIRC bridge handler initialized
[ 2.071896] cx25821: driver version 0.0.106 loaded
[ 2.072860] usbcore: registered new interface driver pvrusb2
[ 2.072890] pvrusb2: V4L in-tree version:Hauppauge WinTV-PVR-USB2 MPEG2 Encoder/Tuner
[ 2.072897] pvrusb2: Debug mask is 31 (0x1f)
[ 2.073903] sp5100_tco: SP5100/SB800 TCO WatchDog Timer Driver v0.05
[ 2.074490] xen_wdt: Xen WatchDog Timer Driver v0.01
[ 2.075432] xen_wdt: initialized (timeout=60s, nowayout=0)
[ 2.079512] device-mapper: ioctl: 4.26.0-ioctl (2013-08-15) initialised: dm-devel@redhat.com
[ 2.082630] device-mapper: cache-policy-mq: version 1.0.0 loaded
[ 2.082660] device-mapper: cache cleaner: version 1.0.0 loaded
[ 2.082666] Bluetooth: Virtual HCI driver ver 1.3
[ 2.083086] Bluetooth: HCI UART driver ver 2.2
[ 2.083098] Bluetooth: HCI H4 protocol initialized
[ 2.083103] Bluetooth: HCI BCSP protocol initialized
[ 2.083107] Bluetooth: HCILL protocol initialized
[ 2.083111] Bluetooth: HCIATH3K protocol initialized
[ 2.083115] Bluetooth: HCI Three-wire UART (H5) protocol initialized
[ 2.083439] usbcore: registered new interface driver bcm203x
[ 2.084033] usbcore: registered new interface driver bpa10x
[ 2.084374] usbcore: registered new interface driver bfusb
[ 2.084716] usbcore: registered new interface driver btusb
[ 2.085210] usbcore: registered new interface driver ath3k
[ 2.085814] hidraw: raw HID events driver (C) Jiri Kosina
[ 2.088974] usbcore: registered new interface driver usbhid
[ 2.088983] usbhid: USB HID core driver
[ 2.097677] usbcore: registered new interface driver snd-usb-audio
[ 2.097975] usbcore: registered new interface driver snd-ua101
[ 2.098252] usbcore: registered new interface driver snd-usb-usx2y
[ 2.098526] usbcore: registered new interface driver snd-usb-caiaq
[ 2.098797] usbcore: registered new interface driver snd-usb-6fire
[ 2.098882] Netfilter messages via NETLINK v0.30.
[ 2.098913] nfnl_acct: registering with nfnetlink.
[ 2.099088] nf_conntrack version 0.5.0 (3773 buckets, 15092 max)
[ 2.100344] ctnetlink v0.93: registering with nfnetlink.
[ 2.101476] xt_time: kernel timezone is -0000
[ 2.101498] xvda: xvda1 xvda2
[ 2.101509] ip_set: protocol 6
[ 2.101550] IPVS: Registered protocols ()
[ 2.102027] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[ 2.102231] IPVS: Creating netns size=1904 id=0
[ 2.102595] IPVS: ipvs loaded.
[ 2.103069] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 2.103473] TCP: cubic registered
[ 2.103480] NET: Registered protocol family 17
[ 2.104492] Bridge firewalling registered
[ 2.104524] Ebtables v2.0 registered
[ 2.105453] Bluetooth: RFCOMM TTY layer initialized
[ 2.105561] Bluetooth: RFCOMM socket layer initialized
[ 2.105567] Bluetooth: RFCOMM ver 1.11
[ 2.105571] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 2.105576] Bluetooth: BNEP filters: protocol multicast
[ 2.105598] Bluetooth: BNEP socket layer initialized
[ 2.105603] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[ 2.105626] Bluetooth: HIDP socket layer initialized
[ 2.105668] Key type ceph registered
[ 2.107631] libceph: loaded (mon/osd proto 15/24)
[ 2.109119] registered taskstats version 1
[ 2.109623] console [netcon0] enabled
[ 2.109634] netconsole: network logging started
[ 2.109640] drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 2.109837] ALSA device list:
[ 2.109844] No soundcards found.
[ 2.111258] Freeing unused kernel memory: 1068K (ffffffff822f6000 - ffffffff82401000)
[ 2.111275] Write protecting the kernel read-only data: 18432k
[ 2.120228] Freeing unused kernel memory: 1548K (ffff880001a7d000 - ffff880001c00000)
[ 2.121537] Freeing unused kernel memory: 1944K (ffff88000201a000 - ffff880002200000)
[ 2.174907] udevd[1207]: starting version 175
[ 4.324325] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
[ 8.933317] udevd[1429]: starting version 175
[ 15.570258] Adding 523260k swap on /dev/xvda2. Priority:-1 extents:1 across:523260k SS
[ 15.637450] EXT4-fs (xvda1): re-mounted. Opts: (null)
[ 15.896610] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro
[34856.970314] =============================================================================
[34856.970332] BUG kmalloc-2048 (Not tainted): Poison overwritten
[34856.970339] -----------------------------------------------------------------------------
[34856.970339]
[34856.970350] Disabling lock debugging due to kernel taint
[34856.970357] INFO: 0xffff88001b36099b-0xffff88001b36099b. First byte 0x6a instead of 0x6b
[34856.970371] INFO: Allocated in __alloc_skb+0x82/0x290 age=2 cpu=0 pid=2818
[34856.970381] __slab_alloc.isra.57+0x61e/0x6d6
[34856.970389] __kmalloc_node_track_caller+0xc9/0x1b0
[34856.970396] __kmalloc_reserve.isra.49+0x37/0xa0
[34856.970402] __alloc_skb+0x82/0x290
[34856.970408] sk_stream_alloc_skb+0x3f/0x120
[34856.970414] tcp_sendmsg+0x615/0xe20
[34856.970419] inet_sendmsg+0xa6/0x100
[34856.970426] sock_aio_write+0xc2/0xe0
[34856.970433] do_sync_readv_writev+0x4e/0x80
[34856.970439] do_readv_writev+0xe8/0x270
[34856.970444] vfs_writev+0x30/0x60
[34856.970449] SyS_writev+0x54/0xd0
[34856.970454] system_call_fastpath+0x16/0x1b
[34856.970461] INFO: Freed in skb_free_head+0x64/0x70 age=2 cpu=0 pid=0
[34856.970468] __slab_free+0x30/0x561
[34856.970474] kfree+0x1fd/0x210
[34856.970479] skb_free_head+0x64/0x70
[34856.970484] skb_release_data+0xf3/0x100
[34856.970492] skb_release_all+0x23/0x30
[34856.970497] __kfree_skb+0x11/0xa0
[34856.970503] tcp_ack+0x64a/0xf20
[34856.970509] tcp_rcv_established+0xe2/0x5c0
[34856.970515] tcp_v4_do_rcv+0x14e/0x4a0
[34856.970520] tcp_v4_rcv+0x8f2/0x9c0
[34856.970527] ip_local_deliver_finish+0xd8/0x1d0
[34856.970533] ip_local_deliver+0x34/0x90
[34856.970539] ip_rcv_finish+0x150/0x660
[34856.970544] ip_rcv+0x22b/0x370
[34856.970551] __netif_receive_skb_core+0x6e2/0x8b0
[34856.970557] __netif_receive_skb+0x1c/0x70
[34856.970563] INFO: Slab 0xffffea00006cd800 objects=13 used=13 fp=0x (null) flags=0x40000000004080
[34856.970572] INFO: Object 0xffff88001b360948 @offset=2376 fp=0xffff88001b3637b0
[34856.970572]
[34856.973176] Bytes b4 ffff88001b360938: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ
[34856.973176] Object ffff88001b360948: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360958: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360968: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360978: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360988: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360998: 6b 6b 6b 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkjkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3609f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360a98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360aa8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ab8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ac8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ad8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ae8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360af8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360b98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ba8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360be8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360bf8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360c98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ca8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ce8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360cf8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360d98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360da8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360db8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360dc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360dd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360de8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360df8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360e98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ea8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360eb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ec8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ed8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ee8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ef8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f08: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f18: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f28: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f38: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f48: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f58: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f68: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f78: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f88: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360f98: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fa8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fb8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fc8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fd8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360fe8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b360ff8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361008: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361018: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361028: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361038: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361048: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361058: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361068: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361078: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361088: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361098: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b3610f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361108: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361118: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361128: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[34856.973176] Object ffff88001b361138: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk.
[34856.973176] Redzone ffff88001b361148: bb bb bb bb bb bb bb bb ........
[34856.973176] Padding ffff88001b361288: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
[34856.973176] CPU: 1 PID: 2818 Comm: glusterfs Tainted: G B 3.12.0-20131104 #1
[34856.973176] ffff88001b360948 ffff88001ad218e8 ffffffff81a6a8ba ffff88001eb36b40
[34856.973176] ffff88001e802d80 ffff88001ad21918 ffffffff8117af19 ffff88001b36099c
[34856.973176] ffff88001e802d80 000000000000006b ffff88001b360948 ffff88001ad21968
[34856.973176] Call Trace:
[34856.973176] [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[34856.973176] [<ffffffff8117af19>] print_trailer+0xf9/0x160
[34856.973176] [<ffffffff8117b0b2>] check_bytes_and_report+0xe2/0x120
[34856.973176] [<ffffffff8117b2b7>] check_object+0x1c7/0x240
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff81a66cc2>] alloc_debug_processing+0x153/0x168
[34856.973176] [<ffffffff81a672f5>] __slab_alloc.isra.57+0x61e/0x6d6
[34856.973176] [<ffffffff810ef7bb>] ? trace_hardirqs_on_caller+0xfb/0x240
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff810ef624>] ? mark_held_locks+0x94/0x130
[34856.973176] [<ffffffff8117ef49>] __kmalloc_node_track_caller+0xc9/0x1b0
[34856.973176] [<ffffffff818ffac2>] ? __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff818ff9d7>] __kmalloc_reserve.isra.49+0x37/0xa0
[34856.973176] [<ffffffff818ffa92>] ? __alloc_skb+0x52/0x290
[34856.973176] [<ffffffff818ffac2>] __alloc_skb+0x82/0x290
[34856.973176] [<ffffffff810f372a>] ? lock_release+0x12a/0x250
[34856.973176] [<ffffffff819af1af>] sk_stream_alloc_skb+0x3f/0x120
[34856.973176] [<ffffffff819aff85>] tcp_sendmsg+0x615/0xe20
[34856.973176] [<ffffffff819d7e16>] inet_sendmsg+0xa6/0x100
[34856.973176] [<ffffffff819d7d70>] ? inet_autobind+0x60/0x60
[34856.973176] [<ffffffff818f57b2>] sock_aio_write+0xc2/0xe0
[34856.973176] [<ffffffff811899be>] do_sync_readv_writev+0x4e/0x80
[34856.973176] [<ffffffff8118b0e8>] do_readv_writev+0xe8/0x270
[34856.973176] [<ffffffff810f372a>] ? lock_release+0x12a/0x250
[34856.973176] [<ffffffff8118b2a0>] vfs_writev+0x30/0x60
[34856.973176] [<ffffffff8118b444>] SyS_writev+0x54/0xd0
[34856.973176] [<ffffffff81a76379>] system_call_fastpath+0x16/0x1b
[34856.973176] FIX kmalloc-2048: Restoring 0xffff88001b36099b-0xffff88001b36099b=0x6b
[34856.973176]
[34856.973176] FIX kmalloc-2048: Marking all objects used
[81155.636434] =============================================================================
[81155.636459] BUG skbuff_head_cache (Tainted: G B ): Poison overwritten
[81155.636469] -----------------------------------------------------------------------------
[81155.636469]
[81155.636481] INFO: 0xffff88001e1e7b1b-0xffff88001e1e7b1b. First byte 0x6a instead of 0x6b
[81155.636495] INFO: Allocated in build_skb+0x2f/0x1c0 age=3 cpu=0 pid=0
[81155.636506] __slab_alloc.isra.57+0x61e/0x6d6
[81155.636513] kmem_cache_alloc+0x10a/0x140
[81155.636519] build_skb+0x2f/0x1c0
[81155.636524] __netdev_alloc_skb+0x91/0xf0
[81155.636532] xennet_alloc_rx_buffers+0xb6/0x340
[81155.636538] xennet_poll+0x6c7/0xdb0
[81155.636545] net_rx_action+0x159/0x250
[81155.636551] __do_softirq+0xea/0x1e0
[81155.636557] call_softirq+0x1c/0x30
[81155.636564] do_softirq+0x95/0x100
[81155.636569] irq_exit+0xa2/0xd0
[81155.636576] xen_evtchn_do_upcall+0x2f/0x40
[81155.636582] xen_do_hypervisor_callback+0x1e/0x30
[81155.636595] INFO: Freed in __kfree_skb+0x3a/0xa0 age=2 cpu=0 pid=0
[81155.636603] __slab_free+0x30/0x561
[81155.636608] kmem_cache_free+0x212/0x230
[81155.636614] __kfree_skb+0x3a/0xa0
[81155.636620] tcp_rcv_established+0x33b/0x5c0
[81155.636627] tcp_v4_do_rcv+0x14e/0x4a0
[81155.636633] tcp_v4_rcv+0x8f2/0x9c0
[81155.636639] ip_local_deliver_finish+0xd8/0x1d0
[81155.636647] ip_local_deliver+0x34/0x90
[81155.636653] ip_rcv_finish+0x150/0x660
[81155.636659] ip_rcv+0x22b/0x370
[81155.636664] __netif_receive_skb_core+0x6e2/0x8b0
[81155.636670] __netif_receive_skb+0x1c/0x70
[81155.636676] netif_receive_skb+0x1e/0x100
[81155.636681] xennet_poll+0x8a4/0xdb0
[81155.636687] net_rx_action+0x159/0x250
[81155.636692] __do_softirq+0xea/0x1e0
[81155.636698] INFO: Slab 0xffffea0000787980 objects=14 used=14 fp=0x (null) flags=0x40000000004080
[81155.636707] INFO: Object 0xffff88001e1e7b00 @offset=6912 fp=0xffff88001e1e6b40
[81155.636707]
[81155.636719] Bytes b4 ffff88001e1e7af0: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
[81155.636728] Object ffff88001e1e7b00: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636738] Object ffff88001e1e7b10: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6a 6b 6b 6b 6b kkkkkkkkkkkjkkkk
[81155.636748] Object ffff88001e1e7b20: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636757] Object ffff88001e1e7b30: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636766] Object ffff88001e1e7b40: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636775] Object ffff88001e1e7b50: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636785] Object ffff88001e1e7b60: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636794] Object ffff88001e1e7b70: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636803] Object ffff88001e1e7b80: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636812] Object ffff88001e1e7b90: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636821] Object ffff88001e1e7ba0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636830] Object ffff88001e1e7bb0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636839] Object ffff88001e1e7bc0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636848] Object ffff88001e1e7bd0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[81155.636857] Object ffff88001e1e7be0: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
[81155.636869] Redzone ffff88001e1e7be8: bb bb bb bb bb bb bb bb ........
[81155.636881] Padding ffff88001e1e7d28: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZZZZZ
[81155.636890] Padding ffff88001e1e7d38: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
[81155.636912] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G B 3.12.0-20131104 #1
[81155.636914] ffff88001e1e7b00 ffff88001fc039d0 ffffffff81a6a8ba ffffffff822134e0
[81155.636920] ffff88001e93e6c0 ffff88001fc03a00 ffffffff8117af19 ffff88001e1e7b1c
[81155.636924] ffff88001e93e6c0 000000000000006b ffff88001e1e7b00 ffff88001fc03a50
[81155.636928] Call Trace:
[81155.636930] <IRQ> [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[81155.636936] [<ffffffff8117af19>] print_trailer+0xf9/0x160
[81155.636940] [<ffffffff8117b0b2>] check_bytes_and_report+0xe2/0x120
[81155.636943] [<ffffffff8117b2b7>] check_object+0x1c7/0x240
[81155.636947] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636950] [<ffffffff81a66cc2>] alloc_debug_processing+0x153/0x168
[81155.636954] [<ffffffff81a672f5>] __slab_alloc.isra.57+0x61e/0x6d6
[81155.636957] [<ffffffff819a058d>] ? ip_local_deliver_finish+0x17d/0x1d0
[81155.636961] [<ffffffff819a044c>] ? ip_local_deliver_finish+0x3c/0x1d0
[81155.636964] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636968] [<ffffffff819a0754>] ? ip_local_deliver+0x34/0x90
[81155.636971] [<ffffffff8199ff00>] ? ip_rcv_finish+0x150/0x660
[81155.636975] [<ffffffff8117ce7a>] kmem_cache_alloc+0x10a/0x140
[81155.636978] [<ffffffff818fe78f>] ? build_skb+0x2f/0x1c0
[81155.636981] [<ffffffff818fe78f>] build_skb+0x2f/0x1c0
[81155.636985] [<ffffffff819004b1>] __netdev_alloc_skb+0x91/0xf0
[81155.636988] [<ffffffff816d50a6>] xennet_alloc_rx_buffers+0xb6/0x340
[81155.636991] [<ffffffff8190d1be>] ? netif_receive_skb+0x1e/0x100
[81155.636995] [<ffffffff816d6fb7>] xennet_poll+0x6c7/0xdb0
[81155.636999] [<ffffffff8190d669>] net_rx_action+0x159/0x250
[81155.637003] [<ffffffff8109fa3a>] __do_softirq+0xea/0x1e0
[81155.637006] [<ffffffff81a777bc>] call_softirq+0x1c/0x30
[81155.637009] [<ffffffff81010dc5>] do_softirq+0x95/0x100
[81155.637013] [<ffffffff8109fce2>] irq_exit+0xa2/0xd0
[81155.637016] [<ffffffff814bb2af>] xen_evtchn_do_upcall+0x2f/0x40
[81155.637019] [<ffffffff81a7781e>] xen_do_hypervisor_callback+0x1e/0x30
[81155.637021] <EOI> [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[81155.637027] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
[81155.637031] [<ffffffff81008da0>] ? xen_safe_halt+0x10/0x20
[81155.637034] [<ffffffff810185cd>] ? default_idle+0x1d/0x30
[81155.637038] [<ffffffff81018f0e>] ? arch_cpu_idle+0x2e/0x40
[81155.637042] [<ffffffff810db410>] ? cpu_startup_entry+0x80/0x1d0
[81155.637046] [<ffffffff81a5ebf1>] ? rest_init+0xc1/0xd0
[81155.637049] [<ffffffff81a5eb30>] ? csum_partial_copy_generic+0x170/0x170
[81155.637054] [<ffffffff8230bee6>] ? start_kernel+0x3d3/0x3e0
[81155.637058] [<ffffffff8230b912>] ? repair_env_string+0x5e/0x5e
[81155.637061] [<ffffffff810d89f7>] ? __add_preferred_console.constprop.18+0x87/0xb0
[81155.637065] [<ffffffff8230b5f8>] ? x86_64_start_reservations+0x2a/0x2c
[81155.637069] [<ffffffff8230ecd9>] ? xen_start_kernel+0x563/0x56d
[81155.637072] FIX skbuff_head_cache: Restoring 0xffff88001e1e7b1b-0xffff88001e1e7b1b=0x6b
[81155.637072]
[81155.637084] FIX skbuff_head_cache: Marking all objects used
[81234.562076] BUG: unable to handle kernel paging request at ffffffff80014120
[81234.562131] IP: [<ffffffff80014120>] 0xffffffff8001411f
[81234.562159] PGD 220f067 PUD 2210067 PMD 0
[81234.562186] Oops: 0010 [#1] PREEMPT SMP
[81234.562192] Modules linked in:
[81234.562198] CPU: 1 PID: 7992 Comm: smbd Tainted: G B 3.12.0-20131104 #1
[81234.562206] task: ffff88001d934780 ti: ffff88000eb28000 task.ti: ffff88000eb28000
[81234.562212] RIP: e030:[<ffffffff80014120>] [<ffffffff80014120>] 0xffffffff8001411f
[81234.562219] RSP: e02b:ffff88000eb29e10 EFLAGS: 00010246
[81234.562224] RAX: ffffffff80014120 RBX: fffffffffffffff4 RCX: 0000000000000000
[81234.562230] RDX: 000000000007e000 RSI: 0000000000000000 RDI: 0000000000000000
[81234.562235] RBP: ffff88000eb29e28 R08: 0000000000000022 R09: 00007ffffffff000
[81234.562241] R10: 000000000000007e R11: fffffffffffffff4 R12: 000000000007e000
[81234.562246] R13: 0000000000000022 R14: ffff88001b25cac0 R15: 000000000007e000
[81234.562254] FS: 00007fdc5cc42720(0000) GS:ffff88001fd00000(0000) knlGS:0000000000000000
[81234.562260] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[81234.562265] CR2: ffffffff80014120 CR3: 0000000004d1d000 CR4: 0000000000000660
[81234.562271] Stack:
[81234.562274] ffffffff8115ce9d 0000000000000003 0000000000000000 ffff88000eb29e88
[81234.562283] ffffffff8115fb4a 0000000000000000 0000000000000000 000000000000007e
[81234.562292] 0000000000000000 ffff88000eb29e88 ffff88001b25cb50 0000000000000000
[81234.562300] Call Trace:
[81234.562308] [<ffffffff8115ce9d>] ? get_unmapped_area+0x9d/0x150
[81234.562314] [<ffffffff8115fb4a>] do_mmap_pgoff+0x10a/0x3e0
[81234.562321] [<ffffffff8114ba10>] vm_mmap_pgoff+0x70/0xb0
[81234.562326] [<ffffffff8115e38c>] SyS_mmap_pgoff+0x1cc/0x260
[81234.562334] [<ffffffff814209ee>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[81234.562342] [<ffffffff81013f1d>] SyS_mmap+0x1d/0x20
[81234.562349] [<ffffffff81a76379>] system_call_fastpath+0x16/0x1b
[81234.562354] Code: Bad RIP value.
[81234.562363] RIP [<ffffffff80014120>] 0xffffffff8001411f
[81234.562368] RSP <ffff88000eb29e10>
[81234.562372] CR2: ffffffff80014120
[81234.562379] ---[ end trace 4d2d192c2478dd2a ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-11-11 14:57 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-20 18:05 kernel BUG at mm/slub.c:3338! RIP [<ffffffff8117912c>] kfree+0x20c/0x210 Sander Eikelenboom
2013-10-22 15:12 ` Christoph Lameter
2013-10-22 15:29 ` Sander Eikelenboom
2013-11-09 18:18 ` Sander Eikelenboom
2013-11-11 14:57 ` Christoph Lameter
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®