On Wed, May 08, 2013 at 06:57:36PM -0500, Robin Holt wrote: > Thomas, > > We are seeing failures booting medium sized machines which I think is > a change in expectations that dyntick put on x86's start_secondary. > > During boot of cpus, we see an occassional panic in tick_do_broadcast at > > 195 if (!cpumask_empty(mask)) { > 196 /* > 197 * It might be necessary to actually check whether the devices > 198 * have different broadcast functions. For now, just use the > 199 * one of the first device. This works as long as we have this > 200 * misfeature only on x86 (lapic) > 201 */ > 202 td = &per_cpu(tick_cpu_device, cpumask_first(mask)); > 203 td->evtdev->broadcast(mask); > ^^^^^^ > NULL --------+ Seems not related to full dynticks. I can reproduce it at every boot with CONFIG_NO_HZ_FULL=n. Attached is the config that triggers it and the log: [ 0.000000] Linux version 3.9.0+ (fweisbec@phenom) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5.1) ) #2 SMP PREEMPT Sat May 11 16:05:21 CEST 2013 [ 0.000000] Command line: root=/dev/sda1 ro ignore_loglevel rcu_nocbs=1-2 nohz_full=1-2 console=ttyS0,115200n8 [ 0.000000] KERNEL supported cpus: [ 0.000000] Intel GenuineIntel [ 0.000000] CPU: vendor_id 'AuthenticAMD' unknown, using generic init. [ 0.000000] CPU: Your system may be unstable. [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f3ff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009f400-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000cffdffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000cffe0000-0x00000000cffe2fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x00000000cffe3000-0x00000000cffeffff] ACPI data [ 0.000000] BIOS-e820: [mem 0x00000000cfff0000-0x00000000cfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000107ffffff] usable [ 0.000000] debug: ignoring loglevel setting. [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.5 present. [ 0.000000] DMI: FUJITSU SIEMENS AMD690VM-FMH/AMD690VM-FMH, BIOS V5.13 03/14/2008 [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] e820: last_pfn = 0x108000 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: uncachable [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-C7FFF write-protect [ 0.000000] C8000-FFFFF uncachable [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 000000000000 mask FFFF80000000 write-back [ 0.000000] 1 base 000080000000 mask FFFFC0000000 write-back [ 0.000000] 2 base 0000C0000000 mask FFFFF0000000 write-back [ 0.000000] 3 base 000100000000 mask FFFFF8000000 write-back [ 0.000000] 4 disabled [ 0.000000] 5 disabled [ 0.000000] 6 disabled [ 0.000000] 7 disabled [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 [ 0.000000] e820: update [mem 0xd0000000-0xffffffff] usable ==> reserved [ 0.000000] e820: last_pfn = 0xcffe0 max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f3dc0-0x000f3dcf] mapped at [ffff8800000f3dc0] [ 0.000000] Scanning 1 areas for low memory corruption [ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576 [ 0.000000] Using GB pages for direct mapping [ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff] [ 0.000000] [mem 0x00000000-0x000fffff] page 4k [ 0.000000] BRK [0x036f7000, 0x036f7fff] PGTABLE [ 0.000000] BRK [0x036f8000, 0x036f8fff] PGTABLE [ 0.000000] BRK [0x036f9000, 0x036f9fff] PGTABLE [ 0.000000] init_memory_mapping: [mem 0x107e00000-0x107ffffff] [ 0.000000] [mem 0x107e00000-0x107ffffff] page 2M [ 0.000000] BRK [0x036fa000, 0x036fafff] PGTABLE [ 0.000000] init_memory_mapping: [mem 0x104000000-0x107dfffff] [ 0.000000] [mem 0x104000000-0x107dfffff] page 2M [ 0.000000] init_memory_mapping: [mem 0x100000000-0x103ffffff] [ 0.000000] [mem 0x100000000-0x103ffffff] page 2M [ 0.000000] init_memory_mapping: [mem 0x80000000-0xcffdffff] [ 0.000000] [mem 0x80000000-0xbfffffff] page 1G [ 0.000000] [mem 0xc0000000-0xcfdfffff] page 2M [ 0.000000] [mem 0xcfe00000-0xcffdffff] page 4k [ 0.000000] BRK [0x036fb000, 0x036fbfff] PGTABLE [ 0.000000] init_memory_mapping: [mem 0x00100000-0x7fffffff] [ 0.000000] [mem 0x00100000-0x001fffff] page 4k [ 0.000000] [mem 0x00200000-0x3fffffff] page 2M [ 0.000000] [mem 0x40000000-0x7fffffff] page 1G [ 0.000000] ACPI: RSDP 00000000000f7cc0 00024 (v02 FSC ) [ 0.000000] ACPI: RSDT 00000000cffe3040 0003C (v01 FSC PC 42302E31 AWRD 00000000) [ 0.000000] ACPI: FACP 00000000cffe30c0 00074 (v01 FSC PC 42302E31 AWRD 00000000) [ 0.000000] ACPI: DSDT 00000000cffe3180 049C0 (v01 FSC AWRDACPI 00001000 MSFT 03000000) [ 0.000000] ACPI: FACS 00000000cffe0000 00040 [ 0.000000] ACPI: SSDT 00000000cffe7c80 00544 (v01 PTLTD POWERNOW 00000001 LTP 00000001) [ 0.000000] ACPI: HPET 00000000cffe8240 00038 (v01 FSC PC 42302E31 AWRD 00000098) [ 0.000000] ACPI: SLIC 00000000cffe82c0 00176 (v01 FSC PC 42302E31 AWRD 00000001) [ 0.000000] ACPI: MCFG 00000000cffe8480 0003C (v01 FSC PC 42302E31 AWRD 00000000) [ 0.000000] ACPI: APIC 00000000cffe7b80 00084 (v01 FSC PC 42302E31 AWRD 00000000) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000107ffffff] [ 0.000000] Initmem setup node 0 [mem 0x00000000-0x107ffffff] [ 0.000000] NODE_DATA [mem 0x107fd8000-0x107ffefff] [ 0.000000] [ffffea0000000000-ffffea00041fffff] PMD -> [ffff880104000000-ffff8801075fffff] on node 0 [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x107ffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009efff] [ 0.000000] node 0: [mem 0x00100000-0xcffdffff] [ 0.000000] node 0: [mem 0x100000000-0x107ffffff] [ 0.000000] On node 0 totalpages: 884606 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3998 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 13248 pages used for memmap [ 0.000000] DMA32 zone: 847840 pages, LIFO batch:31 [ 0.000000] Normal zone: 512 pages used for memmap [ 0.000000] Normal zone: 32768 pages, LIFO batch:7 [ 0.000000] ACPI: PM-Timer IO Port: 0x4008 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] ACPI: NR_CPUS/possible_cpus limit of 2 reached. Processor 2/0x2 ignored. [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] ACPI: NR_CPUS/possible_cpus limit of 2 reached. Processor 3/0x3 ignored. [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ2 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x10b9a201 base: 0xfed00000 [ 0.000000] smpboot: 4 Processors exceeds NR_CPUS limit of 2 [ 0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs [ 0.000000] nr_irqs_gsi: 40 [ 0.000000] e820: [mem 0xd0000000-0xdfffffff] available for PCI devices [ 0.000000] setup_percpu: NR_CPUS:2 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 477 pages/cpu @ffff880103c00000 s1921280 r8192 d24320 u2097152 [ 0.000000] pcpu-alloc: s1921280 r8192 d24320 u2097152 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 870761 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: root=/dev/sda1 ro ignore_loglevel rcu_nocbs=1-2 nohz_full=1-2 console=ttyS0,115200n8 [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Memory: 3372776k/4325376k available (14604k kernel code, 786952k absent, 165648k reserved, 9565k data, 3456k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] RCU debugfs-based tracing is enabled. [ 0.000000] RCU lockdep checking is enabled. [ 0.000000] Experimental no-CBs for all CPUs [ 0.000000] Experimental no-CBs CPUs: 0-1. [ 0.000000] NR_IRQS:4352 nr_irqs:512 16 [ 0.000000] Console: colour VGA+ 80x25 [ 0.000000] console [ttyS0] enabled [ 0.000000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.000000] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.000000] ... MAX_LOCK_DEPTH: 48 [ 0.000000] ... MAX_LOCKDEP_KEYS: 8191 [ 0.000000] ... CLASSHASH_SIZE: 4096 [ 0.000000] ... MAX_LOCKDEP_ENTRIES: 16384 [ 0.000000] ... MAX_LOCKDEP_CHAINS: 32768 [ 0.000000] ... CHAINHASH_SIZE: 16384 [ 0.000000] memory used by lock dependency info: 6367 kB [ 0.000000] per task-struct memory footprint: 2688 bytes [ 0.000000] ODEBUG: 49 of 49 active objects replaced [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Fast TSC calibration failed [ 0.000000] tsc: Unable to calibrate against PIT [ 0.000000] tsc: using HPET reference calibration [ 0.000000] tsc: Detected 2299.710 MHz processor [ 0.000000] tsc: Marking TSC unstable due to TSCs unsynchronized [ 0.011000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4599.42 BogoMIPS (lpj=2299710) [ 0.012024] pid_max: default: 32768 minimum: 301 [ 0.014303] Security Framework initialized [ 0.015031] SELinux: Initializing. [ 0.016185] SELinux: Starting in permissive mode [ 0.022138] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.032281] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 0.037180] Mount-cache hash table entries: 256 [ 0.053269] Initializing cgroup subsys devices [ 0.054124] Initializing cgroup subsys freezer [ 0.055090] Initializing cgroup subsys net_cls [ 0.056051] Initializing cgroup subsys blkio [ 0.057033] Initializing cgroup subsys perf_event [ 0.059062] Initializing cgroup subsys net_prio [ 0.061168] mce: CPU supports 6 MCE banks [ 0.062041] mce: unknown CPU type - not enabling MCE support [ 0.063027] numa_add_cpu cpu 0 node 0: mask now 0 [ 0.065017] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.065017] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.065017] tlb_flushall_shift: -1 [ 0.066280] Freeing SMP alternatives: 40k freed [ 0.067055] ACPI: Core revision 20130328 [ 0.703059] ACPI: All ACPI Tables successfully acquired [ 0.707111] ftrace: allocating 44867 entries in 176 pages [ 0.726591] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.737707] smpboot: CPU0: AuthenticAMD AMD Phenom(tm) 9600 Quad-Core Processor (fam: 10, model: 02, stepping: 02) [ 0.743000] Performance Events: [ 0.757505] ftrace: Allocated trace_printk buffers [ 0.775835] NMI watchdog: disabled (cpu0): hardware events not enabled [ 0.786881] SMP alternatives: lockdep: fixing up alternatives [ 0.787129] smpboot: Booting Node 0, Processors #1 OK [ 0.011000] numa_add_cpu cpu 1 node 0: mask now 0-1 [ 0.011000] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 [ 0.011000] IP: [] clockevents_set_mode+0x16/0x70 [ 0.011000] PGD 0 [ 0.011000] Oops: 0000 [#1] PREEMPT SMP [ 0.011000] Modules linked in: [ 0.011000] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.9.0+ #2 [ 0.011000] Hardware name: FUJITSU SIEMENS AMD690VM-FMH/AMD690VM-FMH, BIOS V5.13 03/14/2008 [ 0.011000] task: ffff880102e6ca40 ti: ffff880102e66000 task.ti: ffff880102e66000 [ 0.011000] RIP: 0010:[] [] clockevents_set_mode+0x16/0x70 [ 0.011000] RSP: 0000:ffff880103e03ef8 EFLAGS: 00010082 [ 0.011000] RAX: ffff880103e0d620 RBX: 0000000000000001 RCX: 0000000000000000 [ 0.011000] RDX: 0000000000000003 RSI: 0000000000000003 RDI: 0000000000000000 [ 0.011000] RBP: ffff880103e03f08 R08: 0000000000000000 R09: 0000000000000001 [ 0.011000] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880103826a80 [ 0.011000] R13: 000000000000d640 R14: ffff880102e67e78 R15: 0000000000000000 [ 0.011000] FS: 0000000000000000(0000) GS:ffff880103e00000(0000) knlGS:0000000000000000 [ 0.011000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.011000] CR2: 0000000000000038 CR3: 000000000260c000 CR4: 00000000000006a0 [ 0.011000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.011000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.011000] Stack: [ 0.011000] 0000000000000001 ffff880103826a80 ffff880103e03f28 ffffffff81132113 [ 0.011000] 000000000000d640 0000000000000001 ffff880103e03f58 ffffffff811339c2 [ 0.011000] 0000000000000001 ffffffffffffffc8 0000000000000000 ffff880102e67e78 [ 0.011000] Call Trace: [ 0.011000] [ 0.011000] [] tick_check_oneshot_broadcast+0x53/0x90 [ 0.011000] [] tick_check_idle+0x32/0xe0 [ 0.011000] [] irq_enter+0x7c/0x90 [ 0.011000] [] do_IRQ+0x3e/0xe0 [ 0.011000] [] common_interrupt+0x6c/0x6c [ 0.011000] [ 0.011000] [] ? start_secondary+0x218/0x277 [ 0.011000] [] ? start_secondary+0x213/0x277 [ 0.011000] Code: dc 48 83 c4 08 5b c9 c3 eb 0b 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 48 83 ec 10 48 89 1c 24 4c 89 64 24 08 66 66 66 66 90 <39> 77 38 48 89 fb 41 89 f4 74 13 48 89 fe 44 89 e7 ff 53 50 41 [ 0.011000] RIP [] clockevents_set_mode+0x16/0x70 [ 0.011000] RSP [ 0.011000] CR2: 0000000000000038 [ 0.866000] BUG: unable to handle kernel [ 0.011000] ---[ end trace bd42872bae65a76c ]--- [ 0.011000] Kernel panic - not syncing: Fatal exception in interrupt NULL pointer dereference at 0000000000000048 [ 0.866000] IP: [] tick_do_broadcast+0x5e/0x100 [ 0.866000] PGD 0 [ 0.866000] Oops: 0000 [#2] PREEMPT SMP [ 0.866000] Modules linked in: [ 0.866000] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G D 3.9.0+ #2 [ 0.866000] Hardware name: FUJITSU SIEMENS AMD690VM-FMH/AMD690VM-FMH, BIOS V5.13 03/14/2008 [ 0.866000] task: ffff880102c20000 ti: ffff88010336a000 task.ti: ffff88010336a000 [ 0.866000] RIP: 0010:[] [] tick_do_broadcast+0x5e/0x100 [ 0.866000] RSP: 0000:ffff880103c03c40 EFLAGS: 00010002 [ 0.866000] RAX: 0000000000000000 RBX: ffff8801038263f0 RCX: 0000000000000001 [ 0.866000] RDX: ffff8801038263f0 RSI: 0000000000000002 RDI: ffff8801038263f0 [ 0.866000] RBP: ffff880103c03c60 R08: 0000000000000000 R09: 0000000000000008 [ 0.866000] R10: 0000000000000001 R11: 0000000000000000 R12: 000000000000d620 [ 0.866000] R13: 0000000000000000 R14: ffff880103c00080 R15: 0000000000000000 [ 0.866000] FS: 0000000000000000(0000) GS:ffff880103c00000(0000) knlGS:0000000000000000 [ 0.866000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.866000] CR2: 0000000000000048 CR3: 000000000260c000 CR4: 00000000000006b0 [ 0.866000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.866000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.866000] Stack: [ 0.866000] ffffffff81132be5 0000000000000000 ffffffff82617e00 ffffffff82624700 [ 0.866000] ffff880103c03c70 ffffffff81132c11 ffff880103c03c90 ffffffff81132c34 [ 0.866000] ffffffff82617e00 ffff88010380c3d0 ffff880103c03ca0 ffffffff8104cd75 [ 0.866000] Call Trace: [ 0.866000] [ 0.866000] [] ? tick_do_periodic_broadcast+0x15/0x50 [ 0.866000] [] tick_do_periodic_broadcast+0x41/0x50 [ 0.866000] [] tick_handle_periodic_broadcast+0x14/0x50 [ 0.866000] [] timer_interrupt+0x15/0x20 [ 0.866000] [] handle_irq_event_percpu+0x9d/0x3b0 [ 0.866000] [] handle_irq_event+0x48/0x70 [ 0.866000] [] handle_edge_irq+0x6d/0x130 [ 0.866000] [] handle_irq+0x71/0x190 [ 0.866000] [] ? vtime_account_system+0x52/0x60 [ 0.866000] [] do_IRQ+0x5d/0xe0 [ 0.866000] [] common_interrupt+0x6c/0x6c [ 0.866000] [] ? __do_softirq+0xc1/0x470 [ 0.866000] [] ? __do_softirq+0xcb/0x470 [ 0.866000] [] ? __do_softirq+0xc1/0x470 [ 0.866000] [] ? _raw_spin_unlock+0x35/0x60 [ 0.866000] [] irq_exit+0xed/0x100 [ 0.866000] [] smp_apic_timer_interrupt+0x6b/0x98 [ 0.866000] [] apic_timer_interrupt+0x6c/0x80 [ 0.866000] [ 0.866000] [] ? mark_held_locks+0x90/0x150 [ 0.866000] [] ? _raw_spin_unlock_irq+0x36/0x70 [ 0.866000] [] ? _raw_spin_unlock_irq+0x30/0x70 [ 0.866000] [] __schedule+0x925/0x9c0 [ 0.866000] [] schedule+0x29/0x70 [ 0.866000] [] native_cpu_up+0x76e/0x966 [ 0.866000] [] cpu_up+0xfb/0x17b [ 0.866000] [] smp_init+0x4e/0x8e [ 0.866000] [] kernel_init_freeable+0x21a/0x32e [ 0.866000] [] ? rest_init+0x160/0x160 [ 0.866000] [] kernel_init+0xe/0xf0 [ 0.866000] [] ret_from_fork+0x7c/0xb0 [ 0.866000] [] ? rest_init+0x160/0x160 [ 0.866000] Code: 85 c0 75 2a 48 63 35 ba 30 65 01 48 89 df 49 c7 c4 20 d6 00 00 e8 53 f4 52 00 89 c0 48 89 df 48 8b 04 c5 40 26 78 82 4a 8b 04 20 50 48 48 83 c4 10 5b 41 5c c9 c3 eb 04 90 90 90 90 39 f0 73 [ 0.866000] RIP [] tick_do_broadcast+0x5e/0x100 [ 0.866000] RSP [ 0.866000] CR2: 0000000000000048 [ 0.866000] ---[ end trace bd42872bae65a76d ]---