FYI, we noticed the following commit: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master commit 4cd13c21b207e80ddb1144c576500098f2d5f882 ("softirq: Let ksoftirqd do its job") in testcase: boot on test machine: qemu-system-x86_64 -enable-kvm -smp 2 -m 512M caused below changes: +------------------------------------------------+------------+------------+ | | b8129a1f6a | 4cd13c21b2 | +------------------------------------------------+------------+------------+ | boot_successes | 0 | 0 | | boot_failures | 80 | 80 | | calltrace:SyS_open | 78 | 80 | | BUG:workqueue_lockup-pool | 80 | 46 | | IP-Config:Auto-configuration_of_network_failed | 0 | 5 | | INFO:rcu_sched_self-detected_stall_on_CPU | 0 | 33 | | INFO:rcu_sched_detected_stalls_on_CPUs/tasks | 0 | 31 | | calltrace:mark_rodata_ro | 0 | 33 | | invoked_oom-killer:gfp_mask=0x | 0 | 3 | | Mem-Info | 0 | 3 | | Out_of_memory:Kill_process | 0 | 2 | | BUG:kernel_hang_in_test_stage | 0 | 27 | +------------------------------------------------+------------+------------+ [ 23.488574] Freeing unused kernel memory: 3428K (ffffffff8390e000 - ffffffff83c67000) [ 23.489865] Write protecting the kernel read-only data: 34816k [ 23.510721] Freeing unused kernel memory: 1864K (ffff88000262e000 - ffff880002800000) [ 23.527185] Freeing unused kernel memory: 128K (ffff8800031e0000 - ffff880003200000) [ 135.341384] INFO: rcu_sched self-detected stall on CPU [ 135.345265] INFO: rcu_sched detected stalls on CPUs/tasks: [ 135.345294] Task dump for CPU 0: [ 135.345305] swapper/0 R running task 0 1 0 0x00080008 [ 135.345317] ffffffff8100401a ffff880017ae2000 ffffffff82629061 0000000000000000 [ 135.345327] ffffffff8326e260 fffffac2858c5000 ffff880017aefc50 ffffffff81159fee [ 135.345338] ffff880017aefe68 8000000000000161 ffff880017aefca8 ffffffff8100401a [ 135.345340] Call Trace: [ 135.345353] [] ? trace_hardirqs_on_thunk+0x1a/0x1c [ 135.345363] [] ? retint_kernel+0x10/0x10 [ 135.345372] [] ? trace_hardirqs_on_caller+0x1ae/0x2e0 [ 135.345381] [] ? trace_hardirqs_on_thunk+0x1a/0x1c [ 135.345390] [] ? irq_exit+0xd0/0x180 [ 135.345398] [] ? retint_kernel+0x10/0x10 [ 135.345408] [] ? note_page+0xac7/0xea0 [ 135.345417] [] ? note_page+0xac7/0xea0 [ 135.345426] [] ? note_page+0x46c/0xea0 [ 135.345436] [] ? ptdump_walk_pgd_level_core+0x582/0x5e0 [ 135.345445] [] ? note_page+0xea0/0xea0 [ 135.345455] [] ? ptdump_walk_pgd_level_checkwx+0x17/0x20 [ 135.345463] [] ? mark_rodata_ro+0x10c/0x120 [ 135.345472] [] ? kernel_init+0x34/0x180 [ 135.345480] [] ? ret_from_fork+0x1f/0x40 [ 135.345488] [] ? rest_init+0x180/0x180 [ 135.372126] Task dump for CPU 0: [ 135.372637] swapper/0 R running task 0 1 0 0x00080008 [ 135.378686] 0000000000000000 ffff880017e07de8 ffffffff8112bfed ffffffff8112bed3 [ 135.382196] ffff880017e07df8 0000000041b58ab3 ffffffff830edea2 ffffffff8112be10 [ 135.386448] 0000000000000046 ffffffff00000000 ffffffff832c7038 ffffffff8125ebf0 [ 135.387952] Call Trace: [ 135.388373] [] sched_show_task+0x1dd/0x320 [ 135.389427] [] ? sched_show_task+0xc3/0x320 [ 135.390613] [] ? SyS_sched_rr_get_interval+0x320/0x320 [ 135.391683] [] ? rcu_dump_cpu_stacks+0x5c/0x182 [ 135.392629] [] ? rcu_dump_cpu_stacks+0x115/0x182 [ 135.393838] [] dump_cpu_task+0x4d/0x60 [ 135.394768] [] rcu_dump_cpu_stacks+0x130/0x182 [ 135.395762] [] rcu_check_callbacks+0x10e4/0x1180 [ 135.396770] [] ? raise_softirq+0x7c/0x90 [ 135.398051] [] update_process_times+0x30/0x60 [ 135.399086] [] tick_nohz_handler+0xf4/0x230 [ 135.400050] [] local_apic_timer_interrupt+0x4a/0x80 [ 135.401279] [] smp_apic_timer_interrupt+0x52/0x70 [ 135.402346] [] apic_timer_interrupt+0x91/0xa0 [ 135.403327] [] ? note_page+0xac7/0xea0 [ 135.404432] [] ? note_page+0x46c/0xea0 [ 135.405380] [] ptdump_walk_pgd_level_core+0x582/0x5e0 [ 135.406445] [] ? note_page+0xea0/0xea0 [ 135.407438] [] ptdump_walk_pgd_level_checkwx+0x17/0x20 [ 135.408585] [] mark_rodata_ro+0x10c/0x120 [ 135.409474] [] kernel_init+0x34/0x180 [ 135.410789] [] ret_from_fork+0x1f/0x40 [ 135.411660] [] ? rest_init+0x180/0x180 [ 158.650460] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 158.666306] BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 127s! [ 158.666320] Showing busy workqueues and worker pools: [ 158.666324] workqueue events: flags=0x0 [ 158.666351] pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 [ 158.666362] pending: vmstat_shepherd [ 158.666803] Writes: Total: 4 Max/Min: 0/0 Fail: 0 [ 158.753030] random: init: uninitialized urandom read (12 bytes read) Thanks, Xiaolong