From: Ben Greear <greearb@candelatech.com>
To: LKML <linux-kernel@vger.kernel.org>
Cc: linux-usb@vger.kernel.org
Subject: Deadlock in usb subsystem on shutdown, 6.18.3+
Date: Tue, 13 Jan 2026 16:21:07 -0800 [thread overview]
Message-ID: <869c32e4-7bfc-c3c0-92a8-f5a3f9ea3ae3@candelatech.com> (raw)
Hello,
We caught a deadlock that appears to be in the USB code during shutdown.
We do a lot of reboots and normally all goes well, so I don't think we
can reliably reproduce the problem.
INFO: task systemd-shutdow:1 blocked for more than 180 seconds.
Tainted: G S O 6.18.3+ #33
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:systemd-shutdow state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00080001
Call Trace:
<TASK>
__schedule+0x46b/0x1140
schedule+0x23/0xc0
schedule_preempt_disabled+0x11/0x20
__mutex_lock.constprop.0+0x4f7/0x9a0
device_shutdown+0xa0/0x220
kernel_restart+0x36/0x90
__do_sys_reboot+0x127/0x220
? do_writev+0x76/0x110
? do_writev+0x76/0x110
do_syscall_64+0x50/0x6d0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fad03531087
RSP: 002b:00007ffe137cf918 EFLAGS: 00000246 ORIG_RAX: 00000000000000a9
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fad03531087
RDX: 0000000001234567 RSI: 0000000028121969 RDI: 00000000fee1dead
RBP: 00007ffe137cfac0 R08: 0000000000000069 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
</TASK>
INFO: task systemd-shutdow:1 is blocked on a mutex likely owned by task kworker/4:1:16648.
INFO: task kworker/4:2:1520 blocked for more than 360 seconds.
Tainted: G S O 6.18.3+ #33
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/4:2 state:D stack:0 pid:1520 tgid:1520 ppid:2 task_flags:0x4288060 flags:0x00080000
Workqueue: events __usb_queue_reset_device
Call Trace:
<TASK>
__schedule+0x46b/0x1140
? schedule_timeout+0x79/0xf0
schedule+0x23/0xc0
usb_kill_urb+0x7b/0xc0
? housekeeping_affine+0x30/0x30
usb_start_wait_urb+0xd6/0x160
usb_control_msg+0xe2/0x140
hub_port_init+0x647/0xf70
usb_reset_and_verify_device+0x191/0x4a0
? device_release_driver_internal+0x4a/0x200
usb_reset_device+0x138/0x280
__usb_queue_reset_device+0x35/0x50
process_one_work+0x17e/0x390
worker_thread+0x2c8/0x3e0
? process_one_work+0x390/0x390
kthread+0xf7/0x1f0
? kthreads_online_cpu+0x100/0x100
? kthreads_online_cpu+0x100/0x100
ret_from_fork+0x114/0x140
? kthreads_online_cpu+0x100/0x100
ret_from_fork_asm+0x11/0x20
</TASK>
INFO: task kworker/4:1:16648 blocked for more than 360 seconds.
Tainted: G S O 6.18.3+ #33
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/4:1 state:D stack:0 pid:16648 tgid:16648 ppid:2 task_flags:0x4288060 flags:0x00080000
Workqueue: events __usb_queue_reset_device
Call Trace:
<TASK>
__schedule+0x46b/0x1140
schedule+0x23/0xc0
usb_kill_urb+0x7b/0xc0
? housekeeping_affine+0x30/0x30
usb_hcd_flush_endpoint+0xb2/0x160
usb_disable_interface+0xbb/0xe0
usb_unbind_interface+0x11e/0x290
? kernfs_remove_by_name_ns+0xb0/0xd0
device_release_driver_internal+0x197/0x200
usb_forced_unbind_intf+0x4d/0xa0
usb_reset_device+0xe1/0x280
__usb_queue_reset_device+0x35/0x50
process_one_work+0x17e/0x390
worker_thread+0x2c8/0x3e0
? process_one_work+0x390/0x390
kthread+0xf7/0x1f0
? kthreads_online_cpu+0x100/0x100
? kthreads_online_cpu+0x100/0x100
ret_from_fork+0x114/0x140
? kthreads_online_cpu+0x100/0x100
ret_from_fork_asm+0x11/0x20
</TASK>
INFO: task kworker/6:1:16649 blocked for more than 360 seconds.
Tainted: G S O 6.18.3+ #33
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
task:kworker/6:1 state:D stack:0 pid:16649 tgid:16649 ppid:2 task_flags:0x4208060 flags:0x00080000
Workqueue: usb_hub_wq hub_event
Call Trace:
<TASK>
__schedule+0x46b/0x1140
schedule+0x23/0xc0
schedule_preempt_disabled+0x11/0x20
__mutex_lock.constprop.0+0x4f7/0x9a0
? update_load_avg+0x7f/0x790
? update_curr+0x14d/0x180
hub_event+0x172/0x1b30
? finish_task_switch.isra.0+0x8d/0x270
? __schedule+0x473/0x1140
process_one_work+0x17e/0x390
worker_thread+0x2c8/0x3e0
? process_one_work+0x390/0x390
kthread+0xf7/0x1f0
? kthreads_online_cpu+0x100/0x100
? kthreads_online_cpu+0x100/0x100
ret_from_fork+0x114/0x140
? kthreads_online_cpu+0x100/0x100
ret_from_fork_asm+0x11/0x20
</TASK>
INFO: task kworker/6:1:16649 is blocked on a mutex likely owned by task kworker/4:2:1520.
NMI backtrace for cpu 14
CPU: 14 UID: 0 PID: 141 Comm: khungtaskd Kdump: loaded Tainted: G S O 6.18.3+ #33 PREEMPT(full)
Tainted: [S]=CPU_OUT_OF_SPEC, [O]=OOT_MODULE
Hardware name: Default string /Default string, BIOS 5.27 08/08/2025
Call Trace:
<TASK>
dump_stack_lvl+0x60/0x80
nmi_cpu_backtrace+0xca/0x100
? lapic_can_unplug_cpu+0xa0/0xa0
nmi_trigger_cpumask_backtrace+0xc5/0xe0
watchdog+0x6ce/0x6f0
? proc_dohung_task_timeout_secs+0x30/0x30
kthread+0xf7/0x1f0
? kthreads_online_cpu+0x100/0x100
? kthreads_online_cpu+0x100/0x100
ret_from_fork+0x114/0x140
? kthreads_online_cpu+0x100/0x100
ret_from_fork_asm+0x11/0x20
</TASK>
Sending NMI from CPU 14 to CPUs 0-13,15-19:
NMI backtrace for cpu 0 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 1 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 2 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 3 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 4 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 5 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 6 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 7 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 8 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 9 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 10 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 11 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 12 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 13 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 15 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 16 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 17 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 18 skipped: idling at intel_idle+0x50/0x80
NMI backtrace for cpu 19 skipped: idling at intel_idle+0x50/0x80
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
next reply other threads:[~2026-01-14 0:27 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-14 0:21 Ben Greear [this message]
2026-01-14 2:45 ` Hillf Danton
2026-01-14 14:36 ` Ben Greear
2026-01-14 15:13 ` Alan Stern
2026-01-14 17:51 ` Ben Greear
2026-01-14 19:26 ` Alan Stern
2026-01-14 20:14 ` Oliver Neukum
2026-01-20 17:29 ` Ben Greear
2026-01-29 9:28 ` Michal Pecio
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=869c32e4-7bfc-c3c0-92a8-f5a3f9ea3ae3@candelatech.com \
--to=greearb@candelatech.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-usb@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®