180kB/s 1m10s =============================================== BUG: circular semaphore deadlock detected! ----------------------------------------------- ksoftirqd/0/3 is deadlocking current task rpciod/0/250 1) rpciod/0/250 is trying to acquire this lock: [dfefa1a4] {r:0,a:-1,&dev->xmit_lock} .. held by: ksoftirqd/0/ 3 [c17946b0, 105] ... acquired at: qdisc_restart+0x262/0x280 ... at: qdisc_restart+0x262/0x280 2) ksoftirqd/0/3 is blocked on this lock: [c045cd80] {r:0,a:-1,ptype_lock} .. held by: rpciod/0/ 250 [df7ace40, 106] ... acquired at: qdisc_restart+0x24f/0x280 ------------------------------ | showing all locks held by: | (rpciod/0/250 [df7ace40, 106]): ------------------------------ #001: [c045cd80] {r:0,a:-1,ptype_lock} ... acquired at: qdisc_restart+0x24f/0x280 ------------------------------ | showing all locks held by: | (ksoftirqd/0/3 [c17946b0, 105]): ------------------------------ #001: [dfefa1a4] {r:0,a:-1,&dev->xmit_lock} ... acquired at: qdisc_restart+0x262/0x280 ksoftirqd/0/3's [blocked] stackdump: c179fe64 00000046 c17946b0 c140c820 c179e000 c17946b0 c179fe20 c0138fed c013deeb c0108e33 00000202 c01148ec 00000000 00000001 dce400a0 00000000 c179e000 c17946b0 df7ace40 c140c820 00002102 439c2336 0000015a c179488c Call Trace: [] schedule+0x40/0x140 (36) [] down_write_mutex+0x1e1/0x290 (92) [] dev_queue_xmit_nit+0x41/0x130 (36) [] qdisc_restart+0x24f/0x280 (56) [] net_tx_action+0xbf/0x150 (44) [] ___do_softirq+0xc2/0x120 (48) [] _do_softirq+0x25/0x30 (8) [] ksoftirqd+0xc6/0x100 (24) [] kthread+0xbb/0xc0 (48) [] kernel_thread_helper+0x5/0xc (1048969236) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [] .... __schedule+0x4f/0x850 .....[] .. ( <= schedule+0x40/0x140) .. [] .... _spin_lock_irqsave+0x22/0x80 .....[] .. ( <= __schedule+0xee/0x850) rpciod/0/250's [current] stackdump: [] check_deadlock+0x1d3/0x270 [] task_blocks_on_sem+0x165/0x1c0 [] down_write_mutex+0x18b/0x290 [] __mutex_lock+0x44/0x60 [] _mutex_lock+0x1d/0x30 [] qdisc_restart+0x262/0x280 [] ip_finish_output+0xd9/0x280 [] ip_fragment+0x6bf/0x860 [] ip_output+0x8b/0xa0 [] ip_push_pending_frames+0x356/0x520 [] udp_push_pending_frames+0x160/0x260 [] udp_sendmsg+0x3b2/0x7b0 [] inet_sendmsg+0x50/0x60 [] sock_sendmsg+0xc0/0xe0 [] kernel_sendmsg+0x47/0x60 [] udp_sendpage+0x121/0x150 [] inet_sendpage+0x8f/0xc0 [] xdr_sendpages+0x17f/0x280 showing all tasks: s init/ 1 [c17953d0, 117] (not blocked) s migration/0/ 2 [c1794d40, 0] (not blocked) R ksoftirqd/0/ 3 [c17946b0, 105] blocked on: [c045cd80] {r:0,a:-1,ptype_lock} .. held by: rpciod/0/ 250 [df7ace40, 106] ... acquired at: qdisc_restart+0x24f/0x280 s desched/0/ 4 [c1794020, 105] (not blocked) s events/0/ 5 [c17b53f0, 98] (not blocked) s khelper/ 6 [c17b4d60, 115] (not blocked) s kthread/ 7 [c17b46d0, 107] (not blocked) s kblockd/0/ 8 [c17b4040, 105] (not blocked) s khubd/ 9 [dfdd5410, 125] (not blocked) s pdflush/ 10 [dfdd4d80, 118] (not blocked) s pdflush/ 11 [dfdd46f0, 116] (not blocked) s aio/0/ 13 [dfdfd430, 107] (not blocked) s kswapd0/ 12 [dfdd4060, 116] (not blocked) s IRQ 8/ 14 [dfdfcda0, 56] (not blocked) s IRQ 12/ 16 [dfdfc080, 54] (not blocked) s IRQ 6/ 17 [dfee5450, 50] (not blocked) s kseriod/ 15 [dfdfc710, 125] (not blocked) s IRQ 14/ 18 [dfee4dc0, 51] (not blocked) s IRQ 15/ 19 [dfee4730, 52] (not blocked) s IRQ 16/ 20 [dfee40a0, 53] (not blocked) s scsi_eh_0/ 21 [dff25470, 119] blocked on: [dff27fa8] {r:0,a:-1,sem.lock} .. held by: scsi_eh_0/ 21 [dff25470, 119] ... acquired at: scsi_error_handler+0x5a/0x1d0 s ahc_dv_0/ 22 [dff24de0, 116] blocked on: [dfed99a8] {r:0,a:-1,&ahc->platform_dat} .. held by: ahc_dv_0/ 22 [dff24de0, 116] ... acquired at: ahc_linux_release_simq+0xdd/0xf0 s IRQ 1/ 23 [dff240c0, 55] (not blocked) s kjournald/ 24 [dff24750, 116] (not blocked) s IRQ 4/ 153 [dfad40e0, 57] (not blocked) s IRQ 3/ 156 [dfad4770, 58] (not blocked) s IRQ 18/ 239 [df4f74b0, 59] (not blocked) s portmap/ 244 [df4f6790, 116] (not blocked) R rpciod/0/ 250 [df7ace40, 106] (not blocked) s lockd/ 251 [df7ac7b0, 118] (not blocked) s syslogd/ 335 [df7ac120, 116] (not blocked) s klogd/ 338 [df4f6100, 117] (not blocked) s famd/ 355 [dee7ee80, 119] (not blocked) s gpm/ 359 [df4f6e20, 116] (not blocked) s inetd/ 364 [dfad4e00, 120] (not blocked) s ntop/ 371 [dee7e7f0, 116] (not blocked) s ntop/ 460 [dce24850, 116] (not blocked) s ntop/ 461 [dd5d4180, 117] (not blocked) s ntop/ 462 [dcd1e1a0, 117] (not blocked) s ntop/ 463 [dcd1f550, 117] (not blocked) s ntop/ 487 [dc45d5b0, 117] (not blocked) s ntop/ 492 [dc8c8870, 117] (not blocked) R ntop/ 494 [dc8c81e0, 115] (not blocked) s sshd/ 380 [dd5d5530, 120] (not blocked) s xfs/ 390 [df7ad4d0, 116] (not blocked) s rpc.statd/ 478 [dd5d4ea0, 117] (not blocked) s atd/ 512 [dcd1eec0, 118] (not blocked) s cron/ 518 [dce241c0, 117] (not blocked) s getty/ 532 [dfad5490, 117] (not blocked) s getty/ 533 [dce24ee0, 117] (not blocked) s getty/ 534 [dcd1e830, 117] (not blocked) s getty/ 535 [dce25570, 117] (not blocked) s getty/ 536 [dc45c890, 117] (not blocked) s getty/ 538 [df35b5d0, 117] (not blocked) s getty/ 539 [df35af40, 116] (not blocked) s tcsh/ 540 [dee7e160, 117] (not blocked) s csh/ 577 [dd5d4810, 117] (not blocked) s mass_test/ 3320 [df35a220, 118] (not blocked) s find/ 3322 [dc45cf20, 119] (not blocked) R apt-get/ 9083 [caf51690, 115] (not blocked) R http/ 9706 [caf502e0, 115] (not blocked) R find/11732 [dee7f510, 120] (not blocked) --------------------------- | showing all locks held: | --------------------------- #001: [cac38168] {r:0,a:-1,kernel/fork.c:342} .. held by: find/11732 [dee7f510, 120] ... acquired at: exit_mmap+0x2a/0x130 #002: [c03fcc00] {r:0,a:-1,&zone->lru_lock} .. held by: find/11732 [dee7f510, 120] ... acquired at: __page_cache_release+0x2d/0x120 #003: [c045cd80] {r:0,a:-1,ptype_lock} .. held by: rpciod/0/ 250 [df7ace40, 106] ... acquired at: qdisc_restart+0x24f/0x280 #004: [dfefa1a4] {r:0,a:-1,&dev->xmit_lock} .. held by: ksoftirqd/0/ 3 [c17946b0, 105] ... acquired at: qdisc_restart+0x262/0x280 #005: [dfefa1e8] {r:0,a:-1,&dev->queue_lock} .. held by: rpciod/0/ 250 [df7ace40, 106] ... acquired at: dev_queue_xmit+0x1fe/0x2d0 #006: [c0455300] {r:0,a:-1,tasklist_lock} .. held by: rpciod/0/ 250 [df7ace40, 106] ... acquired at: show_all_locks+0x30/0x130 =============================================