From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933927AbYD1M34 (ORCPT ); Mon, 28 Apr 2008 08:29:56 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1765226AbYD1M3s (ORCPT ); Mon, 28 Apr 2008 08:29:48 -0400 Received: from pentafluge.infradead.org ([213.146.154.40]:38798 "EHLO pentafluge.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1765067AbYD1M3r (ORCPT ); Mon, 28 Apr 2008 08:29:47 -0400 Subject: AIO/DIO lockup/crash From: Peter Zijlstra To: linux-kernel , linux-aio Cc: Zach Brown , Clark Williams Content-Type: text/plain Date: Mon, 28 Apr 2008 14:29:42 +0200 Message-Id: <1209385782.13978.17.camel@twins> Mime-Version: 1.0 X-Mailer: Evolution 2.22.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi guys, I'm getting this (and various variations thereof - like crashing in the PI chain code on -rt) when running aio-dio-invalidate-failure for a few hours. (dual core opteron - single spindle - ext3) Is this a known issue? I'll run the same on current -git overnight to see if it went away :-) [ 1796.238953] BUG: soft lockup - CPU#1 stuck for 11s! [aio-dio-invalid:3037] [ 1796.245794] CPU 1: [ 1796.247802] Modules linked in: autofs4 binfmt_misc ext2 psmouse evbug evdev i2c_piix4 i2c_core pcspkr thermal processor button sr_mod cdrom sg shpchp pci_hotplug sd_mod ext3 jbd mbcache uhci_hcd ohci_hcd ehci_hcd usbcore [ 1796.267532] Pid: 3037, comm: aio-dio-invalid Not tainted 2.6.24.4 #194 [ 1796.274023] RIP: 0010:[] [] _spin_lock_irqsave+0x63/0x90 [ 1796.282517] RSP: 0018:ffff81007fba7ce0 EFLAGS: 00000246 [ 1796.287800] RAX: 0000000000000000 RBX: ffff81007fba7cf0 RCX: 0000000000001000 [ 1796.294895] RDX: 0000000000000213 RSI: ffff810067dbc740 RDI: 0000000000000001 [ 1796.301993] RBP: ffff81007fba7c60 R08: 0000000000000101 R09: 000000000169aa28 [ 1796.309090] R10: 000000000169aa28 R11: 0000000000000003 R12: ffffffff8020d0c6 [ 1796.316187] R13: ffff81007fba7c60 R14: ffff81007eaddc00 R15: ffff81007eaddf24 [ 1796.323283] FS: 00002b489f45db00(0000) GS:ffff81007fb6cac0(0000) knlGS:0000000000000000 [ 1796.331330] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1796.337043] CR2: 00000000008c7f1c CR3: 0000000068610000 CR4: 00000000000006e0 [ 1796.344140] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1796.351237] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1796.358334] [ 1796.358334] Call Trace: [ 1796.362244] [] dio_bio_end_aio+0x3a/0xe0 [ 1796.368405] [] bio_endio+0x19/0x40 [ 1796.373430] [] req_bio_endio+0x4e/0xa0 [ 1796.378800] [] __end_that_request_first+0x1a4/0x3c0 [ 1796.385292] [] end_that_request_chunk+0x9/0x10 [ 1796.391354] [] scsi_end_request+0x3b/0x110 [ 1796.397069] [] scsi_io_completion+0xa5/0x3b0 [ 1796.402958] [] _spin_unlock_irqrestore+0x16/0x40 [ 1796.409192] [] scsi_finish_command+0x99/0xf0 [ 1796.415079] [] scsi_softirq_done+0x115/0x150 [ 1796.420967] [] blk_done_softirq+0x6b/0x80 [ 1796.426598] [] __do_softirq+0x64/0xd0 [ 1796.431883] [] call_softirq+0x1c/0x30 [ 1796.437166] [] do_softirq+0x3d/0x90 [ 1796.442276] [] irq_exit+0x88/0xa0 [ 1796.447213] [] do_IRQ+0x85/0x100 [ 1796.452064] [] ret_from_intr+0x0/0xa [ 1796.457258] [] _spin_lock_irqsave+0x6e/0x90 [ 1796.463678] [] _spin_lock_irqsave+0x3e/0x90 [ 1796.469479] [] dio_bio_submit+0x2d/0x90 [ 1796.474935] [] dio_send_cur_page+0x9e/0xa0 [ 1796.480648] [] submit_page_section+0x3e/0x130 [ 1796.486623] [] __blockdev_direct_IO+0x979/0xc50 [ 1796.492783] [] :ext3:ext3_direct_IO+0xaf/0x1c0 [ 1796.498847] [] :ext3:ext3_get_block+0x0/0x110 [ 1796.504825] [] generic_file_direct_IO+0xba/0x160 [ 1796.511059] [] generic_file_direct_write+0x6f/0x130 [ 1796.517551] [] __generic_file_aio_write_nolock+0x383/0x440 [ 1796.524650] [] generic_file_aio_write+0x64/0xd0 [ 1796.530802] [] :ext3:ext3_file_write+0x26/0xc0 [ 1796.536865] [] :ext3:ext3_file_write+0x0/0xc0 [ 1796.542841] [] aio_rw_vect_retry+0x6f/0x180 [ 1796.548642] [] aio_rw_vect_retry+0x0/0x180 [ 1796.554355] [] aio_run_iocb+0x49/0x110 [ 1796.559725] [] io_submit_one+0x1d3/0x3f0 [ 1796.565268] [] sys_io_submit+0xde/0x140 [ 1796.570725] [] tracesys+0xdc/0xe1 [ 1796.575661]