From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758325Ab0JWUdX (ORCPT ); Sat, 23 Oct 2010 16:33:23 -0400 Received: from mail-bw0-f46.google.com ([209.85.214.46]:52415 "EHLO mail-bw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753406Ab0JWUdV (ORCPT ); Sat, 23 Oct 2010 16:33:21 -0400 DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:from:to:cc:in-reply-to:references:content-type:date :message-id:mime-version:x-mailer:content-transfer-encoding; b=dKSriqVVXCDMN0oZPLfqxb9otMQnPcj2HGrYhxjqrok9w2ZiAxqBTBWLHU9sTJ3HEF tdyj55K20THkJKUvh/1MaypfDivcvrLQQZmSONPRvMwkh2txu7mSJO5+EA4lADGfphjy a3682jHq6mYTnN1YxMwsvYgN4AyzA1EYkTKgg= Subject: Re: [GIT PULL] Throtl bug (was Re: [origin tree boot failure] Re: [GIT PULL] core block bits for 2.6.37-rc1) From: Maxim Levitsky To: Jens Axboe Cc: Ingo Molnar , Tejun Heo , Linus Torvalds , "linux-kernel@vger.kernel.org" , Vivek Goyal In-Reply-To: <4CC32CE3.2090202@fusionio.com> References: <4CC143F5.3060202@fusionio.com> <20101023152959.GA20930@elte.hu> <4CC31283.1060607@fusionio.com> <4CC318AE.8010209@fusionio.com> <20101023182127.GA4160@elte.hu> <4CC32CE3.2090202@fusionio.com> Content-Type: text/plain; charset="UTF-8" Date: Sat, 23 Oct 2010 22:33:13 +0200 Message-ID: <1287865993.3322.3.camel@maxim-laptop> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, 2010-10-23 at 20:43 +0200, Jens Axboe wrote: > On 2010-10-23 20:21, Ingo Molnar wrote: > > > > * Jens Axboe wrote: > > > >>> Looks like a fairly straight forward case of uninitialized memory and > >>> blk_sync_queue() -> throtl_shutdown_timer() -> cancel_delayed_work_sync(). > >>> > >>> Will get that fixed up. > >> > >> It frees q->td in blk_cleanup_queue(), but doesn't clear q->td. When the final put > >> happens, blk_sync_queue() is called and then ends up doing the > >> cancel_delayed_work_sync() on freed memory. > >> > >> Two possible fixes: > >> > >> - Clear ->td when the queue is goin dead. May require other ->td == NULL > >> checks in the code, so I opted for: > >> > >> - Move the free to when the queue is really going away, post doing the > >> blk_sync_queue() call. > >> > >> The below should fix it. > >> > >> Signed-off-by: Jens Axboe > > > > This did the trick, thanks Jens! > > Great, thanks for testing/reporting! I added your reported/tested-by. > > Linus, please pull this single fix, better get this out the door since > I'll be travelling very shortly. > > > git://git.kernel.dk/linux-2.6-block.git for-2.6.37/core > > Jens Axboe (1): > block: fix use-after-free bug in blk throttle code > > block/blk-core.c | 2 -- > block/blk-sysfs.c | 2 ++ > 2 files changed, 2 insertions(+), 2 deletions(-) > I have here very similar bug. Must have been caused by this patch series. I pulled that tree, but that didn't affect anything. System oopses/panics on removal of any hotplugable device. (reproduced with xD, MemoryStick, and USB mass storage). Here is backtrace for MemoryStick card: <6>[ 24.138665] r592: IRQ: card removed <1>[ 24.228293] BUG: unable to handle kernel NULL pointer dereference at 00000000000001f8 <1>[ 24.228966] IP: [<00000000000001f8>] 0x1f8 <4>[ 24.230739] PGD 0 <0>[ 24.231182] Oops: 0010 [#1] PREEMPT SMP <0>[ 24.231182] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda/sda3/alignment_offset <4>[ 24.231182] CPU 1 <4>[ 24.231182] Modules linked in: dm_crypt firewire_net usb_storage usb_libusual cpufreq_powersave cpufreq_conservative cpufreq_userspace uvcvideo videodev v4l2_compat_ioctl32 acpi_cpufreq iwl3945 iwlcore snd_hda_codec_realtek mac80211 mperf r852 iTCO_wdt coretemp uhci_hcd sm_common ir_lirc_codec mspro_block snd_hda_intel ms_block ehci_hcd sdhci_pci lirc_dev joydev sbp2 nand snd_hda_codec cfg80211 firewire_ohci sdhci ir_sony_decoder ieee1394 nand_ids usbcore r592 ir_jvc_decoder snd_hwdep mmc_core nand_ecc ir_rc6_decoder ene_ir snd_pcm tg3 ir_rc5_decoder firewire_core mtd battery memstick ac ir_nec_decoder psmouse snd_page_alloc libphy sunrpc ir_core sg evdev serio_raw dm_mirror dm_region_hash dm_log dm_mod nouveau ttm drm_kms_helper drm i2c_algo_bit thermal video <4>[ 32.881606] <4>[ 32.881606] Pid: 543, comm: kworker/u:4 Not tainted 2.6.36+ #191 Nettiling/Aspire 5720 <4>[ 32.881606] RIP: 0010:[<00000000000001f8>] [<00000000000001f8>] 0x1f8 <4>[ 32.881606] RSP: 0018:ffff880037a03ab8 EFLAGS: 00010086 <4>[ 32.881606] RAX: ffff88007c0ebc00 RBX: ffff880037af9470 RCX: 0000000000000000 <4>[ 32.881606] RDX: 0000000000000019 RSI: 0000000000000001 RDI: ffff880037af9470 <4>[ 32.881606] RBP: ffff880037a03ad0 R08: 0000000000000000 R09: 0000000000000001 <4>[ 32.881606] R10: 00000000000002f0 R11: 0000000000000000 R12: ffff880037af9470 <4>[ 32.881606] R13: ffff880075d6a870 R14: ffff880075bfb560 R15: 0000000000000282 <4>[ 32.881606] FS: 0000000000000000(0000) GS:ffff88007f200000(0000) knlGS:0000000000000000 <4>[ 32.881606] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b <4>[ 32.881606] CR2: 00000000000001f8 CR3: 000000007a046000 CR4: 00000000000006e0 <4>[ 32.881606] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 <4>[ 32.881606] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 <4>[ 32.881606] Process kworker/u:4 (pid: 543, threadinfo ffff880037a02000, task ffff88007c5b0000) <0>[ 32.881606] Stack: <4>[ 32.881606] ffffffff811c42a2 ffff880037a03af0 ffff880037af9470 ffff880037a03af0 <4>[ 32.881606] <0> ffffffff811c525a ffff880077250040 ffff880077250040 ffff880037a03b10 <4>[ 32.881606] <0> ffffffff811cebb2 ffff880075d6a800 ffff880075d6a8a8 ffff880037a03b30 <0>[ 32.881606] Call Trace: <4>[ 32.881606] [] ? elv_drain_elevator+0x22/0x70 <4>[ 32.881606] [] elv_quiesce_start+0x3a/0xc0 <4>[ 32.881606] [] disk_replace_part_tbl+0x42/0x70 <4>[ 32.881606] [] disk_release+0x23/0x50 <4>[ 32.881606] [] device_release+0x22/0x90 <4>[ 32.881606] [] kobject_release+0x8d/0x1a0 <4>[ 32.881606] [] ? kobject_release+0x0/0x1a0 <4>[ 32.881606] [] kref_put+0x37/0x70 <4>[ 32.881606] [] kobject_put+0x27/0x60 <4>[ 32.881606] [] put_disk+0x12/0x20 <4>[ 32.881606] [] mspro_block_disk_release+0xa3/0xb0 [mspro_block] <4>[ 32.881606] [] mspro_block_remove+0xcd/0x140 [mspro_block] <4>[ 32.881606] [] memstick_device_remove+0x35/0x60 [memstick] <4>[ 32.881606] [] __device_release_driver+0x70/0xe0 <4>[ 32.881606] [] device_release_driver+0x2a/0x40 <4>[ 32.881606] [] bus_remove_device+0xb5/0x120 <4>[ 32.881606] [] device_del+0x127/0x1d0 <4>[ 32.881606] [] device_unregister+0x1d/0x60 <4>[ 32.881606] [] memstick_check+0x241/0x360 [memstick] <4>[ 32.881606] [] process_one_work+0x1c0/0x4d0 <4>[ 32.881606] [] ? process_one_work+0x162/0x4d0 <4>[ 32.881606] [] ? memstick_check+0x0/0x360 [memstick] <4>[ 32.881606] [] worker_thread+0x156/0x410 <4>[ 32.881606] [] ? worker_thread+0x0/0x410 <4>[ 32.881606] [] kthread+0xb6/0xc0 <4>[ 32.881606] [] ? finish_task_switch+0x46/0xe0 <4>[ 32.881606] [] kernel_thread_helper+0x4/0x10 <4>[ 32.881606] [] ? kthread+0x0/0xc0 <4>[ 32.881606] [] ? kernel_thread_helper+0x0/0x10 <0>[ 32.881606] Code: Bad RIP value. <1>[ 32.881606] RIP [<00000000000001f8>] 0x1f8 <4>[ 32.881606] RSP <0>[ 32.881606] CR2: 00000000000001f8 <4>[ 32.881606] ---[ end trace ca0206dec4457aff ]--- Best regards, Maxim Levitsky