From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751476Ab1GVXVH (ORCPT ); Fri, 22 Jul 2011 19:21:07 -0400 Received: from mail001.aei.ca ([206.123.6.130]:49451 "EHLO mail001.aei.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751331Ab1GVXVE (ORCPT ); Fri, 22 Jul 2011 19:21:04 -0400 From: Ed Tomlinson To: Linux Kernel Mailing List Subject: Re: Linux 3.0 release - btrfs possible locking deadlock Date: Fri, 22 Jul 2011 19:21:00 -0400 User-Agent: KMail/1.13.7 (Linux/3.0.0-crc+; KDE/4.6.5; x86_64; ; ) Cc: Linus Torvalds , Chris Mason , linux-btrfs@vger.kernel.org, Josef Bacik References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201107221921.00798.edt@aei.ca> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thursday 21 July 2011 22:59:53 Linus Torvalds wrote: > So there it is. Gone are the 2.6. days, and 3.0 is out. > Hi, Managed to get this with btrfs rsync(ing) from ext4 to a btrfs fs with three partitions using raid1. [16018.211493] device fsid f7186eeb-60df-4b1a-890a-4a1eb42f81fe devid 1 transid 10 /dev/sdd4 [16018.230643] btrfs: use lzo compression [16018.234619] btrfs: enabling disk space caching [25949.414011] [25949.414011] ======================================================= [25949.416549] [ INFO: possible circular locking dependency detected ] [25949.423187] 3.0.0-crc+ #348 [25949.423187] ------------------------------------------------------- [25949.423187] rsync/20237 is trying to acquire lock: [25949.423187] (btrfs-extent-01){+.+...}, at: [] btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [25949.423187] but task is already holding lock: [25949.423187] (&(&eb->lock)->rlock){+.+...}, at: [] btrfs_clear_lock_blocking+0x22/0x30 [btrfs] [25949.423187] [25949.423187] which lock already depends on the new lock. [25949.423187] [25949.423187] [25949.423187] the existing dependency chain (in reverse order) is: [25949.423187] [25949.423187] -> #1 (&(&eb->lock)->rlock){+.+...}: [25949.423187] [] lock_acquire+0x95/0x140 [25949.423187] [] _raw_spin_lock+0x3b/0x50 [25949.423187] [] btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] btrfs_search_slot+0x2e9/0x800 [btrfs] [25949.423187] [] lookup_inline_extent_backref+0xbe/0x490 [btrfs] [25949.423187] [] __btrfs_free_extent+0x13b/0x900 [btrfs] [25949.423187] [] run_clustered_refs+0x823/0xaf0 [btrfs] [25949.423187] [] btrfs_run_delayed_refs+0xcd/0x290 [btrfs] [25949.423187] [] btrfs_commit_transaction+0x8b/0x9d0 [btrfs] [25949.423187] [] transaction_kthread+0x2b6/0x2e0 [btrfs] [25949.423187] [] kthread+0xb6/0xc0 [25949.423187] [] kernel_thread_helper+0x4/0x10 [25949.423187] [25949.423187] -> #0 (btrfs-extent-01){+.+...}: [25949.423187] [] __lock_acquire+0x1588/0x16a0 [25949.423187] [] lock_acquire+0x95/0x140 [25949.423187] [] _raw_spin_lock+0x3b/0x50 [25949.423187] [] btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] btrfs_search_slot+0x2e9/0x800 [btrfs] [25949.423187] [] btrfs_lookup_dir_item+0x82/0x120 [btrfs] [25949.423187] [] btrfs_lookup_dentry+0xc5/0x4c0 [btrfs] [25949.423187] [] btrfs_lookup+0x24/0x70 [btrfs] [25949.423187] [] d_alloc_and_lookup+0xc3/0x100 [25949.423187] [] do_lookup+0x260/0x480 [25949.423187] [] walk_component+0x60/0x1f0 [25949.423187] [] path_lookupat+0xea/0x620 [25949.423187] [] do_path_lookup+0x35/0x1c0 [25949.423187] [] user_path_at+0x98/0xe0 [25949.423187] [] vfs_fstatat+0x4c/0x90 [25949.423187] [] vfs_lstat+0x1e/0x20 [25949.423187] [] sys_newlstat+0x24/0x50 [25949.423187] [] system_call_fastpath+0x16/0x1b [25949.423187] [25949.423187] other info that might help us debug this: [25949.423187] [25949.423187] Possible unsafe locking scenario: [25949.423187] [25949.423187] CPU0 CPU1 [25949.423187] ---- ---- [25949.423187] lock(&(&eb->lock)->rlock); [25949.423187] lock(btrfs-extent-01); [25949.423187] lock(&(&eb->lock)->rlock); [25949.423187] lock(btrfs-extent-01); [25949.423187] [25949.423187] *** DEADLOCK *** [25949.423187] [25949.423187] 2 locks held by rsync/20237: [25949.423187] #0: (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [] do_lookup+0x21a/0x480 [25949.423187] #1: (&(&eb->lock)->rlock){+.+...}, at: [] btrfs_clear_lock_blocking+0x22/0x30 [btrfs] [25949.423187] [25949.423187] stack backtrace: [25949.423187] Pid: 20237, comm: rsync Not tainted 3.0.0-crc+ #348 [25949.423187] Call Trace: [25949.423187] [] print_circular_bug+0x20e/0x2f0 [25949.423187] [] __lock_acquire+0x1588/0x16a0 [25949.423187] [] ? verify_parent_transid+0xcb/0x290 [btrfs] [25949.423187] [] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] lock_acquire+0x95/0x140 [25949.423187] [] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] _raw_spin_lock+0x3b/0x50 [25949.423187] [] ? btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] btrfs_try_spin_lock+0x78/0xb0 [btrfs] [25949.423187] [] btrfs_search_slot+0x2e9/0x800 [btrfs] [25949.423187] [] ? __lock_acquire+0x1ea/0x16a0 [25949.423187] [] btrfs_lookup_dir_item+0x82/0x120 [btrfs] [25949.423187] [] ? kmem_cache_alloc+0xde/0x1e0 [25949.423187] [] btrfs_lookup_dentry+0xc5/0x4c0 [btrfs] [25949.423187] [] ? do_raw_spin_lock+0xde/0x1c0 [25949.423187] [] ? sub_preempt_count+0x51/0x60 [25949.423187] [] btrfs_lookup+0x24/0x70 [btrfs] [25949.423187] [] d_alloc_and_lookup+0xc3/0x100 [25949.423187] [] do_lookup+0x260/0x480 [25949.423187] [] walk_component+0x60/0x1f0 [25949.423187] [] path_lookupat+0xea/0x620 [25949.423187] [] ? might_fault+0x53/0xb0 [25949.423187] [] do_path_lookup+0x35/0x1c0 [25949.423187] [] user_path_at+0x98/0xe0 [25949.423187] [] ? might_fault+0x9c/0xb0 [25949.423187] [] ? might_fault+0x53/0xb0 [25949.423187] [] ? cp_new_stat+0xf8/0x110 [25949.423187] [] vfs_fstatat+0x4c/0x90 [25949.423187] [] vfs_lstat+0x1e/0x20 [25949.423187] [] sys_newlstat+0x24/0x50 [25949.423187] [] ? trace_hardirqs_on_caller+0x14d/0x190 [25949.423187] [] ? trace_hardirqs_on_thunk+0x3a/0x3f [25949.423187] [] system_call_fastpath+0x16/0x1b Kernel is 3.0.0 without any extras. Ideas? Ed