From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752681Ab2G0APr (ORCPT ); Thu, 26 Jul 2012 20:15:47 -0400 Received: from va3ehsobe003.messaging.microsoft.com ([216.32.180.13]:49218 "EHLO va3outboundpool.messaging.microsoft.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752457Ab2G0APp (ORCPT ); Thu, 26 Jul 2012 20:15:45 -0400 X-Forefront-Antispam-Report: CIP:160.33.194.231;KIP:(null);UIP:(null);IPV:NLI;H:usculsndmail04v.am.sony.com;RD:mail.sonyusa.com;EFVD:NLI X-SpamScore: -3 X-BigFish: VPS-3(zzbb2dI98dI1432Izz1202hzz8275bh8275dhz2fh2a8h668h839hd25hf0ah107ah10d2h) Message-ID: <5011DD9B.1030901@am.sony.com> Date: Thu, 26 Jul 2012 17:15:23 -0700 From: Frank Rowand Reply-To: User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110428 Fedora/3.1.10-1.fc14 Thunderbird/3.1.10 MIME-Version: 1.0 To: Steven Rostedt , , CC: "linux-kernel@vger.kernel.org" , linux-rt-users , Thomas Gleixner , Carsten Emde , John Kacur Subject: Re: [PATCH RT 05/12] slab: Prevent local lock deadlock References: <20120718223910.898476857@goodmis.org> <20120718223957.856443728@goodmis.org> In-Reply-To: <20120718223957.856443728@goodmis.org> Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-OriginatorOrg: am.sony.com Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/18/12 15:39, Steven Rostedt wrote: > From: Thomas Gleixner > > On RT we avoid the cross cpu function calls and take the per cpu local > locks instead. Now the code missed that taking the local lock on the > cpu which runs the code must use the proper local lock functions and > not a simple spin_lock(). Otherwise it deadlocks later when trying to > acquire the local lock with the proper function. > > Reported-and-tested-by: Chris Pringle > Signed-off-by: Thomas Gleixner > Signed-off-by: Steven Rostedt > --- > mm/slab.c | 26 ++++++++++++++++++++++---- > 1 file changed, 22 insertions(+), 4 deletions(-) This patch leads to a warning during boot on the ARM pandaboard: [ 0.225097] Brought up 2 CPUs [ 0.225097] SMP: Total of 2 processors activated (2007.19 BogoMIPS). [ 0.225952] [ 0.225982] ============================================= [ 0.225982] [ INFO: possible recursive locking detected ] [ 0.225982] 3.0.36-rt58 #1 [ 0.225982] --------------------------------------------- [ 0.225982] swapper/0/1 is trying to acquire lock: [ 0.226013] (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [] do_ccupdate_local+0x18/0x44 [ 0.226043] [ 0.226043] but task is already holding lock: [ 0.226043] (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [] lock_slab_on+0x48/0x134 [ 0.226074] [ 0.226074] other info that might help us debug this: [ 0.226074] Possible unsafe locking scenario: [ 0.226074] [ 0.226074] CPU0 [ 0.226074] ---- [ 0.226074] lock(&per_cpu(slab_lock, __cpu).lock); [ 0.226104] lock(&per_cpu(slab_lock, __cpu).lock); [ 0.226104] [ 0.226104] *** DEADLOCK *** [ 0.226104] [ 0.226104] May be due to missing lock nesting notation [ 0.226104] [ 0.226104] 2 locks held by swapper/0/1: [ 0.226135] #0: (cache_chain_mutex){+.+.+.}, at: [] kmem_cache_create+0x74/0x4bc [ 0.226135] #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [] lock_slab_on+0x48/0x134 [ 0.226165] [ 0.226165] stack backtrace: [ 0.226196] [] (unwind_backtrace+0x0/0xf0) from [] (__lock_acquire+0x1984/0x1ce8) [ 0.226196] [] (__lock_acquire+0x1984/0x1ce8) from [] (lock_acquire+0x100/0x120) [ 0.226226] [] (lock_acquire+0x100/0x120) from [] (rt_spin_lock+0x4c/0x5c) [ 0.226257] [] (rt_spin_lock+0x4c/0x5c) from [] (do_ccupdate_local+0x18/0x44) [ 0.226257] [] (do_ccupdate_local+0x18/0x44) from [] (slab_on_each_cpu+0x2c/0x64) [ 0.226287] [] (slab_on_each_cpu+0x2c/0x64) from [] (do_tune_cpucache+0xd8/0x3e8) [ 0.226287] [] (do_tune_cpucache+0xd8/0x3e8) from [] (enable_cpucache+0x50/0xcc) [ 0.226318] [] (enable_cpucache+0x50/0xcc) from [] (kmem_cache_create+0x3d0/0x4bc) [ 0.226318] [] (kmem_cache_create+0x3d0/0x4bc) from [] (init_tmpfs+0x3c/0xe8) [ 0.226348] [] (init_tmpfs+0x3c/0xe8) from [] (kernel_init+0x80/0x150) [ 0.226379] [] (kernel_init+0x80/0x150) from [] (kernel_thread_exit+0x0/0x8) [ 0.239776] omap_hwmod: _populate_mpu_rt_base found no _mpu_rt_va for emif_fw [ 0.239776] omap_hwmod: _populate_mpu_rt_base found no _mpu_rt_va for l3_instr Config is from arch/arm/configs/omap2plus_defconfig plus: CONFIG_USB_EHCI_HCD=y CONFIG_USB_NET_SMSC95XX=y CONFIG_PREEMPT_RT_FULL=y -Frank