From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751738Ab2GSNFp (ORCPT ); Thu, 19 Jul 2012 09:05:45 -0400 Received: from hrndva-omtalb.mail.rr.com ([71.74.56.122]:21143 "EHLO hrndva-omtalb.mail.rr.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751114Ab2GSNFn (ORCPT ); Thu, 19 Jul 2012 09:05:43 -0400 X-Authority-Analysis: v=2.0 cv=ZuBv2qHG c=1 sm=0 a=s5Htg7xnQOKvHEu9STBOug==:17 a=OpT9cpI26MMA:10 a=mMwgJEC8K9oA:10 a=5SG0PmZfjMsA:10 a=Q9fys5e9bTEA:10 a=meVymXHHAAAA:8 a=ayC55rCoAAAA:8 a=xiBM9GFvRCpO1eyLGvoA:9 a=PUjeQqilurYA:10 a=s5Htg7xnQOKvHEu9STBOug==:117 X-Cloudmark-Score: 0 X-Originating-IP: 72.230.195.127 Message-ID: <1342703141.12353.24.camel@gandalf.stny.rr.com> Subject: Re: [PATCH RT 00/12] [ANNOUNCE] 3.0.36-rt58-rc1 stable review From: Steven Rostedt To: Mike Galbraith Cc: linux-kernel@vger.kernel.org, linux-rt-users , Thomas Gleixner , Carsten Emde , John Kacur Date: Thu, 19 Jul 2012 09:05:41 -0400 In-Reply-To: <1342670426.7342.8.camel@marge.simpson.net> References: <20120718223910.898476857@goodmis.org> <1342670426.7342.8.camel@marge.simpson.net> Content-Type: text/plain; charset="ISO-8859-15" X-Mailer: Evolution 3.4.3-1 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2012-07-19 at 06:00 +0200, Mike Galbraith wrote: > On Wed, 2012-07-18 at 18:39 -0400, Steven Rostedt wrote: > > > Please test the patches too. > > Your hotplug stress test script made x3550 M3 box fall over. It took a > bit, but down she went. 64 core test box fell over quickly, but that's > very far from virgin source.. seems to be the same though. Thanks for the report. I know a few areas in the hotplug code that can still deadlock (but are hard to hit). But there's no easy fix for them. Basically, the only thing we can do is redesign cpu hotplug (I think someone is already trying to do that ;-). But these patches do fix the main issues of cpu hotplug (albeit, making the code even uglier). The panic below isn't telling much. We really need to know what the other CPUs were up to. This call trace is just telling us that one of the CPUs is waiting for other CPUs to stop or to finish something up. -- Steve > > [ 255.016043] CPU 1 MCA<0>Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 7 > Pid: 9914, comm: migration/7 Not tainted 3.0.36-rt57 #49 > Call Trace: > [] panic+0x9b/0x1b0 > [] watchdog_overflow_callback+0xd7/0xe0 > [] __perf_event_overflow+0x9d/0x240 > [] ? perf_event_update_userpage+0x9b/0xe0 > [] perf_event_overflow+0x14/0x20 > [] intel_pmu_handle_irq+0x177/0x230 > [] perf_event_nmi_handler+0x39/0xc0 > [] notifier_call_chain+0x4d/0x70 > [] __atomic_notifier_call_chain+0x43/0x60 > [] atomic_notifier_call_chain+0x11/0x20 > [] notify_die+0x2e/0x30 > [] default_do_nmi+0x39/0x200 > [] do_nmi+0x78/0x80 > [] nmi+0x20/0x30 > [] ? stop_machine_cpu_stop+0x6a/0xe0 > <> [] cpu_stopper_thread+0xf4/0x1d0 > [] ? wait_for_stop_done+0xa0/0xa0 > [] ? __schedule+0x2c7/0x630 > [] ? cpu_stop_queue_work+0x70/0x70 > [] ? cpu_stop_queue_work+0x70/0x70 > [] kthread+0xa6/0xb0 > [] ? do_exit+0x278/0x450 > [] ? __switch_to+0xf2/0x370 > [] ? finish_task_switch+0x55/0xd0 > [] kernel_thread_helper+0x4/0x10 > [] ? __init_kthread_worker+0x50/0x50 > [] ? gs_change+0x13/0x13 >