From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753306AbaBEQ5H (ORCPT ); Wed, 5 Feb 2014 11:57:07 -0500 Received: from cdptpa-outbound-snat.email.rr.com ([107.14.166.226]:33317 "EHLO cdptpa-oedge-vip.email.rr.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753091AbaBEQ5G (ORCPT ); Wed, 5 Feb 2014 11:57:06 -0500 Date: Wed, 5 Feb 2014 11:57:00 -0500 From: Steven Rostedt To: Steven Rostedt Cc: LKML , linux-rt-users , Thomas Gleixner , Sebastian Andrzej Siewior , Clark Williams , "Luis Claudio R. Goncalves" , John Kacur , Ulrich Obergfell , stable-rt@vger.kernel.org Subject: Re: [PATCH RT] rt: Make cpu_chill() use hrtimer instead of msleep() Message-ID: <20140205115700.3107d0e2@gandalf.local.home> In-Reply-To: <20140205115125.3cc950ce@gandalf.local.home> References: <20140205115125.3cc950ce@gandalf.local.home> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-RR-Connecting-IP: 107.14.168.142:25 X-Cloudmark-Score: 0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 5 Feb 2014 11:51:25 -0500 Steven Rostedt wrote: > > Ulrich Obergfell pointed out that cpu_chill() calls msleep() which is woken > up by the ksoftirqd running the TIMER softirq. But as the cpu_chill() is > called from softirq context, it may block the ksoftirqd() from running, in > which case, it may never wake up the msleep() causing the deadlock. > > I checked the vmcore, and irq/74-qla2xxx is stuck in the msleep() call, > running on CPU 8. The one ksoftirqd that is stuck, happens to be the one that > runs on CPU 8, and it is blocked on a lock held by irq/74-qla2xxx. As that > ksoftirqd is the one that will wake up irq/74-qla2xxx, and it happens to be > blocked on a lock that irq/74-qla2xxx holds, we have our deadlock. > > The solution is not to convert the cpu_chill() back to a cpu_relax() as that > will re-create a possible live lock that the cpu_chill() fixed earlier, and may > also leave this bug open on other softirqs. The fix is to remove the > dependency on ksoftirqd from cpu_chill(). That is, instead of calling > msleep() that requires ksoftirqd to wake it up, use the > hrtimer_nanosleep() code that does the wakeup from hard irq context. > > Found-by: Ulrich Obergfell > Signed-off-by: Steven Rostedt > I should have added: Cc: stable-rt@vger.kernel.org -- Steve