From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751324AbdKUPYk (ORCPT ); Tue, 21 Nov 2017 10:24:40 -0500 Received: from mail.kernel.org ([198.145.29.99]:39038 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751145AbdKUPYj (ORCPT ); Tue, 21 Nov 2017 10:24:39 -0500 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E8CF2192B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=goodmis.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=rostedt@goodmis.org Date: Tue, 21 Nov 2017 10:24:36 -0500 From: Steven Rostedt To: Clark Williams Cc: Sebastian Andrzej Siewior , LKML , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Daniel Bristot de Oliveira , John Kacur , Scott Wood , Mike Galbraith Subject: Re: [PATCH RT] fix IPI balancing for 4.14-rt Message-ID: <20171121102436.3e25655f@gandalf.local.home> In-Reply-To: <20171121091425.7577dc02@tagon> References: <20171006140504.48dc32c1@gandalf.local.home> <20171121091425.7577dc02@tagon> X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 21 Nov 2017 09:14:25 -0600 Clark Williams wrote: > From 8ea8311b75a40bdea03e7f8228a0578b6367e9d1 Mon Sep 17 00:00:00 2001 > From: Clark Williams > Date: Mon, 20 Nov 2017 14:26:12 -0600 > Subject: [PATCH] [rt] sched/rt: fix panic in double_lock_balance with > simplified IPI RT balancing > > I was testing 4.14-rt1 on a large system (cores == 96) and saw that > we were getting into an rt balancing storm, so I tried applying Steven's > patch (not upstream yet): > > sched/rt: Simplify the IPI rt balancing logic > > Booting the resulting kernel yielded a panic in > double_lock_balance() due to irqs not being disabled. > > This patch changes the calls to raw_spin_{lock,unlock} in the > function rto_push_irq_work_function, to be raw_spin_{lock,unlock}_irq. > Not sure if that's too heavy a hammer, but the resulting kernel boots > and runs and survives 12h runs of rteval. Once Steven's patch goes in > upstream, we'll need something like this in RT. > > Signed-off-by: Clark Williams > --- > kernel/sched/rt.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c > index 57fb251dd8ce..a5cd0cea2f0f 100644 > --- a/kernel/sched/rt.c > +++ b/kernel/sched/rt.c > @@ -2008,9 +2008,9 @@ void rto_push_irq_work_func(struct irq_work *work) > * When it gets updated, a check is made if a push is possible. > */ > if (has_pushable_tasks(rq)) { > - raw_spin_lock(&rq->lock); > + raw_spin_lock_irq(&rq->lock); > push_rt_tasks(rq); > - raw_spin_unlock(&rq->lock); > + raw_spin_unlock_irq(&rq->lock); This looks buggy to me. You know you just indiscriminately enabled interrupts here. > } > > raw_spin_lock(&rq->rd->rto_lock); Why is this patch necessary? Is it because you have the irq_work running in non hard irq context? I think you need something like this instead (if you haven't already added it): -- Steve Index: linux-rt.git/kernel/sched/topology.c =================================================================== --- linux-rt.git.orig/kernel/sched/topology.c +++ linux-rt.git/kernel/sched/topology.c @@ -257,6 +257,7 @@ static int init_rootdomain(struct root_d rd->rto_cpu = -1; raw_spin_lock_init(&rd->rto_lock); init_irq_work(&rd->rto_push_work, rto_push_irq_work_func); + rd->rto_push_work.flags |= IRQ_WORK_HARD_IRQ; #endif init_dl_bw(&rd->dl_bw);