From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753188AbdKWRZn (ORCPT ); Thu, 23 Nov 2017 12:25:43 -0500 Received: from Galois.linutronix.de ([146.0.238.70]:44318 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753023AbdKWRZm (ORCPT ); Thu, 23 Nov 2017 12:25:42 -0500 Date: Thu, 23 Nov 2017 18:25:39 +0100 From: Sebastian Andrzej Siewior To: Steven Rostedt Cc: Clark Williams , LKML , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Daniel Bristot de Oliveira , John Kacur , Scott Wood , Mike Galbraith Subject: Re: [PATCH RT] fix IPI balancing for 4.14-rt Message-ID: <20171123172538.GB32454@linutronix.de> References: <20171006140504.48dc32c1@gandalf.local.home> <20171121091425.7577dc02@tagon> <20171121102436.3e25655f@gandalf.local.home> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20171121102436.3e25655f@gandalf.local.home> User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2017-11-21 10:24:36 [-0500], Steven Rostedt wrote: > On Tue, 21 Nov 2017 09:14:25 -0600 > Clark Williams wrote: > > > I was testing 4.14-rt1 on a large system (cores == 96) and saw that > > we were getting into an rt balancing storm, so I tried applying Steven's > > patch (not upstream yet): > > > > sched/rt: Simplify the IPI rt balancing logic > > > Why is this patch necessary? > > Is it because you have the irq_work running in non hard irq context? I > think you need something like this instead (if you haven't already > added it): I cherry-picked commit 4bdced5c9a29 ("sched/rt: Simplify the IPI based RT balancing logic") and while refreshing the queue I noticed that the irq_work struct moved and added the fix below into the original patch where the IRQ_WORK_HARD_IRQ flag was added. > -- Steve > > Index: linux-rt.git/kernel/sched/topology.c > =================================================================== > --- linux-rt.git.orig/kernel/sched/topology.c > +++ linux-rt.git/kernel/sched/topology.c > @@ -257,6 +257,7 @@ static int init_rootdomain(struct root_d > rd->rto_cpu = -1; > raw_spin_lock_init(&rd->rto_lock); > init_irq_work(&rd->rto_push_work, rto_push_irq_work_func); > + rd->rto_push_work.flags |= IRQ_WORK_HARD_IRQ; > #endif > > init_dl_bw(&rd->dl_bw); Sebastian