From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1757802AbbBEPVt (ORCPT <rfc822;w@1wt.eu>);
	Thu, 5 Feb 2015 10:21:49 -0500
Received: from casper.infradead.org ([85.118.1.10]:56262 "EHLO
	casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1753480AbbBEPVs (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Thu, 5 Feb 2015 10:21:48 -0500
Date: Thu, 5 Feb 2015 16:21:44 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: LKML <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Clark Williams <williams@redhat.com>,
        linux-rt-users <linux-rt-users@vger.kernel.org>,
        Mike Galbraith <umgwanakikbuti@gmail.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: [RFC][PATCH] sched/rt: Use IPI to trigger RT task push migration
 instead of pulling
Message-ID: <20150205152144.GM21418@twins.programming.kicks-ass.net>
References: <20150204143906.742287b4@gandalf.local.home>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150204143906.742287b4@gandalf.local.home>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Feb 04, 2015 at 02:39:06PM -0500, Steven Rostedt wrote:
> The problem is that if you have 100 CPUs having an RT task schedule out
> (just like cyclictest -d0 -t -i100 does a lot), and if there's one rq
> that has more than one RT task scheduled on it (overloaded), each
> of those 100 CPUs are going to try to get that second RT task on that
> lonely rq, and each of those 100 CPUs are going to take that lonely
> rq's lock! Then what happens if the task running on that rq wants to
> schedule? Well, it needs to wait behind 100 CPUs contending for its rq
> lock, and we see a HUGE latency (1ms or more).
> 
> What does this patch do instead? Instead of trying to fight for the rq,
> if it sees that there's an overloaded rq, it simply sends an IPI to
> that CPU to have it push the overloaded RT task off to another CPU.

So can't we flip the problem around; 99 overloaded cpus and 1 going
'low', then we IPI the 99, and they're all going to try and push their
tasks on the one (now) sad cpu?