From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753967Ab3LJOhI (ORCPT ); Tue, 10 Dec 2013 09:37:08 -0500 Received: from mail-wi0-f180.google.com ([209.85.212.180]:41570 "EHLO mail-wi0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753807Ab3LJOhG (ORCPT ); Tue, 10 Dec 2013 09:37:06 -0500 Date: Tue, 10 Dec 2013 15:37:03 +0100 From: Frederic Weisbecker To: "Paul E. McKenney" Cc: linux-kernel@vger.kernel.org Subject: Re: Will CPU 0 be forever prohibited from NO_HZ_FULL status? Message-ID: <20131210143702.GB10633@localhost.localdomain> References: <20131204193957.GA2847@linux.vnet.ibm.com> <20131205012053.GC6683@localhost.localdomain> <20131205025037.GM15492@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20131205025037.GM15492@linux.vnet.ibm.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Dec 04, 2013 at 06:50:37PM -0800, Paul E. McKenney wrote: > On Thu, Dec 05, 2013 at 02:20:55AM +0100, Frederic Weisbecker wrote: > > On Wed, Dec 04, 2013 at 11:39:57AM -0800, Paul E. McKenney wrote: > > > Hello, Frederic, > > > > > > Just realized that I could further decrease RT latency of one of my "shut > > > up RCU on NO_HZ_FULL CPUs" patches if I relied on CPU 0 always having > > > a scheduling-clock tick unless the entire system is idle. The trick > > > is that I could then rely on CPU 0 to detect RCU CPU stall warnings, > > > and remove the checking from the other CPUs. > > > > > > Thoughts? > > > > You're right on time as I'm currently working on that :) > > So the plan is to allow timekeeping to be handled by a set of CPUs (cpu_housekeeping_mask > > which I guess should be ~nohz_full_mask & cpu_online_mask). I think it will be better > > for powersaving. I guess you could balance the RCU stall checks in this > > set of housekeeping CPUs? > > > > It should be very easy to make the rcu sysidle stuff to support that housekeeping set, > > I just looked into it and all we need to do is to turn the several "cpu == tick_do_timer" > > checks into something like is_housekeeping_cpu(cpu). And may be a few easy details, like which > > CPU from the housekeeping set should get the kick IPI, well the first one available should be a good start, > > of course I expect some issues with cpu hotplug. > > But other than that, RCU sysidle detection is mostly ready to support tracking only a given subset > > of CPUs instead of all of them. That's in fact what it already does currently by excluding the > > fixed boot timekeeping CPU. > > > > So I'm working on that and should have some patches ready soon. > > Thank you for the info! Nice to know that RCU will continue to be able > to rely on there being at least one housekeeping CPU. ;-) > > At that point, tick_nohz_full_cpu() would still be a good way for RCU > to distinguish housekeeping CPUs from working CPUs, correct? Correct! > > > In fact I just realized that all the sysidle detection infrastructure is there and working > > but we forgot to plug it in the tick engine, and thus we are still running > > with periodic CPU 0 even with CONFIG_NO_HZ_FULL_SYSIDLE=y. Anyway I have a few changes > > ready to enable that, lets hope testing will be ok :) > > Indeed! ;-) > > The CONFIG_NO_HZ_FULL_SYSIDLE=y might complicate things a bit. But I > guess the problem would be a corner case -- the system entered sysidle > mode with a grace period pending, which should eventually wake up the > corresponding grace-period kthread, which might be prevented from ever > running due to high load or something. If that problem arises, I will > fix it. I see. Well we'll find out. In the meantime I successfully plugged sysidle detection with full dynticks and it surprisingly works like a charm. Which makes me think there must be some bug in my patches that make things working by accident :) I'll post soon. Thanks.