From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752474AbcEKRiD (ORCPT ); Wed, 11 May 2016 13:38:03 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:45756 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752138AbcEKRh7 (ORCPT ); Wed, 11 May 2016 13:37:59 -0400 Date: Wed, 11 May 2016 19:37:52 +0200 From: Peter Zijlstra To: Matt Fleming Cc: mingo@kernel.org, linux-kernel@vger.kernel.org, clm@fb.com, mgalbraith@suse.de, tglx@linutronix.de, fweisbec@gmail.com Subject: Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared Message-ID: <20160511173752.GH3192@twins.programming.kicks-ass.net> References: <20160509104807.284575300@infradead.org> <20160509105210.642395937@infradead.org> <20160511115555.GT2839@codeblueprint.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160511115555.GT2839@codeblueprint.co.uk> User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 11, 2016 at 12:55:56PM +0100, Matt Fleming wrote: > This breaks my POWER7 box which presumably doesn't have SD_SHARE_PKG_RESOURCES, > index 978b3ef2d87e..d27153adee4d 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -7920,7 +7920,8 @@ static inline void set_cpu_sd_state_busy(void) > goto unlock; > sd->nohz_idle = 0; > > - atomic_inc(&sd->shared->nr_busy_cpus); > + if (sd->shared) > + atomic_inc(&sd->shared->nr_busy_cpus); > unlock: > rcu_read_unlock(); > } Ah, no, the problem is that while it does have SHARE_PKG_RESOURCES (in its SMT domain -- SMT threads share all cache after all), I failed to connect the sched_domain_shared structure for it. Does something like this also work? --- --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6389,10 +6389,6 @@ sd_init(struct sched_domain_topology_lev sd->cache_nice_tries = 1; sd->busy_idx = 2; - sd->shared = *per_cpu_ptr(sdd->sds, sd_id); - atomic_inc(&sd->shared->ref); - atomic_set(&sd->shared->nr_busy_cpus, sd_weight); - #ifdef CONFIG_NUMA } else if (sd->flags & SD_NUMA) { sd->cache_nice_tries = 2; @@ -6414,6 +6410,16 @@ sd_init(struct sched_domain_topology_lev sd->idle_idx = 1; } + /* + * For all levels sharing cache; connect a sched_domain_shared + * instance. + */ + if (sd->flags & SH_SHARED_PKG_RESOURCES) { + sd->shared = *per_cpu_ptr(sdd->sds, sd_id); + atomic_inc(&sd->shared->ref); + atomic_set(&sd->shared->nr_busy_cpus, sd_weight); + } + sd->private = sdd; return sd;