From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756928AbdDFJ0M (ORCPT ); Thu, 6 Apr 2017 05:26:12 -0400 Received: from Galois.linutronix.de ([146.0.238.70]:36043 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755707AbdDFJZq (ORCPT ); Thu, 6 Apr 2017 05:25:46 -0400 Date: Thu, 6 Apr 2017 11:25:42 +0200 From: Sebastian Andrzej Siewior To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Thomas Gleixner , Mike Galbraith , Ingo Molnar , "Rafael J . Wysocki" Subject: Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask Message-ID: <20170406092542.emc3lmqu2ywt3opy@linutronix.de> References: <20170404184202.20376-1-bigeasy@linutronix.de> <20170405073943.GA17266@gmail.com> <20170405083753.7eszej2njds4ovdb@linutronix.de> <20170406061622.GA19979@gmail.com> <20170406073832.e7bu4ldpfuq44ui6@linutronix.de> <20170406080139.GA22069@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20170406080139.GA22069@gmail.com> User-Agent: NeoMutt/20170306 (1.8.0) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2017-04-06 10:01:39 [+0200], Ingo Molnar wrote: > > * Sebastian Andrzej Siewior wrote: > > > On 2017-04-06 08:16:22 [+0200], Ingo Molnar wrote: > > > > > > * Sebastian Andrzej Siewior wrote: > > > > > > > On 2017-04-05 09:39:43 [+0200], Ingo Molnar wrote: > > > > > > > > > > So maybe we could add the following facility: > > > > > > > > > > ptr = sched_migrate_to_cpu_save(cpu); > > > > > > > > > > ... > > > > > > > > > > sched_migrate_to_cpu_restore(ptr); > > > > > > BTW., and I'm sure this has come up before, but why doesn't migrate_disable() use > > > a simple per task flag that the scheduler migration code takes into account? > > > > we could add that. But right now there are two spots which look at the > > counter to decide whether or not migration is disabled. > > > > > It should be functionally equivalent to the current solution, and it appears to > > > have a heck of a smaller cross section with the rest of the scheduler. > > > > > > I.e.: > > > > > > static inline void migrate_disable(void) > > > { > > > current->migration_disabled++; > > > } > > > > > > ... > > > > > > static inline void migrate_enable(void) > > > { > > > current->migration_disabled--; > > > } > > > > > > or so? Then add this flag as a condition to can_migrate_task() et al. > > > > > > While we generally dislike such flags as they wreck havoc with the scheduler if > > > overused, the cpus_allowed based solution has the exact same effect so it's not > > > like it's a step backwards - and it should also be much faster and less intrusive. > > > > So you are saying that we drop the cpus_ptr + cpus_mask fields again and > > instead add a task-flag to ensure that the scheduler does not migrate > > the task to another CPU? > > Yeah - but no need to add a per-task flag if we already have a counter. > > > > Am I missing some complication? > > > > We do have the counter. We have need to ensure that the CPU is not going away > > while we are in a migrate_disable() region since we can be scheduled out. So the > > CPU can't go offline until we leave that region. > > Yeah. But it should be relatively straightforward to extend the logic that makes > sure that a CPU does not go away from under tasks pinned to that CPU alone, right? I used get_online_cpus() that is enough. > > #define migrate_disable() sched_migrate_to_cpu_save(-1) > > > > int sched_migrate_to_cpu_save(int cpu) > > So if we have a ->migration_disabled counter then we don't need the > sched_migrate_to_cpu_save()/restore() complication, right? correct. Unless (as you suggested) we want a migrate to specific CPU we just the function until the BUG() statement and don't need to check the CPU we are on. > Sorry if this is a back and forth - I was somehow convinced that we do need to > frob the cpus_allowed mask to get this functionality - but in hindsight I think > the counter should be enough. > > I.e. just have a counter and these two APIs: > > static inline void migrate_disable(void) > { > current->migration_disabled++; plus if (current->migration_disabled == 1) get_online_cpus() > } > > ... > > static inline void migrate_enable(void) > { > current->migration_disabled--; > } > > ... and make sure the scheduler migration code plus the CPU hotplug code considers > the counter. So on the sched part I need go through all places where it looks at the mask and make sure it ignores the CPU switch decision. This could be doable. On the CPU hotplug I don't see a way around get_online_cpus(). It is essentially there to ensure a CPU does not go away. The other way around it would be to remove the CPU from the online_mask followed by going through all task left on the CPU waiting for them leave the CPU. And I remember peterz saying that he wanted to make get_online_cpus() fast and stay with that. > Would this work, and would this be the simplest all around solution? Let me try this with get_online_cpus() (as suggested) and without cpus_ptr. > Thanks, > > Ingo Sebastian