From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934169AbdDFHip convert rfc822-to-8bit (ORCPT ); Thu, 6 Apr 2017 03:38:45 -0400 Received: from Galois.linutronix.de ([146.0.238.70]:35627 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932371AbdDFHii (ORCPT ); Thu, 6 Apr 2017 03:38:38 -0400 Date: Thu, 6 Apr 2017 09:38:33 +0200 From: Sebastian Andrzej Siewior To: Ingo Molnar Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Thomas Gleixner , Mike Galbraith , Ingo Molnar , "Rafael J . Wysocki" Subject: Re: [RFC PATCH] kernel: sched: Provide a pointer to the valid CPU mask Message-ID: <20170406073832.e7bu4ldpfuq44ui6@linutronix.de> References: <20170404184202.20376-1-bigeasy@linutronix.de> <20170405073943.GA17266@gmail.com> <20170405083753.7eszej2njds4ovdb@linutronix.de> <20170406061622.GA19979@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8BIT In-Reply-To: <20170406061622.GA19979@gmail.com> User-Agent: NeoMutt/20170306 (1.8.0) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2017-04-06 08:16:22 [+0200], Ingo Molnar wrote: > > * Sebastian Andrzej Siewior wrote: > > > On 2017-04-05 09:39:43 [+0200], Ingo Molnar wrote: > > > > > > So maybe we could add the following facility: > > > > > > ptr = sched_migrate_to_cpu_save(cpu); > > > > > > ... > > > > > > sched_migrate_to_cpu_restore(ptr); > > BTW., and I'm sure this has come up before, but why doesn't migrate_disable() use > a simple per task flag that the scheduler migration code takes into account? we could add that. But right now there are two spots which look at the counter to decide whether or not migration is disabled. > It should be functionally equivalent to the current solution, and it appears to > have a heck of a smaller cross section with the rest of the scheduler. > > I.e.: > > static inline void migrate_disable(void) > { > current->migration_disabled++; > } > > ... > > static inline void migrate_enable(void) > { > current->migration_disabled--; > } > > or so? Then add this flag as a condition to can_migrate_task() et al. > > While we generally dislike such flags as they wreck havoc with the scheduler if > overused, the cpus_allowed based solution has the exact same effect so it's not > like it's a step backwards - and it should also be much faster and less intrusive. So you are saying that we drop the cpus_ptr + cpus_mask fields again and instead add a task-flag to ensure that the scheduler does not migrate the task to another CPU? > Am I missing some complication? We do have the counter. We have need to ensure that the CPU is not going away while we are in a migrate_disable() region since we can be scheduled out. So the CPU can't go offline until we leave that region. This version uses get_online_cpus() while in -RT we have something called "pin_current_cpu()". This is a lightweight version of get_online_cpus() which should go away… Right now I have this and I need to test this and complete CPU hop part: #define migrate_disable() sched_migrate_to_cpu_save(-1) int sched_migrate_to_cpu_save(int cpu) { struct task_struct *p = current; if (in_atomic()) { #ifdef CONFIG_SCHED_DEBUG p->migrate_disable_atomic++; if (cpu >= 0) WARN_ON_ONCE(!cpumask_equal(p->cpus_ptr, cpumask_of(cpu))); #endif return raw_smp_processor_id(); } #ifdef CONFIG_SCHED_DEBUG WARN_ON_ONCE(p->migrate_disable_atomic); #endif if (p->migrate_disable) { p->migrate_disable++; #ifdef CONFIG_SCHED_DEBUG if (cpu >= 0) WARN_ON_ONCE(!cpumask_equal(p->cpus_ptr, cpumask_of(cpu))); #endif return raw_smp_processor_id(); } get_online_cpus(); preempt_disable(); p->migrate_disable = 1; if (cpu < 0) { p->cpus_ptr = &cpumask_of(task_cpu(raw_smp_processor_id())); } else { if (!cpu_online(cpu)) { preempt_enable(); put_online_cpus(); WARN(1, "CPU is offline\n"); return -ENODEV; } p->cpus_ptr = &cpumask_of(task_cpu((cpu))); } t->nr_cpus = 1; preempt_enable(); if (cpumask_equal(p->cpus_ptr, cpumask_of(cpu))) return cpu; /* move to the correct CPU */ BUG(); return raw_smp_processor_id(); } The task-flag / p->migrate_disable() counter is used in two spots in do_set_cpus_allowed(); __set_cpus_allowed_ptr(); so that a change to the affinity mask does not force a CPU hop. > Thanks, > > Ingo Sebastian