From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934821AbbIVUfW (ORCPT ); Tue, 22 Sep 2015 16:35:22 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37198 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934743AbbIVUe0 (ORCPT ); Tue, 22 Sep 2015 16:34:26 -0400 From: Luiz Capitulino To: linux-kernel@vger.kernel.org Cc: riel@redhat.com, rafael.j.wysocki@intel.com, mingo@kernel.org Subject: [RFC 2/3] sched/idle: make cpu_idle_force_poll per-cpu Date: Tue, 22 Sep 2015 16:34:21 -0400 Message-Id: <1442954062-28578-3-git-send-email-lcapitulino@redhat.com> In-Reply-To: <1442954062-28578-1-git-send-email-lcapitulino@redhat.com> References: <1442954062-28578-1-git-send-email-lcapitulino@redhat.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation to support setting idle polling behavior at run-time, this commit makes the cpu_idle_force_poll global counter a per-cpu data. The new per-cpu data is actually a struct, and new helper functions are added in order to maintain the same semantics cpu_idle_force_poll used to have. This change should not be visible to arch code calling cpu_idle_poll_ctrl(). Signed-off-by: Luiz Capitulino --- kernel/sched/idle.c | 42 +++++++++++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c index 93d0657..3060977 100644 --- a/kernel/sched/idle.c +++ b/kernel/sched/idle.c @@ -24,29 +24,49 @@ void sched_idle_set_state(struct cpuidle_state *idle_state) idle_set_state(this_rq(), idle_state); } -static int __read_mostly cpu_idle_force_poll; +struct idle_poll { + int force_poll; +}; + +static DEFINE_PER_CPU(struct idle_poll, idle_poll) = { + .force_poll = 0, +}; + +static bool this_cpu_idle_poll(void) +{ + return per_cpu(idle_poll, smp_processor_id()).force_poll > 0; +} + +static void cpu_idle_poll_set_all(int v) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(idle_poll, cpu).force_poll = v; +} void cpu_idle_poll_ctrl(bool enable) { - if (enable) { - cpu_idle_force_poll++; - } else { - cpu_idle_force_poll--; - WARN_ON_ONCE(cpu_idle_force_poll < 0); + int *p, cpu; + + for_each_possible_cpu(cpu) { + p = &per_cpu(idle_poll, cpu).force_poll; + enable == true ? ++*p : --*p; + WARN_ON_ONCE(*p < 0); } } #ifdef CONFIG_GENERIC_IDLE_POLL_SETUP static int __init cpu_idle_poll_setup(char *__unused) { - cpu_idle_force_poll = 1; + cpu_idle_poll_set_all(1); return 1; } __setup("nohlt", cpu_idle_poll_setup); static int __init cpu_idle_nopoll_setup(char *__unused) { - cpu_idle_force_poll = 0; + cpu_idle_poll_set_all(0); return 1; } __setup("hlt", cpu_idle_nopoll_setup); @@ -58,7 +78,7 @@ static inline void cpu_idle_poll(void) trace_cpu_idle_rcuidle(0, smp_processor_id()); local_irq_enable(); while (!tif_need_resched() && - (cpu_idle_force_poll || tick_check_broadcast_expired())) + (this_cpu_idle_poll() || tick_check_broadcast_expired())) cpu_relax(); trace_cpu_idle_rcuidle(PWR_EVENT_EXIT, smp_processor_id()); rcu_idle_exit(); @@ -71,7 +91,7 @@ void __weak arch_cpu_idle_exit(void) { } void __weak arch_cpu_idle_dead(void) { } void __weak arch_cpu_idle(void) { - cpu_idle_force_poll = 1; + cpu_idle_poll_set_all(1); local_irq_enable(); } @@ -242,7 +262,7 @@ static void cpu_idle_loop(void) * know that the IPI is going to arrive right * away */ - if (cpu_idle_force_poll || tick_check_broadcast_expired()) + if (this_cpu_idle_poll() || tick_check_broadcast_expired()) cpu_idle_poll(); else cpuidle_idle_call(); -- 2.1.0