From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752250AbdK1Jem (ORCPT ); Tue, 28 Nov 2017 04:34:42 -0500 Received: from mail-lf0-f67.google.com ([209.85.215.67]:43537 "EHLO mail-lf0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752167AbdK1Jee (ORCPT ); Tue, 28 Nov 2017 04:34:34 -0500 X-Google-Smtp-Source: AGs4zMbUl3Md3UE4q851qkI/bKwDV5NLvwv+qul+dspIpW0U8X4lGNUOJ8gafTkl1NbEwtUk4JelCw== From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 28 Nov 2017 10:34:22 +0100 To: Mike Galbraith Cc: Uladzislau Rezki , Atish Patra , Peter Zijlstra , Joel Fernandes , LKML , Brendan Jackman , Josef Bacik , Ingo Molnar Subject: Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window. Message-ID: <20171128093422.5hmrkonj3z6twy2y@pc636> References: <1509427662-25114-1-git-send-email-atish.patra@oracle.com> <1509427662-25114-2-git-send-email-atish.patra@oracle.com> <20171031082009.rxxa57goto6q5xld@hirez.programming.kicks-ass.net> <49e98b00-80c7-b3a4-30fd-bccb382d002b@oracle.com> <20171123105247.wcl2fiypge2pvile@pc636> <1511442781.6505.26.camel@gmx.de> <20171124102636.zqqjqa3sru7ebh4k@pc636> <1511549190.8029.233.camel@gmx.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1511549190.8029.233.camel@gmx.de> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Nov 24, 2017 at 07:46:30PM +0100, Mike Galbraith wrote: > On Fri, 2017-11-24 at 11:26 +0100, Uladzislau Rezki wrote: > > > > I guess there is misunderstanding here. The main goal is not to cover > > pinned case, for sure. I was thinking more about below points: > > > > - Extend a claim_wake_up logic for making an ILB/NO_HZ decision more > > predictable (that is good for mobile workloads). Because as it is > > right now it simply returns a first CPU in a "nohz" mask and if we > > know that CPU has been claimed i think it is worth to go with another > > ILB core, since waking up on a remote CPU + doing nohz_idle_balance > > does not improve wake-up latency and is a miss from ilb point of view. > > Using unsynchronized access of a flag set by chaotic events all over > the box? Well, entering NO_HZ mode and checking nohz.idle_cpus_mask nohz_balancer_kick() -> find_new_ilb() are obviously asynchronous events and like you say is chaotic except of accessing to the cpumask variable (i mean atomically). So do not see any issues with that. > > > If you have any proposal, i would be appreciated if you could > > share your specific view. > > My view is you're barking up the wrong tree: you're making the idle > data SIS is using more accurate, but I question the benefit.  That it > makes an imperfect placement decision occasionally due to raciness is > nearly meaningless compared to the cost of frequent bounce. Before sitting down and start testing, i just illustrated how we can apply claim_wake_up to ilb asking community a specific view on it: drawbacks, pros/cons, proposals etc. > > Better (but still not perfect, that requires atomics) state information > doesn't meaningfully improve decision quality. > I agree, it should be atomic, something like below: core.c +/* cpus with claim wake-up set bit */ +cpumask_t cpu_claim_wake_up_map; ... @@ -1559,6 +1562,7 @@ int select_task_rq(struct task_struct *p, int cpu, int sd_flags, int wake_flags) !cpu_online(cpu))) cpu = select_fallback_rq(task_cpu(p), p); + cpumask_test_and_set_cpu(cpu, &cpu_claim_wake_up_map); return cpu; } ... + if (cpumask_test_cpu(cpu, &cpu_claim_wake_up_map)) + return 0; + return 1; } fair.c: static inline int find_new_ilb(void) { - int ilb = cpumask_first(nohz.idle_cpus_mask); + cpumask_t cpumask; + int ilb; + + cpumask_andnot(&cpumask, nohz.idle_cpus_mask, + &cpu_claim_wake_up_map); + + ilb = cpumask_first(&cpumask); > > Considering a core as not-idle when somebody tends to wake up a task on > > it is a good point. If you have any specific example when it is bad, > > please share it. > > I'm not sure how that will work out. Probably like most everything wrt > SIS, first comes "woohoo" then "aw crap", and off to the trash it goes. > -- Uladzislau Rezki