Re: [PATCH RFC/TEST] sched: make sync affine wakeups work

[Mike Galbraith <umgwanakikbuti@gmail.com>]

On Sun, 2014-05-04 at 17:14 +0530, Preeti Murthy wrote: 
> Hi Rik, Mike
> 
> On Fri, May 2, 2014 at 12:00 PM, Rik van Riel <riel@redhat.com> wrote:
> > On 05/02/2014 02:13 AM, Mike Galbraith wrote:
> >> On Fri, 2014-05-02 at 00:42 -0400, Rik van Riel wrote:
> >>
> >>> Whether or not this is the right thing to do remains to be seen,
> >>> but it does allow us to verify whether or not the wake_affine
> >>> strategy of always doing affine wakeups and only disabling them
> >>> in a specific circumstance is sound, or needs rethinking...
> >>
> >> Yes, it needs rethinking.
> >>
> >> I know why you want to try this, yes, select_idle_sibling() is very much
> >> a two faced little bitch.
> >
> > My biggest problem with select_idle_sibling and wake_affine in
> > general is that it will override NUMA placement, even when
> > processes only wake each other up infrequently...
> 
> As far as my understanding goes, the logic in select_task_rq_fair()
> does wake_affine() or calls select_idle_sibling() only at those
> levels of sched domains where the flag SD_WAKE_AFFINE is set.
> This flag is not set at the numa domain and hence they will not be
> balancing across numa nodes. So I don't understand how
> *these functions* are affecting NUMA placements.

Depends on how far away node yonder is I suppose.

static inline int sd_local_flags(int level)
{
        if (sched_domains_numa_distance[level] > RECLAIM_DISTANCE)
                return 0;

        return SD_BALANCE_EXEC | SD_BALANCE_FORK | SD_WAKE_AFFINE;
}