From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1755428AbdESNbX (ORCPT <rfc822;w@1wt.eu>);
        Fri, 19 May 2017 09:31:23 -0400
Received: from foss.arm.com ([217.140.101.70]:43964 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1750768AbdESNbU (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 19 May 2017 09:31:20 -0400
Subject: Re: [PATCH V3 1/2] sched/fair: Fix load_balance() affinity redo path
To: Jeffrey Hugo <jhugo@codeaurora.org>, Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>, linux-kernel@vger.kernel.org
References: <1495136163-27440-1-git-send-email-jhugo@codeaurora.org>
 <1495136163-27440-2-git-send-email-jhugo@codeaurora.org>
Cc: Austin Christ <austinwc@codeaurora.org>,
        Tyler Baicar <tbaicar@codeaurora.org>,
        Timur Tabi <timur@codeaurora.org>
From: Dietmar Eggemann <dietmar.eggemann@arm.com>
Message-ID: <642f9111-71ae-398d-58ad-74f930533e70@arm.com>
Date: Fri, 19 May 2017 14:31:17 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <1495136163-27440-2-git-send-email-jhugo@codeaurora.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 18/05/17 20:36, Jeffrey Hugo wrote:

[...]

> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d711093..a5d41b1 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -8220,7 +8220,24 @@ static int load_balance(int this_cpu, struct rq *this_rq,
>  		/* All tasks on this runqueue were pinned by CPU affinity */
>  		if (unlikely(env.flags & LBF_ALL_PINNED)) {
>  			cpumask_clear_cpu(cpu_of(busiest), cpus);
> -			if (!cpumask_empty(cpus)) {
> +			/*
> +			 * dst_cpu is not a valid busiest cpu in the following
> +			 * check since load cannot be pulled from dst_cpu to be
> +			 * put on dst_cpu.
> +			 */
> +			cpumask_clear_cpu(env.dst_cpu, cpus);
> +			/*
> +			 * Go back to "redo" iff the load-balance cpumask
> +			 * contains other potential busiest cpus for the
> +			 * current sched domain.
> +			 */
> +			if (cpumask_intersects(cpus, sched_domain_span(env.sd))) {
> +				/*
> +				 * Now that the check has passed, reenable
> +				 * dst_cpu so that load can be calculated on
> +				 * it in the redo path.
> +				 */
> +				cpumask_set_cpu(env.dst_cpu, cpus);

IMHO, this will work nicely and its way easier.

Another idea might be to check if the LBF_ALL_PINNED is set when we
check if we should clean the imbalance flag.

@@ -8307,14 +8307,13 @@ static int load_balance(int this_cpu, struct rq *this_rq,
         * We reach balance although we may have faced some affinity
         * constraints. Clear the imbalance flag if it was set.
         */
-       if (sd_parent) {
+       if (sd_parent && !(env.flags & LBF_ALL_PINNED)) {
                int *group_imbalance = &sd_parent->groups->sgc->imbalance;

                if (*group_imbalance)
                        *group_imbalance = 0;
        }

But I think preventing a needless redo loop is even better ...