From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755023AbbCRHGd (ORCPT ); Wed, 18 Mar 2015 03:06:33 -0400 Received: from m50-112.126.com ([123.125.50.112]:35037 "EHLO m50-112.126.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754926AbbCRHGb (ORCPT ); Wed, 18 Mar 2015 03:06:31 -0400 X-Greylist: delayed 1896 seconds by postgrey-1.27 at vger.kernel.org; Wed, 18 Mar 2015 03:06:31 EDT From: Xunlei Pang To: linux-kernel@vger.kernel.org Cc: Peter Zijlstra , Steven Rostedt , Juri Lelli , Xunlei Pang Subject: [PATCH] sched/fair: Restore env status before goto redo in load_balance() Date: Wed, 18 Mar 2015 14:31:02 +0800 Message-Id: <1426660262-27526-1-git-send-email-xlpang@126.com> X-Mailer: git-send-email 1.9.1 X-CM-TRANSID: j9KowACnLDowHAlVTaOZFA--.991S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxZr18Jw4rXFWDKFyDAw18Xwb_yoW5Aw48p3 9avFWrtF4Dt3W8J39avF4v9r4Sqr1fur47JFnrJ3WSyF45Wr1jyr1Sq3W3uFWjvF95tFs0 qr9IqryUuasFg3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jk8n5UUUUU= X-Originating-IP: [210.21.223.3] X-CM-SenderInfo: p0ost0bj6rjloofrz/1tbimhjAv1GfVfzA+QAAsj Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Xunlei Pang In load_balance(), some members of lb_env will be assigned with new values in LBF_DST_PINNED case. But lb_env::flags may still retain LBF_ALL_PINNED if no proper tasks were found afterwards due to another balance, task affinity changing, etc, which can really happen because busiest rq lock has already been released. This is wrong, for example with env.dst_cpu assigned new_dst_cpu when going back to "redo" label, it may cause should_we_balance() to return false which is unreasonable. This patch restores proper status of env before "goto redo", and improves "out_all_pinned" and "out_one_pinned" labels. Signed-off-by: Xunlei Pang --- kernel/sched/fair.c | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ee595ef..45bbda1 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6843,6 +6843,7 @@ static int load_balance(int this_cpu, struct rq *this_rq, .dst_cpu = this_cpu, .dst_rq = this_rq, .dst_grpmask = sched_group_cpus(sd->groups), + .new_dst_cpu = -1, .idle = idle, .loop_break = sched_nr_migrate_break, .cpus = cpus, @@ -6977,12 +6978,19 @@ more_balance: /* All tasks on this runqueue were pinned by CPU affinity */ if (unlikely(env.flags & LBF_ALL_PINNED)) { cpumask_clear_cpu(cpu_of(busiest), cpus); - if (!cpumask_empty(cpus)) { - env.loop = 0; - env.loop_break = sched_nr_migrate_break; - goto redo; + if (env.new_dst_cpu != -1) { + env.new_dst_cpu = -1; + cpumask_or(cpus, cpus, + sched_group_cpus(sd->groups)); + cpumask_and(cpus, cpus, cpu_active_mask); + + env.dst_cpu = this_cpu; + env.dst_rq = this_rq; } - goto out_all_pinned; + env.flags &= ~LBF_SOME_PINNED; + env.loop = 0; + env.loop_break = sched_nr_migrate_break; + goto redo; } } @@ -7009,7 +7017,7 @@ more_balance: raw_spin_unlock_irqrestore(&busiest->lock, flags); env.flags |= LBF_ALL_PINNED; - goto out_one_pinned; + goto out_active_balanced; } /* @@ -7058,26 +7066,23 @@ more_balance: out_balanced: /* * We reach balance although we may have faced some affinity - * constraints. Clear the imbalance flag if it was set. + * constraints. + * + * When LBF_ALL_PINNED was not set, clear the imbalance flag + * if it was set. */ - if (sd_parent) { + if (sd_parent && !(env.flags & LBF_ALL_PINNED)) { int *group_imbalance = &sd_parent->groups->sgc->imbalance; if (*group_imbalance) *group_imbalance = 0; } -out_all_pinned: - /* - * We reach balance because all tasks are pinned at this level so - * we can't migrate them. Let the imbalance flag set so parent level - * can try to migrate them. - */ schedstat_inc(sd, lb_balanced[idle]); sd->nr_balance_failed = 0; -out_one_pinned: +out_active_balanced: /* tune up the balancing interval */ if (((env.flags & LBF_ALL_PINNED) && sd->balance_interval < MAX_PINNED_INTERVAL) || -- 1.9.1