From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752757AbcAFSum (ORCPT ); Wed, 6 Jan 2016 13:50:42 -0500 Received: from terminus.zytor.com ([198.137.202.10]:54712 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752055AbcAFSui (ORCPT ); Wed, 6 Jan 2016 13:50:38 -0500 Date: Wed, 6 Jan 2016 10:49:53 -0800 From: tip-bot for Yuyang Du Message-ID: Cc: steve.muckle@linaro.org, peterz@infradead.org, tglx@linutronix.de, torvalds@linux-foundation.org, dietmar.eggemann@arm.com, mingo@kernel.org, linux-kernel@vger.kernel.org, vincent.guittot@linaro.org, morten.rasmussen@arm.com, yuyang.du@intel.com, efault@gmx.de, patrick.bellasi@arm.com, Juri.Lelli@arm.com, hpa@zytor.com Reply-To: linux-kernel@vger.kernel.org, vincent.guittot@linaro.org, morten.rasmussen@arm.com, yuyang.du@intel.com, steve.muckle@linaro.org, peterz@infradead.org, dietmar.eggemann@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, mingo@kernel.org, Juri.Lelli@arm.com, hpa@zytor.com, efault@gmx.de, patrick.bellasi@arm.com In-Reply-To: <20151216233427.GJ28098@intel.com> References: <20151216233427.GJ28098@intel.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:sched/core] sched/fair: Fix new task' s load avg removed from source CPU in wake_up_new_task() Git-Commit-ID: 0905f04eb21fc1c2e690bed5d0418a061d56c225 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 0905f04eb21fc1c2e690bed5d0418a061d56c225 Gitweb: http://git.kernel.org/tip/0905f04eb21fc1c2e690bed5d0418a061d56c225 Author: Yuyang Du AuthorDate: Thu, 17 Dec 2015 07:34:27 +0800 Committer: Ingo Molnar CommitDate: Wed, 6 Jan 2016 11:06:29 +0100 sched/fair: Fix new task's load avg removed from source CPU in wake_up_new_task() If a newly created task is selected to go to a different CPU in fork balance when it wakes up the first time, its load averages should not be removed from the source CPU since they are never added to it before. The same is also applicable to a never used group entity. Fix it in remove_entity_load_avg(): when entity's last_update_time is 0, simply return. This should precisely identify the case in question, because in other migrations, the last_update_time is set to 0 after remove_entity_load_avg(). Reported-by: Steve Muckle Signed-off-by: Yuyang Du [peterz: cfs_rq_last_update_time] Signed-off-by: Peter Zijlstra (Intel) Cc: Dietmar Eggemann Cc: Juri Lelli Cc: Linus Torvalds Cc: Mike Galbraith Cc: Morten Rasmussen Cc: Patrick Bellasi Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Vincent Guittot Link: http://lkml.kernel.org/r/20151216233427.GJ28098@intel.com Signed-off-by: Ingo Molnar --- kernel/sched/fair.c | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 93efb96..1926606 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -2900,27 +2900,45 @@ dequeue_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) max_t(s64, cfs_rq->runnable_load_sum - se->avg.load_sum, 0); } -/* - * Task first catches up with cfs_rq, and then subtract - * itself from the cfs_rq (task must be off the queue now). - */ -void remove_entity_load_avg(struct sched_entity *se) -{ - struct cfs_rq *cfs_rq = cfs_rq_of(se); - u64 last_update_time; - #ifndef CONFIG_64BIT +static inline u64 cfs_rq_last_update_time(struct cfs_rq *cfs_rq) +{ u64 last_update_time_copy; + u64 last_update_time; do { last_update_time_copy = cfs_rq->load_last_update_time_copy; smp_rmb(); last_update_time = cfs_rq->avg.last_update_time; } while (last_update_time != last_update_time_copy); + + return last_update_time; +} #else - last_update_time = cfs_rq->avg.last_update_time; +static inline u64 cfs_rq_last_update_time(struct cfs_rq *cfs_rq) +{ + return cfs_rq->avg.last_update_time; +} #endif +/* + * Task first catches up with cfs_rq, and then subtract + * itself from the cfs_rq (task must be off the queue now). + */ +void remove_entity_load_avg(struct sched_entity *se) +{ + struct cfs_rq *cfs_rq = cfs_rq_of(se); + u64 last_update_time; + + /* + * Newly created task or never used group entity should not be removed + * from its (source) cfs_rq + */ + if (se->avg.last_update_time == 0) + return; + + last_update_time = cfs_rq_last_update_time(cfs_rq); + __update_load_avg(last_update_time, cpu_of(rq_of(cfs_rq)), &se->avg, 0, 0, NULL); atomic_long_add(se->avg.load_avg, &cfs_rq->removed_load_avg); atomic_long_add(se->avg.util_avg, &cfs_rq->removed_util_avg);