From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751454AbcH3NM2 (ORCPT ); Tue, 30 Aug 2016 09:12:28 -0400 Received: from mailout3.samsung.com ([203.254.224.33]:51492 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750790AbcH3NM0 (ORCPT ); Tue, 30 Aug 2016 09:12:26 -0400 X-AuditID: cbfee68e-f79cb6d000006cfe-18-57c586377862 From: Jeehong Kim To: mingo@redhat.com Cc: peterz@infradead.org, linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org, ezjjilong@gmail.com, Jeehong Kim Subject: [PATCH V2] sched/fair: Fix that tasks are not constrained by cfs_b->quota on hotplug core, when hotplug core is offline and then online. Date: Tue, 30 Aug 2016 22:12:40 +0900 Message-id: <1472562760-20842-1-git-send-email-jhez.kim@samsung.com> X-Mailer: git-send-email 1.9.1 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNLMWRmVeSWpSXmKPExsVy+t8zLV3ztqPhBs97LS0O/dvNaNH9po/d 4uTs84wWV49OYrbYekva4vKuOWwWlw4sYLI43nuAyeLWg0YWB06PnbPusntsXqHl8X7fVTaP vi2rGD0+b5ILYI3isklJzcksSy3St0vgyth78RxzwQ/hiuvPpRsY/wl0MXJySAiYSGy7foYZ whaTuHBvPVsXIxeHkMBKRomJX6YwwRRNnvCcCSKxlFHi69kOdgjnO6PEn62X2ECq2AQ0JO42 32MBsUUERCQOPtwONopZYAqjxJcV+8ESwgKTGSXe7ssCsVkEVCWmfLoKNImDg1fARWJSRxjE NjmJk8cms4L0Sgj0s0vMnviNDaJeQOLb5EMsIPUSArISmw5AnS0pcXDFDZYJjIILGBlWMYqm FiQXFCelFxnpFSfmFpfmpesl5+duYoSEcd8OxpsHrA8xCnAwKvHwcqw7Ei7EmlhWXJl7iNEU aMNEZinR5HxgtOSVxBsamxlZmJqYGhuZW5opifMmSP0MFhJITyxJzU5NLUgtii8qzUktPsTI xMEp1cBYcS5DVHRH1lHnC4+3Sp66zNXS2Wy0uDsr9frXYusZfRc2MbT3CFndfukz8/xbf1Mm zrADSyUuLrx+5I3Qh18uxYblKnxtEzW4m4NC5/77aevS4eA26Ybn71cOc76Y7WPfmlTJKDe/ WaXsTWuF+u71jjz+7OJcr6p5zl4pOrFZ2Fz7YENcw2wlluKMREMt5qLiRABKoRHiXgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprBIsWRmVeSWpSXmKPExsVy+t9jAV3ztqPhBmv7TSwO/dvNaNH9po/d 4uTs84wWV49OYrbYekva4vKuOWwWlw4sYLI43nuAyeLWg0YWB06PnbPusntsXqHl8X7fVTaP vi2rGD0+b5ILYI1qYLTJSE1MSS1SSM1Lzk/JzEu3VfIOjneONzUzMNQ1tLQwV1LIS8xNtVVy 8QnQdcvMAbpHSaEsMacUKBSQWFyspG+HaUJoiJuuBUxjhK5vSBBcj5EBGkhYx5ix9+I55oIf whXXn0s3MP4T6GLk5JAQMJGYPOE5E4QtJnHh3nq2LkYuDiGBpYwSX892sEM43xkl/my9xAZS xSagIXG3+R4LiC0iICJx8OF2sA5mgSmMEl9W7AdLCAtMZpR4uy8LxGYRUJWY8ukq0CQODl4B F4lJHWEQ2+QkTh6bzDqBkXsBI8MqRonUguSC4qT0XKO81HK94sTc4tK8dL3k/NxNjOBYeSa9 g/HwLvdDjAIcjEo8vAfyjoQLsSaWFVfmHmKU4GBWEuF1bD0aLsSbklhZlVqUH19UmpNafIjR FGj/RGYp0eR8YBznlcQbGpuYGVkamRmbmBsbK4nzPv6/LkxIID2xJDU7NbUgtQimj4mDU6qB cbKXSpj2lQ7ntzNMSy88MbaJu/nqzg7eJSxrp7Ydbz68zK73ZJLdpA7RIIXzhvzKavG8lVl7 HcpmNZSEqC7Q3Pz86o/n3JumfJANes+3PfiGVFuegLJ2mtTV9VmT/23dds/k6xtfee7uu7ft yu13T2A9FRZwws6pzebUfl9xNuHUl2eKjpswKLEUZyQaajEXFScCAPFBWg6rAgAA DLP-Filter: Pass X-MTR: 20000000000000000@CPGS X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In case that CONFIG_HOTPLUG_CPU and CONFIG_CFS_BANDWIDTH is turned on and tasks in bandwidth controlled task group run on hotplug core, the tasks are not controlled by cfs_b->quota when hotplug core is offline and then online. The remaining tasks in task group consume all of cfs_b->quota on other cores. The cause of this problem is described as below: 1. When hotplug core is offline while tasks in task group run on hotplug core, unregister_fair_sched_group() deletes leaf_cfs_rq_list of tg->cfs_rq[cpu] from &rq_of(cfs_rq)->leaf_cfs_rq_list. 2. Then, when hotplug core is online, update_runtime_enabled() registers cfs_b->quota on cfs_rq->runtime_enabled of all leaf cfs_rq on runqueue. However, because this is before enqueue_entity() adds &cfs_rq->leaf_cfs_rq_list on &rq_of(cfs_rq)->leaf_cfs_rq_list, cfs->quota is not register on cfs_rq->runtime_enabled. To resolve this problem, this patch makes update_runtime_enabled() registers cfs_b->quota by using walk_tg_tree_from(). Signed-off-by: Jeehong Kim --- V2: update_runtime_enabled() walks the whole tg tree, instead of register quota on every enqueue. kernel/sched/fair.c | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 6488815..63b376d 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4373,17 +4373,27 @@ static void destroy_cfs_bandwidth(struct cfs_bandwidth *cfs_b) hrtimer_cancel(&cfs_b->slack_timer); } -static void __maybe_unused update_runtime_enabled(struct rq *rq) +static int __maybe_unused __update_runtime_enabled(struct task_group *tg, void *data) { - struct cfs_rq *cfs_rq; + struct rq *rq = data; + struct cfs_rq *cfs_rq = tg->cfs_rq[cpu_of(rq)]; + struct cfs_bandwidth *cfs_b = &cfs_rq->tg->cfs_bandwidth; - for_each_leaf_cfs_rq(rq, cfs_rq) { - struct cfs_bandwidth *cfs_b = &cfs_rq->tg->cfs_bandwidth; + raw_spin_lock(&cfs_b->lock); + cfs_rq->runtime_enabled = cfs_b->quota != RUNTIME_INF; + raw_spin_unlock(&cfs_b->lock); - raw_spin_lock(&cfs_b->lock); - cfs_rq->runtime_enabled = cfs_b->quota != RUNTIME_INF; - raw_spin_unlock(&cfs_b->lock); - } + return 0; +} + +static void __maybe_unused update_runtime_enabled(struct rq *rq) +{ + struct cfs_rq *cfs_rq = &rq->cfs; + + /* register cfs_b->quota on the whole tg tree */ + rcu_read_lock(); + walk_tg_tree_from(cfs_rq->tg, __update_runtime_enabled, tg_nop, (void *)rq); + rcu_read_unlock(); } --