From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753301AbdASPR6 (ORCPT ); Thu, 19 Jan 2017 10:17:58 -0500 Received: from smtprelay0231.hostedemail.com ([216.40.44.231]:54954 "EHLO smtprelay.hostedemail.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753003AbdASPRy (ORCPT ); Thu, 19 Jan 2017 10:17:54 -0500 X-Session-Marker: 726F737465647440676F6F646D69732E6F7267 X-Spam-Summary: 2,0,0,,d41d8cd98f00b204,rostedt@goodmis.org,:::::::,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3874:4250:5007:6261:7774:7903:7974:8660:10004:10400:10562:10848:11026:11658:11914:12043:12296:12555:12760:13142:13148:13161:13229:13230:13439:14096:14097:14181:14394:14659:14721:14819:21067:21080:21325:21451:30054,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: act73_3a8bd8929209 X-Filterd-Recvd-Size: 3223 Date: Thu, 19 Jan 2017 10:17:03 -0500 From: Steven Rostedt To: LKML Cc: Peter Zijlstra , Ingo Molnar , Andrew Morton Subject: [PATCH] sched: Optimize pick_next_task for idle_sched_class too Message-ID: <20170119101703.2abeaeb6@gandalf.local.home> X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When running my likely/unlikely profiler, I noticed that the SCHED_DEADLINE's pick_next_task_dl() unlikely case of (!dl_rq->dl_nr_running) was always being hit. There's two cases where this can happen. First, there's an optimization in pick_next_task() for the likely case that the only tasks running on the run queue are SCHED_OTHER tasks. In a normal system, this is the case most of the time. When this is true, only the pick_next_task() of the fair_sched_class is called. If an RT or DEADLINE task is queued, then the other pick_next_task()s of the other sched classes are called in sequence. The SCHED_DEADLINE pick_next_task() is called first, and that unlikely() case is hit if there's no deadline tasks available. This happens when an RT task is queued (first case). But tracing revealed that this happens in another very common case. The case where the system goes from idle to running any task, including SCHED_OTHER. This is because the idle task has a different sched class than the fair_sched_class. The optimization has: if (prev->sched_class == fair_sched_class && rq->nr_running == rq->cfs.h_nr_running) { When going from SCHED_OTHER to idle, this optimization is hit, because the SCHED_OTHER task is of the fair_sched_class, and rq->nr_running and rq->cfs.h_nr_running are both zero. But when we go from idle to SCHED_OTHER, the first test fails. prev->sched_class is equal to idle_sched_class, and this causes both the pick_next_task() of deadline and RT sched classes to be called unnecessarily. Signed-off-by: Steven Rostedt (VMware) --- diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 154fd68..e2c6d3b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -3259,13 +3259,15 @@ static inline struct task_struct * pick_next_task(struct rq *rq, struct task_struct *prev, struct pin_cookie cookie) { const struct sched_class *class = &fair_sched_class; + const struct sched_class *idle_class = &idle_sched_class; struct task_struct *p; /* * Optimization: we know that if all tasks are in * the fair class we can call that function directly: */ - if (likely(prev->sched_class == class && + if (likely((prev->sched_class == class || + prev->sched_class == idle_class) && rq->nr_running == rq->cfs.h_nr_running)) { p = fair_sched_class.pick_next_task(rq, prev, cookie); if (unlikely(p == RETRY_TASK))