From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC641C7618D for ; Thu, 6 Apr 2023 20:21:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239016AbjDFUVQ (ORCPT ); Thu, 6 Apr 2023 16:21:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238928AbjDFUVB (ORCPT ); Thu, 6 Apr 2023 16:21:01 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C92F83D0 for ; Thu, 6 Apr 2023 13:21:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680812460; x=1712348460; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=UpPZ9Wr2ID6fVnMIXrYeJqlEArUgoKya/Nc91Ii6rZw=; b=G18mRTMyMNG0U0Bz4SPdL6U65IuCEsWN3pEFFkbGg1jdU1RTSOkOB12z GEZhfMN1ZOxAoEJtl/cZqcDfEyjeBnhAnG9XA7VJ3fbIKspJ83hGTRv+Y Ln7tbHP590BNrPOydL+Qa3R5WzkfdQyLkth7y/l2A1kSN4VJK91ThO1sx rybjck2lwakWoIooVSZvXeLx6DCSGzIsAaAf2Ky5w3Y4tdYgFaJ1PAaOv lBK1caq+5unCJaFxZRMj4YygfuAgrhI6mvfv7MeNxL4uh21ulba03+Y2f 0guLaIH1pcQYfpOeHHcRVwIwNxvz4WEboQKmFKBqTtres8lbK3hwXXiad A==; X-IronPort-AV: E=McAfee;i="6600,9927,10672"; a="407957729" X-IronPort-AV: E=Sophos;i="5.98,323,1673942400"; d="scan'208";a="407957729" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Apr 2023 13:20:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10672"; a="861529875" X-IronPort-AV: E=Sophos;i="5.98,323,1673942400"; d="scan'208";a="861529875" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orsmga005.jf.intel.com with ESMTP; 06 Apr 2023 13:20:58 -0700 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , Ionela Voinescu , x86@kernel.org, linux-kernel@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v4 04/12] sched/fair: Let low-priority cores help high-priority busy SMT cores Date: Thu, 6 Apr 2023 13:31:40 -0700 Message-Id: <20230406203148.19182-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230406203148.19182-1-ricardo.neri-calderon@linux.intel.com> References: <20230406203148.19182-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Using asym_packing priorities within an SMT core is straightforward. Just follow the priorities that hardware indicates. When balancing load from an SMT core, also consider the idle state of its siblings. Priorities do not reflect that an SMT core divides its throughput among all its busy siblings. They only makes sense when exactly one sibling is busy. Indicate that active balance is needed if the destination CPU has lower priority than the source CPU but the latter has busy SMT siblings. Make find_busiest_queue() not skip higher-priority SMT cores with more than busy sibling. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Ionela Voinescu Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-kernel@vger.kernel.org Tested-by: Zhang Rui Suggested-by: Valentin Schneider Signed-off-by: Ricardo Neri --- Changes since v3: * Improved the logic to determine whether CPU priority should be followed. Also, wrapped this logic in a helper function. (Vincent G./ Peter) Changes since v2: * Introduced this patch. Changes since v1: * N/A --- kernel/sched/fair.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b6bbe0300635..b151e93ec316 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10488,8 +10488,15 @@ static struct rq *find_busiest_queue(struct lb_env *env, nr_running == 1) continue; - /* Make sure we only pull tasks from a CPU of lower priority */ + /* + * Make sure we only pull tasks from a CPU of lower priority + * when balancing between SMT siblings. + * + * If balancing between cores, let lower priority CPUs help + * SMT cores with more than one busy sibling. + */ if ((env->sd->flags & SD_ASYM_PACKING) && + sched_use_asym_prio(env->sd, i) && sched_asym_prefer(i, env->dst_cpu) && nr_running == 1) continue; @@ -10582,10 +10589,15 @@ asym_active_balance(struct lb_env *env) * priority CPUs in order to pack all tasks in the highest priority * CPUs. When done between cores, do it only if the whole core if the * whole core is idle. + * + * If @env::src_cpu is an SMT core with busy siblings, let + * the lower priority @env::dst_cpu help it. Do not follow + * CPU priority. */ return env->idle != CPU_NOT_IDLE && (env->sd->flags & SD_ASYM_PACKING) && sched_use_asym_prio(env->sd, env->dst_cpu) && - sched_asym_prefer(env->dst_cpu, env->src_cpu); + (sched_asym_prefer(env->dst_cpu, env->src_cpu) || + !sched_use_asym_prio(env->sd, env->src_cpu)); } static inline bool -- 2.25.1