From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 063BF30AACD for ; Wed, 24 Jun 2026 05:05:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.15 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782277537; cv=none; b=GoQBTmtPl4Vv/a/UINnAcO96ElVWpPAiHmg5hLP04TGyky7xmAKKmQqN0fRaquOpXhWTwj9XWyJr3EYr5XXrM2H2/8UhX2LxUypdpb5Zef+OO7r3ChNaSu9WNBm57F55GS+pNFcMCQqqKviQQWwzfNh2X0LYz9Fc+ZtRJJHfM3A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782277537; c=relaxed/simple; bh=aIV9T6F3/CjWKBJkLXoPaL1uxEOnpul3+EVLUNTlLps=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=u7sZOG50OnLZ0229IRxVnAzJOc3SbSkOytjcmyt2u4IQU3adCpra8Ue/WKDeQgMcJLdFIWgkuBT5HGVW0Yr4H7heUnDtsfhPAvRIGDMNT1gpi1buaYJfBbP+4UeQnyQU1lwVVOefQ7g72qW8TQNdAGQ/Mk1sEQ+g1wmnsw3i+M8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XdkKdd0R; arc=none smtp.client-ip=192.198.163.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XdkKdd0R" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782277535; x=1813813535; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=aIV9T6F3/CjWKBJkLXoPaL1uxEOnpul3+EVLUNTlLps=; b=XdkKdd0RmWMMJXpBfcxB9AsICqs3vmmiIA+x8zrfd9ZhfN58v3WHtUPX 7w8tTuPyAFmdFDLEaU+EQreWpjCZ9vKateeyUufXXgrGHDqtw+XEabESH 0t7tIuceRG5F2O+HmOfgRBVoiJYcDP0JpcP0NlHs/z3FWqbOmWTGSy4Lj z29C+xJoQpyW4syRXg6sdIktWNDW5GOS5Gz+NG42rlPBwmFHWUL2f7AUS GStiK5fhN4Zndq0/Gye1vZD5vioOEMpmjndFMyWOm/T8cwGl1zARIWEO5 SFyG0iYozGAcm/zGsyOQlGUqQOMMNWDqVPvYmiMGz84RhY/c4H8U1sN8y Q==; X-CSE-ConnectionGUID: yxoELCQ6SciLooAlqHYy1g== X-CSE-MsgGUID: HqZSFAyYRZ6M4T3MWk9Xdw== X-IronPort-AV: E=McAfee;i="6800,10657,11826"; a="83151718" X-IronPort-AV: E=Sophos;i="6.24,221,1774335600"; d="scan'208";a="83151718" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2026 22:05:34 -0700 X-CSE-ConnectionGUID: WglNxHg1QaGRmeKcIthzyw== X-CSE-MsgGUID: xjK0ljgdSQ+fopJdv3tn+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.24,221,1774335600"; d="scan'208";a="279884886" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orviesa002.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2026 22:05:34 -0700 Date: Tue, 23 Jun 2026 22:14:57 -0700 From: Ricardo Neri To: Vincent Guittot Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song , "Rafael J. Wysocki" , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v5 6/6] sched/topology: Do not clear SD_PREFER_SIBLING in domains with clusters Message-ID: <20260624051457.GA16816@ranerica-svr.sc.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> <20260622-rneri-fix-cas-clusters-v5-6-19968f2d1497@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) On Tue, Jun 23, 2026 at 09:26:57AM +0200, Vincent Guittot wrote: > On Tue, 23 Jun 2026 at 01:55, Ricardo Neri > wrote: > > > > Some topologies have scheduling domains that contain CPUs of asymmetric > > capacity, grouped into two or more clusters of equal-capacity CPUs > > sharing an L2 cache. When CONFIG_SCHED_CLUSTER is enabled, load must be > > balanced across these clusters. > > > > Do not clear SD_PREFER_SIBLING in the child domains to indicate to the > > load balancer that it should spread load among cluster siblings. > > > > Checks for capacity in update_sd_pick_busiest(), > > sched_balance_find_src_group(), and sched_balance_find_src_rq() prevent > > migrations from high- to low-capacity CPUs if the busiest group is not > > overloaded. > > > > CPUs with spare capacity, big or small, have always helped overloaded > > groups. Once the overloading condition disappears, misfit load will still > > be used to move high-utilization tasks to bigger CPUs if they have spare > > capacity. > > > > Adding the SD_PREFER_SIBLING flag shifts load balancing in shared-LLC > > domains from equalizing the number of idle CPUs to equalizing the number > > of running tasks. This also enables migrations among clusters from newly- > > idle load balance, where the outgoing task is already dequeued but the CPU > > has not yet transitioned to idle. > > > > Reviewed-by: Tim Chen > > Tested-by: Christian Loehle > > Signed-off-by: Ricardo Neri > > --- > > Changes in v5: > > * Improved inline comments for accuracy. > > * Added Tested-by tag from Christian. Thanks! > > > > Changes in v4: > > * Added Reviewed-by tag from Tim. Thanks! > > > > Changes in v3: > > * Updated documentation of SD_PREFER_SIBLING. > > * Expanded the patch description to explain the behavior when overloaded > > groups are involved. > > > > Changes in v2: > > * Reworded the patch description for clarity. > > * Kept parentheses around bitwise operators for clarity. > > --- > > include/linux/sched/sd_flags.h | 3 ++- > > kernel/sched/topology.c | 14 ++++++++++++-- > > 2 files changed, 14 insertions(+), 3 deletions(-) > > > > diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h > > index 42839cfa2778..f9a46fb8cacf 100644 > > --- a/include/linux/sched/sd_flags.h > > +++ b/include/linux/sched/sd_flags.h > > @@ -147,7 +147,8 @@ SD_FLAG(SD_ASYM_PACKING, SDF_NEEDS_GROUPS) > > * Prefer to place tasks in a sibling domain > > * > > * Set up until domains start spanning NUMA nodes. Close to being a SHARED_CHILD > > - * flag, but cleared below domains with SD_ASYM_CPUCAPACITY. > > + * flag, but cleared below domains with SD_ASYM_CPUCAPACITY unless those child > > + * domains have clusters of CPUs sharing cache. > > * > > * NEEDS_GROUPS: Load balancing flag. > > */ > > diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c > > index 622e2e01974c..261b407d0936 100644 > > --- a/kernel/sched/topology.c > > +++ b/kernel/sched/topology.c > > @@ -1995,8 +1995,18 @@ sd_init(struct sched_domain_topology_level *tl, > > /* > > * Convert topological properties into behaviour. > > */ > > - /* Don't attempt to spread across CPUs of different capacities. */ > > - if ((sd->flags & SD_ASYM_CPUCAPACITY) && sd->child) > > + /* > > + * Don't attempt to spread across CPUs of different capacities. > > + * > > + * If the child domain has clusters of CPUs sharing L2 cache, keep the > > + * flag to spread tasks across clusters of identical capacity. Checks in > > + * the load balancer prevent task migrations from high- to low-capacity > > + * CPUs unless the source group is overloaded. Migrations to a lower- > > + * capacity CPU can happen if a higher-capacity group is overloaded and > > + * a lower-capacity CPU has spare capacity. > > + */ > > + if ((sd->flags & SD_ASYM_CPUCAPACITY) && sd->child && > > + !(sd->child->flags & SD_CLUSTER)) > > sd->child->flags &= ~SD_PREFER_SIBLING; > > Last time I looked at this patch I was balanced between your proposal > above and simply keeping SD_PREFER_SIBLING for all HMP topologies. As > added in the comment: > " Checks in > * the load balancer prevent task migrations from high- to low-capacity > * CPUs unless the source group is overloaded. > " > So, why should we bother for (SD_ASYM_CPUCAPACITY && !SD_CLUSTER) topology ? No reason, AFAICS. I just wanted to restrict the change to the target topology of this patchset. But you raise a good point: given the checks in place in the load balancer, it should be OK to keep SD_PREFER_SIBLING in all asymmetric topologies. I will run a few experiments to confirm.