From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C9A644BC94 for ; Mon, 29 Jun 2026 16:15:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.19 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782749712; cv=none; b=BF/c2UoDxjClsiqgQy7CvNzpg9foVFGnHDhbMZfKCCdJ3fwcCY3PiEyZ2tKpU8DNfAJjsY7r+UWoAlV/FLCPAgvhXlEmqwIZKOvN3HFtr+wvyV/PkJYsEcm2tkcJQMwgIAJ1tpt0V9O0t8pCoG/tVPIHu5I/5LMPZwvmx68ZBVQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782749712; c=relaxed/simple; bh=rraBM92G4yg4qTVeO4xDSpDaveLcd0nS4A5TveK7M98=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZJYOAjE5TLaSDqqAAwKqRftPpIJL96/uxFU/oYcdGrIwIEYdoPHYNNg7nVDXpzFQCq9tJP5K7qGQo7Jhj+QZNURdPRrRLU/Q9pgIuDTsVgSMi2KjnZ5qqpeDrtscgzarT3fLN1PiAsWsF0TBuEgLHp0MIgzYqOroSdU48GsgcPk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FJEgudBF; arc=none smtp.client-ip=192.198.163.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FJEgudBF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782749710; x=1814285710; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=rraBM92G4yg4qTVeO4xDSpDaveLcd0nS4A5TveK7M98=; b=FJEgudBF6qkH4WVfdZl0pM41650TkTKtH0Pv2tVDFFdzyMRS2BBUQaju JSr4ACZ2hOiLL6t+XSyEbMeinsC3E7b3FOUoy+pUX46qnfjvWjsbHg2I2 dS+lGfx/JSsOjiS0EOq41hrV2BIx4JiXAqi5qzTUhC0HfoDP89IFIiK8Z SsKlC3gebX5CVPPLugy29gw+tq3iZsZ/fheX5rjSEE5ZTFQynWYBt26DH LhOtgL3k8gYxh1zIEzJyirvE9vOihokO58n1qXbkZ52BKwbb9y2RvqFK+ VUXWdLifrHzJu726+l7zd4PppG1beitrXnszypJeXCJkyBdI9cwsLpefn Q==; X-CSE-ConnectionGUID: vIcR0dUtQEuVNY74+y/erQ== X-CSE-MsgGUID: vtPHjm/CRw2QX+/VHjpmlA== X-IronPort-AV: E=McAfee;i="6800,10657,11832"; a="82441439" X-IronPort-AV: E=Sophos;i="6.24,232,1774335600"; d="scan'208";a="82441439" Received: from fmviesa009.fm.intel.com ([10.60.135.149]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2026 09:14:54 -0700 X-CSE-ConnectionGUID: 4ZdLRvVFRYW7aMeRXOM+Eg== X-CSE-MsgGUID: PGjto1SKRvOKQmeBufGKFg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.24,232,1774335600"; d="scan'208";a="245644256" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmviesa009.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2026 09:14:54 -0700 Date: Mon, 29 Jun 2026 09:24:14 -0700 From: Ricardo Neri To: Andrea Righi Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , Christian Loehle , K Prateek Nayak , Barry Song , "Rafael J. Wysocki" , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org, Vincent Guittot Subject: Re: [PATCH v5 0/6] sched: Fix cluster scheduling in the presence of asymmetric capacity Message-ID: <20260629162414.GB23570@ranerica-svr.sc.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-0-19968f2d1497@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) On Mon, Jun 29, 2026 at 03:10:49PM +0200, Andrea Righi wrote: > Hi Ricardo, > > On Mon, Jun 22, 2026 at 05:05:50PM -0700, Ricardo Neri wrote: > > Hi, > > > > This is v5 of the patch series. The only changes are optimizations to check > > same-capacity cluster and SMT siblings only when needed. Please read the > > changelog for details. > > > > Cluster scheduling aims to maximize performance by spreading load across > > clusters of CPUs that share mid-level resources [2]. It works well on > > uniform systems, but it breaks down on topologies with big and small > > cores arranged in clusters. As a result, it fails on several generations > > of Intel processors already shipped and upcoming. > > > > Consider the topology below of big (B) cores and clusters of small (s) > > cores. > > ------ ------ > > | B | | B | ----------------- ----------------- > > | | | | | s | s | s | s | | s | s | s | s | > > ------ ------ ----------------- ----------------- > > | L2 | | L2 | | L2 | | L2 | > > ------------------------------------------------------- > > | L3 | > > ------------------------------------------------------- > > > > On a partially busy system (one with idle CPUs; busy CPUs have one task > > each), scheduling for asymmetric capacity ensures that misfit tasks land on > > the big CPUs. The remaining tasks, misfit or not, run on the small CPUs. > > When CONFIG_SCHED_CLUSTER is enabled, these remaining tasks are supposed to > > be evenly spread among the small-CPU clusters. Today, this does not > > happen. > > > > Several issues in the load balancer prevent a small CPU in one cluster > > from pulling tasks from another: > > > > a) update_sd_pick_busiest() may select a fully_busy group with higher > > per-CPU capacity as the busiest, preventing a subsequent fully_busy > > group of equal capacity from being correctly selected. > > b) Misfit-load statistics are used to identify tasks that would benefit > > from migrating to bigger CPUs. Accounting misfit load is pointless if > > the destination CPU is equally small, and it also blocks balancing > > between clusters. > > c) Due to b), groups that are truly has_spare or fully_busy get > > misclassified as misfit_task. update_sd_pick_busiest() then skips > > them, since a small destination CPU cannot help with misfit tasks. > > d) Once a busiest group has been identified, sched_balance_find_src_rq() > > will refuse to migrate tasks to CPUs of equal capacity, even when > > doing so is precisely what is required to balance small-CPU clusters. > > e) The SD_PREFER_SIBLING flag is missing from scheduling domains with > > asymmetric capacity, preventing the balancer from equalizing load > > across sibling small-core clusters. > > > > Together, these issues prevent cluster-level balancing on systems with > > asymmetric CPU capacity. > > > > This series addresses each problem and restores the intended behavior. > > Details, rationale, and code changes are explained in each patch. > > > > I tested these patches on Alder Lake, which has both SMT Pcores and > > clusters of Ecores. I tested with SMT both disabled and enabled. I also > > tested on Lunar Lake and Panther Lake, which have an Ecore cluster not > > connected to the L3 cache. I repeated the same experiment with > > CONFIG_SCHED_CLUSTER disabled. The load balancer behaves as expected. > > > > Christian also tested this patchset on a synthetic arm64 qemu topology and > > the expected behavior [3]. > > > > Link: https://lore.kernel.org/all/20260509180955.1840064-1-arighi@nvidia.com/ [1] > > Link: https://lore.kernel.org/r/20210924085104.44806-1-21cnbao@gmail.com/ [2] > > Link: https://lore.kernel.org/all/e08492e0-d9f3-4574-8841-b633db008507@arm.com/ [3] > > I looked at this series and did some tests on Vera Rubin (arm64). The series > does not directly target this topology, because the system has no usable > SD_CLUSTER domain. Nevertheless, there are changes to the common fair > load-balancing and SD_ASYM_CPUCAPACITY paths, so I tested it just to check for > any potential regression. > > I used DCPerf MediaWiki and a CPU-intensive SGEMM workload based on NVPL. The > results showed no measurable performance differences, with all observed > variations falling within normal run-to-run variability. Based on these results, > the series looks good on this platform as well. > > Tested-by: Andrea Righi Thanks Andrea! I will add this details in the cover letter of the next version.