From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C9EB336EDE for ; Mon, 29 Jun 2026 16:23:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782750204; cv=none; b=Ozgei9/ZBrENTNpTkh6xeW/mhNh5Aca5C9NnLuNKABLphpNUTKPBBcdnWVD4qOYAeyp1kf1D7HDLDiXIX5kC3s9dnV7tBx81VINxuhpB2/QnYXuWd+ZdFioy8tYoS5Wtbq89L9GnIChM1t3QtHivzziM6c+0JF3+9IJRUhlCYVk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782750204; c=relaxed/simple; bh=Yl1I7B73lxoXPM405ZzQa64YJ++xg7S3HLK/o8GBODo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oPb7Zs7R82+548anIBVKAUP9b4R715ZEtqZ/eAsVz9tZWd85XI+zj61kLo7L7mXjq1wqVOvWgn2K2fyym7FqZKSs7dwSB8d8Nk6Zd7LGpqMpzJhGdt2TocnmfIN9eYyTwqmZN6FnHTk4eJU1xPNZlfZ/xPIApzogTqxE44Z8ZXM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RXNTfyVX; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RXNTfyVX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1782750203; x=1814286203; h=date:from:to:cc:subject:message-id:references: mime-version:content-transfer-encoding:in-reply-to; bh=Yl1I7B73lxoXPM405ZzQa64YJ++xg7S3HLK/o8GBODo=; b=RXNTfyVXfO+sJfvvobbe8PTA+H0owaxmpLRDjPdLZcUGX8emiS4Js2UN SBfstNb9Y6emeEkFNaC38TMuh7p+z0LyaYXnJO5M7O2pVcOQZTp4eGess QYIgjP9wxYhMZGOqoCWRWVaiS9LRKpv7iLCN/kbz292Z6i+YETcXumsl4 lp6+6guat0pLH8PJLHrj5wucLmFlJfxftV8pxQ6WMLhbmbVQ0NfaqyiIL aHDl8yabdusWnjRlpSuwqnY5zRjYSDZWYL5rOH1wBeqFjsK/SweUeoxQD vxxkyt1ozT+0y1hgj/oUXdhCo1bc/KaNnPR/mtPwPDkhchedGGkPGL3FU w==; X-CSE-ConnectionGUID: yR8blIOaSf6EOs86fM695A== X-CSE-MsgGUID: 2CBYI0sDTa6BmfeUCYl2Yg== X-IronPort-AV: E=McAfee;i="6800,10657,11832"; a="83645101" X-IronPort-AV: E=Sophos;i="6.24,232,1774335600"; d="scan'208";a="83645101" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2026 09:23:22 -0700 X-CSE-ConnectionGUID: 8yF7G4teTyWN2NwmqisVcA== X-CSE-MsgGUID: XfD5gOo7SaK0My1IfvN7gA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.24,232,1774335600"; d="scan'208";a="290124729" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by orviesa001.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jun 2026 09:23:22 -0700 Date: Mon, 29 Jun 2026 09:32:42 -0700 From: Ricardo Neri To: Vincent Guittot Cc: "Rafael J. Wysocki (Intel)" , Christian Loehle , g@ranerica-svr.sc.intel.com, Ingo Molnar , Peter Zijlstra , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Tim C Chen , Chen Yu , K Prateek Nayak , Barry Song , Andrea Righi , Len Brown , ricardo.neri@intel.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v5 5/6] sched/fair: Allow load balancing between CPUs of identical capacity Message-ID: <20260629163242.GC23570@ranerica-svr.sc.intel.com> References: <20260622-rneri-fix-cas-clusters-v5-5-19968f2d1497@linux.intel.com> <20260624052514.GA16859@ranerica-svr.sc.intel.com> <20260626001138.GA19085@ranerica-svr.sc.intel.com> <20260627020253.GA20409@ranerica-svr.sc.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) On Mon, Jun 29, 2026 at 05:58:56PM +0200, Vincent Guittot wrote: > On Mon, 29 Jun 2026 at 17:54, Rafael J. Wysocki (Intel) > wrote: > > > > On Mon, Jun 29, 2026 at 5:35 PM Vincent Guittot > > wrote: > > > > > > On Sat, 27 Jun 2026 at 03:53, Ricardo Neri > > > wrote: > > > > > > > > On Fri, Jun 26, 2026 at 04:50:12PM +0200, Vincent Guittot wrote: > > > > > On Fri, 26 Jun 2026 at 02:02, Ricardo Neri > > > > > wrote: > > > > > > > > > > > > On Tue, Jun 23, 2026 at 10:25:14PM -0700, Ricardo Neri wrote: > > > > > > > On Tue, Jun 23, 2026 at 08:45:23AM +0100, Christian Loehle wrote: > > > > > > > > On 6/23/26 08:20, Vincent Guittot wrote: > > > > > > > > > On Tue, 23 Jun 2026 at 01:55, Ricardo Neri > > > > > > > > > wrote: > > > > > > > > >> > > > > > > > > >> sched_balance_find_src_rq() avoids selecting a runqueue with a single > > > > > > > > >> running task as busiest if doing so results in migrating the task to a > > > > > > > > >> CPU with less than ~5% of extra capacity. It also unintentionally > > > > > > > > >> prevents migrations between CPUs of identical capacity. > > > > > > > > >> > > > > > > > > >> When CONFIG_SCHED_CLUSTER is enabled, load should be balanced across > > > > > > > > >> clusters of CPUs with the same capacity. Allowing migration between CPUs > > > > > > > > >> of identical capacity is necessary to meet this goal. > > > > > > > > >> > > > > > > > > >> Use arch_scale_cpu_capacity() to reflect architectural capacity, excluding > > > > > > > > > > > > > > > > > > capacity_of() reflects not only RT and irq pressure but also thermal > > > > > > > > > pressure or system frequency capping. > > > > > > > > > If dst cluster is under thermal mitigation but the source cluster is > > > > > > > > > not, we probably shouldn't spread tasks across both clusters. > > > > > > > > > Have you considered using get_actual_cpu_capacity() instead of > > > > > > > > > arch_scale_cpu_capacity() ? > > > > > > > > > > > > > > > > Replacing arch_scale_cpu_capacity() with get_actual_cpu_capacity() > > > > > > > > would make the == comparison below very unlikely to be true FWIW. > > > > > > > > > > > > > > Yes, this is what I thought too. I did not try with get_actual_cpu_capacity(), > > > > > > > though. Perhaps on Intel processors it would work since rq->avg_hw.load_avg > > > > > > > is not used, IIUC. I am not sure about cpufreq_pressure. I need to check. > > > > > > > > > > > > > > Still, it may work for Intel processors but not for ARM ones. > > > > > > > > > > > > > > > I think it's fine like that, I will prepare a follow-up anyway to make > > > > > > > > it work for our "almost equal capacity" cluster systems and then also > > > > > > > > consider switching to get_actual_cpu_capacity() since we include a margin > > > > > > > > anyway. > > > > > > > > > > > > > > Great! > > > > > > > > > > > > I confirmed that does not use rq->avg_hw.load_avg nor cpufreq_pressure. > > > > > > > > > > I'm not surprised that intel don't use rq->avg_hw.load_avg but I'm > > > > > pretty sure that you use cpufreq_pressure, because any call to > > > > > freq_qos_add_request(..., FREQ_QOS_MAX), like scaling_max_freq, will > > > > > update cpufreq_pressure. > > > > > > > > But in cpufreq_update_pressure() a non-zero pressure can be only computed > > > > if arch_scale_freq_ref() returns non-zero. x86 does not implement this > > > > function. > > > > > > > > The check max_freq <= capped_freq is always true because the default > > > > arch_scale_freq_ref() returns 0. The computed pressure is always 0. Am I > > > > missing something? > > > > > > Yeah, I forgot that Intel has not enabled it. You should consider > > > enable it because of the below Sure, I was thinking about the same. > > > > Aren't things supposed to still work if it is not enabled though? In my experiments, after capping frequency I did not observe changes in the behavior of the load balancer. > > The scheduler is not aware of any cpufreq qos max_freq constraint > applied on the CPUs and it consideres CPUs being able to run at their > max capacity I prototyped code for arch_scale_freq_ref() (along with using get_actual_cpu_capacity()) in my patchset. I did observe the load balancer moving tasks towards uncapped CPUs. I can post the patch for arch_scale_freq_ref() separately.