From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 96040138490 for ; Wed, 25 Mar 2026 11:17:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774437425; cv=none; b=q8gSOuHhMPXjDvHTm+vHe1TGi+Hk6mzspXUfHuIPX1JNXHscnt3zatJ8vKftqvuEw6jFUJMGC2v9iuGvJY+DE2rnP6wqi5idQv5X1E0luEVmhXZ+jkcuHZOY2218eRJtYfBlaM+aXWk/1o7OPaVWStSjTEnhYsdOaRZW+Fr4jeE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774437425; c=relaxed/simple; bh=wMP4vjtlJ9D0oKttbrgRG6DwBaFnjN/+CVJt/jPAMeo=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=DpH8n7VIfDRskUZIs2Z4N+pebjIYX/tsE0gaC9j8cUzJXGZ7hKbeR+GASG2dHMOMGORA2fQiRWs2egBUp3+pEWhi+EwiCyR8xGip/4KJZb7zvUTi7bLBH2QYCL6qf1/HjxCs0XzMunAwNNVq6fNetEvgwXzNDVqfpxltALln/vc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=V9b7h8f1; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="V9b7h8f1" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EE6261CDD; Wed, 25 Mar 2026 04:16:56 -0700 (PDT) Received: from [192.168.178.6] (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27EBD3F915; Wed, 25 Mar 2026 04:17:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1774437422; bh=wMP4vjtlJ9D0oKttbrgRG6DwBaFnjN/+CVJt/jPAMeo=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=V9b7h8f1pGoXAHpg4UUzYneuSqpunNMaRxArcmuUxU52QgpFvhXP2oIHMKlfF6nZA PO9ilAjDoFMnRO5neorgvDsVlBb+4X23AGjr/MYhP0MANV46yi7hdOmzm7LmEdXlXr AK85kqhiKna6ZK/XN9uUsoXhv+LHiZ70IoNnY6no= Message-ID: <1d9b4abf-4b70-4775-92b8-924ced316578@arm.com> Date: Wed, 25 Mar 2026 12:16:59 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] sched/topology: Avoid spurious asymmetry from CPU capacity noise To: Andrea Righi Cc: Christian Loehle , Vincent Guittot , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-kernel@vger.kernel.org, Felix Abecassis References: <20260324005509.1134981-1-arighi@nvidia.com> <0fb05951-1f2f-474f-9f7c-9f0f15a5f675@arm.com> <86cb3979-02cd-4171-80fd-df20cb3430cb@arm.com> Content-Language: en-GB From: Dietmar Eggemann In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On 25.03.26 10:32, Andrea Righi wrote: > On Wed, Mar 25, 2026 at 10:23:09AM +0100, Dietmar Eggemann wrote: >> On 24.03.26 12:01, Andrea Righi wrote: >>> Hi Dietmar, >>> >>> On Tue, Mar 24, 2026 at 11:29:24AM +0100, Dietmar Eggemann wrote: >>>> On 24.03.26 10:46, Andrea Righi wrote: >>>>> Hi Christian, >>>>> >>>>> On Tue, Mar 24, 2026 at 08:08:22AM +0000, Christian Loehle wrote: >>>>>> On 3/24/26 07:55, Christian Loehle wrote: >>>>>>> On 3/24/26 07:39, Vincent Guittot wrote: >>>>>>>> On Tue, 24 Mar 2026 at 01:55, Andrea Righi wrote: >> >> [...] >> >>>> The first time we observed this on NVIDIA Grace, we wondered whether >>>> there might be functionality outside the task scheduler that makes use >>>> of these slightly heterogeneous CPU capacity values from CPPC—and >>>> whether the dependency on task scheduling was simply an overlooked >>>> phenomenon. >>>> >>>> And then there was DCPerf Mediawiki on 72 CPUs system always scoring >>>> better with sched_asym_cpucap_active() = TRUE (mentioned already by >>>> Chris L. in: >>>> https://lore.kernel.org/r/15ffdeb3-a0f3-4b88-92c0-17ffb03b0574@arm.com >>> >>> Yeah, I think Chris' asym-packing approach might be the safest thing to do. >>> >>> At the same time it would be nice to improve asym-capacity to introduce >>> some concept of SMT awareness, that was my original attempt with >>> https://lore.kernel.org/all/20260318092214.130908-1-arighi@nvidia.com, >>> since we may see similar asym-capacity benefits on Vera (that has SMT, >>> unlike Grace). What do you think? >> >> We never found a good way to specify a CPU capacity in the SMT case (EAS >> and energy model included). So comparing CPU capacity w/ utilization, CPU >> overutilization detection etc. definitions get more blurry. > > Hm... so should we just avoid calling select_idle_capacity() when SMT is > enabled to prevent waking up tasks on both SMT siblings when there are > fully-idle SMT cores? Yeah, pretty much. So prefer (2) over (1). IMHO, we do have a similar issue here. Can we say that a logical CPU is idle if its SMT sibling isn't? But at least we don't have to use any CPU cap/util comparison there. select_idle_sibling() 8132 if (sched_smt_active()) { 8133 has_idle_core = test_idle_cores(target); 8134 8135 if (!has_idle_core && cpus_share_cache(prev, target)) { <-- (1) 8136 i = select_idle_smt(p, sd, prev); 8137 if ((unsigned int)i < nr_cpumask_bits) 8138 return i; 8139 } 8140 } 8141 8142 i = select_idle_cpu(p, sd, has_idle_core, target); <-- (2a) 8143 if ((unsigned)i < nr_cpumask_bits) 8144 return i select_idle_cpu() 7926 for_each_cpu_wrap(cpu, cpus, target + 1) { 7927 if (has_idle_core) { 7928 i = select_idle_core(p, cpu, cpus, &idle_cpu); <-- (2b) 7929 if ((unsigned int)i < nr_cpumask_bits) 7930 return i; 7931 7932 } else { 7933 if (--nr <= 0) 7934 return -1; 7935 idle_cpu = __select_idle_cpu(cpu, p); 7936 if ((unsigned int)idle_cpu < nr_cpumask_bits) 7937 break; 7938 } 7939 } [...]