From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6E45839EF36 for ; Tue, 7 Apr 2026 11:50:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775562657; cv=none; b=OAoFv8d/6taewqExfZ4M7Ew2PYo9AyemOUeU6GQ6K7a+lxHVEg/ktuVTC2VMPyRKo34YEtzoyhZ56PrpO91b6sB3bxai/Gpi0e2HnDPK10YYv9qqnQqyODxS4EY+qh62jNdxrmNDB9b7gNhillaTDKHJGzdFujrCLDtlr7CrrpU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775562657; c=relaxed/simple; bh=91aOWo8sTvZO3+sTZoW5VUshAS+ljFJs/o5EN7Mdj+o=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=r5A9s4KOtVVUJJafS5bCbmbfyvnYh5PiZPW7v3oIN1IRZpDA3dTg4qwYQIelIgOKnWZEh7pTJSpuqBD3R/MNuawZmeLLFzc2p0Af+y4ZSA9tNpQbSIPd5LccCBz084i28/F+/j/+KjslALrebbBn3S8vlW0pbej7bGBjHjuhEoY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b=KW1SZU5s; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=arm.com header.i=@arm.com header.b="KW1SZU5s" Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0A53A1AED; Tue, 7 Apr 2026 04:50:49 -0700 (PDT) Received: from [192.168.178.6] (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BE70E3F7D8; Tue, 7 Apr 2026 04:50:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=arm.com; s=foss; t=1775562654; bh=91aOWo8sTvZO3+sTZoW5VUshAS+ljFJs/o5EN7Mdj+o=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=KW1SZU5sgPn+VBXjPSSS2QIw7nNAS+1jOpGUfvhuxbCZkD4UcYDckQttirqoEbrcq 5i4KMt/OqiE1LuZpFgObzxSwidl1cwjHsd7FhcMfth0PHqwdsV+kUf1MCry4QRBZyw owcdGTt7ujK6T7TtXiC/SSMgVBDS/wBTyPrpe5lA= Message-ID: Date: Tue, 7 Apr 2026 13:50:51 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/4] sched/fair: SMT-aware asymmetric CPU capacity To: Andrea Righi Cc: Vincent Guittot , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , linux-kernel@vger.kernel.org References: <20260326151211.1862600-1-arighi@nvidia.com> <193f1cd1-ced3-4b37-83af-ea43a7e5e3d0@arm.com> <9886a7d3-fb54-4637-8b4c-1f35272f4882@arm.com> Content-Language: en-GB From: Dietmar Eggemann In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 03.04.26 22:44, Andrea Righi wrote: > On Fri, Apr 03, 2026 at 04:46:03PM +0200, Andrea Righi wrote: >> On Fri, Apr 03, 2026 at 01:47:17PM +0200, Dietmar Eggemann wrote: > ... >>>> Looking at the data: >>>> - SIS_UTIL doesn't seem relevant in this case (differences are within >>>> error range), >>>> - ASYM_CPU_CAPACITY seems to provide a small throughput gain, but it seems >>>> more beneficial for tail latency reduction, >>>> - the ILB SMT patch seems to slightly improve throughput, but the biggest >>>> benefit is still coming from ASYM_CPU_CAPACITY. >>> >>>> Overall, also in this case it seems beneficial to use ASYM_CPU_CAPACITY >>>> rather than equalizing the capacities. >>>> >>>> That said, I'm still not sure why ASYM is helping. The frequency asymmetry >>> >>> OK, I still would be more comfortable with this when I would now why >>> this is :-) >> >> Working on this. :) > > Alright, I think I found something. I tried to make sis() behave more like sic() > by adding the same SMT "full idle core" check in the fast path and removing the > extra select_idle_smt(prev) hop from the LLC idle path. > > Essentially this: > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 7bebceb5ed9df..19fffa2df2d36 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -7651,29 +7651,6 @@ static int select_idle_core(struct task_struct *p, int core, struct cpumask *cpu > return -1; > } > > -/* > - * Scan the local SMT mask for idle CPUs. > - */ > -static int select_idle_smt(struct task_struct *p, struct sched_domain *sd, int target) > -{ > - int cpu; > - > - for_each_cpu_and(cpu, cpu_smt_mask(target), p->cpus_ptr) { > - if (cpu == target) > - continue; > - /* > - * Check if the CPU is in the LLC scheduling domain of @target. > - * Due to isolcpus, there is no guarantee that all the siblings are in the domain. > - */ > - if (!cpumask_test_cpu(cpu, sched_domain_span(sd))) > - continue; > - if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) > - return cpu; So it is this returning of CPU from the smt mask rather than the for_each_cpu_wrap(cpu, cpus, target + 1) __select_idle_cpu() if (choose_idle_cpu(cpu, p) && ...) return cpu where cpus is cpumask_and(cpus, sched_domain_span(MC), p->cpus_ptr) I wonder wether this has anything to do with your NVIDIA Spatial Multithreading (SMT) versus Traditional (time-shared resources) SMT? > - } > - > - return -1; > -} > - > #else /* !CONFIG_SCHED_SMT: */ > > static inline void set_idle_cores(int cpu, int val) > @@ -7690,11 +7667,6 @@ static inline int select_idle_core(struct task_struct *p, int core, struct cpuma > return __select_idle_cpu(core, p); > } > > -static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd, int target) > -{ > - return -1; > -} > - > #endif /* !CONFIG_SCHED_SMT */ > > /* > @@ -7859,7 +7831,7 @@ static inline bool asym_fits_cpu(unsigned long util, > (util_fits_cpu(util, util_min, util_max, cpu) > 0); > } > > - return true; > + return !sched_smt_active() || is_core_idle(cpu); > } This change seems to be orthogonal to the removal of select_idle_smt() for sis()? BTW, the is_core_idle() in asym_fits_cpu() (used for those early return CPU conditions in sis()) is something we don't have on the NO_ASYM side where we only use choose_idle_cpu(). > /* > @@ -7964,16 +7936,9 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target) > if (!sd) > return target; > > - if (sched_smt_active()) { > + if (sched_smt_active()) > has_idle_core = test_idle_cores(target); > > - if (!has_idle_core && cpus_share_cache(prev, target)) { > - i = select_idle_smt(p, sd, prev); > - if ((unsigned int)i < nr_cpumask_bits) > - return i; > - } > - } > - > i = select_idle_cpu(p, sd, has_idle_core, target); > if ((unsigned)i < nr_cpumask_bits) > return i; > > --- > > With this applied, I see identical performance between NO_ASYM and ASYM+SMT. Interesting! > I'm not suggesting to apply this, but that seems to be the reason why ASYM+SMT > performs better in my case. > > -Andrea