From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A5193D4133 for ; Wed, 25 Mar 2026 12:48:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774442941; cv=none; b=bTRhNbivFK/er7Gi4iMXOjBkGC/NFkKWXczDBAnrOR3Eds5u9LSmQow5Zv0VjxinKjv1c4AYINQH8Z+Vv6ozCfywIuqaCduQRfwMTuOjJxmyYyGQkIOJU2wofGQRjtNcZO9joGIhUYCTd+VIEkwwVu1SGKHX42hUYDB3ScGzsps= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774442941; c=relaxed/simple; bh=DFzhC7QKmUnskD6XfWADVzD3QL1LKFsGMizpg2y2Eew=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Jnnk2nsJLHV876akezDiwwgkIof91noJs5jIn6teY/QHuyhfgr5LpZopp86yqEswxDOaq07z6Q5c8OGkueCq3ucLT30kW02qqw6nCSfj3oogauGHXKn5h0LuY0zmbw92DusdOkXwlybqmFFWdwoAQJN/HyI3UG2obhlC3kIhl18= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Uo7xI9to; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Uo7xI9to" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774442939; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dzJgeIIKndH48c0tE7uhqXUCSDf9/JdB0wm2KghTqFA=; b=Uo7xI9toANAAf2/uYnF0XPfZ2MY+FVmVnYpWR6lSQRSLHNPSY81EgtLTXQDgvXqPlwL7KJ eMxRP6F7pPMMh9tK5DFlOWbzJHga5P0j13CinqCgvcxcS4MjFMhfO9rdTLhOm51q6myM7O cktO9F5Ysaty69UZGCeZbOxYSdnnj14= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-82-wSstCPlVMiGhni-NqB7lyA-1; Wed, 25 Mar 2026 08:48:55 -0400 X-MC-Unique: wSstCPlVMiGhni-NqB7lyA-1 X-Mimecast-MFC-AGG-ID: wSstCPlVMiGhni-NqB7lyA_1774442934 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1728F18005BB; Wed, 25 Mar 2026 12:48:53 +0000 (UTC) Received: from pauld.westford.csb (unknown [10.44.35.0]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BAF8019560B1; Wed, 25 Mar 2026 12:48:44 +0000 (UTC) Date: Wed, 25 Mar 2026 08:48:40 -0400 From: Phil Auld To: Andrea Righi Cc: Dietmar Eggemann , Christian Loehle , Vincent Guittot , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-kernel@vger.kernel.org, Felix Abecassis Subject: Re: [PATCH] sched/topology: Avoid spurious asymmetry from CPU capacity noise Message-ID: <20260325124840.GA98184@pauld.westford.csb> References: <20260324005509.1134981-1-arighi@nvidia.com> <0fb05951-1f2f-474f-9f7c-9f0f15a5f675@arm.com> <86cb3979-02cd-4171-80fd-df20cb3430cb@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 On Wed, Mar 25, 2026 at 10:32:28AM +0100 Andrea Righi wrote: > On Wed, Mar 25, 2026 at 10:23:09AM +0100, Dietmar Eggemann wrote: > > On 24.03.26 12:01, Andrea Righi wrote: > > > Hi Dietmar, > > > > > > On Tue, Mar 24, 2026 at 11:29:24AM +0100, Dietmar Eggemann wrote: > > >> On 24.03.26 10:46, Andrea Righi wrote: > > >>> Hi Christian, > > >>> > > >>> On Tue, Mar 24, 2026 at 08:08:22AM +0000, Christian Loehle wrote: > > >>>> On 3/24/26 07:55, Christian Loehle wrote: > > >>>>> On 3/24/26 07:39, Vincent Guittot wrote: > > >>>>>> On Tue, 24 Mar 2026 at 01:55, Andrea Righi wrote: > > > > [...] > > > > >> The first time we observed this on NVIDIA Grace, we wondered whether > > >> there might be functionality outside the task scheduler that makes use > > >> of these slightly heterogeneous CPU capacity values from CPPC—and > > >> whether the dependency on task scheduling was simply an overlooked > > >> phenomenon. > > >> > > >> And then there was DCPerf Mediawiki on 72 CPUs system always scoring > > >> better with sched_asym_cpucap_active() = TRUE (mentioned already by > > >> Chris L. in: > > >> https://lore.kernel.org/r/15ffdeb3-a0f3-4b88-92c0-17ffb03b0574@arm.com > > > > > > Yeah, I think Chris' asym-packing approach might be the safest thing to do. > > > > > > At the same time it would be nice to improve asym-capacity to introduce > > > some concept of SMT awareness, that was my original attempt with > > > https://lore.kernel.org/all/20260318092214.130908-1-arighi@nvidia.com, > > > since we may see similar asym-capacity benefits on Vera (that has SMT, > > > unlike Grace). What do you think? > > > > We never found a good way to specify a CPU capacity in the SMT case (EAS > > and energy model included). So comparing CPU capacity w/ utilization, CPU > > overutilization detection etc. definitions get more blurry. > > Hm... so should we just avoid calling select_idle_capacity() when SMT is > enabled to prevent waking up tasks on both SMT siblings when there are > fully-idle SMT cores? > That might be a good idea. Especially if it's general and not tied to EAS/ASYM. I'm getting some requests for something like that. CHeers, Phil > > > > But in case you now want to hide these small CPU capacity differences from > > asym-cpucap setup you won't run into this 'SD_SHARE_CPUCAPACITY + > > SD_ASYM_CPUCAPACITY'. > > > > You still will have small differences in sched group capacities but this > > is covered by load-balance. > > > > BTW, you should have seen on Vera ?: > > > > sd_int() [kernel/sched/.topology.c] > > > > 1720 WARN_ONCE((sd->flags & (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY)) == > > 1721 (SD_SHARE_CPUCAPACITY | SD_ASYM_CPUCAPACITY), > > 1722 "CPU capacity asymmetry not supported on SMT\n"); > > Yep, I've seen that. :) > > Thanks, > -Andrea > --