From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC2543168E6 for ; Wed, 10 Dec 2025 15:14:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765379665; cv=none; b=psU7J+qPMXwwQQfeo1PDeTZPrr1pcoPJCaeaeMSGe2gTSbQRw/sDlsn4fnfVWTo58ipZ6kFLSYhtzlmWm1pqwJF8pz9hR3z0Sd5Grvmr7pJ/fNPzcnrecjQd53TXVo1mzioBdl0XUAwZInUi/6jiQt8oW5dtef/g9lpLg2GrH04= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765379665; c=relaxed/simple; bh=CH0kodd/6noysQ7ek7FKFanwr8QfId0tc9jWEEvXAgQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FfzOW15nJDAu0KqFEgtJpMuoFy8OTLhVKByHDykVvdTNtCkPfZgk3Itd1sNJzOD9uWoAl7nth0x6UOjQO209dXPB/qrUYX32a+++AXxGMtbV/ikmUkvKOnIxg+rLRXy9kDPy/4tY20i+awfhEwv3i7hFaU04L9CabhrsHpncGBM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=khHuZ1Vv; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="khHuZ1Vv" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=t4mskKzGFPcBDqPyKUF3Z9/FrWQmd1YiBvpwyJaPaJY=; b=khHuZ1VvkZDd58MBEJc8aDcIrt ryjbSi5eKAMNqm+XJw/HQQR6oZExVKl7mXquJrNthkQOTllGWyDXU8Z4H1CygwiwCyuQRPouEis6J IGoX9MUzEsVTzTvTk5J5z+5uJG518wch4/spLt+YPIC+Sqimtyg/yEPWOls4ptW8KZNw5H/3OTHS2 LKMbvf7j/OX+5jAR+zeHEfHOH+I+sjIjcQ1r/mPRTaLPmsKhWyauI1fTsAQ+DBGbIWLS2noDujuzo 6FDYyfJf7AvKNSCcgoKNzpeLFuqRIDggB47wOk/MHDOat2DpYn7UBn1sRBTHEiJX1StZuW4uBLyQx 8+mF14XA==; Received: from 2001-1c00-8d85-5700-266e-96ff-fe07-7dcc.cable.dynamic.v6.ziggo.nl ([2001:1c00:8d85:5700:266e:96ff:fe07:7dcc] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1vTL1V-0000000DU2s-3ggz; Wed, 10 Dec 2025 14:18:34 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 4F9B1302EC2; Wed, 10 Dec 2025 16:13:50 +0100 (CET) Date: Wed, 10 Dec 2025 16:13:50 +0100 From: Peter Zijlstra To: "Chen, Yu C" Cc: Tim Chen , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 02/23] sched/cache: Record per-LLC utilization to guide cache-aware scheduling decisions Message-ID: <20251210151350.GL3707837@noisy.programming.kicks-ass.net> References: <20251209112134.GK3707891@noisy.programming.kicks-ass.net> <58a92e45-3b43-4938-b10f-65f7fa7c5017@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <58a92e45-3b43-4938-b10f-65f7fa7c5017@intel.com> On Wed, Dec 10, 2025 at 11:02:39PM +0900, Chen, Yu C wrote: > On 12/9/2025 8:21 PM, Peter Zijlstra wrote: > > On Wed, Dec 03, 2025 at 03:07:21PM -0800, Tim Chen wrote: > > > > > diff --git a/include/linux/sched/topology.h b/include/linux/sched/topology.h > > > index bbcfdf12aa6e..0ba4697d74ba 100644 > > > --- a/include/linux/sched/topology.h > > > +++ b/include/linux/sched/topology.h > > > @@ -68,6 +68,10 @@ struct sched_domain_shared { > > > atomic_t nr_busy_cpus; > > > int has_idle_cores; > > > int nr_idle_scan; > > > +#ifdef CONFIG_SCHED_CACHE > > > + unsigned long util_avg; > > > + unsigned long capacity ____cacheline_aligned_in_smp; > > > > This cacheline annotation confuses me, see below. > > > > > +#endif > > > }; > > > struct sched_domain { > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > > index cb82f558dc5b..b9f336300f14 100644 > > > --- a/kernel/sched/fair.c > > > +++ b/kernel/sched/fair.c > > > @@ -9622,6 +9622,29 @@ static inline int task_is_ineligible_on_dst_cpu(struct task_struct *p, int dest_ > > > return 0; > > > } > > > +#ifdef CONFIG_SCHED_CACHE > > > +/* Called from load balancing paths with rcu_read_lock held */ > > > +static __maybe_unused bool get_llc_stats(int cpu, unsigned long *util, > > > + unsigned long *cap) > > > +{ > > > + struct sched_domain_shared *sd_share; > > > + > > > + sd_share = rcu_dereference(per_cpu(sd_llc_shared, cpu)); > > > + if (!sd_share) > > > + return false; > > > + > > > + *util = READ_ONCE(sd_share->util_avg); > > > + *cap = READ_ONCE(sd_share->capacity); > > > > You placed capacity on a separate line, forcing the above to be 2 > > distinct lines. That seems... sub-optimal? > > > > The reason capacity was placed in a separate cache line > is that writes to capacity are not very frequent(cpu hotplug > should not happen too-frequently), while writes to util_avg > tend to be relatively frequent(changes frequently). > If capacity and util_avg were placed in the same cache line, > I’m thinking writes to util_avg might invalidate the entire > cache line. This could potentially cause cache misses when > capacity is read elsewhere, which might lead to false sharing? But its introduced here and already read/written together. Is this not premature optimization?