From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABCC736C593 for ; Tue, 24 Feb 2026 09:43:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771926203; cv=none; b=teL997VA0Dm40pkyGC7F5QmSRL6O+Gd+yxewMe6bxCqqXplib3dX0i69S1ayfcdwcLBtZk+EuSFl5glXrgC6k/Gh+vnDHtV8cVAnkc3FQ8BLcmUH9mRNeSu8Hps7lXCOFeIv53He7Nvh01i+tZQspWlCBA+2vneB/I7lQ6foGsI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771926203; c=relaxed/simple; bh=9EnqI2AT/ynb8Tl0uGDhawplFS2GsNIxptEFERG0Qu8=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=aurV6mgxbF13AoP9XUIzBuBDTfCW1pUeWCxv7/OruThX++QQvxJ2O8YBIP62X7VIoR8Wc00E9Gyfi/2hHAliEv5jzJNAjDbZ3ws9I636n0gf7dZiXchllsTJYVvk5Kr2W+uni7BrJ1EmW81gbaD7d3OXgUGDDiZJy7V/OsHFBDg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dRPjv+Hz; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dRPjv+Hz" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61O844tF2347021; Tue, 24 Feb 2026 09:42:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=78QEPW adtxQR2Pm8h1v0EtJSAGoVqAhghiNgyGmeVNY=; b=dRPjv+HzWOCt5+bzh65yGS /ym9UhK2T3aKkc8Hjekdhzu1sHjlXiDwEL5sSUQ9xksOafKWc5RWJVJNE4sWY1FO cT59tfafM1taCMZSAJyl6hd/3WQWjiYXYqciCtM97/uTc5SMqxip5Q+YKX+Pwv5S bGSSjCfid2ZBuuMVZFQK7B1CZKVrYo5YuZTPzVFyz9fvBdr2XUB+r3jaSObdZI8q Mvq78oksDEmcXgiRh/V1o+NTBXrXY0Cw93jZGVV/GM/Z8KrtoURXzKvNpzBwtb+U psm24EkWW+WZllnN7KNFwssIgcEGmy3Cyhgq2/IavdX09urib/ccn8jDuALvlwqw == Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4ch858gayg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 24 Feb 2026 09:42:29 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 61O7kpfw030342; Tue, 24 Feb 2026 09:42:27 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cfrhk88n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 24 Feb 2026 09:42:27 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 61O9gQlv21168834 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 24 Feb 2026 09:42:26 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 886DF5806F; Tue, 24 Feb 2026 09:42:26 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 584CF5807B; Tue, 24 Feb 2026 09:42:16 +0000 (GMT) Received: from [9.98.110.226] (unknown [9.98.110.226]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Tue, 24 Feb 2026 09:42:16 +0000 (GMT) Message-ID: Date: Tue, 24 Feb 2026 15:12:11 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 15/21] sched/cache: Disable cache aware scheduling for processes with high thread counts To: Peter Zijlstra Cc: Tim Chen , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Chen Yu , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Qais Yousef , Libo Chen , linux-kernel@vger.kernel.org, Madadi Vineeth Reddy References: <20260219165507.GN1395266@noisy.programming.kicks-ass.net> <44a3bf9b-b728-4c33-8972-dbd1a3a873e2@linux.ibm.com> <20260220095327.GG2995752@noisy.programming.kicks-ass.net> Content-Language: en-US From: Madadi Vineeth Reddy In-Reply-To: <20260220095327.GG2995752@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjI0MDA4MSBTYWx0ZWRfX/02OwblzC6aI 9LKck1lnx1CtXSkKwDr9ELcIETj3DQRneucu0qU9e/5EE4eJFc66hCFTgtk36nQE8eKhQWDD+jd gP06eihP6oZUvGxVNaglTtmxYUdQ6PQS/UFi0In45GC+0w5NNnq/VZyz+JlHhHrOm9oVKE7ZoIy jd/j9kUiehfKiZ7+2ohcM2rzGvLhBsfTXzv/TUtSU/6l9n1sFLW5HDj0dXeBIqTLue3vc0phIKP KEOfzS6MPLHiyKEoSSnHyMMfg9J4CY2q/tQxGOm4IhYn97Vd+tbcBVwkB34zCtiQ0EO3Jb6F1pb JOv7CCtq4l7+09PYDYHhacmNpMFOjN+Qx06D0pH49maX2CU3t2bloy9mZuc3NtNnx2NVUod57td ThzrXTBPsOyQ0Zn/ZPbIjq/u1gXQGsbZXHwsfqB998HEnYEbICTItYcQIWTYgq4ONsWUvXyoThR hDnidqXmQsyG9WzaBSg== X-Proofpoint-GUID: Q6b-iNtpRg3EdNo5KGIvx2qZZuoah8Qd X-Authority-Analysis: v=2.4 cv=S4HUAYsP c=1 sm=1 tr=0 ts=699d7285 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=IkcTkHD0fZMA:10 a=HzLeVaNsDn8A:10 a=VkNPw1HP01LnGYTKEx00:22 a=Mpw57Om8IfrbqaoTuvik:22 a=GgsMoib0sEa3-_RKJdDe:22 a=1mVJ_uiqAAAA:8 a=ZY7IewyQZ3kdXkKQZlAA:9 a=QEXdDO2ut3YA:10 a=h67g7WpEjx8dfGT80pje:22 X-Proofpoint-ORIG-GUID: Tsa8y1ljqGTmEBFXU6OVe_PBC6RtB1dN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-02-24_01,2026-02-23_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 clxscore=1015 impostorscore=0 malwarescore=0 bulkscore=0 phishscore=0 adultscore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2602240081 Hi Peter, Sorry for the delayed response. Wanted to be sure before responding. On 20/02/26 15:23, Peter Zijlstra wrote: > On Fri, Feb 20, 2026 at 12:10:21PM +0530, Madadi Vineeth Reddy wrote: >> Hi Peter, >> >> On 19/02/26 22:25, Peter Zijlstra wrote: >>> On Wed, Feb 18, 2026 at 11:24:05PM +0530, Madadi Vineeth Reddy wrote: >>>> Is there a way to make this useful for architectures with small LLC >>>> sizes? One possible approach we were exploring is to have LLC at a >>>> hemisphere level that comprise multiple SMT4 cores. >>> >>> Is this hemisphere an actual physical cache level, or would that be >>> artificial? >> >> It's artificial. There is no cache being shared at this level but this is >> still the level where some amount of cache-snooping takes place and it is >> relatively faster to access the data from the caches of the cores >> within this domain. >> >> We verified with this producer consumer workload where the producer >> and consumer threads placed in the same hemisphere showed measurably >> better latency compared to cross-hemisphere placement. > > So I just read the Power10 Wikipedia entry; that seems to suggest there > actually is a significant L3 at the hemisphere level. > > That thing states that Power10 has: > > - 16 cores in two hemispheres of 8 cores each. > - each core has 2M L2 cache > - each hemi has 64M of L3 cache The Wikipedia entry is incorrect. On Power10, L3 is at the SMT4 small core level (4M per core), not at the hemisphere level. This is documented in the Power10 user manual [1] (Page 175). L3 is also a victim cache on Power10. > > Then there appears to be a 'funny' in that there's always one 'dead' > core, so you end up with 8+7, and the small hemi looses an 8M L3 slice > due to that. > > Now, I'm just reading a Wiki pages written by a random person on the > interweb, so perhaps this is wrong (in which case I would suggest you Yes, the Wikipedia page is wrong on this. We will get it corrected with proper references. [1] https://files.openpower.foundation/s/EgCy7C43p2NSRfR Thanks, Vineeth > get someone from IBM to go and edit that page and provide references), > or there has been a miscommunication somewhere else, and perhaps there > really is L3 at the hemi level, and arch/powerpc/ 'forgot' to expose > that :-)