From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33F0C320A0D for ; Tue, 17 Feb 2026 10:07:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771322828; cv=none; b=XWp73qgDePxNU5pLdvaNEQ35M0zd3MsBhDkGudlM0Bk0h3/ZfGLxEosApzvNUPI9wT2kzp8j0WRRcOAkoXnaMxDMucXFQLZhvfWwZ6uWLDZag+puN0J6dFUgpT34AFFXv4MpAbEw9pYGftFzX9d50tiIQQXqNC7qQ7aaU4cFC4A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771322828; c=relaxed/simple; bh=dmlW/7RnBew9v3Y12rT6WN/0+iLeaYemJpYq3qGCahI=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=T/pqV11GXY/reqtimac/jbI0RYpjj5SGIWFH3crneefWSI1MFP9Pq573cqO5HsZkzBhF5axwPiBulDDy/Wfw2HrtfkhGiufJjNwcu64kiyjB1MkYauHKzhHK85EBsXWoqf0C2AN1bSZI9thi8nx2+0Yt0TjaSR8NHuixeuS1yws= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=RPdhAdck; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="RPdhAdck" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61H8B8fm3203603; Tue, 17 Feb 2026 10:06:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=oFttOk BRf8kybXMFTI7QPyh3TSIrSSod+Sjp5HlCnzw=; b=RPdhAdckRQpu32HjWRr+Q6 3DoqJCgPfgEP/QsCO5FCy2MKaGAHTbzaq+8gXwXwpMt/sU2UDXJFTrl34SBC9qgI emPffdsyTVfkyyj2VPaED/7XaRsX4u8lzD1G/vfVgIQND+tUKvr+JWBDAdhb8jlG Le+dLae8IAHooEBj5CIZmD+dRPtsYQoIFodvXsunD+r+2+WIHYE3Ov78ezysdp9b YbVeTRpC3CPn8kSObOTCWerer9jb3ptuq/QCDOFG9N9h2KXwrDRAFLsaRW529cZu 62U50OVZWBU43npbvX8l/FtIfzaV8yyAPVnSidx5lqRO5mOkqnPzrli2uuCtg52Q == Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4caj6ukd5s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Feb 2026 10:06:02 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 61H8VMiC017792; Tue, 17 Feb 2026 10:06:00 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ccb289ymk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Feb 2026 10:06:00 +0000 Received: from smtpav01.wdc07v.mail.ibm.com (smtpav01.wdc07v.mail.ibm.com [10.39.53.228]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 61HA5xQS28836378 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Feb 2026 10:05:59 GMT Received: from smtpav01.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3252258066; Tue, 17 Feb 2026 10:05:59 +0000 (GMT) Received: from smtpav01.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B5EC058065; Tue, 17 Feb 2026 10:05:43 +0000 (GMT) Received: from [9.67.35.207] (unknown [9.67.35.207]) by smtpav01.wdc07v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Feb 2026 10:05:43 +0000 (GMT) Message-ID: <796e4859-82da-422a-925f-7aac7cb86822@linux.ibm.com> Date: Tue, 17 Feb 2026 15:35:40 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 04/21] sched/cache: Make LLC id continuous To: "Chen, Yu C" Cc: Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Qais Yousef , Libo Chen , linux-kernel@vger.kernel.org, Tim Chen , Madadi Vineeth Reddy References: <60a05a3f50d14a7bf3b968f62cca87893c5c552c.1770760558.git.tim.c.chen@linux.intel.com> <437fef08-cabe-461f-a2d2-4bc385e9d513@linux.ibm.com> <54e60704-b0f3-44df-9b83-070806b5a00c@intel.com> Content-Language: en-US From: Madadi Vineeth Reddy In-Reply-To: <54e60704-b0f3-44df-9b83-070806b5a00c@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=E+/AZKdl c=1 sm=1 tr=0 ts=69943d8a cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=IkcTkHD0fZMA:10 a=HzLeVaNsDn8A:10 a=VkNPw1HP01LnGYTKEx00:22 a=Mpw57Om8IfrbqaoTuvik:22 a=GgsMoib0sEa3-_RKJdDe:22 a=QyXUC8HyAAAA:8 a=NxqAXs4mv2Rzev04essA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjE3MDA4MCBTYWx0ZWRfX1VwO/OayO+pD 0NtYzs20v+VKRaERpeLnjGpgRD8p2tP4tzKvlPxNn6J7yEvpDTmlTCfE8Y8C2jM2BsGLE735vB7 +xBaF8Sn1e+YuQDZ261aOfDoTDJ5ajH0I9cOdbFjP5kGOUDJw5Poz4Al30HhoWNNg6fG6+01j86 qRwfndfgW6LRaGE7wRT/sOfu/TvyQIRLJxsYO/hFUwC3q1QOFLmeKBmpIphnIeNYPSCcDhm8sEi S21KUWnMLrUy8lcPr4W2WVWPMi83zRm62LS6YiMKQSNr05WVkurNqrELQiCO70AACQ9a8dOKLht XA3bkpamkPooXtM8uy9Nq455qEl+U7DuNQBlL6FwtBY5wP85X7JpuekTHUKoj6eeKbBk1WRvP5a b0oczh9QRSfOSLaz/I6LBTGK4UPCZek2H3RknYB3WIKcC9gvUCdRhb/83gir9MXkXKQ3O+iCFe8 XKHQEfo2RQ9XD1SJs4Q== X-Proofpoint-ORIG-GUID: 94ERxC1rnHC_HZiYTC0A5SJXqqgE-ynx X-Proofpoint-GUID: FdGYs2HpcDzfSSXuHmWqya2OmQvFbvRm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-02-17_01,2026-02-16_04,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 priorityscore=1501 malwarescore=0 phishscore=0 bulkscore=0 suspectscore=0 spamscore=0 clxscore=1015 impostorscore=0 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2602170080 On 15/02/26 19:55, Chen, Yu C wrote: > On 2/15/2026 1:53 AM, Madadi Vineeth Reddy wrote: >> On 11/02/26 03:48, Tim Chen wrote: >>> From: Chen Yu >>> >>> Introduce an index mapping between CPUs and their LLCs. This provides >>> a continuous per LLC index needed for cache-aware load balancing in >>> later patches. >>> >>> The existing per_cpu llc_id usually points to the first CPU of the >>> LLC domain, which is sparse and unsuitable as an array index. Using >>> llc_id directly would waste memory. >>> >>> With the new mapping, CPUs in the same LLC share a continuous id: >>> >>>    per_cpu(llc_id, CPU=0...15)  = 0 >>>    per_cpu(llc_id, CPU=16...31) = 1 >>>    per_cpu(llc_id, CPU=32...47) = 2 >>>    ... >>> >>> Once a CPU has been assigned an llc_id, this ID persists even when >>> the CPU is taken offline and brought back online, which can facilitate >>> the management of the ID. >> >> tl_max_llcs is never reset across multiple invocations of build_sched_domains(). >> While this preserves LLC IDs across normal CPU hotplug events, I'm wondering about >> scenarios where hardware topology changes, such as physically removing/replacing >> CPU sockets. >> >> Example scenario: >> Boot with 3 LLCs: IDs {0,1,2}, tl_max_llcs=3 >> Physical hardware change removes LLC 1 >> New hardware added at a different position gets ID=3 >> After multiple such events: System has 4 LLCs but IDs {0,2,5,7}, tl_max_llcs=8 >> > > I agree that keeping tl_max_llcs non-decreasing might waste some space. The > original motivation for introducing a dynamic sd_llc_id was mainly that a > static sd_llc_id[NR_LLC] is not suitable, as we cannot find a proper upper > limit for NR_LLC-an arbitrary value for NR_LLC is unacceptable. That is to > say, tl_max_llcs serves as the historical maximum LLC index that has ever > been detected - like other terms such as CPU id. It is possible that the > number of available LLCs shrinks due to CPU offline after boot-up. A value > of tl_max_llcs=8 indicates that this system once had 8 valid LLCs. On the > other hand, dense mapping is a side effect of dynamically allocating sd_llc_id. > >> This creates gaps in the ID space. However, I understand this trade-off might be >> intentional since physical topology changes are rare, and resetting tl_max_llcs and >> all sd_llc_id values would rebuild IDs on every invocation of build_sched_domains(). >> >> Would like to know your thoughts on overhead of resetting tl_max_llcs and sd_llc_id >> so that IDs are rebuilt on each invocation of build_sched_domains() to always maintain >> a dense mapping. >> > > The current implementation is intentionally kept simple for easier review, and > I agree that strictly enforcing a dense mapping for sd_llc_id - by recalculating > the actual maximum LLC count (max_llcs) whenever the CPU topology changes - could > be an optimization direction once the basic version has been accepted. I assume what > you are suggesting is that we could reset tl_max_llcs/max_llcs/sd_llc_id for CPUs > in doms_new[i] within partition_sched_domains_locked() - and then rebuild these > values in build_sched_domains() accordingly. One risk here is a race condition when > modifying the llc_id of a specific CPU - but off the top of my head, valid_llc_buf() > should help prevent out-of-range access to sd->pf caused by such races. > Thoughts? Yes, resetting and rebuilding would maintain dense mapping. Given the added complexity of race conditions vs. minimal benefit (gaps only occur with physical topology changes), I think the current approach is better. We can revisit it once this version goes through. Thanks, Vineeth > > thanks, > Chenyu >