From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B76FB28371 for ; Tue, 17 Feb 2026 21:20:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771363215; cv=none; b=TVwd5JbFQsiKihns8qOQX4LzxoKWEwcme7PfQymWpy6Mv/B6Z/dWAxjaCwP6GhQHQM81EimWKMpvHr2De9g1h5AwPDgH2QVLPG6g93Aukr3fggKeDnpG9lx76ZXuXtMsUH2JjNTVQkWQZPIIQfX5rwduocpmZwb2ArxPM09vf68= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771363215; c=relaxed/simple; bh=m2ZxfzzFFOv+ur/maEjpt7b59xJyDzoVYXroyAfIVKs=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=iRpzySOz5VAp7osA/TfUz0w4pA/SSY6nhkuWDRQD6mTzYghcH4oCBWY1vi5Do5UH2CKi1ndxwgRCfEvYzTVfBWXDQzGvV63/lKw2N5QkBPSiUqAiUk7EJHP+c+NRvY38fd9O4DL6k8SB+h/OyRC1mS4yX9wdnA/8/0s2n5Q2zYk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LvV4gZQ7; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LvV4gZQ7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1771363213; x=1802899213; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=m2ZxfzzFFOv+ur/maEjpt7b59xJyDzoVYXroyAfIVKs=; b=LvV4gZQ7SSpH/caI57BCHx7Th4W3m0NSLLMfLASAR6+4nvTRc3BGztO7 I7nuZ6ejjSa0YA6CkeC0A1fOS5Sg6mR5Y7krTVvfn7w5qU8Hrz6ugV3/P IlQaUtfcWfvw3sOKGeigFCwPAZj3UYTtX+uoPySWQ6npyV1xg5JIWhzKZ PSbnViBNFIny68ADkxGsCXU3p+S1O8yk+VHqdyX4RuGFiPkZq65tYEQxW uqv1whDQuOYgbxUDfa8Y8sZuv3qyc6UzQ5a+11gkriheZSh8Uqu676C+N Gj7MdIXY2pyfOP56t1TH631BDbk0bVrVpit4I6KRnS+J/8NJXNPEb3jsk Q==; X-CSE-ConnectionGUID: ZCXNs6tySh2c8zYS8SiNTA== X-CSE-MsgGUID: r9u7d7mERk+T8XMuzown5Q== X-IronPort-AV: E=McAfee;i="6800,10657,11704"; a="97903536" X-IronPort-AV: E=Sophos;i="6.21,297,1763452800"; d="scan'208";a="97903536" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2026 13:20:12 -0800 X-CSE-ConnectionGUID: d0CcnO+4R/6MgDruXH2Qtg== X-CSE-MsgGUID: Upf9au3nRK+tXOrLrfn2cQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,297,1763452800"; d="scan'208";a="237004875" Received: from unknown (HELO [10.241.243.83]) ([10.241.243.83]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2026 13:20:12 -0800 Message-ID: <107997571531b4fdd653f5f72b7af6969410d8e3.camel@linux.intel.com> Subject: Re: [PATCH v3 04/21] sched/cache: Make LLC id continuous From: Tim Chen To: Madadi Vineeth Reddy , "Chen, Yu C" Cc: Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Qais Yousef , Libo Chen , linux-kernel@vger.kernel.org Date: Tue, 17 Feb 2026 13:20:11 -0800 In-Reply-To: <796e4859-82da-422a-925f-7aac7cb86822@linux.ibm.com> References: <60a05a3f50d14a7bf3b968f62cca87893c5c552c.1770760558.git.tim.c.chen@linux.intel.com> <437fef08-cabe-461f-a2d2-4bc385e9d513@linux.ibm.com> <54e60704-b0f3-44df-9b83-070806b5a00c@intel.com> <796e4859-82da-422a-925f-7aac7cb86822@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Tue, 2026-02-17 at 15:35 +0530, Madadi Vineeth Reddy wrote: > On 15/02/26 19:55, Chen, Yu C wrote: > > On 2/15/2026 1:53 AM, Madadi Vineeth Reddy wrote: > > > On 11/02/26 03:48, Tim Chen wrote: > > > > From: Chen Yu > > > >=20 > > > > Introduce an index mapping between CPUs and their LLCs. This provid= es > > > > a continuous per LLC index needed for cache-aware load balancing in > > > > later patches. > > > >=20 > > > > The existing per_cpu llc_id usually points to the first CPU of the > > > > LLC domain, which is sparse and unsuitable as an array index. Using > > > > llc_id directly would waste memory. > > > >=20 > > > > With the new mapping, CPUs in the same LLC share a continuous id: > > > >=20 > > > > =C2=A0=C2=A0 per_cpu(llc_id, CPU=3D0...15)=C2=A0 =3D 0 > > > > =C2=A0=C2=A0 per_cpu(llc_id, CPU=3D16...31) =3D 1 > > > > =C2=A0=C2=A0 per_cpu(llc_id, CPU=3D32...47) =3D 2 > > > > =C2=A0=C2=A0 ... > > > >=20 > > > > Once a CPU has been assigned an llc_id, this ID persists even when > > > > the CPU is taken offline and brought back online, which can facilit= ate > > > > the management of the ID. > > >=20 > > > tl_max_llcs is never reset across multiple invocations of build_sched= _domains(). > > > While this preserves LLC IDs across normal CPU hotplug events, I'm wo= ndering about > > > scenarios where hardware topology changes, such as physically removin= g/replacing > > > CPU sockets. > > >=20 > > > Example scenario: > > > Boot with 3 LLCs: IDs {0,1,2}, tl_max_llcs=3D3 > > > Physical hardware change removes LLC 1 > > > New hardware added at a different position gets ID=3D3 > > > After multiple such events: System has 4 LLCs but IDs {0,2,5,7}, tl_m= ax_llcs=3D8 > > >=20 > >=20 > > I agree that keeping tl_max_llcs non-decreasing might waste some space.= The > > original motivation for introducing a dynamic sd_llc_id was mainly that= a > > static sd_llc_id[NR_LLC] is not suitable, as we cannot find a proper up= per > > limit for NR_LLC-an arbitrary value for NR_LLC is unacceptable. That is= to > > say, tl_max_llcs serves as the historical maximum LLC index that has ev= er > > been detected - like other terms such as CPU id. It is possible that th= e > > number of available LLCs shrinks due to CPU offline after boot-up. A va= lue > > of tl_max_llcs=3D8 indicates that this system once had 8 valid LLCs. On= the > > other hand, dense mapping is a side effect of dynamically allocating sd= _llc_id. > >=20 > > > This creates gaps in the ID space. However, I understand this trade-o= ff might be > > > intentional since physical topology changes are rare, and resetting t= l_max_llcs and > > > all sd_llc_id values would rebuild IDs on every invocation of build_s= ched_domains(). > > >=20 > > > Would like to know your thoughts on overhead of resetting tl_max_llcs= and sd_llc_id > > > so that IDs are rebuilt on each invocation of build_sched_domains() t= o always maintain > > > a dense mapping. > > >=20 > >=20 > > The current implementation is intentionally kept simple for easier revi= ew, and > > I agree that strictly enforcing a dense mapping for sd_llc_id - by reca= lculating > > the actual maximum LLC count (max_llcs) whenever the CPU topology chang= es - could > > be an optimization direction once the basic version has been accepted. = I assume what > > you are suggesting is that we could reset tl_max_llcs/max_llcs/sd_llc_i= d for CPUs > > in doms_new[i] within partition_sched_domains_locked() - and then rebui= ld these > > values in build_sched_domains() accordingly. One risk here is a race co= ndition when > > modifying the llc_id of a specific CPU - but off the top of my head, va= lid_llc_buf() > > should help prevent out-of-range access to sd->pf caused by such races. > > Thoughts? >=20 > Yes, resetting and rebuilding would maintain dense mapping. Given the add= ed complexity=20 > of race conditions vs. minimal benefit (gaps only occur with physical top= ology changes), > I think the current approach is better. We can revisit it once this versi= on goes through. >=20 The current implementation keep LLC id unchanged across sched domain rebuil= d. The idea was to allow pf[id] to be kept across rebuilds, and point to the same LLC. That said, now that we clear pf[id] across sched domain rebuild, this const= raint can be relaxed. And it should be okay to change the LLC id from the perspectiv= e of cache aware scheduling. However, there could be some transient races with cpus_share_cache() while = the LLC id got changed, which the current implementation avoid. Tim