From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB88A15539A for ; Thu, 19 Feb 2026 21:47:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771537662; cv=none; b=YoIXB7Z4rO2UMmRbfi0VcJENlUPHfSkZYh18qe3P4J4zC1RiKOu2e1xKVWWf67RC1HCfmi4i7eoGxw0ED0QAY2/1JuH/Bq8/cGw0CMSPp5FPmh3U6Y4RtF4IYwoodgr66Wtyuv2V1fz0H8wxYEHdXHlfIRoXJ7L+rX+Mbwey60A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771537662; c=relaxed/simple; bh=MQNjGBavSVdm/9wJ/6sLaN6O9pivate3Cj8pHJugVtM=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=ZgeoJXKl9gqDHw2h2F8wAX66sRyWgp5iQavqB+zKEJXLFt756ZwlbmFrfAN6eK/beZsRRaQms+VbPb0WpGidei/1JnFfEHwgNboYp2q0/32AoLoNYyzxzc459E0E4Tk6JjOpmDYQtYbtqKVwNv6q21a9PSbMiNxOBwlzCmncl1U= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gPLjLu6Z; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gPLjLu6Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1771537661; x=1803073661; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=MQNjGBavSVdm/9wJ/6sLaN6O9pivate3Cj8pHJugVtM=; b=gPLjLu6ZiUr6nla/KkGclWc1Nb+K0oR4lZ3e3SRFPeR3GyLnfNqZXABh hfI0hKehGKDVlkOv88wLMoFv8I6rsT7K/q9rCmo0DRcn0k/MU134DlBDc T0IpJq2tUNbrxG2Z07MAzwes3OcAu3hCHYMnX+IwYRwhbHDnR0q2vnAn2 9/sf+tCPWM5z3X6c71u/21DvLMYP4O8+nV0DmXZMqFga1Td3++D2hisYx f6iNg27ESLohKTUqysMQqyzPNlg98CTj8nLhLqzEo30NU4I/6/GqD76EY upCrfTWERcdtpWtfHDJ9znOZXpdIh1jPfA9WGXfDqD2kTzgjStZ5p8YyE g==; X-CSE-ConnectionGUID: aOZJm8/ySdutKIKAUB6T2Q== X-CSE-MsgGUID: N1C6LBqHSpSY1/9MHI794g== X-IronPort-AV: E=McAfee;i="6800,10657,11706"; a="95257216" X-IronPort-AV: E=Sophos;i="6.21,300,1763452800"; d="scan'208";a="95257216" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2026 13:47:40 -0800 X-CSE-ConnectionGUID: KZOXmUo2SeaUMsffaPfPjg== X-CSE-MsgGUID: j7s6E4eESHOCoz6DZSd02w== X-ExtLoop1: 1 Received: from unknown (HELO [10.241.243.83]) ([10.241.243.83]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Feb 2026 13:47:39 -0800 Message-ID: <2822d23a5f3601073b95627075f7d1eac20e9694.camel@linux.intel.com> Subject: Re: [PATCH v3 00/21] Cache Aware Scheduling From: Tim Chen To: Qais Yousef , Peter Zijlstra Cc: Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Libo Chen , linux-kernel@vger.kernel.org Date: Thu, 19 Feb 2026 13:47:39 -0800 In-Reply-To: <20260219194845.jqu5ydhid7csukpk@airbuntu> References: <20260219140828.a7pyzupun7lsdw34@airbuntu> <20260219144108.GI1282955@noisy.programming.kicks-ass.net> <20260219194845.jqu5ydhid7csukpk@airbuntu> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Thu, 2026-02-19 at 19:48 +0000, Qais Yousef wrote: > On 02/19/26 15:41, Peter Zijlstra wrote: > > On Thu, Feb 19, 2026 at 02:08:28PM +0000, Qais Yousef wrote: > > > On 02/10/26 14:18, Tim Chen wrote: > > > > This patch series introduces infrastructure for cache-aware load > > > > balancing, with the goal of co-locating tasks that share data withi= n > > > > the same Last Level Cache (LLC) domain. By improving cache locality= , > > > > the scheduler can reduce cache bouncing and cache misses, ultimatel= y > > > > improving data access efficiency. The design builds on the initial > > > > prototype from Peter [1]. > > > >=20 > > > > This initial implementation treats threads within the same process = as > > > > entities that are likely to share data. During load balancing, the > > >=20 > > > This is a very aggressive assumption. From what I've seen, only few t= asks truly > > > share data. Lumping everything in a process together is an easy way t= o > > > classify, but I think we can do better. > >=20 > > Not without more information. And that is something we can always add > > later. But like you well know, it is an uphill battle to get programs t= o > > explain/annotate themselves. >=20 > Yes. I think we should be able to come up with a daemon to profile a work= load > on a machine and come up with a recommendation of tasks that have data > co-dependency. >=20 > Note I strongly against programs specifying this themselves. We need to p= rovide > a service that helps with the correct tagging - ie: it is an admin only > operation. >=20 > >=20 > > The alternative is sampling things using the PMU, see which process is > > trying to access which data, but that too is non-trivial, not to mentio= n > > it will get people really upset for consuming PMU resources. >=20 > I was hoping we can tell which data structures are shared between tasks w= ith > perf? >=20 > I am thinking this is not something that need to run continuously. But > disocvered one time off on a machine or once every update. The profiling = can be > done once (on demand) I believe. >=20 > Still if someone really wants to tag all the tasks for a process to stay > together, I think this is fine if that's what they want. I can envision that with tagging tasks with the same cookie that's analogou= s to what we are doing for core scheduling. Or grouping tasks by tagging a cgroup. >=20 > >=20 > > Starting things with a simple assumption is fine. This can always be > > extended. Gotta start somewhere and all that. It currently groups thing= s > > by mm_struct, but it would be fairly straight forward to allow userspac= e > > to group tasks manually. > >=20 > > > > scheduler attempts to aggregate such threads onto the same LLC doma= in > > > > whenever possible. > > >=20 > > > I admit yet to look fully at the series. But I must ask, why are you = deferring > > > to load balance and not looking at wake up path? LB should be for cor= rections. > > > When wake up path is doing wrong decision all the time, LB (which is = super slow > > > to react) is too late to start grouping tasks? What am I missing? > >=20 > > There used to be wakeup steering, but I'm not sure that still exists in > > this version (still need to read beyond the first few patches). It isn'= t > > hard to add. > >=20 > > But I think Tim and Chen have mostly been looking at 'enterprise' > > workloads. > >=20 > > > In my head Core Scheduling is already doing what we want. We just nee= d to > > > extend it to be a bit more relaxed (best effort rather than completel= y strict > > > for security reasons today). This will be a lot more flexible and wil= l allow > > > tasks to be co-located from the get-go. And it will defer the respons= ibility of > > > tagging to userspace. If they do better or worse, it's on them :) It = seems you > > > already hit a corner case where the grouping was a bad idea and doing= some > > > magic with thread numbers to alleviate it. > >=20 > > No, Core scheduling does completely the wrong thing. Core scheduling is > > set up to do co-scheduling, because that's what was required for that > > whole speculation trainwreck. And that is very much not what you want o= r > > need here. > >=20 > > You simply want a preference to co-locate things that use the same data= . > > Which really is a completely different thing. >=20 > Hmm. Isn't the infra the same? We have a group of tasks tagged with a coo= kie > that needs to be co-located. Core scheduling is strict to keep them on th= e same > physical core, but the concept can be extended to co-locate on LLC or clo= sest > cache? >=20 In my understanding, core scheduling doesn't try to place the tasks with the same cookie on the same core, but the tasks can safely be scheduled together in SMTs on a core. However, we can certainly use a similar cookie mechanism to indicate tasks should be scheduled close to each other cache wise. > >=20 > > > FWIW I have come across cases on mobile world were co-locating on a c= luster or > > > a 'big' core with big L2 cache can benefit a small group of tasks. So= the > > > concept is generally beneficial as cache hierarchies are not symmetri= cal in > > > more systems now. Even on symmetrical systems, there can be cases mad= e where > > > two small data dependent task can benefit from packing on a single CP= U. > >=20 > > Sure, we all know this. pipe-bench is a prime example, it flies if you > > co-locate them on the same CPU. It tanks if you pull them apart (except > > SMT siblings, those are mostly good too). >=20 > +1 >=20 > >=20 > > > I know this changes the direction being made here; but I strongly bel= ieve the > > > right way is to extend wake up path rather than lump it solely in LB = (IIUC). > >=20 > > You're really going to need both, and LB really is the more complicated > > part. On a busy/loaded system, LB will completely wreck things for you > > if it doesn't play ball. >=20 > Yes I wasn't advocating for wake up both only of course. But I didn't rea= d all > the details but I saw no wake up done. >=20 > And generally as I think I have been indicating here and there; we do nee= d to > unify the wakeup and LB decision tree. With push lb this unification beco= me > a piece of cake if the wakeup path already handles the case. The current = LB > is a big beast. And will be slow to react for many systems. I think as long as we have up to date information on load at the time of pu= sh in push lb, so we don't cause over aggregation and too much load imbalance, it will be viable to make such aggregation at wake up. Tim