From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 963872D877A for ; Thu, 19 Feb 2026 19:48:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771530531; cv=none; b=GgoY6YOiQwu3A4Xt9yFqSc/YaqVpDreYLJ1mVwIys2mtNCPvkQ5XqNUYLyqWNp09VjZyqmzmawAbmPi4DQtOxyYgMyXmp2n1WFfNybem9CpDpLh4RKXEvl1UU6iXZmF1VPXbSMjfZAi44YnwBAIErPOnQe33DejMxvHt1JaPkng= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771530531; c=relaxed/simple; bh=rfogsnam97ZPKbBqpVXVSyQVn5qvolgBtWzJvvUA9Ic=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sNiJqvtyuPyTXf2I5SR3XRNzcFTjaUAw7I0uIh8drS0KTcJE5lIpsUzxIfwpiKn7E2VOgUIdq3GdDUfroFK1PEwLlW4URxcvJHQmE1tsxoGzw3A+KTxJOJxxBCcmYyrbBfzHt0ulxmC5EN4CohQqsswPnggLVqulqlj+uF4OSUE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=GkXzc/rH; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="GkXzc/rH" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-4806bf39419so17410425e9.1 for ; Thu, 19 Feb 2026 11:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1771530528; x=1772135328; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=p8GLbECZRc02xb/2IrPLHoc8OM+vJU9kq/nSsJ84DvY=; b=GkXzc/rHujX1dwTZKaBecRFx6pSTytLEXQcCeu6GssQ9AEjIsgaZLGrpznGy4r0sTB Fodcqar+E4ifFKFXtq5QsY81GpWw4Rr1VCdh/eW/fwnzgAsu8s2hwG/tye2udFEyAlQG ub/454Pv5XALlYZ+dnlRksKQZDa5tnK7gGCQopyoPl6tVOJZGDZxbfuXaDf8zCacSErx yPEes7ZcACFk2pdp310BC9g8oH59Ff52c4T5ZgGcqE3QSMZR+HDxUSVNz8Y8/0Prohh7 Xk8HAtZ4QWp2mFTmwUmGJ0CfkSjWjRrywOwgjOW5E3rjv+wGuBVc+DLRrk5uAoOpDeXA /+6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771530528; x=1772135328; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p8GLbECZRc02xb/2IrPLHoc8OM+vJU9kq/nSsJ84DvY=; b=tYyUK0bg7/tj9s6i4WIpMDIVE/KudhigqU8Vm+i0fRVWxbYLF1h6pBJGGsENvVda2V Xvp4LjtsBG1w2MiIBgjSDB1Me1YA8SRqJptr2B7OskH6ijV+c4ATPHRit5AWsfIOE5mV 9KmrLf86L21YZXG7Vi4dqnGRILQQ2/ua4mR1HSF7Gh5FgfHmzTEokhvMBVJfk/s3KDEq B0+5+FIgdY9QsLLyhgUwha2gDdV0i/HQVO6INNCyNazU6sQAamItUeU43x0eIPw4jO7Q OvWv8VBwNWd7dIptNw5+dQUo6D4Bo/4iR8iLjEdqediIA7AudFzH0iIKZCmIWExtbrFc i1yg== X-Forwarded-Encrypted: i=1; AJvYcCW8wvS3ol3JLIM1d6zL4FZFAjJE4JY6PeuH1dxSNP0myRVbW/nef4lwQi3ockjgNE+lz/vdAJqYLoVcZ9s=@vger.kernel.org X-Gm-Message-State: AOJu0YztbXH2vzOiUBX+BpHQE2alSoMgtYPPwopCs4ObP3UmZh9HBLra joxivLL8HN4qvTfv8hFcuJN5AFO44WlH3xYsJQDvKJpbiBWeNarP+0UK4Ptxpcjcx0o= X-Gm-Gg: AZuq6aLvCEjTrb/m0s4Bs70h8rElQTIkOOjbeoqQpMe4Z9F4QTDhKYNzCdXtrykEyAF xBYd+2MLRxwqedTweq4AS4g1STPesAL+exqajOd6wVXJjOXSB0Qr2DlYogm0xOKZ/Sv1FEAUMeJ 4TRzntypPt3CXdYkD31z7GkTCjvJl4LPujgYmbFps/1Mr4Qyz4748b05Qtb3cCHR/2f1HtX9Dep llU3CMlYfY2ow/sT9frNH9ReQiPHiS1Ze6NkiIra3IS7oGZiZn9OcVeyGfOzmm2Uj5DjxK36UEu JrX74iYusuYoqKvW1MlLae7gW4YoC5PE1lg8KDPi/c2pg4PLiuM4xe/ZbTwPDY8DNTgNw2xq2ls kOpxNQLbrqbiMEsLFmczUVTFdIrCYkZtdsIwK6zg7uL5hF11uSswTddJENKJaB4dgep+U8qXa30 7iH54DIFejv+l692wlUlBxrAu9jrXS7b+/hTf2nInsCyGlL0mb579EWPYID+1e5b7uQ5XW X-Received: by 2002:a05:6000:290c:b0:437:71b2:6f1a with SMTP id ffacd0b85a97d-43962540637mr5892718f8f.0.1771530527541; Thu, 19 Feb 2026 11:48:47 -0800 (PST) Received: from airbuntu (host86-169-41-76.range86-169.btcentralplus.com. [86.169.41.76]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43796ac8075sm48361318f8f.29.2026.02.19.11.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Feb 2026 11:48:47 -0800 (PST) Date: Thu, 19 Feb 2026 19:48:45 +0000 From: Qais Yousef To: Peter Zijlstra Cc: Tim Chen , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Libo Chen , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 00/21] Cache Aware Scheduling Message-ID: <20260219194845.jqu5ydhid7csukpk@airbuntu> References: <20260219140828.a7pyzupun7lsdw34@airbuntu> <20260219144108.GI1282955@noisy.programming.kicks-ass.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20260219144108.GI1282955@noisy.programming.kicks-ass.net> On 02/19/26 15:41, Peter Zijlstra wrote: > On Thu, Feb 19, 2026 at 02:08:28PM +0000, Qais Yousef wrote: > > On 02/10/26 14:18, Tim Chen wrote: > > > This patch series introduces infrastructure for cache-aware load > > > balancing, with the goal of co-locating tasks that share data within > > > the same Last Level Cache (LLC) domain. By improving cache locality, > > > the scheduler can reduce cache bouncing and cache misses, ultimately > > > improving data access efficiency. The design builds on the initial > > > prototype from Peter [1]. > > > > > > This initial implementation treats threads within the same process as > > > entities that are likely to share data. During load balancing, the > > > > This is a very aggressive assumption. From what I've seen, only few tasks truly > > share data. Lumping everything in a process together is an easy way to > > classify, but I think we can do better. > > Not without more information. And that is something we can always add > later. But like you well know, it is an uphill battle to get programs to > explain/annotate themselves. Yes. I think we should be able to come up with a daemon to profile a workload on a machine and come up with a recommendation of tasks that have data co-dependency. Note I strongly against programs specifying this themselves. We need to provide a service that helps with the correct tagging - ie: it is an admin only operation. > > The alternative is sampling things using the PMU, see which process is > trying to access which data, but that too is non-trivial, not to mention > it will get people really upset for consuming PMU resources. I was hoping we can tell which data structures are shared between tasks with perf? I am thinking this is not something that need to run continuously. But disocvered one time off on a machine or once every update. The profiling can be done once (on demand) I believe. Still if someone really wants to tag all the tasks for a process to stay together, I think this is fine if that's what they want. > > Starting things with a simple assumption is fine. This can always be > extended. Gotta start somewhere and all that. It currently groups things > by mm_struct, but it would be fairly straight forward to allow userspace > to group tasks manually. > > > > scheduler attempts to aggregate such threads onto the same LLC domain > > > whenever possible. > > > > I admit yet to look fully at the series. But I must ask, why are you deferring > > to load balance and not looking at wake up path? LB should be for corrections. > > When wake up path is doing wrong decision all the time, LB (which is super slow > > to react) is too late to start grouping tasks? What am I missing? > > There used to be wakeup steering, but I'm not sure that still exists in > this version (still need to read beyond the first few patches). It isn't > hard to add. > > But I think Tim and Chen have mostly been looking at 'enterprise' > workloads. > > > In my head Core Scheduling is already doing what we want. We just need to > > extend it to be a bit more relaxed (best effort rather than completely strict > > for security reasons today). This will be a lot more flexible and will allow > > tasks to be co-located from the get-go. And it will defer the responsibility of > > tagging to userspace. If they do better or worse, it's on them :) It seems you > > already hit a corner case where the grouping was a bad idea and doing some > > magic with thread numbers to alleviate it. > > No, Core scheduling does completely the wrong thing. Core scheduling is > set up to do co-scheduling, because that's what was required for that > whole speculation trainwreck. And that is very much not what you want or > need here. > > You simply want a preference to co-locate things that use the same data. > Which really is a completely different thing. Hmm. Isn't the infra the same? We have a group of tasks tagged with a cookie that needs to be co-located. Core scheduling is strict to keep them on the same physical core, but the concept can be extended to co-locate on LLC or closest cache? > > > FWIW I have come across cases on mobile world were co-locating on a cluster or > > a 'big' core with big L2 cache can benefit a small group of tasks. So the > > concept is generally beneficial as cache hierarchies are not symmetrical in > > more systems now. Even on symmetrical systems, there can be cases made where > > two small data dependent task can benefit from packing on a single CPU. > > Sure, we all know this. pipe-bench is a prime example, it flies if you > co-locate them on the same CPU. It tanks if you pull them apart (except > SMT siblings, those are mostly good too). +1 > > > I know this changes the direction being made here; but I strongly believe the > > right way is to extend wake up path rather than lump it solely in LB (IIUC). > > You're really going to need both, and LB really is the more complicated > part. On a busy/loaded system, LB will completely wreck things for you > if it doesn't play ball. Yes I wasn't advocating for wake up both only of course. But I didn't read all the details but I saw no wake up done. And generally as I think I have been indicating here and there; we do need to unify the wakeup and LB decision tree. With push lb this unification become a piece of cake if the wakeup path already handles the case. The current LB is a big beast. And will be slow to react for many systems.