From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16E7B2628D for ; Tue, 21 Apr 2026 00:34:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776731687; cv=none; b=lSf+ns422uUEfpXNdOZ+rlEn5WgfRqo16/+qWYYWYwaNUcFql6ZCLcvGHzOcGNKy6kIssqWUymbuszR9LkCjJ8WJINxFIIi893f1KX1qsXYs/r1J69YEWPYh4gr1HNTF2zghJfxzyxNgkyIp+IHO4923kPEDvRilvVDXLqq7Umc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776731687; c=relaxed/simple; bh=I8ipxSvI5GASXc8c8ZUlcGIS7pya7a+h8TOJSXJtSh8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ucuo95X73vHqhs2rARf/Vd/6ebRU+mBgT5aMTD9lK2fJRXBPYGxA+NfwJnQiEK4aQ2lPqXiYyKB0L7DNtMrwOO2sXeG0tOIszhNR7RjpFdJfIFe4hDbLfkyx7Y4UMta90Tvm3qRyC/n9wfhEw/orpioRZau3CgetRv/rGUXUsyY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20251104.gappssmtp.com header.i=@layalina-io.20251104.gappssmtp.com header.b=H+T8hBWd; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20251104.gappssmtp.com header.i=@layalina-io.20251104.gappssmtp.com header.b="H+T8hBWd" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-b936331786dso463764866b.3 for ; Mon, 20 Apr 2026 17:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20251104.gappssmtp.com; s=20251104; t=1776731683; x=1777336483; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=x3d6K0iTFkgVWl0Y3hwsPpm62z40ZxpsN6GQuxG83wU=; b=H+T8hBWd3IU3hu6UY3QR2NqPWONTiHPXY6QDdUgU7LD6LuzeWyxbTXtVflutUyDFjj hEz0DxLQoyhJSX7I0LFs5WsWJILfklFK5hcL9GOlBRxzhgp0qdX0joSUPS2iPuHNYciS x4iN9Q0v9oqVk7F4k1PL15MfSMeKjZXuSyaDOGVnUS0GunqekDWrmOYNWLRdlgaEOdH9 l/ZLBxqOUs+lAWCZ8CR/ry1Bl2Z7n7blo99x2U5v9NPpbBiHc7gKhT+nE8pjAr2VpC+z DAEkh8gg9SAsTRbWYmmHZHMEAlJj9L7VOVMuMKqMUyusBzYuIjYfpZUlqDR5PcbC/jrp n30Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776731683; x=1777336483; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=x3d6K0iTFkgVWl0Y3hwsPpm62z40ZxpsN6GQuxG83wU=; b=VZuuLg2R/R4oH8wFsAo8FFWbEcIJ+DQKoAnGPbgkCznFG9ncykMyBd4oLotR0lBOKx pqbCOVWmwcY1BQIcoOEJJxYdBWyHHAmqupqXqnogED/H8IVzTq2HyRgX2J/kHKDun07Y L3s/7KtWP8mcgkxXNlDk7FiV7xRZZOEn/cYzquNtVZR4plRA8p38QDyMUgTDi/xFrhzF +2IC6vcAf+wVcQPT6WgrfP9fofjxXxKg9Hhhlv3p757krY4KL51fLVnOfMjWhgW5YPrN 6bR6aLc6DTI+B644BOrTmnDeTCwasGL2bs5tuLfwQF1zNaQlu0TqSsoslxgzYVfotAfL 7n7g== X-Forwarded-Encrypted: i=1; AFNElJ8Yjr76evThCIjA/fm6u7WwpYA75C3HMe2D3I23iWXBSHfCqkm0g/PbRff10Z9AUPsIPjQFJLnH/dsVocA=@vger.kernel.org X-Gm-Message-State: AOJu0YwRKzPWUxjNS9m3pOP8kpO0GzMKfhM9cGQBdCOvt6e0ggWgz2L7 yoYC3zOor5v2spD8rWlAalF4IixEnVH1WjH1LQuYTEpamhR5203T2o83M1e8ICHyzPE= X-Gm-Gg: AeBDiesklTfKFwKvDlmSOhSp8gnTMVFg4kTcx7iEfWZnHPuHPFoyfwkSXjtDBYHcerL dDPNsKGHrz84XOvfzEDMT7xf2iDXOZulFYMpuyfYhGYxGTobDgGw1/OGYERpODWDhpXzXMjoG76 Ebb04b2dZlI2j/JzurhzTn6k6WEsZqnwjkK7KZjCqmIQzEZmZtLzA+MXTaAIxiDdeHodqpM76ww ys9cW/avXndzg6lDcB3Hwyx/dDIDjcv+yNv0gNG2pygweswwfOM6z1ls9xZqE3brqeEoJEdwzAW Zzpzb8VCIUmuPXz8rvvl7aucGplJIkuw45ldwY07i9wQn5QaOKmJZ53wavPIF0KOrqQcrxWhjPb w6R4l8dQIlb3O2YHPbyqus9uk2VX0yXtHS4EJxbH56JaMZaEt2Zuo0yxtQ+uoPhc5UIoCgF61Q9 YPX3dHnk+wqKbPT0sarMIkKgpk7ywqX3MDn9Z19Lo= X-Received: by 2002:a17:907:742:b0:b96:e593:fd1e with SMTP id a640c23a62f3a-ba415bae09cmr797799966b.0.1776731683148; Mon, 20 Apr 2026 17:34:43 -0700 (PDT) Received: from airbuntu ([146.70.179.103]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ba451cdfd27sm399993466b.26.2026.04.20.17.34.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Apr 2026 17:34:42 -0700 (PDT) Date: Tue, 21 Apr 2026 01:34:38 +0100 From: Qais Yousef To: "Chen, Yu C" Cc: Tim Chen , Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Libo Chen , linux-kernel@vger.kernel.org Subject: Re: [Patch v4 00/22] Cache aware scheduling Message-ID: <20260421003438.whnn2gvv4gkfcmx5@airbuntu> References: <20260416002749.muyrcycmtabksav4@airbuntu> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: On 04/20/26 17:01, Chen, Yu C wrote: > On 4/16/2026 8:27 AM, Qais Yousef wrote: > > On 04/01/26 14:52, Tim Chen wrote: > > [ ... ] > > > > > I posted schedqos announcement yesterday, which I think (hope) would be the > > right way to address these concerns about tagging tasks. > > > > https://lore.kernel.org/lkml/20260415000910.2h5misvwc45bdumu@airbuntu/ > > > > Thanks, I'll take a look at this. > > > It would be trivial to add experimental branch to add new QoS flavour to say > > NUMA_SENSITIVE etc. I am still trying to think of a generic description to > > address a number of use cases (see Execution Profiles in README.md), not just > > this particular numa sensitive one, but the experimental branch should help > > iterate and drive the kernel development for wake up path + push lb instead of > > using load balance which I really doubt will work well in practice since this > > is slow to react, and you're relying on overcommitting the system by default by > > making every task of every process data dependent and require it to be > > co-located. > > I am not certain which strategy is preferable, as it largely depends > on the use case and workload. We intend to evaluate push-based load > balancing on top of the existing lb-based cache-aware placement logic. I'll defer to Vincent here, but I would have thought lb-based approach can go away. > > > I think in practice admins will care about specific applications to > > be kept within a single LLC, and if they are willing to spend the effort, they > > can tag specific tasks of a specific application. > > > > It seems to me that there are multiple use cases. In one scenario, > the administrator (including daemons) is responsible for tagging > workloads. In another, users prefer the OS to handle automatic > placement without any userspace involvement. How do you define this automatic placement? AFAICS you're just grouping all tasks of a specific process to stay within the same LLC and hitting overcommit issues which you're workingaround with this load balancer only based approach? I think in practice there will be many corner cases where state is not optimal and we'd end up with heuristics to 'balance' things out and sensitivity to independent changes disturbing this fragile balance causing weird regressions and us slowly has less flexibility to move and shuffle code (okay, maybe too much doom and gloom, but we've been by this in the past :)). I am not sure how many of these tests stressed the system with multiple critical processes running concurrently? By making it a userspace problem they have to figure out the right balance and we can focus on providing the right mechanism. > > > Also QoS IMHO should be viewed as a scarce resource. For best effort delivery > > (which is the best we can do in reality, this is not hard real time system), it > > is easier to provide good best effort when the average noise level is low, ie: > > few tasks are required to be kept within the same LLC. If we overcommit often, > > we will crumble often. So IMHO the key is to delegate to userspace to tag, and > > I suppose there are two scenarios. The first is enabling/disabling > aggregation > for a group of tasks, and the second is task tagging. For the first > scenario, > this can be applied either process-wide or cgroup-wide by providing a flag, Cgroup-wide tagging doesn't make sense IMO. Process-wide yes. What does it mean to group all processes in the same cgroup from cache locality PoV? It just seems random setup based on something specific in userspace on how these cgroups are setup that assumes one process per group? I don't think we can generalize if that's the case. Admins can use cpuset to statically partition based on cgroup if they want to ensure a group of processes are confined to the same LLC? > without requiring users to explicitly tag individual tasks. The second > scenario > is an enhancement to support fine-grained control over a specific task. If > schedqos only supports scenario2, the user has to tag every task to support > scenario1. You can do tagging process-wide (as I mentioned in the announcement, I think it's a poor man's way for quick tagging until people learn to do better), not just per-task eg: { "PostgreSQL": { "qos": ["QOS_USER_INTERACTIVE", "QOS_NUMA_SENSITIVE"] } } which will tag every task forked by this binary with a cookie and QoS data dependency tag tell the scheduler all tasks with the same cookie need to stay within the same LLC. We can bikeshed naming and the tagging details, but the actual implementation principal should be the same: keep tasks with the same cookie and data dep tag on the same LLC at wake up; and let push lb handle occasional strayed task. If users want to be smart and tag specific tasks only, the implementation would be identical, it's just there are fewer tasks tagged.