From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6158C3148A7 for ; Thu, 15 Jan 2026 21:47:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.17 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768513641; cv=none; b=iCGf7QkXIA8pc3fUQkbdWFnckPXhR71Gq1YCaFiNn5BMDnVFCbdw+sN+lnc0BSC7Yebrim+GOiBEP2uN5GH5HIKYGl0/DlhMjfiRor2IEv1JrpcTsp0kMFWRDHzIAelEybA7q8WRnIOlg0Bpd4DcHb9NwMd4qS8aXwAC1foxxEg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768513641; c=relaxed/simple; bh=tkrvprG0fYUBlqEAKpWpwOSgFKIFEg+Ru9Pe0JGdwWE=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=Ne0MW5G42z/oqOn21y7vh9eu8trnU8/Fh847nehahbRp+0iRViGBRwCzpNrDpNpdds8vT1EEQ08h0bIF+b5DqTo6r0FpOiXI6Jo3HmKI0qqYuqWffMpJ4O2x6YdWaN8iT0UOdL1xN4ptxq1U0Z4fS4CXgV5JVXSIvFUQN90WBQY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HRyNMeFR; arc=none smtp.client-ip=192.198.163.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HRyNMeFR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1768513639; x=1800049639; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=tkrvprG0fYUBlqEAKpWpwOSgFKIFEg+Ru9Pe0JGdwWE=; b=HRyNMeFRCjblRDD7k6Mg8uj2aGcdkHNednOPk+d+uDcLqFncRpYa3nsx CHOb+f2EOVmsms395fZBapE8VoafBgEqwGXE/KYwdanmKNLYCJFCNgSFj 2HTAR2TJOAq+0SEadY3itsfpiWsgEHCvjCvut0JUD/r7hcjcSe6xPR+f5 k+D8wSnxDWyj82Hw+IeO2rptU09NprmKm03NNd/CeDFmHoKkRO32ZUcJ+ 1oOiWTMTKQv2r9YziZPUOCGwOqzBJv3aAFtdUz5LbnY/I8cgzJ6zei6zT wpG52WkxB9TqHugmLSqXxuAaw25Wr+sjignQ+5D2D7E4fDtRGZMMI/k3/ g==; X-CSE-ConnectionGUID: 9usPfenLTWqq1JCMH5ruFA== X-CSE-MsgGUID: cl7i6WD0RWisaAn2DYAPBw== X-IronPort-AV: E=McAfee;i="6800,10657,11672"; a="69734697" X-IronPort-AV: E=Sophos;i="6.21,229,1763452800"; d="scan'208";a="69734697" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jan 2026 13:47:18 -0800 X-CSE-ConnectionGUID: cWyrCxC8RfS4gA4EjNB+9A== X-CSE-MsgGUID: 2XO4gBxWTzyQDyfAzVuahw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,229,1763452800"; d="scan'208";a="209927753" Received: from unknown (HELO [10.241.243.18]) ([10.241.243.18]) by fmviesa004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jan 2026 13:47:18 -0800 Message-ID: Subject: Re: [PATCH v2 01/23] sched/cache: Introduce infrastructure for cache-aware load balancing From: Tim Chen To: Vern Hao , "Chen, Yu C" Cc: Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org, Peter Zijlstra , Ingo Molnar , K Prateek Nayak , Vincent Guittot , "Gautham R . Shenoy" Date: Thu, 15 Jan 2026 13:47:06 -0800 In-Reply-To: <7d5bb7c4-abc5-470e-84fe-72a3b1d3a2f4@gmail.com> References: <06f0d7edbc3185ec730b50b3b00d87ace44169b3.1764801860.git.tim.c.chen@linux.intel.com> <7e4640a2-f79f-4f14-b099-d97bfd842b37@intel.com> <7d5bb7c4-abc5-470e-84fe-72a3b1d3a2f4@gmail.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Wed, 2025-12-17 at 09:17 +0800, Vern Hao wrote: > On 2025/12/16 14:12, Chen, Yu C wrote: > > On 12/11/2025 5:03 PM, Vern Hao wrote: > > > Hi, Peter, Chen Yu and Tim: > > >=20 > > > On 2025/12/4 07:07, Tim Chen wrote: > > > > From: "Peter Zijlstra (Intel)" > > > >=20 > > > > Adds infrastructure to enable cache-aware load balancing, > > > > which improves cache locality by grouping tasks that share resource= s > > > > within the same cache domain. This reduces cache misses and improve= s > > > > overall data access efficiency. > > > >=20 > > > > In this initial implementation, threads belonging to the same proce= ss > > > > are treated as entities that likely share working sets. The mechani= sm > > > > tracks per-process CPU occupancy across cache domains and attempts = to > > > > migrate threads toward cache-hot domains where their process alread= y > > > > has active threads, thereby enhancing locality. > > > >=20 > > > > This provides a basic model for cache affinity. While the current c= ode > > > > targets the last-level cache (LLC), the approach could be extended = to > > > > other domain types such as clusters (L2) or node-internal groupings= . > > > >=20 > > > > At present, the mechanism selects the CPU within an LLC that has th= e > > > > highest recent runtime. Subsequent patches in this series will use = this > > > > information in the load-balancing path to guide task placement towa= rd > > > > preferred LLCs. > > > >=20 > > > > In the future, more advanced policies could be integrated through N= UMA > > > > balancing-for example, migrating a task to its preferred LLC when s= pare > > > > capacity exists, or swapping tasks across LLCs to improve cache=20 > > > > affinity. > > > > Grouping of tasks could also be generalized from that of a process > > > > to be that of a NUMA group, or be user configurable. > > > >=20 > > > > Originally-by: Peter Zijlstra (Intel) > > > > Signed-off-by: Chen Yu > > > > Signed-off-by: Tim Chen > > > > --- > > > >=20 > > > > Notes: > > > > =C2=A0=C2=A0=C2=A0=C2=A0 v1->v2: > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Restore the original CPU= scan to cover all online CPUs, > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 rather than scanning wit= hin the preferred NUMA node. > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (Peter Zijlstra) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Use rq->curr instead of = rq->donor. (K Prateek Nayak) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Minor fix in task_tick_c= ache() to use > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (mm->mm_sched_epoch >= =3D rq->cpu_epoch) > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 to avoid mm_sched_epoch = going backwards. > > > >=20 > > > > =C2=A0 include/linux/mm_types.h |=C2=A0 44 +++++++ > > > > =C2=A0 include/linux/sched.h=C2=A0=C2=A0=C2=A0 |=C2=A0 11 ++ > > > > =C2=A0 init/Kconfig=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 11 ++ > > > > =C2=A0 kernel/fork.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 6 + > > > > =C2=A0 kernel/sched/core.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2= =A0 6 + > > > > =C2=A0 kernel/sched/fair.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 258=20 > > > > +++++++++++++++++++++++++++++++++++++++ > > > > =C2=A0 kernel/sched/sched.h=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0=C2=A0 8= ++ > > > > =C2=A0 7 files changed, 344 insertions(+) > > > >=20 > > > > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > > > > index 90e5790c318f..1ea16ef90566 100644 > > > > --- a/include/linux/mm_types.h > > > > +++ b/include/linux/mm_types.h > > > > @@ -939,6 +939,11 @@ typedef struct { > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 DECLARE_BITMAP(__mm_flags, NUM_MM_FL= AG_BITS); > > > > =C2=A0 } __private mm_flags_t; > > > > +struct mm_sched { > > > > +=C2=A0=C2=A0=C2=A0 u64 runtime; > > > > +=C2=A0=C2=A0=C2=A0 unsigned long epoch; > > > > +}; > > > > + > > > > =C2=A0 struct kioctx_table; > > > > =C2=A0 struct iommu_mm_data; > > > > =C2=A0 struct mm_struct { > > > > @@ -1029,6 +1034,17 @@ struct mm_struct { > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */ > > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 raw_spinlock= _t cpus_allowed_lock; > > > > =C2=A0 #endif > > > > +#ifdef CONFIG_SCHED_CACHE > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * Track per-cpu-p= er-process occupancy as a proxy for cache=20 > > > > residency. > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * See account_mm_= sched() and ... > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */ > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct mm_sched __percp= u *pcpu_sched; > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 raw_spinlock_t mm_sched= _lock; > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long mm_sched_= epoch; > > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int mm_sched_cpu; > > > As we discussed earlier=EF=BC=8CI continue to believe that dedicating= =20 > > > 'mm_sched_cpu' to handle the aggregated hotspots of all threads is= =20 > > > inappropriate, as the multiple threads lack a necessary correlation= =20 > > > in our real application. > > >=20 > > > So, I was wondering if we could put this variable into struct=20 > > > task_struct, That allows us to better monitor the hotspot CPU of each= =20 > > > thread, despite some details needing consideration. > > >=20 > >=20 > > I suppose you are suggesting a fine-grained control for a set of tasks. > > Process-scope aggregation could be a start as the default strategy( > > conservative, benefit multi-thread workloads that share data per proces= s, > > not introduce regression). >=20 > Yes, in our real-world business scenarios at Tencent, I have indeed=20 > encountered this issue where multiple threads are divided into several= =20 > categories to handle different transactions, so they are not share the= =20 > hot data, the 'mm_sched_cpu'=C2=A0 does not represent all of their task, = so=20 > add a control interface such as cgroup or others will be a good idea. >=20 Yes, the grouping and aggregating of tasks by process will not cover your usage scenario. Chen Yu and I had quite a bit of discussions among us and here're our thoughts. In the initial version of cache aware scheduling, process based aggregation is a sensible default. Once this basic option is merged in mainline we will= consider adding other options for task grouping. For example, setting a flag in a cgroup cpu controller to indicate that tasks in a cgroup could benefit from being consolidated in a LLC. We think that you can put your threads in each category in each of its own cgroup. Will that meet your need? Things like mm_sched_cpu ... etc will be abstracted out, where the grouping= structure in mm is abstracted as cache_group. So we will have something like cache_group_sched_cpu instead of mm_sched_cpu. Tim > >=20 > > On top of that, I wonder if we could provide task-scope control like > > sched_setattr(), similar to core-scheduling cookie mechanism, for > > users that want aggressive aggregation. But before doing that, we need = a > > mechanism that that leverages a monitor system(like PMU) to figure out > There will maybe a trouble, If the environment is running on a VM, We=20 > could use tags to differentiate these tasks and do some tests to verify= =20 > the performance difference between unifying the |mm_sched_cpu| and not= =20 > unifying. > > if putting these tasks together would bring benefit(if I understand > > Steven's suggestion correctly on LPC), or detection tasks that share > > resource, then maybe leverage QOS interfaces to enable the cache-aware > > aggregation(something Qias mentioned on the LPC). > >=20 > > thanks, > > Chenyu > >=20