From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54AC3246762 for ; Sat, 14 Feb 2026 16:14:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771085673; cv=none; b=d+RYJQyfQFXNyBHl906kgfpbHb+HSdfj/2W3a3c97ARwSXQNc+ND2YQJ1wD/QxdpT+wHryt1AGJg0/f4+2NkUkpaQa5IVcXVrvg6mmwLlNd6yOjaUNOKuvQD1//3B39z6yxNCi82aO+cjJmr/XRrMXXISHqSFYa/3t7r9Igj4nQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771085673; c=relaxed/simple; bh=HRoI2hx1cM8HeTY8AhEBxXD9alZtVpp4EP1XYL4dd4s=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=b5R0DyfXSKHKFgg7B5lb4yeGksIEV0DE14M0UHNWTYr6w7P18fQ9vSj81py8iyniDfdPwJqmLSx1gj1QqneOZCoxRXs4jWJ2zr617hotvNCoS8yuabftewsG/qOCWQvB3xdyARr8EII3y8DxYMH6ND0uhiVBnksFoG/6TQ77Dro= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=AK5ebVrA; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="AK5ebVrA" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61ECJ66I133735; Sat, 14 Feb 2026 16:13:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=qCP9aD 7AgtFX5moEYFiijcIX+iQH0+jvSrQ3npIZFTQ=; b=AK5ebVrAykltbE2+pAbqNW PJZOk/z2PGLHWh1H+O6RdUcJkUNSXmcfBrsitJ1+CbmTUZ7ReE9lpT/Dvys+mLfb 0jhJ9UbG7kG3jCJGaZzzGfdbs4Y1XUqZJitZqsExJ46OgJnB+7eIxapG+z7ZxLkS +DQgdU7EvRFKgmnbJoh3MZC+b49Z0437vFpJYLzrCILRjVQdz/utjhFHmKPN4qvi +MqYdE32MLlzCaTqhLSLR/q7w9cZ8tIESAIHcAV7URhgF4zZ4OVqjVJM2IXLoCcA GkxFnKElmceh2DsWlD8J/TKElYZiV0sLrcZGazsT9B5vTT6V2zYvtk+zLp6+kzlw == Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cajcqh7up-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 14 Feb 2026 16:13:09 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 61EG3w05002570; Sat, 14 Feb 2026 16:13:08 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4c6fqt39ye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 14 Feb 2026 16:13:08 +0000 Received: from smtpav05.wdc07v.mail.ibm.com (smtpav05.wdc07v.mail.ibm.com [10.39.53.232]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 61EGD7LT33358366 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 14 Feb 2026 16:13:07 GMT Received: from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 095D658053; Sat, 14 Feb 2026 16:13:07 +0000 (GMT) Received: from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D883B58043; Sat, 14 Feb 2026 16:12:49 +0000 (GMT) Received: from [9.67.107.13] (unknown [9.67.107.13]) by smtpav05.wdc07v.mail.ibm.com (Postfix) with ESMTP; Sat, 14 Feb 2026 16:12:49 +0000 (GMT) Message-ID: Date: Sat, 14 Feb 2026 21:42:46 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 03/21] sched/cache: Introduce helper functions to enforce LLC migration policy To: Tim Chen Cc: Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Chen Yu , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Qais Yousef , Libo Chen , linux-kernel@vger.kernel.org, Madadi Vineeth Reddy References: <7475922f6020abe5d458a136b0c88fe24e823091.1770760558.git.tim.c.chen@linux.intel.com> Content-Language: en-US From: Madadi Vineeth Reddy In-Reply-To: <7475922f6020abe5d458a136b0c88fe24e823091.1770760558.git.tim.c.chen@linux.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-GUID: u9VVWb1oULpu9RyoR58pxK-aSsbiDyvW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjE0MDEzMyBTYWx0ZWRfX8SF4QPoEKWxr EM8PDZ5rarVsTElT1Y0AzDSaC2/wIJw2BRp39i9x0teCmYosfD/u/0lHkTmHqjh9CIZQ7D+zF0t U+K8Zp1Ia32fM1UJektmtrGZG0IVikadrR39sxjXAfdUUN5vP59vu7m8YHlfodSfoTJOsaG0ekO ck9HYA2LlVXi26L/pjjC+oacWxO6jRy35iEdx6o0dwYPoVZLwJLuCpYxUgulUhtC9H7L4yWZSAT ef+fdcK8A0p7qzEL+/P668ttPIhLBLTe7xMcyiXSQHvh/McV00BzW/wfh+gcaVm2hgRfFRJvmKP HIloYOC5two1+fdlhYBQ7TOvT0XRmpzEN0jTihN1la+sEvPKgUnsFnJuJL0X93B5XMpZGUFPEKY TfxgZot9LPBsmWI6Rc5pDneizsHjHpPjiNzyPuG4DLsF6sITRdx2kHbtNtOPJ+7YlXgZXA/j0mO H6aE00Rii/me6kpvAdA== X-Authority-Analysis: v=2.4 cv=UPXQ3Sfy c=1 sm=1 tr=0 ts=69909f16 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=IkcTkHD0fZMA:10 a=HzLeVaNsDn8A:10 a=VkNPw1HP01LnGYTKEx00:22 a=Mpw57Om8IfrbqaoTuvik:22 a=GgsMoib0sEa3-_RKJdDe:22 a=QyXUC8HyAAAA:8 a=GfmkCuvP4TI_01APN-gA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: xNOgwHU6CMpaWn0TQfnk7XMC7pXXBnO6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-02-14_02,2026-02-13_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 phishscore=0 impostorscore=0 adultscore=0 bulkscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2602140133 On 11/02/26 03:48, Tim Chen wrote: > From: Chen Yu > > Cache-aware scheduling aggregates threads onto their preferred LLC, > mainly through load balancing. When the preferred LLC becomes > saturated, more threads are still placed there, increasing latency. > A mechanism is needed to limit aggregation so that the preferred LLC > does not become overloaded. > > Introduce helper functions can_migrate_llc() and > can_migrate_llc_task() to enforce the LLC migration policy: > > 1. Aggregate a task to its preferred LLC if both source and > destination LLCs are not too busy, or if doing so will not > leave the preferred LLC much more imbalanced than the > non-preferred one (>20% utilization difference, a little > higher than imbalance_pct(17%) of the LLC domain as hysteresis). > 2. Allow moving a task from overloaded preferred LLC to a non > preferred LLC if this will not cause the non preferred LLC > to become too imbalanced to cause a later migration back. > 3. If both LLCs are too busy, let the generic load balance to > spread the tasks. > > Further (hysteresis)action could be taken in the future to prevent tasks > from being migrated into and out of the preferred LLC frequently (back and > forth): the threshold for migrating a task out of its preferred LLC should > be higher than that for migrating it into the LLC. > > Co-developed-by: Tim Chen > Signed-off-by: Tim Chen > Signed-off-by: Chen Yu > --- > > Notes: > v2->v3: > No change. > > kernel/sched/fair.c | 153 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 153 insertions(+) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index dfeb107f2cfd..bf5f39a01017 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -9689,6 +9689,27 @@ static inline int task_is_ineligible_on_dst_cpu(struct task_struct *p, int dest_ > } > > #ifdef CONFIG_SCHED_CACHE > +/* > + * The margin used when comparing LLC utilization with CPU capacity. > + * It determines the LLC load level where active LLC aggregation is > + * done. > + * Derived from fits_capacity(). > + * > + * (default: ~50%) > + */ > +#define fits_llc_capacity(util, max) \ > + ((util) * 2 < (max)) > + > +/* > + * The margin used when comparing utilization. > + * is 'util1' noticeably greater than 'util2' > + * Derived from capacity_greater(). > + * Bias is in perentage. > + */ > +/* Allows dst util to be bigger than src util by up to bias percent */ > +#define util_greater(util1, util2) \ > + ((util1) * 100 > (util2) * 120) > + > /* Called from load balancing paths with rcu_read_lock held */ > static __maybe_unused bool get_llc_stats(int cpu, unsigned long *util, > unsigned long *cap) > @@ -9704,6 +9725,138 @@ static __maybe_unused bool get_llc_stats(int cpu, unsigned long *util, > > return true; > } > + > +/* > + * Decision matrix according to the LLC utilization. To > + * decide whether we can do task aggregation across LLC. > + * > + * By default, 50% is the threshold for treating the LLC > + * as busy. The reason for choosing 50% is to avoid saturation > + * of SMT-2, and it is also a safe cutoff for other SMT-n > + * platforms. > + * > + * 20% is the utilization imbalance percentage to decide > + * if the preferred LLC is busier than the non-preferred LLC. > + * 20 is a little higher than the LLC domain's imbalance_pct > + * 17. The hysteresis is used to avoid task bouncing between the > + * preferred LLC and the non-preferred LLC. > + * > + * 1. moving towards the preferred LLC, dst is the preferred > + * LLC, src is not. > + * > + * src \ dst 30% 40% 50% 60% > + * 30% Y Y Y N > + * 40% Y Y Y Y > + * 50% Y Y G G > + * 60% Y Y G G > + * According to this matrix (which I assume shows utilization after migration), G is expected for src=50% and dst=50%. However, the code performs the "both busy" check before adjusting src_util and dst_util: if (!fits_llc_capacity(dst_util, dst_cap) && !fits_llc_capacity(src_util, src_cap)) return mig_unrestricted; src_util = src_util - tsk_util; dst_util = dst_util + tsk_util; For example, with a 10% task migrating from src_util=60% to dst_util=40%: The check evaluates: !fits(40) && !fits(60) = false && true = false - Doesn't return mig_unrestricted - After adjustment: src=50%, dst=50% - Falls through to return mig_llc (Y) But the matrix indicates 50%/50% should be G, not Y. Moving this check after the utilization adjustment would make it consistent with the documented matrix. Thanks, Vineeth > + * 2. moving out of the preferred LLC, src is the preferred > + * LLC, dst is not: > + * > + * src \ dst 30% 40% 50% 60% > + * 30% N N N N > + * 40% N N N N > + * 50% N N G G > + * 60% Y N G G > + * > + * src : src_util > + * dst : dst_util > + * Y : Yes, migrate > + * N : No, do not migrate > + * G : let the Generic load balance to even the load. > + * > + * The intention is that if both LLCs are quite busy, cache aware > + * load balance should not be performed, and generic load balance > + * should take effect. However, if one is busy and the other is not, > + * the preferred LLC capacity(50%) and imbalance criteria(20%) should > + * be considered to determine whether LLC aggregation should be > + * performed to bias the load towards the preferred LLC. > + */ > + > +/* migration decision, 3 states are orthogonal. */ > +enum llc_mig { > + mig_forbid = 0, /* N: Don't migrate task, respect LLC preference */ > + mig_llc, /* Y: Do LLC preference based migration */ > + mig_unrestricted /* G: Don't restrict generic load balance migration */ > +}; > + > +/* > + * Check if task can be moved from the source LLC to the > + * destination LLC without breaking cache aware preferrence. > + * src_cpu and dst_cpu are arbitrary CPUs within the source > + * and destination LLCs, respectively. > + */ > +static enum llc_mig can_migrate_llc(int src_cpu, int dst_cpu, > + unsigned long tsk_util, > + bool to_pref) > +{ > + unsigned long src_util, dst_util, src_cap, dst_cap; > + > + if (!get_llc_stats(src_cpu, &src_util, &src_cap) || > + !get_llc_stats(dst_cpu, &dst_util, &dst_cap)) > + return mig_unrestricted; > + > + if (!fits_llc_capacity(dst_util, dst_cap) && > + !fits_llc_capacity(src_util, src_cap)) > + return mig_unrestricted; > + > + src_util = src_util < tsk_util ? 0 : src_util - tsk_util; > + dst_util = dst_util + tsk_util; > + if (to_pref) { > + /* > + * Don't migrate if we will get preferred LLC too > + * heavily loaded and if the dest is much busier > + * than the src, in which case migration will > + * increase the imbalance too much. > + */ > + if (!fits_llc_capacity(dst_util, dst_cap) && > + util_greater(dst_util, src_util)) > + return mig_forbid; > + } else { > + /* > + * Don't migrate if we will leave preferred LLC > + * too idle, or if this migration leads to the > + * non-preferred LLC falls within sysctl_aggr_imb percent > + * of preferred LLC, leading to migration again > + * back to preferred LLC. > + */ > + if (fits_llc_capacity(src_util, src_cap) || > + !util_greater(src_util, dst_util)) > + return mig_forbid; > + } > + return mig_llc; > +} > + > +/* > + * Check if task p can migrate from source LLC to > + * destination LLC in terms of cache aware load balance. > + */ > +static __maybe_unused enum llc_mig can_migrate_llc_task(int src_cpu, int dst_cpu, > + struct task_struct *p) > +{ > + struct mm_struct *mm; > + bool to_pref; > + int cpu; > + > + mm = p->mm; > + if (!mm) > + return mig_unrestricted; > + > + cpu = mm->sc_stat.cpu; > + if (cpu < 0 || cpus_share_cache(src_cpu, dst_cpu)) > + return mig_unrestricted; > + > + if (cpus_share_cache(dst_cpu, cpu)) > + to_pref = true; > + else if (cpus_share_cache(src_cpu, cpu)) > + to_pref = false; > + else > + return mig_unrestricted; > + > + return can_migrate_llc(src_cpu, dst_cpu, > + task_util(p), to_pref); > +} > + > #else > static inline bool get_llc_stats(int cpu, unsigned long *util, > unsigned long *cap)