From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93A4339572D for ; Thu, 22 Jan 2026 20:43:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769114626; cv=none; b=CR3wBmudo0JW2cf9LRYCBkfu0fkLMttoyY6rIN/P6MZCWfCA/SspiNn/D//E6LdxudYUwiK64ta3CqFHEF/16vWzGKvI/d0DNoAzAU8m+7CInVbgwWqmT/gOcply5sBISIelN52Q4ovZfZ097aBOWZGC8wYbBx5E/8XwlbDVuyc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769114626; c=relaxed/simple; bh=iebWDPffnQ9fDz3jrXZla7CVG4GRimEoyujZwuC6gGU=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=St4wEpxcRBymS/liQzvPW66UNA0KgGuukI/Rmm0my+loEkk5Od0xGLe/9wEzDqA0O1bKaO+7vobz5RcDIfUNtVH/a/Q1T6kB4HaVo1Dbr85oq1swD/+lLM8O1XWxtBSqTX4/BKhK795b+x0IMapvESAqCypkvNOtiTLc/xFQvGY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=F2jqG81S; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F2jqG81S" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1769114618; x=1800650618; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=iebWDPffnQ9fDz3jrXZla7CVG4GRimEoyujZwuC6gGU=; b=F2jqG81S5i3p6M11UcVDgIBv5XkWxwZn510dY7+w8yGsJojEPbRxDhK8 8GLABNQxDlfZXOHMuH9SvMCsLO1S77Fl5dYHHkrqVpqkw9LOt3N2HuEHM MkDEAFLJqf4/u4KwOR8LIrT6y1T6JLWspAHo2S1sqGQUE92jyZplU08E+ r/Qje43/RZGYAnaGFS55uQnyJZ0Su9171CJKxZuQVTXbbqeAiYc2JiXve o2WSsbvmaMgT0pdgQqItNzI6nqhUIn+2RCPi0UYhV51rdqbYYJecdhJSo SJga7UaMTKgTrT8dFUydAt3zVtTKm6JyYOtFIeB97IJX5bzd5aQr5pWXZ Q==; X-CSE-ConnectionGUID: anVJt4RcRMKyLRJWGBmekA== X-CSE-MsgGUID: T/xYjm02QKSMsLzbbfJwUA== X-IronPort-AV: E=McAfee;i="6800,10657,11679"; a="87783455" X-IronPort-AV: E=Sophos;i="6.21,246,1763452800"; d="scan'208";a="87783455" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2026 12:43:33 -0800 X-CSE-ConnectionGUID: j2Cp0TXBSX+WtGB6htuBJA== X-CSE-MsgGUID: wOa8QVBAQj2Breh/jxMqcQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,246,1763452800"; d="scan'208";a="206740603" Received: from tfalcon-desk.amr.corp.intel.com (HELO [10.125.108.184]) ([10.125.108.184]) by fmviesa006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2026 12:43:32 -0800 Message-ID: Subject: Re: [PATCH v2 03/23] sched/cache: Introduce helper functions to enforce LLC migration policy From: Tim Chen To: Yangyu Chen , Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot Cc: Chen Yu , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org Date: Thu, 22 Jan 2026 12:43:31 -0800 In-Reply-To: References: <12e90c8c26c690b40e48cc1e03c785f2f99fafa8.1764801860.git.tim.c.chen@linux.intel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Fri, 2026-01-23 at 02:13 +0800, Yangyu Chen wrote: >=20 > On 4/12/2025 07:07, Tim Chen wrote: > > From: Chen Yu > >=20 > > Cache-aware scheduling aggregates threads onto their preferred LLC, > > mainly through load balancing. When the preferred LLC becomes > > saturated, more threads are still placed there, increasing latency. > > A mechanism is needed to limit aggregation so that the preferred LLC > > does not become overloaded. > >=20 > > Introduce helper functions can_migrate_llc() and > > can_migrate_llc_task() to enforce the LLC migration policy: > >=20 > > 1. Aggregate a task to its preferred LLC if both source and > > destination LLCs are not too busy (<50% utilization), >=20 > Hi Chen Yu and Tim Chen, >=20 > I would like to ask why here chooses 50% for LLC busy. For example, a=20 > common AMD Zen 3-5 has 8 cores per LLC. When these servers turned SMT=20 > off, an 8-thread process can not be scheduled within 1 LLC. I think here= =20 > can be 100% by default. For 100% you would be likely have over aggregation and contention on the LL= C by aggregating everything there. We tested some workload that has hot working set footprint that's a = significant fraction of the LLC and found that 100% would cause significant degradation to performance. 50% is what we found to be a reasonable=C2=A0 default value. This is a tunnable so it could be changed by admin if desired. Tim =20 >=20 > Thanks, > Yangyu Chen >=20 > > or if doing so will not leave the preferred LLC much more > > imbalanced than the non-preferred one (>20% utilization > > difference, similar to imbalance_pct of the LLC domain). > > 2. Allow moving a task from overloaded preferred LLC to a non prefer= red > > LLC if this will not cause the non preferred LLC to become > > too imbalanced to cause a later migration back. > > 3. If both LLCs are too busy, let the generic load balance to spread > > the tasks. > >=20 > > Further (hysteresis)action could be taken in the future to prevent task= s > > from being migrated into and out of the preferred LLC frequently (back = and > > forth): the threshold for migrating a task out of its preferred LLC sho= uld > > be higher than that for migrating it into the LLC. > >=20 > > Since aggregation tends to make the preferred LLC busier than others, > > the imbalance tolerance is controlled by llc_imb_pct. If set to 0, > > tasks may still aggregate to the preferred LLC as long as it is > > not more utilized than the source LLC, preserving the preference. > >=20 > > Co-developed-by: Tim Chen > > Signed-off-by: Chen Yu > > Signed-off-by: Tim Chen > > --- > >=20 > > Notes: > > v1->v2: > > No change. > >=20 > > kernel/sched/fair.c | 153 ++++++++++++++++++++++++++++++++++++++++++= + > > kernel/sched/sched.h | 5 ++ > > 2 files changed, 158 insertions(+) > >=20 > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index b9f336300f14..710ed9943d27 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -1205,6 +1205,9 @@ static s64 update_se(struct rq *rq, struct sched_= entity *se) > > #define EPOCH_PERIOD (HZ / 100) /* 10 ms */ > > #define EPOCH_LLC_AFFINITY_TIMEOUT 5 /* 50 ms */ > > =20 > > +__read_mostly unsigned int llc_overload_pct =3D 50; > > +__read_mostly unsigned int llc_imb_pct =3D 20; > > + > > static int llc_id(int cpu) > > { > > if (cpu < 0) > > @@ -9623,6 +9626,27 @@ static inline int task_is_ineligible_on_dst_cpu(= struct task_struct *p, int dest_ > > } > > =20 > > #ifdef CONFIG_SCHED_CACHE > > +/* > > + * The margin used when comparing LLC utilization with CPU capacity. > > + * Parameter llc_overload_pct determines the LLC load level where > > + * active LLC aggregation is done. > > + * Derived from fits_capacity(). > > + * > > + * (default: ~50%) > > + */ > > +#define fits_llc_capacity(util, max) \ > > + ((util) * 100 < (max) * llc_overload_pct) > > + > > +/* > > + * The margin used when comparing utilization. > > + * is 'util1' noticeably greater than 'util2' > > + * Derived from capacity_greater(). > > + * Bias is in perentage. > > + */ > > +/* Allows dst util to be bigger than src util by up to bias percent */ > > +#define util_greater(util1, util2) \ > > + ((util1) * 100 > (util2) * (100 + llc_imb_pct)) > > + > > /* Called from load balancing paths with rcu_read_lock held */ > > static __maybe_unused bool get_llc_stats(int cpu, unsigned long *util= , > > unsigned long *cap) > > @@ -9638,6 +9662,135 @@ static __maybe_unused bool get_llc_stats(int cp= u, unsigned long *util, > > =20 > > return true; > > } > > + > > +/* > > + * Decision matrix according to the LLC utilization. To > > + * decide whether we can do task aggregation across LLC. > > + * > > + * By default, 50% is the threshold to treat the LLC as busy, > > + * and 20% is the utilization imbalance percentage to decide > > + * if the preferred LLC is busier than the non-preferred LLC. > > + * The hysteresis is used to avoid task bouncing between the > > + * preferred LLC and the non-preferred LLC. > > + * > > + * 1. moving towards the preferred LLC, dst is the preferred > > + * LLC, src is not. > > + * > > + * src \ dst 30% 40% 50% 60% > > + * 30% Y Y Y N > > + * 40% Y Y Y Y > > + * 50% Y Y G G > > + * 60% Y Y G G > > + * > > + * 2. moving out of the preferred LLC, src is the preferred > > + * LLC, dst is not: > > + * > > + * src \ dst 30% 40% 50% 60% > > + * 30% N N N N > > + * 40% N N N N > > + * 50% N N G G > > + * 60% Y N G G > > + * > > + * src : src_util > > + * dst : dst_util > > + * Y : Yes, migrate > > + * N : No, do not migrate > > + * G : let the Generic load balance to even the load. > > + * > > + * The intention is that if both LLCs are quite busy, cache aware > > + * load balance should not be performed, and generic load balance > > + * should take effect. However, if one is busy and the other is not, > > + * the preferred LLC capacity(50%) and imbalance criteria(20%) should > > + * be considered to determine whether LLC aggregation should be > > + * performed to bias the load towards the preferred LLC. > > + */ > > + > > +/* migration decision, 3 states are orthogonal. */ > > +enum llc_mig { > > + mig_forbid =3D 0, /* N: Don't migrate task, respect LLC preference *= / > > + mig_llc, /* Y: Do LLC preference based migration */ > > + mig_unrestricted /* G: Don't restrict generic load balance migration = */ > > +}; > > + > > +/* > > + * Check if task can be moved from the source LLC to the > > + * destination LLC without breaking cache aware preferrence. > > + * src_cpu and dst_cpu are arbitrary CPUs within the source > > + * and destination LLCs, respectively. > > + */ > > +static enum llc_mig can_migrate_llc(int src_cpu, int dst_cpu, > > + unsigned long tsk_util, > > + bool to_pref) > > +{ > > + unsigned long src_util, dst_util, src_cap, dst_cap; > > + > > + if (!get_llc_stats(src_cpu, &src_util, &src_cap) || > > + !get_llc_stats(dst_cpu, &dst_util, &dst_cap)) > > + return mig_unrestricted; > > + > > + if (!fits_llc_capacity(dst_util, dst_cap) && > > + !fits_llc_capacity(src_util, src_cap)) > > + return mig_unrestricted; > > + > > + src_util =3D src_util < tsk_util ? 0 : src_util - tsk_util; > > + dst_util =3D dst_util + tsk_util; > > + if (to_pref) { > > + /* > > + * llc_imb_pct is the imbalance allowed between > > + * preferred LLC and non-preferred LLC. > > + * Don't migrate if we will get preferred LLC too > > + * heavily loaded and if the dest is much busier > > + * than the src, in which case migration will > > + * increase the imbalance too much. > > + */ > > + if (!fits_llc_capacity(dst_util, dst_cap) && > > + util_greater(dst_util, src_util)) > > + return mig_forbid; > > + } else { > > + /* > > + * Don't migrate if we will leave preferred LLC > > + * too idle, or if this migration leads to the > > + * non-preferred LLC falls within sysctl_aggr_imb percent > > + * of preferred LLC, leading to migration again > > + * back to preferred LLC. > > + */ > > + if (fits_llc_capacity(src_util, src_cap) || > > + !util_greater(src_util, dst_util)) > > + return mig_forbid; > > + } > > + return mig_llc; > > +} > > + > > +/* > > + * Check if task p can migrate from source LLC to > > + * destination LLC in terms of cache aware load balance. > > + */ > > +static __maybe_unused enum llc_mig can_migrate_llc_task(int src_cpu, i= nt dst_cpu, > > + struct task_struct *p) > > +{ > > + struct mm_struct *mm; > > + bool to_pref; > > + int cpu; > > + > > + mm =3D p->mm; > > + if (!mm) > > + return mig_unrestricted; > > + > > + cpu =3D mm->mm_sched_cpu; > > + if (cpu < 0 || cpus_share_cache(src_cpu, dst_cpu)) > > + return mig_unrestricted; > > + > > + if (cpus_share_cache(dst_cpu, cpu)) > > + to_pref =3D true; > > + else if (cpus_share_cache(src_cpu, cpu)) > > + to_pref =3D false; > > + else > > + return mig_unrestricted; > > + > > + return can_migrate_llc(src_cpu, dst_cpu, > > + task_util(p), to_pref); > > +} > > + > > #else > > static inline bool get_llc_stats(int cpu, unsigned long *util, > > unsigned long *cap) > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > > index 84118b522f22..bf72c5bab506 100644 > > --- a/kernel/sched/sched.h > > +++ b/kernel/sched/sched.h > > @@ -2828,6 +2828,11 @@ extern unsigned int sysctl_numa_balancing_scan_p= eriod_max; > > extern unsigned int sysctl_numa_balancing_scan_size; > > extern unsigned int sysctl_numa_balancing_hot_threshold; > > =20 > > +#ifdef CONFIG_SCHED_CACHE > > +extern unsigned int llc_overload_pct; > > +extern unsigned int llc_imb_pct; > > +#endif > > + > > #ifdef CONFIG_SCHED_HRTICK > > =20 > > /* >=20