From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCE86372B31 for ; Tue, 17 Feb 2026 18:34:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771353279; cv=none; b=orgSfKoJuYEY4CmyxpvJqEHb2wn03OT/chN3g5pk7Uxgzmam9Uv6Zh9wUgqb9wcyNOMC1lxF1JlF0KpqPAOp6+RPPCKBvWx5hiJ7WQzzJpJegDHbdRqEPTmMn46iASZVqFFt3i5/hqxRQjbF5+CCFqdUiP58O2tk4nDFiKq4j/s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771353279; c=relaxed/simple; bh=ZAtTcxVZ0aaIFnE2F5zdbTndygQTFbvuuyyaJFH7wZw=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=aewY16LnM17XPYOLdT1ETdjUQQeULwC8nwUCVg7R/X7NW6HRJoeGqpSklJbNE9ouKpkF5XEZPMvzwQJEJHi9Di/HKvMTz1rxVq0RSgp7FsDIrCD5MClMMPfyN6WmEOJDV8oPgVzomdalM2mwSSdpbdww6I3hMcJ9VF6dw/oTbTo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=VamuPHQw; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="VamuPHQw" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 61H98Fbm3945141; Tue, 17 Feb 2026 18:33:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=lYLbtC YMVfrFbGAVE49dGgVkBaGMqwiIxU0z4G/TdBA=; b=VamuPHQwWr83bP75ydAJb3 Fx907lKzEHS4BtGpYcKYbvxuw55aRcV/n5iQHybiQkm1uEM40C1Elwx/vYSVT8Rh XvQL8trXvdt2uPqXbIiB7UUt3yf/28GA4FnK5njTy9acIZVwx2rRtn/0OLF0B7Gu 3m1KNCv0LOSWLlUAeLm9vIzPtlPFWydqlYsWF4KFaEgIpBSnadJPxPeNEviUxSq/ NayM61q71/bQN6J3QHOb4ivsdP7v2zsnwpTRsw1gRF2iwThfbnhW7ndNNVf0q6jQ Z4Zulce0bqMG6Fi0UwaU4NfyACWr4euqq014GWhkI3hpFitTnJM6h+Q0ZP2PYC4A == Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cajcjcryn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Feb 2026 18:33:43 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 61HFoCRb001375; Tue, 17 Feb 2026 18:33:42 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4ccb2bc5jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Feb 2026 18:33:42 +0000 Received: from smtpav06.dal12v.mail.ibm.com (smtpav06.dal12v.mail.ibm.com [10.241.53.105]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 61HIXfQI23003808 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Feb 2026 18:33:41 GMT Received: from smtpav06.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 937E458059; Tue, 17 Feb 2026 18:33:41 +0000 (GMT) Received: from smtpav06.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8041958043; Tue, 17 Feb 2026 18:33:24 +0000 (GMT) Received: from [9.67.177.101] (unknown [9.67.177.101]) by smtpav06.dal12v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Feb 2026 18:33:23 +0000 (GMT) Message-ID: <51bac0c9-863b-42cb-8c5c-7a6355840e19@linux.ibm.com> Date: Wed, 18 Feb 2026 00:03:21 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 11/21] sched/cache: Prioritize tasks preferring destination LLC during balancing To: Tim Chen Cc: Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Qais Yousef , Libo Chen , linux-kernel@vger.kernel.org, Madadi Vineeth Reddy References: <4754991218da7da039a0891b0b9647f6eabd5716.1770760558.git.tim.c.chen@linux.intel.com> Content-Language: en-US From: Madadi Vineeth Reddy In-Reply-To: <4754991218da7da039a0891b0b9647f6eabd5716.1770760558.git.tim.c.chen@linux.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-ORIG-GUID: 5eXZniV_utaV6-r0AdV917Gw-Qfi76ty X-Authority-Analysis: v=2.4 cv=Md9hep/f c=1 sm=1 tr=0 ts=6994b488 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=IkcTkHD0fZMA:10 a=HzLeVaNsDn8A:10 a=VkNPw1HP01LnGYTKEx00:22 a=Mpw57Om8IfrbqaoTuvik:22 a=GgsMoib0sEa3-_RKJdDe:22 a=zd2uoN0lAAAA:8 a=QyXUC8HyAAAA:8 a=s2nnpIPtkhTUzEHI370A:9 a=QEXdDO2ut3YA:10 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMjE3MDE0OCBTYWx0ZWRfX/yWw8PlX6b9Z azBAKg86B4nBKpAmvzlTmw/0eFsfxS2j2//hvVUnKxKC78rwci+Ye7gdk033SIeYTY+FuCeWcvd dDSMELN4lUQDeURSvM20f7AJgxcXJNYasIFyRLu2N7KGMUKLNJqvptxi0Cebngfq4lSwaAv0wxL Y9qS2gWdV57f60/9p25TZ00YC434yS7hyjfDWwEuYf+y1WLSRtxPCFxntm2obOJhy+JREnng1Ke mYAQmC0qXrNJeNcdK5BqAAPklJp6m7yoraGrNQbl/grS3juhx8OzMfo7fPjC+7V2GjxkJvjRelt 5qCh4pdKPbGz5vFY7hX49EVbtkYUOOhLqgwrSM7tY263aEqxycHR+elz76Up+jh8upV8RKemEc3 bCHqZqwEquxE2pAxGrwoWI5zKFLnZGtWnxWUQ/B81sjGcIqhn/rfq/R6wd2AKsVbjBKUfEErAD3 6u6CUvz0HMg9V6Dz3jw== X-Proofpoint-GUID: DogdHl1RKqwHUOF7VKglkVcK-VbJCRe4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-02-17_03,2026-02-16_04,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 impostorscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2601150000 definitions=main-2602170148 On 11/02/26 03:48, Tim Chen wrote: > During LLC load balancing, first check for tasks that prefer the > destination LLC and balance them to it before others. > > Mark source sched groups containing tasks preferring non local LLCs > with the group_llc_balance flag. This ensures the load balancer later > pulls or pushes these tasks toward their preferred LLCs. > > The load balancer selects the busiest sched_group and migrates tasks > to less busy groups to distribute load across CPUs. > > With cache-aware scheduling enabled, the busiest sched_group is > the one with most tasks preferring the destination LLC. If > the group has the llc_balance flag set, cache aware load balancing is > triggered. > > Introduce the helper function update_llc_busiest() to identify the > sched_group with the most tasks preferring the destination LLC. > > Suggested-by: K Prateek Nayak > Co-developed-by: Chen Yu > Signed-off-by: Chen Yu > Signed-off-by: Tim Chen > --- > > Notes: > v2->v3: > Consider sd->nr_balance_failed when deciding whether > LLC load balance should be used. > (Peter Zijlstra) > > kernel/sched/fair.c | 77 ++++++++++++++++++++++++++++++++++++++++++++- > 1 file changed, 76 insertions(+), 1 deletion(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index b0cf4424d198..43dcf2827298 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -9649,6 +9649,11 @@ enum group_type { > * from balancing the load across the system. > */ > group_imbalanced, > + /* > + * There are tasks running on non-preferred LLC, possible to move > + * them to their preferred LLC without creating too much imbalance. > + */ > + group_llc_balance, > /* > * The CPU is overloaded and can't provide expected CPU cycles to all > * tasks. > @@ -10561,6 +10566,7 @@ struct sg_lb_stats { > enum group_type group_type; > unsigned int group_asym_packing; /* Tasks should be moved to preferred CPU */ > unsigned int group_smt_balance; /* Task on busy SMT be moved */ > + unsigned int group_llc_balance; /* Tasks should be moved to preferred LLC */ > unsigned long group_misfit_task_load; /* A CPU has a task too big for its capacity */ > #ifdef CONFIG_NUMA_BALANCING > unsigned int nr_numa_running; > @@ -10819,6 +10825,9 @@ group_type group_classify(unsigned int imbalance_pct, > if (group_is_overloaded(imbalance_pct, sgs)) > return group_overloaded; > > + if (sgs->group_llc_balance) > + return group_llc_balance; > + group_llc_balance is placed before group_imbalanced. In cases where a group is both imbalanced and contains tasks preferring the destination LLC, LLC balancing will be selected first. I assume the reasoning is that migrating tasks toward their preferred LLC may also help reduce imbalance, and in cases where the goals conflict, the nr_balance_failed / cache_nice_tries logic will eventually fall back to regular load balancing. Is that the intended policy? It might be helpful to briefly mention this reasoning in the changelog, since this ordering changes balancing priority. Thanks, Vineeth > if (sg_imbalanced(group)) > return group_imbalanced; > > @@ -11012,11 +11021,66 @@ static void record_sg_llc_stats(struct lb_env *env, > if (unlikely(READ_ONCE(sd_share->capacity) != sgs->group_capacity)) > WRITE_ONCE(sd_share->capacity, sgs->group_capacity); > } > + > +/* > + * Do LLC balance on sched group that contains LLC, and have tasks preferring > + * to run on LLC in idle dst_cpu. > + */ > +static inline bool llc_balance(struct lb_env *env, struct sg_lb_stats *sgs, > + struct sched_group *group) > +{ > + if (!sched_cache_enabled()) > + return false; > + > + if (env->sd->flags & SD_SHARE_LLC) > + return false; > + > + /* > + * Don't do cache aware balancing if there > + * are too many balance failures. > + * > + * Should fall back to regular load balancing > + * after repeated cache aware balance failures. > + */ > + if (env->sd->nr_balance_failed >= > + env->sd->cache_nice_tries + 1) > + return false; > + > + if (sgs->nr_pref_dst_llc && > + can_migrate_llc(cpumask_first(sched_group_span(group)), > + env->dst_cpu, 0, true) == mig_llc) > + return true; > + > + return false; > +} > + > +static bool update_llc_busiest(struct lb_env *env, > + struct sg_lb_stats *busiest, > + struct sg_lb_stats *sgs) > +{ > + /* > + * There are more tasks that want to run on dst_cpu's LLC. > + */ > + return sgs->nr_pref_dst_llc > busiest->nr_pref_dst_llc; > +} > #else > static inline void record_sg_llc_stats(struct lb_env *env, struct sg_lb_stats *sgs, > struct sched_group *group) > { > } > + > +static inline bool llc_balance(struct lb_env *env, struct sg_lb_stats *sgs, > + struct sched_group *group) > +{ > + return false; > +} > + > +static bool update_llc_busiest(struct lb_env *env, > + struct sg_lb_stats *busiest, > + struct sg_lb_stats *sgs) > +{ > + return false; > +} > #endif > > /** > @@ -11118,6 +11182,10 @@ static inline void update_sg_lb_stats(struct lb_env *env, > /* Check for loaded SMT group to be balanced to dst CPU */ > if (smt_balance(env, sgs, group)) > sgs->group_smt_balance = 1; > + > + /* Check for tasks in this group can be moved to their preferred LLC */ > + if (llc_balance(env, sgs, group)) > + sgs->group_llc_balance = 1; > } > > sgs->group_type = group_classify(env->sd->imbalance_pct, group, sgs); > @@ -11181,6 +11249,10 @@ static bool update_sd_pick_busiest(struct lb_env *env, > /* Select the overloaded group with highest avg_load. */ > return sgs->avg_load > busiest->avg_load; > > + case group_llc_balance: > + /* Select the group with most tasks preferring dst LLC */ > + return update_llc_busiest(env, busiest, sgs); > + > case group_imbalanced: > /* > * Select the 1st imbalanced group as we don't have any way to > @@ -11443,6 +11515,7 @@ static bool update_pick_idlest(struct sched_group *idlest, > return false; > break; > > + case group_llc_balance: > case group_imbalanced: > case group_asym_packing: > case group_smt_balance: > @@ -11575,6 +11648,7 @@ sched_balance_find_dst_group(struct sched_domain *sd, struct task_struct *p, int > return NULL; > break; > > + case group_llc_balance: > case group_imbalanced: > case group_asym_packing: > case group_smt_balance: > @@ -12074,7 +12148,8 @@ static struct sched_group *sched_balance_find_src_group(struct lb_env *env) > * group's child domain. > */ > if (sds.prefer_sibling && local->group_type == group_has_spare && > - sibling_imbalance(env, &sds, busiest, local) > 1) > + (busiest->group_type == group_llc_balance || > + sibling_imbalance(env, &sds, busiest, local) > 1)) > goto force_balance; > > if (busiest->group_type != group_overloaded) {