From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C595130F7F3 for ; Fri, 20 Feb 2026 18:14:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771611254; cv=none; b=eOLTUjd5nGhDu9PXvAReDgw1b0ouJiaToZ9SsEu/QhbZUU1cyMISweI9I3ZtgoopogCWF+B3G+b9Tnu014b4kEhC+bGMnVK4yoty/2Y9R+ztPZUyKkrw3HlgsDIAca6kOX5m3IqFPCXf3cSBQKX2C7EUcunP3mA2TTiPN8FXK1I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771611254; c=relaxed/simple; bh=ergkJDuWdfptNIM8sb2uc1p5Sb3U0as1QKTIUcIZW/c=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=KddPhSX08bxbv0b56aqqn3XwssPfaZwuiW3+iUZMLdI52FG6WVKro4qA6q2UuAvwTil+z8vRH+1idpa9HGctmx9NdJe4o9U9ODwgkdLY/LKSLVapr16UrxGBT00JFEPffgSl7ksVpjLCDh71DFVZpWc1kGIh1dii7a44c1F6SBA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CMxAwY9G; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CMxAwY9G" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1771611252; x=1803147252; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=ergkJDuWdfptNIM8sb2uc1p5Sb3U0as1QKTIUcIZW/c=; b=CMxAwY9Gh3emcmJnmxr3qdJ1YDG5k+0UeOv/l12Dx6+x0PM4LXnOeEow FFzpggiJ9aLse3TVMa51Mt5Rcbq/jA/87NR+HzmfI0boEn39GKXyK7cYu jvN6qFKBB0dnt3YUIMLxX/VUA6RCNSAfbQvjTNYOsqY7n+68/ResSzoim 7ckqrjBsPe4KLZTCLG4SMxqMCT7j6nzohJ+ELPxMxK5SPKt4XQD6uvV6L Fgw55YQIFO8YQE3QrC1oUfwS/OybwBDG8ZTiBachr+PgH1gQklLsKjAqI Ai/uoGmCuUjw8i8v46i72ZjqJ9Gcb4q6u/fnlzVEu1sKjJ8weAqSL5L/J Q==; X-CSE-ConnectionGUID: gly572y6TymV/UAntHJXsg== X-CSE-MsgGUID: OdqkglgCR3KGwO/NuFQqtw== X-IronPort-AV: E=McAfee;i="6800,10657,11707"; a="90292512" X-IronPort-AV: E=Sophos;i="6.21,302,1763452800"; d="scan'208";a="90292512" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Feb 2026 10:14:11 -0800 X-CSE-ConnectionGUID: TjMAIt97QYm1K/XpGAtUpg== X-CSE-MsgGUID: tsaru63ZT0uDKLgPHxfv1g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,302,1763452800"; d="scan'208";a="213012815" Received: from spandruv-mobl5.amr.corp.intel.com (HELO [10.125.110.105]) ([10.125.110.105]) by fmviesa006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Feb 2026 10:14:10 -0800 Message-ID: <43645f25299018e91fcf7464b8f389787c0e9419.camel@linux.intel.com> Subject: Re: [PATCH v3 00/21] Cache Aware Scheduling From: Tim Chen To: Qais Yousef Cc: "Chen, Yu C" , Peter Zijlstra , Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , Josh Don , Gavin Guo , Libo Chen , linux-kernel@vger.kernel.org Date: Fri, 20 Feb 2026 10:14:09 -0800 In-Reply-To: <20260220032941.cw7zdlcy7n6fxmsj@airbuntu> References: <20260219140828.a7pyzupun7lsdw34@airbuntu> <20260219144108.GI1282955@noisy.programming.kicks-ass.net> <66a23acb-973e-4987-a006-18c50e18d3aa@intel.com> <4514b6aef56d0ae144ebd56df9211c6599744633.camel@linux.intel.com> <20260220032941.cw7zdlcy7n6fxmsj@airbuntu> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Fri, 2026-02-20 at 03:29 +0000, Qais Yousef wrote: > On 02/19/26 10:11, Tim Chen wrote: > > On Thu, 2026-02-19 at 23:07 +0800, Chen, Yu C wrote: > > > Hi Peter, Qais, > > >=20 > > > On 2/19/2026 10:41 PM, Peter Zijlstra wrote: > > > > On Thu, Feb 19, 2026 at 02:08:28PM +0000, Qais Yousef wrote: > > > > > On 02/10/26 14:18, Tim Chen wrote: > > >=20 > > > [ ... ] > > >=20 > > > > >=20 > > > > > I admit yet to look fully at the series. But I must ask, why are = you deferring > > > > > to load balance and not looking at wake up path? LB should be for= corrections. > > > > > When wake up path is doing wrong decision all the time, LB (which= is super slow > > > > > to react) is too late to start grouping tasks? What am I missing? > > > >=20 > > > > There used to be wakeup steering, but I'm not sure that still exist= s in > > > > this version (still need to read beyond the first few patches). It = isn't > > > > hard to add. > > > >=20 > > >=20 > > > Please let me explain a little more about why we did this in the > > > load balance path. Yes, the original version implemented cache-aware > > > scheduling only in the wakeup path. According to our testing, this ap= peared > > > to cause some task bouncing issues across LLCs. This was due to confl= icts > > > with the legacy load balancer, which tries to spread tasks to differe= nt=20 > > > LLCs. > > > So as Peter said, the load balancer should be taken care of anyway. L= ater, > > > we kept only the cache aware logic in the load balancer, and the test= =20 > > > results > > > became much more stable, so we kept it as is. The wakeup path more or= less > > > aggregates the wakees(threads within the same process) within the LLC= in=20 > > > the > > > wakeup fast path, so we have not changed it for now. > > >=20 > > > Let me copy the changelog from the previous patch version: > > >=20 > > > " > > > In previous versions, aggregation of tasks were done in the > > > wake up path, without making load balancing paths aware of > > > LLC (Last-Level-Cache) preference. This led to the following > > > problems: > > >=20 > > > 1) Aggregation of tasks during wake up led to load imbalance > > > between LLCs > > > 2) Load balancing tried to even out the load between LLCs > > > 3) Wake up tasks aggregation happened at a faster rate and > > > load balancing moved tasks in opposite directions, leading > > > to continuous and excessive task migrations and regressions > > > in benchmarks like schbench. > > >=20 > > > In this version, load balancing is made cache-aware. The main > > > idea of cache-aware load balancing consists of two parts: > > >=20 > > > 1) Identify tasks that prefer to run on their hottest LLC and > > > move them there. > > > 2) Prevent generic load balancing from moving a task out of > > > its hottest LLC. > > > " > > >=20 > >=20 > > Another reason why we moved away from doing things in the wake up > > path is load imbalance consideration. Wake up path does not have > > the most up to date load information in the LLC sched domains as > > in the load balance path. So you may actually have everyone rushed >=20 > What's the reason wake up doesn't have the latest info? Is this a limitat= ion of > these large systems where stats updates are too expensive to do? Is it no= t > fixable at all? You will need to sum the load for each run queue for each LLC to get an accurate picture. That will be too expensive on the wake up path. Tim >=20 > > into each's favorite LLC and causes LLC overload. And load balance > > will have to undo this. This led to frequent task migrations that > > hurts performance. > >=20 > > It is better to consider LLC preference in the load balance path > > so we can aggregate tasks while still keeping load imbalance under > > control. > >=20 > > Tim