From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A98E7325494 for ; Tue, 16 Dec 2025 22:45:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765925167; cv=none; b=na/GZXoFuF6z3jPpnFdZorou0zOv0uHO8dVU3Oo0v8lC/FQWGLxl9wRfBUM5HwU3bu4aJcFtpezv3eCCG/yVFM9zVreWZ/uFgPyIKUoT4hKkhPoBGmH0buwBJUY/4k4PGGKgySy93VuzO4Bgc3oSyajC851QHJpZw7vmTlQMt9Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765925167; c=relaxed/simple; bh=Sf9mD0+biixabHmDy5Qeu89Zjw+H/OCCqSOgqKkxZ0Y=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=a3XnguHtyCBu9T4TK+f11UScshF0mE7umFEHmz0g26Rd3FsO5MD0VU63NJ9fVbrul5h4soBhI9rbZdvtr7kxHQhJEcZP8Cbge3piHAWNhJTwYJ2dWzP/byDZzC8HzLwr5aMActYly7Ys3rXBTHF2mKqdZDyfs4vrqq38V5yf4SQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZYrBYXPN; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZYrBYXPN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1765925161; x=1797461161; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=Sf9mD0+biixabHmDy5Qeu89Zjw+H/OCCqSOgqKkxZ0Y=; b=ZYrBYXPNOE8BvEHfVCBoC6gYlgfGJTaABIcmqxpG5k5dkthx9Z0PC5Ya Jfh3fYTLFGkU5OltfkidIrV+sX0mbgWxO3N8tqkvCIrabjYRzjBWaMcnw RZqii+qczPkrlQ44O03PDtqQyzOWS9AsrZ+irVFSV674Gs54nNPOeaSS5 OMfOZXaPLESMJiQa9/EjhQ8MsAj24NE950QSjpVvxlJC6kD2GguauQXhD jgDJUEOImVNqjrjZd9ZWWxXd+kIUpFxZb96AGUpyArGOCfl6UBpQJyfPW +p7ZSP4FPn+2KckIeRATb9EPYYJZmRv3s8XSJubGocq6O9fg2AiU+Stpk Q==; X-CSE-ConnectionGUID: cdY24KxxT0qJoFtqer8ypw== X-CSE-MsgGUID: +0oZbJMYSSW2Hm3mTkCwFg== X-IronPort-AV: E=McAfee;i="6800,10657,11644"; a="71483295" X-IronPort-AV: E=Sophos;i="6.21,154,1763452800"; d="scan'208";a="71483295" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2025 14:45:59 -0800 X-CSE-ConnectionGUID: 4oJCXRQhQWiQSOkVPbYJqQ== X-CSE-MsgGUID: bKzSYFfWS9mt8X1g6foQlQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,154,1763452800"; d="scan'208";a="197426580" Received: from schen9-mobl4.amr.corp.intel.com (HELO [10.125.110.205]) ([10.125.110.205]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2025 14:45:58 -0800 Message-ID: <181d7d3b57439976d773f25b99c2df1d677bb8e6.camel@linux.intel.com> Subject: Re: [PATCH v2 07/23] sched/cache: Introduce per runqueue task LLC preference counter From: Tim Chen To: Peter Zijlstra Cc: Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org Date: Tue, 16 Dec 2025 14:45:57 -0800 In-Reply-To: <20251211103150.GA3707891@noisy.programming.kicks-ass.net> References: <63091f7ca7bb473fbc176af86a87d27a07a6e149.1764801860.git.tim.c.chen@linux.intel.com> <20251210125114.GS3707891@noisy.programming.kicks-ass.net> <20251211103150.GA3707891@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Thu, 2025-12-11 at 11:31 +0100, Peter Zijlstra wrote: > On Wed, Dec 10, 2025 at 10:49:14AM -0800, Tim Chen wrote: > > On Wed, 2025-12-10 at 13:51 +0100, Peter Zijlstra wrote: > > > On Wed, Dec 03, 2025 at 03:07:26PM -0800, Tim Chen wrote: >=20 > > > Would it perhaps be easier to stick this thing in rq->sd rather than = in > > > rq->nr_pref_llc. That way it automagically switches with the 'new' > > > domain. And then, with a bit of care, a singe load-balance pass shoul= d > > > see a consistent view (there should not be reloads of rq->sd -- which > > > will be a bit of an audit I suppose). > >=20 > > We need nr_pref_llc information at the runqueue level because the load = balancer=C2=A0 > > must identify which specific rq has the largest number of tasks that=C2= =A0 > > prefer a given destination LLC. If we move the counter to the LLC=E2=80= =99s sd=C2=A0 > > level, we would only know the aggregate number of tasks in the entire L= LC=C2=A0 > > that prefer that destination=E2=80=94not which rq they reside on. Witho= ut per-rq=C2=A0 > > counts, we would not be able to select the correct source rq to pull ta= sks from. > >=20 > > The only way this could work at the LLC-sd level is if all CPUs within= =C2=A0 > > the LLC shared a single runqueue, which is not the case today. > >=20 > > Let me know if I understand your comments correctly. >=20 > So the sched_domain instances are per-cpu (hence the need for > sched_domain_shared). So irrespective of what level you stick them at (I > was thinking the bottom most, but it really doesn't matter) they will be > per CPU. One side effect of that is when rebuild_sched_domains() got triggered, all rq->sd is getting reallocated. So we'll lose the old LLC preferences till we have time to re-sample process occupancy. I think it is okay as long as the call to rebuild_sched_domains() too frequently. Is this assumption correct? Tim=20