From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AFA2363C5D for ; Wed, 4 Feb 2026 22:58:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770245928; cv=none; b=rDDghb7593rHR2fvqlpOkmT7rdE1/VObnhw0qxo+LajZM3n/h68HcjZq7k8sjbNZL/S4M+t+hYljoyNV3uUyyvQoFF4tT5lWSirOWWNcYACfYSoERLtuN3Xsu+46CAiIpuoilBg6MEjF3ARrMacdRGRhfnjNnP2S8HjM9fbkE2w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770245928; c=relaxed/simple; bh=jU+eFmr8zBESXwupkDGNESRl4aAqdDfSAvwNMj2oGcI=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=KbAlSyt78nhAaJ6jAk1ibiqa4IUsVwxH4qGPMZzvVsPkFJLjcfM9VycbExRcHkvQ0joYjXfuUO9Lm8CYT7pWScji2m4WgpjGyijEqc3jcMAkTJgWCzclhE4vSEDdEIYzyCA4Z+RcAxQL+QA/xiFwzEgDfHUVmPsxqSX1tdpI64A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FXwRqiLm; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FXwRqiLm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1770245928; x=1801781928; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=jU+eFmr8zBESXwupkDGNESRl4aAqdDfSAvwNMj2oGcI=; b=FXwRqiLmsvh3y3PkV9q9AiOpXUrktAl7HQ25FsBMu/qh3bSHK2FaGKgH vul46g7g0JF2pxJPHu/8LnDnyvUwKuOwOhgrOUQXdTZuqIPRCAi3S6SW4 bfZBxaoHRede4ofHv1Y79z12RdRpVd5W8sJBhCh7NfKr4+xrw8/8kZ682 FSpcHsy6Mmjz/LywjGebd0NHZsKwzWgsoi3Z4zQGeqib8iZn2oBE5MSaX HacoP2f3p0QViCUIH2AXPgYsq8BXvA79V2krFuYgmt231ybUCuv3F4WnQ FyXoEmQmL7aXe0e6XBT6JPr9Uv3mVDXP8MFLtHTE072ll3QbJdwqqURq0 w==; X-CSE-ConnectionGUID: Mc+/5WabRsSwufYfxeFOZQ== X-CSE-MsgGUID: exehcsGzSoytPEmZkyC+Yg== X-IronPort-AV: E=McAfee;i="6800,10657,11691"; a="71170055" X-IronPort-AV: E=Sophos;i="6.21,273,1763452800"; d="scan'208";a="71170055" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2026 14:58:47 -0800 X-CSE-ConnectionGUID: 3lCUcZO3QaCfqu8gHKKIIg== X-CSE-MsgGUID: HC4iKMP6QEmvKfyaKKqC2w== X-ExtLoop1: 1 Received: from unknown (HELO [10.241.243.83]) ([10.241.243.83]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2026 14:58:46 -0800 Message-ID: Subject: Re: [PATCH] sched/topology: Check average distances to remote packages From: Tim Chen To: Kyle Meyer , bp@alien8.de, dave.hansen@linux.intel.com, mingo@redhat.com, peterz@infradead.org, tglx@kernel.org, vinicius.gomes@intel.com Cc: brgerst@gmail.com, hpa@zytor.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, patryk.wlazlyn@linux.intel.com, rafael.j.wysocki@intel.com, russ.anderson@hpe.com, x86@kernel.org, yu.c.chen@intel.com, zhao1.liu@intel.com Date: Wed, 04 Feb 2026 14:58:46 -0800 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Tue, 2026-01-27 at 11:02 -0600, Kyle Meyer wrote: > Granite Rapids (GNR) and Clearwater Forest (CWF) average distances to > remote packages to fix scheduler domains, see [1] for more information. >=20 > A warning and backtrace are printed when sub-NUMA clustering (SNC) is > enabled and there are more than 2 packages because the average distances > to remote packages could be different, skewing the single average remote > distance. >=20 > This is unnecessary when the average distances to remote packages are > the same. >=20 > Support single average remote distance on systems with more than 2 > packages, preventing unnecessary warnings and backtraces by checking if > average distances to remote packages are the same. >=20 > [1] commit 4d6dd05d07d0 ("sched/topology: Fix sched domain build error fo= r GNR, CWF in SNC-3 mode"). >=20 > Signed-off-by: Kyle Meyer > --- >=20 > The warning and backtrace were noticed on a 16 socket GNR system with SNC= -2 enabled. >=20 > --- > arch/x86/kernel/smpboot.c | 70 ++++++++++++++++++++++++++++----------- > 1 file changed, 51 insertions(+), 19 deletions(-) >=20 > diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c > index 5cd6950ab672..4467716f4054 100644 > --- a/arch/x86/kernel/smpboot.c > +++ b/arch/x86/kernel/smpboot.c > @@ -518,27 +518,70 @@ static int avg_remote_numa_distance(void) > { > int i, j; > int distance, nr_remote, total_distance; > + int max_pkgs =3D topology_max_packages(); > + int cpu, pkg, pkg_avg_distance; > + int *pkg_total_distance; > + int *pkg_nr_remote; Minor nit, pkg_nr_remote needs to be initialized as NULL. Otherwise if allocation of pkg_total_distance failed and we go to cleanup, we can call kfree(pkg_nr_remote) with a non null uninitialized pkg_nr_remote pointer. Maybe do int *pkg_total_distance=3DNULL, *pkg_nr_remote=3DNULL; in declaration. > =20 > if (sched_avg_remote_distance > 0) > return sched_avg_remote_distance; > =20 > + sched_avg_remote_distance =3D REMOTE_DISTANCE; > + > nr_remote =3D 0; > total_distance =3D 0; > + > + pkg_total_distance =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); > + if (!pkg_total_distance) > + goto cleanup; > + > + pkg_nr_remote =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); > + if (!pkg_nr_remote) > + goto cleanup; > + > for_each_node_state(i, N_CPU) { > for_each_node_state(j, N_CPU) { > distance =3D node_distance(i, j); > =20 > - if (distance >=3D REMOTE_DISTANCE) { > - nr_remote++; > - total_distance +=3D distance; > - } > + if (distance < REMOTE_DISTANCE) > + continue; > + > + nr_remote++; > + total_distance +=3D distance; > + > + cpu =3D cpumask_first(cpumask_of_node(j)); > + if (cpu >=3D nr_cpu_ids) > + continue; > + > + pkg =3D topology_physical_package_id(cpu); > + pkg_total_distance[pkg] +=3D distance; > + pkg_nr_remote[pkg]++; > } > } > - if (nr_remote) > - sched_avg_remote_distance =3D total_distance / nr_remote; > - else > - sched_avg_remote_distance =3D REMOTE_DISTANCE; > =20 > + if (!nr_remote) > + goto cleanup; > + > + sched_avg_remote_distance =3D total_distance / nr_remote; > + > + /* > + * Single average remote distance won't be appropriate if different > + * packages have different distances to remote packages. > + */ > + for (i =3D 0; i < max_pkgs; i++) { > + if (!pkg_nr_remote[i]) > + continue; > + > + pkg_avg_distance =3D pkg_total_distance[i] / pkg_nr_remote[i]; > + > + pr_debug("sched: Avg. distance to remote package %d: %d\n", i, pkg_avg= _distance); > + > + if (pkg_avg_distance !=3D sched_avg_remote_distance) > + WARN_ONCE(1, "sched: Avg. distances to remote packages are different\= n"); > + } > +cleanup: > + kfree(pkg_nr_remote); > + kfree(pkg_total_distance); > return sched_avg_remote_distance; > } > =20 > @@ -564,18 +607,7 @@ int arch_sched_node_distance(int from, int to) > * in the remote package in the same sched group. > * Simplify NUMA domains and avoid extra NUMA levels including > * different remote NUMA nodes and local nodes. > - * > - * GNR and CWF don't expect systems with more than 2 packages > - * and more than 2 hops between packages. Single average remote > - * distance won't be appropriate if there are more than 2 > - * packages as average distance to different remote packages > - * could be different. > */ > - WARN_ONCE(topology_max_packages() > 2, > - "sched: Expect only up to 2 packages for GNR or CWF, " > - "but saw %d packages when building sched domains.", > - topology_max_packages()); > - > d =3D avg_remote_numa_distance(); > } > return d;