From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B3E928541A for ; Tue, 9 Dec 2025 11:59:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765281573; cv=none; b=cMyozoiz/H8UzhWh9N4QA1xUviPQCaq51PjUQnXnPUvXNfZwioUKsKP8/8Gkr6lxjvOV15Nej9sPEUCMmy5ruPx259VogZujEzOCavKzOxKCE6wKY3Si8DqywaQUSgWkGxts3XVQ091sWz0KGtYXjChtxUw47su/91Jz69B7VQE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765281573; c=relaxed/simple; bh=xAeADyOyptCYts7VwpCOSBWp+0K6zDUiVLeESyyI1hY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=KE30+O8nQ3V5YSvtVJy0ZmVMKh47dQCBe17NCw7LdwesxQiaW66wkQD3sH1YEaNynDJTGPvSVKef2nRCtPyUrX9vZBOUi/bJr/FG9n6gm32XO8NjV/FOiVyDU7fDdcqDGv5Lr1b7girNS1w7QhEq1iOs+mGIMtaNLga3sMKkpWQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=OsnVBiLE; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OsnVBiLE" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0cV+ZbQ932iYk5nxhP+uFBJvNFruGgceBmTWJ/wSjSs=; b=OsnVBiLEKOYf4FLTKtJjZUdcjP NAL4/n8p+nexjHygIZbaynGsVjxcQebmTe+IYoLgYMMr8gO3EhDz6sB7xDarSwe4XerZxrr6rulwo kzkTAjNNdxk5gb0N9zlJmbDSCgubZaeiSdIzKNEcTrIrG8hlzGQp4s1blAn05jZdNNtk0w4G2eP98 fi44hwKxBldMbcJce+FfKr17l85VnpqyLoe0wbPykVuOwJRgRhdOwPvHIUx988iNbl5yQb/6A2bCA fJHfBD2qKvPROmcG3782iZOlfZCdyLIgWXj8G95Itop2dJKvd2d8u5eZjNThpFy2E9Wp++HkPzz6k NrzaZQ5Q==; Received: from 77-249-17-252.cable.dynamic.v4.ziggo.nl ([77.249.17.252] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1vSvVB-0000000BhM4-3WTL; Tue, 09 Dec 2025 11:03:30 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 86AF630045C; Tue, 09 Dec 2025 12:58:46 +0100 (CET) Date: Tue, 9 Dec 2025 12:58:46 +0100 From: Peter Zijlstra To: Tim Chen Cc: Ingo Molnar , K Prateek Nayak , "Gautham R . Shenoy" , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Madadi Vineeth Reddy , Hillf Danton , Shrikanth Hegde , Jianyong Wu , Yangyu Chen , Tingyin Duan , Vern Hao , Vern Hao , Len Brown , Aubrey Li , Zhao Liu , Chen Yu , Chen Yu , Adam Li , Aaron Lu , Tim Chen , linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 04/23] sched/cache: Make LLC id continuous Message-ID: <20251209115846.GL3707891@noisy.programming.kicks-ass.net> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Wed, Dec 03, 2025 at 03:07:23PM -0800, Tim Chen wrote: > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 710ed9943d27..0a3918269906 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -1210,10 +1210,17 @@ __read_mostly unsigned int llc_imb_pct = 20; > > static int llc_id(int cpu) > { > + int llc; > + > if (cpu < 0) > return -1; > > + llc = per_cpu(sd_llc_id, cpu); > + /* avoid race with cpu hotplug */ > + if (unlikely(llc >= max_llcs)) > + return -1; > + > + return llc; > } > > void mm_init_sched(struct mm_struct *mm, struct mm_sched __percpu *_pcpu_sched) > @@ -668,6 +670,55 @@ DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity); > DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity); > DEFINE_STATIC_KEY_FALSE(sched_cluster_active); > > +/* > + * Assign continuous llc id for the CPU, and return > + * the assigned llc id. > + */ > +static int update_llc_id(struct sched_domain *sd, > + int cpu) > +{ > + int id = per_cpu(sd_llc_id, cpu), i; > + > + if (id >= 0) > + return id; > + > + if (sd) { > + /* Look for any assigned id and reuse it.*/ > + for_each_cpu(i, sched_domain_span(sd)) { > + id = per_cpu(sd_llc_id, i); > + > + if (id >= 0) { > + per_cpu(sd_llc_id, cpu) = id; > + return id; > + } > + } > + } > + > + /* > + * When 1. there is no id assigned to this LLC domain, > + * or 2. the sd is NULL, we reach here. > + * Consider the following scenario, > + * CPU0~CPU95 are in the node0, CPU96~CPU191 are > + * in the node1. During bootup, maxcpus=96 is > + * appended. > + * case 1: When running cpu_attach_domain(CPU24) > + * during boot up, CPU24 is the first CPU in its > + * non-NULL LLC domain. However, > + * its corresponding llc id has not been assigned yet. > + * > + * case 2: After boot up, the CPU100 is brought up > + * via sysfs manually. As a result, CPU100 has only a > + * Numa domain attached, because CPU100 is the only CPU > + * of a sched domain, all its bottom domains are degenerated. > + * The LLC domain pointer sd is NULL for CPU100. > + * > + * For both cases, we want to increase the number of LLCs. > + */ > + per_cpu(sd_llc_id, cpu) = max_llcs++; > + > + return per_cpu(sd_llc_id, cpu); > +} I'm not sure I follow. So partition_sched_domains() first calls detach_destroy_domains() on the old set, and then build_sched_domains() on the new set. Do detach_destroy_domain() will do: cpu_attach_domain(NULL,..); That is, it will explicitly attach the NULL sched_domain to a CPU. At which point I feel update_llc_id() should be returning -1, no? Then later, build_sched_domains() will set a !NULL sched_domain, at which point update_llc_id() can set a real value. This should then also get rid of that weird max_llcs check in llc_id(), right?