From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF3122CA6 for ; Sun, 12 Jan 2025 02:50:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=96.67.55.147 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736650247; cv=none; b=Udi0dPUJS6R5WhV8jUjv5SF/lPr9GpozXJfu560zd+g+0HRk1CdoCnsdx22nxhgASQfjbJo1bJt3xPJYoUEv4gAJQu12xDK/Bh1ihJTdniczmbH944ivrqfUBCh7mZ1oQxDhW2EktZ5oLjRVlf2GiqlNh0vumhMCYTFBLE/VbR4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736650247; c=relaxed/simple; bh=ug2WHQ05K4un0wjQ7AndcLyw/j+DpAoSu+SnAtW7geg=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=tTk02zbAOYoalBBdDsxDg/lPGjISa1lQrKoIG/KuN87OPl+/FWRiPJ6EwMfGzyCFGZejbsfSd6VF6djs8kQLIEXXVSoJgOF57KizTkWoXnsszSZ4NroaSOC25YK90ds3uanmaJ6xNNWn1N73ixsIXnk4aD+Z/AsP7BG0uY5yjU4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com; spf=pass smtp.mailfrom=shelob.surriel.com; arc=none smtp.client-ip=96.67.55.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shelob.surriel.com Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tWo05-000000000d5-3ptU; Sat, 11 Jan 2025 21:46:53 -0500 Message-ID: <0a24ce5d1ae0c782c9c676466bd7051693852997.camel@surriel.com> Subject: Re: [PATCH v3 00/12] AMD broadcast TLB invalidation From: Rik van Riel To: Dave Hansen , x86@kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, akpm@linux-foundation.org, nadav.amit@gmail.com, zhengqi.arch@bytedance.com, linux-mm@kvack.org Date: Sat, 11 Jan 2025 21:46:53 -0500 In-Reply-To: References: <20241230175550.4046587-1-riel@surriel.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.1 (3.54.1-1.fc41) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: riel@surriel.com On Mon, 2025-01-06 at 11:03 -0800, Dave Hansen wrote: >=20 > So can we call them "global", "shared" or "system" ASIDs, please? >=20 I have renamed them to global ASIDs. > Second, the TLB_NR_DYN_ASIDS was picked because it's roughly the > number > of distinct PCIDs that the CPU can keep in the TLB at once (at least > on > Intel). Let's say a CPU has 6 mm's in the per-cpu ASID space and > another > 6 in the shared/broadcast space. At that point, PCIDs might not be > doing > much good because the TLB can't store entries for 12 PCIDs. >=20 If the CPU has 12 runnable processes, we may have various other performance issues, too, like the system simply not having enough CPU power to run all the runnable tasks. Most of the systems I have looked at seem to average between .2 and 2 runnable tasks per CPU, depending on whether the workload is CPU bound, or memory/IO bound. > Is there any comprehension in this series? Should we be indexing > cpu_tlbstate.ctxs[] by a *context* number rather than by the ASID > that > it's running as? >=20 We only need the cpu_tlbstate.ctxs[] for the per-CPU ASID space, in order to look up what process is assigned which slot. We do not need it for global ASID numbers, which are always the same everywhere. > Last, I'm not 100% convinced we want to do this whole thing. The > will-it-scale numbers are nice. But given the complexity of this, I > think we need some actual, real end users to stand up and say exactly > how this is important in *PRODUCTION* to them. >=20 Do any of these count? :) https://www.phoronix.com/review/amd-invlpgb-linux I am hoping to gather some real world numbers as well, and will work with some workload owners to get some numbers. --=20 All Rights Reversed.