From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67C921494A8 for ; Tue, 24 Dec 2024 18:25:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735064733; cv=none; b=ateXiS0nlljL4dZJjMN8zRNGMQ2+08BCZssfl7qHN1lSla6rGwZ8X1Gn1Rl5BGx6ZD2QxoA/1Jebv1ynxbk29RaKWHPbjJ6/PMx++YzrqIgRbPedpxj6PwDmKHXVH3+useTDcFuy5DkT7Yrzv3hWdz355ztkx3QzcGr6c3LE4CE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735064733; c=relaxed/simple; bh=A6xCjboCjzBc5AzONP+jWxepsG/UePklhAz38HzUA2g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BI51IN1j989BB2ZmowpgfJ+Urv3qiGeb6l6HhIQqfB0gGJii+RoC1srkCdu+PPjXqGWNwY28Mp/C6lTLiHyD33taT1O1Bj5jo1yVZ9HioLSQs4AsW2YMyuB6Ue/fyjaDzrqax1qXx0fBJhQjEJqhO1lbPMNHGdnZF/cGgfRhDYY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=XZxnAATZ; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="XZxnAATZ" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=eygFBrBXOwlRBs5bhtsvaEFxZnDDs9HHJ94ZRer4BpQ=; b=XZxnAATZAofwsjN+l55Q+GWbYA nuv2EN+OfazcA/Xwv55KF6Dqrm4K2mmveSzN/MKAfDWL26oogPEQcbTA/yhfLMUMByixt2JK9h5Yz /9qaso2bqssnFdEABf5vFmW1idAjQaGuItdSNyHHNBK8N2ll5ueIzgHaNWj0hH73M8hTbOkFQU29X bhp+jqXgMaRb2dMZLhRyLs79zNV9hO0YNY2otClM6f7RVDQiTTiMOQxFD0p4LzbzHJC0BkTcq0p2K hbTz82mJ8etFDXQnAsDbbj/WbaKzjzai3tiwOfZcvTia/OLPgVir4cJlwuaB3Rwk8ilVziS96QL8S r5rc6Mjg==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tQ9ap-00000006e0z-0fNs; Tue, 24 Dec 2024 18:25:19 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 87BCF300478; Tue, 24 Dec 2024 19:25:18 +0100 (CET) Date: Tue, 24 Dec 2024 19:25:18 +0100 From: Peter Zijlstra To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, akpm@linux-foundation.org Subject: Re: [PATCH 09/10] x86/mm: enable AMD translation cache extensions Message-ID: <20241224182518.GB17252@noisy.programming.kicks-ass.net> References: <20241222040717.3096835-1-riel@surriel.com> <20241222040717.3096835-10-riel@surriel.com> <20241222113811.GY11133@noisy.programming.kicks-ass.net> <5ae11e5ac6b278de9ad2ad2badbd5f010543d934.camel@surriel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5ae11e5ac6b278de9ad2ad2badbd5f010543d934.camel@surriel.com> On Sun, Dec 22, 2024 at 10:37:01AM -0500, Rik van Riel wrote: > On Sun, 2024-12-22 at 12:38 +0100, Peter Zijlstra wrote: > > On Sat, Dec 21, 2024 at 11:06:41PM -0500, Rik van Riel wrote: > > > With AMD TCE (translation cache extensions) only the intermediate > > > mappings > > > > Only the leave mapings, as written this all don't make sense, > > Check out page 513 of the AMD manual: > > https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/40332.pdf > > "Translation Cache Extension (TCE) Bit. Bit 15, read/write.  > > Setting this bit to 1 changes how the INVLPG, INVLPGB, and INVPCID > instructions operate on TLB entries. When this bit is 0, these > instructions remove the target PTE from the TLB as well as all  > upper-level table entries that are cached in the TLB, whether or  > not they are associated with the target PTE. When this bit is set, > these instructions will remove the target PTE and only those  > upper-level entries that lead to the target PTE in the page table > hierarchy, leaving unrelated upper-level entries intact. This may > provide a performance benefit. > > Page table management software must be written in a way that takes  > this behavior into account. Software that was written for a  > processor that does not cache upper-level table entries may result  > in stale entries being incorrectly used for translations when TCE  > is enabled. Software that is compatible with TCE mode will operate > in either mode. > > For software using INVLPGB to broadcast TLB invalidations, the > invalidations are controlled by the EFER.TCE value on the processor > executing the INVLPGB instruction. > > Before setting TCE, system software should verify that this feature > is supported by examining the feature flag CPUID Fn8000_0001_ECX[TCE]. > See Section 3.3 “Processor Feature Identification,” on > page 71 for information on using the CPUID instruction" So that makes a ton more sense. > > This suggests that: > 1) TCE does control the "don't make sense" behavior :) Well, you wrote: > > With AMD TCE (translation cache extensions) only the intermediate mappings > > that cover the address range zapped by INVLPG / INVLPGB get invalidated, > > rather than all intermediate mappings getting zapped at every TLB invalidation. And I read that like it would zap only the intermediate mappings rather than the intermediate mappings. Reading it a wee bit more carefully, I see it's not quite as bad, but still not very clear. > 2) Wait, does EFER.TCE need to be set on every CPU > in the system? Could a system run with TCE set > on some CPUs, and cleared on another?! I would imagine it can; I don't think they would recommend anybody do this though.