mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "tip-bot2 for Rik van Riel" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Rik van Riel <riel@surriel.com>,
	"Borislav Petkov (AMD)" <bp@alien8.de>,
	x86@kernel.org, linux-kernel@vger.kernel.org
Subject: [tip: x86/mm] x86/mm: Enable AMD translation cache extensions
Date: Wed, 05 Mar 2025 21:36:22 -0000	[thread overview]
Message-ID: <174121058250.14745.1312436108807852558.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20250226030129.530345-13-riel@surriel.com>

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     b95ef870dcd67c0fbfc57f808c4a0efd0bc2f144
Gitweb:        https://git.kernel.org/tip/b95ef870dcd67c0fbfc57f808c4a0efd0bc2f144
Author:        Rik van Riel <riel@surriel.com>
AuthorDate:    Tue, 25 Feb 2025 22:00:47 -05:00
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Wed, 05 Mar 2025 22:18:16 +01:00

x86/mm: Enable AMD translation cache extensions

With AMD TCE (translation cache extensions) only the intermediate mappings
that cover the address range zapped by INVLPG / INVLPGB get invalidated,
rather than all intermediate mappings getting zapped at every TLB invalidation.

This can help reduce the TLB miss rate, by keeping more intermediate mappings
in the cache.

>From the AMD manual:

Translation Cache Extension (TCE) Bit. Bit 15, read/write. Setting this bit to
1 changes how the INVLPG, INVLPGB, and INVPCID instructions operate on TLB
entries. When this bit is 0, these instructions remove the target PTE from the
TLB as well as all upper-level table entries that are cached in the TLB,
whether or not they are associated with the target PTE.  When this bit is set,
these instructions will remove the target PTE and only those upper-level
entries that lead to the target PTE in the page table hierarchy, leaving
unrelated upper-level entries intact.

  [ bp: use cpu_has()... I know, it is a mess. ]

Signed-off-by: Rik van Riel <riel@surriel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20250226030129.530345-13-riel@surriel.com
---
 arch/x86/include/asm/msr-index.h       | 2 ++
 arch/x86/kernel/cpu/amd.c              | 4 ++++
 tools/arch/x86/include/asm/msr-index.h | 2 ++
 3 files changed, 8 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 72765b2..1aacd6b 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -25,6 +25,7 @@
 #define _EFER_SVME		12 /* Enable virtualization */
 #define _EFER_LMSLE		13 /* Long Mode Segment Limit Enable */
 #define _EFER_FFXSR		14 /* Enable Fast FXSAVE/FXRSTOR */
+#define _EFER_TCE		15 /* Enable Translation Cache Extensions */
 #define _EFER_AUTOIBRS		21 /* Enable Automatic IBRS */
 
 #define EFER_SCE		(1<<_EFER_SCE)
@@ -34,6 +35,7 @@
 #define EFER_SVME		(1<<_EFER_SVME)
 #define EFER_LMSLE		(1<<_EFER_LMSLE)
 #define EFER_FFXSR		(1<<_EFER_FFXSR)
+#define EFER_TCE		(1<<_EFER_TCE)
 #define EFER_AUTOIBRS		(1<<_EFER_AUTOIBRS)
 
 /*
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 7a72ef4..7058533 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -1075,6 +1075,10 @@ static void init_amd(struct cpuinfo_x86 *c)
 
 	/* AMD CPUs don't need fencing after x2APIC/TSC_DEADLINE MSR writes. */
 	clear_cpu_cap(c, X86_FEATURE_APIC_MSRS_FENCE);
+
+	/* Enable Translation Cache Extension */
+	if (cpu_has(c, X86_FEATURE_TCE))
+		msr_set_bit(MSR_EFER, _EFER_TCE);
 }
 
 #ifdef CONFIG_X86_32
diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h
index 3ae84c3..dc1c105 100644
--- a/tools/arch/x86/include/asm/msr-index.h
+++ b/tools/arch/x86/include/asm/msr-index.h
@@ -25,6 +25,7 @@
 #define _EFER_SVME		12 /* Enable virtualization */
 #define _EFER_LMSLE		13 /* Long Mode Segment Limit Enable */
 #define _EFER_FFXSR		14 /* Enable Fast FXSAVE/FXRSTOR */
+#define _EFER_TCE		15 /* Enable Translation Cache Extensions */
 #define _EFER_AUTOIBRS		21 /* Enable Automatic IBRS */
 
 #define EFER_SCE		(1<<_EFER_SCE)
@@ -34,6 +35,7 @@
 #define EFER_SVME		(1<<_EFER_SVME)
 #define EFER_LMSLE		(1<<_EFER_LMSLE)
 #define EFER_FFXSR		(1<<_EFER_FFXSR)
+#define EFER_TCE		(1<<_EFER_TCE)
 #define EFER_AUTOIBRS		(1<<_EFER_AUTOIBRS)
 
 /*

  reply	other threads:[~2025-03-05 21:36 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-26  3:00 [PATCH v14 00/13] AMD broadcast TLB invalidation Rik van Riel
2025-02-26  3:00 ` [PATCH v14 01/13] x86/mm: consolidate full flush threshold decision Rik van Riel
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Consolidate " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-09-02 15:44   ` [BUG] x86/mm: regression after 4a02ed8e1cc3 Giovanni Cabiddu
2025-09-02 15:50     ` Dave Hansen
2025-09-02 16:08       ` Nadav Amit
2025-09-02 16:11         ` Dave Hansen
2025-09-03 14:00       ` Rik van Riel
2025-09-02 16:05     ` Jann Horn
2025-09-02 16:13       ` Jann Horn
2025-09-03 14:18       ` Nadav Amit
2025-09-03 14:42         ` Jann Horn
2025-09-02 16:31     ` Jann Horn
2025-09-02 16:57       ` Giovanni Cabiddu
2025-02-26  3:00 ` [PATCH v14 02/13] x86/mm: get INVLPGB count max from CPUID Rik van Riel
2025-02-28 16:21   ` Borislav Petkov
2025-02-28 19:27   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Add INVLPGB feature and Kconfig entry tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 03/13] x86/mm: add INVLPGB support code Rik van Riel
2025-02-28 18:46   ` Borislav Petkov
2025-02-28 18:51   ` Dave Hansen
2025-02-28 19:47   ` Borislav Petkov
2025-03-03 18:41     ` Dave Hansen
2025-03-03 19:23       ` Dave Hansen
2025-03-04 11:00         ` Borislav Petkov
2025-03-04 15:10           ` Dave Hansen
2025-03-04 16:19             ` Borislav Petkov
2025-03-04 16:57               ` Dave Hansen
2025-03-04 21:12                 ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Add " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 04/13] x86/mm: use INVLPGB for kernel TLB flushes Rik van Riel
2025-02-28 19:00   ` Dave Hansen
2025-02-28 21:43   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Use " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 05/13] x86/mm: use INVLPGB in flush_tlb_all Rik van Riel
2025-02-28 19:18   ` Dave Hansen
2025-03-01 12:20     ` Borislav Petkov
2025-03-01 15:54       ` Rik van Riel
2025-02-28 22:20   ` Borislav Petkov
2025-02-26  3:00 ` [PATCH v14 06/13] x86/mm: use broadcast TLB flushing for page reclaim TLB flushing Rik van Riel
2025-02-28 18:57   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Use broadcast TLB flushing in page reclaim tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 07/13] x86/mm: add global ASID allocation helper functions Rik van Riel
2025-03-02  7:06   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Add " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 08/13] x86/mm: global ASID context switch & TLB flush handling Rik van Riel
2025-03-02  7:58   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Handle global ASID context switch and TLB flush tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 09/13] x86/mm: global ASID process exit helpers Rik van Riel
2025-03-02 12:38   ` Borislav Petkov
2025-03-02 13:53     ` Rik van Riel
2025-03-03 10:16       ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Add " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 10/13] x86/mm: enable broadcast TLB invalidation for multi-threaded processes Rik van Riel
2025-03-03 10:57   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] x86/mm: Enable " tip-bot2 for Rik van Riel
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 11/13] x86/mm: do targeted broadcast flushing from tlbbatch code Rik van Riel
2025-03-03 11:46   ` Borislav Petkov
2025-03-03 21:47     ` Dave Hansen
2025-03-04 11:52       ` Borislav Petkov
2025-03-04 15:24         ` Dave Hansen
2025-03-04 12:52       ` Brendan Jackman
2025-03-04 14:11         ` Borislav Petkov
2025-03-04 15:33           ` Brendan Jackman
2025-03-04 17:51             ` Dave Hansen
2025-02-26  3:00 ` [PATCH v14 12/13] x86/mm: enable AMD translation cache extensions Rik van Riel
2025-03-05 21:36   ` tip-bot2 for Rik van Riel [this message]
2025-03-19 11:04   ` [tip: x86/core] x86/mm: Enable " tip-bot2 for Rik van Riel
2025-02-26  3:00 ` [PATCH v14 13/13] x86/mm: only invalidate final translations with INVLPGB Rik van Riel
2025-03-03 22:40   ` Dave Hansen
2025-03-04 11:53     ` Borislav Petkov
2025-03-03 12:42 ` [PATCH v14 00/13] AMD broadcast TLB invalidation Borislav Petkov
2025-03-03 13:29   ` Borislav Petkov
2025-03-04 12:04 ` [PATCH] x86/mm: Always set the ASID valid bit for the INVLPGB instruction Borislav Petkov
2025-03-04 12:43   ` Borislav Petkov
2025-03-05 21:36   ` [tip: x86/mm] " tip-bot2 for Tom Lendacky
2025-03-19 11:04   ` [tip: x86/core] " tip-bot2 for Tom Lendacky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=174121058250.14745.1312436108807852558.tip-bot2@tip-bot2 \
    --to=tip-bot2@linutronix.de \
    --cc=bp@alien8.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=riel@surriel.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®