mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests
@ 2026-09-16  5:05 Zack Rusin
  2026-09-16  5:05 ` [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data Zack Rusin
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Zack Rusin @ 2026-09-16  5:05 UTC (permalink / raw)
  To: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Kiryl Shutsemau, Rick Edgecombe
  Cc: Thomas Gleixner, Ingo Molnar, Dave Hansen, H . Peter Anvin,
	virtualization, bcm-kernel-feedback-list, linux-kernel,
	Christoph Lameter, Andrew Morton, Tom Lendacky, Bo Gan, linux-mm,
	linux-arch, linux-coco, kvm, Zack Rusin

VMware registers each per-CPU steal-time GPA with the host. An encrypted
guest must first convert that storage to shared memory, but the existing
setup publishes the address without conversion.

Patch 1 makes the decrypted per-CPU section available with
CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
encrypted-guest setup until allocator-backed page-table splitting is
available, converts every possible CPU's storage before publishing any
GPA, and attempts to roll back all conversions on failure.

TDX's conversion callback uses __pa(), so patch 2 preflights every possible
CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
per-CPU storage. This covers percpu_alloc=page and automatic allocator
fallback. AMD encrypted guests support those mappings and are not rejected.
Supporting them in TDX would require a separate conversion-API change.

Conversion need not preserve zeroes, and the host initializes only the
8-byte counter. Patch 2 therefore clears each 64-byte object after
conversion and before registration, without disturbing other decrypted
objects that can share its page.

The intended merge path is tip's x86/vmware branch, following
commit ac26963a1175 ("percpu: Introduce DEFINE_PER_CPU_DECRYPTED").
Per-CPU and asm-generic maintainer Acks are requested for patch 1.

This replaces patch 4 of Alexey's v2 posting:
https://lore.kernel.org/all/20260309235250.2611115-1-alexey.makhalov@broadcom.com/

Zack Rusin (2):
  percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data
  x86/vmware: Decrypt steal-time storage before sharing it

 arch/x86/kernel/cpu/vmware.c      | 96 ++++++++++++++++++++++++++++++-
 include/asm-generic/vmlinux.lds.h |  2 +-
 include/linux/percpu-defs.h       |  2 +-
 3 files changed, 97 insertions(+), 3 deletions(-)


base-commit: fd73f4a6659897191fa0d40695fe370925dd3780
-- 
2.53.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data
  2026-09-16  5:05 [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Zack Rusin
@ 2026-09-16  5:05 ` Zack Rusin
  2026-09-16 11:14   ` Kiryl Shutsemau
  2026-09-16  5:05 ` [PATCH v1 2/2] x86/vmware: Decrypt steal-time storage before sharing it Zack Rusin
  2026-09-16 12:18 ` [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Kiryl Shutsemau
  2 siblings, 1 reply; 7+ messages in thread
From: Zack Rusin @ 2026-09-16  5:05 UTC (permalink / raw)
  To: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Kiryl Shutsemau, Rick Edgecombe
  Cc: Thomas Gleixner, Ingo Molnar, Dave Hansen, H . Peter Anvin,
	virtualization, bcm-kernel-feedback-list, linux-kernel,
	Christoph Lameter, Andrew Morton, Tom Lendacky, Bo Gan, linux-mm,
	linux-arch, linux-coco, kvm, Zack Rusin

Decrypted per-CPU data is gated by AMD_MEM_ENCRYPT. With
INTEL_TDX_GUEST=y and AMD_MEM_ENCRYPT=n it falls back to ordinary
per-CPU storage, so converting a variable to shared memory can expose
unrelated data on the same page.

Gate the declarations and linker section on X86_MEM_ENCRYPT, which both
AMD_MEM_ENCRYPT and INTEL_TDX_GUEST select. Existing AMD configurations
are unchanged.

Keep the decrypted subsection page-aligned at both ends. The per-CPU
template and each allocated per-CPU unit are also page-aligned, so page
conversion cannot expose ordinary per-CPU data. TDX-only configurations
may need additional space for this alignment and isolation.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 include/asm-generic/vmlinux.lds.h | 2 +-
 include/linux/percpu-defs.h       | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index b2988aa12f66..1448e791773e 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -1013,7 +1013,7 @@
  * Note: We use a separate section so that only this section gets
  * decrypted to avoid exposing more than we wish.
  */
-#ifdef CONFIG_AMD_MEM_ENCRYPT
+#ifdef CONFIG_X86_MEM_ENCRYPT
 #define PERCPU_DECRYPTED_SECTION					\
 	. = ALIGN(PAGE_SIZE);						\
 	*(.data..percpu..decrypted)					\
diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h
index dbe3267a0a13..54ce1442d918 100644
--- a/include/linux/percpu-defs.h
+++ b/include/linux/percpu-defs.h
@@ -172,7 +172,7 @@
  * Declaration/definition used for per-CPU variables that should be accessed
  * as decrypted when memory encryption is enabled in the guest.
  */
-#ifdef CONFIG_AMD_MEM_ENCRYPT
+#ifdef CONFIG_X86_MEM_ENCRYPT
 #define DECLARE_PER_CPU_DECRYPTED(type, name)				\
 	DECLARE_PER_CPU_SECTION(type, name, "..decrypted")
 
-- 
2.53.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH v1 2/2] x86/vmware: Decrypt steal-time storage before sharing it
  2026-09-16  5:05 [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Zack Rusin
  2026-09-16  5:05 ` [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data Zack Rusin
@ 2026-09-16  5:05 ` Zack Rusin
  2026-09-16 12:18 ` [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Kiryl Shutsemau
  2 siblings, 0 replies; 7+ messages in thread
From: Zack Rusin @ 2026-09-16  5:05 UTC (permalink / raw)
  To: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Kiryl Shutsemau, Rick Edgecombe
  Cc: Thomas Gleixner, Ingo Molnar, Dave Hansen, H . Peter Anvin,
	virtualization, bcm-kernel-feedback-list, linux-kernel,
	Christoph Lameter, Andrew Morton, Tom Lendacky, Bo Gan, linux-mm,
	linux-arch, linux-coco, kvm, Zack Rusin

VMware's steal-time counter must be in shared memory so the host can
update it. Encrypted guests currently register its address without
decrypting the storage. Move their setup to an early initcall, when the
memory allocator is available for page-table splitting, but before
secondary CPUs start. Leave ordinary guests' setup unchanged.

Convert every possible CPU's storage before registering any address.
Zero each object after conversion, since its contents may not survive
conversion and the host initializes only the counter. Register the boot
CPU with preemption disabled; the existing hotplug callbacks handle the
others. If conversion or boot-CPU registration fails, attempt to re-encrypt
all affected ranges, including a partially converted failing range, and
disable steal time.

TDX's conversion path requires directly mapped memory. Check all per-CPU
objects before converting anything, and disable steal time if a TDX guest
uses the page per-CPU allocator, which supplies vmalloc mappings. This
also covers automatic fallback from the embedded allocator. AMD guests
are unaffected by this restriction because their conversion path supports
those mappings.

Link: https://lore.kernel.org/r/20260309235250.2611115-5-alexey.makhalov@broadcom.com
Co-developed-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Bo Gan <bo.gan@broadcom.com>
Co-developed-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/kernel/cpu/vmware.c | 96 +++++++++++++++++++++++++++++++++++-
 1 file changed, 95 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index 34b73573b108..f49898275d87 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -25,14 +25,19 @@
 #include <linux/init.h>
 #include <linux/export.h>
 #include <linux/clocksource.h>
+#include <linux/cc_platform.h>
 #include <linux/cpu.h>
 #include <linux/efi.h>
+#include <linux/mm.h>
+#include <linux/preempt.h>
 #include <linux/reboot.h>
+#include <linux/set_memory.h>
 #include <linux/static_call.h>
 #include <linux/sched/cputime.h>
 #include <asm/div64.h>
 #include <asm/x86_init.h>
 #include <asm/hypervisor.h>
+#include <asm/cpufeature.h>
 #include <asm/cpuid/api.h>
 #include <asm/timer.h>
 #include <asm/apic.h>
@@ -147,6 +152,7 @@ static struct cyc2ns_data vmware_cyc2ns __ro_after_init;
 static bool vmw_sched_clock __initdata = true;
 static DEFINE_PER_CPU_DECRYPTED(struct vmware_steal_time, vmw_steal_time) __aligned(64);
 static bool has_steal_clock;
+static bool vmw_steal_time_ready;
 static bool steal_acc __initdata = true; /* steal time accounting */
 
 static __init int setup_vmw_sched_clock(char *s)
@@ -280,7 +286,9 @@ static void vmware_disable_steal_time(void)
 
 static void vmware_guest_cpu_init(void)
 {
-	if (has_steal_clock)
+	if (has_steal_clock &&
+	    (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) ||
+	     vmw_steal_time_ready))
 		vmware_register_steal_time();
 }
 
@@ -325,6 +333,92 @@ static int vmware_cpu_down_prepare(unsigned int cpu)
 }
 #endif
 
+static void __init vmware_steal_time_range(int cpu, unsigned long *start,
+					   int *numpages)
+{
+	unsigned long addr = (unsigned long)per_cpu_ptr(&vmw_steal_time, cpu);
+
+	*start = addr & PAGE_MASK;
+	*numpages = DIV_ROUND_UP(offset_in_page(addr) +
+				 sizeof(struct vmware_steal_time), PAGE_SIZE);
+}
+
+static int __init vmware_decrypt_steal_time(void)
+{
+	int cpu, failed_cpu, numpages, ret;
+	unsigned long start;
+
+	if (!has_steal_clock ||
+	    !cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT))
+		return 0;
+
+	/*
+	 * TDX's conversion callback derives the physical range with __pa(),
+	 * so the storage must live in the direct map. The page per-CPU
+	 * allocator, which is also the automatic fallback when the embedding
+	 * allocator fails, hands out vmalloc addresses instead. Reject those
+	 * before converting anything, so no page is left converted.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) {
+		for_each_possible_cpu(cpu) {
+			if (!is_vmalloc_addr(per_cpu_ptr(&vmw_steal_time, cpu)))
+				continue;
+
+			pr_warn("steal time disabled: TDX requires a direct mapping\n");
+			has_steal_clock = false;
+			return 0;
+		}
+	}
+
+	for_each_possible_cpu(cpu) {
+		vmware_steal_time_range(cpu, &start, &numpages);
+		ret = set_memory_decrypted(start, numpages);
+		if (ret) {
+			failed_cpu = cpu;
+			goto rollback;
+		}
+
+		/*
+		 * Conversion need not preserve the zeroes. The host
+		 * initializes only the counter on enable, so the guest
+		 * must initialize the reserved words itself.
+		 */
+		memset(per_cpu_ptr(&vmw_steal_time, cpu), 0,
+		       sizeof(struct vmware_steal_time));
+	}
+
+	vmw_steal_time_ready = true;
+	preempt_disable();
+	vmware_guest_cpu_init();
+	preempt_enable();
+	if (!has_steal_clock) {
+		pr_warn("failed to register boot CPU steal-time memory\n");
+		failed_cpu = nr_cpu_ids;
+		goto rollback_pages;
+	}
+	return 0;
+
+rollback:
+	pr_warn("failed to decrypt steal-time memory for CPU %d: %d\n",
+		failed_cpu, ret);
+
+rollback_pages:
+	for_each_possible_cpu(cpu) {
+		vmware_steal_time_range(cpu, &start, &numpages);
+		ret = set_memory_encrypted(start, numpages);
+		if (ret)
+			pr_warn("failed to re-encrypt steal-time memory for CPU %d: %d\n",
+				cpu, ret);
+		if (cpu == failed_cpu)
+			break;
+	}
+
+	vmw_steal_time_ready = false;
+	has_steal_clock = false;
+	return 0;
+}
+early_initcall(vmware_decrypt_steal_time);
+
 static __init int activate_jump_labels(void)
 {
 	if (has_steal_clock) {
-- 
2.53.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data
  2026-09-16  5:05 ` [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data Zack Rusin
@ 2026-09-16 11:14   ` Kiryl Shutsemau
  0 siblings, 0 replies; 7+ messages in thread
From: Kiryl Shutsemau @ 2026-09-16 11:14 UTC (permalink / raw)
  To: Zack Rusin
  Cc: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Rick Edgecombe, Thomas Gleixner,
	Ingo Molnar, Dave Hansen, H . Peter Anvin, virtualization,
	bcm-kernel-feedback-list, linux-kernel, Christoph Lameter,
	Andrew Morton, Tom Lendacky, Bo Gan, linux-mm, linux-arch,
	linux-coco, kvm

On Wed, Sep 16, 2026 at 01:05:39AM -0400, Zack Rusin wrote:
> Decrypted per-CPU data is gated by AMD_MEM_ENCRYPT. With
> INTEL_TDX_GUEST=y and AMD_MEM_ENCRYPT=n it falls back to ordinary
> per-CPU storage, so converting a variable to shared memory can expose
> unrelated data on the same page.
> 
> Gate the declarations and linker section on X86_MEM_ENCRYPT, which both
> AMD_MEM_ENCRYPT and INTEL_TDX_GUEST select. Existing AMD configurations
> are unchanged.
> 
> Keep the decrypted subsection page-aligned at both ends. The per-CPU
> template and each allocated per-CPU unit are also page-aligned, so page
> conversion cannot expose ordinary per-CPU data. TDX-only configurations
> may need additional space for this alignment and isolation.
> 
> Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
> ---
>  include/asm-generic/vmlinux.lds.h | 2 +-
>  include/linux/percpu-defs.h       | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
> index b2988aa12f66..1448e791773e 100644
> --- a/include/asm-generic/vmlinux.lds.h
> +++ b/include/asm-generic/vmlinux.lds.h
> @@ -1013,7 +1013,7 @@
>   * Note: We use a separate section so that only this section gets
>   * decrypted to avoid exposing more than we wish.
>   */
> -#ifdef CONFIG_AMD_MEM_ENCRYPT
> +#ifdef CONFIG_X86_MEM_ENCRYPT
>  #define PERCPU_DECRYPTED_SECTION					\
>  	. = ALIGN(PAGE_SIZE);						\
>  	*(.data..percpu..decrypted)					\
> diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h
> index dbe3267a0a13..54ce1442d918 100644
> --- a/include/linux/percpu-defs.h
> +++ b/include/linux/percpu-defs.h
> @@ -172,7 +172,7 @@
>   * Declaration/definition used for per-CPU variables that should be accessed
>   * as decrypted when memory encryption is enabled in the guest.
>   */
> -#ifdef CONFIG_AMD_MEM_ENCRYPT
> +#ifdef CONFIG_X86_MEM_ENCRYPT
>  #define DECLARE_PER_CPU_DECRYPTED(type, name)				\
>  	DECLARE_PER_CPU_SECTION(type, name, "..decrypted")

DECLARE_PER_CPU_DECRYPTED() is not used anywhere. Do we want to drop it?

Otherwise:

Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests
  2026-09-16  5:05 [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Zack Rusin
  2026-09-16  5:05 ` [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data Zack Rusin
  2026-09-16  5:05 ` [PATCH v1 2/2] x86/vmware: Decrypt steal-time storage before sharing it Zack Rusin
@ 2026-09-16 12:18 ` Kiryl Shutsemau
  2026-09-16 15:41   ` Zack Rusin
  2 siblings, 1 reply; 7+ messages in thread
From: Kiryl Shutsemau @ 2026-09-16 12:18 UTC (permalink / raw)
  To: Zack Rusin
  Cc: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Rick Edgecombe, Thomas Gleixner,
	Ingo Molnar, Dave Hansen, H . Peter Anvin, virtualization,
	bcm-kernel-feedback-list, linux-kernel, Christoph Lameter,
	Andrew Morton, Tom Lendacky, Bo Gan, linux-mm, linux-arch,
	linux-coco, kvm

On Wed, Sep 16, 2026 at 01:05:38AM -0400, Zack Rusin wrote:
> VMware registers each per-CPU steal-time GPA with the host. An encrypted
> guest must first convert that storage to shared memory, but the existing
> setup publishes the address without conversion.
> 
> Patch 1 makes the decrypted per-CPU section available with
> CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
> encrypted-guest setup until allocator-backed page-table splitting is
> available, converts every possible CPU's storage before publishing any
> GPA, and attempts to roll back all conversions on failure.

I acked 1/2, but I don't like where 2/2 does the conversion.

A variable declared with DEFINE_PER_CPU_DECRYPTED() ends up in a
dedicated, page-aligned linker section. The point of the section is that
one place converts it. Instead every user does it itself: KVM in
sev_map_percpu_data(), and now VMware in vmware_decrypt_steal_time(),
each with its own vendor checks and failure handling. The macro today
only buys page isolation, not the shared mapping its name promises.

The underlying problem is that the whole "decrypted section"
infrastructure is built around SME/SEV and was never generalized.
__bss_decrypted, early_set_memory_decrypted() and
mem_encrypt_free_decrypted_mem() are all under CONFIG_AMD_MEM_ENCRYPT
and implemented in mem_encrypt_amd.c. sme_postprocess_startup() converts
.bss..decrypted only if sme_get_me_mask() is set, so a TDX guest never
shares it. 1/2 moves the per-CPU linker section to X86_MEM_ENCRYPT, but
nothing that would act on that section follows.

Rather than have every TDX user reinvent the conversion, I would rather
see the infrastructure made vendor-neutral: boundary symbols for the
per-CPU decrypted section like the ones .bss..decrypted has, an early
conversion primitive that works on TDX as well as SEV, and a single
conversion of both sections at boot. Then sev_map_percpu_data() and
this driver's loop go away.

> TDX's conversion callback uses __pa(), so patch 2 preflights every possible
> CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
> per-CPU storage. This covers percpu_alloc=page and automatic allocator
> fallback. AMD encrypted guests support those mappings and are not rejected.
> Supporting them in TDX would require a separate conversion-API change.

Refusing vmalloc-backed storage on TDX is the right call, and not because
of the __pa() in the callback. Converting a vmalloc alias means either
fracturing the direct map or leaving a private direct-map alias to a
shared page, and the latter is a guest shutdown the moment
load_unaligned_zeropad() steps into it. See the comment in
tdx_early_init() and the earlier discussion of the same idea:

https://lore.kernel.org/all/xqi2bkulnhen2vax5msbzczlaywx3dsc7ezpn7oo5qn7u7xzap@xmaseinov7tf/

But that decision belongs in the same central place as the conversion.
If the first per-CPU chunk is vmalloc-backed on TDX, the decrypted
section is simply not shared, and users see that, instead of every
driver re-deriving it.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests
  2026-09-16 12:18 ` [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Kiryl Shutsemau
@ 2026-09-16 15:41   ` Zack Rusin
  2026-09-17 14:11     ` Kiryl Shutsemau
  0 siblings, 1 reply; 7+ messages in thread
From: Zack Rusin @ 2026-09-16 15:41 UTC (permalink / raw)
  To: Kiryl Shutsemau
  Cc: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Rick Edgecombe, Thomas Gleixner,
	Ingo Molnar, Dave Hansen, H . Peter Anvin, virtualization,
	bcm-kernel-feedback-list, linux-kernel, Christoph Lameter,
	Andrew Morton, Tom Lendacky, Bo Gan, linux-mm, linux-arch,
	linux-coco, kvm

[-- Attachment #1: Type: text/plain, Size: 5443 bytes --]

On Wed, Sep 16, 2026 at 8:18 AM Kiryl Shutsemau <kas@kernel.org> wrote:
>
> On Wed, Sep 16, 2026 at 01:05:38AM -0400, Zack Rusin wrote:
> > VMware registers each per-CPU steal-time GPA with the host. An encrypted
> > guest must first convert that storage to shared memory, but the existing
> > setup publishes the address without conversion.
> >
> > Patch 1 makes the decrypted per-CPU section available with
> > CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
> > encrypted-guest setup until allocator-backed page-table splitting is
> > available, converts every possible CPU's storage before publishing any
> > GPA, and attempts to roll back all conversions on failure.
>
> I acked 1/2, but I don't like where 2/2 does the conversion.

I saw that, thank you. Since I'll be respinning v2 of this I can go
ahead and drop the unused DECLARE_PER_CPU_DECRYPTED in that one if you
want as well.

> A variable declared with DEFINE_PER_CPU_DECRYPTED() ends up in a
> dedicated, page-aligned linker section. The point of the section is that
> one place converts it. Instead every user does it itself: KVM in
> sev_map_percpu_data(), and now VMware in vmware_decrypt_steal_time(),
> each with its own vendor checks and failure handling. The macro today
> only buys page isolation, not the shared mapping its name promises.
>
> The underlying problem is that the whole "decrypted section"
> infrastructure is built around SME/SEV and was never generalized.
> __bss_decrypted, early_set_memory_decrypted() and
> mem_encrypt_free_decrypted_mem() are all under CONFIG_AMD_MEM_ENCRYPT
> and implemented in mem_encrypt_amd.c. sme_postprocess_startup() converts
> .bss..decrypted only if sme_get_me_mask() is set, so a TDX guest never
> shares it. 1/2 moves the per-CPU linker section to X86_MEM_ENCRYPT, but
> nothing that would act on that section follows.
>
> Rather than have every TDX user reinvent the conversion, I would rather
> see the infrastructure made vendor-neutral: boundary symbols for the
> per-CPU decrypted section like the ones .bss..decrypted has, an early
> conversion primitive that works on TDX as well as SEV, and a single
> conversion of both sections at boot. Then sev_map_percpu_data() and
> this driver's loop go away.
>
> > TDX's conversion callback uses __pa(), so patch 2 preflights every possible
> > CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
> > per-CPU storage. This covers percpu_alloc=page and automatic allocator
> > fallback. AMD encrypted guests support those mappings and are not rejected.
> > Supporting them in TDX would require a separate conversion-API change.
>
> Refusing vmalloc-backed storage on TDX is the right call, and not because
> of the __pa() in the callback. Converting a vmalloc alias means either
> fracturing the direct map or leaving a private direct-map alias to a
> shared page, and the latter is a guest shutdown the moment
> load_unaligned_zeropad() steps into it. See the comment in
> tdx_early_init() and the earlier discussion of the same idea:
>
> https://lore.kernel.org/all/xqi2bkulnhen2vax5msbzczlaywx3dsc7ezpn7oo5qn7u7xzap@xmaseinov7tf/
>
> But that decision belongs in the same central place as the conversion.
> If the first per-CPU chunk is vmalloc-backed on TDX, the decrypted
> section is simply not shared, and users see that, instead of every
> driver re-deriving it.

I went through the linked thread, including the dropped 2022
Quote-buffer alias approach and the 2024 netvsc work's kexec
regression, fix and re-review discussion. I will keep TDX vmalloc
support out of this series and correct the explanation to cover alias
consistency, direct-map splitting and load_unaligned_zeropad(),
including removing the suggestion that per-page GPA lookup alone
resolves it.

After tracing the boot paths, I think there are two distinct pieces
here. The SEV boot GHCB needs .bss..decrypted before BSS is cleared,
whereas the SMP per-CPU instances do not exist until
setup_per_cpu_areas(). KVM then registers its boot-CPU per-CPU buffers
in smp_prepare_boot_cpu(), so the common conversion needs the early
primitive you proposed. A shared early_initcall would be too late for
KVM. The UP registration path probably also needs handling separately

My suggestion would be to do one of two things (or at least I think
those two options are realistic to me, given that I'm no expert here):
- add the central per-CPU infrastructure and migrate KVM/VMware
together, removing both loops and putting the TDX vmalloc rejection
and readiness state in common code. Would you and the KVM maintainers
be happy with that first, and generalizing .bss..decrypted separately
while preserving its earliest SEV setup? afaict the latter also needs
to broaden the annotation and linker guards, and changes kvmclock
storage and sharing on TDX-only builds.
- a smaller VMware fix using a separately allocated, direct-mapped
page per CPU while the common infrastructure is developed. Using the
normal allocator would also defer ordinary-guest registration, unless
I kept their current static storage as a second path. This option
would retain caller-managed sharing and buffer lifetime handling.

The second option is, of course, easier for me, but I'm happy to do
the first to get steal-time storage working for us. Do you have any
thoughts on this?

z

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5414 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests
  2026-09-16 15:41   ` Zack Rusin
@ 2026-09-17 14:11     ` Kiryl Shutsemau
  0 siblings, 0 replies; 7+ messages in thread
From: Kiryl Shutsemau @ 2026-09-17 14:11 UTC (permalink / raw)
  To: Zack Rusin
  Cc: Borislav Petkov, Ajay Kaher, Alexey Makhalov, x86, Dennis Zhou,
	Tejun Heo, Arnd Bergmann, Rick Edgecombe, Thomas Gleixner,
	Ingo Molnar, Dave Hansen, H . Peter Anvin, virtualization,
	bcm-kernel-feedback-list, linux-kernel, Christoph Lameter,
	Andrew Morton, Tom Lendacky, Bo Gan, linux-mm, linux-arch,
	linux-coco, kvm

On Wed, Sep 16, 2026 at 11:41:19AM -0400, Zack Rusin wrote:
> > I acked 1/2, but I don't like where 2/2 does the conversion.
> 
> I saw that, thank you. Since I'll be respinning v2 of this I can go
> ahead and drop the unused DECLARE_PER_CPU_DECRYPTED in that one if you
> want as well.

Yes, please.

> I went through the linked thread, including the dropped 2022
> Quote-buffer alias approach and the 2024 netvsc work's kexec
> regression, fix and re-review discussion. I will keep TDX vmalloc
> support out of this series and correct the explanation to cover alias
> consistency, direct-map splitting and load_unaligned_zeropad(),
> including removing the suggestion that per-page GPA lookup alone
> resolves it.
> 
> After tracing the boot paths, I think there are two distinct pieces
> here. The SEV boot GHCB needs .bss..decrypted before BSS is cleared,
> whereas the SMP per-CPU instances do not exist until
> setup_per_cpu_areas(). KVM then registers its boot-CPU per-CPU buffers
> in smp_prepare_boot_cpu(), so the common conversion needs the early
> primitive you proposed. A shared early_initcall would be too late for
> KVM. The UP registration path probably also needs handling separately

Agreed on all of that. Leave .bss..decrypted alone in this series; the
SEV boot GHCB pins its conversion to the startup code and nothing on
TDX needs it today.

With the primitive called before smp_prepare_boot_cpu() there is no
readiness state to track.

For the vmalloc-backed case, I would rather not handle it at all. Force
the embed allocator when memory encryption is on: ignore
percpu_alloc=page with a warning and don't fall back to page mode if
embed fails, just let it hit the existing panic. On 64-bit embed only
fails on memblock exhaustion at boot or when the NUMA groups spread over
more than 3/4 of vmalloc space. Neither happens in a guest.

> My suggestion would be to do one of two things (or at least I think
> those two options are realistic to me, given that I'm no expert here):
> - add the central per-CPU infrastructure and migrate KVM/VMware
> together, removing both loops and putting the TDX vmalloc rejection
> and readiness state in common code. Would you and the KVM maintainers
> be happy with that first, and generalizing .bss..decrypted separately
> while preserving its earliest SEV setup? afaict the latter also needs
> to broaden the annotation and linker guards, and changes kvmclock
> storage and sharing on TDX-only builds.
> - a smaller VMware fix using a separately allocated, direct-mapped
> page per CPU while the common infrastructure is developed. Using the
> normal allocator would also defer ordinary-guest registration, unless
> I kept their current static storage as a second path. This option
> would retain caller-managed sharing and buffer lifetime handling.
> 
> The second option is, of course, easier for me, but I'm happy to do
> the first to get steal-time storage working for us. Do you have any
> thoughts on this?

The first one, please. It is the better long-term shape and it removes
the KVM loop as well, so the per-CPU section finally does what its
name says.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2026-09-17 14:11 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-16  5:05 [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Zack Rusin
2026-09-16  5:05 ` [PATCH v1 1/2] percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data Zack Rusin
2026-09-16 11:14   ` Kiryl Shutsemau
2026-09-16  5:05 ` [PATCH v1 2/2] x86/vmware: Decrypt steal-time storage before sharing it Zack Rusin
2026-09-16 12:18 ` [PATCH v1 0/2] x86/vmware: Share steal-time storage in encrypted guests Kiryl Shutsemau
2026-09-16 15:41   ` Zack Rusin
2026-09-17 14:11     ` Kiryl Shutsemau

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®