mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Rick Edgecombe <rick.p.edgecombe@intel.com>
To: pbonzini@redhat.com, seanjc@google.com
Cc: rick.p.edgecombe@intel.com, yan.y.zhao@intel.com,
	isaku.yamahata@gmail.com, kai.huang@intel.com,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	tony.lindgren@linux.intel.com, xiaoyao.li@intel.com,
	reinette.chatre@intel.com,
	Isaku Yamahata <isaku.yamahata@intel.com>
Subject: [PATCH v2 11/25] KVM: TDX: Add placeholders for TDX VM/vCPU structures
Date: Wed, 30 Oct 2024 12:00:24 -0700	[thread overview]
Message-ID: <20241030190039.77971-12-rick.p.edgecombe@intel.com> (raw)
In-Reply-To: <20241030190039.77971-1-rick.p.edgecombe@intel.com>

From: Isaku Yamahata <isaku.yamahata@intel.com>

Add TDX's own VM and vCPU structures as placeholder to manage and run
TDX guests.  Also add helper functions to check whether a VM/vCPU is
TDX or normal VMX one, and add helpers to convert between TDX VM/vCPU
and KVM VM/vCPU.

TDX protects guest VMs from malicious host.  Unlike VMX guests, TDX
guests are crypto-protected.  KVM cannot access TDX guests' memory and
vCPU states directly.  Instead, TDX requires KVM to use a set of TDX
architecture-defined firmware APIs (a.k.a TDX module SEAMCALLs) to
manage and run TDX guests.

In fact, the way to manage and run TDX guests and normal VMX guests are
quite different.  Because of that, the current structures
('struct kvm_vmx' and 'struct vcpu_vmx') to manage VMX guests are not
quite suitable for TDX guests.  E.g., the majority of the members of
'struct vcpu_vmx' don't apply to TDX guests.

Introduce TDX's own VM and vCPU structures ('struct kvm_tdx' and 'struct
vcpu_tdx' respectively) for KVM to manage and run TDX guests.  And
instead of building TDX's VM and vCPU structures based on VMX's, build
them directly based on 'struct kvm'.

As a result, TDX and VMX guests will have different VM size and vCPU
size/alignment.

Currently, kvm_arch_alloc_vm() uses 'kvm_x86_ops::vm_size' to allocate
enough space for the VM structure when creating guest.  With TDX guests,
ideally, KVM should allocate the VM structure based on the VM type so
that the precise size can be allocated for VMX and TDX guests.  But this
requires more extensive code change.  For now, simply choose the maximum
size of 'struct kvm_tdx' and 'struct kvm_vmx' for VM structure
allocation for both VMX and TDX guests.  This would result in small
memory waste for each VM which has smaller VM structure size but this is
acceptable.

For simplicity, use the same way for vCPU allocation too.  Otherwise KVM
would need to maintain a separate 'kvm_vcpu_cache' for each VM type.

Note, updating the 'vt_x86_ops::vm_size' needs to be done before calling
kvm_ops_update(), which copies vt_x86_ops to kvm_x86_ops.  However this
happens before TDX module initialization.  Therefore theoretically it is
possible that 'kvm_x86_ops::vm_size' is set to size of 'struct kvm_tdx'
(when it's larger) but TDX actually fails to initialize at a later time.

Again the worst case of this is wasting couple of bytes memory for each
VM.  KVM could choose to update 'kvm_x86_ops::vm_size' at a later time
depending on TDX's status but that would require base KVM module to
export either kvm_x86_ops or kvm_ops_update().

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
uAPI breakout v2:
 - Correct typo for update (Tony)

uAPI breakout v1:
 - Re-add __always_inline to to_kvm_tdx(), to_tdx(). (Sean)
 - Fix bisectability issues in headers (Kai)
 - Add a comment around updating vt_x86_ops.vm_size.
 - Update the comment around updating vcpu_size/align:
   https://lore.kernel.org/kvm/25d2bf93854ae7410d82119227be3cb2ce47c4f2.camel@intel.com/
 - Refine changelog:
   https://lore.kernel.org/kvm/9c592801471a137c51f583065764fbfc3081c016.camel@intel.com/

v19:
 - correctly update ops.vm_size, vcpu_size and, vcpu_align by Xiaoyao

v14 -> v15:
 - use KVM_X86_TDX_VM
---
 arch/x86/kvm/vmx/main.c | 53 ++++++++++++++++++++++++++++++++++++++---
 arch/x86/kvm/vmx/tdx.c  |  2 +-
 arch/x86/kvm/vmx/tdx.h  | 49 +++++++++++++++++++++++++++++++++++++
 3 files changed, 100 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index 053294939eb1..245f7d1f1bd4 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -8,6 +8,39 @@
 #include "posted_intr.h"
 #include "tdx.h"
 
+static __init int vt_hardware_setup(void)
+{
+	int ret;
+
+	ret = vmx_hardware_setup();
+	if (ret)
+		return ret;
+
+	/*
+	 * Update vt_x86_ops::vm_size here so it is ready before
+	 * kvm_ops_update() is called in kvm_x86_vendor_init().
+	 *
+	 * Note, the actual bringing up of TDX must be done after
+	 * kvm_ops_update() because enabling TDX requires enabling
+	 * hardware virtualization first, i.e., all online CPUs must
+	 * be in post-VMXON state.  This means the @vm_size here
+	 * may be updated to TDX's size but TDX may fail to enable
+	 * at later time.
+	 *
+	 * The VMX/VT code could update kvm_x86_ops::vm_size again
+	 * after bringing up TDX, but this would require exporting
+	 * either kvm_x86_ops or kvm_ops_update() from the base KVM
+	 * module, which looks overkill.  Anyway, the worst case here
+	 * is KVM may allocate couple of more bytes than needed for
+	 * each VM.
+	 */
+	if (enable_tdx)
+		vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size,
+				sizeof(struct kvm_tdx));
+
+	return 0;
+}
+
 #define VMX_REQUIRED_APICV_INHIBITS				\
 	(BIT(APICV_INHIBIT_REASON_DISABLED) |			\
 	 BIT(APICV_INHIBIT_REASON_ABSENT) |			\
@@ -161,7 +194,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
 };
 
 struct kvm_x86_init_ops vt_init_ops __initdata = {
-	.hardware_setup = vmx_hardware_setup,
+	.hardware_setup = vt_hardware_setup,
 	.handle_intel_pt_intr = NULL,
 
 	.runtime_ops = &vt_x86_ops,
@@ -178,6 +211,7 @@ module_exit(vt_exit);
 
 static int __init vt_init(void)
 {
+	unsigned vcpu_size, vcpu_align;
 	int r;
 
 	r = vmx_init();
@@ -187,12 +221,25 @@ static int __init vt_init(void)
 	/* tdx_init() has been taken */
 	tdx_bringup();
 
+	/*
+	 * TDX and VMX have different vCPU structures.  Calculate the
+	 * maximum size/align so that kvm_init() can use the larger
+	 * values to create the kmem_vcpu_cache.
+	 */
+	vcpu_size = sizeof(struct vcpu_vmx);
+	vcpu_align = __alignof__(struct vcpu_vmx);
+	if (enable_tdx) {
+		vcpu_size = max_t(unsigned, vcpu_size,
+				sizeof(struct vcpu_tdx));
+		vcpu_align = max_t(unsigned, vcpu_align,
+				__alignof__(struct vcpu_tdx));
+	}
+
 	/*
 	 * Common KVM initialization _must_ come last, after this, /dev/kvm is
 	 * exposed to userspace!
 	 */
-	r = kvm_init(sizeof(struct vcpu_vmx), __alignof__(struct vcpu_vmx),
-		     THIS_MODULE);
+	r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE);
 	if (r)
 		goto err_kvm_init;
 
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index f95a4dbcaf4a..f2830ff2af1d 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -7,7 +7,7 @@
 #undef pr_fmt
 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
 
-static bool enable_tdx __ro_after_init;
+bool enable_tdx __ro_after_init;
 module_param_named(tdx, enable_tdx, bool, 0444);
 
 static enum cpuhp_state tdx_cpuhp_state;
diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
index 766a6121f670..e6a232d58e6a 100644
--- a/arch/x86/kvm/vmx/tdx.h
+++ b/arch/x86/kvm/vmx/tdx.h
@@ -4,9 +4,58 @@
 #ifdef CONFIG_INTEL_TDX_HOST
 void tdx_bringup(void);
 void tdx_cleanup(void);
+
+extern bool enable_tdx;
+
+struct kvm_tdx {
+	struct kvm kvm;
+	/* TDX specific members follow. */
+};
+
+struct vcpu_tdx {
+	struct kvm_vcpu	vcpu;
+	/* TDX specific members follow. */
+};
+
+static inline bool is_td(struct kvm *kvm)
+{
+	return kvm->arch.vm_type == KVM_X86_TDX_VM;
+}
+
+static inline bool is_td_vcpu(struct kvm_vcpu *vcpu)
+{
+	return is_td(vcpu->kvm);
+}
+
+static __always_inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm)
+{
+	return container_of(kvm, struct kvm_tdx, kvm);
+}
+
+static __always_inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu)
+{
+	return container_of(vcpu, struct vcpu_tdx, vcpu);
+}
+
 #else
 static inline void tdx_bringup(void) {}
 static inline void tdx_cleanup(void) {}
+
+#define enable_tdx	0
+
+struct kvm_tdx {
+	struct kvm kvm;
+};
+
+struct vcpu_tdx {
+	struct kvm_vcpu	vcpu;
+};
+
+static inline bool is_td(struct kvm *kvm) { return false; }
+static inline bool is_td_vcpu(struct kvm_vcpu *vcpu) { return false; }
+static inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm) { return NULL; }
+static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return NULL; }
+
 #endif
 
 #endif
-- 
2.47.0


  parent reply	other threads:[~2024-10-30 19:01 UTC|newest]

Thread overview: 103+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-30 19:00 [PATCH v2 00/25] TDX vCPU/VM creation Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 01/25] x86/virt/tdx: Share the global metadata structure for KVM to use Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 02/25] KVM: TDX: Get TDX global information Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 03/25] x86/virt/tdx: Read essential global metadata for KVM Rick Edgecombe
2024-12-06  8:37   ` Xiaoyao Li
2024-12-06 16:13     ` Huang, Kai
2024-12-06 16:18       ` Huang, Kai
2024-12-06 16:24       ` Dave Hansen
2024-12-07  0:00         ` Huang, Kai
2024-12-12  0:31           ` Edgecombe, Rick P
2024-12-21  1:17             ` Huang, Kai
2024-12-21  1:07   ` [PATCH v2.1 " Kai Huang
2024-10-30 19:00 ` [PATCH v2 04/25] x86/virt/tdx: Add tdx_guest_keyid_alloc/free() to alloc and free TDX guest KeyID Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 05/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX KeyID management Rick Edgecombe
2024-11-12 20:09   ` Dave Hansen
2024-11-14  0:01     ` Edgecombe, Rick P
2024-10-30 19:00 ` [PATCH v2 06/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX TD creation Rick Edgecombe
2024-11-12 20:17   ` Dave Hansen
2024-11-12 21:21     ` Edgecombe, Rick P
2024-11-12 21:40       ` Dave Hansen
2024-10-30 19:00 ` [PATCH v2 07/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX vCPU creation Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 08/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX page cache management Rick Edgecombe
2024-10-31  3:57   ` Yan Zhao
2024-10-31 18:57     ` Edgecombe, Rick P
2024-10-31 23:33       ` Huang, Kai
2024-11-13  0:20   ` Dave Hansen
2024-11-13 20:51     ` Edgecombe, Rick P
2024-11-13 21:08       ` Dave Hansen
2024-11-13 21:25         ` Huang, Kai
2024-11-13 22:01           ` Edgecombe, Rick P
2024-11-13 21:44         ` Edgecombe, Rick P
2024-11-13 21:50           ` Dave Hansen
2024-11-13 22:00             ` Edgecombe, Rick P
2024-11-14  0:21               ` Huang, Kai
2024-11-14  0:32                 ` Edgecombe, Rick P
2024-10-30 19:00 ` [PATCH v2 09/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX VM/vCPU field access Rick Edgecombe
2025-01-05  9:45   ` Francesco Lavra
2025-01-06 18:59     ` Edgecombe, Rick P
2024-10-30 19:00 ` [PATCH v2 10/25] x86/virt/tdx: Add SEAMCALL wrappers for TDX flush operations Rick Edgecombe
2024-11-13  1:11   ` Dave Hansen
2024-11-13 21:18     ` Edgecombe, Rick P
2024-11-13 21:41       ` Dave Hansen
2024-11-13 21:48         ` Edgecombe, Rick P
2024-10-30 19:00 ` Rick Edgecombe [this message]
2025-01-05 10:58   ` [PATCH v2 11/25] KVM: TDX: Add placeholders for TDX VM/vCPU structures Francesco Lavra
2025-01-06 19:00     ` Edgecombe, Rick P
2025-01-22  7:52     ` Tony Lindgren
2024-10-30 19:00 ` [PATCH v2 12/25] KVM: TDX: Define TDX architectural definitions Rick Edgecombe
2024-10-30 22:38   ` Huang, Kai
2024-10-30 22:53     ` Huang, Kai
2024-10-30 19:00 ` [PATCH v2 13/25] KVM: TDX: Add TDX "architectural" error codes Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 14/25] KVM: TDX: Add helper functions to print TDX SEAMCALL error Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 15/25] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 16/25] KVM: TDX: Get system-wide info about TDX module on initialization Rick Edgecombe
2024-10-31  9:09   ` Binbin Wu
2024-10-31  9:18     ` Tony Lindgren
2024-10-31  9:22       ` Binbin Wu
2024-10-31  9:23     ` Xiaoyao Li
2024-10-31  9:37       ` Tony Lindgren
2024-10-31 14:27         ` Xiaoyao Li
2024-11-01  8:19           ` Tony Lindgren
2024-12-06  8:45   ` Xiaoyao Li
2024-12-10  9:35     ` Tony Lindgren
2025-01-08  2:34   ` Chao Gao
2025-01-08  5:41     ` Huang, Kai
2024-10-30 19:00 ` [PATCH v2 17/25] KVM: TDX: create/destroy VM structure Rick Edgecombe
2024-11-04  2:03   ` Chao Gao
2024-11-04  5:59     ` Tony Lindgren
2024-10-30 19:00 ` [PATCH v2 18/25] KVM: TDX: Support per-VM KVM_CAP_MAX_VCPUS extension check Rick Edgecombe
2025-01-05 22:12   ` Huang, Kai
2025-01-06 19:09     ` Edgecombe, Rick P
2024-10-30 19:00 ` [PATCH v2 19/25] KVM: TDX: initialize VM with TDX specific parameters Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 20/25] KVM: TDX: Make pmu_intel.c ignore guest TD case Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 21/25] KVM: TDX: Don't offline the last cpu of one package when there's TDX guest Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 22/25] KVM: TDX: create/free TDX vcpu structure Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 23/25] KVM: TDX: Do TDX specific vcpu initialization Rick Edgecombe
2024-10-30 19:00 ` [PATCH v2 24/25] KVM: x86: Introduce KVM_TDX_GET_CPUID Rick Edgecombe
2024-11-01  6:39   ` Binbin Wu
2024-11-01 16:03     ` Edgecombe, Rick P
2025-01-09 11:07   ` Francesco Lavra
2025-01-10  4:29     ` Xiaoyao Li
2025-01-10 10:34       ` Francesco Lavra
2025-01-10  4:47   ` Xiaoyao Li
2025-01-21 20:24     ` Edgecombe, Rick P
2025-01-22  7:43       ` Xiaoyao Li
2025-01-23 19:44         ` Edgecombe, Rick P
2025-01-21 23:19     ` Edgecombe, Rick P
2024-10-30 19:00 ` [PATCH v2 25/25] KVM: x86/mmu: Taking guest pa into consideration when calculate tdp level Rick Edgecombe
2024-10-31 19:21 ` [PATCH v2 00/25] TDX vCPU/VM creation Adrian Hunter
2024-11-11  9:49   ` Tony Lindgren
2024-11-12  7:26     ` Adrian Hunter
2024-11-12  9:57       ` Tony Lindgren
2024-11-12 21:26   ` Edgecombe, Rick P
2024-12-10 18:22 ` Paolo Bonzini
2024-12-23 16:25 ` Paolo Bonzini
2025-01-04  1:43   ` Edgecombe, Rick P
2025-01-05 21:32     ` Huang, Kai
2025-01-07  7:37     ` Tony Lindgren
2025-01-07 12:41       ` Nikolay Borisov
2025-01-08  5:28         ` Tony Lindgren
2025-01-08 15:01           ` Sean Christopherson
2025-01-09  7:04             ` Tony Lindgren
2025-01-22  8:27     ` Tony Lindgren

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241030190039.77971-12-rick.p.edgecombe@intel.com \
    --to=rick.p.edgecombe@intel.com \
    --cc=isaku.yamahata@gmail.com \
    --cc=isaku.yamahata@intel.com \
    --cc=kai.huang@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=reinette.chatre@intel.com \
    --cc=seanjc@google.com \
    --cc=tony.lindgren@linux.intel.com \
    --cc=xiaoyao.li@intel.com \
    --cc=yan.y.zhao@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®