mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Chao Gao <chao.gao@intel.com>
To: linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org,
	kvm@vger.kernel.org, x86@kernel.org
Cc: reinette.chatre@intel.com, ira.weiny@intel.com,
	kai.huang@intel.com, dan.j.williams@intel.com,
	yilun.xu@linux.intel.com, sagis@google.com,
	vannapurve@google.com, paulmck@kernel.org, nik.borisov@suse.com,
	zhenzhong.duan@intel.com, seanjc@google.com,
	rick.p.edgecombe@intel.com, kas@kernel.org,
	dave.hansen@linux.intel.com, vishal.l.verma@intel.com,
	Chao Gao <chao.gao@intel.com>,
	Farrah Chen <farrah.chen@intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH v3 14/26] x86/virt/seamldr: Introduce skeleton for TDX Module updates
Date: Fri, 23 Jan 2026 06:55:22 -0800	[thread overview]
Message-ID: <20260123145645.90444-15-chao.gao@intel.com> (raw)
In-Reply-To: <20260123145645.90444-1-chao.gao@intel.com>

The P-SEAMLDR requires that no TDX Module SEAMCALLs are invoked during a
runtime TDX Module update.

But currently, TDX Module SEAMCALLs are invoked in various contexts and in
parallel across CPUs. Additionally, considering the need to force all vCPUs
out of guest mode, no single lock primitive, except for stop_machine(), can
meet this requirement.

Perform TDX Module updates within stop_machine() as it achieves the
P-SEAMLDR requirements and is an existing well understood mechanism.

TDX Module updates consist of several steps: shutting down the old
module, installing the new module, and initializing the new one and etc.
Some steps must be executed on a single CPU, others serially across all
CPUs, and some can be performed concurrently on all CPUs and there are
ordering requirements between steps. So, all CPUs need to perform the work
in a step-locked manner.

In preparation for adding concrete steps for TDX Module updates,
establish the framework by mimicking multi_cpu_stop(). Specifically, use a
global state machine to control the work done on each CPU and require all
CPUs to acknowledge completion before proceeding to the next stage.

Potential alternative to stop_machine()
=======================================
An alternative approach is to lock all KVM entry points and kick all
vCPUs.  Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
implemented in KVM common code (virt/kvm). Adding a locking mechanism
there would affect all architectures. And to lock only TDX vCPUs, new
logic would be needed to identify TDX vCPUs, which the common code
currently lacks. This would add significant complexity and maintenance
overhead for a TDX-specific use case.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Tested-by: Farrah Chen <farrah.chen@intel.com>
---
v2:
 - refine the changlog to follow context-problem-solution structure
 - move alternative discussions at the end of the changelog
 - add a comment about state machine transition
 - Move rcu_momentary_eqs() call to the else branch.
---
 arch/x86/virt/vmx/tdx/seamldr.c | 71 ++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)

diff --git a/arch/x86/virt/vmx/tdx/seamldr.c b/arch/x86/virt/vmx/tdx/seamldr.c
index d136ef89cd36..06080c648b02 100644
--- a/arch/x86/virt/vmx/tdx/seamldr.c
+++ b/arch/x86/virt/vmx/tdx/seamldr.c
@@ -11,7 +11,9 @@
 #include <linux/cpumask.h>
 #include <linux/irqflags.h>
 #include <linux/mm.h>
+#include <linux/nmi.h>
 #include <linux/slab.h>
+#include <linux/stop_machine.h>
 #include <linux/types.h>
 
 #include <asm/seamldr.h>
@@ -223,6 +225,68 @@ static struct seamldr_params *init_seamldr_params(const u8 *data, u32 size)
 	return alloc_seamldr_params(module, module_size, sig, sig_size);
 }
 
+/*
+ * During a TDX Module update, all CPUs start from TDP_START and progress
+ * to TDP_DONE. Each state is associated with certain work. For some
+ * states, just one CPU needs to perform the work, while other CPUs just
+ * wait during those states.
+ */
+enum tdp_state {
+	TDP_START,
+	TDP_DONE,
+};
+
+static struct {
+	enum tdp_state state;
+	atomic_t thread_ack;
+} tdp_data;
+
+static void set_target_state(enum tdp_state state)
+{
+	/* Reset ack counter. */
+	atomic_set(&tdp_data.thread_ack, num_online_cpus());
+	/* Ensure thread_ack is updated before the new state */
+	smp_wmb();
+	WRITE_ONCE(tdp_data.state, state);
+}
+
+/* Last one to ack a state moves to the next state. */
+static void ack_state(void)
+{
+	if (atomic_dec_and_test(&tdp_data.thread_ack))
+		set_target_state(tdp_data.state + 1);
+}
+
+/*
+ * See multi_cpu_stop() from where this multi-cpu state-machine was
+ * adopted, and the rationale for touch_nmi_watchdog()
+ */
+static int do_seamldr_install_module(void *params)
+{
+	enum tdp_state newstate, curstate = TDP_START;
+	int ret = 0;
+
+	do {
+		/* Chill out and ensure we re-read tdp_data. */
+		cpu_relax();
+		newstate = READ_ONCE(tdp_data.state);
+
+		if (newstate != curstate) {
+			curstate = newstate;
+			switch (curstate) {
+			default:
+				break;
+			}
+			ack_state();
+		} else {
+			touch_nmi_watchdog();
+			rcu_momentary_eqs();
+		}
+	} while (curstate != TDP_DONE);
+
+	return ret;
+}
+
 DEFINE_FREE(free_seamldr_params, struct seamldr_params *,
 	    if (!IS_ERR_OR_NULL(_T)) free_seamldr_params(_T))
 
@@ -237,6 +301,7 @@ DEFINE_FREE(free_seamldr_params, struct seamldr_params *,
 int seamldr_install_module(const u8 *data, u32 size)
 {
 	const struct seamldr_info *info = seamldr_get_info();
+	int ret;
 
 	if (!info)
 		return -EIO;
@@ -258,7 +323,11 @@ int seamldr_install_module(const u8 *data, u32 size)
 		return -EBUSY;
 	}
 
-	/* TODO: Update TDX Module here */
+	set_target_state(TDP_START + 1);
+	ret = stop_machine_cpuslocked(do_seamldr_install_module, params, cpu_online_mask);
+	if (ret)
+		return ret;
+
 	return 0;
 }
 EXPORT_SYMBOL_FOR_MODULES(seamldr_install_module, "tdx-host");
-- 
2.47.3


  parent reply	other threads:[~2026-01-23 15:00 UTC|newest]

Thread overview: 132+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-23 14:55 [PATCH v3 00/26] Runtime TDX Module update support Chao Gao
2026-01-23 14:55 ` [PATCH v3 01/26] x86/virt/tdx: Print SEAMCALL leaf numbers in decimal Chao Gao
2026-01-26 10:01   ` Tony Lindgren
2026-01-28  1:28   ` Binbin Wu
2026-01-28 16:26   ` Dave Hansen
2026-01-29  5:44     ` Chao Gao
2026-01-23 14:55 ` [PATCH v3 02/26] x86/virt/tdx: Use %# prefix for hex values in SEAMCALL error messages Chao Gao
2026-01-26 10:02   ` Tony Lindgren
2026-01-28  1:34   ` Binbin Wu
2026-01-28 12:16     ` Chao Gao
2026-01-28 15:18   ` Dave Hansen
2026-01-23 14:55 ` [PATCH v3 03/26] x86/virt/tdx: Move low level SEAMCALL helpers out of <asm/tdx.h> Chao Gao
2026-01-26 10:02   ` Tony Lindgren
2026-01-28  1:37   ` Binbin Wu
2026-01-28 12:42     ` Chao Gao
2026-01-28 16:31       ` Dave Hansen
2026-01-29 14:02         ` Chao Gao
2026-01-29 16:03           ` Dave Hansen
2026-01-28 16:37   ` Dave Hansen
2026-01-29  8:04     ` Chao Gao
2026-01-23 14:55 ` [PATCH v3 04/26] coco/tdx-host: Introduce a "tdx_host" device Chao Gao
2026-01-26  9:52   ` Tony Lindgren
2026-01-28 16:53     ` Dave Hansen
2026-01-28  3:24   ` Binbin Wu
2026-01-29  7:26     ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 05/26] coco/tdx-host: Expose TDX Module version Chao Gao
2026-01-26  9:54   ` Tony Lindgren
2026-01-28  3:48   ` Binbin Wu
2026-01-28 17:01   ` Dave Hansen
2026-01-29 14:07     ` Chao Gao
2026-01-29  7:38   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 06/26] x86/virt/tdx: Prepare to support P-SEAMLDR SEAMCALLs Chao Gao
2026-01-26 10:05   ` Tony Lindgren
2026-01-28  5:58   ` Binbin Wu
2026-01-28 23:03   ` Dave Hansen
2026-01-29  9:46     ` Xu Yilun
2026-01-29 16:08       ` Dave Hansen
2026-01-29 14:55     ` Chao Gao
2026-01-29 16:59       ` Dave Hansen
2026-01-23 14:55 ` [PATCH v3 07/26] x86/virt/seamldr: Introduce a wrapper for " Chao Gao
2026-01-26 10:12   ` Tony Lindgren
2026-01-28  6:38   ` Binbin Wu
2026-01-28 23:04   ` Dave Hansen
2026-01-30  8:08     ` Chao Gao
2026-01-30 16:23       ` Dave Hansen
2026-01-28 23:36   ` Dave Hansen
2026-01-30 13:21     ` Chao Gao
2026-01-30 16:18       ` Dave Hansen
2026-02-03 12:15         ` Chao Gao
2026-02-03 15:41           ` Sean Christopherson
2026-02-03 16:12             ` Dave Hansen
2026-02-03 23:54             ` Chao Gao
2026-02-05 16:29               ` Sean Christopherson
2026-02-05 16:37                 ` Dave Hansen
2026-01-23 14:55 ` [PATCH v3 08/26] x86/virt/seamldr: Retrieve P-SEAMLDR information Chao Gao
2026-01-26 10:15   ` Tony Lindgren
2026-01-28  6:50   ` Binbin Wu
2026-01-28 23:54   ` Dave Hansen
2026-01-30  4:01     ` Xu Yilun
2026-01-30 16:35       ` Dave Hansen
2026-02-02  0:16         ` Xu Yilun
2026-01-30 13:55     ` Chao Gao
2026-01-30 16:06       ` Dave Hansen
2026-01-28 23:57   ` Dave Hansen
2026-01-30 13:30     ` Chao Gao
2026-01-23 14:55 ` [PATCH v3 09/26] coco/tdx-host: Expose P-SEAMLDR information via sysfs Chao Gao
2026-01-26  9:56   ` Tony Lindgren
2026-01-28  3:07   ` Huang, Kai
2026-01-29  0:08   ` Dave Hansen
2026-01-30 14:44     ` Chao Gao
2026-01-30 16:02       ` Dave Hansen
2026-01-23 14:55 ` [PATCH v3 10/26] coco/tdx-host: Implement FW_UPLOAD sysfs ABI for TDX Module updates Chao Gao
2026-01-26 10:00   ` Tony Lindgren
2026-01-28  3:30   ` Huang, Kai
2026-01-30 14:07   ` Xu Yilun
2026-02-06 17:15   ` Xing, Cedric
2026-01-23 14:55 ` [PATCH v3 11/26] x86/virt/seamldr: Block TDX Module updates if any CPU is offline Chao Gao
2026-01-26 10:16   ` Tony Lindgren
2026-02-02  0:31   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 12/26] x86/virt/seamldr: Verify availability of slots for TDX Module updates Chao Gao
2026-01-26 10:17   ` Tony Lindgren
2026-01-23 14:55 ` [PATCH v3 13/26] x86/virt/seamldr: Allocate and populate a module update request Chao Gao
2026-01-26 10:23   ` Tony Lindgren
2026-01-27  3:21   ` Huang, Kai
2026-01-28 11:28     ` Chao Gao
2026-01-28 22:33       ` Huang, Kai
2026-01-28  4:03   ` Huang, Kai
2026-01-30 14:56     ` Chao Gao
2026-02-02  3:08   ` Xu Yilun
2026-01-23 14:55 ` Chao Gao [this message]
2026-01-26 10:28   ` [PATCH v3 14/26] x86/virt/seamldr: Introduce skeleton for TDX Module updates Tony Lindgren
2026-02-02  6:01   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 15/26] x86/virt/seamldr: Abort updates if errors occurred midway Chao Gao
2026-01-26 10:31   ` Tony Lindgren
2026-02-02  6:08   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 16/26] x86/virt/seamldr: Shut down the current TDX module Chao Gao
2026-01-26 10:42   ` Tony Lindgren
2026-02-02  6:31   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 17/26] x86/virt/tdx: Reset software states after TDX module shutdown Chao Gao
2026-01-26 10:43   ` Tony Lindgren
2026-01-23 14:55 ` [PATCH v3 18/26] x86/virt/seamldr: Log TDX Module update failures Chao Gao
2026-01-26 10:45   ` Tony Lindgren
2026-02-02  7:11   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 19/26] x86/virt/seamldr: Install a new TDX Module Chao Gao
2026-01-26 10:52   ` Tony Lindgren
2026-01-23 14:55 ` [PATCH v3 20/26] x86/virt/seamldr: Do TDX per-CPU initialization after updates Chao Gao
2026-01-26 10:53   ` Tony Lindgren
2026-02-02  7:32   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 21/26] x86/virt/tdx: Establish contexts for the new TDX Module Chao Gao
2026-01-26 10:54   ` Tony Lindgren
2026-01-23 14:55 ` [PATCH v3 22/26] x86/virt/tdx: Update tdx_sysinfo and check features post-update Chao Gao
2026-01-26 11:07   ` Tony Lindgren
2026-02-02  7:33   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 23/26] x86/virt/tdx: Enable TDX Module runtime updates Chao Gao
2026-01-26 11:14   ` Tony Lindgren
2026-02-04 10:03     ` Tony Lindgren
2026-02-02  7:41   ` Xu Yilun
2026-01-23 14:55 ` [PATCH v3 24/26] x86/virt/seamldr: Extend sigstruct to 16KB Chao Gao
2026-01-26 11:15   ` Tony Lindgren
2026-01-27  3:58   ` Huang, Kai
2026-01-28 23:01   ` Huang, Kai
2026-01-30 14:25     ` Chao Gao
2026-02-02 11:57       ` Huang, Kai
2026-01-23 14:55 ` [PATCH v3 25/26] x86/virt/tdx: Avoid updates during update-sensitive operations Chao Gao
2026-01-26 11:23   ` Tony Lindgren
2026-01-23 14:55 ` [PATCH v3 26/26] coco/tdx-host: Set and document TDX Module update expectations Chao Gao
2026-01-26 11:28   ` Tony Lindgren
2026-01-26 22:14   ` dan.j.williams
2026-01-27 12:17     ` Chao Gao
2026-01-27 17:23       ` dan.j.williams
2026-01-28 17:52 ` [PATCH v3 00/26] Runtime TDX Module update support Sagi Shahar
2026-01-29  1:51   ` Chao Gao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260123145645.90444-15-chao.gao@intel.com \
    --to=chao.gao@intel.com \
    --cc=bp@alien8.de \
    --cc=dan.j.williams@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=farrah.chen@intel.com \
    --cc=hpa@zytor.com \
    --cc=ira.weiny@intel.com \
    --cc=kai.huang@intel.com \
    --cc=kas@kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=nik.borisov@suse.com \
    --cc=paulmck@kernel.org \
    --cc=reinette.chatre@intel.com \
    --cc=rick.p.edgecombe@intel.com \
    --cc=sagis@google.com \
    --cc=seanjc@google.com \
    --cc=tglx@linutronix.de \
    --cc=vannapurve@google.com \
    --cc=vishal.l.verma@intel.com \
    --cc=x86@kernel.org \
    --cc=yilun.xu@linux.intel.com \
    --cc=zhenzhong.duan@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®