[PATCH v1 0/4] x86/vmware: Preserve panic diagnostics in vmware.log

[PATCH v1 0/4] x86/vmware: Preserve panic diagnostics in vmware.log

[Zack Rusin]

A virtual machine may have neither guest-local persistent storage nor
working userspace after a panic. The host's vmware.log remains available
after the VM terminates, and a production encrypted guest is the only
component able to export readable panic text. Hyper-V already provides
this class of host-persistent panic reporting.

Add a panic-only VMware logger that copies the newest 4 KiB of the kernel
log and sends it with bounded low-bandwidth RPC work. Ordinary guests
enable text transfer by default; encrypted guests require an administrator
to opt in through kernel.vmware_record_panic_msg. A separate late dumper
reports the structured crash event after built-in dumpers and suppresses
the event while a crash kernel is loaded, because a configured VMware host
may terminate the VM before the reporting hypercall returns.

Patch 4 makes panic diagnostics run before kdump by default on VMware
guests. This follows existing platform uses of
crash_kexec_post_notifiers, with Hyper-V as the closest guest-reporting
precedent. Unlike Hyper-V, VMware exposes no capability bit for this
behavior. An explicit crash_kexec_post_notifiers=0 still wins, and patch 4
can be dropped without affecting patches 1 through 3.

This replaces patches 2 and 3 of Alexey's v2 posting:
https://lore.kernel.org/all/20260309235250.2611115-1-alexey.makhalov@broadcom.com/

The intended merge path is tip's x86/vmware branch. Boris please review the
VMware transport and policy. Petr and John, please review the panic-only
dumper use and registration ordering. Joel and Kees, please review the
sysctl. Baoquan and the KEXEC/KDUMP team, please review the 
loaded-crash-kernel guard and the default in patch 4.

Zack Rusin (4):
  x86/vmware: Add a bounded panic log sender
  x86/vmware: Add the vmware_record_panic_msg sysctl
  x86/vmware: Report guest crashes after kmsg dumpers
  x86/vmware: Run panic diagnostics before kdump by default

 Documentation/admin-guide/sysctl/kernel.rst |  14 ++
 arch/x86/include/asm/vmware.h               |   2 +
 arch/x86/kernel/cpu/vmware.c                | 245 ++++++++++++++++++++
 3 files changed, 261 insertions(+)


base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
-- 
2.53.0

[PATCH v1 1/4] x86/vmware: Add a bounded panic log sender

[Zack Rusin]

A virtual machine may have no guest-local persistent storage or working
userspace after a panic. Preserve the newest kernel log tail in the host
vmware.log, which remains available after the VM terminates.

Allocate one page during early init and register a panic-only kmsg dumper.
Copy the newest records once, prefix them with "log ", and send one
explicit-length RPC message over the low-bandwidth register path. The page
limit bounds panic-time work and avoids sharing guest memory with the host.

Keep the source immutable for up to three attempts. Retry only checkpoint
responses, zero-fill a short final word, and close the single RPC channel
on every exit after open. The panic callback allocates nothing, does not
sleep, and emits no printk messages.

Free the page and leave panic logging disabled if dumper registration
fails, including when CONFIG_PRINTK=n.

Link: https://lore.kernel.org/r/20260309235250.2611115-3-alexey.makhalov@broadcom.com
Co-developed-by: Bo Gan <bo.gan@broadcom.com>
Signed-off-by: Bo Gan <bo.gan@broadcom.com>
Co-developed-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/include/asm/vmware.h |   1 +
 arch/x86/kernel/cpu/vmware.c  | 158 ++++++++++++++++++++++++++++++++++
 2 files changed, 159 insertions(+)

diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 4220dae14a2d..598d4cd448ec 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -57,6 +57,7 @@
 #define VMWARE_HYPERVISOR_MAGIC		0x564d5868U
 
 #define VMWARE_CMD_GETVERSION		10
+#define VMWARE_CMD_MESSAGE		30
 #define VMWARE_CMD_GETHZ		45
 #define VMWARE_CMD_GETVCPU_INFO		68
 #define VMWARE_CMD_STEALCLOCK		91
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index 34b73573b108..bf59653d6e07 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -24,11 +24,14 @@
 #include <linux/dmi.h>
 #include <linux/init.h>
 #include <linux/export.h>
+#include <linux/kmsg_dump.h>
+#include <linux/mm.h>
 #include <linux/clocksource.h>
 #include <linux/cpu.h>
 #include <linux/efi.h>
 #include <linux/reboot.h>
 #include <linux/static_call.h>
+#include <linux/wordpart.h>
 #include <linux/sched/cputime.h>
 #include <asm/div64.h>
 #include <asm/x86_init.h>
@@ -52,6 +55,22 @@
 #define STEALCLOCK_DISABLED        0
 #define STEALCLOCK_ENABLED         1
 
+#define VMWARE_MSG_STATUS_SUCCESS	BIT(16)
+#define VMWARE_MSG_STATUS_CPT		BIT(20)
+
+#define VMWARE_RPCI_PROTOCOL		0x49435052
+#define VMWARE_GUESTMSG_COOKIE		BIT(31)
+
+#define VMWARE_MSG_TYPE(n)		((n) << 16)
+#define VMWARE_MSG_OPEN			VMWARE_MSG_TYPE(0U)
+#define VMWARE_MSG_SENDSIZE		VMWARE_MSG_TYPE(1U)
+#define VMWARE_MSG_SENDPAYLOAD		VMWARE_MSG_TYPE(2U)
+#define VMWARE_MSG_CLOSE		VMWARE_MSG_TYPE(6U)
+
+#define VMWARE_LOG_ATTEMPTS		3
+#define VMWARE_LOG_PREFIX		"log "
+#define VMWARE_LOG_PREFIX_LEN		(sizeof(VMWARE_LOG_PREFIX) - 1)
+
 struct vmware_steal_time {
 	union {
 		u64 clock;	/* stolen time counter in units of vtsc */
@@ -64,6 +83,12 @@ struct vmware_steal_time {
 	u64 reserved[7];
 };
 
+struct vmware_rpc_channel {
+	u16 id;
+	u32 cookie_high;
+	u32 cookie_low;
+};
+
 static unsigned long vmware_tsc_khz __ro_after_init;
 static u8 vmware_hypercall_mode     __ro_after_init;
 
@@ -142,6 +167,139 @@ static unsigned long vmware_get_tsc_khz(void)
 	return vmware_tsc_khz;
 }
 
+static int vmware_log_open(struct vmware_rpc_channel *channel)
+{
+	u32 info, id;
+
+	vmware_hypercall6(VMWARE_CMD_MESSAGE | VMWARE_MSG_OPEN,
+			  VMWARE_RPCI_PROTOCOL | VMWARE_GUESTMSG_COOKIE, 0,
+			  &info, &id, &channel->cookie_high,
+			  &channel->cookie_low);
+	if (!(info & VMWARE_MSG_STATUS_SUCCESS))
+		return -EIO;
+
+	channel->id = upper_16_bits(id);
+	return 0;
+}
+
+static int vmware_log_close(const struct vmware_rpc_channel *channel)
+{
+	u32 info;
+
+	vmware_hypercall5(VMWARE_CMD_MESSAGE | VMWARE_MSG_CLOSE, 0,
+			  (u32)channel->id << 16, channel->cookie_high,
+			  channel->cookie_low, &info);
+
+	return info & VMWARE_MSG_STATUS_SUCCESS ? 0 : -EIO;
+}
+
+static int vmware_log_send_once(const struct vmware_rpc_channel *channel,
+				const char *buffer, size_t length)
+{
+	u32 info;
+
+	vmware_hypercall5(VMWARE_CMD_MESSAGE | VMWARE_MSG_SENDSIZE, length,
+			  (u32)channel->id << 16, channel->cookie_high,
+			  channel->cookie_low, &info);
+	if (!(info & VMWARE_MSG_STATUS_SUCCESS))
+		return info & VMWARE_MSG_STATUS_CPT ? -EAGAIN : -EIO;
+
+	while (length) {
+		size_t bytes = min_t(size_t, length, sizeof(u32));
+		u32 word = 0;
+
+		memcpy(&word, buffer, bytes);
+		vmware_hypercall5(VMWARE_CMD_MESSAGE | VMWARE_MSG_SENDPAYLOAD,
+				  word, (u32)channel->id << 16,
+				  channel->cookie_high, channel->cookie_low,
+				  &info);
+		if (!(info & VMWARE_MSG_STATUS_SUCCESS))
+			return info & VMWARE_MSG_STATUS_CPT ? -EAGAIN : -EIO;
+
+		buffer += bytes;
+		length -= bytes;
+	}
+
+	return 0;
+}
+
+static int vmware_log_send(const struct vmware_rpc_channel *channel,
+			   const char *buffer, size_t length)
+{
+	int attempt, ret;
+
+	for (attempt = 0; attempt < VMWARE_LOG_ATTEMPTS; attempt++) {
+		ret = vmware_log_send_once(channel, buffer, length);
+		if (ret != -EAGAIN)
+			return ret;
+	}
+
+	return -EAGAIN;
+}
+
+static int vmware_log_rpc(const char *buffer, size_t length)
+{
+	struct vmware_rpc_channel channel;
+	int close_ret, ret;
+
+	ret = vmware_log_open(&channel);
+	if (ret)
+		return ret;
+
+	ret = vmware_log_send(&channel, buffer, length);
+	close_ret = vmware_log_close(&channel);
+	if (!ret && close_ret)
+		ret = close_ret;
+
+	return ret;
+}
+
+static struct page *vmware_panic_page;
+
+static void vmware_panic_log_dump(struct kmsg_dumper *dumper,
+				  struct kmsg_dump_detail *detail)
+{
+	struct kmsg_dump_iter iter;
+	char *buffer = page_address(vmware_panic_page);
+	size_t length = 0;
+
+	memcpy(buffer, VMWARE_LOG_PREFIX, VMWARE_LOG_PREFIX_LEN);
+	kmsg_dump_rewind(&iter);
+	(void)kmsg_dump_get_buffer(&iter, true,
+				   buffer + VMWARE_LOG_PREFIX_LEN,
+				   PAGE_SIZE - VMWARE_LOG_PREFIX_LEN, &length);
+	(void)vmware_log_rpc(buffer, length + VMWARE_LOG_PREFIX_LEN);
+}
+
+static struct kmsg_dumper vmware_panic_log_dumper = {
+	.dump = vmware_panic_log_dump,
+	.max_reason = KMSG_DUMP_PANIC,
+};
+
+static int __init vmware_panic_log_init(void)
+{
+	int ret;
+
+	if (!hypervisor_is_type(X86_HYPER_VMWARE))
+		return 0;
+
+	vmware_panic_page = alloc_page(GFP_KERNEL);
+	if (!vmware_panic_page) {
+		pr_err("failed to allocate panic log buffer\n");
+		return 0;
+	}
+
+	ret = kmsg_dump_register(&vmware_panic_log_dumper);
+	if (ret) {
+		pr_err("failed to register panic log dumper: %d\n", ret);
+		__free_page(vmware_panic_page);
+		vmware_panic_page = NULL;
+	}
+
+	return 0;
+}
+early_initcall(vmware_panic_log_init);
+
 #ifdef CONFIG_PARAVIRT
 static struct cyc2ns_data vmware_cyc2ns __ro_after_init;
 static bool vmw_sched_clock __initdata = true;
-- 
2.53.0

[PATCH v1 2/4] x86/vmware: Add the vmware_record_panic_msg sysctl

[Zack Rusin]

Full panic text is useful on ordinary guests. Encrypted guests should
export it only when an administrator opts in. Reporting a crash does not
require transferring the log.

Enable panic recording by default for ordinary VMware guests and disable
it when guest memory encryption is active. Expose the policy as the
boolean kernel.vmware_record_panic_msg sysctl and document that it
controls only the vmware.log transfer.

Register the sysctl only after the panic buffer and dumper are ready, and
only when sysctl support is built. A registration failure leaves the
internal default in force.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 Documentation/admin-guide/sysctl/kernel.rst | 14 +++++++++++++
 arch/x86/kernel/cpu/vmware.c                | 23 +++++++++++++++++++++
 2 files changed, 37 insertions(+)

diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
index b6328cd0f43e..4577e935ddcf 100644
--- a/Documentation/admin-guide/sysctl/kernel.rst
+++ b/Documentation/admin-guide/sysctl/kernel.rst
@@ -1690,6 +1690,20 @@ entry will default to 2 instead of 0.
 = =============================================================
 
 
+vmware_record_panic_msg
+=======================
+
+Controls whether panic kmsg data is written to the host's ``vmware.log``.
+This setting does not control the separate VMware guest-crash event.
+
+= ==============================================================
+0 Do not write panic kmsg data to ``vmware.log``. This is the
+  default for encrypted guests.
+1 Write panic kmsg data to ``vmware.log``. This is the default for
+  ordinary guests.
+= ==============================================================
+
+
 warn_limit
 ==========
 
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index bf59653d6e07..3848811550e9 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -31,6 +31,7 @@
 #include <linux/efi.h>
 #include <linux/reboot.h>
 #include <linux/static_call.h>
+#include <linux/sysctl.h>
 #include <linux/wordpart.h>
 #include <linux/sched/cputime.h>
 #include <asm/div64.h>
@@ -255,6 +256,19 @@ static int vmware_log_rpc(const char *buffer, size_t length)
 }
 
 static struct page *vmware_panic_page;
+static int vmware_record_panic_msg;
+
+static const struct ctl_table vmware_panic_sysctls[] = {
+	{
+		.procname	= "vmware_record_panic_msg",
+		.data		= &vmware_record_panic_msg,
+		.maxlen		= sizeof(vmware_record_panic_msg),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE,
+	},
+};
 
 static void vmware_panic_log_dump(struct kmsg_dumper *dumper,
 				  struct kmsg_dump_detail *detail)
@@ -263,6 +277,9 @@ static void vmware_panic_log_dump(struct kmsg_dumper *dumper,
 	char *buffer = page_address(vmware_panic_page);
 	size_t length = 0;
 
+	if (!READ_ONCE(vmware_record_panic_msg))
+		return;
+
 	memcpy(buffer, VMWARE_LOG_PREFIX, VMWARE_LOG_PREFIX_LEN);
 	kmsg_dump_rewind(&iter);
 	(void)kmsg_dump_get_buffer(&iter, true,
@@ -283,6 +300,9 @@ static int __init vmware_panic_log_init(void)
 	if (!hypervisor_is_type(X86_HYPER_VMWARE))
 		return 0;
 
+	vmware_record_panic_msg =
+		!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT);
+
 	vmware_panic_page = alloc_page(GFP_KERNEL);
 	if (!vmware_panic_page) {
 		pr_err("failed to allocate panic log buffer\n");
@@ -295,6 +315,9 @@ static int __init vmware_panic_log_init(void)
 		__free_page(vmware_panic_page);
 		vmware_panic_page = NULL;
 	}
+	if (vmware_panic_page && IS_ENABLED(CONFIG_SYSCTL) &&
+	    !register_sysctl("kernel", vmware_panic_sysctls))
+		pr_err("failed to register panic log sysctl\n");
 
 	return 0;
 }
-- 
2.53.0

[PATCH v1 3/4] x86/vmware: Report guest crashes after kmsg dumpers

[Zack Rusin]

VMware accepts a structured guest-crash event, but a host configured with
backdoor.coredumpOnCrash may terminate the VM before the hypercall
returns. Sending the event from a panic notifier could prevent later kmsg
dumpers and a loaded crash kernel from running.

Register a second panic-only kmsg dumper with late_initcall_sync(). Kmsg
dumpers run in registration order, placing the report after the early
VMware logger and built-in dumpers registered by then. This leaves no
report before the late initcall, but avoids losing dump output to a
terminating host.

Skip the event when a crash kernel is loaded. The earlier logger can still
write panic text to vmware.log before __crash_kexec(). If kdump later
fails, no structured event is sent; issuing it first could prevent kdump
from running at all.

Issue the event at most once. With CONFIG_PRINTK=n, install a late
INT_MIN panic notifier. An unexpected dumper registration failure with
printk enabled leaves reporting disabled because a notifier fallback
would overtake working dumpers.

Link: https://lore.kernel.org/r/20260309235250.2611115-4-alexey.makhalov@broadcom.com
Co-developed-by: Brennan Lamoreaux <brennan.lamoreaux@broadcom.com>
Signed-off-by: Brennan Lamoreaux <brennan.lamoreaux@broadcom.com>
Co-developed-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Alexey Makhalov <alexey.makhalov@broadcom.com>
Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/include/asm/vmware.h |  1 +
 arch/x86/kernel/cpu/vmware.c  | 63 +++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/arch/x86/include/asm/vmware.h b/arch/x86/include/asm/vmware.h
index 598d4cd448ec..ea2ce5272a00 100644
--- a/arch/x86/include/asm/vmware.h
+++ b/arch/x86/include/asm/vmware.h
@@ -61,6 +61,7 @@
 #define VMWARE_CMD_GETHZ		45
 #define VMWARE_CMD_GETVCPU_INFO		68
 #define VMWARE_CMD_STEALCLOCK		91
+#define VMWARE_CMD_REPORTGUESTCRASH	102
 /*
  * Hypercall command mask:
  *   bits [6:0] command, range [0, 127]
diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index 3848811550e9..c21fc835537f 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -22,10 +22,14 @@
  */
 
 #include <linux/dmi.h>
+#include <linux/atomic.h>
+#include <linux/crash_core.h>
 #include <linux/init.h>
 #include <linux/export.h>
 #include <linux/kmsg_dump.h>
+#include <linux/limits.h>
 #include <linux/mm.h>
+#include <linux/panic_notifier.h>
 #include <linux/clocksource.h>
 #include <linux/cpu.h>
 #include <linux/efi.h>
@@ -323,6 +327,65 @@ static int __init vmware_panic_log_init(void)
 }
 early_initcall(vmware_panic_log_init);
 
+static atomic_t vmware_crash_reported = ATOMIC_INIT(0);
+
+static void vmware_report_guest_crash(void)
+{
+	if (kexec_crash_loaded())
+		return;
+	if (atomic_xchg(&vmware_crash_reported, 1))
+		return;
+
+	vmware_hypercall1(VMWARE_CMD_REPORTGUESTCRASH, 0);
+}
+
+static void vmware_crash_report_dump(struct kmsg_dumper *dumper,
+				     struct kmsg_dump_detail *detail)
+{
+	vmware_report_guest_crash();
+}
+
+static struct kmsg_dumper vmware_crash_report_dumper = {
+	.dump = vmware_crash_report_dump,
+	.max_reason = KMSG_DUMP_PANIC,
+};
+
+static int vmware_crash_report_notify(struct notifier_block *nb,
+				      unsigned long action, void *data)
+{
+	vmware_report_guest_crash();
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block vmware_crash_report_nb = {
+	.notifier_call = vmware_crash_report_notify,
+	.priority = INT_MIN,
+};
+
+static int __init vmware_crash_report_init(void)
+{
+	int ret;
+
+	if (!hypervisor_is_type(X86_HYPER_VMWARE))
+		return 0;
+
+	ret = kmsg_dump_register(&vmware_crash_report_dumper);
+	if (!ret)
+		return 0;
+	if (IS_ENABLED(CONFIG_PRINTK)) {
+		pr_err("failed to register crash report dumper: %d\n", ret);
+		return 0;
+	}
+
+	ret = atomic_notifier_chain_register(&panic_notifier_list,
+					     &vmware_crash_report_nb);
+	if (ret)
+		pr_err("failed to register crash report notifier: %d\n", ret);
+
+	return 0;
+}
+late_initcall_sync(vmware_crash_report_init);
+
 #ifdef CONFIG_PARAVIRT
 static struct cyc2ns_data vmware_cyc2ns __ro_after_init;
 static bool vmw_sched_clock __initdata = true;
-- 
2.53.0

[PATCH v1 4/4] x86/vmware: Run panic diagnostics before kdump by default

[Zack Rusin]

The core panic path enters a loaded crash kernel before running kmsg
dumpers, so the VMware logger cannot preserve the panic in the host log.
Set crash_kexec_post_notifiers during VMware platform setup. setup_arch()
runs before ordinary core parameters are parsed, so an explicit
crash_kexec_post_notifiers=0 still overrides this default. The 0644
parameter also remains writable at runtime.

VMware exposes no capability bit for this behavior, so the default changes
for every VMware guest. Running the logger first adds work to the panic
path and can reduce kdump reliability.

A full log transfer uses 1027 low-bandwidth hypercalls; three checkpoint
attempts are bounded at 3077. In vmcall tests, a maximum-size transfer
took at most 7.47 ms; three complete transfers comprising 3081 calls took
at most 21.58 ms.

The crash report remains suppressed while a crash kernel is loaded because
the host may terminate the VM inside that call. A future report from the
crash kernel or kdump userspace can restore the event after saving the
vmcore.

Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
---
 arch/x86/kernel/cpu/vmware.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
index c21fc835537f..d88c625d8389 100644
--- a/arch/x86/kernel/cpu/vmware.c
+++ b/arch/x86/kernel/cpu/vmware.c
@@ -646,6 +646,7 @@ static void __init vmware_platform_setup(void)
 {
 	u32 eax, ebx, ecx;
 	u64 lpj, tsc_khz;
+	crash_kexec_post_notifiers = true;
 
 	eax = vmware_hypercall3(VMWARE_CMD_GETHZ, UINT_MAX, &ebx, &ecx);
 
-- 
2.53.0

Re: [PATCH v1 2/4] x86/vmware: Add the vmware_record_panic_msg sysctl

[Joel Granados]

[-- Attachment #1: Type: text/plain, Size: 4303 bytes --]

On Tue, Sep 08, 2026 at 02:07:14PM -0400, Zack Rusin wrote:
> Full panic text is useful on ordinary guests. Encrypted guests should
> export it only when an administrator opts in. Reporting a crash does not
> require transferring the log.
> 
> Enable panic recording by default for ordinary VMware guests and disable
> it when guest memory encryption is active. Expose the policy as the
> boolean kernel.vmware_record_panic_msg sysctl and document that it
> controls only the vmware.log transfer.
> 
> Register the sysctl only after the panic buffer and dumper are ready, and
> only when sysctl support is built. A registration failure leaves the
> internal default in force.
> 
> Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
> ---
>  Documentation/admin-guide/sysctl/kernel.rst | 14 +++++++++++++
>  arch/x86/kernel/cpu/vmware.c                | 23 +++++++++++++++++++++
>  2 files changed, 37 insertions(+)
> 
> diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
> index b6328cd0f43e..4577e935ddcf 100644
> --- a/Documentation/admin-guide/sysctl/kernel.rst
> +++ b/Documentation/admin-guide/sysctl/kernel.rst
> @@ -1690,6 +1690,20 @@ entry will default to 2 instead of 0.
>  = =============================================================
>  
>  
> +vmware_record_panic_msg
> +=======================
> +
> +Controls whether panic kmsg data is written to the host's ``vmware.log``.
> +This setting does not control the separate VMware guest-crash event.
> +
> += ==============================================================
> +0 Do not write panic kmsg data to ``vmware.log``. This is the
> +  default for encrypted guests.
> +1 Write panic kmsg data to ``vmware.log``. This is the default for
> +  ordinary guests.
> += ==============================================================

This seems like it can be represented in a boolean value. Any reason why
you did not use proc_dobool for your handler? Would it make sense to
have vmware_record_panic_msg as a boolean?

Best

> +
> +
>  warn_limit
>  ==========
>  
> diff --git a/arch/x86/kernel/cpu/vmware.c b/arch/x86/kernel/cpu/vmware.c
> index bf59653d6e07..3848811550e9 100644
> --- a/arch/x86/kernel/cpu/vmware.c
> +++ b/arch/x86/kernel/cpu/vmware.c
> @@ -31,6 +31,7 @@
>  #include <linux/efi.h>
>  #include <linux/reboot.h>
>  #include <linux/static_call.h>
> +#include <linux/sysctl.h>
>  #include <linux/wordpart.h>
>  #include <linux/sched/cputime.h>
>  #include <asm/div64.h>
> @@ -255,6 +256,19 @@ static int vmware_log_rpc(const char *buffer, size_t length)
>  }
>  
>  static struct page *vmware_panic_page;
> +static int vmware_record_panic_msg;
> +
> +static const struct ctl_table vmware_panic_sysctls[] = {
> +	{
> +		.procname	= "vmware_record_panic_msg",
> +		.data		= &vmware_record_panic_msg,
> +		.maxlen		= sizeof(vmware_record_panic_msg),
> +		.mode		= 0644,
> +		.proc_handler	= proc_dointvec_minmax,
> +		.extra1		= SYSCTL_ZERO,
> +		.extra2		= SYSCTL_ONE,
> +	},
> +};
>  
>  static void vmware_panic_log_dump(struct kmsg_dumper *dumper,
>  				  struct kmsg_dump_detail *detail)
> @@ -263,6 +277,9 @@ static void vmware_panic_log_dump(struct kmsg_dumper *dumper,
>  	char *buffer = page_address(vmware_panic_page);
>  	size_t length = 0;
>  
> +	if (!READ_ONCE(vmware_record_panic_msg))
> +		return;
> +
>  	memcpy(buffer, VMWARE_LOG_PREFIX, VMWARE_LOG_PREFIX_LEN);
>  	kmsg_dump_rewind(&iter);
>  	(void)kmsg_dump_get_buffer(&iter, true,
> @@ -283,6 +300,9 @@ static int __init vmware_panic_log_init(void)
>  	if (!hypervisor_is_type(X86_HYPER_VMWARE))
>  		return 0;
>  
> +	vmware_record_panic_msg =
> +		!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT);
> +
>  	vmware_panic_page = alloc_page(GFP_KERNEL);
>  	if (!vmware_panic_page) {
>  		pr_err("failed to allocate panic log buffer\n");
> @@ -295,6 +315,9 @@ static int __init vmware_panic_log_init(void)
>  		__free_page(vmware_panic_page);
>  		vmware_panic_page = NULL;
>  	}
> +	if (vmware_panic_page && IS_ENABLED(CONFIG_SYSCTL) &&
> +	    !register_sysctl("kernel", vmware_panic_sysctls))
> +		pr_err("failed to register panic log sysctl\n");
>  
>  	return 0;
>  }
> -- 
> 2.53.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [PATCH v1 2/4] x86/vmware: Add the vmware_record_panic_msg sysctl

[Zack Rusin]

[-- Attachment #1: Type: text/plain, Size: 2557 bytes --]

On Thu, Sep 10, 2026 at 4:32 AM Joel Granados <joel.granados@kernel.org> wrote:
>
> On Tue, Sep 08, 2026 at 02:07:14PM -0400, Zack Rusin wrote:
> > Full panic text is useful on ordinary guests. Encrypted guests should
> > export it only when an administrator opts in. Reporting a crash does not
> > require transferring the log.
> >
> > Enable panic recording by default for ordinary VMware guests and disable
> > it when guest memory encryption is active. Expose the policy as the
> > boolean kernel.vmware_record_panic_msg sysctl and document that it
> > controls only the vmware.log transfer.
> >
> > Register the sysctl only after the panic buffer and dumper are ready, and
> > only when sysctl support is built. A registration failure leaves the
> > internal default in force.
> >
> > Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
> > ---
> >  Documentation/admin-guide/sysctl/kernel.rst | 14 +++++++++++++
> >  arch/x86/kernel/cpu/vmware.c                | 23 +++++++++++++++++++++
> >  2 files changed, 37 insertions(+)
> >
> > diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
> > index b6328cd0f43e..4577e935ddcf 100644
> > --- a/Documentation/admin-guide/sysctl/kernel.rst
> > +++ b/Documentation/admin-guide/sysctl/kernel.rst
> > @@ -1690,6 +1690,20 @@ entry will default to 2 instead of 0.
> >  = =============================================================
> >
> >
> > +vmware_record_panic_msg
> > +=======================
> > +
> > +Controls whether panic kmsg data is written to the host's ``vmware.log``.
> > +This setting does not control the separate VMware guest-crash event.
> > +
> > += ==============================================================
> > +0 Do not write panic kmsg data to ``vmware.log``. This is the
> > +  default for encrypted guests.
> > +1 Write panic kmsg data to ``vmware.log``. This is the default for
> > +  ordinary guests.
> > += ==============================================================
>
> This seems like it can be represented in a boolean value. Any reason why
> you did not use proc_dobool for your handler? Would it make sense to
> have vmware_record_panic_msg as a boolean?

Thank you for taking a look! This follows the existing Hyper-V
implementation of hyperv_record_panic_msg, which uses an int and
proc_dointvec_minmax with 0/1 bounds. I thought that consistency
across the codebase was more important even though I agree that bool
and proc_dobool() would be a better fit.

z

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5414 bytes --]

Re: [PATCH v1 2/4] x86/vmware: Add the vmware_record_panic_msg sysctl

[Zack Rusin]

[-- Attachment #1: Type: text/plain, Size: 3169 bytes --]

On Thu, Sep 10, 2026 at 7:44 AM Zack Rusin <zack.rusin@broadcom.com> wrote:
>
> On Thu, Sep 10, 2026 at 4:32 AM Joel Granados <joel.granados@kernel.org> wrote:
> >
> > On Tue, Sep 08, 2026 at 02:07:14PM -0400, Zack Rusin wrote:
> > > Full panic text is useful on ordinary guests. Encrypted guests should
> > > export it only when an administrator opts in. Reporting a crash does not
> > > require transferring the log.
> > >
> > > Enable panic recording by default for ordinary VMware guests and disable
> > > it when guest memory encryption is active. Expose the policy as the
> > > boolean kernel.vmware_record_panic_msg sysctl and document that it
> > > controls only the vmware.log transfer.
> > >
> > > Register the sysctl only after the panic buffer and dumper are ready, and
> > > only when sysctl support is built. A registration failure leaves the
> > > internal default in force.
> > >
> > > Signed-off-by: Zack Rusin <zack.rusin@broadcom.com>
> > > ---
> > >  Documentation/admin-guide/sysctl/kernel.rst | 14 +++++++++++++
> > >  arch/x86/kernel/cpu/vmware.c                | 23 +++++++++++++++++++++
> > >  2 files changed, 37 insertions(+)
> > >
> > > diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
> > > index b6328cd0f43e..4577e935ddcf 100644
> > > --- a/Documentation/admin-guide/sysctl/kernel.rst
> > > +++ b/Documentation/admin-guide/sysctl/kernel.rst
> > > @@ -1690,6 +1690,20 @@ entry will default to 2 instead of 0.
> > >  = =============================================================
> > >
> > >
> > > +vmware_record_panic_msg
> > > +=======================
> > > +
> > > +Controls whether panic kmsg data is written to the host's ``vmware.log``.
> > > +This setting does not control the separate VMware guest-crash event.
> > > +
> > > += ==============================================================
> > > +0 Do not write panic kmsg data to ``vmware.log``. This is the
> > > +  default for encrypted guests.
> > > +1 Write panic kmsg data to ``vmware.log``. This is the default for
> > > +  ordinary guests.
> > > += ==============================================================
> >
> > This seems like it can be represented in a boolean value. Any reason why
> > you did not use proc_dobool for your handler? Would it make sense to
> > have vmware_record_panic_msg as a boolean?
>
> Thank you for taking a look! This follows the existing Hyper-V
> implementation of hyperv_record_panic_msg, which uses an int and
> proc_dointvec_minmax with 0/1 bounds. I thought that consistency
> across the codebase was more important even though I agree that bool
> and proc_dobool() would be a better fit.

I've been thinking about it and the more I look at this code the more
I feel like respinning v2 and changing vmware_record_panic_msg to
bool. Because kernel.hyperv.record_panic_msg is already in, I'm
worried that if I'll do someone might object based on the fact that
the same option for hyperv behaves differenty introducing a subtle
inconsistency. Any strong opinions on that before I go ahead and make
that change?

z

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5414 bytes --]

Re: [PATCH v1 4/4] x86/vmware: Run panic diagnostics before kdump by default

[Petr Mladek]

Adding Guilherme G. Piccoli into Cc.

On Tue 2026-09-08 14:07:16, Zack Rusin wrote:
> The core panic path enters a loaded crash kernel before running kmsg
> dumpers, so the VMware logger cannot preserve the panic in the host log.
> Set crash_kexec_post_notifiers during VMware platform setup. setup_arch()
> runs before ordinary core parameters are parsed, so an explicit
> crash_kexec_post_notifiers=0 still overrides this default. The 0644
> parameter also remains writable at runtime.
> 
> VMware exposes no capability bit for this behavior, so the default changes
> for every VMware guest. Running the logger first adds work to the panic
> path and can reduce kdump reliability.

AFAIK, the quality of the notifiers is varying. Running all notifiers
might reduce the kdump reliability even more.

I do not like much the hack with crash_kexec_post_notifiers. It is
an all or nothing option.

Also it was introduced as a quick hack so that users could decide
what is more important for them. But it is not longer a "user"
decision when some platforms enforce the ordering because they
depend on the notifier.

panic() is problematic and it is about compromises. And we need
to balance what is important, what is safe, and what is optional.
This is why I suggested to introduce more notifiers some time ago,
see https://lore.kernel.org/lkml/YfPxvzSzDLjO5ldp@alley/

Guillermo implemented this, see
https://lore.kernel.org/all/20220427224924.592546-1-gpiccoli@igalia.com/
But it has stalled because it touched too many subsystems and it was
hard to get an agreement.

Maybe, we should start with something simple, and introduce
one more panic notifier as a start. It might be called either:

  + "panic_hypervisor_list" because "crash_kexec_post_notifiers = true"
    seems to be primary set on hypervisors.

But I would rather make it more generic and call it

  + panic_pre_crash_kexec or panic_pre_kdump because there might be
    more notifiers which are either 100% safe and useful or are worth
    the risk before calling crash dump.

We could put there x86/vmware notifiers as a start. And we could later
move there other important notifiers.

How does that sound, please?

> A full log transfer uses 1027 low-bandwidth hypercalls; three checkpoint
> attempts are bounded at 3077. In vmcall tests, a maximum-size transfer
> took at most 7.47 ms; three complete transfers comprising 3081 calls took
> at most 21.58 ms.
> 
> The crash report remains suppressed while a crash kernel is loaded because
> the host may terminate the VM inside that call. A future report from the
> crash kernel or kdump userspace can restore the event after saving the
> vmcore.

Best Regards,
Petr

Re: [PATCH v1 0/4] x86/vmware: Preserve panic diagnostics in vmware.log

[Maaz Mombasawala]

On 9/8/26 11:07 AM, Zack Rusin wrote:
> A virtual machine may have neither guest-local persistent storage nor
> working userspace after a panic. The host's vmware.log remains available
> after the VM terminates, and a production encrypted guest is the only
> component able to export readable panic text. Hyper-V already provides
> this class of host-persistent panic reporting.
> 
> Add a panic-only VMware logger that copies the newest 4 KiB of the kernel
> log and sends it with bounded low-bandwidth RPC work. Ordinary guests
> enable text transfer by default; encrypted guests require an administrator
> to opt in through kernel.vmware_record_panic_msg. A separate late dumper
> reports the structured crash event after built-in dumpers and suppresses
> the event while a crash kernel is loaded, because a configured VMware host
> may terminate the VM before the reporting hypercall returns.
> 
> Patch 4 makes panic diagnostics run before kdump by default on VMware
> guests. This follows existing platform uses of
> crash_kexec_post_notifiers, with Hyper-V as the closest guest-reporting
> precedent. Unlike Hyper-V, VMware exposes no capability bit for this
> behavior. An explicit crash_kexec_post_notifiers=0 still wins, and patch 4
> can be dropped without affecting patches 1 through 3.
> 
> This replaces patches 2 and 3 of Alexey's v2 posting:
> https://lore.kernel.org/all/20260309235250.2611115-1-alexey.makhalov@broadcom.com/
> 
> The intended merge path is tip's x86/vmware branch. Boris please review the
> VMware transport and policy. Petr and John, please review the panic-only
> dumper use and registration ordering. Joel and Kees, please review the
> sysctl. Baoquan and the KEXEC/KDUMP team, please review the 
> loaded-crash-kernel guard and the default in patch 4.
> 
> Zack Rusin (4):
>   x86/vmware: Add a bounded panic log sender
>   x86/vmware: Add the vmware_record_panic_msg sysctl
>   x86/vmware: Report guest crashes after kmsg dumpers
>   x86/vmware: Run panic diagnostics before kdump by default
> 
>  Documentation/admin-guide/sysctl/kernel.rst |  14 ++
>  arch/x86/include/asm/vmware.h               |   2 +
>  arch/x86/kernel/cpu/vmware.c                | 245 ++++++++++++++++++++
>  3 files changed, 261 insertions(+)
> 
> 
> base-commit: cee9395acd8043be0644b25c34bfa86623f2b935


LGTM for the series

Reviewed-by: Maaz Mombasawala <maaz.mombasawala@broadcom.com>

-- 
Maaz Mombasawala <maaz.mombasawala@broadcom.com>

Re: [PATCH v1 4/4] x86/vmware: Run panic diagnostics before kdump by default

[Zack Rusin]

[-- Attachment #1: Type: text/plain, Size: 3802 bytes --]

On Thu, Sep 17, 2026 at 5:07 AM Petr Mladek <pmladek@suse.com> wrote:
>
> Adding Guilherme G. Piccoli into Cc.
>
> On Tue 2026-09-08 14:07:16, Zack Rusin wrote:
> > The core panic path enters a loaded crash kernel before running kmsg
> > dumpers, so the VMware logger cannot preserve the panic in the host log.
> > Set crash_kexec_post_notifiers during VMware platform setup. setup_arch()
> > runs before ordinary core parameters are parsed, so an explicit
> > crash_kexec_post_notifiers=0 still overrides this default. The 0644
> > parameter also remains writable at runtime.
> >
> > VMware exposes no capability bit for this behavior, so the default changes
> > for every VMware guest. Running the logger first adds work to the panic
> > path and can reduce kdump reliability.
>
> AFAIK, the quality of the notifiers is varying. Running all notifiers
> might reduce the kdump reliability even more.
>
> I do not like much the hack with crash_kexec_post_notifiers. It is
> an all or nothing option.
>
> Also it was introduced as a quick hack so that users could decide
> what is more important for them. But it is not longer a "user"
> decision when some platforms enforce the ordering because they
> depend on the notifier.
>
> panic() is problematic and it is about compromises. And we need
> to balance what is important, what is safe, and what is optional.
> This is why I suggested to introduce more notifiers some time ago,
> see https://lore.kernel.org/lkml/YfPxvzSzDLjO5ldp@alley/
>
> Guillermo implemented this, see
> https://lore.kernel.org/all/20220427224924.592546-1-gpiccoli@igalia.com/
> But it has stalled because it touched too many subsystems and it was
> hard to get an agreement.
>
> Maybe, we should start with something simple, and introduce
> one more panic notifier as a start. It might be called either:
>
>   + "panic_hypervisor_list" because "crash_kexec_post_notifiers = true"
>     seems to be primary set on hypervisors.
>
> But I would rather make it more generic and call it
>
>   + panic_pre_crash_kexec or panic_pre_kdump because there might be
>     more notifiers which are either 100% safe and useful or are worth
>     the risk before calling crash dump.
>
> We could put there x86/vmware notifiers as a start. And we could later
> move there other important notifiers.
>
> How does that sound, please?

afaict that sounds good. Keeping it to one new list and one client
should make it manageable to review (which is my biggest worry :) ).

As so the implementation: I'll drop VMware's
crash_kexec_post_notifiers default and start with one generic
panic_pre_kdump_list, with only the VMware log sender on it. The
structured crash-report hypercall (REPORTGUESTCRASH) can terminate the
VM, so it stays late and remains suppressed when a crash kernel is
loaded. Log transfer is still controlled by
kernel.vmware_record_panic_msg, with encrypted guests defaulting to
off.

I think that without that default though, x86 oops_end() can enter
crash_kexec(regs) before reaching panic(), for example with
panic_on_oops=1. To cover that path too, I'd call the chain from
__crash_kexec() after the image check and register capture, under the
existing kexec lock. A second call in vpanic(), immediately before
kmsg_dump_desc(), would cover the fallback path. And I think a
set-once guard would prevent duplicate or recursive dispatch.

That leaves CPU-stop and ordinary notifier ordering unchanged. The
logger would read the kmsg buffer directly, without invoking the other
dumpers, and would need to tolerate NMI context and other CPUs still
running. I'll keep the crash-core call in a separate patch for kdump
review, leaving other clients for follow-ups.

Does that sound good to you?

z

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5414 bytes --]