From: Ashish Kalra <Ashish.Kalra@amd.com>
To: <tglx@kernel.org>, <mingo@redhat.com>, <bp@alien8.de>,
<dave.hansen@linux.intel.com>, <x86@kernel.org>, <hpa@zytor.com>,
<seanjc@google.com>, <peterz@infradead.org>,
<thomas.lendacky@amd.com>, <herbert@gondor.apana.org.au>,
<davem@davemloft.net>, <ardb@kernel.org>
Cc: <pbonzini@redhat.com>, <aik@amd.com>, <Michael.Roth@amd.com>,
<KPrateek.Nayak@amd.com>, <Tycho.Andersen@amd.com>,
<Nathan.Fontenot@amd.com>, <ackerleytng@google.com>,
<jackyli@google.com>, <pgonda@google.com>, <rientjes@google.com>,
<jacobhxu@google.com>, <xin@zytor.com>,
<pawan.kumar.gupta@linux.intel.com>, <babu.moger@amd.com>,
<dyoung@redhat.com>, <nikunj@amd.com>, <darwi@linutronix.de>,
<linux-kernel@vger.kernel.org>, <linux-crypto@vger.kernel.org>,
<kvm@vger.kernel.org>, <linux-coco@lists.linux.dev>
Subject: [PATCH v16 4/5] x86/sev: Perform RMP optimizations asynchronously
Date: Wed, 16 Sep 2026 22:15:14 +0000 [thread overview]
Message-ID: <daa513fbe8f0672b00bc5e026e4399244be6dc72.1789594774.git.ashish.kalra@amd.com> (raw)
In-Reply-To: <cover.1789594774.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
When SNP is enabled, all writes to memory are checked to ensure memory
integrity. This imposes performance overhead on the whole system.
RMPOPT is a new instruction that minimizes the performance overhead of
RMP checks on the hypervisor and on non-SNP guests by allowing RMP
checks to be skipped for 1GB regions of memory that are known not to
contain any SNP guest memory.
Add support for performing RMP optimizations asynchronously using a
dedicated per-CPU workqueue. The workqueue is allocated from an initcall,
and snp_enable_rmpopt() queues the optimization pass.
Shortly after SNP initialization, run an optimization pass over all
physical memory (up to 2TB of system RAM, starting from the lowest
physical memory address aligned down to a 1GB boundary), skipping RMP
checks for 1GB regions that do not contain SNP guest memory (excluding
preassigned pages such as the RMP table and firmware pages).
As SNP guests are launched, RMPUPDATE assigns their private pages to
guest-owned state; when such a page falls within an optimized 1GB
region, the hardware clears that region's RMPOPT optimization and RMP
checks resume there to protect the guest memory.
Since launching SNP guests clears these optimizations, perform them
again asynchronously using the dedicated workqueue.
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Suggested-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
Changes in v15:
- Move the workqueue allocation and the (fixed) optimization range
computation to an initcall; snp_enable_rmpopt() now only programs the
RMPOPT_BASE MSRs and queues the optimization pass.
- Gate rmpopt_capable() on a static rmpopt_enabled bool set when the
workqueue is allocated, instead of an if (rmpopt_wq) check, and drop
rmpopt_wq_mutex.
- Queue both the initial and the teardown pass with mod_delayed_work().
arch/x86/virt/svm/sev.c | 99 +++++++++++++++++++++++++++++++++++++++--
1 file changed, 95 insertions(+), 4 deletions(-)
diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
index 25cb486ff61b..bd97135cacc2 100644
--- a/arch/x86/virt/svm/sev.c
+++ b/arch/x86/virt/svm/sev.c
@@ -19,6 +19,7 @@
#include <linux/iommu.h>
#include <linux/amd-iommu.h>
#include <linux/nospec.h>
+#include <linux/workqueue.h>
#include <asm/sev.h>
#include <asm/processor.h>
@@ -124,7 +125,25 @@ static void *rmp_bookkeeping __ro_after_init;
static u64 probed_rmp_base, probed_rmp_size;
-static phys_addr_t rmpopt_pa_start;
+static u64 rmpopt_pa_start, rmpopt_pa_end;
+
+enum rmpopt_op_type {
+ RMPOPT_OP_VERIFY_AND_REPORT_STATUS,
+ RMPOPT_OP_REPORT_STATUS
+};
+
+static struct workqueue_struct *rmpopt_wq;
+static struct delayed_work rmpopt_delayed_work;
+static bool rmpopt_enabled;
+
+/*
+ * Delay, in milliseconds, before the RMP re-optimization pass runs after an SNP
+ * guest is torn down, passed as the delay to mod_delayed_work(). This coalesces
+ * a burst of teardowns into a single scan and gives each guest's pages time to
+ * be converted back to the shared, hypervisor-owned state. The 10 second value
+ * is a heuristic trading re-optimization latency against scanning too eagerly.
+ */
+#define RMPOPT_WORK_TIMEOUT (10 * MSEC_PER_SEC)
static LIST_HEAD(snp_leaked_pages_list);
static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
@@ -557,6 +576,12 @@ int snp_prepare(void)
}
EXPORT_SYMBOL_FOR_MODULES(snp_prepare, "ccp");
+static void rmpopt_disable(void)
+{
+ if (rmpopt_wq)
+ cancel_delayed_work_sync(&rmpopt_delayed_work);
+}
+
void snp_shutdown(void)
{
u64 syscfg;
@@ -565,6 +590,8 @@ void snp_shutdown(void)
if (syscfg & MSR_AMD64_SYSCFG_SNP_EN)
return;
+ rmpopt_disable();
+
clear_rmp();
on_each_cpu(mfd_reconfigure, NULL, 1);
@@ -580,8 +607,69 @@ EXPORT_SYMBOL_FOR_MODULES(snp_shutdown, "ccp");
static bool rmpopt_capable(void)
{
return cpu_feature_enabled(X86_FEATURE_RMPOPT) &&
- cc_platform_has(CC_ATTR_HOST_SEV_SNP);
+ cc_platform_has(CC_ATTR_HOST_SEV_SNP) && rmpopt_enabled;
+}
+
+/*
+ * RMPOPT optimizations skip RMP checks at 1GB granularity if this range of
+ * memory does not contain any SNP guest memory.
+ *
+ * @pa is a system physical address; RMPOPT operates on the containing 1GB.
+ */
+static void rmpopt(u64 pa)
+{
+ enum rmpopt_op_type op = RMPOPT_OP_VERIFY_AND_REPORT_STATUS;
+ u64 pa_start = ALIGN_DOWN(pa, SZ_1G);
+
+ /* Supported by binutils 2.48+ */
+ asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
+ :: "a" (pa_start), "c" (op)
+ : "memory", "cc");
+}
+
+static void rmpopt_scan_range(void *arg)
+{
+ u64 pa;
+
+ for (pa = rmpopt_pa_start; pa < rmpopt_pa_end; pa += SZ_1G)
+ rmpopt(pa);
+}
+
+static void do_rmpopt_work(struct work_struct *work)
+{
+ /*
+ * Warm up the RMPOPT cache on this pinned per-CPU worker with interrupts
+ * enabled, so the IRQ-disabled fan-out below only issues cache-hit RMPOPTs.
+ */
+ rmpopt_scan_range(NULL);
+
+ on_each_cpu_mask(cpu_primary_thread_mask, rmpopt_scan_range, NULL, true);
+}
+
+static int __init rmpopt_init(void)
+{
+ if (!cpu_feature_enabled(X86_FEATURE_RMPOPT))
+ return 0;
+
+ rmpopt_wq = alloc_workqueue("rmpopt_wq", WQ_PERCPU, 1);
+ if (!rmpopt_wq) {
+ pr_err("Failed to allocate RMPOPT workqueue\n");
+ return 0;
+ }
+
+ INIT_DELAYED_WORK(&rmpopt_delayed_work, do_rmpopt_work);
+
+ /* The optimization range is fixed at boot; compute it once. */
+ rmpopt_pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), SZ_1G);
+ rmpopt_pa_end = ALIGN(PFN_PHYS(max_pfn), SZ_1G);
+ if ((rmpopt_pa_end - rmpopt_pa_start) > SZ_2T)
+ rmpopt_pa_end = rmpopt_pa_start + SZ_2T;
+
+ rmpopt_enabled = true;
+
+ return 0;
}
+device_initcall(rmpopt_init);
void snp_enable_rmpopt(void)
{
@@ -591,8 +679,6 @@ void snp_enable_rmpopt(void)
if (!rmpopt_capable())
return;
- rmpopt_pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), SZ_1G);
-
/*
* Per-CPU RMPOPT tables cover at most 2 TB. Program each core's
* RMPOPT_BASE with the start of RAM to optimize up to 2 TB. The MSR
@@ -610,6 +696,11 @@ void snp_enable_rmpopt(void)
for_each_cpu(cpu, cpu_primary_thread_mask)
wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE,
rmpopt_pa_start | MSR_AMD64_RMPOPT_ENABLE);
+
+ mod_delayed_work(rmpopt_wq, &rmpopt_delayed_work,
+ msecs_to_jiffies(RMPOPT_WORK_TIMEOUT));
+
+ pr_info_once("RMPOPT optimizations enabled\n");
}
EXPORT_SYMBOL_FOR_MODULES(snp_enable_rmpopt, "ccp");
--
2.43.0
next prev parent reply other threads:[~2026-09-16 22:15 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-16 22:13 [PATCH v16 0/5] Add RMPOPT support Ashish Kalra
2026-09-16 22:14 ` [PATCH v16 1/5] x86/cpufeatures: Add X86_FEATURE_RMPOPT feature flag Ashish Kalra
2026-09-16 22:14 ` [PATCH v16 2/5] x86/sev: Disable CPU hotplug while SNP is active Ashish Kalra
[not found] ` <20260916223630.C5E1C1F000FF@smtp.kernel.org>
2026-09-17 2:19 ` Borislav Petkov
2026-09-17 16:31 ` Kalra, Ashish
2026-09-16 22:14 ` [PATCH v16 3/5] x86/sev: Initialize RMPOPT configuration MSRs Ashish Kalra
2026-09-18 18:00 ` Borislav Petkov
2026-09-16 22:15 ` Ashish Kalra [this message]
2026-09-18 22:23 ` [PATCH v16 4/5] x86/sev: Perform RMP optimizations asynchronously Borislav Petkov
2026-09-18 22:46 ` Kalra, Ashish
2026-09-18 23:39 ` Borislav Petkov
2026-09-16 22:15 ` [PATCH v16 5/5] x86/sev: Re-enable RMP optimizations on SNP guest shutdown Ashish Kalra
2026-09-17 19:47 ` [PATCH v16 0/5] Add RMPOPT support Tom Lendacky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=daa513fbe8f0672b00bc5e026e4399244be6dc72.1789594774.git.ashish.kalra@amd.com \
--to=ashish.kalra@amd.com \
--cc=KPrateek.Nayak@amd.com \
--cc=Michael.Roth@amd.com \
--cc=Nathan.Fontenot@amd.com \
--cc=Tycho.Andersen@amd.com \
--cc=ackerleytng@google.com \
--cc=aik@amd.com \
--cc=ardb@kernel.org \
--cc=babu.moger@amd.com \
--cc=bp@alien8.de \
--cc=darwi@linutronix.de \
--cc=dave.hansen@linux.intel.com \
--cc=davem@davemloft.net \
--cc=dyoung@redhat.com \
--cc=herbert@gondor.apana.org.au \
--cc=hpa@zytor.com \
--cc=jackyli@google.com \
--cc=jacobhxu@google.com \
--cc=kvm@vger.kernel.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=nikunj@amd.com \
--cc=pawan.kumar.gupta@linux.intel.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=pgonda@google.com \
--cc=rientjes@google.com \
--cc=seanjc@google.com \
--cc=tglx@kernel.org \
--cc=thomas.lendacky@amd.com \
--cc=x86@kernel.org \
--cc=xin@zytor.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®