* [PATCH v2 1/2] KVM: Stop KVM_PRE_FAULT_MEMORY when the vCPU's dirty ring is full
2026-11-17 7:44 [PATCH v2 0/2] KVM: Bound KVM_PRE_FAULT_MEMORY by the vCPU's dirty ring Fuad Tabba
@ 2026-11-17 7:44 ` Fuad Tabba
2026-11-17 7:44 ` [PATCH v2 2/2] KVM: selftests: Test KVM_PRE_FAULT_MEMORY against a full dirty ring Fuad Tabba
1 sibling, 0 replies; 3+ messages in thread
From: Fuad Tabba @ 2026-11-17 7:44 UTC (permalink / raw)
To: Paolo Bonzini, Sean Christopherson
Cc: kvm, linux-kselftest, linux-doc, linux-kernel, Jonathan Corbet,
Shuah Khan, Isaku Yamahata, Rick Edgecombe, Lorenzo Stoakes,
Alexandru Elisei, Marc Zyngier, Oliver Upton, kvmarm,
Wei-Lin Chang, Will Deacon, Fuad Tabba, stable
Pre-faulting can map a dirty-logged page writable, which pushes it to
the vCPU's dirty ring. KVM_RUN exits with KVM_EXIT_DIRTY_RING_FULL
before the ring overflows, but the pre-fault loop never checks it, so
one call can push more entries than the ring has free, which trips the
WARN in kvm_dirty_ring_push() and overwrites entries userspace hasn't
harvested. On x86:
WARNING: arch/x86/kvm/../../../virt/kvm/dirty_ring.c:225 at kvm_dirty_ring_push+0x7a/0xd0
Call Trace:
make_spte+0x2cc/0x450
kvm_tdp_mmu_map+0x5e1/0x7f0
kvm_tdp_page_fault+0x12c/0x1b0
kvm_mmu_do_page_fault+0x181/0x250
kvm_arch_vcpu_pre_fault_memory+0x136/0x1d0
kvm_vcpu_pre_fault_memory+0xcf/0x180
kvm_vcpu_ioctl+0x455/0x770
Stop the loop once the ring is soft full, with the check KVM_RUN makes
before entering the guest. Return the pages already processed, or
-ENOSPC when there aren't any, since a retry can't succeed until
userspace harvests and resets the ring. s390 has the ioctl but no ring,
so give kvm_dirty_ring_check_request() a stub.
The check also sets run->exit_reason, and arm64's KVM_RUN completes a
pending MMIO exit only if exit_reason is still KVM_EXIT_MMIO. If a
pre-fault stops on a full ring while an MMIO exit is pending, the next
KVM_RUN doesn't complete the exit and the guest repeats the MMIO
access. Userspace can avoid that by completing the exit with KVM_RUN
before it pre-faults.
Fixes: bc1a5cd002116 ("KVM: Add KVM_PRE_FAULT_MEMORY vcpu ioctl to pre-populate guest memory")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
Documentation/virt/kvm/api.rst | 9 +++++++--
include/linux/kvm_dirty_ring.h | 5 +++++
virt/kvm/kvm_main.c | 5 +++++
3 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index e95c9cda085d9..89eb6be9bc6e3 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -6544,6 +6544,7 @@ Errors:
ENOENT The specified `gpa` is outside defined memslots.
ENOEXEC The vCPU has not been initialised (arm64).
EINTR An unmasked signal is pending and no page was processed.
+ ENOSPC The vCPU's dirty ring is full and no page was processed.
EFAULT The parameter address was invalid.
EHWPOISON A poisoned host page was encountered.
EOPNOTSUPP Mapping memory for a GPA is unsupported by the
@@ -6588,7 +6589,9 @@ case, the ioctl can be called in parallel.
When the ioctl returns, the input values are updated to point to the
remaining range. If `size` > 0 on return, the caller can just issue
-the ioctl again with the same `struct kvm_map_memory` argument.
+the ioctl again with the same `struct kvm_pre_fault_memory` argument.
+If that fails with ENOSPC, the vCPU's dirty ring has to be harvested
+and reset first.
On x86, shadow page tables cannot support this ioctl because they
are indexed by virtual address or nested guest physical address.
@@ -8957,7 +8960,9 @@ Therefore, the ioctl must be called *before* reading the content of
the dirty pages.
The dirty ring can get full. When it happens, the KVM_RUN of the
-vcpu will return with exit reason KVM_EXIT_DIRTY_RING_FULL.
+vcpu will return with exit reason KVM_EXIT_DIRTY_RING_FULL, and
+KVM_PRE_FAULT_MEMORY on the vcpu will process no more pages until the
+ring has been harvested and reset.
The dirty ring interface has a major difference comparing to the
KVM_GET_DIRTY_LOG interface in that, when reading the dirty ring from
diff --git a/include/linux/kvm_dirty_ring.h b/include/linux/kvm_dirty_ring.h
index eb10d87adf7d5..729615ae0cf4d 100644
--- a/include/linux/kvm_dirty_ring.h
+++ b/include/linux/kvm_dirty_ring.h
@@ -60,6 +60,11 @@ static inline void kvm_dirty_ring_push(struct kvm_vcpu *vcpu,
{
}
+static inline bool kvm_dirty_ring_check_request(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+
static inline struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring,
u32 offset)
{
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index cf4cb586f1407..ac8b94a386e4a 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -4461,6 +4461,11 @@ static int kvm_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
break;
}
+ if (kvm_dirty_ring_check_request(vcpu)) {
+ r = -ENOSPC;
+ break;
+ }
+
r = kvm_arch_vcpu_pre_fault_memory(vcpu, range);
if (WARN_ON_ONCE(r == 0 || r == -EIO))
break;
--
2.47.3
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH v2 2/2] KVM: selftests: Test KVM_PRE_FAULT_MEMORY against a full dirty ring
2026-11-17 7:44 [PATCH v2 0/2] KVM: Bound KVM_PRE_FAULT_MEMORY by the vCPU's dirty ring Fuad Tabba
2026-11-17 7:44 ` [PATCH v2 1/2] KVM: Stop KVM_PRE_FAULT_MEMORY when the vCPU's dirty ring is full Fuad Tabba
@ 2026-11-17 7:44 ` Fuad Tabba
1 sibling, 0 replies; 3+ messages in thread
From: Fuad Tabba @ 2026-11-17 7:44 UTC (permalink / raw)
To: Paolo Bonzini, Sean Christopherson
Cc: kvm, linux-kselftest, linux-doc, linux-kernel, Jonathan Corbet,
Shuah Khan, Isaku Yamahata, Rick Edgecombe, Lorenzo Stoakes,
Alexandru Elisei, Marc Zyngier, Oliver Upton, kvmarm,
Wei-Lin Chang, Will Deacon, Fuad Tabba
Pre-fault more populated, dirty-logged pages than the vCPU's dirty ring
holds, and check that each call stops before the ring overflows, that a
retry returns ENOSPC without progress, and that each page is reported
exactly once. A read pre-fault can map a page read-only, or
access-tracked on x86 without EPT A/D bits, so skip the case if
pre-faulting doesn't dirty any page.
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
.../selftests/kvm/pre_fault_memory_test.c | 101 ++++++++++++++++++
1 file changed, 101 insertions(+)
diff --git a/tools/testing/selftests/kvm/pre_fault_memory_test.c b/tools/testing/selftests/kvm/pre_fault_memory_test.c
index 3082fe09b95fa..72c501afc873b 100644
--- a/tools/testing/selftests/kvm/pre_fault_memory_test.c
+++ b/tools/testing/selftests/kvm/pre_fault_memory_test.c
@@ -5,6 +5,9 @@
* Author:
* Isaku Yamahata <isaku.yamahata at gmail.com>
*/
+#include <asm/barrier.h>
+#include <linux/bitmap.h>
+#include <linux/bitops.h>
#include <linux/sizes.h>
#include <test_util.h>
@@ -17,6 +20,8 @@
/* Arbitrarily chosen values */
#define TEST_BASE_SIZE SZ_2M
#define TEST_SLOT 10
+/* Large enough for the entries every architecture reserves. */
+#define TEST_RING_SIZE SZ_64K
/* Storage of test info to share with guest code */
struct test_config {
@@ -281,6 +286,101 @@ static void test_pre_fault_memory(unsigned long vm_type, enum vm_mem_backing_src
for_each_guest_mode(__test_pre_fault_memory, &p);
}
+/*
+ * Pre-fault more dirty-logged pages than the ring holds: each call has to stop
+ * before the ring overflows, and no entry may be lost.
+ */
+static void test_pre_fault_dirty_ring(void)
+{
+ const u64 entries = TEST_RING_SIZE / sizeof(struct kvm_dirty_gfn);
+ const u64 npages = entries + entries / 4;
+ struct kvm_pre_fault_memory range, left;
+ struct kvm_dirty_gfn *gfns, *cur;
+ u64 harvested = 0, size;
+ struct kvm_vcpu *vcpu;
+ unsigned long *seen;
+ int ret, stops = 0;
+ struct kvm_vm *vm;
+ u32 fetch = 0;
+ gpa_t gpa;
+
+ if (!kvm_has_cap(KVM_CAP_DIRTY_LOG_RING) && !kvm_has_cap(KVM_CAP_DIRTY_LOG_RING_ACQ_REL)) {
+ pr_info("Skipping the dirty ring test: no dirty ring\n");
+ return;
+ }
+
+ vm = vm_create(1);
+ vm_enable_dirty_ring(vm, TEST_RING_SIZE);
+ vcpu = vm_vcpu_add(vm, 0, NULL);
+ gfns = vcpu_map_dirty_ring(vcpu);
+
+ /* A ring entry is a host page. */
+ size = npages * getpagesize();
+ gpa = align_down(vm->max_gfn * vm->page_size - size, SZ_2M);
+ vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, gpa, TEST_SLOT,
+ vm_num_guest_pages(vm->mode, npages), KVM_MEM_LOG_DIRTY_PAGES);
+
+ /* Populate the pages, so that pre-faulting can map them writable. */
+ memset(addr_gpa2hva(vm, gpa), 1, size);
+
+ seen = bitmap_zalloc(npages);
+ TEST_ASSERT(seen, "Failed to allocate the bitmap of reported pages");
+
+ range = (struct kvm_pre_fault_memory) { .gpa = gpa, .size = size };
+ while (range.size) {
+ u64 n;
+
+ vcpu_ioctl(vcpu, KVM_PRE_FAULT_MEMORY, &range);
+
+ if (range.size) {
+ left = range;
+ ret = __vcpu_ioctl(vcpu, KVM_PRE_FAULT_MEMORY, &range);
+ TEST_ASSERT_VM_VCPU_IOCTL(ret == -1 && errno == ENOSPC,
+ KVM_PRE_FAULT_MEMORY, ret, vm);
+ TEST_ASSERT(range.gpa == left.gpa && range.size == left.size,
+ "KVM_PRE_FAULT_MEMORY made progress on a full ring");
+ stops++;
+ }
+
+ for (n = 0; ; n++) {
+ cur = &gfns[fetch % entries];
+ /* Pairs with the smp_wmb() in kvm_dirty_ring_push(). */
+ if (smp_load_acquire(&cur->flags) != KVM_DIRTY_GFN_F_DIRTY)
+ break;
+
+ TEST_ASSERT(cur->slot == TEST_SLOT && cur->offset < npages,
+ "Ring entry %u: slot %u offset 0x%llx, expected slot %u offset < 0x%lx",
+ fetch, cur->slot, cur->offset, TEST_SLOT, npages);
+ TEST_ASSERT(!__test_and_set_bit(cur->offset, seen),
+ "Ring entry %u: offset 0x%llx reported twice",
+ fetch, cur->offset);
+ /* Pairs with kvm_dirty_gfn_harvested()'s smp_load_acquire(). */
+ smp_store_release(&cur->flags, KVM_DIRTY_GFN_F_RESET);
+ fetch++;
+ }
+
+ /* A read pre-fault can map a page read-only, or access-tracked on x86. */
+ if (!n && !harvested && !range.size) {
+ pr_info("Skipping the dirty ring test: pre-faulting dirtied no pages\n");
+ bitmap_free(seen);
+ kvm_vm_free(vm);
+ return;
+ }
+ TEST_ASSERT(n, "KVM_PRE_FAULT_MEMORY pushed nothing to the ring");
+ TEST_ASSERT(kvm_vm_reset_dirty_ring(vm) == n,
+ "KVM_RESET_DIRTY_RINGS didn't reset %lu entries", n);
+ harvested += n;
+ }
+
+ TEST_ASSERT(harvested == npages, "%lu ring entries for %lu pages", harvested, npages);
+ TEST_ASSERT(stops, "KVM_PRE_FAULT_MEMORY never stopped for a %lu-entry ring", entries);
+ pr_info("Pre-faulted %lu pages through a %lu-entry ring, stopping %d times\n",
+ npages, entries, stops);
+
+ bitmap_free(seen);
+ kvm_vm_free(vm);
+}
+
static void help(char *name)
{
puts("");
@@ -320,5 +420,6 @@ int main(int argc, char *argv[])
test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, backing, false);
test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, backing, true);
#endif
+ test_pre_fault_dirty_ring();
return 0;
}
--
2.47.3
^ permalink raw reply [flat|nested] 3+ messages in thread