[PATCH 0/5] KVM: riscv: Speed up dirty logging write faults

[PATCH 0/5] KVM: riscv: Speed up dirty logging write faults

[Jinyu Tang]

This series speeds up the common dirty logging write fault path for
RISC-V KVM.

When dirty logging is enabled, guest writes to write-protected G-stage
4K leaves currently go through the full fault path and take mmu_lock for
write.  That is expensive once the leaf already exists and
only needs its permission and dirty state updated.

The series first makes the existing MMU notifier locking contract
explicit, switches RISC-V KVM to KVM's rwlock form of mmu_lock, adds an
atomic G-stage PTE update helper, uses it for existing permission
updates, and finally handles dirty logging write faults under the read
side of mmu_lock when a 4K G-stage leaf is already present.

The locking and concurrency model follows existing KVM patterns from x86
and arm64: use an rwlock mmu_lock, keep structural page-table changes on
the write side, and use atomic PTE updates for permission-only changes
that may race with another vCPU.

The fast path marks the memslot page dirty before making the PTE writable
again.  It also sets the PTE dirty bit, so systems that trap when D is
clear do not fall back to the slow path for a writable-but-clean PTE.

Performance was measured with KVM selftests inside a QEMU RISC-V host.

  dirty_log_perf_test -m 19 -v 4 -b 1G -i 3
    base dirty memory avg, iterations 2-3: 53.262493s
    patch 1-4 dirty memory avg, iterations 2-3: 52.735675s
    patch 1-5 dirty memory avg, iterations 2-3: 26.422964s
    patch 1-5 improvement over base: 50.4%

  dirty_log_perf_test -m 19 -v 4 -b 64M -i 3
    base dirty memory avg, iterations 2-3: 3.203170s
    patch 1-5 dirty memory avg, iterations 2-3: 1.762452s
    improvement: 45.0%

Patch 1-4 are prerequisites for the read-side fast path. The
main speedup comes from patch 5 avoiding the full fault path 
for dirty logging write faults.

Jinyu Tang (5):
  KVM: riscv: Rely on common MMU notifier locking
  KVM: riscv: Use an rwlock for mmu_lock
  KVM: riscv: Add a G-stage PTE cmpxchg helper
  KVM: riscv: Update G-stage PTE permissions atomically
  KVM: riscv: Fast-path dirty logging write faults

 arch/riscv/include/asm/kvm_gstage.h |   4 ++
 arch/riscv/include/asm/kvm_host.h   |   2 +
 arch/riscv/kvm/gstage.c             |  35 +++++++---
 arch/riscv/kvm/mmu.c                | 105 +++++++++++++++++++++++-----
 4 files changed, 122 insertions(+), 24 deletions(-)

-- 
2.43.0

[PATCH 1/5] KVM: riscv: Rely on common MMU notifier locking

[Jinyu Tang]

The common KVM invalidation paths call kvm_unmap_gfn_range() with
mmu_lock already held for write.

For the standard MMU notifier path, the call chain is:

  kvm_mmu_notifier_invalidate_range_start()
    kvm_handle_hva_range()
      kvm_unmap_gfn_range()

kvm_mmu_notifier_invalidate_range_start() leaves range.lockless clear.
kvm_handle_hva_range() therefore takes KVM_MMU_LOCK(kvm) before invoking
the handler.

The guest_memfd path has the same locking contract:

  __kvm_gmem_invalidate_begin()
    kvm_mmu_unmap_gfn_range()
      kvm_unmap_gfn_range()

__kvm_gmem_invalidate_begin() explicitly takes KVM_MMU_LOCK(kvm) before
calling kvm_mmu_unmap_gfn_range().

So remove the local trylock and make the common locking contract explicit 
with lockdep_assert_held_write() like x86.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 arch/riscv/kvm/mmu.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 2d3def024..0197e41fc 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -230,18 +230,16 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
 {
 	struct kvm_gstage gstage;
-	bool mmu_locked;
 
 	if (!kvm->arch.pgd)
 		return false;
 
+	lockdep_assert_held_write(&kvm->mmu_lock);
+
 	kvm_riscv_gstage_init(&gstage, kvm);
-	mmu_locked = spin_trylock(&kvm->mmu_lock);
 	kvm_riscv_gstage_unmap_range(&gstage, range->start << PAGE_SHIFT,
 				     (range->end - range->start) << PAGE_SHIFT,
 				     range->may_block);
-	if (mmu_locked)
-		spin_unlock(&kvm->mmu_lock);
 	return false;
 }
 
-- 
2.43.0

[PATCH 2/5] KVM: riscv: Use an rwlock for mmu_lock

[Jinyu Tang]

RISC-V KVM currently uses a spinlock for mmu_lock. That serializes all
G-stage MMU operations, including permission-only updates that do not
allocate or free page-table pages.

Use KVM's rwlock form of mmu_lock, as x86 and arm64 already do. Keep the
existing map, unmap and teardown paths on the write side. This prepares
RISC-V for read-side handling of G-stage permission updates.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 arch/riscv/include/asm/kvm_host.h |  2 ++
 arch/riscv/kvm/gstage.c           |  2 +-
 arch/riscv/kvm/mmu.c              | 24 ++++++++++++------------
 3 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 75b0a951c..60017ceec 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -48,6 +48,8 @@
 
 #define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS_RANGE
 
+#define KVM_HAVE_MMU_RWLOCK
+
 #define KVM_DIRTY_LOG_MANUAL_CAPS	(KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
 					 KVM_DIRTY_LOG_INITIALLY_SET)
 
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index d9fe8be2a..6f934cb4a 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -410,7 +410,7 @@ void kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
 		 * to prevent starvation and lockup detector warnings.
 		 */
 		if (!(gstage->flags & KVM_GSTAGE_FLAGS_LOCAL) && may_block && addr < end)
-			cond_resched_lock(&gstage->kvm->mmu_lock);
+			cond_resched_rwlock_write(&gstage->kvm->mmu_lock);
 	}
 }
 
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 0197e41fc..48f16e52f 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -26,9 +26,9 @@ static void mmu_wp_memory_region(struct kvm *kvm, int slot)
 
 	kvm_riscv_gstage_init(&gstage, kvm);
 
-	spin_lock(&kvm->mmu_lock);
+	write_lock(&kvm->mmu_lock);
 	kvm_riscv_gstage_wp_range(&gstage, start, end);
-	spin_unlock(&kvm->mmu_lock);
+	write_unlock(&kvm->mmu_lock);
 	kvm_flush_remote_tlbs_memslot(kvm, memslot);
 }
 
@@ -65,9 +65,9 @@ int kvm_riscv_mmu_ioremap(struct kvm *kvm, gpa_t gpa, phys_addr_t hpa,
 		if (ret)
 			goto out;
 
-		spin_lock(&kvm->mmu_lock);
+		write_lock(&kvm->mmu_lock);
 		ret = kvm_riscv_gstage_set_pte(&gstage, &pcache, &map);
-		spin_unlock(&kvm->mmu_lock);
+		write_unlock(&kvm->mmu_lock);
 		if (ret)
 			goto out;
 
@@ -85,9 +85,9 @@ void kvm_riscv_mmu_iounmap(struct kvm *kvm, gpa_t gpa, unsigned long size)
 
 	kvm_riscv_gstage_init(&gstage, kvm);
 
-	spin_lock(&kvm->mmu_lock);
+	write_lock(&kvm->mmu_lock);
 	kvm_riscv_gstage_unmap_range(&gstage, gpa, size, false);
-	spin_unlock(&kvm->mmu_lock);
+	write_unlock(&kvm->mmu_lock);
 }
 
 void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
@@ -131,9 +131,9 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
 
 	kvm_riscv_gstage_init(&gstage, kvm);
 
-	spin_lock(&kvm->mmu_lock);
+	write_lock(&kvm->mmu_lock);
 	kvm_riscv_gstage_unmap_range(&gstage, gpa, size, false);
-	spin_unlock(&kvm->mmu_lock);
+	write_unlock(&kvm->mmu_lock);
 }
 
 void kvm_arch_commit_memory_region(struct kvm *kvm,
@@ -504,7 +504,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 	if (logging && !is_write)
 		writable = false;
 
-	spin_lock(&kvm->mmu_lock);
+	write_lock(&kvm->mmu_lock);
 
 	if (mmu_invalidate_retry(kvm, mmu_seq))
 		goto out_unlock;
@@ -527,7 +527,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 
 out_unlock:
 	kvm_release_faultin_page(kvm, page, ret && ret != -EEXIST, writable);
-	spin_unlock(&kvm->mmu_lock);
+	write_unlock(&kvm->mmu_lock);
 	return ret;
 }
 
@@ -556,7 +556,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
 	struct kvm_gstage gstage;
 	void *pgd = NULL;
 
-	spin_lock(&kvm->mmu_lock);
+	write_lock(&kvm->mmu_lock);
 	if (kvm->arch.pgd) {
 		kvm_riscv_gstage_init(&gstage, kvm);
 		kvm_riscv_gstage_unmap_range(&gstage, 0UL,
@@ -566,7 +566,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
 		kvm->arch.pgd_phys = 0;
 		kvm->arch.pgd_levels = 0;
 	}
-	spin_unlock(&kvm->mmu_lock);
+	write_unlock(&kvm->mmu_lock);
 
 	if (pgd)
 		free_pages((unsigned long)pgd, get_order(kvm_riscv_gstage_pgd_size));
-- 
2.43.0

[PATCH 3/5] KVM: riscv: Add a G-stage PTE cmpxchg helper

[Jinyu Tang]

Permission-only G-stage PTE updates can run in parallel once they are
moved to the read side of mmu_lock. Plain set_pte() is not enough for
that case because another CPU may update the same PTE first.

x86 handles the same class of SPTE races with cmpxchg-based updates in
its fast page fault and TDP MMU paths. Add a small RISC-V helper for
atomic G-stage PTE updates. The helper reports contention to the caller
and flushes the target range only when the PTE value actually changes.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 arch/riscv/include/asm/kvm_gstage.h |  4 ++++
 arch/riscv/kvm/gstage.c             | 14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index 9c908432b..afe80e4bf 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -54,6 +54,10 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
 			     struct kvm_mmu_memory_cache *pcache,
 			     const struct kvm_gstage_mapping *map);
 
+bool kvm_riscv_gstage_try_update_pte(struct kvm_gstage *gstage, u32 level,
+				     gpa_t addr, pte_t *ptep,
+				     pte_t old_pte, pte_t new_pte);
+
 int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
 			      struct kvm_mmu_memory_cache *pcache,
 			      gpa_t gpa, phys_addr_t hpa, unsigned long page_size,
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index 6f934cb4a..d70584b9e 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -123,6 +123,20 @@ static void gstage_tlb_flush(struct kvm_gstage *gstage, u32 level, gpa_t addr)
 					       gstage->vmid);
 }
 
+bool kvm_riscv_gstage_try_update_pte(struct kvm_gstage *gstage, u32 level,
+				     gpa_t addr, pte_t *ptep,
+				     pte_t old_pte, pte_t new_pte)
+{
+	if (cmpxchg(&ptep->pte, pte_val(old_pte), pte_val(new_pte)) !=
+	    pte_val(old_pte))
+		return false;
+
+	if (pte_val(old_pte) != pte_val(new_pte))
+		gstage_tlb_flush(gstage, level, addr);
+
+	return true;
+}
+
 int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
 			     struct kvm_mmu_memory_cache *pcache,
 			     const struct kvm_gstage_mapping *map)
-- 
2.43.0

[PATCH 4/5] KVM: riscv: Update G-stage PTE permissions atomically

[Jinyu Tang]

When a fault hits an existing G-stage leaf with the same PFN, KVM only
needs to update the PTE permissions. This path will be used by read-side
fault handling, so it must not overwrite a concurrent PTE update.

Use the cmpxchg helper when relaxing permissions on an existing leaf,
following the same concurrency model used by x86 for atomic SPTE
permission updates. Retry if another CPU changed the PTE first, and use
cpu_relax() while spinning.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 arch/riscv/kvm/gstage.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index d70584b9e..9595169da 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -182,17 +182,22 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
 static void kvm_riscv_gstage_update_pte_prot(struct kvm_gstage *gstage, u32 level,
 					     gpa_t addr, pte_t *ptep, pgprot_t prot)
 {
-	pte_t new_pte;
+	pte_t old_pte, new_pte;
 
-	if (pgprot_val(pte_pgprot(ptep_get(ptep))) == pgprot_val(prot))
-		return;
+	for (;;) {
+		old_pte = ptep_get(ptep);
+		if (pgprot_val(pte_pgprot(old_pte)) == pgprot_val(prot))
+			return;
 
-	new_pte = pfn_pte(pte_pfn(ptep_get(ptep)), prot);
-	new_pte = pte_mkdirty(new_pte);
+		new_pte = pfn_pte(pte_pfn(old_pte), prot);
+		new_pte = pte_mkdirty(new_pte);
 
-	set_pte(ptep, new_pte);
+		if (kvm_riscv_gstage_try_update_pte(gstage, level, addr, ptep,
+						    old_pte, new_pte))
+			return;
 
-	gstage_tlb_flush(gstage, level, addr);
+		cpu_relax();
+	}
 }
 
 int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
-- 
2.43.0

[PATCH 5/5] KVM: riscv: Fast-path dirty logging write faults

[Jinyu Tang]

With dirty logging enabled, guest writes often fault on an existing 4K
G-stage leaf that was write-protected only for dirty tracking. The slow
path still performs the full fault handling flow and takes mmu_lock for
write, even though the page-table shape does not change.

x86 handles the analogous case in its fast page fault path by atomically
making a writable SPTE writable again when the fault is only a
write-protection fault. Add the same style of fast path for RISC-V. If a
write fault hits an existing 4K leaf in a writable dirty-log memslot,
mark the page dirty and atomically set the PTE writable and dirty under
the read side of mmu_lock.

The dirty bitmap is updated before the PTE becomes writable again. The
PTE D bit is also set so systems that trap on a clear D bit do not fall
back to the slow path for a writable but clean PTE.

Signed-off-by: Jinyu Tang <tjytimi@163.com>
---
 arch/riscv/kvm/mmu.c | 75 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 48f16e52f..980059e09 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -419,6 +419,77 @@ static unsigned long transparent_hugepage_adjust(struct kvm *kvm,
 	return PAGE_SIZE;
 }
 
+static bool kvm_riscv_mmu_dirty_log_write_fault_fast(struct kvm *kvm,
+						     struct kvm_memory_slot *memslot,
+						     gpa_t gpa,
+						     struct kvm_gstage_mapping *out_map)
+{
+	struct kvm_gstage gstage;
+	unsigned long mmu_seq;
+	pte_t old_pte, new_pte;
+	pte_t *ptep;
+	gfn_t gfn = gpa >> PAGE_SHIFT;
+	u32 ptep_level;
+	bool dirty_marked = false;
+	bool ret;
+
+	kvm_riscv_gstage_init(&gstage, kvm);
+	mmu_seq = kvm->mmu_invalidate_seq;
+
+	read_lock(&kvm->mmu_lock);
+
+	if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) {
+		ret = false;
+		goto out_unlock;
+	}
+
+	if (!kvm_riscv_gstage_get_leaf(&gstage, gpa, &ptep, &ptep_level) ||
+	    ptep_level) {
+		ret = false;
+		goto out_unlock;
+	}
+
+	for (;;) {
+		old_pte = ptep_get(ptep);
+		if (!(pte_val(old_pte) & _PAGE_LEAF)) {
+			ret = false;
+			break;
+		}
+
+		if (!dirty_marked) {
+			mark_page_dirty_in_slot(kvm, memslot, gfn);
+			dirty_marked = true;
+		}
+
+		if ((pte_val(old_pte) & (_PAGE_WRITE | _PAGE_DIRTY)) ==
+		    (_PAGE_WRITE | _PAGE_DIRTY)) {
+			new_pte = old_pte;
+			ret = true;
+			break;
+		}
+
+		new_pte = pte_mkdirty(pte_mkwrite_novma(old_pte));
+
+		if (kvm_riscv_gstage_try_update_pte(&gstage, ptep_level, gpa,
+						    ptep, old_pte, new_pte)) {
+			ret = true;
+			break;
+		}
+		cpu_relax();
+	}
+
+out_unlock:
+	read_unlock(&kvm->mmu_lock);
+
+	if (ret) {
+		out_map->addr = gpa & PAGE_MASK;
+		out_map->level = 0;
+		out_map->pte = new_pte;
+	}
+
+	return ret;
+}
+
 int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 		      gpa_t gpa, unsigned long hva, bool is_write,
 		      struct kvm_gstage_mapping *out_map)
@@ -442,6 +513,10 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
 	/* Setup initial state of output mapping */
 	memset(out_map, 0, sizeof(*out_map));
 
+	if (is_write && logging &&
+	    kvm_riscv_mmu_dirty_log_write_fault_fast(kvm, memslot, gpa, out_map))
+		return 0;
+
 	/* We need minimum second+third level pages */
 	ret = kvm_mmu_topup_memory_cache(pcache, kvm->arch.pgd_levels);
 	if (ret) {
-- 
2.43.0

Re: [PATCH 5/5] KVM: riscv: Fast-path dirty logging write faults

[Inochi Amaoto]

On Sun, May 17, 2026 at 11:34:27PM +0800, Jinyu Tang wrote:
> With dirty logging enabled, guest writes often fault on an existing 4K
> G-stage leaf that was write-protected only for dirty tracking. The slow
> path still performs the full fault handling flow and takes mmu_lock for
> write, even though the page-table shape does not change.
> 
> x86 handles the analogous case in its fast page fault path by atomically
> making a writable SPTE writable again when the fault is only a
> write-protection fault. Add the same style of fast path for RISC-V. If a
> write fault hits an existing 4K leaf in a writable dirty-log memslot,
> mark the page dirty and atomically set the PTE writable and dirty under
> the read side of mmu_lock.
> 
> The dirty bitmap is updated before the PTE becomes writable again. The
> PTE D bit is also set so systems that trap on a clear D bit do not fall
> back to the slow path for a writable but clean PTE.
> 
> Signed-off-by: Jinyu Tang <tjytimi@163.com>
> ---
>  arch/riscv/kvm/mmu.c | 75 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 75 insertions(+)
> 
> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
> index 48f16e52f..980059e09 100644
> --- a/arch/riscv/kvm/mmu.c
> +++ b/arch/riscv/kvm/mmu.c
> @@ -419,6 +419,77 @@ static unsigned long transparent_hugepage_adjust(struct kvm *kvm,
>  	return PAGE_SIZE;
>  }
>  
> +static bool kvm_riscv_mmu_dirty_log_write_fault_fast(struct kvm *kvm,
> +						     struct kvm_memory_slot *memslot,
> +						     gpa_t gpa,
> +						     struct kvm_gstage_mapping *out_map)
> +{
> +	struct kvm_gstage gstage;
> +	unsigned long mmu_seq;
> +	pte_t old_pte, new_pte;
> +	pte_t *ptep;
> +	gfn_t gfn = gpa >> PAGE_SHIFT;
> +	u32 ptep_level;
> +	bool dirty_marked = false;
> +	bool ret;
> +
> +	kvm_riscv_gstage_init(&gstage, kvm);
> +	mmu_seq = kvm->mmu_invalidate_seq;
> +
> +	read_lock(&kvm->mmu_lock);
> +
> +	if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) {
> +		ret = false;
> +		goto out_unlock;
> +	}
> +

> +	if (!kvm_riscv_gstage_get_leaf(&gstage, gpa, &ptep, &ptep_level) ||
> +	    ptep_level) {
> +		ret = false;
> +		goto out_unlock;
> +	}
> +

Add a check here: if this is a huge page and is a logging page, 
we should fallback to slow path to apply page spltting.

> +	for (;;) {
> +		old_pte = ptep_get(ptep);
> +		if (!(pte_val(old_pte) & _PAGE_LEAF)) {

Use pmd_present() here.

> +			ret = false;
> +			break;
> +		}
> +

> +		if (!dirty_marked) {
> +			mark_page_dirty_in_slot(kvm, memslot, gfn);
> +			dirty_marked = true;
> +		}

Only log dirty entry when the updating is success, otherwise
the page could be record twice in both fast and slow path.

> +
> +		if ((pte_val(old_pte) & (_PAGE_WRITE | _PAGE_DIRTY)) ==
> +		    (_PAGE_WRITE | _PAGE_DIRTY)) {

I think only pte_write is required, for write-protected path,
only write permission is needed to be checked, as we always
set dirty bit. For the future dirty log hardware extension,
the Svadu will update the dirty bit so no need for this logic.

> +			new_pte = old_pte;
> +			ret = true;
> +			break;
> +		}
> +

> +		new_pte = pte_mkdirty(pte_mkwrite_novma(old_pte));

pte_mkyoung is also needed.

> +
> +		if (kvm_riscv_gstage_try_update_pte(&gstage, ptep_level, gpa,
> +						    ptep, old_pte, new_pte)) {
> +			ret = true;
> +			break;
> +		}
> +		cpu_relax();
> +	}
> +
> +out_unlock:
> +	read_unlock(&kvm->mmu_lock);
> +
> +	if (ret) {
> +		out_map->addr = gpa & PAGE_MASK;
> +		out_map->level = 0;
> +		out_map->pte = new_pte;
> +	}
> +
> +	return ret;
> +}
> +
>  int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
>  		      gpa_t gpa, unsigned long hva, bool is_write,
>  		      struct kvm_gstage_mapping *out_map)
> @@ -442,6 +513,10 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
>  	/* Setup initial state of output mapping */
>  	memset(out_map, 0, sizeof(*out_map));
>  
> +	if (is_write && logging &&
> +	    kvm_riscv_mmu_dirty_log_write_fault_fast(kvm, memslot, gpa, out_map))
> +		return 0;
> +
>  	/* We need minimum second+third level pages */
>  	ret = kvm_mmu_topup_memory_cache(pcache, kvm->arch.pgd_levels);
>  	if (ret) {
> -- 
> 2.43.0
> 

Regards,
Inochi

Re: [PATCH 0/5] KVM: riscv: Speed up dirty logging write faults

[Anup Patel]

On Sun, May 17, 2026 at 9:05 PM Jinyu Tang <tjytimi@163.com> wrote:
>
> This series speeds up the common dirty logging write fault path for
> RISC-V KVM.
>
> When dirty logging is enabled, guest writes to write-protected G-stage
> 4K leaves currently go through the full fault path and take mmu_lock for
> write.  That is expensive once the leaf already exists and
> only needs its permission and dirty state updated.
>
> The series first makes the existing MMU notifier locking contract
> explicit, switches RISC-V KVM to KVM's rwlock form of mmu_lock, adds an
> atomic G-stage PTE update helper, uses it for existing permission
> updates, and finally handles dirty logging write faults under the read
> side of mmu_lock when a 4K G-stage leaf is already present.
>
> The locking and concurrency model follows existing KVM patterns from x86
> and arm64: use an rwlock mmu_lock, keep structural page-table changes on
> the write side, and use atomic PTE updates for permission-only changes
> that may race with another vCPU.
>
> The fast path marks the memslot page dirty before making the PTE writable
> again.  It also sets the PTE dirty bit, so systems that trap when D is
> clear do not fall back to the slow path for a writable-but-clean PTE.
>
> Performance was measured with KVM selftests inside a QEMU RISC-V host.
>
>   dirty_log_perf_test -m 19 -v 4 -b 1G -i 3
>     base dirty memory avg, iterations 2-3: 53.262493s
>     patch 1-4 dirty memory avg, iterations 2-3: 52.735675s
>     patch 1-5 dirty memory avg, iterations 2-3: 26.422964s
>     patch 1-5 improvement over base: 50.4%
>
>   dirty_log_perf_test -m 19 -v 4 -b 64M -i 3
>     base dirty memory avg, iterations 2-3: 3.203170s
>     patch 1-5 dirty memory avg, iterations 2-3: 1.762452s
>     improvement: 45.0%
>
> Patch 1-4 are prerequisites for the read-side fast path. The
> main speedup comes from patch 5 avoiding the full fault path
> for dirty logging write faults.
>
> Jinyu Tang (5):
>   KVM: riscv: Rely on common MMU notifier locking
>   KVM: riscv: Use an rwlock for mmu_lock
>   KVM: riscv: Add a G-stage PTE cmpxchg helper
>   KVM: riscv: Update G-stage PTE permissions atomically
>   KVM: riscv: Fast-path dirty logging write faults
>
>  arch/riscv/include/asm/kvm_gstage.h |   4 ++
>  arch/riscv/include/asm/kvm_host.h   |   2 +
>  arch/riscv/kvm/gstage.c             |  35 +++++++---
>  arch/riscv/kvm/mmu.c                | 105 +++++++++++++++++++++++-----
>  4 files changed, 122 insertions(+), 24 deletions(-)
>
> --
> 2.43.0
>

LGTM. I tried this at my end as well.

Reviewed-by: Anup Patel <anup@brainfault.org>

Queued this series for Linux-7.2

Thanks,
Anup

Re: [PATCH 0/5] KVM: riscv: Speed up dirty logging write faults

[patchwork-bot+linux-riscv]

Hello:

This series was applied to riscv/linux.git (fixes)
by Anup Patel <anup@brainfault.org>:

On Sun, 17 May 2026 23:34:22 +0800 you wrote:
> This series speeds up the common dirty logging write fault path for
> RISC-V KVM.
> 
> When dirty logging is enabled, guest writes to write-protected G-stage
> 4K leaves currently go through the full fault path and take mmu_lock for
> write.  That is expensive once the leaf already exists and
> only needs its permission and dirty state updated.
> 
> [...]

Here is the summary with links:
  - [1/5] KVM: riscv: Rely on common MMU notifier locking
    (no matching commit)
  - [2/5] KVM: riscv: Use an rwlock for mmu_lock
    (no matching commit)
  - [3/5] KVM: riscv: Add a G-stage PTE cmpxchg helper
    https://git.kernel.org/riscv/c/7dd416fdd3fb
  - [4/5] KVM: riscv: Update G-stage PTE permissions atomically
    https://git.kernel.org/riscv/c/d7a26a0ba715
  - [5/5] KVM: riscv: Fast-path dirty logging write faults
    https://git.kernel.org/riscv/c/7705be59eb2d

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html