[PATCH] RISC-V: KVM: Fix G-stage page table memory leak on VM destroy

[PATCH] RISC-V: KVM: Fix G-stage page table memory leak on VM destroy

[Yangyu Chen]

kvm_riscv_mmu_free_pgd() tears down the G-stage with
kvm_riscv_gstage_unmap_range() over the whole guest physical address
space. That function looks up each address with
kvm_riscv_gstage_get_leaf(), which walks down to the leaf PTE, and then
clears only that leaf. kvm_riscv_gstage_op_pte() frees a page table only
when it is called on a non-leaf PTE, which never happens here, so every
page table below the PGD is leaked when a VM is destroyed. With 4 KiB
mappings this is one page per 2 MiB of guest memory that was ever
mapped, plus the upper-level tables.

The leaked pages are not slab, vmalloc or page cache, so they show up
only as unaccounted used memory. On a SpacemiT K3 host, a workload that
creates and destroys tens of thousands of small VMs leaked 0.5-1 MiB per
VM and ran the host out of memory after about 20,000 VMs.

Add kvm_riscv_gstage_unmap_all(), which calls kvm_riscv_gstage_op_pte()
with GSTAGE_OP_CLEAR on every PGD entry. That recursively clears all
leaf PTEs and frees each page table below the PGD. Use it in
kvm_riscv_mmu_free_pgd().

Tested with a reproducer script that builds a minimal KVM host kernel
from a source tree and boots it under QEMU 11.0.2 (TCG, rv64 with H,
Sv57x4 G-stage). Its init creates 10 VMs one after another. Each guest
writes one word in every 2 MiB of an 8 GiB RAM region (anonymous memory
with MADV_NOHUGEPAGE, so 4 KiB G-stage mappings) and then shuts down,
so each VM needs 4096 last-level page tables (16 MiB). MemFree lost per
VM, after one warm-up VM:

  before:  16415 kB  (RESULT: LEAK)
  after:    1636 kB  (RESULT: OK, noise)

The bug was found while investigating the host OOM, and the fix and
reproducer were written with the help of an AI coding assistant
(Claude). The analysis, fix and test results above were reviewed by
the submitter.

Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming")
Cc: stable@vger.kernel.org
Signed-off-by: Yangyu Chen <cyy@cyyself.name>
---
How to reproduce:

The reproducer script is at
https://gist.github.com/cyyself/23abb87a8ad9ee8c64ce03cca78669fc
and needs riscv64-linux-gnu-gcc, qemu-system-riscv64 and cpio. Run it
from the top of a kernel source tree:

	$ cd linux
	$ /path/to/kvm-riscv-gstage-leak.sh

It builds a minimal riscv64 KVM host kernel (allnoconfig + KVM) into
.kvm-leak-build/ and boots it under QEMU (TCG, -cpu rv64,h=true) with a
static test program as /init. The test creates 10 VMs one after another.
Each guest touches one page in every 2 MiB of 8 GiB of guest RAM, so it
needs 16 MiB of last-level G-stage page tables. The test then prints how
much MemFree each VM leaves behind. The first run takes about 1.5
minutes, mostly kernel build; reruns take about 10 seconds.

Without this patch:

	10 VMs, each touching one 4 KiB page per 2 MiB of 8 GiB guest RAM
	MemFree 2037664 -> 1873512 kB: 16415 kB lost per VM (a leaking kernel loses ~16384 kB)
	RESULT: LEAK

With this patch:

	MemFree 2054080 -> 2037716 kB: 1636 kB lost per VM (a leaking kernel loses ~16384 kB)
	RESULT: OK

The exit status is 0 for OK, 1 for LEAK and 2 if the test could not
run. IMAGE=<path/to/Image> boots an existing kernel instead of building
one. That kernel needs CONFIG_KVM=y, CONFIG_BLK_DEV_INITRD=y and
CONFIG_DEVTMPFS=y.

I also tested VMs that select Sv39x4, Sv48x4 and Sv57x4 through
KVM_CAP_VM_GPA_BITS, and four VMs with mixed modes alive at the same
time, destroyed one by one while the remaining VMs kept running and
were checked for correct guest memory contents. With this patch, no
memory was lost in any mode and no guest data was corrupted. Without
it, every mode leaked about 16 MiB per VM.
---
 arch/riscv/include/asm/kvm_gstage.h |  2 ++
 arch/riscv/kvm/gstage.c             | 22 ++++++++++++++++++++++
 arch/riscv/kvm/mmu.c                |  3 +--
 3 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index aaf080ba1b77..90ec9e0b34b1 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -80,6 +80,8 @@ bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
 bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
 				  gpa_t start, gpa_t size, bool may_block);
 
+bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage);
+
 bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end);
 
 bool kvm_riscv_gstage_wp_pt_masked(struct kvm_gstage *gstage, gfn_t base_gfn,
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index e5002cb9cbef..c912cdc2021f 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -456,6 +456,28 @@ bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
 	return flush;
 }
 
+/*
+ * Clear every G-stage mapping and free all page tables below the PGD.
+ * kvm_riscv_gstage_unmap_range() only clears leaf PTEs and leaves the
+ * page tables in place, so it cannot be used to tear down a G-stage.
+ */
+bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage)
+{
+	u32 level = gstage->pgd_levels - 1;
+	pte_t *pgd = (pte_t *)gstage->pgd;
+	unsigned long i, page_size;
+	bool flush = false;
+
+	if (gstage_level_to_page_size(gstage, level, &page_size))
+		return false;
+
+	for (i = 0; i < kvm_riscv_gstage_pgd_size / sizeof(pte_t); i++)
+		flush |= kvm_riscv_gstage_op_pte(gstage, i * page_size, &pgd[i],
+						 level, GSTAGE_OP_CLEAR);
+
+	return flush;
+}
+
 bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end)
 {
 	int ret;
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 371fccaf9df0..87838a0f6bb9 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -784,8 +784,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
 	write_lock(&kvm->mmu_lock);
 	if (kvm->arch.pgd) {
 		kvm_riscv_gstage_init(&gstage, kvm);
-		flush = kvm_riscv_gstage_unmap_range(&gstage, 0UL,
-			kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels), false);
+		flush = kvm_riscv_gstage_unmap_all(&gstage);
 		pgd = READ_ONCE(kvm->arch.pgd);
 		kvm->arch.pgd = NULL;
 		kvm->arch.pgd_phys = 0;
-- 
2.53.0

Re: [PATCH] RISC-V: KVM: Fix G-stage page table memory leak on VM destroy

[James Morris]

On Sat, 31 Oct 2026, Yangyu Chen wrote:

> Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yangyu Chen <cyy@cyyself.name>

Tested-by: James Morris <jmorris@namei.org>

Assisted by an LLM.

On a SpacemiT K3 CoM260 host (Sv39x4), a native port of your reproducer 
showed about 16 MiB lost per VM on Fedora's 7.3.1 kernel. With the same 
source rebuilt with this patch, each run averaged under 0.1 MiB lost per 
VM (130 measured VMs total), consistent with noise. Both figures count 
free pages held on the per-CPU lists as free.


-- 
James Morris
<jmorris@namei.org>

Re: [PATCH] RISC-V: KVM: Fix G-stage page table memory leak on VM destroy

[Anup Patel]

On Sat, Oct 31, 2026 at 12:28 AM Yangyu Chen <cyy@cyyself.name> wrote:
>
> kvm_riscv_mmu_free_pgd() tears down the G-stage with
> kvm_riscv_gstage_unmap_range() over the whole guest physical address
> space. That function looks up each address with
> kvm_riscv_gstage_get_leaf(), which walks down to the leaf PTE, and then
> clears only that leaf. kvm_riscv_gstage_op_pte() frees a page table only
> when it is called on a non-leaf PTE, which never happens here, so every
> page table below the PGD is leaked when a VM is destroyed. With 4 KiB
> mappings this is one page per 2 MiB of guest memory that was ever
> mapped, plus the upper-level tables.
>
> The leaked pages are not slab, vmalloc or page cache, so they show up
> only as unaccounted used memory. On a SpacemiT K3 host, a workload that
> creates and destroys tens of thousands of small VMs leaked 0.5-1 MiB per
> VM and ran the host out of memory after about 20,000 VMs.
>
> Add kvm_riscv_gstage_unmap_all(), which calls kvm_riscv_gstage_op_pte()
> with GSTAGE_OP_CLEAR on every PGD entry. That recursively clears all
> leaf PTEs and frees each page table below the PGD. Use it in
> kvm_riscv_mmu_free_pgd().
>
> Tested with a reproducer script that builds a minimal KVM host kernel
> from a source tree and boots it under QEMU 11.0.2 (TCG, rv64 with H,
> Sv57x4 G-stage). Its init creates 10 VMs one after another. Each guest
> writes one word in every 2 MiB of an 8 GiB RAM region (anonymous memory
> with MADV_NOHUGEPAGE, so 4 KiB G-stage mappings) and then shuts down,
> so each VM needs 4096 last-level page tables (16 MiB). MemFree lost per
> VM, after one warm-up VM:
>
>   before:  16415 kB  (RESULT: LEAK)
>   after:    1636 kB  (RESULT: OK, noise)
>
> The bug was found while investigating the host OOM, and the fix and
> reproducer were written with the help of an AI coding assistant
> (Claude). The analysis, fix and test results above were reviewed by
> the submitter.
>
> Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yangyu Chen <cyy@cyyself.name>

Good catch. Thanks for the fix.

Reviewed-by: Anup Patel <anup@brainfault.org>

Queued this patch as a fix for Linux-7.4-rcX

Thanks,
Anup

> ---
> How to reproduce:
>
> The reproducer script is at
> https://gist.github.com/cyyself/23abb87a8ad9ee8c64ce03cca78669fc
> and needs riscv64-linux-gnu-gcc, qemu-system-riscv64 and cpio. Run it
> from the top of a kernel source tree:
>
>         $ cd linux
>         $ /path/to/kvm-riscv-gstage-leak.sh
>
> It builds a minimal riscv64 KVM host kernel (allnoconfig + KVM) into
> .kvm-leak-build/ and boots it under QEMU (TCG, -cpu rv64,h=true) with a
> static test program as /init. The test creates 10 VMs one after another.
> Each guest touches one page in every 2 MiB of 8 GiB of guest RAM, so it
> needs 16 MiB of last-level G-stage page tables. The test then prints how
> much MemFree each VM leaves behind. The first run takes about 1.5
> minutes, mostly kernel build; reruns take about 10 seconds.
>
> Without this patch:
>
>         10 VMs, each touching one 4 KiB page per 2 MiB of 8 GiB guest RAM
>         MemFree 2037664 -> 1873512 kB: 16415 kB lost per VM (a leaking kernel loses ~16384 kB)
>         RESULT: LEAK
>
> With this patch:
>
>         MemFree 2054080 -> 2037716 kB: 1636 kB lost per VM (a leaking kernel loses ~16384 kB)
>         RESULT: OK
>
> The exit status is 0 for OK, 1 for LEAK and 2 if the test could not
> run. IMAGE=<path/to/Image> boots an existing kernel instead of building
> one. That kernel needs CONFIG_KVM=y, CONFIG_BLK_DEV_INITRD=y and
> CONFIG_DEVTMPFS=y.
>
> I also tested VMs that select Sv39x4, Sv48x4 and Sv57x4 through
> KVM_CAP_VM_GPA_BITS, and four VMs with mixed modes alive at the same
> time, destroyed one by one while the remaining VMs kept running and
> were checked for correct guest memory contents. With this patch, no
> memory was lost in any mode and no guest data was corrupted. Without
> it, every mode leaked about 16 MiB per VM.
> ---
>  arch/riscv/include/asm/kvm_gstage.h |  2 ++
>  arch/riscv/kvm/gstage.c             | 22 ++++++++++++++++++++++
>  arch/riscv/kvm/mmu.c                |  3 +--
>  3 files changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
> index aaf080ba1b77..90ec9e0b34b1 100644
> --- a/arch/riscv/include/asm/kvm_gstage.h
> +++ b/arch/riscv/include/asm/kvm_gstage.h
> @@ -80,6 +80,8 @@ bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
>  bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
>                                   gpa_t start, gpa_t size, bool may_block);
>
> +bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage);
> +
>  bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end);
>
>  bool kvm_riscv_gstage_wp_pt_masked(struct kvm_gstage *gstage, gfn_t base_gfn,
> diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
> index e5002cb9cbef..c912cdc2021f 100644
> --- a/arch/riscv/kvm/gstage.c
> +++ b/arch/riscv/kvm/gstage.c
> @@ -456,6 +456,28 @@ bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
>         return flush;
>  }
>
> +/*
> + * Clear every G-stage mapping and free all page tables below the PGD.
> + * kvm_riscv_gstage_unmap_range() only clears leaf PTEs and leaves the
> + * page tables in place, so it cannot be used to tear down a G-stage.
> + */
> +bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage)
> +{
> +       u32 level = gstage->pgd_levels - 1;
> +       pte_t *pgd = (pte_t *)gstage->pgd;
> +       unsigned long i, page_size;
> +       bool flush = false;
> +
> +       if (gstage_level_to_page_size(gstage, level, &page_size))
> +               return false;
> +
> +       for (i = 0; i < kvm_riscv_gstage_pgd_size / sizeof(pte_t); i++)
> +               flush |= kvm_riscv_gstage_op_pte(gstage, i * page_size, &pgd[i],
> +                                                level, GSTAGE_OP_CLEAR);
> +
> +       return flush;
> +}
> +
>  bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end)
>  {
>         int ret;
> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
> index 371fccaf9df0..87838a0f6bb9 100644
> --- a/arch/riscv/kvm/mmu.c
> +++ b/arch/riscv/kvm/mmu.c
> @@ -784,8 +784,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
>         write_lock(&kvm->mmu_lock);
>         if (kvm->arch.pgd) {
>                 kvm_riscv_gstage_init(&gstage, kvm);
> -               flush = kvm_riscv_gstage_unmap_range(&gstage, 0UL,
> -                       kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels), false);
> +               flush = kvm_riscv_gstage_unmap_all(&gstage);
>                 pgd = READ_ONCE(kvm->arch.pgd);
>                 kvm->arch.pgd = NULL;
>                 kvm->arch.pgd_phys = 0;
> --
> 2.53.0
>

Re: [PATCH] RISC-V: KVM: Fix G-stage page table memory leak on VM destroy

[Yangyu Chen]

> On 13 Nov 2026, at 20:16, Anup Patel <anup@brainfault.org> wrote:
> 
> On Sat, Oct 31, 2026 at 12:28 AM Yangyu Chen <cyy@cyyself.name> wrote:
>> 
>> kvm_riscv_mmu_free_pgd() tears down the G-stage with
>> kvm_riscv_gstage_unmap_range() over the whole guest physical address
>> space. That function looks up each address with
>> kvm_riscv_gstage_get_leaf(), which walks down to the leaf PTE, and then
>> clears only that leaf. kvm_riscv_gstage_op_pte() frees a page table only
>> when it is called on a non-leaf PTE, which never happens here, so every
>> page table below the PGD is leaked when a VM is destroyed. With 4 KiB
>> mappings this is one page per 2 MiB of guest memory that was ever
>> mapped, plus the upper-level tables.
>> 
>> The leaked pages are not slab, vmalloc or page cache, so they show up
>> only as unaccounted used memory. On a SpacemiT K3 host, a workload that
>> creates and destroys tens of thousands of small VMs leaked 0.5-1 MiB per
>> VM and ran the host out of memory after about 20,000 VMs.
>> 
>> Add kvm_riscv_gstage_unmap_all(), which calls kvm_riscv_gstage_op_pte()
>> with GSTAGE_OP_CLEAR on every PGD entry. That recursively clears all
>> leaf PTEs and frees each page table below the PGD. Use it in
>> kvm_riscv_mmu_free_pgd().
>> 
>> Tested with a reproducer script that builds a minimal KVM host kernel
>> from a source tree and boots it under QEMU 11.0.2 (TCG, rv64 with H,
>> Sv57x4 G-stage). Its init creates 10 VMs one after another. Each guest
>> writes one word in every 2 MiB of an 8 GiB RAM region (anonymous memory
>> with MADV_NOHUGEPAGE, so 4 KiB G-stage mappings) and then shuts down,
>> so each VM needs 4096 last-level page tables (16 MiB). MemFree lost per
>> VM, after one warm-up VM:
>> 
>>  before:  16415 kB  (RESULT: LEAK)
>>  after:    1636 kB  (RESULT: OK, noise)
>> 
>> The bug was found while investigating the host OOM, and the fix and
>> reproducer were written with the help of an AI coding assistant
>> (Claude). The analysis, fix and test results above were reviewed by
>> the submitter.
>> 
>> Fixes: 9d05c1fee837 ("RISC-V: KVM: Implement stage2 page table programming")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Yangyu Chen <cyy@cyyself.name>
> 
> Good catch. Thanks for the fix.
> 
> Reviewed-by: Anup Patel <anup@brainfault.org>
> 
> Queued this patch as a fix for Linux-7.4-rcX
> 

I think this should also be backported to linux-stable.

Thanks,
Yangyu Chen

> Thanks,
> Anup
> 
>> ---
>> How to reproduce:
>> 
>> The reproducer script is at
>> https://gist.github.com/cyyself/23abb87a8ad9ee8c64ce03cca78669fc
>> and needs riscv64-linux-gnu-gcc, qemu-system-riscv64 and cpio. Run it
>> from the top of a kernel source tree:
>> 
>>        $ cd linux
>>        $ /path/to/kvm-riscv-gstage-leak.sh
>> 
>> It builds a minimal riscv64 KVM host kernel (allnoconfig + KVM) into
>> .kvm-leak-build/ and boots it under QEMU (TCG, -cpu rv64,h=true) with a
>> static test program as /init. The test creates 10 VMs one after another.
>> Each guest touches one page in every 2 MiB of 8 GiB of guest RAM, so it
>> needs 16 MiB of last-level G-stage page tables. The test then prints how
>> much MemFree each VM leaves behind. The first run takes about 1.5
>> minutes, mostly kernel build; reruns take about 10 seconds.
>> 
>> Without this patch:
>> 
>>        10 VMs, each touching one 4 KiB page per 2 MiB of 8 GiB guest RAM
>>        MemFree 2037664 -> 1873512 kB: 16415 kB lost per VM (a leaking kernel loses ~16384 kB)
>>        RESULT: LEAK
>> 
>> With this patch:
>> 
>>        MemFree 2054080 -> 2037716 kB: 1636 kB lost per VM (a leaking kernel loses ~16384 kB)
>>        RESULT: OK
>> 
>> The exit status is 0 for OK, 1 for LEAK and 2 if the test could not
>> run. IMAGE=<path/to/Image> boots an existing kernel instead of building
>> one. That kernel needs CONFIG_KVM=y, CONFIG_BLK_DEV_INITRD=y and
>> CONFIG_DEVTMPFS=y.
>> 
>> I also tested VMs that select Sv39x4, Sv48x4 and Sv57x4 through
>> KVM_CAP_VM_GPA_BITS, and four VMs with mixed modes alive at the same
>> time, destroyed one by one while the remaining VMs kept running and
>> were checked for correct guest memory contents. With this patch, no
>> memory was lost in any mode and no guest data was corrupted. Without
>> it, every mode leaked about 16 MiB per VM.
>> ---
>> arch/riscv/include/asm/kvm_gstage.h |  2 ++
>> arch/riscv/kvm/gstage.c             | 22 ++++++++++++++++++++++
>> arch/riscv/kvm/mmu.c                |  3 +--
>> 3 files changed, 25 insertions(+), 2 deletions(-)
>> 
>> diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
>> index aaf080ba1b77..90ec9e0b34b1 100644
>> --- a/arch/riscv/include/asm/kvm_gstage.h
>> +++ b/arch/riscv/include/asm/kvm_gstage.h
>> @@ -80,6 +80,8 @@ bool kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
>> bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
>>                                  gpa_t start, gpa_t size, bool may_block);
>> 
>> +bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage);
>> +
>> bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end);
>> 
>> bool kvm_riscv_gstage_wp_pt_masked(struct kvm_gstage *gstage, gfn_t base_gfn,
>> diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
>> index e5002cb9cbef..c912cdc2021f 100644
>> --- a/arch/riscv/kvm/gstage.c
>> +++ b/arch/riscv/kvm/gstage.c
>> @@ -456,6 +456,28 @@ bool kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
>>        return flush;
>> }
>> 
>> +/*
>> + * Clear every G-stage mapping and free all page tables below the PGD.
>> + * kvm_riscv_gstage_unmap_range() only clears leaf PTEs and leaves the
>> + * page tables in place, so it cannot be used to tear down a G-stage.
>> + */
>> +bool kvm_riscv_gstage_unmap_all(struct kvm_gstage *gstage)
>> +{
>> +       u32 level = gstage->pgd_levels - 1;
>> +       pte_t *pgd = (pte_t *)gstage->pgd;
>> +       unsigned long i, page_size;
>> +       bool flush = false;
>> +
>> +       if (gstage_level_to_page_size(gstage, level, &page_size))
>> +               return false;
>> +
>> +       for (i = 0; i < kvm_riscv_gstage_pgd_size / sizeof(pte_t); i++)
>> +               flush |= kvm_riscv_gstage_op_pte(gstage, i * page_size, &pgd[i],
>> +                                                level, GSTAGE_OP_CLEAR);
>> +
>> +       return flush;
>> +}
>> +
>> bool kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end)
>> {
>>        int ret;
>> diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
>> index 371fccaf9df0..87838a0f6bb9 100644
>> --- a/arch/riscv/kvm/mmu.c
>> +++ b/arch/riscv/kvm/mmu.c
>> @@ -784,8 +784,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
>>        write_lock(&kvm->mmu_lock);
>>        if (kvm->arch.pgd) {
>>                kvm_riscv_gstage_init(&gstage, kvm);
>> -               flush = kvm_riscv_gstage_unmap_range(&gstage, 0UL,
>> -                       kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels), false);
>> +               flush = kvm_riscv_gstage_unmap_all(&gstage);
>>                pgd = READ_ONCE(kvm->arch.pgd);
>>                kvm->arch.pgd = NULL;
>>                kvm->arch.pgd_phys = 0;
>> --
>> 2.53.0