[PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[Fuad Tabba]

Hi folks,

Two fixes for stage-2 teardown, which since 4ddfab5436b6 takes time
proportional to the size of the IPA space rather than to what is mapped.

Under QEMU TCG ('-cpu max', 16K pages with LPA2), destroying even an
empty VM takes about 7 seconds in pKVM. Without pKVM it takes about 19
seconds if the VM requests a 52-bit IPA (crosvm requests the host's
limit). I would expect it to scale the same way in hardware. I tested
only under QEMU, since I don't have hardware with LPA2.

pKVM gives every VM the host's IPA limit. The first makes it release the
VM's mappings from its tree instead of walking that space. My series
honoring KVM_VM_TYPE_ARM_IPA_SIZE in pKVM [1] (respinning it soon)
doesn't replace the fix, as crosvm requests the limit there too.

The second makes the non-pKVM teardown step over an empty entry larger
than the largest block in one go, rather than start a new walk for each
block-sized chunk of it. It depends on the first, as the lookup it adds
reads fields that hold the mapping tree on a pKVM stage-2.

The third moves the pKVM/non-pKVM split in the teardown into
kvm_vm_s2_ops, with no functional change. The first two apply back to
7.1, except that a backport of the second drops the flags argument 7.4
added to kvm_pgtable_get_leaf().

Based on Linux 7.4-rc1 (6628e7d4c6b25).

Cheers,
/fuad

[1] https://lore.kernel.org/all/20260917091826.2639205-1-fuad.tabba@linux.dev/

Fuad Tabba (3):
  KVM: arm64: Don't walk the whole IPA space when destroying a pKVM
    stage-2
  KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables
  KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/mmu.c              | 34 +++++++++++++++++++++++--------
 arch/arm64/kvm/pkvm.c             | 17 ++++++++++++----
 3 files changed, 41 insertions(+), 12 deletions(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.47.3

[PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Fuad Tabba]

Since commit 4ddfab5436b6 ("KVM: arm64: Reschedule as needed when
destroying the stage-2 page-tables"), kvm_stage2_destroy() steps through
the IPA space in block-sized chunks, and with pKVM each chunk is a
lookup in the VM's mapping tree. However, pKVM gives every VM the host's
IPA limit, so with LPA2 a teardown makes 2^27 lookups with 16K pages
(2^22 with 4K), nearly all of them for empty space.

Walk the mapping tree once instead, and reschedule after passing each
mapping to the reclaim or unshare helper. The unshare helper can't
reschedule itself, as the map and unmap paths call it under the
mmu_lock. A failed unshare now leaves only its own mapping behind, not
the rest of its chunk.

Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c  |  7 ++++++-
 arch/arm64/kvm/pkvm.c | 17 +++++++++++++----
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1db10d8d6db73..42977cf8a7f5d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1022,7 +1022,12 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
 {
 	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
 
-	stage2_destroy_range(pgt, 0, BIT(ia_bits));
+	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
+	if (is_protected_kvm_enabled())
+		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
+	else
+		stage2_destroy_range(pgt, 0, BIT(ia_bits));
+
 	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
 }
 
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 016997288fd23..992ed3ca6bdd9 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -504,6 +504,7 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
 {
 	struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
 	pkvm_handle_t handle = kvm->arch.pkvm.handle;
+	struct pkvm_mapping *mapping;
 
 	if (!handle)
 		return;
@@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
 		kvm->arch.pkvm.is_dying = true;
 	}
 
-	if (kvm_vm_is_protected(kvm))
-		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
-	else
-		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
+	/* Called once for the whole IPA space, so reschedule between mappings */
+	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
+		u64 start = __pkvm_mapping_start(mapping);
+		u64 end = __pkvm_mapping_end(mapping) + 1;
+
+		if (kvm_vm_is_protected(kvm))
+			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
+		else
+			__pkvm_pgtable_stage2_unshare(pgt, start, end);
+
+		cond_resched();
+	}
 }
 
 void pkvm_pgtable_stage2_destroy_pgd(struct kvm_pgtable *pgt)
-- 
2.47.3

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Vincent Donnefort]

On Tue, Nov 03, 2026 at 08:47:40AM +0000, Fuad Tabba wrote:
> Since commit 4ddfab5436b6 ("KVM: arm64: Reschedule as needed when
> destroying the stage-2 page-tables"), kvm_stage2_destroy() steps through
> the IPA space in block-sized chunks, and with pKVM each chunk is a
> lookup in the VM's mapping tree. However, pKVM gives every VM the host's
> IPA limit, so with LPA2 a teardown makes 2^27 lookups with 16K pages
> (2^22 with 4K), nearly all of them for empty space.
> 
> Walk the mapping tree once instead, and reschedule after passing each
> mapping to the reclaim or unshare helper. The unshare helper can't
> reschedule itself, as the map and unmap paths call it under the
> mmu_lock. A failed unshare now leaves only its own mapping behind, not
> the rest of its chunk.

Not a problem now, but don't we make the device assignment story harder?

> 
> Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")

Isn't it an optimisation?

> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>  arch/arm64/kvm/mmu.c  |  7 ++++++-
>  arch/arm64/kvm/pkvm.c | 17 +++++++++++++----
>  2 files changed, 19 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1db10d8d6db73..42977cf8a7f5d 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1022,7 +1022,12 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>  {
>  	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
>  
> -	stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
> +	if (is_protected_kvm_enabled())
> +		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	else
> +		stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +
>  	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>  }
>  
> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
> index 016997288fd23..992ed3ca6bdd9 100644
> --- a/arch/arm64/kvm/pkvm.c
> +++ b/arch/arm64/kvm/pkvm.c
> @@ -504,6 +504,7 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  {
>  	struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
>  	pkvm_handle_t handle = kvm->arch.pkvm.handle;
> +	struct pkvm_mapping *mapping;
>  
>  	if (!handle)
>  		return;
> @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  		kvm->arch.pkvm.is_dying = true;
>  	}
>  
> -	if (kvm_vm_is_protected(kvm))
> -		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
> -	else
> -		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
> +	/* Called once for the whole IPA space, so reschedule between mappings */
> +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> +		u64 start = __pkvm_mapping_start(mapping);
> +		u64 end = __pkvm_mapping_end(mapping) + 1;

__pkvm_mapping_* are rb-tree internals. I think it is better to use explicitly:

  start = gfn_to_gpa(mapping->gfn);
  end = start + (mapping->nr_pages * PAGE_SIZE)

Also, I wonder if we'd better WARN if the mapping isn't aligned with the
destroyed range?

-- 
Vincent

> +
> +		if (kvm_vm_is_protected(kvm))
> +			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
> +		else
> +			__pkvm_pgtable_stage2_unshare(pgt, start, end);
> +
> +		cond_resched();
> +	}
>  }
>  
>  void pkvm_pgtable_stage2_destroy_pgd(struct kvm_pgtable *pgt)
> -- 
> 2.47.3
> 
> 

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Fuad Tabba]

Hi Vincent,

On Tue, 03 Nov 2026 14:52:36 +0000, Vincent Donnefort
<vdonnefort@google.com> wrote:
[...]
> > Walk the mapping tree once instead, and reschedule after passing each
> > mapping to the reclaim or unshare helper. The unshare helper can't
> > reschedule itself, as the map and unmap paths call it under the
> > mmu_lock. A failed unshare now leaves only its own mapping behind, not
> > the rest of its chunk.
>
> Not a problem now, but don't we make the device assignment story harder?

I don't think so. In Android, the host doesn't track the MMIO of a
device assigned to a protected guest. The hypervisor maps it into the
guest, and gives it back to the host in __pkvm_start_teardown_vm(),
before the host reclaims anything. So the walk this patch changes
shouldn't reach it.

>
> >
> > Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
>
> Isn't it an optimisation?

No, it fixes a regression: 4ddfab5436b6 replaced a single walk of the
mapping tree with the 2^27 lookups, as described in the commit
message.

[...]
> > diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
[...]
> > @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
[...]
> > +        for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> > +                u64 start = __pkvm_mapping_start(mapping);
> > +                u64 end = __pkvm_mapping_end(mapping) + 1;
>
> __pkvm_mapping_* are rb-tree internals. I think it is better to use explicitly:
>
>   start = gfn_to_gpa(mapping->gfn);
>   end = start + (mapping->nr_pages * PAGE_SIZE)

Ack.

>
> Also, I wonder if we'd better WARN if the mapping isn't aligned with the
> destroyed range?

No need. The function's only caller passes the whole IPA space, so
every mapping is inside the range.

Cheers,
/fuad

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Marc Zyngier]

On Tue, 03 Nov 2026 16:15:03 +0000,
Fuad Tabba <fuad.tabba@linux.dev> wrote:
> 
> Hi Vincent,
> 
> On Tue, 03 Nov 2026 14:52:36 +0000, Vincent Donnefort
> <vdonnefort@google.com> wrote:
> [...]
> > > Walk the mapping tree once instead, and reschedule after passing each
> > > mapping to the reclaim or unshare helper. The unshare helper can't
> > > reschedule itself, as the map and unmap paths call it under the
> > > mmu_lock. A failed unshare now leaves only its own mapping behind, not
> > > the rest of its chunk.
> >
> > Not a problem now, but don't we make the device assignment story harder?
> 
> I don't think so. In Android, the host doesn't track the MMIO of a
> device assigned to a protected guest. The hypervisor maps it into the
> guest, and gives it back to the host in __pkvm_start_teardown_vm(),
> before the host reclaims anything. So the walk this patch changes
> shouldn't reach it.
> 
> >
> > >
> > > Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
> >
> > Isn't it an optimisation?
> 
> No, it fixes a regression: 4ddfab5436b6 replaced a single walk of the
> mapping tree with the 2^27 lookups, as described in the commit
> message.

But that's a configuration that simply does not exist in HW.

Therefore this isn't a fix, this is an opportunistic optimisation that
may or may not become useful. So not a fix, no backporting.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Fuad Tabba]

On Tue, 3 Nov 2026 at 16:26, Marc Zyngier <maz@kernel.org> wrote:
[...]
> > > > Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
> > >
> > > Isn't it an optimisation?
> >
> > No, it fixes a regression: 4ddfab5436b6 replaced a single walk of the
> > mapping tree with the 2^27 lookups, as described in the commit
> > message.
>
> But that's a configuration that simply does not exist in HW.
>
> Therefore this isn't a fix, this is an opportunistic optimisation that
> may or may not become useful. So not a fix, no backporting.

I'll drop the tag then.

Cheers,
/fuad

>         M.
>
> --
> Without deviation from the norm, progress is not possible.

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Vincent Donnefort]

On Tue, Nov 03, 2026 at 08:47:40AM +0000, Fuad Tabba wrote:
> Since commit 4ddfab5436b6 ("KVM: arm64: Reschedule as needed when
> destroying the stage-2 page-tables"), kvm_stage2_destroy() steps through
> the IPA space in block-sized chunks, and with pKVM each chunk is a
> lookup in the VM's mapping tree. However, pKVM gives every VM the host's
> IPA limit, so with LPA2 a teardown makes 2^27 lookups with 16K pages
> (2^22 with 4K), nearly all of them for empty space.
> 
> Walk the mapping tree once instead, and reschedule after passing each
> mapping to the reclaim or unshare helper. The unshare helper can't
> reschedule itself, as the map and unmap paths call it under the
> mmu_lock. A failed unshare now leaves only its own mapping behind, not
> the rest of its chunk.
> 
> Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>  arch/arm64/kvm/mmu.c  |  7 ++++++-
>  arch/arm64/kvm/pkvm.c | 17 +++++++++++++----
>  2 files changed, 19 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1db10d8d6db73..42977cf8a7f5d 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1022,7 +1022,12 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>  {
>  	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
>  
> -	stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
> +	if (is_protected_kvm_enabled())
> +		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	else
> +		stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +
>  	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>  }
>  
> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
> index 016997288fd23..992ed3ca6bdd9 100644
> --- a/arch/arm64/kvm/pkvm.c
> +++ b/arch/arm64/kvm/pkvm.c
> @@ -504,6 +504,7 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  {
>  	struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
>  	pkvm_handle_t handle = kvm->arch.pkvm.handle;
> +	struct pkvm_mapping *mapping;
>  
>  	if (!handle)
>  		return;
> @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  		kvm->arch.pkvm.is_dying = true;
>  	}
>  
> -	if (kvm_vm_is_protected(kvm))
> -		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
> -	else
> -		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
> +	/* Called once for the whole IPA space, so reschedule between mappings */
> +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> +		u64 start = __pkvm_mapping_start(mapping);
> +		u64 end = __pkvm_mapping_end(mapping) + 1;
> +
> +		if (kvm_vm_is_protected(kvm))
> +			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
> +		else
> +			__pkvm_pgtable_stage2_unshare(pgt, start, end);
> +
> +		cond_resched();
> +	}
>  }

Just thought that it is a bit weird that pkvm_pgtable_stage2_destroy_range()
does cond_resched() while kvm_pgtable_stage2_destroy_range() does not.

How about instead kvm_stage2_destroy() refines start/end with some accessors
calling in pkvm.c pkvm_mapping_iter_first() / pkvm_mapping_iter_last()?

>  
>  void pkvm_pgtable_stage2_destroy_pgd(struct kvm_pgtable *pgt)
> -- 
> 2.47.3
> 
> 

-- 
Vincent

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Oliver Upton]

On Tue, Nov 03, 2026 at 04:37:54PM +0000, Vincent Donnefort wrote:
> > @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
> >  		kvm->arch.pkvm.is_dying = true;
> >  	}
> >  
> > -	if (kvm_vm_is_protected(kvm))
> > -		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
> > -	else
> > -		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
> > +	/* Called once for the whole IPA space, so reschedule between mappings */
> > +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> > +		u64 start = __pkvm_mapping_start(mapping);
> > +		u64 end = __pkvm_mapping_end(mapping) + 1;
> > +
> > +		if (kvm_vm_is_protected(kvm))
> > +			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
> > +		else
> > +			__pkvm_pgtable_stage2_unshare(pgt, start, end);
> > +
> > +		cond_resched();
> > +	}
> >  }
> 
> Just thought that it is a bit weird that pkvm_pgtable_stage2_destroy_range()
> does cond_resched() while kvm_pgtable_stage2_destroy_range() does not.
> 
> How about instead kvm_stage2_destroy() refines start/end with some accessors
> calling in pkvm.c pkvm_mapping_iter_first() / pkvm_mapping_iter_last()?

Yeah, I'd rather we tackle this as a general problem than doing some
pKVM specific hacks.

Seems to me that we do a lot of wasted work even in the non-pKVM case
where the outer loop will continue to iterate KVM_PGTABLE_MIN_BLOCK_LEVEL
worth of IPA even if the underlying table is empty.

What we'd want is for the kvm_pgtable_stage2_destroy_range()
implementation to report back how much of the IPA space it actually
iterated over. We wouldn't want to keep re-visiting the same invalid PGD
entry only because it's larger than our max block size.

Thanks,
Oliver

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Oliver Upton]

On Tue, Nov 03, 2026 at 08:47:31AM -0800, Oliver Upton wrote:
> On Tue, Nov 03, 2026 at 04:37:54PM +0000, Vincent Donnefort wrote:
> > > @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
> > >  		kvm->arch.pkvm.is_dying = true;
> > >  	}
> > >  
> > > -	if (kvm_vm_is_protected(kvm))
> > > -		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
> > > -	else
> > > -		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
> > > +	/* Called once for the whole IPA space, so reschedule between mappings */
> > > +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> > > +		u64 start = __pkvm_mapping_start(mapping);
> > > +		u64 end = __pkvm_mapping_end(mapping) + 1;
> > > +
> > > +		if (kvm_vm_is_protected(kvm))
> > > +			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
> > > +		else
> > > +			__pkvm_pgtable_stage2_unshare(pgt, start, end);
> > > +
> > > +		cond_resched();
> > > +	}
> > >  }
> > 
> > Just thought that it is a bit weird that pkvm_pgtable_stage2_destroy_range()
> > does cond_resched() while kvm_pgtable_stage2_destroy_range() does not.
> > 
> > How about instead kvm_stage2_destroy() refines start/end with some accessors
> > calling in pkvm.c pkvm_mapping_iter_first() / pkvm_mapping_iter_last()?
> 
> Yeah, I'd rather we tackle this as a general problem than doing some
> pKVM specific hacks.
> 
> Seems to me that we do a lot of wasted work even in the non-pKVM case
> where the outer loop will continue to iterate KVM_PGTABLE_MIN_BLOCK_LEVEL
> worth of IPA even if the underlying table is empty.
> 
> What we'd want is for the kvm_pgtable_stage2_destroy_range()
> implementation to report back how much of the IPA space it actually
> iterated over. We wouldn't want to keep re-visiting the same invalid PGD
> entry only because it's larger than our max block size.

Next patch heh :)

[PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Fuad Tabba]

The page-table walker steps over an empty entry in one go, but
kvm_stage2_destroy() starts a new walk for every block-sized chunk of
the IPA space. Tearing down a VM that requests a 52-bit IPA then takes
2^27 walks with 16K pages (2^22 with 4K), however little it maps.
crosvm requests the host's limit.

After destroying each chunk, look up the entry at its start and, if it
is empty and larger than a chunk, skip its whole range. The lookup reads
fields that hold the mapping tree on a pKVM stage-2, so this depends on
the previous patch keeping pKVM out of stage2_destroy_range(). For the
same reason, call kvm_pgtable_stage2_destroy_range() directly rather
than through KVM_PGT_FN().

Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 42977cf8a7f5d..10b425a0141be 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
 	u64 next;
 
 	do {
+		kvm_pte_t pte;
+		s8 level;
+
 		next = stage2_range_addr_end(addr, end);
-		KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
-							     next - addr);
+		kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);
+
+		/*
+		 * The walk freed every table it emptied, so skipping an empty
+		 * entry here doesn't leave anything behind.
+		 */
+		if (!kvm_pgtable_get_leaf(pgt, addr, &pte, &level, 0) && !pte &&
+		    level < KVM_PGTABLE_MIN_BLOCK_LEVEL)
+			next = __stage2_range_addr_end(addr, end, kvm_granule_size(level));
+
 		if (next != end)
 			cond_resched();
 	} while (addr = next, addr != end);
-- 
2.47.3

Re: [PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Suzuki K Poulose]

Hi Fuad

On 03/11/2026 08:47, Fuad Tabba wrote:
> The page-table walker steps over an empty entry in one go, but
> kvm_stage2_destroy() starts a new walk for every block-sized chunk of
> the IPA space. Tearing down a VM that requests a 52-bit IPA then takes
> 2^27 walks with 16K pages (2^22 with 4K), however little it maps.
> crosvm requests the host's limit.
> 

The patch looks good to me, in fact I was thinking about the same thing
yesterday while looking at the same code from a different angle, to move
to the vm_s2_ops abstraction.

> After destroying each chunk, look up the entry at its start and, if it
> is empty and larger than a chunk, skip its whole range. The lookup reads

Could we not lookup first before we destroy, that way we can avoid 
destroy walks ?

> fields that hold the mapping tree on a pKVM stage-2, so this depends on
> the previous patch keeping pKVM out of stage2_destroy_range(). For the
> same reason, call kvm_pgtable_stage2_destroy_range() directly rather
> than through KVM_PGT_FN().



> 
> Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>   arch/arm64/kvm/mmu.c | 15 +++++++++++++--
>   1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 42977cf8a7f5d..10b425a0141be 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
>   	u64 next;
>   
>   	do {
> +		kvm_pte_t pte;
> +		s8 level;
> +
>   		next = stage2_range_addr_end(addr, end);
> -		KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
> -							     next - addr);
> +		kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);

minor nit: Given the kvm_pgtable_stage2_destroy_range() is the only 
called from here, and this caller would benefit from the walk
information, would it make sense to return a hint back ? Not sure how
much of that is possible. We do this in the RMM, always return a hint
of the next possible address. Of course this is not that expensive
as the CCA case and could be dealt in a separate series.

> +
> +		/*
> +		 * The walk freed every table it emptied, so skipping an empty
> +		 * entry here doesn't leave anything behind.
> +		 */
> +		if (!kvm_pgtable_get_leaf(pgt, addr, &pte, &level, 0) && !pte &&
> +		    level < KVM_PGTABLE_MIN_BLOCK_LEVEL)
> +			next = __stage2_range_addr_end(addr, end, kvm_granule_size(level));

Suzuki


> +
>   		if (next != end)
>   			cond_resched();
>   	} while (addr = next, addr != end);

Re: [PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Fuad Tabba]

Hi Suzuki,

On Tue, 03 Nov 2026 10:27:54 +0000, Suzuki K Poulose
<suzuki.poulose@arm.com> wrote:
[...]
> > After destroying each chunk, look up the entry at its start and, if it
> > is empty and larger than a chunk, skip its whole range. The lookup reads
>
> Could we not lookup first before we destroy, that way we can avoid
> destroy walks ?

I'd rather keep it after the walk. Looking first relies on no empty
table being left linked above KVM_PGTABLE_MIN_BLOCK_LEVEL: the lookup
would return one of its empty entries, and the loop would skip past
the table without freeing it.

I don't think anything leaves such a table behind today, but looking
after the walk doesn't depend on that, since the walk frees every
empty table it passes through. That's one extra walk per empty region.
I'll add to the comment why the lookup comes after the walk.

[...]
> > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
[...]
> > @@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
[...]
> >                   next = stage2_range_addr_end(addr, end);
> > -                KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
> > -                                                             next - addr);
> > +                kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);
>
> minor nit: Given the kvm_pgtable_stage2_destroy_range() is the only
> called from here, and this caller would benefit from the walk
> information, would it make sense to return a hint back ? Not sure how
> much of that is possible. We do this in the RMM, always return a hint
> of the next possible address. Of course this is not that expensive
> as the CCA case and could be dealt in a separate series.

kvm_pgtable_stage2_destroy_range() is also called by
kvm_pgtable_stage2_destroy(), which EL2 uses to free a pKVM guest's
stage-2, so I'd rather not change it in a fix.

Cheers,
/fuad

Re: [PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Oliver Upton]

On Tue, Nov 03, 2026 at 04:19:38PM +0000, Fuad Tabba wrote:
> > > @@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
> [...]
> > >                   next = stage2_range_addr_end(addr, end);
> > > -                KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
> > > -                                                             next - addr);
> > > +                kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);
> >
> > minor nit: Given the kvm_pgtable_stage2_destroy_range() is the only
> > called from here, and this caller would benefit from the walk
> > information, would it make sense to return a hint back ? Not sure how
> > much of that is possible. We do this in the RMM, always return a hint
> > of the next possible address. Of course this is not that expensive
> > as the CCA case and could be dealt in a separate series.
> 
> kvm_pgtable_stage2_destroy_range() is also called by
> kvm_pgtable_stage2_destroy(), which EL2 uses to free a pKVM guest's
> stage-2, so I'd rather not change it in a fix.

This one is still an optimization, not a fix. The re-walk after destroy
is a bit fiddly, especially when the inner walk knows exactly what IPA
it stopped at.

Thanks,
Oliver

[PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Fuad Tabba]

Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
like the other stage-2 operations. Both pKVM flavors share
pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
from kvm_phys_size().

No functional change intended.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 97332ffcb1300..b36779bafae9b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
 
 struct kvm_gfn_range;
 struct kvm_s2_fault_desc;
+struct kvm_pgtable;
 
 struct kvm_vm_s2_ops {
 	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
@@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
 	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
 				      phys_addr_t start, u64 size,
 				      bool may_block);
+	void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
 	int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
 };
 
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 10b425a0141be..9df3c80a5acd1 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
 	} while (addr = next, addr != end);
 }
 
-static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
+static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
 {
-	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
+	stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
+	kvm_pgtable_stage2_destroy_pgd(pgt);
+}
 
-	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
-	if (is_protected_kvm_enabled())
-		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
-	else
-		stage2_destroy_range(pgt, 0, BIT(ia_bits));
-
-	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
+static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
+{
+	pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
+	pkvm_pgtable_stage2_destroy_pgd(pgt);
 }
 
 /**
@@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
 	return 0;
 
 out_destroy_pgtable:
-	kvm_stage2_destroy(pgt);
+	kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
 	mmu->pgt = NULL;
 out_free_pgtable:
 	kfree(pgt);
@@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
 	write_unlock(&kvm->mmu_lock);
 
 	if (pgt) {
-		kvm_stage2_destroy(pgt);
+		kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
 		kfree(pgt);
 	}
 
@@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops protected_pkvm_vm_s2_ops = {
 	.vm_age_gfn			= no_age_gfn,
 	.vm_test_age_gfn		= no_age_gfn,
 	.vm_stage2_unmap_range		= no_stage2_unmap_range,
+	.vm_stage2_destroy		= pkvm_stage2_destroy,
 	.vm_mem_abort			= protected_pkvm_mem_abort,
 };
 
@@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
 	.vm_age_gfn			= pkvm_age_gfn,
 	.vm_test_age_gfn		= pkvm_test_age_gfn,
 	.vm_stage2_unmap_range		= pkvm_stage2_unmap_range,
+	.vm_stage2_destroy		= pkvm_stage2_destroy,
 	.vm_mem_abort			= kvm_vm_mem_abort,
 };
 
@@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
 	.vm_age_gfn			= kvm_vm_age_gfn,
 	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
 	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+	.vm_stage2_destroy		= kvm_vm_stage2_destroy,
 	.vm_mem_abort			= kvm_vm_mem_abort,
 };
 
-- 
2.47.3

Re: [PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Suzuki K Poulose]

Hi Fuad

On 03/11/2026 08:47, Fuad Tabba wrote:
> Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
> KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
> like the other stage-2 operations. Both pKVM flavors share
> pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
> from kvm_phys_size().
> 
> No functional change intended.

Someone beat you on this (well almost) ;-)

https://lore.kernel.org/all/4304d0c7-2220-4add-9c71-e6dc6b4fbc11@arm.com


> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>   arch/arm64/include/asm/kvm_host.h |  2 ++
>   arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
>   2 files changed, 15 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 97332ffcb1300..b36779bafae9b 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
>   
>   struct kvm_gfn_range;
>   struct kvm_s2_fault_desc;
> +struct kvm_pgtable;
>   
>   struct kvm_vm_s2_ops {
>   	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
> @@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
>   	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
>   				      phys_addr_t start, u64 size,
>   				      bool may_block);
> +	void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
>   	int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
>   };
>   
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 10b425a0141be..9df3c80a5acd1 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
>   	} while (addr = next, addr != end);
>   }
>   
> -static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
> +static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
>   {
> -	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
> +	stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
> +	kvm_pgtable_stage2_destroy_pgd(pgt);
> +}
>   
> -	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
> -	if (is_protected_kvm_enabled())
> -		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
> -	else
> -		stage2_destroy_range(pgt, 0, BIT(ia_bits));
> -
> -	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
> +static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
> +{
> +	pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
> +	pkvm_pgtable_stage2_destroy_pgd(pgt);
>   }
>   
>   /**
> @@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
>   	return 0;
>   
>   out_destroy_pgtable:
> -	kvm_stage2_destroy(pgt);
> +	kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);

minor nit: should we retain the kvm_stage2_destroy() to dispatch the 
callback ?

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Cheers
Suzuki


>   	mmu->pgt = NULL;
>   out_free_pgtable:
>   	kfree(pgt);
> @@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>   	write_unlock(&kvm->mmu_lock);
>   
>   	if (pgt) {
> -		kvm_stage2_destroy(pgt);
> +		kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
>   		kfree(pgt);
>   	}
>   
> @@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops protected_pkvm_vm_s2_ops = {
>   	.vm_age_gfn			= no_age_gfn,
>   	.vm_test_age_gfn		= no_age_gfn,
>   	.vm_stage2_unmap_range		= no_stage2_unmap_range,
> +	.vm_stage2_destroy		= pkvm_stage2_destroy,
>   	.vm_mem_abort			= protected_pkvm_mem_abort,
>   };
>   
> @@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
>   	.vm_age_gfn			= pkvm_age_gfn,
>   	.vm_test_age_gfn		= pkvm_test_age_gfn,
>   	.vm_stage2_unmap_range		= pkvm_stage2_unmap_range,
> +	.vm_stage2_destroy		= pkvm_stage2_destroy,
>   	.vm_mem_abort			= kvm_vm_mem_abort,
>   };
>   
> @@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
>   	.vm_age_gfn			= kvm_vm_age_gfn,
>   	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
>   	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
> +	.vm_stage2_destroy		= kvm_vm_stage2_destroy,
>   	.vm_mem_abort			= kvm_vm_mem_abort,
>   };
>   

Re: [PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Suzuki K Poulose]

On 03/11/2026 09:37, Suzuki K Poulose wrote:
> Hi Fuad
> 
> On 03/11/2026 08:47, Fuad Tabba wrote:
>> Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
>> KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
>> like the other stage-2 operations. Both pKVM flavors share
>> pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
>> from kvm_phys_size().
>>
>> No functional change intended.
> 
> Someone beat you on this (well almost) ;-)
> 
> https://lore.kernel.org/all/4304d0c7-2220-4add-9c71-e6dc6b4fbc11@arm.com
> 
> 
>>
>> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
>> ---
>>   arch/arm64/include/asm/kvm_host.h |  2 ++
>>   arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
>>   2 files changed, 15 insertions(+), 11 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/ 
>> asm/kvm_host.h
>> index 97332ffcb1300..b36779bafae9b 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
>>   struct kvm_gfn_range;
>>   struct kvm_s2_fault_desc;
>> +struct kvm_pgtable;
>>   struct kvm_vm_s2_ops {
>>       bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
>> @@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
>>       void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
>>                         phys_addr_t start, u64 size,
>>                         bool may_block);
>> +    void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
>>       int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
>>   };
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 10b425a0141be..9df3c80a5acd1 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct 
>> kvm_pgtable *pgt, phys_addr_t addr,
>>       } while (addr = next, addr != end);
>>   }
>> -static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>> +static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
>>   {
>> -    unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
>> +    stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));

Also, now that stage2_destroy_range is only called for normal KVM,
could we fold that in here ?

Cheers
Suzuki


>> +    kvm_pgtable_stage2_destroy_pgd(pgt);
>> +}
>> -    /* pKVM walks its mapping tree, not the IPA space, and 
>> reschedules per mapping */
>> -    if (is_protected_kvm_enabled())
>> -        pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
>> -    else
>> -        stage2_destroy_range(pgt, 0, BIT(ia_bits));
>> -
>> -    KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>> +static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
>> +{
>> +    pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
>> +    pkvm_pgtable_stage2_destroy_pgd(pgt);
>>   }
>>   /**
>> @@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct 
>> kvm_s2_mmu *mmu, unsigned long t
>>       return 0;
>>   out_destroy_pgtable:
>> -    kvm_stage2_destroy(pgt);
>> +    kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
> 
> minor nit: should we retain the kvm_stage2_destroy() to dispatch the 
> callback ?
> 
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> Cheers
> Suzuki
> 
> 
>>       mmu->pgt = NULL;
>>   out_free_pgtable:
>>       kfree(pgt);
>> @@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>>       write_unlock(&kvm->mmu_lock);
>>       if (pgt) {
>> -        kvm_stage2_destroy(pgt);
>> +        kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
>>           kfree(pgt);
>>       }
>> @@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops 
>> protected_pkvm_vm_s2_ops = {
>>       .vm_age_gfn            = no_age_gfn,
>>       .vm_test_age_gfn        = no_age_gfn,
>>       .vm_stage2_unmap_range        = no_stage2_unmap_range,
>> +    .vm_stage2_destroy        = pkvm_stage2_destroy,
>>       .vm_mem_abort            = protected_pkvm_mem_abort,
>>   };
>> @@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops 
>> = {
>>       .vm_age_gfn            = pkvm_age_gfn,
>>       .vm_test_age_gfn        = pkvm_test_age_gfn,
>>       .vm_stage2_unmap_range        = pkvm_stage2_unmap_range,
>> +    .vm_stage2_destroy        = pkvm_stage2_destroy,
>>       .vm_mem_abort            = kvm_vm_mem_abort,
>>   };
>> @@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops 
>> kvm_default_vm_s2_ops = {
>>       .vm_age_gfn            = kvm_vm_age_gfn,
>>       .vm_test_age_gfn        = kvm_vm_test_age_gfn,
>>       .vm_stage2_unmap_range        = kvm_vm_stage2_unmap_range,
>> +    .vm_stage2_destroy        = kvm_vm_stage2_destroy,
>>       .vm_mem_abort            = kvm_vm_mem_abort,
>>   };
> 

Re: [PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Fuad Tabba]

Hi Suzuki,

On Tue, 03 Nov 2026 10:33:15 +0000, Suzuki K Poulose
<suzuki.poulose@arm.com> wrote:
[...]
> > Someone beat you on this (well almost) ;-)

:D

> > https://lore.kernel.org/all/4304d0c7-2220-4add-9c71-e6dc6b4fbc11@arm.com

realm_stage2_destroy() in your sketch doesn't use the range it's
given, so it should fit this hook, which only takes the pgt.

[...]
> >> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> >> index 10b425a0141be..9df3c80a5acd1 100644
> >> --- a/arch/arm64/kvm/mmu.c
> >> +++ b/arch/arm64/kvm/mmu.c
> >> @@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct
> >> kvm_pgtable *pgt, phys_addr_t addr,
> >>       } while (addr = next, addr != end);
> >>   }
> >> -static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
> >> +static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
> >>   {
> >> -    unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
> >> +    stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
>
> Also, now that stage2_destroy_range is only called for normal KVM,
> could we fold that in here ?

Ack.

[...]
> >> @@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct
> >> kvm_s2_mmu *mmu, unsigned long t
> >>       return 0;
> >>   out_destroy_pgtable:
> >> -    kvm_stage2_destroy(pgt);
> >> +    kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
> >
> > minor nit: should we retain the kvm_stage2_destroy() to dispatch the
> > callback ?

I'll keep it in v2.

> >
> > Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Thanks!

Cheers,
/fuad

Re: [PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[Oliver Upton]

Hi Fuad,

On Tue, Nov 03, 2026 at 08:47:39AM +0000, Fuad Tabba wrote:
> Hi folks,
> 
> Two fixes for stage-2 teardown, which since 4ddfab5436b6 takes time
> proportional to the size of the IPA space rather than to what is mapped.
> 
> Under QEMU TCG ('-cpu max', 16K pages with LPA2), destroying even an
> empty VM takes about 7 seconds in pKVM. Without pKVM it takes about 19
> seconds if the VM requests a 52-bit IPA (crosvm requests the host's
> limit). I would expect it to scale the same way in hardware. I tested
> only under QEMU, since I don't have hardware with LPA2.
> 
> pKVM gives every VM the host's IPA limit. The first makes it release the
> VM's mappings from its tree instead of walking that space. My series
> honoring KVM_VM_TYPE_ARM_IPA_SIZE in pKVM [1] (respinning it soon)
> doesn't replace the fix, as crosvm requests the limit there too.
> 
> The second makes the non-pKVM teardown step over an empty entry larger
> than the largest block in one go, rather than start a new walk for each
> block-sized chunk of it. It depends on the first, as the lookup it adds
> reads fields that hold the mapping tree on a pKVM stage-2.
> 
> The third moves the pKVM/non-pKVM split in the teardown into
> kvm_vm_s2_ops, with no functional change. The first two apply back to
> 7.1, except that a backport of the second drops the flags argument 7.4
> added to kvm_pgtable_get_leaf().

I don't see why optimized destroy walks for CCA and pKVM is
fundamentally incompatible with driving them from a loop that
reschedules. Non-preemptible kernels are still very much a thing, I'd
rather not undo the fix we took for that exact use case.

So unless I'm missing something completely obvious, I'd rather that CCA
and pKVM do teardowns from that loop.

Thanks,
Oliver

Re: [PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[Fuad Tabba]

Hi Oliver,

On Tue, 03 Nov 2026 17:03:13 +0000, Oliver Upton <oupton@kernel.org> wrote:
[...]
> I don't see why optimized destroy walks for CCA and pKVM is
> fundamentally incompatible with driving them from a loop that
> reschedules. Non-preemptible kernels are still very much a thing, I'd
> rather not undo the fix we took for that exact use case.
>
> So unless I'm missing something completely obvious, I'd rather that CCA
> and pKVM do teardowns from that loop.

In v2, pKVM's teardown will run from that loop.
kvm_pgtable_stage2_destroy_range() will return where its walk stopped
(Suzuki's suggestion) and pkvm_pgtable_stage2_destroy_range() where
the next mapping starts, so stage2_destroy_range() resumes from there.
That replaces the pKVM path in patch 1 and the lookup in patch 2, and
I'll drop patch 3 and the Fixes: tags.

Thanks,
/fuad