[PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[PATCH 0/3] KVM: arm64: Stop stage-2 teardown scaling with the IPA size

[Fuad Tabba]

Hi folks,

Two fixes for stage-2 teardown, which since 4ddfab5436b6 takes time
proportional to the size of the IPA space rather than to what is mapped.

Under QEMU TCG ('-cpu max', 16K pages with LPA2), destroying even an
empty VM takes about 7 seconds in pKVM. Without pKVM it takes about 19
seconds if the VM requests a 52-bit IPA (crosvm requests the host's
limit). I would expect it to scale the same way in hardware. I tested
only under QEMU, since I don't have hardware with LPA2.

pKVM gives every VM the host's IPA limit. The first makes it release the
VM's mappings from its tree instead of walking that space. My series
honoring KVM_VM_TYPE_ARM_IPA_SIZE in pKVM [1] (respinning it soon)
doesn't replace the fix, as crosvm requests the limit there too.

The second makes the non-pKVM teardown step over an empty entry larger
than the largest block in one go, rather than start a new walk for each
block-sized chunk of it. It depends on the first, as the lookup it adds
reads fields that hold the mapping tree on a pKVM stage-2.

The third moves the pKVM/non-pKVM split in the teardown into
kvm_vm_s2_ops, with no functional change. The first two apply back to
7.1, except that a backport of the second drops the flags argument 7.4
added to kvm_pgtable_get_leaf().

Based on Linux 7.4-rc1 (6628e7d4c6b25).

Cheers,
/fuad

[1] https://lore.kernel.org/all/20260917091826.2639205-1-fuad.tabba@linux.dev/

Fuad Tabba (3):
  KVM: arm64: Don't walk the whole IPA space when destroying a pKVM
    stage-2
  KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables
  KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/mmu.c              | 34 +++++++++++++++++++++++--------
 arch/arm64/kvm/pkvm.c             | 17 ++++++++++++----
 3 files changed, 41 insertions(+), 12 deletions(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.47.3

[PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Fuad Tabba]

Since commit 4ddfab5436b6 ("KVM: arm64: Reschedule as needed when
destroying the stage-2 page-tables"), kvm_stage2_destroy() steps through
the IPA space in block-sized chunks, and with pKVM each chunk is a
lookup in the VM's mapping tree. However, pKVM gives every VM the host's
IPA limit, so with LPA2 a teardown makes 2^27 lookups with 16K pages
(2^22 with 4K), nearly all of them for empty space.

Walk the mapping tree once instead, and reschedule after passing each
mapping to the reclaim or unshare helper. The unshare helper can't
reschedule itself, as the map and unmap paths call it under the
mmu_lock. A failed unshare now leaves only its own mapping behind, not
the rest of its chunk.

Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c  |  7 ++++++-
 arch/arm64/kvm/pkvm.c | 17 +++++++++++++----
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 1db10d8d6db73..42977cf8a7f5d 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1022,7 +1022,12 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
 {
 	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
 
-	stage2_destroy_range(pgt, 0, BIT(ia_bits));
+	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
+	if (is_protected_kvm_enabled())
+		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
+	else
+		stage2_destroy_range(pgt, 0, BIT(ia_bits));
+
 	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
 }
 
diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
index 016997288fd23..992ed3ca6bdd9 100644
--- a/arch/arm64/kvm/pkvm.c
+++ b/arch/arm64/kvm/pkvm.c
@@ -504,6 +504,7 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
 {
 	struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
 	pkvm_handle_t handle = kvm->arch.pkvm.handle;
+	struct pkvm_mapping *mapping;
 
 	if (!handle)
 		return;
@@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
 		kvm->arch.pkvm.is_dying = true;
 	}
 
-	if (kvm_vm_is_protected(kvm))
-		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
-	else
-		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
+	/* Called once for the whole IPA space, so reschedule between mappings */
+	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
+		u64 start = __pkvm_mapping_start(mapping);
+		u64 end = __pkvm_mapping_end(mapping) + 1;
+
+		if (kvm_vm_is_protected(kvm))
+			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
+		else
+			__pkvm_pgtable_stage2_unshare(pgt, start, end);
+
+		cond_resched();
+	}
 }
 
 void pkvm_pgtable_stage2_destroy_pgd(struct kvm_pgtable *pgt)
-- 
2.47.3

[PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Fuad Tabba]

The page-table walker steps over an empty entry in one go, but
kvm_stage2_destroy() starts a new walk for every block-sized chunk of
the IPA space. Tearing down a VM that requests a 52-bit IPA then takes
2^27 walks with 16K pages (2^22 with 4K), however little it maps.
crosvm requests the host's limit.

After destroying each chunk, look up the entry at its start and, if it
is empty and larger than a chunk, skip its whole range. The lookup reads
fields that hold the mapping tree on a pKVM stage-2, so this depends on
the previous patch keeping pKVM out of stage2_destroy_range(). For the
same reason, call kvm_pgtable_stage2_destroy_range() directly rather
than through KVM_PGT_FN().

Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/kvm/mmu.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 42977cf8a7f5d..10b425a0141be 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
 	u64 next;
 
 	do {
+		kvm_pte_t pte;
+		s8 level;
+
 		next = stage2_range_addr_end(addr, end);
-		KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
-							     next - addr);
+		kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);
+
+		/*
+		 * The walk freed every table it emptied, so skipping an empty
+		 * entry here doesn't leave anything behind.
+		 */
+		if (!kvm_pgtable_get_leaf(pgt, addr, &pte, &level, 0) && !pte &&
+		    level < KVM_PGTABLE_MIN_BLOCK_LEVEL)
+			next = __stage2_range_addr_end(addr, end, kvm_granule_size(level));
+
 		if (next != end)
 			cond_resched();
 	} while (addr = next, addr != end);
-- 
2.47.3

[PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Fuad Tabba]

Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
like the other stage-2 operations. Both pKVM flavors share
pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
from kvm_phys_size().

No functional change intended.

Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
---
 arch/arm64/include/asm/kvm_host.h |  2 ++
 arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
 2 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 97332ffcb1300..b36779bafae9b 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
 
 struct kvm_gfn_range;
 struct kvm_s2_fault_desc;
+struct kvm_pgtable;
 
 struct kvm_vm_s2_ops {
 	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
@@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
 	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
 				      phys_addr_t start, u64 size,
 				      bool may_block);
+	void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
 	int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
 };
 
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 10b425a0141be..9df3c80a5acd1 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
 	} while (addr = next, addr != end);
 }
 
-static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
+static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
 {
-	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
+	stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
+	kvm_pgtable_stage2_destroy_pgd(pgt);
+}
 
-	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
-	if (is_protected_kvm_enabled())
-		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
-	else
-		stage2_destroy_range(pgt, 0, BIT(ia_bits));
-
-	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
+static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
+{
+	pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
+	pkvm_pgtable_stage2_destroy_pgd(pgt);
 }
 
 /**
@@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
 	return 0;
 
 out_destroy_pgtable:
-	kvm_stage2_destroy(pgt);
+	kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
 	mmu->pgt = NULL;
 out_free_pgtable:
 	kfree(pgt);
@@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
 	write_unlock(&kvm->mmu_lock);
 
 	if (pgt) {
-		kvm_stage2_destroy(pgt);
+		kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
 		kfree(pgt);
 	}
 
@@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops protected_pkvm_vm_s2_ops = {
 	.vm_age_gfn			= no_age_gfn,
 	.vm_test_age_gfn		= no_age_gfn,
 	.vm_stage2_unmap_range		= no_stage2_unmap_range,
+	.vm_stage2_destroy		= pkvm_stage2_destroy,
 	.vm_mem_abort			= protected_pkvm_mem_abort,
 };
 
@@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
 	.vm_age_gfn			= pkvm_age_gfn,
 	.vm_test_age_gfn		= pkvm_test_age_gfn,
 	.vm_stage2_unmap_range		= pkvm_stage2_unmap_range,
+	.vm_stage2_destroy		= pkvm_stage2_destroy,
 	.vm_mem_abort			= kvm_vm_mem_abort,
 };
 
@@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
 	.vm_age_gfn			= kvm_vm_age_gfn,
 	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
 	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
+	.vm_stage2_destroy		= kvm_vm_stage2_destroy,
 	.vm_mem_abort			= kvm_vm_mem_abort,
 };
 
-- 
2.47.3

Re: [PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Suzuki K Poulose]

Hi Fuad

On 03/11/2026 08:47, Fuad Tabba wrote:
> Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
> KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
> like the other stage-2 operations. Both pKVM flavors share
> pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
> from kvm_phys_size().
> 
> No functional change intended.

Someone beat you on this (well almost) ;-)

https://lore.kernel.org/all/4304d0c7-2220-4add-9c71-e6dc6b4fbc11@arm.com


> 
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>   arch/arm64/include/asm/kvm_host.h |  2 ++
>   arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
>   2 files changed, 15 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index 97332ffcb1300..b36779bafae9b 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
>   
>   struct kvm_gfn_range;
>   struct kvm_s2_fault_desc;
> +struct kvm_pgtable;
>   
>   struct kvm_vm_s2_ops {
>   	bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
> @@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
>   	void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
>   				      phys_addr_t start, u64 size,
>   				      bool may_block);
> +	void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
>   	int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
>   };
>   
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 10b425a0141be..9df3c80a5acd1 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
>   	} while (addr = next, addr != end);
>   }
>   
> -static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
> +static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
>   {
> -	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
> +	stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
> +	kvm_pgtable_stage2_destroy_pgd(pgt);
> +}
>   
> -	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
> -	if (is_protected_kvm_enabled())
> -		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
> -	else
> -		stage2_destroy_range(pgt, 0, BIT(ia_bits));
> -
> -	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
> +static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
> +{
> +	pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
> +	pkvm_pgtable_stage2_destroy_pgd(pgt);
>   }
>   
>   /**
> @@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu, unsigned long t
>   	return 0;
>   
>   out_destroy_pgtable:
> -	kvm_stage2_destroy(pgt);
> +	kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);

minor nit: should we retain the kvm_stage2_destroy() to dispatch the 
callback ?

Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>

Cheers
Suzuki


>   	mmu->pgt = NULL;
>   out_free_pgtable:
>   	kfree(pgt);
> @@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>   	write_unlock(&kvm->mmu_lock);
>   
>   	if (pgt) {
> -		kvm_stage2_destroy(pgt);
> +		kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
>   		kfree(pgt);
>   	}
>   
> @@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops protected_pkvm_vm_s2_ops = {
>   	.vm_age_gfn			= no_age_gfn,
>   	.vm_test_age_gfn		= no_age_gfn,
>   	.vm_stage2_unmap_range		= no_stage2_unmap_range,
> +	.vm_stage2_destroy		= pkvm_stage2_destroy,
>   	.vm_mem_abort			= protected_pkvm_mem_abort,
>   };
>   
> @@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops = {
>   	.vm_age_gfn			= pkvm_age_gfn,
>   	.vm_test_age_gfn		= pkvm_test_age_gfn,
>   	.vm_stage2_unmap_range		= pkvm_stage2_unmap_range,
> +	.vm_stage2_destroy		= pkvm_stage2_destroy,
>   	.vm_mem_abort			= kvm_vm_mem_abort,
>   };
>   
> @@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops kvm_default_vm_s2_ops = {
>   	.vm_age_gfn			= kvm_vm_age_gfn,
>   	.vm_test_age_gfn		= kvm_vm_test_age_gfn,
>   	.vm_stage2_unmap_range		= kvm_vm_stage2_unmap_range,
> +	.vm_stage2_destroy		= kvm_vm_stage2_destroy,
>   	.vm_mem_abort			= kvm_vm_mem_abort,
>   };
>   

Re: [PATCH 2/3] KVM: arm64: Skip empty IPA ranges when destroying stage-2 page-tables

[Suzuki K Poulose]

Hi Fuad

On 03/11/2026 08:47, Fuad Tabba wrote:
> The page-table walker steps over an empty entry in one go, but
> kvm_stage2_destroy() starts a new walk for every block-sized chunk of
> the IPA space. Tearing down a VM that requests a 52-bit IPA then takes
> 2^27 walks with 16K pages (2^22 with 4K), however little it maps.
> crosvm requests the host's limit.
> 

The patch looks good to me, in fact I was thinking about the same thing
yesterday while looking at the same code from a different angle, to move
to the vm_s2_ops abstraction.

> After destroying each chunk, look up the entry at its start and, if it
> is empty and larger than a chunk, skip its whole range. The lookup reads

Could we not lookup first before we destroy, that way we can avoid 
destroy walks ?

> fields that hold the mapping tree on a pKVM stage-2, so this depends on
> the previous patch keeping pKVM out of stage2_destroy_range(). For the
> same reason, call kvm_pgtable_stage2_destroy_range() directly rather
> than through KVM_PGT_FN().



> 
> Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")
> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>   arch/arm64/kvm/mmu.c | 15 +++++++++++++--
>   1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 42977cf8a7f5d..10b425a0141be 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1010,9 +1010,20 @@ static void stage2_destroy_range(struct kvm_pgtable *pgt, phys_addr_t addr,
>   	u64 next;
>   
>   	do {
> +		kvm_pte_t pte;
> +		s8 level;
> +
>   		next = stage2_range_addr_end(addr, end);
> -		KVM_PGT_FN(kvm_pgtable_stage2_destroy_range)(pgt, addr,
> -							     next - addr);
> +		kvm_pgtable_stage2_destroy_range(pgt, addr, next - addr);

minor nit: Given the kvm_pgtable_stage2_destroy_range() is the only 
called from here, and this caller would benefit from the walk
information, would it make sense to return a hint back ? Not sure how
much of that is possible. We do this in the RMM, always return a hint
of the next possible address. Of course this is not that expensive
as the CCA case and could be dealt in a separate series.

> +
> +		/*
> +		 * The walk freed every table it emptied, so skipping an empty
> +		 * entry here doesn't leave anything behind.
> +		 */
> +		if (!kvm_pgtable_get_leaf(pgt, addr, &pte, &level, 0) && !pte &&
> +		    level < KVM_PGTABLE_MIN_BLOCK_LEVEL)
> +			next = __stage2_range_addr_end(addr, end, kvm_granule_size(level));

Suzuki


> +
>   		if (next != end)
>   			cond_resched();
>   	} while (addr = next, addr != end);

Re: [PATCH 3/3] KVM: arm64: Move stage-2 teardown into kvm_vm_s2_ops

[Suzuki K Poulose]

On 03/11/2026 09:37, Suzuki K Poulose wrote:
> Hi Fuad
> 
> On 03/11/2026 08:47, Fuad Tabba wrote:
>> Replace kvm_stage2_destroy()'s is_protected_kvm_enabled() and
>> KVM_PGT_FN() dispatch with a vm_stage2_destroy hook in kvm_vm_s2_ops,
>> like the other stage-2 operations. Both pKVM flavors share
>> pkvm_stage2_destroy(). Like kvm_vm_stage2_destroy(), it takes the range
>> from kvm_phys_size().
>>
>> No functional change intended.
> 
> Someone beat you on this (well almost) ;-)
> 
> https://lore.kernel.org/all/4304d0c7-2220-4add-9c71-e6dc6b4fbc11@arm.com
> 
> 
>>
>> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
>> ---
>>   arch/arm64/include/asm/kvm_host.h |  2 ++
>>   arch/arm64/kvm/mmu.c              | 24 +++++++++++++-----------
>>   2 files changed, 15 insertions(+), 11 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/ 
>> asm/kvm_host.h
>> index 97332ffcb1300..b36779bafae9b 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -233,6 +233,7 @@ struct kvm_vcpu_ops {
>>   struct kvm_gfn_range;
>>   struct kvm_s2_fault_desc;
>> +struct kvm_pgtable;
>>   struct kvm_vm_s2_ops {
>>       bool (*vm_age_gfn)(struct kvm *kvm, struct kvm_gfn_range *range);
>> @@ -243,6 +244,7 @@ struct kvm_vm_s2_ops {
>>       void (*vm_stage2_unmap_range)(struct kvm_s2_mmu *mmu,
>>                         phys_addr_t start, u64 size,
>>                         bool may_block);
>> +    void (*vm_stage2_destroy)(struct kvm_pgtable *pgt);
>>       int (*vm_mem_abort)(const struct kvm_s2_fault_desc *s2fd);
>>   };
>> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
>> index 10b425a0141be..9df3c80a5acd1 100644
>> --- a/arch/arm64/kvm/mmu.c
>> +++ b/arch/arm64/kvm/mmu.c
>> @@ -1029,17 +1029,16 @@ static void stage2_destroy_range(struct 
>> kvm_pgtable *pgt, phys_addr_t addr,
>>       } while (addr = next, addr != end);
>>   }
>> -static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>> +static void kvm_vm_stage2_destroy(struct kvm_pgtable *pgt)
>>   {
>> -    unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
>> +    stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));

Also, now that stage2_destroy_range is only called for normal KVM,
could we fold that in here ?

Cheers
Suzuki


>> +    kvm_pgtable_stage2_destroy_pgd(pgt);
>> +}
>> -    /* pKVM walks its mapping tree, not the IPA space, and 
>> reschedules per mapping */
>> -    if (is_protected_kvm_enabled())
>> -        pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
>> -    else
>> -        stage2_destroy_range(pgt, 0, BIT(ia_bits));
>> -
>> -    KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>> +static void pkvm_stage2_destroy(struct kvm_pgtable *pgt)
>> +{
>> +    pkvm_pgtable_stage2_destroy_range(pgt, 0, kvm_phys_size(pgt->mmu));
>> +    pkvm_pgtable_stage2_destroy_pgd(pgt);
>>   }
>>   /**
>> @@ -1126,7 +1125,7 @@ int kvm_init_stage2_mmu(struct kvm *kvm, struct 
>> kvm_s2_mmu *mmu, unsigned long t
>>       return 0;
>>   out_destroy_pgtable:
>> -    kvm_stage2_destroy(pgt);
>> +    kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
> 
> minor nit: should we retain the kvm_stage2_destroy() to dispatch the 
> callback ?
> 
> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> Cheers
> Suzuki
> 
> 
>>       mmu->pgt = NULL;
>>   out_free_pgtable:
>>       kfree(pgt);
>> @@ -1246,7 +1245,7 @@ void kvm_free_stage2_pgd(struct kvm_s2_mmu *mmu)
>>       write_unlock(&kvm->mmu_lock);
>>       if (pgt) {
>> -        kvm_stage2_destroy(pgt);
>> +        kvm->arch.vm_s2_ops->vm_stage2_destroy(pgt);
>>           kfree(pgt);
>>       }
>> @@ -3152,6 +3151,7 @@ static const struct kvm_vm_s2_ops 
>> protected_pkvm_vm_s2_ops = {
>>       .vm_age_gfn            = no_age_gfn,
>>       .vm_test_age_gfn        = no_age_gfn,
>>       .vm_stage2_unmap_range        = no_stage2_unmap_range,
>> +    .vm_stage2_destroy        = pkvm_stage2_destroy,
>>       .vm_mem_abort            = protected_pkvm_mem_abort,
>>   };
>> @@ -3161,6 +3161,7 @@ static const struct kvm_vm_s2_ops pkvm_vm_s2_ops 
>> = {
>>       .vm_age_gfn            = pkvm_age_gfn,
>>       .vm_test_age_gfn        = pkvm_test_age_gfn,
>>       .vm_stage2_unmap_range        = pkvm_stage2_unmap_range,
>> +    .vm_stage2_destroy        = pkvm_stage2_destroy,
>>       .vm_mem_abort            = kvm_vm_mem_abort,
>>   };
>> @@ -3170,6 +3171,7 @@ static const struct kvm_vm_s2_ops 
>> kvm_default_vm_s2_ops = {
>>       .vm_age_gfn            = kvm_vm_age_gfn,
>>       .vm_test_age_gfn        = kvm_vm_test_age_gfn,
>>       .vm_stage2_unmap_range        = kvm_vm_stage2_unmap_range,
>> +    .vm_stage2_destroy        = kvm_vm_stage2_destroy,
>>       .vm_mem_abort            = kvm_vm_mem_abort,
>>   };
> 

Re: [PATCH 1/3] KVM: arm64: Don't walk the whole IPA space when destroying a pKVM stage-2

[Vincent Donnefort]

On Tue, Nov 03, 2026 at 08:47:40AM +0000, Fuad Tabba wrote:
> Since commit 4ddfab5436b6 ("KVM: arm64: Reschedule as needed when
> destroying the stage-2 page-tables"), kvm_stage2_destroy() steps through
> the IPA space in block-sized chunks, and with pKVM each chunk is a
> lookup in the VM's mapping tree. However, pKVM gives every VM the host's
> IPA limit, so with LPA2 a teardown makes 2^27 lookups with 16K pages
> (2^22 with 4K), nearly all of them for empty space.
> 
> Walk the mapping tree once instead, and reschedule after passing each
> mapping to the reclaim or unshare helper. The unshare helper can't
> reschedule itself, as the map and unmap paths call it under the
> mmu_lock. A failed unshare now leaves only its own mapping behind, not
> the rest of its chunk.

Not a problem now, but don't we make the device assignment story harder?

> 
> Fixes: 4ddfab5436b69 ("KVM: arm64: Reschedule as needed when destroying the stage-2 page-tables")

Isn't it an optimisation?

> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
> ---
>  arch/arm64/kvm/mmu.c  |  7 ++++++-
>  arch/arm64/kvm/pkvm.c | 17 +++++++++++++----
>  2 files changed, 19 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
> index 1db10d8d6db73..42977cf8a7f5d 100644
> --- a/arch/arm64/kvm/mmu.c
> +++ b/arch/arm64/kvm/mmu.c
> @@ -1022,7 +1022,12 @@ static void kvm_stage2_destroy(struct kvm_pgtable *pgt)
>  {
>  	unsigned int ia_bits = VTCR_EL2_IPA(pgt->mmu->vtcr);
>  
> -	stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	/* pKVM walks its mapping tree, not the IPA space, and reschedules per mapping */
> +	if (is_protected_kvm_enabled())
> +		pkvm_pgtable_stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +	else
> +		stage2_destroy_range(pgt, 0, BIT(ia_bits));
> +
>  	KVM_PGT_FN(kvm_pgtable_stage2_destroy_pgd)(pgt);
>  }
>  
> diff --git a/arch/arm64/kvm/pkvm.c b/arch/arm64/kvm/pkvm.c
> index 016997288fd23..992ed3ca6bdd9 100644
> --- a/arch/arm64/kvm/pkvm.c
> +++ b/arch/arm64/kvm/pkvm.c
> @@ -504,6 +504,7 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  {
>  	struct kvm *kvm = kvm_s2_mmu_to_kvm(pgt->mmu);
>  	pkvm_handle_t handle = kvm->arch.pkvm.handle;
> +	struct pkvm_mapping *mapping;
>  
>  	if (!handle)
>  		return;
> @@ -513,10 +514,18 @@ void pkvm_pgtable_stage2_destroy_range(struct kvm_pgtable *pgt,
>  		kvm->arch.pkvm.is_dying = true;
>  	}
>  
> -	if (kvm_vm_is_protected(kvm))
> -		__pkvm_pgtable_stage2_reclaim(pgt, addr, addr + size);
> -	else
> -		__pkvm_pgtable_stage2_unshare(pgt, addr, addr + size);
> +	/* Called once for the whole IPA space, so reschedule between mappings */
> +	for_each_mapping_in_range_safe(pgt, addr, addr + size, mapping) {
> +		u64 start = __pkvm_mapping_start(mapping);
> +		u64 end = __pkvm_mapping_end(mapping) + 1;

__pkvm_mapping_* are rb-tree internals. I think it is better to use explicitly:

  start = gfn_to_gpa(mapping->gfn);
  end = start + (mapping->nr_pages * PAGE_SIZE)

Also, I wonder if we'd better WARN if the mapping isn't aligned with the
destroyed range?

-- 
Vincent

> +
> +		if (kvm_vm_is_protected(kvm))
> +			__pkvm_pgtable_stage2_reclaim(pgt, start, end);
> +		else
> +			__pkvm_pgtable_stage2_unshare(pgt, start, end);
> +
> +		cond_resched();
> +	}
>  }
>  
>  void pkvm_pgtable_stage2_destroy_pgd(struct kvm_pgtable *pgt)
> -- 
> 2.47.3
> 
>