Re: [RFC PATCH 5/5] KVM: arm64: Enable HAFDBS for guests not on migration

[Leonardo Bras <leo.bras@arm.com>]

On Tue, Sep 15, 2026 at 05:10:02PM -0700, Oliver Upton wrote:
> Hi,
> 
> On Tue, Sep 01, 2026 at 06:15:56PM +0100, Leonardo Bras wrote:
> > When dirty-logging is disabled, even non-write faults make a page dirty,
> > which avoids a second fault when the page is actually written to.
> > 
> > On dirty-logging enable, this approach causes all (writable) pages on the
> > memslot to be marked clean, even if they were not written to, which can
> > take a lot of time, while holding the MMU lock, doing atomic writes to
> > PTEs.
> 
> Do you have any performance numbers for this? Enabling HAFDBS seems a
> bit involved to avoid some stores on the first pass.

Not yet, but if the idea does not look too crazy I can find hardware and 
collect some data :)

> 
> > So, if the system supports VHE + HAFDBS, keep the non-write-faulted page as
> > writable-clean, and let HAFDBS update that on demand when a write happens.
> > 
> > When dirty-tracking actually starts, disable HAFDBS as having it on
> > avoids the same fault that is used for dirty-logging.
> > 
> > Signed-off-by: Leonardo Bras <leo.bras@arm.com>
> > ---
> >  arch/arm64/include/asm/kvm_mmu.h    |  6 ++++++
> >  arch/arm64/include/asm/kvm_nested.h |  9 +++++++--
> >  arch/arm64/kvm/arm.c                |  7 +++++++
> >  arch/arm64/kvm/mmu.c                | 26 +++++++++++++++++++++++++-
> >  4 files changed, 45 insertions(+), 3 deletions(-)
> > 
> > diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
> > index 6eae7e7e2a68..3defa1a988d3 100644
> > --- a/arch/arm64/include/asm/kvm_mmu.h
> > +++ b/arch/arm64/include/asm/kvm_mmu.h
> > @@ -383,20 +383,26 @@ static inline void kvm_fault_unlock(struct kvm *kvm)
> >   * and CMOs are NOP'd. This has the effect of no longer requiring a
> >   * KVA for addresses mapped into the S2. The presence of these features
> >   * are thus necessary to support cacheable S2 mapping of VM_PFNMAP.
> >   */
> >  static inline bool kvm_supports_cacheable_pfnmap(void)
> >  {
> >  	return cpus_have_final_cap(ARM64_HAS_STAGE2_FWB) &&
> >  	       cpus_have_final_cap(ARM64_HAS_CACHE_DIC);
> >  }
> >  
> > +static inline bool kvm_supports_hafdbs(struct kvm *kvm)
> > +{
> > +	return IS_ENABLED(CONFIG_ARM64_HW_AFDBM) &&
> 
> cpucap_is_possible() is the right place to park this.
> 

Noted!

> > 						      has_vhe() &&
> 
> I don't see a reason why this needs to be constrained to VHE-only.
> 

Humm, in nVHE would not the host kernel run in EL1?
I thought that this being a feature that depends on EL2 registers host 
would need to be in EL2 to make use of it.

That being said, I understand very little of how this works, so I 
constrained to VHE only at the start.

Would this work in nVHE?

> > +		!kvm_vcpu_has_nv(kvm) && cpus_have_final_cap(ARM64_HW_DBM);
> 
> Same thing goes for nested... KVM can make use of HAFDBS in the
> canonical stage-2 MMU (or even a shadow stage-2) independent of the
> guest hypervisor.
> 

Humm, I remember reaching the conclusion that it could not be used if the 
guest supported NV. Let's say:

L0 - Host	- Has HAFDBS enabled
L1 - Hypervisor - Has HAFDBS disabled
L2 - Guest	-

Let's say guest writes to a page, and the shadow S2 has DBM=1, so it's 
marked as WD by HAFDBS. Since no fault was taken, how would the L1 be able 
to update it's S2 pagetables to mark the page dirty?

(We would have to transverse the Shadow S2 Pagetable updating the original 
S2 pagetable)

I was wondering, thought, that we could emulate it in the last level 
hypervisor, if it's guest does not support nested guests. That would mean 
we can have the last-1 level hypervisor to update the S2 pagetable on the 
last level hypervisor without it having to fault. Ex:

L0 Host - HAFDBS disabled
[...]
Ln-1 Hypervisor - HAFDBS disabled
Ln   Hypervisor - HAFDBS enabled
Ln+1 Guest - No E2H feature

When the guest writes to a page, the host should receive a fault, that IIUC 
have to propagate down up to Ln Hyp. If Ln Hyp has HAFDBS, we could skip 
injecting a fault in Ln Hyp, as  Ln-1 Hyp could emulate HAFDBS and write 
the dirty bit to S2 pagetagle of Ln+1 guest, that resides in Ln memory.

Not sure if the troulbe would be worth, though.
Does it make sense?



> >  #ifdef CONFIG_PTDUMP_STAGE2_DEBUGFS
> >  void kvm_s2_ptdump_create_debugfs(struct kvm *kvm);
> >  void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu);
> >  void kvm_nested_s2_ptdump_remove_debugfs(struct kvm_s2_mmu *mmu);
> >  #else
> >  static inline void kvm_s2_ptdump_create_debugfs(struct kvm *kvm) {}
> >  static inline void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu) {}
> >  static inline void kvm_nested_s2_ptdump_remove_debugfs(struct kvm_s2_mmu *mmu) {}
> >  #endif /* CONFIG_PTDUMP_STAGE2_DEBUGFS */
> >  
> > diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h
> > index 1ed708335809..9242b5d665af 100644
> > --- a/arch/arm64/include/asm/kvm_nested.h
> > +++ b/arch/arm64/include/asm/kvm_nested.h
> > @@ -1,24 +1,29 @@
> >  /* SPDX-License-Identifier: GPL-2.0 */
> >  #ifndef __ARM64_KVM_NESTED_H
> >  #define __ARM64_KVM_NESTED_H
> >  
> >  #include <linux/bitfield.h>
> >  #include <linux/kvm_host.h>
> >  #include <asm/kvm_emulate.h>
> >  #include <asm/kvm_pgtable.h>
> >  
> > -static inline bool vcpu_has_nv(const struct kvm_vcpu *vcpu)
> > +static inline bool kvm_vcpu_has_nv(const struct kvm *kvm)
> 
> The name would suggest this thing takes a vcpu pointer...
> 

Ah, that name was based on
#define kvm_vcpu_has_feature(k, f)  __vcpu_has_feature(&(k)->arch, #(f))

That takes a kvm struct to check the kvm_arch one, instead of looking into 
the vcpu. I did it like this because there were some scenarios it was not 
quite straightforward to get the vcpu to use vcpu_has_nv(), which takes a 
vcpu.


Thanks!
Leo