Re: [PATCH 4/4] KVM: x86: Add vCPU stat for APICv interrupt injections causing #VMEXIT

[Alejandro Jimenez <alejandro.j.jimenez@oracle.com>]

On 6/3/24 20:14, Sean Christopherson wrote:
> On Mon, Apr 29, 2024, Alejandro Jimenez wrote:
>> Even when APICv/AVIC is active, certain guest accesses to its local APIC(s)
>> cannot be fully accelerated, and cause a #VMEXIT to allow the VMM to
>> emulate the behavior and side effects. Expose a counter stat for these
>> specific #VMEXIT types.
>>
>> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
>> Signed-off-by: Alejandro Jimenez <alejandro.j.jimenez@oracle.com>
>> ---
>>   arch/x86/include/asm/kvm_host.h | 1 +
>>   arch/x86/kvm/svm/avic.c         | 7 +++++++
>>   arch/x86/kvm/vmx/vmx.c          | 2 ++
>>   arch/x86/kvm/x86.c              | 1 +
>>   4 files changed, 11 insertions(+)
>>
>> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
>> index e7e3213cefae..388979dfe9f3 100644
>> --- a/arch/x86/include/asm/kvm_host.h
>> +++ b/arch/x86/include/asm/kvm_host.h
>> @@ -1576,6 +1576,7 @@ struct kvm_vcpu_stat {
>>   	u64 guest_mode;
>>   	u64 notify_window_exits;
>>   	u64 apicv_active;
>> +	u64 apicv_unaccelerated_inj;
> 
> The stat name doesn't match the changelog or the code.  The AVIC updates in
> avic_incomplete_ipi_interception() are unaccelerated _injection_, they're
> unaccelarated _delivery_.  And in those cases, the fact that delivery wasn't
> accelerated is relatively uninteresting in most cases.
> 

Yeah, this was my flawed attempt to interpret/implement Paolo's comment in the RFC thread:

"... for example I'd add an interrupt_injections stat for unaccelerated injections causing a vmexit or otherwise hitting lapic.c"

so I incorrectly bundled together APIC accesses that result in #VMEXIT and end up requiring additional emulation (while managing to miss the handle_apic_access() case).


> And avic_unaccelerated_access_interception() and handle_apic_write() don't
> necessarily have anything to do with injection.

apicv_unaccelerated_acccess is perhaps a better name (assuming stat is updated in handle_apic_access() as well)?

> 
> On the flip side, the slow paths for {svm,vmx}_deliver_interrupt() are very
> explicitly unnaccelerated injection.

Now that you highlight this, I think it might be closer to Paolo's idea. i.e. a stat for the slow path on these can be contrasted/compared with the kvm_apicv_accept_irq tracepoint that is hit on the fast path.
My initial reaction would be to update a stat for the fast path, as a confirmation that apicv is active which is how/why I typically use the kvm_apicv_accept_irq tracepoint, but that becomes redundant by having the apicv_active stat on PATCH 1.

So, if you don't think it is useful to have a general apicv_unaccelerated_acccess counter, I can drop this patch.

Thank you,

Alejandro

> 
> It's not entirely clear from the changelog what the end goal of this stat is.
> A singular stat for all APICv/AVIC access VM-Exits seems uninteresting, as such
> a stat essentially just captures that the guest is active.  Maaaybe someone could
> glean info from comparing two VMs, but even that is dubious.  E.g. if a guest is
> doing something function and generating a lot of avic_incomplete_ipi_interception()
> exits, those will likely be in the noise due to the total volume of other AVIC
> exits.
> 
>>   };
>>   
>>   struct x86_instruction_info;
>> diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
>> index 4b74ea91f4e6..274041d3cf66 100644
>> --- a/arch/x86/kvm/svm/avic.c
>> +++ b/arch/x86/kvm/svm/avic.c
>> @@ -517,6 +517,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
>>   			kvm_apic_write_nodecode(vcpu, APIC_ICR);
>>   		else
>>   			kvm_apic_send_ipi(apic, icrl, icrh);
>> +
>> +		++vcpu->stat.apicv_unaccelerated_inj;
>>   		break;
>>   	case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING:
>>   		/*
>> @@ -525,6 +527,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
>>   		 * vcpus. So, we just need to kick the appropriate vcpu.
>>   		 */
>>   		avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh, index);
>> +
>> +		++vcpu->stat.apicv_unaccelerated_inj;
>>   		break;
>>   	case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE:
>>   		WARN_ONCE(1, "Invalid backing page\n");
>> @@ -704,6 +708,9 @@ int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu)
>>   
>>   	trace_kvm_avic_unaccelerated_access(vcpu->vcpu_id, offset,
>>   					    trap, write, vector);
>> +
>> +	++vcpu->stat.apicv_unaccelerated_inj;
>> +
>>   	if (trap) {
>>   		/* Handling Trap */
>>   		WARN_ONCE(!write, "svm: Handling trap read.\n");
>> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
>> index f10b5f8f364b..a7487f12ded1 100644
>> --- a/arch/x86/kvm/vmx/vmx.c
>> +++ b/arch/x86/kvm/vmx/vmx.c
>> @@ -5657,6 +5657,8 @@ static int handle_apic_write(struct kvm_vcpu *vcpu)
>>   {
>>   	unsigned long exit_qualification = vmx_get_exit_qual(vcpu);
>>   
>> +	++vcpu->stat.apicv_unaccelerated_inj;
>> +
>>   	/*
>>   	 * APIC-write VM-Exit is trap-like, KVM doesn't need to advance RIP and
>>   	 * hardware has done any necessary aliasing, offset adjustments, etc...
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index 03cb933920cb..c8730b0fac87 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -307,6 +307,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
>>   	STATS_DESC_IBOOLEAN(VCPU, guest_mode),
>>   	STATS_DESC_COUNTER(VCPU, notify_window_exits),
>>   	STATS_DESC_IBOOLEAN(VCPU, apicv_active),
>> +	STATS_DESC_COUNTER(VCPU, apicv_unaccelerated_inj),
>>   };
>>   
>>   const struct kvm_stats_header kvm_vcpu_stats_header = {
>> -- 
>> 2.39.3
>>