Re: [PATCH v6 00/17] Introducing AMD x2AVIC and hybrid-AVIC modes

[Maxim Levitsky <mlevitsk@redhat.com>]

On Tue, 2022-06-28 at 20:20 +0700, Suthikulpanit, Suravee wrote:
> Maxim,
> 
> On 5/19/2022 5:26 PM, Suravee Suthikulpanit wrote:
> > Introducing support for AMD x2APIC virtualization. This feature is
> > indicated by the CPUID Fn8000_000A EDX[14], and it can be activated
> > by setting bit 31 (enable AVIC) and bit 30 (x2APIC mode) of VMCB
> > offset 60h.
> > 
> > With x2AVIC support, the guest local APIC can be fully virtualized in
> > both xAPIC and x2APIC modes, and the mode can be changed during runtime.
> > For example, when AVIC is enabled, the hypervisor set VMCB bit 31
> > to activate AVIC for each vCPU. Then, it keeps track of each vCPU's
> > APIC mode, and updates VMCB bit 30 to enable/disable x2APIC
> > virtualization mode accordingly.
> > 
> > Besides setting bit VMCB bit 30 and 31, for x2AVIC, kvm_amd driver needs
> > to disable interception for the x2APIC MSR range to allow AVIC hardware
> > to virtualize register accesses.
> > 
> > This series also introduce a partial APIC virtualization (hybrid-AVIC)
> > mode, where APIC register accesses are trapped (i.e. not virtualized
> > by hardware), but leverage AVIC doorbell for interrupt injection.
> > This eliminates need to disable x2APIC in the guest on system without
> > x2AVIC support. (Note: suggested by Maxim)
> > 
> > Testing for v5:
> >    * Test partial AVIC mode by launching a VM with x2APIC mode
> >    * Tested booting a Linux VM with x2APIC physical and logical modes upto 512 vCPUs.
> >    * Test the following nested SVM test use cases:
> > 
> >               L0     |    L1   |   L2
> >         ----------------------------------
> >                 AVIC |    APIC |    APIC
> >                 AVIC |    APIC |  x2APIC
> >          hybrid-AVIC |  x2APIC |    APIC
> >          hybrid-AVIC |  x2APIC |  x2APIC
> >               x2AVIC |    APIC |    APIC
> >               x2AVIC |    APIC |  x2APIC
> >               x2AVIC |  x2APIC |    APIC
> >               x2AVIC |  x2APIC |  x2APIC
> 
> With the commit 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base"),
> APICV/AVIC is now inhibit when the guest kernel boots w/ option "nox2apic" or "x2apic_phys"
> due to APICV_INHIBIT_REASON_APIC_ID_MODIFIED.
> 
> These cases used to work. In theory, we should be able to allow AVIC works in this case.
> Is there a way to modify logic in kvm_lapic_xapic_id_updated() to allow these use cases
> to work w/ APICv/AVIC?
> 
> Best Regards,
> Suravee
> 

This seems very strange, I assume you test the kvm/queue of today,

which contains a fix for a typo I had in the list of inhibit reasons
(commit 5bdae49fc2f689b5f896b54bd9230425d3643dab - KVM: SEV: fix misplaced closing parenthesis)


Could you share more details on the test? How many vCPUs in the guest, is x2apic exposed to the guest?


Looking through the code the the __x2apic_disable, touches the MSR_IA32_APICBASE so I would expect
the APICV_INHIBIT_REASON_APIC_BASE_MODIFIED inhibit to be triggered and not APICV_INHIBIT_REASON_APIC_ID_MODIFIED


I don't see yet how the x2apic_phys can trigger these inhibits.

Best regards,
	Maxim Levitsky