Re: [PATCH v3 0/3] x86/crash: Fix double NMI shootdown bug

[Andrew Cooper <Andrew.Cooper3@citrix.com>]

On 15/11/2022 16:56, Sean Christopherson wrote:
> On Tue, Nov 15, 2022, Andrew Cooper wrote:
>> On 14/11/2022 23:34, Sean Christopherson wrote:
>>> Tom,
>>>
>>> I Cc'd you this time around because the APM doesn't explicitly state that
>>> GIF is set when EFER.SVME is disabled.  KVM's nSVM emulation does set GIF
>>> in this case, but it's not clear whether or not KVM is making up behavior.
>>> If clearing EFER.SVME doesn't set GIF, then patch 1 needs to be modified
>>> to try STGI before clearing EFER.SVME, e.g. if a crash is initiated from
>>> KVM between CLGI and STGI.  Responding CPUs are "safe" because GIF=0 also
>>> blocks NMIs, but the initiating CPU might leave GIF=0 when jumping into
>>> the new kernel.
>> GIF exists independently of EFER.SVME.
>>
>> It is also gets set by the SKINIT instruction, which is why there is an
>> asymmetry on the #UD conditions of STGI and CLGI.
>>
>> STGI is also intended to be used by the DLME once critical
>> initialisation is done, hence why it's dependent on EFER.SVME || SKINIT.
> Gah, stupid APM.  The pseudocode states "EFER.SVME || CPUID.SKINIT", but the
> description and the comment both say that SVM must be enabled to execute SKINIT,
> which made me hope that disabling SVM would reset everything.
>
>   This instruction generates a #UD exception if SVM is not enabled. See “Enabling
>   SVM” in AMD64 Architecture Programmer’s Manual Volume 2: System Instructions,
>   order# 24593.
>
> ...
>
>   IF ((EFER.SVME == 0) && !(CPUID 8000_0001.ECX[SKINIT]) || (!PROTECTED_MODE))
>           EXCEPTION [#UD]  // This instruction can only be executed
>                            // in protected mode with SVM enabled
>                                                      ^^^^^^^^^^^

/sigh - and there needs to be one extra set of brackets to remove the
ambiguity in that pseudocode.


I think it's tied up with the complexity of the VMM LOCK feature.  Just
as with Intel and the SMX inside/outside VMX bits in
MSR_FEATURE_CONTROL, there's a requirement stated in the PPR/BKDG that
firmware offers the option to enable or disable SVM, where disable is
actually set the lock field.

VM_CR.SVM_LOCK can only be configured when EFER.SVME is clear, after
which SKINIT (and STGI) for DTRM is still usable, while general SVM isn't.

In this case, the CPU is operating with EFER.SVME=0 and a working(ish) GIF.


I find it curious that both Intel and AMD have VMM+DTRM technology that
overlap (feature wise), appeared at the same time in silicon, *and* have
slightly weird ways of force disabling VMM while keeping DTRM
operational.  Funny how these two features have played out.

~Andrew