Re: [RFC PATCH 06/24] KVM: SEV: Track ASID->vCPU instead of ASID->VMCB

[Sean Christopherson <seanjc@google.com>]

On Mon, Jun 23, 2025, Tom Lendacky wrote:
> On 6/20/25 18:13, Sean Christopherson wrote:
> > On Wed, Mar 26, 2025, Yosry Ahmed wrote:
> > The more I think about all of this, the less it makes sense.  The *entire* point
> > of an ASID is to tag TLB entries so that a flush isn't required when running code
> > for the same address space.
> > 
> > The main problem I'm struggling with is that, as usual, the APM doesn't properly
> > document anything, and just gives "suggestions" for the VMM.  *sigh*
> > 
> > As I read it, these snippets from the APM are saying ASIDs tag only GPA=>PA entries
> > when NPT is in use.
> > 
> >   TLB entries are tagged with Address Space Identifier (ASID) bits to distinguish
> >   different guest virtual address spaces when shadow page tables are used, or
> >   different guest physical address spaces when nested page tables are used. The
> >   VMM can choose a software strategy in which it keeps multiple shadow page tables,
> >   and/or multiple nested page tables in processors that support nested paging,
> >   up-to-date; the VMM can allocate a different ASID for each shadow or nested
> >   page table. This allows switching to a new process in a guest under shadow
> >   paging (changing CR3 contents), or to a new guest under nested paging (changing
> >   nCR3 contents), without flushing the TLBs.
> > 
> >   Note that because an ASID is associated with the guest's physical address
> >   space, it is common across all of the guest's virtual address spaces within a
> >   processor. This differs from shadow page tables where ASIDs tag individual
> >   guest virtual address spaces. Note also that the same ASID may or may not be
> >   associated with the same address space across all processors in a
> >   multiprocessor system, for either nested tables or shadow tables; this depends
> >   on how the VMM manages ASID assignment.
> > 
> > But then the "15.16.1 TLB Flush" section says this, without any qualification
> > whatsoever that it applies only to shadow paging.
> > 
> >   A MOV-to-CR3, a task switch that changes CR3, or clearing or setting CR0.PG or
> >   bits PGE, PAE, PSE of CR4 affects only the TLB entries belonging to the current
> >   ASID, regardless of whether the operation occurred in host or guest mode. The
> >   current ASID is 0 when the CPU is not inside a guest context.
> > 
> > And honestly, tagging only GPA=>PA entries doesn't make any sense, because
> > GVA=>GPA needs to be tagged with *something*.  And the APM doesn't say anything
> > about caching GPA=>PA translations, only about caching VA=>PA.
> 
> VA=>PA translations are always tagged with a TLB tag value.  Outside of
> SEV-SNP, the TLB tag value is ASID.
> 
> So for those guests, VA=>PA translation are tagged with the ASID.  For
> SEV-SNP guests, see below.
> 
> > 
> > The thing that doesn't fit is that SEV+ uses ASIDs on a per-VM basis.  I suggested
> > per-VM ASIDs for all VM types based solely on that fact, but now I'm wondering if
> > it's SEV+ that crazy and broken.  Because if ASIDs also tag GVA=>GPA, then SEV has
> > a massive architectural security hole, e.g. a malicious hypervisor can coerce the
> > CPU into using a stale GVA=>GPA TLB entry by switching vCPUs and letting guest
> > process with CR3=x access memory for guest process with CR3=y.  But again, if
> > ASIDs don't tag GVA=>GPA, then what provides isolation between vCPUs!?!?!
> 
> No.
> 
> For SEV/SEV-ES guests, the HV (which remains partially trusted) must do a
> TLB flush before running a different VMCB of the same guest, in order to
> avoid this problem. This code is in pre_sev_run().
> 
> For SEV-SNP guests, this is handled automatically by hardware through the
> PCPU_ID and TLB_ID VMSA fields (documented somewhat in APM 15.36.15).

Aha!  I knew I had to be missing something.  Rule #1: don't doubt Kaplan ;-)

> In short, the TLB is tagged with {TLB_ID, ASID} and TLB_ID is managed by
> HW and guaranteed to be different for each vCPU of the guest running on a
> physical core. This ensures that the TLB tag is unique for each guest and
> for each vCPU of the guest.

Thanks Tom, very much appreciated!