[GIT PULL] KVM: x86: MMU changes for 6.6

[Sean Christopherson <seanjc@google.com>]

Please pull MMU changes for 6.6, with a healthy dose of KVMGT cleanups mixed in.
The other highlight is finally purging the old MMU_DEBUG code and replacing it
with CONFIG_KVM_PROVE_MMU.

All KVMGT patches have been reviewed/acked and tested by KVMGT folks.  A *huge*
thanks to them for all the reviews and testing, and to Yan in particular.

If you have time, please take a closer look at commit a328a359d99b ("KVM:
x86/mmu: Use dummy root, backed by zero page, for !visible guest roots"), it's
held up in (limited) testing, but I'd love more eyeballs on it.

The following changes since commit fdf0eaf11452d72945af31804e2a1048ee1b574c:

  Linux 6.5-rc2 (2023-07-16 15:10:37 -0700)

are available in the Git repository at:

  https://github.com/kvm-x86/linux.git tags/kvm-x86-mmu-6.6

for you to fetch changes up to bfd926291c585600ace63ea3b6eb1458aa067f4f:

  KVM: x86/mmu: Include mmu.h in spte.h (2023-08-25 09:03:51 -0700)

----------------------------------------------------------------
KVM x86 MMU changes for 6.6:

 - Rip out the ancient MMU_DEBUG crud and replace the useful bits with
   CONFIG_KVM_PROVE_MMU

 - Overhaul KVM's page-track APIs, and KVMGT's usage, to reduce the API surface
   that is needed by external users (currently only KVMGT), and fix a variety
   of issues in the process

 - Fix KVM's handling of !visible guest roots to avoid premature triple fault
   injection by loading a dummy root backed by the zero page

----------------------------------------------------------------
Like Xu (1):
      KVM: x86/mmu: Move the lockdep_assert of mmu_lock to inside clear_dirty_pt_masked()

Mingwei Zhang (1):
      KVM: x86/mmu: Plumb "struct kvm" all the way to pte_list_remove()

Sean Christopherson (42):
      KVM: x86/mmu: Guard against collision with KVM-defined PFERR_IMPLICIT_ACCESS
      KVM: x86/mmu: Delete pgprintk() and all its usage
      KVM: x86/mmu: Delete rmap_printk() and all its usage
      KVM: x86/mmu: Delete the "dbg" module param
      KVM: x86/mmu: Avoid pointer arithmetic when iterating over SPTEs
      KVM: x86/mmu: Cleanup sanity check of SPTEs at SP free
      KVM: x86/mmu: Rename MMU_WARN_ON() to KVM_MMU_WARN_ON()
      KVM: x86/mmu: Convert "runtime" WARN_ON() assertions to WARN_ON_ONCE()
      KVM: x86/mmu: Bug the VM if a vCPU ends up in long mode without PAE enabled
      KVM: x86/mmu: Replace MMU_DEBUG with proper KVM_PROVE_MMU Kconfig
      KVM: x86/mmu: Use BUILD_BUG_ON_INVALID() for KVM_MMU_WARN_ON() stub
      KVM: x86/mmu: BUG() in rmap helpers iff CONFIG_BUG_ON_DATA_CORRUPTION=y
      drm/i915/gvt: Verify pfn is "valid" before dereferencing "struct page"
      drm/i915/gvt: Verify hugepages are contiguous in physical address space
      drm/i915/gvt: Put the page reference obtained by KVM's gfn_to_pfn()
      drm/i915/gvt: Explicitly check that vGPU is attached before shadowing
      drm/i915/gvt: Error out on an attempt to shadowing an unknown GTT entry type
      drm/i915/gvt: Don't rely on KVM's gfn_to_pfn() to query possible 2M GTT
      drm/i915/gvt: Use an "unsigned long" to iterate over memslot gfns
      drm/i915/gvt: Drop unused helper intel_vgpu_reset_gtt()
      drm/i915/gvt: Protect gfn hash table with vgpu_lock
      KVM: x86/mmu: Move kvm_arch_flush_shadow_{all,memslot}() to mmu.c
      KVM: x86/mmu: Don't rely on page-track mechanism to flush on memslot change
      KVM: x86/mmu: Don't bounce through page-track mechanism for guest PTEs
      KVM: drm/i915/gvt: Drop @vcpu from KVM's ->track_write() hook
      KVM: x86: Reject memslot MOVE operations if KVMGT is attached
      drm/i915/gvt: Don't bother removing write-protection on to-be-deleted slot
      KVM: x86/mmu: Move KVM-only page-track declarations to internal header
      KVM: x86/mmu: Use page-track notifiers iff there are external users
      KVM: x86/mmu: Drop infrastructure for multiple page-track modes
      KVM: x86/mmu: Rename page-track APIs to reflect the new reality
      KVM: x86/mmu: Assert that correct locks are held for page write-tracking
      KVM: x86/mmu: Bug the VM if write-tracking is used but not enabled
      KVM: x86/mmu: Drop @slot param from exported/external page-track APIs
      KVM: x86/mmu: Handle KVM bookkeeping in page-track APIs, not callers
      drm/i915/gvt: Drop final dependencies on KVM internal details
      KVM: x86/mmu: Add helper to convert root hpa to shadow page
      KVM: x86/mmu: Harden new PGD against roots without shadow pages
      KVM: x86/mmu: Harden TDP MMU iteration against root w/o shadow page
      KVM: x86/mmu: Disallow guest from using !visible slots for page tables
      KVM: x86/mmu: Use dummy root, backed by zero page, for !visible guest roots
      KVM: x86/mmu: Include mmu.h in spte.h

Yan Zhao (5):
      drm/i915/gvt: remove interface intel_gvt_is_valid_gfn
      drm/i915/gvt: Don't try to unpin an empty page range
      KVM: x86: Add a new page-track hook to handle memslot deletion
      drm/i915/gvt: switch from ->track_flush_slot() to ->track_remove_region()
      KVM: x86: Remove the unused page-track hook track_flush_slot()

 arch/x86/include/asm/kvm_host.h       |  16 +-
 arch/x86/include/asm/kvm_page_track.h |  73 +++-----
 arch/x86/kvm/Kconfig                  |  13 ++
 arch/x86/kvm/mmu.h                    |   2 +
 arch/x86/kvm/mmu/mmu.c                | 319 ++++++++++++++++------------------
 arch/x86/kvm/mmu/mmu_internal.h       |  24 +--
 arch/x86/kvm/mmu/page_track.c         | 258 +++++++++++++--------------
 arch/x86/kvm/mmu/page_track.h         |  58 +++++++
 arch/x86/kvm/mmu/paging_tmpl.h        |  41 +++--
 arch/x86/kvm/mmu/spte.c               |   6 +-
 arch/x86/kvm/mmu/spte.h               |  21 ++-
 arch/x86/kvm/mmu/tdp_iter.c           |  11 +-
 arch/x86/kvm/mmu/tdp_mmu.c            |  33 ++--
 arch/x86/kvm/x86.c                    |  22 +--
 drivers/gpu/drm/i915/gvt/gtt.c        | 102 ++---------
 drivers/gpu/drm/i915/gvt/gtt.h        |   1 -
 drivers/gpu/drm/i915/gvt/gvt.h        |   3 +-
 drivers/gpu/drm/i915/gvt/kvmgt.c      | 120 +++++--------
 drivers/gpu/drm/i915/gvt/page_track.c |  10 +-
 include/linux/kvm_host.h              |  19 ++
 20 files changed, 568 insertions(+), 584 deletions(-)
 create mode 100644 arch/x86/kvm/mmu/page_track.h