[PATCH v3 00/18] KVM: x86/pmu: Add mediated vPMU PerfMon v5 support

[PATCH v3 00/18] KVM: x86/pmu: Add mediated vPMU PerfMon v5 support

[Zide Chen]

KVM currently caps the guest Intel architectural PMU version at 2. This
series bumps it to PerfMon v5 and adds support for some features that
are introduced in PerfMon v3-v5.

- CPUID.0AH:ECX fixed-counter bitmap (v5): supersedes EDX[4:0] and
  lets KVM/guests represent non-contiguous fixed counters (e.g. some
  E-core server parts skip fixed counter 3).
- Support IA32_PERF_GLOBAL_STATUS_RESET (v4): inherits MSR 0x390 and its
  emulation from IA32_PERF_GLOBAL_OVF_CTRL without KVM code changes;
  document its mediated-PMU passthrough erratum in errata.rst.
- Add support for IA32_PERF_GLOBAL_STATUS_SET (v4): lets software set
  individual bits in the global status MSR.
- Add support for IA32_PERF_GLOBAL_INUSE (v4): reports which counters
  and PMI are currently claimed.
- Implement ANYTHREAD_DEPRECATION capability.

Intel PerfMon v5 doesn't support non-contiguous GP counters; however,
this series lays the groundwork for GP counter bitmap support anyway,
for consistency with the host perf API interface, and prepares for
Perfmon Masking support.

Freeze_PerfMon_On_PMI virtualization is not implemented because, as
commit 3daa96d67274 ("perf/intel: Remove Perfmon-v4 counter_freezing
support") states, the Freeze-on-PMI mechanism violates the perf counter
independence.

Streamlined Freeze_LBRs_On_PMI is not implemented because LBR is not
yet supported on mediated vPMU.

Mutual exclusion between Intel PT and mediated vPMU, to prevent host PT
PMIs from asserting IA32_PERF_GLOBAL_STATUS.Trace_ToPA_PMI (bit 55)
while guest PMU context is loaded, as analyzed by Yosry Ahmed, is left
out of this series and will be addressed in a separate follow-up patch.

Tests are described in v2 cover letter.

Patch series summary:
patch 1-3: Code cleanups.
patch 4-9: Implement PMC bitmap support across perf and KVM.
patch 10-16: Implement PerfMon v4-v5 new features.
patch 17: Advertise PerfMon version up to v5 on Intel hosts.
patch 18: Support fixed counters bitmap in pmu_counters_test selftest.

Rebased on kvm-x86/next: commit dea4f6380f4e ("Merge branch 'vmx'")

v3 changes:
- Dropped v2 patch 4 ("Gate BUFFER_OVF reserved bit on guest DS") (Jim).
- New patch 10 to document the IA32_PERF_GLOBAL_STATUS_RESET passthrough
  erratum in errata.rst.
- New patch 15 to mask only OS/USR bits when filtering fixed counters,
  preventing AnyThread leakage into hardware and preserving METRICS_CLEAR_EN.
- Split MSR 0x391 and MSR 0x392 emulation into separate patches.
- Always intercept IA32_PERF_GLOBAL_STATUS_SET and GLOBAL_INUSE to
  prevent hardware state divergence (Jim).
- Reject bit 63 (CondChgd) writes to GLOBAL_STATUS_SET (Jim).
- Keep the AnyThread bit guest-visible while removing it from the
  hardware-visible states (Jim).
- Address other minor review comments from LKML, also some fixes
  suggested by AI tools.
- Collected Reviewed-by and Acked-by tags.

v2 changes:
- Dropped v1 11/15: KVM: x86/pmu: Emulate streamlined Freeze-LBR-on-PMI
- New patch 4/15 to gate BUFFER_OVF reserved bit on guest DS feature.
- Some renames:
  - all_valid_pmc_mask to pmc_exists;
  - add _is to kvm_{gp,fixed}_pmc_supported();
  - split kvm_for_each_set_pmc_idx() to kvm_for_each_{gp,fixed}_counter().
- Add missing AnyThr handling in IA32_FIXED_CTR_CTRL and advertise
  anythread_deprecated only if kvm_pmu_cap.version >= 5.
- Cap vPMU v3/v4 to v2, as AnyThread cannot be safely virtualized.
- Miscellaneous updates to changelogs and some minor fixes.

Previous versions:
v2: https://lore.kernel.org/kvm/20260827223755.143247-1-zide.chen@intel.com/
v1: https://lore.kernel.org/kvm/amj-5YH-LSwZOLow@google.com/T/#u

Dapeng Mi (6):
  KVM: x86/pmu: Drop nr_arch_{gp,fixed}_counters from kvm_pmu
  perf/x86: Plumb counter bitmap from x86_pmu to x86_pmu_cap
  KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities
  perf/x86: Remove num_counters_{gp,fixed} from x86_pmu_capability
  KVM: x86/pmu: Advertise PerfMon version 5 on Intel hosts
  KVM: selftests: Support fixed counters bitmap in pmu_counters_test

Zide Chen (12):
  KVM: x86/pmu: Remove redundant Perf Global Status MSR bit definitions
  KVM: x86/pmu: Rename all_valid_pmc_idx to pmc_exists
  KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu
  KVM: x86/pmu: Add PMC bitmap accessor helpers
  KVM: x86/pmu: Expose kvm_host_pmu to vendor modules
  KVM: x86/pmu: Document GLOBAL_STATUS_RESET passthrough erratum
  KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_STATUS_SET MSR
  KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_INUSE MSR
  KVM: x86/pmu: Populate CPUID.0AH:ECX fixed-counter bitmap
  KVM: x86/pmu: Factor out fixed counter control bit calculation
  KVM: x86/pmu: Mask only OS/USR bits when filtering fixed counters
  KVM: x86/pmu: Ignore AnyThread bit if CPUID.0AH:EDX[15] is set

 Documentation/virt/kvm/x86/errata.rst         |  10 +-
 arch/x86/events/core.c                        |   8 +-
 arch/x86/include/asm/kvm_host.h               |   9 +-
 arch/x86/include/asm/msr-index.h              |   9 +-
 arch/x86/include/asm/perf_event.h             |  13 +-
 arch/x86/kvm/cpuid.c                          |  21 +-
 arch/x86/kvm/msrs.c                           |  12 +-
 arch/x86/kvm/pmu.c                            |  87 +++++---
 arch/x86/kvm/pmu.h                            |  54 ++++-
 arch/x86/kvm/svm/pmu.c                        |  37 ++--
 arch/x86/kvm/svm/svm.c                        |  18 +-
 arch/x86/kvm/vmx/nested.c                     |   6 +-
 arch/x86/kvm/vmx/pmu_intel.c                  | 196 +++++++++++++-----
 arch/x86/kvm/vmx/vmx.c                        |  28 ++-
 .../selftests/kvm/include/x86/processor.h     |   6 +-
 .../selftests/kvm/x86/pmu_counters_test.c     |  28 ++-
 16 files changed, 379 insertions(+), 164 deletions(-)

-- 
2.56.0

[PATCH v3 01/18] KVM: x86/pmu: Remove redundant Perf Global Status MSR bit definitions

[Zide Chen]

IA32_PERF_GLOBAL_STATUS and its SET/RESET counterparts share bit
definitions with perf_event.h, which is the canonical home for Intel
PMU bit definitions (e.g., PERFEVTSEL, FIXED_CTR_CTRL, GLOBAL_STATUS).
Drop the duplicate definitions from msr-index.h and update KVM code to
use the macros from perf_event.h.

Drop the comment as well, since the macros now match the MSR
naming.

No functional change intended.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/include/asm/msr-index.h |  8 --------
 arch/x86/kvm/vmx/pmu_intel.c     | 11 ++---------
 arch/x86/kvm/vmx/vmx.c           |  2 +-
 3 files changed, 3 insertions(+), 18 deletions(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 3a8e51a0c9e8..30cd04c0ace9 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -1245,14 +1245,6 @@
 
 #define MSR_PERF_METRICS		0x00000329
 
-/* PERF_GLOBAL_OVF_CTL bits */
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT	55
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI		(1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT)
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT		62
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF			(1ULL <<  MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT)
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD_BIT		63
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD			(1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD_BIT)
-
 /* Geode defined MSRs */
 #define MSR_GEODE_BUSCONT_CONF0		0x00001900
 
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 70a8c4816135..36d91215f42a 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -579,17 +579,10 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 			 ((BIT_ULL(pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX));
 	pmu->global_ctrl_rsvd = counter_rsvd;
 
-	/*
-	 * GLOBAL_STATUS and GLOBAL_OVF_CONTROL (a.k.a. GLOBAL_STATUS_RESET)
-	 * share reserved bit definitions.  The kernel just happens to use
-	 * OVF_CTRL for the names.
-	 */
 	pmu->global_status_rsvd = pmu->global_ctrl_rsvd
-			& ~(MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF |
-			    MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD);
+			& ~(GLOBAL_STATUS_BUFFER_OVF | GLOBAL_STATUS_COND_CHG);
 	if (vmx_pt_mode_is_host_guest())
-		pmu->global_status_rsvd &=
-				~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI;
+		pmu->global_status_rsvd &= ~GLOBAL_STATUS_TRACE_TOPAPMI;
 
 	if (perf_capabilities & PERF_CAP_PEBS_FORMAT) {
 		if (perf_capabilities & PERF_CAP_PEBS_BASELINE) {
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index b341af4c6057..efde96e15533 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -8477,7 +8477,7 @@ static unsigned int vmx_handle_intel_pt_intr(void)
 		return 0;
 
 	kvm_make_request(KVM_REQ_PMI, vcpu);
-	__set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT,
+	__set_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT,
 		  (unsigned long *)&vcpu->arch.pmu.global_status);
 	return 1;
 }
-- 
2.56.0

[PATCH v3 02/18] KVM: x86/pmu: Rename all_valid_pmc_idx to pmc_exists

[Zide Chen]

all_valid_pmc_idx is a bitmap of valid PMC indices, not an index
itself; rename it to pmc_exists to better reflect what it holds and
keep the name consistent with other fields in the same struct.

No functional change intended.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
v2:
- Pick the name pmc_exists other than all_valid_pmc_mask. (Sean)
---
 arch/x86/include/asm/kvm_host.h |  2 +-
 arch/x86/kvm/pmu.c              | 10 +++++-----
 arch/x86/kvm/vmx/pmu_intel.c    |  2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index e2eef7057bd3..72b8fbf432a2 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -595,7 +595,7 @@ struct kvm_pmu {
 		DECLARE_BITMAP(reprogram_pmi, X86_PMC_IDX_MAX);
 		atomic64_t __reprogram_pmi;
 	};
-	DECLARE_BITMAP(all_valid_pmc_idx, X86_PMC_IDX_MAX);
+	DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
 	DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX);
 
 	DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX);
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index eef2f792dc9e..83883191d124 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -944,7 +944,7 @@ static void kvm_pmu_reset(struct kvm_vcpu *vcpu)
 	bitmap_zero(pmu->reprogram_pmi, X86_PMC_IDX_MAX);
 	bitmap_zero(pmu->pmc_has_mode_specific_enables, X86_PMC_IDX_MAX);
 
-	kvm_for_each_pmc(pmu, pmc, i, pmu->all_valid_pmc_idx) {
+	kvm_for_each_pmc(pmu, pmc, i, pmu->pmc_exists) {
 		pmc_stop_counter(pmc);
 		pmc->counter = 0;
 		pmc->emulated_counter = 0;
@@ -990,7 +990,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->fixed_ctr_ctrl_rsvd = ~0ull;
 	pmu->pebs_enable_rsvd = ~0ull;
 	pmu->pebs_data_cfg_rsvd = ~0ull;
-	bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
+	bitmap_zero(pmu->pmc_exists, X86_PMC_IDX_MAX);
 
 	if (!vcpu->kvm->arch.enable_pmu)
 		return;
@@ -1011,8 +1011,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
 
-	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
-	bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX,
+	bitmap_set(pmu->pmc_exists, 0, pmu->nr_arch_gp_counters);
+	bitmap_set(pmu->pmc_exists, KVM_FIXED_PMC_BASE_IDX,
 		   pmu->nr_arch_fixed_counters);
 }
 
@@ -1034,7 +1034,7 @@ void kvm_pmu_cleanup(struct kvm_vcpu *vcpu)
 
 	pmu->need_cleanup = false;
 
-	bitmap_andnot(bitmask, pmu->all_valid_pmc_idx,
+	bitmap_andnot(bitmask, pmu->pmc_exists,
 		      pmu->pmc_in_use, X86_PMC_IDX_MAX);
 
 	kvm_for_each_pmc(pmu, pmc, i, bitmask) {
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 36d91215f42a..ee18899614a1 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -560,7 +560,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		lbr_desc->records.nr = 0;
 
 	if (lbr_desc->records.nr)
-		bitmap_set(pmu->all_valid_pmc_idx, INTEL_PMC_IDX_FIXED_VLBR, 1);
+		bitmap_set(pmu->pmc_exists, INTEL_PMC_IDX_FIXED_VLBR, 1);
 
 	if (pmu->version == 1)
 		return;
-- 
2.56.0

[PATCH v3 03/18] KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu

[Zide Chen]

This field stores the bits that are reserved in guest IA32_PERFEVTSELx
MSRs. Rename it to eventsel_rsvd to better reflect its purpose and to
align with other field names in struct kvm_pmu.

Opportunistically, replace magic numbers with macros.

No functional change intended.

Suggested-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Sandipan Das <sandipan.das@amd.com>
---
v3:
- Mention "magic numbers" in changelog to cover AMD as well.

v2:
- Replace magic numbers in AMD pmu->eventsel_rsvd.
---
 arch/x86/include/asm/kvm_host.h |  2 +-
 arch/x86/kvm/pmu.c              |  6 +++++-
 arch/x86/kvm/svm/pmu.c          | 12 +++++++++---
 arch/x86/kvm/vmx/pmu_intel.c    | 12 ++++++------
 4 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 72b8fbf432a2..b8875ed5502c 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -581,7 +581,7 @@ struct kvm_pmu {
 	u64 counter_bitmask[2];
 	u64 global_ctrl_rsvd;
 	u64 global_status_rsvd;
-	u64 reserved_bits;
+	u64 eventsel_rsvd;
 	u64 raw_event_mask;
 	struct kvm_pmc gp_counters[KVM_MAX_NR_GP_COUNTERS];
 	struct kvm_pmc fixed_counters[KVM_MAX_NR_FIXED_COUNTERS];
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 83883191d124..cbd63329cf13 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -983,7 +983,11 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->nr_arch_fixed_counters = 0;
 	pmu->counter_bitmask[KVM_PMC_GP] = 0;
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
-	pmu->reserved_bits = 0xffffffff00200000ull;
+	/*
+	 * KVM is not able to emulate the AnyThread bit due to cross-VM
+	 * information leakage on SMT cores.
+	 */
+	pmu->eventsel_rsvd = GENMASK_ULL(63, 32) | ARCH_PERFMON_EVENTSEL_ANY;
 	pmu->raw_event_mask = X86_RAW_EVENT_MASK;
 	pmu->global_ctrl_rsvd = ~0ull;
 	pmu->global_status_rsvd = ~0ull;
diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index c18286545a7a..52224872e0d4 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -163,7 +163,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	/* MSR_EVNTSELn */
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_EVNTSEL);
 	if (pmc) {
-		data &= ~pmu->reserved_bits;
+		data &= ~pmu->eventsel_rsvd;
 		if (data != pmc->eventsel) {
 			pmc->eventsel = data;
 			pmc->eventsel_hw = (data & ~AMD64_EVENTSEL_HOSTONLY) |
@@ -214,9 +214,15 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 
 	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(48) - 1;
 
-	pmu->reserved_bits = 0xfffffff000280000ull;
+	/*
+	 * AMD PerfEvtSeln registers do not support PinControl, and KVM does
+	 * not emulate AnyThread due to cross-VM information leakage on SMT
+	 * cores (as on the Intel side).
+	 */
+	pmu->eventsel_rsvd = GENMASK_ULL(63, 36) | ARCH_PERFMON_EVENTSEL_PIN_CONTROL |
+			     ARCH_PERFMON_EVENTSEL_ANY;
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) && kvm_vcpu_has_mediated_pmu(vcpu))
-		pmu->reserved_bits &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
+		pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
 
 	pmu->raw_event_mask = AMD64_RAW_EVENT_MASK;
 	/* not applicable to AMD; but clean them to prevent any fall out */
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index ee18899614a1..fadc466cd600 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -388,7 +388,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	struct kvm_pmc *pmc;
 	u32 msr = msr_info->index;
 	u64 data = msr_info->data;
-	u64 reserved_bits, diff;
+	u64 eventsel_rsvd, diff;
 
 	switch (msr) {
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
@@ -436,11 +436,11 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			pmc_write_counter(pmc, data);
 			break;
 		} else if ((pmc = get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0))) {
-			reserved_bits = pmu->reserved_bits;
+			eventsel_rsvd = pmu->eventsel_rsvd;
 			if ((pmc->idx == 2) &&
 			    (pmu->raw_event_mask & HSW_IN_TX_CHECKPOINTED))
-				reserved_bits ^= HSW_IN_TX_CHECKPOINTED;
-			if (data & reserved_bits)
+				eventsel_rsvd ^= HSW_IN_TX_CHECKPOINTED;
+			if (data & eventsel_rsvd)
 				return 1;
 
 			if (data != pmc->eventsel) {
@@ -548,7 +548,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (entry &&
 	    (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) &&
 	    (entry->ebx & (X86_FEATURE_HLE|X86_FEATURE_RTM))) {
-		pmu->reserved_bits ^= HSW_IN_TX;
+		pmu->eventsel_rsvd ^= HSW_IN_TX;
 		pmu->raw_event_mask |= (HSW_IN_TX|HSW_IN_TX_CHECKPOINTED);
 	}
 
@@ -587,7 +587,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (perf_capabilities & PERF_CAP_PEBS_FORMAT) {
 		if (perf_capabilities & PERF_CAP_PEBS_BASELINE) {
 			pmu->pebs_enable_rsvd = counter_rsvd;
-			pmu->reserved_bits &= ~ICL_EVENTSEL_ADAPTIVE;
+			pmu->eventsel_rsvd &= ~ICL_EVENTSEL_ADAPTIVE;
 			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
 			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
 		} else {
-- 
2.56.0

[PATCH v3 04/18] KVM: x86/pmu: Add PMC bitmap accessor helpers

[Zide Chen]

pmu->nr_arch_{gp,fixed}_counters is not able to represent that a PMU
may include non-contiguous GP or fixed counters.

pmu->pmc_exists already holds a bitmap indicating both fixed and
general-purpose counters, and loops over valid counters can be done via
pmu->pmc_exists alone. Extend it to a union so that the u64 alias is
available for convenient mask arithmetic operations.

Add the necessary helpers to prepare for bitmap-based PMC counter
implementation.

No functional change intended.

Co-developed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
v3:
- Add BUILD_BUG_ON(). (Jim)
v2:
- Rename kvm_{gp,fixed}_pmc_supported() to kvm_is_{gp,fixed}_pmc_supported()
  and simplify them by removing the temporary bitmap. (Sean)
- Split kvm_for_each_set_pmc_idx() to kvm_for_each_gp_counter()
  and kvm_for_each_fixed_counter(). (Sean)
---
 arch/x86/include/asm/kvm_host.h |  5 +++-
 arch/x86/kvm/pmu.c              |  2 ++
 arch/x86/kvm/pmu.h              | 53 +++++++++++++++++++++++++++++----
 3 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b8875ed5502c..4f192429ed93 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -595,7 +595,10 @@ struct kvm_pmu {
 		DECLARE_BITMAP(reprogram_pmi, X86_PMC_IDX_MAX);
 		atomic64_t __reprogram_pmi;
 	};
-	DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
+	union {
+		DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
+		u64 pmc_exists64;
+	};
 	DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX);
 
 	DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX);
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index cbd63329cf13..5e174d95f3b7 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -1024,6 +1024,8 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
+	BUILD_BUG_ON(sizeof(pmu->pmc_exists) != sizeof(pmu->pmc_exists64));
+
 	memset(pmu, 0, sizeof(*pmu));
 	kvm_pmu_call(init)(vcpu);
 }
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 090c9bbb74f4..016935c3a8ec 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -88,6 +88,30 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline unsigned long kvm_gp_pmc_mask(struct kvm_pmu *pmu)
+{
+	return pmu->pmc_exists64 &
+	       GENMASK_ULL(KVM_MAX_NR_GP_COUNTERS - 1, 0);
+}
+
+static inline unsigned long kvm_fixed_pmc_mask(struct kvm_pmu *pmu)
+{
+	return (pmu->pmc_exists64 >> KVM_FIXED_PMC_BASE_IDX) &
+	       GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
+}
+
+static inline bool kvm_is_gp_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
+{
+	return idx < KVM_MAX_NR_GP_COUNTERS &&
+	       test_bit(idx, pmu->pmc_exists);
+}
+
+static inline bool kvm_is_fixed_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
+{
+	return idx < KVM_MAX_NR_FIXED_COUNTERS &&
+	       test_bit(KVM_FIXED_PMC_BASE_IDX + idx, pmu->pmc_exists);
+}
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
@@ -104,11 +128,11 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
  */
 static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
 {
-	if (idx < pmu->nr_arch_gp_counters)
+	if (kvm_is_gp_pmc_supported(pmu, idx))
 		return &pmu->gp_counters[idx];
 
 	idx -= KVM_FIXED_PMC_BASE_IDX;
-	if (idx >= 0 && idx < pmu->nr_arch_fixed_counters)
+	if (kvm_is_fixed_pmc_supported(pmu, idx))
 		return &pmu->fixed_counters[idx];
 
 	return NULL;
@@ -120,6 +144,17 @@ static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
 			continue;				\
 		else						\
 
+/*
+ * @mask is expected to be a scalar unsigned long derived from pmu->pmc_exists,
+ * which is already constrained by KVM_MAX_NR_{AMD,INTEL}_{GP,FIXED}_COUNTERS,
+ * so iteration up to KVM's maximum counter count is safe.
+ */
+#define kvm_for_each_gp_counter(i, mask)	\
+	for_each_set_bit((i), &(mask), KVM_MAX_NR_GP_COUNTERS)
+
+#define kvm_for_each_fixed_counter(i, mask)	\
+	for_each_set_bit((i), &(mask), KVM_MAX_NR_FIXED_COUNTERS)
+
 static inline u64 pmc_bitmask(struct kvm_pmc *pmc)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
@@ -168,9 +203,12 @@ static inline bool kvm_valid_perf_global_ctrl(struct kvm_pmu *pmu,
 static inline struct kvm_pmc *get_gp_pmc(struct kvm_pmu *pmu, u32 msr,
 					 u32 base)
 {
-	if (msr >= base && msr < base + pmu->nr_arch_gp_counters) {
+	if (msr >= base && msr < base + KVM_MAX_NR_GP_COUNTERS) {
 		u32 index = array_index_nospec(msr - base,
-					       pmu->nr_arch_gp_counters);
+					       KVM_MAX_NR_GP_COUNTERS);
+
+		if (!kvm_is_gp_pmc_supported(pmu, index))
+			return NULL;
 
 		return &pmu->gp_counters[index];
 	}
@@ -183,9 +221,12 @@ static inline struct kvm_pmc *get_fixed_pmc(struct kvm_pmu *pmu, u32 msr)
 {
 	int base = MSR_CORE_PERF_FIXED_CTR0;
 
-	if (msr >= base && msr < base + pmu->nr_arch_fixed_counters) {
+	if (msr >= base && msr < base + KVM_MAX_NR_FIXED_COUNTERS) {
 		u32 index = array_index_nospec(msr - base,
-					       pmu->nr_arch_fixed_counters);
+					       KVM_MAX_NR_FIXED_COUNTERS);
+
+		if (!kvm_is_fixed_pmc_supported(pmu, index))
+			return NULL;
 
 		return &pmu->fixed_counters[index];
 	}
-- 
2.56.0

[PATCH v3 05/18] KVM: x86/pmu: Drop nr_arch_{gp,fixed}_counters from kvm_pmu

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

Populate and use pmc_exists directly when walking guest-visible PMCs,
instead of relying on pmu->nr_arch_{gp,fixed}_counters as intermediate
state, in preparation for switching KVM's host PMU capabilities from
counter counts to counter bitmaps.

Iterate counters via the newly added pmc_exists based helpers, and
remove the now-redundant nr_arch_{gp,fixed}_counters fields from struct
kvm_pmu. Consolidate PMU MSR intercept setup in VMX and SVM into single
loops.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Consolidate PMU MSR intercept setup into single loops in VMX and SVM.
- Rename fixed_cntr_mask to fixed_mask in intel_pmu_refresh().
- Fix 32-bit zero-extension bug in pebs_enable_rsvd calculation
  by explicitly casting to u64. (Sashiko)

v2:
- Fix AMD legacy PMU MSR intercept handling for guest PMU
  narrowing reconfiguration by adding for_each_clear_bit_from().
- Re-word the stale interception changelog part.
---
 arch/x86/include/asm/kvm_host.h |  2 --
 arch/x86/kvm/pmu.c              | 26 +++++++++----------
 arch/x86/kvm/svm/pmu.c          | 25 ++++++++----------
 arch/x86/kvm/svm/svm.c          | 13 ++++------
 arch/x86/kvm/vmx/nested.c       |  6 +++--
 arch/x86/kvm/vmx/pmu_intel.c    | 46 ++++++++++++++++++++++-----------
 arch/x86/kvm/vmx/vmx.c          | 26 +++++++++----------
 7 files changed, 75 insertions(+), 69 deletions(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 4f192429ed93..8097d106d157 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -570,8 +570,6 @@ struct kvm_pmc {
 
 struct kvm_pmu {
 	u8 version;
-	unsigned nr_arch_gp_counters;
-	unsigned nr_arch_fixed_counters;
 	unsigned available_event_types;
 	u64 fixed_ctr_ctrl;
 	u64 fixed_ctr_ctrl_hw;
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 5e174d95f3b7..6cd67f044b6e 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -787,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return pmu->nr_arch_gp_counters != kvm_host_pmu.num_counters_gp ||
-	       pmu->nr_arch_fixed_counters != kvm_host_pmu.num_counters_fixed;
+	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
+	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
@@ -979,8 +979,6 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	kvm_pmu_reset(vcpu);
 
 	pmu->version = 0;
-	pmu->nr_arch_gp_counters = 0;
-	pmu->nr_arch_fixed_counters = 0;
 	pmu->counter_bitmask[KVM_PMC_GP] = 0;
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
 	/*
@@ -1008,16 +1006,12 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (pmu->nr_arch_gp_counters &&
+	if (kvm_gp_pmc_mask(pmu) &&
 	    (kvm_pmu_has_perf_global_ctrl(pmu) || kvm_vcpu_has_mediated_pmu(vcpu)))
-		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
+		pmu->global_ctrl = kvm_gp_pmc_mask(pmu);
 
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
-
-	bitmap_set(pmu->pmc_exists, 0, pmu->nr_arch_gp_counters);
-	bitmap_set(pmu->pmc_exists, KVM_FIXED_PMC_BASE_IDX,
-		   pmu->nr_arch_fixed_counters);
 }
 
 void kvm_pmu_init(struct kvm_vcpu *vcpu)
@@ -1328,6 +1322,8 @@ static __always_inline u32 gp_eventsel_msr(u32 idx)
 static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	struct kvm_pmc *pmc;
 	u32 i;
 
@@ -1336,14 +1332,14 @@ static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
 	 * is intercepted if hardware has counters that aren't visible to the
 	 * guest (KVM will inject #GP as appropriate).
 	 */
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, gp_mask) {
 		pmc = &pmu->gp_counters[i];
 
 		if (pmc->counter != rdpmc(i))
 			wrmsrq(gp_counter_msr(i), pmc->counter);
 		wrmsrq(gp_eventsel_msr(i), pmc->eventsel_hw);
 	}
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+	kvm_for_each_fixed_counter(i, fixed_mask) {
 		pmc = &pmu->fixed_counters[i];
 
 		if (pmc->counter != rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i))
@@ -1386,6 +1382,8 @@ void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu)
 static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	struct kvm_pmc *pmc;
 	u32 i;
 
@@ -1393,7 +1391,7 @@ static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu)
 	 * Clear selectors and counters to ensure hardware doesn't count using
 	 * guest controls when the host (perf) restores its state.
 	 */
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, gp_mask) {
 		pmc = &pmu->gp_counters[i];
 
 		pmc->counter = rdpmc(i);
@@ -1403,7 +1401,7 @@ static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu)
 			wrmsrq(gp_eventsel_msr(i), 0);
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+	kvm_for_each_fixed_counter(i, fixed_mask) {
 		pmc = &pmu->fixed_counters[i];
 
 		pmc->counter = rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i);
diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index 52224872e0d4..0943ccc1d6b8 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -27,12 +27,11 @@ enum pmu_type {
 
 static struct kvm_pmc *amd_pmu_get_pmc(struct kvm_pmu *pmu, int pmc_idx)
 {
-	unsigned int num_counters = pmu->nr_arch_gp_counters;
-
-	if (pmc_idx >= num_counters)
+	if (!kvm_is_gp_pmc_supported(pmu, pmc_idx))
 		return NULL;
 
-	return &pmu->gp_counters[array_index_nospec(pmc_idx, num_counters)];
+	pmc_idx = array_index_nospec(pmc_idx, KVM_MAX_NR_AMD_GP_COUNTERS);
+	return &pmu->gp_counters[pmc_idx];
 }
 
 static inline struct kvm_pmc *get_gp_pmc_amd(struct kvm_pmu *pmu, u32 msr,
@@ -77,7 +76,7 @@ static int amd_check_rdpmc_early(struct kvm_vcpu *vcpu, unsigned int idx)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
-	if (idx >= pmu->nr_arch_gp_counters)
+	if (!kvm_is_gp_pmc_supported(pmu, idx))
 		return -EINVAL;
 
 	return 0;
@@ -117,7 +116,7 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 		return pmu->version > 1;
 	default:
 		if (msr > MSR_F15H_PERF_CTR5 &&
-		    msr < MSR_F15H_PERF_CTL0 + 2 * pmu->nr_arch_gp_counters)
+		    msr < MSR_F15H_PERF_CTL0 + 2 * hweight_long(kvm_gp_pmc_mask(pmu)))
 			return pmu->version > 1;
 		break;
 	}
@@ -184,6 +183,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 
 static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 {
+	unsigned int nr_gp_counters = AMD64_NUM_COUNTERS;
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	union cpuid_0x80000022_ebx ebx;
 
@@ -197,18 +197,16 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 		BUILD_BUG_ON(x86_feature_cpuid(X86_FEATURE_PERFMON_V2).function != 0x80000022 ||
 			     x86_feature_cpuid(X86_FEATURE_PERFMON_V2).index);
 		ebx.full = kvm_find_cpuid_entry_index(vcpu, 0x80000022, 0)->ebx;
-		pmu->nr_arch_gp_counters = ebx.split.num_core_pmc;
+		nr_gp_counters = ebx.split.num_core_pmc;
 	} else if (guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {
-		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS_CORE;
-	} else {
-		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS;
+		nr_gp_counters = AMD64_NUM_COUNTERS_CORE;
 	}
 
-	pmu->nr_arch_gp_counters = min_t(unsigned int, pmu->nr_arch_gp_counters,
-					 kvm_pmu_cap.num_counters_gp);
+	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
+			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
 
 	if (pmu->version > 1) {
-		pmu->global_ctrl_rsvd = ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1);
+		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
 		pmu->global_status_rsvd = pmu->global_ctrl_rsvd;
 	}
 
@@ -227,7 +225,6 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->raw_event_mask = AMD64_RAW_EVENT_MASK;
 	/* not applicable to AMD; but clean them to prevent any fall out */
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
-	pmu->nr_arch_fixed_counters = 0;
 }
 
 static void amd_pmu_init(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 136069dfcd44..e21abcedea70 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -761,24 +761,21 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 {
 	bool intercept = !kvm_vcpu_has_mediated_pmu(vcpu);
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	int i;
 
 	if (!enable_mediated_pmu)
 		return;
 
 	/* Legacy counters are always available for AMD CPUs with a PMU. */
-	for (i = 0; i < min(pmu->nr_arch_gp_counters, AMD64_NUM_COUNTERS); i++)
+	for (i = 0; i < AMD64_NUM_COUNTERS; i++)
 		svm_set_intercept_for_msr(vcpu, MSR_K7_PERFCTR0 + i,
-					  MSR_TYPE_RW, intercept);
+					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++)
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
-					  MSR_TYPE_RW, intercept);
-
-	for ( ; i < kvm_pmu_cap.num_counters_gp; i++)
-		svm_enable_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
-					     MSR_TYPE_RW);
+					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept = kvm_need_perf_global_ctrl_intercept(vcpu);
 	svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 9b0bfa2f854c..6dade9d26d1a 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -704,6 +704,8 @@ static void nested_vmx_merge_pmu_msr_bitmaps(struct kvm_vcpu *vcpu,
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	int i;
 
 	/*
@@ -713,12 +715,12 @@ static void nested_vmx_merge_pmu_msr_bitmaps(struct kvm_vcpu *vcpu,
 	if (!kvm_vcpu_has_mediated_pmu(vcpu))
 		return;
 
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, gp_mask) {
 		nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PERFCTR0 + i);
 		nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PMC0 + i);
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
+	kvm_for_each_fixed_counter(i, fixed_mask)
 		nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_FIXED_CTR0 + i);
 
 	nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_GLOBAL_CTRL);
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index fadc466cd600..7770200d0e91 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -66,12 +66,13 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 	 * hardware, e.g. to ensure the event filter is enforced.
 	 */
 	u64 old_fixed_ctr_ctrl = pmu->fixed_ctr_ctrl_hw;
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
 	struct kvm_pmc *pmc;
 	int i;
 
 	pmu->fixed_ctr_ctrl = data;
 	pmu->fixed_ctr_ctrl_hw = data;
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+	kvm_for_each_fixed_counter(i, fixed_mask) {
 		u8 new_ctrl = fixed_ctrl_field(data, i);
 		u8 old_ctrl = fixed_ctrl_field(old_fixed_ctr_ctrl, i);
 
@@ -107,6 +108,8 @@ static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct kvm_vcpu *vcpu,
 	if (WARN_ON_ONCE(!pmu->version))
 		return NULL;
 
+	idx &= INTEL_RDPMC_INDEX_MASK;
+
 	/*
 	 * General Purpose (GP) PMCs are supported on all PMUs, and fixed PMCs
 	 * are supported on all architectural PMUs, i.e. on all virtual PMUs
@@ -117,23 +120,25 @@ static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct kvm_vcpu *vcpu,
 	 */
 	switch (type) {
 	case INTEL_RDPMC_FIXED:
+		if (!kvm_is_fixed_pmc_supported(pmu, idx))
+			return NULL;
+
 		counters = pmu->fixed_counters;
-		num_counters = pmu->nr_arch_fixed_counters;
+		num_counters = KVM_MAX_NR_INTEL_FIXED_COUNTERS;
 		bitmask = pmu->counter_bitmask[KVM_PMC_FIXED];
 		break;
 	case INTEL_RDPMC_GP:
+		if (!kvm_is_gp_pmc_supported(pmu, idx))
+			return NULL;
+
 		counters = pmu->gp_counters;
-		num_counters = pmu->nr_arch_gp_counters;
+		num_counters = KVM_MAX_NR_INTEL_GP_COUNTERS;
 		bitmask = pmu->counter_bitmask[KVM_PMC_GP];
 		break;
 	default:
 		return NULL;
 	}
 
-	idx &= INTEL_RDPMC_INDEX_MASK;
-	if (idx >= num_counters)
-		return NULL;
-
 	*mask &= bitmask;
 	return &counters[array_index_nospec(idx, num_counters)];
 }
@@ -495,9 +500,10 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 
 static void intel_pmu_enable_fixed_counter_bits(struct kvm_pmu *pmu, u64 bits)
 {
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
 	int i;
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
+	kvm_for_each_fixed_counter(i, fixed_mask)
 		pmu->fixed_ctr_ctrl_rsvd &= ~intel_fixed_bits_by_idx(i, bits);
 }
 
@@ -509,6 +515,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	union cpuid10_eax eax;
 	union cpuid10_edx edx;
 	u64 perf_capabilities;
+	u64 fixed_mask;
+	int nr_gp_counters;
 	u64 counter_rsvd;
 
 	if (!lbr_desc)
@@ -535,8 +543,6 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (!pmu->version)
 		return;
 
-	pmu->nr_arch_gp_counters = min_t(int, eax.split.num_counters,
-					 kvm_pmu_cap.num_counters_gp);
 	eax.split.bit_width = min_t(int, eax.split.bit_width,
 				    kvm_pmu_cap.bit_width_gp);
 	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(eax.split.bit_width) - 1;
@@ -544,6 +550,17 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				      kvm_pmu_cap.events_mask_len);
 	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
 
+	fixed_mask = BIT_ULL(edx.split.num_counters_fixed) - 1;
+	fixed_mask &= BIT_ULL(kvm_pmu_cap.num_counters_fixed) - 1;
+
+	/*
+	 * The number of counters comes from guest CPUID data. Clamp the value
+	 * to avoid a shift-by-64 in BIT_ULL().
+	 */
+	nr_gp_counters = min_t(int, eax.split.num_counters, X86_PMC_IDX_MAX - 1);
+	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
+			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+
 	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
 	if (entry &&
 	    (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) &&
@@ -565,8 +582,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (pmu->version == 1)
 		return;
 
-	pmu->nr_arch_fixed_counters = min_t(int, edx.split.num_counters_fixed,
-					    kvm_pmu_cap.num_counters_fixed);
+	pmu->pmc_exists64 |= fixed_mask << INTEL_PMC_IDX_FIXED;
 	edx.split.bit_width_fixed = min_t(int, edx.split.bit_width_fixed,
 					  kvm_pmu_cap.bit_width_fixed);
 	pmu->counter_bitmask[KVM_PMC_FIXED] = BIT_ULL(edx.split.bit_width_fixed) - 1;
@@ -575,8 +591,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 						 INTEL_FIXED_0_USER |
 						 INTEL_FIXED_0_ENABLE_PMI);
 
-	counter_rsvd = ~((BIT_ULL(pmu->nr_arch_gp_counters) - 1) |
-			 ((BIT_ULL(pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX));
+	counter_rsvd = ~((u64)kvm_gp_pmc_mask(pmu) |
+			 ((u64)kvm_fixed_pmc_mask(pmu) << KVM_FIXED_PMC_BASE_IDX));
 	pmu->global_ctrl_rsvd = counter_rsvd;
 
 	pmu->global_status_rsvd = pmu->global_ctrl_rsvd
@@ -591,7 +607,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
 			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
 		} else {
-			pmu->pebs_enable_rsvd = ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1);
+			pmu->pebs_enable_rsvd = ~(u64)kvm_gp_pmc_mask(pmu);
 		}
 	}
 }
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index efde96e15533..944ecbf351cf 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -4242,6 +4242,8 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 	bool has_mediated_pmu = kvm_vcpu_has_mediated_pmu(vcpu);
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	bool intercept = !has_mediated_pmu;
 	int i;
 
@@ -4262,25 +4264,21 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
+
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
-					  MSR_TYPE_RW, intercept);
+					  MSR_TYPE_RW, pmc_intercept);
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW,
-					  intercept || !fw_writes_is_enabled(vcpu));
-	}
-	for ( ; i < kvm_pmu_cap.num_counters_gp; i++) {
-		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
-					  MSR_TYPE_RW, true);
-		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i,
-					  MSR_TYPE_RW, true);
+					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
-		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
-					  MSR_TYPE_RW, intercept);
-	for ( ; i < kvm_pmu_cap.num_counters_fixed; i++)
+	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
+
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
-					  MSR_TYPE_RW, true);
+					  MSR_TYPE_RW, pmc_intercept);
+	}
 
 	intercept = kvm_need_perf_global_ctrl_intercept(vcpu);
 	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_STATUS,
-- 
2.56.0

[PATCH v3 06/18] KVM: x86/pmu: Expose kvm_host_pmu to vendor modules

[Zide Chen]

kvm_host_pmu holds the unadulterated host PMU capabilities, which
can be used to compare against guest capabilities to determine whether
certain MSRs should be intercepted by KVM.

Exposing it directly avoids the need to introduce per-field accessors,
which would add boilerplate without hiding any implementation detail
worth encapsulating.

Opportunistically, fix typo "Unadultered" in the comment.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/pmu.c | 5 +++--
 arch/x86/kvm/pmu.h | 1 +
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 6cd67f044b6e..b9c493c6db48 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -27,8 +27,9 @@
 /* This is enough to filter the vast majority of currently defined events. */
 #define KVM_PMU_EVENT_FILTER_MAX_EVENTS 300
 
-/* Unadultered PMU capabilities of the host, i.e. of hardware. */
-static struct x86_pmu_capability __read_mostly kvm_host_pmu;
+/* Unadulterated PMU capabilities of the host, i.e. of hardware. */
+struct x86_pmu_capability __read_mostly kvm_host_pmu;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_host_pmu);
 
 /* KVM's PMU capabilities, i.e. the intersection of KVM and hardware support. */
 struct x86_pmu_capability __read_mostly kvm_pmu_cap;
diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
index 016935c3a8ec..98e6f2443b82 100644
--- a/arch/x86/kvm/pmu.h
+++ b/arch/x86/kvm/pmu.h
@@ -252,6 +252,7 @@ static inline bool pmc_is_locally_enabled(struct kvm_pmc *pmc)
 	return !kvm_pmu_call(pmc_is_disabled_in_current_mode)(pmc);
 }
 
+extern struct x86_pmu_capability kvm_host_pmu;
 extern struct x86_pmu_capability kvm_pmu_cap;
 
 void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops);
-- 
2.56.0

[PATCH v3 07/18] perf/x86: Plumb counter bitmap from x86_pmu to x86_pmu_cap

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

Intel PerfMon v5 introduced CPUID.0AH:ECX to support non-contiguous
fixed counters and Architectural PerfMon Extension leaf (0x23) further
supports non-contiguous general-purpose counters.

num_counters_{gp,fixed} indicates the total number of GP or fixed
counters, but cannot represent non-contiguous counters.

Add cntr_mask and fixed_cntr_mask unions so that KVM can get the
accurate counter availability directly from x86_pmu_cap. The u64
alias is convenient for mask arithmetic, while the bitmap form works
with for_each_set_bit() and friends.

num_counters_{gp,fixed} in x86_pmu_capability will be removed once
callers have been converted to the use of {,fixed_}cntr_mask.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/events/core.c            | 6 ++++--
 arch/x86/include/asm/perf_event.h | 8 ++++++++
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index 8b3ea0adb965..a9b86cecbc78 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -3158,8 +3158,10 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	cap->version		= x86_pmu.version;
 	cap->num_counters_gp	= x86_pmu_num_counters(NULL);
 	cap->num_counters_fixed	= x86_pmu_num_counters_fixed(NULL);
-	cap->bit_width_gp	= cap->num_counters_gp ? x86_pmu.cntval_bits : 0;
-	cap->bit_width_fixed	= cap->num_counters_fixed ? x86_pmu.cntval_bits : 0;
+	cap->cntr_mask64	= x86_pmu.cntr_mask64;
+	cap->fixed_cntr_mask64	= x86_pmu.fixed_cntr_mask64;
+	cap->bit_width_gp	= cap->cntr_mask64 ? x86_pmu.cntval_bits : 0;
+	cap->bit_width_fixed	= cap->fixed_cntr_mask64 ? x86_pmu.cntval_bits : 0;
 	cap->events_mask	= (unsigned int)x86_pmu.events_maskl;
 	cap->events_mask_len	= x86_pmu.events_mask_len;
 	cap->pebs_ept		= x86_pmu.pebs_ept;
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 1eb13673e889..190fda412a61 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -302,6 +302,14 @@ struct x86_pmu_capability {
 	int		version;
 	int		num_counters_gp;
 	int		num_counters_fixed;
+	union {
+		u64	cntr_mask64;
+		DECLARE_BITMAP(cntr_mask, X86_PMC_IDX_MAX);
+	};
+	union {
+		u64	fixed_cntr_mask64;
+		DECLARE_BITMAP(fixed_cntr_mask, X86_PMC_IDX_MAX);
+	};
 	int		bit_width_gp;
 	int		bit_width_fixed;
 	unsigned int	events_mask;
-- 
2.56.0

[PATCH v3 08/18] KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

Intel platforms support non-contiguous fixed counters via CPUID.0AH:ECX
starting with PerfMon v5, and support non-contiguous GP counters
through the Architectural PerfMon Extension (CPUID leaf 23H).

struct x86_pmu_capability now exposes {,fixed_}cntr_mask64 bitmaps,
which may contain sparse bits representing non-contiguous counters.
Switch KVM's kvm_host_pmu and kvm_pmu_cap consumers over to the new
bitmask fields, including iterating kvm_host_pmu bitmasks directly in
VMX and SVM PMU MSR intercept setup.

CPUID.0AH:EAX[15:8] and CPUID.0AH:EDX[4:0] enumerate only contiguous
counters. Derive these values from kvm_pmu_cap.{,fixed_}cntr_mask64 as
the number of consecutive counters starting at index 0.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
---
v3:
- Iterate kvm_host_pmu bitmasks directly in VMX and SVM intercept loops.
- Clamp nr_{gp,fixed}_counters to the number of contiguous counters
  supported by KVM in intel_pmu_refresh(). (Jim, Sashiko)
- Drop redundant !enable_mediated_pmu clamp on fixed_cntr_mask64. (Sashiko)
---
 arch/x86/kvm/cpuid.c         | 14 +++++++++++---
 arch/x86/kvm/msrs.c          | 12 ++++++------
 arch/x86/kvm/pmu.c           | 17 ++++++++---------
 arch/x86/kvm/svm/pmu.c       |  2 +-
 arch/x86/kvm/svm/svm.c       |  7 +++----
 arch/x86/kvm/vmx/pmu_intel.c | 28 +++++++++++++++++++---------
 arch/x86/kvm/vmx/vmx.c       |  4 ++--
 7 files changed, 50 insertions(+), 34 deletions(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index adafd6fa2c54..3310c370c5fa 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1535,10 +1535,18 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 		}
 
 		eax.split.version_id = kvm_pmu_cap.version;
-		eax.split.num_counters = kvm_pmu_cap.num_counters_gp;
+
+		/* Contiguous GP counters only. */
+		eax.split.num_counters =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		eax.split.bit_width = kvm_pmu_cap.bit_width_gp;
 		eax.split.mask_length = kvm_pmu_cap.events_mask_len;
-		edx.split.num_counters_fixed = kvm_pmu_cap.num_counters_fixed;
+
+		/* Contiguous fixed counters only. */
+		edx.split.num_counters_fixed =
+			find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
 		if (kvm_pmu_cap.version)
@@ -1903,7 +1911,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		cpuid_entry_override(entry, CPUID_8000_0022_EAX);
 
-		ebx.split.num_core_pmc = kvm_pmu_cap.num_counters_gp;
+		ebx.split.num_core_pmc = hweight64(kvm_pmu_cap.cntr_mask64);
 		entry->ebx = ebx.full;
 		break;
 	}
diff --git a/arch/x86/kvm/msrs.c b/arch/x86/kvm/msrs.c
index e91650af44c2..01fb130a7d24 100644
--- a/arch/x86/kvm/msrs.c
+++ b/arch/x86/kvm/msrs.c
@@ -2621,20 +2621,20 @@ static void kvm_probe_msr_to_save(u32 msr_index)
 		break;
 	case MSR_ARCH_PERFMON_PERFCTR0 ...
 	     MSR_ARCH_PERFMON_PERFCTR0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_PERFCTR0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_PERFCTR0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_EVENTSEL0 ...
 	     MSR_ARCH_PERFMON_EVENTSEL0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_EVENTSEL0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_EVENTSEL0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_FIXED_CTR0 ...
 	     MSR_ARCH_PERFMON_FIXED_CTR0 + KVM_MAX_NR_FIXED_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_FIXED_CTR0 >=
-		    kvm_pmu_cap.num_counters_fixed)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_FIXED_CTR0) &
+		      kvm_pmu_cap.fixed_cntr_mask64))
 			return;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index b9c493c6db48..27d6a9169821 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -20,7 +20,6 @@
 #include <asm/perf_event.h>
 #include <asm/cpu_device_id.h>
 #include "x86.h"
-#include "cpuid.h"
 #include "lapic.h"
 #include "pmu.h"
 
@@ -156,8 +155,8 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 		 * there are a non-zero number of counters, but fewer than what
 		 * is architecturally required.
 		 */
-		if (!kvm_host_pmu.num_counters_gp ||
-		    WARN_ON_ONCE(kvm_host_pmu.num_counters_gp < min_nr_gp_ctrs))
+		if (!kvm_host_pmu.cntr_mask64 ||
+		    WARN_ON_ONCE(hweight64(kvm_host_pmu.cntr_mask64) < min_nr_gp_ctrs))
 			enable_pmu = false;
 		else if (is_intel && !kvm_host_pmu.version)
 			enable_pmu = false;
@@ -177,10 +176,10 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
 	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
-	kvm_pmu_cap.num_counters_gp = min(kvm_pmu_cap.num_counters_gp,
-					  pmu_ops->MAX_NR_GP_COUNTERS);
-	kvm_pmu_cap.num_counters_fixed = min(kvm_pmu_cap.num_counters_fixed,
-					     KVM_MAX_NR_FIXED_COUNTERS);
+	kvm_pmu_cap.cntr_mask64 &=
+		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
+	kvm_pmu_cap.fixed_cntr_mask64 &=
+		GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
 
 	kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =
 		perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS);
@@ -788,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
-	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
+	return kvm_gp_pmc_mask(pmu) != kvm_host_pmu.cntr_mask64 ||
+	       kvm_fixed_pmc_mask(pmu) != kvm_host_pmu.fixed_cntr_mask64;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index 0943ccc1d6b8..fb5a298244ec 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -203,7 +203,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
-			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+			    kvm_pmu_cap.cntr_mask64;
 
 	if (pmu->version > 1) {
 		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index e21abcedea70..c9ed6a6e8325 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -773,7 +773,7 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, AMD64_NUM_COUNTERS_CORE)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
@@ -5719,9 +5719,8 @@ static __init void svm_set_cpu_caps(void)
 		 * access to enough counters to virtualize "core" support,
 		 * otherwise limit vPMU support to the legacy number of counters.
 		 */
-		if (kvm_pmu_cap.num_counters_gp < AMD64_NUM_COUNTERS_CORE)
-			kvm_pmu_cap.num_counters_gp = min(AMD64_NUM_COUNTERS,
-							  kvm_pmu_cap.num_counters_gp);
+		if (hweight64(kvm_pmu_cap.cntr_mask64) < AMD64_NUM_COUNTERS_CORE)
+			kvm_pmu_cap.cntr_mask64 &= GENMASK_ULL(AMD64_NUM_COUNTERS - 1, 0);
 		else
 			kvm_cpu_cap_check_and_set(X86_FEATURE_PERFCTR_CORE);
 
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 7770200d0e91..ce668ea08da5 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -494,7 +494,8 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	 * have a known encoding for the associated general purpose event.
 	 */
 	eventsel = perf_get_hw_event_config(fixed_pmc_perf_ids[index]);
-	WARN_ON_ONCE(!eventsel && index < kvm_pmu_cap.num_counters_fixed);
+	WARN_ON_ONCE(!eventsel &&
+		     (kvm_pmu_cap.fixed_cntr_mask64 & BIT_ULL(index)));
 	return eventsel;
 }
 
@@ -516,7 +517,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	union cpuid10_edx edx;
 	u64 perf_capabilities;
 	u64 fixed_mask;
-	int nr_gp_counters;
+	int nr_gp_counters, nr_fixed_counters;
 	u64 counter_rsvd;
 
 	if (!lbr_desc)
@@ -550,16 +551,25 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				      kvm_pmu_cap.events_mask_len);
 	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
 
-	fixed_mask = BIT_ULL(edx.split.num_counters_fixed) - 1;
-	fixed_mask &= BIT_ULL(kvm_pmu_cap.num_counters_fixed) - 1;
+	/*
+	 * CPUID.0AH:EDX[4:0] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous fixed counters supported by KVM.
+	 */
+	nr_fixed_counters = min_t(int, edx.split.num_counters_fixed,
+				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+						      KVM_MAX_NR_FIXED_COUNTERS));
+
+	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
+	fixed_mask &= kvm_pmu_cap.fixed_cntr_mask64;
 
 	/*
-	 * The number of counters comes from guest CPUID data. Clamp the value
-	 * to avoid a shift-by-64 in BIT_ULL().
+	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous GP counters supported by KVM.
 	 */
-	nr_gp_counters = min_t(int, eax.split.num_counters, X86_PMC_IDX_MAX - 1);
-	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
-			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+	nr_gp_counters = min_t(int, eax.split.num_counters,
+			       find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+						   KVM_MAX_NR_GP_COUNTERS));
+	pmu->pmc_exists64 = BIT_ULL(nr_gp_counters) - 1;
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
 	if (entry &&
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 944ecbf351cf..9db6107943ec 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -4264,7 +4264,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, INTEL_PMC_MAX_GENERIC) {
 		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
@@ -4273,7 +4273,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+	for_each_set_bit(i, kvm_host_pmu.fixed_cntr_mask, INTEL_PMC_MAX_FIXED) {
 		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
-- 
2.56.0

[PATCH v3 09/18] perf/x86: Remove num_counters_{gp,fixed} from x86_pmu_capability

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

Now that KVM has switched to bitmap-based PMU capabilities,
num_counters_{gp,fixed} can be removed from x86_pmu_capability.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/events/core.c            | 2 --
 arch/x86/include/asm/perf_event.h | 2 --
 2 files changed, 4 deletions(-)

diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
index a9b86cecbc78..990d3be5a07f 100644
--- a/arch/x86/events/core.c
+++ b/arch/x86/events/core.c
@@ -3156,8 +3156,6 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	 * base PMU holds the correct number of counters for P-cores.
 	 */
 	cap->version		= x86_pmu.version;
-	cap->num_counters_gp	= x86_pmu_num_counters(NULL);
-	cap->num_counters_fixed	= x86_pmu_num_counters_fixed(NULL);
 	cap->cntr_mask64	= x86_pmu.cntr_mask64;
 	cap->fixed_cntr_mask64	= x86_pmu.fixed_cntr_mask64;
 	cap->bit_width_gp	= cap->cntr_mask64 ? x86_pmu.cntval_bits : 0;
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 190fda412a61..17b0bc7dfce7 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -300,8 +300,6 @@ union cpuid_0x80000022_ebx {
 
 struct x86_pmu_capability {
 	int		version;
-	int		num_counters_gp;
-	int		num_counters_fixed;
 	union {
 		u64	cntr_mask64;
 		DECLARE_BITMAP(cntr_mask, X86_PMC_IDX_MAX);
-- 
2.56.0

[PATCH v3 10/18] KVM: x86/pmu: Document GLOBAL_STATUS_RESET passthrough erratum

[Zide Chen]

IA32_PERF_GLOBAL_STATUS_RESET, introduced in PerfMon version 4,
inherits the MSR address and architecturally defined bits from
IA32_PERF_GLOBAL_OVF_CTRL, while providing additional bits to clear
new indicators in IA32_PERF_GLOBAL_STATUS.

From an emulation perspective, KVM handles it identically to the
OVF_CTRL MSR: reads return zero, and intercepted writes are guarded
against pmu->global_status_rsvd. No code changes are needed in KVM
to emulate this MSR.

However, to reduce VM exits, KVM passes through this MSR in mediated
PMU mode when GLOBAL_STATUS is also passed through. Consequently,
writes to bits that are valid on the host but unvirtualized by KVM
will not trigger a #GP (unlike when the MSR is intercepted). Except
for the known limitation where host Intel PT in system mode can
assert Trace_ToPA_PMI (bit 55), unvirtualized status bits are not set
while the guest is running, so clearing them has no negative effects.

Document this divergence in errata.rst.

Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- New patch prompted by Jim and Yosry's review on unvirtualized
  status bits.
---
 Documentation/virt/kvm/x86/errata.rst | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/Documentation/virt/kvm/x86/errata.rst b/Documentation/virt/kvm/x86/errata.rst
index a9cf0e004651..161ba6d1beb4 100644
--- a/Documentation/virt/kvm/x86/errata.rst
+++ b/Documentation/virt/kvm/x86/errata.rst
@@ -84,4 +84,12 @@ memory.
 
 Note, this erratum does not affect AMD CPUs, which fully virtualize CR0.CD in
 hardware, i.e. put the CPU caches into "no fill" mode when CR0.CD=1, even when
-running in the guest.
\ No newline at end of file
+running in the guest.
+
+IA32_PERF_GLOBAL_STATUS_RESET
+-----------------------------
+To reduce VM-exits on performance-critical paths, KVM passes through writes
+to IA32_PERF_GLOBAL_STATUS_RESET (0x390) in mediated PMU mode when the guest's
+counters match hardware, without injecting #GP on unvirtualized bits.  Because
+the MSR is read-as-zero and write-to-clear, this cannot leak host information
+to the guest.
-- 
2.56.0

[PATCH v3 11/18] KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_STATUS_SET MSR

[Zide Chen]

Intel PerfMon v4 introduces IA32_PERF_GLOBAL_STATUS_SET (0x391) to
allow software to set individual bits in the global status MSR. Reads
of IA32_PERF_GLOBAL_STATUS_SET always return zero.

IA32_PERF_GLOBAL_STATUS_SET is not part of the VM state, so it doesn't
need to be advertised to userspace, nor saved and restored during live
migration.

Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent the guest from
modifying status bits that are not virtualized by KVM.

Bit 63 (CondChgd) stays unreserved in pmu->global_status_rsvd to
architecturally allow the guest to write this bit on STATUS_RESET (0x390)
to clear it, and allow userspace to set it on GLOBAL_STATUS (0x38E).

However, bit 63 is architecturally reserved in STATUS_SET per the SDM.
Reject writes to bit 63, injecting #GP for guest writes. While older
processors had silicon errata that accepted writes without #GP (e.g.
SKX95), strictly enforce the architectural specification for
deterministic behavior across migrations.

Platform-specific EDS clarifies that CondChgd indicates a change to
IA32_MISC_ENABLE[7], which KVM intercepts and keeps strictly read-only
for guests. This guarantees hardware will never assert CondChgd during
guest execution.

Thus, the architectural asymmetry among these three MSRs is safe; being
unable to restore bit 63 to hardware via STATUS_SET is benign because
the guest will never legitimately observe CondChgd. Filter it out
defensively before writing to hardware in intel_mediated_pmu_load().

Originally-by: Yang Weijiang <weijiang.yang@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Split from combined v2 patch into standalone GLOBAL_STATUS_SET patch.
- Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent passthrough of
  unvirtualized status bits. (Jim)
- Reject writing bit 63 (CondChgd) to IA32_PERF_GLOBAL_STATUS_SET. (Jim)
- Filter out bit 63 before writing to hardware in intel_mediated_pmu_load().
- Move validation for MSR_CORE_PERF_GLOBAL_STATUS_SET to pmu_intel.c rather
  than common pmu.c to avoid exposing the Intel MSR on AMD.

v2:
- Add the missing vmcs02 updates for these two MSRs.
- Change "> 3" to ">= 4" to make the "v4-gated" more obvious and match
  the existing code style.
---
 arch/x86/kvm/pmu.c           | 13 +++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 14 ++++++++++++--
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 27d6a9169821..2cb5bccc7044 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -855,6 +855,7 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
 		msr_info->data = 0;
 		break;
 	default:
@@ -921,6 +922,18 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (!msr_info->host_initiated)
 			pmu->global_status &= ~data;
 		break;
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+		/*
+		 * Bit 63 (CondChgd) is architecturally reserved in STATUS_SET.
+		 * Older CPUs errantly accepted writes without #GP (e.g. SKX95),
+		 * but KVM strictly enforces the architectural spec to ensure
+		 * consistent behavior across migrations.
+		 */
+		if (data & (pmu->global_status_rsvd | GLOBAL_STATUS_COND_CHG))
+			return 1;
+		if (!msr_info->host_initiated)
+			pmu->global_status |= data;
+		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 		if (!msr_info->host_initiated)
 			pmu->global_status |= data & ~pmu->global_status_rsvd;
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index ce668ea08da5..e81a3f8b9b02 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -197,6 +197,8 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		return kvm_pmu_has_perf_global_ctrl(pmu);
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+		return pmu->version >= 4;
 	case MSR_IA32_PEBS_ENABLE:
 		ret = vcpu_get_perf_capabilities(vcpu) & PERF_CAP_PEBS_FORMAT;
 		break;
@@ -827,8 +829,16 @@ static void intel_mediated_pmu_load(struct kvm_vcpu *vcpu)
 	toggle = pmu->global_status ^ global_status;
 	if (global_status & toggle)
 		wrmsrq(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
-	if (pmu->global_status & toggle)
-		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
+
+	/*
+	 * CondChgd is architecturally reserved in GLOBAL_STATUS_SET. Unless
+	 * set by malicious userspace, this bit should never be set in
+	 * pmu->global_status; filter it out defensively to protect against
+	 * a host #GP.
+	 */
+	if ((pmu->global_status & toggle) & ~GLOBAL_STATUS_COND_CHG)
+		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET,
+		       (pmu->global_status & toggle) & ~GLOBAL_STATUS_COND_CHG);
 
 	wrmsrq(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
-- 
2.56.0

[PATCH v3 12/18] KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_INUSE MSR

[Zide Chen]

Intel PerfMon v4 introduces IA32_PERF_GLOBAL_INUSE (0x392) to track
which counters and the PMI are currently claimed by software agents,
allowing independent software agents to check counter availability
without a shared scheduler arbitrating between them.

IA32_PERF_GLOBAL_INUSE is a read-only MSR, and any write attempt
results in a #GP. The MSR is not part of the VM state, so it does
not need to be advertised to userspace, nor saved and restored during
live migration.

Always intercept IA32_PERF_GLOBAL_INUSE to prevent the guest from
erroneously observing filtered fixed counters as idle.

Originally-by: Yang Weijiang <weijiang.yang@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Split from combined v2 patch into standalone GLOBAL_INUSE patch.
- Always intercept MSR_CORE_PERF_GLOBAL_INUSE. (Jim)
- Use '!!' for explicit boolean conversion. (Jim)
- Relocate PERF_GLOBAL_INUSE_PMI_INUSE to perf_event.h.

v2:
- Change intel_pmu_get_global_inuse() to return u64, to match the
  surrounding code style.
- Add the missing vmcs02 updates for these two MSRs.
- Change "> 3" to ">= 4" to make the "v4-gated" more obvious and match
  the existing code style.
---
 arch/x86/include/asm/msr-index.h  |  1 +
 arch/x86/include/asm/perf_event.h |  3 +++
 arch/x86/kvm/vmx/pmu_intel.c      | 36 +++++++++++++++++++++++++++++++
 3 files changed, 40 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 30cd04c0ace9..25dda50fbdc1 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -1242,6 +1242,7 @@
 #define MSR_CORE_PERF_GLOBAL_CTRL	0x0000038f
 #define MSR_CORE_PERF_GLOBAL_OVF_CTRL	0x00000390
 #define MSR_CORE_PERF_GLOBAL_STATUS_SET	0x00000391
+#define MSR_CORE_PERF_GLOBAL_INUSE	0x00000392
 
 #define MSR_PERF_METRICS		0x00000329
 
diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h
index 17b0bc7dfce7..2937e8a27891 100644
--- a/arch/x86/include/asm/perf_event.h
+++ b/arch/x86/include/asm/perf_event.h
@@ -454,6 +454,9 @@ static inline bool is_topdown_idx(int idx)
 #define GLOBAL_STATUS_ARCH_PEBS_THRESHOLD	BIT_ULL(GLOBAL_STATUS_ARCH_PEBS_THRESHOLD_BIT)
 #define GLOBAL_STATUS_PERF_METRICS_OVF_BIT	48
 
+/* Intel IA32_PERF_GLOBAL_INUSE MSR */
+#define PERF_GLOBAL_INUSE_PMI_INUSE	BIT_ULL(63)
+
 #define GLOBAL_CTRL_EN_PERF_METRICS		BIT_ULL(48)
 /*
  * We model guest LBR event tracing as another fixed-mode PMC like BTS.
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index e81a3f8b9b02..1372091dc1f8 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -198,6 +198,7 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		return kvm_pmu_has_perf_global_ctrl(pmu);
 	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+	case MSR_CORE_PERF_GLOBAL_INUSE:
 		return pmu->version >= 4;
 	case MSR_IA32_PEBS_ENABLE:
 		ret = vcpu_get_perf_capabilities(vcpu) & PERF_CAP_PEBS_FORMAT;
@@ -346,6 +347,38 @@ static bool intel_pmu_handle_lbr_msrs_access(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static u64 intel_pmu_get_global_inuse(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
+	bool pmi_inuse = false;
+	u64 eventsel, data = 0;
+	u32 fixed_ctrl;
+	int i;
+
+	kvm_for_each_gp_counter(i, gp_mask) {
+		eventsel = pmu->gp_counters[i].eventsel;
+
+		if (eventsel & ARCH_PERFMON_EVENTSEL_EVENT)
+			data |= BIT_ULL(i);
+		pmi_inuse |= !!(eventsel & ARCH_PERFMON_EVENTSEL_INT);
+	}
+	kvm_for_each_fixed_counter(i, fixed_mask) {
+		fixed_ctrl = fixed_ctrl_field(pmu->fixed_ctr_ctrl, i);
+
+		if (fixed_ctrl & (INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER))
+			data |= BIT_ULL(KVM_FIXED_PMC_BASE_IDX + i);
+		pmi_inuse |= !!(fixed_ctrl & INTEL_FIXED_0_ENABLE_PMI);
+	}
+	pmi_inuse |= pmu->pebs_enable;
+
+	if (pmi_inuse)
+		data |= PERF_GLOBAL_INUSE_PMI_INUSE;
+
+	return data;
+}
+
 static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
@@ -356,6 +389,9 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		msr_info->data = pmu->fixed_ctr_ctrl;
 		break;
+	case MSR_CORE_PERF_GLOBAL_INUSE:
+		msr_info->data = intel_pmu_get_global_inuse(vcpu);
+		break;
 	case MSR_IA32_PEBS_ENABLE:
 		msr_info->data = pmu->pebs_enable;
 		break;
-- 
2.56.0

[PATCH v3 13/18] KVM: x86/pmu: Populate CPUID.0AH:ECX fixed-counter bitmap

[Zide Chen]

PerfMon v5 adds a fixed-counter support bitmap in CPUID.0AH:ECX.
This is the superset of EDX[4:0] which indicates the contiguous
counters starting from 0. For backward compatibility, it's recommended
to use the following to determine if a fixed counter is supported:

FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i);

Originally-by: Yang Weijiang <weijiang.yang@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Remove '&' when populating entry->ecx. (Jim)
- Gate entry->ecx on kvm_pmu_cap.version >= 5 in intel_pmu_refresh() to
  avoid enabling v5 fixed counters on hosts where KVM caps PMU at v2.
---
 arch/x86/kvm/cpuid.c         | 5 ++++-
 arch/x86/kvm/vmx/pmu_intel.c | 8 ++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 3310c370c5fa..00a2ca6d3049 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1554,7 +1554,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		entry->eax = eax.full;
 		entry->ebx = kvm_pmu_cap.events_mask;
-		entry->ecx = 0;
+		if (kvm_pmu_cap.version >= 5)
+			entry->ecx = (u32)kvm_pmu_cap.fixed_cntr_mask64;
+		else
+			entry->ecx = 0;
 		entry->edx = edx.full;
 		break;
 	}
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 1372091dc1f8..37b34b588601 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -597,7 +597,15 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
 						      KVM_MAX_NR_FIXED_COUNTERS));
 
+	/* FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i) */
 	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
+	/*
+	 * Do not enable fixed counters via the v5 ECX bitmap if KVM caps
+	 * PMU version below 5 (e.g. when mediated PMU is disabled or on
+	 * older hosts), matching KVM's advertised CPUID.0AH:ECX capability.
+	 */
+	if (kvm_pmu_cap.version >= 5 && pmu->version >= 5)
+		fixed_mask |= entry->ecx;
 	fixed_mask &= kvm_pmu_cap.fixed_cntr_mask64;
 
 	/*
-- 
2.56.0

[PATCH v3 14/18] KVM: x86/pmu: Factor out fixed counter control bit calculation

[Zide Chen]

Rename intel_pmu_enable_fixed_counter_bits() to
intel_pmu_fixed_ctrl_bits() and have it return the computed fixed
counter control mask instead of updating fixed_ctr_ctrl_rsvd directly.

Accumulate the desired fixed counter control bits during PMU setup and
apply them once in intel_pmu_refresh().

Move the helper earlier in the file so it can also be reused for
upcoming ANYTHREAD_DEPRECATION emulation.

No functional change intended.

Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
---
v2:
- New patch.
---
 arch/x86/kvm/vmx/pmu_intel.c | 32 +++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 37b34b588601..6bd7c8e03c14 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -55,6 +55,18 @@ static struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
 
 #pragma GCC poison to_vmx
 
+static u64 intel_pmu_fixed_ctrl_bits(struct kvm_pmu *pmu, u64 bits)
+{
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	u64 fixed_ctrl_bits = 0;
+	int i;
+
+	kvm_for_each_fixed_counter(i, fixed_mask)
+		fixed_ctrl_bits |= intel_fixed_bits_by_idx(i, bits);
+
+	return fixed_ctrl_bits;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	/*
@@ -537,20 +549,12 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	return eventsel;
 }
 
-static void intel_pmu_enable_fixed_counter_bits(struct kvm_pmu *pmu, u64 bits)
-{
-	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
-	int i;
-
-	kvm_for_each_fixed_counter(i, fixed_mask)
-		pmu->fixed_ctr_ctrl_rsvd &= ~intel_fixed_bits_by_idx(i, bits);
-}
-
 static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct lbr_desc *lbr_desc = vcpu_to_lbr_desc(vcpu);
 	struct kvm_cpuid_entry2 *entry;
+	u64 fixed_enable_bits = 0;
 	union cpuid10_eax eax;
 	union cpuid10_edx edx;
 	u64 perf_capabilities;
@@ -643,9 +647,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 					  kvm_pmu_cap.bit_width_fixed);
 	pmu->counter_bitmask[KVM_PMC_FIXED] = BIT_ULL(edx.split.bit_width_fixed) - 1;
 
-	intel_pmu_enable_fixed_counter_bits(pmu, INTEL_FIXED_0_KERNEL |
-						 INTEL_FIXED_0_USER |
-						 INTEL_FIXED_0_ENABLE_PMI);
+	fixed_enable_bits |= INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER |
+			     INTEL_FIXED_0_ENABLE_PMI;
 
 	counter_rsvd = ~((u64)kvm_gp_pmc_mask(pmu) |
 			 ((u64)kvm_fixed_pmc_mask(pmu) << KVM_FIXED_PMC_BASE_IDX));
@@ -661,11 +664,14 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 			pmu->pebs_enable_rsvd = counter_rsvd;
 			pmu->eventsel_rsvd &= ~ICL_EVENTSEL_ADAPTIVE;
 			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
-			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
+			fixed_enable_bits |= ICL_FIXED_0_ADAPTIVE;
 		} else {
 			pmu->pebs_enable_rsvd = ~(u64)kvm_gp_pmc_mask(pmu);
 		}
 	}
+
+	pmu->fixed_ctr_ctrl_rsvd &=
+		~intel_pmu_fixed_ctrl_bits(pmu, fixed_enable_bits);
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
-- 
2.56.0

[PATCH v3 15/18] KVM: x86/pmu: Mask only OS/USR bits when filtering fixed counters

[Zide Chen]

In preparation for supporting AnyThread deprecation and Fixed Counter 3,
mask only INTEL_FIXED_0_KERNEL and INTEL_FIXED_0_USER when updating
fixed_ctr_ctrl_hw in kvm_mediated_pmu_refresh_event_filter().

Masking the full 0xf nibble would cause two issues:

1. AnyThread for Fixed Counters 0-2: When a counter is allowed, copying
   the full nibble from guest-visible pmu->fixed_ctr_ctrl back into
   hardware-visible pmu->fixed_ctr_ctrl_hw would resurrect AnyThread in
   hardware, defeating SMT cross-VM isolation.

2. Metrics Clear for Fixed Counter 3: If an event filter denies Fixed
   Counter 3, clearing 0xf inadvertently strips METRICS_CLEAR_EN from
   hardware. If the guest sets METRICS_CLEAR_EN and executes native RDPMC,
   metrics will remain un-cleared, returning stale data.

This gates hardware counting without clobbering other control bits,
matching general-purpose counter filtering which only masks the event
ENABLE bit.

Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- New patch.
---
 arch/x86/kvm/pmu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index 2cb5bccc7044..afb3e18a9879 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -557,7 +557,9 @@ static void kvm_mediated_pmu_refresh_event_filter(struct kvm_pmc *pmc)
 			pmc->eventsel_hw |= pmc->eventsel &
 					    ARCH_PERFMON_EVENTSEL_ENABLE;
 	} else {
-		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX, 0xf);
+		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX,
+						    INTEL_FIXED_0_KERNEL |
+						    INTEL_FIXED_0_USER);
 
 		pmu->fixed_ctr_ctrl_hw &= ~mask;
 		if (allowed)
-- 
2.56.0

[PATCH v3 16/18] KVM: x86/pmu: Ignore AnyThread bit if CPUID.0AH:EDX[15] is set

[Zide Chen]

Intel PerfMon v5 introduces the ANYTHREAD_DEPRECATION capability
(CPUID.0AH:EDX[15]) to indicate that AnyThread counting is deprecated
and that writes to the AnyThread bit in IA32_PERFEVTSELx and
IA32_FIXED_CTR_CTRL are ignored.

Preserve the guest-written value in guest-visible state so that guest
RDMSRs observe the AnyThread bit, but strip it before programming
hardware to ensure cross-VM SMT isolation.

Continue to inject #GP when ANYTHREAD_DEPRECATION is not present in the
guest CPUID, e.g. for PerfMon v3/v4 guests or when the capability is
not configured for a PerfMon v5+ guest.

Advertise CPUID.0AH:EDX[15] only for PMU version 5+ guests, since the
bit doesn't exist below version 5.

Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Exclude Fixed Counter 3 bit 14 (METRICS_CLEAR_EN) from the AnyThread mask
  so it remains reserved in fixed_ctr_ctrl_rsvd. (Jim)
- Preserve AnyThread in guest-visible state (eventsel/fixed_ctr_ctrl) to
  match bare-metal RDMSR behavior. (Jim)
- Removed it from the hardware-visible counterparts (_hw) so that it
  doesn't need to explicitly remove this bit in intel_pmu_set_msr().
  (Jim)
- Re-use intel_pmu_fixed_ctrl_bits() for AnyThread bits computation.
  (Sashiko)

v2:
- Advertise anythread_deprecated only if kvm_pmu_cap.version >= 5.
- Add the missing AnyThr bit handling for IA32_FIXED_CTR_CTRL.
- Update the comments to be clearer.
---
 arch/x86/kvm/cpuid.c         |  2 +-
 arch/x86/kvm/vmx/pmu_intel.c | 28 +++++++++++++++++++++++++---
 2 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index 00a2ca6d3049..66e68a867946 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -1549,7 +1549,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
-		if (kvm_pmu_cap.version)
+		if (kvm_pmu_cap.version >= 5)
 			edx.split.anythread_deprecated = 1;
 
 		entry->eax = eax.full;
diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
index 6bd7c8e03c14..961d16e7f31f 100644
--- a/arch/x86/kvm/vmx/pmu_intel.c
+++ b/arch/x86/kvm/vmx/pmu_intel.c
@@ -67,6 +67,13 @@ static u64 intel_pmu_fixed_ctrl_bits(struct kvm_pmu *pmu, u64 bits)
 	return fixed_ctrl_bits;
 }
 
+static u64 intel_pmu_fixed_anythread_mask(struct kvm_pmu *pmu)
+{
+	/* Fixed counter 3 repurposes that bit for METRICS_CLEAR_EN. */
+	return intel_pmu_fixed_ctrl_bits(pmu, INTEL_FIXED_0_ANYTHREAD) &
+	       ~intel_fixed_bits_by_idx(3, INTEL_FIXED_3_METRICS_CLEAR);
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	/*
@@ -83,9 +90,9 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 	int i;
 
 	pmu->fixed_ctr_ctrl = data;
-	pmu->fixed_ctr_ctrl_hw = data;
+	pmu->fixed_ctr_ctrl_hw = data & ~intel_pmu_fixed_anythread_mask(pmu);
 	kvm_for_each_fixed_counter(i, fixed_mask) {
-		u8 new_ctrl = fixed_ctrl_field(data, i);
+		u8 new_ctrl = fixed_ctrl_field(pmu->fixed_ctr_ctrl_hw, i);
 		u8 old_ctrl = fixed_ctrl_field(old_fixed_ctr_ctrl, i);
 
 		if (old_ctrl == new_ctrl)
@@ -500,7 +507,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 
 			if (data != pmc->eventsel) {
 				pmc->eventsel = data;
-				pmc->eventsel_hw = data;
+				pmc->eventsel_hw = data & ~ARCH_PERFMON_EVENTSEL_ANY;
 				kvm_pmu_request_counter_reprogram(pmc);
 			}
 			break;
@@ -672,6 +679,21 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 
 	pmu->fixed_ctr_ctrl_rsvd &=
 		~intel_pmu_fixed_ctrl_bits(pmu, fixed_enable_bits);
+
+	/*
+	 * AnyThread counting is not supported by KVM due to cross-VM
+	 * information leakage concerns on SMT cores. Therefore, AnyThread
+	 * remains unavailable for PerfMon v3/v4 guests, where AnyThread
+	 * deprecation is not enumerated.
+	 *
+	 * On PerfMon v5+ with ANYTHREAD_DEPRECATION, writes to AnyThread bits
+	 * are accepted and preserved on RDMSR, while counting behavior is
+	 * ignored.
+	 */
+	if (pmu->version >= 5 && edx.split.anythread_deprecated) {
+		pmu->eventsel_rsvd &= ~ARCH_PERFMON_EVENTSEL_ANY;
+		pmu->fixed_ctr_ctrl_rsvd &= ~intel_pmu_fixed_anythread_mask(pmu);
+	}
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
-- 
2.56.0

[PATCH v3 17/18] KVM: x86/pmu: Advertise PerfMon version 5 on Intel hosts

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

KVM currently caps the guest PerfMon version at 2 for all Intel
platforms. Now that KVM emulates the basic architectural PMU features
introduced in PerfMon versions 3 through 5, raise the guest PerfMon
version to 5. Features that require additional emulation support, e.g.
architectural LBR will be enabled separately.

PerfMon versions 3 and 4 continue to be exposed as version 2 because
KVM cannot safely virtualize AnyThread due to cross-VM information
leakage. PerfMon v5 is supported because it provides the
AnyThread-deprecated capability, allowing KVM to indicate that
AnyThread is deprecated.

When enable_mediated_pmu is disabled or in the non-Intel paths, KVM
retains the existing cap of version 2.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- Simplify version cap logic since mediated PMU already requires
  version >= 4. (Jim)
v2:
- Cap vPMU v3/v4 to v2, as AnyThread cannot be safely virtualized.
  (Sashiko)
---
 arch/x86/kvm/pmu.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
index afb3e18a9879..f949fdbf359c 100644
--- a/arch/x86/kvm/pmu.c
+++ b/arch/x86/kvm/pmu.c
@@ -175,7 +175,17 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 	}
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
-	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
+
+	/*
+	 * PMU v3/v4 has AnyThread support, which KVM cannot safely virtualize.
+	 * Only expose PMU v5+ for Intel mediated PMU, where AnyThread is
+	 * deprecated; otherwise, cap at PMU v2.
+	 */
+	if (is_intel && enable_mediated_pmu && kvm_pmu_cap.version >= 5)
+		kvm_pmu_cap.version = 5;
+	else
+		kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
+
 	kvm_pmu_cap.cntr_mask64 &=
 		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
 	kvm_pmu_cap.fixed_cntr_mask64 &=
-- 
2.56.0

[PATCH v3 18/18] KVM: selftests: Support fixed counters bitmap in pmu_counters_test

[Zide Chen]

From: Dapeng Mi <dapeng1.mi@linux.intel.com>

On PerfMon v5, CPUID.0AH:ECX represents the fixed counter bitmask,
while EDX[4:0] indicates the number of contiguous fixed counters
starting from 0:

FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i).

Fix test_fixed_counters() to derive supported fixed counters using the
above formula instead of relying solely on EDX[4:0]. This allows the
test to cover non-contiguous fixed counter configurations.

Update this_pmu_has() to take the PMU version into consideration.
Without this fix, it could return incorrect results in test cases such
as test_arch_events(), which configure a PMU version different from
the one originally advertised by KVM without explicitly updating
CPUID.0AH.{ECX,EDX} to match the new PMU version.

For example, after downgrading a guest from PMU v5 to PMU v2,
this_pmu_has() would still consult CPUID.0AH.ECX while KVM ignores it,
resulting in a mismatch in the reported fixed counter capabilities.

Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Co-developed-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Zide Chen <zide.chen@intel.com>
---
v3:
- use the BIT() macro instead of an open-coded left shift. (Sashiko)
- Simplified ternary condition in this_pmu_has(). (Sashiko)
- Also test legacy EDX contiguous counts with ECX=0 in
  test_fixed_counters(). (Sashiko)

v2:
- Fix this_pmu_has() to honor the guest's configured PMU version.
- Update the pmu_vm_create_with_vcpus() calls after rebasing.
---
 .../selftests/kvm/include/x86/processor.h     |  6 +++-
 .../selftests/kvm/x86/pmu_counters_test.c     | 28 +++++++++++++++----
 2 files changed, 28 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h
index ffec2291b1e1..21381db1c4ca 100644
--- a/tools/testing/selftests/kvm/include/x86/processor.h
+++ b/tools/testing/selftests/kvm/include/x86/processor.h
@@ -836,6 +836,7 @@ static __always_inline bool this_cpu_has_p(struct kvm_x86_cpu_property property)
 
 static inline bool this_pmu_has(struct kvm_x86_pmu_feature feature)
 {
+	u8 pmu_version;
 	u32 nr_bits;
 
 	if (feature.f.reg == KVM_CPUID_EBX) {
@@ -845,7 +846,10 @@ static inline bool this_pmu_has(struct kvm_x86_pmu_feature feature)
 
 	GUEST_ASSERT(feature.f.reg == KVM_CPUID_ECX);
 	nr_bits = this_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
-	return nr_bits > feature.f.bit || this_cpu_has(feature.f);
+	pmu_version = this_cpu_property(X86_PROPERTY_PMU_VERSION);
+
+	return nr_bits > feature.f.bit ||
+	       (pmu_version >= 5 && this_cpu_has(feature.f));
 }
 
 static __always_inline u64 this_cpu_supported_xcr0(void)
diff --git a/tools/testing/selftests/kvm/x86/pmu_counters_test.c b/tools/testing/selftests/kvm/x86/pmu_counters_test.c
index c3e784e16348..77d67fa6e3fc 100644
--- a/tools/testing/selftests/kvm/x86/pmu_counters_test.c
+++ b/tools/testing/selftests/kvm/x86/pmu_counters_test.c
@@ -3,6 +3,7 @@
  * Copyright (C) 2023, Tencent, Inc.
  */
 #include <x86intrin.h>
+#include <linux/bitmap.h>
 
 #include "pmu.h"
 #include "processor.h"
@@ -631,25 +632,42 @@ static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters,
 static void test_fixed_counters(u8 pmu_version, u64 perf_capabilities)
 {
 	u8 nr_fixed_counters = kvm_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
+	unsigned long fixed_subset;
 	struct kvm_vcpu **vcpus;
 	struct kvm_vm *vm;
+	u32 fixed_bitmap;
 	int i = 0;
-	u32 k;
 	u8 j;
 
 	pr_info("Testing %u fixed counters, PMU version %u, perf_caps = %lx\n",
 		nr_fixed_counters, pmu_version, perf_capabilities);
 
+	fixed_bitmap = BIT_ULL(nr_fixed_counters) - 1;
+	if (pmu_version >= 5)
+		fixed_bitmap |= kvm_cpu_property(X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK);
 
-	vm = pmu_vm_create_with_vcpus((nr_fixed_counters + 1) * BIT(nr_fixed_counters),
+	vm = pmu_vm_create_with_vcpus(BIT(__builtin_popcount(fixed_bitmap)) + nr_fixed_counters,
 				      guest_test_fixed_counters,
 				      pmu_version, perf_capabilities, &vcpus);
 
-	for (j = 0; j <= nr_fixed_counters; j++) {
-		for (k = 0; k <= (BIT(nr_fixed_counters) - 1); k++)
-			__test_fixed_counters(vcpus[i++], j, k);
+	for (fixed_subset = 0; fixed_subset <= fixed_bitmap; fixed_subset++) {
+		u32 nr_contiguous;
+
+		/*
+		 * The loop walks all values from 0 to fixed_bitmap, so skip any
+		 * value that is not a subset of fixed_bitmap.
+		 */
+		if (fixed_subset & ~fixed_bitmap)
+			continue;
+
+		nr_contiguous = find_first_zero_bit(&fixed_subset,
+						    MAX_NR_FIXED_COUNTERS);
+		__test_fixed_counters(vcpus[i++], nr_contiguous, fixed_subset);
 	}
 
+	for (j = 1; j <= nr_fixed_counters; j++)
+		__test_fixed_counters(vcpus[i++], j, 0);
+
 	pmu_vm_free(vm, vcpus);
 }
 
-- 
2.56.0

Re: [PATCH v3 01/18] KVM: x86/pmu: Remove redundant Perf Global Status MSR bit definitions

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:10PM -0700, Zide Chen wrote:
> IA32_PERF_GLOBAL_STATUS and its SET/RESET counterparts share bit
> definitions with perf_event.h, which is the canonical home for Intel
> PMU bit definitions (e.g., PERFEVTSEL, FIXED_CTR_CTRL, GLOBAL_STATUS).
> Drop the duplicate definitions from msr-index.h and update KVM code to
> use the macros from perf_event.h.
> 
> Drop the comment as well, since the macros now match the MSR
> naming.
> 
> No functional change intended.
> 
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Reviewed-by: Jim Mattson <jmattson@google.com>

Reviewed-by: Yosry Ahmed <yosry@kernel.org>

Re: [PATCH v3 02/18] KVM: x86/pmu: Rename all_valid_pmc_idx to pmc_exists

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:11PM -0700, Zide Chen wrote:
> all_valid_pmc_idx is a bitmap of valid PMC indices, not an index
> itself; rename it to pmc_exists to better reflect what it holds and
> keep the name consistent with other fields in the same struct.
> 
> No functional change intended.
> 
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Reviewed-by: Jim Mattson <jmattson@google.com>

Reviewed-by: Yosry Ahmed <yosry@kernel.org>

Re: [PATCH v3 03/18] KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:12PM -0700, Zide Chen wrote:
> This field stores the bits that are reserved in guest IA32_PERFEVTSELx
> MSRs. Rename it to eventsel_rsvd to better reflect its purpose and to
> align with other field names in struct kvm_pmu.
> 
> Opportunistically, replace magic numbers with macros.
> 
> No functional change intended.
> 
> Suggested-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Reviewed-by: Jim Mattson <jmattson@google.com>
> Reviewed-by: Sandipan Das <sandipan.das@amd.com>
> ---
> v3:
> - Mention "magic numbers" in changelog to cover AMD as well.
> 
> v2:
> - Replace magic numbers in AMD pmu->eventsel_rsvd.
> ---
>  arch/x86/include/asm/kvm_host.h |  2 +-
>  arch/x86/kvm/pmu.c              |  6 +++++-
>  arch/x86/kvm/svm/pmu.c          | 12 +++++++++---
>  arch/x86/kvm/vmx/pmu_intel.c    | 12 ++++++------
>  4 files changed, 21 insertions(+), 11 deletions(-)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 72b8fbf432a2..b8875ed5502c 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -581,7 +581,7 @@ struct kvm_pmu {
>  	u64 counter_bitmask[2];
>  	u64 global_ctrl_rsvd;
>  	u64 global_status_rsvd;
> -	u64 reserved_bits;
> +	u64 eventsel_rsvd;
>  	u64 raw_event_mask;
>  	struct kvm_pmc gp_counters[KVM_MAX_NR_GP_COUNTERS];
>  	struct kvm_pmc fixed_counters[KVM_MAX_NR_FIXED_COUNTERS];
> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> index 83883191d124..cbd63329cf13 100644
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -983,7 +983,11 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
>  	pmu->nr_arch_fixed_counters = 0;
>  	pmu->counter_bitmask[KVM_PMC_GP] = 0;
>  	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
> -	pmu->reserved_bits = 0xffffffff00200000ull;
> +	/*
> +	 * KVM is not able to emulate the AnyThread bit due to cross-VM
> +	 * information leakage on SMT cores.
> +	 */
> +	pmu->eventsel_rsvd = GENMASK_ULL(63, 32) | ARCH_PERFMON_EVENTSEL_ANY;
>  	pmu->raw_event_mask = X86_RAW_EVENT_MASK;
>  	pmu->global_ctrl_rsvd = ~0ull;
>  	pmu->global_status_rsvd = ~0ull;
> diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
> index c18286545a7a..52224872e0d4 100644
> --- a/arch/x86/kvm/svm/pmu.c
> +++ b/arch/x86/kvm/svm/pmu.c
> @@ -163,7 +163,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  	/* MSR_EVNTSELn */
>  	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_EVNTSEL);
>  	if (pmc) {
> -		data &= ~pmu->reserved_bits;
> +		data &= ~pmu->eventsel_rsvd;
>  		if (data != pmc->eventsel) {
>  			pmc->eventsel = data;
>  			pmc->eventsel_hw = (data & ~AMD64_EVENTSEL_HOSTONLY) |
> @@ -214,9 +214,15 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
>  
>  	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(48) - 1;
>  
> -	pmu->reserved_bits = 0xfffffff000280000ull;
> +	/*
> +	 * AMD PerfEvtSeln registers do not support PinControl, and KVM does
> +	 * not emulate AnyThread due to cross-VM information leakage on SMT
> +	 * cores (as on the Intel side).
> +	 */
> +	pmu->eventsel_rsvd = GENMASK_ULL(63, 36) | ARCH_PERFMON_EVENTSEL_PIN_CONTROL |
> +			     ARCH_PERFMON_EVENTSEL_ANY;

It's probably better if we diff against the generic pmu->eventsel_rsvd
set by kvm_pmu_refresh(), and explicitly add/remove bits as needed here,
like the Intel code. Then we also avoid re-iterating the AnyThread
rationale here.

What do you think?


>  	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) && kvm_vcpu_has_mediated_pmu(vcpu))
> -		pmu->reserved_bits &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
> +		pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;

Re: [PATCH v3 04/18] KVM: x86/pmu: Add PMC bitmap accessor helpers

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:13PM -0700, Zide Chen wrote:
> pmu->nr_arch_{gp,fixed}_counters is not able to represent that a PMU
> may include non-contiguous GP or fixed counters.
> 
> pmu->pmc_exists already holds a bitmap indicating both fixed and
> general-purpose counters, and loops over valid counters can be done via
> pmu->pmc_exists alone. Extend it to a union so that the u64 alias is
> available for convenient mask arithmetic operations.
> 
> Add the necessary helpers to prepare for bitmap-based PMC counter
> implementation.
> 
> No functional change intended.
> 
> Co-developed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Reviewed-by: Jim Mattson <jmattson@google.com>
> ---
> v3:
> - Add BUILD_BUG_ON(). (Jim)
> v2:
> - Rename kvm_{gp,fixed}_pmc_supported() to kvm_is_{gp,fixed}_pmc_supported()
>   and simplify them by removing the temporary bitmap. (Sean)
> - Split kvm_for_each_set_pmc_idx() to kvm_for_each_gp_counter()
>   and kvm_for_each_fixed_counter(). (Sean)
> ---
>  arch/x86/include/asm/kvm_host.h |  5 +++-
>  arch/x86/kvm/pmu.c              |  2 ++
>  arch/x86/kvm/pmu.h              | 53 +++++++++++++++++++++++++++++----
>  3 files changed, 53 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index b8875ed5502c..4f192429ed93 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -595,7 +595,10 @@ struct kvm_pmu {
>  		DECLARE_BITMAP(reprogram_pmi, X86_PMC_IDX_MAX);
>  		atomic64_t __reprogram_pmi;
>  	};
> -	DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
> +	union {
> +		DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
> +		u64 pmc_exists64;

__pmc_exists would be more consistent with __reprogam_pmi above.

> +	};
>  	DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX);
>  
>  	DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX);
> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> index cbd63329cf13..5e174d95f3b7 100644
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -1024,6 +1024,8 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
>  {
>  	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
>  
> +	BUILD_BUG_ON(sizeof(pmu->pmc_exists) != sizeof(pmu->pmc_exists64));
> +
>  	memset(pmu, 0, sizeof(*pmu));
>  	kvm_pmu_call(init)(vcpu);
>  }
> diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h
> index 090c9bbb74f4..016935c3a8ec 100644
> --- a/arch/x86/kvm/pmu.h
> +++ b/arch/x86/kvm/pmu.h
> @@ -88,6 +88,30 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
>  	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
>  }
>  
> +static inline unsigned long kvm_gp_pmc_mask(struct kvm_pmu *pmu)
> +{
> +	return pmu->pmc_exists64 &
> +	       GENMASK_ULL(KVM_MAX_NR_GP_COUNTERS - 1, 0);
> +}
> +
> +static inline unsigned long kvm_fixed_pmc_mask(struct kvm_pmu *pmu)
> +{
> +	return (pmu->pmc_exists64 >> KVM_FIXED_PMC_BASE_IDX) &
> +	       GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
> +}
> +
> +static inline bool kvm_is_gp_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
> +{
> +	return idx < KVM_MAX_NR_GP_COUNTERS &&
> +	       test_bit(idx, pmu->pmc_exists);
> +}
> +
> +static inline bool kvm_is_fixed_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
> +{
> +	return idx < KVM_MAX_NR_FIXED_COUNTERS &&
> +	       test_bit(KVM_FIXED_PMC_BASE_IDX + idx, pmu->pmc_exists);
> +}
> +
>  /*
>   * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
>   * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
> @@ -104,11 +128,11 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
>   */
>  static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
>  {
> -	if (idx < pmu->nr_arch_gp_counters)
> +	if (kvm_is_gp_pmc_supported(pmu, idx))
>  		return &pmu->gp_counters[idx];
>  
>  	idx -= KVM_FIXED_PMC_BASE_IDX;
> -	if (idx >= 0 && idx < pmu->nr_arch_fixed_counters)
> +	if (kvm_is_fixed_pmc_supported(pmu, idx))
>  		return &pmu->fixed_counters[idx];
>  
>  	return NULL;
> @@ -120,6 +144,17 @@ static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
>  			continue;				\
>  		else						\
>  
> +/*
> + * @mask is expected to be a scalar unsigned long derived from pmu->pmc_exists,
> + * which is already constrained by KVM_MAX_NR_{AMD,INTEL}_{GP,FIXED}_COUNTERS,
> + * so iteration up to KVM's maximum counter count is safe.
> + */
> +#define kvm_for_each_gp_counter(i, mask)	\
> +	for_each_set_bit((i), &(mask), KVM_MAX_NR_GP_COUNTERS)
> +
> +#define kvm_for_each_fixed_counter(i, mask)	\
> +	for_each_set_bit((i), &(mask), KVM_MAX_NR_FIXED_COUNTERS)

These are not used in this patch, are they?

They should probably be introduced when they are used. Also, do we
always end up passing kvm_gp_pmc_mask() and kvm_fixed_pmc_mask()? If
yes, maybe we should pass in the pmu here instead?

> @@ -183,9 +221,12 @@ static inline struct kvm_pmc *get_fixed_pmc(struct kvm_pmu *pmu, u32 msr)
>  {
>  	int base = MSR_CORE_PERF_FIXED_CTR0;
>  
> -	if (msr >= base && msr < base + pmu->nr_arch_fixed_counters) {
> +	if (msr >= base && msr < base + KVM_MAX_NR_FIXED_COUNTERS) {
>  		u32 index = array_index_nospec(msr - base,
> -					       pmu->nr_arch_fixed_counters);
> +					       KVM_MAX_NR_FIXED_COUNTERS);
> +
> +		if (!kvm_is_fixed_pmc_supported(pmu, index))
> +			return NULL;

Do all callers handle NULL correctly? At least
reprogram_fixed_counters() doesn't look like it. It may be fine since
NULL can already be returned today. Not sure if this series makes it
more likely.

>  
>  		return &pmu->fixed_counters[index];
>  	}
> -- 
> 2.56.0
> 

Re: [PATCH v3 05/18] KVM: x86/pmu: Drop nr_arch_{gp,fixed}_counters from kvm_pmu

[Yosry Ahmed]

> @@ -979,8 +979,6 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
>  	kvm_pmu_reset(vcpu);
>  
>  	pmu->version = 0;
> -	pmu->nr_arch_gp_counters = 0;
> -	pmu->nr_arch_fixed_counters = 0;
>  	pmu->counter_bitmask[KVM_PMC_GP] = 0;
>  	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
>  	/*
> @@ -1008,16 +1006,12 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
>  	 * in the global controls).  Emulate that behavior when refreshing the
>  	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
>  	 */
> -	if (pmu->nr_arch_gp_counters &&
> +	if (kvm_gp_pmc_mask(pmu) &&
>  	    (kvm_pmu_has_perf_global_ctrl(pmu) || kvm_vcpu_has_mediated_pmu(vcpu)))
> -		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
> +		pmu->global_ctrl = kvm_gp_pmc_mask(pmu);
>  
>  	if (kvm_vcpu_has_mediated_pmu(vcpu))
>  		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
> -
> -	bitmap_set(pmu->pmc_exists, 0, pmu->nr_arch_gp_counters);
> -	bitmap_set(pmu->pmc_exists, KVM_FIXED_PMC_BASE_IDX,
> -		   pmu->nr_arch_fixed_counters);

I understand the vendor modules are responsible of initializing this
now, but should we clear it before calling into the vendors like other
fields above?

>  }
>  
>  void kvm_pmu_init(struct kvm_vcpu *vcpu)
> @@ -1328,6 +1322,8 @@ static __always_inline u32 gp_eventsel_msr(u32 idx)
>  static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
>  {
>  	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
> +	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
> +	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);

All these should go away if we pass the pmu into the iterators.

>  	struct kvm_pmc *pmc;
>  	u32 i;
>  
> @@ -1336,14 +1332,14 @@ static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
>  	 * is intercepted if hardware has counters that aren't visible to the
>  	 * guest (KVM will inject #GP as appropriate).
>  	 */
> -	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
> +	kvm_for_each_gp_counter(i, gp_mask) {
>  		pmc = &pmu->gp_counters[i];
>  
>  		if (pmc->counter != rdpmc(i))
>  			wrmsrq(gp_counter_msr(i), pmc->counter);
>  		wrmsrq(gp_eventsel_msr(i), pmc->eventsel_hw);
>  	}
> -	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
> +	kvm_for_each_fixed_counter(i, fixed_mask) {
>  		pmc = &pmu->fixed_counters[i];
>  
>  		if (pmc->counter != rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i))
[..]
> @@ -117,7 +116,7 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
>  		return pmu->version > 1;
>  	default:
>  		if (msr > MSR_F15H_PERF_CTR5 &&
> -		    msr < MSR_F15H_PERF_CTL0 + 2 * pmu->nr_arch_gp_counters)
> +		    msr < MSR_F15H_PERF_CTL0 + 2 * hweight_long(kvm_gp_pmc_mask(pmu)))

This assumes the MSRs are contiguous, which IIUC is always true, but I
wonder if it should also be handled since the series claims to also
prepare for discontiguous GP counters.

>  			return pmu->version > 1;
>  		break;
>  	}
[..]
> @@ -107,6 +108,8 @@ static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct kvm_vcpu *vcpu,
>  	if (WARN_ON_ONCE(!pmu->version))
>  		return NULL;
>  
> +	idx &= INTEL_RDPMC_INDEX_MASK;
> +
>  	/*
>  	 * General Purpose (GP) PMCs are supported on all PMUs, and fixed PMCs
>  	 * are supported on all architectural PMUs, i.e. on all virtual PMUs
> @@ -117,23 +120,25 @@ static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct kvm_vcpu *vcpu,
>  	 */
>  	switch (type) {
>  	case INTEL_RDPMC_FIXED:
> +		if (!kvm_is_fixed_pmc_supported(pmu, idx))
> +			return NULL;
> +
>  		counters = pmu->fixed_counters;
> -		num_counters = pmu->nr_arch_fixed_counters;
> +		num_counters = KVM_MAX_NR_INTEL_FIXED_COUNTERS;
>  		bitmask = pmu->counter_bitmask[KVM_PMC_FIXED];
>  		break;
>  	case INTEL_RDPMC_GP:
> +		if (!kvm_is_gp_pmc_supported(pmu, idx))
> +			return NULL;
> +
>  		counters = pmu->gp_counters;
> -		num_counters = pmu->nr_arch_gp_counters;
> +		num_counters = KVM_MAX_NR_INTEL_GP_COUNTERS;

num_counters is no longer an accurate name, is it? Maybe max_counter
(and use KVM_MAX_* - 1) or max_nr_counters?

>  		bitmask = pmu->counter_bitmask[KVM_PMC_GP];
>  		break;
>  	default:
>  		return NULL;
>  	}
>  
> -	idx &= INTEL_RDPMC_INDEX_MASK;
> -	if (idx >= num_counters)
> -		return NULL;
> -
>  	*mask &= bitmask;
>  	return &counters[array_index_nospec(idx, num_counters)];
>  }
> @@ -495,9 +500,10 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
>  
>  static void intel_pmu_enable_fixed_counter_bits(struct kvm_pmu *pmu, u64 bits)
>  {
> +	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
>  	int i;
>  
> -	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
> +	kvm_for_each_fixed_counter(i, fixed_mask)
>  		pmu->fixed_ctr_ctrl_rsvd &= ~intel_fixed_bits_by_idx(i, bits);
>  }
>  
> @@ -509,6 +515,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  	union cpuid10_eax eax;
>  	union cpuid10_edx edx;
>  	u64 perf_capabilities;
> +	u64 fixed_mask;
> +	int nr_gp_counters;
>  	u64 counter_rsvd;
>  
>  	if (!lbr_desc)
> @@ -535,8 +543,6 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  	if (!pmu->version)
>  		return;
>  
> -	pmu->nr_arch_gp_counters = min_t(int, eax.split.num_counters,
> -					 kvm_pmu_cap.num_counters_gp);
>  	eax.split.bit_width = min_t(int, eax.split.bit_width,
>  				    kvm_pmu_cap.bit_width_gp);
>  	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(eax.split.bit_width) - 1;
> @@ -544,6 +550,17 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  				      kvm_pmu_cap.events_mask_len);
>  	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
>  
> +	fixed_mask = BIT_ULL(edx.split.num_counters_fixed) - 1;
> +	fixed_mask &= BIT_ULL(kvm_pmu_cap.num_counters_fixed) - 1;

Why not do min() like the old code?

> +
> +	/*
> +	 * The number of counters comes from guest CPUID data. Clamp the value
> +	 * to avoid a shift-by-64 in BIT_ULL().
> +	 */
> +	nr_gp_counters = min_t(int, eax.split.num_counters, X86_PMC_IDX_MAX - 1);
> +	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
> +			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);

Why not just do min(eax.split.num_counters, kvm_pmu_cap.num_counters_gp)
directly like it used to be?

> +
>  	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
>  	if (entry &&
>  	    (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) &&
> @@ -565,8 +582,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  	if (pmu->version == 1)
>  		return;
>  
> -	pmu->nr_arch_fixed_counters = min_t(int, edx.split.num_counters_fixed,
> -					    kvm_pmu_cap.num_counters_fixed);
> +	pmu->pmc_exists64 |= fixed_mask << INTEL_PMC_IDX_FIXED;
>  	edx.split.bit_width_fixed = min_t(int, edx.split.bit_width_fixed,
>  					  kvm_pmu_cap.bit_width_fixed);
>  	pmu->counter_bitmask[KVM_PMC_FIXED] = BIT_ULL(edx.split.bit_width_fixed) - 1;
> @@ -575,8 +591,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  						 INTEL_FIXED_0_USER |
>  						 INTEL_FIXED_0_ENABLE_PMI);
>  
> -	counter_rsvd = ~((BIT_ULL(pmu->nr_arch_gp_counters) - 1) |
> -			 ((BIT_ULL(pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX));
> +	counter_rsvd = ~((u64)kvm_gp_pmc_mask(pmu) |
> +			 ((u64)kvm_fixed_pmc_mask(pmu) << KVM_FIXED_PMC_BASE_IDX));
>  	pmu->global_ctrl_rsvd = counter_rsvd;
>  
>  	pmu->global_status_rsvd = pmu->global_ctrl_rsvd
> @@ -591,7 +607,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
>  			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
>  		} else {
> -			pmu->pebs_enable_rsvd = ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1);
> +			pmu->pebs_enable_rsvd = ~(u64)kvm_gp_pmc_mask(pmu);
>  		}
>  	}
>  }

Re: [PATCH v3 06/18] KVM: x86/pmu: Expose kvm_host_pmu to vendor modules

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:15PM -0700, Zide Chen wrote:
> kvm_host_pmu holds the unadulterated host PMU capabilities, which
> can be used to compare against guest capabilities to determine whether
> certain MSRs should be intercepted by KVM.
> 
> Exposing it directly avoids the need to introduce per-field accessors,
> which would add boilerplate without hiding any implementation detail
> worth encapsulating.
> 
> Opportunistically, fix typo "Unadultered" in the comment.
> 
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Reviewed-by: Jim Mattson <jmattson@google.com>

Reviewed-by: Yosry Ahmed <yosry@kernel.org>

Re: [PATCH v3 08/18] KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities

[Yosry Ahmed]

> diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
> index adafd6fa2c54..3310c370c5fa 100644
> --- a/arch/x86/kvm/cpuid.c
> +++ b/arch/x86/kvm/cpuid.c
> @@ -1535,10 +1535,18 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
>  		}
>  
>  		eax.split.version_id = kvm_pmu_cap.version;
> -		eax.split.num_counters = kvm_pmu_cap.num_counters_gp;
> +
> +		/* Contiguous GP counters only. */
> +		eax.split.num_counters =
> +			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
> +					    KVM_MAX_NR_GP_COUNTERS);
>  		eax.split.bit_width = kvm_pmu_cap.bit_width_gp;
>  		eax.split.mask_length = kvm_pmu_cap.events_mask_len;
> -		edx.split.num_counters_fixed = kvm_pmu_cap.num_counters_fixed;
> +
> +		/* Contiguous fixed counters only. */
> +		edx.split.num_counters_fixed =
> +			find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
> +					    KVM_MAX_NR_FIXED_COUNTERS);
>  		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
>  
>  		if (kvm_pmu_cap.version)
> @@ -1903,7 +1911,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
>  
>  		cpuid_entry_override(entry, CPUID_8000_0022_EAX);
>  
> -		ebx.split.num_core_pmc = kvm_pmu_cap.num_counters_gp;
> +		ebx.split.num_core_pmc = hweight64(kvm_pmu_cap.cntr_mask64);

Should this be the number of contiguous counters?

>  		entry->ebx = ebx.full;
>  		break;
>  	}
[..]  
> @@ -516,7 +517,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  	union cpuid10_edx edx;
>  	u64 perf_capabilities;
>  	u64 fixed_mask;
> -	int nr_gp_counters;
> +	int nr_gp_counters, nr_fixed_counters;
>  	u64 counter_rsvd;
>  
>  	if (!lbr_desc)
> @@ -550,16 +551,25 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  				      kvm_pmu_cap.events_mask_len);
>  	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
>  
> -	fixed_mask = BIT_ULL(edx.split.num_counters_fixed) - 1;
> -	fixed_mask &= BIT_ULL(kvm_pmu_cap.num_counters_fixed) - 1;
> +	/*
> +	 * CPUID.0AH:EDX[4:0] can only enumerate contiguous counters. Clamp the
> +	 * value to the number of contiguous fixed counters supported by KVM.
> +	 */
> +	nr_fixed_counters = min_t(int, edx.split.num_counters_fixed,
> +				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
> +						      KVM_MAX_NR_FIXED_COUNTERS));
> +
> +	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
> +	fixed_mask &= kvm_pmu_cap.fixed_cntr_mask64;

Isn't nr_fixed_counters already clamped to the number of contiguous bits
in fixed_cntr_mask?

>  
>  	/*
> -	 * The number of counters comes from guest CPUID data. Clamp the value
> -	 * to avoid a shift-by-64 in BIT_ULL().
> +	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
> +	 * value to the number of contiguous GP counters supported by KVM.
>  	 */
> -	nr_gp_counters = min_t(int, eax.split.num_counters, X86_PMC_IDX_MAX - 1);
> -	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
> -			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
> +	nr_gp_counters = min_t(int, eax.split.num_counters,
> +			       find_first_zero_bit(kvm_pmu_cap.cntr_mask,
> +						   KVM_MAX_NR_GP_COUNTERS));
> +	pmu->pmc_exists64 = BIT_ULL(nr_gp_counters) - 1;
>  
>  	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);

Re: [PATCH v3 10/18] KVM: x86/pmu: Document GLOBAL_STATUS_RESET passthrough erratum

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:19PM -0700, Zide Chen wrote:
> IA32_PERF_GLOBAL_STATUS_RESET, introduced in PerfMon version 4,
> inherits the MSR address and architecturally defined bits from
> IA32_PERF_GLOBAL_OVF_CTRL, while providing additional bits to clear
> new indicators in IA32_PERF_GLOBAL_STATUS.
> 
> From an emulation perspective, KVM handles it identically to the
> OVF_CTRL MSR: reads return zero, and intercepted writes are guarded
> against pmu->global_status_rsvd. No code changes are needed in KVM
> to emulate this MSR.
> 
> However, to reduce VM exits, KVM passes through this MSR in mediated
> PMU mode when GLOBAL_STATUS is also passed through. Consequently,
> writes to bits that are valid on the host but unvirtualized by KVM
> will not trigger a #GP (unlike when the MSR is intercepted). Except
> for the known limitation where host Intel PT in system mode can
> assert Trace_ToPA_PMI (bit 55), unvirtualized status bits are not set
> while the guest is running, so clearing them has no negative effects.
> 
> Document this divergence in errata.rst.
> 
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> ---
> v3:
> - New patch prompted by Jim and Yosry's review on unvirtualized
>   status bits.
> ---
>  Documentation/virt/kvm/x86/errata.rst | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/virt/kvm/x86/errata.rst b/Documentation/virt/kvm/x86/errata.rst
> index a9cf0e004651..161ba6d1beb4 100644
> --- a/Documentation/virt/kvm/x86/errata.rst
> +++ b/Documentation/virt/kvm/x86/errata.rst
> @@ -84,4 +84,12 @@ memory.
>  
>  Note, this erratum does not affect AMD CPUs, which fully virtualize CR0.CD in
>  hardware, i.e. put the CPU caches into "no fill" mode when CR0.CD=1, even when
> -running in the guest.
> \ No newline at end of file
> +running in the guest.
> +
> +IA32_PERF_GLOBAL_STATUS_RESET
> +-----------------------------
> +To reduce VM-exits on performance-critical paths, KVM passes through writes
> +to IA32_PERF_GLOBAL_STATUS_RESET (0x390) in mediated PMU mode when the guest's
> +counters match hardware, without injecting #GP on unvirtualized bits.

Might be worth clarifying that this refers to the number of counters
matching, not the values in the counters.

> +Because the MSR is read-as-zero and write-to-clear, this cannot leak
> +host information to the guest.

It can let the guest clear host state tho, but I assume this isn't a
problem because GLOBAL_STATUS is saved/restored on guest PMU load/put.
Might be worth clarifying.

> -- 
> 2.56.0
> 

Re: [PATCH v3 11/18] KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_STATUS_SET MSR

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:20PM -0700, Zide Chen wrote:
> Intel PerfMon v4 introduces IA32_PERF_GLOBAL_STATUS_SET (0x391) to
> allow software to set individual bits in the global status MSR. Reads
> of IA32_PERF_GLOBAL_STATUS_SET always return zero.
> 
> IA32_PERF_GLOBAL_STATUS_SET is not part of the VM state, so it doesn't
> need to be advertised to userspace, nor saved and restored during live
> migration.
> 
> Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent the guest from
> modifying status bits that are not virtualized by KVM.
> 
> Bit 63 (CondChgd) stays unreserved in pmu->global_status_rsvd to
> architecturally allow the guest to write this bit on STATUS_RESET (0x390)
> to clear it, and allow userspace to set it on GLOBAL_STATUS (0x38E).
> 
> However, bit 63 is architecturally reserved in STATUS_SET per the SDM.
> Reject writes to bit 63, injecting #GP for guest writes. While older
> processors had silicon errata that accepted writes without #GP (e.g.
> SKX95), strictly enforce the architectural specification for
> deterministic behavior across migrations.
> 
> Platform-specific EDS clarifies that CondChgd indicates a change to
> IA32_MISC_ENABLE[7], which KVM intercepts and keeps strictly read-only
> for guests. This guarantees hardware will never assert CondChgd during
> guest execution.
> 
> Thus, the architectural asymmetry among these three MSRs is safe; being
> unable to restore bit 63 to hardware via STATUS_SET is benign because
> the guest will never legitimately observe CondChgd. Filter it out
> defensively before writing to hardware in intel_mediated_pmu_load().
> 
> Originally-by: Yang Weijiang <weijiang.yang@intel.com>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> ---
> v3:
> - Split from combined v2 patch into standalone GLOBAL_STATUS_SET patch.
> - Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent passthrough of
>   unvirtualized status bits. (Jim)
> - Reject writing bit 63 (CondChgd) to IA32_PERF_GLOBAL_STATUS_SET. (Jim)
> - Filter out bit 63 before writing to hardware in intel_mediated_pmu_load().
> - Move validation for MSR_CORE_PERF_GLOBAL_STATUS_SET to pmu_intel.c rather
>   than common pmu.c to avoid exposing the Intel MSR on AMD.
> 
> v2:
> - Add the missing vmcs02 updates for these two MSRs.
> - Change "> 3" to ">= 4" to make the "v4-gated" more obvious and match
>   the existing code style.
> ---
>  arch/x86/kvm/pmu.c           | 13 +++++++++++++
>  arch/x86/kvm/vmx/pmu_intel.c | 14 ++++++++++++--
>  2 files changed, 25 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> index 27d6a9169821..2cb5bccc7044 100644
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -855,6 +855,7 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
>  	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
>  	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
> +	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
>  		msr_info->data = 0;
>  		break;
>  	default:
> @@ -921,6 +922,18 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>  		if (!msr_info->host_initiated)
>  			pmu->global_status &= ~data;
>  		break;
> +	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
> +		/*
> +		 * Bit 63 (CondChgd) is architecturally reserved in STATUS_SET.
> +		 * Older CPUs errantly accepted writes without #GP (e.g. SKX95),
> +		 * but KVM strictly enforces the architectural spec to ensure
> +		 * consistent behavior across migrations.
> +		 */
> +		if (data & (pmu->global_status_rsvd | GLOBAL_STATUS_COND_CHG))
> +			return 1;

Is it intentional to disallow userspace from writing
GLOBAL_STATUS_COND_CHG through MSR_CORE_PERF_GLOBAL_STATUS_SET, even tho
it's allowed through MSR_CORE_PERF_GLOBAL_STATUS?

> +		if (!msr_info->host_initiated)
> +			pmu->global_status |= data;
> +		break;
>  	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
>  		if (!msr_info->host_initiated)
>  			pmu->global_status |= data & ~pmu->global_status_rsvd;
> diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
> index ce668ea08da5..e81a3f8b9b02 100644
> --- a/arch/x86/kvm/vmx/pmu_intel.c
> +++ b/arch/x86/kvm/vmx/pmu_intel.c
> @@ -197,6 +197,8 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
>  	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
>  	case MSR_CORE_PERF_FIXED_CTR_CTRL:
>  		return kvm_pmu_has_perf_global_ctrl(pmu);
> +	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
> +		return pmu->version >= 4;
>  	case MSR_IA32_PEBS_ENABLE:
>  		ret = vcpu_get_perf_capabilities(vcpu) & PERF_CAP_PEBS_FORMAT;
>  		break;
> @@ -827,8 +829,16 @@ static void intel_mediated_pmu_load(struct kvm_vcpu *vcpu)
>  	toggle = pmu->global_status ^ global_status;
>  	if (global_status & toggle)
>  		wrmsrq(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
> -	if (pmu->global_status & toggle)
> -		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
> +
> +	/*
> +	 * CondChgd is architecturally reserved in GLOBAL_STATUS_SET. Unless
> +	 * set by malicious userspace, this bit should never be set in
> +	 * pmu->global_status; filter it out defensively to protect against
> +	 * a host #GP.

IIUC, this will ignore bit 63 completely when written by userspace,
should KVM be doing this?

> +	 */
> +	if ((pmu->global_status & toggle) & ~GLOBAL_STATUS_COND_CHG)

Maybe store this in a variable as it's used in wrmsrq() as well?

> +		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET,
> +		       (pmu->global_status & toggle) & ~GLOBAL_STATUS_COND_CHG);
>  
>  	wrmsrq(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
>  }
> -- 
> 2.56.0
> 

Re: [PATCH v3 13/18] KVM: x86/pmu: Populate CPUID.0AH:ECX fixed-counter bitmap

[Yosry Ahmed]

On Thu, Nov 05, 2026 at 06:39:22PM -0700, Zide Chen wrote:
> PerfMon v5 adds a fixed-counter support bitmap in CPUID.0AH:ECX.
> This is the superset of EDX[4:0] which indicates the contiguous
> counters starting from 0. For backward compatibility, it's recommended
> to use the following to determine if a fixed counter is supported:
> 
> FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i);
> 
> Originally-by: Yang Weijiang <weijiang.yang@intel.com>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
> ---
> v3:
> - Remove '&' when populating entry->ecx. (Jim)
> - Gate entry->ecx on kvm_pmu_cap.version >= 5 in intel_pmu_refresh() to
>   avoid enabling v5 fixed counters on hosts where KVM caps PMU at v2.
> ---
>  arch/x86/kvm/cpuid.c         | 5 ++++-
>  arch/x86/kvm/vmx/pmu_intel.c | 8 ++++++++
>  2 files changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
> index 3310c370c5fa..00a2ca6d3049 100644
> --- a/arch/x86/kvm/cpuid.c
> +++ b/arch/x86/kvm/cpuid.c
> @@ -1554,7 +1554,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
>  
>  		entry->eax = eax.full;
>  		entry->ebx = kvm_pmu_cap.events_mask;
> -		entry->ecx = 0;
> +		if (kvm_pmu_cap.version >= 5)
> +			entry->ecx = (u32)kvm_pmu_cap.fixed_cntr_mask64;
> +		else
> +			entry->ecx = 0;
>  		entry->edx = edx.full;
>  		break;
>  	}
> diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
> index 1372091dc1f8..37b34b588601 100644
> --- a/arch/x86/kvm/vmx/pmu_intel.c
> +++ b/arch/x86/kvm/vmx/pmu_intel.c
> @@ -597,7 +597,15 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
>  				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
>  						      KVM_MAX_NR_FIXED_COUNTERS));
>  
> +	/* FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i) */
>  	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
> +	/*
> +	 * Do not enable fixed counters via the v5 ECX bitmap if KVM caps
> +	 * PMU version below 5 (e.g. when mediated PMU is disabled or on
> +	 * older hosts), matching KVM's advertised CPUID.0AH:ECX capability.
> +	 */
> +	if (kvm_pmu_cap.version >= 5 && pmu->version >= 5)

Having to check both versions is confusing. Can they go out of sync?
Should we clamp pmu->version to kvm_pmu_cap.version and only check the
former?

> +		fixed_mask |= entry->ecx;
>  	fixed_mask &= kvm_pmu_cap.fixed_cntr_mask64;

I see why this is needed now, to re-apply KVM's fixed counter mask after
reading it, but if this is only needed there, it's probably clearer to
do:

	if (version >= 5)
		fixed_mask |= (entry->ecx & kvm_pmu_cap.fixed_cntr_mask64);

Re: [PATCH v3 03/18] KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu

[Chen, Zide]

>> @@ -214,9 +214,15 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
>>  
>>  	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(48) - 1;
>>  
>> -	pmu->reserved_bits = 0xfffffff000280000ull;
>> +	/*
>> +	 * AMD PerfEvtSeln registers do not support PinControl, and KVM does
>> +	 * not emulate AnyThread due to cross-VM information leakage on SMT
>> +	 * cores (as on the Intel side).
>> +	 */
>> +	pmu->eventsel_rsvd = GENMASK_ULL(63, 36) | ARCH_PERFMON_EVENTSEL_PIN_CONTROL |
>> +			     ARCH_PERFMON_EVENTSEL_ANY;
> 
> It's probably better if we diff against the generic pmu->eventsel_rsvd
> set by kvm_pmu_refresh(), and explicitly add/remove bits as needed here,
> like the Intel code. Then we also avoid re-iterating the AnyThread
> rationale here.
> 
> What do you think?


Yes, I think this is better. How about this:

diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
index fb5a298244ec..15d4b94826f8 100644
--- a/arch/x86/kvm/svm/pmu.c
+++ b/arch/x86/kvm/svm/pmu.c
@@ -213,12 +213,11 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
        pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(48) - 1;

/*
-      * AMD PerfEvtSeln registers do not support PinControl, and KVM does
-        * not emulate AnyThread due to cross-VM information leakage on SMT
-        * cores (as on the Intel side).
+        * AMD PerfEvtSeln registers do not support PinControl, but support
+        * a 12-bit EventSelect (bits 35:32 in addition to 7:0).
         */
-       pmu->eventsel_rsvd = GENMASK_ULL(63, 36) |
ARCH_PERFMON_EVENTSEL_PIN_CONTROL |
-                            ARCH_PERFMON_EVENTSEL_ANY;
+       pmu->eventsel_rsvd |= ARCH_PERFMON_EVENTSEL_PIN_CONTROL;
+       pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_EVENT;
        if (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) &&
kvm_vcpu_has_mediated_pmu(vcpu))
                pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;

> 
> 
>>  	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) && kvm_vcpu_has_mediated_pmu(vcpu))
>> -		pmu->reserved_bits &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
>> +		pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;

Re: [PATCH v3 15/18] KVM: x86/pmu: Mask only OS/USR bits when filtering fixed counters

[Jim Mattson]

On Thu, Nov 5, 2026 at 5:50 PM Zide Chen <zide.chen@intel.com> wrote:
>
> In preparation for supporting AnyThread deprecation and Fixed Counter 3,
> mask only INTEL_FIXED_0_KERNEL and INTEL_FIXED_0_USER when updating
> fixed_ctr_ctrl_hw in kvm_mediated_pmu_refresh_event_filter().
>
> Masking the full 0xf nibble would cause two issues:
>
> 1. AnyThread for Fixed Counters 0-2: When a counter is allowed, copying
>    the full nibble from guest-visible pmu->fixed_ctr_ctrl back into
>    hardware-visible pmu->fixed_ctr_ctrl_hw would resurrect AnyThread in
>    hardware, defeating SMT cross-VM isolation.
>
> 2. Metrics Clear for Fixed Counter 3: If an event filter denies Fixed
>    Counter 3, clearing 0xf inadvertently strips METRICS_CLEAR_EN from
>    hardware. If the guest sets METRICS_CLEAR_EN and executes native RDPMC,
>    metrics will remain un-cleared, returning stale data.
>
> This gates hardware counting without clobbering other control bits,
> matching general-purpose counter filtering which only masks the event
> ENABLE bit.
>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>

Re: [PATCH v3 17/18] KVM: x86/pmu: Advertise PerfMon version 5 on Intel hosts

[Jim Mattson]

On Thu, Nov 5, 2026 at 5:50 PM Zide Chen <zide.chen@intel.com> wrote:
>
> From: Dapeng Mi <dapeng1.mi@linux.intel.com>
>
> KVM currently caps the guest PerfMon version at 2 for all Intel
> platforms. Now that KVM emulates the basic architectural PMU features
> introduced in PerfMon versions 3 through 5, raise the guest PerfMon
> version to 5. Features that require additional emulation support, e.g.
> architectural LBR will be enabled separately.
>
> PerfMon versions 3 and 4 continue to be exposed as version 2 because
> KVM cannot safely virtualize AnyThread due to cross-VM information
> leakage. PerfMon v5 is supported because it provides the
> AnyThread-deprecated capability, allowing KVM to indicate that
> AnyThread is deprecated.
>
> When enable_mediated_pmu is disabled or in the non-Intel paths, KVM
> retains the existing cap of version 2.
>
> Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
> Signed-off-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>