Re: [PATCH v2 11/16] KVM: x86/pmu: Emulate the GLOBAL_STATUS_SET and GLOBAL_INUSE MSRs

["Chen, Zide" <zide.chen@intel.com>]

On 11/2/2026 12:32 PM, Jim Mattson wrote:
> On Mon, Nov 2, 2026 at 9:38 AM Chen, Zide <zide.chen@intel.com> wrote:
>>
>>
>>
>> On 10/30/2026 5:13 PM, Yosry Ahmed wrote:
>>>>> @@ -865,6 +867,7 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>>>>>         case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
>>>>>         case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
>>>>>         case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
>>>>> +       case MSR_CORE_PERF_GLOBAL_STATUS_SET:
>>>>>                 msr_info->data = 0;
>>>>>                 break;
>>>>>         default:
>>>>> @@ -931,6 +934,12 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>>>>>                 if (!msr_info->host_initiated)
>>>>>                         pmu->global_status &= ~data;
>>>>>                 break;
>>>>> +       case MSR_CORE_PERF_GLOBAL_STATUS_SET:
>>>>> +               if (data & pmu->global_status_rsvd)
>>>>> +                       return 1;
>>>>> +               if (!msr_info->host_initiated)
>>>>> +                       pmu->global_status |= data;
>>>>> +               break;
>>>>
>>>> According to Intel SDM Vol. 4, Table 2-2 ("IA-32 Architectural MSRs"),
>>>> IA32_PERF_GLOBAL_STATUS.CondChgd[bit 63] cannot be set by writing to
>>>> IA32_PERF_GLOBAL_STATUS_SET, because IA32_PERF_GLOBAL_STATUS_SET[bit 63] is
>>>> reserved.  In other words, it is not possible to save/restore
>>>> IA32_PERF_GLOBAL_STATUS when bit 63 is set.
>>>>
>>>> Because bit 63 is clear in pmu->global_status_rsvd:
>>>>
>>>> 1. An intercepted guest WRMSR(MSR_CORE_PERF_GLOBAL_STATUS_SET, BIT_ULL(63))
>>>>    succeeds instead of injecting #GP(0) and sets bit 63 in
>>>>    pmu->global_status.
>>>> 2. When bit 63 is set in pmu->global_status (whether it was set by
>>>>    kvm_pmu_set_msr() or by intel_mediated_pmu_put()), the next
>>>>    intel_mediated_pmu_load() will try to write an illegal value to
>>>>    MSR_CORE_PERF_GLOBAL_STATUS_SET. I don't think we get a #GP, because
>>>>    every wrmsr is "safe" these days, but the write will be ignored.
>>>
>>> On SPR, it does produce a #GP that gets ignored (and logged) and the bit
>>> is not set.  Interestingly, on CLX, the write goes through and the bit
>>> is set. Didn't check other CPUs.
>>
>> Good catch! I tested a few more CPUs (DMR/CWF/SRF/GNR/NVL/LNL), and all
>> of them #GP on writes that set bit 63.
>>
>> However, SKX, CLX, and ICX accept writes with bit 63 set and do not
>> generate #GP. On those systems, bit 63 also becomes visible when reading
>> back GLOBAL_STATUS. This behavior is documented in their errata:
>>
>> SKX95/CLX89/ICX166. Bit 63 of IA32_PERF_GLOBAL_STATUS_SET MSR (391H) is
>> reserved. Due to this erratum, setting the bit will not result in #GP.
>>
>> SKX:
>> https://cdrdv2-public.intel.com/336065/336065_Intel%20Xeon%20Processor%20Scalable%20Family%20Public%20Specification%20Update_rev17.pdf
>>
>> CLX:
>> https://cdrdv2-public.intel.com/338848/338848_2nd%20Gen%20Intel%C2%AE%20Xeon%C2%AE%20Scalable%20Processors%20Specification%20Update_Rev028US.pdf
>>
>> ICX:
>> https://cdrdv2-public.intel.com/637780/637780_3rd_Gen_Xeon_Scalable_Spec_Update_026US.pdf
>>
>> Given that SKX95/CLX89/ICX166 document this behavior as an erratum,
>> while SPR and newer CPUs appear to follow the SDM and #GP, should KVM
>> continue to inject #GP for writes that set bit 63?
> 
> Ugh. It seems like we have two options, neither of them all that appealing.
> 
> 1. KVM fixes the errata and raises #GP. To ensure consistency, KVM
> always intercepts the MSR on affected CPUs.
> 2. KVM mimics the errata and allows writes to bit 63 when virtualizing
> affected CPUs. This allows the MSR to be passed through, unless an
> unaffected CPU masquerades as an affected one.
> 
> Sean? WDYT?

After looking into this more closely, I think KVM should simply reject
guest writes to bit 63 (CondChgd).

In platform-specific documentation (EDS for SPR, ICX, DMR, PTL, etc.),
this bit is explicitly described as: "Set in case of a change to
IA32_MISC_ENABLES.PERFMON_AVAILABLE (bit 7)", clarifying the SDM
wording "changes to the state of performance monitoring hardware", or
"CondChgd: Status bits of this register have changed.".

This is also consistent with my testing, where I was unable to trigger
CondChgd on the host while exercising other PMU functionality that sets
GLOBAL_STATUS bits.

Since IA32_MISC_ENABLE is intercepted by KVM and is read-only for the
guest, CondChgd should never be set on GLOBAL_STATUS in the guest.

KVM cannot preserve this bit on GLOBAL_STATUS across vPMU context
switches, since it cannot be restored through GLOBAL_STATUS_SET.
That is fine, however, since CondChgd is not set on the guest.

BTW, the history on early CPUs is somewhat messy. As mentioned earlier,
some CPUs have errata that accept writing to bit 63 without #GP
(SKX95, CLX89, CPX89, ICX166, ICL019, etc.). Additionally, the reset
value of this bit is not consistent on some early CPUs, as noted in
commit b292d7a10487 ("perf/x86/intel: ignore CondChgd bit to avoid
false NMI handling").

I'd suggest splitting this into two separate patches: one patch
to emulate each MSR, for clarity.