Re: [PATCH] x86/bhi: avoid hardware mitigation for 'spectre_bhi=vmexit'

[Jon Kohler <jon@nutanix.com>]

> On Sep 12, 2024, at 11:14 AM, Pawan Gupta <pawan.kumar.gupta@linux.intel.com> wrote:
> 
> !-------------------------------------------------------------------|
>  CAUTION: External Email
> 
> |-------------------------------------------------------------------!
> 
> On Thu, Sep 12, 2024 at 07:11:56AM -0700, Jon Kohler wrote:
>> On hardware that supports BHI_DIS_S/X86_FEATURE_BHI_CTRL, do not use
>> hardware mitigation when using BHI_MITIGATION_VMEXIT_ONLY, as this
>> causes the value of MSR_IA32_SPEC_CTRL to change, which inflicts
>> additional KVM overhead.
>> 
>> Example: In a typical eIBRS enabled system, such as Intel SPR, the
>> SPEC_CTRL may be commonly set to val == 1 to reflect eIBRS enablement;
>> however, SPEC_CTRL_BHI_DIS_S causes val == 1025. If the guests that
>> KVM is virtualizing do not also set the guest side value == 1025,
>> KVM will constantly have to wrmsr toggle the guest vs host value on
>> both entry and exit, delaying both.
>> 
>> Signed-off-by: Jon Kohler <jon@nutanix.com>
>> ---
>> arch/x86/kernel/cpu/bugs.c | 12 ++++++++++--
>> 1 file changed, 10 insertions(+), 2 deletions(-)
>> 
>> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
>> index 45675da354f3..df7535f5e882 100644
>> --- a/arch/x86/kernel/cpu/bugs.c
>> +++ b/arch/x86/kernel/cpu/bugs.c
>> @@ -1662,8 +1662,16 @@ static void __init bhi_select_mitigation(void)
>> return;
>> }
>> 
>> - /* Mitigate in hardware if supported */
>> - if (spec_ctrl_bhi_dis())
>> + /*
>> +  * Mitigate in hardware if appropriate.
>> +  * Note: for vmexit only, do not mitigate in hardware to avoid changing
>> +  * the value of MSR_IA32_SPEC_CTRL to include SPEC_CTRL_BHI_DIS_S. If a
>> +  * guest does not also set their own SPEC_CTRL to include this, KVM has
>> +  * to toggle on every vmexit and vmentry if the host value does not
>> +  * match the guest value. Instead, depend on software loop mitigation
>> +  * only.
>> +  */
>> + if (bhi_mitigation != BHI_MITIGATION_VMEXIT_ONLY && spec_ctrl_bhi_dis())
>> return;
> 
> This makes the system vulnerable. The current software loop is not
> effective on parts that support BHI_DIS_S. There is a separate loop for
> SPR, see Listing 2(long sequence) in Software BHB-clearing sequence
> section here:
> 
>  https://urldefense.proofpoint.com/v2/url?u=https-3A__www.intel.com_content_www_us_en_developer_articles_technical_software-2Dsecurity-2Dguidance_technical-2Ddocumentation_branch-2Dhistory-2Dinjection.html&d=DwIBAg&c=s883GpUCOChKOHiocYtGcg&r=NGPRGGo37mQiSXgHKm5rCQ&m=65GTWhwjPejAvs_QaMrB6pw1KqVI5zZUx6r6hsaSoPHDDulpwh96Q7zedYS6CGoT&s=bQ8X4mjfOPUZgWUx7YQA4rkDADXoOX2-pmn-65xPL9g&e=
> 
> It is only worth implementing the long sequence in VMEXIT_ONLY mode if it is
> significantly better than toggling the MSR.

Thanks for the pointer! I hadn’t seen that second sequence. I’ll do measurements on
three cases and come back with data from an SPR system.
1. as-is (wrmsr on entry and exit)
2. Short sequence (as a baseline)
3. Long sequence