mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jon Kohler <jon@nutanix.com>
To: Chao Gao <chao.gao@intel.com>
Cc: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Borislav Petkov <bp@alien8.de>,
	Peter Zijlstra <peterz@infradead.org>,
	Josh Poimboeuf <jpoimboe@kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	X86 ML <x86@kernel.org>, "H. Peter Anvin" <hpa@zytor.com>,
	LKML <linux-kernel@vger.kernel.org>,
	"kvm @ vger . kernel . org" <kvm@vger.kernel.org>
Subject: Re: [PATCH] x86/bhi: avoid hardware mitigation for 'spectre_bhi=vmexit'
Date: Fri, 13 Sep 2024 15:51:01 +0000	[thread overview]
Message-ID: <3244950F-0422-49AD-812D-DE536DAF5D7E@nutanix.com> (raw)
In-Reply-To: <ZuPNmOLJPJsPlufA@intel.com>



> On Sep 13, 2024, at 1:28 AM, Chao Gao <chao.gao@intel.com> wrote:
> 
> !-------------------------------------------------------------------|
>  CAUTION: External Email
> 
> |-------------------------------------------------------------------!
> 
> On Thu, Sep 12, 2024 at 09:24:40AM -0700, Pawan Gupta wrote:
>> On Thu, Sep 12, 2024 at 03:44:38PM +0000, Jon Kohler wrote:
>>>> It is only worth implementing the long sequence in VMEXIT_ONLY mode if it is
>>>> significantly better than toggling the MSR.
>>> 
>>> Thanks for the pointer! I hadn’t seen that second sequence. I’ll do measurements on
>>> three cases and come back with data from an SPR system.
>>> 1. as-is (wrmsr on entry and exit)
>>> 2. Short sequence (as a baseline)
>>> 3. Long sequence
>> 

Pawan,

Thanks for the pointer to the long sequence. I've tested it along with 
Listing 3 (TSX Abort sequence) using KUT tscdeadline_immed test. TSX 
abort sequence performs better unless BHI mitigation is off or 
host/guest spec_ctrl values match, avoiding WRMSR toggling. Having the
values match the DIS_S value is easier said than done across a fleet
that is already using eIBRS heavily.

Test System:
- Intel Xeon Gold 6442Y, microcode 0x2b0005c0
- Linux 6.6.34 + patches, qemu 8.2
- KVM Unit Tests @ latest (17f6f2fd) with tscdeadline_immed + edits:
- Toggle spec ctrl before test in main()
- Use cpu type SapphireRapids-v2

Test string:
TESTNAME=vmexit_tscdeadline_immed TIMEOUT=90s MACHINE= ACCEL= taskset -c 26 ./x86/run x86/vmexit.flat \
-smp 1 -cpu SapphireRapids-v2,+x2apic,+tsc-deadline -append tscdeadline_immed |grep tscdeadline

Test Results:
1. spectre_bhi=on, host spec_ctrl=1025, guest spec_ctrl=1: tscdeadline_immed 3878 (WRMSR toggling)
2. spectre_bhi=on, host spec_ctrl=1025, guest spec_ctrl=1025: tscdeadline_immed 3153 (no WRMSR toggling)
3. spectre_bhi=vmexit, BHB long sequence, host/guest spec_ctrl=1: tscdeadline_immed 3629 (still better than test 1, penalty only on exit)
4. spectre_bhi=vmexit, TSX abort sequence, host/guest spec_ctrl=1: tscdeadline_immed 3294 (best general purpose performance)
5. spectre_bhi=vmexit, TSX abort sequence, host spec_ctrl=1, guest spec_ctrl=1025: tscdeadline_immed 4011 (needs optimization)

In short, there is a significant speedup to be had here.

As for test 5, honest that is somewhat invalid because it would be
dependent on the VMM user space showing BHI_CTRL.

QEMU as an example does not do that, so even with latest qemu and latest
kernel, guests will still use BHB loop even on SPR++ today, and they
could use the TSX loop with this proposed change if the VMM exposes RTM
feature.

I'm happy to post a V2 patch with my TSX changes, or take any other
suggestions here. 

Thanks all,
Jon

>> I wonder if virtual SPEC_CTRL feature introduced in below series can
>> provide speedup, as it can replace the MSR toggling with faster VMCS
>> operations:
> 
> "virtual SPEC_CTRL" won't provide speedup. the wrmsr on entry/exit is still
> need if guest's (effective) value and host's value are different.
> 
> "virtual SPEC_CTRL" just prevents guests from toggling some bits. It doesn't
> switch the MSR between guest value and host value on entry/exit. so, KVM has
> to do the switching with wrmsr/rdmsr instructions. A new feature, "load
> IA32_SPEC_CTRL" VMX control (refer to Chapter 15 in ISE spec[*]), can help but
> it isn't supported on SPR.
> 
> [*]: https://urldefense.proofpoint.com/v2/url?u=https-3A__cdrdv2.intel.com_v1_dl_getContent_671368&d=DwIDaQ&c=s883GpUCOChKOHiocYtGcg&r=NGPRGGo37mQiSXgHKm5rCQ&m=c7SFjczyXeO5McE4firUZaiOVuLBVwLXAzKV9WQqMqKCCEwSvVk0V4cko-falQYo&s=-hskrlhrR4iuT2sz0KkGJn7hCSAGIteu3_TGQzPgh8I&e= 
> 
>> 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_kvm_20240410143446.797262-2D1-2Dchao.gao-40intel.com_&d=DwIDaQ&c=s883GpUCOChKOHiocYtGcg&r=NGPRGGo37mQiSXgHKm5rCQ&m=c7SFjczyXeO5McE4firUZaiOVuLBVwLXAzKV9WQqMqKCCEwSvVk0V4cko-falQYo&s=rsaEdAN9KEjtAMSN-ke4x4R87FgfxsvCsdwbCFk7VOE&e= 
>> 
>> Adding Chao for their opinion.


  reply	other threads:[~2024-09-13 15:51 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-12 14:11 Jon Kohler
2024-09-12 15:14 ` Pawan Gupta
2024-09-12 15:44   ` Jon Kohler
2024-09-12 16:24     ` Pawan Gupta
2024-09-13  5:28       ` Chao Gao
2024-09-13 15:51         ` Jon Kohler [this message]
2024-09-13 17:33           ` Pawan Gupta
2024-09-13 18:01             ` Jon Kohler
2024-09-13 18:39         ` Jim Mattson
2024-09-13 23:04           ` Jim Mattson
2024-09-14  0:16             ` Pawan Gupta
2024-09-14  2:35               ` Jim Mattson
2024-09-13  5:39 ` Chao Gao
2024-09-13 15:52   ` Jon Kohler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3244950F-0422-49AD-812D-DE536DAF5D7E@nutanix.com \
    --to=jon@nutanix.com \
    --cc=bp@alien8.de \
    --cc=chao.gao@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=hpa@zytor.com \
    --cc=jpoimboe@kernel.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=pawan.kumar.gupta@linux.intel.com \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®