From: "Gabriel L. Somlo" <gsomlo@gmail.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: "Radim Krčmář" <rkrcmar@redhat.com>,
linux-kernel@vger.kernel.org,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"H. Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org
Subject: Re: [PATCH] kvm: better MWAIT emulation for guests
Date: Wed, 15 Mar 2017 15:43:03 -0400 [thread overview]
Message-ID: <20170315194302.GC2239@HEDWIG.INI.CMU.EDU> (raw)
In-Reply-To: <20170315212906-mutt-send-email-mst@kernel.org>
Applies cleanly over git://git.kernel.org/pub/scm/virt/kvm/kvm.git,
but then I get:
CC [M] arch/x86/kvm/x86.o
In file included from arch/x86/kvm/x86.c:28:0:
arch/x86/kvm/x86.h: In function ‘kvm_mwait_in_guest’:
arch/x86/kvm/x86.h:231:34: error: ‘CPUID_MWAIT_LEAF’ undeclared (first use in this function)
if (boot_cpu_data.cpuid_level < CPUID_MWAIT_LEAF)
^
arch/x86/kvm/x86.h:231:34: note: each undeclared identifier is reported only once for each function it appears in
arch/x86/kvm/x86.h:234:45: error: ‘mwait_substates’ undeclared (first use in this function)
cpuid(CPUID_MWAIT_LEAF, &eax, &ebx, &ecx, &mwait_substates);
^
arch/x86/kvm/x86.h:236:14: error: ‘CPUID5_ECX_INTERRUPT_BREAK’ undeclared (first use in this function)
if (!(ecx & CPUID5_ECX_INTERRUPT_BREAK))
^
arch/x86/kvm/x86.h:238:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
scripts/Makefile.build:294: recipe for target 'arch/x86/kvm/x86.o' failed
make[2]: *** [arch/x86/kvm/x86.o] Error 1
scripts/Makefile.build:553: recipe for target 'arch/x86/kvm' failed
make[1]: *** [arch/x86/kvm] Error 2
Makefile:1002: recipe for target 'arch/x86' failed
make: *** [arch/x86] Error 2
Did you accidentally leave out something that went into a .h file
somewhere ?
Thx,
--G
On Wed, Mar 15, 2017 at 09:29:57PM +0200, Michael S. Tsirkin wrote:
> On Wed, Mar 15, 2017 at 03:01:12PM -0400, Gabriel L. Somlo wrote:
> > On Wed, Mar 15, 2017 at 08:29:23PM +0200, Michael S. Tsirkin wrote:
> > > On Wed, Mar 15, 2017 at 02:14:26PM -0400, Gabriel L. Somlo wrote:
> > > > Michael,
> > > >
> > > > I tested this on OS X 10.7 (Lion), the last version that doesn't check
> > > > CPUID for MWAIT support.
> > > >
> > > > I used the latest kvm from git://git.kernel.org/pub/scm/virt/kvm/kvm.git
> > > > first as-is, then with your v2 MWAIT patch applied.
> > > >
> > > > Single-(V)CPU guest works as expected (but then again, single-vcpu
> > > > guests worked even back when I tried emulating MWAIT the same as HLT).
> > > >
> > > > When I try starting a SMP guest (with "-smp 4,cores=2"), the guest OS
> > > > hangs after generating some output in text/verbose boot mode -- I gave
> > > > up waiting for it after about 5 minutes. Works fine before your patch,
> > > > which leads me to suspect that, as I feared, MWAIT doesn't wake
> > > > immediately upon another VCPU writing to the MONITOR-ed memory location.
> > > >
> > > > Tangentially, I remember back in the days of OS X 10.7, the
> > > > alternative to exiting guest mode and emulating MWAIT and MONITOR as
> > > > NOPs was to allow them both to run in guest mode.
> > > >
> > > > While poorly documented by Intel at the time, MWAIT at L>0 effectively
> > > > behaves as a NOP (i.e., doesn't actually put the physical core into
> > > > low-power mode, because doing that would allow a guest to effectively
> > > > DOS the host hardware).
> > >
> > > Thanks for the testing, interesting.
> > > Testing with Linux guest seems to show it works.
> > > This could be an interrupt thing not a monitor thing.
> > > Question: does your host CPU have this in its MWAIT leaf?
> > > Bit 01: Supports treating interrupts as break-event for MWAIT, even when interrupts disabled
> >
> > How would I check for this (I'm sorry, haven't hacked on any KVM
> > related thing in a while, so I don't have it "cached") :)
> >
> > >
> > > We really should check that before enabling,
> > > I'll add that.
> > >
> > > >
> > > > Given how unusual it is for a guest to use MONITOR/MWAIT in the first
> > > > place, what's wrong with leaving it all as is (i.e., emulated as NOP)?
> > > >
> > >
> > > I'm really looking into ways to use mwait within Linux guests,
> > > this is just a building block that should help Mac OSX
> > > as a side effect (and we do not want it broken if at all possible).
> >
> > A few years ago I tried really emulating MONITOR and MWAIT for a
> > project -- while not a total abject failure, the resulting patch
> > worked only intermittently (on OS X 10.7, which was the hot new thing
> > at the time, and hadn't started checking CPUID yet).
> >
> > My collected wisdom on the topic from back then is here:
> >
> > http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/mwait.html
> >
> > The problem is that MWAIT is required to wake synchronously with
> > any other "thing" (either another (v)CPU, or DMA, or whatever) writing
> > to the memory location "marked" by the last preceding MONITOR. While
> > interrupts of any kind may also wake an MWAIT, it is strictly not allowed
> > to "miss" a write to the MONITOR-ed memory location. So unless we implement
> > some sort of condition queue that guarantees re-enabling the "parked" vcpu
> > on an intercepted write to a specific memory location by another vcpu,
> > we can't guarantee architecturally correct behavior.
> >
> > If linux uses it in a very specific way that can be "faked" even
> > without ISA compliance, that's OK with me -- but other guest OSs might
> > take the x86 ISA more literally :)
> >
> > Let me know if there's anything else you'd like me to test, now that I
> > have set up a 4.11.0-rc2+ (a.k.a. kvm git master) testing rig...
> >
> > Regards,
> > --Gabe
>
> Doing that corrently in software would be very hard.
> I suspect your host CPU has an issue, sent a patch to
> detect that. Let's see what happens.
>
> --
> MST
next prev parent reply other threads:[~2017-03-15 19:43 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-09 22:29 Michael S. Tsirkin
2017-03-10 0:51 ` Gabriel L. Somlo
2017-03-10 1:12 ` Michael S. Tsirkin
2017-03-13 7:44 ` Wanpeng Li
2017-03-10 23:46 ` Jim Mattson
2017-03-12 0:01 ` Michael S. Tsirkin
2017-03-12 21:18 ` Gabriel L. Somlo
2017-03-13 15:46 ` Radim Krčmář
2017-03-13 16:08 ` Michael S. Tsirkin
2017-03-13 19:39 ` Radim Krčmář
2017-03-13 20:03 ` Michael S. Tsirkin
2017-03-13 21:43 ` Radim Krčmář
2017-03-15 18:14 ` Gabriel L. Somlo
2017-03-15 18:29 ` Michael S. Tsirkin
2017-03-15 19:01 ` Gabriel L. Somlo
2017-03-15 19:05 ` Michael S. Tsirkin
2017-03-15 19:29 ` Michael S. Tsirkin
2017-03-15 19:43 ` Gabriel L. Somlo [this message]
2017-03-15 20:13 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170315194302.GC2239@HEDWIG.INI.CMU.EDU \
--to=gsomlo@gmail.com \
--cc=corbet@lwn.net \
--cc=hpa@zytor.com \
--cc=kvm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®