From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933323AbdKOVFu (ORCPT ); Wed, 15 Nov 2017 16:05:50 -0500 Received: from aserp1040.oracle.com ([141.146.126.69]:21950 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757904AbdKOVFm (ORCPT ); Wed, 15 Nov 2017 16:05:42 -0500 Date: Wed, 15 Nov 2017 16:05:02 -0500 From: Konrad Rzeszutek Wilk To: Wanpeng Li Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Paolo Bonzini , Radim =?utf-8?B?S3LEjW3DocWZ?= , Peter Zijlstra , Wanpeng Li Subject: Re: [PATCH v5 0/4] KVM: X86: Paravirt remote TLB flush Message-ID: <20171115210502.GX21113@char.us.oracle.com> References: <1510567280-19376-1-git-send-email-wanpeng.li@hotmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1510567280-19376-1-git-send-email-wanpeng.li@hotmail.com> User-Agent: Mutt/1.8.3 (2017-05-23) X-Source-IP: aserv0022.oracle.com [141.146.126.234] Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Nov 13, 2017 at 02:01:16AM -0800, Wanpeng Li wrote: > Remote flushing api's does a busy wait which is fine in bare-metal > scenario. But with-in the guest, the vcpus might have been pre-empted > or blocked. In this scenario, the initator vcpu would end up > busy-waiting for a long amount of time. > > This patch set implements para-virt flush tlbs making sure that it > does not wait for vcpus that are sleeping. And all the sleeping vcpus > flush the tlb on guest enter. Idea was discussed here: > https://lkml.org/lkml/2012/2/20/157 > > The best result is achieved when we're overcommiting the host by running > multiple vCPUs on each pCPU. In this case PV tlb flush avoids touching > vCPUs which are not scheduled and avoid the wait on the main CPU. > > In addition, thanks for commit 9e52fc2b50d ("x86/mm: Enable RCU based > page table freeing (CONFIG_HAVE_RCU_TABLE_FREE=y)") > > Test on a Haswell i7 desktop 4 cores (2HT), so 8 pCPUs, running ebizzy > in one linux guest. 8 pCPUS? > > ebizzy -M > vanilla optimized boost > 8 vCPUs 10152 10083 -0.68% > 16 vCPUs 1224 4866 297.5% > 24 vCPUs 1109 3871 249% > 32 vCPUs 1025 3375 229.3% so this is all just one guest? What happens if you have say a 64pCPU machine with eight of these guests? That is more of a realistic workload in todays cloud situations. > > Note: The patchset is rebased against "locking/qspinlock/x86: Avoid > test-and-set when PV_DEDICATED is set" v3 > > v4 -> v5: > * flushmask instead of cpumask > > v3 -> v4: > * use READ_ONCE() > * use try_cmpxchg instead of cmpxchg > * add {} to if > * no FLUSH flags to preserve during set_preempted > * "KVM: X86" prefix to patch subject > > v2 -> v3: > * percpu cpumask > > v1 -> v2: > * a new CPUID feature bit > * fix cmpxchg check > * use kvm_vcpu_flush_tlb() to get the statistics right > * just OR the KVM_VCPU_PREEMPTED in kvm_steal_time_set_preempted > * add a new bool argument to kvm_x86_ops->tlb_flush > * __cpumask_clear_cpu() instead of cpumask_clear_cpu() > * not put cpumask_t on stack > * rebase the patchset against "locking/qspinlock/x86: Avoid > test-and-set when PV_DEDICATED is set" v3 > > Wanpeng Li (4): > KVM: X86: Add vCPU running/preempted state > KVM: X86: Add paravirt remote TLB flush > KVM: X86: introduce invalidate_gpa argument to tlb flush > KVM: X86: Add flush_on_enter before guest enter > > Documentation/virtual/kvm/cpuid.txt | 4 ++++ > arch/x86/include/asm/kvm_host.h | 2 +- > arch/x86/include/uapi/asm/kvm_para.h | 6 +++++ > arch/x86/kernel/kvm.c | 46 ++++++++++++++++++++++++++++++++++-- > arch/x86/kvm/cpuid.c | 3 ++- > arch/x86/kvm/svm.c | 14 +++++------ > arch/x86/kvm/vmx.c | 21 ++++++++-------- > arch/x86/kvm/x86.c | 25 +++++++++++++------- > 8 files changed, 88 insertions(+), 30 deletions(-) > > -- > 2.7.4 >