[PATCH v3 1/2] KVM: X86: Less kvmclock sync induced vmexits after VM boots

[PATCH v3 1/2] KVM: X86: Less kvmclock sync induced vmexits after VM boots

[Wanpeng Li]

From: Wanpeng Li <wanpengli@tencent.com>

In the progress of vCPUs creation, it queues a kvmclock sync worker to
the global
workqueue before each vCPU creation completes. Each worker will be scheduled
after 300 * HZ delay and request a kvmclock update for all vCPUs and kick them
out. This is especially worse when scaling to large VMs due to a lot of vmexits.
Just one worker as a leader to trigger the kvmclock sync request for
all vCPUs is
enough.

Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
 arch/x86/kvm/x86.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index fb5d64e..d0ba2d4 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -9390,8 +9390,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
     if (!kvmclock_periodic_sync)
         return;

-    schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
-                    KVMCLOCK_SYNC_PERIOD);
+    if (kvm->created_vcpus == 1)
+        schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
+                        KVMCLOCK_SYNC_PERIOD);
 }

 void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
--
2.7.4

Re: [PATCH v3 1/2] KVM: X86: Less kvmclock sync induced vmexits after VM boots

[Wanpeng Li]

On Mon, 17 Feb 2020 at 18:36, Wanpeng Li <kernellwp@gmail.com> wrote:
>
> From: Wanpeng Li <wanpengli@tencent.com>
>
> In the progress of vCPUs creation, it queues a kvmclock sync worker to
> the global
> workqueue before each vCPU creation completes. Each worker will be scheduled
> after 300 * HZ delay and request a kvmclock update for all vCPUs and kick them
> out. This is especially worse when scaling to large VMs due to a lot of vmexits.
> Just one worker as a leader to trigger the kvmclock sync request for
> all vCPUs is
> enough.

Sorry for the alignment.

>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> ---
>  arch/x86/kvm/x86.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index fb5d64e..d0ba2d4 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -9390,8 +9390,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
>      if (!kvmclock_periodic_sync)
>          return;
>
> -    schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> -                    KVMCLOCK_SYNC_PERIOD);
> +    if (kvm->created_vcpus == 1)
> +        schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> +                        KVMCLOCK_SYNC_PERIOD);
>  }
>
>  void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
> --
> 2.7.4

Re: [PATCH v3 1/2] KVM: X86: Less kvmclock sync induced vmexits after VM boots

[Paolo Bonzini]

On 17/02/20 11:36, Wanpeng Li wrote:
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index fb5d64e..d0ba2d4 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -9390,8 +9390,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
>      if (!kvmclock_periodic_sync)
>          return;
> 
> -    schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> -                    KVMCLOCK_SYNC_PERIOD);
> +    if (kvm->created_vcpus == 1)
> +        schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> +                        KVMCLOCK_SYNC_PERIOD);

This is called with kvm->lock not held, so you can have
kvm->created_vcpus == 2 by the time you get here.  You can test instead
"if (vcpu->vcpu_idx == 0)".

Thanks,

Paolo

Re: [PATCH v3 1/2] KVM: X86: Less kvmclock sync induced vmexits after VM boots

[Wanpeng Li]

On Mon, 17 Feb 2020 at 19:23, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 17/02/20 11:36, Wanpeng Li wrote:
> >
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index fb5d64e..d0ba2d4 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -9390,8 +9390,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
> >      if (!kvmclock_periodic_sync)
> >          return;
> >
> > -    schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> > -                    KVMCLOCK_SYNC_PERIOD);
> > +    if (kvm->created_vcpus == 1)
> > +        schedule_delayed_work(&kvm->arch.kvmclock_sync_work,
> > +                        KVMCLOCK_SYNC_PERIOD);
>
> This is called with kvm->lock not held, so you can have
> kvm->created_vcpus == 2 by the time you get here.  You can test instead
> "if (vcpu->vcpu_idx == 0)".

Agreed.

    Wanpeng