Re: [PATCH v3 4/8] KVM: Add a module param to allow enabling virtualization when KVM is loaded

[Paolo Bonzini <pbonzini@redhat.com>]

On 6/8/24 02:06, Sean Christopherson wrote:
> Add an off-by-default module param, enable_virt_at_load, to let userspace
> force virtualization to be enabled in hardware when KVM is initialized,
> i.e. just before /dev/kvm is exposed to userspace.  Enabling virtualization
> during KVM initialization allows userspace to avoid the additional latency
> when creating/destroying the first/last VM.  Now that KVM uses the cpuhp
> framework to do per-CPU enabling, the latency could be non-trivial as the
> cpuhup bringup/teardown is serialized across CPUs, e.g. the latency could
> be problematic for use case that need to spin up VMs quickly.
> 
> Enabling virtualizaton during initialization will also allow KVM to setup
> the Intel TDX Module, which requires VMX to be fully enabled, without
> needing additional APIs to temporarily enable virtualization.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>

I think we should enable it by default and wait for someone to complain. 
  Or notice, even.

Paolo

> ---
>   virt/kvm/kvm_main.c | 37 +++++++++++++++++++++++++++++++++++++
>   1 file changed, 37 insertions(+)
> 
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 98e52d12f137..7bdd744e4821 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -5495,6 +5495,9 @@ static struct miscdevice kvm_dev = {
>   };
>   
>   #ifdef CONFIG_KVM_GENERIC_HARDWARE_ENABLING
> +static bool enable_virt_at_load;
> +module_param(enable_virt_at_load, bool, 0444);
> +
>   __visible bool kvm_rebooting;
>   EXPORT_SYMBOL_GPL(kvm_rebooting);
>   
> @@ -5645,15 +5648,41 @@ static void kvm_disable_virtualization(void)
>   	unregister_syscore_ops(&kvm_syscore_ops);
>   	cpuhp_remove_state(CPUHP_AP_KVM_ONLINE);
>   }
> +
> +static int kvm_init_virtualization(void)
> +{
> +	if (enable_virt_at_load)
> +		return kvm_enable_virtualization();
> +
> +	return 0;
> +}
> +
> +static void kvm_uninit_virtualization(void)
> +{
> +	if (enable_virt_at_load)
> +		kvm_disable_virtualization();
> +
> +	WARN_ON(kvm_usage_count);
> +}
>   #else /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */
>   static int kvm_enable_virtualization(void)
>   {
>   	return 0;
>   }
>   
> +static int kvm_init_virtualization(void)
> +{
> +	return 0;
> +}
> +
>   static void kvm_disable_virtualization(void)
>   {
>   
> +}
> +
> +static void kvm_uninit_virtualization(void)
> +{
> +
>   }
>   #endif /* CONFIG_KVM_GENERIC_HARDWARE_ENABLING */
>   
> @@ -6395,6 +6424,10 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module)
>   
>   	kvm_gmem_init(module);
>   
> +	r = kvm_init_virtualization();
> +	if (r)
> +		goto err_virt;
> +
>   	/*
>   	 * Registration _must_ be the very last thing done, as this exposes
>   	 * /dev/kvm to userspace, i.e. all infrastructure must be setup!
> @@ -6408,6 +6441,8 @@ int kvm_init(unsigned vcpu_size, unsigned vcpu_align, struct module *module)
>   	return 0;
>   
>   err_register:
> +	kvm_uninit_virtualization();
> +err_virt:
>   	kvm_vfio_ops_exit();
>   err_vfio:
>   	kvm_async_pf_deinit();
> @@ -6433,6 +6468,8 @@ void kvm_exit(void)
>   	 */
>   	misc_deregister(&kvm_dev);
>   
> +	kvm_uninit_virtualization();
> +
>   	debugfs_remove_recursive(kvm_debugfs_dir);
>   	for_each_possible_cpu(cpu)
>   		free_cpumask_var(per_cpu(cpu_kick_mask, cpu));