From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1752356AbdKJKGs (ORCPT <rfc822;w@1wt.eu>);
        Fri, 10 Nov 2017 05:06:48 -0500
Received: from bombadil.infradead.org ([65.50.211.133]:56393 "EHLO
        bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1750835AbdKJKGq (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 10 Nov 2017 05:06:46 -0500
Date: Fri, 10 Nov 2017 11:06:42 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Wanpeng Li <kernellwp@gmail.com>
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        Paolo Bonzini <pbonzini@redhat.com>,
        Radim Kr??m???? <rkrcmar@redhat.com>,
        Wanpeng Li <wanpeng.li@hotmail.com>
Subject: Re: [PATCH v3 2/4] KVM: Add paravirt remote TLB flush
Message-ID: <20171110100642.hmcapsvggzzmbrsb@hirez.programming.kicks-ass.net>
References: <1510307387-14812-1-git-send-email-wanpeng.li@hotmail.com>
 <1510307387-14812-3-git-send-email-wanpeng.li@hotmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1510307387-14812-3-git-send-email-wanpeng.li@hotmail.com>
User-Agent: NeoMutt/20170609 (1.8.3)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Nov 10, 2017 at 01:49:45AM -0800, Wanpeng Li wrote:
> +static DEFINE_PER_CPU(cpumask_var_t, __pv_tlb_mask);
> +
> +static void kvm_flush_tlb_others(const struct cpumask *cpumask,
> +			const struct flush_tlb_info *info)
> +{
> +	u8 state;
> +	int cpu;
> +	struct kvm_steal_time *src;
> +	struct cpumask *flushmask = this_cpu_cpumask_var_ptr(__pv_tlb_mask);
> +
> +	if (unlikely(!flushmask))
> +		return;
> +
> +	cpumask_copy(flushmask, cpumask);
> +	/*
> +	 * We have to call flush only on online vCPUs. And
> +	 * queue flush_on_enter for pre-empted vCPUs
> +	 */
> +	for_each_cpu(cpu, cpumask) {
> +		src = &per_cpu(steal_time, cpu);
> +		state = src->preempted;

I think that wants to be: state = READ_ONCE(src->preempted);

Because without that its possible for state to get re-loaded between the
check here:

> +		if ((state & KVM_VCPU_PREEMPTED)) {

and its use here.

> +			if (cmpxchg(&src->preempted, state, state |
> +				KVM_VCPU_SHOULD_FLUSH) == state)

You can actually write that like:

			if (try_cmpxchg(&src->preempted, state,
					state | KVM_VCPU_SHOULD_FLUSH))

Which should generate ever so slightly better code (it uses the cmpxchg
ZF instead of doing a superfluous compare).

> +				__cpumask_clear_cpu(cpu, flushmask);
> +		}
> +	}
> +
> +	native_flush_tlb_others(flushmask, info);
> +}