From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DC6EC433E2 for ; Fri, 19 Mar 2021 23:21:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CFBA66198C for ; Fri, 19 Mar 2021 23:21:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229770AbhCSXUc (ORCPT ); Fri, 19 Mar 2021 19:20:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229519AbhCSXUO (ORCPT ); Fri, 19 Mar 2021 19:20:14 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34D4CC061760 for ; Fri, 19 Mar 2021 16:20:14 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id v92so10579349ybi.12 for ; Fri, 19 Mar 2021 16:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=uevn87Pv71vAHsLA8sEmHJbxFzL55foju3lqBq/Tdz0=; b=pIfaTxKK9qfTJHolePPDEnkfaVQ2UiK7abdoXt2KyIbgfIEyPpeTraTeT7elGt3eLN 3xpHkNwtNfS00VbpOiEYo3zjsx+3+3GKMiU6bXvQrvB1Vjif53jAySjo4q696UVb8UtQ bkCABuJHWahi+EsYuJxXDG/MZWEgN4hGqadJtQ4w/eapMnBJ+v0MS1/Xc2i4YCZm7MUw Y/FGK3PwSDTmUogUH8hxcW3iL40fo/CFpKP7kbKTNco7ZsOKEjhERxQ7tyu5uzGXLQaZ NAu5l1tyTyRIzb44xTvlTIFizInm2maD+wRkb31oyTmS/y8ro/gO6OgcxiIbsgak90Or J0bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=uevn87Pv71vAHsLA8sEmHJbxFzL55foju3lqBq/Tdz0=; b=qkel6ukC5v2DSvdfmhA06TQukSLrH8fxA7+sxpnYBt8ENsHPshrNASOELCQbol2Als tN8/o4exAtDvC3AjAOPYzkjdh+REnDk6s3+Nj+nfeQmAvrz64Z0HoahQKU6nd8H40pQF Brxj0ngK9TDlzkOBIWhr7L0htf3w/UtHmYt5HKWkPMWEqijjec/DC+mrxvC+BxP2IKzu oXWKwpNw61o6S4vKHJ1sBVu1+0U2as/f2o1VPUXUjR8IjEyRR8IgvW9tgrSQC4BfddNg 48JVgIA/8zmg3EHgpjAdM/hEBpS9s17d6GvJwUm1ocVXnKkTkVlNDyp/5sUrcTuOI1Mr DaHQ== X-Gm-Message-State: AOAM531KlKofrzxuadECwMlDDmOVZN5iElUYkY4Y66uSJk/ObFtX69Fq ZwyRwdBHwqA1tcjlhkkWAurz++MxpeA= X-Google-Smtp-Source: ABdhPJyh2/iZ1TrtSEuUbyTus519ry6wuAyLAZMiJUykWEfMw7axv6kMUwgAm1cT9PmRWK2yOPSizBgqSZo= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:f:10:115a:eb6e:42f6:f9d5]) (user=seanjc job=sendgmr) by 2002:a25:880c:: with SMTP id c12mr9709753ybl.399.1616196013487; Fri, 19 Mar 2021 16:20:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 19 Mar 2021 16:20:05 -0700 In-Reply-To: <20210319232006.3468382-1-seanjc@google.com> Message-Id: <20210319232006.3468382-2-seanjc@google.com> Mime-Version: 1.0 References: <20210319232006.3468382-1-seanjc@google.com> X-Mailer: git-send-email 2.31.0.rc2.261.g7f71774620-goog Subject: [PATCH 1/2] KVM: x86/mmu: Ensure TLBs are flushed when yielding during GFN range zap From: Sean Christopherson To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ben Gardon Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When flushing a range of GFNs across multiple roots, ensure any pending flush from a previous root is honored before yielding while walking the tables of the current root. Note, kvm_tdp_mmu_zap_gfn_range() now intentionally overwrites it local "flush" with the result to avoid redundant flushes. zap_gfn_range() preserves and return the incoming "flush", unless of course the flush was performed prior to yielding and no new flush was triggered. Fixes: 1af4a96025b3 ("KVM: x86/mmu: Yield in TDU MMU iter even if no SPTES changed") Cc: stable@vger.kernel.org Cc: Ben Gardon Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/tdp_mmu.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index f0c99fa04ef2..6cf08c3c537f 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -86,7 +86,7 @@ static inline struct kvm_mmu_page *tdp_mmu_next_root(struct kvm *kvm, list_for_each_entry(_root, &_kvm->arch.tdp_mmu_roots, link) static bool zap_gfn_range(struct kvm *kvm, struct kvm_mmu_page *root, - gfn_t start, gfn_t end, bool can_yield); + gfn_t start, gfn_t end, bool can_yield, bool flush); void kvm_tdp_mmu_free_root(struct kvm *kvm, struct kvm_mmu_page *root) { @@ -99,7 +99,7 @@ void kvm_tdp_mmu_free_root(struct kvm *kvm, struct kvm_mmu_page *root) list_del(&root->link); - zap_gfn_range(kvm, root, 0, max_gfn, false); + zap_gfn_range(kvm, root, 0, max_gfn, false, false); free_page((unsigned long)root->spt); kmem_cache_free(mmu_page_header_cache, root); @@ -664,20 +664,21 @@ static inline bool tdp_mmu_iter_cond_resched(struct kvm *kvm, * scheduler needs the CPU or there is contention on the MMU lock. If this * function cannot yield, it will not release the MMU lock or reschedule and * the caller must ensure it does not supply too large a GFN range, or the - * operation can cause a soft lockup. + * operation can cause a soft lockup. Note, in some use cases a flush may be + * required by prior actions. Ensure the pending flush is performed prior to + * yielding. */ static bool zap_gfn_range(struct kvm *kvm, struct kvm_mmu_page *root, - gfn_t start, gfn_t end, bool can_yield) + gfn_t start, gfn_t end, bool can_yield, bool flush) { struct tdp_iter iter; - bool flush_needed = false; rcu_read_lock(); tdp_root_for_each_pte(iter, root, start, end) { if (can_yield && - tdp_mmu_iter_cond_resched(kvm, &iter, flush_needed)) { - flush_needed = false; + tdp_mmu_iter_cond_resched(kvm, &iter, flush)) { + flush = false; continue; } @@ -695,11 +696,11 @@ static bool zap_gfn_range(struct kvm *kvm, struct kvm_mmu_page *root, continue; tdp_mmu_set_spte(kvm, &iter, 0); - flush_needed = true; + flush = true; } rcu_read_unlock(); - return flush_needed; + return flush; } /* @@ -714,7 +715,7 @@ bool kvm_tdp_mmu_zap_gfn_range(struct kvm *kvm, gfn_t start, gfn_t end) bool flush = false; for_each_tdp_mmu_root_yield_safe(kvm, root) - flush |= zap_gfn_range(kvm, root, start, end, true); + flush = zap_gfn_range(kvm, root, start, end, true, flush); return flush; } @@ -931,7 +932,7 @@ static int zap_gfn_range_hva_wrapper(struct kvm *kvm, struct kvm_mmu_page *root, gfn_t start, gfn_t end, unsigned long unused) { - return zap_gfn_range(kvm, root, start, end, false); + return zap_gfn_range(kvm, root, start, end, false, false); } int kvm_tdp_mmu_zap_hva_range(struct kvm *kvm, unsigned long start, -- 2.31.0.rc2.261.g7f71774620-goog