From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E7BCC43216 for ; Fri, 27 Aug 2021 00:57:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2A3F9610A6 for ; Fri, 27 Aug 2021 00:57:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244011AbhH0A6k (ORCPT ); Thu, 26 Aug 2021 20:58:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243898AbhH0A6c (ORCPT ); Thu, 26 Aug 2021 20:58:32 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DADE2C0613A3 for ; Thu, 26 Aug 2021 17:57:43 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id q80-20020a25d953000000b0059a45a5f834so4909627ybg.22 for ; Thu, 26 Aug 2021 17:57:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=reply-to:date:in-reply-to:message-id:mime-version:references :subject:from:to:cc; bh=BKACFpkyclmSNZOzZCvoQ85Q5j0ibf+pUktwZSnA2VE=; b=uDGNAKy3X8wgG4FPiD0TolBumhb9mvdkmZ5pXk9jMiSPuEv6IzBrAY2yREw0PMXzi4 9KSpq/MIGbNRkWH+90XdYBr08BMNemkexcswrPOoxwAVc9eNlnrdscRQYZv1qWj5RVJQ Uu5cGBTR2dLrVmB+fbyQ0f2saIhD4/u6CJuZoKrKwJgASrh55N/fvDtm72qpzzHlRHHS 5MsttnXx7/8NWuUSXTy9DIxqvoIYMeSt7SrSLZoZYq6Cz9pNjweGfVRu94/2EucLJNq+ age0a4QpYT7qPSqnVfS8AXaoz7sVZQi9saQiXGh1vd9lEsvZMwvUn3vvO8JTj1WmcE1u xE7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:date:in-reply-to:message-id :mime-version:references:subject:from:to:cc; bh=BKACFpkyclmSNZOzZCvoQ85Q5j0ibf+pUktwZSnA2VE=; b=BGM52/VgXCGx4r7rXC4Ms0cXxzJUZN1JGlUXSAimuQzh8CxTRf/YnneVco7Ek+L+vn iD4fuxjo17BbixNHfbC+lNkydQLYLkEj3lxT2P9t6WCojzVq3sAZocmnJlnAwVzIZaRx gdRl+wlGpizVpd0eb4rrg1+c4kTEPkePR/rFM2CvQVpjH8QEmssC4OqFFgJ3vIm8SixY IKcHo5rztiD46Viau5/1mSUeFxSvY1QosYxykG3yzItagVrUiBGuBOvl14Ujs/ZCUeQ9 ycwU6S5Sx94t4tMJg4EFl3Ry25I5CH2niJnTapI9VeiYo3NNucVhytbKA41ORuvqw6TQ cH+w== X-Gm-Message-State: AOAM533V5NZ2A3NNfs7/vjHAnXBuKs9F+uDH5YP0XqFea+rT8mgU8x0L NmtvFcQ2kGpmiyUkxoukhCM2x496UAg= X-Google-Smtp-Source: ABdhPJw0TCNYmfMyjk3pp1TIQWtZYYXC1+CmXfKBRCTWftHhXnIexf6rTbOO9IR4Dc7O+dyoa9Bys+IIj5k= X-Received: from seanjc798194.pdx.corp.google.com ([2620:15c:90:200:c16c:db05:96b2:1475]) (user=seanjc job=sendgmr) by 2002:a25:4f8a:: with SMTP id d132mr1902818ybb.486.1630025863019; Thu, 26 Aug 2021 17:57:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 26 Aug 2021 17:57:09 -0700 In-Reply-To: <20210827005718.585190-1-seanjc@google.com> Message-Id: <20210827005718.585190-7-seanjc@google.com> Mime-Version: 1.0 References: <20210827005718.585190-1-seanjc@google.com> X-Mailer: git-send-email 2.33.0.259.gc128427fd7-goog Subject: [PATCH 06/15] KVM: x86: Register perf callbacks only when actively handling interrupt From: Sean Christopherson To: Will Deacon , Mark Rutland , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Catalin Marinas , Marc Zyngier , Guo Ren , Nick Hu , Greentime Hu , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Borislav Petkov , x86@kernel.org, Paolo Bonzini , Boris Ostrovsky , Juergen Gross Cc: Alexander Shishkin , Jiri Olsa , Namhyung Kim , James Morse , Alexandru Elisei , Suzuki K Poulose , "H. Peter Anvin" , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Stefano Stabellini , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-csky@vger.kernel.org, linux-riscv@lists.infradead.org, kvm@vger.kernel.org, xen-devel@lists.xenproject.org, Artem Kashkanov , Like Xu , Zhu Lingshan Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Register KVM's perf callback only when handling an interrupt that may be a PMI (sadly this includes IRQs), and unregister the callback immediately after handling the interrupt (or closing the window). Registering the callback on a per-CPU basis (with preemption disabled!), fixes a mostly theoretical bug where perf could dereference a NULL pointer due to KVM unloading and unregistering the callbacks in between perf queries of the callback functions. The precise registration will also allow for future cleanups and optimizations, e.g. the existence of the callbacks can serve as the "in guest" check. Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 27 +++++++++++++++++---------- arch/x86/kvm/x86.h | 10 ++++++++++ include/linux/perf_event.h | 2 ++ kernel/events/core.c | 12 ++++++++++++ 4 files changed, 41 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bae951344e28..bc4ee6ea7752 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8274,28 +8274,31 @@ int kvm_is_in_guest(void) static int kvm_is_user_mode(void) { - int user_mode = 3; + struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); - if (__this_cpu_read(current_vcpu)) - user_mode = static_call(kvm_x86_get_cpl)(__this_cpu_read(current_vcpu)); + if (WARN_ON_ONCE(!vcpu)) + return 0; - return user_mode != 0; + return static_call(kvm_x86_get_cpl)(vcpu) != 0; } static unsigned long kvm_get_guest_ip(void) { - unsigned long ip = 0; + struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); - if (__this_cpu_read(current_vcpu)) - ip = kvm_rip_read(__this_cpu_read(current_vcpu)); + if (WARN_ON_ONCE(!vcpu)) + return 0; - return ip; + return kvm_rip_read(vcpu); } static void kvm_handle_intel_pt_intr(void) { struct kvm_vcpu *vcpu = __this_cpu_read(current_vcpu); + if (WARN_ON_ONCE(!vcpu)) + return; + kvm_make_request(KVM_REQ_PMI, vcpu); __set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT, (unsigned long *)&vcpu->arch.pmu.global_status); @@ -8308,6 +8311,12 @@ static struct perf_guest_info_callbacks kvm_guest_cbs = { .handle_intel_pt_intr = NULL, }; +void kvm_register_perf_callbacks(void) +{ + __perf_register_guest_info_callbacks(&kvm_guest_cbs); +} +EXPORT_SYMBOL_GPL(kvm_register_perf_callbacks); + #ifdef CONFIG_X86_64 static void pvclock_gtod_update_fn(struct work_struct *work) { @@ -11063,7 +11072,6 @@ int kvm_arch_hardware_setup(void *opaque) if (ops->intel_pt_intr_in_guest && ops->intel_pt_intr_in_guest()) kvm_guest_cbs.handle_intel_pt_intr = kvm_handle_intel_pt_intr; - perf_register_guest_info_callbacks(&kvm_guest_cbs); if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES)) supported_xss = 0; @@ -11092,7 +11100,6 @@ int kvm_arch_hardware_setup(void *opaque) void kvm_arch_hardware_unsetup(void) { - perf_unregister_guest_info_callbacks(); kvm_guest_cbs.handle_intel_pt_intr = NULL; static_call(kvm_x86_hardware_unsetup)(); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7d66d63dc55a..5cedc0e8a5d5 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -387,15 +387,25 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm) return kvm->arch.cstate_in_guest; } +void kvm_register_perf_callbacks(void); +static inline void kvm_unregister_perf_callbacks(void) +{ + __perf_unregister_guest_info_callbacks(); +} + DECLARE_PER_CPU(struct kvm_vcpu *, current_vcpu); static inline void kvm_before_interrupt(struct kvm_vcpu *vcpu) { __this_cpu_write(current_vcpu, vcpu); + + kvm_register_perf_callbacks(); } static inline void kvm_after_interrupt(struct kvm_vcpu *vcpu) { + kvm_unregister_perf_callbacks(); + __this_cpu_write(current_vcpu, NULL); } diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index c98253dae037..7a367bf1b78d 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1238,6 +1238,8 @@ extern void perf_event_bpf_event(struct bpf_prog *prog, #ifdef CONFIG_HAVE_GUEST_PERF_EVENTS DECLARE_PER_CPU(struct perf_guest_info_callbacks *, perf_guest_cbs); +extern void __perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs); +extern void __perf_unregister_guest_info_callbacks(void); extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *callbacks); extern void perf_unregister_guest_info_callbacks(void); #endif /* CONFIG_HAVE_GUEST_PERF_EVENTS */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 9bc1375d6ed9..2f28d9d8dc94 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -6485,6 +6485,18 @@ static void perf_pending_event(struct irq_work *entry) #ifdef CONFIG_HAVE_GUEST_PERF_EVENTS DEFINE_PER_CPU(struct perf_guest_info_callbacks *, perf_guest_cbs); +void __perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) +{ + __this_cpu_write(perf_guest_cbs, cbs); +} +EXPORT_SYMBOL_GPL(__perf_register_guest_info_callbacks); + +void __perf_unregister_guest_info_callbacks(void) +{ + __this_cpu_write(perf_guest_cbs, NULL); +} +EXPORT_SYMBOL_GPL(__perf_unregister_guest_info_callbacks); + void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs) { int cpu; -- 2.33.0.259.gc128427fd7-goog