From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E9DDC433FE for ; Sun, 13 Dec 2020 19:11:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4951923406 for ; Sun, 13 Dec 2020 19:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393610AbgLMTKa (ORCPT ); Sun, 13 Dec 2020 14:10:30 -0500 Received: from Galois.linutronix.de ([193.142.43.55]:46450 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726377AbgLMTBo (ORCPT ); Sun, 13 Dec 2020 14:01:44 -0500 Date: Sun, 13 Dec 2020 19:01:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1607886061; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0Nv/PJh7dwxulY0qmnXEP+WSQ7CYQccSMJKPCqnH5Ro=; b=DPDK9NECXeuvZgr1YSi8Cf4dK7gbadnOpdpk1msFx1aFZFdVJ0PD/bASm1FXYLGm7l9+FJ jNi3hfakw0oqBejOLeVeUFZyY2nwFDneMEyaLVArZbtWQJQw11H7CIYQqVRKMM/ShYgABr /Z/DOvVl/NHt9+Yi5/DDMcCiPgD3ojQ2PZ+YCPzqp7cOImecGCXA5cnZ0E1EO4SqwJ9DtY fQnkPXtS/OzWJrUlQoNZeyBufhcLncMV2EXLKhRmf0k03c3aXwUzfM2UC9fYnu5mVs6sDG zCeqzyxEifyxHzHdAnIAeCokkCSWOXH4IK9ODdQcx3CbjlXFbOVivuhV7S0tUA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1607886061; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0Nv/PJh7dwxulY0qmnXEP+WSQ7CYQccSMJKPCqnH5Ro=; b=UuzTZfhSGP/nWyoBsOUy1RfT4GmdU1PR57px07lgU9uXjoHumkbhe8Wm2i//4MSUCKXZ1y glPYsb731ym3EvDQ== From: "tip-bot2 for Joel Fernandes (Google)" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: core/rcu] rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs Cc: Neeraj Upadhyay , Frederic Weisbecker , "Joel Fernandes (Google)" , "Paul E. McKenney" , x86@kernel.org, linux-kernel@vger.kernel.org MIME-Version: 1.0 Message-ID: <160788606053.3364.4345333662369014444.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The following commit has been merged into the core/rcu branch of tip: Commit-ID: bd56e0a4a291bc9db2cbaddef20ec61a1aad4208 Gitweb: https://git.kernel.org/tip/bd56e0a4a291bc9db2cbaddef20ec61a1aad4208 Author: Joel Fernandes (Google) AuthorDate: Wed, 07 Oct 2020 13:50:36 -07:00 Committer: Paul E. McKenney CommitterDate: Thu, 19 Nov 2020 19:37:17 -08:00 rcu/tree: nocb: Avoid raising softirq for offloaded ready-to-execute CBs Testing showed that rcu_pending() can return 1 when offloaded callbacks are ready to execute. This invokes RCU core processing, for example, by raising RCU_SOFTIRQ, eventually resulting in a call to rcu_core(). However, rcu_core() explicitly avoids in any way manipulating offloaded callbacks, which are instead handled by the rcuog and rcuoc kthreads, which work independently of rcu_core(). One exception to this independence is that rcu_core() invokes do_nocb_deferred_wakeup(), however, rcu_pending() also checks rcu_nocb_need_deferred_wakeup() in order to correctly handle this case, invoking rcu_core() when needed. This commit therefore avoids needlessly invoking RCU core processing by checking rcu_segcblist_ready_cbs() only on non-offloaded CPUs. This reduces overhead, for example, by reducing softirq activity. This change passed 30 minute tests of TREE01 through TREE09 each. On TREE08, there is at most 150us from the time that rcu_pending() chose not to invoke RCU core processing to the time when the ready callbacks were invoked by the rcuoc kthread. This provides further evidence that there is no need to invoke rcu_core() for offloaded callbacks that are ready to invoke. Cc: Neeraj Upadhyay Reviewed-by: Frederic Weisbecker Reviewed-by: Neeraj Upadhyay Signed-off-by: Joel Fernandes (Google) Signed-off-by: Paul E. McKenney --- kernel/rcu/tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index d6a015e..50d90ee 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3718,7 +3718,8 @@ static int rcu_pending(int user) return 1; /* Does this CPU have callbacks ready to invoke? */ - if (rcu_segcblist_ready_cbs(&rdp->cblist)) + if (!rcu_segcblist_is_offloaded(&rdp->cblist) && + rcu_segcblist_ready_cbs(&rdp->cblist)) return 1; /* Has RCU gone idle with this CPU needing another grace period? */