From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3554CC07E96 for ; Tue, 6 Jul 2021 16:24:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1568B619B5 for ; Tue, 6 Jul 2021 16:24:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230031AbhGFQ1D (ORCPT ); Tue, 6 Jul 2021 12:27:03 -0400 Received: from mail.kernel.org ([198.145.29.99]:37490 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229935AbhGFQ1B (ORCPT ); Tue, 6 Jul 2021 12:27:01 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3F7EB61C26; Tue, 6 Jul 2021 16:24:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625588662; bh=YsYkZubJJjCoLzPUhl/QuE2E+XD6qj3d4PuzTeoxUmM=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=V+h/QgmK95WRgyoOiXMoRnrgSY2YXp45qEBq6/CsXoWCVgdaYjZYN19Yz+aBlWutj fLlxqpWrHBpllE9rEmKGEpMCWdZYZ+CrRn/Ou4LMJ6D8GLg8RrDkfGCJ+zNYXenTb3 Xjtw8uKx36n/kAniY+VNhnI29oiSs/rrT6KII/jTiYMG79pZ28y2i1/l8peLCX4VuD P8Uhz4GdnBlZeEyjfJAKl2jqurRroFjdyFmedmvl7TpBqgpMJAiSyla4cqYTdfLghm 3z6melMMxvZs9d2YOuezxBJrB+Dp7JJ+t6Ru4adikSPAKSgyYTH0FeuvSJQech2418 V3A5PnE4SznRA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 03D2C5C03E0; Tue, 6 Jul 2021 09:24:22 -0700 (PDT) Date: Tue, 6 Jul 2021 09:24:21 -0700 From: "Paul E. McKenney" To: Boqun Feng Cc: Frederic Weisbecker , Peter Zijlstra , LKML , Uladzislau Rezki , Ingo Molnar , Neeraj Upadhyay , Joel Fernandes Subject: Re: [PATCH 2/2] rcu: Remove needless preemption disablement in rcu_all_qs() Message-ID: <20210706162421.GC4397@paulmck-ThinkPad-P17-Gen-1> Reply-To: paulmck@kernel.org References: <20210705234344.104239-1-frederic@kernel.org> <20210705234344.104239-2-frederic@kernel.org> <20210706123058.GB107277@lothringen> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jul 06, 2021 at 09:28:38PM +0800, Boqun Feng wrote: > On Tue, Jul 06, 2021 at 02:30:58PM +0200, Frederic Weisbecker wrote: > > On Tue, Jul 06, 2021 at 09:51:01AM +0200, Peter Zijlstra wrote: > > > On Tue, Jul 06, 2021 at 01:43:44AM +0200, Frederic Weisbecker wrote: > > > > The preemption is already disabled when we write rcu_data.rcu_urgent_qs. > > > > We can use __this_cpu_write() directly, although that path is mostly > > > > used when CONFIG_PREEMPT=n. > > > > > > > > Signed-off-by: Frederic Weisbecker > > > > Cc: Neeraj Upadhyay > > > > Cc: Joel Fernandes > > > > Cc: Uladzislau Rezki > > > > Cc: Boqun Feng > > > > --- > > > > kernel/rcu/tree_plugin.h | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h > > > > index 27b74352cccf..38b3d01424d7 100644 > > > > --- a/kernel/rcu/tree_plugin.h > > > > +++ b/kernel/rcu/tree_plugin.h > > > > @@ -871,7 +871,7 @@ void rcu_all_qs(void) > > > > preempt_enable(); > > > > return; > > > > } > > > > - this_cpu_write(rcu_data.rcu_urgent_qs, false); > > > > + __this_cpu_write(rcu_data.rcu_urgent_qs, false); > > > > > > There's another subtle difference between this_cpu_write() and > > > __this_cpu_write() aside from preempt. this_cpu_write() is also > > > IRQ-safe, while __this_cpu_write() is not. > > > > > > I've not looked at the usage here to see if that is relevant, but the > > > Changelog only mentioned the preempt side of things, and that argument > > > is incomplete in general. > > > > You're right, I missed that. I see this rcu_urgent_qs is set by > > RCU TASKS from rcu_tasks_wait_gp() (did I missed another path?). > > Not sure if this is called from IRQ nor if it actually matters to > > protect against IRQs for that single write. > > I think __this_cpu_write() being IRQ-unsafe means it may overwrite > percpu writes to other bytes in the same word? Let's say the > rcu_urgent_qs is the lowest byte in the word, the pseduo asm code of > __this_cpu_write() may be: > > __this_cpu_write(ptr, v): > long tmp = *ptr; > tmp &= ~(0xff); > tmp |= v; > *ptr = tmp; > > and the following sequence introduces an overwrite: > > __this_cpu_write(ptr, v): // v is 0, and *ptr is 1 > long tmp = *ptr; // tmp is 1 > > this_cpu_write() // modify another byte of *ptr, make it > // 0xff01 > > tmp &= ~(0xff) // tmp is 0 > tmp |=v; // tmp is 0 > *ptr = tmp; // *ptr is 0, overwrite a percpu write on > // another field. > > I know that many archs have byte-wise store, so compilers don't really > have the reason to generate code as above, but __this_cpu_write() is > just a normal write, nothing prevents this from happenning, unless I'm > missing something here? There can indeed be writes to .rcu_urgent_qs from interrupt handlers, for example in CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels from rcu_read_unlock(). More conventionally, the RCU_SOFTIRQ handler can run on the back of an interrupts, and can invoke rcu_check_quiescent_state(), which invokes rcu_report_qs_rdp(), which invokes rcu_disable_urgency_upon_qs(), which writes to ->rcu_urgent_qs. RCU takes a strict view of data races, so this wants the existing this_cpu_write(). However, RCU very likely has this_cpu_write() calls that should instead be __this_cpu_write() calls and vice versa, so please do continue treating any that you see with an appropriate level of suspicion. Thanx, Paul > Regards, > Boqun > > > > > I'm not quite used to rcu_tasks. Paul?