From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753589AbcEPLKG (ORCPT ); Mon, 16 May 2016 07:10:06 -0400 Received: from merlin.infradead.org ([205.233.59.134]:35651 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752586AbcEPLKD (ORCPT ); Mon, 16 May 2016 07:10:03 -0400 Date: Mon, 16 May 2016 13:09:48 +0200 From: Peter Zijlstra To: Peter Hurley Cc: Waiman Long , Ingo Molnar , linux-kernel@vger.kernel.org, Davidlohr Bueso , Jason Low , Dave Chinner , Scott J Norton , Douglas Hatch , Paul McKenney Subject: Re: [PATCH v2] locking/rwsem: Add reader-owned state to the owner field Message-ID: <20160516110948.GM3193@twins.programming.kicks-ass.net> References: <1462580424-40333-1-git-send-email-Waiman.Long@hpe.com> <5733AC64.6020306@hurleysoftware.com> <20160513150749.GT3192@twins.programming.kicks-ass.net> <573615AD.60300@hurleysoftware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <573615AD.60300@hurleysoftware.com> User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, May 13, 2016 at 10:58:05AM -0700, Peter Hurley wrote: > > Note that barrier() and READ_ONCE() have overlapping but not identical > > results and the combined use actually makes sense here. > > > > Yes, a barrier() anywhere in the loop will force a reload of the > > variable, _however_ it doesn't force that reload to not suffer from > > load tearing. > > > > Using volatile also forces a reload, but also ensures the load cannot > > be torn IFF it is of machine word side and naturally aligned. > > > > So while the READ_ONCE() here is pointless for forcing the reload; > > that's already ensured, we still need to make sure the load isn't torn. > > If load tearing a naturally aligned pointer is a real code generation > possibility then the rcu list code is broken too (which loads ->next > directly; cf. list_for_each_entry_rcu() & list_for_each_entry_lockless()). > > For 4.4, Paul added READ_ONCE() checks for list_empty() et al, but iirc > that had to do with control dependencies and not load tearing. Well, Paul is the one who started the whole load/store tearing thing, so I suppose he knows what he's doing. That said; its a fairly recent as things go so lots of code hasn't been updated yet, and its also a very unlikely thing for a compiler to do; since it mostly doesn't make sense to emit multiple instructions where one will do, so its not a very high priority thing either. But from what I understand, the compiler is free to emit all kinds of nonsense for !volatile loads/stores. > OTOH, this patch might actually produce store-tearing: > > +static inline void rwsem_set_reader_owned(struct rw_semaphore *sem) > +{ > + /* > + * We check the owner value first to make sure that we will only > + * do a write to the rwsem cacheline when it is really necessary > + * to minimize cacheline contention. > + */ > + if (sem->owner != RWSEM_READER_OWNED) > + sem->owner = RWSEM_READER_OWNED; > +} Correct; which is why we should always use {READ,WRITE}_ONCE() for anything that is used locklessly.