From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753841Ab3KTBiO (ORCPT ); Tue, 19 Nov 2013 20:38:14 -0500 Received: from mga09.intel.com ([134.134.136.24]:51391 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753728Ab3KTBht (ORCPT ); Tue, 19 Nov 2013 20:37:49 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.93,535,1378882800"; d="scan'208";a="411505186" Subject: [PATCH v6 4/5] MCS Lock: Barrier corrections From: Tim Chen To: Ingo Molnar , Andrew Morton , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, linux-mm , linux-arch@vger.kernel.org, Linus Torvalds , Waiman Long , Andrea Arcangeli , Alex Shi , Andi Kleen , Michel Lespinasse , Davidlohr Bueso , Matthew R Wilcox , Dave Hansen , Peter Zijlstra , Rik van Riel , Peter Hurley , "Paul E.McKenney" , Tim Chen , Raghavendra K T , George Spelvin , "H. Peter Anvin" , Arnd Bergmann , Aswin Chandramouleeswaran , Scott J Norton , Will Deacon , "Figo.zhang" In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Tue, 19 Nov 2013 17:37:43 -0800 Message-ID: <1384911463.11046.454.camel@schen9-DESK> Mime-Version: 1.0 X-Mailer: Evolution 2.32.3 (2.32.3-1.fc14) Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch corrects the way memory barriers are used in the MCS lock with smp_load_acquire and smp_store_release fucnction. It removes ones that are not needed. It uses architecture specific load-acquire and store-release primitives for synchronization, if available. Generic implementations are provided in case they are not defined even though they may not be optimal. These generic implementation could be removed later on once changes are made in all the relevant header files. Suggested-by: Michel Lespinasse Signed-off-by: Waiman Long Signed-off-by: Jason Low Signed-off-by: Tim Chen --- kernel/locking/mcs_spinlock.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/kernel/locking/mcs_spinlock.c b/kernel/locking/mcs_spinlock.c index 44fb092..6f2ce8e 100644 --- a/kernel/locking/mcs_spinlock.c +++ b/kernel/locking/mcs_spinlock.c @@ -37,15 +37,19 @@ void mcs_spin_lock(struct mcs_spinlock **lock, struct mcs_spinlock *node) node->locked = 0; node->next = NULL; + /* xchg() provides a memory barrier */ prev = xchg(lock, node); if (likely(prev == NULL)) { /* Lock acquired */ return; } ACCESS_ONCE(prev->next) = node; - smp_wmb(); - /* Wait until the lock holder passes the lock down */ - while (!ACCESS_ONCE(node->locked)) + /* + * Wait until the lock holder passes the lock down. + * Using smp_load_acquire() provides a memory barrier that + * ensures subsequent operations happen after the lock is acquired. + */ + while (!(smp_load_acquire(&node->locked))) arch_mutex_cpu_relax(); } EXPORT_SYMBOL_GPL(mcs_spin_lock); @@ -68,7 +72,12 @@ void mcs_spin_unlock(struct mcs_spinlock **lock, struct mcs_spinlock *node) while (!(next = ACCESS_ONCE(node->next))) arch_mutex_cpu_relax(); } - ACCESS_ONCE(next->locked) = 1; - smp_wmb(); + /* + * Pass lock to next waiter. + * smp_store_release() provides a memory barrier to ensure + * all operations in the critical section has been completed + * before unlocking. + */ + smp_store_release(&next->locked, 1); } EXPORT_SYMBOL_GPL(mcs_spin_unlock); -- 1.7.11.7