From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756029AbcIEMoq (ORCPT ); Mon, 5 Sep 2016 08:44:46 -0400 Received: from merlin.infradead.org ([205.233.59.134]:58952 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754845AbcIEMoD (ORCPT ); Mon, 5 Sep 2016 08:44:03 -0400 Message-Id: <20160905124027.796192899@infradead.org> User-Agent: quilt/0.61-1 Date: Mon, 05 Sep 2016 14:36:46 +0200 From: Peter Zijlstra To: Linus Torvalds , Waiman Long , Jason Low , Ding Tianhong , Thomas Gleixner , Will Deacon , Ingo Molnar , Imre Deak , Linux Kernel Mailing List , Davidlohr Bueso , Tim Chen , Terry Rudd , "Paul E. McKenney" , Jason Low , Peter Zijlstra Subject: [RFC][PATCH -v3 10/10] locking/mutex: Implement alternative HANDOFF References: <20160905123636.450529224@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline; filename=peterz-mutex-handoff-opt.patch Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org As mentioned in a previous patch, its possible to implement the handoff logic differently, avoiding the issue where we 'leak' a HADOFF flag. This patch does so, just to show what it looks like; I'm not at all convinced this is worth it. Signed-off-by: Peter Zijlstra (Intel) --- kernel/locking/mutex.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -71,6 +71,32 @@ static inline unsigned long __owner_flag return owner & MUTEX_FLAGS; } +static inline bool __mutex_req_handoff(struct mutex *lock) +{ + unsigned long owner, curr = (unsigned long)current; + + owner = atomic_long_read(&lock->owner); + for (;;) { + unsigned long old, new; + + if (owner & MUTEX_FLAG_HANDOFF) + return false; + + if (__owner_task(owner)) + new = owner | MUTEX_FLAG_HANDOFF; + else + new = curr | __owner_flags(owner); + + old = atomic_long_cmpxchg_acquire(&lock->owner, owner, new); + if (old == owner) + break; + + owner = old; + } + + return !__owner_task(owner); +} + /* * Actual trylock that will work on any unlocked state. * @@ -609,7 +635,6 @@ __mutex_lock_common(struct mutex *lock, struct task_struct *task = current; struct mutex_waiter waiter; unsigned long flags; - bool first = false; struct ww_mutex *ww; int ret; @@ -676,8 +701,9 @@ __mutex_lock_common(struct mutex *lock, schedule_preempt_disabled(); if (__mutex_waiter_is_first(lock, &waiter)) { - first = true; - __mutex_set_flag(lock, MUTEX_FLAG_HANDOFF); + if (__mutex_req_handoff(lock)) + break; + if (mutex_optimistic_spin(lock, ww_ctx, use_ww_ctx, true)) break; } @@ -700,8 +726,6 @@ __mutex_lock_common(struct mutex *lock, mutex_remove_waiter(lock, &waiter, task); if (likely(list_empty(&lock->wait_list))) __mutex_clear_flag(lock, MUTEX_FLAGS); - else if (first && (atomic_long_read(&lock->owner) & MUTEX_FLAG_HANDOFF)) - __mutex_clear_flag(lock, MUTEX_FLAG_HANDOFF); debug_mutex_free_waiter(&waiter);