From: Davidlohr Bueso <dave@stgolabs.net>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@kernel.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Davidlohr Bueso <dave@stgolabs.net>,
linux-kernel@vger.kernel.org, Davidlohr Bueso <dbueso@suse.de>
Subject: [PATCH 8/8] locking/osq: No need for load/acquire when acquire-polling
Date: Sun, 28 Dec 2014 01:11:23 -0800 [thread overview]
Message-ID: <1419757883-4423-9-git-send-email-dave@stgolabs.net> (raw)
In-Reply-To: <1419757883-4423-1-git-send-email-dave@stgolabs.net>
Both mutexes and rwsems took a performance hit when we switched
over from the original mcs code to the cancelable variant (osq).
The reason being the use of smp_load_acquire() when polling for
node->locked. Paul describes the scenario nicely:
https://lkml.org/lkml/2013/11/19/405
The smp_load_acquire() when unqueuing make sense. In addition,
we don't need to worry about leaking the critical region as
osq is only used internally.
This impacts both regular and large levels of concurrency and
hardware, ie on a 40 core system with a disk intensive workload:
disk-1 804.83 ( 0.00%) 828.16 ( 2.90%)
disk-61 8063.45 ( 0.00%) 18181.82 (125.48%)
disk-121 7187.41 ( 0.00%) 20119.17 (179.92%)
disk-181 6933.32 ( 0.00%) 20509.91 (195.82%)
disk-241 6850.81 ( 0.00%) 20397.80 (197.74%)
disk-301 6815.22 ( 0.00%) 20287.58 (197.68%)
disk-361 7080.40 ( 0.00%) 20205.22 (185.37%)
disk-421 7076.13 ( 0.00%) 19957.33 (182.04%)
disk-481 7083.25 ( 0.00%) 19784.06 (179.31%)
disk-541 7038.39 ( 0.00%) 19610.92 (178.63%)
disk-601 7072.04 ( 0.00%) 19464.53 (175.23%)
disk-661 7010.97 ( 0.00%) 19348.23 (175.97%)
disk-721 7069.44 ( 0.00%) 19255.33 (172.37%)
disk-781 7007.58 ( 0.00%) 19103.14 (172.61%)
disk-841 6981.18 ( 0.00%) 18964.22 (171.65%)
disk-901 6968.47 ( 0.00%) 18826.72 (170.17%)
disk-961 6964.61 ( 0.00%) 18708.02 (168.62%)
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
kernel/locking/osq_lock.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c
index 9c6e251..d10dfb9 100644
--- a/kernel/locking/osq_lock.c
+++ b/kernel/locking/osq_lock.c
@@ -109,7 +109,7 @@ bool osq_lock(struct optimistic_spin_queue *lock)
* cmpxchg in an attempt to undo our queueing.
*/
- while (!smp_load_acquire(&node->locked)) {
+ while (!READ_ONCE(node->locked)) {
/*
* If we need to reschedule bail... so we can block.
*/
--
2.1.2
prev parent reply other threads:[~2014-12-28 9:12 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-28 9:11 [PATCH 0/8] locking: Various updates Davidlohr Bueso
2014-12-28 9:11 ` [PATCH 1/8] Documentation/memory-barriers.txt: Fix smp typo Davidlohr Bueso
2014-12-31 2:02 ` Paul E. McKenney
2014-12-28 9:11 ` [PATCH 2/8] locking/mutex: Get rid of use_ww_cxt param in common paths Davidlohr Bueso
2014-12-28 16:23 ` Peter Zijlstra
2014-12-28 18:12 ` Davidlohr Bueso
2014-12-28 9:11 ` [PATCH 3/8] locking/mutex: Checking the stamp is ww only Davidlohr Bueso
2014-12-28 9:11 ` [PATCH 4/8] locking/mutex: Delete useless comment Davidlohr Bueso
2014-12-28 9:11 ` [PATCH 5/8] locking/mutex: Introduce ww_mutex_set_context_slowpath Davidlohr Bueso
2014-12-28 9:11 ` [PATCH 6/8] locking/mcs: Better differentiate between mcs variants Davidlohr Bueso
2015-01-14 14:13 ` Ingo Molnar
2015-01-14 15:55 ` Davidlohr Bueso
2015-01-14 14:16 ` Ingo Molnar
2014-12-28 9:11 ` [PATCH 7/8] locking: Use [READ,ASSIGN]_ONCE() for non-scalar types Davidlohr Bueso
2014-12-28 16:30 ` Peter Zijlstra
2014-12-28 9:11 ` Davidlohr Bueso [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1419757883-4423-9-git-send-email-dave@stgolabs.net \
--to=dave@stgolabs.net \
--cc=dbueso@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®