From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754525Ab1FPBuv (ORCPT <rfc822;w@1wt.eu>);
	Wed, 15 Jun 2011 21:50:51 -0400
Received: from smtp1.linux-foundation.org ([140.211.169.13]:32895 "EHLO
	smtp1.linux-foundation.org" rhost-flags-OK-OK-OK-OK)
	by vger.kernel.org with ESMTP id S1753538Ab1FPBuu (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Wed, 15 Jun 2011 21:50:50 -0400
MIME-Version: 1.0
In-Reply-To: <1308173849.15315.91.camel@twins>
References: <1308097798.17300.142.camel@schen9-DESK> <1308101214.15392.151.camel@sli10-conroe>
 <1308138750.15315.62.camel@twins> <20110615161827.GA11769@tassilo.jf.intel.com>
 <1308156337.2171.23.camel@laptop> <1308163398.17300.147.camel@schen9-DESK>
 <1308169937.15315.88.camel@twins> <4DF91CB9.5080504@linux.intel.com>
 <1308172336.17300.177.camel@schen9-DESK> <1308173849.15315.91.camel@twins>
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Wed, 15 Jun 2011 18:50:18 -0700
Message-ID: <BANLkTim5TPKQ9RdLYRxy=mphOVKw5EXvTA@mail.gmail.com>
Subject: Re: REGRESSION: Performance regressions from switching anon_vma->lock
 to mutex
To: Peter Zijlstra <a.p.zijlstra@chello.nl>,
        Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>, Shaohua Li <shaohua.li@intel.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>,
        KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        David Miller <davem@davemloft.net>,
        Martin Schwidefsky <schwidefsky@de.ibm.com>,
        Russell King <rmk@arm.linux.org.uk>, Paul Mundt <lethal@linux-sh.org>,
        Jeff Dike <jdike@addtoit.com>, Richard Weinberger <richard@nod.at>,
        "Luck, Tony" <tony.luck@intel.com>,
        KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
        Mel Gorman <mel@csn.ul.ie>, Nick Piggin <npiggin@kernel.dk>,
        Namhyung Kim <namhyung@gmail.com>, "Shi, Alex" <alex.shi@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "Rafael J. Wysocki" <rjw@sisk.pl>
Content-Type: text/plain; charset=ISO-8859-1
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jun 15, 2011 at 2:37 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>
> http://programming.kicks-ass.net/sekrit/39-2.txt.bz2
> http://programming.kicks-ass.net/sekrit/tip-2.txt.bz2
>
> tip+sirq+linus is still slightly faster than .39 here,

Hmm. Your profile doesn't show the mutex slowpath at all, so there's a
big difference to the one Tim quoted parts of.

In fact, your profile looks fine. The load clearly spends tons of time
in page faulting and in timing things (that read_hpet thing is
disgusting), but with that in mind, the profile doesn't look scary.
Yes, the 2% spinlock time is bad, but you've clearly not hit the real
lock contention case. The mutex lock shows up, but _way_ below the
spinlock, and the slowpath never shows at all. You end up having
mutex_spin_on_owner at 0.09%, it's not really visible.

Clearly going from your two-socket 12-core thing to Tim's four-socket
40-core case is a big jump. But maybe it really was about RCU, and
even the limited softirq patch that moves the grace period stuff etc
back to softirqs ends up helping.

Tim, have you tried running your bigger load with that patch? You
could try my patch on top too just to match Peter's tree, but I doubt
that's the big first-order issue.

                       Linus