From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752418Ab1AYBLI (ORCPT <rfc822;w@1wt.eu>);
	Mon, 24 Jan 2011 20:11:08 -0500
Received: from mga11.intel.com ([192.55.52.93]:46299 "EHLO mga11.intel.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1751490Ab1AYBLH (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Mon, 24 Jan 2011 20:11:07 -0500
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="4.60,372,1291622400"; 
   d="scan'208";a="880586471"
Subject: Re: more dput lock contentions in 2.6.38-rc?
From: Shaohua Li <shaohua.li@intel.com>
To: Nick Piggin <npiggin@gmail.com>
Cc: "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Nick Piggin <npiggin@kernel.dk>, "Chen, Tim C" <tim.c.chen@intel.com>
In-Reply-To: <AANLkTikOdiQR+rWXzoU9z5sJv9Q2=ehdhi1NCHXT0a-w@mail.gmail.com>
References: <1295915725.1949.967.camel@sli10-conroe>
	 <AANLkTikOdiQR+rWXzoU9z5sJv9Q2=ehdhi1NCHXT0a-w@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Date: Tue, 25 Jan 2011 09:11:04 +0800
Message-ID: <1295917864.1949.970.camel@sli10-conroe>
Mime-Version: 1.0
X-Mailer: Evolution 2.30.3 
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, 2011-01-25 at 09:04 +0800, Nick Piggin wrote:
> On Tue, Jan 25, 2011 at 11:35 AM, Shaohua Li <shaohua.li@intel.com> wrote:
> > Hi,
> > we are testing dbench benchmark and see big drop of 2.6.38-rc compared
> > to 2.6.37 in several machines with 2 sockets or 4 sockets. We have 12
> > disks mount to /mnt/stp/dbenchdata/sd*/ and dbench runs against data of
> > the disks. According to perf, we saw more lock contentions:
> > In 2.6.37: 13.00%        dbench  [kernel.kallsyms]   [k] _raw_spin_lock
> > In 2.6.38-rc: 69.45%        dbench  [kernel.kallsyms]   [k]_raw_spin_lock
> > -     69.45%        dbench  [kernel.kallsyms]   [k] _raw_spin_lock
> >   - _raw_spin_lock
> >      - 48.41% dput
> >         - 61.17% path_put
> >            - 60.47% do_path_lookup
> >               + 53.18% user_path_at
> >               + 42.13% do_filp_open
> >               + 4.69% user_path_parent
> 
> What filesystems are mounted on the path?
ext3 or ext4

> >            - 35.56% d_path
> >                 seq_path
> >                 show_vfsmnt
> >                 seq_read
> >                 vfs_read
> >                 sys_read
> >                 system_call_fastpath
> >                 __GI___libc_read
> 
> This guy is from glibc's statvfs call that dbench uses. It
> parses /proc/mounts for mount flags which is racy (and
> not a good idea to do with any frequency).
> 
> A patch went into the kernel that allows glibc to get the
> flags directly. Not sure about glibc status, I imagine it
> will get there in another decade or two... Can you try
> commenting it out of dbench source code?
Sure, maybe after Chinese new year holiday, sorry.