From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753855AbbFSXdM (ORCPT ); Fri, 19 Jun 2015 19:33:12 -0400 Received: from mga03.intel.com ([134.134.136.65]:1259 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751391AbbFSXdH (ORCPT ); Fri, 19 Jun 2015 19:33:07 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,646,1427785200"; d="scan'208";a="730961067" Date: Fri, 19 Jun 2015 16:33:06 -0700 From: Andi Kleen To: Dave Hansen Cc: dave.hansen@linux.intel.com, akpm@linux-foundation.org, jack@suse.cz, viro@zeniv.linux.org.uk, eparis@redhat.com, john@johnmccutchan.com, rlove@rlove.org, tim.c.chen@linux.intel.com, linux-kernel@vger.kernel.org, paulmck@linux.vnet.ibm.com Subject: Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched files Message-ID: <20150619233306.GT25760@tassilo.jf.intel.com> References: <20150619215025.4F689817@viggo.jf.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150619215025.4F689817@viggo.jf.intel.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jun 19, 2015 at 02:50:25PM -0700, Dave Hansen wrote: > > From: Dave Hansen > > I have a _tiny_ microbenchmark that sits in a loop and writes > single bytes to a file. Writing one byte to a tmpfs file is > around 2x slower than reading one byte from a file, which is a > _bit_ more than I expecte. This is a dumb benchmark, but I think > it's hard to deny that write() is a hot path and we should avoid > unnecessary overhead there. > > I did a 'perf record' of 30-second samples of read and write. > The top item in a diffprofile is srcu_read_lock() from > fsnotify(). There are active inotify fd's from systemd, but > nothing is actually listening to the file or its part of > the filesystem. > > I *think* we can avoid taking the srcu_read_lock() for the > common case where there are no actual marks on the file > being modified *or* the vfsmount. What is so expensive in it? Just the memory barrier in it? Perhaps the function can be tuned in general. -Andi int __srcu_read_lock(struct srcu_struct *sp) { int idx; idx = ACCESS_ONCE(sp->completed) & 0x1; preempt_disable(); __this_cpu_inc(sp->per_cpu_ref->c[idx]); smp_mb(); /* B */ /* Avoid leaking the critical section. */ __this_cpu_inc(sp->per_cpu_ref->seq[idx]); preempt_enable(); return idx; } > > The *_fsnotify_mask is an aggregate of each of the masks from > each mark. If we have nothing set in the masks at all then there > are no marks and no need to do anything with 'ignored masks' > since none exist. This keeps us from having to do the costly > srcu_read_lock() for a check which is very cheap. > > This patch gave a 10.8% speedup in writes/second on my test. > > Signed-off-by: Dave Hansen > Cc: Andrew Morton > Cc: Jan Kara > Cc: Al Viro > Cc: Eric Paris > Cc: John McCutchan > Cc: Robert Love > Cc: Tim Chen > Cc: Andi Kleen > Cc: linux-kernel@vger.kernel.org > --- > > b/fs/notify/fsnotify.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff -puN fs/notify/fsnotify.c~optimize-fsnotify fs/notify/fsnotify.c > --- a/fs/notify/fsnotify.c~optimize-fsnotify 2015-06-19 13:29:53.117283581 -0700 > +++ b/fs/notify/fsnotify.c 2015-06-19 13:29:53.123283853 -0700 > @@ -213,6 +213,16 @@ int fsnotify(struct inode *to_tell, __u3 > !(test_mask & to_tell->i_fsnotify_mask) && > !(mnt && test_mask & mnt->mnt_fsnotify_mask)) > return 0; > + /* > + * Optimization: The *_fsnotify_mask is an aggregate of each of the > + * masks from each mark. If we have nothing set in the masks at > + * all then there are no marks and no need to do anything with > + * 'ignored masks' since none exist. This keeps us from having to > + * do the costly srcu_read_lock() for a check which is very cheap. > + */ > + if (!to_tell->i_fsnotify_mask && > + (!mnt || !mnt->mnt_fsnotify_mask)) > + return 0; > > idx = srcu_read_lock(&fsnotify_mark_srcu); > > _ -- ak@linux.intel.com -- Speaking for myself only -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/