Re: [PATCH 2/2 v2] epoll: Do not take global 'epmutex' for simple topologies

[Nathan Zimmer <nzimmer@sgi.com>]

On Fri, Oct 04, 2013 at 11:16:12AM -0400, Jason Baron wrote:
> On 10/03/2013 05:50 PM, Andrew Morton wrote:
> > On Tue,  1 Oct 2013 17:08:14 +0000 (GMT) Jason Baron <jbaron@akamai.com> wrote:
> >
> >> When calling EPOLL_CTL_ADD for an epoll file descriptor that is attached
> >> directly to a wakeup source, we do not need to take the global 'epmutex',
> >> unless the epoll file descriptor is nested. The purpose of taking
> >> the 'epmutex' on add is to prevent complex topologies such as loops and
> >> deep wakeup paths from forming in parallel through multiple EPOLL_CTL_ADD
> >> operations. However, for the simple case of an epoll file descriptor
> >> attached directly to a wakeup source (with no nesting), we do not need
> >> to hold the 'epmutex'.
> >>
> >> This patch along with 'epoll: optimize EPOLL_CTL_DEL using rcu' improves
> >> scalability on larger systems. Quoting Nathan Zimmer's mail on SPECjbb
> >> performance:
> >>
> >> "
> >> On the 16 socket run the performance went from 35k jOPS to 125k jOPS.
> >> In addition the benchmark when from scaling well on 10 sockets to scaling well
> >> on just over 40 sockets.
> >>
> >> ...
> >>
> >> Currently the benchmark stops scaling at around 40-44 sockets but it seems like
> >> I found a second unrelated bottleneck.
> >> "
> > I couldn't resist fiddling.  Please review
> >
> > From: Andrew Morton <akpm@linux-foundation.org>
> > Subject: epoll-do-not-take-global-epmutex-for-simple-topologies-fix
> >
> > - use `bool' for boolean variables
> > - remove unneeded/undesirable cast of void*
> > - add missed ep_scan_ready_list() kerneldoc 
> 
> Hi Andrew,
> 
> Looks good to me. Feel free to add:
> 
> Reviewed-by: Jason Baron <jbaron@akamai.com>
> 
> Thanks,
> 
> -Jason

Just finished rerunning the benchmarks with this latest patchset, including the
fiddling, and it still looks great.

Thanks,

Nate