From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S262350AbVGNRSL (ORCPT ); Thu, 14 Jul 2005 13:18:11 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S261607AbVGNRQA (ORCPT ); Thu, 14 Jul 2005 13:16:00 -0400 Received: from e33.co.us.ibm.com ([32.97.110.131]:130 "EHLO e33.co.us.ibm.com") by vger.kernel.org with ESMTP id S261621AbVGNROW (ORCPT ); Thu, 14 Jul 2005 13:14:22 -0400 Date: Thu, 14 Jul 2005 12:13:57 -0500 From: serue@us.ibm.com To: "Paul E. McKenney" Cc: serue@us.ibm.com, lkml , Dipankar Sarma , "David A. Wheeler" , Tony Jones Subject: Re: rcu-refcount stacker performance Message-ID: <20050714171357.GA23309@serge.austin.ibm.com> References: <20050714142107.GA20984@serge.austin.ibm.com> <20050714152321.GB1299@us.ibm.com> <20050714134450.GB7296@sergelap.austin.ibm.com> <20050714165936.GE1299@us.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050714165936.GE1299@us.ibm.com> User-Agent: Mutt/1.5.8i Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Quoting Paul E. McKenney (paulmck@us.ibm.com): > On Thu, Jul 14, 2005 at 08:44:50AM -0500, serue@us.ibm.com wrote: > > Quoting Paul E. McKenney (paulmck@us.ibm.com): > > > My guess is that the reference count is indeed costing you quite a > > > bit. I glance quickly at the patch, and most of the uses seem to > > > be of the form: > > > > > > increment ref count > > > rcu_read_lock() > > > do something > > > rcu_read_unlock() > > > decrement ref count > > > > > > Can't these cases rely solely on rcu_read_lock()? Why do you also > > > need to increment the reference count in these cases? > > > > The problem is on module unload: is it possible for CPU1 to be > > on "do something", and sleep, and, while it sleeps, CPU2 does > > rmmod(lsm), so that by the time CPU1 stops sleeping, the code it > > is executing has been freed? > > OK, but in the above case, "do something" cannot be sleeping, since > it is under rcu_read_lock(). Oh, but that's not quite what the code is doing, rather it is doing: rcu_read_lock while get next element from list inc element.refcount rcu_read_unlock do something rcu_read_lock dec refcount rcu_read_unlock What I plan to try next is: rcu_read_lock while get next element from list if (element->owning_module->state != LIVE) continue rcu_read_unlock do something rcu_read_lock rcu_read_unlock > > Because stacker won't remove the lsm from the list of modules > > until mod->exit() is executed, and module_free(mod) happens > > immediately after that, the above scenario seems possible. > > Right, if you have some other code path that sleeps (outside of > rcu_read_lock(), right?), then you need the reference count for that > code path. But the code paths that do not sleep should be able to > dispense with the reference count, reducing the cache-line traffic. Most if not all of the codepaths can sleep, however. So unfortunately that doesn't seem a feasible solution. That's why I'm hoping there is something inherent in the module unload code that I can take advantage of to forego my own refcounting. thanks, -serge