Re: [PATCH v2] Move calc_load call out from xtime_lock protection

[Dimitri Sivanich <sivanich@sgi.com>]

On Sat, Apr 11, 2009 at 06:53:45PM +0200, Thomas Gleixner wrote:
> On Sat, 11 Apr 2009, Dimitri Sivanich wrote:
> 
> > On Sat, Apr 11, 2009 at 11:51:36AM +0200, Thomas Gleixner wrote:
> > > On Fri, 10 Apr 2009, Dimitri Sivanich wrote:
> > > 
> > > > The xtime_lock is being held for long periods on larger systems due
> > > > to an extensive amount of time being spent in calc_load(),
> > > > specifically here:
> > > >   do_timer->update_times->calc_load->count_active_tasks->nr_active()
> > > > 
> > > > On a 64 cpu system I've seen this take approximately 55 usec.
> > > > Presumably it would be worse on larger systems.  This causes other
> > > > cpus to be held off in places such as
> > > > scheduler_tick->sched_clock_tick waiting for the xtime_lock to be
> > > > released.
> > > 
> > > I thought more about that. Why don't we move the calc_load() call into
> > > the timer softirq context and avoid fiddling with all the call sites ?
> > > Also moving calc_load out of the timer interrupt context reduces the
> > > interrupts off section as well.
> > > 
> > Sounds reasonable to me.
> 
> Ok. I looked once more and I wonder if it's possible to avoid the
> whole for_each_online_cpu() business by moving the accounting into the
> scheduler and be a bit more clever than just blindly running over all
> cpus everytime. 
> 
> /me goes back to stare a bit more at the avenrun stuff
> 

Also, if my patch gets replaced by this, you might want to avoid saving intermediate results in calc_load, as my patch attempts to do.