Re: [PATCH] NUMA-Q disable irqbalance

[Andrew Theurer <habanero@us.ibm.com>]

> > People in our benchmarking group (Andrew, cc'ed) have told me that
> > reducing the frequency of IO-APIC reprogramming by a factor of 20 or
> > so improves performance greatly - don't know what HZ that was at, but
> > the whole thing seems a little overenthusiastic to me.

I thought I saw a big difference in 2.4, but my latest runs could not 
reproduce this.  I also tested Andrea's version 
http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.19rc3aa1/30_irq-balance-12
which addresses some of the problems:

[email to Andrea]
> Andrea,
> 
> I have some irqbalance results.  For some reason, Ingo's isn't performing as 
> bad as I thought on 2.4.  Your patch still performs better.  These results 
> are from NetBench, a CIFS network fileserver benchmark on a samba linux
> 4-way 
> P4 server:
> 
> 2.4.19-rc3aa3:
> 
> No Balance    Ingo IRQ Balance        Andrea IRQ Balance
> 794 Mbps              787 Mbps                        792 Mbps
> 
> With hyperthreading:
> 
> No Balance    Ingo IRQ Balance        Andrea IRQ Balance
> 773 Mbps              798 Mbps                        809 Mbps

Before I tried hyperthreading, no balance was always best performance, but 
"unfair".  However, with hyperthreading, (Andrea's) IRQbalance can be the 
best, probably because an interrupt rate of that level just can't be 
processed by "1/2" of a CPU and would benefit from a distrubution.  I did 
make a run on 2.5.25 a few weeks ago.  With IRQbalance turned off, I was 10% 
better.  I have not had a chance to try Andrea's in 2.5 but I suspect it 
would be best overall, since he compensates for HZ:
#define IRQ_BALANCE_INTERVAL (HZ/50) 
and does not seem to reprogram the IO-APIC as often.  IMO, I think Andrea's 
version is a little less aggressive and has less overhead, something I'd 
prefer in 2.5. 

-Andrew Theurer