A note on APIC bus latency

[Jonathan Lundell <jlundell@pobox.com>]

We recently ran into some issues caused by APIC bus latency. I was 
reminded of that by the recent discussion of NAPI and related 
interrupt-performance matters.

Intel processors that predate Pentium 4 but use an APIC transmit APIC 
messages over a serial APIC bus, typically at 16.7 MHz. (Pentium 4 
uses the system bus for APIC messages.)

A message exchange (IO-APIC sends an interrupt message; CPU sends 
back an EOI message) requires from 35 to 48 APIC bus clocks, or 2-3 
microseconds. That gets to be pretty significant compared to packet 
times, especially at Gbit speeds, but even at 100 MHz, and is the 
time required to burst a thousand bytes or more at faster PCI rates.

It's also likely to be significant for inter-processor interrupts, 
though I don't know what the implications are here.
-- 
/Jonathan Lundell.