x86_64 IOAPIC

x86_64 IOAPIC

[SUJIT V]

Hi,
    We have a IBM Qauadcore 8 CPU system running x86_64 linux 2.6.23.
We were doing some performance test. (disk & network).
The eth0 IRQ is pinned to CPU0.  The aacraid interrupt has not been
assigned any smp affinity.

On the system we see in (/proc/interrupts) that when the eth0
interrupt rate is high, the aacraid interrupt counter for CPU0 does
not increase as much compared to aacraid interrupts handled by other
CPUs .
This shows that IOAPIC is doing some load balancing.

I wanted to check the code which is doing this load balancing. I
checked the file arch/x86_64/kernel/io_apic.c. I could not find any
irq balancing stuff out there.
I went through some archived discussions about the x86_64 ioapic irq
balancing. Most of the discussions were about using irqbalance
utility.


Could some one point me to the relevant piece of io_apic code ?


Thanks

Re: x86_64 IOAPIC

[SUJIT V]

Forgot to mention that we are not using the irqbalance utility.

So which part of the kernel io_apic code is doing the irq balancing
which we see in /proc/interrupt.

Thanks


On Thu, Jan 29, 2009 at 2:02 PM, SUJIT V <sujit.linux@gmail.com> wrote:
> Hi,
>    We have a IBM Qauadcore 8 CPU system running x86_64 linux 2.6.23.
> We were doing some performance test. (disk & network).
> The eth0 IRQ is pinned to CPU0.  The aacraid interrupt has not been
> assigned any smp affinity.
>
> On the system we see in (/proc/interrupts) that when the eth0
> interrupt rate is high, the aacraid interrupt counter for CPU0 does
> not increase as much compared to aacraid interrupts handled by other
> CPUs .
> This shows that IOAPIC is doing some load balancing.
>
> I wanted to check the code which is doing this load balancing. I
> checked the file arch/x86_64/kernel/io_apic.c. I could not find any
> irq balancing stuff out there.
> I went through some archived discussions about the x86_64 ioapic irq
> balancing. Most of the discussions were about using irqbalance
> utility.
>
>
> Could some one point me to the relevant piece of io_apic code ?
>
>
> Thanks
>

Re: x86_64 IOAPIC

[Yinghai Lu]

On Thu, Jan 29, 2009 at 2:05 PM, SUJIT V <sujit.linux@gmail.com> wrote:
> Forgot to mention that we are not using the irqbalance utility.
>
> So which part of the kernel io_apic code is doing the irq balancing
> which we see in /proc/interrupt.
>
> Thanks
>
>
> On Thu, Jan 29, 2009 at 2:02 PM, SUJIT V <sujit.linux@gmail.com> wrote:
>> Hi,
>>    We have a IBM Qauadcore 8 CPU system running x86_64 linux 2.6.23.
>> We were doing some performance test. (disk & network).
>> The eth0 IRQ is pinned to CPU0.  The aacraid interrupt has not been
>> assigned any smp affinity.
>>
>> On the system we see in (/proc/interrupts) that when the eth0
>> interrupt rate is high, the aacraid interrupt counter for CPU0 does
>> not increase as much compared to aacraid interrupts handled by other
>> CPUs .
>> This shows that IOAPIC is doing some load balancing.
>>
>> I wanted to check the code which is doing this load balancing. I
>> checked the file arch/x86_64/kernel/io_apic.c. I could not find any
>> irq balancing stuff out there.
>> I went through some archived discussions about the x86_64 ioapic irq
>> balancing. Most of the discussions were about using irqbalance
>> utility.
>>
>>
>> Could some one point me to the relevant piece of io_apic code ?

you are using logical flat mode. and target_cpus and smp_affinity are 0xff.
aka irq is broadcast to all cpus.

YH