Re: PCI: MSI interrupts masked using prohibited method

[Michal Schmidt <mschmidt@redhat.com>]

On Tue, 22 Jul 2008 10:52:26 -0700
Jesse Barnes <jbarnes@virtuousgeek.org> wrote:

> On Tuesday, July 22, 2008 6:56 am Michal Schmidt wrote:
> > This breaks the setting of SMP affinity for MSI interrupts :-(
> > With the patch, writes to /proc/irq/<n>/smp_affinity are ignored
> > for an MSI interrupt.
> 
> It should only break it for devices that don't provide a mask bit.
> But given that we can't really mask generically on those devices,
> maybe that's ok given that it fixes the other problems mentioned in
> this thread...

I looked a bit more into why exactly it breaks. The device for which
MSI IRQ affinity breaks is a "Broadcom Corporation NetXtreme II BCM5708
Gigabit Ethernet" (14e4:164c rev 12) in HP DL360 G5.

The interesting thing is that I can see Destination ID bits of MSI
Message Address change correctly in lspci output. But the interrupt is
still delivered load-balanced to all CPUs even though the Destination
ID identifies the single CPU I asked for. It seems the device only
takes the new Message Address setting into account when the MSI Enable
bit in the Message Control register is changed from 0 to 1. I tested
this by setting the MSI enable bit to 0 and then immediately back to 1
at the end of io_apic_64.c:set_msi_irq_affinity().

Is this a permitted behaviour for the device? I couldn't find anything
in the PCI specification that would mentioned it.

Later I tested it with a different device which does not have maskbits
either (an Intel Ethernet controller). For this device MSI IRQ
migration works without problems and no hackery with the MSI enable bit
was necessary.

Michal