Re: PCI: MSI interrupts masked using prohibited method

[Jesse Barnes <jbarnes@virtuousgeek.org>]

On Friday, June 27, 2008 10:07 am Jesse Barnes wrote:
> On Friday, June 27, 2008 5:17 am David Vrabel wrote:
> > Jesse Barnes wrote:
> > > On Tuesday, June 24, 2008 3:46 am David Vrabel wrote:
> > >> PCI MSI interrupts are masked and unmasked using a method (by writing
> > >> the MSI Enable capability bit) that is prohibited by the PCI
> > >> specification.
> > >
> > > Yeah, it's probably quite a bit slower too (I assume you're talking
> > > about io_apic_64's msi_mask_irq).  Seems like masking this at the
> > > ioapic level would make more sense anyway...
> > >
> > >> This behaviour can cause missed interrupts with some devices if the
> > >> interrupt is asserted by the hardware while MSI is disabled.
> > >>
> > >> I believe the interrupt should be masked/unmasked on the interrupt
> > >> controller (the APIC on x86, for example).   I'm going to test this
> > >> now and see if it works.
> >
> > After further research it seems that MSI interrupts aren't routed via
> > the IO-APIC, so this cannot be done.
> >
> > I think the only solution is to not perform any sort of masking and rely
> > on the device driver being able to handle this.
>
> On x86, they're targetted at the LAPIC block (see section 8 of the IA SDM);
> maybe we could modify the message address or data such that it won't
> generate an interrupt instead?  I think this latest approach is correct in
> the sense that both the system and drivers have to take care that
>   1) we don't miss interrupts, and
>   2) we don't generate spurious unhandled interrupts (as might happen if we
> disable MSI and the device generates a legacy IRQ on a different vector).
>
> But it looks like the real problem is in the system interrupt code that
> handles MSIs.  We should only be disabling MSIs using the capability bit at
> device enable or disable time, not during the normal course of interrupt
> handling, since if we do we may miss device interrupts or have them routed
> to the wrong (legacy) vector.
>
> Cc'ing Ingo & Thomas since they know the core interrupt code pretty well.

Ingo or Matthew, any ideas about this?  The fundamental issue is that if we go 
poke at a device's MSI cap bits during interrupt handling, the device may 
start using regular IRQs instead, potentially on a different vector.  It 
would be good if we could come up with a better way of masking MSIs during 
handling...

Thanks,
Jesse