Re: PCIE

[David Miller <davem@davemloft.net>]

From: "H. Peter Anvin" <hpa@zytor.com>
Date: Sat, 26 May 2007 19:34:26 -0700

> There are systems which only get a single bit indication that an MSI has
> happened.
> 
> Presumably we need something like IRQF_MSI which can be set as
> appropriate depending on the architecture?

Although I don't want to make the IRQ handling subsystems any more
complicated than they already are, one idea I floated around is that
we could seperate CPU irq numbers (the things we pass around today)
and MSI vector numbers.

But I immediately understand how that is unnecessary to some extent,
any platform which needs to deal with that kind of distinction can use
virtual IRQ numbers like sparc64 and powerpc do.

Sparc64 PCI-E controllers, for example, allow you to group several
MSIs into a 'group', and the interrupt source is for the group rather
than the individual MSIs.  When the MSI group interrupt arrives, you
get a descriptor in a per-MSI-group ring buffer that describes the MSI
that arrived.

This descriptor in fact passes on a ton of interesting information,
see arch/sparc64/kernel/pci_sun4v.c:pci_sun4v_msiq_entry.

It gives you the type, the system TSC value at the time of interrupt
arrival (so you could do incredibly cool profiling with this),
the PCI bus/device/fn that generated the MSI vector, the MSI address
that was signalled, the MSI data field, and full information for PCI-E
MSG packets including bus/dev/fn of target, the message routing code,
and the message code.

But we use none of these facilities currently because it's either
impossible or too cumbersome to be useful at the moment.