Re: [RFC] Relaxed PIO read vs. DMA write ordering

[Matthew Wilcox <willy@debian.org>]

On Wed, Jan 07, 2004 at 09:58:02AM -0800, Jesse Barnes wrote:
> I've already talked with Grant a little about this, but I'm having
> second thoughts about the approach we discussed.  PCI-X allows PIO read
> responses to 'pass' DMA writes to system memory when the relaxed
> ordering bit is set in the PCI-X command word _and_ the transaction has
> the relaxed ordering bit set (so called "Relaxed Read Ordering" in
> section 11.2 of the PCI-X addendum).  This effectively 'unserializes'
> PIO vs. DMA transactions so that PIO reads doesn't get stuck behind an
> unrelated DMA writes from the same device; something which can
> potentially take awhile since cacheline ownership has to be acquired,
> etc.

So we want a pci_set_relaxed() macro / function() to set this bit
(otherwise dozens of drivers will start to try to set the bit themselves,
badly).  If this bit *isn't* set, setting the bit in the transaction will have
no effect, right?

> The proposal I gave to Grant added a new readX() variant,
> readX_relaxed(), that drivers could use when they don't need strict
> ordering semantics (this may actually be the majority of cases, but it's
> safer to be strict by default than create a read_ordered and open a
> window for data corruption).  It might be confusing, however, to add yet
> another readX() routine, and there are other ways we might go about it.
> One suggestion was to overload the pci_sync_* calls so that they'd
> explicitly flush DMA writes to system memory, implying that all reads on
> some platforms would use relaxed semantics, but that we'd have to modify
> drivers to add in pci_sync_* calls where needed.

How about always setting the bit in readb() and having a readb_ordered()
which doesn't set the bit in the transaction?  That way, drivers which
call pci_set_relaxed() have the responsibility to verify they're not
relying on these semantics and use readb_ordered() in any places that
they are.

No doubt you're going to smack this idea down by telling me what SN2
firmware currently does ...

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain