Re: [PATCH 0/3] pci: let devices flush DMA to host memory

[Grant Grundler <grundler@parisc-linux.org>]

On Fri, Aug 24, 2007 at 11:02:32AM -0700, akepner@sgi.com wrote:
> 
> On Altix, DMA may be reordered within the NUMA interconnect. 
> This can be a problem with Infiniband, where DMA to Completion 
> Queues can race with data DMA. This patchset allows a driver 
> to associate a memory region with a "dmaflush" attribute, so 
> that writes to the memory region flush in-flight DMA, preventing 
> the CQ/data race.

FYI to linux-pci folks
This patch had previous discussion on LKML:
    http://lkml.org/lkml/2007/8/17/336

James Bottomley at one point obliquely referred to
my OLS2003 paper: "DMA Hints on ia64/PARISC"

After reading the thread, my take is we need a more elegant way for a
device driver to handle registration of DMA regions allocated by user
space. The API would "make this page/region act like dma_alloc_coherent()".
That implies strong ordering between CPU and DMA to/from the device.
Maybe the code is the right thing and I want a name that makes
sense in the context of current DMA API.

On IRC, willy suggested an mmap() flag and that sounds reasonable too
though I don't know if it's feasible. 

hth,
grant

> There are three patches in this set:
> 
> 	[1/3]: add pci_dma_flags_set_dmaflush() to pci interface
> 	[2/3]: redefine pci_dma_flags_set_dmaflush() for sn-ia64
> 	[3/3]: document pci_dma_flags_set_dmaflush()
> 
> And there would be additional patches to IB drivers to make use 
> of the interface, of course. 
> 
> -- 
> Arthur