Re: [PATCH v2 4/5] iommu: Use dma_addr_t for IOVA arguments

[James Bottomley <jbottomley@parallels.com>]

On Fri, 2014-05-09 at 14:19 -0600, Bjorn Helgaas wrote:
> On Fri, May 9, 2014 at 1:52 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Friday 09 May 2014 09:32:26 Bjorn Helgaas wrote:
> >> > There are people who care deeply about the performance of IOMMU API
> >> > map/unmap. It isn't used *just* for virtual machines any more. See
> >> > drivers/infiniband/hw/usnic/usnic_uiom.c for example.
> >>
> >> Of course we should care about IOMMU API performance.  We should also
> >> care about interface consistency, and it seems there's a tradeoff in
> >> this case.  I said "relatively" because I expect map/unmap to be less
> >> frequent than read/write operations that use the mapping.  I don't
> >> know anything about infiniband, so maybe that assumption is false
> >> there.
> >
> > In most drivers using the streaming DMA API, every mapping is used
> > exactly once. Think of network or block drivers: they rarely send
> > the same data twice to the device, and it usually comes from or
> > goes to some user space buffer.
> 
> Oh, good point.  I don't work that high up in the stack, so thanks for
> reminding me of that.

To round this out, for most devices we have two types of mappings: the
mailbox ones, which designate regions of communication memory between
the kernel and the device which are usually permanent mappings, and the
transmission mappings: every bit of data we send to the device is
mapped, sent/received and then unmapped.  The setup and teardown costs
factor into the throughput.  Some high iops devices (like SSD or high
speed net) are peculiarly sensitive to this.

James