Re: IOMMUs was Re: Intel vs AMD x86-64

[Andi Kleen <ak@suse.de>]

On Tue, 24 Feb 2004 10:13:40 -0800
"David S. Miller" <davem@redhat.com> wrote:

> On 24 Feb 2004 15:06:47 +0100
> Andi Kleen <ak@suse.de> wrote:
> 
> > One side effect of this is that the IOMMU TLB flush strategy is a bit
> > dumb, because it has to do config space accesses for it.
> 
> This can be costly, but if you flush the IOMMU like sparc64 does (basically
> it's similar to how KMAPs are flushed on x86), the cost gets real low because
> then you only flush the whole iommu once every time you walk the whole mapping
> table of the iommu.
> 
> I'm sure you've probably thought of this already, just mentioning it in case
> you haven't.

Arjan suggested it some time ago already. In fact I implemented it, it's in the current code.
But it caused data corruption with a few devices, in particular 3ware, so I had 
to disable it again. I didn't find a bug in the code. It worked fine with others. My theory 
was that it triggered some hardware bug that was normally masked by the frequent flushes, but 
I wasn't able to track it down without heavy equipment.

Currently it is in there, but disabled by default. Can be enabled with iommu=nofullflush.

Also the other part of the dumbness is that the flush is global, not per mapping. I guess
you don't have that problem on Sparc64.

Anyways, even with these restrictions having the GART as IOMMU is much better than
doing software bouncing.

-Andi