Re: [PATCH] intel-iommu: Obey coherent_dma_mask for alloc_coherent on passthrough

[Alex Williamson <alex.williamson@hp.com>]

On Mon, 2009-11-09 at 23:02 +0000, David Woodhouse wrote:
> On Fri, 2009-11-06 at 11:41 +0900, FUJITA Tomonori wrote:
> > This is fine for 2.6.32 but we'll cleanly fix this by using
> > swiotlb_dma_ops later, right?
> 
> Well, the idea was that with 'iommu=pt' we'd have passthrough mode for
> _decent_ devices, but the crappy devices without 64-bit DMA would just
> have the IOMMU enabled instead.
> 
> We can see this as a simple classification bug -- we should be checking
> pdev->coherent_dma_mask as well as pdev->dma_mask:
> 
> --- a/drivers/pci/intel-iommu.c
> +++ b/drivers/pci/intel-iommu.c
> @@ -2196,7 +2196,8 @@ static int iommu_should_identity_map(struct pci_dev *pdev, int startup)
>  	 * take them out of the 1:1 domain later.
>  	 */
>  	if (!startup)
> -		return pdev->dma_mask > DMA_BIT_MASK(32);
> +		return (pdev->dma_mask & pdev->coherent_dma_mask) < 
> +			dma_get_required_mask();
>  
>  	return 1;
>  }
> 
> That fixes the case of a 32-bit coherent_dma_mask as it was intended to
> be fixed.
> 
> Unfortunately, Alex's hardware is more broken than that. It also likes
> to do stray reads from unmapped addresses -- addresses which used to be
> mapped at some time in the past. So he really _does_ want the IOMMU
> disabled, or in passthrough mode.
> 
> But I think that's a special case and needs to be handled with a quirk,
> while the above patch actually addresses the problem we claimed we were
> trying to address.

I'm not sure what this quirk looks like, do you have something in mind?

> Handling Alex's broken hardware probably wants to be done with
> 'iommu=off' for now, and then when Chris's swiotlb fallback patches are
> done we can perhaps do something more cunning.

iommu=off means a feature regression from 2.6.31 and kills support for
being able to use VT-d for virtualization for a large percentage of
servers from a major vendor.  I don't think Chris' patches actually
address this since we don't actually know what the DMA mask is for a
device until the driver claims it.  How long do we wait before we drop
the swiotlb?  I think his patch is really intended for the "oops, the
DMAR is broken, the hardware is bad, I can't init the hardware IOMMU,
whew we can fallback to swiotlb".

> I'm slightly reluctant to put the half-arsed 'try to allocate in the
> right region for broken devices but without full swiotlb support' option
> into 2.6.32.

Since the device also makes use of RMRRs, once we have it in the
si_domain, we're stuck.  I think that means we needs swiotlb anytime
we're in passthrough mode.  That's what 2.6.31, can we get it back for
2.6.32?  Thanks,

Alex