From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936478AbdKQVwC (ORCPT ); Fri, 17 Nov 2017 16:52:02 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54538 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S936458AbdKQVvz (ORCPT ); Fri, 17 Nov 2017 16:51:55 -0500 Date: Fri, 17 Nov 2017 14:51:52 -0700 From: Alex Williamson To: Suravee Suthikulpanit Cc: linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, joro@8bytes.org, jroedel@suse.de Subject: Re: [PATCH 1/2] vfio/type1: Adopt fast IOTLB flush interface when unmap IOVAs Message-ID: <20171117145152.41fae930@t450s.home> In-Reply-To: <1510953080-5619-2-git-send-email-Suravee.Suthikulpanit@amd.com> References: <1510953080-5619-1-git-send-email-Suravee.Suthikulpanit@amd.com> <1510953080-5619-2-git-send-email-Suravee.Suthikulpanit@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 17 Nov 2017 21:51:55 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 17 Nov 2017 15:11:19 -0600 Suravee Suthikulpanit wrote: > From: Suravee Suthikulpanit > > VFIO IOMMU type1 currently upmaps IOVA pages synchronously, which requires > IOTLB flushing for every unmapping. This results in large IOTLB flushing > overhead when handling pass-through devices with a large number of mapped > IOVAs (e.g. GPUs). Of course the type of device is really irrelevant, QEMU maps the entire VM address space for any assigned device. > This can be avoided by using the new IOTLB flushing interface. > > Cc: Alex Williamson > Cc: Joerg Roedel > Signed-off-by: Suravee Suthikulpanit > --- > drivers/vfio/vfio_iommu_type1.c | 12 +++++++++--- > 1 file changed, 9 insertions(+), 3 deletions(-) > > diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c > index 92155cc..28a7ab6 100644 > --- a/drivers/vfio/vfio_iommu_type1.c > +++ b/drivers/vfio/vfio_iommu_type1.c > @@ -698,10 +698,12 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, > break; > } > > - unmapped = iommu_unmap(domain->domain, iova, len); > + unmapped = iommu_unmap_fast(domain->domain, iova, len); > if (WARN_ON(!unmapped)) > break; > > + iommu_tlb_range_add(domain->domain, iova, len); > + We should only add @unmapped, not @len, right? > unlocked += vfio_unpin_pages_remote(dma, iova, > phys >> PAGE_SHIFT, > unmapped >> PAGE_SHIFT, > @@ -710,6 +712,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, > > cond_resched(); > } > + iommu_tlb_sync(domain->domain); > > dma->iommu_mapped = false; > if (do_accounting) { > @@ -884,8 +887,11 @@ static int map_try_harder(struct vfio_domain *domain, dma_addr_t iova, > break; > } > > - for (; i < npage && i > 0; i--, iova -= PAGE_SIZE) > - iommu_unmap(domain->domain, iova, PAGE_SIZE); > + for (; i < npage && i > 0; i--, iova -= PAGE_SIZE) { > + iommu_unmap_fast(domain->domain, iova, PAGE_SIZE); > + iommu_tlb_range_add(domain->domain, iova, PAGE_SIZE); > + } > + iommu_tlb_sync(domain->domain); > > return ret; > }