From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD068C43381 for ; Mon, 18 Feb 2019 14:35:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7D94321736 for ; Mon, 18 Feb 2019 14:35:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388051AbfBROfG (ORCPT ); Mon, 18 Feb 2019 09:35:06 -0500 Received: from mx1.redhat.com ([209.132.183.28]:47716 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387748AbfBRN4U (ORCPT ); Mon, 18 Feb 2019 08:56:20 -0500 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3910DC05D41E; Mon, 18 Feb 2019 13:56:19 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-102.ams2.redhat.com [10.36.116.102]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2D5F95D717; Mon, 18 Feb 2019 13:56:14 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [PATCH v4 06/22] iommu: Introduce bind/unbind_guest_msi Date: Mon, 18 Feb 2019 14:54:47 +0100 Message-Id: <20190218135504.25048-7-eric.auger@redhat.com> In-Reply-To: <20190218135504.25048-1-eric.auger@redhat.com> References: <20190218135504.25048-1-eric.auger@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 18 Feb 2019 13:56:19 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On ARM, MSI are translated by the SMMU. An IOVA is allocated for each MSI doorbell. If both the host and the guest are exposed with SMMUs, we end up with 2 different IOVAs allocated by each. guest allocates an IOVA (gIOVA) to map onto the guest MSI doorbell (gDB). The Host allocates another IOVA (hIOVA) to map onto the physical doorbell (hDB). So we end up with 2 untied mappings: S1 S2 gIOVA -> gDB hIOVA -> hDB Currently the PCI device is programmed by the host with hIOVA as MSI doorbell. So this does not work. This patch introduces an API to pass gIOVA/gDB to the host so that gIOVA can be reused by the host instead of re-allocating a new IOVA. So the goal is to create the following nested mapping: S1 S2 gIOVA -> gDB -> hDB and program the PCI device with gIOVA MSI doorbell. Signed-off-by: Eric Auger --- v3 -> v4: - add unbind v2 -> v3: - add a struct device handle --- drivers/iommu/iommu.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/iommu.h | 25 +++++++++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index bcb8eb15426c..c9a06608496d 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1592,6 +1592,40 @@ static void __iommu_detach_device(struct iommu_domain *domain, trace_detach_device_from_domain(dev); } +/** + * iommu_bind_guest_msi - Passes the stage1 GIOVA/GPA mapping of the + * virtual doorbell used by the assigned device @dev. + * + * @domain: iommu domain the stage 1 mapping will be attached to + * @dev: assigned device which uses this stage1 mapping + * @iova: iova llocated by the guest + * @gpa: guest physical address of the virtual doorbell + * @size: granule size used for the mapping + * + * The associated IOVA can be reused by the host to create a nested + * stage2 binding mapping onto the physical doorbell used by @dev + */ + +int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + dma_addr_t giova, phys_addr_t gpa, size_t size) +{ + if (unlikely(!domain->ops->bind_guest_msi)) + return -ENODEV; + + return domain->ops->bind_guest_msi(domain, dev, giova, gpa, size); +} +EXPORT_SYMBOL_GPL(iommu_bind_guest_msi); + +void iommu_unbind_guest_msi(struct iommu_domain *domain, struct device *dev, + dma_addr_t iova) +{ + if (unlikely(!domain->ops->unbind_guest_msi)) + return; + + domain->ops->unbind_guest_msi(domain, dev, iova); +} +EXPORT_SYMBOL_GPL(iommu_unbind_guest_msi); + void iommu_detach_device(struct iommu_domain *domain, struct device *dev) { struct iommu_group *group; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index a3b879d0753c..1b685db845f8 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -190,6 +190,8 @@ struct iommu_resv_region { * @attach_pasid_table: attach a pasid table * @detach_pasid_table: detach the pasid table * @cache_invalidate: invalidate translation caches + * @bind_guest_msi: provides a stage1 giova/gpa MSI doorbell mapping + * @unbind_guest_msi: withdraw a stage1 giova/gpa MSI doorbell mapping */ struct iommu_ops { bool (*capable)(enum iommu_cap); @@ -239,6 +241,11 @@ struct iommu_ops { int (*cache_invalidate)(struct iommu_domain *domain, struct device *dev, struct iommu_cache_invalidate_info *inv_info); + int (*bind_guest_msi)(struct iommu_domain *domain, struct device *dev, + dma_addr_t giova, phys_addr_t gpa, size_t size); + void (*unbind_guest_msi)(struct iommu_domain *domain, + struct device *dev, dma_addr_t giova); + unsigned long pgsize_bitmap; }; @@ -355,6 +362,11 @@ extern void iommu_detach_pasid_table(struct iommu_domain *domain); extern int iommu_cache_invalidate(struct iommu_domain *domain, struct device *dev, struct iommu_cache_invalidate_info *inv_info); +extern int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + dma_addr_t giova, phys_addr_t gpa, size_t size); +extern void iommu_unbind_guest_msi(struct iommu_domain *domain, + struct device *dev, dma_addr_t giova); + extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev); extern struct iommu_domain *iommu_get_dma_domain(struct device *dev); extern int iommu_map(struct iommu_domain *domain, unsigned long iova, @@ -813,6 +825,19 @@ iommu_cache_invalidate(struct iommu_domain *domain, return -ENODEV; } +static inline +int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + dma_addr_t giova, phys_addr_t gpa, size_t size); +{ + return -ENODEV; +} +static inline +int iommu_unbind_guest_msi(struct iommu_domain *domain, struct device *dev, + dma_addr_t giova); +{ + return -ENODEV; +} + #endif /* CONFIG_IOMMU_API */ #ifdef CONFIG_IOMMU_DEBUGFS -- 2.20.1