Re: [RFC PATCH v1 3/4] vfio: Try to enable IOPF for VFIO devices

[Alex Williamson <alex.williamson@redhat.com>]

On Mon, 25 Jan 2021 17:04:01 +0800
Shenming Lu <lushenming@huawei.com> wrote:

> If IOMMU_DEV_FEAT_IOPF is set for the VFIO device, which means that
> the delivering of page faults of this device from the IOMMU is enabled,
> we register the VFIO page fault handler to complete the whole faulting
> path (HW+SW). And add a iopf_enabled field in struct vfio_device to
> record it.
> 
> Signed-off-by: Shenming Lu <lushenming@huawei.com>
> ---
>  drivers/vfio/vfio.c | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
> 
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index ff7797260d0f..fd885d99ee0f 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -97,6 +97,7 @@ struct vfio_device {
>  	struct vfio_group		*group;
>  	struct list_head		group_next;
>  	void				*device_data;
> +	bool				iopf_enabled;
>  };
>  
>  #ifdef CONFIG_VFIO_NOIOMMU
> @@ -532,6 +533,21 @@ static struct vfio_group *vfio_group_get_from_dev(struct device *dev)
>  /**
>   * Device objects - create, release, get, put, search
>   */
> +
> +static void vfio_device_enable_iopf(struct vfio_device *device)
> +{
> +	struct device *dev = device->dev;
> +
> +	if (!iommu_dev_has_feature(dev, IOMMU_DEV_FEAT_IOPF))
> +		return;
> +
> +	if (WARN_ON(iommu_register_device_fault_handler(dev,
> +					vfio_iommu_dev_fault_handler, dev)))

The layering here is wrong, vfio-core doesn't manage the IOMMU, we have
backend IOMMU drivers for that.  We can't even assume we have IOMMU API
support here, that's what the type1 backend handles.  Thanks,

Alex

> +		return;
> +
> +	device->iopf_enabled = true;
> +}
> +
>  static
>  struct vfio_device *vfio_group_create_device(struct vfio_group *group,
>  					     struct device *dev,
> @@ -549,6 +565,8 @@ struct vfio_device *vfio_group_create_device(struct vfio_group *group,
>  	device->group = group;
>  	device->ops = ops;
>  	device->device_data = device_data;
> +	/* By default try to enable IOPF */
> +	vfio_device_enable_iopf(device);
>  	dev_set_drvdata(dev, device);
>  
>  	/* No need to get group_lock, caller has group reference */
> @@ -573,6 +591,8 @@ static void vfio_device_release(struct kref *kref)
>  	mutex_unlock(&group->device_lock);
>  
>  	dev_set_drvdata(device->dev, NULL);
> +	if (device->iopf_enabled)
> +		WARN_ON(iommu_unregister_device_fault_handler(device->dev));
>  
>  	kfree(device);
>