Re: [PATCH v6 02/12] iommu: Add pasid_bits field in struct dev_iommu

[Baolu Lu <baolu.lu@linux.intel.com>]

On 2022/5/10 22:34, Jason Gunthorpe wrote:
> On Tue, May 10, 2022 at 02:17:28PM +0800, Lu Baolu wrote:
> 
>>   int iommu_device_register(struct iommu_device *iommu,
>> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> index 627a3ed5ee8f..afc63fce6107 100644
>> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
>>   	    smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
>>   		master->stall_enabled = true;
>>   
>> +	dev->iommu->pasid_bits = master->ssid_bits;
>>   	return &smmu->iommu;
>>   
>>   err_free_master:
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 2990f80c5e08..99643f897f26 100644
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
>>   			if (pasid_supported(iommu)) {
>>   				int features = pci_pasid_features(pdev);
>>   
>> -				if (features >= 0)
>> +				if (features >= 0) {
>>   					info->pasid_supported = features | 1;
>> +					dev->iommu->pasid_bits =
>> +						fls(pci_max_pasids(pdev)) - 1;
>> +				}
> 
> It is not very nice that both the iommu drivers have to duplicate the
> code to read the pasid capability out of the PCI device.
> 
> IMHO it would make more sense for the iommu layer to report the
> capability of its own HW block only, and for the core code to figure
> out the master's limitation using a bus-specific approach.

Fair enough. The iommu hardware capability could be reported in

/**
  * struct iommu_device - IOMMU core representation of one IOMMU hardware
  *                       instance
  * @list: Used by the iommu-core to keep a list of registered iommus
  * @ops: iommu-ops for talking to this iommu
  * @dev: struct device for sysfs handling
  */
struct iommu_device {
         struct list_head list;
         const struct iommu_ops *ops;
         struct fwnode_handle *fwnode;
         struct device *dev;
};

I haven't checked ARM code yet, but it works for x86 as far as I can
see.

> 
> It is also unfortunate that the enable/disable pasid is inside the
> iommu driver as well - ideally the PCI driver itself would do this
> when it knows it wants to use PASIDs.
> 
> The ordering interaction with ATS makes this look quite annoying
> though. :(
> 
> I'm also not convinced individual IOMMU drivers should be forcing ATS
> on, there are performance and functional implications here. Using ATS
> or not is possibly best left as an administrator policy controlled by
> the core code. Again we seem to have some mess.

Agreed with you. This has already been in my task list. I will start to
solve it after the iommufd tasks.

Best regards,
baolu