Re: [PATCH v8 16/23] dma-buf: Let importers ask how peer-to-peer traffic is routed

["Christian König" <christian.koenig@amd.com>]

On 9/30/26 16:57, Thomas Hellström wrote:
> On Wed, 2026-09-30 at 17:32 +0300, Leon Romanovsky wrote:
>> On Wed, Sep 30, 2026 at 01:52:39PM +0200, Christian König wrote:
>>> On 9/30/26 13:43, Leon Romanovsky wrote:
>>>> On Wed, Sep 30, 2026 at 10:34:56AM +0200, Christian König wrote:
>>>>> On 9/30/26 10:18, Leon Romanovsky wrote:
>>> ...
>>>>>>
>>>>>> At a minimum, exporters need to pass `p2pdma_provider`.
>>>>>
>>>>> No, exactly that is a no-go. The neither the framework nor the
>>>>> importer should see the p2pdma_provider.
>>>>>
>>>>> Only fully translated addresses where the DMA access should
>>>>> happen.
>>>>>
>>>>>>
>>>>>> If I keep the “dma-buf: Let exporters hand out the P2PDMA
>>>>>> provider behind a
>>>>>> buffer” patch, I can move the P2P TLP types back into
>>>>>> `p2pdma.c` and export
>>>>>> only the function that indicates whether ATS is required.
>>>>>>
>>>>>> Is it ok?
>>>>>
>>>>> What you can do is to forward declare enum pci_p2pdma_map_type
>>>>> and than pass that 1 to 1 from the exporter to the importer.
>>>>
>>>> Unfortunately, neither suggestion applies to RDMA NICs. They need
>>>> to know,
>>>> before mapping addresses, whether to create the memory region
>>>> with ATS
>>>> enabled.
>>>
>>> The design principle here is that the final location and access
>>> path of the data isn't determined when the buffer is created.
>>>
>>> The importer first need to attach before it can query such
>>> information from the exporter.
>>
>> In attach yes, this is why importer digs in dma_buf ops to get
>> p2pdma_provide, however it is before addresses are known.
>>
>>>> The importer needs a way to obtain device information from the
>>>> exporter so
>>>> that it can configure itself correctly.
>>>
>>> That won't work with DMA-buf then, the exporter is completely
>>> opaque to the importer and that is for really good reasons.
>>>
>>> Why in the world does the importer needs to know the information
>>> from the exporter before the mapping is created?
>>>
>>> It is the exporter who decides how data is accessed by the importer
>>> and not the other way around.
>>
>> There are several reasons:
>>
>> 1. This is how DMA-BUF MRs are built in RDMA. In mlx5, they rely on
>> the ODP
>>    mechanism, which requires an MKEY to be created first. See commit
>>    90da7dc8206a (“RDMA/mlx5: Support dma-buf based userspace memory
>> region”).
>> 2. P2P routing is a property of devices, not memory. It is known and
>> remains
>>    stable.
>> 3. See the VFIO TPH ST discussion, where the requirement to obtain
>> the
>>    exporter’s P2P information in the importer was raised again.
>>
>> Thanks
> 
> 
> Returning again to Jason's series. Let's say we'd add just the mapping
> type infrastructure, converted users of pcie_p2pdma only to use that
> and then we'd have access to per-mapping-type data. This could actually
> be done as a prereq for this series and merged separately. It's a
> couple of patches only.
> 
> Jason's match() and finish() callbacks could compute the interesting
> routes at attach time,  perhaps even condesed to whether IOVA is used
> and whether ATS translated packages have a direct route (which is what
> mlx5 care about AFAICT). This information is kept outside core dma-buf
> and would be specific to the pcie_p2p mapping type (interconnect) only
> rather than having functions and callbacks bloating the core dma-buf
> structures.

We need to move that to the mapping time because that is when the final location is committed, but apart from that the approach sounds sane to me.

Otherwise the information can change any time there is a new importer attached.

DMA-buf is basically just the frontend for passing mapping information between the exporter and the importer here.

Regards,
Christian.
> Then exactly where the cross-subsystem match() and finish()
> implementations should live I figure remain up for discussion and
> guidance by Christoph?
> 
> Thanks,
> Thomas
> 
> 
>