Re: [PATCH v6 17/18] dma-buf: Let importers ask how peer-to-peer traffic is routed

[Leon Romanovsky <leon@kernel.org>]

On Mon, Sep 14, 2026 at 04:27:00PM +0200, Christian König wrote:
> On 9/14/26 13:22, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@nvidia.com>
> > 
> > Exporters keep the &struct p2pdma_provider backing a buffer in their own
> > private data. An importer cannot reach it, so it has no way to learn how
> > its own peer-to-peer traffic would be routed before it programs its
> > hardware.
> > 
> > Add an optional @p2pdma_provider callback for an exporter to hand that
> > provider out, and dma_buf_p2pdma_map_type() for an importer to ask by TLP
> > class. Exporters keep the provider where it already lives, so this adds an
> > operation rather than changing any existing signature or structure.
> > 
> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > ---
> >  drivers/dma-buf/dma-buf-mapping.c                 | 32 +++++++++++++++++++++++
> >  drivers/infiniband/core/uverbs_std_types_dmabuf.c | 12 +++++++++
> >  drivers/vfio/pci/vfio_pci_dmabuf.c                | 12 +++++++++
> >  include/linux/dma-buf-mapping.h                   |  3 +++
> >  include/linux/dma-buf.h                           | 17 ++++++++++++
> >  5 files changed, 76 insertions(+)
> > 
> > diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-mapping.c
> > index 794acff2546a..fb03a698b381 100644
> > --- a/drivers/dma-buf/dma-buf-mapping.c
> > +++ b/drivers/dma-buf/dma-buf-mapping.c
> > @@ -6,6 +6,38 @@
> >  #include <linux/dma-buf-mapping.h>
> >  #include <linux/dma-resv.h>
> >  
> > +/**
> > + * dma_buf_p2pdma_map_type - How peer-to-peer traffic to a buffer is routed
> > + * @attach:	attachment of the importer that will issue the traffic
> > + * @tlp_flags:	&enum pci_p2pdma_tlp_flags describing the TLPs it will issue
> > + *
> > + * Reports how the PCIe fabric routes @tlp_flags traffic between the buffer
> > + * behind @attach and the importer attached to it, so that an importer can
> > + * choose the TLP attributes that earn it a direct route before it programs
> > + * its hardware.
> > + *
> > + * Return: the mapping type for @tlp_flags traffic, or PCI_P2PDMA_MAP_NONE
> > + * when the exporter names no &struct p2pdma_provider and nothing is known
> > + * about the route.
> > + */
> > +enum pci_p2pdma_map_type
> > +dma_buf_p2pdma_map_type(struct dma_buf_attachment *attach,
> > +			unsigned int tlp_flags)
> > +{
> > +	struct dma_buf *dmabuf = attach->dmabuf;
> > +	struct p2pdma_provider *provider;
> > +
> > +	if (!dmabuf->ops->p2pdma_provider)
> > +		return PCI_P2PDMA_MAP_NONE;
> > +
> > +	provider = dmabuf->ops->p2pdma_provider(dmabuf);
> > +	if (!provider)
> > +		return PCI_P2PDMA_MAP_NONE;
> > +
> > +	return pci_p2pdma_map_type_tlp(provider, attach->dev, tlp_flags);
> 
> Calling PCI subsystem functions from dma-buf is a hard NO-GO.

There are two main reasons for this design:

1. DMA-buf's broken use of scatter-gather lists.
2. Hellwig's NAK on exporting low-level PCI P2P functions. These should be
   used by core code, while in-kernel exporters and importers may be built
   as modules and therefore need EXPORT_SYMBOL() to access PCI P2P
   functionality.

> 
> Then wrappers for DMA-buf backend functions should be in dma-buf.c and not here.
> 
> I shouldn't have allowed the mapping functions to be added to DMA-buf in the first place, all of this belongs either into the DMA layer or the exporter.

Due to item #2, they cannot be in the exporters. Ideally, the DMA layer
should not have PCI-specific paths.

> 
> We should probably have a single callback the exporter provides to fill in a structure with PCI specific information for a mapping.

If you have an idea how to do this without adding EXPORT_SYMBOL to p2p,
let me know and I will implement it.

DMA-buf is the de facto mechanism for setting up and using p2p between
devices. There is no other mechanism (except NVMe which is unique)
in the kernel to achieve this. IMHO, the dmabuf code should try to implement
as much of the common functionality as possible, so that dmabuf consumers
do not have to reinvent their own p2p implementations.

> 

<...>

> > +
> >  static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
> >  	.attach = vfio_pci_dma_buf_attach,
> >  	.map_dma_buf = vfio_pci_dma_buf_map,
> >  	.unmap_dma_buf = vfio_pci_dma_buf_unmap,
> >  	.release = vfio_pci_dma_buf_release,
> > +	.p2pdma_provider = vfio_pci_dma_buf_provider,
> 
> Please split up the patch into the actual DMA-buf changes, exporter changes and importer changes.

Sure, will do.

Thanks