Re: DMA-buf and uncached system memory

["Christian König" <christian.koenig@amd.com>]

Am 23.06.22 um 10:04 schrieb Lucas Stach:
> Am Donnerstag, dem 23.06.2022 um 09:26 +0200 schrieb Christian König:
>> Am 23.06.22 um 09:13 schrieb Pekka Paalanen:
>>> On Thu, 23 Jun 2022 08:59:41 +0200
>>> Christian König <christian.koenig@amd.com> wrote:
>>>
>>>> The exporter isn't doing anything wrong here. DMA-buf are supposed to be
>>>> CPU cached and can also be cache hot.
>>> Hi,
>>>
>>> what is that statement based on?
>> On the design documentation of DMA-buf and the actual driver
>> implementations.
>>
>> Coherency and snooping of the CPU cache is mandatory for devices and
>> root complexes in the PCI specification. Incoherent access is just an
>> extension.
>>
>> We inherited that by basing DMA-buf on the Linux kernel DMA-API which in
>> turn is largely based on the PCI specification.
>>
>>> Were the (mandatory for CPU access) cpu_access_begin/end functions &
>>> ioctls not supposed to ensure that CPU cache is up-to-date / CPU cache
>>> is fully flushed out?
>> No, those functions are to inform the exporter that the importer has
>> started and finished accessing the buffer using the CPU.
>>
>> There is no signaling in the other direction. In other words the
>> exporter doesn't inform the importer about CPU accesses because it is
>> the owner of the buffer.
>>
>> It's the responsibility of the importer to make sure that it can
>> actually access the data in the buffer. If it can't guarantee that the
>> importer shouldn't import the buffer in the first place.
> This is not really correct. DMA-buf inherited the the map/unmap part
> from the DMA API, which on cache coherent architecture is mostly a no-
> op or ties into the IOMMU implementation to set up the pagetables for
> the translation. On non cache coherent architectures this is the point
> where any any necessary cache maintenance happens. DRM breaks this
> model by caching the DMA-buf mapping for performance reasons.

That's not only because of performance reasons, but also because of 
correctness.

At least the Vulkan API and a bunch of OpenGL extensions make it 
mandatory for the buffer to be cache coherent. The kernel is simply not 
informed about domain transfers.

For example you can just do a CPU copy to a ring buffer and the 
expectation is that an already running shader sees that.

> In the DMA API keeping things mapped is also a valid use-case, but then
> you need to do explicit domain transfers via the dma_sync_* family,
> which DMA-buf has not inherited. Again those sync are no-ops on cache
> coherent architectures, but do any necessary cache maintenance on non
> coherent arches.

Correct, yes. Coherency is mandatory for DMA-buf, you can't use 
dma_sync_* on it when you are the importer.

The exporter could of course make use of that because he is the owner of 
the buffer.

Regards,
Christian.

>
> Regards,
> Lucas
>