Contiguous DMA buffer view for a custom device (Intel/x86)

Contiguous DMA buffer view for a custom device (Intel/x86)

[Alex Feinman]

Good morning.
I have a custom PCI device that needs to fill memory buffers of up to 16MB
in size. For various reasons I would like to avoid implementing
scatter-gather support in the device and would much rather prefer a
contiguous buffer view instead. The buffers are allocated using
dma_alloc_coherent or dma_alloc_attrs (as I don't necessarily need coherent
access)

This is on a custom Skylake/Kaby Lake platform

It is my understanding (possibly incorrect) that
a) The contiguous memory space available to subsystems like
videobuf2-dma-contig is limited and I won't be likely to be able tp
allocate say 4 16MB contiguous buffers reliably
b) IOMMU can solve this problem for me by providing a device-specific
contiguous view of a fragmented physical memory allocation
c) In order to enable IOMMU do the above, I need to allocate DRHDs and
DMARs in BIOS initialization (I build my own BIOS)

Please, let me know if I am on the right track? Of course I realize that
implementing SGDMA would be the best option, but short of that and blocking
out some physical memory on boot, what are my options?

Are there any pointers of how exactly achieve this (implementation I could
peruse, articles and whatnot)?

Best regards

Alex Feinman
ReplyForward

Re: Contiguous DMA buffer view for a custom device (Intel/x86)

[Alan Cox]

> b) IOMMU can solve this problem for me by providing a device-specific
> contiguous view of a fragmented physical memory allocation
> c) In order to enable IOMMU do the above, I need to allocate DRHDs and
> DMARs in BIOS initialization (I build my own BIOS)

Yes. The EDK2 firmware toolkit has all the bits you need in it I think.

https://firmware.intel.com/sites/default/files/Intel_WhitePaper_Using_IOMMU_for_DMA_Protection_in_UEFI.pdf

isn't quite on the topic you want but it does explain it fairly well in
passing.

https://software.intel.com/en-us/blogs/2009/03/02/intels-virtualization-for-directed-io-aka-iommu-part-1

is a bit out of date but may help too.

> Please, let me know if I am on the right track? Of course I realize that
> implementing SGDMA would be the best option, but short of that and blocking
> out some physical memory on boot, what are my options?

If performance is absolutely critical simply stealing a chunk of memory
in the firmware and describing it your device some other way is ugly, but
for a custom solution I guess anything goes 8)

Alan

Re: Contiguous DMA buffer view for a custom device (Intel/x86)

[Alex Feinman]

Thanks, this sounds like what I need. However, since this is a
custom-purpose system, I am tempted to go with CMA and simply
preallocated 200MB on start

On Mon, Aug 20, 2018 at 1:58 PM Alan Cox <gnomes@lxorguk.ukuu.org.uk> wrote:
>
> > b) IOMMU can solve this problem for me by providing a device-specific
> > contiguous view of a fragmented physical memory allocation
> > c) In order to enable IOMMU do the above, I need to allocate DRHDs and
> > DMARs in BIOS initialization (I build my own BIOS)
>
> Yes. The EDK2 firmware toolkit has all the bits you need in it I think.
>
> https://firmware.intel.com/sites/default/files/Intel_WhitePaper_Using_IOMMU_for_DMA_Protection_in_UEFI.pdf
>
> isn't quite on the topic you want but it does explain it fairly well in
> passing.
>
> https://software.intel.com/en-us/blogs/2009/03/02/intels-virtualization-for-directed-io-aka-iommu-part-1
>
> is a bit out of date but may help too.
>
> > Please, let me know if I am on the right track? Of course I realize that
> > implementing SGDMA would be the best option, but short of that and blocking
> > out some physical memory on boot, what are my options?
>
> If performance is absolutely critical simply stealing a chunk of memory
> in the firmware and describing it your device some other way is ugly, but
> for a custom solution I guess anything goes 8)
>
> Alan