Re: allocation failed: out of vmalloc space error treating and VIDEO1394 IOC LISTEN CHANNEL ioctl failed problem

["Kristian Høgsberg" <krh@bitplanet.net>]

On 1/15/07, David Moore <dcm@mit.edu> wrote:
> On Mon, 2007-01-15 at 10:20 -0800, Arjan van de Ven wrote:
> > if you need that much you probably should redesign your algorithms to
> > not need vmalloc in the first place....
>
> I think you've convinced me that vmalloc is not a good choice when a
> driver needs a large buffer (many megabytes) for DMA.
>
> In this case, we need a large ring buffer for reception of isochronous
> packets from a firewire device.  If I understand you correctly, you are
> suggesting that this buffer be obtained as followed:
>
> 1. Application performs malloc() in user-space and mmap()s it.
> 2. Driver uses vmalloc_to_page() on every page of the malloc'ed memory
> and constructs a scatter-gather list.
> 3. Map the sg list with pci_map_sg().
> 4. Commence DMA.
>
> Is that correct?  In particular, does it do the right thing in terms of
> pinning the memory and dealing with high memory?
>
> I notice that the block I/O API has some convenience functions for this,
> but this is not a block device.  Are there some other convenience
> functions that can be used?
>
> Forgive me if these are obvious questions -- I'm not the developer of
> video1394, but I'd still like get it right for the new firewire stack
> that's being developed.

David, thanks for bringing this up.  Indeed if vmalloc is not the
right way for allocating big buffers, we need to figure something else
out.  My impression was that the vmalloc group of functions could be
used for big allocations since they don't require the underlying
memory to be physically contiguous.

What I'm doing currently in the new firewire stack is to vmalloc the
memory to be used for isochronous payload and then use
remap_vmalloc_range() to map the memory to the user.  I never access
the contents from the kernel side, I just use vmalloc so I can pass
the pointer to remap_vmalloc_range().  Maybe this is overkill and a
better way to do this is to call get_page() a number of times and
manually add these pages to the process address space without ever
setting up a kernel side mapping for these.

However, what I'd really like to do is to leave it to user space to
allocate the memory as David describes.  In the transmit case, user
space allocates memory (malloc or mmap) and loads the payload into
that buffer.  Then is does an ioctl() on the firewire control device
to indicate the location of this buffer, describe how that payload is
to be split into packets, and optionally a header per packet to
prepend.  The kernel side driver then converts the user space
addresses to pages, pins the pages in question, and sets up dma
programs to transmit the packets.

Likewise for reception, user space allocates buffers for receiving the
data and then instructs the kernel the receive a certain amount of
data into this buffer.  The kernel pins the pages backing the user
space buffer and sets up dma to received into those pages.  Once a
page it full, it's unpinned and userspace is notified.

It's not too difficult from what I'm doing now, I'd just like to give
user space more control over the buffers it uses for streaming (i.e.
letting user space allocate them).  What I'm missing here is: how do I
actually pin a page in memory?  I'm sure it's not too difficult, but I
haven't yet figured it out and I'm sure somebody knows it off the top
of his head.

cheers,
Kristian