Re: How to efficiently handle DMA and cache on ARMv7 ? (was "Is get_user_pages() enough to prevent pages from being swapped out ?")

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

On Friday 07 August 2009 21:01:45 Russell King - ARM Linux wrote:
> On Fri, Aug 07, 2009 at 08:15:01AM -0500, Robin Holt wrote:
> > On Fri, Aug 07, 2009 at 02:07:43PM +0200, Laurent Desnogues wrote:
> > > On Fri, Aug 7, 2009 at 11:54 AM, Jamie Lokier<jamie@shareable.org> 
wrote:
> > > > 1. Does the architecture not prevent speculative instruction
> > > > prefetches from crossing a page boundary?  It would be handy under
> > > > the circumstances.
> > >
> > > There's no such restriction in ARMv7 architecture.
> >
> > Doesn't it prevent them for uncached areas?
>
> "Uncached areas" is very very fuzzy.  Are you talking about a non-cachable
> memory mapping, or a strongly ordered mapping.
>
> I'm afraid that we're going to have to require more precise use of language
> to describe these things - wolley statements like "uncached areas" are now
> just too ambiguous.

Ok. Maybe the kernel mapping from L_PTE_MT_UNCACHED to strongly ordered for 
ARMv6 and up (not sure about how it worked for previous versions) brought some 
confusion. I'll try to be more precise now.

> > I _THOUGHT_ there was an alloc_consistent (or something like that) call on
> > ARM which gave you an uncached mapping where you could do DMA.
>
> The dma_alloc_coherent() does _remap_ memory into a strongly ordered
> mapping.  However, the fully cached mapping remains, which means that
> the CPU can still speculatively prefetch from that memory.

Does that mean that, in theory, all DMA transfers in the DMA_FROM_DEVICE 
direction are currently broken on ARMv7 ?

The ARM Architecture Reference Manual (ARM DDI 0100I) states that

"• If the same memory locations are marked as having different memory types 
(Normal, Device, or Strongly Ordered), for example by the use of synonyms in a 
virtual to physical address mapping, UNPREDICTABLE behavior results.

• If the same memory locations are marked as having different cacheable 
attributes, for example by the use of synonyms in a virtual to physical 
address mapping, UNPREDICTABLE behavior results."

dma_alloc_coherent() ends up calling __dma_alloc(), which allocates pages 
using alloc_pages(), flushes the data cache for the allocated virtual range 
and then simply remaps the pages using PTEs previously allocated from the 
kernel MM.

This would be broken if a fully cached Normal mapping already existed for 
those physical pages. You seem to imply that's the case, but I'm not sure to 
understand why.

> Since we map the fully cached mapping using section (or even supersection)
> mappings for TLB efficiency, we can't change the memory type on a
> per-page basis.
>
> > I also thought there was a dma_* set of functions which remapped as
> > uncached before DMA begins and remapped as normal after DMA has been
> > completed.
>
> You're talking about the deprecated DMA bounce code there.  It's
> basically the same problem since it uses the dma_alloc_coherent()
> interface to gain a source of DMA-able memory.

Regards,

Laurent Pinchart