mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>,
	"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
	"Ian.Campbell@citrix.com" <Ian.Campbell@citrix.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	"konrad.wilk@oracle.com" <konrad.wilk@oracle.com>
Subject: Re: [PATCH v5 11/13] xen: introduce xen_alloc/free_coherent_pages
Date: Fri, 6 Sep 2013 17:52:57 +0100	[thread overview]
Message-ID: <alpine.DEB.2.02.1309061740050.6397@kaball.uk.xensource.com> (raw)
In-Reply-To: <20130906162018.GJ1946@arm.com>

On Fri, 6 Sep 2013, Catalin Marinas wrote:
> On Fri, Sep 06, 2013 at 05:09:52PM +0100, Stefano Stabellini wrote:
> > On Fri, 6 Sep 2013, Catalin Marinas wrote:
> > > On Fri, Sep 06, 2013 at 03:59:02PM +0100, Stefano Stabellini wrote:
> > > > On Fri, 6 Sep 2013, Catalin Marinas wrote:
> > > > > On Thu, Sep 05, 2013 at 05:43:33PM +0100, Stefano Stabellini wrote:
> > > > > > On Thu, 5 Sep 2013, Catalin Marinas wrote:
> > > > > > > On Thu, Aug 29, 2013 at 07:32:32PM +0100, Stefano Stabellini wrote:
> > > > > > > > xen_swiotlb_alloc_coherent needs to allocate a coherent buffer for cpu
> > > > > > > > and devices. On native x86 and ARMv8 is sufficient to call
> > > > > > > > __get_free_pages in order to get a coherent buffer, while on ARM we need
> > > > > > > > to call arm_dma_ops.alloc.
> > > > > > > 
> > > > > > > Don't bet on this for ARMv8. It's not mandated for the architecture, so
> > > > > > > at some point some SoC will require non-cacheable buffers for coherency.
> > > > > > 
> > > > > > I see.
> > > > > > Would it be better if I implemented xen_alloc_coherent_pages on armv8 by
> > > > > > calling arm64_swiotlb_dma_ops.alloc?
> > > > > 
> > > > > What does this buffer do exactly? Is it allocated by guests?
> > > > 
> > > > It is allocated by Dom0 to do DMA to/from a device.
> > > > It is the buffer that is going to be returned by dma_map_ops.alloc to
> > > > the caller:
> > > > 
> > > > On x86:
> > > > dma_map_ops.alloc -> xen_swiotlb_alloc_coherent -> xen_alloc_coherent_pages -> __get_free_pages
> > > > 
> > > > On ARM:
> > > > dma_map_ops.alloc -> xen_swiotlb_alloc_coherent -> xen_alloc_coherent_pages -> arm_dma_ops.alloc
> > > > 
> > > > On ARM64
> > > > dma_map_ops.alloc -> xen_swiotlb_alloc_coherent -> xen_alloc_coherent_pages -> ????
> > > 
> > > OK, I'm getting more confused. Do all the above calls happen in the
> > > guest, Dom0, or a mix?
> > 
> > I guess the confusion comes from a difference in terminology: dom0 is a
> > guest like the others, just a bit more privileged. We usually call domU
> > a normal unprivileged guest.
> 
> Thanks for the explanation.
> 
> > The above calls would happen in Dom0 (when an SMMU is not available).
> 
> So for Dom0, are there cases when it needs xen_swiotlb_alloc_coherent()
> and other cases when it needs the arm_dma_ops.alloc? In Dom0 could we
> not always use the default dma_alloc_coherent()?

Keep in mind that dom0 runs with second stage translation enabled.  This
means that what dom0 thinks is a physical address (machine address in
Xen terminology), it's actually just an intermediate physical address.
Also for the same reason what dom0 thinks is a contiguous buffer, it's
actually only contiguous in the intermediate physical address space.

So every time dom0 wants to allocate a dma-capable buffer it needs to go
via swiotlb-xen, that makes the buffer contiguous in the physical address
space (machine address space in Xen terminology) by issuing an hypercall.
swiotlb-xen also returns the physical address (machine address in Xen
terminology) to the caller.

To answer your question: in absence of an SMMU, all the
dma_alloc_coherent calls in dom0 need to go via xen_swiotlb_alloc_coherent.

xen_swiotlb_alloc_coherent cannot allocate a contigous buffer in physical
address space (see above), but it has to allocate a buffer coherent from
the caching attributes point of view. The hypervisor is going to take
care of making the allocated buffer really contiguous in physical address
space.

So now the problem is: how is xen_swiotlb_alloc_coherent going to
allocate a coherent buffer?
On x86 I can just call __get_free_pages.
On ARM I have to call arm_dma_ops.alloc.
On ARM64 ???

BTW if the Matrix is your kind of fun, I wrote an blog post explaining the
swiotlb Morpheus style:
http://blog.xen.org/index.php/2013/08/14/swiotlb-by-morpheus/


> > They could also happen in a DomU if we assign a physical device to it
> > (and an SMMU is not available).
> 
> The problem is that you don't necessarily know one kind of coherency you
> know for a physical device. As I said, we plan to do this DT-driven.
 
OK, but if I call arm64_swiotlb_dma_ops.alloc passing the right
arguments to it, I should be able to get the right coherency for the
right device, correct?

  reply	other threads:[~2013-09-06 16:55 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-29 18:31 [PATCH v5 0/13] enable swiotlb-xen on arm and arm64 Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 01/13] arm: make SWIOTLB available Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 02/13] arm: introduce a global dma_ops pointer Stefano Stabellini
2013-09-02 14:45   ` [Xen-devel] " Julien Grall
2013-09-05 16:39     ` Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 03/13] arm64: define DMA_ERROR_CODE Stefano Stabellini
2013-09-05 16:05   ` Catalin Marinas
2013-09-05 16:54     ` Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 04/13] arm64: do not initialize arm64_swiotlb if dma_ops is already set Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 05/13] xen/arm,arm64: move Xen initialization earlier Stefano Stabellini
2013-09-05 16:20   ` Catalin Marinas
2013-09-05 16:59     ` Stefano Stabellini
2013-09-06  8:58       ` Ian Campbell
2013-09-06 14:09         ` Catalin Marinas
2013-09-06 14:23           ` Konrad Rzeszutek Wilk
2013-09-06 14:53             ` Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 06/13] xen: introduce XENMEM_exchange_and_pin and XENMEM_unpin Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 07/13] xen: make xen_create_contiguous_region return the dma address Stefano Stabellini
2013-08-30 13:45   ` Konrad Rzeszutek Wilk
2013-08-29 18:32 ` [PATCH v5 08/13] swiotlb-xen: support autotranslate guests Stefano Stabellini
2013-08-30 13:46   ` Konrad Rzeszutek Wilk
2013-08-29 18:32 ` [PATCH v5 09/13] xen/arm,arm64: enable SWIOTLB_XEN Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 10/13] swiotlb-xen: introduce xen_swiotlb_set_dma_mask Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 11/13] xen: introduce xen_alloc/free_coherent_pages Stefano Stabellini
2013-09-05 16:09   ` Catalin Marinas
2013-09-05 16:43     ` Stefano Stabellini
2013-09-06 14:14       ` Catalin Marinas
2013-09-06 14:59         ` Stefano Stabellini
2013-09-06 15:59           ` Catalin Marinas
2013-09-06 16:09             ` Stefano Stabellini
2013-09-06 16:20               ` Catalin Marinas
2013-09-06 16:52                 ` Stefano Stabellini [this message]
2013-09-09 15:51                   ` Catalin Marinas
2013-09-09 16:46                     ` Stefano Stabellini
2013-09-11  9:36                       ` Catalin Marinas
2013-09-11 17:34                         ` Stefano Stabellini
2013-09-12 13:53                           ` Catalin Marinas
2013-09-12 14:44                             ` Stefano Stabellini
2013-09-12 15:04                               ` Catalin Marinas
2013-08-29 18:32 ` [PATCH v5 12/13] swiotlb-xen: use xen_alloc/free_coherent_pages Stefano Stabellini
2013-08-30 13:53   ` Konrad Rzeszutek Wilk
2013-09-05 16:50     ` Stefano Stabellini
2013-09-06 14:17       ` Konrad Rzeszutek Wilk
2013-09-06 15:04         ` Stefano Stabellini
2013-08-29 18:32 ` [PATCH v5 13/13] swiotlb: don't assume that io_tlb_start-io_tlb_end is coherent Stefano Stabellini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.02.1309061740050.6397@kaball.uk.xensource.com \
    --to=stefano.stabellini@eu.citrix.com \
    --cc=Ian.Campbell@citrix.com \
    --cc=catalin.marinas@arm.com \
    --cc=konrad.wilk@oracle.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=xen-devel@lists.xensource.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®