Re: [PATCH v1] dma-contiguous: try local node first for dma_alloc_contiguous()

[Feng Tang <feng.tang@linux.alibaba.com>]

Hi Robin,

On Tue, Apr 14, 2026 at 01:32:11PM +0100, Robin Murphy wrote:
> On 14/04/2026 10:03 am, Feng Tang wrote:
> > There was a bug report on a multi-numa-nodes ARM server that when
> > IOMMU is disabled, the dma_alloc_coherent() function always returns
> > memory from node 0 even for devices attaching to other nodes, while
> > they can get local dma memory when IOMMU is on with the same API.
> > 
> > The reason is, when IOMMU is disabled, the dma_alloc_coherent() will
> > go the direct way and call dma_alloc_contiguous(). The system doesn't
> > have any explicit cma setting (like per-numa cma), and only has a
> > default 64MB cma reserved area (on node 0), where kernel will try
> > first to allocate memory from.
> > 
> > Make the dma allocation more locality friendly by trying first the
> > numa aware allocation alloc_pages_node() before falling back to the
> > reserved cma area.
> > 
> > One more thought is to check the node of the reserved cma area and
> > only call alloc_pages_nodes() when it isn't the same node that the
> > device attaches to.

Thanks for the prompt review!

> Frankly, no. We literally have a feature specifically created for this
> use-case already - if you care about NUMA locality of DMA allocations then
> enable DMA_NUMA_CMA. Or if you don't need CMA at all, then disabling CMA
> would be even better for the overall performance of your system.

That makes sense. I also recommended using the per-numa cma to the reporter.

One thing is this numa unfriendly behavior happens silently, and I haven't
noticed it for long time till I got the bug report, as the normal
distribution enable CONFIG_CMA which provides the 64M cma area on node
0 (mostly) by default. I'm afraid many normal users are experiencing the
same thing without doing any extra explicit cma setting.

Thanks,
Feng

> Thanks,
> Robin.
> 
> > Signed-off-by: Feng Tang <feng.tang@linux.alibaba.com>
> > ---
> >   kernel/dma/contiguous.c | 12 ++++++++++--
> >   1 file changed, 10 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> > index c56004d314dc..0180f40f094e 100644
> > --- a/kernel/dma/contiguous.c
> > +++ b/kernel/dma/contiguous.c
> > @@ -371,8 +371,9 @@ static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t gfp)
> >    */
> >   struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
> >   {
> > -#ifdef CONFIG_DMA_NUMA_CMA
> > +#ifdef CONFIG_NUMA
> >   	int nid = dev_to_node(dev);
> > +	struct page *page;
> >   #endif
> >   	/* CMA can be used only in the context which permits sleeping */
> > @@ -386,7 +387,6 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
> >   #ifdef CONFIG_DMA_NUMA_CMA
> >   	if (nid != NUMA_NO_NODE && !(gfp & (GFP_DMA | GFP_DMA32))) {
> >   		struct cma *cma = dma_contiguous_pernuma_area[nid];
> > -		struct page *page;
> >   		if (cma) {
> >   			page = cma_alloc_aligned(cma, size, gfp);
> > @@ -402,6 +402,14 @@ struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t gfp)
> >   		}
> >   	}
> >   #endif
> > +
> > +#ifdef CONFIG_NUMA
> > +	/* Try first to allocate memory on the same node as the device */
> > +	page = alloc_pages_node(nid, gfp, get_order(size));
> > +	if (page)
> > +		return page;
> > +#endif
> > +
> >   	if (!dma_contiguous_default_area)
> >   		return NULL;