malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[lunar12 lunartwix]

A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
CMA region by user space will cause an error on our ARM 3.18 kernel
platform with a 32MB CMA.

It seems that the malloc in CMA region will be aligned to
CMA_ALIGNMENT everytime even if the requested malloc size is very
small so the CMA region is not available after the malloc operations.

Is there any configuraiton that can change this behavior??

Thanks

Cheers
Ken

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[Michal Hocko]

[CC linux-mm and some usual suspects]

On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
> CMA region by user space will cause an error on our ARM 3.18 kernel
> platform with a 32MB CMA.
> 
> It seems that the malloc in CMA region will be aligned to
> CMA_ALIGNMENT everytime even if the requested malloc size is very
> small so the CMA region is not available after the malloc operations.
> 
> Is there any configuraiton that can change this behavior??
> 
> Thanks
> 
> Cheers
> Ken

-- 
Michal Hocko
SUSE Labs

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[lunar12 lunartwix]

2016-05-18 16:48 GMT+08:00 Michal Hocko <mhocko@kernel.org>:
> [CC linux-mm and some usual suspects]
>
> On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
>> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
>> CMA region by user space will cause an error on our ARM 3.18 kernel
>> platform with a 32MB CMA.
>>
>> It seems that the malloc in CMA region will be aligned to
>> CMA_ALIGNMENT everytime even if the requested malloc size is very
>> small so the CMA region is not available after the malloc operations.
>>
>> Is there any configuraiton that can change this behavior??
>>
>> Thanks
>>
>> Cheers
>> Ken
>
> --
> Michal Hocko
> SUSE Labs

Update more information and any comment would be very appreciated

CMA region (from boot message):
Reserved memory: created CMA memory pool at 0x22e00000, size 80 MiB

User space test program:

    do
    {

        addr = malloc(2*1024);
        memset((void *)addr,2*1024,0x5A);
        vaddr=(unsigned int)addr;

        //get_user_page & page_to_phys in kernel
        ioctl(devfd, IOCTL_MSYS_USER_TO_PHYSICAL, &addr)

        count++;
        paddr=(unsigned int)addr;

        if(paddr>0x22E00000)
        {
            printf("USR:0x%08X 0x%08X %d\n",vaddr,paddr,count);
        }
    } while(addr!=NULL);


System print out:

USR:0x0164B248 0x27C00000 11337
USR:0x0164BA50 0x27C00000 11338
USR:0x0164C258 0x27800000 11339
USR:0x0164CA60 0x27800000 11340
USR:0x0164D268 0x27600000 11341
USR:0x0164DA70 0x27600000 11342
USR:0x0164E278 0x27400000 11343
USR:0x0164EA80 0x27400000 11344
USR:0x0164F288 0x27200000 11345
USR:0x0164FA90 0x27200000 11346
....
It seems that an 2MB CMA would be occpuied every 2 malloc()

Cheers
Ken

Re: malloc() size in CMA region seems to be aligned to CMA_ALIGNMENT

[Joonsoo Kim]

On Wed, May 18, 2016 at 09:15:13PM +0800, lunar12 lunartwix wrote:
> 2016-05-18 16:48 GMT+08:00 Michal Hocko <mhocko@kernel.org>:
> > [CC linux-mm and some usual suspects]

Michal, Thanks.

> >
> > On Tue 17-05-16 23:37:55, lunar12 lunartwix wrote:
> >> A 4MB dma_alloc_coherent  in kernel after malloc(2*1024) 40 times in
> >> CMA region by user space will cause an error on our ARM 3.18 kernel
> >> platform with a 32MB CMA.
> >>
> >> It seems that the malloc in CMA region will be aligned to
> >> CMA_ALIGNMENT everytime even if the requested malloc size is very
> >> small so the CMA region is not available after the malloc operations.
> >>
> >> Is there any configuraiton that can change this behavior??
> >>
> >> Thanks
> >>
> >> Cheers
> >> Ken
> >
> > --
> > Michal Hocko
> > SUSE Labs
> 
> Update more information and any comment would be very appreciated
> 
> CMA region (from boot message):
> Reserved memory: created CMA memory pool at 0x22e00000, size 80 MiB
> 
> User space test program:
> 
>     do
>     {
> 
>         addr = malloc(2*1024);
>         memset((void *)addr,2*1024,0x5A);
>         vaddr=(unsigned int)addr;
> 
>         //get_user_page & page_to_phys in kernel
>         ioctl(devfd, IOCTL_MSYS_USER_TO_PHYSICAL, &addr)
> 
>         count++;
>         paddr=(unsigned int)addr;
> 
>         if(paddr>0x22E00000)
>         {
>             printf("USR:0x%08X 0x%08X %d\n",vaddr,paddr,count);
>         }
>     } while(addr!=NULL);
> 
> 
> System print out:
> 
> USR:0x0164B248 0x27C00000 11337
> USR:0x0164BA50 0x27C00000 11338
> USR:0x0164C258 0x27800000 11339
> USR:0x0164CA60 0x27800000 11340
> USR:0x0164D268 0x27600000 11341
> USR:0x0164DA70 0x27600000 11342
> USR:0x0164E278 0x27400000 11343
> USR:0x0164EA80 0x27400000 11344
> USR:0x0164F288 0x27200000 11345
> USR:0x0164FA90 0x27200000 11346
> ....
> It seems that an 2MB CMA would be occpuied every 2 malloc()

I'm not familiar with device part of CMA but try to analyze.

Above output means that your device maps 2 MB CMA mem to 1 page. I guess
that your device requires such alignment. Could you check
CONFIG_CMA_ALIGNMENT? And, insert to log to below snippet
in drivers/base/dma-contiguous.c to check your device align requirement?

struct page *dma_alloc_from_contiguous(struct device *dev, size_t count,
                                       unsigned int align)
{                                                                                          
        if (align > CONFIG_CMA_ALIGNMENT)
                align = CONFIG_CMA_ALIGNMENT;
        return cma_alloc(dev_get_cma_area(dev), count, align);
}

I guess changing CONFIG_CMA_ALIGNMENT works for you, but, since it
ignore your device align requirement, I'm not sure that it is right solution.

Thanks.