Re: [PATCH] ntb_perf: pass correct struct device to dma_alloc_coherent

[Sanjay R Mehta <sanmehta@amd.com>]

On 12/10/2019 11:01 PM, Logan Gunthorpe wrote:
> [CAUTION: External Email]
>
> On 2019-12-10 6:07 a.m., Sanjay R Mehta wrote:
>> From: Sanjay R Mehta <sanju.mehta@amd.com>
>>
>> Currently, ntb->dev is passed to dma_alloc_coherent
>> and dma_free_coherent calls. The returned dma_addr_t
>> is the CPU physical address. This works fine as long
>> as IOMMU is disabled. But when IOMMU is enabled, we
>> need to make sure that IOVA is returned for dma_addr_t.
>> So the correct way to achieve this is by changing the
>> first parameter of dma_alloc_coherent() as ntb->pdev->dev
>> instead.
>>
>> Fixes: 5648e56 ("NTB: ntb_perf: Add full multi-port NTB API support")
>> Signed-off-by: Sanjay R Mehta <sanju.mehta@amd.com>
> Yes, I did the same thing as one of the patches in my fix-up series that
> never got merged. See [1].
>
> Hopefully you can make better progress than I did.
>
> While you're at it I think it's worth doing the same thing in ntb_tool
> as well as removing the dma_coerce_mask_and_coherent() calls that are in
> the NTB drivers which are meaningless once we get back to using the
> correct DMA device.
Yes, What you said make sense. I will address your comment in the next version of patch, but I will break them down into two separate patches.
>
> Thanks,
>
> Logan
>
> [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flkml%2F20190109192233.5752-3-logang%40deltatee.com%2F&amp;data=02%7C01%7CSanju.Mehta%40amd.com%7C1f043932c1c1425a836908d77d96e265%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637115959326469198&amp;sdata=BoDyfqTML39JhIvO7%2FvYGnKNRKe2rSKgZxJ0uT6dtsQ%3D&amp;reserved=0
>
>> ---
>>  drivers/ntb/test/ntb_perf.c | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
>> index f5df33a..8729838 100644
>> --- a/drivers/ntb/test/ntb_perf.c
>> +++ b/drivers/ntb/test/ntb_perf.c
>> @@ -559,7 +559,7 @@ static void perf_free_inbuf(struct perf_peer *peer)
>>               return;
>>
>>       (void)ntb_mw_clear_trans(peer->perf->ntb, peer->pidx, peer->gidx);
>> -     dma_free_coherent(&peer->perf->ntb->dev, peer->inbuf_size,
>> +     dma_free_coherent(&peer->perf->ntb->pdev->dev, peer->inbuf_size,
>>                         peer->inbuf, peer->inbuf_xlat);
>>       peer->inbuf = NULL;
>>  }
>> @@ -588,8 +588,9 @@ static int perf_setup_inbuf(struct perf_peer *peer)
>>
>>       perf_free_inbuf(peer);
>>
>> -     peer->inbuf = dma_alloc_coherent(&perf->ntb->dev, peer->inbuf_size,
>> -                                      &peer->inbuf_xlat, GFP_KERNEL);
>> +     peer->inbuf = dma_alloc_coherent(&perf->ntb->pdev->dev,
>> +                                      peer->inbuf_size, &peer->inbuf_xlat,
>> +                                      GFP_KERNEL);
>>       if (!peer->inbuf) {
>>               dev_err(&perf->ntb->dev, "Failed to alloc inbuf of %pa\n",
>>                       &peer->inbuf_size);
>>