Re: [PATCH v2] usb: dwc3: gadget: Avoid canceling current request for queuing error

[Wesley Cheng <wcheng@codeaurora.org>]

On 5/3/2021 10:22 PM, Thinh Nguyen wrote:
> Wesley Cheng wrote:
>>
>>
>> On 5/3/2021 8:12 PM, Thinh Nguyen wrote:
>>> Hi Wesley,
>>>
>>> Wesley Cheng wrote:
>>>>
>>>>
>>>> On 5/3/2021 7:20 PM, Thinh Nguyen wrote:
>>>>> Hi,
>>>>>
>>>>> Wesley Cheng wrote:
>>>>>> If an error is received when issuing a start or update transfer
>>>>>> command, the error handler will stop all active requests (including
>>>>>> the current USB request), and call dwc3_gadget_giveback() to notify
>>>>>> function drivers of the requests which have been stopped.  Avoid
>>>>>> having to cancel the current request which is trying to be queued, as
>>>>>> the function driver will handle the EP queue error accordingly.
>>>>>> Simply unmap the request as it was done before, and allow previously
>>>>>> started transfers to be cleaned up.
>>>>>>
>>>>
>>>> Hi Thinh,
>>>>
>>>>>
>>>>> It looks like you're still letting dwc3 stopping and cancelling all the
>>>>> active requests instead letting the function driver doing the dequeue.
>>>>>
>>>>
>>>> Yeah, main issue isn't due to the function driver doing dequeue, but
>>>> having cleanup (ie USB request free) if there is an error during
>>>> usb_ep_queue().
>>>>
>>>> The function driver in question at the moment is the f_fs driver in AIO
>>>> mode.  When async IO is enabled in the FFS driver, every time it queues
>>>> a packet, it will allocate a io_data struct beforehand.  If the
>>>> usb_ep_queue() fails it will free this io_data memory.  Problem is that,
>>>> since the DWC3 gadget calls the completion with -ECONNRESET, the FFS
>>>> driver will also schedule a work item (within io_data struct) to handle
>>>> the completion.  So you end up with a flow like below
>>>>
>>>> allocate io_data (ffs)
>>>>  --> usb_ep_queue()
>>>>    --> __dwc3_gadget_kick_transfer()
>>>>    --> dwc3_send_gadget_ep_cmd(EINVAL)
>>>>    --> dwc3_gadget_ep_cleanup_cancelled_requests()
>>>>    --> dwc3_gadget_giveback(ECONNRESET)
>>>> ffs completion callback
>>>> queue work item within io_data
>>>>  --> usb_ep_queue returns EINVAL
>>>> ffs frees io_data
>>>> ...
>>>>
>>>> work scheduled
>>>>  --> NULL pointer/memory fault as io_data is freed
>>
>> Hi Thinh,
>>
>>>
>>> sounds like a race issue.
>>>
>>
>> It'll always happen if usb_ep_queue() fails with an error. Sorry for not
>> clarifying, but the "..." represents executing in a different context
>> :). Anything above the "..." is in the same context.
>>>>
>>>>> BTW, what kinds of command and error do you see in your setup and for
>>>>> what type endpoint? I'm thinking of letting the function driver to
>>>>> dequeue the requests instead of letting dwc3 automatically
>>>>> ending/cancelling the queued requests. However, it's a bit tricky to do
>>>>> that if the error is -ETIMEDOUT since we're not sure if the controller
>>>>> had already cached the TRBs.
>>>>>
>>>>
>>>> Happens on bulk EPs so far, but I think it wouldn't matter as long as
>>>> its over the FFS interface. (and using async IO transfers)
>>>
>>> Do you know which command and error code? It's strange if
>>> UPDATE_TRANSFER command failed.
>>>
>>
>> Sorry for missing that part of the question.  It is a no xfer resource
>> error on a start transfer command.  So far this happens on low system
>> memory test cases, so there may be some sequences that were missed,
>> which led to this particular command error.
>>
>> Thanks
>> Wesley Cheng

Hi Thinh,

> 
> No xfer resource usually means that the driver attempted to send
> START_TRANSFER without waiting for END_TRANSFER command to complete.
> This may be a dwc3 driver issue. Did you check this?
> 
> Thanks,
> Thinh
> 
> 

Yes, we know the reason why this happens, and its due to one of the
downstream changes we had that led to the scenario above.  Although,
that has been fixed, I still believe the error path is a potential
scenario we'd still want to address.

I think the returning success always on dwc3_gadget_ep_queue(), and
allowing the error in the completion handler/giveback at the function
driver level to do the cleanup is a feasible solution.  Doesn't change
the flow of the DWC3 gadget, and so far all function drivers we've used
handle this in the correct manner.

Thanks
Wesley Cheng

>>
>>>>
>>>>> This seems to add more complexity and I don't have a good solution to
>>>>> it. Since you're already cancelling all the active request anyway, what
>>>>> do you think of always letting dwc3_gadget_ep_queue() to go through with
>>>>> success, but report failure through request completion?
>>>>>
>>>>
>>>> We do have something similar as well downstream (returning success
>>>> always on dwc3_gadget_ep_queue()) and its been working for us also.
>>>> Problem is we don't test the ISOC path much, so this is the only type of
>>>> EP that might come into question...
>>>>
>>>
>>> It should be similiar with isoc. I can't think of a potential issue yet.
>>>
>>>> Coming up with a way to address the concerns you brought up was a bit
>>>> difficult as there were scenarios we needed to consider.  next_request()
>>>> doesn't always have to be the request being queued (even if ep queue
>>>> triggered it).  There was no easy way to determine if kick transfer was
>>>> due to ep queue, but even if there was, we'd need to remember the
>>>> previous point as well.
>>>>
>>>
>>> Yeah, there are a few pitfalls. I don't have a good solution to it if we
>>> want to return failure immediately and let the function driver handle
>>> the dequeue (if it wants to).
>>>
>>> Thanks,
>>> Thinh
>>>
>>
> 

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project