Re: [RFC PATCH v11 6/9] media: tegra: Add Tegra210 Video input driver

[Sowjanya Komatineni <skomatineni@nvidia.com>]

On 5/4/20 5:18 AM, Hans Verkuil wrote:
> On 03/05/2020 00:46, Sowjanya Komatineni wrote:
>> On 5/2/20 1:48 PM, Dmitry Osipenko wrote:
>>> 02.05.2020 19:55, Sowjanya Komatineni пишет:
>>>> Also stop stream ioctl request happens during suspend where both threads
>>>> will be stopped properly. done thread stop happens only after finishing
>>>> all outstanding buffers.
>>> Do you mean that V4L core takes care of stopping the streami on suspend
>>> and re-starting it on resume from suspend?
>>>
>>>> Stop stream request happens from streaming applications so even without
>>>> driver suspend/resume implementation currently, streaming will be
>>>> stopped prior to system  suspend where both threads will be stopped
>>>> properly (after finishing out standing buffers) and will be resumed by
>>>> application on system resume
>>> All userspace is frozen on suspend. System suspension is transparent for
>>> userspace applications. I'm not sure what you're meaning here.
>>>
>>>> Also tested suspending while streaming with this unconditional freeze, I
>>>> don't see any issue as application stops stream where v4l_streamoff gets
>>>> executed during suspend and on resume streaming starts where
>>>> v4l_streamon happens.
>>>>
>>>> So, I don't see any issue with existing implementation of unconditional
>>>> freeze.
>>> I don't understand why freezing is needed at all if V4L core takes care
>>> of stopping the stream on suspend, what is the point? If there is no
>>> real point, then let's make threads non-freezable and done with that.
>> video device fops unlocked_ioctl is set to video_ioctl2() in vi driver.
>>
>> video device fops unlocked_ioctl gets executed with stream off cmd
>> during suspend and stream on cmd during resume which eventually calls
>> v4l_streamoff and v4l_streamon during system suspend/resume.
> That's news to me. The 'only' thing that suspend/resume needs to do is to
> stop the HW DMA on suspend and to restart the HW DMA (and typically reconfigure
> the whole HW video pipeline) on resume. Userspace doesn't do anything special.
>
> That's how e.g. a UVC webcam behaves when you close the lid of a laptop while
> it is streaming and open it again later.
>
> It can be hard to get this right, and I suspect many media drivers will fail
> this test.

when video device node path is kept opened during suspend, on suspend 
entry looks like inode path is tried to closes and on resume opened 
again causing v4l2 stream off/on thru v4l2 ioctl.

Based on our testing even with downstream and user applications, we 
always see video device node path close/open during suspend/resume which 
does v4l2 stream off/on.


>> My understanding to have freezable threads is during system suspend user
>> space applications are frozen prior to kernel freeze and during suspend
>> when opened video character device node gets closed these ioctl gets
>> invoked and stream off during suspend and stream on during resume
>> happens. So probably we still need to use freezable threads to sync with
>> user space application when frozen before really entering suspend.
>>
>> Will wait for Thierry/Hans comment to correct if my above understanding
>> is wrong and help clarify if we need freezable threads at all in this
>> case...
>>
>> Note: I see other drivers using freezable threads for capture drivers.
> Well, it's often a copy-and-paste without truly understanding what is
> going on. You should not assume that the author knew what was happening.
>
> To be honest, I'm not an expert on this either.
>
> Looking at the tegra start/finish threads: they basically look at the
> chan->capture and chan->done lists. Freezing the threads should not be
> a problem as long as the actual suspend/resume doesn't mess with those
> lists. If it does, then it may get tricky to prove that it is safe to
> do suspend/resume (I think).
>
> An alternative is to stop and restart those threads when suspending or
> resuming. Then those threads do not need to be 'freezable' and it might
> be easier to validate the code.
>
> In any case, I do not want to postpone the merger of the upcoming v12 for
> this. Changes can be done in later patches, if needed.
>
> Regards,
>
> 	Hans

Thanks Hans.

Buffers list don't get altered after frozen and during suspend/resume 
till they are out of freeze.

Will remove freezable threads and move on for v12.

Will validate suspend/resume later after all sensor support.

>>
>> Assuming we use freezable threads, I was saying we don't need
>> conditional try_to_freeze() like you pointed because even if finish
>> thread freeze happens prior to frame capture initiated by start thread,
>> vi hardware will still continue to update this single ongoing buffer and
>> will finish max within 200ms and actually there is no direct processing
>> of this done by finish thread itself except that it returns buffers back
>> when done and in this case it returns back when unfreeze/wake up happens.
>>
>> So, I don't see any issue of unconditional try_to_freeze() even with
>> freezable threads.
>>
>> Thanks
>>
>> Sowjanya
>>
>>