[PATCH] media: amphion: check and cancel vpu before release

[PATCH] media: amphion: check and cancel vpu before release

[Ming Qian]

driver need to cancel vpu before releasing the vpu instance,
but it doesn't want to release the m2m_ctx before
the release callback of instance is called,
as driver may access the m2m_ctx in handling some event
which may be received in releasing instance.

check and cancel the unstopped instance before release.

Fixes: d91d7bc85062 ("media: amphion: release m2m ctx when releasing vpu instance")
Signed-off-by: Ming Qian <ming.qian@nxp.com>
---
 drivers/media/platform/amphion/vpu_v4l2.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/platform/amphion/vpu_v4l2.c
index 99ad2f1c5a53..845fc53d8937 100644
--- a/drivers/media/platform/amphion/vpu_v4l2.c
+++ b/drivers/media/platform/amphion/vpu_v4l2.c
@@ -767,6 +767,23 @@ int vpu_v4l2_open(struct file *file, struct vpu_inst *inst)
 	return ret;
 }
 
+static void vpu_v4l2_check_and_cancel(struct file *file, struct vpu_inst *inst)
+{
+	struct vb2_queue *vq;
+
+	vpu_inst_lock(inst);
+
+	vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
+	if (vb2_is_streaming(vq))
+		v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
+
+	vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
+	if (vb2_is_streaming(vq))
+		v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
+
+	vpu_inst_unlock(inst);
+}
+
 int vpu_v4l2_close(struct file *file)
 {
 	struct vpu_dev *vpu = video_drvdata(file);
@@ -774,6 +791,7 @@ int vpu_v4l2_close(struct file *file)
 
 	vpu_trace(vpu->dev, "tgid = %d, pid = %d, inst = %p\n", inst->tgid, inst->pid, inst);
 
+	vpu_v4l2_check_and_cancel(file, inst);
 	call_void_vop(inst, release);
 	vpu_inst_unregister(inst);
 	vpu_inst_put(inst);
-- 
2.38.1

Re: [PATCH] media: amphion: check and cancel vpu before release

[Hans Verkuil]

Hi Ming,

On 11/1/22 09:05, Ming Qian wrote:
> driver need to cancel vpu before releasing the vpu instance,
> but it doesn't want to release the m2m_ctx before
> the release callback of instance is called,
> as driver may access the m2m_ctx in handling some event
> which may be received in releasing instance.
> 
> check and cancel the unstopped instance before release.
> 
> Fixes: d91d7bc85062 ("media: amphion: release m2m ctx when releasing vpu instance")
> Signed-off-by: Ming Qian <ming.qian@nxp.com>
> ---
>  drivers/media/platform/amphion/vpu_v4l2.c | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/media/platform/amphion/vpu_v4l2.c b/drivers/media/platform/amphion/vpu_v4l2.c
> index 99ad2f1c5a53..845fc53d8937 100644
> --- a/drivers/media/platform/amphion/vpu_v4l2.c
> +++ b/drivers/media/platform/amphion/vpu_v4l2.c
> @@ -767,6 +767,23 @@ int vpu_v4l2_open(struct file *file, struct vpu_inst *inst)
>  	return ret;
>  }
>  
> +static void vpu_v4l2_check_and_cancel(struct file *file, struct vpu_inst *inst)
> +{
> +	struct vb2_queue *vq;
> +
> +	vpu_inst_lock(inst);
> +
> +	vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
> +	if (vb2_is_streaming(vq))
> +		v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
> +
> +	vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
> +	if (vb2_is_streaming(vq))
> +		v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
> +
> +	vpu_inst_unlock(inst);

This feels weird. This is normally done in v4l2_m2m_ctx_release(), so
if you need to do this here, doesn't that mean perhaps that the order
of releasing things should be changed? I.e., first call v4l2_m2m_ctx_release()
and only afterwards release the vpu instance.

In other words, are you just fixing the symptom rather than the actual
cause?

Regards,

	Hans

> +}
> +
>  int vpu_v4l2_close(struct file *file)
>  {
>  	struct vpu_dev *vpu = video_drvdata(file);
> @@ -774,6 +791,7 @@ int vpu_v4l2_close(struct file *file)
>  
>  	vpu_trace(vpu->dev, "tgid = %d, pid = %d, inst = %p\n", inst->tgid, inst->pid, inst);
>  
> +	vpu_v4l2_check_and_cancel(file, inst);
>  	call_void_vop(inst, release);
>  	vpu_inst_unregister(inst);
>  	vpu_inst_put(inst);

RE: [EXT] Re: [PATCH] media: amphion: check and cancel vpu before release

[Ming Qian]

>Hi Ming,
>
>On 11/1/22 09:05, Ming Qian wrote:
>> driver need to cancel vpu before releasing the vpu instance, but it
>> doesn't want to release the m2m_ctx before the release callback of
>> instance is called, as driver may access the m2m_ctx in handling some
>> event which may be received in releasing instance.
>>
>> check and cancel the unstopped instance before release.
>>
>> Fixes: d91d7bc85062 ("media: amphion: release m2m ctx when releasing
>> vpu instance")
>> Signed-off-by: Ming Qian <ming.qian@nxp.com>
>> ---
>>  drivers/media/platform/amphion/vpu_v4l2.c | 18 ++++++++++++++++++
>>  1 file changed, 18 insertions(+)
>>
>> diff --git a/drivers/media/platform/amphion/vpu_v4l2.c
>> b/drivers/media/platform/amphion/vpu_v4l2.c
>> index 99ad2f1c5a53..845fc53d8937 100644
>> --- a/drivers/media/platform/amphion/vpu_v4l2.c
>> +++ b/drivers/media/platform/amphion/vpu_v4l2.c
>> @@ -767,6 +767,23 @@ int vpu_v4l2_open(struct file *file, struct vpu_inst
>*inst)
>>       return ret;
>>  }
>>
>> +static void vpu_v4l2_check_and_cancel(struct file *file, struct
>> +vpu_inst *inst) {
>> +     struct vb2_queue *vq;
>> +
>> +     vpu_inst_lock(inst);
>> +
>> +     vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
>> +     if (vb2_is_streaming(vq))
>> +             v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
>> +
>> +     vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
>> +     if (vb2_is_streaming(vq))
>> +             v4l2_m2m_streamoff(file, inst->fh.m2m_ctx, vq->type);
>> +
>> +     vpu_inst_unlock(inst);
>
>This feels weird. This is normally done in v4l2_m2m_ctx_release(), so if you
>need to do this here, doesn't that mean perhaps that the order of releasing
>things should be changed? I.e., first call v4l2_m2m_ctx_release() and only
>afterwards release the vpu instance.
>
>In other words, are you just fixing the symptom rather than the actual cause?
>
>Regards,
>
>        Hans
>

Hi Hans,
    I agree,
    In the first  I indeed call v4l2_m2m_ctx_release() and afterwards release the vpu instance,
But there may be some redundant event triggered by firmware. And they may led to kernel panic as I didn't put the whole event handler into the instance lock.

I just thought if I release the m2m ctx in releasing vpu instance, then thus case can be avoided. But it's really not well thought out.

I'll revert the patch " media: amphion: release m2m ctx when releasing vpu instance", then check the event handler carefully to make sure there is no issue if the m2m ctx has been fixed.

Ming


>> +}
>> +
>>  int vpu_v4l2_close(struct file *file)  {
>>       struct vpu_dev *vpu = video_drvdata(file); @@ -774,6 +791,7 @@
>> int vpu_v4l2_close(struct file *file)
>>
>>       vpu_trace(vpu->dev, "tgid = %d, pid = %d, inst = %p\n",
>> inst->tgid, inst->pid, inst);
>>
>> +     vpu_v4l2_check_and_cancel(file, inst);
>>       call_void_vop(inst, release);
>>       vpu_inst_unregister(inst);
>>       vpu_inst_put(inst);