[PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

Only return the value of the ready field if the VQ is initialized in
which case the value of the field is valid.

Failing to do so can result in virtio_vdpa failing to load if the device
was previously used by vhost_vdpa and the old values are ready.
virtio_vdpa expects to find VQs in "not ready" state.

Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
Signed-off-by: Eli Cohen <elic@nvidia.com>
---
 drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 02a05492204c..f6b680d2ab1c 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
 	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
 	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
 
-	return mvq->ready;
+	return mvq->initialized && mvq->ready;
 }
 
 static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
-- 
2.31.1

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Jason Wang]

在 2021/6/1 上午12:04, Eli Cohen 写道:
> Only return the value of the ready field if the VQ is initialized in
> which case the value of the field is valid.
>
> Failing to do so can result in virtio_vdpa failing to load if the device
> was previously used by vhost_vdpa and the old values are ready.
> virtio_vdpa expects to find VQs in "not ready" state.
>
> Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> Signed-off-by: Eli Cohen <elic@nvidia.com>
> ---
>   drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> index 02a05492204c..f6b680d2ab1c 100644
> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
>   	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
>   	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
>   
> -	return mvq->ready;
> +	return mvq->initialized && mvq->ready;


I think the more suitable fix is to reset mvq->ready during reset. The 
vq_ready should follow the queue_enable semantic in virtio-pci:

"
The device MUST present a 0 in queue_enable on reset.
"

Thanks


>   }
>   
>   static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
> 
> 在 2021/6/1 上午12:04, Eli Cohen 写道:
> > Only return the value of the ready field if the VQ is initialized in
> > which case the value of the field is valid.
> > 
> > Failing to do so can result in virtio_vdpa failing to load if the device
> > was previously used by vhost_vdpa and the old values are ready.
> > virtio_vdpa expects to find VQs in "not ready" state.
> > 
> > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > ---
> >   drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > index 02a05492204c..f6b680d2ab1c 100644
> > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
> >   	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
> >   	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
> > -	return mvq->ready;
> > +	return mvq->initialized && mvq->ready;
> 
> 
> I think the more suitable fix is to reset mvq->ready during reset. The
> vq_ready should follow the queue_enable semantic in virtio-pci:
> 
> "
> The device MUST present a 0 in queue_enable on reset.
> "

ok

> 
> Thanks
> 
> 
> >   }
> >   static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
> 

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
> 
> 在 2021/6/1 上午12:04, Eli Cohen 写道:
> > Only return the value of the ready field if the VQ is initialized in
> > which case the value of the field is valid.
> > 
> > Failing to do so can result in virtio_vdpa failing to load if the device
> > was previously used by vhost_vdpa and the old values are ready.
> > virtio_vdpa expects to find VQs in "not ready" state.
> > 
> > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > ---
> >   drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > index 02a05492204c..f6b680d2ab1c 100644
> > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
> >   	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
> >   	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
> > -	return mvq->ready;
> > +	return mvq->initialized && mvq->ready;
> 
> 
> I think the more suitable fix is to reset mvq->ready during reset. The
> vq_ready should follow the queue_enable semantic in virtio-pci:
> 
> "
> The device MUST present a 0 in queue_enable on reset.
> "

Thinking again, I think we should call set_vq_ready() from
qemu/virtio_vdpa etc. after reset to explicitly set ready to false.

The ready indication is not necessairily a reflection of the hardware
queue:

"Virtual queue ready bit
Writing one (0x1) to this register notifies the device that it can
execute requests from this virtual queue. Reading from this register
returns the last value written to it. Both read and write accesses apply
to the queue selected by writing to QueueSel."


> 
> Thanks
> 
> 
> >   }
> >   static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
> 

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Jason Wang]

在 2021/6/1 下午12:13, Eli Cohen 写道:
> On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
>> 在 2021/6/1 上午12:04, Eli Cohen 写道:
>>> Only return the value of the ready field if the VQ is initialized in
>>> which case the value of the field is valid.
>>>
>>> Failing to do so can result in virtio_vdpa failing to load if the device
>>> was previously used by vhost_vdpa and the old values are ready.
>>> virtio_vdpa expects to find VQs in "not ready" state.
>>>
>>> Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
>>> Signed-off-by: Eli Cohen <elic@nvidia.com>
>>> ---
>>>    drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> index 02a05492204c..f6b680d2ab1c 100644
>>> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>> @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
>>>    	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
>>>    	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
>>> -	return mvq->ready;
>>> +	return mvq->initialized && mvq->ready;
>>
>> I think the more suitable fix is to reset mvq->ready during reset. The
>> vq_ready should follow the queue_enable semantic in virtio-pci:
>>
>> "
>> The device MUST present a 0 in queue_enable on reset.
>> "
> Thinking again, I think we should call set_vq_ready() from
> qemu/virtio_vdpa etc. after reset to explicitly set ready to false.


This is not what I read from the spec and how the current driver behave.

And I don't see why we need to stick to 1 after the reset.


>
> The ready indication is not necessairily a reflection of the hardware
> queue:
>
> "Virtual queue ready bit
> Writing one (0x1) to this register notifies the device that it can
> execute requests from this virtual queue. Reading from this register
> returns the last value written to it. Both read and write accesses apply
> to the queue selected by writing to QueueSel."


My understanding that this applies if not reset in the middle. We can 
clarify this in the spec if needed.

Thanks


>
>
>> Thanks
>>
>>
>>>    }
>>>    static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

On Tue, Jun 01, 2021 at 01:03:36PM +0800, Jason Wang wrote:
> 
> 在 2021/6/1 下午12:13, Eli Cohen 写道:
> > On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
> > > 在 2021/6/1 上午12:04, Eli Cohen 写道:
> > > > Only return the value of the ready field if the VQ is initialized in
> > > > which case the value of the field is valid.
> > > > 
> > > > Failing to do so can result in virtio_vdpa failing to load if the device
> > > > was previously used by vhost_vdpa and the old values are ready.
> > > > virtio_vdpa expects to find VQs in "not ready" state.
> > > > 
> > > > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > > > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > > > ---
> > > >    drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
> > > >    1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > index 02a05492204c..f6b680d2ab1c 100644
> > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
> > > >    	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
> > > >    	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
> > > > -	return mvq->ready;
> > > > +	return mvq->initialized && mvq->ready;
> > > 
> > > I think the more suitable fix is to reset mvq->ready during reset. The
> > > vq_ready should follow the queue_enable semantic in virtio-pci:
> > > 
> > > "
> > > The device MUST present a 0 in queue_enable on reset.
> > > "
> > Thinking again, I think we should call set_vq_ready() from
> > qemu/virtio_vdpa etc. after reset to explicitly set ready to false.
> 
> 
> This is not what I read from the spec and how the current driver behave.
> 
> And I don't see why we need to stick to 1 after the reset.

Like you said below, this must be clarified by the spec. Because my
understanding of the text is that the ready flag is a means for the
driver to tell the device whether it may or may not execute requests. So
following this reasoning, the driver should tell the device when ready
should be zero. As you suggest, after reset.

Meanwhile I am going to change the code to reset ready after device
reset.

> 
> 
> > 
> > The ready indication is not necessairily a reflection of the hardware
> > queue:
> > 
> > "Virtual queue ready bit
> > Writing one (0x1) to this register notifies the device that it can
> > execute requests from this virtual queue. Reading from this register
> > returns the last value written to it. Both read and write accesses apply
> > to the queue selected by writing to QueueSel."
> 
> 
> My understanding that this applies if not reset in the middle. We can
> clarify this in the spec if needed.
> 
> Thanks
> 
> 
> > 
> > 
> > > Thanks
> > > 
> > > 
> > > >    }
> > > >    static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
> 

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Jason Wang]

在 2021/6/1 下午6:21, Eli Cohen 写道:
> On Tue, Jun 01, 2021 at 01:03:36PM +0800, Jason Wang wrote:
>> 在 2021/6/1 下午12:13, Eli Cohen 写道:
>>> On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
>>>> 在 2021/6/1 上午12:04, Eli Cohen 写道:
>>>>> Only return the value of the ready field if the VQ is initialized in
>>>>> which case the value of the field is valid.
>>>>>
>>>>> Failing to do so can result in virtio_vdpa failing to load if the device
>>>>> was previously used by vhost_vdpa and the old values are ready.
>>>>> virtio_vdpa expects to find VQs in "not ready" state.
>>>>>
>>>>> Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
>>>>> Signed-off-by: Eli Cohen <elic@nvidia.com>
>>>>> ---
>>>>>     drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
>>>>>     1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>>> index 02a05492204c..f6b680d2ab1c 100644
>>>>> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>>> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>>> @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
>>>>>     	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
>>>>>     	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
>>>>> -	return mvq->ready;
>>>>> +	return mvq->initialized && mvq->ready;
>>>> I think the more suitable fix is to reset mvq->ready during reset. The
>>>> vq_ready should follow the queue_enable semantic in virtio-pci:
>>>>
>>>> "
>>>> The device MUST present a 0 in queue_enable on reset.
>>>> "
>>> Thinking again, I think we should call set_vq_ready() from
>>> qemu/virtio_vdpa etc. after reset to explicitly set ready to false.
>>
>> This is not what I read from the spec and how the current driver behave.
>>
>> And I don't see why we need to stick to 1 after the reset.
> Like you said below, this must be clarified by the spec.


I will add this to my todo list.


> Because my
> understanding of the text is that the ready flag is a means for the
> driver to tell the device whether it may or may not execute requests. So
> following this reasoning, the driver should tell the device when ready
> should be zero. As you suggest, after reset.


Another thought, reset should prevent the device from executing requests 
on each queue. In this case, presenting vq_ready to zero make sense. 
Otherwise there would be a conflict.

Thanks


>
> Meanwhile I am going to change the code to reset ready after device
> reset.
>
>>
>>> The ready indication is not necessairily a reflection of the hardware
>>> queue:
>>>
>>> "Virtual queue ready bit
>>> Writing one (0x1) to this register notifies the device that it can
>>> execute requests from this virtual queue. Reading from this register
>>> returns the last value written to it. Both read and write accesses apply
>>> to the queue selected by writing to QueueSel."
>>
>> My understanding that this applies if not reset in the middle. We can
>> clarify this in the spec if needed.
>>
>> Thanks
>>
>>
>>>
>>>> Thanks
>>>>
>>>>
>>>>>     }
>>>>>     static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

On Wed, Jun 02, 2021 at 03:29:52PM +0800, Jason Wang wrote:
> 
> 在 2021/6/1 下午6:21, Eli Cohen 写道:
> > On Tue, Jun 01, 2021 at 01:03:36PM +0800, Jason Wang wrote:
> > > 在 2021/6/1 下午12:13, Eli Cohen 写道:
> > > > On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
> > > > > 在 2021/6/1 上午12:04, Eli Cohen 写道:
> > > > > > Only return the value of the ready field if the VQ is initialized in
> > > > > > which case the value of the field is valid.
> > > > > > 
> > > > > > Failing to do so can result in virtio_vdpa failing to load if the device
> > > > > > was previously used by vhost_vdpa and the old values are ready.
> > > > > > virtio_vdpa expects to find VQs in "not ready" state.
> > > > > > 
> > > > > > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > > > > > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > > > > > ---
> > > > > >     drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
> > > > > >     1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > 
> > > > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > > > index 02a05492204c..f6b680d2ab1c 100644
> > > > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > > > @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
> > > > > >     	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
> > > > > >     	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
> > > > > > -	return mvq->ready;
> > > > > > +	return mvq->initialized && mvq->ready;
> > > > > I think the more suitable fix is to reset mvq->ready during reset. The
> > > > > vq_ready should follow the queue_enable semantic in virtio-pci:
> > > > > 
> > > > > "
> > > > > The device MUST present a 0 in queue_enable on reset.
> > > > > "
> > > > Thinking again, I think we should call set_vq_ready() from
> > > > qemu/virtio_vdpa etc. after reset to explicitly set ready to false.
> > > 
> > > This is not what I read from the spec and how the current driver behave.
> > > 
> > > And I don't see why we need to stick to 1 after the reset.
> > Like you said below, this must be clarified by the spec.
> 
> 
> I will add this to my todo list.

Thanks.

> 
> 
> > Because my
> > understanding of the text is that the ready flag is a means for the
> > driver to tell the device whether it may or may not execute requests. So
> > following this reasoning, the driver should tell the device when ready
> > should be zero. As you suggest, after reset.
> 
> 
> Another thought, reset should prevent the device from executing requests on
> each queue. In this case, presenting vq_ready to zero make sense. Otherwise
> there would be a conflict.
> 

Right.

> Thanks
> 
> 
> > 
> > Meanwhile I am going to change the code to reset ready after device
> > reset.
> > 
> > > 
> > > > The ready indication is not necessairily a reflection of the hardware
> > > > queue:
> > > > 
> > > > "Virtual queue ready bit
> > > > Writing one (0x1) to this register notifies the device that it can
> > > > execute requests from this virtual queue. Reading from this register
> > > > returns the last value written to it. Both read and write accesses apply
> > > > to the queue selected by writing to QueueSel."
> > > 
> > > My understanding that this applies if not reset in the middle. We can
> > > clarify this in the spec if needed.
> > > 
> > > Thanks
> > > 
> > > 
> > > > 
> > > > > Thanks
> > > > > 
> > > > > 
> > > > > >     }
> > > > > >     static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
> 

Re: [PATCH 2/3] vdpa/mlx5: Only return vq ready if vq is initialized

[Eli Cohen]

Please abandon this patch. I am sending instead a patch that resets
ready after reset.

On Tue, Jun 01, 2021 at 01:03:36PM +0800, Jason Wang wrote:
> 
> 在 2021/6/1 下午12:13, Eli Cohen 写道:
> > On Tue, Jun 01, 2021 at 10:18:04AM +0800, Jason Wang wrote:
> > > 在 2021/6/1 上午12:04, Eli Cohen 写道:
> > > > Only return the value of the ready field if the VQ is initialized in
> > > > which case the value of the field is valid.
> > > > 
> > > > Failing to do so can result in virtio_vdpa failing to load if the device
> > > > was previously used by vhost_vdpa and the old values are ready.
> > > > virtio_vdpa expects to find VQs in "not ready" state.
> > > > 
> > > > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > > > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > > > ---
> > > >    drivers/vdpa/mlx5/net/mlx5_vnet.c | 2 +-
> > > >    1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > index 02a05492204c..f6b680d2ab1c 100644
> > > > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > > > @@ -1407,7 +1407,7 @@ static bool mlx5_vdpa_get_vq_ready(struct vdpa_device *vdev, u16 idx)
> > > >    	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
> > > >    	struct mlx5_vdpa_virtqueue *mvq = &ndev->vqs[idx];
> > > > -	return mvq->ready;
> > > > +	return mvq->initialized && mvq->ready;
> > > 
> > > I think the more suitable fix is to reset mvq->ready during reset. The
> > > vq_ready should follow the queue_enable semantic in virtio-pci:
> > > 
> > > "
> > > The device MUST present a 0 in queue_enable on reset.
> > > "
> > Thinking again, I think we should call set_vq_ready() from
> > qemu/virtio_vdpa etc. after reset to explicitly set ready to false.
> 
> 
> This is not what I read from the spec and how the current driver behave.
> 
> And I don't see why we need to stick to 1 after the reset.
> 
> 
> > 
> > The ready indication is not necessairily a reflection of the hardware
> > queue:
> > 
> > "Virtual queue ready bit
> > Writing one (0x1) to this register notifies the device that it can
> > execute requests from this virtual queue. Reading from this register
> > returns the last value written to it. Both read and write accesses apply
> > to the queue selected by writing to QueueSel."
> 
> 
> My understanding that this applies if not reset in the middle. We can
> clarify this in the spec if needed.
> 
> Thanks
> 
> 
> > 
> > 
> > > Thanks
> > > 
> > > 
> > > >    }
> > > >    static int mlx5_vdpa_set_vq_state(struct vdpa_device *vdev, u16 idx,
>