[PATCH v1] vdpa/mlx5: Clear vq ready indication upon device reset

[PATCH v1] vdpa/mlx5: Clear vq ready indication upon device reset

[Eli Cohen]

After device reset, the virtqueues are not ready so clear the ready
field.

Failing to do so can result in virtio_vdpa failing to load if the device
was previously used by vhost_vdpa and the old values are ready.
virtio_vdpa expects to find VQs in "not ready" state.

Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
Signed-off-by: Eli Cohen <elic@nvidia.com>
---
v0 --> v1:
  Make sure clear of ready is done only in reset flow

 drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 02a05492204c..eaeae67e0ff0 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -1766,6 +1766,14 @@ static void teardown_driver(struct mlx5_vdpa_net *ndev)
 	mutex_unlock(&ndev->reslock);
 }
 
+static void clear_vqs_ready(struct mlx5_vdpa_net *ndev)
+{
+	int i;
+
+	for (i = ndev->mvdev.max_vqs - 1; i >= 0; i--)
+		ndev->vqs[i].ready = false;
+}
+
 static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
 {
 	struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
@@ -1776,6 +1784,7 @@ static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
 	if (!status) {
 		mlx5_vdpa_info(mvdev, "performing device reset\n");
 		teardown_driver(ndev);
+		clear_vqs_ready(ndev);
 		mlx5_vdpa_destroy_mr(&ndev->mvdev);
 		ndev->mvdev.status = 0;
 		ndev->mvdev.mlx_features = 0;
-- 
2.31.1

Re: [PATCH v1] vdpa/mlx5: Clear vq ready indication upon device reset

[Jason Wang]

在 2021/6/3 下午4:10, Eli Cohen 写道:
> After device reset, the virtqueues are not ready so clear the ready
> field.
>
> Failing to do so can result in virtio_vdpa failing to load if the device
> was previously used by vhost_vdpa and the old values are ready.
> virtio_vdpa expects to find VQs in "not ready" state.
>
> Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> Signed-off-by: Eli Cohen <elic@nvidia.com>
> ---
> v0 --> v1:
>    Make sure clear of ready is done only in reset flow
>
>   drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 +++++++++
>   1 file changed, 9 insertions(+)
>
> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> index 02a05492204c..eaeae67e0ff0 100644
> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> @@ -1766,6 +1766,14 @@ static void teardown_driver(struct mlx5_vdpa_net *ndev)
>   	mutex_unlock(&ndev->reslock);
>   }
>   
> +static void clear_vqs_ready(struct mlx5_vdpa_net *ndev)
> +{
> +	int i;
> +
> +	for (i = ndev->mvdev.max_vqs - 1; i >= 0; i--)
> +		ndev->vqs[i].ready = false;


The patch looks correct but I wonder the reason for the decreasing used 
in the loop.

Usually, it means it has some reason that must be done in that way.

Thanks


> +}
> +
>   static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
>   {
>   	struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
> @@ -1776,6 +1784,7 @@ static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
>   	if (!status) {
>   		mlx5_vdpa_info(mvdev, "performing device reset\n");
>   		teardown_driver(ndev);
> +		clear_vqs_ready(ndev);
>   		mlx5_vdpa_destroy_mr(&ndev->mvdev);
>   		ndev->mvdev.status = 0;
>   		ndev->mvdev.mlx_features = 0;

Re: [PATCH v1] vdpa/mlx5: Clear vq ready indication upon device reset

[Eli Cohen]

On Fri, Jun 04, 2021 at 10:34:29AM +0800, Jason Wang wrote:
> 
> 在 2021/6/3 下午4:10, Eli Cohen 写道:
> > After device reset, the virtqueues are not ready so clear the ready
> > field.
> > 
> > Failing to do so can result in virtio_vdpa failing to load if the device
> > was previously used by vhost_vdpa and the old values are ready.
> > virtio_vdpa expects to find VQs in "not ready" state.
> > 
> > Fixes: 1a86b377aa21 ("vdpa/mlx5: Add VDPA driver for supported mlx5 devices")
> > Signed-off-by: Eli Cohen <elic@nvidia.com>
> > ---
> > v0 --> v1:
> >    Make sure clear of ready is done only in reset flow
> > 
> >   drivers/vdpa/mlx5/net/mlx5_vnet.c | 9 +++++++++
> >   1 file changed, 9 insertions(+)
> > 
> > diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > index 02a05492204c..eaeae67e0ff0 100644
> > --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> > @@ -1766,6 +1766,14 @@ static void teardown_driver(struct mlx5_vdpa_net *ndev)
> >   	mutex_unlock(&ndev->reslock);
> >   }
> > +static void clear_vqs_ready(struct mlx5_vdpa_net *ndev)
> > +{
> > +	int i;
> > +
> > +	for (i = ndev->mvdev.max_vqs - 1; i >= 0; i--)
> > +		ndev->vqs[i].ready = false;
> 
> 
> The patch looks correct but I wonder the reason for the decreasing used in
> the loop.

No special reason. I just copy pasted it from somewhere else in the
code. Will post another version.
> 
> Usually, it means it has some reason that must be done in that way.
> 
> Thanks
> 
> 
> > +}
> > +
> >   static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
> >   {
> >   	struct mlx5_vdpa_dev *mvdev = to_mvdev(vdev);
> > @@ -1776,6 +1784,7 @@ static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status)
> >   	if (!status) {
> >   		mlx5_vdpa_info(mvdev, "performing device reset\n");
> >   		teardown_driver(ndev);
> > +		clear_vqs_ready(ndev);
> >   		mlx5_vdpa_destroy_mr(&ndev->mvdev);
> >   		ndev->mvdev.status = 0;
> >   		ndev->mvdev.mlx_features = 0;
>