[PATCH vfio 0/2] vfio/mlx5: locking updates

[PATCH vfio 0/2] vfio/mlx5: locking updates

[Brett Creeley]

The vfio/pds series for locking updates/fixes in the following link
made some changes that can also be done for other vendor's vfio
drivers. Specifically, changing the reset lock from a spinlock to mutex
and also calling mutex_destroy() in the vfio device release callback.

https://lore.kernel.org/kvm/20231122192532.25791-1-brett.creeley@amd.com/

So, this series makes these changes in order to remain separate from
the vfio/pds series linked above.

Note, that I don't have the required hardware to test on this vendor's
hardware, so help would be appreciated.

Brett Creeley (2):
  vfio/mlx5: Change reset_lock to mutex_lock
  vfio/mlx5: Destroy the state_mutex on release

 drivers/vfio/pci/mlx5/cmd.c  |  4 +++-
 drivers/vfio/pci/mlx5/cmd.h  |  3 +--
 drivers/vfio/pci/mlx5/main.c | 12 ++++++------
 3 files changed, 10 insertions(+), 9 deletions(-)

-- 
2.17.1

[PATCH vfio 1/2] vfio/mlx5: Change reset_lock to mutex_lock

[Brett Creeley]

Based on comments from other vfio vendors and the
maintainer the vfio/pds driver changed the reset_lock
to a mutex_lock. As part of that change it was requested
that the other vendor drivers be changed as well. So,
make the change.

The comment that requested the change for reference:
https://lore.kernel.org/kvm/BN9PR11MB52769E037CB356AB15A0D9B88CA0A@BN9PR11MB5276.namprd11.prod.outlook.com/

Also, make checkpatch happy by moving the lock comment.

Signed-off-by: Brett Creeley <brett.creeley@amd.com>
---
 drivers/vfio/pci/mlx5/cmd.c  |  3 ++-
 drivers/vfio/pci/mlx5/cmd.h  |  3 +--
 drivers/vfio/pci/mlx5/main.c | 12 ++++++------
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index efd1d252cdc9..5cf68ab2bbd7 100644
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -202,6 +202,7 @@ void mlx5vf_cmd_remove_migratable(struct mlx5vf_pci_core_device *mvdev)
 	mlx5_sriov_blocking_notifier_unregister(mvdev->mdev, mvdev->vf_id,
 						&mvdev->nb);
 	destroy_workqueue(mvdev->cb_wq);
+	mutex_destroy(&mvdev->reset_mutex);
 }
 
 void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev,
@@ -238,7 +239,7 @@ void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev,
 		goto end;
 
 	mutex_init(&mvdev->state_mutex);
-	spin_lock_init(&mvdev->reset_lock);
+	mutex_init(&mvdev->reset_mutex);
 	mvdev->nb.notifier_call = mlx5fv_vf_event;
 	ret = mlx5_sriov_blocking_notifier_register(mvdev->mdev, mvdev->vf_id,
 						    &mvdev->nb);
diff --git a/drivers/vfio/pci/mlx5/cmd.h b/drivers/vfio/pci/mlx5/cmd.h
index f2c7227fa683..1cfd126724a2 100644
--- a/drivers/vfio/pci/mlx5/cmd.h
+++ b/drivers/vfio/pci/mlx5/cmd.h
@@ -183,8 +183,7 @@ struct mlx5vf_pci_core_device {
 	/* protect migration state */
 	struct mutex state_mutex;
 	enum vfio_device_mig_state mig_state;
-	/* protect the reset_done flow */
-	spinlock_t reset_lock;
+	struct mutex reset_mutex; /* protect the reset_done flow */
 	struct mlx5_vf_migration_file *resuming_migf;
 	struct mlx5_vf_migration_file *saving_migf;
 	struct mlx5_vhca_page_tracker tracker;
diff --git a/drivers/vfio/pci/mlx5/main.c b/drivers/vfio/pci/mlx5/main.c
index b6ac66c5008d..689edfd750e1 100644
--- a/drivers/vfio/pci/mlx5/main.c
+++ b/drivers/vfio/pci/mlx5/main.c
@@ -1284,16 +1284,16 @@ mlx5vf_pci_step_device_state_locked(struct mlx5vf_pci_core_device *mvdev,
 void mlx5vf_state_mutex_unlock(struct mlx5vf_pci_core_device *mvdev)
 {
 again:
-	spin_lock(&mvdev->reset_lock);
+	mutex_lock(&mvdev->reset_mutex);
 	if (mvdev->deferred_reset) {
 		mvdev->deferred_reset = false;
-		spin_unlock(&mvdev->reset_lock);
+		mutex_unlock(&mvdev->reset_mutex);
 		mvdev->mig_state = VFIO_DEVICE_STATE_RUNNING;
 		mlx5vf_disable_fds(mvdev);
 		goto again;
 	}
 	mutex_unlock(&mvdev->state_mutex);
-	spin_unlock(&mvdev->reset_lock);
+	mutex_unlock(&mvdev->reset_mutex);
 }
 
 static struct file *
@@ -1372,13 +1372,13 @@ static void mlx5vf_pci_aer_reset_done(struct pci_dev *pdev)
 	 * In case the state_mutex was taken already we defer the cleanup work
 	 * to the unlock flow of the other running context.
 	 */
-	spin_lock(&mvdev->reset_lock);
+	mutex_lock(&mvdev->reset_mutex);
 	mvdev->deferred_reset = true;
 	if (!mutex_trylock(&mvdev->state_mutex)) {
-		spin_unlock(&mvdev->reset_lock);
+		mutex_unlock(&mvdev->reset_mutex);
 		return;
 	}
-	spin_unlock(&mvdev->reset_lock);
+	mutex_unlock(&mvdev->reset_mutex);
 	mlx5vf_state_mutex_unlock(mvdev);
 }
 
-- 
2.17.1

[PATCH vfio 2/2] vfio/mlx5: Destroy the state_mutex on release

[Brett Creeley]

The state_mutex is initialized in vfio init, but
never destroyed. This isn't required as mutex_destroy()
doesn't do anything unless lock debugging is enabled.
However, for completeness, fix it.

No fixes tag is added as it doesn't seem worthwhile
for such a trivial and debug only change.

Signed-off-by: Brett Creeley <brett.creeley@amd.com>
---
 drivers/vfio/pci/mlx5/cmd.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index 5cf68ab2bbd7..4ae5295757b4 100644
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -203,6 +203,7 @@ void mlx5vf_cmd_remove_migratable(struct mlx5vf_pci_core_device *mvdev)
 						&mvdev->nb);
 	destroy_workqueue(mvdev->cb_wq);
 	mutex_destroy(&mvdev->reset_mutex);
+	mutex_destroy(&mvdev->state_mutex);
 }
 
 void mlx5vf_cmd_set_migratable(struct mlx5vf_pci_core_device *mvdev,
-- 
2.17.1