[PATCH net-next 10/15] net/mlx5: LAG, do not add a device to an already-active LAG

[Tariq Toukan <tariqt@nvidia.com>]

From: Shay Drory <shayd@nvidia.com>

mlx5_ldev_add_mdev() would add a newly probed device to the LAG even
when the LAG is already active, mutating membership while getters may be
iterating it and while the active mapping assumes a fixed member set.

Reject the join when the group's LAG is active:
__mlx5_lag_dev_add_mdev() returns -EOPNOTSUPP, and mlx5_lag_add_mdev()
treats that as terminal - unregister the HCA devcom component and leave
the device standalone, rather than the -EAGAIN retry used for the
bounded mode-change window.
The device stays out of the LAG until it is reprobed.

This keeps LAG membership stable while active. It is a no-op for
existing configurations, where no device is added to a LAG once it is
active.

In addition, do not register the eswitch devcom component of such a
device, so it does not pair with the eswitches of the LAG members.

Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch_offloads.c   | 4 ++++
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c        | 9 +++++++++
 drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h        | 5 +++++
 3 files changed, 18 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index c712848b202d..70e6dd9561a6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3573,6 +3573,10 @@ void mlx5_esw_offloads_devcom_init(struct mlx5_eswitch *esw,
 	xa_init(&esw->offloads.peer_flows);
 	mutex_init(&esw->offloads.peer_mutex);
 
+	/* do not pair a virt LAG capable VF/SF that is not part of a LAG */
+	if (!mlx5_core_is_pf(esw->dev) && mlx5_virt_lag_nonmember(esw->dev))
+		return;
+
 	if (!MLX5_CAP_ESW(esw->dev, merged_eswitch))
 		return;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
index 9bc4a929cef3..d73bac8441f7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.c
@@ -2449,6 +2449,11 @@ static int __mlx5_lag_dev_add_mdev(struct mlx5_core_dev *dev)
 		mutex_unlock(&ldev->lock);
 		return -EAGAIN;
 	}
+
+	if (mlx5_virt_lag_is_supported(dev) && __mlx5_lag_is_active(ldev)) {
+		mutex_unlock(&ldev->lock);
+		return -EOPNOTSUPP;
+	}
 	mlx5_ldev_get(ldev);
 	err = mlx5_ldev_add_mdev(ldev, dev, 0);
 	if (err) {
@@ -2537,6 +2542,10 @@ void mlx5_lag_add_mdev(struct mlx5_core_dev *dev)
 	err = __mlx5_lag_dev_add_mdev(dev);
 	mlx5_devcom_comp_unlock(dev->priv.hca_devcom_comp);
 
+	if (err == -EOPNOTSUPP) {
+		mlx5_lag_unregister_hca_devcom_comp(dev);
+		return;
+	}
 	if (err) {
 		msleep(100);
 		goto recheck;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
index cbe380ae8bf9..385c767e24ae 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/lag.h
@@ -280,6 +280,11 @@ static inline bool mlx5_virt_lag_is_supported(const struct mlx5_core_dev *dev)
 	return true;
 }
 
+static inline bool mlx5_virt_lag_nonmember(struct mlx5_core_dev *dev)
+{
+	return mlx5_virt_lag_is_supported(dev) && !mlx5_lag_dev(dev);
+}
+
 static inline bool mlx5_lag_is_supported(struct mlx5_core_dev *dev)
 {
 	if (!MLX5_CAP_GEN(dev, vport_group_manager) ||
-- 
2.44.0