[PATCH net-next 13/15] net/mlx5: E-switch, drop master egress rule under a SW vport LAG

[Tariq Toukan <tariqt@nvidia.com>]

From: Shay Drory <shayd@nvidia.com>

VPORT_LAG steers all traffic of a LAG slave to the LAG master. The
eswitch master egress rule does the opposite direction: it bounces
traffic that reached the master's manager vport into the slave eswitch.
With a FW vport LAG the FW steers that bounced packet to the LAG master,
so the two coexist.

The software vport LAG emulation cannot. It is a catch-all rule in the
slave eswitch's egress ACL that forwards to the master's manager vport,
while the master egress rule redirects the packet after the FDB - past
the point where the slave egress rule is evaluated - so the bounced
packet never reaches it.

The two therefore cannot coexist: drop the master egress rule when the
software vport LAG is installed, and restore it when it is removed. Both
rules are keyed by the peer vhca_id, so each slave swaps its own pair
independently. Restoring needs the ACL size, so pass ldev->max_funcs
down to mlx5_eswitch_offloads_vport_lag_del_one(), as the single FDB
path already does.

Also, release the MPESW metadata after the shared FDB is destroyed,
mirroring the creation order. Releasing it first makes all slaves fall
back to the same default uplink metadata, so the restored master egress
rules collide.

Note that vport LAG on its own only gets traffic to the LAG master; the
distribution to the other slaves esw managers, which was managed by
master egress rules, is done by the LAG demux

Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  3 ++-
 .../mellanox/mlx5/core/eswitch_offloads.c     | 24 +++++++++++++++++--
 .../ethernet/mellanox/mlx5/core/lag/mpesw.c   |  2 +-
 .../mellanox/mlx5/core/lag/shared_fdb.c       |  6 +++--
 4 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index e9cbcd73b23f..55f5ea522d46 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -957,7 +957,8 @@ bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 cont
 int mlx5_eswitch_offloads_vport_lag_add_one(struct mlx5_eswitch *master_esw,
 					    struct mlx5_eswitch *slave_esw);
 void mlx5_eswitch_offloads_vport_lag_del_one(struct mlx5_eswitch *master_esw,
-					     struct mlx5_eswitch *slave_esw);
+					     struct mlx5_eswitch *slave_esw,
+					     int max_slaves);
 int mlx5_eswitch_offloads_single_fdb_add_one(struct mlx5_eswitch *master_esw,
 					     struct mlx5_eswitch *slave_esw, int max_slaves);
 void mlx5_eswitch_offloads_single_fdb_del_one(struct mlx5_eswitch *master_esw,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index d633c63b309b..f1a5d7d036cb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3381,12 +3381,32 @@ void mlx5_eswitch_offloads_single_fdb_del_one(struct mlx5_eswitch *master_esw,
 int mlx5_eswitch_offloads_vport_lag_add_one(struct mlx5_eswitch *master_esw,
 					    struct mlx5_eswitch *slave_esw)
 {
-	return esw_set_slave_egress_rule(master_esw->dev, slave_esw->dev);
+	int err;
+
+	err = esw_set_slave_egress_rule(master_esw->dev, slave_esw->dev);
+	if (err)
+		return err;
+
+	esw_unset_master_egress_rule(master_esw->dev, slave_esw->dev);
+
+	return 0;
 }
 
 void mlx5_eswitch_offloads_vport_lag_del_one(struct mlx5_eswitch *master_esw,
-					     struct mlx5_eswitch *slave_esw)
+					     struct mlx5_eswitch *slave_esw,
+					     int max_slaves)
 {
+	int err;
+
+	if (mlx5_sd_is_primary(slave_esw->dev)) {
+		err = esw_set_master_egress_rule(master_esw->dev,
+						 slave_esw->dev, max_slaves);
+		if (err)
+			esw_warn(master_esw->dev,
+				 "Failed to restore master egress rule (%d)\n",
+				 err);
+	}
+
 	esw_unset_slave_egress_rule(master_esw->dev, slave_esw->dev);
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
index 584436668a5c..3dd7fa8de66c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/mpesw.c
@@ -208,10 +208,10 @@ void mlx5_lag_disable_mpesw(struct mlx5_lag *ldev)
 	if (ldev->mode != MLX5_LAG_MODE_MPESW)
 		return;
 
-	mlx5_mpesw_metadata_cleanup(ldev);
 	mlx5_lag_shared_fdb_destroy(ldev, MLX5_LAG_FILTER_ALL);
 	if (mlx5_lag_has_sd_group(ldev))
 		mlx5_mpesw_restore_sd_fdb(ldev);
+	mlx5_mpesw_metadata_cleanup(ldev);
 
 	/* Pair with the master mark taken at MPESW create for VFs. */
 	if (ldev->virt_lag)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c b/drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c
index 404278a15ef7..90209e4b99f7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lag/shared_fdb.c
@@ -126,7 +126,8 @@ int mlx5_lag_create_vport_lag(struct mlx5_lag *ldev, u32 group_id)
 		if (j == master_idx)
 			continue;
 		slave_esw = mlx5_lag_fn(ldev, j)->dev->priv.eswitch;
-		mlx5_eswitch_offloads_vport_lag_del_one(master_esw, slave_esw);
+		mlx5_eswitch_offloads_vport_lag_del_one(master_esw, slave_esw,
+							ldev->max_fns);
 	}
 	return err;
 }
@@ -153,7 +154,8 @@ int mlx5_lag_destroy_vport_lag(struct mlx5_lag *ldev, u32 group_id)
 			continue;
 		dev = mlx5_lag_fn(ldev, i)->dev;
 		mlx5_eswitch_offloads_vport_lag_del_one(master_esw,
-							dev->priv.eswitch);
+							dev->priv.eswitch,
+							ldev->max_fns);
 	}
 	return 0;
 }
-- 
2.44.0