From: Tariq Toukan <tariqt@nvidia.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, <netdev@vger.kernel.org>,
Paolo Abeni <pabeni@redhat.com>,
Sabrina Dubroca <sd@queasysnail.net>
Cc: Boris Pismenny <borisp@nvidia.com>,
Carolina Jubran <cjubran@nvidia.com>,
Cosmin Ratiu <cratiu@nvidia.com>,
Daniel Zahka <daniel.zahka@gmail.com>,
Dragos Tatulea <dtatulea@nvidia.com>,
Emeel Hakim <ehakim@nvidia.com>, "Gal Pressman" <gal@nvidia.com>,
Jianbo Liu <jianbol@nvidia.com>,
Leon Romanovsky <leon@kernel.org>,
open list <linux-kernel@vger.kernel.org>,
<linux-rdma@vger.kernel.org>, Lior Nahmanson <liorna@nvidia.com>,
Mark Bloch <mbloch@nvidia.com>, Parav Pandit <parav@nvidia.com>,
Raed Salem <raeds@nvidia.com>,
Rahul Rameshbabu <rrameshbabu@nvidia.com>,
Roi Dayan <roid@nvidia.com>, Saeed Mahameed <saeedm@nvidia.com>,
Steffen Klassert <steffen.klassert@secunet.com>,
Tariq Toukan <tariqt@nvidia.com>
Subject: [PATCH net 1/7] net/mlx5e: ipsec: Block eswitch mode changes during policy creation
Date: Thu, 17 Sep 2026 20:54:27 +0300 [thread overview]
Message-ID: <20260917175433.4090878-2-tariqt@nvidia.com> (raw)
In-Reply-To: <20260917175433.4090878-1-tariqt@nvidia.com>
From: Cosmin Ratiu <cratiu@nvidia.com>
Eswitch mode changes can tear down the IPsec context while policy
creation is accessing it. The mode-blocking reference acquired when
creating a flow table comes too late: the table lookup already accesses
the IPsec context before taking that reference.
Block mode changes before checking the IPsec context and validating the
policy. Release the temporary reference after successful setup, when the
flow table holds its own reference, or after unwinding on failure.
Unfortunately, simply using mlx5_eswitch_block_mode() for this would
introduce a regression where:
1. An offloaded inbound IPsec policy is added on the uplink.
2. A TC flower rule is added on a VF representor.
3. Another uplink IPsec policy using the same RX tables is added.
Before this change, the 3rd rule would reuse an existing RX/TX table
from 1 and would avoid an mlx5_eswitch_block_mode() check in
rx_get()/tx_get(). After this change, the temporary mode block added
would reject the 3rd rule because esw->user_count > 0.
To avoid that, I was forced to separate the mode blocking from the TC
check by introducing a check_users argument to
mlx5_eswitch_block_mode().
Fixes: a5b8ca9471d3 ("net/mlx5e: Add XFRM policy offload logic")
Signed-off-by: Cosmin Ratiu <cratiu@nvidia.com>
Reviewed-by: Dragos Tatulea <dtatulea@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../mellanox/mlx5/core/en_accel/ipsec.c | 23 +++++++++++++++----
.../mellanox/mlx5/core/en_accel/ipsec_fs.c | 4 ++--
.../net/ethernet/mellanox/mlx5/core/eswitch.c | 11 +++++----
.../net/ethernet/mellanox/mlx5/core/eswitch.h | 10 +++++---
.../mellanox/mlx5/core/eswitch_offloads.c | 7 +++---
5 files changed, 36 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index db260e3d1412..841ecdc2c4d9 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -806,7 +806,7 @@ static int mlx5e_xfrm_add_state(struct net_device *dev,
goto err_xfrm;
}
- err = mlx5_eswitch_block_mode(priv->mdev);
+ err = mlx5_eswitch_block_mode(priv->mdev, true);
if (err)
goto unblock_ipsec;
@@ -1266,18 +1266,28 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x,
int err;
priv = netdev_priv(netdev);
+ /* Block esw mode changes until the policy holds its own block. */
+ err = mlx5_eswitch_block_mode(priv->mdev, false);
+ if (err) {
+ NL_SET_ERR_MSG_MOD(extack, "Eswitch busy, can't add policy");
+ return err;
+ }
+
if (!priv->ipsec) {
NL_SET_ERR_MSG_MOD(extack, "Device doesn't support IPsec packet offload");
- return -EOPNOTSUPP;
+ err = -EOPNOTSUPP;
+ goto unblock_mode;
}
err = mlx5e_xfrm_validate_policy(priv->mdev, x, extack);
if (err)
- return err;
+ goto unblock_mode;
pol_entry = kzalloc_obj(*pol_entry);
- if (!pol_entry)
- return -ENOMEM;
+ if (!pol_entry) {
+ err = -ENOMEM;
+ goto unblock_mode;
+ }
pol_entry->x = x;
pol_entry->ipsec = priv->ipsec;
@@ -1293,6 +1303,7 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x,
goto err_fs;
x->xdo.offload_handle = (unsigned long)pol_entry;
+ mlx5_eswitch_unblock_mode(priv->mdev);
return 0;
err_fs:
@@ -1300,6 +1311,8 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x,
ipsec_busy:
kfree(pol_entry);
NL_SET_ERR_MSG_MOD(extack, "Device failed to offload this policy");
+unblock_mode:
+ mlx5_eswitch_unblock_mode(priv->mdev);
return err;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
index 329608c59313..5c72656c623f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
@@ -1127,7 +1127,7 @@ static int rx_get(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
if (rx->ft.refcnt)
goto skip;
- err = mlx5_eswitch_block_mode(mdev);
+ err = mlx5_eswitch_block_mode(mdev, true);
if (err)
return err;
@@ -1416,7 +1416,7 @@ static int tx_get(struct mlx5_core_dev *mdev, struct mlx5e_ipsec *ipsec,
if (tx->ft.refcnt)
goto skip;
- err = mlx5_eswitch_block_mode(mdev);
+ err = mlx5_eswitch_block_mode(mdev, true);
if (err)
return err;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index fc197d1dc9df..0b48cc7a6734 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -3005,21 +3005,22 @@ void mlx5_esw_put(struct mlx5_core_dev *mdev)
/**
* mlx5_esw_try_lock() - Take a write lock on esw mode lock.
* @esw: eswitch device.
+ * @check_users: reject the lock if eswitch users exist.
*
* Should be called by esw mode change routine.
*
* Return:
- * * 0 - esw mode if successfully locked and refcount is 0.
- * * -EBUSY - refcount is not 0.
- * * -EINVAL - In the middle of switching mode or lock is already held.
+ * * >= 0 - esw mode if successfully locked.
+ * * -EBUSY - mode change in progress or users exist with check_users set.
+ * * -EINVAL - lock is already held.
*/
-int mlx5_esw_try_lock(struct mlx5_eswitch *esw)
+int mlx5_esw_try_lock(struct mlx5_eswitch *esw, bool check_users)
{
if (down_write_trylock(&esw->mode_lock) == 0)
return -EINVAL;
if (esw->eswitch_operation_in_progress ||
- atomic64_read(&esw->user_count) > 0) {
+ (check_users && atomic64_read(&esw->user_count) > 0)) {
up_write(&esw->mode_lock);
return -EBUSY;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 8b1f93b13ea9..4a9a1656f6db 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -946,7 +946,7 @@ bool mlx5_esw_hold(struct mlx5_core_dev *dev);
void mlx5_esw_release(struct mlx5_core_dev *dev);
void mlx5_esw_get(struct mlx5_core_dev *dev);
void mlx5_esw_put(struct mlx5_core_dev *dev);
-int mlx5_esw_try_lock(struct mlx5_eswitch *esw);
+int mlx5_esw_try_lock(struct mlx5_eswitch *esw, bool check_users);
int mlx5_esw_lock(struct mlx5_eswitch *esw);
void mlx5_esw_unlock(struct mlx5_eswitch *esw);
@@ -970,7 +970,7 @@ bool mlx5_eswitch_is_peer(struct mlx5_eswitch *esw,
bool mlx5_eswitch_block_encap(struct mlx5_core_dev *dev, bool from_fdb);
void mlx5_eswitch_unblock_encap(struct mlx5_core_dev *dev);
-int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev);
+int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev, bool check_users);
void mlx5_eswitch_unblock_mode(struct mlx5_core_dev *dev);
static inline int mlx5_eswitch_num_vfs(struct mlx5_eswitch *esw)
@@ -1081,7 +1081,11 @@ static inline void mlx5_eswitch_unblock_encap(struct mlx5_core_dev *dev)
{
}
-static inline int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev) { return 0; }
+static inline int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev,
+ bool check_users)
+{
+ return 0;
+}
static inline void mlx5_eswitch_unblock_mode(struct mlx5_core_dev *dev) {}
static inline bool mlx5_eswitch_block_ipsec(struct mlx5_core_dev *dev)
{
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index eb74b6260168..996490e11625 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -4343,7 +4343,7 @@ static int esw_inline_mode_to_devlink(u8 mlx5_mode, u8 *mode)
return 0;
}
-int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev)
+int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev, bool check_users)
{
struct mlx5_eswitch *esw = dev->priv.eswitch;
int err;
@@ -4351,8 +4351,7 @@ int mlx5_eswitch_block_mode(struct mlx5_core_dev *dev)
if (!mlx5_esw_allowed(esw))
return 0;
- /* Take TC into account */
- err = mlx5_esw_try_lock(esw);
+ err = mlx5_esw_try_lock(esw, check_users);
if (err < 0)
return err;
@@ -4485,7 +4484,7 @@ int mlx5_devlink_eswitch_mode_set(struct devlink *devlink, u16 mode,
return 0;
mlx5_lag_disable_change(esw->dev);
- err = mlx5_esw_try_lock(esw);
+ err = mlx5_esw_try_lock(esw, true);
if (err < 0) {
NL_SET_ERR_MSG_MOD(extack, "Can't change mode, E-Switch is busy");
goto enable_lag;
--
2.44.0
next prev parent reply other threads:[~2026-09-17 18:10 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-17 17:54 [PATCH net 0/7] net/mlx5e: Fix offload lifetime, cleanup and exclusion bugs Tariq Toukan
2026-09-17 17:54 ` Tariq Toukan [this message]
2026-09-17 17:54 ` [PATCH net 2/7] net/mlx5e: ipsec: Block eswitch mode changes before accessing priv->ipsec Tariq Toukan
2026-09-17 17:54 ` [PATCH net 3/7] net/mlx5e: tc: Tie esw & accel blocking refs to the flow's lifetime Tariq Toukan
2026-09-17 17:54 ` [PATCH net 4/7] net/mlx5e: macsec: Track hardware object ownership for SA teardown Tariq Toukan
2026-09-17 17:54 ` [PATCH net 5/7] net/mlx5e: macsec: Delete remaining SecYs during cleanup Tariq Toukan
2026-09-17 17:54 ` [PATCH net 6/7] net/mlx5e: Serialize TC and IPsec offload exclusion counters Tariq Toukan
2026-09-17 17:54 ` [PATCH net 7/7] net/mlx5e: shampo: Do not merge PSP packets Tariq Toukan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260917175433.4090878-2-tariqt@nvidia.com \
--to=tariqt@nvidia.com \
--cc=andrew+netdev@lunn.ch \
--cc=borisp@nvidia.com \
--cc=cjubran@nvidia.com \
--cc=cratiu@nvidia.com \
--cc=daniel.zahka@gmail.com \
--cc=davem@davemloft.net \
--cc=dtatulea@nvidia.com \
--cc=edumazet@google.com \
--cc=ehakim@nvidia.com \
--cc=gal@nvidia.com \
--cc=jianbol@nvidia.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=liorna@nvidia.com \
--cc=mbloch@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=parav@nvidia.com \
--cc=raeds@nvidia.com \
--cc=roid@nvidia.com \
--cc=rrameshbabu@nvidia.com \
--cc=saeedm@nvidia.com \
--cc=sd@queasysnail.net \
--cc=steffen.klassert@secunet.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®