From: Tariq Toukan <tariqt@nvidia.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Jakub Kicinski <kuba@kernel.org>, <netdev@vger.kernel.org>,
Paolo Abeni <pabeni@redhat.com>
Cc: Alexandre Cassen <acassen@corp.free.fr>,
Alex Vesker <valex@nvidia.com>, Erez Shitrit <erezsh@nvidia.com>,
Gal Pressman <gal@nvidia.com>,
Kees Cook <kees+treewide@kernel.org>,
Leon Romanovsky <leon@kernel.org>,
open list <linux-kernel@vger.kernel.org>,
<linux-rdma@vger.kernel.org>, Mark Bloch <mbloch@nvidia.com>,
Moshe Shemesh <moshe@nvidia.com>,
Prathamesh Deshpande <prathameshdeshpande7@gmail.com>,
Saeed Mahameed <saeedm@nvidia.com>,
"Simon Horman" <horms@kernel.org>,
Tariq Toukan <tariqt@nvidia.com>,
"William Theesfeld" <william@theesfeld.net>,
Yevgeny Kliteynik <kliteyn@nvidia.com>
Subject: [PATCH net V2 2/2] net/mlx5: DR, Fix out-of-bounds write when sync to HW fails
Date: Wed, 18 Nov 2026 19:32:27 +0200 [thread overview]
Message-ID: <20261118173227.1957670-3-tariqt@nvidia.com> (raw)
In-Reply-To: <20261118173227.1957670-1-tariqt@nvidia.com>
From: Yevgeny Kliteynik <kliteyn@nvidia.com>
mlx5dr_icm_free_chunk() moves a freed chunk to the "hot" chunks array by
writing it to hot_chunks_arr and, once the accumulated hot memory
crosses a threshold, triggers a sync to HW that flushes the array. The
array is large enough to hold the chunks of a single accumulation cycle,
and relies on the sync path to flush it.
However, if sync to HW fails, hot chunks array is not cleared, and every
subsequent mlx5dr_icm_free_chunk will write past the end of
hot_chunks_arr, resulting in an out-of-bounds write and kernel heap
corruption.
A failed sync to HW also means it is no longer safe to recycle ICM: a
chunk's memory may only be returned to the buddy allocator (and thus be
reused) after a sync has confirmed that HW no longer references it.
The issue was discovered by a fuzzer that repeatedly triggered sync
failure, at which point the out-of-bounds write was reported by KASAN:
KASAN log excerpt: BUG:
slab-out-of-bounds in mlx5dr_icm_free_chunk+0x4cd at call 2
(hot_chunks_num=3 > cap=2).
Fix this by putting the pool into an error state on the first failed
sync to HW, reporting the failure once. The sync is not retried, and
while the pool is in this state:
- mlx5dr_icm_free_chunk() doesn't queue freed chunks; the segment is
left reserved and is reclaimed when the pool is destroyed.
- mlx5dr_icm_alloc_chunk() doesn't serve new chunks.
Fixes: 4519fc45beeb ("net/mlx5: DR, Keep track of hot ICM chunks in an array instead of list")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../mlx5/core/steering/sws/dr_icm_pool.c | 32 +++++++++++++++++--
1 file changed, 29 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/steering/sws/dr_icm_pool.c b/drivers/net/ethernet/mellanox/mlx5/core/steering/sws/dr_icm_pool.c
index 1b928b40686e..a906a72601c4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/steering/sws/dr_icm_pool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/steering/sws/dr_icm_pool.c
@@ -33,6 +33,11 @@ struct mlx5dr_icm_pool {
u64 hot_memory_size;
/* hot memory size threshold for triggering sync */
u64 th;
+ /* Set when a sync to HW fails - no new chunks are served, freed
+ * chunks are no longer queued, and their ICM is not returned to
+ * the buddy allocator.
+ */
+ bool error_state;
};
struct mlx5dr_icm_dm {
@@ -368,7 +373,15 @@ static int dr_icm_pool_sync_all_buddy_pools(struct mlx5dr_icm_pool *pool)
err = mlx5dr_cmd_sync_steering(pool->dmn->mdev);
if (err) {
- mlx5dr_err(pool->dmn, "Failed to sync to HW (err: %d)\n", err);
+ /* The hot chunks could not be flushed. Put the pool in an
+ * error state so that no further chunks are queued (which
+ * would overflow hot_chunks_arr), and no new chunks are
+ * served. The sync is not retried.
+ */
+ mlx5dr_err(pool->dmn,
+ "Failed to sync to HW (err: %d), disabling ICM pool\n",
+ err);
+ pool->error_state = true;
return err;
}
@@ -443,6 +456,10 @@ mlx5dr_icm_alloc_chunk(struct mlx5dr_icm_pool *pool,
return NULL;
mutex_lock(&pool->mutex);
+
+ if (pool->error_state)
+ goto out;
+
/* find mem, get back the relevant buddy pool and seg in that mem */
ret = dr_icm_handle_buddies_get_mem(pool, chunk_size, &buddy, &seg);
if (ret)
@@ -475,6 +492,14 @@ void mlx5dr_icm_free_chunk(struct mlx5dr_icm_chunk *chunk)
/* move the chunk to the waiting chunks array, AKA "hot" memory */
mutex_lock(&pool->mutex);
+ /* If the pool is in an error state, the chunk must not be queued.
+ * ICM memory is not returned to the buddy allocator - it is
+ * reclaimed only when the pool is destroyed.
+ * Just free the chunk's bookkeeping struct.
+ */
+ if (pool->error_state)
+ goto out;
+
pool->hot_memory_size += mlx5dr_icm_pool_get_chunk_byte_size(chunk);
hot_chunk = &pool->hot_chunks_arr[pool->hot_chunks_num++];
@@ -482,12 +507,13 @@ void mlx5dr_icm_free_chunk(struct mlx5dr_icm_chunk *chunk)
hot_chunk->seg = chunk->seg;
hot_chunk->size = chunk->size;
- kmem_cache_free(chunks_cache, chunk);
-
/* Check if we have chunks that are waiting for sync-ste */
if (dr_icm_pool_is_sync_required(pool))
dr_icm_pool_sync_all_buddy_pools(pool);
+out:
+ kmem_cache_free(chunks_cache, chunk);
+
mutex_unlock(&pool->mutex);
}
--
2.44.0
next prev parent reply other threads:[~2026-11-18 17:33 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 17:32 [PATCH net V2 0/2] net/mlx5: Fix HWS and DR steering memory safety bugs Tariq Toukan
2026-11-18 17:32 ` [PATCH net V2 1/2] net/mlx5: HWS, Handle NULL in vport destination action creation Tariq Toukan
2026-11-18 17:32 ` Tariq Toukan [this message]
2026-11-18 17:37 ` [PATCH net V2 0/2] net/mlx5: Fix HWS and DR steering memory safety bugs netdev-bot+sinfo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118173227.1957670-3-tariqt@nvidia.com \
--to=tariqt@nvidia.com \
--cc=acassen@corp.free.fr \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=erezsh@nvidia.com \
--cc=gal@nvidia.com \
--cc=horms@kernel.org \
--cc=kees+treewide@kernel.org \
--cc=kliteyn@nvidia.com \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=moshe@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=prathameshdeshpande7@gmail.com \
--cc=saeedm@nvidia.com \
--cc=valex@nvidia.com \
--cc=william@theesfeld.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®