Re: [PATCH rdma-next 1/2] RDMA/mlx5: Fix DEVX UMEM mailbox size truncation

[Edward Srouji <edwards@nvidia.com>]

On 11/9/2026 4:48 PM, Jason Gunthorpe wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Mon, Nov 09, 2026 at 04:42:32PM +0200, Edward Srouji wrote:
>> From: Uri Dar <udar@nvidia.com>
>>
>> devx_umem_reg_cmd_alloc() stores the CREATE_UMEM mailbox size, a header
>> plus 8 bytes per MTT, in the u32 cmd->inlen. The number of MTTs is
>> derived from the user-supplied length and pgsz_bitmap, so a UMEM of about
>> 2TB registered with 4KB pages wraps the size, the mailbox is
>> under-allocated and mlx5_ib_populate_pas() writes past its end.
>>
>> Store the size in an int, the in_size type of mlx5_cmd_exec(), where a
>> larger value would turn negative, and compute it with overflow checks so
>> a UMEM whose mailbox does not fit is rejected.
>>
>> Mailbox sizes within 512 bytes of INT_MAX still overflow the block count
>> in mlx5_calc_cmd_blocks(), but the command then fails with -ENOMEM
>> before reaching firmware. This predates the change.
>>
>> Fixes: aeae94579caf ("IB/mlx5: Add DEVX support for memory registration")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Uri Dar <udar@nvidia.com>
>> Reviewed-by: Michael Gur <michaelgur@nvidia.com>
>> Signed-off-by: Edward Srouji <edwards@nvidia.com>
>> ---
>>   drivers/infiniband/hw/mlx5/devx.c | 15 +++++++++------
>>   1 file changed, 9 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
>> index fc6f793dcc6584e6a22b40c57e2df8f24579eb14..1da6c0e440d6e19d8791b012a148f479cfdddad7 100644
>> --- a/drivers/infiniband/hw/mlx5/devx.c
>> +++ b/drivers/infiniband/hw/mlx5/devx.c
>> @@ -113,7 +113,7 @@ struct devx_umem {
>>
>>   struct devx_umem_reg_cmd {
>>        void                            *in;
>> -     u32                             inlen;
>> +     int                             inlen;
> 
> Do not change things which must be unsigned into signed values.

I asked to use check_add_overflow() but it was dumb to ignore this from 
my side. Will fix in V2.

> 
> Jason