From: Edward Srouji <edwards@nvidia.com>
To: Leon Romanovsky <leon@kernel.org>, Jason Gunthorpe <jgg@ziepe.ca>,
"Sabrina Dubroca" <sd@queasysnail.net>
Cc: <linux-rdma@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<netdev@vger.kernel.org>, Edward Srouji <edwards@nvidia.com>,
"Akiva Goldberger" <agoldberger@nvidia.com>,
Michael Gur <michaelgur@nvidia.com>
Subject: [PATCH rdma-next 00/11] RDMA/mlx5: Preparation for SD support
Date: Tue, 17 Nov 2026 16:01:11 +0200 [thread overview]
Message-ID: <20261117-mlx5-sd-prep-v1-0-8efd124c7895@nvidia.com> (raw)
This series prepares mlx5_ib for Socket Direct (SD), where one IB device
spans several core devices, one per PCIe function. Today the driver
assumes a single core device per IB device and reaches it through
dev->mdev all over the code.
Instead, each call site now finds its core device through the resource
it works on: QPs, SRQs, AHs and MWs through their PD, and DEVX objects,
UARs and mmaps through their ucontext. A CQ and an MR store the core
device they were created on, since they cannot always reach their owner
when they need it. The SD series can then change where a resource lives
in the resolvers, without touching the call sites.
The resolvers follow one naming scheme: <resource>_to_mdev() returns a
core device and <resource>_to_dev() an IB device. Where several resource
types are accepted, a _Generic() dispatches to per-type helpers, which
are prefixed with '__' and are not meant to be called directly.
State kept once per IB device, such as the ODP page-fault EQ, delay drop
and local loopback, stays on dev->mdev. The SD series decides how each
of them works across core devices.
This series only changes how each call site finds its core device. With
one core device per IB device every resolver returns dev->mdev, so the
behaviour is unchanged. Pre-existing issues in the touched code are
intentionally left alone so that each patch stays a mechanical
conversion.
Patches 1-2: name device pointers consistently, dev for the IB device
and mdev for the core device.
Patches 3-7: resolve the core device through the PD.
Patches 8-11: resolve the core device through the ucontext, and keep it
on the CQ.
Signed-off-by: Edward Srouji <edwards@nvidia.com>
---
Akiva Goldberger (11):
RDMA/mlx5: Enforce dev/mdev naming for device pointers
RDMA/mlx5: Rename to_mdev() helpers that return an IB device
RDMA/mlx5: Select the PD and SRQ core device through the PD
RDMA/mlx5: Select the core device through PD for QP commands
RDMA/mlx5: Cache the MR core device and resolve the MW's via its PD
RDMA/mlx5: Select the AH core device through their PD
RDMA/mlx5: Select the QP core device through its PD
RDMA/mlx5: Resolve core device from ucontext for DEVX resources
RDMA/mlx5: Cache the CQ core device and resolve it via cq_to_mdev()
RDMA/mlx5: Resolve the bfreg UAR-index core device from the ucontext
RDMA/mlx5: Resolve UAR and mmap core devices via the ucontext
drivers/infiniband/hw/mlx5/ah.c | 20 +-
drivers/infiniband/hw/mlx5/cmd.c | 52 +--
drivers/infiniband/hw/mlx5/cmd.h | 26 +-
drivers/infiniband/hw/mlx5/counters.c | 32 +-
drivers/infiniband/hw/mlx5/cq.c | 120 +++----
drivers/infiniband/hw/mlx5/devx.c | 159 ++++-----
drivers/infiniband/hw/mlx5/devx.h | 10 +-
drivers/infiniband/hw/mlx5/dm.c | 74 +++--
drivers/infiniband/hw/mlx5/dmah.c | 10 +-
drivers/infiniband/hw/mlx5/fs.c | 32 +-
drivers/infiniband/hw/mlx5/gsi.c | 27 +-
drivers/infiniband/hw/mlx5/ib_rep.c | 48 +--
drivers/infiniband/hw/mlx5/ib_virt.c | 12 +-
drivers/infiniband/hw/mlx5/macsec.c | 10 +-
drivers/infiniband/hw/mlx5/mad.c | 18 +-
drivers/infiniband/hw/mlx5/main.c | 270 +++++++--------
drivers/infiniband/hw/mlx5/mlx5_ib.h | 151 +++++++--
drivers/infiniband/hw/mlx5/mr.c | 228 +++++++------
drivers/infiniband/hw/mlx5/odp.c | 64 ++--
drivers/infiniband/hw/mlx5/qos.c | 13 +-
drivers/infiniband/hw/mlx5/qp.c | 580 +++++++++++++++++----------------
drivers/infiniband/hw/mlx5/qp.h | 34 +-
drivers/infiniband/hw/mlx5/qpc.c | 109 ++++---
drivers/infiniband/hw/mlx5/restrack.c | 15 +-
drivers/infiniband/hw/mlx5/srq.c | 50 +--
drivers/infiniband/hw/mlx5/srq.h | 4 +-
drivers/infiniband/hw/mlx5/srq_cmd.c | 124 ++++---
drivers/infiniband/hw/mlx5/std_types.c | 10 +-
drivers/infiniband/hw/mlx5/umr.c | 133 ++++----
drivers/infiniband/hw/mlx5/umr.h | 22 +-
drivers/infiniband/hw/mlx5/wr.c | 15 +-
31 files changed, 1330 insertions(+), 1142 deletions(-)
---
base-commit: 36d1eca030a37baa00a56bf34962f36263181918
change-id: 20261117-mlx5-sd-prep-571490a1fad3
Best regards,
--
Edward Srouji <edwards@nvidia.com>
next reply other threads:[~2026-11-17 14:02 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-17 14:01 Edward Srouji [this message]
2026-11-17 14:01 ` [PATCH rdma-next 01/11] RDMA/mlx5: Enforce dev/mdev naming for device pointers Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 02/11] RDMA/mlx5: Rename to_mdev() helpers that return an IB device Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 03/11] RDMA/mlx5: Select the PD and SRQ core device through the PD Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 04/11] RDMA/mlx5: Select the core device through PD for QP commands Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 05/11] RDMA/mlx5: Cache the MR core device and resolve the MW's via its PD Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 06/11] RDMA/mlx5: Select the AH core device through their PD Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 07/11] RDMA/mlx5: Select the QP core device through its PD Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 08/11] RDMA/mlx5: Resolve core device from ucontext for DEVX resources Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 09/11] RDMA/mlx5: Cache the CQ core device and resolve it via cq_to_mdev() Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 10/11] RDMA/mlx5: Resolve the bfreg UAR-index core device from the ucontext Edward Srouji
2026-11-17 14:01 ` [PATCH rdma-next 11/11] RDMA/mlx5: Resolve UAR and mmap core devices via " Edward Srouji
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261117-mlx5-sd-prep-v1-0-8efd124c7895@nvidia.com \
--to=edwards@nvidia.com \
--cc=agoldberger@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=michaelgur@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=sd@queasysnail.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®