* [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions
@ 2026-10-08 17:18 Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
Hi,
This series by Shahar introduces mlx5_nodnic, a Linux driver for the
NODNIC (No-DRAM NIC) hardware interface present in Mellanox/NVIDIA
ConnectX-family ASICs since 2015.
NODNIC is a first-class hardware capability.
It defines a lightweight NIC function intended for contexts where local
DRAM is unavailable or too costly: embedded controllers, management
planes, pre-boot environments, or any resource-constrained instantiation
of a ConnectX NIC. The hardware sends and receives ordinary Ethernet
frames via SQs, RQs and CQs. There is no encapsulation, no private
protocol, and no FW interposition: packets are forwarded as standard
Ethernet between whatever peers are connected to the function. The
interface is documented in the HW spec and has been in production in
non-Linux OS drivers for over a decade.
Where it is used today:
- Embedded management controllers on ConnectX-class ASICs that need a
network path to the host without consuming a data-path PF.
- Resource-constrained SoC integrations (no local DRAM) where a
minimal-footprint driver is required.
- Pre-boot / early-OS environments (PXE, iSCSI boot) that need a NIC
without a full mlx5e stack.
- BF4 management PFs (one concrete deployment): on BlueField-4 the
firmware exposes a NODNIC PF on the host side and a peer PF on the ARM
side, using standard vport forwarding to connect them. This provides
in-band Ethernet connectivity between the host and the DPU OS - SSH,
PXE, and any standard networking application work over it without any
special handling.
Design notes:
- The driver is separate from mlx5e rather than a mode within it.
NODNIC uses PCI Vendor Specific Capability (VSC) for command
operations - reads and writes over PCI config space - instead of the
DMA-based command interface used by mlx5_core. This is what makes
NODNIC usable in environments without reliable DMA access, and it
makes the two command paths fundamentally incompatible. Folding this
into mlx5e would mean carrying a second command transport and a second
resource model through code that is already dense.
- A single SQ, RQ and shared CQ are exposed per function, with two MSI-X
vectors: the data vector fires on CQEs and drives both TX and RX
completions, and the event vector notifies on port state changes.
This is the hardware's model, not a driver simplification.
- The driver exposes a plain netdev with no private uAPI, no ioctls and
no vendor-specific configuration surface. Everything is driven through
standard netdev and ethtool interfaces.
- The code lives under drivers/net/ethernet/mellanox/mlx5/nodnic/ and
shares no paths with mlx5e, keeping the footprint small and the blast
radius of changes contained.
Regards,
Tariq
V2:
- Fix sparse warnings in the VSC accessors: use __be32 for the
big-endian accessors and drop the redundant le32 conversions, which
were also wrong on big-endian hosts (Simon Horman).
- Cover letter: drop the devlink mention (Simon Horman).
- Replace the open-coded bitfield macros in the VSC code with
GENMASK/BIT and FIELD_GET/u32_replace_bits, and turn the config space
accessor macros into functions.
- Fix line wrapping reported by checkpatch --strict.
- Add a MAINTAINERS entry for the driver.
- Kconfig: select PAGE_POOL, fixing the build when it is not otherwise
enabled.
- Drop inline from a function in a .c file.
V1:
https://lore.kernel.org/all/20260930125708.144767-1-tariqt@nvidia.com/
Shahar Shitrit (5):
net/mlx5: nodnic: Introduce the mlx5_nodnic driver
net/mlx5: nodnic: Implement data path
net/mlx5: nodnic: Add TX/RX statistics
net/mlx5: nodnic: Add ethtool driver info
net/mlx5: nodnic: Add health monitoring
MAINTAINERS | 9 +
drivers/net/ethernet/mellanox/Kconfig | 1 +
drivers/net/ethernet/mellanox/Makefile | 1 +
.../net/ethernet/mellanox/mlx5/nodnic/Kconfig | 19 +
.../ethernet/mellanox/mlx5/nodnic/Makefile | 8 +
.../net/ethernet/mellanox/mlx5/nodnic/en.c | 263 ++++
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 158 +++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 1107 +++++++++++++++++
.../ethernet/mellanox/mlx5/nodnic/health.c | 315 +++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 866 +++++++++++++
.../ethernet/mellanox/mlx5/nodnic/nodnic.h | 105 ++
.../mellanox/mlx5/nodnic/nodnic_ifc.h | 318 +++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.c | 343 +++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.h | 45 +
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 172 +++
.../net/ethernet/mellanox/mlx5/nodnic/rx.h | 25 +
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 230 ++++
.../net/ethernet/mellanox/mlx5/nodnic/tx.h | 61 +
18 files changed, 4046 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
base-commit: 8df0638138d3e0344fd1fb36cf2d1ca1cf5028f0
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
@ 2026-10-08 17:18 ` Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
From: Shahar Shitrit <shshitrit@nvidia.com>
Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC
functions.
It defines a lightweight NIC function intended for contexts where local
DRAM is unavailable or too costly: embedded controllers, management
planes, pre-boot environments, or any resource-constrained instantiation
of a ConnectX NIC. The hardware sends and receives ordinary Ethernet
frames via SQs, RQs and CQs. There is no encapsulation, no private
protocol, and no FW interposition: packets are forwarded as standard
Ethernet between whatever peers are connected to the function. The
interface is documented in the HW spec and has been in production in
non-Linux OS drivers for over a decade.
This patch implements the PCI probe and remove flows, including FLR,
firmware readiness, interface negotiation and capability discovery,
followed by port and ring configuration.
It also adds the VSC access layer for PCI configuration-space accesses
to the initialization segment and to the NODNIC configuration registers.
Set up the event IRQ for link-state notifications, retrieve the port MAC
address, and implement the netdev lifecycle, including ndo_open and
ndo_stop.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
MAINTAINERS | 9 +
drivers/net/ethernet/mellanox/Kconfig | 1 +
drivers/net/ethernet/mellanox/Makefile | 1 +
.../net/ethernet/mellanox/mlx5/nodnic/Kconfig | 18 +
.../ethernet/mellanox/mlx5/nodnic/Makefile | 8 +
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 97 ++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 828 +++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 864 ++++++++++++++++++
.../ethernet/mellanox/mlx5/nodnic/nodnic.h | 105 +++
.../mellanox/mlx5/nodnic/nodnic_ifc.h | 318 +++++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.c | 343 +++++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.h | 45 +
12 files changed, 2637 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 72ca3aab2106..6e06bc31c00f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16998,6 +16998,15 @@ F: drivers/infiniband/hw/mlx5/
F: include/linux/mlx5/
F: include/uapi/rdma/mlx5-abi.h
+MELLANOX MLX5 NODNIC DRIVER
+M: Shahar Shitrit <shshitrit@nvidia.com>
+M: Tariq Toukan <tariqt@nvidia.com>
+L: netdev@vger.kernel.org
+S: Maintained
+W: https://www.nvidia.com/networking/
+Q: https://patchwork.kernel.org/project/netdevbpf/list/
+F: drivers/net/ethernet/mellanox/mlx5/nodnic/
+
MELLANOX MLX5 VDPA DRIVER
M: Dragos Tatulea <dtatulea@nvidia.com>
L: virtualization@lists.linux.dev
diff --git a/drivers/net/ethernet/mellanox/Kconfig b/drivers/net/ethernet/mellanox/Kconfig
index b4f66eb9ddb9..979a029c0171 100644
--- a/drivers/net/ethernet/mellanox/Kconfig
+++ b/drivers/net/ethernet/mellanox/Kconfig
@@ -23,5 +23,6 @@ source "drivers/net/ethernet/mellanox/mlx5/core/Kconfig"
source "drivers/net/ethernet/mellanox/mlxsw/Kconfig"
source "drivers/net/ethernet/mellanox/mlxfw/Kconfig"
source "drivers/net/ethernet/mellanox/mlxbf_gige/Kconfig"
+source "drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig"
endif # NET_VENDOR_MELLANOX
diff --git a/drivers/net/ethernet/mellanox/Makefile b/drivers/net/ethernet/mellanox/Makefile
index d4b5f547a727..ec46837fa729 100644
--- a/drivers/net/ethernet/mellanox/Makefile
+++ b/drivers/net/ethernet/mellanox/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_MLX5_CORE) += mlx5/core/
obj-$(CONFIG_MLXSW_CORE) += mlxsw/
obj-$(CONFIG_MLXFW) += mlxfw/
obj-$(CONFIG_MLXBF_GIGE) += mlxbf_gige/
+obj-$(CONFIG_MLX5_NODNIC) += mlx5/nodnic/
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
new file mode 100644
index 000000000000..461750a9c97c
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config MLX5_NODNIC
+ tristate "Mellanox NODNIC core driver"
+ depends on PCI
+ depends on NETDEVICES && ETHERNET
+ help
+ This driver provides core support for Mellanox NODNIC (NO DRAM NIC)
+ devices. NODNIC is a lightweight network
+ interface driver that uses Vendor Specific Capability (VSC) for
+ PCI communication with Mellanox network adapters.
+
+ This driver enables basic Ethernet networking functionality for
+ supported Mellanox NODNIC hardware.
+
+ To compile this driver as a module, choose M here: the module
+ will be called mlx5_nodnic.
+
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
new file mode 100644
index 000000000000..4955f0427c79
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for Mellanox NODNIC driver
+#
+
+obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
+
+mlx5_nodnic-y := main.o en_cfg.o nodnic_pci_vsc.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
new file mode 100644
index 000000000000..90dc57695e1b
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_EN_H__
+#define __MLX5_NODNIC_EN_H__
+
+#include <linux/if_vlan.h>
+#include <linux/mlx5/device.h>
+#include <linux/mutex.h>
+#include <linux/netdevice.h>
+#include <linux/timer.h>
+#include <linux/types.h>
+
+#define MLX5_NODNIC_RING_ENTRIES 1024
+#define MLX5_NODNIC_CQE_SIZE_B 64
+#define MLX5_NODNIC_SQ_WQEBB_B 64
+#define MLX5_NODNIC_RQ_STRIDE_BYTES 16
+
+enum {
+ MLX5_NODNIC_STATE_OPENED,
+};
+
+enum {
+ MLX5_NODNIC_RCV_DBR,
+ MLX5_NODNIC_SND_DBR,
+ MLX5_NODNIC_DBR_WORDS,
+};
+
+struct mlx5_nodnic_ring {
+ void *wq;
+ dma_addr_t dma_handle;
+ u32 size;
+ u32 num_entries;
+ u32 qn;
+ u32 lkey;
+};
+
+struct mlx5_nodnic_sq {
+ struct device *device;
+ struct net_device *netdev;
+ struct mlx5_nodnic_ring ring;
+
+ u16 pc;
+ u16 cc;
+};
+
+struct mlx5_nodnic_rq {
+ struct mlx5_nodnic_ring ring;
+ u16 pc;
+ u16 cc;
+};
+
+struct mlx5_nodnic_dbr {
+ __be32 *db;
+ dma_addr_t dma_handle;
+ u32 alloc_size;
+};
+
+struct mlx5_nodnic_cq {
+ struct mlx5_nodnic_ring ring;
+ int cqn;
+ int num_entries;
+ int log_num_entries;
+
+ struct mlx5_nodnic_dbr dbr;
+
+ int cc;
+};
+
+/* for FW use only */
+struct mlx5_nodnic_working_buffer {
+ dma_addr_t dma_handle;
+ u32 alloc_size;
+};
+
+struct mlx5_nodnic_priv {
+ struct mlx5_nodnic_core_dev *core_dev;
+ struct net_device *netdev;
+
+ struct mutex state_lock; /* protects Interface state */
+ unsigned long state;
+
+ struct mlx5_nodnic_sq sq;
+ struct mlx5_nodnic_rq rq;
+ struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
+ struct mlx5_nodnic_cq cq;
+ struct mlx5_nodnic_ring working_buffer;
+};
+
+void mlx5_nodnic_build_netdev(struct net_device *netdev);
+struct net_device *mlx5_nodnic_create_netdev(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv);
+void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open(struct net_device *netdev);
+int mlx5_nodnic_close(struct net_device *netdev);
+
+#endif /* __MLX5_NODNIC_EN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
new file mode 100644
index 000000000000..5724ed41d8b8
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -0,0 +1,828 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/bitops.h>
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/interrupt.h>
+#include <linux/log2.h>
+#include <linux/pci.h>
+#include <net/netdev_lock.h>
+#include <net/net_namespace.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+#include "nodnic_pci_vsc.h"
+
+#define MLX5_NODNIC_CQ_INIT_CMD_SN cpu_to_be32(2 << 28)
+
+static int mlx5_nodnic_write_ring_addr(struct mlx5_nodnic_core_dev *dev,
+ dma_addr_t dma_handle, u32 offset_h,
+ u32 offset_l)
+{
+ __be32 addr_h, addr_l;
+ int err;
+
+ addr_h = cpu_to_be32(upper_32_bits(dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_h, addr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring addr high: %d\n",
+ err);
+ return err;
+ }
+
+ addr_l = 0;
+ MLX5_SET(nodnic_q_addr_l, &addr_l, q_addr_l,
+ lower_32_bits(dma_handle) >> 12);
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_l, addr_l);
+ if (err)
+ mlx5_nodnic_core_err(dev, "failed to write ring addr low: %d\n",
+ err);
+ return err;
+}
+
+static u32 mlx5_nodnic_calc_ring_size(struct mlx5_nodnic_core_dev *dev,
+ int stride)
+{
+ u32 ring_size = MLX5_NODNIC_RING_ENTRIES * stride;
+ u32 max_supported = 1U << dev->log_max_ring_size;
+
+ return min(ring_size, max_supported);
+}
+
+static int mlx5_nodnic_write_ring_size(struct mlx5_nodnic_core_dev *dev,
+ int offset, int size)
+{
+ __be32 addr_l;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &addr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read ring size: %d\n",
+ err);
+ return err;
+ }
+
+ MLX5_SET(nodnic_q_addr_l, &addr_l, log_size, ilog2(size));
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset, addr_l);
+ if (err)
+ mlx5_nodnic_core_err(dev, "failed to write ring size: %d\n",
+ err);
+ return err;
+}
+
+static int mlx5_nodnic_read_qn(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, u32 offset)
+{
+ __be32 qn_dword;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &qn_dword);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read queue number: %d\n",
+ err);
+ return err;
+ }
+
+ ring->qn = MLX5_GET(nodnic_ring_q_number, &qn_dword, queue_number);
+
+ return 0;
+}
+
+static int mlx5_nodnic_write_ring_dbr_addr(struct mlx5_nodnic_core_dev *dev,
+ u32 ring_offset)
+{
+ int off_h = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_H;
+ int off_l = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_L;
+ struct mlx5_nodnic_dbr *dbr = &dev->priv->dbr;
+ __be32 dbr_h, dbr_l;
+ int err;
+
+ dbr_l = 0;
+ MLX5_SET(nodnic_ring_dbr_addr_l, &dbr_l, db_record_addr_l,
+ (u32)(dbr->dma_handle >> 2));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_l, dbr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring DBR addr low: %d\n",
+ err);
+ return err;
+ }
+
+ dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_h, dbr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring DBR addr high: %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_alloc_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, int size)
+{
+ dma_addr_t dma_handle;
+ void *buf;
+
+ buf = dma_alloc_coherent(dev->device, size, &dma_handle, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ *ring = (struct mlx5_nodnic_ring) {
+ .wq = buf,
+ .dma_handle = dma_handle,
+ .size = size,
+ };
+
+ return 0;
+}
+
+static void mlx5_nodnic_free_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring)
+{
+ dma_free_coherent(dev->device, ring->size, ring->wq,
+ ring->dma_handle);
+ memset(ring, 0, sizeof(*ring));
+}
+
+static int mlx5_nodnic_init_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, u32 ring_offset,
+ int stride)
+{
+ int ring_addr_h = ring_offset + MLX5_NODNIC_RING_ADDR_H;
+ int ring_addr_l = ring_offset + MLX5_NODNIC_RING_ADDR_L;
+ int size, err;
+
+ size = mlx5_nodnic_calc_ring_size(dev, stride);
+
+ err = mlx5_nodnic_alloc_ring(dev, ring, size);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle, ring_addr_h,
+ ring_addr_l);
+ if (err)
+ goto err_ring_free;
+
+ err = mlx5_nodnic_write_ring_size(dev, ring_addr_l, size);
+ if (err)
+ goto err_ring_free;
+
+ ring->num_entries = size / stride;
+
+ err = mlx5_nodnic_read_qn(dev, ring,
+ ring_offset + MLX5_NODNIC_RING_QUEUE_NUMBER);
+ if (err)
+ goto err_ring_free;
+
+ err = mlx5_nodnic_write_ring_dbr_addr(dev, ring_offset);
+ if (err)
+ goto err_ring_free;
+
+ return 0;
+
+err_ring_free:
+ mlx5_nodnic_free_ring(dev, ring);
+ return err;
+}
+
+static int mlx5_nodnic_alloc_dbr(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_dbr *dbr)
+{
+ const u32 bytes = MLX5_NODNIC_DBR_WORDS * sizeof(__be32);
+ dma_addr_t dma_handle;
+ void *buf;
+
+ buf = dma_alloc_coherent(dev->device, bytes, &dma_handle, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ *dbr = (struct mlx5_nodnic_dbr) {
+ .db = (__be32 *)buf,
+ .alloc_size = bytes,
+ .dma_handle = dma_handle,
+ };
+
+ return 0;
+}
+
+static void mlx5_nodnic_free_dbr(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_dbr *dbr)
+{
+ if (!dbr->db)
+ return;
+
+ dma_free_coherent(dev->device, dbr->alloc_size, dbr->db,
+ dbr->dma_handle);
+}
+
+static int mlx5_nodnic_init_cq_dbr(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_dbr *dbr = &dev->priv->cq.dbr;
+ __be32 dbr_h, dbr_l;
+ int err;
+
+ err = mlx5_nodnic_alloc_dbr(dev, dbr);
+ if (err)
+ return err;
+
+ dbr_l = cpu_to_be32(lower_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_CQ_DBR_ADDR_L,
+ dbr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write CQ DBR addr low: %d\n",
+ err);
+ goto err_free_dbr;
+ }
+
+ dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_CQ_DBR_ADDR_H,
+ dbr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write CQ DBR addr high: %d\n",
+ err);
+ goto err_free_dbr;
+ }
+
+ dev->priv->cq.dbr.db[1] = MLX5_NODNIC_CQ_INIT_CMD_SN;
+
+ return 0;
+
+err_free_dbr:
+ mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
+ return err;
+}
+
+static void mlx5_nodnic_cleanup_cq_dbr(struct mlx5_nodnic_core_dev *dev)
+{
+ mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
+}
+
+/* log_cq_size should be read after rq and sq ring sizes were set. */
+static int mlx5_nodnic_read_cq_ring_size(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ __be32 cq_addr_l;
+ u32 log_cq_size;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_CQ_ADDR_L,
+ &cq_addr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQ ring size: %d\n",
+ err);
+ return err;
+ }
+
+ log_cq_size = MLX5_GET(nodnic_q_addr_l, &cq_addr_l, log_size);
+ cq->num_entries = 1ULL << log_cq_size;
+ cq->log_num_entries = log_cq_size;
+
+ return 0;
+}
+
+static int mlx5_nodnic_init_cq_ring(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ ring = &cq->ring;
+
+ err = mlx5_nodnic_read_cq_ring_size(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_alloc_ring(dev, ring,
+ cq->num_entries * MLX5_NODNIC_CQE_SIZE_B);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle,
+ MLX5_NODNIC_PORT_CQ_ADDR_H,
+ MLX5_NODNIC_PORT_CQ_ADDR_L);
+ if (err)
+ goto err_free_cq;
+
+ return 0;
+
+err_free_cq:
+ mlx5_nodnic_free_ring(dev, &cq->ring);
+ return err;
+}
+
+static void mlx5_nodnic_cleanup_cq_ring(struct mlx5_nodnic_core_dev *dev)
+{
+ return mlx5_nodnic_free_ring(dev, &dev->priv->cq.ring);
+}
+
+static int mlx5_nodnic_read_cq_n(struct mlx5_nodnic_core_dev *dev)
+{
+ __be32 cq_n;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, MLX5_NODNIC_CQ_N,
+ &cq_n);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQ number: %d\n",
+ err);
+ return err;
+ }
+
+ dev->priv->cq.cqn = MLX5_GET(nodnic_cqn, &cq_n, cq_n);
+
+ return 0;
+}
+
+int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_ring *work_buf = &dev->priv->working_buffer;
+ u32 size = 1U << dev->log_working_buffer_size;
+ int err;
+
+ err = mlx5_nodnic_alloc_ring(dev, work_buf, size);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(dev, work_buf->dma_handle,
+ MLX5_NODNIC_PORT_WORK_BUF_ADDR_H,
+ MLX5_NODNIC_PORT_WORK_BUF_ADDR_L);
+ if (err)
+ goto err_free_working_buffer;
+
+ return 0;
+
+err_free_working_buffer:
+ mlx5_nodnic_free_ring(dev, work_buf);
+ return err;
+}
+
+int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ int err;
+
+ cq->cc = 0;
+
+ err = mlx5_nodnic_init_cq_ring(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_init_cq_dbr(dev);
+ if (err)
+ goto err_cleanup_ring;
+
+ err = mlx5_nodnic_read_cq_n(dev);
+ if (err)
+ goto err_cleanup_dbr;
+
+ return 0;
+
+err_cleanup_dbr:
+ mlx5_nodnic_cleanup_cq_dbr(dev);
+err_cleanup_ring:
+ mlx5_nodnic_cleanup_cq_ring(dev);
+ return err;
+}
+
+static void mlx5_nodnic_close_cq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_core_dev *dev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+
+ mlx5_nodnic_cleanup_cq_dbr(dev);
+ mlx5_nodnic_cleanup_cq_ring(dev);
+}
+
+int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ sq->netdev = priv->netdev;
+ sq->device = dev->device;
+ sq->pc = 0;
+ sq->cc = 0;
+ ring = &sq->ring;
+
+ err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_SEND_RING_OFFSET,
+ MLX5_NODNIC_SQ_WQEBB_B);
+ if (err)
+ return err;
+
+ ring->lkey = dev->lkey;
+
+ return 0;
+}
+
+static void mlx5_nodnic_close_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_sq *sq = &dev->priv->sq;
+
+ mlx5_nodnic_free_ring(dev, &sq->ring);
+}
+
+int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ rq->pc = 0;
+ rq->cc = 0;
+ ring = &rq->ring;
+
+ err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_RECEIVE_RING_OFFSET,
+ MLX5_NODNIC_RQ_STRIDE_BYTES);
+ if (err)
+ return err;
+
+ ring->lkey = dev->lkey;
+
+ return 0;
+}
+
+static void mlx5_nodnic_close_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+
+ mlx5_nodnic_free_ring(dev, &rq->ring);
+}
+
+static int mlx5_nodnic_open_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ int err;
+
+ /* one doorbell record shared by both SQ and RQ */
+ err = mlx5_nodnic_alloc_dbr(dev, &dev->priv->dbr);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ goto err_free_dbr;
+
+ err = mlx5_nodnic_open_sq(dev);
+ if (err)
+ goto err_release_vsc;
+
+ err = mlx5_nodnic_open_rq(dev);
+ if (err)
+ goto err_close_sq;
+
+ err = mlx5_nodnic_open_cq(dev);
+ if (err)
+ goto err_close_rq;
+
+ err = mlx5_nodnic_init_working_buffer(dev);
+ if (err)
+ goto err_close_cq;
+
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+
+ return 0;
+
+err_close_cq:
+ mlx5_nodnic_close_cq(&dev->priv->cq);
+err_close_rq:
+ mlx5_nodnic_close_rq(dev);
+err_close_sq:
+ mlx5_nodnic_close_sq(dev);
+err_release_vsc:
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+err_free_dbr:
+ mlx5_nodnic_free_dbr(dev, &dev->priv->dbr);
+
+ return err;
+}
+
+static void mlx5_nodnic_close_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+
+ mlx5_nodnic_free_ring(dev, &priv->working_buffer);
+ mlx5_nodnic_close_cq(&priv->cq);
+ mlx5_nodnic_close_rq(dev);
+ mlx5_nodnic_close_sq(dev);
+ mlx5_nodnic_free_dbr(dev, &priv->dbr);
+}
+
+static void mlx5_nodnic_update_carrier(struct mlx5_nodnic_priv *priv,
+ __be32 event)
+{
+ u32 port_state = MLX5_GET(nodnic_port_event, &event, port_state);
+ bool up = port_state == MLX5_NODNIC_PORT_STATE_ACTIVE;
+
+ /* Force re-notification if state is unchanged */
+ if (up == netif_carrier_ok(priv->netdev)) {
+ netif_carrier_event(priv->netdev);
+ return;
+ }
+ if (up) {
+ netif_carrier_on(priv->netdev);
+ netdev_info(priv->netdev, "Link up\n");
+ } else {
+ netif_carrier_off(priv->netdev);
+ netdev_info(priv->netdev, "Link down\n");
+ }
+}
+
+static int mlx5_nodnic_refresh_carrier_locked(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx;
+ __be32 event;
+ int err;
+
+ vsc_ctx = dev->vsc_ctx;
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_EVENT,
+ &event);
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+
+ if (!err)
+ mlx5_nodnic_update_carrier(priv, event);
+
+ return err;
+}
+
+void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
+{
+ mutex_lock(&priv->state_lock);
+ if (test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state))
+ mlx5_nodnic_refresh_carrier_locked(priv);
+ mutex_unlock(&priv->state_lock);
+}
+
+int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ __be32 val = 0;
+ int err;
+
+ /* data_msix_en, event_msix_en must be set before activating port */
+ MLX5_SET(nodnic_port_network, &val, data_msix_en, 1);
+ MLX5_SET(nodnic_port_network, &val, event_msix_en, 1);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (msix): %d\n",
+ err);
+ return err;
+ }
+
+ MLX5_SET(nodnic_port_network, &val, network_en, 1);
+ MLX5_SET(nodnic_port_network, &val, receive_filter_en, BIT(0));
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (enable): %d\n",
+ err);
+ return err;
+ }
+
+ /* dma enabling requires a separate write */
+ MLX5_SET(nodnic_port_network, &val, dma_en, 1);
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (dma): %d\n",
+ err);
+ return err;
+}
+
+void mlx5_nodnic_disable_port(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ __be32 val;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ &val);
+ if (err)
+ goto out;
+
+ MLX5_SET(nodnic_port_network, &val, dma_en, 0);
+ MLX5_SET(nodnic_port_network, &val, network_en, 0);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err)
+ goto out;
+
+ /* data_msix_en, event_msix_en can't be modified while port is active */
+ MLX5_SET(nodnic_port_network, &val, data_msix_en, 0);
+ MLX5_SET(nodnic_port_network, &val, event_msix_en, 0);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+
+out:
+ if (err)
+ mlx5_nodnic_core_warn(dev,
+ "failed to read/write to vsc err %d\n",
+ err);
+}
+
+static int mlx5_nodnic_init_uar_map(struct mlx5_nodnic_core_dev *dev)
+{
+ void __iomem *base;
+ phys_addr_t paddr;
+ u32 uar_page_size;
+
+ uar_page_size = 1 << dev->log_uar_page_size;
+
+ paddr = dev->bar_addr + (dev->send_ring0_uar_index * uar_page_size);
+
+ base = ioremap(paddr, uar_page_size);
+ if (!base) {
+ mlx5_nodnic_core_err(dev, "failed to map UAR\n");
+ return -ENOMEM;
+ }
+
+ dev->uar_base = base;
+
+ return 0;
+}
+
+static void mlx5_nodnic_cleanup_uar_map(struct mlx5_nodnic_core_dev *dev)
+{
+ if (!dev->uar_base)
+ return;
+
+ iounmap(dev->uar_base);
+ dev->uar_base = NULL;
+}
+
+static int mlx5_nodnic_open_locked(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ int err;
+
+ err = mlx5_nodnic_init_uar_map(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_open_queues(dev);
+ if (err)
+ goto err_uar;
+
+ set_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+
+ err = mlx5_nodnic_refresh_carrier_locked(dev->priv);
+ if (err)
+ netdev_err(dev->priv->netdev,
+ "failed to query carrier, err=%d\n", err);
+
+ return 0;
+
+err_uar:
+ mlx5_nodnic_cleanup_uar_map(dev);
+ return err;
+}
+
+int mlx5_nodnic_open(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ int err;
+
+ mutex_lock(&priv->state_lock);
+ err = mlx5_nodnic_open_locked(netdev);
+ mutex_unlock(&priv->state_lock);
+
+ return err;
+}
+
+static int mlx5_nodnic_close_locked(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+
+ clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ netif_carrier_off(priv->netdev);
+ netdev_info(priv->netdev, "Link down\n");
+ mlx5_nodnic_close_queues(dev);
+
+ return 0;
+}
+
+int mlx5_nodnic_close(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ int err = 0;
+
+ mutex_lock(&priv->state_lock);
+ if (!test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state)) {
+ mutex_unlock(&priv->state_lock);
+ return 0;
+ }
+
+ err = mlx5_nodnic_close_locked(netdev);
+ mutex_unlock(&priv->state_lock);
+
+ mlx5_nodnic_cleanup_uar_map(priv->core_dev);
+
+ return err;
+}
+
+static const struct net_device_ops mlx5_nodnic_netdev_ops = {
+ .ndo_open = mlx5_nodnic_open,
+ .ndo_stop = mlx5_nodnic_close,
+};
+
+void mlx5_nodnic_build_netdev(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *core_dev;
+
+ core_dev = priv->core_dev;
+
+ SET_NETDEV_DEV(netdev, core_dev->device);
+
+ netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
+ netdev->request_ops_lock = true;
+
+ eth_hw_addr_set(netdev, core_dev->mac_address);
+}
+
+static void mlx5_nodnic_init_priv(struct mlx5_nodnic_priv *priv,
+ struct net_device *netdev,
+ struct mlx5_nodnic_core_dev *core_dev)
+{
+ priv->netdev = netdev;
+ priv->core_dev = core_dev;
+ mutex_init(&priv->state_lock);
+}
+
+static void mlx5_nodnic_cleanup_priv(struct mlx5_nodnic_priv *priv)
+{
+ mutex_destroy(&priv->state_lock);
+}
+
+struct net_device *mlx5_nodnic_create_netdev(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv;
+ struct net_device *netdev;
+
+ netdev = alloc_etherdev_mqs(sizeof(struct mlx5_nodnic_priv), 1, 1);
+ if (!netdev)
+ return NULL;
+
+ priv = netdev_priv(netdev);
+ mlx5_nodnic_init_priv(priv, netdev, dev);
+ dev->priv = priv;
+
+ netif_carrier_off(netdev);
+ netif_tx_disable(netdev);
+ dev_net_set(netdev, &init_net);
+
+ return netdev;
+}
+
+void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv;
+ struct net_device *netdev;
+
+ if (!dev->priv)
+ return;
+
+ priv = dev->priv;
+ netdev = priv->netdev;
+ if (!netdev)
+ return;
+
+ if (dev->netdev_registered) {
+ netif_device_detach(netdev);
+ unregister_netdev(netdev);
+ dev->netdev_registered = false;
+ }
+
+ mlx5_nodnic_cleanup_priv(priv);
+ dev->priv = NULL;
+
+ free_netdev(netdev);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
new file mode 100644
index 000000000000..a533232bf6ef
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
@@ -0,0 +1,864 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+#include "nodnic_pci_vsc.h"
+
+MODULE_DESCRIPTION("Nvidia Mellanox MLX5_NODNIC core driver");
+MODULE_LICENSE("Dual BSD/GPL");
+
+#define MLX5_NODNIC_FW_INIT_TIMEOUT_MS 120000
+#define MLX5_NODNIC_FW_INIT_WARN_MS 20000
+#define MLX5_NODNIC_FW_PRE_INIT_WAIT_MS 2
+
+static int mlx5_nodnic_pci_enable_device(struct mlx5_nodnic_core_dev *dev)
+{
+ struct pci_dev *pdev = dev->pdev;
+ int err = 0;
+
+ mutex_lock(&dev->pci_status_mutex);
+ if (dev->pci_status == MLX5_NODNIC_PCI_STATUS_DISABLED) {
+ err = pci_enable_device(pdev);
+ if (!err)
+ dev->pci_status = MLX5_NODNIC_PCI_STATUS_ENABLED;
+ }
+ mutex_unlock(&dev->pci_status_mutex);
+
+ return err;
+}
+
+static void mlx5_nodnic_pci_disable_device(struct mlx5_nodnic_core_dev *dev)
+{
+ struct pci_dev *pdev = dev->pdev;
+
+ mutex_lock(&dev->pci_status_mutex);
+ if (dev->pci_status == MLX5_NODNIC_PCI_STATUS_ENABLED) {
+ pci_disable_device(pdev);
+ dev->pci_status = MLX5_NODNIC_PCI_STATUS_DISABLED;
+ }
+ mutex_unlock(&dev->pci_status_mutex);
+}
+
+static int request_bar(struct pci_dev *pdev)
+{
+ int err;
+
+ if (!(pci_resource_flags(pdev, 0) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "Missing registers BAR, aborting\n");
+ return -ENODEV;
+ }
+
+ err = pci_request_regions(pdev, KBUILD_MODNAME);
+ if (err)
+ dev_err(&pdev->dev, "Couldn't get PCI resources, aborting\n");
+
+ return err;
+}
+
+static void release_bar(struct pci_dev *pdev)
+{
+ pci_release_regions(pdev);
+}
+
+static int set_dma_caps(struct pci_dev *pdev)
+{
+ if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
+ int err;
+
+ dev_warn(&pdev->dev,
+ "couldn't set 64-bit PCI DMA mask\n");
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pdev->dev,
+ "can't set PCI DMA mask, aborting\n");
+ return err;
+ }
+ }
+
+ dma_set_max_seg_size(&pdev->dev, 2u * 1024 * 1024 * 1024);
+
+ return 0;
+}
+
+static void mlx5_nodnic_pci_close(struct mlx5_nodnic_core_dev *dev)
+{
+ pci_free_irq_vectors(dev->pdev);
+ iounmap(dev->iseg);
+ release_bar(dev->pdev);
+ mlx5_nodnic_pci_disable_device(dev);
+ mlx5_nodnic_vsc_cleanup(dev->vsc_ctx);
+}
+
+static int
+mlx5_nodnic_pci_init(struct mlx5_nodnic_core_dev *dev, struct pci_dev *pdev)
+{
+ int err;
+
+ mutex_init(&dev->pci_status_mutex);
+ pci_set_drvdata(dev->pdev, dev);
+
+ dev->bar_addr = pci_resource_start(pdev, 0);
+
+ err = mlx5_nodnic_pci_enable_device(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "cannot enable PCI device, aborting\n");
+ return err;
+ }
+
+ err = request_bar(pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "error requesting BARs, aborting\n");
+ goto err_disable;
+ }
+
+ err = set_dma_caps(pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed setting DMA capabilities mask, aborting\n");
+ goto err_release_bar;
+ }
+
+ dev->iseg = ioremap(dev->bar_addr, sizeof(*dev->iseg));
+ if (!dev->iseg) {
+ err = -ENOMEM;
+ mlx5_nodnic_core_err(dev,
+ "failed mapping initialization segment, aborting\n");
+ goto err_release_bar;
+ }
+
+ dev->vsc_ctx = mlx5_nodnic_vsc_init(dev->pdev);
+ if (!dev->vsc_ctx) {
+ err = -EINVAL;
+ goto err_unmap;
+ }
+
+ if (!pci_find_capability(dev->pdev, PCI_CAP_ID_MSIX)) {
+ mlx5_nodnic_core_err(dev,
+ "msi-x support is not supported in pci, aborting\n");
+ err = -EOPNOTSUPP;
+ goto err_vsc_cleanup;
+ }
+
+ err = pci_alloc_irq_vectors(pdev,
+ MLX5_NODNIC_NUM_MSIX, MLX5_NODNIC_NUM_MSIX,
+ PCI_IRQ_MSIX);
+ if (err < 0) {
+ mlx5_nodnic_core_err(dev, "pci_alloc_irq_vectors failed: %d\n",
+ err);
+ goto err_vsc_cleanup;
+ }
+
+ return 0;
+
+err_vsc_cleanup:
+ mlx5_nodnic_vsc_cleanup(dev->vsc_ctx);
+err_unmap:
+ iounmap(dev->iseg);
+err_release_bar:
+ release_bar(dev->pdev);
+err_disable:
+ mlx5_nodnic_pci_disable_device(dev);
+ return err;
+}
+
+int mlx5_nodnic_get_fw_initializing(struct mlx5_nodnic_core_dev *dev,
+ u8 *initializing)
+{
+ __be32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_INITIALIZING,
+ &data);
+ if (err)
+ return err;
+
+ *initializing = MLX5_GET(nodnic_iseg_initializing_word, &data,
+ initializing);
+
+ return 0;
+}
+
+static int mlx5_nodnic_wait_fw_init(struct mlx5_nodnic_core_dev *dev,
+ const char *init_state)
+{
+ unsigned long timeout =
+ msecs_to_jiffies(MLX5_NODNIC_FW_INIT_TIMEOUT_MS);
+ unsigned long warn_interval =
+ msecs_to_jiffies(MLX5_NODNIC_FW_INIT_WARN_MS);
+ unsigned long warn = jiffies + warn_interval;
+ unsigned long end = jiffies + timeout;
+ u8 fw_initializing;
+ int err;
+
+ do {
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_get_fw_initializing(dev, &fw_initializing);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read FW initializing state: %d\n",
+ err);
+ return err;
+ }
+
+ if (!fw_initializing)
+ break;
+
+ if (time_after(jiffies, end)) {
+ mlx5_nodnic_core_err(dev,
+ "timeout waiting for FW to exit %s state (%u ms)\n",
+ init_state,
+ MLX5_NODNIC_FW_INIT_TIMEOUT_MS);
+ return -ETIMEDOUT;
+ }
+ if (time_after(jiffies, warn)) {
+ mlx5_nodnic_core_warn(dev,
+ "FW still in %s state, %u ms remaining (state=0x%x)\n",
+ init_state,
+ jiffies_to_msecs(end - jiffies),
+ fw_initializing);
+
+ warn = jiffies + warn_interval;
+ }
+ msleep(MLX5_NODNIC_FW_PRE_INIT_WAIT_MS);
+ } while (true);
+
+ return 0;
+}
+
+int mlx5_nodnic_check_supported(struct mlx5_nodnic_core_dev *dev)
+{
+ __be32 initializing_word;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_INITIALIZING,
+ &initializing_word);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed reading NODNIC initializing word: %d\n",
+ err);
+ return err;
+ }
+
+ if (!(MLX5_GET(nodnic_iseg_initializing_word, &initializing_word,
+ nic_interface_supported) &
+ BIT(MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC))) {
+ mlx5_nodnic_core_err(dev,
+ "NODNIC is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_write_nic_interface(struct mlx5_nodnic_core_dev *dev,
+ int interface)
+{
+ __be32 dword;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE,
+ &dword);
+ if (err)
+ return err;
+
+ MLX5_SET(nodnic_iseg_cmdq_addr_l, &dword, nic_interface, interface);
+
+ err = mlx5_nodnic_vsc_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE,
+ dword);
+
+ return err;
+}
+
+int mlx5_nodnic_set_nic_interface(struct mlx5_nodnic_core_dev *dev)
+{
+ int interface = MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC;
+ int err;
+
+ err = mlx5_nodnic_check_supported(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_nic_interface(dev, interface);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "failed to write nic_interface err %d\n",
+ err);
+
+ return err;
+}
+
+static int mlx5_nodnic_read_general_caps(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 hardware_format, revision;
+ bool support_receive_filters;
+ __be32 caps;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_CAPS,
+ &caps);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read general caps: %d\n",
+ err);
+ return err;
+ }
+
+ revision = MLX5_GET(nodnic_gen_cfg_caps, &caps, revision);
+ hardware_format = MLX5_GET(nodnic_gen_cfg_caps, &caps, hardware_format);
+ if (revision != 1) {
+ mlx5_nodnic_core_err(dev, "revision is not 1, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (hardware_format != 1) {
+ mlx5_nodnic_core_err(dev,
+ "hardware format is not 1, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ dev->log_working_buffer_size = MLX5_GET(nodnic_gen_cfg_caps, &caps,
+ log_working_buffer_size);
+ support_receive_filters = MLX5_GET(nodnic_gen_cfg_caps, &caps,
+ support_receive_filter);
+ if (!support_receive_filters) {
+ mlx5_nodnic_core_err(dev,
+ "receive filters are not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_ring_config(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 addr = MLX5_NODNIC_GENERAL_CONFIG_LOG_MAX_RING_SIZE;
+ __be32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, addr, &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read ring config: %d\n",
+ err);
+ return err;
+ }
+
+ dev->log_max_ring_size = MLX5_GET(nodnic_gen_cfg_log_max_ring_size,
+ &data, log_max_ring_size);
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_lkey(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 lkey;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_LKEY,
+ &lkey);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read lkey: %d\n", err);
+ return err;
+ }
+
+ dev->lkey = lkey;
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_cqe_format(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 addr = MLX5_NODNIC_GENERAL_CONFIG_CQE_FORMAT;
+ u8 cqe_format;
+ __be32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, addr, &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQE format: %d\n",
+ err);
+ return err;
+ }
+
+ cqe_format = MLX5_GET(nodnic_gen_cfg_cqe_format, &data, cqe_format);
+ if (cqe_format != 0) {
+ mlx5_nodnic_core_err(dev,
+ "cqe format is not legacy (0x0), aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_uar_index(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 addr = MLX5_NODNIC_PORT_SEND_RING0_UAR_INDEX;
+ u32 uar_index;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx, addr, &uar_index);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read UAR index: %d\n",
+ err);
+ return err;
+ }
+
+ dev->send_ring0_uar_index = uar_index;
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_uar_size(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 addr = MLX5_NODNIC_GENERAL_CONFIG_LOG_UAR_PAGE_SIZE;
+ __be32 uar_size;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, addr, &uar_size);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read UAR size: %d\n", err);
+ return err;
+ }
+
+ dev->log_uar_page_size = MLX5_GET(nodnic_gen_cfg_log_uar_page_size,
+ &uar_size, log_uar_page_size);
+
+ return 0;
+}
+
+static int mlx5_nodnic_validate_feature_caps(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 addr = MLX5_NODNIC_GENERAL_CONFIG_FEATURE_CAPS;
+ u8 support_uar_tx_doorbell, support_bar_cq_ctrl;
+ u8 data_msix_support, event_msix_support;
+ __be32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, addr, &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read feature caps: %d\n",
+ err);
+ return err;
+ }
+
+ support_uar_tx_doorbell = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ support_uar_tx_doorbell);
+ if (!support_uar_tx_doorbell) {
+ mlx5_nodnic_core_err(dev,
+ "UAR tx doorbell is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ support_bar_cq_ctrl = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ support_bar_cq_ctrl);
+ if (!support_bar_cq_ctrl) {
+ mlx5_nodnic_core_err(dev,
+ "UAR cq arming is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ data_msix_support = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ data_msix_support);
+ if (!data_msix_support) {
+ mlx5_nodnic_core_err(dev,
+ "data MSI-X is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ event_msix_support = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ event_msix_support);
+ if (!event_msix_support) {
+ mlx5_nodnic_core_err(dev,
+ "event MSI-X is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_read_config(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read_offset(dev->vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read VSC offset, err=%d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_read_general_caps(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_ring_config(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_lkey(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_cqe_format(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_validate_feature_caps(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_uar_size(dev);
+ if (err)
+ return err;
+
+ return mlx5_nodnic_read_uar_index(dev);
+}
+
+static void mlx5_nodnic_dev_cleanup(struct mlx5_nodnic_core_dev *dev)
+{
+ int interface = MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED;
+ int err;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return;
+
+ err = mlx5_nodnic_write_nic_interface(dev, interface);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err)
+ return;
+
+ mlx5_nodnic_wait_fw_init(dev, "teardown");
+}
+
+static int mlx5_nodnic_dev_init(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ int err;
+
+ err = mlx5_nodnic_wait_fw_init(dev, "pre-FLR");
+ if (err)
+ return err;
+
+ pci_save_state(dev->pdev);
+ err = pcie_flr(dev->pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "pcie_flr failed with error code %d\n",
+ err);
+ return err;
+ }
+ pci_restore_state(dev->pdev);
+
+ err = mlx5_nodnic_wait_fw_init(dev, "post-FLR");
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_set_nic_interface(dev);
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_set_nic_interface failed with error code %d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_wait_fw_init(dev, "post-MLX5_NODNIC setup");
+ if (err)
+ goto disable_nic;
+
+ pci_set_master(dev->pdev);
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ goto disable_nic;
+
+ err = mlx5_nodnic_read_config(dev);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_read_config failed with error code %d\n",
+ err);
+
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+ if (err)
+ goto disable_nic;
+
+ return 0;
+
+disable_nic:
+ mlx5_nodnic_dev_cleanup(dev);
+ return err;
+}
+
+int mlx5_nodnic_validate_port_link_type(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 link_type;
+ __be32 event;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_EVENT,
+ &event);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read port event: %d\n",
+ err);
+ return err;
+ }
+
+ link_type = MLX5_GET(nodnic_port_event, &event, link_type);
+ if (!link_type) {
+ mlx5_nodnic_core_err(dev, "port is not Ethernet, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_read_mac_address(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 mac_h, mac_l;
+ u64 mac;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_MAC_ADDR_H,
+ &mac_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read MAC address high: %d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_MAC_ADDR_L,
+ &mac_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read MAC address low: %d\n",
+ err);
+ return err;
+ }
+
+ mac = ((u64)mac_h << 32) | mac_l;
+
+ u64_to_ether_addr(mac, dev->mac_address);
+
+ return 0;
+}
+
+static int mlx5_nodnic_port_init(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_validate_port_link_type(dev);
+ if (err)
+ goto release;
+
+ err = mlx5_nodnic_read_mac_address(dev);
+ if (err)
+ goto release;
+
+release:
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ return err;
+}
+
+static irqreturn_t mlx5_nodnic_event_irq_thread(int irq, void *data)
+{
+ struct mlx5_nodnic_priv *priv = data;
+
+ if (!netif_device_present(priv->netdev))
+ return IRQ_HANDLED;
+
+ mlx5_nodnic_refresh_carrier(priv);
+
+ return IRQ_HANDLED;
+}
+
+static int mlx5_nodnic_event_irq_setup(struct mlx5_nodnic_core_dev *dev)
+{
+ int irqn, err;
+
+ irqn = pci_irq_vector(dev->pdev, MLX5_NODNIC_EVENT_MSIX);
+ if (irqn < 0) {
+ mlx5_nodnic_core_err(dev,
+ "pci_irq_vector for event failed: %d\n",
+ irqn);
+ return irqn;
+ }
+
+ err = request_threaded_irq(irqn, NULL, mlx5_nodnic_event_irq_thread,
+ IRQF_ONESHOT, "mlx5_nodnic-event",
+ dev->priv);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "request_threaded_irq for event failed\n");
+ return err;
+ }
+
+ dev->event_irqn = irqn;
+
+ return 0;
+}
+
+static void mlx5_nodnic_event_irq_cleanup(struct mlx5_nodnic_core_dev *dev)
+{
+ if (dev->event_irqn < 0)
+ return;
+ free_irq(dev->event_irqn, dev->priv);
+ dev->event_irqn = -1;
+}
+
+static int mlx5_nodnic_probe_one(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct mlx5_nodnic_core_dev *dev;
+ struct net_device *netdev;
+ int err;
+
+ dev = kzalloc_obj(struct mlx5_nodnic_core_dev, GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->device = &pdev->dev;
+ dev->pdev = pdev;
+ dev->event_irqn = -1;
+
+ err = mlx5_nodnic_pci_init(dev, pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_pci_init failed with error code %d\n",
+ err);
+ goto err;
+ }
+
+ err = mlx5_nodnic_dev_init(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_dev_init failed with error code %d\n",
+ err);
+ goto err_pci_cleanup;
+ }
+
+ err = mlx5_nodnic_port_init(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_port_init failed with error code %d\n",
+ err);
+ goto err_dev_cleanup;
+ }
+
+ netdev = mlx5_nodnic_create_netdev(dev);
+ if (!netdev) {
+ err = -ENOMEM;
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_create_netdev failed with error code %d\n",
+ err);
+ goto err_dev_cleanup;
+ }
+
+ mlx5_nodnic_build_netdev(netdev);
+
+ err = mlx5_nodnic_event_irq_setup(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_event_irq_setup failed with error code %d\n",
+ err);
+ goto err_netdev_cleanup;
+ }
+
+ err = register_netdev(netdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "register_netdev failed with error code %d\n",
+ err);
+ goto err_irq_cleanup;
+ }
+ dev->netdev_registered = true;
+
+ dev->state = NODNIC_DEVICE_STATE_UP;
+
+ pci_save_state(pdev);
+
+ return 0;
+
+err_irq_cleanup:
+ mlx5_nodnic_event_irq_cleanup(dev);
+err_netdev_cleanup:
+ mlx5_nodnic_cleanup_netdev(dev);
+err_dev_cleanup:
+ mlx5_nodnic_dev_cleanup(dev);
+err_pci_cleanup:
+ mlx5_nodnic_pci_close(dev);
+err:
+ kfree(dev);
+ return err;
+}
+
+static void mlx5_nodnic_remove_one(struct pci_dev *pdev)
+{
+ struct mlx5_nodnic_core_dev *dev = pci_get_drvdata(pdev);
+
+ if (!dev)
+ return;
+
+ mlx5_nodnic_event_irq_cleanup(dev);
+ mlx5_nodnic_cleanup_netdev(dev);
+ mlx5_nodnic_dev_cleanup(dev);
+ mlx5_nodnic_pci_close(dev);
+ kfree(dev);
+}
+
+static const struct pci_device_id mlx5_nodnic_core_pci_table[] = {
+ { PCI_VDEVICE(MELLANOX, 0xc2d6) }, /* BlueField-4 nodnic rshim PF */
+ { }
+};
+
+MODULE_DEVICE_TABLE(pci, mlx5_nodnic_core_pci_table);
+
+static struct pci_driver mlx5_nodnic_core_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = mlx5_nodnic_core_pci_table,
+ .probe = mlx5_nodnic_probe_one,
+ .remove = mlx5_nodnic_remove_one,
+};
+
+static int __init mlx5_nodnic_init(void)
+{
+ return pci_register_driver(&mlx5_nodnic_core_driver);
+}
+
+static void __exit mlx5_nodnic_cleanup(void)
+{
+ pci_unregister_driver(&mlx5_nodnic_core_driver);
+}
+
+module_init(mlx5_nodnic_init);
+module_exit(mlx5_nodnic_cleanup);
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
new file mode 100644
index 000000000000..2fcae95988d8
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_H__
+#define __MLX5_NODNIC_H__
+
+#include <linux/mlx5/device.h>
+#include <linux/pci.h>
+#include <linux/sched.h>
+
+struct mlx5_nodnic_priv;
+
+#define mlx5_nodnic_core_dbg(__dev, format, ...) \
+ dev_dbg((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+#define mlx5_nodnic_core_err(__dev, format, ...) \
+ dev_err((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+#define mlx5_nodnic_core_warn(__dev, format, ...) \
+ dev_warn((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+enum {
+ MLX5_NODNIC_PORT_STATE_DOWN,
+ MLX5_NODNIC_PORT_STATE_INITIALIZE,
+ MLX5_NODNIC_PORT_STATE_ARMED,
+ MLX5_NODNIC_PORT_STATE_ACTIVE,
+};
+
+enum mlx5_nodnic_pci_status {
+ MLX5_NODNIC_PCI_STATUS_DISABLED,
+ MLX5_NODNIC_PCI_STATUS_ENABLED,
+};
+
+enum mlx5_nodnic_device_state {
+ NODNIC_DEVICE_STATE_UP = 1,
+ NODNIC_DEVICE_STATE_INTERNAL_ERROR,
+};
+
+enum mlx5_nodnic_msix {
+ MLX5_NODNIC_DATA_MSIX,
+ MLX5_NODNIC_EVENT_MSIX,
+ MLX5_NODNIC_NUM_MSIX,
+};
+
+struct mlx5_nodnic_vsc_ctx;
+
+struct mlx5_nodnic_core_dev {
+ struct mlx5_nodnic_priv *priv;
+ struct device *device;
+ struct pci_dev *pdev;
+ bool netdev_registered;
+
+ /* sync pci state */
+ struct mutex pci_status_mutex;
+ enum mlx5_nodnic_pci_status pci_status;
+
+ struct mlx5_init_seg __iomem *iseg;
+ phys_addr_t bar_addr;
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx;
+ void __iomem *uar_base;
+
+ u8 log_working_buffer_size;
+ u8 log_max_ring_size;
+ u8 log_uar_page_size;
+ u32 send_ring0_uar_index;
+
+ u8 mac_address[6];
+ u32 lkey;
+
+ int event_irqn;
+
+ enum mlx5_nodnic_device_state state;
+};
+
+void mlx5_nodnic_start_health_poll(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_stop_health_poll(struct mlx5_nodnic_core_dev *dev);
+
+/*
+ * VSC-gated helpers: the caller must hold the VSC context across these calls.
+ */
+int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_disable_port(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_get_fw_initializing(struct mlx5_nodnic_core_dev *dev,
+ u8 *initializing);
+int mlx5_nodnic_set_nic_interface(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_check_supported(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_write_nic_interface(struct mlx5_nodnic_core_dev *dev,
+ int interface);
+int mlx5_nodnic_read_config(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_validate_port_link_type(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_read_mac_address(struct mlx5_nodnic_core_dev *dev);
+
+/* end of VSC-gated helpers */
+
+#endif /* __MLX5_NODNIC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
new file mode 100644
index 000000000000..31c87b8f40bc
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
@@ -0,0 +1,318 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#ifndef __MLX5_IFC_NODNIC_H__
+#define __MLX5_IFC_NODNIC_H__
+
+#include <linux/mlx5/device.h>
+#include <linux/types.h>
+
+enum {
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_FULL_DRIVER,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_SW_RESET = 0x7,
+};
+
+enum {
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_INTERNAL_ERR = 0x1,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_DEAD_IRISC = 0x7,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HW_FATAL_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_CRC_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_FETCH_PCI_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PAGE_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ASYNCHRONOUS_EQ_BUF_OVERRUN,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_IN_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_INV,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FFSER_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HIGH_TEMP_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PCI_POISONED_ERR = 0x12,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_TRUST_LOCKDOWN_ERR,
+};
+
+/* ISEG */
+#define MLX5_NODNIC_ISEG_NIC_INTERFACE MLX5_BYTE_OFF(nodnic_iseg, cmdq_addr_l)
+#define MLX5_NODNIC_ISEG_INITIALIZING \
+ MLX5_BYTE_OFF(nodnic_iseg, initializing_word)
+#define MLX5_NODNIC_ISEG_NO_DRAM_NIC_OFFSET \
+ MLX5_BYTE_OFF(nodnic_iseg, no_dram_nic_offset)
+#define MLX5_NODNIC_ISEG_CMD_ADDR_L \
+ MLX5_BYTE_OFF(nodnic_iseg_cmdq_addr_l, cmdq_addr_l)
+#define MLX5_NODNIC_ISEG_CLEAR_INT MLX5_BYTE_OFF(nodnic_iseg, clear_int)
+
+/* General configuration */
+#define MLX5_NODNIC_GENERAL_CONFIG_CAPS MLX5_BYTE_OFF(nodnic_gen_cfg, caps)
+#define MLX5_NODNIC_GENERAL_CONFIG_LOG_MAX_RING_SIZE \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, log_max_ring_size)
+#define MLX5_NODNIC_GENERAL_CONFIG_LKEY MLX5_BYTE_OFF(nodnic_gen_cfg, lkey)
+#define MLX5_NODNIC_GENERAL_CONFIG_CQE_FORMAT \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, cqe_format)
+#define MLX5_NODNIC_GENERAL_CONFIG_FEATURE_CAPS \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, feature_caps)
+#define MLX5_NODNIC_GENERAL_CONFIG_LOG_UAR_PAGE_SIZE \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, log_uar_page_size)
+
+/* Port */
+#define MLX5_NODNIC_PORT_OFFSET \
+ MLX5_BYTE_OFF(nodnic_config_seg, port)
+#define MLX5_NODNIC_PORT_FIELD_OFFSET(field) \
+ (MLX5_NODNIC_PORT_OFFSET + MLX5_BYTE_OFF(nodnic_port_settings, field))
+
+#define MLX5_NODNIC_PORT_EVENT MLX5_NODNIC_PORT_FIELD_OFFSET(event)
+#define MLX5_NODNIC_RECEIVE_RING_OFFSET \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(receive_ring0)
+#define MLX5_NODNIC_SEND_RING_OFFSET MLX5_NODNIC_PORT_FIELD_OFFSET(send_ring0)
+#define MLX5_NODNIC_PORT_NETWORK MLX5_NODNIC_PORT_FIELD_OFFSET(network)
+#define MLX5_NODNIC_PORT_MAC_ADDR_H MLX5_NODNIC_PORT_FIELD_OFFSET(mac_h)
+#define MLX5_NODNIC_PORT_MAC_ADDR_L MLX5_NODNIC_PORT_FIELD_OFFSET(mac_l)
+#define MLX5_NODNIC_PORT_CQ_ADDR_H MLX5_NODNIC_PORT_FIELD_OFFSET(cq_addr_h)
+#define MLX5_NODNIC_PORT_CQ_ADDR_L MLX5_NODNIC_PORT_FIELD_OFFSET(cq_addr_l)
+#define MLX5_NODNIC_PORT_WORK_BUF_ADDR_H \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(working_buffer_addr_h)
+#define MLX5_NODNIC_PORT_WORK_BUF_ADDR_L \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(working_buffer_addr_l)
+#define MLX5_NODNIC_PORT_ARM_CQ MLX5_NODNIC_PORT_FIELD_OFFSET(arm_cq)
+#define MLX5_NODNIC_PORT_SEND_RING0_UAR_INDEX \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(send_ring0_uar_index)
+#define MLX5_NODNIC_CQ_DBR_ADDR_H \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cq_dbr_addr_h)
+#define MLX5_NODNIC_CQ_DBR_ADDR_L \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cq_dbr_addr_l)
+#define MLX5_NODNIC_CQ_N \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cqn)
+
+/* Port ring */
+#define MLX5_NODNIC_RING_ADDR_H MLX5_BYTE_OFF(nodnic_ring_config, q_addr_h)
+#define MLX5_NODNIC_RING_ADDR_L MLX5_BYTE_OFF(nodnic_ring_config, q_addr_l)
+#define MLX5_NODNIC_RING_QUEUE_NUMBER \
+ MLX5_BYTE_OFF(nodnic_ring_config, q_number)
+#define MLX5_NODNIC_RING_DBR_ADDR_H \
+ MLX5_BYTE_OFF(nodnic_ring_config, dbr_addr_h)
+#define MLX5_NODNIC_RING_DBR_ADDR_L \
+ MLX5_BYTE_OFF(nodnic_ring_config, dbr_addr_l)
+
+/* mlx5_nodnic configurations are read and written from VSC
+ * in a dword granularity
+ */
+
+struct mlx5_ifc_nodnic_iseg_cmdq_addr_l_bits {
+ u8 cmdq_addr_l[0x14];
+ u8 reserved_at_14[0x1];
+ u8 nic_interface[0x3];
+ u8 log_cmdq_size[0x4];
+ u8 log_cmdq_stride[0x4];
+};
+
+struct mlx5_ifc_nodnic_iseg_initializing_word_bits {
+ u8 initializing[0x1];
+ u8 nic_interface_supported[0x7];
+ u8 embedded_cpu[0x1];
+ u8 driver_reset_ack[0x1];
+ u8 recovery[0x1];
+ u8 pre_boot_bios_hold[0x1];
+ u8 initializing_state[0x4];
+ u8 reset_state_machine[0x4];
+ u8 reserved_at_14[0x1];
+ u8 pre_boot_action[0x3];
+ u8 reserved_at_17[0x8];
+};
+
+struct mlx5_ifc_nodnic_iseg_health_buffer_bits {
+ u8 reserved_at_0[0x100];
+ u8 assert_existptr[0x20];
+ u8 assert_callra[0x20];
+ u8 reserved_at_28[0x20];
+ u8 time[0x20];
+ u8 fw_version[0x20];
+ u8 hw_id[0x20];
+ u8 rfr[0x1];
+ u8 crr[0x1];
+ u8 reserved_at_e2[0x2];
+ u8 valid[0x1];
+ u8 severity[0x3];
+ u8 reserved_at_e8[0x18];
+ u8 irisc_index[0x8];
+ u8 synd[0x8];
+ u8 ext_synd[0x10];
+};
+
+struct mlx5_ifc_nodnic_clear_int_bits {
+ u8 reserved_at_0[0x1f];
+ u8 clear_int[0x1];
+};
+
+struct mlx5_ifc_nodnic_iseg_bits {
+ u8 fw_rev[0x20];
+ u8 cmdif_rev[0x20];
+ u8 rsvd0[0x40];
+ u8 cmdq_addr_h[0x20];
+ struct mlx5_ifc_nodnic_iseg_cmdq_addr_l_bits cmdq_addr_l;
+ u8 command_doorbell_vector[0x20];
+ u8 rsvd1[0xee0];
+ u8 traffic_state_rsvd[0x1e];
+ u8 traffic_state[0x2];
+ struct mlx5_ifc_nodnic_iseg_initializing_word_bits initializing_word;
+ struct mlx5_ifc_nodnic_iseg_health_buffer_bits health_buffer;
+ u8 no_dram_nic_offset[0x20];
+ u8 rsvd2[0x6e40];
+ struct mlx5_ifc_nodnic_clear_int_bits clear_int;
+};
+
+struct mlx5_ifc_nodnic_q_addr_l_bits {
+ u8 q_addr_l[0x14];
+ u8 reserved_at_14[0x6];
+ u8 log_size[0x6];
+};
+
+struct mlx5_ifc_nodnic_ring_db_reg_bits {
+ u8 reserved_at_0[0x8];
+ u8 ring_pi[0x10];
+ u8 reserved_at_18[0x8];
+};
+
+struct mlx5_ifc_nodnic_ring_q_number_bits {
+ u8 reserved_at_0[0x8];
+ u8 queue_number[0x18];
+};
+
+struct mlx5_ifc_nodnic_ring_dbr_addr_l_bits {
+ u8 db_record_addr_l[0x1e];
+ u8 reserved_at_1e[0x2];
+};
+
+struct mlx5_ifc_nodnic_ring_config_bits {
+ u8 q_addr_h[0x20];
+ struct mlx5_ifc_nodnic_q_addr_l_bits q_addr_l;
+ struct mlx5_ifc_nodnic_ring_db_reg_bits db_reg;
+ struct mlx5_ifc_nodnic_ring_q_number_bits q_number;
+ u8 q_key[0x20];
+ u8 pkey[0x20];
+ u8 dbr_addr_h[0x20];
+ u8 dbr_addr_l[0x20];
+};
+
+struct mlx5_ifc_nodnic_port_event_bits {
+ u8 driver_reset_needed[0x1];
+ u8 port_management_change[0x1];
+ u8 reserved_at_2[0x19];
+ u8 link_type[0x1];
+ u8 port_state[0x4];
+};
+
+struct mlx5_ifc_nodnic_port_network_bits {
+ u8 network_en[0x1];
+ u8 dma_en[0x1];
+ u8 promisc_en[0x1];
+ u8 promisc_multicast_en[0x1];
+ u8 vlan_stripping_en[0x1];
+ u8 protocol_filter_ip_ver[0x1];
+ u8 data_msix_en[0x1];
+ u8 event_msix_en[0x1];
+ u8 reserved_at_8[0x8];
+ u8 protocol_filter_en[0x8];
+ u8 reserved_at_18[0x3];
+ u8 receive_filter_en[0x5];
+};
+
+struct mlx5_ifc_nodnic_port_mac_h_bits {
+ u8 reserved_at_0[0x10];
+ u8 mac_h[0x10];
+};
+
+struct mlx5_ifc_nodnic_port_arm_cq_bits {
+ u8 cq_ci[0x18];
+ u8 reserved_at_18[0x8];
+};
+
+struct mlx5_ifc_nodnic_cqn_bits {
+ u8 reserved[0x8];
+ u8 cq_n[0x18];
+};
+
+struct mlx5_ifc_nodnic_port_settings_bits {
+ struct mlx5_ifc_nodnic_port_event_bits event;
+ struct mlx5_ifc_nodnic_port_network_bits network;
+ struct mlx5_ifc_nodnic_port_mac_h_bits mac_h;
+ u8 mac_l[0x20];
+ u8 mac_filters[0x200];
+ u8 gid[0x80];
+ u8 reserved_at_300[0x20];
+ u8 sm_sl_lid[0x20];
+ u8 cq_addr_h[0x20];
+ struct mlx5_ifc_nodnic_q_addr_l_bits cq_addr_l;
+ u8 working_buffer_addr_h[0x20];
+ u8 working_buffer_addr_l[0x20];
+ struct mlx5_ifc_nodnic_port_arm_cq_bits arm_cq;
+ u8 pkey[0x20];
+ struct mlx5_ifc_nodnic_ring_config_bits send_ring0;
+ u8 rsvd_send_ring1[0x100];
+ struct mlx5_ifc_nodnic_ring_config_bits receive_ring0;
+ u8 rsvd_receive_ring1[0x100];
+ u8 protocol_filter_tftp_port[0x20];
+ u8 reserved_at_104[0x20];
+ u8 send_ring0_uar_index[0x20];
+ u8 send_ring1_uar_index[0x20];
+ u8 cq_dbr_addr_h[0x20];
+ u8 cq_dbr_addr_l[0x20];
+ struct mlx5_ifc_nodnic_cqn_bits cqn;
+ u8 padding[0x320];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_caps_bits {
+ u8 revision[0x8];
+ u8 hardware_format[0x8];
+ u8 support_receive_filter[0x1];
+ u8 support_promisc_filter[0x1];
+ u8 support_promisc_multicast[0x1];
+ u8 support_protocol_filter[0x1];
+ u8 reserved_at_13[0x1];
+ u8 log_working_buffer_size[0x3];
+ u8 log_pkey_table_size[0x4]; /* IB */
+ u8 reserved_at_1a[0x3];
+ u8 num_ports[0x1];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_log_max_ring_size_bits {
+ u8 reserved_at_0[0x2];
+ u8 log_max_ring_size[0x6];
+ u8 reserved_at_8[0x18];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_cqe_format_bits {
+ u8 cqe_format[0x4];
+ u8 reserved_at_4[0x1c];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_feature_caps_bits {
+ u8 support_db_record_rx[0x1];
+ u8 reserved_at_1[0x1];
+ u8 support_uar_tx_doorbell[0x1];
+ u8 support_bar_dma_en[0x1];
+ u8 support_bar_cq_ctrl[0x1];
+ u8 vlan_stripping_supported[0x1];
+ u8 data_msix_support[0x1];
+ u8 event_msix_support[0x1];
+ u8 reserved_at_8[0x18];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_log_uar_page_size_bits {
+ u8 reserved_at_0[0x18];
+ u8 log_uar_page_size[0x8];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_bits {
+ struct mlx5_ifc_nodnic_gen_cfg_caps_bits caps;
+ struct mlx5_ifc_nodnic_gen_cfg_log_max_ring_size_bits log_max_ring_size;
+ u8 lkey[0x20];
+ struct mlx5_ifc_nodnic_gen_cfg_cqe_format_bits cqe_format;
+ u8 reserved_at_80[3][0x20];
+ struct mlx5_ifc_nodnic_gen_cfg_feature_caps_bits feature_caps;
+ struct mlx5_ifc_nodnic_gen_cfg_log_uar_page_size_bits log_uar_page_size;
+};
+
+struct mlx5_ifc_nodnic_config_seg_bits {
+ struct mlx5_ifc_nodnic_gen_cfg_bits general;
+ u8 reserved_at_24[55][0x20];
+ struct mlx5_ifc_nodnic_port_settings_bits port;
+};
+
+#endif /* __MLX5_IFC_NODNIC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
new file mode 100644
index 000000000000..6ddbf045a8d4
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
@@ -0,0 +1,343 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/bitfield.h>
+#include <linux/delay.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include "nodnic_ifc.h"
+#include "nodnic_pci_vsc.h"
+
+struct mlx5_nodnic_vsc_ctx {
+ struct pci_dev *pdev;
+ u32 addr; /* VSC init segment address */
+ u32 offset; /* NODNIC config offset */
+};
+
+#define VSC_MAX_RETRIES 2048
+
+enum {
+ MLX5_NODNIC_VSC_CTRL_OFFSET = 0x4,
+ MLX5_NODNIC_VSC_COUNTER_OFFSET = 0x8,
+ MLX5_NODNIC_VSC_SEM_OFFSET = 0xc,
+ MLX5_NODNIC_VSC_ADDR_OFFSET = 0x10,
+ MLX5_NODNIC_VSC_DATA_OFFSET = 0x14,
+};
+
+/* MLX5_NODNIC_VSC_CTRL_OFFSET fields */
+#define MLX5_NODNIC_VSC_CTRL_SPACE GENMASK(15, 0)
+#define MLX5_NODNIC_VSC_CTRL_STATUS GENMASK(31, 29)
+
+/* MLX5_NODNIC_VSC_ADDR_OFFSET fields */
+#define MLX5_NODNIC_VSC_ADDR_SYND BIT(30)
+#define MLX5_NODNIC_VSC_ADDR_FLAG BIT(31)
+
+static int mlx5_nodnic_vsc_pci_read(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int offset, u32 *val)
+{
+ return pci_read_config_dword(ctx->pdev, ctx->addr + offset, val);
+}
+
+static int mlx5_nodnic_vsc_pci_write(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int offset, u32 val)
+{
+ return pci_write_config_dword(ctx->pdev, ctx->addr + offset, val);
+}
+
+struct mlx5_nodnic_vsc_ctx *mlx5_nodnic_vsc_init(struct pci_dev *pdev)
+{
+ struct mlx5_nodnic_vsc_ctx *ctx;
+
+ ctx = kzalloc_obj(struct mlx5_nodnic_vsc_ctx, GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ ctx->pdev = pdev;
+ ctx->addr = pci_find_capability(ctx->pdev, PCI_CAP_ID_VNDR);
+ if (!ctx->addr) {
+ dev_err(&ctx->pdev->dev,
+ "failed to get valid vendor specific ID\n");
+ kfree(ctx);
+ return NULL;
+ }
+
+ return ctx;
+}
+
+void mlx5_nodnic_vsc_cleanup(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ kfree(ctx);
+}
+
+int mlx5_nodnic_vsc_read_offset(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ u32 offset;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_cpu32(ctx,
+ MLX5_NODNIC_ISEG_NO_DRAM_NIC_OFFSET,
+ &offset);
+ if (err) {
+ dev_warn(&ctx->pdev->dev,
+ "failed to read no_dram_nic_offset err %d\n", err);
+ return err;
+ }
+
+ ctx->offset = offset;
+ return 0;
+}
+
+static int mlx5_nodnic_vsc_gw_lock(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ struct pci_dev *pdev = ctx->pdev;
+ u32 lock_val, counter = 0;
+ int err, retries = 0;
+
+ pci_cfg_access_lock(pdev);
+
+ do {
+ if (retries > VSC_MAX_RETRIES) {
+ err = -EBUSY;
+ goto pci_unlock;
+ }
+ if (pci_channel_offline(pdev)) {
+ err = -EACCES;
+ goto pci_unlock;
+ }
+
+ /* Check if semaphore is already locked */
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_SEM_OFFSET,
+ &lock_val);
+ if (err)
+ goto pci_unlock;
+
+ if (lock_val) {
+ retries++;
+ usleep_range(1000, 2000);
+ continue;
+ }
+
+ /* Read and write counter value, if written value is
+ * the same, semaphore was acquired successfully.
+ */
+ err = mlx5_nodnic_vsc_pci_read(ctx,
+ MLX5_NODNIC_VSC_COUNTER_OFFSET,
+ &counter);
+ if (err)
+ goto pci_unlock;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_SEM_OFFSET,
+ counter);
+ if (err)
+ goto pci_unlock;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_SEM_OFFSET,
+ &lock_val);
+ if (err)
+ goto pci_unlock;
+
+ retries++;
+ } while (counter != lock_val);
+
+ return 0;
+
+pci_unlock:
+ pci_cfg_access_unlock(pdev);
+ return err;
+}
+
+static int mlx5_nodnic_vsc_gw_unlock(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_SEM_OFFSET,
+ MLX5_NODNIC_VSC_UNLOCK);
+ pci_cfg_access_unlock(ctx->pdev);
+
+ return err;
+}
+
+static int
+mlx5_nodnic_vsc_gw_set_space(struct mlx5_nodnic_vsc_ctx *ctx, u16 space)
+{
+ u32 val;
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, &val);
+ if (err)
+ return err;
+
+ val = u32_replace_bits(val, space, MLX5_NODNIC_VSC_CTRL_SPACE);
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, val);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, &val);
+ if (err)
+ return err;
+
+ if (!FIELD_GET(MLX5_NODNIC_VSC_CTRL_STATUS, val))
+ return -EINVAL;
+
+ return 0;
+}
+
+int mlx5_nodnic_vsc_context_acquire(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_gw_lock(ctx);
+ if (err) {
+ dev_warn(&ctx->pdev->dev, "failed to lock vsc, err %d\n", err);
+ return err;
+ }
+
+ err = mlx5_nodnic_vsc_gw_set_space(ctx, MLX5_NODNIC_VSC_SPACE_NODNIC);
+ if (err) {
+ dev_warn(&ctx->pdev->dev,
+ "failed to set vsc space, err %d\n", err);
+ mlx5_nodnic_vsc_gw_unlock(ctx);
+ return err;
+ }
+
+ return err;
+}
+
+void mlx5_nodnic_vsc_context_release(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ mlx5_nodnic_vsc_gw_unlock(ctx);
+}
+
+static int mlx5_nodnic_vsc_wait_on_flag(struct mlx5_nodnic_vsc_ctx *ctx,
+ u8 expected_val)
+{
+ int err, retries = 0;
+ u32 flag;
+
+ do {
+ if (retries > VSC_MAX_RETRIES)
+ return -EBUSY;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ &flag);
+ if (err)
+ return err;
+ flag = FIELD_GET(MLX5_NODNIC_VSC_ADDR_FLAG, flag);
+ retries++;
+
+ if ((retries & 0xf) == 0)
+ usleep_range(1000, 2000);
+
+ } while (flag != expected_val);
+
+ return 0;
+}
+
+static int mlx5_nodnic_vsc_read(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ /* Write the address with the flag and syndrome bits cleared */
+ address &= ~(MLX5_NODNIC_VSC_ADDR_FLAG | MLX5_NODNIC_VSC_ADDR_SYND);
+
+ err = mlx5_nodnic_vsc_pci_write(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ address);
+ if (err)
+ goto out;
+
+ err = mlx5_nodnic_vsc_wait_on_flag(ctx, 1);
+ if (err)
+ goto out;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_DATA_OFFSET, data);
+out:
+ return err;
+}
+
+static int mlx5_nodnic_vsc_write(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_DATA_OFFSET, data);
+ if (err)
+ goto out;
+
+ /* Write the address with the flag bit set and the syndrome bit
+ * cleared.
+ */
+ address &= ~(MLX5_NODNIC_VSC_ADDR_FLAG | MLX5_NODNIC_VSC_ADDR_SYND);
+ address |= MLX5_NODNIC_VSC_ADDR_FLAG;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ address);
+ if (err)
+ return err;
+
+ /* Wait for the hardware to clear the flag to 0x0,
+ * indicating the write operation has completed.
+ */
+ err = mlx5_nodnic_vsc_wait_on_flag(ctx, 0);
+
+out:
+ return err;
+}
+
+/* The PCI config space accessors already convert between the little-endian
+ * bus and CPU byte order. The *_cpu32 accessors hold the dword value in CPU
+ * byte order. The *_be32 accessors hold it in the big-endian layout used by
+ * MLX5_GET() and MLX5_SET().
+ */
+int mlx5_nodnic_vsc_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 *data)
+{
+ u32 val;
+ int err;
+
+ err = mlx5_nodnic_vsc_read(ctx, address, &val);
+ if (!err)
+ *data = cpu_to_be32(val);
+
+ return err;
+}
+
+int mlx5_nodnic_vsc_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ return mlx5_nodnic_vsc_read(ctx, address, data);
+}
+
+int mlx5_nodnic_vsc_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 data)
+{
+ return mlx5_nodnic_vsc_write(ctx, address, be32_to_cpu(data));
+}
+
+int mlx5_nodnic_vsc_cfg_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 *data)
+{
+ return mlx5_nodnic_vsc_read_be32(ctx, ctx->offset + address, data);
+}
+
+int mlx5_nodnic_vsc_cfg_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 data)
+{
+ return mlx5_nodnic_vsc_write_be32(ctx, ctx->offset + address, data);
+}
+
+int mlx5_nodnic_vsc_cfg_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ return mlx5_nodnic_vsc_read(ctx, ctx->offset + address, data);
+}
+
+int mlx5_nodnic_vsc_cfg_write_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ return mlx5_nodnic_vsc_write(ctx, ctx->offset + address, data);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
new file mode 100644
index 000000000000..b56d7abc4a90
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_PCI_VSC_H__
+#define __MLX5_NODNIC_PCI_VSC_H__
+
+#include <linux/pci.h>
+#include <linux/types.h>
+
+struct mlx5_nodnic_vsc_ctx;
+
+enum mlx5_nodnic_vsc_state {
+ MLX5_NODNIC_VSC_UNLOCK,
+ MLX5_NODNIC_VSC_LOCK,
+};
+
+enum {
+ MLX5_NODNIC_VSC_SPACE_NODNIC = 0x4,
+};
+
+struct mlx5_nodnic_vsc_ctx *mlx5_nodnic_vsc_init(struct pci_dev *pdev);
+void mlx5_nodnic_vsc_cleanup(struct mlx5_nodnic_vsc_ctx *ctx);
+int mlx5_nodnic_vsc_read_offset(struct mlx5_nodnic_vsc_ctx *ctx);
+int mlx5_nodnic_vsc_context_acquire(struct mlx5_nodnic_vsc_ctx *ctx);
+void mlx5_nodnic_vsc_context_release(struct mlx5_nodnic_vsc_ctx *ctx);
+
+/* base address - init segment copy in VSC */
+int mlx5_nodnic_vsc_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 *data);
+int mlx5_nodnic_vsc_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 data);
+int mlx5_nodnic_vsc_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+
+/* base address - nodnic configuration registers in VSC */
+int mlx5_nodnic_vsc_cfg_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 *data);
+int mlx5_nodnic_vsc_cfg_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, __be32 data);
+int mlx5_nodnic_vsc_cfg_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+int mlx5_nodnic_vsc_cfg_write_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data);
+
+#endif /* __MLX5_NODNIC_PCI_VSC_H__ */
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
@ 2026-10-08 17:18 ` Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
From: Shahar Shitrit <shshitrit@nvidia.com>
Implement the TX and RX data paths for the mlx5_nodnic driver.
Add TX handling to build and post WQEs, DMA-map packets, ring the
BlueFlame doorbell, and process send completions. Handle SQ exhaustion
by stopping and waking the TX queue as needed.
Add page pool backed RX handling to allocate and post receive buffers,
process receive completions, build SKBs, and deliver packets through
NAPI/GRO.
Implement NAPI polling and CQE dispatch for TX and RX completions,
including RX WQE reposting and CQ re-arming.
Wire up ndo_start_xmit and ndo_features_check, and enable TX/RX
checksum offload support.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/nodnic/Kconfig | 1 +
.../ethernet/mellanox/mlx5/nodnic/Makefile | 2 +-
.../net/ethernet/mellanox/mlx5/nodnic/en.c | 263 ++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 35 +++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 220 +++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 169 +++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.h | 25 ++
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 226 +++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/tx.h | 61 ++++
9 files changed, 1001 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
index 461750a9c97c..f53f68a98228 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
@@ -4,6 +4,7 @@ config MLX5_NODNIC
tristate "Mellanox NODNIC core driver"
depends on PCI
depends on NETDEVICES && ETHERNET
+ select PAGE_POOL
help
This driver provides core support for Mellanox NODNIC (NO DRAM NIC)
devices. NODNIC is a lightweight network
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
index 4955f0427c79..25865d7244d8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -5,4 +5,4 @@
obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
-mlx5_nodnic-y := main.o en_cfg.o nodnic_pci_vsc.o
+mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
new file mode 100644
index 000000000000..3572aff50194
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
@@ -0,0 +1,263 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/interrupt.h>
+#include <linux/mlx5/cq.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "rx.h"
+#include "tx.h"
+
+#define MLX5_NODNIC_TX_CQ_POLL_BUDGET 128
+
+enum mlx5_nodnic_cqe_result {
+ MLX5_NODNIC_CQE_TX,
+ MLX5_NODNIC_CQE_RX,
+ MLX5_NODNIC_CQE_IGNORED,
+};
+
+static const char *mlx5_nodnic_cqe_err_syndrome_str(u8 syndrome)
+{
+ switch (syndrome) {
+ case MLX5_CQE_SYNDROME_LOCAL_LENGTH_ERR:
+ return "LOCAL_LENGTH_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR:
+ return "LOCAL_QP_OP_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_PROT_ERR:
+ return "LOCAL_PROT_ERR";
+ case MLX5_CQE_SYNDROME_WR_FLUSH_ERR:
+ return "WR_FLUSH_ERR";
+ case MLX5_CQE_SYNDROME_MW_BIND_ERR:
+ return "MW_BIND_ERR";
+ case MLX5_CQE_SYNDROME_BAD_RESP_ERR:
+ return "BAD_RESP_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_ACCESS_ERR:
+ return "LOCAL_ACCESS_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR:
+ return "REMOTE_INVAL_REQ_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_ACCESS_ERR:
+ return "REMOTE_ACCESS_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_OP_ERR:
+ return "REMOTE_OP_ERR";
+ case MLX5_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR:
+ return "TRANSPORT_RETRY_EXC_ERR";
+ case MLX5_CQE_SYNDROME_RNR_RETRY_EXC_ERR:
+ return "RNR_RETRY_EXC_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_ABORTED_ERR:
+ return "REMOTE_ABORTED_ERR";
+ default:
+ return "UNKNOWN";
+ }
+}
+
+static void mlx5_nodnic_handle_err_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe64,
+ int napi_budget)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct mlx5_err_cqe *err_cqe = (struct mlx5_err_cqe *)cqe64;
+ u8 vendor_syndrome, syndrome;
+ const char *queue_type;
+ u16 wqe_counter;
+ u8 opcode;
+
+ opcode = err_cqe->op_own >> 4;
+ syndrome = err_cqe->syndrome;
+ vendor_syndrome = err_cqe->vendor_err_synd;
+ wqe_counter = be16_to_cpu(err_cqe->wqe_counter);
+
+ queue_type = (opcode == OPCODE_CQ_SEND_ERR) ? "SQ" : "RQ";
+
+ netdev_err(priv->netdev,
+ "%s CQE error: syndrome=0x%02x (%s), vendor_syndrome=0x%02x, wqe_counter=%u, cq_ci=%d\n",
+ queue_type,
+ syndrome, mlx5_nodnic_cqe_err_syndrome_str(syndrome),
+ vendor_syndrome, wqe_counter, cq->cc);
+
+ print_hex_dump(KERN_ERR, "mlx5_nodnic err CQE: ", DUMP_PREFIX_OFFSET,
+ 16, 1, err_cqe, sizeof(*err_cqe), false);
+
+ if (opcode == OPCODE_CQ_SEND_ERR) {
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ u16 sq_pi = mlx5_nodnic_sq_get_pi(sq, wqe_counter);
+
+ mlx5_nodnic_complete_send_wqe(sq, &sq->wqe_info[sq_pi],
+ napi_budget);
+ } else {
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u16 ci = wqe_counter & (rq->ring.num_entries - 1);
+
+ mlx5_nodnic_release_rx_wqe(rq, ci);
+ }
+}
+
+static u32 mlx5_nodnic_cq_get_ci(struct mlx5_nodnic_cq *cq)
+{
+ return cq->cc & (cq->num_entries - 1);
+}
+
+static u8 mlx5_nodnic_cq_get_sw_ownership_val(struct mlx5_nodnic_cq *cq)
+{
+ return (cq->cc >> cq->log_num_entries) & 1;
+}
+
+static struct mlx5_cqe64 *mlx5_nodnic_cq_get_cqe(struct mlx5_nodnic_cq *cq)
+{
+ u8 cqe_ownership_bit, sw_ownership_val;
+ struct mlx5_cqe64 *cqe;
+ u32 ci;
+
+ ci = mlx5_nodnic_cq_get_ci(cq);
+ cqe = (struct mlx5_cqe64 *)cq->ring.wq + ci;
+
+ cqe_ownership_bit = cqe->op_own & MLX5_NODNIC_CQE_OWNER_MASK;
+ sw_ownership_val = mlx5_nodnic_cq_get_sw_ownership_val(cq);
+ if (cqe_ownership_bit != sw_ownership_val)
+ return NULL;
+
+ /* ensure cqe content is read after cqe ownership bit */
+ dma_rmb();
+
+ return cqe;
+}
+
+static u8 mlx5_nodnic_cq_get_opcode(struct mlx5_cqe64 *cqe)
+{
+ return cqe->op_own >> 4;
+}
+
+static enum mlx5_nodnic_cqe_result
+mlx5_nodnic_handle_cqe(struct mlx5_nodnic_cq *cq, struct mlx5_cqe64 *cqe,
+ int budget)
+{
+ struct net_device *netdev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->netdev;
+ u8 opcode = mlx5_nodnic_cq_get_opcode(cqe);
+ enum mlx5_nodnic_cqe_result result;
+
+ switch (opcode) {
+ case OPCODE_CQ_SEND:
+ mlx5_nodnic_handle_send_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_TX;
+ break;
+
+ case OPCODE_CQ_RECV:
+ mlx5_nodnic_handle_recv_cqe(cq, cqe);
+ result = MLX5_NODNIC_CQE_RX;
+ break;
+
+ case OPCODE_CQ_SEND_ERR:
+ mlx5_nodnic_handle_err_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_TX;
+ break;
+
+ case OPCODE_CQ_RECV_ERR:
+ mlx5_nodnic_handle_err_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_RX;
+ break;
+
+ case OPCODE_CQ_INVALID:
+ /* Invalid/stale CQE - silently consume it.
+ * This can occur after device restart when stale CQEs
+ * from the previous session are still in the hardware CQ.
+ */
+ result = MLX5_NODNIC_CQE_IGNORED;
+ break;
+
+ default:
+ netdev_err(netdev,
+ "unexpected CQE opcode 0x%02x at cq_ci=%d\n",
+ opcode, cq->cc);
+ result = MLX5_NODNIC_CQE_IGNORED;
+ break;
+ }
+
+ cq->cc++;
+ return result;
+}
+
+void mlx5_nodnic_arm_cq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_core_dev *dev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+ __be32 doorbell[2];
+ u32 sn, ci, val;
+
+ sn = cq->arm_sn & 3;
+ ci = cq->cc & 0xffffff;
+ val = sn << 28 | MLX5_CQ_DB_REQ_NOT | ci;
+
+ cq->dbr.db[0] = cpu_to_be32(ci);
+ /* arm_db is the second dword of the CQ doorbell record */
+ cq->dbr.db[1] = cpu_to_be32(val);
+
+ /* Make sure that the doorbell record in host memory is
+ * written before ringing the doorbell via PCI MMIO.
+ */
+ wmb();
+
+ doorbell[0] = cpu_to_be32(val);
+ doorbell[1] = cpu_to_be32(cq->cqn);
+
+ mlx5_write64(doorbell, dev->uar_base + MLX5_CQ_DOORBELL);
+}
+
+int mlx5_nodnic_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct mlx5_nodnic_cq *cq = container_of(napi,
+ struct mlx5_nodnic_cq, napi);
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ int tx_work_done = 0, rx_work_done = 0;
+ enum mlx5_nodnic_cqe_result result;
+ struct mlx5_cqe64 *cqe;
+
+ while (true) {
+ cqe = mlx5_nodnic_cq_get_cqe(cq);
+ if (!cqe)
+ break;
+
+ result = mlx5_nodnic_handle_cqe(cq, cqe, budget);
+ switch (result) {
+ case MLX5_NODNIC_CQE_TX:
+ if (++tx_work_done >= MLX5_NODNIC_TX_CQ_POLL_BUDGET)
+ goto poll_done;
+ break;
+ case MLX5_NODNIC_CQE_RX:
+ if (++rx_work_done >= budget)
+ goto poll_done;
+ break;
+ case MLX5_NODNIC_CQE_IGNORED:
+ break;
+ }
+ }
+
+poll_done:
+ if (!mlx5_nodnic_rq_post_wqes(priv))
+ return budget;
+
+ if (rx_work_done >= budget)
+ return budget;
+
+ if (napi_complete_done(napi, rx_work_done))
+ mlx5_nodnic_arm_cq(cq);
+
+ return rx_work_done;
+}
+
+netdev_features_t
+mlx5_nodnic_check_features(struct sk_buff *skb, struct net_device *netdev,
+ netdev_features_t features)
+{
+ features = vlan_features_check(skb, features);
+
+ /* Validate if the tunneled packet is being offloaded by HW */
+ if (skb->encapsulation && (features & NETIF_F_CSUM_MASK))
+ return features & ~NETIF_F_CSUM_MASK;
+
+ return features;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index 90dc57695e1b..19858e9da064 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -4,10 +4,12 @@
#ifndef __MLX5_NODNIC_EN_H__
#define __MLX5_NODNIC_EN_H__
+#include <net/page_pool/helpers.h>
#include <linux/if_vlan.h>
#include <linux/mlx5/device.h>
#include <linux/mutex.h>
#include <linux/netdevice.h>
+#include <linux/skbuff.h>
#include <linux/timer.h>
#include <linux/types.h>
@@ -16,16 +18,32 @@
#define MLX5_NODNIC_SQ_WQEBB_B 64
#define MLX5_NODNIC_RQ_STRIDE_BYTES 16
+#define MLX5_NODNIC_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN)
+#define MLX5_NODNIC_HW_MTU (ETH_DATA_LEN + MLX5_NODNIC_ETH_HARD_MTU)
+#define MLX5_NODNIC_SKB_HEADROOM_BYTES NET_SKB_PAD
+
enum {
MLX5_NODNIC_STATE_OPENED,
};
+enum {
+ MLX5_NODNIC_CQE_OWNER_MASK = 1,
+};
+
enum {
MLX5_NODNIC_RCV_DBR,
MLX5_NODNIC_SND_DBR,
MLX5_NODNIC_DBR_WORDS,
};
+enum mlx5_nodnic_cqe_opcode {
+ OPCODE_CQ_SEND,
+ OPCODE_CQ_RECV = 0x2,
+ OPCODE_CQ_SEND_ERR = 0xD,
+ OPCODE_CQ_RECV_ERR,
+ OPCODE_CQ_INVALID,
+};
+
struct mlx5_nodnic_ring {
void *wq;
dma_addr_t dma_handle;
@@ -35,17 +53,24 @@ struct mlx5_nodnic_ring {
u32 lkey;
};
+struct mlx5_nodnic_tx_wqe_info;
+
struct mlx5_nodnic_sq {
struct device *device;
struct net_device *netdev;
+ struct netdev_queue *txq;
struct mlx5_nodnic_ring ring;
+ struct mlx5_nodnic_tx_wqe_info *wqe_info;
+
u16 pc;
u16 cc;
};
struct mlx5_nodnic_rq {
+ struct page_pool *page_pool;
struct mlx5_nodnic_ring ring;
+ struct page **pages;
u16 pc;
u16 cc;
};
@@ -62,9 +87,13 @@ struct mlx5_nodnic_cq {
int num_entries;
int log_num_entries;
+ int irqn;
struct mlx5_nodnic_dbr dbr;
+ unsigned int arm_sn;
int cc;
+
+ struct napi_struct napi;
};
/* for FW use only */
@@ -94,4 +123,10 @@ void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev);
int mlx5_nodnic_open(struct net_device *netdev);
int mlx5_nodnic_close(struct net_device *netdev);
+void mlx5_nodnic_arm_cq(struct mlx5_nodnic_cq *cq);
+int mlx5_nodnic_napi_poll(struct napi_struct *napi, int budget);
+netdev_features_t mlx5_nodnic_check_features(struct sk_buff *skb,
+ struct net_device *netdev,
+ netdev_features_t features);
+
#endif /* __MLX5_NODNIC_EN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index 5724ed41d8b8..9657c43e9958 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -15,6 +15,8 @@
#include "nodnic_ifc.h"
#include "nodnic.h"
#include "nodnic_pci_vsc.h"
+#include "rx.h"
+#include "tx.h"
#define MLX5_NODNIC_CQ_INIT_CMD_SN cpu_to_be32(2 << 28)
@@ -347,6 +349,46 @@ static int mlx5_nodnic_read_cq_n(struct mlx5_nodnic_core_dev *dev)
return 0;
}
+static irqreturn_t mlx5_nodnic_cq_irq_handler(int irq, void *data)
+{
+ struct mlx5_nodnic_cq *cq = data;
+
+ cq->arm_sn++;
+ napi_schedule_irqoff(&cq->napi);
+
+ return IRQ_HANDLED;
+}
+
+static int mlx5_nodnic_setup_data_irq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct net_device *netdev = priv->netdev;
+ int irqn, err;
+
+ irqn = pci_irq_vector(priv->core_dev->pdev, MLX5_NODNIC_DATA_MSIX);
+ if (irqn < 0) {
+ netdev_err(netdev, "pci_irq_vector failed: %d\n", irqn);
+ return irqn;
+ }
+
+ err = request_irq(irqn, mlx5_nodnic_cq_irq_handler, 0,
+ "mlx5_nodnic-cq", cq);
+ if (err) {
+ netdev_err(netdev, "request_irq failed\n");
+ return err;
+ }
+ cq->irqn = irqn;
+
+ return 0;
+}
+
+static void mlx5_nodnic_cleanup_data_irq(struct mlx5_nodnic_cq *cq)
+{
+ free_irq(cq->irqn, cq);
+}
+
int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_ring *work_buf = &dev->priv->working_buffer;
@@ -376,6 +418,7 @@ int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
int err;
cq->cc = 0;
+ cq->arm_sn = 0;
err = mlx5_nodnic_init_cq_ring(dev);
if (err)
@@ -389,8 +432,19 @@ int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
if (err)
goto err_cleanup_dbr;
+ netif_napi_add_config_locked(dev->priv->netdev, &cq->napi,
+ mlx5_nodnic_napi_poll, 0);
+
+ err = mlx5_nodnic_setup_data_irq(cq);
+ if (err)
+ goto err_napi_del;
+
+ netif_napi_set_irq_locked(&cq->napi, cq->irqn);
+
return 0;
+err_napi_del:
+ netif_napi_del_locked(&cq->napi);
err_cleanup_dbr:
mlx5_nodnic_cleanup_cq_dbr(dev);
err_cleanup_ring:
@@ -403,6 +457,8 @@ static void mlx5_nodnic_close_cq(struct mlx5_nodnic_cq *cq)
struct mlx5_nodnic_core_dev *dev =
container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+ mlx5_nodnic_cleanup_data_irq(cq);
+ netif_napi_del_locked(&cq->napi);
mlx5_nodnic_cleanup_cq_dbr(dev);
mlx5_nodnic_cleanup_cq_ring(dev);
}
@@ -427,14 +483,28 @@ int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev)
ring->lkey = dev->lkey;
+ sq->wqe_info = kvcalloc(ring->num_entries,
+ sizeof(struct mlx5_nodnic_tx_wqe_info),
+ GFP_KERNEL);
+ if (!sq->wqe_info) {
+ err = -ENOMEM;
+ goto err_free_ring;
+ }
+
return 0;
+
+err_free_ring:
+ mlx5_nodnic_free_ring(dev, ring);
+ return err;
}
static void mlx5_nodnic_close_sq(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_sq *sq = &dev->priv->sq;
+ mlx5_nodnic_sq_free_pending(sq);
mlx5_nodnic_free_ring(dev, &sq->ring);
+ kvfree(sq->wqe_info);
}
int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev)
@@ -573,6 +643,143 @@ void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
mutex_unlock(&priv->state_lock);
}
+static int mlx5_nodnic_activate_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct net_device *netdev = priv->netdev;
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ struct page_pool_params pp_params = {
+ .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+ .order = 0,
+ .pool_size = rq->ring.num_entries,
+ .nid = dev_to_node(dev->device),
+ .dev = dev->device,
+ .napi = &priv->cq.napi,
+ .netdev = netdev,
+ .dma_dir = DMA_FROM_DEVICE,
+ .offset = MLX5_NODNIC_SKB_HEADROOM_BYTES,
+ .max_len = MLX5_NODNIC_HW_MTU,
+ };
+ int err;
+
+ rq->page_pool = page_pool_create(&pp_params);
+ if (IS_ERR(rq->page_pool)) {
+ err = PTR_ERR(rq->page_pool);
+ return err;
+ }
+
+ err = mlx5_nodnic_rq_alloc_pages(rq);
+ if (err) {
+ netdev_err(netdev, "failed to alloc RX pages, err=%d\n", err);
+ goto err_destroy_page_pool;
+ }
+
+ netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_RX, &priv->cq.napi);
+
+ return 0;
+
+err_destroy_page_pool:
+ page_pool_destroy(rq->page_pool);
+
+ return err;
+}
+
+static void mlx5_nodnic_deactivate_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct net_device *netdev = dev->priv->netdev;
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+
+ netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_RX, NULL);
+
+ mlx5_nodnic_rq_free_pages(rq);
+ page_pool_destroy(rq->page_pool);
+}
+
+static void mlx5_nodnic_tx_disable_queue(struct netdev_queue *txq)
+{
+ __netif_tx_lock_bh(txq);
+ netif_tx_stop_queue(txq);
+ __netif_tx_unlock_bh(txq);
+}
+
+static void mlx5_nodnic_activate_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+
+ sq->txq = netdev_get_tx_queue(priv->netdev, 0);
+ netdev_tx_reset_queue(sq->txq);
+ netif_tx_start_queue(sq->txq);
+ netif_queue_set_napi(sq->netdev, 0, NETDEV_QUEUE_TYPE_TX,
+ &priv->cq.napi);
+}
+
+static void mlx5_nodnic_deactivate_sq(struct mlx5_nodnic_sq *sq)
+{
+ netif_queue_set_napi(sq->netdev, 0, NETDEV_QUEUE_TYPE_TX, NULL);
+ mlx5_nodnic_tx_disable_queue(sq->txq);
+}
+
+static void mlx5_nodnic_init_cqes_ownership(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_cqe64 *cqe;
+
+ for (int i = 0; i < cq->num_entries; i++) {
+ cqe = (struct mlx5_cqe64 *)
+ ((u8 *)cq->ring.wq + i * MLX5_NODNIC_CQE_SIZE_B);
+ cqe->op_own = 0xf1;
+ }
+}
+
+static int mlx5_nodnic_activate_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ int err;
+
+ napi_enable_locked(&cq->napi);
+
+ mlx5_nodnic_init_cqes_ownership(cq);
+
+ err = mlx5_nodnic_activate_rq(dev);
+ if (err)
+ goto err_napi;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ goto err_rq;
+
+ err = mlx5_nodnic_enable_port(dev);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err)
+ goto err_rq;
+
+ mlx5_nodnic_arm_cq(cq);
+ napi_schedule(&cq->napi);
+
+ mlx5_nodnic_activate_sq(dev);
+
+ return 0;
+
+err_rq:
+ mlx5_nodnic_deactivate_rq(dev);
+err_napi:
+ napi_disable_locked(&cq->napi);
+
+ return err;
+}
+
+static void mlx5_nodnic_deactivate_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ if (!mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx)) {
+ mlx5_nodnic_disable_port(dev);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ }
+
+ mlx5_nodnic_deactivate_sq(&dev->priv->sq);
+ napi_disable_locked(&dev->priv->cq.napi);
+ mlx5_nodnic_deactivate_rq(dev);
+}
+
int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
@@ -692,6 +899,10 @@ static int mlx5_nodnic_open_locked(struct net_device *netdev)
set_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ err = mlx5_nodnic_activate_queues(dev);
+ if (err)
+ goto err_close;
+
err = mlx5_nodnic_refresh_carrier_locked(dev->priv);
if (err)
netdev_err(dev->priv->netdev,
@@ -699,6 +910,9 @@ static int mlx5_nodnic_open_locked(struct net_device *netdev)
return 0;
+err_close:
+ clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ mlx5_nodnic_close_queues(dev);
err_uar:
mlx5_nodnic_cleanup_uar_map(dev);
return err;
@@ -721,6 +935,7 @@ static int mlx5_nodnic_close_locked(struct net_device *netdev)
struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ mlx5_nodnic_deactivate_queues(dev);
clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
netif_carrier_off(priv->netdev);
netdev_info(priv->netdev, "Link down\n");
@@ -751,6 +966,8 @@ int mlx5_nodnic_close(struct net_device *netdev)
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
+ .ndo_start_xmit = mlx5_nodnic_xmit,
+ .ndo_features_check = mlx5_nodnic_check_features,
};
void mlx5_nodnic_build_netdev(struct net_device *netdev)
@@ -765,6 +982,9 @@ void mlx5_nodnic_build_netdev(struct net_device *netdev)
netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
netdev->request_ops_lock = true;
+ netdev->hw_features = NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
+ netdev->features = netdev->hw_features;
+
eth_hw_addr_set(netdev, core_dev->mac_address);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
new file mode 100644
index 000000000000..7736a1f24ea5
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
@@ -0,0 +1,169 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/errno.h>
+#include <linux/if_vlan.h>
+#include <linux/kernel.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "rx.h"
+
+int mlx5_nodnic_rq_alloc_pages(struct mlx5_nodnic_rq *rq)
+{
+ int entries = rq->ring.num_entries;
+ struct page *page;
+
+ rq->pages = kvcalloc(rq->ring.num_entries, sizeof(*rq->pages),
+ GFP_KERNEL);
+ if (!rq->pages)
+ return -ENOMEM;
+
+ for (int i = 0; i < entries; i++) {
+ page = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!page))
+ goto err_free_pages;
+
+ rq->pages[i] = page;
+ }
+
+ return 0;
+
+err_free_pages:
+ mlx5_nodnic_rq_free_pages(rq);
+ return -ENOMEM;
+}
+
+void mlx5_nodnic_rq_free_pages(struct mlx5_nodnic_rq *rq)
+{
+ u32 i;
+
+ for (i = 0; i < rq->ring.num_entries; i++) {
+ if (!rq->pages[i])
+ continue;
+
+ page_pool_put_full_page(rq->page_pool,
+ rq->pages[i], false);
+ }
+
+ kvfree(rq->pages);
+}
+
+static void mlx5_nodnic_rq_push_db(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+
+ /* ensure wqes are visible to device before updating dbr */
+ dma_wmb();
+
+ /* update local DB record */
+ priv->dbr.db[MLX5_NODNIC_RCV_DBR] = cpu_to_be32(rq->pc & 0xffff);
+}
+
+static u16 mlx5_nodnic_rq_get_pi(struct mlx5_nodnic_rq *rq)
+{
+ return rq->pc & (rq->ring.num_entries - 1);
+}
+
+static int mlx5_nodnic_rq_post_one(struct mlx5_nodnic_rq *rq)
+{
+ u16 pi = mlx5_nodnic_rq_get_pi(rq);
+ struct mlx5_nodnic_rx_wqe *wqe;
+
+ if (!rq->pages[pi]) {
+ rq->pages[pi] = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!rq->pages[pi]))
+ return -ENOMEM;
+ }
+
+ wqe = (struct mlx5_nodnic_rx_wqe *)
+ ((u8 *)rq->ring.wq + pi * MLX5_NODNIC_RQ_STRIDE_BYTES);
+ wqe->byte_count = cpu_to_be32(MLX5_NODNIC_HW_MTU);
+ wqe->lkey = cpu_to_be32(rq->ring.lkey);
+ wqe->addr = cpu_to_be64(page_pool_get_dma_addr(rq->pages[pi]) +
+ MLX5_NODNIC_SKB_HEADROOM_BYTES);
+ rq->pc++;
+
+ return 0;
+}
+
+bool mlx5_nodnic_rq_post_wqes(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u32 avail;
+ int i;
+
+ avail = rq->ring.num_entries - (u16)(rq->pc - rq->cc);
+ for (i = 0; i < avail; i++) {
+ if (mlx5_nodnic_rq_post_one(rq))
+ break;
+ }
+
+ if (i)
+ mlx5_nodnic_rq_push_db(priv);
+
+ return i == avail;
+}
+
+static void nodnic_handle_rx_csum(struct net_device *netdev,
+ const struct mlx5_cqe64 *cqe,
+ struct sk_buff *skb)
+{
+ skb->ip_summed = CHECKSUM_NONE;
+
+ if (unlikely(!(netdev->features & NETIF_F_RXCSUM)))
+ return;
+
+ if (likely((cqe->hds_ip_ext & CQE_L3_OK) &&
+ (cqe->hds_ip_ext & CQE_L4_OK)))
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+}
+
+void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ u32 byte_count = be32_to_cpu(cqe->byte_cnt);
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u16 wqe_counter, ci;
+ struct sk_buff *skb;
+ struct page *page;
+
+ wqe_counter = be16_to_cpu(cqe->wqe_counter);
+ ci = wqe_counter & (rq->ring.num_entries - 1);
+ page = rq->pages[ci];
+
+ page_pool_dma_sync_for_cpu(rq->page_pool, page, 0, byte_count);
+
+ skb = napi_build_skb(page_address(page), PAGE_SIZE);
+ if (unlikely(!skb))
+ goto err_recycle;
+
+ skb_mark_for_recycle(skb);
+ rq->pages[ci] = NULL;
+
+ skb_reserve(skb, MLX5_NODNIC_SKB_HEADROOM_BYTES);
+ skb_put(skb, byte_count);
+ skb->protocol = eth_type_trans(skb, priv->netdev);
+ nodnic_handle_rx_csum(priv->netdev, cqe, skb);
+
+ napi_gro_receive(&cq->napi, skb);
+ rq->cc++;
+
+ return;
+
+err_recycle:
+ mlx5_nodnic_release_rx_wqe(rq, ci);
+}
+
+void mlx5_nodnic_release_rx_wqe(struct mlx5_nodnic_rq *rq, u16 ci)
+{
+ page_pool_put_full_page(rq->page_pool, rq->pages[ci], true);
+ rq->pages[ci] = NULL;
+ rq->cc++;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
new file mode 100644
index 000000000000..ccdfa3e69824
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_RX_H__
+#define __MLX5_NODNIC_RX_H__
+
+struct mlx5_nodnic_priv;
+struct mlx5_nodnic_cq;
+struct mlx5_nodnic_rq;
+struct mlx5_cqe64;
+
+struct mlx5_nodnic_rx_wqe {
+ __be32 byte_count;
+ __be32 lkey;
+ __be64 addr;
+};
+
+int mlx5_nodnic_rq_alloc_pages(struct mlx5_nodnic_rq *rq);
+void mlx5_nodnic_rq_free_pages(struct mlx5_nodnic_rq *rq);
+bool mlx5_nodnic_rq_post_wqes(struct mlx5_nodnic_priv *priv);
+void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe);
+void mlx5_nodnic_release_rx_wqe(struct mlx5_nodnic_rq *rq, u16 ci);
+
+#endif /* __MLX5_NODNIC_RX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
new file mode 100644
index 000000000000..f2efc8d3f5b8
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/mlx5/doorbell.h>
+#include <linux/net.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "tx.h"
+
+#define MLX5_NODNIC_OPCODE_SEND 0x0a
+#define MLX5_NODNIC_DS_COUNT 3
+#define MLX5_NODNIC_CE_ALWAYS 0x2
+#define MLX5_NODNIC_SQ_STOP_ROOM 1
+
+static void mlx5_nodnic_tx_build_wqe_eseg(struct sk_buff *skb,
+ struct mlx5_nodnic_wqe_eth_seg *eseg)
+{
+ u8 flags = 0;
+
+ if (likely(skb->ip_summed == CHECKSUM_PARTIAL))
+ flags |= MLX5_NODNIC_ETH_WQE_L3_CSUM |
+ MLX5_NODNIC_ETH_WQE_L4_CSUM;
+
+ eseg->cs_flags_mss = cpu_to_be32((u32)flags << 24);
+ eseg->inline_hdr_sz = 0;
+}
+
+static void
+mlx5_nodnic_tx_build_wqe_ctrl(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_wqe_ctrl_seg *ctrl)
+{
+ ctrl->opmod_idx_opcode = cpu_to_be32((sq->pc << 8) |
+ MLX5_NODNIC_OPCODE_SEND);
+ ctrl->qpn_ds = cpu_to_be32((sq->ring.qn << 8) | MLX5_NODNIC_DS_COUNT);
+ ctrl->fm_ce_se = cpu_to_be32(MLX5_NODNIC_CE_ALWAYS << 2);
+ ctrl->general_id = 0;
+}
+
+static void mlx5_nodnic_tx_fill_wqe_info(struct mlx5_nodnic_sq *sq,
+ struct sk_buff *skb,
+ dma_addr_t dma_addr)
+{
+ u16 pi = mlx5_nodnic_sq_get_pi(sq, sq->pc);
+
+ sq->wqe_info[pi].skb = skb;
+ sq->wqe_info[pi].num_bytes = skb->len;
+ sq->wqe_info[pi].dma = dma_addr;
+}
+
+static int
+mlx5_nodnic_tx_build_wqe_ds(struct mlx5_nodnic_sq *sq, struct sk_buff *skb,
+ struct mlx5_nodnic_wqe_data_seg *dseg)
+{
+ dma_addr_t dma_addr = 0;
+
+ dseg->byte_count = cpu_to_be32(skb->len & 0x7FFFFFFF);
+ dseg->lkey = cpu_to_be32(sq->ring.lkey);
+ dma_addr = dma_map_single(sq->device, skb->data, skb->len,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(sq->device, dma_addr)))
+ return -ENOMEM;
+
+ dseg->addr = cpu_to_be64(dma_addr);
+
+ mlx5_nodnic_tx_fill_wqe_info(sq, skb, dma_addr);
+
+ return 0;
+}
+
+static __always_inline int
+mlx5_nodnic_tx_write_wqe(struct mlx5_nodnic_sq *sq, struct sk_buff *skb,
+ struct mlx5_nodnic_tx_wqe *wqe)
+{
+ int err;
+
+ memset(wqe, 0, MLX5_NODNIC_SQ_WQEBB_B);
+ mlx5_nodnic_tx_build_wqe_ctrl(sq, &wqe->ctrl);
+ mlx5_nodnic_tx_build_wqe_eseg(skb, &wqe->eth);
+ err = mlx5_nodnic_tx_build_wqe_ds(sq, skb, &wqe->data);
+ if (err)
+ return err;
+
+ /* Ensure WQE stores are visible to the device before ringing the db */
+ dma_wmb();
+
+ return 0;
+}
+
+static int mlx5_nodnic_tx_is_sq_full(struct mlx5_nodnic_sq *sq)
+{
+ u16 pending = (u16)(sq->pc - sq->cc);
+
+ return pending == sq->ring.num_entries - MLX5_NODNIC_SQ_STOP_ROOM;
+}
+
+static int mlx5_nodnic_tx_get_next_sq_offset(struct mlx5_nodnic_sq *sq)
+{
+ u16 pi;
+
+ pi = mlx5_nodnic_sq_get_pi(sq, sq->pc);
+ return pi * MLX5_NODNIC_SQ_WQEBB_B;
+}
+
+static void mlx5_nodnic_tx_notify_hw(struct mlx5_nodnic_priv *priv,
+ void __iomem *uar_base, u16 pc,
+ struct mlx5_nodnic_wqe_ctrl_seg *ctrl)
+{
+ /* ensure wqe is visible to device before updating doorbell record */
+ dma_wmb();
+
+ priv->dbr.db[MLX5_NODNIC_SND_DBR] = cpu_to_be32(pc);
+
+ /* ensure doorbell record is visible to device before ringing the
+ * doorbell
+ */
+ wmb();
+
+ mlx5_write64((__be32 *)ctrl, uar_base + MLX5_BF_OFFSET);
+}
+
+netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_tx_wqe *wqe;
+ u16 next_offset;
+ int err;
+
+ if (unlikely(mlx5_nodnic_tx_is_sq_full(sq)))
+ netif_tx_stop_queue(sq->txq);
+
+ next_offset = mlx5_nodnic_tx_get_next_sq_offset(sq);
+
+ wqe = (struct mlx5_nodnic_tx_wqe *)((u8 *)sq->ring.wq + next_offset);
+
+ err = mlx5_nodnic_tx_write_wqe(sq, skb, wqe);
+ if (err) {
+ dev_kfree_skb_any(skb);
+
+ return NETDEV_TX_OK;
+ }
+
+ sq->pc += 1;
+
+ netdev_tx_sent_queue(sq->txq, skb->len);
+
+ mlx5_nodnic_tx_notify_hw(priv, dev->uar_base, sq->pc, &wqe->ctrl);
+
+ return NETDEV_TX_OK;
+}
+
+u16 mlx5_nodnic_sq_get_pi(struct mlx5_nodnic_sq *sq, u16 pc)
+{
+ return pc & (sq->ring.num_entries - 1);
+}
+
+void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe, int napi_budget)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_tx_wqe_info *wi;
+ u16 pi;
+
+ pi = mlx5_nodnic_sq_get_pi(sq, be16_to_cpu(cqe->wqe_counter));
+ wi = &sq->wqe_info[pi];
+
+ mlx5_nodnic_complete_send_wqe(sq, wi, napi_budget);
+}
+
+void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_tx_wqe_info *wi,
+ int napi_budget)
+{
+ struct netdev_queue *txq = sq->txq;
+
+ dma_unmap_single(sq->device, wi->dma, wi->num_bytes,
+ DMA_TO_DEVICE);
+ napi_consume_skb(wi->skb, napi_budget);
+
+ netdev_tx_completed_queue(txq, 1, wi->num_bytes);
+
+ sq->cc++;
+
+ if (netif_tx_queue_stopped(txq))
+ netif_tx_wake_queue(txq);
+}
+
+void mlx5_nodnic_sq_free_pending(struct mlx5_nodnic_sq *sq)
+{
+ struct mlx5_nodnic_tx_wqe_info *wi;
+ unsigned int completed_bytes = 0;
+ unsigned int completed_pkts = 0;
+ u16 sqcc = sq->cc;
+ u16 ci;
+
+ while (sqcc != sq->pc) {
+ ci = mlx5_nodnic_sq_get_pi(sq, sqcc);
+ wi = &sq->wqe_info[ci];
+
+ dma_unmap_single(sq->device, wi->dma,
+ wi->num_bytes, DMA_TO_DEVICE);
+
+ completed_pkts++;
+ completed_bytes += wi->num_bytes;
+
+ dev_kfree_skb_any(wi->skb);
+
+ sqcc++;
+ }
+
+ if (completed_pkts)
+ netdev_tx_completed_queue(sq->txq, completed_pkts,
+ completed_bytes);
+ sq->cc = sqcc;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
new file mode 100644
index 000000000000..b93a27dffb92
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_TX_H__
+#define __MLX5_NODNIC_TX_H__
+
+#include <linux/netdevice.h>
+#include <linux/types.h>
+#include <linux/skbuff.h>
+
+struct mlx5_nodnic_sq;
+struct mlx5_nodnic_cq;
+struct mlx5_cqe64;
+
+struct mlx5_nodnic_tx_wqe_info {
+ struct sk_buff *skb;
+ u32 num_bytes;
+ dma_addr_t dma;
+};
+
+enum {
+ MLX5_NODNIC_ETH_WQE_L3_CSUM = 1 << 6,
+ MLX5_NODNIC_ETH_WQE_L4_CSUM = 1 << 7,
+};
+
+struct mlx5_nodnic_wqe_ctrl_seg {
+ __be32 opmod_idx_opcode;
+ __be32 qpn_ds;
+ __be32 fm_ce_se;
+ __be32 general_id;
+} __packed;
+
+struct mlx5_nodnic_wqe_eth_seg {
+ __be32 swp_offsets;
+ __be32 cs_flags_mss;
+ __be32 flow_metadata;
+ __be32 inline_hdr_sz;
+} __packed;
+
+struct mlx5_nodnic_wqe_data_seg {
+ __be32 byte_count;
+ __be32 lkey;
+ __be64 addr;
+} __packed;
+
+struct mlx5_nodnic_tx_wqe {
+ struct mlx5_nodnic_wqe_ctrl_seg ctrl;
+ struct mlx5_nodnic_wqe_eth_seg eth;
+ struct mlx5_nodnic_wqe_data_seg data;
+} __packed;
+
+netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *dev);
+u16 mlx5_nodnic_sq_get_pi(struct mlx5_nodnic_sq *sq, u16 pc);
+void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe, int napi_budget);
+void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_tx_wqe_info *wi,
+ int napi_budget);
+void mlx5_nodnic_sq_free_pending(struct mlx5_nodnic_sq *sq);
+
+#endif /* __MLX5_NODNIC_TX_H__ */
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
@ 2026-10-08 17:18 ` Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
4 siblings, 1 reply; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
From: Shahar Shitrit <shshitrit@nvidia.com>
Add TX and RX packet and byte statistics, as well as TX drop
accounting.
Update the counters from the data path and expose them through
ndo_get_stats64.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 14 +++++++++++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 25 +++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 3 +++
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 4 +++
4 files changed, 46 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index 19858e9da064..d811bfe242ac 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -44,6 +44,17 @@ enum mlx5_nodnic_cqe_opcode {
OPCODE_CQ_INVALID,
};
+struct mlx5_nodnic_sq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 dropped;
+};
+
+struct mlx5_nodnic_rq_stats {
+ u64 packets;
+ u64 bytes;
+};
+
struct mlx5_nodnic_ring {
void *wq;
dma_addr_t dma_handle;
@@ -109,6 +120,9 @@ struct mlx5_nodnic_priv {
struct mutex state_lock; /* protects Interface state */
unsigned long state;
+ struct mlx5_nodnic_sq_stats sq_stats;
+ struct mlx5_nodnic_rq_stats rq_stats;
+
struct mlx5_nodnic_sq sq;
struct mlx5_nodnic_rq rq;
struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index 9657c43e9958..cc8b3d3fb0c6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -963,10 +963,35 @@ int mlx5_nodnic_close(struct net_device *netdev)
return err;
}
+static void mlx5_nodnic_update_sq_stats(const struct mlx5_nodnic_sq_stats *sq,
+ struct rtnl_link_stats64 *s)
+{
+ s->tx_packets = sq->packets;
+ s->tx_bytes = sq->bytes;
+ s->tx_dropped = sq->dropped;
+}
+
+static void mlx5_nodnic_update_rq_stats(const struct mlx5_nodnic_rq_stats *rq,
+ struct rtnl_link_stats64 *s)
+{
+ s->rx_packets = rq->packets;
+ s->rx_bytes = rq->bytes;
+}
+
+static void mlx5_nodnic_get_stats64(struct net_device *netdev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+
+ mlx5_nodnic_update_sq_stats(&priv->sq_stats, stats);
+ mlx5_nodnic_update_rq_stats(&priv->rq_stats, stats);
+}
+
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
.ndo_start_xmit = mlx5_nodnic_xmit,
+ .ndo_get_stats64 = mlx5_nodnic_get_stats64,
.ndo_features_check = mlx5_nodnic_check_features,
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
index 7736a1f24ea5..808740ec3910 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
@@ -152,6 +152,9 @@ void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
skb->protocol = eth_type_trans(skb, priv->netdev);
nodnic_handle_rx_csum(priv->netdev, cqe, skb);
+ priv->rq_stats.packets++;
+ priv->rq_stats.bytes += byte_count;
+
napi_gro_receive(&cq->napi, skb);
rq->cc++;
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
index f2efc8d3f5b8..6958c8a51231 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
@@ -143,6 +143,7 @@ netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *netdev)
err = mlx5_nodnic_tx_write_wqe(sq, skb, wqe);
if (err) {
+ priv->sq_stats.dropped++;
dev_kfree_skb_any(skb);
return NETDEV_TX_OK;
@@ -176,6 +177,9 @@ void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
wi = &sq->wqe_info[pi];
mlx5_nodnic_complete_send_wqe(sq, wi, napi_budget);
+
+ priv->sq_stats.packets++;
+ priv->sq_stats.bytes += wi->num_bytes;
}
void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next V2 4/5] net/mlx5: nodnic: Add ethtool driver info
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
` (2 preceding siblings ...)
2026-10-08 17:18 ` [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
@ 2026-10-08 17:18 ` Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
4 siblings, 0 replies; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
From: Shahar Shitrit <shshitrit@nvidia.com>
Implement the get_drvinfo ethtool operation to report the driver name,
firmware version, and PCI bus information.
Read the firmware revision from the initialization segment and expose
it together with the driver information through ethtool.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index cc8b3d3fb0c6..033db1999c74 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -987,6 +987,39 @@ static void mlx5_nodnic_get_stats64(struct net_device *netdev,
mlx5_nodnic_update_rq_stats(&priv->rq_stats, stats);
}
+static u16 mlx5_nodnic_fw_rev_maj(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->fw_rev) & 0xffff;
+}
+
+static u16 mlx5_nodnic_fw_rev_min(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->fw_rev) >> 16;
+}
+
+static u16 mlx5_nodnic_fw_rev_sub(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->cmdif_rev_fw_sub) & 0xffff;
+}
+
+static void mlx5_nodnic_get_drvinfo(struct net_device *netdev,
+ struct ethtool_drvinfo *drvinfo)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *mdev = priv->core_dev;
+
+ strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver));
+ snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version),
+ "%d.%d.%04d", mlx5_nodnic_fw_rev_maj(mdev),
+ mlx5_nodnic_fw_rev_min(mdev), mlx5_nodnic_fw_rev_sub(mdev));
+ strscpy(drvinfo->bus_info, dev_name(mdev->device),
+ sizeof(drvinfo->bus_info));
+}
+
+static const struct ethtool_ops mlx5_nodnic_ethtool_ops = {
+ .get_drvinfo = mlx5_nodnic_get_drvinfo,
+};
+
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
@@ -1005,6 +1038,7 @@ void mlx5_nodnic_build_netdev(struct net_device *netdev)
SET_NETDEV_DEV(netdev, core_dev->device);
netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
+ netdev->ethtool_ops = &mlx5_nodnic_ethtool_ops;
netdev->request_ops_lock = true;
netdev->hw_features = NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
` (3 preceding siblings ...)
2026-10-08 17:18 ` [PATCH net-next V2 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
@ 2026-10-08 17:18 ` Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
4 siblings, 1 reply; 10+ messages in thread
From: Tariq Toukan @ 2026-10-08 17:18 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea, Simon Horman
From: Shahar Shitrit <shshitrit@nvidia.com>
Monitor firmware health by periodically polling the health buffer from
the initialization segment.
Detect fatal device conditions, firmware health counter stalls, and
syndrome changes. Report health errors together with their severity
and mark the device as unhealthy on fatal conditions.
Start and stop health polling as part of the device probe and remove
flows.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/nodnic/Makefile | 2 +-
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 12 +
.../ethernet/mellanox/mlx5/nodnic/health.c | 315 ++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 2 +
4 files changed, 330 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
index 25865d7244d8..1d09edeaf3af 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -5,4 +5,4 @@
obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
-mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o
+mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o health.o
\ No newline at end of file
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index d811bfe242ac..90e9f273d454 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -113,6 +113,16 @@ struct mlx5_nodnic_working_buffer {
u32 alloc_size;
};
+struct mlx5_nodnic_health {
+ struct health_buffer __iomem *health;
+ __be32 __iomem *health_counter;
+ struct timer_list timer;
+ u32 prev_counter;
+ int miss_counter;
+ u8 synd;
+ u32 fatal_error;
+};
+
struct mlx5_nodnic_priv {
struct mlx5_nodnic_core_dev *core_dev;
struct net_device *netdev;
@@ -128,6 +138,8 @@ struct mlx5_nodnic_priv {
struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
struct mlx5_nodnic_cq cq;
struct mlx5_nodnic_ring working_buffer;
+
+ struct mlx5_nodnic_health health;
};
void mlx5_nodnic_build_netdev(struct net_device *netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
new file mode 100644
index 000000000000..a981a5195c7e
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
@@ -0,0 +1,315 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/kernel.h>
+#include <linux/random.h>
+#include <linux/timer.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+
+#define MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS 2000
+#define MLX5_NODNIC_MAX_MISSES 3
+
+enum nodnic_rfr_severity_bit_offsets {
+ NODNIC_CRR_BIT_OFFSET = 0x6,
+ NODNIC_RFR_BIT_OFFSET = 0x7,
+};
+
+enum {
+ MLX5_NODNIC_SEVERITY_MASK = 0x7,
+ MLX5_NODNIC_SEVERITY_VALID_MASK = 0x8,
+};
+
+enum {
+ MLX5_NODNIC_SENSOR_NO_ERR,
+ MLX5_NODNIC_SENSOR_PCI_COMM_ERR,
+ MLX5_NODNIC_SENSOR_PCI_ERR,
+ MLX5_NODNIC_SENSOR_NIC_DISABLED,
+ MLX5_NODNIC_SENSOR_NIC_SW_RESET,
+ MLX5_NODNIC_SENSOR_FW_SYND_RFR,
+};
+
+static unsigned long get_next_poll_jiffies(void)
+{
+ return jiffies + msecs_to_jiffies(MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS) +
+ get_random_u32_below(HZ);
+}
+
+static bool sensor_pci_not_working(struct mlx5_nodnic_health *health)
+{
+ struct health_buffer __iomem *h = health->health;
+
+ return (ioread32be(&h->fw_ver) == 0xffffffff);
+}
+
+static u8 mlx5_nodnic_get_nic_state(struct mlx5_nodnic_core_dev *dev)
+{
+ return (ioread32be(&dev->iseg->cmdq_addr_l_sz) >> 8) & 7;
+}
+
+static int mlx5_nodnic_health_get_rfr(u8 rfr_severity)
+{
+ return rfr_severity >> NODNIC_RFR_BIT_OFFSET;
+}
+
+static bool sensor_fw_synd_rfr(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+ struct health_buffer __iomem *h = health->health;
+ u8 synd = ioread8(&h->synd);
+ u8 rfr;
+
+ rfr = mlx5_nodnic_health_get_rfr(ioread8(&h->rfr_severity));
+
+ if (rfr && synd)
+ mlx5_nodnic_core_dbg(dev, "FW requests reset, synd: %d\n",
+ synd);
+
+ return rfr && synd;
+}
+
+static
+u32 mlx5_nodnic_health_check_fatal_sensors(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 nic_state;
+
+ if (pci_channel_offline(dev->pdev))
+ return MLX5_NODNIC_SENSOR_PCI_ERR;
+ if (sensor_pci_not_working(&dev->priv->health))
+ return MLX5_NODNIC_SENSOR_PCI_COMM_ERR;
+
+ nic_state = mlx5_nodnic_get_nic_state(dev);
+ if (nic_state == MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED)
+ return MLX5_NODNIC_SENSOR_NIC_DISABLED;
+ if (nic_state == MLX5_NODNIC_ISEG_NIC_INTERFACE_SW_RESET)
+ return MLX5_NODNIC_SENSOR_NIC_SW_RESET;
+
+ if (sensor_fw_synd_rfr(dev))
+ return MLX5_NODNIC_SENSOR_FW_SYND_RFR;
+
+ return MLX5_NODNIC_SENSOR_NO_ERR;
+}
+
+static const char *mlx5_nodnic_fatal_error_str(u32 err)
+{
+ switch (err) {
+ case MLX5_NODNIC_SENSOR_PCI_COMM_ERR: return "PCI communication error";
+ case MLX5_NODNIC_SENSOR_PCI_ERR: return "PCI error";
+ case MLX5_NODNIC_SENSOR_NIC_DISABLED: return "NIC disabled";
+ case MLX5_NODNIC_SENSOR_NIC_SW_RESET: return "NIC SW reset";
+ case MLX5_NODNIC_SENSOR_FW_SYND_RFR: return "FW synd RFR";
+ default: return "unknown";
+ }
+}
+
+static int mlx5_nodnic_health_get_crr(u8 rfr_severity)
+{
+ return (rfr_severity >> NODNIC_CRR_BIT_OFFSET) & 0x01;
+}
+
+static int mlx5_nodnic_health_get_severity(u8 rfr_severity)
+{
+ return rfr_severity & MLX5_NODNIC_SEVERITY_VALID_MASK ?
+ rfr_severity & MLX5_NODNIC_SEVERITY_MASK : LOGLEVEL_ERR;
+}
+
+/* Syndrome values from health buffer (same as mlx5_ifc.h) */
+static const char *mlx5_nodnic_hsynd_str(u8 synd)
+{
+ switch (synd) {
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_INTERNAL_ERR:
+ return "firmware internal error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_DEAD_IRISC:
+ return "irisc not responding";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HW_FATAL_ERR:
+ return "unrecoverable hardware error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_CRC_ERR:
+ return "firmware CRC error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_FETCH_PCI_ERR:
+ return "ICM fetch PCI error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PAGE_ERR:
+ return "HW fatal error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ASYNCHRONOUS_EQ_BUF_OVERRUN:
+ return "async EQ buffer overrun";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_IN_ERR:
+ return "EQ error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_INV:
+ return "Invalid EQ referenced";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FFSER_ERR:
+ return "FFSER error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HIGH_TEMP_ERR:
+ return "High temperature";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PCI_POISONED_ERR:
+ return "ICM fetch PCI data poisoned error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_TRUST_LOCKDOWN_ERR:
+ return "Trust lockdown error";
+ default:
+ return "unrecognized error";
+ }
+}
+
+static const char *mlx5_nodnic_loglevel_str(int level)
+{
+ switch (level) {
+ case LOGLEVEL_EMERG:
+ return "EMERGENCY";
+ case LOGLEVEL_ALERT:
+ return "ALERT";
+ case LOGLEVEL_CRIT:
+ return "CRITICAL";
+ case LOGLEVEL_ERR:
+ return "ERROR";
+ case LOGLEVEL_WARNING:
+ return "WARNING";
+ case LOGLEVEL_NOTICE:
+ return "NOTICE";
+ case LOGLEVEL_INFO:
+ return "INFO";
+ case LOGLEVEL_DEBUG:
+ return "DEBUG";
+ }
+ return "Unknown log level";
+}
+
+static const char *mlx5_nodnic_severity_to_kern(int severity)
+{
+ if (severity <= LOGLEVEL_ERR)
+ return KERN_ERR;
+
+ if (severity <= LOGLEVEL_WARNING)
+ return KERN_WARNING;
+
+ return KERN_INFO;
+}
+
+static void mlx5_nodnic_print_health_info(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+ struct health_buffer __iomem *h = health->health;
+ struct device *device = dev->device;
+ u8 synd = ioread8(&h->synd);
+ const char *kern_severity;
+ u8 rfr_severity;
+ int severity;
+ int i;
+
+ if (!synd)
+ return;
+
+ if (sensor_pci_not_working(&dev->priv->health)) {
+ mlx5_nodnic_core_err(dev, "PCI slot is unavailable\n");
+ return;
+ }
+
+ rfr_severity = ioread8(&h->rfr_severity);
+ severity = mlx5_nodnic_health_get_severity(rfr_severity);
+ kern_severity = mlx5_nodnic_severity_to_kern(severity);
+
+ dev_printk(kern_severity, device,
+ "Health issue observed, %s, severity(%d) %s:\n",
+ mlx5_nodnic_hsynd_str(synd), severity,
+ mlx5_nodnic_loglevel_str(severity));
+
+ for (i = 0; i < ARRAY_SIZE(h->assert_var); i++)
+ dev_printk(kern_severity, device, "assert_var[%d] 0x%08x\n", i,
+ ioread32be(h->assert_var + i));
+
+ dev_printk(kern_severity, device, "assert_exit_ptr 0x%08x\n",
+ ioread32be(&h->assert_exit_ptr));
+ dev_printk(kern_severity, device, "assert_callra 0x%08x\n",
+ ioread32be(&h->assert_callra));
+ dev_printk(kern_severity, device, "raw fw_ver 0x%08x\n",
+ ioread32be(&h->fw_ver));
+ dev_printk(kern_severity, device, "time %u\n", ioread32be(&h->time));
+ dev_printk(kern_severity, device, "hw_id 0x%08x\n",
+ ioread32be(&h->hw_id));
+ dev_printk(kern_severity, device, "rfr %d\n",
+ mlx5_nodnic_health_get_rfr(rfr_severity));
+ dev_printk(kern_severity, device, "crr %d\n",
+ mlx5_nodnic_health_get_crr(rfr_severity));
+ dev_printk(kern_severity, device, "severity %d (%s)\n", severity,
+ mlx5_nodnic_loglevel_str(severity));
+ dev_printk(kern_severity, device, "irisc_index %d\n",
+ ioread8(&h->irisc_index));
+ dev_printk(kern_severity, device, "synd 0x%x: %s\n", synd,
+ mlx5_nodnic_hsynd_str(synd));
+ dev_printk(kern_severity, device, "ext_synd 0x%04x\n",
+ ioread16be(&h->ext_synd));
+
+ if (mlx5_nodnic_health_get_crr(rfr_severity))
+ mlx5_nodnic_core_warn(dev, "Cold reset is required\n");
+}
+
+static void mlx5_nodnic_poll_health(struct timer_list *t)
+{
+ struct mlx5_nodnic_priv *priv = timer_container_of(priv, t,
+ health.timer);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_health *health = &priv->health;
+ struct health_buffer __iomem *h = health->health;
+ u32 fatal_error, count;
+ u8 prev_synd;
+
+ if (dev->state == NODNIC_DEVICE_STATE_INTERNAL_ERROR)
+ return;
+
+ fatal_error = mlx5_nodnic_health_check_fatal_sensors(dev);
+
+ if (fatal_error && !health->fatal_error) {
+ mlx5_nodnic_core_err(dev, "fatal error detected: %s (%u)\n",
+ mlx5_nodnic_fatal_error_str(fatal_error),
+ fatal_error);
+ priv->health.fatal_error = fatal_error;
+ dev->state = NODNIC_DEVICE_STATE_INTERNAL_ERROR;
+ mlx5_nodnic_print_health_info(dev);
+ return;
+ }
+
+ count = ioread32be(health->health_counter);
+ if (count == health->prev_counter)
+ ++health->miss_counter;
+ else
+ health->miss_counter = 0;
+
+ health->prev_counter = count;
+ if (health->miss_counter == MLX5_NODNIC_MAX_MISSES) {
+ mlx5_nodnic_core_err(dev,
+ "device's health compromised - reached miss count\n");
+ health->synd = ioread8(&h->synd);
+ mlx5_nodnic_print_health_info(dev);
+ }
+
+ prev_synd = health->synd;
+ health->synd = ioread8(&h->synd);
+ if (health->synd && health->synd != prev_synd)
+ mlx5_nodnic_print_health_info(dev);
+
+ mod_timer(&health->timer, get_next_poll_jiffies());
+}
+
+void mlx5_nodnic_start_health_poll(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+
+ timer_setup(&health->timer, mlx5_nodnic_poll_health, 0);
+ health->fatal_error = MLX5_NODNIC_SENSOR_NO_ERR;
+ health->miss_counter = 0;
+ health->prev_counter = 0;
+ health->synd = 0;
+ health->health = &dev->iseg->health;
+ health->health_counter = &dev->iseg->health_counter;
+
+ health->timer.expires = jiffies +
+ msecs_to_jiffies(MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS);
+ add_timer(&health->timer);
+}
+
+void mlx5_nodnic_stop_health_poll(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+
+ timer_delete_sync(&health->timer);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
index a533232bf6ef..7194f7438527 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
@@ -804,6 +804,7 @@ static int mlx5_nodnic_probe_one(struct pci_dev *pdev,
dev->netdev_registered = true;
dev->state = NODNIC_DEVICE_STATE_UP;
+ mlx5_nodnic_start_health_poll(dev);
pci_save_state(pdev);
@@ -829,6 +830,7 @@ static void mlx5_nodnic_remove_one(struct pci_dev *pdev)
if (!dev)
return;
+ mlx5_nodnic_stop_health_poll(dev);
mlx5_nodnic_event_irq_cleanup(dev);
mlx5_nodnic_cleanup_netdev(dev);
mlx5_nodnic_dev_cleanup(dev);
--
2.44.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver
2026-10-08 17:18 ` [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
@ 2026-10-12 17:19 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-10-12 17:19 UTC (permalink / raw)
To: tariqt
Cc: andrew+netdev, davem, edumazet, kuba, netdev, pabeni, cjubran,
gal, linux-kernel, shshitrit, leon, mbloch, saeedm, dtatulea,
horms
Thank you for your contribution! Sashiko AI review found 10 potential
issue(s) to consider.
Critical: 1 · High: 4 · Medium: 1 · Low: 4
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Critical] mlx5: Fatal SQ/RQ size calculation mismatch causes DMA
buffer overflow
- [High] Raw positive PCIBIOS_* codes from config-space accessors reach
the PCI probe return, leading to use-after-free and double kfree.
- [High] Device DMA buffers can be freed while the device still has DMA
enabled.
- [High] mlx5: CQ log_size hardware register is overwritten with zero
during init
- [High] mlx5: 32-bit multiplication overflow in UAR physical address
calculation
- [Medium] The only device this driver binds, 0x15b3:0xc2d6, is already
listed by mlx5_core in mgt_ifc_device_ids[] (fw_reset.c:394,…
- [Low] mlx5_nodnic_build_netdev() installs the MAC that
mlx5_nodnic_read_mac_address() read from device registers…
- [Low] (Baseline: not valid; fixed later in the series.) This patch
registers a netdev whose ops have no ndo_start_xmit and no RX/NAPI…
- [Low] (Baseline: not valid.) The device-supplied size exponents
(log_cq_size, log_max_ring_size, log_uar_page_size) are used in shifts…
- [Low] (Baseline: not valid.) The VSC read/write helpers mask address
bits 31:30 instead of rejecting them, and do not check the syndrome…
--
net/mlx5: nodnic: Introduce the mlx5_nodnic driver
This commit adds mlx5_nodnic, a PCI driver for DRAM-less ConnectX NIC
functions. It covers probe and remove (FLR, firmware readiness polling,
interface negotiation and capability discovery). It also adds a VSC layer
for config-space access to the init segment and the NODNIC registers, and
a netdev with ndo_open/ndo_stop and a link-event IRQ.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
> new file mode 100644
> index 0000000000000..5724ed41d8b89
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
> @@ -0,0 +1,828 @@
[ ... ]
> +/* log_cq_size should be read after rq and sq ring sizes were set. */
> +static int mlx5_nodnic_read_cq_ring_size(struct mlx5_nodnic_core_dev *dev)
> +{
[ ... ]
> + log_cq_size = MLX5_GET(nodnic_q_addr_l, &cq_addr_l, log_size);
> + cq->num_entries = 1ULL << log_cq_size;
> + cq->log_num_entries = log_cq_size;
[Severity: Low]
This isn't a bug, but log_cq_size comes straight from a device register.
The same is true of log_max_ring_size in mlx5_nodnic_calc_ring_size() and
log_uar_page_size in mlx5_nodnic_init_uar_map(). All three are used in
shifts and size calculations with no range check.
mlx5_core also trusts FW-reported exponents, so this is only a hardening
question. Would a sanity check on these values be worth adding?
[ ... ]
> +static void mlx5_nodnic_close_queues(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_priv *priv = dev->priv;
> +
> + mlx5_nodnic_free_ring(dev, &priv->working_buffer);
> + mlx5_nodnic_close_cq(&priv->cq);
> + mlx5_nodnic_close_rq(dev);
> + mlx5_nodnic_close_sq(dev);
> + mlx5_nodnic_free_dbr(dev, &priv->dbr);
> +}
[Severity: High]
Can the device still DMA into these buffers after they are freed?
mlx5_nodnic_open_queues() writes the ring, CQ, doorbell record and working
buffer addresses to the device. Nothing here clears those registers
before dma_free_coherent().
This patch doesn't call mlx5_nodnic_enable_port() yet. Later in the
series, "net/mlx5: nodnic: Implement data path" adds
mlx5_nodnic_deactivate_queues(), but it only disables the port if the VSC
acquire succeeds:
if (!mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx)) {
mlx5_nodnic_disable_port(dev);
mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
}
mlx5_nodnic_deactivate_sq(&dev->priv->sq);
napi_disable_locked(&dev->priv->cq.napi);
mlx5_nodnic_deactivate_rq(dev);
The acquire can fail with -EBUSY after VSC_MAX_RETRIES, with -EACCES, or
on a set_space failure. A write inside mlx5_nodnic_disable_port() can
also fail, and that only produces a warning. In any of these cases the RX
pages go back to the page pool and the coherent buffers are freed. At
that point network_en and dma_en are still set and bus mastering is
still on.
The error paths in mlx5_nodnic_init_ring(), mlx5_nodnic_init_cq_ring()
and mlx5_nodnic_init_working_buffer() follow the same pattern. Each one
frees a buffer after its address has already been written to the device.
In mlx5_nodnic_remove_one(), mlx5_nodnic_cleanup_netdev() frees all of
these buffers first. Only after that does mlx5_nodnic_dev_cleanup() set
the interface to DISABLED and pci_disable_device() clear bus mastering.
Should the device side be shut down, or pci_clear_master() called as a
fallback, before these buffers are freed?
[ ... ]
> +static const struct net_device_ops mlx5_nodnic_netdev_ops = {
> + .ndo_open = mlx5_nodnic_open,
> + .ndo_stop = mlx5_nodnic_close,
> +};
[Severity: Low]
This isn't a bug, but this patch registers a netdev with no ndo_start_xmit
and no RX path. netif_tx_disable() in mlx5_nodnic_create_netdev() keeps
the TX queue stopped, so the core never calls the missing callback.
The data path is added later in the series by "net/mlx5: nodnic:
Implement data path". Is it intended that this patch registers an
interface that can't pass traffic yet?
> +
> +void mlx5_nodnic_build_netdev(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + struct mlx5_nodnic_core_dev *core_dev;
> +
> + core_dev = priv->core_dev;
> +
> + SET_NETDEV_DEV(netdev, core_dev->device);
> +
> + netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
> + netdev->request_ops_lock = true;
> +
> + eth_hw_addr_set(netdev, core_dev->mac_address);
[Severity: Low]
Should the MAC from mlx5_nodnic_read_mac_address() be checked with
is_valid_ether_addr() before it is installed here? If the device reports
an all-zero or multicast address, that address is registered as the
interface's unicast address.
For comparison, mlx5e_set_netdev_dev_addr() in mlx5_core falls back to
eth_hw_addr_random() when the FW MAC is zero.
[ ... ]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
> new file mode 100644
> index 0000000000000..a533232bf6ef4
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
> @@ -0,0 +1,864 @@
[ ... ]
> +static const struct pci_device_id mlx5_nodnic_core_pci_table[] = {
> + { PCI_VDEVICE(MELLANOX, 0xc2d6) }, /* BlueField-4 nodnic rshim PF */
> + { }
> +};
> +
> +MODULE_DEVICE_TABLE(pci, mlx5_nodnic_core_pci_table);
> +
> +static struct pci_driver mlx5_nodnic_core_driver = {
> + .name = KBUILD_MODNAME,
> + .id_table = mlx5_nodnic_core_pci_table,
> + .probe = mlx5_nodnic_probe_one,
> + .remove = mlx5_nodnic_remove_one,
> +};
[Severity: Medium]
How is this driver meant to interact with mlx5_core's sync FW reset?
mlx5_core already lists 0xc2d6 in mgt_ifc_device_ids[] in
drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c:
{ PCI_VDEVICE(MELLANOX, 0xc2d6) }, /* BlueField4 MGT interface device ID */
Because of that entry, mlx5_check_dev_ids() lets the reset go ahead with
this function present. mlx5_pci_link_toggle() then saves and locks every
function on the bus, toggles the link and restores config space:
list_for_each_entry(sdev, &bridge_bus->devices, bus_list) {
pci_save_state(sdev);
pci_cfg_access_lock(sdev);
}
This driver has no .err_handler and no reset_prepare/reset_done hooks,
and none are added later in the series. Take a devlink reload with action
fw_activate on the PF as an example. Afterwards, the device-side state
this driver programmed would be gone: nic_interface=NO_DRAM_NIC, the
ring/CQ/DBR/working buffer addresses, and the port and MSI-X enables. The
netdev would stay registered with stale software state.
The commit message doesn't mention this coupling with mlx5_core's reset
flow. Is it expected?
[ ... ]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
> new file mode 100644
> index 0000000000000..6ddbf045a8d48
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
> @@ -0,0 +1,343 @@
[ ... ]
> +static int mlx5_nodnic_vsc_pci_read(struct mlx5_nodnic_vsc_ctx *ctx,
> + unsigned int offset, u32 *val)
> +{
> + return pci_read_config_dword(ctx->pdev, ctx->addr + offset, val);
> +}
> +
> +static int mlx5_nodnic_vsc_pci_write(struct mlx5_nodnic_vsc_ctx *ctx,
> + unsigned int offset, u32 val)
> +{
> + return pci_write_config_dword(ctx->pdev, ctx->addr + offset, val);
> +}
[Severity: High]
Should these convert the return value with pcibios_err_to_errno()?
pci_read_config_dword() and pci_write_config_dword() return positive
PCIBIOS_* codes, for example PCIBIOS_DEVICE_NOT_FOUND once the device is
disconnected. Every caller up the stack only checks if (err) and passes
the value up unchanged, so mlx5_nodnic_probe_one() can end up returning
a positive value:
mlx5_nodnic_probe_one()
mlx5_nodnic_dev_init()
mlx5_nodnic_wait_fw_init()
mlx5_nodnic_vsc_context_acquire()
mlx5_nodnic_vsc_gw_lock()
mlx5_nodnic_vsc_pci_read()
On that path, probe calls kfree(dev) and returns the positive value.
drvdata, set in mlx5_nodnic_pci_init(), still points at dev.
local_pci_probe() treats a positive return as success:
if (rc < 0) {
pci_dev->driver = NULL;
pm_runtime_put_sync(dev);
return rc;
}
...
pci_warn(pci_dev, "Driver probe function unexpectedly returned %d\n",
rc);
return 0;
So the driver stays bound, and the next unbind calls
mlx5_nodnic_remove_one(). It gets the freed dev from pci_get_drvdata(),
releases the IRQ vectors, the iseg mapping, the regions and vsc_ctx a
second time, and then calls kfree(dev) again.
mlx5_nodnic_vsc_gw_lock() checks pci_channel_offline() only at the top of
each loop iteration. Probe can also spend minutes in
mlx5_nodnic_wait_fw_init(). A surprise removal or DPC event in that
window looks like enough to hit this.
ndo_open can return the same kind of value through
mlx5_nodnic_open()->mlx5_nodnic_open_locked()->mlx5_nodnic_open_queues()
->mlx5_nodnic_vsc_context_acquire(). netif_open() only checks for
ret < 0:
ret = __dev_open(dev, extack);
if (ret < 0)
return ret;
rtmsg_ifinfo(RTM_NEWLINK, dev, IFF_UP | IFF_RUNNING, GFP_KERNEL, 0, NULL);
call_netdevice_notifiers(NETDEV_UP, dev);
That would send RTM_NEWLINK and NETDEV_UP for an interface that
__dev_open() left down.
[ ... ]
> +static int mlx5_nodnic_vsc_read(struct mlx5_nodnic_vsc_ctx *ctx,
> + unsigned int address, u32 *data)
> +{
> + int err;
> +
> + /* Write the address with the flag and syndrome bits cleared */
> + address &= ~(MLX5_NODNIC_VSC_ADDR_FLAG | MLX5_NODNIC_VSC_ADDR_SYND);
[Severity: Low]
This isn't a bug, but this masks off bits 31:30 of the address instead of
rejecting it. Neither mlx5_nodnic_vsc_read() nor mlx5_nodnic_vsc_write()
checks the syndrome bit after the flag changes.
Those address bits can only be set if the FW-supplied ctx->offset is
1 GiB or more. mlx5_vsc_gw_read() and mlx5_vsc_gw_write() in mlx5_core
skip the syndrome check as well.
Should an out-of-range address or a reported syndrome be returned as an
error here?
[ ... ]
> @@ -0,0 +1,828 @@
> +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
> +// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
> +
> +#include <linux/bitops.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/etherdevice.h>
> +#include <linux/ethtool.h>
> +#include <linux/interrupt.h>
> +#include <linux/log2.h>
> +#include <linux/pci.h>
> +#include <net/netdev_lock.h>
> +#include <net/net_namespace.h>
> +
> +#include "en.h"
> +#include "nodnic_ifc.h"
> +#include "nodnic.h"
> +#include "nodnic_pci_vsc.h"
> +
> +#define MLX5_NODNIC_CQ_INIT_CMD_SN cpu_to_be32(2 << 28)
> +
> +static int mlx5_nodnic_write_ring_addr(struct mlx5_nodnic_core_dev *dev,
> + dma_addr_t dma_handle, u32 offset_h,
> + u32 offset_l)
> +{
> + __be32 addr_h, addr_l;
> + int err;
> +
> + addr_h = cpu_to_be32(upper_32_bits(dma_handle));
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_h, addr_h);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write ring addr high: %d\n",
> + err);
> + return err;
> + }
> +
> + addr_l = 0;
> + MLX5_SET(nodnic_q_addr_l, &addr_l, q_addr_l,
> + lower_32_bits(dma_handle) >> 12);
> +
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_l, addr_l);
> + if (err)
> + mlx5_nodnic_core_err(dev, "failed to write ring addr low: %d\n",
> + err);
> + return err;
> +}
> +
> +static u32 mlx5_nodnic_calc_ring_size(struct mlx5_nodnic_core_dev *dev,
> + int stride)
> +{
> + u32 ring_size = MLX5_NODNIC_RING_ENTRIES * stride;
> + u32 max_supported = 1U << dev->log_max_ring_size;
> +
> + return min(ring_size, max_supported);
> +}
> +
> +static int mlx5_nodnic_write_ring_size(struct mlx5_nodnic_core_dev *dev,
> + int offset, int size)
> +{
> + __be32 addr_l;
> + int err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &addr_l);
> + if (err) {
> + mlx5_nodnic_core_err(dev, "failed to read ring size: %d\n",
> + err);
> + return err;
> + }
> +
> + MLX5_SET(nodnic_q_addr_l, &addr_l, log_size, ilog2(size));
> +
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset, addr_l);
> + if (err)
> + mlx5_nodnic_core_err(dev, "failed to write ring size: %d\n",
> + err);
> + return err;
> +}
> +
> +static int mlx5_nodnic_read_qn(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_ring *ring, u32 offset)
> +{
> + __be32 qn_dword;
> + int err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &qn_dword);
> + if (err) {
> + mlx5_nodnic_core_err(dev, "failed to read queue number: %d\n",
> + err);
> + return err;
> + }
> +
> + ring->qn = MLX5_GET(nodnic_ring_q_number, &qn_dword, queue_number);
> +
> + return 0;
> +}
> +
> +static int mlx5_nodnic_write_ring_dbr_addr(struct mlx5_nodnic_core_dev *dev,
> + u32 ring_offset)
> +{
> + int off_h = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_H;
> + int off_l = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_L;
> + struct mlx5_nodnic_dbr *dbr = &dev->priv->dbr;
> + __be32 dbr_h, dbr_l;
> + int err;
> +
> + dbr_l = 0;
> + MLX5_SET(nodnic_ring_dbr_addr_l, &dbr_l, db_record_addr_l,
> + (u32)(dbr->dma_handle >> 2));
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_l, dbr_l);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write ring DBR addr low: %d\n",
> + err);
> + return err;
> + }
> +
> + dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_h, dbr_h);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write ring DBR addr high: %d\n",
> + err);
> + return err;
> + }
> +
> + return 0;
> +}
> +
> +static int mlx5_nodnic_alloc_ring(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_ring *ring, int size)
> +{
> + dma_addr_t dma_handle;
> + void *buf;
> +
> + buf = dma_alloc_coherent(dev->device, size, &dma_handle, GFP_KERNEL);
> + if (!buf)
> + return -ENOMEM;
> +
> + *ring = (struct mlx5_nodnic_ring) {
> + .wq = buf,
> + .dma_handle = dma_handle,
> + .size = size,
> + };
> +
> + return 0;
> +}
> +
> +static void mlx5_nodnic_free_ring(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_ring *ring)
> +{
> + dma_free_coherent(dev->device, ring->size, ring->wq,
> + ring->dma_handle);
> + memset(ring, 0, sizeof(*ring));
> +}
> +
> +static int mlx5_nodnic_init_ring(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_ring *ring, u32 ring_offset,
> + int stride)
> +{
> + int ring_addr_h = ring_offset + MLX5_NODNIC_RING_ADDR_H;
> + int ring_addr_l = ring_offset + MLX5_NODNIC_RING_ADDR_L;
> + int size, err;
> +
> + size = mlx5_nodnic_calc_ring_size(dev, stride);
> +
> + err = mlx5_nodnic_alloc_ring(dev, ring, size);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle, ring_addr_h,
> + ring_addr_l);
> + if (err)
> + goto err_ring_free;
> +
> + err = mlx5_nodnic_write_ring_size(dev, ring_addr_l, size);
> + if (err)
> + goto err_ring_free;
> +
> + ring->num_entries = size / stride;
> +
> + err = mlx5_nodnic_read_qn(dev, ring,
> + ring_offset + MLX5_NODNIC_RING_QUEUE_NUMBER);
> + if (err)
> + goto err_ring_free;
> +
> + err = mlx5_nodnic_write_ring_dbr_addr(dev, ring_offset);
> + if (err)
> + goto err_ring_free;
> +
> + return 0;
> +
> +err_ring_free:
> + mlx5_nodnic_free_ring(dev, ring);
> + return err;
> +}
> +
> +static int mlx5_nodnic_alloc_dbr(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_dbr *dbr)
> +{
> + const u32 bytes = MLX5_NODNIC_DBR_WORDS * sizeof(__be32);
> + dma_addr_t dma_handle;
> + void *buf;
> +
> + buf = dma_alloc_coherent(dev->device, bytes, &dma_handle, GFP_KERNEL);
> + if (!buf)
> + return -ENOMEM;
> +
> + *dbr = (struct mlx5_nodnic_dbr) {
> + .db = (__be32 *)buf,
> + .alloc_size = bytes,
> + .dma_handle = dma_handle,
> + };
> +
> + return 0;
> +}
> +
> +static void mlx5_nodnic_free_dbr(struct mlx5_nodnic_core_dev *dev,
> + struct mlx5_nodnic_dbr *dbr)
> +{
> + if (!dbr->db)
> + return;
> +
> + dma_free_coherent(dev->device, dbr->alloc_size, dbr->db,
> + dbr->dma_handle);
> +}
> +
> +static int mlx5_nodnic_init_cq_dbr(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_dbr *dbr = &dev->priv->cq.dbr;
> + __be32 dbr_h, dbr_l;
> + int err;
> +
> + err = mlx5_nodnic_alloc_dbr(dev, dbr);
> + if (err)
> + return err;
> +
> + dbr_l = cpu_to_be32(lower_32_bits(dbr->dma_handle));
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
> + MLX5_NODNIC_CQ_DBR_ADDR_L,
> + dbr_l);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write CQ DBR addr low: %d\n",
> + err);
> + goto err_free_dbr;
> + }
> +
> + dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
> + err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
> + MLX5_NODNIC_CQ_DBR_ADDR_H,
> + dbr_h);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write CQ DBR addr high: %d\n",
> + err);
> + goto err_free_dbr;
> + }
> +
> + dev->priv->cq.dbr.db[1] = MLX5_NODNIC_CQ_INIT_CMD_SN;
> +
> + return 0;
> +
> +err_free_dbr:
> + mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
> + return err;
> +}
> +
> +static void mlx5_nodnic_cleanup_cq_dbr(struct mlx5_nodnic_core_dev *dev)
> +{
> + mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
> +}
> +
> +/* log_cq_size should be read after rq and sq ring sizes were set. */
> +static int mlx5_nodnic_read_cq_ring_size(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_cq *cq = &dev->priv->cq;
> + __be32 cq_addr_l;
> + u32 log_cq_size;
> + int err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
> + MLX5_NODNIC_PORT_CQ_ADDR_L,
> + &cq_addr_l);
> + if (err) {
> + mlx5_nodnic_core_err(dev, "failed to read CQ ring size: %d\n",
> + err);
> + return err;
> + }
> +
> + log_cq_size = MLX5_GET(nodnic_q_addr_l, &cq_addr_l, log_size);
> + cq->num_entries = 1ULL << log_cq_size;
> + cq->log_num_entries = log_cq_size;
> +
> + return 0;
> +}
> +
> +static int mlx5_nodnic_init_cq_ring(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_cq *cq = &dev->priv->cq;
> + struct mlx5_nodnic_ring *ring;
> + int err;
> +
> + ring = &cq->ring;
> +
> + err = mlx5_nodnic_read_cq_ring_size(dev);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_alloc_ring(dev, ring,
> + cq->num_entries * MLX5_NODNIC_CQE_SIZE_B);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle,
[Severity: High]
Does this leave the CQ log_size field set to zero?
mlx5_nodnic_read_cq_ring_size() reads log_size out of
MLX5_NODNIC_PORT_CQ_ADDR_L and sizes the CQ from it. Right after that,
mlx5_nodnic_write_ring_addr() starts from addr_l = 0, sets only q_addr_l
and writes the whole dword back to the same register. That clears the
log_size bits that were just read.
For the SQ and RQ, mlx5_nodnic_init_ring() follows the address write
with mlx5_nodnic_write_ring_size(), which does a read-modify-write and
puts log_size back. mlx5_nodnic_init_cq_ring() has no equivalent step,
and nothing else in mlx5_nodnic_open_cq() writes the CQ size.
FW might stop looking at log_size in this register once it has set it.
If it doesn't, the device would see a single-entry CQ while
cq->num_entries in the driver says otherwise. Completions would then
keep landing on the first CQE.
Should mlx5_nodnic_write_ring_addr() preserve the other bits of the low
dword with a read-modify-write? Or should the CQ path rewrite log_size
after the address, the way the SQ/RQ path does?
> + MLX5_NODNIC_PORT_CQ_ADDR_H,
> + MLX5_NODNIC_PORT_CQ_ADDR_L);
> + if (err)
> + goto err_free_cq;
> +
> + return 0;
> +
> +err_free_cq:
> + mlx5_nodnic_free_ring(dev, &cq->ring);
> + return err;
> +}
> +
> +static void mlx5_nodnic_cleanup_cq_ring(struct mlx5_nodnic_core_dev *dev)
> +{
> + return mlx5_nodnic_free_ring(dev, &dev->priv->cq.ring);
> +}
> +
> +static int mlx5_nodnic_read_cq_n(struct mlx5_nodnic_core_dev *dev)
> +{
> + __be32 cq_n;
> + int err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, MLX5_NODNIC_CQ_N,
> + &cq_n);
> + if (err) {
> + mlx5_nodnic_core_err(dev, "failed to read CQ number: %d\n",
> + err);
> + return err;
> + }
> +
> + dev->priv->cq.cqn = MLX5_GET(nodnic_cqn, &cq_n, cq_n);
> +
> + return 0;
> +}
> +
> +int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_ring *work_buf = &dev->priv->working_buffer;
> + u32 size = 1U << dev->log_working_buffer_size;
> + int err;
> +
> + err = mlx5_nodnic_alloc_ring(dev, work_buf, size);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_write_ring_addr(dev, work_buf->dma_handle,
> + MLX5_NODNIC_PORT_WORK_BUF_ADDR_H,
> + MLX5_NODNIC_PORT_WORK_BUF_ADDR_L);
> + if (err)
> + goto err_free_working_buffer;
> +
> + return 0;
> +
> +err_free_working_buffer:
> + mlx5_nodnic_free_ring(dev, work_buf);
> + return err;
> +}
> +
> +int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_cq *cq = &dev->priv->cq;
> + int err;
> +
> + cq->cc = 0;
> +
> + err = mlx5_nodnic_init_cq_ring(dev);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_init_cq_dbr(dev);
> + if (err)
> + goto err_cleanup_ring;
> +
> + err = mlx5_nodnic_read_cq_n(dev);
> + if (err)
> + goto err_cleanup_dbr;
> +
> + return 0;
> +
> +err_cleanup_dbr:
> + mlx5_nodnic_cleanup_cq_dbr(dev);
> +err_cleanup_ring:
> + mlx5_nodnic_cleanup_cq_ring(dev);
> + return err;
> +}
> +
> +static void mlx5_nodnic_close_cq(struct mlx5_nodnic_cq *cq)
> +{
> + struct mlx5_nodnic_core_dev *dev =
> + container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
> +
> + mlx5_nodnic_cleanup_cq_dbr(dev);
> + mlx5_nodnic_cleanup_cq_ring(dev);
> +}
> +
> +int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_priv *priv = dev->priv;
> + struct mlx5_nodnic_sq *sq = &priv->sq;
> + struct mlx5_nodnic_ring *ring;
> + int err;
> +
> + sq->netdev = priv->netdev;
> + sq->device = dev->device;
> + sq->pc = 0;
> + sq->cc = 0;
> + ring = &sq->ring;
> +
> + err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_SEND_RING_OFFSET,
> + MLX5_NODNIC_SQ_WQEBB_B);
> + if (err)
> + return err;
> +
> + ring->lkey = dev->lkey;
> +
> + return 0;
> +}
> +
> +static void mlx5_nodnic_close_sq(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_sq *sq = &dev->priv->sq;
> +
> + mlx5_nodnic_free_ring(dev, &sq->ring);
> +}
> +
> +int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_rq *rq = &dev->priv->rq;
> + struct mlx5_nodnic_ring *ring;
> + int err;
> +
> + rq->pc = 0;
> + rq->cc = 0;
> + ring = &rq->ring;
> +
> + err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_RECEIVE_RING_OFFSET,
> + MLX5_NODNIC_RQ_STRIDE_BYTES);
> + if (err)
> + return err;
> +
> + ring->lkey = dev->lkey;
> +
> + return 0;
> +}
> +
> +static void mlx5_nodnic_close_rq(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_rq *rq = &dev->priv->rq;
> +
> + mlx5_nodnic_free_ring(dev, &rq->ring);
> +}
> +
> +static int mlx5_nodnic_open_queues(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
> + int err;
> +
> + /* one doorbell record shared by both SQ and RQ */
> + err = mlx5_nodnic_alloc_dbr(dev, &dev->priv->dbr);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
> + if (err)
> + goto err_free_dbr;
> +
> + err = mlx5_nodnic_open_sq(dev);
> + if (err)
> + goto err_release_vsc;
> +
> + err = mlx5_nodnic_open_rq(dev);
> + if (err)
> + goto err_close_sq;
> +
> + err = mlx5_nodnic_open_cq(dev);
> + if (err)
> + goto err_close_rq;
> +
> + err = mlx5_nodnic_init_working_buffer(dev);
> + if (err)
> + goto err_close_cq;
> +
> + mlx5_nodnic_vsc_context_release(vsc_ctx);
> +
> + return 0;
> +
> +err_close_cq:
> + mlx5_nodnic_close_cq(&dev->priv->cq);
> +err_close_rq:
> + mlx5_nodnic_close_rq(dev);
> +err_close_sq:
> + mlx5_nodnic_close_sq(dev);
> +err_release_vsc:
> + mlx5_nodnic_vsc_context_release(vsc_ctx);
> +err_free_dbr:
> + mlx5_nodnic_free_dbr(dev, &dev->priv->dbr);
> +
> + return err;
> +}
> +
> +static void mlx5_nodnic_close_queues(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_priv *priv = dev->priv;
> +
> + mlx5_nodnic_free_ring(dev, &priv->working_buffer);
> + mlx5_nodnic_close_cq(&priv->cq);
> + mlx5_nodnic_close_rq(dev);
> + mlx5_nodnic_close_sq(dev);
> + mlx5_nodnic_free_dbr(dev, &priv->dbr);
> +}
> +
> +static void mlx5_nodnic_update_carrier(struct mlx5_nodnic_priv *priv,
> + __be32 event)
> +{
> + u32 port_state = MLX5_GET(nodnic_port_event, &event, port_state);
> + bool up = port_state == MLX5_NODNIC_PORT_STATE_ACTIVE;
> +
> + /* Force re-notification if state is unchanged */
> + if (up == netif_carrier_ok(priv->netdev)) {
> + netif_carrier_event(priv->netdev);
> + return;
> + }
> + if (up) {
> + netif_carrier_on(priv->netdev);
> + netdev_info(priv->netdev, "Link up\n");
> + } else {
> + netif_carrier_off(priv->netdev);
> + netdev_info(priv->netdev, "Link down\n");
> + }
> +}
> +
> +static int mlx5_nodnic_refresh_carrier_locked(struct mlx5_nodnic_priv *priv)
> +{
> + struct mlx5_nodnic_core_dev *dev = priv->core_dev;
> + struct mlx5_nodnic_vsc_ctx *vsc_ctx;
> + __be32 event;
> + int err;
> +
> + vsc_ctx = dev->vsc_ctx;
> + err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_EVENT,
> + &event);
> + mlx5_nodnic_vsc_context_release(vsc_ctx);
> +
> + if (!err)
> + mlx5_nodnic_update_carrier(priv, event);
> +
> + return err;
> +}
> +
> +void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
> +{
> + mutex_lock(&priv->state_lock);
> + if (test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state))
> + mlx5_nodnic_refresh_carrier_locked(priv);
> + mutex_unlock(&priv->state_lock);
> +}
> +
> +int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
> + __be32 val = 0;
> + int err;
> +
> + /* data_msix_en, event_msix_en must be set before activating port */
> + MLX5_SET(nodnic_port_network, &val, data_msix_en, 1);
> + MLX5_SET(nodnic_port_network, &val, event_msix_en, 1);
> + err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + val);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write port network (msix): %d\n",
> + err);
> + return err;
> + }
> +
> + MLX5_SET(nodnic_port_network, &val, network_en, 1);
> + MLX5_SET(nodnic_port_network, &val, receive_filter_en, BIT(0));
> +
> + err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + val);
> + if (err) {
> + mlx5_nodnic_core_err(dev,
> + "failed to write port network (enable): %d\n",
> + err);
> + return err;
> + }
> +
> + /* dma enabling requires a separate write */
> + MLX5_SET(nodnic_port_network, &val, dma_en, 1);
> +
> + err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + val);
> + if (err)
> + mlx5_nodnic_core_err(dev,
> + "failed to write port network (dma): %d\n",
> + err);
> + return err;
> +}
> +
> +void mlx5_nodnic_disable_port(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
> + __be32 val;
> + int err;
> +
> + err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + &val);
> + if (err)
> + goto out;
> +
> + MLX5_SET(nodnic_port_network, &val, dma_en, 0);
> + MLX5_SET(nodnic_port_network, &val, network_en, 0);
> + err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + val);
> + if (err)
> + goto out;
> +
> + /* data_msix_en, event_msix_en can't be modified while port is active */
> + MLX5_SET(nodnic_port_network, &val, data_msix_en, 0);
> + MLX5_SET(nodnic_port_network, &val, event_msix_en, 0);
> + err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
> + val);
> +
> +out:
> + if (err)
> + mlx5_nodnic_core_warn(dev,
> + "failed to read/write to vsc err %d\n",
> + err);
> +}
> +
> +static int mlx5_nodnic_init_uar_map(struct mlx5_nodnic_core_dev *dev)
> +{
> + void __iomem *base;
> + phys_addr_t paddr;
> + u32 uar_page_size;
> +
> + uar_page_size = 1 << dev->log_uar_page_size;
> +
> + paddr = dev->bar_addr + (dev->send_ring0_uar_index * uar_page_size);
[Severity: High]
Can this multiplication wrap?
dev->send_ring0_uar_index and uar_page_size are both u32, so the product
is computed in 32 bits and only widened to phys_addr_t for the addition
to dev->bar_addr. With a 4 KiB UAR page, any index of 0x100000 or more
wraps. ioremap() then maps a page at the wrong offset in BAR 0, possibly
overlapping the init segment or other device registers.
dev->uar_base is meant to be the TX doorbell page; probe insists on
support_uar_tx_doorbell. With a wrapped offset, doorbell writes would go
to whatever register happens to sit at that offset.
Should the product be computed as
(phys_addr_t)dev->send_ring0_uar_index * uar_page_size? Both values come
from FW, so it also seems worth checking that the mapped range fits
inside pci_resource_len(pdev, 0) before calling ioremap().
> +
> + base = ioremap(paddr, uar_page_size);
> + if (!base) {
> + mlx5_nodnic_core_err(dev, "failed to map UAR\n");
> + return -ENOMEM;
> + }
> +
> + dev->uar_base = base;
> +
> + return 0;
> +}
> +
> +static void mlx5_nodnic_cleanup_uar_map(struct mlx5_nodnic_core_dev *dev)
> +{
> + if (!dev->uar_base)
> + return;
> +
> + iounmap(dev->uar_base);
> + dev->uar_base = NULL;
> +}
> +
> +static int mlx5_nodnic_open_locked(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + struct mlx5_nodnic_core_dev *dev = priv->core_dev;
> + int err;
> +
> + err = mlx5_nodnic_init_uar_map(dev);
> + if (err)
> + return err;
> +
> + err = mlx5_nodnic_open_queues(dev);
> + if (err)
> + goto err_uar;
> +
> + set_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
> +
> + err = mlx5_nodnic_refresh_carrier_locked(dev->priv);
> + if (err)
> + netdev_err(dev->priv->netdev,
> + "failed to query carrier, err=%d\n", err);
> +
> + return 0;
> +
> +err_uar:
> + mlx5_nodnic_cleanup_uar_map(dev);
> + return err;
> +}
> +
> +int mlx5_nodnic_open(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + int err;
> +
> + mutex_lock(&priv->state_lock);
> + err = mlx5_nodnic_open_locked(netdev);
> + mutex_unlock(&priv->state_lock);
> +
> + return err;
> +}
> +
> +static int mlx5_nodnic_close_locked(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + struct mlx5_nodnic_core_dev *dev = priv->core_dev;
> +
> + clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
> + netif_carrier_off(priv->netdev);
> + netdev_info(priv->netdev, "Link down\n");
> + mlx5_nodnic_close_queues(dev);
> +
> + return 0;
> +}
> +
> +int mlx5_nodnic_close(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + int err = 0;
> +
> + mutex_lock(&priv->state_lock);
> + if (!test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state)) {
> + mutex_unlock(&priv->state_lock);
> + return 0;
> + }
> +
> + err = mlx5_nodnic_close_locked(netdev);
> + mutex_unlock(&priv->state_lock);
> +
> + mlx5_nodnic_cleanup_uar_map(priv->core_dev);
> +
> + return err;
> +}
> +
> +static const struct net_device_ops mlx5_nodnic_netdev_ops = {
> + .ndo_open = mlx5_nodnic_open,
> + .ndo_stop = mlx5_nodnic_close,
> +};
> +
> +void mlx5_nodnic_build_netdev(struct net_device *netdev)
> +{
> + struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
> + struct mlx5_nodnic_core_dev *core_dev;
> +
> + core_dev = priv->core_dev;
> +
> + SET_NETDEV_DEV(netdev, core_dev->device);
> +
> + netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
> + netdev->request_ops_lock = true;
> +
> + eth_hw_addr_set(netdev, core_dev->mac_address);
> +}
> +
> +static void mlx5_nodnic_init_priv(struct mlx5_nodnic_priv *priv,
> + struct net_device *netdev,
> + struct mlx5_nodnic_core_dev *core_dev)
> +{
> + priv->netdev = netdev;
> + priv->core_dev = core_dev;
> + mutex_init(&priv->state_lock);
> +}
> +
> +static void mlx5_nodnic_cleanup_priv(struct mlx5_nodnic_priv *priv)
> +{
> + mutex_destroy(&priv->state_lock);
> +}
> +
> +struct net_device *mlx5_nodnic_create_netdev(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_priv *priv;
> + struct net_device *netdev;
> +
> + netdev = alloc_etherdev_mqs(sizeof(struct mlx5_nodnic_priv), 1, 1);
> + if (!netdev)
> + return NULL;
> +
> + priv = netdev_priv(netdev);
> + mlx5_nodnic_init_priv(priv, netdev, dev);
> + dev->priv = priv;
> +
> + netif_carrier_off(netdev);
> + netif_tx_disable(netdev);
> + dev_net_set(netdev, &init_net);
> +
> + return netdev;
> +}
> +
> +void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_priv *priv;
> + struct net_device *netdev;
> +
> + if (!dev->priv)
> + return;
> +
> + priv = dev->priv;
> + netdev = priv->netdev;
> + if (!netdev)
> + return;
> +
> + if (dev->netdev_registered) {
> + netif_device_detach(netdev);
> + unregister_netdev(netdev);
> + dev->netdev_registered = false;
> + }
> +
> + mlx5_nodnic_cleanup_priv(priv);
> + dev->priv = NULL;
> +
> + free_netdev(netdev);
> +}
[Severity: Critical]
Is the log_size written here meant to be in bytes or in entries?
mlx5_nodnic_calc_ring_size() returns a byte count,
MLX5_NODNIC_RING_ENTRIES * stride. That is 1024 * 64 = 64 KiB for the
SQ and 1024 * 16 = 16 KiB for the RQ. mlx5_nodnic_write_ring_size() then
programs ilog2() of that byte count into the log_size field of
nodnic_q_addr_l.
The CQ path reads the same log_size field of nodnic_q_addr_l and treats
it as an entry count. mlx5_nodnic_read_cq_ring_size() sets
cq->num_entries to 1 << log_cq_size, and mlx5_nodnic_init_cq_ring()
allocates num_entries * MLX5_NODNIC_CQE_SIZE_B bytes.
If the SQ/RQ field has the same meaning, the device is told the SQ has
65536 WQEBBs and the RQ has 16384 entries. Only 1024 of each were
allocated with dma_alloc_coherent(). Once the device's view of the ring
goes past the allocation, it would read or write beyond the end of the
coherent buffer.
The clamp in mlx5_nodnic_calc_ring_size() has the same problem. It
compares the byte count against 1U << dev->log_max_ring_size, so if
log_max_ring_size is in entries, the cap is off by a factor of stride.
log_max_ring_size is also a 6-bit FW field, so a value of 32 or more
makes the shift undefined.
Could you confirm which units the HW spec uses for log_size and
log_max_ring_size, and make the SQ, RQ and CQ paths agree on them?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261008171821.660739-1-tariqt%40nvidia.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path
2026-10-08 17:18 ` [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
@ 2026-10-12 17:19 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-10-12 17:19 UTC (permalink / raw)
To: tariqt
Cc: andrew+netdev, davem, edumazet, kuba, netdev, pabeni, cjubran,
gal, linux-kernel, shshitrit, leon, mbloch, saeedm, dtatulea,
horms
Thank you for your contribution! Sashiko AI review found 9 potential
issue(s) to consider.
Critical: 0 · High: 3 · Medium: 6 · Low: 0
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [High] mlx5_nodnic_activate_queues() (en_cfg.c) calls
napi_enable_locked() first.
- [High] mlx5_nodnic_deactivate_queues() (en_cfg.c, added in this patch)
calls mlx5_nodnic_disable_port() only if…
- [High] TX stop/wake between mlx5_nodnic_xmit() and
mlx5_nodnic_complete_send_wqe() uses plain, unannotated sq->cc/sq->pc…
- [Medium] mlx5_nodnic_handle_err_cqe()
(drivers/net/ethernet/mellanox/mlx5/nodnic/en.c) handles…
- [Medium] mlx5_nodnic_napi_poll() (en.c) does not handle budget == 0.
- [Medium] In mlx5_nodnic_napi_poll() (en.c), when '++tx_work_done >=
MLX5_NODNIC_TX_CQ_POLL_BUDGET' (128) the loop jumps to poll_done while…
- [Medium] The CQ consumer-index doorbell record (cq->dbr.db[0], set_ci)
is written only in mlx5_nodnic_arm_cq(), which runs only after…
- [Medium] mlx5_nodnic_handle_recv_cqe() (rx.c) passes the device-
reported byte_cnt to page_pool_dma_sync_for_cpu() and skb_put()…
- [Medium] mlx5_nodnic_enable_port() makes three separate PORT_NETWORK
writes: data/event MSI-X enable, then network_en plus…
--
net/mlx5: nodnic: Implement data path
This commit implements the TX and RX data paths for the mlx5_nodnic driver.
It adds WQE posting with a BlueFlame doorbell, page pool backed RX, and a
single NAPI context that dispatches TX and RX CQEs. It also wires up
ndo_start_xmit and ndo_features_check and enables TX/RX checksum offload.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
> new file mode 100644
> index 0000000000000..3572aff50194f
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
[ ... ]
> +static void mlx5_nodnic_handle_err_cqe(struct mlx5_nodnic_cq *cq,
> + struct mlx5_cqe64 *cqe64,
> + int napi_budget)
> +{
[ ... ]
> + netdev_err(priv->netdev,
> + "%s CQE error: syndrome=0x%02x (%s), vendor_syndrome=0x%02x, wqe_counter=%u, cq_ci=%d\n",
> + queue_type,
> + syndrome, mlx5_nodnic_cqe_err_syndrome_str(syndrome),
> + vendor_syndrome, wqe_counter, cq->cc);
> +
> + print_hex_dump(KERN_ERR, "mlx5_nodnic err CQE: ", DUMP_PREFIX_OFFSET,
> + 16, 1, err_cqe, sizeof(*err_cqe), false);
> +
> + if (opcode == OPCODE_CQ_SEND_ERR) {
> + struct mlx5_nodnic_sq *sq = &priv->sq;
> + u16 sq_pi = mlx5_nodnic_sq_get_pi(sq, wqe_counter);
> +
> + mlx5_nodnic_complete_send_wqe(sq, &sq->wqe_info[sq_pi],
> + napi_budget);
> + } else {
> + struct mlx5_nodnic_rq *rq = &priv->rq;
> + u16 ci = wqe_counter & (rq->ring.num_entries - 1);
> +
> + mlx5_nodnic_release_rx_wqe(rq, ci);
> + }
> +}
[Severity: Medium]
Is there a recovery path for the SQ or RQ after an error CQE?
mlx5_nodnic_handle_err_cqe() calls netdev_err() and does a 64 byte
print_hex_dump() at KERN_ERR. Neither is rate limited. It then completes or
releases the one WQE named by wqe_counter and carries on. Nothing stops,
resets or recovers the queue.
The driver decodes MLX5_CQE_SYNDROME_WR_FLUSH_ERR, so it seems to expect
mlx5 flush-after-error behaviour. If so, one non-flush error (for example
LOCAL_LENGTH_ERR) would make every later TX WQE come back as a flush error.
mlx5_nodnic_napi_poll() also reposts every released RX WQE through
mlx5_nodnic_rq_post_wqes(). An errored RQ would flush them straight back.
Could one hardware error then become a flood of KERN_ERR output from
softirq context? TX and RX would stay dead until the interface is bounced,
while the queue and carrier still look up.
[ ... ]
> +int mlx5_nodnic_napi_poll(struct napi_struct *napi, int budget)
> +{
[ ... ]
> + while (true) {
> + cqe = mlx5_nodnic_cq_get_cqe(cq);
> + if (!cqe)
> + break;
> +
> + result = mlx5_nodnic_handle_cqe(cq, cqe, budget);
> + switch (result) {
> + case MLX5_NODNIC_CQE_TX:
> + if (++tx_work_done >= MLX5_NODNIC_TX_CQ_POLL_BUDGET)
> + goto poll_done;
> + break;
> + case MLX5_NODNIC_CQE_RX:
> + if (++rx_work_done >= budget)
> + goto poll_done;
> + break;
[Severity: Medium]
Does this poll handle budget == 0? netpoll calls it with a zero budget:
net/core/netpoll.c:poll_one_napi() {
...
work = napi->poll(napi, 0);
...
}
The next CQE is fully handled before the budget is checked.
For an RX CQE, mlx5_nodnic_handle_recv_cqe() calls
page_pool_dma_sync_for_cpu(), napi_build_skb() and napi_gro_receive(). For
an RX error CQE, mlx5_nodnic_release_rx_wqe() calls
page_pool_put_full_page() with allow_direct set to true.
Only after that does ++rx_work_done >= budget exit the loop. poll_done then
always calls mlx5_nodnic_rq_post_wqes(), which allocates from the page pool.
Documentation/networking/napi.rst says that with a zero budget only TX
completions should be processed, and page pool and RX APIs must not be
used.
With netconsole on this netdev, netpoll can run from hardirq or printk
context. Can this path then touch the page pool's lockless cache, the
per-CPU NAPI skb cache and GRO? Could those get corrupted if it interrupts
a softirq user on the same CPU?
> + case MLX5_NODNIC_CQE_IGNORED:
> + break;
> + }
> + }
> +
> +poll_done:
> + if (!mlx5_nodnic_rq_post_wqes(priv))
> + return budget;
> +
> + if (rx_work_done >= budget)
> + return budget;
[Severity: Medium]
On both return budget paths, does the device ever see an updated CQ
consumer index?
The CQ doorbell record consumer index is only written in
mlx5_nodnic_arm_cq():
cq->dbr.db[0] = cpu_to_be32(ci);
That only runs after napi_complete_done() succeeds.
Under sustained RX that fills the budget on every pass, NAPI never
completes. cq->cc keeps advancing in software, but db[0] keeps the value
from the last arm.
Meanwhile mlx5_nodnic_rq_post_wqes() keeps reposting RX WQEs based on the
software rq->cc. mlx5_nodnic_complete_send_wqe() keeps freeing SQ slots
and waking TX.
Suppose the device uses set_ci for CQ overrun detection like other mlx5
devices. Could it then see more outstanding CQEs than the CQ size and
report an overrun or stall, even though software has consumed every entry?
For comparison, mlx5e updates the CQ doorbell record with
mlx5_cqwq_update_db_record() at the end of every poll pass.
> +
> + if (napi_complete_done(napi, rx_work_done))
> + mlx5_nodnic_arm_cq(cq);
> +
> + return rx_work_done;
> +}
[Severity: Medium]
When the loop exits because tx_work_done reached
MLX5_NODNIC_TX_CQ_POLL_BUDGET, more CQEs may still be pending:
case MLX5_NODNIC_CQE_TX:
if (++tx_work_done >= MLX5_NODNIC_TX_CQ_POLL_BUDGET)
goto poll_done;
Suppose the RQ refill succeeds and rx_work_done < budget. This path then
calls napi_complete_done(), re-arms the CQ and returns less than budget,
even though polling stopped with work left.
Should this return budget instead, as mlx5e does when its TX poll hits its
limit?
As written, the remaining TX completions, and any RX CQEs behind them,
depend on the device raising an event when armed with unconsumed CQEs. If
it only signals on new CQE writes, TX completions could stall with the SQ
stopped. There is no ndo_tx_timeout to recover.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
> index 5724ed41d8b89..9657c43e9958f 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
[ ... ]
> @@ -573,6 +643,143 @@ void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
[ ... ]
> +static int mlx5_nodnic_activate_queues(struct mlx5_nodnic_core_dev *dev)
> +{
> + struct mlx5_nodnic_cq *cq = &dev->priv->cq;
> + int err;
> +
> + napi_enable_locked(&cq->napi);
> +
> + mlx5_nodnic_init_cqes_ownership(cq);
> +
> + err = mlx5_nodnic_activate_rq(dev);
> + if (err)
> + goto err_napi;
> +
> + err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
> + if (err)
> + goto err_rq;
> +
> + err = mlx5_nodnic_enable_port(dev);
> + mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
> + if (err)
> + goto err_rq;
[Severity: Medium]
If mlx5_nodnic_enable_port() fails after its first or second write, is the
port left partly enabled?
mlx5_nodnic_enable_port() makes three separate PORT_NETWORK writes: MSI-X
enable, then network_en plus receive_filter_en, then dma_en. It returns
early on failure and does not roll back.
This unwind goes to err_rq without calling mlx5_nodnic_disable_port().
mlx5_nodnic_open_locked() then reaches:
err_close:
clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
mlx5_nodnic_close_queues(dev);
That frees the rings, DBRs and working buffer, but their addresses are
still programmed in the device.
The dma_en write might take effect but still be reported as failed, for
example on a VSC flag-wait timeout. Could the device then DMA into freed
memory?
mlx5_nodnic_disable_port() also notes that data_msix_en and event_msix_en
can't be modified while the port is active. Could the leftover state make
the first enable_port write on the next open ineffective?
> +
> + mlx5_nodnic_arm_cq(cq);
> + napi_schedule(&cq->napi);
> +
> + mlx5_nodnic_activate_sq(dev);
> +
> + return 0;
> +
> +err_rq:
> + mlx5_nodnic_deactivate_rq(dev);
> +err_napi:
> + napi_disable_locked(&cq->napi);
> +
> + return err;
> +}
[Severity: High]
Is this unwind order safe?
mlx5_nodnic_activate_rq() creates the page pool with
.napi = &priv->cq.napi. On the err_rq path, mlx5_nodnic_deactivate_rq()
frees rq->pages and calls page_pool_destroy() while NAPI is still enabled.
Only after that does err_napi call napi_disable_locked(). The
err_destroy_page_pool path inside mlx5_nodnic_activate_rq() has the same
problem.
page_pool_destroy() reaches this check through
page_pool_disable_direct_recycling()->napi_assert_will_not_race():
WARN_ON(!test_bit(NAPI_STATE_SCHED, &napi->state));
napi_enable_locked() has already cleared that bit. So wouldn't any open
failure after page_pool_create() hit this WARN? That covers RX page
allocation, VSC acquire and enable_port failures. The pool is also torn
down without synchronizing against a running poll.
mlx5_nodnic_deactivate_queues() uses the other order: it calls
napi_disable_locked() before mlx5_nodnic_deactivate_rq().
A related question is why napi_enable_locked() runs first. It comes before
mlx5_nodnic_init_cqes_ownership(), before mlx5_nodnic_activate_rq() sets up
rq->page_pool and rq->pages, and before mlx5_nodnic_activate_sq() sets
sq->txq.
napi_enable_locked()->napi_restore_config() re-hashes the napi_id kept from
the previous session. A busy-polling socket can then call
mlx5_nodnic_napi_poll() straight away, without the netdev or state lock.
On reopen, rq->pages still points at the array freed by
mlx5_nodnic_rq_free_pages() during the previous close:
kvfree(rq->pages);
rq->page_pool still points at the pool destroyed then, and open_rq has
reset rq->pc and rq->cc to 0. Can
mlx5_nodnic_rq_post_wqes()->mlx5_nodnic_rq_post_one() then read
rq->pages[pi] and call page_pool_dev_alloc_pages(rq->page_pool) on freed
memory?
Before the ownership init runs, the zeroed CQ ring has op_own == 0. That
matches software ownership 0 and opcode OPCODE_CQ_SEND. Would a poll in
that window complete a bogus send through wqe_info[0] and push sq->cc past
sq->pc?
> +
> +static void mlx5_nodnic_deactivate_queues(struct mlx5_nodnic_core_dev *dev)
> +{
> + if (!mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx)) {
> + mlx5_nodnic_disable_port(dev);
> + mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
> + }
> +
> + mlx5_nodnic_deactivate_sq(&dev->priv->sq);
> + napi_disable_locked(&dev->priv->cq.napi);
> + mlx5_nodnic_deactivate_rq(dev);
> +}
[Severity: High]
What happens to the device if mlx5_nodnic_vsc_context_acquire() fails here?
Acquire can fail with -EBUSY when the gateway semaphore is still held after
VSC_MAX_RETRIES in mlx5_nodnic_vsc_gw_lock(). It can also fail with
-EACCES, or when setting the space fails. In all of these cases
mlx5_nodnic_disable_port() is skipped. Even when it is called, it returns
void and only warns if its read or write of PORT_NETWORK fails.
Either way teardown continues. mlx5_nodnic_deactivate_rq() returns the
device-owned RX pages to the page pool and destroys it.
mlx5_nodnic_close_queues() then frees the coherent memory:
mlx5_nodnic_free_ring(dev, &priv->working_buffer);
mlx5_nodnic_close_cq(&priv->cq);
mlx5_nodnic_close_rq(dev);
mlx5_nodnic_close_sq(dev);
mlx5_nodnic_free_dbr(dev, &priv->dbr);
At that point the device still has network_en and dma_en set and the ring
and DBR addresses programmed. Its RX WQEs still point at the freed pages.
Can an incoming packet then be DMA-written into pages that are back in the
page allocator? Can the device keep fetching the freed SQ ring and doorbell
records?
Even when the disable write succeeds, nothing reads back or waits for
in-flight DMA to drain.
[ ... ]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
> new file mode 100644
> index 0000000000000..7736a1f24ea5b
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
[ ... ]
> +void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
> + struct mlx5_cqe64 *cqe)
> +{
[ ... ]
> + u32 byte_count = be32_to_cpu(cqe->byte_cnt);
[ ... ]
> + page_pool_dma_sync_for_cpu(rq->page_pool, page, 0, byte_count);
> +
> + skb = napi_build_skb(page_address(page), PAGE_SIZE);
> + if (unlikely(!skb))
> + goto err_recycle;
> +
> + skb_mark_for_recycle(skb);
> + rq->pages[ci] = NULL;
> +
> + skb_reserve(skb, MLX5_NODNIC_SKB_HEADROOM_BYTES);
> + skb_put(skb, byte_count);
[Severity: Medium]
This is mostly a hardening question. byte_count comes straight from the
CQE and is passed to page_pool_dma_sync_for_cpu() and skb_put(). It is
never checked against the posted buffer size (MLX5_NODNIC_HW_MTU) or the
skb tailroom.
A CQE with byte_cnt above roughly PAGE_SIZE - NET_SKB_PAD minus the
skb_shared_info size would hit skb_over_panic()->BUG().
This needs a buggy or misbehaving device. Every RX WQE is posted with
byte_count = MLX5_NODNIC_HW_MTU, mlx5-family hardware normally reports
oversize frames as LOCAL_LENGTH_ERR, and the mlx5e linear RX path doesn't
check this either. Would a cheap bound check here still be worthwhile?
[ ... ]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
> new file mode 100644
> index 0000000000000..f2efc8d3f5b85
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
[ ... ]
> +static int mlx5_nodnic_tx_is_sq_full(struct mlx5_nodnic_sq *sq)
> +{
> + u16 pending = (u16)(sq->pc - sq->cc);
> +
> + return pending == sq->ring.num_entries - MLX5_NODNIC_SQ_STOP_ROOM;
> +}
[ ... ]
> +netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *netdev)
> +{
[ ... ]
> + if (unlikely(mlx5_nodnic_tx_is_sq_full(sq)))
> + netif_tx_stop_queue(sq->txq);
> +
> + next_offset = mlx5_nodnic_tx_get_next_sq_offset(sq);
> +
> + wqe = (struct mlx5_nodnic_tx_wqe *)((u8 *)sq->ring.wq + next_offset);
> +
> + err = mlx5_nodnic_tx_write_wqe(sq, skb, wqe);
> + if (err) {
> + dev_kfree_skb_any(skb);
> +
> + return NETDEV_TX_OK;
> + }
[Severity: High]
Is the stop/wake handshake between mlx5_nodnic_xmit() and
mlx5_nodnic_complete_send_wqe() safe on weakly ordered CPUs?
sq->pc and sq->cc use plain loads and stores with no barriers. xmit does
not re-check after stopping, and the full test only matches
pending == num_entries - 1.
When pending == N - 1, xmit stops the queue and still posts, which leaves
the ring full (pending == N). NAPI then does:
sq->cc++;
if (netif_tx_queue_stopped(txq))
netif_tx_wake_queue(txq);
On the next xmit, nothing orders the load of sq->cc after the observation
that XOFF was cleared. On arm64 or POWER that load can return the old cc.
pending is then N, the == N - 1 check fails, and the queue is not stopped.
That post uses the slot just freed, so the ring is now truly full with the
queue still running.
The xmit after that again sees pending == N and does not stop. It writes
the WQE and wqe_info at pc & (N - 1), the slot of the oldest WQE the
hardware still owns.
Can this corrupt an in-flight descriptor, lose its skb and DMA mapping, and
let pc run past cc + N?
On the completion side, cc++ followed by test_bit() of XOFF has no full
barrier to pair with xmit's load of cc followed by set_bit().
The dma mapping failure path above also returns after netif_tx_stop_queue()
without posting anything. With that path in the mix, could the queue end up
stopped with nothing outstanding to wake it?
netif_txq_maybe_stop() and netif_txq_completed_wake() in
include/net/netdev_queues.h provide the smp_mb(), a >= test and the
re-check. Would they fit here?
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261008171821.660739-1-tariqt%40nvidia.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics
2026-10-08 17:18 ` [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
@ 2026-10-12 17:19 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-10-12 17:19 UTC (permalink / raw)
To: tariqt
Cc: andrew+netdev, davem, edumazet, kuba, netdev, pabeni, cjubran,
gal, linux-kernel, shshitrit, leon, mbloch, saeedm, dtatulea,
horms
Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.
Critical: 0 · High: 0 · Medium: 2 · Low: 1
If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).
- [Medium] mlx5_nodnic_handle_send_cqe() reads wi->num_bytes for the
tx_bytes update (tx.c:182) after mlx5_nodnic_complete_send_wqe() has…
- [Medium] The new 64-bit counters in struct mlx5_nodnic_sq_stats and
struct mlx5_nodnic_rq_stats (en.h:47-56) are plain u64 with no…
- [Low] The new RX packet and byte counters in
mlx5_nodnic_handle_recv_cqe() (rx.c:155-156) are incremented only…
--
net/mlx5: nodnic: Add TX/RX statistics
This commit adds TX and RX packet and byte counters to the nodnic driver,
plus a TX drop counter. The data path updates the counters, and
ndo_get_stats64 exposes them.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
> index 19858e9da064a..d811bfe242ac6 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
> @@ -44,6 +44,17 @@ enum mlx5_nodnic_cqe_opcode {
> OPCODE_CQ_INVALID,
> };
>
> +struct mlx5_nodnic_sq_stats {
> + u64 packets;
> + u64 bytes;
> + u64 dropped;
> +};
> +
> +struct mlx5_nodnic_rq_stats {
> + u64 packets;
> + u64 bytes;
> +};
[Severity: Medium]
Can these counters tear when they are read on a 32-bit kernel?
They are plain u64 fields. There is no u64_stats_sync and no
READ_ONCE()/WRITE_ONCE(). Three places write them with plain ++ and +=:
mlx5_nodnic_handle_recv_cqe() - rq_stats packets/bytes, NAPI
mlx5_nodnic_handle_send_cqe() - sq_stats packets/bytes, NAPI
mlx5_nodnic_xmit() - sq_stats dropped, ndo_start_xmit
mlx5_nodnic_get_stats64() reads them with plain loads from process context
through dev_get_stats(). It shares no lock with the writers.
MLX5_NODNIC in Kconfig does not depend on 64BIT, and
include/linux/mlx5/doorbell.h has a BITS_PER_LONG != 64 fallback, so
32-bit builds look supported. On 32-bit, a u64 add is two 32-bit stores.
A reader such as ip -s link or /proc/net/dev could then see rx_bytes or
tx_bytes jump by about 2^32. On 64-bit these are still unannotated data
races that KCSAN would report.
Should these use u64_stats_t, with u64_stats_update_begin()/end() in the
writers and u64_stats_fetch_begin()/retry() in mlx5_nodnic_get_stats64()?
dropped is written from xmit context, while packets and bytes are written
from NAPI. So the TX side would need separate syncp instances, or separate
handling for dropped, so that each syncp has only one writer.
[ ... ]
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
> index 7736a1f24ea5b..808740ec39102 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
> @@ -152,6 +152,9 @@ void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
> skb->protocol = eth_type_trans(skb, priv->netdev);
> nodnic_handle_rx_csum(priv->netdev, cqe, skb);
>
> + priv->rq_stats.packets++;
> + priv->rq_stats.bytes += byte_count;
> +
[Severity: Low]
Should good frames that are dropped because napi_build_skb() failed also be
counted? Earlier in mlx5_nodnic_handle_recv_cqe():
skb = napi_build_skb(page_address(page), PAGE_SIZE);
if (unlikely(!skb))
goto err_recycle;
This jumps past both counter updates. The rtnl_link_stats64 kernel-doc
says @rx_packets includes "packets which host had to drop at various
stages of processing (even in the driver)", and that @rx_bytes
corresponds to @rx_packets.
Could the counters be incremented before the allocation? The failure
could also be counted in rx_dropped.
> napi_gro_receive(&cq->napi, skb);
> rq->cc++;
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
> index f2efc8d3f5b85..6958c8a51231d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
[ ... ]
> @@ -176,6 +177,9 @@ void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
> wi = &sq->wqe_info[pi];
>
> mlx5_nodnic_complete_send_wqe(sq, wi, napi_budget);
> +
> + priv->sq_stats.packets++;
> + priv->sq_stats.bytes += wi->num_bytes;
[Severity: Medium]
Could wi->num_bytes already belong to a different packet when it is read
here?
mlx5_nodnic_complete_send_wqe() gives the slot back to the producer before
it returns:
sq->cc++;
if (netif_tx_queue_stopped(txq))
netif_tx_wake_queue(txq);
mlx5_nodnic_xmit() stops the queue when the ring is one entry short of
full, but it still posts the current packet:
if (unlikely(mlx5_nodnic_tx_is_sq_full(sq)))
netif_tx_stop_queue(sq->txq);
So the ring can become completely full (pc - cc == num_entries). Once this
completion increments cc and wakes the queue, mlx5_nodnic_xmit() on
another CPU computes pi = pc & mask, which is the slot that was just
completed. That CPU holds only the tx queue lock, which NAPI does not take.
mlx5_nodnic_tx_fill_wqe_info() then does:
sq->wqe_info[pi].num_bytes = skb->len;
If that write lands before the read above, tx_bytes gets the new packet's
length. The new packet is then counted again when it completes.
Could wi->num_bytes be saved in a local variable before calling
mlx5_nodnic_complete_send_wqe()? Another option is to update the stats
before cc is advanced and the queue is woken.
> }
>
> void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261008171821.660739-1-tariqt%40nvidia.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring
2026-10-08 17:18 ` [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
@ 2026-10-12 17:19 ` netdev-bot+sashiko
0 siblings, 0 replies; 10+ messages in thread
From: netdev-bot+sashiko @ 2026-10-12 17:19 UTC (permalink / raw)
To: tariqt
Cc: andrew+netdev, davem, edumazet, kuba, netdev, pabeni, cjubran,
gal, linux-kernel, shshitrit, leon, mbloch, saeedm, dtatulea,
horms
Thank you for your contribution! Sashiko AI review found more severe
issue(s) in other patches of this series. This mail is only an FYI, in case
you want to address the nit picks below when you respin.
Low priority / nit picks for your consideration:
- [Low] The patch removes the trailing newline from
drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile.
--
net/mlx5: nodnic: Add health monitoring
This commit adds firmware health monitoring to the nodnic driver. A timer
periodically polls the health buffer in the initialization segment. It
detects fatal sensors, stalls in the health counter, and syndrome changes.
Polling is started from probe and stopped in remove.
> diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
> index 25865d7244d82..1d09edeaf3af8 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
> +++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
> @@ -5,4 +5,4 @@
>
> obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
>
> -mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o
> +mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o health.o
> \ No newline at end of file
[Severity: Low]
This isn't a bug, but the trailing newline at the end of the nodnic
Makefile has been dropped. Before this patch the mlx5_nodnic-y line ended
with a newline. Could it be put back so that checkpatch doesn't warn and
the next change to this file doesn't show an unrelated whitespace hunk?
[ ... ]
--
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261008171821.660739-1-tariqt%40nvidia.com
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-10-12 17:19 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-08 17:18 [PATCH net-next V2 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
2026-10-08 17:18 ` [PATCH net-next V2 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
2026-10-08 17:18 ` [PATCH net-next V2 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
2026-10-12 17:19 ` netdev-bot+sashiko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®