* [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions
@ 2026-09-30 12:57 Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
Hi,
This series by Shahar introduces mlx5_nodnic, a Linux driver for the
NODNIC (No-DRAM NIC) hardware interface present in Mellanox/NVIDIA
ConnectX-family ASICs since 2015.
NODNIC is a first-class hardware capability.
It defines a lightweight NIC function intended for contexts where local
DRAM is unavailable or too costly: embedded controllers, management
planes, pre-boot environments, or any resource-constrained instantiation
of a ConnectX NIC. The hardware sends and receives ordinary Ethernet
frames via SQs, RQs and CQs. There is no encapsulation, no private
protocol, and no FW interposition: packets are forwarded as standard
Ethernet between whatever peers are connected to the function. The
interface is documented in the HW spec and has been in production in
non-Linux OS drivers for over a decade.
Where it is used today:
- Embedded management controllers on ConnectX-class ASICs that need a
network path to the host without consuming a data-path PF.
- Resource-constrained SoC integrations (no local DRAM) where a
minimal-footprint driver is required.
- Pre-boot / early-OS environments (PXE, iSCSI boot) that need a NIC
without a full mlx5e stack.
- BF4 management PFs (one concrete deployment): on BlueField-4 the
firmware exposes a NODNIC PF on the host side and a peer PF on the ARM
side, using standard vport forwarding to connect them. This provides
in-band Ethernet connectivity between the host and the DPU OS - SSH,
PXE, and any standard networking application work over it without any
special handling.
Design notes:
- The driver is separate from mlx5e rather than a mode within it.
NODNIC uses PCI Vendor Specific Capability (VSC) for command
operations - reads and writes over PCI config space - instead of the
DMA-based command interface used by mlx5_core. This is what makes
NODNIC usable in environments without reliable DMA access, and it
makes the two command paths fundamentally incompatible. Folding this
into mlx5e would mean carrying a second command transport and a second
resource model through code that is already dense.
- A single SQ, RQ and shared CQ are exposed per function, with two MSI-X
vectors: the data vector fires on CQEs and drives both TX and RX
completions, and the event vector notifies on port state changes.
This is the hardware's model, not a driver simplification.
- The driver exposes a plain netdev with no private uAPI, no ioctls and
no vendor-specific configuration surface. Everything is driven through
standard netdev, ethtool and devlink interfaces.
- The code lives under drivers/net/ethernet/mellanox/mlx5/nodnic/ and
shares no paths with mlx5e, keeping the footprint small and the blast
radius of changes contained.
Regards,
Tariq
Shahar Shitrit (5):
net/mlx5: nodnic: Introduce the mlx5_nodnic driver
net/mlx5: nodnic: Implement data path
net/mlx5: nodnic: Add TX/RX statistics
net/mlx5: nodnic: Add ethtool driver info
net/mlx5: nodnic: Add health monitoring
drivers/net/ethernet/mellanox/Kconfig | 1 +
drivers/net/ethernet/mellanox/Makefile | 1 +
.../net/ethernet/mellanox/mlx5/nodnic/Kconfig | 18 +
.../ethernet/mellanox/mlx5/nodnic/Makefile | 8 +
.../net/ethernet/mellanox/mlx5/nodnic/en.c | 263 ++++
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 158 +++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 1107 +++++++++++++++++
.../ethernet/mellanox/mlx5/nodnic/health.c | 315 +++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 876 +++++++++++++
.../ethernet/mellanox/mlx5/nodnic/nodnic.h | 105 ++
.../mellanox/mlx5/nodnic/nodnic_ifc.h | 318 +++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.c | 418 +++++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.h | 45 +
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 172 +++
.../net/ethernet/mellanox/mlx5/nodnic/rx.h | 25 +
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 230 ++++
.../net/ethernet/mellanox/mlx5/nodnic/tx.h | 61 +
17 files changed, 4121 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
base-commit: 47a1446725732cd3996edf607e8739334bbf4d78
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
@ 2026-09-30 12:57 ` Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
From: Shahar Shitrit <shshitrit@nvidia.com>
Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC
functions.
It defines a lightweight NIC function intended for contexts where local
DRAM is unavailable or too costly: embedded controllers, management
planes, pre-boot environments, or any resource-constrained instantiation
of a ConnectX NIC. The hardware sends and receives ordinary Ethernet
frames via SQs, RQs and CQs. There is no encapsulation, no private
protocol, and no FW interposition: packets are forwarded as standard
Ethernet between whatever peers are connected to the function. The
interface is documented in the HW spec and has been in production in
non-Linux OS drivers for over a decade.
This patch implements the PCI probe and remove flows, including FLR,
firmware readiness, interface negotiation and capability discovery,
followed by port and ring configuration.
It also adds the VSC access layer for PCI configuration-space accesses
to the initialization segment and to the NODNIC configuration registers.
Set up the event IRQ for link-state notifications, retrieve the port MAC
address, and implement the netdev lifecycle, including ndo_open and
ndo_stop.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
drivers/net/ethernet/mellanox/Kconfig | 1 +
drivers/net/ethernet/mellanox/Makefile | 1 +
.../net/ethernet/mellanox/mlx5/nodnic/Kconfig | 18 +
.../ethernet/mellanox/mlx5/nodnic/Makefile | 8 +
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 97 ++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 828 +++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 874 ++++++++++++++++++
.../ethernet/mellanox/mlx5/nodnic/nodnic.h | 105 +++
.../mellanox/mlx5/nodnic/nodnic_ifc.h | 318 +++++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.c | 418 +++++++++
.../mellanox/mlx5/nodnic/nodnic_pci_vsc.h | 45 +
11 files changed, 2713 insertions(+)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
diff --git a/drivers/net/ethernet/mellanox/Kconfig b/drivers/net/ethernet/mellanox/Kconfig
index b4f66eb9ddb9..979a029c0171 100644
--- a/drivers/net/ethernet/mellanox/Kconfig
+++ b/drivers/net/ethernet/mellanox/Kconfig
@@ -23,5 +23,6 @@ source "drivers/net/ethernet/mellanox/mlx5/core/Kconfig"
source "drivers/net/ethernet/mellanox/mlxsw/Kconfig"
source "drivers/net/ethernet/mellanox/mlxfw/Kconfig"
source "drivers/net/ethernet/mellanox/mlxbf_gige/Kconfig"
+source "drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig"
endif # NET_VENDOR_MELLANOX
diff --git a/drivers/net/ethernet/mellanox/Makefile b/drivers/net/ethernet/mellanox/Makefile
index d4b5f547a727..ec46837fa729 100644
--- a/drivers/net/ethernet/mellanox/Makefile
+++ b/drivers/net/ethernet/mellanox/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_MLX5_CORE) += mlx5/core/
obj-$(CONFIG_MLXSW_CORE) += mlxsw/
obj-$(CONFIG_MLXFW) += mlxfw/
obj-$(CONFIG_MLXBF_GIGE) += mlxbf_gige/
+obj-$(CONFIG_MLX5_NODNIC) += mlx5/nodnic/
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
new file mode 100644
index 000000000000..461750a9c97c
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Kconfig
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config MLX5_NODNIC
+ tristate "Mellanox NODNIC core driver"
+ depends on PCI
+ depends on NETDEVICES && ETHERNET
+ help
+ This driver provides core support for Mellanox NODNIC (NO DRAM NIC)
+ devices. NODNIC is a lightweight network
+ interface driver that uses Vendor Specific Capability (VSC) for
+ PCI communication with Mellanox network adapters.
+
+ This driver enables basic Ethernet networking functionality for
+ supported Mellanox NODNIC hardware.
+
+ To compile this driver as a module, choose M here: the module
+ will be called mlx5_nodnic.
+
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
new file mode 100644
index 000000000000..4955f0427c79
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for Mellanox NODNIC driver
+#
+
+obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
+
+mlx5_nodnic-y := main.o en_cfg.o nodnic_pci_vsc.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
new file mode 100644
index 000000000000..90dc57695e1b
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_EN_H__
+#define __MLX5_NODNIC_EN_H__
+
+#include <linux/if_vlan.h>
+#include <linux/mlx5/device.h>
+#include <linux/mutex.h>
+#include <linux/netdevice.h>
+#include <linux/timer.h>
+#include <linux/types.h>
+
+#define MLX5_NODNIC_RING_ENTRIES 1024
+#define MLX5_NODNIC_CQE_SIZE_B 64
+#define MLX5_NODNIC_SQ_WQEBB_B 64
+#define MLX5_NODNIC_RQ_STRIDE_BYTES 16
+
+enum {
+ MLX5_NODNIC_STATE_OPENED,
+};
+
+enum {
+ MLX5_NODNIC_RCV_DBR,
+ MLX5_NODNIC_SND_DBR,
+ MLX5_NODNIC_DBR_WORDS,
+};
+
+struct mlx5_nodnic_ring {
+ void *wq;
+ dma_addr_t dma_handle;
+ u32 size;
+ u32 num_entries;
+ u32 qn;
+ u32 lkey;
+};
+
+struct mlx5_nodnic_sq {
+ struct device *device;
+ struct net_device *netdev;
+ struct mlx5_nodnic_ring ring;
+
+ u16 pc;
+ u16 cc;
+};
+
+struct mlx5_nodnic_rq {
+ struct mlx5_nodnic_ring ring;
+ u16 pc;
+ u16 cc;
+};
+
+struct mlx5_nodnic_dbr {
+ __be32 *db;
+ dma_addr_t dma_handle;
+ u32 alloc_size;
+};
+
+struct mlx5_nodnic_cq {
+ struct mlx5_nodnic_ring ring;
+ int cqn;
+ int num_entries;
+ int log_num_entries;
+
+ struct mlx5_nodnic_dbr dbr;
+
+ int cc;
+};
+
+/* for FW use only */
+struct mlx5_nodnic_working_buffer {
+ dma_addr_t dma_handle;
+ u32 alloc_size;
+};
+
+struct mlx5_nodnic_priv {
+ struct mlx5_nodnic_core_dev *core_dev;
+ struct net_device *netdev;
+
+ struct mutex state_lock; /* protects Interface state */
+ unsigned long state;
+
+ struct mlx5_nodnic_sq sq;
+ struct mlx5_nodnic_rq rq;
+ struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
+ struct mlx5_nodnic_cq cq;
+ struct mlx5_nodnic_ring working_buffer;
+};
+
+void mlx5_nodnic_build_netdev(struct net_device *netdev);
+struct net_device *mlx5_nodnic_create_netdev(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv);
+void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open(struct net_device *netdev);
+int mlx5_nodnic_close(struct net_device *netdev);
+
+#endif /* __MLX5_NODNIC_EN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
new file mode 100644
index 000000000000..75dbd2071110
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -0,0 +1,828 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/bitops.h>
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/interrupt.h>
+#include <linux/log2.h>
+#include <linux/pci.h>
+#include <net/netdev_lock.h>
+#include <net/net_namespace.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+#include "nodnic_pci_vsc.h"
+
+#define MLX5_NODNIC_CQ_INIT_CMD_SN cpu_to_be32(2 << 28)
+
+static int mlx5_nodnic_write_ring_addr(struct mlx5_nodnic_core_dev *dev,
+ dma_addr_t dma_handle, u32 offset_h,
+ u32 offset_l)
+{
+ u32 addr_h, addr_l;
+ int err;
+
+ addr_h = cpu_to_be32(upper_32_bits(dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_h, addr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring addr high: %d\n",
+ err);
+ return err;
+ }
+
+ addr_l = 0;
+ MLX5_SET(nodnic_q_addr_l, &addr_l, q_addr_l,
+ lower_32_bits(dma_handle) >> 12);
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset_l, addr_l);
+ if (err)
+ mlx5_nodnic_core_err(dev, "failed to write ring addr low: %d\n",
+ err);
+ return err;
+}
+
+static u32 mlx5_nodnic_calc_ring_size(struct mlx5_nodnic_core_dev *dev,
+ int stride)
+{
+ u32 ring_size = MLX5_NODNIC_RING_ENTRIES * stride;
+ u32 max_supported = 1U << dev->log_max_ring_size;
+
+ return min(ring_size, max_supported);
+}
+
+static int mlx5_nodnic_write_ring_size(struct mlx5_nodnic_core_dev *dev,
+ int offset, int size)
+{
+ u32 addr_l;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &addr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read ring size: %d\n",
+ err);
+ return err;
+ }
+
+ MLX5_SET(nodnic_q_addr_l, &addr_l, log_size, ilog2(size));
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, offset, addr_l);
+ if (err)
+ mlx5_nodnic_core_err(dev, "failed to write ring size: %d\n",
+ err);
+ return err;
+}
+
+static int mlx5_nodnic_read_qn(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, u32 offset)
+{
+ u32 qn_dword;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, offset, &qn_dword);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read queue number: %d\n",
+ err);
+ return err;
+ }
+
+ ring->qn = MLX5_GET(nodnic_ring_q_number, &qn_dword, queue_number);
+
+ return 0;
+}
+
+static int mlx5_nodnic_write_ring_dbr_addr(struct mlx5_nodnic_core_dev *dev,
+ u32 ring_offset)
+{
+ int off_h = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_H;
+ int off_l = ring_offset + MLX5_NODNIC_RING_DBR_ADDR_L;
+ struct mlx5_nodnic_dbr *dbr = &dev->priv->dbr;
+ u32 dbr_h, dbr_l;
+ int err;
+
+ dbr_l = 0;
+ MLX5_SET(nodnic_ring_dbr_addr_l, &dbr_l, db_record_addr_l,
+ (u32)(dbr->dma_handle >> 2));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_l, dbr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring DBR addr low: %d\n",
+ err);
+ return err;
+ }
+
+ dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx, off_h, dbr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write ring DBR addr high: %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_alloc_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, int size)
+{
+ dma_addr_t dma_handle;
+ void *buf;
+
+ buf = dma_alloc_coherent(dev->device, size, &dma_handle, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ *ring = (struct mlx5_nodnic_ring) {
+ .wq = buf,
+ .dma_handle = dma_handle,
+ .size = size,
+ };
+
+ return 0;
+}
+
+static void mlx5_nodnic_free_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring)
+{
+ dma_free_coherent(dev->device, ring->size, ring->wq,
+ ring->dma_handle);
+ memset(ring, 0, sizeof(*ring));
+}
+
+static int mlx5_nodnic_init_ring(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_ring *ring, u32 ring_offset,
+ int stride)
+{
+ int ring_addr_h = ring_offset + MLX5_NODNIC_RING_ADDR_H;
+ int ring_addr_l = ring_offset + MLX5_NODNIC_RING_ADDR_L;
+ int size, err;
+
+ size = mlx5_nodnic_calc_ring_size(dev, stride);
+
+ err = mlx5_nodnic_alloc_ring(dev, ring, size);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle, ring_addr_h,
+ ring_addr_l);
+ if (err)
+ goto err_ring_free;
+
+ err = mlx5_nodnic_write_ring_size(dev, ring_addr_l, size);
+ if (err)
+ goto err_ring_free;
+
+ ring->num_entries = size / stride;
+
+ err = mlx5_nodnic_read_qn(dev, ring,
+ ring_offset + MLX5_NODNIC_RING_QUEUE_NUMBER);
+ if (err)
+ goto err_ring_free;
+
+ err = mlx5_nodnic_write_ring_dbr_addr(dev, ring_offset);
+ if (err)
+ goto err_ring_free;
+
+ return 0;
+
+err_ring_free:
+ mlx5_nodnic_free_ring(dev, ring);
+ return err;
+}
+
+static int mlx5_nodnic_alloc_dbr(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_dbr *dbr)
+{
+ const u32 bytes = MLX5_NODNIC_DBR_WORDS * sizeof(__be32);
+ dma_addr_t dma_handle;
+ void *buf;
+
+ buf = dma_alloc_coherent(dev->device, bytes, &dma_handle, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ *dbr = (struct mlx5_nodnic_dbr) {
+ .db = (__be32 *)buf,
+ .alloc_size = bytes,
+ .dma_handle = dma_handle,
+ };
+
+ return 0;
+}
+
+static void mlx5_nodnic_free_dbr(struct mlx5_nodnic_core_dev *dev,
+ struct mlx5_nodnic_dbr *dbr)
+{
+ if (!dbr->db)
+ return;
+
+ dma_free_coherent(dev->device, dbr->alloc_size, dbr->db,
+ dbr->dma_handle);
+}
+
+static int mlx5_nodnic_init_cq_dbr(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_dbr *dbr = &dev->priv->cq.dbr;
+ u32 dbr_h, dbr_l;
+ int err;
+
+ err = mlx5_nodnic_alloc_dbr(dev, dbr);
+ if (err)
+ return err;
+
+ dbr_l = cpu_to_be32(lower_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_CQ_DBR_ADDR_L,
+ dbr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write CQ DBR addr low: %d\n",
+ err);
+ goto err_free_dbr;
+ }
+
+ dbr_h = cpu_to_be32(upper_32_bits(dbr->dma_handle));
+ err = mlx5_nodnic_vsc_cfg_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_CQ_DBR_ADDR_H,
+ dbr_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write CQ DBR addr high: %d\n",
+ err);
+ goto err_free_dbr;
+ }
+
+ dev->priv->cq.dbr.db[1] = MLX5_NODNIC_CQ_INIT_CMD_SN;
+
+ return 0;
+
+err_free_dbr:
+ mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
+ return err;
+}
+
+static void mlx5_nodnic_cleanup_cq_dbr(struct mlx5_nodnic_core_dev *dev)
+{
+ mlx5_nodnic_free_dbr(dev, &dev->priv->cq.dbr);
+}
+
+/* log_cq_size should be read after rq and sq ring sizes were set. */
+static int mlx5_nodnic_read_cq_ring_size(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ u32 log_cq_size;
+ u32 cq_addr_l;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_CQ_ADDR_L,
+ &cq_addr_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQ ring size: %d\n",
+ err);
+ return err;
+ }
+
+ log_cq_size = MLX5_GET(nodnic_q_addr_l, &cq_addr_l, log_size);
+ cq->num_entries = 1ULL << log_cq_size;
+ cq->log_num_entries = log_cq_size;
+
+ return 0;
+}
+
+static int mlx5_nodnic_init_cq_ring(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ ring = &cq->ring;
+
+ err = mlx5_nodnic_read_cq_ring_size(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_alloc_ring(dev, ring,
+ cq->num_entries * MLX5_NODNIC_CQE_SIZE_B);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(dev, ring->dma_handle,
+ MLX5_NODNIC_PORT_CQ_ADDR_H,
+ MLX5_NODNIC_PORT_CQ_ADDR_L);
+ if (err)
+ goto err_free_cq;
+
+ return 0;
+
+err_free_cq:
+ mlx5_nodnic_free_ring(dev, &cq->ring);
+ return err;
+}
+
+static void mlx5_nodnic_cleanup_cq_ring(struct mlx5_nodnic_core_dev *dev)
+{
+ return mlx5_nodnic_free_ring(dev, &dev->priv->cq.ring);
+}
+
+static int mlx5_nodnic_read_cq_n(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 cq_n;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx, MLX5_NODNIC_CQ_N,
+ &cq_n);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQ number: %d\n",
+ err);
+ return err;
+ }
+
+ dev->priv->cq.cqn = MLX5_GET(nodnic_cqn, &cq_n, cq_n);
+
+ return 0;
+}
+
+int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_ring *work_buf = &dev->priv->working_buffer;
+ u32 size = 1U << dev->log_working_buffer_size;
+ int err;
+
+ err = mlx5_nodnic_alloc_ring(dev, work_buf, size);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_ring_addr(
+ dev, work_buf->dma_handle,
+ MLX5_NODNIC_PORT_WORKING_BUFFER_ADDR_H,
+ MLX5_NODNIC_PORT_WORKING_BUFFER_ADDR_L);
+ if (err)
+ goto err_free_working_buffer;
+
+ return 0;
+
+err_free_working_buffer:
+ mlx5_nodnic_free_ring(dev, work_buf);
+ return err;
+}
+
+int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ int err;
+
+ cq->cc = 0;
+
+ err = mlx5_nodnic_init_cq_ring(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_init_cq_dbr(dev);
+ if (err)
+ goto err_cleanup_ring;
+
+ err = mlx5_nodnic_read_cq_n(dev);
+ if (err)
+ goto err_cleanup_dbr;
+
+ return 0;
+
+err_cleanup_dbr:
+ mlx5_nodnic_cleanup_cq_dbr(dev);
+err_cleanup_ring:
+ mlx5_nodnic_cleanup_cq_ring(dev);
+ return err;
+}
+
+static void mlx5_nodnic_close_cq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_core_dev *dev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+
+ mlx5_nodnic_cleanup_cq_dbr(dev);
+ mlx5_nodnic_cleanup_cq_ring(dev);
+}
+
+int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ sq->netdev = priv->netdev;
+ sq->device = dev->device;
+ sq->pc = 0;
+ sq->cc = 0;
+ ring = &sq->ring;
+
+ err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_SEND_RING_OFFSET,
+ MLX5_NODNIC_SQ_WQEBB_B);
+ if (err)
+ return err;
+
+ ring->lkey = dev->lkey;
+
+ return 0;
+}
+
+static void mlx5_nodnic_close_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_sq *sq = &dev->priv->sq;
+
+ mlx5_nodnic_free_ring(dev, &sq->ring);
+}
+
+int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+ struct mlx5_nodnic_ring *ring;
+ int err;
+
+ rq->pc = 0;
+ rq->cc = 0;
+ ring = &rq->ring;
+
+ err = mlx5_nodnic_init_ring(dev, ring, MLX5_NODNIC_RECEIVE_RING_OFFSET,
+ MLX5_NODNIC_RQ_STRIDE_BYTES);
+ if (err)
+ return err;
+
+ ring->lkey = dev->lkey;
+
+ return 0;
+}
+
+static void mlx5_nodnic_close_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+
+ mlx5_nodnic_free_ring(dev, &rq->ring);
+}
+
+static int mlx5_nodnic_open_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ int err;
+
+ /* one doorbell record shared by both SQ and RQ */
+ err = mlx5_nodnic_alloc_dbr(dev, &dev->priv->dbr);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ goto err_free_dbr;
+
+ err = mlx5_nodnic_open_sq(dev);
+ if (err)
+ goto err_release_vsc;
+
+ err = mlx5_nodnic_open_rq(dev);
+ if (err)
+ goto err_close_sq;
+
+ err = mlx5_nodnic_open_cq(dev);
+ if (err)
+ goto err_close_rq;
+
+ err = mlx5_nodnic_init_working_buffer(dev);
+ if (err)
+ goto err_close_cq;
+
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+
+ return 0;
+
+err_close_cq:
+ mlx5_nodnic_close_cq(&dev->priv->cq);
+err_close_rq:
+ mlx5_nodnic_close_rq(dev);
+err_close_sq:
+ mlx5_nodnic_close_sq(dev);
+err_release_vsc:
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+err_free_dbr:
+ mlx5_nodnic_free_dbr(dev, &dev->priv->dbr);
+
+ return err;
+}
+
+static void mlx5_nodnic_close_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+
+ mlx5_nodnic_free_ring(dev, &priv->working_buffer);
+ mlx5_nodnic_close_cq(&priv->cq);
+ mlx5_nodnic_close_rq(dev);
+ mlx5_nodnic_close_sq(dev);
+ mlx5_nodnic_free_dbr(dev, &priv->dbr);
+}
+
+static void mlx5_nodnic_update_carrier(struct mlx5_nodnic_priv *priv, u32 event)
+{
+ u32 port_state = MLX5_GET(nodnic_port_event, &event, port_state);
+ bool up = port_state == MLX5_NODNIC_PORT_STATE_ACTIVE;
+
+ /* Force re-notification if state is unchanged */
+ if (up == netif_carrier_ok(priv->netdev)) {
+ netif_carrier_event(priv->netdev);
+ return;
+ }
+ if (up) {
+ netif_carrier_on(priv->netdev);
+ netdev_info(priv->netdev, "Link up\n");
+ } else {
+ netif_carrier_off(priv->netdev);
+ netdev_info(priv->netdev, "Link down\n");
+ }
+}
+
+static int mlx5_nodnic_refresh_carrier_locked(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx;
+ u32 event;
+ int err;
+
+ vsc_ctx = dev->vsc_ctx;
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_EVENT,
+ &event);
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+
+ if (!err)
+ mlx5_nodnic_update_carrier(priv, event);
+
+ return err;
+}
+
+void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
+{
+ mutex_lock(&priv->state_lock);
+ if (test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state))
+ mlx5_nodnic_refresh_carrier_locked(priv);
+ mutex_unlock(&priv->state_lock);
+}
+
+int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ u32 val = 0;
+ int err;
+
+ /* data_msix_en, event_msix_en must be set before activating port */
+ MLX5_SET(nodnic_port_network, &val, data_msix_en, 1);
+ MLX5_SET(nodnic_port_network, &val, event_msix_en, 1);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (msix): %d\n",
+ err);
+ return err;
+ }
+
+ MLX5_SET(nodnic_port_network, &val, network_en, 1);
+ MLX5_SET(nodnic_port_network, &val, receive_filter_en, BIT(0));
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (enable): %d\n",
+ err);
+ return err;
+ }
+
+ /* dma enabling requires a separate write */
+ MLX5_SET(nodnic_port_network, &val, dma_en, 1);
+
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "failed to write port network (dma): %d\n",
+ err);
+ return err;
+}
+
+void mlx5_nodnic_disable_port(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ int err;
+ u32 val;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ &val);
+ if (err)
+ goto out;
+
+ MLX5_SET(nodnic_port_network, &val, dma_en, 0);
+ MLX5_SET(nodnic_port_network, &val, network_en, 0);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+ if (err)
+ goto out;
+
+ /* data_msix_en, event_msix_en can't be modified while port is active */
+ MLX5_SET(nodnic_port_network, &val, data_msix_en, 0);
+ MLX5_SET(nodnic_port_network, &val, event_msix_en, 0);
+ err = mlx5_nodnic_vsc_cfg_write_be32(vsc_ctx, MLX5_NODNIC_PORT_NETWORK,
+ val);
+
+out:
+ if (err)
+ mlx5_nodnic_core_warn(dev,
+ "failed to read/write to vsc err %d\n",
+ err);
+}
+
+static int mlx5_nodnic_init_uar_map(struct mlx5_nodnic_core_dev *dev)
+{
+ void __iomem *base;
+ phys_addr_t paddr;
+ u32 uar_page_size;
+
+ uar_page_size = 1 << dev->log_uar_page_size;
+
+ paddr = dev->bar_addr + (dev->send_ring0_uar_index * uar_page_size);
+
+ base = ioremap(paddr, uar_page_size);
+ if (!base) {
+ mlx5_nodnic_core_err(dev, "failed to map UAR\n");
+ return -ENOMEM;
+ }
+
+ dev->uar_base = base;
+
+ return 0;
+}
+
+static void mlx5_nodnic_cleanup_uar_map(struct mlx5_nodnic_core_dev *dev)
+{
+ if (!dev->uar_base)
+ return;
+
+ iounmap(dev->uar_base);
+ dev->uar_base = NULL;
+}
+
+static int mlx5_nodnic_open_locked(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ int err;
+
+ err = mlx5_nodnic_init_uar_map(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_open_queues(dev);
+ if (err)
+ goto err_uar;
+
+ set_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+
+ err = mlx5_nodnic_refresh_carrier_locked(dev->priv);
+ if (err)
+ netdev_err(dev->priv->netdev,
+ "failed to query carrier, err=%d\n", err);
+
+ return 0;
+
+err_uar:
+ mlx5_nodnic_cleanup_uar_map(dev);
+ return err;
+}
+
+int mlx5_nodnic_open(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ int err;
+
+ mutex_lock(&priv->state_lock);
+ err = mlx5_nodnic_open_locked(netdev);
+ mutex_unlock(&priv->state_lock);
+
+ return err;
+}
+
+static int mlx5_nodnic_close_locked(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+
+ clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ netif_carrier_off(priv->netdev);
+ netdev_info(priv->netdev, "Link down\n");
+ mlx5_nodnic_close_queues(dev);
+
+ return 0;
+}
+
+int mlx5_nodnic_close(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ int err = 0;
+
+ mutex_lock(&priv->state_lock);
+ if (!test_bit(MLX5_NODNIC_STATE_OPENED, &priv->state)) {
+ mutex_unlock(&priv->state_lock);
+ return 0;
+ }
+
+ err = mlx5_nodnic_close_locked(netdev);
+ mutex_unlock(&priv->state_lock);
+
+ mlx5_nodnic_cleanup_uar_map(priv->core_dev);
+
+ return err;
+}
+
+static const struct net_device_ops mlx5_nodnic_netdev_ops = {
+ .ndo_open = mlx5_nodnic_open,
+ .ndo_stop = mlx5_nodnic_close,
+};
+
+void mlx5_nodnic_build_netdev(struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *core_dev;
+
+ core_dev = priv->core_dev;
+
+ SET_NETDEV_DEV(netdev, core_dev->device);
+
+ netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
+ netdev->request_ops_lock = true;
+
+ eth_hw_addr_set(netdev, core_dev->mac_address);
+}
+
+static void mlx5_nodnic_init_priv(struct mlx5_nodnic_priv *priv,
+ struct net_device *netdev,
+ struct mlx5_nodnic_core_dev *core_dev)
+{
+ priv->netdev = netdev;
+ priv->core_dev = core_dev;
+ mutex_init(&priv->state_lock);
+}
+
+static void mlx5_nodnic_cleanup_priv(struct mlx5_nodnic_priv *priv)
+{
+ mutex_destroy(&priv->state_lock);
+}
+
+struct net_device *mlx5_nodnic_create_netdev(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv;
+ struct net_device *netdev;
+
+ netdev = alloc_etherdev_mqs(sizeof(struct mlx5_nodnic_priv), 1, 1);
+ if (!netdev)
+ return NULL;
+
+ priv = netdev_priv(netdev);
+ mlx5_nodnic_init_priv(priv, netdev, dev);
+ dev->priv = priv;
+
+ netif_carrier_off(netdev);
+ netif_tx_disable(netdev);
+ dev_net_set(netdev, &init_net);
+
+ return netdev;
+}
+
+void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv;
+ struct net_device *netdev;
+
+ if (!dev->priv)
+ return;
+
+ priv = dev->priv;
+ netdev = priv->netdev;
+ if (!netdev)
+ return;
+
+ if (dev->netdev_registered) {
+ netif_device_detach(netdev);
+ unregister_netdev(netdev);
+ dev->netdev_registered = false;
+ }
+
+ mlx5_nodnic_cleanup_priv(priv);
+ dev->priv = NULL;
+
+ free_netdev(netdev);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
new file mode 100644
index 000000000000..d77d9dbe3c89
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
@@ -0,0 +1,874 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/dma-mapping.h>
+#include <linux/etherdevice.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+#include "nodnic_pci_vsc.h"
+
+MODULE_DESCRIPTION("Nvidia Mellanox MLX5_NODNIC core driver");
+MODULE_LICENSE("Dual BSD/GPL");
+
+#define MLX5_NODNIC_FW_INIT_TIMEOUT_MS 120000
+#define MLX5_NODNIC_FW_INIT_WARN_MS 20000
+#define MLX5_NODNIC_FW_PRE_INIT_WAIT_MS 2
+
+static int mlx5_nodnic_pci_enable_device(struct mlx5_nodnic_core_dev *dev)
+{
+ struct pci_dev *pdev = dev->pdev;
+ int err = 0;
+
+ mutex_lock(&dev->pci_status_mutex);
+ if (dev->pci_status == MLX5_NODNIC_PCI_STATUS_DISABLED) {
+ err = pci_enable_device(pdev);
+ if (!err)
+ dev->pci_status = MLX5_NODNIC_PCI_STATUS_ENABLED;
+ }
+ mutex_unlock(&dev->pci_status_mutex);
+
+ return err;
+}
+
+static void mlx5_nodnic_pci_disable_device(struct mlx5_nodnic_core_dev *dev)
+{
+ struct pci_dev *pdev = dev->pdev;
+
+ mutex_lock(&dev->pci_status_mutex);
+ if (dev->pci_status == MLX5_NODNIC_PCI_STATUS_ENABLED) {
+ pci_disable_device(pdev);
+ dev->pci_status = MLX5_NODNIC_PCI_STATUS_DISABLED;
+ }
+ mutex_unlock(&dev->pci_status_mutex);
+}
+
+static int request_bar(struct pci_dev *pdev)
+{
+ int err;
+
+ if (!(pci_resource_flags(pdev, 0) & IORESOURCE_MEM)) {
+ dev_err(&pdev->dev, "Missing registers BAR, aborting\n");
+ return -ENODEV;
+ }
+
+ err = pci_request_regions(pdev, KBUILD_MODNAME);
+ if (err)
+ dev_err(&pdev->dev, "Couldn't get PCI resources, aborting\n");
+
+ return err;
+}
+
+static void release_bar(struct pci_dev *pdev)
+{
+ pci_release_regions(pdev);
+}
+
+static int set_dma_caps(struct pci_dev *pdev)
+{
+ if (dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
+ int err;
+
+ dev_warn(&pdev->dev,
+ "couldn't set 64-bit PCI DMA mask\n");
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pdev->dev,
+ "can't set PCI DMA mask, aborting\n");
+ return err;
+ }
+ }
+
+ dma_set_max_seg_size(&pdev->dev, 2u * 1024 * 1024 * 1024);
+
+ return 0;
+}
+
+static void mlx5_nodnic_pci_close(struct mlx5_nodnic_core_dev *dev)
+{
+ pci_free_irq_vectors(dev->pdev);
+ iounmap(dev->iseg);
+ release_bar(dev->pdev);
+ mlx5_nodnic_pci_disable_device(dev);
+ mlx5_nodnic_vsc_cleanup(dev->vsc_ctx);
+}
+
+static int
+mlx5_nodnic_pci_init(struct mlx5_nodnic_core_dev *dev, struct pci_dev *pdev)
+{
+ int err;
+
+ mutex_init(&dev->pci_status_mutex);
+ pci_set_drvdata(dev->pdev, dev);
+
+ dev->bar_addr = pci_resource_start(pdev, 0);
+
+ err = mlx5_nodnic_pci_enable_device(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "cannot enable PCI device, aborting\n");
+ return err;
+ }
+
+ err = request_bar(pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "error requesting BARs, aborting\n");
+ goto err_disable;
+ }
+
+ err = set_dma_caps(pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed setting DMA capabilities mask, aborting\n");
+ goto err_release_bar;
+ }
+
+ dev->iseg = ioremap(dev->bar_addr, sizeof(*dev->iseg));
+ if (!dev->iseg) {
+ err = -ENOMEM;
+ mlx5_nodnic_core_err(dev,
+ "failed mapping initialization segment, aborting\n");
+ goto err_release_bar;
+ }
+
+ dev->vsc_ctx = mlx5_nodnic_vsc_init(dev->pdev);
+ if (!dev->vsc_ctx) {
+ err = -EINVAL;
+ goto err_unmap;
+ }
+
+ if (!pci_find_capability(dev->pdev, PCI_CAP_ID_MSIX)) {
+ mlx5_nodnic_core_err(dev,
+ "msi-x support is not supported in pci, aborting\n");
+ err = -EOPNOTSUPP;
+ goto err_vsc_cleanup;
+ }
+
+ err = pci_alloc_irq_vectors(pdev,
+ MLX5_NODNIC_NUM_MSIX, MLX5_NODNIC_NUM_MSIX,
+ PCI_IRQ_MSIX);
+ if (err < 0) {
+ mlx5_nodnic_core_err(dev, "pci_alloc_irq_vectors failed: %d\n",
+ err);
+ goto err_vsc_cleanup;
+ }
+
+ return 0;
+
+err_vsc_cleanup:
+ mlx5_nodnic_vsc_cleanup(dev->vsc_ctx);
+err_unmap:
+ iounmap(dev->iseg);
+err_release_bar:
+ release_bar(dev->pdev);
+err_disable:
+ mlx5_nodnic_pci_disable_device(dev);
+ return err;
+}
+
+int mlx5_nodnic_get_fw_initializing(struct mlx5_nodnic_core_dev *dev,
+ u8 *initializing)
+{
+ u32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_INITIALIZING,
+ &data);
+ if (err)
+ return err;
+
+ *initializing = MLX5_GET(nodnic_iseg_initializing_word, &data,
+ initializing);
+
+ return 0;
+}
+
+static int mlx5_nodnic_wait_fw_init(struct mlx5_nodnic_core_dev *dev,
+ const char *init_state)
+{
+ unsigned long timeout =
+ msecs_to_jiffies(MLX5_NODNIC_FW_INIT_TIMEOUT_MS);
+ unsigned long warn_interval =
+ msecs_to_jiffies(MLX5_NODNIC_FW_INIT_WARN_MS);
+ unsigned long warn = jiffies + warn_interval;
+ unsigned long end = jiffies + timeout;
+ u8 fw_initializing;
+ int err;
+
+ do {
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_get_fw_initializing(dev, &fw_initializing);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read FW initializing state: %d\n",
+ err);
+ return err;
+ }
+
+ if (!fw_initializing)
+ break;
+
+ if (time_after(jiffies, end)) {
+ mlx5_nodnic_core_err(dev,
+ "timeout waiting for FW to exit %s state (%u ms)\n",
+ init_state,
+ MLX5_NODNIC_FW_INIT_TIMEOUT_MS);
+ return -ETIMEDOUT;
+ }
+ if (time_after(jiffies, warn)) {
+ mlx5_nodnic_core_warn(dev,
+ "FW still in %s state, %u ms remaining (state=0x%x)\n",
+ init_state,
+ jiffies_to_msecs(end - jiffies),
+ fw_initializing);
+
+ warn = jiffies + warn_interval;
+ }
+ msleep(MLX5_NODNIC_FW_PRE_INIT_WAIT_MS);
+ } while (true);
+
+ return 0;
+}
+
+int mlx5_nodnic_check_supported(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 initializing_word;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_INITIALIZING,
+ &initializing_word);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed reading NODNIC initializing word: %d\n",
+ err);
+ return err;
+ }
+
+ if (!(MLX5_GET(nodnic_iseg_initializing_word, &initializing_word,
+ nic_interface_supported) &
+ BIT(MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC))) {
+ mlx5_nodnic_core_err(dev,
+ "NODNIC is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_write_nic_interface(struct mlx5_nodnic_core_dev *dev,
+ int interface)
+{
+ u32 dword;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE,
+ &dword);
+ if (err)
+ return err;
+
+ MLX5_SET(nodnic_iseg_cmdq_addr_l, &dword, nic_interface, interface);
+
+ err = mlx5_nodnic_vsc_write_be32(dev->vsc_ctx,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE,
+ dword);
+
+ return err;
+}
+
+int mlx5_nodnic_set_nic_interface(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_check_supported(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_write_nic_interface(
+ dev, MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "failed to write nic_interface err %d\n",
+ err);
+
+ return err;
+}
+
+static int mlx5_nodnic_read_general_caps(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 hardware_format, revision;
+ bool support_receive_filters;
+ u32 caps;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_CAPS,
+ &caps);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read general caps: %d\n",
+ err);
+ return err;
+ }
+
+ revision = MLX5_GET(nodnic_gen_cfg_caps, &caps, revision);
+ hardware_format = MLX5_GET(nodnic_gen_cfg_caps, &caps, hardware_format);
+ if (revision != 1) {
+ mlx5_nodnic_core_err(dev, "revision is not 1, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (hardware_format != 1) {
+ mlx5_nodnic_core_err(dev,
+ "hardware format is not 1, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ dev->log_working_buffer_size = MLX5_GET(nodnic_gen_cfg_caps, &caps,
+ log_working_buffer_size);
+ support_receive_filters = MLX5_GET(nodnic_gen_cfg_caps, &caps,
+ support_receive_filter);
+ if (!support_receive_filters) {
+ mlx5_nodnic_core_err(dev,
+ "receive filters are not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_ring_config(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(
+ dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_LOG_MAX_RING_SIZE,
+ &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read ring config: %d\n",
+ err);
+ return err;
+ }
+
+ dev->log_max_ring_size = MLX5_GET(nodnic_gen_cfg_log_max_ring_size,
+ &data, log_max_ring_size);
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_lkey(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 lkey;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_LKEY,
+ &lkey);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read lkey: %d\n", err);
+ return err;
+ }
+
+ dev->lkey = lkey;
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_cqe_format(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 cqe_format;
+ u32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(
+ dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_CQE_FORMAT,
+ &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read CQE format: %d\n",
+ err);
+ return err;
+ }
+
+ cqe_format = MLX5_GET(nodnic_gen_cfg_cqe_format, &data, cqe_format);
+ if (cqe_format != 0) {
+ mlx5_nodnic_core_err(dev,
+ "cqe format is not legacy (0x0), aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_uar_index(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 uar_index;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(
+ dev->vsc_ctx,
+ MLX5_NODNIC_PORT_SEND_RING0_UAR_INDEX,
+ &uar_index);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read UAR index: %d\n",
+ err);
+ return err;
+ }
+
+ dev->send_ring0_uar_index = uar_index;
+
+ return 0;
+}
+
+static int mlx5_nodnic_read_uar_size(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 uar_size;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(
+ dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_LOG_UAR_PAGE_SIZE,
+ &uar_size);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read UAR size: %d\n", err);
+ return err;
+ }
+
+ dev->log_uar_page_size = MLX5_GET(nodnic_gen_cfg_log_uar_page_size,
+ &uar_size, log_uar_page_size);
+
+ return 0;
+}
+
+static int mlx5_nodnic_validate_feature_caps(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 support_uar_tx_doorbell, support_bar_cq_ctrl;
+ u8 data_msix_support, event_msix_support;
+ u32 data;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(
+ dev->vsc_ctx,
+ MLX5_NODNIC_GENERAL_CONFIG_FEATURE_CAPS,
+ &data);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read feature caps: %d\n",
+ err);
+ return err;
+ }
+
+ support_uar_tx_doorbell = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ support_uar_tx_doorbell);
+ if (!support_uar_tx_doorbell) {
+ mlx5_nodnic_core_err(dev,
+ "UAR tx doorbell is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ support_bar_cq_ctrl = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ support_bar_cq_ctrl);
+ if (!support_bar_cq_ctrl) {
+ mlx5_nodnic_core_err(dev,
+ "UAR cq arming is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ data_msix_support = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ data_msix_support);
+ if (!data_msix_support) {
+ mlx5_nodnic_core_err(dev,
+ "data MSI-X is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ event_msix_support = MLX5_GET(nodnic_gen_cfg_feature_caps, &data,
+ event_msix_support);
+ if (!event_msix_support) {
+ mlx5_nodnic_core_err(dev,
+ "event MSI-X is not supported, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_read_config(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read_offset(dev->vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read VSC offset, err=%d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_read_general_caps(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_ring_config(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_lkey(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_cqe_format(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_validate_feature_caps(dev);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_read_uar_size(dev);
+ if (err)
+ return err;
+
+ return mlx5_nodnic_read_uar_index(dev);
+}
+
+static void mlx5_nodnic_dev_cleanup(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return;
+
+ err = mlx5_nodnic_write_nic_interface(
+ dev, MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err)
+ return;
+
+ mlx5_nodnic_wait_fw_init(dev, "teardown");
+}
+
+static int mlx5_nodnic_dev_init(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
+ int err;
+
+ err = mlx5_nodnic_wait_fw_init(dev, "pre-FLR");
+ if (err)
+ return err;
+
+ pci_save_state(dev->pdev);
+ err = pcie_flr(dev->pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "pcie_flr failed with error code %d\n",
+ err);
+ return err;
+ }
+ pci_restore_state(dev->pdev);
+
+ err = mlx5_nodnic_wait_fw_init(dev, "post-FLR");
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_set_nic_interface(dev);
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_set_nic_interface failed with error code %d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_wait_fw_init(dev, "post-MLX5_NODNIC setup");
+ if (err)
+ goto disable_nic;
+
+ pci_set_master(dev->pdev);
+
+ err = mlx5_nodnic_vsc_context_acquire(vsc_ctx);
+ if (err)
+ goto disable_nic;
+
+ err = mlx5_nodnic_read_config(dev);
+ if (err)
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_read_config failed with error code %d\n",
+ err);
+
+ mlx5_nodnic_vsc_context_release(vsc_ctx);
+ if (err)
+ goto disable_nic;
+
+ return 0;
+
+disable_nic:
+ mlx5_nodnic_dev_cleanup(dev);
+ return err;
+}
+
+int mlx5_nodnic_validate_port_link_type(struct mlx5_nodnic_core_dev *dev)
+{
+ u8 link_type;
+ u32 event;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_be32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_EVENT,
+ &event);
+ if (err) {
+ mlx5_nodnic_core_err(dev, "failed to read port event: %d\n",
+ err);
+ return err;
+ }
+
+ link_type = MLX5_GET(nodnic_port_event, &event, link_type);
+ if (!link_type) {
+ mlx5_nodnic_core_err(dev, "port is not Ethernet, aborting\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+int mlx5_nodnic_read_mac_address(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 mac_h, mac_l;
+ u64 mac;
+ int err;
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_MAC_ADDR_H,
+ &mac_h);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read MAC address high: %d\n",
+ err);
+ return err;
+ }
+
+ err = mlx5_nodnic_vsc_cfg_read_cpu32(dev->vsc_ctx,
+ MLX5_NODNIC_PORT_MAC_ADDR_L,
+ &mac_l);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "failed to read MAC address low: %d\n",
+ err);
+ return err;
+ }
+
+ mac = ((u64)mac_h << 32) | mac_l;
+
+ u64_to_ether_addr(mac, dev->mac_address);
+
+ return 0;
+}
+
+static int mlx5_nodnic_port_init(struct mlx5_nodnic_core_dev *dev)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_validate_port_link_type(dev);
+ if (err)
+ goto release;
+
+ err = mlx5_nodnic_read_mac_address(dev);
+ if (err)
+ goto release;
+
+release:
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ return err;
+}
+
+static irqreturn_t mlx5_nodnic_event_irq_thread(int irq, void *data)
+{
+ struct mlx5_nodnic_priv *priv = data;
+
+ if (!netif_device_present(priv->netdev))
+ return IRQ_HANDLED;
+
+ mlx5_nodnic_refresh_carrier(priv);
+
+ return IRQ_HANDLED;
+}
+
+static int mlx5_nodnic_event_irq_setup(struct mlx5_nodnic_core_dev *dev)
+{
+ int irqn, err;
+
+ irqn = pci_irq_vector(dev->pdev, MLX5_NODNIC_EVENT_MSIX);
+ if (irqn < 0) {
+ mlx5_nodnic_core_err(dev,
+ "pci_irq_vector for event failed: %d\n",
+ irqn);
+ return irqn;
+ }
+
+ err = request_threaded_irq(irqn, NULL, mlx5_nodnic_event_irq_thread,
+ IRQF_ONESHOT, "mlx5_nodnic-event",
+ dev->priv);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "request_threaded_irq for event failed\n");
+ return err;
+ }
+
+ dev->event_irqn = irqn;
+
+ return 0;
+}
+
+static void mlx5_nodnic_event_irq_cleanup(struct mlx5_nodnic_core_dev *dev)
+{
+ if (dev->event_irqn < 0)
+ return;
+ free_irq(dev->event_irqn, dev->priv);
+ dev->event_irqn = -1;
+}
+
+static int mlx5_nodnic_probe_one(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct mlx5_nodnic_core_dev *dev;
+ struct net_device *netdev;
+ int err;
+
+ dev = kzalloc_obj(struct mlx5_nodnic_core_dev, GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->device = &pdev->dev;
+ dev->pdev = pdev;
+ dev->event_irqn = -1;
+
+ err = mlx5_nodnic_pci_init(dev, pdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_pci_init failed with error code %d\n",
+ err);
+ goto err;
+ }
+
+ err = mlx5_nodnic_dev_init(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_dev_init failed with error code %d\n",
+ err);
+ goto err_pci_cleanup;
+ }
+
+ err = mlx5_nodnic_port_init(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_port_init failed with error code %d\n",
+ err);
+ goto err_dev_cleanup;
+ }
+
+ netdev = mlx5_nodnic_create_netdev(dev);
+ if (!netdev) {
+ err = -ENOMEM;
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_create_netdev failed with error code %d\n",
+ err);
+ goto err_dev_cleanup;
+ }
+
+ mlx5_nodnic_build_netdev(netdev);
+
+ err = mlx5_nodnic_event_irq_setup(dev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "mlx5_nodnic_event_irq_setup failed with error code %d\n",
+ err);
+ goto err_netdev_cleanup;
+ }
+
+ err = register_netdev(netdev);
+ if (err) {
+ mlx5_nodnic_core_err(dev,
+ "register_netdev failed with error code %d\n",
+ err);
+ goto err_irq_cleanup;
+ }
+ dev->netdev_registered = true;
+
+ dev->state = NODNIC_DEVICE_STATE_UP;
+
+ pci_save_state(pdev);
+
+ return 0;
+
+err_irq_cleanup:
+ mlx5_nodnic_event_irq_cleanup(dev);
+err_netdev_cleanup:
+ mlx5_nodnic_cleanup_netdev(dev);
+err_dev_cleanup:
+ mlx5_nodnic_dev_cleanup(dev);
+err_pci_cleanup:
+ mlx5_nodnic_pci_close(dev);
+err:
+ kfree(dev);
+ return err;
+}
+
+static void mlx5_nodnic_remove_one(struct pci_dev *pdev)
+{
+ struct mlx5_nodnic_core_dev *dev = pci_get_drvdata(pdev);
+
+ if (!dev)
+ return;
+
+ mlx5_nodnic_event_irq_cleanup(dev);
+ mlx5_nodnic_cleanup_netdev(dev);
+ mlx5_nodnic_dev_cleanup(dev);
+ mlx5_nodnic_pci_close(dev);
+ kfree(dev);
+}
+
+static const struct pci_device_id mlx5_nodnic_core_pci_table[] = {
+ { PCI_VDEVICE(MELLANOX, 0xc2d6) }, /* BlueField-4 nodnic rshim PF */
+ { }
+};
+
+MODULE_DEVICE_TABLE(pci, mlx5_nodnic_core_pci_table);
+
+static struct pci_driver mlx5_nodnic_core_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = mlx5_nodnic_core_pci_table,
+ .probe = mlx5_nodnic_probe_one,
+ .remove = mlx5_nodnic_remove_one,
+};
+
+static int __init mlx5_nodnic_init(void)
+{
+ return pci_register_driver(&mlx5_nodnic_core_driver);
+}
+
+static void __exit mlx5_nodnic_cleanup(void)
+{
+ pci_unregister_driver(&mlx5_nodnic_core_driver);
+}
+
+module_init(mlx5_nodnic_init);
+module_exit(mlx5_nodnic_cleanup);
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
new file mode 100644
index 000000000000..2fcae95988d8
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic.h
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_H__
+#define __MLX5_NODNIC_H__
+
+#include <linux/mlx5/device.h>
+#include <linux/pci.h>
+#include <linux/sched.h>
+
+struct mlx5_nodnic_priv;
+
+#define mlx5_nodnic_core_dbg(__dev, format, ...) \
+ dev_dbg((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+#define mlx5_nodnic_core_err(__dev, format, ...) \
+ dev_err((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+#define mlx5_nodnic_core_warn(__dev, format, ...) \
+ dev_warn((__dev)->device, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
+enum {
+ MLX5_NODNIC_PORT_STATE_DOWN,
+ MLX5_NODNIC_PORT_STATE_INITIALIZE,
+ MLX5_NODNIC_PORT_STATE_ARMED,
+ MLX5_NODNIC_PORT_STATE_ACTIVE,
+};
+
+enum mlx5_nodnic_pci_status {
+ MLX5_NODNIC_PCI_STATUS_DISABLED,
+ MLX5_NODNIC_PCI_STATUS_ENABLED,
+};
+
+enum mlx5_nodnic_device_state {
+ NODNIC_DEVICE_STATE_UP = 1,
+ NODNIC_DEVICE_STATE_INTERNAL_ERROR,
+};
+
+enum mlx5_nodnic_msix {
+ MLX5_NODNIC_DATA_MSIX,
+ MLX5_NODNIC_EVENT_MSIX,
+ MLX5_NODNIC_NUM_MSIX,
+};
+
+struct mlx5_nodnic_vsc_ctx;
+
+struct mlx5_nodnic_core_dev {
+ struct mlx5_nodnic_priv *priv;
+ struct device *device;
+ struct pci_dev *pdev;
+ bool netdev_registered;
+
+ /* sync pci state */
+ struct mutex pci_status_mutex;
+ enum mlx5_nodnic_pci_status pci_status;
+
+ struct mlx5_init_seg __iomem *iseg;
+ phys_addr_t bar_addr;
+ struct mlx5_nodnic_vsc_ctx *vsc_ctx;
+ void __iomem *uar_base;
+
+ u8 log_working_buffer_size;
+ u8 log_max_ring_size;
+ u8 log_uar_page_size;
+ u32 send_ring0_uar_index;
+
+ u8 mac_address[6];
+ u32 lkey;
+
+ int event_irqn;
+
+ enum mlx5_nodnic_device_state state;
+};
+
+void mlx5_nodnic_start_health_poll(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_stop_health_poll(struct mlx5_nodnic_core_dev *dev);
+
+/*
+ * VSC-gated helpers: the caller must hold the VSC context across these calls.
+ */
+int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev);
+void mlx5_nodnic_disable_port(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_get_fw_initializing(struct mlx5_nodnic_core_dev *dev,
+ u8 *initializing);
+int mlx5_nodnic_set_nic_interface(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_check_supported(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_write_nic_interface(struct mlx5_nodnic_core_dev *dev,
+ int interface);
+int mlx5_nodnic_read_config(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_validate_port_link_type(struct mlx5_nodnic_core_dev *dev);
+int mlx5_nodnic_read_mac_address(struct mlx5_nodnic_core_dev *dev);
+
+/* end of VSC-gated helpers */
+
+#endif /* __MLX5_NODNIC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
new file mode 100644
index 000000000000..59de0644ea89
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_ifc.h
@@ -0,0 +1,318 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#ifndef __MLX5_IFC_NODNIC_H__
+#define __MLX5_IFC_NODNIC_H__
+
+#include <linux/mlx5/device.h>
+#include <linux/types.h>
+
+enum {
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_FULL_DRIVER,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_NO_DRAM_NIC,
+ MLX5_NODNIC_ISEG_NIC_INTERFACE_SW_RESET = 0x7,
+};
+
+enum {
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_INTERNAL_ERR = 0x1,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_DEAD_IRISC = 0x7,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HW_FATAL_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_CRC_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_FETCH_PCI_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PAGE_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ASYNCHRONOUS_EQ_BUF_OVERRUN,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_IN_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_INV,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FFSER_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HIGH_TEMP_ERR,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PCI_POISONED_ERR = 0x12,
+ MLX5_NODNIC_ISEG_HEALTH_SYNDROME_TRUST_LOCKDOWN_ERR,
+};
+
+/* ISEG */
+#define MLX5_NODNIC_ISEG_NIC_INTERFACE MLX5_BYTE_OFF(nodnic_iseg, cmdq_addr_l)
+#define MLX5_NODNIC_ISEG_INITIALIZING \
+ MLX5_BYTE_OFF(nodnic_iseg, initializing_word)
+#define MLX5_NODNIC_ISEG_NO_DRAM_NIC_OFFSET \
+ MLX5_BYTE_OFF(nodnic_iseg, no_dram_nic_offset)
+#define MLX5_NODNIC_ISEG_CMD_ADDR_L \
+ MLX5_BYTE_OFF(nodnic_iseg_cmdq_addr_l, cmdq_addr_l)
+#define MLX5_NODNIC_ISEG_CLEAR_INT MLX5_BYTE_OFF(nodnic_iseg, clear_int)
+
+/* General configuration */
+#define MLX5_NODNIC_GENERAL_CONFIG_CAPS MLX5_BYTE_OFF(nodnic_gen_cfg, caps)
+#define MLX5_NODNIC_GENERAL_CONFIG_LOG_MAX_RING_SIZE \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, log_max_ring_size)
+#define MLX5_NODNIC_GENERAL_CONFIG_LKEY MLX5_BYTE_OFF(nodnic_gen_cfg, lkey)
+#define MLX5_NODNIC_GENERAL_CONFIG_CQE_FORMAT \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, cqe_format)
+#define MLX5_NODNIC_GENERAL_CONFIG_FEATURE_CAPS \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, feature_caps)
+#define MLX5_NODNIC_GENERAL_CONFIG_LOG_UAR_PAGE_SIZE \
+ MLX5_BYTE_OFF(nodnic_gen_cfg, log_uar_page_size)
+
+/* Port */
+#define MLX5_NODNIC_PORT_OFFSET \
+ MLX5_BYTE_OFF(nodnic_config_seg, port)
+#define MLX5_NODNIC_PORT_FIELD_OFFSET(field) \
+ (MLX5_NODNIC_PORT_OFFSET + MLX5_BYTE_OFF(nodnic_port_settings, field))
+
+#define MLX5_NODNIC_PORT_EVENT MLX5_NODNIC_PORT_FIELD_OFFSET(event)
+#define MLX5_NODNIC_RECEIVE_RING_OFFSET \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(receive_ring0)
+#define MLX5_NODNIC_SEND_RING_OFFSET MLX5_NODNIC_PORT_FIELD_OFFSET(send_ring0)
+#define MLX5_NODNIC_PORT_NETWORK MLX5_NODNIC_PORT_FIELD_OFFSET(network)
+#define MLX5_NODNIC_PORT_MAC_ADDR_H MLX5_NODNIC_PORT_FIELD_OFFSET(mac_h)
+#define MLX5_NODNIC_PORT_MAC_ADDR_L MLX5_NODNIC_PORT_FIELD_OFFSET(mac_l)
+#define MLX5_NODNIC_PORT_CQ_ADDR_H MLX5_NODNIC_PORT_FIELD_OFFSET(cq_addr_h)
+#define MLX5_NODNIC_PORT_CQ_ADDR_L MLX5_NODNIC_PORT_FIELD_OFFSET(cq_addr_l)
+#define MLX5_NODNIC_PORT_WORKING_BUFFER_ADDR_H \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(working_buffer_addr_h)
+#define MLX5_NODNIC_PORT_WORKING_BUFFER_ADDR_L \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(working_buffer_addr_l)
+#define MLX5_NODNIC_PORT_ARM_CQ MLX5_NODNIC_PORT_FIELD_OFFSET(arm_cq)
+#define MLX5_NODNIC_PORT_SEND_RING0_UAR_INDEX \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(send_ring0_uar_index)
+#define MLX5_NODNIC_CQ_DBR_ADDR_H \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cq_dbr_addr_h)
+#define MLX5_NODNIC_CQ_DBR_ADDR_L \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cq_dbr_addr_l)
+#define MLX5_NODNIC_CQ_N \
+ MLX5_NODNIC_PORT_FIELD_OFFSET(cqn)
+
+/* Port ring */
+#define MLX5_NODNIC_RING_ADDR_H MLX5_BYTE_OFF(nodnic_ring_config, q_addr_h)
+#define MLX5_NODNIC_RING_ADDR_L MLX5_BYTE_OFF(nodnic_ring_config, q_addr_l)
+#define MLX5_NODNIC_RING_QUEUE_NUMBER \
+ MLX5_BYTE_OFF(nodnic_ring_config, q_number)
+#define MLX5_NODNIC_RING_DBR_ADDR_H \
+ MLX5_BYTE_OFF(nodnic_ring_config, dbr_addr_h)
+#define MLX5_NODNIC_RING_DBR_ADDR_L \
+ MLX5_BYTE_OFF(nodnic_ring_config, dbr_addr_l)
+
+/* mlx5_nodnic configurations are read and written from VSC
+ * in a dword granularity
+ */
+
+struct mlx5_ifc_nodnic_iseg_cmdq_addr_l_bits {
+ u8 cmdq_addr_l[0x14];
+ u8 reserved_at_14[0x1];
+ u8 nic_interface[0x3];
+ u8 log_cmdq_size[0x4];
+ u8 log_cmdq_stride[0x4];
+};
+
+struct mlx5_ifc_nodnic_iseg_initializing_word_bits {
+ u8 initializing[0x1];
+ u8 nic_interface_supported[0x7];
+ u8 embedded_cpu[0x1];
+ u8 driver_reset_ack[0x1];
+ u8 recovery[0x1];
+ u8 pre_boot_bios_hold[0x1];
+ u8 initializing_state[0x4];
+ u8 reset_state_machine[0x4];
+ u8 reserved_at_14[0x1];
+ u8 pre_boot_action[0x3];
+ u8 reserved_at_17[0x8];
+};
+
+struct mlx5_ifc_nodnic_iseg_health_buffer_bits {
+ u8 reserved_at_0[0x100];
+ u8 assert_existptr[0x20];
+ u8 assert_callra[0x20];
+ u8 reserved_at_28[0x20];
+ u8 time[0x20];
+ u8 fw_version[0x20];
+ u8 hw_id[0x20];
+ u8 rfr[0x1];
+ u8 crr[0x1];
+ u8 reserved_at_e2[0x2];
+ u8 valid[0x1];
+ u8 severity[0x3];
+ u8 reserved_at_e8[0x18];
+ u8 irisc_index[0x8];
+ u8 synd[0x8];
+ u8 ext_synd[0x10];
+};
+
+struct mlx5_ifc_nodnic_clear_int_bits {
+ u8 reserved_at_0[0x1f];
+ u8 clear_int[0x1];
+};
+
+struct mlx5_ifc_nodnic_iseg_bits {
+ u8 fw_rev[0x20];
+ u8 cmdif_rev[0x20];
+ u8 rsvd0[0x40];
+ u8 cmdq_addr_h[0x20];
+ struct mlx5_ifc_nodnic_iseg_cmdq_addr_l_bits cmdq_addr_l;
+ u8 command_doorbell_vector[0x20];
+ u8 rsvd1[0xee0];
+ u8 traffic_state_rsvd[0x1e];
+ u8 traffic_state[0x2];
+ struct mlx5_ifc_nodnic_iseg_initializing_word_bits initializing_word;
+ struct mlx5_ifc_nodnic_iseg_health_buffer_bits health_buffer;
+ u8 no_dram_nic_offset[0x20];
+ u8 rsvd2[0x6e40];
+ struct mlx5_ifc_nodnic_clear_int_bits clear_int;
+};
+
+struct mlx5_ifc_nodnic_q_addr_l_bits {
+ u8 q_addr_l[0x14];
+ u8 reserved_at_14[0x6];
+ u8 log_size[0x6];
+};
+
+struct mlx5_ifc_nodnic_ring_db_reg_bits {
+ u8 reserved_at_0[0x8];
+ u8 ring_pi[0x10];
+ u8 reserved_at_18[0x8];
+};
+
+struct mlx5_ifc_nodnic_ring_q_number_bits {
+ u8 reserved_at_0[0x8];
+ u8 queue_number[0x18];
+};
+
+struct mlx5_ifc_nodnic_ring_dbr_addr_l_bits {
+ u8 db_record_addr_l[0x1e];
+ u8 reserved_at_1e[0x2];
+};
+
+struct mlx5_ifc_nodnic_ring_config_bits {
+ u8 q_addr_h[0x20];
+ struct mlx5_ifc_nodnic_q_addr_l_bits q_addr_l;
+ struct mlx5_ifc_nodnic_ring_db_reg_bits db_reg;
+ struct mlx5_ifc_nodnic_ring_q_number_bits q_number;
+ u8 q_key[0x20];
+ u8 pkey[0x20];
+ u8 dbr_addr_h[0x20];
+ u8 dbr_addr_l[0x20];
+};
+
+struct mlx5_ifc_nodnic_port_event_bits {
+ u8 driver_reset_needed[0x1];
+ u8 port_management_change[0x1];
+ u8 reserved_at_2[0x19];
+ u8 link_type[0x1];
+ u8 port_state[0x4];
+};
+
+struct mlx5_ifc_nodnic_port_network_bits {
+ u8 network_en[0x1];
+ u8 dma_en[0x1];
+ u8 promisc_en[0x1];
+ u8 promisc_multicast_en[0x1];
+ u8 vlan_stripping_en[0x1];
+ u8 protocol_filter_ip_ver[0x1];
+ u8 data_msix_en[0x1];
+ u8 event_msix_en[0x1];
+ u8 reserved_at_8[0x8];
+ u8 protocol_filter_en[0x8];
+ u8 reserved_at_18[0x3];
+ u8 receive_filter_en[0x5];
+};
+
+struct mlx5_ifc_nodnic_port_mac_h_bits {
+ u8 reserved_at_0[0x10];
+ u8 mac_h[0x10];
+};
+
+struct mlx5_ifc_nodnic_port_arm_cq_bits {
+ u8 cq_ci[0x18];
+ u8 reserved_at_18[0x8];
+};
+
+struct mlx5_ifc_nodnic_cqn_bits {
+ u8 reserved[0x8];
+ u8 cq_n[0x18];
+};
+
+struct mlx5_ifc_nodnic_port_settings_bits {
+ struct mlx5_ifc_nodnic_port_event_bits event;
+ struct mlx5_ifc_nodnic_port_network_bits network;
+ struct mlx5_ifc_nodnic_port_mac_h_bits mac_h;
+ u8 mac_l[0x20];
+ u8 mac_filters[0x200];
+ u8 gid[0x80];
+ u8 reserved_at_300[0x20];
+ u8 sm_sl_lid[0x20];
+ u8 cq_addr_h[0x20];
+ struct mlx5_ifc_nodnic_q_addr_l_bits cq_addr_l;
+ u8 working_buffer_addr_h[0x20];
+ u8 working_buffer_addr_l[0x20];
+ struct mlx5_ifc_nodnic_port_arm_cq_bits arm_cq;
+ u8 pkey[0x20];
+ struct mlx5_ifc_nodnic_ring_config_bits send_ring0;
+ u8 rsvd_send_ring1[0x100];
+ struct mlx5_ifc_nodnic_ring_config_bits receive_ring0;
+ u8 rsvd_receive_ring1[0x100];
+ u8 protocol_filter_tftp_port[0x20];
+ u8 reserved_at_104[0x20];
+ u8 send_ring0_uar_index[0x20];
+ u8 send_ring1_uar_index[0x20];
+ u8 cq_dbr_addr_h[0x20];
+ u8 cq_dbr_addr_l[0x20];
+ struct mlx5_ifc_nodnic_cqn_bits cqn;
+ u8 padding[0x320];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_caps_bits {
+ u8 revision[0x8];
+ u8 hardware_format[0x8];
+ u8 support_receive_filter[0x1];
+ u8 support_promisc_filter[0x1];
+ u8 support_promisc_multicast[0x1];
+ u8 support_protocol_filter[0x1];
+ u8 reserved_at_13[0x1];
+ u8 log_working_buffer_size[0x3];
+ u8 log_pkey_table_size[0x4]; /* IB */
+ u8 reserved_at_1a[0x3];
+ u8 num_ports[0x1];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_log_max_ring_size_bits {
+ u8 reserved_at_0[0x2];
+ u8 log_max_ring_size[0x6];
+ u8 reserved_at_8[0x18];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_cqe_format_bits {
+ u8 cqe_format[0x4];
+ u8 reserved_at_4[0x1c];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_feature_caps_bits {
+ u8 support_db_record_rx[0x1];
+ u8 reserved_at_1[0x1];
+ u8 support_uar_tx_doorbell[0x1];
+ u8 support_bar_dma_en[0x1];
+ u8 support_bar_cq_ctrl[0x1];
+ u8 vlan_stripping_supported[0x1];
+ u8 data_msix_support[0x1];
+ u8 event_msix_support[0x1];
+ u8 reserved_at_8[0x18];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_log_uar_page_size_bits {
+ u8 reserved_at_0[0x18];
+ u8 log_uar_page_size[0x8];
+};
+
+struct mlx5_ifc_nodnic_gen_cfg_bits {
+ struct mlx5_ifc_nodnic_gen_cfg_caps_bits caps;
+ struct mlx5_ifc_nodnic_gen_cfg_log_max_ring_size_bits log_max_ring_size;
+ u8 lkey[0x20];
+ struct mlx5_ifc_nodnic_gen_cfg_cqe_format_bits cqe_format;
+ u8 reserved_at_80[3][0x20];
+ struct mlx5_ifc_nodnic_gen_cfg_feature_caps_bits feature_caps;
+ struct mlx5_ifc_nodnic_gen_cfg_log_uar_page_size_bits log_uar_page_size;
+};
+
+struct mlx5_ifc_nodnic_config_seg_bits {
+ struct mlx5_ifc_nodnic_gen_cfg_bits general;
+ u8 reserved_at_24[55][0x20];
+ struct mlx5_ifc_nodnic_port_settings_bits port;
+};
+
+#endif /* __MLX5_IFC_NODNIC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
new file mode 100644
index 000000000000..9e9cdf375c8e
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.c
@@ -0,0 +1,418 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/delay.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include "nodnic_ifc.h"
+#include "nodnic_pci_vsc.h"
+
+struct mlx5_nodnic_vsc_ctx {
+ struct pci_dev *pdev;
+ u32 addr; /* VSC init segment address */
+ u32 offset; /* NODNIC config offset */
+};
+
+#define MLX5_NODNIC_ONES32(size) \
+ ((size) ? (0xffffffff >> (32 - (size))) : 0)
+#define MLX5_NODNIC_MASK32(offset, size) \
+ (MLX5_NODNIC_ONES32(size) << (offset))
+#define MLX5_NODNIC_EXTRACT_C(source, offset, size) \
+ ((((u32)(source)) >> (offset)) & MLX5_NODNIC_ONES32(size))
+#define MLX5_NODNIC_EXTRACT(src, start, len) \
+ (((len) == 32) ? (src) : MLX5_NODNIC_EXTRACT_C(src, start, len))
+#define MLX5_NODNIC_MERGE_C(rsrc1, rsrc2, start, len) \
+ ((((rsrc2) << (start)) & (MLX5_NODNIC_MASK32((start), (len)))) | \
+ ((rsrc1) & (~MLX5_NODNIC_MASK32((start), (len)))))
+#define MLX5_NODNIC_MERGE(rsrc1, rsrc2, start, len) \
+ (((len) == 32) ? (rsrc2) : \
+ MLX5_NODNIC_MERGE_C(rsrc1, rsrc2, start, len))
+#define mlx5_nodnic_vsc_pci_read(ctx, offset, val) \
+ pci_read_config_dword((ctx)->pdev, (ctx)->addr + (offset), (val))
+#define mlx5_nodnic_vsc_pci_write(ctx, offset, val) \
+ pci_write_config_dword((ctx)->pdev, (ctx)->addr + (offset), (val))
+#define VSC_MAX_RETRIES 2048
+
+enum {
+ MLX5_NODNIC_VSC_CTRL_OFFSET = 0x4,
+ MLX5_NODNIC_VSC_COUNTER_OFFSET = 0x8,
+ MLX5_NODNIC_VSC_SEMAPHORE_OFFSET = 0xc,
+ MLX5_NODNIC_VSC_ADDR_OFFSET = 0x10,
+ MLX5_NODNIC_VSC_DATA_OFFSET = 0x14,
+
+ MLX5_NODNIC_VSC_FLAG_BIT_OFFS = 31,
+ MLX5_NODNIC_VSC_FLAG_BIT_LEN = 1,
+
+ MLX5_NODNIC_VSC_SYND_BIT_OFFS = 30,
+ MLX5_NODNIC_VSC_SYND_BIT_LEN = 1,
+
+ MLX5_NODNIC_VSC_ADDR_BIT_OFFS = 0,
+ MLX5_NODNIC_VSC_ADDR_BIT_LEN = 30,
+
+ MLX5_NODNIC_VSC_SPACE_BIT_OFFS = 0,
+ MLX5_NODNIC_VSC_SPACE_BIT_LEN = 16,
+
+ MLX5_NODNIC_VSC_SIZE_VLD_BIT_OFFS = 28,
+ MLX5_NODNIC_VSC_SIZE_VLD_BIT_LEN = 1,
+
+ MLX5_NODNIC_VSC_STATUS_BIT_OFFS = 29,
+ MLX5_NODNIC_VSC_STATUS_BIT_LEN = 3,
+};
+
+struct mlx5_nodnic_vsc_ctx *mlx5_nodnic_vsc_init(struct pci_dev *pdev)
+{
+ struct mlx5_nodnic_vsc_ctx *ctx;
+
+ ctx = kzalloc_obj(struct mlx5_nodnic_vsc_ctx, GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ ctx->pdev = pdev;
+ ctx->addr = pci_find_capability(ctx->pdev, PCI_CAP_ID_VNDR);
+ if (!ctx->addr) {
+ dev_err(&ctx->pdev->dev,
+ "failed to get valid vendor specific ID\n");
+ kfree(ctx);
+ return NULL;
+ }
+
+ return ctx;
+}
+
+void mlx5_nodnic_vsc_cleanup(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ kfree(ctx);
+}
+
+int mlx5_nodnic_vsc_read_offset(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ u32 offset;
+ int err;
+
+ err = mlx5_nodnic_vsc_read_cpu32(ctx,
+ MLX5_NODNIC_ISEG_NO_DRAM_NIC_OFFSET,
+ &offset);
+ if (err) {
+ dev_warn(&ctx->pdev->dev,
+ "failed to read no_dram_nic_offset err %d\n", err);
+ return err;
+ }
+
+ ctx->offset = offset;
+ return 0;
+}
+
+static int mlx5_nodnic_vsc_gw_lock(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ struct pci_dev *pdev = ctx->pdev;
+ u32 lock_val, counter = 0;
+ int err, retries = 0;
+
+ pci_cfg_access_lock(pdev);
+
+ do {
+ if (retries > VSC_MAX_RETRIES) {
+ err = -EBUSY;
+ goto pci_unlock;
+ }
+ if (pci_channel_offline(pdev)) {
+ err = -EACCES;
+ goto pci_unlock;
+ }
+
+ /* Check if semaphore is already locked */
+ err = mlx5_nodnic_vsc_pci_read(ctx,
+ MLX5_NODNIC_VSC_SEMAPHORE_OFFSET,
+ &lock_val);
+ if (err)
+ goto pci_unlock;
+
+ if (lock_val) {
+ retries++;
+ usleep_range(1000, 2000);
+ continue;
+ }
+
+ /* Read and write counter value, if written value is
+ * the same, semaphore was acquired successfully.
+ */
+ err = mlx5_nodnic_vsc_pci_read(ctx,
+ MLX5_NODNIC_VSC_COUNTER_OFFSET,
+ &counter);
+ if (err)
+ goto pci_unlock;
+
+ err = mlx5_nodnic_vsc_pci_write(
+ ctx, MLX5_NODNIC_VSC_SEMAPHORE_OFFSET, counter);
+ if (err)
+ goto pci_unlock;
+
+ err = mlx5_nodnic_vsc_pci_read(
+ ctx, MLX5_NODNIC_VSC_SEMAPHORE_OFFSET, &lock_val);
+ if (err)
+ goto pci_unlock;
+
+ retries++;
+ } while (counter != lock_val);
+
+ return 0;
+
+pci_unlock:
+ pci_cfg_access_unlock(pdev);
+ return err;
+}
+
+static int mlx5_nodnic_vsc_gw_unlock(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_write(
+ ctx, MLX5_NODNIC_VSC_SEMAPHORE_OFFSET, MLX5_NODNIC_VSC_UNLOCK);
+ pci_cfg_access_unlock(ctx->pdev);
+
+ return err;
+}
+
+static int
+mlx5_nodnic_vsc_gw_set_space(struct mlx5_nodnic_vsc_ctx *ctx, u16 space)
+{
+ u32 val;
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, &val);
+ if (err)
+ return err;
+
+ val = MLX5_NODNIC_MERGE(
+ val, space, MLX5_NODNIC_VSC_SPACE_BIT_OFFS,
+ MLX5_NODNIC_VSC_SPACE_BIT_LEN);
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, val);
+ if (err)
+ return err;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_CTRL_OFFSET, &val);
+ if (err)
+ return err;
+
+ if (MLX5_NODNIC_EXTRACT(
+ val, MLX5_NODNIC_VSC_STATUS_BIT_OFFS,
+ MLX5_NODNIC_VSC_STATUS_BIT_LEN) == 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+int mlx5_nodnic_vsc_context_acquire(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_gw_lock(ctx);
+ if (err) {
+ dev_warn(&ctx->pdev->dev, "failed to lock vsc, err %d\n", err);
+ return err;
+ }
+
+ err = mlx5_nodnic_vsc_gw_set_space(ctx, MLX5_NODNIC_VSC_SPACE_NODNIC);
+ if (err) {
+ dev_warn(&ctx->pdev->dev,
+ "failed to set vsc space, err %d\n", err);
+ mlx5_nodnic_vsc_gw_unlock(ctx);
+ return err;
+ }
+
+ return err;
+}
+
+void mlx5_nodnic_vsc_context_release(struct mlx5_nodnic_vsc_ctx *ctx)
+{
+ mlx5_nodnic_vsc_gw_unlock(ctx);
+}
+
+static int mlx5_nodnic_vsc_wait_on_flag(struct mlx5_nodnic_vsc_ctx *ctx,
+ u8 expected_val)
+{
+ int err, retries = 0;
+ u32 flag;
+
+ do {
+ if (retries > VSC_MAX_RETRIES)
+ return -EBUSY;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ &flag);
+ if (err)
+ return err;
+ flag = MLX5_NODNIC_EXTRACT(flag,
+ MLX5_NODNIC_VSC_FLAG_BIT_OFFS,
+ MLX5_NODNIC_VSC_FLAG_BIT_LEN);
+ retries++;
+
+ if ((retries & 0xf) == 0)
+ usleep_range(1000, 2000);
+
+ } while (flag != expected_val);
+
+ return 0;
+}
+
+static int mlx5_nodnic_vsc_read(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ /* Write address and CLEAR flag and syndrome.
+ * We mask the real_address to ensure the Flag and Syndrome bit
+ * positions are 0x0. Assuming MLX5_NODNIC_VSC_SYND_BIT_OFFS is
+ * the start of the flag/syndrome field.
+ */
+ u32 mask = ~(((1U << MLX5_NODNIC_VSC_FLAG_BIT_LEN) - 1) <<
+ MLX5_NODNIC_VSC_SYND_BIT_OFFS) &
+ ~(((1U << MLX5_NODNIC_VSC_SYND_BIT_LEN) - 1) <<
+ (MLX5_NODNIC_VSC_SYND_BIT_OFFS +
+ MLX5_NODNIC_VSC_FLAG_BIT_LEN));
+
+ /* Apply the mask to clear the bits */
+ address &= mask;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ address);
+ if (err)
+ goto out;
+
+ err = mlx5_nodnic_vsc_wait_on_flag(ctx, 1);
+ if (err)
+ goto out;
+
+ err = mlx5_nodnic_vsc_pci_read(ctx, MLX5_NODNIC_VSC_DATA_OFFSET, data);
+out:
+ return err;
+}
+
+static int mlx5_nodnic_vsc_write(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ u32 clear_mask;
+ int err;
+
+ err = mlx5_nodnic_vsc_pci_write(ctx, MLX5_NODNIC_VSC_DATA_OFFSET, data);
+ if (err)
+ goto out;
+
+ /* Prepare the address register.
+ * We must:
+ * 1. Set the Address bits.
+ * 2. Set the Flag bit to 0x1.
+ * 3. Clear the Syndrome bits to 0x0.
+ */
+
+ clear_mask = ~(((1U << MLX5_NODNIC_VSC_FLAG_BIT_LEN) - 1) <<
+ MLX5_NODNIC_VSC_FLAG_BIT_OFFS) &
+ ~(((1U << MLX5_NODNIC_VSC_SYND_BIT_LEN) - 1) <<
+ MLX5_NODNIC_VSC_SYND_BIT_OFFS);
+
+ address &= clear_mask;
+ address |= (1U << MLX5_NODNIC_VSC_FLAG_BIT_OFFS);
+
+ err = mlx5_nodnic_vsc_pci_write(ctx,
+ MLX5_NODNIC_VSC_ADDR_OFFSET,
+ address);
+ if (err)
+ return err;
+
+ /* Wait for the hardware to clear the flag to 0x0,
+ * indicating the write operation has completed.
+ */
+ err = mlx5_nodnic_vsc_wait_on_flag(ctx, 0);
+
+out:
+ return err;
+}
+
+static u32 mlx5_nodnic_le32_to_be32(u32 data)
+{
+ return cpu_to_be32(le32_to_cpu(data));
+}
+
+static u32 mlx5_nodnic_be32_to_le32(u32 data)
+{
+ return cpu_to_le32(be32_to_cpu(data));
+}
+
+int mlx5_nodnic_vsc_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read(ctx, address, data);
+ if (!err)
+ *data = mlx5_nodnic_le32_to_be32(*data);
+
+ return err;
+}
+
+int mlx5_nodnic_vsc_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read(ctx, address, data);
+ if (!err)
+ *data = le32_to_cpu(*data);
+
+ return err;
+}
+
+int mlx5_nodnic_vsc_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ data = mlx5_nodnic_be32_to_le32(data);
+
+ return mlx5_nodnic_vsc_write(ctx, address, data);
+}
+
+int mlx5_nodnic_vsc_cfg_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read(ctx, ctx->offset + address, data);
+ if (err)
+ return err;
+
+ *data = mlx5_nodnic_le32_to_be32(*data);
+
+ return 0;
+}
+
+int mlx5_nodnic_vsc_cfg_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ data = mlx5_nodnic_be32_to_le32(data);
+
+ return mlx5_nodnic_vsc_write(ctx, ctx->offset + address, data);
+}
+
+int mlx5_nodnic_vsc_cfg_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data)
+{
+ int err;
+
+ err = mlx5_nodnic_vsc_read(ctx, ctx->offset + address, data);
+ if (err)
+ return err;
+
+ *data = le32_to_cpu(*data);
+
+ return 0;
+}
+
+int mlx5_nodnic_vsc_cfg_write_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data)
+{
+ data = cpu_to_le32(data);
+
+ return mlx5_nodnic_vsc_write(ctx, ctx->offset + address, data);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
new file mode 100644
index 000000000000..b48ee7abf83f
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/nodnic_pci_vsc.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_PCI_VSC_H__
+#define __MLX5_NODNIC_PCI_VSC_H__
+
+#include <linux/pci.h>
+#include <linux/types.h>
+
+struct mlx5_nodnic_vsc_ctx;
+
+enum mlx5_nodnic_vsc_state {
+ MLX5_NODNIC_VSC_UNLOCK,
+ MLX5_NODNIC_VSC_LOCK,
+};
+
+enum {
+ MLX5_NODNIC_VSC_SPACE_NODNIC = 0x4,
+};
+
+struct mlx5_nodnic_vsc_ctx *mlx5_nodnic_vsc_init(struct pci_dev *pdev);
+void mlx5_nodnic_vsc_cleanup(struct mlx5_nodnic_vsc_ctx *ctx);
+int mlx5_nodnic_vsc_read_offset(struct mlx5_nodnic_vsc_ctx *ctx);
+int mlx5_nodnic_vsc_context_acquire(struct mlx5_nodnic_vsc_ctx *ctx);
+void mlx5_nodnic_vsc_context_release(struct mlx5_nodnic_vsc_ctx *ctx);
+
+/* base address - init segment copy in VSC */
+int mlx5_nodnic_vsc_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+int mlx5_nodnic_vsc_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data);
+int mlx5_nodnic_vsc_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+
+/* base address - nodnic configuration registers in VSC */
+int mlx5_nodnic_vsc_cfg_read_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+int mlx5_nodnic_vsc_cfg_write_be32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data);
+int mlx5_nodnic_vsc_cfg_read_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 *data);
+int mlx5_nodnic_vsc_cfg_write_cpu32(struct mlx5_nodnic_vsc_ctx *ctx,
+ unsigned int address, u32 data);
+
+#endif /* __MLX5_NODNIC_PCI_VSC_H__ */
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next 2/5] net/mlx5: nodnic: Implement data path
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
@ 2026-09-30 12:57 ` Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
From: Shahar Shitrit <shshitrit@nvidia.com>
Implement the TX and RX data paths for the mlx5_nodnic driver.
Add TX handling to build and post WQEs, DMA-map packets, ring the
BlueFlame doorbell, and process send completions. Handle SQ exhaustion
by stopping and waking the TX queue as needed.
Add page pool backed RX handling to allocate and post receive buffers,
process receive completions, build SKBs, and deliver packets through
NAPI/GRO.
Implement NAPI polling and CQE dispatch for TX and RX completions,
including RX WQE reposting and CQ re-arming.
Wire up ndo_start_xmit and ndo_features_check, and enable TX/RX
checksum offload support.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/nodnic/Makefile | 2 +-
.../net/ethernet/mellanox/mlx5/nodnic/en.c | 263 ++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 35 +++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 220 +++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 169 +++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.h | 25 ++
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 226 +++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/tx.h | 61 ++++
8 files changed, 1000 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
index 4955f0427c79..25865d7244d8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -5,4 +5,4 @@
obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
-mlx5_nodnic-y := main.o en_cfg.o nodnic_pci_vsc.o
+mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
new file mode 100644
index 000000000000..3572aff50194
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.c
@@ -0,0 +1,263 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/interrupt.h>
+#include <linux/mlx5/cq.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "rx.h"
+#include "tx.h"
+
+#define MLX5_NODNIC_TX_CQ_POLL_BUDGET 128
+
+enum mlx5_nodnic_cqe_result {
+ MLX5_NODNIC_CQE_TX,
+ MLX5_NODNIC_CQE_RX,
+ MLX5_NODNIC_CQE_IGNORED,
+};
+
+static const char *mlx5_nodnic_cqe_err_syndrome_str(u8 syndrome)
+{
+ switch (syndrome) {
+ case MLX5_CQE_SYNDROME_LOCAL_LENGTH_ERR:
+ return "LOCAL_LENGTH_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR:
+ return "LOCAL_QP_OP_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_PROT_ERR:
+ return "LOCAL_PROT_ERR";
+ case MLX5_CQE_SYNDROME_WR_FLUSH_ERR:
+ return "WR_FLUSH_ERR";
+ case MLX5_CQE_SYNDROME_MW_BIND_ERR:
+ return "MW_BIND_ERR";
+ case MLX5_CQE_SYNDROME_BAD_RESP_ERR:
+ return "BAD_RESP_ERR";
+ case MLX5_CQE_SYNDROME_LOCAL_ACCESS_ERR:
+ return "LOCAL_ACCESS_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR:
+ return "REMOTE_INVAL_REQ_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_ACCESS_ERR:
+ return "REMOTE_ACCESS_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_OP_ERR:
+ return "REMOTE_OP_ERR";
+ case MLX5_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR:
+ return "TRANSPORT_RETRY_EXC_ERR";
+ case MLX5_CQE_SYNDROME_RNR_RETRY_EXC_ERR:
+ return "RNR_RETRY_EXC_ERR";
+ case MLX5_CQE_SYNDROME_REMOTE_ABORTED_ERR:
+ return "REMOTE_ABORTED_ERR";
+ default:
+ return "UNKNOWN";
+ }
+}
+
+static void mlx5_nodnic_handle_err_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe64,
+ int napi_budget)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct mlx5_err_cqe *err_cqe = (struct mlx5_err_cqe *)cqe64;
+ u8 vendor_syndrome, syndrome;
+ const char *queue_type;
+ u16 wqe_counter;
+ u8 opcode;
+
+ opcode = err_cqe->op_own >> 4;
+ syndrome = err_cqe->syndrome;
+ vendor_syndrome = err_cqe->vendor_err_synd;
+ wqe_counter = be16_to_cpu(err_cqe->wqe_counter);
+
+ queue_type = (opcode == OPCODE_CQ_SEND_ERR) ? "SQ" : "RQ";
+
+ netdev_err(priv->netdev,
+ "%s CQE error: syndrome=0x%02x (%s), vendor_syndrome=0x%02x, wqe_counter=%u, cq_ci=%d\n",
+ queue_type,
+ syndrome, mlx5_nodnic_cqe_err_syndrome_str(syndrome),
+ vendor_syndrome, wqe_counter, cq->cc);
+
+ print_hex_dump(KERN_ERR, "mlx5_nodnic err CQE: ", DUMP_PREFIX_OFFSET,
+ 16, 1, err_cqe, sizeof(*err_cqe), false);
+
+ if (opcode == OPCODE_CQ_SEND_ERR) {
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ u16 sq_pi = mlx5_nodnic_sq_get_pi(sq, wqe_counter);
+
+ mlx5_nodnic_complete_send_wqe(sq, &sq->wqe_info[sq_pi],
+ napi_budget);
+ } else {
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u16 ci = wqe_counter & (rq->ring.num_entries - 1);
+
+ mlx5_nodnic_release_rx_wqe(rq, ci);
+ }
+}
+
+static u32 mlx5_nodnic_cq_get_ci(struct mlx5_nodnic_cq *cq)
+{
+ return cq->cc & (cq->num_entries - 1);
+}
+
+static u8 mlx5_nodnic_cq_get_sw_ownership_val(struct mlx5_nodnic_cq *cq)
+{
+ return (cq->cc >> cq->log_num_entries) & 1;
+}
+
+static struct mlx5_cqe64 *mlx5_nodnic_cq_get_cqe(struct mlx5_nodnic_cq *cq)
+{
+ u8 cqe_ownership_bit, sw_ownership_val;
+ struct mlx5_cqe64 *cqe;
+ u32 ci;
+
+ ci = mlx5_nodnic_cq_get_ci(cq);
+ cqe = (struct mlx5_cqe64 *)cq->ring.wq + ci;
+
+ cqe_ownership_bit = cqe->op_own & MLX5_NODNIC_CQE_OWNER_MASK;
+ sw_ownership_val = mlx5_nodnic_cq_get_sw_ownership_val(cq);
+ if (cqe_ownership_bit != sw_ownership_val)
+ return NULL;
+
+ /* ensure cqe content is read after cqe ownership bit */
+ dma_rmb();
+
+ return cqe;
+}
+
+static u8 mlx5_nodnic_cq_get_opcode(struct mlx5_cqe64 *cqe)
+{
+ return cqe->op_own >> 4;
+}
+
+static enum mlx5_nodnic_cqe_result
+mlx5_nodnic_handle_cqe(struct mlx5_nodnic_cq *cq, struct mlx5_cqe64 *cqe,
+ int budget)
+{
+ struct net_device *netdev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->netdev;
+ u8 opcode = mlx5_nodnic_cq_get_opcode(cqe);
+ enum mlx5_nodnic_cqe_result result;
+
+ switch (opcode) {
+ case OPCODE_CQ_SEND:
+ mlx5_nodnic_handle_send_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_TX;
+ break;
+
+ case OPCODE_CQ_RECV:
+ mlx5_nodnic_handle_recv_cqe(cq, cqe);
+ result = MLX5_NODNIC_CQE_RX;
+ break;
+
+ case OPCODE_CQ_SEND_ERR:
+ mlx5_nodnic_handle_err_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_TX;
+ break;
+
+ case OPCODE_CQ_RECV_ERR:
+ mlx5_nodnic_handle_err_cqe(cq, cqe, budget);
+ result = MLX5_NODNIC_CQE_RX;
+ break;
+
+ case OPCODE_CQ_INVALID:
+ /* Invalid/stale CQE - silently consume it.
+ * This can occur after device restart when stale CQEs
+ * from the previous session are still in the hardware CQ.
+ */
+ result = MLX5_NODNIC_CQE_IGNORED;
+ break;
+
+ default:
+ netdev_err(netdev,
+ "unexpected CQE opcode 0x%02x at cq_ci=%d\n",
+ opcode, cq->cc);
+ result = MLX5_NODNIC_CQE_IGNORED;
+ break;
+ }
+
+ cq->cc++;
+ return result;
+}
+
+void mlx5_nodnic_arm_cq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_core_dev *dev =
+ container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+ __be32 doorbell[2];
+ u32 sn, ci, val;
+
+ sn = cq->arm_sn & 3;
+ ci = cq->cc & 0xffffff;
+ val = sn << 28 | MLX5_CQ_DB_REQ_NOT | ci;
+
+ cq->dbr.db[0] = cpu_to_be32(ci);
+ /* arm_db is the second dword of the CQ doorbell record */
+ cq->dbr.db[1] = cpu_to_be32(val);
+
+ /* Make sure that the doorbell record in host memory is
+ * written before ringing the doorbell via PCI MMIO.
+ */
+ wmb();
+
+ doorbell[0] = cpu_to_be32(val);
+ doorbell[1] = cpu_to_be32(cq->cqn);
+
+ mlx5_write64(doorbell, dev->uar_base + MLX5_CQ_DOORBELL);
+}
+
+int mlx5_nodnic_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct mlx5_nodnic_cq *cq = container_of(napi,
+ struct mlx5_nodnic_cq, napi);
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ int tx_work_done = 0, rx_work_done = 0;
+ enum mlx5_nodnic_cqe_result result;
+ struct mlx5_cqe64 *cqe;
+
+ while (true) {
+ cqe = mlx5_nodnic_cq_get_cqe(cq);
+ if (!cqe)
+ break;
+
+ result = mlx5_nodnic_handle_cqe(cq, cqe, budget);
+ switch (result) {
+ case MLX5_NODNIC_CQE_TX:
+ if (++tx_work_done >= MLX5_NODNIC_TX_CQ_POLL_BUDGET)
+ goto poll_done;
+ break;
+ case MLX5_NODNIC_CQE_RX:
+ if (++rx_work_done >= budget)
+ goto poll_done;
+ break;
+ case MLX5_NODNIC_CQE_IGNORED:
+ break;
+ }
+ }
+
+poll_done:
+ if (!mlx5_nodnic_rq_post_wqes(priv))
+ return budget;
+
+ if (rx_work_done >= budget)
+ return budget;
+
+ if (napi_complete_done(napi, rx_work_done))
+ mlx5_nodnic_arm_cq(cq);
+
+ return rx_work_done;
+}
+
+netdev_features_t
+mlx5_nodnic_check_features(struct sk_buff *skb, struct net_device *netdev,
+ netdev_features_t features)
+{
+ features = vlan_features_check(skb, features);
+
+ /* Validate if the tunneled packet is being offloaded by HW */
+ if (skb->encapsulation && (features & NETIF_F_CSUM_MASK))
+ return features & ~NETIF_F_CSUM_MASK;
+
+ return features;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index 90dc57695e1b..19858e9da064 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -4,10 +4,12 @@
#ifndef __MLX5_NODNIC_EN_H__
#define __MLX5_NODNIC_EN_H__
+#include <net/page_pool/helpers.h>
#include <linux/if_vlan.h>
#include <linux/mlx5/device.h>
#include <linux/mutex.h>
#include <linux/netdevice.h>
+#include <linux/skbuff.h>
#include <linux/timer.h>
#include <linux/types.h>
@@ -16,16 +18,32 @@
#define MLX5_NODNIC_SQ_WQEBB_B 64
#define MLX5_NODNIC_RQ_STRIDE_BYTES 16
+#define MLX5_NODNIC_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN)
+#define MLX5_NODNIC_HW_MTU (ETH_DATA_LEN + MLX5_NODNIC_ETH_HARD_MTU)
+#define MLX5_NODNIC_SKB_HEADROOM_BYTES NET_SKB_PAD
+
enum {
MLX5_NODNIC_STATE_OPENED,
};
+enum {
+ MLX5_NODNIC_CQE_OWNER_MASK = 1,
+};
+
enum {
MLX5_NODNIC_RCV_DBR,
MLX5_NODNIC_SND_DBR,
MLX5_NODNIC_DBR_WORDS,
};
+enum mlx5_nodnic_cqe_opcode {
+ OPCODE_CQ_SEND,
+ OPCODE_CQ_RECV = 0x2,
+ OPCODE_CQ_SEND_ERR = 0xD,
+ OPCODE_CQ_RECV_ERR,
+ OPCODE_CQ_INVALID,
+};
+
struct mlx5_nodnic_ring {
void *wq;
dma_addr_t dma_handle;
@@ -35,17 +53,24 @@ struct mlx5_nodnic_ring {
u32 lkey;
};
+struct mlx5_nodnic_tx_wqe_info;
+
struct mlx5_nodnic_sq {
struct device *device;
struct net_device *netdev;
+ struct netdev_queue *txq;
struct mlx5_nodnic_ring ring;
+ struct mlx5_nodnic_tx_wqe_info *wqe_info;
+
u16 pc;
u16 cc;
};
struct mlx5_nodnic_rq {
+ struct page_pool *page_pool;
struct mlx5_nodnic_ring ring;
+ struct page **pages;
u16 pc;
u16 cc;
};
@@ -62,9 +87,13 @@ struct mlx5_nodnic_cq {
int num_entries;
int log_num_entries;
+ int irqn;
struct mlx5_nodnic_dbr dbr;
+ unsigned int arm_sn;
int cc;
+
+ struct napi_struct napi;
};
/* for FW use only */
@@ -94,4 +123,10 @@ void mlx5_nodnic_cleanup_netdev(struct mlx5_nodnic_core_dev *dev);
int mlx5_nodnic_open(struct net_device *netdev);
int mlx5_nodnic_close(struct net_device *netdev);
+void mlx5_nodnic_arm_cq(struct mlx5_nodnic_cq *cq);
+int mlx5_nodnic_napi_poll(struct napi_struct *napi, int budget);
+netdev_features_t mlx5_nodnic_check_features(struct sk_buff *skb,
+ struct net_device *netdev,
+ netdev_features_t features);
+
#endif /* __MLX5_NODNIC_EN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index 75dbd2071110..d73e2ec1662b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -15,6 +15,8 @@
#include "nodnic_ifc.h"
#include "nodnic.h"
#include "nodnic_pci_vsc.h"
+#include "rx.h"
+#include "tx.h"
#define MLX5_NODNIC_CQ_INIT_CMD_SN cpu_to_be32(2 << 28)
@@ -347,6 +349,46 @@ static int mlx5_nodnic_read_cq_n(struct mlx5_nodnic_core_dev *dev)
return 0;
}
+static irqreturn_t mlx5_nodnic_cq_irq_handler(int irq, void *data)
+{
+ struct mlx5_nodnic_cq *cq = data;
+
+ cq->arm_sn++;
+ napi_schedule_irqoff(&cq->napi);
+
+ return IRQ_HANDLED;
+}
+
+static int mlx5_nodnic_setup_data_irq(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct net_device *netdev = priv->netdev;
+ int irqn, err;
+
+ irqn = pci_irq_vector(priv->core_dev->pdev, MLX5_NODNIC_DATA_MSIX);
+ if (irqn < 0) {
+ netdev_err(netdev, "pci_irq_vector failed: %d\n", irqn);
+ return irqn;
+ }
+
+ err = request_irq(irqn, mlx5_nodnic_cq_irq_handler, 0,
+ "mlx5_nodnic-cq", cq);
+ if (err) {
+ netdev_err(netdev, "request_irq failed\n");
+ return err;
+ }
+ cq->irqn = irqn;
+
+ return 0;
+}
+
+static void mlx5_nodnic_cleanup_data_irq(struct mlx5_nodnic_cq *cq)
+{
+ free_irq(cq->irqn, cq);
+}
+
int mlx5_nodnic_init_working_buffer(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_ring *work_buf = &dev->priv->working_buffer;
@@ -377,6 +419,7 @@ int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
int err;
cq->cc = 0;
+ cq->arm_sn = 0;
err = mlx5_nodnic_init_cq_ring(dev);
if (err)
@@ -390,8 +433,19 @@ int mlx5_nodnic_open_cq(struct mlx5_nodnic_core_dev *dev)
if (err)
goto err_cleanup_dbr;
+ netif_napi_add_config_locked(dev->priv->netdev, &cq->napi,
+ mlx5_nodnic_napi_poll, 0);
+
+ err = mlx5_nodnic_setup_data_irq(cq);
+ if (err)
+ goto err_napi_del;
+
+ netif_napi_set_irq_locked(&cq->napi, cq->irqn);
+
return 0;
+err_napi_del:
+ netif_napi_del_locked(&cq->napi);
err_cleanup_dbr:
mlx5_nodnic_cleanup_cq_dbr(dev);
err_cleanup_ring:
@@ -404,6 +458,8 @@ static void mlx5_nodnic_close_cq(struct mlx5_nodnic_cq *cq)
struct mlx5_nodnic_core_dev *dev =
container_of(cq, struct mlx5_nodnic_priv, cq)->core_dev;
+ mlx5_nodnic_cleanup_data_irq(cq);
+ netif_napi_del_locked(&cq->napi);
mlx5_nodnic_cleanup_cq_dbr(dev);
mlx5_nodnic_cleanup_cq_ring(dev);
}
@@ -428,14 +484,28 @@ int mlx5_nodnic_open_sq(struct mlx5_nodnic_core_dev *dev)
ring->lkey = dev->lkey;
+ sq->wqe_info = kvcalloc(ring->num_entries,
+ sizeof(struct mlx5_nodnic_tx_wqe_info),
+ GFP_KERNEL);
+ if (!sq->wqe_info) {
+ err = -ENOMEM;
+ goto err_free_ring;
+ }
+
return 0;
+
+err_free_ring:
+ mlx5_nodnic_free_ring(dev, ring);
+ return err;
}
static void mlx5_nodnic_close_sq(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_sq *sq = &dev->priv->sq;
+ mlx5_nodnic_sq_free_pending(sq);
mlx5_nodnic_free_ring(dev, &sq->ring);
+ kvfree(sq->wqe_info);
}
int mlx5_nodnic_open_rq(struct mlx5_nodnic_core_dev *dev)
@@ -573,6 +643,143 @@ void mlx5_nodnic_refresh_carrier(struct mlx5_nodnic_priv *priv)
mutex_unlock(&priv->state_lock);
}
+static int mlx5_nodnic_activate_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct net_device *netdev = priv->netdev;
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ struct page_pool_params pp_params = {
+ .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV,
+ .order = 0,
+ .pool_size = rq->ring.num_entries,
+ .nid = dev_to_node(dev->device),
+ .dev = dev->device,
+ .napi = &priv->cq.napi,
+ .netdev = netdev,
+ .dma_dir = DMA_FROM_DEVICE,
+ .offset = MLX5_NODNIC_SKB_HEADROOM_BYTES,
+ .max_len = MLX5_NODNIC_HW_MTU,
+ };
+ int err;
+
+ rq->page_pool = page_pool_create(&pp_params);
+ if (IS_ERR(rq->page_pool)) {
+ err = PTR_ERR(rq->page_pool);
+ return err;
+ }
+
+ err = mlx5_nodnic_rq_alloc_pages(rq);
+ if (err) {
+ netdev_err(netdev, "failed to alloc RX pages, err=%d\n", err);
+ goto err_destroy_page_pool;
+ }
+
+ netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_RX, &priv->cq.napi);
+
+ return 0;
+
+err_destroy_page_pool:
+ page_pool_destroy(rq->page_pool);
+
+ return err;
+}
+
+static void mlx5_nodnic_deactivate_rq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct net_device *netdev = dev->priv->netdev;
+ struct mlx5_nodnic_rq *rq = &dev->priv->rq;
+
+ netif_queue_set_napi(netdev, 0, NETDEV_QUEUE_TYPE_RX, NULL);
+
+ mlx5_nodnic_rq_free_pages(rq);
+ page_pool_destroy(rq->page_pool);
+}
+
+static void mlx5_nodnic_tx_disable_queue(struct netdev_queue *txq)
+{
+ __netif_tx_lock_bh(txq);
+ netif_tx_stop_queue(txq);
+ __netif_tx_unlock_bh(txq);
+}
+
+static void mlx5_nodnic_activate_sq(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_priv *priv = dev->priv;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+
+ sq->txq = netdev_get_tx_queue(priv->netdev, 0);
+ netdev_tx_reset_queue(sq->txq);
+ netif_tx_start_queue(sq->txq);
+ netif_queue_set_napi(sq->netdev, 0, NETDEV_QUEUE_TYPE_TX,
+ &priv->cq.napi);
+}
+
+static void mlx5_nodnic_deactivate_sq(struct mlx5_nodnic_sq *sq)
+{
+ netif_queue_set_napi(sq->netdev, 0, NETDEV_QUEUE_TYPE_TX, NULL);
+ mlx5_nodnic_tx_disable_queue(sq->txq);
+}
+
+static void mlx5_nodnic_init_cqes_ownership(struct mlx5_nodnic_cq *cq)
+{
+ struct mlx5_cqe64 *cqe;
+
+ for (int i = 0; i < cq->num_entries; i++) {
+ cqe = (struct mlx5_cqe64 *)
+ ((u8 *)cq->ring.wq + i * MLX5_NODNIC_CQE_SIZE_B);
+ cqe->op_own = 0xf1;
+ }
+}
+
+static int mlx5_nodnic_activate_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_cq *cq = &dev->priv->cq;
+ int err;
+
+ napi_enable_locked(&cq->napi);
+
+ mlx5_nodnic_init_cqes_ownership(cq);
+
+ err = mlx5_nodnic_activate_rq(dev);
+ if (err)
+ goto err_napi;
+
+ err = mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx);
+ if (err)
+ goto err_rq;
+
+ err = mlx5_nodnic_enable_port(dev);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ if (err)
+ goto err_rq;
+
+ mlx5_nodnic_arm_cq(cq);
+ napi_schedule(&cq->napi);
+
+ mlx5_nodnic_activate_sq(dev);
+
+ return 0;
+
+err_rq:
+ mlx5_nodnic_deactivate_rq(dev);
+err_napi:
+ napi_disable_locked(&cq->napi);
+
+ return err;
+}
+
+static void mlx5_nodnic_deactivate_queues(struct mlx5_nodnic_core_dev *dev)
+{
+ if (!mlx5_nodnic_vsc_context_acquire(dev->vsc_ctx)) {
+ mlx5_nodnic_disable_port(dev);
+ mlx5_nodnic_vsc_context_release(dev->vsc_ctx);
+ }
+
+ mlx5_nodnic_deactivate_sq(&dev->priv->sq);
+ napi_disable_locked(&dev->priv->cq.napi);
+ mlx5_nodnic_deactivate_rq(dev);
+}
+
int mlx5_nodnic_enable_port(struct mlx5_nodnic_core_dev *dev)
{
struct mlx5_nodnic_vsc_ctx *vsc_ctx = dev->vsc_ctx;
@@ -692,6 +899,10 @@ static int mlx5_nodnic_open_locked(struct net_device *netdev)
set_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ err = mlx5_nodnic_activate_queues(dev);
+ if (err)
+ goto err_close;
+
err = mlx5_nodnic_refresh_carrier_locked(dev->priv);
if (err)
netdev_err(dev->priv->netdev,
@@ -699,6 +910,9 @@ static int mlx5_nodnic_open_locked(struct net_device *netdev)
return 0;
+err_close:
+ clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
+ mlx5_nodnic_close_queues(dev);
err_uar:
mlx5_nodnic_cleanup_uar_map(dev);
return err;
@@ -721,6 +935,7 @@ static int mlx5_nodnic_close_locked(struct net_device *netdev)
struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ mlx5_nodnic_deactivate_queues(dev);
clear_bit(MLX5_NODNIC_STATE_OPENED, &priv->state);
netif_carrier_off(priv->netdev);
netdev_info(priv->netdev, "Link down\n");
@@ -751,6 +966,8 @@ int mlx5_nodnic_close(struct net_device *netdev)
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
+ .ndo_start_xmit = mlx5_nodnic_xmit,
+ .ndo_features_check = mlx5_nodnic_check_features,
};
void mlx5_nodnic_build_netdev(struct net_device *netdev)
@@ -765,6 +982,9 @@ void mlx5_nodnic_build_netdev(struct net_device *netdev)
netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
netdev->request_ops_lock = true;
+ netdev->hw_features = NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
+ netdev->features = netdev->hw_features;
+
eth_hw_addr_set(netdev, core_dev->mac_address);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
new file mode 100644
index 000000000000..7736a1f24ea5
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
@@ -0,0 +1,169 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/errno.h>
+#include <linux/if_vlan.h>
+#include <linux/kernel.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "rx.h"
+
+int mlx5_nodnic_rq_alloc_pages(struct mlx5_nodnic_rq *rq)
+{
+ int entries = rq->ring.num_entries;
+ struct page *page;
+
+ rq->pages = kvcalloc(rq->ring.num_entries, sizeof(*rq->pages),
+ GFP_KERNEL);
+ if (!rq->pages)
+ return -ENOMEM;
+
+ for (int i = 0; i < entries; i++) {
+ page = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!page))
+ goto err_free_pages;
+
+ rq->pages[i] = page;
+ }
+
+ return 0;
+
+err_free_pages:
+ mlx5_nodnic_rq_free_pages(rq);
+ return -ENOMEM;
+}
+
+void mlx5_nodnic_rq_free_pages(struct mlx5_nodnic_rq *rq)
+{
+ u32 i;
+
+ for (i = 0; i < rq->ring.num_entries; i++) {
+ if (!rq->pages[i])
+ continue;
+
+ page_pool_put_full_page(rq->page_pool,
+ rq->pages[i], false);
+ }
+
+ kvfree(rq->pages);
+}
+
+static void mlx5_nodnic_rq_push_db(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+
+ /* ensure wqes are visible to device before updating dbr */
+ dma_wmb();
+
+ /* update local DB record */
+ priv->dbr.db[MLX5_NODNIC_RCV_DBR] = cpu_to_be32(rq->pc & 0xffff);
+}
+
+static u16 mlx5_nodnic_rq_get_pi(struct mlx5_nodnic_rq *rq)
+{
+ return rq->pc & (rq->ring.num_entries - 1);
+}
+
+static int mlx5_nodnic_rq_post_one(struct mlx5_nodnic_rq *rq)
+{
+ u16 pi = mlx5_nodnic_rq_get_pi(rq);
+ struct mlx5_nodnic_rx_wqe *wqe;
+
+ if (!rq->pages[pi]) {
+ rq->pages[pi] = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!rq->pages[pi]))
+ return -ENOMEM;
+ }
+
+ wqe = (struct mlx5_nodnic_rx_wqe *)
+ ((u8 *)rq->ring.wq + pi * MLX5_NODNIC_RQ_STRIDE_BYTES);
+ wqe->byte_count = cpu_to_be32(MLX5_NODNIC_HW_MTU);
+ wqe->lkey = cpu_to_be32(rq->ring.lkey);
+ wqe->addr = cpu_to_be64(page_pool_get_dma_addr(rq->pages[pi]) +
+ MLX5_NODNIC_SKB_HEADROOM_BYTES);
+ rq->pc++;
+
+ return 0;
+}
+
+bool mlx5_nodnic_rq_post_wqes(struct mlx5_nodnic_priv *priv)
+{
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u32 avail;
+ int i;
+
+ avail = rq->ring.num_entries - (u16)(rq->pc - rq->cc);
+ for (i = 0; i < avail; i++) {
+ if (mlx5_nodnic_rq_post_one(rq))
+ break;
+ }
+
+ if (i)
+ mlx5_nodnic_rq_push_db(priv);
+
+ return i == avail;
+}
+
+static void nodnic_handle_rx_csum(struct net_device *netdev,
+ const struct mlx5_cqe64 *cqe,
+ struct sk_buff *skb)
+{
+ skb->ip_summed = CHECKSUM_NONE;
+
+ if (unlikely(!(netdev->features & NETIF_F_RXCSUM)))
+ return;
+
+ if (likely((cqe->hds_ip_ext & CQE_L3_OK) &&
+ (cqe->hds_ip_ext & CQE_L4_OK)))
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+}
+
+void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ u32 byte_count = be32_to_cpu(cqe->byte_cnt);
+ struct mlx5_nodnic_rq *rq = &priv->rq;
+ u16 wqe_counter, ci;
+ struct sk_buff *skb;
+ struct page *page;
+
+ wqe_counter = be16_to_cpu(cqe->wqe_counter);
+ ci = wqe_counter & (rq->ring.num_entries - 1);
+ page = rq->pages[ci];
+
+ page_pool_dma_sync_for_cpu(rq->page_pool, page, 0, byte_count);
+
+ skb = napi_build_skb(page_address(page), PAGE_SIZE);
+ if (unlikely(!skb))
+ goto err_recycle;
+
+ skb_mark_for_recycle(skb);
+ rq->pages[ci] = NULL;
+
+ skb_reserve(skb, MLX5_NODNIC_SKB_HEADROOM_BYTES);
+ skb_put(skb, byte_count);
+ skb->protocol = eth_type_trans(skb, priv->netdev);
+ nodnic_handle_rx_csum(priv->netdev, cqe, skb);
+
+ napi_gro_receive(&cq->napi, skb);
+ rq->cc++;
+
+ return;
+
+err_recycle:
+ mlx5_nodnic_release_rx_wqe(rq, ci);
+}
+
+void mlx5_nodnic_release_rx_wqe(struct mlx5_nodnic_rq *rq, u16 ci)
+{
+ page_pool_put_full_page(rq->page_pool, rq->pages[ci], true);
+ rq->pages[ci] = NULL;
+ rq->cc++;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
new file mode 100644
index 000000000000..ccdfa3e69824
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_RX_H__
+#define __MLX5_NODNIC_RX_H__
+
+struct mlx5_nodnic_priv;
+struct mlx5_nodnic_cq;
+struct mlx5_nodnic_rq;
+struct mlx5_cqe64;
+
+struct mlx5_nodnic_rx_wqe {
+ __be32 byte_count;
+ __be32 lkey;
+ __be64 addr;
+};
+
+int mlx5_nodnic_rq_alloc_pages(struct mlx5_nodnic_rq *rq);
+void mlx5_nodnic_rq_free_pages(struct mlx5_nodnic_rq *rq);
+bool mlx5_nodnic_rq_post_wqes(struct mlx5_nodnic_priv *priv);
+void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe);
+void mlx5_nodnic_release_rx_wqe(struct mlx5_nodnic_rq *rq, u16 ci);
+
+#endif /* __MLX5_NODNIC_RX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
new file mode 100644
index 000000000000..fcde1a9086f7
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/mlx5/doorbell.h>
+#include <linux/net.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic.h"
+#include "tx.h"
+
+#define MLX5_NODNIC_OPCODE_SEND 0x0a
+#define MLX5_NODNIC_DS_COUNT 3
+#define MLX5_NODNIC_CE_ALWAYS 0x2
+#define MLX5_NODNIC_SQ_STOP_ROOM 1
+
+static void mlx5_nodnic_tx_build_wqe_eseg(struct sk_buff *skb,
+ struct mlx5_nodnic_wqe_eth_seg *eseg)
+{
+ u8 flags = 0;
+
+ if (likely(skb->ip_summed == CHECKSUM_PARTIAL))
+ flags |= MLX5_NODNIC_ETH_WQE_L3_CSUM |
+ MLX5_NODNIC_ETH_WQE_L4_CSUM;
+
+ eseg->cs_flags_mss = cpu_to_be32((u32)flags << 24);
+ eseg->inline_hdr_sz = 0;
+}
+
+static inline void
+mlx5_nodnic_tx_build_wqe_ctrl(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_wqe_ctrl_seg *ctrl)
+{
+ ctrl->opmod_idx_opcode = cpu_to_be32((sq->pc << 8) |
+ MLX5_NODNIC_OPCODE_SEND);
+ ctrl->qpn_ds = cpu_to_be32((sq->ring.qn << 8) | MLX5_NODNIC_DS_COUNT);
+ ctrl->fm_ce_se = cpu_to_be32(MLX5_NODNIC_CE_ALWAYS << 2);
+ ctrl->general_id = 0;
+}
+
+static void mlx5_nodnic_tx_fill_wqe_info(struct mlx5_nodnic_sq *sq,
+ struct sk_buff *skb,
+ dma_addr_t dma_addr)
+{
+ u16 pi = mlx5_nodnic_sq_get_pi(sq, sq->pc);
+
+ sq->wqe_info[pi].skb = skb;
+ sq->wqe_info[pi].num_bytes = skb->len;
+ sq->wqe_info[pi].dma = dma_addr;
+}
+
+static int
+mlx5_nodnic_tx_build_wqe_ds(struct mlx5_nodnic_sq *sq, struct sk_buff *skb,
+ struct mlx5_nodnic_wqe_data_seg *dseg)
+{
+ dma_addr_t dma_addr = 0;
+
+ dseg->byte_count = cpu_to_be32(skb->len & 0x7FFFFFFF);
+ dseg->lkey = cpu_to_be32(sq->ring.lkey);
+ dma_addr = dma_map_single(sq->device, skb->data, skb->len,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(sq->device, dma_addr)))
+ return -ENOMEM;
+
+ dseg->addr = cpu_to_be64(dma_addr);
+
+ mlx5_nodnic_tx_fill_wqe_info(sq, skb, dma_addr);
+
+ return 0;
+}
+
+static __always_inline int
+mlx5_nodnic_tx_write_wqe(struct mlx5_nodnic_sq *sq, struct sk_buff *skb,
+ struct mlx5_nodnic_tx_wqe *wqe)
+{
+ int err;
+
+ memset(wqe, 0, MLX5_NODNIC_SQ_WQEBB_B);
+ mlx5_nodnic_tx_build_wqe_ctrl(sq, &wqe->ctrl);
+ mlx5_nodnic_tx_build_wqe_eseg(skb, &wqe->eth);
+ err = mlx5_nodnic_tx_build_wqe_ds(sq, skb, &wqe->data);
+ if (err)
+ return err;
+
+ /* Ensure WQE stores are visible to the device before ringing the db */
+ dma_wmb();
+
+ return 0;
+}
+
+static int mlx5_nodnic_tx_is_sq_full(struct mlx5_nodnic_sq *sq)
+{
+ u16 pending = (u16)(sq->pc - sq->cc);
+
+ return pending == sq->ring.num_entries - MLX5_NODNIC_SQ_STOP_ROOM;
+}
+
+static int mlx5_nodnic_tx_get_next_sq_offset(struct mlx5_nodnic_sq *sq)
+{
+ u16 pi;
+
+ pi = mlx5_nodnic_sq_get_pi(sq, sq->pc);
+ return pi * MLX5_NODNIC_SQ_WQEBB_B;
+}
+
+static void mlx5_nodnic_tx_notify_hw(struct mlx5_nodnic_priv *priv,
+ void __iomem *uar_base, u16 pc,
+ struct mlx5_nodnic_wqe_ctrl_seg *ctrl)
+{
+ /* ensure wqe is visible to device before updating doorbell record */
+ dma_wmb();
+
+ priv->dbr.db[MLX5_NODNIC_SND_DBR] = cpu_to_be32(pc);
+
+ /* ensure doorbell record is visible to device before ringing the
+ * doorbell
+ */
+ wmb();
+
+ mlx5_write64((__be32 *)ctrl, uar_base + MLX5_BF_OFFSET);
+}
+
+netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *netdev)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_tx_wqe *wqe;
+ u16 next_offset;
+ int err;
+
+ if (unlikely(mlx5_nodnic_tx_is_sq_full(sq)))
+ netif_tx_stop_queue(sq->txq);
+
+ next_offset = mlx5_nodnic_tx_get_next_sq_offset(sq);
+
+ wqe = (struct mlx5_nodnic_tx_wqe *)((u8 *)sq->ring.wq + next_offset);
+
+ err = mlx5_nodnic_tx_write_wqe(sq, skb, wqe);
+ if (err) {
+ dev_kfree_skb_any(skb);
+
+ return NETDEV_TX_OK;
+ }
+
+ sq->pc += 1;
+
+ netdev_tx_sent_queue(sq->txq, skb->len);
+
+ mlx5_nodnic_tx_notify_hw(priv, dev->uar_base, sq->pc, &wqe->ctrl);
+
+ return NETDEV_TX_OK;
+}
+
+u16 mlx5_nodnic_sq_get_pi(struct mlx5_nodnic_sq *sq, u16 pc)
+{
+ return pc & (sq->ring.num_entries - 1);
+}
+
+void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe, int napi_budget)
+{
+ struct mlx5_nodnic_priv *priv = container_of(cq,
+ struct mlx5_nodnic_priv,
+ cq);
+ struct mlx5_nodnic_sq *sq = &priv->sq;
+ struct mlx5_nodnic_tx_wqe_info *wi;
+ u16 pi;
+
+ pi = mlx5_nodnic_sq_get_pi(sq, be16_to_cpu(cqe->wqe_counter));
+ wi = &sq->wqe_info[pi];
+
+ mlx5_nodnic_complete_send_wqe(sq, wi, napi_budget);
+}
+
+void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_tx_wqe_info *wi,
+ int napi_budget)
+{
+ struct netdev_queue *txq = sq->txq;
+
+ dma_unmap_single(sq->device, wi->dma, wi->num_bytes,
+ DMA_TO_DEVICE);
+ napi_consume_skb(wi->skb, napi_budget);
+
+ netdev_tx_completed_queue(txq, 1, wi->num_bytes);
+
+ sq->cc++;
+
+ if (netif_tx_queue_stopped(txq))
+ netif_tx_wake_queue(txq);
+}
+
+void mlx5_nodnic_sq_free_pending(struct mlx5_nodnic_sq *sq)
+{
+ struct mlx5_nodnic_tx_wqe_info *wi;
+ unsigned int completed_bytes = 0;
+ unsigned int completed_pkts = 0;
+ u16 sqcc = sq->cc;
+ u16 ci;
+
+ while (sqcc != sq->pc) {
+ ci = mlx5_nodnic_sq_get_pi(sq, sqcc);
+ wi = &sq->wqe_info[ci];
+
+ dma_unmap_single(sq->device, wi->dma,
+ wi->num_bytes, DMA_TO_DEVICE);
+
+ completed_pkts++;
+ completed_bytes += wi->num_bytes;
+
+ dev_kfree_skb_any(wi->skb);
+
+ sqcc++;
+ }
+
+ if (completed_pkts)
+ netdev_tx_completed_queue(sq->txq, completed_pkts,
+ completed_bytes);
+ sq->cc = sqcc;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
new file mode 100644
index 000000000000..b93a27dffb92
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#ifndef __MLX5_NODNIC_TX_H__
+#define __MLX5_NODNIC_TX_H__
+
+#include <linux/netdevice.h>
+#include <linux/types.h>
+#include <linux/skbuff.h>
+
+struct mlx5_nodnic_sq;
+struct mlx5_nodnic_cq;
+struct mlx5_cqe64;
+
+struct mlx5_nodnic_tx_wqe_info {
+ struct sk_buff *skb;
+ u32 num_bytes;
+ dma_addr_t dma;
+};
+
+enum {
+ MLX5_NODNIC_ETH_WQE_L3_CSUM = 1 << 6,
+ MLX5_NODNIC_ETH_WQE_L4_CSUM = 1 << 7,
+};
+
+struct mlx5_nodnic_wqe_ctrl_seg {
+ __be32 opmod_idx_opcode;
+ __be32 qpn_ds;
+ __be32 fm_ce_se;
+ __be32 general_id;
+} __packed;
+
+struct mlx5_nodnic_wqe_eth_seg {
+ __be32 swp_offsets;
+ __be32 cs_flags_mss;
+ __be32 flow_metadata;
+ __be32 inline_hdr_sz;
+} __packed;
+
+struct mlx5_nodnic_wqe_data_seg {
+ __be32 byte_count;
+ __be32 lkey;
+ __be64 addr;
+} __packed;
+
+struct mlx5_nodnic_tx_wqe {
+ struct mlx5_nodnic_wqe_ctrl_seg ctrl;
+ struct mlx5_nodnic_wqe_eth_seg eth;
+ struct mlx5_nodnic_wqe_data_seg data;
+} __packed;
+
+netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *dev);
+u16 mlx5_nodnic_sq_get_pi(struct mlx5_nodnic_sq *sq, u16 pc);
+void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
+ struct mlx5_cqe64 *cqe, int napi_budget);
+void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
+ struct mlx5_nodnic_tx_wqe_info *wi,
+ int napi_budget);
+void mlx5_nodnic_sq_free_pending(struct mlx5_nodnic_sq *sq);
+
+#endif /* __MLX5_NODNIC_TX_H__ */
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next 3/5] net/mlx5: nodnic: Add TX/RX statistics
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
@ 2026-09-30 12:57 ` Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
4 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
From: Shahar Shitrit <shshitrit@nvidia.com>
Add TX and RX packet and byte statistics, as well as TX drop
accounting.
Update the counters from the data path and expose them through
ndo_get_stats64.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 14 +++++++++++
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 25 +++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/rx.c | 3 +++
.../net/ethernet/mellanox/mlx5/nodnic/tx.c | 4 +++
4 files changed, 46 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index 19858e9da064..d811bfe242ac 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -44,6 +44,17 @@ enum mlx5_nodnic_cqe_opcode {
OPCODE_CQ_INVALID,
};
+struct mlx5_nodnic_sq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 dropped;
+};
+
+struct mlx5_nodnic_rq_stats {
+ u64 packets;
+ u64 bytes;
+};
+
struct mlx5_nodnic_ring {
void *wq;
dma_addr_t dma_handle;
@@ -109,6 +120,9 @@ struct mlx5_nodnic_priv {
struct mutex state_lock; /* protects Interface state */
unsigned long state;
+ struct mlx5_nodnic_sq_stats sq_stats;
+ struct mlx5_nodnic_rq_stats rq_stats;
+
struct mlx5_nodnic_sq sq;
struct mlx5_nodnic_rq rq;
struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index d73e2ec1662b..9f3ec04e7a7b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -963,10 +963,35 @@ int mlx5_nodnic_close(struct net_device *netdev)
return err;
}
+static void mlx5_nodnic_update_sq_stats(const struct mlx5_nodnic_sq_stats *sq,
+ struct rtnl_link_stats64 *s)
+{
+ s->tx_packets = sq->packets;
+ s->tx_bytes = sq->bytes;
+ s->tx_dropped = sq->dropped;
+}
+
+static void mlx5_nodnic_update_rq_stats(const struct mlx5_nodnic_rq_stats *rq,
+ struct rtnl_link_stats64 *s)
+{
+ s->rx_packets = rq->packets;
+ s->rx_bytes = rq->bytes;
+}
+
+static void mlx5_nodnic_get_stats64(struct net_device *netdev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+
+ mlx5_nodnic_update_sq_stats(&priv->sq_stats, stats);
+ mlx5_nodnic_update_rq_stats(&priv->rq_stats, stats);
+}
+
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
.ndo_start_xmit = mlx5_nodnic_xmit,
+ .ndo_get_stats64 = mlx5_nodnic_get_stats64,
.ndo_features_check = mlx5_nodnic_check_features,
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
index 7736a1f24ea5..808740ec3910 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/rx.c
@@ -152,6 +152,9 @@ void mlx5_nodnic_handle_recv_cqe(struct mlx5_nodnic_cq *cq,
skb->protocol = eth_type_trans(skb, priv->netdev);
nodnic_handle_rx_csum(priv->netdev, cqe, skb);
+ priv->rq_stats.packets++;
+ priv->rq_stats.bytes += byte_count;
+
napi_gro_receive(&cq->napi, skb);
rq->cc++;
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
index fcde1a9086f7..b6f2368155ce 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/tx.c
@@ -143,6 +143,7 @@ netdev_tx_t mlx5_nodnic_xmit(struct sk_buff *skb, struct net_device *netdev)
err = mlx5_nodnic_tx_write_wqe(sq, skb, wqe);
if (err) {
+ priv->sq_stats.dropped++;
dev_kfree_skb_any(skb);
return NETDEV_TX_OK;
@@ -176,6 +177,9 @@ void mlx5_nodnic_handle_send_cqe(struct mlx5_nodnic_cq *cq,
wi = &sq->wqe_info[pi];
mlx5_nodnic_complete_send_wqe(sq, wi, napi_budget);
+
+ priv->sq_stats.packets++;
+ priv->sq_stats.bytes += wi->num_bytes;
}
void mlx5_nodnic_complete_send_wqe(struct mlx5_nodnic_sq *sq,
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next 4/5] net/mlx5: nodnic: Add ethtool driver info
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
` (2 preceding siblings ...)
2026-09-30 12:57 ` [PATCH net-next 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
@ 2026-09-30 12:57 ` Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
4 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
From: Shahar Shitrit <shshitrit@nvidia.com>
Implement the get_drvinfo ethtool operation to report the driver name,
firmware version, and PCI bus information.
Read the firmware revision from the initialization segment and expose
it together with the driver information through ethtool.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/nodnic/en_cfg.c | 34 +++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
index 9f3ec04e7a7b..88a59d7ad7ed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en_cfg.c
@@ -987,6 +987,39 @@ static void mlx5_nodnic_get_stats64(struct net_device *netdev,
mlx5_nodnic_update_rq_stats(&priv->rq_stats, stats);
}
+static u16 mlx5_nodnic_fw_rev_maj(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->fw_rev) & 0xffff;
+}
+
+static u16 mlx5_nodnic_fw_rev_min(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->fw_rev) >> 16;
+}
+
+static u16 mlx5_nodnic_fw_rev_sub(struct mlx5_nodnic_core_dev *dev)
+{
+ return ioread32be(&dev->iseg->cmdif_rev_fw_sub) & 0xffff;
+}
+
+static void mlx5_nodnic_get_drvinfo(struct net_device *netdev,
+ struct ethtool_drvinfo *drvinfo)
+{
+ struct mlx5_nodnic_priv *priv = netdev_priv(netdev);
+ struct mlx5_nodnic_core_dev *mdev = priv->core_dev;
+
+ strscpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver));
+ snprintf(drvinfo->fw_version, sizeof(drvinfo->fw_version),
+ "%d.%d.%04d", mlx5_nodnic_fw_rev_maj(mdev),
+ mlx5_nodnic_fw_rev_min(mdev), mlx5_nodnic_fw_rev_sub(mdev));
+ strscpy(drvinfo->bus_info, dev_name(mdev->device),
+ sizeof(drvinfo->bus_info));
+}
+
+static const struct ethtool_ops mlx5_nodnic_ethtool_ops = {
+ .get_drvinfo = mlx5_nodnic_get_drvinfo,
+};
+
static const struct net_device_ops mlx5_nodnic_netdev_ops = {
.ndo_open = mlx5_nodnic_open,
.ndo_stop = mlx5_nodnic_close,
@@ -1005,6 +1038,7 @@ void mlx5_nodnic_build_netdev(struct net_device *netdev)
SET_NETDEV_DEV(netdev, core_dev->device);
netdev->netdev_ops = &mlx5_nodnic_netdev_ops;
+ netdev->ethtool_ops = &mlx5_nodnic_ethtool_ops;
netdev->request_ops_lock = true;
netdev->hw_features = NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH net-next 5/5] net/mlx5: nodnic: Add health monitoring
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
` (3 preceding siblings ...)
2026-09-30 12:57 ` [PATCH net-next 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
@ 2026-09-30 12:57 ` Tariq Toukan
4 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-09-30 12:57 UTC (permalink / raw)
To: Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
netdev, Paolo Abeni
Cc: Carolina Jubran, Gal Pressman, open list, Shahar Shitrit,
Tariq Toukan, Leon Romanovsky, Mark Bloch, Saeed Mahameed,
Dragos Tatulea
From: Shahar Shitrit <shshitrit@nvidia.com>
Monitor firmware health by periodically polling the health buffer from
the initialization segment.
Detect fatal device conditions, firmware health counter stalls, and
syndrome changes. Report health errors together with their severity
and mark the device as unhealthy on fatal conditions.
Start and stop health polling as part of the device probe and remove
flows.
Signed-off-by: Shahar Shitrit <shshitrit@nvidia.com>
Reviewed-by: Carolina Jubran <cjubran@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/nodnic/Makefile | 2 +-
.../net/ethernet/mellanox/mlx5/nodnic/en.h | 12 +
.../ethernet/mellanox/mlx5/nodnic/health.c | 315 ++++++++++++++++++
.../net/ethernet/mellanox/mlx5/nodnic/main.c | 2 +
4 files changed, 330 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
index 25865d7244d8..1d09edeaf3af 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/Makefile
@@ -5,4 +5,4 @@
obj-$(CONFIG_MLX5_NODNIC) += mlx5_nodnic.o
-mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o
+mlx5_nodnic-y := main.o en.o en_cfg.o tx.o rx.o nodnic_pci_vsc.o health.o
\ No newline at end of file
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
index d811bfe242ac..90e9f273d454 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/en.h
@@ -113,6 +113,16 @@ struct mlx5_nodnic_working_buffer {
u32 alloc_size;
};
+struct mlx5_nodnic_health {
+ struct health_buffer __iomem *health;
+ __be32 __iomem *health_counter;
+ struct timer_list timer;
+ u32 prev_counter;
+ int miss_counter;
+ u8 synd;
+ u32 fatal_error;
+};
+
struct mlx5_nodnic_priv {
struct mlx5_nodnic_core_dev *core_dev;
struct net_device *netdev;
@@ -128,6 +138,8 @@ struct mlx5_nodnic_priv {
struct mlx5_nodnic_dbr dbr; /* one shared dbr for TX and RX */
struct mlx5_nodnic_cq cq;
struct mlx5_nodnic_ring working_buffer;
+
+ struct mlx5_nodnic_health health;
};
void mlx5_nodnic_build_netdev(struct net_device *netdev);
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
new file mode 100644
index 000000000000..a981a5195c7e
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/health.c
@@ -0,0 +1,315 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+// Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+#include <linux/kernel.h>
+#include <linux/random.h>
+#include <linux/timer.h>
+#include <linux/types.h>
+
+#include "en.h"
+#include "nodnic_ifc.h"
+#include "nodnic.h"
+
+#define MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS 2000
+#define MLX5_NODNIC_MAX_MISSES 3
+
+enum nodnic_rfr_severity_bit_offsets {
+ NODNIC_CRR_BIT_OFFSET = 0x6,
+ NODNIC_RFR_BIT_OFFSET = 0x7,
+};
+
+enum {
+ MLX5_NODNIC_SEVERITY_MASK = 0x7,
+ MLX5_NODNIC_SEVERITY_VALID_MASK = 0x8,
+};
+
+enum {
+ MLX5_NODNIC_SENSOR_NO_ERR,
+ MLX5_NODNIC_SENSOR_PCI_COMM_ERR,
+ MLX5_NODNIC_SENSOR_PCI_ERR,
+ MLX5_NODNIC_SENSOR_NIC_DISABLED,
+ MLX5_NODNIC_SENSOR_NIC_SW_RESET,
+ MLX5_NODNIC_SENSOR_FW_SYND_RFR,
+};
+
+static unsigned long get_next_poll_jiffies(void)
+{
+ return jiffies + msecs_to_jiffies(MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS) +
+ get_random_u32_below(HZ);
+}
+
+static bool sensor_pci_not_working(struct mlx5_nodnic_health *health)
+{
+ struct health_buffer __iomem *h = health->health;
+
+ return (ioread32be(&h->fw_ver) == 0xffffffff);
+}
+
+static u8 mlx5_nodnic_get_nic_state(struct mlx5_nodnic_core_dev *dev)
+{
+ return (ioread32be(&dev->iseg->cmdq_addr_l_sz) >> 8) & 7;
+}
+
+static int mlx5_nodnic_health_get_rfr(u8 rfr_severity)
+{
+ return rfr_severity >> NODNIC_RFR_BIT_OFFSET;
+}
+
+static bool sensor_fw_synd_rfr(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+ struct health_buffer __iomem *h = health->health;
+ u8 synd = ioread8(&h->synd);
+ u8 rfr;
+
+ rfr = mlx5_nodnic_health_get_rfr(ioread8(&h->rfr_severity));
+
+ if (rfr && synd)
+ mlx5_nodnic_core_dbg(dev, "FW requests reset, synd: %d\n",
+ synd);
+
+ return rfr && synd;
+}
+
+static
+u32 mlx5_nodnic_health_check_fatal_sensors(struct mlx5_nodnic_core_dev *dev)
+{
+ u32 nic_state;
+
+ if (pci_channel_offline(dev->pdev))
+ return MLX5_NODNIC_SENSOR_PCI_ERR;
+ if (sensor_pci_not_working(&dev->priv->health))
+ return MLX5_NODNIC_SENSOR_PCI_COMM_ERR;
+
+ nic_state = mlx5_nodnic_get_nic_state(dev);
+ if (nic_state == MLX5_NODNIC_ISEG_NIC_INTERFACE_DISABLED)
+ return MLX5_NODNIC_SENSOR_NIC_DISABLED;
+ if (nic_state == MLX5_NODNIC_ISEG_NIC_INTERFACE_SW_RESET)
+ return MLX5_NODNIC_SENSOR_NIC_SW_RESET;
+
+ if (sensor_fw_synd_rfr(dev))
+ return MLX5_NODNIC_SENSOR_FW_SYND_RFR;
+
+ return MLX5_NODNIC_SENSOR_NO_ERR;
+}
+
+static const char *mlx5_nodnic_fatal_error_str(u32 err)
+{
+ switch (err) {
+ case MLX5_NODNIC_SENSOR_PCI_COMM_ERR: return "PCI communication error";
+ case MLX5_NODNIC_SENSOR_PCI_ERR: return "PCI error";
+ case MLX5_NODNIC_SENSOR_NIC_DISABLED: return "NIC disabled";
+ case MLX5_NODNIC_SENSOR_NIC_SW_RESET: return "NIC SW reset";
+ case MLX5_NODNIC_SENSOR_FW_SYND_RFR: return "FW synd RFR";
+ default: return "unknown";
+ }
+}
+
+static int mlx5_nodnic_health_get_crr(u8 rfr_severity)
+{
+ return (rfr_severity >> NODNIC_CRR_BIT_OFFSET) & 0x01;
+}
+
+static int mlx5_nodnic_health_get_severity(u8 rfr_severity)
+{
+ return rfr_severity & MLX5_NODNIC_SEVERITY_VALID_MASK ?
+ rfr_severity & MLX5_NODNIC_SEVERITY_MASK : LOGLEVEL_ERR;
+}
+
+/* Syndrome values from health buffer (same as mlx5_ifc.h) */
+static const char *mlx5_nodnic_hsynd_str(u8 synd)
+{
+ switch (synd) {
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_INTERNAL_ERR:
+ return "firmware internal error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_DEAD_IRISC:
+ return "irisc not responding";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HW_FATAL_ERR:
+ return "unrecoverable hardware error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FW_CRC_ERR:
+ return "firmware CRC error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_FETCH_PCI_ERR:
+ return "ICM fetch PCI error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PAGE_ERR:
+ return "HW fatal error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ASYNCHRONOUS_EQ_BUF_OVERRUN:
+ return "async EQ buffer overrun";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_IN_ERR:
+ return "EQ error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_EQ_INV:
+ return "Invalid EQ referenced";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_FFSER_ERR:
+ return "FFSER error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_HIGH_TEMP_ERR:
+ return "High temperature";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_ICM_PCI_POISONED_ERR:
+ return "ICM fetch PCI data poisoned error";
+ case MLX5_NODNIC_ISEG_HEALTH_SYNDROME_TRUST_LOCKDOWN_ERR:
+ return "Trust lockdown error";
+ default:
+ return "unrecognized error";
+ }
+}
+
+static const char *mlx5_nodnic_loglevel_str(int level)
+{
+ switch (level) {
+ case LOGLEVEL_EMERG:
+ return "EMERGENCY";
+ case LOGLEVEL_ALERT:
+ return "ALERT";
+ case LOGLEVEL_CRIT:
+ return "CRITICAL";
+ case LOGLEVEL_ERR:
+ return "ERROR";
+ case LOGLEVEL_WARNING:
+ return "WARNING";
+ case LOGLEVEL_NOTICE:
+ return "NOTICE";
+ case LOGLEVEL_INFO:
+ return "INFO";
+ case LOGLEVEL_DEBUG:
+ return "DEBUG";
+ }
+ return "Unknown log level";
+}
+
+static const char *mlx5_nodnic_severity_to_kern(int severity)
+{
+ if (severity <= LOGLEVEL_ERR)
+ return KERN_ERR;
+
+ if (severity <= LOGLEVEL_WARNING)
+ return KERN_WARNING;
+
+ return KERN_INFO;
+}
+
+static void mlx5_nodnic_print_health_info(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+ struct health_buffer __iomem *h = health->health;
+ struct device *device = dev->device;
+ u8 synd = ioread8(&h->synd);
+ const char *kern_severity;
+ u8 rfr_severity;
+ int severity;
+ int i;
+
+ if (!synd)
+ return;
+
+ if (sensor_pci_not_working(&dev->priv->health)) {
+ mlx5_nodnic_core_err(dev, "PCI slot is unavailable\n");
+ return;
+ }
+
+ rfr_severity = ioread8(&h->rfr_severity);
+ severity = mlx5_nodnic_health_get_severity(rfr_severity);
+ kern_severity = mlx5_nodnic_severity_to_kern(severity);
+
+ dev_printk(kern_severity, device,
+ "Health issue observed, %s, severity(%d) %s:\n",
+ mlx5_nodnic_hsynd_str(synd), severity,
+ mlx5_nodnic_loglevel_str(severity));
+
+ for (i = 0; i < ARRAY_SIZE(h->assert_var); i++)
+ dev_printk(kern_severity, device, "assert_var[%d] 0x%08x\n", i,
+ ioread32be(h->assert_var + i));
+
+ dev_printk(kern_severity, device, "assert_exit_ptr 0x%08x\n",
+ ioread32be(&h->assert_exit_ptr));
+ dev_printk(kern_severity, device, "assert_callra 0x%08x\n",
+ ioread32be(&h->assert_callra));
+ dev_printk(kern_severity, device, "raw fw_ver 0x%08x\n",
+ ioread32be(&h->fw_ver));
+ dev_printk(kern_severity, device, "time %u\n", ioread32be(&h->time));
+ dev_printk(kern_severity, device, "hw_id 0x%08x\n",
+ ioread32be(&h->hw_id));
+ dev_printk(kern_severity, device, "rfr %d\n",
+ mlx5_nodnic_health_get_rfr(rfr_severity));
+ dev_printk(kern_severity, device, "crr %d\n",
+ mlx5_nodnic_health_get_crr(rfr_severity));
+ dev_printk(kern_severity, device, "severity %d (%s)\n", severity,
+ mlx5_nodnic_loglevel_str(severity));
+ dev_printk(kern_severity, device, "irisc_index %d\n",
+ ioread8(&h->irisc_index));
+ dev_printk(kern_severity, device, "synd 0x%x: %s\n", synd,
+ mlx5_nodnic_hsynd_str(synd));
+ dev_printk(kern_severity, device, "ext_synd 0x%04x\n",
+ ioread16be(&h->ext_synd));
+
+ if (mlx5_nodnic_health_get_crr(rfr_severity))
+ mlx5_nodnic_core_warn(dev, "Cold reset is required\n");
+}
+
+static void mlx5_nodnic_poll_health(struct timer_list *t)
+{
+ struct mlx5_nodnic_priv *priv = timer_container_of(priv, t,
+ health.timer);
+ struct mlx5_nodnic_core_dev *dev = priv->core_dev;
+ struct mlx5_nodnic_health *health = &priv->health;
+ struct health_buffer __iomem *h = health->health;
+ u32 fatal_error, count;
+ u8 prev_synd;
+
+ if (dev->state == NODNIC_DEVICE_STATE_INTERNAL_ERROR)
+ return;
+
+ fatal_error = mlx5_nodnic_health_check_fatal_sensors(dev);
+
+ if (fatal_error && !health->fatal_error) {
+ mlx5_nodnic_core_err(dev, "fatal error detected: %s (%u)\n",
+ mlx5_nodnic_fatal_error_str(fatal_error),
+ fatal_error);
+ priv->health.fatal_error = fatal_error;
+ dev->state = NODNIC_DEVICE_STATE_INTERNAL_ERROR;
+ mlx5_nodnic_print_health_info(dev);
+ return;
+ }
+
+ count = ioread32be(health->health_counter);
+ if (count == health->prev_counter)
+ ++health->miss_counter;
+ else
+ health->miss_counter = 0;
+
+ health->prev_counter = count;
+ if (health->miss_counter == MLX5_NODNIC_MAX_MISSES) {
+ mlx5_nodnic_core_err(dev,
+ "device's health compromised - reached miss count\n");
+ health->synd = ioread8(&h->synd);
+ mlx5_nodnic_print_health_info(dev);
+ }
+
+ prev_synd = health->synd;
+ health->synd = ioread8(&h->synd);
+ if (health->synd && health->synd != prev_synd)
+ mlx5_nodnic_print_health_info(dev);
+
+ mod_timer(&health->timer, get_next_poll_jiffies());
+}
+
+void mlx5_nodnic_start_health_poll(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+
+ timer_setup(&health->timer, mlx5_nodnic_poll_health, 0);
+ health->fatal_error = MLX5_NODNIC_SENSOR_NO_ERR;
+ health->miss_counter = 0;
+ health->prev_counter = 0;
+ health->synd = 0;
+ health->health = &dev->iseg->health;
+ health->health_counter = &dev->iseg->health_counter;
+
+ health->timer.expires = jiffies +
+ msecs_to_jiffies(MLX5_NODNIC_HEALTH_POLL_INTERVAL_MS);
+ add_timer(&health->timer);
+}
+
+void mlx5_nodnic_stop_health_poll(struct mlx5_nodnic_core_dev *dev)
+{
+ struct mlx5_nodnic_health *health = &dev->priv->health;
+
+ timer_delete_sync(&health->timer);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
index d77d9dbe3c89..d6720c72905f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/nodnic/main.c
@@ -814,6 +814,7 @@ static int mlx5_nodnic_probe_one(struct pci_dev *pdev,
dev->netdev_registered = true;
dev->state = NODNIC_DEVICE_STATE_UP;
+ mlx5_nodnic_start_health_poll(dev);
pci_save_state(pdev);
@@ -839,6 +840,7 @@ static void mlx5_nodnic_remove_one(struct pci_dev *pdev)
if (!dev)
return;
+ mlx5_nodnic_stop_health_poll(dev);
mlx5_nodnic_event_irq_cleanup(dev);
mlx5_nodnic_cleanup_netdev(dev);
mlx5_nodnic_dev_cleanup(dev);
--
2.44.0
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-09-30 12:58 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-30 12:57 [PATCH net-next 0/5] net/mlx5: Introduce mlx5_nodnic, a driver for DRAM-less NVIDIA Mellanox NIC functions Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 1/5] net/mlx5: nodnic: Introduce the mlx5_nodnic driver Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 2/5] net/mlx5: nodnic: Implement data path Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 3/5] net/mlx5: nodnic: Add TX/RX statistics Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 4/5] net/mlx5: nodnic: Add ethtool driver info Tariq Toukan
2026-09-30 12:57 ` [PATCH net-next 5/5] net/mlx5: nodnic: Add health monitoring Tariq Toukan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®