mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues
@ 2026-11-13  5:47 han.junyang
  2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
                   ` (3 more replies)
  0 siblings, 4 replies; 8+ messages in thread
From: han.junyang @ 2026-11-13  5:47 UTC (permalink / raw)
  To: andrew+netdev, davem, edumazet, kuba, pabeni, horms
  Cc: linux-kernel, netdev, han.junyang, ran.ming, han.chengfei, zhang.yanze

From: Junyang Han <han.junyang@zte.com.cn>

This series continues the DingHai (ZXDH) PF driver bring-up: after PCI
probing, it verifies the firmware version contract, waits for the
RISC-V management core to become ready, sets up the MSI-X interrupt
pools, and creates the async event queues through which the firmware
will report events.

Some notes on the IRQ design:

The vector space is partitioned into per-purpose pools (async, RDMA,
vq). Event queues share a vector through an atomic notifier chain
attached to the IRQ, and the sharing degree is governed by pool
thresholds. The IRQ table hangs off a void *priv in the shared core
device: the PF, MPF and SF core devices each carry a different pool
layout behind that pointer, so a type-specific struct in en_pf.c keeps
the shared header free of PF-only details.

Changes in v6:
- Guard the IRQ table teardown in patch 2 rather than deferring the
  guards to patch 3, so the probe error paths of the MSI-X patch are
  safe on their own and bisection cannot land on an unguarded cleanup
  (sashiko review of v5).
- Review findings not taken from v5:
  The sashiko automated review of v5 [1], which was not posted to the
  list, reported four findings. The IRQ table teardown NULL
  dereferences on the probe error paths were real and are fixed in
  this revision. The remaining three restate earlier findings or
  do not apply: the BAR0 bounds item repeats the BAR 0 entry below;
  the 20 s legacy wait is the legacy-firmware detection itself:
  firmware that predates the compat region never populates it and
  the driver reports and proceeds in legacy mode after the timeout
  (see Changes in v4); and the health_version read cannot race the
  RISC-V core initialization: the readiness skip fires only for
  firmware with patch < ZXDH_HPIRQ_PATCH, which never reports the
  power-on flag, while firmware that does report health_version
  carries patch >= ZXDH_HPIRQ_PATCH and initializes the health
  buffer before the compat block reports ready.

  [1] https://sashiko.dev/#/patchset/20261014151200489685SCIlolykW0NGfL5EtQ%40zte.com.cn

Changes in v5:
- Drop __packed from zxdh_fw_compat and zxdh_health_buffer: both
  structs are naturally aligned with no padding.

Changes in v4:
- Fold the irq and eq table create() steps into their init()
  counterparts and route every probe error path through the same
  destroy() functions remove() uses, so cleanup has a single
  authority from the start (review feedback on v3).
- Give the pool name a buffer bound of its own (16 bytes) instead of
  sharing the 100-byte IRQ name buffer, so W=1 builds stay clean of
  -Wformat-truncation (review feedback on v3).
- Lower the compat region wait from 200 s to 20 s: the firmware
  publishes the block within 10 s of boot, so twice that leaves
  margin, and the wait now sits far below the 180 s udev event
  window raised on the thread (review feedback on v3).
- Wait for the compat block with a per-field readiness check instead
  of only the first dword: the firmware fills the fields one by one,
  and a dword-granular check can read a half-populated block (the
  "single write" justification given in v3 was wrong).
- Log the legacy-firmware assumption when the compat region wait
  times out, making the intentionally ignored timeout visible.

Changes in v3:
- Unmap the modern config MMIO regions on the probe error paths added
  in this series; they used to leak when the fw compat check or the
  RISC-V readiness wait failed.
- Skip the RISC-V readiness wait for firmware without the compat
  region: the erased patch field read as 0xffff and defeated the skip.
- Make fw_minor unsigned so firmware minor versions >= 128 are not
  rejected through sign wrap.
- Balance the per-CPU IRQ accounting on release, so pool teardown does
  not trip its leftover WARN; set the IRQ affinity for real with
  irq_set_affinity_and_hint() instead of only updating the hint; name
  IRQs after their pool instead of a hardcoded prefix; assert that the
  pool is empty at free instead of force-releasing leftovers.
- Clear eq->irq when the async IRQ request fails, so teardown does not
  treat the ERR_PTR as a live IRQ.

Review findings not taken:

- xa_alloc() with a NULL entry does not fail: __xa_alloc() turns a
  NULL entry into the internal zero entry as a reservation
  (lib/xarray.c), which is the reservation semantics the pool relies
  on.
- BAR 0 length checks against a truncated bar: the bar layout is part
  of the board firmware contract and the driver does not defend
  against a broken device, as settled during the review of the
  earlier device bring-up series (Andrew Lunn).

Changes in v2:
- Convert both firmware readiness polls to readx_poll_timeout()
  (Andrew Lunn).
- Read the fw compat block field by field through ioread*() accessors
  instead of an ioread32_rep() bulk copy, which misplaces the u8/u16
  fields on big-endian; no __le annotations are needed since each
  accessor converts from little-endian (Andrew Lunn).
- Use kref for the IRQ reference count (Andrew Lunn).

Junyang Han (3):
  dinghai: add firmware version check and RISC-V readiness polling
  dinghai: add MSI-X interrupt pools
  dinghai: add async event queue for firmware notifications

 drivers/net/ethernet/zte/dinghai/Makefile   |   2 +-
 drivers/net/ethernet/zte/dinghai/en_pf.c    | 223 +++++++++++
 drivers/net/ethernet/zte/dinghai/en_pf.h    |  56 +++
 drivers/net/ethernet/zte/dinghai/zxdh_eq.c  | 137 +++++++
 drivers/net/ethernet/zte/dinghai/zxdh_eq.h  |  60 +++
 drivers/net/ethernet/zte/dinghai/zxdh_irq.c | 406 ++++++++++++++++++++
 drivers/net/ethernet/zte/dinghai/zxdh_irq.h |  73 ++++
 7 files changed, 956 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_eq.c
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_eq.h
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_irq.c
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_irq.h

-- 
2.27.0

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH net-next v6 1/3] dinghai: add firmware version check and  RISC-V readiness polling
  2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
@ 2026-11-13  5:50 ` han.junyang
  2026-11-19 12:05   ` Simon Horman
  2026-11-13  5:52 ` [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools han.junyang
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 8+ messages in thread
From: han.junyang @ 2026-11-13  5:50 UTC (permalink / raw)
  To: andrew+netdev, davem, edumazet, kuba, pabeni, horms
  Cc: linux-kernel, netdev, han.junyang, ran.ming, han.chengfei, zhang.yanze

From: Junyang Han <han.junyang@zte.com.cn>

The DingHai firmware publishes a version compatibility block and a
RISC-V health buffer at fixed offsets within BAR 0.

After the PCI capabilities are mapped, poll the compatibility block
until the firmware populates it (the region reads as all ones until
then) and verify the driver/firmware version contract. Then wait for
the RISC-V management core to set its power-on flag in the health
buffer before the rest of the probe continues.

Firmware images that support neither self-healing (health version
other than 1) nor hot-plug interrupts (patch level below
ZXDH_HPIRQ_PATCH) do not report the power-on flag and skip the
readiness wait.

Signed-off-by: Junyang Han <han.junyang@zte.com.cn>
---
 drivers/net/ethernet/zte/dinghai/en_pf.c | 130 +++++++++++++++++++++++
 drivers/net/ethernet/zte/dinghai/en_pf.h |  46 ++++++++
 2 files changed, 176 insertions(+)

diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
index 86d437408820..ddff9b0438fb 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.c
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.c
@@ -6,6 +6,8 @@

 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
 #include <net/devlink.h>
 #include <linux/dma-mapping.h>
 #include "en_pf.h"
@@ -369,6 +371,120 @@ int zxdh_pf_modern_cfg_init(struct zxdh_core_dev *zxdh_dev)
 	return ret;
 }

+/* The firmware fills the compatibility block field by field, so it is
+ * only ready to be read once every field we consume has lost its
+ * erased value.
+ */
+static bool zxdh_pf_fw_compat_ready(struct zxdh_fw_compat __iomem *compat)
+{
+	return ioread8(&compat->module_id) != 0xffU &&
+	       ioread8(&compat->major) != 0xffU &&
+	       ioread8(&compat->fw_minor) != 0xffU &&
+	       ioread8(&compat->drv_minor) != 0xffU &&
+	       ioread16(&compat->patch) != 0xffffU;
+}
+
+/* Read the firmware version block and verify the driver/firmware
+ * version contract.
+ */
+static int zxdh_pf_fw_compat_check(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_pf_dev *pf_dev = zxdh_dev->priv;
+	struct zxdh_fw_compat __iomem *compat;
+	struct zxdh_fw_compat *fw_compat;
+	bool ready;
+	int err;
+
+	fw_compat = &pf_dev->fw_compat;
+	compat = pf_dev->pci_ioremap_addr[0] + ZXDH_FW_COMPAT_OFFSET;
+
+	/* The firmware publishes the block within 10 s of boot; wait twice
+	 * that to leave margin.
+	 */
+	err = readx_poll_timeout(zxdh_pf_fw_compat_ready, compat, ready,
+				 ready, USEC_PER_SEC,
+				 ZXDH_FW_COMPAT_TIMEOUT_SEC * USEC_PER_SEC);
+	if (err)
+		dev_info(zxdh_dev->device,
+			 "compat region not populated, assuming legacy firmware\n");
+
+	/* Firmware predating the compatibility region keeps the erased
+	 * pattern, which fails the module id check below and defers the
+	 * decision to the readiness wait.
+	 */
+	fw_compat->module_id = ioread8(&compat->module_id);
+	fw_compat->major = ioread8(&compat->major);
+	fw_compat->fw_minor = ioread8(&compat->fw_minor);
+	fw_compat->drv_minor = ioread8(&compat->drv_minor);
+	fw_compat->patch = ioread16(&compat->patch);
+
+	if (fw_compat->module_id != ZXDH_MODULE_ID) {
+		dev_info(zxdh_dev->device,
+			 "unknown module id %u, skip fw compat check\n",
+			 fw_compat->module_id);
+		/* Unknown firmware is treated as predating the HPIRQ
+		 * patch, so that the readiness wait is skipped.
+		 */
+		fw_compat->patch = 0;
+		return 0;
+	}
+
+	if (fw_compat->major != ZXDH_MAJOR) {
+		dev_err(zxdh_dev->device,
+			"driver major %u incompatible with firmware major %u\n",
+			ZXDH_MAJOR, fw_compat->major);
+		return -EINVAL;
+	}
+
+	if (fw_compat->fw_minor < ZXDH_FW_MINOR) {
+		dev_err(zxdh_dev->device,
+			"firmware fw_minor %u older than required %u\n",
+			fw_compat->fw_minor, ZXDH_FW_MINOR);
+		return -EINVAL;
+	}
+
+	if (fw_compat->drv_minor > ZXDH_DRV_MINOR) {
+		dev_err(zxdh_dev->device,
+			"driver drv_minor %u older than required by firmware %u\n",
+			ZXDH_DRV_MINOR, fw_compat->drv_minor);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+/* Wait for the RISC-V management core of the firmware to finish
+ * booting, so that later probe steps can talk to it.
+ */
+static int zxdh_pf_wait_riscv_ready(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_pf_dev *pf_dev = zxdh_dev->priv;
+	struct zxdh_health_buffer __iomem *hb;
+	u8 health_version;
+	u8 power_on;
+	int err;
+
+	hb = pf_dev->pci_ioremap_addr[0] + ZXDH_RISCV_HB_OFFSET;
+	health_version = ioread8(&hb->health_version);
+
+	/* Firmware that supports neither self-healing nor hot-plug
+	 * interrupts does not report the power-on flag.
+	 */
+	if (health_version != 1 &&
+	    pf_dev->fw_compat.patch < ZXDH_HPIRQ_PATCH)
+		return 0;
+
+	err = readx_poll_timeout(ioread8, &hb->riscv_power_on, power_on,
+				 power_on == 1, USEC_PER_SEC,
+				 ZXDH_RISCV_READY_TIMEOUT_SEC * USEC_PER_SEC);
+	if (err) {
+		dev_err(zxdh_dev->device, "timed out waiting for riscv power on\n");
+		return err;
+	}
+
+	return 0;
+}
+
 static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	struct zxdh_core_dev *zxdh_dev;
@@ -405,10 +521,24 @@ static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		goto err_cfg_init;
 	}

+	ret = zxdh_pf_fw_compat_check(zxdh_dev);
+	if (ret) {
+		dev_err(&pdev->dev, "zxdh_pf_fw_compat_check failed: %d\n", ret);
+		goto err_modern_cfg;
+	}
+
+	ret = zxdh_pf_wait_riscv_ready(zxdh_dev);
+	if (ret) {
+		dev_err(&pdev->dev, "zxdh_pf_wait_riscv_ready failed: %d\n", ret);
+		goto err_modern_cfg;
+	}
+
 	devlink_register(devlink);

 	return 0;

+err_modern_cfg:
+	zxdh_pf_modern_cfg_uninit(zxdh_dev);
 err_cfg_init:
 	zxdh_pf_pci_close(zxdh_dev);
 err_pci_init:
diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.h b/drivers/net/ethernet/zte/dinghai/en_pf.h
index 7373dee8d1a9..730cb084a8ae 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.h
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.h
@@ -29,6 +29,51 @@
 #define ZXDH_PF_ALIGN2			2
 #define ZXDH_PF_MAP_MINLEN2		2

+/* Fixed offsets of the firmware interface regions within BAR 0. */
+#define ZXDH_RISCV_HB_OFFSET	0x5300
+#define ZXDH_FW_COMPAT_OFFSET	0x5400
+
+/* Driver/firmware version contract. The firmware publishes its side of
+ * the contract in the region at ZXDH_FW_COMPAT_OFFSET.
+ */
+#define ZXDH_MODULE_ID		1
+#define ZXDH_MAJOR		1
+#define ZXDH_FW_MINOR		0
+#define ZXDH_DRV_MINOR		0
+/* Firmware patch level that introduced hot-plug IRQ support. */
+#define ZXDH_HPIRQ_PATCH	4
+
+#define ZXDH_FW_COMPAT_TIMEOUT_SEC	20
+#define ZXDH_RISCV_READY_TIMEOUT_SEC	40
+
+/* Firmware version compatibility block at ZXDH_FW_COMPAT_OFFSET.
+ * Fields are read through ioread*(), which converts from little-endian.
+ */
+struct zxdh_fw_compat {
+	u8 module_id;
+	u8 major;
+	u8 fw_minor;
+	u8 drv_minor;
+	u16 patch;
+	u16 rsv;
+};
+
+/* Health buffer at ZXDH_RISCV_HB_OFFSET, maintained by the RISC-V
+ * management core of the firmware. Fields are read through ioread*().
+ */
+struct zxdh_health_buffer {
+	u32 synd;		/* Bitmask of active syndrome flags. */
+	u32 health_counter;	/* Incremented heartbeat counter. */
+	u8 status;
+	u8 rfr;
+	u8 fw_exception;
+	u8 riscv_power_on;	/* Set to 1 once the core finished booting. */
+	u8 fw_version[32];
+	u8 pf_status[5];
+	u8 health_version;	/* 1 when the firmware supports self-healing. */
+	u8 rsv1[30];
+};
+
 struct zxdh_core_dev {
 	struct device *device;
 	struct pci_dev *pdev;
@@ -54,6 +99,7 @@ struct zxdh_pf_dev {
 	s32 modern_bars;
 	void __iomem *pci_ioremap_addr[6];
 	u32 dev_cfg_bar_off;
+	struct zxdh_fw_compat fw_compat;
 };

 void *zxdh_core_alloc_priv(struct zxdh_core_dev *zxdh_dev, size_t size);
-- 
2.27.0

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools
  2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
  2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
@ 2026-11-13  5:52 ` han.junyang
  2026-11-21 23:31   ` Jakub Kicinski
  2026-11-13  5:53 ` [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications han.junyang
  2026-11-21 23:44 ` [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues patchwork-bot+netdevbpf
  3 siblings, 1 reply; 8+ messages in thread
From: han.junyang @ 2026-11-13  5:52 UTC (permalink / raw)
  To: andrew+netdev, davem, edumazet, kuba, pabeni, horms
  Cc: linux-kernel, netdev, han.junyang, ran.ming, han.chengfei, zhang.yanze

From: Junyang Han <han.junyang@zte.com.cn>

Allocate the fixed MSI-X vector layout of the device and manage the
vectors in per-purpose pools: the async event queues first, a range
reserved for RDMA in between and the vq queue pairs last. This series
wires up the async pool; the vq pool comes with the netdev series.

IRQs are reference counted so that several event queues can share one
vector. The pool hands out the least loaded matching IRQ once it passes
the pool minimum threshold, and binds newly created IRQs to the least
loaded CPU of the requested affinity mask. Interrupt delivery fans out
through an atomic notifier chain attached to each IRQ, which the async
event queue setup posted later in this series hooks into.

Signed-off-by: Junyang Han <han.junyang@zte.com.cn>
---
 drivers/net/ethernet/zte/dinghai/Makefile   |   2 +-
 drivers/net/ethernet/zte/dinghai/en_pf.c    |  87 +++++
 drivers/net/ethernet/zte/dinghai/en_pf.h    |   8 +
 drivers/net/ethernet/zte/dinghai/zxdh_irq.c | 406 ++++++++++++++++++++
 drivers/net/ethernet/zte/dinghai/zxdh_irq.h |  73 ++++
 5 files changed, 575 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_irq.c
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_irq.h

diff --git a/drivers/net/ethernet/zte/dinghai/Makefile b/drivers/net/ethernet/zte/dinghai/Makefile
index e2f4da33df59..f4a8f9480291 100644
--- a/drivers/net/ethernet/zte/dinghai/Makefile
+++ b/drivers/net/ethernet/zte/dinghai/Makefile
@@ -4,4 +4,4 @@
 #

 obj-$(CONFIG_DINGHAI_PF) += dinghai10e.o
-dinghai10e-y := en_pf.o
+dinghai10e-y := en_pf.o zxdh_irq.o
diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
index ddff9b0438fb..5927f1f39b02 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.c
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.c
@@ -8,6 +8,8 @@
 #include <linux/pci.h>
 #include <linux/io.h>
 #include <linux/iopoll.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
 #include <net/devlink.h>
 #include <linux/dma-mapping.h>
 #include "en_pf.h"
@@ -27,6 +29,14 @@ static const struct pci_device_id zxdh_pf_pci_table[] = {

 MODULE_DEVICE_TABLE(pci, zxdh_pf_pci_table);

+struct zxdh_pf_irq_table {
+	struct zxdh_irq_pool *async_pool;
+};
+
+/* IRQ compaction thresholds of the async pool, in number of queues. */
+#define ZXDH_PF_ASYNC_IRQ_MIN_COMP	0
+#define ZXDH_PF_ASYNC_IRQ_MAX_COMP	7
+
 void *zxdh_core_alloc_priv(struct zxdh_core_dev *zxdh_dev, size_t size)
 {
 	void *priv = kzalloc(size, GFP_KERNEL);
@@ -485,6 +495,74 @@ static int zxdh_pf_wait_riscv_ready(struct zxdh_core_dev *zxdh_dev)
 	return 0;
 }

+static int zxdh_pf_irq_pools_init(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_pf_irq_table *pf_irq_table = zxdh_dev->irq_table.priv;
+	struct zxdh_irq_pool *pool;
+
+	pool = zxdh_irq_pool_alloc(zxdh_dev, 0, ZXDH_ASYNC_CHANNELS_NUM,
+				   "zxdh_pf_async", ZXDH_PF_ASYNC_IRQ_MIN_COMP,
+				   ZXDH_PF_ASYNC_IRQ_MAX_COMP);
+	if (IS_ERR(pool))
+		return PTR_ERR(pool);
+
+	pf_irq_table->async_pool = pool;
+
+	return 0;
+}
+
+static void zxdh_pf_irq_pools_destroy(struct zxdh_pf_irq_table *pf_irq_table)
+{
+	if (pf_irq_table->async_pool)
+		zxdh_irq_pool_free(pf_irq_table->async_pool);
+}
+
+int zxdh_pf_irq_table_init(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
+	struct zxdh_pf_irq_table *priv;
+	int total_vec;
+
+	priv = kvzalloc_obj(*priv, GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+	table->priv = priv;
+
+	total_vec = ZXDH_VQS_CHANNELS_NUM + ZXDH_ASYNC_CHANNELS_NUM +
+		    ZXDH_RDMA_CHANNELS_NUM;
+	total_vec = pci_alloc_irq_vectors(zxdh_dev->pdev, total_vec, total_vec,
+					  PCI_IRQ_MSIX);
+	if (total_vec < 0) {
+		dev_err(zxdh_dev->device, "pci_alloc_irq_vectors failed: %d\n",
+			total_vec);
+		return total_vec;
+	}
+
+	return zxdh_pf_irq_pools_init(zxdh_dev);
+}
+
+void zxdh_pf_irq_table_destroy(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
+
+	if (table->priv)
+		zxdh_pf_irq_pools_destroy(table->priv);
+	kvfree(table->priv);
+	table->priv = NULL;
+	pci_free_irq_vectors(zxdh_dev->pdev);
+}
+
+struct zxdh_irq *zxdh_pf_async_irq_request(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
+	struct zxdh_pf_irq_table *pf_irq_table = table->priv;
+
+	if (!pf_irq_table->async_pool)
+		return NULL;
+
+	return zxdh_get_irq_of_pool(pf_irq_table->async_pool);
+}
+
 static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 {
 	struct zxdh_core_dev *zxdh_dev;
@@ -533,10 +611,18 @@ static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		goto err_modern_cfg;
 	}

+	ret = zxdh_pf_irq_table_init(zxdh_dev);
+	if (ret) {
+		dev_err(&pdev->dev, "zxdh_pf_irq_table_init failed: %d\n", ret);
+		goto err_irq_table;
+	}
+
 	devlink_register(devlink);

 	return 0;

+err_irq_table:
+	zxdh_pf_irq_table_destroy(zxdh_dev);
 err_modern_cfg:
 	zxdh_pf_modern_cfg_uninit(zxdh_dev);
 err_cfg_init:
@@ -554,6 +640,7 @@ static void zxdh_pf_remove(struct pci_dev *pdev)
 	struct devlink *devlink = priv_to_devlink(zxdh_dev);

 	devlink_unregister(devlink);
+	zxdh_pf_irq_table_destroy(zxdh_dev);
 	zxdh_pf_modern_cfg_uninit(zxdh_dev);
 	zxdh_pf_pci_close(zxdh_dev);
 	zxdh_core_free_priv(zxdh_dev);
diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.h b/drivers/net/ethernet/zte/dinghai/en_pf.h
index 730cb084a8ae..3de96488a810 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.h
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.h
@@ -9,6 +9,8 @@

 #include <linux/types.h>

+#include "zxdh_irq.h"
+
 #define ZXDH_PF_VENDOR_ID	0x1cf2
 #define ZXDH_PF_DEVICE_ID	0x8040
 #define ZXDH_VF_DEVICE_ID	0x8041
@@ -78,6 +80,7 @@ struct zxdh_core_dev {
 	struct device *device;
 	struct pci_dev *pdev;
 	struct devlink *devlink;
+	struct zxdh_irq_table irq_table;
 	void *priv;
 };

@@ -118,4 +121,9 @@ int zxdh_pf_device_cfg_init(struct zxdh_core_dev *zxdh_dev);
 void zxdh_pf_modern_cfg_uninit(struct zxdh_core_dev *zxdh_dev);
 int zxdh_pf_modern_cfg_init(struct zxdh_core_dev *zxdh_dev);

+/* PF IRQ table (en_pf.c) */
+int zxdh_pf_irq_table_init(struct zxdh_core_dev *zxdh_dev);
+void zxdh_pf_irq_table_destroy(struct zxdh_core_dev *zxdh_dev);
+struct zxdh_irq *zxdh_pf_async_irq_request(struct zxdh_core_dev *zxdh_dev);
+
 #endif /* __ZXDH_EN_PF_H__ */
diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_irq.c b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
new file mode 100644
index 000000000000..bc5348dfa5ba
--- /dev/null
+++ b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
@@ -0,0 +1,406 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ZTE DingHai Ethernet driver - IRQ pool management
+ * Copyright (c) 2022-2026, ZTE Corporation.
+ *
+ * Device vectors are managed in per-purpose pools. An IRQ carries a
+ * refcount so that several event queues can share one vector; on
+ * raise, the interrupt is fanned out through an atomic notifier chain
+ * attached to the IRQ.
+ */
+
+#include <linux/cpumask.h>
+#include <linux/device.h>
+#include <linux/interrupt.h>
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/xarray.h>
+
+#include "en_pf.h"
+#include "zxdh_irq.h"
+
+static void zxdh_cpu_put(struct zxdh_irq_pool *pool, int cpu)
+{
+	pool->irqs_per_cpu[cpu]--;
+}
+
+/* Release callback of the IRQ kref, called with pool->lock held. */
+static void zxdh_irq_release(struct kref *kref)
+{
+	struct zxdh_irq *irq = container_of(kref, struct zxdh_irq, refcount);
+	struct zxdh_irq_pool *pool = irq->pool;
+
+	lockdep_assert_held(&pool->lock);
+	xa_erase(&pool->irqs, irq->index);
+	zxdh_cpu_put(pool, cpumask_first(irq->mask));
+	/* free_irq() requires the affinity hint to be cleared before it is
+	 * called; the asymmetry with the set path in zxdh_irq_alloc() is
+	 * intentional.
+	 */
+	irq_update_affinity_hint(irq->irqn, NULL);
+	free_cpumask_var(irq->mask);
+	free_irq(irq->irqn, irq);
+	kfree(irq);
+}
+
+/* Drop one reference; returns 1 if the IRQ was released. */
+static int zxdh_irq_put(struct zxdh_irq *irq)
+{
+	struct zxdh_irq_pool *pool = irq->pool;
+	int released;
+
+	mutex_lock(&pool->lock);
+	released = kref_put(&irq->refcount, zxdh_irq_release);
+	mutex_unlock(&pool->lock);
+
+	return released;
+}
+
+static int zxdh_irq_get(struct zxdh_irq *irq)
+{
+	return kref_get_unless_zero(&irq->refcount);
+}
+
+static irqreturn_t zxdh_irq_int_handler(int irq, void *data)
+{
+	struct zxdh_irq *zxdh_irq = data;
+
+	atomic_notifier_call_chain(&zxdh_irq->nh, 0, NULL);
+
+	return IRQ_HANDLED;
+}
+
+static struct zxdh_irq *zxdh_irq_alloc(struct zxdh_irq_pool *pool, int vecidx,
+				       const struct cpumask *affinity)
+{
+	struct zxdh_core_dev *zxdh_dev = pool->dev;
+	struct zxdh_irq *irq;
+	int err;
+	int cpu;
+
+	irq = kzalloc_obj(*irq, GFP_KERNEL);
+	if (!irq)
+		return ERR_PTR(-ENOMEM);
+
+	irq->pool = pool;
+	irq->irqn = pci_irq_vector(zxdh_dev->pdev, vecidx);
+	if (irq->irqn < 0) {
+		err = irq->irqn;
+		goto err_irqn;
+	}
+
+	ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh);
+	snprintf(irq->name, ZXDH_MAX_IRQ_NAME, "%s_%d@pci:%s", pool->name,
+		 vecidx, pci_name(zxdh_dev->pdev));
+
+	err = request_irq(irq->irqn, zxdh_irq_int_handler, 0, irq->name, irq);
+	if (err) {
+		dev_err(zxdh_dev->device, "request_irq failed: %d\n", err);
+		goto err_irqn;
+	}
+
+	if (!zalloc_cpumask_var(&irq->mask, GFP_KERNEL)) {
+		dev_err(zxdh_dev->device, "zalloc_cpumask_var failed\n");
+		err = -ENOMEM;
+		goto err_cpumask;
+	}
+
+	if (affinity) {
+		cpumask_copy(irq->mask, affinity);
+	} else {
+		/* No preference requested; spread over all online CPUs. */
+		for_each_online_cpu(cpu)
+			cpumask_set_cpu(cpu, irq->mask);
+	}
+	irq_set_affinity_and_hint(irq->irqn, irq->mask);
+
+	kref_init(&irq->refcount);
+	irq->index = vecidx;
+	err = xa_err(xa_store(&pool->irqs, irq->index, irq, GFP_KERNEL));
+	if (err) {
+		dev_err(zxdh_dev->device, "xa_store failed for irq %u: %d\n",
+			irq->index, err);
+		goto err_xa;
+	}
+
+	return irq;
+
+err_xa:
+	irq_update_affinity_hint(irq->irqn, NULL);
+	free_cpumask_var(irq->mask);
+err_cpumask:
+	free_irq(irq->irqn, irq);
+err_irqn:
+	kfree(irq);
+	return ERR_PTR(err);
+}
+
+int zxdh_irq_attach_nb(struct zxdh_irq *irq, struct notifier_block *nb)
+{
+	int err;
+
+	if (!zxdh_irq_get(irq))
+		return -ENOENT;
+
+	err = atomic_notifier_chain_register(&irq->nh, nb);
+	if (err)
+		zxdh_irq_put(irq);
+
+	return err;
+}
+
+int zxdh_irq_detach_nb(struct zxdh_irq *irq, struct notifier_block *nb)
+{
+	int err;
+
+	err = atomic_notifier_chain_unregister(&irq->nh, nb);
+	zxdh_irq_put(irq);
+
+	return err;
+}
+
+/* Release one or more IRQs back to their pool. */
+void zxdh_irqs_release_vectors(struct zxdh_irq **irqs, int nirqs)
+{
+	int i;
+
+	for (i = 0; i < nirqs; i++) {
+		synchronize_irq(irqs[i]->irqn);
+		zxdh_irq_put(irqs[i]);
+	}
+}
+
+static void zxdh_cpu_get(struct zxdh_irq_pool *pool, int cpu)
+{
+	pool->irqs_per_cpu[cpu]++;
+}
+
+/* Find the least loaded CPU, i.e. the CPU with the fewest IRQs bound
+ * to it, within the requested mask.
+ */
+static int zxdh_cpu_get_least_loaded(struct zxdh_irq_pool *pool,
+				     const struct cpumask *req_mask)
+{
+	int best_cpu = -1;
+	int cpu;
+
+	for_each_cpu_and(cpu, req_mask, cpu_online_mask) {
+		/* This CPU has no IRQs yet, no need to look further. */
+		if (!pool->irqs_per_cpu[cpu]) {
+			best_cpu = cpu;
+			break;
+		}
+		if (best_cpu < 0)
+			best_cpu = cpu;
+		if (pool->irqs_per_cpu[cpu] < pool->irqs_per_cpu[best_cpu])
+			best_cpu = cpu;
+	}
+
+	if (best_cpu < 0) {
+		dev_err(pool->dev->device,
+			"no online CPU in affinity mask (%*pbl)\n",
+			cpumask_pr_args(req_mask));
+		best_cpu = cpumask_first(cpu_online_mask);
+	}
+	pool->irqs_per_cpu[best_cpu]++;
+
+	return best_cpu;
+}
+
+/* Create a new IRQ from the pool and bind it to the requested mask. */
+static struct zxdh_irq *zxdh_irq_pool_request_irq(struct zxdh_irq_pool *pool,
+						  const struct cpumask *req_mask)
+{
+	const struct cpumask *affinity = req_mask;
+	cpumask_var_t auto_mask;
+	struct zxdh_irq *irq;
+	u32 irq_index;
+	int err;
+
+	if (!zalloc_cpumask_var(&auto_mask, GFP_KERNEL))
+		return ERR_PTR(-ENOMEM);
+
+	err = xa_alloc(&pool->irqs, &irq_index, NULL, pool->xa_num_irqs,
+		       GFP_KERNEL);
+	if (err) {
+		if (err == -EBUSY)
+			err = -EUSERS;
+		goto err_xa;
+	}
+
+	if (cpumask_weight(req_mask) > 1) {
+		/* Bind to the least loaded CPU of the request. */
+		cpumask_set_cpu(zxdh_cpu_get_least_loaded(pool, req_mask),
+				auto_mask);
+		affinity = auto_mask;
+	} else {
+		zxdh_cpu_get(pool, cpumask_first(req_mask));
+	}
+
+	irq = zxdh_irq_alloc(pool, irq_index, affinity);
+	if (IS_ERR(irq))
+		goto err_alloc;
+
+	free_cpumask_var(auto_mask);
+
+	return irq;
+
+err_alloc:
+	/* Undo the per-CPU accounting done above. */
+	zxdh_cpu_put(pool, cpumask_first(affinity));
+	xa_erase(&pool->irqs, irq_index);
+err_xa:
+	free_cpumask_var(auto_mask);
+	return err ? ERR_PTR(err) : irq;
+}
+
+/* Look for the IRQ with the smallest refcount whose affinity mask is a
+ * subset of the requested mask.
+ */
+static struct zxdh_irq *zxdh_irq_pool_find_least_loaded(struct zxdh_irq_pool *pool,
+							const struct cpumask *req_mask)
+{
+	struct zxdh_irq *irq = NULL;
+	struct zxdh_irq *iter;
+	unsigned long index;
+	int iter_refcount;
+
+	lockdep_assert_held(&pool->lock);
+	xa_for_each_range(&pool->irqs, index, iter,
+			  pool->xa_num_irqs.min, pool->xa_num_irqs.max) {
+		iter_refcount = refcount_read(&iter->refcount.refcount);
+
+		/* Skip IRQs whose mask is not a subset of the request. */
+		if (!cpumask_subset(iter->mask, req_mask))
+			continue;
+		/* IRQ below the minimum threshold found, take it. */
+		if (iter_refcount < pool->min_threshold)
+			return iter;
+		if (!irq ||
+		    iter_refcount < refcount_read(&irq->refcount.refcount))
+			irq = iter;
+	}
+
+	return irq;
+}
+
+/* Request an IRQ for the given mask: share the least loaded existing
+ * matching IRQ once it passes the pool minimum threshold, otherwise
+ * create a new one.
+ */
+static struct zxdh_irq *zxdh_irq_affinity_request(struct zxdh_irq_pool *pool,
+						  const struct cpumask *req_mask)
+{
+	struct zxdh_irq *least_loaded_irq;
+	struct zxdh_irq *new_irq;
+
+	mutex_lock(&pool->lock);
+
+	least_loaded_irq = zxdh_irq_pool_find_least_loaded(pool, req_mask);
+	if (least_loaded_irq &&
+	    refcount_read(&least_loaded_irq->refcount.refcount) < pool->min_threshold)
+		goto out;
+
+	/* No IRQ below the minimum threshold, try to create a new one. */
+	new_irq = zxdh_irq_pool_request_irq(pool, req_mask);
+	if (IS_ERR(new_irq)) {
+		if (!least_loaded_irq) {
+			mutex_unlock(&pool->lock);
+			return new_irq;
+		}
+		/* Could not create a new IRQ for the requested affinity,
+		 * share the existing one.
+		 */
+		dev_warn(pool->dev->device,
+			 "irq pool %s exhausted, sharing irqs\n",
+			 pool->name);
+		goto out;
+	}
+
+	least_loaded_irq = new_irq;
+	goto unlock;
+
+out:
+	/* Holding pool->lock keeps the IRQ alive, so this cannot fail. */
+	zxdh_irq_get(least_loaded_irq);
+	if (refcount_read(&least_loaded_irq->refcount.refcount) > pool->max_threshold)
+		dev_warn(pool->dev->device,
+			 "irq %u overloaded, pool %s, %u refs on this irq\n",
+			 pci_irq_vector(pool->dev->pdev,
+					least_loaded_irq->index),
+			 pool->name,
+			 refcount_read(&least_loaded_irq->refcount.refcount) /
+			 ZXDH_EQ_REFS_PER_IRQ);
+unlock:
+	mutex_unlock(&pool->lock);
+
+	return least_loaded_irq;
+}
+
+/* Request an IRQ from the pool, spread over the online CPUs. */
+struct zxdh_irq *zxdh_get_irq_of_pool(struct zxdh_irq_pool *pool)
+{
+	cpumask_var_t req_mask;
+	struct zxdh_irq *irq;
+
+	if (!zalloc_cpumask_var(&req_mask, GFP_KERNEL))
+		return ERR_PTR(-ENOMEM);
+	cpumask_copy(req_mask, cpu_online_mask);
+
+	irq = zxdh_irq_affinity_request(pool, req_mask);
+
+	free_cpumask_var(req_mask);
+
+	return irq;
+}
+
+struct zxdh_irq_pool *zxdh_irq_pool_alloc(struct zxdh_core_dev *zxdh_dev,
+					  int start, int size, const char *name,
+					  u32 min_threshold, u32 max_threshold)
+{
+	struct zxdh_irq_pool *pool;
+	u16 *irqs_per_cpu;
+
+	irqs_per_cpu = kcalloc(nr_cpu_ids, sizeof(*irqs_per_cpu), GFP_KERNEL);
+	if (!irqs_per_cpu)
+		return ERR_PTR(-ENOMEM);
+
+	pool = kvzalloc_obj(*pool, GFP_KERNEL);
+	if (!pool) {
+		kfree(irqs_per_cpu);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	pool->dev = zxdh_dev;
+	pool->irqs_per_cpu = irqs_per_cpu;
+	mutex_init(&pool->lock);
+	xa_init_flags(&pool->irqs, XA_FLAGS_ALLOC);
+	pool->xa_num_irqs.min = start;
+	pool->xa_num_irqs.max = start + size - 1;
+	strscpy(pool->name, name, sizeof(pool->name));
+
+	/* Thresholds are expressed in queue references, two per IRQ. */
+	pool->min_threshold = min_threshold * ZXDH_EQ_REFS_PER_IRQ;
+	pool->max_threshold = max_threshold * ZXDH_EQ_REFS_PER_IRQ;
+
+	return pool;
+}
+
+void zxdh_irq_pool_free(struct zxdh_irq_pool *pool)
+{
+	u32 cpu;
+
+	WARN_ON_ONCE(!xa_empty(&pool->irqs));
+	xa_destroy(&pool->irqs);
+	mutex_destroy(&pool->lock);
+
+	if (pool->irqs_per_cpu) {
+		for_each_online_cpu(cpu)
+			WARN_ON(pool->irqs_per_cpu[cpu]);
+		kfree(pool->irqs_per_cpu);
+	}
+
+	kvfree(pool);
+}
diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_irq.h b/drivers/net/ethernet/zte/dinghai/zxdh_irq.h
new file mode 100644
index 000000000000..a969e04f3ce4
--- /dev/null
+++ b/drivers/net/ethernet/zte/dinghai/zxdh_irq.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * ZTE DingHai Ethernet driver - IRQ pool management
+ * Copyright (c) 2022-2026, ZTE Corporation.
+ */
+
+#ifndef __ZXDH_IRQ_H__
+#define __ZXDH_IRQ_H__
+
+#include <linux/cpumask.h>
+#include <linux/kref.h>
+#include <linux/mutex.h>
+#include <linux/notifier.h>
+#include <linux/types.h>
+#include <linux/xarray.h>
+
+struct zxdh_core_dev;
+
+/* Number of MSI-X vectors per purpose. The layout of the vector space
+ * is a hardware interface:
+ *   [0, 8)    async event queues
+ *   [8, 14)   reserved for RDMA (unused for now)
+ *   [14, 78)  vq rx/tx queue pairs
+ */
+#define ZXDH_ASYNC_CHANNELS_NUM		8
+#define ZXDH_RDMA_CHANNELS_NUM		6
+#define ZXDH_VQS_CHANNELS_NUM		64
+
+#define ZXDH_MAX_IRQ_NAME	100
+#define ZXDH_MAX_POOL_NAME	16
+#define ZXDH_EQ_REFS_PER_IRQ	2
+
+/* Reference counted interrupt. Event queues share an IRQ through the
+ * notifier chain, which is fired from hard IRQ context.
+ */
+struct zxdh_irq {
+	struct atomic_notifier_head nh;
+	cpumask_var_t mask;		/* interrupt affinity */
+	char name[ZXDH_MAX_IRQ_NAME];
+	struct zxdh_irq_pool *pool;
+	u32 index;			/* vector index */
+	int irqn;			/* Linux IRQ number */
+	struct kref refcount;
+};
+
+/* Pool of vectors dedicated to one purpose, identified by the
+ * [xa_num_irqs.min, xa_num_irqs.max] vector range.
+ */
+struct zxdh_irq_pool {
+	char name[ZXDH_MAX_POOL_NAME];
+	struct xa_limit xa_num_irqs;
+	struct mutex lock;		/* serializes IRQ creation */
+	struct xarray irqs;
+	u32 min_threshold;		/* in queue references */
+	u32 max_threshold;		/* in queue references */
+	u16 *irqs_per_cpu;		/* number of IRQs bound per CPU */
+	struct zxdh_core_dev *dev;
+};
+
+struct zxdh_irq_table {
+	void *priv;
+};
+
+struct zxdh_irq *zxdh_get_irq_of_pool(struct zxdh_irq_pool *pool);
+int zxdh_irq_attach_nb(struct zxdh_irq *irq, struct notifier_block *nb);
+int zxdh_irq_detach_nb(struct zxdh_irq *irq, struct notifier_block *nb);
+void zxdh_irqs_release_vectors(struct zxdh_irq **irqs, int nirqs);
+struct zxdh_irq_pool *zxdh_irq_pool_alloc(struct zxdh_core_dev *zxdh_dev,
+					  int start, int size, const char *name,
+					  u32 min_threshold, u32 max_threshold);
+void zxdh_irq_pool_free(struct zxdh_irq_pool *pool);
+
+#endif /* __ZXDH_IRQ_H__ */
-- 
2.27.0

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH net-next v6 3/3] dinghai: add async event queue for firmware  notifications
  2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
  2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
  2026-11-13  5:52 ` [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools han.junyang
@ 2026-11-13  5:53 ` han.junyang
  2026-11-21 23:31   ` Jakub Kicinski
  2026-11-21 23:44 ` [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues patchwork-bot+netdevbpf
  3 siblings, 1 reply; 8+ messages in thread
From: han.junyang @ 2026-11-13  5:53 UTC (permalink / raw)
  To: andrew+netdev, davem, edumazet, kuba, pabeni, horms
  Cc: linux-kernel, netdev, han.junyang, ran.ming, han.chengfei, zhang.yanze

From: Junyang Han <han.junyang@zte.com.cn>

Add the event queue table and the first async event queue, fed by the
RISC-V management core of the firmware. The queue owns one MSI-X
vector taken from the async pool and, on interrupt, reads the event_id
from the PF receive sub-channel of the BAR message area, maps it to
an event type and fans it out through the per-event-type notifier
chains of the table. The PF probe creates and tears down the queue
through the existing IRQ pool interfaces.

The BAR message channel handshake that tells the firmware which
vectors to signal comes with the BAR message subsystem in a later
series; the PF side wired here is live as soon as that channel is
enabled. Subscribers for the RISCV_READY event register on the
notifier chains in later series as well.

Signed-off-by: Junyang Han <han.junyang@zte.com.cn>
---
 drivers/net/ethernet/zte/dinghai/Makefile  |   2 +-
 drivers/net/ethernet/zte/dinghai/en_pf.c   |  16 ++-
 drivers/net/ethernet/zte/dinghai/en_pf.h   |   2 +
 drivers/net/ethernet/zte/dinghai/zxdh_eq.c | 137 +++++++++++++++++++++
 drivers/net/ethernet/zte/dinghai/zxdh_eq.h |  60 +++++++++
 5 files changed, 211 insertions(+), 6 deletions(-)
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_eq.c
 create mode 100644 drivers/net/ethernet/zte/dinghai/zxdh_eq.h

diff --git a/drivers/net/ethernet/zte/dinghai/Makefile b/drivers/net/ethernet/zte/dinghai/Makefile
index f4a8f9480291..2dc6d68cc0e1 100644
--- a/drivers/net/ethernet/zte/dinghai/Makefile
+++ b/drivers/net/ethernet/zte/dinghai/Makefile
@@ -4,4 +4,4 @@
 #

 obj-$(CONFIG_DINGHAI_PF) += dinghai10e.o
-dinghai10e-y := en_pf.o zxdh_irq.o
+dinghai10e-y := en_pf.o zxdh_irq.o zxdh_eq.o
diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
index 5927f1f39b02..f7400d66bc69 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.c
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.c
@@ -552,13 +552,10 @@ void zxdh_pf_irq_table_destroy(struct zxdh_core_dev *zxdh_dev)
 	pci_free_irq_vectors(zxdh_dev->pdev);
 }

+/* Fetch a vector from the async pool for one async event queue. */
 struct zxdh_irq *zxdh_pf_async_irq_request(struct zxdh_core_dev *zxdh_dev)
 {
-	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
-	struct zxdh_pf_irq_table *pf_irq_table = table->priv;
-
-	if (!pf_irq_table->async_pool)
-		return NULL;
+	struct zxdh_pf_irq_table *pf_irq_table = zxdh_dev->irq_table.priv;

 	return zxdh_get_irq_of_pool(pf_irq_table->async_pool);
 }
@@ -617,10 +614,18 @@ static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		goto err_irq_table;
 	}

+	ret = zxdh_pf_eq_table_init(zxdh_dev);
+	if (ret) {
+		dev_err(&pdev->dev, "zxdh_pf_eq_table_init failed: %d\n", ret);
+		goto err_eq_table;
+	}
+
 	devlink_register(devlink);

 	return 0;

+err_eq_table:
+	zxdh_pf_eq_table_destroy(zxdh_dev);
 err_irq_table:
 	zxdh_pf_irq_table_destroy(zxdh_dev);
 err_modern_cfg:
@@ -640,6 +645,7 @@ static void zxdh_pf_remove(struct pci_dev *pdev)
 	struct devlink *devlink = priv_to_devlink(zxdh_dev);

 	devlink_unregister(devlink);
+	zxdh_pf_eq_table_destroy(zxdh_dev);
 	zxdh_pf_irq_table_destroy(zxdh_dev);
 	zxdh_pf_modern_cfg_uninit(zxdh_dev);
 	zxdh_pf_pci_close(zxdh_dev);
diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.h b/drivers/net/ethernet/zte/dinghai/en_pf.h
index 3de96488a810..b48c786ef48a 100644
--- a/drivers/net/ethernet/zte/dinghai/en_pf.h
+++ b/drivers/net/ethernet/zte/dinghai/en_pf.h
@@ -9,6 +9,7 @@

 #include <linux/types.h>

+#include "zxdh_eq.h"
 #include "zxdh_irq.h"

 #define ZXDH_PF_VENDOR_ID	0x1cf2
@@ -81,6 +82,7 @@ struct zxdh_core_dev {
 	struct pci_dev *pdev;
 	struct devlink *devlink;
 	struct zxdh_irq_table irq_table;
+	struct zxdh_eq_table eq_table;
 	void *priv;
 };

diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_eq.c b/drivers/net/ethernet/zte/dinghai/zxdh_eq.c
new file mode 100644
index 000000000000..3e70f4f3063f
--- /dev/null
+++ b/drivers/net/ethernet/zte/dinghai/zxdh_eq.c
@@ -0,0 +1,137 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * ZTE DingHai Ethernet driver - event queue plumbing
+ * Copyright (c) 2022-2026, ZTE Corporation.
+ *
+ * Async event queues are the delivery path for firmware notifications:
+ * a queue owns one MSI-X vector of the async pool and fans interrupts
+ * out to the per-event-type notifier chains of the event queue table.
+ * The BAR message channel handshake that tells the firmware which
+ * vectors to signal comes with the BAR message subsystem in a later
+ * series; the PF side wired here is live as soon as that channel is
+ * enabled.
+ */
+
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/mutex.h>
+#include <linux/notifier.h>
+#include <linux/slab.h>
+
+#include "en_pf.h"
+#include "zxdh_eq.h"
+#include "zxdh_irq.h"
+
+/* PF side of the event queue table: one async EQ fed by the RISC-V
+ * management core.
+ */
+struct zxdh_pf_eq_table {
+	struct zxdh_eq_async riscv_eq;
+};
+
+/* Fetch the event_id the RISC-V core posted in the PF receive
+ * sub-channel of the BAR message area. The field lives at byte offset
+ * 2 of the 12-byte message header, so a single 32-bit read of the
+ * sub-channel start covers it.
+ */
+static u16 zxdh_eq_event_id_get(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_pf_dev *pf_dev = zxdh_dev->priv;
+	void __iomem *subchan;
+
+	subchan = pf_dev->pci_ioremap_addr[0] + ZXDH_BAR_MSG_SUBCHAN_RECV;
+
+	return ioread32(subchan) >> 16;
+}
+
+static enum zxdh_eq_event_type zxdh_eq_event_type_get(u16 event_id)
+{
+	if (event_id == ZXDH_MSG_MODULE_RISC_READY)
+		return ZXDH_EQ_EVENT_TYPE_RISCV_READY;
+
+	/* Only the RISC-V ready notification is dispatched in this
+	 * series; the remaining firmware events arrive with the BAR
+	 * message subsystem.
+	 */
+	return ZXDH_EQ_EVENT_TYPE_MAX;
+}
+
+static int zxdh_eq_async_riscv_int(struct notifier_block *nb,
+				   unsigned long action, void *data)
+{
+	struct zxdh_eq_async *eq = container_of(nb, struct zxdh_eq_async, irq_nb);
+	struct zxdh_core_dev *zxdh_dev = eq->priv;
+	enum zxdh_eq_event_type event_type;
+
+	event_type = zxdh_eq_event_type_get(zxdh_eq_event_id_get(zxdh_dev));
+	if (event_type == ZXDH_EQ_EVENT_TYPE_MAX)
+		return NOTIFY_DONE;
+
+	atomic_notifier_call_chain(&zxdh_dev->eq_table.nh[event_type],
+				   event_type, NULL);
+
+	return NOTIFY_OK;
+}
+
+int zxdh_pf_eq_table_init(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_eq_table *table = &zxdh_dev->eq_table;
+	struct zxdh_pf_eq_table *priv;
+	struct zxdh_eq_async *eq;
+	int err;
+	int i;
+
+	priv = kvzalloc_obj(*priv, GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+	table->priv = priv;
+
+	mutex_init(&table->lock);
+	for (i = 0; i < ZXDH_EQ_EVENT_TYPE_MAX; i++)
+		ATOMIC_INIT_NOTIFIER_HEAD(&table->nh[i]);
+
+	eq = &priv->riscv_eq;
+	eq->priv = zxdh_dev;
+	eq->irq = zxdh_pf_async_irq_request(zxdh_dev);
+	if (IS_ERR(eq->irq)) {
+		err = PTR_ERR(eq->irq);
+		goto err_irq;
+	}
+
+	eq->irq_nb.notifier_call = zxdh_eq_async_riscv_int;
+	err = zxdh_irq_attach_nb(eq->irq, &eq->irq_nb);
+	if (err)
+		goto err_attach;
+
+	return 0;
+
+err_attach:
+	zxdh_irqs_release_vectors(&eq->irq, 1);
+err_irq:
+	eq->irq = NULL;
+	return err;
+}
+
+void zxdh_pf_eq_table_destroy(struct zxdh_core_dev *zxdh_dev)
+{
+	struct zxdh_pf_eq_table *pf_eq_table = zxdh_dev->eq_table.priv;
+	struct zxdh_eq_table *table = &zxdh_dev->eq_table;
+	struct zxdh_eq_async *eq;
+
+	if (!pf_eq_table)
+		return;
+
+	eq = &pf_eq_table->riscv_eq;
+
+	mutex_lock(&table->lock);
+	if (eq->irq) {
+		zxdh_irq_detach_nb(eq->irq, &eq->irq_nb);
+		zxdh_irqs_release_vectors(&eq->irq, 1);
+		eq->irq = NULL;
+	}
+	mutex_unlock(&table->lock);
+
+	mutex_destroy(&table->lock);
+	kvfree(table->priv);
+	table->priv = NULL;
+}
diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_eq.h b/drivers/net/ethernet/zte/dinghai/zxdh_eq.h
new file mode 100644
index 000000000000..ac9df4763059
--- /dev/null
+++ b/drivers/net/ethernet/zte/dinghai/zxdh_eq.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * ZTE DingHai Ethernet driver - event queue abstraction
+ * Copyright (c) 2022-2026, ZTE Corporation.
+ */
+
+#ifndef __ZXDH_EQ_H__
+#define __ZXDH_EQ_H__
+
+#include <linux/mutex.h>
+#include <linux/notifier.h>
+
+struct zxdh_core_dev;
+struct zxdh_irq;
+
+/* BAR message area: shared-memory mailbox between the driver and the
+ * firmware, split into sub-channels of 2048 bytes. Notifications from
+ * the RISC-V management core are posted to the PF receive sub-channel,
+ * one interval above the area base.
+ */
+#define ZXDH_BAR_MSG_OFFSET		0x2000
+#define ZXDH_BAR_MSG_SUBCHAN_INTERVAL	2048
+#define ZXDH_BAR_MSG_SUBCHAN_RECV	(ZXDH_BAR_MSG_OFFSET +	\
+					 ZXDH_BAR_MSG_SUBCHAN_INTERVAL)
+
+/* event_id of the "RISC-V core ready" firmware notification, carried
+ * at byte offset 2 of the BAR message header.
+ */
+#define ZXDH_MSG_MODULE_RISC_READY	13
+
+/* Event types the upper layers can subscribe to over the per-type
+ * notifier chains of the event queue table. The full catalogue lives
+ * in the firmware; only the ones dispatched in this series are named.
+ */
+enum zxdh_eq_event_type {
+	ZXDH_EQ_EVENT_TYPE_RISCV_READY = 0,
+	ZXDH_EQ_EVENT_TYPE_MAX,
+};
+
+/* Asynchronous event queue delivering firmware notifications to the
+ * driver without a request/reply handshake.
+ */
+struct zxdh_eq_async {
+	struct zxdh_irq *irq;
+	struct notifier_block irq_nb;
+	void *priv;
+};
+
+struct zxdh_eq_table {
+	/* Notifier chains fanned out by event type. */
+	struct atomic_notifier_head nh[ZXDH_EQ_EVENT_TYPE_MAX];
+	/* Serializes async EQ setup/teardown. */
+	struct mutex lock;
+	void *priv;
+};
+
+int zxdh_pf_eq_table_init(struct zxdh_core_dev *zxdh_dev);
+void zxdh_pf_eq_table_destroy(struct zxdh_core_dev *zxdh_dev);
+
+#endif /* __ZXDH_EQ_H__ */
-- 
2.27.0

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling
  2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
@ 2026-11-19 12:05   ` Simon Horman
  0 siblings, 0 replies; 8+ messages in thread
From: Simon Horman @ 2026-11-19 12:05 UTC (permalink / raw)
  To: han.junyang
  Cc: Simon Horman, andrew+netdev, davem, edumazet, kuba, pabeni,
	linux-kernel, netdev, ran.ming, han.chengfei, zhang.yanze

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
Full review at: https://sashiko.dev/#/patchset/20261113135038394NowoSxTdiA32KKAPcTmCa@zte.com.cn?part=1
---
dinghai: add firmware version check and RISC-V readiness polling

This commit polls the BAR 0 firmware compatibility block to verify the
driver/firmware version contract during PF probe and waits for the RISC-V
management core power-on flag when supported.

> diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
> index 86d4374088203..ddff9b0438fb3 100644
> --- a/drivers/net/ethernet/zte/dinghai/en_pf.c
> +++ b/drivers/net/ethernet/zte/dinghai/en_pf.c

[ ... ]

> @@ -369,6 +371,120 @@ int zxdh_pf_modern_cfg_init(struct zxdh_core_dev *zxdh_dev)
>  	return ret;
>  }
>  
> +/* The firmware fills the compatibility block field by field, so it is
> + * only ready to be read once every field we consume has lost its
> + * erased value.
> + */
> +static bool zxdh_pf_fw_compat_ready(struct zxdh_fw_compat __iomem *compat)
> +{
> +	return ioread8(&compat->module_id) != 0xffU &&
> +	       ioread8(&compat->major) != 0xffU &&
> +	       ioread8(&compat->fw_minor) != 0xffU &&
> +	       ioread8(&compat->drv_minor) != 0xffU &&
> +	       ioread16(&compat->patch) != 0xffffU;
> +}
> +
> +/* Read the firmware version block and verify the driver/firmware
> + * version contract.
> + */
> +static int zxdh_pf_fw_compat_check(struct zxdh_core_dev *zxdh_dev)
> +{
> +	struct zxdh_pf_dev *pf_dev = zxdh_dev->priv;
> +	struct zxdh_fw_compat __iomem *compat;
> +	struct zxdh_fw_compat *fw_compat;
> +	bool ready;
> +	int err;
> +
> +	fw_compat = &pf_dev->fw_compat;
> +	compat = pf_dev->pci_ioremap_addr[0] + ZXDH_FW_COMPAT_OFFSET;
> +
> +	/* The firmware publishes the block within 10 s of boot; wait twice
> +	 * that to leave margin.
> +	 */
> +	err = readx_poll_timeout(zxdh_pf_fw_compat_ready, compat, ready,
> +				 ready, USEC_PER_SEC,
> +				 ZXDH_FW_COMPAT_TIMEOUT_SEC * USEC_PER_SEC);
> +	if (err)
> +		dev_info(zxdh_dev->device,
> +			 "compat region not populated, assuming legacy firmware\n");
> +
> +	/* Firmware predating the compatibility region keeps the erased
> +	 * pattern, which fails the module id check below and defers the
> +	 * decision to the readiness wait.
> +	 */
> +	fw_compat->module_id = ioread8(&compat->module_id);
> +	fw_compat->major = ioread8(&compat->major);
> +	fw_compat->fw_minor = ioread8(&compat->fw_minor);
> +	fw_compat->drv_minor = ioread8(&compat->drv_minor);
> +	fw_compat->patch = ioread16(&compat->patch);

[Severity: Medium]
Does falling through on timeout here read and validate a partially populated
compatibility block?

Since the firmware populates compat field by field, readx_poll_timeout() can
time out after module_id is set to ZXDH_MODULE_ID while later fields still
hold 0xff or 0xffff.  In that case the module_id check does not bail out, and
zxdh_pf_fw_compat_check() either fails probe with -EINVAL (if major or
drv_minor is still 0xff) or records fw_compat->patch as 0xffff, causing
zxdh_pf_wait_riscv_ready() to wait 40 seconds for riscv_power_on.  Should the
if (err) branch set fw_compat->patch = 0 and return 0 directly?

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools
  2026-11-13  5:52 ` [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools han.junyang
@ 2026-11-21 23:31   ` Jakub Kicinski
  0 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2026-11-21 23:31 UTC (permalink / raw)
  To: han.junyang
  Cc: Jakub Kicinski, andrew+netdev, davem, edumazet, pabeni, horms,
	linux-kernel, netdev, ran.ming, han.chengfei, zhang.yanze

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
dinghai: add MSI-X interrupt pools

This commit allocates the device's fixed 78-entry MSI-X vector layout. It
also adds a reference counted IRQ pool implementation in zxdh_irq.c. Only
the async event queue pool is wired up so far, and interrupts are fanned out
through a per-IRQ atomic notifier chain.

> diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
> index ddff9b0438fb3..5927f1f39b02a 100644
> --- a/drivers/net/ethernet/zte/dinghai/en_pf.c
> +++ b/drivers/net/ethernet/zte/dinghai/en_pf.c

[ ... ]

> @@ -485,6 +495,74 @@ static int zxdh_pf_wait_riscv_ready(struct zxdh_core_dev *zxdh_dev)

[ ... ]

> +int zxdh_pf_irq_table_init(struct zxdh_core_dev *zxdh_dev)
> +{
> +	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
> +	struct zxdh_pf_irq_table *priv;
> +	int total_vec;
> +
> +	priv = kvzalloc_obj(*priv, GFP_KERNEL);
> +	if (!priv)
> +		return -ENOMEM;
> +	table->priv = priv;
> +
> +	total_vec = ZXDH_VQS_CHANNELS_NUM + ZXDH_ASYNC_CHANNELS_NUM +
> +		    ZXDH_RDMA_CHANNELS_NUM;
> +	total_vec = pci_alloc_irq_vectors(zxdh_dev->pdev, total_vec, total_vec,
> +					  PCI_IRQ_MSIX);
> +	if (total_vec < 0) {
> +		dev_err(zxdh_dev->device, "pci_alloc_irq_vectors failed: %d\n",
> +			total_vec);
> +		return total_vec;
> +	}

[Severity: Medium]
This asks for exactly 78 MSI-X vectors (min_vecs == max_vecs). It doesn't
check the function's MSI-X table size with pci_msix_vec_count(), and there
is no smaller fallback layout. If the call fails, zxdh_pf_probe() aborts.

The same probe path is also bound to the VF device ID:

    { PCI_DEVICE(ZXDH_PF_VENDOR_ID, ZXDH_VF_DEVICE_ID) },

Will a VF, or any function with fewer than 78 MSI-X table entries, now fail
to probe? This commit only uses the 8 async vectors, but it requires all 78.

The commit message calls this "the fixed MSI-X vector layout of the
device". Could it also say whether VFs expose the same table size, and that
probe now fails without the full set?

[Severity: Low]
Probe now requires MSI-X. But DINGHAI_PF in
drivers/net/ethernet/zte/dinghai/Kconfig only gets depends on PCI from its
parent DINGHAI:

    config DINGHAI_PF
        tristate "ZTE DingHai PF (Physical Function) driver"

With CONFIG_PCI=y and CONFIG_PCI_MSI=n, the pci_alloc_irq_vectors_affinity()
stub returns -ENOSPC unless PCI_IRQ_INTX is set and min_vecs is 1. Neither
holds here, so the driver builds but can never bind.

Should DINGHAI_PF depend on PCI_MSI, as other MSI-X only ethernet drivers
do (amazon, google, huawei/hinic3, intel/idpf and others)?

> diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_irq.c b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
> new file mode 100644
> index 0000000000000..bc5348dfa5ba3
> --- /dev/null
> +++ b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
> @@ -0,0 +1,406 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * ZTE DingHai Ethernet driver - IRQ pool management
> + * Copyright (c) 2022-2026, ZTE Corporation.

[Severity: Medium]
The pool code in this file looks closely derived from mlx5's
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c. That file carries:

    // SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
    /* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */

The functions match up as follows:

    zxdh_cpu_get()/zxdh_cpu_put()      -> cpu_get()/cpu_put()
    zxdh_cpu_get_least_loaded()        -> cpu_get_least_loaded()
    zxdh_irq_pool_request_irq()        -> irq_pool_request_irq()
    zxdh_irq_pool_find_least_loaded()  -> irq_pool_find_least_loaded()
    zxdh_irq_affinity_request()        -> mlx5_irq_affinity_request()

The control flow is the same, apart from the renames:

  - the early break on a CPU with no IRQs
  - the cpumask_first(cpu_online_mask) fallback
  - the cpumask_weight() > 1 split
  - the goto out/unlock structure and the "sharing" fallback
  - the overload message that divides by the refs per IRQ constant

The comment in zxdh_irq_release() about clearing the affinity hint before
free_irq() is a rewording of the one in irq_release() in mlx5's pci_irq.c.

Should this file keep the NVIDIA copyright notice, and should the commit
message mention where the code comes from? Picking the GPL-2.0 side of the
dual license is fine, but the original copyright line is dropped here.

[ ... ]

> +static irqreturn_t zxdh_irq_int_handler(int irq, void *data)
> +{
> +	struct zxdh_irq *zxdh_irq = data;
> +
> +	atomic_notifier_call_chain(&zxdh_irq->nh, 0, NULL);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static struct zxdh_irq *zxdh_irq_alloc(struct zxdh_irq_pool *pool, int vecidx,
> +				       const struct cpumask *affinity)
> +{

[ ... ]

> +	ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh);
> +	snprintf(irq->name, ZXDH_MAX_IRQ_NAME, "%s_%d@pci:%s", pool->name,
> +		 vecidx, pci_name(zxdh_dev->pdev));
> +
> +	err = request_irq(irq->irqn, zxdh_irq_int_handler, 0, irq->name, irq);

[Severity: Medium]
Can a firmware event get lost here? request_irq() unmasks the MSI-X entry
while irq->nh is still empty, and the only consumer is attached later.

At the end of the series (after "dinghai: add async event queue for
firmware notifications"), the order is:

zxdh_pf_probe()
  zxdh_pf_wait_riscv_ready()        /* firmware already up */
  zxdh_pf_irq_table_init()          /* MSI-X enabled, entries masked */
  zxdh_pf_eq_table_init()
    zxdh_pf_async_irq_request()
      zxdh_get_irq_of_pool()
        zxdh_irq_affinity_request()
          zxdh_irq_pool_request_irq()
            zxdh_irq_alloc()
              request_irq()         /* unmasked, chain empty */
    zxdh_irq_attach_nb()            /* notifier registered here */

A message raised while the entry was masked is latched in the PBA and sent
when the entry is unmasked. If that happens before zxdh_irq_attach_nb(),
zxdh_irq_int_handler() walks an empty chain and returns IRQ_HANDLED.
Nothing re-reads the BAR message channel after the notifier is attached.

Would using IRQF_NO_AUTOEN and calling enable_irq() after the notifier is
attached avoid this? Re-checking the event source after registration would
also work.

Whether this happens depends on firmware signalling async vector 0 before
the driver's vector handshake, which a later series is said to add. No
subscriber registers on eq_table.nh within this series yet.

[ ... ]

> +	if (affinity) {
> +		cpumask_copy(irq->mask, affinity);
> +	} else {
> +		/* No preference requested; spread over all online CPUs. */
> +		for_each_online_cpu(cpu)
> +			cpumask_set_cpu(cpu, irq->mask);
> +	}

[Severity: Low]
This isn't a bug, but can the affinity == NULL branch ever run? The only
caller, zxdh_irq_pool_request_irq(), always passes req_mask or auto_mask.
Before that, it already dereferences req_mask in cpumask_weight() and
cpumask_first(). zxdh_get_irq_of_pool() always passes a copy of
cpu_online_mask.

The comment suggests NULL is a valid input, but a NULL mask would crash
before reaching this branch. Could the branch be removed?

[ ... ]

> +/* Find the least loaded CPU, i.e. the CPU with the fewest IRQs bound
> + * to it, within the requested mask.
> + */
> +static int zxdh_cpu_get_least_loaded(struct zxdh_irq_pool *pool,
> +				     const struct cpumask *req_mask)
> +{

[ ... ]

> +	pool->irqs_per_cpu[best_cpu]++;
> +
> +	return best_cpu;
> +}

[Severity: Low]
This isn't a bug, but the comment says this only finds the least loaded
CPU. It also increments irqs_per_cpu[best_cpu].

zxdh_irq_pool_request_irq() depends on that increment. It calls
zxdh_cpu_get() only in the single CPU branch, and undoes the count with
zxdh_cpu_put() on error. If someone followed the comment and added a
zxdh_cpu_get() to the multi CPU branch, the CPU would be counted twice and
the WARN_ON() in zxdh_irq_pool_free() would fire.

The lock in struct zxdh_irq_pool in zxdh_irq.h has a similar problem. It is
documented as:

    struct mutex lock;		/* serializes IRQ creation */

The code relies on it for more than creation:

  - zxdh_irq_release() and zxdh_irq_pool_find_least_loaded() both assert it
  - the "cannot fail" zxdh_irq_get() in zxdh_irq_affinity_request() is only
    safe because release is serialized by it
  - it covers all irqs_per_cpu updates

Could both comments describe what the code relies on?

[ ... ]

> +/* Request an IRQ for the given mask: share the least loaded existing
> + * matching IRQ once it passes the pool minimum threshold, otherwise
> + * create a new one.
> + */
> +static struct zxdh_irq *zxdh_irq_affinity_request(struct zxdh_irq_pool *pool,
> +						  const struct cpumask *req_mask)
> +{
> +	struct zxdh_irq *least_loaded_irq;
> +	struct zxdh_irq *new_irq;
> +
> +	mutex_lock(&pool->lock);
> +
> +	least_loaded_irq = zxdh_irq_pool_find_least_loaded(pool, req_mask);
> +	if (least_loaded_irq &&
> +	    refcount_read(&least_loaded_irq->refcount.refcount) < pool->min_threshold)
> +		goto out;

[Severity: Medium]
Does the header comment describe the opposite of what the code does? It
says an existing IRQ is shared "once it passes the pool minimum threshold".
The commit message says the same:

    The pool hands out the least loaded matching IRQ once it passes
    the pool minimum threshold

The code shares an IRQ only while its refcount is below min_threshold. At
or above the threshold, it creates a new IRQ with
zxdh_irq_pool_request_irq(). It shares only if that fails because the pool
is exhausted.

The early return in zxdh_irq_pool_find_least_loaded() and the inner
comments here match the code, not the header comment or the commit message.

For the async pool, ZXDH_PF_ASYNC_IRQ_MIN_COMP is 0:

    #define ZXDH_PF_ASYNC_IRQ_MIN_COMP	0

So pool->min_threshold is 0, and this unsigned comparison can never be true.
Every async request creates a new vector until all 8 are used. Only then
does it share, with the "exhausted" warning.

Is that the intended policy? If so, could the header comment and the commit
message be updated to match?

[ ... ]

> +out:
> +	/* Holding pool->lock keeps the IRQ alive, so this cannot fail. */
> +	zxdh_irq_get(least_loaded_irq);
> +	if (refcount_read(&least_loaded_irq->refcount.refcount) > pool->max_threshold)
> +		dev_warn(pool->dev->device,
> +			 "irq %u overloaded, pool %s, %u refs on this irq\n",
> +			 pci_irq_vector(pool->dev->pdev,
> +					least_loaded_irq->index),
> +			 pool->name,
> +			 refcount_read(&least_loaded_irq->refcount.refcount) /
> +			 ZXDH_EQ_REFS_PER_IRQ);

[Severity: Low]
This isn't a bug, but the units look mixed up here. Each event queue takes
two references: one from zxdh_get_irq_of_pool() and one from
zxdh_irq_attach_nb(). So refcount / ZXDH_EQ_REFS_PER_IRQ is the number of
queues, not "refs on this irq". The mlx5 version prints "%u EQs on this
irq".

The comment in zxdh_irq_pool_alloc() has the same mix-up:

    /* Thresholds are expressed in queue references, two per IRQ. */

The factor is two per queue, not per IRQ, and the ZXDH_EQ_REFS_PER_IRQ
name has the same problem.

This warning also prints the int returned by pci_irq_vector() with %u.
Could it use least_loaded_irq->irqn instead?

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications
  2026-11-13  5:53 ` [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications han.junyang
@ 2026-11-21 23:31   ` Jakub Kicinski
  0 siblings, 0 replies; 8+ messages in thread
From: Jakub Kicinski @ 2026-11-21 23:31 UTC (permalink / raw)
  To: han.junyang
  Cc: Jakub Kicinski, andrew+netdev, davem, edumazet, pabeni, horms,
	linux-kernel, netdev, ran.ming, han.chengfei, zhang.yanze

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
--- Patch [3]: [PATCH net-next v6 3/3] dinghai: add async event queue for firmware
 notifications ---
commit 0beec1cea24be93e6e75a8732e384d617376a13d
Author: Junyang Han <han.junyang@zte.com.cn>

dinghai: add async event queue for firmware notifications

This adds an event queue table and one async event queue that receives
firmware notifications from the RISC-V management core. The queue takes
one MSI-X vector from the async pool. Its interrupt handler reads an
event_id from the PF receive sub-channel of the BAR message area and
dispatches it through per-event-type notifier chains, and the PF
probe/remove paths create and tear down the queue.

> diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
> index 5927f1f39b02a..f7400d66bc690 100644
> --- a/drivers/net/ethernet/zte/dinghai/en_pf.c
> +++ b/drivers/net/ethernet/zte/dinghai/en_pf.c

[ ... ]

> @@ -617,10 +614,18 @@ static int zxdh_pf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>  		goto err_irq_table;
>  	}
>  
> +	ret = zxdh_pf_eq_table_init(zxdh_dev);
> +	if (ret) {
> +		dev_err(&pdev->dev, "zxdh_pf_eq_table_init failed: %d\n", ret);
> +		goto err_eq_table;
> +	}

[Severity: Low]
Does this queue have any users in this series? Every probe now takes an
async-pool vector and installs a hard IRQ handler:

zxdh_pf_eq_table_init()
  zxdh_pf_async_irq_request()
    zxdh_get_irq_of_pool()
      zxdh_irq_affinity_request()
        zxdh_irq_pool_request_irq()
          zxdh_irq_alloc()
            request_irq()

When the interrupt fires, zxdh_eq_async_riscv_int() dispatches to
zxdh_dev->eq_table.nh[], but nothing registers on those chains. Under
drivers/net/ethernet/zte/, the only reference to eq_table.nh[] is the
atomic_notifier_call_chain() call in zxdh_eq_async_riscv_int().

zxdh_eq.h also exports only zxdh_pf_eq_table_init() and
zxdh_pf_eq_table_destroy(), with no register/unregister helper. Later
subscribers would have to reach into struct zxdh_eq_table directly.

The commit message says:

  "the PF side wired here is live as soon as that channel is enabled.
  Subscribers for the RISCV_READY event register on the notifier chains
  in later series as well."

Even with the channel enabled, every RISCV_READY dispatch would go to an
empty chain. Could this be merged together with the BAR message channel
and the first subscriber? Then the design could be reviewed against its
consumers, including the event payload, valid/ack handling and ownership
of the mailbox offsets.

[ ... ]

> diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_eq.c b/drivers/net/ethernet/zte/dinghai/zxdh_eq.c
> new file mode 100644
> index 0000000000000..3e70f4f3063fe
> --- /dev/null
> +++ b/drivers/net/ethernet/zte/dinghai/zxdh_eq.c

[ ... ]

> +/* Fetch the event_id the RISC-V core posted in the PF receive
> + * sub-channel of the BAR message area. The field lives at byte offset
> + * 2 of the 12-byte message header, so a single 32-bit read of the
> + * sub-channel start covers it.
> + */
> +static u16 zxdh_eq_event_id_get(struct zxdh_core_dev *zxdh_dev)
> +{
> +	struct zxdh_pf_dev *pf_dev = zxdh_dev->priv;
> +	void __iomem *subchan;
> +
> +	subchan = pf_dev->pci_ioremap_addr[0] + ZXDH_BAR_MSG_SUBCHAN_RECV;
> +
> +	return ioread32(subchan) >> 16;

[Severity: Medium]
Should the handler check that a new message was actually posted before
it uses this event_id?

The >> 16 discards bytes 0-1 of the header. In the 12-byte BAR message
header, byte 0 holds the per-message flag bits (valid, sync, emec, ack,
poll, usr).

The BAR mailbox protocol isn't in the tree yet, so this is based on the
DPDK zxdh PMD. Its header layout matches the comment here (event_id at
offset 2, RISC_READY = 13). Its receive path rejects any header without
the valid bit in zxdh_bar_chan_msg_header_check().

zxdh_irq_int_handler() calls the chain on every interrupt:

	atomic_notifier_call_chain(&zxdh_irq->nh, 0, NULL);

	return IRQ_HANDLED;

so nothing earlier filters these events. If the mailbox still says 13,
ZXDH_EQ_EVENT_TYPE_RISCV_READY would be dispatched again on any
interrupt on this vector. That includes a stale header from an earlier
notification, an uninitialized mailbox, or a spurious MSI-X. It also
covers interrupts meant for other users if zxdh_irq_affinity_request()
shares the vector when the pool runs out.

Can subscribers then receive a RISCV_READY event that the firmware
didn't post for this interrupt?

> +}

[ ... ]

> +static int zxdh_eq_async_riscv_int(struct notifier_block *nb,
> +				   unsigned long action, void *data)
> +{
> +	struct zxdh_eq_async *eq = container_of(nb, struct zxdh_eq_async, irq_nb);
> +	struct zxdh_core_dev *zxdh_dev = eq->priv;
> +	enum zxdh_eq_event_type event_type;
> +
> +	event_type = zxdh_eq_event_type_get(zxdh_eq_event_id_get(zxdh_dev));
> +	if (event_type == ZXDH_EQ_EVENT_TYPE_MAX)
> +		return NOTIFY_DONE;
> +
> +	atomic_notifier_call_chain(&zxdh_dev->eq_table.nh[event_type],
> +				   event_type, NULL);
> +
> +	return NOTIFY_OK;

[Severity: Medium]
What gives the receive sub-channel back to the firmware once the
message is handled? This handler is its only consumer, and it returns
without writing anything to the mailbox.

In the DPDK zxdh receive path, zxdh_bar_irq_recv() resets the header's
valid field to usable. For sync messages it also writes a reply header
with the ack bit set.

Without this, could the RISC-V core keep seeing an unconsumed message
and fail to post the next notification? Could a sync-flagged sender
wait for an ack until it times out? The old header would also stay in
the mailbox, so the stale re-dispatch described above could repeat on
every later interrupt.

> +}

[ ... ]

> +void zxdh_pf_eq_table_destroy(struct zxdh_core_dev *zxdh_dev)
> +{
> +	struct zxdh_pf_eq_table *pf_eq_table = zxdh_dev->eq_table.priv;
> +	struct zxdh_eq_table *table = &zxdh_dev->eq_table;
> +	struct zxdh_eq_async *eq;
> +
> +	if (!pf_eq_table)
> +		return;
> +
> +	eq = &pf_eq_table->riscv_eq;
> +
> +	mutex_lock(&table->lock);
> +	if (eq->irq) {
> +		zxdh_irq_detach_nb(eq->irq, &eq->irq_nb);
> +		zxdh_irqs_release_vectors(&eq->irq, 1);
> +		eq->irq = NULL;
> +	}
> +	mutex_unlock(&table->lock);

[Severity: Low]
This isn't a bug, but what does table->lock protect? zxdh_eq.h
documents it as:

	/* Serializes async EQ setup/teardown. */
	struct mutex lock;

zxdh_pf_eq_table_init() only calls mutex_init(). It then sets eq->irq
and calls zxdh_irq_attach_nb() without taking the lock. This
mutex_lock() in zxdh_pf_eq_table_destroy() is the only place it is
acquired.

The driver core already serializes probe and remove, and the hard IRQ
path can't take a mutex. Holding the lock here also keeps it held across
synchronize_rcu() in atomic_notifier_chain_unregister(),
synchronize_irq(), pool->lock and free_irq().

Could the lock be removed, or the comment changed to match what it
actually guarantees?

> +
> +	mutex_destroy(&table->lock);
> +	kvfree(table->priv);
> +	table->priv = NULL;
> +}

[ ... ]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues
  2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
                   ` (2 preceding siblings ...)
  2026-11-13  5:53 ` [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications han.junyang
@ 2026-11-21 23:44 ` patchwork-bot+netdevbpf
  3 siblings, 0 replies; 8+ messages in thread
From: patchwork-bot+netdevbpf @ 2026-11-21 23:44 UTC (permalink / raw)
  To: Junyang Han
  Cc: andrew+netdev, davem, edumazet, kuba, pabeni, horms,
	linux-kernel, netdev, ran.ming, han.chengfei, zhang.yanze

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Fri, 13 Nov 2026 13:47:43 +0800 (CST) you wrote:
> From: Junyang Han <han.junyang@zte.com.cn>
> 
> This series continues the DingHai (ZXDH) PF driver bring-up: after PCI
> probing, it verifies the firmware version contract, waits for the
> RISC-V management core to become ready, sets up the MSI-X interrupt
> pools, and creates the async event queues through which the firmware
> will report events.
> 
> [...]

Here is the summary with links:
  - [net-next,v6,1/3] dinghai: add firmware version check and RISC-V readiness polling
    https://git.kernel.org/netdev/net-next/c/a7e1c56d1f95
  - [net-next,v6,2/3] dinghai: add MSI-X interrupt pools
    (no matching commit)
  - [net-next,v6,3/3] dinghai: add async event queue for firmware notifications
    (no matching commit)

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-21 23:44 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
2026-11-19 12:05   ` Simon Horman
2026-11-13  5:52 ` [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools han.junyang
2026-11-21 23:31   ` Jakub Kicinski
2026-11-13  5:53 ` [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications han.junyang
2026-11-21 23:31   ` Jakub Kicinski
2026-11-21 23:44 ` [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues patchwork-bot+netdevbpf

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®