mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: han.junyang@zte.com.cn
Cc: Jakub Kicinski <kuba@kernel.org>,
	andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com,
	pabeni@redhat.com, horms@kernel.org,
	linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
	ran.ming@zte.com.cn, han.chengfei@zte.com.cn,
	zhang.yanze@zte.com.cn
Subject: Re: [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools
Date: Sat, 21 Nov 2026 15:31:37 -0800	[thread overview]
Message-ID: <20261121233137.1369263-1-kuba@kernel.org> (raw)
In-Reply-To: <20261113135220713uvE1CCvGgC9U59hVZGJsZ@zte.com.cn>

This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
---
dinghai: add MSI-X interrupt pools

This commit allocates the device's fixed 78-entry MSI-X vector layout. It
also adds a reference counted IRQ pool implementation in zxdh_irq.c. Only
the async event queue pool is wired up so far, and interrupts are fanned out
through a per-IRQ atomic notifier chain.

> diff --git a/drivers/net/ethernet/zte/dinghai/en_pf.c b/drivers/net/ethernet/zte/dinghai/en_pf.c
> index ddff9b0438fb3..5927f1f39b02a 100644
> --- a/drivers/net/ethernet/zte/dinghai/en_pf.c
> +++ b/drivers/net/ethernet/zte/dinghai/en_pf.c

[ ... ]

> @@ -485,6 +495,74 @@ static int zxdh_pf_wait_riscv_ready(struct zxdh_core_dev *zxdh_dev)

[ ... ]

> +int zxdh_pf_irq_table_init(struct zxdh_core_dev *zxdh_dev)
> +{
> +	struct zxdh_irq_table *table = &zxdh_dev->irq_table;
> +	struct zxdh_pf_irq_table *priv;
> +	int total_vec;
> +
> +	priv = kvzalloc_obj(*priv, GFP_KERNEL);
> +	if (!priv)
> +		return -ENOMEM;
> +	table->priv = priv;
> +
> +	total_vec = ZXDH_VQS_CHANNELS_NUM + ZXDH_ASYNC_CHANNELS_NUM +
> +		    ZXDH_RDMA_CHANNELS_NUM;
> +	total_vec = pci_alloc_irq_vectors(zxdh_dev->pdev, total_vec, total_vec,
> +					  PCI_IRQ_MSIX);
> +	if (total_vec < 0) {
> +		dev_err(zxdh_dev->device, "pci_alloc_irq_vectors failed: %d\n",
> +			total_vec);
> +		return total_vec;
> +	}

[Severity: Medium]
This asks for exactly 78 MSI-X vectors (min_vecs == max_vecs). It doesn't
check the function's MSI-X table size with pci_msix_vec_count(), and there
is no smaller fallback layout. If the call fails, zxdh_pf_probe() aborts.

The same probe path is also bound to the VF device ID:

    { PCI_DEVICE(ZXDH_PF_VENDOR_ID, ZXDH_VF_DEVICE_ID) },

Will a VF, or any function with fewer than 78 MSI-X table entries, now fail
to probe? This commit only uses the 8 async vectors, but it requires all 78.

The commit message calls this "the fixed MSI-X vector layout of the
device". Could it also say whether VFs expose the same table size, and that
probe now fails without the full set?

[Severity: Low]
Probe now requires MSI-X. But DINGHAI_PF in
drivers/net/ethernet/zte/dinghai/Kconfig only gets depends on PCI from its
parent DINGHAI:

    config DINGHAI_PF
        tristate "ZTE DingHai PF (Physical Function) driver"

With CONFIG_PCI=y and CONFIG_PCI_MSI=n, the pci_alloc_irq_vectors_affinity()
stub returns -ENOSPC unless PCI_IRQ_INTX is set and min_vecs is 1. Neither
holds here, so the driver builds but can never bind.

Should DINGHAI_PF depend on PCI_MSI, as other MSI-X only ethernet drivers
do (amazon, google, huawei/hinic3, intel/idpf and others)?

> diff --git a/drivers/net/ethernet/zte/dinghai/zxdh_irq.c b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
> new file mode 100644
> index 0000000000000..bc5348dfa5ba3
> --- /dev/null
> +++ b/drivers/net/ethernet/zte/dinghai/zxdh_irq.c
> @@ -0,0 +1,406 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * ZTE DingHai Ethernet driver - IRQ pool management
> + * Copyright (c) 2022-2026, ZTE Corporation.

[Severity: Medium]
The pool code in this file looks closely derived from mlx5's
drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c. That file carries:

    // SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
    /* Copyright (c) 2021, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */

The functions match up as follows:

    zxdh_cpu_get()/zxdh_cpu_put()      -> cpu_get()/cpu_put()
    zxdh_cpu_get_least_loaded()        -> cpu_get_least_loaded()
    zxdh_irq_pool_request_irq()        -> irq_pool_request_irq()
    zxdh_irq_pool_find_least_loaded()  -> irq_pool_find_least_loaded()
    zxdh_irq_affinity_request()        -> mlx5_irq_affinity_request()

The control flow is the same, apart from the renames:

  - the early break on a CPU with no IRQs
  - the cpumask_first(cpu_online_mask) fallback
  - the cpumask_weight() > 1 split
  - the goto out/unlock structure and the "sharing" fallback
  - the overload message that divides by the refs per IRQ constant

The comment in zxdh_irq_release() about clearing the affinity hint before
free_irq() is a rewording of the one in irq_release() in mlx5's pci_irq.c.

Should this file keep the NVIDIA copyright notice, and should the commit
message mention where the code comes from? Picking the GPL-2.0 side of the
dual license is fine, but the original copyright line is dropped here.

[ ... ]

> +static irqreturn_t zxdh_irq_int_handler(int irq, void *data)
> +{
> +	struct zxdh_irq *zxdh_irq = data;
> +
> +	atomic_notifier_call_chain(&zxdh_irq->nh, 0, NULL);
> +
> +	return IRQ_HANDLED;
> +}
> +
> +static struct zxdh_irq *zxdh_irq_alloc(struct zxdh_irq_pool *pool, int vecidx,
> +				       const struct cpumask *affinity)
> +{

[ ... ]

> +	ATOMIC_INIT_NOTIFIER_HEAD(&irq->nh);
> +	snprintf(irq->name, ZXDH_MAX_IRQ_NAME, "%s_%d@pci:%s", pool->name,
> +		 vecidx, pci_name(zxdh_dev->pdev));
> +
> +	err = request_irq(irq->irqn, zxdh_irq_int_handler, 0, irq->name, irq);

[Severity: Medium]
Can a firmware event get lost here? request_irq() unmasks the MSI-X entry
while irq->nh is still empty, and the only consumer is attached later.

At the end of the series (after "dinghai: add async event queue for
firmware notifications"), the order is:

zxdh_pf_probe()
  zxdh_pf_wait_riscv_ready()        /* firmware already up */
  zxdh_pf_irq_table_init()          /* MSI-X enabled, entries masked */
  zxdh_pf_eq_table_init()
    zxdh_pf_async_irq_request()
      zxdh_get_irq_of_pool()
        zxdh_irq_affinity_request()
          zxdh_irq_pool_request_irq()
            zxdh_irq_alloc()
              request_irq()         /* unmasked, chain empty */
    zxdh_irq_attach_nb()            /* notifier registered here */

A message raised while the entry was masked is latched in the PBA and sent
when the entry is unmasked. If that happens before zxdh_irq_attach_nb(),
zxdh_irq_int_handler() walks an empty chain and returns IRQ_HANDLED.
Nothing re-reads the BAR message channel after the notifier is attached.

Would using IRQF_NO_AUTOEN and calling enable_irq() after the notifier is
attached avoid this? Re-checking the event source after registration would
also work.

Whether this happens depends on firmware signalling async vector 0 before
the driver's vector handshake, which a later series is said to add. No
subscriber registers on eq_table.nh within this series yet.

[ ... ]

> +	if (affinity) {
> +		cpumask_copy(irq->mask, affinity);
> +	} else {
> +		/* No preference requested; spread over all online CPUs. */
> +		for_each_online_cpu(cpu)
> +			cpumask_set_cpu(cpu, irq->mask);
> +	}

[Severity: Low]
This isn't a bug, but can the affinity == NULL branch ever run? The only
caller, zxdh_irq_pool_request_irq(), always passes req_mask or auto_mask.
Before that, it already dereferences req_mask in cpumask_weight() and
cpumask_first(). zxdh_get_irq_of_pool() always passes a copy of
cpu_online_mask.

The comment suggests NULL is a valid input, but a NULL mask would crash
before reaching this branch. Could the branch be removed?

[ ... ]

> +/* Find the least loaded CPU, i.e. the CPU with the fewest IRQs bound
> + * to it, within the requested mask.
> + */
> +static int zxdh_cpu_get_least_loaded(struct zxdh_irq_pool *pool,
> +				     const struct cpumask *req_mask)
> +{

[ ... ]

> +	pool->irqs_per_cpu[best_cpu]++;
> +
> +	return best_cpu;
> +}

[Severity: Low]
This isn't a bug, but the comment says this only finds the least loaded
CPU. It also increments irqs_per_cpu[best_cpu].

zxdh_irq_pool_request_irq() depends on that increment. It calls
zxdh_cpu_get() only in the single CPU branch, and undoes the count with
zxdh_cpu_put() on error. If someone followed the comment and added a
zxdh_cpu_get() to the multi CPU branch, the CPU would be counted twice and
the WARN_ON() in zxdh_irq_pool_free() would fire.

The lock in struct zxdh_irq_pool in zxdh_irq.h has a similar problem. It is
documented as:

    struct mutex lock;		/* serializes IRQ creation */

The code relies on it for more than creation:

  - zxdh_irq_release() and zxdh_irq_pool_find_least_loaded() both assert it
  - the "cannot fail" zxdh_irq_get() in zxdh_irq_affinity_request() is only
    safe because release is serialized by it
  - it covers all irqs_per_cpu updates

Could both comments describe what the code relies on?

[ ... ]

> +/* Request an IRQ for the given mask: share the least loaded existing
> + * matching IRQ once it passes the pool minimum threshold, otherwise
> + * create a new one.
> + */
> +static struct zxdh_irq *zxdh_irq_affinity_request(struct zxdh_irq_pool *pool,
> +						  const struct cpumask *req_mask)
> +{
> +	struct zxdh_irq *least_loaded_irq;
> +	struct zxdh_irq *new_irq;
> +
> +	mutex_lock(&pool->lock);
> +
> +	least_loaded_irq = zxdh_irq_pool_find_least_loaded(pool, req_mask);
> +	if (least_loaded_irq &&
> +	    refcount_read(&least_loaded_irq->refcount.refcount) < pool->min_threshold)
> +		goto out;

[Severity: Medium]
Does the header comment describe the opposite of what the code does? It
says an existing IRQ is shared "once it passes the pool minimum threshold".
The commit message says the same:

    The pool hands out the least loaded matching IRQ once it passes
    the pool minimum threshold

The code shares an IRQ only while its refcount is below min_threshold. At
or above the threshold, it creates a new IRQ with
zxdh_irq_pool_request_irq(). It shares only if that fails because the pool
is exhausted.

The early return in zxdh_irq_pool_find_least_loaded() and the inner
comments here match the code, not the header comment or the commit message.

For the async pool, ZXDH_PF_ASYNC_IRQ_MIN_COMP is 0:

    #define ZXDH_PF_ASYNC_IRQ_MIN_COMP	0

So pool->min_threshold is 0, and this unsigned comparison can never be true.
Every async request creates a new vector until all 8 are used. Only then
does it share, with the "exhausted" warning.

Is that the intended policy? If so, could the header comment and the commit
message be updated to match?

[ ... ]

> +out:
> +	/* Holding pool->lock keeps the IRQ alive, so this cannot fail. */
> +	zxdh_irq_get(least_loaded_irq);
> +	if (refcount_read(&least_loaded_irq->refcount.refcount) > pool->max_threshold)
> +		dev_warn(pool->dev->device,
> +			 "irq %u overloaded, pool %s, %u refs on this irq\n",
> +			 pci_irq_vector(pool->dev->pdev,
> +					least_loaded_irq->index),
> +			 pool->name,
> +			 refcount_read(&least_loaded_irq->refcount.refcount) /
> +			 ZXDH_EQ_REFS_PER_IRQ);

[Severity: Low]
This isn't a bug, but the units look mixed up here. Each event queue takes
two references: one from zxdh_get_irq_of_pool() and one from
zxdh_irq_attach_nb(). So refcount / ZXDH_EQ_REFS_PER_IRQ is the number of
queues, not "refs on this irq". The mlx5 version prints "%u EQs on this
irq".

The comment in zxdh_irq_pool_alloc() has the same mix-up:

    /* Thresholds are expressed in queue references, two per IRQ. */

The factor is two per queue, not per IRQ, and the ZXDH_EQ_REFS_PER_IRQ
name has the same problem.

This warning also prints the int returned by pci_irq_vector() with %u.
Could it use least_loaded_irq->irqn instead?

  reply	other threads:[~2026-11-21 23:31 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13  5:47 [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues han.junyang
2026-11-13  5:50 ` [PATCH net-next v6 1/3] dinghai: add firmware version check and RISC-V readiness polling han.junyang
2026-11-19 12:05   ` Simon Horman
2026-11-13  5:52 ` [PATCH net-next v6 2/3] dinghai: add MSI-X interrupt pools han.junyang
2026-11-21 23:31   ` Jakub Kicinski [this message]
2026-11-13  5:53 ` [PATCH net-next v6 3/3] dinghai: add async event queue for firmware notifications han.junyang
2026-11-21 23:31   ` Jakub Kicinski
2026-11-21 23:44 ` [PATCH net-next v6 0/3] dinghai: firmware handshake, MSI-X pools and async event queues patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121233137.1369263-1-kuba@kernel.org \
    --to=kuba@kernel.org \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=han.chengfei@zte.com.cn \
    --cc=han.junyang@zte.com.cn \
    --cc=horms@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=ran.ming@zte.com.cn \
    --cc=zhang.yanze@zte.com.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®