Re: [PATCH] irqchip/gic-v3-its: Enable dynamic MSI-X allocation

[Vlad Poenaru <vlad.wing@gmail.com>]

Two updates on this report.

First, a correction: it is not specific to our debug config. Our
production config hits it as well, on every Grace + BCM57508 machine
we tried, at the same point in boot.

Second, the root cause is in bnxt_en, and this commit only exposes it.
When the channel count is raised, bnxt_check_rings() allocates the
extra MSI-X vectors before firmware has reserved them for the
function. On BCM57508, writes to MSI-X table entries beyond the
reserved IRQs don't stick, so the message written when the vectors
are activated is lost, and the entries stay zero after the
reservation.
x86 never notices because GENERIC_IRQ_RESERVATION_MODE makes the PCI
MSI domains MSI_FLAG_MUST_REACTIVATE and the message is written again
at request_irq() time. With this commit, arm64 GICv3 takes the same
dynamic path but writes the message only once, at allocation.

Instrumenting the driver showed it directly. After the channel count
went from 18 to 32, MSI-X entries 18-31 read back as all zero, while
the cached msi_msg and the ITS LPI mapping were correct. Exactly the
14 rings on those vectors lost their completions.

The fix is posted to netdev:

  https://lore.kernel.org/all/20261023121321.2127263-1-vlad.wing@gmail.com/

With it, the same machine runs cleanly through the channel change and
the ring resize. I don't think the ITS change needs to be reverted.

#regzbot monitor: https://lore.kernel.org/all/20261023121321.2127263-1-vlad.wing@gmail.com/

Thanks,
Vlad