Re: [RFC PATCH] irqchip/gic-v3-its: enable dynamic MSI-X allocation

[Jinqian Yang <yangjinqian1@huawei.com>]

On 2026/6/24 15:07, Marc Zyngier wrote:
> On Wed, 24 Jun 2026 03:53:45 +0100,
> Jinqian Yang <yangjinqian1@huawei.com> wrote:
>>
>> On ARM64 platforms with GICv3 ITS, VFIO PCI passthrough currently
>> cannot dynamically allocate MSI-X vectors after MSI-X has been
>> enabled. When QEMU needs to extend the vector range, it must
>> disable MSI-X, free all interrupts, then re-enable with a larger
>> allocation. This creates an interrupt loss window for already-active
>> vectors.
>>
>> Consider HNS3 with RoCE: NIC and RDMA share one PCI device and
>> ITS DeviceID, with MSI-X vectors partitioned as NIC (lower range)
>> then RoCE (starting at base_vector = num_nic_msi). In VFIO
>> passthrough, loading hns_roce after hns3 forces QEMU to tear down
>> all interrupts before re-allocating the larger range. During this
>> process, NIC interrupts may be lost. Testing confirmed that this
>> occasionally occurs, causing the network port reset to fail.
> 
> Well, that's what you get for not exposing differentiated functions.
> Eventually, you face the reality that this is a poor design.
> 

Fair point, though this is not unique to HNS3.. All major NIC+RDMA
vendors share the same PCI function.

>>
>> ITS_MSI_FLAGS_SUPPORTED lacks MSI_FLAG_PCI_MSIX_ALLOC_DYN, causing
>> pci_msix_can_alloc_dyn() to return false. VFIO then sets
>> has_dyn_msix=false and never clears VFIO_IRQ_INFO_NORESIZE for
>> MSI-X, keeping the old "disable and reallocate" behavior.
>>
>> The essential prerequisite for enabling this flag is the fix to
>> msi_prepare() call timing (commit 1396e89e09f0 ("genirq/msi: Move
>> prepare() call to per-device allocation")): msi_prepare() is
>> now called once at per-device domain creation with hwsize, so ITS
>> creates an ITT with sufficient capacity for all MSI-X vectors.
>> Without this fix, msi_prepare() was called per-allocation with
>> semi-random nvec, maybe resulting in an ITT too small for dynamic
>> vector addition.
> 
> How is this paragraph relevant? The kernel has had this fix for over a
> year, and backporting this series is not something I plan to ever do.
> 

Will remove from commit msg.

>>
>> With this in place, dynamic MSI-X allocation works correctly:
>> msi_domain_alloc_irq_at() uses populate_alloc_info() to copy the
>> pre-prepared alloc_data without re-invoking msi_prepare(), so each
>> new vector simply gets a LPI entry in the already-allocated ITT,
>> without affecting existing vectors.
>>
>> Signed-off-by: Jinqian Yang <yangjinqian1@huawei.com>
>> ---
>>   drivers/irqchip/irq-gic-its-msi-parent.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/irqchip/irq-gic-its-msi-parent.c b/drivers/irqchip/irq-gic-its-msi-parent.c
>> index b9257103a999..b2b9d2068bb1 100644
>> --- a/drivers/irqchip/irq-gic-its-msi-parent.c
>> +++ b/drivers/irqchip/irq-gic-its-msi-parent.c
>> @@ -18,7 +18,8 @@
>>   
>>   #define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
>>   				 MSI_FLAG_PCI_MSIX      |	\
>> -				 MSI_FLAG_MULTI_PCI_MSI)
>> +				 MSI_FLAG_MULTI_PCI_MSI |	\
>> +				 MSI_FLAG_PCI_MSIX_ALLOC_DYN)
>>   
>>   static int its_translate_frame_address(struct fwnode_handle *msi_node, phys_addr_t *pa)
>>   {
> 
> What has this been tested with? In which conditions?
> 

Tested on Hisilicon HIP09 (ARM64, GICv3/GICv4.1) with latest
upstream kernel and QEMU 8.2.

VFIO passthrough of HNS3 NIC to VM: load both hns3 and
hns_roce_hw_v2 drivers, then trigger FLR. Without the flag,
QEMU disables/re-enables MSI-X around FLR, causing occasional
link up failure due to interrupt loss.

Thanks,
Jinqian