Re: [PATCH 3/7] vfio/pci: Add qcom-vfio-pci variant driver

[Jose Ignacio Tornos Martinez <jtornosm@redhat.com>]

Hi Jason,

Thanks for the follow-up.

> It breaks because VFIO will only cache *actual* MSI-X table entries,
> it does not have visibility into anything that
> platform_device_msi_init_and_alloc_irqs()

You're right, the current hook is on SET_IRQS which is tied to standard
PCI MSI allocation. If the driver moved to device MSI domains, this specific
hook would stop working.

However, and I was referring to this, the variant driver is a
self-contained module, it doesn't prevent anyone from moving
ath11k/ath12k to device MSI domains. If that work happens, the
variant driver can be adapted to hook the new allocation path, or
replaced entirely.

> The only reason this works for you at all is because the driver
> hackily copies the vectors from the real MSI-X table so it can look in
> the "cache" to find their true physical versions.
>
> Which is my general objection, I would like to see the driver to use
> platform_device_msi_init_and_alloc_irqs() and don't want to get stuck
> unable to do that because it would break this.

I understand the concern, but I haven't seen any active work on
moving ath11k/ath12k to device MSI domains, and the VM problem would
remain exactly the same after that move (as you acknowledged). It would
mean waiting for a refactoring that has no active work or timeline yet,
while users remain without a working solution.

> Yes, but it does actually solve the problem in all its forms..

From my research, there is currently no existing work on device
MSI domain virtualization in KVM. Building it would require
coordinated changes across multiple subsystems (KVM, QEMU, VFIO,
IRQ core, guest drivers, arch code) and several maintainer trees,
and there is no KVM hypercall that bridges to VFIO, so the
architecture would need to be designed from scratch.

In the meantime, the variant driver provides an immediate solution
using an established kernel pattern, for users with deployed hardware.
It can coexist and be replaced when a generic mechanism arrives.

That said, I'm open to restructuring the current approach as a more
generic VFIO mechanism for exposing host-side device metadata to VM
guests, where MSI address caching would be one quirk type rather than
a Qualcomm-specific driver. This way the framework would be reusable
for any device with similar needs. Would that direction be more
acceptable as an intermediate step?

> Everyone else seems to know this stuff doesn't work for
> virtualization and doesn't try to do something like that.

The firmware is what it is, and the hardware is deployed. The kernel
regularly supports devices with non-ideal firmware designs through
quirks and device-specific drivers, and this is what I am trying to
do with the VFIO variant driver pattern (as it was previously done
for other drivers).

> Why do people care so much about this? I always thought it was a bit
> odd?

WiFi passthrough to VMs is useful for network function virtualization,
security isolation (running the wireless stack in a separate VM),
edge computing, and development/testing environments. These are real
use cases with real users who have this hardware deployed today.
Qualcomm wireless cards are widely deployed, perform well, and could be
actively used in these scenarios.

Thanks

Best regards
Jose Ignacio