Re: Is PCI passthrough of the AMD XDNA device possible?

[Lizhi Hou <lizhi.hou@amd.com>]

On 7/2/25 02:14, Marcello Sylvester Bauer wrote:
> Greetings,
>
> I have some questions about making the AMD XDNA NPU accessible in a 
> virtualized environment. I tried using VFIO on QEMU, but I could not 
> get it to work.
>
> Here is a brief rundown of the resulting issues:
>
> Simply adding the device to QEMU ("-device 
> vfio-pci,host=0000:c4:00.1,addr=4") results in the following error:
> ```
> amdxdna 0000:c4:00.1: [drm] *ERROR* aie2_init: Enable PASID failed, 
> ret -19
> amdxdna 0000:c4:00.1: [drm] *ERROR* amdxdna_probe: Hardware init 
> failed, ret -19
> ```
> This makes sense, as the drive relies on the IOMMU for the PASID 
> extension.
>
> Fortunately, QEMU supports hardware-assisted IOMMU virtualization with 
> the AMD-vIOMMU device ("-device amd-iommu,intremap=on").
> While the driver can enable the PASID extension with this device, it 
> fails to establish communication with PSP during firmware validation:
> ```
> amdxdna 0000:00:04.1: [drm] *ERROR* fw return error 0x8
> amdxdna 0000:00:04.1: [drm] *ERROR* failed to validate fw, ret -5
> amdxdna 0000:00:04.1: [drm] *ERROR* aie2_hw_start: failed to start 
> psp, ret -5
> amdxdna 0000:00:04.1: [drm] *ERROR* aie2_smu_exec: smu cmd 4 timed out
> amdxdna 0000:00:04.1: [drm] *ERROR* aie2_smu_fini: Power off failed, 
> ret -110
> amdxdna 0000:00:04.1: [drm] *ERROR* aie2_init: start npu failed, ret -5
> amdxdna 0000:00:04.1: [drm] *ERROR* amdxdna_probe: Hardware init 
> failed, ret -5
> amdxdna 0000:00:04.1: probe with driver amdxdna failed with error -5
> ```
>
> Note that it also causes other devices on the host system to 
> malfunction, such as the GPU:
> ```
> amdgpu 0000:c3:00.0: amdgpu: SMU: I'm not done with your previous 
> command: SMN_C2PMSG_66:0x0000000F SMN_C2PMSG_82:0x00000007
> amdgpu 0000:c3:00.0: amdgpu: SMU: I'm not done with your previous 
> command: SMN_C2PMSG_66:0x0000000F SMN_C2PMSG_82:0x00000007
> amdgpu 0000:c3:00.0: amdgpu: SMU: I'm not done with your previous 
> command: SMN_C2PMSG_66:0x0000000F SMN_C2PMSG_82:0x00000007
> amdgpu 0000:c3:00.0: amdgpu: Failed to disable gfxoff!
> ```
>
> Now to the questions in this regard:
>
> Are there any known limitations or information regarding the 
> virtualization of AMD XNDA?
> Is there public documentation on how the device interacts with the PSP 
> BAR, and if not, what does firmware error 8 mean?

Unfortunately, XDNA hardware/firmware does not support passthrough. The 
validate fw error with passthrough setup is known. If you would use XDNA 
in VM, you may try virtio solution: https://github.com/amd/virtio-npu


Thanks,

Lizhi

>
> Thanks for the support.
>
> Kind regards,
> Marcello
> -- 
> hardware: Lenovo ThinkPad T14s Gen 6 (AMD Ryzen AI 7 PRO 360)
> QEMU: v10.0.0
> Linux host: v6.15.4
> Linux guest: 6.14.0-22-generic (Ubuntu 25.04)
>
>