Linux Kernel build bug with AMD_IOMMU_V2=M and HSA_AMD=Y

Linux Kernel build bug with AMD_IOMMU_V2=M and HSA_AMD=Y

[David Niklas]

[-- Attachment #1: Type: text/plain, Size: 2799 bytes --]

Hello,
(There are so many maintainers for kfd_iommu.c I feel like I'm spamming.)

When compiling for Linux version 5.11.12 using the AMDGPU GPU driver
with HSA_AMD enabled, I get the below set of errors. To work around this,
I need to change AMD_IOMMU_V2 from M to Y. This bug doesn't affect linux
kernel version 5.6 as it requires AMD_IOMMU_V2 to by Y when HSA_AMD is
enabled.
I'd bisect and request the removal of the relevant patch, but it's
possible that building the linux kernel should work this way and so a fix,
not a patch removal, is what should be issued.
I'm attaching my kernel config for 5.11.

Thanks,
David

PS: I made an official bug report in case you'd prefer that:
https://bugzilla.kernel.org/show_bug.cgi?id=212619

drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
`kfd_iommu_bind_process_to_device': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:120:
undefined reference to `amd_iommu_bind_pasid'
drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
`kfd_iommu_unbind_process': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:138:
undefined reference to `amd_iommu_unbind_pasid'
drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
`kfd_iommu_suspend': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:292:
undefined reference to
`amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:293:
undefined reference to `amd_iommu_set_invalid_ppr_cb'
drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
`kfd_iommu_resume': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:312:
undefined reference to
`amd_iommu_init_device' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:316:
undefined reference to
`amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:318:
undefined reference to
`amd_iommu_set_invalid_ppr_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:323:
undefined reference to
`amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:324:
undefined reference to
`amd_iommu_set_invalid_ppr_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:325:
undefined reference to
`amd_iommu_free_device' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:232:
undefined reference to `amd_iommu_bind_pasid'
drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
`kfd_iommu_suspend': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:294:
undefined reference to `amd_iommu_free_device' Makefile:1166: recipe for
target 'vmlinux' failed make: *** [vmlinux] Error 1

[-- Attachment #2: kernel-build-amdgpu-bug.conf.xz --]
[-- Type: application/x-xz, Size: 50448 bytes --]

Re: Linux Kernel build bug with AMD_IOMMU_V2=M and HSA_AMD=Y

[Felix Kuehling]

This should have been fixed by this commit in amd-staging-drm-next: 
https://lore.kernel.org/patchwork/patch/1392368/

commit b8aff1f3a0b3d8434f8ccf5d3017137c29aca77b
Author: Felix Kuehling <Felix.Kuehling@amd.com>
Date:   Mon Mar 8 22:15:42 2021 -0500

     drm/amdkfd: fix build error with AMD_IOMMU_V2=m
     
     Using 'imply AMD_IOMMU_V2' does not guarantee that the driver can link
     against the exported functions. If the GPU driver is built-in but the
     IOMMU driver is a loadable module, the kfd_iommu.c file is indeed
     built but does not work:
     
     x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_bind_process_to_device':
     kfd_iommu.c:(.text+0x516): undefined reference to `amd_iommu_bind_pasid'
     x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_unbind_process':
     kfd_iommu.c:(.text+0x691): undefined reference to `amd_iommu_unbind_pasid'
     x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_suspend':
     kfd_iommu.c:(.text+0x966): undefined reference to `amd_iommu_set_invalidate_ctx_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0x97f): undefined reference to `amd_iommu_set_invalid_ppr_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0x9a4): undefined reference to `amd_iommu_free_device'
     x86_64-linux-ld: drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: in function `kfd_iommu_resume':
     kfd_iommu.c:(.text+0xa9a): undefined reference to `amd_iommu_init_device'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xadc): undefined reference to `amd_iommu_set_invalidate_ctx_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xaff): undefined reference to `amd_iommu_set_invalid_ppr_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xc72): undefined reference to `amd_iommu_bind_pasid'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xe08): undefined reference to `amd_iommu_set_invalidate_ctx_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xe26): undefined reference to `amd_iommu_set_invalid_ppr_cb'
     x86_64-linux-ld: kfd_iommu.c:(.text+0xe42): undefined reference to `amd_iommu_free_device'
     
     Use IS_REACHABLE to only build IOMMU-V2 support if the amd_iommu symbols
     are reachable by the amdkfd driver. Output a warning if they are not,
     because that may not be what the user was expecting.
     
     Fixes: 64d1c3a43a6f ("drm/amdkfd: Centralize IOMMUv2 code and make it conditional")
     Reported-by: Arnd Bergmann <arnd@arndb.de>
     Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
     Reviewed-by: Christian König <christian.koenig@amd.com>

Regards,
   Felix

On 2021-04-08 7:04 p.m., David Niklas wrote:
> Hello,
> (There are so many maintainers for kfd_iommu.c I feel like I'm spamming.)
>
> When compiling for Linux version 5.11.12 using the AMDGPU GPU driver
> with HSA_AMD enabled, I get the below set of errors. To work around this,
> I need to change AMD_IOMMU_V2 from M to Y. This bug doesn't affect linux
> kernel version 5.6 as it requires AMD_IOMMU_V2 to by Y when HSA_AMD is
> enabled.
> I'd bisect and request the removal of the relevant patch, but it's
> possible that building the linux kernel should work this way and so a fix,
> not a patch removal, is what should be issued.
> I'm attaching my kernel config for 5.11.
>
> Thanks,
> David
>
> PS: I made an official bug report in case you'd prefer that:
> https://bugzilla.kernel.org/show_bug.cgi?id=212619
>
> drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
> `kfd_iommu_bind_process_to_device': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:120:
> undefined reference to `amd_iommu_bind_pasid'
> drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
> `kfd_iommu_unbind_process': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:138:
> undefined reference to `amd_iommu_unbind_pasid'
> drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
> `kfd_iommu_suspend': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:292:
> undefined reference to
> `amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:293:
> undefined reference to `amd_iommu_set_invalid_ppr_cb'
> drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
> `kfd_iommu_resume': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:312:
> undefined reference to
> `amd_iommu_init_device' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:316:
> undefined reference to
> `amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:318:
> undefined reference to
> `amd_iommu_set_invalid_ppr_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:323:
> undefined reference to
> `amd_iommu_set_invalidate_ctx_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:324:
> undefined reference to
> `amd_iommu_set_invalid_ppr_cb' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:325:
> undefined reference to
> `amd_iommu_free_device' /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:232:
> undefined reference to `amd_iommu_bind_pasid'
> drivers/gpu/drm/amd/amdkfd/kfd_iommu.o: In function
> `kfd_iommu_suspend': /root/working/linux-5.11.12/drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_iommu.c:294:
> undefined reference to `amd_iommu_free_device' Makefile:1166: recipe for
> target 'vmlinux' failed make: *** [vmlinux] Error 1