amdgpu driver rebinding broken by "drm/amd: Clean up kfd node on surprise disconnect"

amdgpu driver rebinding broken by "drm/amd: Clean up kfd node on surprise disconnect"

[Cal Peake]

Hi,

The recent commit 28695ca09d32: "drm/amd: Clean up kfd node on surprise 
disconnect," has broken something with my VMs utilizing PCI passthrough.
Before launching the VM, I unbind a secondary Radeon GPU from the amdgpu 
driver and then to the vfio-pci driver.

Previously, everything would just work and I'd get the following kernel 
output:

  amdgpu 0000:14:00.0: amdgpu: amdgpu: finishing device.
  [drm] amdgpu: ttm finalized
  vfio-pci 0000:14:00.0: vgaarb: VGA decodes changed: olddecodes=none,decodes=io+mem:owns=none
  vfio-pci 0000:14:00.1: enabling device (0000 -> 0002)
  vfio-pci 0000:14:00.3: enabling device (0000 -> 0002)

However, now when doing the rebinding: the host display (a different 
Radeon GPU) will freeze up and after a short time, the system will reset 
(thanks to a watchdog I believe) and I get this in the kernel log:

  amdgpu 0000:14:00.0: amdgpu: amdgpu: finishing device.
  vfio-pci 0000:14:00.1: Unable to change power state from D3hot to D0, device inaccessible
  vfio-pci 0000:14:00.3: Unable to change power state from D3hot to D0, device inaccessible
  vfio-pci 0000:14:00.2: Unable to change power state from D3hot to D0, device inaccessible


Indeed, backing out the commit from kernels 6.12.67, 6.12.68, and 6.18.8 
gets things back to working.

Please let me know if you have any ideas or if there is anymore debugging 
info I can provide.

Thanks,

-- 
Cal Peake

Re: amdgpu driver rebinding broken by "drm/amd: Clean up kfd node on surprise disconnect"

[Mario Limonciello]

On 2/3/26 4:27 PM, Cal Peake wrote:
> Hi,
> 
> The recent commit 28695ca09d32: "drm/amd: Clean up kfd node on surprise
> disconnect," has broken something with my VMs utilizing PCI passthrough.
> Before launching the VM, I unbind a secondary Radeon GPU from the amdgpu
> driver and then to the vfio-pci driver.
> 
> Previously, everything would just work and I'd get the following kernel
> output:
> 
>    amdgpu 0000:14:00.0: amdgpu: amdgpu: finishing device.
>    [drm] amdgpu: ttm finalized
>    vfio-pci 0000:14:00.0: vgaarb: VGA decodes changed: olddecodes=none,decodes=io+mem:owns=none
>    vfio-pci 0000:14:00.1: enabling device (0000 -> 0002)
>    vfio-pci 0000:14:00.3: enabling device (0000 -> 0002)
> 
> However, now when doing the rebinding: the host display (a different
> Radeon GPU) will freeze up and after a short time, the system will reset
> (thanks to a watchdog I believe) and I get this in the kernel log:
> 
>    amdgpu 0000:14:00.0: amdgpu: amdgpu: finishing device.
>    vfio-pci 0000:14:00.1: Unable to change power state from D3hot to D0, device inaccessible
>    vfio-pci 0000:14:00.3: Unable to change power state from D3hot to D0, device inaccessible
>    vfio-pci 0000:14:00.2: Unable to change power state from D3hot to D0, device inaccessible
> 
> 
> Indeed, backing out the commit from kernels 6.12.67, 6.12.68, and 6.18.8
> gets things back to working.
> 
> Please let me know if you have any ideas or if there is anymore debugging
> info I can provide.
> 
> Thanks,
> 

Here's a solution:
https://lore.kernel.org/amd-gfx/20260205164254.4091912-1-mario.limonciello@amd.com/

Re: amdgpu driver rebinding broken by "drm/amd: Clean up kfd node on surprise disconnect"

[Cal Peake]

On Thu, 5 Feb 2026, Mario Limonciello wrote:

> On 2/3/26 4:27 PM, Cal Peake wrote:
> > Hi,
> > 
> > The recent commit 28695ca09d32: "drm/amd: Clean up kfd node on surprise
> > disconnect," has broken something with my VMs utilizing PCI passthrough.
> > Before launching the VM, I unbind a secondary Radeon GPU from the amdgpu
> > driver and then to the vfio-pci driver.
> > 
> > [...]
> > 
> Here's a solution:
> https://lore.kernel.org/amd-gfx/20260205164254.4091912-1-mario.limonciello@amd.com/
> 

Thank you, Mario, that does the trick.

-- 
Cal Peake