[PATCH] vfio/nvgrace-gpu: Skip PFN registration for empty memory regions

[PATCH] vfio/nvgrace-gpu: Skip PFN registration for empty memory regions

[Andrea Parri]

nvgrace_gpu_open_device() registers nvdev->resmem only when it has a
length, but registers nvdev->usemem unconditionally.  In fallback mode,
where the ACPI memory properties are missing or the C2C link is down,
usemem.memlength is zero, so the empty region is still registered.

register_pfn_address_space() then inserts a node covering
[0, ULONG_MAX].  Nothing removes it, because the fallback device table
closes through vfio_pci_core_close_device() rather than
nvgrace_gpu_close_device().  A second open fails with -EBUSY, and once
the device is unbound the node outlives the nvdev that contains it, so a
later registration or a memory-failure event walks freed memory.

The node also captures every memory-failure report for a PFN with no
struct page.  The callback rejects every VMA of the fallback device, so
no process is killed, but the failure is logged as MF_RECOVERED rather
than MF_IGNORED.

Guard the usemem registration with its length, matching the resmem
handling.  Fallback mode then registers nothing and the existing close
path is correct.  Guard the unregistration the same way.

On the unfixed tree a second open after unbind reports:

  BUG: KASAN: slab-use-after-free in interval_tree_iter_first+0x209/0x310
  ...
   interval_tree_iter_first+0x209/0x310
   register_pfn_address_space+0x97/0x100
   nvgrace_gpu_open_device+0x21c/0x730
   vfio_df_open+0x213/0x520
  ...
  Allocated by task 152:
   _vfio_alloc_device+0x3a/0x500
   nvgrace_gpu_probe+0xcb/0x660

The fix should be safe because memlength already decided whether the
range was meaningful: full mode always has a nonzero usemem.memlength,
and fallback mode never calls nvgrace_gpu_close_device().  The guard
only skips a registration that could never have been unregistered.

Fixes: e5f19b619fa0 ("vfio/nvgrace-gpu: register device memory for poison handling")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
Tested on x86-64 under virtme-ng with KASAN, not on Grace hardware. The
driver was bound through driver_override to a QEMU cxl-type3 function,
whose CXL Device DVSEC reports the memory as valid and active, so probe
takes the fallback path; the device was opened through a noiommu VFIO
cdev. Without the patch the second open fails with -EBUSY and, after
unbind and rebind, the next open hits the KASAN report quoted above.
With the patch neither happens.

 drivers/vfio/pci/nvgrace-gpu/main.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
index 963fd8ded20d1..326b991732148 100644
--- a/drivers/vfio/pci/nvgrace-gpu/main.c
+++ b/drivers/vfio/pci/nvgrace-gpu/main.c
@@ -207,9 +207,12 @@ static int nvgrace_gpu_open_device(struct vfio_device *core_vdev)
 			goto error_exit;
 	}
 
-	ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev, &nvdev->usemem);
-	if (ret && ret != -EOPNOTSUPP)
-		goto register_mem_failed;
+	if (nvdev->usemem.memlength) {
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev,
+							      &nvdev->usemem);
+		if (ret && ret != -EOPNOTSUPP)
+			goto register_mem_failed;
+	}
 
 	vfio_pci_core_finish_enable(vdev);
 	nvdev->bar0_base = io;
@@ -235,7 +238,8 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
 	if (nvdev->resmem.memlength)
 		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
 
-	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
+	if (nvdev->usemem.memlength)
+		unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
 
 	/* Unmap the mapping to the device memory cached region */
 	if (nvdev->usemem.memaddr) {

base-commit: f49a343b305c0b6c19a3b50c0bbf10bcd0e2e8fd
-- 
2.53.0

Re: [PATCH] vfio/nvgrace-gpu: Skip PFN registration for empty memory regions

[Alex Williamson]

On Fri, 25 Sep 2026 18:10:16 +0200
Andrea Parri <parri.andrea@gmail.com> wrote:

> nvgrace_gpu_open_device() registers nvdev->resmem only when it has a
> length, but registers nvdev->usemem unconditionally.  In fallback mode,
> where the ACPI memory properties are missing or the C2C link is down,
> usemem.memlength is zero, so the empty region is still registered.
> 
> register_pfn_address_space() then inserts a node covering
> [0, ULONG_MAX].  Nothing removes it, because the fallback device table
> closes through vfio_pci_core_close_device() rather than
> nvgrace_gpu_close_device().  A second open fails with -EBUSY, and once
> the device is unbound the node outlives the nvdev that contains it, so a
> later registration or a memory-failure event walks freed memory.
> 
> The node also captures every memory-failure report for a PFN with no
> struct page.  The callback rejects every VMA of the fallback device, so
> no process is killed, but the failure is logged as MF_RECOVERED rather
> than MF_IGNORED.
> 
> Guard the usemem registration with its length, matching the resmem
> handling.  Fallback mode then registers nothing and the existing close
> path is correct.  Guard the unregistration the same way.
> 
> On the unfixed tree a second open after unbind reports:
> 
>   BUG: KASAN: slab-use-after-free in interval_tree_iter_first+0x209/0x310
>   ...
>    interval_tree_iter_first+0x209/0x310
>    register_pfn_address_space+0x97/0x100
>    nvgrace_gpu_open_device+0x21c/0x730
>    vfio_df_open+0x213/0x520
>   ...
>   Allocated by task 152:
>    _vfio_alloc_device+0x3a/0x500
>    nvgrace_gpu_probe+0xcb/0x660
> 
> The fix should be safe because memlength already decided whether the
> range was meaningful: full mode always has a nonzero usemem.memlength,
> and fallback mode never calls nvgrace_gpu_close_device().  The guard
> only skips a registration that could never have been unregistered.
> 
> Fixes: e5f19b619fa0 ("vfio/nvgrace-gpu: register device memory for poison handling")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
> Tested on x86-64 under virtme-ng with KASAN, not on Grace hardware. The
> driver was bound through driver_override to a QEMU cxl-type3 function,
> whose CXL Device DVSEC reports the memory as valid and active, so probe
> takes the fallback path; the device was opened through a noiommu VFIO
> cdev. Without the patch the second open fails with -EBUSY and, after
> unbind and rebind, the next open hits the KASAN report quoted above.
> With the patch neither happens.
> 
>  drivers/vfio/pci/nvgrace-gpu/main.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
> index 963fd8ded20d1..326b991732148 100644
> --- a/drivers/vfio/pci/nvgrace-gpu/main.c
> +++ b/drivers/vfio/pci/nvgrace-gpu/main.c
> @@ -207,9 +207,12 @@ static int nvgrace_gpu_open_device(struct vfio_device *core_vdev)
>  			goto error_exit;
>  	}
>  
> -	ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev, &nvdev->usemem);
> -	if (ret && ret != -EOPNOTSUPP)
> -		goto register_mem_failed;
> +	if (nvdev->usemem.memlength) {
> +		ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev,
> +							      &nvdev->usemem);
> +		if (ret && ret != -EOPNOTSUPP)
> +			goto register_mem_failed;
> +	}
>  
>  	vfio_pci_core_finish_enable(vdev);
>  	nvdev->bar0_base = io;
> @@ -235,7 +238,8 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
>  	if (nvdev->resmem.memlength)
>  		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
>  
> -	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
> +	if (nvdev->usemem.memlength)
> +		unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
>  
>  	/* Unmap the mapping to the device memory cached region */
>  	if (nvdev->usemem.memaddr) {
> 
> base-commit: f49a343b305c0b6c19a3b50c0bbf10bcd0e2e8fd

Ankit, any comments?

It seems to me though that the reason this bug exists is because
nvgrace_gpu_pci_core_ops.open_device is trying to handle both real
nvgrace devices, where usemem.memlength cannot be zero, alongside
non-nvgrace devices where that can easily be forgotten.

The above patches the failure of that model, but maybe the better
approach is to split them such that the failures like this cannot
occur, ie. a core-only wrapper that just calls vfio_pci_core_enable()
and vfio_pci_core_finish_enable(), or some other refactor such that the
core-only path never handles the nvdev.  Thanks,

Alex