Re: [PATCH] vfio/nvgrace-gpu: Skip PFN registration for empty memory regions

[Alex Williamson <alex@shazbot.org>]

On Fri, 25 Sep 2026 18:10:16 +0200
Andrea Parri <parri.andrea@gmail.com> wrote:

> nvgrace_gpu_open_device() registers nvdev->resmem only when it has a
> length, but registers nvdev->usemem unconditionally.  In fallback mode,
> where the ACPI memory properties are missing or the C2C link is down,
> usemem.memlength is zero, so the empty region is still registered.
> 
> register_pfn_address_space() then inserts a node covering
> [0, ULONG_MAX].  Nothing removes it, because the fallback device table
> closes through vfio_pci_core_close_device() rather than
> nvgrace_gpu_close_device().  A second open fails with -EBUSY, and once
> the device is unbound the node outlives the nvdev that contains it, so a
> later registration or a memory-failure event walks freed memory.
> 
> The node also captures every memory-failure report for a PFN with no
> struct page.  The callback rejects every VMA of the fallback device, so
> no process is killed, but the failure is logged as MF_RECOVERED rather
> than MF_IGNORED.
> 
> Guard the usemem registration with its length, matching the resmem
> handling.  Fallback mode then registers nothing and the existing close
> path is correct.  Guard the unregistration the same way.
> 
> On the unfixed tree a second open after unbind reports:
> 
>   BUG: KASAN: slab-use-after-free in interval_tree_iter_first+0x209/0x310
>   ...
>    interval_tree_iter_first+0x209/0x310
>    register_pfn_address_space+0x97/0x100
>    nvgrace_gpu_open_device+0x21c/0x730
>    vfio_df_open+0x213/0x520
>   ...
>   Allocated by task 152:
>    _vfio_alloc_device+0x3a/0x500
>    nvgrace_gpu_probe+0xcb/0x660
> 
> The fix should be safe because memlength already decided whether the
> range was meaningful: full mode always has a nonzero usemem.memlength,
> and fallback mode never calls nvgrace_gpu_close_device().  The guard
> only skips a registration that could never have been unregistered.
> 
> Fixes: e5f19b619fa0 ("vfio/nvgrace-gpu: register device memory for poison handling")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
> Tested on x86-64 under virtme-ng with KASAN, not on Grace hardware. The
> driver was bound through driver_override to a QEMU cxl-type3 function,
> whose CXL Device DVSEC reports the memory as valid and active, so probe
> takes the fallback path; the device was opened through a noiommu VFIO
> cdev. Without the patch the second open fails with -EBUSY and, after
> unbind and rebind, the next open hits the KASAN report quoted above.
> With the patch neither happens.
> 
>  drivers/vfio/pci/nvgrace-gpu/main.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
> index 963fd8ded20d1..326b991732148 100644
> --- a/drivers/vfio/pci/nvgrace-gpu/main.c
> +++ b/drivers/vfio/pci/nvgrace-gpu/main.c
> @@ -207,9 +207,12 @@ static int nvgrace_gpu_open_device(struct vfio_device *core_vdev)
>  			goto error_exit;
>  	}
>  
> -	ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev, &nvdev->usemem);
> -	if (ret && ret != -EOPNOTSUPP)
> -		goto register_mem_failed;
> +	if (nvdev->usemem.memlength) {
> +		ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev,
> +							      &nvdev->usemem);
> +		if (ret && ret != -EOPNOTSUPP)
> +			goto register_mem_failed;
> +	}
>  
>  	vfio_pci_core_finish_enable(vdev);
>  	nvdev->bar0_base = io;
> @@ -235,7 +238,8 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
>  	if (nvdev->resmem.memlength)
>  		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
>  
> -	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
> +	if (nvdev->usemem.memlength)
> +		unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
>  
>  	/* Unmap the mapping to the device memory cached region */
>  	if (nvdev->usemem.memaddr) {
> 
> base-commit: f49a343b305c0b6c19a3b50c0bbf10bcd0e2e8fd

Ankit, any comments?

It seems to me though that the reason this bug exists is because
nvgrace_gpu_pci_core_ops.open_device is trying to handle both real
nvgrace devices, where usemem.memlength cannot be zero, alongside
non-nvgrace devices where that can easily be forgotten.

The above patches the failure of that model, but maybe the better
approach is to split them such that the failures like this cannot
occur, ie. a core-only wrapper that just calls vfio_pci_core_enable()
and vfio_pci_core_finish_enable(), or some other refactor such that the
core-only path never handles the nvdev.  Thanks,

Alex