[PATCH v2 1/2] vfio/nvgrace-gpu: Use a core-only open_device in fallback mode

[Andrea Parri <parri.andrea@gmail.com>]

When the ACPI memory properties are missing or the C2C link is down, the
device is exposed through nvgrace_gpu_pci_core_ops.  That table opens
through nvgrace_gpu_open_device() but closes through
vfio_pci_core_close_device(), so open and close are not symmetric.

nvgrace_gpu_open_device() registers nvdev->usemem for poison handling
unconditionally.  In fallback mode usemem.memlength is zero, and
register_pfn_address_space() inserts a node covering [0, ULONG_MAX].
Nothing removes it: a second open fails with -EBUSY, and once the device
is unbound the node outlives the nvdev that contains it.  The next open
after a rebind reports:

  BUG: KASAN: slab-use-after-free in interval_tree_iter_first+0x209/0x310
  Read of size 8 at addr ffff88800894aa30 by task nvgrace_repro/203
  Call Trace:
   <TASK>
   dump_stack_lvl+0x5d/0x80
   print_report+0x153/0x4c6
   kasan_report+0xe4/0x1a0
   interval_tree_iter_first+0x209/0x310
   register_pfn_address_space+0x97/0x100
   nvgrace_gpu_open_device+0x21c/0x730
   vfio_df_open+0x213/0x520
   vfio_df_ioctl_bind_iommufd+0x3e8/0x6e0
   vfio_device_fops_unl_ioctl+0x782/0x1570
   __x64_sys_ioctl+0x13c/0x1c0
   do_syscall_64+0x106/0x630
   entry_SYSCALL_64_after_hwframe+0x77/0x7f
   </TASK>

  Allocated by task 188:
   ...
   _vfio_alloc_device+0x3a/0x500
   nvgrace_gpu_probe+0xcb/0x660
   local_pci_probe+0xcf/0x170
   ...

  Freed by task 188:
   ...
   kfree+0x20d/0x600
   device_release+0xcf/0x250
   kobject_put+0x152/0x2f0
   pci_device_remove+0xb0/0x1c0
   device_release_driver_internal+0x38d/0x540
   unbind_store+0xde/0x100
   ...

Give the fallback table its own open_device that only calls
vfio_pci_core_enable() and vfio_pci_core_finish_enable(), the
counterpart of vfio_pci_core_close_device().  Fallback mode then never
touches the nvgrace memory regions, as Alex suggested.

This should be safe because the fallback table uses the vfio-pci core
read, write and mmap handlers, so nothing there uses the state
nvgrace_gpu_open_device() sets up.  The one visible difference is that a
fallback open no longer fails when BAR0 cannot be mapped, which matches
vfio-pci.

Fixes: e5f19b619fa0 ("vfio/nvgrace-gpu: register device memory for poison handling")
Cc: stable@vger.kernel.org
Suggested-by: Alex Williamson <alex@shazbot.org>
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
v2: Replace the usemem.memlength guards with a core-only open_device
    for the fallback table (Alex)
    Ankit confirmed fallback mode does not need the BAR0 check:
    https://lore.kernel.org/r/SAWPR12MB99926787B53C4FF469A6F3BE7AB09D2@SAWPR12MB999267.namprd12.prod.outlook.com/
v1: https://lore.kernel.org/r/20260925161024.189771-1-parri.andrea@gmail.com/

 drivers/vfio/pci/nvgrace-gpu/main.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
index 963fd8ded20d1..208463c218088 100644
--- a/drivers/vfio/pci/nvgrace-gpu/main.c
+++ b/drivers/vfio/pci/nvgrace-gpu/main.c
@@ -1083,6 +1083,21 @@ static const struct vfio_device_ops nvgrace_gpu_pci_ops = {
 	.detach_ioas	= vfio_iommufd_physical_detach_ioas,
 };
 
+static int nvgrace_gpu_core_open_device(struct vfio_device *core_vdev)
+{
+	struct vfio_pci_core_device *vdev =
+		container_of(core_vdev, struct vfio_pci_core_device, vdev);
+	int ret;
+
+	ret = vfio_pci_core_enable(vdev);
+	if (ret)
+		return ret;
+
+	vfio_pci_core_finish_enable(vdev);
+
+	return 0;
+}
+
 static const struct vfio_pci_device_ops nvgrace_gpu_pci_dev_core_ops = {
 	.get_dmabuf_phys = vfio_pci_core_get_dmabuf_phys,
 };
@@ -1091,7 +1106,7 @@ static const struct vfio_device_ops nvgrace_gpu_pci_core_ops = {
 	.name		= "nvgrace-gpu-vfio-pci-core",
 	.init		= vfio_pci_core_init_dev,
 	.release	= vfio_pci_core_release_dev,
-	.open_device	= nvgrace_gpu_open_device,
+	.open_device	= nvgrace_gpu_core_open_device,
 	.close_device	= vfio_pci_core_close_device,
 	.ioctl		= vfio_pci_core_ioctl,
 	.get_region_info_caps = vfio_pci_ioctl_get_region_info,
-- 
2.53.0