[PATCH] drm/vgem: fix IDR slot and xa_node memory leak in vgem_fence_signal_ioctl()

[PATCH] drm/vgem: fix IDR slot and xa_node memory leak in vgem_fence_signal_ioctl()

[Hui Peng]

In `vgem_fence_signal_ioctl()`, `idr_replace(&vfile->fence_idr, NULL,
arg->fence)` is called instead of `idr_remove(&vfile->fence_idr,
arg->fence)`.

Because `idr_replace()` replaces the slot's pointer with `NULL` in the
underlying XArray without freeing the IDR slot (`xa_erase()`), the ID
remains allocated until the vgem file descriptor is closed. Repeated
`DRM_IOCTL_VGEM_FENCE_ATTACH` + `DRM_IOCTL_VGEM_FENCE_SIGNAL` cycles on
a long-lived vgem fd permanently accumulate `xa_node` slab allocations
in the kernel and exhaust the per-fd IDR space.

Replace `idr_replace(&vfile->fence_idr, NULL, arg->fence)` with
`idr_remove(&vfile->fence_idr, arg->fence)`.

Fixes: 407779848445 ("drm/vgem: Attach sw fences to exported vGEM dma-buf (ioctl)")
Assisted-by: LLM
Signed-off-by: Hui Peng <benquike@gmail.com>

---
 drivers/gpu/drm/vgem/vgem_fence.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/vgem/vgem_fence.c b/drivers/gpu/drm/vgem/vgem_fence.c
index 0e53ba7db629..5793be469b3c 100644
--- a/drivers/gpu/drm/vgem/vgem_fence.c
+++ b/drivers/gpu/drm/vgem/vgem_fence.c
@@ -200,12 +200,10 @@ int vgem_fence_signal_ioctl(struct drm_device *dev,
 		return -EINVAL;
 
 	mutex_lock(&vfile->fence_mutex);
-	fence = idr_replace(&vfile->fence_idr, NULL, arg->fence);
+	fence = idr_remove(&vfile->fence_idr, arg->fence);
 	mutex_unlock(&vfile->fence_mutex);
 	if (!fence)
 		return -ENOENT;
-	if (IS_ERR(fence))
-		return PTR_ERR(fence);
 
 	if (dma_fence_is_signaled(fence))
 		ret = -ETIMEDOUT;
-- 
2.55.0.1082.g2b9226bbc0-goog