[PATCH] drm/nouveau/kms: fix overflow in block size calculation in nouveau_check_bl_size()

[PATCH] drm/nouveau/kms: fix overflow in block size calculation in nouveau_check_bl_size()

[Alexey Nepomnyashih]

Prevent potential overflow in nouveau_check_bl_size() when calculating
bl_size. Although bl_size is a 64-bit value, the intermediate
multiplication of 32-bit operands (bw, bh, tile_mode, and gob_size) may
overflow before being assigned. gob_size is 256 or 512, and tile_mode is
validated to be ≤ 31, but bw and bh can still be large enough to trigger
overflow. Cast bw to uint64_t to ensure proper 64-bit arithmetic.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Fixes: 4f5746c863db ("drm/nouveau/kms: Check framebuffer size against bo")
Signed-off-by: Alexey Nepomnyashih <sdl@nppct.ru>
---
 drivers/gpu/drm/nouveau/nouveau_display.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c
index add006fc8d81..0363711ee0ee 100644
--- a/drivers/gpu/drm/nouveau/nouveau_display.c
+++ b/drivers/gpu/drm/nouveau/nouveau_display.c
@@ -239,7 +239,7 @@ nouveau_check_bl_size(struct nouveau_drm *drm, struct nouveau_bo *nvbo,
 	bh = nouveau_get_height_in_blocks(h, gobs_in_block, drm->client.device.info.family);
 	gob_size = nouveau_get_gob_size(drm->client.device.info.family);
 
-	bl_size = bw * bh * gobs_in_block * gob_size;
+	bl_size = (uint64_t)bw * bh * gobs_in_block * gob_size;
 
 	DRM_DEBUG_KMS("offset=%u stride=%u h=%u gobs_in_block=%u bw=%u bh=%u gob_size=%u bl_size=%llu size=%zu\n",
 		      offset, stride, h, gobs_in_block, bw, bh, gob_size,
-- 
2.43.0