[PATCH] Remove variable-sized stack in drivers/char/fbmem.c

[PATCH] Remove variable-sized stack in drivers/char/fbmem.c

[Art Haas]

Hi.

The following patch replaces the variably sized stack with a kmalloc()
invocation. A brief search around the code suggests that memory
allocation as this function was doing is unusual.

Art Haas

===== drivers/video/fbmem.c 1.96 vs edited =====
--- 1.96/drivers/video/fbmem.c	Mon Apr 12 12:55:32 2004
+++ edited/drivers/video/fbmem.c	Tue Apr 13 16:53:38 2004
@@ -980,21 +980,29 @@
 int
 fb_blank(struct fb_info *info, int blank)
 {	
-	/* ??? Variable sized stack allocation.  */
-	u16 black[info->cmap.len];
+	int res;
+	size_t size;
+	u16 * black = NULL;
 	struct fb_cmap cmap;
 	
 	if (info->fbops->fb_blank && !info->fbops->fb_blank(blank, info))
 		return 0;
 	if (blank) { 
-		memset(black, 0, info->cmap.len * sizeof(u16));
+		size = info->cmap.len * sizeof(*black);
+		black = kmalloc(size, GFP_KERNEL);
+		if (!black)
+			return -ENOMEM;
+		memset(black, 0, size);
 		cmap.red = cmap.green = cmap.blue = black;
 		cmap.transp = info->cmap.transp ? black : NULL;
 		cmap.start = info->cmap.start;
 		cmap.len = info->cmap.len;
 	} else
 		cmap = info->cmap;
-	return fb_set_cmap(&cmap, 1, info);
+	res = fb_set_cmap(&cmap, 1, info);
+	if (black)
+		kfree(black);
+	return res;
 }
 
 static int 
-- 
Man once surrendering his reason, has no remaining guard against absurdities
the most monstrous, and like a ship without rudder, is the sport of every wind.

-Thomas Jefferson to James Smith, 1822