[PATCH] video: imxfb: always allocate 256 entries for the color map

[PATCH] video: imxfb: always allocate 256 entries for the color map

[Martin Kaiser]

The current code calculates the number of color map entries as
1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when
written to an int variable. As a consequence, the subsequent copying
of the default (non-empty) color map into our newly allocated color map
fails and imxfb's probe function returns an error.

On both imx1 and imx21 platforms, the color map is used only for modes
with <= 8bpp. By allocating 256 entries for the color map, we're on the
safe side.

Signed-off-by: Martin Kaiser <martin@kaiser.cx>
---
Re-sending: corrected a typo in the LKML address, sorry for that.

 drivers/video/fbdev/imxfb.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c
index fe0c4ee..1b0faad 100644
--- a/drivers/video/fbdev/imxfb.c
+++ b/drivers/video/fbdev/imxfb.c
@@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev)
 	 */
 	imxfb_check_var(&info->var, info);
 
-	ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0);
+	/*
+	 * For modes > 8bpp, the color map is bypassed.
+	 * Therefore, 256 entries are enough.
+	 */
+	ret = fb_alloc_cmap(&info->cmap, 256, 0);
 	if (ret < 0)
 		goto failed_cmap;
 
-- 
1.7.10.4

Re: [PATCH] video: imxfb: always allocate 256 entries for the color map

[Sascha Hauer]

On Mon, Jan 02, 2017 at 11:24:02PM +0100, Martin Kaiser wrote:
> The current code calculates the number of color map entries as
> 1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when
> written to an int variable. As a consequence, the subsequent copying
> of the default (non-empty) color map into our newly allocated color map
> fails and imxfb's probe function returns an error.
> 
> On both imx1 and imx21 platforms, the color map is used only for modes
> with <= 8bpp. By allocating 256 entries for the color map, we're on the
> safe side.
> 
> Signed-off-by: Martin Kaiser <martin@kaiser.cx>

Acked-by: Sascha Hauer <s.hauer@pengutronix.de>

Sascha

> ---
> Re-sending: corrected a typo in the LKML address, sorry for that.
> 
>  drivers/video/fbdev/imxfb.c |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c
> index fe0c4ee..1b0faad 100644
> --- a/drivers/video/fbdev/imxfb.c
> +++ b/drivers/video/fbdev/imxfb.c
> @@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev)
>  	 */
>  	imxfb_check_var(&info->var, info);
>  
> -	ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0);
> +	/*
> +	 * For modes > 8bpp, the color map is bypassed.
> +	 * Therefore, 256 entries are enough.
> +	 */
> +	ret = fb_alloc_cmap(&info->cmap, 256, 0);
>  	if (ret < 0)
>  		goto failed_cmap;
>  
> -- 
> 1.7.10.4
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH] video: imxfb: always allocate 256 entries for the color map

[Bartlomiej Zolnierkiewicz]

Hi,

On Monday, January 09, 2017 01:59:42 PM Sascha Hauer wrote:
> On Mon, Jan 02, 2017 at 11:24:02PM +0100, Martin Kaiser wrote:
> > The current code calculates the number of color map entries as
> > 1 << info->var.bits_per_pixel. For 32bpp modes, 1 << 32 is 0 when
> > written to an int variable. As a consequence, the subsequent copying
> > of the default (non-empty) color map into our newly allocated color map
> > fails and imxfb's probe function returns an error.
> > 
> > On both imx1 and imx21 platforms, the color map is used only for modes
> > with <= 8bpp. By allocating 256 entries for the color map, we're on the
> > safe side.
> > 
> > Signed-off-by: Martin Kaiser <martin@kaiser.cx>
> 
> Acked-by: Sascha Hauer <s.hauer@pengutronix.de>

Thanks, patch queued for 4.11.

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung R&D Institute Poland
Samsung Electronics

> Sascha
> 
> > ---
> > Re-sending: corrected a typo in the LKML address, sorry for that.
> > 
> >  drivers/video/fbdev/imxfb.c |    6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/video/fbdev/imxfb.c b/drivers/video/fbdev/imxfb.c
> > index fe0c4ee..1b0faad 100644
> > --- a/drivers/video/fbdev/imxfb.c
> > +++ b/drivers/video/fbdev/imxfb.c
> > @@ -985,7 +985,11 @@ static int imxfb_probe(struct platform_device *pdev)
> >  	 */
> >  	imxfb_check_var(&info->var, info);
> >  
> > -	ret = fb_alloc_cmap(&info->cmap, 1 << info->var.bits_per_pixel, 0);
> > +	/*
> > +	 * For modes > 8bpp, the color map is bypassed.
> > +	 * Therefore, 256 entries are enough.
> > +	 */
> > +	ret = fb_alloc_cmap(&info->cmap, 256, 0);
> >  	if (ret < 0)
> >  		goto failed_cmap;