Re: [PATCH] x86: simplefb: avoid overflow

["H. Peter Anvin" <hpa@zytor.com>]

On 09/06/2013 03:59 AM, Geert Uytterhoeven wrote:
> On Fri, Sep 6, 2013 at 11:55 AM, David Herrmann <dh.herrmann@gmail.com> wrote:
>> On Fri, Sep 6, 2013 at 11:32 AM, Tom Gundersen <teg@jklm.no> wrote:
>>> lfb_size can easily be say 4M, which would make the bitshit overflow and
>>> the test fail.
>>>
>>> Signed-off-by: Tom Gundersen <teg@jklm.no>
>>> Cc: David Herrmann <dh.herrmann@gmail.com>
>>> Cc: H. Peter Anvin <hpa@zytor.com>
>>> ---
>>>   arch/x86/kernel/sysfb_simplefb.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/arch/x86/kernel/sysfb_simplefb.c b/arch/x86/kernel/sysfb_simplefb.c
>>> index 22513e9..fff44a5 100644
>>> --- a/arch/x86/kernel/sysfb_simplefb.c
>>> +++ b/arch/x86/kernel/sysfb_simplefb.c
>>> @@ -72,7 +72,7 @@ __init int create_simplefb(const struct screen_info *si,
>>>           * the part that is occupied by the framebuffer */
>>>          len = mode->height * mode->stride;
>>>          len = PAGE_ALIGN(len);
>>> -       if (len > si->lfb_size << 16) {
>>> +       if (len > ((unsigned long) si->lfb_size) << 16) {
>
> On 32-bit, "unsigned long" is the same size as __u32, so this doesn't
> make any difference.
>
>> Nice catch. vesafb uses "lfb_size * 65535" which causes an implicit
>> cast. I thought <<16 looks nicer but that doesn't do any implicit
>> cast..
>
> "lfb_size * 65535" is the same. "lfb_size" is __u32, "65535" is int.
> So there's no implicit cast. Or am I missing something?
>

<< 16 is * 65536 not 65535...

	-hpa