[tip:x86/mm] x86: Fix mmap random address range

[tip:x86/mm] x86: Fix mmap random address range

[tip-bot for Ludwig Nussel]

Commit-ID:  9af0c7a6fa860698d080481f24a342ba74b68982
Gitweb:     http://git.kernel.org/tip/9af0c7a6fa860698d080481f24a342ba74b68982
Author:     Ludwig Nussel <ludwig.nussel@suse.de>
AuthorDate: Tue, 15 Nov 2011 14:46:46 -0800
Committer:  Ingo Molnar <mingo@elte.hu>
CommitDate: Mon, 5 Dec 2011 17:07:23 +0100

x86: Fix mmap random address range

On x86_32 casting the unsigned int result of get_random_int() to
long may result in a negative value.  On x86_32 the range of
mmap_rnd() therefore was -255 to 255.  The 32bit mode on x86_64
used 0 to 255 as intended.

The bug was introduced by 675a081 ("x86: unify mmap_{32|64}.c")
in January 2008.

Signed-off-by: Ludwig Nussel <ludwig.nussel@suse.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: harvey.harrison@gmail.com
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Harvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Link: http://lkml.kernel.org/r/201111152246.pAFMklOB028527@wpaz5.hot.corp.google.com
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 arch/x86/mm/mmap.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index 4b5ba85..845df68 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -75,9 +75,9 @@ static unsigned long mmap_rnd(void)
 	*/
 	if (current->flags & PF_RANDOMIZE) {
 		if (mmap_is_ia32())
-			rnd = (long)get_random_int() % (1<<8);
+			rnd = get_random_int() % (1<<8);
 		else
-			rnd = (long)(get_random_int() % (1<<28));
+			rnd = get_random_int() % (1<<28);
 	}
 	return rnd << PAGE_SHIFT;
 }

Re: [tip:x86/mm] x86: Fix mmap random address range

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 809 bytes --]

On Mon, 05 Dec 2011 09:52:17 PST, tip-bot for Ludwig Nussel said:
> Commit-ID:  9af0c7a6fa860698d080481f24a342ba74b68982
> Gitweb:     http://git.kernel.org/tip/9af0c7a6fa860698d080481f24a342ba74b68982
> Author:     Ludwig Nussel <ludwig.nussel@suse.de>
> AuthorDate: Tue, 15 Nov 2011 14:46:46 -0800
> Committer:  Ingo Molnar <mingo@elte.hu>
> CommitDate: Mon, 5 Dec 2011 17:07:23 +0100
>
> x86: Fix mmap random address range
>
> On x86_32 casting the unsigned int result of get_random_int() to
> long may result in a negative value.  On x86_32 the range of
> mmap_rnd() therefore was -255 to 255.  The 32bit mode on x86_64
> used 0 to 255 as intended.

Will somebody please hand me a big dose of caffeine and explain to me
how the range could have been -255 to 255?  -256 maybe, or -128 to 127,
but -255??!?

[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]