brk() should check randomize_va_space rather then CONFIG_COMPAT_BRK

brk() should check randomize_va_space rather then CONFIG_COMPAT_BRK

[Frank Heckenbach]

The brk() syscall checks CONFIG_COMPAT_BRK to decide whether the brk
area starts just after the end of the code+bss.

Since CONFIG_COMPAT_BRK can be overridden by
/proc/sys/kernel/randomize_va_space, it should rather check this
variable (which is initialized in mm/memory.c from
CONFIG_COMPAT_BRK), see the patch below.

I've tested it with an old libc5 binary: Without the patch, if the
kernel was configured without CONFIG_COMPAT_BRK, the first malloc()
fails with any setting of randomize_va_space. With the patch
applied, the program runs correctly with randomize_va_space < 2, but
not with randomize_va_space == 2, as it should be.

--- linux-2.6.37.2/mm/mmap.c.orig	2011-02-25 00:09:00.000000000 +0100
+++ linux-2.6.37.2/mm/mmap.c	2011-02-26 01:13:53.000000000 +0100
@@ -252,11 +252,11 @@
 
 	down_write(&mm->mmap_sem);
 
-#ifdef CONFIG_COMPAT_BRK
-	min_brk = mm->end_code;
-#else
-	min_brk = mm->start_brk;
-#endif
+	if (randomize_va_space < 2)
+		min_brk = mm->end_code;
+	else
+		min_brk = mm->start_brk;
+
 	if (brk < min_brk)
 		goto out;
 

Re: brk() should check randomize_va_space rather then CONFIG_COMPAT_BRK

[Jiri Kosina]

On Sun, 27 Feb 2011, Frank Heckenbach wrote:

> The brk() syscall checks CONFIG_COMPAT_BRK to decide whether the brk
> area starts just after the end of the code+bss.
> 
> Since CONFIG_COMPAT_BRK can be overridden by
> /proc/sys/kernel/randomize_va_space, it should rather check this
> variable (which is initialized in mm/memory.c from
> CONFIG_COMPAT_BRK), see the patch below.
> 
> I've tested it with an old libc5 binary: Without the patch, if the
> kernel was configured without CONFIG_COMPAT_BRK, the first malloc()
> fails with any setting of randomize_va_space. With the patch
> applied, the program runs correctly with randomize_va_space < 2, but
> not with randomize_va_space == 2, as it should be.
> 
> --- linux-2.6.37.2/mm/mmap.c.orig	2011-02-25 00:09:00.000000000 +0100
> +++ linux-2.6.37.2/mm/mmap.c	2011-02-26 01:13:53.000000000 +0100
> @@ -252,11 +252,11 @@
>  
>  	down_write(&mm->mmap_sem);
>  
> -#ifdef CONFIG_COMPAT_BRK
> -	min_brk = mm->end_code;
> -#else
> -	min_brk = mm->start_brk;
> -#endif
> +	if (randomize_va_space < 2)
> +		min_brk = mm->end_code;
> +	else
> +		min_brk = mm->start_brk;
> +
>  	if (brk < min_brk)
>  		goto out;

Hi Frank,

how do you avoid race here? 

More precisely -- randomize_va_space can be changed in runtime, when 
already running processess have been started with different 
randomize_va_space value (and thus the shifting of mm->brk already 
happened in arch_randomize_brk())

-- 
Jiri Kosina
SUSE Labs, Novell Inc.

Re: brk() should check randomize_va_space rather then CONFIG_COMPAT_BRK

[Jiri Kosina]

On Mon, 28 Feb 2011, Jiri Kosina wrote:

> > The brk() syscall checks CONFIG_COMPAT_BRK to decide whether the brk
> > area starts just after the end of the code+bss.
> > 
> > Since CONFIG_COMPAT_BRK can be overridden by
> > /proc/sys/kernel/randomize_va_space, it should rather check this
> > variable (which is initialized in mm/memory.c from
> > CONFIG_COMPAT_BRK), see the patch below.
> > 
> > I've tested it with an old libc5 binary: Without the patch, if the
> > kernel was configured without CONFIG_COMPAT_BRK, the first malloc()
> > fails with any setting of randomize_va_space. With the patch
> > applied, the program runs correctly with randomize_va_space < 2, but
> > not with randomize_va_space == 2, as it should be.
> > 
> > --- linux-2.6.37.2/mm/mmap.c.orig	2011-02-25 00:09:00.000000000 +0100
> > +++ linux-2.6.37.2/mm/mmap.c	2011-02-26 01:13:53.000000000 +0100
> > @@ -252,11 +252,11 @@
> >  
> >  	down_write(&mm->mmap_sem);
> >  
> > -#ifdef CONFIG_COMPAT_BRK
> > -	min_brk = mm->end_code;
> > -#else
> > -	min_brk = mm->start_brk;
> > -#endif
> > +	if (randomize_va_space < 2)
> > +		min_brk = mm->end_code;
> > +	else
> > +		min_brk = mm->start_brk;
> > +
> >  	if (brk < min_brk)
> >  		goto out;
> 
> Hi Frank,
> 
> how do you avoid race here? 
> 
> More precisely -- randomize_va_space can be changed in runtime, when 
> already running processess have been started with different 
> randomize_va_space value (and thus the shifting of mm->brk already 
> happened in arch_randomize_brk())

Oh, and also see the commit

	commit 5520e89485252c759ee60d313e9422447659947b
	Author: Jiri Kosina <jkosina@suse.cz>
	Date:   Thu Jan 13 15:47:23 2011 -0800

	    brk: fix min_brk lower bound computation for COMPAT_BRK

which is quite relevant here as well (and you seem to be sending patch 
against code that doesn't have this patch applied).

-- 
Jiri Kosina
SUSE Labs, Novell Inc.

Re: brk() should check randomize_va_space rather then CONFIG_COMPAT_BRK

[Frank Heckenbach]

Jiri Kosina wrote:

> On Mon, 28 Feb 2011, Jiri Kosina wrote:
> 
> > how do you avoid race here?
> > 
> > More precisely -- randomize_va_space can be changed in runtime, when 
> > already running processess have been started with different 
> > randomize_va_space value (and thus the shifting of mm->brk already 
> > happened in arch_randomize_brk())

I see. So would it help setting mm->brk = mm->start_brk =
mm->end_code if randomize_va_space < 2 in load_elf_binary() (and
elsewhere if needed -- not sure if other binformats are affected)?

> Oh, and also see the commit
> 
> 	commit 5520e89485252c759ee60d313e9422447659947b
> 	Author: Jiri Kosina <jkosina@suse.cz>
> 	Date:   Thu Jan 13 15:47:23 2011 -0800
> 
> 	    brk: fix min_brk lower bound computation for COMPAT_BRK
> 
> which is quite relevant here as well

AFAICS, this patch only changes the CONFIG_COMPAT_BRK case. I'm more
interested in the !CONFIG_COMPAT_BRK case (since my old binaries
work with CONFIG_COMPAT_BRK as is). Since e.g., Debian's default
kernel doesn't set CONFIG_COMPAT_BRK, I had hoped I could get them
to run by setting randomize_va_space to 0 or 1.

> (and you seem to be sending patch
> against code that doesn't have this patch applied).

Well, I used the last stable release; I hadn't noticed there was a
more recent change, sorry.

Frank