Rebuilding and checksumming the Linux kernel

Rebuilding and checksumming the Linux kernel

[Edgardo Hames]

I'm working on a project with teams spread across the world and we all
work on the same repository patching the kernel and then integrating
into a common main branch. Even though we label the source code, we
would like to make sure that we are all building the same kernel by
running md5sum on the resulting binary.

Right now, this is not possible because the kernel includes a
timestamp and a build number on the binary. What are this timestamps
used for? Can we just remove them? Is there any other thing that may
lead to different binaries? (we are using the exact toolchains and
build machines, so we can assume that the same executable is generated
if the input source is the same).

Please, cc me in your replies because I'm not subscribed to the list.

Thanks for your time,
Ed

Re: Rebuilding and checksumming the Linux kernel

[Arjan van de Ven]

On Sun, 2007-04-01 at 15:56 -0300, Edgardo Hames wrote:
> I'm working on a project with teams spread across the world and we all
> work on the same repository patching the kernel and then integrating
> into a common main branch. Even though we label the source code, we
> would like to make sure that we are all building the same kernel by
> running md5sum on the resulting binary.
> 
> Right now, this is not possible because the kernel includes a
> timestamp and a build number on the binary. What are this timestamps
> used for? Can we just remove them? Is there any other thing that may
> lead to different binaries? (we are using the exact toolchains and
> build machines, so we can assume that the same executable is generated
> if the input source is the same).

elfutils has a "elfcmp" program that may be useful for you

another trick could be to run objdump -d on the vmlinux, and then
compare the assembly file.. you could even grep out any known date like
things that way,,

Re: Rebuilding and checksumming the Linux kernel

[Jan Engelhardt]

On Apr 1 2007 15:56, Edgardo Hames wrote:
>
> I'm working on a project with teams spread across the world and we all
> work on the same repository patching the kernel and then integrating
> into a common main branch. Even though we label the source code, we
> would like to make sure that we are all building the same kernel by
> running md5sum on the resulting binary.

If you just want to make sure that interfaces match,
CONFIG_MODULE_SRCVERSION_ALL should be doing the right thing.


Jan
-- 

RE: Rebuilding and checksumming the Linux kernel

[David Schwartz]

> I'm working on a project with teams spread across the world and we all
> work on the same repository patching the kernel and then integrating
> into a common main branch. Even though we label the source code, we
> would like to make sure that we are all building the same kernel by
> running md5sum on the resulting binary.

> Right now, this is not possible because the kernel includes a
> timestamp and a build number on the binary. What are this timestamps
> used for? Can we just remove them? Is there any other thing that may
> lead to different binaries? (we are using the exact toolchains and
> build machines, so we can assume that the same executable is generated
> if the input source is the same).

I think this is wrong in principle even if you can get it to work in
practice. If you want to all work on the same kernel, then keep a single
copy somewhere that you all work from. There is no reason the same source
code will or should always produce the same output. Compilers are not
generally required to be deterministic.

DS

Re: Rebuilding and checksumming the Linux kernel

[Sam Ravnborg]

On Sun, Apr 01, 2007 at 03:56:30PM -0300, Edgardo Hames wrote:
> I'm working on a project with teams spread across the world and we all
> work on the same repository patching the kernel and then integrating
> into a common main branch. Even though we label the source code, we
> would like to make sure that we are all building the same kernel by
> running md5sum on the resulting binary.
> 
> Right now, this is not possible because the kernel includes a
> timestamp and a build number on the binary. What are this timestamps
> used for? Can we just remove them? Is there any other thing that may
> lead to different binaries? (we are using the exact toolchains and
> build machines, so we can assume that the same executable is generated
> if the input source is the same).

Soon you will be able to say:

KBUILD_BUILD_TIMESTAMP="2007-02-30 01:02.03" KBUILD_BUILD_VERSION=99 make

It will hit -mm soon and mainline in next merge window.
Patch is in kbuild.git at git.kernel.org.

	Sam

Re: Rebuilding and checksumming the Linux kernel

[Edgardo Hames]

On 4/1/07, Sam Ravnborg <sam@ravnborg.org> wrote:
> On Sun, Apr 01, 2007 at 03:56:30PM -0300, Edgardo Hames wrote:
> > I'm working on a project with teams spread across the world and we all
> > work on the same repository patching the kernel and then integrating
> > into a common main branch. Even though we label the source code, we
> > would like to make sure that we are all building the same kernel by
> > running md5sum on the resulting binary.
> >
> > Right now, this is not possible because the kernel includes a
> > timestamp and a build number on the binary. What are this timestamps
> > used for? Can we just remove them? Is there any other thing that may
> > lead to different binaries? (we are using the exact toolchains and
> > build machines, so we can assume that the same executable is generated
> > if the input source is the same).
>
> Soon you will be able to say:
>
> KBUILD_BUILD_TIMESTAMP="2007-02-30 01:02.03" KBUILD_BUILD_VERSION=99 make
>
> It will hit -mm soon and mainline in next merge window.
> Patch is in kbuild.git at git.kernel.org.

This is so much more than I could have asked for! I will try this
patch on my version of the kernel.

Thanks a lot,
Ed
-- 
Encontrá a "Tu psicópata favorito" http://tuxmaniac.blogspot.com

The future is not what it used to be.
-- Paul Valéry

Re: Rebuilding and checksumming the Linux kernel

[Edgardo Hames]

On Apr 1, 6:10 pm, "David Schwartz"  wrote:
> > I'm working on a project with teams spread across the world and we all
> > work on the same repository patching the kernel and then integrating
> > into a common main branch. Even though we label the source code, we
> > would like to make sure that we are all building the same kernel by
> > running md5sum on the resulting binary.
> > Right now, this is not possible because the kernel includes a
> > timestamp and a build number on the binary. What are this timestamps
> > used for? Can we just remove them? Is there any other thing that may
> > lead to different binaries? (we are using the exact toolchains and
> > build machines, so we can assume that the same executable is generated
> > if the input source is the same).
>
> I think this is wrong in principle even if you can get it to work in
> practice. If you want to all work on the same kernel, then keep a single
> copy somewhere that you all work from. There is no reason the same source
> code will or should always produce the same output. Compilers are not
> generally required to be deterministic.

Could you please be a little more specific? We keep toolchains in our
SCM tool, so we can always retrieve a particular version of them and
rebuild with them. Why shouldn't we get the same binary then? I'm
really interested in hearing your opinion on this practice.

Thanks,
Ed
-- 
Encontrá a "Tu psicópata favorito" http://tuxmaniac.blogspot.com

The future is not what it used to be.
-- Paul Valéry

Re: Rebuilding and checksumming the Linux kernel

[Jan Engelhardt]

On Apr 1 2007 22:03, Edgardo Hames wrote:

>> code will or should always produce the same output. Compilers are not
>> generally required to be deterministic.
>
> Could you please be a little more specific? We keep toolchains in our
> SCM tool,

Compile this, shush.

#include <stdio.h>
int main(void)
{
	printf("%s %s\n", __DATE__, __TIME__);
	return 0;
}


Jan
--