2.6.22.6 - Discrepancy between running and on-disk kernels

2.6.22.6 - Discrepancy between running and on-disk kernels

[Yan]

Hello,

I have been trying to compare the code from the on-disk compressed kernel
that was booted and the running kernel extracted from /dev/kmem. I extracted
the kernel's code from the disk image by stripping the head.S and similar
and gunzipping it, and extracted the kernel from /dev/kmem by reading data
between _text and _etext symbol offsets.

I then ran both through a disassembler and diff'ed the outputs. Predictably,
the disassembly was similar, but not identical. Some instructions (e.g. bts)
had a 'lock' prefix, where as others had a 'nop' in its place.

There were other differences with some instructions like mfence. Everything
else matched just fine, the differences were mostly in memory-referencing
instructions.

My question is, what can be changing the kernel between being on static
storage and being loaded? Or am I just being fooled into thinking something
changed it? I'm thinking that it had something to do with mutex locking
and SMP, but I looked through boot code and tried googling and came up with
nothing. Another possibility is me interpreting non-instructions as
instructions.

Example:

The following pattern repeats a lot:

1325c1326,1327
<     1089: f0 0f b3 04 24          lock btr %eax,(%esp)
---
>     1089: 90                      nop
>     108a: 0f b3 04 24             btr    %eax,(%esp)

As well as other changes:

94383,94384c94832,94835
<    47bb3: f0 83 04 24 00          lock addl $0x0,(%esp)
<    47bb8: f0 ff 05 7c 30 80 c0    lock incl 0xc080307c
---
>    47bb3: 0f ae f0                mfence
>    47bb6: 89 f6                   mov    %esi,%esi
>    47bb8: 90                      nop
>    47bb9: ff 05 7c 30 80 c0       incl   0xc080307c


(I'm not subscribed; please cc any responses to me)

Thanks in advance,
Yan

Re: 2.6.22.6 - Discrepancy between running and on-disk kernels

[Arjan van de Ven]

On Fri, 14 Mar 2008 10:51:43 -0400
Yan <rottled@gmail.com> wrote:

> Hello,
> 
> I have been trying to compare the code from the on-disk compressed
> kernel that was booted and the running kernel extracted
> from /dev/kmem. I extracted the kernel's code from the disk image by
> stripping the head.S and similar and gunzipping it, and extracted the
> kernel from /dev/kmem by reading data between _text and _etext symbol
> offsets.
> 
> I then ran both through a disassembler and diff'ed the outputs.
> Predictably, the disassembly was similar, but not identical. Some
> instructions (e.g. bts) had a 'lock' prefix, where as others had a
> 'nop' in its place.
> 
> There were other differences with some instructions like mfence.
> Everything else matched just fine, the differences were mostly in
> memory-referencing instructions.
> 
> My question is, what can be changing the kernel between being on
> static storage and being loaded?

the kernel code is self-patching, it gets modified to match your system
during boot time. So you cannot assume that the kernel on disk and the kernel
in memory are identical.
(Same goes for modules)

Re: 2.6.22.6 - Discrepancy between running and on-disk kernels

[Yan]

Can you point me to where in the source that self-patching happens? I
tried grep'ing
through the source for anything relevant, and came out empty handed.

-yan

On Fri, Mar 14, 2008 at 12:00 PM, Arjan van de Ven <arjan@infradead.org> wrote:
> On Fri, 14 Mar 2008 10:51:43 -0400
>  Yan <rottled@gmail.com> wrote:
>
>  > Hello,
>  >
>  > I have been trying to compare the code from the on-disk compressed
>  > kernel that was booted and the running kernel extracted
>  > from /dev/kmem. I extracted the kernel's code from the disk image by
>  > stripping the head.S and similar and gunzipping it, and extracted the
>  > kernel from /dev/kmem by reading data between _text and _etext symbol
>  > offsets.
>  >
>  > I then ran both through a disassembler and diff'ed the outputs.
>  > Predictably, the disassembly was similar, but not identical. Some
>  > instructions (e.g. bts) had a 'lock' prefix, where as others had a
>  > 'nop' in its place.
>  >
>  > There were other differences with some instructions like mfence.
>  > Everything else matched just fine, the differences were mostly in
>  > memory-referencing instructions.
>  >
>  > My question is, what can be changing the kernel between being on
>  > static storage and being loaded?
>
>  the kernel code is self-patching, it gets modified to match your system
>  during boot time. So you cannot assume that the kernel on disk and the kernel
>  in memory are identical.
>  (Same goes for modules)
>

Re: 2.6.22.6 - Discrepancy between running and on-disk kernels

[Arjan van de Ven]

On Fri, 14 Mar 2008 12:08:16 -0400
Yan <rottled@gmail.com> wrote:

> Can you point me to where in the source that self-patching happens? I
> tried grep'ing
> through the source for anything relevant, and came out empty handed.


in the apply_alternatives() function

> 
> -yan
> 
> On Fri, Mar 14, 2008 at 12:00 PM, Arjan van de Ven
> <arjan@infradead.org> wrote:
> > On Fri, 14 Mar 2008 10:51:43 -0400
> >  Yan <rottled@gmail.com> wrote:
> >
> >  > Hello,
> >  >
> >  > I have been trying to compare the code from the on-disk
> >  > compressed kernel that was booted and the running kernel
> >  > extracted from /dev/kmem. I extracted the kernel's code from the
> >  > disk image by stripping the head.S and similar and gunzipping
> >  > it, and extracted the kernel from /dev/kmem by reading data
> >  > between _text and _etext symbol offsets.
> >  >
> >  > I then ran both through a disassembler and diff'ed the outputs.
> >  > Predictably, the disassembly was similar, but not identical. Some
> >  > instructions (e.g. bts) had a 'lock' prefix, where as others had
> >  > a 'nop' in its place.
> >  >
> >  > There were other differences with some instructions like mfence.
> >  > Everything else matched just fine, the differences were mostly in
> >  > memory-referencing instructions.
> >  >
> >  > My question is, what can be changing the kernel between being on
> >  > static storage and being loaded?
> >
> >  the kernel code is self-patching, it gets modified to match your
> > system during boot time. So you cannot assume that the kernel on
> > disk and the kernel in memory are identical.
> >  (Same goes for modules)
> >


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org