Re: [RFC] Unify KVM kernel-space and user-space code into a single project

[Ingo Molnar <mingo@elte.hu>]

* Avi Kivity <avi@redhat.com> wrote:

> > Well, for what it's worth, I rarely ever use anything else. My virtual 
> > disks are raw so I can loop mount them easily, and I can also switch my 
> > guest kernels from outside... without ever needing to mount those disks.
> 
> Curious, what do you use them for?
> 
> btw, if you build your kernel outside the guest, then you already have 
> access to all its symbols, without needing anything further.

There's two errors with your argument:

1) you are assuming that it's only about kernel symbols

Look at this 'perf report' output:

# Samples: 7127509216
#
# Overhead     Command                  Shared Object  Symbol
# ........  ..........  .............................  ......
#
    19.14%         git  git                            [.] lookup_object
    15.16%        perf  git                            [.] lookup_object
     4.74%        perf  libz.so.1.2.3                  [.] inflate
     4.52%         git  libz.so.1.2.3                  [.] inflate
     4.21%        perf  libz.so.1.2.3                  [.] inflate_table
     3.94%         git  libz.so.1.2.3                  [.] inflate_table
     3.29%         git  git                            [.] find_pack_entry_one
     3.24%         git  libz.so.1.2.3                  [.] inflate_fast
     2.96%        perf  libz.so.1.2.3                  [.] inflate_fast
     2.96%         git  git                            [.] decode_tree_entry
     2.80%        perf  libc-2.11.90.so                [.] __strlen_sse42
     2.56%         git  libc-2.11.90.so                [.] __strlen_sse42
     1.98%        perf  libc-2.11.90.so                [.] __GI_memcpy
     1.71%        perf  git                            [.] decode_tree_entry
     1.53%         git  libc-2.11.90.so                [.] __GI_memcpy
     1.48%         git  git                            [.] lookup_blob
     1.30%         git  git                            [.] process_tree
     1.30%        perf  git                            [.] process_tree
     0.90%        perf  git                            [.] tree_entry
     0.82%        perf  git                            [.] lookup_blob
     0.78%         git  [kernel.kallsyms]              [k] kstat_irqs_cpu

kernel symbols are only a small portion of the symbols. (a single line in this 
case)

To get to those other symbols we have to read the ELF symbols of those 
binaries in the guest filesystem, in the post-processing/reporting phase. This 
is both complex to do and relatively slow so we dont want to (and cannot) do 
this at sample time from IRQ context or NMI context ...

Also, many aspects of reporting are interactive so it's done lazily or 
on-demand. So we need ready access to the guest filesystem - for those guests 
which decide to integrate with the host for this.

2) the 'SystemTap mistake'

You are assuming that the symbols of the kernel when it got built got saved 
properly and are discoverable easily. In reality those symbols can be erased 
by a make clean, can be modified by a new build, can be misplaced and can 
generally be hard to find because each distro puts them in a different 
installation path.

My 10+ years experience with kernel instrumentation solutions is that 
kernel-driven, self-sufficient, robust, trustable, well-enumerated sources of 
information work far better in practice.

The thing is, in this thread i'm forced to repeat the same basic facts again 
and again. Could you _PLEASE_, pretty please, when it comes to instrumentation 
details, at least _read the mails_ of the guys who actually ... write and 
maintain Linux instrumentation code? This is getting ridiculous really.

Thanks,

	Ingo