[PATCH v8 0/3] KVM: perf: kvm events analysis tool

[Dong Hao <haodong@linux.vnet.ibm.com>]

From: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>

Changelog:
the changes from Avi's comments:
- move the definition of x86 fault vector to asm/kvm.h
- drop the second patch which introduced new tracpoints

the changes from David's comments:
- use scnprintf instead of snprintf
- drop exclusive check for '-a' and '-p', after this, we should
  append -a to track all guest, -p to track the specified guest
- improve the help usage
- fix a possible memory leak
- some cleanups

the changes from Andrew Jones's comments:
- move stat related code to util/stat.c

Thank you very much for your patience!

This patchset introduces a perf-based tool (perf kvm stat record/report)
which can analyze kvm events more smartly. Below is the presentation slice
on 2012 Japan LinuxCon:
http://events.linuxfoundation.org/images/stories/pdf/lcjp2012_guangrong.pdf
You can get more details from it. If any questions/comments, please feel free
to let us know.

This patchset is based on Arnaldo's git tree perf/core branch.

Usage:
- kvm stat
  run a command and gather performance counter statistics, it is the alias of
  perf stat

- trace kvm events:
  perf kvm stat record, or, if other tracepoints are interesting as well, we
  can append the events like this:
  perf kvm stat record -e timer:* -a

  If many guests are running, we can track the specified guest by using -p or
  --pid. -a is used to track events generated by all guests.

- show the result:
  perf kvm stat report

The output example is following:
# pgrep qemu
13005
13059

total 2 guests are running on the host

Then, track the guest whose pid is 13059:
# ./perf kvm stat record -p 13059
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.253 MB perf.data.guest (~11065 samples) ]

See the vmexit events:
# ./perf kvm stat report --event=vmexit


Analyze events for all VCPUs:

             VM-EXIT    Samples  Samples%     Time%         Avg time

         APIC_ACCESS        460    70.55%     0.01%     22.44us ( +-   1.75% )
                 HLT         93    14.26%    99.98% 832077.26us ( +-  10.42% )
  EXTERNAL_INTERRUPT         64     9.82%     0.00%     35.35us ( +-  14.21% )
   PENDING_INTERRUPT         24     3.68%     0.00%      9.29us ( +-  31.39% )
           CR_ACCESS          7     1.07%     0.00%      8.12us ( +-   5.76% )
      IO_INSTRUCTION          3     0.46%     0.00%     18.00us ( +-  11.79% )
       EXCEPTION_NMI          1     0.15%     0.00%      5.83us ( +-   -nan% )

Total Samples:652, Total events handled time:77396109.80us.

See the mmio events:
# ./perf kvm stat report --event=mmio


Analyze events for all VCPUs:

         MMIO Access    Samples  Samples%     Time%         Avg time

        0xfee00380:W        387    84.31%    79.28%      8.29us ( +-   3.32% )
        0xfee00300:W         24     5.23%     9.96%     16.79us ( +-   1.97% )
        0xfee00300:R         24     5.23%     7.83%     13.20us ( +-   3.00% )
        0xfee00310:W         24     5.23%     2.93%      4.94us ( +-   3.84% )

Total Samples:459, Total events handled time:4044.59us.

See the ioport event:
# ./perf kvm stat report --event=ioport


Analyze events for all VCPUs:

      IO Port Access    Samples  Samples%     Time%         Avg time

         0xc050:POUT          3   100.00%   100.00%     13.75us ( +-  10.83% )

Total Samples:3, Total events handled time:41.26us.

And, --vcpu is used to track the specified vcpu and --key is used to sort the
result:
# ./perf kvm stat report --event=vmexit --vcpu=0 --key=time


Analyze events for VCPU 0:

             VM-EXIT    Samples  Samples%     Time%         Avg time

                 HLT         27    13.85%    99.97% 405790.24us ( +-  12.70% )
  EXTERNAL_INTERRUPT         13     6.67%     0.00%     27.94us ( +-  22.26% )
         APIC_ACCESS        146    74.87%     0.03%     21.69us ( +-   2.91% )
      IO_INSTRUCTION          2     1.03%     0.00%     17.77us ( +-  20.56% )
           CR_ACCESS          2     1.03%     0.00%      8.55us ( +-   6.47% )
   PENDING_INTERRUPT          5     2.56%     0.00%      6.27us ( +-   3.94% )

Total Samples:195, Total events handled time:10959950.90us.


Dong Hao (3):
  KVM: x86: export svm/vmx exit code and vector code to userspace
  perf: move stat related code to util/stat.c
  KVM: perf: kvm events analysis tool

 arch/x86/include/asm/kvm.h            |   16 +
 arch/x86/include/asm/kvm_host.h       |   16 -
 arch/x86/include/asm/svm.h            |  205 +++++---
 arch/x86/include/asm/vmx.h            |  127 ++++--
 arch/x86/kvm/trace.h                  |   89 ----
 tools/perf/Documentation/perf-kvm.txt |   30 ++-
 tools/perf/MANIFEST                   |    3 +
 tools/perf/Makefile                   |    1 +
 tools/perf/builtin-kvm.c              |  840 ++++++++++++++++++++++++++++++++-
 tools/perf/builtin-stat.c             |   56 +---
 tools/perf/util/header.c              |   59 +++-
 tools/perf/util/header.h              |    1 +
 tools/perf/util/stat.c                |   57 +++
 tools/perf/util/stat.h                |   16 +
 tools/perf/util/thread.h              |    2 +
 15 files changed, 1235 insertions(+), 283 deletions(-)
 create mode 100644 tools/perf/util/stat.c
 create mode 100644 tools/perf/util/stat.h

-- 
1.7.2.5