Re: [PATCH V3 00/25] perf tools: Introduce an abstraction for AUX Area and Instruction Tracing

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Fri, Apr 24, 2015 at 10:29:42PM +0300, Adrian Hunter escreveu:
> Hi
> 
> Here is V3 patches for the introduction of an abstraction for
> using the AUX area and Instruction tracing.
> 
> The patches can also be found here:
> 
> 	http://git.infradead.org/users/ahunter/linux-perf.git
> 
> An example perf.data file and build id archive can be found here:
> 
> 	http://git.infradead.org/~ahunter/tfr/
> 
> There is also a tar of the 3 most relevant files with debugging
> symbols. These need to be placed in under the correct paths in
> /usr/lib/debug to get symbols.

Thanks a lot for making those files available, I'll try and push a first
batch from what is in perf/core by now, the try those files, and then
apply some more, hopefully finishing the outstanding ones over the
weekend.

- Arnaldo
 
> Changes in V3:
> 
>    New patch:
>       perf tools: Amend mmap ref counting for the AUX area mmap
> 
>    Move some code under arch:
>       perf tools: Add Intel PT support
>       perf tools: Add Intel BTS support
> 
>    Updated documentation:
>       perf report: Add Instruction Tracing support
>       perf auxtrace: Add option to synthesize events for transactions
>       perf tools: Take Intel PT into use
>       perf tools: Add Intel BTS support
> 
>    Patches already applied:
>       perf header: Add AUX area tracing feature
>       perf evlist: Add support for mmapping an AUX area buffer
>       perf tools: Add user events for AUX area tracing
>       perf tools: Add support for AUX area recording
>       perf record: Add basic AUX area tracing support
>       perf record: Extend -m option for AUX area tracing mmap pages
>       perf tools: Add a user event for AUX area tracing errors
>       perf session: Add hooks to allow transparent decoding of AUX area tracing data
>       perf session: Add instruction tracing options
>       perf auxtrace: Add helpers for AUX area tracing errors
>       perf auxtrace: Add helpers for queuing AUX area tracing data
>       perf auxtrace: Add a heap for sorting AUX area tracing queues
>       perf auxtrace: Add processing for AUX area tracing events
>       perf auxtrace: Add a hashtable for caching
>       perf tools: Add member to struct dso for an instruction cache
>       perf script: Add Instruction Tracing support
>       perf inject: Re-pipe AUX area tracing events
>       perf inject: Add Instruction Tracing support
>       perf script: Add field option 'flags' to print sample flags
>       perf tools: Add aux_watermark member of struct perf_event_attr
> 
> Changes in V2:
> 
>    Get rid of MIN()
>       perf auxtrace: Add helpers for AUX area tracing errors
>       perf inject: Re-pipe AUX area tracing events
>       perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
> 
> 
> Intel BTS can be used on most recent Intel CPUs. Intel PT
> is available on Broadwell.
> 
> Examples:
> 
> 	Trace 'ls' with Intel BTS userspace only
> 
> 	perf record --per-thread -e intel_bts//u ls
> 	perf report
> 	perf script
> 
> 	Trace 'ls' with Intel BTS kernel and userspace
> 
> 	~/libexec/perf-core/perf-with-kcore record bts-ls --per-thread -e intel_bts// -- ls
> 	~/libexec/perf-core/perf-with-kcore report bts-ls
> 	~/libexec/perf-core/perf-with-kcore script bts-ls
> 
> 	Trace 'ls' with Intel PT userspace only
> 
> 	perf record -e intel_pt//u ls
> 	perf report
> 	perf script
> 
> 	Trace 'ls' with Intel PT kernel and userspace
> 
> 	~/libexec/perf-core/perf-with-kcore record pt-ls -e intel_pt// -- ls
> 	~/libexec/perf-core/perf-with-kcore report pt-ls
> 	~/libexec/perf-core/perf-with-kcore script pt-ls
> 
> 
> The abstraction has two separate aspects:
> 	1. recording AUX area data
> 	2. processing AUX area data
> 
> Recording consists of mmapping a separate buffer and copying
> the data into the perf.data file.  The buffer is an AUX area
> buffer.  The data is written preceded by a new user event
> PERF_RECORD_AUXTRACE.  The data is too big to fit in the event
> but follows immediately afterward. Session processing has to
> skip to get to the next event header in a similar fashion to
> the existing PERF_RECORD_HEADER_TRACING_DATA
> event.  The main recording patches are:
> 
>       perf evlist: Add support for mmapping an AUX area buffer
>       perf tools: Add user events for AUX area tracing
>       perf tools: Add support for AUX area recording
>       perf record: Add basic AUX area tracing support
> 
> Processing consists of providing hooks in session processing
> to enable a decoder to see all the events and deliver synthesized
> events transparently into the event stream.  The main processing
> patch is:
> 
>       perf session: Add hooks to allow transparent decoding of AUX area tracing data
> 
> 
> Adrian Hunter (25):
>       perf tools: Amend mmap ref counting for the AUX area mmap
>       perf script: Always allow fields 'addr' and 'cpu' for auxtrace
>       perf report: Add Instruction Tracing support
>       perf tools: Add AUX area tracing index
>       perf tools: Hit all build ids when AUX area tracing
>       perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
>       perf auxtrace: Add option to synthesize events for transactions
>       perf tools: Add support for PERF_RECORD_AUX
>       perf tools: Add support for PERF_RECORD_ITRACE_START
>       perf tools: Add AUX area tracing Snapshot Mode
>       perf record: Add AUX area tracing Snapshot Mode support
>       perf auxtrace: Add Intel PT as an AUX area tracing type
>       perf tools: Add Intel PT packet decoder
>       perf tools: Add Intel PT instruction decoder
>       perf tools: Add Intel PT log
>       perf tools: Add Intel PT decoder
>       perf tools: Add Intel PT support
>       perf tools: Take Intel PT into use
>       perf tools: Allow auxtrace data alignment
>       perf tools: Add Intel BTS support
>       perf tools: Output sample flags and insn_len from intel_pt
>       perf tools: Output sample flags and insn_len from intel_bts
>       perf tools: Intel PT to always update thread stack trace number
>       perf tools: Intel BTS to always update thread stack trace number
>       perf tools: Add example call-graph script
> 
>  tools/build/Makefile.build                         |    2 +
>  tools/perf/.gitignore                              |    2 +
>  tools/perf/Documentation/intel-bts.txt             |   69 +
>  tools/perf/Documentation/intel-pt.txt              |  467 +++++
>  tools/perf/Documentation/perf-inject.txt           |    9 +-
>  tools/perf/Documentation/perf-record.txt           |    7 +
>  tools/perf/Documentation/perf-report.txt           |   28 +
>  tools/perf/Documentation/perf-script.txt           |    9 +-
>  tools/perf/Makefile.perf                           |    8 +-
>  tools/perf/arch/x86/util/Build                     |    5 +
>  tools/perf/arch/x86/util/auxtrace.c                |   83 +
>  tools/perf/arch/x86/util/intel-bts.c               |  458 +++++
>  tools/perf/arch/x86/util/intel-pt.c                |  752 ++++++++
>  tools/perf/arch/x86/util/pmu.c                     |   15 +
>  tools/perf/builtin-buildid-list.c                  |    9 +
>  tools/perf/builtin-inject.c                        |   78 +-
>  tools/perf/builtin-record.c                        |  172 +-
>  tools/perf/builtin-report.c                        |   11 +
>  tools/perf/builtin-script.c                        |   29 +-
>  tools/perf/config/Makefile                         |    5 +
>  tools/perf/perf.h                                  |    3 +
>  .../scripts/python/call-graph-from-postgresql.py   |  285 +++
>  tools/perf/tests/make                              |    2 +
>  tools/perf/util/Build                              |    5 +-
>  tools/perf/util/auxtrace.c                         |  322 +++-
>  tools/perf/util/auxtrace.h                         |  220 +++
>  tools/perf/util/event.c                            |   39 +
>  tools/perf/util/event.h                            |   24 +
>  tools/perf/util/evlist.c                           |    2 +-
>  tools/perf/util/header.c                           |   31 +-
>  tools/perf/util/intel-bts.c                        |  919 ++++++++++
>  tools/perf/util/intel-bts.h                        |   43 +
>  tools/perf/util/intel-pt-decoder/Build             |   14 +
>  .../perf/util/intel-pt-decoder/intel-pt-decoder.c  | 1738 ++++++++++++++++++
>  .../perf/util/intel-pt-decoder/intel-pt-decoder.h  |   89 +
>  .../util/intel-pt-decoder/intel-pt-insn-decoder.c  |  246 +++
>  .../util/intel-pt-decoder/intel-pt-insn-decoder.h  |   65 +
>  tools/perf/util/intel-pt-decoder/intel-pt-log.c    |  155 ++
>  tools/perf/util/intel-pt-decoder/intel-pt-log.h    |   52 +
>  .../util/intel-pt-decoder/intel-pt-pkt-decoder.c   |  400 +++++
>  .../util/intel-pt-decoder/intel-pt-pkt-decoder.h   |   64 +
>  tools/perf/util/intel-pt.c                         | 1897 ++++++++++++++++++++
>  tools/perf/util/intel-pt.h                         |   57 +
>  tools/perf/util/machine.c                          |   21 +
>  tools/perf/util/machine.h                          |    4 +
>  tools/perf/util/parse-options.h                    |    4 +
>  tools/perf/util/session.c                          |   32 +
>  tools/perf/util/session.h                          |    1 +
>  tools/perf/util/tool.h                             |    2 +
>  49 files changed, 8882 insertions(+), 72 deletions(-)
>  create mode 100644 tools/perf/Documentation/intel-bts.txt
>  create mode 100644 tools/perf/Documentation/intel-pt.txt
>  create mode 100644 tools/perf/arch/x86/util/auxtrace.c
>  create mode 100644 tools/perf/arch/x86/util/intel-bts.c
>  create mode 100644 tools/perf/arch/x86/util/intel-pt.c
>  create mode 100644 tools/perf/arch/x86/util/pmu.c
>  create mode 100644 tools/perf/scripts/python/call-graph-from-postgresql.py
>  create mode 100644 tools/perf/util/intel-bts.c
>  create mode 100644 tools/perf/util/intel-bts.h
>  create mode 100644 tools/perf/util/intel-pt-decoder/Build
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.c
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-decoder.h
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.c
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-insn-decoder.h
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.c
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-log.h
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.c
>  create mode 100644 tools/perf/util/intel-pt-decoder/intel-pt-pkt-decoder.h
>  create mode 100644 tools/perf/util/intel-pt.c
>  create mode 100644 tools/perf/util/intel-pt.h
> 
> 
> Regards
> Adrian