Re: [PATCH perf/core 00/13] perf memory/refcnt leak fixes

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Wed, Nov 18, 2015 at 03:40:09PM +0900, Masami Hiramatsu escreveu:
> Hi,
> 
> Here is a series to fix some memory leaks and refcount
> leaks on map and dso. This also includes the refcnt APIs
> with backtrace debugging feature.

Cool, I wonder if this could be usable in the kernel proper... Is there
such a facility there? I'll check.

But thanks for doing this work, I'll go thru the fixes first, then look
at the debugging feature.

- Arnaldo
 
> The story has started from the posible memory leak report
> reported by Wnag Nan.
> I've tried to use valgrind to ensure the perf probe doesn't
> have other memory leaks. The result is here:
> 
>   ----
>   # valgrind ./perf probe vfs_read
>   ==17521== Memcheck, a memory error detector
>   ==17521== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
>   ==17521== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
>   ==17521== Command: ./perf probe vfs_read
>   ==17521==
>   Added new event:
>     probe:vfs_read       (on vfs_read)
>   
>   You can now use it in all perf tools, such as:
>   
>           perf record -e probe:vfs_read -aR sleep 1
>   
>   ==17521==
>   ==17521== HEAP SUMMARY:
>   ==17521==     in use at exit: 3,512,761 bytes in 38,012 blocks
>   ==17521==   total heap usage: 74,723 allocs, 36,711 frees, 24,014,927
>   bytes allocated
>   ==17521==
>   ==17521== LEAK SUMMARY:
>   ==17521==    definitely lost: 6,857 bytes in 49 blocks
>   ==17521==    indirectly lost: 3,501,287 bytes in 37,891 blocks
>   ==17521==      possibly lost: 0 bytes in 0 blocks
>   ==17521==    still reachable: 4,617 bytes in 72 blocks
>   ==17521==         suppressed: 0 bytes in 0 blocks
>   ==17521== Rerun with --leak-check=full to see details of leaked memory
>   ==17521==
>   ==17521== For counts of detected and suppressed errors, rerun with: -v
>   ==17521== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
>   ----
> 
> Oops! It leaked almost 4 MB memories. I've tried to find
> the root causes, and what I've found is there are many
> leaks in not only perf-probe specific code, but also maps
> and dsos (and some other pieces).
> 
> The first 3 patches are just fixing 'easy' memory leaks. However,
> most of the leaks are caused by refcnt. Since valgrind seems not
> able to debug this kind of issues, I introduced a hand-made refcnt
> backtrace APIs for debugging.
> The rest of patches are for fixing refcnt leak bugs and replcing
> refcnt apis.
> 
> After all, most of the issues are gone, except for just a few issues
> in elfutils. I'll continue to investigate that.
> 
>   ----
>   valgrind ./perf probe vfs_read
>   ==29521== Memcheck, a memory error detector
>   ==29521== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
>   ==29521== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
>   ==29521== Command: ./perf probe vfs_read
>   ==29521==
>   Added new event:
>     probe:vfs_read       (on vfs_read)
>   
>   You can now use it in all perf tools, such as:
>   
>           perf record -e probe:vfs_read -aR sleep 1
>   
>   ==29521==
>   ==29521== HEAP SUMMARY:
>   ==29521==     in use at exit: 5,137 bytes in 75 blocks
>   ==29521==   total heap usage: 74,723 allocs, 74,648 frees, 24,014,927
>   bytes allocated
>   ==29521==
>   ==29521== LEAK SUMMARY:
>   ==29521==    definitely lost: 520 bytes in 3 blocks
>   ==29521==    indirectly lost: 0 bytes in 0 blocks
>   ==29521==      possibly lost: 0 bytes in 0 blocks
>   ==29521==    still reachable: 4,617 bytes in 72 blocks
>   ==29521==         suppressed: 0 bytes in 0 blocks
>   ==29521== Rerun with --leak-check=full to see details of leaked memory
>   ==29521==
>   ==29521== For counts of detected and suppressed errors, rerun with: -v
>   ==29521== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
>   ----
> 
> Anyway, I decided to release these fixes and the debugging feature
> because at least this will improve perf quality.
> 
> Thank you,
> 
> ---
> 
> Masami Hiramatsu (13):
>       perf probe: Fix to free temporal Dwarf_Frame
>       perf: Make perf_exec_path always returns malloc'd string
>       perf: Introduce generic refcount APIs with debug feature
>       perf: make map to use refcnt
>       perf: Fix machine__findnew_module_map to put registered map
>       perf: Fix machine__destroy_kernel_maps to put vmlinux_maps
>       perf: Fix to destroy kernel maps when machine exits
>       perf: Fix to put new map after inserting to map_groups in dso__load_sym
>       perf: Make dso to use refcnt for debug
>       perf: Fix __dsos__addnew to put dso after adding it to the list
>       perf: Fix machine__create_kernel_maps to put kernel dso
>       perf: Fix machine__findnew_module_map to put dso
>       perf: Fix dso__load_sym to put dso
> 
> 
>  tools/perf/config/Makefile     |    5 ++
>  tools/perf/util/Build          |    1 
>  tools/perf/util/dso.c          |    9 ++-
>  tools/perf/util/exec_cmd.c     |   20 ++++--
>  tools/perf/util/exec_cmd.h     |    5 +-
>  tools/perf/util/help.c         |    6 +-
>  tools/perf/util/machine.c      |   17 ++++-
>  tools/perf/util/map.c          |    7 +-
>  tools/perf/util/map.h          |    3 +
>  tools/perf/util/probe-finder.c |    9 ++-
>  tools/perf/util/refcnt.c       |  125 ++++++++++++++++++++++++++++++++++++++++
>  tools/perf/util/refcnt.h       |   65 +++++++++++++++++++++
>  tools/perf/util/symbol-elf.c   |    4 +
>  13 files changed, 250 insertions(+), 26 deletions(-)
>  create mode 100644 tools/perf/util/refcnt.c
>  create mode 100644 tools/perf/util/refcnt.h
> 
> 
> -- 
> Masami HIRAMATSU
> Linux Technology Research Center, System Productivity Research Dept.
> Center for Technology Innovation - Systems Engineering 
> Hitachi, Ltd., Research & Development Group
> E-mail: masami.hiramatsu.pt@hitachi.com