From: David Miller <davem@davemloft.net>
To: acme@ghostprotocols.net
Cc: linux-kernel@vger.kernel.org
Subject: perf crashes related to map/sym mismatch
Date: Sun, 25 Mar 2012 01:57:19 -0400 (EDT) [thread overview]
Message-ID: <20120325.015719.1170644257291462749.davem@davemloft.net> (raw)
I have perf crashes that eminate in two different ways, but the cause
seems to be identical. The two symptoms are:
1) SYM rbtree corruption, pointers have their low bits set.
2) segmentation fault in symbol__inc_addr_samples
h->addr[offset]++ crashes because offset is "huge" and
offset is "huge" because addr < sym->start
It turns out that #2 is what causes #1, incrementing random addresses
eventually hits a SYM rbtree linkage address thus corrupting the
pointer to be odd.
Why is "addr" smaller than sym->start? It's because the 'map' used to
perform ->map_ip() and adjust "ip" in perf_top__record_precise_ip() is
different from the 'map' used earlier to invoke ->map_ip() to
calculate the final al->addr value in thread__find_addr_map().
Basically if al->map != he->ms.map we are in trouble.
As best I can tell this happens because the hist entry sort routines
do not take the map into account when doing comparisons of whether the
symbols of two hist entries are equal.
So you can end up with a hist entry from a lookup which uses a 'map'
on a DSO which is stale and has subsequently been updated from a more
recent MMAP event. In my case we have two map objects of libc, the
older one in the hist_entry covers:
start = 0xf77bc000
end = 0xf7928000
whereas the newer one in the al->map covers:
start = 0xf765c000
end = 0xf77c8000
The hist_entry map is probably in the current thread's removed_maps
tree, and indeed there is an explicit comment about this in
map_groups__flush()
So it looks like we did a lookup on a symbol in libc pre-exec() and
created a hist_entry for it, then we flush the map groups on the
exec() and in the newly exec'd program we then do a lookup on the same
symbol and this is where we find the hist_entry with the out-of-date
map information.
Perhaps that the right thing to do is to explicitly detect this
situation and flush out the hist_entry with the stale map information
so we can create one with more uptodate mapping info.
next reply other threads:[~2012-03-25 5:57 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-03-25 5:57 David Miller [this message]
2012-03-26 17:23 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120325.015719.1170644257291462749.davem@davemloft.net \
--to=davem@davemloft.net \
--cc=acme@ghostprotocols.net \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®