From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Tom Zanussi <tom.zanussi@linux.intel.com>
Cc: rostedt@goodmis.org, masami.hiramatsu.pt@hitachi.com,
namhyung@kernel.org, peterz@infradead.org,
linux-kernel@vger.kernel.org,
Daniel Borkmann <daniel@iogearbox.net>,
netdev@vger.kernel.org
Subject: Re: [RFC][PATCH 00/10] Add trace event support to eBPF
Date: Thu, 18 Feb 2016 20:16:23 -0800 [thread overview]
Message-ID: <20160219041622.GA88036@ast-mbp.thefacebook.com> (raw)
In-Reply-To: <1455830838.14359.58.camel@tzanussi-mobl.amr.corp.intel.com>
On Thu, Feb 18, 2016 at 03:27:18PM -0600, Tom Zanussi wrote:
> On Tue, 2016-02-16 at 20:51 -0800, Alexei Starovoitov wrote:
> > On Tue, Feb 16, 2016 at 04:35:27PM -0600, Tom Zanussi wrote:
> > > On Sun, 2016-02-14 at 01:02 +0100, Alexei Starovoitov wrote:
> > > > On Fri, Feb 12, 2016 at 10:11:18AM -0600, Tom Zanussi wrote:
> >
>
> # ./funccount.py '*spin*'
>
> Which on my machine resulted in a hard lockup on all CPUs. I'm not set
thanks for the report. looks like something got broken. After:
# ./funccount.par '*spin*'
Tracing 12 functions for "*spin*"... Hit Ctrl-C to end.
^C
ADDR FUNC COUNT
ffffffff810aeb91 mutex_spin_on_owner 530
ffffffff8177f241 _raw_spin_unlock_bh 1325
ffffffff810aebe1 mutex_optimistic_spin 1696
ffffffff8177f581 _raw_spin_lock_bh 1985
ffffffff8177f511 _raw_spin_trylock 55337
ffffffff8177f3c1 _raw_spin_lock_irq 787875
ffffffff8177f551 _raw_spin_lock 2211324
ffffffff8177f331 _raw_spin_lock_irqsave 3556740
ffffffff8177f1c1 __lock_text_start 3593983
[ 275.175524] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 11
it seems kprobe cleanup is racing with bpf cleanup...
> > > surrounding that even in the comments. I guess I'd have to spend a few
> > > hours reading the BPF code and the verifier even, to understand that.
> >
> > not sure what is your goal. Runtime lookup via field name is not acceptable
> > whether it's cached or not. There is no place for strcmp in the critical path.
>
> Exactly - that's why I was asking about a 'begin probe', in order to do
> the lookup once, in an non-critical path.
It is critical path. When program is called million times for_each{strcmp}
at the beginning of every program is unacceptable overhead.
In the crash above, Ctrl-C was pressed in a split second, yet bpf
already processed 2.2M + 3.5M + 3.5M events and then hung while unloading.
In the upcoming tracepoint+bpf patches the programs will have
direct access to tracepoint data without wasting time on strcmp.
The steps to do that were already outlined in the previous email.
prev parent reply other threads:[~2016-02-19 4:16 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-12 16:11 Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 01/10] tracing: Move some constants to ring_buffer.h Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 02/10] eBPF: Add BPF_PROG_TYPE_TRACE_EVENT prog type Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 03/10] tracing: Add an 'accessor' function to ftrace_event_field Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 04/10] tracing: Add trace event accessor functions Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 05/10] eBPF/tracing: Add eBPF trace event field access helpers Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 06/10] tracing: Add kprobe/uprobe support for TRACE_EVENT eBPF progs Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 07/10] tracing: Add eBPF program support to static trace events Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 08/10] samples/bpf: Add support for trace events to the eBPF loading code Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 09/10] samples/bpf: Add readcounts-by-pid example Tom Zanussi
2016-02-12 16:11 ` [RFC][PATCH 10/10] samples/bpf: Add kprobe-event-fields example Tom Zanussi
2016-02-14 0:02 ` [RFC][PATCH 00/10] Add trace event support to eBPF Alexei Starovoitov
2016-02-16 22:35 ` Tom Zanussi
2016-02-17 4:51 ` Alexei Starovoitov
2016-02-18 21:27 ` Tom Zanussi
2016-02-18 22:40 ` Daniel Borkmann
2016-02-19 4:16 ` Alexei Starovoitov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160219041622.GA88036@ast-mbp.thefacebook.com \
--to=alexei.starovoitov@gmail.com \
--cc=daniel@iogearbox.net \
--cc=linux-kernel@vger.kernel.org \
--cc=masami.hiramatsu.pt@hitachi.com \
--cc=namhyung@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tom.zanussi@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®