Re: [PATCH] tracing: use raw spinlocks instead of spinlocks

[Ingo Molnar <mingo@elte.hu>]

* Steven Rostedt <rostedt@goodmis.org> wrote:

> On Mon, 3 Nov 2008, Frederic Weisbecker wrote:
> 
> > When I tried to figure out why my experimental function's return 
> > tracer was hanging, I discovered that it was partially caused by 
> > the fact that the ring buffer might use the usual spinlocks during 
> > entry insertion.
> > 
> > ring_buffer_lock_reserve() -> rb_reserve_next_event() -> 
> > __rb_reserve_next() -> spin_lock_irqsave()
> > 
> > Since this last function is traced, the result is a recursion 
> > during the trace. I guess it happens too with the function tracer.
> > 
> > We should use the raw_spin_locks which are not traced.
> > 
> > Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
> 
> Frederic,
> 
> Ingo has been very adamant about not using raw_spin_locks in the 
> ring buffers. My original code did this, and he nacked it. The 
> reason being (and he eventually convinced me) was that by using raw, 
> we not only do not trace the locking, we also remove the lock 
> checking. This code can easily produce deadlocks, so we do not want 
> the lock checking removed.
> 
> The real fix is to find a way in your tracer to detect the 
> recursion, and be able to prevent it. Like the atomic disables I use 
> in ftrace. It does the same thing. It leaves the lockdep checking on 
> its own locks, but can also detect if the lock checking caused it to 
> recurse. When the recusion is detected, the tracer itself will not 
> trace.

i'm wondering, does the changing to raw-spinlocks fix the deadlock? 
It's generally just the lack of recursion checking that is causing 
lockdep troubles - and recursion checking we want for all the more 
intrusive ftrace plugins anyway.

Frederic, do you have trouble finding the source of the deadlock? In 
theory the NMI watchdog should be able to punch through it. (if not 
then we need to improve things so that it can)

	Ingo