[PATCH] Fix lockup on panic with lockdep

[PATCH] Fix lockup on panic with lockdep

[Derek Basehore]

If we don't call mutex_acquire at the beginning of console_unblank, we can run
into a lockup on the logbuf_lock between console_unlock and printk during panic.
What happens in console_unlock is:

-locks logbuf_lock
-calls mutex_release
 -which calls printk
  -which locks logbuf_lock

This fixes the problem by calling console_trylock (which calls mutex_acquire)
instead of directly accessing the semaphore in console_unblank and moves the
mutex_release to after we unlock logbuf_lock in console_unlock (interrupts are
still disabled).

Signed-off-by: Derek Basehore <dbasehore@chromium.org>
---
 kernel/printk/printk.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 7228258..c599ab5 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -2084,7 +2084,6 @@ skip:
 		local_irq_restore(flags);
 	}
 	console_locked = 0;
-	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
 
 	/* Release the exclusive_console once it is used */
 	if (unlikely(exclusive_console))
@@ -2092,6 +2091,7 @@ skip:
 
 	raw_spin_unlock(&logbuf_lock);
 
+	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
 	up(&console_sem);
 
 	/*
@@ -2137,7 +2137,7 @@ void console_unblank(void)
 	 * oops_in_progress is set to 1..
 	 */
 	if (oops_in_progress) {
-		if (down_trylock(&console_sem) != 0)
+		if (!console_trylock())
 			return;
 	} else
 		console_lock();
-- 
1.9.1.423.g4596e3a

Re: [PATCH] Fix lockup on panic with lockdep

[Andrew Morton]

On Mon, 12 May 2014 17:41:54 -0700 Derek Basehore <dbasehore@chromium.org> wrote:

> If we don't call mutex_acquire at the beginning of console_unblank, we can run
> into a lockup on the logbuf_lock between console_unlock and printk during panic.
> What happens in console_unlock is:
> 
> -locks logbuf_lock
> -calls mutex_release
>  -which calls printk
>   -which locks logbuf_lock
> 
> This fixes the problem by calling console_trylock (which calls mutex_acquire)
> instead of directly accessing the semaphore in console_unblank and moves the
> mutex_release to after we unlock logbuf_lock in console_unlock (interrupts are
> still disabled).

Please take a look at linux-next, where this code has changed a lot.

> index 7228258..c599ab5 100644
> --- a/kernel/printk/printk.c
> +++ b/kernel/printk/printk.c
> @@ -2084,7 +2084,6 @@ skip:
>  		local_irq_restore(flags);
>  	}
>  	console_locked = 0;
> -	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
>  
>  	/* Release the exclusive_console once it is used */
>  	if (unlikely(exclusive_console))
> @@ -2092,6 +2091,7 @@ skip:
>  
>  	raw_spin_unlock(&logbuf_lock);
>  
> +	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
>  	up(&console_sem);
>  
>  	/*
> @@ -2137,7 +2137,7 @@ void console_unblank(void)
>  	 * oops_in_progress is set to 1..
>  	 */
>  	if (oops_in_progress) {
> -		if (down_trylock(&console_sem) != 0)
> +		if (!console_trylock())
>  			return;
>  	} else
>  		console_lock();

Although the code is quite different, an equivalent change appears to
have been made already.  I'm not sure that bugfix was intentional ;)

Re: [PATCH] Fix lockup on panic with lockdep

[Jan Kara]

On Tue 13-05-14 13:29:58, Andrew Morton wrote:
> On Mon, 12 May 2014 17:41:54 -0700 Derek Basehore <dbasehore@chromium.org> wrote:
> 
> > If we don't call mutex_acquire at the beginning of console_unblank, we can run
> > into a lockup on the logbuf_lock between console_unlock and printk during panic.
> > What happens in console_unlock is:
> > 
> > -locks logbuf_lock
> > -calls mutex_release
> >  -which calls printk
> >   -which locks logbuf_lock
> > 
> > This fixes the problem by calling console_trylock (which calls mutex_acquire)
> > instead of directly accessing the semaphore in console_unblank and moves the
> > mutex_release to after we unlock logbuf_lock in console_unlock (interrupts are
> > still disabled).
> 
> Please take a look at linux-next, where this code has changed a lot.
> 
> > index 7228258..c599ab5 100644
> > --- a/kernel/printk/printk.c
> > +++ b/kernel/printk/printk.c
> > @@ -2084,7 +2084,6 @@ skip:
> >  		local_irq_restore(flags);
> >  	}
> >  	console_locked = 0;
> > -	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
> >  
> >  	/* Release the exclusive_console once it is used */
> >  	if (unlikely(exclusive_console))
> > @@ -2092,6 +2091,7 @@ skip:
> >  
> >  	raw_spin_unlock(&logbuf_lock);
> >  
> > +	mutex_release(&console_lock_dep_map, 1, _RET_IP_);
> >  	up(&console_sem);
> >  
> >  	/*
> > @@ -2137,7 +2137,7 @@ void console_unblank(void)
> >  	 * oops_in_progress is set to 1..
> >  	 */
> >  	if (oops_in_progress) {
> > -		if (down_trylock(&console_sem) != 0)
> > +		if (!console_trylock())
> >  			return;
> >  	} else
> >  		console_lock();
> 
> Although the code is quite different, an equivalent change appears to
> have been made already.  I'm not sure that bugfix was intentional ;)
  It was intentional. It would take extra work to not fix the bug when I
was cleaning up those lockdep annotations so I decided to just fix it... ;)

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR