[PATCH] ftrace/direct: Fix lockup in modify_ftrace_direct_multi

[PATCH] ftrace/direct: Fix lockup in modify_ftrace_direct_multi

[Jiri Olsa]

We can't call unregister_ftrace_function under ftrace_lock.

Fixes: ed29271894aa ("ftrace/direct: Do not disable when switching direct callers")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 kernel/trace/ftrace.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index b4ed1a301232..fc49e8809a56 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -5602,10 +5602,11 @@ int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr)
 		}
 	}
 
+	mutex_unlock(&ftrace_lock);
+
 	/* Removing the tmp_ops will add the updated direct callers to the functions */
 	unregister_ftrace_function(&tmp_ops);
 
-	mutex_unlock(&ftrace_lock);
  out_direct:
 	mutex_unlock(&direct_mutex);
 	return err;
-- 
2.32.0

Re: [PATCH] ftrace/direct: Fix lockup in modify_ftrace_direct_multi

[Steven Rostedt]

On Tue,  9 Nov 2021 12:42:17 +0100
Jiri Olsa <jolsa@redhat.com> wrote:

> We can't call unregister_ftrace_function under ftrace_lock.

Ouch. 

Do you have a sample module that triggers this?

I thought we had something that tested this code, but I don't see it in my
tree.

-- Steve

> 
> Fixes: ed29271894aa ("ftrace/direct: Do not disable when switching direct callers")
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>  kernel/trace/ftrace.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index b4ed1a301232..fc49e8809a56 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -5602,10 +5602,11 @@ int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr)
>  		}
>  	}
>  
> +	mutex_unlock(&ftrace_lock);
> +
>  	/* Removing the tmp_ops will add the updated direct callers to the functions */
>  	unregister_ftrace_function(&tmp_ops);
>  
> -	mutex_unlock(&ftrace_lock);
>   out_direct:
>  	mutex_unlock(&direct_mutex);
>  	return err;

Re: [PATCH] ftrace/direct: Fix lockup in modify_ftrace_direct_multi

[Jiri Olsa]

On Tue, Nov 09, 2021 at 11:02:37AM -0500, Steven Rostedt wrote:
> On Tue,  9 Nov 2021 12:42:17 +0100
> Jiri Olsa <jolsa@redhat.com> wrote:
> 
> > We can't call unregister_ftrace_function under ftrace_lock.
> 
> Ouch. 
> 
> Do you have a sample module that triggers this?
> 
> I thought we had something that tested this code, but I don't see it in my
> tree.

I triggered that with my bpf batch changes

there's not test for modify at the moment,
just register/unregister I'll add one

jirka

> 
> -- Steve
> 
> > 
> > Fixes: ed29271894aa ("ftrace/direct: Do not disable when switching direct callers")
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > ---
> >  kernel/trace/ftrace.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > index b4ed1a301232..fc49e8809a56 100644
> > --- a/kernel/trace/ftrace.c
> > +++ b/kernel/trace/ftrace.c
> > @@ -5602,10 +5602,11 @@ int modify_ftrace_direct_multi(struct ftrace_ops *ops, unsigned long addr)
> >  		}
> >  	}
> >  
> > +	mutex_unlock(&ftrace_lock);
> > +
> >  	/* Removing the tmp_ops will add the updated direct callers to the functions */
> >  	unregister_ftrace_function(&tmp_ops);
> >  
> > -	mutex_unlock(&ftrace_lock);
> >   out_direct:
> >  	mutex_unlock(&direct_mutex);
> >  	return err;
>