[PATCH] kprobes: Document how optimized kprobes are removed from module unload

[PATCH] kprobes: Document how optimized kprobes are removed from module unload

[Steven Rostedt]

From: Steven Rostedt (VMware) <rostedt@goodmis.org>

Thomas discovered a bug where the kprobe trace tests had a race
condition where the kprobe_optimizer called from a delayed work queue
that does the optimizing and "unoptimizing" of a kprobe, can try to
modify the text after it has been freed by the init code.

The kprobe trace selftest is a special case, and Thomas and myself
investigated to see if there's a chance that this could also be a bug
with module unloading, as the code is not obvious to how it handles
this. After adding lots of printks, I figured it out. Thomas suggested
that this should be commented so that others will not have to go
through this exercise again.

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 7367e0e..ac386f6 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -2183,6 +2183,12 @@ static int kprobes_module_callback(struct notifier_block *nb,
 				 * The vaddr this probe is installed will soon
 				 * be vfreed buy not synced to disk. Hence,
 				 * disarming the breakpoint isn't needed.
+				 *
+				 * Note, this will also move any optimized probes
+				 * that are pending to be removed from their
+				 * corresponding lists to the freeing_list and
+				 * will not be touched by the delayed
+				 * kprobe_optimizer work handler.
 				 */
 				kill_kprobe(p);
 			}

Re: [PATCH] kprobes: Document how optimized kprobes are removed from module unload

[Masami Hiramatsu]

On Tue, 16 May 2017 14:58:35 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> From: Steven Rostedt (VMware) <rostedt@goodmis.org>
> 
> Thomas discovered a bug where the kprobe trace tests had a race
> condition where the kprobe_optimizer called from a delayed work queue
> that does the optimizing and "unoptimizing" of a kprobe, can try to
> modify the text after it has been freed by the init code.
> 
> The kprobe trace selftest is a special case, and Thomas and myself
> investigated to see if there's a chance that this could also be a bug
> with module unloading, as the code is not obvious to how it handles
> this. After adding lots of printks, I figured it out. Thomas suggested
> that this should be commented so that others will not have to go
> through this exercise again.
> 

OK, and I prefer this comment to move into kill_kprobe() right
before calling kill_optimized_kprobe() because that actually
does it.

Thank you,

> Suggested-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> index 7367e0e..ac386f6 100644
> --- a/kernel/kprobes.c
> +++ b/kernel/kprobes.c
> @@ -2183,6 +2183,12 @@ static int kprobes_module_callback(struct notifier_block *nb,
>  				 * The vaddr this probe is installed will soon
>  				 * be vfreed buy not synced to disk. Hence,
>  				 * disarming the breakpoint isn't needed.
> +				 *
> +				 * Note, this will also move any optimized probes
> +				 * that are pending to be removed from their
> +				 * corresponding lists to the freeing_list and
> +				 * will not be touched by the delayed
> +				 * kprobe_optimizer work handler.
>  				 */
>  				kill_kprobe(p);
>  			}


-- 
Masami Hiramatsu <mhiramat@kernel.org>

Re: [PATCH] kprobes: Document how optimized kprobes are removed from module unload

[Steven Rostedt]

On Wed, 17 May 2017 10:47:07 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Tue, 16 May 2017 14:58:35 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
> > From: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > 
> > Thomas discovered a bug where the kprobe trace tests had a race
> > condition where the kprobe_optimizer called from a delayed work queue
> > that does the optimizing and "unoptimizing" of a kprobe, can try to
> > modify the text after it has been freed by the init code.
> > 
> > The kprobe trace selftest is a special case, and Thomas and myself
> > investigated to see if there's a chance that this could also be a bug
> > with module unloading, as the code is not obvious to how it handles
> > this. After adding lots of printks, I figured it out. Thomas suggested
> > that this should be commented so that others will not have to go
> > through this exercise again.
> >   
> 
> OK, and I prefer this comment to move into kill_kprobe() right
> before calling kill_optimized_kprobe() because that actually
> does it.

Actually, I can add a comment in both places. I didn't put it into
kill_kprobe() because, yes that's where it is done. I placed it in the
module handler because that's where one will look to see if module
unloading doesn't have any race conditions with the delayed handling.

-- Steve


> 
> Thank you,
> 
> > Suggested-by: Thomas Gleixner <tglx@linutronix.de>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> > diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> > index 7367e0e..ac386f6 100644
> > --- a/kernel/kprobes.c
> > +++ b/kernel/kprobes.c
> > @@ -2183,6 +2183,12 @@ static int kprobes_module_callback(struct notifier_block *nb,
> >  				 * The vaddr this probe is installed will soon
> >  				 * be vfreed buy not synced to disk. Hence,
> >  				 * disarming the breakpoint isn't needed.
> > +				 *
> > +				 * Note, this will also move any optimized probes
> > +				 * that are pending to be removed from their
> > +				 * corresponding lists to the freeing_list and
> > +				 * will not be touched by the delayed
> > +				 * kprobe_optimizer work handler.
> >  				 */
> >  				kill_kprobe(p);
> >  			}  
> 
> 

Re: [PATCH] kprobes: Document how optimized kprobes are removed from module unload

[Masami Hiramatsu]

On Tue, 16 May 2017 22:46:02 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Wed, 17 May 2017 10:47:07 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Tue, 16 May 2017 14:58:35 -0400
> > Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > > From: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > 
> > > Thomas discovered a bug where the kprobe trace tests had a race
> > > condition where the kprobe_optimizer called from a delayed work queue
> > > that does the optimizing and "unoptimizing" of a kprobe, can try to
> > > modify the text after it has been freed by the init code.
> > > 
> > > The kprobe trace selftest is a special case, and Thomas and myself
> > > investigated to see if there's a chance that this could also be a bug
> > > with module unloading, as the code is not obvious to how it handles
> > > this. After adding lots of printks, I figured it out. Thomas suggested
> > > that this should be commented so that others will not have to go
> > > through this exercise again.
> > >   
> > 
> > OK, and I prefer this comment to move into kill_kprobe() right
> > before calling kill_optimized_kprobe() because that actually
> > does it.
> 
> Actually, I can add a comment in both places. I didn't put it into
> kill_kprobe() because, yes that's where it is done. I placed it in the
> module handler because that's where one will look to see if module
> unloading doesn't have any race conditions with the delayed handling.

OK, I see.

Acked-by: Masami Hiramatsu <mhiramat@kernel.org>

Thanks,

> 
> -- Steve
> 
> 
> > 
> > Thank you,
> > 
> > > Suggested-by: Thomas Gleixner <tglx@linutronix.de>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > > diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> > > index 7367e0e..ac386f6 100644
> > > --- a/kernel/kprobes.c
> > > +++ b/kernel/kprobes.c
> > > @@ -2183,6 +2183,12 @@ static int kprobes_module_callback(struct notifier_block *nb,
> > >  				 * The vaddr this probe is installed will soon
> > >  				 * be vfreed buy not synced to disk. Hence,
> > >  				 * disarming the breakpoint isn't needed.
> > > +				 *
> > > +				 * Note, this will also move any optimized probes
> > > +				 * that are pending to be removed from their
> > > +				 * corresponding lists to the freeing_list and
> > > +				 * will not be touched by the delayed
> > > +				 * kprobe_optimizer work handler.
> > >  				 */
> > >  				kill_kprobe(p);
> > >  			}  
> > 
> > 
> 


-- 
Masami Hiramatsu <mhiramat@kernel.org>