mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: Tomas Glozar <tglozar@redhat.com>
Cc: linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org,
	jkacur@redhat.com,
	"Luis Claudio R. Goncalves" <lgoncalv@redhat.com>
Subject: Re: [PATCH] tracing/timerlat: Check tlat_var for NULL in timerlat_fd_release
Date: Thu, 5 Sep 2024 08:31:39 -0400	[thread overview]
Message-ID: <20240905083139.34698d60@gandalf.local.home> (raw)
In-Reply-To: <CAP4=nvQPPd5nb3ao4tc9dCyd+fy30wX6=dziFEJ_AyaJOpnCmQ@mail.gmail.com>

On Thu, 5 Sep 2024 12:38:37 +0200
Tomas Glozar <tglozar@redhat.com> wrote:

> st 4. 9. 2024 v 16:23 odesílatel Steven Rostedt <rostedt@goodmis.org> napsal:
> >
> > When running my tests, the second one would end up deadlocking and
> > triggering lockdep. I found a way to do basically the same thing with a
> > cpumask and no added locking. I'm currently testing it and will be sending
> > out a patch later today (if it passes the tests).
> >
> > -- Steve
> >  
> 
> Oh that's unfortunate. Your cpumask patch does protect from timerlat
> trying to stop a user workload thread via kthread_stop, but I don't
> think it prevents this race in the code from the other patch:
> 
> static int timerlat_fd_release(struct inode *inode, struct file *file)
> {
>     ...
>     mutex_lock(&interface_lock);
>     <-- this can now run at the same time as tlat_var_reset(), since
> the latter is not done under interface_lock in the latest version
>     ...
> if (tlat_var->kthread)
>         <-- if tlat_var is zeroed here, we still panic on NULL dereference
> hrtimer_cancel(&tlat_var->timer);
>     ...
> }

Right, but my other patch was protecting the osn variable too, which I
don't think is necessary.

> 
> Either way, the results are much better: before, the kernel panicked
> in <10 iterations of the while loop of the reproducer; with the
> cpumask patch and the tlat_var->kthread check patch, it has been
> running on my test VM for a few hours already with no panic.

I think the hrtimer change is missing this:

diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 8543d941b870..8e611dcee68a 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -259,6 +259,9 @@ static inline void tlat_var_reset(void)
 {
 	struct timerlat_variables *tlat_var;
 	int cpu;
+
+	/* Synchronize with the timerlat interfaces */
+	mutex_lock(&interface_lock);
 	/*
 	 * So far, all the values are initialized as 0, so
 	 * zeroing the structure is perfect.
@@ -269,6 +272,7 @@ static inline void tlat_var_reset(void)
 			hrtimer_cancel(&tlat_var->timer);
 		memset(tlat_var, 0, sizeof(*tlat_var));
 	}
+	mutex_unlock(&interface_lock);
 }
 #else /* CONFIG_TIMERLAT_TRACER */
 #define tlat_var_reset()	do {} while (0)

I'll add it and start testing it. I haven't pushed to Linus yet.

-- Steve

      reply	other threads:[~2024-09-05 12:30 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-08-20 13:00 tglozar
2024-08-21 20:03 ` Steven Rostedt
2024-08-22  9:32   ` Tomas Glozar
2024-08-22 14:32     ` Steven Rostedt
2024-08-22 14:37       ` Steven Rostedt
2024-08-22 11:20   ` Luis Claudio R. Goncalves
2024-08-22 14:34     ` Steven Rostedt
2024-08-23 16:54 ` Steven Rostedt
2024-08-23 18:52   ` Steven Rostedt
2024-08-26 13:01     ` Tomas Glozar
2024-08-26 17:26       ` Steven Rostedt
2024-08-27 14:34         ` Tomas Glozar
2024-08-29 23:40           ` Luis Claudio R. Goncalves
2024-08-30  0:31             ` Luis Claudio R. Goncalves
2024-09-03 12:47           ` Tomas Glozar
2024-09-03 15:00             ` Steven Rostedt
2024-09-04 14:21             ` Steven Rostedt
2024-09-05 10:38               ` Tomas Glozar
2024-09-05 12:31                 ` Steven Rostedt [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240905083139.34698d60@gandalf.local.home \
    --to=rostedt@goodmis.org \
    --cc=jkacur@redhat.com \
    --cc=lgoncalv@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=tglozar@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®