Re: [PATCH v2] perf/core: Avoid removing shared pmu_context on unregister

[Peter Zijlstra <peterz@infradead.org>]

On Fri, May 12, 2017 at 12:45:25PM +0100, Chris Wilson wrote:
> In commit 1fd7e4169954 ("perf/core: Remove perf_cpu_context::unique_pmu"),
> the search for another user of the pmu_cpu_context was removed, and so
> we unconditionally free it during perf_pmu_unregister. This leads to
> random corruption later and a BUG at mm/percpu.c:689.
> 
> v2: Check for shared pmu_contexts under the mutex.
> 
> Fixes: 1fd7e4169954 ("perf/core: Remove perf_cpu_context::unique_pmu")
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: David Carrillo-Cisneros <davidcc@google.com>
> Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: <stable@vger.kernel.org> # v4.11+
> ---
>  kernel/events/core.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index aaefaa27e1a6..4f60f66b35ad 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -8983,10 +8983,12 @@ EXPORT_SYMBOL_GPL(perf_pmu_register);
>  void perf_pmu_unregister(struct pmu *pmu)
>  {
>  	int remove_device;
> +	int remove_context;
>  
>  	mutex_lock(&pmus_lock);
>  	remove_device = pmu_bus_running;
>  	list_del_rcu(&pmu->entry);
> +	remove_context = !find_pmu_context(pmu->task_ctx_nr);
>  	mutex_unlock(&pmus_lock);
>  
>  	/*
> @@ -9005,7 +9007,8 @@ void perf_pmu_unregister(struct pmu *pmu)
>  		device_del(pmu->dev);
>  		put_device(pmu->dev);
>  	}
> -	free_pmu_context(pmu);
> +	if (remove_context)
> +		free_pmu_context(pmu);
>  }
>  EXPORT_SYMBOL_GPL(perf_pmu_unregister);

I was recently made aware of this patch again; which for some reason
never got resumbitted.

Looking at it I'm not at all sure it is correct.

The first clue is that only task_ctx_nr == perf_sw_context PMUs should
ever be sharing a context; which was noted in the original patch
discussion but that never made it in a comment:

  https://lkml.kernel.org/r/20170118192454.58008-3-davidcc@google.com

And the software PMUs _should_ never get unregistered. Of course it
looks like some:

  arch/powerpc/perf/imc-pmu.c
  drivers/perf/arm_spe_pmu.c

seem to do just that. But I doubt you're running with any of those
drivers active.

Aah, it looks like Will actually fixed this when he did that SPE driver,
see commit:

  df0062b27ebf ("perf/core: Avoid freeing static PMU contexts when PMU is unregistered")

Still, there is another bug there, we should not be doing idr_remove()
outside the lock.

Still, no idea what you hit and why. Or if either or both of these
patches will fix that.

---
diff --git a/kernel/events/core.c b/kernel/events/core.c
index c80549bf82c6..a7ab1d31208c 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -9425,9 +9425,7 @@ static void free_pmu_context(struct pmu *pmu)
 	if (pmu->task_ctx_nr > perf_invalid_context)
 		return;
 
-	mutex_lock(&pmus_lock);
 	free_percpu(pmu->pmu_cpu_context);
-	mutex_unlock(&pmus_lock);
 }
 
 /*
@@ -9697,6 +9695,7 @@ void perf_pmu_unregister(struct pmu *pmu)
 	synchronize_srcu(&pmus_srcu);
 	synchronize_rcu();
 
+	mutex_lock(&pmus_lock);
 	free_percpu(pmu->pmu_disable_count);
 	if (pmu->type >= PERF_TYPE_MAX)
 		idr_remove(&pmu_idr, pmu->type);
@@ -9707,6 +9706,7 @@ void perf_pmu_unregister(struct pmu *pmu)
 		put_device(pmu->dev);
 	}
 	free_pmu_context(pmu);
+	mutex_unlock(&pmus_lock);
 }
 EXPORT_SYMBOL_GPL(perf_pmu_unregister);