From: Peter Zijlstra <peterz@infradead.org>
To: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Intel Graphics Development <intel-gfx@lists.freedesktop.org>,
LKML <linux-kernel@vger.kernel.org>,
Chris Wilson <chris@chris-wilson.co.uk>,
Mika Kuoppala <mika.kuoppala@intel.com>,
Thomas Gleixner <tglx@linutronix.de>,
Marta Lofstedt <marta.lofstedt@intel.com>,
Daniel Vetter <daniel.vetter@intel.com>
Subject: Re: [PATCH 2/2] drm/i915: Use rcu instead of stop_machine in set_wedged
Date: Fri, 6 Oct 2017 13:10:39 +0200 [thread overview]
Message-ID: <20171006111039.pwp7ijxclt4fchxp@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20171006090637.25545-2-daniel.vetter@ffwll.ch>
On Fri, Oct 06, 2017 at 11:06:37AM +0200, Daniel Vetter wrote:
> I pondered whether we should annotate engine->submit_request as __rcu
> and use rcu_assign_pointer and rcu_dereference on it. But the reason
> behind those is to make sure the compiler/cpu barriers are there for
> when you have an actual data structure you point at, to make sure all
> the writes are seen correctly on the read side. But we just have a
> function pointer, and .text isn't changed, so no need for these
> barriers and hence no need for annotations.
synchronize_*() provides an smp_mb() on the calling CPU and ensures an
smp_mb() on all other CPUs before completion, such that everybody agrees
on the state prior to calling syncrhonize_rcu(). So yes, no additional
ordering requirements.
> drivers/gpu/drm/i915/i915_gem.c | 31 +++++++++--------------
> drivers/gpu/drm/i915/i915_gem_request.c | 2 ++
> drivers/gpu/drm/i915/selftests/i915_gem_request.c | 2 ++
> 3 files changed, 16 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index ab8c6946fea4..e79a6ca60265 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -3020,16 +3020,8 @@ static void nop_submit_request(struct drm_i915_gem_request *request)
> intel_engine_init_global_seqno(request->engine, request->global_seqno);
> }
>
> +static void engine_complete_requests(struct intel_engine_cs *engine)
> {
> /* Mark all executing requests as skipped */
> engine->cancel_requests(engine);
>
> @@ -3041,24 +3033,25 @@ static void engine_set_wedged(struct intel_engine_cs *engine)
> intel_engine_last_submit(engine));
> }
>
> +void i915_gem_set_wedged(struct drm_i915_private *i915)
> {
> struct intel_engine_cs *engine;
> enum intel_engine_id id;
>
> for_each_engine(engine, i915, id)
> + engine->submit_request = nop_submit_request;
>
> + /* Make sure no one is running the old callback before we proceed with
> + * cancelling requests and resetting the completion tracking. Otherwise
> + * we might submit a request to the hardware which never completes.
> + */
ARGH @ horrid comment style..
http://lkml.iu.edu/hypermail/linux/kernel/1607.1/00627.html
:-)
> + synchronize_rcu();
>
> + for_each_engine(engine, i915, id)
> + engine_complete_requests(engine);
>
> + set_bit(I915_WEDGED, &i915->gpu_error.flags);
> + wake_up_all(&i915->gpu_error.reset_queue);
> }
>
> bool i915_gem_unset_wedged(struct drm_i915_private *i915)
> diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> index b100b38f1dd2..ef78a85cb845 100644
> --- a/drivers/gpu/drm/i915/i915_gem_request.c
> +++ b/drivers/gpu/drm/i915/i915_gem_request.c
> @@ -556,7 +556,9 @@ submit_notify(struct i915_sw_fence *fence, enum i915_sw_fence_notify state)
> switch (state) {
> case FENCE_COMPLETE:
> trace_i915_gem_request_submit(request);
> + rcu_read_lock();
> request->engine->submit_request(request);
> + rcu_read_unlock();
> break;
>
> case FENCE_FREE:
> diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_request.c b/drivers/gpu/drm/i915/selftests/i915_gem_request.c
> index 78b9f811707f..a999161e8db1 100644
> --- a/drivers/gpu/drm/i915/selftests/i915_gem_request.c
> +++ b/drivers/gpu/drm/i915/selftests/i915_gem_request.c
> @@ -215,7 +215,9 @@ static int igt_request_rewind(void *arg)
> }
> i915_gem_request_get(vip);
> i915_add_request(vip);
> + rcu_read_lock();
> request->engine->submit_request(request);
> + rcu_read_unlock();
>
> mutex_unlock(&i915->drm.struct_mutex);
Yes, this is a correct and good replacement, however, you said:
> Chris said parts of the reasons for going with stop_machine() was that
> it's no overhead for the fast-path. But these callbacks use irqsave
> spinlocks and do a bunch of MMIO, and rcu_read_lock is _real_ fast.
This means that you could simply do synchronize_sched() without the
addition of rcu_read_lock()s and still be fine.
next prev parent reply other threads:[~2017-10-06 11:11 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-06 9:06 [PATCH 1/2] drm/i915: Preallocate our mmu notifier workequeu to unbreak cpu hotplug deadlock Daniel Vetter
2017-10-06 9:06 ` [PATCH 2/2] drm/i915: Use rcu instead of stop_machine in set_wedged Daniel Vetter
2017-10-06 9:17 ` Chris Wilson
2017-10-06 10:12 ` Thomas Gleixner
2017-10-06 11:12 ` Peter Zijlstra
2017-10-06 14:12 ` Daniel Vetter
2017-10-06 11:03 ` Chris Wilson
2017-10-06 14:20 ` Daniel Vetter
[not found] ` <150731094807.1676.8763340289583726886@mail.alporthouse.com>
2017-10-09 9:12 ` Daniel Vetter
[not found] ` <150731147225.1676.14663520819978219878@mail.alporthouse.com>
2017-10-09 9:26 ` Daniel Vetter
2017-10-06 11:10 ` Peter Zijlstra [this message]
2017-10-06 9:23 ` [PATCH 1/2] drm/i915: Preallocate our mmu notifier workequeu to unbreak cpu hotplug deadlock Chris Wilson
2017-10-06 9:48 ` Chris Wilson
2017-10-06 11:34 ` [Intel-gfx] " Tvrtko Ursulin
2017-10-06 14:23 ` Daniel Vetter
2017-10-06 14:44 ` Tvrtko Ursulin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171006111039.pwp7ijxclt4fchxp@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=chris@chris-wilson.co.uk \
--cc=daniel.vetter@ffwll.ch \
--cc=daniel.vetter@intel.com \
--cc=intel-gfx@lists.freedesktop.org \
--cc=linux-kernel@vger.kernel.org \
--cc=marta.lofstedt@intel.com \
--cc=mika.kuoppala@intel.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®