From: "Paul E. McKenney" <paulmck@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Joel Fernandes <joel@joelfernandes.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-kernel@vger.kernel.org, Boqun Feng <boqun.feng@gmail.com>,
"Rafael J. Wysocki" <rjw@rjwysocki.net>
Subject: Re: RCU vs NOHZ
Date: Thu, 29 Sep 2022 08:20:44 -0700 [thread overview]
Message-ID: <20220929152044.GE4196@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <YzV5vqoLInptafJm@hirez.programming.kicks-ass.net>
On Thu, Sep 29, 2022 at 12:55:58PM +0200, Peter Zijlstra wrote:
> On Sat, Sep 17, 2022 at 07:25:08AM -0700, Paul E. McKenney wrote:
> > On Fri, Sep 16, 2022 at 11:20:14AM +0200, Peter Zijlstra wrote:
> > > On Fri, Sep 16, 2022 at 12:58:17AM -0700, Paul E. McKenney wrote:
> > >
> > > > To the best of my knowledge at this point in time, agreed. Who knows
> > > > what someone will come up with next week? But for people running certain
> > > > types of real-time and HPC workloads, context tracking really does handle
> > > > both idle and userspace transitions.
> > >
> > > Sure, but idle != nohz. Nohz is where we disable the tick, and currently
> > > RCU can inhibit this -- rcu_needs_cpu().
> >
> > Exactly. For non-nohz userspace execution, the tick is still running
> > anyway, so RCU of course won't be inhibiting its disabling. And in that
> > case, RCU's hook is the tick interrupt itself. RCU's hook is passed a
> > flag saying whether the interrupt came from userspace or from kernel.
>
> I'm not sure how we ended up here; this is completely irrelevant and I'm
> not disagreeing with it.
>
> > > AFAICT there really isn't an RCU hook for this, not through context
> > > tracking not through anything else.
> >
> > There is a directly invoked RCU hook for any transition that enables or
> > disables the tick, namely the ct_*_enter() and ct_*_exit() functions,
> > that is, those functions formerly known as rcu_*_enter() and rcu_*_exit().
>
> Context tracking doesn't know about NOHZ, therefore RCU can't either.
> Context tracking knows about IDLE, but not all IDLE is NOHZ-IDLE.
>
> Specifically we have:
>
> ct_{idle,irq,nmi,user,kernel}_enter()
>
> And none of them are related to NOHZ in the slightest. So no, RCU does
> not have a NOHZ callback.
>
> I'm still thikning you're conflating NOHZ_FULL (stopping the tick when
> in userspace) and regular NOHZ (stopping the tick when idle).
>
> > And this of course means that any additional schemes to reduce RCU's
> > power consumption must be compared (with real measurements on real
> > hardware!) to Joel et al.'s work, whether in combination or as an
> > alternative. And either way, the power savings must of course justify
> > the added code and complexity.
>
> Well, Joel's lazy scheme has the difficulty that you can wreck things by
> improperly marking the callback as lazy when there's an explicit
> dependency on it. The talk even called that out.
>
> I was hoping to construct a scheme that doesn't need the whole lazy
> approach.
>
>
> To recap; we want the CPU to go into deeper idle states, no?
>
> RCU can currently inhibit this by having callbacks pending for this CPU
> -- in this case RCU inhibits NOHZ-IDLE and deep power states are not
> selected or less effective.
>
> Now, deep idle states actually purge the caches, so cache locality
> cannot be an argument to keep the callbacks local.
>
> We know when we're doing deep idle we stop the tick.
>
> So why not, when stopping the tick, move the RCU pending crud elsewhere
> and let the CPU get on with going idle instead of inhibiting the
> stopping of the tick and wrecking deep idle?
Because doing so in the past has cost more energy than is saved.
Thanx, Paul
next prev parent reply other threads:[~2022-09-29 15:21 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-15 8:39 Peter Zijlstra
2022-09-15 15:37 ` Joel Fernandes
2022-09-15 16:06 ` Paul E. McKenney
2022-09-15 18:50 ` Peter Zijlstra
2022-09-15 19:14 ` Paul E. McKenney
2022-09-15 22:30 ` Peter Zijlstra
2022-09-16 3:40 ` Joel Fernandes
2022-09-16 7:58 ` Paul E. McKenney
2022-09-16 9:20 ` Peter Zijlstra
2022-09-16 18:11 ` Joel Fernandes
2022-09-17 13:35 ` Peter Zijlstra
2022-09-17 13:52 ` Joel Fernandes
2022-09-17 14:28 ` Paul E. McKenney
2022-09-17 14:25 ` Paul E. McKenney
2022-09-21 21:36 ` Paul E. McKenney
2022-09-23 15:12 ` Joel Fernandes
2022-09-23 16:01 ` Paul E. McKenney
2022-09-23 17:47 ` Joel Fernandes
2022-09-23 18:15 ` Paul E. McKenney
2022-09-29 11:06 ` Peter Zijlstra
2022-09-29 10:55 ` Peter Zijlstra
2022-09-29 15:20 ` Paul E. McKenney [this message]
2022-09-29 15:46 ` Paul E. McKenney
2022-09-29 16:05 ` Joel Fernandes
2022-09-29 16:23 ` Peter Zijlstra
2022-09-29 16:42 ` Paul E. McKenney
2022-09-29 19:01 ` Peter Zijlstra
2022-09-29 19:52 ` Paul E. McKenney
2022-09-29 16:18 ` Peter Zijlstra
2022-09-29 16:36 ` Paul E. McKenney
2022-09-29 19:08 ` Peter Zijlstra
2022-09-29 19:56 ` Paul E. McKenney
2022-09-30 14:48 ` Paul E. McKenney
2022-09-23 19:47 Joel Fernandes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220929152044.GE4196@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=boqun.feng@gmail.com \
--cc=fweisbec@gmail.com \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rjw@rjwysocki.net \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®