From: "Paul E. McKenney" <paulmck@kernel.org>
To: Joel Fernandes <joel@joelfernandes.org>
Cc: Frederic Weisbecker <frederic@kernel.org>,
rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
rushikesh.s.kadam@intel.com, urezki@gmail.com,
neeraj.iitr10@gmail.com, rostedt@goodmis.org
Subject: Re: [PATCH v6 1/4] rcu: Make call_rcu() lazy to save power
Date: Mon, 26 Sep 2022 10:33:57 -0700 [thread overview]
Message-ID: <20220926173357.GN4196@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <YzG/hoePbBumpaBV@google.com>
On Mon, Sep 26, 2022 at 03:04:38PM +0000, Joel Fernandes wrote:
> On Mon, Sep 26, 2022 at 12:00:45AM +0200, Frederic Weisbecker wrote:
> > On Sat, Sep 24, 2022 at 09:00:39PM -0400, Joel Fernandes wrote:
> > >
> > >
> > > > On Sep 24, 2022, at 7:28 PM, Joel Fernandes <joel@joelfernandes.org> wrote:
> > > >
> > > > Hi Frederic, thanks for the response, replies
> > > > below courtesy fruit company’s device:
> > > >
> > > >>> On Sep 24, 2022, at 6:46 PM, Frederic Weisbecker <frederic@kernel.org> wrote:
> > > >>>
> > > >>> On Thu, Sep 22, 2022 at 10:01:01PM +0000, Joel Fernandes (Google) wrote:
> > > >>> @@ -3902,7 +3939,11 @@ static void rcu_barrier_entrain(struct rcu_data *rdp)
> > > >>> rdp->barrier_head.func = rcu_barrier_callback;
> > > >>> debug_rcu_head_queue(&rdp->barrier_head);
> > > >>> rcu_nocb_lock(rdp);
> > > >>> - WARN_ON_ONCE(!rcu_nocb_flush_bypass(rdp, NULL, jiffies));
> > > >>> + /*
> > > >>> + * Flush the bypass list, but also wake up the GP thread as otherwise
> > > >>> + * bypass/lazy CBs maynot be noticed, and can cause real long delays!
> > > >>> + */
> > > >>> + WARN_ON_ONCE(!rcu_nocb_flush_bypass(rdp, NULL, jiffies, FLUSH_BP_WAKE));
> > > >>
> > > >> This fixes an issue that goes beyond lazy implementation. It should be done
> > > >> in a separate patch, handling rcu_segcblist_entrain() as well, with "Fixes: " tag.
> > > >
> > > > I wanted to do that, however on discussion with
> > > > Paul I thought of making this optimization only for
> > > > all lazy bypass CBs. That makes it directly related
> > > > this patch since the laziness notion is first
> > > > introduced here. On the other hand I could make
> > > > this change in a later patch since we are not
> > > > super bisectable anyway courtesy of the last
> > > > patch (which is not really an issue if the CONFIG
> > > > is kept off during someone’s bisection.
> > >
> > > Or are we saying it’s worth doing the wake up for rcu barrier even for
> > > regular bypass CB? That’d save 2 jiffies on rcu barrier. If we agree it’s
> > > needed, then yes splitting the patch makes sense.
> > >
> > > Please let me know your opinions, thanks,
> > >
> > > - Joel
> >
> > Sure, I mean since we are fixing the buggy rcu_barrier_entrain() anyway, let's
> > just fix bypass as well. Such as in the following (untested):
>
> Got it. This sounds good to me, and will simplify the code a bit more for sure.
>
> I guess a question for Paul - are you Ok with rcu_barrier() causing wake ups
> if the bypass list has any non-lazy CBs as well? That should be OK, IMO.
In theory, I am OK with it. In practice, you are the guys with the
hardware that can measure power consumption, not me! ;-)
> > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > index b39e97175a9e..a0df964abb0e 100644
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -3834,6 +3834,8 @@ static void rcu_barrier_entrain(struct rcu_data *rdp)
> > {
> > unsigned long gseq = READ_ONCE(rcu_state.barrier_sequence);
> > unsigned long lseq = READ_ONCE(rdp->barrier_seq_snap);
> > + bool wake_nocb = false;
> > + bool was_alldone = false;
> >
> > lockdep_assert_held(&rcu_state.barrier_lock);
> > if (rcu_seq_state(lseq) || !rcu_seq_state(gseq) || rcu_seq_ctr(lseq) != rcu_seq_ctr(gseq))
> > @@ -3842,6 +3844,8 @@ static void rcu_barrier_entrain(struct rcu_data *rdp)
> > rdp->barrier_head.func = rcu_barrier_callback;
> > debug_rcu_head_queue(&rdp->barrier_head);
> > rcu_nocb_lock(rdp);
> > + if (rcu_rdp_is_offloaded(rdp) && !rcu_segcblist_pend_cbs(&rdp->cblist))
> > + was_alldone = true;
> > WARN_ON_ONCE(!rcu_nocb_flush_bypass(rdp, NULL, jiffies));
> > if (rcu_segcblist_entrain(&rdp->cblist, &rdp->barrier_head)) {
> > atomic_inc(&rcu_state.barrier_cpu_count);
> > @@ -3849,7 +3853,12 @@ static void rcu_barrier_entrain(struct rcu_data *rdp)
> > debug_rcu_head_unqueue(&rdp->barrier_head);
> > rcu_barrier_trace(TPS("IRQNQ"), -1, rcu_state.barrier_sequence);
> > }
> > + if (was_alldone && rcu_segcblist_pend_cbs(&rdp->cblist))
> > + wake_nocb = true;
> > rcu_nocb_unlock(rdp);
> > + if (wake_nocb)
> > + wake_nocb_gp(rdp, false);
> > +
>
> Thanks for the code snippet, I like how you are checking if the bypass list
> is empty, without actually checking it ;-)
That certainly is consistent with the RCU philosophy. :-)
Thanx, Paul
next prev parent reply other threads:[~2022-09-26 17:57 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-22 22:01 [PATCH v6 0/4] rcu: call_rcu() power improvements Joel Fernandes (Google)
2022-09-22 22:01 ` [PATCH v6 1/4] rcu: Make call_rcu() lazy to save power Joel Fernandes (Google)
2022-09-23 21:44 ` Paul E. McKenney
2022-09-24 16:20 ` Joel Fernandes
2022-09-24 21:11 ` Paul E. McKenney
2022-09-24 22:56 ` Joel Fernandes
2022-09-25 17:31 ` Joel Fernandes
2022-09-26 17:42 ` Paul E. McKenney
2022-09-26 21:07 ` Joel Fernandes
2022-09-26 22:37 ` Paul E. McKenney
2022-09-26 23:33 ` Joel Fernandes
2022-09-26 23:53 ` Paul E. McKenney
2022-10-03 19:33 ` Joel Fernandes
2022-10-03 19:49 ` Paul E. McKenney
2022-09-24 22:46 ` Frederic Weisbecker
2022-09-24 23:28 ` Joel Fernandes
2022-09-25 1:00 ` Joel Fernandes
2022-09-25 22:00 ` Frederic Weisbecker
2022-09-26 15:04 ` Joel Fernandes
2022-09-26 17:33 ` Paul E. McKenney [this message]
2022-09-26 23:37 ` Joel Fernandes
2022-09-25 22:09 ` Frederic Weisbecker
2022-09-26 17:45 ` Paul E. McKenney
2022-09-25 8:57 ` Uladzislau Rezki
2022-09-25 17:46 ` Joel Fernandes
2022-09-26 17:48 ` Paul E. McKenney
2022-09-26 19:32 ` Uladzislau Rezki
2022-09-26 21:02 ` Joel Fernandes
2022-09-26 22:32 ` Paul E. McKenney
2022-09-26 23:47 ` Joel Fernandes
2022-09-26 23:59 ` Paul E. McKenney
2022-09-27 1:49 ` Joel Fernandes
2022-09-27 3:22 ` Paul E. McKenney
2022-09-27 13:05 ` Joel Fernandes
2022-09-27 14:14 ` Paul E. McKenney
2022-09-27 14:22 ` Joel Fernandes
2022-09-27 14:30 ` Paul E. McKenney
2022-09-27 15:25 ` Joel Fernandes
2022-09-27 15:59 ` Paul E. McKenney
[not found] ` <CAEXW_YRpAjvmBPzRA-hRQpuaDuZUzfndLb3q+e3BUyWprg5wkQ@mail.gmail.com>
2022-09-27 3:21 ` Paul E. McKenney
2022-09-26 22:27 ` Paul E. McKenney
2022-09-26 19:39 ` Uladzislau Rezki
2022-09-26 20:54 ` Joel Fernandes
2022-09-26 22:35 ` Paul E. McKenney
2022-09-26 23:44 ` Joel Fernandes
2022-09-26 23:57 ` Paul E. McKenney
2022-09-27 1:16 ` Joel Fernandes
2022-09-27 3:20 ` Paul E. McKenney
2022-09-27 14:08 ` Uladzislau Rezki
2022-09-27 14:30 ` Joel Fernandes
2022-09-27 14:59 ` Uladzislau Rezki
2022-09-27 15:13 ` Uladzislau Rezki
2022-09-27 21:31 ` Uladzislau Rezki
2022-09-27 22:05 ` Joel Fernandes
2022-09-27 22:29 ` Joel Fernandes
2022-09-30 16:11 ` Uladzislau Rezki
2022-10-04 11:35 ` Uladzislau Rezki
2022-10-04 18:06 ` Joel Fernandes
2022-09-27 15:14 ` Joel Fernandes
2022-09-22 22:01 ` [PATCH v6 2/4] rcu: shrinker for lazy rcu Joel Fernandes (Google)
2022-09-22 22:01 ` [PATCH v6 3/4] rcuscale: Add laziness and kfree tests Joel Fernandes (Google)
2022-09-22 22:01 ` [PATCH v6 4/4] percpu-refcount: Use call_rcu_flush() for atomic switch Joel Fernandes (Google)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220926173357.GN4196@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=frederic@kernel.org \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=neeraj.iitr10@gmail.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=rushikesh.s.kadam@intel.com \
--cc=urezki@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®