From: "Paul E. McKenney" <paulmck@kernel.org>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Phil Auld <pauld@redhat.com>, Alex Belits <abelits@marvell.com>,
Nicolas Saenz Julienne <nsaenz@kernel.org>,
Xiongfeng Wang <wangxiongfeng2@huawei.com>,
Neeraj Upadhyay <quic_neeraju@quicinc.com>,
Thomas Gleixner <tglx@linutronix.de>,
Yu Liao <liaoyu15@huawei.com>, Boqun Feng <boqun.feng@gmail.com>,
Marcelo Tosatti <mtosatti@redhat.com>,
Paul Gortmaker <paul.gortmaker@windriver.com>,
Uladzislau Rezki <uladzislau.rezki@sony.com>,
Joel Fernandes <joel@joelfernandes.org>
Subject: Re: [PATCH 18/19] rcu/context_tracking: Merge dynticks counter and context tracking states
Date: Fri, 11 Mar 2022 09:28:44 -0800 [thread overview]
Message-ID: <20220311172844.GJ4285@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <20220311163525.GF227945@lothringen>
On Fri, Mar 11, 2022 at 05:35:25PM +0100, Frederic Weisbecker wrote:
> On Thu, Mar 10, 2022 at 12:32:22PM -0800, Paul E. McKenney wrote:
> > On Wed, Mar 02, 2022 at 04:48:09PM +0100, Frederic Weisbecker wrote:
> > > Updating the context tracking state and the RCU dynticks counter
> > > atomically in a single operation is a first step towards improving CPU
> > > isolation. This makes the context tracking state updates fully ordered
> > > and therefore allow for later enhancements such as postponing some work
> > > while a task is running isolated in userspace until it ever comes back
> > > to the kernel.
> > >
> > > The state field becomes divided in two parts:
> > >
> > > 1) Lower bits for context tracking state:
> > >
> > > CONTEXT_IDLE = 1,
> > > CONTEXT_USER = 2,
> > > CONTEXT_GUEST = 4,
> >
> > And the CONTEXT_DISABLED value of -1 works because you can have only
> > one of the above three bits set at a time?
> >
> > Except that RCU needs this to unconditionally at least distinguish
> > between kernel and idle, given the prevalence of CONFIG_NO_HZ_IDLE=y.
> > So does the CONTEXT_DISABLED really happen anymore?
> >
> > A few more questions interspersed below.
>
> The value of CONTEXT_DISABLED is never stored in the ct->state. It is just
> returned as is when CONTEXT_TRACKING is disabled. So this shouldn't conflict
> with RCU.
Whew! ;-)
> > > @@ -452,15 +453,16 @@ void noinstr __ct_user_exit(enum ctx_state state)
> > > * Exit RCU idle mode while entering the kernel because it can
> > > * run a RCU read side critical section anytime.
> > > */
> > > - rcu_eqs_exit(true);
> > > + ct_kernel_enter(true, RCU_DYNTICKS_IDX - state);
> > > if (state == CONTEXT_USER) {
> > > instrumentation_begin();
> > > vtime_user_exit(current);
> > > trace_user_exit(0);
> > > instrumentation_end();
> > > }
> > > + } else {
> > > + atomic_sub(state, &ct->state);
> >
> > OK, atomic_sub() got my attention. What is going on here? ;-)
>
> Right :-)
>
> So that's when context tracking user is running but RCU doesn't
> track user. This is for example when NO_HZ_FULL=n but VIRT_CPU_ACCOUNTING_GEN=y.
>
> I might remove that standalone VIRT_CPU_ACCOUNTING_GEN=y one day but for now
> it's there.
>
> Anyway so in this case we only want to track KERNEL <-> USER from context
> tracking POV, but we don't need the DYNTICKS_RCU_IDX part, hence the spared
> ordering.
>
> But it still needs to be atomic because NMIs may increase DYNTICKS_RCU_IDX on
> the same field.
OK, so the idea is because NO_HZ_FULL=n, RCU doesn't care about user
space execution?
How about looking at it the other way? Is there some reason that RCU
shouldn't take advantage of the userspace-execution information when it
exists? For example, in the NO_HZ_FULL=n but VIRT_CPU_ACCOUNTING_GEN=y
case, is there some chance that RCU would be ignoring a non-noinstr
function?
> > > @@ -548,7 +550,7 @@ EXPORT_SYMBOL_GPL(context_tracking);
> > > void ct_idle_enter(void)
> > > {
> > > lockdep_assert_irqs_disabled();
> > > - rcu_eqs_enter(false);
> > > + ct_kernel_exit(false, RCU_DYNTICKS_IDX + CONTEXT_IDLE);
> > > }
> > > EXPORT_SYMBOL_GPL(ct_idle_enter);
> > >
> > > @@ -566,7 +568,7 @@ void ct_idle_exit(void)
> > > unsigned long flags;
> > >
> > > local_irq_save(flags);
> > > - rcu_eqs_exit(false);
> > > + ct_kernel_enter(false, RCU_DYNTICKS_IDX - CONTEXT_IDLE);
> >
> > Nice! This works because all transitions must be either from or
> > to kernel context, correct?
>
> Exactly. There is no such thing as IDLE -> USER -> GUEST, etc...
> There has to be KERNEL in the middle of each. Because we never
> call rcu_idle_enter() -> rcu_user_enter() for example. The has to be
> rcu_idle_exit() in the middle.
>
> (famous last words).
Works for me, for the moment, anyway. ;-)
> > > /* Return true if the specified CPU is currently idle from an RCU viewpoint. */
> > > @@ -321,8 +321,7 @@ bool rcu_dynticks_zero_in_eqs(int cpu, int *vp)
> > > int snap;
> > >
> > > // If not quiescent, force back to earlier extended quiescent state.
> > > - snap = ct_dynticks_cpu(cpu) & ~0x1;
> > > -
> > > + snap = ct_dynticks_cpu(cpu) & ~RCU_DYNTICKS_IDX;
> >
> > Do we also need to get rid of the low-order bits? Or is that happening
> > elsewhere? Or is there some reason that they can stick around?
>
> Yep, ct_dynticks_cpu() clears the low order CONTEXT_* bits.
Whew! ;-)
> > > diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
> > > index 9bf5cc79d5eb..1ac48c804006 100644
> > > --- a/kernel/rcu/tree_stall.h
> > > +++ b/kernel/rcu/tree_stall.h
> > > @@ -459,7 +459,7 @@ static void print_cpu_stall_info(int cpu)
> > > rdp->rcu_iw_pending ? (int)min(delta, 9UL) + '0' :
> > > "!."[!delta],
> > > ticks_value, ticks_title,
> > > - rcu_dynticks_snap(cpu) & 0xfff,
> > > + (rcu_dynticks_snap(cpu) >> RCU_DYNTICKS_SHIFT) & 0xfff ,
> >
> > Actually, the low-ordder several bits are useful when debugging, so
> > could you please not shift them away? Maybe also go to 0xffff to allow
> > for more bits taken?
>
> Yeah that makes sense, I'll change that.
>
> Thanks a lot for the reviews!
Thank you for the series!
Thanx, Paul
next prev parent reply other threads:[~2022-03-11 17:28 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-02 15:47 [PATCH 00/19] rcu/context-tracking: Merge RCU eqs-dynticks counter to context tracking Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 01/19] context_tracking: Rename __context_tracking_enter/exit() to __ct_user_enter/exit() Frederic Weisbecker
2022-03-10 19:27 ` Paul E. McKenney
2022-03-02 15:47 ` [PATCH 02/19] context_tracking: Rename context_tracking_user_enter/exit() to user_enter/exit_callable() Frederic Weisbecker
2022-03-05 13:59 ` Peter Zijlstra
2022-03-09 20:53 ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 03/19] context_tracking: Rename context_tracking_enter/exit() to ct_user_enter/exit() Frederic Weisbecker
2022-03-05 14:02 ` Peter Zijlstra
2022-03-09 21:21 ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 04/19] context_tracking: Rename context_tracking_cpu_set() to context_tracking_cpu_track_user() Frederic Weisbecker
2022-03-05 14:03 ` Peter Zijlstra
2022-03-09 21:11 ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 05/19] context_tracking: Split user tracking Kconfig Frederic Weisbecker
2022-03-10 19:43 ` Paul E. McKenney
2022-03-11 15:49 ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 06/19] context_tracking: Take idle eqs entrypoints over RCU Frederic Weisbecker
2022-03-05 14:05 ` Peter Zijlstra
2022-03-09 21:12 ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 07/19] context_tracking: Take IRQ " Frederic Weisbecker
2022-03-10 19:46 ` Paul E. McKenney
2022-03-02 15:47 ` [PATCH 08/19] context_tracking: Take NMI " Frederic Weisbecker
2022-03-10 19:47 ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 09/19] rcu/context-tracking: Remove rcu_irq_enter/exit() Frederic Weisbecker
2022-03-05 14:16 ` Peter Zijlstra
2022-03-09 22:25 ` Frederic Weisbecker
2022-03-02 15:48 ` [PATCH 10/19] rcu/context_tracking: Move dynticks counter to context tracking Frederic Weisbecker
2022-03-10 20:00 ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 11/19] rcu/context_tracking: Move dynticks_nesting " Frederic Weisbecker
2022-03-10 20:01 ` Paul E. McKenney
2022-03-12 23:23 ` Peter Zijlstra
2022-03-02 15:48 ` [PATCH 12/19] rcu/context_tracking: Move dynticks_nmi_nesting " Frederic Weisbecker
2022-03-10 20:02 ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 13/19] rcu/context-tracking: Move deferred nocb resched " Frederic Weisbecker
2022-03-10 20:04 ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 14/19] rcu/context-tracking: Move RCU-dynticks internal functions to context_tracking Frederic Weisbecker
2022-03-10 20:07 ` Paul E. McKenney
2022-03-11 16:02 ` Frederic Weisbecker
2022-03-11 16:14 ` Paul E. McKenney
2022-03-12 23:10 ` Peter Zijlstra
2022-03-02 15:48 ` [PATCH 15/19] rcu/context-tracking: Remove unused and/or unecessary middle functions Frederic Weisbecker
2022-03-09 16:40 ` nicolas saenz julienne
2022-03-11 15:19 ` Frederic Weisbecker
2022-03-02 15:48 ` [PATCH 16/19] context_tracking: Convert state to atomic_t Frederic Weisbecker
2022-03-09 17:17 ` nicolas saenz julienne
2022-03-11 15:24 ` Frederic Weisbecker
2022-03-12 22:54 ` Peter Zijlstra
2022-03-21 13:32 ` Will Deacon
2022-03-02 15:48 ` [PATCH 17/19] rcu/context-tracking: Use accessor for dynticks counter value Frederic Weisbecker
2022-03-10 20:08 ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 18/19] rcu/context_tracking: Merge dynticks counter and context tracking states Frederic Weisbecker
2022-03-10 20:32 ` Paul E. McKenney
2022-03-11 16:35 ` Frederic Weisbecker
2022-03-11 17:28 ` Paul E. McKenney [this message]
2022-03-02 15:48 ` [PATCH 19/19] context_tracking: Exempt CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK from non-active tracking Frederic Weisbecker
2022-03-08 16:15 ` nicolas saenz julienne
2022-03-11 15:16 ` Frederic Weisbecker
2022-03-11 11:37 ` [PATCH 00/19] rcu/context-tracking: Merge RCU eqs-dynticks counter to context tracking nicolas saenz julienne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220311172844.GJ4285@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=abelits@marvell.com \
--cc=boqun.feng@gmail.com \
--cc=frederic@kernel.org \
--cc=joel@joelfernandes.org \
--cc=liaoyu15@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mtosatti@redhat.com \
--cc=nsaenz@kernel.org \
--cc=paul.gortmaker@windriver.com \
--cc=pauld@redhat.com \
--cc=peterz@infradead.org \
--cc=quic_neeraju@quicinc.com \
--cc=tglx@linutronix.de \
--cc=uladzislau.rezki@sony.com \
--cc=wangxiongfeng2@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®