mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Paul E. McKenney" <paulmck@kernel.org>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Phil Auld <pauld@redhat.com>, Alex Belits <abelits@marvell.com>,
	Nicolas Saenz Julienne <nsaenz@kernel.org>,
	Xiongfeng Wang <wangxiongfeng2@huawei.com>,
	Neeraj Upadhyay <quic_neeraju@quicinc.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Yu Liao <liaoyu15@huawei.com>, Boqun Feng <boqun.feng@gmail.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Paul Gortmaker <paul.gortmaker@windriver.com>,
	Uladzislau Rezki <uladzislau.rezki@sony.com>,
	Joel Fernandes <joel@joelfernandes.org>
Subject: Re: [PATCH 18/19] rcu/context_tracking: Merge dynticks counter and context tracking states
Date: Fri, 11 Mar 2022 09:28:44 -0800	[thread overview]
Message-ID: <20220311172844.GJ4285@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <20220311163525.GF227945@lothringen>

On Fri, Mar 11, 2022 at 05:35:25PM +0100, Frederic Weisbecker wrote:
> On Thu, Mar 10, 2022 at 12:32:22PM -0800, Paul E. McKenney wrote:
> > On Wed, Mar 02, 2022 at 04:48:09PM +0100, Frederic Weisbecker wrote:
> > > Updating the context tracking state and the RCU dynticks counter
> > > atomically in a single operation is a first step towards improving CPU
> > > isolation. This makes the context tracking state updates fully ordered
> > > and therefore allow for later enhancements such as postponing some work
> > > while a task is running isolated in userspace until it ever comes back
> > > to the kernel.
> > > 
> > > The state field becomes divided in two parts:
> > > 
> > > 1) Lower bits for context tracking state:
> > > 
> > >    	CONTEXT_IDLE = 1,
> > > 	CONTEXT_USER = 2,
> > > 	CONTEXT_GUEST = 4,
> > 
> > And the CONTEXT_DISABLED value of -1 works because you can have only
> > one of the above three bits set at a time?
> > 
> > Except that RCU needs this to unconditionally at least distinguish
> > between kernel and idle, given the prevalence of CONFIG_NO_HZ_IDLE=y.
> > So does the CONTEXT_DISABLED really happen anymore?
> > 
> > A few more questions interspersed below.
> 
> The value of CONTEXT_DISABLED is never stored in the ct->state. It is just
> returned as is when CONTEXT_TRACKING is disabled. So this shouldn't conflict
> with RCU.

Whew!  ;-)

> > > @@ -452,15 +453,16 @@ void noinstr __ct_user_exit(enum ctx_state state)
> > >  			 * Exit RCU idle mode while entering the kernel because it can
> > >  			 * run a RCU read side critical section anytime.
> > >  			 */
> > > -			rcu_eqs_exit(true);
> > > +			ct_kernel_enter(true, RCU_DYNTICKS_IDX - state);
> > >  			if (state == CONTEXT_USER) {
> > >  				instrumentation_begin();
> > >  				vtime_user_exit(current);
> > >  				trace_user_exit(0);
> > >  				instrumentation_end();
> > >  			}
> > > +		} else {
> > > +			atomic_sub(state, &ct->state);
> > 
> > OK, atomic_sub() got my attention.  What is going on here?  ;-)
> 
> Right :-)
> 
> So that's when context tracking user is running but RCU doesn't
> track user. This is for example when NO_HZ_FULL=n but VIRT_CPU_ACCOUNTING_GEN=y.
> 
> I might remove that standalone VIRT_CPU_ACCOUNTING_GEN=y one day but for now
> it's there.
> 
> Anyway so in this case we only want to track KERNEL <-> USER from context
> tracking POV, but we don't need the DYNTICKS_RCU_IDX part, hence the spared
> ordering.
> 
> But it still needs to be atomic because NMIs may increase DYNTICKS_RCU_IDX on
> the same field.

OK, so the idea is because NO_HZ_FULL=n, RCU doesn't care about user
space execution?

How about looking at it the other way?  Is there some reason that RCU
shouldn't take advantage of the userspace-execution information when it
exists?  For example, in the NO_HZ_FULL=n but VIRT_CPU_ACCOUNTING_GEN=y
case, is there some chance that RCU would be ignoring a non-noinstr
function?

> > > @@ -548,7 +550,7 @@ EXPORT_SYMBOL_GPL(context_tracking);
> > >  void ct_idle_enter(void)
> > >  {
> > >  	lockdep_assert_irqs_disabled();
> > > -	rcu_eqs_enter(false);
> > > +	ct_kernel_exit(false, RCU_DYNTICKS_IDX + CONTEXT_IDLE);
> > >  }
> > >  EXPORT_SYMBOL_GPL(ct_idle_enter);
> > >  
> > > @@ -566,7 +568,7 @@ void ct_idle_exit(void)
> > >  	unsigned long flags;
> > >  
> > >  	local_irq_save(flags);
> > > -	rcu_eqs_exit(false);
> > > +	ct_kernel_enter(false, RCU_DYNTICKS_IDX - CONTEXT_IDLE);
> > 
> > Nice!  This works because all transitions must be either from or
> > to kernel context, correct?
> 
> Exactly. There is no such thing as IDLE -> USER -> GUEST, etc...
> There has to be KERNEL in the middle of each. Because we never
> call rcu_idle_enter() -> rcu_user_enter() for example. The has to be
> rcu_idle_exit() in the middle.
> 
> (famous last words).

Works for me, for the moment, anyway.  ;-)

> > >  /* Return true if the specified CPU is currently idle from an RCU viewpoint.  */
> > > @@ -321,8 +321,7 @@ bool rcu_dynticks_zero_in_eqs(int cpu, int *vp)
> > >  	int snap;
> > >  
> > >  	// If not quiescent, force back to earlier extended quiescent state.
> > > -	snap = ct_dynticks_cpu(cpu) & ~0x1;
> > > -
> > > +	snap = ct_dynticks_cpu(cpu) & ~RCU_DYNTICKS_IDX;
> > 
> > Do we also need to get rid of the low-order bits?  Or is that happening
> > elsewhere?  Or is there some reason that they can stick around?
> 
> Yep, ct_dynticks_cpu() clears the low order CONTEXT_* bits.

Whew!  ;-)

> > > diff --git a/kernel/rcu/tree_stall.h b/kernel/rcu/tree_stall.h
> > > index 9bf5cc79d5eb..1ac48c804006 100644
> > > --- a/kernel/rcu/tree_stall.h
> > > +++ b/kernel/rcu/tree_stall.h
> > > @@ -459,7 +459,7 @@ static void print_cpu_stall_info(int cpu)
> > >  			rdp->rcu_iw_pending ? (int)min(delta, 9UL) + '0' :
> > >  				"!."[!delta],
> > >  	       ticks_value, ticks_title,
> > > -	       rcu_dynticks_snap(cpu) & 0xfff,
> > > +	       (rcu_dynticks_snap(cpu) >> RCU_DYNTICKS_SHIFT) & 0xfff ,
> > 
> > Actually, the low-ordder several bits are useful when debugging, so
> > could you please not shift them away?  Maybe also go to 0xffff to allow
> > for more bits taken?
> 
> Yeah that makes sense, I'll change that.
> 
> Thanks a lot for the reviews!

Thank you for the series!

							Thanx, Paul

  reply	other threads:[~2022-03-11 17:28 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-02 15:47 [PATCH 00/19] rcu/context-tracking: Merge RCU eqs-dynticks counter to context tracking Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 01/19] context_tracking: Rename __context_tracking_enter/exit() to __ct_user_enter/exit() Frederic Weisbecker
2022-03-10 19:27   ` Paul E. McKenney
2022-03-02 15:47 ` [PATCH 02/19] context_tracking: Rename context_tracking_user_enter/exit() to user_enter/exit_callable() Frederic Weisbecker
2022-03-05 13:59   ` Peter Zijlstra
2022-03-09 20:53     ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 03/19] context_tracking: Rename context_tracking_enter/exit() to ct_user_enter/exit() Frederic Weisbecker
2022-03-05 14:02   ` Peter Zijlstra
2022-03-09 21:21     ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 04/19] context_tracking: Rename context_tracking_cpu_set() to context_tracking_cpu_track_user() Frederic Weisbecker
2022-03-05 14:03   ` Peter Zijlstra
2022-03-09 21:11     ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 05/19] context_tracking: Split user tracking Kconfig Frederic Weisbecker
2022-03-10 19:43   ` Paul E. McKenney
2022-03-11 15:49     ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 06/19] context_tracking: Take idle eqs entrypoints over RCU Frederic Weisbecker
2022-03-05 14:05   ` Peter Zijlstra
2022-03-09 21:12     ` Frederic Weisbecker
2022-03-02 15:47 ` [PATCH 07/19] context_tracking: Take IRQ " Frederic Weisbecker
2022-03-10 19:46   ` Paul E. McKenney
2022-03-02 15:47 ` [PATCH 08/19] context_tracking: Take NMI " Frederic Weisbecker
2022-03-10 19:47   ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 09/19] rcu/context-tracking: Remove rcu_irq_enter/exit() Frederic Weisbecker
2022-03-05 14:16   ` Peter Zijlstra
2022-03-09 22:25     ` Frederic Weisbecker
2022-03-02 15:48 ` [PATCH 10/19] rcu/context_tracking: Move dynticks counter to context tracking Frederic Weisbecker
2022-03-10 20:00   ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 11/19] rcu/context_tracking: Move dynticks_nesting " Frederic Weisbecker
2022-03-10 20:01   ` Paul E. McKenney
2022-03-12 23:23   ` Peter Zijlstra
2022-03-02 15:48 ` [PATCH 12/19] rcu/context_tracking: Move dynticks_nmi_nesting " Frederic Weisbecker
2022-03-10 20:02   ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 13/19] rcu/context-tracking: Move deferred nocb resched " Frederic Weisbecker
2022-03-10 20:04   ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 14/19] rcu/context-tracking: Move RCU-dynticks internal functions to context_tracking Frederic Weisbecker
2022-03-10 20:07   ` Paul E. McKenney
2022-03-11 16:02     ` Frederic Weisbecker
2022-03-11 16:14       ` Paul E. McKenney
2022-03-12 23:10   ` Peter Zijlstra
2022-03-02 15:48 ` [PATCH 15/19] rcu/context-tracking: Remove unused and/or unecessary middle functions Frederic Weisbecker
2022-03-09 16:40   ` nicolas saenz julienne
2022-03-11 15:19     ` Frederic Weisbecker
2022-03-02 15:48 ` [PATCH 16/19] context_tracking: Convert state to atomic_t Frederic Weisbecker
2022-03-09 17:17   ` nicolas saenz julienne
2022-03-11 15:24     ` Frederic Weisbecker
2022-03-12 22:54   ` Peter Zijlstra
2022-03-21 13:32     ` Will Deacon
2022-03-02 15:48 ` [PATCH 17/19] rcu/context-tracking: Use accessor for dynticks counter value Frederic Weisbecker
2022-03-10 20:08   ` Paul E. McKenney
2022-03-02 15:48 ` [PATCH 18/19] rcu/context_tracking: Merge dynticks counter and context tracking states Frederic Weisbecker
2022-03-10 20:32   ` Paul E. McKenney
2022-03-11 16:35     ` Frederic Weisbecker
2022-03-11 17:28       ` Paul E. McKenney [this message]
2022-03-02 15:48 ` [PATCH 19/19] context_tracking: Exempt CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK from non-active tracking Frederic Weisbecker
2022-03-08 16:15   ` nicolas saenz julienne
2022-03-11 15:16     ` Frederic Weisbecker
2022-03-11 11:37 ` [PATCH 00/19] rcu/context-tracking: Merge RCU eqs-dynticks counter to context tracking nicolas saenz julienne

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220311172844.GJ4285@paulmck-ThinkPad-P17-Gen-1 \
    --to=paulmck@kernel.org \
    --cc=abelits@marvell.com \
    --cc=boqun.feng@gmail.com \
    --cc=frederic@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=liaoyu15@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mtosatti@redhat.com \
    --cc=nsaenz@kernel.org \
    --cc=paul.gortmaker@windriver.com \
    --cc=pauld@redhat.com \
    --cc=peterz@infradead.org \
    --cc=quic_neeraju@quicinc.com \
    --cc=tglx@linutronix.de \
    --cc=uladzislau.rezki@sony.com \
    --cc=wangxiongfeng2@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®