Re: [PATCH] rcu: Guard deferred QS on kernel exit behind need_deferred_qs() check

[Frederic Weisbecker <frederic@kernel.org>]

Le Tue, Sep 15, 2026 at 03:52:41PM +0530, Mukesh Kumar Chaurasiya (IBM) a écrit :
> ct_kernel_exit() unconditionally calls rcu_preempt_deferred_qs(current)
> on every return to userspace.

When nohz_full is active, right?

> On the common fast path nothing is
> actually deferred, so this is a needless write to
> current->rcu_read_unlock_special

When nothing is to be deferred, rcu_preempt_deferred_qs() does nothing, right?

And if some nohz_full workloads involve rare syscalls, they usually run a single
task, so no preemption that would trigger a deferred qs.


> -- a word that lives on the task_struct
> and is therefore subject to cross-CPU cache-line traffic.
> 
> On weakly-ordered architectures such as ppc64le, rcu_read_lock() and
> rcu_read_unlock() already issue lwsync/isync barriers and touch that
> same cache line in the syscall body.  Bouncing it again at syscall exit
> adds measurable overhead, particularly on workloads with a high syscall
> rate (e.g. SELinux-heavy workloads where every AVC check issues a
> system call).
> 
> Introduce rcu_ct_kernel_exit_qs() which wraps the deferred-QS call with
> a rcu_preempt_need_deferred_qs() guard, matching the pattern already
> used in rcu_flavor_sched_clock_irq():
> 
>   notrace void rcu_ct_kernel_exit_qs(void)
>   {
>           if (rcu_preempt_need_deferred_qs(current))
>                   rcu_preempt_deferred_qs(current);
>   }
> 
> The declaration is added to <linux/rcutree.h> and a stub no-op is added
> to <linux/rcutiny.h> so that TINY_RCU builds are unaffected.
> 
> ct_kernel_exit() is updated to call rcu_ct_kernel_exit_qs() in place of
> the direct rcu_preempt_deferred_qs() call.  Semantics are identical when
> a deferred QS is actually pending; only the unnecessary write on the
> fast path is eliminated.
> 
> Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
> ---
>  include/linux/rcutiny.h   |  1 +
>  include/linux/rcutree.h   |  1 +
>  kernel/context_tracking.c |  2 +-
>  kernel/rcu/tree.c         | 19 +++++++++++++++++++
>  4 files changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/rcutiny.h b/include/linux/rcutiny.h
> index e56ded733b1b..dcad641eb2c2 100644
> --- a/include/linux/rcutiny.h
> +++ b/include/linux/rcutiny.h
> @@ -120,6 +120,7 @@ static inline bool rcu_preempt_need_deferred_qs(struct task_struct *t)
>  	return false;
>  }
>  static inline void rcu_preempt_deferred_qs(struct task_struct *t) { }
> +static inline void rcu_ct_kernel_exit_qs(void) { }
>  void rcu_scheduler_starting(void);
>  static inline void rcu_end_inkernel_boot(void) { }
>  static inline bool rcu_inkernel_boot_has_ended(void) { return true; }
> diff --git a/include/linux/rcutree.h b/include/linux/rcutree.h
> index 16a04202888b..d623f2a7d3fc 100644
> --- a/include/linux/rcutree.h
> +++ b/include/linux/rcutree.h
> @@ -87,6 +87,7 @@ static inline void rcu_irq_exit_check_preempt(void) { }
>  
>  struct task_struct;
>  void rcu_preempt_deferred_qs(struct task_struct *t);
> +void rcu_ct_kernel_exit_qs(void);
>  
>  void exit_rcu(void);
>  
> diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> index a743e7ffa6c0..011018214c6d 100644
> --- a/kernel/context_tracking.c
> +++ b/kernel/context_tracking.c
> @@ -118,7 +118,7 @@ static void noinstr ct_kernel_exit(bool user, int offset)
>  	lockdep_assert_irqs_disabled();
>  	trace_rcu_watching(TPS("End"), ct_nesting(), 0, ct_rcu_watching());
>  	WARN_ON_ONCE(IS_ENABLED(CONFIG_RCU_EQS_DEBUG) && !user && !is_idle_task(current));
> -	rcu_preempt_deferred_qs(current);
> +	rcu_ct_kernel_exit_qs();
>  
>  	// instrumentation for the noinstr ct_kernel_exit_state()
>  	instrument_atomic_write(&ct->state, sizeof(ct->state));
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 96848fc1f02b..c23478f70c17 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -368,6 +368,25 @@ notrace void rcu_momentary_eqs(void)
>  }
>  EXPORT_SYMBOL_GPL(rcu_momentary_eqs);
>  
> +/**
> + * rcu_ct_kernel_exit_qs - report deferred QS on syscall/exception exit if needed
> + *
> + * Called from ct_kernel_exit() on every return to userspace.  Guards the
> + * rcu_preempt_deferred_qs() call with rcu_preempt_need_deferred_qs() so that
> + * on the common fast path -- where nothing is deferred -- we avoid the
> + * cache-line traffic on current->rcu_read_unlock_special that the unconditional
> + * call causes.  This is particularly significant on weakly-ordered architectures
> + * (e.g. ppc64le) where rcu_read_lock/unlock issue lwsync/isync barriers and
> + * already touch that cache line in the syscall body.
> + *
> + * Follows the same pattern used by rcu_flavor_sched_clock_irq().
> + */
> +notrace void rcu_ct_kernel_exit_qs(void)
> +{
> +	if (rcu_preempt_need_deferred_qs(current))
> +		rcu_preempt_deferred_qs(current);
> +}
> +

rcu_preempt_deferred_qs() already has a rcu_preempt_need_deferred_qs() fast
path. Am I missing something?

Thanks.


>  /**
>   * rcu_is_cpu_rrupt_from_idle - see if 'interrupted' from idle
>   *
> -- 
> 2.55.0
> 

-- 
Frederic Weisbecker
SUSE Labs