[PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 8693 bytes --]

From: David Woodhouse <dwmw@amazon.co.uk>

For a dedicated srcu_struct whose read-side critical sections are
short, atomic, and usually absent — such as one converted from a
spinning lock — the common case at synchronize time is that there are
no readers at all. Even synchronize_srcu_expedited() still costs the
caller an unconditional sleep and two trips through the SRCU workqueue
to discover that: tens of microseconds of machinery to wait for
nothing. And synchronize_srcu()'s auto-expedite heuristic explicitly
declines to expedite within exp_holdoff (25µs) of the previous grace
period, which is exactly the regime a burst of back-to-back
invalidations puts such a domain in.

Provide try_synchronize_srcu(), which proves the no-readers case
inline and returns true without sleeping, without the workqueue, and
without advancing the grace-period sequence (so cookies from
get_state_synchronize_srcu() and queued callbacks are unaffected). If
the proof fails, it returns false and the caller falls back:

	if (!try_synchronize_srcu(ssp))
		synchronize_srcu_expedited(ssp);

For Tree SRCU the proof sums both epochs' unlock counters, executes a
full barrier, then sums both epochs' lock counters. Equality proves a
moment within this function at which no reader existed: a reader
entering between the sums inflates only the lock sum (spurious
fallback, safe), and a reader whose increment is unobserved has not
yet returned from srcu_read_lock() — the barrier pairing with
__srcu_read_lock() guarantees such a reader sees every store the
caller made beforehand, so it is not a reader the caller is obliged
to wait for. Summing both epochs means no index flip is required, and
without a flip the counter-wrap concerns of
srcu_readers_active_idx_check() do not arise. Readers of the _fast()
flavors elide the read-side barrier this depends on; any sign of them
(in the unlock-side rdm mask, which is gathered unconditionally)
disqualifies the fast path.

For Tiny SRCU (!SMP) both nesting counts being zero already proves no
reader exists — a mid-section reader could only be preempted or in an
interrupt, either of which leaves its count visibly elevated — and
program order on the sole CPU provides all the required ordering.

The immediate motivation is a proposed conversion of KVM's
gfn_to_pfn_cache to use SRCU¹, where the mmu_notifier invalidation
path must drain readers of a cache (in the manner of a TLB shootdown)
before the primary MMU zaps the backing page. Those readers are short
non-sleeping fast paths, some in contexts which cannot sleep (hardirq
event channel delivery, the scheduler's sched-out hook); measurement
under a worst-case invalidation flood shows 98.8% of drains complete inline
in 4-16µs where the expedited grace period took 32-128µs, with the
wait dominated by workqueue round-trip latency, not by readers.

¹ https://lore.kernel.org/all/20260811132237.102400-1-dwmw2@infradead.org/

More potential use cases already exist in the tree with the same
no-readers-common-case profile: kvm->irq_srcu takes half a dozen
expedited grace periods in the irqfd and routing-update paths (bursts
of which, at VM boot, fall inside the exp_holdoff window), and mshv's
pt_irq_srcu is the same shape.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Assisted-by: Claude:claude-mythos-5
---
Compile-tested for Tiny SRCU (tinyconfig); the Tree version has had
the KVM gfn_to_pfn_cache conversion soaking on it under an adversarial
invalidation flood.

 include/linux/srcu.h  |  1 +
 kernel/rcu/srcutiny.c | 23 ++++++++++++++
 kernel/rcu/srcutree.c | 70 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 94 insertions(+)

diff --git a/include/linux/srcu.h b/include/linux/srcu.h
index a54ce9e808b9..0d7543c7becc 100644
--- a/include/linux/srcu.h
+++ b/include/linux/srcu.h
@@ -91,6 +91,7 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
 		void (*func)(struct rcu_head *head));
 void cleanup_srcu_struct(struct srcu_struct *ssp);
 void synchronize_srcu(struct srcu_struct *ssp);
+bool try_synchronize_srcu(struct srcu_struct *ssp);
 
 #define SRCU_GET_STATE_COMPLETED 0x1
 
diff --git a/kernel/rcu/srcutiny.c b/kernel/rcu/srcutiny.c
index a2e2d516e51b..f07159cae241 100644
--- a/kernel/rcu/srcutiny.c
+++ b/kernel/rcu/srcutiny.c
@@ -260,6 +260,29 @@ void synchronize_srcu(struct srcu_struct *ssp)
 }
 EXPORT_SYMBOL_GPL(synchronize_srcu);
 
+/**
+ * try_synchronize_srcu - inline grace period for a reader-free srcu_struct
+ * @ssp: srcu_struct with which to synchronize.
+ *
+ * If @ssp provably has no readers in either epoch, provide the
+ * synchronize_srcu() guarantee to the caller immediately, without
+ * sleeping. Returns true on success; on failure the caller must fall
+ * back to synchronize_srcu().
+ *
+ * On !SMP a reader can only be mid-critical-section if it was
+ * preempted (or is running in an interrupt which preempted us), in
+ * which case its nesting count is visibly non-zero. Both counts being
+ * zero therefore proves that no reader exists, and any reader which
+ * begins after this function returns will, by program order on this
+ * sole CPU, observe every store the caller made before calling it.
+ */
+bool try_synchronize_srcu(struct srcu_struct *ssp)
+{
+	return !READ_ONCE(ssp->srcu_lock_nesting[0]) &&
+	       !READ_ONCE(ssp->srcu_lock_nesting[1]);
+}
+EXPORT_SYMBOL_GPL(try_synchronize_srcu);
+
 /*
  * get_state_synchronize_srcu - Provide an end-of-grace-period cookie
  */
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index 2601566c254a..76a78336feaa 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -1645,6 +1645,76 @@ void synchronize_srcu(struct srcu_struct *ssp)
 }
 EXPORT_SYMBOL_GPL(synchronize_srcu);
 
+/**
+ * try_synchronize_srcu - inline grace period for a reader-free srcu_struct
+ * @ssp: srcu_struct with which to synchronize.
+ *
+ * If @ssp provably has no readers in either epoch, provide the
+ * synchronize_srcu() guarantee to the caller immediately: without
+ * sleeping, without a trip through the SRCU workqueue, and without
+ * advancing the grace-period sequence. Returns true on success; on
+ * failure the caller must fall back to synchronize_srcu() or
+ * synchronize_srcu_expedited().
+ *
+ * This serves dedicated srcu_struct structures whose read-side critical
+ * sections are short, atomic, and usually absent — where even an
+ * expedited grace period costs two trips through the workqueue and an
+ * unconditional sleep of the caller, three orders of magnitude more
+ * than the check below.
+ *
+ * Only readers of the srcu_read_lock() and srcu_read_lock_nmisafe()
+ * flavors are compatible with this proof; if the _fast() flavors have
+ * ever been used on @ssp, this function always returns false.
+ */
+bool try_synchronize_srcu(struct srcu_struct *ssp)
+{
+	unsigned long unlocks0, unlocks1;
+	unsigned long rdm0, rdm1;
+
+	check_init_srcu_struct(ssp);
+
+	/*
+	 * Order the caller's prior stores before the counter reads below.
+	 * Pairs (store-buffering pattern) with the smp_mb() in
+	 * __srcu_read_lock(): any reader whose lock increment is not
+	 * observed by the sums below is guaranteed to observe, within its
+	 * critical section, every store the caller made before calling
+	 * this function.
+	 */
+	smp_mb();
+
+	unlocks0 = srcu_readers_unlock_idx(ssp, 0, &rdm0);
+	unlocks1 = srcu_readers_unlock_idx(ssp, 1, &rdm1);
+
+	/*
+	 * Reader flavors which elide the read-side smp_mb() that the
+	 * pairing above depends on cannot be proven absent this way;
+	 * they need a real grace period.
+	 */
+	if ((rdm0 | rdm1) & SRCU_READ_FLAVOR_SLOWGP)
+		return false;
+
+	/*
+	 * As in srcu_readers_active_idx_check(): ensure that a lock is
+	 * always counted if the corresponding unlock is counted, so that
+	 * a reader racing with these sums can only inflate the lock sum
+	 * and force the (safe) fallback. Summing both epochs means no
+	 * index flip is needed: a stable equality proves there was a
+	 * moment in this function at which no readers existed at all.
+	 */
+	smp_mb();
+
+	if (!srcu_readers_lock_idx(ssp, 0, false, unlocks0))
+		return false;
+	if (!srcu_readers_lock_idx(ssp, 1, false, unlocks1))
+		return false;
+
+	/* Order the caller's subsequent accesses after the proof. */
+	smp_mb();
+	return true;
+}
+EXPORT_SYMBOL_GPL(try_synchronize_srcu);
+
 /**
  * get_state_synchronize_srcu - Provide an end-of-grace-period cookie
  * @ssp: srcu_struct to provide cookie for.
-- 
2.43.0


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 9249 bytes --]

Argh, a random syzbot fix has polluted my recipient cache; adding Sean for real.

On Wed, 2026-08-12 at 13:32 +0100, David Woodhouse wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> For a dedicated srcu_struct whose read-side critical sections are
> short, atomic, and usually absent — such as one converted from a
> spinning lock — the common case at synchronize time is that there are
> no readers at all. Even synchronize_srcu_expedited() still costs the
> caller an unconditional sleep and two trips through the SRCU workqueue
> to discover that: tens of microseconds of machinery to wait for
> nothing. And synchronize_srcu()'s auto-expedite heuristic explicitly
> declines to expedite within exp_holdoff (25µs) of the previous grace
> period, which is exactly the regime a burst of back-to-back
> invalidations puts such a domain in.
> 
> Provide try_synchronize_srcu(), which proves the no-readers case
> inline and returns true without sleeping, without the workqueue, and
> without advancing the grace-period sequence (so cookies from
> get_state_synchronize_srcu() and queued callbacks are unaffected). If
> the proof fails, it returns false and the caller falls back:
> 
> 	if (!try_synchronize_srcu(ssp))
> 		synchronize_srcu_expedited(ssp);
> 
> For Tree SRCU the proof sums both epochs' unlock counters, executes a
> full barrier, then sums both epochs' lock counters. Equality proves a
> moment within this function at which no reader existed: a reader
> entering between the sums inflates only the lock sum (spurious
> fallback, safe), and a reader whose increment is unobserved has not
> yet returned from srcu_read_lock() — the barrier pairing with
> __srcu_read_lock() guarantees such a reader sees every store the
> caller made beforehand, so it is not a reader the caller is obliged
> to wait for. Summing both epochs means no index flip is required, and
> without a flip the counter-wrap concerns of
> srcu_readers_active_idx_check() do not arise. Readers of the _fast()
> flavors elide the read-side barrier this depends on; any sign of them
> (in the unlock-side rdm mask, which is gathered unconditionally)
> disqualifies the fast path.
> 
> For Tiny SRCU (!SMP) both nesting counts being zero already proves no
> reader exists — a mid-section reader could only be preempted or in an
> interrupt, either of which leaves its count visibly elevated — and
> program order on the sole CPU provides all the required ordering.
> 
> The immediate motivation is a proposed conversion of KVM's
> gfn_to_pfn_cache to use SRCU¹, where the mmu_notifier invalidation
> path must drain readers of a cache (in the manner of a TLB shootdown)
> before the primary MMU zaps the backing page. Those readers are short
> non-sleeping fast paths, some in contexts which cannot sleep (hardirq
> event channel delivery, the scheduler's sched-out hook); measurement
> under a worst-case invalidation flood shows 98.8% of drains complete inline
> in 4-16µs where the expedited grace period took 32-128µs, with the
> wait dominated by workqueue round-trip latency, not by readers.
> 
> ¹ https://lore.kernel.org/all/20260811132237.102400-1-dwmw2@infradead.org/
> 
> More potential use cases already exist in the tree with the same
> no-readers-common-case profile: kvm->irq_srcu takes half a dozen
> expedited grace periods in the irqfd and routing-update paths (bursts
> of which, at VM boot, fall inside the exp_holdoff window), and mshv's
> pt_irq_srcu is the same shape.
> 
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Assisted-by: Claude:claude-mythos-5
> ---
> Compile-tested for Tiny SRCU (tinyconfig); the Tree version has had
> the KVM gfn_to_pfn_cache conversion soaking on it under an adversarial
> invalidation flood.
> 
>  include/linux/srcu.h  |  1 +
>  kernel/rcu/srcutiny.c | 23 ++++++++++++++
>  kernel/rcu/srcutree.c | 70 +++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 94 insertions(+)
> 
> diff --git a/include/linux/srcu.h b/include/linux/srcu.h
> index a54ce9e808b9..0d7543c7becc 100644
> --- a/include/linux/srcu.h
> +++ b/include/linux/srcu.h
> @@ -91,6 +91,7 @@ void call_srcu(struct srcu_struct *ssp, struct rcu_head *head,
>  		void (*func)(struct rcu_head *head));
>  void cleanup_srcu_struct(struct srcu_struct *ssp);
>  void synchronize_srcu(struct srcu_struct *ssp);
> +bool try_synchronize_srcu(struct srcu_struct *ssp);
>  
>  #define SRCU_GET_STATE_COMPLETED 0x1
>  
> diff --git a/kernel/rcu/srcutiny.c b/kernel/rcu/srcutiny.c
> index a2e2d516e51b..f07159cae241 100644
> --- a/kernel/rcu/srcutiny.c
> +++ b/kernel/rcu/srcutiny.c
> @@ -260,6 +260,29 @@ void synchronize_srcu(struct srcu_struct *ssp)
>  }
>  EXPORT_SYMBOL_GPL(synchronize_srcu);
>  
> +/**
> + * try_synchronize_srcu - inline grace period for a reader-free srcu_struct
> + * @ssp: srcu_struct with which to synchronize.
> + *
> + * If @ssp provably has no readers in either epoch, provide the
> + * synchronize_srcu() guarantee to the caller immediately, without
> + * sleeping. Returns true on success; on failure the caller must fall
> + * back to synchronize_srcu().
> + *
> + * On !SMP a reader can only be mid-critical-section if it was
> + * preempted (or is running in an interrupt which preempted us), in
> + * which case its nesting count is visibly non-zero. Both counts being
> + * zero therefore proves that no reader exists, and any reader which
> + * begins after this function returns will, by program order on this
> + * sole CPU, observe every store the caller made before calling it.
> + */
> +bool try_synchronize_srcu(struct srcu_struct *ssp)
> +{
> +	return !READ_ONCE(ssp->srcu_lock_nesting[0]) &&
> +	       !READ_ONCE(ssp->srcu_lock_nesting[1]);
> +}
> +EXPORT_SYMBOL_GPL(try_synchronize_srcu);
> +
>  /*
>   * get_state_synchronize_srcu - Provide an end-of-grace-period cookie
>   */
> diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
> index 2601566c254a..76a78336feaa 100644
> --- a/kernel/rcu/srcutree.c
> +++ b/kernel/rcu/srcutree.c
> @@ -1645,6 +1645,76 @@ void synchronize_srcu(struct srcu_struct *ssp)
>  }
>  EXPORT_SYMBOL_GPL(synchronize_srcu);
>  
> +/**
> + * try_synchronize_srcu - inline grace period for a reader-free srcu_struct
> + * @ssp: srcu_struct with which to synchronize.
> + *
> + * If @ssp provably has no readers in either epoch, provide the
> + * synchronize_srcu() guarantee to the caller immediately: without
> + * sleeping, without a trip through the SRCU workqueue, and without
> + * advancing the grace-period sequence. Returns true on success; on
> + * failure the caller must fall back to synchronize_srcu() or
> + * synchronize_srcu_expedited().
> + *
> + * This serves dedicated srcu_struct structures whose read-side critical
> + * sections are short, atomic, and usually absent — where even an
> + * expedited grace period costs two trips through the workqueue and an
> + * unconditional sleep of the caller, three orders of magnitude more
> + * than the check below.
> + *
> + * Only readers of the srcu_read_lock() and srcu_read_lock_nmisafe()
> + * flavors are compatible with this proof; if the _fast() flavors have
> + * ever been used on @ssp, this function always returns false.
> + */
> +bool try_synchronize_srcu(struct srcu_struct *ssp)
> +{
> +	unsigned long unlocks0, unlocks1;
> +	unsigned long rdm0, rdm1;
> +
> +	check_init_srcu_struct(ssp);
> +
> +	/*
> +	 * Order the caller's prior stores before the counter reads below.
> +	 * Pairs (store-buffering pattern) with the smp_mb() in
> +	 * __srcu_read_lock(): any reader whose lock increment is not
> +	 * observed by the sums below is guaranteed to observe, within its
> +	 * critical section, every store the caller made before calling
> +	 * this function.
> +	 */
> +	smp_mb();
> +
> +	unlocks0 = srcu_readers_unlock_idx(ssp, 0, &rdm0);
> +	unlocks1 = srcu_readers_unlock_idx(ssp, 1, &rdm1);
> +
> +	/*
> +	 * Reader flavors which elide the read-side smp_mb() that the
> +	 * pairing above depends on cannot be proven absent this way;
> +	 * they need a real grace period.
> +	 */
> +	if ((rdm0 | rdm1) & SRCU_READ_FLAVOR_SLOWGP)
> +		return false;
> +
> +	/*
> +	 * As in srcu_readers_active_idx_check(): ensure that a lock is
> +	 * always counted if the corresponding unlock is counted, so that
> +	 * a reader racing with these sums can only inflate the lock sum
> +	 * and force the (safe) fallback. Summing both epochs means no
> +	 * index flip is needed: a stable equality proves there was a
> +	 * moment in this function at which no readers existed at all.
> +	 */
> +	smp_mb();
> +
> +	if (!srcu_readers_lock_idx(ssp, 0, false, unlocks0))
> +		return false;
> +	if (!srcu_readers_lock_idx(ssp, 1, false, unlocks1))
> +		return false;
> +
> +	/* Order the caller's subsequent accesses after the proof. */
> +	smp_mb();
> +	return true;
> +}
> +EXPORT_SYMBOL_GPL(try_synchronize_srcu);
> +
>  /**
>   * get_state_synchronize_srcu - Provide an end-of-grace-period cookie
>   * @ssp: srcu_struct to provide cookie for.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[Kunwu Chan]

Hi David,

Thanks for the patch.

On Wed, 12 Aug 2026 13:32:48 +0100 David Woodhouse <dwmw2@infradead.org> wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> For a dedicated srcu_struct whose read-side critical sections are
> short, atomic, and usually absent — such as one converted from a
> spinning lock — the common case at synchronize time is that there are
> no readers at all. Even synchronize_srcu_expedited() still costs the
> caller an unconditional sleep and two trips through the SRCU workqueue

If the common case is that there are no readers and the new fast path
is intended to avoid the SRCU grace-period machinery, could you explain
what property of SRCU is still required by the proposed users that
prevents using ordinary RCU instead?

> And synchronize_srcu()'s auto-expedite heuristic explicitly
> declines to expedite within exp_holdoff (25µs) of the previous grace
> period, which is exactly the regime a burst of back-to-back
> invalidations puts such a domain in.
> 

Could this use case instead be addressed by making exp_holdoff
configurable per srcu_struct, or otherwise allowing a dedicated SRCU
domain to use a more aggressive expedite policy?

If not, what makes the proposed try path preferable to changing the
expedite policy?

> Provide try_synchronize_srcu(), which proves the no-readers case
> inline and returns true without sleeping, without the workqueue, and
> without advancing the grace-period sequence (so cookies from
> get_state_synchronize_srcu() and queued callbacks are unaffected). If

Could you also clarify why observing no readers during this
check is sufficient for the reclamation/happens-before guarantee
required by the proposed callers?

Since a successful try_synchronize_srcu() does not advance srcu_gp_seq, 
flip the SRCU index, or process callbacks, could we document more 
explicitly that a successful try provides the required 
reclamation/happens-before guarantee to the caller, but does not 
constitute an SRCU grace-period completion for the state/callback APIs?


> the proof fails, it returns false and the caller falls back:
> 
> 	if (!try_synchronize_srcu(ssp))
> 		synchronize_srcu_expedited(ssp);
> 
> For Tree SRCU the proof sums both epochs' unlock counters, executes a
> full barrier, then sums both epochs' lock counters. Equality proves a

One question about the barriers: the existing
srcu_readers_active_idx_check() relies on the barrier between the
unlock-counter and lock-counter reads to provide the ordering with the
reader-side barrier; the !did_gp path there has no barrier before the
unlock-counter read.
 
[...]
> +	smp_mb();
> +
> +	unlocks0 = srcu_readers_unlock_idx(ssp, 0, &rdm0);
> +	unlocks1 = srcu_readers_unlock_idx(ssp, 1, &rdm1);
> +
> +	/*
> +	 * Reader flavors which elide the read-side smp_mb() that the
> +	 * pairing above depends on cannot be proven absent this way;
> +	 * they need a real grace period.
> +	 */
> +	if ((rdm0 | rdm1) & SRCU_READ_FLAVOR_SLOWGP)
> +		return false;
> +
> +	/*
> +	 * As in srcu_readers_active_idx_check(): ensure that a lock is
> +	 * always counted if the corresponding unlock is counted, so that
> +	 * a reader racing with these sums can only inflate the lock sum
> +	 * and force the (safe) fallback. Summing both epochs means no
> +	 * index flip is needed: a stable equality proves there was a
> +	 * moment in this function at which no readers existed at all.
> +	 */
> +	smp_mb();

And, the middle smp_mb() appears to occupy the same position, while
the final smp_mb() orders the caller's subsequent accesses after a
successful proof.

What ordering edge does the first smp_mb() establish that is not
already provided by the middle and final barriers? I couldn't identify
a specific execution requiring it from the current comment or code
structure.

Could you point out the ordering requirement for the first smp_mb(),
or provide an LKMM argument showing why it is needed?

If there is no additional ordering requirement, could it be removed to
avoid an extra full memory barrier in this fast path?

> +
> +	if (!srcu_readers_lock_idx(ssp, 0, false, unlocks0))
> +		return false;
> +	if (!srcu_readers_lock_idx(ssp, 1, false, unlocks1))
> +		return false;
> +
> +	/* Order the caller's subsequent accesses after the proof. */
> +	smp_mb();
> +	return true;
> +}
> +EXPORT_SYMBOL_GPL(try_synchronize_srcu);

One more question about the documented fallback: synchronize_srcu()
and __synchronize_srcu() use
RCU_LOCKDEP_WARN(lockdep_is_held(ssp) || ...) to catch invocation from
within a same-type SRCU read-side critical section.

try_synchronize_srcu() does not appear to perform the corresponding
check. A caller inside an SRCU read-side critical section would fail
the fast-path proof and then follow the documented expedited fallback,
resulting in a self-deadlock without the corresponding lockdep diagnostic.

Would it make sense to perform the same lockdep check in the try path?

When applying this to current rcu/dev, the patch also needs to account
for the additional bool is_atomic argument to
check_init_srcu_struct(). Please rebase accordingly.

Thanks,
KunWu
> +
>  /**
>   * get_state_synchronize_srcu - Provide an end-of-grace-period cookie
>   * @ssp: srcu_struct to provide cookie for.
> -- 
> 2.43.0
> 
> 

Sent using hkml (https://github.com/sjp38/hackermail)

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 4369 bytes --]

On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> If the common case is that there are no readers and the new fast path
> is intended to avoid the SRCU grace-period machinery, could you explain
> what property of SRCU is still required by the proposed users that
> prevents using ordinary RCU instead?

My original version did use RCU, but Sean asked me to switch to a
dedicated per-vm SRCU¹ because a system-wide grace period doesn't work
well on large KVM hosting systems.

¹ https://lore.kernel.org/all/anS8C4Cxqy6ehCo3@google.com/

> Could this use case instead be addressed by making exp_holdoff
> configurable per srcu_struct, or otherwise allowing a dedicated SRCU
> domain to use a more aggressive expedite policy?

That doesn't help much; the other concern is latency. And the mere act
of bouncing to the workqueue only to realise we can wake up again was
adding 100µs or so every time for nothing. *Plus* once you schedule at
all, you start to see really high tail latencies of >20ms too — which
is why we've also started looking at synchronize_srcu_atomic() to
eliminate that completely. If we get *that*, then maybe I don't
personally care as much about try_synchronize_srcu(), although it might
be a useful extension to synchronize_srcu_expedited() — this *would* be
the 'more aggressive expedite policy' of which you speak, for the case
where there are no readers.

> Could you also clarify why observing no readers during this
> check is sufficient for the reclamation/happens-before guarantee
> required by the proposed callers?

Both halves of the grace-period guarantee are provided when it returns
true. For any reader, either:

   • its lock increment was observed by the sums and its unlock was
     too — the reader's final smp_mb() then orders its whole critical
     section before the caller's subsequent accesses (that reader has
     completed), or

   • its lock increment was not observed — in which case the
     store-buffering pairing with the smp_mb() in __srcu_read_lock()
     guarantees that reader observes every store the caller made
     before calling: it is a "new" reader for the purposes of the
     grace period, exactly as if it had started after a real
     synchronize_srcu() returned, or

   • its lock increment was observed but its unlock was not — the one
     case where the reader may still be running with no guarantee it
     saw anything. But that reader inflates the lock sum above the
     unlock sum, so this is precisely the case in which we return
     false and the caller takes the real grace period.

It's the same argument srcu_readers_active_idx_check() makes; the only
novelty is summing both epochs so no flip is needed.

> Since a successful try_synchronize_srcu() does not advance srcu_gp_seq,
> flip the SRCU index, or process callbacks, could we document more
> explicitly that a successful try provides the required
> reclamation/happens-before guarantee to the caller, but does not
> constitute an SRCU grace-period completion for the state/callback APIs?

Yes; done in v2.

> What ordering edge does the first smp_mb() establish that is not
> already provided by the middle and final barriers?

Er.... none? :)

We need the caller's prior stores ordered before the *lock*-counter
reads, but the middle barrier already does that for us. Dropped in v2,
and the comment on the middle barrier updated.

> Would it make sense to perform the same lockdep check in the try path?

Ack.

> When applying this to current rcu/dev, the patch also needs to account
> for the additional bool is_atomic argument to
> check_init_srcu_struct(). Please rebase accordingly.

Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
should retract my 'if we get that' above... but let's see how it
survives.

I also have more than just this patch to rebase onto rcu/dev — unlike
some, I'm not just hacking RCU for the fun of it; I get my jollies
elsewhere :)

Btw, the current commit c68271c3f83a message says "no index flip, no
grace-period sequence update" but that isn't true of the implementation
any more, is it? In fact, it *always* does those things, and doesn't
take the try-path optimisation by counting both epochs, that I'm doing
here?

Thanks for the careful review.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[KunWu Chan]

Hi David,

Thanks for the detailed explanations and for addressing the review
comments in v2.

On Fri, Aug 21, 2026 at 7:53 PM David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> > If the common case is that there are no readers and the new fast path
> > is intended to avoid the SRCU grace-period machinery, could you explain
> > what property of SRCU is still required by the proposed users that
> > prevents using ordinary RCU instead?
>
> My original version did use RCU, but Sean asked me to switch to a
> dedicated per-vm SRCU¹ because a system-wide grace period doesn't work
> well on large KVM hosting systems.

The rationale for the per-VM SRCU and the no-reader fast path is much
clearer now. In particular, I understand that the per-VM SRCU is mainly
about having a dedicated synchronization domain, while the try path
avoids the unnecessary workqueue and scheduling latency in the
no-reader case.

>
> ¹ https://lore.kernel.org/all/anS8C4Cxqy6ehCo3@google.com/
>
> > Could this use case instead be addressed by making exp_holdoff
> > configurable per srcu_struct, or otherwise allowing a dedicated SRCU
> > domain to use a more aggressive expedite policy?
>
> That doesn't help much; the other concern is latency. And the mere act
> of bouncing to the workqueue only to realise we can wake up again was
> adding 100µs or so every time for nothing. *Plus* once you schedule at
> all, you start to see really high tail latencies of >20ms too — which
> is why we've also started looking at synchronize_srcu_atomic() to
> eliminate that completely. If we get *that*, then maybe I don't
> personally care as much about try_synchronize_srcu(), although it might
> be a useful extension to synchronize_srcu_expedited() — this *would* be
> the 'more aggressive expedite policy' of which you speak, for the case
> where there are no readers.
>

In particular, tuning exp_holdoff would not address the workqueue round-trip
and the resulting tail latency.

> > Could you also clarify why observing no readers during this
> > check is sufficient for the reclamation/happens-before guarantee
> > required by the proposed callers?
>
> Both halves of the grace-period guarantee are provided when it returns
> true. For any reader, either:
>
>    • its lock increment was observed by the sums and its unlock was
>      too — the reader's final smp_mb() then orders its whole critical
>      section before the caller's subsequent accesses (that reader has
>      completed), or
>
>    • its lock increment was not observed — in which case the
>      store-buffering pairing with the smp_mb() in __srcu_read_lock()
>      guarantees that reader observes every store the caller made
>      before calling: it is a "new" reader for the purposes of the
>      grace period, exactly as if it had started after a real
>      synchronize_srcu() returned, or
>
>    • its lock increment was observed but its unlock was not — the one
>      case where the reader may still be running with no guarantee it
>      saw anything. But that reader inflates the lock sum above the
>      unlock sum, so this is precisely the case in which we return
>      false and the caller takes the real grace period.
>
> It's the same argument srcu_readers_active_idx_check() makes; the only
> novelty is summing both epochs so no flip is needed.
>
> > Since a successful try_synchronize_srcu() does not advance srcu_gp_seq,
> > flip the SRCU index, or process callbacks, could we document more
> > explicitly that a successful try provides the required
> > reclamation/happens-before guarantee to the caller, but does not
> > constitute an SRCU grace-period completion for the state/callback APIs?
>
> Yes; done in v2.
>
> > What ordering edge does the first smp_mb() establish that is not
> > already provided by the middle and final barriers?
>
> Er.... none? :)
>
> We need the caller's prior stores ordered before the *lock*-counter
> reads, but the middle barrier already does that for us. Dropped in v2,
> and the comment on the middle barrier updated.
>
> > Would it make sense to perform the same lockdep check in the try path?
>
> Ack.
>
> > When applying this to current rcu/dev, the patch also needs to account
> > for the additional bool is_atomic argument to
> > check_init_srcu_struct(). Please rebase accordingly.
>
> Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> should retract my 'if we get that' above... but let's see how it
> survives.
>
> I also have more than just this patch to rebase onto rcu/dev — unlike
> some, I'm not just hacking RCU for the fun of it; I get my jollies
> elsewhere :)
>
> Btw, the current commit c68271c3f83a message says "no index flip, no
> grace-period sequence update" but that isn't true of the implementation
> any more, is it? In fact, it *always* does those things, and doesn't
> take the try-path optimisation by counting both epochs, that I'm doing
> here?
>

The v2 changes address the points I raised, including removing the
redundant leading smp_mb(), adding the lockdep check, and clarifying
that a successful try does not constitute an SRCU grace-period
completion for the state/callback APIs.

I'll take another look at v2, in particular the two-epoch counter proof
and its correspondence with srcu_readers_active_idx_check().

Thanks again for the detailed explanations.

Thanks,
Kunwu

> Thanks for the careful review.

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[Paul E. McKenney]

On Fri, Aug 21, 2026 at 12:53:22PM +0100, David Woodhouse wrote:
> On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:

[ . . . ]

> > When applying this to current rcu/dev, the patch also needs to account
> > for the additional bool is_atomic argument to
> > check_init_srcu_struct(). Please rebase accordingly.
> 
> Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> should retract my 'if we get that' above... but let's see how it
> survives.

It is starting to shape up.  Here is what is left, that I know of, anyway:

o	A bit more work to get rcutorture to put up with atomic SRCU.

o	Tree synchronize_srcu_atomic() fastpath.

o	Disable preemption.

o	Reorganize commits to fix the Tiny hunks that got applied
	in Tree commits, and likely other similar issues.

o	More work on error checking for use of banned primitives
	with atomic SRCU.

o	Update documentation.

Not that it matters, but atomic Tiny SRCU seems to be doing quite well.

> I also have more than just this patch to rebase onto rcu/dev — unlike
> some, I'm not just hacking RCU for the fun of it; I get my jollies
> elsewhere :)
> 
> Btw, the current commit c68271c3f83a message says "no index flip, no
> grace-period sequence update" but that isn't true of the implementation
> any more, is it? In fact, it *always* does those things, and doesn't
> take the try-path optimisation by counting both epochs, that I'm doing
> here?

The current "dev" branch is here:

dc5bf5c8fb26 ("fixup! srcutree: Add an atomic Tree SRCU").

Tree SRCU always does an index flip, though a single index flip can
serve multiple concurrent calls to synchronze_srcu_atomic().

Tiny SRCU has the fast path, but must advance the index anyway for the
benefit of get_state_synchronize_srcu() and friends.  It just bumps it
in one shot.

Tree SRCU will also have to bump the index, but will be able to
do so manually, without doing all the grace-period processing.

> Thanks for the careful review.

What David said!  ;-)

							Thanx, Paul

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[Kunwu Chan]

Hi Paul,

I'd be interested in helping with one of these. Are there any items
that are not already being worked on and would be useful for me to
take on? I'm particularly interested in the Tree
`synchronize_srcu_atomic()` fastpath, if that part is still open.

Thanks,
Kunwu


August 26, 2026 at 7:49 AM, "Paul E. McKenney" <paulmck@kernel.org mailto:paulmck@kernel.org?to=%22Paul%20E.%20McKenney%22%20%3Cpaulmck%40kernel.org%3E > wrote:


> 
> On Fri, Aug 21, 2026 at 12:53:22PM +0100, David Woodhouse wrote:
> 
> > 
> > On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> > 
> [ . . . ]
> 
> > 
> > When applying this to current rcu/dev, the patch also needs to account
> >  for the additional bool is_atomic argument to
> >  check_init_srcu_struct(). Please rebase accordingly.
> >  
> >  Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> >  should retract my 'if we get that' above... but let's see how it
> >  survives.
> > 
> It is starting to shape up. Here is what is left, that I know of, anyway:
> 
> o A bit more work to get rcutorture to put up with atomic SRCU.
> 
> o Tree synchronize_srcu_atomic() fastpath.
> 
> o Disable preemption.
> 
> o Reorganize commits to fix the Tiny hunks that got applied
>  in Tree commits, and likely other similar issues.
> 
> o More work on error checking for use of banned primitives
>  with atomic SRCU.
> 
> o Update documentation.
> 
> Not that it matters, but atomic Tiny SRCU seems to be doing quite well.
> 
> > 
> > I also have more than just this patch to rebase onto rcu/dev — unlike
> >  some, I'm not just hacking RCU for the fun of it; I get my jollies
> >  elsewhere :)
> >  
> >  Btw, the current commit c68271c3f83a message says "no index flip, no
> >  grace-period sequence update" but that isn't true of the implementation
> >  any more, is it? In fact, it *always* does those things, and doesn't
> >  take the try-path optimisation by counting both epochs, that I'm doing
> >  here?
> > 
> The current "dev" branch is here:
> 
> dc5bf5c8fb26 ("fixup! srcutree: Add an atomic Tree SRCU").
> 
> Tree SRCU always does an index flip, though a single index flip can
> serve multiple concurrent calls to synchronze_srcu_atomic().
> 
> Tiny SRCU has the fast path, but must advance the index anyway for the
> benefit of get_state_synchronize_srcu() and friends. It just bumps it
> in one shot.
> 
> Tree SRCU will also have to bump the index, but will be able to
> do so manually, without doing all the grace-period processing.
> 
> > 
> > Thanks for the careful review.
> > 
> What David said! ;-)
> 
>  Thanx, Paul
>

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[Paul E. McKenney]

On Wed, Aug 26, 2026 at 03:26:18AM +0000, Kunwu Chan wrote:
> Hi Paul,
> 
> I'd be interested in helping with one of these. Are there any items
> that are not already being worked on and would be useful for me to
> take on? I'm particularly interested in the Tree
> `synchronize_srcu_atomic()` fastpath, if that part is still open.

Please feel free to give it a shot.  The similar fastpath in Tiny SRCU
is a partial guide.  The trick is that get_state_synchronize_srcu()
and poll_state_synchronize_srcu() need to keep working.

The commit that I just pushed to -rcu on branch "dev" seems to have a
working rcutorture.  Seems.  ;-)

							Thanx, Paul

> Thanks,
> Kunwu
> 
> 
> August 26, 2026 at 7:49 AM, "Paul E. McKenney" <paulmck@kernel.org mailto:paulmck@kernel.org?to=%22Paul%20E.%20McKenney%22%20%3Cpaulmck%40kernel.org%3E > wrote:
> 
> 
> > 
> > On Fri, Aug 21, 2026 at 12:53:22PM +0100, David Woodhouse wrote:
> > 
> > > 
> > > On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> > > 
> > [ . . . ]
> > 
> > > 
> > > When applying this to current rcu/dev, the patch also needs to account
> > >  for the additional bool is_atomic argument to
> > >  check_init_srcu_struct(). Please rebase accordingly.
> > >  
> > >  Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> > >  should retract my 'if we get that' above... but let's see how it
> > >  survives.
> > > 
> > It is starting to shape up. Here is what is left, that I know of, anyway:
> > 
> > o A bit more work to get rcutorture to put up with atomic SRCU.
> > 
> > o Tree synchronize_srcu_atomic() fastpath.
> > 
> > o Disable preemption.
> > 
> > o Reorganize commits to fix the Tiny hunks that got applied
> >  in Tree commits, and likely other similar issues.
> > 
> > o More work on error checking for use of banned primitives
> >  with atomic SRCU.
> > 
> > o Update documentation.
> > 
> > Not that it matters, but atomic Tiny SRCU seems to be doing quite well.
> > 
> > > 
> > > I also have more than just this patch to rebase onto rcu/dev — unlike
> > >  some, I'm not just hacking RCU for the fun of it; I get my jollies
> > >  elsewhere :)
> > >  
> > >  Btw, the current commit c68271c3f83a message says "no index flip, no
> > >  grace-period sequence update" but that isn't true of the implementation
> > >  any more, is it? In fact, it *always* does those things, and doesn't
> > >  take the try-path optimisation by counting both epochs, that I'm doing
> > >  here?
> > > 
> > The current "dev" branch is here:
> > 
> > dc5bf5c8fb26 ("fixup! srcutree: Add an atomic Tree SRCU").
> > 
> > Tree SRCU always does an index flip, though a single index flip can
> > serve multiple concurrent calls to synchronze_srcu_atomic().
> > 
> > Tiny SRCU has the fast path, but must advance the index anyway for the
> > benefit of get_state_synchronize_srcu() and friends. It just bumps it
> > in one shot.
> > 
> > Tree SRCU will also have to bump the index, but will be able to
> > do so manually, without doing all the grace-period processing.
> > 
> > > 
> > > Thanks for the careful review.
> > > 
> > What David said! ;-)
> > 
> >  Thanx, Paul
> >

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[KunWu Chan]

On Wed, Aug 26, 2026 at 11:53 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> On Wed, Aug 26, 2026 at 03:26:18AM +0000, Kunwu Chan wrote:
> > Hi Paul,
> >
> > I'd be interested in helping with one of these. Are there any items
> > that are not already being worked on and would be useful for me to
> > take on? I'm particularly interested in the Tree
> > `synchronize_srcu_atomic()` fastpath, if that part is still open.
>
> Please feel free to give it a shot.  The similar fastpath in Tiny SRCU
> is a partial guide.  The trick is that get_state_synchronize_srcu()
> and poll_state_synchronize_srcu() need to keep working.
>
> The commit that I just pushed to -rcu on branch "dev" seems to have a
> working rcutorture.  Seems.  ;-)
>
>                                                         Thanx, Paul

Thanks Paul,

I'll start by looking at the Tiny fastpath and how the state-based
synchronization APIs depend on the SRCU index, then work through
the corresponding Tree fastpath.

Thanks,
Kunwu


> > Thanks,
> > Kunwu
> >
> >
> > August 26, 2026 at 7:49 AM, "Paul E. McKenney" <paulmck@kernel.org mailto:paulmck@kernel.org?to=%22Paul%20E.%20McKenney%22%20%3Cpaulmck%40kernel.org%3E > wrote:
> >
> >
> > >
> > > On Fri, Aug 21, 2026 at 12:53:22PM +0100, David Woodhouse wrote:
> > >
> > > >
> > > > On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> > > >
> > > [ . . . ]
> > >
> > > >
> > > > When applying this to current rcu/dev, the patch also needs to account
> > > >  for the additional bool is_atomic argument to
> > > >  check_init_srcu_struct(). Please rebase accordingly.
> > > >
> > > >  Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> > > >  should retract my 'if we get that' above... but let's see how it
> > > >  survives.
> > > >
> > > It is starting to shape up. Here is what is left, that I know of, anyway:
> > >
> > > o A bit more work to get rcutorture to put up with atomic SRCU.
> > >
> > > o Tree synchronize_srcu_atomic() fastpath.
> > >
> > > o Disable preemption.
> > >
> > > o Reorganize commits to fix the Tiny hunks that got applied
> > >  in Tree commits, and likely other similar issues.
> > >
> > > o More work on error checking for use of banned primitives
> > >  with atomic SRCU.
> > >
> > > o Update documentation.
> > >
> > > Not that it matters, but atomic Tiny SRCU seems to be doing quite well.
> > >
> > > >
> > > > I also have more than just this patch to rebase onto rcu/dev — unlike
> > > >  some, I'm not just hacking RCU for the fun of it; I get my jollies
> > > >  elsewhere :)
> > > >
> > > >  Btw, the current commit c68271c3f83a message says "no index flip, no
> > > >  grace-period sequence update" but that isn't true of the implementation
> > > >  any more, is it? In fact, it *always* does those things, and doesn't
> > > >  take the try-path optimisation by counting both epochs, that I'm doing
> > > >  here?
> > > >
> > > The current "dev" branch is here:
> > >
> > > dc5bf5c8fb26 ("fixup! srcutree: Add an atomic Tree SRCU").
> > >
> > > Tree SRCU always does an index flip, though a single index flip can
> > > serve multiple concurrent calls to synchronze_srcu_atomic().
> > >
> > > Tiny SRCU has the fast path, but must advance the index anyway for the
> > > benefit of get_state_synchronize_srcu() and friends. It just bumps it
> > > in one shot.
> > >
> > > Tree SRCU will also have to bump the index, but will be able to
> > > do so manually, without doing all the grace-period processing.
> > >
> > > >
> > > > Thanks for the careful review.
> > > >
> > > What David said! ;-)
> > >
> > >  Thanx, Paul
> > >

Re: [PATCH] srcu: Add try_synchronize_srcu() for callers which can prove readers absent

[Paul E. McKenney]

On Wed, Aug 26, 2026 at 12:19:58PM +0800, KunWu Chan wrote:
> On Wed, Aug 26, 2026 at 11:53 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > On Wed, Aug 26, 2026 at 03:26:18AM +0000, Kunwu Chan wrote:
> > > Hi Paul,
> > >
> > > I'd be interested in helping with one of these. Are there any items
> > > that are not already being worked on and would be useful for me to
> > > take on? I'm particularly interested in the Tree
> > > `synchronize_srcu_atomic()` fastpath, if that part is still open.
> >
> > Please feel free to give it a shot.  The similar fastpath in Tiny SRCU
> > is a partial guide.  The trick is that get_state_synchronize_srcu()
> > and poll_state_synchronize_srcu() need to keep working.
> >
> > The commit that I just pushed to -rcu on branch "dev" seems to have a
> > working rcutorture.  Seems.  ;-)
> >
> >                                                         Thanx, Paul
> 
> Thanks Paul,
> 
> I'll start by looking at the Tiny fastpath and how the state-based
> synchronization APIs depend on the SRCU index, then work through
> the corresponding Tree fastpath.

Very good!  And please do not forget memory-ordering issues.

							Thanx, Paul

> Thanks,
> Kunwu
> 
> 
> > > Thanks,
> > > Kunwu
> > >
> > >
> > > August 26, 2026 at 7:49 AM, "Paul E. McKenney" <paulmck@kernel.org mailto:paulmck@kernel.org?to=%22Paul%20E.%20McKenney%22%20%3Cpaulmck%40kernel.org%3E > wrote:
> > >
> > >
> > > >
> > > > On Fri, Aug 21, 2026 at 12:53:22PM +0100, David Woodhouse wrote:
> > > >
> > > > >
> > > > > On Fri, 2026-08-21 at 11:22 +0800, Kunwu Chan wrote:
> > > > >
> > > > [ . . . ]
> > > >
> > > > >
> > > > > When applying this to current rcu/dev, the patch also needs to account
> > > > >  for the additional bool is_atomic argument to
> > > > >  check_init_srcu_struct(). Please rebase accordingly.
> > > > >
> > > > >  Ooh, shiny, I hadn't spotted that the atomic thing had landed. Maybe I
> > > > >  should retract my 'if we get that' above... but let's see how it
> > > > >  survives.
> > > > >
> > > > It is starting to shape up. Here is what is left, that I know of, anyway:
> > > >
> > > > o A bit more work to get rcutorture to put up with atomic SRCU.
> > > >
> > > > o Tree synchronize_srcu_atomic() fastpath.
> > > >
> > > > o Disable preemption.
> > > >
> > > > o Reorganize commits to fix the Tiny hunks that got applied
> > > >  in Tree commits, and likely other similar issues.
> > > >
> > > > o More work on error checking for use of banned primitives
> > > >  with atomic SRCU.
> > > >
> > > > o Update documentation.
> > > >
> > > > Not that it matters, but atomic Tiny SRCU seems to be doing quite well.
> > > >
> > > > >
> > > > > I also have more than just this patch to rebase onto rcu/dev — unlike
> > > > >  some, I'm not just hacking RCU for the fun of it; I get my jollies
> > > > >  elsewhere :)
> > > > >
> > > > >  Btw, the current commit c68271c3f83a message says "no index flip, no
> > > > >  grace-period sequence update" but that isn't true of the implementation
> > > > >  any more, is it? In fact, it *always* does those things, and doesn't
> > > > >  take the try-path optimisation by counting both epochs, that I'm doing
> > > > >  here?
> > > > >
> > > > The current "dev" branch is here:
> > > >
> > > > dc5bf5c8fb26 ("fixup! srcutree: Add an atomic Tree SRCU").
> > > >
> > > > Tree SRCU always does an index flip, though a single index flip can
> > > > serve multiple concurrent calls to synchronze_srcu_atomic().
> > > >
> > > > Tiny SRCU has the fast path, but must advance the index anyway for the
> > > > benefit of get_state_synchronize_srcu() and friends. It just bumps it
> > > > in one shot.
> > > >
> > > > Tree SRCU will also have to bump the index, but will be able to
> > > > do so manually, without doing all the grace-period processing.
> > > >
> > > > >
> > > > > Thanks for the careful review.
> > > > >
> > > > What David said! ;-)
> > > >
> > > >  Thanx, Paul
> > > >