Re: [PATCH v2 0/7] srcu: NMI-safe atomic SRCU readers, and their irqsave forms

[David Woodhouse <dwmw2@infradead.org>]

[-- Attachment #1: Type: text/plain, Size: 3532 bytes --]

On Wed, 2026-11-11 at 09:59 -0500, Mathieu Desnoyers wrote:
> Hi Paul,
> 
> This series adapts atomic SRCU to the needs of RCU pseudo-transactions
> [1], as we discussed [2].  Their readers resolve a proxy within a short
> section, in any context, NMI included.  A writer waits for these
> sections without sleeping, from task, softirq or hardirq context.
> Atomic SRCU fits, with one fix, one rule made explicit and two
> additions:
> 
> - Patch 1 fixes Tiny synchronize_srcu_atomic().  With
>   CONFIG_PREEMPTION=y, it called synchronize_rcu(), which ends an RCU
>   grace period on Tiny RCU.  That is wrong when the caller is in an
>   interrupt handler or within rcu_read_lock().
> - Patch 2 is about calling synchronize_srcu_atomic() from an interrupt
>   handler.  All calls on that srcu_struct, task context included, must
>   then disable interrupts around it.  Otherwise, the interrupt handler
>   may spin forever on the CPU of a waiter it interrupted.  Patch 2
>   documents this rule, and warns when it is broken.
> - Patch 3 makes the atomic readers NMI-safe.  They use the counter
>   increments of __srcu_read_lock_nmisafe(), which keep the full barriers
>   that synchronize_srcu_atomic() pairs with.
> - Patch 4 adds srcu_read_lock_atomic_irqsave() and
>   srcu_read_unlock_atomic_irqrestore().  They disable interrupts instead
>   of preemption, for a domain whose writers wait with interrupts
>   disabled.
> - Patch 5 makes rcutorture run readers from NMI, for the NMI-safe reader
>   flavors.
> - Patch 6 makes rcutorture alternate the atomic readers of each CPU with
>   irqsave readers, with a new atomic_irqsave parameter.  It also adds an
>   SRCU-P run of this mode to torture.sh.
> - Patch 7 makes rcutorture check what patch 1 fixes.
> 
> Patch 2 does not make synchronize_srcu_atomic() disable interrupts
> itself.  A reader that disables only preemption may wait for an IPI to
> the waiter's CPU, and that IPI would never run.  Only the caller knows
> that its readers disable interrupts.
> 
> On architectures without NMI-safe this_cpu_inc(), every atomic reader
> now pays an atomic increment at lock and at unlock.  The update side is
> unchanged.  On architectures with NMI-safe this_cpu_inc(), such as x86,
> the readers use the same counter increments as before.  David agreed to
> the change, as long as synchronize_srcu_atomic() stays within the 4-16us
> it was designed for [3].

Tested-by: David Woodhouse <dwmw@amazon.co.uk>

I gave it a spin on the 192-way Granite Rapids box I've been using for
the KVM gfn-to-pfn-cache conversion to atomic SRCU.

All my torture tests pass, including the mmu_notifier race test which
does a full invalidate/refresh cycle for every one of 5000 event
channel deliveries. That works out at over 1.2 million calls to
synchronize_srcu_atomic() per run, of which 99.4% are below 8µs:

                with series          without
    [4, 8)µs     99.388%             99.458%
    [8, 16)       0.583%             0.508%
    [16, 32)      0.026%             0.032%
    > 32µs        0.003%             0.002%
    worst        128-256µs bucket    512µs-1ms bucket
    (n)          1235551             1126320

So it's basically identical to the same measurement without your
series, which is fairly much as we expected.

Also, rcutorture with reader_flavor=0x10 ran clean for five minutes,
for whatever that's worth on a config where NEED_SRCU_NMI_SAFE is off.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6179 bytes --]