Re: [PATCH 00/17] KVM: Use atomic SRCU for gfn-to-pfn cache, reinstate guest mode for x86 nesting

[Kunwu Chan <kunwu.chan@gmail.com>]

On Tue, 22 Sep 2026 12:37:43 +0200 David Woodhouse <dwmw2@infradead.org> wrote:

> On Tue, 2026-09-22 at 11:16 +0800, KunWu Chan wrote:
> > Do you happen to have any numbers comparing the GPC invalidation
> > latency with regular SRCU vs. `synchronize_srcu_atomic()`? If there
> > are also numbers with the reader-free fastpath, that would be useful
> > for understanding its impact as well.
> 
> Yeah, I built some latency tests and was posting results in the earlier
> thread¹, on a few different test hosts.
> 
> I compared against the existing rwlock, as well as SRCU both with and
> without the try_synchronize_srcu() fast path. Mostly looking at the
> invalidation latency, since that was Sean's stated concern with the
> original RCU-based proof of concept.
> 
> All from the same test: 12 concurrent guest-memory invalidation
> reproducers hammering the Xen shinfo/vcpu_info caches, 300 second
> windows, measuring the invalidation drain end-to-end.
> 
> 192-way Granite Rapids, PREEMPT_RT production config:
> 
>   rwlock (before this series)              avg  4.4µs   max 3.85ms
>   synchronize_srcu_expedited() drain       avg  8.6µs   max 810µs
>   synchronize_srcu_atomic() + fastpath     avg  ~3µs    max 801µs
> 
> The A/B numbers I have for the reader-free fast path were on different
> hardware (128-way Ice Lake, production-like config):
> 
>   synchronize_srcu_atomic(), no fastpath   avg 8.0µs   max 6.0ms
>   with the inline no-readers proof         avg 3.6µs   max 326µs
> 
> If you want, it isn't much effort for me to tell my friend to redo any
> of the measurements.
> 
> ¹ https://lore.kernel.org/all/0d4af6318ac67486858be1df8d436147b444a2d2.camel@infradead.org/
> 

Hi David,

Thanks, this is very helpful. I've put the results together below.

KVM GPC invalidation drain latency

128-way Ice Lake:
┌──────────────────────────────────┬──────────────┬──────────────┐
│ Implementation                   │ Avg          │ Max          │
├──────────────────────────────────┼──────────────┼──────────────┤
│ Baseline (rwlock)                │ not provided │ not provided │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_expedited()     │ not provided │ not provided │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_atomic()        │ 8.0us        │ 6.0ms        │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_atomic() +      │ 3.6us        │ 326us        │
│   reader-free fastpath           │              │              │
└──────────────────────────────────┴──────────────┴──────────────┘

192-way Granite Rapids:
┌──────────────────────────────────┬──────────────┬──────────────┐
│ Implementation                   │ Avg          │ Max          │
├──────────────────────────────────┼──────────────┼──────────────┤
│ Baseline (rwlock)                │ 4.4us        │ 3.85ms       │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_expedited()     │ 8.6us        │ 810us        │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_atomic()        │ not provided │ not provided │
├──────────────────────────────────┼──────────────┼──────────────┤
│ synchronize_srcu_atomic() +      │ ~3us         │ 801us        │
│   reader-free fastpath           │              │              │
└──────────────────────────────────┴──────────────┴──────────────┘
Note: Avg / Max are the average and maximum end-to-end invalidation drain latency.
The measurements use 12 concurrent guest-memory invalidation reproducers 
hammering the Xen shinfo/vcpu_info caches over 300-second windows.

On the 128-way Ice Lake system, the reader-free fastpath reduces the
average latency from 8.0us to 3.6us, and the maximum from 6.0ms to
326us.

The 192-way Granite Rapids result is from a separate hardware
configuration, so I kept it separate from the 128-way A/B comparison.

The only missing comparison is the 192-way Granite Rapids result for
synchronize_srcu_atomic() without the reader-free fastpath. If you
already have that result, it would be useful to add it. No need to
rerun the measurement just for this table if you don't have it.

Could you please confirm that I transcribed the numbers correctly?

Thanks,
Kunwu