[PATCH 2/2] KVM: arm64: timer: Avoid the irq descriptor lock on vCPU load

[Amir Noohi <amirmnoohi@gmail.com>]

When the host interrupt controller has an active state, every vCPU load
sets the active state of the host timer interrupts with
irq_set_irqchip_state(): once for the virtual timer and, on VHE, once
more for the physical timer. On GICv5 hosts the state is also cleared on
every vCPU put, and with a userspace irqchip it is also cleared after a
guest exit when the virtual timer is not pending.

The timer interrupts are per CPU interrupts, so all CPUs share a single
interrupt descriptor and irq_set_irqchip_state() acquires its lock. CPUs
which load vCPUs concurrently therefore serialize on that lock, although
each of them only writes its own redistributor (and, when setting the
active state on GICv3, reads it back while holding the lock) or, on
GICv5, a system register.

This does not require over-subscription. On a 32 CPU Ampere Altra
(Neoverse-N1, GICv3, VHE) with an in-kernel GICv3, a test which makes N
vCPUs exit to userspace in a loop, each vCPU thread pinned to its own
otherwise idle CPU and running from its own guest page, gets 319694
exits/s from a single vCPU, but only 119943 per vCPU with 16 vCPUs and
58722 with 31. The aggregate rate stops growing at 8 vCPUs. At 8 vCPUs,
122800 of the 122847 lock contention events within 5 seconds had
__irq_get_desc_lock() or irq_set_irqchip_state() on the stack.

Use irq_set_percpu_irqchip_state(), which does not acquire the
descriptor lock. All callers of set_timer_irq_phys_active() run with
preemption disabled, and the timer interrupts are requested in
kvm_timer_hyp_init() and never freed afterwards. On hosts where KVM
stacks its own interrupt chip on top of the timer interrupts, its
irq_set_irqchip_state() callback masks or unmasks the parent interrupt
on the calling CPU, which is equally CPU local.

Same test and kernel configuration, exits/s per vCPU, mean of three 5
second runs:

                in-kernel GICv3           userspace irqchip
  vCPUs       before      after          before      after
      1       319694     324584          319670     345088
      8       278579     328736          112423     343984
     16       119943     328887           56120     282905
     31        58722     326056           28114     250586

With an in-kernel GICv3 the per vCPU rate no longer depends on the
number of vCPUs, and at 8 vCPUs no lock contention events have either
function on the stack.

Fixes: b103cc3f10c0 ("KVM: arm/arm64: Avoid timer save/restore in vcpu entry/exit")
Signed-off-by: Amir Noohi <amirmnoohi@gmail.com>
---
 arch/arm64/kvm/arch_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c
index 6ac3321f4c57..a9103625dea3 100644
--- a/arch/arm64/kvm/arch_timer.c
+++ b/arch/arm64/kvm/arch_timer.c
@@ -679,7 +679,7 @@ static void timer_restore_state(struct arch_timer_context *ctx)
 static inline void set_timer_irq_phys_active(struct arch_timer_context *ctx, bool active)
 {
 	int r;
-	r = irq_set_irqchip_state(ctx->host_timer_irq, IRQCHIP_STATE_ACTIVE, active);
+	r = irq_set_percpu_irqchip_state(ctx->host_timer_irq, IRQCHIP_STATE_ACTIVE, active);
 	WARN_ON(r);
 }
 
-- 
2.53.0