Re: [PATCH v15 12/37] KVM: arm64: CCA: Support the VGIC in realms

[Steven Price <steven.price@arm.com>]

On 23/07/2026 07:56, Kohei Enju wrote:
> On 07/22 14:31, Steven Price wrote:
>> On 22/07/2026 09:27, Kohei Enju wrote:
>>> On 07/15 15:28, Steven Price wrote:
>>>> The RMM provides emulation of a VGIC to the realm guest. With RMM v2.0
>>>> the registers are passed in the system registers so this works similar
>>>> to a normal guest, but kvm_arch_vcpu_put() need reordering to early out,
>>>> and realm guests don't support GICv2 even if the host does.
>>>>
>>>> Signed-off-by: Steven Price <steven.price@arm.com>
>>>
>>> Hi Steven,
>>
>> Hi Kohei,
>>
>>> I've been testing this series and found that when the host CPU doesn't have
>>> ARM64_HAS_ICH_HCR_EL2_TDIR this series doesn't work as expected.
>>
>> Thanks for testing!
>>
>>> Since commit 2a28810cbb8b ("KVM: arm64: GICv3: Detect and work around the lack
>>> of ICV_DIR_EL1 trapping"), when the host CPU doesn't support this feature, KVM
>>> traps all GIC sysreg accesses in the common group. However, currently trap
>>> handlers for ICC_{PMR,RPR,CTLR}_EL1 registers are missing [0]. So when Realm
>>> guests try to access those registers, KVM traps them but just emits the warning
>>> shown in [1], and Realm guests fail to boot.
>>>
>>> As far as I can tell, the CCA requirements don't require the
>>> ARM64_HAS_ICH_HCR_EL2_TDIR feature. If that's the case, this seems to be a
>>> problem. Is there any workaround for this issue, or should we implement trap
>>> handlers for those registers?
>>
>> As Marc has already said in his reply, I'm really surprised you have a
>> CPU which implements CCA but doesn't have ARM64_HAS_ICH_HCR_EL2_TDIR.
>> Can you give some more details about the platform you are testing on?
> 
> Thank you for taking a look, Steve.
> 
> Unfortunately, due to the company's policy, I can't share any details
> about the platform I'm currently testing on. When the time is right,
> I'll be happy to do so.
> 
>>
>> Ultimately there are two options here - either don't support CCA (so
>> detect the lack of ARM64_HAS_ICH_HCR_EL2_TDIR and bail out early), or
>> plumb in the trap handlers - as commit 2a28810cbb8b points out this
>> isn't something we really want to support if we can avoid it.
> 
> Yes, the former makes perfect sense if there would be no systems that
> supports CCA without TDIR. However, if such systems do exist, I'd be
> interested in exploring the latter approach.
> 
>>
>> Hence I'm interested to know where this sits between "hacked up test
>> system" and "production hardware". E.g. if this is an emulator it might
>> be possible to just enable the CPU feature.
> 
> Again, I can't share that right now, but I'd like to share more when the
> time comes.

Given Marc's response for now I'm not going to add support for this. 
"When the time comes" we can revisit whether it makes sense to have 
upstream support and what that might look like.

However, I did ask an AI tool to have a go at implementing this, and it 
came up with the (very lightly tested) patch below. That might at least 
give you something you can test. It doesn't meet Marc's request that it
reuses the existing implementation, and it took the AI a few goes at
getting something that "works" so I suspect it might have bugs - I
haven't reviewed the code myself.

Thanks,
Steve

----8<----
From 7aaaa5089c05f19bf00df2b651fa4bd7d738384c Mon Sep 17 00:00:00 2001
From: Steven Price <steven.price@arm.com>
Date: Thu, 23 Jul 2026 11:49:36 +0100
Subject: [PATCH] KVM: arm64: CCA: Handle GICv3 CPU interface traps for realms

On CPUs that lack ICH_HCR_EL2.TDIR, KVM works around the missing DIR
trap by setting ICH_HCR_EL2.TC and trapping the GICv3 common CPU
interface registers. Normal guests handle these traps in hyp via the
VGIC CPU interface emulation, but realm guests exit through the RMM and
reach KVM's host-side sysreg emulation instead.

That path only handles ICC_DIR_EL1 and treats the rest of the trapped
common group as undefined. A realm guest can therefore take an
unexpected UNDEF when accessing registers such as ICC_PMR_EL1 or
ICC_CTLR_EL1 on systems that require the TC workaround.

Handle the GICv3 common CPU interface registers directly from the realm
sysreg exit path. Reads are satisfied from the saved VGIC state and
writes update the VGIC shadow state. For PMR and CTLR writes, also
restore the VMCR/APR state to the live GIC CPU interface before
re-entering the REC, matching the effect a direct guest sysreg write
would have had.

Signed-off-by: Steven Price <steven.price@arm.com>
---
 arch/arm64/kvm/rmi-exit.c | 119 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)

diff --git a/arch/arm64/kvm/rmi-exit.c b/arch/arm64/kvm/rmi-exit.c
index 78d9189fd5ca..0fad8e68d920 100644
--- a/arch/arm64/kvm/rmi-exit.c
+++ b/arch/arm64/kvm/rmi-exit.c
@@ -3,17 +3,25 @@
  * Copyright (C) 2023-2026 ARM Ltd.
  */
 
+#include <linux/bitfield.h>
+#include <linux/irqchip/arm-gic-v3.h>
 #include <linux/kvm_host.h>
 #include <kvm/arm_hypercalls.h>
 #include <kvm/arm_psci.h>
 
 #include <linux/arm-smccc-rmi.h>
 #include <asm/kvm_emulate.h>
+#include <asm/kvm_hyp.h>
 #include <asm/kvm_rmi.h>
 #include <asm/kvm_mmu.h>
+#include <asm/sysreg.h>
+
+#include "vgic/vgic.h"
 
 typedef int (*exit_handler_fn)(struct kvm_vcpu *vcpu);
 
+#define GICV3_IDLE_PRIORITY	0xff
+
 static int rec_exit_reason_notimpl(struct kvm_vcpu *vcpu)
 {
 	vcpu_err(vcpu, "Unhandled exit reason from realm (ESR: %#llx)\n",
@@ -48,6 +56,114 @@ static int rec_exit_sync_iabt(struct kvm_vcpu *vcpu)
 	return -ENXIO;
 }
 
+static int rec_get_gicv3_bpr_min(void)
+{
+	return 8 - (FIELD_GET(ICH_VTR_EL2_PREbits,
+			      kvm_vgic_global_state.ich_vtr_el2) + 1);
+}
+
+static u8 rec_get_gicv3_active_priority(struct kvm_vcpu *vcpu)
+{
+	struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3;
+	u8 prio = 0;
+
+	for (int i = 0; i <= vgic_v3_max_apr_idx(vcpu); i++) {
+		u32 val = cpuif->vgic_ap0r[i] | cpuif->vgic_ap1r[i];
+
+		if (!val) {
+			prio += 32;
+			continue;
+		}
+
+		return (prio + __ffs(val)) << rec_get_gicv3_bpr_min();
+	}
+
+	return GICV3_IDLE_PRIORITY;
+}
+
+static void rec_restore_gicv3_vmcr_aprs(struct kvm_vcpu *vcpu)
+{
+	preempt_disable();
+	kvm_call_hyp(__vgic_v3_restore_vmcr_aprs,
+		     &vcpu->arch.vgic_cpu.vgic_v3);
+	preempt_enable();
+}
+
+/*
+ * Realm exits are handled after the RMM has returned to the host, so use the
+ * saved VGIC shadow state instead of the hyp-side live sysreg helpers.
+ */
+static bool rec_exit_gicv3_sys_reg(struct kvm_vcpu *vcpu, bool is_write)
+{
+	struct realm_rec *rec = &vcpu->arch.rec;
+	struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+	struct vgic_vmcr vmcr;
+	unsigned long esr = kvm_vcpu_get_esr(vcpu);
+	int rt = kvm_vcpu_sys_get_rt(vcpu);
+	u32 sysreg = esr_sys64_to_sysreg(esr);
+	u64 val;
+
+	if (!kvm_has_gicv3(vcpu->kvm))
+		return false;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+
+	switch (sysreg) {
+	case SYS_ICC_DIR_EL1:
+		if (!is_write)
+			return false;
+
+		vgic_v3_deactivate(vcpu, rec->run->exit.gprs[rt]);
+		return true;
+	case SYS_ICC_PMR_EL1:
+		if (is_write) {
+			vmcr.pmr = FIELD_GET(ICC_PMR_EL1_MASK,
+					     rec->run->exit.gprs[rt]);
+			vgic_set_vmcr(vcpu, &vmcr);
+			rec_restore_gicv3_vmcr_aprs(vcpu);
+		} else {
+			rec->run->enter.gprs[rt] = FIELD_PREP(ICC_PMR_EL1_MASK,
+							      vmcr.pmr);
+		}
+
+		return true;
+	case SYS_ICC_CTLR_EL1:
+		if (is_write) {
+			val = rec->run->exit.gprs[rt];
+			vmcr.cbpr = FIELD_GET(ICC_CTLR_EL1_CBPR_MASK, val);
+			vmcr.eoim = FIELD_GET(ICC_CTLR_EL1_EOImode_MASK, val);
+			vgic_set_vmcr(vcpu, &vmcr);
+			rec_restore_gicv3_vmcr_aprs(vcpu);
+		} else {
+			val  = FIELD_PREP(ICC_CTLR_EL1_PRI_BITS_MASK,
+					  vgic_cpu->num_pri_bits - 1);
+			val |= FIELD_PREP(ICC_CTLR_EL1_ID_BITS_MASK,
+					  vgic_cpu->num_id_bits);
+			val |= FIELD_PREP(ICC_CTLR_EL1_SEIS_MASK,
+					  FIELD_GET(ICH_VTR_EL2_SEIS,
+						    kvm_vgic_global_state.ich_vtr_el2));
+			val |= FIELD_PREP(ICC_CTLR_EL1_A3V_MASK,
+					  FIELD_GET(ICH_VTR_EL2_A3V,
+						    kvm_vgic_global_state.ich_vtr_el2));
+			val |= FIELD_PREP(ICC_CTLR_EL1_CBPR_MASK,
+					  vmcr.cbpr);
+			val |= FIELD_PREP(ICC_CTLR_EL1_EOImode_MASK,
+					  vmcr.eoim);
+			rec->run->enter.gprs[rt] = val;
+		}
+
+		return true;
+	case SYS_ICC_RPR_EL1:
+		if (is_write)
+			return false;
+
+		rec->run->enter.gprs[rt] = rec_get_gicv3_active_priority(vcpu);
+		return true;
+	default:
+		return false;
+	}
+}
+
 static int rec_exit_sys_reg(struct kvm_vcpu *vcpu)
 {
 	struct realm_rec *rec = &vcpu->arch.rec;
@@ -59,6 +175,9 @@ static int rec_exit_sys_reg(struct kvm_vcpu *vcpu)
 	if (is_write)
 		vcpu_set_reg(vcpu, rt, rec->run->exit.gprs[rt]);
 
+	if (rec_exit_gicv3_sys_reg(vcpu, is_write))
+		return 1;
+
 	ret = kvm_handle_sys_reg(vcpu);
 	if (!is_write)
 		rec->run->enter.gprs[rt] = vcpu_get_reg(vcpu, rt);
-- 
2.43.0