From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47FB1339399 for ; Tue, 6 Oct 2026 03:42:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791258175; cv=none; b=L8h4CTT38Xjy76uTAyCjt3kw5+oijJJGI81aVtpuXF4TVhquhGJFEtSHFl7BrX9Fm3UVZlrfHiTqEmb+uw/ohgACJRrpxsgthbz2HGjWLi+nWcafkyqTS7cw59SiFyPM+LrReisz5FIeqy3bx+8zE/KH0JihYdgCsPC3MJPQpc8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791258175; c=relaxed/simple; bh=ewL6r4+WUG3zvSaf1v33KUoNnFiMKovzld70FKrMj7E=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hdznNq4bIMY7f6isLHvGcRioTpgLC36HfFUceq0h8M1AFLfEawelxvZVbrBoNenlGHw7Nd4epAckDAZRGpS98QA4KIGY5QXpCmwbaPkfoFIxxKRoW1ztOB2mRQS5yjuB3Lmd1ELApwhWL8HOTIH48SJEvhkoGur4k2jXOQ9DUU4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SzSL4ZVt; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=l/NTFriw; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SzSL4ZVt"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="l/NTFriw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1791258172; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6ciyrU+oMiPwGK5aZ+ysyOdFUKDXGXj8SaQGvfUaxVI=; b=SzSL4ZVtnOgWxgW+Sc8lI62BiquJcdntraIk3q9uipWIi34VW7vRyMReJCU8dCG8aSmqzk tY46IQbhQvZwkzx/TgSO8wSAn460BlQEnqxrxXtHwf1jHoSMF/L7VClKwXMD489RkJwZHO BhO5d2puaTfidFgHmOMUmwmkI19aRAk= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-32-JdUiGnynM3eFcQCxpE8Rmw-1; Mon, 05 Oct 2026 23:42:50 -0400 X-MC-Unique: JdUiGnynM3eFcQCxpE8Rmw-1 X-Mimecast-MFC-AGG-ID: JdUiGnynM3eFcQCxpE8Rmw_1791258170 Received: by mail-pj1-f72.google.com with SMTP id 98e67ed59e1d1-39512608fb1so4814150a91.1 for ; Mon, 05 Oct 2026 20:42:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1791258170; x=1791862970; darn=vger.kernel.org; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to :content-type; bh=6ciyrU+oMiPwGK5aZ+ysyOdFUKDXGXj8SaQGvfUaxVI=; b=l/NTFriwGYNqvbzSsEhrRb5pT92oYAe8m38nGoUynJ3SxlOc44z7k8eiUyippkEIIK eXDElz3kIK9f6yCKQpIHdhU1nY31QRO3M4E6zTlbS+FQ0E8JUmAcDzuw1zlExnrPx/2Q EZep/wv6uSm/Ogs4lOJGvYd1awH/QchBy2SfCIGWkLE75sXIzi27XAQsK9QcJ5077F/w pBabnIcUWh3MjvuPrNV/Sa47pXRis8YGh3nb7UCtOc4x/0LXEXYYXKhKBONk77toVzeR wp2SFc5QBvvIE4X4q/9uq+B2q90oT2PMY9fwWTwTGtsxH6CEr2+dDJdNOYaR9zCffROq 7+aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791258170; x=1791862970; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=6ciyrU+oMiPwGK5aZ+ysyOdFUKDXGXj8SaQGvfUaxVI=; b=gGjSMkd7s5WyzsLDFLc3REJMjhXXbCuUjOYGNithsa7QNLBUqrstOy4G1arwmv86/w F0DCMDJU/kMibno6YqlFurOUP0kP8dnWOgpCDwnVgaEfD7RzZRdEkai+QpFAW2cglo67 5GYIOKBKO51cAi0Qfy/fmYpzTmPcdH8FbhwI0xHwHcn5fn43SEuPP04ev4fmK7MRzDu3 6b3Cc4YA9ai8bQ0uhOcdrCvtHdR9o2l0Fx10OO90aGzQLxdUUCYYAoAdGzzScSyA8DWW q0lMtYZL+61Jl3ZddFh2d8+jmpKQtgsX2RDQXSdyqRNbqA7D+xb9tomx4xNavNPbEixs olXg== X-Forwarded-Encrypted: i=1; AKwUvBx31Htm8reYGmLYQ38dHWJ+0mRaKDbbpzE1nlCddMQRHANUQ3ExflxW6nXBkIYqFqDzD1kYMQMMGdQTM4c=@vger.kernel.org X-Gm-Message-State: AFq9FYLPZMyHNWjkgtuQYZtH15S1tKYu7rWAegRLEpSudybeWciwmgTN fEx7MqH1jfZMWin2tlSKp8PsJNbLXMIvVx9+RGvg4PZG8K5s0u3ix3q0GwlYy1fh+rTtdVCIaFG xzOrDViNk9eDcwTTYjLMBWX+KHOFaHN/CWnUqpm1KGPxK3hTVsaKVWnaVPyCBcD4KcA== X-Gm-Gg: AYBFou3/SiIAajno//MNqG+s+jbARJN0c5yeZ0nXgeLDxYvLrqswN2dWSBdbyF39DIS xpICD3cw3lr+EbYetxd9OdL5J81/nvTVC0+2u0CZWQ/3B4k5tAiHGAgLxBG3H6A/3z3a/5CbKCF ZFYVW8Py63239dVF9zII7Wb7iX2dvEod9vZ/r1e/mLWGaOB1wRZaHg5Y2JTjcB6tkgZXISmIALf YB9ZF+pVWh7DXEhW34WRw/mFt7uAsvbHXJxPobv7mxIjCtJc2OonrahIEGVqBBQsabOqt8VdeeO 7ilHZwIaOC38/2MvASyAWVHW/jmI3dagKB3rT0bL9CM4lr7vKHQ/n4maMGMfl3Ff13sj7UdU/4M m/2FDjppmJuhLn5/gMuMAkkZatv/7l+Lkb/z3SgvnSA== X-Received: by 2002:a17:90b:2886:b0:3a8:2c7e:8986 with SMTP id 98e67ed59e1d1-3a82c7e90c0mr3024688a91.17.1791258169453; Mon, 05 Oct 2026 20:42:49 -0700 (PDT) X-Received: by 2002:a17:90b:2886:b0:3a8:2c7e:8986 with SMTP id 98e67ed59e1d1-3a82c7e90c0mr3024671a91.17.1791258168948; Mon, 05 Oct 2026 20:42:48 -0700 (PDT) Received: from [192.168.68.52] (n175-34-8-244.mrk21.qld.optusnet.com.au. [175.34.8.244]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a7e4b00de6sm5567281a91.1.2026.10.05.20.42.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Oct 2026 20:42:47 -0700 (PDT) Message-ID: <9bbb3b3c-522f-4deb-9f32-0fcf731dd0bb@redhat.com> Date: Tue, 6 Oct 2026 13:42:38 +1000 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v22 18/23] KVM: arm64: CCA: Introduce Realms To: Suzuki K Poulose , kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: maz@kernel.org, will@kernel.org, catalin.marinas@arm.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, steven.price@arm.com, aneesh.kumar@kernel.org, oupton@kernel.org, joey.gouly@arm.com, tabba@google.com, yuzenghui@huawei.com, linux-coco@lists.linux.dev, gankulkarni@os.amperecomputing.com, sdonthineni@nvidia.com, alpergun@google.com, fj0570is@fujitsu.com, WeiLin.Chang@arm.com, lpieralisi@kernel.org, enju.kohei@fujitsu.com, sudeep.holla@arm.com, jonathan.cameron@oss.qualcomm.com References: <20261005090754.2140522-1-suzuki.poulose@arm.com> <20261005090754.2140522-19-suzuki.poulose@arm.com> Content-Language: en-US From: Gavin Shan In-Reply-To: <20261005090754.2140522-19-suzuki.poulose@arm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 10/5/26 7:07 PM, Suzuki K Poulose wrote: > From: Steven Price > > Add foundational work for supporting Realms. > - Add a new VM flavor. > - At KVM init, check if the KVM can support Realms (though not functional > yet) and will be advertised by static key kvm_rmi_is_available. This > will be turned on in a later patches, once we have all the bits and > pieces ready. For now check if we are blessed with KVM_MODE_RMM. > - Add realm specific tracking in kvm_arch. Since Realm and protected pKVM > states are mutually exclusive, move them into a union. > > Please note that we cannot create Realm VMs yet. This requires further > changes to the UABI and core RMI driver support, which will come later. > > Reviewed-by: Jonathan Cameron > Signed-off-by: Steven Price > Co-developed-by: Suzuki K Poulose > Signed-off-by: Suzuki K Poulose > --- > arch/arm64/include/asm/kvm_emulate.h | 16 ++++++++ > arch/arm64/include/asm/kvm_host.h | 19 ++++++--- > arch/arm64/include/asm/kvm_rmi.h | 61 ++++++++++++++++++++++++++++ > arch/arm64/include/asm/virt.h | 1 + > arch/arm64/kvm/Makefile | 2 +- > arch/arm64/kvm/arm.c | 6 +++ > arch/arm64/kvm/mmu.c | 1 + > arch/arm64/kvm/rmi.c | 18 ++++++++ > 8 files changed, 118 insertions(+), 6 deletions(-) > create mode 100644 arch/arm64/include/asm/kvm_rmi.h > create mode 100644 arch/arm64/kvm/rmi.c > > diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h > index a3c1928bdf743..d360a8b05b8bf 100644 > --- a/arch/arm64/include/asm/kvm_emulate.h > +++ b/arch/arm64/include/asm/kvm_emulate.h > @@ -793,4 +793,20 @@ static inline void kvm_reset_vcpu_psci(struct kvm_vcpu *vcpu, > vcpu_set_reg(vcpu, 0, reset_state->r0); > } > > +static inline enum realm_state kvm_realm_state(struct kvm *kvm) > +{ > + return READ_ONCE(kvm->arch.realm.state); > +} > + > +static inline void kvm_set_realm_state(struct kvm *kvm, > + enum realm_state new_state) > +{ > + WRITE_ONCE(kvm->arch.realm.state, new_state); > +} > + > +static inline bool kvm_realm_is_created(struct kvm *kvm) > +{ > + return kvm_vm_is_realm(kvm) && kvm_realm_state(kvm) != REALM_STATE_NONE; > +} > + > #endif /* __ARM64_KVM_EMULATE_H__ */ > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index dfa9d4ec61a76..3debffef638a4 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -27,6 +27,7 @@ > #include > #include > #include > +#include > #include > > #define __KVM_HAVE_ARCH_INTC_INITIALIZED > @@ -334,6 +335,7 @@ enum kvm_arm_vm_flavor { > VM_PKVM, /* Normal guests on pKVM */ > MARKER(__VM_PROTECTED), > VM_PROTECTED_PKVM, /* Protected VM */ > + VM_REALM, /* CCA */ > VM_FLAVOR_MAX > }; > > @@ -450,11 +452,14 @@ struct kvm_arch { > /* Count the number of VNCR_EL2 TLBs */ > atomic_t vncr_tlb_count; > > - /* > - * For an untrusted host VM, 'pkvm.handle' is used to lookup > - * the associated pKVM instance in the hypervisor. > - */ > - struct kvm_protected_vm pkvm; > + union { > + /* > + * For an untrusted host VM, 'pkvm.handle' is used to lookup > + * the associated pKVM instance in the hypervisor. > + */ > + struct kvm_protected_vm pkvm; > + struct realm realm; > + }; > > #ifdef CONFIG_PTDUMP_STAGE2_DEBUGFS > /* Nested virtualization info */ > @@ -1565,6 +1570,10 @@ struct kvm *kvm_arch_alloc_vm(void); > (__kvm && kvm_vm_is_protected_pkvm(__kvm)); \ > }) > > + > +#define kvm_vm_is_realm(kvm) ((kvm)->arch.vm_flavor == VM_REALM) > +#define vcpu_is_rec(vcpu) kvm_vm_is_realm((vcpu)->kvm) > + vcpu_is_rec() isn't safely called in nVHE hyp stub. So do we need add something similiar to what we had for vcpu_is_protected(vcpu) in PATCH[05]? #ifdef __KVM_NVHE_HYPERVISOR__ /* vcpu_is_rec() isn't expected to called in nVHE hyp stub */ #define vcpu_is_rec(vcpu) BUILD_BUG_ON(1) #else #define vcpu_is_rec(vcpu) kvm_vm_is_realm((vcpu)->kvm) #endif > #define kvm_vm_hyp_is_distrusting(kvm) ((kvm)->arch.vm_flavor >= __VM_DISTRUSTING_HYP) > > int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature); > diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_rmi.h > new file mode 100644 > index 0000000000000..44f5c75a27b5b > --- /dev/null > +++ b/arch/arm64/include/asm/kvm_rmi.h > @@ -0,0 +1,61 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Copyright (C) 2023-2026 ARM Ltd. > + */ > + > +#ifndef __ASM_KVM_RMI_H > +#define __ASM_KVM_RMI_H > + > +/** > + * enum realm_state - State of a Realm > + * > + * Mirrors the RMM's Realm lifecycle states where they are meaningful to KVM, > + * with REALM_STATE_DYING being a KVM-internal state used to prevent further > + * requests while teardown is in progress. KVM does not track REALM_SYSTEM_OFF > + * or REALM_ZOMBIE separately as they naturally lead to teardown. > + */ > +enum realm_state { > + /** > + * @REALM_STATE_NONE: > + * Realm has not yet been created. rmi_realm_create() has not > + * yet been called. > + */ > + REALM_STATE_NONE, > + /** > + * @REALM_STATE_NEW: > + * Realm is under construction, rmi_realm_create() has been > + * called, but it is not yet activated. Pages may be populated. ^^^^^ Granule instead of page is the term applicable to RMM. Lets use a generic term 'memory' here. Also, lets mention the name of the function used to populate the memory (granules), which is consistent to the comments for REALM_STATE_ACTIVE. Memory may be populated by rmi_rtt_data_map_init(). > + */ > + REALM_STATE_NEW, > + /** > + * @REALM_STATE_ACTIVE: > + * Realm has been created and is eligible for execution with > + * rmi_rec_enter(). Pages may no longer be populated with > + * rmi_data_create(). > + */ > + REALM_STATE_ACTIVE, rmi_data_create() is an invalid function name. Memory can't longer be populated using rmi_rtt_data_map_init(). > + /** > + * @REALM_STATE_DYING: > + * Realm is in the process of being destroyed or has already been > + * destroyed. > + */ > + REALM_STATE_DYING, > + /** > + * @REALM_STATE_DEAD: > + * Realm has been destroyed. > + */ > + REALM_STATE_DEAD > +}; > + > +/** > + * struct realm - Additional per VM data for a Realm > + * > + * @state: The lifetime state machine for the realm > + */ > +struct realm { > + enum realm_state state; > +}; > + > +void kvm_init_rmi(void); > + > +#endif /* __ASM_KVM_RMI_H */ > diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h > index b546703c3ab9a..92cec42952f42 100644 > --- a/arch/arm64/include/asm/virt.h > +++ b/arch/arm64/include/asm/virt.h > @@ -87,6 +87,7 @@ void __hyp_reset_vectors(void); > bool is_kvm_arm_initialised(void); > > DECLARE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); > +DECLARE_STATIC_KEY_FALSE(kvm_rmi_is_available); > > static inline bool is_pkvm_initialized(void) > { > diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile > index 59612d2f277c1..ed3cf30eb06e7 100644 > --- a/arch/arm64/kvm/Makefile > +++ b/arch/arm64/kvm/Makefile > @@ -16,7 +16,7 @@ CFLAGS_handle_exit.o += -Wno-override-init > kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ > inject_fault.o va_layout.o handle_exit.o config.o \ > guest.o debug.o reset.o sys_regs.o stacktrace.o \ > - vgic-sys-reg-v3.o fpsimd.o pkvm.o \ > + vgic-sys-reg-v3.o fpsimd.o pkvm.o rmi.o \ > arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \ > vgic/vgic.o vgic/vgic-init.o \ > vgic/vgic-irqfd.o vgic/vgic-v2.o \ > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index 294cea4cc8271..37ef2ba4e43e2 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -42,6 +42,7 @@ > #include > #include > #include > +#include > #include > #include > > @@ -112,6 +113,8 @@ long kvm_get_cap_for_kvm_ioctl(unsigned int ioctl, long *ext) > return -EINVAL; > } > > +DEFINE_STATIC_KEY_FALSE(kvm_rmi_is_available); > + > DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); > > DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base); > @@ -2239,6 +2242,7 @@ static const struct kvm_vcpu_ops *arm64_vcpu_ops[] = { > KVM_VCPU_OPS(VM_VHE, vhe_vcpu_ops), > KVM_VCPU_OPS(VM_PKVM, pkvm_vcpu_ops), > KVM_VCPU_OPS(VM_PROTECTED_PKVM, pkvm_vcpu_ops), > + KVM_VCPU_OPS(VM_REALM, realm_vcpu_ops), > }; > > static void kvm_init_vcpu_ops(struct kvm_vcpu *vcpu) > @@ -3192,6 +3196,8 @@ static __init int kvm_arm_init(void) > > in_hyp_mode = is_kernel_in_hyp_mode(); > > + kvm_init_rmi(); > + > if (cpus_have_final_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) || > cpus_have_final_cap(ARM64_WORKAROUND_1508412)) > kvm_info("Guests without required CPU erratum workarounds can deadlock system!\n" \ > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > index 58bc6a85f48b4..697f1ba6667a9 100644 > --- a/arch/arm64/kvm/mmu.c > +++ b/arch/arm64/kvm/mmu.c > @@ -2927,6 +2927,7 @@ static const struct kvm_vm_s2_ops *arm64_vm_s2_ops[] = { > KVM_VM_S2_OPS(VM_NVHE, kvm_default_vm_s2_ops), > KVM_VM_S2_OPS(VM_PKVM, pkvm_vm_s2_ops), > KVM_VM_S2_OPS(VM_PROTECTED_PKVM, protected_pkvm_vm_s2_ops), > + KVM_VM_S2_OPS(VM_REALM, realm_vm_s2_ops), > }; > > static int kvm_vm_init_vm_s2_ops(struct kvm *kvm) > diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c > new file mode 100644 > index 0000000000000..5ecc8b3498698 > --- /dev/null > +++ b/arch/arm64/kvm/rmi.c > @@ -0,0 +1,18 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Copyright (C) 2023-2026 ARM Ltd. > + */ > + > +#include > + > +#include > + > +void kvm_init_rmi(void) > +{ > + if (kvm_get_mode() != KVM_MODE_RMM) > + return; > + > + /* TODO: Check if the RMI is available */ > + > + /* Future patch will enable static branch kvm_rmi_is_available */ > +} Thanks, Gavin