Re: [PATCH v12 06/46] arm64: RMI: Define the user ABI

[Steven Price <steven.price@arm.com>]

On 23/01/2026 16:47, Suzuki K Poulose wrote:
> Hi Steven
> 
> On 17/12/2025 10:10, Steven Price wrote:
>> There is one CAP which identified the presence of CCA, and two ioctls.
>> One ioctl is used to populate memory and the other is used when user
>> space is providing the PSCI implementation to identify the target of the
>> operation.
>>
> 
> Could we split these changes to the corresponding implementation patches
> ? That might give us a full picture of how these UAPIs fit in the bigger
> picture.

I could do that, but I was deliberately keeping this as a patch which
contains the uABI interface so that the whole uABI is visible in one
place. I was trying to reduce the amount to review for those who aren't
so interested in the implementation details (especially for this series
where we know it's going to change somewhat with the move to RMM v2.0).

>> Signed-off-by: Steven Price <steven.price@arm.com>
>> ---
>> Changes since v11:
>>   * Completely reworked to be more implicit. Rather than having explicit
>>     CAP operations to progress the realm construction these operations
>>     are done when needed (on populating and on first vCPU run).
>>   * Populate and PSCI complete are promoted to proper ioctls.
>> Changes since v10:
>>   * Rename symbols from RME to RMI.
>> Changes since v9:
>>   * Improvements to documentation.
>>   * Bump the magic number for KVM_CAP_ARM_RME to avoid conflicts.
>> Changes since v8:
>>   * Minor improvements to documentation following review.
>>   * Bump the magic numbers to avoid conflicts.
>> Changes since v7:
>>   * Add documentation of new ioctls
>>   * Bump the magic numbers to avoid conflicts
>> Changes since v6:
>>   * Rename some of the symbols to make their usage clearer and avoid
>>     repetition.
>> Changes from v5:
>>   * Actually expose the new VCPU capability (KVM_ARM_VCPU_REC) by bumping
>>     KVM_VCPU_MAX_FEATURES - note this also exposes KVM_ARM_VCPU_HAS_EL2!
>> ---
>>   Documentation/virt/kvm/api.rst | 57 ++++++++++++++++++++++++++++++++++
>>   include/uapi/linux/kvm.h       | 23 ++++++++++++++
>>   2 files changed, 80 insertions(+)
>>
>> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/
>> api.rst
>> index 01a3abef8abb..2d5dc7e48954 100644
>> --- a/Documentation/virt/kvm/api.rst
>> +++ b/Documentation/virt/kvm/api.rst
>> @@ -6517,6 +6517,54 @@ the capability to be present.
>>     `flags` must currently be zero.
>>   +4.144 KVM_ARM_VCPU_RMI_PSCI_COMPLETE
>> +------------------------------------
>> +
>> +:Capability: KVM_CAP_ARM_RMI
>> +:Architectures: arm64
>> +:Type: vcpu ioctl
>> +:Parameters: struct kvm_arm_rmi_psci_complete (in)
>> +:Returns: 0 if successful, < 0 on error
>> +
>> +::
>> +
>> +  struct kvm_arm_rmi_psci_complete {
>> +    __u64 target_mpidr;
>> +    __u32 psci_status;
>> +    __u32 padding[3];
>> +  };
>> +
>> +Where PSCI functions are handled by user space, the RMM needs to be
>> informed of
>> +the target of the operation using `target_mpidr`, along with the status
>> +(`psci_status`). The RMM v1.0 specification defines two functions
>> that require
>> +this call: PSCI_CPU_ON and PSCI_AFFINITY_INFO.
>> +
>> +If the kernel is handling PSCI then this is done automatically and
>> the VMM
>> +doesn't need to call this ioctl.
>> +
>> +4.145 KVM_ARM_RMI_POPULATE
>> +--------------------------
>> +
>> +:Capability: KVM_CAP_ARM_RMI
>> +:Architectures: arm64
>> +:Type: vm ioctl
>> +:Parameters: struct kvm_arm_rmi_populate (in)
>> +:Returns: number of bytes populated, < 0 on error
>> +
>> +::
>> +
>> +  struct kvm_arm_rmi_populate {
>> +    __u64 base;
>> +    __u64 size;
>> +    __u64 source_uaddr;
>> +    __u32 flags;
>> +    __u32 reserved;
>> +  };
>> +
>> +Populate a region of protected address space by copying the data from
>> the user
>> +space pointer provided. This is only valid before any VCPUs have been
>> run.
>> +The ioctl might not populate the entire region and user space may
>> have to
>> +repeatedly call it (with updated pointers) to populate the entire
>> region.
> 
> It may be a good idea to spill out the restrictions on calling this,
> right ? We expect that the source_uaddr is a separate memory area
> from the "DRAM" (in shared mode) ?

Yes - I'd tried to say that by mentioning it's a "user space pointer",
but I could make that more explicit.

> Can this be called before/after converting the entire memory of the
> Guest to Private ? I believe, it must be done after the "initial
> change all of DRAM to private" ?

It can be called before or after setting RIPAS and it implicitly sets
RIPAS to RAM (I'll add something to make that explicit).

With RMM v1.0 there's an ugly problem that INIT_RIPAS fails on a region
which is already RIPAS RAM. This should be fixed with RMM v2.0. There's
a workaround in patch 17 currently:

		/* FIXME: Ugly hack to skip regions which are
		 * already RIPAS_RAM
		 */

But that should go away.

Note that with this proposed uAPI there is no mechanism for user space
to directly set the RIPAS - it's assumed that all guestmem_fd regions
should be set to RIPAS RAM at the time of the first VCPU run - which
means the VMM doesn't need to do anything special.

In the future we could consider a guestmem_fd flag or extra ioctl to
allow the VMM to control RIPAS at a finer granularity. But so far we
don't seem to have any compelling reason to expose that.

Thanks,
Steve

> Suzuki
> 
>>     .. _kvm_run:
>>   @@ -8765,6 +8813,15 @@ helpful if user space wants to emulate
>> instructions which are not
>>   This capability can be enabled dynamically even if VCPUs were already
>>   created and are running.
>>   +7.47 KVM_CAP_ARM_RMI
>> +--------------------
>> +
>> +:Architectures: arm64
>> +:Target: VM
>> +:Parameters: None
>> +
>> +This capability indicates that support for CCA realms is available.
>> +
>>   8. Other capabilities.
>>   ======================
>>   diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index dddb781b0507..8e66ba9c81db 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -974,6 +974,7 @@ struct kvm_enable_cap {
>>   #define KVM_CAP_GUEST_MEMFD_FLAGS 244
>>   #define KVM_CAP_ARM_SEA_TO_USER 245
>>   #define KVM_CAP_S390_USER_OPEREXEC 246
>> +#define KVM_CAP_ARM_RMI 247
>>     struct kvm_irq_routing_irqchip {
>>       __u32 irqchip;
>> @@ -1628,4 +1629,26 @@ struct kvm_pre_fault_memory {
>>       __u64 padding[5];
>>   };
>>   +/* Available with KVM_CAP_ARM_RMI, only for VMs with
>> KVM_VM_TYPE_ARM_REALM  */
>> +struct kvm_arm_rmi_psci_complete {
>> +    __u64 target_mpidr;
>> +    __u32 psci_status;
>> +    __u32 padding[3];
>> +};
>> +
>> +#define KVM_ARM_VCPU_RMI_PSCI_COMPLETE    _IOW(KVMIO, 0xd6, struct
>> kvm_arm_rmi_psci_complete)
>> +
>> +/* Available with KVM_CAP_ARM_RMI, only for VMs with
>> KVM_VM_TYPE_ARM_REALM */
>> +struct kvm_arm_rmi_populate {
>> +    __u64 base;
>> +    __u64 size;
>> +    __u64 source_uaddr;
>> +    __u32 flags;
>> +    __u32 reserved;
>> +};
>> +
>> +#define KVM_ARM_RMI_POPULATE_FLAGS_MEASURE    (1 << 0)
>> +
>> +#define KVM_ARM_RMI_POPULATE    _IOW(KVMIO, 0xd7, struct
>> kvm_arm_rmi_populate)
>> +
>>   #endif /* __LINUX_KVM_H */
>