Re: [PATCH v3 0/3] arm64: realm: Fix DMA address for devices

[Suzuki K Poulose <suzuki.poulose@arm.com>]

Hi Marek,


On 27/02/2025 14:41, Suzuki K Poulose wrote:
> Linux can be run as a Confidential Guest in Arm CCA from Linux v6.13. The address
> space (GPA or IPA) of a Realm VM is split into two halves, with private bottom
> half and shared top half. In Linux we treat the "top" bit of the IPA space as
> an attribute, to indicate whether it is shared or not (MSB == 1 implies shared).
> Stage2 (GPA to PA) translations used by the CPU accesses, cover the full IPA space,
> and are managed by RMM. The "top" bit as attribute is only a software construct.
> 
> At present any device passed through to a Realm is treated as untrusted and the
> Realm uses bounce buffering for any DMA, using the "decrypted" (shared) DMA
> buffers (i.e., IPA with top bit set). In Linux, we only send the "DMA" address
> masking the "top" bit. In Arm CCA, SMMU for untrusted devices are managed by the
> non-secure Host and thus it can be confusing for the host/device when an unmasked
> address is provided. Given there could be other hypervisors than Linux/KVM
> running Arm CCA guests, the Realm Guest must adhere to a single convention for
> the DMA address. This gets further complicated when we add support for trusted
> devices, which can DMA into the full Realm memory space, once accepted. Thus,
> a DMA masked address (with "top" bit lost) will prevent a trusted device from
> accessing a shared buffer.
> 
> To resolve this Arm has decided to standardise the DMA address used by the Realm
> to include the full IPA address bits (including the "top" bit, which Linux uses
> as an attribute). This implies, any DMA to a shared buffer must have the top bit
> of the IPA space set.
> 
> There is already a provision to do this in phys_to_dma* and dma_to_phys(), but
> that is specific to AMD SME and is quite the opposite of what we need for Arm CCA.
> i.e., For Arm CCA we need to set the bit for "decrypted" DMA and clear the bit
> for "encrypted".
> 
> This series converts the existing __sme_* helpers to a bit more generalised versions :
> dma_addr_decrypted() and dma_encrypted(). Also, while converting a DMA address back
> to CPU physical address requires clearing off any "encryption/decryption" bits.
> I have named this "dma_addr_canonical()". (The other options are :
>    * dma_addr_clear_encryption - Well, not just for encryption, but we clear decryption
>      too, so not ideal.
>    * dma_addr_normal
>    * dma_addr_clear
>    * dma_addr_default
> 
> This also implies that the VMMs must take care to :
> 
>   1. Create the S2-SMMU mappings for VFIO at the "unprotected" alias.
>   2. Always mask the "top" bit off any IPA it receives from the Realm for DMA.
>      KVM already does that today and no changes are required.
> 
> A kvmtool branch with the changes above is available here [1]. There are two
> patches [2] & [3], that are really required on top of the Arm CCA support.
> 
> Ideally it would be good to get this backported to v6.13 stable kernel releases
> to make sure that they are compliant with this change.
> 

Please could you take a look at this series and let us know your
thoughts ? If you are happy with the changes, are you happy to pull
this through the DMA MAP tree ? The relevant bits have been reviewed/
acked by people (arm64 and AMD bits).

Kind regards
Suzuki


> Changes since v2:
>   Link: https://lkml.kernel.org/r/20250219220751.1276854-1-suzuki.poulose@arm.com
>    - Collect Acks & Reviews for Patch 1
>    - Rename helpers
>    	dma_encrypted		=> dma_addr_encrypted
> 	dma_decrypted		=> dma_addr_unencrypted
> 	dma_clear_encryption	=> dma_addr_canonical
>    - For Arm CCA, use PROT_NS_SHARED, set/clear only the top IPA bit.
>    - Drop dma_addr_encrypted() helper for Arm CCA as it is a NOP
> 
> Changes since v1
>   Link: https://lkml.kernel.org/r/20250212171411.951874-1-suzuki.poulose@arm.com
>   - Follow Robin's suggestion to generalise the DMA address conversion helpers
>     to provide dma_{encrypte,decrypted,clear_encryption}. See PATCH 2 for more
>     details.
>   - Add a fix to the ordering of "__sme_clr" for dma_to_phys (PATCH 1)
> 
> [1] git@git.gitlab.arm.com:linux-arm/kvmtool-cca.git cca/guest-dma-alias/v1
> [2] https://gitlab.arm.com/linux-arm/kvmtool-cca/-/commit/ea37a6eb968abe4c75be4a8a90808714657c2ef7
> [3] https://gitlab.arm.com/linux-arm/kvmtool-cca/-/commit/8afd0d5e6a7ee444dd0c1565fe94ecd831054a29
> 
> Cc: Will Deacon <will@kernel.org>
> Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Cc: Steven Price <steven.price@arm.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Marek Szyprowski <m.szyprowski@samsung.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Cc: Gavin Shan <gshan@redhat.com>
> 
> Suzuki K Poulose (3):
>    dma: Fix encryption bit clearing for dma_to_phys
>    dma: Introduce generic dma_addr_*crypted helpers
>    arm64: realm: Use aliased addresses for device DMA to shared buffers
> 
>   arch/arm64/include/asm/mem_encrypt.h | 11 +++++++++++
>   include/linux/dma-direct.h           | 13 +++++++++----
>   include/linux/mem_encrypt.h          | 23 +++++++++++++++++++++++
>   3 files changed, 43 insertions(+), 4 deletions(-)
>