Re: [PATCH v2 2/2] vfio/type1: Set IOMMU_MMIO in dma->prot for MMIO-backed addresses

[Tom Lendacky <thomas.lendacky@amd.com>]

On 11/7/25 12:32, Jason Gunthorpe wrote:
> On Fri, Nov 07, 2025 at 11:56:51AM -0600, Tom Lendacky wrote:
> 
>> When you are on bare-metal, or in the hypervisor, System Memory Encryption
>> (SME) deals with the encryption bit set in the page table entries
>> (including the nested page table entries for guests). 
> 
> So "decrypted" means something about AMD's unique memory encryption
> scheme on bare metal but in a CC guest it is a cross arch 'shared with
> hypervisor' flag?

Note, that if the encryption bit is not set in the guest, then the host
encryption key is used if the underlying NPT leaf entry has the encryption
bit set. In that case, both the host and guest can read the memory, with
the memory still being encrypted in physical memory.

> 
> What about CXL memory? What about ZONE_DEVICE coherent memory? Do
> these get the C bit set too?

When CXL memory is presented as system memory to the OS it does support
the encryption bit. So when pages are allocated for the guest, the memory
pages will be encrypted with the guest key.

Not sure what you mean by ZONE_DEVICE coherent memory. Is it presented to
the system as system physical memory that the hypervisor can allocate as
guest memory?

> 
> :( :( :(
> 
>> In the guest (prior to Trusted I/O / TDISP), decrypted (or shared) memory
>> is used because a device cannot DMA to or from guest memory using the
>> guest encryption key. So all DMA must go to "decrypted" memory or be
>> bounce-buffered through "decrypted" memory (SWIOTLB) - basically memory
>> that does not get encrypted/decrypted using the guest encryption key.
> 
> Where is the code for this? As I wrote we always do sme_set in the
> iommu driver, even on guests, even for "decrypted" bounce buffered
> memory.
> 
> That sounds like a bug by your explanation?
> 
> Does this mean vIOMMU has never worked in AMD CC guests?

I assume by vIOMMU you mean a VMM-emulated IOMMU in the guest. This does
does not work today with AMD CC guests since it requires the hypervisor to
read the guest IOMMU buffers in order to emulate the behavior and those
buffers are encrypted. So there is no vIOMMU support today in AMD CC
guests.

There was a patch series submitted a while back to allocate the IOMMU
buffers in shared memory in order to support a (non-secure) vIOMMU in the
guest in order to support >255 vCPUs, but that was rejected in favor of
using kvm-msi-ext-dest-id.

https://lore.kernel.org/linux-iommu/20240430152430.4245-1-suravee.suthikulpanit@amd.com/

> 
>> It is not until we get to Trusted I/O / TDISP where devices will be able
>> to DMA directly to guest encrypted memory and guests will require secure
>> MMIO addresses which will need the encryption bit set (Alexey can correct
>> me on the TIO statements if they aren't correct, as he is closer to it all).
> 
> So in this case we do need to do sme_set on MMIO even though that MMIO
> is not using the dram encryption key?

@Alexey will be able to provide more details on how this works.

Thanks,
Tom

> 
> Jason