Re: [PATCH v2 2/2] vfio/type1: Set IOMMU_MMIO in dma->prot for MMIO-backed addresses

[Vasant Hegde <vasant.hegde@amd.com>]

On 11/8/2025 1:29 AM, Tom Lendacky wrote:
> On 11/7/25 12:32, Jason Gunthorpe wrote:
>> On Fri, Nov 07, 2025 at 11:56:51AM -0600, Tom Lendacky wrote:
>>
>>> When you are on bare-metal, or in the hypervisor, System Memory Encryption
>>> (SME) deals with the encryption bit set in the page table entries
>>> (including the nested page table entries for guests). 
>>
>> So "decrypted" means something about AMD's unique memory encryption
>> scheme on bare metal but in a CC guest it is a cross arch 'shared with
>> hypervisor' flag?
> 
> Note, that if the encryption bit is not set in the guest, then the host
> encryption key is used if the underlying NPT leaf entry has the encryption
> bit set. In that case, both the host and guest can read the memory, with
> the memory still being encrypted in physical memory.
> 
>>
>> What about CXL memory? What about ZONE_DEVICE coherent memory? Do
>> these get the C bit set too?
> 
> When CXL memory is presented as system memory to the OS it does support
> the encryption bit. So when pages are allocated for the guest, the memory
> pages will be encrypted with the guest key.
> 
> Not sure what you mean by ZONE_DEVICE coherent memory. Is it presented to
> the system as system physical memory that the hypervisor can allocate as
> guest memory?
> 
>>
>> :( :( :(
>>
>>> In the guest (prior to Trusted I/O / TDISP), decrypted (or shared) memory
>>> is used because a device cannot DMA to or from guest memory using the
>>> guest encryption key. So all DMA must go to "decrypted" memory or be
>>> bounce-buffered through "decrypted" memory (SWIOTLB) - basically memory
>>> that does not get encrypted/decrypted using the guest encryption key.
>>
>> Where is the code for this? As I wrote we always do sme_set in the
>> iommu driver, even on guests, even for "decrypted" bounce buffered
>> memory.
>>
>> That sounds like a bug by your explanation?
>>
>> Does this mean vIOMMU has never worked in AMD CC guests?
> 
> I assume by vIOMMU you mean a VMM-emulated IOMMU in the guest. This does
> does not work today with AMD CC guests since it requires the hypervisor to
> read the guest IOMMU buffers in order to emulate the behavior and those
> buffers are encrypted. So there is no vIOMMU support today in AMD CC
> guests.
> 
> There was a patch series submitted a while back to allocate the IOMMU
> buffers in shared memory in order to support a (non-secure) vIOMMU in the
> guest in order to support >255 vCPUs, but that was rejected in favor of
> using kvm-msi-ext-dest-id.
> 
> https://lore.kernel.org/linux-iommu/20240430152430.4245-1-suravee.suthikulpanit@amd.com/
> 
>>
>>> It is not until we get to Trusted I/O / TDISP where devices will be able
>>> to DMA directly to guest encrypted memory and guests will require secure
>>> MMIO addresses which will need the encryption bit set (Alexey can correct
>>> me on the TIO statements if they aren't correct, as he is closer to it all).
>>
>> So in this case we do need to do sme_set on MMIO even though that MMIO
>> is not using the dram encryption key?

Yes. Its mapped to GPA (at least IOMMU VF MMIO BAR, I believe its same for TIO
device as well) and we need to set 'C' bit.

-Vasan



> 
> @Alexey will be able to provide more details on how this works.
> 
> Thanks,
> Tom
> 
>>
>> Jason
>