Re: [RFC PATCH v1 04/38] tsm: Support DMA Allocation from private memory

[Alexey Kardashevskiy <aik@amd.com>]

On 28/7/25 20:03, Jason Gunthorpe wrote:
> On Mon, Jul 28, 2025 at 07:21:41PM +0530, Aneesh Kumar K.V (Arm) wrote:
>> @@ -48,3 +49,12 @@ int set_memory_decrypted(unsigned long addr, int numpages)
>>   	return crypt_ops->decrypt(addr, numpages);
>>   }
>>   EXPORT_SYMBOL_GPL(set_memory_decrypted);
>> +
>> +bool force_dma_unencrypted(struct device *dev)
>> +{
>> +	if (dev->tdi_enabled)
>> +		return false;
> 
> Is this OK? I see code like this:
> 
> static inline dma_addr_t phys_to_dma_direct(struct device *dev,
> 		phys_addr_t phys)
> {
> 	if (force_dma_unencrypted(dev))
> 		return phys_to_dma_unencrypted(dev, phys);
> 	return phys_to_dma(dev, phys);

I did write this in the first place so I'll comment :)

> 
> What are the ARM rules for generating dma addreses?
> 
> 1) Device is T=0, memory is unencrypted, call dma_addr_unencrypted()
>     and do "top bit IBA set"
> 
> 2) Device is T=1, memory is encrypted, use the phys_to_dma() normally
> 
> 3) Device it T=1, memory is uncrypted, use the phys_to_dma()
>     normally??? Seems odd, I would have guessed the DMA address sould
>     be the same as case #1?
> 
> Can you document this in a comment?


On AMD, T=1 only encrypts the PCIe trafic, when a DMA request hits the IOMMU, the IOMMU decrypts it and then decides whether to encrypt it with a memory key: if there is secure vIOMMU - it will do what Cbit says in the guest IOMMU table (this is in the works) oooor just always set Cbit without guest vIOMMU (which is a big knob per a device and this is what my patches do now).

And with vIOMMU, I'd expect phys_to_dma_direct() not to be called as this one is in a direct map path.

> 
>> diff --git a/include/linux/device.h b/include/linux/device.h
>> index 4940db137fff..d62e0dd9d8ee 100644
>> --- a/include/linux/device.h
>> +++ b/include/linux/device.h
>> @@ -688,6 +688,7 @@ struct device {
>>   #ifdef CONFIG_IOMMU_DMA
>>   	bool			dma_iommu:1;
>>   #endif
>> +	bool			tdi_enabled:1;
>>   };
> 
> I would give the dev->tdi_enabled a clearer name, maybe
> dev->encrypted_dma_supported ?


May be but "_enabled", not "_supported". And, ideally, with vIOMMU, at least AMD won't be needing it.

> 
> Also need to think carefully of a bitfield is OK here, we can't
> locklessly change a bitfield so need to audit that all members are set
> under, probably, the device lock or some other single threaded hand
> waving. It seems believable it is like that but should be checked out,
> and add a lockdep if it relies on the device lock.

True.

> 
> Jason
> 

-- 
Alexey