Re: [RFC PATCH 08/12] vfio/pci: Create host unaccessible dma-buf for private device

[Alexey Kardashevskiy <aik@amd.com>]

On 18/1/25 00:25, Jason Gunthorpe wrote:
> On Fri, Jan 17, 2025 at 09:57:40AM +0800, Baolu Lu wrote:
>> On 1/15/25 21:01, Jason Gunthorpe wrote:
>>> On Wed, Jan 15, 2025 at 11:57:05PM +1100, Alexey Kardashevskiy wrote:
>>>> On 15/1/25 00:35, Jason Gunthorpe wrote:
>>>>> On Tue, Jun 18, 2024 at 07:28:43AM +0800, Xu Yilun wrote:
>>>>>
>>>>>>> is needed so the secure world can prepare anything it needs prior to
>>>>>>> starting the VM.
>>>>>> OK. From Dan's patchset there are some touch point for vendor tsm
>>>>>> drivers to do secure world preparation. e.g. pci_tsm_ops::probe().
>>>>>>
>>>>>> Maybe we could move to Dan's thread for discussion.
>>>>>>
>>>>>> https://lore.kernel.org/linux-
>>>>>> coco/173343739517.1074769.13134786548545925484.stgit@dwillia2-
>>>>>> xfh.jf.intel.com/
>>>>> I think Dan's series is different, any uapi from that series should
>>>>> not be used in the VMM case. We need proper vfio APIs for the VMM to
>>>>> use. I would expect VFIO to be calling some of that infrastructure.
>>>> Something like this experiment?
>>>>
>>>> https://github.com/aik/linux/commit/
>>>> ce052512fb8784e19745d4cb222e23cabc57792e
>>> Yeah, maybe, though I don't know which of vfio/iommufd/kvm should be
>>> hosting those APIs, the above does seem to be a reasonable direction.
>>>
>>> When the various fds are closed I would expect the kernel to unbind
>>> and restore the device back.
>>
>> I am curious about the value of tsm binding against an iomnufd_vdevice
>> instead of the physical iommufd_device.
> 
> Interesting question
>   
>> It is likely that the kvm pointer should be passed to iommufd during the
>> creation of a viommu object.
> 
> Yes, I fully expect this
> 
>> If my recollection is correct, the arm
>> smmu-v3 needs it to obtain the vmid to setup the userspace event queue:
> 
> Right now it will use a VMID unrelated to KVM. BTM support on ARM will
> require syncing the VMID with KVM.
> 
> AMD and Intel may require the KVM for some reason as well.
> 
> For CC I'm expecting the KVM fd to be the handle for the cVM, so any
> RPCs that want to call into the secure world need the KVM FD to get
> the cVM's identifier. Ie a "bind to cVM" RPC will need the PCI
> information and the cVM's handle.

And keep KVM fd open until unbind? Or just for the short time to call 
the PSP?

>  From that perspective it does make sense that any cVM related APIs,
> like "bind to cVM" would be against the VDEVICE where we have a link
> to the VIOMMU which has the KVM. On the iommufd side the VIOMMU is
> part of the object hierarchy, but does not necessarily have to force a
> vIOMMU to appear in the cVM.

Well, in my sketch it "appears" as an ability to make GUEST TIO REQUEST 
calls (guest <-> secure FW protocol).

> But it also seems to me that VFIO should be able to support putting
> the device into the RUN state without involving KVM or cVMs.

AMD's TDI bind handler in the PSP wants a guest handle ("GCTX") and a 
guest device BDFn, and VFIO has no desire to dive into this KVM business 
beyond IOMMUFD.

And then this GUEST TIO REQUEST which is used for 1) enabling secure 
part of IOMMU (so it relates to IOMMUFD)  2) enabling secure MMIO (which 
is more VFIO business).

We can do all sorts of things but the lifetime of these entangled 
objects is tricky sometimes. Thanks,


>> Intel TDX connect implementation also needs a reference to the kvm
>> pointer to obtain the secure EPT information. This is crucial because
>> the CPU's page table must be shared with the iommu.
> 
> I thought kvm folks were NAKing this sharing entirely? Or is the
> secure EPT in the secure world and not directly managed by Linux?
> 
> AFAIK AMD is going to mirror the iommu page table like today.
> 
> ARM, I suspect, will not have an "EPT" under Linux control, so
> whatever happens will be hidden in their secure world.
> 
> Jason

-- 
Alexey