Re: [PATCH v2] vhost: introduce mdev based hardware backend

[Jason Wang <jasowang@redhat.com>]

On 2019/10/24 下午4:03, Jason Wang wrote:
>
> On 2019/10/24 下午12:21, Tiwei Bie wrote:
>> On Wed, Oct 23, 2019 at 06:29:21PM +0800, Jason Wang wrote:
>>> On 2019/10/23 下午6:11, Tiwei Bie wrote:
>>>> On Wed, Oct 23, 2019 at 03:25:00PM +0800, Jason Wang wrote:
>>>>> On 2019/10/23 下午3:07, Tiwei Bie wrote:
>>>>>> On Wed, Oct 23, 2019 at 01:46:23PM +0800, Jason Wang wrote:
>>>>>>> On 2019/10/23 上午11:02, Tiwei Bie wrote:
>>>>>>>> On Tue, Oct 22, 2019 at 09:30:16PM +0800, Jason Wang wrote:
>>>>>>>>> On 2019/10/22 下午5:52, Tiwei Bie wrote:
>>>>>>>>>> This patch introduces a mdev based hardware vhost backend.
>>>>>>>>>> This backend is built on top of the same abstraction used
>>>>>>>>>> in virtio-mdev and provides a generic vhost interface for
>>>>>>>>>> userspace to accelerate the virtio devices in guest.
>>>>>>>>>>
>>>>>>>>>> This backend is implemented as a mdev device driver on top
>>>>>>>>>> of the same mdev device ops used in virtio-mdev but using
>>>>>>>>>> a different mdev class id, and it will register the device
>>>>>>>>>> as a VFIO device for userspace to use. Userspace can setup
>>>>>>>>>> the IOMMU with the existing VFIO container/group APIs and
>>>>>>>>>> then get the device fd with the device name. After getting
>>>>>>>>>> the device fd of this device, userspace can use vhost ioctls
>>>>>>>>>> to setup the backend.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
>>>>>>>>>> ---
>>>>>>>>>> This patch depends on below series:
>>>>>>>>>> https://lkml.org/lkml/2019/10/17/286
>>>>>>>>>>
>>>>>>>>>> v1 -> v2:
>>>>>>>>>> - Replace _SET_STATE with _SET_STATUS (MST);
>>>>>>>>>> - Check status bits at each step (MST);
>>>>>>>>>> - Report the max ring size and max number of queues (MST);
>>>>>>>>>> - Add missing MODULE_DEVICE_TABLE (Jason);
>>>>>>>>>> - Only support the network backend w/o multiqueue for now;
>>>>>>>>> Any idea on how to extend it to support devices other than 
>>>>>>>>> net? I think we
>>>>>>>>> want a generic API or an API that could be made generic in the 
>>>>>>>>> future.
>>>>>>>>>
>>>>>>>>> Do we want to e.g having a generic vhost mdev for all kinds of 
>>>>>>>>> devices or
>>>>>>>>> introducing e.g vhost-net-mdev and vhost-scsi-mdev?
>>>>>>>> One possible way is to do what vhost-user does. I.e. Apart from
>>>>>>>> the generic ring, features, ... related ioctls, we also introduce
>>>>>>>> device specific ioctls when we need them. As vhost-mdev just needs
>>>>>>>> to forward configs between parent and userspace and even won't
>>>>>>>> cache any info when possible,
>>>>>>> So it looks to me this is only possible if we expose e.g 
>>>>>>> set_config and
>>>>>>> get_config to userspace.
>>>>>> The set_config and get_config interface isn't really everything
>>>>>> of device specific settings. We also have ctrlq in virtio-net.
>>>>> Yes, but it could be processed by the exist API. Isn't it? Just 
>>>>> set ctrl vq
>>>>> address and let parent to deal with that.
>>>> I mean how to expose ctrlq related settings to userspace?
>>>
>>> I think it works like:
>>>
>>> 1) userspace find ctrl_vq is supported
>>>
>>> 2) then it can allocate memory for ctrl vq and set its address through
>>> vhost-mdev
>>>
>>> 3) userspace can populate ctrl vq itself
>> I see. That is to say, userspace e.g. QEMU will program the
>> ctrl vq with the existing VHOST_*_VRING_* ioctls, and parent
>> drivers should know that the addresses used in ctrl vq are
>> host virtual addresses in vhost-mdev's case.
>
>
> That's really good point. And that means parent needs to differ vhost 
> from virtio. It should work. 


HVA may only work when we have something similar to VHOST_SET_OWNER 
which can reuse MM of its owner.


> But is there any chance to use DMA address? I'm asking since the API 
> then tends to be device specific. 


I wonder whether we can introduce MAP IOMMU notifier and get DMA 
mappings from that.

Thanks