From: Jason Wang <jasowang@redhat.com>
To: Yongji Xie <xieyongji@bytedance.com>
Cc: mst <mst@redhat.com>, Liu Xiaodong <xiaodong.liu@intel.com>,
Maxime Coquelin <maxime.coquelin@redhat.com>,
Stefan Hajnoczi <stefanha@redhat.com>,
songmuchun@bytedance.com,
virtualization <virtualization@lists.linux-foundation.org>,
linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 0/5] VDUSE: Support registering userspace memory as bounce buffer
Date: Thu, 14 Jul 2022 10:59:17 +0800 [thread overview]
Message-ID: <07a8e952-1347-8920-3a78-0200b865e0e9@redhat.com> (raw)
In-Reply-To: <CACycT3vgaOrLVq+GDRK1PqqBRCkUAU0bYH=2CDvudsX0F9FBDA@mail.gmail.com>
在 2022/7/11 15:24, Yongji Xie 写道:
> On Mon, Jul 11, 2022 at 2:02 PM Jason Wang <jasowang@redhat.com> wrote:
>> On Fri, Jul 8, 2022 at 5:53 PM Yongji Xie <xieyongji@bytedance.com> wrote:
>>> On Fri, Jul 8, 2022 at 4:38 PM Jason Wang <jasowang@redhat.com> wrote:
>>>> On Wed, Jul 6, 2022 at 6:16 PM Yongji Xie <xieyongji@bytedance.com> wrote:
>>>>> On Wed, Jul 6, 2022 at 5:30 PM Jason Wang <jasowang@redhat.com> wrote:
>>>>>> On Wed, Jul 6, 2022 at 1:05 PM Xie Yongji <xieyongji@bytedance.com> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> This series introduces some new ioctls: VDUSE_IOTLB_GET_INFO,
>>>>>>> VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM to support
>>>>>>> registering and de-registering userspace memory for IOTLB
>>>>>>> as bounce buffer in virtio-vdpa case.
>>>>>>>
>>>>>>> The VDUSE_IOTLB_GET_INFO ioctl can help user to query IOLTB
>>>>>>> information such as bounce buffer size. Then user can use
>>>>>>> those information on VDUSE_IOTLB_REG_UMEM and
>>>>>>> VDUSE_IOTLB_DEREG_UMEM ioctls to register and de-register
>>>>>>> userspace memory for IOTLB.
>>>>>>>
>>>>>>> During registering and de-registering, the DMA data in use
>>>>>>> would be copied from kernel bounce pages to userspace bounce
>>>>>>> pages and back.
>>>>>>>
>>>>>>> With this feature, some existing application such as SPDK
>>>>>>> and DPDK can leverage the datapath of VDUSE directly and
>>>>>>> efficiently as discussed before [1][2]. They can register
>>>>>>> some preallocated hugepages to VDUSE to avoid an extra
>>>>>>> memcpy from bounce-buffer to hugepages.
>>>>>> This is really interesting.
>>>>>>
>>>>>> But a small concern on uAPI is that this seems to expose the VDUSE
>>>>>> internal implementation (bounce buffer) to userspace. We tried hard to
>>>>>> hide it via the GET_FD before. Anyway can we keep it?
>>>>>>
>>>>> Another way is changing GET_FD ioctl to add a flag or reuse 'perm'
>>>>> field to indicate whether a IOVA region supports userspace memory
>>>>> registration. Then userspace can use
>>>>> VDUSE_IOTLB_REG_UMEM/VDUSE_IOTLB_DEREG_UMEM to register/deregister
>>>>> userspace memory for this IOVA region.
>>>> Looks better.
>>>>
>>> OK.
>>>
>>>>> Any suggestions?
>>>> I wonder what's the value of keeping the compatibility with the kernel
>>>> mmaped bounce buffer. It means we need to take extra care on e.g data
>>>> copying when reg/reg user space memory.
>>>>
>>> I'm not sure I get your point on the compatibility with the kernel
>>> bounce buffer. Do you mean they use the same iova region?
>> Yes.
>>
>>> The userspace daemon might crash or reboot. In those cases, we still
>>> need a kernel buffer to store/recover the data.
>> Yes, this should be a good point.
>>
>>>> Can we simply allow the third kind of fd that only works for umem registration?
>>>>
>>> Do you mean using another iova region for umem?
>> I meant having a new kind of fd that only allows umem registration.
>>
> OK. It seems to be a little complicated to allow mapping a registered
> user memory via a new fd, e.g. how to handle the mapping if the
> userspace daemon exits but the fd is already passed to another
> process.
>
>>> I think we don't need
>>> a fd in umem case since the userspace daemon can access the memory
>>> directly without using mmap() to map it into the address space in
>>> advance.
>> Ok, I will have a look at the code and get back.
>>
> OK. Looking forward to your reply.
Looks good overall.
Just few comments.
Thanks
>
> Thanks,
> Yongji
>
prev parent reply other threads:[~2022-07-14 2:59 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-06 5:04 Xie Yongji
2022-07-06 5:04 ` [PATCH v2 1/5] vduse: Remove unnecessary spin lock protection Xie Yongji
2022-07-13 5:43 ` Jason Wang
2022-07-13 11:08 ` Yongji Xie
2022-07-14 2:27 ` Jason Wang
2022-07-14 6:06 ` Yongji Xie
2022-07-13 5:57 ` Michael S. Tsirkin
2022-07-13 10:39 ` Yongji Xie
2022-07-06 5:05 ` [PATCH v2 2/5] vduse: Use memcpy_{to,from}_page() in do_bounce() Xie Yongji
2022-07-13 5:59 ` Jason Wang
2022-07-06 5:05 ` [PATCH v2 3/5] vduse: Support using userspace pages as bounce buffer Xie Yongji
2022-07-06 5:05 ` [PATCH v2 4/5] vduse: Support querying IOLTB information Xie Yongji
2022-07-14 2:51 ` Jason Wang
2022-07-14 5:58 ` Yongji Xie
2022-07-06 5:05 ` [PATCH v2 5/5] vduse: Support registering userspace memory for IOTLB Xie Yongji
2022-07-06 6:08 ` kernel test robot
2022-07-07 23:01 ` kernel test robot
2022-07-08 1:14 ` kernel test robot
2022-07-06 9:30 ` [PATCH v2 0/5] VDUSE: Support registering userspace memory as bounce buffer Jason Wang
2022-07-06 10:15 ` Yongji Xie
2022-07-08 8:38 ` Jason Wang
2022-07-08 9:53 ` Yongji Xie
2022-07-11 6:02 ` Jason Wang
2022-07-11 7:24 ` Yongji Xie
2022-07-14 2:59 ` Jason Wang [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=07a8e952-1347-8920-3a78-0200b865e0e9@redhat.com \
--to=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=maxime.coquelin@redhat.com \
--cc=mst@redhat.com \
--cc=songmuchun@bytedance.com \
--cc=stefanha@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=xiaodong.liu@intel.com \
--cc=xieyongji@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®