From: Jason Wang <jasowang@redhat.com>
To: Si-Wei Liu <si-wei.liu@oracle.com>,
Eugenio Perez Martin <eperezma@redhat.com>
Cc: virtualization <virtualization@lists.linux-foundation.org>,
kvm list <kvm@vger.kernel.org>,
"Michael S. Tsirkin" <mst@redhat.com>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
Stefano Garzarella <sgarzare@redhat.com>,
Longpeng <longpeng2@huawei.com>,
Zhu Lingshan <lingshan.zhu@intel.com>,
Martin Petrus Hubertus Habets <martinh@xilinx.com>,
Harpreet Singh Anand <hanand@xilinx.com>,
dinang@xilinx.com, Eli Cohen <elic@nvidia.com>,
Laurent Vivier <lvivier@redhat.com>,
pabloc@xilinx.com, "Dawar, Gautam" <gautam.dawar@amd.com>,
Xie Yongji <xieyongji@bytedance.com>,
habetsm.xilinx@gmail.com,
Christophe JAILLET <christophe.jaillet@wanadoo.fr>,
tanuj.kamde@amd.com, Wu Zongyong <wuzongyong@linux.alibaba.com>,
martinpo@xilinx.com, Cindy Lu <lulu@redhat.com>,
ecree.xilinx@gmail.com, Parav Pandit <parav@nvidia.com>,
Dan Carpenter <dan.carpenter@oracle.com>,
Zhang Min <zhang.min9@zte.com.cn>
Subject: Re: [PATCH 1/4] vdpa: Add stop operation
Date: Tue, 24 May 2022 10:45:54 +0800 [thread overview]
Message-ID: <333e957d-af84-24fb-6636-843a9dcfc1e2@redhat.com> (raw)
In-Reply-To: <9f68802c-2692-7321-f916-670ee0abfc40@oracle.com>
在 2022/5/24 08:01, Si-Wei Liu 写道:
>
>
> On 5/23/2022 4:54 PM, Si-Wei Liu wrote:
>>
>>
>> On 5/23/2022 12:20 PM, Eugenio Perez Martin wrote:
>>> On Sat, May 21, 2022 at 12:13 PM Si-Wei Liu <si-wei.liu@oracle.com>
>>> wrote:
>>>>
>>>>
>>>> On 5/20/2022 10:23 AM, Eugenio Pérez wrote:
>>>>> This operation is optional: It it's not implemented, backend
>>>>> feature bit
>>>>> will not be exposed.
>>>>>
>>>>> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
>>>>> ---
>>>>> include/linux/vdpa.h | 6 ++++++
>>>>> 1 file changed, 6 insertions(+)
>>>>>
>>>>> diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
>>>>> index 15af802d41c4..ddfebc4e1e01 100644
>>>>> --- a/include/linux/vdpa.h
>>>>> +++ b/include/linux/vdpa.h
>>>>> @@ -215,6 +215,11 @@ struct vdpa_map_file {
>>>>> * @reset: Reset device
>>>>> * @vdev: vdpa device
>>>>> * Returns integer: success (0) or
>>>>> error (< 0)
>>>>> + * @stop: Stop or resume the device (optional,
>>>>> but it must
>>>>> + * be implemented if require device stop)
>>>>> + * @vdev: vdpa device
>>>>> + * @stop: stop (true), not stop (false)
>>>>> + * Returns integer: success (0) or
>>>>> error (< 0)
>>>> Is this uAPI meant to address all use cases described in the full
>>>> blown
>>>> _F_STOP virtio spec proposal, such as:
>>>>
>>>> --------------%<--------------
>>>>
>>>> ...... the device MUST finish any in flight
>>>> operations after the driver writes STOP. Depending on the device, it
>>>> can do it
>>>> in many ways as long as the driver can recover its normal operation
>>>> if it
>>>> resumes the device without the need of resetting it:
>>>>
>>>> - Drain and wait for the completion of all pending requests until a
>>>> convenient avail descriptor. Ignore any other posterior
>>>> descriptor.
>>>> - Return a device-specific failure for these descriptors, so the
>>>> driver
>>>> can choose to retry or to cancel them.
>>>> - Mark them as done even if they are not, if the kind of device can
>>>> assume to lose them.
>>>> --------------%<--------------
>>>>
>>> Right, this is totally underspecified in this series.
>>>
>>> I'll expand on it in the next version, but that text proposed to
>>> virtio-comment was complicated and misleading. I find better to get
>>> the previous version description. Would the next description work?
>>>
>>> ```
>>> After the return of ioctl, the device MUST finish any pending
>>> operations like
>>> in flight requests. It must also preserve all the necessary state (the
>>> virtqueue vring base plus the possible device specific states)
>> Hmmm, "possible device specific states" is a bit vague. Does it
>> require the device to save any device internal state that is not
>> defined in the virtio spec - such as any failed in-flight requests to
>> resubmit upon resume? Or you would lean on SVQ to intercept it in
>> depth and save it with some other means? I think network device also
>> has internal state such as flow steering state that needs bookkeeping
>> as well.
> Noted that I understand you may introduce additional feature call
> similar to VHOST_USER_GET_INFLIGHT_FD for (failed) in-flight request,
> but since that's is a get interface, I assume the actual state
> preserving should still take place in this STOP call.
>
Yes, I think so.
> -Siwei
>
>>
>> A follow-up question is what is the use of the `stop` argument of
>> false, does it require the device to support resume?
Yes, this is required by the hypervisor e.g for Qemu it supports vm
stop/resume.
>> I seem to recall this is something to abandon in favor of device
>> reset plus setting queue base/addr after. Or it's just a optional
>> feature that may be device specific (if one can do so in simple way).
Rest is more like a workarond consider we don't have a stop API.
Consider we don't add stop at the beginning, it can only be an optional
feature.
Thanks
>>
>> -Siwei
>>
>>> that is required
>>> for restoring in the future.
>>>
>>> In the future, we will provide features similar to
>>> VHOST_USER_GET_INFLIGHT_FD
>>> so the device can save pending operations.
>>> ```
>>>
>>> Thanks for pointing it out!
>>>
>>>
>>>
>>>
>>>
>>>> E.g. do I assume correctly all in flight requests are flushed after
>>>> return from this uAPI call? Or some of pending requests may be subject
>>>> to loss or failure? How does the caller/user specify these various
>>>> options (if there are) for device stop?
>>>>
>>>> BTW, it would be nice to add the corresponding support to vdpa_sim_blk
>>>> as well to demo the stop handling. To just show it on vdpa-sim-net
>>>> IMHO
>>>> is perhaps not so convincing.
>>>>
>>>> -Siwei
>>>>
>>>>> * @get_config_size: Get the size of the configuration space
>>>>> includes
>>>>> * fields that are conditional on
>>>>> feature bits.
>>>>> * @vdev: vdpa device
>>>>> @@ -316,6 +321,7 @@ struct vdpa_config_ops {
>>>>> u8 (*get_status)(struct vdpa_device *vdev);
>>>>> void (*set_status)(struct vdpa_device *vdev, u8 status);
>>>>> int (*reset)(struct vdpa_device *vdev);
>>>>> + int (*stop)(struct vdpa_device *vdev, bool stop);
>>>>> size_t (*get_config_size)(struct vdpa_device *vdev);
>>>>> void (*get_config)(struct vdpa_device *vdev, unsigned int
>>>>> offset,
>>>>> void *buf, unsigned int len);
>>
>
next prev parent reply other threads:[~2022-05-24 2:46 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-20 17:23 [PATCH 0/4] Implement vdpasim " Eugenio Pérez
2022-05-20 17:23 ` [PATCH 1/4] vdpa: Add " Eugenio Pérez
2022-05-21 10:13 ` Si-Wei Liu
2022-05-23 19:20 ` Eugenio Perez Martin
2022-05-23 23:54 ` Si-Wei Liu
2022-05-24 0:01 ` Si-Wei Liu
2022-05-24 2:45 ` Jason Wang [this message]
2022-05-24 7:38 ` Eugenio Perez Martin
2022-05-24 7:09 ` Stefano Garzarella
2022-05-24 7:42 ` Eugenio Perez Martin
2022-05-24 7:51 ` Stefano Garzarella
2022-05-20 17:23 ` [PATCH 2/4] vhost-vdpa: introduce STOP backend feature bit Eugenio Pérez
2022-05-21 10:24 ` Si-Wei Liu
2022-05-23 9:57 ` Eugenio Perez Martin
2022-05-20 17:23 ` [PATCH 3/4] vhost-vdpa: uAPI to stop the device Eugenio Pérez
2022-05-21 5:20 ` kernel test robot
2022-05-21 8:36 ` Martin Habets
2022-05-23 8:11 ` Eugenio Perez Martin
2022-05-20 17:23 ` [PATCH 4/4] vdpa_sim: Implement stop vdpa op Eugenio Pérez
2022-05-23 8:27 ` Stefano Garzarella
2022-05-23 19:07 ` Eugenio Perez Martin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=333e957d-af84-24fb-6636-843a9dcfc1e2@redhat.com \
--to=jasowang@redhat.com \
--cc=christophe.jaillet@wanadoo.fr \
--cc=dan.carpenter@oracle.com \
--cc=dinang@xilinx.com \
--cc=ecree.xilinx@gmail.com \
--cc=elic@nvidia.com \
--cc=eperezma@redhat.com \
--cc=gautam.dawar@amd.com \
--cc=habetsm.xilinx@gmail.com \
--cc=hanand@xilinx.com \
--cc=kvm@vger.kernel.org \
--cc=lingshan.zhu@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longpeng2@huawei.com \
--cc=lulu@redhat.com \
--cc=lvivier@redhat.com \
--cc=martinh@xilinx.com \
--cc=martinpo@xilinx.com \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabloc@xilinx.com \
--cc=parav@nvidia.com \
--cc=sgarzare@redhat.com \
--cc=si-wei.liu@oracle.com \
--cc=tanuj.kamde@amd.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=wuzongyong@linux.alibaba.com \
--cc=xieyongji@bytedance.com \
--cc=zhang.min9@zte.com.cn \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®