From: Tony Krowiak <akrowiak@linux.ibm.com>
To: Halil Pasic <pasic@linux.ibm.com>
Cc: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
kvm@vger.kernel.org, jjherne@linux.ibm.com, freude@linux.ibm.com,
borntraeger@de.ibm.com, cohuck@redhat.com,
mjrosato@linux.ibm.com, alex.williamson@redhat.com,
kwankhede@nvidia.com, fiuczy@linux.ibm.com
Subject: Re: [PATCH v17 08/15] s390/vfio-ap: keep track of active guests
Date: Tue, 11 Jan 2022 17:19:35 -0500 [thread overview]
Message-ID: <f3696ce0-5a9f-fefb-5606-055139866d99@linux.ibm.com> (raw)
In-Reply-To: <fcce7cc6-6ac7-b22a-a957-80e59a0f4e83@linux.ibm.com>
On 1/11/22 16:58, Tony Krowiak wrote:
>
>
> On 12/29/21 22:33, Halil Pasic wrote:
>> On Thu, 21 Oct 2021 11:23:25 -0400
>> Tony Krowiak <akrowiak@linux.ibm.com> wrote:
>>
>>> The vfio_ap device driver registers for notification when the
>>> pointer to
>>> the KVM object for a guest is set. Let's store the KVM pointer as
>>> well as
>>> the pointer to the mediated device when the KVM pointer is set.
>> [..]
>>
>>
>>> struct ap_matrix_dev {
>>> ...
>>> struct rw_semaphore guests_lock;
>>> struct list_head guests;
>>> ...
>>> }
>>>
>>> The 'guests_lock' field is a r/w semaphore to control access to the
>>> 'guests' field. The 'guests' field is a list of ap_guest
>>> structures containing the KVM and matrix_mdev pointers for each active
>>> guest. An ap_guest structure will be stored into the list whenever the
>>> vfio_ap device driver is notified that the KVM pointer has been set and
>>> removed when notified that the KVM pointer has been cleared.
>>>
>> Is this about the field or about the list including all the nodes? This
>> reads lie guests_lock only protects the head element, which makes no
>> sense to me. Because of how these lists work.
>
> It locks the list, I can rewrite the description.
Ignore this response and read the answers to your comments below.
>
>
>>
>> The narrowest scope that could make sense is all the list_head stuff
>> in the entire list. I.e. one would only need the lock to traverse or
>> manipulate the list, while the payload would still be subject to
>> the matrix_dev->lock mutex.
>
> The matrix_dev->guests lock is needed whenever the kvm->lock
> is needed because the struct ap_guest object is created and the
> struct kvm assigned to it when the kvm pointer is set
> (vfio_ap_mdev_set_kvm function). So, in order to access the
> ap_guest object and retrieve the kvm pointer, we have to ensure
> the ap_guest_object is still available. The fact we can get the
> kvm pointer from the ap_matrix_mdev object just makes things
> more efficient - i.e., we won't have to traverse the list.
>
> Whenever the kvm->lock and matrix_dev->lock mutexes must
> be held, the order is:
>
> matrix_dev->guests_lock
> matrix_dev->guests->kvm->lock
> matrix_dev->lock
>
> There are times where all three locks are not required; for example,
> the handle_pqap and vfio_ap_mdev_probe/remove functions only
> require the matrix_dev->lock because it does not need to lock kvm.
>
>>
>> [..]
>>
>>> +struct ap_guest {
>>> + struct kvm *kvm;
>>> + struct list_head node;
>>> +};
>>> +
>>> /**
>>> * struct ap_matrix_dev - Contains the data for the matrix device.
>>> *
>>> @@ -39,6 +44,9 @@
>>> * single ap_matrix_mdev device. It's quite coarse but we
>>> don't
>>> * expect much contention.
>>> * @vfio_ap_drv: the vfio_ap device driver
>>> + * @guests_lock: r/w semaphore for protecting access to @guests
>>> + * @guests: list of guests (struct ap_guest) using AP devices
>>> bound to the
>>> + * vfio_ap device driver.
>> Please compare the above. Also if it is only about the access to the
>> list, then you could drop the lock right after create, and not keep it
>> till the very end of vfio_ap_mdev_set_kvm(). Right?
>
> That would be true if it only controlled access to the list, but as I
> explained above, that is not its sole purpose.
>
>>
>> In any case I'm skeptical about this whole struct ap_guest business. To
>> me, it looks like something that just makes things more obscure and
>> complicated without any real benefit.
>
> I'm open to other ideas, but you'll have to come up with a way
> to take the kvm->lock before the matrix_mdev->lock in the
> vfio_ap_mdev_probe_queue and vfio_ap_mdev_remove_queue
> functions where we don't have access to the ap_matrix_mdev
> object to which the APQN is assigned and has the pointer to the
> kvm object.
>
> In order to retrieve the matrix_mdev, we need the matrix_dev->lock.
> In order to hot plug/unplug the queue, we need the kvm->lock.
> There's your catch-22 that needs to be solved. This design is my
> attempt to solve that.
>
>>
>> Regards,
>> Halil
>>
>>> */
>>> struct ap_matrix_dev {
>>> struct device device;
>>> @@ -47,6 +55,8 @@ struct ap_matrix_dev {
>>> struct list_head mdev_list;
>>> struct mutex lock;
>>> struct ap_driver *vfio_ap_drv;
>>> + struct rw_semaphore guests_lock;
>>> + struct list_head guests;
>>> };
>>> extern struct ap_matrix_dev *matrix_dev;
>
next prev parent reply other threads:[~2022-01-11 22:19 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-21 15:23 [PATCH v17 00/15] s390/vfio-ap: dynamic configuration support Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 01/15] s390/vfio-ap: Set pqap hook when vfio_ap module is loaded Tony Krowiak
2021-12-27 8:21 ` Halil Pasic
2022-01-11 21:13 ` Tony Krowiak
2022-01-04 16:22 ` Jason J. Herne
2022-01-11 17:29 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 02/15] s390/vfio-ap: use new AP bus interface to search for queue devices Tony Krowiak
2021-12-27 8:25 ` Halil Pasic
2022-01-11 17:32 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 03/15] s390/vfio-ap: move probe and remove callbacks to vfio_ap_ops.c Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 04/15] s390/vfio-ap: manage link between queue struct and matrix mdev Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 05/15] s390/vfio-ap: introduce shadow APCB Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 06/15] s390/vfio-ap: refresh guest's APCB by filtering APQNs assigned to mdev Tony Krowiak
2021-12-27 8:53 ` Halil Pasic
2022-01-11 21:19 ` Tony Krowiak
2022-01-12 11:52 ` Halil Pasic
2022-01-15 0:31 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 07/15] s390/vfio-ap: allow assignment of unavailable AP queues to mdev device Tony Krowiak
2021-12-27 9:06 ` Halil Pasic
2021-10-21 15:23 ` [PATCH v17 08/15] s390/vfio-ap: keep track of active guests Tony Krowiak
2021-12-30 2:04 ` Halil Pasic
2022-01-11 21:27 ` Tony Krowiak
2021-12-30 3:33 ` Halil Pasic
2022-01-11 21:58 ` Tony Krowiak
2022-01-11 22:19 ` Tony Krowiak [this message]
2022-01-12 14:25 ` Halil Pasic
2022-01-15 0:29 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 09/15] s390/vfio-ap: allow hot plug/unplug of AP resources using mdev device Tony Krowiak
2022-01-09 21:36 ` Halil Pasic
2022-01-11 22:42 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 10/15] s390/vfio-ap: reset queues after adapter/domain unassignment Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 11/15] s390/ap: driver callback to indicate resource in use Tony Krowiak
2021-11-04 11:27 ` Harald Freudenberger
2021-11-04 15:48 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 12/15] s390/vfio-ap: implement in-use callback for vfio_ap driver Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 13/15] s390/vfio-ap: sysfs attribute to display the guest's matrix Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 14/15] s390/ap: notify drivers on config changed and scan complete callbacks Tony Krowiak
2021-11-04 12:06 ` Harald Freudenberger
2021-11-04 15:50 ` Tony Krowiak
2021-11-05 8:23 ` Harald Freudenberger
2021-11-05 13:15 ` Harald Freudenberger
2021-11-08 14:27 ` Tony Krowiak
2021-11-08 14:26 ` Tony Krowiak
2022-02-04 10:43 ` Halil Pasic
2022-02-07 19:39 ` Tony Krowiak
2022-02-08 1:38 ` Halil Pasic
2022-02-08 3:27 ` Tony Krowiak
2021-10-21 15:23 ` [PATCH v17 15/15] s390/vfio-ap: update docs to include dynamic config support Tony Krowiak
2021-10-27 14:24 ` [PATCH v17 00/15] s390/vfio-ap: dynamic configuration support Tony Krowiak
2021-11-02 19:23 ` Tony Krowiak
2021-11-15 15:45 ` Tony Krowiak
2021-11-22 16:12 ` Tony Krowiak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f3696ce0-5a9f-fefb-5606-055139866d99@linux.ibm.com \
--to=akrowiak@linux.ibm.com \
--cc=alex.williamson@redhat.com \
--cc=borntraeger@de.ibm.com \
--cc=cohuck@redhat.com \
--cc=fiuczy@linux.ibm.com \
--cc=freude@linux.ibm.com \
--cc=jjherne@linux.ibm.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mjrosato@linux.ibm.com \
--cc=pasic@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®