[PATCH 0/2] vhost-scsi: preserve event ordering and fix fallback deadlock

[PATCH 0/2] vhost-scsi: preserve event ordering and fix fallback deadlock

[physicalmtea]

From: Jia Jia <physicalmtea@gmail.com>

The vhost-scsi event list is populated with llist_add(), but completion
walks the detached list without reversing it.

Trigger: concurrent hotplug (ln -s). Each event is inserted at the
head of the pending list. Multiple events must already be stacked on
the llist before the vhost worker runs
vhost_scsi_complete_events(false). The whole list is then detached
and walked directly, so later-enqueued events are delivered to the
guest first.

Queued hotplug and hotunplug events can therefore be delivered in
reverse order.  Fix that ordering first, then fix the fallback path
that can relock the event virtqueue mutex while already holding it.

Patch 2 is based on patch 1 because both changes update the event
completion path. The first patch is otherwise independent of the
fallback deadlock.

Sashiko AI flagged this while reviewing
the vhost-scsi event queue fix.
This is a pre-existing self-deadlock. It was reproduced in a
follow-up test.

Trigger: vq->worker == NULL. vhost_vq_work_queue() then returns false,
and a subsequent vhost_scsi_do_plug() call deadlocks. I do not know what
normal condition gets us here; the normal vhost-scsi worker detach/reset
paths do not reach this code. The only reproduction I could come up with
was killing the vhost-scsi worker. This still looks like a low-probability
condition.

Jia Jia (2):
  vhost-scsi: preserve event ordering
  vhost-scsi: do not relock event vq mutex on send_evt fallback

 drivers/vhost/scsi.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

[PATCH 1/2] vhost-scsi: preserve event ordering

[physicalmtea]

From: Jia Jia <physicalmtea@gmail.com>

vhost_scsi_send_evt() uses llist_add(), so pending events are LIFO.
vhost_scsi_complete_events() walks the detached list directly.
This can deliver a later hotplug or hotunplug event before an earlier
one.

Reverse the detached list before processing it.
This keeps delivery in enqueue order.

Fixes: a6c9af87363c ("tcm_vhost: Add hotplug/hotunplug support")
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
---
 drivers/vhost/scsi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 7a1f39a32..37f681aab 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -637,7 +637,7 @@ static void vhost_scsi_complete_events(struct vhost_scsi *vs, bool drop)
 	struct llist_node *llnode;
 
 	mutex_lock(&vq->mutex);
-	llnode = llist_del_all(&vs->vs_event_list);
+	llnode = llist_reverse_order(llist_del_all(&vs->vs_event_list));
 	llist_for_each_entry_safe(evt, t, llnode, list) {
 		if (!drop)
 			vhost_scsi_do_evt_work(vs, evt);
-- 
2.34.1

Re: [PATCH 1/2] vhost-scsi: preserve event ordering

[Mike Christie]

On 9/15/26 4:42 AM, physicalmtea@gmail.com wrote:
> diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
> index 7a1f39a32..37f681aab 100644
> --- a/drivers/vhost/scsi.c
> +++ b/drivers/vhost/scsi.c
> @@ -637,7 +637,7 @@ static void vhost_scsi_complete_events(struct vhost_scsi *vs, bool drop)
>   	struct llist_node *llnode;
>   
>   	mutex_lock(&vq->mutex);
> -	llnode = llist_del_all(&vs->vs_event_list);
> +	llnode = llist_reverse_order(llist_del_all(&vs->vs_event_list));
>   	llist_for_each_entry_safe(evt, t, llnode, list) {
>   		if (!drop)
>   			vhost_scsi_do_evt_work(vs, evt);
> -- 
> 2.34.1
> 
> 

Reviewed-by: Mike Christie <michael.christie@oracle.com>

[PATCH 2/2] vhost-scsi: do not relock event vq mutex on send_evt fallback

[physicalmtea]

From: Jia Jia <physicalmtea@gmail.com>

vhost_scsi_send_evt() is called with the event virtqueue mutex held.
If the worker is gone, the fallback currently calls
vhost_scsi_complete_events(), which tries to acquire the same mutex again
and deadlocks the caller.

Split event completion into a helper for callers that already hold the
mutex and a locking wrapper for the event worker. Use the helper on the
fallback path.

Link: https://lore.kernel.org/all/20260905005352.1E5B01F00A3D@smtp.kernel.org/
Fixes: b1b2ce58ed23 ("vhost-scsi: Handle vhost_vq_work_queue failures for events")
Signed-off-by: Jia Jia <physicalmtea@gmail.com>
Reviewed-by: Mike Christie <michael.christie@oracle.com>
---
 drivers/vhost/scsi.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 37f681aab..9cd181cb1 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -630,19 +630,26 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt)
 	vhost_scsi_log_write(vq, vq_log, log_num);
 }
 
-static void vhost_scsi_complete_events(struct vhost_scsi *vs, bool drop)
+/* Caller must hold the event virtqueue mutex. */
+static void __vhost_scsi_complete_events(struct vhost_scsi *vs, bool drop)
 {
-	struct vhost_virtqueue *vq = &vs->vqs[VHOST_SCSI_VQ_EVT].vq;
 	struct vhost_scsi_evt *evt, *t;
 	struct llist_node *llnode;
 
-	mutex_lock(&vq->mutex);
 	llnode = llist_reverse_order(llist_del_all(&vs->vs_event_list));
 	llist_for_each_entry_safe(evt, t, llnode, list) {
 		if (!drop)
 			vhost_scsi_do_evt_work(vs, evt);
 		vhost_scsi_free_evt(vs, evt);
 	}
+}
+
+static void vhost_scsi_complete_events(struct vhost_scsi *vs, bool drop)
+{
+	struct vhost_virtqueue *vq = &vs->vqs[VHOST_SCSI_VQ_EVT].vq;
+
+	mutex_lock(&vq->mutex);
+	__vhost_scsi_complete_events(vs, drop);
 	mutex_unlock(&vq->mutex);
 }
 
@@ -1829,7 +1836,7 @@ vhost_scsi_send_evt(struct vhost_scsi *vs, struct vhost_virtqueue *vq,
 
 	llist_add(&evt->list, &vs->vs_event_list);
 	if (!vhost_vq_work_queue(vq, &vs->vs_event_work))
-		vhost_scsi_complete_events(vs, true);
+		__vhost_scsi_complete_events(vs, true);
 }
 
 static void vhost_scsi_evt_handle_kick(struct vhost_work *work)
-- 
2.34.1