[RFC PATCH v3 0/6] scsi: release completed passthrough requests before user copyback

[RFC PATCH v3 0/6] scsi: release completed passthrough requests before user copyback

[Yingjie Wang]

This rerolls the request-lifetime series after the v2 review:

  https://lore.kernel.org/r/cover.1793003363.git.x-professor@outlook.com

The two BSG copyback fixes and James Houghton's cancellation fix that were
prerequisites for the previous draft are now in the SCSI fixes tree as
a52f2ad45726, ffb7b533a479 and 2b2bb55990cf.  This series is based on
04feb5fd2c23 from that tree.

The affected passthrough paths can copy data to userspace, or unmap a user
buffer that requires copyback, while still retaining a completed request.
If that access faults and the fault needs I/O from the same queue, the task
can wait for a tag held by the request it would release only after the fault
returns.

Changes since v2:

- 1/6 through 4/6 are unchanged apart from rebasing onto the merged
  prerequisites.
- 5/6 adds an io_uring helper that transfers the storage reserved for the
  command's copied SQE to the driver after every SQE field has been consumed.
  The transfer invalidates ioucmd->sqe, rejects multishot users, and makes a
  later -EAGAIN return fail instead of reissuing a command whose SQE storage
  has been overwritten.  It adds neither an allocation nor object growth.
- 6/6 uses 96 bytes of that storage for the bounded SCSI sense snapshot.
  End-I/O saves the result and sense data, queues task work, and returns
  RQ_END_IO_FREE for every completion.  Task work only uses the saved BIO,
  result and sense data; it never accesses the request or scsi_cmnd.  A
  canceled task-work callback or failed sense copy reports a zero sense
  length.

Compared with the v3 plan I posted on October 29, this removes the remaining
sense-bearing case that would have held its tag until task work:

  https://lore.kernel.org/r/20261029.scsi-lifetime-v3-plan.7ad5c4111d3b@outlook.com

It also avoids the extra per-command sense allocation discussed during v2
review.  I am marking this RFC because the new io_uring helper and its
no-reissue contract need review from the io_uring side.

Validation used the exact tip below:

- Each of the six intermediate commits builds independently with W=1 on
  x86-64.  The final tree also builds the affected objects on i386 and on
  x86-64 with CONFIG_IO_URING disabled and SCSI modular.
- A full x86-64 KASAN and lockdep bzImage builds successfully.
- In QEMU with virtio-scsi disk and CD-ROM devices, 20 regression iterations
  passed 960 assertions plus 6 linked-SQE assertions.  The linked test wrote
  a real 18-byte sense snapshot over the reused SQE storage before completing
  the linked command.
- Trace validation passed 100 synchronous and 20 deferred lifecycle windows.
  Every deferred window retained zero completed requests before task work.
  The run recorded 40 CDDA calls and no BUG, WARNING, KASAN, UBSAN or lockdep
  report.
- Four independent exact-image performance runs with successful depth-16
  INQUIRY and a 96-byte response buffer measured 12.825 us/request in normal
  mode and 13.004 us/request with DEFER_TASKRUN.  The existing baseline was
  13.726 and 12.559 us/request, while the sent v2 allocation measured 17.647
  and 16.665 us/request.  These debug-kernel virtual-device numbers show that
  the v2 allocation overhead is absent; their run-to-run spread does not
  support a general performance claim.

The i386 configuration was build-tested but not booted.  CDDA testing covers
the virtual device's expected error path, not audio extraction from real
media.  Transport BSG (block/bsg-lib.c) remains outside this series because
it uses a separate queue and needs an independent dependency analysis.

v1: https://lore.kernel.org/r/cover.1792237341.git.x-professor@outlook.com
Review follow-up: https://lore.kernel.org/r/20261022.scsi-copyback-review.31b8d6a70f4c@outlook.com
CDDA reply: https://lore.kernel.org/r/20261022.scsi-cdda-lifetime.8f31a2c97b42@outlook.com

Yingjie Wang (6):
  scsi: core: release SG_IO request before user copyback
  scsi: core: release legacy ioctl request before user copyback
  scsi: sr: release CDDA request before user copyback
  scsi: bsg: release synchronous request before user copyback
  io_uring/cmd: allow reusing SQE storage for completion data
  scsi: bsg: release io_uring requests from end-I/O

 drivers/scsi/scsi_bsg.c      | 116 +++++++++++++++++++++++------------
 drivers/scsi/scsi_ioctl.c    |  34 +++++++---
 drivers/scsi/sr.c            |   3 +
 include/linux/io_uring/cmd.h |  24 ++++++++
 io_uring/uring_cmd.c         |  22 +++++++
 5 files changed, 151 insertions(+), 48 deletions(-)


base-commit: 04feb5fd2c23894ebc2cdaf4a4cc38b8667b17ac
-- 
2.43.0

[RFC PATCH v3 1/6] scsi: core: release SG_IO request before user copyback

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

The block-device SG_IO path retains its completed request while copying
sense data and unmapping the user data buffer. Either operation can fault.
If the fault needs I/O from the same queue and completed requests retain
the available tags, the nested I/O cannot make progress.

Fill the kernel-resident output header and save the bounded sense data
before releasing the request. Then perform the user copies and unmap the
original BIO chain, preserving the existing sense-copy error precedence.
The saved mapping owns its BIO references independently of the request.

The fixed request pool was introduced by scsi-mq; the later move of this
ioctl implementation into drivers/scsi did not introduce the dependency.

Fixes: d285203cf647 ("scsi: add support for a blk-mq based I/O path.")
Link: https://lore.kernel.org/r/20261022.scsi-copyback-review.31b8d6a70f4c@outlook.com
Cc: stable@vger.kernel.org
Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 drivers/scsi/scsi_ioctl.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index bf049d9cb719..b588aa9059e3 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -398,6 +398,8 @@ static int scsi_complete_sghdr_rq(struct request *rq, struct sg_io_hdr *hdr,
 		struct bio *bio)
 {
 	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
+	u8 sense_buffer[SCSI_SENSE_BUFFERSIZE];
+	unsigned int sense_len = 0;
 	int r, ret = 0;
 
 	/*
@@ -417,10 +419,16 @@ static int scsi_complete_sghdr_rq(struct request *rq, struct sg_io_hdr *hdr,
 	hdr->sb_len_wr = 0;
 
 	if (scmd->sense_len && hdr->sbp) {
-		int len = min((unsigned int) hdr->mx_sb_len, scmd->sense_len);
+		sense_len = min3(hdr->mx_sb_len, scmd->sense_len,
+				 sizeof(sense_buffer));
+		memcpy(sense_buffer, scmd->sense_buffer, sense_len);
+	}
+
+	blk_mq_free_request(rq);
 
-		if (!copy_to_user(hdr->sbp, scmd->sense_buffer, len))
-			hdr->sb_len_wr = len;
+	if (sense_len) {
+		if (!copy_to_user(hdr->sbp, sense_buffer, sense_len))
+			hdr->sb_len_wr = sense_len;
 		else
 			ret = -EFAULT;
 	}
@@ -493,7 +501,7 @@ static int sg_io(struct scsi_device *sdev, struct sg_io_hdr *hdr,
 
 	hdr->duration = jiffies_to_msecs(jiffies - start_time);
 
-	ret = scsi_complete_sghdr_rq(rq, hdr, bio);
+	return scsi_complete_sghdr_rq(rq, hdr, bio);
 
 out_put_request:
 	blk_mq_free_request(rq);
-- 
2.43.0

[RFC PATCH v3 2/6] scsi: core: release legacy ioctl request before user copyback

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

SCSI_IOCTL_SEND_COMMAND copies sense or command data to userspace while
the completed request still occupies its queue tag. A fault that needs
the same queue can therefore wait for a resource held by the ioctl.

Save the command status and the compatibility-limited sense data, then
release the request before copying to userspace. The data buffer is a
separate allocation and remains valid until the existing buffer cleanup.
Keep the positive SCSI status and -EFAULT return conventions unchanged.

This complements the SG_IO fix by covering the legacy
SCSI_IOCTL_SEND_COMMAND path.

Fixes: d285203cf647 ("scsi: add support for a blk-mq based I/O path.")
Link: https://lore.kernel.org/r/20261022.scsi-copyback-review.31b8d6a70f4c@outlook.com
Cc: stable@vger.kernel.org
Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 drivers/scsi/scsi_ioctl.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/scsi_ioctl.c b/drivers/scsi/scsi_ioctl.c
index b588aa9059e3..4f97516c795c 100644
--- a/drivers/scsi/scsi_ioctl.c
+++ b/drivers/scsi/scsi_ioctl.c
@@ -538,6 +538,8 @@ static int sg_scsi_ioctl(struct request_queue *q, bool open_for_write,
 		struct scsi_ioctl_command __user *sic)
 {
 	struct request *rq;
+	u8 sense_buffer[16];
+	unsigned int sense_len = 0;
 	int err;
 	unsigned int in_len, out_len, bytes, opcode, cmdlen;
 	struct scsi_cmnd *scmd;
@@ -626,17 +628,21 @@ static int sg_scsi_ioctl(struct request_queue *q, bool open_for_write,
 	blk_execute_rq(rq, false);
 
 	err = scmd->result & 0xff;	/* only 8 bit SCSI status */
+	if (err && scmd->sense_len && scmd->sense_buffer) {
+		/* limit sense len for backward compatibility */
+		sense_len = min(scmd->sense_len, sizeof(sense_buffer));
+		memcpy(sense_buffer, scmd->sense_buffer, sense_len);
+	}
+	blk_mq_free_request(rq);
+
 	if (err) {
-		if (scmd->sense_len && scmd->sense_buffer) {
-			/* limit sense len for backward compatibility */
-			if (copy_to_user(sic->data, scmd->sense_buffer,
-					 min(scmd->sense_len, 16U)))
-				err = -EFAULT;
-		}
+		if (sense_len && copy_to_user(sic->data, sense_buffer, sense_len))
+			err = -EFAULT;
 	} else {
 		if (copy_to_user(sic->data, buffer, out_len))
 			err = -EFAULT;
 	}
+	goto error_free_buffer;
 
 error:
 	blk_mq_free_request(rq);
-- 
2.43.0

[RFC PATCH v3 3/6] scsi: sr: release CDDA request before user copyback

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

sr_read_cdda_bpc() retains its completed request while unmapping the user
buffer. For a bounced data-in mapping, unmapping can copy to userspace
and fault. A fault that needs I/O from the same queue cannot progress if
the completed request retains the available tag.

Release the request after consuming its command result and sense data,
before unmapping the original BIO chain saved at submission. No request
or SCSI command field is accessed after the release. The mapping's BIO
references remain owned by blk_rq_unmap_user().

The scsi-mq request pool introduced this tag dependency. The later move
of the CDDA implementation from drivers/cdrom into sr was a relocation.

Fixes: d285203cf647 ("scsi: add support for a blk-mq based I/O path.")
Link: https://lore.kernel.org/r/20261022.scsi-cdda-lifetime.8f31a2c97b42@outlook.com
Cc: stable@vger.kernel.org
Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 drivers/scsi/sr.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/scsi/sr.c b/drivers/scsi/sr.c
index 4f2e8d1be6da..c64c25e9f404 100644
--- a/drivers/scsi/sr.c
+++ b/drivers/scsi/sr.c
@@ -969,8 +969,11 @@ static int sr_read_cdda_bpc(struct cdrom_device_info *cdi, void __user *ubuf,
 		ret = -EIO;
 	}
 
+	blk_mq_free_request(rq);
 	if (blk_rq_unmap_user(bio))
 		ret = -EFAULT;
+	return ret;
+
 out_put_request:
 	blk_mq_free_request(rq);
 	return ret;
-- 
2.43.0

[RFC PATCH v3 4/6] scsi: bsg: release synchronous request before user copyback

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

The synchronous SCSI BSG ioctl copies sense data and unmaps its user
data buffer while retaining the completed request. Copyback can fault
and need I/O from the same queue, which cannot progress when completed
requests retain the available tags.

Fill the kernel-resident output fields and save the bounded sense data
before releasing the request. Perform both user copies afterward, using
the original BIO chain saved at mapping time. Preserve the copyback
error precedence established by the separate error-propagation fix.

The SG_IO v4 implementation predates scsi-mq. The fixed scsi-mq request
pool introduced this dependency; moving the callback into scsi_bsg.c
did not change its ownership ordering.

Fixes: d285203cf647 ("scsi: add support for a blk-mq based I/O path.")
Link: https://lore.kernel.org/r/20261022.scsi-copyback-review.31b8d6a70f4c@outlook.com
Cc: stable@vger.kernel.org
Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 drivers/scsi/scsi_bsg.c | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index 3cf1caad02f7..5ffbd1843861 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -207,6 +207,8 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
 	struct scsi_cmnd *scmd;
 	struct request *rq;
 	struct bio *bio;
+	u8 sense_buffer[SCSI_SENSE_BUFFERSIZE];
+	unsigned int sense_len = 0;
 	int ret, unmap_ret;
 
 	if (hdr->protocol != BSG_PROTOCOL_SCSI  ||
@@ -266,24 +268,28 @@ static int scsi_bsg_sg_io_fn(struct request_queue *q, struct sg_io_v4 *hdr,
 	hdr->response_len = 0;
 
 	if (scmd->sense_len && hdr->response) {
-		int len = min_t(unsigned int, hdr->max_response_len,
-				scmd->sense_len);
-
-		if (copy_to_user(uptr64(hdr->response), scmd->sense_buffer,
-				 len))
-			ret = -EFAULT;
-		else
-			hdr->response_len = len;
+		sense_len = min3(hdr->max_response_len, scmd->sense_len,
+				 sizeof(sense_buffer));
+		memcpy(sense_buffer, scmd->sense_buffer, sense_len);
 	}
 
 	if (rq_data_dir(rq) == READ)
 		hdr->din_resid = scmd->resid_len;
 	else
 		hdr->dout_resid = scmd->resid_len;
+	blk_mq_free_request(rq);
+
+	if (sense_len) {
+		if (copy_to_user(uptr64(hdr->response), sense_buffer, sense_len))
+			ret = -EFAULT;
+		else
+			hdr->response_len = sense_len;
+	}
 
 	unmap_ret = blk_rq_unmap_user(bio);
 	if (!ret)
 		ret = unmap_ret;
+	return ret;
 
 out_put_request:
 	blk_mq_free_request(rq);
-- 
2.43.0

[RFC PATCH v3 5/6] io_uring/cmd: allow reusing SQE storage for completion data

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

uring_cmd requests already allocate enough asynchronous state to hold a
copied 128-byte SQE. Once a driver has consumed every SQE field and
committed to asynchronous completion, that storage is no longer needed.

Add a helper that transfers the storage to the driver until command
completion without growing the request or adding an allocation. Invalidate
the SQE pointer on transfer and fail a subsequent -EAGAIN return with -EIO
so a command cannot be reissued after overwriting its SQE copy.

Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 include/linux/io_uring/cmd.h | 24 ++++++++++++++++++++++++
 io_uring/uring_cmd.c         | 22 ++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h
index e18d4af62e1a..6a9c06bcccc6 100644
--- a/include/linux/io_uring/cmd.h
+++ b/include/linux/io_uring/cmd.h
@@ -8,6 +8,7 @@
 #include <linux/blk-mq.h>
 
 /* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */
+#define IORING_URING_CMD_SQE_CONSUMED	BIT(29)
 #define IORING_URING_CMD_CANCELABLE	(1U << 30)
 /* io_uring_cmd is being issued again */
 #define IORING_URING_CMD_REISSUE	(1U << 31)
@@ -82,6 +83,22 @@ int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *ioucmd,
  */
 void __io_uring_cmd_done(struct io_uring_cmd *, unsigned issue_flags);
 
+/**
+ * io_uring_cmd_get_completion_data - reuse the command's SQE copy storage
+ * @ioucmd: command that has consumed all of its SQE fields
+ * @len: amount of completion storage required
+ *
+ * Transfer the per-request storage reserved for an SQE copy to the command.
+ * After a successful call, @ioucmd->sqe is invalid and the command must not be
+ * reissued. The returned storage remains valid until io_uring_cmd_done*().
+ * Multishot commands cannot use this helper.
+ *
+ * Return: completion storage on success, or NULL if it is unavailable or too
+ * large.
+ */
+void *io_uring_cmd_get_completion_data(struct io_uring_cmd *ioucmd,
+				       size_t len);
+
 void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd,
 			    io_req_tw_func_t task_work_cb,
 			    unsigned flags);
@@ -140,6 +157,13 @@ static inline void __io_uring_cmd_done(struct io_uring_cmd *cmd,
 				       unsigned issue_flags)
 {
 }
+
+static inline void *
+io_uring_cmd_get_completion_data(struct io_uring_cmd *ioucmd, size_t len)
+{
+	return NULL;
+}
+
 static inline void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd,
 			    io_req_tw_func_t task_work_cb, unsigned flags)
 {
diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c
index b5486b56ee0c..87b1b300ae20 100644
--- a/io_uring/uring_cmd.c
+++ b/io_uring/uring_cmd.c
@@ -166,6 +166,23 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd, unsigned issue_flags)
 }
 EXPORT_SYMBOL_GPL(__io_uring_cmd_done);
 
+void *io_uring_cmd_get_completion_data(struct io_uring_cmd *ioucmd, size_t len)
+{
+	struct io_kiocb *req = cmd_to_io_kiocb(ioucmd);
+	struct io_async_cmd *ac = req->async_data;
+
+	if (!ac || !ioucmd->sqe ||
+	    ioucmd->flags & (IORING_URING_CMD_MULTISHOT |
+			     IORING_URING_CMD_SQE_CONSUMED) ||
+	    len > sizeof(ac->sqes))
+		return NULL;
+
+	ioucmd->flags |= IORING_URING_CMD_SQE_CONSUMED;
+	ioucmd->sqe = NULL;
+	return ac->sqes;
+}
+EXPORT_SYMBOL_GPL(io_uring_cmd_get_completion_data);
+
 int io_uring_cmd_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 {
 	struct io_uring_cmd *ioucmd = io_kiocb_to_cmd(req, struct io_uring_cmd);
@@ -257,11 +274,16 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags)
 
 	ret = file->f_op->uring_cmd(ioucmd, issue_flags);
 	if (ret == -EAGAIN) {
+		if (ioucmd->flags & IORING_URING_CMD_SQE_CONSUMED) {
+			ret = -EIO;
+			goto done;
+		}
 		ioucmd->flags |= IORING_URING_CMD_REISSUE;
 		return ret;
 	}
 	if (ret == -EIOCBQUEUED)
 		return ret;
+done:
 	if (ret < 0)
 		req_set_fail(req);
 	io_req_uring_cleanup(req, issue_flags);
-- 
2.43.0

[RFC PATCH v3 6/6] scsi: bsg: release io_uring requests from end-I/O

[Yingjie Wang]

From: Yingjie Wang <x-professor@outlook.com>

The BSG io_uring end-I/O callback leaves completed requests for task work
to release. The request keeps its tag until that work runs. If the
submitting task first faults on memory backed by the same queue, it can
block waiting for the retained tag and never run the task work that frees
it.

Snapshot the SCSI result and bounded sense data in end-I/O, queue task
work, and return RQ_END_IO_FREE for every completion. Task work now only
unmaps the saved BIO, copies the saved sense data, and posts the CQE; it
never dereferences the request or scsi_cmnd.

Store the sense snapshot in io_uring completion data that was already
allocated for the command SQE copy. This adds no per-command allocation
and does not grow the request. If task work is canceled or the sense copy
fails, clear the reported sense length because no bytes reached user
space.

Fixes: 7b6d3255e7f8 ("scsi: bsg: add io_uring passthrough handler")
Link: https://lore.kernel.org/r/20261022.scsi-copyback-review.31b8d6a70f4c@outlook.com
Cc: stable@vger.kernel.org
Signed-off-by: Yingjie Wang <x-professor@outlook.com>
---
 drivers/scsi/scsi_bsg.c | 94 ++++++++++++++++++++++++++++-------------
 1 file changed, 64 insertions(+), 30 deletions(-)

diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index 5ffbd1843861..4794278374b1 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -16,9 +16,13 @@
  */
 struct scsi_bsg_uring_cmd_pdu {
 	struct bio *bio;		/* mapped user buffer, unmap in task work */
-	struct request *req;		/* block request, freed in task work */
+	u8 *sense_buffer;		/* completion storage owned by io_uring */
 	u64 response_addr;		/* user space response buffer address */
-	u32 max_response_len;		/* user response buffer size */
+	/* max_response_len is replaced by res2 after the end-I/O snapshot. */
+	union {
+		u32 max_response_len;	/* maximum sense snapshot size */
+		u64 res2;		/* saved completion result */
+	};
 };
 static_assert(sizeof(struct scsi_bsg_uring_cmd_pdu) <= sizeof_field(struct io_uring_cmd, pdu));
 
@@ -28,17 +32,22 @@ static inline struct scsi_bsg_uring_cmd_pdu *scsi_bsg_uring_cmd_pdu(
 	return io_uring_cmd_to_pdu(ioucmd, struct scsi_bsg_uring_cmd_pdu);
 }
 
-/* Task work: build res2 (layout in uapi/linux/bsg.h) and copy sense to user. */
+static u64 scsi_bsg_res2_clear_sense_len(u64 res2)
+{
+	return bsg_scsi_res2_build(bsg_scsi_res2_device_status(res2),
+				   bsg_scsi_res2_driver_status(res2),
+				   bsg_scsi_res2_host_status(res2), 0,
+				   bsg_scsi_res2_resid_len(res2));
+}
+
+/* Task work: unmap the saved BIO and copy sense to user space. */
 static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 {
 	struct io_uring_cmd *ioucmd = io_uring_cmd_from_tw(tw_req);
 	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
-	struct request *rq = pdu->req;
-	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(rq);
-	u64 res2;
+	u64 res2 = pdu->res2;
+	u8 sense_len = bsg_scsi_res2_sense_len(res2);
 	int ret = 0;
-	u8 driver_status = 0;
-	u8 sense_len_wr = 0;
 
 	if (pdu->bio)
 		ret = blk_rq_unmap_user(pdu->bio);
@@ -48,27 +57,20 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 	 * has exited, this may run from a kworker or in a different task, so
 	 * don't copy the sense data to the submitter's address.
 	 */
-	if (unlikely(tw.cancel))
+	if (unlikely(tw.cancel)) {
 		ret = -ECANCELED;
-
-	if (scsi_status_is_check_condition(scmd->result)) {
-		driver_status = DRIVER_SENSE;
-		if (pdu->response_addr && !tw.cancel)
-			sense_len_wr = min_t(unsigned int, pdu->max_response_len,
-					     scmd->sense_len);
-	}
-
-	if (sense_len_wr) {
-		if (copy_to_user(uptr64(pdu->response_addr), scmd->sense_buffer,
-				 sense_len_wr))
+		sense_len = 0;
+	} else if (sense_len) {
+		if (copy_to_user(uptr64(pdu->response_addr), pdu->sense_buffer,
+				 sense_len)) {
 			ret = -EFAULT;
+			sense_len = 0;
+		}
 	}
 
-	res2 = bsg_scsi_res2_build(status_byte(scmd->result), driver_status,
-				  host_byte(scmd->result), sense_len_wr,
-				  scmd->resid_len);
+	if (!sense_len)
+		res2 = scsi_bsg_res2_clear_sense_len(res2);
 
-	blk_mq_free_request(rq);
 	io_uring_cmd_done32(ioucmd, ret, res2,
 			    IO_URING_CMD_TASK_WORK_ISSUE_FLAGS);
 }
@@ -78,9 +80,25 @@ static enum rq_end_io_ret scsi_bsg_uring_cmd_done(struct request *req,
 						  const struct io_comp_batch *iocb)
 {
 	struct io_uring_cmd *ioucmd = req->end_io_data;
+	struct scsi_bsg_uring_cmd_pdu *pdu = scsi_bsg_uring_cmd_pdu(ioucmd);
+	struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
+	u8 driver_status = 0;
+	u8 sense_len = 0;
 
+	if (scsi_status_is_check_condition(scmd->result)) {
+		driver_status = DRIVER_SENSE;
+		if (pdu->response_addr) {
+			sense_len = min_t(unsigned int, pdu->max_response_len,
+					  scmd->sense_len);
+			memcpy(pdu->sense_buffer, scmd->sense_buffer,
+			       sense_len);
+		}
+	}
+	pdu->res2 = bsg_scsi_res2_build(status_byte(scmd->result),
+					driver_status, host_byte(scmd->result),
+					sense_len, scmd->resid_len);
 	io_uring_cmd_do_in_task_lazy(ioucmd, scsi_bsg_uring_task_cb);
-	return RQ_END_IO_NONE;
+	return RQ_END_IO_FREE;
 }
 
 static int scsi_bsg_map_user_buffer(struct request *req,
@@ -123,8 +141,15 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 	u32 dout_xfer_len = READ_ONCE(cmd->dout_xfer_len);
 	u64 din_xferp = READ_ONCE(cmd->din_xferp);
 	u32 din_xfer_len = READ_ONCE(cmd->din_xfer_len);
+	u64 response_addr = READ_ONCE(cmd->response);
+	u32 max_response_len = min_t(u32, READ_ONCE(cmd->max_response_len),
+				     SCSI_SENSE_BUFFERSIZE);
+	u32 timeout_ms = READ_ONCE(cmd->timeout_ms);
 	int ret;
 
+	if (ioucmd->flags & IORING_URING_CMD_MULTISHOT)
+		return -EOPNOTSUPP;
+
 	if (cmd->protocol != BSG_PROTOCOL_SCSI ||
 	    cmd->subprotocol != BSG_SUB_PROTOCOL_SCSI_CMD)
 		return -EINVAL;
@@ -168,8 +193,8 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 		goto out_free_req;
 	}
 
-	pdu->response_addr = cmd->response;
-	pdu->max_response_len = cmd->max_response_len;
+	pdu->response_addr = response_addr;
+	pdu->max_response_len = max_response_len;
 
 	if (dout_xfer_len || din_xfer_len) {
 		bool is_write = dout_xfer_len > 0;
@@ -186,16 +211,25 @@ static int scsi_bsg_uring_cmd(struct request_queue *q, struct io_uring_cmd *iouc
 		pdu->bio = NULL;
 	}
 
-	req->timeout = cmd->timeout_ms ?
-		msecs_to_jiffies(cmd->timeout_ms) : BLK_DEFAULT_SG_TIMEOUT;
+	req->timeout = timeout_ms ? msecs_to_jiffies(timeout_ms) :
+				   BLK_DEFAULT_SG_TIMEOUT;
+
+	pdu->sense_buffer =
+		io_uring_cmd_get_completion_data(ioucmd, SCSI_SENSE_BUFFERSIZE);
+	if (WARN_ON_ONCE(!pdu->sense_buffer)) {
+		ret = -EIO;
+		goto out_unmap_user;
+	}
 
 	req->end_io = scsi_bsg_uring_cmd_done;
 	req->end_io_data = ioucmd;
-	pdu->req = req;
 
 	blk_execute_rq_nowait(req, false);
 	return -EIOCBQUEUED;
 
+out_unmap_user:
+	if (pdu->bio)
+		blk_rq_unmap_user(pdu->bio);
 out_free_req:
 	blk_mq_free_request(req);
 	return ret;
-- 
2.43.0