[PATCH] block: only fail dma-buf bios that need block layer PI

[PATCH] block: only fail dma-buf bios that need block layer PI

[Kyle Bader]

From: Kyle Bader <kbader@ibm.com>

Any dma-buf I/O to a device with a PI checksum fails today.
blkdev_init_dma_buf_io_ctx() returns -EOPNOTSUPP at registration, and
blk_mq_submit_bio() fails each dma-buf bio with BLK_STS_NOTSUPP. The
checks prevent a crash: bi_dmabuf_map shares a union with bi_io_vec,
and bio_integrity_generate() and bio_integrity_verify() walk bi_io_vec
to compute the guards over the data.

Only the BI_ACT_CHECK action walks the data, and a dma-buf bio does not
need it in two cases. In the first case, the submitter attaches its own
integrity payload, for example with the io_uring PI attribute
(IORING_RW_ATTR_FLAG_PI). Then bio_integrity_action() returns 0, and
the device checks the PI as the submitter asked. The block layer only
touches the integrity buffer, to remap the ref tags. In the second
case, read_verify or write_generate is off. Then the device inserts and
strips the PI, or the block layer allocates an integrity buffer without
PI in it. This is the same as for any other bio.

The first case lets an application read data into device memory, for
example a GPU buffer or an NVMe CMB. The device checks each block and
puts the guards in host memory. The application can then check the
data, or combine the guards into a checksum of the whole buffer. The
CPU does not read the data.

If bio_integrity_action() asks for BI_ACT_CHECK, fail the dma-buf bio,
and otherwise let it through. Remove the check at registration, because
the integrity payload comes with each I/O. With the default integrity
settings, a dma-buf I/O without an integrity payload still fails with
-EOPNOTSUPP.

Link: https://lore.kernel.org/all/bbb87d6b-b7a3-487a-8be4-27f3f9ce4a1b@samsung.com/
Assisted-by: LLM
Signed-off-by: Kyle Bader <kbader@ibm.com>
---
This applies to for-7.4/io_uring-dmabuf (45b716456dd9).

Anuj's check in v10 fixed the crash in blk_integrity_iterate(). This
patch keeps that check for the bios that need block layer PI, and lets
the other dma-buf bios through.

The sysfs knobs change the integrity flags only with the queue frozen,
and blk_mq_submit_bio() holds a queue reference from the new check to
bio_integrity_prep(). So the two calls to bio_integrity_action() give
the same result for a bio.

I tested in a QEMU 10.2.2 guest, with udmabuf and with an NVMe CMB
that a small module exports as a dma-buf. The namespaces have 4 KiB
blocks with these metadata formats:

 - 16 bytes, PI type 1 with the 64b guard
 - 64 bytes, PI type 1 with the 64b guard in the last 16 bytes
 - 8 bytes, PI type 1 with the 16b guard
 - no metadata

With a user PI buffer, reads and writes pass on all three PI formats,
and the data and the PI match. This is also true for a 2 MiB I/O that
the block layer splits, and for four commands with one PI slice each.
A write with one bad guard fails with -EILSEQ from the device. Without
a PI buffer, a dma-buf I/O fails with -EOPNOTSUPP. If read_verify or
write_generate is off, that I/O passes. Pavel's rw-dmabuf test from
liburing rw-dmabuf-tests-v8 passes on the namespace without metadata.
On the PI namespaces it passes with read_verify=0 and write_generate=0,
and with the defaults it fails with -EOPNOTSUPP on the first read.
The tree builds with CONFIG_BLK_DEV_INTEGRITY=n.

Without this patch, every PI case fails at registration with
-EOPNOTSUPP, also with read_verify and write_generate off.

If a liburing test helps, I can turn the test program into one.

 block/blk-mq.c | 9 ++++++++-
 block/fops.c   | 2 --
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 7fff1305b..3fc9fdeb3 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3138,8 +3138,15 @@ void blk_mq_submit_bio(struct bio *bio)
 		goto queue_exit;
 	}
 
+	/*
+	 * The block layer cannot generate or verify PI for a dma-buf bio, as
+	 * that walks the data pages.  Only fail the bio if it needs that, not
+	 * if the submitter attached its own integrity payload or if PI
+	 * generation or verification is turned off for the device.
+	 */
 	if (unlikely(op_is_dmabuf(bio->bi_opf) &&
-		     bdev_has_integrity_csum(bio->bi_bdev))) {
+		     bdev_has_integrity_csum(bio->bi_bdev) &&
+		     (bio_integrity_action(bio) & BI_ACT_CHECK))) {
 		bio_endio_status(bio, BLK_STS_NOTSUPP);
 		goto queue_exit;
 	}
diff --git a/block/fops.c b/block/fops.c
index 01bc8afe3..5c8024011 100644
--- a/block/fops.c
+++ b/block/fops.c
@@ -947,8 +947,6 @@ static int blkdev_init_dma_buf_io_ctx(struct file *file,
 		return -EINVAL;
 	if (!disk->fops->init_dma_buf_io_ctx)
 		return -EOPNOTSUPP;
-	if (bdev_has_integrity_csum(bdev))
-		return -EOPNOTSUPP;
 	return disk->fops->init_dma_buf_io_ctx(bdev, ctx);
 }
 

base-commit: 45b716456dd99488a0cef58482120dc0474aa696
-- 
2.54.0

Re: [PATCH] block: only fail dma-buf bios that need block layer PI

[Anuj gupta]

Looks ok to me
Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>

Re: [PATCH] block: only fail dma-buf bios that need block layer PI

[Pavel Begunkov]

On 10/28/26 18:14, Kyle Bader wrote:
> From: Kyle Bader <kbader@ibm.com>
> 
> Any dma-buf I/O to a device with a PI checksum fails today.
> blkdev_init_dma_buf_io_ctx() returns -EOPNOTSUPP at registration, and
> blk_mq_submit_bio() fails each dma-buf bio with BLK_STS_NOTSUPP. The
> checks prevent a crash: bi_dmabuf_map shares a union with bi_io_vec,
> and bio_integrity_generate() and bio_integrity_verify() walk bi_io_vec
> to compute the guards over the data.
> 
> Only the BI_ACT_CHECK action walks the data, and a dma-buf bio does not
> need it in two cases. In the first case, the submitter attaches its own
> integrity payload, for example with the io_uring PI attribute
> (IORING_RW_ATTR_FLAG_PI). Then bio_integrity_action() returns 0, and
> the device checks the PI as the submitter asked. The block layer only
> touches the integrity buffer, to remap the ref tags. In the second
> case, read_verify or write_generate is off. Then the device inserts and
> strips the PI, or the block layer allocates an integrity buffer without
> PI in it. This is the same as for any other bio.
> 
> The first case lets an application read data into device memory, for
> example a GPU buffer or an NVMe CMB. The device checks each block and
> puts the guards in host memory. The application can then check the
> data, or combine the guards into a checksum of the whole buffer. The
> CPU does not read the data.
> 
> If bio_integrity_action() asks for BI_ACT_CHECK, fail the dma-buf bio,
> and otherwise let it through. Remove the check at registration, because
> the integrity payload comes with each I/O. With the default integrity
> settings, a dma-buf I/O without an integrity payload still fails with
> -EOPNOTSUPP.
> 
> Link: https://lore.kernel.org/all/bbb87d6b-b7a3-487a-8be4-27f3f9ce4a1b@samsung.com/
> Assisted-by: LLM
> Signed-off-by: Kyle Bader <kbader@ibm.com>
> ---
> This applies to for-7.4/io_uring-dmabuf (45b716456dd9).
> 
> Anuj's check in v10 fixed the crash in blk_integrity_iterate(). This
> patch keeps that check for the bios that need block layer PI, and lets
> the other dma-buf bios through.
> 
> The sysfs knobs change the integrity flags only with the queue frozen,
> and blk_mq_submit_bio() holds a queue reference from the new check to
> bio_integrity_prep(). So the two calls to bio_integrity_action() give
> the same result for a bio.
> 
> I tested in a QEMU 10.2.2 guest, with udmabuf and with an NVMe CMB
> that a small module exports as a dma-buf. The namespaces have 4 KiB
> blocks with these metadata formats:
> 
>   - 16 bytes, PI type 1 with the 64b guard
>   - 64 bytes, PI type 1 with the 64b guard in the last 16 bytes
>   - 8 bytes, PI type 1 with the 16b guard
>   - no metadata
> 
> With a user PI buffer, reads and writes pass on all three PI formats,
> and the data and the PI match. This is also true for a 2 MiB I/O that
> the block layer splits, and for four commands with one PI slice each.
> A write with one bad guard fails with -EILSEQ from the device. Without
> a PI buffer, a dma-buf I/O fails with -EOPNOTSUPP. If read_verify or
> write_generate is off, that I/O passes. Pavel's rw-dmabuf test from
> liburing rw-dmabuf-tests-v8 passes on the namespace without metadata.
> On the PI namespaces it passes with read_verify=0 and write_generate=0,
> and with the defaults it fails with -EOPNOTSUPP on the first read.
> The tree builds with CONFIG_BLK_DEV_INTEGRITY=n.
> 
> Without this patch, every PI case fails at registration with
> -EOPNOTSUPP, also with read_verify and write_generate off.

Thanks, looks good from a quick glance, I'll take another look in
a bit. It might be nice to somehow support kernel generated PI
as well later, looks like there will be a good bunch of cases that
might want to have CPU access.

> If a liburing test helps, I can turn the test program into one.
> 
>   block/blk-mq.c | 9 ++++++++-
>   block/fops.c   | 2 --
>   2 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 7fff1305b..3fc9fdeb3 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -3138,8 +3138,15 @@ void blk_mq_submit_bio(struct bio *bio)
>   		goto queue_exit;
>   	}
>   
> +	/*
> +	 * The block layer cannot generate or verify PI for a dma-buf bio, as
> +	 * that walks the data pages.  Only fail the bio if it needs that, not
> +	 * if the submitter attached its own integrity payload or if PI
> +	 * generation or verification is turned off for the device.
> +	 */
>   	if (unlikely(op_is_dmabuf(bio->bi_opf) &&
> -		     bdev_has_integrity_csum(bio->bi_bdev))) {
> +		     bdev_has_integrity_csum(bio->bi_bdev) &&
> +		     (bio_integrity_action(bio) & BI_ACT_CHECK))) {
>   		bio_endio_status(bio, BLK_STS_NOTSUPP);
>   		goto queue_exit;
>   	}
> diff --git a/block/fops.c b/block/fops.c
> index 01bc8afe3..5c8024011 100644
> --- a/block/fops.c
> +++ b/block/fops.c
> @@ -947,8 +947,6 @@ static int blkdev_init_dma_buf_io_ctx(struct file *file,
>   		return -EINVAL;
>   	if (!disk->fops->init_dma_buf_io_ctx)
>   		return -EOPNOTSUPP;
> -	if (bdev_has_integrity_csum(bdev))
> -		return -EOPNOTSUPP;
>   	return disk->fops->init_dma_buf_io_ctx(bdev, ctx);
>   }
>   
> 
> base-commit: 45b716456dd99488a0cef58482120dc0474aa696

-- 
Pavel Begunkov

Re: [PATCH] block: only fail dma-buf bios that need block layer PI

[Christoph Hellwig]

On Wed, Oct 28, 2026 at 06:14:12PM +0000, Kyle Bader wrote:
> From: Kyle Bader <kbader@ibm.com>
> 
> Any dma-buf I/O to a device with a PI checksum fails today.
> blkdev_init_dma_buf_io_ctx() returns -EOPNOTSUPP at registration, and
> blk_mq_submit_bio() fails each dma-buf bio with BLK_STS_NOTSUPP. The
> checks prevent a crash: bi_dmabuf_map shares a union with bi_io_vec,
> and bio_integrity_generate() and bio_integrity_verify() walk bi_io_vec
> to compute the guards over the data.

Please cut down the commit log to the essentials.  It's basicallyr
currently all P2P I/O to devices advertizing integrity data is rejected,
but it should only reject those walking the bio_vecs.

Re: [PATCH] block: only fail dma-buf bios that need block layer PI

[Christoph Hellwig]

On Thu, Oct 29, 2026 at 11:51:31AM +0000, Pavel Begunkov wrote:
> Thanks, looks good from a quick glance, I'll take another look in
> a bit. It might be nice to somehow support kernel generated PI
> as well later, looks like there will be a good bunch of cases that
> might want to have CPU access.

I don't think the kernel auto-PI makes sense for P2P.  The data lives
on a remote device, so generating PI for it would have to transfer
the data to the host.  Instead we probably should just force PRACT
for that case.  It would make sense to support PI generation for
dmabufs using local memory, though.