[PATCH] dm-verity: use bdev_rw_virt() in verity_recheck()

[PATCH] dm-verity: use bdev_rw_virt() in verity_recheck()

[Chris Wiseman]

verity_ctr() creates a dedicated dm_io_client (v->io) for every
dm-verity target, but its only caller is verity_recheck(), which reads
a single data block into a page from v->recheck_pool after a hash
mismatch.  Because dm_io_client_create() preallocates a mempool of 16
bios and 16 biovec-max (4 KiB) vectors per target, each dm-verity
target permanently pins ~70 KiB of unreclaimable slab that a single-page
read never uses (~2.4 MiB across 35 targets on Android, or ~4.1 MiB
across 59 targets).

Replace dm_io() in verity_recheck() with bdev_rw_virt() and remove
v->io.  bdev_rw_virt() puts the bio and single bio_vec on the stack and
waits for completion, while keeping v->recheck_pool per-target.

Fixes: 9177f3c0dea6 ("dm-verity: recheck the hash after a failure")
Link: https://lore.kernel.org/dm-devel/20250423115013.4004577-1-weilongping@oppo.com/
Signed-off-by: Chris Wiseman <chriswiseman@google.com>
---
TEST (ARM64 QEMU, 59 dm-verity targets, sha256, 4K blocks, mem_profiling=1):

/proc/allocinfo growth from 0 to 35 targets (bytes):
                                     BASE     PATCHED       DELTA
  bio.c:biovec_init_pool        4,587,520   2,293,760  -2,293,760
  bio.c:bioset_init             1,196,160   1,052,800    -143,360
  dm-io.c:dm_io_client_create     107,520      53,760     -53,760
  mempool.c:mempool_init_node      36,680      23,240     -13,440
  Total (35 targets)            5,927,880   3,423,560  -2,504,320 (-71,552 B/target)

Unreclaimable slab saved by target count:
   8 targets:   572,416 B (  559 KiB)
  15 targets: 1,073,280 B (1,048 KiB)
  35 targets: 2,504,320 B (2,446 KiB)  [SUnreclaim: BASE +8,436 kB vs PATCHED +5,960 kB]
  59 targets: 4,221,568 B (4,123 KiB)

 drivers/md/dm-verity-target.c | 24 +++---------------------
 drivers/md/dm-verity.h        |  2 --
 2 files changed, 3 insertions(+), 23 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 6d589867623c..7c240b3a470f 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -379,21 +379,13 @@ static noinline int verity_recheck(struct dm_verity *v, struct dm_verity_io *io,
 	struct page *page;
 	void *buffer;
 	int r;
-	struct dm_io_request io_req;
-	struct dm_io_region io_loc;
 
 	page = mempool_alloc(&v->recheck_pool, GFP_NOIO);
 	buffer = page_to_virt(page);
 
-	io_req.bi_opf = REQ_OP_READ;
-	io_req.mem.type = DM_IO_KMEM;
-	io_req.mem.ptr.addr = buffer;
-	io_req.notify.fn = NULL;
-	io_req.client = v->io;
-	io_loc.bdev = v->data_dev->bdev;
-	io_loc.sector = cur_block << (v->data_dev_block_bits - SECTOR_SHIFT);
-	io_loc.count = 1 << (v->data_dev_block_bits - SECTOR_SHIFT);
-	r = dm_io(&io_req, 1, &io_loc, NULL, NULL, IOPRIO_DEFAULT);
+	r = bdev_rw_virt(v->data_dev->bdev,
+			 cur_block << (v->data_dev_block_bits - SECTOR_SHIFT),
+			 buffer, 1 << v->data_dev_block_bits, REQ_OP_READ);
 	if (unlikely(r))
 		goto free_ret;
 
@@ -1058,8 +1050,6 @@ static void verity_dtr(struct dm_target *ti)
 		destroy_workqueue(v->verify_wq);
 
 	mempool_exit(&v->recheck_pool);
-	if (v->io)
-		dm_io_client_destroy(v->io);
 
 	if (v->bufio)
 		dm_bufio_client_destroy(v->bufio);
@@ -1604,14 +1594,6 @@ static int verity_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 		goto bad;
 	}
 
-	v->io = dm_io_client_create();
-	if (IS_ERR(v->io)) {
-		r = PTR_ERR(v->io);
-		v->io = NULL;
-		ti->error = "Cannot allocate dm io";
-		goto bad;
-	}
-
 	v->bufio = dm_bufio_client_create(v->hash_dev->bdev,
 		1 << v->hash_dev_block_bits, 1, sizeof(struct buffer_aux),
 		dm_bufio_alloc_callback, NULL,
diff --git a/drivers/md/dm-verity.h b/drivers/md/dm-verity.h
index e104a651c657..0c42ac99e631 100644
--- a/drivers/md/dm-verity.h
+++ b/drivers/md/dm-verity.h
@@ -11,7 +11,6 @@
 #ifndef DM_VERITY_H
 #define DM_VERITY_H
 
-#include <linux/dm-io.h>
 #include <linux/dm-bufio.h>
 #include <linux/device-mapper.h>
 #include <linux/interrupt.h>
@@ -80,7 +79,6 @@ struct dm_verity {
 
 	char *signature_key_desc; /* signature keyring reference */
 
-	struct dm_io_client *io;
 	mempool_t recheck_pool;
 };
 
-- 
2.56.0

Re: [PATCH] dm-verity: use bdev_rw_virt() in verity_recheck()

[Eric Biggers]

On Tue, Nov 03, 2026 at 06:50:28PM +0000, Chris Wiseman wrote:
> verity_ctr() creates a dedicated dm_io_client (v->io) for every
> dm-verity target, but its only caller is verity_recheck(), which reads
> a single data block into a page from v->recheck_pool after a hash
> mismatch.  Because dm_io_client_create() preallocates a mempool of 16
> bios and 16 biovec-max (4 KiB) vectors per target, each dm-verity
> target permanently pins ~70 KiB of unreclaimable slab that a single-page
> read never uses (~2.4 MiB across 35 targets on Android, or ~4.1 MiB
> across 59 targets).
> 
> Replace dm_io() in verity_recheck() with bdev_rw_virt() and remove
> v->io.  bdev_rw_virt() puts the bio and single bio_vec on the stack and
> waits for completion, while keeping v->recheck_pool per-target.
> 
> Fixes: 9177f3c0dea6 ("dm-verity: recheck the hash after a failure")
> Link: https://lore.kernel.org/dm-devel/20250423115013.4004577-1-weilongping@oppo.com/
> Signed-off-by: Chris Wiseman <chriswiseman@google.com>

Reviewed-by: Eric Biggers <ebiggers@kernel.org>

- Eric

Re: [PATCH] dm-verity: use bdev_rw_virt() in verity_recheck()

[Sami Tolvanen]

Hi Chris,

On Tue, Nov 03, 2026 at 06:50:28PM +0000, Chris Wiseman wrote:
> verity_ctr() creates a dedicated dm_io_client (v->io) for every
> dm-verity target, but its only caller is verity_recheck(), which reads
> a single data block into a page from v->recheck_pool after a hash
> mismatch.  Because dm_io_client_create() preallocates a mempool of 16
> bios and 16 biovec-max (4 KiB) vectors per target, each dm-verity
> target permanently pins ~70 KiB of unreclaimable slab that a single-page
> read never uses (~2.4 MiB across 35 targets on Android, or ~4.1 MiB
> across 59 targets).
> 
> Replace dm_io() in verity_recheck() with bdev_rw_virt() and remove
> v->io.  bdev_rw_virt() puts the bio and single bio_vec on the stack and
> waits for completion, while keeping v->recheck_pool per-target.
> 
> Fixes: 9177f3c0dea6 ("dm-verity: recheck the hash after a failure")
> Link: https://lore.kernel.org/dm-devel/20250423115013.4004577-1-weilongping@oppo.com/
> Signed-off-by: Chris Wiseman <chriswiseman@google.com>
> ---
> TEST (ARM64 QEMU, 59 dm-verity targets, sha256, 4K blocks, mem_profiling=1):
> 
> /proc/allocinfo growth from 0 to 35 targets (bytes):
>                                      BASE     PATCHED       DELTA
>   bio.c:biovec_init_pool        4,587,520   2,293,760  -2,293,760
>   bio.c:bioset_init             1,196,160   1,052,800    -143,360
>   dm-io.c:dm_io_client_create     107,520      53,760     -53,760
>   mempool.c:mempool_init_node      36,680      23,240     -13,440
>   Total (35 targets)            5,927,880   3,423,560  -2,504,320 (-71,552 B/target)
> 
> Unreclaimable slab saved by target count:
>    8 targets:   572,416 B (  559 KiB)
>   15 targets: 1,073,280 B (1,048 KiB)
>   35 targets: 2,504,320 B (2,446 KiB)  [SUnreclaim: BASE +8,436 kB vs PATCHED +5,960 kB]
>   59 targets: 4,221,568 B (4,123 KiB)
> 
>  drivers/md/dm-verity-target.c | 24 +++---------------------
>  drivers/md/dm-verity.h        |  2 --
>  2 files changed, 3 insertions(+), 23 deletions(-)

Looks like a nice improvement!

Reviewed-by: Sami Tolvanen <samitolvanen@google.com>

Sami