[PATCH] dm-crypt: defer async write submission to write thread

[PATCH] dm-crypt: defer async write submission to write thread

[Josef Schlehofer]

With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
bio directly, even when called from kcryptd_async_done(). An async
cipher may run this callback in softirq or with BH disabled, where
submit_bio_noacct() must not be called because it can sleep.

Submit inline only when async is false. Async completions are sent to
the write thread, as is already done for submit_from_crypt_cpus. Use
the async flag rather than in_interrupt(), since callback context
depends on the cipher driver. Sync ciphers still submit inline.

Tested with cryptd in QEMU (x86_64 and arm64) with
CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.

In QEMU:

- 4k writes logging "sleeping function called from invalid context":
  100/100 unpatched, 0/155 patched.
- nr_requests=4 on a loop device: 0/6 unpatched runs completed,
  4/4 patched runs completed.

On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
6th unpatched run and did not appear in 30 patched runs.

Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
Cc: stable@vger.kernel.org
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
Based on dm-7.4 (dab527d83462).

To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:

  modprobe -a cryptd dm_crypt loop
  truncate -s 256M /dev/shm/nowq.img
  L=$(losetup -f)
  losetup "$L" /dev/shm/nowq.img
  KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
  dmsetup create nowq --table "0 $((256 * 2048)) crypt capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue no_write_workqueue"
  D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
  dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
  dmesg | grep 'sleeping function called'

For the nr_requests=4 case, run this instead of the dd. The writes did
not finish without this patch:

  echo 4 > /sys/block/${L##*/}/queue/nr_requests
  for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i * 512)) conv=fsync & done; wait

 drivers/md/dm-crypt.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 8853e9af1c3d..c0552a300e62 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async)
 	/* crypt_convert should have filled the clone bio */
 	BUG_ON(io->ctx.iter_out.bi_size);
 
-	if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
-	    test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
+	if (likely(!async) &&
+	    (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
+	     test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
 		dm_submit_bio_remap(io->base_bio, clone);
 		return;
 	}
-- 
2.54.0 (Apple Git-157)

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Damien Le Moal]

On 11/5/26 16:27, Josef Schlehofer wrote:
> With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
> bio directly, even when called from kcryptd_async_done(). An async
> cipher may run this callback in softirq or with BH disabled, where
> submit_bio_noacct() must not be called because it can sleep.
> 
> Submit inline only when async is false. Async completions are sent to
> the write thread, as is already done for submit_from_crypt_cpus. Use
> the async flag rather than in_interrupt(), since callback context
> depends on the cipher driver. Sync ciphers still submit inline.
> 
> Tested with cryptd in QEMU (x86_64 and arm64) with
> CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.

Originally, this was added to avoid write reordering for zoned block devices.
So did you test this change with that (scsi_debug or null-blk) ? You can also
use an emulated NVMe ZNS device in QEMU.

> 
> In QEMU:
> 
> - 4k writes logging "sleeping function called from invalid context":
>   100/100 unpatched, 0/155 patched.
> - nr_requests=4 on a loop device: 0/6 unpatched runs completed,
>   4/4 patched runs completed.
> 
> On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
> 6th unpatched run and did not appear in 30 patched runs.
> 
> Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
> Cc: stable@vger.kernel.org
> Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
> ---
> Based on dm-7.4 (dab527d83462).
> 
> To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
> CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:
> 
>   modprobe -a cryptd dm_crypt loop
>   truncate -s 256M /dev/shm/nowq.img
>   L=$(losetup -f)
>   losetup "$L" /dev/shm/nowq.img
>   KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
>   dmsetup create nowq --table "0 $((256 * 2048)) crypt capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue no_write_workqueue"
>   D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
>   dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
>   dmesg | grep 'sleeping function called'
> 
> For the nr_requests=4 case, run this instead of the dd. The writes did
> not finish without this patch:
> 
>   echo 4 > /sys/block/${L##*/}/queue/nr_requests
>   for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i * 512)) conv=fsync & done; wait
> 
>  drivers/md/dm-crypt.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
> index 8853e9af1c3d..c0552a300e62 100644
> --- a/drivers/md/dm-crypt.c
> +++ b/drivers/md/dm-crypt.c
> @@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async)
>  	/* crypt_convert should have filled the clone bio */
>  	BUG_ON(io->ctx.iter_out.bi_size);
>  
> -	if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
> -	    test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
> +	if (likely(!async) &&
> +	    (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
> +	     test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
>  		dm_submit_bio_remap(io->base_bio, clone);
>  		return;
>  	}


-- 
Damien Le Moal
Western Digital Research

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Mikulas Patocka]

On Mon, 9 Nov 2026, Damien Le Moal wrote:

> On 11/5/26 16:27, Josef Schlehofer wrote:
> > With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
> > bio directly, even when called from kcryptd_async_done(). An async
> > cipher may run this callback in softirq or with BH disabled, where
> > submit_bio_noacct() must not be called because it can sleep.
> > 
> > Submit inline only when async is false. Async completions are sent to
> > the write thread, as is already done for submit_from_crypt_cpus. Use
> > the async flag rather than in_interrupt(), since callback context
> > depends on the cipher driver. Sync ciphers still submit inline.
> > 
> > Tested with cryptd in QEMU (x86_64 and arm64) with
> > CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.
> 
> Originally, this was added to avoid write reordering for zoned block devices.
> So did you test this change with that (scsi_debug or null-blk) ? You can also
> use an emulated NVMe ZNS device in QEMU.

So, should we disable asynchronous crypto operations for zoned devices? 
Would you send a patch for that?

Mikulas