[PATCH] dm-crypt: defer async write submission to write thread

[PATCH] dm-crypt: defer async write submission to write thread

[Josef Schlehofer]

With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
bio directly, even when called from kcryptd_async_done(). An async
cipher may run this callback in softirq or with BH disabled, where
submit_bio_noacct() must not be called because it can sleep.

Submit inline only when async is false. Async completions are sent to
the write thread, as is already done for submit_from_crypt_cpus. Use
the async flag rather than in_interrupt(), since callback context
depends on the cipher driver. Sync ciphers still submit inline.

Tested with cryptd in QEMU (x86_64 and arm64) with
CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.

In QEMU:

- 4k writes logging "sleeping function called from invalid context":
  100/100 unpatched, 0/155 patched.
- nr_requests=4 on a loop device: 0/6 unpatched runs completed,
  4/4 patched runs completed.

On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
6th unpatched run and did not appear in 30 patched runs.

Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
Cc: stable@vger.kernel.org
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
---
Based on dm-7.4 (dab527d83462).

To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:

  modprobe -a cryptd dm_crypt loop
  truncate -s 256M /dev/shm/nowq.img
  L=$(losetup -f)
  losetup "$L" /dev/shm/nowq.img
  KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
  dmsetup create nowq --table "0 $((256 * 2048)) crypt capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue no_write_workqueue"
  D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
  dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
  dmesg | grep 'sleeping function called'

For the nr_requests=4 case, run this instead of the dd. The writes did
not finish without this patch:

  echo 4 > /sys/block/${L##*/}/queue/nr_requests
  for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i * 512)) conv=fsync & done; wait

 drivers/md/dm-crypt.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 8853e9af1c3d..c0552a300e62 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async)
 	/* crypt_convert should have filled the clone bio */
 	BUG_ON(io->ctx.iter_out.bi_size);
 
-	if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
-	    test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
+	if (likely(!async) &&
+	    (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
+	     test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
 		dm_submit_bio_remap(io->base_bio, clone);
 		return;
 	}
-- 
2.54.0 (Apple Git-157)

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Damien Le Moal]

On 11/5/26 16:27, Josef Schlehofer wrote:
> With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
> bio directly, even when called from kcryptd_async_done(). An async
> cipher may run this callback in softirq or with BH disabled, where
> submit_bio_noacct() must not be called because it can sleep.
> 
> Submit inline only when async is false. Async completions are sent to
> the write thread, as is already done for submit_from_crypt_cpus. Use
> the async flag rather than in_interrupt(), since callback context
> depends on the cipher driver. Sync ciphers still submit inline.
> 
> Tested with cryptd in QEMU (x86_64 and arm64) with
> CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.

Originally, this was added to avoid write reordering for zoned block devices.
So did you test this change with that (scsi_debug or null-blk) ? You can also
use an emulated NVMe ZNS device in QEMU.

> 
> In QEMU:
> 
> - 4k writes logging "sleeping function called from invalid context":
>   100/100 unpatched, 0/155 patched.
> - nr_requests=4 on a loop device: 0/6 unpatched runs completed,
>   4/4 patched runs completed.
> 
> On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
> 6th unpatched run and did not appear in 30 patched runs.
> 
> Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
> Cc: stable@vger.kernel.org
> Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
> ---
> Based on dm-7.4 (dab527d83462).
> 
> To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
> CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:
> 
>   modprobe -a cryptd dm_crypt loop
>   truncate -s 256M /dev/shm/nowq.img
>   L=$(losetup -f)
>   losetup "$L" /dev/shm/nowq.img
>   KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
>   dmsetup create nowq --table "0 $((256 * 2048)) crypt capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue no_write_workqueue"
>   D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
>   dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
>   dmesg | grep 'sleeping function called'
> 
> For the nr_requests=4 case, run this instead of the dd. The writes did
> not finish without this patch:
> 
>   echo 4 > /sys/block/${L##*/}/queue/nr_requests
>   for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i * 512)) conv=fsync & done; wait
> 
>  drivers/md/dm-crypt.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
> index 8853e9af1c3d..c0552a300e62 100644
> --- a/drivers/md/dm-crypt.c
> +++ b/drivers/md/dm-crypt.c
> @@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async)
>  	/* crypt_convert should have filled the clone bio */
>  	BUG_ON(io->ctx.iter_out.bi_size);
>  
> -	if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
> -	    test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
> +	if (likely(!async) &&
> +	    (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
> +	     test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
>  		dm_submit_bio_remap(io->base_bio, clone);
>  		return;
>  	}


-- 
Damien Le Moal
Western Digital Research

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Mikulas Patocka]

On Mon, 9 Nov 2026, Damien Le Moal wrote:

> On 11/5/26 16:27, Josef Schlehofer wrote:
> > With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
> > bio directly, even when called from kcryptd_async_done(). An async
> > cipher may run this callback in softirq or with BH disabled, where
> > submit_bio_noacct() must not be called because it can sleep.
> > 
> > Submit inline only when async is false. Async completions are sent to
> > the write thread, as is already done for submit_from_crypt_cpus. Use
> > the async flag rather than in_interrupt(), since callback context
> > depends on the cipher driver. Sync ciphers still submit inline.
> > 
> > Tested with cryptd in QEMU (x86_64 and arm64) with
> > CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.
> 
> Originally, this was added to avoid write reordering for zoned block devices.
> So did you test this change with that (scsi_debug or null-blk) ? You can also
> use an emulated NVMe ZNS device in QEMU.

So, should we disable asynchronous crypto operations for zoned devices? 
Would you send a patch for that?

Mikulas

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Josef Schlehofer]

On Mon, 9 Nov 2026, Mikulas Patocka wrote:
> On Mon, 9 Nov 2026, Damien Le Moal wrote:
>> Originally, this was added to avoid write reordering for zoned block devices.
>> So did you test this change with that (scsi_debug or null-blk) ? You can also
>> use an emulated NVMe ZNS device in QEMU.
>
> So, should we disable asynchronous crypto operations for zoned devices?

I tested dm-7.4 with and without the patch in QEMU, using null_blk,
scsi_debug and an emulated ZNS device. I tested both cryptd and a
synchronous cipher.

My write and read-back checks passed on both arm64 and x86_64. I also
ran fio's t/zbd suite and blktests zbd on arm64. All tests passed with
both kernels.

I also used a kprobe to check whether async=1 was ever used in
kcryptd_crypt_write_io_submit() on zoned devices. It wasn't, since
DM_CRYPT_WRITE_INLINE keeps those writes on the async=0 path.

The only exception was blktests zbd/013, which stalled during cleanup on
both kernels under heavy host load. The LUKS volume was still mapped
after luksClose, causing null_blk unloading to wait.

I couldn't find any issues with this change or with async crypto on
zoned devices, so whether to disable it is up to you and Damien.

Thanks,
Josef

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Damien Le Moal]

On 2026/11/10 13:29, Josef Schlehofer wrote:
> On Mon, 9 Nov 2026, Mikulas Patocka wrote:
>> On Mon, 9 Nov 2026, Damien Le Moal wrote:
>>> Originally, this was added to avoid write reordering for zoned block devices.
>>> So did you test this change with that (scsi_debug or null-blk) ? You can also
>>> use an emulated NVMe ZNS device in QEMU.
>>
>> So, should we disable asynchronous crypto operations for zoned devices?
> 
> I tested dm-7.4 with and without the patch in QEMU, using null_blk,
> scsi_debug and an emulated ZNS device. I tested both cryptd and a
> synchronous cipher.
> 
> My write and read-back checks passed on both arm64 and x86_64. I also
> ran fio's t/zbd suite and blktests zbd on arm64. All tests passed with
> both kernels.

My bad. Since we switched the block layer to zone write plugging, a DM device
only ever sees a single write command per zone. So regardless of what the target
driver does with that command, things are fine for writes to zoned block
devices. The only gotcha here is that if the target driver splits the write,
then the fragments must be ordered correctly to be sequential. If I remember
correctly, I patched dm-crypt to avoid such splitting for exactly that reason.

> 
> I also used a kprobe to check whether async=1 was ever used in
> kcryptd_crypt_write_io_submit() on zoned devices. It wasn't, since
> DM_CRYPT_WRITE_INLINE keeps those writes on the async=0 path.
> 
> The only exception was blktests zbd/013, which stalled during cleanup on
> both kernels under heavy host load. The LUKS volume was still mapped
> after luksClose, causing null_blk unloading to wait.
> 
> I couldn't find any issues with this change or with async crypto on
> zoned devices, so whether to disable it is up to you and Damien.

Thanks for checking. async is fine for zoned devices. Feel free to focus on
other comments that mention that this may enable async even if the user does not
want that.

-- 
Damien Le Moal
Western Digital Research

Re: [PATCH] dm-crypt: defer async write submission to write thread

[Ignat Korchagin]

On Thu, Nov 5, 2026 at 7:27 AM Josef Schlehofer
<pepe.schlehofer@gmail.com> wrote:
>
> With no_write_workqueue, kcryptd_crypt_write_io_submit() submits the
> bio directly, even when called from kcryptd_async_done(). An async
> cipher may run this callback in softirq or with BH disabled, where
> submit_bio_noacct() must not be called because it can sleep.
>
> Submit inline only when async is false. Async completions are sent to
> the write thread, as is already done for submit_from_crypt_cpus. Use
> the async flag rather than in_interrupt(), since callback context
> depends on the cipher driver. Sync ciphers still submit inline.

At the very least I think it may be an unexpected/undesirable setup
for the user: they enable "no_write_workqueue" expecting a fully
inline setup, but still get io offloaded. Perhaps in such cases we can
do a pr_warn_once() to let the user know their setup is impossible?

>
> Tested with cryptd in QEMU (x86_64 and arm64) with
> CONFIG_DEBUG_ATOMIC_SLEEP, and on a P2020 running 6.18.55 without it.
>
> In QEMU:
>
> - 4k writes logging "sleeping function called from invalid context":
>   100/100 unpatched, 0/155 patched.
> - nr_requests=4 on a loop device: 0/6 unpatched runs completed,
>   4/4 patched runs completed.
>
> On the P2020, a warning in blk_mq_run_hw_queue() appeared in the
> 6th unpatched run and did not appear in 30 patched runs.
>
> Fixes: 39d42fa96ba1 ("dm crypt: add flags to optionally bypass kcryptd workqueues")
> Cc: stable@vger.kernel.org
> Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
> ---
> Based on dm-7.4 (dab527d83462).
>
> To reproduce, with CONFIG_DEBUG_ATOMIC_SLEEP, DM_CRYPT, BLK_DEV_LOOP,
> CRYPTO_CRYPTD, CRYPTO_XTS and CRYPTO_AES:
>
>   modprobe -a cryptd dm_crypt loop
>   truncate -s 256M /dev/shm/nowq.img
>   L=$(losetup -f)
>   losetup "$L" /dev/shm/nowq.img
>   KEY=$(od -An -v -tx1 -N64 /dev/urandom | tr -d ' \n')
>   dmsetup create nowq --table "0 $((256 * 2048)) crypt capi:cryptd(xts-aes-lib)-plain64 $KEY 0 $L 0 2 no_read_workqueue no_write_workqueue"
>   D=/dev/dm-$(dmsetup info -c --noheadings -o minor nowq)
>   dd if=/dev/urandom of="$D" bs=4k count=1 conv=fsync
>   dmesg | grep 'sleeping function called'
>
> For the nr_requests=4 case, run this instead of the dd. The writes did
> not finish without this patch:
>
>   echo 4 > /sys/block/${L##*/}/queue/nr_requests
>   for i in 1 2 3 4; do dd if=/dev/urandom of="$D" bs=64k count=512 seek=$((i * 512)) conv=fsync & done; wait
>
>  drivers/md/dm-crypt.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
> index 8853e9af1c3d..c0552a300e62 100644
> --- a/drivers/md/dm-crypt.c
> +++ b/drivers/md/dm-crypt.c
> @@ -1982,8 +1982,9 @@ static void kcryptd_crypt_write_io_submit(struct dm_crypt_io *io, int async)
>         /* crypt_convert should have filled the clone bio */
>         BUG_ON(io->ctx.iter_out.bi_size);
>
> -       if ((likely(!async) && test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags)) ||
> -           test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags)) {
> +       if (likely(!async) &&
> +           (test_bit(DM_CRYPT_NO_OFFLOAD, &cc->flags) ||
> +            test_bit(DM_CRYPT_NO_WRITE_WORKQUEUE, &cc->flags))) {
>                 dm_submit_bio_remap(io->base_bio, clone);
>                 return;
>         }
> --
> 2.54.0 (Apple Git-157)
>
>

Ignat