[PATCH v7 0/2] dm zoned: fix idle reclaim busy loops

[PATCH v7 0/2] dm zoned: fix idle reclaim busy loops

[Kyrylo Biryuk]

Two dm-zoned reclaim fixes for the 7.4 cycle and stable, split from the
earlier series as Mikulas asked after reverting v4 from dm-7.4: these
two carry Fixes and Cc: stable; the third change, "dm zoned: keep
polling for idle reclaim after a reclaim pass", will follow separately
for 7.5 and depends on patch 1 of this series.

1/2 stops dmz_reclaim_work() from rescheduling itself at once when a
pass cannot make progress (-ENOSPC: no zone to copy into; -EBUSY: no
zone to reclaim). On an idle target that retry ran without pause and
called dmz_check_dev() on every pass. 2/2 stops idle reclaim from
copying a random zone into another random zone, which frees nothing;
its -ENOSPC result relies on the back-off from 1/2, so the order
matters for backports too.

v5: https://lore.kernel.org/r/20261013145522.17499-1-illofspeed@gmail.com
Mikulas's request:
https://lore.kernel.org/r/b7e35437-de42-d078-e75a-1f08541eb725@redhat.com
Damien on v5 4/4:
https://lore.kernel.org/r/bf59b38e-635d-48b8-861f-e50961472e92@kernel.org

Changes since v5 (posted on 2026-10-13; a v6 was prepared but never
sent):
- Rebased on 7.4-rc1 without conflicts; its dm-zoned sources are
  identical to those of 7.3-rc7, on which v5 was based.
- 1/2 (was "dm zoned: back off when reclaim finds no destination zone")
  also takes the -EBUSY back-off from v5 4/4, as Damien asked ("This is
  undoing what patch 1 does"), but not v5 4/4's dmz_should_reclaim()
  change (idle reclaim only when a zone holds data). Without the
  polling patch that change stopped the idle poll after a complete idle
  drain, so later data was not reclaimed until the next allocation;
  the back-off alone stops both loops. Hannes's and Damien's
  Reviewed-by from v5 1/4 are dropped because the patch changed, as is
  Hannes's later one on v3 1/3 (2026-10-21).
- 2/2: code unchanged since v3; Damien's Reviewed-by kept, Hannes's
  added from his reply to v3 2/3 (2026-10-21).
- The polling patch is no longer part of this series (see above).
  Hannes, your comment on v3 3/3 concerns code that v4 moved into
  dmz_schedule_reclaim(), as Damien asked. The polling patch keeps that
  form, which avoids one extra run of the work after a pass that leaves
  reclaim not needed; if you prefer the unconditional reschedule,
  please say so and I will change it.
- The Assisted-by tags are gone, following commit 1e035297ffda ("docs:
  remove Assisted-by references"); tool use is described below.
- Commit messages: test results for exactly these patches on 7.4-rc1;
  measured rates are SCSI request counts, not commands reaching the
  drives.

Tested on 7.4-rc1 with emulated host-managed drives (scsi_debug),
RAM-backed: idle request rate and reclaim passes against the stock
module (1/2), the random-to-random case on a two-device target (2/2),
and checksums of all written data; details are in the commit messages.
md/dm-zoned reassembly after target removal was also clean, and a target
suspend and a "reclaim" message each landed during a copy with no hang
or kernel warning.

Also tested on three 27 TB host-managed SATA drives, each a
single-device dm-zoned target (998 random zones of 256 MiB) used as an
md RAID5 member under dm-crypt and XFS, with the stock module and with
both patches built out of tree for 7.2.10 (whose dm-zoned sources differ
from 7.4-rc1 only in three places where 7.2.10 open-codes
clear_and_wake_up_bit()). One run per module, kprobes attached
throughout; the drives sit behind a SATA port multiplier on a Marvell
88SE9235, and two other drives on the same controller were being
zero-filled at about 218 MB/s each, which affects the absolute numbers:
- idle for 300 s with all random zones free: stock, about 32.5 million
  reclaim passes per worker, each with a TEST UNIT READY that libata
  completes in software (about 108,000 SCSI requests per second per
  member) and about 915 s of system CPU time across all CPUs; patched,
  29-30 passes per worker, no dmz_check_dev() call, 58 requests per
  member and about 46 s of system CPU time. Those passes issue no I/O:
  the requests come with the target's periodic metadata flush;
- 300 s of single-threaded random 4 KiB O_DIRECT overwrites spread over
  1,100 chunks (more chunks than random zones), with dm-zoned's free
  random zone percentage at or below 30 (at most 309 of 998 free) in
  72-79% of the samples: 2,457 (stock) and 2,395 (patched) writes,
  99th percentile latency 0.49 s and 0.48 s;
- a target suspend with the array running, and removal of one
  member's dm-zoned target after the array was stopped, each while
  reclaim was copying a full zone (patched): each waited about 11 s
  for the copy to end, with no hang or kernel warning (with stock,
  reclaim on the member to be suspended had stopped after a busy pass,
  so there was no copy to overlap; the stock removal also waited for
  its copy);
- sha256 of all written data, read back at the end of each run: no
  mismatch; md parity checks of every stripe of the suspend/removal
  test and of a random sample (about 4%) of the other written stripes:
  no mismatch.

2/2's case does not arise on these single-device targets. As with the
stock module, reclaim still stops after a pass that ends while the
target is busy and reclaim is not needed (here: after a busy pass at
31% free, no pass for 600 s); the polling patch addresses that.

Not tested:
- the back-off while the target is busy: in the overwrite test every
  reclaim pass copied a zone, so no pass ended with -ENOSPC or -EBUSY;
- the -ENOSPC back-off on a single-device target without cache zones
  (found by code inspection) and recovery after a sequential zone is
  freed (neither test setup could establish those states);
- the effect of the cache-to-random fallback that 2/2 keeps: 6 such
  copies completed while the two-device target was being filled, but
  the test of its effect could not establish its starting state;
- system reboot and shutdown; only stopping and reassembling the md
  array and the dm-zoned targets was exercised;
- stable kernels at run time, other than the 7.2.10 runs above. Both
  patches apply with at most a 2-line offset to 7.3.1, 7.2.15, 6.18.60,
  6.12.115, 6.6.161, 6.1.191, 5.15.224 and 5.10.273, and the dm-zoned
  objects build there without warnings (x86-64 defconfig plus dm-zoned,
  W=1).

I used LLM coding assistants for root-cause analysis, drafts of the
code and changelogs, the test harnesses and reviews. I reviewed the code
and changed it where needed, and I am responsible for these patches.

Kyrylo Biryuk (2):
  dm zoned: back off when reclaim cannot make progress
  dm zoned: do not reclaim a random zone into another random zone

 drivers/md/dm-zoned-reclaim.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.43.0

[PATCH v7 1/2] dm zoned: back off when reclaim cannot make progress

[Kyrylo Biryuk]

When a reclaim pass cannot make progress, dmz_reclaim_work() still ends
with dmz_schedule_reclaim(). If the target is idle, dmz_should_reclaim()
keeps returning true, the work is queued again with no delay and the
worker spins on the same failing pass. This patch handles two such
cases:

  - dmz_do_reclaim() fails with -ENOSPC because no free zone is
    available to copy the data into, for example on a target without
    cache zones that has no free sequential zone left. The next patch
    also makes this the expected result when an idle target with cache
    zones has random zones but no free sequential zone.
  - dmz_do_reclaim() finds no zone it can lock for reclaim and returns
    -EBUSY. Since commit 90a9b8693f1b ("dm zoned: reclaim random zones
    when idle"), dmz_should_reclaim() returns true on an idle target
    whenever the total number of random (and cache) zones is non-zero,
    whether or not any of them holds data. Each pass then also runs
    dmz_check_dev(), which checks every device for media events (a TEST
    UNIT READY on SCSI disks).

The -EBUSY loop lasts as long as the target stays idle; it ends when a
pass finishes while the target is busy and has enough free zones. On
three 27 TB host-managed SATA drives used as md RAID5 members (7.2.10
with the module built out of tree, all random zones free, target idle
for 300 seconds, kprobes attached, two other drives on the controller
being zero-filled), each member's SCSI device counted about 108,000
requests per second with the stock module and the system used about
915 s of CPU time, against about 46 s with this series. libata completes
TEST UNIT READY in software, so these counters do not measure commands
sent to the drives; the cost is CPU time.

When a pass fails with -ENOSPC or -EBUSY, retrying at once is unlikely
to make progress: re-arm the idle poll instead, so that reclaim is
retried after DMZ_IDLE_PERIOD unless a zone allocation or a "reclaim"
message queues it earlier. On a busy target, zone allocations can
restart reclaim immediately when free space is low enough to require
reclaim. Re-arming also keeps the poll running after a complete idle
drain. The poll can still stop after a pass that ends while reclaim is
not needed; that is unchanged.

Skip the device check for these two results: they normally mean that
there is no zone to copy into or no zone to reclaim, not that a device
failed, and the I/O paths check the devices themselves. Use
queue_delayed_work() so that work an allocation has already queued is
not postponed.

A pass that fails with an I/O error, for example on an unreadable block
of a device that stays online, is still retried at once while the
target is idle; this patch does not change that.

Tested on 7.4-rc1 with this patch and the next one, on three emulated
host-managed drives (scsi_debug, 1 GiB, 32 MiB zones) each under
dm-zoned as md RAID5 members, all random zones free and the target idle
for 300 seconds. With the stock module each member's SCSI device counted
about 70.2 million requests and the reclaim workers used 726 s of CPU.
With this series: 75 requests per member, 30 passes per worker (one per
DMZ_IDLE_PERIOD) and 0.08 s of CPU. Data written during the tests read
back with matching checksums, and no kernel warnings were logged.

Fixes: 3b1a94c88b79 ("dm zoned: drive-managed zoned block device target")
Fixes: 90a9b8693f1b ("dm zoned: reclaim random zones when idle")
Cc: stable@vger.kernel.org
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index c041413c729..077a8a50197 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -537,6 +537,16 @@ static void dmz_reclaim_work(struct work_struct *work)
 		dmz_nr_rnd_zones(zmd, zrc->dev_idx));
 
 	ret = dmz_do_reclaim(zrc);
+	if (ret == -ENOSPC || ret == -EBUSY) {
+		/*
+		 * No zone to copy into (-ENOSPC) or no zone to reclaim
+		 * (-EBUSY): retrying at once cannot make progress and would
+		 * spin. Poll again after the idle period instead, without
+		 * postponing work that an allocation has already queued.
+		 */
+		queue_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
+		return;
+	}
 	if (ret && ret != -EINTR) {
 		if (!dmz_check_dev(zmd))
 			return;
-- 
2.43.0

Re: [PATCH v7 1/2] dm zoned: back off when reclaim cannot make progress

[Damien Le Moal]

On 11/4/26 07:04, Kyrylo Biryuk wrote:
> When a reclaim pass cannot make progress, dmz_reclaim_work() still ends
> with dmz_schedule_reclaim(). If the target is idle, dmz_should_reclaim()
> keeps returning true, the work is queued again with no delay and the
> worker spins on the same failing pass. This patch handles two such
> cases:
> 
>   - dmz_do_reclaim() fails with -ENOSPC because no free zone is
>     available to copy the data into, for example on a target without
>     cache zones that has no free sequential zone left. The next patch
>     also makes this the expected result when an idle target with cache
>     zones has random zones but no free sequential zone.
>   - dmz_do_reclaim() finds no zone it can lock for reclaim and returns
>     -EBUSY. Since commit 90a9b8693f1b ("dm zoned: reclaim random zones
>     when idle"), dmz_should_reclaim() returns true on an idle target
>     whenever the total number of random (and cache) zones is non-zero,
>     whether or not any of them holds data. Each pass then also runs
>     dmz_check_dev(), which checks every device for media events (a TEST
>     UNIT READY on SCSI disks).
> 
> The -EBUSY loop lasts as long as the target stays idle; it ends when a
> pass finishes while the target is busy and has enough free zones. On
> three 27 TB host-managed SATA drives used as md RAID5 members (7.2.10
> with the module built out of tree, all random zones free, target idle
> for 300 seconds, kprobes attached, two other drives on the controller
> being zero-filled), each member's SCSI device counted about 108,000
> requests per second with the stock module and the system used about
> 915 s of CPU time, against about 46 s with this series. libata completes
> TEST UNIT READY in software, so these counters do not measure commands
> sent to the drives; the cost is CPU time.
> 
> When a pass fails with -ENOSPC or -EBUSY, retrying at once is unlikely
> to make progress: re-arm the idle poll instead, so that reclaim is
> retried after DMZ_IDLE_PERIOD unless a zone allocation or a "reclaim"
> message queues it earlier. On a busy target, zone allocations can
> restart reclaim immediately when free space is low enough to require
> reclaim. Re-arming also keeps the poll running after a complete idle
> drain. The poll can still stop after a pass that ends while reclaim is
> not needed; that is unchanged.
> 
> Skip the device check for these two results: they normally mean that
> there is no zone to copy into or no zone to reclaim, not that a device
> failed, and the I/O paths check the devices themselves. Use
> queue_delayed_work() so that work an allocation has already queued is
> not postponed.
> 
> A pass that fails with an I/O error, for example on an unreadable block
> of a device that stays online, is still retried at once while the
> target is idle; this patch does not change that.
> 
> Tested on 7.4-rc1 with this patch and the next one, on three emulated
> host-managed drives (scsi_debug, 1 GiB, 32 MiB zones) each under
> dm-zoned as md RAID5 members, all random zones free and the target idle
> for 300 seconds. With the stock module each member's SCSI device counted
> about 70.2 million requests and the reclaim workers used 726 s of CPU.
> With this series: 75 requests per member, 30 passes per worker (one per
> DMZ_IDLE_PERIOD) and 0.08 s of CPU. Data written during the tests read
> back with matching checksums, and no kernel warnings were logged.
> 
> Fixes: 3b1a94c88b79 ("dm zoned: drive-managed zoned block device target")
> Fixes: 90a9b8693f1b ("dm zoned: reclaim random zones when idle")
> Cc: stable@vger.kernel.org
> Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>

Looks good.

Reviewed-by: Damien Le Moal <dlemoal@kernel.org>

-- 
Damien Le Moal
Western Digital Research

[PATCH v7 2/2] dm zoned: do not reclaim a random zone into another random zone

[Kyrylo Biryuk]

dmz_reclaim_rnd_data() falls back to a random destination zone when no
free sequential zone is available and the target has cache zones. When
the target is idle and the cache zone list is empty,
dmz_get_rnd_zone_for_reclaim() selects random zones of the zoned device
as well, so a random zone is then copied into another random zone. That
frees nothing: the next pass picks the next random zone and copies it
again, for as long as the target stays idle.

Seen on a two-device target (256 GiB regular cache device plus a 27 TB
host-managed drive) used as an md RAID5 member: after a full rebuild
every chunk was mapped and no sequential zone was free, and the idle
drive kept reading and writing while the free zone counters
in "dmsetup status" did not change.

Only switch to a random destination zone when the zone being reclaimed
is a cache zone: moving it to the zoned device still frees cache space.

This depends on "dm zoned: back off when reclaim cannot make progress":
without it, dmz_reclaim_work() would retry the resulting -ENOSPC at
once, and run dmz_check_dev(), for as long as the target stays idle.

Tested on 7.4-rc1 together with the previous patch, on a two-device
target (a 512 MiB RAM-backed regular cache device plus a 1 GiB
scsi_debug host-managed drive) filled until no sequential zone was
free, then idle for 60 seconds: the stock module made 877
random-to-random copies; with this series it made none, and each pass
failed with -ENOSPC once per DMZ_IDLE_PERIOD without a device check.

Fixes: c5c788595292 ("dm zoned: start reclaim with sequential zones")
Cc: stable@vger.kernel.org
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index 077a8a50197..152cb878811 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -288,7 +288,13 @@ static int dmz_reclaim_rnd_data(struct dmz_reclaim *zrc, struct dm_zone *dzone)
 again:
 	szone = dmz_alloc_zone(zmd, zrc->dev_idx,
 			       alloc_flags | DMZ_ALLOC_RECLAIM);
-	if (!szone && alloc_flags == DMZ_ALLOC_SEQ && dmz_nr_cache_zones(zmd)) {
+	/*
+	 * If no free sequential zone is available, a cache zone can still be
+	 * moved into a free random zone of the zoned device, which frees cache
+	 * space. Do not do this for a random zone: moving it into another
+	 * random zone frees nothing and, on an idle target, repeats forever.
+	 */
+	if (!szone && alloc_flags == DMZ_ALLOC_SEQ && dmz_is_cache(dzone)) {
 		alloc_flags = DMZ_ALLOC_RND;
 		goto again;
 	}
-- 
2.43.0