[PATCH v5 0/4] dm zoned: fix idle reclaim problems

[PATCH v5 0/4] dm zoned: fix idle reclaim problems

[Kyrylo Biryuk]

Mikulas,

I noticed too late that v3 of this series is already in dm-7.4 and
for-next (24646f6ed3, 2cca719c63, 5829edd027), so v4 crossed with it.
This v5 is v4 plus one more fix; if you prefer to keep v3, I can send
the v4 changes and patch 4 as incremental patches on top of dm-7.4
instead - please tell me which you want.

Patch 4 is new. While measuring an idle pool I found that an idle
target whose random (and cache) zones are all free never stops running
its reclaim work: since 90a9b8693f1b ("dm zoned: reclaim random zones
when idle"), dmz_should_reclaim() treats "idle" plus a non-zero total
number of random zones as "reclaim now", dmz_do_reclaim() finds no
candidate and returns -EBUSY, dmz_check_dev() checks every device for
media events (a TEST UNIT READY on SCSI disks), and the work is requeued
with no delay. In mainline the loop can end when a pass finishes while
the target is busy with enough free zones; with patch 3 the idle poll
stays armed, so the loop recurs whenever the target becomes idle.

Measured on 7.3-rc7 with three emulated host-managed members of an md
RAID5 (idle, 30 s average):

  stock dm-zoned        181,195 SCSI commands/s   32 % kernel CPU
  v4 (patches 1-3)      187,707 SCSI commands/s   31 % kernel CPU
  v5 (patches 1-4)            0 SCSI commands/s    0 % kernel CPU

With patch 4, the random zones filled by a 2 GiB write burst are still
reclaimed once the target is idle, and on three 27 TB host-managed
drives (7.2.10 with these patches) an idle pool went from about 155,000
TEST UNIT READY commands per second per drive to none.

Changes since v4:
 - new patch 4 (above)
 - patch 3: Reviewed-by from Damien Le Moal
 - patches 1-3 otherwise unchanged

Kyrylo Biryuk (4):
  dm zoned: back off when reclaim finds no destination zone
  dm zoned: do not reclaim a random zone into another random zone
  dm zoned: keep polling for idle reclaim after a reclaim pass
  dm zoned: do not busy-loop idle reclaim when no zone needs reclaim

 drivers/md/dm-zoned-reclaim.c | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

[PATCH v5 1/4] dm zoned: back off when reclaim finds no destination zone

[Kyrylo Biryuk]

When dmz_do_reclaim() fails with -ENOSPC because no free zone is
available to copy the data into, dmz_reclaim_work() still ends with
dmz_schedule_reclaim(). If the target is idle, dmz_should_reclaim()
keeps returning true, the work is queued again with no delay and the
worker spins on the same failing pass.

Re-arm the idle poll instead, so that reclaim is retried after
DMZ_IDLE_PERIOD. The next patch makes -ENOSPC the expected result when
an idle target has random zones but no free sequential zone.

Tested on 7.3-rc7 together with the next two patches, on emulated
host-managed drives (scsi_debug zbc=managed, and the tcmu-runner ZBC
handler under md RAID5). Earlier versions of the series ran on 7.2.6 on
three 27 TB host-managed drives used as md RAID5 members.

Fixes: 3b1a94c88b79 ("dm zoned: drive-managed zoned block device target")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Assisted-by: LLM
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index c041413c729..315f0da99f3 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -540,6 +540,14 @@ static void dmz_reclaim_work(struct work_struct *work)
 	if (ret && ret != -EINTR) {
 		if (!dmz_check_dev(zmd))
 			return;
+		if (ret == -ENOSPC) {
+			/*
+			 * No zone to copy into: nothing can progress now.
+			 * Rescheduling immediately would spin; poll instead.
+			 */
+			mod_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
+			return;
+		}
 	}
 
 	dmz_schedule_reclaim(zrc);
-- 
2.43.0

[PATCH v5 2/4] dm zoned: do not reclaim a random zone into another random zone

[Kyrylo Biryuk]

dmz_reclaim_rnd_data() falls back to a random destination zone when no
free sequential zone is available and the target has cache zones. When
the target is idle and the cache zone list is empty,
dmz_get_rnd_zone_for_reclaim() selects random zones of the zoned device
as well, so a random zone is then copied into another random zone. That
frees nothing: the next pass picks the next random zone and copies it
again, for as long as the target stays idle.

Seen on a two-device target (256 GiB regular cache device plus a 27 TB
host-managed drive) used as an md RAID5 member: after a full rebuild
every chunk was mapped and no sequential zone was free, and the idle
drive read and wrote about 72 MB/s each without pause for hours while
the free zone counters in "dmsetup status" did not change. Reproduced on
emulated host-managed drives (tcmu-runner ZBC handler): 340 MB/s of
copying on an idle target, 0 MB/s with this patch.

Only switch to a random destination zone when the zone being reclaimed
is a cache zone: moving it to the zoned device still frees cache space.

Fixes: c5c788595292 ("dm zoned: start reclaim with sequential zones")
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Assisted-by: LLM
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index 315f0da99f3..88d56d3c86d 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -288,7 +288,13 @@ static int dmz_reclaim_rnd_data(struct dmz_reclaim *zrc, struct dm_zone *dzone)
 again:
 	szone = dmz_alloc_zone(zmd, zrc->dev_idx,
 			       alloc_flags | DMZ_ALLOC_RECLAIM);
-	if (!szone && alloc_flags == DMZ_ALLOC_SEQ && dmz_nr_cache_zones(zmd)) {
+	/*
+	 * If no free sequential zone is available, a cache zone can still be
+	 * moved into a free random zone of the zoned device, which frees cache
+	 * space. Do not do this for a random zone: moving it into another
+	 * random zone frees nothing and, on an idle target, repeats forever.
+	 */
+	if (!szone && alloc_flags == DMZ_ALLOC_SEQ && dmz_is_cache(dzone)) {
 		alloc_flags = DMZ_ALLOC_RND;
 		goto again;
 	}
-- 
2.43.0

[PATCH v5 3/4] dm zoned: keep polling for idle reclaim after a reclaim pass

[Kyrylo Biryuk]

dmz_reclaim_work() re-arms its DMZ_IDLE_PERIOD poll only when
dmz_should_reclaim() returns false at the start of a pass (and, with the
first patch of this series, when a pass fails with -ENOSPC). After a pass
it calls dmz_schedule_reclaim(), which queues the work only if reclaim
is needed right now. When a pass ends while the target is still busy and
has enough free zones, nothing is queued at all. Once the target goes
idle, nothing re-evaluates dmz_should_reclaim() until another zone
allocation, a "reclaim" message or a resume schedules the work, so the
random or cache zones that filled up under load stay full.

Seen on md RAID5 members on 27 TB host-managed drives: after heavy
writes the random zones stayed full for hours on idle targets; the
workaround was a timer sending "dmsetup message <dev> 0 reclaim". With
this patch the emptied and filled zones were reclaimed within minutes of
the targets going idle, without the timer.

When reclaim is not needed right now, make dmz_schedule_reclaim() queue
the idle poll, so the work stays queued after every pass. It uses
queue_delayed_work(), which leaves already pending work alone, so the
other callers (zone allocation and the "reclaim" message) neither
postpone a pending poll nor delay a pending immediate reclaim.

Fixes: 3b1a94c88b79 ("dm zoned: drive-managed zoned block device target")
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Assisted-by: LLM
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index 88d56d3c86d..f1f9adf6039 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -641,7 +641,8 @@ void dmz_reclaim_bio_acc(struct dmz_reclaim *zrc)
 }
 
 /*
- * Start reclaim if necessary.
+ * Start reclaim if necessary. Otherwise make sure that an idle poll stays
+ * queued, without delaying work that is already pending.
  */
 void dmz_schedule_reclaim(struct dmz_reclaim *zrc)
 {
@@ -649,4 +650,6 @@ void dmz_schedule_reclaim(struct dmz_reclaim *zrc)
 
 	if (dmz_should_reclaim(zrc, p_unmap))
 		mod_delayed_work(zrc->wq, &zrc->work, 0);
+	else
+		queue_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
 }
-- 
2.43.0

[PATCH v5 4/4] dm zoned: do not busy-loop idle reclaim when no zone needs reclaim

[Kyrylo Biryuk]

An idle target whose random (and cache) zones are all unmapped keeps its
reclaim work running without pause:

  - since commit 90a9b8693f1b ("dm zoned: reclaim random zones when
    idle"), dmz_should_reclaim() returns true when the target is idle and
    the total number of random (and cache) zones is non-zero, whether or
    not any of them holds data;
  - dmz_do_reclaim() then finds no candidate and returns -EBUSY;
  - dmz_reclaim_work() runs dmz_check_dev(), which checks every device
    for media events (a TEST UNIT READY on SCSI disks), and
    dmz_schedule_reclaim() requeues the work with no delay because
    dmz_should_reclaim() is still true.

Without the previous patch, a pass ending while the target is busy and
has enough free zones can leave reclaim unqueued, which ends the loop;
keeping the idle poll armed makes it recur whenever the target becomes
idle. On three 27 TB host-managed drives used
without cache devices (all random zones free) it kept about three CPUs
busy and sent about 155,000 TEST UNIT READY commands per second to each
drive while the target was idle.

Only reclaim on idle when a random or cache zone actually holds data (the
intent of the "p_unmap < 100" test that commit replaced), and
handle -EBUSY like -ENOSPC: nothing can progress and no I/O failed, so
poll again after the idle period instead of rescheduling at once, and
skip the device check. Use queue_delayed_work() for that poll so work an
allocation has already queued is not postponed.

Fixes: 90a9b8693f1b ("dm zoned: reclaim random zones when idle")
Assisted-by: LLM
Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>
---
 drivers/md/dm-zoned-reclaim.c | 26 +++++++++++++++-----------
 1 file changed, 15 insertions(+), 11 deletions(-)

diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
index f1f9adf6039..0d62ade1466 100644
--- a/drivers/md/dm-zoned-reclaim.c
+++ b/drivers/md/dm-zoned-reclaim.c
@@ -470,9 +470,10 @@ static unsigned int dmz_reclaim_percentage(struct dmz_reclaim *zrc)
  */
 static bool dmz_should_reclaim(struct dmz_reclaim *zrc, unsigned int p_unmap)
 {
-	unsigned int nr_reclaim;
+	unsigned int nr_reclaim, nr_unmap;
 
 	nr_reclaim = dmz_nr_rnd_zones(zrc->metadata, zrc->dev_idx);
+	nr_unmap = dmz_nr_unmap_rnd_zones(zrc->metadata, zrc->dev_idx);
 
 	if (dmz_nr_cache_zones(zrc->metadata)) {
 		/*
@@ -483,10 +484,11 @@ static bool dmz_should_reclaim(struct dmz_reclaim *zrc, unsigned int p_unmap)
 		if (zrc->dev_idx == 0)
 			return false;
 		nr_reclaim += dmz_nr_cache_zones(zrc->metadata);
+		nr_unmap += dmz_nr_unmap_cache_zones(zrc->metadata);
 	}
 
-	/* Reclaim when idle */
-	if (dmz_target_idle(zrc) && nr_reclaim)
+	/* Reclaim when idle, but only if a random or cache zone holds data */
+	if (dmz_target_idle(zrc) && nr_reclaim > nr_unmap)
 		return true;
 
 	/* If there are still plenty of cache zones, do not reclaim */
@@ -543,17 +545,19 @@ static void dmz_reclaim_work(struct work_struct *work)
 		dmz_nr_rnd_zones(zmd, zrc->dev_idx));
 
 	ret = dmz_do_reclaim(zrc);
+	if (ret == -ENOSPC || ret == -EBUSY) {
+		/*
+		 * No zone to copy into (-ENOSPC) or no zone to reclaim
+		 * (-EBUSY): nothing can progress now and no I/O failed.
+		 * Rescheduling immediately would spin; poll instead, without
+		 * postponing work that an allocation has already queued.
+		 */
+		queue_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
+		return;
+	}
 	if (ret && ret != -EINTR) {
 		if (!dmz_check_dev(zmd))
 			return;
-		if (ret == -ENOSPC) {
-			/*
-			 * No zone to copy into: nothing can progress now.
-			 * Rescheduling immediately would spin; poll instead.
-			 */
-			mod_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
-			return;
-		}
 	}
 
 	dmz_schedule_reclaim(zrc);
-- 
2.43.0

Re: [PATCH v5 4/4] dm zoned: do not busy-loop idle reclaim when no zone needs reclaim

[Damien Le Moal]

On 2026/10/13 16:55, Kyrylo Biryuk wrote:
> An idle target whose random (and cache) zones are all unmapped keeps its
> reclaim work running without pause:
> 
>   - since commit 90a9b8693f1b ("dm zoned: reclaim random zones when
>     idle"), dmz_should_reclaim() returns true when the target is idle and
>     the total number of random (and cache) zones is non-zero, whether or
>     not any of them holds data;
>   - dmz_do_reclaim() then finds no candidate and returns -EBUSY;
>   - dmz_reclaim_work() runs dmz_check_dev(), which checks every device
>     for media events (a TEST UNIT READY on SCSI disks), and
>     dmz_schedule_reclaim() requeues the work with no delay because
>     dmz_should_reclaim() is still true.
> 
> Without the previous patch, a pass ending while the target is busy and
> has enough free zones can leave reclaim unqueued, which ends the loop;
> keeping the idle poll armed makes it recur whenever the target becomes
> idle. On three 27 TB host-managed drives used
> without cache devices (all random zones free) it kept about three CPUs
> busy and sent about 155,000 TEST UNIT READY commands per second to each
> drive while the target was idle.
> 
> Only reclaim on idle when a random or cache zone actually holds data (the
> intent of the "p_unmap < 100" test that commit replaced), and
> handle -EBUSY like -ENOSPC: nothing can progress and no I/O failed, so
> poll again after the idle period instead of rescheduling at once, and
> skip the device check. Use queue_delayed_work() for that poll so work an
> allocation has already queued is not postponed.
> 
> Fixes: 90a9b8693f1b ("dm zoned: reclaim random zones when idle")
> Assisted-by: LLM
> Signed-off-by: Kyrylo Biryuk <illofspeed@gmail.com>

This is undoing what patch 1 does. So can you squash this into patch 1 to avoid
the churn?

> ---
>  drivers/md/dm-zoned-reclaim.c | 26 +++++++++++++++-----------
>  1 file changed, 15 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/md/dm-zoned-reclaim.c b/drivers/md/dm-zoned-reclaim.c
> index f1f9adf6039..0d62ade1466 100644
> --- a/drivers/md/dm-zoned-reclaim.c
> +++ b/drivers/md/dm-zoned-reclaim.c
> @@ -470,9 +470,10 @@ static unsigned int dmz_reclaim_percentage(struct dmz_reclaim *zrc)
>   */
>  static bool dmz_should_reclaim(struct dmz_reclaim *zrc, unsigned int p_unmap)
>  {
> -	unsigned int nr_reclaim;
> +	unsigned int nr_reclaim, nr_unmap;
>  
>  	nr_reclaim = dmz_nr_rnd_zones(zrc->metadata, zrc->dev_idx);
> +	nr_unmap = dmz_nr_unmap_rnd_zones(zrc->metadata, zrc->dev_idx);
>  
>  	if (dmz_nr_cache_zones(zrc->metadata)) {
>  		/*
> @@ -483,10 +484,11 @@ static bool dmz_should_reclaim(struct dmz_reclaim *zrc, unsigned int p_unmap)
>  		if (zrc->dev_idx == 0)
>  			return false;
>  		nr_reclaim += dmz_nr_cache_zones(zrc->metadata);
> +		nr_unmap += dmz_nr_unmap_cache_zones(zrc->metadata);
>  	}
>  
> -	/* Reclaim when idle */
> -	if (dmz_target_idle(zrc) && nr_reclaim)
> +	/* Reclaim when idle, but only if a random or cache zone holds data */
> +	if (dmz_target_idle(zrc) && nr_reclaim > nr_unmap)
>  		return true;
>  
>  	/* If there are still plenty of cache zones, do not reclaim */
> @@ -543,17 +545,19 @@ static void dmz_reclaim_work(struct work_struct *work)
>  		dmz_nr_rnd_zones(zmd, zrc->dev_idx));
>  
>  	ret = dmz_do_reclaim(zrc);
> +	if (ret == -ENOSPC || ret == -EBUSY) {
> +		/*
> +		 * No zone to copy into (-ENOSPC) or no zone to reclaim
> +		 * (-EBUSY): nothing can progress now and no I/O failed.
> +		 * Rescheduling immediately would spin; poll instead, without
> +		 * postponing work that an allocation has already queued.
> +		 */
> +		queue_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
> +		return;
> +	}
>  	if (ret && ret != -EINTR) {
>  		if (!dmz_check_dev(zmd))
>  			return;
> -		if (ret == -ENOSPC) {
> -			/*
> -			 * No zone to copy into: nothing can progress now.
> -			 * Rescheduling immediately would spin; poll instead.
> -			 */
> -			mod_delayed_work(zrc->wq, &zrc->work, DMZ_IDLE_PERIOD);
> -			return;
> -		}
>  	}
>  
>  	dmz_schedule_reclaim(zrc);


-- 
Damien Le Moal
Western Digital Research

Re: [PATCH v5 0/4] dm zoned: fix idle reclaim problems

[Mikulas Patocka]

On Tue, 13 Oct 2026, Kyrylo Biryuk wrote:

> Mikulas,
> 
> I noticed too late that v3 of this series is already in dm-7.4 and
> for-next (24646f6ed3, 2cca719c63, 5829edd027), so v4 crossed with it.
> This v5 is v4 plus one more fix; if you prefer to keep v3, I can send
> the v4 changes and patch 4 as incremental patches on top of dm-7.4
> instead - please tell me which you want.

Hi

Linus doesn't want development being done close to the start of the merge 
window (or during the merge window), so I reverted the patches from my 
tree and I will postpone them.

I suggest - separate the patches into bugfixes (they have the "Fixes:" 
line and the "Cc: stable@vger.kernel.org" line) and performance 
enhancements (without "Fixes:" and "Cc: stable@vger.kernel.org"). The 
bugfixes may go in later during the 7.4 cycle, the performance 
enhancements will wait for the 7.5 merge window.

Mikulas

> Patch 4 is new. While measuring an idle pool I found that an idle
> target whose random (and cache) zones are all free never stops running
> its reclaim work: since 90a9b8693f1b ("dm zoned: reclaim random zones
> when idle"), dmz_should_reclaim() treats "idle" plus a non-zero total
> number of random zones as "reclaim now", dmz_do_reclaim() finds no
> candidate and returns -EBUSY, dmz_check_dev() checks every device for
> media events (a TEST UNIT READY on SCSI disks), and the work is requeued
> with no delay. In mainline the loop can end when a pass finishes while
> the target is busy with enough free zones; with patch 3 the idle poll
> stays armed, so the loop recurs whenever the target becomes idle.
> 
> Measured on 7.3-rc7 with three emulated host-managed members of an md
> RAID5 (idle, 30 s average):
> 
>   stock dm-zoned        181,195 SCSI commands/s   32 % kernel CPU
>   v4 (patches 1-3)      187,707 SCSI commands/s   31 % kernel CPU
>   v5 (patches 1-4)            0 SCSI commands/s    0 % kernel CPU
> 
> With patch 4, the random zones filled by a 2 GiB write burst are still
> reclaimed once the target is idle, and on three 27 TB host-managed
> drives (7.2.10 with these patches) an idle pool went from about 155,000
> TEST UNIT READY commands per second per drive to none.
> 
> Changes since v4:
>  - new patch 4 (above)
>  - patch 3: Reviewed-by from Damien Le Moal
>  - patches 1-3 otherwise unchanged
> 
> Kyrylo Biryuk (4):
>   dm zoned: back off when reclaim finds no destination zone
>   dm zoned: do not reclaim a random zone into another random zone
>   dm zoned: keep polling for idle reclaim after a reclaim pass
>   dm zoned: do not busy-loop idle reclaim when no zone needs reclaim
> 
>  drivers/md/dm-zoned-reclaim.c | 31 ++++++++++++++++++++++++++-----
>  1 file changed, 26 insertions(+), 5 deletions(-)
>