* [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
@ 2026-11-18 17:56 Miroslav Cimerman
2026-11-18 20:37 ` Qu Wenruo
0 siblings, 1 reply; 6+ messages in thread
From: Miroslav Cimerman @ 2026-11-18 17:56 UTC (permalink / raw)
To: linux-btrfs
Cc: David Sterba, Chris Mason, Qu Wenruo, Naohiro Aota,
Johannes Thumshirn, linux-kernel
On zoned devices scrub can't rewrite a bad copy in place. It calls
btrfs_repair_one_zone() to relocate the block group instead, and counts
the repaired blocks as corrected.
On a degraded mount btrfs_repair_one_zone() relocates nothing ("Do not
attempt to repair in degraded state"), but scrub still counts the
blocks as corrected and logs "fixed up error" for each. The bad copies
stay, and the next scrub finds and "fixes" the same errors again. With
zoned RAID1 mounted -o degraded with both devices present and read
errors in one device's metadata zone, two scrubs in a row both report:
Error summary: read=96
Corrected: 36
Uncorrectable: 60
Count such errors as uncorrectable, which the uapi header defines as
"no intact copy was found or the writeback failed". Dev-replace, which
writes the repaired blocks to the target device, and read-only scrub
are not changed.
Fixes: 54765392a1b9 ("btrfs: scrub: introduce helper to queue a stripe for scrub")
Signed-off-by: Miroslav Cimerman <mc@raidlab.org>
---
Keeping these errors as corrected and only logging that the repair was
skipped would leave a false positive: the counters would still report
the bad copies as fixed, and the next scrub finds them again. A new
counter for errors with an intact copy that were not written back would
be another option, but it changes the uapi. A read-only scrub, which
also counts errors with an intact copy as corrected without writing
anything, is not changed here.
Found with FuzzyRAID, a fault-injection tester for RAID that is not
public at the moment. The analysis, the fix and the commit message
were prepared with the help of an LLM, which also tested it. I reviewed
everything.
It also applies on top of "btrfs: scrub: fix several races which can
lead to UAF" (v2), with offsets.
fs/btrfs/scrub.c | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 98362c91bce4..9dcb0e7e3d0b 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -323,6 +323,12 @@ struct scrub_error_records {
*/
unsigned long init_error_bitmap;
+ /*
+ * Set if the repaired blocks could not be written back, thus all the
+ * initial errors are still on disk.
+ */
+ bool repair_skipped;
+
unsigned int nr_io_errors;
unsigned int nr_csum_errors;
unsigned int nr_meta_errors;
@@ -992,7 +998,7 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
struct btrfs_fs_info *fs_info = sctx->fs_info;
struct btrfs_device *dev = NULL;
const unsigned long extent_bitmap = scrub_bitmap_read_has_extent(stripe);
- const unsigned long error_bitmap = scrub_bitmap_read_error(stripe);
+ unsigned long error_bitmap = scrub_bitmap_read_error(stripe);
u64 physical = 0;
int nr_data_sectors = 0;
int nr_meta_sectors = 0;
@@ -1003,6 +1009,11 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
if (test_bit(SCRUB_STRIPE_FLAG_NO_REPORT, &stripe->state))
return;
+ /* Not written back, the repaired blocks are still bad on disk. */
+ if (errors->repair_skipped)
+ bitmap_or(&error_bitmap, &error_bitmap,
+ &errors->init_error_bitmap, stripe->nr_sectors);
+
/*
* Init needed infos for error reporting.
*
@@ -1227,6 +1238,15 @@ static void scrub_stripe_read_repair_worker(struct work_struct *work)
if (!sctx->readonly && !bitmap_empty(&repaired, stripe->nr_sectors)) {
if (btrfs_is_zoned(fs_info)) {
btrfs_repair_one_zone(fs_info, sctx->stripes[0].bg->start);
+ /*
+ * On a degraded mount btrfs_repair_one_zone() does not
+ * relocate the block group and the bad copies stay,
+ * unless dev-replace writes the repaired blocks to the
+ * target device.
+ */
+ if (btrfs_test_opt(fs_info, DEGRADED) &&
+ !sctx->is_dev_replace)
+ errors.repair_skipped = true;
} else {
scrub_write_sectors(sctx, stripe, repaired, false);
wait_scrub_stripe_io(stripe);
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
2026-11-18 17:56 [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount Miroslav Cimerman
@ 2026-11-18 20:37 ` Qu Wenruo
2026-11-19 1:22 ` Miroslav Cimerman
0 siblings, 1 reply; 6+ messages in thread
From: Qu Wenruo @ 2026-11-18 20:37 UTC (permalink / raw)
To: Miroslav Cimerman, linux-btrfs
Cc: David Sterba, Chris Mason, Naohiro Aota, Johannes Thumshirn,
linux-kernel
在 2026/11/19 04:26, Miroslav Cimerman 写道:
> On zoned devices scrub can't rewrite a bad copy in place. It calls
> btrfs_repair_one_zone() to relocate the block group instead, and counts
> the repaired blocks as corrected.
>
> On a degraded mount btrfs_repair_one_zone() relocates nothing ("Do not
> attempt to repair in degraded state"),
I think this is a little weird, any special reason for this in the first
place?
> but scrub still counts the
> blocks as corrected and logs "fixed up error" for each. The bad copies
> stay, and the next scrub finds and "fixes" the same errors again. With
> zoned RAID1 mounted -o degraded with both devices present and read
> errors in one device's metadata zone, two scrubs in a row both report:
>
> Error summary: read=96
> Corrected: 36
> Uncorrectable: 60
>
> Count such errors as uncorrectable, which the uapi header defines as
> "no intact copy was found or the writeback failed". Dev-replace, which
> writes the repaired blocks to the target device, and read-only scrub
> are not changed.
>
> Fixes: 54765392a1b9 ("btrfs: scrub: introduce helper to queue a stripe for scrub")
I do not think this is the correct fixes tag, that commit is just
calling btrfs_repair_one_zone(), and btrfs is already calling that before.
> Signed-off-by: Miroslav Cimerman <mc@raidlab.org>
> ---
> Keeping these errors as corrected and only logging that the repair was
> skipped would leave a false positive: the counters would still report
> the bad copies as fixed, and the next scrub finds them again. A new
> counter for errors with an intact copy that were not written back would
> be another option, but it changes the uapi. A read-only scrub, which
> also counts errors with an intact copy as corrected without writing
> anything, is not changed here.
>
> Found with FuzzyRAID, a fault-injection tester for RAID that is not
> public at the moment. The analysis, the fix and the commit message
> were prepared with the help of an LLM, which also tested it. I reviewed
> everything.
>
> It also applies on top of "btrfs: scrub: fix several races which can
> lead to UAF" (v2), with offsets.
>
> fs/btrfs/scrub.c | 22 +++++++++++++++++++++-
> 1 file changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
> index 98362c91bce4..9dcb0e7e3d0b 100644
> --- a/fs/btrfs/scrub.c
> +++ b/fs/btrfs/scrub.c
> @@ -323,6 +323,12 @@ struct scrub_error_records {
> */
> unsigned long init_error_bitmap;
>
> + /*
> + * Set if the repaired blocks could not be written back, thus all the
> + * initial errors are still on disk.
> + */
> + bool repair_skipped;
> +
> unsigned int nr_io_errors;
> unsigned int nr_csum_errors;
> unsigned int nr_meta_errors;
> @@ -992,7 +998,7 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
> struct btrfs_fs_info *fs_info = sctx->fs_info;
> struct btrfs_device *dev = NULL;
> const unsigned long extent_bitmap = scrub_bitmap_read_has_extent(stripe);
> - const unsigned long error_bitmap = scrub_bitmap_read_error(stripe);
> + unsigned long error_bitmap = scrub_bitmap_read_error(stripe);
> u64 physical = 0;
> int nr_data_sectors = 0;
> int nr_meta_sectors = 0;
> @@ -1003,6 +1009,11 @@ static void scrub_stripe_report_errors(struct scrub_ctx *sctx,
> if (test_bit(SCRUB_STRIPE_FLAG_NO_REPORT, &stripe->state))
> return;
>
> + /* Not written back, the repaired blocks are still bad on disk. */
> + if (errors->repair_skipped)
> + bitmap_or(&error_bitmap, &error_bitmap,
> + &errors->init_error_bitmap, stripe->nr_sectors);
> +
> /*
> * Init needed infos for error reporting.
> *
> @@ -1227,6 +1238,15 @@ static void scrub_stripe_read_repair_worker(struct work_struct *work)
> if (!sctx->readonly && !bitmap_empty(&repaired, stripe->nr_sectors)) {
> if (btrfs_is_zoned(fs_info)) {
> btrfs_repair_one_zone(fs_info, sctx->stripes[0].bg->start);
> + /*
> + * On a degraded mount btrfs_repair_one_zone() does not
> + * relocate the block group and the bad copies stay,
> + * unless dev-replace writes the repaired blocks to the
> + * target device.
> + */
> + if (btrfs_test_opt(fs_info, DEGRADED) &&
> + !sctx->is_dev_replace)
> + errors.repair_skipped = true;
> } else {
> scrub_write_sectors(sctx, stripe, repaired, false);
> wait_scrub_stripe_io(stripe);
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
2026-11-18 20:37 ` Qu Wenruo
@ 2026-11-19 1:22 ` Miroslav Cimerman
2026-11-19 2:37 ` Qu Wenruo
0 siblings, 1 reply; 6+ messages in thread
From: Miroslav Cimerman @ 2026-11-19 1:22 UTC (permalink / raw)
To: Qu Wenruo
Cc: linux-btrfs, David Sterba, Chris Mason, Naohiro Aota,
Johannes Thumshirn, linux-kernel
Qu Wenruo <wqu@suse.com> wrote:
> 在 2026/11/19 04:26, Miroslav Cimerman 写道:
> > On a degraded mount btrfs_repair_one_zone() relocates nothing ("Do not
> > attempt to repair in degraded state"),
>
> I think this is a little weird, any special reason for this in the first
> place?
It was probably meant to check for a missing device (so that a mirror's
redundancy won't be downgraded by the relocation). But because it checks
with the mount flag, a -o degraded mount with no missing devices does
not get bad copies repaired by scrub.
I propose a v2 series, where 1/2 replaces the current mount flag check
with a missing devices check, and 2/2 is this patch modified to use the
same missing devices check as 1/2, so they are in sync. 2/2 is still
needed because for example RAID1C3 can have all: a good copy, bad copy,
and a missing device.
Are you okay with the proposal, Qu?
> > Fixes: 54765392a1b9 ("btrfs: scrub: introduce helper to queue a stripe for scrub")
>
> I do not think this is the correct fixes tag, that commit is just
> calling btrfs_repair_one_zone(), and btrfs is already calling that before.
You are right, the Fixes tag should be:
Fixes: f7ef5287a63d ("btrfs: zoned: relocate block group to repair IO failure in zoned filesystems")
Thanks,
Miroslav
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
2026-11-19 1:22 ` Miroslav Cimerman
@ 2026-11-19 2:37 ` Qu Wenruo
2026-11-19 21:49 ` Miroslav Cimerman
0 siblings, 1 reply; 6+ messages in thread
From: Qu Wenruo @ 2026-11-19 2:37 UTC (permalink / raw)
To: Miroslav Cimerman, Qu Wenruo
Cc: linux-btrfs, David Sterba, Chris Mason, Naohiro Aota,
Johannes Thumshirn, linux-kernel
在 2026/11/19 11:52, Miroslav Cimerman 写道:
> Qu Wenruo <wqu@suse.com> wrote:
>
>> 在 2026/11/19 04:26, Miroslav Cimerman 写道:
>
>>> On a degraded mount btrfs_repair_one_zone() relocates nothing ("Do not
>>> attempt to repair in degraded state"),
>>
>> I think this is a little weird, any special reason for this in the first
>> place?
>
> It was probably meant to check for a missing device (so that a mirror's
> redundancy won't be downgraded by the relocation).
If it's a workaround for the well known new-SINGLE chunks during
degraded writes, it may be a little over-killed.
A lot of btrfs-progs commands will do give extra warning when such
unexpected profiles are detected, so end users can later balance them
out after recovering.
(It's not the best solution, but it is not zoned specific either)
I'm more concerned if it's some extra requirement for zoned RAID
profiles, e.g. to keep the writer pointers in sync etc.
> But because it checks
> with the mount flag, a -o degraded mount with no missing devices does
> not get bad copies repaired by scrub.
>
> I propose a v2 series, where 1/2 replaces the current mount flag check
> with a missing devices check, and 2/2 is this patch modified to use the
> same missing devices check as 1/2, so they are in sync. 2/2 is still
> needed because for example RAID1C3 can have all: a good copy, bad copy,
> and a missing device.
And I'd prefer if you can get some helper to know if this block group
can be repaired, so that no duplicated checks between repair and scrub.
Thanks,
Qu
>
> Are you okay with the proposal, Qu?
>
>>> Fixes: 54765392a1b9 ("btrfs: scrub: introduce helper to queue a stripe for scrub")
>>
>> I do not think this is the correct fixes tag, that commit is just
>> calling btrfs_repair_one_zone(), and btrfs is already calling that before.
>
> You are right, the Fixes tag should be:
>
> Fixes: f7ef5287a63d ("btrfs: zoned: relocate block group to repair IO failure in zoned filesystems")
>
> Thanks,
> Miroslav
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
2026-11-19 2:37 ` Qu Wenruo
@ 2026-11-19 21:49 ` Miroslav Cimerman
2026-11-19 22:33 ` Qu Wenruo
0 siblings, 1 reply; 6+ messages in thread
From: Miroslav Cimerman @ 2026-11-19 21:49 UTC (permalink / raw)
To: Qu Wenruo
Cc: Qu Wenruo, linux-btrfs, David Sterba, Chris Mason, Naohiro Aota,
Johannes Thumshirn, linux-kernel
Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> If it's a workaround for the well known new-SINGLE chunks during
> degraded writes, it may be a little over-killed.
True, balance and block group reclaim already relocate block groups on
degraded mounts with no such check.
> I'm more concerned if it's some extra requirement for zoned RAID
> profiles, e.g. to keep the writer pointers in sync etc.
None that the repair would break: a regular mount needs equal write
pointers, but a missing device falls behind on every degraded write
anyway, not only on a repair. The problem is when that device comes
back stale: two days ago my tester found that a degraded mount could
then allocate over committed extents, which aborted the transaction or
crashed. The series "btrfs: zoned: RAID1 block groups with missing or
stale stripes" fixes that:
https://lore.kernel.org/linux-btrfs/20261119213835.103131-1-mc@raidlab.org/
In testing with the skip removed, the relocation with a device really
missing works: RAID1 on three devices stays RAID1, RAID1C3 on three
devices gets single block groups, a second scrub is clean, and the
missing device mounts normally when it comes back.
One thing it needs: without the skip, a degraded mount that reads a bad
copy starts the relocation during mount. In testing, it raced with the
mount, which failed and hit the uncommitted transaction ASSERT on its
error path. "btrfs: zoned: wait for the relocating repair kthread
during unmount" prevents that: with it, no repair starts before
BTRFS_FS_OPEN is set:
https://lore.kernel.org/linux-btrfs/20261118004808.201460-1-mc@raidlab.org/
> And I'd prefer if you can get some helper to know if this block group
> can be repaired, so that no duplicated checks between repair and scrub.
With that repair kthread patch, the skip can be dropped completely,
so no check is left to share, and scrub needs no change for degraded
mounts. It still counts errors as corrected when a repair is not
started, e.g. while another one runs, just as on a regular mount.
In testing, with the series and the repair kthread patch applied and
the skip removed, a stale device came back. A few scrubs on the
degraded mount relocated the block groups it was behind on, and then a
regular mount worked again, without a balance. Only a block group where
scrub finds no error still needs a balance, e.g. when the missed range
holds only nodatasum data.
One downside: when fewer devices are left than the profile needs, the
repair relocates the block group as single, without the missing
device's copy. A RAID1C3 or RAID1C4 block group then keeps one copy
until a balance. Without the repair, it would get its copies back when
the device returns.
Are you OK with a v2 that drops the skip completely, on top of that
repair kthread patch?
Thanks,
Miroslav
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount
2026-11-19 21:49 ` Miroslav Cimerman
@ 2026-11-19 22:33 ` Qu Wenruo
0 siblings, 0 replies; 6+ messages in thread
From: Qu Wenruo @ 2026-11-19 22:33 UTC (permalink / raw)
To: Miroslav Cimerman, Qu Wenruo
Cc: linux-btrfs, David Sterba, Chris Mason, Naohiro Aota,
Johannes Thumshirn, linux-kernel
在 2026/11/20 08:19, Miroslav Cimerman 写道:
> Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>> If it's a workaround for the well known new-SINGLE chunks during
>> degraded writes, it may be a little over-killed.
>
> True, balance and block group reclaim already relocate block groups on
> degraded mounts with no such check.
>
>> I'm more concerned if it's some extra requirement for zoned RAID
>> profiles, e.g. to keep the writer pointers in sync etc.
>
> None that the repair would break: a regular mount needs equal write
> pointers, but a missing device falls behind on every degraded write
> anyway, not only on a repair. The problem is when that device comes
> back stale: two days ago my tester found that a degraded mount could
> then allocate over committed extents, which aborted the transaction or
> crashed. The series "btrfs: zoned: RAID1 block groups with missing or
> stale stripes" fixes that:
> https://lore.kernel.org/linux-btrfs/20261119213835.103131-1-mc@raidlab.org/
>
> In testing with the skip removed, the relocation with a device really
> missing works: RAID1 on three devices stays RAID1, RAID1C3 on three
> devices gets single block groups, a second scrub is clean, and the
> missing device mounts normally when it comes back.
>
> One thing it needs: without the skip, a degraded mount that reads a bad
> copy starts the relocation during mount. In testing, it raced with the
> mount, which failed and hit the uncommitted transaction ASSERT on its
> error path. "btrfs: zoned: wait for the relocating repair kthread
> during unmount" prevents that: with it, no repair starts before
> BTRFS_FS_OPEN is set:
> https://lore.kernel.org/linux-btrfs/20261118004808.201460-1-mc@raidlab.org/
>
>> And I'd prefer if you can get some helper to know if this block group
>> can be repaired, so that no duplicated checks between repair and scrub.
>
> With that repair kthread patch, the skip can be dropped completely,
> so no check is left to share, and scrub needs no change for degraded
> mounts. It still counts errors as corrected when a repair is not
> started, e.g. while another one runs, just as on a regular mount.
>
> In testing, with the series and the repair kthread patch applied and
> the skip removed, a stale device came back. A few scrubs on the
> degraded mount relocated the block groups it was behind on, and then a
> regular mount worked again, without a balance. Only a block group where
> scrub finds no error still needs a balance, e.g. when the missed range
> holds only nodatasum data.
>
> One downside: when fewer devices are left than the profile needs, the
> repair relocates the block group as single, without the missing
> device's copy. A RAID1C3 or RAID1C4 block group then keeps one copy
> until a balance. Without the repair, it would get its copies back when
> the device returns.
>
> Are you OK with a v2 that drops the skip completely, on top of that
> repair kthread patch?
I'm fine dropping that check, but it would be better to be a dedicated
patch for extra review from zoned experts.
Thanks,
Qu
>
> Thanks,
> Miroslav
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-11-19 22:33 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 17:56 [PATCH] btrfs: scrub: don't count zoned repairs as corrected on a degraded mount Miroslav Cimerman
2026-11-18 20:37 ` Qu Wenruo
2026-11-19 1:22 ` Miroslav Cimerman
2026-11-19 2:37 ` Qu Wenruo
2026-11-19 21:49 ` Miroslav Cimerman
2026-11-19 22:33 ` Qu Wenruo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®