mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Zhang Yi <yi.zhang@huaweicloud.com>
To: "Jose Fernandez (Anthropic)" <jose.fernandez@linux.dev>,
	Theodore Ts'o <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>, Baokun Li <libaokun@linux.alibaba.com>,
	Ojaswin Mujoo <ojaswin@linux.ibm.com>,
	Andreas Dilger <adilger.kernel@dilger.ca>,
	"Ritesh Harjani (IBM)" <ritesh.list@gmail.com>,
	guolingxing <guolingxing@tju.edu.cn>,
	Josef Bacik <josef@toxicpanda.com>,
	Chris Mason <mason@kernel.org>,
	linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org,
	regressions@lists.linux.dev
Subject: Re: [REGRESSION] ext4: silent data loss when a split at I/O completion gets ENOSPC
Date: Wed, 18 Nov 2026 11:44:08 +0800	[thread overview]
Message-ID: <10712329-a587-496d-babf-620ebfd0e60e@huaweicloud.com> (raw)
In-Reply-To: <20261117-jf-ext4-enospc-reply-zhangyi-v9-b19eeb966ad0@linux.dev>

On 11/17/2026 10:23 PM, Jose Fernandez (Anthropic) wrote:
> On Tue, Nov 17, 2026 Zhang Yi wrote:
>> I think the solution should be based on Ojaswin's counting approach in
>> [1] or the PENDING_CONVERSION solution we discussed, and then, after
>> extent splitting fails, try to flush the journal and retry.
> 
> Thanks for the quick POC. Our agents ran it on [1] rebased on
> 23c62304dde2 with the report's reproducer, in two separate setups (QEMU
> without KVM). None of 72 files lost data in either, against 42 and 53 of
> 72 with [1] alone.
> 
> They found two gaps:
> 
> - The new branch in ext4_ext_handle_unwritten_extents() skips both
>   ext4_update_inode_fsync_trans() calls, so fdatasync() and O_DSYNC can
>   return before the commit. On a full filesystem mounted with
>   commit=3600, a shutdown test lost the write in all 13 buffered
>   fdatasync() and O_DSYNC cases that reach the zeroout. They pass with
>   [1] alone and with that call added.

Yes, this is a pre-existing issue introduced by 5f1a1cccebf8 ("ext4:
correct the mapping status if the extent has been zeroed"), but it
healed itself after 5f18f60d56c0 when extent splitting was deferred
until after I/O completion.

Now the current POC brings the splitting before submitting I/O logic
back in the bh buffered I/O path, so the issue has resurfaced. I can
address this issue together in the formal fix patch.

> 
> - Direct writes into preallocated files can still get ENOSPC with the
>   POC, on a few percent of files in both setups, with no data lost.
> 

Could you provide a reproducer? My guess is that high concurrency causes
the blocks freed by my current process to be taken by I/O issued by
other processes, since we can no longer simply fall back to zeroing the
entire extent as before, which was basically always effective.

> The POC tries the zeroout before the commit and retry. Should it be the
> other way around? 

I agree that doing zeroout first and then retrying does look a bit odd,
and it is not friendly to large extents. However, since the previous
logic would just do the zeroing directly when space allocation failed
during splitting, the current POC does not break anything.

Besides, Ojaswin plans to refactor this part later and move zeroout into
the fast path, so I suggest we keep it this way for now to reduce the
probability of ENOSPC, and discuss further approaches later.

> And will you take guolingxing's patch ([2] in my report) as well?

Yeah, we should merge this patch as well.

Best Regards,
Yi.


  reply	other threads:[~2026-11-18  3:44 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  2:27 Jose Fernandez (Anthropic)
2026-11-17 11:37 ` Zhang Yi
2026-11-17 14:23   ` Jose Fernandez (Anthropic)
2026-11-18  3:44     ` Zhang Yi [this message]
2026-11-18 14:06       ` Jose Fernandez (Anthropic)
2026-11-18  7:55   ` Ojaswin Mujoo
2026-11-18 10:48     ` Zhang Yi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=10712329-a587-496d-babf-620ebfd0e60e@huaweicloud.com \
    --to=yi.zhang@huaweicloud.com \
    --cc=adilger.kernel@dilger.ca \
    --cc=guolingxing@tju.edu.cn \
    --cc=jack@suse.cz \
    --cc=jose.fernandez@linux.dev \
    --cc=josef@toxicpanda.com \
    --cc=libaokun@linux.alibaba.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mason@kernel.org \
    --cc=ojaswin@linux.ibm.com \
    --cc=regressions@lists.linux.dev \
    --cc=ritesh.list@gmail.com \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®