[PATCH v7 00/31] ext4: use iomap for regular file's buffered I/O path

[PATCH v7 00/31] ext4: use iomap for regular file's buffered I/O path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Hi all,

This is v7 of the series converting ext4's buffered I/O path from
buffer_head to iomap. This iteration doesn't have significant changes
since v6. It just fixes bugs pointed out by Sashiko, improves commit
messages and comments to prevent false positives from Sashiko as well
as some small optimizations. Rebased on 7.3-rc4 (93f51579e7df). Besides,
I noticed that my V6 has already been merged into the ext4 dev branch.
If you'd like me to send separate follow-up patches, just let me know.

You can get commits here:

 https://github.com/zhangyi089/linux/commits/ext4_buffered_iomap_v7/

Patch 01-02: Simplify truncate operations and drop unnecessary ordered
             I/O when zeroing EOF blocks. This prepares for the iomap
	     conversion.
Patch 03-20: Extend ext4_map_blocks() and implement core iomap buffered
             read/write, writeback, mmap, and partial block zeroing.
Patch 21-29: Handle ordered sequence for zeroing the EOF partial block
             and updating i_disksize when performing file extending on
	     unaligned EOF files.
Patch 30-31: Enable iomap buffered I/O path.


Tests and Performance:

Tested with xfstests-bld using -g auto, fast_commit, and 64k
configurations. No new test failures were observed, aside from the
known occasional failure of generic/127, which is an issue in the
MM large folio split logic and warrants a separate fix.

 https://lore.kernel.org/linux-fsdevel/a638a8fb-c184-4069-ae33-379ec12cd514@huaweicloud.com/

This is a patch set that fixes this issue and is awaiting upstream:

 https://lore.kernel.org/linux-mm/20260928120833.3440834-1-yi.zhang@huaweicloud.com/

Performance was tested with FIO on a 4-core virtual machine with an
Intel(R) Xeon(R) Platinum 8380 CPU, backed by a 150GB RAM-backed
virtual-io block device (Since there are no significant changes from
v5, the performance data remains unchanged).

Write perf tests (MiB/s)
========================

  Sync  : 1 = fsync() after each write
  OW    : 1 = overwrite existing blocks
  Dnc   : 1 = RWF_DONTCACHE used

  #1    : only write page cache, no writeback
  #2    : writeback triggered (set dirty_background_bytes = 100MB and
          dirty_bytes = 200MB)


  Config                   BS       ext4+bh   ext4+iomap    delta
  ---------------------------------------------------------------
  Sync=0 OW=0 Dnc=0 (#1)   1k          41.8         42.0    ~0%
  Sync=0 OW=0 Dnc=0 (#1)   4k           170          176    +4%
  Sync=0 OW=0 Dnc=0 (#1)   64k         1816         1981    +9%
  Sync=0 OW=0 Dnc=0 (#1)   1m          4295         5780   +35%

  Sync=0 OW=0 Dnc=0 (#2)   1k          42.6         43.2    +2%
  Sync=0 OW=0 Dnc=0 (#2)   4k           137          140    +2%
  Sync=0 OW=0 Dnc=0 (#2)   64k          734          957   +30%
  Sync=0 OW=0 Dnc=0 (#2)   1m          1460         1683   +15%

  Sync=0 OW=0 Dnc=1        1k          14.6         15.0    +3%
  Sync=0 OW=0 Dnc=1        4k          67.9         61.9    -9%
  Sync=0 OW=0 Dnc=1        64k          415          698   +68%
  Sync=0 OW=0 Dnc=1        1m          1404         2883  +105%

  Sync=1 OW=0 Dnc=0        1k          0.71         0.60   -15%
  Sync=1 OW=0 Dnc=0        4k          2.04         1.89    -7%
  Sync=1 OW=0 Dnc=0        64k         32.6         32.4    -1%
  Sync=1 OW=0 Dnc=0        1m           253          277   +10%

  Sync=0 OW=1 Dnc=0 (#1)   1k          19.1         21.0   +10%
  Sync=0 OW=1 Dnc=0 (#1)   4k           164          179    +9%
  Sync=0 OW=1 Dnc=0 (#1)   64k         1776         1867    +5%
  Sync=0 OW=1 Dnc=0 (#1)   1m          4105         4879   +19%

  Sync=0 OW=1 Dnc=0 (#2)   1k          19.9         21.6    +8%
  Sync=0 OW=1 Dnc=0 (#2)   4k           148          157    +6%
  Sync=0 OW=1 Dnc=0 (#2)   64k          843          840    ~0%
  Sync=0 OW=1 Dnc=0 (#2)   1m          1579         1806   +14%

  Sync=0 OW=1 Dnc=1        1k          17.4         17.9    +3%
  Sync=0 OW=1 Dnc=1        4k          80.4         87.7    +9%
  Sync=0 OW=1 Dnc=1        64k          806          923   +14%
  Sync=0 OW=1 Dnc=1        1m          1070         1122    +5%

  Sync=1 OW=1 Dnc=0        1k          0.82         0.82     0%
  Sync=1 OW=1 Dnc=0        4k          3.55         3.56    ~0%
  Sync=1 OW=1 Dnc=0        64k         48.4         49.8    +3%
  Sync=1 OW=1 Dnc=0        1m           325          336    +4%


Read tests (MiB/s)
==================

  READ HOLE           : read of a hole
  READ DATA           : read of file data without pre-exists page cache
  READ PRECACHED DATA : read of file data with warm page cache

  Scenario                 BS       ext4+bh   ext4+iomap    delta
  ---------------------------------------------------------------
  READ HOLE                1k          46.6         47.1    +1%
  READ HOLE                4k           189          198    +5%
  READ HOLE                64k         2011         2116    +5%
  READ HOLE                1m          4538         4684    +3%

  READ DATA                1k          44.6         50.4   +13%
  READ DATA                4k           178          182    +2%
  READ DATA                64k          741          768    +4%
  READ DATA                1m          1143         1210    +6%

  READ PRECACHED DATA      1k          50.8         50.7    ~0%
  READ PRECACHED DATA      4k           179          173    -3%
  READ PRECACHED DATA      64k         2009         1989    -1%
  READ PRECACHED DATA      1m          6296         6294    ~0%

Large I/O scenarios show significant write performance improvement,
while small I/O in synchronous and non-cached scenarios experience a
slight regression. Read performance shows no significant difference.


Changes since v6:
 - Rebased on 7.3-rc4 (93f51579e7df).
 - Collects RVB labels from Ojaswin.
 - Drop patch 03 "ext4: skip ordered I/O wait when zeroing beyond
   i_disksize block" in v6, since this optimization is not safe under
   concurrent writeback and file extending.
 - In patch 06, add comments about passing empty @handle parameter for
   ext4_map_blocks() and simplify the error return paths.
 - In patch 07, add missing check_block_validity() while returning
   unwritten blocks.
 - In patch 08, explicitly handle the newly added zero return value from
   ext4_map_blocks() when meeting a hole during block allocation in
   mpage_map_one_extent().
 - Add patch 14, ensure there are enough credits for unwritten extents
   to unwritten extents conversion on the iomap writeback completion
   path.
 - In patch 15, add retry limitation on block allocation during
   writeback.
 - In patch 19, add printing of the pblk for the file mapping.
 - In patch 21, use clear_and_wake_up_bit() instead of open coding.
 - In patch 22, Optimize the pending I/O writeback logic for the
   blocksize < PAGE_SIZE case, to avoid an unnecessary separate
   submission when the writeback range and the pending range are in the
   same page. (Ojaswin)
 - In patch 25, fix the issue where i_disksize is not updated when
   performing a block-aligned zero range in synchronous mode.
 - In patch 27, check disksize when setting the DISKSIZE_GROW_PENDING bit
   because we drop the previous optimization in ext4_block_zero_eof() in
   patch 03 (v6), we need a reliable i_disksize when setting this bit.
 - In patch 28, only emit the wait disksize pending trace when the bit is
   actually set.
 - Improve commit messages in patches 04, 13 and 26 to prevent false
   positives from Sashiko.

v6:     https://lore.kernel.org/linux-ext4/20260903123543.2302999-1-yi.zhang@huaweicloud.com/
v5:     https://lore.kernel.org/linux-ext4/20260814093331.1703882-1-yi.zhang@huaweicloud.com/
v4:     https://lore.kernel.org/linux-ext4/20260511072344.191271-1-yi.zhang@huaweicloud.com/
v3:     https://lore.kernel.org/linux-ext4/20260422021042.4157510-1-yi.zhang@huaweicloud.com/
v2:     https://lore.kernel.org/linux-ext4/20260203062523.3869120-1-yi.zhang@huawei.com/
v1:     https://lore.kernel.org/linux-ext4/20241022111059.2566137-1-yi.zhang@huaweicloud.com/
RFC v4: https://lore.kernel.org/linux-ext4/20240410142948.2817554-1-yi.zhang@huaweicloud.com/
RFC v3: https://lore.kernel.org/linux-ext4/20240127015825.1608160-1-yi.zhang@huaweicloud.com/
RFC v2: https://lore.kernel.org/linux-ext4/20240102123918.799062-1-yi.zhang@huaweicloud.com/
RFC v1: https://lore.kernel.org/linux-ext4/20231123125121.4064694-1-yi.zhang@huaweicloud.com/

Comments and suggestions are welcome!

Thanks,
Yi.


Original Cover-letter
=====================

This series adds the iomap buffered I/O path support for regular files.
It implements the core iomap APIs on ext4 and introduces the
'buffered_iomap' mount option to enable the iomap buffered I/O path.
This supports default features, default mount options and bigalloc
feature. However, this does not support online defragmentation, inline
data, fsverify, fscrypt, non-extent inodes, and data=journal mode, we
will fall to buffered_head I/O path automatically if these features and
options are used.

This iomap buffered I/O path is not enabled by default because the
preceding features are not supported. Users can explicitly enable or
disable it via 'buffered_iomap' and 'nobuffered_iomap' mount options.

Key considerations

1. Lock ordering difference

   The lock ordering of folio lock and transaction start in the iomap
   path is the opposite of that in the buffer_head path.

2. data=ordered mode is not used

   Two main reasons:
   a) The lock ordering of folio lock and transaction start for
      data=ordered mode is opposite to the iomap path, which would cause
      a deadlock.
   b) The iomap writeback path does not support partial folio submission
      (required by data=ordered mode when block size < folio size, and
      it is currently handled by ext4_bio_write_folio()), which would
      also cause a deadlock.

   To replace data=ordered mode functionality:

   - For append write: Always allocate unwritten extents (dioread_nolock
     behavior) to prevent stale data exposure.

   - For post-EOF partial block zeroing, mark the inode as
     DISKSIZE_GROW_PENDING after zeroing the EOF block, and postpone
     updating i_disksize to i_size until the zeroed block has been
     written to disk or discarded (Suggested by Jan).

     Tradeoffs:
     1) i_disksize must be updated from io_end processing, so this
        mechanism is restricted to the buffered I/O iomap path due to
        locking constraints. This limitation will eventually go away
	once everything is converted to iomap.
     2) After a crash, i_disksize may be updated while the file content
        shows zeros. This is acceptable for simplicity and does not
	break any existing guarantees.

   - For online defragmentation: Not supported yet, needs further
     consideration.

3. Always enable dioread_nolock

   Two main reasons:
   a) Since data=ordered mode cannot be used, allocating written blocks
      directly would expose stale data.
   b) To optimize writeback, we should allocate blocks based on writeback
      length rather than per-folio mapping. Direct written allocation
      would over-allocate blocks.

   dioread_nolock has been the default mount option for many years, and
   Jan pointed out that we may no longer need to disable it, so gradually
   remove this mount option in the future.


Zhang Yi (31):
  ext4: simplify size updating in ext4_setattr()
  ext4: factor out ext4_truncate_[up|down]()
  ext4: set EXT4_MAP_NEW flag for delayed allocated blocks
  ext4: recheck extent status tree before block allocation
  ext4: fix orig_mlen initialization in ext4_map_blocks()
  ext4: allow ext4_map_blocks() to start its own transaction handle
  ext4: avoid unnecessary transaction in ext4_map_blocks() for unwritten
    extents
  ext4: skip block allocation for holes in the data submission path
  ext4: add iomap address space operations for buffered I/O
  ext4: implement buffered read path using iomap
  ext4: pass out extent seq counter when mapping da blocks
  ext4: do not use data=ordered mode for inodes using buffered iomap
    path
  ext4: implement buffered write path using iomap
  ext4: rework handle credit accounting for unwritten extent conversion
  ext4: implement writeback path using iomap
  ext4: implement mmap path using iomap
  ext4: implement partial block zero range path using iomap
  ext4: drain writeback before removing extents on the iomap path
  ext4: add block mapping tracepoints for iomap buffered I/O path
  ext4: disable online defrag when inode using iomap buffered I/O path
  ext4: add EXT4_STATE_DISKSIZE_GROW_PENDING state bit and helpers
  ext4: submit and wait for pending disksize-grow I/O on writeback
  ext4: advance i_disksize to i_size upon disksize-grow I/O completion
  ext4: defer i_disksize update while DISKSIZE_GROW_PENDING is set
  ext4: submit and wait for disksize-grow I/O in fallocate paths
  ext4: clear DISKSIZE_GROW_PENDING on truncate or error
  ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block
  ext4: add tracepoints for DISKSIZE_GROW_PENDING set, clear, and wait
  ext4: add tracepoints for EOF block zeroing and disksize-grow I/O
  ext4: partially enable iomap for the buffered I/O path of regular
    files
  ext4: introduce a mount option for iomap buffered I/O path

 fs/ext4/ext4.h              |   83 ++-
 fs/ext4/ext4_jbd2.c         |    8 +-
 fs/ext4/ext4_jbd2.h         |    7 +-
 fs/ext4/extents.c           |  105 ++-
 fs/ext4/file.c              |   20 +-
 fs/ext4/ialloc.c            |    1 +
 fs/ext4/inode.c             | 1194 +++++++++++++++++++++++++++++++----
 fs/ext4/migrate.c           |    2 +
 fs/ext4/move_extent.c       |   11 +
 fs/ext4/page-io.c           |  201 ++++++
 fs/ext4/super.c             |   38 +-
 include/trace/events/ext4.h |  209 ++++++
 12 files changed, 1715 insertions(+), 164 deletions(-)

-- 
2.52.0

[PATCH v7 01/31] ext4: simplify size updating in ext4_setattr()

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

The logic for updating the file size in ext4_setattr() is currently
somewhat messy. By directly entering the error-handling path after
failing to add an orphan inode, the unnecessary recovery process
involving old_disksize and the file size can be avoided.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 26f0f9714f03c..ddee4c50bd859 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -6080,7 +6080,6 @@ int ext4_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 	if (attr->ia_valid & ATTR_SIZE) {
 		handle_t *handle;
 		loff_t oldsize = inode->i_size;
-		loff_t old_disksize;
 		int shrink = (attr->ia_size < inode->i_size);
 
 		if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))) {
@@ -6164,6 +6163,8 @@ int ext4_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 			if (ext4_handle_valid(handle) && shrink) {
 				error = ext4_orphan_add(handle, inode);
 				orphan = 1;
+				if (error)
+					goto out_handle;
 			}
 
 			if (shrink)
@@ -6179,23 +6180,18 @@ int ext4_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 					(attr->ia_size > 0 ? attr->ia_size - 1 : 0) >>
 					inode->i_sb->s_blocksize_bits);
 
-			down_write(&EXT4_I(inode)->i_data_sem);
-			old_disksize = EXT4_I(inode)->i_disksize;
-			EXT4_I(inode)->i_disksize = attr->ia_size;
-
 			/*
 			 * We have to update i_size under i_data_sem together
 			 * with i_disksize to avoid races with writeback code
-			 * running ext4_wb_update_i_disksize().
+			 * updating disksize in mpage_map_and_submit_extent().
 			 */
-			if (!error)
-				i_size_write(inode, attr->ia_size);
-			else
-				EXT4_I(inode)->i_disksize = old_disksize;
+			down_write(&EXT4_I(inode)->i_data_sem);
+			i_size_write(inode, attr->ia_size);
+			EXT4_I(inode)->i_disksize = attr->ia_size;
 			up_write(&EXT4_I(inode)->i_data_sem);
-			rc = ext4_mark_inode_dirty(handle, inode);
-			if (!error)
-				error = rc;
+
+			error = ext4_mark_inode_dirty(handle, inode);
+out_handle:
 			ext4_journal_stop(handle);
 			if (error)
 				goto out_mmap_sem;
-- 
2.52.0

[PATCH v7 02/31] ext4: factor out ext4_truncate_[up|down]()

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Refactor ext4_setattr() by introducing two helper functions,
ext4_truncate_up() and ext4_truncate_down(), to handle size changes. The
current ATTR_SIZE processing consolidates checks for both shrinking and
non-shrinking cases, leading to cluttered code. Separating the
truncation paths improves readability.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/inode.c | 199 +++++++++++++++++++++++++++---------------------
 1 file changed, 112 insertions(+), 87 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index ddee4c50bd859..d0ccafa554f8d 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -5982,6 +5982,112 @@ static void ext4_wait_for_tail_page_commit(struct inode *inode)
 	}
 }
 
+/*
+ * Set i_size and i_disksize to 'newsize'.
+ *
+ * Both i_rwsem and i_data_sem are required here to avoid races between
+ * generic append writeback and concurrent truncate that also modify
+ * i_size and i_disksize.
+ */
+static inline void ext4_set_inode_size(struct inode *inode, loff_t newsize)
+{
+	WARN_ON_ONCE(S_ISREG(inode->i_mode) && !inode_is_locked(inode));
+
+	down_write(&EXT4_I(inode)->i_data_sem);
+	i_size_write(inode, newsize);
+	EXT4_I(inode)->i_disksize = newsize;
+	up_write(&EXT4_I(inode)->i_data_sem);
+}
+
+static int ext4_truncate_up(struct inode *inode, loff_t oldsize, loff_t newsize)
+{
+	ext4_lblk_t old_lblk, new_lblk;
+	handle_t *handle;
+	int ret;
+
+	if (!IS_ALIGNED(oldsize | newsize, i_blocksize(inode))) {
+		ret = ext4_inode_attach_jinode(inode);
+		if (ret)
+			return ret;
+	}
+
+	inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode));
+	if (!IS_ALIGNED(oldsize, i_blocksize(inode))) {
+		ret = ext4_block_zero_eof(inode, oldsize, LLONG_MAX);
+		if (ret)
+			return ret;
+	}
+
+	handle = ext4_journal_start(inode, EXT4_HT_INODE, 3);
+	if (IS_ERR(handle))
+		return PTR_ERR(handle);
+
+	old_lblk = oldsize > 0 ? (oldsize - 1) >> inode->i_blkbits : 0;
+	new_lblk = newsize > 0 ? (newsize - 1) >> inode->i_blkbits : 0;
+	ext4_fc_track_range(handle, inode, old_lblk, new_lblk);
+
+	ext4_set_inode_size(inode, newsize);
+
+	ret = ext4_mark_inode_dirty(handle, inode);
+	ext4_journal_stop(handle);
+	if (ret)
+		return ret;
+	/*
+	 * isize extend must be called outside an active handle due to
+	 * the lock ordering of transaction start and folio lock in the
+	 * iomap buffered I/O path (folio lock -> transaction start).
+	 */
+	pagecache_isize_extended(inode, oldsize, newsize);
+	return 0;
+}
+
+static int ext4_truncate_down(struct inode *inode, loff_t oldsize,
+			      loff_t newsize, int *orphan)
+{
+	ext4_lblk_t start_lblk;
+	handle_t *handle;
+	int ret;
+
+	/* Do not change i_size. */
+	if (newsize == oldsize)
+		goto truncate;
+
+	/* Shrink. */
+	handle = ext4_journal_start(inode, EXT4_HT_INODE, 3);
+	if (IS_ERR(handle))
+		return PTR_ERR(handle);
+
+	if (ext4_handle_valid(handle)) {
+		ret = ext4_orphan_add(handle, inode);
+		*orphan = 1;
+		if (ret) {
+			ext4_journal_stop(handle);
+			return ret;
+		}
+	}
+
+	start_lblk = newsize > 0 ? (newsize - 1) >> inode->i_blkbits : 0;
+	ext4_fc_track_range(handle, inode, start_lblk, EXT_MAX_BLOCKS - 1);
+
+	ext4_set_inode_size(inode, newsize);
+
+	ret = ext4_mark_inode_dirty(handle, inode);
+	ext4_journal_stop(handle);
+	if (ret)
+		return ret;
+
+	if (ext4_should_journal_data(inode))
+		ext4_wait_for_tail_page_commit(inode);
+truncate:
+	/*
+	 * Truncate pagecache after we've waited for commit in data=journal
+	 * mode to make pages freeable.  Call ext4_truncate() even if
+	 * i_size didn't change to truncate possible preallocated blocks.
+	 */
+	truncate_pagecache(inode, newsize);
+	return ext4_truncate(inode);
+}
+
 /*
  * ext4_setattr()
  *
@@ -6078,7 +6184,6 @@ int ext4_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 	}
 
 	if (attr->ia_valid & ATTR_SIZE) {
-		handle_t *handle;
 		loff_t oldsize = inode->i_size;
 		int shrink = (attr->ia_size < inode->i_size);
 
@@ -6130,94 +6235,14 @@ int ext4_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 			goto err_out;
 		}
 
-		if (attr->ia_size != inode->i_size) {
-			/* attach jbd2 jinode for EOF folio tail zeroing */
-			if (attr->ia_size & (inode->i_sb->s_blocksize - 1) ||
-			    oldsize & (inode->i_sb->s_blocksize - 1)) {
-				error = ext4_inode_attach_jinode(inode);
-				if (error)
-					goto out_mmap_sem;
-			}
-
-			/*
-			 * Update c/mtime and tail zero the EOF folio on
-			 * truncate up. ext4_truncate() handles the shrink case
-			 * below.
-			 */
-			if (!shrink) {
-				inode_set_mtime_to_ts(inode,
-						      inode_set_ctime_current(inode));
-				if (oldsize & (inode->i_sb->s_blocksize - 1)) {
-					error = ext4_block_zero_eof(inode,
-							oldsize, LLONG_MAX);
-					if (error)
-						goto out_mmap_sem;
-				}
-			}
-
-			handle = ext4_journal_start(inode, EXT4_HT_INODE, 3);
-			if (IS_ERR(handle)) {
-				error = PTR_ERR(handle);
-				goto out_mmap_sem;
-			}
-			if (ext4_handle_valid(handle) && shrink) {
-				error = ext4_orphan_add(handle, inode);
-				orphan = 1;
-				if (error)
-					goto out_handle;
-			}
-
-			if (shrink)
-				ext4_fc_track_range(handle, inode,
-					(attr->ia_size > 0 ? attr->ia_size - 1 : 0) >>
-					inode->i_sb->s_blocksize_bits,
-					EXT_MAX_BLOCKS - 1);
-			else
-				ext4_fc_track_range(
-					handle, inode,
-					(oldsize > 0 ? oldsize - 1 : oldsize) >>
-					inode->i_sb->s_blocksize_bits,
-					(attr->ia_size > 0 ? attr->ia_size - 1 : 0) >>
-					inode->i_sb->s_blocksize_bits);
-
-			/*
-			 * We have to update i_size under i_data_sem together
-			 * with i_disksize to avoid races with writeback code
-			 * updating disksize in mpage_map_and_submit_extent().
-			 */
-			down_write(&EXT4_I(inode)->i_data_sem);
-			i_size_write(inode, attr->ia_size);
-			EXT4_I(inode)->i_disksize = attr->ia_size;
-			up_write(&EXT4_I(inode)->i_data_sem);
-
-			error = ext4_mark_inode_dirty(handle, inode);
-out_handle:
-			ext4_journal_stop(handle);
-			if (error)
-				goto out_mmap_sem;
-			if (!shrink) {
-				pagecache_isize_extended(inode, oldsize,
-							 inode->i_size);
-			} else if (ext4_should_journal_data(inode)) {
-				ext4_wait_for_tail_page_commit(inode);
-			}
+		if (attr->ia_size > oldsize)
+			error = ext4_truncate_up(inode, oldsize, attr->ia_size);
+		else {
+			/* Shrink or do not change i_size. */
+			error = ext4_truncate_down(inode, oldsize,
+						   attr->ia_size, &orphan);
 		}
 
-		/*
-		 * Truncate pagecache after we've waited for commit
-		 * in data=journal mode to make pages freeable.
-		 */
-		truncate_pagecache(inode, inode->i_size);
-		/*
-		 * Call ext4_truncate() even if i_size didn't change to
-		 * truncate possible preallocated blocks.
-		 */
-		if (attr->ia_size <= oldsize) {
-			rc = ext4_truncate(inode);
-			if (rc)
-				error = rc;
-		}
-out_mmap_sem:
 		filemap_invalidate_unlock(inode->i_mapping);
 	}
 
-- 
2.52.0

[PATCH v7 03/31] ext4: set EXT4_MAP_NEW flag for delayed allocated blocks

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Set EXT4_MAP_NEW in ext4_da_map_blocks() to properly indicate that a
new delayed allocation block has been inserted, allowing callers to
distinguish newly created delayed extents from existing ones.

Currently, the buffer_head caller, ext4_da_get_block_prep(), does not
consume this flag. It intercepts EXT4_MAP_DELAYED and returns early, and
unconditionally calls set_buffer_new(bh) so EXT4_MAP_NEW is not used.
The flag is prepared for the iomap buffered I/O path added later.

Reported-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Link: https://lore.kernel.org/linux-ext4/cc05c17d-163e-4251-b2c9-aa3a6f9555d7@huaweicloud.com/
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index d0ccafa554f8d..c0f8ff495dd82 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -1990,7 +1990,7 @@ static int ext4_da_map_blocks(struct inode *inode, struct ext4_map_blocks *map)
 		}
 	}
 
-	map->m_flags |= EXT4_MAP_DELAYED;
+	map->m_flags |= EXT4_MAP_DELAYED | EXT4_MAP_NEW;
 	retval = ext4_insert_delayed_blocks(inode, map->m_lblk, map->m_len);
 	if (!retval)
 		map->m_seq = READ_ONCE(EXT4_I(inode)->i_es_seq);
-- 
2.52.0

[PATCH v7 04/31] ext4: recheck extent status tree before block allocation

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

After acquiring i_data_sem in write mode, recheck that the mapping
found via the extent status tree or disk query has not changed. A
racing truncate may have trimmed the extent between the earlier lookup
and the write lock acquisition, since writeback does not hold i_rwsem
or the folio locks covering the full extent. This could cause
ext4_map_create_blocks() to allocate blocks beyond the truncated range,
potentially leading to quota leaks in the upcomming iomap buffered
writeback path, because the iomap writeback infrastructure caches
extents beyond the folio range.

Therefore, if we find a valid extent and the sequence number has
changed, retry the entire lookup to obtain the correct trimmed mapping.

Note that a retry only happens when i_es_seq has actually changed, which
requires another thread to hold i_data_sem in write mode, so each retry
implies real lock contention, and the writer yields on the rwsem
slowpath rather than spinning. Moreover, the read-write lock also tries
its best to guarantee fairness, so even though no retry limit is added
here, livelock should not occur in theory. Also, the retry here mirrors
the existing write_ops->iomap_valid() + IOMAP_F_STALE mechanism in
iomap, which also rechecks the sequence number and retries without any
upper bound, and has not shown livelock problems in practice.

In addition, if ext4_map_query_blocks() fails, return the error
immediately. Otherwise the following sequence number comparison would
use a map->m_seq that may not have been updated, and silently
continuing on an error is not advisable.

Suggested-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/linux-ext4/b0781809-4759-4e12-be17-71555b764f48@gmail.com/
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index c0f8ff495dd82..739f8d3914653 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -734,6 +734,7 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	else
 		ext4_check_map_extents_env(inode);
 
+create_retry:
 	/* Lookup extent status tree firstly */
 	if (ext4_es_lookup_extent(inode, map->m_lblk, NULL, &es, &map->m_seq)) {
 		if (ext4_es_is_written(&es) || ext4_es_is_unwritten(&es)) {
@@ -784,6 +785,8 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	down_read(&EXT4_I(inode)->i_data_sem);
 	retval = ext4_map_query_blocks(handle, inode, map, flags);
 	up_read((&EXT4_I(inode)->i_data_sem));
+	if (retval < 0)
+		return retval;
 
 found:
 	if (retval > 0 && map->m_flags & EXT4_MAP_MAPPED) {
@@ -820,6 +823,21 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	 * with create == 1 flag.
 	 */
 	down_write(&EXT4_I(inode)->i_data_sem);
+
+	/*
+	 * Check the validity of the mapping found via the extent status
+	 * tree or the disk query. A racing truncate may have changed the
+	 * extent, since writeback may not hold i_rwsem or the folio locks
+	 * covering the full extent(e.g., the iomap writeback path may
+	 * allocate an extent that extends beyond the length of the locked
+	 * folio in advance).
+	 */
+	if (map->m_seq != READ_ONCE(EXT4_I(inode)->i_es_seq)) {
+		up_write(&EXT4_I(inode)->i_data_sem);
+		map->m_flags = 0;
+		map->m_len = orig_mlen;
+		goto create_retry;
+	}
 	retval = ext4_map_create_blocks(handle, inode, map, flags);
 	up_write((&EXT4_I(inode)->i_data_sem));
 
-- 
2.52.0

[PATCH v7 05/31] ext4: fix orig_mlen initialization in ext4_map_blocks()

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Save orig_mlen after clamping map->m_len to INT_MAX. Otherwise, the
unclamped value may be passed below, bypassing its overflow protection.

Fixes: 5bb12b1837c0 ("ext4: Add support for EXT4_GET_BLOCKS_QUERY_LEAF_BLOCKS")
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 739f8d3914653..64d223eda0ffe 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -703,7 +703,7 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	struct extent_status es;
 	int retval;
 	int ret = 0;
-	unsigned int orig_mlen = map->m_len;
+	unsigned int orig_mlen;
 #ifdef ES_AGGRESSIVE_TEST
 	struct ext4_map_blocks orig_map;
 
@@ -719,6 +719,7 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	 */
 	if (unlikely(map->m_len > INT_MAX))
 		map->m_len = INT_MAX;
+	orig_mlen = map->m_len;
 
 	/* We can handle the block number less than EXT_MAX_BLOCKS */
 	if (unlikely(map->m_lblk >= EXT_MAX_BLOCKS))
-- 
2.52.0

[PATCH v7 06/31] ext4: allow ext4_map_blocks() to start its own transaction handle

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Make ext4_map_blocks() start its own transaction handle when the caller
does not provide one. The handle is started after the lookup path
confirms that allocation is actually needed, and is stopped at the
unified out exit path. This avoids unnecessarily starting a handle for
pure mapping queries. This prepares for the buffered iomap writeback
conversion, which improves performance for fragile overwrite cases.

Suggested-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/linux-ext4/i536qqwj5eyulec3r2ki2ycnelqdd4bkpat2drn7t72t6p622k@ktueynysgo3j/
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 64d223eda0ffe..553b767148c7b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -686,6 +686,11 @@ int ext4_map_create_blocks(handle_t *handle, struct inode *inode,
  * Otherwise, call with ext4_ind_map_blocks() to handle indirect mapping
  * based files
  *
+ * ext4_map_blocks() may also be called with a NULL @handle. In that case
+ * it starts its own transaction handle only when block allocation is
+ * actually needed, i.e. after the lookup confirms the blocks are not yet
+ * mapped, and stops it before returning.
+ *
  * On success, it returns the number of blocks being mapped or allocated.
  * If flags doesn't contain EXT4_GET_BLOCKS_CREATE the blocks are
  * pre-allocated and unwritten, the resulting @map is marked as unwritten.
@@ -703,6 +708,7 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	struct extent_status es;
 	int retval;
 	int ret = 0;
+	bool internal_handle = false;
 	unsigned int orig_mlen;
 #ifdef ES_AGGRESSIVE_TEST
 	struct ext4_map_blocks orig_map;
@@ -787,13 +793,13 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	retval = ext4_map_query_blocks(handle, inode, map, flags);
 	up_read((&EXT4_I(inode)->i_data_sem));
 	if (retval < 0)
-		return retval;
+		goto out;
 
 found:
 	if (retval > 0 && map->m_flags & EXT4_MAP_MAPPED) {
 		ret = check_block_validity(inode, map);
 		if (ret != 0)
-			return ret;
+			goto out;
 	}
 
 	/* If it is only a block(s) look up */
@@ -813,8 +819,17 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 		 * ext4_ext_map_blocks()
 		 */
 		if (!(flags & EXT4_GET_BLOCKS_CONVERT_UNWRITTEN))
-			return retval;
+			goto out;
 
+	if (!handle) {
+		handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS,
+				ext4_chunk_trans_blocks(inode, orig_mlen));
+		if (IS_ERR(handle)) {
+			ret = PTR_ERR(handle);
+			goto out;
+		}
+		internal_handle = true;
+	}
 
 	ext4_fc_track_inode(handle, inode);
 	/*
@@ -845,12 +860,12 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	if (retval < 0)
 		ext_debug(inode, "failed with err %d\n", retval);
 	if (retval <= 0)
-		return retval;
+		goto out;
 
 	if (map->m_flags & EXT4_MAP_MAPPED) {
 		ret = check_block_validity(inode, map);
 		if (ret != 0)
-			return ret;
+			goto out;
 
 		/*
 		 * Inodes with freshly allocated blocks where contents will be
@@ -872,12 +887,16 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 				ret = ext4_jbd2_inode_add_write(handle, inode,
 						start_byte, length);
 			if (ret)
-				return ret;
+				goto out;
 		}
 	}
 	ext4_fc_track_range(handle, inode, map->m_lblk, map->m_lblk +
 			    map->m_len - 1);
-	return retval;
+out:
+	if (internal_handle)
+		ext4_journal_stop(handle);
+
+	return ret ? ret : retval;
 }
 
 /*
-- 
2.52.0

[PATCH v7 07/31] ext4: avoid unnecessary transaction in ext4_map_blocks() for unwritten extents

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

When ext4_map_blocks() finds an unwritten extent in the extent cache and
the caller is willing to accept unwritten extents without conversion,
there is no need to start a journal transaction since no metadata update
is required. This avoids unnecessary transaction overhead in the
upcoming iomap writeback path when overwriting already-allocated
unwritten extents.

One thing to be careful about, as the comment in ext4_map_blocks()
states, if the flags contain EXT4_GET_BLOCKS_CREATE, the function will
mark @map as mapped.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 553b767148c7b..6b0489cbb1319 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -812,14 +812,25 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 	 * Note that if blocks have been preallocated
 	 * ext4_ext_map_blocks() returns with buffer head unmapped
 	 */
-	if (retval > 0 && map->m_flags & EXT4_MAP_MAPPED)
+	if (retval > 0) {
 		/*
-		 * If we need to convert extent to unwritten
-		 * we continue and do the actual work in
-		 * ext4_ext_map_blocks()
+		 * If we need to convert written extent to unwritten or
+		 * convert unwritten extent to written, continue and do
+		 * the actual work in ext4_ext_map_blocks().
 		 */
-		if (!(flags & EXT4_GET_BLOCKS_CONVERT_UNWRITTEN))
+		if (map->m_flags & EXT4_MAP_MAPPED &&
+		    !(flags & EXT4_GET_BLOCKS_CONVERT_UNWRITTEN))
 			goto out;
+		if (map->m_flags & EXT4_MAP_UNWRITTEN &&
+		    (flags & EXT4_GET_BLOCKS_UNWRIT_EXT) &&
+		    !(flags & EXT4_GET_BLOCKS_CONVERT)) {
+			/* Contains EXT4_GET_BLOCKS_CREATE - mark mapped. */
+			ret = check_block_validity(inode, map);
+			if (!ret)
+				map->m_flags |= EXT4_MAP_MAPPED;
+			goto out;
+		}
+	}
 
 	if (!handle) {
 		handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS,
-- 
2.52.0

[PATCH v7 08/31] ext4: skip block allocation for holes in the data submission path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

When ext4_map_blocks() is called from the data submission path and I/O
end extent conversion path (EXT4_GET_BLOCKS_IO_SUBMIT), it should not
allocate blocks if the lookup returns a hole.

The writeback path can legitimately encounter dirty ranges that map to
holes. For example, when a folio straddles i_size and the tail beyond
i_size is dirtied via a mmap write. Allocating blocks for such ranges is
wrong because there is no data to write back, the dirty bits should
simply be discarded without submitting I/O.

This prepares for the buffered iomap writeback conversion, mirrors the
existing buffer_head writeback path, where mpage_add_bh_to_extent() skip
unmapped buffers and ext4_bio_write_folio() clears their dirty bits.
Since ext4_map_blocks() can now return zero for a hole,
mpage_map_one_extent() must handle it as well. However, because those
holes are already filtered out by mpage_add_bh_to_extent(), it only maps
buffers that are delayed or unwritten, so a hole can never occur in
normal operation. Add a WARN_ON_ONCE() and bail out with -EINVAL rather
than proceeding with an invalid physical block number of zero.

In the ioend extent conversion path, holes are also not expected because
we should wait for folio writeback before punching hole. If one is
encountered, it likely indicates a failure in the concurrency
protection, so ext4_map_blocks() returns zero, and we keep warning and
bail out with -EINVAL to surface the failure rather than continuing
conversion on torn data. Atomic writes in
ext4_convert_unwritten_extents_atomic() already bail out similarly.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 6b0489cbb1319..778cf539f6df2 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -830,6 +830,13 @@ int ext4_map_blocks(handle_t *handle, struct inode *inode,
 				map->m_flags |= EXT4_MAP_MAPPED;
 			goto out;
 		}
+	} else if (retval == 0) {
+		/*
+		 * Do not allocate blocks for holes in the context of
+		 * data submission path.
+		 */
+		if (!map->m_flags && (flags & EXT4_GET_BLOCKS_IO_SUBMIT))
+			goto out;
 	}
 
 	if (!handle) {
@@ -2428,6 +2435,12 @@ static int mpage_map_one_extent(handle_t *handle, struct mpage_da_data *mpd)
 	err = ext4_map_blocks(handle, inode, map, get_blocks_flags);
 	if (err < 0)
 		return err;
+	/*
+	 * A hole? This should never happen since mpage_add_bh_to_extent()
+	 * has filtered it out.
+	 */
+	if (WARN_ON_ONCE(!err))
+		return -EINVAL;
 	if (dioread_nolock && (map->m_flags & EXT4_MAP_UNWRITTEN)) {
 		if (!mpd->io_submit.io_end->handle &&
 		    ext4_handle_valid(handle)) {
-- 
2.52.0

[PATCH v7 09/31] ext4: add iomap address space operations for buffered I/O

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Introduce initial support for iomap in the buffered I/O path for regular
files on ext4.

  - Add a new inode state flag EXT4_STATE_BUFFERED_IOMAP to indicate the
    inode uses iomap instead of buffer_head for buffered I/O
  - Add helper ext4_inode_buffered_iomap() to check the flag
  - Add new address space operations ext4_iomap_aops with callbacks that
    will use generic iomap implementations
  - Add ext4_iomap_aops to ext4_set_aops() when the flag is set

The following callbacks(read_folio(), readahead(), writepages()) are
provided as placeholders and will be implemented in later patches.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/ext4.h  |  7 +++++++
 fs/ext4/inode.c | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 724a27e8be613..24ec205da2d75 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -2049,6 +2049,7 @@ enum {
 	EXT4_STATE_FC_FLUSHING_DATA,	/* Fast commit flushing data */
 	EXT4_STATE_ORPHAN_FILE,		/* Inode orphaned in orphan file */
 	EXT4_STATE_FC_REQUEUE,		/* Inode modified during fast commit */
+	EXT4_STATE_BUFFERED_IOMAP,	/* Inode use iomap for buffered IO */
 };
 
 #define EXT4_INODE_BIT_FNS(name, field, offset)				\
@@ -2148,6 +2149,12 @@ static inline struct mapping_metadata_bhs *ext4_i_metadata_bhs(
 	return READ_ONCE(EXT4_I(inode)->i_metadata_bhs);
 }
 
+/* Whether the inode pass through the iomap infrastructure for buffered I/O */
+static inline bool ext4_inode_buffered_iomap(struct inode *inode)
+{
+	return ext4_test_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP);
+}
+
 /*
  * Codes for operating systems
  */
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 778cf539f6df2..fc2aae64e2fb0 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3969,6 +3969,22 @@ const struct iomap_ops ext4_iomap_report_ops = {
 	.iomap_next = ext4_iomap_next_report,
 };
 
+static int ext4_iomap_read_folio(struct file *file, struct folio *folio)
+{
+	return 0;
+}
+
+static void ext4_iomap_readahead(struct readahead_control *rac)
+{
+
+}
+
+static int ext4_iomap_writepages(struct address_space *mapping,
+				 struct writeback_control *wbc)
+{
+	return 0;
+}
+
 /*
  * For data=journal mode, folio should be marked dirty only when it was
  * writeably mapped. When that happens, it was already attached to the
@@ -4055,6 +4071,20 @@ static const struct address_space_operations ext4_da_aops = {
 	.swap_activate		= ext4_iomap_swap_activate,
 };
 
+static const struct address_space_operations ext4_iomap_aops = {
+	.read_folio		= ext4_iomap_read_folio,
+	.readahead		= ext4_iomap_readahead,
+	.writepages		= ext4_iomap_writepages,
+	.dirty_folio		= iomap_dirty_folio,
+	.bmap			= ext4_bmap,
+	.invalidate_folio	= iomap_invalidate_folio,
+	.release_folio		= iomap_release_folio,
+	.migrate_folio		= filemap_migrate_folio,
+	.is_partially_uptodate  = iomap_is_partially_uptodate,
+	.error_remove_folio	= generic_error_remove_folio,
+	.swap_activate		= ext4_iomap_swap_activate,
+};
+
 static const struct address_space_operations ext4_dax_aops = {
 	.writepages		= ext4_dax_writepages,
 	.dirty_folio		= noop_dirty_folio,
@@ -4076,6 +4106,8 @@ void ext4_set_aops(struct inode *inode)
 	}
 	if (IS_DAX(inode))
 		inode->i_mapping->a_ops = &ext4_dax_aops;
+	else if (ext4_inode_buffered_iomap(inode))
+		inode->i_mapping->a_ops = &ext4_iomap_aops;
 	else if (test_opt(inode->i_sb, DELALLOC))
 		inode->i_mapping->a_ops = &ext4_da_aops;
 	else
-- 
2.52.0

[PATCH v7 10/31] ext4: implement buffered read path using iomap

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Implement the iomap read path for ext4 by introducing a new
ext4_iomap_buffered_read_ops instance. This provides the read_folio()
and readahead() callbacks for ext4_iomap_aops. The implementation
introduces:

 - ext4_iomap_map_blocks(): Helper function to query extent mappings for
   a given read range using ext4_map_blocks() and convert the mapping
   information to iomap type
 - ext4_iomap_buffered_read_begin(): The iomap_begin callbacks that maps
   blocks, validates filesystem state, and populates the iomap. It
   returns -ERANGE for inline data which is not yet supported.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 48 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 47 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index fc2aae64e2fb0..2d6494c5b56c1 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3969,14 +3969,60 @@ const struct iomap_ops ext4_iomap_report_ops = {
 	.iomap_next = ext4_iomap_next_report,
 };
 
+static int ext4_iomap_map_blocks(struct inode *inode, loff_t offset,
+		loff_t length, struct ext4_map_blocks *map)
+{
+	u8 blkbits = inode->i_blkbits;
+
+	if ((offset >> blkbits) > EXT4_MAX_LOGICAL_BLOCK)
+		return -EINVAL;
+
+	/* Calculate the first and last logical blocks respectively. */
+	map->m_lblk = offset >> blkbits;
+	map->m_len = min_t(loff_t, (offset + length - 1) >> blkbits,
+			   EXT4_MAX_LOGICAL_BLOCK) - map->m_lblk + 1;
+
+	return ext4_map_blocks(NULL, inode, map, 0);
+}
+
+static int ext4_iomap_buffered_read_begin(struct inode *inode, loff_t offset,
+		loff_t length, unsigned int flags, struct iomap *iomap,
+		struct iomap *srcmap)
+{
+	struct ext4_map_blocks map;
+	int ret;
+
+	if (unlikely(ext4_forced_shutdown(inode->i_sb)))
+		return -EIO;
+
+	/* Inline data support is not yet available. */
+	if (WARN_ON_ONCE(ext4_has_inline_data(inode)))
+		return -ERANGE;
+
+	ret = ext4_iomap_map_blocks(inode, offset, length, &map);
+	if (ret < 0)
+		return ret;
+
+	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
+	return 0;
+}
+
+static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_buffered_read_next,
+			      ext4_iomap_buffered_read_begin);
+
+static const struct iomap_ops ext4_iomap_buffered_read_ops = {
+	.iomap_next = ext4_iomap_buffered_read_next,
+};
+
 static int ext4_iomap_read_folio(struct file *file, struct folio *folio)
 {
+	iomap_bio_read_folio(folio, &ext4_iomap_buffered_read_ops);
 	return 0;
 }
 
 static void ext4_iomap_readahead(struct readahead_control *rac)
 {
-
+	iomap_bio_readahead(rac, &ext4_iomap_buffered_read_ops);
 }
 
 static int ext4_iomap_writepages(struct address_space *mapping,
-- 
2.52.0

[PATCH v7 11/31] ext4: pass out extent seq counter when mapping da blocks

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

The iomap buffered write path does not hold the folio lock between
mapping the inode extent and copying data. Therefore, it can race
with writeback that modifies the extent type (e.g., from unwritten to
written). This can lead to data corruption on partial writes, as
iomap_block_needs_zeroing() may return a false positive based on a
stale extent. The iomap infrastructure uses the sequence counter
stored in the inode to detect such stale mappings.

Commit 07c440e8da8f ("ext4: pass out extent seq counter when mapping
blocks") added the m_seq field to ext4_map_blocks to pass out extent
sequence numbers, but it missed two callsites within
ext4_da_map_blocks(). These callsites are on the delayed allocation
path, which is needed in the iomap buffered write path. Pass out the
sequence counter to ensure stale mappings can be detected.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 2d6494c5b56c1..3d218fa8c0791 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -1977,7 +1977,7 @@ static int ext4_da_map_blocks(struct inode *inode, struct ext4_map_blocks *map)
 	ext4_check_map_extents_env(inode);
 
 	/* Lookup extent status tree firstly */
-	if (ext4_es_lookup_extent(inode, map->m_lblk, NULL, &es, NULL)) {
+	if (ext4_es_lookup_extent(inode, map->m_lblk, NULL, &es, &map->m_seq)) {
 		map->m_len = min_t(unsigned int, map->m_len,
 				   es.es_len - (map->m_lblk - es.es_lblk));
 
@@ -2030,7 +2030,7 @@ static int ext4_da_map_blocks(struct inode *inode, struct ext4_map_blocks *map)
 	 * is held in write mode, before inserting a new da entry in
 	 * the extent status tree.
 	 */
-	if (ext4_es_lookup_extent(inode, map->m_lblk, NULL, &es, NULL)) {
+	if (ext4_es_lookup_extent(inode, map->m_lblk, NULL, &es, &map->m_seq)) {
 		map->m_len = min_t(unsigned int, map->m_len,
 				   es.es_len - (map->m_lblk - es.es_lblk));
 
-- 
2.52.0

[PATCH v7 12/31] ext4: do not use data=ordered mode for inodes using buffered iomap path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

The data=ordered mode introduces two fundamental conflicts with the
iomap buffered write path, leading to potential deadlocks.

1) Lock ordering conflict
   In the iomap writeback path, each folio is processed sequentially:
   the folio lock is acquired first, followed by starting a transaction
   to create block mappings.

   In data=ordered mode, writeback triggered by the journal commit
   process may attempt to acquire a folio lock that is already held by
   iomap background writeback process. Meanwhile, iomap, under that same
   folio lock, may start a new transaction to map other blocks on this
   folio and wait for the currently committing transaction to finish,
   resulting in a deadlock.

   Trans N commit                    background writeback(via iomap)

   journal_submit_data_buffers()
    ext4_journal_submit_inode_data_buffers()
     iomap_writepages()
                                     iomap_writepages()
                                     folio_lock()
      folio_lock()  -- wait
                                      iomap_writeback_folio()
                                       iomap_writeback_range()
                                        ext4_journal_start()
                                         start new transaction
                                          -- wait for trans N commit,
					     DEADLOCK
                                         ext4_map_blocks()

   Currently, in the buffer_head writeback path, this is handled by
   starting the transaction before taking any folio locks for writeback.

2) Partial folio submission not supported
   When block size < folio size, a folio may contain both mapped and
   unmapped blocks. In data=ordered mode, a deadlock can occur if the
   journal waits (pure JI_WAIT_DATA) for such a folio to be written back
   while background writeback has already started on it (with the
   writeback flag set). The problem is that mapping the remaining
   delalloc blocks can deadlock because the writeback flag is not
   cleared until the entire folio is processed and committed.

   T0: Assume we have a folio contains four blocks, from front to back,
       they are A, B, C, D. The block B and C are holes, and the last
       block D is written in delalloc mode (the block is not allocated
       yet).
   T1: The background writeback process starts to write back data, set
       writeback flag on the folio, allocates block D, and adds it to
       transaction N's order list of jbd2 in pure JI_WAIT_DATA mode.
   T2: This folio completes the writeback and clears the writeback flag.
   T3: Before transaction N commit, we buffered write block A to C.
   T4: Transaction N commit and folio writeback are running
       concurrently.

   Trans N commit                   background writeback(via iomap)

                                    iomap_writeback_folio()
                                     folio_start_writeback()
                                      -- set writeback flag
   jbd2_journal_finish_inode_data_buffers()
    __filemap_fdatawait_range()
     -- wait writeback flag to clear
                                      iomap_writeback_range()
                                       ext4_journal_start()
                                        start new transaction
                                         -- wait for trans N commit,
                                            DEADLOCK
                                       ext4_map_block() (B, C)

   Currently, in the buffer_head writeback path, this is handled by:

   1. Partial folio submission — already-allocated buffers can be
      submitted first. The writeback flag is cleared after I/O
      completes, preventing block allocation while the writeback flag is
      set.
   2. Allocation order — the transaction is started first, then blocks
      are allocated, the writeback flag is set, and finally the
      allocated buffers submission begins.

To support data=ordered mode, the iomap core would need two invasive
changes:
 - Acquire the transaction handle before locking any folio for
   writeback.
 - Support partial folio submission.

Both changes are complicated and risk performance regressions.
Therefore, we must avoid using data=ordered mode when converting to the
iomap path.

Currently, data=ordered mode is used in three scenarios:
 - Append write
 - Post-EOF partial block truncate-up followed by append write
 - Online defragmentation

We can address the first two without data=ordered mode:
 - For append write: always allocate unwritten blocks (i.e. always
   enable dioread_nolock), preserving the behavior of current
   extent-type inodes.
 - For post-EOF truncate-up + append write: postpone updating i_disksize
   until after the zeroed partial block has been written back.

Online defragmentation does not yet support iomap; this can be resolved
separately in the future.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/ext4_jbd2.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h
index 2fbf48b3dfe21..be54e93bde0b1 100644
--- a/fs/ext4/ext4_jbd2.h
+++ b/fs/ext4/ext4_jbd2.h
@@ -379,7 +379,12 @@ static inline int ext4_should_journal_data(struct inode *inode)
 
 static inline int ext4_should_order_data(struct inode *inode)
 {
-	return ext4_inode_journal_mode(inode) & EXT4_INODE_ORDERED_DATA_MODE;
+	/*
+	 * inodes using the iomap buffered I/O path do not use the
+	 * data=ordered mode.
+	 */
+	return !ext4_inode_buffered_iomap(inode) &&
+		(ext4_inode_journal_mode(inode) & EXT4_INODE_ORDERED_DATA_MODE);
 }
 
 static inline int ext4_should_writeback_data(struct inode *inode)
-- 
2.52.0

[PATCH v7 13/31] ext4: implement buffered write path using iomap

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Introduce two new iomap_ops instances for ext4 buffered writes:

 - ext4_iomap_buffered_da_write_ops: for delayed allocation mode, using
   ext4_da_map_blocks() to map delalloc extents.
 - ext4_iomap_buffered_write_ops: for non-delayed allocation mode, using
   ext4_map_blocks() to directly allocate blocks.

Also add ext4_iomap_valid() for the iomap infrastructure to check extent
validity. This is useful for the iomap buffered write path because it
does not hold the folio lock between mapping the inode extent and
copying data. So it can race with writeback, which may modify or stale
the extent type.

Key changes and considerations:

 - Unwritten extents for new blocks (dioread_nolock always on)
   Since data=ordered mode is not used to prevent stale data exposure in
   the non-delayed allocation path, new blocks are always allocated as
   unwritten extents.

 - Short write and write failure handling
   a. Delalloc path: On short write or failure, the stale delalloc range
      must be dropped and its space reservation released. Otherwise, a
      clean folio may cover leftover delalloc extents, causing
      inaccurate space reservation accounting.
   b. Non-delalloc path: No cleanup of allocated blocks is needed on
      short write since leaving the preallocated blocks behind doesn't
      cause any real issue. These blocks are visible to users and are
      not permanently leaked.

 - Adjust the call frequency of ext4_nonda_switch() so that it is no
   longer called once per folio in the buffer_head path, but instead
   once per write. For code simplicity, ext4_iomap_buffered_da_write_ops
   will be shared with the later page-fault path. Since
   ext4_nonda_switch() -> try_to_writeback_inodes_sb() synchronously
   waits for writeback, which could lead to deadlock under the same
   folio lock acquired by iomap_page_mkwrite(), we can no longer call
   ext4_nonda_switch() inside ext4_iomap_buffered_da_write_ops. This may
   increase the probability of a writeback-time allocation failure, but
   the expected real-world impact is small, so it can be implemented
   this way for now.

 - Lock ordering reversal
   The folio lock and transaction start ordering is reversed compared to
   the buffer_head buffered write path. To handle this, the journal
   handle must be stopped in iomap_begin() callbacks. The lock ordering
   documentation in super.c has been updated accordingly.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/ext4.h  |   4 ++
 fs/ext4/file.c  |  20 +++++++-
 fs/ext4/inode.c | 130 ++++++++++++++++++++++++++++++++++++++++++++++--
 fs/ext4/super.c |  10 ++--
 4 files changed, 156 insertions(+), 8 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 24ec205da2d75..98295ef7069ac 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3165,6 +3165,7 @@ int ext4_walk_page_buffers(handle_t *handle,
 int do_journal_get_write_access(handle_t *handle, struct inode *inode,
 				struct buffer_head *bh);
 void ext4_set_inode_mapping_order(struct inode *inode);
+int ext4_nonda_switch(struct super_block *sb);
 #define FALL_BACK_TO_NONDELALLOC 1
 #define EXT4_WRITE_DATA_INLINE	 2
 
@@ -4047,6 +4048,9 @@ static inline void ext4_clear_io_unwritten_flag(ext4_io_end_t *io_end)
 
 extern const struct iomap_ops ext4_iomap_ops;
 extern const struct iomap_ops ext4_iomap_report_ops;
+extern const struct iomap_ops ext4_iomap_buffered_write_ops;
+extern const struct iomap_ops ext4_iomap_buffered_da_write_ops;
+extern const struct iomap_write_ops ext4_iomap_write_ops;
 
 int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
 		unsigned flags, struct iomap *iomap, struct iomap *srcmap);
diff --git a/fs/ext4/file.c b/fs/ext4/file.c
index 374b4bc25bd5d..50d3c92709c8b 100644
--- a/fs/ext4/file.c
+++ b/fs/ext4/file.c
@@ -330,6 +330,21 @@ static ssize_t ext4_write_checks(struct kiocb *iocb, struct iov_iter *from)
 	return count;
 }
 
+static ssize_t ext4_iomap_buffered_write(struct kiocb *iocb,
+					 struct iov_iter *from)
+{
+	struct inode *inode = file_inode(iocb->ki_filp);
+	const struct iomap_ops *iomap_ops;
+
+	if (test_opt(inode->i_sb, DELALLOC) && !ext4_nonda_switch(inode->i_sb))
+		iomap_ops = &ext4_iomap_buffered_da_write_ops;
+	else
+		iomap_ops = &ext4_iomap_buffered_write_ops;
+
+	return iomap_file_buffered_write(iocb, from, iomap_ops,
+					 &ext4_iomap_write_ops, NULL);
+}
+
 static ssize_t ext4_buffered_write_iter(struct kiocb *iocb,
 					struct iov_iter *from)
 {
@@ -351,7 +366,10 @@ static ssize_t ext4_buffered_write_iter(struct kiocb *iocb,
 	if (ret <= 0)
 		goto out;
 
-	ret = generic_perform_write(iocb, from);
+	if (ext4_inode_buffered_iomap(inode))
+		ret = ext4_iomap_buffered_write(iocb, from);
+	else
+		ret = generic_perform_write(iocb, from);
 
 out:
 	inode_unlock(inode);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 3d218fa8c0791..92ed73842d7d9 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3149,7 +3149,7 @@ static int ext4_dax_writepages(struct address_space *mapping,
 	return ret;
 }
 
-static int ext4_nonda_switch(struct super_block *sb)
+int ext4_nonda_switch(struct super_block *sb)
 {
 	s64 free_clusters, dirty_clusters;
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
@@ -3521,6 +3521,15 @@ static bool ext4_inode_datasync_dirty(struct inode *inode)
 	return inode_state_read_once(inode) & I_DIRTY_DATASYNC;
 }
 
+static bool ext4_iomap_valid(struct inode *inode, const struct iomap *iomap)
+{
+	return iomap->validity_cookie == READ_ONCE(EXT4_I(inode)->i_es_seq);
+}
+
+const struct iomap_write_ops ext4_iomap_write_ops = {
+	.iomap_valid = ext4_iomap_valid,
+};
+
 static void ext4_set_iomap(struct inode *inode, struct iomap *iomap,
 			   struct ext4_map_blocks *map, loff_t offset,
 			   loff_t length, unsigned int flags)
@@ -3555,6 +3564,8 @@ static void ext4_set_iomap(struct inode *inode, struct iomap *iomap,
 	    !ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS))
 		iomap->flags |= IOMAP_F_MERGED;
 
+	iomap->validity_cookie = map->m_seq;
+
 	/*
 	 * Flags passed to ext4_map_blocks() for direct I/O writes can result
 	 * in m_flags having both EXT4_MAP_MAPPED and EXT4_MAP_UNWRITTEN bits
@@ -3970,7 +3981,8 @@ const struct iomap_ops ext4_iomap_report_ops = {
 };
 
 static int ext4_iomap_map_blocks(struct inode *inode, loff_t offset,
-		loff_t length, struct ext4_map_blocks *map)
+				 loff_t length, struct ext4_map_blocks *map,
+				 int flags)
 {
 	u8 blkbits = inode->i_blkbits;
 
@@ -3982,7 +3994,10 @@ static int ext4_iomap_map_blocks(struct inode *inode, loff_t offset,
 	map->m_len = min_t(loff_t, (offset + length - 1) >> blkbits,
 			   EXT4_MAX_LOGICAL_BLOCK) - map->m_lblk + 1;
 
-	return ext4_map_blocks(NULL, inode, map, 0);
+	if (flags & EXT4_GET_BLOCKS_DELALLOC_RESERVE)
+		return ext4_da_map_blocks(inode, map);
+
+	return ext4_map_blocks(NULL, inode, map, flags);
 }
 
 static int ext4_iomap_buffered_read_begin(struct inode *inode, loff_t offset,
@@ -3999,7 +4014,7 @@ static int ext4_iomap_buffered_read_begin(struct inode *inode, loff_t offset,
 	if (WARN_ON_ONCE(ext4_has_inline_data(inode)))
 		return -ERANGE;
 
-	ret = ext4_iomap_map_blocks(inode, offset, length, &map);
+	ret = ext4_iomap_map_blocks(inode, offset, length, &map, 0);
 	if (ret < 0)
 		return ret;
 
@@ -4007,6 +4022,113 @@ static int ext4_iomap_buffered_read_begin(struct inode *inode, loff_t offset,
 	return 0;
 }
 
+static int ext4_iomap_buffered_do_write_begin(struct inode *inode,
+		loff_t offset, loff_t length, unsigned int flags,
+		struct iomap *iomap, struct iomap *srcmap, bool delalloc)
+{
+	int ret, retries = 0;
+	struct ext4_map_blocks map;
+	int map_flags;
+
+	ret = ext4_emergency_state(inode->i_sb);
+	if (unlikely(ret))
+		return ret;
+
+	/* Inline data and non-extent are not supported. */
+	if (WARN_ON_ONCE(ext4_has_inline_data(inode)))
+		return -ERANGE;
+	if (WARN_ON_ONCE(!ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)))
+		return -EINVAL;
+	if (WARN_ON_ONCE(!(flags & IOMAP_WRITE)))
+		return -EINVAL;
+
+	map_flags = delalloc ? EXT4_GET_BLOCKS_DELALLOC_RESERVE :
+			       EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT;
+retry:
+	ret = ext4_iomap_map_blocks(inode, offset, length, &map, map_flags);
+	if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries))
+		goto retry;
+	if (ret < 0)
+		return ret;
+
+	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
+	return 0;
+}
+
+static int ext4_iomap_buffered_write_begin(struct inode *inode,
+		loff_t offset, loff_t length, unsigned int flags,
+		struct iomap *iomap, struct iomap *srcmap)
+{
+	return ext4_iomap_buffered_do_write_begin(inode, offset, length, flags,
+						  iomap, srcmap, false);
+}
+
+static int ext4_iomap_buffered_da_write_begin(struct inode *inode,
+		loff_t offset, loff_t length, unsigned int flags,
+		struct iomap *iomap, struct iomap *srcmap)
+{
+	return ext4_iomap_buffered_do_write_begin(inode, offset, length, flags,
+						  iomap, srcmap, true);
+}
+
+/*
+ * On write failure, drop the stale delayed allocation range and release
+ * its reserved space for both start and end blocks. Otherwise, we may
+ * leave a range of delayed extents covered by a clean folio, which can
+ * result in inaccurate space reservation accounting.
+ */
+static void ext4_iomap_punch_delalloc(struct inode *inode, loff_t offset,
+				     loff_t length, struct iomap *iomap)
+{
+	down_write(&EXT4_I(inode)->i_data_sem);
+	ext4_es_remove_extent(inode, offset >> inode->i_blkbits,
+			DIV_ROUND_UP_ULL(length, EXT4_BLOCK_SIZE(inode->i_sb)));
+	up_write(&EXT4_I(inode)->i_data_sem);
+}
+
+static int ext4_iomap_buffered_da_write_end(struct inode *inode, loff_t offset,
+					    loff_t length, ssize_t written,
+					    unsigned int flags,
+					    struct iomap *iomap)
+{
+	loff_t start_byte, end_byte;
+
+	/* If we didn't reserve the blocks, we're not allowed to punch them. */
+	if (iomap->type != IOMAP_DELALLOC || !(iomap->flags & IOMAP_F_NEW))
+		return 0;
+
+	/* Nothing to do if we've written the entire delalloc extent */
+	start_byte = iomap_last_written_block(inode, offset, written);
+	end_byte = round_up(offset + length, i_blocksize(inode));
+	if (start_byte >= end_byte)
+		return 0;
+
+	filemap_invalidate_lock(inode->i_mapping);
+	iomap_write_delalloc_release(inode, start_byte, end_byte, flags,
+				     iomap, ext4_iomap_punch_delalloc);
+	filemap_invalidate_unlock(inode->i_mapping);
+	return 0;
+}
+
+/*
+ * Since we always allocate unwritten extents, there is no need for
+ * iomap_end to clean up allocated blocks on a short write.
+ */
+static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_buffered_write_next,
+			      ext4_iomap_buffered_write_begin);
+
+const struct iomap_ops ext4_iomap_buffered_write_ops = {
+	.iomap_next = ext4_iomap_buffered_write_next,
+};
+
+static DEFINE_IOMAP_ITER_NEXT_END(ext4_iomap_buffered_da_write_next,
+				  ext4_iomap_buffered_da_write_begin,
+				  ext4_iomap_buffered_da_write_end);
+
+const struct iomap_ops ext4_iomap_buffered_da_write_ops = {
+	.iomap_next = ext4_iomap_buffered_da_write_next,
+};
+
 static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_buffered_read_next,
 			      ext4_iomap_buffered_read_begin);
 
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index bca0dc87d0b7c..30150094f2a54 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -104,9 +104,13 @@ static const struct fs_parameter_spec ext4_param_specs[];
  *   -> page lock -> i_data_sem (rw)
  *
  * buffered write path:
- * sb_start_write -> i_mutex -> mmap_lock
- * sb_start_write -> i_mutex -> transaction start -> page lock ->
- *   i_data_sem (rw)
+ * sb_start_write -> i_rwsem (w) -> mmap_lock
+ * - buffer_head path:
+ *   sb_start_write -> i_rwsem (w) -> transaction start -> folio lock ->
+ *     i_data_sem (rw)
+ * - iomap path:
+ *   sb_start_write -> i_rwsem (w) -> transaction start -> i_data_sem (rw)
+ *   sb_start_write -> i_rwsem (w) -> folio lock (not under an active handle)
  *
  * truncate:
  * sb_start_write -> i_mutex -> invalidate_lock (w) -> i_mmap_rwsem (w) ->
-- 
2.52.0

[PATCH v7 14/31] ext4: rework handle credit accounting for unwritten extent conversion

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

While converting the buffered I/O path to iomap, we noticed that the
writeback completion can hand over an ioend which was merged across an
arbitrary number of extents because iomap_ioend_try_merge() doesn't
limit the length of the merged ioends, while
ext4_convert_unwritten_extents() converts all of them with the handle
provided by the caller, and callers calculate the credits of that handle
without knowing the exact number of extents. Converting more extents
than that within one transaction can exhaust its credits.

Prepare for the iomap path nevertheless by making the conversion loop
check the remaining credits before each extent with
ext4_journal_ensure_credits(), so that the transaction is extended, or
restarted once it is full, as needed. Also estimate the credits with
ext4_meta_trans_blocks(..., alloc_extents = 0), since no data blocks are
allocated here, and manage an internally started transaction as a whole
instead of starting and stopping one for each extent.

The buffer_head writeback path is safe, as it passes a handle whose
credits are reserved by ext4_meta_trans_blocks(inode, bpf, bpf, 0),
which is enough to convert one extent per block of a folio, so the
reservation is sufficient and the handle is never restarted there.

This also helps the fallocate path, where ext4_alloc_file_blocks() sizes
the handle for the conversion with ext4_chunk_trans_blocks() as well,
i.e. for a single extent only, while a concurrent modification of the
extent tree can make the conversion cover more extents than the handle
has credits for.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/extents.c | 53 ++++++++++++++++++++++++++++-------------------
 1 file changed, 32 insertions(+), 21 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 76038b6c36552..83d040e36b682 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -5137,27 +5137,36 @@ int ext4_convert_unwritten_extents(handle_t *handle, struct inode *inode,
 	int ret = 0, ret2 = 0, ret3 = 0;
 	struct ext4_map_blocks map;
 	unsigned int blkbits = inode->i_blkbits;
-	unsigned int credits = 0;
+	unsigned int credits;
+	bool internal_handle = !handle;
 
 	map.m_lblk = offset >> blkbits;
 	map.m_len = max_blocks = EXT4_MAX_BLOCKS(len, offset, blkbits);
 
-	if (!handle) {
-		/*
-		 * credits to insert 1 extent into extent tree
-		 */
-		credits = ext4_chunk_trans_blocks(inode, max_blocks);
+	/* Credits to convert one extent in this range to written state. */
+	credits = ext4_meta_trans_blocks(inode, max_blocks, 1, 0);
+
+	if (internal_handle) {
+		handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS, credits);
+		if (IS_ERR(handle)) {
+			ret = PTR_ERR(handle);
+			goto out;
+		}
 	}
 
 	while (max_blocks) {
-		if (credits) {
-			handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS,
-						    credits);
-			if (IS_ERR(handle)) {
-				ret = PTR_ERR(handle);
-				break;
-			}
-		}
+		/*
+		 * The caller cannot know how many extents the range covers,
+		 * so make sure the transaction can take one more extent,
+		 * extending or restarting it once it is full.  The reserved
+		 * handle of the buffer_head writeback path is safe as it has
+		 * enough credits for the conversion, so it is not restarted
+		 * here.
+		 */
+		ret = ext4_journal_ensure_credits(handle, credits, 0);
+		if (ret < 0)
+			break;
+
 		/*
 		 * Do not cache any unrelated extents, as it does not hold the
 		 * i_rwsem or invalidate_lock, which could corrupt the extent
@@ -5177,11 +5186,6 @@ int ext4_convert_unwritten_extents(handle_t *handle, struct inode *inode,
 		}
 
 		ret2 = ext4_mark_inode_dirty(handle, inode);
-		if (credits) {
-			ret3 = ext4_journal_stop(handle);
-			if (unlikely(ret3))
-				ret2 = ret3;
-		}
 		ret = ret < 0 ? ret : ret2;
 		if (ret)
 			break;
@@ -5189,6 +5193,13 @@ int ext4_convert_unwritten_extents(handle_t *handle, struct inode *inode,
 		map.m_lblk += map.m_len;
 		map.m_len = (max_blocks -= map.m_len);
 	}
+
+	if (internal_handle) {
+		ret3 = ext4_journal_stop(handle);
+		if (!ret)
+			ret = ret3;
+	}
+out:
 	/* Converted some or all blocks successfully? */
 	if (converted)
 		*converted = conv_blocks;
@@ -5203,8 +5214,8 @@ int ext4_convert_unwritten_io_end_vec(handle_t *handle, ext4_io_end_t *io_end)
 
 	/*
 	 * This is somewhat ugly but the idea is clear: When transaction is
-	 * reserved, everything goes into it. Otherwise we rather start several
-	 * smaller transactions for conversion of each extent separately.
+	 * reserved, everything goes into it. Otherwise the conversion runs
+	 * its own transaction, extending or restarting it as it goes.
 	 */
 	if (handle) {
 		handle = ext4_journal_start_reserved(handle,
-- 
2.52.0

[PATCH v7 15/31] ext4: implement writeback path using iomap

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Add the iomap writeback path for ext4 buffered I/O. This introduces:

 - ext4_iomap_writepages(): the main writeback entry point.
 - ext4_writeback_ops: a new iomap_writeback_ops instance to handle
   block mapping and I/O submission.
 - A new end I/O work handler ext4_iomap_finish_ioend() is added for
   converting unwritten extents, updating file size, and handling
   DATA_ERR_ABORT after I/O completion.

Core implementation details:

 - ->writeback_range() callback
   Calls ext4_iomap_map_writeback_range() to map and allocate blocks
   using the enhanced ext4_map_blocks(). ext4_map_blocks() now starts
   its own transaction internally when it needs to allocate blocks.
   For performance, when a block range is not yet allocated, it
   allocates based on the writeback length and delalloc extent length
   (ext4_map_blocks() will trim the allocated length), rather than
   allocating a single folio at a time. The folio is then added to
   an iomap_ioend instance.

 - ->writeback_submit() callback
   Registers ext4_iomap_end_bio() as the end bio callback. This callback
   schedules a worker to handle:
   - Unwritten extent conversion.
   - i_disksize update after data is written back.
   - Journal abort on writeback I/O failure.

 - The new end I/O work handler ext4_iomap_finish_ioend()
   This reuses i_rsv_conversion_work and i_rsv_conversion_list, which
   are now used in the buffer_head writeback path for memory saving,
   because an inode can never go through both buffer_head and iomap
   paths. Those are not initialized in this patch but will be when the
   iomap path is formally enabled later.

Key changes and considerations:

 - Append write and unwritten extents
   Since data=ordered mode is not used to prevent stale data exposure
   during append writebacks, new blocks are always allocated as
   unwritten extents (i.e. always enable dioread_nolock), and i_disksize
   update is postponed until I/O completion. Additionally, the deadlock
   that the reserve handle was expected to resolve does not occur
   anymore. Therefore, the end I/O worker can start a normal journal
   handle instead of a reserve handle when converting unwritten extents.

 - Lock ordering
   The ->writeback_range() callback runs under the folio lock, requiring
   the journal handle to be started under that same lock. This reverses
   the order compared to the buffer_head writeback path. The lock
   ordering documentation in super.c has been updated accordingly.

 - Don't cache writes
   The iomap infrastructure sets the BIO_COMPLETE_IN_TASK flag when
   submitting I/O, so the ioend will be processed in task context.
   However, if a private defer worker is to be started, this flag must
   be cleared explicitly to avoid double deferral. In the future, all
   private defer work should be moved to the generic bio complete in
   task framework.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h    |   8 ++-
 fs/ext4/inode.c   | 147 +++++++++++++++++++++++++++++++++++++++++++++-
 fs/ext4/page-io.c | 122 ++++++++++++++++++++++++++++++++++++++
 fs/ext4/super.c   |   5 +-
 4 files changed, 278 insertions(+), 4 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 98295ef7069ac..03fa90d2986ff 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -1208,8 +1208,10 @@ struct ext4_inode_info {
 	/* Lock protecting lists below */
 	spinlock_t i_completed_io_lock;
 	/*
-	 * Completed IOs that need unwritten extents handling and have
-	 * transaction reserved
+	 * Completed IOs that need unwritten extents handling and have a
+	 * transaction reserved for the buffer_head writeback path, and
+	 * also used by the iomap writeback path to queue ioends needing
+	 * unwritten extents conversion, i_disksize update, etc.
 	 */
 	struct list_head i_rsv_conversion_list;
 	struct work_struct i_rsv_conversion_work;
@@ -3991,6 +3993,8 @@ void ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *page,
 		size_t len);
 extern struct ext4_io_end_vec *ext4_alloc_io_end_vec(ext4_io_end_t *io_end);
 extern struct ext4_io_end_vec *ext4_last_io_end_vec(ext4_io_end_t *io_end);
+extern void ext4_iomap_end_io(struct work_struct *work);
+extern void ext4_iomap_end_bio(struct bio *bio);
 
 /* mmp.c */
 extern int ext4_multi_mount_protect(struct super_block *, ext4_fsblk_t);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 92ed73842d7d9..78be37cd3fa2b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -44,6 +44,7 @@
 #include <linux/iversion.h>
 
 #include "ext4_jbd2.h"
+#include "ext4_extents.h"
 #include "xattr.h"
 #include "acl.h"
 #include "truncate.h"
@@ -4147,10 +4148,154 @@ static void ext4_iomap_readahead(struct readahead_control *rac)
 	iomap_bio_readahead(rac, &ext4_iomap_buffered_read_ops);
 }
 
+
+static int ext4_iomap_map_writeback_range(struct iomap_writepage_ctx *wpc,
+					  loff_t offset, unsigned int dirty_len)
+{
+	struct inode *inode = wpc->inode;
+	struct super_block *sb = inode->i_sb;
+	struct journal_s *journal = EXT4_SB(sb)->s_journal;
+	struct ext4_map_blocks map;
+	ext4_lblk_t index = offset >> inode->i_blkbits;
+	unsigned int blk_len, blk_end;
+	int retries = 0;
+	int ret;
+
+	ret = ext4_emergency_state(sb);
+	if (unlikely(ret))
+		return ret;
+
+	/* Check validity of the cached writeback mapping. */
+	if (offset >= wpc->iomap.offset &&
+	    offset < wpc->iomap.offset + wpc->iomap.length &&
+	    ext4_iomap_valid(inode, &wpc->iomap))
+		return 0;
+
+	blk_len = dirty_len >> inode->i_blkbits;
+	blk_end = umin(wpc->wbc->range_end >> inode->i_blkbits, UINT_MAX - 1);
+	if (blk_end > index + blk_len)
+		blk_len = blk_end - index + 1;
+
+retry:
+	map.m_lblk = index;
+	map.m_len = min_t(unsigned int, MAX_WRITEPAGES_EXTENT_LEN, blk_len);
+	ret = ext4_map_blocks(NULL, inode, &map,
+			      EXT4_GET_BLOCKS_CREATE_UNWRIT_EXT |
+			      EXT4_GET_BLOCKS_METADATA_NOFAIL |
+			      EXT4_GET_BLOCKS_IO_SUBMIT |
+			      EXT4_EX_NOCACHE);
+	if (ret < 0) {
+		if (ext4_emergency_state(sb))
+			return ret;
+
+		/*
+		 * Retry transient ENOSPC errors, if
+		 * ext4_count_free_blocks() is non-zero, a commit
+		 * should free up blocks.
+		 */
+		if (ret == -ENOSPC && journal && ext4_count_free_clusters(sb) &&
+		    ++retries <= 3) {
+			jbd2_journal_force_commit_nested(journal);
+			goto retry;
+		}
+
+		ext4_msg(sb, KERN_CRIT,
+			 "Delayed block allocation failed for inode %llu at logical offset %llu with max blocks %u with error %d",
+			 inode->i_ino, (unsigned long long)map.m_lblk,
+			 (unsigned int)map.m_len, -ret);
+		ext4_msg(sb, KERN_CRIT,
+			 "This should not happen!! Data will be lost\n");
+		if (ret == -ENOSPC)
+			ext4_print_free_blocks(inode);
+		return ret;
+	}
+
+	ext4_set_iomap(inode, &wpc->iomap, &map, offset, dirty_len, 0);
+	return 0;
+}
+
+static void ext4_iomap_discard_folio(struct folio *folio, loff_t pos)
+{
+	struct inode *inode = folio->mapping->host;
+	loff_t length = folio_pos(folio) + folio_size(folio) - pos;
+
+	ext4_iomap_punch_delalloc(inode, pos, length, NULL);
+}
+
+static ssize_t ext4_iomap_writeback_range(struct iomap_writepage_ctx *wpc,
+					  struct folio *folio, u64 offset,
+					  unsigned int len, u64 end_pos)
+{
+	ssize_t ret;
+
+	ret = ext4_iomap_map_writeback_range(wpc, offset, len);
+	if (!ret)
+		ret = iomap_add_to_ioend(wpc, folio, offset, end_pos, len);
+	if (ret < 0)
+		ext4_iomap_discard_folio(folio, offset);
+	return ret;
+}
+
+static int ext4_iomap_writeback_submit(struct iomap_writepage_ctx *wpc,
+				       int error)
+{
+	struct iomap_ioend *ioend = wpc->wb_ctx;
+	struct ext4_inode_info *ei = EXT4_I(ioend->io_inode);
+
+	/*
+	 * After I/O completion, a worker needs to be scheduled when:
+	 * 1) Unwritten extents require conversion.
+	 * 2) The file size needs to be extended.
+	 * 3) The journal needs to be aborted due to an I/O error.
+	 */
+	if ((ioend->io_flags & IOMAP_IOEND_UNWRITTEN) ||
+	    (ioend->io_offset + ioend->io_size > READ_ONCE(ei->i_disksize)) ||
+	    test_opt(ioend->io_inode->i_sb, DATA_ERR_ABORT))
+		ioend->io_bio.bi_end_io = ext4_iomap_end_bio;
+
+	/*
+	 * ext4_iomap_end_bio() always defers endio processing, disable
+	 * generic BIO in task to avoid double deferral since we will use
+	 * a private defer endio handler in process context.
+	 *
+	 * TODO: Switch all defer handlers to the generic bio complete
+	 * in task framework.
+	 */
+	if (ioend->io_bio.bi_end_io)
+		bio_clear_flag(&ioend->io_bio, BIO_COMPLETE_IN_TASK);
+
+	return iomap_ioend_writeback_submit(wpc, error);
+}
+
+static const struct iomap_writeback_ops ext4_writeback_ops = {
+	.writeback_range = ext4_iomap_writeback_range,
+	.writeback_submit = ext4_iomap_writeback_submit,
+};
+
 static int ext4_iomap_writepages(struct address_space *mapping,
 				 struct writeback_control *wbc)
 {
-	return 0;
+	struct inode *inode = mapping->host;
+	struct super_block *sb = inode->i_sb;
+	long nr = wbc->nr_to_write;
+	int alloc_ctx, ret;
+	struct iomap_writepage_ctx wpc = {
+		.inode = inode,
+		.wbc = wbc,
+		.ops = &ext4_writeback_ops,
+	};
+
+	ret = ext4_emergency_state(sb);
+	if (unlikely(ret))
+		return ret;
+
+	alloc_ctx = ext4_writepages_down_read(sb);
+	trace_ext4_writepages(inode, wbc);
+	ret = iomap_writepages(&wpc);
+	trace_ext4_writepages_result(inode, wbc, ret, nr - wbc->nr_to_write);
+	ext4_writepages_up_read(sb, alloc_ctx);
+
+	return ret;
 }
 
 /*
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 0236b6b9785a2..9b0e12b5463cf 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -22,6 +22,7 @@
 #include <linux/bio.h>
 #include <linux/workqueue.h>
 #include <linux/kernel.h>
+#include <linux/iomap.h>
 #include <linux/slab.h>
 #include <linux/mm.h>
 #include <linux/sched/mm.h>
@@ -547,3 +548,124 @@ void ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *folio,
 		io_submit_add_bh(io, inode, folio, bh);
 	} while ((bh = bh->b_this_page) != head);
 }
+
+static int ext4_iomap_wb_update_disksize(handle_t *handle, struct inode *inode,
+					 loff_t end)
+{
+	loff_t new_disksize = end;
+	struct ext4_inode_info *ei = EXT4_I(inode);
+	int ret;
+
+	/*
+	 * Races with truncate are avoided by checking i_size under
+	 * i_data_sem.
+	 */
+	down_write(&ei->i_data_sem);
+	new_disksize = min(new_disksize, i_size_read(inode));
+	if (new_disksize > ei->i_disksize)
+		WRITE_ONCE(ei->i_disksize, new_disksize);
+	up_write(&ei->i_data_sem);
+	ret = ext4_mark_inode_dirty(handle, inode);
+	if (ret)
+		EXT4_ERROR_INODE_ERR(inode, -ret, "Failed to mark inode dirty");
+
+	return ret;
+}
+
+static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
+{
+	struct inode *inode = ioend->io_inode;
+	struct super_block *sb = inode->i_sb;
+	loff_t pos = ioend->io_offset;
+	size_t size = ioend->io_size;
+	loff_t end = pos + size;
+	handle_t *handle;
+	int credits;
+	int ret, err;
+
+	ret = blk_status_to_errno(ioend->io_bio.bi_status);
+	if (unlikely(ret)) {
+		if (test_opt(sb, DATA_ERR_ABORT) && !ext4_emergency_state(sb))
+			jbd2_journal_abort(EXT4_SB(sb)->s_journal, ret);
+		goto out;
+	}
+
+	if (!(ioend->io_flags & IOMAP_IOEND_UNWRITTEN) &&
+	    end <= READ_ONCE(EXT4_I(inode)->i_disksize))
+		goto out;
+
+	/*
+	 * We may need to convert one extent, update the i_disksize and
+	 * dirty the inode.
+	 */
+	credits = ext4_chunk_trans_blocks(inode,
+			EXT4_MAX_BLOCKS(size, pos, inode->i_blkbits));
+	handle = ext4_journal_start(inode, EXT4_HT_EXT_CONVERT, credits);
+	if (IS_ERR(handle)) {
+		ret = PTR_ERR(handle);
+		goto out_err;
+	}
+
+	/* Update on-disk size after I/O is completed. */
+	if (end > READ_ONCE(EXT4_I(inode)->i_disksize)) {
+		ret = ext4_iomap_wb_update_disksize(handle, inode, end);
+		if (ret)
+			goto out_journal;
+	}
+
+	if (ioend->io_flags & IOMAP_IOEND_UNWRITTEN)
+		ret = ext4_convert_unwritten_extents(handle, inode, pos,
+						     size, NULL);
+
+out_journal:
+	err = ext4_journal_stop(handle);
+	if (!ret)
+		ret = err;
+out_err:
+	if (ret < 0 && !ext4_emergency_state(sb)) {
+		ext4_msg(sb, KERN_EMERG,
+			 "failed to convert unwritten extents to written extents or update inode size -- potential data loss! (inode %llu, error %d)",
+			 inode->i_ino, ret);
+	}
+out:
+	iomap_finish_ioends(ioend, ret);
+}
+
+/*
+ * Work on buffered iomap completed IO, to convert unwritten extents to
+ * mapped extents
+ */
+void ext4_iomap_end_io(struct work_struct *work)
+{
+	struct ext4_inode_info *ei = container_of(work, struct ext4_inode_info,
+						  i_rsv_conversion_work);
+	struct iomap_ioend *ioend;
+	struct list_head ioend_list;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ei->i_completed_io_lock, flags);
+	list_replace_init(&ei->i_rsv_conversion_list, &ioend_list);
+	spin_unlock_irqrestore(&ei->i_completed_io_lock, flags);
+
+	iomap_sort_ioends(&ioend_list);
+	while (!list_empty(&ioend_list)) {
+		ioend = list_entry(ioend_list.next, struct iomap_ioend, io_list);
+		list_del_init(&ioend->io_list);
+		iomap_ioend_try_merge(ioend, &ioend_list);
+		ext4_iomap_finish_ioend(ioend);
+	}
+}
+
+void ext4_iomap_end_bio(struct bio *bio)
+{
+	struct iomap_ioend *ioend = iomap_ioend_from_bio(bio);
+	struct ext4_inode_info *ei = EXT4_I(ioend->io_inode);
+	unsigned long flags;
+
+	spin_lock_irqsave(&ei->i_completed_io_lock, flags);
+	if (list_empty(&ei->i_rsv_conversion_list))
+		queue_work(EXT4_SB(ioend->io_inode->i_sb)->rsv_conversion_wq,
+			   &ei->i_rsv_conversion_work);
+	list_add_tail(&ioend->io_list, &ei->i_rsv_conversion_list);
+	spin_unlock_irqrestore(&ei->i_completed_io_lock, flags);
+}
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 30150094f2a54..6d2d323604f96 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -123,7 +123,10 @@ static const struct fs_parameter_spec ext4_param_specs[];
  * sb_start_write -> i_mutex -> transaction start -> i_data_sem (rw)
  *
  * writepages:
- * transaction start -> page lock(s) -> i_data_sem (rw)
+ * - buffer_head path:
+ *   transaction start -> folio lock(s) -> i_data_sem (rw)
+ * - iomap path:
+ *   folio lock -> transaction start -> i_data_sem (rw)
  */
 
 static const struct fs_context_operations ext4_context_ops = {
-- 
2.52.0

[PATCH v7 16/31] ext4: implement mmap path using iomap

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Introduce ext4_iomap_page_mkwrite() to implement the mmap iomap path
for ext4. The heavy lifting is delegated to iomap_page_mkwrite(), which
only requires ext4_iomap_buffered_write_ops and
ext4_iomap_buffered_da_write_ops to allocate and map blocks.

Note that the lock ordering between folio lock and transaction start in
this path is reversed compared to the buffer_head buffered write path.
The lock ordering documentation in super.c has been updated accordingly.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/inode.c | 32 +++++++++++++++++++++++++++++++-
 fs/ext4/super.c |  8 ++++++--
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 78be37cd3fa2b..f8721fd6eb22b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4040,7 +4040,7 @@ static int ext4_iomap_buffered_do_write_begin(struct inode *inode,
 		return -ERANGE;
 	if (WARN_ON_ONCE(!ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)))
 		return -EINVAL;
-	if (WARN_ON_ONCE(!(flags & IOMAP_WRITE)))
+	if (WARN_ON_ONCE(!(flags & (IOMAP_WRITE | IOMAP_FAULT))))
 		return -EINVAL;
 
 	map_flags = delalloc ? EXT4_GET_BLOCKS_DELALLOC_RESERVE :
@@ -4098,6 +4098,14 @@ static int ext4_iomap_buffered_da_write_end(struct inode *inode, loff_t offset,
 	if (iomap->type != IOMAP_DELALLOC || !(iomap->flags & IOMAP_F_NEW))
 		return 0;
 
+	/*
+	 * iomap_page_mkwrite() will never fail in a way that requires delalloc
+	 * extents that it allocated to be revoked.  Hence never try to release
+	 * them here.
+	 */
+	if (flags & IOMAP_FAULT)
+		return 0;
+
 	/* Nothing to do if we've written the entire delalloc extent */
 	start_byte = iomap_last_written_block(inode, offset, written);
 	end_byte = round_up(offset + length, i_blocksize(inode));
@@ -7309,6 +7317,23 @@ static int ext4_block_page_mkwrite(struct inode *inode, struct folio *folio,
 	return ret;
 }
 
+static vm_fault_t ext4_iomap_page_mkwrite(struct vm_fault *vmf)
+{
+	struct inode *inode = file_inode(vmf->vma->vm_file);
+	const struct iomap_ops *iomap_ops;
+
+	/*
+	 * ext4_nonda_switch() could writeback this folio, so have to
+	 * call it before lock folio.
+	 */
+	if (test_opt(inode->i_sb, DELALLOC) && !ext4_nonda_switch(inode->i_sb))
+		iomap_ops = &ext4_iomap_buffered_da_write_ops;
+	else
+		iomap_ops = &ext4_iomap_buffered_write_ops;
+
+	return iomap_page_mkwrite(vmf, iomap_ops, NULL);
+}
+
 vm_fault_t ext4_page_mkwrite(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
@@ -7335,6 +7360,11 @@ vm_fault_t ext4_page_mkwrite(struct vm_fault *vmf)
 	if (err)
 		goto out_ret;
 
+	if (ext4_inode_buffered_iomap(inode)) {
+		ret = ext4_iomap_page_mkwrite(vmf);
+		goto out;
+	}
+
 	/*
 	 * On data journalling we skip straight to the transaction handle:
 	 * there's no delalloc; page truncated will be checked later; the
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 6d2d323604f96..1c2395aa1d53a 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -100,8 +100,12 @@ static const struct fs_parameter_spec ext4_param_specs[];
  * Lock ordering
  *
  * page fault path:
- * mmap_lock -> sb_start_pagefault -> invalidate_lock (r) -> transaction start
- *   -> page lock -> i_data_sem (rw)
+ * - buffer_head path:
+ *   mmap_lock -> sb_start_pagefault -> invalidate_lock (r) ->
+ *     transaction start -> folio lock -> i_data_sem (rw)
+ * - iomap path:
+ *   mmap_lock -> sb_start_pagefault -> invalidate_lock (r) ->
+ *     folio lock -> transaction start -> i_data_sem (rw)
  *
  * buffered write path:
  * sb_start_write -> i_rwsem (w) -> mmap_lock
-- 
2.52.0

[PATCH v7 17/31] ext4: implement partial block zero range path using iomap

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Introduce a new iomap_ops instance, ext4_iomap_zero_ops, along with
ext4_block_iomap_zero_range() to implement block zeroing via the iomap
infrastructure for ext4.

ext4_block_iomap_zero_range() calls iomap_zero_range() with
ext4_iomap_zero_begin() as the callback. The callback locates the
range and populates the iomap mapping. If the range is mapped,
iomap_zero_iter() in the iomap core zeros the partial block
directly. If the range is an unwritten extent within EOF, the
callback collects a dirty folio batch via iomap_fill_dirty_folios()
so that iomap_zero_iter() can zero those folios directly, bypassing
a separate slow flush operation that would otherwise be needed to
convert the unwritten extent.

Note that ext4_iomap_zero_begin() can race with concurrent writeback:
after it queries an unwritten extent, writeback may convert it to
written and complete on the folio before iomap_fill_dirty_folios() scans
the range. The empty batch then makes iomap_zero_iter() skip zeroing,
leaving stale on-disk data.

  zero_range                      writeback
  ----------------------          ----------------------
  ext4_block_zero_range()
    iomap_zero_range()
      iomap_iter()
        ext4_iomap_zero_begin()
          ext4_iomap_map_blocks()
            -> extent is UNWRITTEN
                                  ext4_convert_unwritten_extents_endio()
                                    -> extent is converted to WRITTEN
                                    -> folio is clean
          iomap_fill_dirty_folios()
            filemap_get_folios_dirty()
            -> folio is clean, not added to batch
          ext4_set_iomap() -> IOMAP_UNWRITTEN
      iomap_zero_iter()
        __iomap_get_folio() -> NULL (empty batch)
        iomap_iter_advance_full()  <-- zeroing skipped

  [later read returns stale on-disk data]   <--   CORRUPTION

Therefore, we retry the extent lookup when iomap_fill_dirty_folios()
adds nothing and the i_es_seq cookie captured at the first lookup has
advanced, indicating the race actually occurred.

Other important constraints:

Zeroing out under an active journal handle can cause deadlock, as the
lock/handle ordering is inconsistent with the iomap writeback path.
Therefore, ext4_block_iomap_zero_range() must not be called under an
active handle. In addition, for post-EOF zeroing, the caller cannot
rely on data=ordered mode to persist the zeroed data before
i_disksize is updated.

Subsequent patches will address this by deferring i_disksize update
to i_size until after the zeroed data has been written back.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index f8721fd6eb22b..35a2c96535c03 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4119,6 +4119,67 @@ static int ext4_iomap_buffered_da_write_end(struct inode *inode, loff_t offset,
 	return 0;
 }
 
+static int ext4_iomap_zero_begin(struct inode *inode,
+		loff_t offset, loff_t length, unsigned int flags,
+		struct iomap *iomap, struct iomap *srcmap)
+{
+	struct iomap_iter *iter = container_of(iomap, struct iomap_iter, iomap);
+	struct ext4_map_blocks map;
+	u8 blkbits = inode->i_blkbits;
+	unsigned int iomap_flags;
+	int ret;
+
+	ret = ext4_emergency_state(inode->i_sb);
+	if (unlikely(ret))
+		return ret;
+
+	if (WARN_ON_ONCE(!(flags & IOMAP_ZERO)))
+		return -EINVAL;
+
+again:
+	ret = ext4_iomap_map_blocks(inode, offset, length, &map, 0);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * Look up dirty folios for unwritten mappings within EOF. Providing
+	 * this bypasses the flush iomap uses to trigger extent conversion
+	 * when unwritten mappings have dirty pagecache in need of zeroing.
+	 */
+	iomap_flags = 0;
+	if (map.m_flags & EXT4_MAP_UNWRITTEN) {
+		loff_t start = ((loff_t)map.m_lblk) << blkbits;
+		loff_t end = ((loff_t)map.m_lblk + map.m_len) << blkbits;
+		unsigned int count;
+
+		count = iomap_fill_dirty_folios(iter, &start, end,
+						&iomap_flags);
+		if ((start >> blkbits) < map.m_lblk + map.m_len)
+			map.m_len = (start >> blkbits) - map.m_lblk;
+
+		/*
+		 * This can be raced by a concurrent writeback that cleans
+		 * the folio and converts the unwritten extent to written.
+		 * Recheck the mapping after a folio lock round in
+		 * iomap_fill_dirty_folios().
+		 */
+		if (count == 0 &&
+		    map.m_seq != READ_ONCE(EXT4_I(inode)->i_es_seq))
+			goto again;
+	}
+
+	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
+	iomap->flags |= iomap_flags;
+
+	return 0;
+}
+
+static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_zero_next, ext4_iomap_zero_begin);
+
+static const struct iomap_ops ext4_iomap_zero_ops = {
+	.iomap_next = ext4_iomap_zero_next,
+};
+
 /*
  * Since we always allocate unwritten extents, there is no need for
  * iomap_end to clean up allocated blocks on a short write.
@@ -4582,6 +4643,48 @@ static int ext4_block_journalled_zero_range(struct inode *inode, loff_t from,
 	return err;
 }
 
+static int ext4_block_iomap_zero_range(struct inode *inode, loff_t from,
+				       loff_t length, bool *did_zero,
+				       bool *zero_written)
+{
+	int ret;
+
+	/*
+	 * Zeroing out under an active handle can cause deadlock since
+	 * the order of acquiring the folio lock and starting a handle is
+	 * inconsistent with the iomap writeback procedure.
+	 */
+	if (WARN_ON_ONCE(ext4_handle_valid(journal_current_handle())))
+		return -EINVAL;
+
+	/* The zeroing scope should not extend across a block. */
+	if (WARN_ON_ONCE((from >> inode->i_blkbits) !=
+			 ((from + length - 1) >> inode->i_blkbits)))
+		return -EINVAL;
+
+	if (!(EXT4_SB(inode->i_sb)->s_mount_state & EXT4_ORPHAN_FS) &&
+	    !(inode_state_read_once(inode) & (I_NEW | I_FREEING)))
+		WARN_ON_ONCE(!inode_is_locked(inode) &&
+			!rwsem_is_locked(&inode->i_mapping->invalidate_lock));
+
+	ret = iomap_zero_range(inode, from, length, did_zero,
+			       &ext4_iomap_zero_ops, &ext4_iomap_write_ops,
+			       NULL);
+	if (ret)
+		return ret;
+
+	/*
+	 * TODO: The iomap does not distinguish between different types
+	 * of zeroing operations. So we always set zero_written whenever
+	 * zeroing is performed, which may cause unnecessary folio
+	 * flushing when zeroing occurs on delayed-allocated blocks.
+	 */
+	if (did_zero && zero_written)
+		*zero_written = *did_zero;
+
+	return 0;
+}
+
 /*
  * Zeros out a mapping of length 'length' starting from file offset
  * 'from'.  The range to be zero'd must be contained with in one block.
@@ -4608,6 +4711,9 @@ static int ext4_block_zero_range(struct inode *inode,
 	} else if (ext4_should_journal_data(inode)) {
 		return ext4_block_journalled_zero_range(inode, from, length,
 							did_zero);
+	} else if (ext4_inode_buffered_iomap(inode)) {
+		return ext4_block_iomap_zero_range(inode, from, length,
+						   did_zero, zero_written);
 	}
 	return ext4_block_do_zero_range(inode, from, length, did_zero,
 					zero_written);
@@ -4656,6 +4762,10 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
 	 * truncating up or performing an append write, because there might be
 	 * exposing stale on-disk data which may caused by concurrent post-EOF
 	 * mmap write during folio writeback.
+	 *
+	 * TODO: In the iomap path, handle this by tracking the ordered range
+	 * and updating i_disksize to i_size after the zeroed data has been
+	 * written back.
 	 */
 	if (ext4_should_order_data(inode) &&
 	    did_zero && zero_written && !IS_DAX(inode)) {
-- 
2.52.0

[PATCH v7 18/31] ext4: drain writeback before removing extents on the iomap path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Because the iomap infrastructure does not always create an ifs to
manage sub-folio state when folio size is larger than blocksize,
invalidating a partial dirty folio during punch hole may fail to
clear the dirty state of the affected range. As a result, writeback
of that folio may observe a hole. At writeback submit time,
ext4_map_blocks() already handles this case and will not allocate
blocks. However, when punch hole races with writeback, the
following scenario can cause I/O completion to encounter a hole.

  punch hole                        writeback
  ----------                        ---------
  ext4_punch_hole()
    ext4_truncate_page_cache_block_range()
      iomap_invalidate_folio() [partial folio]
        iomap_clear_range_dirty()
        -- no ifs, sub-block dirty bits NOT cleared
                                    ext4_iomap_writepages()
                                      iomap_writepages()
                                        ext4_iomap_writeback_submit()
                                          ext4_iomap_map_writeback_range()
                                            ext4_map_blocks(IO_SUBMIT)
                                              -> extent exists, not a hole
                                          submit_io()  -> bio in flight
    down_write(&i_data_sem)
    ext4_es_remove_extent()
    ext4_ext_remove_space()
      -> extent removed, hole inserted
    up_write(&i_data_sem)
    [ transaction commits, freed blocks released to buddy ]
    [ allocator reuses freed blocks for another inode ]
                                    [bio completes]
                                    ext4_iomap_finish_ioend()
                                      ext4_convert_unwritten_extents()
                                        ext4_map_blocks(IO_CONVERT_EXT)
                                          -> returns 0 (hole found)
                                    [bio writes to reallocated blocks ]
                                    -> silent data corruption

If the bio completes after the freed blocks have been reallocated to
another inode, it silently overwrites them and causes data corruption.
Therefore, on the iomap path, drain writeback in ext4_punch_hole() after
partial zeroing and before starting the freeing transaction, so any bio
in flight on a partial folio completes against blocks still owned by
this inode before extent removal.  Once extent removal begins, the ES
hole insert and the existing IO_SUBMIT short-circuit prevent further bio
submission.

This is a workaround. The proper fix is for the iomap infrastructure to
always maintain an ifs when folio size is larger than blocksize, so that
partial invalidate reliably clears sub-block dirty bits and no draining
is needed.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 fs/ext4/inode.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 35a2c96535c03..35767c3a7ffff 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -5028,7 +5028,14 @@ int ext4_punch_hole(struct file *file, loff_t offset, loff_t length)
 	ret = ext4_zero_partial_blocks(inode, offset, length, &partial_zeroed);
 	if (ret)
 		return ret;
-	if (((file->f_flags & O_SYNC) || IS_SYNC(inode)) && partial_zeroed) {
+	/*
+	 * On the iomap path, partial invalidate of a folio without an ifs
+	 * leaves sub-block dirty bits uncleared.  Drain writeback before
+	 * removing extents so that any bio in flight on a partial folio
+	 * completes against blocks still owned by this inode.
+	 */
+	if (ext4_inode_buffered_iomap(inode) ||
+	    (((file->f_flags & O_SYNC) || IS_SYNC(inode)) && partial_zeroed)) {
 		ret = filemap_write_and_wait_range(inode->i_mapping, offset,
 						   end - 1);
 		if (ret)
-- 
2.52.0

[PATCH v7 19/31] ext4: add block mapping tracepoints for iomap buffered I/O path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Add tracepoints for iomap buffered read, write, partial block zeroing,
and writeback operations to help debug the iomap buffered I/O path.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/inode.c             |  6 +++++
 include/trace/events/ext4.h | 47 +++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 35767c3a7ffff..1c7083a9293e9 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4019,6 +4019,8 @@ static int ext4_iomap_buffered_read_begin(struct inode *inode, loff_t offset,
 	if (ret < 0)
 		return ret;
 
+	trace_ext4_iomap_buffered_read_begin(inode, &map, offset, length,
+					     flags);
 	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
 	return 0;
 }
@@ -4052,6 +4054,8 @@ static int ext4_iomap_buffered_do_write_begin(struct inode *inode,
 	if (ret < 0)
 		return ret;
 
+	trace_ext4_iomap_buffered_write_begin(inode, &map, offset, length,
+					      flags);
 	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
 	return 0;
 }
@@ -4168,6 +4172,7 @@ static int ext4_iomap_zero_begin(struct inode *inode,
 			goto again;
 	}
 
+	trace_ext4_iomap_zero_begin(inode, &map, offset, length, flags);
 	ext4_set_iomap(inode, iomap, &map, offset, length, flags);
 	iomap->flags |= iomap_flags;
 
@@ -4279,6 +4284,7 @@ static int ext4_iomap_map_writeback_range(struct iomap_writepage_ctx *wpc,
 		return ret;
 	}
 
+	trace_ext4_iomap_map_writeback_range(inode, &map, offset, dirty_len, 0);
 	ext4_set_iomap(inode, &wpc->iomap, &map, offset, dirty_len, 0);
 	return 0;
 }
diff --git a/include/trace/events/ext4.h b/include/trace/events/ext4.h
index 7028a28316fa3..bcf80f518e61d 100644
--- a/include/trace/events/ext4.h
+++ b/include/trace/events/ext4.h
@@ -3157,6 +3157,53 @@ TRACE_EVENT(ext4_move_extent_exit,
 		  __entry->ret)
 );
 
+DECLARE_EVENT_CLASS(ext4_set_iomap_class,
+	TP_PROTO(struct inode *inode, struct ext4_map_blocks *map,
+		 loff_t offset, loff_t length, unsigned int flags),
+	TP_ARGS(inode, map, offset, length, flags),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(ext4_lblk_t, m_lblk)
+		__field(ext4_fsblk_t, pblk)
+		__field(unsigned int, m_len)
+		__field(unsigned int, m_flags)
+		__field(u64, m_seq)
+		__field(loff_t, offset)
+		__field(loff_t, length)
+		__field(unsigned int, iomap_flags)
+	),
+	TP_fast_assign(
+		__entry->dev		= inode->i_sb->s_dev;
+		__entry->ino		= inode->i_ino;
+		__entry->m_lblk		= map->m_lblk;
+		__entry->pblk		= map->m_pblk;
+		__entry->m_len		= map->m_len;
+		__entry->m_flags	= map->m_flags;
+		__entry->m_seq		= map->m_seq;
+		__entry->offset		= offset;
+		__entry->length		= length;
+		__entry->iomap_flags	= flags;
+
+	),
+	TP_printk("dev %d:%d ino %llu m_lblk %u pblk %llu m_len %u m_flags %s m_seq %llu orig_off 0x%llx orig_len 0x%llx iomap_flags 0x%x",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->m_lblk, __entry->pblk, __entry->m_len,
+		  show_mflags(__entry->m_flags), __entry->m_seq,
+		  __entry->offset, __entry->length, __entry->iomap_flags)
+)
+
+#define DEFINE_SET_IOMAP_EVENT(name) \
+DEFINE_EVENT(ext4_set_iomap_class, name, \
+	TP_PROTO(struct inode *inode, struct ext4_map_blocks *map, \
+		 loff_t offset, loff_t length, unsigned int flags), \
+	TP_ARGS(inode, map, offset, length, flags))
+
+DEFINE_SET_IOMAP_EVENT(ext4_iomap_buffered_read_begin);
+DEFINE_SET_IOMAP_EVENT(ext4_iomap_buffered_write_begin);
+DEFINE_SET_IOMAP_EVENT(ext4_iomap_map_writeback_range);
+DEFINE_SET_IOMAP_EVENT(ext4_iomap_zero_begin);
+
 #endif /* _TRACE_EXT4_H */
 
 /* This part must be outside protection */
-- 
2.52.0

[PATCH v7 20/31] ext4: disable online defrag when inode using iomap buffered I/O path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Online defragmentation does not currently support inodes using the
iomap buffered I/O path. The existing implementation relies on
buffer_head for sub-folio block management and data=ordered mode for
data consistency, both of which are incompatible with the iomap path.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/move_extent.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c
index 3329b7ad5dbd8..948ef3f44df5e 100644
--- a/fs/ext4/move_extent.c
+++ b/fs/ext4/move_extent.c
@@ -476,6 +476,17 @@ static int mext_check_validity(struct inode *orig_inode,
 		return -EOPNOTSUPP;
 	}
 
+	/*
+	 * TODO: support online defrag for inodes that use the buffered
+	 * I/O iomap path.
+	 */
+	if (ext4_inode_buffered_iomap(orig_inode) ||
+	    ext4_inode_buffered_iomap(donor_inode)) {
+		ext4_msg(sb, KERN_ERR,
+			 "Online defrag not supported for inode with iomap buffered IO path");
+		return -EOPNOTSUPP;
+	}
+
 	if (donor_inode->i_mode & (S_ISUID|S_ISGID)) {
 		ext4_debug("ext4 move extent: suid or sgid is set to donor file [ino:orig %llu, donor %llu]\n",
 			   orig_inode->i_ino, donor_inode->i_ino);
-- 
2.52.0

[PATCH v7 21/31] ext4: add EXT4_STATE_DISKSIZE_GROW_PENDING state bit and helpers

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Inodes using the iomap buffered I/O path do not use data=ordered
mode, so the zeroed EOF block that straddles i_disksize needs explicit
tracking to ensure it is written back before i_disksize is advanced.

Add the EXT4_STATE_DISKSIZE_GROW_PENDING inode state bit and three
helpers: ext4_iomap_clear_disksize_pending() to atomically clear the
bit and wake waiters, ext4_iomap_wait_disksize_pending() to block until
the bit is cleared, and ext4_iomap_get_disksize_pending_range() to
compute the pending range from i_disksize.

These will be used by subsequent patches to serialize i_disksize
updates with the writeback of the zeroed EOF block.

Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h  |  7 ++++++
 fs/ext4/inode.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 03fa90d2986ff..1c3d736fb700b 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -2052,6 +2052,9 @@ enum {
 	EXT4_STATE_ORPHAN_FILE,		/* Inode orphaned in orphan file */
 	EXT4_STATE_FC_REQUEUE,		/* Inode modified during fast commit */
 	EXT4_STATE_BUFFERED_IOMAP,	/* Inode use iomap for buffered IO */
+	EXT4_STATE_DISKSIZE_GROW_PENDING,
+					/* Has zeroed EOF block straddles
+					 * i_disksize awaiting writeback */
 };
 
 #define EXT4_INODE_BIT_FNS(name, field, offset)				\
@@ -3219,6 +3222,10 @@ extern int ext4_chunk_trans_blocks(struct inode *, int nrblocks);
 extern int ext4_chunk_trans_extent(struct inode *inode, int nrblocks);
 extern int ext4_meta_trans_blocks(struct inode *inode, int lblocks,
 				  int pextents, int alloc_extents);
+void ext4_iomap_clear_disksize_pending(struct inode *inode);
+void ext4_iomap_wait_disksize_pending(struct inode *inode);
+unsigned int ext4_iomap_get_disksize_pending_range(struct inode *inode,
+						   loff_t *start);
 extern int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end);
 
 #define EXT4_PARTIAL_ZERO_START	0x1
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 1c7083a9293e9..9c45a0c01d1c1 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -126,6 +126,63 @@ void ext4_inode_csum_set(struct inode *inode, struct ext4_inode *raw,
 		raw->i_checksum_hi = cpu_to_le16(csum >> 16);
 }
 
+/*
+ * Clear the disksize-grow-pending state and wake up all waiters.
+ * Called when the pending zeroed EOF block which straddles i_disksize
+ * has completed writeback or its folio is discarded.
+ */
+void ext4_iomap_clear_disksize_pending(struct inode *inode)
+{
+	clear_and_wake_up_bit(
+		ext4_inode_state_wait_bit(EXT4_STATE_DISKSIZE_GROW_PENDING),
+		ext4_inode_state_wait_word(inode));
+}
+
+/*
+ * Wait for the disksize-grow-pending zeroed EOF block which straddles
+ * i_disksize to be written back or cleared.
+ */
+void ext4_iomap_wait_disksize_pending(struct inode *inode)
+{
+	wait_on_bit(ext4_inode_state_wait_word(inode),
+		    ext4_inode_state_wait_bit(EXT4_STATE_DISKSIZE_GROW_PENDING),
+		    TASK_UNINTERRUPTIBLE);
+}
+
+/*
+ * Get the range of the disksize-grow-pending zeroed EOF block range
+ * which straddles i_disksize if the EXT4_STATE_DISKSIZE_GROW_PENDING
+ * bit is set.
+ *
+ * Return the pending range, or zero if the BIT has already been cleared.
+ */
+unsigned int ext4_iomap_get_disksize_pending_range(struct inode *inode,
+						   loff_t *start)
+{
+	unsigned int blocksize = i_blocksize(inode);
+	loff_t disksize;
+
+	if (!ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING))
+		return 0;
+
+	/*
+	 * The pending bit should be set only when i_disksize is not
+	 * block-size aligned. While set, i_disksize must not be advanced,
+	 * and the bit must be cleared when i_disksize is shrunk.
+	 */
+	down_read(&EXT4_I(inode)->i_data_sem);
+	disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+	if (!ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING) ||
+	    WARN_ON_ONCE(IS_ALIGNED(disksize, blocksize))) {
+		up_read(&EXT4_I(inode)->i_data_sem);
+		return 0;
+	}
+
+	up_read(&EXT4_I(inode)->i_data_sem);
+	*start = disksize;
+	return blocksize - (disksize & (blocksize - 1));
+}
+
 static inline int ext4_begin_ordered_truncate(struct inode *inode,
 					      loff_t new_size)
 {
-- 
2.52.0

[PATCH v7 22/31] ext4: submit and wait for pending disksize-grow I/O on writeback

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

When the current writeback pass begins beyond the disksize-grow-pending
zeroed EOF block, the ioend worker would otherwise have to wait for the
pending EOF block to complete before it can advance i_disksize.
Otherwise the old EOF block could be exposed as stale data once
i_disksize advances past it.

Therefore, introduce the ioend mechanism for the pending range, tag
ioends that cover the pending zeroed EOF block which straddles
i_disksize with EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO in
ext4_iomap_writeback_submit(), and clear the bit and wake up all waiters
in ext4_iomap_end_bio() when such an ioend completes.

Clearing EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO does not depend on whether
the disksize grow I/O succeeds. That is, even if the I/O fails, we still
allow subsequent writes in the range to update i_disksize. This is
consistent with the previous behavior, and we rely on data_err=abort to
prevent metadata updates when data write failures occur.

In order to avoid the ioend that passes the pending range waiting for a
long time, proactively submit the pending range first in
ext4_iomap_writepages() so it completes in parallel with the rest of the
writeback.

Note that the handling of discarding the zeroed EOF folio will be
processed later, otherwise the bit will be set forever.
EXT4_STATE_DISKSIZE_GROW_PENDING will be set after everthing is done.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h    |  6 +++++
 fs/ext4/inode.c   | 56 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/ext4/page-io.c | 40 +++++++++++++++++++++++++++++++++
 3 files changed, 101 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 1c3d736fb700b..089dbd39c5c24 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3986,6 +3986,12 @@ extern int ext4_move_extents(struct file *o_filp, struct file *d_filp,
 			     __u64 len, __u64 *moved_len);
 
 /* page-io.c */
+/*
+ * The I/O range covers the zeroed EOF block that straddles i_disksize
+ * and will advance it upon completion.
+ */
+#define EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO	1UL
+
 extern int __init ext4_init_pageio(void);
 extern void ext4_exit_pageio(void);
 extern ext4_io_end_t *ext4_init_io_end(struct inode *inode, gfp_t flags);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 9c45a0c01d1c1..76e070eaa6f89 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4372,7 +4372,10 @@ static int ext4_iomap_writeback_submit(struct iomap_writepage_ctx *wpc,
 				       int error)
 {
 	struct iomap_ioend *ioend = wpc->wb_ctx;
-	struct ext4_inode_info *ei = EXT4_I(ioend->io_inode);
+	struct inode *inode = ioend->io_inode;
+	struct ext4_inode_info *ei = EXT4_I(inode);
+	unsigned int blocksize = i_blocksize(inode);
+	loff_t pstart, plen;
 
 	/*
 	 * After I/O completion, a worker needs to be scheduled when:
@@ -4385,6 +4388,21 @@ static int ext4_iomap_writeback_submit(struct iomap_writepage_ctx *wpc,
 	    test_opt(ioend->io_inode->i_sb, DATA_ERR_ABORT))
 		ioend->io_bio.bi_end_io = ext4_iomap_end_bio;
 
+	/*
+	 * Mark the I/O as DISKSIZE_GROW_IO by setting io_private to
+	 * EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO if it covers the pending range.
+	 * Such I/O will allow or trigger i_disksize advancement in the
+	 * ioend worker.
+	 */
+	plen = ext4_iomap_get_disksize_pending_range(inode, &pstart);
+	if (plen &&
+	    round_down(ioend->io_offset, blocksize) <= pstart &&
+	    round_up(ioend->io_offset + ioend->io_size, blocksize) >=
+			pstart + plen) {
+		ioend->io_bio.bi_end_io = ext4_iomap_end_bio;
+		ioend->io_private = (void *)EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO;
+	}
+
 	/*
 	 * ext4_iomap_end_bio() always defers endio processing, disable
 	 * generic BIO in task to avoid double deferral since we will use
@@ -4404,6 +4422,36 @@ static const struct iomap_writeback_ops ext4_writeback_ops = {
 	.writeback_submit = ext4_iomap_writeback_submit,
 };
 
+/*
+ * If the current writeback range begins after the pending zeroed EOF
+ * block range which straddles i_disksize, issue a separate writeback to
+ * flush it first, so as to avoid prolonged waiting.
+ */
+static void ext4_iomap_wb_submit_zeroed_eof(struct inode *inode,
+					    struct writeback_control *wbc)
+{
+	struct address_space *mapping = inode->i_mapping;
+	loff_t pstart, plen, range_start;
+
+	if (wbc->range_cyclic)
+		range_start = (loff_t)mapping->writeback_index << PAGE_SHIFT;
+	else
+		range_start = wbc->range_start;
+
+	plen = ext4_iomap_get_disksize_pending_range(inode, &pstart);
+	if (!plen)
+		return;
+	if (round_down(range_start, mapping_min_folio_nrbytes(mapping)) <
+	    pstart + plen)
+		return;
+
+	/* Keep the caller's sync mode to avoid stalling the background flusher. */
+	if (wbc->sync_mode == WB_SYNC_ALL)
+		filemap_fdatawrite_range(mapping, pstart, pstart + plen - 1);
+	else
+		filemap_flush_range(mapping, pstart, pstart + plen - 1);
+}
+
 static int ext4_iomap_writepages(struct address_space *mapping,
 				 struct writeback_control *wbc)
 {
@@ -4421,6 +4469,12 @@ static int ext4_iomap_writepages(struct address_space *mapping,
 	if (unlikely(ret))
 		return ret;
 
+	/*
+	 * Submit the pending zeroed EOF block range if the entire
+	 * writeback range lies beyond it.
+	 */
+	ext4_iomap_wb_submit_zeroed_eof(inode, wbc);
+
 	alloc_ctx = ext4_writepages_down_read(sb);
 	trace_ext4_writepages(inode, wbc);
 	ret = iomap_writepages(&wpc);
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 9b0e12b5463cf..697e12a54a49a 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -549,6 +549,34 @@ void ext4_bio_write_folio(struct ext4_io_submit *io, struct folio *folio,
 	} while ((bh = bh->b_this_page) != head);
 }
 
+/*
+ * If the current writeback range starts beyond the zeroed EOF pending
+ * range that straddles i_disksize, wait for the zeroed data from
+ * ext4_block_zero_eof() to be written out first. Otherwise, extending
+ * i_disksize may expose stale data in the old EOF block.
+ */
+static void ext4_iomap_wb_disksize_pending_wait(struct inode *inode,
+						loff_t pos, size_t size)
+{
+	loff_t disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+	loff_t pstart, plen;
+
+	/*
+	 * Overwrite I/Os and I/Os covering the EOF block do not need to
+	 * wait: the former do not advance i_disksize past the pending
+	 * boundary, and the latter are the pending I/O itself (cleared in
+	 * the bio completion path).
+	 */
+	if (pos < round_up(disksize, i_blocksize(inode)))
+		return;
+
+	plen = ext4_iomap_get_disksize_pending_range(inode, &pstart);
+	if (!plen || pos < pstart + plen)
+		return;
+
+	ext4_iomap_wait_disksize_pending(inode);
+}
+
 static int ext4_iomap_wb_update_disksize(handle_t *handle, struct inode *inode,
 					 loff_t end)
 {
@@ -594,6 +622,9 @@ static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
 	    end <= READ_ONCE(EXT4_I(inode)->i_disksize))
 		goto out;
 
+	/* Wait for disksize-pending zeroed data to be written out. */
+	ext4_iomap_wb_disksize_pending_wait(inode, pos, size);
+
 	/*
 	 * We may need to convert one extent, update the i_disksize and
 	 * dirty the inode.
@@ -660,8 +691,17 @@ void ext4_iomap_end_bio(struct bio *bio)
 {
 	struct iomap_ioend *ioend = iomap_ioend_from_bio(bio);
 	struct ext4_inode_info *ei = EXT4_I(ioend->io_inode);
+	unsigned long io_mode = (unsigned long)ioend->io_private;
 	unsigned long flags;
 
+	/*
+	 * This is a disksize-pending I/O: clear the disksize-pending
+	 * state set in ext4_block_zero_eof() and wake up all waiters
+	 * that will update the inode i_disksize.
+	 */
+	if (io_mode == EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO)
+		ext4_iomap_clear_disksize_pending(ioend->io_inode);
+
 	spin_lock_irqsave(&ei->i_completed_io_lock, flags);
 	if (list_empty(&ei->i_rsv_conversion_list))
 		queue_work(EXT4_SB(ioend->io_inode->i_sb)->rsv_conversion_wq,
-- 
2.52.0

[PATCH v7 23/31] ext4: advance i_disksize to i_size upon disksize-grow I/O completion

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

When the disksize-grow-pending zeroed EOF I/O completes, i_disksize has
to be advanced. Advancing it only to the end of that specific I/O would
discard any later i_disksize updates from concurrent fallocate or
similar operations, causing filesystem inconsistency.

Scanning dirty or writeback folios beyond the current position to
compute a safe advance target is expensive and racy with concurrent
fallocate, so instead advance i_disksize directly to i_size. This may
expose zeroed data (not stale data) after crash recovery when dirty data
in the range is not yet on disk, but only for unaligned append writes,
which is deemed acceptable.

To support this, teach ext4_iomap_wb_update_disksize() to take an
is_disksize_grow flag and advance i_disksize to i_size when set, and
have ext4_iomap_finish_ioend() pass the flag based on the
EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO tag of the completing ioend.

Note that EXT4_STATE_DISKSIZE_GROW_PENDING is not set for now, and it
will be set after everthing is done.

Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/page-io.c | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 697e12a54a49a..4f1176b9332f2 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -578,9 +578,9 @@ static void ext4_iomap_wb_disksize_pending_wait(struct inode *inode,
 }
 
 static int ext4_iomap_wb_update_disksize(handle_t *handle, struct inode *inode,
-					 loff_t end)
+					 loff_t end, bool is_disksize_grow)
 {
-	loff_t new_disksize = end;
+	loff_t new_disksize, i_size;
 	struct ext4_inode_info *ei = EXT4_I(inode);
 	int ret;
 
@@ -589,7 +589,34 @@ static int ext4_iomap_wb_update_disksize(handle_t *handle, struct inode *inode,
 	 * i_data_sem.
 	 */
 	down_write(&ei->i_data_sem);
-	new_disksize = min(new_disksize, i_size_read(inode));
+	i_size = i_size_read(inode);
+
+	/*
+	 * EXT4_STATE_DISKSIZE_GROW_PENDING is cleared when the pending
+	 * I/O completes. However, another thread may have re-set the bit
+	 * between that point and here, meaning i_disksize has already
+	 * been advanced and a new EOF zeroing has been initiated. In that
+	 * case, do not advance i_disksize to i_size; leave it to the
+	 * next pending grow ioend.
+	 */
+	if (is_disksize_grow &&
+	    ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING))
+		is_disksize_grow = false;
+
+	/*
+	 * Update i_disksize to i_size when EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO
+	 * completes. This is safe because we never directly allocate written
+	 * blocks during buffered writes.
+	 *
+	 * This ensures that i_disksize is correctly advanced during
+	 * truncate-up or append fallocate on a block-unaligned file,
+	 * preventing it from remaining stale. The tradeoff is that zeroed
+	 * data may be exposed after crash recovery if dirty data in this
+	 * range is not yet on disk, but stale data will never be exposed.
+	 * This is because the extent is only converted to written state
+	 * after the data has been persisted.
+	 */
+	new_disksize = is_disksize_grow ? i_size : min(end, i_size);
 	if (new_disksize > ei->i_disksize)
 		WRITE_ONCE(ei->i_disksize, new_disksize);
 	up_write(&ei->i_data_sem);
@@ -607,6 +634,8 @@ static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
 	loff_t pos = ioend->io_offset;
 	size_t size = ioend->io_size;
 	loff_t end = pos + size;
+	unsigned long io_mode = (unsigned long)ioend->io_private;
+	bool is_disksize_grow = (io_mode == EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO);
 	handle_t *handle;
 	int credits;
 	int ret, err;
@@ -619,7 +648,7 @@ static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
 	}
 
 	if (!(ioend->io_flags & IOMAP_IOEND_UNWRITTEN) &&
-	    end <= READ_ONCE(EXT4_I(inode)->i_disksize))
+	    end <= READ_ONCE(EXT4_I(inode)->i_disksize) && !is_disksize_grow)
 		goto out;
 
 	/* Wait for disksize-pending zeroed data to be written out. */
@@ -638,8 +667,9 @@ static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
 	}
 
 	/* Update on-disk size after I/O is completed. */
-	if (end > READ_ONCE(EXT4_I(inode)->i_disksize)) {
-		ret = ext4_iomap_wb_update_disksize(handle, inode, end);
+	if (end > READ_ONCE(EXT4_I(inode)->i_disksize) || is_disksize_grow) {
+		ret = ext4_iomap_wb_update_disksize(handle, inode, end,
+						    is_disksize_grow);
 		if (ret)
 			goto out_journal;
 	}
-- 
2.52.0

[PATCH v7 24/31] ext4: defer i_disksize update while DISKSIZE_GROW_PENDING is set

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Operations like append allocate, zero range, and truncate update
i_disksize directly. If the new i_disksize exceeds the original value
while the zeroed EOF block is still awaiting writeback, metadata may be
persisted before the zeroed data, exposing stale data on crash.

Defer i_disksize updates while EXT4_STATE_DISKSIZE_GROW_PENDING is set;
the ioend worker for the pending block will advance i_disksize to i_size
once the zeroed data is written back. The tradeoff is that i_disksize
may lag i_size transiently, but this is observable only to callers that
read i_disksize directly.

Introduce __ext4_set_i_disksize() to centralize the bit check for
callers already holding i_data_sem (ext4_ext_truncate and
ext4_set_inode_size), and refactor ext4_update_inode_size() to take
i_data_sem itself and check the bit atomically with i_size_write(), so
the ioend worker observes the latest i_size under the same lock.

Note that in the O_SYNC mode of ext4_do_fallocate() or
ext4_zero_range(), we need to flush out pending blocks that can update
i_disksize synchronously. This will be handled later, and
EXT4_STATE_DISKSIZE_GROW_PENDING will also be set after everything is
done.

Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h    | 47 ++++++++++++++++++++++++++++++++++++++++++-----
 fs/ext4/extents.c |  2 +-
 fs/ext4/inode.c   |  8 +++++---
 3 files changed, 48 insertions(+), 9 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 089dbd39c5c24..b26ce3183bacb 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3605,30 +3605,67 @@ do {								\
 #define EXT4_FREECLUSTERS_WATERMARK 0
 #endif
 
-/* Update i_disksize. Requires i_rwsem to avoid races with truncate */
+/*
+ * Update i_disksize. Requires i_rwsem to avoid races with truncate.
+ *
+ * In the iomap buffered I/O path, the EXT4_STATE_DISKSIZE_GROW_PENDING
+ * inode state bit indicates that the zeroed EOF partial block which
+ * straddles i_disksize is still waiting writeback.  In that case,
+ * i_disksize will be updated after the pending zeroed data has been
+ * written out.
+ */
 static inline void ext4_update_i_disksize(struct inode *inode, loff_t newsize)
 {
 	WARN_ON_ONCE(S_ISREG(inode->i_mode) &&
 		     !inode_is_locked(inode));
 	down_write(&EXT4_I(inode)->i_data_sem);
-	if (newsize > EXT4_I(inode)->i_disksize)
+	if (newsize > EXT4_I(inode)->i_disksize &&
+	    !ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING))
 		WRITE_ONCE(EXT4_I(inode)->i_disksize, newsize);
 	up_write(&EXT4_I(inode)->i_data_sem);
 }
 
-/* Update i_size, i_disksize. Requires i_rwsem to avoid races with truncate */
+static inline void __ext4_set_i_disksize(struct inode *inode, loff_t newsize)
+{
+	WARN_ON_ONCE(!rwsem_is_locked(&EXT4_I(inode)->i_data_sem));
+
+	if (newsize < EXT4_I(inode)->i_disksize ||
+	    !ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING))
+		WRITE_ONCE(EXT4_I(inode)->i_disksize, newsize);
+}
+
+/*
+ * Update i_size and i_disksize to @newsize.  Requires i_rwsem to avoid
+ * races with truncate.
+ *
+ * In the iomap buffered I/O path, i_disksize is updated only if no zeroed
+ * pending block straddles i_disksize (EXT4_STATE_DISKSIZE_GROW_PENDING
+ * clear), otherwise the ioend worker for the pending block will advance
+ * i_disksize once the pending block is written back.  Both updates happen
+ * under i_data_sem so that the writeback ioend worker can always see the
+ * latest i_size under the same semaphore.
+ *
+ * Returns 0 if nothing changed, 1 if i_size was raised, 2 if i_disksize
+ * was raised, or 3 if both were.
+ */
 static inline int ext4_update_inode_size(struct inode *inode, loff_t newsize)
 {
 	int changed = 0;
 
+	if (newsize <= inode->i_size && newsize <= EXT4_I(inode)->i_disksize)
+		return 0;
+
+	down_write(&EXT4_I(inode)->i_data_sem);
 	if (newsize > inode->i_size) {
 		i_size_write(inode, newsize);
 		changed = 1;
 	}
-	if (newsize > EXT4_I(inode)->i_disksize) {
-		ext4_update_i_disksize(inode, newsize);
+	if (newsize > EXT4_I(inode)->i_disksize &&
+	    !ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING)) {
+		WRITE_ONCE(EXT4_I(inode)->i_disksize, newsize);
 		changed |= 2;
 	}
+	up_write(&EXT4_I(inode)->i_data_sem);
 	return changed;
 }
 
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 83d040e36b682..b52e5fb156894 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -4561,7 +4561,7 @@ int ext4_ext_truncate(handle_t *handle, struct inode *inode)
 	 */
 
 	/* we have to know where to truncate from in crash case */
-	EXT4_I(inode)->i_disksize = inode->i_size;
+	__ext4_set_i_disksize(inode, inode->i_size);
 	err = ext4_mark_inode_dirty(handle, inode);
 	if (err)
 		return err;
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 76e070eaa6f89..5644807cc875b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -6635,8 +6635,10 @@ static void ext4_wait_for_tail_page_commit(struct inode *inode)
  * Set i_size and i_disksize to 'newsize'.
  *
  * Both i_rwsem and i_data_sem are required here to avoid races between
- * generic append writeback and concurrent truncate that also modify
- * i_size and i_disksize.
+ * generic append writeback (or zeroed pending I/O writeback) and
+ * concurrent operations (e.g., fallocate, truncate) that also modify
+ * i_size and i_disksize. This also ensures that the writeback ioend worker
+ * observes the latest i_size under the same lock protection.
  */
 static inline void ext4_set_inode_size(struct inode *inode, loff_t newsize)
 {
@@ -6644,7 +6646,7 @@ static inline void ext4_set_inode_size(struct inode *inode, loff_t newsize)
 
 	down_write(&EXT4_I(inode)->i_data_sem);
 	i_size_write(inode, newsize);
-	EXT4_I(inode)->i_disksize = newsize;
+	__ext4_set_i_disksize(inode, newsize);
 	up_write(&EXT4_I(inode)->i_data_sem);
 }
 
-- 
2.52.0

[PATCH v7 25/31] ext4: submit and wait for disksize-grow I/O in fallocate paths

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Collapse range and insert range update i_disksize directly under
i_data_sem. If the operation runs while the zeroed EOF block is still
awaiting writeback, i_disksize could advance past the zeroed boundary
before the zeroed data is persisted, exposing stale data on crash.

Deferring i_disksize updates like fallocate and zero_range is not an
option here because the shift would move written extents beyond the
current i_disksize. So flush and wait for the pending zeroed EOF block
before these operations advance i_disksize. Since these operations
already perform writeback, the extra flush does not add significant
overhead.

In addition, for ext4_update_disksize_before_punch(), if the punch
discards the pending block, the zeroed data will never be written back
before advancing i_disksize, so it is also necessary to sync the
pending EOF range there.

Finally, for the SYNC variants of zero_range and fallocate, this also
guarantees the i_disksize update is persisted on the synchronous
return.

Note that EXT4_STATE_DISKSIZE_GROW_PENDING is not set for now, and it
will be set after everything is done.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h    |  2 ++
 fs/ext4/extents.c | 50 ++++++++++++++++++++++++++++++++++++++++-------
 fs/ext4/inode.c   | 41 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 86 insertions(+), 7 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index b26ce3183bacb..504dce9fdc6b9 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3226,6 +3226,8 @@ void ext4_iomap_clear_disksize_pending(struct inode *inode);
 void ext4_iomap_wait_disksize_pending(struct inode *inode);
 unsigned int ext4_iomap_get_disksize_pending_range(struct inode *inode,
 						   loff_t *start);
+extern int ext4_iomap_sync_zeroed_eof(struct inode *inode,
+				      loff_t offset, loff_t end);
 extern int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end);
 
 #define EXT4_PARTIAL_ZERO_START	0x1
diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index b52e5fb156894..a10d8c6e3cf1f 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -4857,6 +4857,17 @@ static long ext4_zero_range(struct file *file, loff_t offset,
 		if (ret)
 			return ret;
 	}
+
+	/*
+	 * In SYNC mode, sync the pending zeroed EOF block to ensure the
+	 * i_disksize update is persisted.
+	 */
+	if (((file->f_flags & O_SYNC) || IS_SYNC(inode)) && new_size) {
+		ret = ext4_iomap_sync_zeroed_eof(inode, 0, LLONG_MAX);
+		if (ret)
+			return ret;
+	}
+
 	/* Finish zeroing out if it doesn't contain partial block */
 	if (IS_ALIGNED(offset | end, blocksize))
 		return ret;
@@ -4928,10 +4939,20 @@ static long ext4_do_fallocate(struct file *file, loff_t offset,
 	if (ret)
 		goto out;
 
-	if (((file->f_flags & O_SYNC) || IS_SYNC(inode)) &&
-	    EXT4_SB(inode->i_sb)->s_journal) {
-		ret = ext4_fc_commit(EXT4_SB(inode->i_sb)->s_journal,
-					EXT4_I(inode)->i_sync_tid);
+	if ((file->f_flags & O_SYNC) || IS_SYNC(inode)) {
+		/*
+		 * Sync the pending zeroed EOF block to ensure the
+		 * i_disksize update is persisted.
+		 */
+		if (new_size) {
+			ret = ext4_iomap_sync_zeroed_eof(inode, 0, LLONG_MAX);
+			if (ret)
+				goto out;
+		}
+		if (EXT4_SB(inode->i_sb)->s_journal) {
+			ret = ext4_fc_commit(EXT4_SB(inode->i_sb)->s_journal,
+						EXT4_I(inode)->i_sync_tid);
+		}
 	}
 out:
 	trace_ext4_fallocate_exit(inode, offset,
@@ -5673,6 +5694,14 @@ static int ext4_collapse_range(struct file *file, loff_t offset, loff_t len)
 	if (end >= inode->i_size)
 		return -EINVAL;
 
+	/*
+	 * Persist the pending zeroed EOF block to ensure i_disksize
+	 * can be safely updated thereafter.
+	 */
+	ret = ext4_iomap_sync_zeroed_eof(inode, 0, LLONG_MAX);
+	if (ret)
+		return ret;
+
 	/*
 	 * Write tail of the last page before removed range and data that
 	 * will be shifted since they will get removed from the page cache
@@ -5722,7 +5751,7 @@ static int ext4_collapse_range(struct file *file, loff_t offset, loff_t len)
 
 	new_size = inode->i_size - len;
 	i_size_write(inode, new_size);
-	EXT4_I(inode)->i_disksize = new_size;
+	__ext4_set_i_disksize(inode, new_size);
 
 	up_write(&EXT4_I(inode)->i_data_sem);
 	ret = ext4_mark_inode_dirty(handle, inode);
@@ -5775,6 +5804,14 @@ static int ext4_insert_range(struct file *file, loff_t offset, loff_t len)
 	if (len > inode->i_sb->s_maxbytes - inode->i_size)
 		return -EFBIG;
 
+	/*
+	 * Persist the pending zeroed EOF block to ensure i_disksize
+	 * can be safely updated thereafter.
+	 */
+	ret = ext4_iomap_sync_zeroed_eof(inode, 0, LLONG_MAX);
+	if (ret)
+		return ret;
+
 	/*
 	 * Write out all dirty pages. Need to round down to align start offset
 	 * to page size boundary for page size > block size.
@@ -5794,8 +5831,7 @@ static int ext4_insert_range(struct file *file, loff_t offset, loff_t len)
 	ext4_fc_mark_ineligible(sb, EXT4_FC_REASON_FALLOC_RANGE, handle);
 
 	/* Expand file to avoid data loss if there is error while shifting */
-	inode->i_size += len;
-	EXT4_I(inode)->i_disksize += len;
+	ext4_update_inode_size(inode, inode->i_size + len);
 	ret = ext4_mark_inode_dirty(handle, inode);
 	if (ret)
 		goto out_handle;
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 5644807cc875b..55f77b52e1685 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4836,6 +4836,39 @@ static int ext4_block_zero_range(struct inode *inode,
 					zero_written);
 }
 
+/*
+ * Submit and wait for the pending zeroed EOF block range to complete
+ * if the given range [@offset, @end) fully covers it.  Must be called
+ * outside the context of an active journal handle and hold the i_rwsem.
+ */
+int ext4_iomap_sync_zeroed_eof(struct inode *inode, loff_t offset, loff_t end)
+{
+	loff_t pstart, plen;
+	int ret;
+
+	if (!ext4_inode_buffered_iomap(inode))
+		return 0;
+
+	plen = ext4_iomap_get_disksize_pending_range(inode, &pstart);
+	if (!plen || offset > pstart || end < pstart + plen)
+		return 0;
+
+	ret = filemap_write_and_wait_range(inode->i_mapping, pstart,
+					   pstart + plen - 1);
+	if (ret)
+		return ret;
+
+	/*
+	 * The pending bit should have been cleared by the ioend worker,
+	 * which runs before the folio writeback flag is cleared.  The
+	 * caller holds i_rwsem, so no concurrent ext4_block_zero_eof()
+	 * can re-set it.
+	 */
+	WARN_ON_ONCE(ext4_test_inode_state(inode,
+				EXT4_STATE_DISKSIZE_GROW_PENDING));
+	return 0;
+}
+
 /*
  * Zero out a mapping from file offset 'from' up to the end of the block
  * which corresponds to 'from' or to the given 'end' inside this block.
@@ -4994,6 +5027,14 @@ int ext4_update_disksize_before_punch(struct inode *inode, loff_t offset,
 	if (offset > size)
 		return 0;
 
+	/*
+	 * We are going to punch the pending zeroed EOF block, persist
+	 * it to ensure i_disksize can be safely updated thereafter.
+	 */
+	ret = ext4_iomap_sync_zeroed_eof(inode, offset, offset + len);
+	if (ret)
+		return ret;
+
 	if (offset + len < size)
 		size = offset + len;
 	if (EXT4_I(inode)->i_disksize >= size)
-- 
2.52.0

[PATCH v7 26/31] ext4: clear DISKSIZE_GROW_PENDING on truncate or error

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

The disksize-grow-pending state is set when a zeroed EOF block is queued
for writeback and cleared by the ioend completion path once writeback
finishes. However, the zeroed block may be discarded before writeback
completes — through folio discard, truncate, or unlink and inode
eviction.

In any of these cases, leaving the bit set would block subsequent
writeback indefinitely. Therefore, we must clear it on all paths that
invalidate the pending block before writeback completes:
 - In ext4_iomap_discard_folio(), on folio discard.
 - In ext4_evict_inode(), when an unlinked inode is destroyed (don't
   bother with an inode that has nlink > 0, since it is only evicted
   after its dirty pages have been written out, and also don't bother
   with bad inodes, since they can never have the bit set).

In ext4_truncate_down(), truncating past the pending zeroed EOF block
also invalidates the pending disksize update, so the bit must be cleared
there as well.

Finally, add a WARN_ON in ext4_destroy_inode() to catch any inode
destroyed with the bit still set. The check is skipped when the
filesystem is in an error or emergency state, as the pending block may
not have been written back in those cases.

Note that EXT4_STATE_DISKSIZE_GROW_PENDING is not set for now, and it
will be set after everthing is done.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/inode.c | 24 +++++++++++++++++++++++-
 fs/ext4/super.c |  8 ++++++++
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 55f77b52e1685..9b6916609000c 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -266,6 +266,8 @@ void ext4_evict_inode(struct inode *inode)
 
 	if (ext4_should_order_data(inode))
 		ext4_begin_ordered_truncate(inode, 0);
+	if (ext4_inode_buffered_iomap(inode))
+		ext4_iomap_clear_disksize_pending(inode);
 	truncate_inode_pages_final(&inode->i_data);
 
 	/*
@@ -4350,8 +4352,17 @@ static void ext4_iomap_discard_folio(struct folio *folio, loff_t pos)
 {
 	struct inode *inode = folio->mapping->host;
 	loff_t length = folio_pos(folio) + folio_size(folio) - pos;
+	loff_t pstart, plen;
 
 	ext4_iomap_punch_delalloc(inode, pos, length, NULL);
+
+	/*
+	 * Clear the disksize-grow-pending state if the zeroed EOF block
+	 * fails to write back and is discarded.
+	 */
+	plen = ext4_iomap_get_disksize_pending_range(inode, &pstart);
+	if (plen && pos <= pstart && folio_next_pos(folio) >= pstart + plen)
+		ext4_iomap_clear_disksize_pending(inode);
 }
 
 static ssize_t ext4_iomap_writeback_range(struct iomap_writepage_ctx *wpc,
@@ -6761,7 +6772,18 @@ static int ext4_truncate_down(struct inode *inode, loff_t oldsize,
 	start_lblk = newsize > 0 ? (newsize - 1) >> inode->i_blkbits : 0;
 	ext4_fc_track_range(handle, inode, start_lblk, EXT_MAX_BLOCKS - 1);
 
-	ext4_set_inode_size(inode, newsize);
+	down_write(&EXT4_I(inode)->i_data_sem);
+	/*
+	 * Truncate the zeroed EOF block invalidates the pending disksize
+	 * update, so clear the disksize-grow-pending state.
+	 */
+	if (ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING) &&
+	    (newsize <= EXT4_I(inode)->i_disksize))
+		ext4_iomap_clear_disksize_pending(inode);
+
+	i_size_write(inode, newsize);
+	__ext4_set_i_disksize(inode, newsize);
+	up_write(&EXT4_I(inode)->i_data_sem);
 
 	ret = ext4_mark_inode_dirty(handle, inode);
 	ext4_journal_stop(handle);
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 1c2395aa1d53a..73735fd336b91 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1497,6 +1497,14 @@ static void ext4_destroy_inode(struct inode *inode)
 			 "Inode %llu (%p): i_reserved_data_blocks (%u) not cleared!",
 			 inode->i_ino, EXT4_I(inode),
 			 EXT4_I(inode)->i_reserved_data_blocks);
+
+	if (!(EXT4_SB(inode->i_sb)->s_mount_state & EXT4_ERROR_FS) &&
+	    !ext4_emergency_state(inode->i_sb) &&
+	    WARN_ON_ONCE(ext4_test_inode_state(inode,
+				EXT4_STATE_DISKSIZE_GROW_PENDING)))
+		ext4_msg(inode->i_sb, KERN_ERR,
+			 "Inode %llu (%p): EXT4_STATE_DISKSIZE_GROW_PENDING not cleared!",
+			 inode->i_ino, EXT4_I(inode));
 }
 
 static void ext4_shutdown(struct super_block *sb)
-- 
2.52.0

[PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

In the iomap buffered I/O path, data=ordered mode is not used, so the
zeroed EOF block has no implicit ordering with later i_disksize updates.
Without the pending state being set, i_disksize can be advanced past the
zeroed block before writeback completes, exposing stale data after a
crash.

Previous patches added the consumer side of the
disksize-grow-pending mechanism: the state bit, clear and wait helpers,
and ioend tagging. Now add ext4_iomap_mark_disksize_pending() and call
it from ext4_block_zero_eof() after zeroing the tail of the block that
straddles i_disksize.

The helper locks the folio, waits for any in-flight writeback on it to
complete, then sets EXT4_STATE_DISKSIZE_GROW_PENDING only if the folio
is still dirty. Waiting for writeback prevents folio_test_dirty() from
returning false mid-writeback, which would cause us to skip the pending
state while zeroed data is still in flight. The dirty check then avoids
setting the bit when the data has already been written back.

Suggested-by: Jan Kara <jack@suse.cz>
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/inode.c | 106 ++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 94 insertions(+), 12 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 9b6916609000c..3bdb5f3b119ab 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4847,6 +4847,89 @@ static int ext4_block_zero_range(struct inode *inode,
 					zero_written);
 }
 
+/*
+ * Inodes using the iomap buffered I/O path do not use data=ordered mode.
+ * Therefore, we mark the inode as disksize-grow-pending after zeroing the
+ * EOF block. The zeroed block will be submitted before any subsequent
+ * data.
+ *
+ * In the I/O completion path, ext4_iomap_wb_disksize_pending_wait() will
+ * wait for I/O completion before advancing i_disksize if the write
+ * extends beyond the zeroed boundary.
+ *
+ * When zeroed I/O is in progress, operations that extend i_disksize are
+ * handled as follows:
+ *
+ *  - Truncate up, append fallocate and zero_range:
+ *    Defer the update. The file size will be updated to i_size by the
+ *    end_io handler once the ongoing pending I/O completes.
+ *
+ *  - Insert range and collapse range operations:
+ *    Wait synchronously for the relevant I/O to complete before updating
+ *    i_disksize.
+ */
+static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
+{
+	struct folio *folio;
+
+	folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
+	if (IS_ERR(folio))
+		/* Already in writeback and cleared? */
+		return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
+
+	/*
+	 * Ensure that in-flight writeback, possibly started after
+	 * iomap_zero_range() unlocked the folio, has completed. Without
+	 * this wait folio_test_dirty() below may miss the zeroed data
+	 * (writeback clears PG_dirty), causing us to skip the
+	 * disksize-grow-pending tracking and potentially expose stale
+	 * on-disk data.
+	 */
+	folio_wait_writeback(folio);
+	WARN_ON_ONCE(folio_test_writeback(folio));
+
+	/*
+	 * If the zeroed range does not overlap the existing on-disk tail
+	 * block, the zeroed data lies beyond the currently on-disk data
+	 * and will be written back before i_disksize is advanced past it,
+	 * so no stale data can be exposed.
+	 *
+	 * Checking i_disksize here (after folio_wait_writeback()) is
+	 * necessary and safe. If a delalloc writeback of this folio was
+	 * in-flight, it could be raced by a concurrent mmap write which
+	 * corrupts the tail block beyond i_size but the i_disksize is not
+	 * advanced. folio_wait_writeback() has waited for its completion
+	 * and the ioend has advanced i_disksize accordingly. If writeback
+	 * had not started, we don't need to mark any pending state because
+	 * any future writeback will carry the pagecache content that now
+	 * includes the zeroed data, so no stale data can appear on disk
+	 * even without the pending tracking.
+	 */
+	if (from >= round_up(READ_ONCE(EXT4_I(inode)->i_disksize),
+			     i_blocksize(inode)))
+		goto out;
+
+	/*
+	 * Mark the inode as disksize-grow-pending. The zeroed block will
+	 * be written out by the generic writepages cycle or any other
+	 * syncing operation.
+	 *
+	 * Multiple overlapping unaligned EOF writes should not happen,
+	 * because we only mark the pending state after zeroing the on-disk
+	 * EOF block, and i_disksize can only be updated after the previous
+	 * zeroed pending block has been written back or the dirty folio
+	 * has been discarded.
+	 */
+	if (likely(folio_test_dirty(folio) &&
+		   !ext4_test_inode_state(inode,
+					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
+		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
+out:
+	folio_unlock(folio);
+	folio_put(folio);
+	return 0;
+}
+
 /*
  * Submit and wait for the pending zeroed EOF block range to complete
  * if the given range [@offset, @end) fully covers it.  Must be called
@@ -4923,21 +5006,20 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
 	 * truncating up or performing an append write, because there might be
 	 * exposing stale on-disk data which may caused by concurrent post-EOF
 	 * mmap write during folio writeback.
-	 *
-	 * TODO: In the iomap path, handle this by tracking the ordered range
-	 * and updating i_disksize to i_size after the zeroed data has been
-	 * written back.
 	 */
-	if (ext4_should_order_data(inode) &&
-	    did_zero && zero_written && !IS_DAX(inode)) {
-		handle_t *handle;
+	if (did_zero && zero_written && !IS_DAX(inode)) {
+		if (ext4_should_order_data(inode)) {
+			handle_t *handle;
 
-		handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
-		if (IS_ERR(handle))
-			return PTR_ERR(handle);
+			handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
+			if (IS_ERR(handle))
+				return PTR_ERR(handle);
 
-		err = ext4_jbd2_inode_add_write(handle, inode, from, length);
-		ext4_journal_stop(handle);
+			err = ext4_jbd2_inode_add_write(handle, inode, from,
+							length);
+			ext4_journal_stop(handle);
+		} else if (ext4_inode_buffered_iomap(inode))
+			err = ext4_iomap_mark_disksize_pending(inode, from);
 		if (err)
 			return err;
 	}
-- 
2.52.0

[PATCH v7 28/31] ext4: add tracepoints for DISKSIZE_GROW_PENDING set, clear, and wait

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Add trace events ext4_iomap_mark_disksize_pending(),
ext4_iomap_clear_disksize_pending(), and
ext4_iomap_wait_disksize_pending() to track disksize-grow-pending
state changes and waiting.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/inode.c             | 15 ++++++++++++---
 include/trace/events/ext4.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 3 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 3bdb5f3b119ab..920760ab6ea7b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -133,9 +133,12 @@ void ext4_inode_csum_set(struct inode *inode, struct ext4_inode *raw,
  */
 void ext4_iomap_clear_disksize_pending(struct inode *inode)
 {
-	clear_and_wake_up_bit(
+	if (!test_and_clear_wake_up_bit(
 		ext4_inode_state_wait_bit(EXT4_STATE_DISKSIZE_GROW_PENDING),
-		ext4_inode_state_wait_word(inode));
+		ext4_inode_state_wait_word(inode)))
+		return;
+
+	trace_ext4_iomap_clear_disksize_pending(inode);
 }
 
 /*
@@ -144,6 +147,10 @@ void ext4_iomap_clear_disksize_pending(struct inode *inode)
  */
 void ext4_iomap_wait_disksize_pending(struct inode *inode)
 {
+	/* Only emit the trace when the bit is actually set */
+	if (ext4_test_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING))
+		trace_ext4_iomap_wait_disksize_pending(inode);
+
 	wait_on_bit(ext4_inode_state_wait_word(inode),
 		    ext4_inode_state_wait_bit(EXT4_STATE_DISKSIZE_GROW_PENDING),
 		    TASK_UNINTERRUPTIBLE);
@@ -4922,8 +4929,10 @@ static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
 	 */
 	if (likely(folio_test_dirty(folio) &&
 		   !ext4_test_inode_state(inode,
-					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
+					  EXT4_STATE_DISKSIZE_GROW_PENDING))) {
+		trace_ext4_iomap_mark_disksize_pending(inode);
 		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
+	}
 out:
 	folio_unlock(folio);
 	folio_put(folio);
diff --git a/include/trace/events/ext4.h b/include/trace/events/ext4.h
index bcf80f518e61d..d33456d06b001 100644
--- a/include/trace/events/ext4.h
+++ b/include/trace/events/ext4.h
@@ -3204,6 +3204,39 @@ DEFINE_SET_IOMAP_EVENT(ext4_iomap_buffered_write_begin);
 DEFINE_SET_IOMAP_EVENT(ext4_iomap_map_writeback_range);
 DEFINE_SET_IOMAP_EVENT(ext4_iomap_zero_begin);
 
+DECLARE_EVENT_CLASS(ext4_iomap_disksize_pending,
+	TP_PROTO(struct inode *inode),
+	TP_ARGS(inode),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(loff_t, i_disksize)
+	),
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->i_disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+	),
+	TP_printk("dev %d:%d ino %llu i_disksize %lld",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->i_disksize)
+);
+
+DEFINE_EVENT(ext4_iomap_disksize_pending, ext4_iomap_mark_disksize_pending,
+	TP_PROTO(struct inode *inode),
+	TP_ARGS(inode)
+);
+
+DEFINE_EVENT(ext4_iomap_disksize_pending, ext4_iomap_clear_disksize_pending,
+	TP_PROTO(struct inode *inode),
+	TP_ARGS(inode)
+);
+
+DEFINE_EVENT(ext4_iomap_disksize_pending, ext4_iomap_wait_disksize_pending,
+	TP_PROTO(struct inode *inode),
+	TP_ARGS(inode)
+);
+
 #endif /* _TRACE_EXT4_H */
 
 /* This part must be outside protection */
-- 
2.52.0

[PATCH v7 29/31] ext4: add tracepoints for EOF block zeroing and disksize-grow I/O

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Add tracepoints to track the disksize-grow-pending lifecycle in the
writeback path and the block-zero-EOF entry point:

- ext4_iomap_wb_disksize_pending_submit: ioend is marked as
  DISKSIZE_GROW_IO in ext4_iomap_writeback_submit().
- ext4_iomap_wb_disksize_pending_complete: ioend of type
  DISKSIZE_GROW_IO completes in ext4_iomap_finish_ioend().
- ext4_iomap_wb_disksize_pending_wait: ioend worker waits for the
  pending zeroed EOF block to complete.
- ext4_iomap_wb_update_disksize: i_disksize is advanced in
  ext4_iomap_wb_update_disksize, including the new value and
  whether the update is a disksize-grow completion.
- ext4_block_zero_eof: ext4_block_zero_eof is called with the range
  and zeroing outcome, capturing the producer-side entry point.

Together with the previous mark/clear/wait tracepoints, these cover the
full lifetime of the disksize-grow-pending state.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/inode.c             |   3 +
 fs/ext4/page-io.c           |  11 ++-
 include/trace/events/ext4.h | 129 ++++++++++++++++++++++++++++++++++++
 3 files changed, 142 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 920760ab6ea7b..c3a6f3dce3d86 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4417,6 +4417,8 @@ static int ext4_iomap_writeback_submit(struct iomap_writepage_ctx *wpc,
 	    round_down(ioend->io_offset, blocksize) <= pstart &&
 	    round_up(ioend->io_offset + ioend->io_size, blocksize) >=
 			pstart + plen) {
+		trace_ext4_iomap_wb_disksize_pending_submit(inode,
+				ioend->io_offset, ioend->io_size);
 		ioend->io_bio.bi_end_io = ext4_iomap_end_bio;
 		ioend->io_private = (void *)EXT4_IOMAP_IOEND_DISKSIZE_GROW_IO;
 	}
@@ -5033,6 +5035,7 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
 			return err;
 	}
 
+	trace_ext4_block_zero_eof(inode, from, length, did_zero, zero_written);
 	return 0;
 }
 
diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c
index 4f1176b9332f2..734d856a6d478 100644
--- a/fs/ext4/page-io.c
+++ b/fs/ext4/page-io.c
@@ -31,6 +31,8 @@
 #include "xattr.h"
 #include "acl.h"
 
+#include <trace/events/ext4.h>
+
 static struct kmem_cache *io_end_cachep;
 static struct kmem_cache *io_end_vec_cachep;
 
@@ -574,6 +576,7 @@ static void ext4_iomap_wb_disksize_pending_wait(struct inode *inode,
 	if (!plen || pos < pstart + plen)
 		return;
 
+	trace_ext4_iomap_wb_disksize_pending_wait(inode, pos, size);
 	ext4_iomap_wait_disksize_pending(inode);
 }
 
@@ -617,8 +620,11 @@ static int ext4_iomap_wb_update_disksize(handle_t *handle, struct inode *inode,
 	 * after the data has been persisted.
 	 */
 	new_disksize = is_disksize_grow ? i_size : min(end, i_size);
-	if (new_disksize > ei->i_disksize)
+	if (new_disksize > ei->i_disksize) {
+		trace_ext4_iomap_wb_update_disksize(inode, end, i_size,
+				ei->i_disksize, new_disksize, is_disksize_grow);
 		WRITE_ONCE(ei->i_disksize, new_disksize);
+	}
 	up_write(&ei->i_data_sem);
 	ret = ext4_mark_inode_dirty(handle, inode);
 	if (ret)
@@ -641,6 +647,9 @@ static void ext4_iomap_finish_ioend(struct iomap_ioend *ioend)
 	int ret, err;
 
 	ret = blk_status_to_errno(ioend->io_bio.bi_status);
+	if (is_disksize_grow)
+		trace_ext4_iomap_wb_disksize_pending_complete(ioend->io_inode,
+				ioend->io_offset, ioend->io_size, ret);
 	if (unlikely(ret)) {
 		if (test_opt(sb, DATA_ERR_ABORT) && !ext4_emergency_state(sb))
 			jbd2_journal_abort(EXT4_SB(sb)->s_journal, ret);
diff --git a/include/trace/events/ext4.h b/include/trace/events/ext4.h
index d33456d06b001..92048da9dae43 100644
--- a/include/trace/events/ext4.h
+++ b/include/trace/events/ext4.h
@@ -3237,6 +3237,135 @@ DEFINE_EVENT(ext4_iomap_disksize_pending, ext4_iomap_wait_disksize_pending,
 	TP_ARGS(inode)
 );
 
+/* disksize pending I/O tracepoints for iomap Buffered I/O path */
+DECLARE_EVENT_CLASS(ext4_iomap_wb_disksize_pending,
+	TP_PROTO(struct inode *inode, loff_t io_offset, size_t io_size),
+	TP_ARGS(inode, io_offset, io_size),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(loff_t, io_offset)
+		__field(size_t, io_size)
+		__field(loff_t, i_size)
+		__field(loff_t, i_disksize)
+	),
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->io_offset = io_offset;
+		__entry->io_size = io_size;
+		__entry->i_size = i_size_read(inode);
+		__entry->i_disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+	),
+	TP_printk("dev %d:%d ino %llu io_offset %lld io_size %zu i_size %lld i_disksize %lld",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->io_offset, __entry->io_size,
+		  __entry->i_size, __entry->i_disksize)
+);
+
+DEFINE_EVENT(ext4_iomap_wb_disksize_pending,
+	ext4_iomap_wb_disksize_pending_submit,
+	TP_PROTO(struct inode *inode, loff_t io_offset, size_t io_size),
+	TP_ARGS(inode, io_offset, io_size)
+);
+
+DEFINE_EVENT(ext4_iomap_wb_disksize_pending,
+	ext4_iomap_wb_disksize_pending_wait,
+	TP_PROTO(struct inode *inode, loff_t io_offset, size_t io_size),
+	TP_ARGS(inode, io_offset, io_size)
+);
+
+TRACE_EVENT(ext4_iomap_wb_disksize_pending_complete,
+	TP_PROTO(struct inode *inode, loff_t io_offset, size_t io_size,
+		 int ret),
+	TP_ARGS(inode, io_offset, io_size, ret),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(loff_t, io_offset)
+		__field(size_t, io_size)
+		__field(loff_t, i_size)
+		__field(loff_t, i_disksize)
+		__field(int, ret)
+	),
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->io_offset = io_offset;
+		__entry->io_size = io_size;
+		__entry->i_size = i_size_read(inode);
+		__entry->i_disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+		__entry->ret = ret;
+	),
+	TP_printk("dev %d:%d ino %llu io_offset %lld io_size %zu ret %d i_size %lld i_disksize %lld",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->io_offset, __entry->io_size,
+		  __entry->ret, __entry->i_size, __entry->i_disksize)
+);
+
+/* i_disksize update tracepoint */
+TRACE_EVENT(ext4_iomap_wb_update_disksize,
+	TP_PROTO(struct inode *inode, loff_t end, loff_t i_size,
+		 loff_t i_disksize, loff_t new_disksize,
+		 bool is_disksize_grow),
+	TP_ARGS(inode, end, i_size, i_disksize, new_disksize,
+		is_disksize_grow),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(loff_t, end)
+		__field(loff_t, i_size)
+		__field(loff_t, i_disksize)
+		__field(loff_t, new_disksize)
+		__field(bool, is_disksize_grow)
+	),
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->end = end;
+		__entry->i_size = i_size;
+		__entry->i_disksize = i_disksize;
+		__entry->new_disksize = new_disksize;
+		__entry->is_disksize_grow = is_disksize_grow;
+	),
+	TP_printk("dev %d:%d ino %llu end %lld i_size %lld i_disksize %lld new_disksize %lld is_disksize_grow %d",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->end, __entry->i_size,
+		  __entry->i_disksize, __entry->new_disksize,
+		  __entry->is_disksize_grow)
+);
+
+/* Block zero EOF tracepoint */
+TRACE_EVENT(ext4_block_zero_eof,
+	TP_PROTO(struct inode *inode, loff_t from, loff_t length,
+		 bool did_zero, bool zero_written),
+	TP_ARGS(inode, from, length, did_zero, zero_written),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(u64, ino)
+		__field(loff_t, from)
+		__field(loff_t, length)
+		__field(loff_t, i_size)
+		__field(loff_t, i_disksize)
+		__field(bool, did_zero)
+		__field(bool, zero_written)
+	),
+	TP_fast_assign(
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->ino = inode->i_ino;
+		__entry->from = from;
+		__entry->length = length;
+		__entry->i_size = inode->i_size;
+		__entry->i_disksize = READ_ONCE(EXT4_I(inode)->i_disksize);
+		__entry->did_zero = did_zero;
+		__entry->zero_written = zero_written;
+	),
+	TP_printk("dev %d:%d ino %llu zero EOF from %lld length %lld i_size %lld i_disksize %lld did_zero %d zero_written %d",
+		  MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino,
+		  __entry->from, __entry->length, __entry->i_size,
+		  __entry->i_disksize, __entry->did_zero, __entry->zero_written)
+);
+
 #endif /* _TRACE_EXT4_H */
 
 /* This part must be outside protection */
-- 
2.52.0

[PATCH v7 30/31] ext4: partially enable iomap for the buffered I/O path of regular files

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Introduce ext4_enable_buffered_iomap() to determine whether a regular
file inode should use the iomap buffered I/O path. We now support the
default filesystem features, mount options, and the bigalloc feature.
However, inline data, fsverity, fscrypt, indirect inode type, and
data=journal mode are not fully supported.

The decision is made at inode initialization time in __ext4_new_inode()
and __ext4_iget() by setting the EXT4_STATE_BUFFERED_IOMAP state flag.
If any of these unsupported features are met, the inode silently falls
back to the traditional buffer_head path. Switching the buffered I/O
path on an active inode is not supported, with the exception of changing
a per-inode journal flag.

For features like encryption, verity, and inline data that can be
dynamically enabled at the superblock level, checking the global feature
flag avoids the complexity of toggling the path on individual inodes.

Additionally:

 - Extend ext4_inode_journal_mode() to force ordered mode for inodes
   using the iomap path under a data=journal mount. For the global data
   journal mode (EXT4_MOUNT_JOURNAL_DATA), dynamic enablement is
   deferred until the next inode re-initialization. For the per-inode
   data journal mode (EXT4_INODE_JOURNAL_DATA), dynamic changes take
   effect immediately, as it is safe to switch address_space operations
   and drop all page cache under i_rwsem and filemap_invalidate_lock.

 - Add a WARN_ON_ONCE() guard in _ext4_get_block() to catch inodes
   using the iomap path from accidentally entering the legacy
   buffer_head writeback path.

 - Place silent BUFFERED_IOMAP checks in ext4_do_writepages() and
   ext4_iomap_writepages() under the writepages rwsem. Unlike
   _ext4_get_block(), the writeback path does not hold i_rwsem or
   invalidate_lock and can race with ext4_change_inode_journal_flag()
   switching the inode's a_ops.

 - Reject extent-to-indirect migration via ext4_ind_migrate() for inodes
   on the iomap path.

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h      |   1 +
 fs/ext4/ext4_jbd2.c |   8 +++-
 fs/ext4/ialloc.c    |   1 +
 fs/ext4/inode.c     | 105 +++++++++++++++++++++++++++++++++++++++++++-
 fs/ext4/migrate.c   |   2 +
 5 files changed, 113 insertions(+), 4 deletions(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 504dce9fdc6b9..405e256a1802b 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3170,6 +3170,7 @@ int ext4_walk_page_buffers(handle_t *handle,
 int do_journal_get_write_access(handle_t *handle, struct inode *inode,
 				struct buffer_head *bh);
 void ext4_set_inode_mapping_order(struct inode *inode);
+void ext4_enable_buffered_iomap(struct inode *inode);
 int ext4_nonda_switch(struct super_block *sb);
 #define FALL_BACK_TO_NONDELALLOC 1
 #define EXT4_WRITE_DATA_INLINE	 2
diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c
index 53ddedb52a6f7..a4664ddecdcd6 100644
--- a/fs/ext4/ext4_jbd2.c
+++ b/fs/ext4/ext4_jbd2.c
@@ -17,8 +17,12 @@ int ext4_inode_journal_mode(struct inode *inode)
 	    test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA ||
 	    (ext4_test_inode_flag(inode, EXT4_INODE_JOURNAL_DATA) &&
 	    !test_opt(inode->i_sb, DELALLOC))) {
-		/* We do not support data journalling for encrypted data */
-		if (S_ISREG(inode->i_mode) && IS_ENCRYPTED(inode))
+		/*
+		 * We do not support data journalling for encrypted data
+		 * and buffered IOMAP path.
+		 */
+		if (S_ISREG(inode->i_mode) &&
+		    (IS_ENCRYPTED(inode) || ext4_inode_buffered_iomap(inode)))
 			return EXT4_INODE_ORDERED_DATA_MODE;  /* ordered */
 		return EXT4_INODE_JOURNAL_DATA_MODE;	/* journal data */
 	}
diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c
index a5831fc536db9..f97a2f4904ebb 100644
--- a/fs/ext4/ialloc.c
+++ b/fs/ext4/ialloc.c
@@ -1346,6 +1346,7 @@ struct inode *__ext4_new_inode(struct mnt_idmap *idmap,
 		}
 	}
 
+	ext4_enable_buffered_iomap(inode);
 	ext4_set_inode_mapping_order(inode);
 
 	ext4_update_inode_fsync_trans(handle, inode, 1);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index c3a6f3dce3d86..a07ce368e6e82 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -1039,6 +1039,9 @@ static int _ext4_get_block(struct inode *inode, sector_t iblock,
 
 	if (ext4_has_inline_data(inode))
 		return -ERANGE;
+	/* inode using the iomap buffered I/O path should not go here. */
+	if (WARN_ON_ONCE(ext4_inode_buffered_iomap(inode)))
+		return -EINVAL;
 
 	map.m_lblk = iblock;
 	map.m_len = bh->b_size >> inode->i_blkbits;
@@ -2915,6 +2918,13 @@ static int ext4_do_writepages(struct mpage_da_data *mpd)
 	if (!mapping->nrpages || !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
 		goto out_writepages;
 
+	/*
+	 * Does ext4_change_inode_journal_flag() change the inode's
+	 * buffered I/O path?
+	 */
+	if (ext4_inode_buffered_iomap(inode))
+		goto out_writepages;
+
 	/*
 	 * If the filesystem has aborted, it is read-only, so return
 	 * right away instead of dumping stack traces later on that
@@ -4053,6 +4063,9 @@ static int ext4_iomap_map_blocks(struct inode *inode, loff_t offset,
 {
 	u8 blkbits = inode->i_blkbits;
 
+	/* inode using the buffer_head buffered I/O path should not go here. */
+	if (WARN_ON_ONCE(!ext4_inode_buffered_iomap(inode)))
+		return -EINVAL;
 	if ((offset >> blkbits) > EXT4_MAX_LOGICAL_BLOCK)
 		return -EINVAL;
 
@@ -4496,6 +4509,15 @@ static int ext4_iomap_writepages(struct address_space *mapping,
 	ext4_iomap_wb_submit_zeroed_eof(inode, wbc);
 
 	alloc_ctx = ext4_writepages_down_read(sb);
+	/*
+	 * Does ext4_change_inode_journal_flag() change the inode's
+	 * buffered I/O path?
+	 */
+	if (!ext4_inode_buffered_iomap(inode)) {
+		ext4_writepages_up_read(sb, alloc_ctx);
+		return ret;
+	}
+
 	trace_ext4_writepages(inode, wbc);
 	ret = iomap_writepages(&wpc);
 	trace_ext4_writepages_result(inode, wbc, ret, nr - wbc->nr_to_write);
@@ -6076,6 +6098,81 @@ static int check_igot_inode(struct inode *inode, ext4_iget_flags flags,
 	return -EFSCORRUPTED;
 }
 
+/*
+ * Determine whether an inode should use the iomap buffered I/O path.
+ * EXT4_STATE_BUFFERED_IOMAP is generally set at inode initialization
+ * time. Online switching of the buffered I/O path on an active inode is
+ * NOT supported, with the exception of changing a per-inode journal
+ * flag.
+ *
+ * For features like inline data, fsverity, and encryption that can be
+ * dynamically enabled or disabled, we check the superblock-level
+ * feature flags. If any of these is globally enabled, no inode is
+ * allowed into the iomap buffered I/O path. This avoids the complexity
+ * of dynamic toggling.
+ *
+ * For the global data journal mode (EXT4_MOUNT_JOURNAL_DATA), dynamic
+ * change through remount is deferred. It will only become available
+ * after the inode is re-initialized (i.e., after the last reference
+ * drops and the inode is re-read from disk with the journal flag
+ * cleared).
+ *
+ * For the per-inode data journal mode (EXT4_INODE_JOURNAL_DATA),
+ * dynamic changes take effect immediately. This is safe because
+ * address_space operations can be switched and all page cache can be
+ * dropped under i_rwsem and filemap_invalidate_lock.
+ *
+ * For extent-to-indirect block migration (via EXT4_IOC_SETFLAGS
+ * clearing EXT4_EXTENTS_FL), this operation is directly rejected for
+ * inodes using the iomap path.
+ */
+void ext4_enable_buffered_iomap(struct inode *inode)
+{
+	struct super_block *sb = inode->i_sb;
+
+	if (!S_ISREG(inode->i_mode))
+		return;
+	if (ext4_test_inode_flag(inode, EXT4_INODE_EA_INODE))
+		return;
+
+	/* Unsupported Features */
+	if (ext4_has_feature_inline_data(sb))
+		return;
+	if (ext4_has_feature_verity(sb))
+		return;
+	if (ext4_has_feature_encrypt(sb))
+		return;
+	if (test_opt(sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA ||
+	    ext4_test_inode_flag(inode, EXT4_INODE_JOURNAL_DATA))
+		return;
+	if (!(ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)))
+		return;
+
+	ext4_set_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP);
+
+	/*
+	 * Install the iomap end_io handler on the shared conversion
+	 * work.  This is safe at inode initialization and during the
+	 * buffered I/O path changes where we flush all pending
+	 * writebacks and drop page cache under i_rwsem and
+	 * filemap_invalidate_lock.
+	 */
+	INIT_WORK(&EXT4_I(inode)->i_rsv_conversion_work, ext4_iomap_end_io);
+}
+
+static void ext4_disable_buffered_iomap(struct inode *inode)
+{
+	ext4_clear_inode_state(inode, EXT4_STATE_BUFFERED_IOMAP);
+
+	/*
+	 * Reinstall the buffer_head end_io handler on the shared
+	 * conversion work.  This is safe during the buffered I/O path
+	 * changes where we flush all pending writebacks and drop page
+	 * cache under i_rwsem and filemap_invalidate_lock.
+	 */
+	INIT_WORK(&EXT4_I(inode)->i_rsv_conversion_work, ext4_end_io_rsv_work);
+}
+
 void ext4_set_inode_mapping_order(struct inode *inode)
 {
 	struct super_block *sb = inode->i_sb;
@@ -6390,6 +6487,8 @@ struct inode *__ext4_iget(struct super_block *sb, unsigned long ino,
 	if (ret)
 		goto bad_inode;
 
+	ext4_enable_buffered_iomap(inode);
+
 	if (S_ISREG(inode->i_mode)) {
 		inode->i_op = &ext4_file_inode_operations;
 		inode->i_fop = &ext4_file_operations;
@@ -7620,9 +7719,10 @@ int ext4_change_inode_journal_flag(struct inode *inode, int val)
 	 * the inode's in-core data-journaling state flag now.
 	 */
 
-	if (val)
+	if (val) {
 		ext4_set_inode_flag(inode, EXT4_INODE_JOURNAL_DATA);
-	else {
+		ext4_disable_buffered_iomap(inode);
+	} else {
 		err = jbd2_journal_flush(journal, 0);
 		if (err < 0) {
 			jbd2_journal_unlock_updates(journal);
@@ -7631,6 +7731,7 @@ int ext4_change_inode_journal_flag(struct inode *inode, int val)
 			return err;
 		}
 		ext4_clear_inode_flag(inode, EXT4_INODE_JOURNAL_DATA);
+		ext4_enable_buffered_iomap(inode);
 	}
 	ext4_set_aops(inode);
 	ext4_set_inode_mapping_order(inode);
diff --git a/fs/ext4/migrate.c b/fs/ext4/migrate.c
index 5d60ef10fe110..09931d3ba2c6a 100644
--- a/fs/ext4/migrate.c
+++ b/fs/ext4/migrate.c
@@ -621,6 +621,8 @@ int ext4_ind_migrate(struct inode *inode)
 
 	if (ext4_has_feature_bigalloc(inode->i_sb))
 		return -EOPNOTSUPP;
+	if (ext4_inode_buffered_iomap(inode))
+		return -EOPNOTSUPP;
 
 	/*
 	 * In order to get correct extent info, force all delayed allocation
-- 
2.52.0

[PATCH v7 31/31] ext4: introduce a mount option for iomap buffered I/O path

[Zhang Yi]

From: Zhang Yi <yi.zhang@huawei.com>

Since the iomap buffered I/O path does not yet support all existing ext4
features, it cannot be enabled by default. Introduce the
'buffered_iomap' and 'nobuffered_iomap' mount options to explicitly
enable or disable the iomap buffered I/O path for regular files.

Toggling this option via remount is allowed. The change of I/O path will
not take effect immediately. It will be deferred. The new setting will
only take effect after the inode is re-initialized (i.e., after the last
reference is dropped and the inode is re-read from disk).

Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
---
 fs/ext4/ext4.h  | 1 +
 fs/ext4/inode.c | 6 ++++++
 fs/ext4/super.c | 7 +++++++
 3 files changed, 14 insertions(+)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index 405e256a1802b..8cfe77b45e21b 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -1321,6 +1321,7 @@ struct ext4_inode_info {
 						    * scanning in mballoc
 						    */
 #define EXT4_MOUNT2_ABORT		0x00000100 /* Abort filesystem */
+#define EXT4_MOUNT2_BUFFERED_IOMAP	0x00000200 /* Use iomap for buffered I/O */
 
 #define clear_opt(sb, opt)		EXT4_SB(sb)->s_mount_opt &= \
 						~EXT4_MOUNT_##opt
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index a07ce368e6e82..1853fbdc2a5af 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -6125,11 +6125,17 @@ static int check_igot_inode(struct inode *inode, ext4_iget_flags flags,
  * For extent-to-indirect block migration (via EXT4_IOC_SETFLAGS
  * clearing EXT4_EXTENTS_FL), this operation is directly rejected for
  * inodes using the iomap path.
+ *
+ * When remounting to toggle the buffered_iomap mount option, the change
+ * of I/O path is deferred as well, it will be available after the inode
+ * is re-initialized.
  */
 void ext4_enable_buffered_iomap(struct inode *inode)
 {
 	struct super_block *sb = inode->i_sb;
 
+	if (!test_opt2(sb, BUFFERED_IOMAP))
+		return;
 	if (!S_ISREG(inode->i_mode))
 		return;
 	if (ext4_test_inode_flag(inode, EXT4_INODE_EA_INODE))
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 73735fd336b91..7db34ba0ea593 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1747,6 +1747,7 @@ enum {
 	Opt_discard, Opt_nodiscard, Opt_init_itable, Opt_noinit_itable,
 	Opt_max_dir_size_kb, Opt_nojournal_checksum, Opt_nombcache,
 	Opt_no_prefetch_block_bitmaps, Opt_mb_optimize_scan,
+	Opt_buffered_iomap, Opt_nobuffered_iomap,
 	Opt_errors, Opt_data, Opt_data_err, Opt_jqfmt, Opt_dax_type,
 #ifdef CONFIG_EXT4_DEBUG
 	Opt_fc_debug_max_replay, Opt_fc_debug_force
@@ -1885,6 +1886,8 @@ static const struct fs_parameter_spec ext4_param_specs[] = {
 	fsparam_flag	("no_prefetch_block_bitmaps",
 						Opt_no_prefetch_block_bitmaps),
 	fsparam_s32	("mb_optimize_scan",	Opt_mb_optimize_scan),
+	fsparam_flag	("buffered_iomap",	Opt_buffered_iomap),
+	fsparam_flag	("nobuffered_iomap",	Opt_nobuffered_iomap),
 	fsparam_string	("check",		Opt_removed),	/* mount option from ext2/3 */
 	fsparam_flag	("nocheck",		Opt_removed),	/* mount option from ext2/3 */
 	fsparam_flag	("reservation",		Opt_removed),	/* mount option from ext2/3 */
@@ -1978,6 +1981,10 @@ static const struct mount_opts {
 	{Opt_nombcache, EXT4_MOUNT_NO_MBCACHE, MOPT_SET},
 	{Opt_no_prefetch_block_bitmaps, EXT4_MOUNT_NO_PREFETCH_BLOCK_BITMAPS,
 	 MOPT_SET},
+	{Opt_buffered_iomap, EXT4_MOUNT2_BUFFERED_IOMAP,
+	 MOPT_SET | MOPT_2 | MOPT_EXT4_ONLY},
+	{Opt_nobuffered_iomap, EXT4_MOUNT2_BUFFERED_IOMAP,
+	 MOPT_CLEAR | MOPT_2 | MOPT_EXT4_ONLY},
 #ifdef CONFIG_EXT4_DEBUG
 	{Opt_fc_debug_force, EXT4_MOUNT2_JOURNAL_FAST_COMMIT,
 	 MOPT_SET | MOPT_2 | MOPT_EXT4_ONLY},
-- 
2.52.0

[syzbot ci] Re: ext4: use iomap for regular file's buffered I/O path

[syzbot ci]

syzbot ci has tested the following series

[v7] ext4: use iomap for regular file's buffered I/O path
https://lore.kernel.org/all/20261009103033.2920530-1-yi.zhang@huaweicloud.com
* [PATCH v7 01/31] ext4: simplify size updating in ext4_setattr()
* [PATCH v7 02/31] ext4: factor out ext4_truncate_[up|down]()
* [PATCH v7 03/31] ext4: set EXT4_MAP_NEW flag for delayed allocated blocks
* [PATCH v7 04/31] ext4: recheck extent status tree before block allocation
* [PATCH v7 05/31] ext4: fix orig_mlen initialization in ext4_map_blocks()
* [PATCH v7 06/31] ext4: allow ext4_map_blocks() to start its own transaction handle
* [PATCH v7 07/31] ext4: avoid unnecessary transaction in ext4_map_blocks() for unwritten extents
* [PATCH v7 08/31] ext4: skip block allocation for holes in the data submission path
* [PATCH v7 09/31] ext4: add iomap address space operations for buffered I/O
* [PATCH v7 10/31] ext4: implement buffered read path using iomap
* [PATCH v7 11/31] ext4: pass out extent seq counter when mapping da blocks
* [PATCH v7 12/31] ext4: do not use data=ordered mode for inodes using buffered iomap path
* [PATCH v7 13/31] ext4: implement buffered write path using iomap
* [PATCH v7 14/31] ext4: rework handle credit accounting for unwritten extent conversion
* [PATCH v7 15/31] ext4: implement writeback path using iomap
* [PATCH v7 16/31] ext4: implement mmap path using iomap
* [PATCH v7 17/31] ext4: implement partial block zero range path using iomap
* [PATCH v7 18/31] ext4: drain writeback before removing extents on the iomap path
* [PATCH v7 19/31] ext4: add block mapping tracepoints for iomap buffered I/O path
* [PATCH v7 20/31] ext4: disable online defrag when inode using iomap buffered I/O path
* [PATCH v7 21/31] ext4: add EXT4_STATE_DISKSIZE_GROW_PENDING state bit and helpers
* [PATCH v7 22/31] ext4: submit and wait for pending disksize-grow I/O on writeback
* [PATCH v7 23/31] ext4: advance i_disksize to i_size upon disksize-grow I/O completion
* [PATCH v7 24/31] ext4: defer i_disksize update while DISKSIZE_GROW_PENDING is set
* [PATCH v7 25/31] ext4: submit and wait for disksize-grow I/O in fallocate paths
* [PATCH v7 26/31] ext4: clear DISKSIZE_GROW_PENDING on truncate or error
* [PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block
* [PATCH v7 28/31] ext4: add tracepoints for DISKSIZE_GROW_PENDING set, clear, and wait
* [PATCH v7 29/31] ext4: add tracepoints for EOF block zeroing and disksize-grow I/O
* [PATCH v7 30/31] ext4: partially enable iomap for the buffered I/O path of regular files
* [PATCH v7 31/31] ext4: introduce a mount option for iomap buffered I/O path

and found the following issue:
WARNING in ext4_do_writepages

Full report is available here:
https://ci.syzbot.org/series/3711c951-acf4-4ebe-a920-9cc89b6be2ff

***

WARNING in ext4_do_writepages

tree:      vfs
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/vfs/vfs.git
base:      3d399224425573875b6f6f1181bd8cf9a28cc7d1
arch:      amd64
compiler:  Debian clang version 22.1.8 (++20260613092233+e80beda6e255-1~exp1~20260613092250.77), Debian LLD 22.1.8
config:    https://ci.syzbot.org/builds/c3c01f8c-846d-4c7a-955a-fd7427af6b0d/config

------------[ cut here ]------------
!err
WARNING: fs/ext4/inode.c:2512 at ext4_do_writepages+0x33b3/0x44a0, CPU#0: kworker/u9:3/147
Modules linked in:
CPU: 0 UID: 0 PID: 147 Comm: kworker/u9:3 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Workqueue: writeback wb_workfn (flush-7:1)
RIP: 0010:ext4_do_writepages+0x33b3/0x44a0
Code: 0f 0b 90 e9 b6 d9 ff ff e8 6a e2 36 ff 90 0f 0b 90 e9 e6 dc ff ff e8 5c e2 36 ff 90 0f 0b 90 e9 94 df ff ff e8 4e e2 36 ff 90 <0f> 0b 90 bb ea ff ff ff e9 ae f8 ff ff e8 3b e2 36 ff 48 8d 3d 64
RSP: 0018:ffffc90003106ce0 EFLAGS: 00010293
RAX: ffffffff8290e912 RBX: 0000000000000000 RCX: ffff8881053f9e00
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffffc900031070f0 R08: ffff888025da4a37 R09: 1ffff11004bb4946
R10: dffffc0000000000 R11: ffffed1004bb4947 R12: ffffc900031071a0
R13: dffffc0000000000 R14: 0000000000000000 R15: 1ffff11004bb495d
FS:  0000000000000000(0000) GS:ffff88818d6c9000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00005555565a5a28 CR3: 0000000031f66000 CR4: 00000000000006f0
Call Trace:
 <TASK>
 ext4_writepages+0x241/0x3b0
 do_writepages+0x338/0x560
 __writeback_single_inode+0x136/0x1220
 writeback_sb_inodes+0x9de/0x1b00
 wb_writeback+0x41c/0xad0
 wb_workfn+0x431/0x10f0
 process_scheduled_works+0xc3d/0x1630
 worker_thread+0xa47/0xfb0
 kthread+0x38b/0x480
 ret_from_fork+0x514/0xb70
 ret_from_fork_asm+0x1a/0x30
 </TASK>


***

If these findings have caused you to resend the series or submit a
separate fix, please add the following tag to your commit message:
  Tested-by: syzbot@syzkaller.appspotmail.com

---
This report is generated by a bot. It may contain errors.
syzbot ci engineers can be reached at syzkaller@googlegroups.com.

To test a fix for this bug, please reply with `#syz test`
(on a separate line) and attach the patch to the email.

Notes:
- The patch will be applied on top of the tested series (as an
  incremental fix).
- To test a new version of the whole series, please send it directly
  to syzbot@lists.linux.dev.
- Arguments like custom git repos and branches are not supported.

Re: [syzbot ci] Re: ext4: use iomap for regular file's buffered I/O path

[Zhang Yi]

On 10/10/2026 1:54 AM, syzbot ci wrote:
> syzbot ci has tested the following series
> 

[...]

> and found the following issue:
> WARNING in ext4_do_writepages
> 
> Full report is available here:
> https://ci.syzbot.org/series/3711c951-acf4-4ebe-a920-9cc89b6be2ff
> 
> ***
> 
> WARNING in ext4_do_writepages
> 
> tree:      vfs
> URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/vfs/vfs.git
> base:      3d399224425573875b6f6f1181bd8cf9a28cc7d1
> arch:      amd64
> compiler:  Debian clang version 22.1.8 (++20260613092233+e80beda6e255-1~exp1~20260613092250.77), Debian LLD 22.1.8
> config:    https://ci.syzbot.org/builds/c3c01f8c-846d-4c7a-955a-fd7427af6b0d/config
> 
> ------------[ cut here ]------------
> !err
> WARNING: fs/ext4/inode.c:2512 at ext4_do_writepages+0x33b3/0x44a0, CPU#0: kworker/u9:3/147
> Modules linked in:
> CPU: 0 UID: 0 PID: 147 Comm: kworker/u9:3 Not tainted syzkaller #0 PREEMPT(full) 
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
> Workqueue: writeback wb_workfn (flush-7:1)
> RIP: 0010:ext4_do_writepages+0x33b3/0x44a0
> Code: 0f 0b 90 e9 b6 d9 ff ff e8 6a e2 36 ff 90 0f 0b 90 e9 e6 dc ff ff e8 5c e2 36 ff 90 0f 0b 90 e9 94 df ff ff e8 4e e2 36 ff 90 <0f> 0b 90 bb ea ff ff ff e9 ae f8 ff ff e8 3b e2 36 ff 48 8d 3d 64
> RSP: 0018:ffffc90003106ce0 EFLAGS: 00010293
> RAX: ffffffff8290e912 RBX: 0000000000000000 RCX: ffff8881053f9e00
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> RBP: ffffc900031070f0 R08: ffff888025da4a37 R09: 1ffff11004bb4946
> R10: dffffc0000000000 R11: ffffed1004bb4947 R12: ffffc900031071a0
> R13: dffffc0000000000 R14: 0000000000000000 R15: 1ffff11004bb495d
> FS:  0000000000000000(0000) GS:ffff88818d6c9000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00005555565a5a28 CR3: 0000000031f66000 CR4: 00000000000006f0
> Call Trace:
>  <TASK>
>  ext4_writepages+0x241/0x3b0
>  do_writepages+0x338/0x560
>  __writeback_single_inode+0x136/0x1220
>  writeback_sb_inodes+0x9de/0x1b00
>  wb_writeback+0x41c/0xad0
>  wb_workfn+0x431/0x10f0
>  process_scheduled_works+0xc3d/0x1630
>  worker_thread+0xa47/0xfb0
>  kthread+0x38b/0x480
>  ret_from_fork+0x514/0xb70
>  ret_from_fork_asm+0x1a/0x30
>  </TASK>
> 
> 

This WARNING was newly introduced by patch 8, "ext4: skip block
allocation for holes in the data submission path". It doesn't actually
cause a new regression, it just appears to expose a pre-existing issue
in the buffer_head buffered I/O path, where an inconsistency between
the bh and extent state could potentially lead to allocating a block and
writing data into a hole. The logic of the patch itself is fine. To
avoid noise, I can drop the WARN_ON_ONCE() while still keeping return
-EINVAL. Thoughts?

Thanks,
Yi.

> ***
> 
> If these findings have caused you to resend the series or submit a
> separate fix, please add the following tag to your commit message:
>   Tested-by: syzbot@syzkaller.appspotmail.com
> 
> ---
> This report is generated by a bot. It may contain errors.
> syzbot ci engineers can be reached at syzkaller@googlegroups.com.
> 
> To test a fix for this bug, please reply with `#syz test`
> (on a separate line) and attach the patch to the email.
> 
> Notes:
> - The patch will be applied on top of the tested series (as an
>   incremental fix).
> - To test a new version of the whole series, please send it directly
>   to syzbot@lists.linux.dev.
> - Arguments like custom git repos and branches are not supported.

Re: [PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block

[Ojaswin Mujoo]

On Fri, Oct 09, 2026 at 06:30:28PM +0800, Zhang Yi wrote:
> From: Zhang Yi <yi.zhang@huawei.com>
> 
> In the iomap buffered I/O path, data=ordered mode is not used, so the
> zeroed EOF block has no implicit ordering with later i_disksize updates.
> Without the pending state being set, i_disksize can be advanced past the
> zeroed block before writeback completes, exposing stale data after a
> crash.
> 
> Previous patches added the consumer side of the
> disksize-grow-pending mechanism: the state bit, clear and wait helpers,
> and ioend tagging. Now add ext4_iomap_mark_disksize_pending() and call
> it from ext4_block_zero_eof() after zeroing the tail of the block that
> straddles i_disksize.
> 
> The helper locks the folio, waits for any in-flight writeback on it to
> complete, then sets EXT4_STATE_DISKSIZE_GROW_PENDING only if the folio
> is still dirty. Waiting for writeback prevents folio_test_dirty() from
> returning false mid-writeback, which would cause us to skip the pending
> state while zeroed data is still in flight. The dirty check then avoids
> setting the bit when the data has already been written back.
> 
> Suggested-by: Jan Kara <jack@suse.cz>
> Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
> ---
>  fs/ext4/inode.c | 106 ++++++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 94 insertions(+), 12 deletions(-)
> 
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 9b6916609000c..3bdb5f3b119ab 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -4847,6 +4847,89 @@ static int ext4_block_zero_range(struct inode *inode,
>  					zero_written);
>  }
>  
> +/*
> + * Inodes using the iomap buffered I/O path do not use data=ordered mode.
> + * Therefore, we mark the inode as disksize-grow-pending after zeroing the
> + * EOF block. The zeroed block will be submitted before any subsequent
> + * data.
> + *
> + * In the I/O completion path, ext4_iomap_wb_disksize_pending_wait() will
> + * wait for I/O completion before advancing i_disksize if the write
> + * extends beyond the zeroed boundary.
> + *
> + * When zeroed I/O is in progress, operations that extend i_disksize are
> + * handled as follows:
> + *
> + *  - Truncate up, append fallocate and zero_range:
> + *    Defer the update. The file size will be updated to i_size by the
> + *    end_io handler once the ongoing pending I/O completes.
> + *
> + *  - Insert range and collapse range operations:
> + *    Wait synchronously for the relevant I/O to complete before updating
> + *    i_disksize.
> + */
> +static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
> +{
> +	struct folio *folio;
> +
> +	folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
> +	if (IS_ERR(folio))
> +		/* Already in writeback and cleared? */
> +		return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
> +
> +	/*
> +	 * Ensure that in-flight writeback, possibly started after
> +	 * iomap_zero_range() unlocked the folio, has completed. Without
> +	 * this wait folio_test_dirty() below may miss the zeroed data
> +	 * (writeback clears PG_dirty), causing us to skip the
> +	 * disksize-grow-pending tracking and potentially expose stale
> +	 * on-disk data.
> +	 */
> +	folio_wait_writeback(folio);
> +	WARN_ON_ONCE(folio_test_writeback(folio));
> +
> +	/*
> +	 * If the zeroed range does not overlap the existing on-disk tail
> +	 * block, the zeroed data lies beyond the currently on-disk data
> +	 * and will be written back before i_disksize is advanced past it,
> +	 * so no stale data can be exposed.
> +	 *
> +	 * Checking i_disksize here (after folio_wait_writeback()) is
> +	 * necessary and safe. If a delalloc writeback of this folio was
> +	 * in-flight, it could be raced by a concurrent mmap write which
> +	 * corrupts the tail block beyond i_size but the i_disksize is not
> +	 * advanced. folio_wait_writeback() has waited for its completion
> +	 * and the ioend has advanced i_disksize accordingly. If writeback
> +	 * had not started, we don't need to mark any pending state because
> +	 * any future writeback will carry the pagecache content that now
> +	 * includes the zeroed data, so no stale data can appear on disk
> +	 * even without the pending tracking.
> +	 */

Hey Zhang,

So in continuation for our discussion at [1], thanks for the info and
yes I think the race mentioned there cannot happen. But I was still
trying to look at this path for my own understanding and there's another
sequence of events I'd like to discuss with you. Mostly the same from
last but what if the writeback moves after
ext4_iomap_makr_zero_pending():  

    Initial state: i_size = i_disksize = 2k

    1. pwrite(4k,6k)
      - ext4_block_zero_eof(2k,4k)
          set inode state DISK_SIZE_GROW_PENDING
      - i_size=6k, i_disksize=2k
    2. writeback(2k,4k)
      - submits a GROW_IO ioend for 0,4k
      - i_size=6k, i_disksize=2k (unchanged)
    3. pwrite(8k,10k) - part 1
      - ext4_block_zero_eof(6k,8k)
      - zeroes 6k,8k
      - ext4_iomap_mark_zero_pending(from=6k)
        - folio is unlocked and not under writeback yet
        - keeps DISK_SIZE_GROW_PENDING set
      - iomap_write_iter not called yet...
    4. mmap write at (6k,8k)
      - I dont see anything stopping this?
      - i_size is still 6k so this is eof write.
    5. writeback(4k,6k)
      - submits IO for 4k,8k
      - i_size=6k, i_disksize=2k (unchanged)
    6. ioend for 2. (2k,4k)
      - clears DISKSIZE_GROW_PENDING
      - GROW_IO so i_disksize updated
      - i_size=6k, i_disksize=6k
    7. ioend for 5. (4k,8k)
      - DISKSIZE_GROW_PENDING is cleared
      - i_disksize=6k,i_size=6k - no updates
      - 4k,8k converted written - data beyond EOF from 4.
    8. 3. continues
      - enter iomap_write_iter
      - pagecache_i_size_extended()
        - page_mkwrite(6k folio) and mark dirty
        - zero 6k,8k
      - i_size=10k, i_disksize=6k
    9. writeback(8k,10k)
      - DISKSIZE_GROW_PENDING cleared so just submit (8k,10k)
    10. ioend for 8.
      - DISKSIZE_GROW_PENDING cleared so no waiting
      - i_size=10k, i_disksize=10k - updated
      - if we crash now, 6k,8k non-zero data exposed

I looked at the code surrounding ext4_iomap_mkwrite() and
ext4_buffered_write_iter() and I don't think there's any sort of
serialization against this sequence right? Or am I missing something
again :) 

Regards,
ojaswin

[1] https://lore.kernel.org/linux-ext4/179153611724.860602.5266334742364847927.b4-ty@b4/T/#m72e1710d172cf7699b6f05e29a270e45951800b0
> +	if (from >= round_up(READ_ONCE(EXT4_I(inode)->i_disksize),
> +			     i_blocksize(inode)))
> +		goto out;
> +
> +	/*
> +	 * Mark the inode as disksize-grow-pending. The zeroed block will
> +	 * be written out by the generic writepages cycle or any other
> +	 * syncing operation.
> +	 *
> +	 * Multiple overlapping unaligned EOF writes should not happen,
> +	 * because we only mark the pending state after zeroing the on-disk
> +	 * EOF block, and i_disksize can only be updated after the previous
> +	 * zeroed pending block has been written back or the dirty folio
> +	 * has been discarded.
> +	 */
> +	if (likely(folio_test_dirty(folio) &&
> +		   !ext4_test_inode_state(inode,
> +					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
> +		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
> +out:
> +	folio_unlock(folio);
> +	folio_put(folio);
> +	return 0;
> +}
> +
>  /*
>   * Submit and wait for the pending zeroed EOF block range to complete
>   * if the given range [@offset, @end) fully covers it.  Must be called
> @@ -4923,21 +5006,20 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
>  	 * truncating up or performing an append write, because there might be
>  	 * exposing stale on-disk data which may caused by concurrent post-EOF
>  	 * mmap write during folio writeback.
> -	 *
> -	 * TODO: In the iomap path, handle this by tracking the ordered range
> -	 * and updating i_disksize to i_size after the zeroed data has been
> -	 * written back.
>  	 */
> -	if (ext4_should_order_data(inode) &&
> -	    did_zero && zero_written && !IS_DAX(inode)) {
> -		handle_t *handle;
> +	if (did_zero && zero_written && !IS_DAX(inode)) {
> +		if (ext4_should_order_data(inode)) {
> +			handle_t *handle;
>  
> -		handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
> -		if (IS_ERR(handle))
> -			return PTR_ERR(handle);
> +			handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
> +			if (IS_ERR(handle))
> +				return PTR_ERR(handle);
>  
> -		err = ext4_jbd2_inode_add_write(handle, inode, from, length);
> -		ext4_journal_stop(handle);
> +			err = ext4_jbd2_inode_add_write(handle, inode, from,
> +							length);
> +			ext4_journal_stop(handle);
> +		} else if (ext4_inode_buffered_iomap(inode))
> +			err = ext4_iomap_mark_disksize_pending(inode, from);
>  		if (err)
>  			return err;
>  	}
> -- 
> 2.52.0
> 

Re: [PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block

[Zhang Yi]

On 10/10/2026 5:00 PM, Ojaswin Mujoo wrote:
> On Fri, Oct 09, 2026 at 06:30:28PM +0800, Zhang Yi wrote:
>> From: Zhang Yi <yi.zhang@huawei.com>
>>
>> In the iomap buffered I/O path, data=ordered mode is not used, so the
>> zeroed EOF block has no implicit ordering with later i_disksize updates.
>> Without the pending state being set, i_disksize can be advanced past the
>> zeroed block before writeback completes, exposing stale data after a
>> crash.
>>
>> Previous patches added the consumer side of the
>> disksize-grow-pending mechanism: the state bit, clear and wait helpers,
>> and ioend tagging. Now add ext4_iomap_mark_disksize_pending() and call
>> it from ext4_block_zero_eof() after zeroing the tail of the block that
>> straddles i_disksize.
>>
>> The helper locks the folio, waits for any in-flight writeback on it to
>> complete, then sets EXT4_STATE_DISKSIZE_GROW_PENDING only if the folio
>> is still dirty. Waiting for writeback prevents folio_test_dirty() from
>> returning false mid-writeback, which would cause us to skip the pending
>> state while zeroed data is still in flight. The dirty check then avoids
>> setting the bit when the data has already been written back.
>>
>> Suggested-by: Jan Kara <jack@suse.cz>
>> Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
>> ---
>>   fs/ext4/inode.c | 106 ++++++++++++++++++++++++++++++++++++++++++------
>>   1 file changed, 94 insertions(+), 12 deletions(-)
>>
>> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
>> index 9b6916609000c..3bdb5f3b119ab 100644
>> --- a/fs/ext4/inode.c
>> +++ b/fs/ext4/inode.c
>> @@ -4847,6 +4847,89 @@ static int ext4_block_zero_range(struct inode *inode,
>>   					zero_written);
>>   }
>>   
>> +/*
>> + * Inodes using the iomap buffered I/O path do not use data=ordered mode.
>> + * Therefore, we mark the inode as disksize-grow-pending after zeroing the
>> + * EOF block. The zeroed block will be submitted before any subsequent
>> + * data.
>> + *
>> + * In the I/O completion path, ext4_iomap_wb_disksize_pending_wait() will
>> + * wait for I/O completion before advancing i_disksize if the write
>> + * extends beyond the zeroed boundary.
>> + *
>> + * When zeroed I/O is in progress, operations that extend i_disksize are
>> + * handled as follows:
>> + *
>> + *  - Truncate up, append fallocate and zero_range:
>> + *    Defer the update. The file size will be updated to i_size by the
>> + *    end_io handler once the ongoing pending I/O completes.
>> + *
>> + *  - Insert range and collapse range operations:
>> + *    Wait synchronously for the relevant I/O to complete before updating
>> + *    i_disksize.
>> + */
>> +static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
>> +{
>> +	struct folio *folio;
>> +
>> +	folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
>> +	if (IS_ERR(folio))
>> +		/* Already in writeback and cleared? */
>> +		return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
>> +
>> +	/*
>> +	 * Ensure that in-flight writeback, possibly started after
>> +	 * iomap_zero_range() unlocked the folio, has completed. Without
>> +	 * this wait folio_test_dirty() below may miss the zeroed data
>> +	 * (writeback clears PG_dirty), causing us to skip the
>> +	 * disksize-grow-pending tracking and potentially expose stale
>> +	 * on-disk data.
>> +	 */
>> +	folio_wait_writeback(folio);
>> +	WARN_ON_ONCE(folio_test_writeback(folio));
>> +
>> +	/*
>> +	 * If the zeroed range does not overlap the existing on-disk tail
>> +	 * block, the zeroed data lies beyond the currently on-disk data
>> +	 * and will be written back before i_disksize is advanced past it,
>> +	 * so no stale data can be exposed.
>> +	 *
>> +	 * Checking i_disksize here (after folio_wait_writeback()) is
>> +	 * necessary and safe. If a delalloc writeback of this folio was
>> +	 * in-flight, it could be raced by a concurrent mmap write which
>> +	 * corrupts the tail block beyond i_size but the i_disksize is not
>> +	 * advanced. folio_wait_writeback() has waited for its completion
>> +	 * and the ioend has advanced i_disksize accordingly. If writeback
>> +	 * had not started, we don't need to mark any pending state because
>> +	 * any future writeback will carry the pagecache content that now
>> +	 * includes the zeroed data, so no stale data can appear on disk
>> +	 * even without the pending tracking.
>> +	 */
> 
> Hey Zhang,
> 
> So in continuation for our discussion at [1], thanks for the info and
> yes I think the race mentioned there cannot happen. But I was still
> trying to look at this path for my own understanding and there's another
> sequence of events I'd like to discuss with you. Mostly the same from
> last but what if the writeback moves after
> ext4_iomap_makr_zero_pending():
> 
>      Initial state: i_size = i_disksize = 2k
> 
>      1. pwrite(4k,6k)
>        - ext4_block_zero_eof(2k,4k)
>            set inode state DISK_SIZE_GROW_PENDING
>        - i_size=6k, i_disksize=2k
>      2. writeback(2k,4k)
>        - submits a GROW_IO ioend for 0,4k
>        - i_size=6k, i_disksize=2k (unchanged)
>      3. pwrite(8k,10k) - part 1
>        - ext4_block_zero_eof(6k,8k)
>        - zeroes 6k,8k
>        - ext4_iomap_mark_zero_pending(from=6k)
>          - folio is unlocked and not under writeback yet
>          - keeps DISK_SIZE_GROW_PENDING set
>        - iomap_write_iter not called yet...
>      4. mmap write at (6k,8k)
>        - I dont see anything stopping this?
>        - i_size is still 6k so this is eof write.
>      5. writeback(4k,6k)
>        - submits IO for 4k,8k
>        - i_size=6k, i_disksize=2k (unchanged)
>      6. ioend for 2. (2k,4k)
>        - clears DISKSIZE_GROW_PENDING
>        - GROW_IO so i_disksize updated
>        - i_size=6k, i_disksize=6k
>      7. ioend for 5. (4k,8k)
>        - DISKSIZE_GROW_PENDING is cleared
>        - i_disksize=6k,i_size=6k - no updates
>        - 4k,8k converted written - data beyond EOF from 4.
>      8. 3. continues
>        - enter iomap_write_iter
>        - pagecache_i_size_extended()
>          - page_mkwrite(6k folio) and mark dirty
>          - zero 6k,8k
>        - i_size=10k, i_disksize=6k
>      9. writeback(8k,10k)
>        - DISKSIZE_GROW_PENDING cleared so just submit (8k,10k)
>      10. ioend for 8.
>        - DISKSIZE_GROW_PENDING cleared so no waiting
>        - i_size=10k, i_disksize=10k - updated
>        - if we crash now, 6k,8k non-zero data exposed
> 
> I looked at the code surrounding ext4_iomap_mkwrite() and
> ext4_buffered_write_iter() and I don't think there's any sort of
> serialization against this sequence right? Or am I missing something
> again :)
> 

Hi Ojaswin,

If I understand correctly, your scenario is essentially a concurrent
mmap write and append write, that is, a post-EOF mmap write is performed
after the append write does the EOF zeroing but before i_size is
advanced, right?

The scenario can be simplified as follows:

   Initial state: i_size = i_disksize = 2k

   1. pwrite (4k,6k)
     - ext4_block_zero_eof(2k,4k), zero 2k,4k and dirty folio
     - iomap_write_iter not called yet... so i_size is still 2k
   2. mmap write at (2k,4k)
     - i_size is still 2k (so this is still an EOF write?)
   3. writeback (0,4k) since it's dirty
     - submits I/O for (0,4k), data is written to disk
   4. 1 continue and then writeback
     - i_size=6k
     - submits I/O for (4,8k)
     - i_disksize=6k, 2k,4k non-zero data exposed

Actually, I think we don't need to consider this kind of concurrent
scenario, since it is undefined behavior in itself. Even if we
don't write this part of the mmap data back to disk, we can still
read it from memory, right? What ext4_block_zero_eof() needs to do
is zero out the post-EOF stale data written by the mmap before the
append write.

Thanks,
Yi


> Regards,
> ojaswin
> 
> [1] https://lore.kernel.org/linux-ext4/179153611724.860602.5266334742364847927.b4-ty@b4/T/#m72e1710d172cf7699b6f05e29a270e45951800b0
>> +	if (from >= round_up(READ_ONCE(EXT4_I(inode)->i_disksize),
>> +			     i_blocksize(inode)))
>> +		goto out;
>> +
>> +	/*
>> +	 * Mark the inode as disksize-grow-pending. The zeroed block will
>> +	 * be written out by the generic writepages cycle or any other
>> +	 * syncing operation.
>> +	 *
>> +	 * Multiple overlapping unaligned EOF writes should not happen,
>> +	 * because we only mark the pending state after zeroing the on-disk
>> +	 * EOF block, and i_disksize can only be updated after the previous
>> +	 * zeroed pending block has been written back or the dirty folio
>> +	 * has been discarded.
>> +	 */
>> +	if (likely(folio_test_dirty(folio) &&
>> +		   !ext4_test_inode_state(inode,
>> +					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
>> +		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
>> +out:
>> +	folio_unlock(folio);
>> +	folio_put(folio);
>> +	return 0;
>> +}
>> +
>>   /*
>>    * Submit and wait for the pending zeroed EOF block range to complete
>>    * if the given range [@offset, @end) fully covers it.  Must be called
>> @@ -4923,21 +5006,20 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
>>   	 * truncating up or performing an append write, because there might be
>>   	 * exposing stale on-disk data which may caused by concurrent post-EOF
>>   	 * mmap write during folio writeback.
>> -	 *
>> -	 * TODO: In the iomap path, handle this by tracking the ordered range
>> -	 * and updating i_disksize to i_size after the zeroed data has been
>> -	 * written back.
>>   	 */
>> -	if (ext4_should_order_data(inode) &&
>> -	    did_zero && zero_written && !IS_DAX(inode)) {
>> -		handle_t *handle;
>> +	if (did_zero && zero_written && !IS_DAX(inode)) {
>> +		if (ext4_should_order_data(inode)) {
>> +			handle_t *handle;
>>   
>> -		handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
>> -		if (IS_ERR(handle))
>> -			return PTR_ERR(handle);
>> +			handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
>> +			if (IS_ERR(handle))
>> +				return PTR_ERR(handle);
>>   
>> -		err = ext4_jbd2_inode_add_write(handle, inode, from, length);
>> -		ext4_journal_stop(handle);
>> +			err = ext4_jbd2_inode_add_write(handle, inode, from,
>> +							length);
>> +			ext4_journal_stop(handle);
>> +		} else if (ext4_inode_buffered_iomap(inode))
>> +			err = ext4_iomap_mark_disksize_pending(inode, from);
>>   		if (err)
>>   			return err;
>>   	}
>> -- 
>> 2.52.0
>>

Re: [PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block

[Ojaswin Mujoo]

On Sat, Oct 10, 2026 at 05:50:24PM +0800, Zhang Yi wrote:
> On 10/10/2026 5:00 PM, Ojaswin Mujoo wrote:
> > On Fri, Oct 09, 2026 at 06:30:28PM +0800, Zhang Yi wrote:
> > > From: Zhang Yi <yi.zhang@huawei.com>
> > > 
> > > In the iomap buffered I/O path, data=ordered mode is not used, so the
> > > zeroed EOF block has no implicit ordering with later i_disksize updates.
> > > Without the pending state being set, i_disksize can be advanced past the
> > > zeroed block before writeback completes, exposing stale data after a
> > > crash.
> > > 
> > > Previous patches added the consumer side of the
> > > disksize-grow-pending mechanism: the state bit, clear and wait helpers,
> > > and ioend tagging. Now add ext4_iomap_mark_disksize_pending() and call
> > > it from ext4_block_zero_eof() after zeroing the tail of the block that
> > > straddles i_disksize.
> > > 
> > > The helper locks the folio, waits for any in-flight writeback on it to
> > > complete, then sets EXT4_STATE_DISKSIZE_GROW_PENDING only if the folio
> > > is still dirty. Waiting for writeback prevents folio_test_dirty() from
> > > returning false mid-writeback, which would cause us to skip the pending
> > > state while zeroed data is still in flight. The dirty check then avoids
> > > setting the bit when the data has already been written back.
> > > 
> > > Suggested-by: Jan Kara <jack@suse.cz>
> > > Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
> > > ---
> > >   fs/ext4/inode.c | 106 ++++++++++++++++++++++++++++++++++++++++++------
> > >   1 file changed, 94 insertions(+), 12 deletions(-)
> > > 
> > > diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> > > index 9b6916609000c..3bdb5f3b119ab 100644
> > > --- a/fs/ext4/inode.c
> > > +++ b/fs/ext4/inode.c
> > > @@ -4847,6 +4847,89 @@ static int ext4_block_zero_range(struct inode *inode,
> > >   					zero_written);
> > >   }
> > > +/*
> > > + * Inodes using the iomap buffered I/O path do not use data=ordered mode.
> > > + * Therefore, we mark the inode as disksize-grow-pending after zeroing the
> > > + * EOF block. The zeroed block will be submitted before any subsequent
> > > + * data.
> > > + *
> > > + * In the I/O completion path, ext4_iomap_wb_disksize_pending_wait() will
> > > + * wait for I/O completion before advancing i_disksize if the write
> > > + * extends beyond the zeroed boundary.
> > > + *
> > > + * When zeroed I/O is in progress, operations that extend i_disksize are
> > > + * handled as follows:
> > > + *
> > > + *  - Truncate up, append fallocate and zero_range:
> > > + *    Defer the update. The file size will be updated to i_size by the
> > > + *    end_io handler once the ongoing pending I/O completes.
> > > + *
> > > + *  - Insert range and collapse range operations:
> > > + *    Wait synchronously for the relevant I/O to complete before updating
> > > + *    i_disksize.
> > > + */
> > > +static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
> > > +{
> > > +	struct folio *folio;
> > > +
> > > +	folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
> > > +	if (IS_ERR(folio))
> > > +		/* Already in writeback and cleared? */
> > > +		return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
> > > +
> > > +	/*
> > > +	 * Ensure that in-flight writeback, possibly started after
> > > +	 * iomap_zero_range() unlocked the folio, has completed. Without
> > > +	 * this wait folio_test_dirty() below may miss the zeroed data
> > > +	 * (writeback clears PG_dirty), causing us to skip the
> > > +	 * disksize-grow-pending tracking and potentially expose stale
> > > +	 * on-disk data.
> > > +	 */
> > > +	folio_wait_writeback(folio);
> > > +	WARN_ON_ONCE(folio_test_writeback(folio));
> > > +
> > > +	/*
> > > +	 * If the zeroed range does not overlap the existing on-disk tail
> > > +	 * block, the zeroed data lies beyond the currently on-disk data
> > > +	 * and will be written back before i_disksize is advanced past it,
> > > +	 * so no stale data can be exposed.
> > > +	 *
> > > +	 * Checking i_disksize here (after folio_wait_writeback()) is
> > > +	 * necessary and safe. If a delalloc writeback of this folio was
> > > +	 * in-flight, it could be raced by a concurrent mmap write which
> > > +	 * corrupts the tail block beyond i_size but the i_disksize is not
> > > +	 * advanced. folio_wait_writeback() has waited for its completion
> > > +	 * and the ioend has advanced i_disksize accordingly. If writeback
> > > +	 * had not started, we don't need to mark any pending state because
> > > +	 * any future writeback will carry the pagecache content that now
> > > +	 * includes the zeroed data, so no stale data can appear on disk
> > > +	 * even without the pending tracking.
> > > +	 */
> > 
> > Hey Zhang,
> > 
> > So in continuation for our discussion at [1], thanks for the info and
> > yes I think the race mentioned there cannot happen. But I was still
> > trying to look at this path for my own understanding and there's another
> > sequence of events I'd like to discuss with you. Mostly the same from
> > last but what if the writeback moves after
> > ext4_iomap_makr_zero_pending():
> > 
> >      Initial state: i_size = i_disksize = 2k
> > 
> >      1. pwrite(4k,6k)
> >        - ext4_block_zero_eof(2k,4k)
> >            set inode state DISK_SIZE_GROW_PENDING
> >        - i_size=6k, i_disksize=2k
> >      2. writeback(2k,4k)
> >        - submits a GROW_IO ioend for 0,4k
> >        - i_size=6k, i_disksize=2k (unchanged)
> >      3. pwrite(8k,10k) - part 1
> >        - ext4_block_zero_eof(6k,8k)
> >        - zeroes 6k,8k
> >        - ext4_iomap_mark_zero_pending(from=6k)
> >          - folio is unlocked and not under writeback yet
> >          - keeps DISK_SIZE_GROW_PENDING set
> >        - iomap_write_iter not called yet...
> >      4. mmap write at (6k,8k)
> >        - I dont see anything stopping this?
> >        - i_size is still 6k so this is eof write.
> >      5. writeback(4k,6k)
> >        - submits IO for 4k,8k
> >        - i_size=6k, i_disksize=2k (unchanged)
> >      6. ioend for 2. (2k,4k)
> >        - clears DISKSIZE_GROW_PENDING
> >        - GROW_IO so i_disksize updated
> >        - i_size=6k, i_disksize=6k
> >      7. ioend for 5. (4k,8k)
> >        - DISKSIZE_GROW_PENDING is cleared
> >        - i_disksize=6k,i_size=6k - no updates
> >        - 4k,8k converted written - data beyond EOF from 4.
> >      8. 3. continues
> >        - enter iomap_write_iter
> >        - pagecache_i_size_extended()
> >          - page_mkwrite(6k folio) and mark dirty
> >          - zero 6k,8k
> >        - i_size=10k, i_disksize=6k
> >      9. writeback(8k,10k)
> >        - DISKSIZE_GROW_PENDING cleared so just submit (8k,10k)
> >      10. ioend for 8.
> >        - DISKSIZE_GROW_PENDING cleared so no waiting
> >        - i_size=10k, i_disksize=10k - updated
> >        - if we crash now, 6k,8k non-zero data exposed
> > 
> > I looked at the code surrounding ext4_iomap_mkwrite() and
> > ext4_buffered_write_iter() and I don't think there's any sort of
> > serialization against this sequence right? Or am I missing something
> > again :)
> > 
> 
> Hi Ojaswin,
> 
> If I understand correctly, your scenario is essentially a concurrent
> mmap write and append write, that is, a post-EOF mmap write is performed
> after the append write does the EOF zeroing but before i_size is
> advanced, right?
> 
> The scenario can be simplified as follows:
> 
>   Initial state: i_size = i_disksize = 2k
> 
>   1. pwrite (4k,6k)
>     - ext4_block_zero_eof(2k,4k), zero 2k,4k and dirty folio
>     - iomap_write_iter not called yet... so i_size is still 2k
>   2. mmap write at (2k,4k)
>     - i_size is still 2k (so this is still an EOF write?)
>   3. writeback (0,4k) since it's dirty
>     - submits I/O for (0,4k), data is written to disk
>   4. 1 continue and then writeback
>     - i_size=6k
>     - submits I/O for (4,8k)
>     - i_disksize=6k, 2k,4k non-zero data exposed

Yep, looks right.
> 
> Actually, I think we don't need to consider this kind of concurrent
> scenario, since it is undefined behavior in itself. Even if we
> don't write this part of the mmap data back to disk, we can still

It will be zeroed again when i_size grows in iomap_write_iter()
-> pagecache_isize_extended() but yeah a mapped reader might be able to
read it before that happens. 

> read it from memory, right? What ext4_block_zero_eof() needs to do
> is zero out the post-EOF stale data written by the mmap before the
> append write.

Okay got it so we care more about not exposing data that was
previously on disk rather than a race with mmap. We do have some
xfstests that seem to care about the later as well (like generic/363)
but I get that the race is a very small window here and needs us to
crash at a specific time. Plus its a parallel mmap written data rather
than stale data so hopefully it's harmless even if a bit of it is
exposed after crash.

Thanks,
ojaswin

> 
> Thanks,
> Yi
> 
> 
> > Regards,
> > ojaswin
> > 
> > [1] https://lore.kernel.org/linux-ext4/179153611724.860602.5266334742364847927.b4-ty@b4/T/#m72e1710d172cf7699b6f05e29a270e45951800b0
> > > +	if (from >= round_up(READ_ONCE(EXT4_I(inode)->i_disksize),
> > > +			     i_blocksize(inode)))
> > > +		goto out;
> > > +
> > > +	/*
> > > +	 * Mark the inode as disksize-grow-pending. The zeroed block will
> > > +	 * be written out by the generic writepages cycle or any other
> > > +	 * syncing operation.
> > > +	 *
> > > +	 * Multiple overlapping unaligned EOF writes should not happen,
> > > +	 * because we only mark the pending state after zeroing the on-disk
> > > +	 * EOF block, and i_disksize can only be updated after the previous
> > > +	 * zeroed pending block has been written back or the dirty folio
> > > +	 * has been discarded.
> > > +	 */
> > > +	if (likely(folio_test_dirty(folio) &&
> > > +		   !ext4_test_inode_state(inode,
> > > +					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
> > > +		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
> > > +out:
> > > +	folio_unlock(folio);
> > > +	folio_put(folio);
> > > +	return 0;
> > > +}
> > > +
> > >   /*
> > >    * Submit and wait for the pending zeroed EOF block range to complete
> > >    * if the given range [@offset, @end) fully covers it.  Must be called
> > > @@ -4923,21 +5006,20 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
> > >   	 * truncating up or performing an append write, because there might be
> > >   	 * exposing stale on-disk data which may caused by concurrent post-EOF
> > >   	 * mmap write during folio writeback.
> > > -	 *
> > > -	 * TODO: In the iomap path, handle this by tracking the ordered range
> > > -	 * and updating i_disksize to i_size after the zeroed data has been
> > > -	 * written back.
> > >   	 */
> > > -	if (ext4_should_order_data(inode) &&
> > > -	    did_zero && zero_written && !IS_DAX(inode)) {
> > > -		handle_t *handle;
> > > +	if (did_zero && zero_written && !IS_DAX(inode)) {
> > > +		if (ext4_should_order_data(inode)) {
> > > +			handle_t *handle;
> > > -		handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
> > > -		if (IS_ERR(handle))
> > > -			return PTR_ERR(handle);
> > > +			handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
> > > +			if (IS_ERR(handle))
> > > +				return PTR_ERR(handle);
> > > -		err = ext4_jbd2_inode_add_write(handle, inode, from, length);
> > > -		ext4_journal_stop(handle);
> > > +			err = ext4_jbd2_inode_add_write(handle, inode, from,
> > > +							length);
> > > +			ext4_journal_stop(handle);
> > > +		} else if (ext4_inode_buffered_iomap(inode))
> > > +			err = ext4_iomap_mark_disksize_pending(inode, from);
> > >   		if (err)
> > >   			return err;
> > >   	}
> > > -- 
> > > 2.52.0
> > > 
> 

Re: [PATCH v7 27/31] ext4: set DISKSIZE_GROW_PENDING after zeroing unaligned EOF block

[Zhang Yi]

On 10/10/2026 9:49 PM, Ojaswin Mujoo wrote:
> On Sat, Oct 10, 2026 at 05:50:24PM +0800, Zhang Yi wrote:
>> On 10/10/2026 5:00 PM, Ojaswin Mujoo wrote:
>>> On Fri, Oct 09, 2026 at 06:30:28PM +0800, Zhang Yi wrote:
>>>> From: Zhang Yi <yi.zhang@huawei.com>
>>>>
>>>> In the iomap buffered I/O path, data=ordered mode is not used, so the
>>>> zeroed EOF block has no implicit ordering with later i_disksize updates.
>>>> Without the pending state being set, i_disksize can be advanced past the
>>>> zeroed block before writeback completes, exposing stale data after a
>>>> crash.
>>>>
>>>> Previous patches added the consumer side of the
>>>> disksize-grow-pending mechanism: the state bit, clear and wait helpers,
>>>> and ioend tagging. Now add ext4_iomap_mark_disksize_pending() and call
>>>> it from ext4_block_zero_eof() after zeroing the tail of the block that
>>>> straddles i_disksize.
>>>>
>>>> The helper locks the folio, waits for any in-flight writeback on it to
>>>> complete, then sets EXT4_STATE_DISKSIZE_GROW_PENDING only if the folio
>>>> is still dirty. Waiting for writeback prevents folio_test_dirty() from
>>>> returning false mid-writeback, which would cause us to skip the pending
>>>> state while zeroed data is still in flight. The dirty check then avoids
>>>> setting the bit when the data has already been written back.
>>>>
>>>> Suggested-by: Jan Kara <jack@suse.cz>
>>>> Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
>>>> ---
>>>>    fs/ext4/inode.c | 106 ++++++++++++++++++++++++++++++++++++++++++------
>>>>    1 file changed, 94 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
>>>> index 9b6916609000c..3bdb5f3b119ab 100644
>>>> --- a/fs/ext4/inode.c
>>>> +++ b/fs/ext4/inode.c
>>>> @@ -4847,6 +4847,89 @@ static int ext4_block_zero_range(struct inode *inode,
>>>>    					zero_written);
>>>>    }
>>>> +/*
>>>> + * Inodes using the iomap buffered I/O path do not use data=ordered mode.
>>>> + * Therefore, we mark the inode as disksize-grow-pending after zeroing the
>>>> + * EOF block. The zeroed block will be submitted before any subsequent
>>>> + * data.
>>>> + *
>>>> + * In the I/O completion path, ext4_iomap_wb_disksize_pending_wait() will
>>>> + * wait for I/O completion before advancing i_disksize if the write
>>>> + * extends beyond the zeroed boundary.
>>>> + *
>>>> + * When zeroed I/O is in progress, operations that extend i_disksize are
>>>> + * handled as follows:
>>>> + *
>>>> + *  - Truncate up, append fallocate and zero_range:
>>>> + *    Defer the update. The file size will be updated to i_size by the
>>>> + *    end_io handler once the ongoing pending I/O completes.
>>>> + *
>>>> + *  - Insert range and collapse range operations:
>>>> + *    Wait synchronously for the relevant I/O to complete before updating
>>>> + *    i_disksize.
>>>> + */
>>>> +static int ext4_iomap_mark_disksize_pending(struct inode *inode, loff_t from)
>>>> +{
>>>> +	struct folio *folio;
>>>> +
>>>> +	folio = filemap_lock_folio(inode->i_mapping, from >> PAGE_SHIFT);
>>>> +	if (IS_ERR(folio))
>>>> +		/* Already in writeback and cleared? */
>>>> +		return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
>>>> +
>>>> +	/*
>>>> +	 * Ensure that in-flight writeback, possibly started after
>>>> +	 * iomap_zero_range() unlocked the folio, has completed. Without
>>>> +	 * this wait folio_test_dirty() below may miss the zeroed data
>>>> +	 * (writeback clears PG_dirty), causing us to skip the
>>>> +	 * disksize-grow-pending tracking and potentially expose stale
>>>> +	 * on-disk data.
>>>> +	 */
>>>> +	folio_wait_writeback(folio);
>>>> +	WARN_ON_ONCE(folio_test_writeback(folio));
>>>> +
>>>> +	/*
>>>> +	 * If the zeroed range does not overlap the existing on-disk tail
>>>> +	 * block, the zeroed data lies beyond the currently on-disk data
>>>> +	 * and will be written back before i_disksize is advanced past it,
>>>> +	 * so no stale data can be exposed.
>>>> +	 *
>>>> +	 * Checking i_disksize here (after folio_wait_writeback()) is
>>>> +	 * necessary and safe. If a delalloc writeback of this folio was
>>>> +	 * in-flight, it could be raced by a concurrent mmap write which
>>>> +	 * corrupts the tail block beyond i_size but the i_disksize is not
>>>> +	 * advanced. folio_wait_writeback() has waited for its completion
>>>> +	 * and the ioend has advanced i_disksize accordingly. If writeback
>>>> +	 * had not started, we don't need to mark any pending state because
>>>> +	 * any future writeback will carry the pagecache content that now
>>>> +	 * includes the zeroed data, so no stale data can appear on disk
>>>> +	 * even without the pending tracking.
>>>> +	 */
>>>
>>> Hey Zhang,
>>>
>>> So in continuation for our discussion at [1], thanks for the info and
>>> yes I think the race mentioned there cannot happen. But I was still
>>> trying to look at this path for my own understanding and there's another
>>> sequence of events I'd like to discuss with you. Mostly the same from
>>> last but what if the writeback moves after
>>> ext4_iomap_makr_zero_pending():
>>>
>>>       Initial state: i_size = i_disksize = 2k
>>>
>>>       1. pwrite(4k,6k)
>>>         - ext4_block_zero_eof(2k,4k)
>>>             set inode state DISK_SIZE_GROW_PENDING
>>>         - i_size=6k, i_disksize=2k
>>>       2. writeback(2k,4k)
>>>         - submits a GROW_IO ioend for 0,4k
>>>         - i_size=6k, i_disksize=2k (unchanged)
>>>       3. pwrite(8k,10k) - part 1
>>>         - ext4_block_zero_eof(6k,8k)
>>>         - zeroes 6k,8k
>>>         - ext4_iomap_mark_zero_pending(from=6k)
>>>           - folio is unlocked and not under writeback yet
>>>           - keeps DISK_SIZE_GROW_PENDING set
>>>         - iomap_write_iter not called yet...
>>>       4. mmap write at (6k,8k)
>>>         - I dont see anything stopping this?
>>>         - i_size is still 6k so this is eof write.
>>>       5. writeback(4k,6k)
>>>         - submits IO for 4k,8k
>>>         - i_size=6k, i_disksize=2k (unchanged)
>>>       6. ioend for 2. (2k,4k)
>>>         - clears DISKSIZE_GROW_PENDING
>>>         - GROW_IO so i_disksize updated
>>>         - i_size=6k, i_disksize=6k
>>>       7. ioend for 5. (4k,8k)
>>>         - DISKSIZE_GROW_PENDING is cleared
>>>         - i_disksize=6k,i_size=6k - no updates
>>>         - 4k,8k converted written - data beyond EOF from 4.
>>>       8. 3. continues
>>>         - enter iomap_write_iter
>>>         - pagecache_i_size_extended()
>>>           - page_mkwrite(6k folio) and mark dirty
>>>           - zero 6k,8k
>>>         - i_size=10k, i_disksize=6k
>>>       9. writeback(8k,10k)
>>>         - DISKSIZE_GROW_PENDING cleared so just submit (8k,10k)
>>>       10. ioend for 8.
>>>         - DISKSIZE_GROW_PENDING cleared so no waiting
>>>         - i_size=10k, i_disksize=10k - updated
>>>         - if we crash now, 6k,8k non-zero data exposed
>>>
>>> I looked at the code surrounding ext4_iomap_mkwrite() and
>>> ext4_buffered_write_iter() and I don't think there's any sort of
>>> serialization against this sequence right? Or am I missing something
>>> again :)
>>>
>>
>> Hi Ojaswin,
>>
>> If I understand correctly, your scenario is essentially a concurrent
>> mmap write and append write, that is, a post-EOF mmap write is performed
>> after the append write does the EOF zeroing but before i_size is
>> advanced, right?
>>
>> The scenario can be simplified as follows:
>>
>>    Initial state: i_size = i_disksize = 2k
>>
>>    1. pwrite (4k,6k)
>>      - ext4_block_zero_eof(2k,4k), zero 2k,4k and dirty folio
>>      - iomap_write_iter not called yet... so i_size is still 2k
>>    2. mmap write at (2k,4k)
>>      - i_size is still 2k (so this is still an EOF write?)
>>    3. writeback (0,4k) since it's dirty
>>      - submits I/O for (0,4k), data is written to disk
>>    4. 1 continue and then writeback
>>      - i_size=6k
>>      - submits I/O for (4,8k)
>>      - i_disksize=6k, 2k,4k non-zero data exposed
> 
> Yep, looks right.
>>
>> Actually, I think we don't need to consider this kind of concurrent
>> scenario, since it is undefined behavior in itself. Even if we
>> don't write this part of the mmap data back to disk, we can still
> 
> It will be zeroed again when i_size grows in iomap_write_iter()
> -> pagecache_isize_extended() but yeah a mapped reader might be able to
> read it before that happens.

In fact, for our current scenario, pagecache_isize_extended() is a
no-op, since blocksize equals PAGE_SIZE, so it will no longer zero out
the post-EOF data written by mmap. What pagecache_isize_extended()
zeroes is the data in the folio that doesn't belong to the block
containing the current EOF when blocksize is smaller than folio size,
whereas ext4_block_zero_eof() zeroes the post-EOF part inside the tail
block. Their responsibilities are not the same.

So in the scenario you describe, the data written by mmap will remain in
memory for a long time, and this is not a small window. Since we don't
care about this concurrent scenario, puting ext4_block_zero_eof() before
the write is safe.

Cheers,
Yi.

> 
>> read it from memory, right? What ext4_block_zero_eof() needs to do
>> is zero out the post-EOF stale data written by the mmap before the
>> append write.
> 
> Okay got it so we care more about not exposing data that was
> previously on disk rather than a race with mmap. We do have some
> xfstests that seem to care about the later as well (like generic/363)
> but I get that the race is a very small window here and needs us to
> crash at a specific time. Plus its a parallel mmap written data rather
> than stale data so hopefully it's harmless even if a bit of it is
> exposed after crash.
> 
> Thanks,
> ojaswin
> 
>>
>> Thanks,
>> Yi
>>
>>
>>> Regards,
>>> ojaswin
>>>
>>> [1] https://lore.kernel.org/linux-ext4/179153611724.860602.5266334742364847927.b4-ty@b4/T/#m72e1710d172cf7699b6f05e29a270e45951800b0
>>>> +	if (from >= round_up(READ_ONCE(EXT4_I(inode)->i_disksize),
>>>> +			     i_blocksize(inode)))
>>>> +		goto out;
>>>> +
>>>> +	/*
>>>> +	 * Mark the inode as disksize-grow-pending. The zeroed block will
>>>> +	 * be written out by the generic writepages cycle or any other
>>>> +	 * syncing operation.
>>>> +	 *
>>>> +	 * Multiple overlapping unaligned EOF writes should not happen,
>>>> +	 * because we only mark the pending state after zeroing the on-disk
>>>> +	 * EOF block, and i_disksize can only be updated after the previous
>>>> +	 * zeroed pending block has been written back or the dirty folio
>>>> +	 * has been discarded.
>>>> +	 */
>>>> +	if (likely(folio_test_dirty(folio) &&
>>>> +		   !ext4_test_inode_state(inode,
>>>> +					  EXT4_STATE_DISKSIZE_GROW_PENDING)))
>>>> +		ext4_set_inode_state(inode, EXT4_STATE_DISKSIZE_GROW_PENDING);
>>>> +out:
>>>> +	folio_unlock(folio);
>>>> +	folio_put(folio);
>>>> +	return 0;
>>>> +}
>>>> +
>>>>    /*
>>>>     * Submit and wait for the pending zeroed EOF block range to complete
>>>>     * if the given range [@offset, @end) fully covers it.  Must be called
>>>> @@ -4923,21 +5006,20 @@ int ext4_block_zero_eof(struct inode *inode, loff_t from, loff_t end)
>>>>    	 * truncating up or performing an append write, because there might be
>>>>    	 * exposing stale on-disk data which may caused by concurrent post-EOF
>>>>    	 * mmap write during folio writeback.
>>>> -	 *
>>>> -	 * TODO: In the iomap path, handle this by tracking the ordered range
>>>> -	 * and updating i_disksize to i_size after the zeroed data has been
>>>> -	 * written back.
>>>>    	 */
>>>> -	if (ext4_should_order_data(inode) &&
>>>> -	    did_zero && zero_written && !IS_DAX(inode)) {
>>>> -		handle_t *handle;
>>>> +	if (did_zero && zero_written && !IS_DAX(inode)) {
>>>> +		if (ext4_should_order_data(inode)) {
>>>> +			handle_t *handle;
>>>> -		handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
>>>> -		if (IS_ERR(handle))
>>>> -			return PTR_ERR(handle);
>>>> +			handle = ext4_journal_start(inode, EXT4_HT_MISC, 1);
>>>> +			if (IS_ERR(handle))
>>>> +				return PTR_ERR(handle);
>>>> -		err = ext4_jbd2_inode_add_write(handle, inode, from, length);
>>>> -		ext4_journal_stop(handle);
>>>> +			err = ext4_jbd2_inode_add_write(handle, inode, from,
>>>> +							length);
>>>> +			ext4_journal_stop(handle);
>>>> +		} else if (ext4_inode_buffered_iomap(inode))
>>>> +			err = ext4_iomap_mark_disksize_pending(inode, from);
>>>>    		if (err)
>>>>    			return err;
>>>>    	}
>>>> -- 
>>>> 2.52.0
>>>>
>>