* [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE
@ 2026-11-16 4:08 Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 01/20] f2fs: use block-based index and size calculations in more paths Daeho Jeong
` (19 more replies)
0 siblings, 20 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
F2FS has required the filesystem block size to equal PAGE_SIZE. On a
kernel with 16KB pages, this means an existing F2FS image with 4KB
blocks cannot be mounted, e.g. the userdata partition of an Android
device that moves from a 4KB page kernel to a 16KB page kernel.
This series lets F2FS mount and use a filesystem whose block size is
smaller than PAGE_SIZE, such as 4KB blocks on a 16KB page kernel. The
on-disk format does not change.
With subpage blocks, even an order-0 folio holds several blocks, so
the series builds on the large folio support from Nanzhe Zhao [1] and
tracks uptodate and dirty state per block in f2fs_folio_state for any
folio that holds more than one block. Most paths then work in blocks
instead of pages:
- Patches 1-2 convert more page based index and size calculations to
block based ones, and add the superblock and mount infrastructure.
Subpage block sizes are still rejected at this point.
- Patch 3 extends f2fs_folio_state with per-block state, identified by
a magic number, and allocates it on demand with __GFP_NOFAIL.
- Patches 4-7 handle directories, the per-inode dirty counters, the
read path and GC, and the write path and writeback per block.
- Patches 8-19 adapt the remaining features: range operations and
mmap, pinned files, collapse/insert range, move range,
defragmentation, swapfile, fs-verity, atomic writes, inline data,
online resize and compression, and remove the -EOPNOTSUPP checks
that patch 2 added for the features that need them.
- Patch 20 stops rejecting subpage block sizes.
When the block size equals PAGE_SIZE, the behavior does not change,
except for the fixes noted in the individual patches. The following
are not limited to subpage blocks or large folios:
- Patch 1: __complete_revoke_list() and __clone_blkaddrs() compute
offsets in 64 bits on 32-bit architectures.
- Patch 8: an extending direct write updates i_size in ->end_io(),
before iomap invalidates the page cache of the written range.
- Patch 10: collapse and insert range hold i_gc_rwsem[WRITE] and
invalidate_lock until the moved blocks are written back and the page
cache is dropped.
- Patch 11: move range zeroes the cached data past the old EOF of the
destination before extending it, and drops the source blocks from
the page cache also after an exchange that failed part way.
- Patch 12: defragmentation reads compressed files through
->read_folio(). A block of a compressed cluster that was not in the
page cache was read without decompression and written back as file
data.
- Patch 15: aborting atomic writes holds i_gc_rwsem[WRITE] and
invalidate_lock.
A few of the patches adjust code from the large folio series [1]:
- f2fs_folio_has_ffs() checks the folio state magic instead of
requiring a large folio, since order-0 multi-block folios also carry
a folio state, and the page private flags can legitimately be set
before a state is attached.
- f2fs_ffs_find_or_alloc() cannot fail with __GFP_NOFAIL, so the
allocation failure handling of its callers is removed.
- The inode dirty blocks are counted from the dirty bits of the folio
state, and ->dirty_folio() marks the uptodate blocks of a folio
dirty, since mm does not tell which blocks were modified.
- Defragmentation and swapfile activation mark only the target block
dirty instead of the whole page.
Changes in v2:
Addressed the review comments from Chao Yu and Nanzhe Zhao, the
warning reported by the kernel test robot, and the findings of the
Sashiko review. v1 is at [2].
- Patch 1 (block based calculations): rename the subject ("in more
paths"). Use F2FS_BLK_ALIGN() in dir_blocks(). Compute the new
i_size in __clone_blkaddrs() in 64 bits.
- Patch 2 (superblock and mount): probe the 2nd superblock from the
smallest block size and stop at the first one found, so that a stale
superblock of a larger block size is not picked, and return its
error instead of the error of the last probe. Bound log_sectorsize
by log_blocksize and remove F2FS_MAX_LOG_SECTOR_SIZE. Check the
requested inline_xattr_size in f2fs_check_opt_consistency(). Do not
set *recovery when no valid superblock is found. Report a failed
read of a superblock again.
- Patch 3 (folio state): remove the unused variable in
prepare_large_folio_write_begin(). Count the submitted blocks in
nwritten also on error. Do not clear the uptodate flag of the folio
of an inline inode in f2fs_invalidate_folio(). Explain the use of
__GFP_NOFAIL in the commit message.
- Patch 4 (directories): read the name length in f2fs_delete_entry()
under the folio lock. Clear the gcing flag of the folio when its
last dirty block is freed. Simplify f2fs_dentry_to_lblk() and use
f2fs_folio_lblk_address().
- Patch 5 (dirty counters): commit message only.
- Patch 6 (read path and GC): keep the folio state of a folio that is
not fully uptodate in ->release_folio while the folio has other
users. Read the block again in f2fs_get_read_data_folio() and
f2fs_find_data_folio() when the folio state was released while the
folio was unlocked, instead of failing. Do not map the blocks past
EOF in f2fs_read_data_large_folio(). Read under memalloc_nofs in
ra_data_block(), where GC locks are held. Rename
f2fs_submit_page_read_sync() to f2fs_submit_block_read_sync().
- Patch 7 (write path and writeback): count the inode dirty blocks
from the dirty bits of the folio state instead of per folio. Make
->dirty_folio() mark the uptodate blocks within EOF dirty, and add
f2fs_folio_mark_dirty() for the updates made by f2fs, which mark
only the blocks they modify. Keep f2fs_balance_fs_folio_locked() in
write_begin instead of balancing after the folio is unlocked. Drop
the pending write that a failed submission leaves behind in
f2fs_convert_inline_folio(). Keep the folio dirty and accounted when
flush_inline_data() fails.
- Patch 8 (range operations and mmap): update i_size of an extending
direct write in ->end_io(). In page_mkwrite, reserve blocks only
where it may allocate, i.e. not for a pinned file or a compressed
cluster, do not balance with the folio locked, and read the blocks
that are not uptodate after a partial invalidation. Check the dirty
state of a reserved block only for SEEK_DATA.
- Patch 9 (pinned files): keep failing page_mkwrite on a device
aliasing file with -EIO.
- Patch 10 (collapse and insert range): hold i_gc_rwsem[WRITE] and
invalidate_lock from before the first writeback until the moved
blocks are written back and the page cache is dropped.
- Patch 11 (move range): zero the cached data past the old EOF of the
destination with f2fs_zero_post_eof_page() before the move. Drop the
source blocks from the page cache also after an exchange that failed
part way. Call pagecache_isize_extended() for the gap between the
old EOF and the moved data.
- Patch 12 (defragmentation): read compressed files through
->read_folio() as well, and lock and redirty the folio that was read
instead of looking it up again (f2fs_get_defrag_folio()).
- Patch 13 (swapfile): use f2fs_folio_mark_dirty().
- Patch 14 (fs-verity): stop the data reads of a verity file at i_size
in f2fs_read_data_large_folio(). Do not zero the blocks past i_size
on read while verity is being enabled.
- Patch 15 (atomic writes): hold i_gc_rwsem[WRITE] and invalidate_lock
while aborting atomic writes, except at eviction.
- Patch 16 (inline data): redirty the folio and stop on any error of
the inline writeback in f2fs_write_cache_folios(). Note the effect
on inline inodes with large folios in the commit message.
- Patch 18 (compression): raise the minimum cluster size only when a
page has more than four blocks, reject a smaller compress_log_size
in the mount option and in F2FS_IOC_SET_COMPRESS_OPTION, fail the
open of a file that already has a smaller cluster with -EOPNOTSUPP
instead of failing its inode sanity check, and document it. Dirty
every block of a partially dirty compressed cluster before it is
rewritten. Attach a folio state to the multi-block folios of a
rewritten cluster before they are dirtied. Count a read folio of a
compressed file once in the iostat. Do not cut the range that
redirty_blocks() dirties at EOF. Remove inode_inc_dirty_block() and
inode_dec_dirty_block().
- Patches 17, 19 and 20: no change.
- The patches also use the F2FS_BLK_* helpers in more places, as
suggested in the review.
Testing:
Built every patch for x86_64, arm64 (16KB pages), arm and i386.
Ran the xfstests auto group (excluding the clone, dedupe and thin
groups) on arm64 QEMU with mount options "discard,fsync_mode=strict"
in these combinations of page size / block size:
- 16KB/4KB: default, compression (lz4, zstd, lz4hc, lzo-rle with
64KB clusters and checksum), compression+encryption,
encryption+verity, mode=lfs, atgc
- 4KB/4KB: default, compression, encryption+verity,
compression+encryption, mode=lfs
- 16KB/16KB: default, compression, encryption+verity,
compression+encryption, mode=lfs
plus generic/019, 521, 522 and 780 for each combination, and a 24 hour
power-off recovery test with fsstress for each of the three
combinations.
There were no crashes or hangs, and the power-off recovery tests found
no corruption. Two failures are worth a note, and neither depends on
this series:
- generic/455 failed once on 16KB/4KB with encryption+verity, because
fsync() returned before its node block was written. [3] fixes it.
- generic/551 was killed by the OOM killer in three of the compression
runs. It is killed the same way without this series.
Also ran:
- The xfstests quick group with lockdep (PROVE_LOCKING and
DEBUG_ATOMIC_SLEEP) on 16KB/4KB, default and compression. The only
reports were in generic/574 with compression, where
f2fs_decompress_cluster() may sleep in the read completion. This
happens without this series as well, and [4] fixes it.
- 16 processes writing through mmap at the same time until the
filesystem is full, on a 1GB image with 16KB pages and 4KB blocks,
with and without large folios and with and without mode=lfs, as
asked in the review of v1. They got SIGBUS only after the free
space ran out, the data read back was correct, and fsck found no
error.
- A 32-bit ARM kernel (4KB pages) with files beyond 4GB.
[1] https://lore.kernel.org/linux-f2fs-devel/20261028154407.1601430-1-nzzhao.sigma@gmail.com/
[2] https://lore.kernel.org/linux-f2fs-devel/20261102193952.3002018-1-daeho43@gmail.com/
[3] https://lore.kernel.org/linux-f2fs-devel/20261103101208.4550-1-seungbum.ju@gmail.com/
[4] https://lore.kernel.org/linux-f2fs-devel/20261112143355.2923939-2-jprusakowski@google.com/
Daeho Jeong (20):
f2fs: use block-based index and size calculations in more paths
f2fs: support subpage block size superblock and mount
f2fs: extend f2fs_folio_state for subpage blocks
f2fs: support subpage blocks in directory operations
f2fs: convert inode dirty page counters to dirty block counters
f2fs: support subpage blocks in data read path and GC
f2fs: support subpage blocks in data write path and writeback
f2fs: adapt range operations and mmap for subpage blocks
f2fs: support subpage blocks in pinned files
f2fs: support subpage blocks in collapse and insert range
f2fs: support subpage blocks in move range
f2fs: support subpage blocks in defragmentation
f2fs: support subpage blocks in swapfile activation
f2fs: support subpage blocks in fs-verity enablement
f2fs: support subpage blocks in atomic writes
f2fs: preserve inline data in subpage block mode
f2fs: support subpage blocks in online resize
f2fs: support subpage blocks in compression
f2fs: count compress cache entries in pages for its memory limit
f2fs: enable subpage block size support
Documentation/filesystems/f2fs.rst | 7 +-
fs/f2fs/checkpoint.c | 67 +-
fs/f2fs/compress.c | 569 ++++++---
fs/f2fs/data.c | 1725 ++++++++++++++++++----------
fs/f2fs/dir.c | 126 +-
fs/f2fs/f2fs.h | 237 ++--
fs/f2fs/file.c | 716 ++++++++----
fs/f2fs/gc.c | 103 +-
fs/f2fs/inline.c | 23 +-
fs/f2fs/inode.c | 3 +-
fs/f2fs/node.c | 26 +-
fs/f2fs/segment.c | 14 +-
fs/f2fs/super.c | 180 ++-
fs/f2fs/verity.c | 5 -
include/linux/f2fs_fs.h | 2 +-
15 files changed, 2494 insertions(+), 1309 deletions(-)
base-commit: b9d7379d10d43386b59082da8c9a0b111e054781
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 01/20] f2fs: use block-based index and size calculations in more paths
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
` (18 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Several paths still convert between file offsets and block indices with
PAGE_SHIFT or PAGE_SIZE, assuming that a page holds exactly one
filesystem block. Use the F2FS_BLK_* helpers there so that they stay
correct when the block size is smaller than PAGE_SIZE.
No functional change when the block size equals PAGE_SIZE, except on
32-bit architectures: __complete_revoke_list() now computes the
truncation offset and __clone_blkaddrs() the new i_size in 64 bits, so
that they no longer overflow beyond 4GB.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 5 +++--
fs/f2fs/dir.c | 3 +--
fs/f2fs/file.c | 50 +++++++++++++++++++++++------------------------
fs/f2fs/segment.c | 5 +++--
4 files changed, 32 insertions(+), 31 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 4ae61138a003..aacc92839f61 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1897,8 +1897,9 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
}
got_it:
if (new_i_size && i_size_read(inode) <
- ((loff_t)(index + 1) << PAGE_SHIFT))
- f2fs_i_size_write(inode, ((loff_t)(index + 1) << PAGE_SHIFT));
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1))
+ f2fs_i_size_write(inode,
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1));
return folio;
}
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 70847313ca6a..a3cc3be88738 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -39,8 +39,7 @@ extern struct kmem_cache *f2fs_cf_name_slab;
static unsigned long dir_blocks(struct inode *inode)
{
- return ((unsigned long long) (i_size_read(inode) + PAGE_SIZE - 1))
- >> PAGE_SHIFT;
+ return F2FS_BLK_ALIGN(F2FS_I_SB(inode), i_size_read(inode));
}
static unsigned int dir_buckets(unsigned int level, int dir_level)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 05061254c3b2..f8a444064ec8 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1543,7 +1543,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
if (do_replace[i] || blkaddr[i] == NULL_ADDR) {
struct dnode_of_data dn;
struct node_info ni;
- size_t new_size;
+ loff_t new_size;
pgoff_t ilen;
set_new_dnode(&dn, dst_inode, NULL, NULL, 0);
@@ -1576,7 +1576,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
}
dn.ofs_in_node++;
i++;
- new_size = (loff_t)(dst + i) << PAGE_SHIFT;
+ new_size = F2FS_BLK_TO_BYTES(sbi, dst + i);
if (dst_inode->i_size < new_size)
f2fs_i_size_write(dst_inode, new_size);
} while (--ilen && (do_replace[i] || blkaddr[i] == NULL_ADDR));
@@ -1593,7 +1593,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
return PTR_ERR(fsrc);
src_off = offset_in_folio(fsrc,
- (loff_t)(src + i) << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, src + i));
/*
* For same-inode operations (collapse/insert), src and
@@ -1623,10 +1623,10 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
}
dst_off = offset_in_folio(fdst,
- (loff_t)(dst + i) << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, dst + i));
f2fs_folio_wait_writeback(fdst, true, true);
memcpy_folio(fdst, dst_off, fsrc, src_off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
if (folio_test_large(fdst)) {
if (!f2fs_ffs_find_or_alloc(fdst)) {
f2fs_folio_put(fdst, true);
@@ -1635,15 +1635,15 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
return -ENOMEM;
}
f2fs_ffs_mark_subrange_uptodate(fdst,
- dst_off, PAGE_SIZE);
+ dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
}
folio_mark_dirty(fdst);
if (same_folio && i_size_read(dst_inode) <
- ((loff_t)(dst + i + 1) << PAGE_SHIFT))
+ F2FS_BLK_TO_BYTES(sbi, dst + i + 1))
f2fs_i_size_write(dst_inode,
- ((loff_t)(dst + i + 1) << PAGE_SHIFT));
+ F2FS_BLK_TO_BYTES(sbi, dst + i + 1));
folio_set_f2fs_gcing(fdst);
f2fs_folio_put(fdst, true);
if (!same_folio)
@@ -1715,9 +1715,9 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_lock_context lc;
- pgoff_t nrpages = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
- pgoff_t start = offset >> PAGE_SHIFT;
- pgoff_t end = (offset + len) >> PAGE_SHIFT;
+ pgoff_t nrpages = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t start = F2FS_BYTES_TO_BLK(sbi, offset);
+ pgoff_t end = F2FS_BYTES_TO_BLK(sbi, offset + len);
int ret;
f2fs_balance_fs(sbi, true);
@@ -1988,10 +1988,10 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- pg_start = offset >> PAGE_SHIFT;
- pg_end = (offset + len) >> PAGE_SHIFT;
+ pg_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ pg_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
delta = pg_end - pg_start;
- idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
/* avoid gc operation during block exchange */
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
@@ -3104,10 +3104,10 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
f2fs_balance_fs(sbi, true);
inode_lock(inode);
- pg_start = range->start >> PAGE_SHIFT;
+ pg_start = F2FS_BYTES_TO_BLK(sbi, range->start);
pg_end = min_t(pgoff_t,
- (range->start + range->len) >> PAGE_SHIFT,
- DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE));
+ F2FS_BYTES_TO_BLK(sbi, range->start + range->len),
+ F2FS_BLK_ALIGN(sbi, i_size_read(inode)));
if (is_inode_flag_set(inode, FI_COMPRESS_RELEASED) ||
f2fs_is_atomic_file(inode)) {
@@ -3124,8 +3124,8 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
/* writeback all dirty pages in the range */
err = filemap_write_and_wait_range(inode->i_mapping,
- (loff_t)pg_start << PAGE_SHIFT,
- ((loff_t)pg_end << PAGE_SHIFT) - 1);
+ F2FS_BLK_TO_BYTES(sbi, pg_start),
+ F2FS_BLK_TO_BYTES(sbi, pg_end) - 1);
if (err)
goto out;
@@ -3249,7 +3249,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
unlock_out:
inode_unlock(inode);
if (!err) {
- range->len = (u64)total << PAGE_SHIFT;
+ range->len = F2FS_BLK_TO_BYTES(sbi, total);
stat_inc_defrag_blk_count(sbi, total);
}
return err;
@@ -4354,7 +4354,7 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
while (page_idx < last_idx) {
struct dnode_of_data dn;
@@ -4532,7 +4532,7 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
while (page_idx < last_idx) {
struct dnode_of_data dn;
@@ -4997,7 +4997,7 @@ static int f2fs_ioc_decompress_file(struct file *filp)
if (!atomic_read(&fi->i_compr_blocks))
goto out;
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
@@ -5084,7 +5084,7 @@ static int f2fs_ioc_compress_file(struct file *filp)
set_inode_flag(inode, FI_ENABLE_COMPRESS);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 31ac93a93021..2b0821c2c18b 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -285,6 +285,7 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index,
static void __complete_revoke_list(struct inode *inode, struct list_head *head,
bool revoke)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct revoke_entry *cur, *tmp;
pgoff_t start_index = 0;
bool truncate = is_inode_flag_set(inode, FI_ATOMIC_REPLACE);
@@ -303,7 +304,7 @@ static void __complete_revoke_list(struct inode *inode, struct list_head *head,
}
if (!revoke && truncate)
- f2fs_do_truncate_blocks(inode, start_index * PAGE_SIZE, false);
+ f2fs_do_truncate_blocks(inode, F2FS_BLK_TO_BYTES(sbi, start_index), false);
}
static int __f2fs_commit_atomic_write(struct inode *inode)
@@ -315,7 +316,7 @@ static int __f2fs_commit_atomic_write(struct inode *inode)
struct list_head revoke_list;
block_t blkaddr;
struct dnode_of_data dn;
- pgoff_t len = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ pgoff_t len = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
pgoff_t off = 0, blen, index;
int ret = 0, i;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 02/20] f2fs: support subpage block size superblock and mount
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 01/20] f2fs: use block-based index and size calculations in more paths Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
` (17 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Add the infrastructure to mount F2FS with a block size smaller than
PAGE_SIZE, e.g. 4KB blocks on a 16KB page kernel.
- Add helpers to convert between block indices, folio indices and
offsets within a folio.
- Take the block size from the 1st superblock, which is at the same
offset for every block size. If it is invalid, probe the 2nd
superblock with each supported block size from the smallest, and
stop at the first one found, even if it is invalid, so that a stale
superblock of a larger block size is not picked.
- Allow log_blocksize up to PAGE_SHIFT, and write back only the target
superblock block instead of the whole folio.
- Bound log_sectorsize by log_blocksize, so that
log_sectors_per_block + log_sectorsize cannot wrap around, and remove
F2FS_MAX_LOG_SECTOR_SIZE, which has no user left.
- Check the requested inline_xattr_size, not the current one,
against the block size in f2fs_check_opt_consistency().
- Keep rejecting subpage block sizes, and return -EOPNOTSUPP for the
features that do not support them yet. Later patches add the support
and lift these restrictions.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/compress.c | 2 +
fs/f2fs/f2fs.h | 61 +++++++++++---
fs/f2fs/file.c | 42 ++++++++++
fs/f2fs/super.c | 178 +++++++++++++++++++++++++++++-----------
fs/f2fs/verity.c | 3 +
include/linux/f2fs_fs.h | 2 +-
6 files changed, 226 insertions(+), 62 deletions(-)
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index c7910688c318..0b0f31af6626 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -550,6 +550,8 @@ bool f2fs_is_compress_backend_ready(struct inode *inode)
{
if (!f2fs_compressed_file(inode))
return true;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return false;
return f2fs_cops[F2FS_I(inode)->i_compress_algorithm];
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 50a5f845176d..3f83d01489d4 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1372,11 +1372,11 @@ struct f2fs_io_info {
};
#define F2FS_FIO_LBLK(fio) \
- (((fio)->folio)->index + (fio)->folio_offset)
+ (f2fs_folio_lblk((fio)->sbi, (fio)->folio) + (fio)->folio_offset)
#define F2FS_FIO_BLKCNT(fio) \
((fio)->folio_blkcnt)
#define F2FS_FIO_BIO_OFFSET(fio) \
- ((fio)->folio_offset << PAGE_SHIFT)
+ F2FS_BLK_TO_BYTES((fio)->sbi, (fio)->folio_offset)
#define F2FS_FIO_BIO_SIZE(fio) \
F2FS_BLK_TO_BYTES((fio)->sbi, F2FS_FIO_BLKCNT(fio))
@@ -2326,19 +2326,59 @@ static inline void make_dentry_ptr_block(struct inode *inode,
SIZE_OF_DIR_ENTRY * entries;
}
+static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return F2FS_BYTES_TO_BLK(sbi, folio_size(folio));
+}
+
+static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
+{
+ return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
+}
+
+static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return F2FS_BYTES_TO_BLK(sbi, folio_pos(folio));
+}
+
+static inline pgoff_t f2fs_lblk_to_folio_index(struct f2fs_sb_info *sbi,
+ pgoff_t index)
+{
+ return F2FS_BLK_TO_BYTES(sbi, index) >> PAGE_SHIFT;
+}
+
+static inline pgoff_t f2fs_folio_blk_offset(struct f2fs_sb_info *sbi,
+ const struct folio *folio,
+ pgoff_t index)
+{
+ return index - f2fs_folio_lblk(sbi, folio);
+}
+
+static inline size_t f2fs_folio_byte_offset(struct f2fs_sb_info *sbi,
+ const struct folio *folio,
+ pgoff_t index)
+{
+ return F2FS_BLK_TO_BYTES(sbi, f2fs_folio_blk_offset(sbi, folio, index));
+}
+
+static inline void *f2fs_folio_lblk_address(struct f2fs_sb_info *sbi,
+ struct folio *folio, pgoff_t index)
+{
+ return folio_address(folio) + f2fs_folio_byte_offset(sbi, folio, index);
+}
+
static inline struct f2fs_super_block *F2FS_RAW_SUPER(struct f2fs_sb_info *sbi)
{
return (struct f2fs_super_block *)(sbi->raw_super);
}
-static inline struct f2fs_super_block *F2FS_SUPER_BLOCK(struct folio *folio,
- pgoff_t index)
+static inline struct f2fs_super_block *
+F2FS_SUPER_BLOCK(struct f2fs_sb_info *sbi, struct folio *folio, pgoff_t index)
{
- pgoff_t idx_in_folio = index % folio_nr_pages(folio);
-
return (struct f2fs_super_block *)
- (page_address(folio_page(folio, idx_in_folio)) +
- F2FS_SUPER_OFFSET);
+ (f2fs_folio_lblk_address(sbi, folio, index) + F2FS_SUPER_OFFSET);
}
static inline struct f2fs_checkpoint *F2FS_CKPT(struct f2fs_sb_info *sbi)
@@ -3905,11 +3945,10 @@ static inline bool f2fs_is_time_consistent(struct inode *inode)
static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
bool ret;
if (dsync) {
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
-
spin_lock(&sbi->inode_lock[DIRTY_META]);
ret = list_empty(&F2FS_I(inode)->gdirty_list);
spin_unlock(&sbi->inode_lock[DIRTY_META]);
@@ -3917,7 +3956,7 @@ static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
}
if (!is_inode_flag_set(inode, FI_AUTO_RECOVER) ||
file_keep_isize(inode) ||
- i_size_read(inode) & ~PAGE_MASK)
+ i_size_read(inode) & F2FS_BLKSIZE_MASK(sbi))
return false;
if (!f2fs_is_time_consistent(inode))
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index f8a444064ec8..5a6f0ff9f377 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2207,6 +2207,15 @@ static long f2fs_fallocate(struct file *file, int mode,
goto out;
}
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
+ if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
+ f2fs_is_pinned_file(inode) || f2fs_is_atomic_file(inode) ||
+ f2fs_compressed_file(inode)) {
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+ }
+
ret = file_modified(file);
if (ret)
goto out;
@@ -2455,6 +2464,9 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
loff_t isize;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!(filp->f_mode & FMODE_WRITE))
return -EBADF;
@@ -3262,6 +3274,9 @@ static int f2fs_ioc_defragment(struct file *filp, unsigned long arg)
struct f2fs_defragment range;
int err;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -3484,6 +3499,9 @@ static int f2fs_ioc_move_range(struct file *filp, unsigned long arg)
{
struct f2fs_move_range range;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(file_inode(filp))))
+ return -EOPNOTSUPP;
+
if (copy_from_user(&range, (struct f2fs_move_range __user *)arg,
sizeof(range)))
return -EFAULT;
@@ -3734,6 +3752,9 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
__u32 pin;
int ret = 0;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (get_user(pin, (__u32 __user *)arg))
return -EFAULT;
@@ -4114,6 +4135,9 @@ static int f2fs_ioc_resize_fs(struct file *filp, unsigned long arg)
struct f2fs_sb_info *sbi = F2FS_I_SB(file_inode(filp));
__u64 block_count;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -4131,6 +4155,9 @@ static int f2fs_ioc_enable_verity(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return -EOPNOTSUPP;
+
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (!f2fs_sb_has_verity(F2FS_I_SB(inode))) {
@@ -4311,6 +4338,9 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
int ret;
int writecount;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4506,6 +4536,9 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
unsigned int reserved_blocks = 0;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4827,6 +4860,9 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
struct f2fs_comp_option option;
int ret = 0;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4965,6 +5001,9 @@ static int f2fs_ioc_decompress_file(struct file *filp)
pgoff_t page_idx = 0, last_idx, cluster_idx;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5046,6 +5085,9 @@ static int f2fs_ioc_compress_file(struct file *filp)
pgoff_t page_idx = 0, last_idx, cluster_idx;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 78dbf1e92285..f17bae198fe5 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1613,8 +1613,8 @@ static int f2fs_check_opt_consistency(struct fs_context *fc,
min_size = MIN_INLINE_XATTR_SIZE;
max_size = MAX_INLINE_XATTR_SIZE(F2FS_BLKSIZE(sbi));
- if (F2FS_OPTION(sbi).inline_xattr_size < min_size ||
- F2FS_OPTION(sbi).inline_xattr_size > max_size) {
+ if (F2FS_CTX_INFO(ctx).inline_xattr_size < min_size ||
+ F2FS_CTX_INFO(ctx).inline_xattr_size > max_size) {
f2fs_err(sbi, "inline xattr size is out of range: %d ~ %d",
min_size, max_size);
return -EINVAL;
@@ -4000,7 +4000,7 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
folio_lock(folio);
folio_wait_writeback(folio);
if (update)
- memcpy(F2FS_SUPER_BLOCK(folio, index), F2FS_RAW_SUPER(sbi),
+ memcpy(F2FS_SUPER_BLOCK(sbi, folio, index), F2FS_RAW_SUPER(sbi),
sizeof(struct f2fs_super_block));
folio_mark_dirty(folio);
folio_clear_dirty_for_io(folio);
@@ -4010,9 +4010,10 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
bio = bio_alloc(sbi->sb->s_bdev, 1, opf, GFP_NOFS);
/* it doesn't need to set crypto context for superblock update */
- bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(sbi, folio->index);
+ bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(sbi, index);
- if (!bio_add_folio(bio, folio, folio_size(folio), 0))
+ if (!bio_add_folio(bio, folio, F2FS_BLKSIZE(sbi),
+ f2fs_folio_byte_offset(sbi, folio, index)))
f2fs_bug_on(sbi, 1);
ret = submit_bio_wait(bio);
@@ -4025,7 +4026,7 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
struct folio *folio, pgoff_t index)
{
- struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
+ struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(sbi, folio, index);
struct super_block *sb = sbi->sb;
u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);
u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);
@@ -4107,11 +4108,13 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
res = "internally";
} else {
/*
- * __f2fs_commit_super() will access log_blocksize
- * in SECTOR_FROM_BLOCK(), init it in advance.
+ * __f2fs_commit_super() will access log_blocksize and
+ * blocksize to locate the superblock, init them in
+ * advance.
*/
sbi->log_blocksize =
le32_to_cpu(raw_super->log_blocksize);
+ sbi->blocksize = BIT(sbi->log_blocksize);
err = __f2fs_commit_super(sbi, folio, index, false);
res = err ? "failed" : "done";
}
@@ -4129,7 +4132,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
{
block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main;
block_t total_sections, blocks_per_seg;
- struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
+ struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(sbi, folio, index);
size_t crc_offset = 0;
__u32 crc = 0;
@@ -4155,14 +4158,23 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
}
}
- /* only support block_size equals to PAGE_SIZE */
- if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
- f2fs_info(sbi, "Invalid log_blocksize (%u), supports only %u",
+ /* Keep one or more filesystem blocks in each base-page folio. */
+ if (le32_to_cpu(raw_super->log_blocksize) <
+ F2FS_MIN_LOG_BLOCKSIZE ||
+ le32_to_cpu(raw_super->log_blocksize) > F2FS_MAX_LOG_BLOCKSIZE) {
+ f2fs_info(sbi, "Invalid log_blocksize (%u), supports %u..%u",
le32_to_cpu(raw_super->log_blocksize),
- PAGE_SHIFT);
+ F2FS_MIN_LOG_BLOCKSIZE, F2FS_MAX_LOG_BLOCKSIZE);
return -EFSCORRUPTED;
}
+ /* Subpage block size is enabled after all paths support it. */
+ if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
+ f2fs_info(sbi, "Subpage block size (%u) is not supported yet",
+ le32_to_cpu(raw_super->log_blocksize));
+ return -EOPNOTSUPP;
+ }
+
/* check log blocks per segment */
if (le32_to_cpu(raw_super->log_blocks_per_seg) != 9) {
f2fs_info(sbi, "Invalid log blocks per segment (%u)",
@@ -4172,7 +4184,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
/* Currently, support 512/1024/2048/4096/16K bytes sector size */
if (le32_to_cpu(raw_super->log_sectorsize) >
- F2FS_MAX_LOG_SECTOR_SIZE ||
+ le32_to_cpu(raw_super->log_blocksize) ||
le32_to_cpu(raw_super->log_sectorsize) <
F2FS_MIN_LOG_SECTOR_SIZE) {
f2fs_info(sbi, "Invalid log sectorsize (%u)",
@@ -4181,7 +4193,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
}
if (le32_to_cpu(raw_super->log_sectors_per_block) +
le32_to_cpu(raw_super->log_sectorsize) !=
- F2FS_MAX_LOG_SECTOR_SIZE) {
+ le32_to_cpu(raw_super->log_blocksize)) {
f2fs_info(sbi, "Invalid log sectors per block(%u) log sectorsize(%u)",
le32_to_cpu(raw_super->log_sectors_per_block),
le32_to_cpu(raw_super->log_sectorsize));
@@ -4697,6 +4709,61 @@ static int init_blkz_info(struct f2fs_sb_info *sbi, int devi)
}
#endif
+static int check_super_block(struct f2fs_sb_info *sbi,
+ struct f2fs_super_block *super,
+ int block, int *valid_super_block,
+ struct f2fs_super_block **raw_super)
+{
+ struct folio *folio;
+ int err;
+
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, block), NULL);
+ if (IS_ERR(folio)) {
+ f2fs_err(sbi, "Unable to read %dth superblock", block + 1);
+ return PTR_ERR(folio);
+ }
+
+ if (block) {
+ struct f2fs_super_block *raw =
+ F2FS_SUPER_BLOCK(sbi, folio, block);
+ bool magic_ok = le32_to_cpu(raw->magic) == F2FS_SUPER_MAGIC;
+ u32 log_bs = le32_to_cpu(raw->log_blocksize);
+
+ /* Skip unmatched magic during probing to avoid noisy logs */
+ if (!*raw_super && !magic_ok) {
+ err = -ENOENT;
+ goto out_put;
+ }
+
+ /*
+ * The 2nd superblock was located with sbi->log_blocksize.
+ * Check it before sanity_check_raw_super(), which may write
+ * the superblock back with its own log_blocksize. Leave an
+ * unmatched magic for sanity_check_raw_super() to report.
+ */
+ if (magic_ok && log_bs != sbi->log_blocksize) {
+ f2fs_info(sbi, "Mismatched log_blocksize (%u) in 2nd superblock, expected %u",
+ log_bs, sbi->log_blocksize);
+ err = -EFSCORRUPTED;
+ goto out_put;
+ }
+ }
+
+ err = sanity_check_raw_super(sbi, folio, block);
+ if (!err && !*raw_super) {
+ memcpy(super, F2FS_SUPER_BLOCK(sbi, folio, block),
+ sizeof(*super));
+ *valid_super_block = block;
+ *raw_super = super;
+ sbi->log_blocksize = le32_to_cpu(super->log_blocksize);
+ sbi->blocksize = BIT(sbi->log_blocksize);
+ }
+out_put:
+ folio_put(folio);
+ return err;
+}
+
/*
* Read f2fs raw super block.
* Because we have two copies of super block, so read both of them
@@ -4707,52 +4774,54 @@ static int read_raw_super_block(struct f2fs_sb_info *sbi,
struct f2fs_super_block **raw_super,
int *valid_super_block, int *recovery)
{
- struct super_block *sb = sbi->sb;
- int block;
- struct folio *folio;
struct f2fs_super_block *super;
- int err = 0;
+ int log_bs, err, ret, err2 = -ENOENT;
super = kzalloc_obj(struct f2fs_super_block);
if (!super)
return -ENOMEM;
- for (block = 0; block < 2; block++) {
- folio = read_mapping_folio(sb->s_bdev->bd_mapping, block, NULL);
- if (IS_ERR(folio)) {
- f2fs_err(sbi, "Unable to read %dth superblock",
- block + 1);
- err = PTR_ERR(folio);
- *recovery = 1;
- continue;
- }
+ /* 1. Read 1st superblock (block 0) */
+ err = check_super_block(sbi, super, 0, valid_super_block, raw_super);
+ if (err) {
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 1st superblock");
+ *recovery = 1;
+ }
- /* sanity checking of raw super */
- err = sanity_check_raw_super(sbi, folio, block);
- if (err) {
- f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock",
- block + 1);
- folio_put(folio);
+ /* 2. Read 2nd superblock (block 1) */
+ if (*raw_super) {
+ if (check_super_block(sbi, super, 1, valid_super_block, raw_super)) {
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 2nd superblock");
*recovery = 1;
- continue;
}
-
- if (!*raw_super) {
- memcpy(super, F2FS_SUPER_BLOCK(folio, block),
- sizeof(*super));
- *valid_super_block = block;
- *raw_super = super;
+ } else {
+ for (log_bs = F2FS_MIN_LOG_BLOCKSIZE;
+ log_bs <= F2FS_MAX_LOG_BLOCKSIZE; log_bs++) {
+ sbi->log_blocksize = log_bs;
+ sbi->blocksize = BIT(log_bs);
+ ret = check_super_block(sbi, super, 1, valid_super_block,
+ raw_super);
+ if (!ret)
+ break;
+ /* A 2nd superblock was found: do not probe larger blocks. */
+ if (ret != -ENOENT) {
+ err2 = ret;
+ break;
+ }
}
- folio_put(folio);
+ if (!*raw_super)
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 2nd superblock");
}
- /* No valid superblock */
- if (!*raw_super)
+ /*
+ * No valid superblock: return the error of the 2nd superblock if one
+ * was found, or the error of the 1st superblock otherwise.
+ */
+ if (!*raw_super) {
kfree(super);
- else
- err = 0;
-
- return err;
+ return err2 != -ENOENT ? err2 : err;
+ }
+ return 0;
}
int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
@@ -4777,7 +4846,8 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
/* write back-up superblock first */
index = sbi->valid_super_block ? 0 : 1;
- folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, index), NULL);
if (IS_ERR(folio))
return PTR_ERR(folio);
err = __f2fs_commit_super(sbi, folio, index, true);
@@ -4789,7 +4859,8 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
/* write current valid superblock */
index = sbi->valid_super_block;
- folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, index), NULL);
if (IS_ERR(folio))
return PTR_ERR(folio);
err = __f2fs_commit_super(sbi, folio, index, true);
@@ -5263,6 +5334,13 @@ static int f2fs_fill_super(struct super_block *sb, struct fs_context *fc)
}
mutex_init(&sbi->flush_lock);
+ /*
+ * The 1st superblock is at F2FS_SUPER_OFFSET of block 0 for every
+ * block size, so locate it with the minimum supported block size.
+ */
+ sbi->log_blocksize = F2FS_MIN_LOG_BLOCKSIZE;
+ sbi->blocksize = F2FS_MIN_BLKSIZE;
+
err = read_raw_super_block(sbi, &raw_super, &valid_super_block,
&recovery);
if (err)
diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
index 4a2cea4288f9..af1b7f21c1d8 100644
--- a/fs/f2fs/verity.c
+++ b/fs/f2fs/verity.c
@@ -94,6 +94,9 @@ static int f2fs_begin_enable_verity(struct file *filp)
struct inode *inode = file_inode(filp);
int err;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return -EOPNOTSUPP;
+
if (f2fs_verity_in_progress(inode))
return -EBUSY;
diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h
index 3081702b1ddb..4007f54d7db4 100644
--- a/include/linux/f2fs_fs.h
+++ b/include/linux/f2fs_fs.h
@@ -13,8 +13,8 @@
#define F2FS_SUPER_OFFSET 1024 /* byte-size offset */
#define F2FS_MIN_LOG_SECTOR_SIZE 9 /* 9 bits for 512 bytes */
-#define F2FS_MAX_LOG_SECTOR_SIZE PAGE_SHIFT /* Max is Block Size */
#define F2FS_MIN_LOG_BLOCKSIZE 12
+#define F2FS_MAX_LOG_BLOCKSIZE PAGE_SHIFT
#define F2FS_MIN_BLKSIZE 4096UL
#define F2FS_MAX_BLKSIZE PAGE_SIZE
#define F2FS_MAX_EXTENSION 64 /* # of extension entries */
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 03/20] f2fs: extend f2fs_folio_state for subpage blocks
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 01/20] f2fs: use block-based index and size calculations in more paths Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
` (16 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, even an order-0 folio holds several blocks, so
track uptodate and dirty state per block for any folio that holds more
than one block, not only for large folios.
- Identify the folio state by a magic number. An order-0 folio can carry
one now, so folio_test_large() no longer tells it apart from the
context of a compressed page, which folio->private can also point to.
Carry the page private flags over when the state is attached.
- Make the uptodate and dirty helpers take block indices, and attach the
state on demand in them. Some of their callers cannot return an error,
e.g. f2fs_set_link() and f2fs_delete_entry() once a directory folio
holds several blocks, so allocate the state with __GFP_NOFAIL, as
ifs_alloc() does in iomap, and drop the allocation failure handling
of the callers. The write paths attach the state before a folio is
dirtied, so writeback does not normally have to allocate it.
- Handle per-block state in invalidation, dirtying and writeback.
- Clear PAGECACHE_TAG_TOWRITE as well as PAGECACHE_TAG_DIRTY in
f2fs_clear_page_cache_dirty_tag().
Some of these changes also apply to large folios when the block size
equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 440 ++++++++++++++++++++++++++--------------------
fs/f2fs/f2fs.h | 35 ++--
fs/f2fs/file.c | 18 +-
fs/f2fs/segment.c | 2 +-
4 files changed, 269 insertions(+), 226 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index aacc92839f61..b913d53e96e7 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -96,15 +96,14 @@ struct bio_post_read_ctx {
static bool __ffs_mark_subrange_uptodate(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
- unsigned int start, end;
-
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int start = F2FS_BYTES_TO_BLK(sbi, offset);
+ unsigned int end = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
bitmap_set(ffs->state, start, end - start);
- return bitmap_full(ffs->state, nr_subpages);
+ return bitmap_full(ffs->state, nr_blocks);
}
/*
@@ -122,7 +121,7 @@ static bool __ffs_mark_subrange_uptodate(struct folio *folio,
* release the bio's reference to the decompress_io_ctx of the page's cluster.
*/
/*
- * Update read_pages_pending.
+ * Update read_blocks_pending.
*/
static inline void f2fs_update_read_folio_pending(struct folio *folio, int nr)
{
@@ -134,7 +133,7 @@ static inline void f2fs_update_read_folio_pending(struct folio *folio, int nr)
ffs = (struct f2fs_folio_state *)folio->private;
spin_lock_irqsave(&ffs->state_lock, flags);
- ffs->read_pages_pending += nr;
+ ffs->read_blocks_pending += nr;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
@@ -173,8 +172,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
if (uptodate)
uptodate = __ffs_mark_subrange_uptodate(folio, ffs,
fi.offset, fi.length);
- ffs->read_pages_pending -= nr_pages;
- finished = !ffs->read_pages_pending;
+ ffs->read_blocks_pending -= nr_pages;
+ finished = !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
@@ -372,18 +371,18 @@ bool f2fs_ffs_has_dirty_subpage(struct folio *folio)
{
struct f2fs_folio_state *ffs;
unsigned long flags;
- unsigned int nr_subpages;
+ unsigned int nr_blocks;
bool dirty;
if (!f2fs_folio_has_ffs(folio))
return false;
ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
+ nr_blocks = f2fs_blocks_per_folio(F2FS_F_SB(folio), folio);
spin_lock_irqsave(&ffs->state_lock, flags);
- dirty = find_next_bit(ffs->state, 2 * nr_subpages,
- nr_subpages) < 2 * nr_subpages;
+ dirty = find_next_bit(ffs->state, 2 * nr_blocks,
+ nr_blocks) < 2 * nr_blocks;
spin_unlock_irqrestore(&ffs->state_lock, flags);
return dirty;
@@ -402,7 +401,7 @@ static void f2fs_ffs_put_write_pending(struct folio *folio, unsigned int nr)
goto release;
ffs = (struct f2fs_folio_state *)folio->private;
- if (!atomic_sub_and_test(nr, &ffs->write_pages_pending))
+ if (!atomic_sub_and_test(nr, &ffs->write_blocks_pending))
return;
release:
if (!f2fs_ffs_has_dirty_subpage(folio))
@@ -1572,7 +1571,7 @@ static int f2fs_submit_page_read_sync(struct inode *inode, struct folio *folio,
REQ_OP_READ, true);
/* Wait until all bios have completed (pending drops back to our bias). */
- while (READ_ONCE(ffs->read_pages_pending) != 1)
+ while (READ_ONCE(ffs->read_blocks_pending) != 1)
f2fs_io_schedule_timeout(DEFAULT_SCHEDULE_TIMEOUT);
/* Remove the bias. */
@@ -1693,12 +1692,8 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
if (IS_ERR(folio))
return folio;
- if (folio_test_large(folio)) {
- if (!f2fs_ffs_find_or_alloc(folio)) {
- err = -ENOMEM;
- goto put_err;
- }
- }
+ if (folio_test_large(folio))
+ f2fs_ffs_find_or_alloc(folio);
if (f2fs_lookup_read_extent_cache_block(inode, index,
&dn.data_blkaddr)) {
@@ -1821,12 +1816,8 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
* for a fully uptodate folio; a partially uptodate one cannot be
* rebuilt and will fail the subpage check below.
*/
- if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio)) {
- if (!f2fs_ffs_find_or_alloc(folio)) {
- f2fs_folio_put(folio, true);
- return ERR_PTR(-ENOMEM);
- }
- }
+ if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio))
+ f2fs_ffs_find_or_alloc(folio);
if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, index))) {
f2fs_folio_put(folio, true);
@@ -1862,12 +1853,6 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
return ERR_PTR(-ENOMEM);
}
- if (folio_test_large(folio) && !f2fs_ffs_find_or_alloc(folio)) {
- f2fs_folio_put(folio, true);
- f2fs_put_cache(ientry, true);
- return ERR_PTR(-ENOMEM);
- }
-
set_new_dnode(&dn, inode, ientry, NULL, 0);
err = f2fs_reserve_block(&dn, index);
if (err) {
@@ -2900,29 +2885,43 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio)
{
- struct f2fs_folio_state *ffs;
- unsigned int nr_subpages = folio_nr_pages(folio);
- unsigned long private = (unsigned long)folio->private;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_test_large(folio));
- f2fs_bug_on(F2FS_F_SB(folio),
- test_bit(F2FS_FOLIO_PRIVATE_NOT_POINTER, &private));
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ struct f2fs_folio_state *ffs = folio->private;
+ unsigned int nr_blocks;
+ unsigned long private_flags;
if (f2fs_folio_has_ffs(folio))
- return (struct f2fs_folio_state *)folio->private;
+ return ffs;
- ffs = f2fs_kmalloc(F2FS_F_SB(folio),
- struct_size(ffs, state, BITS_TO_LONGS(2 * nr_subpages)),
- GFP_NOFS | __GFP_ZERO);
- if (!ffs)
- return NULL;
+ /* Carry over the page private flags kept in folio->private. */
+ private_flags = f2fs_folio_get_private_flags(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ ffs = kzalloc(struct_size(ffs, state, BITS_TO_LONGS(2 * nr_blocks)),
+ GFP_NOIO | __GFP_NOFAIL);
+
+ ffs->magic = F2FS_FOLIO_STATE_MAGIC;
spin_lock_init(&ffs->state_lock);
+ ffs->private_flags = private_flags;
if (folio_test_uptodate(folio))
- bitmap_set(ffs->state, 0, nr_subpages);
- if (folio_test_dirty(folio))
- bitmap_set(ffs->state, nr_subpages, nr_subpages);
+ bitmap_set(ffs->state, 0, nr_blocks);
+ if (folio_test_dirty(folio)) {
+ struct inode *inode = folio->mapping->host;
+ pgoff_t last_lblk = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
+ pgoff_t valid_blocks = 0;
+ if (f2fs_verity_in_progress(inode))
+ valid_blocks = nr_blocks;
+ else if (first_lblk < last_lblk)
+ valid_blocks = min_t(pgoff_t, nr_blocks,
+ last_lblk - first_lblk);
+
+ if (valid_blocks)
+ bitmap_set(ffs->state, nr_blocks, valid_blocks);
+ }
+
+ folio_detach_private(folio);
folio_attach_private(folio, ffs);
return ffs;
}
@@ -2940,17 +2939,17 @@ static void ffs_detach_free(struct folio *folio)
if (!ffs)
return;
- WARN_ON_ONCE(ffs->read_pages_pending != 0);
- WARN_ON_ONCE(atomic_read(&ffs->write_pages_pending));
+ WARN_ON_ONCE(ffs->read_blocks_pending != 0);
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending));
+ ffs->magic = 0;
kfree(ffs);
}
bool f2fs_ffs_test_blk_uptodate(const struct folio *folio, pgoff_t index)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
struct f2fs_folio_state *ffs;
- unsigned int idx;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_contains(folio, index));
+ pgoff_t offset;
if (folio_test_uptodate(folio))
return true;
@@ -2958,63 +2957,60 @@ bool f2fs_ffs_test_blk_uptodate(const struct folio *folio, pgoff_t index)
if (!f2fs_folio_has_ffs(folio))
return false;
- ffs = (struct f2fs_folio_state *)folio->private;
- idx = index - folio->index;
- return test_bit(idx, ffs->state);
+ ffs = folio->private;
+ offset = f2fs_folio_blk_offset(sbi, folio, index);
+ if (WARN_ON_ONCE(offset >= f2fs_blocks_per_folio(sbi, folio)))
+ return false;
+
+ if (test_bit(offset, ffs->state)) {
+ /* Order the bit test before reading the block data */
+ smp_rmb();
+ return true;
+ }
+ return false;
}
void f2fs_ffs_mark_subrange_uptodate(struct folio *folio, size_t offset,
size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
unsigned long flags;
bool mark_uptodate = false;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
- if (!f2fs_folio_has_ffs(folio)) {
+ if (!f2fs_is_multiblock_folio(sbi, folio)) {
folio_mark_uptodate(folio);
return;
}
- ffs = (struct f2fs_folio_state *)folio->private;
- spin_lock_irqsave(&ffs->state_lock, flags);
- mark_uptodate = __ffs_mark_subrange_uptodate(folio, ffs, offset, len) &&
- !ffs->read_pages_pending;
- spin_unlock_irqrestore(&ffs->state_lock, flags);
- if (mark_uptodate)
+ /*
+ * Without f2fs_folio_state, the folio uptodate flag covers every block,
+ * so marking the whole folio needs no allocation. This keeps bio
+ * completion paths such as decompression free of allocation.
+ */
+ if (!f2fs_folio_has_ffs(folio) && !offset && len >= folio_size(folio)) {
folio_mark_uptodate(folio);
-}
-
-static void ffs_clear_subrange_uptodate(struct folio *folio,
- size_t offset, size_t len)
-{
- struct f2fs_folio_state *ffs;
- unsigned int nr_subpages, start, end;
- unsigned long flags;
-
- f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
-
- if (!f2fs_folio_has_ffs(folio))
return;
+ }
- ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
-
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
- bitmap_clear(ffs->state, start, end - start);
+ mark_uptodate = __ffs_mark_subrange_uptodate(folio, ffs, offset, len) &&
+ !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
+ if (mark_uptodate)
+ folio_mark_uptodate(folio);
}
static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+
if (f2fs_folio_has_ffs(folio)) {
- ffs_clear_subrange_uptodate(folio,
- (loff_t)i << PAGE_SHIFT, PAGE_SIZE);
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio,
+ F2FS_BLK_TO_BYTES(sbi, i), F2FS_BLKSIZE(sbi));
} else {
folio_clear_uptodate(folio);
folio_clear_f2fs_gcing(folio);
@@ -3023,33 +3019,36 @@ static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
bool f2fs_ffs_test_blk_dirty(const struct folio *folio, pgoff_t index)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
struct f2fs_folio_state *ffs;
- unsigned int idx, nr_subpages;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_contains(folio, index));
+ unsigned int nr_blocks;
+ pgoff_t offset;
if (!f2fs_folio_has_ffs(folio))
return folio_test_dirty(folio);
- ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
- idx = index - folio->index;
- return test_bit(nr_subpages + idx, ffs->state);
+ ffs = folio->private;
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ offset = f2fs_folio_blk_offset(sbi, folio, index);
+ if (WARN_ON_ONCE(offset >= nr_blocks))
+ return false;
+
+ return test_bit(nr_blocks + offset, ffs->state);
}
static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned int start, end, before;
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
- before = bitmap_weight(ffs->state, 2 * nr_subpages);
- bitmap_set(ffs->state, nr_subpages + start, end - start);
- return bitmap_weight(ffs->state, 2 * nr_subpages) - before;
+ before = bitmap_weight(ffs->state, 2 * nr_blocks);
+ bitmap_set(ffs->state, nr_blocks + start, end - start);
+ return bitmap_weight(ffs->state, 2 * nr_blocks) - before;
}
/*
@@ -3062,16 +3061,17 @@ static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
size_t offset, size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
unsigned int set;
unsigned long flags;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
- if (!f2fs_folio_has_ffs(folio))
+ if (!f2fs_is_multiblock_folio(sbi, folio))
return;
- ffs = (struct f2fs_folio_state *)folio->private;
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
set = __ffs_mark_subrange_dirty(folio, ffs, offset, len);
@@ -3081,22 +3081,54 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
f2fs_account_dirty_blocks(folio->mapping->host, set);
}
+void f2fs_ffs_clear_subrange_uptodate(struct folio *folio,
+ size_t offset, size_t len)
+{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ struct f2fs_folio_state *ffs;
+ unsigned int nr_blocks, start, end;
+ unsigned long flags;
+
+ f2fs_bug_on(sbi, offset + len > folio_size(folio));
+
+ if (!f2fs_folio_has_ffs(folio)) {
+ if (!folio_test_uptodate(folio))
+ return;
+ /* Without ffs, clearing the whole folio needs no allocation. */
+ if (!offset && len >= folio_size(folio)) {
+ folio_clear_uptodate(folio);
+ return;
+ }
+ }
+
+ ffs = f2fs_ffs_find_or_alloc(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
+
+ spin_lock_irqsave(&ffs->state_lock, flags);
+ bitmap_clear(ffs->state, start, end - start);
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ if (folio_test_uptodate(folio))
+ folio_clear_uptodate(folio);
+}
+
static bool __ffs_clear_subrange_dirty(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len,
unsigned int *cleared)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned int start, end, before;
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
- before = bitmap_weight(ffs->state, 2 * nr_subpages);
- bitmap_clear(ffs->state, nr_subpages + start, end - start);
- *cleared = before - bitmap_weight(ffs->state, 2 * nr_subpages);
- return find_next_bit(ffs->state, 2 * nr_subpages, nr_subpages) <
- 2 * nr_subpages;
+ before = bitmap_weight(ffs->state, 2 * nr_blocks);
+ bitmap_clear(ffs->state, nr_blocks + start, end - start);
+ *cleared = before - bitmap_weight(ffs->state, 2 * nr_blocks);
+ return find_next_bit(ffs->state, 2 * nr_blocks, nr_blocks) <
+ 2 * nr_blocks;
}
/**
@@ -3117,10 +3149,19 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
unsigned int cleared = 0;
bool dirty;
- if (!f2fs_folio_has_ffs(folio))
- return false;
+ if (!f2fs_folio_has_ffs(folio)) {
+ /*
+ * Without ffs, the folio dirty flag covers every block. Callers
+ * that clear the flag first (writeback, GC) must have ffs.
+ */
+ if (!folio_test_dirty(folio))
+ return false;
+ /* Clearing the whole folio leaves no dirty block. */
+ if (!offset && len >= folio_size(folio))
+ return false;
+ }
- ffs = (struct f2fs_folio_state *)folio->private;
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len, &cleared);
spin_unlock_irqrestore(&ffs->state_lock, flags);
@@ -3131,55 +3172,37 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
return dirty;
}
-static unsigned int ffs_next_dirty_subpage(struct f2fs_folio_state *ffs,
- const struct folio *folio, unsigned int start,
- unsigned int end)
-{
- unsigned int nr_subpages = folio_nr_pages(folio);
-
- return find_next_bit(ffs->state, nr_subpages + end + 1,
- nr_subpages + start) - nr_subpages;
-}
-
-static unsigned int ffs_next_clean_subpage(struct f2fs_folio_state *ffs,
- const struct folio *folio, unsigned int start,
- unsigned int end)
-{
- unsigned int nr_subpages = folio_nr_pages(folio);
-
- return find_next_zero_bit(ffs->state, nr_subpages + end + 1,
- nr_subpages + start) - nr_subpages;
-}
-
-static unsigned int ffs_find_dirty_range(struct folio *folio,
- u64 *range_start, u64 range_end)
+bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
+ unsigned int to, unsigned int *start,
+ unsigned int *count)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
- unsigned int start, end, nr_pages;
-
- if (*range_start >= range_end)
- return 0;
-
- if (!f2fs_folio_has_ffs(folio))
- return range_end - *range_start;
-
- ffs = (struct f2fs_folio_state *)folio->private;
- start = offset_in_folio(folio, *range_start) >> PAGE_SHIFT;
- end = DIV_ROUND_UP(min_not_zero(offset_in_folio(folio, range_end),
- folio_size(folio)), PAGE_SIZE) - 1;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned long flags;
- start = ffs_next_dirty_subpage(ffs, folio, start, end);
- if (start > end)
- return 0;
+ if (from >= to || to > nr_blocks)
+ return false;
- if (start == end)
- nr_pages = 1;
- else
- nr_pages = ffs_next_clean_subpage(ffs, folio,
- start + 1, end) - start;
+ if (!f2fs_folio_has_ffs(folio)) {
+ if (!folio_test_dirty(folio))
+ return false;
+ *start = from;
+ *count = to - from;
+ return true;
+ }
- *range_start = folio_pos(folio) + ((u64)start << PAGE_SHIFT);
- return (u64)nr_pages << PAGE_SHIFT;
+ ffs = folio->private;
+ spin_lock_irqsave(&ffs->state_lock, flags);
+ *start = find_next_bit(ffs->state, nr_blocks + to, nr_blocks + from) - nr_blocks;
+ if (*start >= to) {
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ return false;
+ }
+ *count = find_next_zero_bit(ffs->state, nr_blocks + to,
+ nr_blocks + *start) - nr_blocks - *start;
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ return true;
}
static bool find_next_valid_block(const struct folio *folio,
@@ -3307,13 +3330,9 @@ static int f2fs_read_data_large_folio(struct inode *inode,
/*
* Allocate the folio state before doing anything with the
* folio, so that the block mapping helpers below can rely on
- * it, and fail early if memory is not available.
+ * it.
*/
ffs = f2fs_ffs_find_or_alloc(folio);
- if (!ffs) {
- ret = -ENOMEM;
- goto err_out;
- }
}
index = folio->index;
offset = 0;
@@ -3393,7 +3412,7 @@ static int f2fs_read_data_large_folio(struct inode *inode,
continue;
}
- /* We must increment read_pages_pending before possible BIOs submitting
+ /* We must increment read_blocks_pending before possible BIOs submitting
* to prevent from premature folio_end_read() call on folio
*/
if (folio_test_large(folio)) {
@@ -3847,7 +3866,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
__folio_start_writeback(folio, folio_test_dirty(folio));
if (f2fs_folio_has_ffs(folio)) {
ffs = (struct f2fs_folio_state *)folio->private;
- atomic_inc(&ffs->write_pages_pending);
+ atomic_inc(&ffs->write_blocks_pending);
pending_added = true;
}
@@ -3927,7 +3946,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
if (f2fs_folio_has_ffs(folio)) {
ffs = (struct f2fs_folio_state *)folio->private;
- atomic_inc(&ffs->write_pages_pending);
+ atomic_inc(&ffs->write_blocks_pending);
}
set_new_dnode(&dn, dn_inode, NULL, NULL, 0);
@@ -4499,7 +4518,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
struct f2fs_lock_context lc;
u64 pos = 0;
u64 end_pos = 0;
- u32 r_len = 0;
int err = 0;
int submitted = 0;
int nwritten = 0;
@@ -4633,29 +4651,60 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_start_writeback(folio);
+ /*
+ * PG_dirty was already cleared by writeback_iter(), and a
+ * single-block folio has no ffs dirty bitmap, so the dirty
+ * range cannot be derived. Write the whole folio.
+ */
+ if (!folio_test_large(folio)) {
+ err = f2fs_write_folio_dirty_range(folio, &submitted,
+ wbc, io_type, &pos, end_pos - pos);
+ folio_submitted += submitted;
+ nwritten += submitted;
+ if (err)
+ goto out;
+ goto written;
+ }
+
+ if (!f2fs_folio_has_ffs(folio)) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
+ f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
+ } else {
+ ffs = (struct f2fs_folio_state *)folio->private;
+ }
/*
* Large folios can be submitted as multiple bios, which may
* complete while we are still submitting the remaining dirty
- * ranges. Hold a bias on write_pages_pending to prevent
+ * ranges. Hold a bias on write_blocks_pending to prevent
* folio_end_writeback() from being called early.
*/
- if (!bias_added && f2fs_folio_has_ffs(folio)) {
- ffs = (struct f2fs_folio_state *)folio->private;
- WARN_ON_ONCE(atomic_read(&ffs->write_pages_pending) != 0);
- atomic_inc(&ffs->write_pages_pending);
+ if (!bias_added) {
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
bias_added = true;
}
- while ((r_len = ffs_find_dirty_range(folio, &pos, end_pos))) {
+ unsigned int from = 0, start, count;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int to_blk = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, end_pos - folio_pos(folio)),
+ nr_blocks);
+
+ while (f2fs_ffs_dirty_blk_range(folio, from, to_blk,
+ &start, &count)) {
submitted = 0;
+ pos = folio_pos(folio) + F2FS_BLK_TO_BYTES(sbi, start);
err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos, r_len);
+ wbc, io_type, &pos,
+ F2FS_BLK_TO_BYTES(sbi, count));
folio_submitted += submitted;
nwritten += submitted;
if (err)
goto out;
+ from = start + count;
}
+written:
/* If none of the subpages of a folio is submitted, keep the atomic flag */
if (folio_submitted &&
f2fs_is_atomic_file(inode) &&
@@ -5141,7 +5190,6 @@ static int prepare_large_folio_write_begin(struct inode *inode,
unsigned int len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- struct f2fs_folio_state *ffs;
size_t ori_off = offset_in_folio(folio, pos);
size_t need_off = ori_off;
int err = 0;
@@ -5170,9 +5218,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
}
}
- ffs = f2fs_ffs_find_or_alloc(folio);
- if (!ffs)
- return -ENOMEM;
+ f2fs_ffs_find_or_alloc(folio);
/* Skip read if the folio is already fully uptodate or the write
* covers the entire folio.
@@ -5248,8 +5294,7 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
len = min_t(unsigned int, len, folio_size(folio) - ori_off);
- if (!f2fs_ffs_find_or_alloc(folio))
- return -ENOMEM;
+ f2fs_ffs_find_or_alloc(folio);
/* Inline data must have been converted before reaching here. */
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
@@ -5533,31 +5578,37 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
if (inode->i_ino >= F2FS_ROOT_INO(sbi) &&
(offset || length != folio_size(folio))) {
- size_t clear_start = round_up(offset, PAGE_SIZE);
- size_t clear_end = round_down(offset + length, PAGE_SIZE);
+ size_t clear_start = round_up(offset, F2FS_BLKSIZE(sbi));
+ size_t clear_end = round_down(offset + length, F2FS_BLKSIZE(sbi));
size_t clear_length;
bool dirty;
/*
* If the truncated range falls within a single subpage, no
- * subpage state needs to be cleared.
+ * subpage state needs to be cleared. The folio of an inline
+ * inode has no subpage state; its uptodate flag tells that the
+ * inline data was loaded, so it must not be cleared.
*/
- if (clear_start >= clear_end || !f2fs_folio_has_ffs(folio))
+ if (clear_start >= clear_end ||
+ !f2fs_is_multiblock_folio(sbi, folio) ||
+ f2fs_has_inline_data(inode))
return;
clear_length = clear_end - clear_start;
dirty = f2fs_ffs_clear_subrange_dirty_and_test(folio,
clear_start, clear_length);
- ffs_clear_subrange_uptodate(folio, clear_start, clear_length);
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio, clear_start,
+ clear_length);
/*
* If the truncated subrange happens to clear the remaining
* dirty bitmap of the whole folio, cancel the folio-level
* dirty state.
*/
- if (!dirty && folio_test_dirty(folio))
+ if (!dirty && folio_test_dirty(folio)) {
+ f2fs_clear_page_cache_dirty_tag(folio);
folio_cancel_dirty(folio);
+ }
if (!dirty && !folio_test_writeback(folio))
folio_clear_f2fs_gcing(folio);
return;
@@ -5592,6 +5643,7 @@ static bool f2fs_dirty_data_folio(struct address_space *mapping,
struct folio *folio)
{
struct inode *inode = mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
trace_f2fs_set_page_dirty(folio, DATA);
@@ -5599,13 +5651,13 @@ static bool f2fs_dirty_data_folio(struct address_space *mapping,
bool uptodate = true;
if (f2fs_folio_has_ffs(folio)) {
- struct f2fs_folio_state *ffs =
- (struct f2fs_folio_state *)folio->private;
+ struct f2fs_folio_state *ffs = folio->private;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned long flags;
spin_lock_irqsave(&ffs->state_lock, flags);
- uptodate = bitmap_full(ffs->state, folio_nr_pages(folio)) &&
- !ffs->read_pages_pending;
+ uptodate = bitmap_full(ffs->state, nr_blocks) &&
+ !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
if (uptodate)
@@ -5957,6 +6009,8 @@ void f2fs_clear_page_cache_dirty_tag(struct folio *folio)
xa_lock_irqsave(&mapping->i_pages, flags);
__xa_clear_mark(&mapping->i_pages, folio->index,
PAGECACHE_TAG_DIRTY);
+ __xa_clear_mark(&mapping->i_pages, folio->index,
+ PAGECACHE_TAG_TOWRITE);
xa_unlock_irqrestore(&mapping->i_pages, flags);
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 3f83d01489d4..249c962434b5 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1630,13 +1630,17 @@ static inline void f2fs_clear_bit(unsigned int nr, char *addr);
* folio->private is a wrapped pointer.
*/
+#define F2FS_FOLIO_STATE_MAGIC 0xF5F2FF50
+
struct f2fs_folio_state {
+ u32 magic;
spinlock_t state_lock;
- unsigned int read_pages_pending;
- atomic_t write_pages_pending;
+ unsigned int read_blocks_pending;
+ atomic_t write_blocks_pending;
unsigned long private_flags;
- /* state[0..nr_subpages - 1] tracks uptodate subpages.
- * state[nr_subpages..2 * nr_subpages - 1] tracks dirty subpages.
+ /*
+ * state[0..nr_blocks - 1] tracks uptodate blocks.
+ * state[nr_blocks..2 * nr_blocks - 1] tracks dirty blocks.
*/
unsigned long state[];
};
@@ -2337,6 +2341,12 @@ static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
}
+static inline bool f2fs_is_multiblock_folio(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return folio_size(folio) > F2FS_BLKSIZE(sbi);
+}
+
static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
const struct folio *folio)
{
@@ -2483,18 +2493,9 @@ static inline bool f2fs_folio_has_ffs(const struct folio *folio)
{
unsigned long private = (unsigned long)folio->private;
- if (!folio_test_large(folio) || !private)
+ if (!private || (private & BIT(F2FS_FOLIO_PRIVATE_NOT_POINTER)))
return false;
-
- /*
- * A large folio should always carry f2fs_folio_state rather than
- * non-pointer private flags at its folio->private.
- */
- if (unlikely(test_bit(F2FS_FOLIO_PRIVATE_NOT_POINTER, &private))) {
- f2fs_bug_on(F2FS_F_SB(folio), 1);
- return false;
- }
- return true;
+ return *((u32 *)private) == F2FS_FOLIO_STATE_MAGIC;
}
static inline void clear_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
@@ -4529,6 +4530,10 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio, size_t offset, size_t len
bool f2fs_ffs_has_dirty_subpage(struct folio *folio);
bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
size_t len);
+void f2fs_ffs_clear_subrange_uptodate(struct folio *folio, size_t offset,
+ size_t len);
+bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
+ unsigned int to, unsigned int *start, unsigned int *count);
void f2fs_write_failed(struct inode *inode, loff_t to);
void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);
bool f2fs_release_folio(struct folio *folio, gfp_t wait);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 5a6f0ff9f377..28fccc735a39 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -238,12 +238,6 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
goto out_sem;
}
- if (folio_test_large(folio) && !f2fs_ffs_find_or_alloc(folio)) {
- folio_unlock(folio);
- err = -ENOMEM;
- goto out_sem;
- }
-
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
for (i = 0; i < nr; i++) {
@@ -1628,12 +1622,6 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
memcpy_folio(fdst, dst_off, fsrc, src_off,
F2FS_BLKSIZE(sbi));
if (folio_test_large(fdst)) {
- if (!f2fs_ffs_find_or_alloc(fdst)) {
- f2fs_folio_put(fdst, true);
- if (!same_folio)
- f2fs_folio_put(fsrc, true);
- return -ENOMEM;
- }
f2fs_ffs_mark_subrange_uptodate(fdst,
dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
@@ -4975,16 +4963,12 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
* bitmap as well, otherwise f2fs_write_cache_folios()
* finds no dirty range and skips the whole folio.
*/
- if (!f2fs_ffs_find_or_alloc(folio)) {
- ret = -ENOMEM;
- goto next;
- }
+ f2fs_ffs_find_or_alloc(folio);
f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
}
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
-next:
redirty_idx = folio_next_index(folio);
folio_unlock(folio);
folio_put_refs(folio, 2);
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 2b0821c2c18b..93f165151c0f 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -4193,7 +4193,7 @@ static void do_write_block(struct f2fs_summary *sum, struct f2fs_io_info *fio)
fio->old_blkaddr, fio->new_blkaddr, err);
/*
* For a large folio, the caller rolls back
- * write_pages_pending and ends writeback when the last
+ * write_blocks_pending and ends writeback when the last
* pending write is gone.
*/
if (!f2fs_folio_has_ffs(folio))
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 04/20] f2fs: support subpage blocks in directory operations
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (2 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
` (15 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a directory folio holds several dentry blocks.
Address a dentry block within its folio by its block index, and mark
only the modified dentry block dirty. When an empty dentry block is
freed, clear only the state of that block and keep the folio dirty if
other blocks in it are still dirty. If no dirty block is left, clear the
gcing flag of the folio too.
Also read the name length in f2fs_delete_entry() under the folio lock,
so that the dentry bitmap is not updated with a value read without it.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/dir.c | 123 +++++++++++++++++++++++++++++++++--------------
fs/f2fs/f2fs.h | 2 +
fs/f2fs/inline.c | 6 +--
3 files changed, 91 insertions(+), 40 deletions(-)
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index a3cc3be88738..91929c250085 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -37,6 +37,29 @@ static inline bool f2fs_should_fallback_to_linear(struct inode *dir)
extern struct kmem_cache *f2fs_cf_name_slab;
#endif
+static inline pgoff_t f2fs_dentry_to_lblk(struct f2fs_sb_info *sbi,
+ struct folio *folio, void *de)
+{
+ return f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, de - folio_address(folio));
+}
+
+void f2fs_mark_dentry_folio_dirty(struct inode *dir, struct folio *folio,
+ pgoff_t bidx)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
+
+ if (f2fs_has_subpage_blocks(sbi)) {
+ size_t off = f2fs_folio_byte_offset(sbi, folio, bidx);
+
+ f2fs_ffs_mark_subrange_uptodate(folio, off, F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_dirty(folio, off, F2FS_BLKSIZE(sbi));
+ } else if (!folio_test_uptodate(folio)) {
+ folio_mark_uptodate(folio);
+ }
+ folio_mark_dirty(folio);
+}
+
static unsigned long dir_blocks(struct inode *inode)
{
return F2FS_BLK_ALIGN(F2FS_I_SB(inode), i_size_read(inode));
@@ -189,15 +212,16 @@ static unsigned long dir_block_index(unsigned int level,
}
static struct f2fs_dir_entry *find_in_block(struct inode *dir,
- struct folio *dentry_folio,
+ struct folio *dentry_folio, pgoff_t bidx,
const struct f2fs_filename *fname,
int *max_slots,
bool use_hash)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
void *dentry_blk;
struct f2fs_dentry_ptr d;
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(sbi, dentry_folio, bidx);
make_dentry_ptr_block(dir, &d, dentry_blk);
return f2fs_find_target_dentry(&d, fname, max_slots, use_hash);
@@ -288,7 +312,7 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
unsigned int nbucket, nblock;
unsigned int bidx, end_block, bucket_no;
struct f2fs_dir_entry *de = NULL;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
bool room = false;
int max_slots;
@@ -305,11 +329,11 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
while (bidx < end_block) {
/* no need to allocate new dentry pages to all the indices */
struct folio *dentry_folio;
- dentry_folio = f2fs_find_data_folio(dir, bidx, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(dir, bidx, &next_blkofs);
if (IS_ERR(dentry_folio)) {
if (PTR_ERR(dentry_folio) == -ENOENT) {
room = true;
- bidx = next_pgofs;
+ bidx = next_blkofs;
continue;
} else {
*dentry_block = dentry_folio;
@@ -317,7 +341,8 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
}
}
- de = find_in_block(dir, dentry_folio, fname, &max_slots, use_hash);
+ de = find_in_block(dir, dentry_folio, bidx, fname, &max_slots,
+ use_hash);
if (IS_ERR(de)) {
f2fs_folio_put(dentry_folio, false);
*dentry_block = ERR_CAST(de);
@@ -353,7 +378,7 @@ struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
const struct f2fs_filename *fname,
void **dentry_block)
{
- unsigned long npages = dir_blocks(dir);
+ unsigned long nblocks = dir_blocks(dir);
struct f2fs_dir_entry *de = NULL;
unsigned int max_depth;
unsigned int level;
@@ -369,7 +394,7 @@ struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
goto out;
}
- if (npages == 0)
+ if (nblocks == 0)
goto out;
max_depth = F2FS_I(dir)->i_current_depth;
@@ -467,7 +492,8 @@ void f2fs_set_link(struct inode *dir, struct f2fs_dir_entry *de,
f2fs_folio_wait_writeback(folio, true, true);
de->ino = cpu_to_le32(inode->i_ino);
de->file_type = fs_umode_to_ftype(inode->i_mode);
- folio_mark_dirty(folio);
+ f2fs_mark_dentry_folio_dirty(dir, folio,
+ f2fs_dentry_to_lblk(F2FS_I_SB(dir), folio, de));
}
inode_set_mtime_to_ts(dir, inode_set_ctime_current(dir));
@@ -538,12 +564,12 @@ static int make_empty_dir(struct inode *inode,
if (IS_ERR(dentry_folio))
return PTR_ERR(dentry_folio);
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(inode), dentry_folio, 0);
make_dentry_ptr_block(inode, &d, dentry_blk);
f2fs_do_make_empty_dir(inode, parent, &d);
- folio_mark_dirty(dentry_folio);
+ f2fs_mark_dentry_folio_dirty(inode, dentry_folio, 0);
f2fs_folio_put(dentry_folio, true);
return 0;
}
@@ -736,7 +762,8 @@ int f2fs_add_regular_entry(struct inode *dir, const struct f2fs_filename *fname,
if (IS_ERR(dentry_folio))
return PTR_ERR(dentry_folio);
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(dir),
+ dentry_folio, block);
make_dentry_ptr_block(dir, &d, dentry_blk);
bit_pos = f2fs_room_for_filename(d.bitmap, slots, d.max);
if (bit_pos < d.max)
@@ -763,7 +790,7 @@ int f2fs_add_regular_entry(struct inode *dir, const struct f2fs_filename *fname,
f2fs_update_dentry(ino, mode, &d, &fname->disk_name, fname->hash,
bit_pos);
- folio_mark_dirty(dentry_folio);
+ f2fs_mark_dentry_folio_dirty(dir, dentry_folio, block);
if (inode) {
f2fs_i_pino_write(inode, dir->i_ino);
@@ -900,26 +927,29 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
struct f2fs_dentry_ptr d;
struct folio *folio;
unsigned int bit_pos;
- int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
+ int slots;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
pgoff_t index;
int i;
- f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
+ f2fs_update_time(sbi, REQ_TIME);
- if (F2FS_OPTION(F2FS_I_SB(dir)).fsync_mode == FSYNC_MODE_STRICT)
- f2fs_add_ino_entry(F2FS_I_SB(dir), dir->i_ino, TRANS_DIR_INO);
+ if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT)
+ f2fs_add_ino_entry(sbi, dir->i_ino, TRANS_DIR_INO);
if (f2fs_has_inline_dentry(dir))
return f2fs_delete_inline_entry(dentry,
f2fs_dentry_cache(dentry_block), dir, inode);
folio = f2fs_dentry_folio(dentry_block);
- index = folio->index;
folio_lock(folio);
f2fs_folio_wait_writeback(folio, true, true);
- dentry_blk = folio_address(folio);
+ /* Read the dentry under the folio lock. */
+ slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
+ index = f2fs_dentry_to_lblk(sbi, folio, dentry);
+ dentry_blk = f2fs_folio_lblk_address(sbi, folio, index);
make_dentry_ptr_block(dir, &d, dentry_blk);
bit_pos = dentry - d.dentry;
for (i = 0; i < slots; i++)
@@ -927,18 +957,34 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
/* Let's check and deallocate this dentry page */
bit_pos = find_next_bit_le(d.bitmap, d.max, 0);
- folio_mark_dirty(folio);
+ f2fs_mark_dentry_folio_dirty(dir, folio, index);
if (bit_pos == d.max &&
!f2fs_truncate_hole(dir, index, index + 1)) {
+ if (f2fs_has_subpage_blocks(sbi)) {
+ size_t off = f2fs_folio_byte_offset(sbi, folio, index);
+ bool dirty;
+
+ dirty = f2fs_ffs_clear_subrange_dirty_and_test(folio,
+ off, F2FS_BLKSIZE(sbi));
+ f2fs_ffs_clear_subrange_uptodate(folio, off,
+ F2FS_BLKSIZE(sbi));
+ if (dirty)
+ goto keep_dirty;
+ /* No dirty block is left to be written for GC. */
+ folio_clear_f2fs_gcing(folio);
+ }
f2fs_clear_page_cache_dirty_tag(folio);
folio_clear_dirty_for_io(folio);
- folio_clear_uptodate(folio);
- folio_detach_private(folio);
+ if (!f2fs_has_subpage_blocks(sbi)) {
+ folio_clear_uptodate(folio);
+ folio_detach_private(folio);
+ inode_dec_dirty_block(dir);
+ }
- inode_dec_dirty_block(dir);
f2fs_remove_dirty_inode(dir);
}
+keep_dirty:
f2fs_folio_put(folio, true);
inode_set_mtime_to_ts(dir, inode_set_ctime_current(dir));
@@ -960,20 +1006,21 @@ bool f2fs_empty_dir(struct inode *dir)
return f2fs_empty_inline_dir(dir);
while (bidx < nblock) {
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
struct folio *dentry_folio;
- dentry_folio = f2fs_find_data_folio(dir, bidx, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(dir, bidx, &next_blkofs);
if (IS_ERR(dentry_folio)) {
if (PTR_ERR(dentry_folio) == -ENOENT) {
- bidx = next_pgofs;
+ bidx = next_blkofs;
continue;
} else {
return false;
}
}
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(dir),
+ dentry_folio, bidx);
make_dentry_ptr_block(dir, &d, dentry_blk);
if (bidx == 0)
bit_pos = 2;
@@ -1079,7 +1126,7 @@ int f2fs_fill_dentries(struct dir_context *ctx, struct f2fs_dentry_ptr *d,
static int f2fs_readdir(struct file *file, struct dir_context *ctx)
{
struct inode *inode = file_inode(file);
- unsigned long npages = dir_blocks(inode);
+ unsigned long nblocks = dir_blocks(inode);
void *dentry_blk = NULL;
struct file_ra_state *ra = &file->f_ra;
loff_t start_pos = ctx->pos;
@@ -1088,6 +1135,8 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
unsigned int n = (unsigned long)ctx->pos / entries;
struct f2fs_dentry_ptr d;
struct fscrypt_str fstr = FSTR_INIT(NULL, 0);
+ pgoff_t f_last = nblocks ? f2fs_lblk_to_folio_index(sbi, nblocks - 1) : 0;
+ pgoff_t f_index, ra_pages;
int err = 0;
if (IS_ENCRYPTED(inode)) {
@@ -1105,9 +1154,9 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
goto out_free;
}
- for (; n < npages; ctx->pos = n * entries) {
+ for (; n < nblocks; ctx->pos = n * entries) {
struct folio *dentry_folio;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
/* allow readdir() to be interrupted */
if (fatal_signal_pending(current)) {
@@ -1117,23 +1166,25 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
cond_resched();
/* readahead for multi pages of dir */
- if (npages - n > 1 && !ra_has_index(ra, n))
- page_cache_sync_readahead(inode->i_mapping, ra, file, n,
- min(npages - n, (pgoff_t)MAX_DIR_RA_PAGES));
+ f_index = f2fs_lblk_to_folio_index(sbi, n);
+ ra_pages = f_last - f_index + 1;
+ if (nblocks - n > 1 && !ra_has_index(ra, f_index))
+ page_cache_sync_readahead(inode->i_mapping, ra, file, f_index,
+ min(ra_pages, (pgoff_t)MAX_DIR_RA_PAGES));
- dentry_folio = f2fs_find_data_folio(inode, n, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(inode, n, &next_blkofs);
if (IS_ERR(dentry_folio)) {
err = PTR_ERR(dentry_folio);
if (err == -ENOENT) {
err = 0;
- n = next_pgofs;
+ n = next_blkofs;
continue;
} else {
goto out_free;
}
}
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(sbi, dentry_folio, n);
make_dentry_ptr_block(inode, &d, dentry_blk);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 249c962434b5..876407cbdbb5 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4193,6 +4193,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_blk,
int f2fs_do_tmpfile(struct inode *inode, struct inode *dir,
struct f2fs_filename *fname);
bool f2fs_empty_dir(struct inode *dir);
+void f2fs_mark_dentry_folio_dirty(struct inode *dir, struct folio *folio,
+ pgoff_t bidx);
static inline int f2fs_add_link(struct dentry *dentry, struct inode *inode)
{
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index 49905aa08418..ba0aff8e5643 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -463,10 +463,6 @@ static int f2fs_move_inline_dirents(struct inode *dir,
memcpy(dst.dentry, src.dentry, SIZE_OF_DIR_ENTRY * src.max);
memcpy(dst.filename, src.filename, src.max * F2FS_SLOT_LEN);
- if (!folio_test_uptodate(folio))
- folio_mark_uptodate(folio);
- folio_mark_dirty(folio);
-
/* clear inline dir and flag after data writeback */
f2fs_truncate_inline_inode(dir, ientry, 0);
@@ -484,6 +480,8 @@ static int f2fs_move_inline_dirents(struct inode *dir,
f2fs_i_depth_write(dir, 1);
if (i_size_read(dir) < F2FS_BLKSIZE(F2FS_I_SB(dir)))
f2fs_i_size_write(dir, F2FS_BLKSIZE(F2FS_I_SB(dir)));
+
+ f2fs_mark_dentry_folio_dirty(dir, folio, 0);
out:
f2fs_folio_put(folio, true);
return err;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 05/20] f2fs: convert inode dirty page counters to dirty block counters
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (3 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
` (14 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The per-inode dirty counter is compared against block based thresholds
such as min_fsync_blocks and BLKS_PER_SEG, and it already holds blocks:
its helpers take page counts and convert them. Rename it to
dirty_blocks and make the helpers take block counts, and convert to
pages where a counter is compared with page based values. The
conversions are no-ops when the block size equals PAGE_SIZE.
The callers that account one block for a folio are left as they are,
which is still correct when the block size equals PAGE_SIZE. Later
patches convert them.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/checkpoint.c | 4 ++--
fs/f2fs/data.c | 13 +++++++------
fs/f2fs/f2fs.h | 36 +++++++++++++++++-------------------
fs/f2fs/file.c | 16 ++++++++--------
fs/f2fs/inode.c | 2 +-
fs/f2fs/node.c | 3 ++-
fs/f2fs/super.c | 2 +-
7 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 158d97fc349a..1060ee651756 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -1316,7 +1316,7 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
{
int flag = (type == DIR_INODE) ? FI_DIRTY_DIR : FI_DIRTY_FILE;
- if (get_dirty_pages(inode) || !is_inode_flag_set(inode, flag))
+ if (get_dirty_blocks(inode) || !is_inode_flag_set(inode, flag))
return;
list_del_init(&F2FS_I(inode)->dirty_list);
@@ -1325,7 +1325,7 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
}
/*
- * Account @nr dirty pages (negative when dirtiness is cleared) for @inode.
+ * Account @nr dirty blocks (negative when dirtiness is cleared) for @inode.
*/
void f2fs_account_dirty_blocks(struct inode *inode, int nr)
{
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index b913d53e96e7..ffa2233d66f2 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4279,7 +4279,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
folio_batch_init(&fbatch);
- if (get_dirty_pages(mapping->host) <=
+ if (get_dirty_blocks(mapping->host) <=
SM_I(F2FS_M_SB(mapping))->min_hot_blocks)
set_inode_flag(mapping->host, FI_HOT_DATA);
else
@@ -4524,7 +4524,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
bool op_locked = false;
bool retry = false;
- if (get_dirty_pages(inode) <= SM_I(sbi)->min_hot_blocks)
+ if (get_dirty_blocks(inode) <= SM_I(sbi)->min_hot_blocks)
set_inode_flag(inode, FI_HOT_DATA);
else
clear_inode_flag(inode, FI_HOT_DATA);
@@ -4788,7 +4788,7 @@ static inline bool __should_serialize_io(struct inode *inode,
return true;
if (wbc->sync_mode != WB_SYNC_ALL)
return true;
- if (get_dirty_pages(inode) >= SM_I(F2FS_I_SB(inode))->min_seq_blocks)
+ if (get_dirty_blocks(inode) >= SM_I(F2FS_I_SB(inode))->min_seq_blocks)
return true;
return false;
}
@@ -4827,7 +4827,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
bool locked = false;
/* skip writing if there is no dirty page in this inode */
- if (!get_dirty_pages(inode) && wbc->sync_mode == WB_SYNC_NONE)
+ if (!get_dirty_blocks(inode) && wbc->sync_mode == WB_SYNC_NONE)
return 0;
/* during POR, we don't need to trigger writepage at all. */
@@ -4836,7 +4836,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
if ((S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode)) &&
wbc->sync_mode == WB_SYNC_NONE &&
- get_dirty_pages(inode) < nr_caches_to_skip(sbi, DATA) &&
+ get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
f2fs_available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
@@ -4895,7 +4895,8 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
return ret;
skip_write:
- wbc->pages_skipped += get_dirty_pages(inode);
+ wbc->pages_skipped += get_dirty_blocks(inode) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
update_skipped_write(sbi, wbc);
trace_f2fs_writepages(mapping->host, wbc, DATA);
return 0;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 876407cbdbb5..9cf9583a4428 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -957,7 +957,7 @@ struct f2fs_inode_info {
unsigned long flags[BITS_TO_LONGS(FI_MAX)]; /* use to pass per-file flags */
unsigned int ioprio_hint; /* hint for IO priority */
struct f2fs_rwsem i_sem; /* protect fi info */
- atomic_t dirty_pages; /* # of dirty pages */
+ atomic_t dirty_blocks; /* # of dirty blocks */
f2fs_hash_t chash; /* hash value of given file name */
unsigned int clevel; /* maximum level of given file name */
struct task_struct *task; /* lookup and create consistency */
@@ -2336,6 +2336,8 @@ static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
return F2FS_BYTES_TO_BLK(sbi, folio_size(folio));
}
+#define F2FS_LOG_BLKS_PER_PAGE(sbi) (PAGE_SHIFT - (sbi)->log_blocksize)
+
static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
{
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
@@ -2979,40 +2981,36 @@ static inline void dec_cache_count(struct f2fs_sb_info *sbi, int count_type)
sub_cache_counts(sbi, count_type, 1);
}
-/*
- * Account @nr dirty pages for @inode; one page is
- * (PAGE_SIZE >> log_blocksize) filesystem blocks.
- */
-static inline void inode_add_dirty_blocks(struct inode *inode, int nr)
+static inline void inode_add_dirty_blocks(struct inode *inode,
+ unsigned int nr_blocks)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize);
- atomic_add(blocks, &F2FS_I(inode)->dirty_pages);
+ atomic_add(nr_blocks, &F2FS_I(inode)->dirty_blocks);
if (S_ISDIR(inode->i_mode))
- add_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_DENTS, nr_blocks);
else
- add_cache_counts(sbi, F2FS_DIRTY_DATA, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_DATA, nr_blocks);
if (IS_NOQUOTA(inode))
- add_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_QDATA, nr_blocks);
}
-static inline void inode_sub_dirty_blocks(struct inode *inode, int nr)
+static inline void inode_sub_dirty_blocks(struct inode *inode,
+ unsigned int nr_blocks)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize);
if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
!S_ISLNK(inode->i_mode))
return;
- atomic_sub(blocks, &F2FS_I(inode)->dirty_pages);
+ atomic_sub(nr_blocks, &F2FS_I(inode)->dirty_blocks);
if (S_ISDIR(inode->i_mode))
- sub_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_DENTS, nr_blocks);
else
- sub_cache_counts(sbi, F2FS_DIRTY_DATA, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_DATA, nr_blocks);
if (IS_NOQUOTA(inode))
- sub_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_QDATA, nr_blocks);
}
static inline void inode_inc_dirty_block(struct inode *inode)
@@ -3052,9 +3050,9 @@ static inline s64 get_nr_caches(struct f2fs_sb_info *sbi, int count_type)
return atomic_read(&sbi->nr_caches[count_type]);
}
-static inline int get_dirty_pages(struct inode *inode)
+static inline int get_dirty_blocks(struct inode *inode)
{
- return atomic_read(&F2FS_I(inode)->dirty_pages);
+ return atomic_read(&F2FS_I(inode)->dirty_blocks);
}
static inline int get_blocktype_secs(struct f2fs_sb_info *sbi, int block_type)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 28fccc735a39..e0d246397ad0 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -376,7 +376,7 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
goto go_write;
/* if fdatasync is triggered, let's do in-place-update */
- if (datasync || get_dirty_pages(inode) <= SM_I(sbi)->min_fsync_blocks)
+ if (datasync || get_dirty_blocks(inode) <= SM_I(sbi)->min_fsync_blocks)
set_inode_flag(inode, FI_NEED_IPU);
ret = file_write_and_wait_range(file, start, end);
clear_inode_flag(inode, FI_NEED_IPU);
@@ -2495,9 +2495,9 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
* Should wait end_io to count F2FS_WB_CP_DATA correctly by
* f2fs_is_atomic_file.
*/
- if (get_dirty_pages(inode))
- f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%llu, npages=%u",
- inode->i_ino, get_dirty_pages(inode));
+ if (get_dirty_blocks(inode))
+ f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%llu, nblocks=%u",
+ inode->i_ino, get_dirty_blocks(inode));
ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
if (ret)
goto out_unlock;
@@ -2519,7 +2519,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
F2FS_I(fi->cow_inode)->atomic_inode = inode;
} else {
/* Reuse the already created COW inode */
- f2fs_bug_on(sbi, get_dirty_pages(fi->cow_inode));
+ f2fs_bug_on(sbi, get_dirty_blocks(fi->cow_inode));
invalidate_mapping_pages(fi->cow_inode->i_mapping, 0, -1);
@@ -4877,7 +4877,7 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
goto out;
}
- if (f2fs_is_mmap_file(inode) || get_dirty_pages(inode)) {
+ if (f2fs_is_mmap_file(inode) || get_dirty_blocks(inode)) {
ret = -EBUSY;
goto out;
}
@@ -5033,7 +5033,7 @@ static int f2fs_ioc_decompress_file(struct file *filp)
if (ret < 0)
break;
- if (get_dirty_pages(inode) >= BLKS_PER_SEG(sbi)) {
+ if (get_dirty_blocks(inode) >= BLKS_PER_SEG(sbi)) {
ret = filemap_fdatawrite(inode->i_mapping);
if (ret < 0)
break;
@@ -5123,7 +5123,7 @@ static int f2fs_ioc_compress_file(struct file *filp)
if (ret < 0)
break;
- if (get_dirty_pages(inode) >= BLKS_PER_SEG(sbi)) {
+ if (get_dirty_blocks(inode) >= BLKS_PER_SEG(sbi)) {
ret = filemap_fdatawrite(inode->i_mapping);
if (ret < 0)
break;
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index f5a4dac98ef2..5c6a64b31b05 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -872,7 +872,7 @@ static void f2fs_pre_evict_inode(struct inode *inode)
test_opt(sbi, COMPRESS_CACHE) && f2fs_compressed_file(inode))
f2fs_invalidate_compress_pages(sbi, inode->i_ino);
- f2fs_bug_on(sbi, get_dirty_pages(inode));
+ f2fs_bug_on(sbi, get_dirty_blocks(inode));
f2fs_remove_dirty_inode(inode);
f2fs_remove_donate_inode(inode);
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index e53871f32c35..cb8affb2c1e0 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -85,7 +85,8 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
} else if (type == DIRTY_DENTS) {
if (bdi_wb_dirty_exceeded(sbi->sb->s_bdi))
return false;
- mem_size = get_nr_caches(sbi, F2FS_DIRTY_DENTS);
+ mem_size = get_nr_caches(sbi, F2FS_DIRTY_DENTS) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 1);
} else if (type == INO_ENTRIES) {
int i;
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index f17bae198fe5..9b01ead00edc 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1855,7 +1855,7 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)
init_once((void *) fi);
/* Initialize f2fs-specific inode info */
- atomic_set(&fi->dirty_pages, 0);
+ atomic_set(&fi->dirty_blocks, 0);
atomic_set(&fi->i_compr_blocks, 0);
atomic_set(&fi->open_count, 0);
atomic_set(&fi->writeback, 0);
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 06/20] f2fs: support subpage blocks in data read path and GC
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (4 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
` (13 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a folio can be partially uptodate, so reads and GC
have to work per block.
- Read only the non-uptodate blocks of a multi-block folio, and send
every inode to this path in subpage mode.
- Do not use the folio state when a single bio reads the whole folio.
- Implement ->is_partially_uptodate.
- Convert block indices to folio indices in the data folio getters and
GC, and prefetch the whole folio in GC so that its blocks are read
together.
- Fix error handling in f2fs_read_data_large_folio().
- Zero the blocks past EOF in f2fs_read_data_large_folio() instead of
reading them, as f2fs_read_single_page() does. With subpage blocks,
they share a page with the last block of the file and can be seen
through mmap.
- Test the per-block uptodate state in f2fs_find_data_folio() under the
folio lock, as ->release_folio can free the folio state otherwise. A
fully uptodate folio is still returned without the lock.
- Keep the folio state of a folio that is not fully uptodate in
->release_folio while the folio has other users. Otherwise a block
that a lockless reader, e.g. a dentry lookup, is using can be read
from disk again.
- Get sbi from the folio in f2fs_finish_read_bio() only after the
compressed page check, as nothing pins the inode of a compressed page
once its cluster is unlocked.
- Rename helpers that now take block indices, e.g.
f2fs_submit_page_read() to f2fs_submit_block_read().
The read changes also apply to large folios when the block size equals
PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 434 ++++++++++++++++++++++++++++++-------------------
fs/f2fs/f2fs.h | 20 ++-
fs/f2fs/file.c | 22 +--
fs/f2fs/gc.c | 87 +++++++---
fs/f2fs/node.c | 6 +-
5 files changed, 358 insertions(+), 211 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index ffa2233d66f2..971d74d2f010 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -145,13 +145,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
bio_for_each_folio_all(fi, bio) {
struct folio *folio = fi.folio;
- /*
- * Cache sbi before the pending count is dropped: a sync reader
- * may then release the folio while this bio is still in its
- * end_io, so F2FS_F_SB(folio) is no longer safe to use below.
- */
- struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- unsigned nr_pages = fi.length >> PAGE_SHIFT;
+ struct f2fs_sb_info *sbi;
+ unsigned int nr_blocks;
bool finished = true;
bool uptodate = bio->bi_status == BLK_STS_OK;
@@ -164,6 +159,10 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
continue;
}
+ sbi = F2FS_F_SB(folio);
+ nr_blocks = F2FS_BYTES_TO_BLK(sbi, fi.length);
+ sub_cache_counts(sbi, F2FS_RD_DATA, nr_blocks);
+
if (f2fs_folio_has_ffs(folio)) {
struct f2fs_folio_state *ffs =
(struct f2fs_folio_state *)folio->private;
@@ -172,14 +171,11 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
if (uptodate)
uptodate = __ffs_mark_subrange_uptodate(folio, ffs,
fi.offset, fi.length);
- ffs->read_blocks_pending -= nr_pages;
+ ffs->read_blocks_pending -= nr_blocks;
finished = !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
- while (nr_pages--)
- dec_cache_count(sbi, F2FS_RD_DATA);
-
if (finished)
folio_end_read(folio, uptodate);
}
@@ -1477,7 +1473,7 @@ void f2fs_submit_cache_write(struct f2fs_io_info *fio)
static struct bio *f2fs_grab_read_bio(struct inode *inode,
struct fsverity_info *vi, block_t blkaddr,
- unsigned nr_pages, blk_opf_t op_flag,
+ unsigned int nr_blocks, blk_opf_t op_flag,
pgoff_t first_idx, bool for_write)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1487,7 +1483,7 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode,
sector_t sector;
struct block_device *bdev = f2fs_target_device(sbi, blkaddr, §or);
- bio = bio_alloc_bioset(bdev, bio_max_segs(nr_pages),
+ bio = bio_alloc_bioset(bdev, bio_max_segs(nr_blocks),
REQ_OP_READ | op_flag,
for_write ? GFP_NOIO : GFP_KERNEL, &f2fs_bioset);
bio->bi_iter.bi_sector = sector;
@@ -1521,27 +1517,29 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode,
}
/* This can handle encryption stuffs */
-static void f2fs_submit_page_read(struct inode *inode, struct fsverity_info *vi,
- struct folio *folio, pgoff_t index,
- block_t blkaddr, blk_opf_t op_flags,
- bool for_write)
+static void f2fs_submit_block_read(struct inode *inode,
+ struct fsverity_info *vi,
+ struct folio *folio, pgoff_t bidx,
+ block_t blkaddr, blk_opf_t op_flags,
+ bool for_write)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct bio *bio;
size_t offset = 0;
- if (f2fs_folio_has_ffs(folio)) {
- offset = offset_in_folio(folio, (loff_t)index << PAGE_SHIFT);
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ f2fs_ffs_find_or_alloc(folio);
+ offset = f2fs_folio_byte_offset(sbi, folio, bidx);
f2fs_update_read_folio_pending(folio, 1);
}
- bio = f2fs_grab_read_bio(inode, vi, blkaddr, 1, op_flags, index,
+ bio = f2fs_grab_read_bio(inode, vi, blkaddr, 1, op_flags, bidx,
for_write);
/* wait for GCed page writeback via generic cache */
f2fs_wait_on_block_writeback(inode, blkaddr);
- if (!bio_add_folio(bio, folio, PAGE_SIZE, offset))
+ if (!bio_add_folio(bio, folio, F2FS_BLKSIZE(sbi), offset))
f2fs_bug_on(sbi, 1);
inc_cache_count(sbi, F2FS_RD_DATA);
@@ -1550,12 +1548,12 @@ static void f2fs_submit_page_read(struct inode *inode, struct fsverity_info *vi,
}
/*
- * Synchronously read a single 4K subpage by reusing f2fs_submit_page_read()
+ * Synchronously read a single block by reusing f2fs_submit_block_read()
* so that iostat, trace, blk-crypto and post-read handling are all preserved.
* The caller must have already allocated ffs for the folio.
*/
-static int f2fs_submit_page_read_sync(struct inode *inode, struct folio *folio,
- pgoff_t index, block_t blkaddr)
+static int f2fs_submit_block_read_sync(struct inode *inode, struct folio *folio,
+ pgoff_t index, block_t blkaddr)
{
struct f2fs_folio_state *ffs =
(struct f2fs_folio_state *)folio->private;
@@ -1567,8 +1565,8 @@ static int f2fs_submit_page_read_sync(struct inode *inode, struct folio *folio,
* We are holding the folio lock, so use GFP_NOIO for the bio
* allocation to avoid entering mm reclaim.
*/
- f2fs_submit_page_read(inode, NULL, folio, index, blkaddr,
- REQ_OP_READ, true);
+ f2fs_submit_block_read(inode, NULL, folio, index, blkaddr,
+ REQ_OP_READ, true);
/* Wait until all bios have completed (pending drops back to our bias). */
while (READ_ONCE(ffs->read_blocks_pending) != 1)
@@ -1680,24 +1678,23 @@ static inline struct fsverity_info *f2fs_need_verity(const struct inode *inode,
return NULL;
}
-struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
- blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs)
+struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t bidx,
+ blk_opf_t op_flags, bool for_write, pgoff_t *next_blkofs)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, bidx);
struct dnode_of_data dn;
struct folio *folio;
int err;
- folio = f2fs_grab_cache_folio(mapping, index, for_write);
+ folio = f2fs_grab_cache_folio(mapping, f_index, for_write);
if (IS_ERR(folio))
return folio;
- if (folio_test_large(folio))
- f2fs_ffs_find_or_alloc(folio);
-
- if (f2fs_lookup_read_extent_cache_block(inode, index,
+ if (f2fs_lookup_read_extent_cache_block(inode, bidx,
&dn.data_blkaddr)) {
- if (!f2fs_is_valid_blkaddr(F2FS_I_SB(inode), dn.data_blkaddr,
+ if (!f2fs_is_valid_blkaddr(sbi, dn.data_blkaddr,
DATA_GENERIC_ENHANCE_READ)) {
err = -EFSCORRUPTED;
goto put_err;
@@ -1706,29 +1703,29 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
}
set_new_dnode(&dn, inode, NULL, NULL, 0);
- err = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
+ err = f2fs_get_dnode_of_data(&dn, bidx, LOOKUP_NODE);
if (err) {
- if (err == -ENOENT && next_pgofs)
- *next_pgofs = f2fs_get_next_page_offset(&dn, index);
+ if (err == -ENOENT && next_blkofs)
+ *next_blkofs = f2fs_get_next_block_offset(&dn, bidx);
goto put_err;
}
f2fs_put_dnode(&dn);
if (unlikely(dn.data_blkaddr == NULL_ADDR)) {
err = -ENOENT;
- if (next_pgofs)
- *next_pgofs = index + 1;
+ if (next_blkofs)
+ *next_blkofs = bidx + 1;
goto put_err;
}
if (dn.data_blkaddr != NEW_ADDR &&
- !f2fs_is_valid_blkaddr(F2FS_I_SB(inode),
+ !f2fs_is_valid_blkaddr(sbi,
dn.data_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
goto put_err;
}
got_it:
- if (f2fs_ffs_test_blk_uptodate(folio, index)) {
+ if (f2fs_ffs_test_blk_uptodate(folio, bidx)) {
folio_unlock(folio);
return folio;
}
@@ -1741,17 +1738,17 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
* f2fs_init_inode_metadata.
*/
if (dn.data_blkaddr == NEW_ADDR) {
- size_t offset = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t offset = f2fs_folio_byte_offset(sbi, folio, bidx);
- folio_zero_segment(folio, offset, offset + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, offset, PAGE_SIZE);
+ folio_zero_segment(folio, offset, offset + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, offset, F2FS_BLKSIZE(sbi));
folio_unlock(folio);
return folio;
}
- f2fs_submit_page_read(inode, f2fs_need_verity(inode, index),
- folio, index, dn.data_blkaddr, op_flags, for_write);
+ f2fs_submit_block_read(inode, f2fs_need_verity(inode, bidx),
+ folio, bidx, dn.data_blkaddr, op_flags,
+ for_write);
return folio;
put_err:
@@ -1759,32 +1756,53 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
return ERR_PTR(err);
}
-struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
- pgoff_t *next_pgofs)
+struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t bidx,
+ pgoff_t *next_blkofs)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, bidx);
struct folio *folio;
- folio = f2fs_filemap_get_folio(mapping, index, FGP_ACCESSED, 0);
+ folio = f2fs_filemap_get_folio(mapping, f_index, FGP_ACCESSED, 0);
if (IS_ERR(folio))
goto read;
if (folio_test_uptodate(folio))
return folio;
- f2fs_folio_put(folio, false);
+ /* The folio state is freed under the folio lock. */
+ folio_lock(folio);
+ if (folio->mapping == mapping &&
+ f2fs_ffs_test_blk_uptodate(folio, bidx)) {
+ folio_unlock(folio);
+ return folio;
+ }
+ f2fs_folio_put(folio, true);
read:
- folio = f2fs_get_read_data_folio(inode, index, 0, false, next_pgofs);
+ folio = f2fs_get_read_data_folio(inode, bidx, 0, false, next_blkofs);
if (IS_ERR(folio))
return folio;
if (folio_test_uptodate(folio))
return folio;
- folio_wait_locked(folio);
- if (unlikely(!folio_test_uptodate(folio))) {
- f2fs_folio_put(folio, false);
+ folio_lock(folio);
+ if (unlikely(folio->mapping != mapping ||
+ !f2fs_ffs_test_blk_uptodate(folio, bidx))) {
+ /*
+ * Reclaim can free the folio state of a clean folio while it
+ * is unlocked, losing the uptodate state of the block.
+ */
+ bool freed = folio->mapping == mapping &&
+ f2fs_is_multiblock_folio(sbi, folio) &&
+ !f2fs_folio_has_ffs(folio);
+
+ f2fs_folio_put(folio, true);
+ if (freed)
+ goto read;
return ERR_PTR(-EIO);
}
+ folio_unlock(folio);
return folio;
}
@@ -1793,13 +1811,15 @@ struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
* Because, the callers, functions in dir.c and GC, should be able to know
* whether this page exists or not.
*/
-struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
+struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t bidx,
bool for_write)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
struct folio *folio;
- folio = f2fs_get_read_data_folio(inode, index, 0, for_write, NULL);
+repeat:
+ folio = f2fs_get_read_data_folio(inode, bidx, 0, for_write, NULL);
if (IS_ERR(folio))
return folio;
@@ -1812,15 +1832,17 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
}
/*
- * ffs can be released while the folio was unlocked above. Rebuild it
- * for a fully uptodate folio; a partially uptodate one cannot be
- * rebuilt and will fail the subpage check below.
+ * ffs can be released while the folio was unlocked above, losing the
+ * uptodate state of the blocks of a partially uptodate folio. Read the
+ * block again in that case.
*/
- if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio))
- f2fs_ffs_find_or_alloc(folio);
+ if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, bidx))) {
+ bool freed = f2fs_is_multiblock_folio(sbi, folio) &&
+ !f2fs_folio_has_ffs(folio);
- if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, index))) {
f2fs_folio_put(folio, true);
+ if (freed)
+ goto repeat;
return ERR_PTR(-EIO);
}
return folio;
@@ -1838,12 +1860,14 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
struct folio *f2fs_get_new_data_folio(struct inode *inode,
struct f2fs_cached_block *ientry, pgoff_t index, bool new_i_size)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, index);
struct folio *folio;
struct dnode_of_data dn;
int err;
- folio = f2fs_grab_cache_folio(mapping, index, true);
+ folio = f2fs_grab_cache_folio(mapping, f_index, true);
if (IS_ERR(folio)) {
/*
* before exiting, we should make sure ientry will be released
@@ -1866,25 +1890,24 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
goto got_it;
if (dn.data_blkaddr == NEW_ADDR) {
- size_t off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t off = f2fs_folio_byte_offset(sbi, folio, index);
- folio_zero_segment(folio, off, off + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, off, PAGE_SIZE);
+ folio_zero_segment(folio, off, off + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, off, F2FS_BLKSIZE(sbi));
} else {
f2fs_folio_put(folio, true);
/* if ientry exists, blkaddr should be NEW_ADDR */
- f2fs_bug_on(F2FS_I_SB(inode), ientry);
+ f2fs_bug_on(sbi, ientry);
folio = f2fs_get_lock_data_folio(inode, index, true);
if (IS_ERR(folio))
return folio;
}
got_it:
if (new_i_size && i_size_read(inode) <
- F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1))
+ F2FS_BLK_TO_BYTES(sbi, index + 1))
f2fs_i_size_write(inode,
- F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1));
+ F2FS_BLK_TO_BYTES(sbi, index + 1));
return folio;
}
@@ -1965,7 +1988,7 @@ int f2fs_get_block_locked(struct dnode_of_data *dn, pgoff_t index)
static int f2fs_map_no_dnode(struct inode *inode,
struct f2fs_map_blocks *map, struct dnode_of_data *dn,
- pgoff_t pgoff)
+ pgoff_t blkofs)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1978,10 +2001,10 @@ static int f2fs_map_no_dnode(struct inode *inode,
(is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN) || f2fs_cp_error(sbi)))
return -EIO;
- if (map->m_next_pgofs)
- *map->m_next_pgofs = f2fs_get_next_page_offset(dn, pgoff);
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = f2fs_get_next_block_offset(dn, blkofs);
if (map->m_next_extent)
- *map->m_next_extent = f2fs_get_next_page_offset(dn, pgoff);
+ *map->m_next_extent = f2fs_get_next_block_offset(dn, blkofs);
return 0;
}
@@ -2188,19 +2211,19 @@ int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, int flag)
goto sync_out;
case F2FS_GET_BLOCK_FIEMAP:
if (blkaddr == NULL_ADDR) {
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
goto sync_out;
}
break;
case F2FS_GET_BLOCK_DIO:
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
break;
default:
/* for defragment case */
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
goto sync_out;
}
}
@@ -2341,7 +2364,7 @@ static bool __f2fs_overwrite_io(struct inode *inode, loff_t pos, size_t len,
return false;
map.m_lblk = F2FS_BYTES_TO_BLK(F2FS_I_SB(inode), pos);
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -2444,7 +2467,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_map_blocks map;
sector_t start_blk, last_blk, blk_len, max_len;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
u64 logical = 0, phys = 0, size = 0;
u32 flags = 0;
int ret = 0;
@@ -2494,7 +2517,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
memset(&map, 0, sizeof(map));
map.m_lblk = start_blk;
map.m_len = blk_len;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = NO_CHECK_TYPE;
if (compr_cluster) {
@@ -2508,7 +2531,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
/* HOLE */
if (!compr_cluster && !(map.m_flags & F2FS_MAP_FLAGS)) {
- start_blk = next_pgofs;
+ start_blk = next_blkofs;
if (F2FS_BLK_TO_BYTES(sbi, start_blk) < maxbytes)
goto prep_next;
@@ -2644,7 +2667,7 @@ static int f2fs_read_single_page(struct inode *inode, struct fsverity_info *vi,
if (block_in_file > map->m_lblk &&
block_in_file < (map->m_lblk + map->m_len))
goto got_it;
- } else if (block_in_file < *map->m_next_pgofs) {
+ } else if (block_in_file < *map->m_next_blkofs) {
goto got_it;
}
@@ -3209,23 +3232,26 @@ static bool find_next_valid_block(const struct folio *folio,
size_t orig_off, size_t *need_off,
size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
size_t start = orig_off;
size_t end = start + len;
size_t head, tail;
pgoff_t index;
- if (start & (PAGE_SIZE - 1)) {
- head = round_down(start, PAGE_SIZE);
- index = folio->index + (head >> PAGE_SHIFT);
+ if (start & F2FS_BLKSIZE_MASK(sbi)) {
+ head = round_down(start, F2FS_BLKSIZE(sbi));
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, head);
if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
*need_off = head;
return true;
}
}
- if (end & (PAGE_SIZE - 1)) {
- tail = round_down(end, PAGE_SIZE);
- index = folio->index + (tail >> PAGE_SHIFT);
+ if (end & F2FS_BLKSIZE_MASK(sbi)) {
+ tail = round_down(end, F2FS_BLKSIZE(sbi));
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, tail);
if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
*need_off = tail;
return true;
@@ -3248,26 +3274,34 @@ static unsigned int ffs_next_nonuptodate_subpage(struct f2fs_folio_state *ffs,
}
static void f2fs_skip_fully_uptodate_front(struct folio *folio,
- pgoff_t *index, pgoff_t *offset, unsigned int *nrpages,
- unsigned int *max_nr_pages)
+ pgoff_t *index, pgoff_t *offset, unsigned int *nr_blocks,
+ unsigned int *max_nr_blocks)
{
struct f2fs_folio_state *ffs;
unsigned int next, skipped;
- if (!f2fs_folio_has_ffs(folio) || !*nrpages)
+ if (folio_test_uptodate(folio)) {
+ *index += *nr_blocks;
+ *offset += *nr_blocks;
+ *max_nr_blocks -= *nr_blocks;
+ *nr_blocks = 0;
+ return;
+ }
+
+ if (!f2fs_folio_has_ffs(folio) || !*nr_blocks)
return;
ffs = (struct f2fs_folio_state *)folio->private;
next = ffs_next_nonuptodate_subpage(ffs, *offset,
- *offset + *nrpages - 1);
+ *offset + *nr_blocks - 1);
skipped = next - *offset;
if (!skipped)
return;
*index += skipped;
*offset += skipped;
- *nrpages -= skipped;
- *max_nr_pages -= skipped;
+ *nr_blocks -= skipped;
+ *max_nr_blocks -= skipped;
}
static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
@@ -3278,7 +3312,7 @@ static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
struct f2fs_folio_state *ffs;
unsigned int next, end;
- if (!f2fs_folio_has_ffs(folio) || *len_blks <= 1)
+ if (*len_blks <= 1)
return;
if (f2fs_is_multi_device(sbi)) {
@@ -3289,8 +3323,11 @@ static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
}
*len_blks = min_t(unsigned int, *len_blks,
(unsigned int)fscrypt_limit_io_blocks(inode,
- folio->index + offset,
- *len_blks));
+ f2fs_folio_lblk(sbi, folio) + offset,
+ *len_blks));
+
+ if (!f2fs_folio_has_ffs(folio) || *len_blks <= 1)
+ return;
ffs = (struct f2fs_folio_state *)folio->private;
end = offset + *len_blks - 1;
@@ -3303,13 +3340,16 @@ static int f2fs_read_data_large_folio(struct inode *inode,
struct fsverity_info *vi,
struct readahead_control *rac, struct folio *folio)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct bio *bio = NULL;
sector_t last_block_in_bio = 0;
struct f2fs_map_blocks map = {0, };
- pgoff_t index, offset, next_pgofs = 0;
- unsigned max_nr_pages = rac ? readahead_count(rac) :
- folio_nr_pages(folio);
- unsigned nrpages, len_blks;
+ pgoff_t index, offset, next_blkofs = 0;
+ unsigned int max_nr_blocks = rac ?
+ (readahead_count(rac) << F2FS_LOG_BLKS_PER_PAGE(sbi)) :
+ f2fs_blocks_per_folio(sbi, folio);
+ unsigned int nr_blocks, len_blks;
+ sector_t last_block_in_file;
struct f2fs_folio_state *ffs;
int ret = 0;
bool folio_in_bio = false;
@@ -3322,35 +3362,35 @@ static int f2fs_read_data_large_folio(struct inode *inode,
if (!folio)
goto out;
- f2fs_update_read_folio_count(F2FS_I_SB(inode), folio);
+ f2fs_update_read_folio_count(sbi, folio);
folio_in_bio = false;
- ffs = NULL;
- if (folio_test_large(folio)) {
- /*
- * Allocate the folio state before doing anything with the
- * folio, so that the block mapping helpers below can rely on
- * it.
- */
- ffs = f2fs_ffs_find_or_alloc(folio);
- }
- index = folio->index;
+ index = f2fs_folio_lblk(sbi, folio);
offset = 0;
- nrpages = folio_nr_pages(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ last_block_in_file = F2FS_BLK_ALIGN(sbi, f2fs_readpage_limit(inode));
- for (; nrpages;
- nrpages -= len_blks, max_nr_pages -= len_blks,
+ for (; nr_blocks;
+ nr_blocks -= len_blks, max_nr_blocks -= len_blks,
index += len_blks, offset += len_blks) {
sector_t block_nr;
+ bool whole_folio_in_bio;
unsigned int i;
+ size_t page_offset;
len_blks = 1;
f2fs_skip_fully_uptodate_front(folio, &index, &offset,
- &nrpages, &max_nr_pages);
- if (!nrpages)
+ &nr_blocks, &max_nr_blocks);
+ if (!nr_blocks)
break;
+ page_offset = F2FS_BLK_TO_BYTES(sbi, offset);
+
+ /* The blocks past EOF are zeroed below without mapping. */
+ if (index >= last_block_in_file)
+ goto got_it;
+
/*
* Map blocks using the previous result first.
*/
@@ -3358,7 +3398,7 @@ static int f2fs_read_data_large_folio(struct inode *inode,
if (index > map.m_lblk &&
index < (map.m_lblk + map.m_len))
goto got_it;
- } else if (index < next_pgofs) {
+ } else if (index < next_blkofs) {
/* hole case */
goto got_it;
}
@@ -3368,26 +3408,30 @@ static int f2fs_read_data_large_folio(struct inode *inode,
* done with this page.
*/
memset(&map, 0, sizeof(map));
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = NO_CHECK_TYPE;
map.m_lblk = index;
- map.m_len = max_nr_pages;
+ map.m_len = max_nr_blocks;
ret = f2fs_map_blocks(inode, &map, F2FS_GET_BLOCK_DEFAULT);
if (ret)
goto err_out;
got_it:
- if ((map.m_flags & F2FS_MAP_MAPPED)) {
+ /* Zero the blocks past EOF, as f2fs_read_single_page() does. */
+ if ((map.m_flags & F2FS_MAP_MAPPED) &&
+ index < last_block_in_file) {
block_nr = map.m_pblk + index - map.m_lblk;
- len_blks = min_t(unsigned int, nrpages, max_nr_pages);
+ len_blks = min_t(unsigned int, nr_blocks, max_nr_blocks);
len_blks = min_t(unsigned int, len_blks,
(unsigned int)(map.m_lblk + map.m_len - index));
+ len_blks = min_t(unsigned int, len_blks,
+ last_block_in_file - index);
f2fs_truncate_read_extent(inode, folio, offset,
block_nr, &len_blks);
for (i = 0; i < len_blks; i++) {
- if (!f2fs_is_valid_blkaddr(F2FS_I_SB(inode),
+ if (!f2fs_is_valid_blkaddr(sbi,
block_nr + i,
DATA_GENERIC_ENHANCE_READ)) {
ret = -EFSCORRUPTED;
@@ -3395,18 +3439,21 @@ static int f2fs_read_data_large_folio(struct inode *inode,
}
}
+ whole_folio_in_bio = !f2fs_folio_has_ffs(folio) &&
+ offset == 0 &&
+ len_blks == f2fs_blocks_per_folio(sbi, folio);
} else {
- size_t page_offset = offset << PAGE_SHIFT;
-
- folio_zero_range(folio, page_offset, PAGE_SIZE);
- if (vi && !fsverity_verify_blocks(vi, folio, PAGE_SIZE, page_offset)) {
+ folio_zero_range(folio, page_offset, F2FS_BLKSIZE(sbi));
+ if (vi && !fsverity_verify_blocks(vi, folio,
+ F2FS_BLKSIZE(sbi), page_offset)) {
ret = -EIO;
goto err_out;
}
- if (folio_test_large(folio)) {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irq(&ffs->state_lock);
__ffs_mark_subrange_uptodate(folio, ffs,
- page_offset, PAGE_SIZE);
+ page_offset, F2FS_BLKSIZE(sbi));
spin_unlock_irq(&ffs->state_lock);
}
continue;
@@ -3415,7 +3462,10 @@ static int f2fs_read_data_large_folio(struct inode *inode,
/* We must increment read_blocks_pending before possible BIOs submitting
* to prevent from premature folio_end_read() call on folio
*/
- if (folio_test_large(folio)) {
+ ffs = NULL;
+ if (f2fs_is_multiblock_folio(sbi, folio) && !whole_folio_in_bio) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
+
/* set the bitmap to wait */
f2fs_update_read_folio_pending(folio, len_blks);
}
@@ -3424,16 +3474,16 @@ static int f2fs_read_data_large_folio(struct inode *inode,
* This page will go to BIO. Do we need to send this
* BIO off first?
*/
- if (bio && (!page_is_mergeable(F2FS_I_SB(inode), bio,
+ if (bio && (!page_is_mergeable(sbi, bio,
last_block_in_bio, block_nr) ||
!f2fs_crypt_mergeable_bio(bio, inode, index, NULL))) {
submit_and_realloc:
- f2fs_submit_read_bio(F2FS_I_SB(inode), bio, DATA);
+ f2fs_submit_read_bio(sbi, bio, DATA);
bio = NULL;
}
if (bio == NULL)
bio = f2fs_grab_read_bio(inode, vi,
- block_nr, max_nr_pages,
+ block_nr, max_nr_blocks,
f2fs_ra_op_flags(rac),
index, false);
@@ -3444,33 +3494,33 @@ static int f2fs_read_data_large_folio(struct inode *inode,
for (i = 0; i < len_blks; i++)
f2fs_wait_on_block_writeback(inode, block_nr + i);
- if (!bio_add_folio(bio, folio,
- len_blks * F2FS_BLKSIZE(F2FS_I_SB(inode)),
- offset << PAGE_SHIFT))
+ if (!bio_add_folio(bio, folio, F2FS_BLK_TO_BYTES(sbi, len_blks),
+ page_offset))
goto submit_and_realloc;
folio_in_bio = true;
- for (i = 0; i < len_blks; i++)
- inc_cache_count(F2FS_I_SB(inode), F2FS_RD_DATA);
- f2fs_update_iostat(F2FS_I_SB(inode), NULL, FS_DATA_READ_IO,
- len_blks * F2FS_BLKSIZE(F2FS_I_SB(inode)));
+ add_cache_counts(sbi, F2FS_RD_DATA, len_blks);
+ f2fs_update_iostat(sbi, NULL, FS_DATA_READ_IO,
+ F2FS_BLK_TO_BYTES(sbi, len_blks));
last_block_in_bio = block_nr + len_blks - 1;
}
trace_f2fs_read_folio(folio, DATA);
-err_out:
- if (!folio_in_bio)
- folio_end_read(folio, !ret);
- if (ret)
- goto out;
+ if (!folio_in_bio) {
+ if (!folio_test_uptodate(folio))
+ folio_mark_uptodate(folio);
+ folio_unlock(folio);
+ }
if (rac) {
folio = readahead_folio(rac);
goto next_folio;
}
out:
- f2fs_submit_read_bio(F2FS_I_SB(inode), bio, DATA);
- if (ret && folio_in_bio) {
- /* Wait bios and clear uptodate. */
- folio_lock(folio);
+ f2fs_submit_read_bio(sbi, bio, DATA);
+ return ret;
+err_out:
+ f2fs_submit_read_bio(sbi, bio, DATA);
+ /* The read completion unlocks a folio in the bio, not uptodate. */
+ if (!folio_in_bio) {
folio_clear_uptodate(folio);
folio_unlock(folio);
}
@@ -3501,13 +3551,12 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
pgoff_t nc_cluster_idx = NULL_CLUSTER;
pgoff_t index;
#endif
- pgoff_t next_pgofs = 0;
+ pgoff_t next_blkofs = 0;
unsigned nr_pages = rac ? readahead_count(rac) : 1;
- struct address_space *mapping = rac ? rac->mapping : folio->mapping;
unsigned max_nr_pages = nr_pages;
int ret = 0;
- if (mapping_large_folio_support(mapping))
+ if (f2fs_has_multiblock_folios(inode))
return f2fs_read_data_large_folio(inode, vi, rac, folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
@@ -3522,7 +3571,7 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
map.m_lblk = 0;
map.m_len = 0;
map.m_flags = 0;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -3895,7 +3944,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
/*
* Do not clear gcing flag on failure here: the caller will redirty
* the folio and keep the flag so that it can be guaranteed by
- * checkpoint, or drop the flag by itself, e.g. move_data_page().
+ * checkpoint, or drop the flag by itself, e.g. move_data_cache_block().
*/
if (!submitted && pending_added)
f2fs_ffs_put_write_pending(folio, 1);
@@ -5276,7 +5325,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
DATA_GENERIC_ENHANCE_READ))
return -EFSCORRUPTED;
- err = f2fs_submit_page_read_sync(inode, folio, index, blkaddr);
+ err = f2fs_submit_block_read_sync(inode, folio, index, blkaddr);
if (err)
return err;
}
@@ -5327,8 +5376,7 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
DATA_GENERIC_ENHANCE_READ))
return -EFSCORRUPTED;
- err = f2fs_submit_page_read_sync(inode, folio, index,
- blkaddr);
+ err = f2fs_submit_block_read_sync(inode, folio, index, blkaddr);
if (err)
return err;
balance:
@@ -5485,9 +5533,9 @@ static int f2fs_write_begin(const struct kiocb *iocb,
* folio's owner; otherwise an unencrypted @inode whose COW inode
* is encrypted hits a NULL ->i_crypt_info on decryption.
*/
- f2fs_submit_page_read(inode,
- NULL, /* can't write to fsverity files */
- folio, index, blkaddr, 0, true);
+ f2fs_submit_block_read(inode,
+ NULL, /* can't write to fsverity files */
+ folio, index, blkaddr, 0, true);
folio_lock(folio);
if (unlikely(folio->mapping != mapping)) {
@@ -5636,6 +5684,16 @@ bool f2fs_release_folio(struct folio *folio, gfp_t wait)
if (folio_test_dirty(folio))
return false;
+ /*
+ * The folio state holds the uptodate state of the blocks of a folio
+ * that is not fully uptodate. Keep it while the folio has users other
+ * than the caller, e.g. a dentry lookup that parses a block without
+ * the folio lock, so that the block is not read again under them.
+ */
+ if (f2fs_folio_has_ffs(folio) && !folio_test_uptodate(folio) &&
+ folio_ref_count(folio) > folio_expected_ref_count(folio) + 1)
+ return false;
+
ffs_detach_free(folio);
return true;
}
@@ -5728,7 +5786,7 @@ static sector_t f2fs_bmap(struct address_space *mapping, sector_t block)
memset(&map, 0, sizeof(map));
map.m_lblk = block;
map.m_len = 1;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_seg_type = NO_CHECK_TYPE;
if (!f2fs_map_blocks(inode, &map, F2FS_GET_BLOCK_BMAP))
@@ -5847,7 +5905,7 @@ static int check_swap_activate(struct swap_info_struct *sis,
memset(&map, 0, sizeof(map));
map.m_lblk = cur_lblock;
map.m_len = last_lblock - cur_lblock;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -5987,6 +6045,43 @@ static void f2fs_swap_deactivate(struct file *file)
}
#endif
+static bool f2fs_is_partially_uptodate(struct folio *folio, size_t from,
+ size_t count)
+{
+ struct inode *inode = folio->mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct f2fs_folio_state *ffs;
+ unsigned int first, last, i;
+
+ if (folio_test_uptodate(folio))
+ return true;
+
+ if (f2fs_has_inline_data(inode) || !f2fs_is_multiblock_folio(sbi, folio))
+ return false;
+
+ if (!f2fs_folio_has_ffs(folio))
+ return false;
+
+ if (from >= folio_size(folio))
+ return false;
+
+ count = min_t(size_t, folio_size(folio) - from, count);
+ if (!count)
+ return false;
+
+ first = F2FS_BYTES_TO_BLK(sbi, from);
+ last = F2FS_BYTES_TO_BLK(sbi, from + count - 1);
+
+ ffs = folio->private;
+
+ for (i = first; i <= last; i++) {
+ if (!test_bit(i, ffs->state))
+ return false;
+ }
+
+ return true;
+}
+
const struct address_space_operations f2fs_dblock_aops = {
.read_folio = f2fs_read_data_folio,
.readahead = f2fs_readahead,
@@ -5997,6 +6092,7 @@ const struct address_space_operations f2fs_dblock_aops = {
.migrate_folio = filemap_migrate_folio,
.invalidate_folio = f2fs_invalidate_folio,
.release_folio = f2fs_release_folio,
+ .is_partially_uptodate = f2fs_is_partially_uptodate,
.bmap = f2fs_bmap,
.swap_activate = f2fs_swap_activate,
.swap_deactivate = f2fs_swap_deactivate,
@@ -6087,13 +6183,13 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_map_blocks map = { NULL, };
- pgoff_t next_pgofs = 0;
+ pgoff_t next_blkofs = 0;
int err;
map.m_lblk = F2FS_BYTES_TO_BLK(sbi, offset);
map.m_len = F2FS_BYTES_TO_BLK(sbi, offset + length - 1) -
map.m_lblk + 1;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = f2fs_rw_hint_to_seg_type(F2FS_I_SB(inode),
inode->i_write_hint);
if (flags & IOMAP_WRITE && iomap->private) {
@@ -6146,7 +6242,7 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
return -ENOTBLK;
if (map.m_pblk == NULL_ADDR) {
- iomap->length = F2FS_BLK_TO_BYTES(sbi, next_pgofs) -
+ iomap->length = F2FS_BLK_TO_BYTES(sbi, next_blkofs) -
iomap->offset;
iomap->type = IOMAP_HOLE;
} else if (map.m_pblk == NEW_ADDR) {
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 9cf9583a4428..f7234283d33d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -836,7 +836,7 @@ struct f2fs_map_blocks {
unsigned int m_len;
unsigned int m_flags;
unsigned long m_last_pblk; /* last allocated block, only used for DIO in LFS mode */
- pgoff_t *m_next_pgofs; /* point next possible non-hole pgofs */
+ pgoff_t *m_next_blkofs; /* point next possible non-hole blkofs */
pgoff_t *m_next_extent; /* point to next possible extent */
int m_seg_type;
bool m_may_create; /* indicate it is from write path */
@@ -2349,6 +2349,12 @@ static inline bool f2fs_is_multiblock_folio(struct f2fs_sb_info *sbi,
return folio_size(folio) > F2FS_BLKSIZE(sbi);
}
+static inline bool f2fs_has_multiblock_folios(struct inode *inode)
+{
+ return mapping_large_folio_support(inode->i_mapping) ||
+ f2fs_has_subpage_blocks(F2FS_I_SB(inode));
+}
+
static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
const struct folio *folio)
{
@@ -4240,7 +4246,7 @@ bool f2fs_is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino);
int f2fs_get_node_info(struct f2fs_sb_info *sbi, nid_t nid,
struct node_info *ni, bool checkpoint_context);
-pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs);
+pgoff_t f2fs_get_next_block_offset(struct dnode_of_data *dn, pgoff_t blkofs);
int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode);
int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from);
int f2fs_truncate_xattr_node(struct inode *inode);
@@ -4503,11 +4509,11 @@ int f2fs_reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count);
int f2fs_reserve_new_block(struct dnode_of_data *dn);
int f2fs_get_block_locked(struct dnode_of_data *dn, pgoff_t index);
int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index);
-struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
- blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs);
-struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
- pgoff_t *next_pgofs);
-struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
+struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t bidx,
+ blk_opf_t op_flags, bool for_write, pgoff_t *next_blkofs);
+struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t bidx,
+ pgoff_t *next_blkofs);
+struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t bidx,
bool for_write);
struct folio *f2fs_get_new_data_folio(struct inode *inode,
struct f2fs_cached_block *ientry, pgoff_t index, bool new_i_size);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index e0d246397ad0..b5fa4df9f2e3 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -571,7 +571,7 @@ static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
} else if (err == -ENOENT) {
/* direct node does not exists */
if (whence == SEEK_DATA) {
- pgofs = f2fs_get_next_page_offset(&dn, pgofs);
+ pgofs = f2fs_get_next_block_offset(&dn, pgofs);
continue;
} else {
goto found;
@@ -1356,7 +1356,7 @@ int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
err = f2fs_get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
if (err) {
if (err == -ENOENT) {
- pg_start = f2fs_get_next_page_offset(&dn,
+ pg_start = f2fs_get_next_block_offset(&dn,
pg_start);
continue;
}
@@ -2026,7 +2026,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
loff_t len, int mode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- struct f2fs_map_blocks map = { .m_next_pgofs = NULL,
+ struct f2fs_map_blocks map = { .m_next_blkofs = NULL,
.m_next_extent = NULL, .m_seg_type = NO_CHECK_TYPE,
.m_may_create = true };
struct f2fs_gc_control gc_control = { .victim_segno = NULL_SEGNO,
@@ -3095,7 +3095,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
.m_seg_type = NO_CHECK_TYPE,
.m_may_create = false };
struct extent_info ei = {};
- pgoff_t pg_start, pg_end, next_pgofs;
+ pgoff_t pg_start, pg_end, next_blkofs;
unsigned int total = 0, sec_num;
block_t blk_end = 0;
bool fragmented = false;
@@ -3139,7 +3139,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
}
map.m_lblk = pg_start;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
/*
* lookup mapping info in dnode page cache, skip defragmenting if all
@@ -3153,7 +3153,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
goto out;
if (!(map.m_flags & F2FS_MAP_FLAGS)) {
- map.m_lblk = next_pgofs;
+ map.m_lblk = next_blkofs;
continue;
}
@@ -3200,7 +3200,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
goto clear_out;
if (!(map.m_flags & F2FS_MAP_FLAGS)) {
- map.m_lblk = next_pgofs;
+ map.m_lblk = next_blkofs;
goto check;
}
@@ -4092,7 +4092,7 @@ int f2fs_precache_extents(struct inode *inode)
map.m_lblk = 0;
map.m_pblk = 0;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = &m_next_extent;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -4385,7 +4385,7 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
if (ret) {
f2fs_unlock_op(sbi, &lc);
if (ret == -ENOENT) {
- page_idx = f2fs_get_next_page_offset(&dn,
+ page_idx = f2fs_get_next_block_offset(&dn,
page_idx);
ret = 0;
continue;
@@ -4567,7 +4567,7 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
if (ret) {
f2fs_unlock_op(sbi, &lc);
if (ret == -ENOENT) {
- page_idx = f2fs_get_next_page_offset(&dn,
+ page_idx = f2fs_get_next_block_offset(&dn,
page_idx);
ret = 0;
continue;
@@ -4735,7 +4735,7 @@ static int f2fs_sec_trim_file(struct file *filp, unsigned long arg)
ret = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
if (ret) {
if (ret == -ENOENT) {
- index = f2fs_get_next_page_offset(&dn, index);
+ index = f2fs_get_next_block_offset(&dn, index);
continue;
}
goto out;
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 337ecf4df945..274c86511cde 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1246,7 +1246,9 @@ static int ra_data_block(struct inode *inode, pgoff_t index)
}
f2fs_up_read(&F2FS_I(inode)->i_sem);
- folio = f2fs_grab_cache_folio(mapping, index, true);
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, index);
+
+ folio = f2fs_grab_cache_folio(mapping, f_index, true);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
goto out_iput;
@@ -1323,6 +1325,35 @@ static int ra_data_block(struct inode *inode, pgoff_t index)
return err;
}
+/*
+ * Prefetch the whole folio holding @bidx with one read, so that contiguous
+ * blocks in the folio share a bio. Skip the folio if it is locked, e.g. by
+ * the read issued for a previous block in the folio.
+ */
+static void ra_data_folio(struct inode *inode, pgoff_t bidx)
+{
+ struct address_space *mapping = inode->i_mapping;
+ struct folio *folio;
+
+ folio = __filemap_get_folio(mapping,
+ f2fs_lblk_to_folio_index(F2FS_I_SB(inode), bidx),
+ FGP_LOCK | FGP_CREAT | FGP_NOWAIT,
+ mapping_gfp_mask(mapping));
+ if (IS_ERR(folio))
+ return;
+
+ if (folio_test_uptodate(folio)) {
+ folio_unlock(folio);
+ } else {
+ /* GC locks are held, so do not recurse into fs reclaim. */
+ unsigned int nofs_flags = memalloc_nofs_save();
+
+ mapping->a_ops->read_folio(NULL, folio);
+ memalloc_nofs_restore(nofs_flags);
+ }
+ folio_put(folio);
+}
+
/*
* Move data block via meta cache while keeping locked data page.
* This can be used to move blocks, aka LBAs, directly on disk.
@@ -1367,8 +1398,10 @@ static int move_data_block(struct inode *inode, block_t bidx,
}
f2fs_up_read(&F2FS_I(inode)->i_sem);
+ pgoff_t f_index = f2fs_lblk_to_folio_index(fio.sbi, bidx);
+
/* do not read out */
- folio = f2fs_grab_cache_folio(mapping, bidx, false);
+ folio = f2fs_grab_cache_folio(mapping, f_index, false);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
goto out_iput;
@@ -1389,7 +1422,9 @@ static int move_data_block(struct inode *inode, block_t bidx,
goto out;
if (unlikely(dn.data_blkaddr == NULL_ADDR)) {
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio,
+ f2fs_folio_byte_offset(fio.sbi, folio, bidx),
+ F2FS_BLKSIZE(fio.sbi));
err = -ENOENT;
goto put_out;
}
@@ -1516,24 +1551,26 @@ static int move_data_block(struct inode *inode, block_t bidx,
return err;
}
-static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
+static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
unsigned int segno, int off)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct folio *folio;
size_t foff = 0;
- bool large = false;
+ bool has_ffs = false;
int err = 0;
folio = f2fs_get_lock_data_folio(inode, bidx, true);
if (IS_ERR(folio))
return PTR_ERR(folio);
- if (f2fs_folio_has_ffs(folio)) {
- large = true;
- foff = offset_in_folio(folio, (loff_t)bidx << PAGE_SHIFT);
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ has_ffs = true;
+ f2fs_ffs_find_or_alloc(folio);
+ foff = f2fs_folio_byte_offset(sbi, folio, bidx);
}
- if (!check_valid_map(F2FS_I_SB(inode), segno, off)) {
+ if (!check_valid_map(sbi, segno, off)) {
err = -ENOENT;
goto out;
}
@@ -1547,13 +1584,13 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
err = -EAGAIN;
goto out;
}
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else {
struct f2fs_io_info fio = {
- .sbi = F2FS_I_SB(inode),
+ .sbi = sbi,
.ino = inode->i_ino,
.type = DATA,
.temp = COLD,
@@ -1561,7 +1598,7 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
.op_flags = REQ_SYNC,
.old_blkaddr = NULL_ADDR,
.folio = folio,
- .folio_offset = bidx - folio->index,
+ .folio_offset = f2fs_folio_blk_offset(sbi, folio, bidx),
.folio_blkcnt = 1,
.encrypted_page = NULL,
.need_lock = LOCK_REQ,
@@ -1572,15 +1609,15 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
retry:
f2fs_folio_wait_writeback(folio, true, true);
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
if (folio_clear_dirty_for_io(folio)) {
- if (!large) {
+ if (!has_ffs) {
inode_dec_dirty_block(inode);
f2fs_remove_dirty_inode(inode);
} else if (f2fs_ffs_clear_subrange_dirty_and_test(folio,
- foff, PAGE_SIZE)) {
+ foff, F2FS_BLKSIZE(sbi))) {
folio_mark_dirty(folio);
}
}
@@ -1598,15 +1635,16 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
* Keep gcing flag for redirtied folio, so that
* it can be guaranteed by checkpoint.
*/
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff,
+ F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else if (!f2fs_ffs_has_dirty_subpage(folio)) {
/* migrate this block in later GC */
folio_clear_f2fs_gcing(folio);
}
- } else if (large && folio_test_dirty(folio)) {
+ } else if (has_ffs && folio_test_dirty(folio)) {
/*
* The folio is left both dirty and under writeback,
* submit the cached bio before unlocking it, otherwise
@@ -1742,6 +1780,13 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
continue;
}
+ if (f2fs_has_subpage_blocks(sbi)) {
+ ra_data_folio(inode, start_bidx);
+ f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
+ add_gc_inode(gc_list, inode);
+ continue;
+ }
+
data_folio = f2fs_get_read_data_folio(inode, start_bidx,
REQ_RAHEAD, true, NULL);
f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
@@ -1785,7 +1830,7 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
err = move_data_block(inode, start_bidx,
gc_type, segno, off);
else
- err = move_data_page(inode, start_bidx, gc_type,
+ err = move_data_cache_block(inode, start_bidx, gc_type,
segno, off);
if (!err && (gc_type == FG_GC ||
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index cb8affb2c1e0..c48b12208e20 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -701,7 +701,7 @@ static void f2fs_ra_node_caches(struct f2fs_cached_block *parent, int start, int
blk_finish_plug(&plug);
}
-pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
+pgoff_t f2fs_get_next_block_offset(struct dnode_of_data *dn, pgoff_t blkofs)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
const long direct_index = ADDRS_PER_INODE(dn->inode);
@@ -714,7 +714,7 @@ pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
pgoff_t base = 0;
if (!dn->max_level)
- return pgofs + 1;
+ return blkofs + 1;
while (max_level-- > cur_level)
skipped_unit *= NIDS_PER_BLOCK(sbi);
@@ -733,7 +733,7 @@ pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
f2fs_bug_on(F2FS_I_SB(dn->inode), 1);
}
- return ((pgofs - base) / skipped_unit + 1) * skipped_unit + base;
+ return ((blkofs - base) / skipped_unit + 1) * skipped_unit + base;
}
/*
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 07/20] f2fs: support subpage blocks in data write path and writeback
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (5 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
` (12 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With blocks smaller than PAGE_SIZE, a folio holds several blocks, so
write_begin, write_end and writeback have to handle individual blocks
instead of the whole folio.
- Read, zero and reserve the blocks touched by a write per block in
write_begin, and mark only the written blocks dirty in write_end.
- Write back only the dirty blocks of a multi-block folio and complete
write I/O per block. In subpage mode every inode is written back by
f2fs_write_cache_folios(), which now handles dentry and quota folios
and checkpoint errors as f2fs_write_single_data_page() does. Both use
the new f2fs_cp_controlled_inode() and f2fs_wb_folio_action() helpers
for these rules.
It also resets the error for each folio, so that the remaining folios
are still written back after an error, as writeback_iter() expects
for integrity writeback, instead of redirtying them.
f2fs_write_folio_dirty_range() cleans up each block where it is
written or skipped, and advances last_disk_size only up to the last
block it submitted. On an error, it clears the dirty state of the
blocks before the failed one, so that only the failed block and the
ones after it are written again.
- Count inode dirty blocks from the dirty bits of the folio state, and
count every block of a folio dirtied without one, e.g. after a read
of the whole folio or through folio_mark_dirty(). Attaching a folio
state to a dirty folio converts its count to the dirty bits, so the
counter stays balanced.
- ->dirty_folio() does not know which blocks were modified, so it marks
the uptodate blocks of the folio within EOF dirty. Otherwise, a folio
dirtied by mm after writeback cleared its dirty bits, e.g. through a
pinned page, is cleaned without being written. The updates made by
f2fs mark only the blocks they modify and dirty the folio with
f2fs_folio_mark_dirty(), which keeps the other blocks clean.
- Compressed and atomic files are not supported yet in subpage mode:
write_begin returns -EOPNOTSUPP for them. Inline data is converted
first.
Some of these changes also apply to large folios when the block size
equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/checkpoint.c | 65 ++---
fs/f2fs/compress.c | 2 +-
fs/f2fs/data.c | 583 ++++++++++++++++++++++++++-----------------
fs/f2fs/dir.c | 4 +-
fs/f2fs/f2fs.h | 3 +
fs/f2fs/file.c | 23 +-
fs/f2fs/gc.c | 20 +-
fs/f2fs/inline.c | 17 +-
fs/f2fs/node.c | 11 +-
9 files changed, 433 insertions(+), 295 deletions(-)
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 1060ee651756..05edd76f5a81 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -1324,29 +1324,6 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
stat_dec_dirty_inode(F2FS_I_SB(inode), type);
}
-/*
- * Account @nr dirty blocks (negative when dirtiness is cleared) for @inode.
- */
-void f2fs_account_dirty_blocks(struct inode *inode, int nr)
-{
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
-
- if (!nr)
- return;
-
- spin_lock(&sbi->inode_lock[type]);
- if (nr > 0) {
- if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH))
- __add_dirty_inode(inode, type);
- inode_add_dirty_blocks(inode, nr);
- } else {
- inode_sub_dirty_blocks(inode, -nr);
- __remove_dirty_inode(inode, type);
- }
- spin_unlock(&sbi->inode_lock[type]);
-}
-
void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio)
{
if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
@@ -1354,16 +1331,13 @@ void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio)
return;
/*
- * Large folios have a subpage dirty bitmap, so defer their dirty accounting
- * to the f2fs_ffs_*subrange_dirty*() helpers. This avoids nesting
- * ffs->state_lock with the inode dirty list lock.
+ * The dirty blocks of a folio with a folio state are accounted when
+ * their dirty bits are set. Without it, every block of the folio is
+ * dirty.
*/
- if (f2fs_folio_has_ffs(folio)) {
- folio_set_f2fs_reference(folio);
- return;
- }
-
- f2fs_account_dirty_blocks(inode, 1);
+ if (!f2fs_folio_has_ffs(folio))
+ f2fs_account_dirty_blocks(inode,
+ f2fs_blocks_per_folio(F2FS_I_SB(inode), folio));
folio_set_f2fs_reference(folio);
}
@@ -1385,6 +1359,33 @@ void f2fs_remove_dirty_inode(struct inode *inode)
spin_unlock(&sbi->inode_lock[type]);
}
+/*
+ * Account @nr dirty blocks (negative when they are cleaned) for @inode.
+ */
+void f2fs_account_dirty_blocks(struct inode *inode, int nr)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
+
+ if (!nr)
+ return;
+
+ if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
+ !S_ISLNK(inode->i_mode))
+ return;
+
+ if (nr < 0) {
+ inode_sub_dirty_blocks(inode, -nr);
+ return;
+ }
+
+ spin_lock(&sbi->inode_lock[type]);
+ if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH))
+ __add_dirty_inode(inode, type);
+ inode_add_dirty_blocks(inode, nr);
+ spin_unlock(&sbi->inode_lock[type]);
+}
+
int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type,
bool from_cp)
{
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index 0b0f31af6626..c63622b6c49a 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -1090,7 +1090,7 @@ static void set_cluster_dirty(struct compress_ctx *cc)
if (!cc->rpages[i])
continue;
folio = page_folio(cc->rpages[i]);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
}
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 971d74d2f010..54b2024ece7b 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -389,7 +389,7 @@ bool f2fs_ffs_has_dirty_subpage(struct folio *folio)
* clear F2FS_FOLIO_PRIVATE_ONGOING_MIGRATION if no dirty subpage is left,
* and end writeback if it was started.
*/
-static void f2fs_ffs_put_write_pending(struct folio *folio, unsigned int nr)
+void f2fs_ffs_put_write_pending(struct folio *folio, unsigned int nr)
{
struct f2fs_folio_state *ffs;
@@ -420,8 +420,7 @@ static void f2fs_write_end_bio(struct bio *bio)
bio_for_each_folio_all(fi, bio) {
struct folio *folio = fi.folio;
- unsigned int nr_pages = fi.length >> PAGE_SHIFT;
- unsigned int npages = nr_pages;
+ unsigned int nr_blocks = F2FS_BYTES_TO_BLK(sbi, fi.length);
enum count_type type;
#ifdef CONFIG_F2FS_FS_COMPRESSION
@@ -441,8 +440,7 @@ static void f2fs_write_end_bio(struct bio *bio)
}
}
- while (nr_pages--)
- dec_cache_count(sbi, type);
+ sub_cache_counts(sbi, type, nr_blocks);
/*
* we should access sbi before folio_end_writeback() to
@@ -452,7 +450,7 @@ static void f2fs_write_end_bio(struct bio *bio)
wq_has_sleeper(&sbi->cp_wait))
wake_up(&sbi->cp_wait);
- f2fs_ffs_put_write_pending(folio, npages);
+ f2fs_ffs_put_write_pending(folio, nr_blocks);
}
bio_put(bio);
@@ -2906,12 +2904,29 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
}
#endif
+/* The blocks of @folio which can be dirty, i.e. the ones within EOF. */
+static unsigned int f2fs_folio_valid_blocks(struct inode *inode,
+ struct folio *folio)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ pgoff_t last_lblk = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
+
+ if (f2fs_verity_in_progress(inode))
+ return nr_blocks;
+ if (first_lblk < last_lblk)
+ return min_t(pgoff_t, nr_blocks, last_lblk - first_lblk);
+ return 0;
+}
+
struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs = folio->private;
unsigned int nr_blocks;
unsigned long private_flags;
+ int dirty_blocks = 0;
if (f2fs_folio_has_ffs(folio))
return ffs;
@@ -2929,29 +2944,29 @@ struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio)
if (folio_test_uptodate(folio))
bitmap_set(ffs->state, 0, nr_blocks);
if (folio_test_dirty(folio)) {
- struct inode *inode = folio->mapping->host;
- pgoff_t last_lblk = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
- pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
- pgoff_t valid_blocks = 0;
-
- if (f2fs_verity_in_progress(inode))
- valid_blocks = nr_blocks;
- else if (first_lblk < last_lblk)
- valid_blocks = min_t(pgoff_t, nr_blocks,
- last_lblk - first_lblk);
+ unsigned int valid_blocks = f2fs_folio_valid_blocks(
+ folio->mapping->host, folio);
if (valid_blocks)
bitmap_set(ffs->state, nr_blocks, valid_blocks);
+ /*
+ * A folio dirtied without a folio state was accounted as a
+ * whole. Its dirty bits are accounted from now on.
+ */
+ dirty_blocks = (int)valid_blocks - (int)nr_blocks;
}
folio_detach_private(folio);
folio_attach_private(folio, ffs);
+ if (dirty_blocks)
+ f2fs_account_dirty_blocks(folio->mapping->host, dirty_blocks);
return ffs;
}
static void ffs_detach_free(struct folio *folio)
{
struct f2fs_folio_state *ffs;
+ unsigned int nr_blocks;
if (!f2fs_folio_has_ffs(folio)) {
folio_detach_private(folio);
@@ -2964,6 +2979,10 @@ static void ffs_detach_free(struct folio *folio)
WARN_ON_ONCE(ffs->read_blocks_pending != 0);
WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending));
+ /* The dirty blocks must have been cleaned and unaccounted. */
+ nr_blocks = f2fs_blocks_per_folio(F2FS_F_SB(folio), folio);
+ WARN_ON_ONCE(find_next_bit(ffs->state, 2 * nr_blocks, nr_blocks) <
+ 2 * nr_blocks);
ffs->magic = 0;
kfree(ffs);
}
@@ -3031,7 +3050,7 @@ static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- if (f2fs_folio_has_ffs(folio)) {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
f2fs_ffs_clear_subrange_uptodate(folio,
F2FS_BLK_TO_BYTES(sbi, i), F2FS_BLKSIZE(sbi));
} else {
@@ -3059,21 +3078,6 @@ bool f2fs_ffs_test_blk_dirty(const struct folio *folio, pgoff_t index)
return test_bit(nr_blocks + offset, ffs->state);
}
-static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
- struct f2fs_folio_state *ffs, size_t offset, size_t len)
-{
- struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- unsigned int start, end, before;
-
- start = F2FS_BYTES_TO_BLK(sbi, offset);
- end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
-
- before = bitmap_weight(ffs->state, 2 * nr_blocks);
- bitmap_set(ffs->state, nr_blocks + start, end - start);
- return bitmap_weight(ffs->state, 2 * nr_blocks) - before;
-}
-
/*
* Mark the given range dirty for writeback.
*
@@ -3086,7 +3090,8 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
- unsigned int set;
+ unsigned int nr_blocks;
+ unsigned int start, end, before, set;
unsigned long flags;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
@@ -3095,11 +3100,17 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
return;
ffs = f2fs_ffs_find_or_alloc(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
spin_lock_irqsave(&ffs->state_lock, flags);
- set = __ffs_mark_subrange_dirty(folio, ffs, offset, len);
+ before = bitmap_weight(ffs->state, 2 * nr_blocks);
+ bitmap_set(ffs->state, start + nr_blocks, end - start);
+ set = bitmap_weight(ffs->state, 2 * nr_blocks) - before;
spin_unlock_irqrestore(&ffs->state_lock, flags);
+ /* Outside ffs->state_lock, which the I/O completion takes. */
if (set)
f2fs_account_dirty_blocks(folio->mapping->host, set);
}
@@ -3161,8 +3172,7 @@ static bool __ffs_clear_subrange_dirty(struct folio *folio,
* @len: byte length of the subrange
*
* Clear the dirty bits of the 4K subpages covered by [offset, offset + len),
- * account the cleared subpages against the inode dirty counters, and return
- * whether the folio still has any dirty subpage left.
+ * and return whether the folio still has any dirty subpage left.
*/
bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
size_t len)
@@ -3190,11 +3200,28 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
spin_unlock_irqrestore(&ffs->state_lock, flags);
if (cleared)
- f2fs_account_dirty_blocks(folio->mapping->host, -cleared);
+ f2fs_account_dirty_blocks(folio->mapping->host, -(int)cleared);
return dirty;
}
+/*
+ * All blocks of @folio are clean now, e.g. written back or truncated. Clear
+ * its dirty bits, or without a folio state, unaccount all of its blocks if
+ * it was @dirty.
+ */
+void f2fs_folio_clear_dirty_blocks(struct folio *folio, bool dirty)
+{
+ struct inode *inode = folio->mapping->host;
+
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ folio_size(folio));
+ else if (dirty)
+ f2fs_account_dirty_blocks(inode,
+ -(int)f2fs_blocks_per_folio(F2FS_I_SB(inode), folio));
+}
+
bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
unsigned int to, unsigned int *start,
unsigned int *count)
@@ -3954,7 +3981,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type,
- u64 *pos, u64 len)
+ u64 start, u64 end)
{
struct inode *inode = folio->mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -3962,19 +3989,19 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
folio_test_f2fs_atomic(folio);
struct inode *dn_inode = atomic_commit ?
F2FS_I(inode)->cow_inode : inode;
- struct f2fs_folio_state *ffs = NULL;
- u64 folio_start = folio_pos(folio);
- pgoff_t start_idx = (*pos - folio_start) >> PAGE_SHIFT;
- pgoff_t end_idx = (*pos - folio_start + len - 1) >> PAGE_SHIFT;
+ u64 pos = folio_pos(folio);
+ pgoff_t start_idx = F2FS_BYTES_TO_BLK(sbi, start - pos);
+ pgoff_t end_idx = F2FS_BYTES_TO_BLK(sbi, end - 1 - pos);
+ pgoff_t last_idx = 0;
+ pgoff_t i;
+ int local_submitted = 0;
int err = 0;
- for (pgoff_t i = start_idx; i <= end_idx && !err; i++) {
+ for (i = start_idx; i <= end_idx; i++) {
struct dnode_of_data dn;
struct node_info ni;
- pgoff_t data_idx = folio->index + i;
+ pgoff_t data_idx = f2fs_folio_lblk(sbi, folio) + i;
bool ipu_force = false;
- bool dn_held = false;
- bool submitted_blk = false;
struct f2fs_io_info fio = {
.sbi = sbi,
.ino = inode->i_ino,
@@ -3994,7 +4021,12 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
};
if (f2fs_folio_has_ffs(folio)) {
- ffs = (struct f2fs_folio_state *)folio->private;
+ struct f2fs_folio_state *ffs =
+ (struct f2fs_folio_state *)folio->private;
+
+ /* Write only dirty blocks */
+ if (!f2fs_ffs_test_blk_dirty(folio, data_idx))
+ continue;
atomic_inc(&ffs->write_blocks_pending);
}
@@ -4006,7 +4038,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
if (!f2fs_is_valid_blkaddr(sbi, fio.old_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
- goto block_done;
+ goto skip_block;
}
ipu_force = true;
goto got_it;
@@ -4026,24 +4058,22 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
LOOKUP_NODE);
}
if (err && err != -ENOENT)
- goto block_done;
- dn_held = true;
+ goto skip_block;
- fio.old_blkaddr = dn.data_blkaddr;
-
- /* This page is already truncated */
- if (err == -ENOENT || fio.old_blkaddr == NULL_ADDR) {
- f2fs_clear_truncated_page_state(folio, i);
+ /* This block is already truncated */
+ if (err || dn.data_blkaddr == NULL_ADDR) {
err = 0;
- goto block_done;
+ f2fs_clear_truncated_page_state(folio, i);
+ goto skip_block;
}
+ fio.old_blkaddr = dn.data_blkaddr;
got_it:
if (__is_valid_data_blkaddr(fio.old_blkaddr) &&
!f2fs_is_valid_blkaddr(sbi, fio.old_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
- goto block_done;
+ goto skip_block;
}
if (fio.meta_gc)
@@ -4053,61 +4083,109 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
(__is_valid_data_blkaddr(fio.old_blkaddr) &&
need_inplace_update(&fio))) {
f2fs_put_dnode(&dn);
- dn_held = false;
err = f2fs_inplace_write_data(&fio);
- trace_f2fs_do_write_data_page(folio, IPU);
if (err)
- goto block_done;
+ goto skip_block_no_dnode;
- if (submitted)
- (*submitted)++;
- submitted_blk = true;
+ local_submitted++;
+ last_idx = data_idx;
set_inode_flag(inode, FI_UPDATE_WRITE);
- goto block_done;
+ trace_f2fs_do_write_data_page(folio, IPU);
+ continue;
}
err = f2fs_get_node_info(sbi, dn.nid, &ni, false);
if (err)
- goto block_done;
+ goto skip_block;
fio.version = ni.version;
f2fs_outplace_write_data(&dn, &fio);
- /*
- * The outplace write path failed to submit this block.
- */
+ f2fs_put_dnode(&dn);
+ /* The outplace write path failed to submit this block. */
if (!fio.submitted) {
err = -EIO;
- goto block_done;
+ goto skip_block_no_dnode;
}
- if (submitted)
- (*submitted)++;
- submitted_blk = true;
+ local_submitted++;
+ last_idx = data_idx;
set_inode_flag(inode, FI_APPEND_WRITE);
trace_f2fs_do_write_data_page(folio, OPU);
-block_done:
- if (dn_held)
- f2fs_put_dnode(&dn);
+ continue;
+
+skip_block:
+ f2fs_put_dnode(&dn);
+skip_block_no_dnode:
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_put_write_pending(folio, 1);
+ if (err)
+ break;
+ }
+
+ if (err) {
/*
- * If this block was not submitted, roll back its
- * write pages pending.
+ * The blocks before the failed one were written or skipped, so
+ * keep only the failed block and the ones after it dirty.
*/
- if (!submitted_blk && ffs)
- f2fs_ffs_put_write_pending(folio, 1);
- if (!err) {
- *pos = ((loff_t)data_idx + 1) << PAGE_SHIFT;
- spin_lock(&F2FS_I(inode)->i_size_lock);
- if (F2FS_I(inode)->last_disk_size < *pos)
- F2FS_I(inode)->last_disk_size = *pos;
- spin_unlock(&F2FS_I(inode)->i_size_lock);
- } else {
- file_set_keep_isize(inode);
+ if (i > start_idx) {
+ size_t len = F2FS_BLK_TO_BYTES(sbi, i - start_idx);
+
+ f2fs_ffs_clear_subrange_dirty_and_test(folio,
+ start - pos, len);
}
+ file_set_keep_isize(inode);
+ } else if (local_submitted) {
+ loff_t psize = F2FS_BLK_TO_BYTES(sbi, last_idx + 1);
+
+ if (psize > i_size_read(inode))
+ psize = i_size_read(inode);
+
+ spin_lock(&F2FS_I(inode)->i_size_lock);
+ if (F2FS_I(inode)->last_disk_size < psize)
+ F2FS_I(inode)->last_disk_size = psize;
+ spin_unlock(&F2FS_I(inode)->i_size_lock);
}
+ if (submitted)
+ *submitted = local_submitted;
return err;
}
+/* Dentry and quota blocks are controlled by checkpoint. */
+static bool f2fs_cp_controlled_inode(struct inode *inode)
+{
+ return S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode);
+}
+
+enum f2fs_wb_action {
+ F2FS_WB_WRITE, /* write back the folio */
+ F2FS_WB_REDIRTY, /* keep the folio dirty */
+ F2FS_WB_DROP, /* drop the dirty data */
+};
+
+/* Decide what to do with a dirty data folio before writing it back. */
+static enum f2fs_wb_action f2fs_wb_folio_action(struct f2fs_sb_info *sbi,
+ struct inode *inode)
+{
+ /* we should bypass data pages to proceed the kworker jobs */
+ if (unlikely(f2fs_cp_error(sbi))) {
+ mapping_set_error(inode->i_mapping, -EIO);
+ /*
+ * Keep dirty dentry folios for the latest directory structure,
+ * and keep data folios in remount-ro mode.
+ */
+ if ((S_ISDIR(inode->i_mode) &&
+ !is_sbi_flag_set(sbi, SBI_IS_CLOSE)) ||
+ F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
+ return F2FS_WB_REDIRTY;
+ return F2FS_WB_DROP;
+ }
+
+ if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
+ return F2FS_WB_REDIRTY;
+ return F2FS_WB_WRITE;
+}
+
int f2fs_write_single_data_page(struct folio *folio, int *submitted,
struct bio **bio,
sector_t *last_block,
@@ -4148,26 +4226,15 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
trace_f2fs_writepage(folio, DATA);
- /* we should bypass data pages to proceed the kworker jobs */
- if (unlikely(f2fs_cp_error(sbi))) {
- mapping_set_error(folio->mapping, -EIO);
- /*
- * don't drop any dirty dentry pages for keeping lastest
- * directory structure.
- */
- if (S_ISDIR(inode->i_mode) &&
- !is_sbi_flag_set(sbi, SBI_IS_CLOSE))
- goto redirty_out;
-
- /* keep data pages in remount-ro mode */
- if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
- goto redirty_out;
+ switch (f2fs_wb_folio_action(sbi, inode)) {
+ case F2FS_WB_REDIRTY:
+ goto redirty_out;
+ case F2FS_WB_DROP:
goto out;
+ case F2FS_WB_WRITE:
+ break;
}
- if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
- goto redirty_out;
-
if (folio->index < end_index ||
f2fs_verity_in_progress(inode) ||
compr_blocks)
@@ -4183,8 +4250,7 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
folio_zero_segment(folio, offset, folio_size(folio));
write:
- /* Dentry/quota blocks are controlled by checkpoint */
- if (S_ISDIR(inode->i_mode) || quota_inode) {
+ if (f2fs_cp_controlled_inode(inode)) {
struct f2fs_lock_context lc;
/*
@@ -4235,14 +4301,14 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
goto redirty_out;
out:
- inode_dec_dirty_block(inode);
+ f2fs_folio_clear_dirty_blocks(folio, true);
if (err) {
folio_clear_uptodate(folio);
folio_clear_f2fs_gcing(folio);
}
folio_unlock(folio);
- if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) &&
- !F2FS_I(inode)->wb_task && allow_balance)
+ if (!f2fs_cp_controlled_inode(inode) &&
+ !F2FS_I(inode)->wb_task && allow_balance)
f2fs_balance_fs(sbi, need_balance_fs);
if (unlikely(f2fs_cp_error(sbi))) {
@@ -4571,6 +4637,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
int submitted = 0;
int nwritten = 0;
bool op_locked = false;
+ bool node_locked = false;
bool retry = false;
if (get_dirty_blocks(inode) <= SM_I(sbi)->min_hot_blocks)
@@ -4582,20 +4649,15 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
struct f2fs_folio_state *ffs = NULL;
u64 isize;
size_t poff;
- size_t clear_len;
pgoff_t end_index;
bool verity_in_progress;
int folio_submitted = 0;
bool bias_added = false;
submitted = 0;
+ err = 0;
retry = false;
- if (err) {
- folio_redirty_for_writepage(wbc, folio);
- goto retry_out;
- }
-
if (atomic_read(&sbi->wb_sync_req[DATA]) &&
wbc->sync_mode == WB_SYNC_NONE) {
folio_redirty_for_writepage(wbc, folio);
@@ -4608,11 +4670,22 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
goto retry_out;
}
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ WARN_ON_ONCE(1);
+ mapping_set_error(mapping, -EOPNOTSUPP);
+ folio_clear_dirty_for_io(folio);
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ err = -EOPNOTSUPP;
+ goto retry_out;
+ }
+
/*
* Writing back one large folio performs multiple block allocations, so
* best-effort balance the filesystem before writing it back.
*/
- if (!F2FS_I(inode)->wb_task && has_not_enough_free_secs(sbi, 0, 0)) {
+ if (!f2fs_cp_controlled_inode(inode) &&
+ !F2FS_I(inode)->wb_task && has_not_enough_free_secs(sbi, 0, 0)) {
folio_redirty_for_writepage(wbc, folio);
retry = true;
folio_unlock(folio);
@@ -4647,31 +4720,37 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
/*
- * If cp_error is set, all subpages of this folio are not
- * written back unless the filesystem is mounted with
- * errors=remount-ro. The check is placed after the retry
- * block so it also covers cp_error being set while the folio
- * lock was dropped for f2fs_balance_fs() or for the trylock_op()
- * backoff. PG_dirty is clear at this point, so both the
- * redirty and the drop below leave a coherent state.
+ * Checked after the retry block, so that it also covers cp_error
+ * being set while the folio lock was dropped for f2fs_balance_fs()
+ * or for the trylock_op() backoff.
*/
- if (unlikely(f2fs_cp_error(sbi))) {
- mapping_set_error(mapping, -EIO);
- if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY) {
- folio_redirty_for_writepage(wbc, folio);
- goto retry_out;
- }
- goto out;
+ switch (f2fs_wb_folio_action(sbi, inode)) {
+ case F2FS_WB_REDIRTY:
+ folio_redirty_for_writepage(wbc, folio);
+ goto retry_out;
+ case F2FS_WB_DROP:
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ goto retry_out;
+ case F2FS_WB_WRITE:
+ break;
}
- /* To avoid dealing with the complexity for one subrange is in bio
- * while we trylock_op failed before writing another subrange.
- * Try to lock_op before any subrange write for the folio.
- */
- if (!op_locked) {
+ if (f2fs_cp_controlled_inode(inode)) {
+ /*
+ * Quota writes need node_write to avoid block
+ * allocation during checkpoint.
+ */
+ if (IS_NOQUOTA(inode)) {
+ f2fs_down_read_trace(&sbi->node_write, &lc);
+ node_locked = true;
+ }
+ } else {
+ /* To avoid dealing with the complexity for one subrange is in bio
+ * while we trylock_op failed before writing another subrange.
+ * Try to lock_op before any subrange write for the folio.
+ */
if (!f2fs_trylock_op(sbi, &lc)) {
folio_redirty_for_writepage(wbc, folio);
- err = 0;
if (wbc->sync_mode != WB_SYNC_ALL)
goto retry_out;
@@ -4701,59 +4780,44 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_start_writeback(folio);
/*
- * PG_dirty was already cleared by writeback_iter(), and a
- * single-block folio has no ffs dirty bitmap, so the dirty
- * range cannot be derived. Write the whole folio.
+ * PG_dirty was already cleared by writeback_iter(). A single-block
+ * folio has no ffs, so its whole folio is written.
*/
- if (!folio_test_large(folio)) {
- err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos, end_pos - pos);
- folio_submitted += submitted;
- nwritten += submitted;
- if (err)
- goto out;
- goto written;
- }
-
- if (!f2fs_folio_has_ffs(folio)) {
- ffs = f2fs_ffs_find_or_alloc(folio);
- f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
- } else {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ if (!f2fs_folio_has_ffs(folio)) {
+ /*
+ * PG_dirty, which was cleared for writeback,
+ * covered every block and the folio was
+ * accounted as a whole. Account its dirty
+ * bits instead.
+ */
+ WARN_ON_ONCE(folio_test_dirty(folio));
+ f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
+ f2fs_account_dirty_blocks(inode,
+ -(int)f2fs_blocks_per_folio(sbi, folio));
+ }
ffs = (struct f2fs_folio_state *)folio->private;
- }
- /*
- * Large folios can be submitted as multiple bios, which may
- * complete while we are still submitting the remaining dirty
- * ranges. Hold a bias on write_blocks_pending to prevent
- * folio_end_writeback() from being called early.
- */
- if (!bias_added) {
- WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
- atomic_inc(&ffs->write_blocks_pending);
- bias_added = true;
+ /*
+ * Multiblock folios can be submitted as multiple bios,
+ * which may complete while we are still submitting the
+ * remaining dirty ranges. Hold a bias on
+ * write_blocks_pending to prevent folio_end_writeback()
+ * from being called early.
+ */
+ if (!bias_added) {
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
+ bias_added = true;
+ }
}
- unsigned int from = 0, start, count;
- unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- unsigned int to_blk = min_t(unsigned int,
- F2FS_BLK_ALIGN(sbi, end_pos - folio_pos(folio)),
- nr_blocks);
-
- while (f2fs_ffs_dirty_blk_range(folio, from, to_blk,
- &start, &count)) {
- submitted = 0;
- pos = folio_pos(folio) + F2FS_BLK_TO_BYTES(sbi, start);
- err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos,
- F2FS_BLK_TO_BYTES(sbi, count));
- folio_submitted += submitted;
- nwritten += submitted;
- if (err)
- goto out;
- from = start + count;
- }
+ err = f2fs_write_folio_dirty_range(folio, &submitted, wbc, io_type,
+ pos, end_pos);
+ folio_submitted += submitted;
+ nwritten += submitted;
+ if (err)
+ goto out;
-written:
/* If none of the subpages of a folio is submitted, keep the atomic flag */
if (folio_submitted &&
f2fs_is_atomic_file(inode) &&
@@ -4761,27 +4825,10 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_clear_f2fs_atomic(folio);
out:
- /*
- * Clear the dirty state of the subranges that were written back.
- * If some subranges are still dirty (e.g. the folio was only
- * partially written back), or if the writeback failed, the
- * folio must stay dirty so that the remaining subranges are
- * written back later.
- *
- * On success, clear dirty bits of the whole folio rather than
- * up to isize: subpages beyond EOF can still carry dirty bits
- * (e.g. fill_zero() w/ FALLOC_FL_KEEP_SIZE, BG_GC on blocks
- * beyond EOF, or f2fs_ffs_find_or_alloc() on a dirty folio),
- * they will never be written back, and would keep the folio
- * redirtied forever.
- */
- clear_len = err ? pos - folio_pos(folio) : folio_size(folio);
- if (f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
- clear_len) || err) {
+ if (!err)
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ else
folio_redirty_for_writepage(wbc, folio);
- } else if (!f2fs_folio_has_ffs(folio)) {
- inode_dec_dirty_block(inode);
- }
/*
* Release the bias taken above and we end writeback here if
@@ -4791,7 +4838,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
f2fs_ffs_put_write_pending(folio, 1);
} else if (!folio_submitted && folio_test_writeback(folio)) {
/*
- * Order-0 folios take no bias, so end the writeback
+ * Single-block folios take no bias, so end the writeback
* if nothing was submitted.
*/
folio_end_writeback(folio);
@@ -4803,6 +4850,11 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
op_locked = false;
}
+ if (node_locked) {
+ f2fs_up_read_trace(&sbi->node_write, &lc);
+ node_locked = false;
+ }
+
folio_unlock(folio);
}
@@ -4883,10 +4935,10 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
goto skip_write;
- if ((S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode)) &&
- wbc->sync_mode == WB_SYNC_NONE &&
- get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
- f2fs_available_free_memory(sbi, DIRTY_DENTS))
+ if (f2fs_cp_controlled_inode(inode) &&
+ wbc->sync_mode == WB_SYNC_NONE &&
+ get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
+ f2fs_available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
/* skip writing in file defragment preparing stage */
@@ -4913,7 +4965,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (mapping_large_folio_support(inode->i_mapping) &&
+ if (f2fs_has_multiblock_folios(inode) &&
!f2fs_has_inline_data(inode))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
@@ -5147,7 +5199,7 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
int err = 0;
block_t ori_blk_addr = NULL_ADDR;
bool cow_has_reserved_block = false;
- bool append = ((loff_t)index << PAGE_SHIFT) >= i_size_read(inode);
+ bool append = F2FS_BLK_TO_BYTES(sbi, index) >= i_size_read(inode);
/*
* On the first attempt, an append write can skip the COW lookup and
@@ -5244,6 +5296,10 @@ static int prepare_large_folio_write_begin(struct inode *inode,
size_t need_off = ori_off;
int err = 0;
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode)))
+ return -EOPNOTSUPP;
+
/*
* when the write did not come through f2fs_write_iter, we preallocate all
* blocks in the write range here.
@@ -5276,8 +5332,12 @@ static int prepare_large_folio_write_begin(struct inode *inode,
if (folio_test_uptodate(folio) || len == folio_size(folio))
return 0;
+ /*
+ * Skip read if the write starts at a block boundary and reaches EOF,
+ * and zero the rest of the folio instead.
+ */
if (!f2fs_verity_in_progress(inode) &&
- !(pos & (PAGE_SIZE - 1)) && (pos + len) >= i_size_read(inode)) {
+ !(pos & F2FS_BLKSIZE_MASK(sbi)) && (pos + len) >= i_size_read(inode)) {
size_t zoff = ori_off + len;
if (zoff < folio_size(folio))
@@ -5291,7 +5351,8 @@ static int prepare_large_folio_write_begin(struct inode *inode,
while (find_next_valid_block(folio, ori_off,
&need_off, len)) {
struct dnode_of_data dn;
- pgoff_t index = folio->index + (need_off >> PAGE_SHIFT);
+ pgoff_t index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, need_off);
block_t blkaddr;
if (!f2fs_lookup_read_extent_cache_block(inode, index,
@@ -5307,12 +5368,13 @@ static int prepare_large_folio_write_begin(struct inode *inode,
f2fs_put_dnode(&dn);
if (blkaddr == NEW_ADDR) {
- size_t off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t off = f2fs_folio_byte_offset(sbi,
+ folio, index);
- folio_zero_segment(folio, off, off + PAGE_SIZE);
+ folio_zero_segment(folio, off,
+ off + F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_uptodate(folio, off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
continue;
}
}
@@ -5410,12 +5472,18 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto fail;
}
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ err = -EOPNOTSUPP;
+ goto fail;
+ }
+
/*
* We should check this at this moment to avoid deadlock on inode page
* and #0 page. The locking rule for inline_data conversion should be:
* folio_lock(folio #0) -> folio_lock(inode_page)
*/
- if (index != 0) {
+ if (index != 0 || f2fs_has_subpage_blocks(sbi)) {
err = f2fs_convert_inline_inode(inode);
if (err)
goto fail;
@@ -5445,6 +5513,7 @@ static int f2fs_write_begin(const struct kiocb *iocb,
#endif
repeat:
+ need_balance = false;
/*
* Do not use FGP_STABLE to avoid deadlock.
* Will wait that below with our IO control.
@@ -5478,17 +5547,18 @@ static int f2fs_write_begin(const struct kiocb *iocb,
*/
f2fs_folio_wait_writeback(folio, false, true);
- if (f2fs_is_atomic_file(inode) && !folio_test_large(folio))
+ if (f2fs_is_atomic_file(inode) && !f2fs_is_multiblock_folio(sbi, folio)) {
err = prepare_atomic_write_begin(sbi, inode, folio->index,
&blkaddr, &need_balance, retried);
- else if (!folio_test_large(folio))
+ } else if (!f2fs_is_multiblock_folio(sbi, folio)) {
err = prepare_write_begin(sbi, folio, pos, blen,
&blkaddr, &need_balance);
- else if (f2fs_is_atomic_file(inode))
+ } else if (f2fs_is_atomic_file(inode)) {
err = prepare_large_folio_atomic_write_begin(inode,
mapping, folio, pos, len, retried);
- else
+ } else {
err = prepare_large_folio_write_begin(inode, folio, pos, blen);
+ }
if (err) {
if (err == -EAGAIN) {
retried = true;
@@ -5504,7 +5574,7 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto repeat;
}
- if (folio_test_large(folio))
+ if (f2fs_is_multiblock_folio(sbi, folio))
return 0;
if (blen == folio_size(folio) || folio_test_uptodate(folio))
@@ -5574,7 +5644,7 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!folio_test_uptodate(folio)) {
if (unlikely(copied != len)) {
copied = 0;
- } else if (folio_test_large(folio)) {
+ } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
f2fs_ffs_mark_subrange_uptodate(folio,
offset_in_folio(folio, pos), len);
} else {
@@ -5599,10 +5669,10 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!copied)
goto unlock_out;
- if (folio_test_large(folio))
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio))
f2fs_ffs_mark_subrange_dirty(folio, offset_in_folio(folio, pos),
copied);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
if (f2fs_is_atomic_file(inode))
folio_set_f2fs_atomic(folio);
@@ -5655,6 +5725,7 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
* dirty state.
*/
if (!dirty && folio_test_dirty(folio)) {
+ f2fs_remove_dirty_inode(inode);
f2fs_clear_page_cache_dirty_tag(folio);
folio_cancel_dirty(folio);
}
@@ -5663,11 +5734,8 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
return;
}
- if (f2fs_folio_has_ffs(folio)) {
- f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
- folio_size(folio));
- } else if (folio_test_dirty(folio)) {
- inode_dec_dirty_block(inode);
+ if (f2fs_folio_has_ffs(folio) || folio_test_dirty(folio)) {
+ f2fs_folio_clear_dirty_blocks(folio, true);
f2fs_remove_dirty_inode(inode);
}
@@ -5698,7 +5766,7 @@ bool f2fs_release_folio(struct folio *folio, gfp_t wait)
return true;
}
-static bool f2fs_dirty_data_folio(struct address_space *mapping,
+static bool __f2fs_dirty_data_folio(struct address_space *mapping,
struct folio *folio)
{
struct inode *inode = mapping->host;
@@ -5731,6 +5799,55 @@ static bool f2fs_dirty_data_folio(struct address_space *mapping,
return false;
}
+/*
+ * Dirty @folio for an update made by f2fs itself, which has marked the
+ * dirty blocks of a multi-block folio with f2fs_ffs_mark_subrange_dirty().
+ */
+bool f2fs_folio_mark_dirty(struct folio *folio)
+{
+ return __f2fs_dirty_data_folio(folio->mapping, folio);
+}
+
+/*
+ * mm dirtied @folio without telling which blocks were modified, e.g. through
+ * a writable mapping or a pinned page: mark its uptodate blocks within EOF
+ * dirty.
+ */
+static void f2fs_ffs_mark_uptodate_blocks_dirty(struct inode *inode,
+ struct folio *folio)
+{
+ struct f2fs_folio_state *ffs = folio->private;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(F2FS_I_SB(inode), folio);
+ unsigned int valid_blocks = f2fs_folio_valid_blocks(inode, folio);
+ unsigned int i, set = 0;
+ unsigned long flags;
+
+ spin_lock_irqsave(&ffs->state_lock, flags);
+ for (i = 0; i < valid_blocks; i++) {
+ if (test_bit(i, ffs->state) &&
+ !__test_and_set_bit(nr_blocks + i, ffs->state))
+ set++;
+ }
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+
+ if (set)
+ f2fs_account_dirty_blocks(inode, set);
+}
+
+static bool f2fs_dirty_data_folio(struct address_space *mapping,
+ struct folio *folio)
+{
+ struct inode *inode = mapping->host;
+
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_uptodate_blocks_dirty(inode, folio);
+ else
+ /* A folio state is attached under the folio lock. */
+ WARN_ON_ONCE(!folio_test_locked(folio) &&
+ f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio));
+ return __f2fs_dirty_data_folio(mapping, folio);
+}
+
static sector_t f2fs_bmap_compress(struct inode *inode, sector_t block)
{
@@ -5847,7 +5964,7 @@ static int f2fs_migrate_blocks(struct inode *inode, block_t start_blk,
f2fs_ffs_mark_subrange_dirty(folio,
offset_in_folio(folio, (loff_t)blkidx << PAGE_SHIFT),
PAGE_SIZE);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
f2fs_folio_put(folio, true);
}
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 91929c250085..c4ca8ba8cd03 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -57,7 +57,7 @@ void f2fs_mark_dentry_folio_dirty(struct inode *dir, struct folio *folio,
} else if (!folio_test_uptodate(folio)) {
folio_mark_uptodate(folio);
}
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
}
static unsigned long dir_blocks(struct inode *inode)
@@ -979,9 +979,9 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
if (!f2fs_has_subpage_blocks(sbi)) {
folio_clear_uptodate(folio);
folio_detach_private(folio);
- inode_dec_dirty_block(dir);
}
+ f2fs_folio_clear_dirty_blocks(folio, true);
f2fs_remove_dirty_inode(dir);
}
keep_dirty:
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index f7234283d33d..449f8c672810 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4531,7 +4531,10 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
int compr_blocks, bool allow_balance);
bool f2fs_ffs_test_blk_uptodate(const struct folio *folio, pgoff_t index);
struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio);
+bool f2fs_folio_mark_dirty(struct folio *folio);
+void f2fs_folio_clear_dirty_blocks(struct folio *folio, bool dirty);
bool f2fs_ffs_test_blk_dirty(const struct folio *folio, pgoff_t index);
+void f2fs_ffs_put_write_pending(struct folio *folio, unsigned int nr);
void f2fs_ffs_mark_subrange_dirty(struct folio *folio, size_t offset, size_t len);
bool f2fs_ffs_has_dirty_subpage(struct folio *folio);
bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index b5fa4df9f2e3..4c4eb02eb486 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -258,7 +258,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
if (folio_test_large(folio))
f2fs_ffs_mark_subrange_dirty(folio, 0, dirty_len);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
f2fs_update_iostat(sbi, inode, APP_MAPPED_IO, dirty_len);
f2fs_update_time(sbi, REQ_TIME);
@@ -856,7 +856,7 @@ static int truncate_partial_data_page(struct inode *inode, u64 from,
*/
f2fs_ffs_mark_subrange_dirty(folio, folio_off,
blk_end - folio_off);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
}
f2fs_folio_put(folio, true);
return 0;
@@ -1339,7 +1339,7 @@ static int fill_zero(struct inode *inode, pgoff_t index,
* f2fs_get_new_data_folio() above.
*/
f2fs_ffs_mark_subrange_dirty(folio, folio_off, len);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
f2fs_folio_put(folio, true);
return 0;
}
@@ -1627,7 +1627,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
F2FS_BLKSIZE(sbi));
}
- folio_mark_dirty(fdst);
+ f2fs_folio_mark_dirty(fdst);
if (same_folio && i_size_read(dst_inode) <
F2FS_BLK_TO_BYTES(sbi, dst + i + 1))
f2fs_i_size_write(dst_inode,
@@ -3222,7 +3222,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
f2fs_ffs_mark_subrange_dirty(folio,
offset_in_folio(folio, (loff_t)idx << PAGE_SHIFT),
PAGE_SIZE);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
f2fs_folio_put(folio, true);
@@ -4967,7 +4967,7 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
}
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
redirty_idx = folio_next_index(folio);
folio_unlock(folio);
@@ -5561,12 +5561,13 @@ static int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *iter,
return ret;
}
- if (!dio && mapping_large_folio_support(inode->i_mapping)) {
+ if (!dio && f2fs_has_multiblock_folios(inode)) {
/*
- * Preallocate all blocks touched by a large-folio buffered write so
- * the regular write_begin path does not need to unlock the folio for
- * f2fs_balance_fs(). Rechecking large-folio state after unlock is
- * unreliable since partial truncation can split the folio.
+ * Preallocate all blocks touched by a multiblock-folio buffered
+ * write so the regular write_begin path does not need to unlock
+ * the folio for f2fs_balance_fs(). Rechecking large-folio
+ * state after unlock is unreliable since partial truncation can
+ * split the folio.
*/
map.m_lblk = F2FS_BYTES_TO_BLK(sbi, pos);
map.m_len = F2FS_BLK_ALIGN(sbi, pos + count);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 274c86511cde..c681f3558e85 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1586,7 +1586,7 @@ static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
}
if (has_ffs)
f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else {
struct f2fs_io_info fio = {
@@ -1611,15 +1611,15 @@ static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
if (has_ffs)
f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
if (folio_clear_dirty_for_io(folio)) {
- if (!has_ffs) {
- inode_dec_dirty_block(inode);
- f2fs_remove_dirty_inode(inode);
- } else if (f2fs_ffs_clear_subrange_dirty_and_test(folio,
- foff, F2FS_BLKSIZE(sbi))) {
- folio_mark_dirty(folio);
- }
+ /* The other dirty blocks keep the folio dirty. */
+ if (!has_ffs)
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ else if (f2fs_ffs_clear_subrange_dirty_and_test(folio,
+ foff, F2FS_BLKSIZE(sbi)))
+ f2fs_folio_mark_dirty(folio);
+ f2fs_remove_dirty_inode(inode);
}
folio_set_f2fs_gcing(folio);
@@ -1638,7 +1638,7 @@ static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
if (has_ffs)
f2fs_ffs_mark_subrange_dirty(folio, foff,
F2FS_BLKSIZE(sbi));
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else if (!f2fs_ffs_has_dirty_subpage(folio)) {
/* migrate this block in later GC */
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index ba0aff8e5643..032c1cf8c615 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -187,7 +187,7 @@ int f2fs_convert_inline_folio(struct dnode_of_data *dn, struct folio *folio)
f2fs_bug_on(F2FS_F_SB(folio), folio_test_writeback(folio));
f2fs_do_read_inline_data(folio, dn->inode_entry);
- folio_mark_dirty(folio);
+ f2fs_folio_mark_dirty(folio);
/* clear dirty state */
dirty = folio_clear_dirty_for_io(folio);
@@ -196,10 +196,23 @@ int f2fs_convert_inline_folio(struct dnode_of_data *dn, struct folio *folio)
folio_start_writeback(folio);
fio.old_blkaddr = dn->data_blkaddr;
set_inode_flag(dn->inode, FI_HOT_DATA);
+ if (f2fs_is_multiblock_folio(fio.sbi, folio)) {
+ struct f2fs_folio_state *ffs = f2fs_ffs_find_or_alloc(folio);
+
+ atomic_inc(&ffs->write_blocks_pending);
+ }
f2fs_outplace_write_data(dn, &fio);
+ /* Drop the pending write that a failed submission leaves behind. */
+ if (!fio.submitted && f2fs_is_multiblock_folio(fio.sbi, folio))
+ f2fs_ffs_put_write_pending(folio, 1);
f2fs_folio_wait_writeback(folio, true, true);
if (dirty) {
- inode_dec_dirty_block(dn->inode);
+ /*
+ * The folio of inline data was dirtied without a folio state,
+ * so it was accounted as a whole.
+ */
+ f2fs_account_dirty_blocks(dn->inode,
+ -(int)f2fs_blocks_per_folio(fio.sbi, folio));
f2fs_remove_dirty_inode(dn->inode);
}
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index c48b12208e20..f1eae132c4d3 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -1700,10 +1700,13 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino)
goto folio_out;
ret = f2fs_write_inline_data(inode, folio);
- inode_dec_dirty_block(inode);
- f2fs_remove_dirty_inode(inode);
- if (ret)
- folio_mark_dirty(folio);
+ if (ret) {
+ /* Keep it dirty as it was, with its dirty blocks accounted. */
+ filemap_dirty_folio(folio->mapping, folio);
+ } else {
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ f2fs_remove_dirty_inode(inode);
+ }
folio_out:
f2fs_folio_put(folio, true);
iput_out:
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 08/20] f2fs: adapt range operations and mmap for subpage blocks
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (6 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
` (11 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
- A writable PTE covers the whole folio, so page_mkwrite prepares
every block of the folio up to EOF and dirties only those blocks. It
reserves them only where it may allocate, i.e. not for a pinned file
or a compressed cluster.
- Align zero range, punch hole, preallocation and truncation to the
block size instead of PAGE_SIZE. When i_size grows, write-protect the
folio straddling the old EOF so that stores to its new blocks fault
into page_mkwrite.
- Update i_size of an extending direct write in ->end_io(), before
iomap invalidates the page cache of the written range, as ext4 does.
Otherwise, a read in between zero-fills the blocks beyond the old
i_size in the folio straddling it and marks them uptodate, which hides
the data just written.
- Unmap every base page overlapping a punched or zeroed range, and
truncate the page cache without truncate_pagecache_range(), whose
unmap underflows for a range ending in the first page.
- For SEEK_DATA, treat a NEW_ADDR block as data only if the block
itself is dirty.
- When truncation zeroes the tail of a page, mark only the block
holding the new EOF dirty, not the blocks after it.
The SEEK_DATA change also applies to large folios when the block size
equals PAGE_SIZE. The direct write change applies to any block size, so
i_size is also updated before the sync of an O_DSYNC direct write.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 69 ++++++++++--
fs/f2fs/f2fs.h | 4 +-
fs/f2fs/file.c | 300 +++++++++++++++++++++++++++++++++----------------
3 files changed, 261 insertions(+), 112 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 54b2024ece7b..98ceea49e426 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -3257,7 +3257,7 @@ bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
static bool find_next_valid_block(const struct folio *folio,
size_t orig_off, size_t *need_off,
- size_t len)
+ size_t len, bool read_full)
{
struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
size_t start = orig_off;
@@ -3265,6 +3265,20 @@ static bool find_next_valid_block(const struct folio *folio,
size_t head, tail;
pgoff_t index;
+ /* Find every block that is not uptodate, not only the partial ones. */
+ if (read_full) {
+ for (head = round_down(*need_off, F2FS_BLKSIZE(sbi)); head < end;
+ head += F2FS_BLKSIZE(sbi)) {
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, head);
+ if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
+ *need_off = head;
+ return true;
+ }
+ }
+ return false;
+ }
+
if (start & F2FS_BLKSIZE_MASK(sbi)) {
head = round_down(start, F2FS_BLKSIZE(sbi));
index = f2fs_folio_lblk(sbi, folio) +
@@ -5289,7 +5303,8 @@ static bool f2fs_balance_fs_folio_locked(struct folio *folio,
static int prepare_large_folio_write_begin(struct inode *inode,
struct folio *folio, loff_t pos,
- unsigned int len)
+ unsigned int len, bool read_full,
+ bool preallocated, bool balance)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
size_t ori_off = offset_in_folio(folio, pos);
@@ -5301,11 +5316,11 @@ static int prepare_large_folio_write_begin(struct inode *inode,
return -EOPNOTSUPP;
/*
- * when the write did not come through f2fs_write_iter, we preallocate all
- * blocks in the write range here.
+ * f2fs_balance_fs_folio_locked() unlocks the folio to balance. Skip it
+ * for page_mkwrite (!@balance), which cannot unlock the folio and has
+ * called f2fs_balance_fs() before locking it.
*/
- if (!is_inode_flag_set(inode, FI_PREALLOCATED_ALL) &&
- !IS_DEVICE_ALIASING(inode)) {
+ if (!preallocated && !IS_DEVICE_ALIASING(inode)) {
pgoff_t first = F2FS_BYTES_TO_BLK(sbi, pos);
pgoff_t last = F2FS_BLK_ALIGN(sbi, pos + len);
pgoff_t idx;
@@ -5318,8 +5333,9 @@ static int prepare_large_folio_write_begin(struct inode *inode,
&node_changed);
if (err)
return err;
- if (f2fs_balance_fs_folio_locked(folio, folio->mapping,
- node_changed))
+ if (balance &&
+ f2fs_balance_fs_folio_locked(folio, folio->mapping,
+ node_changed))
return -EAGAIN;
}
}
@@ -5329,14 +5345,15 @@ static int prepare_large_folio_write_begin(struct inode *inode,
/* Skip read if the folio is already fully uptodate or the write
* covers the entire folio.
*/
- if (folio_test_uptodate(folio) || len == folio_size(folio))
+ if (folio_test_uptodate(folio) ||
+ (!read_full && len == folio_size(folio)))
return 0;
/*
* Skip read if the write starts at a block boundary and reaches EOF,
* and zero the rest of the folio instead.
*/
- if (!f2fs_verity_in_progress(inode) &&
+ if (!read_full && !f2fs_verity_in_progress(inode) &&
!(pos & F2FS_BLKSIZE_MASK(sbi)) && (pos + len) >= i_size_read(inode)) {
size_t zoff = ori_off + len;
@@ -5349,7 +5366,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
while (find_next_valid_block(folio, ori_off,
- &need_off, len)) {
+ &need_off, len, read_full)) {
struct dnode_of_data dn;
pgoff_t index = f2fs_folio_lblk(sbi, folio) +
F2FS_BYTES_TO_BLK(sbi, need_off);
@@ -5449,6 +5466,20 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
return 0;
}
+int f2fs_prepare_subpage_mkwrite(struct inode *inode, struct folio *folio,
+ loff_t pos, unsigned int len, bool need_alloc)
+{
+ /*
+ * mkwrite runs without inode_lock, so FI_PREALLOCATED_ALL may have
+ * been set by a concurrent write() for a different range: reserve
+ * the blocks here, unless @need_alloc is false, as for a pinned file
+ * or a compressed cluster. It calls f2fs_balance_fs() before locking
+ * the folio, which it cannot unlock here, so do not balance.
+ */
+ return prepare_large_folio_write_begin(inode, folio, pos, len, true,
+ !need_alloc, false);
+}
+
static int f2fs_write_begin(const struct kiocb *iocb,
struct address_space *mapping,
loff_t pos, unsigned len, struct folio **foliop,
@@ -5557,7 +5588,10 @@ static int f2fs_write_begin(const struct kiocb *iocb,
err = prepare_large_folio_atomic_write_begin(inode,
mapping, folio, pos, len, retried);
} else {
- err = prepare_large_folio_write_begin(inode, folio, pos, blen);
+ err = prepare_large_folio_write_begin(inode, folio, pos, blen,
+ false,
+ is_inode_flag_set(inode, FI_PREALLOCATED_ALL),
+ true);
}
if (err) {
if (err == -EAGAIN) {
@@ -5633,6 +5667,7 @@ static int f2fs_write_end(const struct kiocb *iocb,
struct folio *folio, void *fsdata)
{
struct inode *inode = folio->mapping->host;
+ loff_t old_size = i_size_read(inode);
trace_f2fs_write_end(inode, pos, len, copied);
@@ -5662,6 +5697,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (pos + copied > i_size_read(inode) &&
!f2fs_verity_in_progress(inode))
f2fs_i_size_write(inode, pos + copied);
+ if (i_size_read(inode) > old_size)
+ pagecache_isize_extended(inode, old_size, pos);
return copied;
}
#endif
@@ -5687,6 +5724,14 @@ static int f2fs_write_end(const struct kiocb *iocb,
unlock_out:
f2fs_folio_put(folio, true);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
+
+ /*
+ * The folio straddling the old EOF may still be mapped writable, and
+ * the blocks between the old EOF and pos in it are not prepared.
+ * Write-protect it so that mmap stores to them fault.
+ */
+ if (i_size_read(inode) > old_size)
+ pagecache_isize_extended(inode, old_size, pos);
return copied;
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 449f8c672810..f588603dd1d0 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4094,7 +4094,7 @@ int f2fs_getattr(const struct mnt_idmap *idmap, const struct path *path,
struct kstat *stat, u32 request_mask, unsigned int flags);
int f2fs_setattr(const struct mnt_idmap *idmap, struct dentry *dentry,
struct iattr *attr);
-int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);
+int f2fs_truncate_hole(struct inode *inode, pgoff_t blk_start, pgoff_t blk_end);
void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count);
int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,
bool readonly, bool need_lock);
@@ -4543,6 +4543,8 @@ void f2fs_ffs_clear_subrange_uptodate(struct folio *folio, size_t offset,
size_t len);
bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
unsigned int to, unsigned int *start, unsigned int *count);
+int f2fs_prepare_subpage_mkwrite(struct inode *inode, struct folio *folio,
+ loff_t pos, unsigned int len, bool need_alloc);
void f2fs_write_failed(struct inode *inode, loff_t to);
void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);
bool f2fs_release_folio(struct folio *folio, gfp_t wait);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 4c4eb02eb486..bec7988ca54b 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -36,23 +36,24 @@
#include <trace/events/f2fs.h>
#include <uapi/linux/f2fs.h>
-static int fill_zero(struct inode *inode, pgoff_t index,
+static int fill_zero(struct inode *inode, pgoff_t bidx,
loff_t start, loff_t len);
static int do_zero_post_eof_page(struct inode *inode, loff_t new_size)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
loff_t old_size = i_size_read(inode);
unsigned int offset, len;
- pgoff_t index;
+ pgoff_t bidx;
int err;
- offset = old_size & (PAGE_SIZE - 1);
+ offset = old_size & F2FS_BLKSIZE_MASK(sbi);
if (!offset)
return 0;
- len = min_t(loff_t, PAGE_SIZE - offset, new_size - old_size);
- index = old_size >> PAGE_SHIFT;
+ len = min_t(loff_t, F2FS_BLKSIZE(sbi) - offset, new_size - old_size);
+ bidx = F2FS_BYTES_TO_BLK(sbi, old_size);
if (f2fs_has_inline_data(inode)) {
/* data post eof should be always zero */
@@ -63,7 +64,7 @@ static int do_zero_post_eof_page(struct inode *inode, loff_t new_size)
return err;
}
- err = fill_zero(inode, index, offset, len);
+ err = fill_zero(inode, bidx, offset, len);
if (err)
return err;
return filemap_write_and_wait_range(inode->i_mapping,
@@ -144,11 +145,11 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
bool need_alloc = !f2fs_is_pinned_file(inode);
pgoff_t pidx = folio->index + folio_page_idx(folio, vmf->page);
loff_t pos = (loff_t)pidx << PAGE_SHIFT;
+ pgoff_t bidx = F2FS_BYTES_TO_BLK(sbi, pos);
loff_t isize;
loff_t folio_start;
loff_t valid_end;
size_t dirty_len;
- size_t subpage_off;
pgoff_t i, nr;
int err = 0;
vm_fault_t ret;
@@ -212,14 +213,17 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
folio_lock(folio);
isize = i_size_read(inode);
folio_start = folio_pos(folio);
- subpage_off = offset_in_folio(folio, pos);
valid_end = min_t(loff_t, folio_start + folio_size(folio), isize);
dirty_len = valid_end - folio_start;
+ /*
+ * With subpage blocks, the folio can stay mapped after a partial
+ * invalidation, so its blocks that are not uptodate are read below.
+ */
if (unlikely(folio->mapping != inode->i_mapping ||
pos >= isize ||
- !f2fs_ffs_test_blk_uptodate(folio,
- folio->index + (subpage_off >> PAGE_SHIFT)))) {
+ (!f2fs_has_subpage_blocks(sbi) &&
+ !f2fs_ffs_test_blk_uptodate(folio, bidx)))) {
folio_unlock(folio);
err = -EFAULT;
goto out_sem;
@@ -240,6 +244,25 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
+ if (f2fs_has_subpage_blocks(sbi)) {
+ /*
+ * A writable PTE covers the whole base folio. Like ext4's
+ * block_page_mkwrite(), prepare every filesystem block through EOF
+ * before returning it writable, but dirty only those blocks. This
+ * prevents a later store in a sibling 4K block from bypassing block
+ * allocation while also keeping blocks beyond i_size clean.
+ */
+ f2fs_folio_wait_writeback(folio, false, true);
+ err = f2fs_prepare_subpage_mkwrite(inode, folio,
+ folio_pos(folio), dirty_len,
+ need_alloc);
+ if (err) {
+ folio_unlock(folio);
+ goto out_sem;
+ }
+ goto zero_tail;
+ }
+
for (i = 0; i < nr; i++) {
err = f2fs_get_block_mkwrite(inode, folio, folio->index + i,
need_alloc);
@@ -249,6 +272,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
}
}
+zero_tail:
/* page is wholly or partially inside EOF */
if (folio_start + folio_size(folio) > isize) {
size_t offset = offset_in_folio(folio, isize);
@@ -256,8 +280,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
folio_zero_segment(folio, offset, folio_size(folio));
}
- if (folio_test_large(folio))
- f2fs_ffs_mark_subrange_dirty(folio, 0, dirty_len);
+ f2fs_ffs_mark_subrange_dirty(folio, 0, dirty_len);
f2fs_folio_mark_dirty(folio);
f2fs_update_iostat(sbi, inode, APP_MAPPED_IO, dirty_len);
@@ -503,7 +526,9 @@ static bool __found_offset(struct address_space *mapping,
{
block_t blkaddr = f2fs_data_blkaddr(dn);
struct inode *inode = mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
bool compressed_cluster = false;
+ bool dirty = false;
if (f2fs_compressed_file(inode)) {
block_t first_blkaddr = data_blkaddr(dn->inode, dn->node_entry,
@@ -512,12 +537,38 @@ static bool __found_offset(struct address_space *mapping,
compressed_cluster = first_blkaddr == COMPRESS_ADDR;
}
+ /* Only SEEK_DATA needs to know whether a reserved block is dirty. */
+ if (blkaddr == NEW_ADDR && whence == SEEK_DATA) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, index);
+
+ if (xa_get_mark(&mapping->i_pages, f_index, PAGECACHE_TAG_DIRTY)) {
+ struct folio *folio = filemap_get_folio(mapping, f_index);
+
+ if (!IS_ERR(folio)) {
+ /*
+ * ffs is freed under the folio lock. We hold the
+ * dnode, so only trylock the folio, and take the
+ * block as dirty, as the dirty tag says, if the
+ * folio is busy.
+ */
+ if (folio_trylock(folio)) {
+ if (folio->mapping == mapping)
+ dirty = f2fs_ffs_test_blk_dirty(folio,
+ index);
+ folio_unlock(folio);
+ } else {
+ dirty = true;
+ }
+ f2fs_folio_put(folio, false);
+ }
+ }
+ }
+
switch (whence) {
case SEEK_DATA:
if (__is_valid_data_blkaddr(blkaddr))
return true;
- if (blkaddr == NEW_ADDR &&
- xa_get_mark(&mapping->i_pages, index, PAGECACHE_TAG_DIRTY))
+ if (dirty)
return true;
if (compressed_cluster)
return true;
@@ -815,20 +866,22 @@ void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count)
dn->ofs_in_node, nr_free);
}
-static int truncate_partial_data_page(struct inode *inode, u64 from,
+static int truncate_partial_data_block(struct inode *inode, u64 from,
bool cache_only)
{
- pgoff_t index = from >> PAGE_SHIFT;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ pgoff_t bidx = F2FS_BYTES_TO_BLK(sbi, from);
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, bidx);
struct address_space *mapping = inode->i_mapping;
struct folio *folio;
size_t folio_off;
- size_t blk_end;
+ size_t zero_end;
- if (!(from & (PAGE_SIZE - 1)) && !cache_only)
+ if (!(from & F2FS_BLKSIZE_MASK(sbi)) && !cache_only)
return 0;
if (cache_only) {
- folio = filemap_lock_folio(mapping, index);
+ folio = filemap_lock_folio(mapping, f_index);
if (IS_ERR(folio))
return 0;
if (folio_test_uptodate(folio))
@@ -837,25 +890,26 @@ static int truncate_partial_data_page(struct inode *inode, u64 from,
return 0;
}
- folio = f2fs_get_lock_data_folio(inode, index, true);
+ folio = f2fs_get_lock_data_folio(inode, bidx, true);
if (IS_ERR(folio))
return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
truncate_out:
f2fs_folio_wait_writeback(folio, true, true);
folio_off = offset_in_folio(folio, from);
- blk_end = min_t(size_t, round_up(folio_off, PAGE_SIZE),
+ zero_end = min_t(size_t, round_up(folio_off, PAGE_SIZE),
folio_size(folio));
- folio_zero_segment(folio, folio_off, blk_end);
+ folio_zero_segment(folio, folio_off, zero_end);
/* An encrypted inode should have a key and truncate the last page. */
- f2fs_bug_on(F2FS_I_SB(inode), cache_only && IS_ENCRYPTED(inode));
+ f2fs_bug_on(sbi, cache_only && IS_ENCRYPTED(inode));
if (!cache_only) {
/*
* The folio was read back above, so the subpage is
- * already uptodate and dirty data stays valid.
+ * already uptodate and dirty data stays valid. Mark only
+ * the block holding the new EOF dirty.
*/
f2fs_ffs_mark_subrange_dirty(folio, folio_off,
- blk_end - folio_off);
+ round_up(folio_off, F2FS_BLKSIZE(sbi)) - folio_off);
f2fs_folio_mark_dirty(folio);
}
f2fs_folio_put(folio, true);
@@ -870,7 +924,7 @@ int f2fs_do_truncate_blocks(struct inode *inode, u64 from, bool lock)
pgoff_t free_from;
int count = 0, err = 0;
struct f2fs_cached_block *ientry;
- bool truncate_page = false;
+ bool truncate_block = false;
trace_f2fs_truncate_blocks_enter(inode, from);
@@ -915,7 +969,7 @@ int f2fs_do_truncate_blocks(struct inode *inode, u64 from, bool lock)
if (f2fs_has_inline_data(inode)) {
f2fs_truncate_inline_inode(inode, ientry, from);
f2fs_put_cache(ientry, true);
- truncate_page = true;
+ truncate_block = true;
goto out;
}
@@ -944,9 +998,9 @@ int f2fs_do_truncate_blocks(struct inode *inode, u64 from, bool lock)
if (lock)
f2fs_unlock_op(sbi, &lc);
free_partial:
- /* lastly zero out the first data page */
+ /* lastly zero out the first data block */
if (!err)
- err = truncate_partial_data_page(inode, from, truncate_page);
+ err = truncate_partial_data_block(inode, from, truncate_block);
out_err:
trace_f2fs_truncate_blocks_exit(inode, err);
return err;
@@ -1019,9 +1073,8 @@ int f2fs_truncate(struct inode *inode)
* Always truncate page #0 to avoid page cache
* leak in evict() path.
*/
- truncate_inode_pages_range(inode->i_mapping,
- F2FS_BLK_TO_BYTES(sbi, 0),
- F2FS_BLK_END_BYTES(sbi, 0));
+ truncate_inode_pages_range(inode->i_mapping, 0,
+ PAGE_SIZE - 1);
return err;
}
}
@@ -1310,7 +1363,7 @@ const struct inode_operations f2fs_file_inode_operations = {
.fileattr_set = f2fs_fileattr_set,
};
-static int fill_zero(struct inode *inode, pgoff_t index,
+static int fill_zero(struct inode *inode, pgoff_t bidx,
loff_t start, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1324,15 +1377,14 @@ static int fill_zero(struct inode *inode, pgoff_t index,
f2fs_balance_fs(sbi, true);
f2fs_lock_op(sbi, &lc);
- folio = f2fs_get_new_data_folio(inode, NULL, index, false);
+ folio = f2fs_get_new_data_folio(inode, NULL, bidx, false);
f2fs_unlock_op(sbi, &lc);
if (IS_ERR(folio))
return PTR_ERR(folio);
f2fs_folio_wait_writeback(folio, true, true);
- folio_off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT) + start;
+ folio_off = f2fs_folio_byte_offset(sbi, folio, bidx) + start;
folio_zero_range(folio, folio_off, len);
/*
* This subpage must be uptodate here, which is guaranteed by
@@ -1344,41 +1396,68 @@ static int fill_zero(struct inode *inode, pgoff_t index,
return 0;
}
-int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
+int f2fs_truncate_hole(struct inode *inode, pgoff_t blk_start, pgoff_t blk_end)
{
int err;
- while (pg_start < pg_end) {
+ while (blk_start < blk_end) {
struct dnode_of_data dn;
pgoff_t end_offset, count;
set_new_dnode(&dn, inode, NULL, NULL, 0);
- err = f2fs_get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
+ err = f2fs_get_dnode_of_data(&dn, blk_start, LOOKUP_NODE);
if (err) {
if (err == -ENOENT) {
- pg_start = f2fs_get_next_block_offset(&dn,
- pg_start);
+ blk_start = f2fs_get_next_block_offset(&dn,
+ blk_start);
continue;
}
return err;
}
end_offset = ADDRS_PER_PAGE(dn.node_entry, inode);
- count = min(end_offset - dn.ofs_in_node, pg_end - pg_start);
+ count = min(end_offset - dn.ofs_in_node, blk_end - blk_start);
f2fs_bug_on(F2FS_I_SB(inode), count == 0 || count > end_offset);
f2fs_truncate_data_blocks_range(&dn, count);
f2fs_put_dnode(&dn);
- pg_start += count;
+ blk_start += count;
}
return 0;
}
+/*
+ * unmap_mapping_range() computes page count solely from holelen, which causes
+ * under-unmapping when start is not PAGE_SIZE aligned. Outward-align to PAGE_SIZE
+ * to guarantee all overlapping base pages are unmapped.
+ *
+ * Then truncate the page cache without truncate_pagecache_range(), whose own
+ * unmap underflows when the range ends in the first page and unmaps the rest
+ * of the file.
+ */
+static void f2fs_truncate_pagecache_range(struct inode *inode,
+ loff_t start, loff_t end)
+{
+ struct address_space *mapping = inode->i_mapping;
+ loff_t unmap_start, unmap_len;
+
+ if (!f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
+ truncate_pagecache_range(inode, start, end - 1);
+ return;
+ }
+
+ unmap_start = round_down(start, PAGE_SIZE);
+ unmap_len = round_up(end, PAGE_SIZE) - unmap_start;
+ unmap_mapping_range(mapping, unmap_start, unmap_len, 0);
+ truncate_inode_pages_range(mapping, start, end - 1);
+}
+
static int f2fs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
{
- pgoff_t pg_start, pg_end;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ pgoff_t blk_start, blk_end;
loff_t off_start, off_end;
int ret;
@@ -1390,47 +1469,46 @@ static int f2fs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- off_start = offset & (PAGE_SIZE - 1);
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ off_start = offset & F2FS_BLKSIZE_MASK(sbi);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- if (pg_start == pg_end) {
- ret = fill_zero(inode, pg_start, off_start,
+ if (blk_start == blk_end) {
+ ret = fill_zero(inode, blk_start, off_start,
off_end - off_start);
if (ret)
return ret;
} else {
if (off_start) {
- ret = fill_zero(inode, pg_start++, off_start,
- PAGE_SIZE - off_start);
+ ret = fill_zero(inode, blk_start++, off_start,
+ F2FS_BLKSIZE(sbi) - off_start);
if (ret)
return ret;
}
if (off_end) {
- ret = fill_zero(inode, pg_end, 0, off_end);
+ ret = fill_zero(inode, blk_end, 0, off_end);
if (ret)
return ret;
}
- if (pg_start < pg_end) {
- loff_t blk_start, blk_end;
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ if (blk_start < blk_end) {
+ loff_t byte_start, byte_end;
struct f2fs_lock_context lc;
f2fs_balance_fs(sbi, true);
- blk_start = (loff_t)pg_start << PAGE_SHIFT;
- blk_end = (loff_t)pg_end << PAGE_SHIFT;
+ byte_start = F2FS_BLK_TO_BYTES(sbi, blk_start);
+ byte_end = F2FS_BLK_TO_BYTES(sbi, blk_end);
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- truncate_pagecache_range(inode, blk_start, blk_end - 1);
+ f2fs_truncate_pagecache_range(inode, byte_start, byte_end);
f2fs_lock_op(sbi, &lc);
- ret = f2fs_truncate_hole(inode, pg_start, pg_end);
+ ret = f2fs_truncate_hole(inode, blk_start, blk_end);
f2fs_unlock_op(sbi, &lc);
filemap_invalidate_unlock(inode->i_mapping);
@@ -1621,7 +1699,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
f2fs_folio_wait_writeback(fdst, true, true);
memcpy_folio(fdst, dst_off, fsrc, src_off,
F2FS_BLKSIZE(sbi));
- if (folio_test_large(fdst)) {
+ if (f2fs_is_multiblock_folio(sbi, fdst)) {
f2fs_ffs_mark_subrange_uptodate(fdst,
dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
@@ -1830,7 +1908,7 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
- pgoff_t index, pg_start, pg_end;
+ pgoff_t blk_start, blk_end;
loff_t new_size = i_size_read(inode);
loff_t off_start, off_end;
int ret = 0;
@@ -1851,31 +1929,33 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
if (ret)
return ret;
- pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- off_start = offset & (PAGE_SIZE - 1);
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ off_start = offset & F2FS_BLKSIZE_MASK(sbi);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- if (pg_start == pg_end) {
- ret = fill_zero(inode, pg_start, off_start,
+ if (blk_start == blk_end) {
+ ret = fill_zero(inode, blk_start, off_start,
off_end - off_start);
if (ret)
return ret;
new_size = max_t(loff_t, new_size, offset + len);
} else {
+ pgoff_t index;
+
if (off_start) {
- ret = fill_zero(inode, pg_start++, off_start,
- PAGE_SIZE - off_start);
+ ret = fill_zero(inode, blk_start++, off_start,
+ F2FS_BLKSIZE(sbi) - off_start);
if (ret)
return ret;
new_size = max_t(loff_t, new_size,
- (loff_t)pg_start << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, blk_start));
}
- for (index = pg_start; index < pg_end;) {
+ for (index = blk_start; index < blk_end;) {
struct dnode_of_data dn;
struct f2fs_lock_context lc;
unsigned int end_offset;
@@ -1884,9 +1964,9 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(mapping);
- truncate_pagecache_range(inode,
- (loff_t)index << PAGE_SHIFT,
- ((loff_t)pg_end << PAGE_SHIFT) - 1);
+ f2fs_truncate_pagecache_range(inode,
+ F2FS_BLK_TO_BYTES(sbi, index),
+ F2FS_BLK_TO_BYTES(sbi, blk_end));
f2fs_lock_op(sbi, &lc);
@@ -1900,7 +1980,7 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
}
end_offset = ADDRS_PER_PAGE(dn.node_entry, inode);
- end = min(pg_end, end_offset - dn.ofs_in_node + index);
+ end = min(blk_end, end_offset - dn.ofs_in_node + index);
ret = f2fs_do_zero_range(&dn, index, end);
f2fs_put_dnode(&dn);
@@ -1916,11 +1996,11 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
index = end;
new_size = max_t(loff_t, new_size,
- (loff_t)index << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, index));
}
if (off_end) {
- ret = fill_zero(inode, pg_end, 0, off_end);
+ ret = fill_zero(inode, blk_end, 0, off_end);
if (ret)
goto out;
@@ -1930,10 +2010,14 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
out:
if (new_size > i_size_read(inode)) {
- if (mode & FALLOC_FL_KEEP_SIZE)
+ if (mode & FALLOC_FL_KEEP_SIZE) {
file_set_keep_isize(inode);
- else
+ } else {
+ loff_t old_size = i_size_read(inode);
+
f2fs_i_size_write(inode, new_size);
+ pagecache_isize_extended(inode, old_size, new_size);
+ }
}
return ret;
}
@@ -2034,7 +2118,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
.should_migrate_blocks = false,
.err_gc_skipped = true,
.nr_free_secs = 0 };
- pgoff_t pg_start, pg_end;
+ pgoff_t blk_start, blk_end;
loff_t new_size;
loff_t off_end;
block_t expanded = 0;
@@ -2054,12 +2138,12 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
f2fs_balance_fs(sbi, true);
- pg_start = ((unsigned long long)offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long)offset + len) >> PAGE_SHIFT;
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- map.m_lblk = pg_start;
- map.m_len = pg_end - pg_start;
+ map.m_lblk = blk_start;
+ map.m_len = blk_end - blk_start;
if (off_end)
map.m_len++;
@@ -2071,9 +2155,9 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
block_t sec_len;
if (map.m_lblk % sec_blks) {
- pg_start = rounddown(map.m_lblk, sec_blks);
- map.m_lblk = pg_start;
- map.m_len = pg_end - pg_start;
+ blk_start = rounddown(map.m_lblk, sec_blks);
+ map.m_lblk = blk_start;
+ map.m_len = blk_end - blk_start;
if (off_end)
map.m_len++;
}
@@ -2089,7 +2173,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
f2fs_warn_ratelimited(sbi,
"ino:%llu, start:%lu, end:%lu, need to trigger GC to "
"reclaim enough free segment when checkpoint is enabled",
- inode->i_ino, pg_start, pg_end);
+ inode->i_ino, blk_start, blk_end);
goto out_err;
}
}
@@ -2135,20 +2219,24 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
if (!expanded)
return err;
- last_off = pg_start + expanded - 1;
+ last_off = blk_start + expanded - 1;
/* update new size to the failed position */
- new_size = (last_off == pg_end) ? offset + len :
- (loff_t)(last_off + 1) << PAGE_SHIFT;
+ new_size = (last_off == blk_end) ? offset + len :
+ F2FS_BLK_TO_BYTES(sbi, last_off + 1);
} else {
- new_size = ((loff_t)pg_end << PAGE_SHIFT) + off_end;
+ new_size = F2FS_BLK_TO_BYTES(sbi, blk_end) + off_end;
}
if (new_size > i_size_read(inode)) {
- if (mode & FALLOC_FL_KEEP_SIZE)
+ if (mode & FALLOC_FL_KEEP_SIZE) {
file_set_keep_isize(inode);
- else
+ } else {
+ loff_t old_size = i_size_read(inode);
+
f2fs_i_size_write(inode, new_size);
+ pagecache_isize_extended(inode, old_size, new_size);
+ }
}
return err;
@@ -5623,11 +5711,27 @@ static ssize_t f2fs_buffered_write_iter(struct kiocb *iocb,
static int f2fs_dio_write_end_io(struct kiocb *iocb, ssize_t size, int error,
unsigned int flags)
{
- struct f2fs_sb_info *sbi = F2FS_I_SB(file_inode(iocb->ki_filp));
+ struct inode *inode = file_inode(iocb->ki_filp);
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ /* iomap advances ki_pos after ->end_io() */
+ loff_t end = iocb->ki_pos + size;
+ loff_t old_size = i_size_read(inode);
dec_cache_count(sbi, F2FS_DIO_WRITE);
if (error)
return error;
+
+ /*
+ * Update i_size before iomap invalidates the page cache of the
+ * written range. Otherwise, a read in between zero-fills the blocks
+ * beyond the old i_size in the folio straddling it and marks them
+ * uptodate, which hides the data just written. An extending write
+ * waits for the I/O, so this runs in the task holding inode_lock.
+ */
+ if (size && end > old_size) {
+ f2fs_i_size_write(inode, end);
+ pagecache_isize_extended(inode, old_size, end);
+ }
f2fs_update_time(sbi, REQ_TIME);
f2fs_update_iostat(sbi, NULL, APP_DIRECT_IO, size);
return 0;
@@ -5758,8 +5862,6 @@ static ssize_t f2fs_dio_write_iter(struct kiocb *iocb, struct iov_iter *from,
if (ret < 0)
goto out;
- if (pos + ret > inode->i_size)
- f2fs_i_size_write(inode, pos + ret);
if (!do_opu)
set_inode_flag(inode, FI_UPDATE_WRITE);
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 09/20] f2fs: support subpage blocks in pinned files
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (7 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
` (10 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Pinned files must not allocate blocks outside fallocate(). As write()
does, return -EIO from page_mkwrite when the folio range up to EOF is
not fully allocated, and remove the -EOPNOTSUPP checks.
page_mkwrite keeps failing a device aliasing file with -EIO, as it does
when the block size equals PAGE_SIZE. Its block addresses are only in
the extent cache, so f2fs_overwrite_io() does not catch it.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index bec7988ca54b..72c601d240d4 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -142,7 +142,8 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
struct folio *folio = page_folio(vmf->page);
struct inode *inode = file_inode(vmf->vma->vm_file);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- bool need_alloc = !f2fs_is_pinned_file(inode);
+ bool pinned = f2fs_is_pinned_file(inode);
+ bool need_alloc = !pinned;
pgoff_t pidx = folio->index + folio_page_idx(folio, vmf->page);
loff_t pos = (loff_t)pidx << PAGE_SHIFT;
pgoff_t bidx = F2FS_BYTES_TO_BLK(sbi, pos);
@@ -245,6 +246,19 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
if (f2fs_has_subpage_blocks(sbi)) {
+ /*
+ * A device aliasing file has its block addresses only in the
+ * extent cache, where f2fs_overwrite_io() finds them. Fail it
+ * as f2fs_get_block_mkwrite() does.
+ */
+ if (pinned &&
+ (IS_DEVICE_ALIASING(inode) ||
+ !f2fs_overwrite_io(inode, folio_pos(folio), dirty_len))) {
+ folio_unlock(folio);
+ err = -EIO;
+ goto out_sem;
+ }
+
/*
* A writable PTE covers the whole base folio. Like ext4's
* block_page_mkwrite(), prepare every filesystem block through EOF
@@ -2285,8 +2299,7 @@ static long f2fs_fallocate(struct file *file, int mode,
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
- f2fs_is_pinned_file(inode) || f2fs_is_atomic_file(inode) ||
- f2fs_compressed_file(inode)) {
+ f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
@@ -3828,9 +3841,6 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
__u32 pin;
int ret = 0;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (get_user(pin, (__u32 __user *)arg))
return -EFAULT;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 10/20] f2fs: support subpage blocks in collapse and insert range
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (8 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 11/20] f2fs: support subpage blocks in move range Daeho Jeong
` (9 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The folio holding offset may hold blocks before the shifted range.
Write back the page cache from the PAGE_SIZE boundary below offset
under invalidate_lock before truncating it from there, so that no
data is lost. Detect src and dst in the same folio by folio index
instead of block index, as a folio holds several blocks. Remove the
-EOPNOTSUPP check.
Also take i_gc_rwsem[WRITE] and invalidate_lock before that writeback
and keep them until the moved blocks are written back and the page
cache is dropped, as move range does. After the block exchange, the
page cache holds the old data of the source blocks until then. With
subpage blocks, page_mkwrite dirties every block of a folio up to EOF,
so a store to a neighbouring block in between writes the old data back,
over a hole or over the moved data. This also keeps GC away from the
inode until then, for any block size.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 115 ++++++++++++++++++++++++++++---------------------
1 file changed, 65 insertions(+), 50 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 72c601d240d4..d4472143dea0 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1671,6 +1671,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
} else {
struct folio *fsrc, *fdst;
size_t src_off, dst_off;
+ pgoff_t dst_fidx = f2fs_lblk_to_folio_index(sbi, dst + i);
bool same_folio;
fsrc = f2fs_get_lock_data_folio(src_inode,
@@ -1683,11 +1684,11 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
/*
* For same-inode operations (collapse/insert), src and
- * dst may fall within the same large folio. Detect this
+ * dst may fall within the same folio. Detect this
* to avoid self-deadlock on folio lock.
*/
same_folio = src_inode == dst_inode &&
- folio_contains(fsrc, dst + i);
+ folio_contains(fsrc, dst_fidx);
if (same_folio) {
struct dnode_of_data dn;
@@ -1795,30 +1796,30 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_lock_context lc;
- pgoff_t nrpages = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t nrblocks = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
pgoff_t start = F2FS_BYTES_TO_BLK(sbi, offset);
pgoff_t end = F2FS_BYTES_TO_BLK(sbi, offset + len);
int ret;
- f2fs_balance_fs(sbi, true);
-
- /* avoid gc operation during block exchange */
- f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
- filemap_invalidate_lock(inode->i_mapping);
+ /*
+ * Write back under invalidate_lock, which blocks page_mkwrite, so
+ * that truncate_pagecache() below cannot drop a racing mmap store to
+ * the blocks before @offset in the head folio.
+ */
+ ret = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ if (ret)
+ return ret;
ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
if (ret)
- goto out_unlock;
+ return ret;
f2fs_lock_op(sbi, &lc);
f2fs_drop_extent_tree(inode);
- truncate_pagecache(inode, offset);
- ret = __exchange_data_block(inode, inode, end, start, nrpages - end, true);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
+ ret = __exchange_data_block(inode, inode, end, start, nrblocks - end, true);
f2fs_unlock_op(sbi, &lc);
-
-out_unlock:
- filemap_invalidate_unlock(inode->i_mapping);
- f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
}
@@ -1839,28 +1840,35 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
- if (ret)
- return ret;
+ f2fs_balance_fs(F2FS_I_SB(inode), true);
+
+ /*
+ * Avoid GC during the block exchange, and keep both locks until the
+ * moved blocks are written back and the page cache is dropped. Until
+ * then, it holds the old data of the source blocks, which neither
+ * page_mkwrite nor GC may write back.
+ */
+ f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
+ filemap_invalidate_lock(inode->i_mapping);
ret = f2fs_do_collapse(inode, offset, len);
if (ret)
- return ret;
+ goto out_unlock;
/* write out all moved pages, if possible */
- filemap_invalidate_lock(inode->i_mapping);
- ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
+ ret = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
if (ret)
goto out_unlock;
- truncate_pagecache(inode, offset);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
new_size = i_size_read(inode) - len;
ret = f2fs_truncate_blocks(inode, new_size, true);
-out_unlock:
- filemap_invalidate_unlock(inode->i_mapping);
if (!ret)
f2fs_i_size_write(inode, new_size);
+out_unlock:
+ filemap_invalidate_unlock(inode->i_mapping);
+ f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
}
@@ -2040,7 +2048,7 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
- pgoff_t nr, pg_start, pg_end, delta, idx;
+ pgoff_t nr, blk_start, blk_end, delta, idx;
loff_t new_size;
int ret = 0;
@@ -2053,8 +2061,8 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
return -EINVAL;
/* insert range should be aligned to block size of f2fs. */
- if (offset & F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)) ||
- len & F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)))
+ if (offset & F2FS_BLKSIZE_MASK(sbi) ||
+ len & F2FS_BLKSIZE_MASK(sbi))
return -EINVAL;
ret = f2fs_convert_inline_inode(inode);
@@ -2069,29 +2077,34 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(mapping, offset, LLONG_MAX);
- if (ret)
- return ret;
-
- pg_start = F2FS_BYTES_TO_BLK(sbi, offset);
- pg_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- delta = pg_end - pg_start;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
+ delta = blk_end - blk_start;
idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
/* avoid gc operation during block exchange */
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(mapping);
+ /*
+ * Write back under invalidate_lock, which blocks page_mkwrite, so
+ * that truncate_pagecache() below cannot drop a racing mmap store to
+ * the blocks before @offset in the head folio.
+ */
+ ret = filemap_write_and_wait_range(mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ if (ret)
+ goto out_unlock;
+
ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
if (ret)
goto out_unlock;
- truncate_pagecache(inode, offset);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
- while (!ret && idx > pg_start) {
+ while (!ret && idx > blk_start) {
struct f2fs_lock_context lc;
- nr = idx - pg_start;
+ nr = idx - blk_start;
if (nr > delta)
nr = delta;
idx -= nr;
@@ -2103,20 +2116,23 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
idx + delta, nr, false);
f2fs_unlock_op(sbi, &lc);
}
-out_unlock:
- filemap_invalidate_unlock(mapping);
- f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
if (ret)
- return ret;
+ goto out_unlock;
- /* write out all moved pages, if possible */
- filemap_invalidate_lock(mapping);
- ret = filemap_write_and_wait_range(mapping, offset, LLONG_MAX);
- truncate_pagecache(inode, offset);
- filemap_invalidate_unlock(mapping);
+ /*
+ * Write out all moved pages, if possible. Keep both locks until the
+ * page cache is dropped. Until then, it holds the old data of the
+ * source blocks, which neither page_mkwrite nor GC may write back.
+ */
+ ret = filemap_write_and_wait_range(mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
if (!ret)
f2fs_i_size_write(inode, new_size);
+out_unlock:
+ filemap_invalidate_unlock(mapping);
+ f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
return ret;
}
@@ -2298,8 +2314,7 @@ static long f2fs_fallocate(struct file *file, int mode,
}
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
- f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
+ if (f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 11/20] f2fs: support subpage blocks in move range
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (9 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
` (8 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Folios at both ends of the moved ranges can straddle the range
boundaries. Write back and truncate the page cache from PAGE_SIZE
boundaries, and hold invalidate_lock of both inodes during the exchange
so that faults and readahead cannot bring back stale data. After the
exchange, also after one that failed part way, drop the source blocks
that it read into the page cache, and call pagecache_isize_extended()
for the gap between the old EOF of dst and the moved data. The page
cache was not truncated before, so this also applies when the block
size equals PAGE_SIZE. Remove the -EOPNOTSUPP check.
Zero the cached data past the old EOF of dst with
f2fs_zero_post_eof_page() before the move extends dst over it, as the
other paths that extend a file do. Otherwise data stored past EOF
through a mapping becomes file data. This also applies when the block
size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 73 +++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 58 insertions(+), 15 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index d4472143dea0..4499a461be86 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -3435,7 +3435,7 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
size_t olen = len;
loff_t dst_max_i_size = 0;
loff_t dst_osize, dst_end;
- int ret;
+ int ret, err;
if (file_in->f_path.mnt != file_out->f_path.mnt ||
src->i_sb != dst->i_sb)
@@ -3516,17 +3516,6 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
if (ret)
goto out_unlock;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(src->i_mapping,
- pos_in, pos_in + len);
- if (ret)
- goto out_unlock;
-
- ret = filemap_write_and_wait_range(dst->i_mapping,
- pos_out, pos_out + len);
- if (ret)
- goto out_unlock;
-
f2fs_balance_fs(sbi, true);
f2fs_down_write(&F2FS_I(src)->i_gc_rwsem[WRITE]);
@@ -3536,6 +3525,39 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
goto out_src;
}
+ /*
+ * Hold invalidate_lock across writeback, truncation and the block
+ * exchange so neither faults nor readahead can repopulate the page
+ * cache with pre-exchange data.
+ */
+ filemap_invalidate_lock_two(src->i_mapping, dst->i_mapping);
+
+ /*
+ * Zero the cached data past the old EOF of dst, such as mmap stores
+ * past EOF, before the move extends dst over it.
+ */
+ ret = f2fs_zero_post_eof_page(dst, dst_end, false, false);
+ if (ret)
+ goto out_inval;
+
+ /* write out all dirty pages from offset */
+ ret = filemap_write_and_wait_range(src->i_mapping,
+ round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ if (ret)
+ goto out_inval;
+
+ ret = filemap_write_and_wait_range(dst->i_mapping,
+ round_down(pos_out, PAGE_SIZE),
+ round_up(pos_out + len, PAGE_SIZE) - 1);
+ if (ret)
+ goto out_inval;
+
+ truncate_pagecache_range(src, round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ truncate_pagecache_range(dst, round_down(pos_out, PAGE_SIZE),
+ round_up(pos_out + len, PAGE_SIZE) - 1);
+
f2fs_lock_op(sbi, &lc);
ret = __exchange_data_block(src, dst, F2FS_BYTES_TO_BLK(sbi, pos_in),
F2FS_BYTES_TO_BLK(sbi, pos_out),
@@ -3549,6 +3571,30 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
}
f2fs_unlock_op(sbi, &lc);
+ /*
+ * __clone_blkaddrs() reads the checkpointed source blocks into the
+ * page cache before punching them, so drop them again. Do it after a
+ * failed exchange too: the blocks moved before the failure were
+ * punched as well.
+ */
+ err = filemap_write_and_wait_range(src->i_mapping,
+ round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ if (!err)
+ truncate_pagecache_range(src, round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ if (!ret)
+ ret = err;
+
+ /*
+ * Write-protect the folio straddling the old EOF of dst, so that mmap
+ * stores to the blocks between it and the moved data fault.
+ */
+ if (i_size_read(dst) > dst_osize)
+ pagecache_isize_extended(dst, dst_osize, pos_out);
+out_inval:
+ filemap_invalidate_unlock_two(src->i_mapping, dst->i_mapping);
+
if (src != dst)
f2fs_up_write(&F2FS_I(dst)->i_gc_rwsem[WRITE]);
out_src:
@@ -3603,9 +3649,6 @@ static int f2fs_ioc_move_range(struct file *filp, unsigned long arg)
{
struct f2fs_move_range range;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(file_inode(filp))))
- return -EOPNOTSUPP;
-
if (copy_from_user(&range, (struct f2fs_move_range __user *)arg,
sizeof(range)))
return -EFAULT;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 12/20] f2fs: support subpage blocks in defragmentation
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (10 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 11/20] f2fs: support subpage blocks in move range Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
` (7 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Mark only the target block, not its whole page, dirty when redirtying
data for defragmentation. Remove the -EOPNOTSUPP check.
Read a multi-block folio once through ->read_folio() instead of each
block synchronously, and redirty the folio that was read after locking
it. This also applies to large folios when the block size equals
PAGE_SIZE.
Read compressed files through ->read_folio() as well, with any block
size. f2fs_get_lock_data_folio() reads the mapped block as it is, so a
block of a compressed cluster that was not in the page cache was read
without decompression, and defragmentation wrote the compressed data
back as file data.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 44 ++++++++++++++++++++++++++++++++++++++------
1 file changed, 38 insertions(+), 6 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 4499a461be86..26dd06961389 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -3202,6 +3202,41 @@ static int f2fs_ioc_write_checkpoint(struct file *filp)
return ret;
}
+/*
+ * Return the locked folio of a block to redirty it for defragmentation.
+ *
+ * A compressed file is read through ->read_folio(), which decompresses the
+ * cluster: f2fs_get_lock_data_folio() would read the mapped block as it is
+ * and leave compressed data in the page cache. A multi-block folio is read
+ * the same way, in one I/O instead of one synchronous read per block.
+ */
+static struct folio *f2fs_get_defrag_folio(struct file *filp, pgoff_t bidx)
+{
+ struct inode *inode = file_inode(filp);
+ struct address_space *mapping = inode->i_mapping;
+ pgoff_t index = f2fs_lblk_to_folio_index(F2FS_I_SB(inode), bidx);
+ struct folio *folio;
+
+ if (!f2fs_compressed_file(inode) && !f2fs_has_multiblock_folios(inode))
+ return f2fs_get_lock_data_folio(inode, bidx, true);
+
+repeat:
+ folio = read_mapping_folio(mapping, index, filp);
+ if (IS_ERR(folio))
+ return folio;
+
+ folio_lock(folio);
+ if (unlikely(folio->mapping != mapping)) {
+ f2fs_folio_put(folio, true);
+ goto repeat;
+ }
+ if (unlikely(!folio_test_uptodate(folio))) {
+ f2fs_folio_put(folio, true);
+ return ERR_PTR(-EIO);
+ }
+ return folio;
+}
+
static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
struct file *filp,
struct f2fs_defragment *range)
@@ -3327,7 +3362,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
cnt < BLKS_PER_SEG(sbi)) {
struct folio *folio;
- folio = f2fs_get_lock_data_folio(inode, idx, true);
+ folio = f2fs_get_defrag_folio(filp, idx);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
goto clear_out;
@@ -3336,8 +3371,8 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
f2fs_folio_wait_writeback(folio, true, true);
f2fs_ffs_mark_subrange_dirty(folio,
- offset_in_folio(folio, (loff_t)idx << PAGE_SHIFT),
- PAGE_SIZE);
+ f2fs_folio_byte_offset(sbi, folio, idx),
+ F2FS_BLKSIZE(sbi));
f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
f2fs_folio_put(folio, true);
@@ -3378,9 +3413,6 @@ static int f2fs_ioc_defragment(struct file *filp, unsigned long arg)
struct f2fs_defragment range;
int err;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 13/20] f2fs: support subpage blocks in swapfile activation
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (11 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
` (6 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The swap subsystem works in PAGE_SIZE units. Ignore a partial tail
page, require PAGE_SIZE aligned extents, and convert block counts and
addresses to pages for swap. When migrating blocks, mark only the
migrated block, not its whole page, dirty in the folio state.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 43 ++++++++++++++++++++++++++++---------------
1 file changed, 28 insertions(+), 15 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 98ceea49e426..f23f29d45763 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -6007,8 +6007,8 @@ static int f2fs_migrate_blocks(struct inode *inode, block_t start_blk,
}
f2fs_ffs_mark_subrange_dirty(folio,
- offset_in_folio(folio, (loff_t)blkidx << PAGE_SHIFT),
- PAGE_SIZE);
+ f2fs_folio_byte_offset(sbi, folio, blkidx),
+ F2FS_BLKSIZE(sbi));
f2fs_folio_mark_dirty(folio);
f2fs_folio_put(folio, true);
}
@@ -6040,8 +6040,10 @@ static int check_swap_activate(struct swap_info_struct *sis,
struct address_space *mapping = swap_file->f_mapping;
struct inode *inode = mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ unsigned int blks_per_page = F2FS_BLKS_PER_PAGE(sbi);
block_t cur_lblock;
block_t last_lblock;
+ block_t max_lblock;
block_t pblock;
block_t lowest_pblock = -1;
block_t highest_pblock = 0;
@@ -6056,9 +6058,12 @@ static int check_swap_activate(struct swap_info_struct *sis,
* to be very smart.
*/
cur_lblock = 0;
- last_lblock = F2FS_BYTES_TO_BLK(sbi, i_size_read(inode));
+ /* swap uses whole pages only; ignore a partial tail page */
+ last_lblock = round_down(F2FS_BYTES_TO_BLK(sbi, i_size_read(inode)),
+ blks_per_page);
+ max_lblock = min_t(u64, last_lblock, (u64)sis->max * blks_per_page);
- while (cur_lblock < last_lblock && cur_lblock < sis->max) {
+ while (cur_lblock < max_lblock) {
struct f2fs_map_blocks map;
bool last_extent = false;
retry:
@@ -6093,7 +6098,7 @@ static int check_swap_activate(struct swap_info_struct *sis,
not_aligned++;
nr_pblocks = roundup(nr_pblocks, blks_per_sec);
- if (cur_lblock + nr_pblocks > sis->max)
+ if (cur_lblock + nr_pblocks > max_lblock)
nr_pblocks -= blks_per_sec;
/* this extent is last one */
@@ -6114,20 +6119,28 @@ static int check_swap_activate(struct swap_info_struct *sis,
goto retry;
}
- if (cur_lblock + nr_pblocks >= sis->max)
- nr_pblocks = sis->max - cur_lblock;
+ if (cur_lblock + nr_pblocks >= max_lblock)
+ nr_pblocks = max_lblock - cur_lblock;
+
+ if (pblock % blks_per_page || nr_pblocks % blks_per_page) {
+ f2fs_err(sbi, "Swapfile extent is not aligned to PAGE_SIZE");
+ ret = -EINVAL;
+ goto out;
+ }
if (cur_lblock) { /* exclude the header page */
- if (pblock < lowest_pblock)
- lowest_pblock = pblock;
- if (pblock + nr_pblocks - 1 > highest_pblock)
- highest_pblock = pblock + nr_pblocks - 1;
+ if (pblock / blks_per_page < lowest_pblock)
+ lowest_pblock = pblock / blks_per_page;
+ if ((pblock + nr_pblocks) / blks_per_page - 1 > highest_pblock)
+ highest_pblock = (pblock + nr_pblocks) / blks_per_page - 1;
}
/*
* We found a PAGE_SIZE-length, PAGE_SIZE-aligned run of blocks
*/
- ret = add_swap_extent(sis, cur_lblock, nr_pblocks, pblock);
+ ret = add_swap_extent(sis, cur_lblock / blks_per_page,
+ nr_pblocks / blks_per_page,
+ pblock / blks_per_page);
if (ret < 0)
goto out;
nr_extents += ret;
@@ -6136,9 +6149,9 @@ static int check_swap_activate(struct swap_info_struct *sis,
ret = nr_extents;
*span = 1 + highest_pblock - lowest_pblock;
if (cur_lblock == 0)
- cur_lblock = 1; /* force Empty message */
- sis->max = cur_lblock;
- sis->pages = cur_lblock - 1;
+ cur_lblock = blks_per_page; /* force Empty message */
+ sis->max = cur_lblock / blks_per_page;
+ sis->pages = sis->max - 1;
out:
if (not_aligned)
f2fs_warn(sbi, "Swapfile (%u) is not align to section: 1) creat(), 2) ioctl(F2FS_IOC_SET_PIN_FILE), 3) fallocate(%lu * N)",
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 14/20] f2fs: support subpage blocks in fs-verity enablement
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (12 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
` (5 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Check verity per block. While enabling verity, dirty and write back
only the data blocks below i_size in the folios before the verity
metadata. Do not zero the blocks past i_size on read while enabling
verity, since GC can read a folio of the verity metadata that is already
written. Remove the -EOPNOTSUPP checks. The changes for enabling
verity also apply to large folios when the block size equals PAGE_SIZE.
Stop the data reads of a verity file at i_size in
f2fs_read_data_large_folio(), so that the blocks past EOF in the folio
that spans EOF are zeroed instead of read. A pinned file can have such
blocks allocated without being zeroed, and fs-verity rejects non-zero
data past EOF. The reads of the verity metadata still go past i_size.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 29 +++++++++++++++++++++--------
fs/f2fs/f2fs.h | 5 +++++
fs/f2fs/file.c | 3 ---
fs/f2fs/verity.c | 8 --------
4 files changed, 26 insertions(+), 19 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index f23f29d45763..9f7f8214d10e 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1671,7 +1671,7 @@ int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index)
static inline struct fsverity_info *f2fs_need_verity(const struct inode *inode,
pgoff_t idx)
{
- if (idx < DIV_ROUND_UP(inode->i_size, PAGE_SIZE))
+ if (idx < F2FS_BLK_ALIGN(F2FS_SB(inode->i_sb), inode->i_size))
return fsverity_get_info(inode);
return NULL;
}
@@ -2619,7 +2619,12 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
static inline loff_t f2fs_readpage_limit(struct inode *inode)
{
- if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode))
+ /*
+ * The verity metadata is past i_size. GC can read it while verity is
+ * being enabled, so do not zero it.
+ */
+ if (IS_ENABLED(CONFIG_FS_VERITY) &&
+ (IS_VERITY(inode) || f2fs_verity_in_progress(inode)))
return F2FS_BLK_TO_BYTES(F2FS_I_SB(inode),
max_file_blocks(F2FS_I_SB(inode), inode));
@@ -2913,10 +2918,10 @@ static unsigned int f2fs_folio_valid_blocks(struct inode *inode,
pgoff_t last_lblk = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
- if (f2fs_verity_in_progress(inode))
- return nr_blocks;
if (first_lblk < last_lblk)
return min_t(pgoff_t, nr_blocks, last_lblk - first_lblk);
+ if (f2fs_verity_in_progress(inode))
+ return nr_blocks;
return 0;
}
@@ -3409,7 +3414,14 @@ static int f2fs_read_data_large_folio(struct inode *inode,
index = f2fs_folio_lblk(sbi, folio);
offset = 0;
nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- last_block_in_file = F2FS_BLK_ALIGN(sbi, f2fs_readpage_limit(inode));
+ /*
+ * A data read of a verity file, which has vi, stops at i_size. A
+ * pinned file can have blocks past EOF that were allocated without
+ * being zeroed, and fs-verity rejects non-zero data past EOF. Only
+ * the reads of the verity metadata, which have no vi, go past i_size.
+ */
+ last_block_in_file = F2FS_BLK_ALIGN(sbi, vi ? i_size_read(inode) :
+ f2fs_readpage_limit(inode));
for (; nr_blocks;
nr_blocks -= len_blks, max_nr_blocks -= len_blks,
@@ -3719,7 +3731,7 @@ static int f2fs_read_data_folio(struct file *file, struct folio *folio)
return ret;
}
- vi = f2fs_need_verity(inode, folio->index);
+ vi = f2fs_need_verity(inode, f2fs_folio_lblk(F2FS_I_SB(inode), folio));
if (vi)
fsverity_readahead(vi, folio->index, folio_nr_pages(folio));
return f2fs_mpage_readpages(inode, vi, NULL, folio);
@@ -3739,7 +3751,8 @@ static void f2fs_readahead(struct readahead_control *rac)
if (f2fs_has_inline_data(inode))
return;
- vi = f2fs_need_verity(inode, readahead_index(rac));
+ vi = f2fs_need_verity(inode, F2FS_BYTES_TO_BLK(F2FS_I_SB(inode),
+ readahead_pos(rac)));
if (vi)
fsverity_readahead(vi, readahead_index(rac),
readahead_count(rac));
@@ -4777,7 +4790,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
op_locked = true;
}
- if (!verity_in_progress) {
+ if (!verity_in_progress || end_pos <= f2fs_verity_metadata_pos(inode)) {
poff = offset_in_folio(folio, isize);
end_index = isize >> PAGE_SHIFT;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index f588603dd1d0..f84d6f4cd1eb 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -5349,6 +5349,11 @@ static inline bool f2fs_allow_multi_device_dio(struct f2fs_sb_info *sbi,
return sbi->aligned_blksize;
}
+static inline loff_t f2fs_verity_metadata_pos(const struct inode *inode)
+{
+ return round_up(i_size_read(inode), 65536);
+}
+
#ifdef CONFIG_F2FS_FAULT_INJECTION
extern int f2fs_build_fault_attr(struct f2fs_sb_info *sbi, unsigned long rate,
unsigned long type, enum fault_option fo);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 26dd06961389..8540613e3b22 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -4331,9 +4331,6 @@ static int f2fs_ioc_enable_verity(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
- return -EOPNOTSUPP;
-
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (!f2fs_sb_has_verity(F2FS_I_SB(inode))) {
diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
index af1b7f21c1d8..5f579d0b917b 100644
--- a/fs/f2fs/verity.c
+++ b/fs/f2fs/verity.c
@@ -32,11 +32,6 @@
#define F2FS_VERIFY_VER (1)
-static inline loff_t f2fs_verity_metadata_pos(const struct inode *inode)
-{
- return round_up(inode->i_size, 65536);
-}
-
/*
* Write some verity metadata to the inode for FS_IOC_ENABLE_VERITY.
* kernel_write() can't be used because the file descriptor is readonly.
@@ -94,9 +89,6 @@ static int f2fs_begin_enable_verity(struct file *filp)
struct inode *inode = file_inode(filp);
int err;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
- return -EOPNOTSUPP;
-
if (f2fs_verity_in_progress(inode))
return -EBUSY;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 15/20] f2fs: support subpage blocks in atomic writes
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (13 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
` (4 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Writeback already writes dirty blocks to the COW inode per block. Make
write_begin of atomic files reserve and read COW blocks per block as
well, and remove the -EOPNOTSUPP checks. Skipping the read of the
blocks that a write fully covers also applies to large folios when the
block size equals PAGE_SIZE.
Aborting atomic writes drops the page cache and restores i_size without
i_gc_rwsem[WRITE] and invalidate_lock. With subpage blocks, a page fault
in between can make a folio writable with only the blocks within the
smaller i_size prepared, and a read can zero-fill the blocks beyond it
and mark them uptodate, while they are inside the restored i_size. GC
reads through the page cache as well once FI_ATOMIC_FILE is cleared, and
then moves the zeroes. Hold both locks across it as
f2fs_ioc_start_atomic_write() does, except at eviction, where no one
else uses the inode.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 24 ++++++++++++++++--------
fs/f2fs/file.c | 30 ++++++++++++++++++++++--------
fs/f2fs/inode.c | 1 +
fs/f2fs/segment.c | 7 +++++++
4 files changed, 46 insertions(+), 16 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 9f7f8214d10e..b4f16afccc51 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4698,7 +4698,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ f2fs_compressed_file(inode)) {
WARN_ON_ONCE(1);
mapping_set_error(mapping, -EOPNOTSUPP);
folio_clear_dirty_for_io(folio);
@@ -5325,7 +5325,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
int err = 0;
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode)))
+ f2fs_compressed_file(inode))
return -EOPNOTSUPP;
/*
@@ -5441,13 +5441,16 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
/* Inline data must have been converted before reaching here. */
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
- start = folio->index + (ori_off >> PAGE_SHIFT);
- end = folio->index + ((ori_off + len - 1) >> PAGE_SHIFT);
+ start = f2fs_folio_lblk(sbi, folio) + F2FS_BYTES_TO_BLK(sbi, ori_off);
+ end = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, ori_off + len - 1);
for (index = start; index <= end; index++) {
+ loff_t block_start = F2FS_BLK_TO_BYTES(sbi, index);
+ loff_t block_end = block_start + F2FS_BLKSIZE(sbi);
block_t blkaddr = NULL_ADDR;
bool node_changed = false;
- size_t off = (index - folio->index) << PAGE_SHIFT;
+ size_t off = f2fs_folio_byte_offset(sbi, folio, index);
err = prepare_atomic_write_begin(sbi, inode, index,
&blkaddr, &node_changed,
@@ -5455,12 +5458,17 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
if (err)
return err;
+ /* Skip read if the write covers the entire block. */
+ if (pos <= block_start && pos + len >= block_end)
+ goto balance;
+
if (f2fs_ffs_test_blk_uptodate(folio, index))
goto balance;
if (blkaddr == NEW_ADDR) {
- folio_zero_segment(folio, off, off + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, off, PAGE_SIZE);
+ folio_zero_segment(folio, off, off + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, off,
+ F2FS_BLKSIZE(sbi));
goto balance;
}
@@ -5517,7 +5525,7 @@ static int f2fs_write_begin(const struct kiocb *iocb,
}
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ f2fs_compressed_file(inode)) {
err = -EOPNOTSUPP;
goto fail;
}
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 8540613e3b22..d6304b705e06 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2314,7 +2314,7 @@ static long f2fs_fallocate(struct file *file, int mode,
}
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if (f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
+ if (f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
@@ -2358,6 +2358,23 @@ static long f2fs_fallocate(struct file *file, int mode,
return ret;
}
+/*
+ * Abort atomic writes with inode_lock held. Until i_size is restored, GC,
+ * page faults and reads would see the i_size of the aborted writes, so keep
+ * them away as f2fs_ioc_start_atomic_write() does.
+ */
+static void f2fs_abort_atomic_write_locked(struct inode *inode, bool clean)
+{
+ if (!f2fs_is_atomic_file(inode))
+ return;
+
+ f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
+ filemap_invalidate_lock(inode->i_mapping);
+ f2fs_abort_atomic_write(inode, clean);
+ filemap_invalidate_unlock(inode->i_mapping);
+ f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
+}
+
static int f2fs_release_file(struct inode *inode, struct file *filp)
{
if (atomic_dec_and_test(&F2FS_I(inode)->open_count))
@@ -2372,7 +2389,7 @@ static int f2fs_release_file(struct inode *inode, struct file *filp)
return 0;
inode_lock(inode);
- f2fs_abort_atomic_write(inode, true);
+ f2fs_abort_atomic_write_locked(inode, true);
inode_unlock(inode);
return 0;
@@ -2391,7 +2408,7 @@ static int f2fs_file_flush(struct file *file, fl_owner_t id)
if (F2FS_I(inode)->atomic_write_task == current &&
(current->flags & PF_EXITING)) {
inode_lock(inode);
- f2fs_abort_atomic_write(inode, true);
+ f2fs_abort_atomic_write_locked(inode, true);
inode_unlock(inode);
}
@@ -2568,9 +2585,6 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
loff_t isize;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!(filp->f_mode & FMODE_WRITE))
return -EBADF;
@@ -2702,7 +2716,7 @@ static int f2fs_ioc_commit_atomic_write(struct file *filp)
if (!ret)
ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 0, true);
- f2fs_abort_atomic_write(inode, ret);
+ f2fs_abort_atomic_write_locked(inode, ret);
} else {
ret = f2fs_do_sync_file(filp, 0, LLONG_MAX, 1, false);
}
@@ -2730,7 +2744,7 @@ static int f2fs_ioc_abort_atomic_write(struct file *filp)
inode_lock(inode);
- f2fs_abort_atomic_write(inode, true);
+ f2fs_abort_atomic_write_locked(inode, true);
inode_unlock(inode);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index 5c6a64b31b05..2bc15fa192be 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -851,6 +851,7 @@ static void f2fs_pre_evict_inode(struct inode *inode)
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode_info *fi = F2FS_I(inode);
+ /* No one else uses the inode now, so the locks are not needed. */
f2fs_abort_atomic_write(inode, true);
if (fi->cow_inode && f2fs_is_cow_file(fi->cow_inode)) {
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 93f165151c0f..5bff57723e2d 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -186,6 +186,13 @@ bool f2fs_need_SSR(struct f2fs_sb_info *sbi)
SM_I(sbi)->min_ssr_sections + reserved_sections(sbi));
}
+/*
+ * The callers hold i_gc_rwsem[WRITE] and invalidate_lock as
+ * f2fs_ioc_start_atomic_write() does, so that no page is read, by GC either,
+ * or made writable between dropping the page cache and restoring i_size, i.e.
+ * with the i_size of the aborted writes. Only the eviction does not, as no one
+ * else uses the inode then.
+ */
void f2fs_abort_atomic_write(struct inode *inode, bool clean)
{
struct f2fs_inode_info *fi = F2FS_I(inode);
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 16/20] f2fs: preserve inline data in subpage block mode
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (14 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
` (3 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
A write to an inline inode converted it to a regular inode whenever the
folio held multiple blocks, allocating a block even for small files.
Convert only when the write goes beyond MAX_INLINE_DATA; otherwise read
the inline data into the folio and write it back to the inode block in
f2fs_write_cache_folios(). This also applies to an inline inode that
uses large folios when the block size equals PAGE_SIZE: its writeback
now goes through f2fs_write_cache_folios() instead of
f2fs_write_cache_pages().
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 54 +++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 45 insertions(+), 9 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index b4f16afccc51..7dd6764a9363 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4804,6 +4804,16 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
}
+ if (f2fs_has_inline_data(inode)) {
+ err = f2fs_write_inline_data(inode, folio);
+ if (!err) {
+ f2fs_folio_clear_dirty_blocks(folio, true);
+ goto retry_out;
+ }
+ folio_redirty_for_writepage(wbc, folio);
+ goto retry_out;
+ }
+
folio_start_writeback(folio);
/*
@@ -4992,8 +5002,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (f2fs_has_multiblock_folios(inode) &&
- !f2fs_has_inline_data(inode))
+ if (f2fs_has_multiblock_folios(inode))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
ret = f2fs_write_cache_pages(mapping, wbc, io_type);
@@ -5060,6 +5069,16 @@ void f2fs_write_failed(struct inode *inode, loff_t to)
}
}
+/* Fill @folio from the inline data for a write within the inline area */
+static void prepare_inline_write_begin(struct inode *inode,
+ struct folio *folio, struct f2fs_cached_block *ientry)
+{
+ f2fs_do_read_inline_data(folio, ientry);
+ set_inode_flag(inode, FI_DATA_EXIST);
+ if (inode->i_nlink)
+ f2fs_cache_set_inline(ientry);
+}
+
static int prepare_write_begin(struct f2fs_sb_info *sbi,
struct folio *folio, loff_t pos, unsigned int len,
block_t *blk_addr, bool *node_changed)
@@ -5113,10 +5132,7 @@ static int prepare_write_begin(struct f2fs_sb_info *sbi,
if (f2fs_has_inline_data(inode)) {
if (pos + len <= MAX_INLINE_DATA(inode)) {
- f2fs_do_read_inline_data(folio, ientry);
- set_inode_flag(inode, FI_DATA_EXIST);
- if (inode->i_nlink)
- f2fs_cache_set_inline(ientry);
+ prepare_inline_write_begin(inode, folio, ientry);
goto out;
}
err = f2fs_convert_inline_folio(&dn, folio);
@@ -5535,7 +5551,8 @@ static int f2fs_write_begin(const struct kiocb *iocb,
* and #0 page. The locking rule for inline_data conversion should be:
* folio_lock(folio #0) -> folio_lock(inode_page)
*/
- if (index != 0 || f2fs_has_subpage_blocks(sbi)) {
+ if (index != 0 || (f2fs_has_subpage_blocks(sbi) &&
+ pos + len > MAX_INLINE_DATA(inode))) {
err = f2fs_convert_inline_inode(inode);
if (err)
goto fail;
@@ -5592,6 +5609,23 @@ static int f2fs_write_begin(const struct kiocb *iocb,
blen = min_t(unsigned int, len,
folio_size(folio) - offset_in_folio(folio, pos));
+ if (f2fs_has_inline_data(inode) && f2fs_is_multiblock_folio(sbi, folio)) {
+ struct f2fs_lock_context lc;
+ struct f2fs_cached_block *ientry;
+
+ f2fs_map_lock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ ientry = f2fs_get_inode_cache(sbi, inode->i_ino);
+ if (IS_ERR(ientry)) {
+ f2fs_map_unlock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ err = PTR_ERR(ientry);
+ goto put_folio;
+ }
+ prepare_inline_write_begin(inode, folio, ientry);
+ f2fs_put_cache(ientry, true);
+ f2fs_map_unlock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ return 0;
+ }
+
/*
* Flush the cached IO of this folio before touching it, especially
* for the large folio prepare below which reads and updates subpage
@@ -5700,7 +5734,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!folio_test_uptodate(folio)) {
if (unlikely(copied != len)) {
copied = 0;
- } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
+ } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio) &&
+ !f2fs_has_inline_data(inode)) {
f2fs_ffs_mark_subrange_uptodate(folio,
offset_in_folio(folio, pos), len);
} else {
@@ -5727,7 +5762,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!copied)
goto unlock_out;
- if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio))
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio) &&
+ !f2fs_has_inline_data(inode))
f2fs_ffs_mark_subrange_dirty(folio, offset_in_folio(folio, pos),
copied);
f2fs_folio_mark_dirty(folio);
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 17/20] f2fs: support subpage blocks in online resize
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (15 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 18/20] f2fs: support subpage blocks in compression Daeho Jeong
` (2 subsequent siblings)
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Resize works in blocks, segments and sections, independent of
PAGE_SIZE, so remove the -EOPNOTSUPP check.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index d6304b705e06..64c91827a5f4 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -4325,9 +4325,6 @@ static int f2fs_ioc_resize_fs(struct file *filp, unsigned long arg)
struct f2fs_sb_info *sbi = F2FS_I_SB(file_inode(filp));
__u64 block_count;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 18/20] f2fs: support subpage blocks in compression
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (16 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 20/20] f2fs: enable subpage block size support Daeho Jeong
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a cluster spans fewer folios than blocks, and
several compressed blocks share a bounce folio.
- Require a cluster to cover at least one page. This raises the minimum
compress_log_size only when a page has more than four blocks, e.g. to
64KB for 4KB blocks on a 64KB page kernel, where the mount option and
F2FS_IOC_SET_COMPRESS_OPTION reject a smaller one. A file that already
has a smaller cluster is handled like one with an unsupported
compression algorithm: it can be looked up and removed, but opening it
fails with -EOPNOTSUPP.
- Use block offsets and sizes for compressed data, keep each compressed
block at its block offset within the cluster, and count pending
compressed I/O in blocks so that a shared bounce folio is not
decompressed or freed early.
- Submit only the written block of a shared bounce folio in
f2fs_submit_page_write(), at its offset in the folio.
- Keep compressed files on the page based read and write paths, and
compress a cluster only when all of its blocks are dirty and below
EOF.
- A folio with per-block state may hold uptodate blocks newer than the
on-disk cluster, so decompress into a bounce page and copy back only
the blocks that are not uptodate.
- Cache compressed blocks one block at a time in the compress cache.
- Lock only the unlocked pages in cancel_cluster_writeback() to avoid a
self-deadlock.
- Attach a folio state to the multi-block folios of a rewritten cluster
before they are dirtied, as the other write paths do.
- Attach a folio state to multi-block folios that have none when
compressed writeback starts, as the raw write path does. Otherwise the
write completion, which does not hold the folio lock, can detach the
page private flags while a task is moving them into a new folio state.
- Take sbi from the dic to count the blocks of a compressed page in
f2fs_finish_read_bio().
- Write the blocks of a multi-block folio in
f2fs_write_single_data_page() as f2fs_write_cache_folios() does,
taking the write_blocks_pending bias with a new helper shared by the
two and dropping it with f2fs_ffs_put_write_pending().
- Count a read folio of a compressed file once in the iostat:
f2fs_mpage_readpages() counts the folios that go through its loop,
and f2fs_read_data_large_folio() counts the others.
- Remove the -EOPNOTSUPP checks.
- Remove inode_inc_dirty_block() and inode_dec_dirty_block(), which
have no caller left.
The bounce folios are now freed when the last block of a cluster
completes, also when the block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
Documentation/filesystems/f2fs.rst | 7 +-
fs/f2fs/compress.c | 567 ++++++++++++++++++++---------
fs/f2fs/data.c | 293 +++++++++------
fs/f2fs/f2fs.h | 71 ++--
fs/f2fs/file.c | 55 +--
fs/f2fs/super.c | 7 +-
6 files changed, 657 insertions(+), 343 deletions(-)
diff --git a/Documentation/filesystems/f2fs.rst b/Documentation/filesystems/f2fs.rst
index b79bb67ee78e..4d827c8a98d8 100644
--- a/Documentation/filesystems/f2fs.rst
+++ b/Documentation/filesystems/f2fs.rst
@@ -319,7 +319,12 @@ compress_algorithm=%s:%d Control compress algorithm and its compress level, now,
========= ===========
compress_log_size=%u Support configuring compress cluster size. The size will
- be 4KB * (1 << %u). The default and minimum sizes are 16KB.
+ be block size * (1 << %u). The default and minimum sizes
+ are four blocks, i.e. 16KB with 4KB blocks. A cluster has
+ to cover at least one page, so they are one page if a page
+ has more than four blocks, e.g. 64KB with 4KB blocks on a
+ 64KB page kernel, where a file with a smaller cluster
+ cannot be opened (-EOPNOTSUPP).
compress_extension=%s Support adding specified extension, so that f2fs can enable
compression on those corresponding files, e.g. if all files
with '.ext' has high compression rate, we can set the '.ext'
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index c63622b6c49a..d0e989254ffe 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -57,14 +57,15 @@ struct f2fs_compress_ops {
bool (*is_level_valid)(int level);
};
-static unsigned int offset_in_cluster(struct compress_ctx *cc, pgoff_t index)
+static unsigned int offset_in_cluster(struct compress_ctx *cc, pgoff_t lblk)
{
- return index & (cc->cluster_size - 1);
+ return (lblk & (cc->cluster_size - 1)) >>
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(cc->inode));
}
-static pgoff_t cluster_idx(struct compress_ctx *cc, pgoff_t index)
+static pgoff_t cluster_idx(struct compress_ctx *cc, pgoff_t lblk)
{
- return index >> cc->log_cluster_size;
+ return lblk >> cc->log_cluster_size;
}
static pgoff_t start_idx_of_cluster(struct compress_ctx *cc)
@@ -114,7 +115,9 @@ static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock)
static void f2fs_put_rpages(struct compress_ctx *cc)
{
- f2fs_drop_rpages(cc, cc->cluster_size, false);
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+
+ f2fs_drop_rpages(cc, cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi), false);
}
static void f2fs_unlock_rpages(struct compress_ctx *cc, int len)
@@ -125,9 +128,11 @@ static void f2fs_unlock_rpages(struct compress_ctx *cc, int len)
static void f2fs_put_rpages_wbc(struct compress_ctx *cc,
struct writeback_control *wbc, bool redirty, bool unlock)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
unsigned int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
if (!cc->rpages[i])
continue;
if (redirty)
@@ -158,22 +163,24 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc, bool reuse)
cc->rpages = NULL;
cc->nr_rpages = 0;
cc->nr_cpages = 0;
- cc->valid_nr_cpages = 0;
+ cc->valid_nr_cblocks = 0;
if (!reuse)
cc->cluster_idx = NULL_CLUSTER;
}
void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct folio *folio)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
unsigned int cluster_ofs;
+ pgoff_t lblk = f2fs_folio_lblk(sbi, folio);
- if (!f2fs_cluster_can_merge_page(cc, folio->index))
- f2fs_bug_on(F2FS_I_SB(cc->inode), 1);
+ if (!f2fs_cluster_can_merge_block(cc, lblk))
+ f2fs_bug_on(sbi, 1);
- cluster_ofs = offset_in_cluster(cc, folio->index);
+ cluster_ofs = offset_in_cluster(cc, lblk);
cc->rpages[cluster_ofs] = folio_page(folio, 0);
cc->nr_rpages++;
- cc->cluster_idx = cluster_idx(cc, folio->index);
+ cc->cluster_idx = cluster_idx(cc, lblk);
}
#ifdef CONFIG_F2FS_FS_LZO
@@ -184,7 +191,7 @@ static int lzo_init_compress_ctx(struct compress_ctx *cc)
if (!cc->private)
return -ENOMEM;
- cc->clen = lzo1x_worst_compress(PAGE_SIZE << cc->log_cluster_size);
+ cc->clen = lzo1x_worst_compress(F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) << cc->log_cluster_size);
return 0;
}
@@ -220,10 +227,11 @@ static int lzo_decompress_pages(struct decompress_io_ctx *dic)
return -EIO;
}
- if (dic->rlen != PAGE_SIZE << dic->log_cluster_size) {
+ if (dic->rlen != F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size) {
f2fs_err_ratelimited(dic->sbi,
"lzo invalid rlen:%zu, expected:%lu",
- dic->rlen, PAGE_SIZE << dic->log_cluster_size);
+ dic->rlen, (unsigned long)F2FS_BLKSIZE(dic->sbi) <<
+ dic->log_cluster_size);
return -EIO;
}
return 0;
@@ -256,7 +264,7 @@ static int lz4_init_compress_ctx(struct compress_ctx *cc)
* adapt worst compress case, because lz4 compressor can handle
* output budget properly.
*/
- cc->clen = cc->rlen - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ cc->clen = cc->rlen - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
return 0;
}
@@ -300,10 +308,11 @@ static int lz4_decompress_pages(struct decompress_io_ctx *dic)
return -EIO;
}
- if (ret != PAGE_SIZE << dic->log_cluster_size) {
+ if (ret != F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size) {
f2fs_err_ratelimited(dic->sbi,
"lz4 invalid ret:%d, expected:%lu",
- ret, PAGE_SIZE << dic->log_cluster_size);
+ ret, (unsigned long)F2FS_BLKSIZE(dic->sbi) <<
+ dic->log_cluster_size);
return -EIO;
}
return 0;
@@ -358,7 +367,7 @@ static int zstd_init_compress_ctx(struct compress_ctx *cc)
cc->private = workspace;
cc->private2 = stream;
- cc->clen = cc->rlen - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ cc->clen = cc->rlen - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
return 0;
}
@@ -375,7 +384,7 @@ static int zstd_compress_pages(struct compress_ctx *cc)
zstd_in_buffer inbuf;
zstd_out_buffer outbuf;
int src_size = cc->rlen;
- int dst_size = src_size - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ int dst_size = src_size - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
int ret;
inbuf.pos = 0;
@@ -419,7 +428,7 @@ static int zstd_init_decompress_ctx(struct decompress_io_ctx *dic)
void *workspace;
unsigned int workspace_size;
unsigned int max_window_size =
- MAX_COMPRESS_WINDOW_SIZE(dic->log_cluster_size);
+ MAX_COMPRESS_WINDOW_SIZE(dic->sbi, dic->log_cluster_size);
workspace_size = zstd_dstream_workspace_bound(max_window_size);
@@ -475,7 +484,7 @@ static int zstd_decompress_pages(struct decompress_io_ctx *dic)
f2fs_err_ratelimited(dic->sbi,
"%s ZSTD invalid rlen:%zu, expected:%lu",
__func__, dic->rlen,
- PAGE_SIZE << dic->log_cluster_size);
+ (unsigned long)F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size);
return -EIO;
}
@@ -550,7 +559,9 @@ bool f2fs_is_compress_backend_ready(struct inode *inode)
{
if (!f2fs_compressed_file(inode))
return true;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ /* A cluster smaller than a page is not supported */
+ if (F2FS_I(inode)->i_log_cluster_size <
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(inode)))
return false;
return f2fs_cops[F2FS_I(inode)->i_compress_algorithm];
}
@@ -621,13 +632,23 @@ static void *f2fs_vmap(struct page **pages, unsigned int count)
return buf;
}
+static void f2fs_vunmap(void *vaddr, unsigned int count)
+{
+ if (!vaddr)
+ return;
+ vm_unmap_ram((void *)((unsigned long)vaddr & PAGE_MASK), count);
+}
+
static int f2fs_compress_pages(struct compress_ctx *cc)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
struct f2fs_inode_info *fi = F2FS_I(cc->inode);
const struct f2fs_compress_ops *cops =
f2fs_cops[fi->i_compress_algorithm];
- unsigned int max_len, new_nr_cpages;
+ unsigned int max_len, new_cblocks;
+ unsigned int nr_rfolios, needed_folios;
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(sbi) ? 1 : 0;
+ void *cbuf_vaddr;
u32 chksum = 0;
int i, ret;
@@ -640,9 +661,12 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
goto out;
}
+ nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+
max_len = COMPRESS_HEADER_SIZE + cc->clen;
- cc->nr_cpages = DIV_ROUND_UP(max_len, PAGE_SIZE);
- cc->valid_nr_cpages = cc->nr_cpages;
+ cc->valid_nr_cblocks = F2FS_BLK_ALIGN(sbi, max_len);
+ cc->nr_cpages = DIV_ROUND_UP(cbuf_blk_ofs + cc->valid_nr_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
cc->cpages = page_array_alloc(sbi, cc->nr_cpages);
if (!cc->cpages) {
@@ -653,23 +677,24 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
for (i = 0; i < cc->nr_cpages; i++)
cc->cpages[i] = f2fs_compress_alloc_page();
- cc->rbuf = f2fs_vmap(cc->rpages, cc->cluster_size);
+ cc->rbuf = f2fs_vmap(cc->rpages, nr_rfolios);
if (!cc->rbuf) {
ret = -ENOMEM;
goto out_free_cpages;
}
- cc->cbuf = f2fs_vmap(cc->cpages, cc->nr_cpages);
- if (!cc->cbuf) {
+ cbuf_vaddr = f2fs_vmap(cc->cpages, cc->nr_cpages);
+ if (!cbuf_vaddr) {
ret = -ENOMEM;
goto out_vunmap_rbuf;
}
+ cc->cbuf = cbuf_vaddr + F2FS_BLK_TO_BYTES(sbi, cbuf_blk_ofs);
ret = cops->compress_pages(cc);
if (ret)
goto out_vunmap_cbuf;
- max_len = PAGE_SIZE * (cc->cluster_size - 1) - COMPRESS_HEADER_SIZE;
+ max_len = F2FS_BLK_TO_BYTES(sbi, cc->cluster_size - 1) - COMPRESS_HEADER_SIZE;
if (cc->clen > max_len) {
ret = -EAGAIN;
@@ -685,17 +710,19 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
for (i = 0; i < COMPRESS_DATA_RESERVED_SIZE; i++)
cc->cbuf->reserved[i] = cpu_to_le32(0);
- new_nr_cpages = DIV_ROUND_UP(cc->clen + COMPRESS_HEADER_SIZE, PAGE_SIZE);
+ new_cblocks = F2FS_BLK_ALIGN(sbi, cc->clen + COMPRESS_HEADER_SIZE);
- /* zero out any unused part of the last page */
+ /* zero out any unused part of the last block */
memset(&cc->cbuf->cdata[cc->clen], 0,
- (new_nr_cpages * PAGE_SIZE) -
+ F2FS_BLK_TO_BYTES(sbi, new_cblocks) -
(cc->clen + COMPRESS_HEADER_SIZE));
- vm_unmap_ram(cc->cbuf, cc->nr_cpages);
- vm_unmap_ram(cc->rbuf, cc->cluster_size);
+ f2fs_vunmap(cc->cbuf, cc->nr_cpages);
+ f2fs_vunmap(cc->rbuf, nr_rfolios);
- for (i = new_nr_cpages; i < cc->nr_cpages; i++) {
+ needed_folios = DIV_ROUND_UP(cbuf_blk_ofs + new_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
+ for (i = needed_folios; i < cc->nr_cpages; i++) {
f2fs_compress_free_page(cc->cpages[i]);
cc->cpages[i] = NULL;
}
@@ -703,16 +730,16 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
if (cops->destroy_compress_ctx)
cops->destroy_compress_ctx(cc);
- cc->valid_nr_cpages = new_nr_cpages;
+ cc->valid_nr_cblocks = new_cblocks;
trace_f2fs_compress_pages_end(cc->inode, cc->cluster_idx,
cc->clen, ret);
return 0;
out_vunmap_cbuf:
- vm_unmap_ram(cc->cbuf, cc->nr_cpages);
+ f2fs_vunmap(cc->cbuf, cc->nr_cpages);
out_vunmap_rbuf:
- vm_unmap_ram(cc->rbuf, cc->cluster_size);
+ f2fs_vunmap(cc->rbuf, nr_rfolios);
out_free_cpages:
for (i = 0; i < cc->nr_cpages; i++) {
if (cc->cpages[i])
@@ -741,7 +768,7 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
const struct f2fs_compress_ops *cops =
f2fs_cops[fi->i_compress_algorithm];
bool bypass_callback = false;
- int ret;
+ int i, ret;
trace_f2fs_decompress_pages_start(dic->inode, dic->cluster_idx,
dic->cluster_size, fi->i_compress_algorithm);
@@ -758,9 +785,9 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
}
dic->clen = le32_to_cpu(dic->cbuf->clen);
- dic->rlen = PAGE_SIZE << dic->log_cluster_size;
+ dic->rlen = F2FS_BLKSIZE(sbi) << dic->log_cluster_size;
- if (dic->clen > PAGE_SIZE * dic->nr_cpages - COMPRESS_HEADER_SIZE) {
+ if (dic->clen > F2FS_BLK_TO_BYTES(sbi, dic->valid_nr_cblocks) - COMPRESS_HEADER_SIZE) {
ret = -EFSCORRUPTED;
/* Avoid f2fs_commit_super in irq context */
@@ -787,6 +814,31 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
}
}
+ /*
+ * Folios with f2fs_folio_state were decompressed into bounce pages.
+ * Copy only the blocks that are not uptodate, so that newer data in
+ * the uptodate (possibly dirty) blocks is kept.
+ */
+ for (i = 0; !ret && i < dic->nr_rpages; i++) {
+ struct page *rpage = dic->rpages[i];
+ struct folio *folio;
+ pgoff_t lblk;
+ unsigned int b;
+
+ if (!rpage || dic->tpages[i] == rpage)
+ continue;
+
+ folio = page_folio(rpage);
+ lblk = f2fs_folio_lblk(sbi, folio);
+ for (b = 0; b < f2fs_blocks_per_folio(sbi, folio); b++) {
+ size_t off = F2FS_BLK_TO_BYTES(sbi, b);
+
+ if (!f2fs_ffs_test_blk_uptodate(folio, lblk + b))
+ memcpy_page(rpage, off, dic->tpages[i], off,
+ F2FS_BLKSIZE(sbi));
+ }
+ }
+
out_release:
f2fs_release_decomp_mem(dic, bypass_callback, false);
@@ -796,8 +848,8 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
f2fs_decompress_end_io(dic, ret, in_task);
}
-static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
- struct folio *folio, nid_t ino, block_t blkaddr);
+static void f2fs_cache_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset, nid_t ino, block_t blkaddr);
/*
* This is called when a page of a compressed cluster has been read from disk
@@ -805,29 +857,35 @@ static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
* page being waited on in the cluster, and if so, it decompresses the cluster
* (or in the case of a failure, cleans up without actually decompressing).
*/
-void f2fs_end_read_compressed_page(struct folio *folio, bool failed,
- block_t blkaddr, bool in_task)
+void f2fs_end_read_compressed_page(struct folio *folio, size_t offset,
+ bool failed, block_t blkaddr, unsigned int nr_blocks,
+ bool in_task)
{
struct decompress_io_ctx *dic = folio->private;
struct f2fs_sb_info *sbi = dic->sbi;
+ int i;
- dec_cache_count(sbi, F2FS_RD_DATA);
+ sub_cache_counts(sbi, F2FS_RD_DATA, nr_blocks);
if (failed)
WRITE_ONCE(dic->failed, true);
- else if (blkaddr && in_task)
- f2fs_cache_compressed_page(sbi, folio,
- dic->inode->i_ino, blkaddr);
+ else if (blkaddr && in_task) {
+ for (i = 0; i < nr_blocks; i++)
+ f2fs_cache_compressed_block(sbi, folio,
+ offset + F2FS_BLK_TO_BYTES(sbi, i),
+ dic->inode->i_ino,
+ blkaddr + i);
+ }
- if (atomic_dec_and_test(&dic->remaining_pages))
+ if (atomic_sub_and_test(nr_blocks, &dic->remaining_blocks))
f2fs_decompress_cluster(dic, in_task);
}
-static bool is_page_in_cluster(struct compress_ctx *cc, pgoff_t index)
+static bool is_block_in_cluster(struct compress_ctx *cc, pgoff_t lblk)
{
if (cc->cluster_idx == NULL_CLUSTER)
return true;
- return cc->cluster_idx == cluster_idx(cc, index);
+ return cc->cluster_idx == cluster_idx(cc, lblk);
}
bool f2fs_cluster_is_empty(struct compress_ctx *cc)
@@ -837,19 +895,40 @@ bool f2fs_cluster_is_empty(struct compress_ctx *cc)
static bool f2fs_cluster_is_full(struct compress_ctx *cc)
{
- return cc->cluster_size == cc->nr_rpages;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ int i;
+
+ if (cc->nr_rpages != nr_folios)
+ return false;
+
+ /* PG_dirty is cleared for writeback; without ffs it covered every block */
+ for (i = 0; i < nr_folios; i++) {
+ struct folio *folio = page_folio(cc->rpages[i]);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int start, count;
+
+ if (f2fs_folio_has_ffs(folio) &&
+ (!f2fs_ffs_dirty_blk_range(folio, 0, nr_blocks,
+ &start, &count) ||
+ count != nr_blocks))
+ return false;
+ }
+ return true;
}
-bool f2fs_cluster_can_merge_page(struct compress_ctx *cc, pgoff_t index)
+bool f2fs_cluster_can_merge_block(struct compress_ctx *cc, pgoff_t lblk)
{
if (f2fs_cluster_is_empty(cc))
return true;
- return is_page_in_cluster(cc, index);
+ return is_block_in_cluster(cc, lblk);
}
bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
int index, int nr_pages, bool uptodate)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
unsigned long pgidx = page_folio(pages[index])->index;
int i = uptodate ? 0 : 1;
@@ -857,13 +936,13 @@ bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
* when uptodate set to true, try to check all pages in cluster is
* uptodate or not.
*/
- if (uptodate && (pgidx % cc->cluster_size))
+ if (uptodate && (pgidx % nr_folios))
return false;
- if (nr_pages - index < cc->cluster_size)
+ if (nr_pages - index < nr_folios)
return false;
- for (; i < cc->cluster_size; i++) {
+ for (; i < nr_folios; i++) {
struct folio *folio = page_folio(pages[index + i]);
if (folio->index != pgidx + i)
@@ -877,17 +956,19 @@ bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
static bool cluster_has_invalid_data(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
loff_t i_size = i_size_read(cc->inode);
- unsigned nr_pages = DIV_ROUND_UP(i_size, PAGE_SIZE);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct page *page = cc->rpages[i];
- f2fs_bug_on(F2FS_I_SB(cc->inode), !page);
+ f2fs_bug_on(sbi, !page);
- /* beyond EOF */
- if (page_folio(page)->index >= nr_pages)
+ /* any block beyond EOF */
+ if (folio_pos(page_folio(page)) +
+ folio_size(page_folio(page)) - F2FS_BLKSIZE(sbi) >= i_size)
return true;
}
return false;
@@ -1014,17 +1095,17 @@ static int f2fs_decompressed_blocks(struct inode *inode,
}
/* return whether cluster is compressed one or not */
-int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
+int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t lblk)
{
return __f2fs_cluster_blocks(inode,
- index >> F2FS_I(inode)->i_log_cluster_size,
+ lblk >> F2FS_I(inode)->i_log_cluster_size,
CLUSTER_IS_COMPR);
}
/* return whether cluster contains non raw blocks or not */
-bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index)
+bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t lblk)
{
- unsigned int cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size;
+ unsigned int cluster_idx = lblk >> F2FS_I(inode)->i_log_cluster_size;
return f2fs_decompressed_blocks(inode, cluster_idx) !=
F2FS_I(inode)->i_cluster_size;
@@ -1045,34 +1126,65 @@ static bool cluster_may_compress(struct compress_ctx *cc)
static void set_cluster_writeback(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
- if (cc->rpages[i])
- set_page_writeback(cc->rpages[i]);
+ for (i = 0; i < nr_folios; i++) {
+ struct folio *folio;
+
+ if (!cc->rpages[i])
+ continue;
+
+ folio = page_folio(cc->rpages[i]);
+ set_page_writeback(cc->rpages[i]);
+ /*
+ * Attach a folio state before the I/O, as the raw write path
+ * does, so that the write completion, which does not hold the
+ * folio lock, never detaches the page private flags kept in
+ * folio->private. Without it, the cleared PG_dirty covered
+ * every block, so mark them dirty.
+ */
+ if (f2fs_is_multiblock_folio(sbi, folio) &&
+ !f2fs_folio_has_ffs(folio)) {
+ f2fs_ffs_mark_subrange_dirty(folio, 0, folio_size(folio));
+ /* It was accounted as a whole, now its dirty bits are. */
+ f2fs_account_dirty_blocks(cc->inode,
+ -(int)f2fs_blocks_per_folio(sbi, folio));
+ }
}
}
static void cancel_cluster_writeback(struct compress_ctx *cc,
- struct compress_io_ctx *cic, int submitted)
+ struct compress_io_ctx *cic, int failed_blk)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
/* Wait for submitted IOs. */
- if (submitted > 1) {
+ if (failed_blk > 1) {
f2fs_submit_merged_write(F2FS_I_SB(cc->inode), DATA);
- while (atomic_read(&cic->pending_pages) !=
- (cc->valid_nr_cpages - submitted + 1))
+ while (atomic_read(&cic->pending_blocks) !=
+ (cc->valid_nr_cblocks - failed_blk + 1))
f2fs_io_schedule_timeout(DEFAULT_SCHEDULE_TIMEOUT);
}
/* Cancel writeback and stay locked. */
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio = page_folio(cc->rpages[i]);
+ unsigned int last_blk_in_folio =
+ ((i + 1) << F2FS_LOG_BLKS_PER_PAGE(sbi)) - 1;
- if (i < submitted) {
- inode_inc_dirty_block(cc->inode);
+ if (last_blk_in_folio < failed_blk) {
folio_lock(folio);
+ /* f2fs_write_compressed_pages() cleared its dirty blocks */
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0,
+ folio_size(folio));
+ else
+ f2fs_account_dirty_blocks(cc->inode,
+ f2fs_blocks_per_folio(sbi, folio));
}
folio_clear_f2fs_gcing(folio);
if (folio_test_writeback(folio))
@@ -1082,21 +1194,28 @@ static void cancel_cluster_writeback(struct compress_ctx *cc,
static void set_cluster_dirty(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio;
if (!cc->rpages[i])
continue;
folio = page_folio(cc->rpages[i]);
+ /*
+ * The whole cluster is rewritten. This also attaches a folio
+ * state to a multi-block folio before it is dirtied.
+ */
+ f2fs_ffs_mark_subrange_dirty(folio, 0, folio_size(folio));
f2fs_folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
}
}
static int prepare_compress_overwrite(struct compress_ctx *cc,
- struct page **pagep, pgoff_t index, void **fsdata)
+ struct page **pagep, pgoff_t lblk, void **fsdata)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
struct address_space *mapping = cc->inode->i_mapping;
@@ -1115,9 +1234,13 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
if (ret)
return ret;
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+
/* keep folio reference to avoid page reclaim */
- for (i = 0; i < cc->cluster_size; i++) {
- folio = f2fs_filemap_get_folio(mapping, start_idx + i,
+ for (i = 0; i < nr_folios; i++) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, start_idx) + i;
+
+ folio = f2fs_filemap_get_folio(mapping, f_index,
fgp_flag, GFP_NOFS);
if (IS_ERR(folio)) {
ret = PTR_ERR(folio);
@@ -1147,10 +1270,12 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
goto out;
}
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, start_idx) + i;
+
f2fs_bug_on(sbi, cc->rpages[i]);
- folio = filemap_lock_folio(mapping, start_idx + i);
+ folio = filemap_lock_folio(mapping, f_index);
if (IS_ERR(folio)) {
/* folio could be truncated */
goto release_and_retry;
@@ -1171,7 +1296,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
if (likely(!ret)) {
*fsdata = cc->rpages;
- *pagep = cc->rpages[offset_in_cluster(cc, index)];
+ *pagep = cc->rpages[offset_in_cluster(cc, lblk)];
return cc->cluster_size;
}
@@ -1184,23 +1309,23 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
}
int f2fs_prepare_compress_overwrite(struct inode *inode,
- struct page **pagep, pgoff_t index, void **fsdata)
+ struct page **pagep, pgoff_t lblk, void **fsdata)
{
struct compress_ctx cc = {
.inode = inode,
.log_cluster_size = F2FS_I(inode)->i_log_cluster_size,
.cluster_size = F2FS_I(inode)->i_cluster_size,
- .cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size,
+ .cluster_idx = lblk >> F2FS_I(inode)->i_log_cluster_size,
.rpages = NULL,
.nr_rpages = 0,
.vi = NULL, /* can't write to fsverity files */
};
- return prepare_compress_overwrite(&cc, pagep, index, fsdata);
+ return prepare_compress_overwrite(&cc, pagep, lblk, fsdata);
}
bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
- pgoff_t index, unsigned copied)
+ pgoff_t lblk, unsigned int copied)
{
struct compress_ctx cc = {
@@ -1210,7 +1335,7 @@ bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
.rpages = fsdata,
};
struct folio *folio = page_folio(cc.rpages[0]);
- bool first_index = (index == folio->index);
+ bool first_index = (lblk == f2fs_folio_lblk(F2FS_I_SB(inode), folio));
if (copied)
set_cluster_dirty(&cc);
@@ -1223,12 +1348,15 @@ bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
void *fsdata = NULL;
struct page *pagep;
struct page **rpages;
int log_cluster_size = F2FS_I(inode)->i_log_cluster_size;
- pgoff_t start_idx = from >> (PAGE_SHIFT + log_cluster_size) <<
+ pgoff_t start_idx = (from >> (sbi->log_blocksize + log_cluster_size)) <<
log_cluster_size;
+ unsigned int nr_folios = (1 << log_cluster_size) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
int err;
@@ -1252,9 +1380,9 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
rpages = fsdata;
- for (i = (1 << log_cluster_size) - 1; i >= 0; i--) {
+ for (i = nr_folios - 1; i >= 0; i--) {
struct folio *folio = page_folio(rpages[i]);
- loff_t start = (loff_t)folio->index << PAGE_SHIFT;
+ loff_t start = folio_pos(folio);
loff_t offset = from > start ? from - start : 0;
folio_zero_segment(folio, offset, folio_size(folio));
@@ -1266,14 +1394,31 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
f2fs_compress_write_end(inode, fsdata, start_idx, true);
err = filemap_write_and_wait_range(inode->i_mapping,
- round_down(from, 1 << log_cluster_size << PAGE_SHIFT),
+ round_down(from, 1 << log_cluster_size << sbi->log_blocksize),
LLONG_MAX);
if (err)
return err;
truncate_pagecache(inode, from);
- return f2fs_do_truncate_blocks(inode, round_up(from, PAGE_SIZE), lock);
+ return f2fs_do_truncate_blocks(inode, round_up(from, F2FS_BLKSIZE(sbi)),
+ lock);
+}
+
+/*
+ * Page cache index tagged on the @i-th bounce page of a cluster. With
+ * subpage blocks, compressed block k is kept at block offset k of the
+ * cluster, so the first bounce page also covers the unused slot of the
+ * COMPRESS_ADDR block and the bounce pages map 1:1 to the folios of the
+ * cluster. Otherwise there is no bounce page for COMPRESS_ADDR and page
+ * i holds block i + 1.
+ */
+static pgoff_t f2fs_cpage_index(struct f2fs_sb_info *sbi, pgoff_t start_idx,
+ unsigned int i)
+{
+ if (f2fs_has_subpage_blocks(sbi))
+ return f2fs_lblk_to_folio_index(sbi, start_idx) + i;
+ return start_idx + 1 + i;
}
static int f2fs_write_compressed_pages(struct compress_ctx *cc,
@@ -1303,8 +1448,9 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
struct node_info ni;
struct compress_io_ctx *cic;
struct f2fs_lock_context lc;
+ unsigned int nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
pgoff_t start_idx = start_idx_of_cluster(cc);
- unsigned int last_index = cc->cluster_size - 1;
+ unsigned int blk_idx, c_folio_idx, c_folio_ofs;
loff_t psize;
int i, err;
bool quota_inode = IS_NOQUOTA(inode);
@@ -1338,10 +1484,10 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
goto out_put_dnode;
}
- folio = page_folio(cc->rpages[last_index]);
+ folio = page_folio(cc->rpages[nr_rfolios - 1]);
psize = folio_next_pos(folio);
- err = f2fs_get_node_info(fio.sbi, dn.nid, &ni, false);
+ err = f2fs_get_node_info(sbi, dn.nid, &ni, false);
if (err)
goto out_put_dnode;
@@ -1353,18 +1499,25 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
cic->magic = F2FS_COMPRESSED_PAGE_MAGIC;
cic->inode = inode;
- atomic_set(&cic->pending_pages, cc->valid_nr_cpages);
- cic->rpages = page_array_alloc(sbi, cc->cluster_size);
+ atomic_set(&cic->pending_blocks, cc->valid_nr_cblocks);
+ cic->rpages = page_array_alloc(sbi, nr_rfolios);
if (!cic->rpages)
goto out_put_cic;
- cic->nr_rpages = cc->cluster_size;
+ cic->nr_rpages = nr_rfolios;
+ cic->cpages = cc->cpages;
+ cic->nr_cpages = cc->nr_cpages;
+ cc->cpages = NULL;
- for (i = 0; i < cc->valid_nr_cpages; i++) {
- f2fs_set_compressed_page(cc->cpages[i], inode,
- page_folio(cc->rpages[i + 1])->index, cic);
- fio.compressed_page = cc->cpages[i];
+ for (i = 0; i < cic->nr_cpages; i++) {
+ if (!cic->cpages[i])
+ break;
+
+ f2fs_set_compressed_page(cic->cpages[i], inode,
+ f2fs_cpage_index(sbi, start_idx, i), cic);
+ }
+ for (i = 0; i < cc->valid_nr_cblocks; i++) {
fio.old_blkaddr = data_blkaddr(dn.inode, dn.node_entry,
dn.ofs_in_node + i + 1);
@@ -1374,14 +1527,15 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
set_cluster_writeback(cc);
- for (i = 0; i < cc->cluster_size; i++)
+ for (i = 0; i < nr_rfolios; i++)
cic->rpages[i] = cc->rpages[i];
for (i = 0; i < cc->cluster_size; i++, dn.ofs_in_node++) {
block_t blkaddr;
+ unsigned int r_idx = i >> F2FS_LOG_BLKS_PER_PAGE(sbi);
blkaddr = f2fs_data_blkaddr(&dn);
- fio.folio = page_folio(cc->rpages[i]);
+ fio.folio = page_folio(cc->rpages[r_idx]);
fio.old_blkaddr = blkaddr;
/* cluster header */
@@ -1397,7 +1551,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
if (fio.compr_blocks && __is_valid_data_blkaddr(blkaddr))
fio.compr_blocks++;
- if (i > cc->valid_nr_cpages) {
+ if (i > cc->valid_nr_cblocks) {
if (__is_valid_data_blkaddr(blkaddr)) {
f2fs_invalidate_blocks(sbi, blkaddr, 1);
f2fs_update_data_blkaddr(&dn, NEW_ADDR);
@@ -1407,9 +1561,13 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
f2fs_bug_on(fio.sbi, blkaddr == NULL_ADDR);
- fio.compressed_page = cc->cpages[i - 1];
+ blk_idx = f2fs_has_subpage_blocks(sbi) ? i : (i - 1);
+ c_folio_idx = blk_idx >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ c_folio_ofs = blk_idx % F2FS_BLKS_PER_PAGE(sbi);
+
+ fio.compressed_page = cic->cpages[c_folio_idx];
+ fio.folio_offset = c_folio_ofs;
- cc->cpages[i - 1] = NULL;
fio.submitted = 0;
f2fs_outplace_write_data(&dn, &fio);
if (unlikely(!fio.submitted)) {
@@ -1419,14 +1577,19 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
}
(*submitted)++;
unlock_continue:
- inode_dec_dirty_block(cc->inode);
- folio_unlock(fio.folio);
+ if ((i + 1) % F2FS_BLKS_PER_PAGE(sbi) == 0) {
+ f2fs_folio_clear_dirty_blocks(fio.folio, true);
+ folio_unlock(fio.folio);
+ }
}
+ /* count pages like the raw write path */
+ *submitted = DIV_ROUND_UP(*submitted, F2FS_BLKS_PER_PAGE(sbi));
+
if (fio.compr_blocks)
f2fs_i_compr_blocks_update(inode, fio.compr_blocks - 1, false);
- f2fs_i_compr_blocks_update(inode, cc->valid_nr_cpages, true);
- add_compr_block_stat(inode, cc->valid_nr_cpages);
+ f2fs_i_compr_blocks_update(inode, cc->valid_nr_cblocks, true);
+ add_compr_block_stat(inode, cc->valid_nr_cblocks);
set_inode_flag(cc->inode, FI_APPEND_WRITE);
@@ -1442,13 +1605,18 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
spin_unlock(&fi->i_size_lock);
f2fs_put_rpages(cc);
- page_array_free(sbi, cc->cpages, cc->nr_cpages);
- cc->cpages = NULL;
f2fs_destroy_compress_ctx(cc, false);
return 0;
out_free_page_array:
- page_array_free(sbi, cic->rpages, cc->cluster_size);
+ page_array_free(sbi, cic->rpages, nr_rfolios);
+
+ for (i = 0; i < cic->nr_cpages; i++) {
+ if (!cic->cpages[i])
+ continue;
+ f2fs_compress_free_page(cic->cpages[i]);
+ }
+ page_array_free(sbi, cic->cpages, cic->nr_cpages);
out_put_cic:
kmem_cache_free(cic_entry_slab, cic);
out_put_dnode:
@@ -1459,18 +1627,21 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
else
f2fs_unlock_op(sbi, &lc);
out_free:
- for (i = 0; i < cc->valid_nr_cpages; i++) {
- f2fs_compress_free_page(cc->cpages[i]);
- cc->cpages[i] = NULL;
+ if (cc->cpages) {
+ for (i = 0; i < cc->nr_cpages; i++) {
+ if (!cc->cpages[i])
+ continue;
+ f2fs_compress_free_page(cc->cpages[i]);
+ }
+ page_array_free(sbi, cc->cpages, cc->nr_cpages);
+ cc->cpages = NULL;
}
- page_array_free(sbi, cc->cpages, cc->nr_cpages);
- cc->cpages = NULL;
return -EAGAIN;
}
-void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
+void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio,
+ unsigned int nr_blocks)
{
- struct page *page = &folio->page;
struct f2fs_sb_info *sbi = bio->bi_private;
struct compress_io_ctx *cic = folio->private;
enum count_type type = WB_DATA_TYPE(folio, true);
@@ -1479,13 +1650,17 @@ void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
if (unlikely(bio->bi_status != BLK_STS_OK))
mapping_set_error(cic->inode->i_mapping, -EIO);
- f2fs_compress_free_page(page);
-
- if (atomic_dec_return(&cic->pending_pages)) {
- dec_cache_count(sbi, type);
+ if (!atomic_sub_and_test(nr_blocks, &cic->pending_blocks)) {
+ sub_cache_counts(sbi, type, nr_blocks);
return;
}
+ for (i = 0; i < cic->nr_cpages; i++) {
+ if (cic->cpages[i])
+ f2fs_compress_free_page(cic->cpages[i]);
+ }
+ page_array_free(sbi, cic->cpages, cic->nr_cpages);
+
for (i = 0; i < cic->nr_rpages; i++) {
struct folio *rfolio;
@@ -1499,12 +1674,12 @@ void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
kmem_cache_free(cic_entry_slab, cic);
/*
- * Make sure dec_cache_count() is the last access to sbi.
+ * Make sure sub_cache_counts() is the last access to sbi.
* Once it drops the F2FS_WB_CP_DATA counter to zero, the
* unmount thread can proceed to destroy sbi and
* sbi->page_array_slab.
*/
- dec_cache_count(sbi, type);
+ sub_cache_counts(sbi, type, nr_blocks);
}
static int f2fs_write_raw_pages(struct compress_ctx *cc,
@@ -1517,10 +1692,12 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
struct f2fs_lock_context lc;
int submitted, compr_blocks, i;
int ret = 0;
+ unsigned int nr_folios = cc->cluster_size >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
compr_blocks = f2fs_compressed_blocks(cc);
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
if (!cc->rpages[i])
continue;
@@ -1535,7 +1712,7 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
if (compr_blocks > 0)
f2fs_lock_op(sbi, &lc);
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio;
if (!cc->rpages[i])
@@ -1595,6 +1772,34 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
return ret;
}
+/*
+ * A compressed cluster is rewritten as a whole, as set_cluster_dirty() does.
+ * With subpage blocks, a dirty folio can have only some of its blocks dirty
+ * (e.g. defragment), so dirty the rest before trying to compress.
+ */
+static void dirty_partial_compressed_cluster(struct compress_ctx *cc)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ int i;
+
+ if (cc->nr_rpages != nr_folios || f2fs_cluster_is_full(cc))
+ return;
+ for (i = 0; i < nr_folios; i++)
+ if (!folio_test_uptodate(page_folio(cc->rpages[i])))
+ return;
+ if (f2fs_compressed_blocks(cc) <= 0)
+ return;
+ for (i = 0; i < nr_folios; i++) {
+ struct folio *folio = page_folio(cc->rpages[i]);
+
+ /* A folio without a folio state is dirty as a whole. */
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0,
+ folio_size(folio));
+ }
+}
+
int f2fs_write_multi_pages(struct compress_ctx *cc,
int *submitted,
struct writeback_control *wbc,
@@ -1603,6 +1808,8 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
int err;
*submitted = 0;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(cc->inode)))
+ dirty_partial_compressed_cluster(cc);
if (cluster_may_compress(cc)) {
err = f2fs_compress_pages(cc);
if (err == -EAGAIN) {
@@ -1639,17 +1846,27 @@ static int f2fs_prepare_decomp_mem(struct decompress_io_ctx *dic,
bool pre_alloc)
{
const struct f2fs_compress_ops *cops = f2fs_cops[dic->compress_algorithm];
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(dic->sbi);
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(dic->sbi) ? 1 : 0;
+ void *cbuf_vaddr;
int i;
if (!allow_memalloc_for_decomp(dic->sbi, pre_alloc))
return 0;
- dic->tpages = page_array_alloc(dic->sbi, dic->cluster_size);
+ dic->tpages = page_array_alloc(dic->sbi, nr_rfolios);
if (!dic->tpages)
return -ENOMEM;
- for (i = 0; i < dic->cluster_size; i++) {
- if (dic->rpages[i]) {
+ for (i = 0; i < nr_rfolios; i++) {
+ /*
+ * A folio with f2fs_folio_state may have uptodate blocks newer
+ * than the on-disk cluster (e.g., dirty blocks left after a
+ * partial truncate). Decompress it into a bounce page instead,
+ * see f2fs_decompress_cluster().
+ */
+ if (dic->rpages[i] &&
+ !f2fs_folio_has_ffs(page_folio(dic->rpages[i]))) {
dic->tpages[i] = dic->rpages[i];
continue;
}
@@ -1657,14 +1874,16 @@ static int f2fs_prepare_decomp_mem(struct decompress_io_ctx *dic,
dic->tpages[i] = f2fs_compress_alloc_page();
}
- dic->rbuf = f2fs_vmap(dic->tpages, dic->cluster_size);
+ dic->rbuf = f2fs_vmap(dic->tpages, nr_rfolios);
if (!dic->rbuf)
return -ENOMEM;
- dic->cbuf = f2fs_vmap(dic->cpages, dic->nr_cpages);
- if (!dic->cbuf)
+ cbuf_vaddr = f2fs_vmap(dic->cpages, dic->nr_cpages);
+ if (!cbuf_vaddr)
return -ENOMEM;
+ dic->cbuf = cbuf_vaddr + F2FS_BLK_TO_BYTES(dic->sbi, cbuf_blk_ofs);
+
if (cops->init_decompress_ctx)
return cops->init_decompress_ctx(dic);
@@ -1675,6 +1894,7 @@ static void f2fs_release_decomp_mem(struct decompress_io_ctx *dic,
bool bypass_destroy_callback, bool pre_alloc)
{
const struct f2fs_compress_ops *cops = f2fs_cops[dic->compress_algorithm];
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(dic->sbi);
if (!allow_memalloc_for_decomp(dic->sbi, pre_alloc))
return;
@@ -1682,11 +1902,15 @@ static void f2fs_release_decomp_mem(struct decompress_io_ctx *dic,
if (!bypass_destroy_callback && cops->destroy_decompress_ctx)
cops->destroy_decompress_ctx(dic);
- if (dic->cbuf)
- vm_unmap_ram(dic->cbuf, dic->nr_cpages);
+ if (dic->cbuf) {
+ f2fs_vunmap(dic->cbuf, dic->nr_cpages);
+ dic->cbuf = NULL;
+ }
- if (dic->rbuf)
- vm_unmap_ram(dic->rbuf, dic->cluster_size);
+ if (dic->rbuf) {
+ f2fs_vunmap(dic->rbuf, nr_rfolios);
+ dic->rbuf = NULL;
+ }
}
static void f2fs_free_dic(struct decompress_io_ctx *dic,
@@ -1697,13 +1921,17 @@ struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc)
struct decompress_io_ctx *dic;
pgoff_t start_idx = start_idx_of_cluster(cc);
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(sbi) ? 1 : 0;
+ unsigned int nr_cpages = DIV_ROUND_UP(cbuf_blk_ofs + cc->valid_nr_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
int i, ret;
dic = f2fs_kmem_cache_alloc(dic_entry_slab, GFP_F2FS_ZERO, false, sbi);
if (!dic)
return ERR_PTR(-ENOMEM);
- dic->rpages = page_array_alloc(sbi, cc->cluster_size);
+ dic->rpages = page_array_alloc(sbi, nr_rfolios);
if (!dic->rpages) {
kmem_cache_free(dic_entry_slab, dic);
return ERR_PTR(-ENOMEM);
@@ -1713,31 +1941,32 @@ struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc)
dic->inode = cc->inode;
dic->sbi = sbi;
dic->compress_algorithm = F2FS_I(cc->inode)->i_compress_algorithm;
- atomic_set(&dic->remaining_pages, cc->nr_cpages);
+ atomic_set(&dic->remaining_blocks, cc->valid_nr_cblocks);
dic->cluster_idx = cc->cluster_idx;
dic->cluster_size = cc->cluster_size;
dic->log_cluster_size = cc->log_cluster_size;
- dic->nr_cpages = cc->nr_cpages;
+ dic->valid_nr_cblocks = cc->valid_nr_cblocks;
+ dic->nr_cpages = nr_cpages;
refcount_set(&dic->refcnt, 1);
dic->failed = false;
dic->vi = cc->vi;
- for (i = 0; i < dic->cluster_size; i++)
+ for (i = 0; i < nr_rfolios; i++)
dic->rpages[i] = cc->rpages[i];
- dic->nr_rpages = cc->cluster_size;
+ dic->nr_rpages = nr_rfolios;
- dic->cpages = page_array_alloc(sbi, dic->nr_cpages);
+ dic->cpages = page_array_alloc(sbi, nr_cpages);
if (!dic->cpages) {
ret = -ENOMEM;
goto out_free;
}
- for (i = 0; i < dic->nr_cpages; i++) {
+ for (i = 0; i < nr_cpages; i++) {
struct page *page;
page = f2fs_compress_alloc_page();
f2fs_set_compressed_page(page, cc->inode,
- start_idx + i + 1, dic);
+ f2fs_cpage_index(sbi, start_idx, i), dic);
dic->cpages[i] = page;
}
@@ -1758,18 +1987,19 @@ static void f2fs_free_dic(struct decompress_io_ctx *dic,
int i;
/* use sbi in dic to avoid UFA of dic->inode*/
struct f2fs_sb_info *sbi = dic->sbi;
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
f2fs_release_decomp_mem(dic, bypass_destroy_callback, true);
if (dic->tpages) {
- for (i = 0; i < dic->cluster_size; i++) {
- if (dic->rpages[i])
+ for (i = 0; i < nr_rfolios; i++) {
+ if (dic->tpages[i] == dic->rpages[i])
continue;
if (!dic->tpages[i])
continue;
f2fs_compress_free_page(dic->tpages[i]);
}
- page_array_free(sbi, dic->tpages, dic->cluster_size);
+ page_array_free(sbi, dic->tpages, nr_rfolios);
}
if (dic->cpages) {
@@ -1812,15 +2042,19 @@ static void f2fs_verify_cluster(struct work_struct *work)
int i;
/* Verify, update, and unlock the decompressed pages. */
- for (i = 0; i < dic->cluster_size; i++) {
+ for (i = 0; i < dic->nr_rpages; i++) {
struct page *rpage = dic->rpages[i];
struct folio *rfolio;
if (!rpage)
continue;
rfolio = page_folio(rpage);
- if (fsverity_verify_folio(dic->vi, rfolio))
+ if (fsverity_verify_folio(dic->vi, rfolio)) {
+ if (f2fs_is_multiblock_folio(dic->sbi, rfolio))
+ f2fs_ffs_mark_subrange_uptodate(rfolio, 0,
+ folio_size(rfolio));
folio_mark_uptodate(rfolio);
+ }
folio_unlock(rfolio);
}
@@ -1849,16 +2083,20 @@ void f2fs_decompress_end_io(struct decompress_io_ctx *dic, bool failed,
}
/* Update and unlock the cluster's pagecache pages. */
- for (i = 0; i < dic->cluster_size; i++) {
+ for (i = 0; i < dic->nr_rpages; i++) {
struct page *rpage = dic->rpages[i];
if (!rpage)
continue;
- if (failed)
+ if (failed) {
ClearPageUptodate(rpage);
- else
+ } else {
+ if (f2fs_is_multiblock_folio(dic->sbi, page_folio(rpage)))
+ f2fs_ffs_mark_subrange_uptodate(page_folio(rpage), 0,
+ folio_size(page_folio(rpage)));
SetPageUptodate(rpage);
+ }
unlock_page(rpage);
}
@@ -1916,8 +2154,8 @@ void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
f2fs_drop_cache_range(COMPRESS_CACHE(sbi), blkaddr, len, false);
}
-static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
- struct folio *folio, nid_t ino, block_t blkaddr)
+static void f2fs_cache_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset, nid_t ino, block_t blkaddr)
{
struct f2fs_cached_block *entry;
@@ -1961,13 +2199,14 @@ static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
}
entry->ino = ino;
- memcpy(cache_address(entry), folio_address(folio), sbi->blocksize);
+ memcpy(cache_address(entry), folio_address(folio) + offset,
+ sbi->blocksize);
f2fs_cache_set_uptodate(entry);
f2fs_put_cache(entry, true);
}
-bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
- block_t blkaddr)
+bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi, struct folio *folio,
+ size_t offset, block_t blkaddr)
{
struct f2fs_cached_block *entry;
bool hitted = false;
@@ -1982,8 +2221,8 @@ bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
if (f2fs_is_compress_cache(entry) &&
f2fs_cache_test_uptodate(entry)) {
atomic_inc(&sbi->compress_page_hit);
- memcpy(folio_address(folio),
- cache_address(entry), folio_size(folio));
+ memcpy(folio_address(folio) + offset,
+ cache_address(entry), sbi->blocksize);
hitted = true;
}
f2fs_put_cache(entry, true);
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 7dd6764a9363..cdc9e6e8633c 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -150,12 +150,17 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
bool finished = true;
bool uptodate = bio->bi_status == BLK_STS_OK;
- if (!folio_test_large(folio) &&
- f2fs_is_compressed_page(folio)) {
+ if (f2fs_is_compressed_page(folio)) {
+ struct decompress_io_ctx *dic = folio->private;
+
+ /* The cluster may be unlocked, so do not use its inode. */
+ nr_blocks = F2FS_BYTES_TO_BLK(dic->sbi, fi.length);
if (ctx && !ctx->decompression_attempted)
- f2fs_end_read_compressed_page(folio, true, 0,
+ f2fs_end_read_compressed_page(folio, fi.offset,
+ true, 0, nr_blocks,
in_task);
- f2fs_put_folio_dic(folio, in_task);
+ while (nr_blocks--)
+ f2fs_put_folio_dic(folio, in_task);
continue;
}
@@ -259,20 +264,24 @@ static void f2fs_verify_and_finish_bio(struct bio *bio, bool in_task)
static void f2fs_handle_step_decompress(struct bio_post_read_ctx *ctx,
bool in_task)
{
+ struct f2fs_sb_info *sbi = ctx->sbi;
struct folio_iter fi;
bool all_compressed = true;
block_t blkaddr = ctx->fs_blkaddr;
bio_for_each_folio_all(fi, ctx->bio) {
struct folio *folio = fi.folio;
+ unsigned int nr_blocks = F2FS_BYTES_TO_BLK(sbi, fi.length);
- if (f2fs_is_compressed_page(folio))
- f2fs_end_read_compressed_page(folio, false, blkaddr,
- in_task);
- else
+ if (f2fs_is_compressed_page(folio)) {
+ f2fs_end_read_compressed_page(folio, fi.offset,
+ false, blkaddr,
+ nr_blocks, in_task);
+ } else {
all_compressed = false;
+ }
- blkaddr++;
+ blkaddr += nr_blocks;
}
ctx->decompression_attempted = true;
@@ -425,7 +434,7 @@ static void f2fs_write_end_bio(struct bio *bio)
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_is_compressed_page(folio)) {
- f2fs_compress_write_end_io(bio, folio);
+ f2fs_compress_write_end_io(bio, folio, nr_blocks);
continue;
}
#endif
@@ -1337,9 +1346,14 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
io->fio = *fio;
}
- bio_offset = bio_folio == fio->folio ? F2FS_FIO_BIO_OFFSET(fio) : 0;
- bio_len = bio_folio == fio->folio ? F2FS_FIO_BIO_SIZE(fio) :
- folio_size(bio_folio);
+ /* A compressed page holds several blocks with subpage blocks. */
+ if (bio_folio == fio->folio || fio->compressed_page) {
+ bio_offset = F2FS_FIO_BIO_OFFSET(fio);
+ bio_len = F2FS_FIO_BIO_SIZE(fio);
+ } else {
+ bio_offset = 0;
+ bio_len = folio_size(bio_folio);
+ }
if (!bio_add_folio(io->bio, bio_folio, bio_len, bio_offset)) {
__submit_merged_bio(io);
@@ -2741,7 +2755,7 @@ static int f2fs_read_single_page(struct inode *inode, struct fsverity_info *vi,
#ifdef CONFIG_F2FS_FS_COMPRESSION
int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
- unsigned nr_pages, sector_t *last_block_in_bio,
+ unsigned int nr_blocks, sector_t *last_block_in_bio,
struct readahead_control *rac, bool for_write)
{
struct dnode_of_data dn;
@@ -2749,6 +2763,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct bio *bio = *bio_ret;
unsigned int start_idx = cc->cluster_idx << cc->log_cluster_size;
+ unsigned int nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
sector_t last_block_in_file;
const unsigned int blocksize = F2FS_BLKSIZE(sbi);
struct decompress_io_ctx *dic = NULL;
@@ -2770,7 +2785,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
blocksize - 1);
/* get rid of pages beyond EOF */
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_rfolios; i++) {
struct page *page = cc->rpages[i];
struct folio *folio;
@@ -2778,10 +2793,9 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
continue;
folio = page_folio(page);
- if ((sector_t)folio->index >= last_block_in_file) {
+ if ((sector_t)f2fs_folio_lblk(sbi, folio) >= last_block_in_file) {
folio_zero_segment(folio, 0, folio_size(folio));
- if (!folio_test_uptodate(folio))
- folio_mark_uptodate(folio);
+ f2fs_ffs_mark_subrange_uptodate(folio, 0, folio_size(folio));
} else if (!folio_test_uptodate(folio)) {
continue;
}
@@ -2824,14 +2838,14 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
ret = -EFAULT;
goto out_put_dnode;
}
- cc->nr_cpages++;
+ cc->valid_nr_cblocks++;
if (!from_dnode && i >= ei.c_len)
break;
}
/* nothing to decompress */
- if (cc->nr_cpages == 0) {
+ if (cc->valid_nr_cblocks == 0) {
ret = 0;
goto out_put_dnode;
}
@@ -2842,8 +2856,12 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
goto out_put_dnode;
}
- for (i = 0; i < cc->nr_cpages; i++) {
- struct folio *folio = page_folio(dic->cpages[i]);
+ for (i = 0; i < cc->valid_nr_cblocks; i++) {
+ unsigned int blk_idx = f2fs_has_subpage_blocks(sbi) ? (1 + i) : i;
+ unsigned int c_folio_idx = blk_idx >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ unsigned int c_folio_ofs = blk_idx % F2FS_BLKS_PER_PAGE(sbi);
+ struct folio *folio = page_folio(dic->cpages[c_folio_idx]);
+ size_t offset = c_folio_ofs * blocksize;
block_t blkaddr;
struct bio_post_read_ctx *ctx;
@@ -2853,8 +2871,8 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
f2fs_wait_on_block_writeback(inode, blkaddr);
- if (f2fs_load_compressed_folio(sbi, folio, blkaddr)) {
- if (atomic_dec_and_test(&dic->remaining_pages)) {
+ if (f2fs_load_compressed_block(sbi, folio, offset, blkaddr)) {
+ if (atomic_dec_and_test(&dic->remaining_blocks)) {
f2fs_decompress_cluster(dic, true);
break;
}
@@ -2863,7 +2881,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
if (bio && (!page_is_mergeable(sbi, bio,
*last_block_in_bio, blkaddr) ||
- !f2fs_crypt_mergeable_bio(bio, inode, folio->index, NULL))) {
+ !f2fs_crypt_mergeable_bio(bio, inode, start_idx + 1 + i, NULL))) {
submit_and_realloc:
f2fs_submit_read_bio(sbi, bio, DATA);
bio = NULL;
@@ -2871,11 +2889,11 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
if (!bio)
bio = f2fs_grab_read_bio(inode, cc->vi, blkaddr,
- nr_pages - i,
+ nr_blocks - i,
f2fs_ra_op_flags(rac),
- folio->index, for_write);
+ start_idx + 1 + i, for_write);
- if (!bio_add_folio(bio, folio, blocksize, 0))
+ if (!bio_add_folio(bio, folio, blocksize, offset))
goto submit_and_realloc;
ctx = get_post_read_ctx(bio);
@@ -2898,7 +2916,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
if (from_dnode)
f2fs_put_dnode(&dn);
out:
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_rfolios; i++) {
if (cc->rpages[i]) {
ClearPageUptodate(cc->rpages[i]);
unlock_page(cc->rpages[i]);
@@ -3408,7 +3426,13 @@ static int f2fs_read_data_large_folio(struct inode *inode,
if (!folio)
goto out;
- f2fs_update_read_folio_count(sbi, folio);
+ /*
+ * f2fs_mpage_readpages() counts the folios of the compressed files
+ * that it reads cluster by cluster, i.e. those without large folios.
+ */
+ if (!f2fs_compressed_file(inode) ||
+ mapping_large_folio_support(inode->i_mapping))
+ f2fs_update_read_folio_count(sbi, folio);
folio_in_bio = false;
index = f2fs_folio_lblk(sbi, folio);
@@ -3599,7 +3623,7 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
.rpages = NULL,
.cpages = NULL,
.nr_rpages = 0,
- .nr_cpages = 0,
+ .valid_nr_cblocks = 0,
};
pgoff_t nc_cluster_idx = NULL_CLUSTER;
pgoff_t index;
@@ -3609,14 +3633,19 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
unsigned max_nr_pages = nr_pages;
int ret = 0;
- if (f2fs_has_multiblock_folios(inode))
+ if (mapping_large_folio_support(inode->i_mapping) ||
+ (f2fs_has_subpage_blocks(F2FS_I_SB(inode)) &&
+ !f2fs_compressed_file(inode)))
return f2fs_read_data_large_folio(inode, vi, rac, folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_compressed_file(inode)) {
+ unsigned int nr_rfolios = cc.cluster_size >>
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(inode));
+
index = rac ? readahead_index(rac) : folio->index;
- max_nr_pages = round_up(index + nr_pages, cc.cluster_size) -
- round_down(index, cc.cluster_size);
+ max_nr_pages = round_up(index + nr_pages, nr_rfolios) -
+ round_down(index, nr_rfolios);
}
#endif
@@ -3638,16 +3667,17 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
f2fs_update_read_folio_count(F2FS_I_SB(inode), folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
- index = folio->index;
+ pgoff_t f_lblk = f2fs_folio_lblk(F2FS_I_SB(inode), folio);
if (!f2fs_compressed_file(inode))
goto read_single_page;
/* there are remained compressed pages, submit them */
- if (!f2fs_cluster_can_merge_page(&cc, index)) {
+ if (!f2fs_cluster_can_merge_block(&cc, f_lblk)) {
cc.vi = vi;
ret = f2fs_read_multi_pages(&cc, &bio,
- max_nr_pages,
+ max_nr_pages <<
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(inode)),
&last_block_in_bio,
rac, false);
f2fs_destroy_compress_ctx(&cc, false);
@@ -3655,15 +3685,15 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
goto set_error_page;
}
if (cc.cluster_idx == NULL_CLUSTER) {
- if (nc_cluster_idx == index >> cc.log_cluster_size)
+ if (nc_cluster_idx == f_lblk >> cc.log_cluster_size)
goto read_single_page;
- ret = f2fs_is_compressed_cluster(inode, index);
+ ret = f2fs_is_compressed_cluster(inode, f_lblk);
if (ret < 0)
goto set_error_page;
else if (!ret) {
nc_cluster_idx =
- index >> cc.log_cluster_size;
+ f_lblk >> cc.log_cluster_size;
goto read_single_page;
}
@@ -3679,15 +3709,24 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
read_single_page:
#endif
- ret = f2fs_read_single_page(inode, vi, folio, max_nr_pages,
- &map, &bio, &last_block_in_bio,
- rac);
- if (ret) {
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
+ ret = f2fs_read_data_large_folio(inode, vi, NULL, folio);
+ if (ret)
+ continue;
+ } else {
+ ret = f2fs_read_single_page(inode, vi, folio,
+ max_nr_pages, &map, &bio,
+ &last_block_in_bio, rac);
+ if (ret) {
#ifdef CONFIG_F2FS_FS_COMPRESSION
set_error_page:
#endif
- folio_zero_segment(folio, 0, folio_size(folio));
- folio_unlock(folio);
+ /* Keep the blocks that are uptodate in ffs */
+ if (!f2fs_folio_has_ffs(folio))
+ folio_zero_segment(folio, 0,
+ folio_size(folio));
+ folio_unlock(folio);
+ }
}
#ifdef CONFIG_F2FS_FS_COMPRESSION
next_page:
@@ -3699,7 +3738,8 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
if (nr_pages == 1 && !f2fs_cluster_is_empty(&cc)) {
cc.vi = vi;
ret = f2fs_read_multi_pages(&cc, &bio,
- max_nr_pages,
+ max_nr_pages <<
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(inode)),
&last_block_in_bio,
rac, false);
f2fs_destroy_compress_ctx(&cc, false);
@@ -4008,7 +4048,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type,
- u64 start, u64 end)
+ u64 start, u64 end,
+ int compr_blocks)
{
struct inode *inode = folio->mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -4041,6 +4082,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
.folio_blkcnt = 1,
.encrypted_page = NULL,
.submitted = 0,
+ .compr_blocks = (i == 0) ? compr_blocks : 0,
.need_lock = LOCK_DONE,
.meta_gc = f2fs_meta_inode_gc_required(inode) ? 1 : 0,
.io_type = io_type,
@@ -4051,8 +4093,12 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct f2fs_folio_state *ffs =
(struct f2fs_folio_state *)folio->private;
- /* Write only dirty blocks */
- if (!f2fs_ffs_test_blk_dirty(folio, data_idx))
+ /*
+ * Write only dirty blocks, or all blocks of a
+ * compressed cluster.
+ */
+ if (!compr_blocks &&
+ !f2fs_ffs_test_blk_dirty(folio, data_idx))
continue;
atomic_inc(&ffs->write_blocks_pending);
}
@@ -4127,6 +4173,9 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
fio.version = ni.version;
+ if (fio.compr_blocks && fio.old_blkaddr == COMPRESS_ADDR)
+ f2fs_i_compr_blocks_update(inode, fio.compr_blocks - 1, false);
+
f2fs_outplace_write_data(&dn, &fio);
f2fs_put_dnode(&dn);
/* The outplace write path failed to submit this block. */
@@ -4213,6 +4262,33 @@ static enum f2fs_wb_action f2fs_wb_folio_action(struct f2fs_sb_info *sbi,
return F2FS_WB_WRITE;
}
+/*
+ * Hold a bias on write_blocks_pending while the blocks of a multi-block
+ * folio are submitted, so that their completions cannot end the folio
+ * writeback early. Without ffs, the folio dirty flag covered @len bytes.
+ */
+static void f2fs_folio_get_write_bias(struct folio *folio, size_t len)
+{
+ struct f2fs_folio_state *ffs;
+
+ if (!f2fs_folio_has_ffs(folio)) {
+ struct inode *inode = folio->mapping->host;
+
+ /*
+ * PG_dirty, which was cleared for writeback, covered every
+ * block and the folio was accounted as a whole. Account its
+ * dirty bits instead.
+ */
+ WARN_ON_ONCE(folio_test_dirty(folio));
+ f2fs_ffs_mark_subrange_dirty(folio, 0, len);
+ f2fs_account_dirty_blocks(inode,
+ -(int)f2fs_blocks_per_folio(F2FS_I_SB(inode), folio));
+ }
+ ffs = folio->private;
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
+}
+
int f2fs_write_single_data_page(struct folio *folio, int *submitted,
struct bio **bio,
sector_t *last_block,
@@ -4231,6 +4307,7 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
bool need_balance_fs = false;
bool quota_inode = IS_NOQUOTA(inode);
int err = 0;
+ int local_submitted = 0;
struct f2fs_io_info fio = {
.sbi = sbi,
.ino = inode->i_ino,
@@ -4277,6 +4354,46 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
folio_zero_segment(folio, offset, folio_size(folio));
write:
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ struct f2fs_lock_context lc;
+ unsigned int to_blk = f2fs_blocks_per_folio(sbi, folio);
+ u64 pos = folio_pos(folio);
+
+ if (!compr_blocks && !f2fs_verity_in_progress(inode))
+ to_blk = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, i_size - pos),
+ to_blk);
+
+ /* Deadlock due to between page->lock and f2fs_lock_op */
+ if (!compr_blocks && !f2fs_trylock_op(sbi, &lc)) {
+ err = -EAGAIN;
+ goto redirty_out;
+ }
+
+ folio_start_writeback(folio);
+ f2fs_folio_get_write_bias(folio, F2FS_BLK_TO_BYTES(sbi, to_blk));
+
+ err = f2fs_write_folio_dirty_range(folio, &local_submitted,
+ wbc, io_type, pos,
+ pos + F2FS_BLK_TO_BYTES(sbi, to_blk),
+ compr_blocks);
+
+ if (!compr_blocks)
+ f2fs_unlock_op(sbi, &lc);
+
+ /* The callers drop the submitted count on error. */
+ if (err && local_submitted)
+ f2fs_submit_merged_write_cond(sbi, NULL, folio);
+
+ if (!err)
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ folio_size(folio));
+
+ f2fs_ffs_put_write_pending(folio, 1);
+
+ goto done;
+ }
+
if (f2fs_cp_controlled_inode(inode)) {
struct f2fs_lock_context lc;
@@ -4346,7 +4463,9 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
}
if (submitted)
- *submitted = fio.submitted;
+ *submitted = f2fs_is_multiblock_folio(sbi, folio) ?
+ DIV_ROUND_UP(local_submitted, F2FS_BLKS_PER_PAGE(sbi)) :
+ fio.submitted;
return 0;
@@ -4391,10 +4510,10 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
.rpages = NULL,
.nr_rpages = 0,
.cpages = NULL,
- .valid_nr_cpages = 0,
+ .valid_nr_cblocks = 0,
.rbuf = NULL,
.cbuf = NULL,
- .rlen = PAGE_SIZE * F2FS_I(inode)->i_cluster_size,
+ .rlen = F2FS_BLK_TO_BYTES(sbi, F2FS_I(inode)->i_cluster_size),
.private = NULL,
};
#endif
@@ -4483,6 +4602,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
void *fsdata = NULL;
struct page *pagep;
int ret2;
+ pgoff_t f_lblk = f2fs_folio_lblk(sbi, folio);
ret = f2fs_init_compress_ctx(&cc);
if (ret) {
@@ -4490,8 +4610,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
break;
}
- if (!f2fs_cluster_can_merge_page(&cc,
- folio->index)) {
+ if (!f2fs_cluster_can_merge_block(&cc, f_lblk)) {
ret = f2fs_write_multi_pages(&cc,
&submitted, wbc, io_type);
if (!ret)
@@ -4511,14 +4630,14 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
ret2 = f2fs_prepare_compress_overwrite(
inode, &pagep,
- folio->index, &fsdata);
+ f_lblk, &fsdata);
if (ret2 < 0) {
ret = ret2;
done = 1;
break;
} else if (ret2 &&
(!f2fs_compress_write_end(inode,
- fsdata, folio->index, 1) ||
+ fsdata, f_lblk, 1) ||
!f2fs_all_cluster_page_ready(&cc,
pages, i, nr_pages,
false))) {
@@ -4673,7 +4792,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
clear_inode_flag(inode, FI_HOT_DATA);
while ((folio = writeback_iter(mapping, wbc, folio, &err))) {
- struct f2fs_folio_state *ffs = NULL;
u64 isize;
size_t poff;
pgoff_t end_index;
@@ -4697,16 +4815,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
goto retry_out;
}
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode)) {
- WARN_ON_ONCE(1);
- mapping_set_error(mapping, -EOPNOTSUPP);
- folio_clear_dirty_for_io(folio);
- f2fs_folio_clear_dirty_blocks(folio, true);
- err = -EOPNOTSUPP;
- goto retry_out;
- }
-
/*
* Writing back one large folio performs multiple block allocations, so
* best-effort balance the filesystem before writing it back.
@@ -4821,35 +4929,12 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
* folio has no ffs, so its whole folio is written.
*/
if (f2fs_is_multiblock_folio(sbi, folio)) {
- if (!f2fs_folio_has_ffs(folio)) {
- /*
- * PG_dirty, which was cleared for writeback,
- * covered every block and the folio was
- * accounted as a whole. Account its dirty
- * bits instead.
- */
- WARN_ON_ONCE(folio_test_dirty(folio));
- f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
- f2fs_account_dirty_blocks(inode,
- -(int)f2fs_blocks_per_folio(sbi, folio));
- }
- ffs = (struct f2fs_folio_state *)folio->private;
- /*
- * Multiblock folios can be submitted as multiple bios,
- * which may complete while we are still submitting the
- * remaining dirty ranges. Hold a bias on
- * write_blocks_pending to prevent folio_end_writeback()
- * from being called early.
- */
- if (!bias_added) {
- WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
- atomic_inc(&ffs->write_blocks_pending);
- bias_added = true;
- }
+ f2fs_folio_get_write_bias(folio, end_pos - pos);
+ bias_added = true;
}
err = f2fs_write_folio_dirty_range(folio, &submitted, wbc, io_type,
- pos, end_pos);
+ pos, end_pos, 0);
folio_submitted += submitted;
nwritten += submitted;
if (err)
@@ -5002,7 +5087,8 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (f2fs_has_multiblock_folios(inode))
+ if (mapping_large_folio_support(mapping) ||
+ (f2fs_has_subpage_blocks(sbi) && !f2fs_compressed_file(inode)))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
ret = f2fs_write_cache_pages(mapping, wbc, io_type);
@@ -5340,10 +5426,6 @@ static int prepare_large_folio_write_begin(struct inode *inode,
size_t need_off = ori_off;
int err = 0;
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode))
- return -EOPNOTSUPP;
-
/*
* f2fs_balance_fs_folio_locked() unlocks the folio to balance. Skip it
* for page_mkwrite (!@balance), which cannot unlock the folio and has
@@ -5540,12 +5622,6 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto fail;
}
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode)) {
- err = -EOPNOTSUPP;
- goto fail;
- }
-
/*
* We should check this at this moment to avoid deadlock on inode page
* and #0 page. The locking rule for inline_data conversion should be:
@@ -5566,11 +5642,13 @@ static int f2fs_write_begin(const struct kiocb *iocb,
*fsdata = NULL;
+ pgoff_t f_lblk = F2FS_BYTES_TO_BLK(sbi, pos);
+
if (len == PAGE_SIZE && !(f2fs_is_atomic_file(inode)))
goto repeat;
ret = f2fs_prepare_compress_overwrite(inode, &page,
- index, fsdata);
+ f_lblk, fsdata);
if (ret < 0) {
err = ret;
goto fail;
@@ -5747,7 +5825,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
#ifdef CONFIG_F2FS_FS_COMPRESSION
/* overwrite compressed file */
if (f2fs_compressed_file(inode) && fsdata) {
- f2fs_compress_write_end(inode, fsdata, folio->index, copied);
+ f2fs_compress_write_end(inode, fsdata,
+ f2fs_folio_lblk(F2FS_I_SB(inode), folio), copied);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (pos + copied > i_size_read(inode) &&
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index f84d6f4cd1eb..013df25ccbac 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1696,7 +1696,7 @@ struct compress_ctx {
unsigned int nr_rpages; /* total page number in rpages */
struct page **cpages; /* pages store compressed data in cluster */
unsigned int nr_cpages; /* total page number in cpages */
- unsigned int valid_nr_cpages; /* valid page number in cpages */
+ unsigned int valid_nr_cblocks; /* valid block number in cpages */
void *rbuf; /* virtual mapped address on rpages */
struct compress_data *cbuf; /* virtual mapped address on cpages */
size_t rlen; /* valid data length in rbuf */
@@ -1712,7 +1712,9 @@ struct compress_io_ctx {
struct inode *inode; /* inode the context belong to */
struct page **rpages; /* pages store raw data in cluster */
unsigned int nr_rpages; /* total page number in rpages */
- atomic_t pending_pages; /* in-flight compressed page count */
+ atomic_t pending_blocks; /* in-flight compressed block count */
+ struct page **cpages; /* bounce pages store compressed data */
+ unsigned int nr_cpages; /* total page number in cpages */
};
/* Context for decompressing one cluster on the read IO path */
@@ -1727,6 +1729,7 @@ struct decompress_io_ctx {
unsigned int nr_rpages; /* total page number in rpages */
struct page **cpages; /* pages store compressed data in cluster */
unsigned int nr_cpages; /* total page number in cpages */
+ unsigned int valid_nr_cblocks; /* valid compressed block number */
struct page **tpages; /* temp pages to pad holes in cluster */
void *rbuf; /* virtual mapped address on rpages */
struct compress_data *cbuf; /* virtual mapped address on cpages */
@@ -1734,16 +1737,16 @@ struct decompress_io_ctx {
size_t clen; /* valid data length in cbuf */
/*
- * The number of compressed pages remaining to be read in this cluster.
- * This is initially nr_cpages. It is decremented by 1 each time a page
- * has been read (or failed to be read). When it reaches 0, the cluster
- * is decompressed (or an error is reported).
+ * The number of compressed blocks remaining to be read in this cluster.
+ * This is initially valid_nr_cblocks. It is decremented by the number
+ * of blocks each time they have been read (or failed to be read). When
+ * it reaches 0, the cluster is decompressed (or an error is reported).
*
* If an error occurs before all the pages have been submitted for I/O,
* then this will never reach 0. In this case the I/O submitter is
* responsible for calling f2fs_decompress_end_io() instead.
*/
- atomic_t remaining_pages;
+ atomic_t remaining_blocks;
/*
* Number of references to this decompress_io_ctx.
@@ -1770,7 +1773,7 @@ struct decompress_io_ctx {
#define NULL_CLUSTER ((unsigned int)(~0))
#define MIN_COMPRESS_LOG_SIZE 2
#define MAX_COMPRESS_LOG_SIZE 8
-#define MAX_COMPRESS_WINDOW_SIZE(log_size) ((PAGE_SIZE) << (log_size))
+#define MAX_COMPRESS_WINDOW_SIZE(sbi, log_size) ((F2FS_BLKSIZE(sbi)) << (log_size))
struct f2fs_gc_kthread {
struct task_struct *f2fs_gc_task;
@@ -2338,6 +2341,12 @@ static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
#define F2FS_LOG_BLKS_PER_PAGE(sbi) (PAGE_SHIFT - (sbi)->log_blocksize)
+static inline unsigned int min_compress_log_size(struct f2fs_sb_info *sbi)
+{
+ return max_t(unsigned int, MIN_COMPRESS_LOG_SIZE,
+ F2FS_LOG_BLKS_PER_PAGE(sbi));
+}
+
static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
{
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
@@ -3019,16 +3028,6 @@ static inline void inode_sub_dirty_blocks(struct inode *inode,
sub_cache_counts(sbi, F2FS_DIRTY_QDATA, nr_blocks);
}
-static inline void inode_inc_dirty_block(struct inode *inode)
-{
- inode_add_dirty_blocks(inode, 1);
-}
-
-static inline void inode_dec_dirty_block(struct inode *inode)
-{
- inode_sub_dirty_blocks(inode, 1);
-}
-
static inline void inc_atomic_write_cnt(struct inode *inode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -4994,20 +4993,22 @@ enum cluster_check_type {
bool f2fs_is_compressed_page(struct folio *folio);
struct folio *f2fs_compress_control_folio(struct folio *folio);
int f2fs_prepare_compress_overwrite(struct inode *inode,
- struct page **pagep, pgoff_t index, void **fsdata);
+ struct page **pagep, pgoff_t lblk, void **fsdata);
bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
- pgoff_t index, unsigned copied);
+ pgoff_t lblk, unsigned int copied);
int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock);
-void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio);
+void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio,
+ unsigned int nr_blocks);
bool f2fs_is_compress_backend_ready(struct inode *inode);
bool f2fs_is_compress_level_valid(int alg, int lvl);
int __init f2fs_init_compress_mempool(void);
void f2fs_destroy_compress_mempool(void);
void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task);
-void f2fs_end_read_compressed_page(struct folio *folio, bool failed,
- block_t blkaddr, bool in_task);
+void f2fs_end_read_compressed_page(struct folio *folio, size_t offset,
+ bool failed, block_t blkaddr,
+ unsigned int nr_blocks, bool in_task);
bool f2fs_cluster_is_empty(struct compress_ctx *cc);
-bool f2fs_cluster_can_merge_page(struct compress_ctx *cc, pgoff_t index);
+bool f2fs_cluster_can_merge_block(struct compress_ctx *cc, pgoff_t lblk);
bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
int index, int nr_pages, bool uptodate);
bool f2fs_sanity_check_cluster(struct dnode_of_data *dn);
@@ -5016,13 +5017,13 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type);
-int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index);
-bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index);
+int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t lblk);
+bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t lblk);
void f2fs_update_read_extent_tree_range_compressed(struct inode *inode,
pgoff_t fofs, block_t blkaddr,
unsigned int llen, unsigned int c_len);
int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
- unsigned nr_pages, sector_t *last_block_in_bio,
+ unsigned int nr_blocks, sector_t *last_block_in_bio,
struct readahead_control *rac, bool for_write);
struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc);
void f2fs_decompress_end_io(struct decompress_io_ctx *dic, bool failed,
@@ -5040,8 +5041,8 @@ int __init f2fs_init_compress_cache(void);
void f2fs_destroy_compress_cache(void);
void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
block_t blkaddr, unsigned int len);
-bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
- block_t blkaddr);
+bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi, struct folio *folio,
+ size_t offset, block_t blkaddr);
void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi, nid_t ino);
#define inc_compr_inode_stat(inode) \
do { \
@@ -5075,7 +5076,8 @@ static inline void f2fs_destroy_compress_mempool(void) { }
static inline void f2fs_decompress_cluster(struct decompress_io_ctx *dic,
bool in_task) { }
static inline void f2fs_end_read_compressed_page(struct folio *folio,
- bool failed, block_t blkaddr, bool in_task)
+ size_t offset, bool failed, block_t blkaddr,
+ unsigned int nr_blocks, bool in_task)
{
WARN_ON_ONCE(1);
}
@@ -5093,17 +5095,18 @@ static inline int __init f2fs_init_compress_cache(void) { return 0; }
static inline void f2fs_destroy_compress_cache(void) { }
static inline void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
block_t blkaddr, unsigned int len) { }
-static inline bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi,
- struct folio *folio, block_t blkaddr) { return false; }
+static inline bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset,
+ block_t blkaddr) { return false; }
static inline void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi,
nid_t ino) { }
#define inc_compr_inode_stat(inode) do { } while (0)
static inline int f2fs_is_compressed_cluster(
struct inode *inode,
- pgoff_t index) { return 0; }
+ pgoff_t lblk) { return 0; }
static inline bool f2fs_is_sparse_cluster(
struct inode *inode,
- pgoff_t index) { return true; }
+ pgoff_t lblk) { return true; }
static inline void f2fs_update_read_extent_tree_range_compressed(
struct inode *inode,
pgoff_t fofs, block_t blkaddr,
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 64c91827a5f4..c80dbbe76f8f 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -179,7 +179,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_compressed_file(inode)) {
- int ret = f2fs_is_compressed_cluster(inode, pidx);
+ int ret = f2fs_is_compressed_cluster(inode, bidx);
if (ret < 0) {
err = ret;
@@ -1032,7 +1032,8 @@ int f2fs_truncate_blocks(struct inode *inode, u64 from, bool lock)
*/
if (f2fs_compressed_file(inode))
free_from = round_up(from,
- F2FS_I(inode)->i_cluster_size << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode),
+ F2FS_I(inode)->i_cluster_size));
#endif
err = f2fs_do_truncate_blocks(inode, free_from, lock);
@@ -2313,13 +2314,6 @@ static long f2fs_fallocate(struct file *file, int mode,
goto out;
}
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if (f2fs_compressed_file(inode)) {
- ret = -EOPNOTSUPP;
- goto out;
- }
- }
-
ret = file_modified(file);
if (ret)
goto out;
@@ -4522,9 +4516,6 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
int ret;
int writecount;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4720,9 +4711,6 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
unsigned int reserved_blocks = 0;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -5044,9 +5032,6 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
struct f2fs_comp_option option;
int ret = 0;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -5057,7 +5042,7 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
sizeof(option)))
return -EFAULT;
- if (option.log_cluster_size < MIN_COMPRESS_LOG_SIZE ||
+ if (option.log_cluster_size < min_compress_log_size(sbi) ||
option.log_cluster_size > MAX_COMPRESS_LOG_SIZE ||
option.algorithm >= COMPRESS_MAX)
return -EINVAL;
@@ -5151,15 +5136,15 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
off = offset_in_folio(folio, (loff_t)redirty_idx << PAGE_SHIFT);
dlen = (size_t)(next_idx - redirty_idx) << PAGE_SHIFT;
- if (folio_test_large(folio)) {
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
/*
- * A compressed inode can keep using large folios, in
+ * f2fs_folio_mark_dirty() does not dirty the blocks of a folio
+ * state. A compressed inode can keep using large folios, in
* which case the compression flag only works as a hint
* until the inode is evicted. Update the subpage dirty
* bitmap as well, otherwise f2fs_write_cache_folios()
* finds no dirty range and skips the whole folio.
*/
- f2fs_ffs_find_or_alloc(folio);
f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
}
@@ -5178,12 +5163,9 @@ static int f2fs_ioc_decompress_file(struct file *filp)
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode_info *fi = F2FS_I(inode);
- pgoff_t page_idx = 0, last_idx, cluster_idx;
+ pgoff_t lblk = 0, last_idx, cluster_idx;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5220,12 +5202,14 @@ static int f2fs_ioc_decompress_file(struct file *filp)
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
- page_idx = cluster_idx << fi->i_log_cluster_size;
+ lblk = cluster_idx << fi->i_log_cluster_size;
- if (!f2fs_is_compressed_cluster(inode, page_idx))
+ if (!f2fs_is_compressed_cluster(inode, lblk))
continue;
- ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
+ ret = redirty_blocks(inode,
+ f2fs_lblk_to_folio_index(sbi, lblk),
+ fi->i_cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi));
if (ret < 0)
break;
@@ -5262,12 +5246,9 @@ static int f2fs_ioc_compress_file(struct file *filp)
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode_info *fi = F2FS_I(inode);
- pgoff_t page_idx = 0, last_idx, cluster_idx;
+ pgoff_t lblk = 0, last_idx, cluster_idx;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5310,12 +5291,14 @@ static int f2fs_ioc_compress_file(struct file *filp)
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
- page_idx = cluster_idx << fi->i_log_cluster_size;
+ lblk = cluster_idx << fi->i_log_cluster_size;
- if (f2fs_is_sparse_cluster(inode, page_idx))
+ if (f2fs_is_sparse_cluster(inode, lblk))
continue;
- ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
+ ret = redirty_blocks(inode,
+ f2fs_lblk_to_folio_index(sbi, lblk),
+ fi->i_cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi));
if (ret < 0)
break;
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 9b01ead00edc..ac024c089710 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1447,6 +1447,11 @@ static int f2fs_check_compression(struct fs_context *fc,
ctx_clear_opt_mask(ctx, F2FS_MOUNT_COMPRESS_CACHE);
return 0;
}
+ if ((ctx->spec_mask & F2FS_SPEC_compress_log_size) &&
+ F2FS_CTX_INFO(ctx).compress_log_size < min_compress_log_size(sbi)) {
+ f2fs_err(sbi, "Compress cluster log size is out of range");
+ return -EINVAL;
+ }
if (ctx->spec_mask & F2FS_SPEC_compress_extension) {
cnt = F2FS_CTX_INFO(ctx).compress_ext_cnt;
for (i = 0; i < F2FS_CTX_INFO(ctx).compress_ext_cnt; i++) {
@@ -2630,7 +2635,7 @@ static void default_options(struct f2fs_sb_info *sbi, bool remount)
F2FS_OPTION(sbi).s_resgid = make_kgid(&init_user_ns, F2FS_DEF_RESGID);
if (f2fs_sb_has_compression(sbi)) {
F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
- F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
+ F2FS_OPTION(sbi).compress_log_size = min_compress_log_size(sbi);
F2FS_OPTION(sbi).compress_ext_cnt = 0;
F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
}
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 19/20] f2fs: count compress cache entries in pages for its memory limit
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (17 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 18/20] f2fs: support subpage blocks in compression Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 20/20] f2fs: enable subpage block size support Daeho Jeong
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The compress cache holds one filesystem block per entry, but
f2fs_available_free_memory() compares the number of entries with a
percentage of free memory counted in pages. With subpage blocks, this
limits the compress cache to a fraction (block size / PAGE_SIZE) of
the memory that compress_percent allows. Convert the entry count to
pages before the comparison.
No functional change when the block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/node.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index f1eae132c4d3..c83a5d79f2f3 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -116,10 +116,12 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
/*
* free memory is lower than watermark or cached page count
- * exceed threshold, deny caching compress page.
+ * exceed threshold, deny caching compress page. A cache
+ * entry holds one block, so convert the entry count to pages.
*/
res = (free_ram > avail_ram * sbi->compress_watermark / 100) &&
- (COMPRESS_CACHE(sbi)->num_entries <
+ ((COMPRESS_CACHE(sbi)->num_entries >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi)) <
free_ram * sbi->compress_percent / 100);
#else
res = false;
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 20/20] f2fs: enable subpage block size support
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (18 preceding siblings ...)
2026-11-16 4:08 ` [PATCH v2 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
@ 2026-11-16 4:08 ` Daeho Jeong
19 siblings, 0 replies; 21+ messages in thread
From: Daeho Jeong @ 2026-11-16 4:08 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
All paths now handle filesystem blocks smaller than PAGE_SIZE, so stop
rejecting a superblock whose log_blocksize is below PAGE_SHIFT.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/super.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index ac024c089710..2f9ef2abfba8 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4173,13 +4173,6 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
return -EFSCORRUPTED;
}
- /* Subpage block size is enabled after all paths support it. */
- if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
- f2fs_info(sbi, "Subpage block size (%u) is not supported yet",
- le32_to_cpu(raw_super->log_blocksize));
- return -EOPNOTSUPP;
- }
-
/* check log blocks per segment */
if (le32_to_cpu(raw_super->log_blocks_per_seg) != 9) {
f2fs_info(sbi, "Invalid log blocks per segment (%u)",
--
2.56.0.611.g1030f0b940-goog
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2026-11-16 4:09 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 4:08 [PATCH v2 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 01/20] f2fs: use block-based index and size calculations in more paths Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 11/20] f2fs: support subpage blocks in move range Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 18/20] f2fs: support subpage blocks in compression Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
2026-11-16 4:08 ` [PATCH v2 20/20] f2fs: enable subpage block size support Daeho Jeong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®