* [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE
@ 2026-11-02 19:39 Daeho Jeong
2026-11-02 19:39 ` [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths Daeho Jeong
` (19 more replies)
0 siblings, 20 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
F2FS has required the filesystem block size to equal PAGE_SIZE. On a
kernel with 16KB pages, this means an existing F2FS image with 4KB
blocks cannot be mounted, e.g. the userdata partition of an Android
device that moves from a 4KB page kernel to a 16KB page kernel.
This series lets F2FS mount and use a filesystem whose block size is
smaller than PAGE_SIZE, such as 4KB blocks on a 16KB page kernel. The
on-disk format does not change.
With subpage blocks, even an order-0 folio holds several blocks, so
the series builds on the large folio support from Nanzhe Zhao [1] and
tracks uptodate and dirty state per block in f2fs_folio_state for any
folio that holds more than one block. Most paths then work in blocks
instead of pages:
- Patches 1-2 convert the remaining page based index and size
calculations to block based ones, and add the superblock and mount
infrastructure. Subpage block sizes are still rejected at this
point.
- Patch 3 extends f2fs_folio_state with per-block state, identified by
a magic number, and allocates it on demand with __GFP_NOFAIL.
- Patches 4-7 handle directories, the per-inode dirty counters, the
read path and GC, and the write path and writeback per block.
- Patches 8-19 adapt the remaining features: range operations and
mmap, pinned files, collapse/insert range, move range,
defragmentation, swapfile, fs-verity, atomic writes, inline data,
online resize and compression, and remove the -EOPNOTSUPP checks
that patch 2 added for the features that need them.
- Patch 20 stops rejecting subpage block sizes.
When the block size equals PAGE_SIZE, the behavior does not change,
except for the fixes noted in the individual patches, some of which
also apply to large folios.
A few of the patches adjust code from the large folio series [1]:
- f2fs_folio_has_ffs() checks the folio state magic instead of
requiring a large folio, since order-0 multi-block folios also carry
a folio state, and the page private flags can legitimately be set
before a state is attached.
- f2fs_ffs_find_or_alloc() cannot fail with __GFP_NOFAIL, so the NULL
checks of its result in f2fs_get_lock_data_folio() and
redirty_blocks() are removed.
- Defragmentation and swapfile activation mark only the target block
dirty instead of the whole page.
- redirty_blocks() does not dirty the blocks past EOF.
Testing:
Built every patch for x86_64 and arm64 (16KB pages), and the whole
series for arm (32-bit).
Ran the xfstests auto group (excluding the clone, dedupe and thin
groups) on arm64 QEMU with mount options "discard,fsync_mode=strict"
in these combinations of page size / block size:
- 16KB/4KB: default, compression (lz4, zstd, lz4hc, lzo-rle with
64KB clusters and checksum), compression+encryption,
encryption+verity, mode=lfs, atgc
- 4KB/4KB: default, compression, encryption+verity,
compression+encryption, mode=lfs
- 16KB/16KB: default, compression, encryption+verity,
compression+encryption, mode=lfs
plus generic/019, 521 and 522 for each combination, and a 24 hour
power-off recovery test with fsstress for each of the three
combinations.
There were no crashes or hangs, and the power-off recovery tests found
no corruption.
[1] https://lore.kernel.org/linux-f2fs-devel/20261028154407.1601430-1-nzzhao.sigma@gmail.com/
Daeho Jeong (20):
f2fs: use block-based index and size calculations in remaining paths
f2fs: support subpage block size superblock and mount
f2fs: extend f2fs_folio_state for subpage blocks
f2fs: support subpage blocks in directory operations
f2fs: convert inode dirty page counters to dirty block counters
f2fs: support subpage blocks in data read path and GC
f2fs: support subpage blocks in data write path and writeback
f2fs: adapt range operations and mmap for subpage blocks
f2fs: support subpage blocks in pinned files
f2fs: support subpage blocks in collapse and insert range
f2fs: support subpage blocks in move range
f2fs: support subpage blocks in defragmentation
f2fs: support subpage blocks in swapfile activation
f2fs: support subpage blocks in fs-verity enablement
f2fs: support subpage blocks in atomic writes
f2fs: preserve inline data in subpage block mode
f2fs: support subpage blocks in online resize
f2fs: support subpage blocks in compression
f2fs: count compress cache entries in pages for its memory limit
f2fs: enable subpage block size support
fs/f2fs/checkpoint.c | 41 +-
fs/f2fs/compress.c | 518 +++++++++----
fs/f2fs/data.c | 1552 +++++++++++++++++++++++----------------
fs/f2fs/dir.c | 124 +++-
fs/f2fs/f2fs.h | 225 ++++--
fs/f2fs/file.c | 553 ++++++++------
fs/f2fs/gc.c | 89 ++-
fs/f2fs/inline.c | 13 +-
fs/f2fs/inode.c | 4 +-
fs/f2fs/node.c | 17 +-
fs/f2fs/segment.c | 7 +-
fs/f2fs/super.c | 162 ++--
fs/f2fs/verity.c | 5 -
include/linux/f2fs_fs.h | 1 +
14 files changed, 2078 insertions(+), 1233 deletions(-)
base-commit: 4c5e07aaeeb8a27d769a7e4e13d0b26aea0cb108
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-03 7:04 ` [f2fs-dev] " Chao Yu
2026-11-02 19:39 ` [PATCH 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
` (18 subsequent siblings)
19 siblings, 1 reply; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Several paths still convert between file offsets and block indices with
PAGE_SHIFT or PAGE_SIZE, assuming that a page holds exactly one
filesystem block. Use the F2FS_BLK_* helpers there so that they stay
correct when the block size is smaller than PAGE_SIZE.
No functional change when the block size equals PAGE_SIZE, except that
__complete_revoke_list() now computes the truncation offset in 64 bits
and no longer overflows beyond 4GB on 32-bit architectures.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 5 +++--
fs/f2fs/dir.c | 4 ++--
fs/f2fs/file.c | 48 +++++++++++++++++++++++------------------------
fs/f2fs/segment.c | 5 +++--
4 files changed, 32 insertions(+), 30 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 4ae61138a003..aacc92839f61 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1897,8 +1897,9 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
}
got_it:
if (new_i_size && i_size_read(inode) <
- ((loff_t)(index + 1) << PAGE_SHIFT))
- f2fs_i_size_write(inode, ((loff_t)(index + 1) << PAGE_SHIFT));
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1))
+ f2fs_i_size_write(inode,
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1));
return folio;
}
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 70847313ca6a..3eb0b7255951 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -39,8 +39,8 @@ extern struct kmem_cache *f2fs_cf_name_slab;
static unsigned long dir_blocks(struct inode *inode)
{
- return ((unsigned long long) (i_size_read(inode) + PAGE_SIZE - 1))
- >> PAGE_SHIFT;
+ return F2FS_BYTES_TO_BLK(F2FS_I_SB(inode),
+ i_size_read(inode) + F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)));
}
static unsigned int dir_buckets(unsigned int level, int dir_level)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 05061254c3b2..93d3088c96f0 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1576,7 +1576,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
}
dn.ofs_in_node++;
i++;
- new_size = (loff_t)(dst + i) << PAGE_SHIFT;
+ new_size = F2FS_BLK_TO_BYTES(sbi, dst + i);
if (dst_inode->i_size < new_size)
f2fs_i_size_write(dst_inode, new_size);
} while (--ilen && (do_replace[i] || blkaddr[i] == NULL_ADDR));
@@ -1593,7 +1593,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
return PTR_ERR(fsrc);
src_off = offset_in_folio(fsrc,
- (loff_t)(src + i) << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, src + i));
/*
* For same-inode operations (collapse/insert), src and
@@ -1623,10 +1623,10 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
}
dst_off = offset_in_folio(fdst,
- (loff_t)(dst + i) << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, dst + i));
f2fs_folio_wait_writeback(fdst, true, true);
memcpy_folio(fdst, dst_off, fsrc, src_off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
if (folio_test_large(fdst)) {
if (!f2fs_ffs_find_or_alloc(fdst)) {
f2fs_folio_put(fdst, true);
@@ -1635,15 +1635,15 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
return -ENOMEM;
}
f2fs_ffs_mark_subrange_uptodate(fdst,
- dst_off, PAGE_SIZE);
+ dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
}
folio_mark_dirty(fdst);
if (same_folio && i_size_read(dst_inode) <
- ((loff_t)(dst + i + 1) << PAGE_SHIFT))
+ F2FS_BLK_TO_BYTES(sbi, dst + i + 1))
f2fs_i_size_write(dst_inode,
- ((loff_t)(dst + i + 1) << PAGE_SHIFT));
+ F2FS_BLK_TO_BYTES(sbi, dst + i + 1));
folio_set_f2fs_gcing(fdst);
f2fs_folio_put(fdst, true);
if (!same_folio)
@@ -1715,9 +1715,9 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_lock_context lc;
- pgoff_t nrpages = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
- pgoff_t start = offset >> PAGE_SHIFT;
- pgoff_t end = (offset + len) >> PAGE_SHIFT;
+ pgoff_t nrpages = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t start = F2FS_BYTES_TO_BLK(sbi, offset);
+ pgoff_t end = F2FS_BYTES_TO_BLK(sbi, offset + len);
int ret;
f2fs_balance_fs(sbi, true);
@@ -1988,10 +1988,10 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- pg_start = offset >> PAGE_SHIFT;
- pg_end = (offset + len) >> PAGE_SHIFT;
+ pg_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ pg_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
delta = pg_end - pg_start;
- idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
/* avoid gc operation during block exchange */
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
@@ -3104,10 +3104,10 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
f2fs_balance_fs(sbi, true);
inode_lock(inode);
- pg_start = range->start >> PAGE_SHIFT;
+ pg_start = F2FS_BYTES_TO_BLK(sbi, range->start);
pg_end = min_t(pgoff_t,
- (range->start + range->len) >> PAGE_SHIFT,
- DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE));
+ F2FS_BYTES_TO_BLK(sbi, range->start + range->len),
+ F2FS_BLK_ALIGN(sbi, i_size_read(inode)));
if (is_inode_flag_set(inode, FI_COMPRESS_RELEASED) ||
f2fs_is_atomic_file(inode)) {
@@ -3124,8 +3124,8 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
/* writeback all dirty pages in the range */
err = filemap_write_and_wait_range(inode->i_mapping,
- (loff_t)pg_start << PAGE_SHIFT,
- ((loff_t)pg_end << PAGE_SHIFT) - 1);
+ F2FS_BLK_TO_BYTES(sbi, pg_start),
+ F2FS_BLK_TO_BYTES(sbi, pg_end) - 1);
if (err)
goto out;
@@ -3249,7 +3249,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
unlock_out:
inode_unlock(inode);
if (!err) {
- range->len = (u64)total << PAGE_SHIFT;
+ range->len = F2FS_BLK_TO_BYTES(sbi, total);
stat_inc_defrag_blk_count(sbi, total);
}
return err;
@@ -4354,7 +4354,7 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
while (page_idx < last_idx) {
struct dnode_of_data dn;
@@ -4532,7 +4532,7 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
f2fs_down_write(&fi->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
while (page_idx < last_idx) {
struct dnode_of_data dn;
@@ -4997,7 +4997,7 @@ static int f2fs_ioc_decompress_file(struct file *filp)
if (!atomic_read(&fi->i_compr_blocks))
goto out;
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
@@ -5084,7 +5084,7 @@ static int f2fs_ioc_compress_file(struct file *filp)
set_inode_flag(inode, FI_ENABLE_COMPRESS);
- last_idx = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ last_idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 31ac93a93021..2b0821c2c18b 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -285,6 +285,7 @@ static int __replace_atomic_write_block(struct inode *inode, pgoff_t index,
static void __complete_revoke_list(struct inode *inode, struct list_head *head,
bool revoke)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct revoke_entry *cur, *tmp;
pgoff_t start_index = 0;
bool truncate = is_inode_flag_set(inode, FI_ATOMIC_REPLACE);
@@ -303,7 +304,7 @@ static void __complete_revoke_list(struct inode *inode, struct list_head *head,
}
if (!revoke && truncate)
- f2fs_do_truncate_blocks(inode, start_index * PAGE_SIZE, false);
+ f2fs_do_truncate_blocks(inode, F2FS_BLK_TO_BYTES(sbi, start_index), false);
}
static int __f2fs_commit_atomic_write(struct inode *inode)
@@ -315,7 +316,7 @@ static int __f2fs_commit_atomic_write(struct inode *inode)
struct list_head revoke_list;
block_t blkaddr;
struct dnode_of_data dn;
- pgoff_t len = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
+ pgoff_t len = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
pgoff_t off = 0, blen, index;
int ret = 0, i;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 02/20] f2fs: support subpage block size superblock and mount
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-02 19:39 ` [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
` (17 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Add the infrastructure to mount F2FS with a block size smaller than
PAGE_SIZE, e.g. 4KB blocks on a 16KB page kernel.
- Add helpers to convert between block indices, folio indices and
offsets within a folio.
- Take the block size from the 1st superblock, which is at the same
offset for every block size. If it is invalid, probe the 2nd
superblock with each supported block size.
- Allow log_blocksize up to PAGE_SHIFT, and write back only the target
superblock block instead of the whole folio.
- Keep rejecting subpage block sizes, and return -EOPNOTSUPP for the
features that do not support them yet. Later patches add the support
and lift these restrictions.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/compress.c | 2 +
fs/f2fs/f2fs.h | 61 ++++++++++++---
fs/f2fs/file.c | 42 +++++++++++
fs/f2fs/super.c | 160 ++++++++++++++++++++++++++++------------
fs/f2fs/verity.c | 3 +
include/linux/f2fs_fs.h | 1 +
6 files changed, 212 insertions(+), 57 deletions(-)
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index c7910688c318..0b0f31af6626 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -550,6 +550,8 @@ bool f2fs_is_compress_backend_ready(struct inode *inode)
{
if (!f2fs_compressed_file(inode))
return true;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return false;
return f2fs_cops[F2FS_I(inode)->i_compress_algorithm];
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 50a5f845176d..11de12a3e804 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1372,11 +1372,11 @@ struct f2fs_io_info {
};
#define F2FS_FIO_LBLK(fio) \
- (((fio)->folio)->index + (fio)->folio_offset)
+ (f2fs_folio_lblk((fio)->sbi, (fio)->folio) + (fio)->folio_offset)
#define F2FS_FIO_BLKCNT(fio) \
((fio)->folio_blkcnt)
#define F2FS_FIO_BIO_OFFSET(fio) \
- ((fio)->folio_offset << PAGE_SHIFT)
+ F2FS_BLK_TO_BYTES((fio)->sbi, (fio)->folio_offset)
#define F2FS_FIO_BIO_SIZE(fio) \
F2FS_BLK_TO_BYTES((fio)->sbi, F2FS_FIO_BLKCNT(fio))
@@ -2326,19 +2326,59 @@ static inline void make_dentry_ptr_block(struct inode *inode,
SIZE_OF_DIR_ENTRY * entries;
}
+static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return folio_size(folio) >> sbi->log_blocksize;
+}
+
+static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
+{
+ return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
+}
+
+static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return ((loff_t)folio->index << PAGE_SHIFT) >> sbi->log_blocksize;
+}
+
+static inline pgoff_t f2fs_lblk_to_folio_index(struct f2fs_sb_info *sbi,
+ pgoff_t index)
+{
+ return F2FS_BLK_TO_BYTES(sbi, index) >> PAGE_SHIFT;
+}
+
+static inline pgoff_t f2fs_folio_blk_offset(struct f2fs_sb_info *sbi,
+ const struct folio *folio,
+ pgoff_t index)
+{
+ return index - f2fs_folio_lblk(sbi, folio);
+}
+
+static inline size_t f2fs_folio_byte_offset(struct f2fs_sb_info *sbi,
+ const struct folio *folio,
+ pgoff_t index)
+{
+ return F2FS_BLK_TO_BYTES(sbi, f2fs_folio_blk_offset(sbi, folio, index));
+}
+
+static inline void *f2fs_folio_lblk_address(struct f2fs_sb_info *sbi,
+ struct folio *folio, pgoff_t index)
+{
+ return folio_address(folio) + f2fs_folio_byte_offset(sbi, folio, index);
+}
+
static inline struct f2fs_super_block *F2FS_RAW_SUPER(struct f2fs_sb_info *sbi)
{
return (struct f2fs_super_block *)(sbi->raw_super);
}
-static inline struct f2fs_super_block *F2FS_SUPER_BLOCK(struct folio *folio,
- pgoff_t index)
+static inline struct f2fs_super_block *
+F2FS_SUPER_BLOCK(struct f2fs_sb_info *sbi, struct folio *folio, pgoff_t index)
{
- pgoff_t idx_in_folio = index % folio_nr_pages(folio);
-
return (struct f2fs_super_block *)
- (page_address(folio_page(folio, idx_in_folio)) +
- F2FS_SUPER_OFFSET);
+ (f2fs_folio_lblk_address(sbi, folio, index) + F2FS_SUPER_OFFSET);
}
static inline struct f2fs_checkpoint *F2FS_CKPT(struct f2fs_sb_info *sbi)
@@ -3905,11 +3945,10 @@ static inline bool f2fs_is_time_consistent(struct inode *inode)
static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
bool ret;
if (dsync) {
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
-
spin_lock(&sbi->inode_lock[DIRTY_META]);
ret = list_empty(&F2FS_I(inode)->gdirty_list);
spin_unlock(&sbi->inode_lock[DIRTY_META]);
@@ -3917,7 +3956,7 @@ static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
}
if (!is_inode_flag_set(inode, FI_AUTO_RECOVER) ||
file_keep_isize(inode) ||
- i_size_read(inode) & ~PAGE_MASK)
+ i_size_read(inode) & (F2FS_BLKSIZE(sbi) - 1))
return false;
if (!f2fs_is_time_consistent(inode))
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 93d3088c96f0..82eb62301c48 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2207,6 +2207,15 @@ static long f2fs_fallocate(struct file *file, int mode,
goto out;
}
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
+ if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
+ f2fs_is_pinned_file(inode) || f2fs_is_atomic_file(inode) ||
+ f2fs_compressed_file(inode)) {
+ ret = -EOPNOTSUPP;
+ goto out;
+ }
+ }
+
ret = file_modified(file);
if (ret)
goto out;
@@ -2455,6 +2464,9 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
loff_t isize;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!(filp->f_mode & FMODE_WRITE))
return -EBADF;
@@ -3262,6 +3274,9 @@ static int f2fs_ioc_defragment(struct file *filp, unsigned long arg)
struct f2fs_defragment range;
int err;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -3484,6 +3499,9 @@ static int f2fs_ioc_move_range(struct file *filp, unsigned long arg)
{
struct f2fs_move_range range;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(file_inode(filp))))
+ return -EOPNOTSUPP;
+
if (copy_from_user(&range, (struct f2fs_move_range __user *)arg,
sizeof(range)))
return -EFAULT;
@@ -3734,6 +3752,9 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
__u32 pin;
int ret = 0;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (get_user(pin, (__u32 __user *)arg))
return -EFAULT;
@@ -4114,6 +4135,9 @@ static int f2fs_ioc_resize_fs(struct file *filp, unsigned long arg)
struct f2fs_sb_info *sbi = F2FS_I_SB(file_inode(filp));
__u64 block_count;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
@@ -4131,6 +4155,9 @@ static int f2fs_ioc_enable_verity(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return -EOPNOTSUPP;
+
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (!f2fs_sb_has_verity(F2FS_I_SB(inode))) {
@@ -4311,6 +4338,9 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
int ret;
int writecount;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4506,6 +4536,9 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
unsigned int reserved_blocks = 0;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4827,6 +4860,9 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
struct f2fs_comp_option option;
int ret = 0;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4965,6 +5001,9 @@ static int f2fs_ioc_decompress_file(struct file *filp)
pgoff_t page_idx = 0, last_idx, cluster_idx;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5046,6 +5085,9 @@ static int f2fs_ioc_compress_file(struct file *filp)
pgoff_t page_idx = 0, last_idx, cluster_idx;
int ret;
+ if (f2fs_has_subpage_blocks(sbi))
+ return -EOPNOTSUPP;
+
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 78dbf1e92285..f34558b661eb 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4000,7 +4000,7 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
folio_lock(folio);
folio_wait_writeback(folio);
if (update)
- memcpy(F2FS_SUPER_BLOCK(folio, index), F2FS_RAW_SUPER(sbi),
+ memcpy(F2FS_SUPER_BLOCK(sbi, folio, index), F2FS_RAW_SUPER(sbi),
sizeof(struct f2fs_super_block));
folio_mark_dirty(folio);
folio_clear_dirty_for_io(folio);
@@ -4010,9 +4010,10 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
bio = bio_alloc(sbi->sb->s_bdev, 1, opf, GFP_NOFS);
/* it doesn't need to set crypto context for superblock update */
- bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(sbi, folio->index);
+ bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(sbi, index);
- if (!bio_add_folio(bio, folio, folio_size(folio), 0))
+ if (!bio_add_folio(bio, folio, F2FS_BLKSIZE(sbi),
+ f2fs_folio_byte_offset(sbi, folio, index)))
f2fs_bug_on(sbi, 1);
ret = submit_bio_wait(bio);
@@ -4025,7 +4026,7 @@ static int __f2fs_commit_super(struct f2fs_sb_info *sbi, struct folio *folio,
static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
struct folio *folio, pgoff_t index)
{
- struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
+ struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(sbi, folio, index);
struct super_block *sb = sbi->sb;
u32 segment0_blkaddr = le32_to_cpu(raw_super->segment0_blkaddr);
u32 cp_blkaddr = le32_to_cpu(raw_super->cp_blkaddr);
@@ -4107,11 +4108,13 @@ static inline bool sanity_check_area_boundary(struct f2fs_sb_info *sbi,
res = "internally";
} else {
/*
- * __f2fs_commit_super() will access log_blocksize
- * in SECTOR_FROM_BLOCK(), init it in advance.
+ * __f2fs_commit_super() will access log_blocksize and
+ * blocksize to locate the superblock, init them in
+ * advance.
*/
sbi->log_blocksize =
le32_to_cpu(raw_super->log_blocksize);
+ sbi->blocksize = BIT(sbi->log_blocksize);
err = __f2fs_commit_super(sbi, folio, index, false);
res = err ? "failed" : "done";
}
@@ -4129,7 +4132,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
{
block_t segment_count, segs_per_sec, secs_per_zone, segment_count_main;
block_t total_sections, blocks_per_seg;
- struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(folio, index);
+ struct f2fs_super_block *raw_super = F2FS_SUPER_BLOCK(sbi, folio, index);
size_t crc_offset = 0;
__u32 crc = 0;
@@ -4155,14 +4158,23 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
}
}
- /* only support block_size equals to PAGE_SIZE */
- if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
- f2fs_info(sbi, "Invalid log_blocksize (%u), supports only %u",
+ /* Keep one or more filesystem blocks in each base-page folio. */
+ if (le32_to_cpu(raw_super->log_blocksize) <
+ F2FS_MIN_LOG_BLOCKSIZE ||
+ le32_to_cpu(raw_super->log_blocksize) > F2FS_MAX_LOG_BLOCKSIZE) {
+ f2fs_info(sbi, "Invalid log_blocksize (%u), supports %u..%u",
le32_to_cpu(raw_super->log_blocksize),
- PAGE_SHIFT);
+ F2FS_MIN_LOG_BLOCKSIZE, F2FS_MAX_LOG_BLOCKSIZE);
return -EFSCORRUPTED;
}
+ /* Subpage block size is enabled after all paths support it. */
+ if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
+ f2fs_info(sbi, "Subpage block size (%u) is not supported yet",
+ le32_to_cpu(raw_super->log_blocksize));
+ return -EOPNOTSUPP;
+ }
+
/* check log blocks per segment */
if (le32_to_cpu(raw_super->log_blocks_per_seg) != 9) {
f2fs_info(sbi, "Invalid log blocks per segment (%u)",
@@ -4181,7 +4193,7 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
}
if (le32_to_cpu(raw_super->log_sectors_per_block) +
le32_to_cpu(raw_super->log_sectorsize) !=
- F2FS_MAX_LOG_SECTOR_SIZE) {
+ le32_to_cpu(raw_super->log_blocksize)) {
f2fs_info(sbi, "Invalid log sectors per block(%u) log sectorsize(%u)",
le32_to_cpu(raw_super->log_sectors_per_block),
le32_to_cpu(raw_super->log_sectorsize));
@@ -4703,56 +4715,103 @@ static int init_blkz_info(struct f2fs_sb_info *sbi, int devi)
* to get the first valid one. If any one of them is broken, we pass
* them recovery flag back to the caller.
*/
+static int check_super_block(struct f2fs_sb_info *sbi,
+ struct f2fs_super_block *super,
+ int block, int *valid_super_block,
+ struct f2fs_super_block **raw_super)
+{
+ struct folio *folio;
+ int err;
+
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, block), NULL);
+ if (IS_ERR(folio))
+ return PTR_ERR(folio);
+
+ /* Skip unmatched magic during probing to avoid noisy logs */
+ if (block != 0 && !*raw_super &&
+ le32_to_cpu(F2FS_SUPER_BLOCK(sbi, folio, block)->magic) !=
+ F2FS_SUPER_MAGIC) {
+ folio_put(folio);
+ return -EINVAL;
+ }
+
+ /*
+ * The 2nd superblock was located with sbi->log_blocksize. Check it
+ * before sanity_check_raw_super(), which may write the superblock
+ * back with its own log_blocksize.
+ */
+ err = 0;
+ if (block) {
+ u32 log_bs = le32_to_cpu(F2FS_SUPER_BLOCK(sbi, folio,
+ block)->log_blocksize);
+
+ if (log_bs != sbi->log_blocksize) {
+ f2fs_info(sbi, "Mismatched log_blocksize (%u) in 2nd superblock, expected %u",
+ log_bs, sbi->log_blocksize);
+ err = -EFSCORRUPTED;
+ }
+ }
+ if (!err)
+ err = sanity_check_raw_super(sbi, folio, block);
+ if (!err && !*raw_super) {
+ memcpy(super, F2FS_SUPER_BLOCK(sbi, folio, block),
+ sizeof(*super));
+ *valid_super_block = block;
+ *raw_super = super;
+ sbi->log_blocksize = le32_to_cpu(super->log_blocksize);
+ sbi->blocksize = BIT(sbi->log_blocksize);
+ }
+ folio_put(folio);
+ return err;
+}
+
static int read_raw_super_block(struct f2fs_sb_info *sbi,
struct f2fs_super_block **raw_super,
int *valid_super_block, int *recovery)
{
- struct super_block *sb = sbi->sb;
- int block;
- struct folio *folio;
struct f2fs_super_block *super;
- int err = 0;
+ int log_bs, err;
super = kzalloc_obj(struct f2fs_super_block);
if (!super)
return -ENOMEM;
- for (block = 0; block < 2; block++) {
- folio = read_mapping_folio(sb->s_bdev->bd_mapping, block, NULL);
- if (IS_ERR(folio)) {
- f2fs_err(sbi, "Unable to read %dth superblock",
- block + 1);
- err = PTR_ERR(folio);
- *recovery = 1;
- continue;
- }
+ /* 1. Read 1st superblock (block 0) */
+ err = check_super_block(sbi, super, 0, valid_super_block, raw_super);
+ if (err) {
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 1st superblock");
+ *recovery = 1;
+ }
- /* sanity checking of raw super */
- err = sanity_check_raw_super(sbi, folio, block);
- if (err) {
- f2fs_err(sbi, "Can't find valid F2FS filesystem in %dth superblock",
- block + 1);
- folio_put(folio);
+ /* 2. Read 2nd superblock (block 1) */
+ if (*raw_super) {
+ if (check_super_block(sbi, super, 1, valid_super_block, raw_super)) {
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 2nd superblock");
*recovery = 1;
- continue;
}
-
+ } else {
+ for (log_bs = F2FS_MIN_LOG_BLOCKSIZE;
+ log_bs <= F2FS_MAX_LOG_BLOCKSIZE; log_bs++) {
+ sbi->log_blocksize = log_bs;
+ sbi->blocksize = BIT(log_bs);
+ err = check_super_block(sbi, super, 1, valid_super_block,
+ raw_super);
+ if (!err)
+ break;
+ }
if (!*raw_super) {
- memcpy(super, F2FS_SUPER_BLOCK(folio, block),
- sizeof(*super));
- *valid_super_block = block;
- *raw_super = super;
+ f2fs_err(sbi, "Can't find valid F2FS filesystem in 2nd superblock");
+ *recovery = 1;
}
- folio_put(folio);
}
- /* No valid superblock */
- if (!*raw_super)
+ /* No valid superblock: return the error of the last probe. */
+ if (!*raw_super) {
kfree(super);
- else
- err = 0;
-
- return err;
+ return err;
+ }
+ return 0;
}
int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
@@ -4777,7 +4836,8 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
/* write back-up superblock first */
index = sbi->valid_super_block ? 0 : 1;
- folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, index), NULL);
if (IS_ERR(folio))
return PTR_ERR(folio);
err = __f2fs_commit_super(sbi, folio, index, true);
@@ -4789,7 +4849,8 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover)
/* write current valid superblock */
index = sbi->valid_super_block;
- folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping, index, NULL);
+ folio = read_mapping_folio(sbi->sb->s_bdev->bd_mapping,
+ f2fs_lblk_to_folio_index(sbi, index), NULL);
if (IS_ERR(folio))
return PTR_ERR(folio);
err = __f2fs_commit_super(sbi, folio, index, true);
@@ -5263,6 +5324,13 @@ static int f2fs_fill_super(struct super_block *sb, struct fs_context *fc)
}
mutex_init(&sbi->flush_lock);
+ /*
+ * The 1st superblock is at F2FS_SUPER_OFFSET of block 0 for every
+ * block size, so locate it with the minimum supported block size.
+ */
+ sbi->log_blocksize = F2FS_MIN_LOG_BLOCKSIZE;
+ sbi->blocksize = F2FS_MIN_BLKSIZE;
+
err = read_raw_super_block(sbi, &raw_super, &valid_super_block,
&recovery);
if (err)
diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
index 4a2cea4288f9..af1b7f21c1d8 100644
--- a/fs/f2fs/verity.c
+++ b/fs/f2fs/verity.c
@@ -94,6 +94,9 @@ static int f2fs_begin_enable_verity(struct file *filp)
struct inode *inode = file_inode(filp);
int err;
+ if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
+ return -EOPNOTSUPP;
+
if (f2fs_verity_in_progress(inode))
return -EBUSY;
diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h
index 3081702b1ddb..1b3276d57bce 100644
--- a/include/linux/f2fs_fs.h
+++ b/include/linux/f2fs_fs.h
@@ -15,6 +15,7 @@
#define F2FS_MIN_LOG_SECTOR_SIZE 9 /* 9 bits for 512 bytes */
#define F2FS_MAX_LOG_SECTOR_SIZE PAGE_SHIFT /* Max is Block Size */
#define F2FS_MIN_LOG_BLOCKSIZE 12
+#define F2FS_MAX_LOG_BLOCKSIZE PAGE_SHIFT
#define F2FS_MIN_BLKSIZE 4096UL
#define F2FS_MAX_BLKSIZE PAGE_SIZE
#define F2FS_MAX_EXTENSION 64 /* # of extension entries */
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 03/20] f2fs: extend f2fs_folio_state for subpage blocks
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-02 19:39 ` [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths Daeho Jeong
2026-11-02 19:39 ` [PATCH 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
` (16 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, even an order-0 folio holds several blocks, so
track uptodate and dirty state per block for any folio that holds more
than one block, not only for large folios.
- Identify the folio state by a magic number, since folio->private can
also hold page private flags, and carry those flags over when the
state is attached.
- Make the uptodate and dirty helpers take block indices. Allocate the
state on demand with __GFP_NOFAIL, so callers no longer handle an
allocation failure.
- Handle per-block state in invalidation, dirtying and writeback.
- Clear PAGECACHE_TAG_TOWRITE as well as PAGECACHE_TAG_DIRTY in
f2fs_clear_page_cache_dirty_tag().
Some of these changes also apply to large folios when the block size
equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 430 ++++++++++++++++++++++++++--------------------
fs/f2fs/f2fs.h | 35 ++--
fs/f2fs/file.c | 18 +-
fs/f2fs/segment.c | 2 +-
4 files changed, 264 insertions(+), 221 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index aacc92839f61..e983b59e4a65 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -96,15 +96,14 @@ struct bio_post_read_ctx {
static bool __ffs_mark_subrange_uptodate(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
- unsigned int start, end;
-
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int start = F2FS_BYTES_TO_BLK(sbi, offset);
+ unsigned int end = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
bitmap_set(ffs->state, start, end - start);
- return bitmap_full(ffs->state, nr_subpages);
+ return bitmap_full(ffs->state, nr_blocks);
}
/*
@@ -122,7 +121,7 @@ static bool __ffs_mark_subrange_uptodate(struct folio *folio,
* release the bio's reference to the decompress_io_ctx of the page's cluster.
*/
/*
- * Update read_pages_pending.
+ * Update read_blocks_pending.
*/
static inline void f2fs_update_read_folio_pending(struct folio *folio, int nr)
{
@@ -134,7 +133,7 @@ static inline void f2fs_update_read_folio_pending(struct folio *folio, int nr)
ffs = (struct f2fs_folio_state *)folio->private;
spin_lock_irqsave(&ffs->state_lock, flags);
- ffs->read_pages_pending += nr;
+ ffs->read_blocks_pending += nr;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
@@ -173,8 +172,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
if (uptodate)
uptodate = __ffs_mark_subrange_uptodate(folio, ffs,
fi.offset, fi.length);
- ffs->read_pages_pending -= nr_pages;
- finished = !ffs->read_pages_pending;
+ ffs->read_blocks_pending -= nr_pages;
+ finished = !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
@@ -372,18 +371,18 @@ bool f2fs_ffs_has_dirty_subpage(struct folio *folio)
{
struct f2fs_folio_state *ffs;
unsigned long flags;
- unsigned int nr_subpages;
+ unsigned int nr_blocks;
bool dirty;
if (!f2fs_folio_has_ffs(folio))
return false;
ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
+ nr_blocks = f2fs_blocks_per_folio(F2FS_F_SB(folio), folio);
spin_lock_irqsave(&ffs->state_lock, flags);
- dirty = find_next_bit(ffs->state, 2 * nr_subpages,
- nr_subpages) < 2 * nr_subpages;
+ dirty = find_next_bit(ffs->state, 2 * nr_blocks,
+ nr_blocks) < 2 * nr_blocks;
spin_unlock_irqrestore(&ffs->state_lock, flags);
return dirty;
@@ -402,7 +401,7 @@ static void f2fs_ffs_put_write_pending(struct folio *folio, unsigned int nr)
goto release;
ffs = (struct f2fs_folio_state *)folio->private;
- if (!atomic_sub_and_test(nr, &ffs->write_pages_pending))
+ if (!atomic_sub_and_test(nr, &ffs->write_blocks_pending))
return;
release:
if (!f2fs_ffs_has_dirty_subpage(folio))
@@ -1572,7 +1571,7 @@ static int f2fs_submit_page_read_sync(struct inode *inode, struct folio *folio,
REQ_OP_READ, true);
/* Wait until all bios have completed (pending drops back to our bias). */
- while (READ_ONCE(ffs->read_pages_pending) != 1)
+ while (READ_ONCE(ffs->read_blocks_pending) != 1)
f2fs_io_schedule_timeout(DEFAULT_SCHEDULE_TIMEOUT);
/* Remove the bias. */
@@ -1693,12 +1692,8 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
if (IS_ERR(folio))
return folio;
- if (folio_test_large(folio)) {
- if (!f2fs_ffs_find_or_alloc(folio)) {
- err = -ENOMEM;
- goto put_err;
- }
- }
+ if (folio_test_large(folio))
+ f2fs_ffs_find_or_alloc(folio);
if (f2fs_lookup_read_extent_cache_block(inode, index,
&dn.data_blkaddr)) {
@@ -1821,12 +1816,8 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
* for a fully uptodate folio; a partially uptodate one cannot be
* rebuilt and will fail the subpage check below.
*/
- if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio)) {
- if (!f2fs_ffs_find_or_alloc(folio)) {
- f2fs_folio_put(folio, true);
- return ERR_PTR(-ENOMEM);
- }
- }
+ if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio))
+ f2fs_ffs_find_or_alloc(folio);
if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, index))) {
f2fs_folio_put(folio, true);
@@ -1862,11 +1853,8 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
return ERR_PTR(-ENOMEM);
}
- if (folio_test_large(folio) && !f2fs_ffs_find_or_alloc(folio)) {
- f2fs_folio_put(folio, true);
- f2fs_put_cache(ientry, true);
- return ERR_PTR(-ENOMEM);
- }
+ if (folio_test_large(folio))
+ f2fs_ffs_find_or_alloc(folio);
set_new_dnode(&dn, inode, ientry, NULL, 0);
err = f2fs_reserve_block(&dn, index);
@@ -2900,29 +2888,43 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio)
{
- struct f2fs_folio_state *ffs;
- unsigned int nr_subpages = folio_nr_pages(folio);
- unsigned long private = (unsigned long)folio->private;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_test_large(folio));
- f2fs_bug_on(F2FS_F_SB(folio),
- test_bit(F2FS_FOLIO_PRIVATE_NOT_POINTER, &private));
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ struct f2fs_folio_state *ffs = folio->private;
+ unsigned int nr_blocks;
+ unsigned long private_flags;
if (f2fs_folio_has_ffs(folio))
- return (struct f2fs_folio_state *)folio->private;
+ return ffs;
- ffs = f2fs_kmalloc(F2FS_F_SB(folio),
- struct_size(ffs, state, BITS_TO_LONGS(2 * nr_subpages)),
- GFP_NOFS | __GFP_ZERO);
- if (!ffs)
- return NULL;
+ /* Carry over the page private flags kept in folio->private. */
+ private_flags = f2fs_folio_get_private_flags(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ ffs = kzalloc(struct_size(ffs, state, BITS_TO_LONGS(2 * nr_blocks)),
+ GFP_NOIO | __GFP_NOFAIL);
+
+ ffs->magic = F2FS_FOLIO_STATE_MAGIC;
spin_lock_init(&ffs->state_lock);
+ ffs->private_flags = private_flags;
if (folio_test_uptodate(folio))
- bitmap_set(ffs->state, 0, nr_subpages);
- if (folio_test_dirty(folio))
- bitmap_set(ffs->state, nr_subpages, nr_subpages);
+ bitmap_set(ffs->state, 0, nr_blocks);
+ if (folio_test_dirty(folio)) {
+ struct inode *inode = folio->mapping->host;
+ pgoff_t last_lblk = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
+ pgoff_t valid_blocks = 0;
+ if (f2fs_verity_in_progress(inode))
+ valid_blocks = nr_blocks;
+ else if (first_lblk < last_lblk)
+ valid_blocks = min_t(pgoff_t, nr_blocks,
+ last_lblk - first_lblk);
+
+ if (valid_blocks)
+ bitmap_set(ffs->state, nr_blocks, valid_blocks);
+ }
+
+ folio_detach_private(folio);
folio_attach_private(folio, ffs);
return ffs;
}
@@ -2940,17 +2942,17 @@ static void ffs_detach_free(struct folio *folio)
if (!ffs)
return;
- WARN_ON_ONCE(ffs->read_pages_pending != 0);
- WARN_ON_ONCE(atomic_read(&ffs->write_pages_pending));
+ WARN_ON_ONCE(ffs->read_blocks_pending != 0);
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending));
+ ffs->magic = 0;
kfree(ffs);
}
bool f2fs_ffs_test_blk_uptodate(const struct folio *folio, pgoff_t index)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
struct f2fs_folio_state *ffs;
- unsigned int idx;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_contains(folio, index));
+ pgoff_t offset;
if (folio_test_uptodate(folio))
return true;
@@ -2958,63 +2960,60 @@ bool f2fs_ffs_test_blk_uptodate(const struct folio *folio, pgoff_t index)
if (!f2fs_folio_has_ffs(folio))
return false;
- ffs = (struct f2fs_folio_state *)folio->private;
- idx = index - folio->index;
- return test_bit(idx, ffs->state);
+ ffs = folio->private;
+ offset = f2fs_folio_blk_offset(sbi, folio, index);
+ if (WARN_ON_ONCE(offset >= f2fs_blocks_per_folio(sbi, folio)))
+ return false;
+
+ if (test_bit(offset, ffs->state)) {
+ /* Order the bit test before reading the block data */
+ smp_rmb();
+ return true;
+ }
+ return false;
}
void f2fs_ffs_mark_subrange_uptodate(struct folio *folio, size_t offset,
size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
unsigned long flags;
bool mark_uptodate = false;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
- if (!f2fs_folio_has_ffs(folio)) {
+ if (!f2fs_is_multiblock_folio(sbi, folio)) {
folio_mark_uptodate(folio);
return;
}
- ffs = (struct f2fs_folio_state *)folio->private;
- spin_lock_irqsave(&ffs->state_lock, flags);
- mark_uptodate = __ffs_mark_subrange_uptodate(folio, ffs, offset, len) &&
- !ffs->read_pages_pending;
- spin_unlock_irqrestore(&ffs->state_lock, flags);
- if (mark_uptodate)
+ /*
+ * Without f2fs_folio_state, the folio uptodate flag covers every block,
+ * so marking the whole folio needs no allocation. This keeps bio
+ * completion paths such as decompression free of allocation.
+ */
+ if (!f2fs_folio_has_ffs(folio) && !offset && len >= folio_size(folio)) {
folio_mark_uptodate(folio);
-}
-
-static void ffs_clear_subrange_uptodate(struct folio *folio,
- size_t offset, size_t len)
-{
- struct f2fs_folio_state *ffs;
- unsigned int nr_subpages, start, end;
- unsigned long flags;
-
- f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
-
- if (!f2fs_folio_has_ffs(folio))
return;
+ }
- ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
-
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
- bitmap_clear(ffs->state, start, end - start);
+ mark_uptodate = __ffs_mark_subrange_uptodate(folio, ffs, offset, len) &&
+ !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
+ if (mark_uptodate)
+ folio_mark_uptodate(folio);
}
static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+
if (f2fs_folio_has_ffs(folio)) {
- ffs_clear_subrange_uptodate(folio,
- (loff_t)i << PAGE_SHIFT, PAGE_SIZE);
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio,
+ F2FS_BLK_TO_BYTES(sbi, i), F2FS_BLKSIZE(sbi));
} else {
folio_clear_uptodate(folio);
folio_clear_f2fs_gcing(folio);
@@ -3023,33 +3022,36 @@ static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
bool f2fs_ffs_test_blk_dirty(const struct folio *folio, pgoff_t index)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
struct f2fs_folio_state *ffs;
- unsigned int idx, nr_subpages;
-
- f2fs_bug_on(F2FS_F_SB(folio), !folio_contains(folio, index));
+ unsigned int nr_blocks;
+ pgoff_t offset;
if (!f2fs_folio_has_ffs(folio))
return folio_test_dirty(folio);
- ffs = (struct f2fs_folio_state *)folio->private;
- nr_subpages = folio_nr_pages(folio);
- idx = index - folio->index;
- return test_bit(nr_subpages + idx, ffs->state);
+ ffs = folio->private;
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ offset = f2fs_folio_blk_offset(sbi, folio, index);
+ if (WARN_ON_ONCE(offset >= nr_blocks))
+ return false;
+
+ return test_bit(nr_blocks + offset, ffs->state);
}
static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned int start, end, before;
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
- before = bitmap_weight(ffs->state, 2 * nr_subpages);
- bitmap_set(ffs->state, nr_subpages + start, end - start);
- return bitmap_weight(ffs->state, 2 * nr_subpages) - before;
+ before = bitmap_weight(ffs->state, 2 * nr_blocks);
+ bitmap_set(ffs->state, nr_blocks + start, end - start);
+ return bitmap_weight(ffs->state, 2 * nr_blocks) - before;
}
/*
@@ -3062,16 +3064,17 @@ static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
size_t offset, size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
unsigned int set;
unsigned long flags;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
- if (!f2fs_folio_has_ffs(folio))
+ if (!f2fs_is_multiblock_folio(sbi, folio))
return;
- ffs = (struct f2fs_folio_state *)folio->private;
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
set = __ffs_mark_subrange_dirty(folio, ffs, offset, len);
@@ -3081,22 +3084,54 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
f2fs_account_dirty_blocks(folio->mapping->host, set);
}
+void f2fs_ffs_clear_subrange_uptodate(struct folio *folio,
+ size_t offset, size_t len)
+{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ struct f2fs_folio_state *ffs;
+ unsigned int nr_blocks, start, end;
+ unsigned long flags;
+
+ f2fs_bug_on(sbi, offset + len > folio_size(folio));
+
+ if (!f2fs_folio_has_ffs(folio)) {
+ if (!folio_test_uptodate(folio))
+ return;
+ /* Without ffs, clearing the whole folio needs no allocation. */
+ if (!offset && len >= folio_size(folio)) {
+ folio_clear_uptodate(folio);
+ return;
+ }
+ }
+
+ ffs = f2fs_ffs_find_or_alloc(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
+
+ spin_lock_irqsave(&ffs->state_lock, flags);
+ bitmap_clear(ffs->state, start, end - start);
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ if (folio_test_uptodate(folio))
+ folio_clear_uptodate(folio);
+}
+
static bool __ffs_clear_subrange_dirty(struct folio *folio,
struct f2fs_folio_state *ffs, size_t offset, size_t len,
unsigned int *cleared)
{
- unsigned int nr_subpages = folio_nr_pages(folio);
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned int start, end, before;
- start = offset >> PAGE_SHIFT;
- end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- end = min(end, nr_subpages);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
- before = bitmap_weight(ffs->state, 2 * nr_subpages);
- bitmap_clear(ffs->state, nr_subpages + start, end - start);
- *cleared = before - bitmap_weight(ffs->state, 2 * nr_subpages);
- return find_next_bit(ffs->state, 2 * nr_subpages, nr_subpages) <
- 2 * nr_subpages;
+ before = bitmap_weight(ffs->state, 2 * nr_blocks);
+ bitmap_clear(ffs->state, nr_blocks + start, end - start);
+ *cleared = before - bitmap_weight(ffs->state, 2 * nr_blocks);
+ return find_next_bit(ffs->state, 2 * nr_blocks, nr_blocks) <
+ 2 * nr_blocks;
}
/**
@@ -3117,10 +3152,19 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
unsigned int cleared = 0;
bool dirty;
- if (!f2fs_folio_has_ffs(folio))
- return false;
+ if (!f2fs_folio_has_ffs(folio)) {
+ /*
+ * Without ffs, the folio dirty flag covers every block. Callers
+ * that clear the flag first (writeback, GC) must have ffs.
+ */
+ if (!folio_test_dirty(folio))
+ return false;
+ /* Clearing the whole folio leaves no dirty block. */
+ if (!offset && len >= folio_size(folio))
+ return false;
+ }
- ffs = (struct f2fs_folio_state *)folio->private;
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len, &cleared);
spin_unlock_irqrestore(&ffs->state_lock, flags);
@@ -3131,55 +3175,37 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
return dirty;
}
-static unsigned int ffs_next_dirty_subpage(struct f2fs_folio_state *ffs,
- const struct folio *folio, unsigned int start,
- unsigned int end)
-{
- unsigned int nr_subpages = folio_nr_pages(folio);
-
- return find_next_bit(ffs->state, nr_subpages + end + 1,
- nr_subpages + start) - nr_subpages;
-}
-
-static unsigned int ffs_next_clean_subpage(struct f2fs_folio_state *ffs,
- const struct folio *folio, unsigned int start,
- unsigned int end)
-{
- unsigned int nr_subpages = folio_nr_pages(folio);
-
- return find_next_zero_bit(ffs->state, nr_subpages + end + 1,
- nr_subpages + start) - nr_subpages;
-}
-
-static unsigned int ffs_find_dirty_range(struct folio *folio,
- u64 *range_start, u64 range_end)
+bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
+ unsigned int to, unsigned int *start,
+ unsigned int *count)
{
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
- unsigned int start, end, nr_pages;
-
- if (*range_start >= range_end)
- return 0;
-
- if (!f2fs_folio_has_ffs(folio))
- return range_end - *range_start;
-
- ffs = (struct f2fs_folio_state *)folio->private;
- start = offset_in_folio(folio, *range_start) >> PAGE_SHIFT;
- end = DIV_ROUND_UP(min_not_zero(offset_in_folio(folio, range_end),
- folio_size(folio)), PAGE_SIZE) - 1;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned long flags;
- start = ffs_next_dirty_subpage(ffs, folio, start, end);
- if (start > end)
- return 0;
+ if (from >= to || to > nr_blocks)
+ return false;
- if (start == end)
- nr_pages = 1;
- else
- nr_pages = ffs_next_clean_subpage(ffs, folio,
- start + 1, end) - start;
+ if (!f2fs_folio_has_ffs(folio)) {
+ if (!folio_test_dirty(folio))
+ return false;
+ *start = from;
+ *count = to - from;
+ return true;
+ }
- *range_start = folio_pos(folio) + ((u64)start << PAGE_SHIFT);
- return (u64)nr_pages << PAGE_SHIFT;
+ ffs = folio->private;
+ spin_lock_irqsave(&ffs->state_lock, flags);
+ *start = find_next_bit(ffs->state, nr_blocks + to, nr_blocks + from) - nr_blocks;
+ if (*start >= to) {
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ return false;
+ }
+ *count = find_next_zero_bit(ffs->state, nr_blocks + to,
+ nr_blocks + *start) - nr_blocks - *start;
+ spin_unlock_irqrestore(&ffs->state_lock, flags);
+ return true;
}
static bool find_next_valid_block(const struct folio *folio,
@@ -3307,13 +3333,9 @@ static int f2fs_read_data_large_folio(struct inode *inode,
/*
* Allocate the folio state before doing anything with the
* folio, so that the block mapping helpers below can rely on
- * it, and fail early if memory is not available.
+ * it.
*/
ffs = f2fs_ffs_find_or_alloc(folio);
- if (!ffs) {
- ret = -ENOMEM;
- goto err_out;
- }
}
index = folio->index;
offset = 0;
@@ -3393,7 +3415,7 @@ static int f2fs_read_data_large_folio(struct inode *inode,
continue;
}
- /* We must increment read_pages_pending before possible BIOs submitting
+ /* We must increment read_blocks_pending before possible BIOs submitting
* to prevent from premature folio_end_read() call on folio
*/
if (folio_test_large(folio)) {
@@ -3847,7 +3869,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
__folio_start_writeback(folio, folio_test_dirty(folio));
if (f2fs_folio_has_ffs(folio)) {
ffs = (struct f2fs_folio_state *)folio->private;
- atomic_inc(&ffs->write_pages_pending);
+ atomic_inc(&ffs->write_blocks_pending);
pending_added = true;
}
@@ -3927,7 +3949,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
if (f2fs_folio_has_ffs(folio)) {
ffs = (struct f2fs_folio_state *)folio->private;
- atomic_inc(&ffs->write_pages_pending);
+ atomic_inc(&ffs->write_blocks_pending);
}
set_new_dnode(&dn, dn_inode, NULL, NULL, 0);
@@ -4499,7 +4521,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
struct f2fs_lock_context lc;
u64 pos = 0;
u64 end_pos = 0;
- u32 r_len = 0;
int err = 0;
int submitted = 0;
int nwritten = 0;
@@ -4633,29 +4654,60 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_start_writeback(folio);
+ /*
+ * PG_dirty was already cleared by writeback_iter(), and a
+ * single-block folio has no ffs dirty bitmap, so the dirty
+ * range cannot be derived. Write the whole folio.
+ */
+ if (!folio_test_large(folio)) {
+ err = f2fs_write_folio_dirty_range(folio, &submitted,
+ wbc, io_type, &pos, end_pos - pos);
+ folio_submitted += submitted;
+ if (err)
+ goto out;
+ nwritten += submitted;
+ goto written;
+ }
+
+ if (!f2fs_folio_has_ffs(folio)) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
+ f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
+ } else {
+ ffs = (struct f2fs_folio_state *)folio->private;
+ }
/*
* Large folios can be submitted as multiple bios, which may
* complete while we are still submitting the remaining dirty
- * ranges. Hold a bias on write_pages_pending to prevent
+ * ranges. Hold a bias on write_blocks_pending to prevent
* folio_end_writeback() from being called early.
*/
- if (!bias_added && f2fs_folio_has_ffs(folio)) {
- ffs = (struct f2fs_folio_state *)folio->private;
- WARN_ON_ONCE(atomic_read(&ffs->write_pages_pending) != 0);
- atomic_inc(&ffs->write_pages_pending);
+ if (f2fs_folio_has_ffs(folio) && !bias_added) {
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
bias_added = true;
}
- while ((r_len = ffs_find_dirty_range(folio, &pos, end_pos))) {
+ unsigned int from = 0, start, count;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int to_blk = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, end_pos - folio_pos(folio)),
+ nr_blocks);
+
+ while (f2fs_ffs_dirty_blk_range(folio, from, to_blk,
+ &start, &count)) {
submitted = 0;
+ pos = folio_pos(folio) + F2FS_BLK_TO_BYTES(sbi, start);
err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos, r_len);
+ wbc, io_type, &pos,
+ F2FS_BLK_TO_BYTES(sbi, count));
folio_submitted += submitted;
nwritten += submitted;
if (err)
goto out;
+ from = start + count;
}
+written:
/* If none of the subpages of a folio is submitted, keep the atomic flag */
if (folio_submitted &&
f2fs_is_atomic_file(inode) &&
@@ -5171,8 +5223,6 @@ static int prepare_large_folio_write_begin(struct inode *inode,
}
ffs = f2fs_ffs_find_or_alloc(folio);
- if (!ffs)
- return -ENOMEM;
/* Skip read if the folio is already fully uptodate or the write
* covers the entire folio.
@@ -5248,8 +5298,7 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
len = min_t(unsigned int, len, folio_size(folio) - ori_off);
- if (!f2fs_ffs_find_or_alloc(folio))
- return -ENOMEM;
+ f2fs_ffs_find_or_alloc(folio);
/* Inline data must have been converted before reaching here. */
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
@@ -5533,8 +5582,8 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
if (inode->i_ino >= F2FS_ROOT_INO(sbi) &&
(offset || length != folio_size(folio))) {
- size_t clear_start = round_up(offset, PAGE_SIZE);
- size_t clear_end = round_down(offset + length, PAGE_SIZE);
+ size_t clear_start = round_up(offset, F2FS_BLKSIZE(sbi));
+ size_t clear_end = round_down(offset + length, F2FS_BLKSIZE(sbi));
size_t clear_length;
bool dirty;
@@ -5548,16 +5597,18 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
clear_length = clear_end - clear_start;
dirty = f2fs_ffs_clear_subrange_dirty_and_test(folio,
clear_start, clear_length);
- ffs_clear_subrange_uptodate(folio, clear_start, clear_length);
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio, clear_start,
+ clear_length);
/*
* If the truncated subrange happens to clear the remaining
* dirty bitmap of the whole folio, cancel the folio-level
* dirty state.
*/
- if (!dirty && folio_test_dirty(folio))
+ if (!dirty && folio_test_dirty(folio)) {
+ f2fs_clear_page_cache_dirty_tag(folio);
folio_cancel_dirty(folio);
+ }
if (!dirty && !folio_test_writeback(folio))
folio_clear_f2fs_gcing(folio);
return;
@@ -5592,6 +5643,7 @@ static bool f2fs_dirty_data_folio(struct address_space *mapping,
struct folio *folio)
{
struct inode *inode = mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
trace_f2fs_set_page_dirty(folio, DATA);
@@ -5599,13 +5651,13 @@ static bool f2fs_dirty_data_folio(struct address_space *mapping,
bool uptodate = true;
if (f2fs_folio_has_ffs(folio)) {
- struct f2fs_folio_state *ffs =
- (struct f2fs_folio_state *)folio->private;
+ struct f2fs_folio_state *ffs = folio->private;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
unsigned long flags;
spin_lock_irqsave(&ffs->state_lock, flags);
- uptodate = bitmap_full(ffs->state, folio_nr_pages(folio)) &&
- !ffs->read_pages_pending;
+ uptodate = bitmap_full(ffs->state, nr_blocks) &&
+ !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
if (uptodate)
@@ -5957,6 +6009,8 @@ void f2fs_clear_page_cache_dirty_tag(struct folio *folio)
xa_lock_irqsave(&mapping->i_pages, flags);
__xa_clear_mark(&mapping->i_pages, folio->index,
PAGECACHE_TAG_DIRTY);
+ __xa_clear_mark(&mapping->i_pages, folio->index,
+ PAGECACHE_TAG_TOWRITE);
xa_unlock_irqrestore(&mapping->i_pages, flags);
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 11de12a3e804..114d3d7bcd3e 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1630,13 +1630,17 @@ static inline void f2fs_clear_bit(unsigned int nr, char *addr);
* folio->private is a wrapped pointer.
*/
+#define F2FS_FOLIO_STATE_MAGIC 0xF5F2FF50
+
struct f2fs_folio_state {
+ u32 magic;
spinlock_t state_lock;
- unsigned int read_pages_pending;
- atomic_t write_pages_pending;
+ unsigned int read_blocks_pending;
+ atomic_t write_blocks_pending;
unsigned long private_flags;
- /* state[0..nr_subpages - 1] tracks uptodate subpages.
- * state[nr_subpages..2 * nr_subpages - 1] tracks dirty subpages.
+ /*
+ * state[0..nr_blocks - 1] tracks uptodate blocks.
+ * state[nr_blocks..2 * nr_blocks - 1] tracks dirty blocks.
*/
unsigned long state[];
};
@@ -2337,6 +2341,12 @@ static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
}
+static inline bool f2fs_is_multiblock_folio(struct f2fs_sb_info *sbi,
+ const struct folio *folio)
+{
+ return folio_size(folio) > F2FS_BLKSIZE(sbi);
+}
+
static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
const struct folio *folio)
{
@@ -2483,18 +2493,9 @@ static inline bool f2fs_folio_has_ffs(const struct folio *folio)
{
unsigned long private = (unsigned long)folio->private;
- if (!folio_test_large(folio) || !private)
+ if (!private || (private & BIT(F2FS_FOLIO_PRIVATE_NOT_POINTER)))
return false;
-
- /*
- * A large folio should always carry f2fs_folio_state rather than
- * non-pointer private flags at its folio->private.
- */
- if (unlikely(test_bit(F2FS_FOLIO_PRIVATE_NOT_POINTER, &private))) {
- f2fs_bug_on(F2FS_F_SB(folio), 1);
- return false;
- }
- return true;
+ return *((u32 *)private) == F2FS_FOLIO_STATE_MAGIC;
}
static inline void clear_sbi_flag(struct f2fs_sb_info *sbi, unsigned int type)
@@ -4529,6 +4530,10 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio, size_t offset, size_t len
bool f2fs_ffs_has_dirty_subpage(struct folio *folio);
bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
size_t len);
+void f2fs_ffs_clear_subrange_uptodate(struct folio *folio, size_t offset,
+ size_t len);
+bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
+ unsigned int to, unsigned int *start, unsigned int *count);
void f2fs_write_failed(struct inode *inode, loff_t to);
void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);
bool f2fs_release_folio(struct folio *folio, gfp_t wait);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 82eb62301c48..e339cfd84b27 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -238,12 +238,6 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
goto out_sem;
}
- if (folio_test_large(folio) && !f2fs_ffs_find_or_alloc(folio)) {
- folio_unlock(folio);
- err = -ENOMEM;
- goto out_sem;
- }
-
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
for (i = 0; i < nr; i++) {
@@ -1628,12 +1622,6 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
memcpy_folio(fdst, dst_off, fsrc, src_off,
F2FS_BLKSIZE(sbi));
if (folio_test_large(fdst)) {
- if (!f2fs_ffs_find_or_alloc(fdst)) {
- f2fs_folio_put(fdst, true);
- if (!same_folio)
- f2fs_folio_put(fsrc, true);
- return -ENOMEM;
- }
f2fs_ffs_mark_subrange_uptodate(fdst,
dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
@@ -4975,16 +4963,12 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
* bitmap as well, otherwise f2fs_write_cache_folios()
* finds no dirty range and skips the whole folio.
*/
- if (!f2fs_ffs_find_or_alloc(folio)) {
- ret = -ENOMEM;
- goto next;
- }
+ f2fs_ffs_find_or_alloc(folio);
f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
}
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
-next:
redirty_idx = folio_next_index(folio);
folio_unlock(folio);
folio_put_refs(folio, 2);
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 2b0821c2c18b..93f165151c0f 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -4193,7 +4193,7 @@ static void do_write_block(struct f2fs_summary *sum, struct f2fs_io_info *fio)
fio->old_blkaddr, fio->new_blkaddr, err);
/*
* For a large folio, the caller rolls back
- * write_pages_pending and ends writeback when the last
+ * write_blocks_pending and ends writeback when the last
* pending write is gone.
*/
if (!f2fs_folio_has_ffs(folio))
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 04/20] f2fs: support subpage blocks in directory operations
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (2 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
` (15 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a directory folio holds several dentry blocks.
Address a dentry block within its folio by its block index, and mark
only the modified dentry block dirty. When an empty dentry block is
freed, clear only the state of that block and keep the folio dirty if
other blocks in it are still dirty.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/dir.c | 120 +++++++++++++++++++++++++++++++++--------------
fs/f2fs/f2fs.h | 2 +
fs/f2fs/inline.c | 6 +--
3 files changed, 89 insertions(+), 39 deletions(-)
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 3eb0b7255951..aa3511c6f8fe 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -37,6 +37,31 @@ static inline bool f2fs_should_fallback_to_linear(struct inode *dir)
extern struct kmem_cache *f2fs_cf_name_slab;
#endif
+static inline pgoff_t f2fs_dentry_to_lblk(struct f2fs_sb_info *sbi,
+ struct folio *folio, void *de)
+{
+ size_t off = round_down((size_t)(de - folio_address(folio)),
+ (size_t)F2FS_BLKSIZE(sbi));
+
+ return f2fs_folio_lblk(sbi, folio) + (off >> sbi->log_blocksize);
+}
+
+void f2fs_mark_dentry_folio_dirty(struct inode *dir, struct folio *folio,
+ pgoff_t bidx)
+{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
+
+ if (f2fs_has_subpage_blocks(sbi)) {
+ size_t off = f2fs_folio_byte_offset(sbi, folio, bidx);
+
+ f2fs_ffs_mark_subrange_uptodate(folio, off, F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_dirty(folio, off, F2FS_BLKSIZE(sbi));
+ } else if (!folio_test_uptodate(folio)) {
+ folio_mark_uptodate(folio);
+ }
+ folio_mark_dirty(folio);
+}
+
static unsigned long dir_blocks(struct inode *inode)
{
return F2FS_BYTES_TO_BLK(F2FS_I_SB(inode),
@@ -190,15 +215,16 @@ static unsigned long dir_block_index(unsigned int level,
}
static struct f2fs_dir_entry *find_in_block(struct inode *dir,
- struct folio *dentry_folio,
+ struct folio *dentry_folio, pgoff_t bidx,
const struct f2fs_filename *fname,
int *max_slots,
bool use_hash)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
void *dentry_blk;
struct f2fs_dentry_ptr d;
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(sbi, dentry_folio, bidx);
make_dentry_ptr_block(dir, &d, dentry_blk);
return f2fs_find_target_dentry(&d, fname, max_slots, use_hash);
@@ -289,7 +315,7 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
unsigned int nbucket, nblock;
unsigned int bidx, end_block, bucket_no;
struct f2fs_dir_entry *de = NULL;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
bool room = false;
int max_slots;
@@ -306,11 +332,11 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
while (bidx < end_block) {
/* no need to allocate new dentry pages to all the indices */
struct folio *dentry_folio;
- dentry_folio = f2fs_find_data_folio(dir, bidx, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(dir, bidx, &next_blkofs);
if (IS_ERR(dentry_folio)) {
if (PTR_ERR(dentry_folio) == -ENOENT) {
room = true;
- bidx = next_pgofs;
+ bidx = next_blkofs;
continue;
} else {
*dentry_block = dentry_folio;
@@ -318,7 +344,8 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
}
}
- de = find_in_block(dir, dentry_folio, fname, &max_slots, use_hash);
+ de = find_in_block(dir, dentry_folio, bidx, fname, &max_slots,
+ use_hash);
if (IS_ERR(de)) {
f2fs_folio_put(dentry_folio, false);
*dentry_block = ERR_CAST(de);
@@ -354,7 +381,7 @@ struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
const struct f2fs_filename *fname,
void **dentry_block)
{
- unsigned long npages = dir_blocks(dir);
+ unsigned long nblocks = dir_blocks(dir);
struct f2fs_dir_entry *de = NULL;
unsigned int max_depth;
unsigned int level;
@@ -370,7 +397,7 @@ struct f2fs_dir_entry *__f2fs_find_entry(struct inode *dir,
goto out;
}
- if (npages == 0)
+ if (nblocks == 0)
goto out;
max_depth = F2FS_I(dir)->i_current_depth;
@@ -468,7 +495,8 @@ void f2fs_set_link(struct inode *dir, struct f2fs_dir_entry *de,
f2fs_folio_wait_writeback(folio, true, true);
de->ino = cpu_to_le32(inode->i_ino);
de->file_type = fs_umode_to_ftype(inode->i_mode);
- folio_mark_dirty(folio);
+ f2fs_mark_dentry_folio_dirty(dir, folio,
+ f2fs_dentry_to_lblk(F2FS_I_SB(dir), folio, de));
}
inode_set_mtime_to_ts(dir, inode_set_ctime_current(dir));
@@ -539,12 +567,12 @@ static int make_empty_dir(struct inode *inode,
if (IS_ERR(dentry_folio))
return PTR_ERR(dentry_folio);
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(inode), dentry_folio, 0);
make_dentry_ptr_block(inode, &d, dentry_blk);
f2fs_do_make_empty_dir(inode, parent, &d);
- folio_mark_dirty(dentry_folio);
+ f2fs_mark_dentry_folio_dirty(inode, dentry_folio, 0);
f2fs_folio_put(dentry_folio, true);
return 0;
}
@@ -737,7 +765,8 @@ int f2fs_add_regular_entry(struct inode *dir, const struct f2fs_filename *fname,
if (IS_ERR(dentry_folio))
return PTR_ERR(dentry_folio);
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(dir),
+ dentry_folio, block);
make_dentry_ptr_block(dir, &d, dentry_blk);
bit_pos = f2fs_room_for_filename(d.bitmap, slots, d.max);
if (bit_pos < d.max)
@@ -764,7 +793,7 @@ int f2fs_add_regular_entry(struct inode *dir, const struct f2fs_filename *fname,
f2fs_update_dentry(ino, mode, &d, &fname->disk_name, fname->hash,
bit_pos);
- folio_mark_dirty(dentry_folio);
+ f2fs_mark_dentry_folio_dirty(dir, dentry_folio, block);
if (inode) {
f2fs_i_pino_write(inode, dir->i_ino);
@@ -902,25 +931,28 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
struct folio *folio;
unsigned int bit_pos;
int slots = GET_DENTRY_SLOTS(le16_to_cpu(dentry->name_len));
+ struct f2fs_sb_info *sbi = F2FS_I_SB(dir);
+ size_t block_offset;
pgoff_t index;
int i;
- f2fs_update_time(F2FS_I_SB(dir), REQ_TIME);
+ f2fs_update_time(sbi, REQ_TIME);
- if (F2FS_OPTION(F2FS_I_SB(dir)).fsync_mode == FSYNC_MODE_STRICT)
- f2fs_add_ino_entry(F2FS_I_SB(dir), dir->i_ino, TRANS_DIR_INO);
+ if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT)
+ f2fs_add_ino_entry(sbi, dir->i_ino, TRANS_DIR_INO);
if (f2fs_has_inline_dentry(dir))
return f2fs_delete_inline_entry(dentry,
f2fs_dentry_cache(dentry_block), dir, inode);
folio = f2fs_dentry_folio(dentry_block);
- index = folio->index;
folio_lock(folio);
f2fs_folio_wait_writeback(folio, true, true);
- dentry_blk = folio_address(folio);
+ index = f2fs_dentry_to_lblk(sbi, folio, dentry);
+ block_offset = f2fs_folio_byte_offset(sbi, folio, index);
+ dentry_blk = folio_address(folio) + block_offset;
make_dentry_ptr_block(dir, &d, dentry_blk);
bit_pos = dentry - d.dentry;
for (i = 0; i < slots; i++)
@@ -928,18 +960,31 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
/* Let's check and deallocate this dentry page */
bit_pos = find_next_bit_le(d.bitmap, d.max, 0);
- folio_mark_dirty(folio);
+ f2fs_mark_dentry_folio_dirty(dir, folio, index);
if (bit_pos == d.max &&
!f2fs_truncate_hole(dir, index, index + 1)) {
+ if (f2fs_has_subpage_blocks(sbi)) {
+ bool dirty;
+
+ dirty = f2fs_ffs_clear_subrange_dirty_and_test(folio,
+ block_offset, F2FS_BLKSIZE(sbi));
+ f2fs_ffs_clear_subrange_uptodate(folio, block_offset,
+ F2FS_BLKSIZE(sbi));
+ if (dirty)
+ goto keep_dirty;
+ }
f2fs_clear_page_cache_dirty_tag(folio);
folio_clear_dirty_for_io(folio);
- folio_clear_uptodate(folio);
- folio_detach_private(folio);
+ if (!f2fs_has_subpage_blocks(sbi)) {
+ folio_clear_uptodate(folio);
+ folio_detach_private(folio);
+ inode_dec_dirty_block(dir);
+ }
- inode_dec_dirty_block(dir);
f2fs_remove_dirty_inode(dir);
}
+keep_dirty:
f2fs_folio_put(folio, true);
inode_set_mtime_to_ts(dir, inode_set_ctime_current(dir));
@@ -961,20 +1006,21 @@ bool f2fs_empty_dir(struct inode *dir)
return f2fs_empty_inline_dir(dir);
while (bidx < nblock) {
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
struct folio *dentry_folio;
- dentry_folio = f2fs_find_data_folio(dir, bidx, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(dir, bidx, &next_blkofs);
if (IS_ERR(dentry_folio)) {
if (PTR_ERR(dentry_folio) == -ENOENT) {
- bidx = next_pgofs;
+ bidx = next_blkofs;
continue;
} else {
return false;
}
}
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(F2FS_I_SB(dir),
+ dentry_folio, bidx);
make_dentry_ptr_block(dir, &d, dentry_blk);
if (bidx == 0)
bit_pos = 2;
@@ -1080,7 +1126,7 @@ int f2fs_fill_dentries(struct dir_context *ctx, struct f2fs_dentry_ptr *d,
static int f2fs_readdir(struct file *file, struct dir_context *ctx)
{
struct inode *inode = file_inode(file);
- unsigned long npages = dir_blocks(inode);
+ unsigned long nblocks = dir_blocks(inode);
void *dentry_blk = NULL;
struct file_ra_state *ra = &file->f_ra;
loff_t start_pos = ctx->pos;
@@ -1089,6 +1135,8 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
unsigned int n = (unsigned long)ctx->pos / entries;
struct f2fs_dentry_ptr d;
struct fscrypt_str fstr = FSTR_INIT(NULL, 0);
+ pgoff_t f_last = nblocks ? f2fs_lblk_to_folio_index(sbi, nblocks - 1) : 0;
+ pgoff_t f_index, ra_pages;
int err = 0;
if (IS_ENCRYPTED(inode)) {
@@ -1106,9 +1154,9 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
goto out_free;
}
- for (; n < npages; ctx->pos = n * entries) {
+ for (; n < nblocks; ctx->pos = n * entries) {
struct folio *dentry_folio;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
/* allow readdir() to be interrupted */
if (fatal_signal_pending(current)) {
@@ -1118,23 +1166,25 @@ static int f2fs_readdir(struct file *file, struct dir_context *ctx)
cond_resched();
/* readahead for multi pages of dir */
- if (npages - n > 1 && !ra_has_index(ra, n))
- page_cache_sync_readahead(inode->i_mapping, ra, file, n,
- min(npages - n, (pgoff_t)MAX_DIR_RA_PAGES));
+ f_index = f2fs_lblk_to_folio_index(sbi, n);
+ ra_pages = f_last - f_index + 1;
+ if (nblocks - n > 1 && !ra_has_index(ra, f_index))
+ page_cache_sync_readahead(inode->i_mapping, ra, file, f_index,
+ min(ra_pages, (pgoff_t)MAX_DIR_RA_PAGES));
- dentry_folio = f2fs_find_data_folio(inode, n, &next_pgofs);
+ dentry_folio = f2fs_find_data_folio(inode, n, &next_blkofs);
if (IS_ERR(dentry_folio)) {
err = PTR_ERR(dentry_folio);
if (err == -ENOENT) {
err = 0;
- n = next_pgofs;
+ n = next_blkofs;
continue;
} else {
goto out_free;
}
}
- dentry_blk = folio_address(dentry_folio);
+ dentry_blk = f2fs_folio_lblk_address(sbi, dentry_folio, n);
make_dentry_ptr_block(inode, &d, dentry_blk);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 114d3d7bcd3e..3822824326d7 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4193,6 +4193,8 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_blk,
int f2fs_do_tmpfile(struct inode *inode, struct inode *dir,
struct f2fs_filename *fname);
bool f2fs_empty_dir(struct inode *dir);
+void f2fs_mark_dentry_folio_dirty(struct inode *dir, struct folio *folio,
+ pgoff_t bidx);
static inline int f2fs_add_link(struct dentry *dentry, struct inode *inode)
{
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index 49905aa08418..ba0aff8e5643 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -463,10 +463,6 @@ static int f2fs_move_inline_dirents(struct inode *dir,
memcpy(dst.dentry, src.dentry, SIZE_OF_DIR_ENTRY * src.max);
memcpy(dst.filename, src.filename, src.max * F2FS_SLOT_LEN);
- if (!folio_test_uptodate(folio))
- folio_mark_uptodate(folio);
- folio_mark_dirty(folio);
-
/* clear inline dir and flag after data writeback */
f2fs_truncate_inline_inode(dir, ientry, 0);
@@ -484,6 +480,8 @@ static int f2fs_move_inline_dirents(struct inode *dir,
f2fs_i_depth_write(dir, 1);
if (i_size_read(dir) < F2FS_BLKSIZE(F2FS_I_SB(dir)))
f2fs_i_size_write(dir, F2FS_BLKSIZE(F2FS_I_SB(dir)));
+
+ f2fs_mark_dentry_folio_dirty(dir, folio, 0);
out:
f2fs_folio_put(folio, true);
return err;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 05/20] f2fs: convert inode dirty page counters to dirty block counters
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (3 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
` (14 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The per-inode dirty counter is compared against block based thresholds
such as min_fsync_blocks and BLKS_PER_SEG, so with subpage blocks it
has to count blocks instead of pages. Rename it to dirty_blocks, make
its helpers take block counts, and convert to pages where it is
compared with page based values. The conversions are no-ops when the
block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/checkpoint.c | 4 ++--
fs/f2fs/data.c | 13 +++++++------
fs/f2fs/f2fs.h | 36 +++++++++++++++++-------------------
fs/f2fs/file.c | 16 ++++++++--------
fs/f2fs/inode.c | 2 +-
fs/f2fs/node.c | 3 ++-
fs/f2fs/super.c | 2 +-
7 files changed, 38 insertions(+), 38 deletions(-)
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 158d97fc349a..1060ee651756 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -1316,7 +1316,7 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
{
int flag = (type == DIR_INODE) ? FI_DIRTY_DIR : FI_DIRTY_FILE;
- if (get_dirty_pages(inode) || !is_inode_flag_set(inode, flag))
+ if (get_dirty_blocks(inode) || !is_inode_flag_set(inode, flag))
return;
list_del_init(&F2FS_I(inode)->dirty_list);
@@ -1325,7 +1325,7 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
}
/*
- * Account @nr dirty pages (negative when dirtiness is cleared) for @inode.
+ * Account @nr dirty blocks (negative when dirtiness is cleared) for @inode.
*/
void f2fs_account_dirty_blocks(struct inode *inode, int nr)
{
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index e983b59e4a65..f2254e872a3d 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4282,7 +4282,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
folio_batch_init(&fbatch);
- if (get_dirty_pages(mapping->host) <=
+ if (get_dirty_blocks(mapping->host) <=
SM_I(F2FS_M_SB(mapping))->min_hot_blocks)
set_inode_flag(mapping->host, FI_HOT_DATA);
else
@@ -4527,7 +4527,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
bool op_locked = false;
bool retry = false;
- if (get_dirty_pages(inode) <= SM_I(sbi)->min_hot_blocks)
+ if (get_dirty_blocks(inode) <= SM_I(sbi)->min_hot_blocks)
set_inode_flag(inode, FI_HOT_DATA);
else
clear_inode_flag(inode, FI_HOT_DATA);
@@ -4791,7 +4791,7 @@ static inline bool __should_serialize_io(struct inode *inode,
return true;
if (wbc->sync_mode != WB_SYNC_ALL)
return true;
- if (get_dirty_pages(inode) >= SM_I(F2FS_I_SB(inode))->min_seq_blocks)
+ if (get_dirty_blocks(inode) >= SM_I(F2FS_I_SB(inode))->min_seq_blocks)
return true;
return false;
}
@@ -4830,7 +4830,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
bool locked = false;
/* skip writing if there is no dirty page in this inode */
- if (!get_dirty_pages(inode) && wbc->sync_mode == WB_SYNC_NONE)
+ if (!get_dirty_blocks(inode) && wbc->sync_mode == WB_SYNC_NONE)
return 0;
/* during POR, we don't need to trigger writepage at all. */
@@ -4839,7 +4839,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
if ((S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode)) &&
wbc->sync_mode == WB_SYNC_NONE &&
- get_dirty_pages(inode) < nr_caches_to_skip(sbi, DATA) &&
+ get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
f2fs_available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
@@ -4898,7 +4898,8 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
return ret;
skip_write:
- wbc->pages_skipped += get_dirty_pages(inode);
+ wbc->pages_skipped += get_dirty_blocks(inode) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
update_skipped_write(sbi, wbc);
trace_f2fs_writepages(mapping->host, wbc, DATA);
return 0;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 3822824326d7..bc67416a6663 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -957,7 +957,7 @@ struct f2fs_inode_info {
unsigned long flags[BITS_TO_LONGS(FI_MAX)]; /* use to pass per-file flags */
unsigned int ioprio_hint; /* hint for IO priority */
struct f2fs_rwsem i_sem; /* protect fi info */
- atomic_t dirty_pages; /* # of dirty pages */
+ atomic_t dirty_blocks; /* # of dirty blocks */
f2fs_hash_t chash; /* hash value of given file name */
unsigned int clevel; /* maximum level of given file name */
struct task_struct *task; /* lookup and create consistency */
@@ -2336,6 +2336,8 @@ static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
return folio_size(folio) >> sbi->log_blocksize;
}
+#define F2FS_LOG_BLKS_PER_PAGE(sbi) (PAGE_SHIFT - (sbi)->log_blocksize)
+
static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
{
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
@@ -2979,40 +2981,36 @@ static inline void dec_cache_count(struct f2fs_sb_info *sbi, int count_type)
sub_cache_counts(sbi, count_type, 1);
}
-/*
- * Account @nr dirty pages for @inode; one page is
- * (PAGE_SIZE >> log_blocksize) filesystem blocks.
- */
-static inline void inode_add_dirty_blocks(struct inode *inode, int nr)
+static inline void inode_add_dirty_blocks(struct inode *inode,
+ unsigned int nr_blocks)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize);
- atomic_add(blocks, &F2FS_I(inode)->dirty_pages);
+ atomic_add(nr_blocks, &F2FS_I(inode)->dirty_blocks);
if (S_ISDIR(inode->i_mode))
- add_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_DENTS, nr_blocks);
else
- add_cache_counts(sbi, F2FS_DIRTY_DATA, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_DATA, nr_blocks);
if (IS_NOQUOTA(inode))
- add_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks);
+ add_cache_counts(sbi, F2FS_DIRTY_QDATA, nr_blocks);
}
-static inline void inode_sub_dirty_blocks(struct inode *inode, int nr)
+static inline void inode_sub_dirty_blocks(struct inode *inode,
+ unsigned int nr_blocks)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize);
if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
!S_ISLNK(inode->i_mode))
return;
- atomic_sub(blocks, &F2FS_I(inode)->dirty_pages);
+ atomic_sub(nr_blocks, &F2FS_I(inode)->dirty_blocks);
if (S_ISDIR(inode->i_mode))
- sub_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_DENTS, nr_blocks);
else
- sub_cache_counts(sbi, F2FS_DIRTY_DATA, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_DATA, nr_blocks);
if (IS_NOQUOTA(inode))
- sub_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks);
+ sub_cache_counts(sbi, F2FS_DIRTY_QDATA, nr_blocks);
}
static inline void inode_inc_dirty_block(struct inode *inode)
@@ -3052,9 +3050,9 @@ static inline s64 get_nr_caches(struct f2fs_sb_info *sbi, int count_type)
return atomic_read(&sbi->nr_caches[count_type]);
}
-static inline int get_dirty_pages(struct inode *inode)
+static inline int get_dirty_blocks(struct inode *inode)
{
- return atomic_read(&F2FS_I(inode)->dirty_pages);
+ return atomic_read(&F2FS_I(inode)->dirty_blocks);
}
static inline int get_blocktype_secs(struct f2fs_sb_info *sbi, int block_type)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index e339cfd84b27..97c052b98559 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -376,7 +376,7 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
goto go_write;
/* if fdatasync is triggered, let's do in-place-update */
- if (datasync || get_dirty_pages(inode) <= SM_I(sbi)->min_fsync_blocks)
+ if (datasync || get_dirty_blocks(inode) <= SM_I(sbi)->min_fsync_blocks)
set_inode_flag(inode, FI_NEED_IPU);
ret = file_write_and_wait_range(file, start, end);
clear_inode_flag(inode, FI_NEED_IPU);
@@ -2495,9 +2495,9 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
* Should wait end_io to count F2FS_WB_CP_DATA correctly by
* f2fs_is_atomic_file.
*/
- if (get_dirty_pages(inode))
- f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%llu, npages=%u",
- inode->i_ino, get_dirty_pages(inode));
+ if (get_dirty_blocks(inode))
+ f2fs_warn(sbi, "Unexpected flush for atomic writes: ino=%llu, nblocks=%u",
+ inode->i_ino, get_dirty_blocks(inode));
ret = filemap_write_and_wait_range(inode->i_mapping, 0, LLONG_MAX);
if (ret)
goto out_unlock;
@@ -2519,7 +2519,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
F2FS_I(fi->cow_inode)->atomic_inode = inode;
} else {
/* Reuse the already created COW inode */
- f2fs_bug_on(sbi, get_dirty_pages(fi->cow_inode));
+ f2fs_bug_on(sbi, get_dirty_blocks(fi->cow_inode));
invalidate_mapping_pages(fi->cow_inode->i_mapping, 0, -1);
@@ -4877,7 +4877,7 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
goto out;
}
- if (f2fs_is_mmap_file(inode) || get_dirty_pages(inode)) {
+ if (f2fs_is_mmap_file(inode) || get_dirty_blocks(inode)) {
ret = -EBUSY;
goto out;
}
@@ -5033,7 +5033,7 @@ static int f2fs_ioc_decompress_file(struct file *filp)
if (ret < 0)
break;
- if (get_dirty_pages(inode) >= BLKS_PER_SEG(sbi)) {
+ if (get_dirty_blocks(inode) >= BLKS_PER_SEG(sbi)) {
ret = filemap_fdatawrite(inode->i_mapping);
if (ret < 0)
break;
@@ -5123,7 +5123,7 @@ static int f2fs_ioc_compress_file(struct file *filp)
if (ret < 0)
break;
- if (get_dirty_pages(inode) >= BLKS_PER_SEG(sbi)) {
+ if (get_dirty_blocks(inode) >= BLKS_PER_SEG(sbi)) {
ret = filemap_fdatawrite(inode->i_mapping);
if (ret < 0)
break;
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index f5a4dac98ef2..5c6a64b31b05 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -872,7 +872,7 @@ static void f2fs_pre_evict_inode(struct inode *inode)
test_opt(sbi, COMPRESS_CACHE) && f2fs_compressed_file(inode))
f2fs_invalidate_compress_pages(sbi, inode->i_ino);
- f2fs_bug_on(sbi, get_dirty_pages(inode));
+ f2fs_bug_on(sbi, get_dirty_blocks(inode));
f2fs_remove_dirty_inode(inode);
f2fs_remove_donate_inode(inode);
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index e53871f32c35..cb8affb2c1e0 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -85,7 +85,8 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
} else if (type == DIRTY_DENTS) {
if (bdi_wb_dirty_exceeded(sbi->sb->s_bdi))
return false;
- mem_size = get_nr_caches(sbi, F2FS_DIRTY_DENTS);
+ mem_size = get_nr_caches(sbi, F2FS_DIRTY_DENTS) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
res = mem_size < ((avail_ram * nm_i->ram_thresh / 100) >> 1);
} else if (type == INO_ENTRIES) {
int i;
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index f34558b661eb..fa74564adcfb 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1855,7 +1855,7 @@ static struct inode *f2fs_alloc_inode(struct super_block *sb)
init_once((void *) fi);
/* Initialize f2fs-specific inode info */
- atomic_set(&fi->dirty_pages, 0);
+ atomic_set(&fi->dirty_blocks, 0);
atomic_set(&fi->i_compr_blocks, 0);
atomic_set(&fi->open_count, 0);
atomic_set(&fi->writeback, 0);
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 06/20] f2fs: support subpage blocks in data read path and GC
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (4 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
` (13 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a folio can be partially uptodate, so reads and GC
have to work per block.
- Read only the non-uptodate blocks of a multi-block folio, and send
every inode to this path in subpage mode.
- Do not use the folio state when a single bio reads the whole folio.
- Implement ->is_partially_uptodate.
- Convert block indices to folio indices in the data folio getters and
GC, and prefetch the whole folio in GC so that its blocks are read
together.
- Fix error handling in f2fs_read_data_large_folio().
- Zero the blocks past EOF in f2fs_read_data_large_folio() instead of
reading them, as f2fs_read_single_page() does. With subpage blocks,
they share a page with the last block of the file and can be seen
through mmap.
- Test the per-block uptodate state in f2fs_find_data_folio() under the
folio lock, as ->release_folio can free the folio state otherwise. A
fully uptodate folio is still returned without the lock.
- Get sbi from the folio in f2fs_finish_read_bio() only after the
compressed page check, as nothing pins the inode of a compressed page
once its cluster is unlocked.
- Rename helpers that now take block indices, e.g.
f2fs_submit_page_read() to f2fs_submit_block_read().
The read changes also apply to large folios when the block size equals
PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 389 +++++++++++++++++++++++++++++--------------------
fs/f2fs/f2fs.h | 20 ++-
fs/f2fs/file.c | 22 +--
fs/f2fs/gc.c | 82 ++++++++---
fs/f2fs/node.c | 6 +-
5 files changed, 319 insertions(+), 200 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index f2254e872a3d..07b7aeb94350 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -145,13 +145,8 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
bio_for_each_folio_all(fi, bio) {
struct folio *folio = fi.folio;
- /*
- * Cache sbi before the pending count is dropped: a sync reader
- * may then release the folio while this bio is still in its
- * end_io, so F2FS_F_SB(folio) is no longer safe to use below.
- */
- struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- unsigned nr_pages = fi.length >> PAGE_SHIFT;
+ struct f2fs_sb_info *sbi;
+ unsigned int nr_blocks;
bool finished = true;
bool uptodate = bio->bi_status == BLK_STS_OK;
@@ -164,6 +159,10 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
continue;
}
+ sbi = F2FS_F_SB(folio);
+ nr_blocks = fi.length >> sbi->log_blocksize;
+ sub_cache_counts(sbi, F2FS_RD_DATA, nr_blocks);
+
if (f2fs_folio_has_ffs(folio)) {
struct f2fs_folio_state *ffs =
(struct f2fs_folio_state *)folio->private;
@@ -172,14 +171,11 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
if (uptodate)
uptodate = __ffs_mark_subrange_uptodate(folio, ffs,
fi.offset, fi.length);
- ffs->read_blocks_pending -= nr_pages;
+ ffs->read_blocks_pending -= nr_blocks;
finished = !ffs->read_blocks_pending;
spin_unlock_irqrestore(&ffs->state_lock, flags);
}
- while (nr_pages--)
- dec_cache_count(sbi, F2FS_RD_DATA);
-
if (finished)
folio_end_read(folio, uptodate);
}
@@ -1477,7 +1473,7 @@ void f2fs_submit_cache_write(struct f2fs_io_info *fio)
static struct bio *f2fs_grab_read_bio(struct inode *inode,
struct fsverity_info *vi, block_t blkaddr,
- unsigned nr_pages, blk_opf_t op_flag,
+ unsigned int nr_blocks, blk_opf_t op_flag,
pgoff_t first_idx, bool for_write)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1487,7 +1483,7 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode,
sector_t sector;
struct block_device *bdev = f2fs_target_device(sbi, blkaddr, §or);
- bio = bio_alloc_bioset(bdev, bio_max_segs(nr_pages),
+ bio = bio_alloc_bioset(bdev, bio_max_segs(nr_blocks),
REQ_OP_READ | op_flag,
for_write ? GFP_NOIO : GFP_KERNEL, &f2fs_bioset);
bio->bi_iter.bi_sector = sector;
@@ -1521,27 +1517,29 @@ static struct bio *f2fs_grab_read_bio(struct inode *inode,
}
/* This can handle encryption stuffs */
-static void f2fs_submit_page_read(struct inode *inode, struct fsverity_info *vi,
- struct folio *folio, pgoff_t index,
- block_t blkaddr, blk_opf_t op_flags,
- bool for_write)
+static void f2fs_submit_block_read(struct inode *inode,
+ struct fsverity_info *vi,
+ struct folio *folio, pgoff_t bidx,
+ block_t blkaddr, blk_opf_t op_flags,
+ bool for_write)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct bio *bio;
size_t offset = 0;
- if (f2fs_folio_has_ffs(folio)) {
- offset = offset_in_folio(folio, (loff_t)index << PAGE_SHIFT);
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ f2fs_ffs_find_or_alloc(folio);
+ offset = f2fs_folio_byte_offset(sbi, folio, bidx);
f2fs_update_read_folio_pending(folio, 1);
}
- bio = f2fs_grab_read_bio(inode, vi, blkaddr, 1, op_flags, index,
+ bio = f2fs_grab_read_bio(inode, vi, blkaddr, 1, op_flags, bidx,
for_write);
/* wait for GCed page writeback via generic cache */
f2fs_wait_on_block_writeback(inode, blkaddr);
- if (!bio_add_folio(bio, folio, PAGE_SIZE, offset))
+ if (!bio_add_folio(bio, folio, F2FS_BLKSIZE(sbi), offset))
f2fs_bug_on(sbi, 1);
inc_cache_count(sbi, F2FS_RD_DATA);
@@ -1550,7 +1548,7 @@ static void f2fs_submit_page_read(struct inode *inode, struct fsverity_info *vi,
}
/*
- * Synchronously read a single 4K subpage by reusing f2fs_submit_page_read()
+ * Synchronously read a single block by reusing f2fs_submit_block_read()
* so that iostat, trace, blk-crypto and post-read handling are all preserved.
* The caller must have already allocated ffs for the folio.
*/
@@ -1567,8 +1565,8 @@ static int f2fs_submit_page_read_sync(struct inode *inode, struct folio *folio,
* We are holding the folio lock, so use GFP_NOIO for the bio
* allocation to avoid entering mm reclaim.
*/
- f2fs_submit_page_read(inode, NULL, folio, index, blkaddr,
- REQ_OP_READ, true);
+ f2fs_submit_block_read(inode, NULL, folio, index, blkaddr,
+ REQ_OP_READ, true);
/* Wait until all bios have completed (pending drops back to our bias). */
while (READ_ONCE(ffs->read_blocks_pending) != 1)
@@ -1680,24 +1678,24 @@ static inline struct fsverity_info *f2fs_need_verity(const struct inode *inode,
return NULL;
}
-struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
- blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs)
+struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t bidx,
+ blk_opf_t op_flags, bool for_write, pgoff_t *next_blkofs)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_has_subpage_blocks(sbi) ?
+ f2fs_lblk_to_folio_index(sbi, bidx) : bidx;
struct dnode_of_data dn;
struct folio *folio;
int err;
- folio = f2fs_grab_cache_folio(mapping, index, for_write);
+ folio = f2fs_grab_cache_folio(mapping, f_index, for_write);
if (IS_ERR(folio))
return folio;
- if (folio_test_large(folio))
- f2fs_ffs_find_or_alloc(folio);
-
- if (f2fs_lookup_read_extent_cache_block(inode, index,
+ if (f2fs_lookup_read_extent_cache_block(inode, bidx,
&dn.data_blkaddr)) {
- if (!f2fs_is_valid_blkaddr(F2FS_I_SB(inode), dn.data_blkaddr,
+ if (!f2fs_is_valid_blkaddr(sbi, dn.data_blkaddr,
DATA_GENERIC_ENHANCE_READ)) {
err = -EFSCORRUPTED;
goto put_err;
@@ -1706,29 +1704,29 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
}
set_new_dnode(&dn, inode, NULL, NULL, 0);
- err = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
+ err = f2fs_get_dnode_of_data(&dn, bidx, LOOKUP_NODE);
if (err) {
- if (err == -ENOENT && next_pgofs)
- *next_pgofs = f2fs_get_next_page_offset(&dn, index);
+ if (err == -ENOENT && next_blkofs)
+ *next_blkofs = f2fs_get_next_block_offset(&dn, bidx);
goto put_err;
}
f2fs_put_dnode(&dn);
if (unlikely(dn.data_blkaddr == NULL_ADDR)) {
err = -ENOENT;
- if (next_pgofs)
- *next_pgofs = index + 1;
+ if (next_blkofs)
+ *next_blkofs = bidx + 1;
goto put_err;
}
if (dn.data_blkaddr != NEW_ADDR &&
- !f2fs_is_valid_blkaddr(F2FS_I_SB(inode),
+ !f2fs_is_valid_blkaddr(sbi,
dn.data_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
goto put_err;
}
got_it:
- if (f2fs_ffs_test_blk_uptodate(folio, index)) {
+ if (f2fs_ffs_test_blk_uptodate(folio, bidx)) {
folio_unlock(folio);
return folio;
}
@@ -1741,17 +1739,17 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
* f2fs_init_inode_metadata.
*/
if (dn.data_blkaddr == NEW_ADDR) {
- size_t offset = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t offset = f2fs_folio_byte_offset(sbi, folio, bidx);
- folio_zero_segment(folio, offset, offset + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, offset, PAGE_SIZE);
+ folio_zero_segment(folio, offset, offset + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, offset, F2FS_BLKSIZE(sbi));
folio_unlock(folio);
return folio;
}
- f2fs_submit_page_read(inode, f2fs_need_verity(inode, index),
- folio, index, dn.data_blkaddr, op_flags, for_write);
+ f2fs_submit_block_read(inode, f2fs_need_verity(inode, bidx),
+ folio, bidx, dn.data_blkaddr, op_flags,
+ for_write);
return folio;
put_err:
@@ -1759,32 +1757,44 @@ struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
return ERR_PTR(err);
}
-struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
- pgoff_t *next_pgofs)
+struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t bidx,
+ pgoff_t *next_blkofs)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_has_subpage_blocks(sbi) ?
+ f2fs_lblk_to_folio_index(sbi, bidx) : bidx;
struct folio *folio;
- folio = f2fs_filemap_get_folio(mapping, index, FGP_ACCESSED, 0);
+ folio = f2fs_filemap_get_folio(mapping, f_index, FGP_ACCESSED, 0);
if (IS_ERR(folio))
goto read;
if (folio_test_uptodate(folio))
return folio;
- f2fs_folio_put(folio, false);
+ /* The folio state is freed under the folio lock. */
+ folio_lock(folio);
+ if (folio->mapping == mapping &&
+ f2fs_ffs_test_blk_uptodate(folio, bidx)) {
+ folio_unlock(folio);
+ return folio;
+ }
+ f2fs_folio_put(folio, true);
read:
- folio = f2fs_get_read_data_folio(inode, index, 0, false, next_pgofs);
+ folio = f2fs_get_read_data_folio(inode, bidx, 0, false, next_blkofs);
if (IS_ERR(folio))
return folio;
if (folio_test_uptodate(folio))
return folio;
- folio_wait_locked(folio);
- if (unlikely(!folio_test_uptodate(folio))) {
- f2fs_folio_put(folio, false);
+ folio_lock(folio);
+ if (unlikely(folio->mapping != mapping ||
+ !f2fs_ffs_test_blk_uptodate(folio, bidx))) {
+ f2fs_folio_put(folio, true);
return ERR_PTR(-EIO);
}
+ folio_unlock(folio);
return folio;
}
@@ -1793,13 +1803,13 @@ struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
* Because, the callers, functions in dir.c and GC, should be able to know
* whether this page exists or not.
*/
-struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
+struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t bidx,
bool for_write)
{
struct address_space *mapping = inode->i_mapping;
struct folio *folio;
- folio = f2fs_get_read_data_folio(inode, index, 0, for_write, NULL);
+ folio = f2fs_get_read_data_folio(inode, bidx, 0, for_write, NULL);
if (IS_ERR(folio))
return folio;
@@ -1819,7 +1829,7 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
if (folio_test_large(folio) && !f2fs_folio_has_ffs(folio))
f2fs_ffs_find_or_alloc(folio);
- if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, index))) {
+ if (unlikely(!f2fs_ffs_test_blk_uptodate(folio, bidx))) {
f2fs_folio_put(folio, true);
return ERR_PTR(-EIO);
}
@@ -1838,12 +1848,15 @@ struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
struct folio *f2fs_get_new_data_folio(struct inode *inode,
struct f2fs_cached_block *ientry, pgoff_t index, bool new_i_size)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
+ pgoff_t f_index = f2fs_has_subpage_blocks(sbi) ?
+ f2fs_lblk_to_folio_index(sbi, index) : index;
struct folio *folio;
struct dnode_of_data dn;
int err;
- folio = f2fs_grab_cache_folio(mapping, index, true);
+ folio = f2fs_grab_cache_folio(mapping, f_index, true);
if (IS_ERR(folio)) {
/*
* before exiting, we should make sure ientry will be released
@@ -1869,25 +1882,24 @@ struct folio *f2fs_get_new_data_folio(struct inode *inode,
goto got_it;
if (dn.data_blkaddr == NEW_ADDR) {
- size_t off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t off = f2fs_folio_byte_offset(sbi, folio, index);
- folio_zero_segment(folio, off, off + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, off, PAGE_SIZE);
+ folio_zero_segment(folio, off, off + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, off, F2FS_BLKSIZE(sbi));
} else {
f2fs_folio_put(folio, true);
/* if ientry exists, blkaddr should be NEW_ADDR */
- f2fs_bug_on(F2FS_I_SB(inode), ientry);
+ f2fs_bug_on(sbi, ientry);
folio = f2fs_get_lock_data_folio(inode, index, true);
if (IS_ERR(folio))
return folio;
}
got_it:
if (new_i_size && i_size_read(inode) <
- F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1))
+ F2FS_BLK_TO_BYTES(sbi, index + 1))
f2fs_i_size_write(inode,
- F2FS_BLK_TO_BYTES(F2FS_I_SB(inode), index + 1));
+ F2FS_BLK_TO_BYTES(sbi, index + 1));
return folio;
}
@@ -1968,7 +1980,7 @@ int f2fs_get_block_locked(struct dnode_of_data *dn, pgoff_t index)
static int f2fs_map_no_dnode(struct inode *inode,
struct f2fs_map_blocks *map, struct dnode_of_data *dn,
- pgoff_t pgoff)
+ pgoff_t blkofs)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1981,10 +1993,10 @@ static int f2fs_map_no_dnode(struct inode *inode,
(is_sbi_flag_set(sbi, SBI_IS_SHUTDOWN) || f2fs_cp_error(sbi)))
return -EIO;
- if (map->m_next_pgofs)
- *map->m_next_pgofs = f2fs_get_next_page_offset(dn, pgoff);
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = f2fs_get_next_block_offset(dn, blkofs);
if (map->m_next_extent)
- *map->m_next_extent = f2fs_get_next_page_offset(dn, pgoff);
+ *map->m_next_extent = f2fs_get_next_block_offset(dn, blkofs);
return 0;
}
@@ -2191,19 +2203,19 @@ int f2fs_map_blocks(struct inode *inode, struct f2fs_map_blocks *map, int flag)
goto sync_out;
case F2FS_GET_BLOCK_FIEMAP:
if (blkaddr == NULL_ADDR) {
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
goto sync_out;
}
break;
case F2FS_GET_BLOCK_DIO:
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
break;
default:
/* for defragment case */
- if (map->m_next_pgofs)
- *map->m_next_pgofs = pgofs + 1;
+ if (map->m_next_blkofs)
+ *map->m_next_blkofs = pgofs + 1;
goto sync_out;
}
}
@@ -2344,7 +2356,7 @@ static bool __f2fs_overwrite_io(struct inode *inode, loff_t pos, size_t len,
return false;
map.m_lblk = F2FS_BYTES_TO_BLK(F2FS_I_SB(inode), pos);
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -2447,7 +2459,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_map_blocks map;
sector_t start_blk, last_blk, blk_len, max_len;
- pgoff_t next_pgofs;
+ pgoff_t next_blkofs;
u64 logical = 0, phys = 0, size = 0;
u32 flags = 0;
int ret = 0;
@@ -2497,7 +2509,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
memset(&map, 0, sizeof(map));
map.m_lblk = start_blk;
map.m_len = blk_len;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = NO_CHECK_TYPE;
if (compr_cluster) {
@@ -2511,7 +2523,7 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
/* HOLE */
if (!compr_cluster && !(map.m_flags & F2FS_MAP_FLAGS)) {
- start_blk = next_pgofs;
+ start_blk = next_blkofs;
if (F2FS_BLK_TO_BYTES(sbi, start_blk) < maxbytes)
goto prep_next;
@@ -2647,7 +2659,7 @@ static int f2fs_read_single_page(struct inode *inode, struct fsverity_info *vi,
if (block_in_file > map->m_lblk &&
block_in_file < (map->m_lblk + map->m_len))
goto got_it;
- } else if (block_in_file < *map->m_next_pgofs) {
+ } else if (block_in_file < *map->m_next_blkofs) {
goto got_it;
}
@@ -3212,23 +3224,26 @@ static bool find_next_valid_block(const struct folio *folio,
size_t orig_off, size_t *need_off,
size_t len)
{
+ struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
size_t start = orig_off;
size_t end = start + len;
size_t head, tail;
pgoff_t index;
- if (start & (PAGE_SIZE - 1)) {
- head = round_down(start, PAGE_SIZE);
- index = folio->index + (head >> PAGE_SHIFT);
+ if (start & (F2FS_BLKSIZE(sbi) - 1)) {
+ head = round_down(start, F2FS_BLKSIZE(sbi));
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, head);
if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
*need_off = head;
return true;
}
}
- if (end & (PAGE_SIZE - 1)) {
- tail = round_down(end, PAGE_SIZE);
- index = folio->index + (tail >> PAGE_SHIFT);
+ if (end & (F2FS_BLKSIZE(sbi) - 1)) {
+ tail = round_down(end, F2FS_BLKSIZE(sbi));
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, tail);
if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
*need_off = tail;
return true;
@@ -3251,26 +3266,34 @@ static unsigned int ffs_next_nonuptodate_subpage(struct f2fs_folio_state *ffs,
}
static void f2fs_skip_fully_uptodate_front(struct folio *folio,
- pgoff_t *index, pgoff_t *offset, unsigned int *nrpages,
- unsigned int *max_nr_pages)
+ pgoff_t *index, pgoff_t *offset, unsigned int *nr_blocks,
+ unsigned int *max_nr_blocks)
{
struct f2fs_folio_state *ffs;
unsigned int next, skipped;
- if (!f2fs_folio_has_ffs(folio) || !*nrpages)
+ if (folio_test_uptodate(folio)) {
+ *index += *nr_blocks;
+ *offset += *nr_blocks;
+ *max_nr_blocks -= *nr_blocks;
+ *nr_blocks = 0;
+ return;
+ }
+
+ if (!f2fs_folio_has_ffs(folio) || !*nr_blocks)
return;
ffs = (struct f2fs_folio_state *)folio->private;
next = ffs_next_nonuptodate_subpage(ffs, *offset,
- *offset + *nrpages - 1);
+ *offset + *nr_blocks - 1);
skipped = next - *offset;
if (!skipped)
return;
*index += skipped;
*offset += skipped;
- *nrpages -= skipped;
- *max_nr_pages -= skipped;
+ *nr_blocks -= skipped;
+ *max_nr_blocks -= skipped;
}
static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
@@ -3281,7 +3304,7 @@ static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
struct f2fs_folio_state *ffs;
unsigned int next, end;
- if (!f2fs_folio_has_ffs(folio) || *len_blks <= 1)
+ if (*len_blks <= 1)
return;
if (f2fs_is_multi_device(sbi)) {
@@ -3292,8 +3315,11 @@ static void f2fs_truncate_read_extent(struct inode *inode, struct folio *folio,
}
*len_blks = min_t(unsigned int, *len_blks,
(unsigned int)fscrypt_limit_io_blocks(inode,
- folio->index + offset,
- *len_blks));
+ f2fs_folio_lblk(sbi, folio) + offset,
+ *len_blks));
+
+ if (!f2fs_folio_has_ffs(folio) || *len_blks <= 1)
+ return;
ffs = (struct f2fs_folio_state *)folio->private;
end = offset + *len_blks - 1;
@@ -3306,13 +3332,16 @@ static int f2fs_read_data_large_folio(struct inode *inode,
struct fsverity_info *vi,
struct readahead_control *rac, struct folio *folio)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct bio *bio = NULL;
sector_t last_block_in_bio = 0;
struct f2fs_map_blocks map = {0, };
- pgoff_t index, offset, next_pgofs = 0;
- unsigned max_nr_pages = rac ? readahead_count(rac) :
- folio_nr_pages(folio);
- unsigned nrpages, len_blks;
+ pgoff_t index, offset, next_blkofs = 0;
+ unsigned int max_nr_blocks = rac ?
+ (readahead_count(rac) * F2FS_BLKS_PER_PAGE(sbi)) :
+ f2fs_blocks_per_folio(sbi, folio);
+ unsigned int nr_blocks, len_blks;
+ sector_t last_block_in_file;
struct f2fs_folio_state *ffs;
int ret = 0;
bool folio_in_bio = false;
@@ -3325,35 +3354,32 @@ static int f2fs_read_data_large_folio(struct inode *inode,
if (!folio)
goto out;
- f2fs_update_read_folio_count(F2FS_I_SB(inode), folio);
+ f2fs_update_read_folio_count(sbi, folio);
folio_in_bio = false;
- ffs = NULL;
- if (folio_test_large(folio)) {
- /*
- * Allocate the folio state before doing anything with the
- * folio, so that the block mapping helpers below can rely on
- * it.
- */
- ffs = f2fs_ffs_find_or_alloc(folio);
- }
- index = folio->index;
+ index = f2fs_folio_lblk(sbi, folio);
offset = 0;
- nrpages = folio_nr_pages(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ last_block_in_file = F2FS_BYTES_TO_BLK(sbi, f2fs_readpage_limit(inode) +
+ F2FS_BLKSIZE(sbi) - 1);
- for (; nrpages;
- nrpages -= len_blks, max_nr_pages -= len_blks,
+ for (; nr_blocks;
+ nr_blocks -= len_blks, max_nr_blocks -= len_blks,
index += len_blks, offset += len_blks) {
sector_t block_nr;
+ bool whole_folio_in_bio;
unsigned int i;
+ size_t page_offset;
len_blks = 1;
f2fs_skip_fully_uptodate_front(folio, &index, &offset,
- &nrpages, &max_nr_pages);
- if (!nrpages)
+ &nr_blocks, &max_nr_blocks);
+ if (!nr_blocks)
break;
+ page_offset = F2FS_BLK_TO_BYTES(sbi, offset);
+
/*
* Map blocks using the previous result first.
*/
@@ -3361,7 +3387,7 @@ static int f2fs_read_data_large_folio(struct inode *inode,
if (index > map.m_lblk &&
index < (map.m_lblk + map.m_len))
goto got_it;
- } else if (index < next_pgofs) {
+ } else if (index < next_blkofs) {
/* hole case */
goto got_it;
}
@@ -3371,26 +3397,30 @@ static int f2fs_read_data_large_folio(struct inode *inode,
* done with this page.
*/
memset(&map, 0, sizeof(map));
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = NO_CHECK_TYPE;
map.m_lblk = index;
- map.m_len = max_nr_pages;
+ map.m_len = max_nr_blocks;
ret = f2fs_map_blocks(inode, &map, F2FS_GET_BLOCK_DEFAULT);
if (ret)
goto err_out;
got_it:
- if ((map.m_flags & F2FS_MAP_MAPPED)) {
+ /* Zero the blocks past EOF, as f2fs_read_single_page() does. */
+ if ((map.m_flags & F2FS_MAP_MAPPED) &&
+ index < last_block_in_file) {
block_nr = map.m_pblk + index - map.m_lblk;
- len_blks = min_t(unsigned int, nrpages, max_nr_pages);
+ len_blks = min_t(unsigned int, nr_blocks, max_nr_blocks);
len_blks = min_t(unsigned int, len_blks,
(unsigned int)(map.m_lblk + map.m_len - index));
+ len_blks = min_t(unsigned int, len_blks,
+ last_block_in_file - index);
f2fs_truncate_read_extent(inode, folio, offset,
block_nr, &len_blks);
for (i = 0; i < len_blks; i++) {
- if (!f2fs_is_valid_blkaddr(F2FS_I_SB(inode),
+ if (!f2fs_is_valid_blkaddr(sbi,
block_nr + i,
DATA_GENERIC_ENHANCE_READ)) {
ret = -EFSCORRUPTED;
@@ -3398,18 +3428,21 @@ static int f2fs_read_data_large_folio(struct inode *inode,
}
}
+ whole_folio_in_bio = !f2fs_folio_has_ffs(folio) &&
+ offset == 0 &&
+ len_blks == f2fs_blocks_per_folio(sbi, folio);
} else {
- size_t page_offset = offset << PAGE_SHIFT;
-
- folio_zero_range(folio, page_offset, PAGE_SIZE);
- if (vi && !fsverity_verify_blocks(vi, folio, PAGE_SIZE, page_offset)) {
+ folio_zero_range(folio, page_offset, F2FS_BLKSIZE(sbi));
+ if (vi && !fsverity_verify_blocks(vi, folio,
+ F2FS_BLKSIZE(sbi), page_offset)) {
ret = -EIO;
goto err_out;
}
- if (folio_test_large(folio)) {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irq(&ffs->state_lock);
__ffs_mark_subrange_uptodate(folio, ffs,
- page_offset, PAGE_SIZE);
+ page_offset, F2FS_BLKSIZE(sbi));
spin_unlock_irq(&ffs->state_lock);
}
continue;
@@ -3418,7 +3451,10 @@ static int f2fs_read_data_large_folio(struct inode *inode,
/* We must increment read_blocks_pending before possible BIOs submitting
* to prevent from premature folio_end_read() call on folio
*/
- if (folio_test_large(folio)) {
+ ffs = NULL;
+ if (f2fs_is_multiblock_folio(sbi, folio) && !whole_folio_in_bio) {
+ ffs = f2fs_ffs_find_or_alloc(folio);
+
/* set the bitmap to wait */
f2fs_update_read_folio_pending(folio, len_blks);
}
@@ -3427,16 +3463,16 @@ static int f2fs_read_data_large_folio(struct inode *inode,
* This page will go to BIO. Do we need to send this
* BIO off first?
*/
- if (bio && (!page_is_mergeable(F2FS_I_SB(inode), bio,
+ if (bio && (!page_is_mergeable(sbi, bio,
last_block_in_bio, block_nr) ||
!f2fs_crypt_mergeable_bio(bio, inode, index, NULL))) {
submit_and_realloc:
- f2fs_submit_read_bio(F2FS_I_SB(inode), bio, DATA);
+ f2fs_submit_read_bio(sbi, bio, DATA);
bio = NULL;
}
if (bio == NULL)
bio = f2fs_grab_read_bio(inode, vi,
- block_nr, max_nr_pages,
+ block_nr, max_nr_blocks,
f2fs_ra_op_flags(rac),
index, false);
@@ -3447,33 +3483,33 @@ static int f2fs_read_data_large_folio(struct inode *inode,
for (i = 0; i < len_blks; i++)
f2fs_wait_on_block_writeback(inode, block_nr + i);
- if (!bio_add_folio(bio, folio,
- len_blks * F2FS_BLKSIZE(F2FS_I_SB(inode)),
- offset << PAGE_SHIFT))
+ if (!bio_add_folio(bio, folio, len_blks * F2FS_BLKSIZE(sbi),
+ page_offset))
goto submit_and_realloc;
folio_in_bio = true;
- for (i = 0; i < len_blks; i++)
- inc_cache_count(F2FS_I_SB(inode), F2FS_RD_DATA);
- f2fs_update_iostat(F2FS_I_SB(inode), NULL, FS_DATA_READ_IO,
- len_blks * F2FS_BLKSIZE(F2FS_I_SB(inode)));
+ add_cache_counts(sbi, F2FS_RD_DATA, len_blks);
+ f2fs_update_iostat(sbi, NULL, FS_DATA_READ_IO,
+ len_blks * F2FS_BLKSIZE(sbi));
last_block_in_bio = block_nr + len_blks - 1;
}
trace_f2fs_read_folio(folio, DATA);
-err_out:
- if (!folio_in_bio)
- folio_end_read(folio, !ret);
- if (ret)
- goto out;
+ if (!folio_in_bio) {
+ if (!folio_test_uptodate(folio))
+ folio_mark_uptodate(folio);
+ folio_unlock(folio);
+ }
if (rac) {
folio = readahead_folio(rac);
goto next_folio;
}
out:
- f2fs_submit_read_bio(F2FS_I_SB(inode), bio, DATA);
- if (ret && folio_in_bio) {
- /* Wait bios and clear uptodate. */
- folio_lock(folio);
+ f2fs_submit_read_bio(sbi, bio, DATA);
+ return ret;
+err_out:
+ f2fs_submit_read_bio(sbi, bio, DATA);
+ /* The read completion unlocks a folio in the bio, not uptodate. */
+ if (!folio_in_bio) {
folio_clear_uptodate(folio);
folio_unlock(folio);
}
@@ -3504,13 +3540,12 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
pgoff_t nc_cluster_idx = NULL_CLUSTER;
pgoff_t index;
#endif
- pgoff_t next_pgofs = 0;
+ pgoff_t next_blkofs = 0;
unsigned nr_pages = rac ? readahead_count(rac) : 1;
- struct address_space *mapping = rac ? rac->mapping : folio->mapping;
unsigned max_nr_pages = nr_pages;
int ret = 0;
- if (mapping_large_folio_support(mapping))
+ if (f2fs_has_multiblock_folios(inode))
return f2fs_read_data_large_folio(inode, vi, rac, folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
@@ -3525,7 +3560,7 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
map.m_lblk = 0;
map.m_len = 0;
map.m_flags = 0;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -5489,9 +5524,9 @@ static int f2fs_write_begin(const struct kiocb *iocb,
* folio's owner; otherwise an unencrypted @inode whose COW inode
* is encrypted hits a NULL ->i_crypt_info on decryption.
*/
- f2fs_submit_page_read(inode,
- NULL, /* can't write to fsverity files */
- folio, index, blkaddr, 0, true);
+ f2fs_submit_block_read(inode,
+ NULL, /* can't write to fsverity files */
+ folio, index, blkaddr, 0, true);
folio_lock(folio);
if (unlikely(folio->mapping != mapping)) {
@@ -5728,7 +5763,7 @@ static sector_t f2fs_bmap(struct address_space *mapping, sector_t block)
memset(&map, 0, sizeof(map));
map.m_lblk = block;
map.m_len = 1;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_seg_type = NO_CHECK_TYPE;
if (!f2fs_map_blocks(inode, &map, F2FS_GET_BLOCK_BMAP))
@@ -5847,7 +5882,7 @@ static int check_swap_activate(struct swap_info_struct *sis,
memset(&map, 0, sizeof(map));
map.m_lblk = cur_lblock;
map.m_len = last_lblock - cur_lblock;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = NULL;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -5987,6 +6022,43 @@ static void f2fs_swap_deactivate(struct file *file)
}
#endif
+static bool f2fs_is_partially_uptodate(struct folio *folio, size_t from,
+ size_t count)
+{
+ struct inode *inode = folio->mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ struct f2fs_folio_state *ffs;
+ unsigned int first, last, i;
+
+ if (folio_test_uptodate(folio))
+ return true;
+
+ if (f2fs_has_inline_data(inode) || !f2fs_is_multiblock_folio(sbi, folio))
+ return false;
+
+ if (!f2fs_folio_has_ffs(folio))
+ return false;
+
+ if (from >= folio_size(folio))
+ return false;
+
+ count = min_t(size_t, folio_size(folio) - from, count);
+ if (!count)
+ return false;
+
+ first = F2FS_BYTES_TO_BLK(sbi, from);
+ last = F2FS_BYTES_TO_BLK(sbi, from + count - 1);
+
+ ffs = folio->private;
+
+ for (i = first; i <= last; i++) {
+ if (!test_bit(i, ffs->state))
+ return false;
+ }
+
+ return true;
+}
+
const struct address_space_operations f2fs_dblock_aops = {
.read_folio = f2fs_read_data_folio,
.readahead = f2fs_readahead,
@@ -5997,6 +6069,7 @@ const struct address_space_operations f2fs_dblock_aops = {
.migrate_folio = filemap_migrate_folio,
.invalidate_folio = f2fs_invalidate_folio,
.release_folio = f2fs_release_folio,
+ .is_partially_uptodate = f2fs_is_partially_uptodate,
.bmap = f2fs_bmap,
.swap_activate = f2fs_swap_activate,
.swap_deactivate = f2fs_swap_deactivate,
@@ -6087,13 +6160,13 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_map_blocks map = { NULL, };
- pgoff_t next_pgofs = 0;
+ pgoff_t next_blkofs = 0;
int err;
map.m_lblk = F2FS_BYTES_TO_BLK(sbi, offset);
map.m_len = F2FS_BYTES_TO_BLK(sbi, offset + length - 1) -
map.m_lblk + 1;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
map.m_seg_type = f2fs_rw_hint_to_seg_type(F2FS_I_SB(inode),
inode->i_write_hint);
if (flags & IOMAP_WRITE && iomap->private) {
@@ -6146,7 +6219,7 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
return -ENOTBLK;
if (map.m_pblk == NULL_ADDR) {
- iomap->length = F2FS_BLK_TO_BYTES(sbi, next_pgofs) -
+ iomap->length = F2FS_BLK_TO_BYTES(sbi, next_blkofs) -
iomap->offset;
iomap->type = IOMAP_HOLE;
} else if (map.m_pblk == NEW_ADDR) {
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index bc67416a6663..cf7294068415 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -836,7 +836,7 @@ struct f2fs_map_blocks {
unsigned int m_len;
unsigned int m_flags;
unsigned long m_last_pblk; /* last allocated block, only used for DIO in LFS mode */
- pgoff_t *m_next_pgofs; /* point next possible non-hole pgofs */
+ pgoff_t *m_next_blkofs; /* point next possible non-hole blkofs */
pgoff_t *m_next_extent; /* point to next possible extent */
int m_seg_type;
bool m_may_create; /* indicate it is from write path */
@@ -2349,6 +2349,12 @@ static inline bool f2fs_is_multiblock_folio(struct f2fs_sb_info *sbi,
return folio_size(folio) > F2FS_BLKSIZE(sbi);
}
+static inline bool f2fs_has_multiblock_folios(struct inode *inode)
+{
+ return mapping_large_folio_support(inode->i_mapping) ||
+ f2fs_has_subpage_blocks(F2FS_I_SB(inode));
+}
+
static inline pgoff_t f2fs_folio_lblk(struct f2fs_sb_info *sbi,
const struct folio *folio)
{
@@ -4240,7 +4246,7 @@ bool f2fs_is_checkpointed_node(struct f2fs_sb_info *sbi, nid_t nid);
bool f2fs_need_inode_block_update(struct f2fs_sb_info *sbi, nid_t ino);
int f2fs_get_node_info(struct f2fs_sb_info *sbi, nid_t nid,
struct node_info *ni, bool checkpoint_context);
-pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs);
+pgoff_t f2fs_get_next_block_offset(struct dnode_of_data *dn, pgoff_t blkofs);
int f2fs_get_dnode_of_data(struct dnode_of_data *dn, pgoff_t index, int mode);
int f2fs_truncate_inode_blocks(struct inode *inode, pgoff_t from);
int f2fs_truncate_xattr_node(struct inode *inode);
@@ -4503,11 +4509,11 @@ int f2fs_reserve_new_blocks(struct dnode_of_data *dn, blkcnt_t count);
int f2fs_reserve_new_block(struct dnode_of_data *dn);
int f2fs_get_block_locked(struct dnode_of_data *dn, pgoff_t index);
int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index);
-struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t index,
- blk_opf_t op_flags, bool for_write, pgoff_t *next_pgofs);
-struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t index,
- pgoff_t *next_pgofs);
-struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t index,
+struct folio *f2fs_get_read_data_folio(struct inode *inode, pgoff_t bidx,
+ blk_opf_t op_flags, bool for_write, pgoff_t *next_blkofs);
+struct folio *f2fs_find_data_folio(struct inode *inode, pgoff_t bidx,
+ pgoff_t *next_blkofs);
+struct folio *f2fs_get_lock_data_folio(struct inode *inode, pgoff_t bidx,
bool for_write);
struct folio *f2fs_get_new_data_folio(struct inode *inode,
struct f2fs_cached_block *ientry, pgoff_t index, bool new_i_size);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 97c052b98559..1ce4553b1d01 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -571,7 +571,7 @@ static loff_t f2fs_seek_block(struct file *file, loff_t offset, int whence)
} else if (err == -ENOENT) {
/* direct node does not exists */
if (whence == SEEK_DATA) {
- pgofs = f2fs_get_next_page_offset(&dn, pgofs);
+ pgofs = f2fs_get_next_block_offset(&dn, pgofs);
continue;
} else {
goto found;
@@ -1356,7 +1356,7 @@ int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
err = f2fs_get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
if (err) {
if (err == -ENOENT) {
- pg_start = f2fs_get_next_page_offset(&dn,
+ pg_start = f2fs_get_next_block_offset(&dn,
pg_start);
continue;
}
@@ -2026,7 +2026,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
loff_t len, int mode)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
- struct f2fs_map_blocks map = { .m_next_pgofs = NULL,
+ struct f2fs_map_blocks map = { .m_next_blkofs = NULL,
.m_next_extent = NULL, .m_seg_type = NO_CHECK_TYPE,
.m_may_create = true };
struct f2fs_gc_control gc_control = { .victim_segno = NULL_SEGNO,
@@ -3095,7 +3095,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
.m_seg_type = NO_CHECK_TYPE,
.m_may_create = false };
struct extent_info ei = {};
- pgoff_t pg_start, pg_end, next_pgofs;
+ pgoff_t pg_start, pg_end, next_blkofs;
unsigned int total = 0, sec_num;
block_t blk_end = 0;
bool fragmented = false;
@@ -3139,7 +3139,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
}
map.m_lblk = pg_start;
- map.m_next_pgofs = &next_pgofs;
+ map.m_next_blkofs = &next_blkofs;
/*
* lookup mapping info in dnode page cache, skip defragmenting if all
@@ -3153,7 +3153,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
goto out;
if (!(map.m_flags & F2FS_MAP_FLAGS)) {
- map.m_lblk = next_pgofs;
+ map.m_lblk = next_blkofs;
continue;
}
@@ -3200,7 +3200,7 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
goto clear_out;
if (!(map.m_flags & F2FS_MAP_FLAGS)) {
- map.m_lblk = next_pgofs;
+ map.m_lblk = next_blkofs;
goto check;
}
@@ -4092,7 +4092,7 @@ int f2fs_precache_extents(struct inode *inode)
map.m_lblk = 0;
map.m_pblk = 0;
- map.m_next_pgofs = NULL;
+ map.m_next_blkofs = NULL;
map.m_next_extent = &m_next_extent;
map.m_seg_type = NO_CHECK_TYPE;
map.m_may_create = false;
@@ -4385,7 +4385,7 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
if (ret) {
f2fs_unlock_op(sbi, &lc);
if (ret == -ENOENT) {
- page_idx = f2fs_get_next_page_offset(&dn,
+ page_idx = f2fs_get_next_block_offset(&dn,
page_idx);
ret = 0;
continue;
@@ -4567,7 +4567,7 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
if (ret) {
f2fs_unlock_op(sbi, &lc);
if (ret == -ENOENT) {
- page_idx = f2fs_get_next_page_offset(&dn,
+ page_idx = f2fs_get_next_block_offset(&dn,
page_idx);
ret = 0;
continue;
@@ -4735,7 +4735,7 @@ static int f2fs_sec_trim_file(struct file *filp, unsigned long arg)
ret = f2fs_get_dnode_of_data(&dn, index, LOOKUP_NODE);
if (ret) {
if (ret == -ENOENT) {
- index = f2fs_get_next_page_offset(&dn, index);
+ index = f2fs_get_next_block_offset(&dn, index);
continue;
}
goto out;
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 337ecf4df945..6c24001325b2 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1246,7 +1246,9 @@ static int ra_data_block(struct inode *inode, pgoff_t index)
}
f2fs_up_read(&F2FS_I(inode)->i_sem);
- folio = f2fs_grab_cache_folio(mapping, index, true);
+ pgoff_t f_index = f2fs_has_subpage_blocks(sbi) ?
+ f2fs_lblk_to_folio_index(sbi, index) : index;
+ folio = f2fs_grab_cache_folio(mapping, f_index, true);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
goto out_iput;
@@ -1323,6 +1325,30 @@ static int ra_data_block(struct inode *inode, pgoff_t index)
return err;
}
+/*
+ * Prefetch the whole folio holding @bidx with one read, so that contiguous
+ * blocks in the folio share a bio. Skip the folio if it is locked, e.g. by
+ * the read issued for a previous block in the folio.
+ */
+static void ra_data_folio(struct inode *inode, pgoff_t bidx)
+{
+ struct address_space *mapping = inode->i_mapping;
+ struct folio *folio;
+
+ folio = __filemap_get_folio(mapping,
+ f2fs_lblk_to_folio_index(F2FS_I_SB(inode), bidx),
+ FGP_LOCK | FGP_CREAT | FGP_NOWAIT,
+ mapping_gfp_mask(mapping));
+ if (IS_ERR(folio))
+ return;
+
+ if (folio_test_uptodate(folio))
+ folio_unlock(folio);
+ else
+ mapping->a_ops->read_folio(NULL, folio);
+ folio_put(folio);
+}
+
/*
* Move data block via meta cache while keeping locked data page.
* This can be used to move blocks, aka LBAs, directly on disk.
@@ -1367,8 +1393,10 @@ static int move_data_block(struct inode *inode, block_t bidx,
}
f2fs_up_read(&F2FS_I(inode)->i_sem);
+ pgoff_t f_index = f2fs_has_subpage_blocks(fio.sbi) ?
+ f2fs_lblk_to_folio_index(fio.sbi, bidx) : bidx;
/* do not read out */
- folio = f2fs_grab_cache_folio(mapping, bidx, false);
+ folio = f2fs_grab_cache_folio(mapping, f_index, false);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
goto out_iput;
@@ -1389,7 +1417,9 @@ static int move_data_block(struct inode *inode, block_t bidx,
goto out;
if (unlikely(dn.data_blkaddr == NULL_ADDR)) {
- folio_clear_uptodate(folio);
+ f2fs_ffs_clear_subrange_uptodate(folio,
+ f2fs_folio_byte_offset(fio.sbi, folio, bidx),
+ F2FS_BLKSIZE(fio.sbi));
err = -ENOENT;
goto put_out;
}
@@ -1516,24 +1546,26 @@ static int move_data_block(struct inode *inode, block_t bidx,
return err;
}
-static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
+static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
unsigned int segno, int off)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct folio *folio;
size_t foff = 0;
- bool large = false;
+ bool has_ffs = false;
int err = 0;
folio = f2fs_get_lock_data_folio(inode, bidx, true);
if (IS_ERR(folio))
return PTR_ERR(folio);
- if (f2fs_folio_has_ffs(folio)) {
- large = true;
- foff = offset_in_folio(folio, (loff_t)bidx << PAGE_SHIFT);
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ has_ffs = true;
+ f2fs_ffs_find_or_alloc(folio);
+ foff = f2fs_folio_byte_offset(sbi, folio, bidx);
}
- if (!check_valid_map(F2FS_I_SB(inode), segno, off)) {
+ if (!check_valid_map(sbi, segno, off)) {
err = -ENOENT;
goto out;
}
@@ -1547,13 +1579,13 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
err = -EAGAIN;
goto out;
}
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else {
struct f2fs_io_info fio = {
- .sbi = F2FS_I_SB(inode),
+ .sbi = sbi,
.ino = inode->i_ino,
.type = DATA,
.temp = COLD,
@@ -1561,7 +1593,7 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
.op_flags = REQ_SYNC,
.old_blkaddr = NULL_ADDR,
.folio = folio,
- .folio_offset = bidx - folio->index,
+ .folio_offset = f2fs_folio_blk_offset(sbi, folio, bidx),
.folio_blkcnt = 1,
.encrypted_page = NULL,
.need_lock = LOCK_REQ,
@@ -1572,15 +1604,15 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
retry:
f2fs_folio_wait_writeback(folio, true, true);
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
if (folio_clear_dirty_for_io(folio)) {
- if (!large) {
+ if (!has_ffs) {
inode_dec_dirty_block(inode);
f2fs_remove_dirty_inode(inode);
} else if (f2fs_ffs_clear_subrange_dirty_and_test(folio,
- foff, PAGE_SIZE)) {
+ foff, F2FS_BLKSIZE(sbi))) {
folio_mark_dirty(folio);
}
}
@@ -1598,15 +1630,16 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type,
* Keep gcing flag for redirtied folio, so that
* it can be guaranteed by checkpoint.
*/
- if (large)
- f2fs_ffs_mark_subrange_dirty(folio, foff, PAGE_SIZE);
+ if (has_ffs)
+ f2fs_ffs_mark_subrange_dirty(folio, foff,
+ F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
} else if (!f2fs_ffs_has_dirty_subpage(folio)) {
/* migrate this block in later GC */
folio_clear_f2fs_gcing(folio);
}
- } else if (large && folio_test_dirty(folio)) {
+ } else if (has_ffs && folio_test_dirty(folio)) {
/*
* The folio is left both dirty and under writeback,
* submit the cached bio before unlocking it, otherwise
@@ -1742,6 +1775,13 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
continue;
}
+ if (f2fs_has_subpage_blocks(sbi)) {
+ ra_data_folio(inode, start_bidx);
+ f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
+ add_gc_inode(gc_list, inode);
+ continue;
+ }
+
data_folio = f2fs_get_read_data_folio(inode, start_bidx,
REQ_RAHEAD, true, NULL);
f2fs_up_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
@@ -1785,7 +1825,7 @@ static int gc_data_segment(struct f2fs_sb_info *sbi, struct f2fs_summary *sum,
err = move_data_block(inode, start_bidx,
gc_type, segno, off);
else
- err = move_data_page(inode, start_bidx, gc_type,
+ err = move_data_cache_block(inode, start_bidx, gc_type,
segno, off);
if (!err && (gc_type == FG_GC ||
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index cb8affb2c1e0..c48b12208e20 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -701,7 +701,7 @@ static void f2fs_ra_node_caches(struct f2fs_cached_block *parent, int start, int
blk_finish_plug(&plug);
}
-pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
+pgoff_t f2fs_get_next_block_offset(struct dnode_of_data *dn, pgoff_t blkofs)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(dn->inode);
const long direct_index = ADDRS_PER_INODE(dn->inode);
@@ -714,7 +714,7 @@ pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
pgoff_t base = 0;
if (!dn->max_level)
- return pgofs + 1;
+ return blkofs + 1;
while (max_level-- > cur_level)
skipped_unit *= NIDS_PER_BLOCK(sbi);
@@ -733,7 +733,7 @@ pgoff_t f2fs_get_next_page_offset(struct dnode_of_data *dn, pgoff_t pgofs)
f2fs_bug_on(F2FS_I_SB(dn->inode), 1);
}
- return ((pgofs - base) / skipped_unit + 1) * skipped_unit + base;
+ return ((blkofs - base) / skipped_unit + 1) * skipped_unit + base;
}
/*
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 07/20] f2fs: support subpage blocks in data write path and writeback
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (5 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
` (12 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With blocks smaller than PAGE_SIZE, a folio holds several blocks, so
write_begin, write_end and writeback have to handle individual blocks
instead of the whole folio.
- Read, zero and reserve the blocks touched by a write per block in
write_begin, and mark only the written blocks dirty in write_end.
- Write back only the dirty blocks of a multi-block folio and complete
write I/O per block. In subpage mode every inode is written back by
f2fs_write_cache_folios(), which now handles dentry and quota folios
and checkpoint errors as f2fs_write_single_data_page() does. Both use
the new f2fs_cp_controlled_inode() and f2fs_wb_folio_action() helpers
for these rules.
It also resets the error for each folio, so that the remaining folios
are still written back after an error, as writeback_iter() expects
for integrity writeback, instead of redirtying them.
f2fs_write_folio_dirty_range() cleans up each block where it is
written or skipped, and advances last_disk_size only up to the last
block it submitted. On an error, it clears the dirty state of the
blocks before the failed one, so that only the failed block and the
ones after it are written again.
- Count inode dirty blocks per folio: add all blocks of a folio when
it is dirtied and remove them when it is cleaned, instead of counting
the dirty bits of the folio state. A multi-block folio can be dirtied
without a folio state, e.g. after a read of the whole folio or
through folio_mark_dirty(), and a state attached later starts with
dirty bits that were never counted, so the bit count cannot stay
balanced. A partially dirty folio is counted as fully dirty.
- Compressed and atomic files are not supported yet in subpage mode:
write_begin returns -EOPNOTSUPP for them. Inline data is converted
first.
Some of these changes also apply to large folios when the block size
equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/checkpoint.c | 39 +---
fs/f2fs/data.c | 486 ++++++++++++++++++++++---------------------
fs/f2fs/dir.c | 2 +-
fs/f2fs/f2fs.h | 1 -
fs/f2fs/file.c | 11 +-
fs/f2fs/gc.c | 11 +-
fs/f2fs/inline.c | 7 +-
fs/f2fs/node.c | 2 +-
8 files changed, 275 insertions(+), 284 deletions(-)
diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c
index 1060ee651756..a9a2b9d93231 100644
--- a/fs/f2fs/checkpoint.c
+++ b/fs/f2fs/checkpoint.c
@@ -1324,46 +1324,21 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type)
stat_dec_dirty_inode(F2FS_I_SB(inode), type);
}
-/*
- * Account @nr dirty blocks (negative when dirtiness is cleared) for @inode.
- */
-void f2fs_account_dirty_blocks(struct inode *inode, int nr)
+void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE;
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- if (!nr)
- return;
-
- spin_lock(&sbi->inode_lock[type]);
- if (nr > 0) {
- if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH))
- __add_dirty_inode(inode, type);
- inode_add_dirty_blocks(inode, nr);
- } else {
- inode_sub_dirty_blocks(inode, -nr);
- __remove_dirty_inode(inode, type);
- }
- spin_unlock(&sbi->inode_lock[type]);
-}
-
-void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio)
-{
if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) &&
!S_ISLNK(inode->i_mode))
return;
- /*
- * Large folios have a subpage dirty bitmap, so defer their dirty accounting
- * to the f2fs_ffs_*subrange_dirty*() helpers. This avoids nesting
- * ffs->state_lock with the inode dirty list lock.
- */
- if (f2fs_folio_has_ffs(folio)) {
- folio_set_f2fs_reference(folio);
- return;
- }
-
- f2fs_account_dirty_blocks(inode, 1);
+ spin_lock(&sbi->inode_lock[type]);
+ if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH))
+ __add_dirty_inode(inode, type);
+ inode_add_dirty_blocks(inode, nr_blocks);
+ spin_unlock(&sbi->inode_lock[type]);
folio_set_f2fs_reference(folio);
}
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 07b7aeb94350..41246f83b31d 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -420,8 +420,7 @@ static void f2fs_write_end_bio(struct bio *bio)
bio_for_each_folio_all(fi, bio) {
struct folio *folio = fi.folio;
- unsigned int nr_pages = fi.length >> PAGE_SHIFT;
- unsigned int npages = nr_pages;
+ unsigned int nr_blocks = fi.length >> sbi->log_blocksize;
enum count_type type;
#ifdef CONFIG_F2FS_FS_COMPRESSION
@@ -441,8 +440,7 @@ static void f2fs_write_end_bio(struct bio *bio)
}
}
- while (nr_pages--)
- dec_cache_count(sbi, type);
+ sub_cache_counts(sbi, type, nr_blocks);
/*
* we should access sbi before folio_end_writeback() to
@@ -452,7 +450,7 @@ static void f2fs_write_end_bio(struct bio *bio)
wq_has_sleeper(&sbi->cp_wait))
wake_up(&sbi->cp_wait);
- f2fs_ffs_put_write_pending(folio, npages);
+ f2fs_ffs_put_write_pending(folio, nr_blocks);
}
bio_put(bio);
@@ -3023,7 +3021,7 @@ static void f2fs_clear_truncated_page_state(struct folio *folio, pgoff_t i)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- if (f2fs_folio_has_ffs(folio)) {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
f2fs_ffs_clear_subrange_uptodate(folio,
F2FS_BLK_TO_BYTES(sbi, i), F2FS_BLKSIZE(sbi));
} else {
@@ -3051,21 +3049,6 @@ bool f2fs_ffs_test_blk_dirty(const struct folio *folio, pgoff_t index)
return test_bit(nr_blocks + offset, ffs->state);
}
-static unsigned int __ffs_mark_subrange_dirty(struct folio *folio,
- struct f2fs_folio_state *ffs, size_t offset, size_t len)
-{
- struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
- unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- unsigned int start, end, before;
-
- start = F2FS_BYTES_TO_BLK(sbi, offset);
- end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
-
- before = bitmap_weight(ffs->state, 2 * nr_blocks);
- bitmap_set(ffs->state, nr_blocks + start, end - start);
- return bitmap_weight(ffs->state, 2 * nr_blocks) - before;
-}
-
/*
* Mark the given range dirty for writeback.
*
@@ -3078,7 +3061,8 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
struct f2fs_folio_state *ffs;
- unsigned int set;
+ unsigned int nr_blocks;
+ unsigned int start, end;
unsigned long flags;
f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio));
@@ -3087,13 +3071,13 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio,
return;
ffs = f2fs_ffs_find_or_alloc(folio);
+ nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ start = F2FS_BYTES_TO_BLK(sbi, offset);
+ end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
spin_lock_irqsave(&ffs->state_lock, flags);
- set = __ffs_mark_subrange_dirty(folio, ffs, offset, len);
+ bitmap_set(ffs->state, start + nr_blocks, end - start);
spin_unlock_irqrestore(&ffs->state_lock, flags);
-
- if (set)
- f2fs_account_dirty_blocks(folio->mapping->host, set);
}
void f2fs_ffs_clear_subrange_uptodate(struct folio *folio,
@@ -3129,19 +3113,16 @@ void f2fs_ffs_clear_subrange_uptodate(struct folio *folio,
}
static bool __ffs_clear_subrange_dirty(struct folio *folio,
- struct f2fs_folio_state *ffs, size_t offset, size_t len,
- unsigned int *cleared)
+ struct f2fs_folio_state *ffs, size_t offset, size_t len)
{
struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- unsigned int start, end, before;
+ unsigned int start, end;
start = F2FS_BYTES_TO_BLK(sbi, offset);
end = min_t(unsigned int, F2FS_BLK_ALIGN(sbi, offset + len), nr_blocks);
- before = bitmap_weight(ffs->state, 2 * nr_blocks);
bitmap_clear(ffs->state, nr_blocks + start, end - start);
- *cleared = before - bitmap_weight(ffs->state, 2 * nr_blocks);
return find_next_bit(ffs->state, 2 * nr_blocks, nr_blocks) <
2 * nr_blocks;
}
@@ -3153,15 +3134,13 @@ static bool __ffs_clear_subrange_dirty(struct folio *folio,
* @len: byte length of the subrange
*
* Clear the dirty bits of the 4K subpages covered by [offset, offset + len),
- * account the cleared subpages against the inode dirty counters, and return
- * whether the folio still has any dirty subpage left.
+ * and return whether the folio still has any dirty subpage left.
*/
bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
size_t len)
{
struct f2fs_folio_state *ffs;
unsigned long flags;
- unsigned int cleared = 0;
bool dirty;
if (!f2fs_folio_has_ffs(folio)) {
@@ -3178,12 +3157,9 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset,
ffs = f2fs_ffs_find_or_alloc(folio);
spin_lock_irqsave(&ffs->state_lock, flags);
- dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len, &cleared);
+ dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len);
spin_unlock_irqrestore(&ffs->state_lock, flags);
- if (cleared)
- f2fs_account_dirty_blocks(folio->mapping->host, -cleared);
-
return dirty;
}
@@ -3943,7 +3919,7 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type,
- u64 *pos, u64 len)
+ u64 start, u64 end)
{
struct inode *inode = folio->mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -3951,19 +3927,19 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
folio_test_f2fs_atomic(folio);
struct inode *dn_inode = atomic_commit ?
F2FS_I(inode)->cow_inode : inode;
- struct f2fs_folio_state *ffs = NULL;
- u64 folio_start = folio_pos(folio);
- pgoff_t start_idx = (*pos - folio_start) >> PAGE_SHIFT;
- pgoff_t end_idx = (*pos - folio_start + len - 1) >> PAGE_SHIFT;
+ u64 pos = folio_pos(folio);
+ pgoff_t start_idx = (start - pos) >> sbi->log_blocksize;
+ pgoff_t end_idx = (end - 1 - pos) >> sbi->log_blocksize;
+ pgoff_t last_idx = 0;
+ pgoff_t i;
+ int local_submitted = 0;
int err = 0;
- for (pgoff_t i = start_idx; i <= end_idx && !err; i++) {
+ for (i = start_idx; i <= end_idx; i++) {
struct dnode_of_data dn;
struct node_info ni;
- pgoff_t data_idx = folio->index + i;
+ pgoff_t data_idx = f2fs_folio_lblk(sbi, folio) + i;
bool ipu_force = false;
- bool dn_held = false;
- bool submitted_blk = false;
struct f2fs_io_info fio = {
.sbi = sbi,
.ino = inode->i_ino,
@@ -3983,7 +3959,12 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
};
if (f2fs_folio_has_ffs(folio)) {
- ffs = (struct f2fs_folio_state *)folio->private;
+ struct f2fs_folio_state *ffs =
+ (struct f2fs_folio_state *)folio->private;
+
+ /* Write only dirty blocks */
+ if (!f2fs_ffs_test_blk_dirty(folio, data_idx))
+ continue;
atomic_inc(&ffs->write_blocks_pending);
}
@@ -3995,7 +3976,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
if (!f2fs_is_valid_blkaddr(sbi, fio.old_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
- goto block_done;
+ goto skip_block;
}
ipu_force = true;
goto got_it;
@@ -4015,24 +3996,22 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
LOOKUP_NODE);
}
if (err && err != -ENOENT)
- goto block_done;
- dn_held = true;
+ goto skip_block;
- fio.old_blkaddr = dn.data_blkaddr;
-
- /* This page is already truncated */
- if (err == -ENOENT || fio.old_blkaddr == NULL_ADDR) {
- f2fs_clear_truncated_page_state(folio, i);
+ /* This block is already truncated */
+ if (err || dn.data_blkaddr == NULL_ADDR) {
err = 0;
- goto block_done;
+ f2fs_clear_truncated_page_state(folio, i);
+ goto skip_block;
}
+ fio.old_blkaddr = dn.data_blkaddr;
got_it:
if (__is_valid_data_blkaddr(fio.old_blkaddr) &&
!f2fs_is_valid_blkaddr(sbi, fio.old_blkaddr,
DATA_GENERIC_ENHANCE)) {
err = -EFSCORRUPTED;
- goto block_done;
+ goto skip_block;
}
if (fio.meta_gc)
@@ -4042,61 +4021,109 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
(__is_valid_data_blkaddr(fio.old_blkaddr) &&
need_inplace_update(&fio))) {
f2fs_put_dnode(&dn);
- dn_held = false;
err = f2fs_inplace_write_data(&fio);
- trace_f2fs_do_write_data_page(folio, IPU);
if (err)
- goto block_done;
+ goto skip_block_no_dnode;
- if (submitted)
- (*submitted)++;
- submitted_blk = true;
+ local_submitted++;
+ last_idx = data_idx;
set_inode_flag(inode, FI_UPDATE_WRITE);
- goto block_done;
+ trace_f2fs_do_write_data_page(folio, IPU);
+ continue;
}
err = f2fs_get_node_info(sbi, dn.nid, &ni, false);
if (err)
- goto block_done;
+ goto skip_block;
fio.version = ni.version;
f2fs_outplace_write_data(&dn, &fio);
- /*
- * The outplace write path failed to submit this block.
- */
+ f2fs_put_dnode(&dn);
+ /* The outplace write path failed to submit this block. */
if (!fio.submitted) {
err = -EIO;
- goto block_done;
+ goto skip_block_no_dnode;
}
- if (submitted)
- (*submitted)++;
- submitted_blk = true;
+ local_submitted++;
+ last_idx = data_idx;
set_inode_flag(inode, FI_APPEND_WRITE);
trace_f2fs_do_write_data_page(folio, OPU);
-block_done:
- if (dn_held)
- f2fs_put_dnode(&dn);
+ continue;
+
+skip_block:
+ f2fs_put_dnode(&dn);
+skip_block_no_dnode:
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_put_write_pending(folio, 1);
+ if (err)
+ break;
+ }
+
+ if (err) {
/*
- * If this block was not submitted, roll back its
- * write pages pending.
+ * The blocks before the failed one were written or skipped, so
+ * keep only the failed block and the ones after it dirty.
*/
- if (!submitted_blk && ffs)
- f2fs_ffs_put_write_pending(folio, 1);
- if (!err) {
- *pos = ((loff_t)data_idx + 1) << PAGE_SHIFT;
- spin_lock(&F2FS_I(inode)->i_size_lock);
- if (F2FS_I(inode)->last_disk_size < *pos)
- F2FS_I(inode)->last_disk_size = *pos;
- spin_unlock(&F2FS_I(inode)->i_size_lock);
- } else {
- file_set_keep_isize(inode);
+ if (i > start_idx) {
+ size_t len = F2FS_BLK_TO_BYTES(sbi, i - start_idx);
+
+ f2fs_ffs_clear_subrange_dirty_and_test(folio,
+ start - pos, len);
}
+ file_set_keep_isize(inode);
+ } else if (local_submitted) {
+ loff_t psize = (loff_t)(last_idx + 1) << sbi->log_blocksize;
+
+ if (psize > i_size_read(inode))
+ psize = i_size_read(inode);
+
+ spin_lock(&F2FS_I(inode)->i_size_lock);
+ if (F2FS_I(inode)->last_disk_size < psize)
+ F2FS_I(inode)->last_disk_size = psize;
+ spin_unlock(&F2FS_I(inode)->i_size_lock);
}
+ if (submitted)
+ *submitted = local_submitted;
return err;
}
+/* Dentry and quota blocks are controlled by checkpoint. */
+static bool f2fs_cp_controlled_inode(struct inode *inode)
+{
+ return S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode);
+}
+
+enum f2fs_wb_action {
+ F2FS_WB_WRITE, /* write back the folio */
+ F2FS_WB_REDIRTY, /* keep the folio dirty */
+ F2FS_WB_DROP, /* drop the dirty data */
+};
+
+/* Decide what to do with a dirty data folio before writing it back. */
+static enum f2fs_wb_action f2fs_wb_folio_action(struct f2fs_sb_info *sbi,
+ struct inode *inode)
+{
+ /* we should bypass data pages to proceed the kworker jobs */
+ if (unlikely(f2fs_cp_error(sbi))) {
+ mapping_set_error(inode->i_mapping, -EIO);
+ /*
+ * Keep dirty dentry folios for the latest directory structure,
+ * and keep data folios in remount-ro mode.
+ */
+ if ((S_ISDIR(inode->i_mode) &&
+ !is_sbi_flag_set(sbi, SBI_IS_CLOSE)) ||
+ F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
+ return F2FS_WB_REDIRTY;
+ return F2FS_WB_DROP;
+ }
+
+ if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
+ return F2FS_WB_REDIRTY;
+ return F2FS_WB_WRITE;
+}
+
int f2fs_write_single_data_page(struct folio *folio, int *submitted,
struct bio **bio,
sector_t *last_block,
@@ -4137,26 +4164,15 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
trace_f2fs_writepage(folio, DATA);
- /* we should bypass data pages to proceed the kworker jobs */
- if (unlikely(f2fs_cp_error(sbi))) {
- mapping_set_error(folio->mapping, -EIO);
- /*
- * don't drop any dirty dentry pages for keeping lastest
- * directory structure.
- */
- if (S_ISDIR(inode->i_mode) &&
- !is_sbi_flag_set(sbi, SBI_IS_CLOSE))
- goto redirty_out;
-
- /* keep data pages in remount-ro mode */
- if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY)
- goto redirty_out;
+ switch (f2fs_wb_folio_action(sbi, inode)) {
+ case F2FS_WB_REDIRTY:
+ goto redirty_out;
+ case F2FS_WB_DROP:
goto out;
+ case F2FS_WB_WRITE:
+ break;
}
- if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
- goto redirty_out;
-
if (folio->index < end_index ||
f2fs_verity_in_progress(inode) ||
compr_blocks)
@@ -4172,8 +4188,7 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
folio_zero_segment(folio, offset, folio_size(folio));
write:
- /* Dentry/quota blocks are controlled by checkpoint */
- if (S_ISDIR(inode->i_mode) || quota_inode) {
+ if (f2fs_cp_controlled_inode(inode)) {
struct f2fs_lock_context lc;
/*
@@ -4224,14 +4239,14 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
goto redirty_out;
out:
- inode_dec_dirty_block(inode);
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
if (err) {
folio_clear_uptodate(folio);
folio_clear_f2fs_gcing(folio);
}
folio_unlock(folio);
- if (!S_ISDIR(inode->i_mode) && !IS_NOQUOTA(inode) &&
- !F2FS_I(inode)->wb_task && allow_balance)
+ if (!f2fs_cp_controlled_inode(inode) &&
+ !F2FS_I(inode)->wb_task && allow_balance)
f2fs_balance_fs(sbi, need_balance_fs);
if (unlikely(f2fs_cp_error(sbi))) {
@@ -4560,6 +4575,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
int submitted = 0;
int nwritten = 0;
bool op_locked = false;
+ bool node_locked = false;
bool retry = false;
if (get_dirty_blocks(inode) <= SM_I(sbi)->min_hot_blocks)
@@ -4571,20 +4587,15 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
struct f2fs_folio_state *ffs = NULL;
u64 isize;
size_t poff;
- size_t clear_len;
pgoff_t end_index;
bool verity_in_progress;
int folio_submitted = 0;
bool bias_added = false;
submitted = 0;
+ err = 0;
retry = false;
- if (err) {
- folio_redirty_for_writepage(wbc, folio);
- goto retry_out;
- }
-
if (atomic_read(&sbi->wb_sync_req[DATA]) &&
wbc->sync_mode == WB_SYNC_NONE) {
folio_redirty_for_writepage(wbc, folio);
@@ -4597,11 +4608,22 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
goto retry_out;
}
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ WARN_ON_ONCE(1);
+ mapping_set_error(mapping, -EOPNOTSUPP);
+ folio_clear_dirty_for_io(folio);
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
+ err = -EOPNOTSUPP;
+ goto retry_out;
+ }
+
/*
* Writing back one large folio performs multiple block allocations, so
* best-effort balance the filesystem before writing it back.
*/
- if (!F2FS_I(inode)->wb_task && has_not_enough_free_secs(sbi, 0, 0)) {
+ if (!f2fs_cp_controlled_inode(inode) &&
+ !F2FS_I(inode)->wb_task && has_not_enough_free_secs(sbi, 0, 0)) {
folio_redirty_for_writepage(wbc, folio);
retry = true;
folio_unlock(folio);
@@ -4636,31 +4658,39 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
/*
- * If cp_error is set, all subpages of this folio are not
- * written back unless the filesystem is mounted with
- * errors=remount-ro. The check is placed after the retry
- * block so it also covers cp_error being set while the folio
- * lock was dropped for f2fs_balance_fs() or for the trylock_op()
- * backoff. PG_dirty is clear at this point, so both the
- * redirty and the drop below leave a coherent state.
+ * Checked after the retry block, so that it also covers cp_error
+ * being set while the folio lock was dropped for f2fs_balance_fs()
+ * or for the trylock_op() backoff.
*/
- if (unlikely(f2fs_cp_error(sbi))) {
- mapping_set_error(mapping, -EIO);
- if (F2FS_OPTION(sbi).errors == MOUNT_ERRORS_READONLY) {
- folio_redirty_for_writepage(wbc, folio);
- goto retry_out;
- }
- goto out;
+ switch (f2fs_wb_folio_action(sbi, inode)) {
+ case F2FS_WB_REDIRTY:
+ folio_redirty_for_writepage(wbc, folio);
+ goto retry_out;
+ case F2FS_WB_DROP:
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ folio_size(folio));
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
+ goto retry_out;
+ case F2FS_WB_WRITE:
+ break;
}
- /* To avoid dealing with the complexity for one subrange is in bio
- * while we trylock_op failed before writing another subrange.
- * Try to lock_op before any subrange write for the folio.
- */
- if (!op_locked) {
+ if (f2fs_cp_controlled_inode(inode)) {
+ /*
+ * Quota writes need node_write to avoid block
+ * allocation during checkpoint.
+ */
+ if (IS_NOQUOTA(inode)) {
+ f2fs_down_read_trace(&sbi->node_write, &lc);
+ node_locked = true;
+ }
+ } else {
+ /* To avoid dealing with the complexity for one subrange is in bio
+ * while we trylock_op failed before writing another subrange.
+ * Try to lock_op before any subrange write for the folio.
+ */
if (!f2fs_trylock_op(sbi, &lc)) {
folio_redirty_for_writepage(wbc, folio);
- err = 0;
if (wbc->sync_mode != WB_SYNC_ALL)
goto retry_out;
@@ -4690,59 +4720,34 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_start_writeback(folio);
/*
- * PG_dirty was already cleared by writeback_iter(), and a
- * single-block folio has no ffs dirty bitmap, so the dirty
- * range cannot be derived. Write the whole folio.
+ * PG_dirty was already cleared by writeback_iter(). A single-block
+ * folio has no ffs, so its whole folio is written.
*/
- if (!folio_test_large(folio)) {
- err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos, end_pos - pos);
- folio_submitted += submitted;
- if (err)
- goto out;
- nwritten += submitted;
- goto written;
- }
-
- if (!f2fs_folio_has_ffs(folio)) {
- ffs = f2fs_ffs_find_or_alloc(folio);
- f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
- } else {
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ if (!f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
ffs = (struct f2fs_folio_state *)folio->private;
+ /*
+ * Multiblock folios can be submitted as multiple bios,
+ * which may complete while we are still submitting the
+ * remaining dirty ranges. Hold a bias on
+ * write_blocks_pending to prevent folio_end_writeback()
+ * from being called early.
+ */
+ if (!bias_added) {
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
+ bias_added = true;
+ }
}
- /*
- * Large folios can be submitted as multiple bios, which may
- * complete while we are still submitting the remaining dirty
- * ranges. Hold a bias on write_blocks_pending to prevent
- * folio_end_writeback() from being called early.
- */
- if (f2fs_folio_has_ffs(folio) && !bias_added) {
- WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
- atomic_inc(&ffs->write_blocks_pending);
- bias_added = true;
- }
-
- unsigned int from = 0, start, count;
- unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
- unsigned int to_blk = min_t(unsigned int,
- F2FS_BLK_ALIGN(sbi, end_pos - folio_pos(folio)),
- nr_blocks);
- while (f2fs_ffs_dirty_blk_range(folio, from, to_blk,
- &start, &count)) {
- submitted = 0;
- pos = folio_pos(folio) + F2FS_BLK_TO_BYTES(sbi, start);
- err = f2fs_write_folio_dirty_range(folio, &submitted,
- wbc, io_type, &pos,
- F2FS_BLK_TO_BYTES(sbi, count));
- folio_submitted += submitted;
- nwritten += submitted;
- if (err)
- goto out;
- from = start + count;
- }
+ err = f2fs_write_folio_dirty_range(folio, &submitted, wbc, io_type,
+ pos, end_pos);
+ folio_submitted += submitted;
+ nwritten += submitted;
+ if (err)
+ goto out;
-written:
/* If none of the subpages of a folio is submitted, keep the atomic flag */
if (folio_submitted &&
f2fs_is_atomic_file(inode) &&
@@ -4750,26 +4755,12 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
folio_clear_f2fs_atomic(folio);
out:
- /*
- * Clear the dirty state of the subranges that were written back.
- * If some subranges are still dirty (e.g. the folio was only
- * partially written back), or if the writeback failed, the
- * folio must stay dirty so that the remaining subranges are
- * written back later.
- *
- * On success, clear dirty bits of the whole folio rather than
- * up to isize: subpages beyond EOF can still carry dirty bits
- * (e.g. fill_zero() w/ FALLOC_FL_KEEP_SIZE, BG_GC on blocks
- * beyond EOF, or f2fs_ffs_find_or_alloc() on a dirty folio),
- * they will never be written back, and would keep the folio
- * redirtied forever.
- */
- clear_len = err ? pos - folio_pos(folio) : folio_size(folio);
- if (f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
- clear_len) || err) {
+ if (!err) {
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ folio_size(folio));
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
+ } else {
folio_redirty_for_writepage(wbc, folio);
- } else if (!f2fs_folio_has_ffs(folio)) {
- inode_dec_dirty_block(inode);
}
/*
@@ -4780,7 +4771,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
f2fs_ffs_put_write_pending(folio, 1);
} else if (!folio_submitted && folio_test_writeback(folio)) {
/*
- * Order-0 folios take no bias, so end the writeback
+ * Single-block folios take no bias, so end the writeback
* if nothing was submitted.
*/
folio_end_writeback(folio);
@@ -4792,6 +4783,11 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
op_locked = false;
}
+ if (node_locked) {
+ f2fs_up_read_trace(&sbi->node_write, &lc);
+ node_locked = false;
+ }
+
folio_unlock(folio);
}
@@ -4872,10 +4868,10 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING)))
goto skip_write;
- if ((S_ISDIR(inode->i_mode) || IS_NOQUOTA(inode)) &&
- wbc->sync_mode == WB_SYNC_NONE &&
- get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
- f2fs_available_free_memory(sbi, DIRTY_DENTS))
+ if (f2fs_cp_controlled_inode(inode) &&
+ wbc->sync_mode == WB_SYNC_NONE &&
+ get_dirty_blocks(inode) < nr_caches_to_skip(sbi, DATA) &&
+ f2fs_available_free_memory(sbi, DIRTY_DENTS))
goto skip_write;
/* skip writing in file defragment preparing stage */
@@ -4902,7 +4898,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (mapping_large_folio_support(inode->i_mapping) &&
+ if (f2fs_has_multiblock_folios(inode) &&
!f2fs_has_inline_data(inode))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
@@ -5136,7 +5132,7 @@ static int prepare_atomic_write_begin(struct f2fs_sb_info *sbi,
int err = 0;
block_t ori_blk_addr = NULL_ADDR;
bool cow_has_reserved_block = false;
- bool append = ((loff_t)index << PAGE_SHIFT) >= i_size_read(inode);
+ bool append = F2FS_BLK_TO_BYTES(sbi, index) >= i_size_read(inode);
/*
* On the first attempt, an append write can skip the COW lookup and
@@ -5226,7 +5222,7 @@ static bool f2fs_balance_fs_folio_locked(struct folio *folio,
static int prepare_large_folio_write_begin(struct inode *inode,
struct folio *folio, loff_t pos,
- unsigned int len)
+ unsigned int len, bool *need_balance)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_folio_state *ffs;
@@ -5234,9 +5230,13 @@ static int prepare_large_folio_write_begin(struct inode *inode,
size_t need_off = ori_off;
int err = 0;
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode)))
+ return -EOPNOTSUPP;
+
/*
- * when the write did not come through f2fs_write_iter, we preallocate all
- * blocks in the write range here.
+ * Reserve blocks without f2fs_balance_fs() since the folio is locked;
+ * the caller balances after unlocking it if *need_balance is set.
*/
if (!is_inode_flag_set(inode, FI_PREALLOCATED_ALL) &&
!IS_DEVICE_ALIASING(inode)) {
@@ -5252,9 +5252,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
&node_changed);
if (err)
return err;
- if (f2fs_balance_fs_folio_locked(folio, folio->mapping,
- node_changed))
- return -EAGAIN;
+ *need_balance |= node_changed;
}
}
@@ -5266,8 +5264,12 @@ static int prepare_large_folio_write_begin(struct inode *inode,
if (folio_test_uptodate(folio) || len == folio_size(folio))
return 0;
+ /*
+ * Skip read if the write starts at a block boundary and reaches EOF,
+ * and zero the rest of the folio instead.
+ */
if (!f2fs_verity_in_progress(inode) &&
- !(pos & (PAGE_SIZE - 1)) && (pos + len) >= i_size_read(inode)) {
+ !(pos & (F2FS_BLKSIZE(sbi) - 1)) && (pos + len) >= i_size_read(inode)) {
size_t zoff = ori_off + len;
if (zoff < folio_size(folio))
@@ -5281,7 +5283,8 @@ static int prepare_large_folio_write_begin(struct inode *inode,
while (find_next_valid_block(folio, ori_off,
&need_off, len)) {
struct dnode_of_data dn;
- pgoff_t index = folio->index + (need_off >> PAGE_SHIFT);
+ pgoff_t index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, need_off);
block_t blkaddr;
if (!f2fs_lookup_read_extent_cache_block(inode, index,
@@ -5297,12 +5300,13 @@ static int prepare_large_folio_write_begin(struct inode *inode,
f2fs_put_dnode(&dn);
if (blkaddr == NEW_ADDR) {
- size_t off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT);
+ size_t off = f2fs_folio_byte_offset(sbi,
+ folio, index);
- folio_zero_segment(folio, off, off + PAGE_SIZE);
+ folio_zero_segment(folio, off,
+ off + F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_uptodate(folio, off,
- PAGE_SIZE);
+ F2FS_BLKSIZE(sbi));
continue;
}
}
@@ -5401,12 +5405,18 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto fail;
}
+ if (f2fs_has_subpage_blocks(sbi) &&
+ (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ err = -EOPNOTSUPP;
+ goto fail;
+ }
+
/*
* We should check this at this moment to avoid deadlock on inode page
* and #0 page. The locking rule for inline_data conversion should be:
* folio_lock(folio #0) -> folio_lock(inode_page)
*/
- if (index != 0) {
+ if (index != 0 || f2fs_has_subpage_blocks(sbi)) {
err = f2fs_convert_inline_inode(inode);
if (err)
goto fail;
@@ -5469,17 +5479,19 @@ static int f2fs_write_begin(const struct kiocb *iocb,
*/
f2fs_folio_wait_writeback(folio, false, true);
- if (f2fs_is_atomic_file(inode) && !folio_test_large(folio))
+ if (f2fs_is_atomic_file(inode) && !f2fs_is_multiblock_folio(sbi, folio)) {
err = prepare_atomic_write_begin(sbi, inode, folio->index,
&blkaddr, &need_balance, retried);
- else if (!folio_test_large(folio))
+ } else if (!f2fs_is_multiblock_folio(sbi, folio)) {
err = prepare_write_begin(sbi, folio, pos, blen,
&blkaddr, &need_balance);
- else if (f2fs_is_atomic_file(inode))
+ } else if (f2fs_is_atomic_file(inode)) {
err = prepare_large_folio_atomic_write_begin(inode,
mapping, folio, pos, len, retried);
- else
- err = prepare_large_folio_write_begin(inode, folio, pos, blen);
+ } else {
+ err = prepare_large_folio_write_begin(inode, folio, pos, blen,
+ &need_balance);
+ }
if (err) {
if (err == -EAGAIN) {
retried = true;
@@ -5495,7 +5507,7 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto repeat;
}
- if (folio_test_large(folio))
+ if (f2fs_is_multiblock_folio(sbi, folio))
return 0;
if (blen == folio_size(folio) || folio_test_uptodate(folio))
@@ -5565,7 +5577,7 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!folio_test_uptodate(folio)) {
if (unlikely(copied != len)) {
copied = 0;
- } else if (folio_test_large(folio)) {
+ } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
f2fs_ffs_mark_subrange_uptodate(folio,
offset_in_folio(folio, pos), len);
} else {
@@ -5590,7 +5602,7 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!copied)
goto unlock_out;
- if (folio_test_large(folio))
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio))
f2fs_ffs_mark_subrange_dirty(folio, offset_in_folio(folio, pos),
copied);
folio_mark_dirty(folio);
@@ -5642,6 +5654,9 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
* dirty state.
*/
if (!dirty && folio_test_dirty(folio)) {
+ inode_sub_dirty_blocks(inode,
+ f2fs_blocks_per_folio(sbi, folio));
+ f2fs_remove_dirty_inode(inode);
f2fs_clear_page_cache_dirty_tag(folio);
folio_cancel_dirty(folio);
}
@@ -5650,11 +5665,8 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length)
return;
}
- if (f2fs_folio_has_ffs(folio)) {
- f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
- folio_size(folio));
- } else if (folio_test_dirty(folio)) {
- inode_dec_dirty_block(inode);
+ if (folio_test_dirty(folio)) {
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
f2fs_remove_dirty_inode(inode);
}
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index aa3511c6f8fe..7f5dc2354a27 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -979,9 +979,9 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block,
if (!f2fs_has_subpage_blocks(sbi)) {
folio_clear_uptodate(folio);
folio_detach_private(folio);
- inode_dec_dirty_block(dir);
}
+ inode_sub_dirty_blocks(dir, f2fs_blocks_per_folio(sbi, folio));
f2fs_remove_dirty_inode(dir);
}
keep_dirty:
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index cf7294068415..aab1095ad5ff 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4457,7 +4457,6 @@ void f2fs_remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino);
int f2fs_recover_orphan_inodes(struct f2fs_sb_info *sbi);
int f2fs_get_valid_checkpoint(struct f2fs_sb_info *sbi);
void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio);
-void f2fs_account_dirty_blocks(struct inode *inode, int nr);
void f2fs_remove_dirty_inode(struct inode *inode);
int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type,
bool from_cp);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 1ce4553b1d01..be548f39e33e 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -5561,12 +5561,13 @@ static int f2fs_preallocate_blocks(struct kiocb *iocb, struct iov_iter *iter,
return ret;
}
- if (!dio && mapping_large_folio_support(inode->i_mapping)) {
+ if (!dio && f2fs_has_multiblock_folios(inode)) {
/*
- * Preallocate all blocks touched by a large-folio buffered write so
- * the regular write_begin path does not need to unlock the folio for
- * f2fs_balance_fs(). Rechecking large-folio state after unlock is
- * unreliable since partial truncation can split the folio.
+ * Preallocate all blocks touched by a multiblock-folio buffered
+ * write so the regular write_begin path does not need to unlock
+ * the folio for f2fs_balance_fs(). Rechecking large-folio
+ * state after unlock is unreliable since partial truncation can
+ * split the folio.
*/
map.m_lblk = F2FS_BYTES_TO_BLK(sbi, pos);
map.m_len = F2FS_BLK_ALIGN(sbi, pos + count);
diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
index 6c24001325b2..f041a036836e 100644
--- a/fs/f2fs/gc.c
+++ b/fs/f2fs/gc.c
@@ -1608,13 +1608,12 @@ static int move_data_cache_block(struct inode *inode, block_t bidx, int gc_type,
f2fs_ffs_mark_subrange_dirty(folio, foff, F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
if (folio_clear_dirty_for_io(folio)) {
- if (!has_ffs) {
- inode_dec_dirty_block(inode);
- f2fs_remove_dirty_inode(inode);
- } else if (f2fs_ffs_clear_subrange_dirty_and_test(folio,
- foff, F2FS_BLKSIZE(sbi))) {
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
+ f2fs_remove_dirty_inode(inode);
+ if (has_ffs &&
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, foff,
+ F2FS_BLKSIZE(sbi)))
folio_mark_dirty(folio);
- }
}
folio_set_f2fs_gcing(folio);
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index ba0aff8e5643..757be9f8d069 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -196,10 +196,15 @@ int f2fs_convert_inline_folio(struct dnode_of_data *dn, struct folio *folio)
folio_start_writeback(folio);
fio.old_blkaddr = dn->data_blkaddr;
set_inode_flag(dn->inode, FI_HOT_DATA);
+ if (f2fs_is_multiblock_folio(fio.sbi, folio)) {
+ struct f2fs_folio_state *ffs = f2fs_ffs_find_or_alloc(folio);
+
+ atomic_inc(&ffs->write_blocks_pending);
+ }
f2fs_outplace_write_data(dn, &fio);
f2fs_folio_wait_writeback(folio, true, true);
if (dirty) {
- inode_dec_dirty_block(dn->inode);
+ inode_sub_dirty_blocks(dn->inode, f2fs_blocks_per_folio(fio.sbi, folio));
f2fs_remove_dirty_inode(dn->inode);
}
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index c48b12208e20..b95bb5c27b0b 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -1700,7 +1700,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino)
goto folio_out;
ret = f2fs_write_inline_data(inode, folio);
- inode_dec_dirty_block(inode);
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
f2fs_remove_dirty_inode(inode);
if (ret)
folio_mark_dirty(folio);
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 08/20] f2fs: adapt range operations and mmap for subpage blocks
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (6 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
` (11 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
- A writable PTE covers the whole folio, so page_mkwrite prepares
every block of the folio up to EOF and dirties only those blocks.
- Align zero range, punch hole, preallocation and truncation to the
block size instead of PAGE_SIZE. When i_size grows, write-protect the
folio straddling the old EOF so that stores to its new blocks fault
into page_mkwrite.
- Unmap every base page overlapping a punched or zeroed range, and
truncate the page cache without truncate_pagecache_range(), whose
unmap underflows for a range ending in the first page.
- For SEEK_DATA, treat a NEW_ADDR block as data only if the block
itself is dirty.
- When truncation zeroes the tail of a page, mark only the block
holding the new EOF dirty, not the blocks after it.
The SEEK_DATA change also applies to large folios when the block size
equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 58 +++++++++--
fs/f2fs/f2fs.h | 4 +-
fs/f2fs/file.c | 263 ++++++++++++++++++++++++++++++++-----------------
3 files changed, 228 insertions(+), 97 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 41246f83b31d..77915a2eb190 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -3198,7 +3198,7 @@ bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
static bool find_next_valid_block(const struct folio *folio,
size_t orig_off, size_t *need_off,
- size_t len)
+ size_t len, bool read_full)
{
struct f2fs_sb_info *sbi = F2FS_M_SB(folio->mapping);
size_t start = orig_off;
@@ -3206,6 +3206,20 @@ static bool find_next_valid_block(const struct folio *folio,
size_t head, tail;
pgoff_t index;
+ /* Find every block that is not uptodate, not only the partial ones. */
+ if (read_full) {
+ for (head = round_down(*need_off, F2FS_BLKSIZE(sbi)); head < end;
+ head += F2FS_BLKSIZE(sbi)) {
+ index = f2fs_folio_lblk(sbi, folio) +
+ F2FS_BYTES_TO_BLK(sbi, head);
+ if (!f2fs_ffs_test_blk_uptodate(folio, index)) {
+ *need_off = head;
+ return true;
+ }
+ }
+ return false;
+ }
+
if (start & (F2FS_BLKSIZE(sbi) - 1)) {
head = round_down(start, F2FS_BLKSIZE(sbi));
index = f2fs_folio_lblk(sbi, folio) +
@@ -5222,7 +5236,8 @@ static bool f2fs_balance_fs_folio_locked(struct folio *folio,
static int prepare_large_folio_write_begin(struct inode *inode,
struct folio *folio, loff_t pos,
- unsigned int len, bool *need_balance)
+ unsigned int len, bool read_full,
+ bool preallocated, bool *need_balance)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_folio_state *ffs;
@@ -5238,8 +5253,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
* Reserve blocks without f2fs_balance_fs() since the folio is locked;
* the caller balances after unlocking it if *need_balance is set.
*/
- if (!is_inode_flag_set(inode, FI_PREALLOCATED_ALL) &&
- !IS_DEVICE_ALIASING(inode)) {
+ if (!preallocated && !IS_DEVICE_ALIASING(inode)) {
pgoff_t first = F2FS_BYTES_TO_BLK(sbi, pos);
pgoff_t last = F2FS_BLK_ALIGN(sbi, pos + len);
pgoff_t idx;
@@ -5261,14 +5275,15 @@ static int prepare_large_folio_write_begin(struct inode *inode,
/* Skip read if the folio is already fully uptodate or the write
* covers the entire folio.
*/
- if (folio_test_uptodate(folio) || len == folio_size(folio))
+ if (folio_test_uptodate(folio) ||
+ (!read_full && len == folio_size(folio)))
return 0;
/*
* Skip read if the write starts at a block boundary and reaches EOF,
* and zero the rest of the folio instead.
*/
- if (!f2fs_verity_in_progress(inode) &&
+ if (!read_full && !f2fs_verity_in_progress(inode) &&
!(pos & (F2FS_BLKSIZE(sbi) - 1)) && (pos + len) >= i_size_read(inode)) {
size_t zoff = ori_off + len;
@@ -5281,7 +5296,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
while (find_next_valid_block(folio, ori_off,
- &need_off, len)) {
+ &need_off, len, read_full)) {
struct dnode_of_data dn;
pgoff_t index = f2fs_folio_lblk(sbi, folio) +
F2FS_BYTES_TO_BLK(sbi, need_off);
@@ -5382,6 +5397,20 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
return 0;
}
+int f2fs_prepare_subpage_mkwrite(struct inode *inode, struct folio *folio,
+ loff_t pos, unsigned int len)
+{
+ bool need_balance = false;
+
+ /*
+ * mkwrite runs without inode_lock, so FI_PREALLOCATED_ALL may have
+ * been set by a concurrent write() for a different range. It calls
+ * f2fs_balance_fs() before locking the folio, so ignore need_balance.
+ */
+ return prepare_large_folio_write_begin(inode, folio, pos, len, true,
+ false, &need_balance);
+}
+
static int f2fs_write_begin(const struct kiocb *iocb,
struct address_space *mapping,
loff_t pos, unsigned len, struct folio **foliop,
@@ -5490,7 +5519,9 @@ static int f2fs_write_begin(const struct kiocb *iocb,
mapping, folio, pos, len, retried);
} else {
err = prepare_large_folio_write_begin(inode, folio, pos, blen,
- &need_balance);
+ false,
+ is_inode_flag_set(inode, FI_PREALLOCATED_ALL),
+ &need_balance);
}
if (err) {
if (err == -EAGAIN) {
@@ -5566,6 +5597,7 @@ static int f2fs_write_end(const struct kiocb *iocb,
struct folio *folio, void *fsdata)
{
struct inode *inode = folio->mapping->host;
+ loff_t old_size = i_size_read(inode);
trace_f2fs_write_end(inode, pos, len, copied);
@@ -5595,6 +5627,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (pos + copied > i_size_read(inode) &&
!f2fs_verity_in_progress(inode))
f2fs_i_size_write(inode, pos + copied);
+ if (i_size_read(inode) > old_size)
+ pagecache_isize_extended(inode, old_size, pos);
return copied;
}
#endif
@@ -5620,6 +5654,14 @@ static int f2fs_write_end(const struct kiocb *iocb,
unlock_out:
f2fs_folio_put(folio, true);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
+
+ /*
+ * The folio straddling the old EOF may still be mapped writable, and
+ * the blocks between the old EOF and pos in it are not prepared.
+ * Write-protect it so that mmap stores to them fault.
+ */
+ if (i_size_read(inode) > old_size)
+ pagecache_isize_extended(inode, old_size, pos);
return copied;
}
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index aab1095ad5ff..8e3224eacdb5 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -4094,7 +4094,7 @@ int f2fs_getattr(const struct mnt_idmap *idmap, const struct path *path,
struct kstat *stat, u32 request_mask, unsigned int flags);
int f2fs_setattr(const struct mnt_idmap *idmap, struct dentry *dentry,
struct iattr *attr);
-int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end);
+int f2fs_truncate_hole(struct inode *inode, pgoff_t blk_start, pgoff_t blk_end);
void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count);
int f2fs_do_shutdown(struct f2fs_sb_info *sbi, unsigned int flag,
bool readonly, bool need_lock);
@@ -4539,6 +4539,8 @@ void f2fs_ffs_clear_subrange_uptodate(struct folio *folio, size_t offset,
size_t len);
bool f2fs_ffs_dirty_blk_range(struct folio *folio, unsigned int from,
unsigned int to, unsigned int *start, unsigned int *count);
+int f2fs_prepare_subpage_mkwrite(struct inode *inode, struct folio *folio,
+ loff_t pos, unsigned int len);
void f2fs_write_failed(struct inode *inode, loff_t to);
void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length);
bool f2fs_release_folio(struct folio *folio, gfp_t wait);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index be548f39e33e..e68553cef777 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -41,18 +41,19 @@ static int fill_zero(struct inode *inode, pgoff_t index,
static int do_zero_post_eof_page(struct inode *inode, loff_t new_size)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
loff_t old_size = i_size_read(inode);
unsigned int offset, len;
- pgoff_t index;
+ pgoff_t bidx;
int err;
- offset = old_size & (PAGE_SIZE - 1);
+ offset = old_size & (F2FS_BLKSIZE(sbi) - 1);
if (!offset)
return 0;
- len = min_t(loff_t, PAGE_SIZE - offset, new_size - old_size);
- index = old_size >> PAGE_SHIFT;
+ len = min_t(loff_t, F2FS_BLKSIZE(sbi) - offset, new_size - old_size);
+ bidx = F2FS_BYTES_TO_BLK(sbi, old_size);
if (f2fs_has_inline_data(inode)) {
/* data post eof should be always zero */
@@ -63,7 +64,7 @@ static int do_zero_post_eof_page(struct inode *inode, loff_t new_size)
return err;
}
- err = fill_zero(inode, index, offset, len);
+ err = fill_zero(inode, bidx, offset, len);
if (err)
return err;
return filemap_write_and_wait_range(inode->i_mapping,
@@ -144,11 +145,11 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
bool need_alloc = !f2fs_is_pinned_file(inode);
pgoff_t pidx = folio->index + folio_page_idx(folio, vmf->page);
loff_t pos = (loff_t)pidx << PAGE_SHIFT;
+ pgoff_t bidx = F2FS_BYTES_TO_BLK(sbi, pos + offset_in_page(vmf->address));
loff_t isize;
loff_t folio_start;
loff_t valid_end;
size_t dirty_len;
- size_t subpage_off;
pgoff_t i, nr;
int err = 0;
vm_fault_t ret;
@@ -212,14 +213,12 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
folio_lock(folio);
isize = i_size_read(inode);
folio_start = folio_pos(folio);
- subpage_off = offset_in_folio(folio, pos);
valid_end = min_t(loff_t, folio_start + folio_size(folio), isize);
dirty_len = valid_end - folio_start;
if (unlikely(folio->mapping != inode->i_mapping ||
pos >= isize ||
- !f2fs_ffs_test_blk_uptodate(folio,
- folio->index + (subpage_off >> PAGE_SHIFT)))) {
+ !f2fs_ffs_test_blk_uptodate(folio, bidx))) {
folio_unlock(folio);
err = -EFAULT;
goto out_sem;
@@ -240,6 +239,24 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
+ if (f2fs_has_subpage_blocks(sbi)) {
+ /*
+ * A writable PTE covers the whole base folio. Like ext4's
+ * block_page_mkwrite(), prepare every filesystem block through EOF
+ * before returning it writable, but dirty only those blocks. This
+ * prevents a later store in a sibling 4K block from bypassing block
+ * allocation while also keeping blocks beyond i_size clean.
+ */
+ f2fs_folio_wait_writeback(folio, false, true);
+ err = f2fs_prepare_subpage_mkwrite(inode, folio,
+ folio_pos(folio), dirty_len);
+ if (err) {
+ folio_unlock(folio);
+ goto out_sem;
+ }
+ goto zero_tail;
+ }
+
for (i = 0; i < nr; i++) {
err = f2fs_get_block_mkwrite(inode, folio, folio->index + i,
need_alloc);
@@ -249,6 +266,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
}
}
+zero_tail:
/* page is wholly or partially inside EOF */
if (folio_start + folio_size(folio) > isize) {
size_t offset = offset_in_folio(folio, isize);
@@ -256,8 +274,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
folio_zero_segment(folio, offset, folio_size(folio));
}
- if (folio_test_large(folio))
- f2fs_ffs_mark_subrange_dirty(folio, 0, dirty_len);
+ f2fs_ffs_mark_subrange_dirty(folio, 0, dirty_len);
folio_mark_dirty(folio);
f2fs_update_iostat(sbi, inode, APP_MAPPED_IO, dirty_len);
@@ -503,7 +520,9 @@ static bool __found_offset(struct address_space *mapping,
{
block_t blkaddr = f2fs_data_blkaddr(dn);
struct inode *inode = mapping->host;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
bool compressed_cluster = false;
+ bool dirty = false;
if (f2fs_compressed_file(inode)) {
block_t first_blkaddr = data_blkaddr(dn->inode, dn->node_entry,
@@ -512,12 +531,37 @@ static bool __found_offset(struct address_space *mapping,
compressed_cluster = first_blkaddr == COMPRESS_ADDR;
}
+ if (blkaddr == NEW_ADDR) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, index);
+
+ if (xa_get_mark(&mapping->i_pages, f_index, PAGECACHE_TAG_DIRTY)) {
+ struct folio *folio = filemap_get_folio(mapping, f_index);
+
+ if (!IS_ERR(folio)) {
+ /*
+ * ffs is freed under the folio lock. We hold the
+ * dnode, so only trylock the folio, and take the
+ * block as dirty, as the dirty tag says, if the
+ * folio is busy.
+ */
+ if (folio_trylock(folio)) {
+ if (folio->mapping == mapping)
+ dirty = f2fs_ffs_test_blk_dirty(folio,
+ index);
+ folio_unlock(folio);
+ } else {
+ dirty = true;
+ }
+ f2fs_folio_put(folio, false);
+ }
+ }
+ }
+
switch (whence) {
case SEEK_DATA:
if (__is_valid_data_blkaddr(blkaddr))
return true;
- if (blkaddr == NEW_ADDR &&
- xa_get_mark(&mapping->i_pages, index, PAGECACHE_TAG_DIRTY))
+ if (dirty)
return true;
if (compressed_cluster)
return true;
@@ -815,20 +859,23 @@ void f2fs_truncate_data_blocks_range(struct dnode_of_data *dn, int count)
dn->ofs_in_node, nr_free);
}
-static int truncate_partial_data_page(struct inode *inode, u64 from,
+static int truncate_partial_data_block(struct inode *inode, u64 from,
bool cache_only)
{
- pgoff_t index = from >> PAGE_SHIFT;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ pgoff_t bidx = F2FS_BYTES_TO_BLK(sbi, from);
+ pgoff_t f_index = f2fs_has_subpage_blocks(sbi) ?
+ f2fs_lblk_to_folio_index(sbi, bidx) : (from >> PAGE_SHIFT);
struct address_space *mapping = inode->i_mapping;
struct folio *folio;
size_t folio_off;
size_t blk_end;
- if (!(from & (PAGE_SIZE - 1)) && !cache_only)
+ if (!(from & (F2FS_BLKSIZE(sbi) - 1)) && !cache_only)
return 0;
if (cache_only) {
- folio = filemap_lock_folio(mapping, index);
+ folio = filemap_lock_folio(mapping, f_index);
if (IS_ERR(folio))
return 0;
if (folio_test_uptodate(folio))
@@ -837,7 +884,7 @@ static int truncate_partial_data_page(struct inode *inode, u64 from,
return 0;
}
- folio = f2fs_get_lock_data_folio(inode, index, true);
+ folio = f2fs_get_lock_data_folio(inode, bidx, true);
if (IS_ERR(folio))
return PTR_ERR(folio) == -ENOENT ? 0 : PTR_ERR(folio);
truncate_out:
@@ -848,14 +895,15 @@ static int truncate_partial_data_page(struct inode *inode, u64 from,
folio_zero_segment(folio, folio_off, blk_end);
/* An encrypted inode should have a key and truncate the last page. */
- f2fs_bug_on(F2FS_I_SB(inode), cache_only && IS_ENCRYPTED(inode));
+ f2fs_bug_on(sbi, cache_only && IS_ENCRYPTED(inode));
if (!cache_only) {
/*
* The folio was read back above, so the subpage is
- * already uptodate and dirty data stays valid.
+ * already uptodate and dirty data stays valid. Mark only
+ * the block holding the new EOF dirty.
*/
f2fs_ffs_mark_subrange_dirty(folio, folio_off,
- blk_end - folio_off);
+ round_up(folio_off, F2FS_BLKSIZE(sbi)) - folio_off);
folio_mark_dirty(folio);
}
f2fs_folio_put(folio, true);
@@ -944,9 +992,9 @@ int f2fs_do_truncate_blocks(struct inode *inode, u64 from, bool lock)
if (lock)
f2fs_unlock_op(sbi, &lc);
free_partial:
- /* lastly zero out the first data page */
+ /* lastly zero out the first data block */
if (!err)
- err = truncate_partial_data_page(inode, from, truncate_page);
+ err = truncate_partial_data_block(inode, from, truncate_page);
out_err:
trace_f2fs_truncate_blocks_exit(inode, err);
return err;
@@ -1310,7 +1358,7 @@ const struct inode_operations f2fs_file_inode_operations = {
.fileattr_set = f2fs_fileattr_set,
};
-static int fill_zero(struct inode *inode, pgoff_t index,
+static int fill_zero(struct inode *inode, pgoff_t bidx,
loff_t start, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -1324,15 +1372,14 @@ static int fill_zero(struct inode *inode, pgoff_t index,
f2fs_balance_fs(sbi, true);
f2fs_lock_op(sbi, &lc);
- folio = f2fs_get_new_data_folio(inode, NULL, index, false);
+ folio = f2fs_get_new_data_folio(inode, NULL, bidx, false);
f2fs_unlock_op(sbi, &lc);
if (IS_ERR(folio))
return PTR_ERR(folio);
f2fs_folio_wait_writeback(folio, true, true);
- folio_off = offset_in_folio(folio,
- (loff_t)index << PAGE_SHIFT) + start;
+ folio_off = f2fs_folio_byte_offset(sbi, folio, bidx) + start;
folio_zero_range(folio, folio_off, len);
/*
* This subpage must be uptodate here, which is guaranteed by
@@ -1344,41 +1391,68 @@ static int fill_zero(struct inode *inode, pgoff_t index,
return 0;
}
-int f2fs_truncate_hole(struct inode *inode, pgoff_t pg_start, pgoff_t pg_end)
+int f2fs_truncate_hole(struct inode *inode, pgoff_t blk_start, pgoff_t blk_end)
{
int err;
- while (pg_start < pg_end) {
+ while (blk_start < blk_end) {
struct dnode_of_data dn;
pgoff_t end_offset, count;
set_new_dnode(&dn, inode, NULL, NULL, 0);
- err = f2fs_get_dnode_of_data(&dn, pg_start, LOOKUP_NODE);
+ err = f2fs_get_dnode_of_data(&dn, blk_start, LOOKUP_NODE);
if (err) {
if (err == -ENOENT) {
- pg_start = f2fs_get_next_block_offset(&dn,
- pg_start);
+ blk_start = f2fs_get_next_block_offset(&dn,
+ blk_start);
continue;
}
return err;
}
end_offset = ADDRS_PER_PAGE(dn.node_entry, inode);
- count = min(end_offset - dn.ofs_in_node, pg_end - pg_start);
+ count = min(end_offset - dn.ofs_in_node, blk_end - blk_start);
f2fs_bug_on(F2FS_I_SB(inode), count == 0 || count > end_offset);
f2fs_truncate_data_blocks_range(&dn, count);
f2fs_put_dnode(&dn);
- pg_start += count;
+ blk_start += count;
}
return 0;
}
+/*
+ * unmap_mapping_range() computes page count solely from holelen, which causes
+ * under-unmapping when start is not PAGE_SIZE aligned. Outward-align to PAGE_SIZE
+ * to guarantee all overlapping base pages are unmapped.
+ *
+ * Then truncate the page cache without truncate_pagecache_range(), whose own
+ * unmap underflows when the range ends in the first page and unmaps the rest
+ * of the file.
+ */
+static void f2fs_truncate_pagecache_range(struct inode *inode,
+ loff_t start, loff_t end)
+{
+ struct address_space *mapping = inode->i_mapping;
+ loff_t unmap_start, unmap_len;
+
+ if (!f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
+ truncate_pagecache_range(inode, start, end - 1);
+ return;
+ }
+
+ unmap_start = round_down(start, PAGE_SIZE);
+ unmap_len = round_up(end, PAGE_SIZE) - unmap_start;
+ unmap_mapping_range(mapping, unmap_start, unmap_len, 0);
+ truncate_inode_pages_range(mapping, start, end - 1);
+}
+
static int f2fs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
{
- pgoff_t pg_start, pg_end;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ pgoff_t blk_start, blk_end;
loff_t off_start, off_end;
int ret;
@@ -1390,47 +1464,46 @@ static int f2fs_punch_hole(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- off_start = offset & (PAGE_SIZE - 1);
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ off_start = offset & F2FS_BLKSIZE_MASK(sbi);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- if (pg_start == pg_end) {
- ret = fill_zero(inode, pg_start, off_start,
+ if (blk_start == blk_end) {
+ ret = fill_zero(inode, blk_start, off_start,
off_end - off_start);
if (ret)
return ret;
} else {
if (off_start) {
- ret = fill_zero(inode, pg_start++, off_start,
- PAGE_SIZE - off_start);
+ ret = fill_zero(inode, blk_start++, off_start,
+ F2FS_BLKSIZE(sbi) - off_start);
if (ret)
return ret;
}
if (off_end) {
- ret = fill_zero(inode, pg_end, 0, off_end);
+ ret = fill_zero(inode, blk_end, 0, off_end);
if (ret)
return ret;
}
- if (pg_start < pg_end) {
- loff_t blk_start, blk_end;
- struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ if (blk_start < blk_end) {
+ loff_t byte_start, byte_end;
struct f2fs_lock_context lc;
f2fs_balance_fs(sbi, true);
- blk_start = (loff_t)pg_start << PAGE_SHIFT;
- blk_end = (loff_t)pg_end << PAGE_SHIFT;
+ byte_start = F2FS_BLK_TO_BYTES(sbi, blk_start);
+ byte_end = F2FS_BLK_TO_BYTES(sbi, blk_end);
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
- truncate_pagecache_range(inode, blk_start, blk_end - 1);
+ f2fs_truncate_pagecache_range(inode, byte_start, byte_end);
f2fs_lock_op(sbi, &lc);
- ret = f2fs_truncate_hole(inode, pg_start, pg_end);
+ ret = f2fs_truncate_hole(inode, blk_start, blk_end);
f2fs_unlock_op(sbi, &lc);
filemap_invalidate_unlock(inode->i_mapping);
@@ -1621,7 +1694,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
f2fs_folio_wait_writeback(fdst, true, true);
memcpy_folio(fdst, dst_off, fsrc, src_off,
F2FS_BLKSIZE(sbi));
- if (folio_test_large(fdst)) {
+ if (f2fs_is_multiblock_folio(sbi, fdst)) {
f2fs_ffs_mark_subrange_uptodate(fdst,
dst_off, F2FS_BLKSIZE(sbi));
f2fs_ffs_mark_subrange_dirty(fdst, dst_off,
@@ -1830,7 +1903,7 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
- pgoff_t index, pg_start, pg_end;
+ pgoff_t blk_start, blk_end;
loff_t new_size = i_size_read(inode);
loff_t off_start, off_end;
int ret = 0;
@@ -1851,31 +1924,33 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
if (ret)
return ret;
- pg_start = ((unsigned long long) offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long) offset + len) >> PAGE_SHIFT;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- off_start = offset & (PAGE_SIZE - 1);
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ off_start = offset & F2FS_BLKSIZE_MASK(sbi);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- if (pg_start == pg_end) {
- ret = fill_zero(inode, pg_start, off_start,
+ if (blk_start == blk_end) {
+ ret = fill_zero(inode, blk_start, off_start,
off_end - off_start);
if (ret)
return ret;
new_size = max_t(loff_t, new_size, offset + len);
} else {
+ pgoff_t index;
+
if (off_start) {
- ret = fill_zero(inode, pg_start++, off_start,
- PAGE_SIZE - off_start);
+ ret = fill_zero(inode, blk_start++, off_start,
+ F2FS_BLKSIZE(sbi) - off_start);
if (ret)
return ret;
new_size = max_t(loff_t, new_size,
- (loff_t)pg_start << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, blk_start));
}
- for (index = pg_start; index < pg_end;) {
+ for (index = blk_start; index < blk_end;) {
struct dnode_of_data dn;
struct f2fs_lock_context lc;
unsigned int end_offset;
@@ -1884,9 +1959,9 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(mapping);
- truncate_pagecache_range(inode,
- (loff_t)index << PAGE_SHIFT,
- ((loff_t)pg_end << PAGE_SHIFT) - 1);
+ f2fs_truncate_pagecache_range(inode,
+ F2FS_BLK_TO_BYTES(sbi, index),
+ F2FS_BLK_TO_BYTES(sbi, blk_end));
f2fs_lock_op(sbi, &lc);
@@ -1900,7 +1975,7 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
}
end_offset = ADDRS_PER_PAGE(dn.node_entry, inode);
- end = min(pg_end, end_offset - dn.ofs_in_node + index);
+ end = min(blk_end, end_offset - dn.ofs_in_node + index);
ret = f2fs_do_zero_range(&dn, index, end);
f2fs_put_dnode(&dn);
@@ -1916,11 +1991,11 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
index = end;
new_size = max_t(loff_t, new_size,
- (loff_t)index << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(sbi, index));
}
if (off_end) {
- ret = fill_zero(inode, pg_end, 0, off_end);
+ ret = fill_zero(inode, blk_end, 0, off_end);
if (ret)
goto out;
@@ -1930,10 +2005,14 @@ static int f2fs_zero_range(struct inode *inode, loff_t offset, loff_t len,
out:
if (new_size > i_size_read(inode)) {
- if (mode & FALLOC_FL_KEEP_SIZE)
+ if (mode & FALLOC_FL_KEEP_SIZE) {
file_set_keep_isize(inode);
- else
+ } else {
+ loff_t old_size = i_size_read(inode);
+
f2fs_i_size_write(inode, new_size);
+ pagecache_isize_extended(inode, old_size, new_size);
+ }
}
return ret;
}
@@ -2034,7 +2113,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
.should_migrate_blocks = false,
.err_gc_skipped = true,
.nr_free_secs = 0 };
- pgoff_t pg_start, pg_end;
+ pgoff_t blk_start, blk_end;
loff_t new_size;
loff_t off_end;
block_t expanded = 0;
@@ -2054,12 +2133,12 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
f2fs_balance_fs(sbi, true);
- pg_start = ((unsigned long long)offset) >> PAGE_SHIFT;
- pg_end = ((unsigned long long)offset + len) >> PAGE_SHIFT;
- off_end = (offset + len) & (PAGE_SIZE - 1);
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
+ off_end = (offset + len) & F2FS_BLKSIZE_MASK(sbi);
- map.m_lblk = pg_start;
- map.m_len = pg_end - pg_start;
+ map.m_lblk = blk_start;
+ map.m_len = blk_end - blk_start;
if (off_end)
map.m_len++;
@@ -2071,9 +2150,9 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
block_t sec_len;
if (map.m_lblk % sec_blks) {
- pg_start = rounddown(map.m_lblk, sec_blks);
- map.m_lblk = pg_start;
- map.m_len = pg_end - pg_start;
+ blk_start = rounddown(map.m_lblk, sec_blks);
+ map.m_lblk = blk_start;
+ map.m_len = blk_end - blk_start;
if (off_end)
map.m_len++;
}
@@ -2089,7 +2168,7 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
f2fs_warn_ratelimited(sbi,
"ino:%llu, start:%lu, end:%lu, need to trigger GC to "
"reclaim enough free segment when checkpoint is enabled",
- inode->i_ino, pg_start, pg_end);
+ inode->i_ino, blk_start, blk_end);
goto out_err;
}
}
@@ -2135,20 +2214,24 @@ static int f2fs_expand_inode_data(struct inode *inode, loff_t offset,
if (!expanded)
return err;
- last_off = pg_start + expanded - 1;
+ last_off = blk_start + expanded - 1;
/* update new size to the failed position */
- new_size = (last_off == pg_end) ? offset + len :
- (loff_t)(last_off + 1) << PAGE_SHIFT;
+ new_size = (last_off == blk_end) ? offset + len :
+ F2FS_BLK_TO_BYTES(sbi, last_off + 1);
} else {
- new_size = ((loff_t)pg_end << PAGE_SHIFT) + off_end;
+ new_size = F2FS_BLK_TO_BYTES(sbi, blk_end) + off_end;
}
if (new_size > i_size_read(inode)) {
- if (mode & FALLOC_FL_KEEP_SIZE)
+ if (mode & FALLOC_FL_KEEP_SIZE) {
file_set_keep_isize(inode);
- else
+ } else {
+ loff_t old_size = i_size_read(inode);
+
f2fs_i_size_write(inode, new_size);
+ pagecache_isize_extended(inode, old_size, new_size);
+ }
}
return err;
@@ -5758,8 +5841,12 @@ static ssize_t f2fs_dio_write_iter(struct kiocb *iocb, struct iov_iter *from,
if (ret < 0)
goto out;
- if (pos + ret > inode->i_size)
+ if (pos + ret > inode->i_size) {
+ loff_t old_size = inode->i_size;
+
f2fs_i_size_write(inode, pos + ret);
+ pagecache_isize_extended(inode, old_size, pos + ret);
+ }
if (!do_opu)
set_inode_flag(inode, FI_UPDATE_WRITE);
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 09/20] f2fs: support subpage blocks in pinned files
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (7 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
` (10 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Pinned files must not allocate blocks outside fallocate(). As write()
does, return -EIO from page_mkwrite when the folio range up to EOF is
not fully allocated, and remove the -EOPNOTSUPP checks.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index e68553cef777..36da0d558ef0 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -240,6 +240,13 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
nr = DIV_ROUND_UP(dirty_len, PAGE_SIZE);
if (f2fs_has_subpage_blocks(sbi)) {
+ if (f2fs_is_pinned_file(inode) &&
+ !f2fs_overwrite_io(inode, folio_pos(folio), dirty_len)) {
+ folio_unlock(folio);
+ err = -EIO;
+ goto out_sem;
+ }
+
/*
* A writable PTE covers the whole base folio. Like ext4's
* block_page_mkwrite(), prepare every filesystem block through EOF
@@ -2280,8 +2287,7 @@ static long f2fs_fallocate(struct file *file, int mode,
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
- f2fs_is_pinned_file(inode) || f2fs_is_atomic_file(inode) ||
- f2fs_compressed_file(inode)) {
+ f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
@@ -3823,9 +3829,6 @@ static int f2fs_ioc_set_pin_file(struct file *filp, unsigned long arg)
__u32 pin;
int ret = 0;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (get_user(pin, (__u32 __user *)arg))
return -EFAULT;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 10/20] f2fs: support subpage blocks in collapse and insert range
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (8 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 11/20] f2fs: support subpage blocks in move range Daeho Jeong
` (9 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The folio holding offset may hold blocks before the shifted range.
Write back the page cache from the PAGE_SIZE boundary below offset
under invalidate_lock before truncating it from there, so that no
data is lost. Detect src and dst in the same folio by folio index
instead of block index, as a folio holds several blocks. Remove the
-EOPNOTSUPP check.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 72 +++++++++++++++++++++++++++++---------------------
1 file changed, 42 insertions(+), 30 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 36da0d558ef0..3ef810653bb2 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1659,6 +1659,7 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
} else {
struct folio *fsrc, *fdst;
size_t src_off, dst_off;
+ pgoff_t dst_fidx = f2fs_lblk_to_folio_index(sbi, dst + i);
bool same_folio;
fsrc = f2fs_get_lock_data_folio(src_inode,
@@ -1671,11 +1672,11 @@ static int __clone_blkaddrs(struct inode *src_inode, struct inode *dst_inode,
/*
* For same-inode operations (collapse/insert), src and
- * dst may fall within the same large folio. Detect this
+ * dst may fall within the same folio. Detect this
* to avoid self-deadlock on folio lock.
*/
same_folio = src_inode == dst_inode &&
- folio_contains(fsrc, dst + i);
+ folio_contains(fsrc, dst_fidx);
if (same_folio) {
struct dnode_of_data dn;
@@ -1783,7 +1784,7 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_lock_context lc;
- pgoff_t nrpages = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
+ pgoff_t nrblocks = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
pgoff_t start = F2FS_BYTES_TO_BLK(sbi, offset);
pgoff_t end = F2FS_BYTES_TO_BLK(sbi, offset + len);
int ret;
@@ -1794,14 +1795,24 @@ static int f2fs_do_collapse(struct inode *inode, loff_t offset, loff_t len)
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(inode->i_mapping);
+ /*
+ * Write back under invalidate_lock, which blocks page_mkwrite, so
+ * that truncate_pagecache() below cannot drop a racing mmap store to
+ * the blocks before @offset in the head folio.
+ */
+ ret = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ if (ret)
+ goto out_unlock;
+
ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
if (ret)
goto out_unlock;
f2fs_lock_op(sbi, &lc);
f2fs_drop_extent_tree(inode);
- truncate_pagecache(inode, offset);
- ret = __exchange_data_block(inode, inode, end, start, nrpages - end, true);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
+ ret = __exchange_data_block(inode, inode, end, start, nrblocks - end, true);
f2fs_unlock_op(sbi, &lc);
out_unlock:
@@ -1827,21 +1838,17 @@ static int f2fs_collapse_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
- if (ret)
- return ret;
-
ret = f2fs_do_collapse(inode, offset, len);
if (ret)
return ret;
/* write out all moved pages, if possible */
filemap_invalidate_lock(inode->i_mapping);
- ret = filemap_write_and_wait_range(inode->i_mapping, offset, LLONG_MAX);
+ ret = filemap_write_and_wait_range(inode->i_mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
if (ret)
goto out_unlock;
- truncate_pagecache(inode, offset);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
new_size = i_size_read(inode) - len;
ret = f2fs_truncate_blocks(inode, new_size, true);
@@ -2028,7 +2035,7 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct address_space *mapping = inode->i_mapping;
- pgoff_t nr, pg_start, pg_end, delta, idx;
+ pgoff_t nr, blk_start, blk_end, delta, idx;
loff_t new_size;
int ret = 0;
@@ -2041,8 +2048,8 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
return -EINVAL;
/* insert range should be aligned to block size of f2fs. */
- if (offset & F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)) ||
- len & F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)))
+ if (offset & F2FS_BLKSIZE_MASK(sbi) ||
+ len & F2FS_BLKSIZE_MASK(sbi))
return -EINVAL;
ret = f2fs_convert_inline_inode(inode);
@@ -2057,29 +2064,34 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
if (ret)
return ret;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(mapping, offset, LLONG_MAX);
- if (ret)
- return ret;
-
- pg_start = F2FS_BYTES_TO_BLK(sbi, offset);
- pg_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
- delta = pg_end - pg_start;
+ blk_start = F2FS_BYTES_TO_BLK(sbi, offset);
+ blk_end = F2FS_BYTES_TO_BLK(sbi, offset + len);
+ delta = blk_end - blk_start;
idx = F2FS_BLK_ALIGN(sbi, i_size_read(inode));
/* avoid gc operation during block exchange */
f2fs_down_write(&F2FS_I(inode)->i_gc_rwsem[WRITE]);
filemap_invalidate_lock(mapping);
+ /*
+ * Write back under invalidate_lock, which blocks page_mkwrite, so
+ * that truncate_pagecache() below cannot drop a racing mmap store to
+ * the blocks before @offset in the head folio.
+ */
+ ret = filemap_write_and_wait_range(mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ if (ret)
+ goto out_unlock;
+
ret = f2fs_zero_post_eof_page(inode, offset + len, false, false);
if (ret)
goto out_unlock;
- truncate_pagecache(inode, offset);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
- while (!ret && idx > pg_start) {
+ while (!ret && idx > blk_start) {
struct f2fs_lock_context lc;
- nr = idx - pg_start;
+ nr = idx - blk_start;
if (nr > delta)
nr = delta;
idx -= nr;
@@ -2099,8 +2111,9 @@ static int f2fs_insert_range(struct inode *inode, loff_t offset, loff_t len)
/* write out all moved pages, if possible */
filemap_invalidate_lock(mapping);
- ret = filemap_write_and_wait_range(mapping, offset, LLONG_MAX);
- truncate_pagecache(inode, offset);
+ ret = filemap_write_and_wait_range(mapping,
+ round_down(offset, PAGE_SIZE), LLONG_MAX);
+ truncate_pagecache(inode, round_down(offset, PAGE_SIZE));
filemap_invalidate_unlock(mapping);
if (!ret)
@@ -2286,8 +2299,7 @@ static long f2fs_fallocate(struct file *file, int mode,
}
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if ((mode & (FALLOC_FL_COLLAPSE_RANGE | FALLOC_FL_INSERT_RANGE)) ||
- f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
+ if (f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 11/20] f2fs: support subpage blocks in move range
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (9 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
` (8 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Folios at both ends of the moved ranges can straddle the range
boundaries. Write back and truncate the page cache from PAGE_SIZE
boundaries, and hold invalidate_lock of both inodes during the exchange
so that faults and readahead cannot bring back stale data. The page
cache was not truncated before, so this also applies when the block
size equals PAGE_SIZE. Remove the -EOPNOTSUPP check.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 45 +++++++++++++++++++++++++++++++--------------
1 file changed, 31 insertions(+), 14 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 3ef810653bb2..0fad86267256 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -3501,17 +3501,6 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
if (ret)
goto out_unlock;
- /* write out all dirty pages from offset */
- ret = filemap_write_and_wait_range(src->i_mapping,
- pos_in, pos_in + len);
- if (ret)
- goto out_unlock;
-
- ret = filemap_write_and_wait_range(dst->i_mapping,
- pos_out, pos_out + len);
- if (ret)
- goto out_unlock;
-
f2fs_balance_fs(sbi, true);
f2fs_down_write(&F2FS_I(src)->i_gc_rwsem[WRITE]);
@@ -3521,6 +3510,31 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
goto out_src;
}
+ /*
+ * Hold invalidate_lock across writeback, truncation and the block
+ * exchange so neither faults nor readahead can repopulate the page
+ * cache with pre-exchange data.
+ */
+ filemap_invalidate_lock_two(src->i_mapping, dst->i_mapping);
+
+ /* write out all dirty pages from offset */
+ ret = filemap_write_and_wait_range(src->i_mapping,
+ round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ if (ret)
+ goto out_inval;
+
+ ret = filemap_write_and_wait_range(dst->i_mapping,
+ round_down(pos_out, PAGE_SIZE),
+ round_up(pos_out + len, PAGE_SIZE) - 1);
+ if (ret)
+ goto out_inval;
+
+ truncate_pagecache_range(src, round_down(pos_in, PAGE_SIZE),
+ round_up(pos_in + len, PAGE_SIZE) - 1);
+ truncate_pagecache_range(dst, round_down(pos_out, PAGE_SIZE),
+ round_up(pos_out + len, PAGE_SIZE) - 1);
+
f2fs_lock_op(sbi, &lc);
ret = __exchange_data_block(src, dst, F2FS_BYTES_TO_BLK(sbi, pos_in),
F2FS_BYTES_TO_BLK(sbi, pos_out),
@@ -3534,6 +3548,12 @@ static int f2fs_move_file_range(struct file *file_in, loff_t pos_in,
}
f2fs_unlock_op(sbi, &lc);
+ /* the exchange may have extended dst past its old EOF folio */
+ if (i_size_read(dst) > dst_osize)
+ pagecache_isize_extended(dst, dst_osize, i_size_read(dst));
+out_inval:
+ filemap_invalidate_unlock_two(src->i_mapping, dst->i_mapping);
+
if (src != dst)
f2fs_up_write(&F2FS_I(dst)->i_gc_rwsem[WRITE]);
out_src:
@@ -3588,9 +3608,6 @@ static int f2fs_ioc_move_range(struct file *filp, unsigned long arg)
{
struct f2fs_move_range range;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(file_inode(filp))))
- return -EOPNOTSUPP;
-
if (copy_from_user(&range, (struct f2fs_move_range __user *)arg,
sizeof(range)))
return -EFAULT;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 12/20] f2fs: support subpage blocks in defragmentation
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (10 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 11/20] f2fs: support subpage blocks in move range Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
` (7 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Mark only the target block, not its whole page, dirty when redirtying
data for defragmentation, and read a multi-block folio once instead of
each block synchronously. Remove the -EOPNOTSUPP check. Reading the
folio once also applies to large folios when the block size equals
PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 27 +++++++++++++++++++++------
1 file changed, 21 insertions(+), 6 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 0fad86267256..29cfc0e2719a 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -3312,6 +3312,20 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
cnt < BLKS_PER_SEG(sbi)) {
struct folio *folio;
+ /*
+ * Read the whole folio in one I/O; otherwise each block
+ * of a multi-block folio is read synchronously below.
+ */
+ if (f2fs_has_multiblock_folios(inode)) {
+ folio = read_mapping_folio(inode->i_mapping,
+ f2fs_lblk_to_folio_index(sbi, idx), filp);
+ if (IS_ERR(folio)) {
+ err = PTR_ERR(folio);
+ goto clear_out;
+ }
+ folio_put(folio);
+ }
+
folio = f2fs_get_lock_data_folio(inode, idx, true);
if (IS_ERR(folio)) {
err = PTR_ERR(folio);
@@ -3320,9 +3334,13 @@ static int f2fs_defragment_range(struct f2fs_sb_info *sbi,
f2fs_folio_wait_writeback(folio, true, true);
- f2fs_ffs_mark_subrange_dirty(folio,
- offset_in_folio(folio, (loff_t)idx << PAGE_SHIFT),
- PAGE_SIZE);
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ size_t off = f2fs_folio_byte_offset(sbi, folio, idx);
+
+ f2fs_ffs_mark_subrange_dirty(folio, off,
+ F2FS_BLKSIZE(sbi));
+ }
+
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
f2fs_folio_put(folio, true);
@@ -3363,9 +3381,6 @@ static int f2fs_ioc_defragment(struct file *filp, unsigned long arg)
struct f2fs_defragment range;
int err;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 13/20] f2fs: support subpage blocks in swapfile activation
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (11 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
` (6 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The swap subsystem works in PAGE_SIZE units. Ignore a partial tail
page, require PAGE_SIZE aligned extents, and convert block counts and
addresses to pages for swap. When migrating blocks, mark only the
migrated block, not its whole page, dirty in the folio state.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 43 ++++++++++++++++++++++++++++---------------
1 file changed, 28 insertions(+), 15 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 77915a2eb190..929a2beef379 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -5876,8 +5876,8 @@ static int f2fs_migrate_blocks(struct inode *inode, block_t start_blk,
}
f2fs_ffs_mark_subrange_dirty(folio,
- offset_in_folio(folio, (loff_t)blkidx << PAGE_SHIFT),
- PAGE_SIZE);
+ f2fs_folio_byte_offset(sbi, folio, blkidx),
+ F2FS_BLKSIZE(sbi));
folio_mark_dirty(folio);
f2fs_folio_put(folio, true);
}
@@ -5909,8 +5909,10 @@ static int check_swap_activate(struct swap_info_struct *sis,
struct address_space *mapping = swap_file->f_mapping;
struct inode *inode = mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
+ unsigned int blks_per_page = F2FS_BLKS_PER_PAGE(sbi);
block_t cur_lblock;
block_t last_lblock;
+ block_t max_lblock;
block_t pblock;
block_t lowest_pblock = -1;
block_t highest_pblock = 0;
@@ -5925,9 +5927,12 @@ static int check_swap_activate(struct swap_info_struct *sis,
* to be very smart.
*/
cur_lblock = 0;
- last_lblock = F2FS_BYTES_TO_BLK(sbi, i_size_read(inode));
+ /* swap uses whole pages only; ignore a partial tail page */
+ last_lblock = round_down(F2FS_BYTES_TO_BLK(sbi, i_size_read(inode)),
+ blks_per_page);
+ max_lblock = min_t(u64, last_lblock, (u64)sis->max * blks_per_page);
- while (cur_lblock < last_lblock && cur_lblock < sis->max) {
+ while (cur_lblock < max_lblock) {
struct f2fs_map_blocks map;
bool last_extent = false;
retry:
@@ -5962,7 +5967,7 @@ static int check_swap_activate(struct swap_info_struct *sis,
not_aligned++;
nr_pblocks = roundup(nr_pblocks, blks_per_sec);
- if (cur_lblock + nr_pblocks > sis->max)
+ if (cur_lblock + nr_pblocks > max_lblock)
nr_pblocks -= blks_per_sec;
/* this extent is last one */
@@ -5983,20 +5988,28 @@ static int check_swap_activate(struct swap_info_struct *sis,
goto retry;
}
- if (cur_lblock + nr_pblocks >= sis->max)
- nr_pblocks = sis->max - cur_lblock;
+ if (cur_lblock + nr_pblocks >= max_lblock)
+ nr_pblocks = max_lblock - cur_lblock;
+
+ if (pblock % blks_per_page || nr_pblocks % blks_per_page) {
+ f2fs_err(sbi, "Swapfile extent is not aligned to PAGE_SIZE");
+ ret = -EINVAL;
+ goto out;
+ }
if (cur_lblock) { /* exclude the header page */
- if (pblock < lowest_pblock)
- lowest_pblock = pblock;
- if (pblock + nr_pblocks - 1 > highest_pblock)
- highest_pblock = pblock + nr_pblocks - 1;
+ if (pblock / blks_per_page < lowest_pblock)
+ lowest_pblock = pblock / blks_per_page;
+ if ((pblock + nr_pblocks) / blks_per_page - 1 > highest_pblock)
+ highest_pblock = (pblock + nr_pblocks) / blks_per_page - 1;
}
/*
* We found a PAGE_SIZE-length, PAGE_SIZE-aligned run of blocks
*/
- ret = add_swap_extent(sis, cur_lblock, nr_pblocks, pblock);
+ ret = add_swap_extent(sis, cur_lblock / blks_per_page,
+ nr_pblocks / blks_per_page,
+ pblock / blks_per_page);
if (ret < 0)
goto out;
nr_extents += ret;
@@ -6005,9 +6018,9 @@ static int check_swap_activate(struct swap_info_struct *sis,
ret = nr_extents;
*span = 1 + highest_pblock - lowest_pblock;
if (cur_lblock == 0)
- cur_lblock = 1; /* force Empty message */
- sis->max = cur_lblock;
- sis->pages = cur_lblock - 1;
+ cur_lblock = blks_per_page; /* force Empty message */
+ sis->max = cur_lblock / blks_per_page;
+ sis->pages = sis->max - 1;
out:
if (not_aligned)
f2fs_warn(sbi, "Swapfile (%u) is not align to section: 1) creat(), 2) ioctl(F2FS_IOC_SET_PIN_FILE), 3) fallocate(%lu * N)",
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 14/20] f2fs: support subpage blocks in fs-verity enablement
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (12 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
` (5 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Check verity per block. While enabling verity, dirty and write back
only the data blocks below i_size in the folios before the verity
metadata. Remove the -EOPNOTSUPP checks. The changes for enabling
verity also apply to large folios when the block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 15 ++++++++-------
fs/f2fs/f2fs.h | 5 +++++
fs/f2fs/file.c | 3 ---
fs/f2fs/verity.c | 8 --------
4 files changed, 13 insertions(+), 18 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 929a2beef379..a23fd4b01a4c 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -1671,7 +1671,7 @@ int f2fs_reserve_block(struct dnode_of_data *dn, pgoff_t index)
static inline struct fsverity_info *f2fs_need_verity(const struct inode *inode,
pgoff_t idx)
{
- if (idx < DIV_ROUND_UP(inode->i_size, PAGE_SIZE))
+ if (idx < F2FS_BLK_ALIGN(F2FS_SB(inode->i_sb), inode->i_size))
return fsverity_get_info(inode);
return NULL;
}
@@ -2924,11 +2924,11 @@ struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio)
pgoff_t first_lblk = f2fs_folio_lblk(sbi, folio);
pgoff_t valid_blocks = 0;
- if (f2fs_verity_in_progress(inode))
- valid_blocks = nr_blocks;
- else if (first_lblk < last_lblk)
+ if (first_lblk < last_lblk)
valid_blocks = min_t(pgoff_t, nr_blocks,
last_lblk - first_lblk);
+ else if (f2fs_verity_in_progress(inode))
+ valid_blocks = nr_blocks;
if (valid_blocks)
bitmap_set(ffs->state, nr_blocks, valid_blocks);
@@ -3657,7 +3657,7 @@ static int f2fs_read_data_folio(struct file *file, struct folio *folio)
return ret;
}
- vi = f2fs_need_verity(inode, folio->index);
+ vi = f2fs_need_verity(inode, f2fs_folio_lblk(F2FS_I_SB(inode), folio));
if (vi)
fsverity_readahead(vi, folio->index, folio_nr_pages(folio));
return f2fs_mpage_readpages(inode, vi, NULL, folio);
@@ -3677,7 +3677,8 @@ static void f2fs_readahead(struct readahead_control *rac)
if (f2fs_has_inline_data(inode))
return;
- vi = f2fs_need_verity(inode, readahead_index(rac));
+ vi = f2fs_need_verity(inode, F2FS_BYTES_TO_BLK(F2FS_I_SB(inode),
+ readahead_pos(rac)));
if (vi)
fsverity_readahead(vi, readahead_index(rac),
readahead_count(rac));
@@ -4717,7 +4718,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
op_locked = true;
}
- if (!verity_in_progress) {
+ if (!verity_in_progress || end_pos <= f2fs_verity_metadata_pos(inode)) {
poff = offset_in_folio(folio, isize);
end_index = isize >> PAGE_SHIFT;
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 8e3224eacdb5..8e38d46c446c 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -5345,6 +5345,11 @@ static inline bool f2fs_allow_multi_device_dio(struct f2fs_sb_info *sbi,
return sbi->aligned_blksize;
}
+static inline loff_t f2fs_verity_metadata_pos(const struct inode *inode)
+{
+ return round_up(i_size_read(inode), 65536);
+}
+
#ifdef CONFIG_F2FS_FAULT_INJECTION
extern int f2fs_build_fault_attr(struct f2fs_sb_info *sbi, unsigned long rate,
unsigned long type, enum fault_option fo);
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 29cfc0e2719a..1c440b5fa9bd 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -4273,9 +4273,6 @@ static int f2fs_ioc_enable_verity(struct file *filp, unsigned long arg)
{
struct inode *inode = file_inode(filp);
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
- return -EOPNOTSUPP;
-
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (!f2fs_sb_has_verity(F2FS_I_SB(inode))) {
diff --git a/fs/f2fs/verity.c b/fs/f2fs/verity.c
index af1b7f21c1d8..5f579d0b917b 100644
--- a/fs/f2fs/verity.c
+++ b/fs/f2fs/verity.c
@@ -32,11 +32,6 @@
#define F2FS_VERIFY_VER (1)
-static inline loff_t f2fs_verity_metadata_pos(const struct inode *inode)
-{
- return round_up(inode->i_size, 65536);
-}
-
/*
* Write some verity metadata to the inode for FS_IOC_ENABLE_VERITY.
* kernel_write() can't be used because the file descriptor is readonly.
@@ -94,9 +89,6 @@ static int f2fs_begin_enable_verity(struct file *filp)
struct inode *inode = file_inode(filp);
int err;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
- return -EOPNOTSUPP;
-
if (f2fs_verity_in_progress(inode))
return -EBUSY;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 15/20] f2fs: support subpage blocks in atomic writes
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (13 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
` (4 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Writeback already writes dirty blocks to the COW inode per block. Make
write_begin of atomic files reserve and read COW blocks per block as
well, and remove the -EOPNOTSUPP checks. Skipping the read of the
blocks that a write fully covers also applies to large folios when the
block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 24 ++++++++++++++++--------
fs/f2fs/file.c | 5 +----
2 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index a23fd4b01a4c..4e4d1b0d6004 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4624,7 +4624,7 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ f2fs_compressed_file(inode)) {
WARN_ON_ONCE(1);
mapping_set_error(mapping, -EOPNOTSUPP);
folio_clear_dirty_for_io(folio);
@@ -5247,7 +5247,7 @@ static int prepare_large_folio_write_begin(struct inode *inode,
int err = 0;
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode)))
+ f2fs_compressed_file(inode))
return -EOPNOTSUPP;
/*
@@ -5359,13 +5359,16 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
/* Inline data must have been converted before reaching here. */
f2fs_bug_on(sbi, f2fs_has_inline_data(inode));
- start = folio->index + (ori_off >> PAGE_SHIFT);
- end = folio->index + ((ori_off + len - 1) >> PAGE_SHIFT);
+ start = f2fs_folio_lblk(sbi, folio) + (ori_off >> sbi->log_blocksize);
+ end = f2fs_folio_lblk(sbi, folio) +
+ ((ori_off + len - 1) >> sbi->log_blocksize);
for (index = start; index <= end; index++) {
+ loff_t block_start = F2FS_BLK_TO_BYTES(sbi, index);
+ loff_t block_end = block_start + F2FS_BLKSIZE(sbi);
block_t blkaddr = NULL_ADDR;
bool node_changed = false;
- size_t off = (index - folio->index) << PAGE_SHIFT;
+ size_t off = f2fs_folio_byte_offset(sbi, folio, index);
err = prepare_atomic_write_begin(sbi, inode, index,
&blkaddr, &node_changed,
@@ -5373,12 +5376,17 @@ static int prepare_large_folio_atomic_write_begin(struct inode *inode,
if (err)
return err;
+ /* Skip read if the write covers the entire block. */
+ if (pos <= block_start && pos + len >= block_end)
+ goto balance;
+
if (f2fs_ffs_test_blk_uptodate(folio, index))
goto balance;
if (blkaddr == NEW_ADDR) {
- folio_zero_segment(folio, off, off + PAGE_SIZE);
- f2fs_ffs_mark_subrange_uptodate(folio, off, PAGE_SIZE);
+ folio_zero_segment(folio, off, off + F2FS_BLKSIZE(sbi));
+ f2fs_ffs_mark_subrange_uptodate(folio, off,
+ F2FS_BLKSIZE(sbi));
goto balance;
}
@@ -5436,7 +5444,7 @@ static int f2fs_write_begin(const struct kiocb *iocb,
}
if (f2fs_has_subpage_blocks(sbi) &&
- (f2fs_compressed_file(inode) || f2fs_is_atomic_file(inode))) {
+ f2fs_compressed_file(inode)) {
err = -EOPNOTSUPP;
goto fail;
}
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 1c440b5fa9bd..91f792cf879f 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -2299,7 +2299,7 @@ static long f2fs_fallocate(struct file *file, int mode,
}
if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if (f2fs_is_atomic_file(inode) || f2fs_compressed_file(inode)) {
+ if (f2fs_compressed_file(inode)) {
ret = -EOPNOTSUPP;
goto out;
}
@@ -2553,9 +2553,6 @@ static int f2fs_ioc_start_atomic_write(struct file *filp, bool truncate)
loff_t isize;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!(filp->f_mode & FMODE_WRITE))
return -EBADF;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 16/20] f2fs: preserve inline data in subpage block mode
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (14 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
` (3 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
A write to an inline inode converted it to a regular inode whenever the
folio held multiple blocks, allocating a block even for small files.
Convert only when the write goes beyond MAX_INLINE_DATA; otherwise read
the inline data into the folio and write it back to the inode block in
f2fs_write_cache_folios().
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/data.c | 59 ++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 50 insertions(+), 9 deletions(-)
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 4e4d1b0d6004..a7cc7231b572 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -4732,6 +4732,21 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
}
}
+ if (f2fs_has_inline_data(inode)) {
+ err = f2fs_write_inline_data(inode, folio);
+ if (!err) {
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ folio_size(folio));
+ inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
+ goto retry_out;
+ }
+ if (err != -EAGAIN) {
+ folio_redirty_for_writepage(wbc, folio);
+ goto retry_out;
+ }
+ err = 0;
+ }
+
folio_start_writeback(folio);
/*
@@ -4913,8 +4928,7 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (f2fs_has_multiblock_folios(inode) &&
- !f2fs_has_inline_data(inode))
+ if (f2fs_has_multiblock_folios(inode))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
ret = f2fs_write_cache_pages(mapping, wbc, io_type);
@@ -4981,6 +4995,16 @@ void f2fs_write_failed(struct inode *inode, loff_t to)
}
}
+/* Fill @folio from the inline data for a write within the inline area */
+static void prepare_inline_write_begin(struct inode *inode,
+ struct folio *folio, struct f2fs_cached_block *ientry)
+{
+ f2fs_do_read_inline_data(folio, ientry);
+ set_inode_flag(inode, FI_DATA_EXIST);
+ if (inode->i_nlink)
+ f2fs_cache_set_inline(ientry);
+}
+
static int prepare_write_begin(struct f2fs_sb_info *sbi,
struct folio *folio, loff_t pos, unsigned int len,
block_t *blk_addr, bool *node_changed)
@@ -5034,10 +5058,7 @@ static int prepare_write_begin(struct f2fs_sb_info *sbi,
if (f2fs_has_inline_data(inode)) {
if (pos + len <= MAX_INLINE_DATA(inode)) {
- f2fs_do_read_inline_data(folio, ientry);
- set_inode_flag(inode, FI_DATA_EXIST);
- if (inode->i_nlink)
- f2fs_cache_set_inline(ientry);
+ prepare_inline_write_begin(inode, folio, ientry);
goto out;
}
err = f2fs_convert_inline_folio(&dn, folio);
@@ -5454,7 +5475,8 @@ static int f2fs_write_begin(const struct kiocb *iocb,
* and #0 page. The locking rule for inline_data conversion should be:
* folio_lock(folio #0) -> folio_lock(inode_page)
*/
- if (index != 0 || f2fs_has_subpage_blocks(sbi)) {
+ if (index != 0 || (f2fs_has_subpage_blocks(sbi) &&
+ pos + len > MAX_INLINE_DATA(inode))) {
err = f2fs_convert_inline_inode(inode);
if (err)
goto fail;
@@ -5510,6 +5532,23 @@ static int f2fs_write_begin(const struct kiocb *iocb,
blen = min_t(unsigned int, len,
folio_size(folio) - offset_in_folio(folio, pos));
+ if (f2fs_has_inline_data(inode) && f2fs_is_multiblock_folio(sbi, folio)) {
+ struct f2fs_lock_context lc;
+ struct f2fs_cached_block *ientry;
+
+ f2fs_map_lock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ ientry = f2fs_get_inode_cache(sbi, inode->i_ino);
+ if (IS_ERR(ientry)) {
+ f2fs_map_unlock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ err = PTR_ERR(ientry);
+ goto put_folio;
+ }
+ prepare_inline_write_begin(inode, folio, ientry);
+ f2fs_put_cache(ientry, true);
+ f2fs_map_unlock(sbi, &lc, F2FS_GET_BLOCK_PRE_AIO);
+ return 0;
+ }
+
/*
* Flush the cached IO of this folio before touching it, especially
* for the large folio prepare below which reads and updates subpage
@@ -5618,7 +5657,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!folio_test_uptodate(folio)) {
if (unlikely(copied != len)) {
copied = 0;
- } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
+ } else if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio) &&
+ !f2fs_has_inline_data(inode)) {
f2fs_ffs_mark_subrange_uptodate(folio,
offset_in_folio(folio, pos), len);
} else {
@@ -5645,7 +5685,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
if (!copied)
goto unlock_out;
- if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio))
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio) &&
+ !f2fs_has_inline_data(inode))
f2fs_ffs_mark_subrange_dirty(folio, offset_in_folio(folio, pos),
copied);
folio_mark_dirty(folio);
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 17/20] f2fs: support subpage blocks in online resize
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (15 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 18/20] f2fs: support subpage blocks in compression Daeho Jeong
` (2 subsequent siblings)
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
Resize works in blocks, segments and sections, independent of
PAGE_SIZE, so remove the -EOPNOTSUPP check.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/file.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 91f792cf879f..67ed6be8952f 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -4250,9 +4250,6 @@ static int f2fs_ioc_resize_fs(struct file *filp, unsigned long arg)
struct f2fs_sb_info *sbi = F2FS_I_SB(file_inode(filp));
__u64 block_count;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 18/20] f2fs: support subpage blocks in compression
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (16 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
2026-11-02 19:39 ` [PATCH 20/20] f2fs: enable subpage block size support Daeho Jeong
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
With subpage blocks, a cluster spans fewer folios than blocks, and
several compressed blocks share a bounce folio.
- Require a cluster to cover at least one page.
- Use block offsets and sizes for compressed data, keep each compressed
block at its block offset within the cluster, and count pending
compressed I/O in blocks so that a shared bounce folio is not
decompressed or freed early.
- Submit only the written block of a shared bounce folio in
f2fs_submit_page_write(), at its offset in the folio.
- Keep compressed files on the page based read and write paths, and
compress a cluster only when all of its blocks are dirty and below
EOF.
- A folio with per-block state may hold uptodate blocks newer than the
on-disk cluster, so decompress into a bounce page and copy back only
the blocks that are not uptodate.
- Cache compressed blocks one block at a time in the compress cache.
- Lock only the unlocked pages in cancel_cluster_writeback() to avoid a
self-deadlock.
- Attach a folio state to multi-block folios when compressed writeback
starts, as the raw write path does. Otherwise the write completion,
which does not hold the folio lock, can detach the page private flags
while a task is moving them into a new folio state.
- Take sbi from the dic to count the blocks of a compressed page in
f2fs_finish_read_bio().
- Write the blocks of a multi-block folio in
f2fs_write_single_data_page() as f2fs_write_cache_folios() does,
taking the write_blocks_pending bias with a new helper shared by the
two and dropping it with f2fs_ffs_put_write_pending().
- Remove the -EOPNOTSUPP checks.
The bounce folios are now freed when the last block of a cluster
completes, also when the block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/compress.c | 520 +++++++++++++++++++++++++++++++--------------
fs/f2fs/data.c | 248 +++++++++++++--------
fs/f2fs/f2fs.h | 61 +++---
fs/f2fs/file.c | 65 +++---
fs/f2fs/inode.c | 2 +-
fs/f2fs/super.c | 7 +-
6 files changed, 582 insertions(+), 321 deletions(-)
diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c
index 0b0f31af6626..5b37ada5b111 100644
--- a/fs/f2fs/compress.c
+++ b/fs/f2fs/compress.c
@@ -57,14 +57,15 @@ struct f2fs_compress_ops {
bool (*is_level_valid)(int level);
};
-static unsigned int offset_in_cluster(struct compress_ctx *cc, pgoff_t index)
+static unsigned int offset_in_cluster(struct compress_ctx *cc, pgoff_t lblk)
{
- return index & (cc->cluster_size - 1);
+ return (lblk & (cc->cluster_size - 1)) >>
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(cc->inode));
}
-static pgoff_t cluster_idx(struct compress_ctx *cc, pgoff_t index)
+static pgoff_t cluster_idx(struct compress_ctx *cc, pgoff_t lblk)
{
- return index >> cc->log_cluster_size;
+ return lblk >> cc->log_cluster_size;
}
static pgoff_t start_idx_of_cluster(struct compress_ctx *cc)
@@ -114,7 +115,9 @@ static void f2fs_drop_rpages(struct compress_ctx *cc, int len, bool unlock)
static void f2fs_put_rpages(struct compress_ctx *cc)
{
- f2fs_drop_rpages(cc, cc->cluster_size, false);
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+
+ f2fs_drop_rpages(cc, cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi), false);
}
static void f2fs_unlock_rpages(struct compress_ctx *cc, int len)
@@ -125,9 +128,11 @@ static void f2fs_unlock_rpages(struct compress_ctx *cc, int len)
static void f2fs_put_rpages_wbc(struct compress_ctx *cc,
struct writeback_control *wbc, bool redirty, bool unlock)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
unsigned int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
if (!cc->rpages[i])
continue;
if (redirty)
@@ -158,22 +163,24 @@ void f2fs_destroy_compress_ctx(struct compress_ctx *cc, bool reuse)
cc->rpages = NULL;
cc->nr_rpages = 0;
cc->nr_cpages = 0;
- cc->valid_nr_cpages = 0;
+ cc->valid_nr_cblocks = 0;
if (!reuse)
cc->cluster_idx = NULL_CLUSTER;
}
void f2fs_compress_ctx_add_page(struct compress_ctx *cc, struct folio *folio)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
unsigned int cluster_ofs;
+ pgoff_t lblk = f2fs_folio_lblk(sbi, folio);
- if (!f2fs_cluster_can_merge_page(cc, folio->index))
- f2fs_bug_on(F2FS_I_SB(cc->inode), 1);
+ if (!f2fs_cluster_can_merge_block(cc, lblk))
+ f2fs_bug_on(sbi, 1);
- cluster_ofs = offset_in_cluster(cc, folio->index);
+ cluster_ofs = offset_in_cluster(cc, lblk);
cc->rpages[cluster_ofs] = folio_page(folio, 0);
cc->nr_rpages++;
- cc->cluster_idx = cluster_idx(cc, folio->index);
+ cc->cluster_idx = cluster_idx(cc, lblk);
}
#ifdef CONFIG_F2FS_FS_LZO
@@ -184,7 +191,7 @@ static int lzo_init_compress_ctx(struct compress_ctx *cc)
if (!cc->private)
return -ENOMEM;
- cc->clen = lzo1x_worst_compress(PAGE_SIZE << cc->log_cluster_size);
+ cc->clen = lzo1x_worst_compress(F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) << cc->log_cluster_size);
return 0;
}
@@ -220,10 +227,11 @@ static int lzo_decompress_pages(struct decompress_io_ctx *dic)
return -EIO;
}
- if (dic->rlen != PAGE_SIZE << dic->log_cluster_size) {
+ if (dic->rlen != F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size) {
f2fs_err_ratelimited(dic->sbi,
"lzo invalid rlen:%zu, expected:%lu",
- dic->rlen, PAGE_SIZE << dic->log_cluster_size);
+ dic->rlen, (unsigned long)F2FS_BLKSIZE(dic->sbi) <<
+ dic->log_cluster_size);
return -EIO;
}
return 0;
@@ -256,7 +264,7 @@ static int lz4_init_compress_ctx(struct compress_ctx *cc)
* adapt worst compress case, because lz4 compressor can handle
* output budget properly.
*/
- cc->clen = cc->rlen - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ cc->clen = cc->rlen - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
return 0;
}
@@ -300,10 +308,11 @@ static int lz4_decompress_pages(struct decompress_io_ctx *dic)
return -EIO;
}
- if (ret != PAGE_SIZE << dic->log_cluster_size) {
+ if (ret != F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size) {
f2fs_err_ratelimited(dic->sbi,
"lz4 invalid ret:%d, expected:%lu",
- ret, PAGE_SIZE << dic->log_cluster_size);
+ ret, (unsigned long)F2FS_BLKSIZE(dic->sbi) <<
+ dic->log_cluster_size);
return -EIO;
}
return 0;
@@ -358,7 +367,7 @@ static int zstd_init_compress_ctx(struct compress_ctx *cc)
cc->private = workspace;
cc->private2 = stream;
- cc->clen = cc->rlen - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ cc->clen = cc->rlen - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
return 0;
}
@@ -375,7 +384,7 @@ static int zstd_compress_pages(struct compress_ctx *cc)
zstd_in_buffer inbuf;
zstd_out_buffer outbuf;
int src_size = cc->rlen;
- int dst_size = src_size - PAGE_SIZE - COMPRESS_HEADER_SIZE;
+ int dst_size = src_size - F2FS_BLKSIZE(F2FS_I_SB(cc->inode)) - COMPRESS_HEADER_SIZE;
int ret;
inbuf.pos = 0;
@@ -419,7 +428,7 @@ static int zstd_init_decompress_ctx(struct decompress_io_ctx *dic)
void *workspace;
unsigned int workspace_size;
unsigned int max_window_size =
- MAX_COMPRESS_WINDOW_SIZE(dic->log_cluster_size);
+ MAX_COMPRESS_WINDOW_SIZE(dic->sbi, dic->log_cluster_size);
workspace_size = zstd_dstream_workspace_bound(max_window_size);
@@ -475,7 +484,7 @@ static int zstd_decompress_pages(struct decompress_io_ctx *dic)
f2fs_err_ratelimited(dic->sbi,
"%s ZSTD invalid rlen:%zu, expected:%lu",
__func__, dic->rlen,
- PAGE_SIZE << dic->log_cluster_size);
+ (unsigned long)F2FS_BLKSIZE(dic->sbi) << dic->log_cluster_size);
return -EIO;
}
@@ -550,8 +559,6 @@ bool f2fs_is_compress_backend_ready(struct inode *inode)
{
if (!f2fs_compressed_file(inode))
return true;
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode)))
- return false;
return f2fs_cops[F2FS_I(inode)->i_compress_algorithm];
}
@@ -621,13 +628,23 @@ static void *f2fs_vmap(struct page **pages, unsigned int count)
return buf;
}
+static void f2fs_vunmap(void *vaddr, unsigned int count)
+{
+ if (!vaddr)
+ return;
+ vm_unmap_ram((void *)((unsigned long)vaddr & PAGE_MASK), count);
+}
+
static int f2fs_compress_pages(struct compress_ctx *cc)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
struct f2fs_inode_info *fi = F2FS_I(cc->inode);
const struct f2fs_compress_ops *cops =
f2fs_cops[fi->i_compress_algorithm];
- unsigned int max_len, new_nr_cpages;
+ unsigned int max_len, new_cblocks;
+ unsigned int nr_rfolios, needed_folios;
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(sbi) ? 1 : 0;
+ void *cbuf_vaddr;
u32 chksum = 0;
int i, ret;
@@ -640,9 +657,12 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
goto out;
}
+ nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+
max_len = COMPRESS_HEADER_SIZE + cc->clen;
- cc->nr_cpages = DIV_ROUND_UP(max_len, PAGE_SIZE);
- cc->valid_nr_cpages = cc->nr_cpages;
+ cc->valid_nr_cblocks = DIV_ROUND_UP(max_len, F2FS_BLKSIZE(sbi));
+ cc->nr_cpages = DIV_ROUND_UP(cbuf_blk_ofs + cc->valid_nr_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
cc->cpages = page_array_alloc(sbi, cc->nr_cpages);
if (!cc->cpages) {
@@ -653,23 +673,24 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
for (i = 0; i < cc->nr_cpages; i++)
cc->cpages[i] = f2fs_compress_alloc_page();
- cc->rbuf = f2fs_vmap(cc->rpages, cc->cluster_size);
+ cc->rbuf = f2fs_vmap(cc->rpages, nr_rfolios);
if (!cc->rbuf) {
ret = -ENOMEM;
goto out_free_cpages;
}
- cc->cbuf = f2fs_vmap(cc->cpages, cc->nr_cpages);
- if (!cc->cbuf) {
+ cbuf_vaddr = f2fs_vmap(cc->cpages, cc->nr_cpages);
+ if (!cbuf_vaddr) {
ret = -ENOMEM;
goto out_vunmap_rbuf;
}
+ cc->cbuf = cbuf_vaddr + (cbuf_blk_ofs << sbi->log_blocksize);
ret = cops->compress_pages(cc);
if (ret)
goto out_vunmap_cbuf;
- max_len = PAGE_SIZE * (cc->cluster_size - 1) - COMPRESS_HEADER_SIZE;
+ max_len = F2FS_BLKSIZE(sbi) * (cc->cluster_size - 1) - COMPRESS_HEADER_SIZE;
if (cc->clen > max_len) {
ret = -EAGAIN;
@@ -685,17 +706,19 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
for (i = 0; i < COMPRESS_DATA_RESERVED_SIZE; i++)
cc->cbuf->reserved[i] = cpu_to_le32(0);
- new_nr_cpages = DIV_ROUND_UP(cc->clen + COMPRESS_HEADER_SIZE, PAGE_SIZE);
+ new_cblocks = DIV_ROUND_UP(cc->clen + COMPRESS_HEADER_SIZE, F2FS_BLKSIZE(sbi));
- /* zero out any unused part of the last page */
+ /* zero out any unused part of the last block */
memset(&cc->cbuf->cdata[cc->clen], 0,
- (new_nr_cpages * PAGE_SIZE) -
+ (new_cblocks * F2FS_BLKSIZE(sbi)) -
(cc->clen + COMPRESS_HEADER_SIZE));
- vm_unmap_ram(cc->cbuf, cc->nr_cpages);
- vm_unmap_ram(cc->rbuf, cc->cluster_size);
+ f2fs_vunmap(cc->cbuf, cc->nr_cpages);
+ f2fs_vunmap(cc->rbuf, nr_rfolios);
- for (i = new_nr_cpages; i < cc->nr_cpages; i++) {
+ needed_folios = DIV_ROUND_UP(cbuf_blk_ofs + new_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
+ for (i = needed_folios; i < cc->nr_cpages; i++) {
f2fs_compress_free_page(cc->cpages[i]);
cc->cpages[i] = NULL;
}
@@ -703,16 +726,16 @@ static int f2fs_compress_pages(struct compress_ctx *cc)
if (cops->destroy_compress_ctx)
cops->destroy_compress_ctx(cc);
- cc->valid_nr_cpages = new_nr_cpages;
+ cc->valid_nr_cblocks = new_cblocks;
trace_f2fs_compress_pages_end(cc->inode, cc->cluster_idx,
cc->clen, ret);
return 0;
out_vunmap_cbuf:
- vm_unmap_ram(cc->cbuf, cc->nr_cpages);
+ f2fs_vunmap(cc->cbuf, cc->nr_cpages);
out_vunmap_rbuf:
- vm_unmap_ram(cc->rbuf, cc->cluster_size);
+ f2fs_vunmap(cc->rbuf, nr_rfolios);
out_free_cpages:
for (i = 0; i < cc->nr_cpages; i++) {
if (cc->cpages[i])
@@ -741,7 +764,7 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
const struct f2fs_compress_ops *cops =
f2fs_cops[fi->i_compress_algorithm];
bool bypass_callback = false;
- int ret;
+ int i, ret;
trace_f2fs_decompress_pages_start(dic->inode, dic->cluster_idx,
dic->cluster_size, fi->i_compress_algorithm);
@@ -758,9 +781,9 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
}
dic->clen = le32_to_cpu(dic->cbuf->clen);
- dic->rlen = PAGE_SIZE << dic->log_cluster_size;
+ dic->rlen = F2FS_BLKSIZE(sbi) << dic->log_cluster_size;
- if (dic->clen > PAGE_SIZE * dic->nr_cpages - COMPRESS_HEADER_SIZE) {
+ if (dic->clen > F2FS_BLKSIZE(sbi) * dic->valid_nr_cblocks - COMPRESS_HEADER_SIZE) {
ret = -EFSCORRUPTED;
/* Avoid f2fs_commit_super in irq context */
@@ -787,6 +810,31 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
}
}
+ /*
+ * Folios with f2fs_folio_state were decompressed into bounce pages.
+ * Copy only the blocks that are not uptodate, so that newer data in
+ * the uptodate (possibly dirty) blocks is kept.
+ */
+ for (i = 0; !ret && i < dic->nr_rpages; i++) {
+ struct page *rpage = dic->rpages[i];
+ struct folio *folio;
+ pgoff_t lblk;
+ unsigned int b;
+
+ if (!rpage || dic->tpages[i] == rpage)
+ continue;
+
+ folio = page_folio(rpage);
+ lblk = f2fs_folio_lblk(sbi, folio);
+ for (b = 0; b < f2fs_blocks_per_folio(sbi, folio); b++) {
+ size_t off = (size_t)b << sbi->log_blocksize;
+
+ if (!f2fs_ffs_test_blk_uptodate(folio, lblk + b))
+ memcpy_page(rpage, off, dic->tpages[i], off,
+ F2FS_BLKSIZE(sbi));
+ }
+ }
+
out_release:
f2fs_release_decomp_mem(dic, bypass_callback, false);
@@ -796,8 +844,8 @@ void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task)
f2fs_decompress_end_io(dic, ret, in_task);
}
-static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
- struct folio *folio, nid_t ino, block_t blkaddr);
+static void f2fs_cache_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset, nid_t ino, block_t blkaddr);
/*
* This is called when a page of a compressed cluster has been read from disk
@@ -805,29 +853,35 @@ static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
* page being waited on in the cluster, and if so, it decompresses the cluster
* (or in the case of a failure, cleans up without actually decompressing).
*/
-void f2fs_end_read_compressed_page(struct folio *folio, bool failed,
- block_t blkaddr, bool in_task)
+void f2fs_end_read_compressed_page(struct folio *folio, size_t offset,
+ bool failed, block_t blkaddr, unsigned int nr_blocks,
+ bool in_task)
{
struct decompress_io_ctx *dic = folio->private;
struct f2fs_sb_info *sbi = dic->sbi;
+ int i;
- dec_cache_count(sbi, F2FS_RD_DATA);
+ sub_cache_counts(sbi, F2FS_RD_DATA, nr_blocks);
if (failed)
WRITE_ONCE(dic->failed, true);
- else if (blkaddr && in_task)
- f2fs_cache_compressed_page(sbi, folio,
- dic->inode->i_ino, blkaddr);
+ else if (blkaddr && in_task) {
+ for (i = 0; i < nr_blocks; i++)
+ f2fs_cache_compressed_block(sbi, folio,
+ offset + i * F2FS_BLKSIZE(sbi),
+ dic->inode->i_ino,
+ blkaddr + i);
+ }
- if (atomic_dec_and_test(&dic->remaining_pages))
+ if (atomic_sub_and_test(nr_blocks, &dic->remaining_blocks))
f2fs_decompress_cluster(dic, in_task);
}
-static bool is_page_in_cluster(struct compress_ctx *cc, pgoff_t index)
+static bool is_block_in_cluster(struct compress_ctx *cc, pgoff_t lblk)
{
if (cc->cluster_idx == NULL_CLUSTER)
return true;
- return cc->cluster_idx == cluster_idx(cc, index);
+ return cc->cluster_idx == cluster_idx(cc, lblk);
}
bool f2fs_cluster_is_empty(struct compress_ctx *cc)
@@ -837,19 +891,40 @@ bool f2fs_cluster_is_empty(struct compress_ctx *cc)
static bool f2fs_cluster_is_full(struct compress_ctx *cc)
{
- return cc->cluster_size == cc->nr_rpages;
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ int i;
+
+ if (cc->nr_rpages != nr_folios)
+ return false;
+
+ /* PG_dirty is cleared for writeback; without ffs it covered every block */
+ for (i = 0; i < nr_folios; i++) {
+ struct folio *folio = page_folio(cc->rpages[i]);
+ unsigned int nr_blocks = f2fs_blocks_per_folio(sbi, folio);
+ unsigned int start, count;
+
+ if (f2fs_folio_has_ffs(folio) &&
+ (!f2fs_ffs_dirty_blk_range(folio, 0, nr_blocks,
+ &start, &count) ||
+ count != nr_blocks))
+ return false;
+ }
+ return true;
}
-bool f2fs_cluster_can_merge_page(struct compress_ctx *cc, pgoff_t index)
+bool f2fs_cluster_can_merge_block(struct compress_ctx *cc, pgoff_t lblk)
{
if (f2fs_cluster_is_empty(cc))
return true;
- return is_page_in_cluster(cc, index);
+ return is_block_in_cluster(cc, lblk);
}
bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
int index, int nr_pages, bool uptodate)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
unsigned long pgidx = page_folio(pages[index])->index;
int i = uptodate ? 0 : 1;
@@ -857,13 +932,13 @@ bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
* when uptodate set to true, try to check all pages in cluster is
* uptodate or not.
*/
- if (uptodate && (pgidx % cc->cluster_size))
+ if (uptodate && (pgidx % nr_folios))
return false;
- if (nr_pages - index < cc->cluster_size)
+ if (nr_pages - index < nr_folios)
return false;
- for (; i < cc->cluster_size; i++) {
+ for (; i < nr_folios; i++) {
struct folio *folio = page_folio(pages[index + i]);
if (folio->index != pgidx + i)
@@ -877,17 +952,19 @@ bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
static bool cluster_has_invalid_data(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
loff_t i_size = i_size_read(cc->inode);
- unsigned nr_pages = DIV_ROUND_UP(i_size, PAGE_SIZE);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct page *page = cc->rpages[i];
- f2fs_bug_on(F2FS_I_SB(cc->inode), !page);
+ f2fs_bug_on(sbi, !page);
- /* beyond EOF */
- if (page_folio(page)->index >= nr_pages)
+ /* any block beyond EOF */
+ if (folio_pos(page_folio(page)) +
+ folio_size(page_folio(page)) - F2FS_BLKSIZE(sbi) >= i_size)
return true;
}
return false;
@@ -1014,17 +1091,17 @@ static int f2fs_decompressed_blocks(struct inode *inode,
}
/* return whether cluster is compressed one or not */
-int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index)
+int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t lblk)
{
return __f2fs_cluster_blocks(inode,
- index >> F2FS_I(inode)->i_log_cluster_size,
+ lblk >> F2FS_I(inode)->i_log_cluster_size,
CLUSTER_IS_COMPR);
}
/* return whether cluster contains non raw blocks or not */
-bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index)
+bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t lblk)
{
- unsigned int cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size;
+ unsigned int cluster_idx = lblk >> F2FS_I(inode)->i_log_cluster_size;
return f2fs_decompressed_blocks(inode, cluster_idx) !=
F2FS_I(inode)->i_cluster_size;
@@ -1045,34 +1122,57 @@ static bool cluster_may_compress(struct compress_ctx *cc)
static void set_cluster_writeback(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
- if (cc->rpages[i])
- set_page_writeback(cc->rpages[i]);
+ for (i = 0; i < nr_folios; i++) {
+ struct folio *folio;
+
+ if (!cc->rpages[i])
+ continue;
+
+ folio = page_folio(cc->rpages[i]);
+ set_page_writeback(cc->rpages[i]);
+ /*
+ * Attach a folio state before the I/O, as the raw write path
+ * does, so that the write completion, which does not hold the
+ * folio lock, never detaches the page private flags kept in
+ * folio->private.
+ */
+ if (f2fs_is_multiblock_folio(sbi, folio))
+ f2fs_ffs_find_or_alloc(folio);
}
}
static void cancel_cluster_writeback(struct compress_ctx *cc,
- struct compress_io_ctx *cic, int submitted)
+ struct compress_io_ctx *cic, int failed_blk)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
/* Wait for submitted IOs. */
- if (submitted > 1) {
+ if (failed_blk > 1) {
f2fs_submit_merged_write(F2FS_I_SB(cc->inode), DATA);
- while (atomic_read(&cic->pending_pages) !=
- (cc->valid_nr_cpages - submitted + 1))
+ while (atomic_read(&cic->pending_blocks) !=
+ (cc->valid_nr_cblocks - failed_blk + 1))
f2fs_io_schedule_timeout(DEFAULT_SCHEDULE_TIMEOUT);
}
+ inode_add_dirty_blocks(cc->inode, failed_blk);
+
/* Cancel writeback and stay locked. */
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio = page_folio(cc->rpages[i]);
+ unsigned int last_blk_in_folio =
+ ((i + 1) << F2FS_LOG_BLKS_PER_PAGE(sbi)) - 1;
- if (i < submitted) {
- inode_inc_dirty_block(cc->inode);
+ if (last_blk_in_folio < failed_blk) {
folio_lock(folio);
+ /* f2fs_write_compressed_pages() cleared its dirty blocks */
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0, PAGE_SIZE);
}
folio_clear_f2fs_gcing(folio);
if (folio_test_writeback(folio))
@@ -1082,21 +1182,26 @@ static void cancel_cluster_writeback(struct compress_ctx *cc,
static void set_cluster_dirty(struct compress_ctx *cc)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio;
if (!cc->rpages[i])
continue;
folio = page_folio(cc->rpages[i]);
+ /* The whole cluster is rewritten */
+ if (f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0, folio_size(folio));
folio_mark_dirty(folio);
folio_set_f2fs_gcing(folio);
}
}
static int prepare_compress_overwrite(struct compress_ctx *cc,
- struct page **pagep, pgoff_t index, void **fsdata)
+ struct page **pagep, pgoff_t lblk, void **fsdata)
{
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
struct address_space *mapping = cc->inode->i_mapping;
@@ -1115,9 +1220,13 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
if (ret)
return ret;
+ unsigned int nr_folios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+
/* keep folio reference to avoid page reclaim */
- for (i = 0; i < cc->cluster_size; i++) {
- folio = f2fs_filemap_get_folio(mapping, start_idx + i,
+ for (i = 0; i < nr_folios; i++) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, start_idx) + i;
+
+ folio = f2fs_filemap_get_folio(mapping, f_index,
fgp_flag, GFP_NOFS);
if (IS_ERR(folio)) {
ret = PTR_ERR(folio);
@@ -1147,10 +1256,12 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
goto out;
}
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
+ pgoff_t f_index = f2fs_lblk_to_folio_index(sbi, start_idx) + i;
+
f2fs_bug_on(sbi, cc->rpages[i]);
- folio = filemap_lock_folio(mapping, start_idx + i);
+ folio = filemap_lock_folio(mapping, f_index);
if (IS_ERR(folio)) {
/* folio could be truncated */
goto release_and_retry;
@@ -1171,7 +1282,7 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
if (likely(!ret)) {
*fsdata = cc->rpages;
- *pagep = cc->rpages[offset_in_cluster(cc, index)];
+ *pagep = cc->rpages[offset_in_cluster(cc, lblk)];
return cc->cluster_size;
}
@@ -1184,23 +1295,23 @@ static int prepare_compress_overwrite(struct compress_ctx *cc,
}
int f2fs_prepare_compress_overwrite(struct inode *inode,
- struct page **pagep, pgoff_t index, void **fsdata)
+ struct page **pagep, pgoff_t lblk, void **fsdata)
{
struct compress_ctx cc = {
.inode = inode,
.log_cluster_size = F2FS_I(inode)->i_log_cluster_size,
.cluster_size = F2FS_I(inode)->i_cluster_size,
- .cluster_idx = index >> F2FS_I(inode)->i_log_cluster_size,
+ .cluster_idx = lblk >> F2FS_I(inode)->i_log_cluster_size,
.rpages = NULL,
.nr_rpages = 0,
.vi = NULL, /* can't write to fsverity files */
};
- return prepare_compress_overwrite(&cc, pagep, index, fsdata);
+ return prepare_compress_overwrite(&cc, pagep, lblk, fsdata);
}
bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
- pgoff_t index, unsigned copied)
+ pgoff_t lblk, unsigned int copied)
{
struct compress_ctx cc = {
@@ -1210,7 +1321,7 @@ bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
.rpages = fsdata,
};
struct folio *folio = page_folio(cc.rpages[0]);
- bool first_index = (index == folio->index);
+ bool first_index = (lblk == f2fs_folio_lblk(F2FS_I_SB(inode), folio));
if (copied)
set_cluster_dirty(&cc);
@@ -1223,12 +1334,15 @@ bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
{
+ struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
void *fsdata = NULL;
struct page *pagep;
struct page **rpages;
int log_cluster_size = F2FS_I(inode)->i_log_cluster_size;
- pgoff_t start_idx = from >> (PAGE_SHIFT + log_cluster_size) <<
+ pgoff_t start_idx = (from >> (sbi->log_blocksize + log_cluster_size)) <<
log_cluster_size;
+ unsigned int nr_folios = (1 << log_cluster_size) >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
int i;
int err;
@@ -1252,9 +1366,9 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
rpages = fsdata;
- for (i = (1 << log_cluster_size) - 1; i >= 0; i--) {
+ for (i = nr_folios - 1; i >= 0; i--) {
struct folio *folio = page_folio(rpages[i]);
- loff_t start = (loff_t)folio->index << PAGE_SHIFT;
+ loff_t start = folio_pos(folio);
loff_t offset = from > start ? from - start : 0;
folio_zero_segment(folio, offset, folio_size(folio));
@@ -1266,14 +1380,15 @@ int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock)
f2fs_compress_write_end(inode, fsdata, start_idx, true);
err = filemap_write_and_wait_range(inode->i_mapping,
- round_down(from, 1 << log_cluster_size << PAGE_SHIFT),
+ round_down(from, 1 << log_cluster_size << sbi->log_blocksize),
LLONG_MAX);
if (err)
return err;
truncate_pagecache(inode, from);
- return f2fs_do_truncate_blocks(inode, round_up(from, PAGE_SIZE), lock);
+ return f2fs_do_truncate_blocks(inode, round_up(from, F2FS_BLKSIZE(sbi)),
+ lock);
}
static int f2fs_write_compressed_pages(struct compress_ctx *cc,
@@ -1303,8 +1418,9 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
struct node_info ni;
struct compress_io_ctx *cic;
struct f2fs_lock_context lc;
+ unsigned int nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
pgoff_t start_idx = start_idx_of_cluster(cc);
- unsigned int last_index = cc->cluster_size - 1;
+ unsigned int blk_idx, c_folio_idx, c_folio_ofs;
loff_t psize;
int i, err;
bool quota_inode = IS_NOQUOTA(inode);
@@ -1338,10 +1454,10 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
goto out_put_dnode;
}
- folio = page_folio(cc->rpages[last_index]);
+ folio = page_folio(cc->rpages[nr_rfolios - 1]);
psize = folio_next_pos(folio);
- err = f2fs_get_node_info(fio.sbi, dn.nid, &ni, false);
+ err = f2fs_get_node_info(sbi, dn.nid, &ni, false);
if (err)
goto out_put_dnode;
@@ -1353,18 +1469,30 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
cic->magic = F2FS_COMPRESSED_PAGE_MAGIC;
cic->inode = inode;
- atomic_set(&cic->pending_pages, cc->valid_nr_cpages);
- cic->rpages = page_array_alloc(sbi, cc->cluster_size);
+ atomic_set(&cic->pending_blocks, cc->valid_nr_cblocks);
+ cic->rpages = page_array_alloc(sbi, nr_rfolios);
if (!cic->rpages)
goto out_put_cic;
- cic->nr_rpages = cc->cluster_size;
+ cic->nr_rpages = nr_rfolios;
+ cic->cpages = cc->cpages;
+ cic->nr_cpages = cc->nr_cpages;
+ cc->cpages = NULL;
+
+ for (i = 0; i < cic->nr_cpages; i++) {
+ pgoff_t fidx;
+
+ if (!cic->cpages[i])
+ break;
+
+ fidx = f2fs_has_subpage_blocks(sbi) ?
+ (f2fs_lblk_to_folio_index(sbi, start_idx) + i) :
+ (start_idx + 1 + i);
- for (i = 0; i < cc->valid_nr_cpages; i++) {
- f2fs_set_compressed_page(cc->cpages[i], inode,
- page_folio(cc->rpages[i + 1])->index, cic);
- fio.compressed_page = cc->cpages[i];
+ f2fs_set_compressed_page(cic->cpages[i], inode, fidx, cic);
+ }
+ for (i = 0; i < cc->valid_nr_cblocks; i++) {
fio.old_blkaddr = data_blkaddr(dn.inode, dn.node_entry,
dn.ofs_in_node + i + 1);
@@ -1374,14 +1502,15 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
set_cluster_writeback(cc);
- for (i = 0; i < cc->cluster_size; i++)
+ for (i = 0; i < nr_rfolios; i++)
cic->rpages[i] = cc->rpages[i];
for (i = 0; i < cc->cluster_size; i++, dn.ofs_in_node++) {
block_t blkaddr;
+ unsigned int r_idx = i >> F2FS_LOG_BLKS_PER_PAGE(sbi);
blkaddr = f2fs_data_blkaddr(&dn);
- fio.folio = page_folio(cc->rpages[i]);
+ fio.folio = page_folio(cc->rpages[r_idx]);
fio.old_blkaddr = blkaddr;
/* cluster header */
@@ -1397,7 +1526,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
if (fio.compr_blocks && __is_valid_data_blkaddr(blkaddr))
fio.compr_blocks++;
- if (i > cc->valid_nr_cpages) {
+ if (i > cc->valid_nr_cblocks) {
if (__is_valid_data_blkaddr(blkaddr)) {
f2fs_invalidate_blocks(sbi, blkaddr, 1);
f2fs_update_data_blkaddr(&dn, NEW_ADDR);
@@ -1407,9 +1536,13 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
f2fs_bug_on(fio.sbi, blkaddr == NULL_ADDR);
- fio.compressed_page = cc->cpages[i - 1];
+ blk_idx = f2fs_has_subpage_blocks(sbi) ? i : (i - 1);
+ c_folio_idx = blk_idx / F2FS_BLKS_PER_PAGE(sbi);
+ c_folio_ofs = blk_idx % F2FS_BLKS_PER_PAGE(sbi);
+
+ fio.compressed_page = cic->cpages[c_folio_idx];
+ fio.folio_offset = c_folio_ofs;
- cc->cpages[i - 1] = NULL;
fio.submitted = 0;
f2fs_outplace_write_data(&dn, &fio);
if (unlikely(!fio.submitted)) {
@@ -1420,13 +1553,20 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
(*submitted)++;
unlock_continue:
inode_dec_dirty_block(cc->inode);
- folio_unlock(fio.folio);
+ if ((i + 1) % F2FS_BLKS_PER_PAGE(sbi) == 0) {
+ f2fs_ffs_clear_subrange_dirty_and_test(fio.folio,
+ 0, PAGE_SIZE);
+ folio_unlock(fio.folio);
+ }
}
+ /* count pages like the raw write path */
+ *submitted = DIV_ROUND_UP(*submitted, F2FS_BLKS_PER_PAGE(sbi));
+
if (fio.compr_blocks)
f2fs_i_compr_blocks_update(inode, fio.compr_blocks - 1, false);
- f2fs_i_compr_blocks_update(inode, cc->valid_nr_cpages, true);
- add_compr_block_stat(inode, cc->valid_nr_cpages);
+ f2fs_i_compr_blocks_update(inode, cc->valid_nr_cblocks, true);
+ add_compr_block_stat(inode, cc->valid_nr_cblocks);
set_inode_flag(cc->inode, FI_APPEND_WRITE);
@@ -1442,13 +1582,18 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
spin_unlock(&fi->i_size_lock);
f2fs_put_rpages(cc);
- page_array_free(sbi, cc->cpages, cc->nr_cpages);
- cc->cpages = NULL;
f2fs_destroy_compress_ctx(cc, false);
return 0;
out_free_page_array:
- page_array_free(sbi, cic->rpages, cc->cluster_size);
+ page_array_free(sbi, cic->rpages, nr_rfolios);
+
+ for (i = 0; i < cic->nr_cpages; i++) {
+ if (!cic->cpages[i])
+ continue;
+ f2fs_compress_free_page(cic->cpages[i]);
+ }
+ page_array_free(sbi, cic->cpages, cic->nr_cpages);
out_put_cic:
kmem_cache_free(cic_entry_slab, cic);
out_put_dnode:
@@ -1459,18 +1604,21 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc,
else
f2fs_unlock_op(sbi, &lc);
out_free:
- for (i = 0; i < cc->valid_nr_cpages; i++) {
- f2fs_compress_free_page(cc->cpages[i]);
- cc->cpages[i] = NULL;
+ if (cc->cpages) {
+ for (i = 0; i < cc->nr_cpages; i++) {
+ if (!cc->cpages[i])
+ continue;
+ f2fs_compress_free_page(cc->cpages[i]);
+ }
+ page_array_free(sbi, cc->cpages, cc->nr_cpages);
+ cc->cpages = NULL;
}
- page_array_free(sbi, cc->cpages, cc->nr_cpages);
- cc->cpages = NULL;
return -EAGAIN;
}
-void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
+void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio,
+ unsigned int nr_blocks)
{
- struct page *page = &folio->page;
struct f2fs_sb_info *sbi = bio->bi_private;
struct compress_io_ctx *cic = folio->private;
enum count_type type = WB_DATA_TYPE(folio, true);
@@ -1479,13 +1627,17 @@ void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
if (unlikely(bio->bi_status != BLK_STS_OK))
mapping_set_error(cic->inode->i_mapping, -EIO);
- f2fs_compress_free_page(page);
-
- if (atomic_dec_return(&cic->pending_pages)) {
- dec_cache_count(sbi, type);
+ if (!atomic_sub_and_test(nr_blocks, &cic->pending_blocks)) {
+ sub_cache_counts(sbi, type, nr_blocks);
return;
}
+ for (i = 0; i < cic->nr_cpages; i++) {
+ if (cic->cpages[i])
+ f2fs_compress_free_page(cic->cpages[i]);
+ }
+ page_array_free(sbi, cic->cpages, cic->nr_cpages);
+
for (i = 0; i < cic->nr_rpages; i++) {
struct folio *rfolio;
@@ -1499,12 +1651,12 @@ void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio)
kmem_cache_free(cic_entry_slab, cic);
/*
- * Make sure dec_cache_count() is the last access to sbi.
+ * Make sure sub_cache_counts() is the last access to sbi.
* Once it drops the F2FS_WB_CP_DATA counter to zero, the
* unmount thread can proceed to destroy sbi and
* sbi->page_array_slab.
*/
- dec_cache_count(sbi, type);
+ sub_cache_counts(sbi, type, nr_blocks);
}
static int f2fs_write_raw_pages(struct compress_ctx *cc,
@@ -1517,10 +1669,12 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
struct f2fs_lock_context lc;
int submitted, compr_blocks, i;
int ret = 0;
+ unsigned int nr_folios = cc->cluster_size >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi);
compr_blocks = f2fs_compressed_blocks(cc);
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
if (!cc->rpages[i])
continue;
@@ -1535,7 +1689,7 @@ static int f2fs_write_raw_pages(struct compress_ctx *cc,
if (compr_blocks > 0)
f2fs_lock_op(sbi, &lc);
- for (i = 0; i < cc->cluster_size; i++) {
+ for (i = 0; i < nr_folios; i++) {
struct folio *folio;
if (!cc->rpages[i])
@@ -1639,17 +1793,27 @@ static int f2fs_prepare_decomp_mem(struct decompress_io_ctx *dic,
bool pre_alloc)
{
const struct f2fs_compress_ops *cops = f2fs_cops[dic->compress_algorithm];
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(dic->sbi);
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(dic->sbi) ? 1 : 0;
+ void *cbuf_vaddr;
int i;
if (!allow_memalloc_for_decomp(dic->sbi, pre_alloc))
return 0;
- dic->tpages = page_array_alloc(dic->sbi, dic->cluster_size);
+ dic->tpages = page_array_alloc(dic->sbi, nr_rfolios);
if (!dic->tpages)
return -ENOMEM;
- for (i = 0; i < dic->cluster_size; i++) {
- if (dic->rpages[i]) {
+ for (i = 0; i < nr_rfolios; i++) {
+ /*
+ * A folio with f2fs_folio_state may have uptodate blocks newer
+ * than the on-disk cluster (e.g., dirty blocks left after a
+ * partial truncate). Decompress it into a bounce page instead,
+ * see f2fs_decompress_cluster().
+ */
+ if (dic->rpages[i] &&
+ !f2fs_folio_has_ffs(page_folio(dic->rpages[i]))) {
dic->tpages[i] = dic->rpages[i];
continue;
}
@@ -1657,14 +1821,16 @@ static int f2fs_prepare_decomp_mem(struct decompress_io_ctx *dic,
dic->tpages[i] = f2fs_compress_alloc_page();
}
- dic->rbuf = f2fs_vmap(dic->tpages, dic->cluster_size);
+ dic->rbuf = f2fs_vmap(dic->tpages, nr_rfolios);
if (!dic->rbuf)
return -ENOMEM;
- dic->cbuf = f2fs_vmap(dic->cpages, dic->nr_cpages);
- if (!dic->cbuf)
+ cbuf_vaddr = f2fs_vmap(dic->cpages, dic->nr_cpages);
+ if (!cbuf_vaddr)
return -ENOMEM;
+ dic->cbuf = cbuf_vaddr + (cbuf_blk_ofs << dic->sbi->log_blocksize);
+
if (cops->init_decompress_ctx)
return cops->init_decompress_ctx(dic);
@@ -1675,6 +1841,7 @@ static void f2fs_release_decomp_mem(struct decompress_io_ctx *dic,
bool bypass_destroy_callback, bool pre_alloc)
{
const struct f2fs_compress_ops *cops = f2fs_cops[dic->compress_algorithm];
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(dic->sbi);
if (!allow_memalloc_for_decomp(dic->sbi, pre_alloc))
return;
@@ -1682,11 +1849,15 @@ static void f2fs_release_decomp_mem(struct decompress_io_ctx *dic,
if (!bypass_destroy_callback && cops->destroy_decompress_ctx)
cops->destroy_decompress_ctx(dic);
- if (dic->cbuf)
- vm_unmap_ram(dic->cbuf, dic->nr_cpages);
+ if (dic->cbuf) {
+ f2fs_vunmap(dic->cbuf, dic->nr_cpages);
+ dic->cbuf = NULL;
+ }
- if (dic->rbuf)
- vm_unmap_ram(dic->rbuf, dic->cluster_size);
+ if (dic->rbuf) {
+ f2fs_vunmap(dic->rbuf, nr_rfolios);
+ dic->rbuf = NULL;
+ }
}
static void f2fs_free_dic(struct decompress_io_ctx *dic,
@@ -1697,13 +1868,17 @@ struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc)
struct decompress_io_ctx *dic;
pgoff_t start_idx = start_idx_of_cluster(cc);
struct f2fs_sb_info *sbi = F2FS_I_SB(cc->inode);
+ unsigned int nr_rfolios = cc->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
+ unsigned int cbuf_blk_ofs = f2fs_has_subpage_blocks(sbi) ? 1 : 0;
+ unsigned int nr_cpages = DIV_ROUND_UP(cbuf_blk_ofs + cc->valid_nr_cblocks,
+ F2FS_BLKS_PER_PAGE(sbi));
int i, ret;
dic = f2fs_kmem_cache_alloc(dic_entry_slab, GFP_F2FS_ZERO, false, sbi);
if (!dic)
return ERR_PTR(-ENOMEM);
- dic->rpages = page_array_alloc(sbi, cc->cluster_size);
+ dic->rpages = page_array_alloc(sbi, nr_rfolios);
if (!dic->rpages) {
kmem_cache_free(dic_entry_slab, dic);
return ERR_PTR(-ENOMEM);
@@ -1713,33 +1888,38 @@ struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc)
dic->inode = cc->inode;
dic->sbi = sbi;
dic->compress_algorithm = F2FS_I(cc->inode)->i_compress_algorithm;
- atomic_set(&dic->remaining_pages, cc->nr_cpages);
+ atomic_set(&dic->remaining_blocks, cc->valid_nr_cblocks);
dic->cluster_idx = cc->cluster_idx;
dic->cluster_size = cc->cluster_size;
dic->log_cluster_size = cc->log_cluster_size;
- dic->nr_cpages = cc->nr_cpages;
+ dic->valid_nr_cblocks = cc->valid_nr_cblocks;
+ dic->nr_cpages = nr_cpages;
refcount_set(&dic->refcnt, 1);
dic->failed = false;
dic->vi = cc->vi;
- for (i = 0; i < dic->cluster_size; i++)
+ for (i = 0; i < nr_rfolios; i++)
dic->rpages[i] = cc->rpages[i];
- dic->nr_rpages = cc->cluster_size;
+ dic->nr_rpages = nr_rfolios;
- dic->cpages = page_array_alloc(sbi, dic->nr_cpages);
+ dic->cpages = page_array_alloc(sbi, nr_cpages);
if (!dic->cpages) {
ret = -ENOMEM;
goto out_free;
}
- for (i = 0; i < dic->nr_cpages; i++) {
+ for (i = 0; i < nr_cpages; i++) {
struct page *page;
+ pgoff_t fidx = f2fs_has_subpage_blocks(sbi) ?
+ (f2fs_lblk_to_folio_index(sbi, start_idx) + i) :
+ (start_idx + 1 + i);
page = f2fs_compress_alloc_page();
- f2fs_set_compressed_page(page, cc->inode,
- start_idx + i + 1, dic);
+ f2fs_set_compressed_page(page, cc->inode, fidx, dic);
dic->cpages[i] = page;
}
+ dic->valid_nr_cblocks = cc->valid_nr_cblocks;
+ dic->nr_cpages = nr_cpages;
ret = f2fs_prepare_decomp_mem(dic, true);
if (ret)
@@ -1758,18 +1938,19 @@ static void f2fs_free_dic(struct decompress_io_ctx *dic,
int i;
/* use sbi in dic to avoid UFA of dic->inode*/
struct f2fs_sb_info *sbi = dic->sbi;
+ unsigned int nr_rfolios = dic->cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi);
f2fs_release_decomp_mem(dic, bypass_destroy_callback, true);
if (dic->tpages) {
- for (i = 0; i < dic->cluster_size; i++) {
- if (dic->rpages[i])
+ for (i = 0; i < nr_rfolios; i++) {
+ if (dic->tpages[i] == dic->rpages[i])
continue;
if (!dic->tpages[i])
continue;
f2fs_compress_free_page(dic->tpages[i]);
}
- page_array_free(sbi, dic->tpages, dic->cluster_size);
+ page_array_free(sbi, dic->tpages, nr_rfolios);
}
if (dic->cpages) {
@@ -1812,15 +1993,19 @@ static void f2fs_verify_cluster(struct work_struct *work)
int i;
/* Verify, update, and unlock the decompressed pages. */
- for (i = 0; i < dic->cluster_size; i++) {
+ for (i = 0; i < dic->nr_rpages; i++) {
struct page *rpage = dic->rpages[i];
struct folio *rfolio;
if (!rpage)
continue;
rfolio = page_folio(rpage);
- if (fsverity_verify_folio(dic->vi, rfolio))
+ if (fsverity_verify_folio(dic->vi, rfolio)) {
+ if (f2fs_is_multiblock_folio(dic->sbi, rfolio))
+ f2fs_ffs_mark_subrange_uptodate(rfolio, 0,
+ folio_size(rfolio));
folio_mark_uptodate(rfolio);
+ }
folio_unlock(rfolio);
}
@@ -1849,16 +2034,20 @@ void f2fs_decompress_end_io(struct decompress_io_ctx *dic, bool failed,
}
/* Update and unlock the cluster's pagecache pages. */
- for (i = 0; i < dic->cluster_size; i++) {
+ for (i = 0; i < dic->nr_rpages; i++) {
struct page *rpage = dic->rpages[i];
if (!rpage)
continue;
- if (failed)
+ if (failed) {
ClearPageUptodate(rpage);
- else
+ } else {
+ if (f2fs_is_multiblock_folio(dic->sbi, page_folio(rpage)))
+ f2fs_ffs_mark_subrange_uptodate(page_folio(rpage), 0,
+ folio_size(page_folio(rpage)));
SetPageUptodate(rpage);
+ }
unlock_page(rpage);
}
@@ -1916,8 +2105,8 @@ void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
f2fs_drop_cache_range(COMPRESS_CACHE(sbi), blkaddr, len, false);
}
-static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
- struct folio *folio, nid_t ino, block_t blkaddr)
+static void f2fs_cache_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset, nid_t ino, block_t blkaddr)
{
struct f2fs_cached_block *entry;
@@ -1961,13 +2150,14 @@ static void f2fs_cache_compressed_page(struct f2fs_sb_info *sbi,
}
entry->ino = ino;
- memcpy(cache_address(entry), folio_address(folio), sbi->blocksize);
+ memcpy(cache_address(entry), folio_address(folio) + offset,
+ sbi->blocksize);
f2fs_cache_set_uptodate(entry);
f2fs_put_cache(entry, true);
}
-bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
- block_t blkaddr)
+bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi, struct folio *folio,
+ size_t offset, block_t blkaddr)
{
struct f2fs_cached_block *entry;
bool hitted = false;
@@ -1982,8 +2172,8 @@ bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
if (f2fs_is_compress_cache(entry) &&
f2fs_cache_test_uptodate(entry)) {
atomic_inc(&sbi->compress_page_hit);
- memcpy(folio_address(folio),
- cache_address(entry), folio_size(folio));
+ memcpy(folio_address(folio) + offset,
+ cache_address(entry), sbi->blocksize);
hitted = true;
}
f2fs_put_cache(entry, true);
diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index a7cc7231b572..869005dfb40b 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -150,12 +150,17 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task)
bool finished = true;
bool uptodate = bio->bi_status == BLK_STS_OK;
- if (!folio_test_large(folio) &&
- f2fs_is_compressed_page(folio)) {
+ if (f2fs_is_compressed_page(folio)) {
+ struct decompress_io_ctx *dic = folio->private;
+
+ /* The cluster may be unlocked, so do not use its inode. */
+ nr_blocks = fi.length >> dic->sbi->log_blocksize;
if (ctx && !ctx->decompression_attempted)
- f2fs_end_read_compressed_page(folio, true, 0,
+ f2fs_end_read_compressed_page(folio, fi.offset,
+ true, 0, nr_blocks,
in_task);
- f2fs_put_folio_dic(folio, in_task);
+ while (nr_blocks--)
+ f2fs_put_folio_dic(folio, in_task);
continue;
}
@@ -265,14 +270,18 @@ static void f2fs_handle_step_decompress(struct bio_post_read_ctx *ctx,
bio_for_each_folio_all(fi, ctx->bio) {
struct folio *folio = fi.folio;
+ struct f2fs_sb_info *sbi = F2FS_F_SB(folio);
+ unsigned int nr_blocks = fi.length >> sbi->log_blocksize;
- if (f2fs_is_compressed_page(folio))
- f2fs_end_read_compressed_page(folio, false, blkaddr,
- in_task);
- else
+ if (f2fs_is_compressed_page(folio)) {
+ f2fs_end_read_compressed_page(folio, fi.offset,
+ false, blkaddr,
+ nr_blocks, in_task);
+ } else {
all_compressed = false;
+ }
- blkaddr++;
+ blkaddr += nr_blocks;
}
ctx->decompression_attempted = true;
@@ -425,7 +434,7 @@ static void f2fs_write_end_bio(struct bio *bio)
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_is_compressed_page(folio)) {
- f2fs_compress_write_end_io(bio, folio);
+ f2fs_compress_write_end_io(bio, folio, nr_blocks);
continue;
}
#endif
@@ -1337,9 +1346,14 @@ void f2fs_submit_page_write(struct f2fs_io_info *fio)
io->fio = *fio;
}
- bio_offset = bio_folio == fio->folio ? F2FS_FIO_BIO_OFFSET(fio) : 0;
- bio_len = bio_folio == fio->folio ? F2FS_FIO_BIO_SIZE(fio) :
- folio_size(bio_folio);
+ /* A compressed page holds several blocks with subpage blocks. */
+ if (bio_folio == fio->folio || fio->compressed_page) {
+ bio_offset = F2FS_FIO_BIO_OFFSET(fio);
+ bio_len = F2FS_FIO_BIO_SIZE(fio);
+ } else {
+ bio_offset = 0;
+ bio_len = folio_size(bio_folio);
+ }
if (!bio_add_folio(io->bio, bio_folio, bio_len, bio_offset)) {
__submit_merged_bio(io);
@@ -2728,7 +2742,7 @@ static int f2fs_read_single_page(struct inode *inode, struct fsverity_info *vi,
#ifdef CONFIG_F2FS_FS_COMPRESSION
int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
- unsigned nr_pages, sector_t *last_block_in_bio,
+ unsigned int nr_blocks, sector_t *last_block_in_bio,
struct readahead_control *rac, bool for_write)
{
struct dnode_of_data dn;
@@ -2765,7 +2779,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
continue;
folio = page_folio(page);
- if ((sector_t)folio->index >= last_block_in_file) {
+ if ((sector_t)f2fs_folio_lblk(sbi, folio) >= last_block_in_file) {
folio_zero_segment(folio, 0, folio_size(folio));
if (!folio_test_uptodate(folio))
folio_mark_uptodate(folio);
@@ -2811,14 +2825,14 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
ret = -EFAULT;
goto out_put_dnode;
}
- cc->nr_cpages++;
+ cc->valid_nr_cblocks++;
if (!from_dnode && i >= ei.c_len)
break;
}
/* nothing to decompress */
- if (cc->nr_cpages == 0) {
+ if (cc->valid_nr_cblocks == 0) {
ret = 0;
goto out_put_dnode;
}
@@ -2829,8 +2843,12 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
goto out_put_dnode;
}
- for (i = 0; i < cc->nr_cpages; i++) {
- struct folio *folio = page_folio(dic->cpages[i]);
+ for (i = 0; i < cc->valid_nr_cblocks; i++) {
+ unsigned int blk_idx = f2fs_has_subpage_blocks(sbi) ? (1 + i) : i;
+ unsigned int c_folio_idx = blk_idx / F2FS_BLKS_PER_PAGE(sbi);
+ unsigned int c_folio_ofs = blk_idx % F2FS_BLKS_PER_PAGE(sbi);
+ struct folio *folio = page_folio(dic->cpages[c_folio_idx]);
+ size_t offset = c_folio_ofs * blocksize;
block_t blkaddr;
struct bio_post_read_ctx *ctx;
@@ -2840,8 +2858,8 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
f2fs_wait_on_block_writeback(inode, blkaddr);
- if (f2fs_load_compressed_folio(sbi, folio, blkaddr)) {
- if (atomic_dec_and_test(&dic->remaining_pages)) {
+ if (f2fs_load_compressed_block(sbi, folio, offset, blkaddr)) {
+ if (atomic_dec_and_test(&dic->remaining_blocks)) {
f2fs_decompress_cluster(dic, true);
break;
}
@@ -2850,7 +2868,7 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
if (bio && (!page_is_mergeable(sbi, bio,
*last_block_in_bio, blkaddr) ||
- !f2fs_crypt_mergeable_bio(bio, inode, folio->index, NULL))) {
+ !f2fs_crypt_mergeable_bio(bio, inode, start_idx + 1 + i, NULL))) {
submit_and_realloc:
f2fs_submit_read_bio(sbi, bio, DATA);
bio = NULL;
@@ -2858,11 +2876,11 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
if (!bio)
bio = f2fs_grab_read_bio(inode, cc->vi, blkaddr,
- nr_pages - i,
+ nr_blocks - i,
f2fs_ra_op_flags(rac),
- folio->index, for_write);
+ start_idx + 1 + i, for_write);
- if (!bio_add_folio(bio, folio, blocksize, 0))
+ if (!bio_add_folio(bio, folio, blocksize, offset))
goto submit_and_realloc;
ctx = get_post_read_ctx(bio);
@@ -3525,7 +3543,7 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
.rpages = NULL,
.cpages = NULL,
.nr_rpages = 0,
- .nr_cpages = 0,
+ .valid_nr_cblocks = 0,
};
pgoff_t nc_cluster_idx = NULL_CLUSTER;
pgoff_t index;
@@ -3535,14 +3553,19 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
unsigned max_nr_pages = nr_pages;
int ret = 0;
- if (f2fs_has_multiblock_folios(inode))
+ if (mapping_large_folio_support(inode->i_mapping) ||
+ (f2fs_has_subpage_blocks(F2FS_I_SB(inode)) &&
+ !f2fs_compressed_file(inode)))
return f2fs_read_data_large_folio(inode, vi, rac, folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_compressed_file(inode)) {
+ unsigned int nr_rfolios = cc.cluster_size >>
+ F2FS_LOG_BLKS_PER_PAGE(F2FS_I_SB(inode));
+
index = rac ? readahead_index(rac) : folio->index;
- max_nr_pages = round_up(index + nr_pages, cc.cluster_size) -
- round_down(index, cc.cluster_size);
+ max_nr_pages = round_up(index + nr_pages, nr_rfolios) -
+ round_down(index, nr_rfolios);
}
#endif
@@ -3564,16 +3587,16 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
f2fs_update_read_folio_count(F2FS_I_SB(inode), folio);
#ifdef CONFIG_F2FS_FS_COMPRESSION
- index = folio->index;
+ pgoff_t f_lblk = f2fs_folio_lblk(F2FS_I_SB(inode), folio);
if (!f2fs_compressed_file(inode))
goto read_single_page;
/* there are remained compressed pages, submit them */
- if (!f2fs_cluster_can_merge_page(&cc, index)) {
+ if (!f2fs_cluster_can_merge_block(&cc, f_lblk)) {
cc.vi = vi;
ret = f2fs_read_multi_pages(&cc, &bio,
- max_nr_pages,
+ max_nr_pages * F2FS_BLKS_PER_PAGE(F2FS_I_SB(inode)),
&last_block_in_bio,
rac, false);
f2fs_destroy_compress_ctx(&cc, false);
@@ -3581,15 +3604,15 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
goto set_error_page;
}
if (cc.cluster_idx == NULL_CLUSTER) {
- if (nc_cluster_idx == index >> cc.log_cluster_size)
+ if (nc_cluster_idx == f_lblk >> cc.log_cluster_size)
goto read_single_page;
- ret = f2fs_is_compressed_cluster(inode, index);
+ ret = f2fs_is_compressed_cluster(inode, f_lblk);
if (ret < 0)
goto set_error_page;
else if (!ret) {
nc_cluster_idx =
- index >> cc.log_cluster_size;
+ f_lblk >> cc.log_cluster_size;
goto read_single_page;
}
@@ -3605,15 +3628,21 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
read_single_page:
#endif
- ret = f2fs_read_single_page(inode, vi, folio, max_nr_pages,
- &map, &bio, &last_block_in_bio,
- rac);
- if (ret) {
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
+ ret = f2fs_read_data_large_folio(inode, vi, NULL, folio);
+ if (ret)
+ continue;
+ } else {
+ ret = f2fs_read_single_page(inode, vi, folio,
+ max_nr_pages, &map, &bio,
+ &last_block_in_bio, rac);
+ if (ret) {
#ifdef CONFIG_F2FS_FS_COMPRESSION
set_error_page:
#endif
- folio_zero_segment(folio, 0, folio_size(folio));
- folio_unlock(folio);
+ folio_zero_segment(folio, 0, folio_size(folio));
+ folio_unlock(folio);
+ }
}
#ifdef CONFIG_F2FS_FS_COMPRESSION
next_page:
@@ -3625,7 +3654,8 @@ static int f2fs_mpage_readpages(struct inode *inode, struct fsverity_info *vi,
if (nr_pages == 1 && !f2fs_cluster_is_empty(&cc)) {
cc.vi = vi;
ret = f2fs_read_multi_pages(&cc, &bio,
- max_nr_pages,
+ max_nr_pages *
+ F2FS_BLKS_PER_PAGE(F2FS_I_SB(inode)),
&last_block_in_bio,
rac, false);
f2fs_destroy_compress_ctx(&cc, false);
@@ -3934,7 +3964,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type,
- u64 start, u64 end)
+ u64 start, u64 end,
+ int compr_blocks)
{
struct inode *inode = folio->mapping->host;
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
@@ -3967,6 +3998,7 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
.folio_blkcnt = 1,
.encrypted_page = NULL,
.submitted = 0,
+ .compr_blocks = (i == 0) ? compr_blocks : 0,
.need_lock = LOCK_DONE,
.meta_gc = f2fs_meta_inode_gc_required(inode) ? 1 : 0,
.io_type = io_type,
@@ -3977,8 +4009,12 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
struct f2fs_folio_state *ffs =
(struct f2fs_folio_state *)folio->private;
- /* Write only dirty blocks */
- if (!f2fs_ffs_test_blk_dirty(folio, data_idx))
+ /*
+ * Write only dirty blocks, or all blocks of a
+ * compressed cluster.
+ */
+ if (!compr_blocks &&
+ !f2fs_ffs_test_blk_dirty(folio, data_idx))
continue;
atomic_inc(&ffs->write_blocks_pending);
}
@@ -4053,6 +4089,9 @@ static int f2fs_write_folio_dirty_range(struct folio *folio, int *submitted,
fio.version = ni.version;
+ if (fio.compr_blocks && fio.old_blkaddr == COMPRESS_ADDR)
+ f2fs_i_compr_blocks_update(inode, fio.compr_blocks - 1, false);
+
f2fs_outplace_write_data(&dn, &fio);
f2fs_put_dnode(&dn);
/* The outplace write path failed to submit this block. */
@@ -4139,6 +4178,22 @@ static enum f2fs_wb_action f2fs_wb_folio_action(struct f2fs_sb_info *sbi,
return F2FS_WB_WRITE;
}
+/*
+ * Hold a bias on write_blocks_pending while the blocks of a multi-block
+ * folio are submitted, so that their completions cannot end the folio
+ * writeback early. Without ffs, the folio dirty flag covered @len bytes.
+ */
+static void f2fs_folio_get_write_bias(struct folio *folio, size_t len)
+{
+ struct f2fs_folio_state *ffs;
+
+ if (!f2fs_folio_has_ffs(folio))
+ f2fs_ffs_mark_subrange_dirty(folio, 0, len);
+ ffs = folio->private;
+ WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
+ atomic_inc(&ffs->write_blocks_pending);
+}
+
int f2fs_write_single_data_page(struct folio *folio, int *submitted,
struct bio **bio,
sector_t *last_block,
@@ -4157,6 +4212,7 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
bool need_balance_fs = false;
bool quota_inode = IS_NOQUOTA(inode);
int err = 0;
+ int local_submitted = 0;
struct f2fs_io_info fio = {
.sbi = sbi,
.ino = inode->i_ino,
@@ -4203,6 +4259,42 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
folio_zero_segment(folio, offset, folio_size(folio));
write:
+ if (f2fs_is_multiblock_folio(sbi, folio)) {
+ struct f2fs_lock_context lc;
+ unsigned int to_blk = f2fs_blocks_per_folio(sbi, folio);
+ u64 pos = folio_pos(folio);
+
+ if (!compr_blocks && !f2fs_verity_in_progress(inode))
+ to_blk = min_t(unsigned int,
+ F2FS_BLK_ALIGN(sbi, i_size - pos),
+ to_blk);
+
+ /* Deadlock due to between page->lock and f2fs_lock_op */
+ if (!compr_blocks && !f2fs_trylock_op(sbi, &lc)) {
+ err = -EAGAIN;
+ goto redirty_out;
+ }
+
+ folio_start_writeback(folio);
+ f2fs_folio_get_write_bias(folio, F2FS_BLK_TO_BYTES(sbi, to_blk));
+
+ err = f2fs_write_folio_dirty_range(folio, &local_submitted,
+ wbc, io_type, pos,
+ pos + F2FS_BLK_TO_BYTES(sbi, to_blk),
+ compr_blocks);
+
+ if (!compr_blocks)
+ f2fs_unlock_op(sbi, &lc);
+
+ if (!err)
+ f2fs_ffs_clear_subrange_dirty_and_test(folio, 0,
+ F2FS_BLK_TO_BYTES(sbi, to_blk));
+
+ f2fs_ffs_put_write_pending(folio, 1);
+
+ goto done;
+ }
+
if (f2fs_cp_controlled_inode(inode)) {
struct f2fs_lock_context lc;
@@ -4272,7 +4364,9 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted,
}
if (submitted)
- *submitted = fio.submitted;
+ *submitted = f2fs_is_multiblock_folio(sbi, folio) ?
+ DIV_ROUND_UP(local_submitted, F2FS_BLKS_PER_PAGE(sbi)) :
+ fio.submitted;
return 0;
@@ -4317,10 +4411,10 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
.rpages = NULL,
.nr_rpages = 0,
.cpages = NULL,
- .valid_nr_cpages = 0,
+ .valid_nr_cblocks = 0,
.rbuf = NULL,
.cbuf = NULL,
- .rlen = PAGE_SIZE * F2FS_I(inode)->i_cluster_size,
+ .rlen = F2FS_BLKSIZE(sbi) * F2FS_I(inode)->i_cluster_size,
.private = NULL,
};
#endif
@@ -4409,6 +4503,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
void *fsdata = NULL;
struct page *pagep;
int ret2;
+ pgoff_t f_lblk = f2fs_folio_lblk(sbi, folio);
ret = f2fs_init_compress_ctx(&cc);
if (ret) {
@@ -4416,8 +4511,7 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
break;
}
- if (!f2fs_cluster_can_merge_page(&cc,
- folio->index)) {
+ if (!f2fs_cluster_can_merge_block(&cc, f_lblk)) {
ret = f2fs_write_multi_pages(&cc,
&submitted, wbc, io_type);
if (!ret)
@@ -4437,14 +4531,14 @@ static int f2fs_write_cache_pages(struct address_space *mapping,
ret2 = f2fs_prepare_compress_overwrite(
inode, &pagep,
- folio->index, &fsdata);
+ f_lblk, &fsdata);
if (ret2 < 0) {
ret = ret2;
done = 1;
break;
} else if (ret2 &&
(!f2fs_compress_write_end(inode,
- fsdata, folio->index, 1) ||
+ fsdata, f_lblk, 1) ||
!f2fs_all_cluster_page_ready(&cc,
pages, i, nr_pages,
false))) {
@@ -4599,7 +4693,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
clear_inode_flag(inode, FI_HOT_DATA);
while ((folio = writeback_iter(mapping, wbc, folio, &err))) {
- struct f2fs_folio_state *ffs = NULL;
u64 isize;
size_t poff;
pgoff_t end_index;
@@ -4623,16 +4716,6 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
goto retry_out;
}
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode)) {
- WARN_ON_ONCE(1);
- mapping_set_error(mapping, -EOPNOTSUPP);
- folio_clear_dirty_for_io(folio);
- inode_sub_dirty_blocks(inode, f2fs_blocks_per_folio(sbi, folio));
- err = -EOPNOTSUPP;
- goto retry_out;
- }
-
/*
* Writing back one large folio performs multiple block allocations, so
* best-effort balance the filesystem before writing it back.
@@ -4754,25 +4837,12 @@ static int f2fs_write_cache_folios(struct address_space *mapping,
* folio has no ffs, so its whole folio is written.
*/
if (f2fs_is_multiblock_folio(sbi, folio)) {
- if (!f2fs_folio_has_ffs(folio))
- f2fs_ffs_mark_subrange_dirty(folio, 0, end_pos - pos);
- ffs = (struct f2fs_folio_state *)folio->private;
- /*
- * Multiblock folios can be submitted as multiple bios,
- * which may complete while we are still submitting the
- * remaining dirty ranges. Hold a bias on
- * write_blocks_pending to prevent folio_end_writeback()
- * from being called early.
- */
- if (!bias_added) {
- WARN_ON_ONCE(atomic_read(&ffs->write_blocks_pending) != 0);
- atomic_inc(&ffs->write_blocks_pending);
- bias_added = true;
- }
+ f2fs_folio_get_write_bias(folio, end_pos - pos);
+ bias_added = true;
}
err = f2fs_write_folio_dirty_range(folio, &submitted, wbc, io_type,
- pos, end_pos);
+ pos, end_pos, 0);
folio_submitted += submitted;
nwritten += submitted;
if (err)
@@ -4928,7 +4998,8 @@ static int __f2fs_write_data_pages(struct address_space *mapping,
account_writeback(inode, true);
blk_start_plug(&plug);
- if (f2fs_has_multiblock_folios(inode))
+ if (mapping_large_folio_support(mapping) ||
+ (f2fs_has_subpage_blocks(sbi) && !f2fs_compressed_file(inode)))
ret = f2fs_write_cache_folios(mapping, wbc, io_type);
else
ret = f2fs_write_cache_pages(mapping, wbc, io_type);
@@ -5267,10 +5338,6 @@ static int prepare_large_folio_write_begin(struct inode *inode,
size_t need_off = ori_off;
int err = 0;
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode))
- return -EOPNOTSUPP;
-
/*
* Reserve blocks without f2fs_balance_fs() since the folio is locked;
* the caller balances after unlocking it if *need_balance is set.
@@ -5464,12 +5531,6 @@ static int f2fs_write_begin(const struct kiocb *iocb,
goto fail;
}
- if (f2fs_has_subpage_blocks(sbi) &&
- f2fs_compressed_file(inode)) {
- err = -EOPNOTSUPP;
- goto fail;
- }
-
/*
* We should check this at this moment to avoid deadlock on inode page
* and #0 page. The locking rule for inline_data conversion should be:
@@ -5490,11 +5551,13 @@ static int f2fs_write_begin(const struct kiocb *iocb,
*fsdata = NULL;
+ pgoff_t f_lblk = F2FS_BYTES_TO_BLK(sbi, pos);
+
if (len == PAGE_SIZE && !(f2fs_is_atomic_file(inode)))
goto repeat;
ret = f2fs_prepare_compress_overwrite(inode, &page,
- index, fsdata);
+ f_lblk, fsdata);
if (ret < 0) {
err = ret;
goto fail;
@@ -5670,7 +5733,8 @@ static int f2fs_write_end(const struct kiocb *iocb,
#ifdef CONFIG_F2FS_FS_COMPRESSION
/* overwrite compressed file */
if (f2fs_compressed_file(inode) && fsdata) {
- f2fs_compress_write_end(inode, fsdata, folio->index, copied);
+ f2fs_compress_write_end(inode, fsdata,
+ f2fs_folio_lblk(F2FS_I_SB(inode), folio), copied);
f2fs_update_time(F2FS_I_SB(inode), REQ_TIME);
if (pos + copied > i_size_read(inode) &&
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 8e38d46c446c..0a11c61ee3cf 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1696,7 +1696,7 @@ struct compress_ctx {
unsigned int nr_rpages; /* total page number in rpages */
struct page **cpages; /* pages store compressed data in cluster */
unsigned int nr_cpages; /* total page number in cpages */
- unsigned int valid_nr_cpages; /* valid page number in cpages */
+ unsigned int valid_nr_cblocks; /* valid block number in cpages */
void *rbuf; /* virtual mapped address on rpages */
struct compress_data *cbuf; /* virtual mapped address on cpages */
size_t rlen; /* valid data length in rbuf */
@@ -1712,7 +1712,9 @@ struct compress_io_ctx {
struct inode *inode; /* inode the context belong to */
struct page **rpages; /* pages store raw data in cluster */
unsigned int nr_rpages; /* total page number in rpages */
- atomic_t pending_pages; /* in-flight compressed page count */
+ atomic_t pending_blocks; /* in-flight compressed block count */
+ struct page **cpages; /* bounce pages store compressed data */
+ unsigned int nr_cpages; /* total page number in cpages */
};
/* Context for decompressing one cluster on the read IO path */
@@ -1727,6 +1729,7 @@ struct decompress_io_ctx {
unsigned int nr_rpages; /* total page number in rpages */
struct page **cpages; /* pages store compressed data in cluster */
unsigned int nr_cpages; /* total page number in cpages */
+ unsigned int valid_nr_cblocks; /* valid compressed block number */
struct page **tpages; /* temp pages to pad holes in cluster */
void *rbuf; /* virtual mapped address on rpages */
struct compress_data *cbuf; /* virtual mapped address on cpages */
@@ -1734,16 +1737,16 @@ struct decompress_io_ctx {
size_t clen; /* valid data length in cbuf */
/*
- * The number of compressed pages remaining to be read in this cluster.
- * This is initially nr_cpages. It is decremented by 1 each time a page
- * has been read (or failed to be read). When it reaches 0, the cluster
- * is decompressed (or an error is reported).
+ * The number of compressed blocks remaining to be read in this cluster.
+ * This is initially valid_nr_cblocks. It is decremented by the number
+ * of blocks each time they have been read (or failed to be read). When
+ * it reaches 0, the cluster is decompressed (or an error is reported).
*
* If an error occurs before all the pages have been submitted for I/O,
* then this will never reach 0. In this case the I/O submitter is
* responsible for calling f2fs_decompress_end_io() instead.
*/
- atomic_t remaining_pages;
+ atomic_t remaining_blocks;
/*
* Number of references to this decompress_io_ctx.
@@ -1770,7 +1773,7 @@ struct decompress_io_ctx {
#define NULL_CLUSTER ((unsigned int)(~0))
#define MIN_COMPRESS_LOG_SIZE 2
#define MAX_COMPRESS_LOG_SIZE 8
-#define MAX_COMPRESS_WINDOW_SIZE(log_size) ((PAGE_SIZE) << (log_size))
+#define MAX_COMPRESS_WINDOW_SIZE(sbi, log_size) ((F2FS_BLKSIZE(sbi)) << (log_size))
struct f2fs_gc_kthread {
struct task_struct *f2fs_gc_task;
@@ -2338,6 +2341,12 @@ static inline unsigned int f2fs_blocks_per_folio(struct f2fs_sb_info *sbi,
#define F2FS_LOG_BLKS_PER_PAGE(sbi) (PAGE_SHIFT - (sbi)->log_blocksize)
+static inline unsigned int min_compress_log_size(struct f2fs_sb_info *sbi)
+{
+ return max_t(unsigned int, MIN_COMPRESS_LOG_SIZE,
+ F2FS_LOG_BLKS_PER_PAGE(sbi));
+}
+
static inline bool f2fs_has_subpage_blocks(struct f2fs_sb_info *sbi)
{
return F2FS_BLKSIZE(sbi) < PAGE_SIZE;
@@ -4990,20 +4999,22 @@ enum cluster_check_type {
bool f2fs_is_compressed_page(struct folio *folio);
struct folio *f2fs_compress_control_folio(struct folio *folio);
int f2fs_prepare_compress_overwrite(struct inode *inode,
- struct page **pagep, pgoff_t index, void **fsdata);
+ struct page **pagep, pgoff_t lblk, void **fsdata);
bool f2fs_compress_write_end(struct inode *inode, void *fsdata,
- pgoff_t index, unsigned copied);
+ pgoff_t lblk, unsigned int copied);
int f2fs_truncate_partial_cluster(struct inode *inode, u64 from, bool lock);
-void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio);
+void f2fs_compress_write_end_io(struct bio *bio, struct folio *folio,
+ unsigned int nr_blocks);
bool f2fs_is_compress_backend_ready(struct inode *inode);
bool f2fs_is_compress_level_valid(int alg, int lvl);
int __init f2fs_init_compress_mempool(void);
void f2fs_destroy_compress_mempool(void);
void f2fs_decompress_cluster(struct decompress_io_ctx *dic, bool in_task);
-void f2fs_end_read_compressed_page(struct folio *folio, bool failed,
- block_t blkaddr, bool in_task);
+void f2fs_end_read_compressed_page(struct folio *folio, size_t offset,
+ bool failed, block_t blkaddr,
+ unsigned int nr_blocks, bool in_task);
bool f2fs_cluster_is_empty(struct compress_ctx *cc);
-bool f2fs_cluster_can_merge_page(struct compress_ctx *cc, pgoff_t index);
+bool f2fs_cluster_can_merge_block(struct compress_ctx *cc, pgoff_t lblk);
bool f2fs_all_cluster_page_ready(struct compress_ctx *cc, struct page **pages,
int index, int nr_pages, bool uptodate);
bool f2fs_sanity_check_cluster(struct dnode_of_data *dn);
@@ -5012,13 +5023,13 @@ int f2fs_write_multi_pages(struct compress_ctx *cc,
int *submitted,
struct writeback_control *wbc,
enum iostat_type io_type);
-int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t index);
-bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t index);
+int f2fs_is_compressed_cluster(struct inode *inode, pgoff_t lblk);
+bool f2fs_is_sparse_cluster(struct inode *inode, pgoff_t lblk);
void f2fs_update_read_extent_tree_range_compressed(struct inode *inode,
pgoff_t fofs, block_t blkaddr,
unsigned int llen, unsigned int c_len);
int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret,
- unsigned nr_pages, sector_t *last_block_in_bio,
+ unsigned int nr_blocks, sector_t *last_block_in_bio,
struct readahead_control *rac, bool for_write);
struct decompress_io_ctx *f2fs_alloc_dic(struct compress_ctx *cc);
void f2fs_decompress_end_io(struct decompress_io_ctx *dic, bool failed,
@@ -5036,8 +5047,8 @@ int __init f2fs_init_compress_cache(void);
void f2fs_destroy_compress_cache(void);
void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
block_t blkaddr, unsigned int len);
-bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi, struct folio *folio,
- block_t blkaddr);
+bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi, struct folio *folio,
+ size_t offset, block_t blkaddr);
void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi, nid_t ino);
#define inc_compr_inode_stat(inode) \
do { \
@@ -5071,7 +5082,8 @@ static inline void f2fs_destroy_compress_mempool(void) { }
static inline void f2fs_decompress_cluster(struct decompress_io_ctx *dic,
bool in_task) { }
static inline void f2fs_end_read_compressed_page(struct folio *folio,
- bool failed, block_t blkaddr, bool in_task)
+ size_t offset, bool failed, block_t blkaddr,
+ unsigned int nr_blocks, bool in_task)
{
WARN_ON_ONCE(1);
}
@@ -5089,17 +5101,18 @@ static inline int __init f2fs_init_compress_cache(void) { return 0; }
static inline void f2fs_destroy_compress_cache(void) { }
static inline void f2fs_invalidate_compress_pages_range(struct f2fs_sb_info *sbi,
block_t blkaddr, unsigned int len) { }
-static inline bool f2fs_load_compressed_folio(struct f2fs_sb_info *sbi,
- struct folio *folio, block_t blkaddr) { return false; }
+static inline bool f2fs_load_compressed_block(struct f2fs_sb_info *sbi,
+ struct folio *folio, size_t offset,
+ block_t blkaddr) { return false; }
static inline void f2fs_invalidate_compress_pages(struct f2fs_sb_info *sbi,
nid_t ino) { }
#define inc_compr_inode_stat(inode) do { } while (0)
static inline int f2fs_is_compressed_cluster(
struct inode *inode,
- pgoff_t index) { return 0; }
+ pgoff_t lblk) { return 0; }
static inline bool f2fs_is_sparse_cluster(
struct inode *inode,
- pgoff_t index) { return true; }
+ pgoff_t lblk) { return true; }
static inline void f2fs_update_read_extent_tree_range_compressed(
struct inode *inode,
pgoff_t fofs, block_t blkaddr,
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 67ed6be8952f..952896640e72 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -178,7 +178,7 @@ static vm_fault_t f2fs_vm_page_mkwrite(struct vm_fault *vmf)
#ifdef CONFIG_F2FS_FS_COMPRESSION
if (f2fs_compressed_file(inode)) {
- int ret = f2fs_is_compressed_cluster(inode, pidx);
+ int ret = f2fs_is_compressed_cluster(inode, bidx);
if (ret < 0) {
err = ret;
@@ -1019,7 +1019,8 @@ int f2fs_truncate_blocks(struct inode *inode, u64 from, bool lock)
*/
if (f2fs_compressed_file(inode))
free_from = round_up(from,
- F2FS_I(inode)->i_cluster_size << PAGE_SHIFT);
+ F2FS_BLK_TO_BYTES(F2FS_I_SB(inode),
+ F2FS_I(inode)->i_cluster_size));
#endif
err = f2fs_do_truncate_blocks(inode, free_from, lock);
@@ -2298,13 +2299,6 @@ static long f2fs_fallocate(struct file *file, int mode,
goto out;
}
- if (f2fs_has_subpage_blocks(F2FS_I_SB(inode))) {
- if (f2fs_compressed_file(inode)) {
- ret = -EOPNOTSUPP;
- goto out;
- }
- }
-
ret = file_modified(file);
if (ret)
goto out;
@@ -4447,9 +4441,6 @@ static int f2fs_release_compress_blocks(struct file *filp, unsigned long arg)
int ret;
int writecount;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4645,9 +4636,6 @@ static int f2fs_reserve_compress_blocks(struct file *filp, unsigned long arg)
unsigned int reserved_blocks = 0;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4969,9 +4957,6 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
struct f2fs_comp_option option;
int ret = 0;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi))
return -EOPNOTSUPP;
@@ -4982,7 +4967,7 @@ static int f2fs_ioc_set_compress_option(struct file *filp, unsigned long arg)
sizeof(option)))
return -EFAULT;
- if (option.log_cluster_size < MIN_COMPRESS_LOG_SIZE ||
+ if (option.log_cluster_size < min_compress_log_size(sbi) ||
option.log_cluster_size > MAX_COMPRESS_LOG_SIZE ||
option.algorithm >= COMPRESS_MAX)
return -EINVAL;
@@ -5076,16 +5061,22 @@ static int redirty_blocks(struct inode *inode, pgoff_t page_idx, int len)
off = offset_in_folio(folio, (loff_t)redirty_idx << PAGE_SHIFT);
dlen = (size_t)(next_idx - redirty_idx) << PAGE_SHIFT;
- if (folio_test_large(folio)) {
+ if (f2fs_is_multiblock_folio(F2FS_I_SB(inode), folio)) {
+ loff_t eof = i_size_read(inode) - folio_pos(folio);
+
/*
- * A compressed inode can keep using large folios, in
+ * folio_mark_dirty() does not dirty the blocks of a folio
+ * state. A compressed inode can keep using large folios, in
* which case the compression flag only works as a hint
* until the inode is evicted. Update the subpage dirty
* bitmap as well, otherwise f2fs_write_cache_folios()
- * finds no dirty range and skips the whole folio.
+ * finds no dirty range and skips the whole folio. Do
+ * not dirty the blocks past EOF.
*/
- f2fs_ffs_find_or_alloc(folio);
- f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
+ if ((loff_t)(off + dlen) > eof)
+ dlen = eof > (loff_t)off ? eof - off : 0;
+ if (dlen)
+ f2fs_ffs_mark_subrange_dirty(folio, off, dlen);
}
folio_mark_dirty(folio);
@@ -5103,12 +5094,9 @@ static int f2fs_ioc_decompress_file(struct file *filp)
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode_info *fi = F2FS_I(inode);
- pgoff_t page_idx = 0, last_idx, cluster_idx;
+ pgoff_t lblk = 0, last_idx, cluster_idx;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5145,12 +5133,14 @@ static int f2fs_ioc_decompress_file(struct file *filp)
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
- page_idx = cluster_idx << fi->i_log_cluster_size;
+ lblk = cluster_idx << fi->i_log_cluster_size;
- if (!f2fs_is_compressed_cluster(inode, page_idx))
+ if (!f2fs_is_compressed_cluster(inode, lblk))
continue;
- ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
+ ret = redirty_blocks(inode,
+ f2fs_lblk_to_folio_index(sbi, lblk),
+ fi->i_cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi));
if (ret < 0)
break;
@@ -5187,12 +5177,9 @@ static int f2fs_ioc_compress_file(struct file *filp)
struct inode *inode = file_inode(filp);
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
struct f2fs_inode_info *fi = F2FS_I(inode);
- pgoff_t page_idx = 0, last_idx, cluster_idx;
+ pgoff_t lblk = 0, last_idx, cluster_idx;
int ret;
- if (f2fs_has_subpage_blocks(sbi))
- return -EOPNOTSUPP;
-
if (!f2fs_sb_has_compression(sbi) ||
F2FS_OPTION(sbi).compress_mode != COMPR_MODE_USER)
return -EOPNOTSUPP;
@@ -5235,12 +5222,14 @@ static int f2fs_ioc_compress_file(struct file *filp)
last_idx >>= fi->i_log_cluster_size;
for (cluster_idx = 0; cluster_idx < last_idx; cluster_idx++) {
- page_idx = cluster_idx << fi->i_log_cluster_size;
+ lblk = cluster_idx << fi->i_log_cluster_size;
- if (f2fs_is_sparse_cluster(inode, page_idx))
+ if (f2fs_is_sparse_cluster(inode, lblk))
continue;
- ret = redirty_blocks(inode, page_idx, fi->i_cluster_size);
+ ret = redirty_blocks(inode,
+ f2fs_lblk_to_folio_index(sbi, lblk),
+ fi->i_cluster_size >> F2FS_LOG_BLKS_PER_PAGE(sbi));
if (ret < 0)
break;
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index 5c6a64b31b05..aad5fbbcc5a2 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -225,7 +225,7 @@ static bool sanity_check_compress_inode(struct inode *inode,
SECTOR_TO_BLOCK(sbi, inode->i_blocks));
return false;
}
- if (ri->i_log_cluster_size < MIN_COMPRESS_LOG_SIZE ||
+ if (ri->i_log_cluster_size < min_compress_log_size(sbi) ||
ri->i_log_cluster_size > MAX_COMPRESS_LOG_SIZE) {
f2fs_warn(sbi,
"%s: inode (ino=%llx) has unsupported log cluster size: %u, run fsck to fix",
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index fa74564adcfb..b2669057b7d2 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1447,6 +1447,11 @@ static int f2fs_check_compression(struct fs_context *fc,
ctx_clear_opt_mask(ctx, F2FS_MOUNT_COMPRESS_CACHE);
return 0;
}
+ if ((ctx->spec_mask & F2FS_SPEC_compress_log_size) &&
+ F2FS_CTX_INFO(ctx).compress_log_size < min_compress_log_size(sbi)) {
+ f2fs_err(sbi, "Compress cluster log size is out of range");
+ return -EINVAL;
+ }
if (ctx->spec_mask & F2FS_SPEC_compress_extension) {
cnt = F2FS_CTX_INFO(ctx).compress_ext_cnt;
for (i = 0; i < F2FS_CTX_INFO(ctx).compress_ext_cnt; i++) {
@@ -2630,7 +2635,7 @@ static void default_options(struct f2fs_sb_info *sbi, bool remount)
F2FS_OPTION(sbi).s_resgid = make_kgid(&init_user_ns, F2FS_DEF_RESGID);
if (f2fs_sb_has_compression(sbi)) {
F2FS_OPTION(sbi).compress_algorithm = COMPRESS_LZ4;
- F2FS_OPTION(sbi).compress_log_size = MIN_COMPRESS_LOG_SIZE;
+ F2FS_OPTION(sbi).compress_log_size = min_compress_log_size(sbi);
F2FS_OPTION(sbi).compress_ext_cnt = 0;
F2FS_OPTION(sbi).compress_mode = COMPR_MODE_FS;
}
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 19/20] f2fs: count compress cache entries in pages for its memory limit
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (17 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 18/20] f2fs: support subpage blocks in compression Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
2026-11-02 19:39 ` [PATCH 20/20] f2fs: enable subpage block size support Daeho Jeong
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
The compress cache holds one filesystem block per entry, but
f2fs_available_free_memory() compares the number of entries with a
percentage of free memory counted in pages. With subpage blocks, this
limits the compress cache to a fraction (block size / PAGE_SIZE) of
the memory that compress_percent allows. Convert the entry count to
pages before the comparison.
No functional change when the block size equals PAGE_SIZE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/node.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c
index b95bb5c27b0b..66b28ae91756 100644
--- a/fs/f2fs/node.c
+++ b/fs/f2fs/node.c
@@ -116,10 +116,12 @@ bool f2fs_available_free_memory(struct f2fs_sb_info *sbi, int type)
/*
* free memory is lower than watermark or cached page count
- * exceed threshold, deny caching compress page.
+ * exceed threshold, deny caching compress page. A cache
+ * entry holds one block, so convert the entry count to pages.
*/
res = (free_ram > avail_ram * sbi->compress_watermark / 100) &&
- (COMPRESS_CACHE(sbi)->num_entries <
+ ((COMPRESS_CACHE(sbi)->num_entries >>
+ F2FS_LOG_BLKS_PER_PAGE(sbi)) <
free_ram * sbi->compress_percent / 100);
#else
res = false;
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 20/20] f2fs: enable subpage block size support
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
` (18 preceding siblings ...)
2026-11-02 19:39 ` [PATCH 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
@ 2026-11-02 19:39 ` Daeho Jeong
19 siblings, 0 replies; 22+ messages in thread
From: Daeho Jeong @ 2026-11-02 19:39 UTC (permalink / raw)
To: linux-kernel, linux-f2fs-devel, kernel-team; +Cc: Daeho Jeong
From: Daeho Jeong <daehojeong@google.com>
All paths now handle filesystem blocks smaller than PAGE_SIZE, so stop
rejecting a superblock whose log_blocksize is below PAGE_SHIFT.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
---
fs/f2fs/super.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index b2669057b7d2..4339e4ea7a12 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -4173,13 +4173,6 @@ static int sanity_check_raw_super(struct f2fs_sb_info *sbi,
return -EFSCORRUPTED;
}
- /* Subpage block size is enabled after all paths support it. */
- if (le32_to_cpu(raw_super->log_blocksize) != PAGE_SHIFT) {
- f2fs_info(sbi, "Subpage block size (%u) is not supported yet",
- le32_to_cpu(raw_super->log_blocksize));
- return -EOPNOTSUPP;
- }
-
/* check log blocks per segment */
if (le32_to_cpu(raw_super->log_blocks_per_seg) != 9) {
f2fs_info(sbi, "Invalid log blocks per segment (%u)",
--
2.56.0.317.g454f8302e1-goog
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [f2fs-dev] [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths
2026-11-02 19:39 ` [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths Daeho Jeong
@ 2026-11-03 7:04 ` Chao Yu
0 siblings, 0 replies; 22+ messages in thread
From: Chao Yu @ 2026-11-03 7:04 UTC (permalink / raw)
To: Daeho Jeong, linux-kernel, linux-f2fs-devel, kernel-team
Cc: chao, Daeho Jeong
On 11/3/26 03:39, Daeho Jeong wrote:
> static unsigned long dir_blocks(struct inode *inode)
> {
> - return ((unsigned long long) (i_size_read(inode) + PAGE_SIZE - 1))
> - >> PAGE_SHIFT;
> + return F2FS_BYTES_TO_BLK(F2FS_I_SB(inode),
> + i_size_read(inode) + F2FS_BLKSIZE_MASK(F2FS_I_SB(inode)));
Equals to F2FS_BLK_ALIGN(F2FS_I_SB(inode), i_size_read(inode))?
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2026-11-03 7:04 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-02 19:39 [PATCH 00/20] f2fs: support block size smaller than PAGE_SIZE Daeho Jeong
2026-11-02 19:39 ` [PATCH 01/20] f2fs: use block-based index and size calculations in remaining paths Daeho Jeong
2026-11-03 7:04 ` [f2fs-dev] " Chao Yu
2026-11-02 19:39 ` [PATCH 02/20] f2fs: support subpage block size superblock and mount Daeho Jeong
2026-11-02 19:39 ` [PATCH 03/20] f2fs: extend f2fs_folio_state for subpage blocks Daeho Jeong
2026-11-02 19:39 ` [PATCH 04/20] f2fs: support subpage blocks in directory operations Daeho Jeong
2026-11-02 19:39 ` [PATCH 05/20] f2fs: convert inode dirty page counters to dirty block counters Daeho Jeong
2026-11-02 19:39 ` [PATCH 06/20] f2fs: support subpage blocks in data read path and GC Daeho Jeong
2026-11-02 19:39 ` [PATCH 07/20] f2fs: support subpage blocks in data write path and writeback Daeho Jeong
2026-11-02 19:39 ` [PATCH 08/20] f2fs: adapt range operations and mmap for subpage blocks Daeho Jeong
2026-11-02 19:39 ` [PATCH 09/20] f2fs: support subpage blocks in pinned files Daeho Jeong
2026-11-02 19:39 ` [PATCH 10/20] f2fs: support subpage blocks in collapse and insert range Daeho Jeong
2026-11-02 19:39 ` [PATCH 11/20] f2fs: support subpage blocks in move range Daeho Jeong
2026-11-02 19:39 ` [PATCH 12/20] f2fs: support subpage blocks in defragmentation Daeho Jeong
2026-11-02 19:39 ` [PATCH 13/20] f2fs: support subpage blocks in swapfile activation Daeho Jeong
2026-11-02 19:39 ` [PATCH 14/20] f2fs: support subpage blocks in fs-verity enablement Daeho Jeong
2026-11-02 19:39 ` [PATCH 15/20] f2fs: support subpage blocks in atomic writes Daeho Jeong
2026-11-02 19:39 ` [PATCH 16/20] f2fs: preserve inline data in subpage block mode Daeho Jeong
2026-11-02 19:39 ` [PATCH 17/20] f2fs: support subpage blocks in online resize Daeho Jeong
2026-11-02 19:39 ` [PATCH 18/20] f2fs: support subpage blocks in compression Daeho Jeong
2026-11-02 19:39 ` [PATCH 19/20] f2fs: count compress cache entries in pages for its memory limit Daeho Jeong
2026-11-02 19:39 ` [PATCH 20/20] f2fs: enable subpage block size support Daeho Jeong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®