From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj2-f4.google.com (mail-pj2-f4.google.com [74.125.227.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA18A4F68CB for ; Tue, 13 Oct 2026 11:57:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.132 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791892630; cv=none; b=QSrze8bQoRAoblcqml/kr7CJ7TZXS/+pTVKUWHPLumWihgHpjw6MxcYfQObIAnQg7zCEUwy85hhpqU/I5B4sDG1Lyk/2yJ5A3gpfRN86p/PXRvQuxV8Z+0MlVBN7UE4XvPwj3Q1jRi2WlikkzcgwUw21Znx0TlQ4RPTrXtKbymU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791892630; c=relaxed/simple; bh=smCHbqcPE8RRAViUyyEXcET2C3Wibr5ngykpjoIcoCc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NoFA5WRnFo9LikL6gUWwDyEsqe1X/JT+zMTw7bYnf4eUg2eipWp8MAOwBoQS04Zi+EekH3jq1sBmt4Bleqs851j/BTQtoGdYLgzUVAIWei0X31X1hy6OmCjeXYj90hcMO5PN7tG0u1FHjGXHDPEaHURUthNkMhx6KJhseknz+xw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=olgNb+po; arc=none smtp.client-ip=74.125.227.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="olgNb+po" Received: by mail-pj2-f4.google.com with SMTP id 98e67ed59e1d1-3a84a12aa97so684051a91.1 for ; Tue, 13 Oct 2026 04:57:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1791892628; x=1792497428; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=G8GZTN6OzJEU0CV9Qcex4KLuipXxqv2mdjpVbUHXdHA=; b=olgNb+po9adOaB6PJdGVf7YPQoKj30SR3qjlgKSZN+TqZJxQwj0u8zjMaewt6B85bM K3kw9hY3oMj3YI+Jk7r8URPG78aVhZlmoBf290vegLb2XL/e3Zs15IFerBMAM6OCClc/ c939MD0BDAmewSeFPkxWjrd2SL1AUq9QK6SUpHVbxyps5aoRyM3NQ4XvlBQsAwa8yhPv Aa3GQQ3GWMAgsH1a7cGou9NexT1L35X046EyjVgt41x1LYpnqejDOAfdpuWYZE2oUfkA cMcQHuHiYj3xFNBeLD46GYj6TWQ67L/4bChOOaROxnP+EmrlWhvj6MOhJ/H9SZo8e/ja mXNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1791892628; x=1792497428; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=G8GZTN6OzJEU0CV9Qcex4KLuipXxqv2mdjpVbUHXdHA=; b=sCbdJMryzsooGSMi1rXLu6rr+XJ44aZNWJlr4Glze8C2HEf7MS2C6seGVbw0hYtLfh z1I8AsFsjeb+jSj/dpQGpSyVzEP34ROdzOLTnFebkaemH6RslqB7c8dagZgoPsb3X4y3 aIL/A/4fBwz8M0GnhUsfe4oUsU4yXWgDjp83WdqKWSOeZ2CrVHPKjo/0hR0fu0wg0LbO J3kYr9iics/nyMhcFGP/17GKl1wmvXFlI78UoPdyRzvcwldKFA+hWIpXGKvL755Ra0F2 IXHDlHeWUMZjXuIMlA4StZ1HKySY//Hyx2ch1SIoIl5eFTlTmaoAC3HvyNqPb/aa0UN3 16PA== X-Forwarded-Encrypted: i=1; AKwUvBwbLm7P3HOxjzzKc/DZd2MLEAmDNSULXV6Z33IjGCcNWCh4rRxhrNtz/O3jfC1KKh3Nc91oeQiFGGNUaeE=@vger.kernel.org X-Gm-Message-State: AFq9FYJ8KeukhRuJJvMrdFGUWZKffCV7YQVyCQHPbNoD6E5z9p7ZrN/h DSPL72QrhyKedTY29oYHxVe8SwkLhYaXY7xJ5/d6FSY5JTvQ3ojGAY0O X-Gm-Gg: AYBFou1r+XEL8JVzSzAqTQ8GEXZ5dk95TmIisDkbsRfUwQS1JNptawzalebBUjUq8rs nFNvu7UNJGgirn+WywWo7WacMBN8W5b7vmgvYGfzCAABp3XyGUZIk8JCVu19goNEeISVywsllaC DGM6jSrRT5zddEDQVHpQJhS6UTcok2VuVwXAj2PhtEXB4TD79awUh/BuxD5/iefYaD4IPrW1SfF r0dN2q5K/VyoBpNdVBGZTPByNoCddSus5MgHmO+49ZVVC3kKCCuy/vcpOCdyjsOttnH8oAaHrQP eoLGA2ubsws6YtUDtlopX3RLLYvPB64X4wLFjpsL6YyxxNOsn1Knl8/nktGwCJwxLZTaln558KS sIeGfZanlQUY9q86P4RmPlaevK2YeQjB7DQkYbecbD5kiF0SDdjHWKaJpMlvB4L8K3c66CpjF3m bUAUcG1jVgmuHo84vbN0fnFqjPl+cJs4h7ySnBwWxLvwlA1Is/rl+0vW/AG88QUVhgedNW9SwSi UFv/3SMN470Fkc7RxpWz7fUcibJgIQAQAkk X-Received: by 2002:a17:90b:1e4a:b0:3aa:9a32:5051 with SMTP id 98e67ed59e1d1-3ab3ab329bcmr10574577a91.45.1791892627740; Tue, 13 Oct 2026 04:57:07 -0700 (PDT) Received: from nzzhao-ThinkCentre-M760t.mioffice.cn ([43.224.245.179]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3ab37139b59sm21248516a91.13.2026.10.13.04.57.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Oct 2026 04:57:07 -0700 (PDT) From: Nanzhe Zhao To: linux-f2fs-devel@lists.sourceforge.net, Jaegeuk Kim , Chao Yu Cc: Barry Song , Nanzhe Zhao , Juan Yescas , Daeho Jeong , Dev Jain , David Hildenbrand , linux-kernel@vger.kernel.org, Bo Zhang , Kalesh Singh , Nanzhe Zhao , Pengfei Li , Ryan Roberts Subject: [PATCH v4 08/18] f2fs: account large folio dirty data at block granularity Date: Tue, 13 Oct 2026 19:56:11 +0800 Message-ID: <20261013115621.968742-9-nzzhao.sigma@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20261013115621.968742-1-nzzhao.sigma@gmail.com> References: <20261013115621.968742-1-nzzhao.sigma@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Nanzhe Zhao Originally inode->dirty_pages was accounted once per folio, but f2fs needs the counter to be accounted in blocks. This introduces batch dirty page add/sub accounting; for a large folio the changed dirty page count is computed from the dirty bitmap bit changes. Found by f2fs/006 (lfs + checkpoint=disable) and generic/363. Signed-off-by: Nanzhe Zhao --- fs/f2fs/checkpoint.c | 38 ++++++++++++++++++++++---- fs/f2fs/compress.c | 4 +-- fs/f2fs/data.c | 37 +++++++++++++++++++------ fs/f2fs/dir.c | 2 +- fs/f2fs/f2fs.h | 65 ++++++++++++++++++++++++++++++++++---------- fs/f2fs/gc.c | 2 +- fs/f2fs/inline.c | 2 +- fs/f2fs/node.c | 2 +- 8 files changed, 117 insertions(+), 35 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index 81c1c2de9..27ea41d7f 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -1316,20 +1316,46 @@ static void __remove_dirty_inode(struct inode *inode, enum inode_type type) stat_dec_dirty_inode(F2FS_I_SB(inode), type); } -void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio) +/* + * Account @nr dirty pages (negative when dirtiness is cleared) for @inode. + */ +void f2fs_account_dirty_blocks(struct inode *inode, int nr) { struct f2fs_sb_info *sbi = F2FS_I_SB(inode); enum inode_type type = S_ISDIR(inode->i_mode) ? DIR_INODE : FILE_INODE; - if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) && - !S_ISLNK(inode->i_mode)) + if (!nr) return; spin_lock(&sbi->inode_lock[type]); - if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH)) - __add_dirty_inode(inode, type); - inode_inc_dirty_pages(inode); + if (nr > 0) { + if (type != FILE_INODE || test_opt(sbi, DATA_FLUSH)) + __add_dirty_inode(inode, type); + inode_add_dirty_blocks(inode, nr); + } else { + inode_sub_dirty_blocks(inode, -nr); + __remove_dirty_inode(inode, type); + } spin_unlock(&sbi->inode_lock[type]); +} + +void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio) +{ + if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) && + !S_ISLNK(inode->i_mode)) + return; + + /* + * Large folios have a subpage dirty bitmap, so defer their dirty accounting + * to the f2fs_ffs_*subrange_dirty*() helpers. This avoids nesting + * ffs->state_lock with the inode dirty list lock. + */ + if (f2fs_folio_has_ffs(folio)) { + folio_set_f2fs_reference(folio); + return; + } + + f2fs_account_dirty_blocks(inode, 1); folio_set_f2fs_reference(folio); } diff --git a/fs/f2fs/compress.c b/fs/f2fs/compress.c index 0719a0e1e..3d26bdd4d 100644 --- a/fs/f2fs/compress.c +++ b/fs/f2fs/compress.c @@ -1067,7 +1067,7 @@ static void cancel_cluster_writeback(struct compress_ctx *cc, struct folio *folio = page_folio(cc->rpages[i]); if (i < submitted) { - inode_inc_dirty_pages(cc->inode); + inode_inc_dirty_block(cc->inode); folio_lock(folio); } folio_clear_f2fs_gcing(folio); @@ -1415,7 +1415,7 @@ static int f2fs_write_compressed_pages(struct compress_ctx *cc, } (*submitted)++; unlock_continue: - inode_dec_dirty_pages(cc->inode); + inode_dec_dirty_block(cc->inode); folio_unlock(fio.folio); } diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index be182b9e8..f994a31f8 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -2897,10 +2897,18 @@ static unsigned int __ffs_mark_subrange_dirty(struct folio *folio, return bitmap_weight(ffs->state, 2 * nr_subpages) - before; } +/* + * Mark the given range dirty for writeback. + * + * Callers must guarantee that the corresponding subpages are already + * uptodate: a dirty subpage always holds the latest data, whose + * on-disk copy may be stale. + */ void f2fs_ffs_mark_subrange_dirty(struct folio *folio, size_t offset, size_t len) { struct f2fs_folio_state *ffs; + unsigned int set; unsigned long flags; f2fs_bug_on(F2FS_F_SB(folio), offset + len > folio_size(folio)); @@ -2911,21 +2919,27 @@ void f2fs_ffs_mark_subrange_dirty(struct folio *folio, ffs = (struct f2fs_folio_state *)folio->private; spin_lock_irqsave(&ffs->state_lock, flags); - __ffs_mark_subrange_dirty(folio, ffs, offset, len); + set = __ffs_mark_subrange_dirty(folio, ffs, offset, len); spin_unlock_irqrestore(&ffs->state_lock, flags); + + if (set) + f2fs_account_dirty_blocks(folio->mapping->host, set); } static bool __ffs_clear_subrange_dirty(struct folio *folio, - struct f2fs_folio_state *ffs, size_t offset, size_t len) + struct f2fs_folio_state *ffs, size_t offset, size_t len, + unsigned int *cleared) { unsigned int nr_subpages = folio_nr_pages(folio); - unsigned int start, end; + unsigned int start, end, before; start = offset >> PAGE_SHIFT; end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT; end = min(end, nr_subpages); + before = bitmap_weight(ffs->state, 2 * nr_subpages); bitmap_clear(ffs->state, nr_subpages + start, end - start); + *cleared = before - bitmap_weight(ffs->state, 2 * nr_subpages); return find_next_bit(ffs->state, 2 * nr_subpages, nr_subpages) < 2 * nr_subpages; } @@ -2937,13 +2951,15 @@ static bool __ffs_clear_subrange_dirty(struct folio *folio, * @len: byte length of the subrange * * Clear the dirty bits of the 4K subpages covered by [offset, offset + len), - * and return whether the folio still has any dirty subpage left. + * account the cleared subpages against the inode dirty counters, and return + * whether the folio still has any dirty subpage left. */ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset, size_t len) { struct f2fs_folio_state *ffs; unsigned long flags; + unsigned int cleared = 0; bool dirty; if (!f2fs_folio_has_ffs(folio)) @@ -2951,9 +2967,12 @@ bool f2fs_ffs_clear_subrange_dirty_and_test(struct folio *folio, size_t offset, ffs = (struct f2fs_folio_state *)folio->private; spin_lock_irqsave(&ffs->state_lock, flags); - dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len); + dirty = __ffs_clear_subrange_dirty(folio, ffs, offset, len, &cleared); spin_unlock_irqrestore(&ffs->state_lock, flags); + if (cleared) + f2fs_account_dirty_blocks(folio->mapping->host, -cleared); + return dirty; } @@ -3854,7 +3873,7 @@ int f2fs_write_single_data_page(struct folio *folio, int *submitted, goto redirty_out; out: - inode_dec_dirty_pages(inode); + inode_dec_dirty_block(inode); if (err) { folio_clear_uptodate(folio); folio_clear_f2fs_gcing(folio); @@ -4330,8 +4349,8 @@ static int f2fs_write_cache_folios(struct address_space *mapping, if (f2fs_ffs_clear_subrange_dirty_and_test(folio, 0, (err ? pos : end_pos) - folio_pos(folio))) folio_redirty_for_writepage(wbc, folio); - else - inode_dec_dirty_pages(inode); + else if (!f2fs_folio_has_ffs(folio)) + inode_dec_dirty_block(inode); /* * Release the bias taken above and we end writeback here if @@ -5160,7 +5179,7 @@ void f2fs_invalidate_folio(struct folio *folio, size_t offset, size_t length) return; if (folio_test_dirty(folio)) { - inode_dec_dirty_pages(inode); + inode_dec_dirty_block(inode); f2fs_remove_dirty_inode(inode); } diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c index 97f7fef6b..70847313c 100644 --- a/fs/f2fs/dir.c +++ b/fs/f2fs/dir.c @@ -937,7 +937,7 @@ void f2fs_delete_entry(struct f2fs_dir_entry *dentry, void *dentry_block, folio_clear_uptodate(folio); folio_detach_private(folio); - inode_dec_dirty_pages(dir); + inode_dec_dirty_block(dir); f2fs_remove_dirty_inode(dir); } f2fs_folio_put(folio, true); diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index f1a37e4fd..e5c1d6a65 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -2895,9 +2895,10 @@ static inline void dec_valid_block_count(struct f2fs_sb_info *sbi, f2fs_i_blocks_write(inode, count, false, true); } -static inline void inc_cache_count(struct f2fs_sb_info *sbi, int count_type) +static inline void add_cache_counts(struct f2fs_sb_info *sbi, + int count_type, int nr) { - atomic_inc(&sbi->nr_caches[count_type]); + atomic_add(nr, &sbi->nr_caches[count_type]); if (count_type == F2FS_DIRTY_DENTS || count_type == F2FS_DIRTY_NODES || @@ -2907,31 +2908,66 @@ static inline void inc_cache_count(struct f2fs_sb_info *sbi, int count_type) set_sbi_flag(sbi, SBI_IS_DIRTY); } -static inline void inode_inc_dirty_pages(struct inode *inode) +static inline void sub_cache_counts(struct f2fs_sb_info *sbi, + int count_type, int nr) { - atomic_inc(&F2FS_I(inode)->dirty_pages); - inc_cache_count(F2FS_I_SB(inode), S_ISDIR(inode->i_mode) ? - F2FS_DIRTY_DENTS : F2FS_DIRTY_DATA); - if (IS_NOQUOTA(inode)) - inc_cache_count(F2FS_I_SB(inode), F2FS_DIRTY_QDATA); + atomic_sub(nr, &sbi->nr_caches[count_type]); +} + +static inline void inc_cache_count(struct f2fs_sb_info *sbi, int count_type) +{ + add_cache_counts(sbi, count_type, 1); } static inline void dec_cache_count(struct f2fs_sb_info *sbi, int count_type) { - atomic_dec(&sbi->nr_caches[count_type]); + sub_cache_counts(sbi, count_type, 1); +} + +/* + * Account @nr dirty pages for @inode; one page is + * (PAGE_SIZE >> log_blocksize) filesystem blocks. + */ +static inline void inode_add_dirty_blocks(struct inode *inode, int nr) +{ + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize); + + atomic_add(blocks, &F2FS_I(inode)->dirty_pages); + if (S_ISDIR(inode->i_mode)) + add_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks); + else + add_cache_counts(sbi, F2FS_DIRTY_DATA, blocks); + if (IS_NOQUOTA(inode)) + add_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks); } -static inline void inode_dec_dirty_pages(struct inode *inode) +static inline void inode_sub_dirty_blocks(struct inode *inode, int nr) { + struct f2fs_sb_info *sbi = F2FS_I_SB(inode); + int blocks = nr * (PAGE_SIZE >> sbi->log_blocksize); + if (!S_ISDIR(inode->i_mode) && !S_ISREG(inode->i_mode) && !S_ISLNK(inode->i_mode)) return; - atomic_dec(&F2FS_I(inode)->dirty_pages); - dec_cache_count(F2FS_I_SB(inode), S_ISDIR(inode->i_mode) ? - F2FS_DIRTY_DENTS : F2FS_DIRTY_DATA); + atomic_sub(blocks, &F2FS_I(inode)->dirty_pages); + if (S_ISDIR(inode->i_mode)) + sub_cache_counts(sbi, F2FS_DIRTY_DENTS, blocks); + else + sub_cache_counts(sbi, F2FS_DIRTY_DATA, blocks); if (IS_NOQUOTA(inode)) - dec_cache_count(F2FS_I_SB(inode), F2FS_DIRTY_QDATA); + sub_cache_counts(sbi, F2FS_DIRTY_QDATA, blocks); +} + +static inline void inode_inc_dirty_block(struct inode *inode) +{ + inode_add_dirty_blocks(inode, 1); +} + +static inline void inode_dec_dirty_block(struct inode *inode) +{ + inode_sub_dirty_blocks(inode, 1); } static inline void inc_atomic_write_cnt(struct inode *inode) @@ -4359,6 +4395,7 @@ void f2fs_remove_orphan_inode(struct f2fs_sb_info *sbi, nid_t ino); int f2fs_recover_orphan_inodes(struct f2fs_sb_info *sbi); int f2fs_get_valid_checkpoint(struct f2fs_sb_info *sbi); void f2fs_update_dirty_folio(struct inode *inode, struct folio *folio); +void f2fs_account_dirty_blocks(struct inode *inode, int nr); void f2fs_remove_dirty_inode(struct inode *inode); int f2fs_sync_dirty_inodes(struct f2fs_sb_info *sbi, enum inode_type type, bool from_cp); diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index 89e50f871..d7fae1890 100644 --- a/fs/f2fs/gc.c +++ b/fs/f2fs/gc.c @@ -1555,7 +1555,7 @@ static int move_data_page(struct inode *inode, block_t bidx, int gc_type, folio_mark_dirty(folio); if (folio_clear_dirty_for_io(folio)) { - inode_dec_dirty_pages(inode); + inode_dec_dirty_block(inode); f2fs_remove_dirty_inode(inode); } diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c index ab7537ff7..49905aa08 100644 --- a/fs/f2fs/inline.c +++ b/fs/f2fs/inline.c @@ -199,7 +199,7 @@ int f2fs_convert_inline_folio(struct dnode_of_data *dn, struct folio *folio) f2fs_outplace_write_data(dn, &fio); f2fs_folio_wait_writeback(folio, true, true); if (dirty) { - inode_dec_dirty_pages(dn->inode); + inode_dec_dirty_block(dn->inode); f2fs_remove_dirty_inode(dn->inode); } diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c index be14dfa31..277c12922 100644 --- a/fs/f2fs/node.c +++ b/fs/f2fs/node.c @@ -1677,7 +1677,7 @@ static void flush_inline_data(struct f2fs_sb_info *sbi, nid_t ino) goto folio_out; ret = f2fs_write_inline_data(inode, folio); - inode_dec_dirty_pages(inode); + inode_dec_dirty_block(inode); f2fs_remove_dirty_inode(inode); if (ret) folio_mark_dirty(folio); -- 2.43.0