From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from m16.mail.126.com (m16.mail.126.com [220.197.31.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C46C332EBB for ; Tue, 15 Sep 2026 04:19:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.9 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789445999; cv=none; b=XGrURTsdEGzbQjNg6AGTQEjic2Lb2p9sqKIYLaf116rnuCcejka59ljls6lIqokWpNgQ1UeHz+V9peP8rp+fcVe2OQtkXAw45SeDLzMJzm5hXRrWC09fyr/ik9ScbMba9+UeJBt7QrFBGOrJd4sgfYSEwRnnoZmmmzDWVsM95Ws= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789445999; c=relaxed/simple; bh=CXMJf1VxpUJkNaOms1lHQUBt1ld2pxaY9LqsuqnoX1Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ieqyhE8y1Nm+SlqGoOJ+65qeUxjvGbHW2FOdpBuVTzz+/9mdTzp/b/cUMfN8DsIVtrTeLsDU9VdrlnZHTj1z+p6lvKGhaZbEKTPSq1HRoKVNT6Uls3tOOIwFQ0UjFGL8PEUpwAJJfez98TtvTPEq1Oa6YxD+2rSck7rYOpYP93g= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=xiaomi.com; spf=pass smtp.mailfrom=126.com; dkim=pass (1024-bit key) header.d=126.com header.i=@126.com header.b=buvhhUul; arc=none smtp.client-ip=220.197.31.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=xiaomi.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=126.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=126.com header.i=@126.com header.b="buvhhUul" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=8q Z+qiQJIRwRVOe+x3CNCs894Wm+YjNv+O3pSDGzKS0=; b=buvhhUul9dq2PHLMfD 6W6GWSHmmNzeDsN38qq7p/SE6bc7QJCeVq2bX6FQb+0R7YtUgNAIRxZPjjk4VQjR hMVjHHG0iRh6BihxqAsdgVyiBP5Zmpz00g5CmthecJL8YM9qw8ZPelIiYw4X2dT+ FyAHjV9Tb4E7uKhDpr4nXOJlA= Received: from nzzhao-ThinkCentre-M760t.mioffice.cn (unknown []) by gzsmtp3 (Coremail) with SMTP id PikvCgDXreo9x6hq3jBvHw--.36918S3; Tue, 15 Sep 2026 12:19:11 +0800 (CST) From: Nanzhe Zhao To: linux-f2fs-devel@lists.sourceforge.net, Jaegeuk Kim , Chao Yu Cc: Barry Song , Juan Yescas , Dev Jain , linux-kernel@vger.kernel.org, David Hildenbrand , Bo Zhang , Kalesh Singh , Nanzhe Zhao , Pengfei Li , Ryan Roberts , Nanzhe Zhao Subject: [PATCH v2 01/14] f2fs: extend folio state for large folio write path Date: Tue, 15 Sep 2026 12:18:56 +0800 Message-ID: <20260915041909.2903887-2-zhaonanzhe@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260915041909.2903887-1-zhaonanzhe@xiaomi.com> References: <20260915041909.2903887-1-zhaonanzhe@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CM-TRANSID:PikvCgDXreo9x6hq3jBvHw--.36918S3 X-Coremail-Antispam: 1Uf129KBjvJXoW3Ww4fKr1DGr1xWF17JryxZrb_yoWDJFWkpF WrGrn0yr48Jry7WF17JF1rZr1Skr92qa1UAFWxG34fA3Waqw1rWF1kJ34jqF43Jry8uFn3 AayYkr1rWF4YyF7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jjksgUUUUU= Sender: nzzhao@126.com X-CM-SenderInfo: xq22xtbr6rjloofrz/xtbBsADGlmqox0AXvwAA3N Large folio write path needs a subpage status bitmap and write pages pending counter, while keeping compatible with f2fs private flags. Move struct f2fs_folio_state to f2fs.h, add private_flags and subpage state bitmap, and change PAGE_PRIVATE functions to be compatible with f2fs_folio_state. Allocate f2fs_folio_state via kzalloc instead of kmem_cache, since the state size depends on the folio order. Note: Now if a path wants to use f2fs_folio_state, it must call `folio_has_ffs` instead of `folio_test_large`` to make check. Signed-off-by: Nanzhe Zhao --- fs/f2fs/data.c | 72 ++++++++++++++++++++++++++++++++------------------ fs/f2fs/f2fs.h | 70 +++++++++++++++++++++++++++++++++++------------- 2 files changed, 98 insertions(+), 44 deletions(-) diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index a3b02c067e89..b3303109a696 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -32,15 +32,9 @@ static struct kmem_cache *bio_post_read_ctx_cache; static struct kmem_cache *bio_entry_slab; -static struct kmem_cache *ffs_entry_slab; static mempool_t *bio_post_read_ctx_pool; static struct bio_set f2fs_bioset; -struct f2fs_folio_state { - spinlock_t state_lock; - unsigned int read_pages_pending; -}; - struct f2fs_bio { struct work_struct work; struct bio bio; @@ -125,6 +119,9 @@ struct bio_post_read_ctx { block_t fs_blkaddr; }; +static bool __ffs_mark_subrange_uptodate(struct folio *folio, + struct f2fs_folio_state *ffs, size_t offset, size_t len); + /* * Update and unlock a bio's pages, and free the bio. * @@ -139,6 +136,23 @@ struct bio_post_read_ctx { * called (i.e., I/O error or decryption error, but *not* verity error), and * release the bio's reference to the decompress_io_ctx of the page's cluster. */ +/* + * Update read_pages_pending. + */ +static inline void f2fs_update_read_folio_pending(struct folio *folio, int nr) +{ + struct f2fs_folio_state *ffs; + unsigned long flags; + + if (!f2fs_folio_has_ffs(folio)) + return; + + ffs = (struct f2fs_folio_state *)folio->private; + spin_lock_irqsave(&ffs->state_lock, flags); + ffs->read_pages_pending += nr; + spin_unlock_irqrestore(&ffs->state_lock, flags); +} + static void f2fs_finish_read_bio(struct bio *bio, bool in_task) { struct folio_iter fi; @@ -147,7 +161,7 @@ static void f2fs_finish_read_bio(struct bio *bio, bool in_task) bio_for_each_folio_all(fi, bio) { struct folio *folio = fi.folio; - unsigned nr_pages = fi.length >> PAGE_SHIFT; + unsigned int nr_pages = fi.length >> PAGE_SHIFT; bool finished = true; if (!folio_test_large(folio) && @@ -2487,17 +2501,31 @@ int f2fs_read_multi_pages(struct compress_ctx *cc, struct bio **bio_ret, } #endif -static struct f2fs_folio_state *ffs_find_or_alloc(struct folio *folio) +struct f2fs_folio_state *f2fs_ffs_find_or_alloc(struct folio *folio) { - struct f2fs_folio_state *ffs = folio->private; + struct f2fs_folio_state *ffs; + unsigned int nr_subpages = folio_nr_pages(folio); + unsigned long private = (unsigned long)folio->private; + + f2fs_bug_on(F2FS_F_SB(folio), !folio_test_large(folio)); + f2fs_bug_on(F2FS_F_SB(folio), + test_bit(PAGE_PRIVATE_NOT_POINTER, &private)); - if (ffs) - return ffs; + if (f2fs_folio_has_ffs(folio)) + return (struct f2fs_folio_state *)folio->private; - ffs = f2fs_kmem_cache_alloc(ffs_entry_slab, - GFP_NOIO | __GFP_ZERO, true, NULL); + ffs = f2fs_kmalloc(F2FS_F_SB(folio), + struct_size(ffs, state, BITS_TO_LONGS(2 * nr_subpages)), + GFP_NOFS | __GFP_ZERO); + if (!ffs) + return NULL; spin_lock_init(&ffs->state_lock); + if (folio_test_uptodate(folio)) + bitmap_set(ffs->state, 0, nr_subpages); + if (folio_test_dirty(folio)) + bitmap_set(ffs->state, nr_subpages, nr_subpages); + folio_attach_private(folio, ffs); return ffs; } @@ -2506,7 +2534,7 @@ static void ffs_detach_free(struct folio *folio) { struct f2fs_folio_state *ffs; - if (!folio_test_large(folio)) { + if (!f2fs_folio_has_ffs(folio)) { folio_detach_private(folio); return; } @@ -2516,7 +2544,8 @@ static void ffs_detach_free(struct folio *folio) return; WARN_ON_ONCE(ffs->read_pages_pending != 0); - kmem_cache_free(ffs_entry_slab, ffs); + WARN_ON_ONCE(atomic_read(&ffs->write_pages_pending)); + kfree(ffs); } static int f2fs_read_data_large_folio(struct inode *inode, @@ -2529,7 +2558,7 @@ static int f2fs_read_data_large_folio(struct inode *inode, pgoff_t index, offset, next_pgofs = 0; unsigned max_nr_pages = rac ? readahead_count(rac) : folio_nr_pages(folio); - unsigned nrpages; + unsigned int nrpages; struct f2fs_folio_state *ffs; int ret = 0; bool folio_in_bio = false; @@ -2605,7 +2634,7 @@ static int f2fs_read_data_large_folio(struct inode *inode, * to prevent from premature folio_end_read() call on folio */ if (folio_test_large(folio)) { - ffs = ffs_find_or_alloc(folio); + ffs = f2fs_ffs_find_or_alloc(folio); /* set the bitmap to wait */ spin_lock_irq(&ffs->state_lock); @@ -4522,21 +4551,12 @@ int __init f2fs_init_bio_entry_cache(void) if (!bio_entry_slab) return -ENOMEM; - ffs_entry_slab = f2fs_kmem_cache_create("f2fs_ffs_slab", - sizeof(struct f2fs_folio_state)); - - if (!ffs_entry_slab) { - kmem_cache_destroy(bio_entry_slab); - return -ENOMEM; - } - return 0; } void f2fs_destroy_bio_entry_cache(void) { kmem_cache_destroy(bio_entry_slab); - kmem_cache_destroy(ffs_entry_slab); } static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index ebc621f302e1..9a2366a6c113 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -1618,6 +1618,18 @@ static inline void f2fs_clear_bit(unsigned int nr, char *addr); * Layout B: lowest bit should be 0 * page.private is a wrapped pointer. */ + +struct f2fs_folio_state { + spinlock_t state_lock; + unsigned int read_pages_pending; + atomic_t write_pages_pending; + unsigned long private_flags; + /* state[0..nr_subpages - 1] tracks uptodate subpages. + * state[nr_subpages..2 * nr_subpages - 1] tracks dirty subpages. + */ + unsigned long state[]; +}; + enum { PAGE_PRIVATE_NOT_POINTER, /* private contains non-pointer data */ PAGE_PRIVATE_ONGOING_MIGRATION, /* data page which is on-going migrating */ @@ -1627,6 +1639,13 @@ enum { PAGE_PRIVATE_MAX }; +static inline bool f2fs_folio_has_ffs(const struct folio *folio) +{ + unsigned long private = (unsigned long)folio->private; + + return folio_test_large(folio) && private; +} + /* For compression */ enum compress_algorithm_type { COMPRESS_LZO, @@ -2738,13 +2757,35 @@ static inline int inc_valid_block_count(struct f2fs_sb_info *sbi, return -ENOSPC; } +static inline unsigned long f2fs_folio_get_private_flags(const struct folio *folio) +{ + unsigned long private = (unsigned long)folio->private; + + if (f2fs_folio_has_ffs(folio)) { + struct f2fs_folio_state *ffs = (struct f2fs_folio_state *)private; + + return READ_ONCE(ffs->private_flags); + } + + if (test_bit(PAGE_PRIVATE_NOT_POINTER, &private)) + return private; + + return 0; +} + +static inline unsigned long *f2fs_folio_flags_addr(struct folio *folio) +{ + if (f2fs_folio_has_ffs(folio)) + return &((struct f2fs_folio_state *)folio->private)->private_flags; + + return (unsigned long *)&folio->private; +} + #define PAGE_PRIVATE_GET_FUNC(name, flagname) \ static inline bool folio_test_f2fs_##name(const struct folio *folio) \ { \ - unsigned long priv = (unsigned long)folio->private; \ - unsigned long v = (1UL << PAGE_PRIVATE_NOT_POINTER) | \ - (1UL << PAGE_PRIVATE_##flagname); \ - return (priv & v) == v; \ + return f2fs_folio_get_private_flags(folio) & \ + (1UL << PAGE_PRIVATE_##flagname); \ } \ static inline bool page_private_##name(struct page *page) \ { \ @@ -2756,14 +2797,10 @@ static inline bool page_private_##name(struct page *page) \ #define PAGE_PRIVATE_SET_FUNC(name, flagname) \ static inline void folio_set_f2fs_##name(struct folio *folio) \ { \ - unsigned long v = (1UL << PAGE_PRIVATE_NOT_POINTER) | \ - (1UL << PAGE_PRIVATE_##flagname); \ if (!folio->private) \ - folio_attach_private(folio, (void *)v); \ - else { \ - v |= (unsigned long)folio->private; \ - folio->private = (void *)v; \ - } \ + folio_attach_private(folio, \ + (void *)BIT(PAGE_PRIVATE_NOT_POINTER)); \ + set_bit(PAGE_PRIVATE_##flagname, f2fs_folio_flags_addr(folio)); \ } \ static inline void set_page_private_##name(struct page *page) \ { \ @@ -2776,13 +2813,10 @@ static inline void set_page_private_##name(struct page *page) \ #define PAGE_PRIVATE_CLEAR_FUNC(name, flagname) \ static inline void folio_clear_f2fs_##name(struct folio *folio) \ { \ - unsigned long v = (unsigned long)folio->private; \ - \ - v &= ~(1UL << PAGE_PRIVATE_##flagname); \ - if (v == (1UL << PAGE_PRIVATE_NOT_POINTER)) \ + clear_bit(PAGE_PRIVATE_##flagname, \ + f2fs_folio_flags_addr(folio)); \ + if (folio->private == (void *)BIT(PAGE_PRIVATE_NOT_POINTER)) \ folio_detach_private(folio); \ - else \ - folio->private = (void *)v; \ } \ static inline void clear_page_private_##name(struct page *page) \ { \ @@ -2808,7 +2842,7 @@ PAGE_PRIVATE_CLEAR_FUNC(atomic, ATOMIC_WRITE); static inline unsigned long folio_get_f2fs_data(struct folio *folio) { - unsigned long data = (unsigned long)folio->private; + unsigned long data = f2fs_folio_get_private_flags(folio); if (!test_bit(PAGE_PRIVATE_NOT_POINTER, &data)) return 0; -- 2.43.0