mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v7 0/7] ovl: only sync what was written through the overlay
@ 2026-11-18 17:03 Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 1/7] ovl: expand fsync mount options Josef Bacik
                   ` (7 more replies)
  0 siblings, 8 replies; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik, Chengguang Xu

v1: https://lore.kernel.org/all/20261027-b4-ovl-containerized-sync-v1-0-802e512379a6@toxicpanda.com/
v2: https://lore.kernel.org/all/20261029-b4-ovl-containerized-sync-v2-0-4e9f08e4a052@toxicpanda.com/
v3: https://lore.kernel.org/all/20261104-b4-ovl-containerized-sync-v3-0-8daa20144f53@toxicpanda.com/
v4: https://lore.kernel.org/all/20261106-b4-ovl-containerized-sync-v4-0-6ddde0b7dd8a@toxicpanda.com/
v5: https://lore.kernel.org/all/20261107-b4-ovl-containerized-sync-v5-0-d34b4ae57fdf@toxicpanda.com/
v6: https://lore.kernel.org/all/20261110-b4-ovl-containerized-sync-v6-0-43d1be16a4d6@toxicpanda.com/

I've completely redesigned this, because there were so many problems
that I kept tripping over trying to maintain state between the upper and
the lower inode. I'm not in love with this design either, but it's the
simplest. I'm hoping somebody spots something obvious and we can do it a
different way.

v6->v7:
- Redone without a flusher. An overlay inode has nothing of its own.
  What is dirty, under writeback or deferred by lazytime is the state of
  the upper inode, and nothing tells overlayfs when that changes. Up to
  v6 the dirty bit of the overlay inode mirrored that state on a bdi of
  the overlay, and most of what the reviews found was the mirror being
  off, or the flusher acting on it. The worst one: with 3000 files open
  for writing and the system over the background threshold the flusher
  of the overlay took 79% of a CPU going over its list. (Sashiko)
- Now the overlay only keeps a set of the inodes that were modified, an
  xarray, patch 5. Only syncfs(), unmount and a read-only remount walk
  it, and they ask the upper inode of each. Periodic and background
  writeback is all up to the upper file system, like without the series.
- The xarray has the marks of the page cache and a sync uses them the
  same way. ->sync_fs() without wait tags what is dirty with towrite,
  starts writeback of those and marks them writeback, with wait it waits
  for what is marked writeback. The one difference is that there is no
  completion to clear the writeback mark. Whoever has waited does, and
  only if the upper inode is clean, another sync may have started it
  again.
- The set holds a reference. An inode leaves it when it's unlinked, or
  when a sync or the shrinker sees the upper inode clean. The shrinker
  part hangs off the ->nr_cached_objects() and ->free_cached_objects()
  hooks. Nobody who walks the set needs a lock against anybody else.
- Gone: the bdi, ->write_inode(), ->drop_inode(), the lazytime patch,
  "ovl: report failed upper writeback in syncfs()", and the special
  cases for the root and for unhashed inodes. 7 patches, v6 had 8.
- syncfs() waits for the writeback of its own inodes, whoever started
  it, and for nobody else's.
- New patch 4: write_inode_for_sync() in fs/fs-writeback.c writes one
  inode the way the flusher does for sync_inodes_sb(), WB_SYNC_ALL with
  for_sync. Up to v6 I pieced that together from sync_inode_metadata()
  without and with wait, so that ext4 wouldn't commit for every inode.
  (Sashiko)
- Deferred timestamps are handed to the upper file system in a sync, or
  when memory is needed, so lazytime keeps working in between.
- syncfs() also reports writeback that the upper file system did earlier
  and that failed. The overlay inode remembers where wb_err of the upper
  mapping was when it entered the set, and a sync compares. An fsync()
  in between doesn't take that away.
- A freeze and a read-only remount of the upper file system are held off
  while an inode is handed over. If one has already begun syncfs() syncs
  the whole upper file system like before. That closes what v6 only
  described in the changelog of patch 7. (Sashiko)
- New patch 2: an upper file system with SB_I_NO_DATA_INTEGRITY, which
  is fuse, doesn't get relaxed. A sync doesn't wait for the fuse server,
  writing the upper inodes one by one does. (Sashiko)
- The helpers that set the impure, redirect, opaque and nlink xattrs
  track the inode themselves, and so does ovl_clear_empty(). A link or a
  rename that failed after them left the inode untracked. A setattr that
  fails tracks it too. (Sashiko)
- Rebased onto 7.4-rc1. v6 was still on the base of the earlier
  versions, and patch 4 conflicted with 4f61ff2ebfb3 ("ovl: use the
  pinned upper dir in ovl_fix_origin()").

v5->v6:
- Lookup marks a merge directory and its parent when it sets the origin
  and impure xattrs on them. (Sashiko)
- New patch 8: if somebody else writes the pages of an upper inode and
  that fails, syncfs() on the overlay reports it. It's best effort, the
  limits are in the changelog, and it's on its own so it can be dropped.
  (Sashiko)
- Left alone: if the deferred timestamps can't be handed over because
  the upper file system is being frozen, the sync of the freeze can miss
  them. They are written after the thaw, it's in the changelog of patch
  7. (Sashiko)

v4->v5:
- Added sync_fs_and_bdev() to fs/sync.c for the ->sync_fs() and block
  device part of sync_filesystem(), so ovl_sync_fs() doesn't open code
  it, patch 2. (Christian)
- ovl_instantiate() marks a new inode and a new hard link dirty.
  (Christian, Sashiko)
- Copy up marks the parent as well, and the inode also when it only adds
  a link to an indexed upper inode. (Sashiko)
- Dropped "ovl: make the root inode appear hashed". A fake hash stays in
  the slab object when the inode is freed, and the next overlay inode in
  it looks hashed. The root isn't tracked at all now, ovl_sync_fs()
  writes its upper inode every time. (Sashiko)
- lazytime got a lot simpler. The overlay inode doesn't get I_DIRTY_TIME
  any more, deferred timestamps are handed to the upper file system when
  ->write_inode() finds them. That takes care of the I/O from dentry
  reclaim, so this doesn't need anything from Jan's deferred inode
  reclaim series. (Sashiko, Christian)
- ovl_sync_upper_inode() could lose an error.
- Left alone: fsync=auto still goes by RENAME_WHITEOUT, no flag. What
  that leaves uncovered is under "Things I know about". (Christian)
- Left alone: if somebody else starts the writeback of an upper inode
  and it fails, syncfs() on the overlay doesn't report it. It never has.
  (Sashiko)

v3->v4:
- Patch 1 is v2 of Amir's patch. It has ovl_tracks_dirty() and picks the
  mode for tmpfs at mount time, so I dropped my version from patch 2.
  fsync=auto stays auto if it doesn't become relaxed. (Amir)

v2->v3:
- Patch 1 is Amir's, it adds fsync=relaxed. Only an overlay with that
  syncs just what was written through it, everybody else syncs the whole
  upper file system as before. fsync=auto, the default, picks relaxed if
  the upper file system supports RENAME_WHITEOUT. (Amir)
- Dropped the fake inodes and the calls in the ovl_do_*() helpers,
  patches 7 to 9 of v2. Upper inodes without an overlay inode are left
  to ->sync_fs() and the block device, like in v1. (Amir)

v1->v2:
- Dropped the superblock flag and the ext4, xfs and btrfs patches. This
  is all in overlayfs now and works the same on any upper file system.
  (Christoph, Amir)
- Upper inodes without an overlay inode get a fake one that goes on the
  dirty list, patches 7 to 9.
- The periodic flusher pass doesn't write the upper inode any more, and
  syncfs() doesn't make ext4 commit per inode.
- Record write errors in s_wb_err of the overlay.
- Make the root inode appear hashed so it can be on the dirty list,
  don't mark unhashed inodes dirty, read i_writecount with acquire.
  (Sashiko)
- Left alone: dirtied_time_when isn't set for an overlay inode that gets
  I_DIRTY_TIME from ->write_inode(). It only matters for the 12 hour
  expiry, where the upper file system writes the timestamps on its own.
  (Sashiko)
- Rewrote my commit messages. (Christoph)

What it does now:

sync_filesystem() writes every inode on a dirty list, calls ->sync_fs()
and writes the block device. Only the first step is expensive, because
of other people's data. With fsync=relaxed overlayfs does that step only
for the upper inodes of the overlay inodes in its set, ->sync_fs()
without wait starts them and with wait waits for them, and does the
other two steps as they are.

Whiteouts and the work directories have no overlay inode, so nobody
writes those inodes. That's fine if ->sync_fs() or the block device gets
all metadata to disk, which is what relaxed asks of the upper file
system.

Things I know about:

 - ext4 without a journal supports RENAME_WHITEOUT, so it gets relaxed.
   It's fine there, its metadata is in the buffers of the block device,
   which we write. I tested it like the others.
 - f2fs and ubifs get relaxed as well and I didn't test them. f2fs
   writes its dirty inodes in the checkpoint. ubifs_sync_fs() doesn't,
   but the only thing overlayfs leaves dirty there on an inode it
   doesn't track is the chmod of the work directory to 0 at mount time,
   right after creating it with mode 0. fsync=on is there for anybody
   who needs more.
 - A default mount that gets relaxed shows that in /proc/mounts.
 - A tracked overlay inode, its upper inode and the dentry of that can't
   be reclaimed while the upper inode is dirty, like a dirty inode of
   any file system. After that they stay until a sync or the shrinker
   looks. The shrinker only queues a work item that does the looking.
   The reference it drops can be the last one to an unlinked upper
   inode, and file systems allocate memory under locks that deleting a
   file needs. Deferred timestamps count as dirty, the work item hands
   them to the upper file system.
 - When a sync reports an error it marks it as seen in wb_err of the
   upper mapping, like an fsync() does. A file that is opened on the
   upper inode after that doesn't get it any more.
 - syncfs() looks at every inode in the set, also at those that the
   upper file system has written by now.
 - A file that is open for writing stays in the set, it may be mapped.
   That doesn't cover pages pinned through a mapping. A direct read into
   one that completes after the last munmap() and close() dirties them
   when the inode may have left the set.
 - An atime update puts the inode into the set as well.
 - A read-only remount of the overlay syncs it, so it now fails once
   with the error of writeback that has failed earlier.
 - The error reporting has gaps, in all of them syncfs() returns 0 like
   it does today. With the upper file system frozen or read-only we
   fall back to syncing all of it and don't look at the inodes, so an
   error is reported by the next syncfs() that can. btrfs records an
   error after it has ended writeback, which can be after we looked.
 - Data that an earlier mount of the same upper directory left dirty,
   with fsync=off for example, isn't written by a syncfs() of a later
   mount. Today it is, along with everything else.
 - Every inode of a relaxed overlay gets a slot in the xarray when it's
   allocated, also those that are never modified, so that tracking one
   can't fail. The index is the address of the inode. i_ino isn't unique
   in an overlay and xa_alloc() needs one of the three marks. All
   overlays allocate from one slab cache, so where their inodes are
   mixed a slot can cost a node of the xarray or more. With 64 overlays
   of 2000 files, same debug kernel as below, it's 21 bytes an inode if
   they're walked one after the other, 89 if all at once, 489 if they
   take turns entry by entry. Plus about 6K for every overlay.
 - If the origin of a copied up file can't be decoded, like in a user
   namespace, a later lookup makes a second overlay inode for the same
   upper inode. If the first one is tracked it pins the upper inode
   after that was unlinked, until a sync or the shrinker looks.

Tested like before, see the v1 mail below, now also with ext2 and ext4
without a journal as the upper file system. New tests for memory
pressure, a frozen and a read-only upper file system, O_TMPFILE, write
errors, and syncfs() next to tasks that create files and next to other
syncfs() calls. Everything that was changed through the overlay is in
the copy of the block device and the 192M of the neighbours aren't,
except with ext2, which doesn't get the new mode by default.

To see that the tests can fail I broke the series in 12 places, a line
or two each, and all of them are caught.

With KASAN and lockdep on, ext4, xfs, btrfs and ext4 without a journal,
one sample each, without and with the series:

  syncfs(), the neighbours' 192M dirty         258-305ms    44-50ms
  unmount, the same                            147-273ms    37-43ms
  syncfs(), the 192M under writeback [4]         42-58ms  138-171ms
  syncfs(), 4000 inodes in the set, dirty       52-153ms   71-207ms
  syncfs(), 10000 inodes in the set, clean       55-57ms  144-146ms

So it isn't free. Walking a tree takes 0 to 3% more system time, which
is what it varies by without the series.

fstests with -overlay, the auto groups of generic and overlay, on ext4
and on xfs: nothing that passes without the series fails with it, and
the run time is within 0.3%.

[2] https://lore.kernel.org/r/20261027-b4-ext2-dirsync-v1-1-3dcd012d518d@toxicpanda.com
[3] https://lore.kernel.org/r/99629e3277656888cbe84bf7490c5e54.josef@toxicpanda.com
[4] Not btrfs, nothing was in flight there without the series.

--- Original email (v1) ---

Hello,

This picks up Chengguang Xu's "implement containerized syncfs for
overlayfs", last posted as RFC V6 in November 2021 [1].

syncfs() on an overlay and unmounting one call sync_filesystem() on the
upper file system, so they write every dirty inode on it no matter who
dirtied it. We run a lot of containers on one upper file system, and
stopping one of them writes out the dirty data of all the others. That
can take a very long time, and s_umount of the upper file system is held
the whole time, so anything that wants it exclusively queues up and then
everybody else queues up behind that.

With this an overlay only writes the upper inodes that were changed
through it, and then has the upper file system commit its metadata.

The V6 discussion ended with a direction but no V7, so that's what I
tried to follow:

 - V6 kept every overlay inode with an upper inode dirty for good. Jan
   and Amir didn't like the cost of walking them. Now an overlay inode
   is marked dirty when the upper inode is changed, mostly from
   ovl_copyattr() as Amir and Miklos suggested.
 - It never looks at the dirty flags of the upper inode, Miklos pointed
   out that they're cleared when writeback starts.
 - ->write_inode() goes through write_inode_now() on the upper inode,
   and keeps the overlay inode dirty while the upper inode is open for
   writing, which covers shared mappings.
 - V6 marked the inodes DONTCACHE, which Jan was worried about. This
   goes back to the v5 patch that keeps dirty overlay inodes cached.

One thing is new. V6 called wait_sb_inodes() on the upper file system to
catch writeback that was started but not finished, which waits for
everybody's writeback. Instead ->write_inode() keeps the overlay inode
dirty while the upper inode has pages tagged dirty or under writeback.
The tags don't have the problem the inode flags have, the writeback tag
is set before the dirty tag is cleared. So a clean overlay inode means
its data is on disk, and syncfs() only waits for its own inodes.

That takes care of the data. Metadata is harder, a lot of what is
changed on the upper file system has no overlay inode that could be on
the dirty list: the root of the overlay isn't hashed, and whiteouts,
index entries, the work directories and their parent have none. File
systems that log their metadata don't care, ->sync_fs() commits it. ext2
does, there a whiteout was gone after a crash. I first had overlayfs
write all of those itself and kept finding more of them, the last one
was the link count of the shared whiteout when a whiteout is removed.

So patch 1 adds a superblock flag, SB_I_OVL_USE_SYNC_FS, by which a file
system says that ->sync_fs() commits all of its metadata, and patches 2
to 4 set it for ext4 with a journal, xfs and btrfs. Without the flag on
the upper file system nothing changes, no inode is marked dirty and
syncfs() and unmount call sync_filesystem() like they always did.

Patches 5, 7, 8 and 10 are Chengguang's with changes noted in them,
patches 6 and 9 are new. I dropped Jan's Reviewed-by tags from V6 since
the patches changed.

lazytime on the upper file system needs its own handling. The flusher
leaves the timestamps in memory for up to 12 hours, so the overlay inode
can't be clean, but it can't stay dirty and pinned for that long either.
Patch 9 gives it I_DIRTY_TIME like the upper inode has, and forwards
->sync_lazytime() to the upper inode.

Things I know about:

 - An overlay inode that loses its last reference while it is dirty is
   kept until it has been written and then sits on the LRU, like with
   any other file system. Clean and unlinked ones are still freed right
   away.
 - Every overlay on such an upper file system gets a bdi.
 - I haven't measured syncfs() with a very large number of dirty inodes.

For testing I put several overlays on one upper file system and dirty
192M next to the overlay under test, through another overlay and
directly. Then syncfs() or unmount the one overlay, copy the block
device as it is at that moment, mount the copy, and check that
everything that was changed through the overlay is in it and that the
192M aren't. That passes with ext4, xfs and btrfs as the upper file
system. With KASAN and lockdep on, syncfs() goes from 239-284ms to
45-48ms and unmount from 126-234ms to 39-44ms. With ext2 and ext4
without a journal the 192M are written as before.

fstests with -overlay, the auto groups of generic and overlay, gives the
same results with and without the series on ext4 and on xfs.

[1] https://lore.kernel.org/r/20211122030038.1938875-1-cgxu519@mykernel.net

Thanks,

Josef

---
Changes in v7:
- Link to v6: https://patch.msgid.link/20261110-b4-ovl-containerized-sync-v6-0-43d1be16a4d6@toxicpanda.com

Changes in v6:
- Link to v5: https://patch.msgid.link/20261107-b4-ovl-containerized-sync-v5-0-d34b4ae57fdf@toxicpanda.com

Changes in v5:
- Link to v4: https://patch.msgid.link/20261106-b4-ovl-containerized-sync-v4-0-6ddde0b7dd8a@toxicpanda.com

Changes in v4:
- Link to v3: https://patch.msgid.link/20261104-b4-ovl-containerized-sync-v3-0-8daa20144f53@toxicpanda.com

Changes in v3:
- Link to v2: https://patch.msgid.link/20261029-b4-ovl-containerized-sync-v2-0-4e9f08e4a052@toxicpanda.com

Changes in v2:
- Link to v1: https://patch.msgid.link/20261027-b4-ovl-containerized-sync-v1-0-802e512379a6@toxicpanda.com

---
Amir Goldstein (1):
      ovl: expand fsync mount options

Chengguang Xu (1):
      ovl: implement containerized syncfs for overlayfs

Josef Bacik (5):
      ovl: no fsync=relaxed without data integrity
      fs: add sync_fs_and_bdev()
      fs: add write_inode_for_sync()
      ovl: add a set of the inodes that a sync has to write
      ovl: track an inode when it is modified

 Documentation/filesystems/overlayfs.rst |  45 ++-
 fs/fs-writeback.c                       |  36 ++
 fs/overlayfs/Kconfig                    |  21 ++
 fs/overlayfs/Makefile                   |   2 +-
 fs/overlayfs/copy_up.c                  |   9 +
 fs/overlayfs/dir.c                      |  11 +
 fs/overlayfs/file.c                     |   7 +-
 fs/overlayfs/inode.c                    |  27 +-
 fs/overlayfs/namei.c                    |   9 +-
 fs/overlayfs/overlayfs.h                |  39 ++-
 fs/overlayfs/ovl_entry.h                |   9 +
 fs/overlayfs/params.c                   |  34 +-
 fs/overlayfs/params.h                   |   2 +
 fs/overlayfs/readdir.c                  |   1 +
 fs/overlayfs/super.c                    |  58 +++-
 fs/overlayfs/sync.c                     | 591 ++++++++++++++++++++++++++++++++
 fs/overlayfs/util.c                     |  16 +-
 fs/sync.c                               |  52 ++-
 include/linux/fs.h                      |   1 +
 include/linux/pagemap.h                 |   1 +
 20 files changed, 920 insertions(+), 51 deletions(-)
---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
change-id: 20261026-b4-ovl-containerized-sync-149bed1afe9a


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 1/7] ovl: expand fsync mount options
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity Josef Bacik
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

From: Amir Goldstein <amir73il@gmail.com>

The fsync mount option currently accepts three values:
"volatile", "strict" and "auto".

All other "enum" mount options have "on" and "off" values and this is
used by the generic _check_overlay_feature helper in fstests to probe
for support for enabling overlayfs features.

This commit adds support for values "on", "off" and "relaxed" and makes
the value of "auto" depend on a kconfig/module option fsync_auto.

The value of "off" behaves almost the same as "volatile", not
persisting changes to upper layer, but it does not create the directory
"$workdir/work/incompat/volatile", which prevents a mount cycle of a
volatile overlayfs. Some users requested to opt-out of this guard rail.

With upperdir on tmpfs, overlayfs now falls back to "fsync=off", because
the other fsync modes are futile on tmpfs.

The new value "relaxed" is a preparation for not syncing the entire
upper fs on syncfs(2) and unmount and instead tracking and syncing only
the upper inodes modified by this overlayfs.

The value of "on" is the traditional overlayfs behavior and it behaves
the same as the current value "auto", releasing the value of "auto" to
differ by the kconfig/module option fsync_auto.

By default, with kconfig/module option fsync_auto=Y, the meaning of the
value "auto" is that filesystems that support RENAME_WHITEOUT default to
"relaxed" and filesystems that do not support RENAME_WHITEOUT default to
the traditional "fsync=on".

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 Documentation/filesystems/overlayfs.rst | 36 ++++++++++++++++++++++++++++-----
 fs/overlayfs/Kconfig                    | 21 +++++++++++++++++++
 fs/overlayfs/overlayfs.h                | 24 +++++++++++++++++++---
 fs/overlayfs/params.c                   | 18 +++++++++++++----
 fs/overlayfs/params.h                   |  2 ++
 fs/overlayfs/super.c                    | 22 ++++++++++++++++++--
 6 files changed, 109 insertions(+), 14 deletions(-)

diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index 16c35b491dad..1a59500834a1 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -826,27 +826,53 @@ The fsync during copy up only guarantees that if a copy up is observed
 after a crash, the observed data is not zeroes or intermediate values
 from the copy up staging area.
 
+Traditionally, syncfs(2) and unmount of an overlay sync the entire upper
+filesystem, including the dirty data of other overlays that share it.
+Overlayfs can be told to only write back the files that were modified
+through the overlay, and then ask the upper filesystem to commit its
+metadata.
+
 On traditional local filesystems with a single journal (e.g. ext4, xfs),
 fsync on a file also persists the parent directory changes, because they
 are usually modified in the same transaction, so metadata durability during
-data copy up effectively comes for free.  Overlayfs further limits risk by
-disallowing network filesystems as upper layer.
+data copy up and the persistence of other upper changes such as whiteouts
+effectively come for free.  Other filesystems (e.g. ubifs) require an explicit
+fsync(2) on the copied up files and directories to persist their metadata.
 
 Overlayfs can be tuned to prefer performance or durability when storing
 to the underlying upper layer.  This is controlled by the "fsync" mount
 option, which supports these values:
 
-- "auto": (default)
+- "volatile": [*]
+    Prefer performance over durability (see `Volatile mount`_)
+- "off":
+    Same as "volatile", but does not prevent mounting the overlay again
+    after unmount (see `Volatile mount`_)
+- "relaxed":
     Call fsync(2) on upper file before completion of data copy up.
     No explicit fsync(2) on directory or metadata-only copy up.
+    syncfs(2) and unmount only write back the files that were modified
+    through the overlay.
+- "auto": (default)
+    "relaxed" if the upper filesystem supports RENAME_WHITEOUT,
+    otherwise "on".
+- "on":
+    syncfs(2) syncs the entire upper filesystem.
+    Copy up behaves like "relaxed".
 - "strict":
+    syncfs(2) syncs the entire upper filesystem.
     Call fsync(2) on upper file and directories before completion of any
     copy up.
-- "volatile": [*]
-    Prefer performance over durability (see `Volatile mount`_)
 
 [*] The mount option "volatile" is an alias to "fsync=volatile".
 
+The default is "fsync=on" instead of "fsync=auto" with the module option
+"fsync_auto=N" or with the kernel config option
+CONFIG_OVERLAY_FS_FSYNC_AUTO=n.
+
+If the upper filesystem has no writeback (e.g. tmpfs), a non-default mode
+falls back to "fsync=off".
+
 
 Volatile mount
 --------------
diff --git a/fs/overlayfs/Kconfig b/fs/overlayfs/Kconfig
index 2ac67e04a6fb..f1b2f3784c76 100644
--- a/fs/overlayfs/Kconfig
+++ b/fs/overlayfs/Kconfig
@@ -108,6 +108,27 @@ config OVERLAY_FS_XINO_AUTO
 
 	  If unsure, say N.
 
+config OVERLAY_FS_FSYNC_AUTO
+	bool "Overlayfs: auto select the fsync mode by default"
+	default y
+	depends on OVERLAY_FS
+	help
+	  If this config option is enabled then overlay filesystems will use
+	  "fsync=auto" by default, which selects "fsync=relaxed" if the upper
+	  filesystem supports RENAME_WHITEOUT and "fsync=on" otherwise.
+	  With "fsync=relaxed", syncfs(2) and unmount of an overlay only write
+	  back the files that were modified through the overlay, instead of
+	  syncing the entire upper filesystem.
+
+	  If this config option is disabled then overlay filesystems will use
+	  "fsync=on" by default.  It is still possible to change the default
+	  with the "fsync_auto" module option or on a filesystem instance
+	  basis with the "fsync" mount option.
+
+	  For more information, see Documentation/filesystems/overlayfs.rst
+
+	  If unsure, say Y.
+
 config OVERLAY_FS_METACOPY
 	bool "Overlayfs: turn on metadata only copy up feature by default"
 	depends on OVERLAY_FS
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 901335026a20..13fbe016ed75 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -102,8 +102,11 @@ enum {
 };
 
 enum {
+	OVL_FSYNC_OFF,
 	OVL_FSYNC_VOLATILE,
+	OVL_FSYNC_RELAXED,
 	OVL_FSYNC_AUTO,
+	OVL_FSYNC_ON,
 	OVL_FSYNC_STRICT,
 };
 
@@ -666,7 +669,8 @@ static inline bool ovl_xino_warn(struct ovl_fs *ofs)
 
 static inline bool ovl_should_sync(struct ovl_fs *ofs)
 {
-	return ofs->config.fsync_mode != OVL_FSYNC_VOLATILE;
+	return ofs->config.fsync_mode != OVL_FSYNC_OFF &&
+	       ofs->config.fsync_mode != OVL_FSYNC_VOLATILE;
 }
 
 static inline bool ovl_should_sync_metadata(struct ovl_fs *ofs)
@@ -674,9 +678,23 @@ static inline bool ovl_should_sync_metadata(struct ovl_fs *ofs)
 	return ofs->config.fsync_mode == OVL_FSYNC_STRICT;
 }
 
-static inline bool ovl_is_volatile(struct ovl_config *config)
+/*
+ * With fsync=relaxed, track the files that were modified through the overlay,
+ * so that syncfs(2) and unmount only write back those.
+ */
+static inline bool ovl_tracks_dirty(struct ovl_fs *ofs)
+{
+	return ofs->config.fsync_mode == OVL_FSYNC_RELAXED;
+}
+
+/*
+ * With fsync=relaxed, syncfs(2) and unmount only write back the files that were
+ * modified through the overlay.  Otherwise they sync the entire upper
+ * filesystem.
+ */
+static inline bool ovl_should_sync_upper_fs(struct ovl_fs *ofs)
 {
-	return config->fsync_mode == OVL_FSYNC_VOLATILE;
+	return ovl_should_sync(ofs) && !ovl_tracks_dirty(ofs);
 }
 
 /*
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index 5a15e5478a6b..549fc1e83ac9 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -28,6 +28,11 @@ module_param_named(xino_auto, ovl_xino_auto_def, bool, 0644);
 MODULE_PARM_DESC(xino_auto,
 		 "Auto enable xino feature");
 
+static bool ovl_fsync_auto_def = IS_ENABLED(CONFIG_OVERLAY_FS_FSYNC_AUTO);
+module_param_named(fsync_auto, ovl_fsync_auto_def, bool, 0644);
+MODULE_PARM_DESC(fsync_auto,
+		 "Default to fsync=auto instead of fsync=on");
+
 static bool ovl_index_def = IS_ENABLED(CONFIG_OVERLAY_FS_INDEX);
 module_param_named(index, ovl_index_def, bool, 0644);
 MODULE_PARM_DESC(index,
@@ -142,8 +147,11 @@ static int ovl_verity_mode_def(void)
 }
 
 static const struct constant_table ovl_parameter_fsync[] = {
+	{ "off",	OVL_FSYNC_OFF      },
 	{ "volatile",	OVL_FSYNC_VOLATILE },
+	{ "relaxed",	OVL_FSYNC_RELAXED  },
 	{ "auto",	OVL_FSYNC_AUTO     },
+	{ "on",		OVL_FSYNC_ON       },
 	{ "strict",	OVL_FSYNC_STRICT   },
 	{}
 };
@@ -153,9 +161,9 @@ static const char *ovl_fsync_mode(struct ovl_config *config)
 	return ovl_parameter_fsync[config->fsync_mode].name;
 }
 
-static int ovl_fsync_mode_def(void)
+int ovl_fsync_mode_def(void)
 {
-	return OVL_FSYNC_AUTO;
+	return ovl_fsync_auto_def ? OVL_FSYNC_AUTO : OVL_FSYNC_ON;
 }
 
 const struct fs_parameter_spec ovl_parameter_spec[] = {
@@ -689,9 +697,11 @@ static int ovl_parse_param(struct fs_context *fc, struct fs_parameter *param)
 		break;
 	case Opt_fsync:
 		config->fsync_mode = result.uint_32;
+		ctx->set.fsync = true;
 		break;
 	case Opt_volatile:
 		config->fsync_mode = OVL_FSYNC_VOLATILE;
+		ctx->set.fsync = true;
 		break;
 	case Opt_userxattr:
 		config->userxattr = true;
@@ -896,8 +906,8 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx,
 		config->index = false;
 	}
 
-	if (!config->upperdir && ovl_is_volatile(config)) {
-		pr_info("option \"volatile\" is meaningless in a non-upper mount, ignoring it.\n");
+	if (!config->upperdir && set.fsync) {
+		pr_info("options \"fsync\" and \"volatile\" are meaningless in a non-upper mount, ignoring them.\n");
 		config->fsync_mode = ovl_fsync_mode_def();
 	}
 
diff --git a/fs/overlayfs/params.h b/fs/overlayfs/params.h
index ffd53cdd8482..5f3572b270f3 100644
--- a/fs/overlayfs/params.h
+++ b/fs/overlayfs/params.h
@@ -15,6 +15,7 @@ struct ovl_opt_set {
 	bool redirect;
 	bool nfs_export;
 	bool index;
+	bool fsync;
 };
 
 #define OVL_MAX_STACK 500
@@ -42,3 +43,4 @@ int ovl_fs_params_verify(const struct ovl_fs_context *ctx,
 			 struct ovl_config *config);
 int ovl_show_options(struct seq_file *m, struct dentry *dentry);
 const char *ovl_xino_mode(struct ovl_config *config);
+int ovl_fsync_mode_def(void);
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 08695a99f4f9..5960974c5af2 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -15,6 +15,7 @@
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
 #include <linux/exportfs.h>
+#include <linux/backing-dev.h>
 #include <linux/file.h>
 #include <linux/fs_context.h>
 #include <linux/fs_parser.h>
@@ -762,6 +763,11 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs,
 	if (!rename_whiteout)
 		pr_warn("upper fs does not support RENAME_WHITEOUT.\n");
 
+	/* fsync=auto that is not resolved to fsync=relaxed means fsync=on */
+	if (ofs->config.fsync_mode == OVL_FSYNC_AUTO && rename_whiteout &&
+	    mnt->mnt_sb->s_bdi != &noop_backing_dev_info)
+		ofs->config.fsync_mode = OVL_FSYNC_RELAXED;
+
 	/*
 	 * Check if upper/work fs supports (trusted|user).overlay.* xattr
 	 */
@@ -814,9 +820,9 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs,
 
 	/*
 	 * For volatile mount, create a incompat/volatile/dirty file to keep
-	 * track of it.
+	 * track of it.  fsync=off allows mounting the overlay again.
 	 */
-	if (ovl_is_volatile(&ofs->config)) {
+	if (ofs->config.fsync_mode == OVL_FSYNC_VOLATILE) {
 		err = ovl_create_volatile_dirty(ofs);
 		if (err < 0) {
 			pr_err("Failed to create volatile/dirty file.\n");
@@ -1481,6 +1487,18 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb)
 			return err;
 
 		upper_sb = ovl_upper_mnt(ofs)->mnt_sb;
+		/*
+		 * An upper fs without writeback has nothing to sync and nothing
+		 * to lose, so every fsync mode behaves like fsync=off.  Keep the
+		 * default mode, which is not shown in mountinfo.
+		 */
+		if (upper_sb->s_bdi == &noop_backing_dev_info &&
+		    ofs->config.fsync_mode != ovl_fsync_mode_def() &&
+		    ofs->config.fsync_mode != OVL_FSYNC_OFF) {
+			pr_warn("upper fs does not support writeback, falling back to fsync=off.\n");
+			ofs->config.fsync_mode = OVL_FSYNC_OFF;
+		}
+
 		if (!ovl_should_sync(ofs)) {
 			ofs->errseq = errseq_sample(&upper_sb->s_wb_err);
 			if (errseq_check(&upper_sb->s_wb_err, ofs->errseq)) {

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 1/7] ovl: expand fsync mount options Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-18 18:00   ` Amir Goldstein
  2026-11-18 17:03 ` [PATCH v7 3/7] fs: add sync_fs_and_bdev() Josef Bacik
                   ` (5 subsequent siblings)
  7 siblings, 1 reply; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

A sync does not wait for a filesystem with SB_I_NO_DATA_INTEGRITY, which is
fuse, so that a server that does not answer cannot block it.

With fsync=relaxed, syncfs() and unmount of the overlay are going to write
the upper inodes one by one and wait for each.  fuse_write_inode() waits
for the server whatever the sync mode is, so they would wait for the
server once for every inode.

Do not resolve fsync=auto to fsync=relaxed on such an upper filesystem, and
fall back to fsync=on if fsync=relaxed was asked for.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 Documentation/filesystems/overlayfs.rst |  4 ++++
 fs/overlayfs/super.c                    | 13 ++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index 1a59500834a1..8a007fbc5257 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -870,6 +870,10 @@ The default is "fsync=on" instead of "fsync=auto" with the module option
 "fsync_auto=N" or with the kernel config option
 CONFIG_OVERLAY_FS_FSYNC_AUTO=n.
 
+An upper filesystem that cannot guarantee data persistence on sync (fuse)
+does not get "fsync=relaxed".  "fsync=auto" means "fsync=on" there, and
+"fsync=relaxed" falls back to it.
+
 If the upper filesystem has no writeback (e.g. tmpfs), a non-default mode
 falls back to "fsync=off".
 
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 5960974c5af2..31a58e28968c 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -765,7 +765,8 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs,
 
 	/* fsync=auto that is not resolved to fsync=relaxed means fsync=on */
 	if (ofs->config.fsync_mode == OVL_FSYNC_AUTO && rename_whiteout &&
-	    mnt->mnt_sb->s_bdi != &noop_backing_dev_info)
+	    mnt->mnt_sb->s_bdi != &noop_backing_dev_info &&
+	    !(mnt->mnt_sb->s_iflags & SB_I_NO_DATA_INTEGRITY))
 		ofs->config.fsync_mode = OVL_FSYNC_RELAXED;
 
 	/*
@@ -1499,6 +1500,16 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb)
 			ofs->config.fsync_mode = OVL_FSYNC_OFF;
 		}
 
+		/*
+		 * A sync does not wait for such an upper fs, whose server may
+		 * not answer.  A relaxed overlay waits for each upper inode.
+		 */
+		if ((upper_sb->s_iflags & SB_I_NO_DATA_INTEGRITY) &&
+		    ofs->config.fsync_mode == OVL_FSYNC_RELAXED) {
+			pr_warn("upper fs does not guarantee data integrity, falling back to fsync=on.\n");
+			ofs->config.fsync_mode = OVL_FSYNC_ON;
+		}
+
 		if (!ovl_should_sync(ofs)) {
 			ofs->errseq = errseq_sample(&upper_sb->s_wb_err);
 			if (errseq_check(&upper_sb->s_wb_err, ofs->errseq)) {

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 3/7] fs: add sync_fs_and_bdev()
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 1/7] ovl: expand fsync mount options Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-19  8:55   ` Jan Kara
  2026-11-18 17:03 ` [PATCH v7 4/7] fs: add write_inode_for_sync() Josef Bacik
                   ` (4 subsequent siblings)
  7 siblings, 1 reply; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

sync_filesystem() writes the inodes, calls ->sync_fs() and writes the block
device, first without waiting and then with.

overlayfs is about to write the inodes of its upper filesystem by itself,
only those that were modified through the overlay, and needs the other two
steps as they are.  Move them to a helper so that overlayfs does not have
to know what they are.

sync_filesystem() calls the inner helper, so it still tests for a read-only
filesystem and for s_umount once.  The exported function does both tests
for its callers.

Nothing calls the exported function yet, no functional change intended.

Suggested-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/sync.c          | 52 +++++++++++++++++++++++++++++++++++++++-------------
 include/linux/fs.h |  1 +
 2 files changed, 40 insertions(+), 13 deletions(-)

diff --git a/fs/sync.c b/fs/sync.c
index 4a84dd837b86..9adc7076c24d 100644
--- a/fs/sync.c
+++ b/fs/sync.c
@@ -22,6 +22,42 @@
 #define VALID_FLAGS (SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE| \
 			SYNC_FILE_RANGE_WAIT_AFTER)
 
+static int __sync_fs_and_bdev(struct super_block *sb, bool wait)
+{
+	int ret;
+
+	if (sb->s_op->sync_fs) {
+		ret = sb->s_op->sync_fs(sb, wait);
+		if (ret)
+			return ret;
+	}
+	if (!wait)
+		return sync_blockdev_nowait(sb->s_bdev);
+	return sync_blockdev(sb->s_bdev);
+}
+
+/**
+ * sync_fs_and_bdev - finish a sync whose inodes the caller has written
+ * @sb: the superblock, the caller holds s_umount
+ * @wait: wait for the I/O to complete
+ *
+ * These are the steps of sync_filesystem() that follow the writing of the
+ * inodes.  This is for a caller that has already written the inodes it cares
+ * about.  Like sync_filesystem() it does nothing on a read-only filesystem.
+ *
+ * Return: 0 on success, a negative error code otherwise.
+ */
+int sync_fs_and_bdev(struct super_block *sb, bool wait)
+{
+	WARN_ON(!rwsem_is_locked(&sb->s_umount));
+
+	if (sb_rdonly(sb))
+		return 0;
+
+	return __sync_fs_and_bdev(sb, wait);
+}
+EXPORT_SYMBOL_GPL(sync_fs_and_bdev);
+
 /*
  * Write out and wait upon all dirty data associated with this
  * superblock.  Filesystem data as well as the underlying block
@@ -29,7 +65,7 @@
  */
 int sync_filesystem(struct super_block *sb)
 {
-	int ret = 0;
+	int ret;
 
 	/*
 	 * We need to be protected against the filesystem going from
@@ -52,22 +88,12 @@ int sync_filesystem(struct super_block *sb)
 	 * at a time.
 	 */
 	writeback_inodes_sb(sb, WB_REASON_SYNC);
-	if (sb->s_op->sync_fs) {
-		ret = sb->s_op->sync_fs(sb, 0);
-		if (ret)
-			return ret;
-	}
-	ret = sync_blockdev_nowait(sb->s_bdev);
+	ret = __sync_fs_and_bdev(sb, false);
 	if (ret)
 		return ret;
 
 	sync_inodes_sb(sb);
-	if (sb->s_op->sync_fs) {
-		ret = sb->s_op->sync_fs(sb, 1);
-		if (ret)
-			return ret;
-	}
-	return sync_blockdev(sb->s_bdev);
+	return __sync_fs_and_bdev(sb, true);
 }
 EXPORT_SYMBOL(sync_filesystem);
 
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 99caef43ff3c..2b016d9dfb88 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -2619,6 +2619,7 @@ extern void __init vfs_caches_init(void);
 
 void emergency_thaw_all(void);
 extern int sync_filesystem(struct super_block *);
+int sync_fs_and_bdev(struct super_block *sb, bool wait);
 extern const struct file_operations def_blk_fops;
 extern const struct file_operations def_chr_fops;
 

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 4/7] fs: add write_inode_for_sync()
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
                   ` (2 preceding siblings ...)
  2026-11-18 17:03 ` [PATCH v7 3/7] fs: add sync_fs_and_bdev() Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-19  9:14   ` Jan Kara
  2026-11-18 17:03 ` [PATCH v7 5/7] ovl: add a set of the inodes that a sync has to write Josef Bacik
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

sync_inodes_sb() has the flusher write every dirty inode of a superblock
with WB_SYNC_ALL and for_sync set.  That writes the pages without waiting
for them, hands over the timestamps that lazytime has deferred while it
holds I_SYNC, and calls ->write_inode() in a way that tells the filesystem
that ->sync_fs() follows, so ext4 does not commit a transaction for every
inode.

overlayfs is about to sync some of the inodes of its upper filesystem, and
none of the exported functions does that for one inode.  write_inode_now()
and sync_inode_metadata() either do not wait for somebody else who is
writing the inode and leave deferred timestamps alone, or they wait for the
pages, clear the error flags of the mapping and make ext4 commit.

Add a function that writes one inode the way a sync does.

Nothing calls it yet.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/fs-writeback.c       | 36 ++++++++++++++++++++++++++++++++++++
 include/linux/pagemap.h |  1 +
 2 files changed, 37 insertions(+)

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index 15260cf2a29b..d265466f12a8 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -3103,6 +3103,42 @@ int write_inode_now(struct inode *inode, int sync)
 }
 EXPORT_SYMBOL(write_inode_now);
 
+/**
+ * write_inode_for_sync - write an inode like a sync of its filesystem does
+ * @inode: inode to write to disk
+ *
+ * This does for one inode what sync_inodes_sb() has the flusher do for every
+ * dirty inode of a superblock.  It writes all dirty pages, the timestamps that
+ * lazytime has deferred and the inode.  It waits for somebody else who is
+ * writing the inode, but not for the pages it writes, and it does not ask the
+ * filesystem to commit anything.
+ *
+ * This is for a caller that syncs some of the inodes of a filesystem.  It has
+ * to wait for the pages with filemap_fdatawait_keep_errors(), like
+ * wait_sb_inodes() does, and to call sync_fs_and_bdev() in the end.
+ *
+ * The caller must have a ref on the inode.
+ *
+ * Return: 0 on success, a negative error code otherwise.
+ */
+int write_inode_for_sync(struct inode *inode)
+{
+	struct writeback_control wbc = {
+		.nr_to_write = LONG_MAX,
+		.sync_mode = WB_SYNC_ALL,
+		.range_start = 0,
+		.range_end = LLONG_MAX,
+		.for_sync = 1,
+	};
+
+	if (!mapping_can_writeback(inode->i_mapping))
+		wbc.nr_to_write = 0;
+
+	might_sleep();
+	return writeback_single_inode(inode, &wbc);
+}
+EXPORT_SYMBOL_GPL(write_inode_for_sync);
+
 /**
  * sync_inode_metadata - write an inode to disk
  * @inode: the inode to sync
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 73af18a37367..172c0fe0453f 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -35,6 +35,7 @@ int filemap_invalidate_pages(struct address_space *mapping,
 			     loff_t pos, loff_t end, bool nowait);
 
 int write_inode_now(struct inode *, int sync);
+int write_inode_for_sync(struct inode *inode);
 int filemap_fdatawrite(struct address_space *);
 int filemap_flush(struct address_space *);
 int filemap_flush_nr(struct address_space *mapping, long *nr_to_write);

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 5/7] ovl: add a set of the inodes that a sync has to write
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
                   ` (3 preceding siblings ...)
  2026-11-18 17:03 ` [PATCH v7 4/7] fs: add write_inode_for_sync() Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 6/7] ovl: track an inode when it is modified Josef Bacik
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

To sync only what was written through an overlay, overlayfs has to know
which upper inodes were modified through it.

An overlay inode has no pages and no metadata of its own.  What is dirty,
under writeback or deferred by lazytime is the state of the upper inode, in
another superblock, and nothing tells overlayfs when it changes.  So do not
mirror that state.  Only remember the overlay inodes that were modified, in
an xarray of the overlay, and let a sync ask the upper inode of each.

The xarray has the marks of the page cache, dirty, towrite and writeback.
This patch only uses the first, the others come with the sync.

The set holds a reference to an inode that has a mark.  The overlay inode
holds the upper dentry, so the upper inode is one pointer away, whatever
the upper filesystem is.

An inode gets its slot in the xarray when it is allocated, and tracking it
only sets a mark, so that cannot fail.  The index is the address of the
inode.  i_ino is not unique in an overlay, and xa_alloc() needs one of the
three marks for itself.  So with fsync=relaxed, which the default resolves
to on most upper filesystems, every inode of the overlay, also one that is
never modified, costs a slot and takes the lock of the xarray when it is
allocated and when it is freed.

All overlays allocate from one slab cache, so where their inodes are mixed
a slot can cost a node of the xarray or more.  With 64 overlays of 2000
files on a guest with KASAN and lockdep it is 21 bytes an inode if they
are walked one after the other, 89 if all at once, and 489 if they take
turns entry by entry.  The tree is tall, about 6K for every overlay.

An inode leaves the set in a sync, when it is unlinked, or when memory is
needed and its upper inode has become clean.  For the last one use the
hooks of the superblock's own shrinker.  Reclaim in a memory cgroup only
runs shrinkers that have objects on a list_lru of that cgroup, which one of
our own would not have.  The hook only queues a work item.  The reference
that is dropped may be the last one to an upper inode that was unlinked,
and filesystems allocate memory under locks that deleting a file needs.

An inode only leaves the set for being clean if its upper inode is seen
clean under the i_lock of the overlay inode, which tracking takes as well.
A modification that came before is visible, and one that comes later puts
the inode back.  So whoever walks the set needs no lock against anybody
else who does.

With lazytime the upper inode holds timestamps back for up to a day.  The
last iput() would write them, but that cannot come while the inode is in
the set.  So the work item hands them to the upper filesystem.

When an inode enters the set, remember where the wb_err of the upper
mapping is.  Writeback that fails from there on, whoever does it, is for a
sync of the overlay to report, and until then the inode is not clean.

Nothing is put into the set yet, and syncfs() still syncs the upper
filesystem.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/overlayfs/Makefile    |   2 +-
 fs/overlayfs/overlayfs.h |  13 ++
 fs/overlayfs/ovl_entry.h |   9 ++
 fs/overlayfs/params.c    |   2 +
 fs/overlayfs/super.c     |  11 +-
 fs/overlayfs/sync.c      | 374 +++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 409 insertions(+), 2 deletions(-)

diff --git a/fs/overlayfs/Makefile b/fs/overlayfs/Makefile
index 5648954f8588..080fb902bafd 100644
--- a/fs/overlayfs/Makefile
+++ b/fs/overlayfs/Makefile
@@ -6,4 +6,4 @@
 obj-$(CONFIG_OVERLAY_FS) += overlay.o
 
 overlay-objs := super.o namei.o util.o inode.o file.o dir.o readdir.o \
-		copy_up.o export.o params.o xattrs.o
+		copy_up.o export.o params.o xattrs.o sync.o
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 13fbe016ed75..fad8ab77ac19 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -65,6 +65,8 @@ enum ovl_inode_flag {
 	OVL_CONST_INO,
 	OVL_HAS_DIGEST,
 	OVL_VERIFIED_DIGEST,
+	/* Modified since a sync has started to write it, under i_lock */
+	OVL_TRACKED,
 };
 
 enum ovl_entry_flag {
@@ -886,6 +888,17 @@ struct inode *ovl_get_inode(struct super_block *sb,
 			    struct ovl_inode_params *oip);
 void ovl_copyattr(struct inode *to);
 
+/* sync.c */
+void ovl_track_init(struct ovl_fs *ofs);
+void ovl_track_exit(struct ovl_fs *ofs);
+int ovl_track_alloc(struct super_block *sb, struct inode *inode);
+void ovl_track_inode(struct inode *inode);
+void ovl_untrack_inode(struct inode *inode);
+void ovl_forget_inode(struct inode *inode);
+void ovl_drop_tracked(struct ovl_fs *ofs);
+long ovl_nr_cached_objects(struct super_block *sb, struct shrink_control *sc);
+long ovl_free_cached_objects(struct super_block *sb, struct shrink_control *sc);
+
 /* vfs fileattr flags read from overlay.protattr xattr to ovl inode */
 #define OVL_PROT_I_FLAGS_MASK  (S_APPEND | S_IMMUTABLE)
 /* vfs fileattr flags copied from real to ovl inode */
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index ac07e8769f9b..44e22b2860d2 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -92,6 +92,13 @@ struct ovl_fs {
 	/* r/o snapshot of upperdir sb's only taken on volatile mounts */
 	errseq_t errseq;
 	bool casefold;
+	/* The inodes that syncfs() and unmount have to write, see sync.c */
+	struct xarray tracked;
+	atomic_long_t nr_tracked;
+	/* Reclaim has this drop what is clean: how many, and where it goes on */
+	struct work_struct release_work;
+	atomic_long_t release_nr;
+	unsigned long release_id[2];
 };
 
 /* Number of lower layers, not including data-only layers */
@@ -164,6 +171,8 @@ struct ovl_inode {
 	const char *redirect;
 	u64 version;
 	unsigned long flags;
+	/* wb_err of the upper mapping up to where a sync has reported it */
+	errseq_t wb_err;
 	struct inode vfs_inode;
 	struct dentry *__upperdentry;
 	struct ovl_entry *oe;
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index 549fc1e83ac9..370dd6341392 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -842,6 +842,7 @@ int ovl_init_fs_context(struct fs_context *fc)
 	fc->ops			= &ovl_context_ops;
 
 	mutex_init(&ofs->whiteout_lock);
+	ovl_track_init(ofs);
 	return 0;
 
 out_err:
@@ -883,6 +884,7 @@ void ovl_free_fs(struct ovl_fs *ofs)
 	kfree(ofs->config.workdir);
 	if (ofs->creator_cred)
 		put_cred(ofs->creator_cred);
+	ovl_track_exit(ofs);
 	kfree(ofs);
 }
 
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 31a58e28968c..5eb510bf1b9a 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -221,6 +221,10 @@ static struct inode *ovl_alloc_inode(struct super_block *sb)
 	oi->__upperdentry = NULL;
 	oi->lowerdata_redirect = NULL;
 	oi->oe = NULL;
+	if (ovl_track_alloc(sb, &oi->vfs_inode)) {
+		kmem_cache_free(ovl_inode_cachep, oi);
+		return NULL;
+	}
 	mutex_init(&oi->lock);
 
 	return &oi->vfs_inode;
@@ -240,6 +244,7 @@ static void ovl_destroy_inode(struct inode *inode)
 {
 	struct ovl_inode *oi = OVL_I(inode);
 
+	ovl_forget_inode(inode);
 	dput(oi->__upperdentry);
 	ovl_stack_put(ovl_lowerstack(oi->oe), ovl_numlower(oi->oe));
 	if (S_ISDIR(inode->i_mode))
@@ -252,8 +257,10 @@ static void ovl_put_super(struct super_block *sb)
 {
 	struct ovl_fs *ofs = OVL_FS(sb);
 
-	if (ofs)
+	if (ofs) {
+		ovl_drop_tracked(ofs);
 		ovl_free_fs(ofs);
+	}
 }
 
 /* Sync real dirty inodes in upper filesystem (if it exists) */
@@ -327,6 +334,8 @@ static const struct super_operations ovl_super_operations = {
 	.drop_inode	= inode_just_drop,
 	.put_super	= ovl_put_super,
 	.sync_fs	= ovl_sync_fs,
+	.nr_cached_objects = ovl_nr_cached_objects,
+	.free_cached_objects = ovl_free_cached_objects,
 	.statfs		= ovl_statfs,
 	.show_options	= ovl_show_options,
 };
diff --git a/fs/overlayfs/sync.c b/fs/overlayfs/sync.c
new file mode 100644
index 000000000000..03dfd903bf08
--- /dev/null
+++ b/fs/overlayfs/sync.c
@@ -0,0 +1,374 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * The inodes that syncfs() and unmount have to write with fsync=relaxed.
+ *
+ * An overlay inode has no pages and no metadata of its own.  What is dirty,
+ * under writeback or deferred by lazytime is the state of the upper inode, and
+ * nothing tells overlayfs when it changes.  So overlayfs does not mirror that
+ * state.  It only remembers which overlay inodes were modified, and a sync
+ * asks the upper inode of each of them.
+ *
+ * The set is an xarray with the marks of the page cache:
+ *
+ *   OVL_X_DIRTY      the upper inode was modified
+ *   OVL_X_TOWRITE    a sync that has begun is going to write it
+ *   OVL_X_WRITEBACK  a sync has started to write it
+ *
+ * The set holds one reference to an inode that has any of them.  They change
+ * under the i_lock of the inode, except that OVL_X_TOWRITE is added to
+ * OVL_X_DIRTY under the lock of the xarray alone.
+ *
+ * There is one difference to the page cache.  No completion clears
+ * OVL_X_WRITEBACK, whoever has waited does.  Another sync may have started the
+ * inode again meanwhile, so the mark only goes when the upper inode is seen
+ * clean.  That way whoever walks the set needs no lock against anybody else who
+ * does.
+ */
+
+#include <linux/fs.h>
+#include <linux/log2.h>
+#include <linux/mount.h>
+#include <linux/pagemap.h>
+#include <linux/writeback.h>
+#include <linux/xarray.h>
+#include <linux/xattr.h>
+#include "overlayfs.h"
+
+#define OVL_X_DIRTY	XA_MARK_0
+#define OVL_X_TOWRITE	XA_MARK_1
+#define OVL_X_WRITEBACK	XA_MARK_2
+
+/*
+ * Index by address.  Inodes are at least their size apart and the shift rounds
+ * down, so it stays unique, and neighbours in a slab share an xarray node.
+ */
+static unsigned long ovl_slot(struct inode *inode)
+{
+	return (unsigned long)OVL_I(inode) >> ilog2(sizeof(struct ovl_inode));
+}
+
+/*
+ * Give a new inode its slot in the set, so that tracking it later cannot fail.
+ * An entry without a mark is never looked at.
+ */
+int ovl_track_alloc(struct super_block *sb, struct inode *inode)
+{
+	struct ovl_fs *ofs = OVL_FS(sb);
+
+	if (!ovl_tracks_dirty(ofs))
+		return 0;
+
+	return xa_insert(&ofs->tracked, ovl_slot(inode), inode, GFP_KERNEL);
+}
+
+/*
+ * @inode is about to be freed.  It is not tracked, or it had a reference.
+ *
+ * Mount settles on the fsync mode after it has allocated the first trap inodes,
+ * so the mode does not tell if @inode has a slot.
+ */
+void ovl_forget_inode(struct inode *inode)
+{
+	struct ovl_fs *ofs = OVL_FS(inode->i_sb);
+
+	WARN_ON_ONCE(ovl_test_flag(OVL_TRACKED, inode));
+	if (!xa_empty(&ofs->tracked))
+		xa_erase(&ofs->tracked, ovl_slot(inode));
+}
+
+static bool ovl_marked(struct ovl_fs *ofs, unsigned long id)
+{
+	return xa_get_mark(&ofs->tracked, id, OVL_X_DIRTY) ||
+	       xa_get_mark(&ofs->tracked, id, OVL_X_TOWRITE) ||
+	       xa_get_mark(&ofs->tracked, id, OVL_X_WRITEBACK);
+}
+
+/* OVL_TRACKED is OVL_X_DIRTY without a walk down the tree */
+static void ovl_set_dirty(struct ovl_fs *ofs, struct inode *inode)
+{
+	unsigned long id = ovl_slot(inode);
+
+	lockdep_assert_held(&inode->i_lock);
+
+	if (ovl_test_flag(OVL_TRACKED, inode))
+		return;
+	if (!ovl_marked(ofs, id)) {
+		struct inode *upper = ovl_inode_upper(inode);
+
+		__iget(inode);
+		atomic_long_inc(&ofs->nr_tracked);
+		/* A sync reports what fails now, or what nobody has seen */
+		OVL_I(inode)->wb_err = filemap_sample_wb_err(upper->i_mapping);
+	}
+	ovl_set_flag(OVL_TRACKED, inode);
+	xa_set_mark(&ofs->tracked, id, OVL_X_DIRTY);
+}
+
+/* OVL_X_DIRTY goes first, nobody adds OVL_X_TOWRITE after that */
+static void ovl_clear_dirty(struct ovl_fs *ofs, struct inode *inode)
+{
+	unsigned long id = ovl_slot(inode);
+
+	lockdep_assert_held(&inode->i_lock);
+
+	ovl_clear_flag(OVL_TRACKED, inode);
+	xa_clear_mark(&ofs->tracked, id, OVL_X_DIRTY);
+	xa_clear_mark(&ofs->tracked, id, OVL_X_TOWRITE);
+}
+
+/*
+ * The upper inode was modified through this overlay, remember the overlay
+ * inode for syncfs() and unmount.  The caller holds a reference to @inode.
+ *
+ * Do not look at the upper inode here.  Its dirty flags are cleared when its
+ * writeback starts, not when it completes.
+ *
+ * Nobody needs an unlinked file to be durable, and the reference would keep its
+ * space.  An O_TMPFILE that can still get a name is not such a file.
+ *
+ * This takes i_lock even if @inode is tracked already, see ovl_untrack().
+ */
+void ovl_track_inode(struct inode *inode)
+{
+	if (!ovl_tracks_dirty(OVL_FS(inode->i_sb)) || !ovl_inode_upper(inode))
+		return;
+
+	spin_lock(&inode->i_lock);
+	if (inode->i_nlink || (inode_state_read(inode) & I_LINKABLE))
+		ovl_set_dirty(OVL_FS(inode->i_sb), inode);
+	spin_unlock(&inode->i_lock);
+}
+
+/*
+ * Is there nothing that a sync would have to do for the upper inode?  The
+ * caller holds i_lock.
+ *
+ * Stores through a shared mapping dirty the pages without a call into
+ * overlayfs, so the inode has to stay tracked while there may be one.  The
+ * mapping holds the real file open for writing.  Read i_writecount first: the
+ * page cache tag is set before the last fput().
+ *
+ * That does not cover pages that are pinned through such a mapping.  A direct
+ * read into it that completes after the last munmap() and close() dirties them
+ * with nobody looking.
+ *
+ * An error that was not reported yet is something to do as well.
+ */
+static bool ovl_upper_clean(struct inode *inode)
+{
+	struct inode *upper = ovl_inode_upper(inode);
+	struct address_space *mapping = upper->i_mapping;
+
+	if (atomic_read_acquire(&upper->i_writecount) > 0)
+		return false;
+
+	return !mapping_tagged(mapping, PAGECACHE_TAG_DIRTY) &&
+	       !mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK) &&
+	       !(inode_state_read_once(upper) & (I_DIRTY_ALL | I_SYNC)) &&
+	       !filemap_check_wb_err(mapping, OVL_I(inode)->wb_err);
+}
+
+enum ovl_untrack {
+	OVL_UNTRACK_ALWAYS,
+	OVL_UNTRACK_CLEAN,	/* if the upper inode is clean */
+	OVL_UNTRACK_UNLINKED,	/* if it has no link and cannot get one */
+};
+
+/*
+ * Take @inode out of the set.  The caller holds a reference to @inode of its
+ * own.
+ *
+ * The condition is looked at under i_lock as well.  A task that has called
+ * ovl_track_inode() before is ordered by i_lock, so what it did to the upper
+ * inode, or to the link count, is visible.  One that calls it later tracks
+ * @inode again.
+ */
+static bool ovl_untrack(struct inode *inode, enum ovl_untrack when)
+{
+	struct ovl_fs *ofs = OVL_FS(inode->i_sb);
+	unsigned long id = ovl_slot(inode);
+	bool done;
+
+	spin_lock(&inode->i_lock);
+	done = ovl_marked(ofs, id);
+	if (done && when == OVL_UNTRACK_CLEAN)
+		done = ovl_upper_clean(inode);
+	else if (done && when == OVL_UNTRACK_UNLINKED)
+		done = !inode->i_nlink &&
+		       (!(inode_state_read(inode) & I_LINKABLE) ||
+			hlist_empty(&inode->i_dentry));
+	if (done) {
+		ovl_clear_dirty(ofs, inode);
+		xa_clear_mark(&ofs->tracked, id, OVL_X_WRITEBACK);
+		atomic_long_dec(&ofs->nr_tracked);
+	}
+	spin_unlock(&inode->i_lock);
+
+	if (done)
+		iput(inode);
+	return done;
+}
+
+/*
+ * A link to @inode was removed, or a dentry of it has gone and it may be an
+ * O_TMPFILE that never got a name.  The caller holds a reference to @inode.
+ */
+void ovl_untrack_inode(struct inode *inode)
+{
+	if (ovl_tracks_dirty(OVL_FS(inode->i_sb)) && !inode->i_nlink)
+		ovl_untrack(inode, OVL_UNTRACK_UNLINKED);
+}
+
+/*
+ * Get a reference to the first inode with @mark from *@id on.  A mark is
+ * cleared before the reference of the set is dropped, so under the lock of the
+ * xarray an inode that has one is alive.
+ */
+static struct inode *ovl_next_marked(struct ovl_fs *ofs, unsigned long *id,
+				     xa_mark_t mark)
+{
+	struct inode *inode;
+
+	xa_lock(&ofs->tracked);
+	inode = xa_find(&ofs->tracked, id, ULONG_MAX, mark);
+	if (inode)
+		ihold(inode);
+	xa_unlock(&ofs->tracked);
+
+	return inode;
+}
+
+/* Does only lazytime keep @upper from being clean? */
+static bool ovl_only_lazytime(struct inode *upper)
+{
+	return (inode_state_read_once(upper) & I_DIRTY_ALL) == I_DIRTY_TIME &&
+	       atomic_read(&upper->i_writecount) <= 0;
+}
+
+/*
+ * Drop @inode if its upper inode is clean.
+ *
+ * lazytime holds timestamps back for up to a day.  The last iput() would write
+ * them, but that cannot come while the overlay inode is tracked.  So hand them
+ * to the upper filesystem.  Nobody has asked for them to be on disk, so there
+ * is nothing to wait for.
+ */
+static void ovl_release_inode(struct vfsmount *mnt, struct inode *inode)
+{
+	struct inode *upper = ovl_inode_upper(inode);
+	int err;
+
+	if (ovl_untrack(inode, OVL_UNTRACK_CLEAN) ||
+	    !ovl_only_lazytime(upper) || mnt_get_write_access(mnt))
+		return;
+
+	err = write_inode_for_sync(upper);
+	mnt_put_write_access(mnt);
+	if (err)
+		errseq_set(&inode->i_sb->s_wb_err, err);
+
+	ovl_untrack(inode, OVL_UNTRACK_CLEAN);
+}
+
+/*
+ * Look at up to *@nr inodes with @mark.  Dropping one writes nothing, so that
+ * also works on an upper filesystem that has become read-only.  It keeps a
+ * freeze out, because on a frozen filesystem dropping the last reference to an
+ * inode that was unlinked sleeps until the thaw.
+ */
+static void ovl_scan_marked(struct ovl_fs *ofs, xa_mark_t mark,
+			    unsigned long *id, unsigned long *nr)
+{
+	struct vfsmount *mnt = ovl_upper_mnt(ofs);
+	struct inode *inode;
+
+	while (*nr && sb_start_write_trylock(mnt->mnt_sb)) {
+		--*nr;
+		inode = ovl_next_marked(ofs, id, mark);
+		if (inode) {
+			ovl_release_inode(mnt, inode);
+			iput(inode);
+		}
+		sb_end_write(mnt->mnt_sb);
+
+		if (!inode) {
+			*id = 0;
+			break;
+		}
+		++*id;
+		cond_resched();
+	}
+}
+
+/*
+ * An inode may be left with OVL_X_WRITEBACK alone, if a sync did not get to
+ * wait for it.
+ */
+static void ovl_release_work(struct work_struct *work)
+{
+	struct ovl_fs *ofs = container_of(work, struct ovl_fs, release_work);
+	unsigned long nr = atomic_long_xchg(&ofs->release_nr, 0);
+
+	ovl_scan_marked(ofs, OVL_X_DIRTY, &ofs->release_id[0], &nr);
+	ovl_scan_marked(ofs, OVL_X_WRITEBACK, &ofs->release_id[1], &nr);
+}
+
+void ovl_track_init(struct ovl_fs *ofs)
+{
+	xa_init_flags(&ofs->tracked, XA_FLAGS_ACCOUNT);
+	INIT_WORK(&ofs->release_work, ovl_release_work);
+}
+
+void ovl_track_exit(struct ovl_fs *ofs)
+{
+	unsigned long id = 0;
+
+	/* Not xa_empty(), an insert that has failed can leave nodes behind */
+	WARN_ON(xa_find(&ofs->tracked, &id, ULONG_MAX, XA_PRESENT));
+	xa_destroy(&ofs->tracked);
+}
+
+/*
+ * Unmount.  The sync of generic_shutdown_super() has written what it could, if
+ * the overlay is not read-only.  evict_inodes() has left alone what is still
+ * in the set.  Trap inodes can be in the xarray as well, without a mark.
+ */
+void ovl_drop_tracked(struct ovl_fs *ofs)
+{
+	struct inode *inode;
+	unsigned long id;
+
+	cancel_work_sync(&ofs->release_work);
+
+	xa_for_each(&ofs->tracked, id, inode) {
+		ihold(inode);
+		ovl_untrack(inode, OVL_UNTRACK_ALWAYS);
+		iput(inode);
+		cond_resched();
+	}
+}
+
+long ovl_nr_cached_objects(struct super_block *sb, struct shrink_control *sc)
+{
+	return atomic_long_read(&OVL_FS(sb)->nr_tracked);
+}
+
+/*
+ * A tracked inode, its upper inode and the dentry of that cannot be reclaimed.
+ * While the upper inode is dirty that is as with any filesystem.  Afterwards
+ * nothing tells overlayfs, so look when memory is needed.
+ *
+ * Not from reclaim itself.  The reference that is dropped may be the last one
+ * to an upper inode that was unlinked, and handing over timestamps takes locks
+ * of an upper inode that is in use.  Filesystems allocate memory under the
+ * locks that both need.
+ */
+long ovl_free_cached_objects(struct super_block *sb, struct shrink_control *sc)
+{
+	struct ovl_fs *ofs = OVL_FS(sb);
+
+	atomic_long_add(sc->nr_to_scan, &ofs->release_nr);
+	queue_work(system_dfl_wq, &ofs->release_work);
+
+	return 0;
+}

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 6/7] ovl: track an inode when it is modified
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
                   ` (4 preceding siblings ...)
  2026-11-18 17:03 ` [PATCH v7 5/7] ovl: add a set of the inodes that a sync has to write Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-18 17:03 ` [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs Josef Bacik
  2026-11-18 18:14 ` [PATCH v7 0/7] ovl: only sync what was written through the overlay Amir Goldstein
  7 siblings, 0 replies; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Josef Bacik

Put an overlay inode into the set of inodes to sync when its upper inode is
modified through the overlay.

Nearly every modification ends in ovl_copyattr(), so do it there.  These do
not, so they track the inode themselves:

 - copy up, for the inode and for its parent
 - ovl_instantiate(), for a new inode and for a new hard link
 - atime updates, if the atime of the upper inode has changed
 - the root, because the mount may have set xattrs on the upper root
 - shared writable mappings
 - the removal of a stale "impure" xattr in readdir
 - the "origin" xattr that lookup sets on a merge directory
 - the helpers that set the "impure", "redirect", "opaque" and "nlink"
   xattrs, and ovl_clear_empty() for the parent.  The operation that calls
   them may still fail, and then it does not get to ovl_copyattr().
 - a setattr that fails, it may have truncated some of the file

Take an inode out of the set when its last link is removed.  Nobody needs
a deleted file to be durable, and the reference of the set would keep its
space.  An O_TMPFILE that can still get a name is tracked like any other
file, and leaves the set when its dentry goes without one.

syncfs() does not look at the set yet.  Until the next patch an inode only
leaves it when it is unlinked, at unmount, or when memory is needed.

Based on work by Chengguang Xu.

Link: https://lore.kernel.org/r/20211122030038.1938875-1-cgxu519@mykernel.net
Suggested-by: Amir Goldstein <amir73il@gmail.com>
Suggested-by: Miklos Szeredi <miklos@szeredi.hu>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 fs/overlayfs/copy_up.c   |  9 +++++++++
 fs/overlayfs/dir.c       | 11 +++++++++++
 fs/overlayfs/file.c      |  7 ++++++-
 fs/overlayfs/inode.c     | 27 +++++++++++++++++++++------
 fs/overlayfs/namei.c     |  9 +++++++--
 fs/overlayfs/overlayfs.h |  1 +
 fs/overlayfs/readdir.c   |  1 +
 fs/overlayfs/super.c     | 11 +++++++++++
 fs/overlayfs/util.c      | 16 ++++++++++++++--
 9 files changed, 81 insertions(+), 11 deletions(-)

diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index c147ad155b79..f5c89af79e15 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -1223,6 +1223,15 @@ static int ovl_copy_up_one(struct dentry *parent, struct dentry *dentry,
 			err = ovl_link_up(&ctx);
 		if (!err && ovl_dentry_needs_data_copy_up_locked(dentry, flags))
 			err = ovl_copy_up_meta_inode_data(&ctx);
+		/*
+		 * All three write to the upper inode, and there may be no later
+		 * write that would track the overlay inode.  The first two
+		 * also add an entry to the upper inode of the parent, set its
+		 * "impure" xattr and restore its timestamps.
+		 */
+		ovl_track_inode(d_inode(dentry));
+		if (parent)
+			ovl_track_inode(d_inode(parent));
 		ovl_copy_up_end(dentry);
 	}
 	do_delayed_call(&done);
diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index 1792e4114621..3d50e0e7183f 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -276,6 +276,9 @@ static int ovl_set_opaque_xerr(struct dentry *dentry, struct dentry *upper,
 	err = ovl_check_setxattr(ofs, upper, OVL_XATTR_OPAQUE, "y", 1, xerr);
 	if (!err)
 		ovl_dentry_set_opaque(dentry);
+	/* A new directory has no inode yet, ovl_instantiate() tracks it */
+	if (d_inode(dentry))
+		ovl_track_inode(d_inode(dentry));
 
 	return err;
 }
@@ -330,6 +333,9 @@ static int ovl_instantiate(struct dentry *dentry, struct inode *inode,
 		inc_nlink(inode);
 	}
 
+	/* The upper inode was just created, or has got a new link */
+	ovl_track_inode(inode);
+
 	if (tmpfile)
 		d_mark_tmpfile(tmpfile, inode);
 
@@ -457,6 +463,8 @@ static struct dentry *ovl_clear_empty(struct dentry *dentry,
 
 	err = ovl_do_rename_rd(&rd);
 	end_renaming(&rd);
+	/* The caller tracks the parent, unless it fails after this */
+	ovl_track_inode(d_inode(dentry->d_parent));
 	if (err)
 		goto out_cleanup_unlocked;
 
@@ -971,6 +979,7 @@ static int ovl_do_remove(struct dentry *dentry, bool is_dir)
 			clear_nlink(dentry->d_inode);
 		else
 			ovl_drop_nlink(dentry);
+		ovl_untrack_inode(dentry->d_inode);
 	}
 	ovl_nlink_end(dentry);
 
@@ -1127,6 +1136,7 @@ static int ovl_set_redirect(struct dentry *dentry, bool samedir)
 	err = ovl_check_setxattr(ofs, ovl_dentry_upper(dentry),
 				 OVL_XATTR_REDIRECT,
 				 redirect, strlen(redirect), -EXDEV);
+	ovl_track_inode(d_inode(dentry));
 	if (!err) {
 		spin_lock(&dentry->d_lock);
 		ovl_dentry_set_redirect(dentry, redirect);
@@ -1339,6 +1349,7 @@ static int ovl_rename_upper(struct ovl_renamedata *ovlrd, struct list_head *list
 			clear_nlink(d_inode(new));
 		else
 			ovl_drop_nlink(new);
+		ovl_untrack_inode(d_inode(new));
 	}
 
 	ovl_dir_modified(old->d_parent, ovl_type_origin(old) ||
diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c
index 7433220d4ad6..95dc34f2690b 100644
--- a/fs/overlayfs/file.c
+++ b/fs/overlayfs/file.c
@@ -472,8 +472,13 @@ static int ovl_mmap(struct file *file, struct vm_area_struct *vma)
 		.cred = ovl_creds(file_inode(file)->i_sb),
 		.accessed = ovl_file_accessed,
 	};
+	int err;
 
-	return backing_file_mmap(of->realfile, vma, &ctx);
+	err = backing_file_mmap(of->realfile, vma, &ctx);
+	/* Stores through the mapping go straight to the upper inode */
+	if (!err && vma_is_shared_maywrite(vma))
+		ovl_track_inode(file_inode(file));
+	return err;
 }
 
 static long ovl_fallocate(struct file *file, int mode, loff_t offset, loff_t len)
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index a11564e90751..08de2e9fdffa 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -84,11 +84,17 @@ int ovl_setattr(const struct mnt_idmap *idmap, struct dentry *dentry,
 		if (err)
 			goto out_put_write;
 
+		/* Before a truncate dirties a page, for errors of writeback */
+		if (winode)
+			ovl_track_inode(dentry->d_inode);
+
 		inode_lock(upperdentry->d_inode);
 		with_ovl_creds(dentry->d_sb)
 			err = ovl_do_notify_change(ofs, upperdentry, attr);
 		if (!err)
 			ovl_copyattr(dentry->d_inode);
+		else
+			ovl_track_inode(dentry->d_inode);
 		inode_unlock(upperdentry->d_inode);
 		ovl_drop_write(dentry);
 
@@ -582,11 +588,17 @@ int ovl_update_time(struct inode *inode, enum fs_update_time type,
 		};
 
 		if (upperpath.dentry) {
+			struct inode *upper = d_inode(upperpath.dentry);
+			struct timespec64 old = inode_get_atime(upper), new;
+
 			if (flags & IOCB_NOWAIT)
 				return -EAGAIN;
 			touch_atime(&upperpath);
-			inode_set_atime_to_ts(inode,
-					      inode_get_atime(d_inode(upperpath.dentry)));
+			new = inode_get_atime(upper);
+			inode_set_atime_to_ts(inode, new);
+			/* touch_atime() may have had nothing to do */
+			if (!timespec64_equal(&old, &new))
+				ovl_track_inode(inode);
 		}
 	}
 	return 0;
@@ -898,7 +910,8 @@ void ovl_inode_init(struct inode *inode, struct ovl_inode_params *oip,
 	oi->lowerdata_redirect = oip->lowerdata_redirect;
 
 	realinode = ovl_inode_real(inode);
-	ovl_copyattr(inode);
+	/* Nothing was changed, the inode is only being looked up */
+	__ovl_copyattr(inode);
 	ovl_copyflags(realinode, inode);
 	ovl_map_ino(inode, ino, fsid);
 }
@@ -975,7 +988,7 @@ static int ovl_set_nlink_common(struct dentry *dentry,
 	struct inode *inode = d_inode(dentry);
 	struct inode *realinode = d_inode(realdentry);
 	char buf[13];
-	int len;
+	int len, err;
 
 	len = snprintf(buf, sizeof(buf), format,
 		       (int) (inode->i_nlink - realinode->i_nlink));
@@ -983,8 +996,10 @@ static int ovl_set_nlink_common(struct dentry *dentry,
 	if (WARN_ON(len >= sizeof(buf)))
 		return -EIO;
 
-	return ovl_setxattr(OVL_FS(inode->i_sb), ovl_dentry_upper(dentry),
-			    OVL_XATTR_NLINK, buf, len);
+	err = ovl_setxattr(OVL_FS(inode->i_sb), ovl_dentry_upper(dentry),
+			   OVL_XATTR_NLINK, buf, len);
+	ovl_track_inode(inode);
+	return err;
 }
 
 int ovl_set_nlink_upper(struct dentry *dentry)
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index c6127edb9be9..353392dbbde4 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -951,7 +951,8 @@ static int ovl_fix_origin(struct ovl_fs *ofs, struct dentry *dentry,
 	ovl_drop_write(dentry);
 out:
 	kfree(fh);
-	return err;
+	/* Tell the caller that @upper was modified, it has no inode yet */
+	return err ? err : 1;
 }
 
 static int ovl_maybe_validate_verity(struct dentry *dentry)
@@ -1102,6 +1103,7 @@ static int ovl_lookup_layers(struct ovl_lookup_ctx *ctx, struct ovl_lookup_data
 	struct dentry *origin = NULL;
 	bool upperopaque = false;
 	bool uppermetacopy = false;
+	bool origin_fixed = false;
 	int metacopy_size = 0;
 	unsigned int i;
 	int err;
@@ -1181,10 +1183,11 @@ static int ovl_lookup_layers(struct ovl_lookup_ctx *ctx, struct ovl_lookup_data
 		 */
 		if (ctx->upperdentry && !ctx->ctr && !ofs->noxattr && d->is_dir) {
 			err = ovl_fix_origin(ofs, dentry, this, ctx->upperdentry);
-			if (err) {
+			if (err < 0) {
 				dput(this);
 				return err;
 			}
+			origin_fixed = err;
 		}
 
 		/*
@@ -1370,6 +1373,8 @@ static int ovl_lookup_layers(struct ovl_lookup_ctx *ctx, struct ovl_lookup_data
 			return PTR_ERR(inode);
 
 		ctx->inode = inode;
+		if (origin_fixed)
+			ovl_track_inode(inode);
 		if (ctx->upperdentry && !uppermetacopy)
 			ovl_set_flag(OVL_UPPERDATA, ctx->inode);
 
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index fad8ab77ac19..f4770b4c0222 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -886,6 +886,7 @@ bool ovl_lookup_trap_inode(struct super_block *sb, struct dentry *dir);
 struct inode *ovl_get_trap_inode(struct super_block *sb, struct dentry *dir);
 struct inode *ovl_get_inode(struct super_block *sb,
 			    struct ovl_inode_params *oip);
+void __ovl_copyattr(struct inode *to);
 void ovl_copyattr(struct inode *to);
 
 /* sync.c */
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
index 2018e3114655..599e76e1294a 100644
--- a/fs/overlayfs/readdir.c
+++ b/fs/overlayfs/readdir.c
@@ -745,6 +745,7 @@ static struct ovl_dir_cache *ovl_cache_get_impure(const struct path *path)
 		if (!ovl_want_write(dentry)) {
 			ovl_removexattr(ofs, ovl_dentry_upper(dentry),
 					OVL_XATTR_IMPURE);
+			ovl_track_inode(inode);
 			ovl_drop_write(dentry);
 		}
 		ovl_clear_flag(OVL_IMPURE, inode);
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 5eb510bf1b9a..7f7d3993bc7a 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -187,8 +187,16 @@ static int ovl_dentry_init(struct dentry *dentry)
 #define ovl_dentry_init NULL
 #endif
 
+/* An O_TMPFILE that has not got a name cannot get one without a dentry */
+static void ovl_d_iput(struct dentry *dentry, struct inode *inode)
+{
+	ovl_untrack_inode(inode);
+	iput(inode);
+}
+
 static const struct dentry_operations ovl_dentry_operations = {
 	.d_real = ovl_d_real,
+	.d_iput = ovl_d_iput,
 	.d_revalidate = ovl_dentry_revalidate,
 	.d_weak_revalidate = ovl_dentry_weak_revalidate,
 	.d_init = ovl_dentry_init,
@@ -197,6 +205,7 @@ static const struct dentry_operations ovl_dentry_operations = {
 #if IS_ENABLED(CONFIG_UNICODE)
 static const struct dentry_operations ovl_dentry_ci_operations = {
 	.d_real = ovl_d_real,
+	.d_iput = ovl_d_iput,
 	.d_revalidate = ovl_dentry_revalidate,
 	.d_weak_revalidate = ovl_dentry_weak_revalidate,
 	.d_hash = generic_ci_d_hash,
@@ -1407,6 +1416,8 @@ static struct dentry *ovl_get_root(struct super_block *sb,
 	ovl_dentry_set_flag(OVL_E_CONNECTED, root);
 	ovl_set_upperdata(d_inode(root));
 	ovl_inode_init(d_inode(root), &oip, ino, fsid);
+	/* The mount may have set xattrs on the upper root */
+	ovl_track_inode(d_inode(root));
 	WARN_ON(!!IS_CASEFOLDED(d_inode(root)) != ofs->casefold);
 	ovl_dentry_init_flags(root, upperdentry, oe, DCACHE_OP_WEAK_REVALIDATE);
 	/* root keeps a reference of upperdentry */
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 521717209b2e..2effec540dac 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -921,6 +921,8 @@ int ovl_set_impure(struct dentry *dentry, struct dentry *upperdentry)
 	err = ovl_check_setxattr(ofs, upperdentry, OVL_XATTR_IMPURE, "y", 1, 0);
 	if (!err)
 		ovl_set_flag(OVL_IMPURE, d_inode(dentry));
+	/* Here, because the operation that needs this may still fail */
+	ovl_track_inode(d_inode(dentry));
 
 	return err;
 }
@@ -1482,7 +1484,7 @@ int ovl_sync_status(struct ovl_fs *ofs)
 }
 
 /*
- * ovl_copyattr() - copy inode attributes from layer to ovl inode
+ * __ovl_copyattr() - copy inode attributes from layer to ovl inode
  *
  * When overlay copies inode information from an upper or lower layer to the
  * relevant overlay inode it will apply the idmapping of the upper or lower
@@ -1492,7 +1494,7 @@ int ovl_sync_status(struct ovl_fs *ofs)
  * map any lower or upper inode owned by id 1001 to id 1000. These mapping
  * helpers are nops when the relevant layer isn't idmapped.
  */
-void ovl_copyattr(struct inode *inode)
+void __ovl_copyattr(struct inode *inode)
 {
 	struct path realpath;
 	struct inode *realinode;
@@ -1516,3 +1518,13 @@ void ovl_copyattr(struct inode *inode)
 	i_size_write(inode, i_size_read(realinode));
 	spin_unlock(&inode->i_lock);
 }
+
+/*
+ * The attributes are copied because the real inode was modified through the
+ * overlay, so this is also where the overlay inode is tracked for a sync.
+ */
+void ovl_copyattr(struct inode *inode)
+{
+	__ovl_copyattr(inode);
+	ovl_track_inode(inode);
+}

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
                   ` (5 preceding siblings ...)
  2026-11-18 17:03 ` [PATCH v7 6/7] ovl: track an inode when it is modified Josef Bacik
@ 2026-11-18 17:03 ` Josef Bacik
  2026-11-18 18:09   ` Amir Goldstein
  2026-11-18 18:14 ` [PATCH v7 0/7] ovl: only sync what was written through the overlay Amir Goldstein
  7 siblings, 1 reply; 14+ messages in thread
From: Josef Bacik @ 2026-11-18 17:03 UTC (permalink / raw)
  To: Miklos Szeredi, Amir Goldstein, Christian Brauner
  Cc: Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	Chengguang Xu, Josef Bacik

From: Chengguang Xu <charliecgxu@tencent.com>

Now overlayfs can only sync own dirty inodes during syncfs,
so remove unnecessary sync_filesystem() on upper file system.

Do that with fsync=relaxed only, which is what the default, fsync=auto,
selects on most filesystems.  fsync=on brings the old behaviour back.

Take the steps of sync_filesystem() for the upper filesystem, but only for
the tracked inodes, and go about it like the page cache does.

Without wait, ->sync_fs() gives every inode that is marked dirty the
towrite mark.  That is what this sync has to write.  Then it starts
writeback of the upper inode of each of them, and replaces dirty and
towrite by the writeback mark.

With wait, for every inode with the writeback mark, it calls
write_inode_for_sync(), which also hands over the timestamps that lazytime
has deferred, and waits for the data, whoever has started to write it.

Both end with sync_fs_and_bdev(), which is what sync_filesystem() does
after the inodes.  Do the same for unmount and for a read-only remount.

A sync tags once, so tasks that modify files do not keep it going.  The
marks are shared by all syncs that are running, so it also writes and waits
for what another one tags ahead of it.  A walk only goes up, so it ends.

In the page cache the completion of the I/O clears the writeback mark.
Here whoever has waited does, and another sync may have started the inode
again meanwhile.  So the mark only goes if the upper inode is seen clean.
Until then every sync sees the inode and waits for it as well, so they do
not have to wait for each other.  Otherwise the inode is marked dirty
again, which also covers an upper inode that is open for writing: stores
through a shared mapping do not call into overlayfs.

That does not cover pages that are pinned through such a mapping.  A direct
read into it that completes after the last munmap() and close() dirties
them when the inode may have left the set, and the next syncfs() does not
write them.

syncfs() now also returns the error of writeback that the flusher of the
upper filesystem has done before, which it did not do so far.  It compares
the wb_err of the upper mapping with what it was when the inode entered the
set.  Unlike the error flags of the mapping, nobody can take that away, an
fsync() for one.  The error is recorded in the superblock of the overlay
under the lock that protects the cursor, so another syncfs() that runs at
the same time gets it as well.

A read-only remount of the overlay syncs it the same way, so it now fails
with such an error, once.

Keep a freeze and a read-only remount of the upper filesystem out while an
inode is handed over.  If one has already begun when ->sync_fs() is to
wait, sync the whole upper filesystem, like before.  If one kept it from
starting an inode and has gone away since, start that inode then.

Upper inodes that have no overlay inode, like whiteouts and the work
directories, are not tracked.  It is up to ->sync_fs() and the block device
to get them to disk.  ext4, xfs and btrfs log all metadata, and ext4
without a journal keeps it in the buffers of the block device.

With many containers on one upper filesystem, stopping one of them no
longer writes out the dirty data of all the others.

Signed-off-by: Chengguang Xu <charliecgxu@tencent.com>
[josef: everything after the first paragraph]
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
 Documentation/filesystems/overlayfs.rst |   7 +-
 fs/overlayfs/overlayfs.h                |   1 +
 fs/overlayfs/params.c                   |  14 +--
 fs/overlayfs/super.c                    |   3 +
 fs/overlayfs/sync.c                     | 217 ++++++++++++++++++++++++++++++++
 5 files changed, 230 insertions(+), 12 deletions(-)

diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
index 8a007fbc5257..00e2cba65e2c 100644
--- a/Documentation/filesystems/overlayfs.rst
+++ b/Documentation/filesystems/overlayfs.rst
@@ -830,7 +830,12 @@ Traditionally, syncfs(2) and unmount of an overlay sync the entire upper
 filesystem, including the dirty data of other overlays that share it.
 Overlayfs can be told to only write back the files that were modified
 through the overlay, and then ask the upper filesystem to commit its
-metadata.
+metadata and write its block device.
+
+In that mode syncfs(2) also returns the error of writeback of such a file
+that has failed earlier, like fsync(2) on the file does.  As after fsync(2),
+a file that is opened on the upper filesystem later does not get that error
+any more.  A read-only remount of the overlay fails with such an error once.
 
 On traditional local filesystems with a single journal (e.g. ext4, xfs),
 fsync on a file also persists the parent directory changes, because they
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index f4770b4c0222..63200551a7d0 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -899,6 +899,7 @@ void ovl_forget_inode(struct inode *inode);
 void ovl_drop_tracked(struct ovl_fs *ofs);
 long ovl_nr_cached_objects(struct super_block *sb, struct shrink_control *sc);
 long ovl_free_cached_objects(struct super_block *sb, struct shrink_control *sc);
+int ovl_sync_tracked(struct super_block *sb, int wait);
 
 /* vfs fileattr flags read from overlay.protattr xattr to ovl inode */
 #define OVL_PROT_I_FLAGS_MASK  (S_APPEND | S_IMMUTABLE)
diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
index 370dd6341392..90111c480440 100644
--- a/fs/overlayfs/params.c
+++ b/fs/overlayfs/params.c
@@ -774,22 +774,14 @@ static int ovl_reconfigure(struct fs_context *fc)
 {
 	struct super_block *sb = fc->root->d_sb;
 	struct ovl_fs *ofs = OVL_FS(sb);
-	struct super_block *upper_sb;
-	int ret = 0;
 
 	if (!(fc->sb_flags & SB_RDONLY) && ovl_force_readonly(ofs))
 		return -EROFS;
 
-	if (fc->sb_flags & SB_RDONLY && !sb_rdonly(sb)) {
-		upper_sb = ovl_upper_mnt(ofs)->mnt_sb;
-		if (ovl_should_sync(ofs)) {
-			down_read(&upper_sb->s_umount);
-			ret = sync_filesystem(upper_sb);
-			up_read(&upper_sb->s_umount);
-		}
-	}
+	if (fc->sb_flags & SB_RDONLY && !sb_rdonly(sb) && ovl_should_sync(ofs))
+		return sync_filesystem(sb);
 
-	return ret;
+	return 0;
 }
 
 static const struct fs_context_operations ovl_context_ops = {
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 7f7d3993bc7a..a2d527f62d75 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -287,6 +287,9 @@ static int ovl_sync_fs(struct super_block *sb, int wait)
 	if (!ret)
 		return ret;
 
+	if (!ovl_should_sync_upper_fs(ofs))
+		return ovl_sync_tracked(sb, wait);
+
 	/*
 	 * Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC).
 	 * All the super blocks will be iterated, including upper_sb.
diff --git a/fs/overlayfs/sync.c b/fs/overlayfs/sync.c
index 03dfd903bf08..55d40cde8984 100644
--- a/fs/overlayfs/sync.c
+++ b/fs/overlayfs/sync.c
@@ -372,3 +372,220 @@ long ovl_free_cached_objects(struct super_block *sb, struct shrink_control *sc)
 
 	return 0;
 }
+
+/*
+ * Keep a freeze and a read-only remount of the upper filesystem out.  Do not
+ * wait for either.
+ */
+static bool ovl_upper_start_write(struct ovl_fs *ofs)
+{
+	struct vfsmount *mnt = ovl_upper_mnt(ofs);
+
+	if (!sb_start_write_trylock(mnt->mnt_sb))
+		return false;
+	if (mnt_get_write_access(mnt)) {
+		sb_end_write(mnt->mnt_sb);
+		return false;
+	}
+	return true;
+}
+
+/*
+ * Like tag_pages_for_writeback(): what is dirty now is what this sync has to
+ * write.
+ */
+static void ovl_tag_for_writeback(struct ovl_fs *ofs)
+{
+	XA_STATE(xas, &ofs->tracked, 0);
+	unsigned int tagged = 0;
+	struct inode *inode;
+
+	xas_lock(&xas);
+	xas_for_each_marked(&xas, inode, ULONG_MAX, OVL_X_DIRTY) {
+		xas_set_mark(&xas, OVL_X_TOWRITE);
+		if (++tagged % XA_CHECK_SCHED)
+			continue;
+
+		xas_pause(&xas);
+		xas_unlock(&xas);
+		cond_resched();
+		xas_lock(&xas);
+	}
+	xas_unlock(&xas);
+}
+
+/*
+ * Start to write the data of the upper inode.  OVL_X_WRITEBACK is set before
+ * OVL_X_DIRTY is cleared, so that a sync always finds one of them, and
+ * OVL_X_DIRTY is cleared before the write, so that a modification that comes
+ * later sets it again.
+ */
+static void ovl_start_inode(struct super_block *sb, struct inode *inode)
+{
+	struct ovl_fs *ofs = OVL_FS(sb);
+	unsigned long id = ovl_slot(inode);
+	bool start;
+	int err;
+
+	spin_lock(&inode->i_lock);
+	/* It may have been unlinked, or another sync was here first */
+	start = xa_get_mark(&ofs->tracked, id, OVL_X_TOWRITE);
+	if (start) {
+		xa_set_mark(&ofs->tracked, id, OVL_X_WRITEBACK);
+		ovl_clear_dirty(ofs, inode);
+	}
+	spin_unlock(&inode->i_lock);
+
+	if (!start)
+		return;
+
+	/* Tags the pages first, so it does not follow an appender */
+	err = filemap_fdatawrite(ovl_inode_upper(inode)->i_mapping);
+	/* If ->sync_fs() returned it the VFS would not call again to wait */
+	if (err)
+		errseq_set(&sb->s_wb_err, err);
+}
+
+/*
+ * Write the upper inode and wait for its data, whoever has started to write it.
+ *
+ * After that OVL_X_WRITEBACK goes if the upper inode is clean.  If it is not,
+ * another sync may have started it again and relies on the mark, or it is open
+ * for writing and the next sync has to write it again.
+ *
+ * Writeback of the upper inode that has failed since @inode has entered the
+ * set, whoever has done it, shows in the wb_err of the mapping.  Nobody takes
+ * that away, unlike the error flags.  It is recorded in the superblock of the
+ * overlay under the lock that protects the cursor.  So another syncfs() that is
+ * here at the same time finds the one or the other.
+ *
+ * btrfs records an error after it has ended writeback.  If that is too late for
+ * the look here, the error is reported once the inode is modified again.
+ */
+static int ovl_wait_inode(struct super_block *sb, struct inode *inode)
+{
+	struct ovl_fs *ofs = OVL_FS(sb);
+	struct inode *upper = ovl_inode_upper(inode);
+	struct address_space *mapping = upper->i_mapping;
+	unsigned long id = ovl_slot(inode);
+	bool drop = false;
+	int err, err2 = 0;
+
+	err = write_inode_for_sync(upper);
+	if (err)
+		errseq_set(&sb->s_wb_err, err);
+	filemap_fdatawait_keep_errors(mapping);
+
+	spin_lock(&inode->i_lock);
+	if (xa_get_mark(&ofs->tracked, id, OVL_X_WRITEBACK)) {
+		err2 = errseq_check_and_advance(&mapping->wb_err,
+						&OVL_I(inode)->wb_err);
+		if (err2)
+			errseq_set(&sb->s_wb_err, err2);
+
+		if (ovl_upper_clean(inode)) {
+			xa_clear_mark(&ofs->tracked, id, OVL_X_WRITEBACK);
+			drop = !ovl_marked(ofs, id);
+			if (drop)
+				atomic_long_dec(&ofs->nr_tracked);
+		} else {
+			ovl_set_dirty(ofs, inode);
+		}
+	}
+	spin_unlock(&inode->i_lock);
+
+	if (drop)
+		iput(inode);
+	return err ? err : err2;
+}
+
+/*
+ * Start every inode that has OVL_X_TOWRITE, or finish every inode that has
+ * OVL_X_WRITEBACK.
+ *
+ * Do not write to an upper filesystem that is frozen or read-only, or on its
+ * way there.  Keep a freeze and a read-only remount out meanwhile, inode by
+ * inode, because writers wait behind a freeze that waits for us.
+ *
+ * Returns false if one of the two has begun.
+ */
+static bool ovl_walk_marked(struct super_block *sb, xa_mark_t mark, int *error)
+{
+	struct ovl_fs *ofs = OVL_FS(sb);
+	struct inode *inode;
+	unsigned long id = 0;
+	int err;
+
+	for (;;) {
+		if (!ovl_upper_start_write(ofs))
+			return false;
+
+		inode = ovl_next_marked(ofs, &id, mark);
+		if (inode && mark == OVL_X_TOWRITE) {
+			ovl_start_inode(sb, inode);
+		} else if (inode) {
+			err = ovl_wait_inode(sb, inode);
+			if (err && !*error)
+				*error = err;
+		}
+		iput(inode);
+		mnt_drop_write(ovl_upper_mnt(ofs));
+
+		if (!inode)
+			return true;
+		id++;
+		cond_resched();
+	}
+}
+
+/*
+ * This tags once, so tasks that modify files do not keep it going.  The marks
+ * are shared with every other sync that is running, and what those tag ahead of
+ * a walk is written and waited for as well.  But a walk only goes up, so it
+ * comes to an end.
+ */
+static bool ovl_sync_inodes(struct super_block *sb, int wait, int *error)
+{
+	if (!wait)
+		ovl_tag_for_writeback(OVL_FS(sb));
+
+	/*
+	 * With @wait this is for what the call before could not start, because
+	 * of a freeze or a remount that has gone away again.
+	 */
+	if (!ovl_walk_marked(sb, OVL_X_TOWRITE, error))
+		return false;
+
+	return !wait || ovl_walk_marked(sb, OVL_X_WRITEBACK, error);
+}
+
+/*
+ * syncfs(), unmount or a read-only remount of an overlay with fsync=relaxed.
+ * sync_filesystem() calls this without @wait first and then with.  The steps
+ * are those that it takes itself for the upper filesystem, except that only the
+ * tracked inodes are written.
+ */
+int ovl_sync_tracked(struct super_block *sb, int wait)
+{
+	struct super_block *upper_sb = ovl_upper_mnt(OVL_FS(sb))->mnt_sb;
+	int ret, err = 0;
+	bool done;
+
+	done = ovl_sync_inodes(sb, wait, &err);
+
+	down_read(&upper_sb->s_umount);
+	/*
+	 * If the upper filesystem is frozen or read-only, or on its way there,
+	 * sync all of it when asked to wait, like without fsync=relaxed.
+	 *
+	 * Otherwise this has to take care of the upper inodes that have no
+	 * overlay inode, like whiteouts and the work directories.
+	 */
+	if (wait && !done)
+		ret = sync_filesystem(upper_sb);
+	else
+		ret = sync_fs_and_bdev(upper_sb, wait);
+	up_read(&upper_sb->s_umount);
+
+	return err ? err : ret;
+}

-- 
2.55.0


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity
  2026-11-18 17:03 ` [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity Josef Bacik
@ 2026-11-18 18:00   ` Amir Goldstein
  0 siblings, 0 replies; 14+ messages in thread
From: Amir Goldstein @ 2026-11-18 18:00 UTC (permalink / raw)
  To: Josef Bacik
  Cc: Miklos Szeredi, Christian Brauner, Alexander Viro, Jan Kara,
	Christoph Hellwig, Chris Mason, Chengguang Xu, linux-unionfs,
	linux-fsdevel, linux-kernel

On Wed, Nov 18, 2026 at 6:03 PM Josef Bacik <josef@toxicpanda.com> wrote:
>
> A sync does not wait for a filesystem with SB_I_NO_DATA_INTEGRITY, which is
> fuse, so that a server that does not answer cannot block it.
>
> With fsync=relaxed, syncfs() and unmount of the overlay are going to write
> the upper inodes one by one and wait for each.  fuse_write_inode() waits
> for the server whatever the sync mode is, so they would wait for the
> server once for every inode.
>
> Do not resolve fsync=auto to fsync=relaxed on such an upper filesystem, and
> fall back to fsync=on if fsync=relaxed was asked for.
>
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>

Ha I very much didn't want to include FUSE in this exclusive club
This was an oversight on my part.

Reviewed-by: Amir Goldstein <amir73il@gmail.com>

> ---
>  Documentation/filesystems/overlayfs.rst |  4 ++++
>  fs/overlayfs/super.c                    | 13 ++++++++++++-
>  2 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
> index 1a59500834a1..8a007fbc5257 100644
> --- a/Documentation/filesystems/overlayfs.rst
> +++ b/Documentation/filesystems/overlayfs.rst
> @@ -870,6 +870,10 @@ The default is "fsync=on" instead of "fsync=auto" with the module option
>  "fsync_auto=N" or with the kernel config option
>  CONFIG_OVERLAY_FS_FSYNC_AUTO=n.
>
> +An upper filesystem that cannot guarantee data persistence on sync (fuse)
> +does not get "fsync=relaxed".  "fsync=auto" means "fsync=on" there, and
> +"fsync=relaxed" falls back to it.
> +
>  If the upper filesystem has no writeback (e.g. tmpfs), a non-default mode
>  falls back to "fsync=off".
>
> diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
> index 5960974c5af2..31a58e28968c 100644
> --- a/fs/overlayfs/super.c
> +++ b/fs/overlayfs/super.c
> @@ -765,7 +765,8 @@ static int ovl_make_workdir(struct super_block *sb, struct ovl_fs *ofs,
>
>         /* fsync=auto that is not resolved to fsync=relaxed means fsync=on */
>         if (ofs->config.fsync_mode == OVL_FSYNC_AUTO && rename_whiteout &&
> -           mnt->mnt_sb->s_bdi != &noop_backing_dev_info)
> +           mnt->mnt_sb->s_bdi != &noop_backing_dev_info &&
> +           !(mnt->mnt_sb->s_iflags & SB_I_NO_DATA_INTEGRITY))
>                 ofs->config.fsync_mode = OVL_FSYNC_RELAXED;
>
>         /*
> @@ -1499,6 +1500,16 @@ static int ovl_fill_super_creds(struct fs_context *fc, struct super_block *sb)
>                         ofs->config.fsync_mode = OVL_FSYNC_OFF;
>                 }
>
> +               /*
> +                * A sync does not wait for such an upper fs, whose server may
> +                * not answer.  A relaxed overlay waits for each upper inode.
> +                */
> +               if ((upper_sb->s_iflags & SB_I_NO_DATA_INTEGRITY) &&
> +                   ofs->config.fsync_mode == OVL_FSYNC_RELAXED) {
> +                       pr_warn("upper fs does not guarantee data integrity, falling back to fsync=on.\n");
> +                       ofs->config.fsync_mode = OVL_FSYNC_ON;
> +               }
> +
>                 if (!ovl_should_sync(ofs)) {
>                         ofs->errseq = errseq_sample(&upper_sb->s_wb_err);
>                         if (errseq_check(&upper_sb->s_wb_err, ofs->errseq)) {
>
> --
> 2.55.0
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs
  2026-11-18 17:03 ` [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs Josef Bacik
@ 2026-11-18 18:09   ` Amir Goldstein
  0 siblings, 0 replies; 14+ messages in thread
From: Amir Goldstein @ 2026-11-18 18:09 UTC (permalink / raw)
  To: Josef Bacik
  Cc: Miklos Szeredi, Christian Brauner, Alexander Viro, Jan Kara,
	Christoph Hellwig, Chris Mason, Chengguang Xu, linux-unionfs,
	linux-fsdevel, linux-kernel, Chengguang Xu

On Wed, Nov 18, 2026 at 6:03 PM Josef Bacik <josef@toxicpanda.com> wrote:
>
> From: Chengguang Xu <charliecgxu@tencent.com>
>
> Now overlayfs can only sync own dirty inodes during syncfs,
> so remove unnecessary sync_filesystem() on upper file system.
>
> Do that with fsync=relaxed only, which is what the default, fsync=auto,
> selects on most filesystems.  fsync=on brings the old behaviour back.
>
> Take the steps of sync_filesystem() for the upper filesystem, but only for
> the tracked inodes, and go about it like the page cache does.
>
> Without wait, ->sync_fs() gives every inode that is marked dirty the
> towrite mark.  That is what this sync has to write.  Then it starts
> writeback of the upper inode of each of them, and replaces dirty and
> towrite by the writeback mark.
>
> With wait, for every inode with the writeback mark, it calls
> write_inode_for_sync(), which also hands over the timestamps that lazytime
> has deferred, and waits for the data, whoever has started to write it.
>
> Both end with sync_fs_and_bdev(), which is what sync_filesystem() does
> after the inodes.  Do the same for unmount and for a read-only remount.
>
> A sync tags once, so tasks that modify files do not keep it going.  The
> marks are shared by all syncs that are running, so it also writes and waits
> for what another one tags ahead of it.  A walk only goes up, so it ends.
>
> In the page cache the completion of the I/O clears the writeback mark.
> Here whoever has waited does, and another sync may have started the inode
> again meanwhile.  So the mark only goes if the upper inode is seen clean.
> Until then every sync sees the inode and waits for it as well, so they do
> not have to wait for each other.  Otherwise the inode is marked dirty
> again, which also covers an upper inode that is open for writing: stores
> through a shared mapping do not call into overlayfs.
>
> That does not cover pages that are pinned through such a mapping.  A direct
> read into it that completes after the last munmap() and close() dirties
> them when the inode may have left the set, and the next syncfs() does not
> write them.
>
> syncfs() now also returns the error of writeback that the flusher of the
> upper filesystem has done before, which it did not do so far.  It compares
> the wb_err of the upper mapping with what it was when the inode entered the
> set.  Unlike the error flags of the mapping, nobody can take that away, an
> fsync() for one.  The error is recorded in the superblock of the overlay
> under the lock that protects the cursor, so another syncfs() that runs at
> the same time gets it as well.
>
> A read-only remount of the overlay syncs it the same way, so it now fails
> with such an error, once.
>
> Keep a freeze and a read-only remount of the upper filesystem out while an
> inode is handed over.  If one has already begun when ->sync_fs() is to
> wait, sync the whole upper filesystem, like before.  If one kept it from
> starting an inode and has gone away since, start that inode then.
>
> Upper inodes that have no overlay inode, like whiteouts and the work
> directories, are not tracked.  It is up to ->sync_fs() and the block device
> to get them to disk.  ext4, xfs and btrfs log all metadata, and ext4
> without a journal keeps it in the buffers of the block device.
>
> With many containers on one upper filesystem, stopping one of them no
> longer writes out the dirty data of all the others.
>
> Signed-off-by: Chengguang Xu <charliecgxu@tencent.com>
> [josef: everything after the first paragraph]
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> ---
>  Documentation/filesystems/overlayfs.rst |   7 +-
>  fs/overlayfs/overlayfs.h                |   1 +
>  fs/overlayfs/params.c                   |  14 +--
>  fs/overlayfs/super.c                    |   3 +
>  fs/overlayfs/sync.c                     | 217 ++++++++++++++++++++++++++++++++
>  5 files changed, 230 insertions(+), 12 deletions(-)
>
> diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst
> index 8a007fbc5257..00e2cba65e2c 100644
> --- a/Documentation/filesystems/overlayfs.rst
> +++ b/Documentation/filesystems/overlayfs.rst
> @@ -830,7 +830,12 @@ Traditionally, syncfs(2) and unmount of an overlay sync the entire upper
>  filesystem, including the dirty data of other overlays that share it.
>  Overlayfs can be told to only write back the files that were modified
>  through the overlay, and then ask the upper filesystem to commit its
> -metadata.
> +metadata and write its block device.
> +
> +In that mode syncfs(2) also returns the error of writeback of such a file
> +that has failed earlier, like fsync(2) on the file does.  As after fsync(2),
> +a file that is opened on the upper filesystem later does not get that error
> +any more.  A read-only remount of the overlay fails with such an error once.
>
>  On traditional local filesystems with a single journal (e.g. ext4, xfs),
>  fsync on a file also persists the parent directory changes, because they
> diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
> index f4770b4c0222..63200551a7d0 100644
> --- a/fs/overlayfs/overlayfs.h
> +++ b/fs/overlayfs/overlayfs.h
> @@ -899,6 +899,7 @@ void ovl_forget_inode(struct inode *inode);
>  void ovl_drop_tracked(struct ovl_fs *ofs);
>  long ovl_nr_cached_objects(struct super_block *sb, struct shrink_control *sc);
>  long ovl_free_cached_objects(struct super_block *sb, struct shrink_control *sc);
> +int ovl_sync_tracked(struct super_block *sb, int wait);
>
>  /* vfs fileattr flags read from overlay.protattr xattr to ovl inode */
>  #define OVL_PROT_I_FLAGS_MASK  (S_APPEND | S_IMMUTABLE)
> diff --git a/fs/overlayfs/params.c b/fs/overlayfs/params.c
> index 370dd6341392..90111c480440 100644
> --- a/fs/overlayfs/params.c
> +++ b/fs/overlayfs/params.c
> @@ -774,22 +774,14 @@ static int ovl_reconfigure(struct fs_context *fc)
>  {
>         struct super_block *sb = fc->root->d_sb;
>         struct ovl_fs *ofs = OVL_FS(sb);
> -       struct super_block *upper_sb;
> -       int ret = 0;
>
>         if (!(fc->sb_flags & SB_RDONLY) && ovl_force_readonly(ofs))
>                 return -EROFS;
>
> -       if (fc->sb_flags & SB_RDONLY && !sb_rdonly(sb)) {
> -               upper_sb = ovl_upper_mnt(ofs)->mnt_sb;
> -               if (ovl_should_sync(ofs)) {
> -                       down_read(&upper_sb->s_umount);
> -                       ret = sync_filesystem(upper_sb);
> -                       up_read(&upper_sb->s_umount);
> -               }
> -       }
> +       if (fc->sb_flags & SB_RDONLY && !sb_rdonly(sb) && ovl_should_sync(ofs))
> +               return sync_filesystem(sb);
>
> -       return ret;
> +       return 0;
>  }
>
>  static const struct fs_context_operations ovl_context_ops = {
> diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
> index 7f7d3993bc7a..a2d527f62d75 100644
> --- a/fs/overlayfs/super.c
> +++ b/fs/overlayfs/super.c
> @@ -287,6 +287,9 @@ static int ovl_sync_fs(struct super_block *sb, int wait)
>         if (!ret)
>                 return ret;
>
> +       if (!ovl_should_sync_upper_fs(ofs))
> +               return ovl_sync_tracked(sb, wait);
> +
>         /*
>          * Not called for sync(2) call or an emergency sync (SB_I_SKIP_SYNC).
>          * All the super blocks will be iterated, including upper_sb.

Maybe also move ovl_sync_fs() to sync.c makes this
a very nice finale ;)

Thanks,
Amir.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 0/7] ovl: only sync what was written through the overlay
  2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
                   ` (6 preceding siblings ...)
  2026-11-18 17:03 ` [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs Josef Bacik
@ 2026-11-18 18:14 ` Amir Goldstein
  7 siblings, 0 replies; 14+ messages in thread
From: Amir Goldstein @ 2026-11-18 18:14 UTC (permalink / raw)
  To: Josef Bacik
  Cc: Miklos Szeredi, Christian Brauner, Alexander Viro, Jan Kara,
	Christoph Hellwig, Chris Mason, Chengguang Xu, linux-unionfs,
	linux-fsdevel, linux-kernel, Chengguang Xu

On Wed, Nov 18, 2026 at 6:03 PM Josef Bacik <josef@toxicpanda.com> wrote:
>
> v1: https://lore.kernel.org/all/20261027-b4-ovl-containerized-sync-v1-0-802e512379a6@toxicpanda.com/
> v2: https://lore.kernel.org/all/20261029-b4-ovl-containerized-sync-v2-0-4e9f08e4a052@toxicpanda.com/
> v3: https://lore.kernel.org/all/20261104-b4-ovl-containerized-sync-v3-0-8daa20144f53@toxicpanda.com/
> v4: https://lore.kernel.org/all/20261106-b4-ovl-containerized-sync-v4-0-6ddde0b7dd8a@toxicpanda.com/
> v5: https://lore.kernel.org/all/20261107-b4-ovl-containerized-sync-v5-0-d34b4ae57fdf@toxicpanda.com/
> v6: https://lore.kernel.org/all/20261110-b4-ovl-containerized-sync-v6-0-43d1be16a4d6@toxicpanda.com/
>
> I've completely redesigned this, because there were so many problems
> that I kept tripping over trying to maintain state between the upper and
> the lower inode. I'm not in love with this design either, but it's the
> simplest. I'm hoping somebody spots something obvious and we can do it a
> different way.

FWIW, I think this is now at least simple enough for me to review ;)

I do expect it may take me some time though.

Thanks,
Amir.



>
> v6->v7:
> - Redone without a flusher. An overlay inode has nothing of its own.
>   What is dirty, under writeback or deferred by lazytime is the state of
>   the upper inode, and nothing tells overlayfs when that changes. Up to
>   v6 the dirty bit of the overlay inode mirrored that state on a bdi of
>   the overlay, and most of what the reviews found was the mirror being
>   off, or the flusher acting on it. The worst one: with 3000 files open
>   for writing and the system over the background threshold the flusher
>   of the overlay took 79% of a CPU going over its list. (Sashiko)
> - Now the overlay only keeps a set of the inodes that were modified, an
>   xarray, patch 5. Only syncfs(), unmount and a read-only remount walk
>   it, and they ask the upper inode of each. Periodic and background
>   writeback is all up to the upper file system, like without the series.
> - The xarray has the marks of the page cache and a sync uses them the
>   same way. ->sync_fs() without wait tags what is dirty with towrite,
>   starts writeback of those and marks them writeback, with wait it waits
>   for what is marked writeback. The one difference is that there is no
>   completion to clear the writeback mark. Whoever has waited does, and
>   only if the upper inode is clean, another sync may have started it
>   again.
> - The set holds a reference. An inode leaves it when it's unlinked, or
>   when a sync or the shrinker sees the upper inode clean. The shrinker
>   part hangs off the ->nr_cached_objects() and ->free_cached_objects()
>   hooks. Nobody who walks the set needs a lock against anybody else.
> - Gone: the bdi, ->write_inode(), ->drop_inode(), the lazytime patch,
>   "ovl: report failed upper writeback in syncfs()", and the special
>   cases for the root and for unhashed inodes. 7 patches, v6 had 8.
> - syncfs() waits for the writeback of its own inodes, whoever started
>   it, and for nobody else's.
> - New patch 4: write_inode_for_sync() in fs/fs-writeback.c writes one
>   inode the way the flusher does for sync_inodes_sb(), WB_SYNC_ALL with
>   for_sync. Up to v6 I pieced that together from sync_inode_metadata()
>   without and with wait, so that ext4 wouldn't commit for every inode.
>   (Sashiko)
> - Deferred timestamps are handed to the upper file system in a sync, or
>   when memory is needed, so lazytime keeps working in between.
> - syncfs() also reports writeback that the upper file system did earlier
>   and that failed. The overlay inode remembers where wb_err of the upper
>   mapping was when it entered the set, and a sync compares. An fsync()
>   in between doesn't take that away.
> - A freeze and a read-only remount of the upper file system are held off
>   while an inode is handed over. If one has already begun syncfs() syncs
>   the whole upper file system like before. That closes what v6 only
>   described in the changelog of patch 7. (Sashiko)
> - New patch 2: an upper file system with SB_I_NO_DATA_INTEGRITY, which
>   is fuse, doesn't get relaxed. A sync doesn't wait for the fuse server,
>   writing the upper inodes one by one does. (Sashiko)
> - The helpers that set the impure, redirect, opaque and nlink xattrs
>   track the inode themselves, and so does ovl_clear_empty(). A link or a
>   rename that failed after them left the inode untracked. A setattr that
>   fails tracks it too. (Sashiko)
> - Rebased onto 7.4-rc1. v6 was still on the base of the earlier
>   versions, and patch 4 conflicted with 4f61ff2ebfb3 ("ovl: use the
>   pinned upper dir in ovl_fix_origin()").
>
> v5->v6:
> - Lookup marks a merge directory and its parent when it sets the origin
>   and impure xattrs on them. (Sashiko)
> - New patch 8: if somebody else writes the pages of an upper inode and
>   that fails, syncfs() on the overlay reports it. It's best effort, the
>   limits are in the changelog, and it's on its own so it can be dropped.
>   (Sashiko)
> - Left alone: if the deferred timestamps can't be handed over because
>   the upper file system is being frozen, the sync of the freeze can miss
>   them. They are written after the thaw, it's in the changelog of patch
>   7. (Sashiko)
>
> v4->v5:
> - Added sync_fs_and_bdev() to fs/sync.c for the ->sync_fs() and block
>   device part of sync_filesystem(), so ovl_sync_fs() doesn't open code
>   it, patch 2. (Christian)
> - ovl_instantiate() marks a new inode and a new hard link dirty.
>   (Christian, Sashiko)
> - Copy up marks the parent as well, and the inode also when it only adds
>   a link to an indexed upper inode. (Sashiko)
> - Dropped "ovl: make the root inode appear hashed". A fake hash stays in
>   the slab object when the inode is freed, and the next overlay inode in
>   it looks hashed. The root isn't tracked at all now, ovl_sync_fs()
>   writes its upper inode every time. (Sashiko)
> - lazytime got a lot simpler. The overlay inode doesn't get I_DIRTY_TIME
>   any more, deferred timestamps are handed to the upper file system when
>   ->write_inode() finds them. That takes care of the I/O from dentry
>   reclaim, so this doesn't need anything from Jan's deferred inode
>   reclaim series. (Sashiko, Christian)
> - ovl_sync_upper_inode() could lose an error.
> - Left alone: fsync=auto still goes by RENAME_WHITEOUT, no flag. What
>   that leaves uncovered is under "Things I know about". (Christian)
> - Left alone: if somebody else starts the writeback of an upper inode
>   and it fails, syncfs() on the overlay doesn't report it. It never has.
>   (Sashiko)
>
> v3->v4:
> - Patch 1 is v2 of Amir's patch. It has ovl_tracks_dirty() and picks the
>   mode for tmpfs at mount time, so I dropped my version from patch 2.
>   fsync=auto stays auto if it doesn't become relaxed. (Amir)
>
> v2->v3:
> - Patch 1 is Amir's, it adds fsync=relaxed. Only an overlay with that
>   syncs just what was written through it, everybody else syncs the whole
>   upper file system as before. fsync=auto, the default, picks relaxed if
>   the upper file system supports RENAME_WHITEOUT. (Amir)
> - Dropped the fake inodes and the calls in the ovl_do_*() helpers,
>   patches 7 to 9 of v2. Upper inodes without an overlay inode are left
>   to ->sync_fs() and the block device, like in v1. (Amir)
>
> v1->v2:
> - Dropped the superblock flag and the ext4, xfs and btrfs patches. This
>   is all in overlayfs now and works the same on any upper file system.
>   (Christoph, Amir)
> - Upper inodes without an overlay inode get a fake one that goes on the
>   dirty list, patches 7 to 9.
> - The periodic flusher pass doesn't write the upper inode any more, and
>   syncfs() doesn't make ext4 commit per inode.
> - Record write errors in s_wb_err of the overlay.
> - Make the root inode appear hashed so it can be on the dirty list,
>   don't mark unhashed inodes dirty, read i_writecount with acquire.
>   (Sashiko)
> - Left alone: dirtied_time_when isn't set for an overlay inode that gets
>   I_DIRTY_TIME from ->write_inode(). It only matters for the 12 hour
>   expiry, where the upper file system writes the timestamps on its own.
>   (Sashiko)
> - Rewrote my commit messages. (Christoph)
>
> What it does now:
>
> sync_filesystem() writes every inode on a dirty list, calls ->sync_fs()
> and writes the block device. Only the first step is expensive, because
> of other people's data. With fsync=relaxed overlayfs does that step only
> for the upper inodes of the overlay inodes in its set, ->sync_fs()
> without wait starts them and with wait waits for them, and does the
> other two steps as they are.
>
> Whiteouts and the work directories have no overlay inode, so nobody
> writes those inodes. That's fine if ->sync_fs() or the block device gets
> all metadata to disk, which is what relaxed asks of the upper file
> system.
>
> Things I know about:
>
>  - ext4 without a journal supports RENAME_WHITEOUT, so it gets relaxed.
>    It's fine there, its metadata is in the buffers of the block device,
>    which we write. I tested it like the others.
>  - f2fs and ubifs get relaxed as well and I didn't test them. f2fs
>    writes its dirty inodes in the checkpoint. ubifs_sync_fs() doesn't,
>    but the only thing overlayfs leaves dirty there on an inode it
>    doesn't track is the chmod of the work directory to 0 at mount time,
>    right after creating it with mode 0. fsync=on is there for anybody
>    who needs more.
>  - A default mount that gets relaxed shows that in /proc/mounts.
>  - A tracked overlay inode, its upper inode and the dentry of that can't
>    be reclaimed while the upper inode is dirty, like a dirty inode of
>    any file system. After that they stay until a sync or the shrinker
>    looks. The shrinker only queues a work item that does the looking.
>    The reference it drops can be the last one to an unlinked upper
>    inode, and file systems allocate memory under locks that deleting a
>    file needs. Deferred timestamps count as dirty, the work item hands
>    them to the upper file system.
>  - When a sync reports an error it marks it as seen in wb_err of the
>    upper mapping, like an fsync() does. A file that is opened on the
>    upper inode after that doesn't get it any more.
>  - syncfs() looks at every inode in the set, also at those that the
>    upper file system has written by now.
>  - A file that is open for writing stays in the set, it may be mapped.
>    That doesn't cover pages pinned through a mapping. A direct read into
>    one that completes after the last munmap() and close() dirties them
>    when the inode may have left the set.
>  - An atime update puts the inode into the set as well.
>  - A read-only remount of the overlay syncs it, so it now fails once
>    with the error of writeback that has failed earlier.
>  - The error reporting has gaps, in all of them syncfs() returns 0 like
>    it does today. With the upper file system frozen or read-only we
>    fall back to syncing all of it and don't look at the inodes, so an
>    error is reported by the next syncfs() that can. btrfs records an
>    error after it has ended writeback, which can be after we looked.
>  - Data that an earlier mount of the same upper directory left dirty,
>    with fsync=off for example, isn't written by a syncfs() of a later
>    mount. Today it is, along with everything else.
>  - Every inode of a relaxed overlay gets a slot in the xarray when it's
>    allocated, also those that are never modified, so that tracking one
>    can't fail. The index is the address of the inode. i_ino isn't unique
>    in an overlay and xa_alloc() needs one of the three marks. All
>    overlays allocate from one slab cache, so where their inodes are
>    mixed a slot can cost a node of the xarray or more. With 64 overlays
>    of 2000 files, same debug kernel as below, it's 21 bytes an inode if
>    they're walked one after the other, 89 if all at once, 489 if they
>    take turns entry by entry. Plus about 6K for every overlay.
>  - If the origin of a copied up file can't be decoded, like in a user
>    namespace, a later lookup makes a second overlay inode for the same
>    upper inode. If the first one is tracked it pins the upper inode
>    after that was unlinked, until a sync or the shrinker looks.
>
> Tested like before, see the v1 mail below, now also with ext2 and ext4
> without a journal as the upper file system. New tests for memory
> pressure, a frozen and a read-only upper file system, O_TMPFILE, write
> errors, and syncfs() next to tasks that create files and next to other
> syncfs() calls. Everything that was changed through the overlay is in
> the copy of the block device and the 192M of the neighbours aren't,
> except with ext2, which doesn't get the new mode by default.
>
> To see that the tests can fail I broke the series in 12 places, a line
> or two each, and all of them are caught.
>
> With KASAN and lockdep on, ext4, xfs, btrfs and ext4 without a journal,
> one sample each, without and with the series:
>
>   syncfs(), the neighbours' 192M dirty         258-305ms    44-50ms
>   unmount, the same                            147-273ms    37-43ms
>   syncfs(), the 192M under writeback [4]         42-58ms  138-171ms
>   syncfs(), 4000 inodes in the set, dirty       52-153ms   71-207ms
>   syncfs(), 10000 inodes in the set, clean       55-57ms  144-146ms
>
> So it isn't free. Walking a tree takes 0 to 3% more system time, which
> is what it varies by without the series.
>
> fstests with -overlay, the auto groups of generic and overlay, on ext4
> and on xfs: nothing that passes without the series fails with it, and
> the run time is within 0.3%.
>
> [2] https://lore.kernel.org/r/20261027-b4-ext2-dirsync-v1-1-3dcd012d518d@toxicpanda.com
> [3] https://lore.kernel.org/r/99629e3277656888cbe84bf7490c5e54.josef@toxicpanda.com
> [4] Not btrfs, nothing was in flight there without the series.
>
> --- Original email (v1) ---
>
> Hello,
>
> This picks up Chengguang Xu's "implement containerized syncfs for
> overlayfs", last posted as RFC V6 in November 2021 [1].
>
> syncfs() on an overlay and unmounting one call sync_filesystem() on the
> upper file system, so they write every dirty inode on it no matter who
> dirtied it. We run a lot of containers on one upper file system, and
> stopping one of them writes out the dirty data of all the others. That
> can take a very long time, and s_umount of the upper file system is held
> the whole time, so anything that wants it exclusively queues up and then
> everybody else queues up behind that.
>
> With this an overlay only writes the upper inodes that were changed
> through it, and then has the upper file system commit its metadata.
>
> The V6 discussion ended with a direction but no V7, so that's what I
> tried to follow:
>
>  - V6 kept every overlay inode with an upper inode dirty for good. Jan
>    and Amir didn't like the cost of walking them. Now an overlay inode
>    is marked dirty when the upper inode is changed, mostly from
>    ovl_copyattr() as Amir and Miklos suggested.
>  - It never looks at the dirty flags of the upper inode, Miklos pointed
>    out that they're cleared when writeback starts.
>  - ->write_inode() goes through write_inode_now() on the upper inode,
>    and keeps the overlay inode dirty while the upper inode is open for
>    writing, which covers shared mappings.
>  - V6 marked the inodes DONTCACHE, which Jan was worried about. This
>    goes back to the v5 patch that keeps dirty overlay inodes cached.
>
> One thing is new. V6 called wait_sb_inodes() on the upper file system to
> catch writeback that was started but not finished, which waits for
> everybody's writeback. Instead ->write_inode() keeps the overlay inode
> dirty while the upper inode has pages tagged dirty or under writeback.
> The tags don't have the problem the inode flags have, the writeback tag
> is set before the dirty tag is cleared. So a clean overlay inode means
> its data is on disk, and syncfs() only waits for its own inodes.
>
> That takes care of the data. Metadata is harder, a lot of what is
> changed on the upper file system has no overlay inode that could be on
> the dirty list: the root of the overlay isn't hashed, and whiteouts,
> index entries, the work directories and their parent have none. File
> systems that log their metadata don't care, ->sync_fs() commits it. ext2
> does, there a whiteout was gone after a crash. I first had overlayfs
> write all of those itself and kept finding more of them, the last one
> was the link count of the shared whiteout when a whiteout is removed.
>
> So patch 1 adds a superblock flag, SB_I_OVL_USE_SYNC_FS, by which a file
> system says that ->sync_fs() commits all of its metadata, and patches 2
> to 4 set it for ext4 with a journal, xfs and btrfs. Without the flag on
> the upper file system nothing changes, no inode is marked dirty and
> syncfs() and unmount call sync_filesystem() like they always did.
>
> Patches 5, 7, 8 and 10 are Chengguang's with changes noted in them,
> patches 6 and 9 are new. I dropped Jan's Reviewed-by tags from V6 since
> the patches changed.
>
> lazytime on the upper file system needs its own handling. The flusher
> leaves the timestamps in memory for up to 12 hours, so the overlay inode
> can't be clean, but it can't stay dirty and pinned for that long either.
> Patch 9 gives it I_DIRTY_TIME like the upper inode has, and forwards
> ->sync_lazytime() to the upper inode.
>
> Things I know about:
>
>  - An overlay inode that loses its last reference while it is dirty is
>    kept until it has been written and then sits on the LRU, like with
>    any other file system. Clean and unlinked ones are still freed right
>    away.
>  - Every overlay on such an upper file system gets a bdi.
>  - I haven't measured syncfs() with a very large number of dirty inodes.
>
> For testing I put several overlays on one upper file system and dirty
> 192M next to the overlay under test, through another overlay and
> directly. Then syncfs() or unmount the one overlay, copy the block
> device as it is at that moment, mount the copy, and check that
> everything that was changed through the overlay is in it and that the
> 192M aren't. That passes with ext4, xfs and btrfs as the upper file
> system. With KASAN and lockdep on, syncfs() goes from 239-284ms to
> 45-48ms and unmount from 126-234ms to 39-44ms. With ext2 and ext4
> without a journal the 192M are written as before.
>
> fstests with -overlay, the auto groups of generic and overlay, gives the
> same results with and without the series on ext4 and on xfs.
>
> [1] https://lore.kernel.org/r/20211122030038.1938875-1-cgxu519@mykernel.net
>
> Thanks,
>
> Josef
>
> ---
> Changes in v7:
> - Link to v6: https://patch.msgid.link/20261110-b4-ovl-containerized-sync-v6-0-43d1be16a4d6@toxicpanda.com
>
> Changes in v6:
> - Link to v5: https://patch.msgid.link/20261107-b4-ovl-containerized-sync-v5-0-d34b4ae57fdf@toxicpanda.com
>
> Changes in v5:
> - Link to v4: https://patch.msgid.link/20261106-b4-ovl-containerized-sync-v4-0-6ddde0b7dd8a@toxicpanda.com
>
> Changes in v4:
> - Link to v3: https://patch.msgid.link/20261104-b4-ovl-containerized-sync-v3-0-8daa20144f53@toxicpanda.com
>
> Changes in v3:
> - Link to v2: https://patch.msgid.link/20261029-b4-ovl-containerized-sync-v2-0-4e9f08e4a052@toxicpanda.com
>
> Changes in v2:
> - Link to v1: https://patch.msgid.link/20261027-b4-ovl-containerized-sync-v1-0-802e512379a6@toxicpanda.com
>
> ---
> Amir Goldstein (1):
>       ovl: expand fsync mount options
>
> Chengguang Xu (1):
>       ovl: implement containerized syncfs for overlayfs
>
> Josef Bacik (5):
>       ovl: no fsync=relaxed without data integrity
>       fs: add sync_fs_and_bdev()
>       fs: add write_inode_for_sync()
>       ovl: add a set of the inodes that a sync has to write
>       ovl: track an inode when it is modified
>
>  Documentation/filesystems/overlayfs.rst |  45 ++-
>  fs/fs-writeback.c                       |  36 ++
>  fs/overlayfs/Kconfig                    |  21 ++
>  fs/overlayfs/Makefile                   |   2 +-
>  fs/overlayfs/copy_up.c                  |   9 +
>  fs/overlayfs/dir.c                      |  11 +
>  fs/overlayfs/file.c                     |   7 +-
>  fs/overlayfs/inode.c                    |  27 +-
>  fs/overlayfs/namei.c                    |   9 +-
>  fs/overlayfs/overlayfs.h                |  39 ++-
>  fs/overlayfs/ovl_entry.h                |   9 +
>  fs/overlayfs/params.c                   |  34 +-
>  fs/overlayfs/params.h                   |   2 +
>  fs/overlayfs/readdir.c                  |   1 +
>  fs/overlayfs/super.c                    |  58 +++-
>  fs/overlayfs/sync.c                     | 591 ++++++++++++++++++++++++++++++++
>  fs/overlayfs/util.c                     |  16 +-
>  fs/sync.c                               |  52 ++-
>  include/linux/fs.h                      |   1 +
>  include/linux/pagemap.h                 |   1 +
>  20 files changed, 920 insertions(+), 51 deletions(-)
> ---
> base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
> change-id: 20261026-b4-ovl-containerized-sync-149bed1afe9a
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 3/7] fs: add sync_fs_and_bdev()
  2026-11-18 17:03 ` [PATCH v7 3/7] fs: add sync_fs_and_bdev() Josef Bacik
@ 2026-11-19  8:55   ` Jan Kara
  0 siblings, 0 replies; 14+ messages in thread
From: Jan Kara @ 2026-11-19  8:55 UTC (permalink / raw)
  To: Josef Bacik
  Cc: Miklos Szeredi, Amir Goldstein, Christian Brauner,
	Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel

On Wed 18-11-26 17:03:26, Josef Bacik wrote:
> sync_filesystem() writes the inodes, calls ->sync_fs() and writes the block
> device, first without waiting and then with.
> 
> overlayfs is about to write the inodes of its upper filesystem by itself,
> only those that were modified through the overlay, and needs the other two
> steps as they are.  Move them to a helper so that overlayfs does not have
> to know what they are.
> 
> sync_filesystem() calls the inner helper, so it still tests for a read-only
> filesystem and for s_umount once.  The exported function does both tests
> for its callers.
> 
> Nothing calls the exported function yet, no functional change intended.
> 
> Suggested-by: Christian Brauner <brauner@kernel.org>
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>

Looks good. Feel free to add:

Reviewed-by: Jan Kara <jack@suse.cz>

								Honza

> ---
>  fs/sync.c          | 52 +++++++++++++++++++++++++++++++++++++++-------------
>  include/linux/fs.h |  1 +
>  2 files changed, 40 insertions(+), 13 deletions(-)
> 
> diff --git a/fs/sync.c b/fs/sync.c
> index 4a84dd837b86..9adc7076c24d 100644
> --- a/fs/sync.c
> +++ b/fs/sync.c
> @@ -22,6 +22,42 @@
>  #define VALID_FLAGS (SYNC_FILE_RANGE_WAIT_BEFORE|SYNC_FILE_RANGE_WRITE| \
>  			SYNC_FILE_RANGE_WAIT_AFTER)
>  
> +static int __sync_fs_and_bdev(struct super_block *sb, bool wait)
> +{
> +	int ret;
> +
> +	if (sb->s_op->sync_fs) {
> +		ret = sb->s_op->sync_fs(sb, wait);
> +		if (ret)
> +			return ret;
> +	}
> +	if (!wait)
> +		return sync_blockdev_nowait(sb->s_bdev);
> +	return sync_blockdev(sb->s_bdev);
> +}
> +
> +/**
> + * sync_fs_and_bdev - finish a sync whose inodes the caller has written
> + * @sb: the superblock, the caller holds s_umount
> + * @wait: wait for the I/O to complete
> + *
> + * These are the steps of sync_filesystem() that follow the writing of the
> + * inodes.  This is for a caller that has already written the inodes it cares
> + * about.  Like sync_filesystem() it does nothing on a read-only filesystem.
> + *
> + * Return: 0 on success, a negative error code otherwise.
> + */
> +int sync_fs_and_bdev(struct super_block *sb, bool wait)
> +{
> +	WARN_ON(!rwsem_is_locked(&sb->s_umount));
> +
> +	if (sb_rdonly(sb))
> +		return 0;
> +
> +	return __sync_fs_and_bdev(sb, wait);
> +}
> +EXPORT_SYMBOL_GPL(sync_fs_and_bdev);
> +
>  /*
>   * Write out and wait upon all dirty data associated with this
>   * superblock.  Filesystem data as well as the underlying block
> @@ -29,7 +65,7 @@
>   */
>  int sync_filesystem(struct super_block *sb)
>  {
> -	int ret = 0;
> +	int ret;
>  
>  	/*
>  	 * We need to be protected against the filesystem going from
> @@ -52,22 +88,12 @@ int sync_filesystem(struct super_block *sb)
>  	 * at a time.
>  	 */
>  	writeback_inodes_sb(sb, WB_REASON_SYNC);
> -	if (sb->s_op->sync_fs) {
> -		ret = sb->s_op->sync_fs(sb, 0);
> -		if (ret)
> -			return ret;
> -	}
> -	ret = sync_blockdev_nowait(sb->s_bdev);
> +	ret = __sync_fs_and_bdev(sb, false);
>  	if (ret)
>  		return ret;
>  
>  	sync_inodes_sb(sb);
> -	if (sb->s_op->sync_fs) {
> -		ret = sb->s_op->sync_fs(sb, 1);
> -		if (ret)
> -			return ret;
> -	}
> -	return sync_blockdev(sb->s_bdev);
> +	return __sync_fs_and_bdev(sb, true);
>  }
>  EXPORT_SYMBOL(sync_filesystem);
>  
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index 99caef43ff3c..2b016d9dfb88 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -2619,6 +2619,7 @@ extern void __init vfs_caches_init(void);
>  
>  void emergency_thaw_all(void);
>  extern int sync_filesystem(struct super_block *);
> +int sync_fs_and_bdev(struct super_block *sb, bool wait);
>  extern const struct file_operations def_blk_fops;
>  extern const struct file_operations def_chr_fops;
>  
> 
> -- 
> 2.55.0
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 4/7] fs: add write_inode_for_sync()
  2026-11-18 17:03 ` [PATCH v7 4/7] fs: add write_inode_for_sync() Josef Bacik
@ 2026-11-19  9:14   ` Jan Kara
  2026-11-19  9:47     ` OGAWA Hirofumi
  0 siblings, 1 reply; 14+ messages in thread
From: Jan Kara @ 2026-11-19  9:14 UTC (permalink / raw)
  To: Josef Bacik
  Cc: Miklos Szeredi, Amir Goldstein, Christian Brauner,
	Alexander Viro, Jan Kara, Christoph Hellwig, Chris Mason,
	Chengguang Xu, linux-unionfs, linux-fsdevel, linux-kernel,
	OGAWA Hirofumi

On Wed 18-11-26 17:03:27, Josef Bacik wrote:
> sync_inodes_sb() has the flusher write every dirty inode of a superblock
> with WB_SYNC_ALL and for_sync set.  That writes the pages without waiting
> for them, hands over the timestamps that lazytime has deferred while it
> holds I_SYNC, and calls ->write_inode() in a way that tells the filesystem
> that ->sync_fs() follows, so ext4 does not commit a transaction for every
> inode.
> 
> overlayfs is about to sync some of the inodes of its upper filesystem, and
> none of the exported functions does that for one inode.  write_inode_now()
> and sync_inode_metadata() either do not wait for somebody else who is
> writing the inode and leave deferred timestamps alone, or they wait for the
> pages, clear the error flags of the mapping and make ext4 commit.
> 
> Add a function that writes one inode the way a sync does.
> 
> Nothing calls it yet.
> 
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>

Looks good. Feel free to add:

Reviewed-by: Jan Kara <jack@suse.cz>

Actually Ogawa wanted the very same function for FAT flushing of dirty data
on close so it looks like a generally useful primitive to have. Just the
description of the function is really hard to parse (at least to me as a
non-native speaker). 

> +/**
> + * write_inode_for_sync - write an inode like a sync of its filesystem does
> + * @inode: inode to write to disk
> + *
> + * This does for one inode what sync_inodes_sb() has the flusher do for every
> + * dirty inode of a superblock.  It writes all dirty pages, the timestamps that
> + * lazytime has deferred and the inode.  It waits for somebody else who is
> + * writing the inode, but not for the pages it writes, and it does not ask the
> + * filesystem to commit anything.
> + *
> + * This is for a caller that syncs some of the inodes of a filesystem.  It has
> + * to wait for the pages with filemap_fdatawait_keep_errors(), like
> + * wait_sb_inodes() does, and to call sync_fs_and_bdev() in the end.

I'd write:
This writes out inode like e.g. sync(2) would do it. On high level that
means we make sure anything dirty is submitted to the storage but we don't
wait for any completion. In particular, it means we wait if there's
somebody else submitting writeback for the inode (I_SYNC state set), we
make sure all dirty pages are submitted for writeback waiting for locks,
and we treat the inode as dirty if it has dirty timestamps. However we
don't wait for any IO to complete and we don't persist other inode-related
metadata through ->sync_inode_metadata(). If the caller wants data
persistency guarantee, it has to wait for IO completion (e.g. using
filemap_fdatawait_keep_errors()) and also persist common filesystem
metadata using sync_fs_and_bdev().

> + *
> + * The caller must have a ref on the inode.
> + *
> + * Return: 0 on success, a negative error code otherwise.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH v7 4/7] fs: add write_inode_for_sync()
  2026-11-19  9:14   ` Jan Kara
@ 2026-11-19  9:47     ` OGAWA Hirofumi
  0 siblings, 0 replies; 14+ messages in thread
From: OGAWA Hirofumi @ 2026-11-19  9:47 UTC (permalink / raw)
  To: Jan Kara
  Cc: Josef Bacik, Miklos Szeredi, Amir Goldstein, Christian Brauner,
	Alexander Viro, Christoph Hellwig, Chris Mason, Chengguang Xu,
	linux-unionfs, linux-fsdevel, linux-kernel

Jan Kara <jack@suse.cz> writes:

> On Wed 18-11-26 17:03:27, Josef Bacik wrote:
>> sync_inodes_sb() has the flusher write every dirty inode of a superblock
>> with WB_SYNC_ALL and for_sync set.  That writes the pages without waiting
>> for them, hands over the timestamps that lazytime has deferred while it
>> holds I_SYNC, and calls ->write_inode() in a way that tells the filesystem
>> that ->sync_fs() follows, so ext4 does not commit a transaction for every
>> inode.
>> 
>> overlayfs is about to sync some of the inodes of its upper filesystem, and
>> none of the exported functions does that for one inode.  write_inode_now()
>> and sync_inode_metadata() either do not wait for somebody else who is
>> writing the inode and leave deferred timestamps alone, or they wait for the
>> pages, clear the error flags of the mapping and make ext4 commit.
>> 
>> Add a function that writes one inode the way a sync does.
>> 
>> Nothing calls it yet.
>> 
>> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
>
> Looks good. Feel free to add:
>
> Reviewed-by: Jan Kara <jack@suse.cz>
>
> Actually Ogawa wanted the very same function for FAT flushing of dirty data
> on close so it looks like a generally useful primitive to have. Just the
> description of the function is really hard to parse (at least to me as a
> non-native speaker). 
>
>> +/**
>> + * write_inode_for_sync - write an inode like a sync of its filesystem does
>> + * @inode: inode to write to disk
>> + *
>> + * This does for one inode what sync_inodes_sb() has the flusher do for every
>> + * dirty inode of a superblock.  It writes all dirty pages, the timestamps that
>> + * lazytime has deferred and the inode.  It waits for somebody else who is
>> + * writing the inode, but not for the pages it writes, and it does not ask the
>> + * filesystem to commit anything.
>> + *
>> + * This is for a caller that syncs some of the inodes of a filesystem.  It has
>> + * to wait for the pages with filemap_fdatawait_keep_errors(), like
>> + * wait_sb_inodes() does, and to call sync_fs_and_bdev() in the end.
>
> I'd write:
> This writes out inode like e.g. sync(2) would do it. On high level that
> means we make sure anything dirty is submitted to the storage but we don't
> wait for any completion. In particular, it means we wait if there's
> somebody else submitting writeback for the inode (I_SYNC state set), we
> make sure all dirty pages are submitted for writeback waiting for locks,
> and we treat the inode as dirty if it has dirty timestamps. However we
> don't wait for any IO to complete and we don't persist other inode-related
> metadata through ->sync_inode_metadata(). If the caller wants data
> persistency guarantee, it has to wait for IO completion (e.g. using
> filemap_fdatawait_keep_errors()) and also persist common filesystem
> metadata using sync_fs_and_bdev().

Looks good (except the name of function though). What I want is, not
emulate sync, it is flushing dirty stuff. Now both are same usage
though, may not be same by future changes.

Thanks.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2026-11-19  9:47 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-18 17:03 [PATCH v7 0/7] ovl: only sync what was written through the overlay Josef Bacik
2026-11-18 17:03 ` [PATCH v7 1/7] ovl: expand fsync mount options Josef Bacik
2026-11-18 17:03 ` [PATCH v7 2/7] ovl: no fsync=relaxed without data integrity Josef Bacik
2026-11-18 18:00   ` Amir Goldstein
2026-11-18 17:03 ` [PATCH v7 3/7] fs: add sync_fs_and_bdev() Josef Bacik
2026-11-19  8:55   ` Jan Kara
2026-11-18 17:03 ` [PATCH v7 4/7] fs: add write_inode_for_sync() Josef Bacik
2026-11-19  9:14   ` Jan Kara
2026-11-19  9:47     ` OGAWA Hirofumi
2026-11-18 17:03 ` [PATCH v7 5/7] ovl: add a set of the inodes that a sync has to write Josef Bacik
2026-11-18 17:03 ` [PATCH v7 6/7] ovl: track an inode when it is modified Josef Bacik
2026-11-18 17:03 ` [PATCH v7 7/7] ovl: implement containerized syncfs for overlayfs Josef Bacik
2026-11-18 18:09   ` Amir Goldstein
2026-11-18 18:14 ` [PATCH v7 0/7] ovl: only sync what was written through the overlay Amir Goldstein

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®