Re: [PATCH 2/2] xfs: skip the fsync flush when a later flush has completed

[Brian Foster <bfoster@redhat.com>]

On Thu, Nov 05, 2026 at 11:49:13AM +0900, Daejun Park via B4 Relay wrote:
> From: Daejun Park <daejun7.park@samsung.com>
> 
> Since commit 47c7d0b19502 ("xfs: fix incorrect log_flushed on
> fsync"), fsync sends its own cache flush whenever another task wrote
> out the iclog it waited for. With concurrent fsyncs, these flushes can
> limit throughput.
> 
> Skip the flush when a flush submitted after the data writes completed
> has finished. Number the fsync's own flush too, so that concurrent
> fsyncs can share it. DAX does not use the new skip, because DAX writes
> do not go through block I/O.
> 
> Suggested-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Daejun Park <daejun7.park@samsung.com>
> ---
> 16 fio jobs of 4k append + fsync, QEMU guest on a 990 PRO without
> power loss protection (virtio-blk, cache=none): 809 -> 4185 fsyncs/s,
> mean fsync latency 19.8 -> 3.8 ms.
> 

Can you include this in the commit log so we have a record of the perf
impact?

> dm-log-writes with 8 fsync writers: no fsync-ed block was lost in 528
> replays, while at least 33257 of 48048 fsyncs skipped their flush
> through the new check. Taking the snapshot before the writeback loses
> 15 of 528. The test needs "dm-log-writes: log earlier flushes' writes
> when a later flush completes", sent to dm-devel.
> ---
>  fs/xfs/xfs_buf.c  | 26 ++++++++++++++++++++++++++
>  fs/xfs/xfs_buf.h  | 19 +++++++++++++++++++
>  fs/xfs/xfs_file.c | 21 +++++++++++++++------
>  3 files changed, 60 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c
> index 9025b177931..6b545d79a41 100644
> --- a/fs/xfs/xfs_buf.c
> +++ b/fs/xfs/xfs_buf.c
> @@ -1785,6 +1785,32 @@ xfs_buftarg_flush_done(
>  		;
>  }
>  
> +/*
> + * Return true if a flush numbered above @seq has completed. All writes that
> + * completed before @seq was read are then on stable storage.
> + */

I think it would be helpful if the comment here and perhaps above
xfs_buftarg_flush_seq() referred to the submit/completion ordering logic
being applied. Maybe even rename seq -> submit_seq as well to make it
more clear. For example, something like:

"Return true if a flush has been submitted and completed since
@submit_seq was sampled. If so, this guarantees writes completed before
@submit_seq are flushed to disk."

But feel free to change that.

> +bool
> +xfs_buftarg_flushed_since(
> +	struct xfs_buftarg	*btp,
> +	s64			seq)
> +{
> +	return atomic64_read(&btp->bt_flush_completed) > seq;
> +}
> +
> +/* Flush the device cache and record it. */
> +int
> +xfs_buftarg_flush(
> +	struct xfs_buftarg	*btp)
> +{
> +	s64			seq = xfs_buftarg_flush_start(btp);
> +	int			error;
> +
> +	error = blkdev_issue_flush(btp->bt_bdev);
> +	if (!error)
> +		xfs_buftarg_flush_done(btp, seq);
> +	return error;
> +}
> +
>  struct xfs_buftarg *
>  xfs_alloc_buftarg(
>  	struct xfs_mount	*mp,
...
> diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c
> index 396aca8c66a..a98361e9ead 100644
> --- a/fs/xfs/xfs_file.c
> +++ b/fs/xfs/xfs_file.c
> @@ -87,7 +87,9 @@ xfs_file_fsync(
>  	int			datasync)
>  {
>  	struct xfs_inode	*ip = XFS_I(file->f_mapping->host);
> +	struct xfs_buftarg	*btp = xfs_inode_buftarg(ip);
>  	int			log_flushed = 0;
> +	s64			flush_seq;
>  	int			error;
>  
>  	trace_xfs_file_fsync(ip);
> @@ -95,6 +97,7 @@ xfs_file_fsync(
>  	error = file_write_and_wait_range(file, start, end);
>  	if (error)
>  		return error;
> +	flush_seq = xfs_buftarg_flush_seq(btp);
>  

Ok, so I think I follow the logic. We write the file, sample the submit
sequence (after write completes), then can optimize away our own flush
if we can confirm that a flush submitted after write completion has
already completed. Right?

That kind of makes me wonder whether we could do something like "flush
or wait on any flush submitted after this seq," but maybe that's
something for another day. ;)

>  	if (xfs_is_shutdown(ip->i_mount))
>  		return -EIO;
> @@ -115,16 +118,22 @@ xfs_file_fsync(
>  	}
>  
>  	/*
> -	 * If the log force was a no-op, we may still need to flush the
> -	 * file data target cache here. This can happen for fdatasync/O_DSYNC
> -	 * when no metadata needed to be committed.

Did we intend to remove the comment on why we flush in the first place?

Brian

> +	 * Flush the file data target unless a flush submitted to it after the
> +	 * data writes above completed has already finished. That is usually
> +	 * the flush before an iclog written out by this log force or by
> +	 * another task, or another fsync's flush.
> +	 *
> +	 * DAX writes are CPU stores, not block I/O, and the check above
> +	 * relies on REQ_PREFLUSH covering completed block I/O. So DAX only
> +	 * skips the flush when this log force flushed, as before.
>  	 *
>  	 * Use the inode's actual file data target rather than assuming the
>  	 * main data device.
>  	 */
> -	if (!log_flushed)
> -		return blkdev_issue_flush(xfs_inode_buftarg(ip)->bt_bdev);
> -	return 0;
> +	if (log_flushed ||
> +	    (!IS_DAX(VFS_I(ip)) && xfs_buftarg_flushed_since(btp, flush_seq)))
> +		return 0;
> +	return xfs_buftarg_flush(btp);
>  }
>  
>  static int
> 
> -- 
> 2.43.0
> 
> 
>