mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Viacheslav Dubeyko <slava@dubeyko.com>
To: xiubo.li@clyso.com, Ilya Dryomov <idryomov@gmail.com>,
	Alex Markuze <amarkuze@redhat.com>
Cc: Jeff Layton <jlayton@kernel.org>,
	ceph-devel@vger.kernel.org,  linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] ceph: revalidate ki_pos for O_APPEND writes after cap acquisition
Date: Tue, 21 Jul 2026 10:23:11 -0700	[thread overview]
Message-ID: <c3e7be316a957f5363322991529c0c4a69937beb.camel@dubeyko.com> (raw)
In-Reply-To: <20260721-ceph-append-fix-v3-1-403d1b5204d3@clyso.com>

On Tue, 2026-07-21 at 13:06 +0800, Xiubo Li via B4 Relay wrote:
> From: Xiubo Li <xiubo.li@clyso.com>
> 
> For O_APPEND writes, ki_pos is set to the current EOF via
> generic_write_checks() after fetching i_size from the MDS.  However,
> ceph_get_caps() may need to wait for Fwx exclusive caps if the write
> extends the file (endoff > i_max_size).  While waiting for Fwx, the
> previous Fwx holder (another client) may have already extended the
> file.  When the MDS grants us Fwx, the cap grant message updates the
> local i_size, but ki_pos remains at the old EOF, causing the append
> write to land at a stale offset and overwrite data from the other
> client.
> 
> Fix by re-reading i_size_read(inode) after ceph_get_caps() returns.
> At this point we hold Fwx exclusive caps, no other client can modify
> the file, and i_size reflects the true EOF from the MDS cap grant.
> No extra MDS round-trip is needed.  Only adjust ki_pos when the EOF
> has actually changed.
> 
> After adjusting ki_pos forward, the write range [pos, pos+count) may
> now exceed the i_max_size that was validated by ceph_get_caps() for
> the old range.  Re-check against i_max_size and truncate the write
> if necessary to stay within the MDS-granted limit.
> 
> Fixes: 8e4473bb50a1 ("ceph: do not execute direct write in parallel
> if O_APPEND is specified")
> Link: https://tracker.ceph.com/issues/7333
> Signed-off-by: Xiubo Li <xiubo.li@clyso.com>
> ---
> Changes in v3:
> - Protect ci->i_max_size access with i_ceph_lock to avoid data race
> with handle_cap_grant() which updates i_max_size from the messenger
> thread.(Viacheslav Dubeyko)
> - Link to v2:
> https://patch.msgid.link/20260718-ceph-append-fix-v2-1-88dcc4f8371f@clyso.com
> 
> Changes in v2:
> - Re-check write range against i_max_size after adjusting ki_pos
> forward, and truncate if it exceeds the MDS-granted limit.(Viacheslav
> Dubeyko)
> - Link to v1:
> https://patch.msgid.link/20260717-ceph-append-fix-v1-1-c51907ac8e36@clyso.com
> ---
>  fs/ceph/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 48 insertions(+)
> 
> diff --git a/fs/ceph/file.c b/fs/ceph/file.c
> index 3823671ab95a..c26bd4fef7d3 100644
> --- a/fs/ceph/file.c
> +++ b/fs/ceph/file.c
> @@ -2489,6 +2489,54 @@ static ssize_t ceph_write_iter(struct kiocb
> *iocb, struct iov_iter *from)
>  	if (err < 0)
>  		goto out;
>  
> +	/*
> +	 * For O_APPEND writes we may have waited for Fwx exclusive
> caps
> +	 * while the previous Fwx holder (another client) extended
> the
> +	 * file.  i_size has been updated via the cap grant message
> from
> +	 * the MDS, but ki_pos is still the old EOF.  Re-read i_size
> here
> +	 * (no extra MDS round-trip needed) and adjust ki_pos to the
> true
> +	 * EOF.  Since we hold Fwx, no other client can change the
> file.
> +	 */
> +	if (iocb->ki_flags & IOCB_APPEND) {
> +		loff_t cur_eof = i_size_read(inode);
> +
> +		if (cur_eof != pos) {
> +			doutc(cl,
> +			      "%p %llx.%llx O_APPEND: pos adjusted
> %lld -> %lld\n",
> +			      inode, ceph_vinop(inode), pos,
> cur_eof);
> +			iocb->ki_pos = cur_eof;
> +			pos = cur_eof;
> +			if (pos >= limit) {
> +				err = -EFBIG;
> +				goto out_caps;
> +			}
> +			iov_iter_truncate(from, limit - pos);
> +			count = iov_iter_count(from);
> +
> +			/*
> +			 * ceph_get_caps() validated the old endoff
> +			 * against i_max_size; adjusting ki_pos
> forward
> +			 * may have shifted the write range beyond
> the
> +			 * granted max_size.  Re-check and truncate
> if
> +			 * necessary.
> +			 */
> +			spin_lock(&ci->i_ceph_lock);
> +			if (pos + count > (loff_t)ci->i_max_size) {
> +				loff_t max_size = ci->i_max_size;
> +
> +				spin_unlock(&ci->i_ceph_lock);
> +				if (pos >= max_size) {
> +					err = -EFBIG;
> +					goto out_caps;
> +				}
> +				iov_iter_truncate(from, max_size -
> pos);
> +				count = iov_iter_count(from);
> +			} else {
> +				spin_unlock(&ci->i_ceph_lock);
> +			}
> +		}
> +	}

Just comment... :) For my taste, it is better to take the i_max_size
under the lock and then to use it:

spin_lock(&ci->i_ceph_lock);
loff_t max_size = ci->i_max_size;
spin_unlock(&ci->i_ceph_lock);

But it is not critical remarks and I don't see any issue with the
current implementation.

Looks good.

Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>

Thanks,
Slava.

> +
>  	err = file_update_time(file);
>  	if (err)
>  		goto out_caps;
> 
> ---
> base-commit: afffcd98d2d22f2e7ebe42f71231d52d4eed8ef7
> change-id: 20260717-ceph-append-fix-9820e3b1a78c
> 
> Best regards,
> --  
> Xiubo Li <xiubo.li@clyso.com>
> 

  reply	other threads:[~2026-07-21 17:23 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-21  5:06 Xiubo Li via B4 Relay
2026-07-21 17:23 ` Viacheslav Dubeyko [this message]
2026-07-22  1:18   ` Xiubo Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c3e7be316a957f5363322991529c0c4a69937beb.camel@dubeyko.com \
    --to=slava@dubeyko.com \
    --cc=amarkuze@redhat.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=idryomov@gmail.com \
    --cc=jlayton@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=xiubo.li@clyso.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®