From: Xiubo Li via B4 Relay <devnull+xiubo.li.clyso.com@kernel.org>
To: Ilya Dryomov <idryomov@gmail.com>,
Alex Markuze <amarkuze@redhat.com>,
Viacheslav Dubeyko <slava@dubeyko.com>,
Milind Changire <mchangir@redhat.com>,
Jeff Layton <jlayton@kernel.org>
Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org,
Dan van der Ster <dan.vanderster@clyso.com>,
Xiubo Li <xiubo.li@clyso.com>
Subject: [PATCH 0/3] ceph: fix the buffered read size metric and make the subvolume metrics per CPU
Date: Wed, 14 Oct 2026 01:20:39 +0200 [thread overview]
Message-ID: <20261014-ceph-metrics-percpu-v1-0-38c5a24fec6e@clyso.com> (raw)
Two problems in the client I/O metrics, reported by Dan van der Ster
while looking at IO500 runs on 144-core Grace clients.
Patch 1: metrics/size has reported 0 bytes for every page cache read
since buffered reads moved to CEPH_OSD_DATA_TYPE_ITER, because
finish_netfs_read() still reads osd_data->length, which only the page
vector type sets.
Patches 2-3: every OSD read and write completion updates the subvolume
metrics through state shared by the whole mount. Even with subvolume
metrics disabled it bumped an atomic64 counter that nothing reads;
with them enabled it took a mount-wide spinlock and searched an rbtree.
Patch 2 drops the unused counters, patch 3 accumulates the counters in
a per-CPU slot and only folds them into the tree when the slot changes
subvolume or when the metrics are sent or dumped.
Tested on x86_64 against a main MDS with SUBVOLUME_METRICS: 16 jobs on
16 CPUs writing and reading two subvolumes and a plain directory. The
read sizes are now reported, and the per-subvolume op and byte totals
summed over every metrics send match the I/O issued.
---
Xiubo Li (3):
ceph: report the requested length in the netfs read metrics
ceph: drop the unused subvolume metrics debug counters
ceph: accumulate subvolume metrics per CPU
fs/ceph/addr.c | 2 +-
fs/ceph/mds_client.c | 6 +-
fs/ceph/subvolume_metrics.c | 172 +++++++++++++++++++++++++++++++-------------
fs/ceph/subvolume_metrics.h | 26 ++-----
4 files changed, 133 insertions(+), 73 deletions(-)
---
base-commit: ed0502cb11f84fff29f374238e526ae94d9bdfde
change-id: 20261014-ceph-metrics-percpu-1cb5047f7150
Best regards,
--
Xiubo Li <xiubo.li@clyso.com>
next reply other threads:[~2026-10-13 23:20 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-13 23:20 Xiubo Li via B4 Relay [this message]
2026-10-13 23:20 ` [PATCH 1/3] ceph: report the requested length in the netfs read metrics Xiubo Li via B4 Relay
2026-10-13 23:20 ` [PATCH 2/3] ceph: drop the unused subvolume metrics debug counters Xiubo Li via B4 Relay
2026-10-13 23:20 ` [PATCH 3/3] ceph: accumulate subvolume metrics per CPU Xiubo Li via B4 Relay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261014-ceph-metrics-percpu-v1-0-38c5a24fec6e@clyso.com \
--to=devnull+xiubo.li.clyso.com@kernel.org \
--cc=amarkuze@redhat.com \
--cc=ceph-devel@vger.kernel.org \
--cc=dan.vanderster@clyso.com \
--cc=idryomov@gmail.com \
--cc=jlayton@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mchangir@redhat.com \
--cc=slava@dubeyko.com \
--cc=xiubo.li@clyso.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®