mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Xiubo Li via B4 Relay <devnull+xiubo.li.clyso.com@kernel.org>
To: Ilya Dryomov <idryomov@gmail.com>,
	Alex Markuze <amarkuze@redhat.com>,
	 Viacheslav Dubeyko <slava@dubeyko.com>,
	 Milind Changire <mchangir@redhat.com>,
	Jeff Layton <jlayton@kernel.org>
Cc: ceph-devel@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Dan van der Ster <dan.vanderster@clyso.com>,
	Xiubo Li <xiubo.li@clyso.com>
Subject: [PATCH 0/3] ceph: fix the buffered read size metric and make the subvolume metrics per CPU
Date: Wed, 14 Oct 2026 01:20:39 +0200	[thread overview]
Message-ID: <20261014-ceph-metrics-percpu-v1-0-38c5a24fec6e@clyso.com> (raw)

Two problems in the client I/O metrics, reported by Dan van der Ster
while looking at IO500 runs on 144-core Grace clients.

Patch 1: metrics/size has reported 0 bytes for every page cache read
since buffered reads moved to CEPH_OSD_DATA_TYPE_ITER, because
finish_netfs_read() still reads osd_data->length, which only the page
vector type sets.

Patches 2-3: every OSD read and write completion updates the subvolume
metrics through state shared by the whole mount.  Even with subvolume
metrics disabled it bumped an atomic64 counter that nothing reads;
with them enabled it took a mount-wide spinlock and searched an rbtree.
Patch 2 drops the unused counters, patch 3 accumulates the counters in
a per-CPU slot and only folds them into the tree when the slot changes
subvolume or when the metrics are sent or dumped.

Tested on x86_64 against a main MDS with SUBVOLUME_METRICS: 16 jobs on
16 CPUs writing and reading two subvolumes and a plain directory.  The
read sizes are now reported, and the per-subvolume op and byte totals
summed over every metrics send match the I/O issued.

---
Xiubo Li (3):
      ceph: report the requested length in the netfs read metrics
      ceph: drop the unused subvolume metrics debug counters
      ceph: accumulate subvolume metrics per CPU

 fs/ceph/addr.c              |   2 +-
 fs/ceph/mds_client.c        |   6 +-
 fs/ceph/subvolume_metrics.c | 172 +++++++++++++++++++++++++++++++-------------
 fs/ceph/subvolume_metrics.h |  26 ++-----
 4 files changed, 133 insertions(+), 73 deletions(-)
---
base-commit: ed0502cb11f84fff29f374238e526ae94d9bdfde
change-id: 20261014-ceph-metrics-percpu-1cb5047f7150

Best regards,
--  
Xiubo Li <xiubo.li@clyso.com>



             reply	other threads:[~2026-10-13 23:20 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-13 23:20 Xiubo Li via B4 Relay [this message]
2026-10-13 23:20 ` [PATCH 1/3] ceph: report the requested length in the netfs read metrics Xiubo Li via B4 Relay
2026-10-13 23:20 ` [PATCH 2/3] ceph: drop the unused subvolume metrics debug counters Xiubo Li via B4 Relay
2026-10-13 23:20 ` [PATCH 3/3] ceph: accumulate subvolume metrics per CPU Xiubo Li via B4 Relay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261014-ceph-metrics-percpu-v1-0-38c5a24fec6e@clyso.com \
    --to=devnull+xiubo.li.clyso.com@kernel.org \
    --cc=amarkuze@redhat.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=dan.vanderster@clyso.com \
    --cc=idryomov@gmail.com \
    --cc=jlayton@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchangir@redhat.com \
    --cc=slava@dubeyko.com \
    --cc=xiubo.li@clyso.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®