From: Ian Rogers <irogers@google.com>
To: irogers@google.com, acme@kernel.org, adrian.hunter@intel.com,
alice.mei.rogers@gmail.com, james.clark@linaro.org,
linux-perf-users@vger.kernel.org, namhyung@kernel.org
Cc: dapeng1.mi@linux.intel.com, leo.yan@linux.dev,
linux-kernel@vger.kernel.org, mingo@redhat.com,
peterz@infradead.org, tmricht@linux.ibm.com
Subject: [PATCH v1 27/49] perf python: Port rw-by-file from Perl to perf module
Date: Sat, 19 Sep 2026 22:21:19 -0700 [thread overview]
Message-ID: <a0b402bbda769485060a543060e543906bec06fd.1789880842.git.irogers@google.com> (raw)
In-Reply-To: <cover.1789880842.git.irogers@google.com>
Replace the legacy Perl script rw-by-file.pl with a standalone Python
script in tools/perf/python/rw-by-file.py using the perf Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Encapsulate per-file-descriptor read/write byte and call count
aggregation in an RwByFile class using perf.session and resolve thread
command names via session.find_thread(pid, sample_tid).
- Add argparse CLI support (-i/--input and target program filter) and
full type annotations.
Add a shell test
(test_rw_by_file_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/rw-by-file.py | 109 ++++++++++++++++++
.../tests/shell/test_rw_by_file_python.sh | 70 +++++++++++
2 files changed, 179 insertions(+)
create mode 100755 tools/perf/python/rw-by-file.py
create mode 100755 tools/perf/tests/shell/test_rw_by_file_python.sh
diff --git a/tools/perf/python/rw-by-file.py b/tools/perf/python/rw-by-file.py
new file mode 100755
index 000000000000..562ee7f7fd7e
--- /dev/null
+++ b/tools/perf/python/rw-by-file.py
@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""Display r/w activity for files read/written to for a given program."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional, Dict
+import perf
+
+class RwByFile:
+ """Tracks and displays read/write activity by file descriptor."""
+ def __init__(self, comm: str) -> None:
+ self.for_comm = comm
+ self.reads: Dict[int, Dict[str, int]] = defaultdict(
+ lambda: {"bytes_requested": 0, "total_reads": 0}
+ )
+ self.writes: Dict[int, Dict[str, int]] = defaultdict(
+ lambda: {"bytes_written": 0, "total_writes": 0}
+ )
+ self.unhandled: Dict[str, int] = defaultdict(int)
+ self.session: Optional[perf.session] = None
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events."""
+ raw_name = str(sample.evsel)
+ event_name = raw_name[6:-1] if raw_name.startswith("evsel(") else raw_name
+
+ pid = sample.sample_pid
+ assert self.session is not None
+ try:
+ thread = self.session.find_thread(pid, sample.sample_tid)
+ comm = (thread.comm() if thread else None) or "unknown"
+ except (TypeError, AttributeError):
+ comm = "unknown"
+
+ if comm != self.for_comm:
+ return
+
+ if event_name == "syscalls:sys_enter_read":
+ try:
+ fd = sample.fd
+ count = sample.count
+ self.reads[fd]["bytes_requested"] += count
+ self.reads[fd]["total_reads"] += 1
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ elif event_name == "syscalls:sys_enter_write":
+ try:
+ fd = sample.fd
+ count = sample.count
+ self.writes[fd]["bytes_written"] += count
+ self.writes[fd]["total_writes"] += 1
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ else:
+ self.unhandled[event_name] += 1
+
+ def print_totals(self) -> None:
+ """Print summary tables."""
+ print(f"file read counts for {self.for_comm}:\n")
+ print(f"{'fd':>6s} {'# reads':>10s} {'bytes_requested':>15s}")
+ print(f"{'-'*6} {'-'*10} {'-'*15}")
+
+ for fd, data in sorted(self.reads.items(),
+ key=lambda kv: kv[1]["bytes_requested"], reverse=True):
+ print(f"{fd:6d} {data['total_reads']:10d} {data['bytes_requested']:15d}")
+
+ print(f"\nfile write counts for {self.for_comm}:\n")
+ print(f"{'fd':>6s} {'# writes':>10s} {'bytes_written':>15s}")
+ print(f"{'-'*6} {'-'*10} {'-'*15}")
+
+ for fd, data in sorted(self.writes.items(),
+ key=lambda kv: kv[1]["bytes_written"], reverse=True):
+ print(f"{fd:6d} {data['total_writes']:10d} {data['bytes_written']:15d}")
+
+ if self.unhandled:
+ print("\nunhandled events:\n")
+ print(f"{'event':<40s} {'count':>10s}")
+ print(f"{'-'*40} {'-'*10}")
+ for event_name, count in self.unhandled.items():
+ print(f"{event_name:<40s} {count:10d}")
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ try:
+ self.session.process_events()
+ finally:
+ self.session = None
+ self.print_totals()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace r/w activity by file")
+ parser.add_argument("comm", help="Filter by command name")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ args = parser.parse_args()
+
+ analyzer = RwByFile(args.comm)
+ try:
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_rw_by_file_python.sh b/tools/perf/tests/shell/test_rw_by_file_python.sh
new file mode 100755
index 000000000000..a305d5eded75
--- /dev/null
+++ b/tools/perf/tests/shell/test_rw_by_file_python.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# rw-by-file python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/rw-by-file.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, rw-by-file.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing rw-by-file.py..."
+
+# Create a perf.data file. Try to get tracepoint data.
+if perf list | grep -q "syscalls:sys_enter_read"; then
+ perf record -e syscalls:sys_enter_read,syscalls:sys_enter_write -a -o "${temp_data}" \
+ -- dd if=/dev/urandom of=/dev/null bs=1M count=10 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no syscalls:sys_enter_read event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes - filtering for "dd" since that's what we ran
+if ! "$PYTHON" "$script_path" -i "${temp_data}" "dd" > "${temp_out}"; then
+ echo "rw-by-file.py test failed"
+ err=1
+else
+ if ! grep -E -q "^ *[0-9]+" "${temp_out}"; then
+ echo "Failed to find metric data rows"
+ err=1
+ else
+ echo "rw-by-file test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
next prev parent reply other threads:[~2026-09-20 5:23 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
2026-09-20 5:20 ` [PATCH v1 01/49] perf python: Update syscall format string to optional positional Ian Rogers
2026-09-20 5:20 ` [PATCH v1 02/49] perf python: Update callchain stubs and session thread lookup Ian Rogers
2026-09-20 5:20 ` [PATCH v1 03/49] perf python: Clean up pylint warnings in ilist.py Ian Rogers
2026-09-20 5:20 ` [PATCH v1 04/49] perf python: Clean up pylint warnings in treport.py Ian Rogers
2026-09-20 5:20 ` [PATCH v1 05/49] perf python: Clean up pylint warnings in tracepoint.py Ian Rogers
2026-09-20 5:20 ` [PATCH v1 06/49] perf python: Clean up pylint warnings in twatch.py Ian Rogers
2026-09-20 5:20 ` [PATCH v1 07/49] perf python: Improve perf script -l descriptions Ian Rogers
2026-09-20 5:21 ` [PATCH v1 08/49] perf python: Expose addr location, transaction, and context_switch Ian Rogers
2026-09-20 5:21 ` [PATCH v1 09/49] perf python: Add Intel PT call_return and itrace capability Ian Rogers
2026-09-20 5:21 ` [PATCH v1 10/49] perf python: Allow KeyboardInterrupt to propagate in LiveSession Ian Rogers
2026-09-20 5:21 ` [PATCH v1 11/49] perf pmu-events: Clean up mypy and pylint issues Ian Rogers
2026-09-20 5:21 ` [PATCH v1 12/49] perf test: Clean up mypy and pylint issues in shell test libraries Ian Rogers
2026-09-20 5:21 ` [PATCH v1 13/49] perf build: Make mypy build test opt-out (NO_MYPY=1) Ian Rogers
2026-09-20 5:21 ` [PATCH v1 14/49] perf build: Make pylint build test opt-out (NO_PYLINT=1) Ian Rogers
2026-09-20 5:21 ` [PATCH v1 15/49] perf Makefile: Install standalone Python scripts during transition Ian Rogers
2026-09-20 5:21 ` [PATCH v1 16/49] perf python: Port stat-cpi to perf module Ian Rogers
2026-09-20 5:21 ` [PATCH v1 17/49] perf python: Port mem-phys-addr " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 18/49] perf python: Port stackcollapse " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 19/49] perf python: Port flamegraph " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 20/49] perf python: Port gecko " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 21/49] perf python: Port event_analyzing_sample " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 22/49] perf python: Port syscall-counts " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 23/49] perf python: Port syscall-counts-by-pid " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 24/49] perf python: Port failed-syscalls-by-pid " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 25/49] perf python: Port failed-syscalls from Perl " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 26/49] perf python: Port sctop " Ian Rogers
2026-09-20 5:21 ` Ian Rogers [this message]
2026-09-20 5:21 ` [PATCH v1 28/49] perf python: Port rw-by-pid from Perl " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 29/49] perf python: Port rwtop " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 30/49] perf python: Port futex-contention " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 31/49] perf python: Port task-analyzer " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 32/49] perf python: Port sched-migration and SchedGui " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 33/49] perf python: Port wakeup-latency from Perl " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 34/49] perf python: Port compaction-times " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 35/49] perf python: Port net_dropmonitor " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 36/49] perf python: Port netdev-times " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 37/49] perf python: Port check-perf-trace " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 38/49] perf python: Port arm-cs-trace-disasm " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 39/49] perf python: Port powerpc-hcalls " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 40/49] perf python: Port intel-pt-events and libxed " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 41/49] perf test: Migrate Intel PT virtual LBR test to Python API Ian Rogers
2026-09-20 5:21 ` [PATCH v1 42/49] perf python: Port export-to-sqlite to perf module Ian Rogers
2026-09-20 5:21 ` [PATCH v1 43/49] perf python: Port export-to-postgresql " Ian Rogers
2026-09-20 5:21 ` [PATCH v1 44/49] perf python: Move and clean up exported-sql-viewer.py Ian Rogers
2026-09-20 5:21 ` [PATCH v1 45/49] perf python: Move and clean up parallel-perf.py Ian Rogers
2026-09-20 5:21 ` [PATCH v1 46/49] perf: Remove libpython support and legacy Python scripts Ian Rogers
2026-09-20 5:21 ` [PATCH v1 47/49] perf Makefile: Update Python script installation path Ian Rogers
2026-09-20 5:21 ` [PATCH v1 48/49] perf script: Support standalone scripts and remove embedded scripting Ian Rogers
2026-09-20 5:21 ` [PATCH v1 49/49] perf Documentation: Update for standalone Python scripts Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a0b402bbda769485060a543060e543906bec06fd.1789880842.git.irogers@google.com \
--to=irogers@google.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alice.mei.rogers@gmail.com \
--cc=dapeng1.mi@linux.intel.com \
--cc=james.clark@linaro.org \
--cc=leo.yan@linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=tmricht@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®