From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f72.google.com (mail-dl1-f72.google.com [74.125.82.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02D4C51AFC4 for ; Wed, 23 Sep 2026 18:13:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.72 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790187237; cv=none; b=f0SdrjHrjR2tvsGD5w7ORBSajl3heYArTFIyEQj3AT6x1tG/n0FkD6LDE7u6bTMyMCtRu6ThLlpvDuBrZLFnmgRyQD9GiZDc9ZFPyMv0t5DLGFiv3ZoQED3sx075FmVrOGdbRNeoMA42KLIjykuMAEB1vADOa5yozr6g2BmJnKE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790187237; c=relaxed/simple; bh=g12EZvZekt6R/6wyFoyXy44IWH+l5NQMFBBy3pRxGyc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=R+qXP+jvOGRhq2ENoUnopG2kUbarwi40tnyGB6w6X7E/z4wSzsdC7+H3nh1eXPhVFnu9Zu4BjhfVgoGQskH42k/Uk1ROdWACpVg8agOx7mO6CufrsM3ERm4+L2KbUfxeyIOeE2qQRDib7MweqhtrvvNv07Dpdpk0Y2Yuz6LtnQk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vf3tPsXP; arc=none smtp.client-ip=74.125.82.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vf3tPsXP" Received: by mail-dl1-f72.google.com with SMTP id a92af1059eb24-14318be7d79so1363820c88.0 for ; Wed, 23 Sep 2026 11:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1790187222; x=1790792022; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=WKre0Si3CV+o1WG/2WLKOz8R0ZGc2BI5qcqtES/NRKo=; b=vf3tPsXPSniWsRyBBhDE+I2RPnc0SfdOpXqCDdbXAIRvdpNiIXMGbijU3Y2HO/qw51 /qe01YQvidWeC/tZ3Lm9S/52FSJmk4uLo3xbWu9rZcyHsv6s8ylSJsgUfHeqq3e+mqcB XtFfMUK3YGAkRVRIdD3gZINHkF6mynpTXuu9UexpE1kv5yODyA/WIcG4U3dTET/GWrT6 a9sUKcTj0H8ryCr0GcXwK3TUQHofkp9WHNTtsr25QPOd90Blow7zANshWelIQi6o11xP wafUYEqoZwgDPbYsbS/8AYi4tl3DGIKn/KvLppmHffZvjQfU5XUptU1eg9MnuZLfnujP HKuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790187222; x=1790792022; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=WKre0Si3CV+o1WG/2WLKOz8R0ZGc2BI5qcqtES/NRKo=; b=mrO/Fa5ql+zDyUbbzRbHQ7chVkbQ+KM6TlYzmAQ8z02m0/NbAQwXT4WBiXcYRkXZkm yMI6R87MVUDbvj1/ZtYMaY9vzreksca8YEErJ4ZdNIu1HTBQnkc31Peqyh6o1j0MMMu9 IMNUFXvd9qFdUDVy/kMdqo5UnanRuyx6CXqw0XmGhgz3kidkahvzr3raCZgWK8Sv9q6R ys0vabaulevxHEwSmTraBUhKMHJRaj2OUlHb00pulihUVPP2hev6vX3WSmn/73E7Sizg aFFIHeMx6pF8kA5/kd7KybdAQ1hL3wHSy+59j6RU7ClficSbBGpV6pSsfYj3M6gmHQ7q lXDQ== X-Forwarded-Encrypted: i=1; AKwUvBysbv1VdAMDvBOCylbbodvYb7Ttcxx4II9GRevhlk3mpgcjlVzTIVekdNnNpnidPZJwaWTOWYDE/WXchUM=@vger.kernel.org X-Gm-Message-State: AFuF++nD2kEgYyD8nWJ0XOKXGiRvNvX7kbjpRSUJuFfIAn4TLZveRdQe VUqy5ohGqM0pjLA6NQ6RlFsYw2Vg2UXTHrNubSPwANV6VuzOOLNo38UIRaeGgVfmJDFTTb4Bac6 XYFIG8IXXOA== X-Received: from dlbts8-n2.prod.google.com ([2002:a05:7022:b048:20b0:144:d7ab:e21b]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7022:1402:b0:143:2719:566c with SMTP id a92af1059eb24-144f9198b49mr3512891c88.40.1790187221594; Wed, 23 Sep 2026 11:13:41 -0700 (PDT) Date: Wed, 23 Sep 2026 11:11:41 -0700 In-Reply-To: <20260923181213.3032038-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260923181213.3032038-1-irogers@google.com> X-Mailer: git-send-email 2.56.0.rc1.310.g51773c2048-goog Message-ID: <20260923181213.3032038-19-irogers@google.com> Subject: [PATCH v3 18/49] perf python: Port stackcollapse to perf module From: Ian Rogers To: irogers@google.com, acme@kernel.org, alice.mei.rogers@gmail.com, namhyung@kernel.org Cc: adrian.hunter@intel.com, dapeng1.mi@linux.intel.com, james.clark@linaro.org, leo.yan@linux.dev, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, tmricht@linux.ibm.com Content-Type: text/plain; charset="UTF-8" Port stackcollapse.py from tools/perf/scripts/python/ to a standalone script in tools/perf/python/ refactored into a StackCollapseAnalyzer class. Improvements compared to the legacy script: - Traverse sample.callchain directly from perf.session without allocating per-event dictionaries, and fall back to sample.symbol when a sample has no callchain. - Replace deprecated optparse with argparse, adding -i/--input alongside --include-tid, --include-pid, --no-comm, --tidy-java, and --kernel. - Handle BrokenPipeError cleanly when output is piped into downstream tools (such as head or flamegraph.pl). Add a shell test (test_stackcollapse_python.sh) using a CPU workload (perf test -w noploop) to verify the standalone script. Assisted-by: Antigravity:gemini-3.1-pro Signed-off-by: Ian Rogers --- tools/perf/python/stackcollapse.py | 145 ++++++++++++++++++ .../tests/shell/test_stackcollapse_python.sh | 78 ++++++++++ 2 files changed, 223 insertions(+) create mode 100755 tools/perf/python/stackcollapse.py create mode 100755 tools/perf/tests/shell/test_stackcollapse_python.sh diff --git a/tools/perf/python/stackcollapse.py b/tools/perf/python/stackcollapse.py new file mode 100755 index 000000000000..0e8a65969db3 --- /dev/null +++ b/tools/perf/python/stackcollapse.py @@ -0,0 +1,145 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +""" +stackcollapse.py - format perf samples with one line per distinct call stack + +This script's output has two space-separated fields. The first is a semicolon +separated stack including the program name (from the "comm" field) and the +function names from the call stack. The second is a count: + + swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2 + +The file is sorted according to the first field. + +Ported from tools/perf/scripts/python/stackcollapse.py +""" +from __future__ import annotations + +import argparse +from collections import defaultdict +import os +import sys +import perf + + +class StackCollapseAnalyzer: + """Accumulates call stacks and prints them collapsed.""" + + def __init__(self, args: argparse.Namespace) -> None: + self.args = args + self.lines: dict[str, int] = defaultdict(int) + self.session: perf.session | None = None + + def tidy_function_name(self, sym: str, dso: str) -> str: + """Beautify function names based on options.""" + if sym is None: + sym = "[unknown]" + + sym = sym.replace(";", ":") + if self.args.tidy_java: + # Beautify Java signatures + sym = sym.replace("<", "") + sym = sym.replace(">", "") + if sym.startswith("L") and "/" in sym: + sym = sym[1:] + try: + sym = sym[:sym.index("(")] + except ValueError: + pass + + if self.args.annotate_kernel and dso == "[kernel.kallsyms]": + return sym + "_[k]" + return sym + + def process_event(self, sample: perf.sample_event) -> None: + """Collect call stack for each sample.""" + stack = [] + callchain = sample.callchain + if callchain is not None: + for node in callchain: + stack.append(self.tidy_function_name(node.symbol, node.dso)) + else: + # Fallback if no callchain + sym = (sample.symbol or '[unknown]') + dso = (sample.dso or '[unknown]') + stack.append(self.tidy_function_name(sym, dso)) + + if self.args.include_comm: + comm = "Unknown" + if self.session is not None: + try: + proc = self.session.find_thread( + sample.sample_pid, sample.sample_tid + ) + if proc: + proc_comm = proc.comm() + if proc_comm is not None: + comm = proc_comm + except TypeError: + pass + comm = str(comm).replace(" ", "_") + sep = "-" + if self.args.include_pid: + comm = f"{comm}{sep}{(sample.sample_pid or 0)}" + sep = "/" + if self.args.include_tid: + comm = f"{comm}{sep}{(sample.sample_tid or 0)}" + stack.append(comm) + + stack_string = ";".join(reversed(stack)) + self.lines[stack_string] += 1 + + def print_totals(self) -> None: + """Print sorted collapsed stacks.""" + try: + for stack in sorted(self.lines): + print(f"{stack} {self.lines[stack]}") + sys.stdout.flush() + except BrokenPipeError: + devnull = os.open(os.devnull, os.O_WRONLY) + os.dup2(devnull, sys.stdout.fileno()) + os.close(devnull) + + +def main(): + """Main function.""" + ap = argparse.ArgumentParser( + description="Format perf samples with one line per distinct call stack" + ) + ap.add_argument("-i", "--input", default="perf.data", help="Input file name") + ap.add_argument("--include-tid", action="store_true", help="include thread id in stack") + ap.add_argument("--include-pid", action="store_true", help="include process id in stack") + ap.add_argument("--no-comm", dest="include_comm", action="store_false", default=True, + help="do not separate stacks according to comm") + ap.add_argument("--tidy-java", action="store_true", help="beautify Java signatures") + ap.add_argument("--kernel", dest="annotate_kernel", action="store_true", + help="annotate kernel functions with _[k]") + + args = ap.parse_args() + + if args.include_tid and not args.include_comm: + print("requesting tid but not comm is invalid", file=sys.stderr) + sys.exit(1) + if args.include_pid and not args.include_comm: + print("requesting pid but not comm is invalid", file=sys.stderr) + sys.exit(1) + + analyzer = StackCollapseAnalyzer(args) + + try: + session = perf.session(perf.data(args.input), sample=analyzer.process_event) + analyzer.session = session + session.process_events() + except IOError as e: + print(f"Error: {e}", file=sys.stderr) + sys.exit(1) + except KeyboardInterrupt: + pass + finally: + analyzer.session = None + + analyzer.print_totals() + + +if __name__ == "__main__": + main() diff --git a/tools/perf/tests/shell/test_stackcollapse_python.sh b/tools/perf/tests/shell/test_stackcollapse_python.sh new file mode 100755 index 000000000000..a3b6c917e842 --- /dev/null +++ b/tools/perf/tests/shell/test_stackcollapse_python.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# stackcollapse python test + +set -e -o pipefail + +shelldir=$(dirname "$0") +# shellcheck source=lib/setup_python.sh +. "${shelldir}"/lib/setup_python.sh + +if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then + echo "Skipping test, perf python module not found" + exit 2 +fi + +script_dir="$(dirname "$0")/../../python" +script_path="${script_dir}/stackcollapse.py" + +if [ ! -f "$script_path" ]; then + echo "Skipping test, stackcollapse.py not found at $script_path" + exit 2 +fi + +err=0 +temp_data="" +temp_out="" + +cleanup() { + rm -f "${temp_data}" "${temp_out}" +} +trap 'cleanup' EXIT TERM INT + +temp_data=$(mktemp /tmp/perf.data.XXXXXX) +temp_out=$(mktemp /tmp/perf.out.XXXXXX) + +echo "Testing stackcollapse.py..." + +# Create a perf.data file with callchains. Use a busy workload rather than +# sleep, as an idle system may not generate any samples at all. +perf record -g -o "${temp_data}" \ + -- perf test -w noploop >/dev/null 2>&1 || \ + { echo "Skipping test, perf record failed"; exit 2; } + +if [ ! -s "${temp_data}" ]; then + echo "Skipping test, perf record failed to create data" + exit 2 +fi + +# Check that the script executes with default options +if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then + echo "stackcollapse.py test failed" + err=1 +else + # It outputs stacks like: swapper;...;... 2 + if [ ! -s "${temp_out}" ]; then + echo "Expected stack traces in output, but output is empty." + err=1 + else + echo "stackcollapse default test passed." + fi +fi + +# Test CLI flags (--include-pid, --include-tid, --tidy-java, --kernel) and BrokenPipeError +if ! "$PYTHON" "$script_path" -i "${temp_data}" \ + --include-pid --include-tid --tidy-java --kernel | head -n 1 > "${temp_out}" || \ + [ ! -s "${temp_out}" ]; then + echo "stackcollapse.py options/pipe test failed" + err=1 +elif ! "$PYTHON" "$script_path" -i "${temp_data}" --no-comm > "${temp_out}" || \ + [ ! -s "${temp_out}" ]; then + echo "stackcollapse.py --no-comm test failed" + err=1 +else + echo "stackcollapse options test passed." +fi +rm -f "${temp_out}" + +exit $err -- 2.56.0.rc1.310.g51773c2048-goog