mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Alireza Haghdoost via B4 Relay <devnull+haghdoost.uber.com@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	 Arnaldo Carvalho de Melo <acme@kernel.org>,
	 Namhyung Kim <namhyung@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	 Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	 Adrian Hunter <adrian.hunter@intel.com>,
	 James Clark <james.clark@linaro.org>,
	Alexei Starovoitov <ast@kernel.org>,
	 Andrii Nakryiko <andriin@fb.com>
Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Alireza Haghdoost <haghdoost@uber.com>
Subject: [PATCH v2 0/4] perf script: Bounded and lazy symbol loading
Date: Sat, 19 Sep 2026 19:33:55 -0700	[thread overview]
Message-ID: <20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@uber.com> (raw)

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, allocating each symbol into an rb-tree held until process
exit. Therefore, a large enough profile turns symbol loading into an
OOM kill.
This is not scalable for profiling a large cgroup with a lot of large
binaries on a production system with limited free memory.

This series adds two independent, opt-in mechanisms and a leading
regression fix they build on:

  [1/4] Fix a broken "#ifdef ELF_C_READ_MMAP" guard so perf actually
        mmaps ELF files instead of malloc'ing section data. This is a
        standalone regression fix that is introduced by 22dd1ac91a77.

  [2/4] --max-symbol-bytes <size>: a byte budget on struct symbol
        allocations (and the lazy index) enforced at the ELF symbol
        loader, degrading to [unknown] with a warning past the cap.
        An unbounded profile doesn't just risk OOM-killing itself. It
        also forces memory pressure on the whole host, pushing the kernel
        to reclaim from co-located latency-sensitive processes. Capping
        it lets the user bound that footprint up front and choose the
        trade-off explicitly.

  [3/4] --lazy-load-symbols: build a compact per-DSO sorted index and
        resolve only the sampled addresses, reading names through the DSO
        data cache at lookup time. On the production fixture, peak RssAnon
        drops from 265 MiB to 39 MiB (6.8x) and wall time from 3.1 s to
        1.85 s (1.7x). Memory optimizations usually cost time; this one
        does not because lazy loading skips a lot of calloc and demangle
        calls.

  [4/4] Shell and unit tests for both options.

Lazy loading handles the common userspace ELF symtab/dynsym path. Eager
loading remains available for dense coverage and for PPC64 .opd and
.gnu_debugdata.

Changes in v2:

- Replace direct pread() name reads with the exact symbol source's DSO data
  cache, preserving split-debuginfo offsets and descriptor reopen behavior.
- Drop the byte-identical-output claim and retain eager loading for PPC64
  .opd and .gnu_debugdata.
- Make the symbol budget atomic and strict, account complete name lengths,
  accept a bare 0 as unlimited, and keep partial zero-sized ranges from
  covering omitted symbols.
- Align lazy lookup with eager duplicate and IFUNC selection, PLT clipping,
  and name-sorted materialization.
- Move option documentation into the feature patches. Add unit and shell
  coverage for cache reopen, truncated names, budget truncation, and skip
  handling.

Link: https://lore.kernel.org/all/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com/
---
Alireza Haghdoost (4):
      perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
      perf script: Add --max-symbol-bytes to bound ELF symbol memory
      perf script: Add --lazy-load-symbols for lazy symbol loading
      perf test: Test lazy symbol loading and symbol memory limits

 tools/perf/Documentation/perf-script.txt           |  28 +
 tools/perf/arch/powerpc/util/sym-handling.c        |   5 +-
 tools/perf/builtin-script.c                        |  44 ++
 tools/perf/tests/Build                             |   1 +
 tools/perf/tests/builtin-test.c                    |   1 +
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 278 ++++++++
 .../tests/shell/script_lazy_load_symbols_skip.sh   |  26 +
 tools/perf/tests/symbol-bytes.c                    | 340 ++++++++++
 tools/perf/tests/tests.h                           |   1 +
 tools/perf/util/dso.c                              |  63 +-
 tools/perf/util/dso.h                              |  49 ++
 tools/perf/util/map.c                              |  30 +-
 tools/perf/util/symbol-elf.c                       | 733 ++++++++++++++++++++-
 tools/perf/util/symbol-minimal.c                   |  17 +
 tools/perf/util/symbol.c                           | 140 +++-
 tools/perf/util/symbol.h                           |  24 +-
 tools/perf/util/symbol_conf.h                      |   2 +
 17 files changed, 1723 insertions(+), 59 deletions(-)
---
base-commit: aa18964dd64511305de0711fed912054da6f5d18
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@uber.com>



             reply	other threads:[~2026-09-20  2:34 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-20  2:33 Alireza Haghdoost via B4 Relay [this message]
2026-09-20  2:33 ` [PATCH v2 1/4] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard Alireza Haghdoost via B4 Relay
2026-09-20 23:52   ` Namhyung Kim
2026-09-20  2:33 ` [PATCH v2 2/4] perf script: Add --max-symbol-bytes to bound ELF symbol memory Alireza Haghdoost via B4 Relay
2026-09-21  0:03   ` Namhyung Kim
2026-09-21  4:27     ` Alireza Haghdoost
2026-09-20  2:33 ` [PATCH v2 3/4] perf script: Add --lazy-load-symbols for lazy symbol loading Alireza Haghdoost via B4 Relay
2026-09-21  0:19   ` Namhyung Kim
2026-09-21  4:45     ` Alireza Haghdoost
2026-09-20  2:33 ` [PATCH v2 4/4] perf test: Test lazy symbol loading and symbol memory limits Alireza Haghdoost via B4 Relay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@uber.com \
    --to=devnull+haghdoost.uber.com@kernel.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=andriin@fb.com \
    --cc=ast@kernel.org \
    --cc=haghdoost@uber.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®