mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Alireza Haghdoost via B4 Relay <devnull+haghdoost.uber.com@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	 Arnaldo Carvalho de Melo <acme@kernel.org>,
	 Namhyung Kim <namhyung@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	 Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	 Adrian Hunter <adrian.hunter@intel.com>,
	 James Clark <james.clark@linaro.org>
Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
	 Alireza Haghdoost <haghdoost@uber.com>
Subject: [PATCH v8 0/7] perf script: Lazy symbol loading
Date: Sat, 21 Nov 2026 08:41:00 -0800	[thread overview]
Message-ID: <20261121-perf-symbol-memory-send-v8-0-370561cbb5a7@uber.com> (raw)

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, and keeps each symbol, with its demangled name, until
process exit. Most of those symbols are never sampled. On a large
profile this turns symbol loading into the main memory cost of perf
script, and in a memory-constrained cgroup into an OOM kill.

With this series, perf script --lazy-load-symbols reads a symbol name
only when a sample needs it. On a perf script run over a 45,073-sample
callchain profile of a large C++ service (about 1M symbols), the output
is identical in all three cases (medians of eleven runs):

                                      time    max RSS
  perf-tools-next                     3.70 s  319 MiB
  v8                                  3.75 s  327 MiB
  v8 --lazy-load-symbols              3.56 s  174 MiB

Eager loading remains the default, and costs 8 more bytes per symbol for
the name pointer. Lazy loading covers userspace ELF .symtab and .dynsym.
The kernel, modules, PPC64 .opd and .gnu_debugdata still load names
eagerly.

The patches:

  [1/7] Split the size, type and binding checks out of
        choose_best_symbol(), so that the loader can tell which aliases
        need their names. No functional change. Unchanged from v7,
        reviewed by Ian.

  [2/7] Make the symbol name a pointer instead of a flexible array, so
        that a name can be set after the symbol is in the rb-tree.
        Unchanged from v7, reviewed by Ian.

  [3/7] Add lazy symbol names to the symbol code: the DSO lookups read
        the name of the symbol they return. Nothing creates such
        symbols yet.

  [4/7] Check that the symbol table file is still the one perf loaded
        before reading names from it. Nothing records a file yet.

  [5/7] Add --lazy-load-symbols, which makes the ELF loader create
        symbols with lazy names and record the file for the check.

  [6/7] Unit tests for lazy names and for the file check.

  [7/7] Shell tests, including parity with eager loading.

perf-tools-next now has the first three patches of Ian's series [1].
Its later patches replace the symbol rb-tree with an array, which would
shrink each symbol further. They conflict with this series, since the
name loading in 3/7 and the alias name loading in 5/7 walk that
rb-tree. This series does not depend on them, and I'll rebase on them
if they land first.

[1] https://lore.kernel.org/all/20261117185305.108003-1-irogers@google.com/

Changes in v8:

- Rebase onto current perf-tools-next, which has Ian's annotation and
  ELF loader changes.
- Split v7 3/4 for review (Namhyung): 3/7 adds lazy names to the symbol
  code, 4/7 adds the check of the symbol table file, and 5/7 adds the
  option and makes the ELF loader create lazy names. The tests are in
  6/7 (unit) and 7/7 (shell).
- 4/7: Also accept a file whose device, inode, size, mtime or ctime
  changed if it has the build ID of the DSO and its .strtab and .dynstr
  did not move, such as a copy of the same build (Namhyung). The
  identity is still checked first, since reading the build ID at each
  reopen more than doubles the time of the worst case.
- 4/7: Compare the ctime too, so a file copied over in place with its
  size and mtime kept (cp -p) is not taken for the same file. Treat a
  string table section that no longer exists or is compressed as
  another file.
- 4/7: Warn once when the file cannot be opened or checked, rather than
  only in debug output, and try again for later names, since the
  failure can be temporary.
- 5/7: Do not skip the .dynsym copies of lazy .symtab symbols. Skipping
  them changed the shape of the rb-tree, which broke lookups past a
  nested entry point and the PLT name of an exported IFUNC. This
  replaces the v7 fix for a zero-size symbol before a copy (Sashiko).
  Duplicate selection now reads the names of each copy, so on the
  profile above, where one binary exports most of its symbols, lazy
  loading keeps its memory saving but loses most of the run time it
  saved in v7. Most of that time goes to reading each name with its own
  pread(). As discussed with Namhyung, a later series will let the DSO
  data cache read the symbol table file, and names will be read through
  it. A prototype with a small read cache gets back about two thirds of
  the v7 speedup for about 1.5 MiB more RSS.
- 6/7: Add tests for a file overwritten in place, for a copy with the
  same build, with moved or missing string tables, and of another build,
  for a failed open, and for a copy whose build ID cannot be read.
- 7/7: Add parity cases for a zero-size symbol before a copy, nested
  entry points, and an exported IFUNC with aliases.

Previous versions:

v7: https://lore.kernel.org/all/20261113-perf-symbol-memory-send-v7-0-d992eced9f9c@uber.com/
v6: https://lore.kernel.org/all/20261028-perf-symbol-memory-send-v6-0-7e90415f0101@uber.com/
v5: https://lore.kernel.org/all/20261005-perf-symbol-memory-send-v5-0-165dceb2b049@uber.com/
v4: https://lore.kernel.org/all/20261002-perf-symbol-memory-send-v4-0-0a592bb2539e@uber.com/
v3: https://lore.kernel.org/all/20260925-perf-symbol-memory-send-v3-0-3e4e234c363b@uber.com/
v2: https://lore.kernel.org/all/20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@uber.com/
v1: https://lore.kernel.org/all/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com/

---
Alireza Haghdoost (7):
      perf symbols: Split out name-independent duplicate symbol selection
      perf symbols: Point to the symbol name instead of embedding it
      perf symbols: Add lazy symbol names
      perf symbols: Check the file before reading lazy symbol names
      perf script: Add --lazy-load-symbols to load symbol names lazily
      perf test: Add lazy symbol loading unit tests
      perf test: Add lazy symbol loading shell tests

 tools/perf/Documentation/perf-script.txt           |  11 +
 tools/perf/builtin-script.c                        |   2 +
 tools/perf/tests/Build                             |   1 +
 tools/perf/tests/builtin-test.c                    |   1 +
 tools/perf/tests/shell/lazy_load_symbols_parity.sh | 447 ++++++++++
 tools/perf/tests/shell/lib/lazy_names.sh           |  20 +
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 190 ++++
 tools/perf/tests/symbol-lazy.c                     | 963 +++++++++++++++++++++
 tools/perf/tests/tests.h                           |   1 +
 tools/perf/ui/browsers/map.c                       |   2 +-
 tools/perf/ui/stdio/hist.c                         |   1 +
 tools/perf/util/dso.c                              |   2 +-
 tools/perf/util/dso.h                              |  36 +-
 tools/perf/util/symbol-elf.c                       | 355 +++++++-
 tools/perf/util/symbol-minimal.c                   |   6 +
 tools/perf/util/symbol.c                           | 141 ++-
 tools/perf/util/symbol.h                           |  31 +-
 tools/perf/util/symbol_conf.h                      |   1 +
 18 files changed, 2183 insertions(+), 28 deletions(-)
---
base-commit: bb27ddbaee56536fb8df0a24f07b526766dfe131
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@uber.com>



             reply	other threads:[~2026-11-21 16:41 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-21 16:41 Alireza Haghdoost via B4 Relay [this message]
2026-11-21 16:41 ` [PATCH v8 1/7] perf symbols: Split out name-independent duplicate symbol selection Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 2/7] perf symbols: Point to the symbol name instead of embedding it Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 3/7] perf symbols: Add lazy symbol names Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 4/7] perf symbols: Check the file before reading " Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 5/7] perf script: Add --lazy-load-symbols to load symbol names lazily Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 6/7] perf test: Add lazy symbol loading unit tests Alireza Haghdoost via B4 Relay
2026-11-21 16:41 ` [PATCH v8 7/7] perf test: Add lazy symbol loading shell tests Alireza Haghdoost via B4 Relay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121-perf-symbol-memory-send-v8-0-370561cbb5a7@uber.com \
    --to=devnull+haghdoost.uber.com@kernel.org \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=haghdoost@uber.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®