[PATCH v7 0/4] perf script: Lazy symbol loading

[Alireza Haghdoost via B4 Relay <devnull+haghdoost.uber.com@kernel.org>]

perf script loads the entire ELF symbol table of every DSO that appears
in a sample, and keeps each symbol, with its demangled name, until
process exit. Most of those symbols are never sampled. On a large
profile this turns symbol loading into the main memory cost of perf
script, and in a memory-constrained cgroup into an OOM kill.

This series adds an opt-in lazy loader for userspace ELF DSOs, after two
preparatory patches:

  [1/4] Split the size, type and binding checks out of
        choose_best_symbol(), so that the loader can tell which aliases
        need their names. No functional change.

  [2/4] Make the symbol name a pointer instead of a flexible array, so
        that a name can be set after the symbol is in the rb-tree.

  [3/4] Add --lazy-load-symbols to load symbol names lazily.

  [4/4] Unit and shell tests.

On a perf script run over a 45,073-sample callchain profile of a large
C++ service (about 1M symbols), the output is identical in all three
cases (medians of seven runs):

                                      time    max RSS
  perf-tools-next                     2.9 s   320 MiB
  v7                                  2.9 s   327 MiB
  v7 --lazy-load-symbols              1.3 s   135 MiB

Eager loading remains the default, and costs 8 more bytes per symbol for
the name pointer. Lazy loading covers userspace ELF .symtab and .dynsym.
The kernel, modules, PPC64 .opd and .gnu_debugdata still load names
eagerly.

Ian's series [1], which removes the priv area and replaces the rb-tree
with an array, would shrink each symbol further. The two series conflict:
both use bit 14 of the symbol flags, and the alias name loading and the
.dynsym copy check in 3/4 walk the rb-tree that [1] replaces. This series
does not depend on it, and I'll rebase on it if it lands first.

[1] https://lore.kernel.org/all/20261113171734.2188066-1-irogers@google.com/

Changes in v7:

- Drop v6 1/5, now in perf-tools-next as 794579ff4959 ("perf symbols:
  Fix broken ELF_C_READ_MMAP fallback guard").
- Rebase onto current perf-tools-next. Since baef30dd8308 ("perf
  symbol-elf: Fix zero-size symbol end calculation across .symtab and
  .dynsym") moved duplicate selection to dso__load_sym(), 3/4 loads the
  alias and IFUNC names there, once after all symbol tables.
- 3/4: Check that the symbol table file is still the one that was
  loaded (device, inode, size, mtime) before reading names from it.
  If a binary is replaced during the run, perf warns once and shows
  "[unknown]" instead of names from the new file.
- 3/4: Load names under the DSO lock instead of a global mutex (Ian,
  Namhyung).
- 3/4: Rename symbol__has_lazy_name() to symbol__needs_lazy_name()
  (Namhyung).
- 3/4: Lazy loading stays opt-in for now; see the reply on v6 4/5.
- 4/4: Add a test for a replaced symbol table file.
- 4/4: Skip instead of failing when names load eagerly (.opd,
  .gnu_debugdata, compressed .strtab), check that test_loop resolved
  before requiring lazy names, skip a kernel DSO given with --dso, and
  include <elf.h>, <stdbool.h> and <stdio.h> (Sashiko).
- Reword the changelogs and trim some comments in 3/4.

Previous versions:

v6: https://lore.kernel.org/all/20261028-perf-symbol-memory-send-v6-0-7e90415f0101@uber.com/
v5: https://lore.kernel.org/all/20261005-perf-symbol-memory-send-v5-0-165dceb2b049@uber.com/
v4: https://lore.kernel.org/all/20261002-perf-symbol-memory-send-v4-0-0a592bb2539e@uber.com/
v3: https://lore.kernel.org/all/20260925-perf-symbol-memory-send-v3-0-3e4e234c363b@uber.com/
v2: https://lore.kernel.org/all/20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@uber.com/
v1: https://lore.kernel.org/all/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com/

---
Alireza Haghdoost (4):
      perf symbols: Split out name-independent duplicate symbol selection
      perf symbols: Point to the symbol name instead of embedding it
      perf script: Add --lazy-load-symbols to load symbol names lazily
      perf test: Test lazy symbol name loading

 tools/perf/Documentation/perf-script.txt           |   9 +
 tools/perf/builtin-script.c                        |   2 +
 tools/perf/tests/Build                             |   1 +
 tools/perf/tests/builtin-test.c                    |   1 +
 tools/perf/tests/shell/lazy_load_symbols_parity.sh | 293 ++++++++++
 tools/perf/tests/shell/lib/lazy_names.sh           |  19 +
 tools/perf/tests/shell/script_lazy_load_symbols.sh | 191 +++++++
 tools/perf/tests/symbol-lazy.c                     | 606 +++++++++++++++++++++
 tools/perf/tests/tests.h                           |   1 +
 tools/perf/ui/browsers/map.c                       |   2 +-
 tools/perf/ui/stdio/hist.c                         |   1 +
 tools/perf/util/dso.c                              |   2 +-
 tools/perf/util/dso.h                              |  20 +-
 tools/perf/util/symbol-elf.c                       | 299 +++++++++-
 tools/perf/util/symbol-minimal.c                   |   6 +
 tools/perf/util/symbol.c                           | 141 ++++-
 tools/perf/util/symbol.h                           |  29 +-
 tools/perf/util/symbol_conf.h                      |   1 +
 18 files changed, 1599 insertions(+), 25 deletions(-)
---
base-commit: a1c3b14c28ebfa79dd56e2743c2fbad0c02a7146
change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9

Best regards,
--  
Alireza Haghdoost <haghdoost@uber.com>