* [PATCH v3 1/6] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 2/6] perf dso: Allow reading DSO data from an explicit file Alireza Haghdoost via B4 Relay
` (5 subsequent siblings)
6 siblings, 0 replies; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
Commit 22dd1ac91a77 ("tools: Remove feature-libelf-mmap feature
detection") replaced perf's compile-time feature test with an #ifdef on
ELF_C_READ_MMAP. ELF_C_READ_MMAP is an Elf_Cmd enumerator rather than a
preprocessor macro, so the condition is always false and perf silently
uses ELF_C_READ.
Perf already requires a sufficiently recent elfutils version that
provides ELF_C_READ_MMAP. Use the enumerator directly instead of
restoring a feature probe or retaining an unreachable fallback.
Fixes: 22dd1ac91a77 ("tools: Remove feature-libelf-mmap feature detection")
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Reviewed-by: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/util/symbol.h | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index d0bac824c79c..46b1649c64fc 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -57,15 +57,7 @@ static inline bool is_livepatch_symbol(const char *str)
return strstarts(str, KLP_SYM_PREFIX);
}
-/*
- * libelf 0.8.x and earlier do not support ELF_C_READ_MMAP;
- * for newer versions we can use mmap to reduce memory usage:
- */
-#ifdef ELF_C_READ_MMAP
-# define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
-#else
-# define PERF_ELF_C_READ_MMAP ELF_C_READ
-#endif
+#define PERF_ELF_C_READ_MMAP ELF_C_READ_MMAP
#ifdef HAVE_LIBELF_SUPPORT
Elf_Scn *elf_section_by_name(Elf *elf, GElf_Ehdr *ep,
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v3 2/6] perf dso: Allow reading DSO data from an explicit file
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 1/6] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection Alireza Haghdoost via B4 Relay
` (4 subsequent siblings)
6 siblings, 0 replies; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
The DSO data cache derives the file to open from the DSO's binary type,
which can resolve to the runtime image rather than the file a symbol
table was read from. With split debuginfo, offsets taken from the
debuginfo file (for example string-table offsets) are then applied to an
unrelated file.
Add dso__data_set_path() so a DSO can be configured to read from one
exact file while keeping the data cache's descriptor eviction and
reopening. Such DSOs may be owned privately rather than being part of a
dsos collection, so drop the assertion that every opened data DSO is in
one.
Add a DSO data test that reads through an explicit path, closes the
descriptor, and reads an uncached offset to exercise reopening.
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
tools/perf/tests/dso-data.c | 42 ++++++++++++++++++++++++++++++++++++++++++
tools/perf/util/dso.c | 43 ++++++++++++++++++++++++++++++++++---------
tools/perf/util/dso.h | 2 ++
3 files changed, 78 insertions(+), 9 deletions(-)
diff --git a/tools/perf/tests/dso-data.c b/tools/perf/tests/dso-data.c
index 46bc3f597260..fbfb2f08d3ba 100644
--- a/tools/perf/tests/dso-data.c
+++ b/tools/perf/tests/dso-data.c
@@ -393,11 +393,53 @@ static int test__dso_data_reopen(struct test_suite *test __maybe_unused, int sub
return 0;
}
+static int test__dso_data_path(struct test_suite *test __maybe_unused, int subtest __maybe_unused)
+{
+ u8 expect[10] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
+ char *file = test_file(TEST_FILE_SIZE);
+ struct dso *dso;
+ long nr, nr_end;
+ u8 buf[10];
+
+ TEST_ASSERT_VAL("No test file", file);
+ nr = open_files_cnt();
+
+ /*
+ * The DSO name does not exist and the DSO is not in a dsos
+ * collection; reads must come from the configured path.
+ */
+ dso = dso__new("/nonexistent/perf-test-dso-data-path");
+ TEST_ASSERT_VAL("Failed to create dso", dso);
+ dso__set_binary_type(dso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO);
+ TEST_ASSERT_VAL("Failed to set path", !dso__data_set_path(dso, file));
+
+ TEST_ASSERT_VAL("Wrong size",
+ dso__data_read_offset(dso, NULL, 10, buf, 10) == 10);
+ TEST_ASSERT_VAL("Wrong data", !memcmp(buf, expect, 10));
+
+ /* An uncached offset after close must reopen the configured path. */
+ dso__data_close(dso);
+ memset(buf, 0, sizeof(buf));
+ TEST_ASSERT_VAL("Wrong size after reopen",
+ dso__data_read_offset(dso, NULL, DSO__DATA_CACHE_SIZE * 2 + 10,
+ buf, 10) == 10);
+ TEST_ASSERT_VAL("Wrong data after reopen",
+ buf[0] == (DSO__DATA_CACHE_SIZE * 2 + 10) % 10);
+
+ dso__data_close(dso);
+ dso__put(dso);
+ unlink(file);
+
+ nr_end = open_files_cnt();
+ TEST_ASSERT_VAL("failed leaking files", nr == nr_end);
+ return 0;
+}
static struct test_case tests__dso_data[] = {
TEST_CASE("read", dso_data),
TEST_CASE("cache", dso_data_cache),
TEST_CASE("reopen", dso_data_reopen),
+ TEST_CASE("explicit path", dso_data_path),
{ .name = NULL, }
};
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 9e90de92fcfa..c88b2a771832 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -532,8 +532,6 @@ static void dso__list_add(struct dso *dso) EXCLUSIVE_LOCKS_REQUIRED(_dso__data_o
#ifdef REFCNT_CHECKING
dso__data(dso)->dso = dso__get(dso);
#endif
- /* Assume the dso is part of dsos, hence the optional reference count above. */
- assert(dso__dsos(dso));
dso__data_open_cnt++;
}
@@ -578,16 +576,22 @@ char *dso__filename_with_chroot(const struct dso *dso, const char *filename)
static char *dso__get_filename(struct dso *dso, const char *root_dir,
bool *decomp)
{
- char *name = malloc(PATH_MAX);
+ char *name;
*decomp = false;
- if (name == NULL)
- return NULL;
-
- if (dso__read_binary_type_filename(dso, dso__binary_type(dso),
- root_dir, name, PATH_MAX))
- goto out;
+ if (dso__data(dso)->path) {
+ name = strdup(dso__data(dso)->path);
+ if (!name)
+ return NULL;
+ } else {
+ name = malloc(PATH_MAX);
+ if (!name)
+ return NULL;
+ if (dso__read_binary_type_filename(dso, dso__binary_type(dso),
+ root_dir, name, PATH_MAX))
+ goto out;
+ }
if (!is_regular_file(name)) {
struct stat st;
@@ -813,6 +817,26 @@ void dso__data_close(struct dso *dso)
mutex_unlock(dso__data_open_lock());
}
+/**
+ * dso__data_set_path - Read @dso's data from an explicit file
+ * @dso: dso object
+ * @path: file to open instead of the path derived from the binary type
+ *
+ * Used when the data must come from one specific file, such as the separate
+ * debuginfo file that a symbol table was read from. Must be called before any
+ * data is read, as already cached data is not invalidated.
+ */
+int dso__data_set_path(struct dso *dso, const char *path)
+{
+ char *new_path = strdup(path);
+
+ if (!new_path)
+ return -ENOMEM;
+ free(dso__data(dso)->path);
+ dso__data(dso)->path = new_path;
+ return 0;
+}
+
static void try_to_open_dso(struct dso *dso, struct machine *machine)
EXCLUSIVE_LOCKS_REQUIRED(_dso__data_open_lock)
{
@@ -1762,6 +1786,7 @@ void dso__delete(struct dso *dso)
dso__data_close(dso);
auxtrace_cache__free(RC_CHK_ACCESS(dso)->auxtrace_cache);
dso_cache__free(dso);
+ zfree(&RC_CHK_ACCESS(dso)->data.path);
dso__free_a2l(dso);
dso__free_a2l_libbfd(dso);
dso__free_libdw(dso);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index e7d5f4bbf894..ff2c91e9a2b9 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -264,6 +264,7 @@ struct dso_data {
#ifdef REFCNT_CHECKING
struct dso *dso;
#endif
+ char *path;
int fd;
int status;
u32 status_seen;
@@ -910,6 +911,7 @@ bool dso__data_get_fd(struct dso *dso, struct machine *machine, int *fd)
EXCLUSIVE_TRYLOCK_FUNCTION(true, _dso__data_open_lock);
void dso__data_put_fd(struct dso *dso) UNLOCK_FUNCTION(_dso__data_open_lock);
void dso__data_close(struct dso *dso) LOCKS_EXCLUDED(_dso__data_open_lock);
+int dso__data_set_path(struct dso *dso, const char *path);
int dso__data_file_size(struct dso *dso, struct machine *machine);
off_t dso__data_size(struct dso *dso, struct machine *machine);
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 1/6] perf symbols: Fix broken ELF_C_READ_MMAP fallback guard Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 2/6] perf dso: Allow reading DSO data from an explicit file Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 19:40 ` Ian Rogers
2026-09-25 19:09 ` [PATCH v3 4/6] perf script: Add --max-symbol-bytes to bound ELF symbol memory Alireza Haghdoost via B4 Relay
` (3 subsequent siblings)
6 siblings, 1 reply; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
symbols__fixup_duplicate() chooses between symbols with the same start
address through choose_best_symbol(), which needs fully constructed
struct symbol objects. A loader that selects among aliases before
allocating symbols cannot use it.
Move the policy into symbol__choose_best(), which compares the size,
name, type and binding of two candidates described by struct
symbol_candidate, and pass the same description to the
arch__choose_best_symbol() hook. choose_best_symbol() becomes a wrapper
that describes two struct symbols. No functional change intended.
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
tools/perf/arch/powerpc/util/sym-handling.c | 6 ++--
tools/perf/util/symbol.c | 43 +++++++++++++++++++++--------
tools/perf/util/symbol.h | 14 +++++++++-
3 files changed, 47 insertions(+), 16 deletions(-)
diff --git a/tools/perf/arch/powerpc/util/sym-handling.c b/tools/perf/arch/powerpc/util/sym-handling.c
index 947bfad7aa59..c263cbfefba5 100644
--- a/tools/perf/arch/powerpc/util/sym-handling.c
+++ b/tools/perf/arch/powerpc/util/sym-handling.c
@@ -10,10 +10,10 @@
#include "probe-event.h"
#include "probe-file.h"
-int arch__choose_best_symbol(struct symbol *syma,
- struct symbol *symb __maybe_unused)
+int arch__choose_best_symbol(const struct symbol_candidate *syma,
+ const struct symbol_candidate *symb __maybe_unused)
{
- char *sym = syma->name;
+ const char *sym = syma->name;
#if !defined(_CALL_ELF) || _CALL_ELF != 2
/* Skip over any initial dot */
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 163652f071c6..4b50250d07fa 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -145,8 +145,8 @@ int __weak arch__compare_symbol_names_n(const char *namea, const char *nameb,
return strncmp(namea, nameb, n);
}
-int __weak arch__choose_best_symbol(struct symbol *syma,
- struct symbol *symb __maybe_unused)
+int __weak arch__choose_best_symbol(const struct symbol_candidate *syma,
+ const struct symbol_candidate *symb __maybe_unused)
{
/* Avoid "SyS" kernel syscall aliases */
if (strlen(syma->name) >= 3 && !strncmp(syma->name, "SyS", 3))
@@ -157,38 +157,39 @@ int __weak arch__choose_best_symbol(struct symbol *syma,
return SYMBOL_A;
}
-static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+int symbol__choose_best(const struct symbol_candidate *syma,
+ const struct symbol_candidate *symb)
{
s64 a;
s64 b;
size_t na, nb;
/* Prefer a symbol with non zero length */
- a = syma->end - syma->start;
- b = symb->end - symb->start;
+ a = syma->size;
+ b = symb->size;
if ((b == 0) && (a > 0))
return SYMBOL_A;
else if ((a == 0) && (b > 0))
return SYMBOL_B;
- if (symbol__type(syma) != symbol__type(symb)) {
- if (symbol__type(syma) == STT_NOTYPE)
+ if (syma->type != symb->type) {
+ if (syma->type == STT_NOTYPE)
return SYMBOL_B;
- if (symbol__type(symb) == STT_NOTYPE)
+ if (symb->type == STT_NOTYPE)
return SYMBOL_A;
}
/* Prefer a non weak symbol over a weak one */
- a = symbol__binding(syma) == STB_WEAK;
- b = symbol__binding(symb) == STB_WEAK;
+ a = syma->binding == STB_WEAK;
+ b = symb->binding == STB_WEAK;
if (b && !a)
return SYMBOL_A;
if (a && !b)
return SYMBOL_B;
/* Prefer a global symbol over a non global one */
- a = symbol__binding(syma) == STB_GLOBAL;
- b = symbol__binding(symb) == STB_GLOBAL;
+ a = syma->binding == STB_GLOBAL;
+ b = symb->binding == STB_GLOBAL;
if (a && !b)
return SYMBOL_A;
if (b && !a)
@@ -213,6 +214,24 @@ static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
return arch__choose_best_symbol(syma, symb);
}
+static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
+{
+ struct symbol_candidate a = {
+ .size = syma->end - syma->start,
+ .name = syma->name,
+ .type = symbol__type(syma),
+ .binding = symbol__binding(syma),
+ };
+ struct symbol_candidate b = {
+ .size = symb->end - symb->start,
+ .name = symb->name,
+ .type = symbol__type(symb),
+ .binding = symbol__binding(symb),
+ };
+
+ return symbol__choose_best(&a, &b);
+}
+
void symbols__fixup_duplicate(struct rb_root_cached *symbols)
{
struct rb_node *nd;
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index 46b1649c64fc..b9fa722a9a14 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -299,10 +299,22 @@ const char *arch__normalize_symbol_name(const char *name);
#define SYMBOL_A 0
#define SYMBOL_B 1
+/* Attributes used to choose between symbols that share a start address. */
+struct symbol_candidate {
+ u64 size;
+ const char *name;
+ u8 type;
+ u8 binding;
+};
+
+int symbol__choose_best(const struct symbol_candidate *a,
+ const struct symbol_candidate *b);
+
int arch__compare_symbol_names(const char *namea, const char *nameb);
int arch__compare_symbol_names_n(const char *namea, const char *nameb,
unsigned int n);
-int arch__choose_best_symbol(struct symbol *syma, struct symbol *symb);
+int arch__choose_best_symbol(const struct symbol_candidate *a,
+ const struct symbol_candidate *b);
enum symbol_tag_include {
SYMBOL_TAG_INCLUDE__NONE = 0,
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection
2026-09-25 19:09 ` [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:40 ` Ian Rogers
2026-09-25 19:55 ` Alireza Haghdoost
0 siblings, 1 reply; 13+ messages in thread
From: Ian Rogers @ 2026-09-25 19:40 UTC (permalink / raw)
To: haghdoost
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
On Fri, Sep 25, 2026 at 12:15 PM Alireza Haghdoost via B4 Relay
<devnull+haghdoost.uber.com@kernel.org> wrote:
>
> From: Alireza Haghdoost <haghdoost@uber.com>
>
> symbols__fixup_duplicate() chooses between symbols with the same start
> address through choose_best_symbol(), which needs fully constructed
> struct symbol objects. A loader that selects among aliases before
> allocating symbols cannot use it.
>
> Move the policy into symbol__choose_best(), which compares the size,
> name, type and binding of two candidates described by struct
> symbol_candidate, and pass the same description to the
> arch__choose_best_symbol() hook. choose_best_symbol() becomes a wrapper
> that describes two struct symbols. No functional change intended.
>
> Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
> ---
> tools/perf/arch/powerpc/util/sym-handling.c | 6 ++--
> tools/perf/util/symbol.c | 43 +++++++++++++++++++++--------
> tools/perf/util/symbol.h | 14 +++++++++-
> 3 files changed, 47 insertions(+), 16 deletions(-)
>
> diff --git a/tools/perf/arch/powerpc/util/sym-handling.c b/tools/perf/arch/powerpc/util/sym-handling.c
> index 947bfad7aa59..c263cbfefba5 100644
> --- a/tools/perf/arch/powerpc/util/sym-handling.c
> +++ b/tools/perf/arch/powerpc/util/sym-handling.c
> @@ -10,10 +10,10 @@
> #include "probe-event.h"
> #include "probe-file.h"
>
> -int arch__choose_best_symbol(struct symbol *syma,
> - struct symbol *symb __maybe_unused)
> +int arch__choose_best_symbol(const struct symbol_candidate *syma,
> + const struct symbol_candidate *symb __maybe_unused)
> {
> - char *sym = syma->name;
> + const char *sym = syma->name;
>
> #if !defined(_CALL_ELF) || _CALL_ELF != 2
> /* Skip over any initial dot */
> diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
> index 163652f071c6..4b50250d07fa 100644
> --- a/tools/perf/util/symbol.c
> +++ b/tools/perf/util/symbol.c
> @@ -145,8 +145,8 @@ int __weak arch__compare_symbol_names_n(const char *namea, const char *nameb,
> return strncmp(namea, nameb, n);
> }
>
> -int __weak arch__choose_best_symbol(struct symbol *syma,
> - struct symbol *symb __maybe_unused)
> +int __weak arch__choose_best_symbol(const struct symbol_candidate *syma,
> + const struct symbol_candidate *symb __maybe_unused)
> {
> /* Avoid "SyS" kernel syscall aliases */
> if (strlen(syma->name) >= 3 && !strncmp(syma->name, "SyS", 3))
> @@ -157,38 +157,39 @@ int __weak arch__choose_best_symbol(struct symbol *syma,
> return SYMBOL_A;
> }
>
> -static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
> +int symbol__choose_best(const struct symbol_candidate *syma,
> + const struct symbol_candidate *symb)
> {
> s64 a;
> s64 b;
> size_t na, nb;
>
> /* Prefer a symbol with non zero length */
> - a = syma->end - syma->start;
> - b = symb->end - symb->start;
> + a = syma->size;
> + b = symb->size;
> if ((b == 0) && (a > 0))
> return SYMBOL_A;
> else if ((a == 0) && (b > 0))
> return SYMBOL_B;
>
> - if (symbol__type(syma) != symbol__type(symb)) {
> - if (symbol__type(syma) == STT_NOTYPE)
> + if (syma->type != symb->type) {
> + if (syma->type == STT_NOTYPE)
> return SYMBOL_B;
> - if (symbol__type(symb) == STT_NOTYPE)
> + if (symb->type == STT_NOTYPE)
> return SYMBOL_A;
> }
>
> /* Prefer a non weak symbol over a weak one */
> - a = symbol__binding(syma) == STB_WEAK;
> - b = symbol__binding(symb) == STB_WEAK;
> + a = syma->binding == STB_WEAK;
> + b = symb->binding == STB_WEAK;
> if (b && !a)
> return SYMBOL_A;
> if (a && !b)
> return SYMBOL_B;
>
> /* Prefer a global symbol over a non global one */
> - a = symbol__binding(syma) == STB_GLOBAL;
> - b = symbol__binding(symb) == STB_GLOBAL;
> + a = syma->binding == STB_GLOBAL;
> + b = symb->binding == STB_GLOBAL;
> if (a && !b)
> return SYMBOL_A;
> if (b && !a)
> @@ -213,6 +214,24 @@ static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
> return arch__choose_best_symbol(syma, symb);
> }
>
> +static int choose_best_symbol(struct symbol *syma, struct symbol *symb)
> +{
> + struct symbol_candidate a = {
> + .size = syma->end - syma->start,
> + .name = syma->name,
> + .type = symbol__type(syma),
> + .binding = symbol__binding(syma),
> + };
> + struct symbol_candidate b = {
> + .size = symb->end - symb->start,
> + .name = symb->name,
> + .type = symbol__type(symb),
> + .binding = symbol__binding(symb),
> + };
> +
> + return symbol__choose_best(&a, &b);
> +}
> +
> void symbols__fixup_duplicate(struct rb_root_cached *symbols)
> {
> struct rb_node *nd;
> diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
> index 46b1649c64fc..b9fa722a9a14 100644
> --- a/tools/perf/util/symbol.h
> +++ b/tools/perf/util/symbol.h
> @@ -299,10 +299,22 @@ const char *arch__normalize_symbol_name(const char *name);
> #define SYMBOL_A 0
> #define SYMBOL_B 1
>
> +/* Attributes used to choose between symbols that share a start address. */
> +struct symbol_candidate {
> + u64 size;
> + const char *name;
> + u8 type;
> + u8 binding;
> +};
> +
> +int symbol__choose_best(const struct symbol_candidate *a,
> + const struct symbol_candidate *b);
> +
> int arch__compare_symbol_names(const char *namea, const char *nameb);
> int arch__compare_symbol_names_n(const char *namea, const char *nameb,
> unsigned int n);
> -int arch__choose_best_symbol(struct symbol *syma, struct symbol *symb);
> +int arch__choose_best_symbol(const struct symbol_candidate *a,
> + const struct symbol_candidate *b);
Given you can change this function definition and only change 1 C
file, is there any reason not to make the function static in symbol.c
and move the struct symbol_candidate definition also to symbol.c?
Thanks,
Ian
>
> enum symbol_tag_include {
> SYMBOL_TAG_INCLUDE__NONE = 0,
>
> --
> Git-157)
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection
2026-09-25 19:40 ` Ian Rogers
@ 2026-09-25 19:55 ` Alireza Haghdoost
2026-09-25 20:21 ` Ian Rogers
0 siblings, 1 reply; 13+ messages in thread
From: Alireza Haghdoost @ 2026-09-25 19:55 UTC (permalink / raw)
To: Ian Rogers
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
> > -int arch__choose_best_symbol(struct symbol *syma, struct symbol *symb);
> > +int arch__choose_best_symbol(const struct symbol_candidate *a,
> > + const struct symbol_candidate *b);
>
> Given you can change this function definition and only change 1 C
> file, is there any reason not to make the function static in symbol.c
> and move the struct symbol_candidate definition also to symbol.c?
>
Thanks for the review. Patch 5 calls symbol__choose_best() from
symbol-elf.c: the lazy index resolves same-address duplicates from its
index entries, not from struct symbols, and uses the same policy as
the eager loader. That's why patch 3 makes it non-static.
struct symbol_candidate has to stay in symbol.h as well, because
arch__choose_best_symbol() takes it and powerpc overrides that weak
function in arch/powerpc/util/sym-handling.c.
I can say this in the patch 3 commit message in v4.
Thanks,
Alireza
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection
2026-09-25 19:55 ` Alireza Haghdoost
@ 2026-09-25 20:21 ` Ian Rogers
0 siblings, 0 replies; 13+ messages in thread
From: Ian Rogers @ 2026-09-25 20:21 UTC (permalink / raw)
To: Alireza Haghdoost
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
On Fri, Sep 25, 2026 at 12:55 PM Alireza Haghdoost <haghdoost@uber.com> wrote:
>
> > > -int arch__choose_best_symbol(struct symbol *syma, struct symbol *symb);
> > > +int arch__choose_best_symbol(const struct symbol_candidate *a,
> > > + const struct symbol_candidate *b);
> >
> > Given you can change this function definition and only change 1 C
> > file, is there any reason not to make the function static in symbol.c
> > and move the struct symbol_candidate definition also to symbol.c?
> >
>
> Thanks for the review. Patch 5 calls symbol__choose_best() from
> symbol-elf.c: the lazy index resolves same-address duplicates from its
> index entries, not from struct symbols, and uses the same policy as
> the eager loader. That's why patch 3 makes it non-static.
>
> struct symbol_candidate has to stay in symbol.h as well, because
> arch__choose_best_symbol() takes it and powerpc overrides that weak
> function in arch/powerpc/util/sym-handling.c.
>
> I can say this in the patch 3 commit message in v4.
Thanks, I saw this when I got to the later patches, sorry for the noise.
Ian
> Thanks,
> Alireza
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v3 4/6] perf script: Add --max-symbol-bytes to bound ELF symbol memory
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
` (2 preceding siblings ...)
2026-09-25 19:09 ` [PATCH v3 3/6] perf symbols: Factor out duplicate symbol selection Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 5/6] perf script: Add --lazy-load-symbols for lazy symbol loading Alireza Haghdoost via B4 Relay
` (2 subsequent siblings)
6 siblings, 0 replies; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
perf script eagerly materializes every ELF symbol into an rb-tree kept
until process exit. Large profiles can therefore consume substantial
anonymous memory, causing perf script to be OOM-killed or forcing the
kernel to reclaim memory from co-located workloads.
This patch adds --max-symbol-bytes to bound struct symbol allocations.
Once the budget is reached, the ELF loader stops loading symbols, warns
once, and lets unresolved addresses appear as [unknown]. This allows
users to bound the memory footprint upfront and explicitly choose between
complete symbolization and avoiding unbounded host memory pressure. perf
record already provides a similar --max-size option to bound disk usage.
The counter includes every symbol__new() allocation, but this patch
enforces the limit only in the ELF loader, which is the source of the
unbounded memory growth addressed here. In this path, reaching the limit
can safely produce [unknown] symbols. Other loaders currently treat a
failed symbol allocation as an error. Capping those paths would therefore
require separate changes whose complexity may outweigh the potential
memory savings.
The cap applies to userspace DSOs, vmlinux-as-ELF, and kernel modules.
Sizes require a B/K/M/G suffix, except that a bare 0 and the default mean
unlimited. Reservations are atomic so concurrent loaders cannot exceed the
limit, and partial zero-sized symbol ranges do not cover omitted
addresses. Allocation and free both charge the stored u16 name length, so
accounting stays balanced; names longer than U16_MAX are charged at the
truncated length.
Document the option with the code that introduces it and add a
concurrent-reservation test.
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
tools/perf/Documentation/perf-script.txt | 10 ++++
tools/perf/builtin-script.c | 42 ++++++++++++++
tools/perf/tests/Build | 1 +
tools/perf/tests/builtin-test.c | 1 +
tools/perf/tests/symbol-bytes.c | 94 ++++++++++++++++++++++++++++++++
tools/perf/tests/tests.h | 1 +
tools/perf/util/symbol-elf.c | 66 ++++++++++++++++++----
tools/perf/util/symbol.c | 91 +++++++++++++++++++++++++++++--
tools/perf/util/symbol.h | 6 ++
tools/perf/util/symbol_conf.h | 1 +
10 files changed, 299 insertions(+), 14 deletions(-)
diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 200ea25891d8..217167a2e56b 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -412,6 +412,16 @@ include::itrace.txt[]
Default: 127
+--max-symbol-bytes::
+ Limit the bytes held in struct symbol allocations for DSOs on the
+ libelf symbol-loader path: userspace DSOs, vmlinux-as-ELF, and kernel
+ modules. This is not a cap on all symbol memory or RSS: symbols from
+ kallsyms, JIT maps, and libbfd are counted but not capped. Accepts a
+ size with a B/K/M/G suffix (e.g. 128M). When the budget is exceeded,
+ the ELF loader stops adding symbols; addresses not covered by symbols
+ already loaded are then printed as [unknown]. A warning is printed.
+ Default: 0 (unlimited).
+
--ns::
Use 9 decimal places when displaying time (i.e. show the nanoseconds)
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 0174489d1c0f..6c459ce6f433 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -68,6 +68,7 @@
#include "util/thread.h"
#include "util/thread_map.h"
#include "util/time-utils.h"
+#include "util/units.h"
#include "util/tool.h"
#include "util/trace-event.h"
#include "util/unwind.h"
@@ -4110,6 +4111,44 @@ static int parse_callret_trace(const struct option *opt __maybe_unused,
return 0;
}
+static int parse_max_symbol_bytes(const struct option *opt,
+ const char *str, int unset)
+{
+ unsigned long *max_bytes = (unsigned long *)opt->value;
+ static struct parse_tag size_tags[] = {
+ { .tag = 'B', .mult = 1 },
+ { .tag = 'K', .mult = 1 << 10 },
+ { .tag = 'M', .mult = 1 << 20 },
+ { .tag = 'G', .mult = 1 << 30 },
+ { .tag = 0 },
+ };
+ unsigned long bytes;
+ size_t len;
+
+ if (unset) {
+ *max_bytes = 0;
+ return 0;
+ }
+
+ if (!strcmp(str, "0")) {
+ *max_bytes = 0;
+ return 0;
+ }
+
+ len = strlen(str);
+ if (len < 2 || !strchr("BKMG", str[len - 1]) ||
+ strspn(str, "0123456789") != len - 1)
+ return -1;
+
+ bytes = parse_tag_value(str, size_tags);
+ if (bytes != (unsigned long)-1) {
+ *max_bytes = bytes;
+ return 0;
+ }
+
+ return -1;
+}
+
int cmd_script(int argc, const char **argv)
{
bool show_full_info = false;
@@ -4210,6 +4249,9 @@ int cmd_script(int argc, const char **argv)
"Set the maximum stack depth when parsing the callchain, "
"anything beyond the specified depth will be ignored. "
"Default: kernel.perf_event_max_stack or " __stringify(PERF_MAX_STACK_DEPTH)),
+ OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
+ "size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
+ parse_max_symbol_bytes),
OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 81c311b131b7..fe92bfdea724 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -67,6 +67,7 @@ perf-test-y += sigtrap.o
perf-test-y += event_groups.o
perf-test-y += hybrid-merge.o
perf-test-y += symbols.o
+perf-test-y += symbol-bytes.o
perf-test-y += util.o
perf-test-y += hwmon_pmu.o
perf-test-y += tool_pmu.o
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index d2f594921e25..45482a7a91d2 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -154,6 +154,7 @@ static struct test_suite *generic_tests[] = {
&suite__event_groups,
&suite__hybrid_merge,
&suite__symbols,
+ &suite__symbol_bytes,
&suite__util,
&suite__subcmd_help,
&suite__kallsyms_split,
diff --git a/tools/perf/tests/symbol-bytes.c b/tools/perf/tests/symbol-bytes.c
new file mode 100644
index 000000000000..d3055f5ed321
--- /dev/null
+++ b/tools/perf/tests/symbol-bytes.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <limits.h>
+#include <pthread.h>
+#include <stdint.h>
+
+#include "debug.h"
+#include "symbol.h"
+#include "symbol_conf.h"
+#include "tests.h"
+
+struct reserve_arg {
+ size_t bytes;
+ bool success;
+};
+
+static void *reserve_bytes(void *data)
+{
+ struct reserve_arg *arg = data;
+
+ arg->success = symbol__try_account_bytes(arg->bytes);
+ return NULL;
+}
+
+static int test__symbol_bytes_reservation(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ enum { NR_THREADS = 8, NR_ALLOWED = 4 };
+ const size_t reservation = 1024;
+ unsigned long saved_max = symbol_conf.max_symbol_bytes;
+ size_t baseline = symbol__bytes_used();
+ struct reserve_arg args[NR_THREADS];
+ pthread_t threads[NR_THREADS];
+ int created = 0, successful = 0;
+ int ret = TEST_FAIL;
+ int i;
+
+ if (baseline > ULONG_MAX - NR_ALLOWED * reservation)
+ return TEST_SKIP;
+
+ symbol_conf.max_symbol_bytes = baseline + NR_ALLOWED * reservation;
+ if (symbol__try_account_bytes(SIZE_MAX)) {
+ pr_debug("overflowing symbol reservation succeeded\n");
+ symbol__unaccount_bytes(SIZE_MAX);
+ goto out;
+ }
+
+ for (i = 0; i < NR_THREADS; i++) {
+ args[i].bytes = reservation;
+ args[i].success = false;
+ if (pthread_create(&threads[i], NULL, reserve_bytes, &args[i]))
+ goto out_join;
+ created++;
+ }
+
+out_join:
+ for (i = 0; i < created; i++)
+ pthread_join(threads[i], NULL);
+ for (i = 0; i < created; i++) {
+ if (args[i].success)
+ successful++;
+ }
+
+ if (created != NR_THREADS || successful != NR_ALLOWED) {
+ pr_debug("symbol reservation count: created %d, successful %d\n",
+ created, successful);
+ goto out_release;
+ }
+ if (symbol__bytes_used() != baseline + NR_ALLOWED * reservation) {
+ pr_debug("symbol reservation exceeded configured budget\n");
+ goto out_release;
+ }
+ ret = TEST_OK;
+
+out_release:
+ for (i = 0; i < created; i++) {
+ if (args[i].success)
+ symbol__unaccount_bytes(args[i].bytes);
+ }
+out:
+ symbol_conf.max_symbol_bytes = saved_max;
+ if (symbol__bytes_used() != baseline)
+ ret = TEST_FAIL;
+ return ret;
+}
+
+static struct test_case tests__symbol_bytes[] = {
+ TEST_CASE("Concurrent strict reservations", symbol_bytes_reservation),
+ { .name = NULL, }
+};
+
+struct test_suite suite__symbol_bytes = {
+ .desc = "Symbol memory accounting",
+ .test_cases = tests__symbol_bytes,
+};
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 9c96f33483d1..4a53e063e73d 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -179,6 +179,7 @@ DECLARE_SUITE(sigtrap);
DECLARE_SUITE(event_groups);
DECLARE_SUITE(hybrid_merge);
DECLARE_SUITE(symbols);
+DECLARE_SUITE(symbol_bytes);
DECLARE_SUITE(util);
DECLARE_SUITE(uncore_event_sorting);
DECLARE_SUITE(subcmd_help);
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index e955c3feddcd..2f7ea1499cbf 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -561,6 +561,13 @@ static bool get_plt_got_name(GElf_Shdr *shdr, size_t i,
return result;
}
+static void symbol_budget_warning(void)
+{
+ pr_warning_once("perf: symbol memory budget exceeded (%lu bytes), "
+ "remaining symbols will be [unknown]\n",
+ symbol_conf.max_symbol_bytes);
+}
+
static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
GElf_Ehdr *ehdr,
char *buf, size_t buf_sz)
@@ -580,11 +587,19 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
get_rela_dyn_info(elf, ehdr, &di, scn);
for (i = 0; i < shdr.sh_size; i += shdr.sh_entsize) {
+ bool budget_exceeded;
+
if (!get_plt_got_name(&shdr, i, &di, buf, buf_sz))
snprintf(buf, buf_sz, "offset_%#" PRIx64 "@plt", (u64)shdr.sh_offset + i);
- sym = symbol__new(shdr.sh_offset + i, shdr.sh_entsize, STB_GLOBAL, STT_FUNC, buf);
- if (!sym)
+ sym = symbol__new_bounded(shdr.sh_offset + i, shdr.sh_entsize,
+ STB_GLOBAL, STT_FUNC, buf, &budget_exceeded);
+ if (!sym) {
+ if (budget_exceeded) {
+ symbol_budget_warning();
+ err = 0;
+ }
goto out;
+ }
symbols__insert(dso__symbols(dso), sym);
}
err = 0;
@@ -615,6 +630,7 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
Elf *elf;
int nr = 0, err = -1;
struct rel_info ri = { .is_rela = false };
+ bool budget_exceeded;
bool lazy_plt;
elf = ss->elf;
@@ -636,9 +652,16 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
return 0;
/* Add a symbol for .plt header */
- plt_sym = symbol__new(shdr_plt.sh_offset, plt_header_size, STB_GLOBAL, STT_FUNC, ".plt");
- if (!plt_sym)
+ plt_sym = symbol__new_bounded(shdr_plt.sh_offset, plt_header_size,
+ STB_GLOBAL, STT_FUNC, ".plt",
+ &budget_exceeded);
+ if (!plt_sym) {
+ if (budget_exceeded) {
+ symbol_budget_warning();
+ return 0;
+ }
goto out_elf_end;
+ }
symbols__insert(dso__symbols(dso), plt_sym);
/* Only x86 has .plt.got */
@@ -756,9 +779,15 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
"offset_%#" PRIx64 "@plt", plt_offset);
free(demangled);
- f = symbol__new(plt_offset, plt_entry_size, STB_GLOBAL, STT_FUNC, sympltname);
- if (!f)
+ f = symbol__new_bounded(plt_offset, plt_entry_size, STB_GLOBAL,
+ STT_FUNC, sympltname, &budget_exceeded);
+ if (!f) {
+ if (budget_exceeded) {
+ symbol_budget_warning();
+ err = 0;
+ }
goto out_elf_end;
+ }
plt_offset += plt_entry_size;
symbols__insert(dso__symbols(dso), f);
@@ -1534,6 +1563,7 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
Elf *elf;
int nr = 0;
bool remap_kernel = false, adjust_kernel_syms = false;
+ bool budget_truncated = false;
u64 max_text_sh_offset = 0;
if (kmap && !kmaps)
@@ -1633,8 +1663,16 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
char *demangled = NULL;
int is_label = elf_sym__is_label(&sym);
const char *section_name;
+ bool budget_exceeded = false;
bool used_opd = false;
+ if (symbol_conf.max_symbol_bytes &&
+ symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+ symbol_budget_warning();
+ budget_truncated = true;
+ break;
+ }
+
if (!is_label && !elf_sym__filter(&sym))
continue;
@@ -1775,10 +1813,17 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
if (demangled != NULL)
elf_name = demangled;
- f = symbol__new(sym.st_value, sym.st_size,
- GELF_ST_BIND(sym.st_info),
- GELF_ST_TYPE(sym.st_info), elf_name);
+ f = symbol__new_bounded(sym.st_value, sym.st_size,
+ GELF_ST_BIND(sym.st_info),
+ GELF_ST_TYPE(sym.st_info), elf_name,
+ &budget_exceeded);
+ if (!f && budget_exceeded) {
+ symbol_budget_warning();
+ budget_truncated = true;
+ }
free(demangled);
+ if (!f && budget_truncated)
+ break;
if (!f)
goto out_elf_end;
@@ -1793,7 +1838,8 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
* For misannotated, zeroed, ASM function sizes.
*/
if (nr > 0) {
- symbols__fixup_end(dso__symbols(dso), false);
+ if (!budget_truncated)
+ symbols__fixup_end(dso__symbols(dso), false);
symbols__fixup_duplicate(dso__symbols(dso));
if (kmap) {
/*
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 4b50250d07fa..cf92a7604a67 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -331,13 +331,82 @@ void symbols__fixup_end(struct rb_root_cached *symbols, bool is_kallsyms)
curr->end = roundup(curr->start, 4096) + 4096;
}
-struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
+static _Atomic size_t symbol_bytes_used;
+
+size_t symbol__bytes_used(void)
+{
+ return atomic_load_explicit(&symbol_bytes_used, memory_order_relaxed);
+}
+
+void symbol__account_bytes(size_t bytes)
+{
+ atomic_fetch_add_explicit(&symbol_bytes_used, bytes, memory_order_relaxed);
+}
+
+bool symbol__try_account_bytes(size_t bytes)
+{
+ size_t old = symbol__bytes_used();
+
+ for (;;) {
+ if (old > SIZE_MAX - bytes)
+ return false;
+ if (symbol_conf.max_symbol_bytes &&
+ (old > symbol_conf.max_symbol_bytes ||
+ bytes > symbol_conf.max_symbol_bytes - old))
+ return false;
+ if (atomic_compare_exchange_weak_explicit(&symbol_bytes_used, &old, old + bytes,
+ memory_order_relaxed,
+ memory_order_relaxed))
+ return true;
+ }
+}
+
+void symbol__unaccount_bytes(size_t bytes)
+{
+ atomic_fetch_sub_explicit(&symbol_bytes_used, bytes, memory_order_relaxed);
+}
+
+/*
+ * Bytes charged for a symbol, derived from the stored u16 namelen so that
+ * allocation and deletion always charge the same amount. Names longer than
+ * U16_MAX are charged at the truncated length.
+ */
+static size_t symbol__charged_bytes(u16 namelen)
+{
+ return symbol_conf.priv_size + sizeof(struct symbol) + namelen + 1;
+}
+
+static struct symbol *__symbol__new(u64 start, u64 len, u8 binding, u8 type,
+ const char *name, bool bounded,
+ bool *budget_exceeded)
{
size_t namelen = strlen(name) + 1;
- struct symbol *sym = calloc(1, (symbol_conf.priv_size +
- sizeof(*sym) + namelen));
- if (sym == NULL)
+ size_t alloc_size, charged;
+ struct symbol *sym;
+
+ if (budget_exceeded)
+ *budget_exceeded = false;
+ if (namelen > SIZE_MAX - sizeof(*sym) ||
+ symbol_conf.priv_size > SIZE_MAX - sizeof(*sym) - namelen)
+ return NULL;
+ alloc_size = symbol_conf.priv_size + sizeof(*sym) + namelen;
+ charged = symbol__charged_bytes(namelen - 1);
+
+ if (bounded && !symbol__try_account_bytes(charged)) {
+ if (budget_exceeded)
+ *budget_exceeded = true;
+ return NULL;
+ }
+
+ sym = calloc(1, alloc_size);
+ if (sym == NULL) {
+ if (bounded)
+ symbol__unaccount_bytes(charged);
return NULL;
+ }
+
+ if (!bounded)
+ symbol__account_bytes(charged);
if (symbol_conf.priv_size) {
if (symbol_conf.init_annotation) {
@@ -360,8 +429,21 @@ struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *
return sym;
}
+struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
+{
+ return __symbol__new(start, len, binding, type, name, false, NULL);
+}
+
+struct symbol *symbol__new_bounded(u64 start, u64 len, u8 binding, u8 type,
+ const char *name, bool *budget_exceeded)
+{
+ return __symbol__new(start, len, binding, type, name, true, budget_exceeded);
+}
+
void symbol__delete(struct symbol *sym)
{
+ size_t charged = symbol__charged_bytes(sym->namelen);
+
if (symbol_conf.priv_size) {
if (symbol_conf.init_annotation) {
struct annotation *notes = symbol__annotation(sym);
@@ -369,6 +451,7 @@ void symbol__delete(struct symbol *sym)
annotation__exit(notes);
}
}
+ symbol__unaccount_bytes(charged);
free(((void *)sym) - symbol_conf.priv_size);
}
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index b9fa722a9a14..de4ac1c51732 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -227,6 +227,12 @@ void symbol__elf_init(void);
int symbol__annotation_init(void);
struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name);
+struct symbol *symbol__new_bounded(u64 start, u64 len, u8 binding, u8 type,
+ const char *name, bool *budget_exceeded);
+size_t symbol__bytes_used(void);
+void symbol__account_bytes(size_t bytes);
+bool symbol__try_account_bytes(size_t bytes);
+void symbol__unaccount_bytes(size_t bytes);
size_t __symbol__fprintf_symname_offs(const struct symbol *sym,
const struct addr_location *al,
bool unknown_as_addr,
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 37d35f42dcc1..30cbc53cbcd0 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -123,6 +123,7 @@ struct symbol_conf {
int pad_output_len_dso;
int group_sort_idx;
int addr_range;
+ unsigned long max_symbol_bytes;
DECLARE_BITMAP(parallelism_filter, MAX_NR_CPUS + 1);
};
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v3 5/6] perf script: Add --lazy-load-symbols for lazy symbol loading
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
` (3 preceding siblings ...)
2026-09-25 19:09 ` [PATCH v3 4/6] perf script: Add --max-symbol-bytes to bound ELF symbol memory Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 19:09 ` [PATCH v3 6/6] perf test: Test lazy symbol loading and symbol memory limits Alireza Haghdoost via B4 Relay
2026-09-25 20:20 ` [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Ian Rogers
6 siblings, 0 replies; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
perf script eagerly materializes eligible symbols from every DSO
encountered in samples. On a production fixture, it loaded about 765k
symbols to resolve about 45k distinct (DSO, symbol) frames, exceeding the
memory available in a memory-constrained cgroup.
This patch adds --lazy-load-symbols for userspace ELF DSOs. It builds a
compact sorted index, resolves sampled addresses by binary search, reads
symbol names through a private data-source DSO, and caches resolved symbols
in the existing rb-tree. The private DSO uses the normal DSO data cache, so
the exact split-debuginfo source can be reopened after descriptor eviction.
If the source cannot be read during preflight, perf discards the index and
eagerly loads that DSO instead.
On the same fixture, peak RssAnon drops from 265 MiB to 39 MiB and wall
time from 3.1 seconds to 1.85 seconds. Memory optimizations usually cost
time; this one does not because lazy loading skips many unnecessary
calloc() calls and demangling operations.
Lazy loading is most effective when samples reference only a small
fraction of the available symbols, such as profiles spanning many large
DSOs. It still builds an index proportional to the total symbol count.
Eager loading remains available for dense symbol coverage or cases
requiring its broader ELF and architecture support.
This does not claim full parity with the eager loader. Lazy loading
supports the common userspace ELF symtab/dynsym case; .gnu_debugdata and
PPC64 .opd continue through the eager loader.
Materialization is serialized with the DSO lock. Name lookups materialize
the remaining index before constructing the name-sorted array. Lazy loading
shares eager duplicate and IFUNC selection, and clips ranges that cross
.plt before synthesizing PLT symbols.
In lazy mode, address lookups always take the DSO lock. Name lookups also
free the index before the name-sorted array is built, even when the
symbol budget stops materialization early, so a DSO with a name array
never changes again. When no PT_LOAD covers a symbol, it falls back to
the section header as the eager loader does, using the runtime section
for NOBITS sections of a debuginfo file.
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
tools/perf/Documentation/perf-script.txt | 16 +
tools/perf/builtin-script.c | 2 +
tools/perf/util/dso.c | 17 +
tools/perf/util/dso.h | 45 +++
tools/perf/util/map.c | 19 +-
tools/perf/util/symbol-elf.c | 662 +++++++++++++++++++++++++++++++
tools/perf/util/symbol-minimal.c | 16 +
tools/perf/util/symbol.c | 9 +
tools/perf/util/symbol_conf.h | 1 +
9 files changed, 786 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 217167a2e56b..615c3ba1aab6 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -412,6 +412,21 @@ include::itrace.txt[]
Default: 127
+--lazy-load-symbols::
+ Resolve symbols lazily instead of eagerly loading the full
+ symbol table of every DSO that appears in a sample. A compact
+ sorted index is built per DSO and only the addresses that appear
+ in samples are materialized into symbols, with names read from the
+ file's string table through the DSO data cache at lookup time. This
+ sharply reduces memory (and usually time) for profiles of large
+ binaries where only a small fraction of the symbol table is
+ referenced. This applies only to userspace ELF DSOs; kernel DSOs and
+ modules always load eagerly. Operations that look up a symbol by name
+ materialize the remainder of that DSO's index first to preserve
+ name-lookup behavior.
+ Output may differ from the default loader for some targets
+ (e.g. PPC64 .opd or .gnu_debugdata). Default: off.
+
--max-symbol-bytes::
Limit the bytes held in struct symbol allocations for DSOs on the
libelf symbol-loader path: userspace DSOs, vmlinux-as-ELF, and kernel
@@ -420,6 +435,7 @@ include::itrace.txt[]
size with a B/K/M/G suffix (e.g. 128M). When the budget is exceeded,
the ELF loader stops adding symbols; addresses not covered by symbols
already loaded are then printed as [unknown]. A warning is printed.
+ With --lazy-load-symbols, the lazy symbol index is also counted.
Default: 0 (unlimited).
--ns::
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 6c459ce6f433..f7b8b5f03ef4 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -4252,6 +4252,8 @@ int cmd_script(int argc, const char **argv)
OPT_CALLBACK(0, "max-symbol-bytes", &symbol_conf.max_symbol_bytes,
"size", "Limit bytes for ELF struct symbol (e.g. 128M; 0=unlimited)",
parse_max_symbol_bytes),
+ OPT_BOOLEAN(0, "lazy-load-symbols", &symbol_conf.lazy_load_symbols,
+ "Resolve symbols lazily instead of loading full symtabs"),
OPT_BOOLEAN(0, "reltime", &reltime, "Show time stamps relative to start"),
OPT_BOOLEAN(0, "deltatime", &deltatime, "Show time stamps relative to previous event"),
OPT_BOOLEAN('I', "show-info", &show_full_info,
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index c88b2a771832..9cbde0a7977e 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -1704,6 +1704,20 @@ void dso__set_sorted_by_name(struct dso *dso)
RC_CHK_ACCESS(dso)->sorted_by_name = true;
}
+void dso__free_ondemand(struct dso *dso)
+{
+ struct dso_ondemand *od = RC_CHK_ACCESS(dso)->ondemand;
+
+ if (!od)
+ return;
+ RC_CHK_ACCESS(dso)->ondemand = NULL;
+ free(od->sorted);
+ symbol__unaccount_bytes(od->nr_alloc * sizeof(*od->sorted));
+ dso__data_close(od->data_dso);
+ dso__put(od->data_dso);
+ free(od);
+}
+
struct dso *dso__new_id(const char *name, const struct dso_id *id)
{
RC_STRUCT(dso) *dso = zalloc(sizeof(*dso) + strlen(name) + 1);
@@ -1785,6 +1799,9 @@ void dso__delete(struct dso *dso)
dso__data_close(dso);
auxtrace_cache__free(RC_CHK_ACCESS(dso)->auxtrace_cache);
+ mutex_lock(dso__lock(dso));
+ dso__free_ondemand(dso);
+ mutex_unlock(dso__lock(dso));
dso_cache__free(dso);
zfree(&RC_CHK_ACCESS(dso)->data.path);
dso__free_a2l(dso);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index ff2c91e9a2b9..6e6c2c7f13a8 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -283,6 +283,27 @@ struct dso_bpf_prog {
struct perf_env *env;
};
+struct sym_idx {
+ u64 start;
+ u64 end;
+ u32 name_off;
+ u8 binding;
+ u8 type;
+ u8 flags;
+};
+
+#define SYM_IDX_FLAG_IFUNC_ALIAS (1 << 0)
+#define SYM_IDX_FLAG_MATERIALIZED (1 << 1)
+
+struct dso_ondemand {
+ struct dso *data_dso;
+ u64 strtab_offset;
+ u64 strtab_size;
+ struct sym_idx *sorted;
+ u32 nr_sorted;
+ u32 nr_alloc;
+};
+
struct auxtrace_cache;
DECLARE_RC_STRUCT(dso) {
@@ -314,6 +335,7 @@ DECLARE_RC_STRUCT(dso) {
char *symsrc_filename;
struct nsinfo *nsinfo;
struct auxtrace_cache *auxtrace_cache;
+ struct dso_ondemand *ondemand;
union { /* Tool specific area */
void *priv;
u64 db_id;
@@ -466,6 +488,16 @@ static inline void dso__set_auxtrace_cache(struct dso *dso, struct auxtrace_cach
RC_CHK_ACCESS(dso)->auxtrace_cache = cache;
}
+static inline struct dso_ondemand *dso__ondemand(struct dso *dso)
+{
+ return RC_CHK_ACCESS(dso)->ondemand;
+}
+
+static inline void dso__set_ondemand(struct dso *dso, struct dso_ondemand *od)
+{
+ RC_CHK_ACCESS(dso)->ondemand = od;
+}
+
static inline struct dso_bpf_prog *dso__bpf_prog(struct dso *dso)
{
return &RC_CHK_ACCESS(dso)->bpf_prog;
@@ -838,6 +870,19 @@ int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type t
const char *root_dir, char *filename, size_t size);
bool is_kernel_module(const char *pathname, int cpumode);
bool dso__needs_decompress(struct dso *dso);
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso));
+struct symbol *dso__find_symbol_ondemand_exact(struct dso *dso, u64 addr)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso));
+void dso__materialize_symbols_ondemand(struct dso *dso)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso));
+const char *dso__read_ondemand_symbol_name(struct dso *data_dso,
+ u64 strtab_offset, u64 strtab_size,
+ u64 name_off, char *buf,
+ size_t buflen, char **to_free,
+ unsigned int *nr_reads);
+void dso__free_ondemand(struct dso *dso)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso));
int dso__decompress_kmodule_fd(struct dso *dso, const char *name);
int dso__decompress_kmodule_path(struct dso *dso, const char *name,
char *pathname, size_t len);
diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index 41cdddc987ee..d0fff55eab92 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -382,10 +382,27 @@ int map__load(struct map *map)
struct symbol *map__find_symbol(struct map *map, u64 addr)
{
+ struct dso *dso;
+ struct symbol *sym;
+
if (map__load(map) < 0)
return NULL;
- return dso__find_symbol(map__dso(map), addr);
+ dso = map__dso(map);
+ if (!symbol_conf.lazy_load_symbols)
+ return dso__find_symbol(dso, addr);
+
+ /*
+ * A lazily loaded DSO inserts symbols into its rb-tree on lookup and
+ * drops its index once fully materialized. Look up and materialize
+ * under the DSO lock so readers never race with either change.
+ */
+ mutex_lock(dso__lock(dso));
+ sym = dso__find_symbol(dso, addr);
+ if (!sym)
+ sym = dso__find_symbol_ondemand(dso, addr);
+ mutex_unlock(dso__lock(dso));
+ return sym;
}
struct symbol *map__find_symbol_by_name_idx(struct map *map, const char *name, size_t *idx)
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 2f7ea1499cbf..bb859bafece1 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -2,6 +2,7 @@
#include <fcntl.h>
#include <stdio.h>
#include <errno.h>
+#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
@@ -12,6 +13,7 @@
#include "libbfd.h"
#include "map.h"
#include "maps.h"
+#include "namespaces.h"
#include "symbol.h"
#include "symsrc.h"
#include "machine.h"
@@ -334,6 +336,7 @@ static bool addend_may_be_ifunc(GElf_Ehdr *ehdr, struct rel_info *ri)
static bool get_ifunc_name(Elf *elf, struct dso *dso, GElf_Ehdr *ehdr,
struct rel_info *ri, char *buf, size_t buf_sz)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso))
{
u64 addr = ri->rela.r_addend;
struct symbol *sym;
@@ -348,6 +351,8 @@ static bool get_ifunc_name(Elf *elf, struct dso *dso, GElf_Ehdr *ehdr,
addr -= phdr.p_vaddr - phdr.p_offset;
sym = dso__find_symbol_nocache(dso, addr);
+ if (!sym && dso__ondemand(dso))
+ sym = dso__find_symbol_ondemand_exact(dso, addr);
/* Expecting the address to be an IFUNC or IFUNC alias */
if (!sym || sym->start != addr ||
@@ -608,6 +613,26 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
return err;
}
+static u32 sym_idx__lower_bound(const struct dso_ondemand *od, u64 addr);
+
+static void dso__clip_ondemand_symbols_at(struct dso *dso, u64 addr)
+{
+ struct dso_ondemand *od = dso__ondemand(dso);
+ u32 lo, i;
+
+ if (!od)
+ return;
+
+ lo = sym_idx__lower_bound(od, addr);
+ if (!lo)
+ return;
+
+ for (i = 0; i < lo; i++) {
+ if (od->sorted[i].end > addr)
+ od->sorted[i].end = addr;
+ }
+}
+
/*
* We need to check if we have a .dynsym, so that we can handle the
* .plt, synthesizing its symbols, that aren't on the symtabs (be it
@@ -616,6 +641,7 @@ static int dso__synthesize_plt_got_symbols(struct dso *dso, Elf *elf,
* have the PLT data stripped out (shdr_rel_plt.sh_type == SHT_NOBITS).
*/
int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso))
{
uint32_t idx;
GElf_Sym sym;
@@ -639,6 +665,13 @@ int dso__synthesize_plt_symbols(struct dso *dso, struct symsrc *ss)
if (!elf_section_by_name(elf, &ehdr, &shdr_plt, ".plt", NULL))
return 0;
+ /*
+ * Zero-sized or oversized ELF symbols can have been extended across
+ * .plt. Clip the index first so lookups cannot attribute PLT addresses
+ * to a preceding symbol before the synthesized PLT symbols are added.
+ */
+ dso__clip_ondemand_symbols_at(dso, shdr_plt.sh_offset);
+
/*
* A symbol from a previous section (e.g. .init) can have been expanded
* by symbols__fixup_end() to overlap .plt. Truncate it before adding
@@ -1544,6 +1577,607 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
return 0;
}
+static int cmp_sym_idx(const void *a, const void *b)
+{
+ const struct sym_idx *sa = a, *sb = b;
+
+ if (sa->start != sb->start)
+ return sa->start < sb->start ? -1 : 1;
+ /*
+ * qsort is not stable. During sorting name_off temporarily holds
+ * the fill ordinal, preserving eager's symtab insertion order for
+ * equal-start aliases. It is restored to st_name afterwards.
+ */
+ if (sa->name_off != sb->name_off)
+ return sa->name_off < sb->name_off ? -1 : 1;
+ return 0;
+}
+
+/*
+ * Return the first entry whose start is not less than @addr. ISO C bsearch()
+ * does not provide an insertion point or guarantee the first equal entry, so
+ * clipping and exact-start alias lookup use this helper.
+ */
+static u32 sym_idx__lower_bound(const struct dso_ondemand *od, u64 addr)
+{
+ u32 lo = 0, hi = od->nr_sorted;
+
+ while (lo < hi) {
+ u32 mid = lo + (hi - lo) / 2;
+
+ if (od->sorted[mid].start < addr)
+ lo = mid + 1;
+ else
+ hi = mid;
+ }
+ return lo;
+}
+
+static int cmp_addr_to_sym_idx(const void *key, const void *entry)
+{
+ u64 addr = *(const u64 *)key;
+ const struct sym_idx *idx = entry;
+
+ if (addr < idx->start)
+ return -1;
+ if (addr >= idx->end)
+ return 1;
+ return 0;
+}
+
+static bool ondemand_sym_ok(Elf *elf, Elf_Data *secstrs,
+ const GElf_Sym *sym, u32 sh_link,
+ uint16_t e_machine)
+{
+ Elf_Scn *sym_sec;
+ GElf_Shdr sym_shdr;
+ int is_label = elf_sym__is_label(sym);
+ const char *name;
+
+ if (!is_label && !elf_sym__filter((GElf_Sym *)sym))
+ return false;
+
+ if (sym->st_shndx == SHN_ABS)
+ return false;
+
+ sym_sec = elf_getscn(elf, sym->st_shndx);
+ if (!sym_sec)
+ return false;
+ if (!gelf_getshdr(sym_sec, &sym_shdr))
+ return false;
+ if (!(sym_shdr.sh_flags & SHF_ALLOC))
+ return false;
+
+ if (is_label && (!secstrs || !elf_sec__filter(&sym_shdr, secstrs)))
+ return false;
+
+ name = elf_strptr(elf, sh_link, sym->st_name);
+ if (!name)
+ return false;
+
+ /*
+ * Reject ARM/AArch64/RISC-V "mapping symbols" ($a/$d/$t/$x), as
+ * the eager loop does. They are zero-size STT_NOTYPE labels in
+ * allocated sections that would otherwise be indexed and fill
+ * forward over real functions, misattributing everything after
+ * them.
+ */
+ if (e_machine == EM_ARM || e_machine == EM_AARCH64) {
+ if (name[0] == '$' && strchr("adtx", name[1]) &&
+ (name[2] == '\0' || name[2] == '.'))
+ return false;
+ }
+ if (e_machine == EM_RISCV) {
+ if (name[0] == '$' && strchr("dx", name[1]))
+ return false;
+ }
+
+ return true;
+}
+
+static void sym_idx__candidate(const struct sym_idx *idx, const char *name,
+ struct symbol_candidate *c)
+{
+ c->size = idx->end - idx->start;
+ c->name = name;
+ c->type = idx->type;
+ c->binding = idx->binding;
+}
+
+/*
+ * Keep one index entry per start address, choosing among aliases with the
+ * same policy as symbols__fixup_duplicate(). Mark a survivor whose group
+ * contained an IFUNC, then shrink the index. Returns the new entry count.
+ */
+static u32 dso_ondemand__dedup_aliases(struct dso *dso, struct dso_ondemand *od,
+ Elf *elf, size_t strtab_idx, u32 count)
+{
+ struct sym_idx *sorted = od->sorted, *shrunk;
+ u32 i, j, out = 0;
+
+ for (i = 0; i < count; i = j) {
+ struct symbol_candidate best, cand;
+ bool has_ifunc = sorted[i].type == STT_GNU_IFUNC;
+ char *best_demangled = NULL, *demangled;
+ const char *name;
+ u32 best_idx = i;
+
+ name = elf_strptr(elf, strtab_idx, sorted[i].name_off);
+ if (name) {
+ best_demangled = dso__demangle_sym(dso, 0, name);
+ if (best_demangled)
+ name = best_demangled;
+ }
+ sym_idx__candidate(&sorted[i], name, &best);
+
+ for (j = i + 1; j < count && sorted[j].start == sorted[i].start; j++) {
+ has_ifunc |= sorted[j].type == STT_GNU_IFUNC;
+ name = elf_strptr(elf, strtab_idx, sorted[j].name_off);
+ if (!best.name || !name)
+ continue;
+
+ demangled = dso__demangle_sym(dso, 0, name);
+ if (demangled)
+ name = demangled;
+ sym_idx__candidate(&sorted[j], name, &cand);
+
+ if (symbol__choose_best(&best, &cand) == SYMBOL_B) {
+ free(best_demangled);
+ best_demangled = demangled;
+ best = cand;
+ best_idx = j;
+ } else {
+ free(demangled);
+ }
+ }
+ free(best_demangled);
+
+ sorted[out] = sorted[best_idx];
+ if (has_ifunc && sorted[out].type != STT_GNU_IFUNC)
+ sorted[out].flags |= SYM_IDX_FLAG_IFUNC_ALIAS;
+ out++;
+ }
+
+ if (out < count) {
+ shrunk = realloc(sorted, out * sizeof(*sorted));
+ if (shrunk) {
+ od->sorted = shrunk;
+ symbol__unaccount_bytes((od->nr_alloc - out) * sizeof(*sorted));
+ od->nr_alloc = out;
+ }
+ }
+ return out;
+}
+
+static int dso__build_ondemand_index(struct dso *dso, struct symsrc *syms_ss,
+ struct symsrc *runtime_ss,
+ int dynsym)
+{
+ struct dso_ondemand *od;
+ Elf *elf = syms_ss->elf;
+ GElf_Ehdr ehdr = syms_ss->ehdr;
+ GElf_Shdr shdr;
+ GElf_Shdr strshdr;
+ Elf_Scn *strscn, *sec_strndx;
+ Elf_Data *syms;
+ GElf_Sym sym;
+ Elf_Data *secstrs = NULL;
+ size_t i, index_bytes, reservation_peak;
+ u32 count = 0, j;
+ u32 *name_offsets;
+ u64 nr_entries, strtab_offset;
+ u64 probe_off;
+ u8 probe;
+
+ /*
+ * GNU debugdata is backed by a temporary decompressed fd rather than a
+ * reopenable source path. Keep using the eager loader for that case.
+ */
+ if (syms_ss->type == DSO_BINARY_TYPE__GNU_DEBUGDATA)
+ return 0;
+
+ if (dynsym)
+ shdr = syms_ss->dynshdr;
+ else
+ shdr = syms_ss->symshdr;
+
+ syms = elf_getdata(dynsym ? syms_ss->dynsym : syms_ss->symtab, NULL);
+ if (!syms)
+ return -1;
+
+ if (!shdr.sh_entsize)
+ return 0;
+
+ nr_entries = shdr.sh_size / shdr.sh_entsize;
+ if (nr_entries > UINT32_MAX)
+ return -EOVERFLOW;
+
+ strscn = elf_getscn(elf, shdr.sh_link);
+ if (!strscn || !gelf_getshdr(strscn, &strshdr))
+ return -1;
+ strtab_offset = strshdr.sh_offset;
+
+ /*
+ * Section name string table, used to match the eager path's
+ * elf_sec__filter() (text/data section check for STT_NOTYPE labels).
+ */
+ sec_strndx = elf_getscn(elf, ehdr.e_shstrndx);
+ if (sec_strndx)
+ secstrs = elf_getdata(sec_strndx, NULL);
+
+ for (i = 0; i < nr_entries; i++) {
+ if (!gelf_getsym(syms, i, &sym))
+ continue;
+ if (ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+ ehdr.e_machine))
+ count++;
+ }
+
+ if (!count)
+ return 0;
+ if (check_mul_overflow((size_t)count, sizeof(*od->sorted),
+ &index_bytes))
+ return -EOVERFLOW;
+
+ /*
+ * Account the index against the symbol memory budget: at 24
+ * bytes/symbol it is the dominant on-demand cost and must count
+ * toward --max-symbol-bytes just like struct symbol allocations do.
+ */
+ if (!symbol__try_account_bytes(index_bytes)) {
+ symbol_budget_warning();
+ return 0;
+ }
+ reservation_peak = symbol__bytes_used();
+
+ od = zalloc(sizeof(*od));
+ if (!od) {
+ symbol__unaccount_bytes(index_bytes);
+ return -1;
+ }
+
+ od->sorted = zalloc(index_bytes);
+ if (!od->sorted) {
+ symbol__unaccount_bytes(index_bytes);
+ free(od);
+ return -1;
+ }
+ od->nr_alloc = count;
+ name_offsets = malloc(count * sizeof(*name_offsets));
+ if (!name_offsets) {
+ symbol__unaccount_bytes(index_bytes);
+ free(od->sorted);
+ free(od);
+ return -1;
+ }
+
+ j = 0;
+ for (i = 0; i < nr_entries; i++) {
+ u64 adjusted;
+ GElf_Phdr phdr;
+
+ if (!gelf_getsym(syms, i, &sym))
+ continue;
+ if (!ondemand_sym_ok(elf, secstrs, &sym, shdr.sh_link,
+ ehdr.e_machine))
+ continue;
+
+ adjusted = sym.st_value;
+
+ if ((ehdr.e_machine == EM_ARM) &&
+ (GELF_ST_TYPE(sym.st_info) == STT_FUNC) &&
+ (adjusted & 1))
+ --adjusted;
+
+ if (elf_read_program_header(runtime_ss->elf, adjusted,
+ &phdr) == 0) {
+ adjusted -= phdr.p_vaddr - phdr.p_offset;
+ } else {
+ Elf_Scn *sym_sec = elf_getscn(elf, sym.st_shndx);
+ GElf_Shdr sym_shdr;
+
+ if (sym_sec && gelf_getshdr(sym_sec, &sym_shdr)) {
+ /*
+ * A NOBITS section in a debuginfo file has an
+ * invalid sh_offset; use the runtime section.
+ */
+ if (sym_shdr.sh_type == SHT_NOBITS) {
+ sym_sec = elf_getscn(runtime_ss->elf,
+ sym.st_shndx);
+ if (!sym_sec ||
+ !gelf_getshdr(sym_sec, &sym_shdr))
+ continue;
+ }
+ adjusted -= sym_shdr.sh_addr - sym_shdr.sh_offset;
+ }
+ }
+
+ od->sorted[j].start = adjusted;
+ od->sorted[j].end = sym.st_size;
+ name_offsets[j] = sym.st_name;
+ od->sorted[j].name_off = j;
+ od->sorted[j].binding = GELF_ST_BIND(sym.st_info);
+ od->sorted[j].type = GELF_ST_TYPE(sym.st_info);
+ j++;
+ }
+ count = j;
+ if (!count) {
+ symbol__unaccount_bytes(index_bytes);
+ free(name_offsets);
+ free(od->sorted);
+ free(od);
+ return 0;
+ }
+
+ qsort(od->sorted, count, sizeof(*od->sorted), cmp_sym_idx);
+ for (i = 0; i < count; i++)
+ od->sorted[i].name_off = name_offsets[od->sorted[i].name_off];
+ free(name_offsets);
+
+ for (i = 0; i < count; i++) {
+ u64 size = od->sorted[i].end;
+
+ if (size > 0)
+ od->sorted[i].end = od->sorted[i].start + size;
+ else if (i + 1 < count)
+ od->sorted[i].end = od->sorted[i + 1].start;
+ else
+ od->sorted[i].end = roundup(od->sorted[i].start, 4096) + 4096;
+ }
+
+ if (!symbol_conf.allow_aliases)
+ count = dso_ondemand__dedup_aliases(dso, od, elf, shdr.sh_link, count);
+
+ if (!symbol_conf.allow_aliases) {
+ for (i = 0; i + 1 < count; i++) {
+ if (od->sorted[i].end > od->sorted[i + 1].start)
+ od->sorted[i].end = od->sorted[i + 1].start;
+ }
+ }
+
+ od->data_dso = dso__new(syms_ss->name);
+ if (!od->data_dso ||
+ dso__data_set_path(od->data_dso, syms_ss->name) < 0)
+ goto out_decline_source;
+ dso__set_binary_type(od->data_dso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO);
+ dso__set_nsinfo(od->data_dso, nsinfo__get(dso__nsinfo(dso)));
+
+ if (od->sorted[0].name_off >= strshdr.sh_size)
+ goto out_decline_source;
+ if (check_add_overflow(strtab_offset,
+ (u64)od->sorted[0].name_off, &probe_off))
+ goto out_decline_source;
+ if (dso__data_read_offset(od->data_dso, NULL, probe_off, &probe, 1) != 1)
+ goto out_decline_source;
+
+ od->strtab_offset = strtab_offset;
+ od->strtab_size = strshdr.sh_size;
+ od->nr_sorted = count;
+
+ dso__set_ondemand(dso, od);
+
+ pr_debug("%s: on-demand index: %u symbols (%zu bytes, %zu bytes total) budget=%zu\n",
+ dso__long_name(dso), count,
+ od->nr_alloc * sizeof(*od->sorted), symbol__bytes_used(),
+ reservation_peak);
+
+ return 1;
+
+out_decline_source:
+ if (od->data_dso) {
+ dso__data_close(od->data_dso);
+ dso__put(od->data_dso);
+ }
+ symbol__unaccount_bytes(od->nr_alloc * sizeof(*od->sorted));
+ free(od->sorted);
+ free(od);
+ return 0;
+}
+
+const char *dso__read_ondemand_symbol_name(struct dso *data_dso,
+ u64 strtab_offset, u64 strtab_size,
+ u64 name_off, char *buf,
+ size_t buflen, char **to_free,
+ unsigned int *nr_reads)
+{
+ ssize_t n;
+ u64 remain;
+ u64 file_off;
+ size_t cap, want;
+
+ *to_free = NULL;
+
+ if (name_off >= strtab_size)
+ return NULL;
+ if (check_add_overflow(strtab_offset, name_off, &file_off))
+ return NULL;
+ remain = strtab_size - name_off;
+ if (nr_reads)
+ *nr_reads = 0;
+
+ want = min((u64)(buflen - 1), remain);
+ if (nr_reads)
+ (*nr_reads)++;
+ n = dso__data_read_offset(data_dso, NULL, file_off, (u8 *)buf, want);
+ if (n <= 0)
+ return NULL;
+ buf[n] = '\0';
+ if (memchr(buf, '\0', n))
+ return buf;
+ if ((size_t)n < want)
+ return NULL;
+
+ cap = 4096;
+ for (;;) {
+ char *tmp;
+
+ want = cap;
+ if (want > remain)
+ want = remain;
+ if (want == 0)
+ break;
+
+ tmp = *to_free ? realloc(*to_free, want + 1) : malloc(want + 1);
+ if (!tmp) {
+ free(*to_free);
+ *to_free = NULL;
+ return NULL;
+ }
+ *to_free = tmp;
+
+ if (nr_reads)
+ (*nr_reads)++;
+ n = dso__data_read_offset(data_dso, NULL, file_off,
+ (u8 *)*to_free, want);
+ if (n <= 0) {
+ free(*to_free);
+ *to_free = NULL;
+ return NULL;
+ }
+ (*to_free)[n] = '\0';
+
+ if (memchr(*to_free, '\0', n))
+ return *to_free;
+ if ((size_t)n < want)
+ break;
+
+ if (want >= remain || (u64)n >= remain)
+ break;
+
+ if (cap > SIZE_MAX / 2)
+ break;
+ cap *= 2;
+ }
+
+ free(*to_free);
+ *to_free = NULL;
+ return NULL;
+}
+
+static struct symbol *dso__materialize_symbol_ondemand(struct dso *dso, u32 pos,
+ bool *budget_exceeded)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso))
+{
+ struct dso_ondemand *od = dso__ondemand(dso);
+ struct sym_idx *idx = &od->sorted[pos];
+ const char *name;
+ char namebuf[1024];
+ char *name_heap = NULL;
+ char *demangled;
+ struct symbol *s = NULL;
+
+ *budget_exceeded = false;
+
+ if (symbol_conf.max_symbol_bytes &&
+ symbol__bytes_used() >= symbol_conf.max_symbol_bytes) {
+ symbol_budget_warning();
+ *budget_exceeded = true;
+ return NULL;
+ }
+
+ name = dso__read_ondemand_symbol_name(od->data_dso, od->strtab_offset,
+ od->strtab_size, idx->name_off,
+ namebuf, sizeof(namebuf),
+ &name_heap, NULL);
+ if (!name)
+ return NULL;
+
+ demangled = dso__demangle_sym(dso, 0, name);
+ if (demangled)
+ name = demangled;
+
+ s = symbol__new_bounded(idx->start, idx->end - idx->start,
+ idx->binding, idx->type, name, budget_exceeded);
+ free(demangled);
+ free(name_heap);
+ if (!s) {
+ if (*budget_exceeded)
+ symbol_budget_warning();
+ return NULL;
+ }
+
+ if (idx->flags & SYM_IDX_FLAG_IFUNC_ALIAS)
+ symbol__set_ifunc_alias(s, true);
+ __symbols__insert(dso__symbols(dso), s);
+ idx->flags |= SYM_IDX_FLAG_MATERIALIZED;
+ return s;
+}
+
+static struct symbol *dso__lookup_symbol_ondemand(struct dso *dso, u32 pos)
+ EXCLUSIVE_LOCKS_REQUIRED(dso__lock(dso))
+{
+ bool budget_exceeded;
+
+ return dso__materialize_symbol_ondemand(dso, pos, &budget_exceeded);
+}
+
+void dso__materialize_symbols_ondemand(struct dso *dso)
+{
+ struct dso_ondemand *od = dso__ondemand(dso);
+ bool budget_exceeded;
+ u32 i;
+
+ if (!od)
+ return;
+ for (i = 0; i < od->nr_sorted; i++) {
+ if (od->sorted[i].flags & SYM_IDX_FLAG_MATERIALIZED)
+ continue;
+ if (!dso__materialize_symbol_ondemand(dso, i, &budget_exceeded) &&
+ budget_exceeded)
+ break;
+ }
+ dso__free_ondemand(dso);
+}
+
+struct symbol *dso__find_symbol_ondemand(struct dso *dso, u64 addr)
+{
+ struct dso_ondemand *od = dso__ondemand(dso);
+ const struct sym_idx *idx;
+ u32 lo, hi, mid;
+
+ if (!od || !od->sorted || !od->data_dso)
+ return NULL;
+
+ if (!symbol_conf.allow_aliases) {
+ idx = bsearch(&addr, od->sorted, od->nr_sorted,
+ sizeof(*od->sorted), cmp_addr_to_sym_idx);
+ return idx ? dso__lookup_symbol_ondemand(dso, idx - od->sorted) : NULL;
+ }
+
+ lo = 0;
+ hi = od->nr_sorted;
+ while (lo < hi) {
+ mid = (lo + hi) / 2;
+ if (addr < od->sorted[mid].start)
+ hi = mid;
+ else if (addr >= od->sorted[mid].end)
+ lo = mid + 1;
+ else
+ return dso__lookup_symbol_ondemand(dso, mid);
+ }
+ return NULL;
+}
+
+struct symbol *dso__find_symbol_ondemand_exact(struct dso *dso, u64 addr)
+{
+ struct dso_ondemand *od = dso__ondemand(dso);
+ u32 lo, mid;
+
+ if (!od || !od->sorted || !od->data_dso)
+ return NULL;
+
+ lo = sym_idx__lower_bound(od, addr);
+ if (lo >= od->nr_sorted || od->sorted[lo].start != addr)
+ return NULL;
+ for (mid = lo; mid < od->nr_sorted &&
+ od->sorted[mid].start == addr; mid++) {
+ if (od->sorted[mid].type == STT_GNU_IFUNC ||
+ od->sorted[mid].flags & SYM_IDX_FLAG_IFUNC_ALIAS)
+ return dso__lookup_symbol_ondemand(dso, mid);
+ }
+ return dso__lookup_symbol_ondemand(dso, lo);
+}
+
static int
dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
struct symsrc *runtime_ss, int kmodule, int dynsym)
@@ -1656,6 +2290,34 @@ dso__load_sym_internal(struct dso *dso, struct map *map, struct symsrc *syms_ss,
if (kmodule && adjust_kernel_syms)
max_text_sh_offset = max_text_section(runtime_ss->elf, &runtime_ss->ehdr);
+ /*
+ * PPC64 ELFv1 function symbols need the eager loop's .opd descriptor
+ * translation. For symtabs, the selected and runtime sources can differ.
+ */
+ if (symbol_conf.lazy_load_symbols && !dso__kernel(dso) && !kmodule &&
+ !syms_ss->opdsec && (dynsym || !runtime_ss->opdsec)) {
+ int oret = 0;
+
+ if (!dynsym && syms_ss->symtab)
+ oret = dso__build_ondemand_index(dso, syms_ss,
+ runtime_ss, 0);
+ else if (dynsym && !dso__ondemand(dso) && syms_ss->dynsym)
+ oret = dso__build_ondemand_index(dso, syms_ss,
+ runtime_ss, 1);
+
+ /*
+ * On hard error, propagate it. If an index was built, the
+ * DSO resolves on demand; skip the eager loop below. If the
+ * build declined (oret == 0, no index -- e.g. no usable
+ * symbols, or no reopenable data source), continue with the eager
+ * loop so the DSO still gets symbols.
+ */
+ if (oret < 0)
+ return oret;
+ if (dso__ondemand(dso))
+ return 1;
+ }
+
curr_dso = dso__get(dso);
elf_symtab__for_each_symbol(syms, nr_syms, idx, sym) {
struct symbol *f;
diff --git a/tools/perf/util/symbol-minimal.c b/tools/perf/util/symbol-minimal.c
index 0a71d1463952..b932ced5f878 100644
--- a/tools/perf/util/symbol-minimal.c
+++ b/tools/perf/util/symbol-minimal.c
@@ -373,6 +373,22 @@ void symbol__elf_init(void)
{
}
+struct symbol *dso__find_symbol_ondemand(struct dso *dso __maybe_unused,
+ u64 addr __maybe_unused)
+{
+ return NULL;
+}
+
+struct symbol *dso__find_symbol_ondemand_exact(struct dso *dso __maybe_unused,
+ u64 addr __maybe_unused)
+{
+ return NULL;
+}
+
+void dso__materialize_symbols_ondemand(struct dso *dso __maybe_unused)
+{
+}
+
bool filename__has_section(const char *filename __maybe_unused, const char *sec __maybe_unused)
{
return false;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index cf92a7604a67..d234f7c29cb2 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -773,6 +773,7 @@ void dso__sort_by_name(struct dso *dso)
if (!dso__sorted_by_name(dso)) {
size_t len = 0;
+ dso__materialize_symbols_ondemand(dso);
dso__set_symbol_names(dso, symbols__sort_by_name(dso__symbols(dso), &len));
if (dso__symbol_names(dso)) {
dso__set_symbol_names_len(dso, len);
@@ -2019,11 +2020,19 @@ int dso__load(struct dso *dso, struct map *map)
}
#ifdef HAVE_LIBBFD_SUPPORT
+#ifdef HAVE_LIBELF_SUPPORT
+ if (is_reg && !symbol_conf.lazy_load_symbols)
+#else
if (is_reg)
+#endif
bfdrc = dso__load_bfd_symbols(dso, name);
#endif
if (is_reg && bfdrc < 0)
sirc = symsrc__init(ss, dso, name, symtab_type);
+#if defined(HAVE_LIBBFD_SUPPORT) && defined(HAVE_LIBELF_SUPPORT)
+ if (is_reg && symbol_conf.lazy_load_symbols && sirc < 0)
+ bfdrc = dso__load_bfd_symbols(dso, name);
+#endif
if (nsexit)
nsinfo__mountns_enter(dso__nsinfo(dso), &nsc);
diff --git a/tools/perf/util/symbol_conf.h b/tools/perf/util/symbol_conf.h
index 30cbc53cbcd0..763d158bc611 100644
--- a/tools/perf/util/symbol_conf.h
+++ b/tools/perf/util/symbol_conf.h
@@ -77,6 +77,7 @@ struct symbol_conf {
no_buildid_mmap2,
guest_code,
lazy_load_kernel_maps,
+ lazy_load_symbols,
keep_exited_threads,
annotate_data_member,
annotate_data_sample,
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v3 6/6] perf test: Test lazy symbol loading and symbol memory limits
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
` (4 preceding siblings ...)
2026-09-25 19:09 ` [PATCH v3 5/6] perf script: Add --lazy-load-symbols for lazy symbol loading Alireza Haghdoost via B4 Relay
@ 2026-09-25 19:09 ` Alireza Haghdoost via B4 Relay
2026-09-25 20:20 ` [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Ian Rogers
6 siblings, 0 replies; 13+ messages in thread
From: Alireza Haghdoost via B4 Relay @ 2026-09-25 19:09 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Ian Rogers, Adrian Hunter, James Clark, Alexei Starovoitov,
Andrii Nakryiko
Cc: linux-perf-users, linux-kernel, Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
Add a perf script shell test for --lazy-load-symbols and
--max-symbol-bytes.
Record a small callchain fixture, require evidence that the controlled
perf DSO built an on-demand index, and compare only extracted occurrences
of the controlled test_loop symbol. This avoids coupling the test to
addresses, diagnostics, or architecture-specific symbols for which the
eager and lazy loaders have documented differences.
Derive the constrained lazy budget from the unlimited run's
index-reservation peak. This gives a deterministic boundary where the
index fits and later materialization reaches the limit. Also verify eager
limiting, malformed size rejection, and one-time warning behavior.
Report unsupported recording, missing controlled output, unavailable
libelf, and unavailable dependent data as skips without replacing a prior
failure. Keep helper returns safe under set -e, and cover skip-status
preservation separately.
Add unit tests for the shared duplicate-selection policy, truncated
string-table reads, lazy address and name lookup across a closed data
descriptor, and address lookups racing name lookups while the symbol
budget truncates materialization. The last test also checks that a DSO
does not change once its name-sorted array has been built.
Add a unit test that loads a DSO (perf itself, or the one given with
--dso) eagerly and lazily and compares every symbol address and name.
Run it from a shell test on a hand-built split-debuginfo binary whose
function section is NOBITS in the debug file and has no PT_LOAD in the
runtime file, so both loaders must fall back to the runtime section
header.
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
---
.../tests/shell/lazy_load_symbols_split_debug.sh | 113 +++++
tools/perf/tests/shell/script_lazy_load_symbols.sh | 278 ++++++++++++
.../tests/shell/script_lazy_load_symbols_skip.sh | 26 ++
tools/perf/tests/symbol-bytes.c | 505 +++++++++++++++++++++
4 files changed, 922 insertions(+)
diff --git a/tools/perf/tests/shell/lazy_load_symbols_split_debug.sh b/tools/perf/tests/shell/lazy_load_symbols_split_debug.sh
new file mode 100755
index 000000000000..4e6335d88377
--- /dev/null
+++ b/tools/perf/tests/shell/lazy_load_symbols_split_debug.sh
@@ -0,0 +1,113 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# lazy symbol loading with split debuginfo
+
+# Build a stripped binary whose symbols live in a --only-keep-debug file,
+# where a function's section is NOBITS (with a different sh_offset) and has
+# no PT_LOAD in the runtime ELF. Lazy and eager loading must then both fall
+# back to the runtime section header and agree on the symbol addresses.
+
+set -e
+
+err=0
+tmpdir=$(mktemp -d /tmp/__perf_test.lazy_split_debug.XXXXX)
+
+cleanup() {
+ rm -rf "${tmpdir}"
+ trap - EXIT TERM INT
+}
+
+trap_cleanup() {
+ echo "Unexpected signal in ${FUNCNAME[1]}"
+ cleanup
+ exit 1
+}
+trap trap_cleanup EXIT TERM INT
+
+skip() {
+ echo "Lazy-load split debuginfo [Skipped: $1]"
+ cleanup
+ exit 2
+}
+
+if ! perf check feature -q libelf; then
+ skip "no libelf support"
+fi
+
+for tool in cc objcopy strip readelf dd; do
+ if ! command -v "${tool}" > /dev/null; then
+ skip "${tool} not found"
+ fi
+done
+
+cat > "${tmpdir}/prog.c" << EOF
+__attribute__((section("splittext"), noinline, used))
+int split_func(int x)
+{
+ return x * 3 + 1;
+}
+
+int main(int argc, char **argv)
+{
+ (void)argv;
+ return split_func(argc);
+}
+EOF
+
+prog="${tmpdir}/prog"
+if ! cc -O1 -g -o "${prog}" "${tmpdir}/prog.c" \
+ -Wl,--section-start=splittext=0x800000 2> /dev/null; then
+ skip "cannot build test program"
+fi
+objcopy --only-keep-debug "${prog}" "${prog}.debug"
+strip -s "${prog}"
+objcopy --add-gnu-debuglink="${prog}.debug" "${prog}"
+
+# The debug file must keep splittext as NOBITS at a stale offset.
+debug_sec=$(readelf -SW "${prog}.debug" | grep ' splittext ' || true)
+run_sec=$(readelf -SW "${prog}" | grep ' splittext ' || true)
+if ! echo "${debug_sec}" | grep -q NOBITS; then
+ skip "splittext is not NOBITS in the debug file"
+fi
+debug_off=$(echo "${debug_sec}" | sed 's/.*splittext *//' | awk '{print $3}')
+run_off=$(echo "${run_sec}" | sed 's/.*splittext *//' | awk '{print $3}')
+if [ -z "${run_off}" ] || [ "${debug_off}" = "${run_off}" ]; then
+ skip "splittext offsets do not differ"
+fi
+
+# Drop the PT_LOAD covering splittext so program header lookup fails.
+phoff=$(readelf -hW "${prog}" | awk '/Start of program headers/ {print $5}')
+phentsize=$(readelf -hW "${prog}" | awk '/Size of program headers/ {print $5}')
+idx=$(readelf -lW "${prog}" | awk '
+ /^Program Headers:/ { in_ph = 1; next }
+ in_ph && /^ Type/ { next }
+ in_ph && /^ *$/ { exit }
+ in_ph && /^ [A-Z]/ {
+ if ($1 == "LOAD" && $3 ~ /^0x0*800000$/) { print n; exit }
+ n++
+ }')
+if [ -z "${phoff}" ] || [ -z "${phentsize}" ] || [ -z "${idx}" ]; then
+ skip "no PT_LOAD for splittext"
+fi
+dd if=/dev/zero of="${prog}" bs=1 seek=$((phoff + idx * phentsize)) count=4 \
+ conv=notrunc 2> /dev/null
+if readelf -lW "${prog}" 2> /dev/null | grep -q 'LOAD .*0x0*800000 '; then
+ echo "Lazy-load split debuginfo [Failed to drop PT_LOAD]"
+ err=1
+fi
+
+if [ "${err}" -eq 0 ]; then
+ export PERF_BUILDID_DIR="${tmpdir}/buildid"
+ if ! perf test --dso "${prog}" "Lazy and eager symbol parity" 2>&1 | \
+ grep -q ': Ok$'; then
+ perf test --dso "${prog}" -vv "Lazy and eager symbol parity" 2>&1 | \
+ grep -E 'symbols$|mismatch|no symbols|no lazy' || true
+ echo "Lazy-load split debuginfo [Failed parity]"
+ err=1
+ else
+ echo "Lazy-load split debuginfo [Success]"
+ fi
+fi
+
+cleanup
+exit ${err}
diff --git a/tools/perf/tests/shell/script_lazy_load_symbols.sh b/tools/perf/tests/shell/script_lazy_load_symbols.sh
new file mode 100755
index 000000000000..7e193c1c3d1f
--- /dev/null
+++ b/tools/perf/tests/shell/script_lazy_load_symbols.sh
@@ -0,0 +1,278 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# perf script lazy symbol loading tests (exclusive)
+#
+# Verifies that --lazy-load-symbols matches the default eager loader for a
+# controlled symbol, and that --max-symbol-bytes caps symbol allocations
+# (emitting [unknown] plus a warning) without crashing.
+
+mark_skip() {
+ if [ "${err}" -eq 0 ]; then
+ err=2
+ fi
+ return 0
+}
+
+if [ "${PERF_LAZY_LOAD_SYMBOLS_TEST_HELPERS:-}" = 1 ]; then
+ return 0
+fi
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/perf_has_symbol.sh
+. "${shelldir}"/lib/perf_has_symbol.sh
+
+testsym="test_loop"
+perf_path=$(readlink -f "$(command -v perf)")
+lazy_index_budget=
+
+skip_test_missing_symbol ${testsym}
+
+if ! perf check feature -q libelf
+then
+ echo "Lazy symbol loading [Skipped no libelf support]"
+ exit 2
+fi
+
+err=0
+temp_dir=$(mktemp -d /tmp/__perf_test.lazy_load.XXXXX)
+perfdata="${temp_dir}/perf.data"
+eager_out="${temp_dir}/eager.out"
+lazy_out="${temp_dir}/lazy.out"
+lazy_err="${temp_dir}/lazy.err"
+eager_sym_out="${temp_dir}/eager.sym.out"
+lazy_sym_out="${temp_dir}/lazy.sym.out"
+
+cleanup() {
+ rm -rf "${temp_dir}"
+ trap - EXIT TERM INT
+}
+
+trap_cleanup() {
+ echo "Unexpected signal in ${FUNCNAME[1]}"
+ cleanup
+ exit 1
+}
+trap trap_cleanup EXIT TERM INT
+
+test_lazy_load_identical() {
+ echo "Lazy-load output matches eager loader"
+
+ # Record a small profile with callchains so symbol resolution runs.
+ if ! perf record -o "${perfdata}" -g -- perf test -w thloop 2> /dev/null
+ then
+ echo "Lazy-load identical [Skipped record not supported]"
+ mark_skip
+ return 0
+ fi
+
+ if ! perf script -i "${perfdata}" 2> /dev/null > "${eager_out}" || \
+ ! perf script -v --lazy-load-symbols -i "${perfdata}" \
+ 2> "${lazy_err}" > "${lazy_out}"
+ then
+ echo "Lazy-load identical [Failed perf script error]"
+ err=1
+ return
+ fi
+ if ! grep -q "on-demand index:" "${lazy_err}"
+ then
+ echo "Lazy-load identical [Failed lazy loader fell back to eager]"
+ err=1
+ return
+ fi
+ lazy_index_budget=$(awk -v dso="${perf_path}: on-demand index:" \
+ 'index($0, dso) { sub(/^.* budget=/, ""); print; exit }' \
+ "${lazy_err}")
+ case "${lazy_index_budget}" in
+ ''|*[!0-9]*)
+ echo "Lazy-load identical [Failed controlled DSO has no index]"
+ err=1
+ return
+ ;;
+ esac
+
+ # The comparison is only meaningful if something actually resolved;
+ # two all-[unknown] outputs would also match.
+ if ! grep -q "${testsym}" "${eager_out}"
+ then
+ echo "Lazy-load identical [Skipped no ${testsym} resolved]"
+ mark_skip
+ return 0
+ fi
+
+ grep -w -o "${testsym}" "${eager_out}" > "${eager_sym_out}"
+ if ! grep -w -o "${testsym}" "${lazy_out}" > "${lazy_sym_out}"
+ then
+ echo "Lazy-load identical [Failed no lazy ${testsym} resolved]"
+ err=1
+ return
+ fi
+
+ if ! cmp -s "${eager_sym_out}" "${lazy_sym_out}"
+ then
+ echo "Lazy-load identical [Failed ${testsym} output differs]"
+ err=1
+ return
+ fi
+ echo "Lazy-load identical [Success]"
+}
+
+test_max_symbol_bytes() {
+ echo "--max-symbol-bytes budget enforcement"
+
+ # Depends on ${perfdata} from test_lazy_load_identical.
+ if [ ! -s "${perfdata}" ]
+ then
+ echo "--max-symbol-bytes budget [Skipped record not supported]"
+ mark_skip
+ return 0
+ fi
+
+ # A tiny budget forces most symbols to be dropped as [unknown],
+ # with a single warning, and must not crash.
+ if ! perf script --max-symbol-bytes=1K -i "${perfdata}" > /dev/null \
+ 2> "${temp_dir}/budget.err"
+ then
+ echo "--max-symbol-bytes budget [Failed nonzero exit]"
+ err=1
+ return
+ fi
+ if ! grep -q "symbol memory budget exceeded" "${temp_dir}/budget.err"
+ then
+ echo "--max-symbol-bytes budget [Failed missing warning]"
+ err=1
+ return
+ fi
+ if perf script --max-symbol-bytes=1Kjunk -i "${perfdata}" \
+ > /dev/null 2>&1
+ then
+ echo "--max-symbol-bytes budget [Failed malformed size accepted]"
+ err=1
+ return
+ fi
+ if ! perf script --max-symbol-bytes=0 -i "${perfdata}" \
+ > /dev/null 2>&1
+ then
+ echo "--max-symbol-bytes budget [Failed zero not accepted]"
+ err=1
+ return
+ fi
+
+ # The unlimited run logged the peak accounted bytes at the controlled
+ # DSO's index reservation, before alias dedup may have released bytes.
+ # Reuse that peak as the budget: deterministic index construction fits,
+ # while subsequent materialization must hit the limit.
+ if ! perf script -v --lazy-load-symbols \
+ --max-symbol-bytes="${lazy_index_budget}B" \
+ -i "${perfdata}" > /dev/null 2> "${temp_dir}/lazy-budget.err"
+ then
+ echo "--max-symbol-bytes lazy budget [Failed nonzero exit]"
+ err=1
+ return
+ fi
+ if ! grep -Fq "${perf_path}: on-demand index:" \
+ "${temp_dir}/lazy-budget.err" ||
+ ! grep -q "symbol memory budget exceeded" "${temp_dir}/lazy-budget.err"
+ then
+ echo "--max-symbol-bytes lazy budget [Failed no indexed budget case]"
+ err=1
+ return
+ fi
+ warnings=$(grep -c "symbol memory budget exceeded" \
+ "${temp_dir}/lazy-budget.err" || true)
+ if [ "${warnings}" -ne 1 ]
+ then
+ echo "--max-symbol-bytes lazy budget [Failed warning count: ${warnings}]"
+ err=1
+ return
+ fi
+ echo "--max-symbol-bytes budget [Success]"
+}
+
+test_budget_truncation_range() {
+ local longsym
+ local first_symbol
+ local trunc_source="${temp_dir}/truncation.S"
+ local trunc_binary="${temp_dir}/truncation"
+ local trunc_data="${temp_dir}/truncation.data"
+ local trunc_out="${temp_dir}/truncation.out"
+ local trunc_err="${temp_dir}/truncation.err"
+
+ echo "--max-symbol-bytes truncation range"
+
+ if [ "$(uname -m)" != x86_64 ]; then
+ echo "--max-symbol-bytes truncation range [Skipped x86_64 only]"
+ mark_skip
+ return 0
+ fi
+
+ longsym=$(printf 's%.0s' {1..900})
+ cat > "${trunc_source}" <<EOF
+ .text
+ .globl ${longsym}
+ .type ${longsym}, @function
+${longsym}:
+ call omitted_symbol
+ mov \$60, %eax
+ xor %edi, %edi
+ syscall
+
+ .globl omitted_symbol
+ .type omitted_symbol, @function
+omitted_symbol:
+ mov \$500000000, %ecx
+1:
+ dec %ecx
+ jnz 1b
+ ret
+ .size omitted_symbol, .-omitted_symbol
+EOF
+ if ! cc -nostdlib -no-pie -Wl,--build-id=none -Wl,-e,"${longsym}" \
+ -o "${trunc_binary}" "${trunc_source}"
+ then
+ echo "--max-symbol-bytes truncation range [Skipped compiler unsupported]"
+ mark_skip
+ return 0
+ fi
+
+ first_symbol=$(readelf -W -s "${trunc_binary}" |
+ awk '$4 == "FUNC" && $7 != "UND" { print $8; exit }')
+ if [ "${first_symbol}" != "${longsym}" ]; then
+ echo "--max-symbol-bytes truncation range [Skipped unexpected symbol order]"
+ mark_skip
+ return 0
+ fi
+
+ if ! perf record -o "${trunc_data}" -e cycles:u -F 1000 -- \
+ "${trunc_binary}" 2> /dev/null
+ then
+ echo "--max-symbol-bytes truncation range [Skipped record not supported]"
+ mark_skip
+ return 0
+ fi
+ if ! perf script --max-symbol-bytes=1K -i "${trunc_data}" -F ip,sym,dso \
+ > "${trunc_out}" 2> "${trunc_err}"
+ then
+ echo "--max-symbol-bytes truncation range [Failed perf script error]"
+ err=1
+ return
+ fi
+
+ if ! grep -q "symbol memory budget exceeded" "${trunc_err}" ||
+ ! grep -F "${trunc_binary}" "${trunc_out}" | grep -q '\[unknown\]' ||
+ grep -Fq "${longsym}" "${trunc_out}"
+ then
+ echo "--max-symbol-bytes truncation range [Failed omitted range resolved]"
+ err=1
+ return
+ fi
+ echo "--max-symbol-bytes truncation range [Success]"
+}
+
+test_lazy_load_identical
+test_max_symbol_bytes
+test_budget_truncation_range
+
+cleanup
+exit $err
diff --git a/tools/perf/tests/shell/script_lazy_load_symbols_skip.sh b/tools/perf/tests/shell/script_lazy_load_symbols_skip.sh
new file mode 100755
index 000000000000..136503863fdd
--- /dev/null
+++ b/tools/perf/tests/shell/script_lazy_load_symbols_skip.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# perf script lazy symbol loading skip status
+
+set -e
+
+shelldir=$(dirname "$0")
+PERF_LAZY_LOAD_SYMBOLS_TEST_HELPERS=1
+. "${shelldir}"/script_lazy_load_symbols.sh
+unset PERF_LAZY_LOAD_SYMBOLS_TEST_HELPERS
+
+err=0
+mark_skip
+if [ "${err}" -ne 2 ]; then
+ echo "Lazy-load skip status [Failed expected 2, got ${err}]"
+ exit 1
+fi
+
+err=1
+mark_skip
+if [ "${err}" -ne 1 ]; then
+ echo "Lazy-load skip status [Failed skip overwrote failure: ${err}]"
+ exit 1
+fi
+
+echo "Lazy-load skip status [Success]"
diff --git a/tools/perf/tests/symbol-bytes.c b/tools/perf/tests/symbol-bytes.c
index d3055f5ed321..eef14e6e573b 100644
--- a/tools/perf/tests/symbol-bytes.c
+++ b/tools/perf/tests/symbol-bytes.c
@@ -1,12 +1,27 @@
// SPDX-License-Identifier: GPL-2.0
+#include <fcntl.h>
+#include <inttypes.h>
#include <limits.h>
#include <pthread.h>
#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+#include <linux/kernel.h>
+#include <linux/zalloc.h>
#include "debug.h"
+#include "dso.h"
+#include "env.h"
+#include "machine.h"
+#include "map.h"
#include "symbol.h"
#include "symbol_conf.h"
#include "tests.h"
+#include "thread.h"
+#include "util.h"
struct reserve_arg {
size_t bytes;
@@ -83,8 +98,498 @@ static int test__symbol_bytes_reservation(struct test_suite *test __maybe_unused
return ret;
}
+static int test__symbol_bytes_duplicate_selection(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ struct duplicate_case {
+ struct symbol_candidate a, b;
+ int expected;
+ } cases[] = {
+ { { 1, "a", STT_FUNC, STB_GLOBAL }, { 0, "b", STT_FUNC, STB_GLOBAL }, SYMBOL_A },
+ { { 1, "a", STT_NOTYPE, STB_GLOBAL }, { 1, "b", STT_FUNC, STB_GLOBAL }, SYMBOL_B },
+ { { 1, "a", STT_FUNC, STB_WEAK }, { 1, "b", STT_FUNC, STB_GLOBAL }, SYMBOL_B },
+ { { 1, "a", STT_FUNC, STB_GLOBAL }, { 1, "b", STT_FUNC, STB_LOCAL }, SYMBOL_A },
+ { { 1, "name", STT_FUNC, STB_GLOBAL },
+ { 1, "_name", STT_FUNC, STB_GLOBAL }, SYMBOL_A },
+ { { 1, "a", STT_FUNC, STB_GLOBAL }, { 1, "long", STT_FUNC, STB_GLOBAL }, SYMBOL_B },
+ };
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(cases); i++) {
+ if (symbol__choose_best(&cases[i].a, &cases[i].b) != cases[i].expected)
+ return TEST_FAIL;
+ }
+ return TEST_OK;
+}
+
+#ifdef HAVE_LIBELF_SUPPORT
+static int truncated_name_case(size_t file_size, unsigned int expected_reads)
+{
+ char path[] = "/tmp/perf-lazy-truncated-XXXXXX";
+ struct dso *data_dso = NULL;
+ char *contents = NULL;
+ char *name_heap = NULL;
+ char namebuf[1024];
+ const char *name;
+ unsigned int nr_reads;
+ int ret = TEST_FAIL;
+ int fd = -1;
+
+ contents = malloc(file_size);
+ if (!contents)
+ goto out;
+ memset(contents, 'a', file_size);
+
+ fd = mkstemp(path);
+ if (fd < 0 || write(fd, contents, file_size) != (ssize_t)file_size)
+ goto out;
+ close(fd);
+ fd = -1;
+
+ data_dso = dso__new(path);
+ if (!data_dso || dso__data_set_path(data_dso, path) < 0)
+ goto out;
+ dso__set_binary_type(data_dso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO);
+ name = dso__read_ondemand_symbol_name(data_dso, 0, 8192, 0,
+ namebuf, sizeof(namebuf),
+ &name_heap, &nr_reads);
+ if (name || name_heap || nr_reads != expected_reads)
+ goto out;
+ ret = TEST_OK;
+out:
+ if (fd >= 0)
+ close(fd);
+ if (data_dso)
+ dso__put(data_dso);
+ unlink(path);
+ free(name_heap);
+ free(contents);
+ return ret;
+}
+
+static int test__symbol_bytes_truncated_name(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ /*
+ * One byte is short in the stack-buffer read. 1023 bytes fills it
+ * exactly, so the following read exercises the heap-buffer path.
+ */
+ if (truncated_name_case(1, 1) != TEST_OK ||
+ truncated_name_case(1023, 2) != TEST_OK)
+ return TEST_FAIL;
+ return TEST_OK;
+}
+
+#define LAZY_SYM_START 0x1000
+#define LAZY_SYM_SIZE 0x10
+#define LAZY_NAME_FMT "lazy_sym_%03u"
+#define LAZY_NAME_LEN sizeof("lazy_sym_000")
+
+static void lazy_name(char *buf, u32 i)
+{
+ snprintf(buf, LAZY_NAME_LEN, LAZY_NAME_FMT, i % 1000);
+}
+
+struct lazy_fixture {
+ char path[32];
+ struct dso *dso;
+ struct map *map;
+ u32 nr;
+};
+
+/*
+ * Build a DSO whose symbols exist only in a lazy index: @nr adjacent
+ * functions named lazy_sym_NNN, with names in a string-table file read
+ * through a private data DSO.
+ */
+static int lazy_fixture__init(struct lazy_fixture *f, u32 nr)
+{
+ struct dso_ondemand *od = NULL;
+ char *strtab;
+ int fd, ret = -1;
+ u32 i;
+
+ memset(f, 0, sizeof(*f));
+ f->nr = nr;
+ strcpy(f->path, "/tmp/perf-lazy-names-XXXXXX");
+
+ strtab = malloc(nr * LAZY_NAME_LEN);
+ if (!strtab)
+ return -1;
+ for (i = 0; i < nr; i++)
+ lazy_name(strtab + i * LAZY_NAME_LEN, i);
+
+ fd = mkstemp(f->path);
+ if (fd < 0) {
+ f->path[0] = '\0';
+ goto out;
+ }
+ if (write(fd, strtab, nr * LAZY_NAME_LEN) != (ssize_t)(nr * LAZY_NAME_LEN)) {
+ close(fd);
+ goto out;
+ }
+ close(fd);
+
+ f->dso = dso__new("/not/the/symbol/source");
+ od = zalloc(sizeof(*od));
+ if (!f->dso || !od)
+ goto out;
+ od->sorted = calloc(nr, sizeof(*od->sorted));
+ od->data_dso = dso__new(f->path);
+ if (!od->sorted || !od->data_dso ||
+ dso__data_set_path(od->data_dso, f->path) < 0)
+ goto out;
+ dso__set_binary_type(od->data_dso, DSO_BINARY_TYPE__SYSTEM_PATH_DSO);
+ od->strtab_size = nr * LAZY_NAME_LEN;
+ od->nr_sorted = nr;
+ od->nr_alloc = nr;
+ for (i = 0; i < nr; i++) {
+ od->sorted[i] = (struct sym_idx) {
+ .start = LAZY_SYM_START + i * LAZY_SYM_SIZE,
+ .end = LAZY_SYM_START + (i + 1) * LAZY_SYM_SIZE,
+ .name_off = i * LAZY_NAME_LEN,
+ .binding = STB_GLOBAL,
+ .type = STT_FUNC,
+ };
+ }
+ if (!symbol__try_account_bytes(od->nr_alloc * sizeof(*od->sorted)))
+ goto out;
+ dso__set_ondemand(f->dso, od);
+ od = NULL;
+ dso__set_loaded(f->dso);
+ f->map = map__new2(0, f->dso);
+ if (f->map)
+ ret = 0;
+out:
+ if (od) {
+ if (od->data_dso)
+ dso__put(od->data_dso);
+ free(od->sorted);
+ free(od);
+ }
+ free(strtab);
+ return ret;
+}
+
+static void lazy_fixture__exit(struct lazy_fixture *f)
+{
+ map__put(f->map);
+ dso__put(f->dso);
+ if (f->path[0])
+ unlink(f->path);
+}
+
+static bool lazy_symbol_ok(const struct symbol *sym, u32 i)
+{
+ char name[LAZY_NAME_LEN];
+
+ lazy_name(name, i);
+ return sym->start == LAZY_SYM_START + i * LAZY_SYM_SIZE && !strcmp(sym->name, name);
+}
+
+static int lazy_nr_symbols(struct dso *dso)
+{
+ struct rb_node *node;
+ int nr = 0;
+
+ for (node = rb_first_cached(dso__symbols(dso)); node; node = rb_next(node))
+ nr++;
+ return nr;
+}
+
+static int test__symbol_bytes_lazy_name_lookup(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ unsigned long saved_max = symbol_conf.max_symbol_bytes;
+ bool saved_lazy = symbol_conf.lazy_load_symbols;
+ size_t baseline = symbol__bytes_used();
+ struct lazy_fixture f;
+ struct symbol *sym;
+ int ret = TEST_FAIL;
+
+ symbol_conf.max_symbol_bytes = 0;
+ symbol_conf.lazy_load_symbols = true;
+ if (lazy_fixture__init(&f, 2))
+ goto out;
+
+ sym = map__find_symbol(f.map, LAZY_SYM_START + 1);
+ if (!sym || !lazy_symbol_ok(sym, 0))
+ goto out;
+
+ /* Name lookup must still work after the data descriptor is closed. */
+ dso__data_close(dso__ondemand(f.dso)->data_dso);
+ sym = map__find_symbol_by_name(f.map, "lazy_sym_001");
+ if (!sym || !lazy_symbol_ok(sym, 1) || dso__ondemand(f.dso))
+ goto out;
+ if (lazy_nr_symbols(f.dso) != 2)
+ goto out;
+ ret = TEST_OK;
+out:
+ lazy_fixture__exit(&f);
+ symbol_conf.max_symbol_bytes = saved_max;
+ symbol_conf.lazy_load_symbols = saved_lazy;
+ if (symbol__bytes_used() != baseline)
+ ret = TEST_FAIL;
+ return ret;
+}
+
+struct lazy_lookup_arg {
+ struct lazy_fixture *f;
+ bool by_name;
+ bool failed;
+};
+
+static void *lazy_lookup(void *data)
+{
+ struct lazy_lookup_arg *arg = data;
+ struct lazy_fixture *f = arg->f;
+ u32 i, round;
+
+ for (round = 0; round < 16; round++) {
+ for (i = 0; i < f->nr; i++) {
+ struct symbol *sym;
+
+ if (arg->by_name) {
+ char name[LAZY_NAME_LEN];
+
+ lazy_name(name, i);
+ sym = map__find_symbol_by_name(f->map, name);
+ } else {
+ sym = map__find_symbol(f->map, LAZY_SYM_START +
+ i * LAZY_SYM_SIZE + 1);
+ }
+ if (sym && !lazy_symbol_ok(sym, i))
+ arg->failed = true;
+ }
+ }
+ return NULL;
+}
+
+/*
+ * Address lookups race with name lookups while the symbol budget stops
+ * materialization part way. Once a name lookup has run, the DSO must no
+ * longer grow, even after budget becomes available again.
+ */
+static int test__symbol_bytes_lazy_budget_race(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ enum { NR_SYMS = 256, NR_ADDR_THREADS = 4, NR_THREADS = NR_ADDR_THREADS + 2 };
+ unsigned long saved_max = symbol_conf.max_symbol_bytes;
+ bool saved_lazy = symbol_conf.lazy_load_symbols;
+ size_t baseline = symbol__bytes_used();
+ size_t charge = symbol_conf.priv_size + sizeof(struct symbol) + LAZY_NAME_LEN;
+ struct lazy_lookup_arg args[NR_THREADS];
+ pthread_t threads[NR_THREADS];
+ struct lazy_fixture f;
+ int created = 0, nr_symbols, i;
+ int ret = TEST_FAIL;
+
+ symbol_conf.max_symbol_bytes = 0;
+ symbol_conf.lazy_load_symbols = true;
+ if (lazy_fixture__init(&f, NR_SYMS))
+ goto out;
+ symbol_conf.max_symbol_bytes = symbol__bytes_used() + NR_SYMS / 4 * charge;
+
+ for (i = 0; i < NR_THREADS; i++) {
+ args[i] = (struct lazy_lookup_arg) {
+ .f = &f,
+ .by_name = i >= NR_ADDR_THREADS,
+ };
+ if (pthread_create(&threads[i], NULL, lazy_lookup, &args[i]))
+ break;
+ created++;
+ }
+ for (i = 0; i < created; i++)
+ pthread_join(threads[i], NULL);
+ if (created != NR_THREADS)
+ goto out;
+ for (i = 0; i < NR_THREADS; i++) {
+ if (args[i].failed) {
+ pr_debug("lazy lookup returned a wrong symbol\n");
+ goto out;
+ }
+ }
+
+ nr_symbols = lazy_nr_symbols(f.dso);
+ if (dso__ondemand(f.dso) || nr_symbols == 0 || nr_symbols > NR_SYMS / 4) {
+ pr_debug("unexpected lazy state: index %p, %d symbols\n",
+ dso__ondemand(f.dso), nr_symbols);
+ goto out;
+ }
+
+ symbol_conf.max_symbol_bytes = 0;
+ for (i = 0; i < NR_SYMS; i++)
+ map__find_symbol(f.map, LAZY_SYM_START + i * LAZY_SYM_SIZE + 1);
+ if (lazy_nr_symbols(f.dso) != nr_symbols) {
+ pr_debug("DSO changed after its name array was built\n");
+ goto out;
+ }
+ ret = TEST_OK;
+out:
+ lazy_fixture__exit(&f);
+ symbol_conf.max_symbol_bytes = saved_max;
+ symbol_conf.lazy_load_symbols = saved_lazy;
+ if (symbol__bytes_used() != baseline)
+ ret = TEST_FAIL;
+ return ret;
+}
+struct sym_entry {
+ u64 start;
+ char *name;
+};
+
+static int cmp_sym_entry(const void *a, const void *b)
+{
+ const struct sym_entry *sa = a, *sb = b;
+
+ if (sa->start != sb->start)
+ return sa->start < sb->start ? -1 : 1;
+ return strcmp(sa->name, sb->name);
+}
+
+static void free_sym_entries(struct sym_entry *entries, size_t nr)
+{
+ size_t i;
+
+ for (i = 0; i < nr; i++)
+ free(entries[i].name);
+ free(entries);
+}
+
+/*
+ * Load @filename on a fresh host machine and return the start and name of
+ * every symbol, sorted. In lazy mode, building the name-sorted array
+ * materializes the whole index.
+ */
+static int load_sym_entries(const char *filename, bool lazy,
+ struct sym_entry **entries_p, size_t *nr_p)
+{
+ struct sym_entry *entries = NULL;
+ struct machine *machine = NULL;
+ struct thread *thread = NULL;
+ struct map *map = NULL;
+ struct perf_env env;
+ struct rb_node *nd;
+ struct dso *dso;
+ size_t nr = 0, alloc = 0;
+ int ret = TEST_FAIL;
+
+ perf_env__init(&env);
+ symbol_conf.lazy_load_symbols = lazy;
+ machine = machine__new_host(&env);
+ if (!machine)
+ goto out;
+ thread = machine__findnew_thread(machine, 100, 100);
+ if (!thread)
+ goto out;
+ map = map__new(machine, 0x100000, 0xffffffff, 0, &dso_id_empty,
+ PROT_EXEC, /*flags=*/0, (char *)filename, thread);
+ if (!map)
+ goto out;
+
+ dso = map__dso(map);
+ if (dso__load(dso, map) <= 0) {
+ pr_debug("%s: no symbols loaded\n", filename);
+ ret = TEST_SKIP;
+ goto out;
+ }
+ if (lazy && !dso__ondemand(dso)) {
+ pr_debug("%s: no lazy index was built\n", filename);
+ ret = TEST_SKIP;
+ goto out;
+ }
+ dso__sort_by_name(dso);
+
+ for (nd = rb_first_cached(dso__symbols(dso)); nd; nd = rb_next(nd)) {
+ struct symbol *sym = rb_entry(nd, struct symbol, rb_node);
+
+ if (nr == alloc) {
+ struct sym_entry *tmp;
+
+ alloc = alloc ? alloc * 2 : 1024;
+ tmp = realloc(entries, alloc * sizeof(*entries));
+ if (!tmp)
+ goto out;
+ entries = tmp;
+ }
+ entries[nr].start = sym->start;
+ entries[nr].name = strdup(sym->name);
+ if (!entries[nr].name)
+ goto out;
+ nr++;
+ }
+ qsort(entries, nr, sizeof(*entries), cmp_sym_entry);
+ *entries_p = entries;
+ *nr_p = nr;
+ entries = NULL;
+ ret = TEST_OK;
+out:
+ if (entries)
+ free_sym_entries(entries, nr);
+ map__put(map);
+ thread__put(thread);
+ machine__delete(machine);
+ perf_env__exit(&env);
+ return ret;
+}
+
+/*
+ * Compare the symbols of a DSO (perf itself, or --dso) loaded eagerly and
+ * lazily. Addresses and names must match.
+ */
+static int test__symbol_bytes_lazy_parity(struct test_suite *test __maybe_unused,
+ int subtest __maybe_unused)
+{
+ unsigned long saved_max = symbol_conf.max_symbol_bytes;
+ bool saved_lazy = symbol_conf.lazy_load_symbols;
+ struct sym_entry *eager = NULL, *lazy = NULL;
+ size_t nr_eager = 0, nr_lazy = 0, i;
+ char filename[PATH_MAX];
+ int ret;
+
+ if (dso_to_test)
+ strlcpy(filename, dso_to_test, sizeof(filename));
+ else
+ perf_exe(filename, sizeof(filename));
+
+ symbol_conf.max_symbol_bytes = 0;
+ ret = load_sym_entries(filename, false, &eager, &nr_eager);
+ if (ret == TEST_OK)
+ ret = load_sym_entries(filename, true, &lazy, &nr_lazy);
+ if (ret != TEST_OK)
+ goto out;
+
+ pr_debug("%s: %zu eager and %zu lazy symbols\n", filename, nr_eager, nr_lazy);
+ for (i = 0; i < nr_eager && i < nr_lazy; i++) {
+ if (cmp_sym_entry(&eager[i], &lazy[i])) {
+ pr_debug("mismatch: eager %#" PRIx64 " %s, lazy %#" PRIx64 " %s\n",
+ eager[i].start, eager[i].name, lazy[i].start, lazy[i].name);
+ ret = TEST_FAIL;
+ goto out;
+ }
+ }
+ if (nr_eager != nr_lazy)
+ ret = TEST_FAIL;
+out:
+ if (eager)
+ free_sym_entries(eager, nr_eager);
+ if (lazy)
+ free_sym_entries(lazy, nr_lazy);
+ symbol_conf.max_symbol_bytes = saved_max;
+ symbol_conf.lazy_load_symbols = saved_lazy;
+ return ret;
+}
+#endif
+
static struct test_case tests__symbol_bytes[] = {
TEST_CASE("Concurrent strict reservations", symbol_bytes_reservation),
+ TEST_CASE("Shared duplicate selection", symbol_bytes_duplicate_selection),
+#ifdef HAVE_LIBELF_SUPPORT
+ TEST_CASE("Truncated lazy symbol names", symbol_bytes_truncated_name),
+ TEST_CASE("Lazy address and name lookup", symbol_bytes_lazy_name_lookup),
+ TEST_CASE("Lazy lookups racing a truncated name lookup", symbol_bytes_lazy_budget_race),
+ TEST_CASE("Lazy and eager symbol parity", symbol_bytes_lazy_parity),
+#endif
{ .name = NULL, }
};
--
Git-157)
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3 0/6] perf script: Bounded and lazy symbol loading
2026-09-25 19:09 [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Alireza Haghdoost via B4 Relay
` (5 preceding siblings ...)
2026-09-25 19:09 ` [PATCH v3 6/6] perf test: Test lazy symbol loading and symbol memory limits Alireza Haghdoost via B4 Relay
@ 2026-09-25 20:20 ` Ian Rogers
2026-09-25 21:28 ` Alireza Haghdoost
6 siblings, 1 reply; 13+ messages in thread
From: Ian Rogers @ 2026-09-25 20:20 UTC (permalink / raw)
To: haghdoost
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
On Fri, Sep 25, 2026 at 12:15 PM Alireza Haghdoost via B4 Relay
<devnull+haghdoost.uber.com@kernel.org> wrote:
>
> perf script loads the entire ELF symbol table of every DSO that appears
> in a sample, allocating each symbol into an rb-tree held until process
> exit. Therefore, a large enough profile turns symbol loading into an
> OOM kill. This does not scale to profiling a large cgroup with many large
> binaries on a production system with limited free memory.
>
> This series adds two independent, opt-in mechanisms, a leading
> regression fix, and two preparatory patches:
>
> [1/6] Fix a broken "#ifdef ELF_C_READ_MMAP" guard so perf actually
> mmaps ELF files instead of malloc'ing section data. This is a
> standalone regression fix for 22dd1ac91a77.
>
> [2/6] Let a DSO read its data from one explicit file through the DSO
> data cache. This fixes the split-debuginfo case where offsets from
> the debuginfo file would be applied to the runtime image.
>
> [3/6] Factor duplicate-symbol selection so it works on symbol
> attributes rather than struct symbol. No functional change.
>
> [4/6] --max-symbol-bytes <size>: a byte budget on struct symbol
> allocations (and the lazy index) enforced at the ELF symbol
> loader, degrading to [unknown] with a warning past the cap.
> An unbounded profile doesn't just risk OOM-killing itself. It
> also forces memory pressure on the whole host, pushing the kernel
> to reclaim from co-located latency-sensitive processes. Capping
> it lets the user bound that footprint up front and choose the
> trade-off explicitly.
>
> [5/6] --lazy-load-symbols: build a compact per-DSO sorted index and
> resolve only the sampled addresses, reading names through the DSO
> data cache at lookup time. On the production fixture, peak RssAnon
> drops from 265 MiB to 39 MiB (6.8x) and wall time from 3.1 s to
> 1.85 s (1.7x). Memory optimizations usually cost time; this one
> does not because lazy loading skips a lot of calloc and demangle
> calls.
>
> [6/6] Shell and unit tests for both options.
>
> Lazy loading handles the common userspace ELF symtab/dynsym path. Eager
> loading remains available for dense coverage and for PPC64 .opd and
> .gnu_debugdata.
So currently we have a symbol as:
```
/**
* A symtab entry. When allocated this may be preceded by an annotation (see
* symbol__annotation) and/or a browser_index (see symbol__browser_index).
*/
struct symbol {
struct rb_node rb_node;
/** Range of symbol [start, end). */
u64 start;
u64 end;
/** Length of the string name. */
u16 namelen;
_Atomic uint16_t flags;
/** Architecture specific. Unused except on PPC where it holds
st_other. */
u8 arch_sym;
/** The name of length namelen associated with the symbol. */
char name[];
};
```
and the struct rb_node is a significant size cost:
```
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long))));
```
A sorted array of symbols would only require 1 pointer per entry,
saving 2 words. The array could reside in the "symbols" variable
passed around something like this:
```
struct symbols {
struct rw_semaphore lock;
struct symbol **symbols;
unsigned int cnt;
unsigned int allocated;
bool sorted;
};
```
This is basically what we do with struct dso and dsos. With a struct
symbols we can add a lock and ensure operations are appropriately
synchronized. On top of this we can build lazy symbol resolution. For
the bounded memory size, I'd prefer something like a reference count
in every symbol (which uses one word we previously saved). We can
periodically scan the symbols array for symbols with a reference count
of 1 and then lower that count, knowing it is safe to release the
symbol's memory because no other thread is referencing it. This would
avoid having symbols appear as "[unknown]". We should really have a
similar scan of the dsos to free up their memory.
If we have a reference count then symbol__annotation and
symbol__browser_index can reference a symbol rather than using
container_of.
I think the patches 4 and 5 do the core work, but they introduce some
warts that I wish didn't have to exist. How do you feel about
reference counting? To avoid reference count leaks we adding a
checking framework that is documented here:
https://perfwiki.github.io/main/reference-count-checking/
Thanks,
Ian
> Changes in v3:
>
> - Rebase onto current perf-tools-next.
> - Pick up Namhyung's Reviewed-by for patch 1.
> - Split the exact-path DSO data support into its own patch (2/6), with a
> DSO data test for reading and reopening through an explicit path.
> - Move the duplicate-selection refactor into a preparatory patch (3/6)
> and factor the whole lazy alias-group handling (traversal, demangling,
> IFUNC propagation, compaction) into one helper.
> - Keep struct symbol::namelen as u16. Charge symbol bytes from the stored
> namelen on both allocation and free, and drop the 64 KiB-name test.
> - Fix lazy-loading races reported by Sashiko: in lazy mode, address
> lookups always take the DSO lock, and building the name-sorted array
> materializes and frees the lazy index even when the budget truncates
> it, so the name array is never invalidated. dso__reset_symbol_names()
> is gone. Add a concurrent budget-truncation test.
> - In lazy mode, when no PT_LOAD covers a symbol and its section is NOBITS
> in the debuginfo file, adjust with the runtime section header as eager
> loading does. Add a lazy/eager symbol parity test and a split-debuginfo
> shell test that exercises this path.
> - Keep each unit test with the code it needs (DSO data in 2/6, budget
> reservation in 4/6); the other tests stay in 6/6.
>
> Link: https://lore.kernel.org/all/20260919-perf-symbol-memory-send-v2-0-495b8f00ad7c@uber.com/
>
> Changes in v2:
>
> - Replace direct pread() name reads with the exact symbol source's DSO data
> cache, preserving split-debuginfo offsets and descriptor reopen behavior.
> - Drop the byte-identical-output claim and retain eager loading for PPC64
> .opd and .gnu_debugdata.
> - Make the symbol budget atomic and strict, account complete name lengths,
> accept a bare 0 as unlimited, and keep partial zero-sized ranges from
> covering omitted symbols.
> - Align lazy lookup with eager duplicate and IFUNC selection, PLT clipping,
> and name-sorted materialization.
> - Move option documentation into the feature patches. Add unit and shell
> coverage for cache reopen, truncated names, budget truncation, and skip
> handling.
>
> Link: https://lore.kernel.org/all/20260915-perf-symbol-memory-send-v1-0-1d3360e21f07@uber.com/
> ---
> Alireza Haghdoost (6):
> perf symbols: Fix broken ELF_C_READ_MMAP fallback guard
> perf dso: Allow reading DSO data from an explicit file
> perf symbols: Factor out duplicate symbol selection
> perf script: Add --max-symbol-bytes to bound ELF symbol memory
> perf script: Add --lazy-load-symbols for lazy symbol loading
> perf test: Test lazy symbol loading and symbol memory limits
>
> tools/perf/Documentation/perf-script.txt | 26 +
> tools/perf/arch/powerpc/util/sym-handling.c | 6 +-
> tools/perf/builtin-script.c | 44 ++
> tools/perf/tests/Build | 1 +
> tools/perf/tests/builtin-test.c | 1 +
> tools/perf/tests/dso-data.c | 42 ++
> .../tests/shell/lazy_load_symbols_split_debug.sh | 113 ++++
> tools/perf/tests/shell/script_lazy_load_symbols.sh | 278 ++++++++
> .../tests/shell/script_lazy_load_symbols_skip.sh | 26 +
> tools/perf/tests/symbol-bytes.c | 599 +++++++++++++++++
> tools/perf/tests/tests.h | 1 +
> tools/perf/util/dso.c | 60 +-
> tools/perf/util/dso.h | 47 ++
> tools/perf/util/map.c | 19 +-
> tools/perf/util/symbol-elf.c | 728 ++++++++++++++++++++-
> tools/perf/util/symbol-minimal.c | 16 +
> tools/perf/util/symbol.c | 143 +++-
> tools/perf/util/symbol.h | 30 +-
> tools/perf/util/symbol_conf.h | 2 +
> 19 files changed, 2133 insertions(+), 49 deletions(-)
> ---
> base-commit: edd8a9fe2eca009599e013a29c421c7a6b5ad1b9
> change-id: 20260915-perf-symbol-memory-send-e7cfca1ac3d9
>
> Best regards,
> --
> Alireza Haghdoost <haghdoost@uber.com>
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3 0/6] perf script: Bounded and lazy symbol loading
2026-09-25 20:20 ` [PATCH v3 0/6] perf script: Bounded and lazy symbol loading Ian Rogers
@ 2026-09-25 21:28 ` Alireza Haghdoost
2026-09-25 21:55 ` Ian Rogers
0 siblings, 1 reply; 13+ messages in thread
From: Alireza Haghdoost @ 2026-09-25 21:28 UTC (permalink / raw)
To: Ian Rogers
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
> So currently we have a symbol as:
> ```
> /**
> * A symtab entry. When allocated this may be preceded by an annotation (see
> * symbol__annotation) and/or a browser_index (see symbol__browser_index).
> */
> struct symbol {
> struct rb_node rb_node;
> /** Range of symbol [start, end). */
> u64 start;
> u64 end;
> /** Length of the string name. */
> u16 namelen;
> _Atomic uint16_t flags;
> /** Architecture specific. Unused except on PPC where it holds
> st_other. */
> u8 arch_sym;
> /** The name of length namelen associated with the symbol. */
> char name[];
> };
> ```
> and the struct rb_node is a significant size cost:
> ```
> struct rb_node {
> unsigned long __rb_parent_color;
> struct rb_node *rb_right;
> struct rb_node *rb_left;
> } __attribute__((aligned(sizeof(long))));
> ```
>
> A sorted array of symbols would only require 1 pointer per entry,
> saving 2 words. The array could reside in the "symbols" variable
> passed around something like this:
> ```
> struct symbols {
> struct rw_semaphore lock;
> struct symbol **symbols;
> unsigned int cnt;
> unsigned int allocated;
> bool sorted;
> };
> ```
> This is basically what we do with struct dso and dsos. With a struct
> symbols we can add a lock and ensure operations are appropriately
> synchronized. On top of this we can build lazy symbol resolution.
>
I agree that a sorted struct symbols array with its own lock would be
a cleaner container than the rb-tree. However, it would not save much
memory on its own. On a sample fixture, perf script materializes about
765k symbols and peaks at 265 MiB RssAnon. Replacing the rb_node with
one pointer per symbol saves 16 bytes each, about 12 MiB in total.
Most of the memory goes to materializing symbols that are never
sampled. Lazy loading avoids that work and brings the peak down to
39 MiB. Lazily materialized symbols are inserted one at a time between
lookups, which suits a tree better than a sorted array.
If the maintainers prefer the sorted array, I'm open to it, but I'd
like to be clear about the cost. The symbol rb-tree is used at around
100 call sites in 18 files under tools/perf, so the conversion would
be a separate series that has to land first. It should also cover both
eager and lazy loading, so there is one container for materialized
symbols rather than a tree in one mode and an array in the other.
> ...For
> the bounded memory size, I'd prefer something like a reference count
> in every symbol (which uses one word we previously saved). We can
> periodically scan the symbols array for symbols with a reference count
> of 1 and then lower that count, knowing it is safe to release the
> symbol's memory because no other thread is referencing it. This would
> avoid having symbols appear as "[unknown]". We should really have a
> similar scan of the dsos to free up their memory.
>
Eviction would bound the steady state, but it doesn't bound the peak
on its own. Eager loading materializes the whole symtab when the DSO
is loaded, before anything can be evicted, and this series does not
add lazy loading for every loader (PPC64 .opd, .gnu_debugdata, the
kernel and modules still load eagerly). --max-symbol-bytes covers
those paths too. It is opt-in and off by default, and is meant for
hosts where perf shares memory with latency-sensitive services and the
operator needs a guaranteed upper bound. Like perf record --max-size,
what happens past the cap is deterministic: perf warns and reports
[unknown].
>
> I think the patches 4 and 5 do the core work, but they introduce some
> warts that I wish didn't have to exist. How do you feel about
> reference counting? To avoid reference count leaks we adding a
> checking framework that is documented here:
> https://perfwiki.github.io/main/reference-count-checking/
Reference counting would be useful for symbol lifetime and for the
annotation and browser_index cleanup, and freeing unreferenced symbols
would reduce steady-state memory. As above, I see it as a complement
to the cap rather than a replacement, since it can't guarantee an
upper bound.
Refcounting struct symbol touches every holder of a symbol pointer,
so I'd suggest doing it as a separate series using the refcount
checking framework rather than in this series.
If there are specific parts of patches 4 and 5 you see as warts,
please point me at them and I'll address them in v4.
For v4 I currently plan to bound the second pass of
dso__build_ondemand_index() by the allocated count, as reported by
Sashiko.
Thanks,
Alireza
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3 0/6] perf script: Bounded and lazy symbol loading
2026-09-25 21:28 ` Alireza Haghdoost
@ 2026-09-25 21:55 ` Ian Rogers
0 siblings, 0 replies; 13+ messages in thread
From: Ian Rogers @ 2026-09-25 21:55 UTC (permalink / raw)
To: Alireza Haghdoost
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
Adrian Hunter, James Clark, Alexei Starovoitov, Andrii Nakryiko,
linux-perf-users, linux-kernel
On Fri, Sep 25, 2026 at 2:28 PM Alireza Haghdoost <haghdoost@uber.com> wrote:
>
> > So currently we have a symbol as:
> > ```
> > /**
> > * A symtab entry. When allocated this may be preceded by an annotation (see
> > * symbol__annotation) and/or a browser_index (see symbol__browser_index).
> > */
> > struct symbol {
> > struct rb_node rb_node;
> > /** Range of symbol [start, end). */
> > u64 start;
> > u64 end;
> > /** Length of the string name. */
> > u16 namelen;
> > _Atomic uint16_t flags;
> > /** Architecture specific. Unused except on PPC where it holds
> > st_other. */
> > u8 arch_sym;
> > /** The name of length namelen associated with the symbol. */
> > char name[];
> > };
> > ```
> > and the struct rb_node is a significant size cost:
> > ```
> > struct rb_node {
> > unsigned long __rb_parent_color;
> > struct rb_node *rb_right;
> > struct rb_node *rb_left;
> > } __attribute__((aligned(sizeof(long))));
> > ```
> >
> > A sorted array of symbols would only require 1 pointer per entry,
> > saving 2 words. The array could reside in the "symbols" variable
> > passed around something like this:
> > ```
> > struct symbols {
> > struct rw_semaphore lock;
> > struct symbol **symbols;
> > unsigned int cnt;
> > unsigned int allocated;
> > bool sorted;
> > };
> > ```
> > This is basically what we do with struct dso and dsos. With a struct
> > symbols we can add a lock and ensure operations are appropriately
> > synchronized. On top of this we can build lazy symbol resolution.
> >
>
> I agree that a sorted struct symbols array with its own lock would be
> a cleaner container than the rb-tree. However, it would not save much
> memory on its own. On a sample fixture, perf script materializes about
> 765k symbols and peaks at 265 MiB RssAnon. Replacing the rb_node with
> one pointer per symbol saves 16 bytes each, about 12 MiB in total.
> Most of the memory goes to materializing symbols that are never
> sampled. Lazy loading avoids that work and brings the peak down to
> 39 MiB. Lazily materialized symbols are inserted one at a time between
> lookups, which suits a tree better than a sorted array.
>
> If the maintainers prefer the sorted array, I'm open to it, but I'd
> like to be clear about the cost. The symbol rb-tree is used at around
> 100 call sites in 18 files under tools/perf, so the conversion would
> be a separate series that has to land first. It should also cover both
> eager and lazy loading, so there is one container for materialized
> symbols rather than a tree in one mode and an array in the other.
>
> > ...For
> > the bounded memory size, I'd prefer something like a reference count
> > in every symbol (which uses one word we previously saved). We can
> > periodically scan the symbols array for symbols with a reference count
> > of 1 and then lower that count, knowing it is safe to release the
> > symbol's memory because no other thread is referencing it. This would
> > avoid having symbols appear as "[unknown]". We should really have a
> > similar scan of the dsos to free up their memory.
> >
>
> Eviction would bound the steady state, but it doesn't bound the peak
> on its own. Eager loading materializes the whole symtab when the DSO
> is loaded, before anything can be evicted, and this series does not
> add lazy loading for every loader (PPC64 .opd, .gnu_debugdata, the
> kernel and modules still load eagerly). --max-symbol-bytes covers
> those paths too. It is opt-in and off by default, and is meant for
> hosts where perf shares memory with latency-sensitive services and the
> operator needs a guaranteed upper bound. Like perf record --max-size,
> what happens past the cap is deterministic: perf warns and reports
> [unknown].
>
> >
> > I think the patches 4 and 5 do the core work, but they introduce some
> > warts that I wish didn't have to exist. How do you feel about
> > reference counting? To avoid reference count leaks we adding a
> > checking framework that is documented here:
> > https://perfwiki.github.io/main/reference-count-checking/
>
> Reference counting would be useful for symbol lifetime and for the
> annotation and browser_index cleanup, and freeing unreferenced symbols
> would reduce steady-state memory. As above, I see it as a complement
> to the cap rather than a replacement, since it can't guarantee an
> upper bound.
>
> Refcounting struct symbol touches every holder of a symbol pointer,
> so I'd suggest doing it as a separate series using the refcount
> checking framework rather than in this series.
>
> If there are specific parts of patches 4 and 5 you see as warts,
> please point me at them and I'll address them in v4.
>
> For v4 I currently plan to bound the second pass of
> dso__build_ondemand_index() by the allocated count, as reported by
> Sashiko.
So, I'm pretty against the complexity of dealing with symbols that may
fail due to a memory pressure issue. I'm also against having memory of
symbols bound, while memory for say debuginfo isn't. Ideally we'd have
a language runtime that'd allow us to express some notion of weakly
holding symbols live. I think we can implement a global function to
reduce the memory footprint by iterating through sessions, machines,
dsos, and symbols, squeezing them when possible. The problem with
implementing that on top of patches 4 and 5 is that it's hard to see
what would be kept, so we'd just revert the changes and start
implementing the reference counting solution again.
In the change there seems to be the addition of helper functions and
structs like symbol_candidate. These appear in header files to be
shared across C files. We have multiple notions of symbols, with and
without libelf. I'd rather the change were more minimal given I think
this is the wrong direction.
Thanks,
Ian
> Thanks,
> Alireza
^ permalink raw reply [flat|nested] 13+ messages in thread