* [PATCH v11 1/4] kallsyms: embed source file:line info in kernel stack traces
2026-11-06 13:24 [PATCH v11 0/4] kallsyms: embed source file:line info in kernel stack traces Sasha Levin
@ 2026-11-06 13:24 ` Sasha Levin
2026-11-06 14:52 ` Helge Deller
2026-11-06 13:24 ` [PATCH v11 2/4] kallsyms: extend lineinfo to loadable modules Sasha Levin
` (2 subsequent siblings)
3 siblings, 1 reply; 6+ messages in thread
From: Sasha Levin @ 2026-11-06 13:24 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Luis Chamberlain, Linus Torvalds,
Richard Weinberger, Juergen Gross, Geert Uytterhoeven,
James Bottomley
Cc: Sasha Levin, Jonathan Corbet, Nathan Chancellor, Nicolas Schier,
Petr Pavlu, Daniel Gomez, Aaron Tomlin, Greg KH, Petr Mladek,
Steven Rostedt, Kees Cook, Peter Zijlstra, Thorsten Leemhuis,
Vlastimil Babka, Helge Deller, Randy Dunlap, Laurent Pinchart,
Vivian Wang, Zhen Lei, Sami Tolvanen, linux-kernel, linux-kbuild,
linux-modules, linux-doc
Add CONFIG_KALLSYMS_LINEINFO, which embeds a compact address-to-line
lookup table in the kernel image so stack traces directly print source
file and line number information:
root@localhost:~# echo c > /proc/sysrq-trigger
[ 11.201987] sysrq: Trigger a crash
[ 11.202831] Kernel panic - not syncing: sysrq triggered crash
[ 11.206218] Call Trace:
[ 11.206501] <TASK>
[ 11.206749] dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:94)
[ 11.207403] vpanic+0x36e/0x620 (kernel/panic.c:650)
[ 11.208565] ? __lock_acquire+0x465/0x2240 (kernel/locking/lockdep.c:4674)
[ 11.209324] panic+0xc9/0xd0 (kernel/panic.c:787)
[ 11.211873] ? find_held_lock+0x2b/0x80 (kernel/locking/lockdep.c:5350)
[ 11.212597] ? lock_release+0xd3/0x300 (kernel/locking/lockdep.c:5535)
[ 11.213312] sysrq_handle_crash+0x1a/0x20 (drivers/tty/sysrq.c:154)
[ 11.214005] __handle_sysrq.cold+0x66/0x256 (drivers/tty/sysrq.c:611)
[ 11.214712] write_sysrq_trigger+0x65/0x80 (drivers/tty/sysrq.c:1221)
[ 11.215424] proc_reg_write+0x1bd/0x3c0 (fs/proc/inode.c:330)
[ 11.216061] vfs_write+0x1c6/0xff0 (fs/read_write.c:686)
[ 11.218848] ksys_write+0xfa/0x200 (fs/read_write.c:740)
[ 11.222394] do_syscall_64+0xf3/0x690 (arch/x86/entry/syscall_64.c:63)
[ 11.223942] entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/entry_64.S:121)
At build time, a new host tool (scripts/gen_lineinfo) reads DWARF
.debug_line from vmlinux using libdw (elfutils), extracts all
address-to-file:line mappings, and generates an assembly file with
sorted parallel arrays (offsets from _text, file IDs, and line
numbers). These are linked into vmlinux as .rodata.
At runtime, kallsyms_lookup_lineinfo() does a binary search on the
table and __sprint_symbol() appends "(file:line)" to each stack frame.
The lookup uses offsets from _text so it works with KASLR, requires no
locks or allocations, and is safe in any context including panic.
The suffix is appended for every symbol variant that already prints an
offset, so %pS, %pSb and the %pB backtrace forms all carry it: arm64
prints frames with %pSb and the generic stack_trace_print() uses %pS.
sprint_symbol_no_offset(), which backs lowercase %ps, stays bare
because callers suffix it with a literal "()".
A line program emits several rows per address, and every row but the
last covers zero bytes, so the generator keeps the last row at each
address. That is the row addr2line reports, and for an inlined call it
names the inlined body rather than the call site.
Rows the compiler marks with line 0 declare that no source location
applies from there on (DWARF5 6.2.2), and the end of a line program's
sequence says the same of the bytes past it. Both are kept, as entries
whose line is zero that the lookup reads as "no annotation", so an
address the compiler deliberately left unattributed reports nothing
instead of inheriting the line above it.
The feature requires CONFIG_DEBUG_INFO (for DWARF data) and libelf
and libdw (from elfutils) on the build host.
Memory footprint, measured with:
make ARCH=x86_64 O=$B defconfig
./scripts/config --file $B/.config -e DEBUG_INFO \
-e DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT
make ARCH=x86_64 O=$B olddefconfig && make ARCH=x86_64 O=$B -j$(nproc)
strip -g $B/vmlinux -o vmlinux.nodbg && stat -c %s vmlinux.nodbg
Table: 1,657,663 entries from 4,234 source files
lineinfo_addrs[] 1,657,663 x u32 = 6.3 MiB
lineinfo_file_ids[] 1,657,663 x u16 = 3.2 MiB
lineinfo_lines[] 1,657,663 x u32 = 6.3 MiB
file_offsets + filenames = 0.1 MiB
Total .rodata increase: = 15.9 MiB
vmlinux (stripped): 52.2 MiB -> 68.2 MiB (+16.0 MiB / +30.6%)
That is the cost of the uncompressed format introduced here; the delta
compression added later in this series brings it down to +8.0 MiB
(+15.3%) on the same config.
Suggested-by: Petr Pavlu <petr.pavlu@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
Documentation/admin-guide/index.rst | 1 +
.../admin-guide/kallsyms-lineinfo.rst | 73 +
MAINTAINERS | 6 +
include/linux/kallsyms.h | 18 +-
init/Kconfig | 20 +
kernel/kallsyms.c | 113 +-
kernel/kallsyms_internal.h | 9 +
scripts/.gitignore | 1 +
scripts/Makefile | 3 +
scripts/Makefile.vmlinux | 4 +
scripts/empty_lineinfo.S | 30 +
scripts/gen_lineinfo.c | 1256 +++++++++++++++++
scripts/kallsyms.c | 11 +
scripts/link-vmlinux.sh | 91 +-
14 files changed, 1625 insertions(+), 11 deletions(-)
create mode 100644 Documentation/admin-guide/kallsyms-lineinfo.rst
create mode 100644 scripts/empty_lineinfo.S
create mode 100644 scripts/gen_lineinfo.c
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index cd28dfe91b060..37456e08fe43c 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -73,6 +73,7 @@ problems and bugs in particular.
ramoops
dynamic-debug-howto
init
+ kallsyms-lineinfo
kdump/index
perf/index
pstore-blk
diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentation/admin-guide/kallsyms-lineinfo.rst
new file mode 100644
index 0000000000000..9bf0190798647
--- /dev/null
+++ b/Documentation/admin-guide/kallsyms-lineinfo.rst
@@ -0,0 +1,73 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================================
+Kallsyms Source Line Info (LINEINFO)
+====================================
+
+Overview
+========
+
+``CONFIG_KALLSYMS_LINEINFO`` embeds DWARF-derived source file and line number
+mappings into the kernel image so that stack traces include
+``(file.c:123)`` annotations next to each symbol. This makes it significantly
+easier to pinpoint the exact source location during debugging, without needing
+to manually cross-reference addresses with ``addr2line``.
+
+Enabling the Feature
+====================
+
+Enable the following kernel configuration options::
+
+ CONFIG_KALLSYMS=y
+ CONFIG_DEBUG_INFO=y
+ CONFIG_KALLSYMS_LINEINFO=y
+
+Build dependency: the host tool ``scripts/gen_lineinfo`` requires ``libelf``
+and ``libdw`` from elfutils. Install the development packages:
+
+- Debian/Ubuntu: ``apt install libdw-dev libelf-dev``
+- Fedora/RHEL: ``dnf install elfutils-devel elfutils-libelf-devel``
+- Arch Linux: ``pacman -S libelf``
+
+Example Output
+==============
+
+Without ``CONFIG_KALLSYMS_LINEINFO``::
+
+ Call Trace:
+ <TASK>
+ dump_stack_lvl+0x5d/0x80
+ do_syscall_64+0x82/0x190
+ entry_SYSCALL_64_after_hwframe+0x76/0x7e
+
+With ``CONFIG_KALLSYMS_LINEINFO``::
+
+ Call Trace:
+ <TASK>
+ dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:123)
+ do_syscall_64+0x82/0x190 (arch/x86/entry/common.c:52)
+ entry_SYSCALL_64_after_hwframe+0x76/0x7e (arch/x86/entry/entry_64.S:121)
+
+Assembly routines such as ``entry_SYSCALL_64_after_hwframe`` are annotated
+too, from the line information the assembler emits when debug info is
+enabled.
+
+Memory Overhead
+===============
+
+The lineinfo tables are stored in ``.rodata``. On an x86_64 ``defconfig``
+with ``CONFIG_DEBUG_INFO`` they hold 1.66 million entries and grow the
+stripped image by 16 MiB, about 10 bytes per entry after deduplication.
+
+Known Limitations
+=================
+
+- **vmlinux only**: Only symbols in the core kernel image are annotated.
+ Module symbols are not covered.
+- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-bit
+ values. Entries beyond 4 GiB from ``_text`` are skipped at build time with
+ a warning.
+- **65535 file limit**: Source file IDs are stored as 16-bit values. Builds
+ with more than 65535 unique source files will fail with an error.
+- **Code without line information**: Functions that no DWARF
+ ``.debug_line`` row covers are not annotated.
diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065d..f21f932f40a24 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14341,6 +14341,12 @@ S: Maintained
F: Documentation/hwmon/k8temp.rst
F: drivers/hwmon/k8temp.c
+KALLSYMS LINEINFO
+M: Sasha Levin <sashal@kernel.org>
+S: Maintained
+F: Documentation/admin-guide/kallsyms-lineinfo.rst
+F: scripts/gen_lineinfo.c
+
KANDOU KB9002 PCIE RETIMER HWMON DRIVER
M: Andy Chung <andy.chung@amd.com>
L: linux-hwmon@vger.kernel.org
diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h
index d5dd54c53ace6..53cc25a6e85d9 100644
--- a/include/linux/kallsyms.h
+++ b/include/linux/kallsyms.h
@@ -16,10 +16,15 @@
#include <asm/sections.h>
#define KSYM_NAME_LEN 512
+
+/* Extra space for " (path/to/file.c:12345)" suffix when lineinfo is enabled */
+#define KSYM_LINEINFO_LEN (IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) ? 128 : 0)
+
#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s %s]") + \
(KSYM_NAME_LEN - 1) + \
2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + \
- (BUILD_ID_SIZE_MAX * 2) + 1)
+ (BUILD_ID_SIZE_MAX * 2) + 1 + \
+ KSYM_LINEINFO_LEN)
struct cred;
struct module;
@@ -96,6 +101,9 @@ extern int sprint_backtrace_build_id(char *buffer, unsigned long address);
int lookup_symbol_name(unsigned long addr, char *symname);
+bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
+ const char **file, unsigned int *line);
+
#else /* !CONFIG_KALLSYMS */
static inline unsigned long kallsyms_lookup_name(const char *name)
@@ -164,6 +172,14 @@ static inline int kallsyms_on_each_match_symbol(int (*fn)(void *, unsigned long)
{
return -EOPNOTSUPP;
}
+
+static inline bool kallsyms_lookup_lineinfo(unsigned long addr,
+ unsigned long sym_start,
+ const char **file,
+ unsigned int *line)
+{
+ return false;
+}
#endif /*CONFIG_KALLSYMS*/
static inline void print_ip_sym(const char *loglvl, unsigned long ip)
diff --git a/init/Kconfig b/init/Kconfig
index 4013781e0a029..dc98f61801c74 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1939,6 +1939,26 @@ config KALLSYMS_ALL
Say N unless you really need all symbols, or kernel live patching.
+config KALLSYMS_LINEINFO
+ bool "Embed source file:line information in stack traces"
+ depends on KALLSYMS && DEBUG_INFO
+ help
+ Embeds an address-to-source-line mapping table in the kernel
+ image so that stack traces directly include file:line information,
+ similar to what scripts/decode_stacktrace.sh provides but without
+ needing external tools or a vmlinux with debug info at runtime.
+
+ When enabled, stack traces will look like:
+
+ kmem_cache_alloc_noprof+0x60/0x630 (mm/slub.c:3456)
+ anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412)
+
+ This requires libelf and libdw (from elfutils) on the build host.
+ Costs 10 bytes per DWARF line-table entry; for x86_64_defconfig
+ with CONFIG_DEBUG_INFO that is about 16MB.
+
+ If unsure, say N.
+
# end of the "standard kernel features (expert users)" menu
config ARCH_HAS_MEMBARRIER_CALLBACKS
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index aec2f06858afd..111df61749f2e 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -467,13 +467,89 @@ static int append_buildid(char *buffer, const char *modname,
#endif /* CONFIG_STACKTRACE_BUILD_ID */
+bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
+ const char **file, unsigned int *line)
+{
+ unsigned long raw_offset, raw_min;
+ unsigned int offset, min_offset = 0, low, high, mid, file_id;
+
+ if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || !lineinfo_num_entries)
+ return false;
+
+ /* Compute offset from _text */
+ if (addr < (unsigned long)_text)
+ return false;
+
+ /*
+ * Round-trip through unsigned int rather than comparing against
+ * UINT_MAX: unsigned long is already 32 bits on 32-bit targets, so
+ * that comparison would be dead code there.
+ */
+ raw_offset = addr - (unsigned long)_text;
+ offset = raw_offset;
+ if (offset != raw_offset)
+ return false;
+
+ /*
+ * The search below returns the closest entry at or below @offset, so
+ * a symbol without line entries of its own (assembly without debug
+ * info, or anything past the _etext cap like .init.text) would
+ * inherit the last entry of whatever precedes it. Bound the result
+ * to entries at or above the resolved symbol's start.
+ */
+ if (sym_start > (unsigned long)_text) {
+ raw_min = sym_start - (unsigned long)_text;
+
+ if (raw_min <= raw_offset)
+ min_offset = raw_min;
+ }
+
+ /* Binary search for largest entry <= offset */
+ low = 0;
+ high = lineinfo_num_entries;
+ while (low < high) {
+ mid = low + (high - low) / 2;
+ if (lineinfo_addrs[mid] <= offset)
+ low = mid + 1;
+ else
+ high = mid;
+ }
+
+ if (low == 0)
+ return false;
+ low--;
+
+ if (lineinfo_addrs[low] < min_offset)
+ return false;
+
+ /*
+ * A zero line is the generator's "no source location applies here"
+ * marker, taken straight from a DWARF line-0 row.
+ */
+ if (!lineinfo_lines[low])
+ return false;
+
+ file_id = lineinfo_file_ids[low];
+ *line = lineinfo_lines[low];
+
+ if (file_id >= lineinfo_num_files)
+ return false;
+
+ if (lineinfo_file_offsets[file_id] >= lineinfo_filenames_size)
+ return false;
+
+ *file = &lineinfo_filenames[lineinfo_file_offsets[file_id]];
+ return true;
+}
+
/* Look up a kernel symbol and return it in a text buffer. */
static int __sprint_symbol(char *buffer, unsigned long address,
- int symbol_offset, int add_offset, int add_buildid)
+ int symbol_offset, int add_offset, int add_buildid,
+ int add_lineinfo)
{
char *modname;
const unsigned char *buildid;
- unsigned long offset, size;
+ unsigned long offset, size, sym_start;
int len;
/* Prevent module removal until modname and modbuildid are printed */
@@ -485,6 +561,7 @@ static int __sprint_symbol(char *buffer, unsigned long address,
if (!len)
return sprintf(buffer, "0x%lx", address - symbol_offset);
+ sym_start = address - offset;
offset -= symbol_offset;
if (add_offset)
@@ -497,6 +574,28 @@ static int __sprint_symbol(char *buffer, unsigned long address,
len += sprintf(buffer + len, "]");
}
+ /*
+ * Annotate every caller that already prints an offset, which covers
+ * %pS and %pSb as well as the %pB backtrace forms: arm64's
+ * dump_backtrace_entry() prints frames with %pSb and the generic
+ * stack_trace_print() uses %pS, so restricting this to the backtrace
+ * entry points would leave both unannotated.
+ *
+ * sprint_symbol_no_offset() is the one variant left bare. It backs
+ * lowercase %ps, and existing format strings tack a literal "()"
+ * onto that to denote a call ("foo() replaced with bar()"), which
+ * "foo (file:line)()" would render unreadable.
+ */
+ if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) && !modname) {
+ const char *li_file;
+ unsigned int li_line;
+
+ if (kallsyms_lookup_lineinfo(address, sym_start,
+ &li_file, &li_line))
+ len += scnprintf(buffer + len, KSYM_SYMBOL_LEN - len,
+ " (%s:%u)", li_file, li_line);
+ }
+
return len;
}
@@ -513,7 +612,7 @@ static int __sprint_symbol(char *buffer, unsigned long address,
*/
int sprint_symbol(char *buffer, unsigned long address)
{
- return __sprint_symbol(buffer, address, 0, 1, 0);
+ return __sprint_symbol(buffer, address, 0, 1, 0, 1);
}
EXPORT_SYMBOL_GPL(sprint_symbol);
@@ -530,7 +629,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol);
*/
int sprint_symbol_build_id(char *buffer, unsigned long address)
{
- return __sprint_symbol(buffer, address, 0, 1, 1);
+ return __sprint_symbol(buffer, address, 0, 1, 1, 1);
}
EXPORT_SYMBOL_GPL(sprint_symbol_build_id);
@@ -547,7 +646,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol_build_id);
*/
int sprint_symbol_no_offset(char *buffer, unsigned long address)
{
- return __sprint_symbol(buffer, address, 0, 0, 0);
+ return __sprint_symbol(buffer, address, 0, 0, 0, 0);
}
EXPORT_SYMBOL_GPL(sprint_symbol_no_offset);
@@ -567,7 +666,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol_no_offset);
*/
int sprint_backtrace(char *buffer, unsigned long address)
{
- return __sprint_symbol(buffer, address, -1, 1, 0);
+ return __sprint_symbol(buffer, address, -1, 1, 0, 1);
}
/**
@@ -587,7 +686,7 @@ int sprint_backtrace(char *buffer, unsigned long address)
*/
int sprint_backtrace_build_id(char *buffer, unsigned long address)
{
- return __sprint_symbol(buffer, address, -1, 1, 1);
+ return __sprint_symbol(buffer, address, -1, 1, 1, 1);
}
/* To avoid using get_symbol_offset for every symbol, we carry prefix along. */
diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h
index 81a867dbe57d4..d7374ce444d81 100644
--- a/kernel/kallsyms_internal.h
+++ b/kernel/kallsyms_internal.h
@@ -15,4 +15,13 @@ extern const u16 kallsyms_token_index[];
extern const unsigned int kallsyms_markers[];
extern const u8 kallsyms_seqs_of_names[];
+extern const u32 lineinfo_num_entries;
+extern const u32 lineinfo_addrs[];
+extern const u16 lineinfo_file_ids[];
+extern const u32 lineinfo_lines[];
+extern const u32 lineinfo_num_files;
+extern const u32 lineinfo_file_offsets[];
+extern const u32 lineinfo_filenames_size;
+extern const char lineinfo_filenames[];
+
#endif // LINUX_KALLSYMS_INTERNAL_H_
diff --git a/scripts/.gitignore b/scripts/.gitignore
index 7004565439107..a7c4896ad8556 100644
--- a/scripts/.gitignore
+++ b/scripts/.gitignore
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
/asn1_compiler
/gen_init_cpio
+/gen_lineinfo
/gen_packed_field_checks
/generate_rust_target
/insert-sys-cert
diff --git a/scripts/Makefile b/scripts/Makefile
index d7fbb7ca098b1..94b09cf7c1839 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -4,6 +4,7 @@
# the kernel for the build process.
hostprogs-always-y += kallsyms
+hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO) += gen_lineinfo
hostprogs-always-$(BUILD_C_RECORDMCOUNT) += recordmcount
hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) += sorttable
hostprogs-always-$(CONFIG_ASN1) += asn1_compiler
@@ -40,6 +41,8 @@ HOSTCFLAGS_asn1_compiler.o = -I$(srctree)/include
HOSTCFLAGS_sign-file.o = $(shell $(HOSTPKG_CONFIG) --cflags libcrypto 2> /dev/null)
HOSTCFLAGS_sign-file.o += -I$(srctree)/tools/include/uapi/
HOSTLDLIBS_sign-file = $(shell $(HOSTPKG_CONFIG) --libs libcrypto 2> /dev/null || echo -lcrypto)
+HOSTCFLAGS_gen_lineinfo.o = $(shell $(HOSTPKG_CONFIG) --cflags libdw 2> /dev/null)
+HOSTLDLIBS_gen_lineinfo = $(shell $(HOSTPKG_CONFIG) --libs libdw 2> /dev/null || echo -ldw -lelf)
ifdef CONFIG_UNWINDER_ORC
ifeq ($(ARCH),x86_64)
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index 95b523fe4f9ee..fddd3a3bc6436 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -78,6 +78,10 @@ ifdef CONFIG_BUILDTIME_TABLE_SORT
vmlinux.unstripped: scripts/sorttable
endif
+ifdef CONFIG_KALLSYMS_LINEINFO
+vmlinux.unstripped: scripts/gen_lineinfo
+endif
+
# vmlinux
# ---------------------------------------------------------------------------
diff --git a/scripts/empty_lineinfo.S b/scripts/empty_lineinfo.S
new file mode 100644
index 0000000000000..e058c41137123
--- /dev/null
+++ b/scripts/empty_lineinfo.S
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * Empty lineinfo stub for the initial vmlinux link.
+ * The real lineinfo is generated from .tmp_vmlinux1 by gen_lineinfo.
+ */
+ .section .rodata, "a"
+ .globl lineinfo_num_entries
+ .balign 4
+lineinfo_num_entries:
+ .long 0
+ .globl lineinfo_num_files
+ .balign 4
+lineinfo_num_files:
+ .long 0
+ .globl lineinfo_addrs
+lineinfo_addrs:
+ .globl lineinfo_file_ids
+lineinfo_file_ids:
+ .globl lineinfo_lines
+lineinfo_lines:
+ .globl lineinfo_file_offsets
+lineinfo_file_offsets:
+ .globl lineinfo_filenames_size
+ .balign 4
+lineinfo_filenames_size:
+ .long 0
+ .globl lineinfo_filenames
+lineinfo_filenames:
diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c
new file mode 100644
index 0000000000000..1b347d07b7ad5
--- /dev/null
+++ b/scripts/gen_lineinfo.c
@@ -0,0 +1,1256 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * gen_lineinfo.c - Generate address-to-source-line lookup tables from DWARF
+ *
+ * Copyright (C) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * Reads DWARF .debug_line from a vmlinux ELF file and outputs an assembly
+ * file containing sorted lookup tables that the kernel uses to annotate
+ * stack traces with source file:line information.
+ *
+ * Requires libelf and libdw from elfutils.
+ */
+
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elfutils/libdw.h>
+#include <dwarf.h>
+#include <elf.h>
+#include <gelf.h>
+#include <limits.h>
+#include <array_size.h>
+#include <hash.h>
+#include <hashtable.h>
+#include <xalloc.h>
+
+#define LINEINFO_PREFIX "gen_lineinfo: "
+
+static bool verbose;
+
+#define verbose_msg(fmt, ...) \
+ do { \
+ if (verbose) \
+ fprintf(stderr, LINEINFO_PREFIX fmt "\n", \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#define warn(fmt, ...) \
+ fprintf(stderr, LINEINFO_PREFIX "warning: " fmt "\n", ##__VA_ARGS__)
+
+#define error(fmt, ...) \
+ do { \
+ fprintf(stderr, LINEINFO_PREFIX "error: " fmt "\n", \
+ ##__VA_ARGS__); \
+ exit(1); \
+ } while (0)
+
+static unsigned int skipped_overflow;
+
+/*
+ * vmlinux mode: end of the invariant .text region. Zero means "no cap"
+ * (graceful fallback when _etext is absent on some build).
+ */
+static unsigned long long text_end_addr;
+
+struct line_entry {
+ unsigned int offset; /* offset from _text */
+ unsigned int file_id;
+ unsigned int line;
+ unsigned int seq; /* line-program row order, breaks offset ties */
+};
+
+/*
+ * Individually allocated so files[] can grow without invalidating the
+ * hlist_node linkage.
+ */
+struct file_entry {
+ struct hlist_node hnode;
+ unsigned int id;
+ unsigned int str_offset;
+ char name[];
+};
+
+static struct line_entry *entries;
+static unsigned int num_entries;
+static unsigned int entries_capacity;
+
+static struct file_entry **files;
+static unsigned int num_files;
+static unsigned int files_capacity;
+
+static HASHTABLE_DEFINE(file_hashtable, 1U << 13);
+
+/* Monotonic row counter; see the seq tie-break in compare_entries(). */
+static unsigned int next_seq;
+
+static void add_entry(unsigned int offset, unsigned int file_id,
+ unsigned int line)
+{
+ if (num_entries >= entries_capacity) {
+ entries_capacity = entries_capacity ? entries_capacity * 2 : 65536;
+ entries = xrealloc(entries, entries_capacity * sizeof(*entries));
+ }
+ entries[num_entries].offset = offset;
+ entries[num_entries].file_id = file_id;
+ entries[num_entries].line = line;
+ entries[num_entries].seq = next_seq++;
+ num_entries++;
+}
+
+static unsigned int find_or_add_file(const char *name)
+{
+ unsigned int key = hash_str(name);
+ struct file_entry *f;
+ size_t len;
+
+ hash_for_each_possible(file_hashtable, f, hnode, key)
+ if (!strcmp(f->name, name))
+ return f->id;
+
+ if (num_files >= 65535)
+ error("too many source files (%u > 65535)", num_files);
+
+ if (num_files >= files_capacity) {
+ files_capacity = files_capacity ? files_capacity * 2 : 4096;
+ files = xrealloc(files, files_capacity * sizeof(*files));
+ }
+
+ len = strlen(name);
+ f = xmalloc(sizeof(*f) + len + 1);
+ memset(f, 0, sizeof(*f));
+ memcpy(f->name, name, len + 1);
+ f->id = num_files;
+
+ files[num_files] = f;
+ hash_add(file_hashtable, &f->hnode, key);
+
+ return num_files++;
+}
+
+/*
+ * Well-known top-level directories in the kernel source tree. Only used
+ * as a last resort, when a path matches none of the build roots below --
+ * e.g. an object compiled outside any of them.
+ */
+static const char * const kernel_dirs[] = {
+ "arch/", "block/", "certs/", "crypto/", "drivers/", "fs/",
+ "include/", "init/", "io_uring/", "ipc/", "kernel/", "lib/",
+ "mm/", "net/", "rust/", "samples/", "scripts/", "security/",
+ "sound/", "tools/", "usr/", "virt/",
+};
+
+/* Absolute build and source roots, longest first. */
+struct path_root {
+ char *path;
+ size_t len;
+};
+
+static struct path_root path_roots[8];
+static unsigned int num_path_roots;
+
+/*
+ * Lexically canonicalize @path in place: collapse repeated slashes, drop
+ * "." components and resolve ".." against the preceding component. Purely
+ * textual -- nothing is stat()ed, because DWARF can name generated files
+ * that do not exist yet when gen_lineinfo runs.
+ */
+static void normalize_path(char *path)
+{
+ bool absolute = path[0] == '/';
+ char *base, *out = path;
+ const char *in = path;
+
+ if (absolute)
+ *out++ = *in++;
+ base = out;
+
+ while (*in) {
+ const char *seg = in;
+ size_t seglen;
+
+ while (*in && *in != '/')
+ in++;
+ seglen = in - seg;
+ while (*in == '/')
+ in++;
+
+ if (!seglen || (seglen == 1 && seg[0] == '.'))
+ continue;
+
+ if (seglen == 2 && seg[0] == '.' && seg[1] == '.') {
+ if (out > base) {
+ /* Pop the previously emitted component. */
+ while (out > base && out[-1] != '/')
+ out--;
+ if (out > base)
+ out--; /* and its separator */
+ continue;
+ }
+ /* "/.." is "/"; a leading ".." in a relative path stays. */
+ if (absolute)
+ continue;
+ }
+
+ /*
+ * Separator first: writing it after the component would land
+ * on the byte @in still points at whenever nothing has been
+ * compacted yet, clobbering the terminator and running the
+ * loop off the end of the string.
+ */
+ if (out > base)
+ *out++ = '/';
+ memmove(out, seg, seglen);
+ out += seglen;
+ }
+
+ if (out == base && !absolute)
+ *out++ = '.';
+ *out = '\0';
+}
+
+static int compare_path_roots(const void *a, const void *b)
+{
+ const struct path_root *ra = a, *rb = b;
+
+ if (ra->len != rb->len)
+ return ra->len > rb->len ? -1 : 1;
+ return 0;
+}
+
+static void add_path_root(const char *path)
+{
+ char buf[PATH_MAX];
+
+ if (!path || !*path)
+ return;
+
+ if (path[0] == '/') {
+ if (snprintf(buf, sizeof(buf), "%s", path) >= (int)sizeof(buf))
+ return;
+ } else {
+ char cwd[PATH_MAX];
+
+ /* kbuild runs host tools with cwd == $objtree. */
+ if (!getcwd(cwd, sizeof(cwd)))
+ return;
+ if (snprintf(buf, sizeof(buf), "%s/%s", cwd, path) >= (int)sizeof(buf))
+ return;
+ }
+
+ normalize_path(buf);
+
+ /* "/" would match every absolute path. */
+ if (!strcmp(buf, "/"))
+ return;
+
+ for (unsigned int i = 0; i < num_path_roots; i++)
+ if (!strcmp(path_roots[i].path, buf))
+ return;
+
+ if (num_path_roots == ARRAY_SIZE(path_roots))
+ return;
+
+ path_roots[num_path_roots].path = xstrdup(buf);
+ path_roots[num_path_roots].len = strlen(buf);
+ num_path_roots++;
+}
+
+/*
+ * Collect the roots that DWARF paths get made relative to. kbuild exports
+ * all three, so no Makefile plumbing is needed: $objtree and $srctree cover
+ * in-tree and O= builds, and $srcroot covers M= external modules, whose
+ * sources live under neither.
+ */
+static void init_path_roots(void)
+{
+ static const char * const vars[] = { "objtree", "srctree", "srcroot" };
+
+ for (unsigned int i = 0; i < ARRAY_SIZE(vars); i++) {
+ const char *val = getenv(vars[i]);
+ char *real;
+
+ if (!val || !*val)
+ continue;
+
+ add_path_root(val);
+
+ /*
+ * Register the resolved form as well, so a symlinked tree
+ * matches whichever spelling the compiler recorded. Only
+ * the roots are resolved this way -- never a DWARF path.
+ */
+ real = realpath(val, NULL);
+ if (real) {
+ add_path_root(real);
+ free(real);
+ }
+ }
+
+ qsort(path_roots, num_path_roots, sizeof(*path_roots),
+ compare_path_roots);
+}
+
+/*
+ * Strip a DWARF filename down to a kernel-tree-relative path.
+ *
+ * Per DWARF, a relative DW_AT_name is relative to the CU's DW_AT_comp_dir,
+ * so the two are joined and canonicalized first. The result is then made
+ * relative to the longest matching build root. Everything after that is a
+ * fallback for objects built outside the tree.
+ */
+static const char *make_relative(const char *path, const char *comp_dir)
+{
+ static char buf[PATH_MAX];
+ const char *p;
+
+ if (path[0] == '/') {
+ if (snprintf(buf, sizeof(buf), "%s", path) >= (int)sizeof(buf))
+ return path;
+ } else if (comp_dir && comp_dir[0] == '/') {
+ if (snprintf(buf, sizeof(buf), "%s/%s", comp_dir, path) >=
+ (int)sizeof(buf))
+ return path;
+ } else {
+ /* Nothing absolute to anchor against. */
+ return path;
+ }
+
+ normalize_path(buf);
+
+ for (unsigned int i = 0; i < num_path_roots; i++) {
+ size_t len = path_roots[i].len;
+
+ if (!strncmp(buf, path_roots[i].path, len) && buf[len] == '/')
+ return buf + len + 1;
+ }
+
+ /*
+ * comp_dir may still be a usable prefix even when it is not one of
+ * the roots -- but only if stripping it leaves a directory
+ * component, otherwise the kernel_dirs scan recovers more.
+ */
+ if (comp_dir) {
+ size_t len = strlen(comp_dir);
+
+ if (!strncmp(buf, comp_dir, len) && buf[len] == '/' &&
+ strchr(buf + len + 1, '/'))
+ return buf + len + 1;
+ }
+
+ for (p = strchr(buf, '/'); p; p = strchr(p + 1, '/'))
+ for (unsigned int i = 0; i < ARRAY_SIZE(kernel_dirs); i++)
+ if (!strncmp(p + 1, kernel_dirs[i],
+ strlen(kernel_dirs[i])))
+ return p + 1;
+
+ p = strrchr(buf, '/');
+ return p ? p + 1 : buf;
+}
+
+static int compare_entries(const void *a, const void *b)
+{
+ const struct line_entry *ea = a;
+ const struct line_entry *eb = b;
+
+ if (ea->offset != eb->offset)
+ return ea->offset < eb->offset ? -1 : 1;
+ /*
+ * Several rows routinely share one address: the line program emits a
+ * row per event, and every row but the last covers zero bytes. Order
+ * them as the program emitted them so deduplicate() can keep the row
+ * that actually describes the instruction. seq is unique, so the
+ * sort is total and the output is reproducible.
+ */
+ if (ea->seq != eb->seq)
+ return ea->seq < eb->seq ? -1 : 1;
+ return 0;
+}
+
+/*
+ * Look up a vmlinux symbol by exact name and return its st_value, or
+ * @fallback if absent. Aborts when @required and the symbol is missing.
+ */
+static unsigned long long find_vmlinux_sym(Elf *elf, const char *name,
+ unsigned long long fallback,
+ bool required)
+{
+ size_t nsyms, i;
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ Elf_Data *data;
+
+ if (!gelf_getshdr(scn, &shdr))
+ continue;
+ if (shdr.sh_type != SHT_SYMTAB || !shdr.sh_entsize)
+ continue;
+
+ data = elf_getdata(scn, NULL);
+ if (!data)
+ continue;
+
+ nsyms = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nsyms; i++) {
+ GElf_Sym sym;
+ const char *sname;
+
+ if (!gelf_getsym(data, i, &sym))
+ continue;
+ sname = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (sname && !strcmp(sname, name))
+ return sym.st_value;
+ }
+ }
+
+ if (required)
+ error("cannot find %s symbol", name);
+ return fallback;
+}
+
+static unsigned long long find_text_addr(Elf *elf)
+{
+ return find_vmlinux_sym(elf, "_text", 0, true);
+}
+
+/*
+ * vmlinux is linked in multiple passes: gen_lineinfo runs against
+ * .tmp_vmlinux1 (which carries an empty lineinfo stub), then real tables
+ * are linked in for the final image. Sections placed AFTER .rodata
+ * (.init.text, .exit.text, ...) shift forward as .rodata grows to hold
+ * the real lineinfo blob, so DWARF addresses we'd capture for them in
+ * pass 1 would be stale in the final kernel. Cap captured addresses at
+ * _etext, the symbol that marks the end of .text — placed before .rodata
+ * in every architecture's vmlinux.lds.S, so its addresses are invariant
+ * across the relink. Returns 0 if _etext is absent (no cap; v3 behavior).
+ */
+static unsigned long long find_text_end_addr(Elf *elf)
+{
+ return find_vmlinux_sym(elf, "_etext", 0, false);
+}
+
+/* Sorted, duplicate-free extents of every function symbol. */
+struct sym_start {
+ unsigned int offset;
+ unsigned int size;
+};
+
+static struct sym_start *sym_starts;
+static unsigned int num_sym_starts;
+static unsigned int sym_starts_capacity;
+
+/*
+ * Every symbol beginning in covered text, whether or not it is a function.
+ * Clang's basic-block sections (CONFIG_PROPELLER_CLANG) split a function
+ * into parts that carry STT_NOTYPE labels and a line-program sequence each,
+ * so resolve_cu_row_groups() cannot ask for STT_FUNC when it needs to know
+ * whether a sequence can begin at an address.
+ */
+static struct sym_start *text_starts;
+static unsigned int num_text_starts;
+static unsigned int text_starts_capacity;
+
+/* Offsets one past the end of each DWARF line-program sequence. */
+static unsigned int *seq_ends;
+static unsigned int num_seq_ends;
+static unsigned int seq_ends_capacity;
+
+static void append_offset(unsigned int **arr, unsigned int *count,
+ unsigned int *capacity, unsigned int value)
+{
+ if (*count >= *capacity) {
+ *capacity = *capacity ? *capacity * 2 : 16384;
+ *arr = xrealloc(*arr, *capacity * sizeof(**arr));
+ }
+ (*arr)[(*count)++] = value;
+}
+
+/*
+ * Record the end of a line-program sequence. @addr is one past the last
+ * covered byte, so the sequence's own coverage is tested using addr - 1.
+ */
+static void record_seq_end(unsigned long long addr,
+ unsigned long long text_addr)
+{
+ unsigned long long raw;
+
+ if (addr <= text_addr)
+ return;
+ if (text_end_addr && addr - 1 >= text_end_addr)
+ return;
+
+ raw = addr - text_addr;
+ if (raw > UINT_MAX)
+ return;
+
+ append_offset(&seq_ends, &num_seq_ends, &seq_ends_capacity,
+ (unsigned int)raw);
+}
+
+static int compare_sym_starts(const void *a, const void *b)
+{
+ const struct sym_start *sa = a, *sb = b;
+
+ if (sa->offset != sb->offset)
+ return sa->offset < sb->offset ? -1 : 1;
+ /* Larger extent first, so the dedup below keeps it. */
+ if (sa->size != sb->size)
+ return sa->size > sb->size ? -1 : 1;
+ return 0;
+}
+
+/* Sort by offset, keeping only the widest symbol at each. */
+static void sort_starts(struct sym_start *a, unsigned int *count)
+{
+ unsigned int n = *count, j = 0;
+
+ if (n < 2)
+ return;
+
+ qsort(a, n, sizeof(*a), compare_sym_starts);
+ for (unsigned int i = 1; i < n; i++) {
+ if (a[i].offset == a[j].offset)
+ continue;
+ if (++j != i)
+ a[j] = a[i];
+ }
+ *count = j + 1;
+}
+
+/*
+ * Collect the extent of every function symbol. deduplicate() uses these to
+ * make sure each function keeps an entry at its own first byte; without that
+ * the kernel's symbol-boundary check rejects the preceding function's entry
+ * and the frame goes unannotated.
+ */
+static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
+{
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ Elf_Data *data;
+ size_t nsyms;
+
+ if (!gelf_getshdr(scn, &shdr))
+ continue;
+ if (shdr.sh_type != SHT_SYMTAB || !shdr.sh_entsize)
+ continue;
+
+ data = elf_getdata(scn, NULL);
+ if (!data)
+ continue;
+
+ nsyms = shdr.sh_size / shdr.sh_entsize;
+ for (size_t i = 0; i < nsyms; i++) {
+ GElf_Sym sym;
+ unsigned long long raw;
+
+ if (!gelf_getsym(data, i, &sym))
+ continue;
+ switch (GELF_ST_TYPE(sym.st_info)) {
+ case STT_FUNC:
+ case STT_NOTYPE:
+ break;
+ default:
+ continue;
+ }
+ if (sym.st_value < text_addr)
+ continue;
+ if (text_end_addr && sym.st_value >= text_end_addr)
+ continue;
+
+ raw = sym.st_value - text_addr;
+ if (raw > UINT_MAX)
+ continue;
+
+ if (num_sym_starts >= sym_starts_capacity) {
+ sym_starts_capacity = sym_starts_capacity ?
+ sym_starts_capacity * 2 : 16384;
+ sym_starts = xrealloc(sym_starts,
+ sym_starts_capacity *
+ sizeof(*sym_starts));
+ }
+ if (num_text_starts >= text_starts_capacity) {
+ text_starts_capacity = text_starts_capacity ?
+ text_starts_capacity * 2 : 16384;
+ text_starts = xrealloc(text_starts,
+ text_starts_capacity *
+ sizeof(*text_starts));
+ }
+ text_starts[num_text_starts].offset = (unsigned int)raw;
+ text_starts[num_text_starts].size =
+ sym.st_size > UINT_MAX ? UINT_MAX :
+ (unsigned int)sym.st_size;
+ num_text_starts++;
+
+ if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
+ continue;
+
+ sym_starts[num_sym_starts].offset = (unsigned int)raw;
+ sym_starts[num_sym_starts].size =
+ sym.st_size > UINT_MAX ? UINT_MAX :
+ (unsigned int)sym.st_size;
+ num_sym_starts++;
+ }
+ }
+
+ sort_starts(sym_starts, &num_sym_starts);
+ sort_starts(text_starts, &num_text_starts);
+
+ /*
+ * An unsized label reaches the next symbol, and a sequence beginning
+ * there cannot run past it.
+ */
+ for (unsigned int i = 1; i < num_text_starts; i++) {
+ if (!text_starts[i - 1].size)
+ text_starts[i - 1].size = text_starts[i].offset -
+ text_starts[i - 1].offset;
+ }
+}
+
+/*
+ * One flag per row read from the compile unit being processed: whether the
+ * row closed a line-program sequence. libdw returns rows sorted by address,
+ * so the flag is the only thing that tells the two kinds of line-0 row apart.
+ */
+static bool *cu_row_seq_end;
+static size_t cu_row_seq_end_cap;
+
+static void cu_rows_reserve(size_t rows)
+{
+ if (rows <= cu_row_seq_end_cap)
+ return;
+ cu_row_seq_end = xrealloc(cu_row_seq_end,
+ rows * sizeof(*cu_row_seq_end));
+ cu_row_seq_end_cap = rows;
+}
+
+/* Size of the symbol starting exactly at @offset, 0 if none. */
+static unsigned int symbol_extent_at(unsigned int offset)
+{
+ unsigned int low = 0, high = num_text_starts;
+
+ while (low < high) {
+ unsigned int mid = low + (high - low) / 2;
+
+ if (text_starts[mid].offset < offset)
+ low = mid + 1;
+ else if (text_starts[mid].offset > offset)
+ high = mid;
+ else
+ return text_starts[mid].size;
+ }
+
+ return 0;
+}
+
+/*
+ * True if the rows a unit has where one of its sequences ended open a new
+ * sequence there, rather than closing the one that ended. libdw cannot say:
+ * both look like an end_sequence followed by a normal row at one address.
+ *
+ * A sequence can only begin at a function, and a unit that begins one goes
+ * on to describe that function -- with a row inside it, or, for a function
+ * the unit covers in a single row, with the sequence end that closes it.
+ * Neither holds for the padding between two functions, which carries a
+ * symbol of its own that no unit describes.
+ */
+static bool starts_new_sequence(unsigned int group, unsigned int next,
+ unsigned int start)
+{
+ unsigned long long end;
+ unsigned int size;
+
+ size = symbol_extent_at(entries[group].offset);
+ if (!size)
+ return false;
+
+ end = (unsigned long long)entries[group].offset + size;
+
+ for (unsigned int t = next; t < num_entries; t++) {
+ if (entries[t].offset > end)
+ break;
+ if (entries[t].offset < end || cu_row_seq_end[t - start])
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Collapse the rows one compile unit emitted at a single address, of which
+ * all but one cover zero bytes.
+ *
+ * Rows that are not sequence ends keep their line-program order here, so the
+ * last of them is the one that describes the address, line 0 included: such a
+ * row declares that no source location applies, which is what addr2line
+ * reports for it too.
+ *
+ * The row closing a sequence is harder. libdw sorts rows by address and
+ * puts an end_sequence ahead of any normal row sharing that address, which
+ * hides the difference between a sequence whose last row lands on its own
+ * end, covering nothing, and a sequence that begins where another ended.
+ * The symbol table settles it, see starts_new_sequence().
+ */
+static void resolve_cu_row_groups(unsigned int start)
+{
+ unsigned int i = start, j = start;
+
+ while (i < num_entries) {
+ unsigned int k = i, normal = i, seq_end = i, keep;
+ bool have_normal = false, have_seq_end = false;
+
+ while (k < num_entries &&
+ entries[k].offset == entries[i].offset) {
+ if (cu_row_seq_end[k - start]) {
+ seq_end = k;
+ have_seq_end = true;
+ } else {
+ normal = k;
+ have_normal = true;
+ }
+ k++;
+ }
+
+ if (!have_normal)
+ keep = seq_end;
+ else if (!have_seq_end)
+ keep = normal;
+ else if (starts_new_sequence(i, k, start))
+ keep = normal;
+ else
+ keep = seq_end;
+
+ cu_row_seq_end[j - start] = cu_row_seq_end[keep - start];
+ entries[j++] = entries[keep];
+ i = k;
+ }
+
+ num_entries = j;
+}
+
+/*
+ * Address spans described by an assembler compile unit. A single .loc there
+ * covers a whole macro expansion, which can emit a function per invocation,
+ * so a row from before a symbol is the row that describes it. In C the same
+ * shape means padding or a split-out fragment carrying a neighbour's line,
+ * which is why synthesize_symbol_starts() asks for this.
+ */
+struct asm_span {
+ unsigned int lo;
+ unsigned int hi;
+};
+
+static struct asm_span *asm_spans;
+static unsigned int num_asm_spans;
+static unsigned int asm_spans_capacity;
+
+static int compare_asm_spans(const void *a, const void *b)
+{
+ const struct asm_span *sa = a, *sb = b;
+
+ if (sa->lo != sb->lo)
+ return sa->lo < sb->lo ? -1 : 1;
+ return 0;
+}
+
+/* True if an assembler unit describes @offset. */
+static bool in_asm_span(unsigned int offset)
+{
+ unsigned int low = 0, high = num_asm_spans;
+
+ while (low < high) {
+ unsigned int mid = low + (high - low) / 2;
+
+ if (asm_spans[mid].lo <= offset)
+ low = mid + 1;
+ else
+ high = mid;
+ }
+
+ return low && offset <= asm_spans[low - 1].hi;
+}
+
+/*
+ * One span per sequence this assembler unit contributed. A unit's code lands
+ * in scattered places once linked, so a span covering the whole unit would
+ * claim whatever sits between. @first_seq_end is where this unit's sequence
+ * ends begin, needed because a sequence reaching the end of the covered range
+ * has its closing row dropped and leaves no marker to stop at.
+ */
+static void record_cu_asm_spans(unsigned int start, unsigned int first_seq_end)
+{
+ unsigned int i = start;
+
+ while (i < num_entries) {
+ unsigned int j = i, hi;
+
+ while (!cu_row_seq_end[j - start] && j + 1 < num_entries)
+ j++;
+
+ hi = entries[j].offset;
+ if (!cu_row_seq_end[j - start]) {
+ for (unsigned int t = first_seq_end; t < num_seq_ends;
+ t++) {
+ if (seq_ends[t] <= hi)
+ continue;
+ if (hi == entries[j].offset || seq_ends[t] < hi)
+ hi = seq_ends[t];
+ }
+ }
+
+ if (num_asm_spans >= asm_spans_capacity) {
+ asm_spans_capacity = asm_spans_capacity ?
+ asm_spans_capacity * 2 : 256;
+ asm_spans = xrealloc(asm_spans, asm_spans_capacity *
+ sizeof(*asm_spans));
+ }
+ asm_spans[num_asm_spans].lo = entries[i].offset;
+ asm_spans[num_asm_spans].hi = hi;
+ num_asm_spans++;
+ i = j + 1;
+ }
+}
+
+static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
+{
+ Dwarf_Off off = 0, next_off;
+ size_t hdr_size;
+
+ while (dwarf_nextcu(dwarf, off, &next_off, &hdr_size,
+ NULL, NULL, NULL) == 0) {
+ Dwarf_Die cudie;
+ Dwarf_Lines *lines;
+ size_t nlines;
+ Dwarf_Attribute attr;
+ const char *comp_dir = NULL;
+ unsigned int cu_first_entry, cu_first_seq_end;
+
+ if (!dwarf_offdie(dwarf, off + hdr_size, &cudie))
+ goto next;
+
+ if (dwarf_attr(&cudie, DW_AT_comp_dir, &attr))
+ comp_dir = dwarf_formstring(&attr);
+
+ if (dwarf_getsrclines(&cudie, &lines, &nlines) != 0)
+ goto next;
+
+ cu_first_entry = num_entries;
+ cu_first_seq_end = num_seq_ends;
+ cu_rows_reserve(nlines);
+
+ for (size_t i = 0; i < nlines; i++) {
+ Dwarf_Line *line = dwarf_onesrcline(lines, i);
+ Dwarf_Addr addr;
+ const char *src;
+ const char *rel;
+ unsigned int file_id, loffset;
+ bool endseq = false;
+ int lineno;
+
+ if (!line)
+ continue;
+
+ if (dwarf_lineaddr(line, &addr) != 0)
+ continue;
+
+ /*
+ * An end_sequence row marks the first address NOT
+ * covered by this sequence; libdw repeats the previous
+ * line number on it, so keeping it as an entry would
+ * extend a function's annotation past its own end.
+ * Record the boundary, which closes an assembler
+ * unit's span, and emit a line-0 entry: a lookup in
+ * the gap before the next sequence then reports
+ * nothing instead of inheriting the line above it, and
+ * synthesize_symbol_starts() sees that coverage
+ * stopped here.
+ */
+ if (dwarf_lineendsequence(line, &endseq) == 0 && endseq) {
+ record_seq_end(addr, text_addr);
+ lineno = 0;
+ } else if (dwarf_lineno(line, &lineno) != 0) {
+ continue;
+ }
+
+ /*
+ * Line 0 means "no source location applies from here"
+ * (DWARF5 6.2.2). Keep such a row rather than drop
+ * it: without it the preceding entry covers the
+ * interval, and an address the compiler explicitly
+ * declined to attribute gets a confident, wrong line.
+ * It needs no file of its own, since the lookup side
+ * reads the zero line as "no annotation" and stops
+ * before it looks one up.
+ */
+ src = dwarf_linesrc(line, NULL, NULL);
+ if (!src && lineno)
+ continue;
+
+ if (addr < text_addr)
+ continue;
+ /*
+ * Skip addresses past _etext. Sections after .rodata
+ * shift when the real lineinfo replaces the empty stub
+ * during the multi-pass vmlinux link, so any address
+ * we'd capture there would be stale by the time the
+ * final kernel runs.
+ *
+ * An end_sequence row addresses the byte after
+ * the code it closes, so judge it by its last
+ * byte, as record_seq_end() does: a sequence
+ * ending at _etext still needs its end row.
+ */
+ if (text_end_addr && addr - endseq >= text_end_addr)
+ continue;
+
+ {
+ unsigned long long raw_offset = addr - text_addr;
+
+ if (raw_offset > UINT_MAX) {
+ skipped_overflow++;
+ continue;
+ }
+ loffset = (unsigned int)raw_offset;
+ }
+
+ if (src) {
+ rel = make_relative(src, comp_dir);
+ file_id = find_or_add_file(rel);
+ } else {
+ file_id = 0;
+ }
+
+ add_entry(loffset, file_id, (unsigned int)lineno);
+ cu_row_seq_end[num_entries - 1 - cu_first_entry] = endseq;
+ }
+
+ resolve_cu_row_groups(cu_first_entry);
+
+ /*
+ * DW_LANG_Mips_Assembler is what gas stamps on a .S unit, and
+ * what tells synthesize_symbol_starts() that a row reaching
+ * over a symbol describes it rather than covering padding.
+ */
+ if (dwarf_srclang(&cudie) == DW_LANG_Mips_Assembler)
+ record_cu_asm_spans(cu_first_entry, cu_first_seq_end);
+next:
+ off = next_off;
+ }
+}
+
+/*
+ * Give every function an entry at its own first byte.
+ *
+ * Compilers routinely emit no line row at a symbol's start: .cold
+ * fragments in particular are covered by a row belonging to the function
+ * they were split out of. That used to resolve fine, but the kernel now
+ * refuses any entry below the resolved symbol's start, so those frames
+ * would print unannotated. Copy the covering row down to the symbol
+ * start instead.
+ *
+ * Two things have to hold before that is honest:
+ *
+ * - the covering entry must carry a location. A sequence that ended
+ * before the symbol leaves its line-0 end marker in between, and the
+ * marker is then the covering entry.
+ *
+ * - the line program must place at least one row inside the symbol, so
+ * we know it describes this symbol's code at all. This is what keeps
+ * the __pfx_* padding stubs unannotated: they are pure alignment
+ * padding, and no compiler ever emits a row inside one.
+ *
+ * Symbols failing either test keep no annotation, which is the correct
+ * answer for hand-written assembly.
+ */
+static void synthesize_symbol_starts(void)
+{
+ unsigned int base_entries = num_entries;
+ unsigned int cursor = 0;
+
+ if (!base_entries || !num_sym_starts)
+ return;
+
+ qsort(asm_spans, num_asm_spans, sizeof(*asm_spans), compare_asm_spans);
+
+ for (unsigned int i = 0; i < num_sym_starts; i++) {
+ unsigned int start = sym_starts[i].offset;
+ unsigned int end = start + sym_starts[i].size;
+
+ while (cursor + 1 < base_entries &&
+ entries[cursor + 1].offset <= start)
+ cursor++;
+
+ if (entries[cursor].offset > start)
+ continue; /* nothing covers this symbol */
+ if (entries[cursor].offset == start)
+ continue; /* already has its own entry */
+ if (!entries[cursor].line)
+ continue; /* coverage stopped before here */
+
+ if (end <= start)
+ continue; /* size overflowed */
+
+ /*
+ * A row reaching over a symbol with none inside it is padding
+ * or a split-out fragment in C, carrying the line of whatever
+ * it came from. In assembly it is one macro expanding to a
+ * function per invocation, which the row does describe.
+ */
+ if (!in_asm_span(start) &&
+ (cursor + 1 >= base_entries ||
+ entries[cursor + 1].offset >= end))
+ continue;
+
+ add_entry(start, entries[cursor].file_id, entries[cursor].line);
+ }
+}
+
+/*
+ * Collapse entries that share an address. Within one compile unit
+ * resolve_cu_row_groups() has already picked the row that describes it,
+ * but rows from different units still meet, typically where one unit's
+ * sequence ends and the next unit's code begins. Coverage is their union:
+ * a real location beats another unit's no-coverage marker. This runs
+ * before synthesize_symbol_starts() and the same-line collapse in
+ * deduplicate(), which both expect one entry per address.
+ */
+static void merge_same_offset(void)
+{
+ unsigned int i, j = 0;
+
+ if (num_entries < 2)
+ return;
+
+ for (i = 1; i < num_entries; i++) {
+ if (entries[i].offset == entries[j].offset) {
+ if (entries[i].line || !entries[j].line)
+ entries[j] = entries[i];
+ continue;
+ }
+ if (++j != i)
+ entries[j] = entries[i];
+ }
+ num_entries = j + 1;
+}
+
+static void deduplicate(void)
+{
+ unsigned int sym_cursor = 0;
+ unsigned int i, j;
+
+ if (num_entries < 2)
+ return;
+
+ /* Sort by offset, then by line-program order. */
+ qsort(entries, num_entries, sizeof(*entries), compare_entries);
+ merge_same_offset();
+
+ synthesize_symbol_starts();
+ qsort(entries, num_entries, sizeof(*entries), compare_entries);
+
+ /*
+ * Drop an entry with the same file:line as the previous kept one: it
+ * is redundant for binary search, since any address between them
+ * resolves to the earlier one.
+ *
+ * Entries sitting on a symbol start are exempt: they are the only
+ * thing standing between that symbol and the kernel's boundary check.
+ */
+ j = 0;
+ for (i = 1; i < num_entries; i++) {
+ bool at_symbol_start;
+
+ while (sym_cursor < num_sym_starts &&
+ sym_starts[sym_cursor].offset < entries[i].offset)
+ sym_cursor++;
+ at_symbol_start = sym_cursor < num_sym_starts &&
+ sym_starts[sym_cursor].offset == entries[i].offset;
+
+ if (!at_symbol_start &&
+ entries[i].file_id == entries[j].file_id &&
+ entries[i].line == entries[j].line)
+ continue;
+
+ j++;
+ if (j != i)
+ entries[j] = entries[i];
+ }
+ num_entries = j + 1;
+}
+
+static void compute_file_offsets(void)
+{
+ unsigned int offset = 0;
+
+ for (unsigned int i = 0; i < num_files; i++) {
+ files[i]->str_offset = offset;
+ offset += strlen(files[i]->name) + 1;
+ }
+}
+
+static void print_escaped_asciz(const char *s)
+{
+ printf("\t.asciz \"");
+ for (; *s; s++) {
+ if (*s == '"' || *s == '\\')
+ putchar('\\');
+ putchar(*s);
+ }
+ printf("\"\n");
+}
+
+static void output_assembly(void)
+{
+ printf("/* SPDX-License-Identifier: GPL-2.0 */\n");
+ printf("/*\n");
+ printf(" * Automatically generated by scripts/gen_lineinfo\n");
+ printf(" * Do not edit.\n");
+ printf(" */\n\n");
+
+ printf("\t.section .rodata, \"a\"\n\n");
+
+ /* Number of entries */
+ printf("\t.globl lineinfo_num_entries\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_num_entries:\n");
+ printf("\t.long %u\n\n", num_entries);
+
+ /* Number of files */
+ printf("\t.globl lineinfo_num_files\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_num_files:\n");
+ printf("\t.long %u\n\n", num_files);
+
+ /* Sorted address offsets from _text */
+ printf("\t.globl lineinfo_addrs\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_addrs:\n");
+ for (unsigned int i = 0; i < num_entries; i++)
+ printf("\t.long 0x%x\n", entries[i].offset);
+ printf("\n");
+
+ /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */
+ printf("\t.globl lineinfo_file_ids\n");
+ printf("\t.balign 2\n");
+ printf("lineinfo_file_ids:\n");
+ for (unsigned int i = 0; i < num_entries; i++)
+ printf("\t.short %u\n", entries[i].file_id);
+ printf("\n");
+
+ /* Line numbers, parallel to addrs */
+ printf("\t.globl lineinfo_lines\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_lines:\n");
+ for (unsigned int i = 0; i < num_entries; i++)
+ printf("\t.long %u\n", entries[i].line);
+ printf("\n");
+
+ /* File string offset table */
+ printf("\t.globl lineinfo_file_offsets\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_file_offsets:\n");
+ for (unsigned int i = 0; i < num_files; i++)
+ printf("\t.long %u\n", files[i]->str_offset);
+ printf("\n");
+
+ /* Filenames size */
+ {
+ unsigned int fsize = 0;
+
+ for (unsigned int i = 0; i < num_files; i++)
+ fsize += strlen(files[i]->name) + 1;
+ printf("\t.globl lineinfo_filenames_size\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_filenames_size:\n");
+ printf("\t.long %u\n\n", fsize);
+ }
+
+ /* Concatenated NUL-terminated filenames */
+ printf("\t.globl lineinfo_filenames\n");
+ printf("lineinfo_filenames:\n");
+ for (unsigned int i = 0; i < num_files; i++)
+ print_escaped_asciz(files[i]->name);
+ printf("\n");
+}
+
+int main(int argc, char *argv[])
+{
+ const char *kbuild_verbose = getenv("KBUILD_VERBOSE");
+ unsigned long long text_addr;
+ Dwarf *dwarf;
+ Elf *elf;
+ int fd;
+
+ if (kbuild_verbose && strchr(kbuild_verbose, '1'))
+ verbose = true;
+
+ while (argc > 2 && (!strcmp(argv[1], "-v") ||
+ !strcmp(argv[1], "--verbose"))) {
+ verbose = true;
+ memmove(&argv[1], &argv[2], (argc - 2) * sizeof(char *));
+ argc--;
+ }
+
+ if (argc != 2) {
+ fprintf(stderr, "Usage: %s [-v] <vmlinux>\n", argv[0]);
+ return 1;
+ }
+
+ init_path_roots();
+
+ fd = open(argv[1], O_RDONLY);
+ if (fd < 0)
+ error("cannot open %s: %s", argv[1], strerror(errno));
+
+ elf_version(EV_CURRENT);
+ elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+ if (!elf)
+ error("elf_begin failed: %s", elf_errmsg(elf_errno()));
+
+ text_addr = find_text_addr(elf);
+ text_end_addr = find_text_end_addr(elf);
+ collect_symbol_starts(elf, text_addr);
+
+ dwarf = dwarf_begin_elf(elf, DWARF_C_READ, NULL);
+ if (!dwarf)
+ error("dwarf_begin_elf failed: %s\n"
+ LINEINFO_PREFIX "error: is %s built with CONFIG_DEBUG_INFO?",
+ dwarf_errmsg(dwarf_errno()), argv[1]);
+
+ process_dwarf(dwarf, text_addr);
+
+ if (skipped_overflow)
+ warn("%u entries skipped (offset > 4 GiB from _text)",
+ skipped_overflow);
+
+ deduplicate();
+ compute_file_offsets();
+
+ verbose_msg("%u entries, %u files", num_entries, num_files);
+
+ output_assembly();
+
+ dwarf_end(dwarf);
+ elf_end(elf);
+ close(fd);
+
+ /* Cleanup */
+ free(entries);
+ free(text_starts);
+ free(asm_spans);
+ free(cu_row_seq_end);
+ free(sym_starts);
+ free(seq_ends);
+ for (unsigned int i = 0; i < num_files; i++)
+ free(files[i]);
+ free(files);
+
+ return 0;
+}
diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
index ecb444078c1ca..756f2fdd0a691 100644
--- a/scripts/kallsyms.c
+++ b/scripts/kallsyms.c
@@ -137,6 +137,17 @@ static bool is_ignored_symbol(const char *name, char type)
return true;
}
+ /*
+ * The generated lineinfo tables (scripts/gen_lineinfo, stubbed by
+ * scripts/empty_lineinfo.S) are read-only data whose size and
+ * addresses change between kallsyms passes. Match them by prefix so
+ * the set cannot drift as the table layout evolves. Text symbols are
+ * exempt, so lib/tests/lineinfo_kunit.c's lineinfo_target_*() stay
+ * resolvable -- the test looks them up by name.
+ */
+ if (toupper(type) != 'T' && !strncmp(name, "lineinfo_", 9))
+ return true;
+
return false;
}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d5710e900bd40..e3b416fb1e99c 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -108,7 +108,7 @@ vmlinux_link()
${ld} ${ldflags} -o ${output} \
${wl}--whole-archive ${objs} ${wl}--no-whole-archive \
${wl}--start-group ${libs} ${wl}--end-group \
- ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
+ ${kallsymso} ${lineinfo_o} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
}
# Check if kallsymso_prev and kallsymso differ
@@ -151,6 +151,43 @@ kallsyms()
kallsymso=${2}.o
}
+# Generate lineinfo tables from DWARF debug info in a temporary vmlinux.
+# ${1} - temporary vmlinux with debug info
+# Output: sets lineinfo_o to the generated .o file
+gen_lineinfo()
+{
+ info LINEINFO .tmp_lineinfo.S
+ if ! scripts/gen_lineinfo "${1}" > .tmp_lineinfo.S; then
+ echo >&2 "Failed to generate lineinfo from ${1}"
+ echo >&2 "Try to disable CONFIG_KALLSYMS_LINEINFO"
+ exit 1
+ fi
+
+ info AS .tmp_lineinfo.o
+ ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
+ -c -o .tmp_lineinfo.o .tmp_lineinfo.S
+
+ lineinfo_o=.tmp_lineinfo.o
+ lineinfo_src=${1}
+}
+
+# Compare the layout of the text the lineinfo table covers, everything
+# below _etext, in two linked images.
+# Lineinfo offsets are relative to _text, so anything that moves text after
+# the table was generated invalidates every one of them. Replacing the
+# stub with the real table grows the image by megabytes, and that is exactly
+# what makes the linker insert branch stubs (see the kallsyms comment
+# below), so this has to be checked rather than assumed.
+text_layout_changed()
+{
+ local covered='$3 == "_etext" { e = 1 } !e && $2 ~ /^[tT]$/ { print $1, $3 }'
+
+ ${NM} -n "${1}" | awk "${covered}" > "${1}.text_layout"
+ ${NM} -n "${2}" | awk "${covered}" > "${2}.text_layout"
+ ! cmp -s "${1}.text_layout" "${2}.text_layout"
+}
+
# Perform kallsyms for the given temporary vmlinux.
sysmap_and_kallsyms()
{
@@ -174,6 +211,9 @@ sorttable()
cleanup()
{
rm -f .btf.*
+ rm -f .tmp_lineinfo.*
+ rm -f .tmp_vmlinux_lineinfo*
+ rm -f .tmp_vmlinux*.text_layout
rm -f .tmp_vmlinux.nm-sort
rm -f System.map
rm -f vmlinux
@@ -202,6 +242,8 @@ fi
btf_vmlinux_bin_o=
btfids_vmlinux=
kallsymso=
+lineinfo_o=
+lineinfo_src=
strip_debug=
generate_map=
@@ -217,10 +259,21 @@ if is_enabled CONFIG_KALLSYMS; then
kallsyms .tmp_vmlinux0.syms .tmp_vmlinux0.kallsyms
fi
+if is_enabled CONFIG_KALLSYMS_LINEINFO; then
+ # Assemble an empty lineinfo stub for the initial link.
+ # The real lineinfo is generated from .tmp_vmlinux1 by gen_lineinfo.
+ ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
+ -c -o .tmp_lineinfo.o "${srctree}/scripts/empty_lineinfo.S"
+ lineinfo_o=.tmp_lineinfo.o
+fi
+
if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
- # The kallsyms linking does not need debug symbols, but the BTF does.
- if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+ # The kallsyms linking does not need debug symbols, but BTF and
+ # lineinfo generation do.
+ if ! is_enabled CONFIG_DEBUG_INFO_BTF &&
+ ! is_enabled CONFIG_KALLSYMS_LINEINFO; then
strip_debug=1
fi
@@ -238,6 +291,10 @@ if is_enabled CONFIG_DEBUG_INFO_BTF; then
btfids_vmlinux=.tmp_vmlinux1.BTF_ids
fi
+if is_enabled CONFIG_KALLSYMS_LINEINFO; then
+ gen_lineinfo .tmp_vmlinux1
+fi
+
if is_enabled CONFIG_KALLSYMS; then
# kallsyms support
@@ -285,6 +342,34 @@ if is_enabled CONFIG_KALLSYMS; then
fi
fi
+if is_enabled CONFIG_KALLSYMS_LINEINFO; then
+ # ${lineinfo_src} was linked against the stub, so its text can sit at
+ # different addresses than the image we are about to ship. When that
+ # happens, regenerate from an image that already carries a real-sized
+ # table and link once more; a second shift would mean the layout is
+ # not settling, which is worth failing over rather than shipping line
+ # numbers that point at the wrong source.
+ lineinfo_last=${kallsyms_sysmap%.syms}
+ lineinfo_last=${lineinfo_last:-.tmp_vmlinux1}
+
+ if text_layout_changed "${lineinfo_src}" "${lineinfo_last}"; then
+ info LINEINFO "text moved, regenerating"
+ strip_debug=
+ vmlinux_link .tmp_vmlinux_lineinfo
+ gen_lineinfo .tmp_vmlinux_lineinfo
+ strip_debug=1
+ vmlinux_link .tmp_vmlinux_lineinfo2
+ if is_enabled CONFIG_KALLSYMS; then
+ sysmap_and_kallsyms .tmp_vmlinux_lineinfo2
+ fi
+ if text_layout_changed "${lineinfo_src}" .tmp_vmlinux_lineinfo2; then
+ echo >&2 "Failed to settle text layout for lineinfo"
+ echo >&2 "Try to disable CONFIG_KALLSYMS_LINEINFO"
+ exit 1
+ fi
+ fi
+fi
+
strip_debug=
if is_enabled CONFIG_VMLINUX_MAP; then
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH v11 1/4] kallsyms: embed source file:line info in kernel stack traces
2026-11-06 13:24 ` [PATCH v11 1/4] " Sasha Levin
@ 2026-11-06 14:52 ` Helge Deller
0 siblings, 0 replies; 6+ messages in thread
From: Helge Deller @ 2026-11-06 14:52 UTC (permalink / raw)
To: Sasha Levin, Andrew Morton, Masahiro Yamada, Luis Chamberlain,
Linus Torvalds, Richard Weinberger, Juergen Gross,
Geert Uytterhoeven, James Bottomley
Cc: Jonathan Corbet, Nathan Chancellor, Nicolas Schier, Petr Pavlu,
Daniel Gomez, Aaron Tomlin, Greg KH, Petr Mladek, Steven Rostedt,
Kees Cook, Peter Zijlstra, Thorsten Leemhuis, Vlastimil Babka,
Randy Dunlap, Laurent Pinchart, Vivian Wang, Zhen Lei,
Sami Tolvanen, linux-kernel, linux-kbuild, linux-modules,
linux-doc
On 11/6/26 14:24, Sasha Levin wrote:
> Add CONFIG_KALLSYMS_LINEINFO, which embeds a compact address-to-line
> lookup table in the kernel image so stack traces directly print source
> file and line number information:
>
> root@localhost:~# echo c > /proc/sysrq-trigger
> [ 11.201987] sysrq: Trigger a crash
> [ 11.202831] Kernel panic - not syncing: sysrq triggered crash
> [ 11.206218] Call Trace:
> [ 11.206501] <TASK>
> [ 11.206749] dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:94)
> [ 11.207403] vpanic+0x36e/0x620 (kernel/panic.c:650)
> [ 11.208565] ? __lock_acquire+0x465/0x2240 (kernel/locking/lockdep.c:4674)
> [ 11.209324] panic+0xc9/0xd0 (kernel/panic.c:787)
> [ 11.211873] ? find_held_lock+0x2b/0x80 (kernel/locking/lockdep.c:5350)
> [ 11.212597] ? lock_release+0xd3/0x300 (kernel/locking/lockdep.c:5535)
> [ 11.213312] sysrq_handle_crash+0x1a/0x20 (drivers/tty/sysrq.c:154)
> [ 11.214005] __handle_sysrq.cold+0x66/0x256 (drivers/tty/sysrq.c:611)
> [ 11.214712] write_sysrq_trigger+0x65/0x80 (drivers/tty/sysrq.c:1221)
> [ 11.215424] proc_reg_write+0x1bd/0x3c0 (fs/proc/inode.c:330)
> [ 11.216061] vfs_write+0x1c6/0xff0 (fs/read_write.c:686)
> [ 11.218848] ksys_write+0xfa/0x200 (fs/read_write.c:740)
> [ 11.222394] do_syscall_64+0xf3/0x690 (arch/x86/entry/syscall_64.c:63)
> [ 11.223942] entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/entry_64.S:121)
>
> At build time, a new host tool (scripts/gen_lineinfo) reads DWARF
> .debug_line from vmlinux using libdw (elfutils), extracts all
> address-to-file:line mappings, and generates an assembly file with
> sorted parallel arrays (offsets from _text, file IDs, and line
> numbers). These are linked into vmlinux as .rodata.
>
> At runtime, kallsyms_lookup_lineinfo() does a binary search on the
> table and __sprint_symbol() appends "(file:line)" to each stack frame.
> The lookup uses offsets from _text so it works with KASLR, requires no
> locks or allocations, and is safe in any context including panic.
>
> The suffix is appended for every symbol variant that already prints an
> offset, so %pS, %pSb and the %pB backtrace forms all carry it: arm64
> prints frames with %pSb and the generic stack_trace_print() uses %pS.
> sprint_symbol_no_offset(), which backs lowercase %ps, stays bare
> because callers suffix it with a literal "()".
>
> A line program emits several rows per address, and every row but the
> last covers zero bytes, so the generator keeps the last row at each
> address. That is the row addr2line reports, and for an inlined call it
> names the inlined body rather than the call site.
>
> Rows the compiler marks with line 0 declare that no source location
> applies from there on (DWARF5 6.2.2), and the end of a line program's
> sequence says the same of the bytes past it. Both are kept, as entries
> whose line is zero that the lookup reads as "no annotation", so an
> address the compiler deliberately left unattributed reports nothing
> instead of inheriting the line above it.
>
> The feature requires CONFIG_DEBUG_INFO (for DWARF data) and libelf
> and libdw (from elfutils) on the build host.
>
> Memory footprint, measured with:
>
> make ARCH=x86_64 O=$B defconfig
> ./scripts/config --file $B/.config -e DEBUG_INFO \
> -e DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT
> make ARCH=x86_64 O=$B olddefconfig && make ARCH=x86_64 O=$B -j$(nproc)
> strip -g $B/vmlinux -o vmlinux.nodbg && stat -c %s vmlinux.nodbg
>
> Table: 1,657,663 entries from 4,234 source files
> lineinfo_addrs[] 1,657,663 x u32 = 6.3 MiB
> lineinfo_file_ids[] 1,657,663 x u16 = 3.2 MiB
> lineinfo_lines[] 1,657,663 x u32 = 6.3 MiB
> file_offsets + filenames = 0.1 MiB
> Total .rodata increase: = 15.9 MiB
>
> vmlinux (stripped): 52.2 MiB -> 68.2 MiB (+16.0 MiB / +30.6%)
>
> That is the cost of the uncompressed format introduced here; the delta
> compression added later in this series brings it down to +8.0 MiB
> (+15.3%) on the same config.
>
> Suggested-by: Petr Pavlu <petr.pavlu@suse.com>
> Signed-off-by: Sasha Levin <sashal@kernel.org>
Tested-by: Helge Deller <deller@gmx.de> # parisc
Acked-by: Helge Deller <deller@gmx.de>
Example:
root@62:/# echo c > /proc/sysrq-trigger
[ 95.740284] sysrq: Trigger a crash
[ 95.745518] Kernel panic - not syncing: sysrq triggered crash
[ 95.746724] CPU: 1 UID: 0 PID: 1 Comm: bash Not tainted 7.4.0-rc1+ #3263 VOLUNTARY
[ 95.746724] Hardware name: 9000/778/B160L
[ 95.746724] Backtrace:
[ 95.746724] [<1041d778>] show_stack+0x54/0x78 (arch/parisc/kernel/traps.c:212)
[ 95.746724] [<10413404>] dump_stack_lvl+0x80/0xc8 (lib/dump_stack.c:122)
[ 95.746724] [<10413468>] dump_stack+0x1c/0x2c (lib/dump_stack.c:130)
[ 95.746724] [<10401288>] vpanic+0x14c/0x340 (kernel/panic.c:561)
[ 95.746724] [<104014b0>] panic+0x34/0x48 (kernel/panic.c:787)
[ 95.746724] [<10e6cbcc>] sysrq_handle_crash+0x20/0x34 (drivers/tty/sysrq.c:154)
[ 95.746724] [<10e6d734>] __handle_sysrq+0xb4/0x1d8 (drivers/tty/sysrq.c:635)
[ 95.746724] [<10e6e1b4>] write_sysrq_trigger+0x90/0xe0 (drivers/tty/sysrq.c:1220)
[ 95.746724] [<10977d88>] proc_reg_write+0xd4/0x120 (fs/proc/inode.c:343)
[ 95.746724] [<1088c9cc>] vfs_write+0xbc/0x468 (fs/read_write.c:690)
[ 95.746724] [<1088cf5c>] ksys_write+0x7c/0x12c (fs/read_write.c:740)
[ 95.746724] [<1088d028>] sys_write+0x1c/0x3c (fs/read_write.c:747)
[ 95.746724] [<10418334>] syscall_exit+0x0/0x10 (arch/parisc/kernel/entry.S:1722)
[ 95.746724]
[ 95.746724] ---[ end Kernel panic - not syncing: sysrq triggered crash ]---
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v11 2/4] kallsyms: extend lineinfo to loadable modules
2026-11-06 13:24 [PATCH v11 0/4] kallsyms: embed source file:line info in kernel stack traces Sasha Levin
2026-11-06 13:24 ` [PATCH v11 1/4] " Sasha Levin
@ 2026-11-06 13:24 ` Sasha Levin
2026-11-06 13:24 ` [PATCH v11 3/4] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction Sasha Levin
2026-11-06 13:24 ` [PATCH v11 4/4] kallsyms: add KUnit tests for lineinfo feature Sasha Levin
3 siblings, 0 replies; 6+ messages in thread
From: Sasha Levin @ 2026-11-06 13:24 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Luis Chamberlain, Linus Torvalds,
Richard Weinberger, Juergen Gross, Geert Uytterhoeven,
James Bottomley
Cc: Sasha Levin, Jonathan Corbet, Nathan Chancellor, Nicolas Schier,
Petr Pavlu, Daniel Gomez, Aaron Tomlin, Greg KH, Petr Mladek,
Steven Rostedt, Kees Cook, Peter Zijlstra, Thorsten Leemhuis,
Vlastimil Babka, Helge Deller, Randy Dunlap, Laurent Pinchart,
Vivian Wang, Zhen Lei, Sami Tolvanen, linux-kernel, linux-kbuild,
linux-modules, linux-doc
Add CONFIG_KALLSYMS_LINEINFO_MODULES, which extends the
CONFIG_KALLSYMS_LINEINFO feature to loadable kernel modules.
At build time, each .ko is post-processed by scripts/gen-mod-lineinfo.sh
(modeled on gen-btf.sh) which runs scripts/gen_lineinfo --module on the
.ko, generates per-section .mod_lineinfo and .init.mod_lineinfo
sections containing compact binary tables of section-relative offsets,
file IDs, line numbers, and filenames, and embeds them back into the
.ko via a partial link (ld -r).
At runtime, module_lookup_lineinfo() walks the section descriptors in
each blob, finds the one whose runtime range contains the queried
address, and binary-searches that section's table. The lookup is
NMI/panic-safe (no locks, no allocations) — the data lives in
read-only module memory and is freed automatically when the module
(or its init memory) is unloaded.
The gen_lineinfo tool gains --module mode which:
- Walks an allowlist of text-like sections (.text, .exit.text,
.init.text), gating each on its presence in the .ko.
- Uses an ELF relocation against each covered section's symbol as the
runtime "anchor", resolved by the module loader's standard
apply_relocations() pass — no implicit base derivation from
mod->mem[].base, no special-cased loader logic.
- Applies the relocations libdw does not: in an ET_REL .ko a compile
unit's DW_AT_abbrev_offset and DW_AT_stmt_list, its DW_FORM_strp
names and the DW_LNE_set_address PCs are all relocations, and libdw
reads zero for every one of them. Only the first compile unit --
whose offsets genuinely are zero -- would otherwise decode, leaving
a module built from several objects with line info for just one of
them. apply_debug_relocations() patches .debug_line, .debug_info,
.debug_str_offsets, .debug_addr, .debug_rnglists and .debug_loclists
in the mutable ELF copy before dwarf_begin_elf() sees it.
- Disambiguates DWARF addresses across sections that all share
sh_addr == 0 in ET_REL files via per-section synthetic biases
applied to .debug_line relocations (handles both abs32 and abs64
width relocs).
- Expands SHF_COMPRESSED debug sections before patching them. With
CONFIG_DEBUG_INFO_COMPRESSED_* the section data is compressed while
relocation offsets address the uncompressed contents, so patching
without expanding first corrupts the stream.
- Applies the arithmetic relocation pairs RISC-V and LoongArch use for
label differences (R_RISCV_ADD16/SUB16 and friends), which clang
emits for line-program address advances even with -mno-relax. Any
relocation left unapplied in a patched debug section aborts the
object instead of shipping line numbers shifted by the gap.
- Handles libdw's ET_REL path-doubling quirk in make_relative().
- Declares empty section stanzas in its output assembly so the
resulting lineinfo.o has LOCAL SECTION symbols rather than GLOBAL
UND ones; otherwise ld -r would not bind the relocation to the
.ko's existing section symbol of the same name and depmod would
warn.
The table describes the module's own section layout, so modfinal builds
it from a partial link and merges it in the link that produces the .ko
rather than post-processing a finished module:
${LD} -r ${KBUILD_LDFLAGS} ${layout_flags} -T module.lds \
-o ${KO}.lineinfo_pre $(filter %.o, $^)
gen-mod-lineinfo.sh ${KO}.lineinfo_pre ${KO}.lineinfo.o
${LD} -r ${KBUILD_LDFLAGS} ${last_flags} -o ${KO} \
${KO}.lineinfo.o ${KO}.lineinfo_pre
Only --be8 and --build-id are held back for the second link: the first
converts the object to a form the linker will not read again, and the
second has to cover the module that ships. Everything else runs in the
first link, where the table is measured, since text linked ahead of the
module would move the code the table describes away from its anchor.
The second link runs without module.lds, which the first one already
applied. Run again, the script finds alloc_tags already renamed to
.codetag.alloc_tags and leaves __start_alloc_tags/__stop_alloc_tags
around an empty range, so allocation profiling never sets up the
module's counters, and it emits the BYTE(0) stub sections of arm and
arm64 a second time, which the module loader rejects.
KBUILD_LDFLAGS carries the target linker emulation, without which an
ARCH=i386 build on a biarch host links the 32-bit objects with the
host's default 64-bit emulation and fails, and -z noexecstack, which
the generated object has no note of its own for.
Order matters: lineinfo.o must come first so its zero-byte text
contributions stay at offset 0 of the merged sections.
The init blob lives in MOD_INIT_RODATA and is revoked via WRITE_ONCE
in do_init_module() before do_free_init() releases the memory; the
module_init_lineinfo_data() reader uses READ_ONCE so concurrent
lookups either see the old pointer (still valid until do_free_init's
synchronize_rcu) or NULL.
The struct module fields are guarded by CONFIG_KALLSYMS_LINEINFO_MODULES
and accessed through inline reader accessors so callers don't
duplicate the guard.
Per-module overhead is approximately 10 bytes per DWARF line entry
plus a small fixed cost per covered section descriptor. The next
patch in this series delta-compresses the per-section streams to ~3-4
bytes per entry.
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
.../admin-guide/kallsyms-lineinfo.rst | 43 +-
MAINTAINERS | 3 +
include/linux/mod_lineinfo.h | 130 ++
include/linux/module.h | 40 +
init/Kconfig | 13 +
kernel/kallsyms.c | 19 +-
kernel/module/kallsyms.c | 213 +++
kernel/module/main.c | 26 +
scripts/Makefile.modfinal | 44 +-
scripts/gen-mod-lineinfo.sh | 45 +
scripts/gen_lineinfo.c | 1447 +++++++++++++++--
11 files changed, 1891 insertions(+), 132 deletions(-)
create mode 100644 include/linux/mod_lineinfo.h
create mode 100755 scripts/gen-mod-lineinfo.sh
diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentation/admin-guide/kallsyms-lineinfo.rst
index 9bf0190798647..9b91adcf82ea3 100644
--- a/Documentation/admin-guide/kallsyms-lineinfo.rst
+++ b/Documentation/admin-guide/kallsyms-lineinfo.rst
@@ -52,22 +52,49 @@ Assembly routines such as ``entry_SYSCALL_64_after_hwframe`` are annotated
too, from the line information the assembler emits when debug info is
enabled.
+Module Support
+==============
+
+``CONFIG_KALLSYMS_LINEINFO_MODULES`` extends the feature to loadable kernel
+modules. When enabled, each ``.ko`` is post-processed at build time to embed
+a ``.mod_lineinfo`` section containing the same kind of address-to-source
+mapping.
+
+Enable in addition to the base options::
+
+ CONFIG_MODULES=y
+ CONFIG_KALLSYMS_LINEINFO_MODULES=y
+
+Stack traces from module code will then include annotations::
+
+ my_driver_func+0x30/0x100 [my_driver] (drivers/foo/bar.c:123)
+
+The ``.mod_lineinfo`` section is loaded into read-only module memory alongside
+the module text. No additional runtime memory allocation is required; the data
+is freed when the module is unloaded.
+
Memory Overhead
===============
-The lineinfo tables are stored in ``.rodata``. On an x86_64 ``defconfig``
-with ``CONFIG_DEBUG_INFO`` they hold 1.66 million entries and grow the
-stripped image by 16 MiB, about 10 bytes per entry after deduplication.
+The vmlinux lineinfo tables are stored in ``.rodata``. On an x86_64
+``defconfig`` with ``CONFIG_DEBUG_INFO`` they hold 1.66 million entries and
+grow the stripped image by 16 MiB, about 10 bytes per entry after
+deduplication.
+
+Per-module lineinfo adds about 10 bytes per entry to each ``.ko`` file, plus
+a small fixed cost per covered section.
Known Limitations
=================
-- **vmlinux only**: Only symbols in the core kernel image are annotated.
- Module symbols are not covered.
-- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-bit
- values. Entries beyond 4 GiB from ``_text`` are skipped at build time with
- a warning.
+- **4 GiB offset limit**: Address offsets from ``_text`` (vmlinux) or
+ ``.text`` base (modules) are stored as 32-bit values. Entries beyond
+ 4 GiB are skipped at build time with a warning.
- **65535 file limit**: Source file IDs are stored as 16-bit values. Builds
with more than 65535 unique source files will fail with an error.
- **Code without line information**: Functions that no DWARF
``.debug_line`` row covers are not annotated.
+- **Module init text**: A module's ``.init.text`` is annotated from a separate
+ ``.init.mod_lineinfo`` section, which is released along with the module's
+ init memory. Traces taken after the module has finished initializing carry
+ no annotation for those addresses.
diff --git a/MAINTAINERS b/MAINTAINERS
index f21f932f40a24..93fbf410506f7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -14345,6 +14345,9 @@ KALLSYMS LINEINFO
M: Sasha Levin <sashal@kernel.org>
S: Maintained
F: Documentation/admin-guide/kallsyms-lineinfo.rst
+F: include/linux/mod_lineinfo.h
+F: lib/tests/lineinfo_kunit.c
+F: scripts/gen-mod-lineinfo.sh
F: scripts/gen_lineinfo.c
KANDOU KB9002 PCIE RETIMER HWMON DRIVER
diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h
new file mode 100644
index 0000000000000..cb0c7af7b3171
--- /dev/null
+++ b/include/linux/mod_lineinfo.h
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * mod_lineinfo.h - Binary format for per-module source line information
+ *
+ * This header defines the layout of the .mod_lineinfo and
+ * .init.mod_lineinfo sections embedded in loadable kernel modules. It
+ * is dual-use: included from both the kernel and the userspace
+ * gen_lineinfo tool.
+ *
+ * Top-level layout (all values in target-native endianness):
+ *
+ * struct mod_lineinfo_root
+ * struct mod_lineinfo_section sections[hdr.num_sections]
+ * ... per-section sub-tables, each pointed at by sections[i].table_offset
+ *
+ * Each mod_lineinfo_section descriptor identifies one ELF text section
+ * covered by the lineinfo blob. Its .anchor field is an ELF relocation
+ * resolved at module-load time to the runtime base of the named section,
+ * eliminating the need to derive the base from mod->mem[].base segments.
+ * If the relocation fails to resolve (e.g. unknown reloc type), .anchor
+ * stays zero and lookups silently degrade to "no annotation".
+ *
+ * Each per-section sub-table is laid out as a stand-alone
+ * mod_lineinfo_header followed by parallel arrays:
+ *
+ * struct mod_lineinfo_header (16 bytes)
+ * u32 addrs[num_entries] -- offsets from this section's base, sorted
+ * u16 file_ids[num_entries] -- parallel to addrs
+ * <2-byte pad if num_entries is odd>
+ * u32 lines[num_entries] -- parallel to addrs
+ * u32 file_offsets[num_files] -- byte offset into filenames[]
+ * char filenames[filenames_size] -- concatenated NUL-terminated strings
+ */
+#ifndef _LINUX_MOD_LINEINFO_H
+#define _LINUX_MOD_LINEINFO_H
+
+#ifdef __KERNEL__
+#include <linux/types.h>
+#include <linux/build_bug.h>
+#include <linux/stddef.h>
+#else
+#include <assert.h>
+#include <stddef.h>
+#include <stdint.h>
+typedef uint32_t u32;
+typedef uint16_t u16;
+typedef uint64_t u64;
+#ifndef __aligned
+#define __aligned(x) __attribute__((__aligned__(x)))
+#endif
+#ifndef static_assert
+#define static_assert(e, ...) _Static_assert(e, #e)
+#endif
+#endif
+
+/*
+ * Per-section descriptor. One entry per ELF text section covered by the
+ * blob (.text, .exit.text, .init.text, ...).
+ */
+struct mod_lineinfo_section {
+ u64 anchor; /* RELOC: runtime base of covered section, or 0 */
+ u32 size; /* covered section size in bytes */
+ u32 table_offset; /* byte offset from blob start to this section's
+ * mod_lineinfo_header */
+} __aligned(8);
+
+/*
+ * Top-level header. Sits at offset 0 of every .mod_lineinfo /
+ * .init.mod_lineinfo section. The generated assembly pads to an 8-byte
+ * boundary after num_sections, so sections[0] must start at offset 8.
+ * The __aligned(8) on struct mod_lineinfo_section guarantees that even on
+ * 32-bit targets where the natural alignment of u64 is smaller (4 on i386,
+ * 2 on m68k) and the compiler would otherwise place sections[] at offset 4.
+ */
+struct mod_lineinfo_root {
+ u32 num_sections;
+ struct mod_lineinfo_section sections[];
+};
+
+static_assert(offsetof(struct mod_lineinfo_root, sections) == 8,
+ "blob layout: sections[] must sit at offset 8 to match the generated assembly");
+static_assert(sizeof(struct mod_lineinfo_section) == 16,
+ "blob layout: section descriptors are 16 bytes in the generated assembly");
+
+struct mod_lineinfo_header {
+ u32 num_entries;
+ u32 num_files;
+ u32 filenames_size; /* total bytes of concatenated filenames */
+};
+
+/* Offset helpers: compute byte offset from the per-section header to each array. */
+
+static inline u32 mod_lineinfo_addrs_off(void)
+{
+ return sizeof(struct mod_lineinfo_header);
+}
+
+/*
+ * The counts come from an on-disk blob and are only validated against the
+ * blob size once the full layout has been summed, so every step widens to
+ * u64: at 10 bytes per entry the u32 sums wrap for counts a caller can
+ * name, which would let a malformed blob pass a bounds check computed from
+ * the wrapped value.
+ */
+static inline u64 mod_lineinfo_file_ids_off(u32 num_entries)
+{
+ return mod_lineinfo_addrs_off() + (u64)num_entries * sizeof(u32);
+}
+
+static inline u64 mod_lineinfo_lines_off(u32 num_entries)
+{
+ /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */
+ u64 off = mod_lineinfo_file_ids_off(num_entries) +
+ (u64)num_entries * sizeof(u16);
+ return (off + 3) & ~3ULL;
+}
+
+static inline u64 mod_lineinfo_file_offsets_off(u32 num_entries)
+{
+ return mod_lineinfo_lines_off(num_entries) +
+ (u64)num_entries * sizeof(u32);
+}
+
+static inline u64 mod_lineinfo_filenames_off(u32 num_entries, u32 num_files)
+{
+ return mod_lineinfo_file_offsets_off(num_entries) +
+ (u64)num_files * sizeof(u32);
+}
+
+#endif /* _LINUX_MOD_LINEINFO_H */
diff --git a/include/linux/module.h b/include/linux/module.h
index 76f4f35964773..40ca276c3212e 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -507,6 +507,12 @@ struct module {
unsigned int btf_inline_data_size;
void *btf_inline_data;
#endif
+#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+ void *lineinfo_data; /* .mod_lineinfo section in MOD_RODATA */
+ unsigned int lineinfo_data_size;
+ void *init_lineinfo_data; /* .init.mod_lineinfo, NULL after init runs */
+ unsigned int init_lineinfo_data_size;
+#endif
#ifdef CONFIG_JUMP_LABEL
struct jump_entry *jump_entries;
unsigned int num_jump_entries;
@@ -1029,6 +1035,40 @@ static inline unsigned long find_kallsyms_symbol_value(struct module *mod,
#endif /* CONFIG_MODULES && CONFIG_KALLSYMS */
+bool module_lookup_lineinfo(struct module *mod, unsigned long addr,
+ unsigned long sym_start,
+ const char **file, unsigned int *line);
+
+/*
+ * Reader accessors so callers don't need to duplicate the
+ * CONFIG_KALLSYMS_LINEINFO_MODULES guard around mod->lineinfo_data /
+ * mod->init_lineinfo_data field access. Setters/clearers in the loader
+ * use the field directly under a matching #ifdef.
+ */
+static inline void *module_lineinfo_data(const struct module *mod,
+ unsigned int *size)
+{
+#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+ *size = mod->lineinfo_data_size;
+ return mod->lineinfo_data;
+#else
+ *size = 0;
+ return NULL;
+#endif
+}
+
+static inline void *module_init_lineinfo_data(const struct module *mod,
+ unsigned int *size)
+{
+#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+ *size = READ_ONCE(mod->init_lineinfo_data_size);
+ return READ_ONCE(mod->init_lineinfo_data);
+#else
+ *size = 0;
+ return NULL;
+#endif
+}
+
/* Define __free(module_put) macro for struct module *. */
DEFINE_FREE(module_put, struct module *, if (_T) module_put(_T))
diff --git a/init/Kconfig b/init/Kconfig
index dc98f61801c74..2a5a584b9e369 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1959,6 +1959,19 @@ config KALLSYMS_LINEINFO
If unsure, say N.
+config KALLSYMS_LINEINFO_MODULES
+ bool "Embed source file:line information in module stack traces"
+ depends on KALLSYMS_LINEINFO && MODULES
+ help
+ Extends KALLSYMS_LINEINFO to loadable kernel modules. Each .ko
+ gets a lineinfo table generated from its DWARF data at build time,
+ so stack traces from module code include (file.c:123) annotations.
+
+ This requires libelf and libdw (from elfutils) on the build host.
+ Costs 10 bytes per DWARF line entry in each .ko.
+
+ If unsure, say N.
+
# end of the "standard kernel features (expert users)" menu
config ARCH_HAS_MEMBARRIER_CALLBACKS
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index 111df61749f2e..77543ac216a92 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -586,12 +586,25 @@ static int __sprint_symbol(char *buffer, unsigned long address,
* onto that to denote a call ("foo() replaced with bar()"), which
* "foo (file:line)()" would render unreadable.
*/
- if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) && !modname) {
+ if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO)) {
const char *li_file;
unsigned int li_line;
+ bool found = false;
+
+ if (!modname) {
+ found = kallsyms_lookup_lineinfo(address, sym_start,
+ &li_file, &li_line);
+ } else if (IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES)) {
+ struct module *mod = __module_address(address);
+
+ if (mod)
+ found = module_lookup_lineinfo(mod, address,
+ sym_start,
+ &li_file,
+ &li_line);
+ }
- if (kallsyms_lookup_lineinfo(address, sym_start,
- &li_file, &li_line))
+ if (found)
len += scnprintf(buffer + len, KSYM_SYMBOL_LEN - len,
" (%s:%u)", li_file, li_line);
}
diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c
index 1ff8ce857d925..a4eaf28fffe75 100644
--- a/kernel/module/kallsyms.c
+++ b/kernel/module/kallsyms.c
@@ -505,3 +505,216 @@ int module_kallsyms_on_each_symbol(const char *modname,
mutex_unlock(&module_mutex);
return ret;
}
+
+#include <linux/mod_lineinfo.h>
+
+/*
+ * Search one per-section sub-table for @section_offset using flat parallel
+ * arrays. @hdr is the per-section header at byte offset @hdr_offset within
+ * @blob. Returns true on hit and populates @file / @line.
+ */
+static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
+ u32 hdr_offset,
+ unsigned int section_offset,
+ unsigned int min_offset,
+ const char **file,
+ unsigned int *line)
+{
+ const struct mod_lineinfo_header *hdr;
+ const u8 *base;
+ const u32 *addrs, *lines, *file_offsets;
+ const u16 *file_ids;
+ const char *filenames;
+ u32 num_entries, num_files, filenames_size;
+ unsigned int low, high, mid;
+ u16 file_id;
+
+ if (hdr_offset > blob_size ||
+ blob_size - hdr_offset < sizeof(*hdr))
+ return false;
+
+ /*
+ * The header and every array it points at are u32-aligned by
+ * construction. Refuse anything else rather than take an alignment
+ * fault here -- this runs from NMI and panic context, where a
+ * recursive fault would lose the backtrace entirely.
+ */
+ if (!IS_ALIGNED(hdr_offset, sizeof(u32)))
+ return false;
+
+ base = (const u8 *)blob + hdr_offset;
+ hdr = (const struct mod_lineinfo_header *)base;
+ num_entries = hdr->num_entries;
+ num_files = hdr->num_files;
+ filenames_size = hdr->filenames_size;
+
+ if (num_entries == 0)
+ return false;
+
+ /*
+ * Check the whole layout against the blob in one go. The offset
+ * helpers sum in u64 precisely because a malformed blob can name
+ * counts whose u32 sum wraps: at 10 bytes per entry across addrs[],
+ * file_ids[] and lines[], num_entries = 0x33333334 wraps to a small
+ * value that any bounds check would happily accept.
+ */
+ {
+ u32 avail = blob_size - hdr_offset;
+ u64 needed = mod_lineinfo_filenames_off(num_entries, num_files);
+
+ if (needed > avail || filenames_size > avail - needed)
+ return false;
+ }
+
+ /*
+ * Filenames are read as NUL-terminated C strings. Require the blob
+ * to end in NUL so a malformed file_offsets entry can never lead the
+ * later "%s" consumer past the end of the section.
+ */
+ if (filenames_size == 0 ||
+ base[mod_lineinfo_filenames_off(num_entries, num_files) +
+ filenames_size - 1] != 0)
+ return false;
+
+ addrs = (const u32 *)(base + mod_lineinfo_addrs_off());
+ file_ids = (const u16 *)(base + mod_lineinfo_file_ids_off(num_entries));
+ lines = (const u32 *)(base + mod_lineinfo_lines_off(num_entries));
+ file_offsets = (const u32 *)(base + mod_lineinfo_file_offsets_off(num_entries));
+ filenames = (const char *)(base + mod_lineinfo_filenames_off(num_entries, num_files));
+
+ /* Binary search for largest entry <= section_offset. */
+ low = 0;
+ high = num_entries;
+ while (low < high) {
+ mid = low + (high - low) / 2;
+ if (addrs[mid] <= section_offset)
+ low = mid + 1;
+ else
+ high = mid;
+ }
+
+ if (low == 0)
+ return false;
+ low--;
+
+ /*
+ * Reject entries below the resolved symbol's start so a symbol
+ * without line entries of its own does not inherit the preceding
+ * symbol's annotation.
+ */
+ if (addrs[low] < min_offset)
+ return false;
+
+ /*
+ * A zero line is the generator's "no source location applies here"
+ * marker, taken straight from a DWARF line-0 row.
+ */
+ if (!lines[low])
+ return false;
+
+ file_id = file_ids[low];
+ if (file_id >= num_files)
+ return false;
+ if (file_offsets[file_id] >= filenames_size)
+ return false;
+
+ *file = &filenames[file_offsets[file_id]];
+ *line = lines[low];
+ return true;
+}
+
+/*
+ * Walk a single .mod_lineinfo / .init.mod_lineinfo blob, find the section
+ * descriptor whose [anchor, anchor+size) range contains @addr, then search
+ * that section's sub-table.
+ */
+static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size,
+ unsigned long addr,
+ unsigned long sym_start,
+ const char **file, unsigned int *line)
+{
+ const struct mod_lineinfo_root *root;
+ u32 i, sections_end;
+
+ if (!blob || blob_size < sizeof(*root))
+ return false;
+
+ /* The section is emitted with .balign 8; see the note above. */
+ if (!IS_ALIGNED((unsigned long)blob, __alignof__(struct mod_lineinfo_section)))
+ return false;
+
+ root = blob;
+ if (root->num_sections == 0)
+ return false;
+
+ if (root->num_sections > U32_MAX / sizeof(struct mod_lineinfo_section))
+ return false;
+ sections_end = sizeof(*root) +
+ root->num_sections * sizeof(struct mod_lineinfo_section);
+ if (sections_end > blob_size)
+ return false;
+
+ for (i = 0; i < root->num_sections; i++) {
+ const struct mod_lineinfo_section *s = &root->sections[i];
+ unsigned long base = (unsigned long)s->anchor;
+ unsigned long offset, min_offset = 0;
+
+ if (!base)
+ continue; /* relocation didn't resolve */
+ if (addr < base)
+ continue;
+ offset = addr - base;
+ /* s->size is u32, so this also bounds offset to u32. */
+ if (offset >= s->size)
+ continue;
+
+ if (sym_start > base && sym_start - base <= offset)
+ min_offset = sym_start - base;
+
+ return module_lookup_lineinfo_section(blob, blob_size,
+ s->table_offset,
+ offset, min_offset,
+ file, line);
+ }
+
+ return false;
+}
+
+/*
+ * Look up source file:line for an address within a loaded module.
+ *
+ * Safe in NMI/panic context: no locks, no allocations.
+ * Caller must hold RCU read lock (or be in a context where the module
+ * cannot be unloaded).
+ */
+bool module_lookup_lineinfo(struct module *mod, unsigned long addr,
+ unsigned long sym_start,
+ const char **file, unsigned int *line)
+{
+ const void *blob;
+ unsigned int size;
+
+ if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES))
+ return false;
+
+ blob = module_lineinfo_data(mod, &size);
+ if (blob && module_lookup_lineinfo_blob(blob, size, addr, sym_start,
+ file, line))
+ return true;
+
+ /*
+ * The init blob lives in MOD_INIT_RODATA and is revoked by
+ * do_init_module() before do_free_init() releases the memory. The
+ * READ_ONCE inside module_init_lineinfo_data() pairs with the
+ * WRITE_ONCE in do_init_module so we never see a partial
+ * pointer/size pair, and an RCU grace period in do_free_init()
+ * guarantees the memory still exists for the duration of any lookup
+ * that captured the pointer before the revocation.
+ */
+ blob = module_init_lineinfo_data(mod, &size);
+ if (blob && module_lookup_lineinfo_blob(blob, size, addr, sym_start,
+ file, line))
+ return true;
+
+ return false;
+}
diff --git a/kernel/module/main.c b/kernel/module/main.c
index 34e0204ea37c1..296a3c57adf60 100644
--- a/kernel/module/main.c
+++ b/kernel/module/main.c
@@ -2739,6 +2739,19 @@ static int find_module_sections(struct module *mod, struct load_info *info)
mod->btf_inline_data = any_section_objs(info, ".BTF.inline", 1,
&mod->btf_inline_data_size);
#endif
+#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+ /*
+ * Use section_objs() (not any_section_objs) — both blobs carry an
+ * ELF anchor relocation that the module loader resolves via its
+ * standard apply_relocations() pass, which only walks SHF_ALLOC
+ * sections. Picking up a non-ALLOC section here would also leave
+ * the pointer dangling into the temporary load image once freed.
+ */
+ mod->lineinfo_data = section_objs(info, ".mod_lineinfo", 1,
+ &mod->lineinfo_data_size);
+ mod->init_lineinfo_data = section_objs(info, ".init.mod_lineinfo", 1,
+ &mod->init_lineinfo_data_size);
+#endif
#ifdef CONFIG_JUMP_LABEL
mod->jump_entries = section_objs(info, "__jump_table",
sizeof(*mod->jump_entries),
@@ -3195,6 +3208,19 @@ static noinline int do_init_module(struct module *mod)
#endif
#if IS_ENABLED(CONFIG_DEBUG_INFO_BTF_INLINE)
mod->btf_inline_data = NULL;
+#endif
+#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+ /*
+ * .init.mod_lineinfo lives in MOD_INIT_RODATA which do_free_init() is
+ * about to release. Clear the pointer so concurrent stack-trace
+ * lookups stop dereferencing it; do_free_init()'s synchronize_rcu()
+ * then waits out any reader that already captured the old pointer.
+ * WRITE_ONCE pairs with the READ_ONCE inside module_init_lineinfo_data()
+ * so the compiler can't tear or reorder the revocation across the
+ * llist_add() that follows.
+ */
+ WRITE_ONCE(mod->init_lineinfo_data, NULL);
+ WRITE_ONCE(mod->init_lineinfo_data_size, 0);
#endif
/*
* We want to free module_init, but be aware that kallsyms may be
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 01a37ec872b90..acefa9195ca60 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -46,8 +46,50 @@ quiet_cmd_btf_ko = BTF [M] $@
$(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
+# Fold the table generation into the link command rather than running it as a
+# separate $(newer-prereqs) step: if_changed also relinks when only the command
+# line changes, e.g. when LDFLAGS_MODULE gains a different --build-id, and a
+# relinked module whose lineinfo step was skipped silently loses its table.
+#
+# The table describes the module's own section layout, so it is built from a
+# partial link that already has that layout and merged in the link that
+# produces the module. Two things are held back for that second link: --be8,
+# because it converts the object to a form the linker will not read again, and
+# --build-id, because the note has to cover the module that is shipped and LLD
+# drops one it merely read. Everything else belongs in the first link, where
+# the table is measured: KBUILD_LDFLAGS_MODULE names an object of its own on
+# PowerPC, arch/powerpc/lib/crtsavres.o, and text linked ahead of the module
+# would move the code the table describes away from the section anchor. The
+# table object goes first in the second link so that its empty .text
+# contributes no bytes and the anchor still resolves to the start of that
+# code. The second link takes no linker script, since the first one already
+# applied module.lds: run again, it finds alloc_tags renamed and leaves
+# __start_alloc_tags/__stop_alloc_tags around an empty range, and it emits
+# the BYTE(0) stub sections of arm and arm64 a second time. The table object
+# is deleted up front because a table left behind by an interrupted build
+# describes a layout this one need not have.
+ifdef CONFIG_KALLSYMS_LINEINFO_MODULES
+lineinfo_ldflags_all = $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE)
+lineinfo_ldflags_last = $(filter --be8 --build-id --build-id=%, \
+ $(lineinfo_ldflags_all))
+
+ cmd_ld_ko_o = \
+ rm -f $@.lineinfo_pre $@.lineinfo.o; \
+ $(LD) -r $(KBUILD_LDFLAGS) \
+ $(filter-out $(lineinfo_ldflags_last), \
+ $(lineinfo_ldflags_all)) \
+ -T $(objtree)/scripts/module.lds \
+ -o $@.lineinfo_pre $(filter %.o, $^); \
+ $(CONFIG_SHELL) $(srctree)/scripts/gen-mod-lineinfo.sh \
+ $@.lineinfo_pre $@.lineinfo.o; \
+ $(LD) -r $(KBUILD_LDFLAGS) $(lineinfo_ldflags_last) -o $@ \
+ $$(test -f $@.lineinfo.o && echo $@.lineinfo.o) \
+ $@.lineinfo_pre; \
+ rm -f $@.lineinfo_pre $@.lineinfo.o
+endif
+
# Re-generate module BTFs if either module's .ko or vmlinux changed
-%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE
+%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) $(and $(CONFIG_KALLSYMS_LINEINFO_MODULES),$(objtree)/scripts/gen_lineinfo $(srctree)/scripts/gen-mod-lineinfo.sh) FORCE
+$(call if_changed,ld_ko_o)
ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+$(if $(newer-prereqs),$(call cmd,btf_ko))
diff --git a/scripts/gen-mod-lineinfo.sh b/scripts/gen-mod-lineinfo.sh
new file mode 100755
index 0000000000000..4a4070b265f83
--- /dev/null
+++ b/scripts/gen-mod-lineinfo.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+#
+# gen-mod-lineinfo.sh - Build a kernel module's source line info object
+#
+# Reads DWARF from a module's partially linked image and emits an object
+# holding the module's .mod_lineinfo section. The section carries an ELF
+# relocation against the module's .text section symbol, so the caller has to
+# link this object into the module for the relocation to ride along to the
+# module loader. Modeled on scripts/gen-btf.sh.
+
+set -e
+
+if [ $# -ne 2 ]; then
+ echo "Usage: $0 <module-prelink.o> <output.o>" >&2
+ exit 1
+fi
+
+IN="$1"
+OUT="$2"
+
+cleanup() {
+ rm -f "${OUT}.S"
+}
+trap cleanup EXIT
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+ set -x
+ ;;
+esac
+
+# Generate assembly from DWARF -- if it fails (no DWARF), silently skip.
+# Leaving no output behind is how the caller is told there is no table.
+if ! ${objtree}/scripts/gen_lineinfo --module "${IN}" > "${OUT}.S"; then
+ exit 0
+fi
+
+# KBUILD_AFLAGS carries the target flags the generated .S has to agree with,
+# e.g. -m32 on a biarch host.
+${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_MODULE} \
+ -c -o "${OUT}" "${OUT}.S"
+
+exit 0
diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c
index 1b347d07b7ad5..124f9e30d768c 100644
--- a/scripts/gen_lineinfo.c
+++ b/scripts/gen_lineinfo.c
@@ -49,16 +49,79 @@ static bool verbose;
exit(1); \
} while (0)
+#include "../include/linux/mod_lineinfo.h"
+
+static int module_mode;
+
static unsigned int skipped_overflow;
+/* Target ELF traits, captured once in main() and reused at emit time. */
+static bool target_64bit;
+static bool target_le;
+
/*
- * vmlinux mode: end of the invariant .text region. Zero means "no cap"
- * (graceful fallback when _etext is absent on some build).
+ * Vmlinux mode only: address range of the *invariant* .text region.
+ * See find_text_end_addr() for why we cap on _etext. text_end_addr == 0
+ * means "no cap available; capture everything above text_addr" (v3
+ * behavior, used as graceful fallback if _etext is absent).
*/
static unsigned long long text_end_addr;
+/*
+ * In module mode we cover several text-like sections, split across two
+ * output blobs by lifecycle:
+ *
+ * .mod_lineinfo -- persistent code (.text, .exit.text); MOD_RODATA
+ * .init.mod_lineinfo -- init code (.init.text); freed with init memory
+ *
+ * In ET_REL .ko files .text/.init.text/.exit.text all have sh_addr == 0,
+ * so DWARF line addresses (which become sh_addr + addend after relocation)
+ * collide across sections. We disambiguate by giving each *present*
+ * covered section a unique synthetic "bias" — a u32 base address — and
+ * adding that bias to relocated values inside apply_debug_relocations().
+ * libdw then yields biased addresses that classify_address() can map back
+ * to a single section unambiguously. The bias is internal to gen_lineinfo
+ * and never leaks into the emitted blob.
+ */
+enum mod_lineinfo_blob {
+ BLOB_PERSISTENT,
+ BLOB_INIT,
+ NUM_BLOBS,
+};
+
+struct covered_section {
+ const char *name; /* ELF section name (e.g. ".text") */
+ enum mod_lineinfo_blob blob;
+ unsigned long long bias;/* synthetic base address (set in resolve_*) */
+ unsigned long long size;
+ bool present; /* found in this .ko */
+ unsigned int sec_index; /* ELF section header index, for reloc matching */
+ unsigned int n_entries; /* DWARF line entries collected for this section */
+};
+
+static struct covered_section all_sections[] = {
+ { .name = ".text", .blob = BLOB_PERSISTENT },
+ { .name = ".exit.text", .blob = BLOB_PERSISTENT },
+ { .name = ".init.text", .blob = BLOB_INIT },
+ { .name = ".noinstr.text", .blob = BLOB_PERSISTENT },
+};
+#define ALL_SECTIONS ARRAY_SIZE(all_sections)
+
+/*
+ * Executable sections present in the .ko but not covered by the blob
+ * (e.g. .static_call.text, .altinstr_replacement, or the per-function
+ * .text.* subsections parisc32 modules keep). They get synthetic biases
+ * from the same cursor as the covered sections so their DWARF sequences
+ * classify unambiguously and can be dropped, instead of aliasing into
+ * .text's [0, size) range.
+ */
+static struct covered_section *extra_sections;
+static unsigned int num_extra_sections;
+static unsigned long long skipped_uncovered;
+
struct line_entry {
- unsigned int offset; /* offset from _text */
+ unsigned int offset; /* offset from covered section's start */
+ unsigned int section_id;/* index into covered_sections[] (module mode only) */
unsigned int file_id;
unsigned int line;
unsigned int seq; /* line-program row order, breaks offset ties */
@@ -88,14 +151,15 @@ static HASHTABLE_DEFINE(file_hashtable, 1U << 13);
/* Monotonic row counter; see the seq tie-break in compare_entries(). */
static unsigned int next_seq;
-static void add_entry(unsigned int offset, unsigned int file_id,
- unsigned int line)
+static void add_entry(unsigned int offset, unsigned int section_id,
+ unsigned int file_id, unsigned int line)
{
if (num_entries >= entries_capacity) {
entries_capacity = entries_capacity ? entries_capacity * 2 : 65536;
entries = xrealloc(entries, entries_capacity * sizeof(*entries));
}
entries[num_entries].offset = offset;
+ entries[num_entries].section_id = section_id;
entries[num_entries].file_id = file_id;
entries[num_entries].line = line;
entries[num_entries].seq = next_seq++;
@@ -308,6 +372,20 @@ static const char *make_relative(const char *path, const char *comp_dir)
static char buf[PATH_MAX];
const char *p;
+ if (path[0] != '/') {
+ /*
+ * libdw doubles relative paths on ET_REL input
+ * (e.g. "a/b.c/a/b.c" -> "a/b.c"). Undo that before the
+ * path is joined to comp_dir, or the halves stop matching.
+ */
+ size_t len = strlen(path);
+ size_t mid = len / 2;
+
+ if (len > 1 && path[mid] == '/' &&
+ !memcmp(path, path + mid + 1, mid))
+ path += mid + 1;
+ }
+
if (path[0] == '/') {
if (snprintf(buf, sizeof(buf), "%s", path) >= (int)sizeof(buf))
return path;
@@ -357,6 +435,9 @@ static int compare_entries(const void *a, const void *b)
const struct line_entry *ea = a;
const struct line_entry *eb = b;
+ /* Group by section first so each per-section table is contiguous. */
+ if (ea->section_id != eb->section_id)
+ return ea->section_id < eb->section_id ? -1 : 1;
if (ea->offset != eb->offset)
return ea->offset < eb->offset ? -1 : 1;
/*
@@ -373,7 +454,8 @@ static int compare_entries(const void *a, const void *b)
/*
* Look up a vmlinux symbol by exact name and return its st_value, or
- * @fallback if absent. Aborts when @required and the symbol is missing.
+ * @fallback if the symbol is absent (lets callers gracefully skip
+ * optional bounds like _etext).
*/
static unsigned long long find_vmlinux_sym(Elf *elf, const char *name,
unsigned long long fallback,
@@ -419,23 +501,43 @@ static unsigned long long find_text_addr(Elf *elf)
}
/*
- * vmlinux is linked in multiple passes: gen_lineinfo runs against
- * .tmp_vmlinux1 (which carries an empty lineinfo stub), then real tables
- * are linked in for the final image. Sections placed AFTER .rodata
- * (.init.text, .exit.text, ...) shift forward as .rodata grows to hold
- * the real lineinfo blob, so DWARF addresses we'd capture for them in
- * pass 1 would be stale in the final kernel. Cap captured addresses at
- * _etext, the symbol that marks the end of .text — placed before .rodata
- * in every architecture's vmlinux.lds.S, so its addresses are invariant
- * across the relink. Returns 0 if _etext is absent (no cap; v3 behavior).
+ * Vmlinux is linked in multiple passes: gen_lineinfo runs against
+ * .tmp_vmlinux1 (which carries the empty lineinfo stub), and the resulting
+ * tables are then linked into the final vmlinux. Sections placed AFTER
+ * .rodata (.init.text, .exit.text, ...) shift forward as the real lineinfo
+ * tables replace the empty stub, so DWARF addresses we'd capture for them
+ * here are stale by the time the kernel runs.
+ *
+ * Cap the captured range at _etext, the symbol that marks the end of the
+ * .text section. .text is placed BEFORE .rodata in every architecture's
+ * vmlinux.lds.S, so its addresses are invariant across the relink.
+ * Returns 0 on architectures or builds that don't expose _etext, in which
+ * case the cap is disabled (preserving the v3 behavior — addresses past
+ * .text remain captured but may be off in stack traces).
*/
static unsigned long long find_text_end_addr(Elf *elf)
{
return find_vmlinux_sym(elf, "_etext", 0, false);
}
+/*
+ * Ordering shared by entries[] and sym_starts[]: section first, then
+ * offset. In module mode every offset is section-relative, so the two
+ * together are what identifies a location.
+ */
+static int compare_sec_off(unsigned int sa, unsigned int oa,
+ unsigned int sb, unsigned int ob)
+{
+ if (sa != sb)
+ return sa < sb ? -1 : 1;
+ if (oa != ob)
+ return oa < ob ? -1 : 1;
+ return 0;
+}
+
/* Sorted, duplicate-free extents of every function symbol. */
struct sym_start {
+ unsigned int section_id;
unsigned int offset;
unsigned int size;
};
@@ -455,56 +557,44 @@ static struct sym_start *text_starts;
static unsigned int num_text_starts;
static unsigned int text_starts_capacity;
-/* Offsets one past the end of each DWARF line-program sequence. */
-static unsigned int *seq_ends;
+/* Locations one past the end of each DWARF line-program sequence. */
+struct seq_end {
+ unsigned int section_id;
+ unsigned int offset;
+};
+
+static struct seq_end *seq_ends;
static unsigned int num_seq_ends;
static unsigned int seq_ends_capacity;
-static void append_offset(unsigned int **arr, unsigned int *count,
- unsigned int *capacity, unsigned int value)
+static void record_seq_end(unsigned int section_id, unsigned int offset)
{
- if (*count >= *capacity) {
- *capacity = *capacity ? *capacity * 2 : 16384;
- *arr = xrealloc(*arr, *capacity * sizeof(**arr));
+ if (num_seq_ends >= seq_ends_capacity) {
+ seq_ends_capacity = seq_ends_capacity ?
+ seq_ends_capacity * 2 : 16384;
+ seq_ends = xrealloc(seq_ends,
+ seq_ends_capacity * sizeof(*seq_ends));
}
- (*arr)[(*count)++] = value;
-}
-
-/*
- * Record the end of a line-program sequence. @addr is one past the last
- * covered byte, so the sequence's own coverage is tested using addr - 1.
- */
-static void record_seq_end(unsigned long long addr,
- unsigned long long text_addr)
-{
- unsigned long long raw;
-
- if (addr <= text_addr)
- return;
- if (text_end_addr && addr - 1 >= text_end_addr)
- return;
-
- raw = addr - text_addr;
- if (raw > UINT_MAX)
- return;
-
- append_offset(&seq_ends, &num_seq_ends, &seq_ends_capacity,
- (unsigned int)raw);
+ seq_ends[num_seq_ends].section_id = section_id;
+ seq_ends[num_seq_ends].offset = offset;
+ num_seq_ends++;
}
static int compare_sym_starts(const void *a, const void *b)
{
const struct sym_start *sa = a, *sb = b;
+ int ret = compare_sec_off(sa->section_id, sa->offset,
+ sb->section_id, sb->offset);
- if (sa->offset != sb->offset)
- return sa->offset < sb->offset ? -1 : 1;
+ if (ret)
+ return ret;
/* Larger extent first, so the dedup below keeps it. */
if (sa->size != sb->size)
return sa->size > sb->size ? -1 : 1;
return 0;
}
-/* Sort by offset, keeping only the widest symbol at each. */
+/* Sort by (section, offset), keeping only the widest symbol at each. */
static void sort_starts(struct sym_start *a, unsigned int *count)
{
unsigned int n = *count, j = 0;
@@ -514,7 +604,8 @@ static void sort_starts(struct sym_start *a, unsigned int *count)
qsort(a, n, sizeof(*a), compare_sym_starts);
for (unsigned int i = 1; i < n; i++) {
- if (a[i].offset == a[j].offset)
+ if (!compare_sec_off(a[i].section_id, a[i].offset,
+ a[j].section_id, a[j].offset))
continue;
if (++j != i)
a[j] = a[i];
@@ -528,7 +619,9 @@ static void sort_starts(struct sym_start *a, unsigned int *count)
* the kernel's symbol-boundary check rejects the preceding function's entry
* and the frame goes unannotated.
*/
-static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
+static void collect_symbol_starts(Elf *elf, unsigned long long text_addr,
+ struct covered_section *sections,
+ unsigned int num_sections)
{
Elf_Scn *scn = NULL;
GElf_Shdr shdr;
@@ -548,8 +641,9 @@ static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
nsyms = shdr.sh_size / shdr.sh_entsize;
for (size_t i = 0; i < nsyms; i++) {
- GElf_Sym sym;
+ unsigned int sec_id = 0;
unsigned long long raw;
+ GElf_Sym sym;
if (!gelf_getsym(data, i, &sym))
continue;
@@ -560,12 +654,32 @@ static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
default:
continue;
}
- if (sym.st_value < text_addr)
- continue;
- if (text_end_addr && sym.st_value >= text_end_addr)
- continue;
- raw = sym.st_value - text_addr;
+ if (module_mode) {
+ /*
+ * ET_REL: st_value is already relative to the
+ * symbol's own section, so only sections the
+ * blob covers are of interest.
+ */
+ for (sec_id = 0; sec_id < num_sections; sec_id++)
+ if (sections[sec_id].present &&
+ sections[sec_id].sec_index ==
+ sym.st_shndx)
+ break;
+ if (sec_id == num_sections)
+ continue;
+ if (sym.st_value >= sections[sec_id].size)
+ continue;
+ raw = sym.st_value;
+ } else {
+ if (sym.st_value < text_addr)
+ continue;
+ if (text_end_addr &&
+ sym.st_value >= text_end_addr)
+ continue;
+ raw = sym.st_value - text_addr;
+ }
+
if (raw > UINT_MAX)
continue;
@@ -583,6 +697,7 @@ static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
text_starts_capacity *
sizeof(*text_starts));
}
+ text_starts[num_text_starts].section_id = sec_id;
text_starts[num_text_starts].offset = (unsigned int)raw;
text_starts[num_text_starts].size =
sym.st_size > UINT_MAX ? UINT_MAX :
@@ -592,6 +707,7 @@ static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
if (GELF_ST_TYPE(sym.st_info) != STT_FUNC)
continue;
+ sym_starts[num_sym_starts].section_id = sec_id;
sym_starts[num_sym_starts].offset = (unsigned int)raw;
sym_starts[num_sym_starts].size =
sym.st_size > UINT_MAX ? UINT_MAX :
@@ -608,12 +724,737 @@ static void collect_symbol_starts(Elf *elf, unsigned long long text_addr)
* there cannot run past it.
*/
for (unsigned int i = 1; i < num_text_starts; i++) {
+ if (text_starts[i].section_id != text_starts[i - 1].section_id)
+ continue;
if (!text_starts[i - 1].size)
text_starts[i - 1].size = text_starts[i].offset -
text_starts[i - 1].offset;
}
}
+/*
+ * Populate @sections[].present/sec_index/size/bias. Sections that don't
+ * exist stay marked absent. Biases are assigned in array order: each
+ * present section gets a base equal to 1 GiB plus the running total of
+ * preceding present sections' sizes, with a gap of 1 to 16 bytes after
+ * each. This guarantees [bias, bias+size) ranges are pairwise disjoint
+ * and fit in u32 as long as the sizes and gaps add up to less than
+ * 3 GiB.
+ *
+ * Starting at 1 GiB keeps an address that no relocation biased out of
+ * every range, as long as it is an offset below 1 GiB into a section
+ * that gets no bias of its own, such as the code lkdtm places in
+ * .rodata. At 0 it would read as an offset into .text. The gap
+ * matters because a sequence can put its last row on its end address:
+ * when that is the end of a section, the row has to land outside every
+ * range rather than at offset 0 of the next section.
+ */
+static void resolve_covered_sections(Elf *elf,
+ struct covered_section *sections,
+ unsigned int num_sections)
+{
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+ size_t shstrndx;
+ unsigned long long cursor = 1ULL << 30;
+
+ if (elf_getshdrstrndx(elf, &shstrndx) != 0)
+ return;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ const char *name;
+
+ if (!gelf_getshdr(scn, &shdr))
+ continue;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ continue;
+ bool covered = false;
+
+ for (unsigned int i = 0; i < num_sections; i++) {
+ if (sections[i].present)
+ continue;
+ if (strcmp(name, sections[i].name))
+ continue;
+ if (shdr.sh_size > UINT_MAX) {
+ warn("section %s exceeds 4 GiB (size=%llu); skipping",
+ name, (unsigned long long)shdr.sh_size);
+ break;
+ }
+ sections[i].sec_index = elf_ndxscn(scn);
+ sections[i].size = shdr.sh_size;
+ sections[i].present = true;
+ covered = true;
+ break;
+ }
+
+ /*
+ * Track every other executable section too, so its DWARF
+ * sequences can be biased into their own range and dropped
+ * instead of polluting a covered section's table.
+ */
+ if (!covered &&
+ (shdr.sh_flags & SHF_EXECINSTR) && (shdr.sh_flags & SHF_ALLOC) &&
+ shdr.sh_size && shdr.sh_size <= UINT_MAX) {
+ struct covered_section *es;
+
+ extra_sections = xrealloc(extra_sections,
+ (num_extra_sections + 1) *
+ sizeof(*extra_sections));
+ es = &extra_sections[num_extra_sections++];
+ memset(es, 0, sizeof(*es));
+ es->name = name;
+ es->sec_index = elf_ndxscn(scn);
+ es->size = shdr.sh_size;
+ es->present = true;
+ }
+ }
+
+ /* Pack present sections into non-overlapping bias ranges. */
+ for (unsigned int i = 0; i < num_sections; i++) {
+ if (!sections[i].present)
+ continue;
+ sections[i].bias = cursor;
+ cursor += sections[i].size;
+ cursor = (cursor + 16) & ~15ULL; /* leave a gap */
+ }
+ for (unsigned int i = 0; i < num_extra_sections; i++) {
+ extra_sections[i].bias = cursor;
+ cursor += extra_sections[i].size;
+ cursor = (cursor + 16) & ~15ULL;
+ }
+}
+
+/* Look up a covered_section by ELF section header index. */
+static struct covered_section *section_by_index(struct covered_section *sections,
+ unsigned int num_sections,
+ unsigned int sec_index)
+{
+ for (unsigned int i = 0; i < num_sections; i++) {
+ if (sections[i].present && sections[i].sec_index == sec_index)
+ return §ions[i];
+ }
+ return NULL;
+}
+
+/*
+ * Apply .rela.debug_line relocations to a mutable copy of .debug_line data.
+ *
+ * elfutils libdw (through at least 0.194) does NOT apply relocations for
+ * ET_REL files when using dwarf_begin_elf(). The internal libdwfl layer
+ * does this via __libdwfl_relocate(), but that API is not public.
+ *
+ * For DWARF5, the .debug_line file name table uses DW_FORM_line_strp
+ * references into .debug_line_str. Without relocation, all these offsets
+ * resolve to 0 (or garbage), causing dwarf_linesrc()/dwarf_filesrc() to
+ * return wrong filenames (typically the comp_dir for every file).
+ *
+ * This function applies the relocations manually so that the patched
+ * .debug_line data can be fed to dwarf_begin_elf() and produce correct
+ * results.
+ *
+ * See elfutils bug https://sourceware.org/bugzilla/show_bug.cgi?id=31447
+ * A fix (dwelf_elf_apply_relocs) was proposed but not yet merged as of
+ * elfutils 0.194: https://sourceware.org/pipermail/elfutils-devel/2024q3/007388.html
+ */
+/*
+ * Determine the relocation type for a 32-bit absolute reference
+ * on the given architecture. Returns 0 if unknown.
+ */
+/*
+ * Constants for the newer architectures are missing from older host ELF
+ * headers -- glibc only gained the LoongArch definitions in 2.36 -- and
+ * gen_lineinfo is built whenever CONFIG_KALLSYMS_LINEINFO is set, on every
+ * architecture. Same guarded-definition pattern as scripts/mod/modpost.c.
+ */
+#ifndef EM_RISCV
+#define EM_RISCV 243
+#endif
+#ifndef R_RISCV_32
+#define R_RISCV_32 1
+#endif
+#ifndef R_RISCV_64
+#define R_RISCV_64 2
+#endif
+#ifndef R_RISCV_ADD8
+#define R_RISCV_ADD8 33
+#endif
+#ifndef R_RISCV_ADD16
+#define R_RISCV_ADD16 34
+#endif
+#ifndef R_RISCV_ADD32
+#define R_RISCV_ADD32 35
+#endif
+#ifndef R_RISCV_ADD64
+#define R_RISCV_ADD64 36
+#endif
+#ifndef R_RISCV_SUB8
+#define R_RISCV_SUB8 37
+#endif
+#ifndef R_RISCV_SUB16
+#define R_RISCV_SUB16 38
+#endif
+#ifndef R_RISCV_SUB32
+#define R_RISCV_SUB32 39
+#endif
+#ifndef R_RISCV_SUB64
+#define R_RISCV_SUB64 40
+#endif
+#ifndef EM_LOONGARCH
+#define EM_LOONGARCH 258
+#endif
+#ifndef R_LARCH_32
+#define R_LARCH_32 1
+#endif
+#ifndef R_LARCH_64
+#define R_LARCH_64 2
+#endif
+#ifndef R_LARCH_ADD8
+#define R_LARCH_ADD8 47
+#endif
+#ifndef R_LARCH_ADD16
+#define R_LARCH_ADD16 48
+#endif
+#ifndef R_LARCH_ADD32
+#define R_LARCH_ADD32 50
+#endif
+#ifndef R_LARCH_ADD64
+#define R_LARCH_ADD64 51
+#endif
+#ifndef R_LARCH_SUB8
+#define R_LARCH_SUB8 52
+#endif
+#ifndef R_LARCH_SUB16
+#define R_LARCH_SUB16 53
+#endif
+#ifndef R_LARCH_SUB32
+#define R_LARCH_SUB32 55
+#endif
+#ifndef R_LARCH_SUB64
+#define R_LARCH_SUB64 56
+#endif
+
+/*
+ * MIPS n64 stores r_info as a symbol index followed by three extra
+ * relocation bytes rather than one integer, so the generic GELF_R_*
+ * macros decode it as nonsense: a type-2 relocation against symbol 9
+ * reads back as type 9 against symbol 33554432. Mirrors
+ * get_rel_type_and_sym() in scripts/mod/modpost.c.
+ */
+static void decode_r_info(const GElf_Ehdr *ehdr, GElf_Xword r_info,
+ unsigned int *r_type, size_t *r_sym)
+{
+ const unsigned int endian_probe = 1;
+ bool host_le = *(const unsigned char *)&endian_probe;
+ bool target_le = ehdr->e_ident[EI_DATA] == ELFDATA2LSB;
+ unsigned char raw[sizeof(r_info)];
+
+ if (ehdr->e_machine != EM_MIPS ||
+ ehdr->e_ident[EI_CLASS] != ELFCLASS64) {
+ *r_type = GELF_R_TYPE(r_info);
+ *r_sym = GELF_R_SYM(r_info);
+ return;
+ }
+
+ /*
+ * libelf byte-swaps r_info as one 64-bit quantity when the object's
+ * endianness differs from ours; undo that to get the field layout
+ * back before picking it apart.
+ */
+ memcpy(raw, &r_info, sizeof(raw));
+ if (target_le != host_le) {
+ unsigned char tmp;
+
+ for (size_t i = 0; i < sizeof(raw) / 2; i++) {
+ tmp = raw[i];
+ raw[i] = raw[sizeof(raw) - 1 - i];
+ raw[sizeof(raw) - 1 - i] = tmp;
+ }
+ }
+
+ *r_type = raw[7];
+ if (target_le)
+ *r_sym = (size_t)raw[0] | (size_t)raw[1] << 8 |
+ (size_t)raw[2] << 16 | (size_t)raw[3] << 24;
+ else
+ *r_sym = (size_t)raw[3] | (size_t)raw[2] << 8 |
+ (size_t)raw[1] << 16 | (size_t)raw[0] << 24;
+}
+
+/*
+ * Some ABIs express a label difference, such as a line program's address
+ * advance, as a pair of arithmetic relocations rather than one absolute
+ * value: RISC-V and LoongArch both do, and clang emits them for
+ * .debug_line even with -mno-relax. Each applies S + A to the value
+ * already stored at the target. Returns the width in bytes, or 0 if
+ * @r_type is not an arithmetic relocation; *@is_sub says which direction.
+ */
+static size_t arith_reloc_width(unsigned int e_machine, unsigned int r_type,
+ bool *is_sub)
+{
+ static const struct {
+ unsigned int machine;
+ unsigned int add;
+ unsigned int sub;
+ size_t width;
+ } tbl[] = {
+ { EM_RISCV, R_RISCV_ADD8, R_RISCV_SUB8, 1 },
+ { EM_RISCV, R_RISCV_ADD16, R_RISCV_SUB16, 2 },
+ { EM_RISCV, R_RISCV_ADD32, R_RISCV_SUB32, 4 },
+ { EM_RISCV, R_RISCV_ADD64, R_RISCV_SUB64, 8 },
+ { EM_LOONGARCH, R_LARCH_ADD8, R_LARCH_SUB8, 1 },
+ { EM_LOONGARCH, R_LARCH_ADD16, R_LARCH_SUB16, 2 },
+ { EM_LOONGARCH, R_LARCH_ADD32, R_LARCH_SUB32, 4 },
+ { EM_LOONGARCH, R_LARCH_ADD64, R_LARCH_SUB64, 8 },
+ };
+
+ for (size_t i = 0; i < ARRAY_SIZE(tbl); i++) {
+ if (tbl[i].machine != e_machine)
+ continue;
+ if (r_type == tbl[i].add) {
+ *is_sub = false;
+ return tbl[i].width;
+ }
+ if (r_type == tbl[i].sub) {
+ *is_sub = true;
+ return tbl[i].width;
+ }
+ }
+ return 0;
+}
+
+/* Relocations we could not apply; the caller refuses to emit a table. */
+static unsigned int unhandled_relocs;
+
+static unsigned int r_type_abs32(unsigned int e_machine)
+{
+ switch (e_machine) {
+ case EM_X86_64: return R_X86_64_32;
+ case EM_386: return R_386_32;
+ case EM_AARCH64: return R_AARCH64_ABS32;
+ case EM_ARM: return R_ARM_ABS32;
+ case EM_RISCV: return R_RISCV_32;
+ case EM_S390: return R_390_32;
+ case EM_MIPS: return R_MIPS_32;
+ case EM_PPC64: return R_PPC64_ADDR32;
+ case EM_PPC: return R_PPC_ADDR32;
+ case EM_LOONGARCH: return R_LARCH_32;
+ case EM_PARISC: return R_PARISC_DIR32;
+ default: return 0;
+ }
+}
+
+/*
+ * Determine the relocation type for a 64-bit absolute reference
+ * on the given architecture. Returns 0 on 32-bit-only architectures
+ * (where DW_LNE_set_address fits in 32 bits and r_type_abs32 covers it).
+ */
+static unsigned int r_type_abs64(unsigned int e_machine)
+{
+ switch (e_machine) {
+ case EM_X86_64: return R_X86_64_64;
+ case EM_AARCH64: return R_AARCH64_ABS64;
+ case EM_RISCV: return R_RISCV_64;
+ case EM_S390: return R_390_64;
+ case EM_MIPS: return R_MIPS_64;
+ case EM_PPC64: return R_PPC64_ADDR64;
+ case EM_LOONGARCH: return R_LARCH_64;
+ case EM_PARISC: return R_PARISC_DIR64;
+ default: return 0;
+ }
+}
+
+/*
+ * Write a 4- or 8-byte unsigned integer in target byte order.
+ * Cross-builds (e.g. x86_64 host -> s390 module) need the patched
+ * .debug_line bytes laid out per the .ko's e_ident[EI_DATA], not the host's.
+ */
+static void elf_write_uint(unsigned char *dst, uint64_t value, size_t size,
+ bool little_endian)
+{
+ if (little_endian) {
+ for (size_t i = 0; i < size; i++)
+ dst[i] = (value >> (i * 8)) & 0xff;
+ } else {
+ for (size_t i = 0; i < size; i++)
+ dst[i] = (value >> ((size - 1 - i) * 8)) & 0xff;
+ }
+}
+
+/* Counterpart to elf_write_uint: read the implicit addend of an SHT_REL
+ * relocation, stored in the relocated field itself in target byte order.
+ */
+static uint64_t elf_read_uint(const unsigned char *src, size_t size,
+ bool little_endian)
+{
+ uint64_t value = 0;
+
+ if (little_endian) {
+ for (size_t i = 0; i < size; i++)
+ value |= (uint64_t)src[i] << (i * 8);
+ } else {
+ for (size_t i = 0; i < size; i++)
+ value |= (uint64_t)src[i] << ((size - 1 - i) * 8);
+ }
+ return value;
+}
+
+/*
+ * Apply one relocation to a debug section. Two reloc widths matter:
+ * abs32 - section-offset refs: DW_FORM_line_strp file-table entries into
+ * .debug_line_str, and DW_AT_stmt_list, DW_AT_abbrev_offset and
+ * DW_FORM_strp refs out of .debug_info
+ * abs64 - DW_LNE_set_address arguments (sequence start PCs)
+ * Without both, libdw sees zeros: it reports wrong filenames, collapses
+ * every sequence to address 0 (collision after dedup), or decodes every
+ * compile unit against the first one's abbrev table and line program.
+ *
+ * @has_addend distinguishes RELA records (explicit @addend) from REL
+ * records, whose addend is read from the relocated field itself.
+ * @bias folds in the covered section's synthetic bias for text-directed
+ * relocations; only .debug_line addresses reach classify_address().
+ */
+static void apply_one_debug_reloc(Elf_Data *dl_data, Elf_Data *sym_data,
+ bool target_le, const GElf_Ehdr *ehdr,
+ unsigned int abs32_type,
+ unsigned int abs64_type, GElf_Xword r_info,
+ GElf_Addr r_offset, GElf_Sxword addend,
+ bool has_addend, bool bias)
+{
+ GElf_Sym sym;
+ unsigned int r_type;
+ size_t r_sym;
+ bool is_abs64 = false;
+ bool is_sub = false;
+ size_t arith_width;
+ size_t width;
+ uint64_t value;
+
+ decode_r_info(ehdr, r_info, &r_type, &r_sym);
+ arith_width = arith_reloc_width(ehdr->e_machine, r_type, &is_sub);
+
+ if (abs32_type && r_type == abs32_type)
+ is_abs64 = false;
+ else if (abs64_type && r_type == abs64_type)
+ is_abs64 = true;
+ else if (arith_width && has_addend)
+ ; /* handled below */
+ else if (r_type) /* type 0 is R_*_NONE everywhere we support */
+ unhandled_relocs++;
+
+ if (!(abs32_type && r_type == abs32_type) &&
+ !(abs64_type && r_type == abs64_type) &&
+ !(arith_width && has_addend))
+ return;
+
+ if (!gelf_getsym(sym_data, r_sym, &sym))
+ return;
+
+ if (arith_width) {
+ uint64_t cur;
+
+ if (r_offset > dl_data->d_size ||
+ dl_data->d_size - r_offset < arith_width)
+ return;
+ /*
+ * A delta, not an address: the section bias would appear in
+ * the ADD and cancel in the matching SUB, so leave it out.
+ */
+ cur = elf_read_uint((unsigned char *)dl_data->d_buf + r_offset,
+ arith_width, target_le);
+ value = (uint64_t)(sym.st_value + addend);
+ cur = is_sub ? cur - value : cur + value;
+ elf_write_uint((unsigned char *)dl_data->d_buf + r_offset,
+ cur, arith_width, target_le);
+ return;
+ }
+
+ width = is_abs64 ? 8 : 4;
+
+ if (r_offset > dl_data->d_size || dl_data->d_size - r_offset < width)
+ return;
+
+ if (!has_addend)
+ addend = (GElf_Sxword)elf_read_uint(
+ (unsigned char *)dl_data->d_buf + r_offset,
+ width, target_le);
+
+ value = (uint64_t)(sym.st_value + addend);
+
+ /*
+ * If the relocation targets one of the tracked text sections, fold
+ * in that section's synthetic bias so the patched DWARF address
+ * lands in a unique numeric range. String-ref relocs
+ * (DW_FORM_line_strp into .debug_line_str) target a different
+ * section, so the symbol-based check correctly excludes them from
+ * biasing — for both abs64 (64-bit ELF) and abs32 (32-bit ELF,
+ * where DW_LNE_set_address is also 4 bytes wide).
+ */
+ if (module_mode && bias) {
+ struct covered_section *cs;
+
+ cs = section_by_index(all_sections, ALL_SECTIONS,
+ sym.st_shndx);
+ if (!cs)
+ cs = section_by_index(extra_sections,
+ num_extra_sections,
+ sym.st_shndx);
+ if (cs)
+ value += cs->bias;
+ }
+
+ if (!is_abs64)
+ value &= 0xffffffffULL;
+
+ elf_write_uint((unsigned char *)dl_data->d_buf + r_offset,
+ value, width, target_le);
+}
+
+/* Walk one .rela.<debug section> / .rel.<debug section> table, if present. */
+static void apply_debug_reloc_table(Elf_Scn *scn, bool is_rela,
+ Elf_Data *dl_data, Elf_Data *sym_data,
+ bool target_le, const GElf_Ehdr *ehdr,
+ unsigned int abs32_type,
+ unsigned int abs64_type, bool bias)
+{
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nrels, i;
+
+ if (!scn)
+ return;
+
+ data = elf_getdata(scn, NULL);
+ if (!data || !gelf_getshdr(scn, &shdr) || !shdr.sh_entsize)
+ return;
+
+ nrels = shdr.sh_size / shdr.sh_entsize;
+
+ for (i = 0; i < nrels; i++) {
+ if (is_rela) {
+ GElf_Rela rela;
+
+ if (!gelf_getrela(data, i, &rela))
+ continue;
+ apply_one_debug_reloc(dl_data, sym_data, target_le,
+ ehdr, abs32_type, abs64_type,
+ rela.r_info, rela.r_offset,
+ rela.r_addend, true, bias);
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ continue;
+ apply_one_debug_reloc(dl_data, sym_data, target_le,
+ ehdr, abs32_type, abs64_type,
+ rel.r_info, rel.r_offset,
+ 0, false, bias);
+ }
+ }
+}
+
+/*
+ * Debug sections libdw reads while decoding compile units and their line
+ * programs. In an ET_REL .ko every reference out of these sections is a
+ * relocation: a CU's DW_AT_abbrev_offset and DW_AT_stmt_list, DW_FORM_strp
+ * names, DW_FORM_line_strp file-table entries and DW_LNE_set_address PCs.
+ * libdw applies none of them, so without this pass every CU but the first
+ * -- whose offsets genuinely are zero -- decodes against the first CU's
+ * abbrev table and line program, and a module built from several objects
+ * gets line info for only the first one.
+ */
+static const struct {
+ const char *name;
+ bool bias;
+} debug_reloc_sections[] = {
+ { ".debug_line", true },
+ { ".debug_info", false },
+ { ".debug_str_offsets", false },
+ { ".debug_addr", false },
+ { ".debug_rnglists", false },
+ { ".debug_loclists", false },
+};
+
+static Elf_Scn *find_scn_by_name(Elf *elf, size_t shstrndx, const char *want)
+{
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ const char *name;
+
+ if (!gelf_getshdr(scn, &shdr))
+ continue;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (name && !strcmp(name, want))
+ return scn;
+ }
+ return NULL;
+}
+
+static Elf_Scn *find_symtab_scn(Elf *elf)
+{
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ if (gelf_getshdr(scn, &shdr) && shdr.sh_type == SHT_SYMTAB)
+ return scn;
+ }
+ return NULL;
+}
+
+static void apply_debug_relocations(Elf *elf)
+{
+ Elf_Scn *symtab_scn;
+ GElf_Ehdr ehdr;
+ GElf_Shdr shdr;
+ unsigned int abs32_type, abs64_type;
+ bool target_le;
+ size_t shstrndx;
+ Elf_Data *sym_data;
+
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ return;
+
+ abs32_type = r_type_abs32(ehdr.e_machine);
+ abs64_type = r_type_abs64(ehdr.e_machine);
+ if (!abs32_type && !abs64_type)
+ error("no known absolute relocation type for ELF machine %u; refusing to emit line info from unrelocated DWARF",
+ ehdr.e_machine);
+ target_le = (ehdr.e_ident[EI_DATA] == ELFDATA2LSB);
+
+ if (elf_getshdrstrndx(elf, &shstrndx) != 0)
+ return;
+
+ symtab_scn = find_symtab_scn(elf);
+ if (!symtab_scn)
+ return;
+ sym_data = elf_getdata(symtab_scn, NULL);
+ if (!sym_data)
+ return;
+
+ for (size_t i = 0; i < ARRAY_SIZE(debug_reloc_sections); i++) {
+ const char *name = debug_reloc_sections[i].name;
+ bool bias = debug_reloc_sections[i].bias;
+ char relname[64];
+ Elf_Scn *dbg_scn, *rela_scn, *rel_scn;
+ Elf_Data *dbg_data;
+
+ dbg_scn = find_scn_by_name(elf, shstrndx, name);
+ if (!dbg_scn)
+ continue;
+
+ /*
+ * CONFIG_DEBUG_INFO_COMPRESSED_* hands us SHF_COMPRESSED
+ * sections. elf_getdata() would return the compressed bytes
+ * while every relocation offset addresses the uncompressed
+ * contents, so patching without expanding first corrupts the
+ * stream and libdw then reads nothing. Expand in place;
+ * libdw sees the same expanded data afterwards.
+ */
+ if (gelf_getshdr(dbg_scn, &shdr) &&
+ (shdr.sh_flags & SHF_COMPRESSED) &&
+ elf_compress(dbg_scn, 0, 0) < 0)
+ error("cannot decompress %s: %s", name,
+ elf_errmsg(elf_errno()));
+
+ dbg_data = elf_getdata(dbg_scn, NULL);
+ if (!dbg_data || !dbg_data->d_buf)
+ continue;
+
+ snprintf(relname, sizeof(relname), ".rela%s", name);
+ rela_scn = find_scn_by_name(elf, shstrndx, relname);
+ snprintf(relname, sizeof(relname), ".rel%s", name);
+ rel_scn = find_scn_by_name(elf, shstrndx, relname);
+
+ /*
+ * RELA (64-bit ELF and most 32-bit targets) carries explicit
+ * addends; REL (i386, arm32, ...) stores the addend in the
+ * relocated field itself.
+ */
+ apply_debug_reloc_table(rela_scn, true, dbg_data, sym_data,
+ target_le, &ehdr, abs32_type,
+ abs64_type, bias);
+ apply_debug_reloc_table(rel_scn, false, dbg_data, sym_data,
+ target_le, &ehdr, abs32_type,
+ abs64_type, bias);
+
+ /*
+ * Anything left unapplied would shift every mapping after it.
+ * Refuse the object rather than emit line numbers that point
+ * at the wrong source lines.
+ */
+ if (unhandled_relocs)
+ error("%s: %u unsupported relocation(s); refusing to emit line info",
+ name, unhandled_relocs);
+ }
+}
+
+/*
+ * Decide which covered_section a (biased) DWARF address belongs to.
+ * apply_debug_relocations() has already added the section's bias to
+ * each line-program PC, so [bias, bias+size) ranges are pairwise disjoint
+ * and a simple linear scan picks the right bucket. Returns the index
+ * within @sections, or @num_sections if @addr falls outside every
+ * present range (caller skips the entry).
+ */
+static unsigned int classify_address(struct covered_section *sections,
+ unsigned int num_sections,
+ unsigned long long addr,
+ unsigned long long *out_offset)
+{
+ for (unsigned int i = 0; i < num_sections; i++) {
+ if (!sections[i].present)
+ continue;
+ if (addr < sections[i].bias)
+ continue;
+ if (addr >= sections[i].bias + sections[i].size)
+ continue;
+ *out_offset = addr - sections[i].bias;
+ return i;
+ }
+ return num_sections;
+}
+
+/*
+ * Classify the end of a line-program sequence. @addr is one past the last
+ * covered byte, so the section it belongs to is resolved from addr - 1.
+ */
+static void classify_seq_end(unsigned long long addr,
+ unsigned long long text_addr,
+ struct covered_section *sections,
+ unsigned int num_sections)
+{
+ unsigned long long raw;
+
+ if (!addr)
+ return;
+
+ if (module_mode) {
+ unsigned long long sec_off;
+ unsigned int sec_id;
+
+ sec_id = classify_address(sections, num_sections, addr - 1,
+ &sec_off);
+ if (sec_id == num_sections || sec_off + 1 > UINT_MAX)
+ return;
+ record_seq_end(sec_id, (unsigned int)sec_off + 1);
+ return;
+ }
+
+ if (addr <= text_addr)
+ return;
+ if (text_end_addr && addr - 1 >= text_end_addr)
+ return;
+
+ raw = addr - text_addr;
+ if (raw > UINT_MAX)
+ return;
+
+ record_seq_end(0, (unsigned int)raw);
+}
+
/*
* One flag per row read from the compile unit being processed: whether the
* row closed a line-program sequence. libdw returns rows sorted by address,
@@ -631,17 +1472,20 @@ static void cu_rows_reserve(size_t rows)
cu_row_seq_end_cap = rows;
}
-/* Size of the symbol starting exactly at @offset, 0 if none. */
-static unsigned int symbol_extent_at(unsigned int offset)
+/* Size of the symbol starting exactly at (@section, @offset), 0 if none. */
+static unsigned int symbol_extent_at(unsigned int section, unsigned int offset)
{
unsigned int low = 0, high = num_text_starts;
while (low < high) {
unsigned int mid = low + (high - low) / 2;
+ int cmp = compare_sec_off(text_starts[mid].section_id,
+ text_starts[mid].offset,
+ section, offset);
- if (text_starts[mid].offset < offset)
+ if (cmp < 0)
low = mid + 1;
- else if (text_starts[mid].offset > offset)
+ else if (cmp > 0)
high = mid;
else
return text_starts[mid].size;
@@ -667,13 +1511,16 @@ static bool starts_new_sequence(unsigned int group, unsigned int next,
unsigned long long end;
unsigned int size;
- size = symbol_extent_at(entries[group].offset);
+ size = symbol_extent_at(entries[group].section_id,
+ entries[group].offset);
if (!size)
return false;
end = (unsigned long long)entries[group].offset + size;
for (unsigned int t = next; t < num_entries; t++) {
+ if (entries[t].section_id != entries[group].section_id)
+ break;
if (entries[t].offset > end)
break;
if (entries[t].offset < end || cu_row_seq_end[t - start])
@@ -707,6 +1554,7 @@ static void resolve_cu_row_groups(unsigned int start)
bool have_normal = false, have_seq_end = false;
while (k < num_entries &&
+ entries[k].section_id == entries[i].section_id &&
entries[k].offset == entries[i].offset) {
if (cu_row_seq_end[k - start]) {
seq_end = k;
@@ -743,6 +1591,7 @@ static void resolve_cu_row_groups(unsigned int start)
* which is why synthesize_symbol_starts() asks for this.
*/
struct asm_span {
+ unsigned int section_id;
unsigned int lo;
unsigned int hi;
};
@@ -755,26 +1604,26 @@ static int compare_asm_spans(const void *a, const void *b)
{
const struct asm_span *sa = a, *sb = b;
- if (sa->lo != sb->lo)
- return sa->lo < sb->lo ? -1 : 1;
- return 0;
+ return compare_sec_off(sa->section_id, sa->lo, sb->section_id, sb->lo);
}
-/* True if an assembler unit describes @offset. */
-static bool in_asm_span(unsigned int offset)
+/* True if an assembler unit describes (@section, @offset). */
+static bool in_asm_span(unsigned int section, unsigned int offset)
{
unsigned int low = 0, high = num_asm_spans;
while (low < high) {
unsigned int mid = low + (high - low) / 2;
- if (asm_spans[mid].lo <= offset)
+ if (compare_sec_off(asm_spans[mid].section_id,
+ asm_spans[mid].lo, section, offset) <= 0)
low = mid + 1;
else
high = mid;
}
- return low && offset <= asm_spans[low - 1].hi;
+ return low && asm_spans[low - 1].section_id == section &&
+ offset <= asm_spans[low - 1].hi;
}
/*
@@ -789,19 +1638,22 @@ static void record_cu_asm_spans(unsigned int start, unsigned int first_seq_end)
unsigned int i = start;
while (i < num_entries) {
- unsigned int j = i, hi;
+ unsigned int sec = entries[i].section_id, j = i, hi;
- while (!cu_row_seq_end[j - start] && j + 1 < num_entries)
+ while (!cu_row_seq_end[j - start] && j + 1 < num_entries &&
+ entries[j + 1].section_id == sec)
j++;
hi = entries[j].offset;
if (!cu_row_seq_end[j - start]) {
for (unsigned int t = first_seq_end; t < num_seq_ends;
t++) {
- if (seq_ends[t] <= hi)
+ if (seq_ends[t].section_id != sec ||
+ seq_ends[t].offset <= hi)
continue;
- if (hi == entries[j].offset || seq_ends[t] < hi)
- hi = seq_ends[t];
+ if (hi == entries[j].offset ||
+ seq_ends[t].offset < hi)
+ hi = seq_ends[t].offset;
}
}
@@ -811,6 +1663,7 @@ static void record_cu_asm_spans(unsigned int start, unsigned int first_seq_end)
asm_spans = xrealloc(asm_spans, asm_spans_capacity *
sizeof(*asm_spans));
}
+ asm_spans[num_asm_spans].section_id = sec;
asm_spans[num_asm_spans].lo = entries[i].offset;
asm_spans[num_asm_spans].hi = hi;
num_asm_spans++;
@@ -818,7 +1671,9 @@ static void record_cu_asm_spans(unsigned int start, unsigned int first_seq_end)
}
}
-static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
+static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr,
+ struct covered_section *sections,
+ unsigned int num_sections)
{
Dwarf_Off off = 0, next_off;
size_t hdr_size;
@@ -850,7 +1705,8 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
Dwarf_Addr addr;
const char *src;
const char *rel;
- unsigned int file_id, loffset;
+ unsigned int file_id, loffset, sec_id;
+ unsigned long long sec_off;
bool endseq = false;
int lineno;
@@ -873,7 +1729,8 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
* stopped here.
*/
if (dwarf_lineendsequence(line, &endseq) == 0 && endseq) {
- record_seq_end(addr, text_addr);
+ classify_seq_end(addr, text_addr, sections,
+ num_sections);
lineno = 0;
} else if (dwarf_lineno(line, &lineno) != 0) {
continue;
@@ -893,31 +1750,59 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
if (!src && lineno)
continue;
- if (addr < text_addr)
- continue;
- /*
- * Skip addresses past _etext. Sections after .rodata
- * shift when the real lineinfo replaces the empty stub
- * during the multi-pass vmlinux link, so any address
- * we'd capture there would be stale by the time the
- * final kernel runs.
- *
- * An end_sequence row addresses the byte after
- * the code it closes, so judge it by its last
- * byte, as record_seq_end() does: a sequence
- * ending at _etext still needs its end row.
- */
- if (text_end_addr && addr - endseq >= text_end_addr)
- continue;
-
- {
- unsigned long long raw_offset = addr - text_addr;
+ if (module_mode) {
+ /*
+ * In ET_REL .ko files .text/.init.text/.exit.text
+ * all share sh_addr == 0; classify_address picks
+ * the right bucket from the explicit ranges we
+ * captured. An end_sequence row addresses the
+ * byte after the sequence, which for a sequence
+ * reaching the end of its section lies past it.
+ * Resolve those from the last byte they cover.
+ */
+ if (endseq && !addr)
+ continue;
+ sec_id = classify_address(sections, num_sections,
+ endseq ? addr - 1 : addr,
+ &sec_off);
+ if (sec_id == num_sections) {
+ skipped_uncovered++;
+ continue;
+ }
+ if (endseq)
+ sec_off++;
+ if (sec_off > UINT_MAX) {
+ skipped_overflow++;
+ continue;
+ }
+ loffset = (unsigned int)sec_off;
+ sections[sec_id].n_entries++;
+ } else {
+ unsigned long long raw_offset;
+ if (addr < text_addr)
+ continue;
+ /*
+ * Skip addresses past _etext. Sections after
+ * .rodata shift when the real lineinfo replaces
+ * the empty stub during the multi-pass vmlinux
+ * link, so any address we'd capture there would
+ * be stale by the time the final kernel runs.
+ *
+ * An end_sequence row addresses the byte after
+ * the code it closes, so judge it by its last
+ * byte, as classify_seq_end() does: a sequence
+ * ending at _etext still needs its end row.
+ */
+ if (text_end_addr && addr - endseq >= text_end_addr)
+ continue;
+ raw_offset = addr - text_addr;
if (raw_offset > UINT_MAX) {
skipped_overflow++;
continue;
}
loffset = (unsigned int)raw_offset;
+ sec_id = 0;
}
if (src) {
@@ -927,7 +1812,7 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
file_id = 0;
}
- add_entry(loffset, file_id, (unsigned int)lineno);
+ add_entry(loffset, sec_id, file_id, (unsigned int)lineno);
cu_row_seq_end[num_entries - 1 - cu_first_entry] = endseq;
}
@@ -980,13 +1865,22 @@ static void synthesize_symbol_starts(void)
qsort(asm_spans, num_asm_spans, sizeof(*asm_spans), compare_asm_spans);
for (unsigned int i = 0; i < num_sym_starts; i++) {
+ unsigned int sec = sym_starts[i].section_id;
unsigned int start = sym_starts[i].offset;
unsigned int end = start + sym_starts[i].size;
+ /*
+ * Both arrays are sorted by (section_id, offset), so one
+ * forward cursor tracks the last entry at or below the symbol.
+ */
while (cursor + 1 < base_entries &&
- entries[cursor + 1].offset <= start)
+ compare_sec_off(entries[cursor + 1].section_id,
+ entries[cursor + 1].offset,
+ sec, start) <= 0)
cursor++;
+ if (entries[cursor].section_id != sec)
+ continue; /* no entries in this section yet */
if (entries[cursor].offset > start)
continue; /* nothing covers this symbol */
if (entries[cursor].offset == start)
@@ -1003,12 +1897,14 @@ static void synthesize_symbol_starts(void)
* it came from. In assembly it is one macro expanding to a
* function per invocation, which the row does describe.
*/
- if (!in_asm_span(start) &&
+ if (!in_asm_span(sec, start) &&
(cursor + 1 >= base_entries ||
+ entries[cursor + 1].section_id != sec ||
entries[cursor + 1].offset >= end))
continue;
- add_entry(start, entries[cursor].file_id, entries[cursor].line);
+ add_entry(start, sec, entries[cursor].file_id,
+ entries[cursor].line);
}
}
@@ -1029,7 +1925,8 @@ static void merge_same_offset(void)
return;
for (i = 1; i < num_entries; i++) {
- if (entries[i].offset == entries[j].offset) {
+ if (!compare_sec_off(entries[i].section_id, entries[i].offset,
+ entries[j].section_id, entries[j].offset)) {
if (entries[i].line || !entries[j].line)
entries[j] = entries[i];
continue;
@@ -1040,15 +1937,21 @@ static void merge_same_offset(void)
num_entries = j + 1;
}
-static void deduplicate(void)
+static void deduplicate(struct covered_section *sections,
+ unsigned int num_sections)
{
unsigned int sym_cursor = 0;
unsigned int i, j;
if (num_entries < 2)
- return;
+ goto count;
- /* Sort by offset, then by line-program order. */
+ /*
+ * Sort by section_id, then offset, then line-program order. This
+ * groups each section's entries contiguously so the per-section
+ * emit can iterate a simple range, and ensures the binary search
+ * invariant (offsets ascending) holds within each section.
+ */
qsort(entries, num_entries, sizeof(*entries), compare_entries);
merge_same_offset();
@@ -1058,7 +1961,10 @@ static void deduplicate(void)
/*
* Drop an entry with the same file:line as the previous kept one: it
* is redundant for binary search, since any address between them
- * resolves to the earlier one.
+ * resolves to the earlier one. Only within one section, though: the
+ * same offset can legitimately appear in two sections (they all
+ * start at sh_addr 0 in ET_REL), and each section is its own
+ * binary-search domain.
*
* Entries sitting on a symbol start are exempt: they are the only
* thing standing between that symbol and the kernel's boundary check.
@@ -1067,11 +1973,24 @@ static void deduplicate(void)
for (i = 1; i < num_entries; i++) {
bool at_symbol_start;
+ if (entries[i].section_id != entries[j].section_id) {
+ j++;
+ if (j != i)
+ entries[j] = entries[i];
+ continue;
+ }
+
while (sym_cursor < num_sym_starts &&
- sym_starts[sym_cursor].offset < entries[i].offset)
+ compare_sec_off(sym_starts[sym_cursor].section_id,
+ sym_starts[sym_cursor].offset,
+ entries[i].section_id,
+ entries[i].offset) < 0)
sym_cursor++;
at_symbol_start = sym_cursor < num_sym_starts &&
- sym_starts[sym_cursor].offset == entries[i].offset;
+ !compare_sec_off(sym_starts[sym_cursor].section_id,
+ sym_starts[sym_cursor].offset,
+ entries[i].section_id,
+ entries[i].offset);
if (!at_symbol_start &&
entries[i].file_id == entries[j].file_id &&
@@ -1083,6 +2002,15 @@ static void deduplicate(void)
entries[j] = entries[i];
}
num_entries = j + 1;
+
+count:
+ /* Recompute per-section n_entries from the deduped array. */
+ if (sections) {
+ for (unsigned int k = 0; k < num_sections; k++)
+ sections[k].n_entries = 0;
+ for (i = 0; i < num_entries; i++)
+ sections[entries[i].section_id].n_entries++;
+ }
}
static void compute_file_offsets(void)
@@ -1180,6 +2108,206 @@ static void output_assembly(void)
printf("\n");
}
+/*
+ * Emit one per-section table in the simple flat-array layout:
+ *
+ * mod_lineinfo_header
+ * addrs[count] (u32, sorted)
+ * file_ids[count] (u16) + 2-byte pad if count is odd
+ * lines[count] (u32)
+ * file_offsets[] (u32)
+ * filenames[]
+ *
+ * @suffix uniquifies labels so multiple tables can coexist in one blob.
+ * Caller has sorted entries[] so this section's entries occupy [first,
+ * first + count).
+ */
+static void emit_section_table(unsigned int first, unsigned int count,
+ const char *suffix)
+{
+ /*
+ * Align before defining the label, not after: the descriptor stores
+ * table_offset as .Lhdr - .Lroot, and every field offset inside the
+ * header is measured from .Lhdr too. Emitting the label first binds
+ * it to the pre-padding address, so the kernel would read the header
+ * out of the padding bytes.
+ */
+ printf("\t.balign 4\n");
+ printf(".Lhdr%s:\n", suffix);
+ printf("\t.long %u\t\t/* num_entries */\n", count);
+ printf("\t.long %u\t\t/* num_files */\n", num_files);
+ printf("\t.long .Lfilenames_end%s - .Lfilenames%s\n\n", suffix, suffix);
+
+ /* addrs[] */
+ for (unsigned int i = 0; i < count; i++)
+ printf("\t.long 0x%x\n", entries[first + i].offset);
+
+ /* file_ids[] */
+ for (unsigned int i = 0; i < count; i++)
+ printf("\t.short %u\n", entries[first + i].file_id);
+ if (count & 1)
+ printf("\t.short 0\t\t/* pad to align lines[] */\n");
+
+ /* lines[] */
+ for (unsigned int i = 0; i < count; i++)
+ printf("\t.long %u\n", entries[first + i].line);
+
+ /* file_offsets[] */
+ printf("\t.balign 4\n");
+ for (unsigned int i = 0; i < num_files; i++)
+ printf("\t.long %u\n", files[i]->str_offset);
+
+ /* filenames[] */
+ printf(".Lfilenames%s:\n", suffix);
+ for (unsigned int i = 0; i < num_files; i++)
+ print_escaped_asciz(files[i]->name);
+ printf(".Lfilenames_end%s:\n", suffix);
+}
+
+/*
+ * Emit one mod_lineinfo_section descriptor. The "anchor" field is a
+ * relocation against the named ELF section symbol; the module loader
+ * resolves it on load to the runtime base of that section.
+ *
+ * On 64-bit ELF: 8-byte slot via .quad <name> (R_*_64 reloc).
+ * On 32-bit ELF: 4-byte reloc via .long <name>, plus 4 bytes of zero
+ * padding. The two halves are ordered to match target endianness so a
+ * naive u64 read on the kernel side recovers the relocated value.
+ */
+static void emit_section_descriptor(const char *section_name,
+ unsigned long long size,
+ const char *table_label,
+ const char *root_label)
+{
+ if (target_64bit) {
+ printf("\t.quad %s\t/* sections[].anchor (RELOC) */\n",
+ section_name);
+ } else if (target_le) {
+ printf("\t.long %s\t/* sections[].anchor low (RELOC) */\n",
+ section_name);
+ printf("\t.long 0\t\t/* sections[].anchor high pad */\n");
+ } else {
+ printf("\t.long 0\t\t/* sections[].anchor high pad */\n");
+ printf("\t.long %s\t/* sections[].anchor low (RELOC) */\n",
+ section_name);
+ }
+ printf("\t.long %llu\t/* sections[].size */\n", size);
+ printf("\t.long %s - %s\t/* sections[].table_offset */\n",
+ table_label, root_label);
+}
+
+/*
+ * Emit one .mod_lineinfo / .init.mod_lineinfo blob. Walks all_sections[]
+ * picking only entries that (a) belong to the requested blob and (b)
+ * actually produced at least one DWARF line entry — sections present in
+ * the .ko but without DWARF (e.g. compiler-generated stub thunks) are
+ * silently skipped. The caller-supplied entries[] is already sorted by
+ * section_id, so each section's entries are contiguous; we walk the
+ * master array in order to compute per-section starting indices.
+ */
+static void emit_blob(const char *output_section,
+ const char *blob_tag,
+ enum mod_lineinfo_blob blob)
+{
+ unsigned int active = 0;
+ unsigned int section_starts[ALL_SECTIONS];
+ unsigned int cursor = 0;
+
+ for (unsigned int i = 0; i < ALL_SECTIONS; i++) {
+ section_starts[i] = cursor;
+ cursor += all_sections[i].n_entries;
+ if (all_sections[i].blob == blob && all_sections[i].n_entries)
+ active++;
+ }
+
+ if (!active)
+ return;
+
+ printf("\t.section %s, \"a\"\n\n", output_section);
+
+ printf("\t.balign 8\n");
+ printf(".Lroot_%s:\n", blob_tag);
+ printf("\t.long %u\t\t/* num_sections */\n", active);
+ /* Pad to align the u64 anchor in sections[0] to 8 bytes. */
+ printf("\t.balign 8\n");
+
+ {
+ unsigned int slot = 0;
+ for (unsigned int i = 0; i < ALL_SECTIONS; i++) {
+ char table_label[64];
+ char root_label[64];
+
+ if (all_sections[i].blob != blob)
+ continue;
+ if (!all_sections[i].n_entries)
+ continue;
+ snprintf(table_label, sizeof(table_label),
+ ".Lhdr_%s_%u", blob_tag, slot);
+ snprintf(root_label, sizeof(root_label),
+ ".Lroot_%s", blob_tag);
+ emit_section_descriptor(all_sections[i].name,
+ all_sections[i].size,
+ table_label, root_label);
+ slot++;
+ }
+ }
+ printf("\n");
+
+ {
+ unsigned int slot = 0;
+
+ for (unsigned int i = 0; i < ALL_SECTIONS; i++) {
+ char suffix[64];
+
+ if (all_sections[i].blob != blob)
+ continue;
+ if (!all_sections[i].n_entries)
+ continue;
+ snprintf(suffix, sizeof(suffix), "_%s_%u",
+ blob_tag, slot);
+ emit_section_table(section_starts[i],
+ all_sections[i].n_entries,
+ suffix);
+ slot++;
+ }
+ }
+ printf("\n");
+}
+
+/*
+ * Declare each text-like section we plan to reference as an empty
+ * SHF_EXECINSTR section in this object. Without these stanzas the
+ * assembler treats `.quad .exit.text` as an undefined external symbol;
+ * after ld -r the resulting GLOBAL UND `.exit.text` doesn't bind to the
+ * .ko's LOCAL SECTION symbol of the same name, leaving depmod with an
+ * unresolved-symbol warning and the loader unable to relocate the anchor.
+ *
+ * Declaring the section here gives lineinfo.o its own local SECTION
+ * symbol; ld -r merges sections by name so the local symbol simply
+ * relocates to offset 0 of the merged section (lineinfo.o is linked
+ * FIRST so its zero-byte contribution stays at the start).
+ */
+static void declare_empty_text_sections(void)
+{
+ for (unsigned int i = 0; i < ALL_SECTIONS; i++) {
+ if (!all_sections[i].present)
+ continue;
+ printf("\t.section %s, \"ax\"\n", all_sections[i].name);
+ }
+ printf("\n");
+}
+
+static void output_module_assembly(void)
+{
+ printf("/* SPDX-License-Identifier: GPL-2.0 */\n");
+ printf("/*\n");
+ printf(" * Automatically generated by scripts/gen_lineinfo --module\n");
+ printf(" * Do not edit.\n");
+ printf(" */\n\n");
+
+ declare_empty_text_sections();
+}
+
int main(int argc, char *argv[])
{
const char *kbuild_verbose = getenv("KBUILD_VERBOSE");
@@ -1198,8 +2326,15 @@ int main(int argc, char *argv[])
argc--;
}
+ if (argc >= 2 && !strcmp(argv[1], "--module")) {
+ module_mode = 1;
+ argv++;
+ argc--;
+ }
+
if (argc != 2) {
- fprintf(stderr, "Usage: %s [-v] <vmlinux>\n", argv[0]);
+ fprintf(stderr, "Usage: %s [-v] [--module] <ELF file>\n",
+ argv[0]);
return 1;
}
@@ -1210,13 +2345,42 @@ int main(int argc, char *argv[])
error("cannot open %s: %s", argv[1], strerror(errno));
elf_version(EV_CURRENT);
- elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+ /*
+ * Module mode patches line-program addresses in the in-memory ELF
+ * data and must never write them back. A private copy-on-write
+ * mapping gives exactly that, so the .ko can stay open read-only.
+ */
+ elf = elf_begin(fd, module_mode ? ELF_C_READ_MMAP_PRIVATE :
+ ELF_C_READ_MMAP, NULL);
if (!elf)
error("elf_begin failed: %s", elf_errmsg(elf_errno()));
- text_addr = find_text_addr(elf);
- text_end_addr = find_text_end_addr(elf);
- collect_symbol_starts(elf, text_addr);
+ {
+ GElf_Ehdr ehdr;
+
+ if (gelf_getehdr(elf, &ehdr) == NULL)
+ error("gelf_getehdr failed");
+ target_64bit = (ehdr.e_ident[EI_CLASS] == ELFCLASS64);
+ target_le = (ehdr.e_ident[EI_DATA] == ELFDATA2LSB);
+ }
+
+ if (module_mode) {
+ /*
+ * .ko files are ET_REL after ld -r. Resolve covered text
+ * sections FIRST so apply_debug_relocations() can use the
+ * assigned biases when patching line-program addresses;
+ * libdw does NOT apply relocations for ET_REL files, so we
+ * patch every debug section it reads.
+ */
+ resolve_covered_sections(elf, all_sections, ALL_SECTIONS);
+ apply_debug_relocations(elf);
+ text_addr = 0; /* unused in module mode */
+ } else {
+ text_addr = find_text_addr(elf);
+ text_end_addr = find_text_end_addr(elf);
+ }
+
+ collect_symbol_starts(elf, text_addr, all_sections, ALL_SECTIONS);
dwarf = dwarf_begin_elf(elf, DWARF_C_READ, NULL);
if (!dwarf)
@@ -1224,18 +2388,59 @@ int main(int argc, char *argv[])
LINEINFO_PREFIX "error: is %s built with CONFIG_DEBUG_INFO?",
dwarf_errmsg(dwarf_errno()), argv[1]);
- process_dwarf(dwarf, text_addr);
+ if (module_mode) {
+ unsigned int persistent_total, init_total;
- if (skipped_overflow)
- warn("%u entries skipped (offset > 4 GiB from _text)",
- skipped_overflow);
+ output_module_assembly(); /* file header only */
- deduplicate();
- compute_file_offsets();
+ /*
+ * Single DWARF pass classifies every line entry into its
+ * covering section (or skips it). Each entry is tagged with
+ * the master-array section_id so per-blob emit can filter.
+ */
+ process_dwarf(dwarf, 0, all_sections, ALL_SECTIONS);
+ deduplicate(all_sections, ALL_SECTIONS);
+ compute_file_offsets();
+
+ emit_blob(".mod_lineinfo", "p", BLOB_PERSISTENT);
+ emit_blob(".init.mod_lineinfo", "i", BLOB_INIT);
+
+ persistent_total = 0;
+ init_total = 0;
+ for (unsigned int i = 0; i < ALL_SECTIONS; i++) {
+ if (all_sections[i].blob == BLOB_PERSISTENT)
+ persistent_total += all_sections[i].n_entries;
+ else if (all_sections[i].blob == BLOB_INIT)
+ init_total += all_sections[i].n_entries;
+ }
+ verbose_msg("persistent %u entries, init %u entries, %u files",
+ persistent_total, init_total, num_files);
+
+ /*
+ * Expected for every module: .static_call.text and friends
+ * are deliberately not covered.
+ */
+ if (skipped_uncovered)
+ verbose_msg("%llu entries dropped (outside covered text sections)",
+ skipped_uncovered);
+
+ if (skipped_overflow)
+ warn("%u entries skipped (offset > 4 GiB)",
+ skipped_overflow);
+ } else {
+ process_dwarf(dwarf, text_addr, NULL, 0);
+
+ if (skipped_overflow)
+ warn("%u entries skipped (offset > 4 GiB from _text)",
+ skipped_overflow);
+
+ deduplicate(NULL, 0);
+ compute_file_offsets();
- verbose_msg("%u entries, %u files", num_entries, num_files);
+ verbose_msg("%u entries, %u files", num_entries, num_files);
- output_assembly();
+ output_assembly();
+ }
dwarf_end(dwarf);
elf_end(elf);
@@ -1248,9 +2453,11 @@ int main(int argc, char *argv[])
free(cu_row_seq_end);
free(sym_starts);
free(seq_ends);
+ free(extra_sections);
for (unsigned int i = 0; i < num_files; i++)
free(files[i]);
free(files);
-
+ for (unsigned int i = 0; i < num_path_roots; i++)
+ free(path_roots[i].path);
return 0;
}
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v11 3/4] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction
2026-11-06 13:24 [PATCH v11 0/4] kallsyms: embed source file:line info in kernel stack traces Sasha Levin
2026-11-06 13:24 ` [PATCH v11 1/4] " Sasha Levin
2026-11-06 13:24 ` [PATCH v11 2/4] kallsyms: extend lineinfo to loadable modules Sasha Levin
@ 2026-11-06 13:24 ` Sasha Levin
2026-11-06 13:24 ` [PATCH v11 4/4] kallsyms: add KUnit tests for lineinfo feature Sasha Levin
3 siblings, 0 replies; 6+ messages in thread
From: Sasha Levin @ 2026-11-06 13:24 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Luis Chamberlain, Linus Torvalds,
Richard Weinberger, Juergen Gross, Geert Uytterhoeven,
James Bottomley
Cc: Sasha Levin, Jonathan Corbet, Nathan Chancellor, Nicolas Schier,
Petr Pavlu, Daniel Gomez, Aaron Tomlin, Greg KH, Petr Mladek,
Steven Rostedt, Kees Cook, Peter Zijlstra, Thorsten Leemhuis,
Vlastimil Babka, Helge Deller, Randy Dunlap, Laurent Pinchart,
Vivian Wang, Zhen Lei, Sami Tolvanen, linux-kernel, linux-kbuild,
linux-modules, linux-doc
Replace the flat uncompressed parallel arrays (lineinfo_addrs[],
lineinfo_file_ids[], lineinfo_lines[]) with a block-indexed,
delta-encoded, ULEB128 varint compressed format.
The sorted address array has small deltas between consecutive entries
(typically 1-50 bytes), file IDs have high locality (delta often 0,
same file), and line numbers change slowly. Delta-encoding followed
by ULEB128 varint compression shrinks most values from 4 bytes to 1.
Entries are grouped into blocks of 64. A small uncompressed block
index (first addr + byte offset per block) enables O(log(N/64)) binary
search, followed by sequential decode of at most 64 entries within the
matching block. All decode state lives on the stack -- zero
allocations, still safe for NMI/panic context.
Measured on x86_64_defconfig + CONFIG_DEBUG_INFO (same recipe as patch
1/4; 1,657,997 entries, 4,235 source files, 25,907 blocks):
Before (flat arrays):
lineinfo_addrs[] 6,631,988 bytes (u32 x 1.66M)
lineinfo_file_ids[] 3,315,994 bytes (u16 x 1.66M)
lineinfo_lines[] 6,631,988 bytes (u32 x 1.66M)
Total: 16,579,970 bytes (15.8 MiB, 10.00 bytes/entry)
After (block-indexed delta + ULEB128):
lineinfo_block_addrs[] 103,628 bytes (101 KiB)
lineinfo_block_offsets[] 103,628 bytes (101 KiB)
lineinfo_data[] 6,070,158 bytes (5.8 MiB)
Total: 6,277,414 bytes (6.0 MiB, 3.79 bytes/entry)
Savings: 9.8 MiB (2.64x reduction)
file_offsets[] and filenames[] are unchanged by this patch and are
excluded from both totals above.
Whole-image effect, measured the same way as patch 1/4:
vmlinux (stripped), no lineinfo: 52.2 MiB
vmlinux (stripped), before this fix: 68.2 MiB (+16.0 MiB / +30.6%)
vmlinux (stripped), after this fix: 60.2 MiB ( +8.0 MiB / +15.3%)
Booted in QEMU and verified with SysRq-l that annotations still work:
default_idle+0x9/0x10 (arch/x86/kernel/process.c:768)
default_idle_call+0x6e/0xb0 (kernel/sched/idle.c:122)
do_idle+0x38f/0x660 (kernel/sched/idle.c:199)
cpu_startup_entry+0x4e/0x60 (kernel/sched/idle.c:453)
rest_init+0x277/0x280 (init/main.c:732)
Suggested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
.../admin-guide/kallsyms-lineinfo.rst | 6 +-
include/linux/mod_lineinfo.h | 265 +++++++++++++++---
init/Kconfig | 8 +-
kernel/kallsyms.c | 60 ++--
kernel/kallsyms_internal.h | 8 +-
kernel/module/kallsyms.c | 127 ++++-----
scripts/empty_lineinfo.S | 20 +-
scripts/gen_lineinfo.c | 185 ++++++++----
8 files changed, 459 insertions(+), 220 deletions(-)
diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentation/admin-guide/kallsyms-lineinfo.rst
index 9b91adcf82ea3..0fc84a361b14e 100644
--- a/Documentation/admin-guide/kallsyms-lineinfo.rst
+++ b/Documentation/admin-guide/kallsyms-lineinfo.rst
@@ -78,10 +78,10 @@ Memory Overhead
The vmlinux lineinfo tables are stored in ``.rodata``. On an x86_64
``defconfig`` with ``CONFIG_DEBUG_INFO`` they hold 1.66 million entries and
-grow the stripped image by 16 MiB, about 10 bytes per entry after
-deduplication.
+grow the stripped image by 8 MiB, about 3.8 bytes per entry after delta
+compression.
-Per-module lineinfo adds about 10 bytes per entry to each ``.ko`` file, plus
+Per-module lineinfo adds about 4 bytes per entry to each ``.ko`` file, plus
a small fixed cost per covered section.
Known Limitations
diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h
index cb0c7af7b3171..2ede8448e4eed 100644
--- a/include/linux/mod_lineinfo.h
+++ b/include/linux/mod_lineinfo.h
@@ -3,9 +3,9 @@
* mod_lineinfo.h - Binary format for per-module source line information
*
* This header defines the layout of the .mod_lineinfo and
- * .init.mod_lineinfo sections embedded in loadable kernel modules. It
- * is dual-use: included from both the kernel and the userspace
- * gen_lineinfo tool.
+ * .init.mod_lineinfo sections embedded in loadable kernel modules. It is
+ * dual-use: included from both the kernel and the userspace gen_lineinfo
+ * tool.
*
* Top-level layout (all values in target-native endianness):
*
@@ -20,16 +20,27 @@
* If the relocation fails to resolve (e.g. unknown reloc type), .anchor
* stays zero and lookups silently degrade to "no annotation".
*
- * Each per-section sub-table is laid out as a stand-alone
- * mod_lineinfo_header followed by parallel arrays:
+ * Each per-section sub-table is laid out exactly as a stand-alone
+ * mod_lineinfo_header followed by its arrays:
*
- * struct mod_lineinfo_header (16 bytes)
- * u32 addrs[num_entries] -- offsets from this section's base, sorted
- * u16 file_ids[num_entries] -- parallel to addrs
- * <2-byte pad if num_entries is odd>
- * u32 lines[num_entries] -- parallel to addrs
+ * struct mod_lineinfo_header
+ * u32 block_addrs[num_blocks] -- first addr per block, for binary search
+ * u32 block_offsets[num_blocks] -- byte offset into compressed data stream
+ * u8 data[data_size] -- LEB128 delta-compressed entries
* u32 file_offsets[num_files] -- byte offset into filenames[]
* char filenames[filenames_size] -- concatenated NUL-terminated strings
+ *
+ * Each sub-array is located by an explicit (offset, size) pair in the
+ * header, similar to a flattened devicetree. All offsets in the per-section
+ * header are relative to that header itself, so a sub-table is fully
+ * self-describing.
+ *
+ * Compressed stream format (per block of LINEINFO_BLOCK_ENTRIES entries):
+ * Entry 0: file_id (ULEB128), line (ULEB128)
+ * addr is in block_addrs[]
+ * Entry 1..N: addr_delta (ULEB128),
+ * file_id_delta (SLEB128),
+ * line_delta (SLEB128)
*/
#ifndef _LINUX_MOD_LINEINFO_H
#define _LINUX_MOD_LINEINFO_H
@@ -44,6 +55,7 @@
#include <stdint.h>
typedef uint32_t u32;
typedef uint16_t u16;
+typedef uint8_t u8;
typedef uint64_t u64;
#ifndef __aligned
#define __aligned(x) __attribute__((__aligned__(x)))
@@ -53,6 +65,8 @@ typedef uint64_t u64;
#endif
#endif
+#define LINEINFO_BLOCK_ENTRIES 64
+
/*
* Per-section descriptor. One entry per ELF text section covered by the
* blob (.text, .exit.text, .init.text, ...).
@@ -84,47 +98,224 @@ static_assert(sizeof(struct mod_lineinfo_section) == 16,
struct mod_lineinfo_header {
u32 num_entries;
+ u32 num_blocks;
u32 num_files;
- u32 filenames_size; /* total bytes of concatenated filenames */
+ u32 blocks_offset; /* offset to block_addrs[] from this header */
+ u32 blocks_size; /* bytes: num_blocks * 2 * sizeof(u32) */
+ u32 data_offset; /* offset to compressed stream */
+ u32 data_size; /* bytes of compressed data */
+ u32 files_offset; /* offset to file_offsets[] */
+ u32 files_size; /* bytes: num_files * sizeof(u32) */
+ u32 filenames_offset;
+ u32 filenames_size;
};
-/* Offset helpers: compute byte offset from the per-section header to each array. */
-
-static inline u32 mod_lineinfo_addrs_off(void)
-{
- return sizeof(struct mod_lineinfo_header);
-}
+/*
+ * Descriptor for a lineinfo table, used by the shared lookup function.
+ * Callers populate this from either linker globals (vmlinux) or a
+ * validated mod_lineinfo_header (modules).
+ */
+struct lineinfo_table {
+ const u32 *blk_addrs;
+ const u32 *blk_offsets;
+ const u8 *data;
+ u32 data_size;
+ const u32 *file_offsets;
+ const char *filenames;
+ u32 num_entries;
+ u32 num_blocks;
+ u32 num_files;
+ u32 filenames_size;
+};
/*
- * The counts come from an on-disk blob and are only validated against the
- * blob size once the full layout has been summed, so every step widens to
- * u64: at 10 bytes per entry the u32 sums wrap for counts a caller can
- * name, which would let a malformed blob pass a bounds check computed from
- * the wrapped value.
+ * Read a ULEB128 varint from a byte stream.
+ * Returns the decoded value and advances *pos past the encoded bytes.
+ * Never reads at or beyond 'end': a value that 'end' cuts short yields
+ * the bits read so far, and one wider than 32 bits is truncated, so
+ * malformed data decodes to a wrong value rather than faulting (safe for
+ * NMI/panic context).
*/
-static inline u64 mod_lineinfo_file_ids_off(u32 num_entries)
+static inline u32 lineinfo_read_uleb128(const u8 *data, u32 *pos, u32 end)
{
- return mod_lineinfo_addrs_off() + (u64)num_entries * sizeof(u32);
-}
+ u32 result = 0;
+ unsigned int shift = 0;
-static inline u64 mod_lineinfo_lines_off(u32 num_entries)
-{
- /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */
- u64 off = mod_lineinfo_file_ids_off(num_entries) +
- (u64)num_entries * sizeof(u16);
- return (off + 3) & ~3ULL;
+ while (*pos < end) {
+ u8 byte = data[*pos];
+ (*pos)++;
+ result |= (u32)(byte & 0x7f) << shift;
+ if (!(byte & 0x80))
+ return result;
+ shift += 7;
+ if (shift >= 32) {
+ /* Malformed: skip remaining continuation bytes */
+ while (*pos < end && (data[*pos] & 0x80))
+ (*pos)++;
+ if (*pos < end)
+ (*pos)++;
+ return result;
+ }
+ }
+ return result;
}
-static inline u64 mod_lineinfo_file_offsets_off(u32 num_entries)
+/* Read an SLEB128 varint. Same safety guarantees as above. */
+static inline int32_t lineinfo_read_sleb128(const u8 *data, u32 *pos, u32 end)
{
- return mod_lineinfo_lines_off(num_entries) +
- (u64)num_entries * sizeof(u32);
+ int32_t result = 0;
+ unsigned int shift = 0;
+ u8 byte = 0;
+
+ while (*pos < end) {
+ byte = data[*pos];
+ (*pos)++;
+ result |= (int32_t)((u32)(byte & 0x7f) << shift);
+ shift += 7;
+ if (!(byte & 0x80))
+ break;
+ if (shift >= 32) {
+ while (*pos < end && (data[*pos] & 0x80))
+ (*pos)++;
+ if (*pos < end)
+ (*pos)++;
+ return result;
+ }
+ }
+
+ /* Sign-extend if the high bit of the last byte was set */
+ if (shift < 32 && (byte & 0x40))
+ result |= -(1 << shift);
+
+ return result;
}
-static inline u64 mod_lineinfo_filenames_off(u32 num_entries, u32 num_files)
+/*
+ * Search a lineinfo table for the source file and line corresponding to a
+ * given offset (from _text for vmlinux, from the covered section's base for
+ * modules). @min_offset is the containing symbol's start in the same offset
+ * space: entries below it belong to a preceding symbol and are rejected.
+ *
+ * Safe for NMI and panic context: no locks, no allocations, all state on stack.
+ * Returns true and sets @file and @line on success; false on any failure.
+ */
+static inline bool lineinfo_search(const struct lineinfo_table *tbl,
+ unsigned int offset,
+ unsigned int min_offset,
+ const char **file, unsigned int *line)
{
- return mod_lineinfo_file_offsets_off(num_entries) +
- (u64)num_files * sizeof(u32);
+ unsigned int low, high, mid, block;
+ unsigned int cur_addr, cur_file_id, cur_line;
+ unsigned int best_addr = 0, best_file_id = 0, best_line = 0;
+ unsigned int block_entries, data_end;
+ bool found = false;
+ u32 pos;
+
+ if (!tbl->num_entries || !tbl->num_blocks)
+ return false;
+
+ /* Binary search on blk_addrs[] to find the right block */
+ low = 0;
+ high = tbl->num_blocks;
+ while (low < high) {
+ mid = low + (high - low) / 2;
+ if (tbl->blk_addrs[mid] <= offset)
+ low = mid + 1;
+ else
+ high = mid;
+ }
+
+ if (low == 0)
+ return false;
+ block = low - 1;
+
+ /* How many entries in this block? */
+ block_entries = LINEINFO_BLOCK_ENTRIES;
+ if (block == tbl->num_blocks - 1) {
+ unsigned int remaining = tbl->num_entries -
+ block * LINEINFO_BLOCK_ENTRIES;
+
+ if (remaining < block_entries)
+ block_entries = remaining;
+ }
+
+ /* Determine end of this block's data in the compressed stream */
+ if (block + 1 < tbl->num_blocks)
+ data_end = tbl->blk_offsets[block + 1];
+ else
+ data_end = tbl->data_size;
+
+ /* Clamp data_end to actual data size */
+ if (data_end > tbl->data_size)
+ data_end = tbl->data_size;
+
+ /* Decode entry 0: addr from blk_addrs, file_id and line from stream */
+ pos = tbl->blk_offsets[block];
+ if (pos >= data_end)
+ return false;
+
+ cur_addr = tbl->blk_addrs[block];
+ cur_file_id = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+ cur_line = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+
+ /* Check entry 0 */
+ if (cur_addr <= offset) {
+ best_addr = cur_addr;
+ best_file_id = cur_file_id;
+ best_line = cur_line;
+ found = true;
+ }
+
+ /* Decode entries 1..N */
+ for (unsigned int i = 1; i < block_entries; i++) {
+ unsigned int addr_delta;
+ int32_t file_delta, line_delta;
+
+ addr_delta = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+ file_delta = lineinfo_read_sleb128(tbl->data, &pos, data_end);
+ line_delta = lineinfo_read_sleb128(tbl->data, &pos, data_end);
+
+ cur_addr += addr_delta;
+ cur_file_id = (unsigned int)((int32_t)cur_file_id + file_delta);
+ cur_line = (unsigned int)((int32_t)cur_line + line_delta);
+
+ if (cur_addr > offset)
+ break;
+
+ best_addr = cur_addr;
+ best_file_id = cur_file_id;
+ best_line = cur_line;
+ found = true;
+ }
+
+ if (!found)
+ return false;
+
+ /*
+ * The best entry is the closest one at or below @offset; reject it
+ * if it lies below the resolved symbol's start, so a symbol without
+ * line entries of its own does not inherit the preceding symbol's
+ * annotation.
+ */
+ if (best_addr < min_offset)
+ return false;
+
+ /*
+ * A zero line is the generator's "no source location applies here"
+ * marker, taken straight from a DWARF line-0 row.
+ */
+ if (!best_line)
+ return false;
+
+ if (best_file_id >= tbl->num_files)
+ return false;
+
+ if (tbl->file_offsets[best_file_id] >= tbl->filenames_size)
+ return false;
+
+ *file = &tbl->filenames[tbl->file_offsets[best_file_id]];
+ *line = best_line;
+ return true;
}
#endif /* _LINUX_MOD_LINEINFO_H */
diff --git a/init/Kconfig b/init/Kconfig
index 2a5a584b9e369..d5839a51442d1 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1954,8 +1954,9 @@ config KALLSYMS_LINEINFO
anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412)
This requires libelf and libdw (from elfutils) on the build host.
- Costs 10 bytes per DWARF line-table entry; for x86_64_defconfig
- with CONFIG_DEBUG_INFO that is about 16MB.
+ Costs about 3.8 bytes per DWARF line-table entry after delta
+ compression; for x86_64_defconfig with CONFIG_DEBUG_INFO that is
+ about 8MB.
If unsure, say N.
@@ -1968,7 +1969,8 @@ config KALLSYMS_LINEINFO_MODULES
so stack traces from module code include (file.c:123) annotations.
This requires libelf and libdw (from elfutils) on the build host.
- Costs 10 bytes per DWARF line entry in each .ko.
+ Costs about 3.8 bytes per DWARF line entry in each .ko after
+ delta compression.
If unsure, say N.
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index 77543ac216a92..395297e27af53 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -467,13 +467,17 @@ static int append_buildid(char *buffer, const char *modname,
#endif /* CONFIG_STACKTRACE_BUILD_ID */
+#include <linux/mod_lineinfo.h>
+
bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
const char **file, unsigned int *line)
{
unsigned long raw_offset, raw_min;
- unsigned int offset, min_offset = 0, low, high, mid, file_id;
+ unsigned int offset, min_offset = 0;
+ struct lineinfo_table tbl;
- if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || !lineinfo_num_entries)
+ if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) ||
+ !lineinfo_num_entries || !lineinfo_num_blocks)
return false;
/* Compute offset from _text */
@@ -491,8 +495,8 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
return false;
/*
- * The search below returns the closest entry at or below @offset, so
- * a symbol without line entries of its own (assembly without debug
+ * The search returns the closest entry at or below the offset, so a
+ * symbol without line entries of its own (assembly without debug
* info, or anything past the _etext cap like .init.text) would
* inherit the last entry of whatever precedes it. Bound the result
* to entries at or above the resolved symbol's start.
@@ -504,42 +508,18 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
min_offset = raw_min;
}
- /* Binary search for largest entry <= offset */
- low = 0;
- high = lineinfo_num_entries;
- while (low < high) {
- mid = low + (high - low) / 2;
- if (lineinfo_addrs[mid] <= offset)
- low = mid + 1;
- else
- high = mid;
- }
-
- if (low == 0)
- return false;
- low--;
-
- if (lineinfo_addrs[low] < min_offset)
- return false;
-
- /*
- * A zero line is the generator's "no source location applies here"
- * marker, taken straight from a DWARF line-0 row.
- */
- if (!lineinfo_lines[low])
- return false;
-
- file_id = lineinfo_file_ids[low];
- *line = lineinfo_lines[low];
-
- if (file_id >= lineinfo_num_files)
- return false;
-
- if (lineinfo_file_offsets[file_id] >= lineinfo_filenames_size)
- return false;
-
- *file = &lineinfo_filenames[lineinfo_file_offsets[file_id]];
- return true;
+ tbl.blk_addrs = lineinfo_block_addrs;
+ tbl.blk_offsets = lineinfo_block_offsets;
+ tbl.data = lineinfo_data;
+ tbl.data_size = lineinfo_data_size;
+ tbl.file_offsets = lineinfo_file_offsets;
+ tbl.filenames = lineinfo_filenames;
+ tbl.num_entries = lineinfo_num_entries;
+ tbl.num_blocks = lineinfo_num_blocks;
+ tbl.num_files = lineinfo_num_files;
+ tbl.filenames_size = lineinfo_filenames_size;
+
+ return lineinfo_search(&tbl, offset, min_offset, file, line);
}
/* Look up a kernel symbol and return it in a text buffer. */
diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h
index d7374ce444d81..ffe4c658067ec 100644
--- a/kernel/kallsyms_internal.h
+++ b/kernel/kallsyms_internal.h
@@ -16,10 +16,12 @@ extern const unsigned int kallsyms_markers[];
extern const u8 kallsyms_seqs_of_names[];
extern const u32 lineinfo_num_entries;
-extern const u32 lineinfo_addrs[];
-extern const u16 lineinfo_file_ids[];
-extern const u32 lineinfo_lines[];
extern const u32 lineinfo_num_files;
+extern const u32 lineinfo_num_blocks;
+extern const u32 lineinfo_block_addrs[];
+extern const u32 lineinfo_block_offsets[];
+extern const u32 lineinfo_data_size;
+extern const u8 lineinfo_data[];
extern const u32 lineinfo_file_offsets[];
extern const u32 lineinfo_filenames_size;
extern const char lineinfo_filenames[];
diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c
index a4eaf28fffe75..e1fd042a42d49 100644
--- a/kernel/module/kallsyms.c
+++ b/kernel/module/kallsyms.c
@@ -509,9 +509,9 @@ int module_kallsyms_on_each_symbol(const char *modname,
#include <linux/mod_lineinfo.h>
/*
- * Search one per-section sub-table for @section_offset using flat parallel
- * arrays. @hdr is the per-section header at byte offset @hdr_offset within
- * @blob. Returns true on hit and populates @file / @line.
+ * Search one per-section sub-table for @section_offset.
+ * @hdr is the per-section header at byte offset @hdr_offset within @blob.
+ * Returns true on hit and populates @file / @line.
*/
static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
u32 hdr_offset,
@@ -521,13 +521,8 @@ static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
unsigned int *line)
{
const struct mod_lineinfo_header *hdr;
- const u8 *base;
- const u32 *addrs, *lines, *file_offsets;
- const u16 *file_ids;
- const char *filenames;
- u32 num_entries, num_files, filenames_size;
- unsigned int low, high, mid;
- u16 file_id;
+ struct lineinfo_table tbl;
+ const void *base;
if (hdr_offset > blob_size ||
blob_size - hdr_offset < sizeof(*hdr))
@@ -543,84 +538,72 @@ static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
return false;
base = (const u8 *)blob + hdr_offset;
- hdr = (const struct mod_lineinfo_header *)base;
- num_entries = hdr->num_entries;
- num_files = hdr->num_files;
- filenames_size = hdr->filenames_size;
+ hdr = base;
- if (num_entries == 0)
+ if (hdr->num_entries == 0 || hdr->num_blocks == 0)
return false;
- /*
- * Check the whole layout against the blob in one go. The offset
- * helpers sum in u64 precisely because a malformed blob can name
- * counts whose u32 sum wraps: at 10 bytes per entry across addrs[],
- * file_ids[] and lines[], num_entries = 0x33333334 wraps to a small
- * value that any bounds check would happily accept.
- */
+ /* Validate each sub-array fits within the remaining blob bytes */
{
u32 avail = blob_size - hdr_offset;
- u64 needed = mod_lineinfo_filenames_off(num_entries, num_files);
- if (needed > avail || filenames_size > avail - needed)
+ if (hdr->blocks_offset > avail ||
+ hdr->blocks_size > avail - hdr->blocks_offset)
+ return false;
+ if (hdr->data_offset > avail ||
+ hdr->data_size > avail - hdr->data_offset)
+ return false;
+ if (hdr->files_offset > avail ||
+ hdr->files_size > avail - hdr->files_offset)
return false;
- }
-
- /*
- * Filenames are read as NUL-terminated C strings. Require the blob
- * to end in NUL so a malformed file_offsets entry can never lead the
- * later "%s" consumer past the end of the section.
- */
- if (filenames_size == 0 ||
- base[mod_lineinfo_filenames_off(num_entries, num_files) +
- filenames_size - 1] != 0)
- return false;
- addrs = (const u32 *)(base + mod_lineinfo_addrs_off());
- file_ids = (const u16 *)(base + mod_lineinfo_file_ids_off(num_entries));
- lines = (const u32 *)(base + mod_lineinfo_lines_off(num_entries));
- file_offsets = (const u32 *)(base + mod_lineinfo_file_offsets_off(num_entries));
- filenames = (const char *)(base + mod_lineinfo_filenames_off(num_entries, num_files));
-
- /* Binary search for largest entry <= section_offset. */
- low = 0;
- high = num_entries;
- while (low < high) {
- mid = low + (high - low) / 2;
- if (addrs[mid] <= section_offset)
- low = mid + 1;
- else
- high = mid;
+ /*
+ * block_addrs[], block_offsets[] and file_offsets[] are read
+ * as u32 arrays, so their offsets need the same alignment
+ * guarantee hdr_offset got above.
+ */
+ if (!IS_ALIGNED(hdr->blocks_offset, sizeof(u32)) ||
+ !IS_ALIGNED(hdr->files_offset, sizeof(u32)))
+ return false;
+ if (hdr->filenames_offset > avail ||
+ hdr->filenames_size > avail - hdr->filenames_offset)
+ return false;
}
- if (low == 0)
- return false;
- low--;
-
/*
- * Reject entries below the resolved symbol's start so a symbol
- * without line entries of its own does not inherit the preceding
- * symbol's annotation.
+ * Validate counts before multiplying by element size — multiplication
+ * could otherwise overflow on 32-bit builds with a malformed blob.
+ * num_blocks contributes (addr,offset) u32 pairs; num_files contributes
+ * one u32 each.
*/
- if (addrs[low] < min_offset)
+ if (hdr->num_blocks > hdr->blocks_size / (2 * sizeof(u32)))
+ return false;
+ if (hdr->num_files > hdr->files_size / sizeof(u32))
return false;
/*
- * A zero line is the generator's "no source location applies here"
- * marker, taken straight from a DWARF line-0 row.
+ * Filenames are read as NUL-terminated C strings. Require the blob
+ * to end in NUL so a malformed file_offsets entry can never lead the
+ * later "%s" consumer past the end of the section.
*/
- if (!lines[low])
- return false;
-
- file_id = file_ids[low];
- if (file_id >= num_files)
- return false;
- if (file_offsets[file_id] >= filenames_size)
+ if (hdr->filenames_size == 0 ||
+ ((const u8 *)base)[hdr->filenames_offset +
+ hdr->filenames_size - 1] != 0)
return false;
- *file = &filenames[file_offsets[file_id]];
- *line = lines[low];
- return true;
+ tbl.blk_addrs = base + hdr->blocks_offset;
+ tbl.blk_offsets = base + hdr->blocks_offset +
+ hdr->num_blocks * sizeof(u32);
+ tbl.data = base + hdr->data_offset;
+ tbl.data_size = hdr->data_size;
+ tbl.file_offsets = base + hdr->files_offset;
+ tbl.filenames = base + hdr->filenames_offset;
+ tbl.num_entries = hdr->num_entries;
+ tbl.num_blocks = hdr->num_blocks;
+ tbl.num_files = hdr->num_files;
+ tbl.filenames_size = hdr->filenames_size;
+
+ return lineinfo_search(&tbl, section_offset, min_offset, file, line);
}
/*
@@ -647,6 +630,7 @@ static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size,
if (root->num_sections == 0)
return false;
+ /* Validate sections[] array fits within the blob */
if (root->num_sections > U32_MAX / sizeof(struct mod_lineinfo_section))
return false;
sections_end = sizeof(*root) +
@@ -682,6 +666,9 @@ static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size,
/*
* Look up source file:line for an address within a loaded module.
+ * Uses the .mod_lineinfo / .init.mod_lineinfo sections embedded in the .ko
+ * at build time. Each section contains one or more per-section sub-tables
+ * keyed by an ELF-relocation-resolved anchor.
*
* Safe in NMI/panic context: no locks, no allocations.
* Caller must hold RCU read lock (or be in a context where the module
diff --git a/scripts/empty_lineinfo.S b/scripts/empty_lineinfo.S
index e058c41137123..edd5b1092f050 100644
--- a/scripts/empty_lineinfo.S
+++ b/scripts/empty_lineinfo.S
@@ -14,12 +14,20 @@ lineinfo_num_entries:
.balign 4
lineinfo_num_files:
.long 0
- .globl lineinfo_addrs
-lineinfo_addrs:
- .globl lineinfo_file_ids
-lineinfo_file_ids:
- .globl lineinfo_lines
-lineinfo_lines:
+ .globl lineinfo_num_blocks
+ .balign 4
+lineinfo_num_blocks:
+ .long 0
+ .globl lineinfo_block_addrs
+lineinfo_block_addrs:
+ .globl lineinfo_block_offsets
+lineinfo_block_offsets:
+ .globl lineinfo_data_size
+ .balign 4
+lineinfo_data_size:
+ .long 0
+ .globl lineinfo_data
+lineinfo_data:
.globl lineinfo_file_offsets
lineinfo_file_offsets:
.globl lineinfo_filenames_size
diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c
index 124f9e30d768c..38bacbd115381 100644
--- a/scripts/gen_lineinfo.c
+++ b/scripts/gen_lineinfo.c
@@ -2013,6 +2013,45 @@ static void deduplicate(struct covered_section *sections,
}
}
+/*
+ * Emit the LEB128 delta-compressed data stream for one block.
+ * @base is the absolute index of the first entry, @count is the number of
+ * entries in this block (<= LINEINFO_BLOCK_ENTRIES). Used by both vmlinux
+ * mode (one section, full entries[]) and module mode (per-section ranges).
+ */
+static void emit_block_data_range(unsigned int base, unsigned int count)
+{
+ if (!count)
+ return;
+
+ /* Entry 0: file_id, line (both unsigned) */
+ printf("\t.uleb128 %u\n", entries[base].file_id);
+ printf("\t.uleb128 %u\n", entries[base].line);
+
+ /* Entries 1..N: addr_delta (unsigned), file/line deltas (signed) */
+ for (unsigned int i = 1; i < count; i++) {
+ unsigned int idx = base + i;
+
+ printf("\t.uleb128 %u\n",
+ entries[idx].offset - entries[idx - 1].offset);
+ printf("\t.sleb128 %d\n",
+ (int)entries[idx].file_id - (int)entries[idx - 1].file_id);
+ printf("\t.sleb128 %d\n",
+ (int)entries[idx].line - (int)entries[idx - 1].line);
+ }
+}
+
+/* Vmlinux-mode wrapper: pick block index out of the global entries[]. */
+static void emit_block_data(unsigned int block)
+{
+ unsigned int base = block * LINEINFO_BLOCK_ENTRIES;
+ unsigned int count = num_entries - base;
+
+ if (count > LINEINFO_BLOCK_ENTRIES)
+ count = LINEINFO_BLOCK_ENTRIES;
+ emit_block_data_range(base, count);
+}
+
static void compute_file_offsets(void)
{
unsigned int offset = 0;
@@ -2036,6 +2075,11 @@ static void print_escaped_asciz(const char *s)
static void output_assembly(void)
{
+ unsigned int num_blocks;
+
+ num_blocks = num_entries ?
+ (num_entries + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0;
+
printf("/* SPDX-License-Identifier: GPL-2.0 */\n");
printf("/*\n");
printf(" * Automatically generated by scripts/gen_lineinfo\n");
@@ -2056,29 +2100,40 @@ static void output_assembly(void)
printf("lineinfo_num_files:\n");
printf("\t.long %u\n\n", num_files);
- /* Sorted address offsets from _text */
- printf("\t.globl lineinfo_addrs\n");
+ /* Number of blocks */
+ printf("\t.globl lineinfo_num_blocks\n");
printf("\t.balign 4\n");
- printf("lineinfo_addrs:\n");
- for (unsigned int i = 0; i < num_entries; i++)
- printf("\t.long 0x%x\n", entries[i].offset);
- printf("\n");
+ printf("lineinfo_num_blocks:\n");
+ printf("\t.long %u\n\n", num_blocks);
- /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */
- printf("\t.globl lineinfo_file_ids\n");
- printf("\t.balign 2\n");
- printf("lineinfo_file_ids:\n");
- for (unsigned int i = 0; i < num_entries; i++)
- printf("\t.short %u\n", entries[i].file_id);
- printf("\n");
+ /* Block first-addresses for binary search */
+ printf("\t.globl lineinfo_block_addrs\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_block_addrs:\n");
+ for (unsigned int i = 0; i < num_blocks; i++)
+ printf("\t.long 0x%x\n", entries[i * LINEINFO_BLOCK_ENTRIES].offset);
- /* Line numbers, parallel to addrs */
- printf("\t.globl lineinfo_lines\n");
+ /* Block byte offsets into compressed stream */
+ printf("\t.globl lineinfo_block_offsets\n");
printf("\t.balign 4\n");
- printf("lineinfo_lines:\n");
- for (unsigned int i = 0; i < num_entries; i++)
- printf("\t.long %u\n", entries[i].line);
- printf("\n");
+ printf("lineinfo_block_offsets:\n");
+ for (unsigned int i = 0; i < num_blocks; i++)
+ printf("\t.long .Lblock_%u - lineinfo_data\n", i);
+
+ /* Compressed data size */
+ printf("\t.globl lineinfo_data_size\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_data_size:\n");
+ printf("\t.long .Ldata_end - lineinfo_data\n\n");
+
+ /* Compressed data stream */
+ printf("\t.globl lineinfo_data\n");
+ printf("lineinfo_data:\n");
+ for (unsigned int i = 0; i < num_blocks; i++) {
+ printf(".Lblock_%u:\n", i);
+ emit_block_data(i);
+ }
+ printf(".Ldata_end:\n\n");
/* File string offset table */
printf("\t.globl lineinfo_file_offsets\n");
@@ -2086,45 +2141,39 @@ static void output_assembly(void)
printf("lineinfo_file_offsets:\n");
for (unsigned int i = 0; i < num_files; i++)
printf("\t.long %u\n", files[i]->str_offset);
- printf("\n");
/* Filenames size */
- {
- unsigned int fsize = 0;
-
- for (unsigned int i = 0; i < num_files; i++)
- fsize += strlen(files[i]->name) + 1;
- printf("\t.globl lineinfo_filenames_size\n");
- printf("\t.balign 4\n");
- printf("lineinfo_filenames_size:\n");
- printf("\t.long %u\n\n", fsize);
- }
+ printf("\t.globl lineinfo_filenames_size\n");
+ printf("\t.balign 4\n");
+ printf("lineinfo_filenames_size:\n");
+ printf("\t.long .Lfilenames_end - lineinfo_filenames\n\n");
/* Concatenated NUL-terminated filenames */
printf("\t.globl lineinfo_filenames\n");
printf("lineinfo_filenames:\n");
for (unsigned int i = 0; i < num_files; i++)
print_escaped_asciz(files[i]->name);
- printf("\n");
+ printf(".Lfilenames_end:\n");
}
/*
- * Emit one per-section table in the simple flat-array layout:
+ * Emit one per-section table. @suffix uniquifies the local labels so
+ * multiple tables can coexist in a single output blob; @blob_root_label
+ * is the symbol for the start of the enclosing blob (used for
+ * table_offset = .Lhdr - .Lroot).
*
- * mod_lineinfo_header
- * addrs[count] (u32, sorted)
- * file_ids[count] (u16) + 2-byte pad if count is odd
- * lines[count] (u32)
- * file_offsets[] (u32)
- * filenames[]
- *
- * @suffix uniquifies labels so multiple tables can coexist in one blob.
- * Caller has sorted entries[] so this section's entries occupy [first,
- * first + count).
+ * Caller has already sorted entries[] so this section's entries occupy
+ * the contiguous range [first, first + count). This function emits
+ * block-relative addresses computed from entries[first + N].offset.
*/
static void emit_section_table(unsigned int first, unsigned int count,
const char *suffix)
{
+ unsigned int num_blocks;
+
+ num_blocks = count ?
+ (count + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0;
+
/*
* Align before defining the label, not after: the descriptor stores
* table_offset as .Lhdr - .Lroot, and every field offset inside the
@@ -2135,29 +2184,45 @@ static void emit_section_table(unsigned int first, unsigned int count,
printf("\t.balign 4\n");
printf(".Lhdr%s:\n", suffix);
printf("\t.long %u\t\t/* num_entries */\n", count);
+ printf("\t.long %u\t\t/* num_blocks */\n", num_blocks);
printf("\t.long %u\t\t/* num_files */\n", num_files);
+ printf("\t.long .Lblk_addrs%s - .Lhdr%s\n", suffix, suffix);
+ printf("\t.long .Lblk_offsets_end%s - .Lblk_addrs%s\n", suffix, suffix);
+ printf("\t.long .Ldata%s - .Lhdr%s\n", suffix, suffix);
+ printf("\t.long .Ldata_end%s - .Ldata%s\n", suffix, suffix);
+ printf("\t.long .Lfile_offsets%s - .Lhdr%s\n", suffix, suffix);
+ printf("\t.long .Lfile_offsets_end%s - .Lfile_offsets%s\n", suffix, suffix);
+ printf("\t.long .Lfilenames%s - .Lhdr%s\n", suffix, suffix);
printf("\t.long .Lfilenames_end%s - .Lfilenames%s\n\n", suffix, suffix);
- /* addrs[] */
- for (unsigned int i = 0; i < count; i++)
- printf("\t.long 0x%x\n", entries[first + i].offset);
-
- /* file_ids[] */
- for (unsigned int i = 0; i < count; i++)
- printf("\t.short %u\n", entries[first + i].file_id);
- if (count & 1)
- printf("\t.short 0\t\t/* pad to align lines[] */\n");
-
- /* lines[] */
- for (unsigned int i = 0; i < count; i++)
- printf("\t.long %u\n", entries[first + i].line);
+ printf(".Lblk_addrs%s:\n", suffix);
+ for (unsigned int i = 0; i < num_blocks; i++)
+ printf("\t.long 0x%x\n",
+ entries[first + i * LINEINFO_BLOCK_ENTRIES].offset);
+
+ printf(".Lblk_offsets%s:\n", suffix);
+ for (unsigned int i = 0; i < num_blocks; i++)
+ printf("\t.long .Lblock%s_%u - .Ldata%s\n", suffix, i, suffix);
+ printf(".Lblk_offsets_end%s:\n\n", suffix);
+
+ printf(".Ldata%s:\n", suffix);
+ for (unsigned int i = 0; i < num_blocks; i++) {
+ unsigned int base = first + i * LINEINFO_BLOCK_ENTRIES;
+ unsigned int n = count - i * LINEINFO_BLOCK_ENTRIES;
+
+ if (n > LINEINFO_BLOCK_ENTRIES)
+ n = LINEINFO_BLOCK_ENTRIES;
+ printf(".Lblock%s_%u:\n", suffix, i);
+ emit_block_data_range(base, n);
+ }
+ printf(".Ldata_end%s:\n", suffix);
- /* file_offsets[] */
printf("\t.balign 4\n");
+ printf(".Lfile_offsets%s:\n", suffix);
for (unsigned int i = 0; i < num_files; i++)
printf("\t.long %u\n", files[i]->str_offset);
+ printf(".Lfile_offsets_end%s:\n\n", suffix);
- /* filenames[] */
printf(".Lfilenames%s:\n", suffix);
for (unsigned int i = 0; i < num_files; i++)
print_escaped_asciz(files[i]->name);
@@ -2437,7 +2502,11 @@ int main(int argc, char *argv[])
deduplicate(NULL, 0);
compute_file_offsets();
- verbose_msg("%u entries, %u files", num_entries, num_files);
+ verbose_msg("%u entries, %u files, %u blocks",
+ num_entries, num_files,
+ num_entries ?
+ (num_entries + LINEINFO_BLOCK_ENTRIES - 1) /
+ LINEINFO_BLOCK_ENTRIES : 0);
output_assembly();
}
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v11 4/4] kallsyms: add KUnit tests for lineinfo feature
2026-11-06 13:24 [PATCH v11 0/4] kallsyms: embed source file:line info in kernel stack traces Sasha Levin
` (2 preceding siblings ...)
2026-11-06 13:24 ` [PATCH v11 3/4] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction Sasha Levin
@ 2026-11-06 13:24 ` Sasha Levin
3 siblings, 0 replies; 6+ messages in thread
From: Sasha Levin @ 2026-11-06 13:24 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Luis Chamberlain, Linus Torvalds,
Richard Weinberger, Juergen Gross, Geert Uytterhoeven,
James Bottomley
Cc: Sasha Levin, Jonathan Corbet, Nathan Chancellor, Nicolas Schier,
Petr Pavlu, Daniel Gomez, Aaron Tomlin, Greg KH, Petr Mladek,
Steven Rostedt, Kees Cook, Peter Zijlstra, Thorsten Leemhuis,
Vlastimil Babka, Helge Deller, Randy Dunlap, Laurent Pinchart,
Vivian Wang, Zhen Lei, Sami Tolvanen, linux-kernel, linux-kbuild,
linux-modules, linux-doc
Add a KUnit test module (CONFIG_LINEINFO_KUNIT_TEST) that verifies the
kallsyms lineinfo feature produces correct source file:line annotations
in stack traces.
Export sprint_backtrace() and sprint_backtrace_build_id() with
EXPORT_SYMBOL_IF_KUNIT() so the test module can exercise the backtrace
APIs.
test_cross_file_inline() pins the row-selection rule: a header helper
is inlined into a caller in another file, and some address inside that
caller has to resolve to the header. Picking any row but the last at a
given address reports the call site instead, which no test that only
looks at its own file can notice.
The test module is built from two objects. A single-object module cannot
distinguish relocated per-compile-unit DWARF references from unrelocated
ones -- its only compile unit sits at offset 0, which is what an
unrelocated reference reads -- so test_second_object_lineinfo() resolves a
target in the second object to cover the general case.
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
kernel/kallsyms.c | 4 +
lib/Kconfig.debug | 10 +
lib/tests/Makefile | 4 +
lib/tests/lineinfo_kunit.c | 1087 +++++++++++++++++++++++++++++
lib/tests/lineinfo_kunit.h | 22 +
lib/tests/lineinfo_kunit_second.c | 21 +
6 files changed, 1148 insertions(+)
create mode 100644 lib/tests/lineinfo_kunit.c
create mode 100644 lib/tests/lineinfo_kunit.h
create mode 100644 lib/tests/lineinfo_kunit_second.c
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index 395297e27af53..a33b5fd619e65 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -32,6 +32,8 @@
#include <linux/bsearch.h>
#include <linux/btf_ids.h>
+#include <kunit/visibility.h>
+
#include "kallsyms_internal.h"
/*
@@ -661,6 +663,7 @@ int sprint_backtrace(char *buffer, unsigned long address)
{
return __sprint_symbol(buffer, address, -1, 1, 0, 1);
}
+EXPORT_SYMBOL_IF_KUNIT(sprint_backtrace);
/**
* sprint_backtrace_build_id - Look up a backtrace symbol and return it in a text buffer
@@ -681,6 +684,7 @@ int sprint_backtrace_build_id(char *buffer, unsigned long address)
{
return __sprint_symbol(buffer, address, -1, 1, 1, 1);
}
+EXPORT_SYMBOL_IF_KUNIT(sprint_backtrace_build_id);
/* To avoid using get_symbol_offset for every symbol, we carry prefix along. */
struct kallsym_iter {
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 2d22638ec13a6..8759182dbd9ea 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -3126,6 +3126,16 @@ config LONGEST_SYM_KUNIT_TEST
If unsure, say N.
+config LINEINFO_KUNIT_TEST
+ tristate "KUnit tests for kallsyms lineinfo" if !KUNIT_ALL_TESTS
+ depends on KUNIT && KALLSYMS_LINEINFO
+ default KUNIT_ALL_TESTS
+ help
+ KUnit tests for the kallsyms source line info feature.
+ Verifies that stack traces include correct (file.c:line) annotations.
+
+ If unsure, say N.
+
config HW_BREAKPOINT_KUNIT_TEST
bool "Test hw_breakpoint constraints accounting" if !KUNIT_ALL_TESTS
depends on HAVE_HW_BREAKPOINT
diff --git a/lib/tests/Makefile b/lib/tests/Makefile
index df0537e1b91fa..39c3742054708 100644
--- a/lib/tests/Makefile
+++ b/lib/tests/Makefile
@@ -39,6 +39,10 @@ obj-$(CONFIG_LIVEUPDATE_TEST) += liveupdate.o
CFLAGS_longest_symbol_kunit.o += $(call cc-disable-warning, missing-prototypes)
obj-$(CONFIG_LONGEST_SYM_KUNIT_TEST) += longest_symbol_kunit.o
+CFLAGS_lineinfo_kunit.o += $(call cc-option,-fno-inline-functions-called-once)
+obj-$(CONFIG_LINEINFO_KUNIT_TEST) += lineinfo_kunit_test.o
+lineinfo_kunit_test-y := lineinfo_kunit.o lineinfo_kunit_second.o
+
obj-$(CONFIG_MEMCPY_KUNIT_TEST) += memcpy_kunit.o
obj-$(CONFIG_MIN_HEAP_KUNIT_TEST) += min_heap_kunit.o
CFLAGS_overflow_kunit.o = $(call cc-disable-warning, tautological-constant-out-of-range-compare)
diff --git a/lib/tests/lineinfo_kunit.c b/lib/tests/lineinfo_kunit.c
new file mode 100644
index 0000000000000..fc1b97ffb5251
--- /dev/null
+++ b/lib/tests/lineinfo_kunit.c
@@ -0,0 +1,1087 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit tests for kallsyms lineinfo (CONFIG_KALLSYMS_LINEINFO).
+ *
+ * Copyright (c) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * Verifies that sprint_symbol() and related APIs append correct
+ * " (file.c:NNN)" annotations to kernel symbol lookups.
+ *
+ * Build with: CONFIG_LINEINFO_KUNIT_TEST=m (or =y)
+ * Run with: ./tools/testing/kunit/kunit.py run lineinfo
+ */
+
+#include <kunit/test.h>
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <asm/sections.h>
+#include <linux/smp.h>
+#include <linux/string.h>
+#include <linux/slab.h>
+#include <linux/mod_lineinfo.h>
+
+#include "lineinfo_kunit.h"
+
+/* --------------- helpers --------------- */
+
+static char *alloc_sym_buf(struct kunit *test)
+{
+ char *buf = kunit_kzalloc(test, KSYM_SYMBOL_LEN, GFP_KERNEL);
+
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+ return buf;
+}
+
+/*
+ * Function-pointer casts yield the descriptor address on architectures
+ * with function descriptors (e.g. ppc64 ELFv1, parisc64); resolve to the
+ * code address the lineinfo tables actually cover.
+ */
+static unsigned long target_addr(const void *fn)
+{
+ return (unsigned long)dereference_function_descriptor((void *)fn);
+}
+
+/*
+ * When the test is built as a module but CONFIG_KALLSYMS_LINEINFO_MODULES
+ * is disabled, the test module's own functions have no lineinfo tables, so
+ * every test that expects annotations on local targets must skip rather
+ * than fail.
+ */
+static void skip_without_module_lineinfo(struct kunit *test)
+{
+ if (IS_MODULE(CONFIG_LINEINFO_KUNIT_TEST) &&
+ !IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES))
+ kunit_skip(test,
+ "KALLSYMS_LINEINFO_MODULES disabled; test module has no lineinfo");
+}
+
+/*
+ * Format a symbol the way a stack trace prints it. Every variant that
+ * prints an offset appends the " (file:line)" annotation: sprint_symbol()
+ * behind %pS, sprint_symbol_build_id() behind %pSb and sprint_backtrace()
+ * behind %pB. Only sprint_symbol_no_offset(), behind %ps, leaves it off.
+ *
+ * sprint_backtrace() subtracts 1 from the address to handle tail-call
+ * return-address corrections; pass @addr + 1 to recover the original.
+ */
+static int sprint_with_lineinfo(char *buf, unsigned long addr)
+{
+ return sprint_backtrace(buf, addr + 1);
+}
+
+/*
+ * Whether two sprint_*() results name the same symbol. Compared on the
+ * output rather than against a spelled-out name, since the PowerPC ELFv1
+ * ABI gives code symbols a leading dot.
+ */
+static bool same_symbol(const char *a, const char *b)
+{
+ size_t len = strcspn(a, "+");
+
+ return len == strcspn(b, "+") && !strncmp(a, b, len);
+}
+
+/*
+ * Return true if @buf contains a lineinfo annotation matching
+ * the pattern " (<path>:<digits>)".
+ *
+ * The path may be a full path like "lib/tests/lineinfo_kunit.c" or
+ * a shortened form from module lineinfo (e.g., just a directory name).
+ */
+static bool has_lineinfo(const char *buf)
+{
+ const char *p, *colon, *end;
+
+ p = strstr(buf, " (");
+ if (!p)
+ return false;
+ p += 2; /* skip " (" */
+
+ colon = strchr(p, ':');
+ if (!colon || colon == p)
+ return false;
+
+ /* After colon: one or more digits then ')' */
+ end = colon + 1;
+ if (*end < '0' || *end > '9')
+ return false;
+ while (*end >= '0' && *end <= '9')
+ end++;
+ return *end == ')';
+}
+
+/*
+ * Extract line number from a lineinfo annotation.
+ * Returns 0 if not found.
+ */
+static unsigned int extract_line(const char *buf)
+{
+ const char *p, *colon;
+ unsigned int line = 0;
+
+ p = strstr(buf, " (");
+ if (!p)
+ return 0;
+
+ colon = strchr(p + 2, ':');
+ if (!colon)
+ return 0;
+
+ colon++;
+ while (*colon >= '0' && *colon <= '9') {
+ line = line * 10 + (*colon - '0');
+ colon++;
+ }
+ return line;
+}
+
+/*
+ * Check if the lineinfo annotation contains the given filename substring.
+ */
+static bool lineinfo_contains_file(const char *buf, const char *name)
+{
+ const char *p, *colon;
+
+ p = strstr(buf, " (");
+ if (!p)
+ return false;
+
+ colon = strchr(p + 2, ':');
+ if (!colon)
+ return false;
+
+ /* Search for @name between '(' and ':' */
+ return strnstr(p + 1, name, colon - p - 1) != NULL;
+}
+
+/* --------------- target functions --------------- */
+
+static noinline int lineinfo_target_normal(void)
+{
+ barrier();
+ return 42;
+}
+
+static noinline int lineinfo_target_short(void)
+{
+ barrier();
+ return 1;
+}
+
+static noinline int lineinfo_target_with_arg(int x)
+{
+ barrier();
+ return x + 1;
+}
+
+static noinline int lineinfo_target_many_lines(void)
+{
+ /*
+ * volatile forces a load/store per statement so the function keeps
+ * one line-table row per source line (and a body large enough for
+ * mid-function lookups) at any optimization level; without it the
+ * compiler folds the arithmetic to a single constant.
+ */
+ volatile int a = 0;
+
+ barrier();
+ a += 1;
+ a += 2;
+ a += 3;
+ a += 4;
+ a += 5;
+ a += 6;
+ a += 7;
+ a += 8;
+ a += 9;
+ a += 10;
+ barrier();
+ return a;
+}
+
+static __always_inline int lineinfo_inline_helper(void)
+{
+ return 99;
+}
+
+static noinline int lineinfo_inline_caller(void)
+{
+ barrier();
+ return lineinfo_inline_helper();
+}
+
+static noinline int lineinfo_cross_file_caller(int x)
+{
+ return lineinfo_header_helper(x);
+}
+
+/* 10-deep call chain */
+static noinline int lineinfo_chain_10(void) { barrier(); return 10; }
+static noinline int lineinfo_chain_9(void) { barrier(); return lineinfo_chain_10(); }
+static noinline int lineinfo_chain_8(void) { barrier(); return lineinfo_chain_9(); }
+static noinline int lineinfo_chain_7(void) { barrier(); return lineinfo_chain_8(); }
+static noinline int lineinfo_chain_6(void) { barrier(); return lineinfo_chain_7(); }
+static noinline int lineinfo_chain_5(void) { barrier(); return lineinfo_chain_6(); }
+static noinline int lineinfo_chain_4(void) { barrier(); return lineinfo_chain_5(); }
+static noinline int lineinfo_chain_3(void) { barrier(); return lineinfo_chain_4(); }
+static noinline int lineinfo_chain_2(void) { barrier(); return lineinfo_chain_3(); }
+static noinline int lineinfo_chain_1(void) { barrier(); return lineinfo_chain_2(); }
+
+/* --------------- Group A: Basic lineinfo presence --------------- */
+
+static void test_normal_function(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf, "lineinfo_kunit.c"),
+ "Wrong file in: %s", buf);
+}
+
+static void test_noinline_function(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_with_arg);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in: %s", buf);
+}
+
+static void test_inline_function(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_inline_caller);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo for inline caller in: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf, "lineinfo_kunit.c"),
+ "Wrong file in: %s", buf);
+}
+
+/*
+ * The inlined body has to win over the call site. Several line-program
+ * rows share the address where an inlined call begins, and all but the
+ * last cover zero bytes; picking any other row reports the caller's file,
+ * which is what addr2line would not do.
+ */
+static void test_cross_file_inline(struct kunit *test)
+{
+ char *first = alloc_sym_buf(test);
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_cross_file_caller);
+ bool found = false;
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(first, addr);
+
+ /*
+ * Walk the function while the symbol still resolves to it, rather
+ * than asking kallsyms for its size: kallsyms_lookup_size_offset()
+ * is not exported to modules.
+ */
+ for (unsigned long i = 0; i < 4096; i++) {
+ sprint_with_lineinfo(buf, addr + i);
+ if (!same_symbol(buf, first))
+ break;
+ if (lineinfo_contains_file(buf, "lineinfo_kunit.h")) {
+ found = true;
+ break;
+ }
+ }
+
+ KUNIT_EXPECT_TRUE_MSG(test, found,
+ "no address in lineinfo_cross_file_caller resolves to the inlined body's header");
+}
+
+static void test_short_function(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_short);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo for short function in: %s", buf);
+}
+
+static void test_many_lines_function(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_many_lines);
+ unsigned int line;
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in: %s", buf);
+ line = extract_line(buf);
+ KUNIT_EXPECT_GT_MSG(test, line, (unsigned int)0,
+ "Line number should be > 0 in: %s", buf);
+}
+
+/* --------------- Group B: Deep call chain --------------- */
+
+typedef int (*chain_fn_t)(void);
+
+static void test_deep_call_chain(struct kunit *test)
+{
+ static const chain_fn_t chain_fns[] = {
+ lineinfo_chain_1, lineinfo_chain_2,
+ lineinfo_chain_3, lineinfo_chain_4,
+ lineinfo_chain_5, lineinfo_chain_6,
+ lineinfo_chain_7, lineinfo_chain_8,
+ lineinfo_chain_9, lineinfo_chain_10,
+ };
+ char *buf = alloc_sym_buf(test);
+ int i, found = 0;
+
+ skip_without_module_lineinfo(test);
+
+ /* Call chain to prevent dead-code elimination */
+ KUNIT_ASSERT_EQ(test, lineinfo_chain_1(), 10);
+
+ for (i = 0; i < ARRAY_SIZE(chain_fns); i++) {
+ unsigned long addr = target_addr(chain_fns[i]);
+
+ sprint_with_lineinfo(buf, addr);
+ if (has_lineinfo(buf))
+ found++;
+ }
+
+ /*
+ * Not every tiny function gets DWARF line info (compiler may
+ * omit it for very small stubs), but at least some should.
+ */
+ KUNIT_EXPECT_GT_MSG(test, found, 0,
+ "None of the 10 chain functions had lineinfo");
+}
+
+/* --------------- Group C: sprint_symbol API variants --------------- */
+
+static void test_sprint_symbol_format(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ sprint_symbol(buf, addr);
+
+ /* Should contain +0x and /0x for offset/size */
+ KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "+0x"),
+ "Missing offset in: %s", buf);
+ KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "/0x"),
+ "Missing size in: %s", buf);
+ /*
+ * Only sprint_symbol_no_offset(), which backs lowercase %ps, stays
+ * bare; see test_sprint_symbol_no_offset().
+ */
+ skip_without_module_lineinfo(test);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "Missing lineinfo in sprint_symbol output: %s",
+ buf);
+}
+
+static void test_sprint_backtrace(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ /* sprint_backtrace subtracts 1 internally to handle tail calls */
+ sprint_backtrace(buf, addr + 1);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in backtrace: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf, "lineinfo_kunit.c"),
+ "Wrong file in backtrace: %s", buf);
+}
+
+static void test_sprint_backtrace_build_id(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_backtrace_build_id(buf, addr + 1);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in backtrace_build_id: %s", buf);
+}
+
+static void test_sprint_symbol_no_offset(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ sprint_symbol_no_offset(buf, addr);
+ /* No "+0x" in output */
+ KUNIT_EXPECT_NULL_MSG(test, strstr(buf, "+0x"),
+ "Unexpected offset in no_offset: %s", buf);
+ /*
+ * The one variant that stays bare: it backs lowercase %ps, which
+ * callers suffix with a literal "()".
+ */
+ KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf),
+ "Unexpected lineinfo in no_offset: %s", buf);
+}
+
+/* --------------- Group D: printk format specifiers --------------- */
+
+static void test_pS_format(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ void *addr = lineinfo_target_normal;
+
+ skip_without_module_lineinfo(test);
+
+ snprintf(buf, KSYM_SYMBOL_LEN, "%pS", addr);
+ /*
+ * %pS uses sprint_symbol(), which annotates: the generic
+ * stack_trace_print() prints every frame with %pS.
+ */
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "Missing lineinfo in %%pS: %s", buf);
+}
+
+static void test_pBb_format(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ /*
+ * %pBb uses sprint_backtrace_build_id which subtracts 1 from the
+ * address, so pass addr+1 to resolve back to the function.
+ */
+ void *addr = (void *)(target_addr(lineinfo_target_normal) + 1);
+
+ skip_without_module_lineinfo(test);
+
+ snprintf(buf, KSYM_SYMBOL_LEN, "%pBb", addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in %%pBb: %s", buf);
+}
+
+static void test_pSR_format(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ void *addr = lineinfo_target_normal;
+
+ skip_without_module_lineinfo(test);
+
+ snprintf(buf, KSYM_SYMBOL_LEN, "%pSR", addr);
+ /* %pSR is a sprint_symbol() variant; same rationale as %pS. */
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "Missing lineinfo in %%pSR: %s", buf);
+}
+
+/* --------------- Group E: Address edge cases --------------- */
+
+static void test_symbol_start_addr(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ /*
+ * sprint_with_lineinfo() passes the start plus one, which
+ * sprint_backtrace() looks up as the start itself but prints as
+ * "+0x1/". Verify the symbol resolves and carries lineinfo.
+ */
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ strnstr(buf, "lineinfo_target_normal",
+ KSYM_SYMBOL_LEN) != NULL,
+ "Didn't resolve to expected function: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo at function start: %s", buf);
+}
+
+static void test_symbol_nonzero_offset(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ /*
+ * sprint_backtrace subtracts 1 internally.
+ * Passing addr+2 resolves to addr+1 which is inside the function
+ * at a non-zero offset.
+ */
+ sprint_backtrace(buf, addr + 2);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ strnstr(buf, "lineinfo_target_normal",
+ KSYM_SYMBOL_LEN) != NULL,
+ "Didn't resolve to expected function: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo at non-zero offset: %s", buf);
+}
+
+static void test_unknown_address(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+
+ sprint_symbol(buf, 1UL);
+ /* Should be "0x1" with no lineinfo */
+ KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "0x1"),
+ "Expected hex address for bogus addr: %s", buf);
+ KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf),
+ "Unexpected lineinfo for bogus addr: %s", buf);
+}
+
+static void test_kernel_function_lineinfo(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(sprint_symbol);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo for sprint_symbol: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf, "kallsyms.c"),
+ "Expected kallsyms.c in: %s", buf);
+}
+
+static void test_init_text_no_stale_lineinfo(struct kunit *test)
+{
+#if IS_BUILTIN(CONFIG_LINEINFO_KUNIT_TEST)
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = (unsigned long)_sinittext + 16;
+
+ /*
+ * The table stops at _etext, so a lookup past it must not inherit
+ * the last covered entry. Not every layout puts .init.text there:
+ * UML links it inside [_text, _etext), where an annotation is the
+ * correct answer and proves nothing about the cap.
+ */
+ if (addr >= (unsigned long)_text && addr < (unsigned long)_etext)
+ kunit_skip(test, "init text is inside [_text, _etext) in this layout");
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf),
+ "Stale lineinfo for init-text address: %s", buf);
+#else
+ kunit_skip(test, "_sinittext not accessible from modules");
+#endif
+}
+
+/* --------------- Group F: Module path --------------- */
+
+static void test_second_object_lineinfo(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_second_file);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo in: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf,
+ "lineinfo_kunit_second.c"),
+ "Wrong file in: %s", buf);
+}
+
+static void test_module_function_lineinfo(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ if (!IS_MODULE(CONFIG_LINEINFO_KUNIT_TEST)) {
+ kunit_skip(test, "Test only meaningful when built as module");
+ return;
+ }
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_EXPECT_NOT_NULL_MSG(test,
+ strstr(buf, "[lineinfo_kunit"),
+ "Missing module name in: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "No lineinfo for module function: %s", buf);
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_contains_file(buf, "lineinfo_kunit.c"),
+ "Wrong file for module function: %s", buf);
+}
+
+/* --------------- Group G: Stress --------------- */
+
+struct lineinfo_stress_data {
+ unsigned long addr;
+ atomic_t failures;
+};
+
+static void lineinfo_stress_fn(void *info)
+{
+ struct lineinfo_stress_data *data = info;
+ char buf[KSYM_SYMBOL_LEN];
+ int i;
+
+ for (i = 0; i < 100; i++) {
+ sprint_with_lineinfo(buf, data->addr);
+ if (!has_lineinfo(buf))
+ atomic_inc(&data->failures);
+ }
+}
+
+static void test_concurrent_sprint_symbol(struct kunit *test)
+{
+ struct lineinfo_stress_data data;
+
+ skip_without_module_lineinfo(test);
+
+ data.addr = target_addr(lineinfo_target_normal);
+ atomic_set(&data.failures, 0);
+
+ on_each_cpu(lineinfo_stress_fn, &data, 1);
+
+ KUNIT_EXPECT_EQ_MSG(test, atomic_read(&data.failures), 0,
+ "Concurrent lineinfo failures detected");
+}
+
+static void test_rapid_sprint_symbol(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+ int i, failures = 0;
+
+ skip_without_module_lineinfo(test);
+
+ for (i = 0; i < 1000; i++) {
+ sprint_with_lineinfo(buf, addr);
+ if (!has_lineinfo(buf))
+ failures++;
+ }
+
+ KUNIT_EXPECT_EQ_MSG(test, failures, 0,
+ "Rapid sprint_symbol failures: %d/1000", failures);
+}
+
+/* --------------- Group H: Safety and plausibility --------------- */
+
+static void test_line_number_plausible(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+ unsigned int line;
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_ASSERT_TRUE(test, has_lineinfo(buf));
+
+ line = extract_line(buf);
+ KUNIT_EXPECT_GT_MSG(test, line, (unsigned int)0,
+ "Line number should be > 0");
+ KUNIT_EXPECT_LT_MSG(test, line, (unsigned int)10000,
+ "Line number %u implausibly large for this file",
+ line);
+}
+
+static void test_buffer_no_overflow(struct kunit *test)
+{
+ const size_t canary_size = 16;
+ char *buf;
+ int i;
+
+ buf = kunit_kzalloc(test, KSYM_SYMBOL_LEN + canary_size, GFP_KERNEL);
+ KUNIT_ASSERT_NOT_NULL(test, buf);
+
+ /* Fill canary area past KSYM_SYMBOL_LEN with 0xAA */
+ memset(buf + KSYM_SYMBOL_LEN, 0xAA, canary_size);
+
+ sprint_with_lineinfo(buf, target_addr(lineinfo_target_normal));
+
+ /* Verify canary bytes are untouched */
+ for (i = 0; i < canary_size; i++) {
+ KUNIT_EXPECT_EQ_MSG(test,
+ (unsigned char)buf[KSYM_SYMBOL_LEN + i],
+ (unsigned char)0xAA,
+ "Buffer overflow at offset %d past KSYM_SYMBOL_LEN",
+ i);
+ }
+}
+
+static void test_dump_stack_no_crash(struct kunit *test)
+{
+ /* Just verify dump_stack() completes without panic */
+ dump_stack();
+ KUNIT_SUCCEED(test);
+}
+
+static void test_sprint_symbol_build_id(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+
+ skip_without_module_lineinfo(test);
+
+ sprint_symbol_build_id(buf, addr);
+ /*
+ * Annotated: this backs %pSb, which is how arm64's
+ * dump_backtrace_entry() prints every frame.
+ */
+ KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf),
+ "Missing lineinfo in sprint_symbol_build_id: %s",
+ buf);
+}
+
+static void test_sleb128_edge_cases(struct kunit *test)
+{
+ u32 pos;
+ int32_t result;
+
+ /* Value 0: single byte 0x00 */
+ {
+ static const u8 data[] = { 0x00 };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)0);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value -1: single byte 0x7F */
+ {
+ static const u8 data[] = { 0x7f };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)-1);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value 1: single byte 0x01 */
+ {
+ static const u8 data[] = { 0x01 };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)1);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value -64: single byte 0x40 */
+ {
+ static const u8 data[] = { 0x40 };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)-64);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value 63: single byte 0x3F */
+ {
+ static const u8 data[] = { 0x3f };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)63);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value -128: two bytes 0x80 0x7F */
+ {
+ static const u8 data[] = { 0x80, 0x7f };
+
+ pos = 0;
+ result = lineinfo_read_sleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)-128);
+ KUNIT_EXPECT_EQ(test, pos, (u32)2);
+ }
+
+ /* Value INT32_MIN: five bytes, high bits land at shift 28 */
+ {
+ static const u8 data[] = { 0x80, 0x80, 0x80, 0x80, 0x78 };
+ const u8 *p = data;
+
+ /* Defeat constant folding so the runtime decoder runs. */
+ OPTIMIZER_HIDE_VAR(p);
+ pos = 0;
+ result = lineinfo_read_sleb128(p, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)INT_MIN);
+ KUNIT_EXPECT_EQ(test, pos, (u32)5);
+ }
+
+ /* Value INT32_MAX: five bytes */
+ {
+ static const u8 data[] = { 0xff, 0xff, 0xff, 0xff, 0x07 };
+ const u8 *p = data;
+
+ OPTIMIZER_HIDE_VAR(p);
+ pos = 0;
+ result = lineinfo_read_sleb128(p, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (int32_t)INT_MAX);
+ KUNIT_EXPECT_EQ(test, pos, (u32)5);
+ }
+}
+
+static void test_uleb128_edge_cases(struct kunit *test)
+{
+ u32 pos, result;
+
+ /* Value 0: single byte 0x00 */
+ {
+ static const u8 data[] = { 0x00 };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (u32)0);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value 127: single byte 0x7F */
+ {
+ static const u8 data[] = { 0x7F };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (u32)127);
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+
+ /* Value 128: two bytes 0x80 0x01 */
+ {
+ static const u8 data[] = { 0x80, 0x01 };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (u32)128);
+ KUNIT_EXPECT_EQ(test, pos, (u32)2);
+ }
+
+ /* Max u32 0xFFFFFFFF: 5 bytes */
+ {
+ static const u8 data[] = { 0xFF, 0xFF, 0xFF, 0xFF, 0x0F };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, sizeof(data));
+ KUNIT_EXPECT_EQ(test, result, (u32)0xFFFFFFFF);
+ KUNIT_EXPECT_EQ(test, pos, (u32)5);
+ }
+
+ /* Truncated input: pos >= end returns 0 */
+ {
+ static const u8 data[] = { 0x80 };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, 0);
+ KUNIT_EXPECT_EQ_MSG(test, result, (u32)0,
+ "Expected 0 for empty input");
+ }
+
+ /* Truncated mid-varint: continuation byte but end reached */
+ {
+ static const u8 data[] = { 0x80 };
+
+ pos = 0;
+ result = lineinfo_read_uleb128(data, &pos, 1);
+ KUNIT_EXPECT_EQ_MSG(test, result, (u32)0,
+ "Expected 0 for truncated varint");
+ KUNIT_EXPECT_EQ(test, pos, (u32)1);
+ }
+}
+
+/*
+ * The symbol-boundary guard must reject an entry belonging to a preceding
+ * symbol, but must still accept one that sits exactly on the symbol start.
+ *
+ * gen_lineinfo used to collapse the second of two entries sharing a
+ * file:line, which left functions whose first entry was collapsed with
+ * nothing at or above their own start -- so this rejected them and the
+ * frame printed unannotated. The generator now keeps (or synthesizes) an
+ * entry at every symbol start; this pins down the contract it relies on.
+ */
+static void test_lineinfo_search_min_offset(struct kunit *test)
+{
+ /*
+ * One block, two entries with identical file:line at offsets 0 and
+ * 0x40 -- exactly the pair deduplicate() used to collapse.
+ * Entry 0 is uleb(file_id), uleb(line); the rest are
+ * uleb(addr_delta), sleb(file_delta), sleb(line_delta).
+ */
+ static const u8 data[] = { 0x00, 0x0a, 0x40, 0x00, 0x00 };
+ static const u32 blk_addrs[] = { 0 };
+ static const u32 blk_offsets[] = { 0 };
+ static const u32 file_offsets[] = { 0 };
+ static const char filenames[] = "test.c";
+ const struct lineinfo_table tbl = {
+ .blk_addrs = blk_addrs,
+ .blk_offsets = blk_offsets,
+ .data = data,
+ .data_size = sizeof(data),
+ .file_offsets = file_offsets,
+ .filenames = filenames,
+ .num_entries = 2,
+ .num_blocks = 1,
+ .num_files = 1,
+ .filenames_size = sizeof(filenames),
+ };
+ const char *file;
+ unsigned int line;
+
+ /* Symbol starts at 0x40 and owns the entry there: must resolve. */
+ file = NULL;
+ line = 0;
+ KUNIT_EXPECT_TRUE_MSG(test,
+ lineinfo_search(&tbl, 0x50, 0x40, &file, &line),
+ "entry exactly at the symbol start was rejected");
+ KUNIT_EXPECT_STREQ(test, file, "test.c");
+ KUNIT_EXPECT_EQ(test, line, 10u);
+
+ /* No min_offset bound: same answer. */
+ file = NULL;
+ line = 0;
+ KUNIT_EXPECT_TRUE(test,
+ lineinfo_search(&tbl, 0x50, 0, &file, &line));
+ KUNIT_EXPECT_EQ(test, line, 10u);
+
+ /*
+ * Symbol starts at 0x60, above every entry: the 0x40 entry belongs
+ * to a preceding symbol and must not leak into this one.
+ */
+ KUNIT_EXPECT_FALSE_MSG(test,
+ lineinfo_search(&tbl, 0x70, 0x60, &file, &line),
+ "entry below the symbol start was accepted");
+}
+
+static void test_line_number_accuracy(struct kunit *test)
+{
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_normal);
+ unsigned int line;
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(buf, addr);
+ KUNIT_ASSERT_TRUE(test, has_lineinfo(buf));
+
+ line = extract_line(buf);
+
+ /*
+ * lineinfo_target_normal is defined around line 103-107.
+ * Allow wide range: KASAN instrumentation and module lineinfo
+ * address mapping can shift the reported line significantly.
+ */
+ KUNIT_EXPECT_GE_MSG(test, line, (unsigned int)50,
+ "Line %u too low for lineinfo_target_normal", line);
+ KUNIT_EXPECT_LE_MSG(test, line, (unsigned int)300,
+ "Line %u too high for lineinfo_target_normal", line);
+}
+
+static void test_many_lines_mid_function(struct kunit *test)
+{
+ char *first = alloc_sym_buf(test);
+ char *buf = alloc_sym_buf(test);
+ unsigned long addr = target_addr(lineinfo_target_many_lines);
+ unsigned int line = 0;
+
+ skip_without_module_lineinfo(test);
+
+ sprint_with_lineinfo(first, addr);
+ KUNIT_ASSERT_TRUE(test, has_lineinfo(first));
+
+ /*
+ * Find an annotated address from 8 bytes in (past the prologue)
+ * onwards. A fixed offset is not enough: instrumentation such as
+ * KCOV or KCSAN can leave stretches that DWARF attributes to line 0,
+ * which correctly print no annotation. Hide the base from the
+ * compiler so this does not become a static mid-function relocation
+ * (sym+8), which objtool flags as a relocation to !ENDBR with
+ * CONFIG_X86_KERNEL_IBT.
+ */
+ OPTIMIZER_HIDE_VAR(addr);
+ for (unsigned long i = 8; i < 4096; i++) {
+ sprint_with_lineinfo(buf, addr + i);
+ if (!same_symbol(buf, first))
+ break;
+ if (has_lineinfo(buf)) {
+ line = extract_line(buf);
+ break;
+ }
+ }
+
+ KUNIT_ASSERT_GT_MSG(test, line, (unsigned int)0,
+ "No annotated address after: %s", first);
+ KUNIT_EXPECT_GE_MSG(test, line, (unsigned int)50,
+ "Line %u too low for mid-function", line);
+ KUNIT_EXPECT_LE_MSG(test, line, (unsigned int)700,
+ "Line %u too high for mid-function", line);
+}
+
+/* --------------- Suite registration --------------- */
+
+static struct kunit_case lineinfo_test_cases[] = {
+ /* Group A: Basic lineinfo presence */
+ KUNIT_CASE(test_normal_function),
+ KUNIT_CASE(test_noinline_function),
+ KUNIT_CASE(test_inline_function),
+ KUNIT_CASE(test_cross_file_inline),
+ KUNIT_CASE(test_short_function),
+ KUNIT_CASE(test_many_lines_function),
+ /* Group B: Deep call chain */
+ KUNIT_CASE(test_deep_call_chain),
+ /* Group C: sprint_symbol API variants */
+ KUNIT_CASE(test_sprint_symbol_format),
+ KUNIT_CASE(test_sprint_backtrace),
+ KUNIT_CASE(test_sprint_backtrace_build_id),
+ KUNIT_CASE(test_sprint_symbol_no_offset),
+ /* Group D: printk format specifiers */
+ KUNIT_CASE(test_pS_format),
+ KUNIT_CASE(test_pBb_format),
+ KUNIT_CASE(test_pSR_format),
+ /* Group E: Address edge cases */
+ KUNIT_CASE(test_symbol_start_addr),
+ KUNIT_CASE(test_symbol_nonzero_offset),
+ KUNIT_CASE(test_unknown_address),
+ KUNIT_CASE(test_kernel_function_lineinfo),
+ KUNIT_CASE(test_init_text_no_stale_lineinfo),
+ /* Group F: Module path */
+ KUNIT_CASE(test_module_function_lineinfo),
+ KUNIT_CASE(test_second_object_lineinfo),
+ /* Group G: Stress */
+ KUNIT_CASE_SLOW(test_concurrent_sprint_symbol),
+ KUNIT_CASE_SLOW(test_rapid_sprint_symbol),
+ /* Group H: Safety and plausibility */
+ KUNIT_CASE(test_line_number_plausible),
+ KUNIT_CASE(test_buffer_no_overflow),
+ KUNIT_CASE(test_dump_stack_no_crash),
+ KUNIT_CASE(test_sprint_symbol_build_id),
+ /* Group I: Encoding/decoding and accuracy */
+ KUNIT_CASE(test_sleb128_edge_cases),
+ KUNIT_CASE(test_uleb128_edge_cases),
+ KUNIT_CASE(test_lineinfo_search_min_offset),
+ KUNIT_CASE(test_line_number_accuracy),
+ KUNIT_CASE(test_many_lines_mid_function),
+ {}
+};
+
+static struct kunit_suite lineinfo_test_suite = {
+ .name = "lineinfo",
+ .test_cases = lineinfo_test_cases,
+};
+kunit_test_suites(&lineinfo_test_suite);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("KUnit tests for kallsyms lineinfo");
+MODULE_AUTHOR("Sasha Levin");
+MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
diff --git a/lib/tests/lineinfo_kunit.h b/lib/tests/lineinfo_kunit.h
new file mode 100644
index 0000000000000..ba436073855c2
--- /dev/null
+++ b/lib/tests/lineinfo_kunit.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LIB_TESTS_LINEINFO_KUNIT_H
+#define _LIB_TESTS_LINEINFO_KUNIT_H
+
+#include <linux/compiler.h>
+
+int lineinfo_target_second_file(int x);
+
+/*
+ * Inlined into lineinfo_cross_file_caller() in lineinfo_kunit.c, so the
+ * addresses it covers must resolve to this header rather than to the
+ * caller's file. __always_inline because the test object is built with
+ * -fno-inline-functions-called-once. The call gives the inlined body an
+ * instruction of its own: an arm64 build can fold "x + 1" into the
+ * caller's return sequence and leave no address attributed to this file.
+ */
+static __always_inline int lineinfo_header_helper(int x)
+{
+ return lineinfo_target_second_file(x) + 1;
+}
+
+#endif /* _LIB_TESTS_LINEINFO_KUNIT_H */
diff --git a/lib/tests/lineinfo_kunit_second.c b/lib/tests/lineinfo_kunit_second.c
new file mode 100644
index 0000000000000..1151de15c1daf
--- /dev/null
+++ b/lib/tests/lineinfo_kunit_second.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Second translation unit of the lineinfo KUnit test module.
+ *
+ * Copyright (c) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * A module built from one object file cannot tell whether per-compile-unit
+ * DWARF references were relocated: its only compile unit sits at offset 0,
+ * which is exactly what an unrelocated reference reads. Keeping a target
+ * function in a second object makes the test cover the general case.
+ */
+
+#include <linux/compiler.h>
+
+#include "lineinfo_kunit.h"
+
+noinline int lineinfo_target_second_file(int x)
+{
+ barrier();
+ return x + 1;
+}
--
2.53.0
^ permalink raw reply [flat|nested] 6+ messages in thread