mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: Andrew Morton <akpm@linux-foundation.org>,
	Masahiro Yamada <masahiroy@kernel.org>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Richard Weinberger <richard@nod.at>,
	Juergen Gross <jgross@suse.com>,
	Geert Uytterhoeven <geert@linux-m68k.org>,
	James Bottomley <James.Bottomley@HansenPartnership.com>
Cc: Sasha Levin <sashal@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
	Nathan Chancellor <nathan@kernel.org>,
	Nicolas Schier <nsc@kernel.org>, Petr Pavlu <petr.pavlu@suse.com>,
	Daniel Gomez <da.gomez@kernel.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	Petr Mladek <pmladek@suse.com>,
	Steven Rostedt <rostedt@goodmis.org>, Kees Cook <kees@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Thorsten Leemhuis <linux@leemhuis.info>,
	Vlastimil Babka <vbabka@kernel.org>, Helge Deller <deller@gmx.de>,
	Randy Dunlap <rdunlap@infradead.org>,
	Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
	Vivian Wang <wangruikang@iscas.ac.cn>,
	Zhen Lei <thunder.leizhen@huawei.com>,
	Sami Tolvanen <samitolvanen@google.com>,
	linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org,
	linux-modules@vger.kernel.org, linux-doc@vger.kernel.org
Subject: [PATCH v9 3/4] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction
Date: Thu, 17 Sep 2026 09:37:24 -0400	[thread overview]
Message-ID: <20260917133727.428546-4-sashal@kernel.org> (raw)
In-Reply-To: <20260917133727.428546-1-sashal@kernel.org>

Replace the flat uncompressed parallel arrays (lineinfo_addrs[],
lineinfo_file_ids[], lineinfo_lines[]) with a block-indexed,
delta-encoded, ULEB128 varint compressed format.

The sorted address array has small deltas between consecutive entries
(typically 1-50 bytes), file IDs have high locality (delta often 0,
same file), and line numbers change slowly.  Delta-encoding followed
by ULEB128 varint compression shrinks most values from 4 bytes to 1.

Entries are grouped into blocks of 64.  A small uncompressed block
index (first addr + byte offset per block) enables O(log(N/64)) binary
search, followed by sequential decode of at most 64 entries within the
matching block.  All decode state lives on the stack -- zero
allocations, still safe for NMI/panic context.

Measured on x86_64_defconfig + CONFIG_DEBUG_INFO (same recipe as patch
1/4; 1,657,997 entries, 4,235 source files, 25,907 blocks):

  Before (flat arrays):
    lineinfo_addrs[]     6,631,988 bytes (u32 x 1.66M)
    lineinfo_file_ids[]  3,315,994 bytes (u16 x 1.66M)
    lineinfo_lines[]     6,631,988 bytes (u32 x 1.66M)
    Total:              16,579,970 bytes (15.8 MiB, 10.00 bytes/entry)

  After (block-indexed delta + ULEB128):
    lineinfo_block_addrs[]    103,628 bytes (101 KiB)
    lineinfo_block_offsets[]  103,628 bytes (101 KiB)
    lineinfo_data[]         6,070,158 bytes (5.8 MiB)
    Total:                  6,277,414 bytes (6.0 MiB, 3.79 bytes/entry)

  Savings: 9.8 MiB (2.64x reduction)

file_offsets[] and filenames[] are unchanged by this patch and are
excluded from both totals above.

Whole-image effect, measured the same way as patch 1/4:

  vmlinux (stripped), no lineinfo:      52.2 MiB
  vmlinux (stripped), before this fix:  68.2 MiB  (+16.0 MiB / +30.6%)
  vmlinux (stripped), after this fix:   60.2 MiB  ( +8.0 MiB / +15.3%)

Booted in QEMU and verified with SysRq-l that annotations still work:

  default_idle+0x9/0x10 (arch/x86/kernel/process.c:768)
  default_idle_call+0x6e/0xb0 (kernel/sched/idle.c:122)
  do_idle+0x38f/0x660 (kernel/sched/idle.c:199)
  cpu_startup_entry+0x4e/0x60 (kernel/sched/idle.c:453)
  rest_init+0x277/0x280 (init/main.c:732)

Suggested-by: Juergen Gross <jgross@suse.com>
Assisted-by: LLM
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 .../admin-guide/kallsyms-lineinfo.rst         |   6 +-
 include/linux/mod_lineinfo.h                  | 263 +++++++++++++++---
 init/Kconfig                                  |   8 +-
 kernel/kallsyms.c                             |  60 ++--
 kernel/kallsyms_internal.h                    |   8 +-
 kernel/module/kallsyms.c                      | 127 ++++-----
 scripts/empty_lineinfo.S                      |  20 +-
 scripts/gen_lineinfo.c                        | 185 ++++++++----
 8 files changed, 457 insertions(+), 220 deletions(-)

diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentation/admin-guide/kallsyms-lineinfo.rst
index 227ed9413be6c..a659f8564b5ed 100644
--- a/Documentation/admin-guide/kallsyms-lineinfo.rst
+++ b/Documentation/admin-guide/kallsyms-lineinfo.rst
@@ -77,10 +77,10 @@ Memory Overhead
 
 The vmlinux lineinfo tables are stored in ``.rodata``.  On an x86_64
 ``defconfig`` with ``CONFIG_DEBUG_INFO`` they hold 1.66 million entries and
-grow the stripped image by 16 MiB, about 10 bytes per entry after
-deduplication.
+grow the stripped image by 8 MiB, about 3.8 bytes per entry after delta
+compression.
 
-Per-module lineinfo adds about 10 bytes per entry to each ``.ko`` file, plus
+Per-module lineinfo adds about 4 bytes per entry to each ``.ko`` file, plus
 a small fixed cost per covered section.
 
 Known Limitations
diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h
index cb0c7af7b3171..98208b5c04acf 100644
--- a/include/linux/mod_lineinfo.h
+++ b/include/linux/mod_lineinfo.h
@@ -3,9 +3,9 @@
  * mod_lineinfo.h - Binary format for per-module source line information
  *
  * This header defines the layout of the .mod_lineinfo and
- * .init.mod_lineinfo sections embedded in loadable kernel modules.  It
- * is dual-use: included from both the kernel and the userspace
- * gen_lineinfo tool.
+ * .init.mod_lineinfo sections embedded in loadable kernel modules.  It is
+ * dual-use: included from both the kernel and the userspace gen_lineinfo
+ * tool.
  *
  * Top-level layout (all values in target-native endianness):
  *
@@ -20,16 +20,27 @@
  * If the relocation fails to resolve (e.g. unknown reloc type), .anchor
  * stays zero and lookups silently degrade to "no annotation".
  *
- * Each per-section sub-table is laid out as a stand-alone
- * mod_lineinfo_header followed by parallel arrays:
+ * Each per-section sub-table is laid out exactly as a stand-alone
+ * mod_lineinfo_header followed by its arrays:
  *
- *   struct mod_lineinfo_header     (16 bytes)
- *   u32 addrs[num_entries]         -- offsets from this section's base, sorted
- *   u16 file_ids[num_entries]      -- parallel to addrs
- *   <2-byte pad if num_entries is odd>
- *   u32 lines[num_entries]         -- parallel to addrs
+ *   struct mod_lineinfo_header
+ *   u32 block_addrs[num_blocks]    -- first addr per block, for binary search
+ *   u32 block_offsets[num_blocks]  -- byte offset into compressed data stream
+ *   u8  data[data_size]            -- LEB128 delta-compressed entries
  *   u32 file_offsets[num_files]    -- byte offset into filenames[]
  *   char filenames[filenames_size] -- concatenated NUL-terminated strings
+ *
+ * Each sub-array is located by an explicit (offset, size) pair in the
+ * header, similar to a flattened devicetree.  All offsets in the per-section
+ * header are relative to that header itself, so a sub-table is fully
+ * self-describing.
+ *
+ * Compressed stream format (per block of LINEINFO_BLOCK_ENTRIES entries):
+ *   Entry 0: file_id (ULEB128), line (ULEB128)
+ *            addr is in block_addrs[]
+ *   Entry 1..N: addr_delta (ULEB128),
+ *               file_id_delta (SLEB128),
+ *               line_delta (SLEB128)
  */
 #ifndef _LINUX_MOD_LINEINFO_H
 #define _LINUX_MOD_LINEINFO_H
@@ -44,6 +55,7 @@
 #include <stdint.h>
 typedef uint32_t u32;
 typedef uint16_t u16;
+typedef uint8_t  u8;
 typedef uint64_t u64;
 #ifndef __aligned
 #define __aligned(x)	__attribute__((__aligned__(x)))
@@ -53,6 +65,8 @@ typedef uint64_t u64;
 #endif
 #endif
 
+#define LINEINFO_BLOCK_ENTRIES 64
+
 /*
  * Per-section descriptor.  One entry per ELF text section covered by the
  * blob (.text, .exit.text, .init.text, ...).
@@ -84,47 +98,222 @@ static_assert(sizeof(struct mod_lineinfo_section) == 16,
 
 struct mod_lineinfo_header {
 	u32 num_entries;
+	u32 num_blocks;
 	u32 num_files;
-	u32 filenames_size;	/* total bytes of concatenated filenames */
+	u32 blocks_offset;	/* offset to block_addrs[] from this header */
+	u32 blocks_size;	/* bytes: num_blocks * 2 * sizeof(u32) */
+	u32 data_offset;	/* offset to compressed stream */
+	u32 data_size;		/* bytes of compressed data */
+	u32 files_offset;	/* offset to file_offsets[] */
+	u32 files_size;		/* bytes: num_files * sizeof(u32) */
+	u32 filenames_offset;
+	u32 filenames_size;
 };
 
-/* Offset helpers: compute byte offset from the per-section header to each array. */
-
-static inline u32 mod_lineinfo_addrs_off(void)
-{
-	return sizeof(struct mod_lineinfo_header);
-}
+/*
+ * Descriptor for a lineinfo table, used by the shared lookup function.
+ * Callers populate this from either linker globals (vmlinux) or a
+ * validated mod_lineinfo_header (modules).
+ */
+struct lineinfo_table {
+	const u32 *blk_addrs;
+	const u32 *blk_offsets;
+	const u8  *data;
+	u32 data_size;
+	const u32 *file_offsets;
+	const char *filenames;
+	u32 num_entries;
+	u32 num_blocks;
+	u32 num_files;
+	u32 filenames_size;
+};
 
 /*
- * The counts come from an on-disk blob and are only validated against the
- * blob size once the full layout has been summed, so every step widens to
- * u64: at 10 bytes per entry the u32 sums wrap for counts a caller can
- * name, which would let a malformed blob pass a bounds check computed from
- * the wrapped value.
+ * Read a ULEB128 varint from a byte stream.
+ * Returns the decoded value and advances *pos past the encoded bytes.
+ * If *pos would exceed 'end', returns 0 and sets *pos = end (safe for
+ * NMI/panic context: no crash, just a missed annotation).
  */
-static inline u64 mod_lineinfo_file_ids_off(u32 num_entries)
+static inline u32 lineinfo_read_uleb128(const u8 *data, u32 *pos, u32 end)
 {
-	return mod_lineinfo_addrs_off() + (u64)num_entries * sizeof(u32);
-}
+	u32 result = 0;
+	unsigned int shift = 0;
 
-static inline u64 mod_lineinfo_lines_off(u32 num_entries)
-{
-	/* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */
-	u64 off = mod_lineinfo_file_ids_off(num_entries) +
-		  (u64)num_entries * sizeof(u16);
-	return (off + 3) & ~3ULL;
+	while (*pos < end) {
+		u8 byte = data[*pos];
+		(*pos)++;
+		result |= (u32)(byte & 0x7f) << shift;
+		if (!(byte & 0x80))
+			return result;
+		shift += 7;
+		if (shift >= 32) {
+			/* Malformed: skip remaining continuation bytes */
+			while (*pos < end && (data[*pos] & 0x80))
+				(*pos)++;
+			if (*pos < end)
+				(*pos)++;
+			return result;
+		}
+	}
+	return result;
 }
 
-static inline u64 mod_lineinfo_file_offsets_off(u32 num_entries)
+/* Read an SLEB128 varint. Same safety guarantees as above. */
+static inline int32_t lineinfo_read_sleb128(const u8 *data, u32 *pos, u32 end)
 {
-	return mod_lineinfo_lines_off(num_entries) +
-	       (u64)num_entries * sizeof(u32);
+	int32_t result = 0;
+	unsigned int shift = 0;
+	u8 byte = 0;
+
+	while (*pos < end) {
+		byte = data[*pos];
+		(*pos)++;
+		result |= (int32_t)((u32)(byte & 0x7f) << shift);
+		shift += 7;
+		if (!(byte & 0x80))
+			break;
+		if (shift >= 32) {
+			while (*pos < end && (data[*pos] & 0x80))
+				(*pos)++;
+			if (*pos < end)
+				(*pos)++;
+			return result;
+		}
+	}
+
+	/* Sign-extend if the high bit of the last byte was set */
+	if (shift < 32 && (byte & 0x40))
+		result |= -(1 << shift);
+
+	return result;
 }
 
-static inline u64 mod_lineinfo_filenames_off(u32 num_entries, u32 num_files)
+/*
+ * Search a lineinfo table for the source file and line corresponding to a
+ * given offset (from _text for vmlinux, from the covered section's base for
+ * modules).  @min_offset is the containing symbol's start in the same offset
+ * space: entries below it belong to a preceding symbol and are rejected.
+ *
+ * Safe for NMI and panic context: no locks, no allocations, all state on stack.
+ * Returns true and sets @file and @line on success; false on any failure.
+ */
+static inline bool lineinfo_search(const struct lineinfo_table *tbl,
+				   unsigned int offset,
+				   unsigned int min_offset,
+				   const char **file, unsigned int *line)
 {
-	return mod_lineinfo_file_offsets_off(num_entries) +
-	       (u64)num_files * sizeof(u32);
+	unsigned int low, high, mid, block;
+	unsigned int cur_addr, cur_file_id, cur_line;
+	unsigned int best_addr = 0, best_file_id = 0, best_line = 0;
+	unsigned int block_entries, data_end;
+	bool found = false;
+	u32 pos;
+
+	if (!tbl->num_entries || !tbl->num_blocks)
+		return false;
+
+	/* Binary search on blk_addrs[] to find the right block */
+	low = 0;
+	high = tbl->num_blocks;
+	while (low < high) {
+		mid = low + (high - low) / 2;
+		if (tbl->blk_addrs[mid] <= offset)
+			low = mid + 1;
+		else
+			high = mid;
+	}
+
+	if (low == 0)
+		return false;
+	block = low - 1;
+
+	/* How many entries in this block? */
+	block_entries = LINEINFO_BLOCK_ENTRIES;
+	if (block == tbl->num_blocks - 1) {
+		unsigned int remaining = tbl->num_entries -
+					block * LINEINFO_BLOCK_ENTRIES;
+
+		if (remaining < block_entries)
+			block_entries = remaining;
+	}
+
+	/* Determine end of this block's data in the compressed stream */
+	if (block + 1 < tbl->num_blocks)
+		data_end = tbl->blk_offsets[block + 1];
+	else
+		data_end = tbl->data_size;
+
+	/* Clamp data_end to actual data size */
+	if (data_end > tbl->data_size)
+		data_end = tbl->data_size;
+
+	/* Decode entry 0: addr from blk_addrs, file_id and line from stream */
+	pos = tbl->blk_offsets[block];
+	if (pos >= data_end)
+		return false;
+
+	cur_addr = tbl->blk_addrs[block];
+	cur_file_id = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+	cur_line = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+
+	/* Check entry 0 */
+	if (cur_addr <= offset) {
+		best_addr = cur_addr;
+		best_file_id = cur_file_id;
+		best_line = cur_line;
+		found = true;
+	}
+
+	/* Decode entries 1..N */
+	for (unsigned int i = 1; i < block_entries; i++) {
+		unsigned int addr_delta;
+		int32_t file_delta, line_delta;
+
+		addr_delta = lineinfo_read_uleb128(tbl->data, &pos, data_end);
+		file_delta = lineinfo_read_sleb128(tbl->data, &pos, data_end);
+		line_delta = lineinfo_read_sleb128(tbl->data, &pos, data_end);
+
+		cur_addr += addr_delta;
+		cur_file_id = (unsigned int)((int32_t)cur_file_id + file_delta);
+		cur_line = (unsigned int)((int32_t)cur_line + line_delta);
+
+		if (cur_addr > offset)
+			break;
+
+		best_addr = cur_addr;
+		best_file_id = cur_file_id;
+		best_line = cur_line;
+		found = true;
+	}
+
+	if (!found)
+		return false;
+
+	/*
+	 * The best entry is the closest one at or below @offset; reject it
+	 * if it lies below the resolved symbol's start, so a symbol without
+	 * line entries of its own does not inherit the preceding symbol's
+	 * annotation.
+	 */
+	if (best_addr < min_offset)
+		return false;
+
+	/*
+	 * A zero line is the generator's "no source location applies here"
+	 * marker, taken straight from a DWARF line-0 row.
+	 */
+	if (!best_line)
+		return false;
+
+	if (best_file_id >= tbl->num_files)
+		return false;
+
+	if (tbl->file_offsets[best_file_id] >= tbl->filenames_size)
+		return false;
+
+	*file = &tbl->filenames[tbl->file_offsets[best_file_id]];
+	*line = best_line;
+	return true;
 }
 
 #endif /* _LINUX_MOD_LINEINFO_H */
diff --git a/init/Kconfig b/init/Kconfig
index debf9c6f9e813..1cba9401d4452 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2147,8 +2147,9 @@ config KALLSYMS_LINEINFO
 	    anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412)
 
 	  This requires libelf and libdw (from elfutils) on the build host.
-	  Costs 10 bytes per DWARF line-table entry; for x86_64_defconfig
-	  with CONFIG_DEBUG_INFO that is about 18MB.
+	  Costs about 3.8 bytes per DWARF line-table entry after delta
+	  compression; for x86_64_defconfig with CONFIG_DEBUG_INFO that is
+	  about 8MB.
 
 	  If unsure, say N.
 
@@ -2161,7 +2162,8 @@ config KALLSYMS_LINEINFO_MODULES
 	  so stack traces from module code include (file.c:123) annotations.
 
 	  This requires libelf and libdw (from elfutils) on the build host.
-	  Costs 10 bytes per DWARF line entry in each .ko.
+	  Costs about 3.8 bytes per DWARF line entry in each .ko after
+	  delta compression.
 
 	  If unsure, say N.
 
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index 77543ac216a92..395297e27af53 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -467,13 +467,17 @@ static int append_buildid(char *buffer,   const char *modname,
 
 #endif /* CONFIG_STACKTRACE_BUILD_ID */
 
+#include <linux/mod_lineinfo.h>
+
 bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
 			      const char **file, unsigned int *line)
 {
 	unsigned long raw_offset, raw_min;
-	unsigned int offset, min_offset = 0, low, high, mid, file_id;
+	unsigned int offset, min_offset = 0;
+	struct lineinfo_table tbl;
 
-	if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || !lineinfo_num_entries)
+	if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) ||
+	    !lineinfo_num_entries || !lineinfo_num_blocks)
 		return false;
 
 	/* Compute offset from _text */
@@ -491,8 +495,8 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
 		return false;
 
 	/*
-	 * The search below returns the closest entry at or below @offset, so
-	 * a symbol without line entries of its own (assembly without debug
+	 * The search returns the closest entry at or below the offset, so a
+	 * symbol without line entries of its own (assembly without debug
 	 * info, or anything past the _etext cap like .init.text) would
 	 * inherit the last entry of whatever precedes it.  Bound the result
 	 * to entries at or above the resolved symbol's start.
@@ -504,42 +508,18 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start,
 			min_offset = raw_min;
 	}
 
-	/* Binary search for largest entry <= offset */
-	low = 0;
-	high = lineinfo_num_entries;
-	while (low < high) {
-		mid = low + (high - low) / 2;
-		if (lineinfo_addrs[mid] <= offset)
-			low = mid + 1;
-		else
-			high = mid;
-	}
-
-	if (low == 0)
-		return false;
-	low--;
-
-	if (lineinfo_addrs[low] < min_offset)
-		return false;
-
-	/*
-	 * A zero line is the generator's "no source location applies here"
-	 * marker, taken straight from a DWARF line-0 row.
-	 */
-	if (!lineinfo_lines[low])
-		return false;
-
-	file_id = lineinfo_file_ids[low];
-	*line = lineinfo_lines[low];
-
-	if (file_id >= lineinfo_num_files)
-		return false;
-
-	if (lineinfo_file_offsets[file_id] >= lineinfo_filenames_size)
-		return false;
-
-	*file = &lineinfo_filenames[lineinfo_file_offsets[file_id]];
-	return true;
+	tbl.blk_addrs	= lineinfo_block_addrs;
+	tbl.blk_offsets	= lineinfo_block_offsets;
+	tbl.data	= lineinfo_data;
+	tbl.data_size	= lineinfo_data_size;
+	tbl.file_offsets = lineinfo_file_offsets;
+	tbl.filenames	= lineinfo_filenames;
+	tbl.num_entries	= lineinfo_num_entries;
+	tbl.num_blocks	= lineinfo_num_blocks;
+	tbl.num_files	= lineinfo_num_files;
+	tbl.filenames_size = lineinfo_filenames_size;
+
+	return lineinfo_search(&tbl, offset, min_offset, file, line);
 }
 
 /* Look up a kernel symbol and return it in a text buffer. */
diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h
index d7374ce444d81..ffe4c658067ec 100644
--- a/kernel/kallsyms_internal.h
+++ b/kernel/kallsyms_internal.h
@@ -16,10 +16,12 @@ extern const unsigned int kallsyms_markers[];
 extern const u8 kallsyms_seqs_of_names[];
 
 extern const u32 lineinfo_num_entries;
-extern const u32 lineinfo_addrs[];
-extern const u16 lineinfo_file_ids[];
-extern const u32 lineinfo_lines[];
 extern const u32 lineinfo_num_files;
+extern const u32 lineinfo_num_blocks;
+extern const u32 lineinfo_block_addrs[];
+extern const u32 lineinfo_block_offsets[];
+extern const u32 lineinfo_data_size;
+extern const u8  lineinfo_data[];
 extern const u32 lineinfo_file_offsets[];
 extern const u32 lineinfo_filenames_size;
 extern const char lineinfo_filenames[];
diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c
index 511cfa58a3e44..1c9491b3617c1 100644
--- a/kernel/module/kallsyms.c
+++ b/kernel/module/kallsyms.c
@@ -506,9 +506,9 @@ int module_kallsyms_on_each_symbol(const char *modname,
 #include <linux/mod_lineinfo.h>
 
 /*
- * Search one per-section sub-table for @section_offset using flat parallel
- * arrays.  @hdr is the per-section header at byte offset @hdr_offset within
- * @blob.  Returns true on hit and populates @file / @line.
+ * Search one per-section sub-table for @section_offset.
+ * @hdr is the per-section header at byte offset @hdr_offset within @blob.
+ * Returns true on hit and populates @file / @line.
  */
 static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
 					   u32 hdr_offset,
@@ -518,13 +518,8 @@ static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
 					   unsigned int *line)
 {
 	const struct mod_lineinfo_header *hdr;
-	const u8 *base;
-	const u32 *addrs, *lines, *file_offsets;
-	const u16 *file_ids;
-	const char *filenames;
-	u32 num_entries, num_files, filenames_size;
-	unsigned int low, high, mid;
-	u16 file_id;
+	struct lineinfo_table tbl;
+	const void *base;
 
 	if (hdr_offset > blob_size ||
 	    blob_size - hdr_offset < sizeof(*hdr))
@@ -540,84 +535,72 @@ static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size,
 		return false;
 
 	base = (const u8 *)blob + hdr_offset;
-	hdr = (const struct mod_lineinfo_header *)base;
-	num_entries = hdr->num_entries;
-	num_files = hdr->num_files;
-	filenames_size = hdr->filenames_size;
+	hdr = base;
 
-	if (num_entries == 0)
+	if (hdr->num_entries == 0 || hdr->num_blocks == 0)
 		return false;
 
-	/*
-	 * Check the whole layout against the blob in one go.  The offset
-	 * helpers sum in u64 precisely because a malformed blob can name
-	 * counts whose u32 sum wraps: at 10 bytes per entry across addrs[],
-	 * file_ids[] and lines[], num_entries = 0x33333334 wraps to a small
-	 * value that any bounds check would happily accept.
-	 */
+	/* Validate each sub-array fits within the remaining blob bytes */
 	{
 		u32 avail = blob_size - hdr_offset;
-		u64 needed = mod_lineinfo_filenames_off(num_entries, num_files);
 
-		if (needed > avail || filenames_size > avail - needed)
+		if (hdr->blocks_offset > avail ||
+		    hdr->blocks_size > avail - hdr->blocks_offset)
+			return false;
+		if (hdr->data_offset > avail ||
+		    hdr->data_size > avail - hdr->data_offset)
+			return false;
+		if (hdr->files_offset > avail ||
+		    hdr->files_size > avail - hdr->files_offset)
 			return false;
-	}
-
-	/*
-	 * Filenames are read as NUL-terminated C strings.  Require the blob
-	 * to end in NUL so a malformed file_offsets entry can never lead the
-	 * later "%s" consumer past the end of the section.
-	 */
-	if (filenames_size == 0 ||
-	    base[mod_lineinfo_filenames_off(num_entries, num_files) +
-		 filenames_size - 1] != 0)
-		return false;
 
-	addrs = (const u32 *)(base + mod_lineinfo_addrs_off());
-	file_ids = (const u16 *)(base + mod_lineinfo_file_ids_off(num_entries));
-	lines = (const u32 *)(base + mod_lineinfo_lines_off(num_entries));
-	file_offsets = (const u32 *)(base + mod_lineinfo_file_offsets_off(num_entries));
-	filenames = (const char *)(base + mod_lineinfo_filenames_off(num_entries, num_files));
-
-	/* Binary search for largest entry <= section_offset. */
-	low = 0;
-	high = num_entries;
-	while (low < high) {
-		mid = low + (high - low) / 2;
-		if (addrs[mid] <= section_offset)
-			low = mid + 1;
-		else
-			high = mid;
+		/*
+		 * block_addrs[], block_offsets[] and file_offsets[] are read
+		 * as u32 arrays, so their offsets need the same alignment
+		 * guarantee hdr_offset got above.
+		 */
+		if (!IS_ALIGNED(hdr->blocks_offset, sizeof(u32)) ||
+		    !IS_ALIGNED(hdr->files_offset, sizeof(u32)))
+			return false;
+		if (hdr->filenames_offset > avail ||
+		    hdr->filenames_size > avail - hdr->filenames_offset)
+			return false;
 	}
 
-	if (low == 0)
-		return false;
-	low--;
-
 	/*
-	 * Reject entries below the resolved symbol's start so a symbol
-	 * without line entries of its own does not inherit the preceding
-	 * symbol's annotation.
+	 * Validate counts before multiplying by element size — multiplication
+	 * could otherwise overflow on 32-bit builds with a malformed blob.
+	 * num_blocks contributes (addr,offset) u32 pairs; num_files contributes
+	 * one u32 each.
 	 */
-	if (addrs[low] < min_offset)
+	if (hdr->num_blocks > hdr->blocks_size / (2 * sizeof(u32)))
+		return false;
+	if (hdr->num_files > hdr->files_size / sizeof(u32))
 		return false;
 
 	/*
-	 * A zero line is the generator's "no source location applies here"
-	 * marker, taken straight from a DWARF line-0 row.
+	 * Filenames are read as NUL-terminated C strings.  Require the blob
+	 * to end in NUL so a malformed file_offsets entry can never lead the
+	 * later "%s" consumer past the end of the section.
 	 */
-	if (!lines[low])
-		return false;
-
-	file_id = file_ids[low];
-	if (file_id >= num_files)
-		return false;
-	if (file_offsets[file_id] >= filenames_size)
+	if (hdr->filenames_size == 0 ||
+	    ((const u8 *)base)[hdr->filenames_offset +
+			       hdr->filenames_size - 1] != 0)
 		return false;
 
-	*file = &filenames[file_offsets[file_id]];
-	*line = lines[low];
-	return true;
+	tbl.blk_addrs	= base + hdr->blocks_offset;
+	tbl.blk_offsets	= base + hdr->blocks_offset +
+			  hdr->num_blocks * sizeof(u32);
+	tbl.data	= base + hdr->data_offset;
+	tbl.data_size	= hdr->data_size;
+	tbl.file_offsets = base + hdr->files_offset;
+	tbl.filenames	= base + hdr->filenames_offset;
+	tbl.num_entries	= hdr->num_entries;
+	tbl.num_blocks	= hdr->num_blocks;
+	tbl.num_files	= hdr->num_files;
+	tbl.filenames_size = hdr->filenames_size;
+
+	return lineinfo_search(&tbl, section_offset, min_offset, file, line);
 }
 
 /*
@@ -644,6 +627,7 @@ static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size,
 	if (root->num_sections == 0)
 		return false;
 
+	/* Validate sections[] array fits within the blob */
 	if (root->num_sections > U32_MAX / sizeof(struct mod_lineinfo_section))
 		return false;
 	sections_end = sizeof(*root) +
@@ -679,6 +663,9 @@ static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size,
 
 /*
  * Look up source file:line for an address within a loaded module.
+ * Uses the .mod_lineinfo / .init.mod_lineinfo sections embedded in the .ko
+ * at build time.  Each section contains one or more per-section sub-tables
+ * keyed by an ELF-relocation-resolved anchor.
  *
  * Safe in NMI/panic context: no locks, no allocations.
  * Caller must hold RCU read lock (or be in a context where the module
diff --git a/scripts/empty_lineinfo.S b/scripts/empty_lineinfo.S
index e058c41137123..edd5b1092f050 100644
--- a/scripts/empty_lineinfo.S
+++ b/scripts/empty_lineinfo.S
@@ -14,12 +14,20 @@ lineinfo_num_entries:
 	.balign 4
 lineinfo_num_files:
 	.long 0
-	.globl lineinfo_addrs
-lineinfo_addrs:
-	.globl lineinfo_file_ids
-lineinfo_file_ids:
-	.globl lineinfo_lines
-lineinfo_lines:
+	.globl lineinfo_num_blocks
+	.balign 4
+lineinfo_num_blocks:
+	.long 0
+	.globl lineinfo_block_addrs
+lineinfo_block_addrs:
+	.globl lineinfo_block_offsets
+lineinfo_block_offsets:
+	.globl lineinfo_data_size
+	.balign 4
+lineinfo_data_size:
+	.long 0
+	.globl lineinfo_data
+lineinfo_data:
 	.globl lineinfo_file_offsets
 lineinfo_file_offsets:
 	.globl lineinfo_filenames_size
diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c
index 3f889e0c2281c..428e49c291fd8 100644
--- a/scripts/gen_lineinfo.c
+++ b/scripts/gen_lineinfo.c
@@ -2018,6 +2018,45 @@ static void deduplicate(struct covered_section *sections,
 	}
 }
 
+/*
+ * Emit the LEB128 delta-compressed data stream for one block.
+ * @base is the absolute index of the first entry, @count is the number of
+ * entries in this block (<= LINEINFO_BLOCK_ENTRIES).  Used by both vmlinux
+ * mode (one section, full entries[]) and module mode (per-section ranges).
+ */
+static void emit_block_data_range(unsigned int base, unsigned int count)
+{
+	if (!count)
+		return;
+
+	/* Entry 0: file_id, line (both unsigned) */
+	printf("\t.uleb128 %u\n", entries[base].file_id);
+	printf("\t.uleb128 %u\n", entries[base].line);
+
+	/* Entries 1..N: addr_delta (unsigned), file/line deltas (signed) */
+	for (unsigned int i = 1; i < count; i++) {
+		unsigned int idx = base + i;
+
+		printf("\t.uleb128 %u\n",
+		       entries[idx].offset - entries[idx - 1].offset);
+		printf("\t.sleb128 %d\n",
+		       (int)entries[idx].file_id - (int)entries[idx - 1].file_id);
+		printf("\t.sleb128 %d\n",
+		       (int)entries[idx].line - (int)entries[idx - 1].line);
+	}
+}
+
+/* Vmlinux-mode wrapper: pick block index out of the global entries[]. */
+static void emit_block_data(unsigned int block)
+{
+	unsigned int base = block * LINEINFO_BLOCK_ENTRIES;
+	unsigned int count = num_entries - base;
+
+	if (count > LINEINFO_BLOCK_ENTRIES)
+		count = LINEINFO_BLOCK_ENTRIES;
+	emit_block_data_range(base, count);
+}
+
 static void compute_file_offsets(void)
 {
 	unsigned int offset = 0;
@@ -2041,6 +2080,11 @@ static void print_escaped_asciz(const char *s)
 
 static void output_assembly(void)
 {
+	unsigned int num_blocks;
+
+	num_blocks = num_entries ?
+		(num_entries + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0;
+
 	printf("/* SPDX-License-Identifier: GPL-2.0 */\n");
 	printf("/*\n");
 	printf(" * Automatically generated by scripts/gen_lineinfo\n");
@@ -2061,29 +2105,40 @@ static void output_assembly(void)
 	printf("lineinfo_num_files:\n");
 	printf("\t.long %u\n\n", num_files);
 
-	/* Sorted address offsets from _text */
-	printf("\t.globl lineinfo_addrs\n");
+	/* Number of blocks */
+	printf("\t.globl lineinfo_num_blocks\n");
 	printf("\t.balign 4\n");
-	printf("lineinfo_addrs:\n");
-	for (unsigned int i = 0; i < num_entries; i++)
-		printf("\t.long 0x%x\n", entries[i].offset);
-	printf("\n");
+	printf("lineinfo_num_blocks:\n");
+	printf("\t.long %u\n\n", num_blocks);
 
-	/* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */
-	printf("\t.globl lineinfo_file_ids\n");
-	printf("\t.balign 2\n");
-	printf("lineinfo_file_ids:\n");
-	for (unsigned int i = 0; i < num_entries; i++)
-		printf("\t.short %u\n", entries[i].file_id);
-	printf("\n");
+	/* Block first-addresses for binary search */
+	printf("\t.globl lineinfo_block_addrs\n");
+	printf("\t.balign 4\n");
+	printf("lineinfo_block_addrs:\n");
+	for (unsigned int i = 0; i < num_blocks; i++)
+		printf("\t.long 0x%x\n", entries[i * LINEINFO_BLOCK_ENTRIES].offset);
 
-	/* Line numbers, parallel to addrs */
-	printf("\t.globl lineinfo_lines\n");
+	/* Block byte offsets into compressed stream */
+	printf("\t.globl lineinfo_block_offsets\n");
 	printf("\t.balign 4\n");
-	printf("lineinfo_lines:\n");
-	for (unsigned int i = 0; i < num_entries; i++)
-		printf("\t.long %u\n", entries[i].line);
-	printf("\n");
+	printf("lineinfo_block_offsets:\n");
+	for (unsigned int i = 0; i < num_blocks; i++)
+		printf("\t.long .Lblock_%u - lineinfo_data\n", i);
+
+	/* Compressed data size */
+	printf("\t.globl lineinfo_data_size\n");
+	printf("\t.balign 4\n");
+	printf("lineinfo_data_size:\n");
+	printf("\t.long .Ldata_end - lineinfo_data\n\n");
+
+	/* Compressed data stream */
+	printf("\t.globl lineinfo_data\n");
+	printf("lineinfo_data:\n");
+	for (unsigned int i = 0; i < num_blocks; i++) {
+		printf(".Lblock_%u:\n", i);
+		emit_block_data(i);
+	}
+	printf(".Ldata_end:\n\n");
 
 	/* File string offset table */
 	printf("\t.globl lineinfo_file_offsets\n");
@@ -2091,45 +2146,39 @@ static void output_assembly(void)
 	printf("lineinfo_file_offsets:\n");
 	for (unsigned int i = 0; i < num_files; i++)
 		printf("\t.long %u\n", files[i]->str_offset);
-	printf("\n");
 
 	/* Filenames size */
-	{
-		unsigned int fsize = 0;
-
-		for (unsigned int i = 0; i < num_files; i++)
-			fsize += strlen(files[i]->name) + 1;
-		printf("\t.globl lineinfo_filenames_size\n");
-		printf("\t.balign 4\n");
-		printf("lineinfo_filenames_size:\n");
-		printf("\t.long %u\n\n", fsize);
-	}
+	printf("\t.globl lineinfo_filenames_size\n");
+	printf("\t.balign 4\n");
+	printf("lineinfo_filenames_size:\n");
+	printf("\t.long .Lfilenames_end - lineinfo_filenames\n\n");
 
 	/* Concatenated NUL-terminated filenames */
 	printf("\t.globl lineinfo_filenames\n");
 	printf("lineinfo_filenames:\n");
 	for (unsigned int i = 0; i < num_files; i++)
 		print_escaped_asciz(files[i]->name);
-	printf("\n");
+	printf(".Lfilenames_end:\n");
 }
 
 /*
- * Emit one per-section table in the simple flat-array layout:
+ * Emit one per-section table.  @suffix uniquifies the local labels so
+ * multiple tables can coexist in a single output blob; @blob_root_label
+ * is the symbol for the start of the enclosing blob (used for
+ * table_offset = .Lhdr - .Lroot).
  *
- *   mod_lineinfo_header
- *   addrs[count]    (u32, sorted)
- *   file_ids[count] (u16) + 2-byte pad if count is odd
- *   lines[count]    (u32)
- *   file_offsets[]  (u32)
- *   filenames[]
- *
- * @suffix uniquifies labels so multiple tables can coexist in one blob.
- * Caller has sorted entries[] so this section's entries occupy [first,
- * first + count).
+ * Caller has already sorted entries[] so this section's entries occupy
+ * the contiguous range [first, first + count).  This function emits
+ * block-relative addresses computed from entries[first + N].offset.
  */
 static void emit_section_table(unsigned int first, unsigned int count,
 			       const char *suffix)
 {
+	unsigned int num_blocks;
+
+	num_blocks = count ?
+		(count + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0;
+
 	/*
 	 * Align before defining the label, not after: the descriptor stores
 	 * table_offset as .Lhdr - .Lroot, and every field offset inside the
@@ -2140,29 +2189,45 @@ static void emit_section_table(unsigned int first, unsigned int count,
 	printf("\t.balign 4\n");
 	printf(".Lhdr%s:\n", suffix);
 	printf("\t.long %u\t\t/* num_entries */\n", count);
+	printf("\t.long %u\t\t/* num_blocks */\n", num_blocks);
 	printf("\t.long %u\t\t/* num_files */\n", num_files);
+	printf("\t.long .Lblk_addrs%s - .Lhdr%s\n", suffix, suffix);
+	printf("\t.long .Lblk_offsets_end%s - .Lblk_addrs%s\n", suffix, suffix);
+	printf("\t.long .Ldata%s - .Lhdr%s\n", suffix, suffix);
+	printf("\t.long .Ldata_end%s - .Ldata%s\n", suffix, suffix);
+	printf("\t.long .Lfile_offsets%s - .Lhdr%s\n", suffix, suffix);
+	printf("\t.long .Lfile_offsets_end%s - .Lfile_offsets%s\n", suffix, suffix);
+	printf("\t.long .Lfilenames%s - .Lhdr%s\n", suffix, suffix);
 	printf("\t.long .Lfilenames_end%s - .Lfilenames%s\n\n", suffix, suffix);
 
-	/* addrs[] */
-	for (unsigned int i = 0; i < count; i++)
-		printf("\t.long 0x%x\n", entries[first + i].offset);
-
-	/* file_ids[] */
-	for (unsigned int i = 0; i < count; i++)
-		printf("\t.short %u\n", entries[first + i].file_id);
-	if (count & 1)
-		printf("\t.short 0\t\t/* pad to align lines[] */\n");
-
-	/* lines[] */
-	for (unsigned int i = 0; i < count; i++)
-		printf("\t.long %u\n", entries[first + i].line);
+	printf(".Lblk_addrs%s:\n", suffix);
+	for (unsigned int i = 0; i < num_blocks; i++)
+		printf("\t.long 0x%x\n",
+		       entries[first + i * LINEINFO_BLOCK_ENTRIES].offset);
+
+	printf(".Lblk_offsets%s:\n", suffix);
+	for (unsigned int i = 0; i < num_blocks; i++)
+		printf("\t.long .Lblock%s_%u - .Ldata%s\n", suffix, i, suffix);
+	printf(".Lblk_offsets_end%s:\n\n", suffix);
+
+	printf(".Ldata%s:\n", suffix);
+	for (unsigned int i = 0; i < num_blocks; i++) {
+		unsigned int base = first + i * LINEINFO_BLOCK_ENTRIES;
+		unsigned int n = count - i * LINEINFO_BLOCK_ENTRIES;
+
+		if (n > LINEINFO_BLOCK_ENTRIES)
+			n = LINEINFO_BLOCK_ENTRIES;
+		printf(".Lblock%s_%u:\n", suffix, i);
+		emit_block_data_range(base, n);
+	}
+	printf(".Ldata_end%s:\n", suffix);
 
-	/* file_offsets[] */
 	printf("\t.balign 4\n");
+	printf(".Lfile_offsets%s:\n", suffix);
 	for (unsigned int i = 0; i < num_files; i++)
 		printf("\t.long %u\n", files[i]->str_offset);
+	printf(".Lfile_offsets_end%s:\n\n", suffix);
 
-	/* filenames[] */
 	printf(".Lfilenames%s:\n", suffix);
 	for (unsigned int i = 0; i < num_files; i++)
 		print_escaped_asciz(files[i]->name);
@@ -2442,7 +2507,11 @@ int main(int argc, char *argv[])
 		deduplicate(NULL, 0);
 		compute_file_offsets();
 
-		verbose_msg("%u entries, %u files", num_entries, num_files);
+		verbose_msg("%u entries, %u files, %u blocks",
+			    num_entries, num_files,
+			    num_entries ?
+			    (num_entries + LINEINFO_BLOCK_ENTRIES - 1) /
+			    LINEINFO_BLOCK_ENTRIES : 0);
 
 		output_assembly();
 	}
-- 
2.53.0


  parent reply	other threads:[~2026-09-17 13:37 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-17 13:37 [PATCH v9 0/4] kallsyms: embed source file:line info in kernel stack traces Sasha Levin
2026-09-17 13:37 ` [PATCH v9 1/4] " Sasha Levin
2026-09-17 13:37 ` [PATCH v9 2/4] kallsyms: extend lineinfo to loadable modules Sasha Levin
2026-09-17 13:37 ` Sasha Levin [this message]
2026-09-17 13:37 ` [PATCH v9 4/4] kallsyms: add KUnit tests for lineinfo feature Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260917133727.428546-4-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=James.Bottomley@HansenPartnership.com \
    --cc=akpm@linux-foundation.org \
    --cc=corbet@lwn.net \
    --cc=da.gomez@kernel.org \
    --cc=deller@gmx.de \
    --cc=geert@linux-m68k.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=jgross@suse.com \
    --cc=kees@kernel.org \
    --cc=laurent.pinchart@ideasonboard.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-modules@vger.kernel.org \
    --cc=linux@leemhuis.info \
    --cc=masahiroy@kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=nathan@kernel.org \
    --cc=nsc@kernel.org \
    --cc=peterz@infradead.org \
    --cc=petr.pavlu@suse.com \
    --cc=pmladek@suse.com \
    --cc=rdunlap@infradead.org \
    --cc=richard@nod.at \
    --cc=rostedt@goodmis.org \
    --cc=samitolvanen@google.com \
    --cc=thunder.leizhen@huawei.com \
    --cc=torvalds@linux-foundation.org \
    --cc=vbabka@kernel.org \
    --cc=wangruikang@iscas.ac.cn \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®