[PATCH bpf-next v5 0/3] bpftool: Make the LLVM disassembler an optional runtime dependency

[PATCH bpf-next v5 0/3] bpftool: Make the LLVM disassembler an optional runtime dependency

[Mark Korenberg via B4 Relay]

libLLVM (~140 MiB on Fedora) is a hard dependency of bpftool, although it
is only used by "bpftool prog dump jited". With this series, bpftool is
no longer linked with libLLVM.so but loads it with dlopen() when it needs
to disassemble JITed programs, so that distributions can make it optional.
Patch 1 fixes a leak found on the way. Patch 2 makes "prog dump jited
... file FILE" work without a disassembler.

Changes in v5:
- Use the soname of libLLVM for dlopen(), not the name of the file it
  points to, which is not in the ld.so path on Ubuntu 24.04 (Sashiko,
  Alexei)
- New patch 2: call disasm_init() only when the image is disassembled,
  so that "prog dump jited ... file FILE" works without libLLVM (Alexei)
- Update common_options.rst, which still said that bpftool links with
  LLVM (Alexei)
- Rebase on bpf-next
- Link to v4: https://lore.kernel.org/r/20261102-bpftool-plugin-v4-0-5a853e973db9@gmail.com

Changes in v4:
- Drop the plugin, load libLLVM.so with dlopen() instead, as suggested by
  Alexei. This also makes "prog dump jited" work with the bpftool built by
  selftests.
- Keep "llvm" in "bpftool version" a build time feature (Alexei)
- Unlike the suggested diff, still link with LLVM built with
  BUILD_SHARED_LIBS=ON, rather than failing to build
- Link to v3: https://lore.kernel.org/r/20261029-bpftool-plugin-v3-0-87af799da40a@gmail.com

Changes in v3:
- Look for the plugin next to the bpftool binary before its install
  directory, so that bpftool run from its build directory does not pick
  an older installed plugin (Sashiko)
- Link to v2: https://lore.kernel.org/r/20261029-bpftool-plugin-v2-0-1a47b2391b0b@gmail.com

Changes in v2:
- Move the LLVMDisasmDispose() fix to a separate patch (Quentin)
- Look for the plugin next to the bpftool binary when it is not installed,
  so that bpftool works from its build directory (Quentin, Sashiko). Not in
  the current directory: bpftool usually runs as root.
- Export a single versioned structure of callbacks from the plugin (Quentin)
- Only report "llvm" in "bpftool version" if the plugin loads (Quentin)
- Handle dlerror() returning NULL (Quentin)
- Rebuild jit_disasm.o if libdir changes between "make" and
  "make install" (Quentin)
- uninstall: use QUIET_UNINST, remove $(libdir)/bpftool (Quentin)
- With a static libLLVM, which a static bpftool requires, link the
  disassembler into bpftool as before (Sashiko). This drops
  LLVM_LINK_STATIC: static LLVM archives are never linked into a shared
  object.
- Commit log and comment fixes (Quentin, Sashiko)
- Rebase on bpf-next
- Link to v1: https://lore.kernel.org/r/20260603-bpftool-plugin-v1-1-68c0ac91f5a3@gmail.com

Tested on Fedora 44 x86_64, LLVM 22: builds with shared, static and
per-component LLVM, and with SKIP_LLVM; "prog dump jited" (plain, JSON
and to a file) on live programs; bpftool, also when installed as
selftests do, loads libLLVM.so only for "prog dump jited" and reports an
error if it is missing; "prog dump jited ... file" works without a
disassembler. Also tested with Ubuntu 24.04 (LLVM 18) userspace, where
bpftool now loads libLLVM.so.18.1 (v4 tried libLLVM.so.1).

The changes since v1 were written with the help of an AI assistant
(Claude), from the review comments; I reviewed and tested them.

---
Mark Korenberg (3):
      bpftool: Fix disposal of the LLVM disassembler context
      bpftool: Don't require a disassembler to dump a JITed program to a file
      bpftool: Make the LLVM disassembler an optional runtime dependency

 tools/bpf/bpftool/Documentation/common_options.rst | 10 ++-
 tools/bpf/bpftool/Makefile                         | 16 +++-
 tools/bpf/bpftool/jit_disasm.c                     | 88 +++++++++++++++++++++-
 tools/bpf/bpftool/prog.c                           |  4 +-
 4 files changed, 110 insertions(+), 8 deletions(-)
---
base-commit: cd5bc55288d4fd999595c8f2af6a5408cbe92a52
change-id: 20260603-bpftool-plugin-c994bc3e0643

Best regards,
-- 
Mark Korenberg <socketpair@gmail.com>

[PATCH bpf-next v5 1/3] bpftool: Fix disposal of the LLVM disassembler context

[Mark Korenberg via B4 Relay]

From: Mark Korenberg <socketpair@gmail.com>

destroy_context() frees the disassembler context with LLVMDisposeMessage(),
which is a plain free() meant for strings returned by LLVM. The context is
therefore never destroyed, leaking about 28 KiB per disassembled function.
Use LLVMDisasmDispose() instead.

Fixes: eb9d1acf634b ("bpftool: Add LLVM as default library for disassembling JIT-ed programs")
Signed-off-by: Mark Korenberg <socketpair@gmail.com>
---
 tools/bpf/bpftool/jit_disasm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
index 04541155e9ccc..5ff144ce84a1c 100644
--- a/tools/bpf/bpftool/jit_disasm.c
+++ b/tools/bpf/bpftool/jit_disasm.c
@@ -115,7 +115,7 @@ init_context(disasm_ctx_t *ctx, const char *arch,
 
 static void destroy_context(disasm_ctx_t *ctx)
 {
-	LLVMDisposeMessage(*ctx);
+	LLVMDisasmDispose(*ctx);
 }
 
 static int

-- 
2.56.0

[PATCH bpf-next v5 2/3] bpftool: Don't require a disassembler to dump a JITed program to a file

[Mark Korenberg via B4 Relay]

From: Mark Korenberg <socketpair@gmail.com>

"bpftool prog dump jited PROG file FILE" only writes the JITed image to
FILE, but do_dump() calls disasm_init() before it parses the arguments,
so the command fails when bpftool has no disassembler.

Call disasm_init() only when the image is disassembled.

Fixes: 29a9c10e4110 ("bpftool: make libbfd optional")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Mark Korenberg <socketpair@gmail.com>
---
 tools/bpf/bpftool/prog.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 43599c79c93ac..15efc6eebffb1 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -894,8 +894,6 @@ static int do_dump(int argc, char **argv)
 	int err = -1;
 
 	if (is_prefix(*argv, "jited")) {
-		if (disasm_init())
-			return -1;
 		mode = DUMP_JITED;
 	} else if (is_prefix(*argv, "xlated")) {
 		mode = DUMP_XLATED;
@@ -959,6 +957,8 @@ static int do_dump(int argc, char **argv)
 		p_err("'visual' is not compatible with JSON output");
 		goto exit_close;
 	}
+	if (mode == DUMP_JITED && !filepath && disasm_init())
+		goto exit_close;
 
 	if (json_output && nb_fds > 1)
 		jsonw_start_array(json_wtr);	/* root array */

-- 
2.56.0

[PATCH bpf-next v5 3/3] bpftool: Make the LLVM disassembler an optional runtime dependency

[Mark Korenberg via B4 Relay]

From: Mark Korenberg <socketpair@gmail.com>

libLLVM is only needed for "bpftool prog dump jited", yet it is a hard
dependency of bpftool: on Fedora 43, installing the ~730 KiB bpftool pulls
in ~140 MiB of llvm20-libs.

When libLLVM is a shared library, don't link bpftool with it. Instead,
disasm_init() loads it with dlopen(), under its soname read at build
time, and resolves the few llvm-c functions bpftool uses with dlsym(),
like perf does for libcapstone. Distributions can then make LLVM an
optional dependency of bpftool.

"bpftool version" still reports "llvm" as a build time feature. Static
LLVM, and LLVM built with one library per component, are linked as before.

Link: https://github.com/libbpf/bpftool/issues/262
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Mark Korenberg <socketpair@gmail.com>
---
 tools/bpf/bpftool/Documentation/common_options.rst | 10 ++-
 tools/bpf/bpftool/Makefile                         | 16 +++-
 tools/bpf/bpftool/jit_disasm.c                     | 86 ++++++++++++++++++++++
 3 files changed, 107 insertions(+), 5 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/common_options.rst b/tools/bpf/bpftool/Documentation/common_options.rst
index 9234b9dab7682..b00fff6195e51 100644
--- a/tools/bpf/bpftool/Documentation/common_options.rst
+++ b/tools/bpf/bpftool/Documentation/common_options.rst
@@ -6,10 +6,12 @@
 -V, --version
     Print bpftool's version number (similar to **bpftool version**), the number
     of the libbpf version in use, and optional features that were included when
-    bpftool was compiled. Optional features include linking against LLVM or
-    libbfd to provide the disassembler for JIT-ted programs (**bpftool prog
-    dump jited**) and usage of BPF skeletons (some features like **bpftool prog
-    profile** or showing pids associated to BPF objects may rely on it).
+    bpftool was compiled. Optional features include LLVM or libbfd to provide
+    the disassembler for JIT-ted programs (**bpftool prog dump jited**) and
+    usage of BPF skeletons (some features like **bpftool prog profile** or
+    showing pids associated to BPF objects may rely on it). A shared libLLVM
+    is only loaded to disassemble JIT-ted programs, so "llvm" is listed even
+    if it is not installed.
 
 -j, --json
     Generate JSON output. For commands that cannot produce JSON, this option
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index b0f7168e79432..49dabf2eded04 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -186,10 +186,24 @@ ifeq ($(HAS_LLVM),1)
   # Makefile includes lib.mk which sets -D_GNU_SOURCE) which would cause
   # compilation error due to redefinition. Let's filter it out here.
   CFLAGS  += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
-  LIBS    += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
+  LIBLLVM_SO := $(shell $(LLVM_CONFIG) --libfiles $(LLVM_CONFIG_LIB_COMPONENTS))
   ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static)
+    LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
     LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
     LIBS += -lstdc++
+  else ifeq ($(words $(LIBLLVM_SO)),1)
+    # Don't link with libLLVM.so. "prog dump jited" loads it with dlopen(),
+    # under the name the linker would put in DT_NEEDED.
+    LIBLLVM_SONAME := $(shell LC_ALL=C $(READELF) -d $(LIBLLVM_SO) | sed -nE \
+	's/.*\(SONAME\).*\[(.*)\]/\1/p')
+    ifeq ($(LIBLLVM_SONAME),)
+      $(error Cannot read the soname of $(LIBLLVM_SO))
+    endif
+    CFLAGS += -DLIBLLVM_SO='"$(LIBLLVM_SONAME)"'
+    LIBS += -ldl
+  else
+    # LLVM built with BUILD_SHARED_LIBS=ON, one library per component
+    LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS))
   endif
   LDFLAGS += $(shell $(LLVM_CONFIG) --ldflags)
 else
diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
index 5ff144ce84a1c..2c646626e2560 100644
--- a/tools/bpf/bpftool/jit_disasm.c
+++ b/tools/bpf/bpftool/jit_disasm.c
@@ -25,6 +25,7 @@
 #include <bpf/libbpf.h>
 
 #ifdef HAVE_LLVM_SUPPORT
+#include <dlfcn.h>
 #include <llvm-c/Core.h>
 #include <llvm-c/Disassembler.h>
 #include <llvm-c/Target.h>
@@ -47,6 +48,89 @@ static int oper_count;
 
 typedef LLVMDisasmContextRef disasm_ctx_t;
 
+#ifdef LIBLLVM_SO
+/*
+ * libLLVM.so is big and only "prog dump jited" needs it. bpftool is not linked
+ * with it. disasm_init() loads it.
+ */
+#define LLVM_FUNCS(F)			\
+	F(LLVMCreateDisasm)		\
+	F(LLVMCreateDisasmCPUFeatures)	\
+	F(LLVMDisasmDispose)		\
+	F(LLVMDisasmInstruction)	\
+	F(LLVMDisposeMessage)		\
+	F(LLVMGetDefaultTargetTriple)	\
+	F(LLVMNormalizeTargetTriple)
+
+#define F(fn) static __typeof__(&fn) p_##fn;
+LLVM_FUNCS(F)
+#undef F
+
+static const struct {
+	const char *name;
+	void *ptr;
+} llvm_funcs[] = {
+#define F(fn) { #fn, &p_##fn },
+	LLVM_FUNCS(F)
+#undef F
+};
+
+/* The functions that LLVMInitializeAll*() in llvm-c/Target.h call */
+static const char * const llvm_init_funcs[] = {
+#define LLVM_TARGET(t)	"LLVMInitialize" #t "TargetInfo",	\
+			"LLVMInitialize" #t "TargetMC",
+#include <llvm/Config/Targets.def>
+#define LLVM_DISASSEMBLER(t) "LLVMInitialize" #t "Disassembler",
+#include <llvm/Config/Disassemblers.def>
+};
+
+#define LLVMCreateDisasm		p_LLVMCreateDisasm
+#define LLVMCreateDisasmCPUFeatures	p_LLVMCreateDisasmCPUFeatures
+#define LLVMDisasmDispose		p_LLVMDisasmDispose
+#define LLVMDisasmInstruction		p_LLVMDisasmInstruction
+#define LLVMDisposeMessage		p_LLVMDisposeMessage
+#define LLVMGetDefaultTargetTriple	p_LLVMGetDefaultTargetTriple
+#define LLVMNormalizeTargetTriple	p_LLVMNormalizeTargetTriple
+
+static void *llvm_sym(void *lib, const char *name)
+{
+	void *sym = dlsym(lib, name);
+
+	if (!sym)
+		p_err("%s is not in %s", name, LIBLLVM_SO);
+	return sym;
+}
+
+int disasm_init(void)
+{
+	void (*init)(void);
+	void *lib, *sym;
+	size_t i;
+
+	lib = dlopen(LIBLLVM_SO, RTLD_NOW);
+	if (!lib) {
+		p_err("failed to load %s: %s", LIBLLVM_SO,
+		      dlerror() ?: "unknown error");
+		return -1;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(llvm_funcs); i++) {
+		sym = llvm_sym(lib, llvm_funcs[i].name);
+		if (!sym)
+			return -1;
+		*(void **)llvm_funcs[i].ptr = sym;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(llvm_init_funcs); i++) {
+		init = llvm_sym(lib, llvm_init_funcs[i]);
+		if (!init)
+			return -1;
+		init();
+	}
+	return 0;
+}
+#endif /* LIBLLVM_SO */
+
 static int printf_json(char *s)
 {
 	s = strtok(s, " \t");
@@ -135,6 +219,7 @@ disassemble_insn(disasm_ctx_t *ctx, unsigned char *image, ssize_t len, int pc,
 	return count;
 }
 
+#ifndef LIBLLVM_SO
 int disasm_init(void)
 {
 	LLVMInitializeAllTargetInfos();
@@ -142,6 +227,7 @@ int disasm_init(void)
 	LLVMInitializeAllDisassemblers();
 	return 0;
 }
+#endif
 #endif /* HAVE_LLVM_SUPPORT */
 
 #ifdef HAVE_LIBBFD_SUPPORT

-- 
2.56.0