[PATCH RFC bpf-next 0/7] bpf: Support struct_ops in light skeletons

[PATCH RFC bpf-next 0/7] bpf: Support struct_ops in light skeletons

[Siddharth Nayyar]

BPF light skeletons (bpftool gen skeleton -L) compile BPF object loading
into a self-contained BPF_PROG_TYPE_SYSCALL loader program (gen_loader)
embedded in a C header, removing any runtime dependency on libbpf or
userspace BTF parsing. Currently, light skeletons do not support
BPF_MAP_TYPE_STRUCT_OPS maps or BPF_PROG_TYPE_STRUCT_OPS programs
because preparing a struct_ops map normally relies on libbpf parsing the
target vmlinux or kernel module BTF in userspace to resolve type IDs,
struct sizes, member byte offsets, and member indices.

This series adds BPF_MAP_TYPE_STRUCT_OPS support for link-based
(SEC(".struct_ops.link")) maps in light skeletons across both vmlinux
and kernel module BTF targets:

- Patch 1 adds a new bpf_btf_get_struct_member_info() helper for
  BPF_PROG_TYPE_SYSCALL programs to query a struct's byte size (when the
  member name is NULL) or a named member's byte offset and member index
  from vmlinux or module BTF at load time.
- Patches 2-5 wire up struct_ops initialization, map creation, program
  loading, and map update in libbpf's gen_loader. At runtime, the loader
  program resolves the kernel wrapper type bpf_struct_ops_<name> and
  target <name> struct BTF IDs (and module BTF FDs) via
  bpf_btf_find_by_name_kind(), queries struct sizes and member
  offsets/indices via bpf_btf_get_struct_member_info(), constructs the
  kernel vdata buffer in the loader data blob, passes the resolved BTF
  metadata to BPF_MAP_CREATE and BPF_PROG_LOAD, and populates the
  callback program FDs before issuing BPF_MAP_UPDATE_ELEM.
- Patch 6 teaches bpftool to generate per-map skel_link_create() attach
  functions and skel__attach() / skel__detach() plumbing for BPF_F_LINK
  struct_ops maps in light skeletons, skips gen_st_ops_shadow() so the
  light skeleton ctx layout is preserved, and rejects non-link
  struct_ops maps.
- Patch 7 adds light skeleton struct_ops selftests covering both vmlinux
  (tcp_congestion_ops) and bpf_testmod (bpf_testmod_ops, including a
  subset struct with reordered members, scalar fields, detach, and
  re-attachment).

Adding a new BPF helper and alternative kfunc approach
======================================================
I am aware that BPF helpers have been frozen for several years in favor
of kfuncs. I propose adding bpf_btf_get_struct_member_info() as a BPF
helper (restricted exclusively to BPF_PROG_TYPE_SYSCALL loader programs,
alongside bpf_btf_find_by_name_kind()) because, to my knowledge, there
is no better way to enable struct_ops support in light skeletons.

Before settling on a BPF helper, I explored an alternative approach
using a kfunc, which hit a roadblock. Calling a kfunc directly from the
light skeleton loader program is not possible because kfunc calls
(BPF_PSEUDO_KFUNC_CALL) require the target kfunc's BTF ID in insn->imm
when the loader program itself is loaded, whereas light skeletons ship
pre-generated loader bytecode without userspace BTF resolution. Although
I was able to resolve a kfunc's BTF ID at runtime via
bpf_btf_find_by_name_kind() and patch and load a separate BPF program to
run the kfunc and gather struct_ops member info, running that program
from the light skeleton loader required re-enabling BPF_PROG_TEST_RUN in
bpf_sys_bpf(). Calling BPF_PROG_TEST_RUN from BPF programs was
explicitly disabled in commit 86f44fcec22c ("bpf: Disallow bpf programs
call prog_run command.") because the verifier cannot perform sufficient
validation of the bpf_attr->test.ctx_in pointer.

Non-link struct_ops support
===========================
I have not enabled non-link struct_ops programs/maps (SEC(".struct_ops")
without BPF_F_LINK) in this series, though I intend to enable them in
the future and can extend this series to support them if maintainers
deem it necessary.

I left non-link struct_ops out for now because in light skeletons,
BPF_MAP_UPDATE_ELEM runs inside the loader program during skel__load().
For non-link struct_ops maps, BPF_MAP_UPDATE_ELEM immediately registers
and activates the struct_ops callbacks in the kernel before
skel__attach() is called, prevents re-attaching after detach, and would
leak an active registration if a later step in the loader program fails
(since loader cleanup only closes FDs via bpf_sys_close() rather than
issuing BPF_MAP_DELETE_ELEM). Supporting non-link struct_ops will
require deferring BPF_MAP_UPDATE_ELEM to userspace skel__attach().

AI assistance and testing
=========================
This series (code, selftests, commit messages, and cover letter) was
developed interactively with an AI coding assistant (Google Gemini),
with me guiding the design step by step across each patch and reviewing
all changes.

The series was built with clang/LLVM and tested in a QEMU x86_64 VM
using test_progs (covering the new struct_ops_lskel selftests for both
vmlinux tcp_congestion_ops and bpf_testmod_ops, as well as existing
struct_ops and light skeleton tests), along with manual verification of
loading and inspecting a struct_ops light skeleton via bpftool.

Feedback on the overall approach and on introducing this
BPF_PROG_TYPE_SYSCALL helper is very welcome.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
Siddharth Nayyar (7):
      bpf: Add bpf_btf_get_struct_member_info() helper
      libbpf: Implement bpf_gen__init_kern_struct_ops() for light skeletons
      libbpf: Support BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create()
      libbpf: Support BPF_PROG_TYPE_STRUCT_OPS in bpf_gen__prog_load()
      libbpf: Implement bpf_gen__struct_ops_update_elem() for light skeletons
      bpftool: Generate attach and detach for struct_ops in light skeletons
      selftests/bpf: Add light skeleton struct_ops tests

 include/linux/bpf.h                                |   1 +
 include/uapi/linux/bpf.h                           |  13 +
 kernel/bpf/btf.c                                   |  73 +++++
 kernel/bpf/syscall.c                               |   2 +
 tools/bpf/bpftool/gen.c                            |  71 ++++-
 tools/include/uapi/linux/bpf.h                     |  13 +
 tools/lib/bpf/bpf_gen_internal.h                   |   7 +
 tools/lib/bpf/gen_loader.c                         | 297 ++++++++++++++++++++-
 tools/lib/bpf/libbpf.c                             |  59 ++--
 tools/lib/bpf/libbpf_internal.h                    |  26 ++
 tools/testing/selftests/bpf/Makefile.skel          |   3 +-
 .../selftests/bpf/prog_tests/struct_ops_lskel.c    | 122 +++++++++
 tools/testing/selftests/bpf/progs/struct_ops_mod.c |  51 ++++
 .../selftests/bpf/progs/struct_ops_vmlinux.c       |  42 +++
 14 files changed, 747 insertions(+), 33 deletions(-)
---
base-commit: 3e18218484126afb9da722080659c038cb2256ab
change-id: 20261102-lskel-struct-ops-03090424cfa2

Best regards,
-- 
Siddharth Nayyar <sidnayyar@google.com>

[PATCH RFC bpf-next 1/7] bpf: Add bpf_btf_get_struct_member_info() helper

[Siddharth Nayyar]

Add a new BPF helper bpf_btf_get_struct_member_info() for
BPF_PROG_TYPE_SYSCALL programs to query struct member offset and index
(or the struct size when the member name is NULL) from vmlinux or
module BTF.

Given a struct's BTF type ID and an optional BTF object FD (0 for
vmlinux BTF), the helper returns a 64-bit value with the member's byte
offset in the lower 32 bits and the member's index in the upper 32 bits
when a member name is provided, or the byte size of the struct when the
member name is NULL.

This helper will be used by BPF light skeletons (gen_loader) to
initialize struct_ops maps and programs against kernel and module BTF
at load time.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 include/linux/bpf.h            |  1 +
 include/uapi/linux/bpf.h       | 13 ++++++++
 kernel/bpf/btf.c               | 73 ++++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/syscall.c           |  2 ++
 tools/include/uapi/linux/bpf.h | 13 ++++++++
 5 files changed, 102 insertions(+)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index b011acc22c50..ab4d995b529c 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -4206,6 +4206,7 @@ extern const struct bpf_func_proto bpf_task_storage_delete_recur_proto;
 extern const struct bpf_func_proto bpf_task_storage_delete_proto;
 extern const struct bpf_func_proto bpf_for_each_map_elem_proto;
 extern const struct bpf_func_proto bpf_btf_find_by_name_kind_proto;
+extern const struct bpf_func_proto bpf_btf_get_struct_member_info_proto;
 extern const struct bpf_func_proto bpf_sk_setsockopt_proto;
 extern const struct bpf_func_proto bpf_sk_getsockopt_proto;
 extern const struct bpf_func_proto bpf_sk_setsockopt_nodelay_proto;
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 62de0919db72..3a630411b920 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -6065,6 +6065,18 @@ union bpf_attr {
  *		0 on success.
  *
  *		**-ENOENT** if the bpf_local_storage cannot be found.
+ *
+ * long bpf_btf_get_struct_member_info(u32 btf_id, int btf_obj_fd, const char *name, int name_sz)
+ *	Description
+ *		Find a struct member with given *name* in the struct type identified
+ *		by *btf_id* in the BTF object referenced by *btf_obj_fd* (or in
+ *		vmlinux BTF if *btf_obj_fd* is 0). If *name* is **NULL** (and
+ *		*name_sz* is 0), return the size of the struct instead.
+ *	Return
+ *		Returns the member's byte offset in the lower 32 bits and the
+ *		member's index in the upper 32 bits when *name* is non-**NULL**,
+ *		or the byte size of the struct when *name* is **NULL**. On error, a
+ *		negative error code is returned.
  */
 #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
 	FN(unspec, 0, ##ctx)				\
@@ -6279,6 +6291,7 @@ union bpf_attr {
 	FN(user_ringbuf_drain, 209, ##ctx)		\
 	FN(cgrp_storage_get, 210, ##ctx)		\
 	FN(cgrp_storage_delete, 211, ##ctx)		\
+	FN(btf_get_struct_member_info, 212, ##ctx)	\
 	/* This helper list is effectively frozen. If you are trying to	\
 	 * add a new helper, you should add a kfunc instead which has	\
 	 * less stability guarantees. See Documentation/bpf/kfuncs.rst	\
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 228b93d4f8c9..8d9188dda34e 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9682,6 +9682,79 @@ const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = {
 	.arg4_type	= ARG_ANYTHING,
 };
 
+BPF_CALL_4(bpf_btf_get_struct_member_info, u32, btf_id, int, btf_obj_fd,
+	   const char *, name, int, name_sz)
+{
+	const struct btf_member *member;
+	const struct btf_type *t;
+	struct btf *btf;
+	long ret = -EINVAL;
+	u32 i;
+
+	if (!name) {
+		if (name_sz)
+			return -EINVAL;
+	} else if (name_sz <= 1 || !*name || name[name_sz - 1]) {
+		return -EINVAL;
+	}
+
+	if (btf_obj_fd) {
+		btf = btf_get_by_fd(btf_obj_fd);
+		if (IS_ERR(btf))
+			return PTR_ERR(btf);
+	} else {
+		btf = bpf_get_btf_vmlinux();
+		if (IS_ERR(btf))
+			return PTR_ERR(btf);
+		if (!btf)
+			return -EINVAL;
+		btf_get(btf);
+	}
+
+	t = btf_type_by_id(btf, btf_id);
+	if (!t || !__btf_type_is_struct(t))
+		goto out;
+
+	if (!name) {
+		ret = t->size;
+		goto out;
+	}
+
+	ret = -ENOENT;
+	for_each_member(i, t, member) {
+		const char *mname = btf_name_by_offset(btf, member->name_off);
+
+		if (!mname || !*mname)
+			continue;
+		if (!strcmp(mname, name)) {
+			u32 moff;
+
+			if (__btf_member_bitfield_size(t, member)) {
+				ret = -EOPNOTSUPP;
+				goto out;
+			}
+			moff = __btf_member_bit_offset(t, member) / 8;
+			ret = moff | (((u64)i) << 32);
+			goto out;
+		}
+	}
+
+out:
+	btf_put(btf);
+	return ret;
+}
+
+const struct bpf_func_proto bpf_btf_get_struct_member_info_proto = {
+	.func		= bpf_btf_get_struct_member_info,
+	.gpl_only	= false,
+	.might_sleep	= true,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_ANYTHING,
+	.arg2_type	= ARG_ANYTHING,
+	.arg3_type	= ARG_PTR_TO_MEM | MEM_RDONLY | PTR_MAYBE_NULL,
+	.arg4_type	= ARG_MEM_SIZE_OR_ZERO,
+};
+
 BTF_ID_LIST_GLOBAL(btf_tracing_ids, MAX_BTF_TRACING_TYPE)
 #define BTF_TRACING_TYPE(name, type) BTF_ID(struct, type)
 BTF_TRACING_TYPE_xxx
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 9004f2673ccf..6e7b6b10603e 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -6756,6 +6756,8 @@ syscall_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		       ? NULL : &bpf_sys_bpf_proto;
 	case BPF_FUNC_btf_find_by_name_kind:
 		return &bpf_btf_find_by_name_kind_proto;
+	case BPF_FUNC_btf_get_struct_member_info:
+		return &bpf_btf_get_struct_member_info_proto;
 	case BPF_FUNC_sys_close:
 		return &bpf_sys_close_proto;
 	case BPF_FUNC_kallsyms_lookup_name:
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 62de0919db72..3a630411b920 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -6065,6 +6065,18 @@ union bpf_attr {
  *		0 on success.
  *
  *		**-ENOENT** if the bpf_local_storage cannot be found.
+ *
+ * long bpf_btf_get_struct_member_info(u32 btf_id, int btf_obj_fd, const char *name, int name_sz)
+ *	Description
+ *		Find a struct member with given *name* in the struct type identified
+ *		by *btf_id* in the BTF object referenced by *btf_obj_fd* (or in
+ *		vmlinux BTF if *btf_obj_fd* is 0). If *name* is **NULL** (and
+ *		*name_sz* is 0), return the size of the struct instead.
+ *	Return
+ *		Returns the member's byte offset in the lower 32 bits and the
+ *		member's index in the upper 32 bits when *name* is non-**NULL**,
+ *		or the byte size of the struct when *name* is **NULL**. On error, a
+ *		negative error code is returned.
  */
 #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
 	FN(unspec, 0, ##ctx)				\
@@ -6279,6 +6291,7 @@ union bpf_attr {
 	FN(user_ringbuf_drain, 209, ##ctx)		\
 	FN(cgrp_storage_get, 210, ##ctx)		\
 	FN(cgrp_storage_delete, 211, ##ctx)		\
+	FN(btf_get_struct_member_info, 212, ##ctx)	\
 	/* This helper list is effectively frozen. If you are trying to	\
 	 * add a new helper, you should add a kfunc instead which has	\
 	 * less stability guarantees. See Documentation/bpf/kfuncs.rst	\

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 2/7] libbpf: Implement bpf_gen__init_kern_struct_ops() for light skeletons

[Siddharth Nayyar]

Add bpf_gen__init_kern_struct_ops() in gen_loader.c and wire it up in
bpf_object__init_kern_struct_ops_maps() when generating a light
skeleton.

At runtime, the generated loader program:
1. Resolves the BTF ID and module BTF FD of the wrapper struct
   bpf_struct_ops_<tname> via bpf_btf_find_by_name_kind().
2. Queries the wrapper struct's size and "data" member offset using
   bpf_btf_get_struct_member_info().
3. Resolves the BTF ID and module BTF FD of the target interface struct
   <tname> for program attach_btf_id resolution.
4. Allocates a 4 KB kern_vdata buffer in the loader data blob and
   populates non-zero scalar/array fields and per-program member offsets
   and attach indices.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/lib/bpf/bpf_gen_internal.h |   4 +
 tools/lib/bpf/gen_loader.c       | 197 +++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c           |  43 ++++-----
 tools/lib/bpf/libbpf_internal.h  |  24 +++++
 4 files changed, 245 insertions(+), 23 deletions(-)

diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h
index 206adf28793d..b424410e8839 100644
--- a/tools/lib/bpf/bpf_gen_internal.h
+++ b/tools/lib/bpf/bpf_gen_internal.h
@@ -84,5 +84,9 @@ void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak,
 			    bool is_typeless, bool is_ld64, int kind, int insn_idx);
 void bpf_gen__record_relo_core(struct bpf_gen *gen, const struct bpf_core_relo *core_relo);
 void bpf_gen__populate_outer_map(struct bpf_gen *gen, int outer_map_idx, int key, int inner_map_idx);
+void bpf_gen__init_kern_struct_ops(struct bpf_gen *gen, struct btf *btf,
+				   struct bpf_struct_ops *st_ops,
+				   __u32 *vtype_id_off,
+				   unsigned int *val_sz_off);
 
 #endif
diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
index 555d4757a898..09713c39f71e 100644
--- a/tools/lib/bpf/gen_loader.c
+++ b/tools/lib/bpf/gen_loader.c
@@ -14,11 +14,14 @@
 #include "hashmap.h"
 #include "bpf_gen_internal.h"
 #include "skel_internal.h"
+#include "relo_core.h"
 
 #define MAX_USED_MAPS	64
 #define MAX_USED_PROGS	32
 #define MAX_KFUNC_DESCS 256
 #define MAX_FD_ARRAY_SZ (MAX_USED_MAPS + MAX_KFUNC_DESCS)
+#define STRUCT_OPS_VALUE_PREFIX "bpf_struct_ops_"
+#define KERN_VDATA_MAX_SIZE 4096
 
 /* The following structure describes the stack layout of the loader program.
  * In addition R6 contains the pointer to context.
@@ -1338,3 +1341,197 @@ void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx)
 	debug_ret(gen, "map_freeze");
 	emit_check_err(gen);
 }
+
+/* Expects:
+ * BPF_REG_8 - (btf_obj_fd << 32) | btf_id
+ *
+ * Overwrites BPF_REG_{0, 1, 2, 3, 4, 7}
+ * Returns result in BPF_REG_7
+ */
+static void emit_bpf_get_struct_member_info(struct bpf_gen *gen, const char *mname)
+{
+	emit(gen, BPF_MOV32_REG(BPF_REG_1, BPF_REG_8));
+	emit(gen, BPF_MOV64_REG(BPF_REG_2, BPF_REG_8));
+	emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_2, 32));
+	if (mname) {
+		int mname_len = strlen(mname) + 1;
+		int mname_off = add_data(gen, mname, mname_len);
+
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_3, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, mname_off));
+		emit(gen, BPF_MOV64_IMM(BPF_REG_4, mname_len));
+	} else {
+		emit(gen, BPF_MOV64_IMM(BPF_REG_3, 0));
+		emit(gen, BPF_MOV64_IMM(BPF_REG_4, 0));
+	}
+	emit(gen, BPF_EMIT_CALL(BPF_FUNC_btf_get_struct_member_info));
+	emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_0));
+	debug_ret(gen, "get_struct_member_info(%s)", mname ?: "NULL");
+	emit_check_err(gen);
+}
+
+/* Expects:
+ * BPF_REG_8 - (btf_obj_fd << 32) | kern_type_id
+ * BPF_REG_9 - kern_data_off
+ */
+static void emit_st_ops_member_val(struct bpf_gen *gen, const char *mname,
+				   const void *mdata, __u32 msize,
+				   int kern_vdata_off)
+{
+	int mdata_off;
+
+	if (libbpf_is_mem_zeroed(mdata, msize))
+		return;
+
+	if (msize > KERN_VDATA_MAX_SIZE) {
+		gen->error = -E2BIG;
+		return;
+	}
+
+	mdata_off = add_data(gen, mdata, msize);
+	emit_bpf_get_struct_member_info(gen, mname);
+
+	/* Extract member byte offset (lower 32 bits of R7), add kern_data_off, and bound it */
+	emit(gen, BPF_MOV32_REG(BPF_REG_2, BPF_REG_7));
+	emit(gen, BPF_ALU32_REG(BPF_ADD, BPF_REG_2, BPF_REG_9));
+	emit(gen, BPF_JMP_IMM(BPF_JLE, BPF_REG_2, KERN_VDATA_MAX_SIZE - msize, 2));
+	emit(gen, BPF_MOV64_IMM(BPF_REG_7, -E2BIG));
+	emit_check_err(gen);
+
+	/* Copy msize bytes from mdata_off into kern_vdata_off + offset */
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, kern_vdata_off));
+	emit(gen, BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2));
+	emit(gen, BPF_MOV64_IMM(BPF_REG_2, msize));
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_3, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, mdata_off));
+	emit(gen, BPF_EMIT_CALL(BPF_FUNC_probe_read_kernel));
+}
+
+/* Expects:
+ * BPF_REG_8 - (btf_obj_fd << 32) | kern_type_id
+ * BPF_REG_9 - kern_data_off
+ */
+static void emit_st_ops_member_prog(struct bpf_gen *gen, const char *mname,
+				    struct bpf_struct_ops *st_ops, __u32 member_idx,
+				    struct bpf_program *prog)
+{
+	int attach_type_off;
+
+	if (st_ops->progs[member_idx] && st_ops->progs[member_idx] != prog)
+		bpf_program__set_autoload(st_ops->progs[member_idx], false);
+	st_ops->progs[member_idx] = prog;
+
+	if (!prog)
+		return;
+
+	st_ops->kern_func_off[member_idx] = add_data(gen, NULL, sizeof(__u32));
+	attach_type_off = add_data(gen, NULL, sizeof(__u32));
+	bpf_program__set_expected_attach_type(prog, attach_type_off);
+
+	emit_bpf_get_struct_member_info(gen, mname);
+
+	/* 1. Store member byte offset (lower 32 bits of R7 + kern_data_off)
+	 * into st_ops->kern_func_off[member_idx]
+	 */
+	emit(gen, BPF_MOV32_REG(BPF_REG_2, BPF_REG_7));
+	emit(gen, BPF_ALU32_REG(BPF_ADD, BPF_REG_2, BPF_REG_9));
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, st_ops->kern_func_off[member_idx]));
+	emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_2, 0));
+
+	/* 2. Store member index (upper 32 bits of R7) into attach_type_off */
+	emit(gen, BPF_MOV64_REG(BPF_REG_2, BPF_REG_7));
+	emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_2, 32));
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, attach_type_off));
+	emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_2, 0));
+}
+
+void bpf_gen__init_kern_struct_ops(struct bpf_gen *gen, struct btf *btf,
+				   struct bpf_struct_ops *st_ops,
+				   __u32 *vtype_id_off,
+				   unsigned int *val_sz_off)
+{
+	struct ksym_relo_desc relo = { .kind = BTF_KIND_STRUCT };
+	const struct btf_member *member;
+	const struct btf_type *type;
+	char tname[192], stname[256];
+	const char *tname_raw;
+	__u32 i;
+
+	type = btf__type_by_id(btf, st_ops->type_id);
+	tname_raw = btf__name_by_offset(btf, type->name_off);
+	snprintf(tname, sizeof(tname), "%.*s",
+		 (int)bpf_core_essential_name_len(tname_raw), tname_raw);
+	snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);
+
+	/* 1. Resolve BTF ID of struct bpf_struct_ops_<tname> */
+	*vtype_id_off = add_data(gen, NULL, sizeof(__u64));
+	relo.name = stname;
+	emit_bpf_find_by_name_kind(gen, &relo);
+	emit_check_err(gen);
+	emit(gen, BPF_MOV64_REG(BPF_REG_8, BPF_REG_7));
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, *vtype_id_off));
+	emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_7, 0));
+
+	/* 2. Use the BTF ID to find the struct_ops value struct size and "data" offset */
+	*val_sz_off = add_data(gen, NULL, sizeof(__u32));
+	emit_bpf_get_struct_member_info(gen, NULL);
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, *val_sz_off));
+	emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_7, 0));
+
+	emit_bpf_get_struct_member_info(gen, "data");
+	emit(gen, BPF_MOV32_REG(BPF_REG_9, BPF_REG_7));
+
+	/* 3. Resolve BTF ID of struct <tname> */
+	st_ops->type_id_off = add_data(gen, NULL, sizeof(__u64));
+	relo.name = tname;
+	emit_bpf_find_by_name_kind(gen, &relo);
+	emit_check_err(gen);
+	emit(gen, BPF_MOV64_REG(BPF_REG_8, BPF_REG_7));
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, st_ops->type_id_off));
+	emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_7, 0));
+
+	/* 4. Allocate and populate space in the data blob for kern_vdata */
+	st_ops->kern_vdata_off = add_data(gen, NULL, KERN_VDATA_MAX_SIZE);
+	member = btf_members(type);
+	for (i = 0; i < btf_vlen(type); i++, member++) {
+		const struct btf_type *mtype;
+		const char *mname;
+		void *mdata;
+		__s64 msize;
+		__u32 moff;
+
+		mname = btf__name_by_offset(btf, member->name_off);
+		if (btf_member_bitfield_size(type, i)) {
+			pr_warn("struct_ops init_kern: bitfield %s is not supported\n",
+				mname);
+			gen->error = -ENOTSUP;
+			return;
+		}
+
+		moff = member->offset / 8;
+		mdata = st_ops->data + moff;
+		msize = btf__resolve_size(btf, member->type);
+		if (msize < 0) {
+			pr_warn("struct_ops init_kern: failed to resolve the size of member %s\n",
+				mname);
+			gen->error = msize;
+			return;
+		}
+
+		mtype = skip_mods_and_typedefs(btf, member->type, NULL);
+		if (btf_is_ptr(mtype)) {
+			struct bpf_program *prog = *(void **)mdata;
+
+			emit_st_ops_member_prog(gen, mname, st_ops, i, prog);
+		} else {
+			emit_st_ops_member_val(gen, mname, mdata, msize,
+					       st_ops->kern_vdata_off);
+		}
+	}
+}
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 0e2a93a42908..7754d6640167 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -523,25 +523,6 @@ struct bpf_program {
 	__u32 subprog_cnt;
 };
 
-struct bpf_struct_ops {
-	struct bpf_program **progs;
-	__u32 *kern_func_off;
-	/* e.g. struct tcp_congestion_ops in bpf_prog's btf format */
-	void *data;
-	/* e.g. struct bpf_struct_ops_tcp_congestion_ops in
-	 *      btf_vmlinux's format.
-	 * struct bpf_struct_ops_tcp_congestion_ops {
-	 *	[... some other kernel fields ...]
-	 *	struct tcp_congestion_ops data;
-	 * }
-	 * kern_vdata-size == sizeof(struct bpf_struct_ops_tcp_congestion_ops)
-	 * bpf_map__init_kern_struct_ops() will populate the "kern_vdata"
-	 * from "data".
-	 */
-	void *kern_vdata;
-	__u32 type_id;
-};
-
 #define DATA_SEC ".data"
 #define PERCPU_SEC ".percpu"
 #define BSS_SEC ".bss"
@@ -1462,7 +1443,7 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map)
 static int bpf_object__init_kern_struct_ops_maps(struct bpf_object *obj)
 {
 	struct bpf_map *map;
-	size_t i;
+	size_t i, j;
 	int err;
 
 	for (i = 0; i < obj->nr_maps; i++) {
@@ -1474,9 +1455,25 @@ static int bpf_object__init_kern_struct_ops_maps(struct bpf_object *obj)
 		if (!map->autocreate)
 			continue;
 
-		err = bpf_map__init_kern_struct_ops(map);
-		if (err)
-			return err;
+		if (obj->gen_loader) {
+			const struct btf_type *type;
+
+			bpf_gen__init_kern_struct_ops(obj->gen_loader, obj->btf,
+						      map->st_ops,
+						      &map->btf_vmlinux_value_type_id,
+						      &map->def.value_size);
+			map->mod_btf_fd = -1;
+			type = btf__type_by_id(obj->btf, map->st_ops->type_id);
+			for (j = 0; j < btf_vlen(type); j++) {
+				if (map->st_ops->progs[j])
+					map->st_ops->progs[j]->attach_btf_id =
+						map->st_ops->type_id_off;
+			}
+		} else {
+			err = bpf_map__init_kern_struct_ops(map);
+			if (err)
+				return err;
+		}
 	}
 
 	return 0;
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index e90b5336bac0..845d6557c922 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -293,6 +293,30 @@ struct btf_map_def {
 	__u64 map_extra;
 };
 
+struct bpf_struct_ops {
+	struct bpf_program **progs;
+	__u32 *kern_func_off;
+	/* e.g. struct tcp_congestion_ops in bpf_prog's btf format */
+	void *data;
+	/* e.g. struct bpf_struct_ops_tcp_congestion_ops in
+	 *      btf_vmlinux's format.
+	 * struct bpf_struct_ops_tcp_congestion_ops {
+	 *	[... some other kernel fields ...]
+	 *	struct tcp_congestion_ops data;
+	 * }
+	 * kern_vdata-size == sizeof(struct bpf_struct_ops_tcp_congestion_ops)
+	 * In standard mode, bpf_map__init_kern_struct_ops() will populate the
+	 * "kern_vdata" buffer from "data".
+	 * In gen_loader mode, bpf_gen__init_kern_struct_ops() will allocate
+	 * space for "kern_vdata" in the loader data blob and record its offset
+	 * in "kern_vdata_off".
+	 */
+	void *kern_vdata;
+	int kern_vdata_off;
+	int type_id_off;
+	__u32 type_id;
+};
+
 int parse_btf_map_def(const char *map_name, struct btf *btf,
 		      const struct btf_type *def_t, bool strict,
 		      struct btf_map_def *map_def, struct btf_map_def *inner_def);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 3/7] libbpf: Support BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create()

[Siddharth Nayyar]

Handle BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create() when generating
a light skeleton:
- Extend map_create_attr size to include value_type_btf_obj_fd.
- Populate attr.btf_fd from the loader stack when
  btf_vmlinux_value_type_id is set.
- Populate attr.value_size and attr.btf_vmlinux_value_type_id from the
  runtime-resolved values in the loader data blob.
- If the struct_ops type resides in a kernel module (btf_obj_fd > 0),
  populate attr.value_type_btf_obj_fd, set BPF_F_VTYPE_BTF_OBJ_FD in
  attr.map_flags, and close value_type_btf_obj_fd after BPF_MAP_CREATE.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/lib/bpf/gen_loader.c | 28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
index 09713c39f71e..6027a2762c35 100644
--- a/tools/lib/bpf/gen_loader.c
+++ b/tools/lib/bpf/gen_loader.c
@@ -508,7 +508,7 @@ void bpf_gen__map_create(struct bpf_gen *gen,
 			 __u32 key_size, __u32 value_size, __u32 max_entries,
 			 struct bpf_map_create_opts *map_attr, int map_idx)
 {
-	int attr_size = offsetofend(union bpf_attr, map_extra);
+	int attr_size = offsetofend(union bpf_attr, value_type_btf_obj_fd);
 	bool close_inner_map_fd = false;
 	int map_create_attr, idx;
 	union bpf_attr attr;
@@ -532,7 +532,7 @@ void bpf_gen__map_create(struct bpf_gen *gen,
 		 map_name, map_idx, map_type, map_attr->btf_value_type_id,
 		 map_create_attr, attr_size);
 
-	if (map_attr->btf_value_type_id)
+	if (map_attr->btf_value_type_id || map_attr->btf_vmlinux_value_type_id)
 		/* populate union bpf_attr with btf_fd saved in the stack earlier */
 		move_stack2blob(gen, attr_field(map_create_attr, btf_fd), 4,
 				stack_off(btf_fd));
@@ -543,6 +543,27 @@ void bpf_gen__map_create(struct bpf_gen *gen,
 				stack_off(inner_map_fd));
 		close_inner_map_fd = true;
 		break;
+	case BPF_MAP_TYPE_STRUCT_OPS:
+		move_blob2blob(gen, attr_field(map_create_attr, value_size), 4,
+			       value_size);
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, map_attr->btf_vmlinux_value_type_id));
+		emit(gen, BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0));
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, map_create_attr));
+		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				      offsetof(union bpf_attr, btf_vmlinux_value_type_id)));
+		emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 32));
+		emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 4));
+		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				      offsetof(union bpf_attr, value_type_btf_obj_fd)));
+		emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1,
+				      offsetof(union bpf_attr, map_flags)));
+		emit(gen, BPF_ALU32_IMM(BPF_OR, BPF_REG_0,
+					tgt_endian(BPF_F_VTYPE_BTF_OBJ_FD)));
+		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0,
+				      offsetof(union bpf_attr, map_flags)));
+		break;
 	default:
 		break;
 	}
@@ -568,6 +589,9 @@ void bpf_gen__map_create(struct bpf_gen *gen,
 	debug_ret(gen, "map_create %s idx %d type %d value_size %d value_btf_id %d",
 		  map_name, map_idx, map_type, value_size,
 		  map_attr->btf_value_type_id);
+	if (map_type == BPF_MAP_TYPE_STRUCT_OPS)
+		emit_sys_close_blob(gen,
+				    attr_field(map_create_attr, value_type_btf_obj_fd));
 	emit_check_err(gen);
 	/* remember map_fd in the stack, if successful */
 	if (map_idx < 0) {

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 4/7] libbpf: Support BPF_PROG_TYPE_STRUCT_OPS in bpf_gen__prog_load()

[Siddharth Nayyar]

For BPF_PROG_TYPE_STRUCT_OPS programs in light skeletons, load_attr's
expected_attach_type and attach_btf_id hold data blob offsets populated
at runtime by bpf_gen__init_kern_struct_ops().

Emit instructions in bpf_gen__prog_load() to copy the resolved member
index into prog_load_attr.expected_attach_type and unpack the 64-bit
(btf_obj_fd << 32) | kern_type_id value from load_attr->attach_btf_id
into prog_load_attr.attach_btf_id and prog_load_attr.attach_btf_obj_fd.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/lib/bpf/gen_loader.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
index 6027a2762c35..aaa491463521 100644
--- a/tools/lib/bpf/gen_loader.c
+++ b/tools/lib/bpf/gen_loader.c
@@ -1151,6 +1151,20 @@ void bpf_gen__prog_load(struct bpf_gen *gen,
 		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
 				      offsetof(union bpf_attr, attach_btf_obj_fd)));
 	}
+	if (prog_type == BPF_PROG_TYPE_STRUCT_OPS) {
+		move_blob2blob(gen, attr_field(prog_load_attr, expected_attach_type), 4,
+			       load_attr->expected_attach_type);
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, load_attr->attach_btf_id));
+		emit(gen, BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_1, 0));
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, prog_load_attr));
+		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
+				      offsetof(union bpf_attr, attach_btf_id)));
+		emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32));
+		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
+				      offsetof(union bpf_attr, attach_btf_obj_fd)));
+	}
 	emit_relos(gen, insns_off);
 	/* emit PROG_LOAD command */
 	emit_sys_bpf(gen, BPF_PROG_LOAD, prog_load_attr, attr_size);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 5/7] libbpf: Implement bpf_gen__struct_ops_update_elem() for light skeletons

[Siddharth Nayyar]

Add bpf_gen__struct_ops_update_elem() in gen_loader.c and call it from
bpf_object_prepare_struct_ops() when generating a light skeleton.

Once all programs are loaded, bpf_gen__struct_ops_update_elem():
1. Closes the attach_btf_obj_fd stored in the upper 32 bits of
   st_ops->type_id_off.
2. Copies each callback program's FD from the loader stack into
   st_ops->kern_vdata_off at the dynamic kernel member offset stored in
   st_ops->kern_func_off[i].
3. Emits BPF_MAP_UPDATE_ELEM to populate the struct_ops map with the
   prepared kern_vdata.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/lib/bpf/bpf_gen_internal.h |  3 +++
 tools/lib/bpf/gen_loader.c       | 58 ++++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.c           | 16 ++++++++---
 tools/lib/bpf/libbpf_internal.h  |  2 ++
 4 files changed, 76 insertions(+), 3 deletions(-)

diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h
index b424410e8839..cd603f82f6ec 100644
--- a/tools/lib/bpf/bpf_gen_internal.h
+++ b/tools/lib/bpf/bpf_gen_internal.h
@@ -88,5 +88,8 @@ void bpf_gen__init_kern_struct_ops(struct bpf_gen *gen, struct btf *btf,
 				   struct bpf_struct_ops *st_ops,
 				   __u32 *vtype_id_off,
 				   unsigned int *val_sz_off);
+void bpf_gen__struct_ops_update_elem(struct bpf_gen *gen, int map_idx,
+				     struct btf *btf,
+				     struct bpf_struct_ops *st_ops);
 
 #endif
diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
index aaa491463521..6dcba83dcc09 100644
--- a/tools/lib/bpf/gen_loader.c
+++ b/tools/lib/bpf/gen_loader.c
@@ -1573,3 +1573,61 @@ void bpf_gen__init_kern_struct_ops(struct bpf_gen *gen, struct btf *btf,
 		}
 	}
 }
+
+void bpf_gen__struct_ops_update_elem(struct bpf_gen *gen, int map_idx,
+				     struct btf *btf,
+				     struct bpf_struct_ops *st_ops)
+{
+	int attr_size = offsetofend(union bpf_attr, flags);
+	const struct btf_type *type;
+	union bpf_attr attr;
+	int key, zero = 0;
+	__u32 i;
+
+	/* 1. Close attach_btf_obj_fd (upper 32 bits of st_ops->type_id_off) */
+	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE,
+					 0, 0, 0, st_ops->type_id_off));
+	emit(gen, BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0));
+	emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_1, 32));
+	__emit_sys_close(gen);
+
+	/* 2. Populate prog_fd into kern_vdata for all callback members */
+	type = btf__type_by_id(btf, st_ops->type_id);
+	for (i = 0; i < btf_vlen(type); i++) {
+		struct bpf_program *prog = st_ops->progs[i];
+		int prog_idx, off_slot;
+
+		if (!prog)
+			continue;
+
+		prog_idx = bpf_program_idx(prog);
+		off_slot = st_ops->kern_func_off[i];
+
+		/* R2 = dynamic kernel member offset from off_slot */
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, off_slot));
+		emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0));
+		emit(gen, BPF_JMP_IMM(BPF_JLE, BPF_REG_2,
+				      KERN_VDATA_MAX_SIZE - sizeof(__u64), 2));
+		emit(gen, BPF_MOV64_IMM(BPF_REG_7, -E2BIG));
+		emit_check_err(gen);
+
+		/* R1 = kern_vdata_off + offset */
+		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
+						 0, 0, 0, st_ops->kern_vdata_off));
+		emit(gen, BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2));
+
+		/* R0 = prog_fd from stack (stack_off(prog_fd[prog_idx])) */
+		emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_10,
+				      stack_off(prog_fd[prog_idx])));
+
+		/* Store prog_fd (R0) into kern_vdata + offset (R1) */
+		emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0));
+	}
+
+	/* 3. Emit BPF_MAP_UPDATE_ELEM command for struct_ops map */
+	memset(&attr, 0, attr_size);
+	key = add_data(gen, &zero, sizeof(zero));
+	emit_map_update_elem(gen, map_idx, &attr, attr_size, key,
+			     st_ops->kern_vdata_off, KERN_VDATA_MAX_SIZE);
+}
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 7754d6640167..d00ff5f8b0b0 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -9772,9 +9772,14 @@ static int bpf_object_prepare_struct_ops(struct bpf_object *obj)
 		if (!map->autocreate)
 			continue;
 
-		err = bpf_map_prepare_vdata(map);
-		if (err)
-			return err;
+		if (obj->gen_loader) {
+			bpf_gen__struct_ops_update_elem(obj->gen_loader, i,
+							obj->btf, map->st_ops);
+		} else {
+			err = bpf_map_prepare_vdata(map);
+			if (err)
+				return err;
+		}
 	}
 
 	return 0;
@@ -10720,6 +10725,11 @@ int bpf_program__set_expected_attach_type(struct bpf_program *prog,
 	return 0;
 }
 
+int bpf_program_idx(const struct bpf_program *prog)
+{
+	return prog - prog->obj->programs;
+}
+
 __u32 bpf_program__flags(const struct bpf_program *prog)
 {
 	return prog->prog_flags;
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 845d6557c922..94c3a4019b11 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -317,6 +317,8 @@ struct bpf_struct_ops {
 	__u32 type_id;
 };
 
+int bpf_program_idx(const struct bpf_program *prog);
+
 int parse_btf_map_def(const char *map_name, struct btf *btf,
 		      const struct btf_type *def_t, bool strict,
 		      struct btf_map_def *map_def, struct btf_map_def *inner_def);

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 6/7] bpftool: Generate attach and detach for struct_ops in light skeletons

[Siddharth Nayyar]

Update codegen_attach_detach() to iterate over BPF_MAP_TYPE_STRUCT_OPS
maps with BPF_F_LINK set. Generate dedicated per-map attach functions
calling skel_link_create(), chain them in skel__attach(), and close the
corresponding link FDs with skel_closenz() in skel__detach().

Also skip gen_st_ops_shadow() when generating light skeletons so
skel->struct_ops is not emitted between skel->maps and skel->progs,
preserving the ctx layout expected by bpf_gen__finish().

Finally, reject struct_ops maps without BPF_F_LINK in light skeletons.
Because the loader program runs BPF_MAP_UPDATE_ELEM during skel__load(),
non-link struct_ops maps would register prematurely before
skel__attach() and leak in the kernel if a later load step fails.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/bpf/bpftool/gen.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 67 insertions(+), 4 deletions(-)

diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
index ee680f7c2116..98e92a34326b 100644
--- a/tools/bpf/bpftool/gen.c
+++ b/tools/bpf/bpftool/gen.c
@@ -569,6 +569,8 @@ static void codegen_asserts(struct bpf_object *obj, const char *obj_name)
 static void codegen_attach_detach(struct bpf_object *obj, const char *obj_name)
 {
 	struct bpf_program *prog;
+	struct bpf_map *map;
+	char ident[256];
 
 	bpf_object__for_each_program(prog, obj) {
 		const char *tp_name;
@@ -608,6 +610,30 @@ static void codegen_attach_detach(struct bpf_object *obj, const char *obj_name)
 			", bpf_program__name(prog));
 	}
 
+	bpf_object__for_each_map(map, obj) {
+		if (!get_map_ident(map, ident, sizeof(ident)))
+			continue;
+		if (bpf_map__type(map) != BPF_MAP_TYPE_STRUCT_OPS)
+			continue;
+		if (!(bpf_map__map_flags(map) & BPF_F_LINK))
+			continue;
+
+		codegen("\
+			\n\
+			\n\
+			static inline int					    \n\
+			%1$s__%2$s__attach(struct %1$s *skel)			    \n\
+			{							    \n\
+				int map_fd = skel->maps.%2$s.map_fd;		    \n\
+				int fd = skel_link_create(map_fd, 0, BPF_STRUCT_OPS);\n\
+										    \n\
+				if (fd > 0)					    \n\
+					skel->links.%2$s_fd = fd;		    \n\
+				return fd;					    \n\
+			}							    \n\
+			", obj_name, ident);
+	}
+
 	codegen("\
 		\n\
 									    \n\
@@ -625,6 +651,20 @@ static void codegen_attach_detach(struct bpf_object *obj, const char *obj_name)
 			", obj_name, bpf_program__name(prog));
 	}
 
+	bpf_object__for_each_map(map, obj) {
+		if (!get_map_ident(map, ident, sizeof(ident)))
+			continue;
+		if (bpf_map__type(map) != BPF_MAP_TYPE_STRUCT_OPS)
+			continue;
+		if (!(bpf_map__map_flags(map) & BPF_F_LINK))
+			continue;
+
+		codegen("\
+			\n\
+				ret = ret < 0 ? ret : %1$s__%2$s__attach(skel);   \n\
+			", obj_name, ident);
+	}
+
 	codegen("\
 		\n\
 			return ret < 0 ? ret : 0;			    \n\
@@ -642,6 +682,20 @@ static void codegen_attach_detach(struct bpf_object *obj, const char *obj_name)
 			", bpf_program__name(prog));
 	}
 
+	bpf_object__for_each_map(map, obj) {
+		if (!get_map_ident(map, ident, sizeof(ident)))
+			continue;
+		if (bpf_map__type(map) != BPF_MAP_TYPE_STRUCT_OPS)
+			continue;
+		if (!(bpf_map__map_flags(map) & BPF_F_LINK))
+			continue;
+
+		codegen("\
+			\n\
+				skel_closenz(skel->links.%1$s_fd);	    \n\
+			", ident);
+	}
+
 	codegen("\
 		\n\
 		}							    \n\
@@ -1323,8 +1377,15 @@ static int do_skeleton(int argc, char **argv)
 			continue;
 		}
 
-		if (bpf_map__type(map) == BPF_MAP_TYPE_STRUCT_OPS)
+		if (bpf_map__type(map) == BPF_MAP_TYPE_STRUCT_OPS) {
+			if (use_loader && !(bpf_map__map_flags(map) & BPF_F_LINK)) {
+				p_err("struct_ops map '%s' requires BPF_F_LINK for light skeleton",
+				      bpf_map__name(map));
+				err = -ENOTSUP;
+				goto out;
+			}
 			attach_map_cnt++;
+		}
 
 		map_cnt++;
 	}
@@ -1385,9 +1446,11 @@ static int do_skeleton(int argc, char **argv)
 	}
 
 	btf = bpf_object__btf(obj);
-	err = gen_st_ops_shadow(obj_name, btf, obj);
-	if (err)
-		goto out;
+	if (!use_loader) {
+		err = gen_st_ops_shadow(obj_name, btf, obj);
+		if (err)
+			goto out;
+	}
 
 	if (prog_cnt) {
 		printf("\tstruct {\n");

-- 
2.56.0.455.g6c5a503b38-goog

[PATCH RFC bpf-next 7/7] selftests/bpf: Add light skeleton struct_ops tests

[Siddharth Nayyar]

Add selftests exercising BPF_MAP_TYPE_STRUCT_OPS support in light
skeletons across both vmlinux and kernel module BTF targets:

- struct_ops_vmlinux.c defines a SEC(".struct_ops.link")
  tcp_congestion_ops map in vmlinux BTF, testing open/load, link
  attachment, callback invocation over a loopback TCP connection,
  and unregistration upon detach.
- struct_ops_mod.c defines two SEC(".struct_ops.link") maps targeting
  bpf_testmod_ops in module BTF (including a reordered subset struct
  bpf_testmod_ops___v2 sharing a callback), testing module BTF FD
  resolution, runtime member offset/index lookup via
  bpf_btf_get_struct_member_info(), scalar member copying, detach,
  and per-map re-attachment.

Signed-off-by: Siddharth Nayyar <sidnayyar@google.com>
---
 tools/testing/selftests/bpf/Makefile.skel          |   3 +-
 .../selftests/bpf/prog_tests/struct_ops_lskel.c    | 122 +++++++++++++++++++++
 tools/testing/selftests/bpf/progs/struct_ops_mod.c |  51 +++++++++
 .../selftests/bpf/progs/struct_ops_vmlinux.c       |  42 +++++++
 4 files changed, 217 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/Makefile.skel b/tools/testing/selftests/bpf/Makefile.skel
index b8eaecc7f429..2726f9081e35 100644
--- a/tools/testing/selftests/bpf/Makefile.skel
+++ b/tools/testing/selftests/bpf/Makefile.skel
@@ -34,7 +34,8 @@ LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h		\
 LSKELS := fexit_sleep.c trace_printk.c trace_vprintk.c map_ptr_kern.c 	\
 	core_kern.c core_kern_overflow.c test_ringbuf.c			\
 	test_ringbuf_n.c test_ringbuf_map_key.c test_ringbuf_write.c    \
-	test_ringbuf_overwrite.c callx_rodata.c
+	test_ringbuf_overwrite.c callx_rodata.c struct_ops_vmlinux.c	\
+	struct_ops_mod.c
 
 LSKELS_SIGNED := fentry_test.c fexit_test.c atomics.c
 
diff --git a/tools/testing/selftests/bpf/prog_tests/struct_ops_lskel.c b/tools/testing/selftests/bpf/prog_tests/struct_ops_lskel.c
new file mode 100644
index 000000000000..4e9bac460953
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/struct_ops_lskel.c
@@ -0,0 +1,122 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include "network_helpers.h"
+#include "struct_ops_vmlinux.lskel.h"
+#include "struct_ops_mod.lskel.h"
+
+static int set_tcp_ca_cb(int fd, void *opts)
+{
+	const char *tcp_ca = opts;
+
+	if (!ASSERT_OK(setsockopt(fd, IPPROTO_TCP, TCP_CONGESTION,
+				  tcp_ca, strlen(tcp_ca)), "setsockopt"))
+		return -1;
+	return 0;
+}
+
+static void test_vmlinux(void)
+{
+	char tcp_ca[] = "bpf_lskel_ca";
+	struct network_helper_opts opts = {
+		.post_socket_cb	= set_tcp_ca_cb,
+		.cb_opts	= tcp_ca,
+	};
+	struct struct_ops_vmlinux_lskel *skel;
+	int srv_fd = -1, cli_fd = -1, sock_fd = -1;
+	int err;
+
+	skel = struct_ops_vmlinux_lskel__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open_and_load"))
+		return;
+
+	err = struct_ops_vmlinux_lskel__attach(skel);
+	if (!ASSERT_OK(err, "attach"))
+		goto cleanup;
+
+	srv_fd = start_server_str(AF_INET6, SOCK_STREAM, NULL, 0, &opts);
+	if (!ASSERT_NEQ(srv_fd, -1, "start_server_str"))
+		goto cleanup;
+
+	cli_fd = connect_to_fd_opts(srv_fd, &opts);
+	if (!ASSERT_NEQ(cli_fd, -1, "connect_to_fd_opts"))
+		goto cleanup;
+
+	ASSERT_GT(skel->bss->ca_init_cnt, 0, "ca_init_cnt");
+
+	close(cli_fd);
+	cli_fd = -1;
+	close(srv_fd);
+	srv_fd = -1;
+
+	struct_ops_vmlinux_lskel__detach(skel);
+	skel->links.lskel_ca_fd = 0;
+
+	/* Verify congestion control was unregistered on detach */
+	sock_fd = socket(AF_INET6, SOCK_STREAM, 0);
+	if (!ASSERT_GE(sock_fd, 0, "socket"))
+		goto cleanup;
+
+	err = setsockopt(sock_fd, IPPROTO_TCP, TCP_CONGESTION,
+			 tcp_ca, strlen(tcp_ca));
+	ASSERT_EQ(err, -1, "setsockopt_after_detach");
+	ASSERT_EQ(errno, ENOENT, "errno_after_detach");
+
+cleanup:
+	if (sock_fd != -1)
+		close(sock_fd);
+	if (cli_fd != -1)
+		close(cli_fd);
+	if (srv_fd != -1)
+		close(srv_fd);
+	struct_ops_vmlinux_lskel__destroy(skel);
+}
+
+static void test_module(void)
+{
+	struct struct_ops_mod_lskel *skel;
+	int err, link_fd;
+
+	if (!env.has_testmod) {
+		test__skip();
+		return;
+	}
+
+	skel = struct_ops_mod_lskel__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open_and_load"))
+		return;
+
+	err = struct_ops_mod_lskel__attach(skel);
+	if (!ASSERT_OK(err, "attach"))
+		goto cleanup;
+
+	ASSERT_GT(skel->links.testmod_1_fd, 0, "testmod_1_fd");
+	ASSERT_GT(skel->links.testmod_2_fd, 0, "testmod_2_fd");
+	ASSERT_EQ(skel->bss->test_1_cnt, 2, "test_1_cnt");
+	ASSERT_EQ(skel->bss->testmod_1_result, 17, "testmod_1_result");
+	ASSERT_EQ(skel->bss->testmod_2_result, 28, "testmod_2_result");
+
+	struct_ops_mod_lskel__detach(skel);
+	skel->links.testmod_1_fd = 0;
+	skel->links.testmod_2_fd = 0;
+	memset(skel->bss, 0, sizeof(*skel->bss));
+
+	link_fd = struct_ops_mod_lskel__testmod_1__attach(skel);
+	if (!ASSERT_GT(link_fd, 0, "testmod_1_attach"))
+		goto cleanup;
+
+	ASSERT_EQ(skel->bss->test_1_cnt, 1, "reattach_test_1_cnt");
+	ASSERT_EQ(skel->bss->testmod_1_result, 17, "reattach_testmod_1_result");
+	ASSERT_EQ(skel->bss->testmod_2_result, 0, "reattach_testmod_2_result");
+
+cleanup:
+	struct_ops_mod_lskel__destroy(skel);
+}
+
+void test_struct_ops_lskel(void)
+{
+	if (test__start_subtest("vmlinux"))
+		test_vmlinux();
+	if (test__start_subtest("module"))
+		test_module();
+}
diff --git a/tools/testing/selftests/bpf/progs/struct_ops_mod.c b/tools/testing/selftests/bpf/progs/struct_ops_mod.c
new file mode 100644
index 000000000000..b1bd7429149d
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/struct_ops_mod.c
@@ -0,0 +1,51 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../test_kmods/bpf_testmod.h"
+
+char _license[] SEC("license") = "GPL";
+
+int test_1_cnt = 0;
+int testmod_1_result = 0;
+int testmod_2_result = 0;
+
+SEC("struct_ops/test_1")
+int BPF_PROG(test_1)
+{
+	test_1_cnt++;
+	return 0;
+}
+
+SEC("struct_ops/test_2")
+void BPF_PROG(test_2, int a, int b)
+{
+	testmod_1_result = a + b;
+}
+
+SEC("struct_ops/test_2_v2")
+void BPF_PROG(test_2_v2, int a, int b)
+{
+	testmod_2_result = a * b;
+}
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops testmod_1 = {
+	.test_1 = (void *)test_1,
+	.test_2 = (void *)test_2,
+	.data = 13,
+};
+
+struct bpf_testmod_ops___v2 {
+	int data;
+	void (*test_2)(int a, int b);
+	int (*test_1)(void);
+};
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops___v2 testmod_2 = {
+	.data = 7,
+	.test_2 = (void *)test_2_v2,
+	.test_1 = (void *)test_1,
+};
diff --git a/tools/testing/selftests/bpf/progs/struct_ops_vmlinux.c b/tools/testing/selftests/bpf/progs/struct_ops_vmlinux.c
new file mode 100644
index 000000000000..1d8b573ad799
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/struct_ops_vmlinux.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "bpf_tracing_net.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+int ca_init_cnt = 0;
+
+SEC("struct_ops/ca_lskel_init")
+void BPF_PROG(ca_lskel_init, struct sock *sk)
+{
+	ca_init_cnt++;
+}
+
+SEC("struct_ops/ca_lskel_cong_control")
+void BPF_PROG(ca_lskel_cong_control, struct sock *sk, __u32 ack, int flag,
+	      const struct rate_sample *rs)
+{
+}
+
+SEC("struct_ops/ca_lskel_ssthresh")
+__u32 BPF_PROG(ca_lskel_ssthresh, struct sock *sk)
+{
+	return tcp_sk(sk)->snd_ssthresh;
+}
+
+SEC("struct_ops/ca_lskel_undo_cwnd")
+__u32 BPF_PROG(ca_lskel_undo_cwnd, struct sock *sk)
+{
+	return tcp_sk(sk)->snd_cwnd;
+}
+
+SEC(".struct_ops.link")
+struct tcp_congestion_ops lskel_ca = {
+	.init = (void *)ca_lskel_init,
+	.cong_control = (void *)ca_lskel_cong_control,
+	.ssthresh = (void *)ca_lskel_ssthresh,
+	.undo_cwnd = (void *)ca_lskel_undo_cwnd,
+	.name = "bpf_lskel_ca",
+};

-- 
2.56.0.455.g6c5a503b38-goog

Re: [PATCH RFC bpf-next 3/7] libbpf: Support BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create()

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2179 bytes --]

> libbpf: Support BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create()

TL;DR: In bpf_gen__map_create(), the struct_ops module BTF fd is closed only
after BPF_MAP_CREATE, so earlier loader failures may leak that fd, and a
retrying daemon may leak one per attempt.

> diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
> --- a/tools/lib/bpf/gen_loader.c
> +++ b/tools/lib/bpf/gen_loader.c

[ ... ]

> @@ -568,6 +589,9 @@ void bpf_gen__map_create(struct bpf_gen *gen,
>  	debug_ret(gen, "map_create %s idx %d type %d value_size %d value_btf_id %d",
>  		  map_name, map_idx, map_type, value_size,
>  		  map_attr->btf_value_type_id);
> +	if (map_type == BPF_MAP_TYPE_STRUCT_OPS)
> +		emit_sys_close_blob(gen,
> +				    attr_field(map_create_attr, value_type_btf_obj_fd));
>  	emit_check_err(gen);

The module BTF fd is only closed here, on the BPF_MAP_CREATE path of this
map.  Is it possible for the loader to leak that fd on earlier failures?

The fd is opened by the code emitted from bpf_gen__init_kern_struct_ops(),
which runs before bpf_object__sanitize_and_load_btf() and
bpf_object__create_maps().  Between those points the loader has several
emit_check_err() jumps to the cleanup label, for example:

  bpf_gen__load_btf()
  an earlier map's BPF_MAP_CREATE
  the struct_ops init sequence checks for a second module struct_ops map

The cleanup code emitted by bpf_gen__init() closes only btf_fd,
inner_map_fd, prog_fd[] and the first nr_maps fd_array entries, so the fd
stored in the upper 32 bits of the vtype_id_off slot is never closed on
those paths.

If one of those steps fails, a module BTF fd stays open in the process that
ran the loader, and a daemon that retries the load leaks one more fd each
time.  bpf_gen__prog_load() closes its module BTF fds "successful or not"
before any error jump.

Could the vtype btf fd also be closed on the cleanup path, or be kept
somewhere the cleanup code already closes?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42289585064

Re: [PATCH RFC bpf-next 1/7] bpf: Add bpf_btf_get_struct_member_info() helper

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1881 bytes --]

> bpf: Add bpf_btf_get_struct_member_info() helper

TL;DR: Design question: the new UAPI helper is added despite the frozen helper
list, which says to use kfuncs; syscall programs can already call kfuncs, and
the commit message doesn't explain why a helper is needed.

> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h

[ ... ]

> @@ -6279,6 +6291,7 @@ union bpf_attr {
>  	FN(user_ringbuf_drain, 209, ##ctx)		\
>  	FN(cgrp_storage_get, 210, ##ctx)		\
>  	FN(cgrp_storage_delete, 211, ##ctx)		\
> +	FN(btf_get_struct_member_info, 212, ##ctx)	\
>  	/* This helper list is effectively frozen. If you are trying to	\
>  	 * add a new helper, you should add a kfunc instead which has	\
>  	 * less stability guarantees. See Documentation/bpf/kfuncs.rst	\

This adds a new UAPI helper id right above the comment saying the helper
list is effectively frozen and that new functionality should be a kfunc.

BPF_PROG_TYPE_SYSCALL programs can already call kfuncs, for example via
register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &generic_kfunc_set) in
kernel/bpf/helpers.c.

Is there a reason this needs to be a helper rather than a kfunc?  Perhaps
the generated light skeleton loader cannot resolve a kfunc BTF id for its
own call instruction, while a helper has a fixed id, but the commit message
only says:

> This helper will be used by BPF light skeletons (gen_loader) to
> initialize struct_ops maps and programs against kernel and module BTF
> at load time.

Could the commit message explain why the freeze is being broken here and
why a kfunc will not work?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42289585064

Re: [PATCH RFC bpf-next 5/7] libbpf: Implement bpf_gen__struct_ops_update_elem() for light skeletons

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2868 bytes --]

> libbpf: Implement bpf_gen__struct_ops_update_elem() for light skeletons

TL;DR: In gen_loader.c, the 8-byte store of a prog fd at a variable offset may
be rejected by the verifier on strict-alignment kernels (e.g. riscv64), so
light skeletons with struct_ops callbacks could fail to load with -EACCES.

> diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
> --- a/tools/lib/bpf/gen_loader.c
> +++ b/tools/lib/bpf/gen_loader.c

[ ... ]

> +void bpf_gen__struct_ops_update_elem(struct bpf_gen *gen, int map_idx,
> +				     struct btf *btf,
> +				     struct bpf_struct_ops *st_ops)
> +{

[ ... ]

> +		/* R2 = dynamic kernel member offset from off_slot */
> +		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
> +						 0, 0, 0, off_slot));
> +		emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0));
> +		emit(gen, BPF_JMP_IMM(BPF_JLE, BPF_REG_2,
> +				      KERN_VDATA_MAX_SIZE - sizeof(__u64), 2));
> +		emit(gen, BPF_MOV64_IMM(BPF_REG_7, -E2BIG));
> +		emit_check_err(gen);
> +
> +		/* R1 = kern_vdata_off + offset */
> +		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
> +						 0, 0, 0, st_ops->kern_vdata_off));
> +		emit(gen, BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2));

[ ... ]

> +		/* Store prog_fd (R0) into kern_vdata + offset (R1) */
> +		emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0));

Can this 8-byte store be rejected by the verifier on strict-alignment
configurations?

After the JLE check, R2 only has var_off = tnum_range(0, 4088), so the low
three bits are unknown.  R1 = blob + kern_vdata_off + R2 then goes through
check_ptr_alignment() -> check_generic_ptr_alignment():

    reg_off = tnum_add(reg->var_off, tnum_const(off));
    if (!tnum_is_aligned(reg_off, size)) {
            ... "misaligned %saccess off %s+%d size %d\n" ...
            return -EACCES;

strict is set for every program when CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
is not enabled, which is the default on riscv64 (it is only selected by
RISCV_EFFICIENT_UNALIGNED_ACCESS, which depends on NONPORTABLE).  The light
skeleton loader is loaded without BPF_F_ANY_ALIGNMENT, so on such kernels a
light skeleton with a struct_ops callback would fail to load with -EACCES.

The other variable-offset blob access in gen_loader, emit_st_ops_member_val(),
passes the pointer to bpf_probe_read_kernel(), and helper memory arguments
are not alignment-checked.  Other direct blob stores use constant, 8-byte
aligned offsets.

Would masking the offset, e.g. BPF_ALU64_IMM(BPF_AND, BPF_REG_2, ~7), or
copying through bpf_probe_read_kernel() make the alignment provable?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42289585064

Re: [PATCH RFC bpf-next 4/7] libbpf: Support BPF_PROG_TYPE_STRUCT_OPS in bpf_gen__prog_load()

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4075 bytes --]

> libbpf: Support BPF_PROG_TYPE_STRUCT_OPS in bpf_gen__prog_load()

TL;DR: In light skeletons, a kernel-module BTF fd for struct_ops may leak
(stay open in the loader's fd table) if any struct_ops program load fails,
since no error-path close is emitted.

> diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
> --- a/tools/lib/bpf/gen_loader.c
> +++ b/tools/lib/bpf/gen_loader.c

[ ... ]

> @@ -1151,6 +1151,20 @@ void bpf_gen__prog_load(struct bpf_gen *gen,
>  		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
>  				      offsetof(union bpf_attr, attach_btf_obj_fd)));
>  	}
> +	if (prog_type == BPF_PROG_TYPE_STRUCT_OPS) {
> +		move_blob2blob(gen, attr_field(prog_load_attr, expected_attach_type), 4,
> +			       load_attr->expected_attach_type);
> +		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
> +						 0, 0, 0, load_attr->attach_btf_id));
> +		emit(gen, BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_1, 0));
> +		emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE,
> +						 0, 0, 0, prog_load_attr));
> +		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
> +				      offsetof(union bpf_attr, attach_btf_id)));
> +		emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32));
> +		emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7,
> +				      offsetof(union bpf_attr, attach_btf_obj_fd)));
> +	}
>  	emit_relos(gen, insns_off);
>  	/* emit PROG_LOAD command */
>  	emit_sys_bpf(gen, BPF_PROG_LOAD, prog_load_attr, attr_size);

For a struct_ops type defined in a kernel module, can the module BTF fd
leak into the process running the light skeleton when a struct_ops
program fails to load?

The block above copies the upper 32 bits of the u64 at
load_attr->attach_btf_id into prog_load_attr.attach_btf_obj_fd.  That
value was produced at runtime by bpf_btf_find_by_name_kind() in
bpf_gen__init_kern_struct_ops().  For a module type it creates a new fd
with __btf_new_fd() and returns:

    ret | ((u64)btf_obj_fd << 32)

so this is an open fd owned by the loader.

The existing code closes module BTF fds whether the load succeeds or not:

    /* successful or not, close btf module FDs used in extern ksyms and attach_btf_obj_fd */
    cleanup_relos(gen, insns_off);
    if (gen->attach_kind) {
            emit_sys_close_blob(gen,
                                attr_field(prog_load_attr, attach_btf_obj_fd));
            gen->attach_kind = 0;
    }
    emit_check_err(gen);

bpf_gen__map_create() does the same for value_type_btf_obj_fd, but the
struct_ops path here does not.  The fd cannot be closed here because all
programs of the map share it.  The only close is in
bpf_gen__struct_ops_update_elem(), added later in the series, which is
emitted from bpf_object_prepare_struct_ops() after all programs are
loaded and so only runs on success.

If BPF_PROG_LOAD fails for any struct_ops program (for example a verifier
rejection), emit_check_err() jumps to gen->cleanup_label.  The cleanup
code emitted by bpf_gen__init() only closes the prog_fd[] stack slots and
the map fds in the blob fd_array:

    for (i = 0; i < nr_progs_sz; i += 4) { ... BPF_FUNC_sys_close ... }
    for (i = 0; i < nr_maps; i++)
            emit_sys_close_blob(gen, blob_fd_array_off(gen, i));

Nothing there knows about st_ops->type_id_off, so the module BTF fd of
every struct_ops map initialised so far stays open in the caller's fd
table.  The same applies to any later failure before
bpf_gen__struct_ops_update_elem() runs, such as another program's load or
an emit_check_err() in emit_relos().  vmlinux struct_ops types are not
affected, since the upper 32 bits are 0 and __emit_sys_close() skips
fd <= 0.

No later commit in the series adds an error-path close.  Should these fds
be added to the cleanup_label path, or closed in an error branch around
the prog-load emit_check_err()?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42289585064

Re: [PATCH RFC bpf-next 2/7] libbpf: Implement bpf_gen__init_kern_struct_ops() for light skeletons

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 6839 bytes --]

> libbpf: Implement bpf_gen__init_kern_struct_ops() for light skeletons

TL;DR:
- Light-skeleton struct_ops loader may overflow into adjacent kern_vdata
  members: kernel member size/kind unchecked
- Module BTF fds may leak on loader failure paths
- Type lookups may resolve in different BTF objects, failing module struct_ops
  loads

> diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c
> --- a/tools/lib/bpf/gen_loader.c
> +++ b/tools/lib/bpf/gen_loader.c

[ ... ]

> +static void emit_st_ops_member_val(struct bpf_gen *gen, const char *mname,
> +				   const void *mdata, __u32 msize,
> +				   int kern_vdata_off)
> +{

[ ... ]

> +	mdata_off = add_data(gen, mdata, msize);
> +	emit_bpf_get_struct_member_info(gen, mname);
> +
> +	/* Extract member byte offset (lower 32 bits of R7), add kern_data_off, and bound it */
> +	emit(gen, BPF_MOV32_REG(BPF_REG_2, BPF_REG_7));
> +	emit(gen, BPF_ALU32_REG(BPF_ADD, BPF_REG_2, BPF_REG_9));
> +	emit(gen, BPF_JMP_IMM(BPF_JLE, BPF_REG_2, KERN_VDATA_MAX_SIZE - msize, 2));
> +	emit(gen, BPF_MOV64_IMM(BPF_REG_7, -E2BIG));
> +	emit_check_err(gen);

The gen path copies msize bytes, where msize is the size of the local
member, to the kernel member's offset.  The only bound here is the 4096
byte kern_vdata buffer.  The kernel member's size and kind, and val_sz,
are not checked.

bpf_map__init_kern_struct_ops() rejects both cases with -ENOTSUP:

    if (BTF_INFO_KIND(mtype->info) != BTF_INFO_KIND(kern_mtype->info)) {
        ...
        return -ENOTSUP;
    }
    ...
    kern_msize = btf__resolve_size(kern_btf, kern_mtype_id);
    if (kern_msize < 0 || msize != kern_msize) {
        ...
        return -ENOTSUP;
    }

If a local struct_ops definition (a ___flavor type, or a vmlinux.h from
another kernel) has a non-zero scalar member that is wider than the
running kernel's, or is a scalar where the kernel has a function pointer,
can the loader write the extra bytes into the following kernel members of
kern_vdata?

If the init_member() callback accepts those values, the map could be
registered with wrong values, and bytes landing in a function pointer slot
are later read as a prog fd by bpf_struct_ops_map_update_elem().

The regular libbpf load of the same object fails cleanly.  Was dropping
these checks intended, or could bpf_btf_get_struct_member_info() also
return the member size, or the loader at least check against val_sz?

[ ... ]

> +void bpf_gen__init_kern_struct_ops(struct bpf_gen *gen, struct btf *btf,
> +				   struct bpf_struct_ops *st_ops,
> +				   __u32 *vtype_id_off,
> +				   unsigned int *val_sz_off)
> +{

[ ... ]

> +	/* 1. Resolve BTF ID of struct bpf_struct_ops_<tname> */
> +	*vtype_id_off = add_data(gen, NULL, sizeof(__u64));
> +	relo.name = stname;
> +	emit_bpf_find_by_name_kind(gen, &relo);
> +	emit_check_err(gen);
> +	emit(gen, BPF_MOV64_REG(BPF_REG_8, BPF_REG_7));
> +	emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE,
> +					 0, 0, 0, *vtype_id_off));
> +	emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_7, 0));

For a struct_ops type that lives in a module, each
bpf_btf_find_by_name_kind() call (steps 1 and 3) creates a new module BTF
fd.  The fd is kept only in the upper 32 bits of the blob slots
*vtype_id_off and st_ops->type_id_off.

The cleanup label emitted by bpf_gen__init() closes only the loader_stack
slots (btf_fd, inner_map_fd, prog_fd[]) and the map entries of fd_array,
so these fds are not closed when emit_check_err() jumps to cleanup.

This commit adds no close at all.  Later in the series, c8839efa9329
("libbpf: Support BPF_MAP_TYPE_STRUCT_OPS in bpf_gen__map_create()")
closes the step 1 fd right after BPF_MAP_CREATE, and b5342aeba917
("libbpf: Implement bpf_gen__struct_ops_update_elem() for light
skeletons") closes the step 3 fd only in bpf_gen__struct_ops_update_elem(),
which runs after every program has loaded.

Can a failure in between leak these BTF fds in the calling process?  For
example a missing kernel member (-ENOENT from
bpf_btf_get_struct_member_info()), the -E2BIG bound check, BTF load, an
earlier map create, or a verifier rejection of any program would each
jump to cleanup with the fds still open.

bpf_gen__prog_load() already closes attach_btf_obj_fd and the kfunc BTF
fds right after PROG_LOAD ("successful or not, close btf module FDs").
Could these fds be kept in extra loader_stack slots that the cleanup label
closes, or be closed before the next emit_check_err() when not needed
later?

[ ... ]

> +	/* 3. Resolve BTF ID of struct <tname> */
> +	st_ops->type_id_off = add_data(gen, NULL, sizeof(__u64));
> +	relo.name = tname;
> +	emit_bpf_find_by_name_kind(gen, &relo);
> +	emit_check_err(gen);

Step 3 looks up struct <tname> with a fresh, global
bpf_btf_find_by_name_kind() call that is not tied to the BTF object where
step 1 found struct bpf_struct_ops_<tname>.  The kernel's bpf_find_btf_id()
searches vmlinux first and then every module BTF in idr order, so the two
lookups can land in different BTF objects.

The non-gen path looks up the value type first and then looks up tname in
the same btf:

    kern_vtype_id = find_ksym_btf_id(obj, stname, BTF_KIND_STRUCT, &btf, mod_btf);
    ...
    kern_type_id = btf__find_by_name_kind(btf, tname, BTF_KIND_STRUCT);

and bpf_map__init_kern_struct_ops() then sets prog->attach_btf_obj_fd to
mod_btf->fd.

A concrete case is net/smc/smc_hs_bpf.c, which registers the smc_hs_ctrl
struct_ops from smc.ko.  struct smc_hs_ctrl is also fully defined in
vmlinux BTF, because net/ipv4/tcp_output.c includes <net/smc.h>.  In gen
mode:

    step 1 finds bpf_struct_ops_smc_hs_ctrl in smc.ko and returns
           (smc_fd << 32) | vtype_id
    step 3 finds smc_hs_ctrl in vmlinux and returns
           (0 << 32) | type_id

With ee35678a566a ("libbpf: Support BPF_PROG_TYPE_STRUCT_OPS in
bpf_gen__prog_load()"), prog_load takes attach_btf_obj_fd from the upper
half of type_id_off, which is 0.  bpf_prog_load() then falls back to
vmlinux BTF, and check_struct_ops_btf_id() -> bpf_struct_ops_find() does
not find the type in vmlinux's struct_ops_tab, so the load fails with
"attach_btf_id %u is not a supported struct" and -ENOTSUPP.

The same mismatch can occur when another module loaded earlier carries its
own copy of struct <tname> in its split BTF.  No later commit in the
series changes this lookup, and the selftests only use bpf_testmod_ops,
which exists only in bpf_testmod.ko.

Could the interface type be resolved within the BTF object returned in
step 1, or could the step 1 fd be paired with the type id when the step 3
result comes from vmlinux?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42289585064