* [PATCH v1 bpf-next 02/18] bpf: Add a btf_id_set8_entry helper
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 03/18] bpf: Build fmod_ret target sets from BTF decl tags Anton Protopopov
` (15 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
The pairs of a struct btf_id_set8 have an anonymous type, and the
btf_id_set8_contains() helper returns a void pointer to the pair it
finds. All its users but one only check if a set contains an ID. The
one which needs the flags, btf_kfunc_id_set_contains(), finds them as
the word next to the ID.
Name the type struct btf_id_set8_pair, and add a btf_id_set8_entry()
helper which returns a pointer to it. Make btf_id_set8_contains()
return bool, like btf_id_set_contains() does, and use the new helper
in btf_kfunc_id_set_contains().
The layout of the set doesn't change. Update the tools copy of
btf_ids.h as well. A later patch uses the new helper to set the flags
of a pair.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
include/linux/btf.h | 8 +++++++-
include/linux/btf_ids.h | 10 ++++++----
kernel/bpf/btf.c | 9 ++++-----
tools/include/linux/btf_ids.h | 10 ++++++----
4 files changed, 23 insertions(+), 14 deletions(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 87fad4076f0e..00495e7833b6 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -558,11 +558,17 @@ static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id)
return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL;
}
-static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+static inline struct btf_id_set8_pair *
+btf_id_set8_entry(const struct btf_id_set8 *set, u32 id)
{
return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func);
}
+static inline bool btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
+{
+ return btf_id_set8_entry(set, id) != NULL;
+}
+
bool btf_param_match_suffix(const struct btf *btf,
const struct btf_param *arg,
const char *suffix);
diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
index f27b7db81100..f5985acf661b 100644
--- a/include/linux/btf_ids.h
+++ b/include/linux/btf_ids.h
@@ -19,13 +19,15 @@ struct btf_id_set {
*/
#define BTF_SET8_FMODRET (1 << 1)
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 228b93d4f8c9..61d2df6776a9 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9973,8 +9973,8 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
enum btf_kfunc_hook hook,
u32 kfunc_btf_id)
{
+ struct btf_id_set8_pair *pair;
struct btf_id_set8 *set;
- u32 *id;
if (hook >= BTF_KFUNC_HOOK_MAX)
return NULL;
@@ -9983,11 +9983,10 @@ static u32 *btf_kfunc_id_set_contains(const struct btf *btf,
set = btf->kfunc_set_tab->sets[hook];
if (!set)
return NULL;
- id = btf_id_set8_contains(set, kfunc_btf_id);
- if (!id)
+ pair = btf_id_set8_entry(set, kfunc_btf_id);
+ if (!pair)
return NULL;
- /* The flags for BTF ID are located next to it */
- return id + 1;
+ return &pair->flags;
}
static bool __btf_kfunc_is_allowed(const struct btf *btf,
diff --git a/tools/include/linux/btf_ids.h b/tools/include/linux/btf_ids.h
index b371ef7241b9..3b5b2856c628 100644
--- a/tools/include/linux/btf_ids.h
+++ b/tools/include/linux/btf_ids.h
@@ -19,13 +19,15 @@ struct btf_id_set {
*/
#define BTF_SET8_FMODRET (1 << 1)
+struct btf_id_set8_pair {
+ u32 id;
+ u32 flags;
+};
+
struct btf_id_set8 {
u32 cnt;
u32 flags;
- struct {
- u32 id;
- u32 flags;
- } pairs[];
+ struct btf_id_set8_pair pairs[];
};
#ifdef CONFIG_DEBUG_INFO_BTF
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 03/18] bpf: Build fmod_ret target sets from BTF decl tags
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 02/18] bpf: Add a btf_id_set8_entry helper Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 04/18] selftests/bpf: Declare bpf_fentry_test1 in tracing tests Anton Protopopov
` (14 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Mark functions tagged with the "bpf_fmodret{,_sleepable}" decl tags
(added by resolve_btfids) as fmodret targets. Add the KF_SLEEPABLE flag
for the latter.
Do this for vmlinux at fs_initcall and for a module before its BTF
becomes visible to anyone else, so nothing can look at the set while it
is built.
The IDs come from the BTF after relocation, so they are final. Unlike
the IDs of a set registered by a module, they must not be relocated, and
the set is installed directly instead of through btf_populate_kfunc_set().
(The register_btf_fmodret_id_set() keeps working, it will be removed by
a later patch.)
The set shares kfunc_set_tab with the kfunc sets, but its flags only say
how a target may be attached. Skip it in btf_kfunc_check_flag(), so a
kfunc which is also an fmod_ret target keeps its own flags, such as
KF_IMPLICIT_ARGS. Otherwise btf_attach_func_proto() finds the flags of
such a kfunc inconsistent and attaching to it fails.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
kernel/bpf/btf.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 113 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 61d2df6776a9..2c45e5fa1d7b 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9307,6 +9307,8 @@ enum {
};
#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+static int btf_register_fmodret_tags(struct btf *btf);
+
struct btf_module {
struct list_head list;
struct module *module;
@@ -9476,6 +9478,15 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,
}
goto out;
}
+ /*
+ * Do this before the BTF is visible to anyone else. A failure
+ * is not fatal: the module loads, but without fmod_ret targets.
+ */
+ err = btf_register_fmodret_tags(btf);
+ if (err)
+ pr_warn("failed to register fmod_ret targets of module [%s]: %d\n",
+ mod->name, err);
+
err = btf_alloc_id(btf);
if (err) {
btf_free(btf);
@@ -10100,7 +10111,9 @@ u32 *btf_kfunc_flags(const struct btf *btf, u32 kfunc_btf_id, const struct bpf_p
}
/*
- * Check a single KF_* @flag on a kfunc across all of its hook sets.
+ * Check a single KF_* @flag on a kfunc across all of its hook sets. The set of
+ * fmod_ret targets is not one of them: its flags only say how a target may be
+ * attached, and a kfunc which is also a target keeps its own flags.
* Returns:
* * 1 if @flag is set
* * 0 if @flag is not set
@@ -10115,6 +10128,8 @@ int btf_kfunc_check_flag(const struct btf *btf, u32 kfunc_btf_id, u32 flag)
u32 *flags;
for (hook = 0; hook < BTF_KFUNC_HOOK_MAX; hook++) {
+ if (hook == BTF_KFUNC_HOOK_FMODRET)
+ continue;
flags = btf_kfunc_id_set_contains(btf, hook, kfunc_btf_id);
if (!flags)
continue;
@@ -10182,6 +10197,103 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
}
EXPORT_SYMBOL_GPL(register_btf_kfunc_id_set);
+/*
+ * Targets of fmod_ret programs are marked in BTF by resolve_btfids, see
+ * BTF_SET8_FMODRET. A "bpf_fmodret" decl tag on a function makes it a target.
+ * A target which sleepable programs may attach to also has a
+ * "bpf_fmodret_sleepable" tag, which never comes without "bpf_fmodret".
+ */
+#define BTF_FMODRET_TAG "bpf_fmodret"
+#define BTF_FMODRET_SLEEPABLE_TAG "bpf_fmodret_sleepable"
+
+/* Return the function which T tags with decl tag NAME, or 0 */
+static u32 btf_fmodret_tag_func(const struct btf *btf, const struct btf_type *t,
+ const char *name)
+{
+ const struct btf_type *func;
+
+ if (!btf_type_is_decl_tag(t) || btf_type_decl_tag(t)->component_idx != -1)
+ return 0;
+ if (strcmp(btf_name_by_offset(btf, t->name_off), name))
+ return 0;
+ func = btf_type_by_id(btf, t->type);
+ if (!func || !btf_type_is_func(func))
+ return 0;
+ return t->type;
+}
+
+/* Build the set of fmod_ret targets of BTF from its decl tags */
+static int btf_register_fmodret_tags(struct btf *btf)
+{
+ u32 id, func_id, nr_types = btf_nr_types(btf), cnt = 0;
+ struct btf_kfunc_set_tab *tab;
+ struct btf_id_set8 *set;
+
+ for (id = btf_start_id(btf); id < nr_types; id++)
+ cnt += !!btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!cnt)
+ return 0;
+
+ set = kmalloc(struct_size(set, pairs, cnt), GFP_KERNEL | __GFP_NOWARN);
+ if (!set)
+ return -ENOMEM;
+ set->flags = 0;
+ set->cnt = 0;
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id), BTF_FMODRET_TAG);
+ if (!func_id)
+ continue;
+ set->pairs[set->cnt].id = func_id;
+ set->pairs[set->cnt].flags = 0;
+ set->cnt++;
+ }
+ sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL);
+
+ /* Each sleepable tag marks a function which is already in the set */
+ for (id = btf_start_id(btf); id < nr_types; id++) {
+ struct btf_id_set8_pair *pair;
+
+ func_id = btf_fmodret_tag_func(btf, btf_type_by_id(btf, id),
+ BTF_FMODRET_SLEEPABLE_TAG);
+ if (!func_id)
+ continue;
+ pair = btf_id_set8_entry(set, func_id);
+ if (pair)
+ pair->flags |= KF_SLEEPABLE;
+ }
+
+ tab = btf->kfunc_set_tab;
+ if (!tab) {
+ tab = kzalloc_obj(*tab, GFP_KERNEL | __GFP_NOWARN);
+ if (!tab) {
+ kfree(set);
+ return -ENOMEM;
+ }
+ btf->kfunc_set_tab = tab;
+ }
+ if (tab->sets[BTF_KFUNC_HOOK_FMODRET]) {
+ kfree(set);
+ return -EEXIST;
+ }
+ tab->sets[BTF_KFUNC_HOOK_FMODRET] = set;
+ return 0;
+}
+
+static int __init btf_fmodret_tags_init(void)
+{
+ struct btf *btf;
+ int err;
+
+ btf = btf_get_module_btf(NULL);
+ if (IS_ERR_OR_NULL(btf))
+ return PTR_ERR_OR_ZERO(btf);
+
+ err = btf_register_fmodret_tags(btf);
+ btf_put(btf);
+ return err;
+}
+fs_initcall(btf_fmodret_tags_init);
+
/* This function must be invoked only from initcalls/module init functions */
int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset)
{
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 04/18] selftests/bpf: Declare bpf_fentry_test1 in tracing tests
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 02/18] bpf: Add a btf_id_set8_entry helper Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 03/18] bpf: Build fmod_ret target sets from BTF decl tags Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 05/18] bpf: Declare fmod_ret targets with BTF_FMODRET sets Anton Protopopov
` (13 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
bpftool puts bpf_fentry_test1 into vmlinux.h only because the function
is in a kfunc set, which exists to let fmod_ret programs attach to it.
A later patch moves it to an fmod_ret set, which does not make a
function a kfunc, and the declaration disappears.
The tests only compare its address. Declare it explicitly, as
get_func_ip_test.c does. The declaration matches the one in vmlinux.h,
so it builds both before and after.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
tools/testing/selftests/bpf/progs/fsession_test.c | 2 ++
tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c | 2 ++
tools/testing/selftests/bpf/progs/tracing_multi_check.c | 2 +-
3 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/progs/fsession_test.c b/tools/testing/selftests/bpf/progs/fsession_test.c
index 86e8a2fe467e..4295f6a78dd1 100644
--- a/tools/testing/selftests/bpf/progs/fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/fsession_test.c
@@ -4,6 +4,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
index bbeea0d512e3..51a7f605f9c5 100644
--- a/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
+++ b/tools/testing/selftests/bpf/progs/get_func_ip_fsession_test.c
@@ -3,6 +3,8 @@
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
+extern int bpf_fentry_test1(int a) __ksym;
+
char _license[] SEC("license") = "GPL";
__u64 test1_entry_result = 0;
diff --git a/tools/testing/selftests/bpf/progs/tracing_multi_check.c b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
index b2959ba71179..9b008ecd6565 100644
--- a/tools/testing/selftests/bpf/progs/tracing_multi_check.c
+++ b/tools/testing/selftests/bpf/progs/tracing_multi_check.c
@@ -8,7 +8,7 @@ char _license[] SEC("license") = "GPL";
int pid = 0;
bool test_cookies = false;
-/* bpf_fentry_test1 is exported as kfunc via vmlinux.h */
+extern int bpf_fentry_test1(int a) __ksym;
extern const void bpf_fentry_test2 __ksym;
extern const void bpf_fentry_test3 __ksym;
extern const void bpf_fentry_test4 __ksym;
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 05/18] bpf: Declare fmod_ret targets with BTF_FMODRET sets
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (2 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 04/18] selftests/bpf: Declare bpf_fentry_test1 in tracing tests Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 06/18] selftests/bpf: Test fmod_ret BTF decl tags and attach permission Anton Protopopov
` (12 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Move the fmod_ret targets of the BPF test functions, MPTCP and bpf_testmod
from registered kfunc sets to BTF_FMODRET sets. Attachment flags stay the
same: bpf_fentry_test1 is still the sleepable target.
These functions are no longer kfuncs, so the bpf_kfunc decl tag, and
with it the vmlinux.h declaration, is gone. Only
bpf_modify_return_test_tp() stays a kfunc, as it is also in the
common_btf_ids kfunc set. Selftests which need bpf_fentry_test1 declare
it themselves.
No code registers an fmod_ret set anymore, as the kernel builds them from
BTF. Remove register_btf_fmodret_id_set(), so a module cannot register a
second set next to the one from its BTF.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
include/linux/btf.h | 1 -
kernel/bpf/btf.c | 7 -------
net/bpf/test_run.c | 12 +++---------
net/mptcp/bpf.c | 15 ++-------------
.../selftests/bpf/test_kmods/bpf_testmod.c | 11 ++---------
5 files changed, 7 insertions(+), 39 deletions(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 00495e7833b6..5522c7fe1373 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -632,7 +632,6 @@ u32 *btf_kfunc_is_modify_return(const struct btf *btf, u32 kfunc_btf_id,
const struct bpf_prog *prog);
int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
const struct btf_kfunc_id_set *s);
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset);
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);
int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt,
struct module *owner);
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 2c45e5fa1d7b..f81a8ae9b92c 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -10294,13 +10294,6 @@ static int __init btf_fmodret_tags_init(void)
}
fs_initcall(btf_fmodret_tags_init);
-/* This function must be invoked only from initcalls/module init functions */
-int register_btf_fmodret_id_set(const struct btf_kfunc_id_set *kset)
-{
- return __register_btf_kfunc_id_set(BTF_KFUNC_HOOK_FMODRET, kset);
-}
-EXPORT_SYMBOL_GPL(register_btf_fmodret_id_set);
-
s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id)
{
struct btf_id_dtor_kfunc_tab *tab = btf->dtor_kfunc_tab;
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index d6c9dc62182e..d63c3b697552 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -643,17 +643,12 @@ CFI_NOSEAL(bpf_kfunc_call_memb_release_dtor);
__bpf_kfunc_end_defs();
-BTF_KFUNCS_START(bpf_test_modify_return_ids)
+BTF_FMODRET_START(bpf_test_modify_return_ids)
BTF_ID_FLAGS(func, bpf_modify_return_test)
BTF_ID_FLAGS(func, bpf_modify_return_test2)
BTF_ID_FLAGS(func, bpf_modify_return_test_tp)
BTF_ID_FLAGS(func, bpf_fentry_test1, KF_SLEEPABLE)
-BTF_KFUNCS_END(bpf_test_modify_return_ids)
-
-static const struct btf_kfunc_id_set bpf_test_modify_return_set = {
- .owner = THIS_MODULE,
- .set = &bpf_test_modify_return_ids,
-};
+BTF_FMODRET_END(bpf_test_modify_return_ids)
BTF_KFUNCS_START(test_sk_check_kfunc_ids)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_release, KF_RELEASE)
@@ -1905,8 +1900,7 @@ static int __init bpf_prog_test_run_init(void)
};
int ret;
- ret = register_btf_fmodret_id_set(&bpf_test_modify_return_set);
- ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
+ ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_prog_test_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_prog_test_kfunc_set);
return ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc,
diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 4cc16cbeb328..d0d489624689 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -20,17 +20,6 @@ struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
return NULL;
}
-BTF_SET8_START(bpf_mptcp_fmodret_ids)
+BTF_FMODRET_START(bpf_mptcp_fmodret_ids)
BTF_ID_FLAGS(func, update_socket_protocol)
-BTF_SET8_END(bpf_mptcp_fmodret_ids)
-
-static const struct btf_kfunc_id_set bpf_mptcp_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_mptcp_fmodret_ids,
-};
-
-static int __init bpf_mptcp_kfunc_init(void)
-{
- return register_btf_fmodret_id_set(&bpf_mptcp_fmodret_set);
-}
-late_initcall(bpf_mptcp_kfunc_init);
+BTF_FMODRET_END(bpf_mptcp_fmodret_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 41ff36f0ca77..91cb311a2dc8 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -2607,15 +2607,9 @@ struct bpf_struct_ops testmod_multi_st_ops = {
extern int bpf_fentry_test1(int a);
-BTF_KFUNCS_START(bpf_testmod_trampoline_count_ids)
+BTF_FMODRET_START(bpf_testmod_trampoline_count_ids)
BTF_ID_FLAGS(func, bpf_testmod_trampoline_count_test)
-BTF_KFUNCS_END(bpf_testmod_trampoline_count_ids)
-
-static const struct
-btf_kfunc_id_set bpf_testmod_trampoline_count_fmodret_set = {
- .owner = THIS_MODULE,
- .set = &bpf_testmod_trampoline_count_ids,
-};
+BTF_FMODRET_END(bpf_testmod_trampoline_count_ids)
static int bpf_testmod_init(void)
{
@@ -2633,7 +2627,6 @@ static int bpf_testmod_init(void)
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set);
ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_testmod_kfunc_set);
- ret = ret ?: register_btf_fmodret_id_set(&bpf_testmod_trampoline_count_fmodret_set);
ret = ret ?: register_bpf_struct_ops(&bpf_bpf_testmod_ops, bpf_testmod_ops);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops2, bpf_testmod_ops2);
ret = ret ?: register_bpf_struct_ops(&bpf_testmod_ops3, bpf_testmod_ops3);
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 06/18] selftests/bpf: Test fmod_ret BTF decl tags and attach permission
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (3 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 05/18] bpf: Declare fmod_ret targets with BTF_FMODRET sets Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 07/18] scripts: Add LLVM plugin infrastructure Anton Protopopov
` (11 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Check that resolve_btfids tags the fmod_ret targets of vmlinux and of
bpf_testmod with "bpf_fmodret", that only bpf_fentry_test1 has the
"bpf_fmodret_sleepable" tag, and that a function which is not a target
has none.
Load minimal fmod_ret programs to check that the kernel builds its
target set from the tags: a target loads, a function which is not a
target does not, and sleepable programs load only for the sleepable
target.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
.../selftests/bpf/prog_tests/fmodret_tags.c | 155 ++++++++++++++++++
1 file changed, 155 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
new file mode 100644
index 000000000000..62efe11af826
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_tags.c
@@ -0,0 +1,155 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <linux/filter.h>
+#include <bpf/btf.h>
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+
+static bool func_has_tag(const struct btf *btf, const char *func, const char *tag)
+{
+ int func_id, id, nr_types;
+
+ func_id = btf__find_by_name_kind(btf, func, BTF_KIND_FUNC);
+ if (func_id < 0)
+ return false;
+
+ nr_types = btf__type_cnt(btf);
+ for (id = 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_decl_tag(t) || t->type != func_id ||
+ btf_decl_tag(t)->component_idx != -1)
+ continue;
+ if (!strcmp(btf__name_by_offset(btf, t->name_off), tag))
+ return true;
+ }
+
+ return false;
+}
+
+static void test_vmlinux_tags(void)
+{
+ struct btf *btf;
+
+ btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(btf, "btf__load_vmlinux_btf"))
+ return;
+
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET), "modify_return_test");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET),
+ "modify_return_test2");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET),
+ "modify_return_test_tp");
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET), "fentry_test1");
+ ASSERT_TRUE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET),
+ "update_socket_protocol");
+
+ /* Only bpf_fentry_test1 is a sleepable target */
+ ASSERT_TRUE(func_has_tag(btf, "bpf_fentry_test1", TAG_FMODRET_SLEEPABLE),
+ "fentry_test1_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test2", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test2_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_modify_return_test_tp", TAG_FMODRET_SLEEPABLE),
+ "modify_return_test_tp_sleepable");
+ ASSERT_FALSE(func_has_tag(btf, "update_socket_protocol", TAG_FMODRET_SLEEPABLE),
+ "update_socket_protocol_sleepable");
+
+ /* A function which is not a target has no tag */
+ ASSERT_GT(btf__find_by_name_kind(btf, "bpf_fentry_test2", BTF_KIND_FUNC), 0,
+ "fentry_test2_exists");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET), "fentry_test2");
+ ASSERT_FALSE(func_has_tag(btf, "bpf_fentry_test2", TAG_FMODRET_SLEEPABLE),
+ "fentry_test2_sleepable");
+
+ btf__free(btf);
+}
+
+static void test_module_tags(void)
+{
+ struct btf *vmlinux_btf, *module_btf;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ return;
+ module_btf = btf__load_module_btf("bpf_testmod", vmlinux_btf);
+ if (!module_btf && errno == ENOENT) {
+ printf("%s:SKIP:bpf_testmod has no BTF\n", __func__);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(module_btf, "btf__load_module_btf"))
+ goto out;
+
+ ASSERT_TRUE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test", TAG_FMODRET),
+ "trampoline_count_test");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_trampoline_count_test",
+ TAG_FMODRET_SLEEPABLE), "trampoline_count_test_sleepable");
+ ASSERT_FALSE(func_has_tag(module_btf, "bpf_testmod_fentry_test1", TAG_FMODRET),
+ "testmod_fentry_test1");
+
+ btf__free(module_btf);
+out:
+ btf__free(vmlinux_btf);
+}
+
+/* Load a minimal fmod_ret program which attaches to function BTF_ID */
+static int load_fmod_ret(int btf_id, __u32 prog_flags)
+{
+ struct bpf_insn insns[] = {
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ };
+ LIBBPF_OPTS(bpf_prog_load_opts, opts,
+ .expected_attach_type = BPF_MODIFY_RETURN,
+ .prog_flags = prog_flags,
+ .attach_btf_id = btf_id,
+ );
+
+ return bpf_prog_load(BPF_PROG_TYPE_TRACING, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+}
+
+static void check_load(const char *func, __u32 prog_flags, bool allowed, const char *name)
+{
+ int btf_id, fd;
+
+ btf_id = libbpf_find_vmlinux_btf_id(func, BPF_MODIFY_RETURN);
+ if (!ASSERT_GT(btf_id, 0, func))
+ return;
+ fd = load_fmod_ret(btf_id, prog_flags);
+ if (allowed)
+ ASSERT_GE(fd, 0, name);
+ else
+ ASSERT_LT(fd, 0, name);
+ if (fd >= 0)
+ close(fd);
+}
+
+static void test_attach_permission(void)
+{
+ check_load("bpf_modify_return_test", 0, true, "target");
+ check_load("bpf_fentry_test1", 0, true, "sleepable_target");
+ check_load("bpf_fentry_test2", 0, false, "not_a_target");
+
+ check_load("bpf_fentry_test1", BPF_F_SLEEPABLE, true, "sleepable_prog_sleepable_target");
+ check_load("bpf_modify_return_test", BPF_F_SLEEPABLE, false,
+ "sleepable_prog_nonsleepable_target");
+}
+
+void test_fmodret_tags(void)
+{
+ if (test__start_subtest("vmlinux"))
+ test_vmlinux_tags();
+ if (test__start_subtest("module"))
+ test_module_tags();
+ if (test__start_subtest("attach_permission"))
+ test_attach_permission();
+}
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 07/18] scripts: Add LLVM plugin infrastructure
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (4 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 06/18] selftests/bpf: Test fmod_ret BTF decl tags and attach permission Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 08/18] bpf: Add the bpf_scan_ir LLVM plugin Anton Protopopov
` (10 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Add scripts/llvm-plugins and the corresponding Makefiles and Kconfig for
LLVM IR passes loaded with -fpass-plugin, similar to the GCC plugins.
Individual plugins register their configuration and shared objects
separately. No plugin is added yet.
Use LLVM_CONFIG to obtain the host compiler and linker flags. Require
llvm-config of the same major version as clang, which loads the plugins:
unlike GCC, clang does not provide the headers for its plugins. Require
also a host C++ compiler which accepts the headers and flags of
llvm-config.
Build plugins without linking LLVM component libraries; their LLVM
symbols are resolved by the compiler loading the shared library. Include
a generated header so objects are rebuilt when enabled plugins change.
Track the headers of a plugin, LLVM ones included, with a dependency
file, as the GCC plugins do.
Export LLVM_PLUGINS_CFLAGS so vDSO, EFI stub and early boot builds can
exclude the plugins, as they exclude the GCC plugins.
Add the plugin infrastructure to the CLANG/LLVM BUILD SUPPORT entry in
MAINTAINERS.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
MAINTAINERS | 2 ++
Makefile | 6 +++++-
arch/Kconfig | 1 +
arch/arm64/kernel/pi/Makefile | 1 +
arch/x86/boot/startup/Makefile | 2 ++
arch/x86/um/vdso/Makefile | 2 +-
drivers/firmware/efi/libstub/Makefile | 2 +-
include/linux/compiler-version.h | 5 +++++
lib/vdso/Makefile.include | 3 +++
scripts/Makefile | 1 +
scripts/Makefile.llvm-plugins | 8 ++++++++
scripts/llvm-plugins/Kconfig | 17 +++++++++++++++++
scripts/llvm-plugins/Makefile | 21 +++++++++++++++++++++
13 files changed, 68 insertions(+), 3 deletions(-)
create mode 100644 scripts/Makefile.llvm-plugins
create mode 100644 scripts/llvm-plugins/Kconfig
create mode 100644 scripts/llvm-plugins/Makefile
diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065..02b6c8963c63 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6562,7 +6562,9 @@ C: irc://irc.libera.chat/clangbuiltlinux
F: Documentation/kbuild/llvm.rst
F: include/linux/compiler-clang.h
F: scripts/Makefile.clang
+F: scripts/Makefile.llvm-plugins
F: scripts/clang-tools/
+F: scripts/llvm-plugins/
K: \b(?i:clang|llvm)\b
CLK API
diff --git a/Makefile b/Makefile
index f895752d4b05..2fcad8109572 100644
--- a/Makefile
+++ b/Makefile
@@ -457,10 +457,13 @@ endif
HOSTCC = $(LLVM_PREFIX)clang$(LLVM_SUFFIX)
HOSTCXX = $(LLVM_PREFIX)clang++$(LLVM_SUFFIX)
+LLVM_CONFIG ?= $(LLVM_PREFIX)llvm-config$(LLVM_SUFFIX)
else
HOSTCC = gcc
HOSTCXX = g++
+LLVM_CONFIG ?= llvm-config
endif
+export LLVM_CONFIG
HOSTRUSTC = rustc
HOSTPKG_CONFIG = pkg-config
@@ -1211,10 +1214,11 @@ include-$(CONFIG_AUTOFDO_CLANG) += scripts/Makefile.autofdo
include-$(CONFIG_PROPELLER_CLANG) += scripts/Makefile.propeller
include-$(CONFIG_WARN_CONTEXT_ANALYSIS) += scripts/Makefile.context-analysis
include-$(CONFIG_GCC_PLUGINS) += scripts/Makefile.gcc-plugins
+include-$(CONFIG_LLVM_PLUGINS) += scripts/Makefile.llvm-plugins
include $(addprefix $(srctree)/, $(include-y))
-# scripts/Makefile.gcc-plugins is intentionally included last.
+# The compiler plugin Makefiles are intentionally included last.
# Do not add $(call cc-option,...) below this line. When you build the kernel
# from the clean source tree, the GCC plugins do not exist at this point.
diff --git a/arch/Kconfig b/arch/Kconfig
index 1163736de1f9..4f1f36d3e82d 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1772,6 +1772,7 @@ config HAVE_GENERIC_TIF_BITS
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"
+source "scripts/llvm-plugins/Kconfig"
config FUNCTION_ALIGNMENT_4B
bool
diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile
index a273ab5af805..5813bb75c992 100644
--- a/arch/arm64/kernel/pi/Makefile
+++ b/arch/arm64/kernel/pi/Makefile
@@ -10,6 +10,7 @@ KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
-D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \
-fno-asynchronous-unwind-tables -fno-unwind-tables \
$(CONFIG_CC_OPT_NO_ADDRSIG)
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# this code may run with the MMU off so disable unaligned accesses
CFLAGS_map_range.o += -mstrict-align
diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile
index 5e499cfb29b5..7036e939097e 100644
--- a/arch/x86/boot/startup/Makefile
+++ b/arch/x86/boot/startup/Makefile
@@ -12,6 +12,8 @@ KBUILD_CFLAGS += -D__DISABLE_EXPORTS -mcmodel=small -fPIC \
# disable ftrace hooks and LTO
KBUILD_CFLAGS := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS))
KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO),$(KBUILD_CFLAGS))
+# LLVM plugins may add data with absolute addresses
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
KMSAN_SANITIZE := n
diff --git a/arch/x86/um/vdso/Makefile b/arch/x86/um/vdso/Makefile
index 7664cbedbe30..0e32c9ae2b5a 100644
--- a/arch/x86/um/vdso/Makefile
+++ b/arch/x86/um/vdso/Makefile
@@ -37,7 +37,7 @@ CFL := $(PROFILING) -mcmodel=small -fPIC -O2 -fasynchronous-unwind-tables -m64 \
$(filter -g%,$(KBUILD_CFLAGS)) -fno-stack-protector \
-fno-omit-frame-pointer -foptimize-sibling-calls
-$(vobjs): KBUILD_CFLAGS += $(CFL)
+$(vobjs): KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS)) $(CFL)
#
# vDSO code runs in userspace and -pg doesn't help with profiling anyway.
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index bbab12b8eb4b..6f69d5d67731 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -6,7 +6,7 @@
# enabled, even if doing so doesn't break the build.
#
-cflags-y := $(KBUILD_CFLAGS)
+cflags-y := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
# x86 links the stub into the decompressor rather than the kernel proper, so
# drop the kernel's code model, calling convention, and stack alignment, and
diff --git a/include/linux/compiler-version.h b/include/linux/compiler-version.h
index ac1665a98a15..e6a5a9f3fbcd 100644
--- a/include/linux/compiler-version.h
+++ b/include/linux/compiler-version.h
@@ -24,6 +24,11 @@
#include <generated/gcc-plugins.h>
#endif
+/* Rebuild objects when an LLVM plugin changes. */
+#ifdef LLVM_PLUGINS
+#include <generated/llvm-plugins.h>
+#endif
+
/*
* If the randstruct seed itself changes (whether for GCC plugins or
* Clang), the entire tree needs to be rebuilt since the randomization of
diff --git a/lib/vdso/Makefile.include b/lib/vdso/Makefile.include
index cedbf15f8087..848d58296ba2 100644
--- a/lib/vdso/Makefile.include
+++ b/lib/vdso/Makefile.include
@@ -1,5 +1,8 @@
# SPDX-License-Identifier: GPL-2.0
+# vDSO code executes outside the kernel.
+KBUILD_CFLAGS := $(filter-out $(LLVM_PLUGINS_CFLAGS),$(KBUILD_CFLAGS))
+
GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
diff --git a/scripts/Makefile b/scripts/Makefile
index d7fbb7ca098b..bc5dacc29207 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -66,6 +66,7 @@ hostprogs += unifdef gen_packed_field_checks
targets += module.lds
subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
+subdir-$(CONFIG_LLVM_PLUGINS) += llvm-plugins
subdir-$(CONFIG_GENKSYMS) += genksyms
subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms
subdir-$(CONFIG_SECURITY_SELINUX) += selinux
diff --git a/scripts/Makefile.llvm-plugins b/scripts/Makefile.llvm-plugins
new file mode 100644
index 000000000000..e18f4907b015
--- /dev/null
+++ b/scripts/Makefile.llvm-plugins
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+LLVM_PLUGIN := $(llvm-plugin-y)
+LLVM_PLUGINS_CFLAGS := $(strip $(addprefix -fpass-plugin=$(objtree)/scripts/llvm-plugins/, \
+ $(LLVM_PLUGIN)) -DLLVM_PLUGINS)
+export LLVM_PLUGIN LLVM_PLUGINS_CFLAGS
+
+KBUILD_CFLAGS += $(LLVM_PLUGINS_CFLAGS)
diff --git a/scripts/llvm-plugins/Kconfig b/scripts/llvm-plugins/Kconfig
new file mode 100644
index 000000000000..37db99159b9c
--- /dev/null
+++ b/scripts/llvm-plugins/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+menuconfig LLVM_PLUGINS
+ bool "LLVM plugins"
+ depends on CC_IS_CLANG
+ depends on $(success,$(LLVM_CONFIG) --version)
+ # The plugins are built against the headers of llvm-config and loaded
+ # by clang, so both must come from the same LLVM release
+ depends on $(success,test "$(shell,$(LLVM_CONFIG) --version 2>/dev/null | cut -d. -f1)" = "$(shell,$(CC) -dumpversion | cut -d. -f1)")
+ # The host C++ compiler must accept the LLVM headers and flags
+ depends on $(success,echo '#include <llvm/Config/llvm-config.h>' | $(HOSTCXX) $(shell,$(LLVM_CONFIG) --cxxflags 2>/dev/null) -x c++ -fsyntax-only -)
+ default y
+ help
+ LLVM plugins add compiler passes to kernel and module builds.
+ Building them requires LLVM development headers and llvm-config
+ of the same major version as clang. Each plugin has its own option
+ below and is built only if that option is enabled.
diff --git a/scripts/llvm-plugins/Makefile b/scripts/llvm-plugins/Makefile
new file mode 100644
index 000000000000..a1f8503c8bcd
--- /dev/null
+++ b/scripts/llvm-plugins/Makefile
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+always-y += $(LLVM_PLUGIN)
+targets += $(LLVM_PLUGIN)
+clean-files += *.so
+
+# Run llvm-config only to build a plugin, not when this Makefile is parsed
+llvm_cxxflags = $(shell $(LLVM_CONFIG) --cxxflags)
+llvm_ldflags = $(shell $(LLVM_CONFIG) --ldflags)
+
+quiet_cmd_llvm_plugin = HOSTCXX $@
+ cmd_llvm_plugin = $(HOSTCXX) -Wp,-MD,$(depfile) $(KBUILD_HOSTCXXFLAGS) \
+ $(llvm_cxxflags) -fPIC -shared $(KBUILD_HOSTLDFLAGS) -o $@ $< \
+ $(llvm_ldflags)
+
+$(obj)/%.so: $(src)/%.cpp FORCE
+ $(call if_changed_dep,llvm_plugin)
+
+$(obj)/../../include/generated/llvm-plugins.h: $(addprefix $(obj)/, $(LLVM_PLUGIN)) FORCE
+ $(call if_changed,touch)
+always-y += ../../include/generated/llvm-plugins.h
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 08/18] bpf: Add the bpf_scan_ir LLVM plugin
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (5 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 07/18] scripts: Add LLVM plugin infrastructure Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 09/18] bpf: Find fmod_ret candidates in bpf_scan_ir Anton Protopopov
` (9 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Add a new bpf_scan_ir.cpp LLVM plugin selected by the new
CONFIG_CLANG_PLUGIN_BPF_SCAN_IR option. The new plugin depends on
DEBUG_INFO_BTF, as the information it produces eventually ends up in
kernel/module BTF. It needs LLVM 19 or later and supports only x86_64
and arm64. For now LTO_CLANG is not supported. It is a planned
follow-up. The new pass runs after optimization over the IR of the
kernel and modules.
The purpose of this plugin is to gather information about functions,
which can be targets for BPF fmodret programs (have paths without side
effects which return errors). The records about functions are stored in
the new .BTF_fmodret_candidates section which is created by plugin and
later is processed by the resolve_btfids tool. The actual plugin logic is
added in the following commits.
Keep .BTF_fmodret_candidates and its relocations in the temporary BTF
link and discard it before the later links, so the build-only function
references do not create runtime relocations. Keep it in modules until
their BTF is generated and remove it then, also when BTF generation is
skipped. Without module BTF, discard it at the module link. Let modpost
accept the section. Let objtool skip its relocations in IBT validation:
a record has the address of its function, which is not an indirect
branch target. The vDSO, the EFI stub and the early boot code are built
without the plugins, so position-independent startup objects do not
contain the absolute function pointers of the records.
The record format and the linker script are in scripts/bpf, so a future
GCC plugin can share them.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
MAINTAINERS | 7 ++
include/asm-generic/vmlinux.lds.h | 3 +-
scripts/Makefile.llvm-plugins | 2 +
scripts/Makefile.modfinal | 1 +
scripts/Makefile.vmlinux | 3 +
scripts/bpf/fmodret.lds | 6 ++
scripts/bpf/fmodret_metadata.h | 29 ++++++
scripts/gen-btf.sh | 4 +-
scripts/link-vmlinux.sh | 7 ++
scripts/llvm-plugins/Kconfig | 16 ++++
scripts/llvm-plugins/bpf_scan_ir.cpp | 126 +++++++++++++++++++++++++++
scripts/mod/modpost.c | 1 +
scripts/module.lds.S | 5 ++
tools/objtool/check.c | 1 +
14 files changed, 209 insertions(+), 2 deletions(-)
create mode 100644 scripts/bpf/fmodret.lds
create mode 100644 scripts/bpf/fmodret_metadata.h
create mode 100644 scripts/llvm-plugins/bpf_scan_ir.cpp
diff --git a/MAINTAINERS b/MAINTAINERS
index 02b6c8963c63..dfbb45ecc716 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5064,6 +5064,13 @@ S: Supported
F: arch/x86/net/
X: arch/x86/net/bpf_jit_comp32.c
+BPF [COMPILER PLUGINS]
+M: Anton Protopopov <a.s.protopopov@gmail.com>
+L: bpf@vger.kernel.org
+S: Maintained
+F: scripts/bpf/
+F: scripts/llvm-plugins/bpf_*
+
BPF [CORE]
M: Alexei Starovoitov <ast@kernel.org>
M: Daniel Borkmann <daniel@iogearbox.net>
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index f8254a88e05d..4e18758b5ffc 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -886,7 +886,8 @@
KLP_SYMID
#define MODINFO \
- .modinfo (INFO) : { KEEP(*(.modinfo)) }
+ .modinfo (INFO) : { KEEP(*(.modinfo)) } \
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
#ifdef CONFIG_GENERIC_BUG
#define BUG_TABLE \
diff --git a/scripts/Makefile.llvm-plugins b/scripts/Makefile.llvm-plugins
index e18f4907b015..9ff364f4db59 100644
--- a/scripts/Makefile.llvm-plugins
+++ b/scripts/Makefile.llvm-plugins
@@ -1,5 +1,7 @@
# SPDX-License-Identifier: GPL-2.0-only
+llvm-plugin-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_scan_ir.so
+
LLVM_PLUGIN := $(llvm-plugin-y)
LLVM_PLUGINS_CFLAGS := $(strip $(addprefix -fpass-plugin=$(objtree)/scripts/llvm-plugins/, \
$(LLVM_PLUGIN)) -DLLVM_PLUGINS)
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 01a37ec872b9..042c31e3bdfb 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -42,6 +42,7 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f $(objtree)/vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
+ $(OBJCOPY) --remove-section=.BTF_fmodret_candidates $@; \
else \
$(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux
index 95b523fe4f9e..c8651454c3d3 100644
--- a/scripts/Makefile.vmlinux
+++ b/scripts/Makefile.vmlinux
@@ -73,6 +73,9 @@ vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD
ifdef CONFIG_DEBUG_INFO_BTF
vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh
endif
+ifdef CONFIG_CLANG_PLUGIN_BPF_SCAN_IR
+vmlinux.unstripped: $(srctree)/scripts/bpf/fmodret.lds
+endif
ifdef CONFIG_BUILDTIME_TABLE_SORT
vmlinux.unstripped: scripts/sorttable
diff --git a/scripts/bpf/fmodret.lds b/scripts/bpf/fmodret.lds
new file mode 100644
index 000000000000..6bdfde9e7679
--- /dev/null
+++ b/scripts/bpf/fmodret.lds
@@ -0,0 +1,6 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+SECTIONS {
+ /DISCARD/ : {
+ *(.BTF_fmodret_candidates)
+ }
+}
diff --git a/scripts/bpf/fmodret_metadata.h b/scripts/bpf/fmodret_metadata.h
new file mode 100644
index 000000000000..209f266f89a3
--- /dev/null
+++ b/scripts/bpf/fmodret_metadata.h
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef FMODRET_METADATA_H
+#define FMODRET_METADATA_H
+
+/*
+ * The records which the BPF compiler plugins write to .BTF_fmodret_candidates
+ * for resolve_btfids, one for each function with values and for each weak
+ * definition. If a function has two weak definitions, the record of the one
+ * which loses the link would describe the body of the other, so both need a
+ * record to make the function ambiguous. A record is packed, target-endian and
+ * aligned to the target pointer size:
+ *
+ * u32 magic, version, size, flags;
+ * ptr function;
+ * u32 nr_values, reserved;
+ * s64 values[nr_values];
+ *
+ * The values are the errors which clean paths of the function return, sorted
+ * and unique.
+ */
+#define FMODRET_META_MAGIC 0x46524d32U
+#define FMODRET_META_VERSION 1U
+#define FMODRET_META_HEADER_SIZE(ptr_size) (24 + (ptr_size))
+#define FMODRET_META_VALUE_SIZE 8
+
+#define FMODRET_FN_WEAK (1U << 0)
+#define FMODRET_FN_RET64 (1U << 1)
+
+#endif
diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
index b17f1189298a..5bca864631e0 100755
--- a/scripts/gen-btf.sh
+++ b/scripts/gen-btf.sh
@@ -120,7 +120,9 @@ gen_btf_o()
embed_btf_data()
{
- ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF ${ELF_FILE}
+ # resolve_btfids has consumed the compiler plugin records of the module
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.BTF \
+ --remove-section=.BTF_fmodret_candidates ${ELF_FILE}
# a module might not have a .BTF_ids or .BTF.base section
btf_base="${ELF_FILE}.BTF.base"
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d5710e900bd4..f39e2d1f1400 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -89,6 +89,13 @@ vmlinux_link()
ldlibs=
fi
+ if is_enabled CONFIG_CLANG_PLUGIN_BPF_SCAN_IR; then
+ if [ "${output}" = ".tmp_vmlinux1" ]; then
+ ldflags="${ldflags} ${wl}--emit-relocs"
+ else
+ ldflags="${ldflags} ${wl}--script=${srctree}/scripts/bpf/fmodret.lds"
+ fi
+ fi
ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}"
# The kallsyms linking does not need debug symbols included.
diff --git a/scripts/llvm-plugins/Kconfig b/scripts/llvm-plugins/Kconfig
index 37db99159b9c..c4d62ae5f65d 100644
--- a/scripts/llvm-plugins/Kconfig
+++ b/scripts/llvm-plugins/Kconfig
@@ -15,3 +15,19 @@ menuconfig LLVM_PLUGINS
Building them requires LLVM development headers and llvm-config
of the same major version as clang. Each plugin has its own option
below and is built only if that option is enabled.
+
+if LLVM_PLUGINS
+
+config CLANG_PLUGIN_BPF_SCAN_IR
+ bool "Scan optimized LLVM IR for BPF"
+ depends on X86_64 || ARM64
+ depends on DEBUG_INFO_BTF
+ depends on !LTO_CLANG
+ depends on CLANG_VERSION >= 190000
+ help
+ Find the functions which return errors on paths without side
+ effects and describe them in .BTF_fmodret_candidates for
+ resolve_btfids. The section is discarded from the final kernel
+ and modules.
+
+endif
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
new file mode 100644
index 000000000000..6700896749d4
--- /dev/null
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* bpf_scan_ir: optimizer-last LLVM IR analysis for BPF. */
+
+#include "../bpf/fmodret_metadata.h"
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Config/llvm-config.h"
+#include "llvm/IR/Analysis.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/Passes/PassBuilder.h"
+#if __has_include("llvm/Plugins/PassPlugin.h")
+#include "llvm/Plugins/PassPlugin.h"
+#else
+#include "llvm/Passes/PassPlugin.h"
+#endif
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+
+using namespace llvm;
+
+namespace {
+
+/* The errors which clean paths of a function return */
+using ReturnValues = SmallVector<int64_t, 8>;
+
+struct FunctionDescription {
+ Function *Fn;
+ unsigned Flags;
+ ReturnValues Values;
+};
+
+static void emitDescriptions(Module &M, ArrayRef<FunctionDescription> Functions)
+{
+ auto *I32 = Type::getInt32Ty(M.getContext());
+ auto *I64 = Type::getInt64Ty(M.getContext());
+ unsigned PtrSize = M.getDataLayout().getPointerSize();
+ auto N = [&](unsigned V) -> Constant * { return ConstantInt::get(I32, V); };
+
+ for (const auto &F : Functions) {
+ unsigned Size = FMODRET_META_HEADER_SIZE(PtrSize) +
+ F.Values.size() * FMODRET_META_VALUE_SIZE;
+ SmallVector<Constant *, 16> Fields = {
+ N(FMODRET_META_MAGIC), N(FMODRET_META_VERSION), N(Size), N(F.Flags),
+ F.Fn, N(F.Values.size()), N(0),
+ };
+
+ for (int64_t V : F.Values)
+ Fields.push_back(ConstantInt::getSigned(I64, V));
+ Constant *Record = ConstantStruct::getAnon(M.getContext(), Fields, true);
+ auto *GV = new GlobalVariable(M, Record->getType(), true, GlobalValue::PrivateLinkage,
+ Record, "fmodret.function." + F.Fn->getName());
+ GV->setSection(".BTF_fmodret_candidates");
+ GV->setAlignment(M.getDataLayout().getPointerABIAlignment(0));
+ appendToUsed(M, {GV});
+ }
+}
+
+#if LLVM_VERSION_MAJOR >= 24
+class BpfScanIRPass : public RequiredPassInfoMixin<BpfScanIRPass>
+#else
+class BpfScanIRPass : public PassInfoMixin<BpfScanIRPass>
+#endif
+{
+public:
+ PreservedAnalyses run(Module &M, ModuleAnalysisManager &MAM)
+ {
+ // Avoid emitting duplicate descriptions if bpf_scan_ir runs again
+ // on the same module.
+ if (M.getNamedMetadata("bpf_scan_ir.done"))
+ return PreservedAnalyses::all();
+
+ SmallVector<FunctionDescription, 32> Descriptions;
+
+ for (Function &F : M) {
+ if (F.isIntrinsic() || F.isDeclaration() || F.hasAvailableExternallyLinkage())
+ continue;
+ Type *Ret = F.getReturnType();
+ unsigned Flags = 0;
+ ReturnValues Values;
+
+ if (!Ret->isIntegerTy(32) && !Ret->isIntegerTy(64))
+ continue;
+ if (F.isWeakForLinker())
+ Flags |= FMODRET_FN_WEAK;
+ if (Ret->isIntegerTy(64))
+ Flags |= FMODRET_FN_RET64;
+ /*
+ * Nothing finds the values yet. A weak definition always
+ * has a record, see fmodret_metadata.h.
+ */
+ if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
+ Descriptions.push_back({&F, Flags, std::move(Values)});
+ }
+ if (Descriptions.empty())
+ return PreservedAnalyses::all();
+ emitDescriptions(M, Descriptions);
+ M.getOrInsertNamedMetadata("bpf_scan_ir.done");
+
+ PreservedAnalyses PA;
+ PA.preserve<FunctionAnalysisManagerModuleProxy>();
+ PA.preserveSet<CFGAnalyses>();
+ return PA;
+ }
+
+ static bool isRequired() { return true; }
+};
+
+} // namespace
+
+extern "C" LLVM_ATTRIBUTE_WEAK PassPluginLibraryInfo llvmGetPassPluginInfo()
+{
+ return {
+ LLVM_PLUGIN_API_VERSION,
+ "bpf_scan_ir",
+ "1.0",
+ [](PassBuilder &PB) {
+ PB.registerOptimizerLastEPCallback(
+ [](ModulePassManager &MPM, OptimizationLevel, auto...) {
+ MPM.addPass(BpfScanIRPass());
+ }
+ );
+ }
+ };
+}
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 75374c64b8cc..3bae98549228 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -772,6 +772,7 @@ static const char *const section_white_list[] =
".llvm.call-graph-profile", /* call graph */
"__llvm_covfun",
"__llvm_covmap",
+ ".BTF_fmodret_candidates",
".klp.symid", /* objtool --klp-symids */
NULL
};
diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index d0f200428957..e6ba499dc876 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -27,6 +27,11 @@ SECTIONS {
__ex_table 0 : ALIGN(4) { KEEP(*(__ex_table)) }
__patchable_function_entries 0 : { *(__patchable_function_entries) }
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+ .BTF_fmodret_candidates 0 (INFO) : { KEEP(*(.BTF_fmodret_candidates)) }
+#else
+ /DISCARD/ : { *(.BTF_fmodret_candidates) }
+#endif
.init.klp_funcs 0 : ALIGN(8) { KEEP(*(.init.klp_funcs)) }
.init.klp_objects 0 : ALIGN(8) { KEEP(*(.init.klp_objects)) }
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index 931c5c412f15..e6120bee4505 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -4657,6 +4657,7 @@ static int validate_ibt(struct objtool_file *file)
!strcmp(sec->name, "__jump_table") ||
!strcmp(sec->name, ".init.klp_funcs") ||
!strcmp(sec->name, "__mcount_loc") ||
+ !strcmp(sec->name, ".BTF_fmodret_candidates") ||
!strcmp(sec->name, ".llvm.call-graph-profile") ||
!strcmp(sec->name, ".llvm_bb_addr_map") ||
!strcmp(sec->name, "__tracepoints") ||
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 09/18] bpf: Find fmod_ret candidates in bpf_scan_ir
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (6 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 08/18] bpf: Add the bpf_scan_ir LLVM plugin Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:02 ` [PATCH v1 bpf-next 10/18] bpf: Allow writes to private stack objects " Anton Protopopov
` (8 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Find the functions which can become fmod_ret targets: functions which
return a 32 or 64 bit integer and have a path from the entry without
observable side effects (a clean path) which returns an error. A
fmod_ret program which returns this error instead of calling the
function then behaves like the path. List the errors in the record of
the function, sorted and unique.
Take only the blocks which no dirty path reaches: after a dirty path
joins a clean one, a branch may depend on what the dirty path did. Only
the block of the return may also be reached by dirty paths, as it
returns the same constant on all paths.
For now an error must be a constant returned by a clean path. The
following changes find more of the errors.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
scripts/llvm-plugins/bpf_scan_ir.cpp | 117 ++++++++++++++++++++++++++-
1 file changed, 113 insertions(+), 4 deletions(-)
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
index 6700896749d4..6828fa73717d 100644
--- a/scripts/llvm-plugins/bpf_scan_ir.cpp
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -3,11 +3,17 @@
#include "../bpf/fmodret_metadata.h"
+#include <algorithm>
+#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/Config/llvm-config.h"
#include "llvm/IR/Analysis.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/CFG.h"
#include "llvm/IR/Constants.h"
#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/PassManager.h"
#include "llvm/Passes/PassBuilder.h"
@@ -22,9 +28,114 @@ using namespace llvm;
namespace {
+static constexpr int MaxErrno = 4095;
+
/* The errors which clean paths of a function return */
using ReturnValues = SmallVector<int64_t, 8>;
+/* Return true and set N if V is an errno constant */
+static bool errnoConstant(const Value *V, int64_t &N)
+{
+ auto *C = dyn_cast<ConstantInt>(V);
+
+ if (!C || (C->getBitWidth() != 32 && C->getBitWidth() != 64))
+ return false;
+
+ N = C->getSExtValue();
+ return N >= -MaxErrno && N < 0;
+}
+
+static bool cleanInstruction(const Instruction &I)
+{
+ if (auto *Intrinsic = dyn_cast<IntrinsicInst>(&I)) {
+ switch (Intrinsic->getIntrinsicID()) {
+ case Intrinsic::assume:
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ case Intrinsic::invariant_start:
+ case Intrinsic::invariant_end:
+ return true;
+ default:
+ break;
+ }
+ }
+ return !I.mayHaveSideEffects();
+}
+
+static bool cleanBB(BasicBlock *BB)
+{
+ for (Instruction &I : *BB) {
+ if (!cleanInstruction(I))
+ return false;
+ }
+ return true;
+}
+
+/* Add the errors which clean paths of F are known to return to VALUES */
+static void cleanReturnValues(Function &F, ReturnValues &Values)
+{
+ SmallPtrSet<BasicBlock *, 32> CleanBBs;
+ SmallVector<BasicBlock *, 32> Worklist;
+ bool Changed;
+
+ /* The clean blocks which the entry reaches through clean blocks */
+ if (cleanBB(&F.getEntryBlock())) {
+ CleanBBs.insert(&F.getEntryBlock());
+ Worklist.push_back(&F.getEntryBlock());
+ }
+ while (!Worklist.empty()) {
+ BasicBlock *BB = Worklist.pop_back_val();
+
+ for (BasicBlock *Successor : successors(BB)) {
+ if (!CleanBBs.contains(Successor) && cleanBB(Successor)) {
+ CleanBBs.insert(Successor);
+ Worklist.push_back(Successor);
+ }
+ }
+ }
+
+ /*
+ * Keep the blocks which only clean paths reach: after a dirty path joins
+ * a clean one, a branch may depend on what the dirty path did
+ */
+ do {
+ Changed = false;
+ for (BasicBlock &BB : F) {
+ if (!CleanBBs.contains(&BB))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB)) {
+ if (!CleanBBs.contains(Pred)) {
+ CleanBBs.erase(&BB);
+ Changed = true;
+ break;
+ }
+ }
+ }
+ } while (Changed);
+
+ /*
+ * The returns of the clean blocks which only clean paths reach or which
+ * one of them branches to. Dirty paths may also reach the latter, but
+ * the returned constant is the same on all paths.
+ */
+ for (BasicBlock &BB : F) {
+ auto *Return = dyn_cast<ReturnInst>(BB.getTerminator());
+ bool Entered = CleanBBs.contains(&BB);
+ int64_t N;
+
+ if (!Return || !Return->getReturnValue() || !cleanBB(&BB))
+ continue;
+ for (BasicBlock *Pred : predecessors(&BB))
+ Entered |= CleanBBs.contains(Pred);
+ if (!Entered)
+ continue;
+ if (errnoConstant(Return->getReturnValue(), N))
+ Values.push_back(N);
+ }
+ llvm::sort(Values);
+ Values.erase(std::unique(Values.begin(), Values.end()), Values.end());
+}
+
struct FunctionDescription {
Function *Fn;
unsigned Flags;
@@ -86,10 +197,8 @@ public:
Flags |= FMODRET_FN_WEAK;
if (Ret->isIntegerTy(64))
Flags |= FMODRET_FN_RET64;
- /*
- * Nothing finds the values yet. A weak definition always
- * has a record, see fmodret_metadata.h.
- */
+ cleanReturnValues(F, Values);
+ /* A weak definition always has a record, see fmodret_metadata.h */
if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
Descriptions.push_back({&F, Flags, std::move(Values)});
}
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 10/18] bpf: Allow writes to private stack objects in bpf_scan_ir
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (7 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 09/18] bpf: Find fmod_ret candidates in bpf_scan_ir Anton Protopopov
@ 2026-11-06 21:02 ` Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 11/18] bpf: Follow PHIs " Anton Protopopov
` (7 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:02 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
A write to a stack object of the function is not observable outside of
it as long as the object has not escaped. Let such writes and memsets
stay on a clean path. A write after the object may have been captured,
a volatile or atomic one and a write to other memory still make the
path dirty.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
scripts/llvm-plugins/bpf_scan_ir.cpp | 85 +++++++++++++++++++++-------
1 file changed, 64 insertions(+), 21 deletions(-)
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
index 6828fa73717d..0fc60b2130b1 100644
--- a/scripts/llvm-plugins/bpf_scan_ir.cpp
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -4,13 +4,17 @@
#include "../bpf/fmodret_metadata.h"
#include <algorithm>
+#include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallVector.h"
+#include "llvm/Analysis/CaptureTracking.h"
+#include "llvm/Analysis/ValueTracking.h"
#include "llvm/Config/llvm-config.h"
#include "llvm/IR/Analysis.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/CFG.h"
#include "llvm/IR/Constants.h"
+#include "llvm/IR/Dominators.h"
#include "llvm/IR/GlobalVariable.h"
#include "llvm/IR/Instructions.h"
#include "llvm/IR/IntrinsicInst.h"
@@ -45,41 +49,76 @@ static bool errnoConstant(const Value *V, int64_t &N)
return N >= -MaxErrno && N < 0;
}
-static bool cleanInstruction(const Instruction &I)
-{
- if (auto *Intrinsic = dyn_cast<IntrinsicInst>(&I)) {
- switch (Intrinsic->getIntrinsicID()) {
- case Intrinsic::assume:
- case Intrinsic::lifetime_start:
- case Intrinsic::lifetime_end:
- case Intrinsic::invariant_start:
- case Intrinsic::invariant_end:
- return true;
- default:
- break;
+class CleanInstructions {
+ DominatorTree &DT;
+ DenseMap<const Instruction *, bool> PrivateWrites;
+
+ bool privateDestination(const Value *Pointer, const Instruction &I)
+ {
+ auto It = PrivateWrites.find(&I);
+ if (It != PrivateWrites.end())
+ return It->second;
+ auto *Allocation = dyn_cast<AllocaInst>(getUnderlyingObject(Pointer));
+ bool Private = false;
+
+ if (Allocation && Allocation->getFunction() == I.getFunction()) {
+ // A later call may receive the initialized object. Only captures
+ // that can precede this write make the write externally visible.
+#if LLVM_VERSION_MAJOR >= 21
+ Private = !PointerMayBeCapturedBefore(Allocation, true, &I, &DT, true, 256);
+#else
+ Private = !PointerMayBeCapturedBefore(Allocation, true, true, &I, &DT, true, 256);
+#endif
}
+ PrivateWrites[&I] = Private;
+ return Private;
}
- return !I.mayHaveSideEffects();
-}
-static bool cleanBB(BasicBlock *BB)
+public:
+ explicit CleanInstructions(DominatorTree &DT) : DT(DT) {}
+
+ bool isClean(const Instruction &I)
+ {
+ if (auto *Store = dyn_cast<StoreInst>(&I))
+ return !Store->isVolatile() && !Store->isAtomic() &&
+ privateDestination(Store->getPointerOperand(), I);
+ if (auto *Memory = dyn_cast<MemIntrinsic>(&I))
+ return !Memory->isVolatile() &&
+ privateDestination(Memory->getRawDest(), I);
+ if (auto *Intrinsic = dyn_cast<IntrinsicInst>(&I)) {
+ switch (Intrinsic->getIntrinsicID()) {
+ case Intrinsic::assume:
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ case Intrinsic::invariant_start:
+ case Intrinsic::invariant_end:
+ return true;
+ default:
+ break;
+ }
+ }
+ return !I.mayHaveSideEffects();
+ }
+};
+
+static bool cleanBB(BasicBlock *BB, CleanInstructions &Clean)
{
for (Instruction &I : *BB) {
- if (!cleanInstruction(I))
+ if (!Clean.isClean(I))
return false;
}
return true;
}
/* Add the errors which clean paths of F are known to return to VALUES */
-static void cleanReturnValues(Function &F, ReturnValues &Values)
+static void cleanReturnValues(Function &F, CleanInstructions &Clean, ReturnValues &Values)
{
SmallPtrSet<BasicBlock *, 32> CleanBBs;
SmallVector<BasicBlock *, 32> Worklist;
bool Changed;
/* The clean blocks which the entry reaches through clean blocks */
- if (cleanBB(&F.getEntryBlock())) {
+ if (cleanBB(&F.getEntryBlock(), Clean)) {
CleanBBs.insert(&F.getEntryBlock());
Worklist.push_back(&F.getEntryBlock());
}
@@ -87,7 +126,7 @@ static void cleanReturnValues(Function &F, ReturnValues &Values)
BasicBlock *BB = Worklist.pop_back_val();
for (BasicBlock *Successor : successors(BB)) {
- if (!CleanBBs.contains(Successor) && cleanBB(Successor)) {
+ if (!CleanBBs.contains(Successor) && cleanBB(Successor, Clean)) {
CleanBBs.insert(Successor);
Worklist.push_back(Successor);
}
@@ -123,7 +162,7 @@ static void cleanReturnValues(Function &F, ReturnValues &Values)
bool Entered = CleanBBs.contains(&BB);
int64_t N;
- if (!Return || !Return->getReturnValue() || !cleanBB(&BB))
+ if (!Return || !Return->getReturnValue() || !cleanBB(&BB, Clean))
continue;
for (BasicBlock *Pred : predecessors(&BB))
Entered |= CleanBBs.contains(Pred);
@@ -183,6 +222,7 @@ public:
return PreservedAnalyses::all();
SmallVector<FunctionDescription, 32> Descriptions;
+ auto &FAM = MAM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
for (Function &F : M) {
if (F.isIntrinsic() || F.isDeclaration() || F.hasAvailableExternallyLinkage())
@@ -197,7 +237,10 @@ public:
Flags |= FMODRET_FN_WEAK;
if (Ret->isIntegerTy(64))
Flags |= FMODRET_FN_RET64;
- cleanReturnValues(F, Values);
+ auto &DT = FAM.getResult<DominatorTreeAnalysis>(F);
+ CleanInstructions Clean(DT);
+
+ cleanReturnValues(F, Clean, Values);
/* A weak definition always has a record, see fmodret_metadata.h */
if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
Descriptions.push_back({&F, Flags, std::move(Values)});
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 11/18] bpf: Follow PHIs in bpf_scan_ir
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (8 preceding siblings ...)
2026-11-06 21:02 ` [PATCH v1 bpf-next 10/18] bpf: Allow writes to private stack objects " Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 12/18] bpf: Follow integer casts " Anton Protopopov
` (6 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
An error often reaches the return through a PHI in the block of the
return which joins it with the results of other paths, some of them
dirty. Take the incoming values of such a PHI on the edges from the
blocks which only clean paths reach. Do not follow a PHI in an earlier
block: a later branch may split its values between clean and dirty
paths.
Take also the errors which LazyValueInfo knows exactly: an incoming
value on its edge and a returned value at the return.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
scripts/llvm-plugins/bpf_scan_ir.cpp | 84 ++++++++++++++++++++++++++--
1 file changed, 79 insertions(+), 5 deletions(-)
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
index 0fc60b2130b1..81de8f941568 100644
--- a/scripts/llvm-plugins/bpf_scan_ir.cpp
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -8,11 +8,13 @@
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/Analysis/CaptureTracking.h"
+#include "llvm/Analysis/LazyValueInfo.h"
#include "llvm/Analysis/ValueTracking.h"
#include "llvm/Config/llvm-config.h"
#include "llvm/IR/Analysis.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/CFG.h"
+#include "llvm/IR/ConstantRange.h"
#include "llvm/IR/Constants.h"
#include "llvm/IR/Dominators.h"
#include "llvm/IR/GlobalVariable.h"
@@ -34,9 +36,40 @@ namespace {
static constexpr int MaxErrno = 4095;
+// is Range in [-4095, 0) ?
+static bool isErrnoRange(const ConstantRange &Range)
+{
+ unsigned Width = Range.getBitWidth();
+
+ if (Width != 32 && Width != 64)
+ return false;
+
+ auto L = APInt(Width, 0) - APInt(Width, MaxErrno);
+ auto R = APInt(Width, 0);
+ return !Range.isEmptySet() && ConstantRange(L, R).contains(Range);
+}
+
/* The errors which clean paths of a function return */
using ReturnValues = SmallVector<int64_t, 8>;
+/* Return true and set N if RANGE holds a single errno */
+static bool singleErrno(const ConstantRange &Range, int64_t &N)
+{
+ const APInt *C = Range.getSingleElement();
+
+ if (!C || !isErrnoRange(Range))
+ return false;
+ N = C->getSExtValue();
+ return true;
+}
+
+/* Return true and set N if the value at use U is a single errno */
+static bool singleErrnoAtUse(const Use &U, LazyValueInfo &LVI, int64_t &N)
+{
+ return U.get()->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeAtUse(U, false), N);
+}
+
/* Return true and set N if V is an errno constant */
static bool errnoConstant(const Value *V, int64_t &N)
{
@@ -49,6 +82,42 @@ static bool errnoConstant(const Value *V, int64_t &N)
return N >= -MaxErrno && N < 0;
}
+/*
+ * Add the errors which V, the value returned in RET, is known to be on the
+ * edges to RET from CLEANBBS to VALUES. V may be a PHI in RET. A PHI in an
+ * earlier block is not followed: a later branch may split its values between
+ * clean and dirty paths.
+ */
+static void collectErrnos(Value *V, BasicBlock *Ret, LazyValueInfo &LVI,
+ const SmallPtrSetImpl<BasicBlock *> &CleanBBs, ReturnValues &Values)
+{
+ int64_t N;
+
+ auto *Phi = dyn_cast<PHINode>(V);
+
+ if (!Phi || Phi->getParent() != Ret)
+ return;
+ for (unsigned i = 0; i < Phi->getNumIncomingValues(); i++) {
+ Value *Incoming = Phi->getIncomingValue(i);
+ BasicBlock *Pred = Phi->getIncomingBlock(i);
+
+ if (!CleanBBs.contains(Pred))
+ continue;
+ if (errnoConstant(Incoming, N)) {
+ Values.push_back(N);
+ continue;
+ }
+ /*
+ * LazyValueInfo knows the ranges of integers only. On an edge it
+ * also uses the branch condition, as in "if (err == -EINVAL) goto
+ * out", where err is the error.
+ */
+ if (Incoming->getType()->isIntegerTy() &&
+ singleErrno(LVI.getConstantRangeOnEdge(Incoming, Pred, Ret), N))
+ Values.push_back(N);
+ }
+}
+
class CleanInstructions {
DominatorTree &DT;
DenseMap<const Instruction *, bool> PrivateWrites;
@@ -111,7 +180,8 @@ static bool cleanBB(BasicBlock *BB, CleanInstructions &Clean)
}
/* Add the errors which clean paths of F are known to return to VALUES */
-static void cleanReturnValues(Function &F, CleanInstructions &Clean, ReturnValues &Values)
+static void cleanReturnValues(Function &F, LazyValueInfo &LVI, CleanInstructions &Clean,
+ ReturnValues &Values)
{
SmallPtrSet<BasicBlock *, 32> CleanBBs;
SmallVector<BasicBlock *, 32> Worklist;
@@ -154,8 +224,7 @@ static void cleanReturnValues(Function &F, CleanInstructions &Clean, ReturnValue
/*
* The returns of the clean blocks which only clean paths reach or which
- * one of them branches to. Dirty paths may also reach the latter, but
- * the returned constant is the same on all paths.
+ * one of them branches to
*/
for (BasicBlock &BB : F) {
auto *Return = dyn_cast<ReturnInst>(BB.getTerminator());
@@ -168,8 +237,11 @@ static void cleanReturnValues(Function &F, CleanInstructions &Clean, ReturnValue
Entered |= CleanBBs.contains(Pred);
if (!Entered)
continue;
- if (errnoConstant(Return->getReturnValue(), N))
+ if (singleErrnoAtUse(Return->getOperandUse(0), LVI, N)) {
Values.push_back(N);
+ continue;
+ }
+ collectErrnos(Return->getReturnValue(), &BB, LVI, CleanBBs, Values);
}
llvm::sort(Values);
Values.erase(std::unique(Values.begin(), Values.end()), Values.end());
@@ -237,10 +309,11 @@ public:
Flags |= FMODRET_FN_WEAK;
if (Ret->isIntegerTy(64))
Flags |= FMODRET_FN_RET64;
+ auto &LVI = FAM.getResult<LazyValueAnalysis>(F);
auto &DT = FAM.getResult<DominatorTreeAnalysis>(F);
CleanInstructions Clean(DT);
- cleanReturnValues(F, Clean, Values);
+ cleanReturnValues(F, LVI, Clean, Values);
/* A weak definition always has a record, see fmodret_metadata.h */
if (!Values.empty() || (Flags & FMODRET_FN_WEAK))
Descriptions.push_back({&F, Flags, std::move(Values)});
@@ -253,6 +326,7 @@ public:
PreservedAnalyses PA;
PA.preserve<FunctionAnalysisManagerModuleProxy>();
PA.preserveSet<CFGAnalyses>();
+ PA.preserve<LazyValueAnalysis>();
return PA;
}
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 12/18] bpf: Follow integer casts in bpf_scan_ir
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (9 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 11/18] bpf: Follow PHIs " Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 13/18] bpf: Check the return values of fmod_ret programs on found targets Anton Protopopov
` (5 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Follow sign extensions and truncations between integers of at least 32
bits in the block of the return. They keep an error, for example an int
error which a function returns as long.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
scripts/llvm-plugins/bpf_scan_ir.cpp | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/scripts/llvm-plugins/bpf_scan_ir.cpp b/scripts/llvm-plugins/bpf_scan_ir.cpp
index 81de8f941568..a3c6336662fc 100644
--- a/scripts/llvm-plugins/bpf_scan_ir.cpp
+++ b/scripts/llvm-plugins/bpf_scan_ir.cpp
@@ -82,17 +82,39 @@ static bool errnoConstant(const Value *V, int64_t &N)
return N >= -MaxErrno && N < 0;
}
+/* A sign extension or a truncation between integers of at least 32 bits */
+static bool preservesErrnoCast(const CastInst *Cast)
+{
+ Type *Src = Cast->getSrcTy(), *Dst = Cast->getDestTy();
+
+ if (Src->isIntegerTy() && Dst->isIntegerTy())
+ return (Cast->getOpcode() == Instruction::SExt ||
+ Cast->getOpcode() == Instruction::Trunc) &&
+ Src->getIntegerBitWidth() >= 32 && Dst->getIntegerBitWidth() >= 32;
+ return false;
+}
+
/*
* Add the errors which V, the value returned in RET, is known to be on the
- * edges to RET from CLEANBBS to VALUES. V may be a PHI in RET. A PHI in an
- * earlier block is not followed: a later branch may split its values between
- * clean and dirty paths.
+ * edges to RET from CLEANBBS to VALUES. Follow the casts in RET which keep the
+ * error to a PHI in RET. A PHI in an earlier block is not followed: a later
+ * branch may split its values between clean and dirty paths.
*/
static void collectErrnos(Value *V, BasicBlock *Ret, LazyValueInfo &LVI,
const SmallPtrSetImpl<BasicBlock *> &CleanBBs, ReturnValues &Values)
{
int64_t N;
+ while (auto *Cast = dyn_cast<CastInst>(V)) {
+ if (Cast->getParent() != Ret || !preservesErrnoCast(Cast))
+ return;
+ if (singleErrnoAtUse(Cast->getOperandUse(0), LVI, N)) {
+ Values.push_back(N);
+ return;
+ }
+ V = Cast->getOperand(0);
+ }
+
auto *Phi = dyn_cast<PHINode>(V);
if (!Phi || Phi->getParent() != Ret)
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 13/18] bpf: Check the return values of fmod_ret programs on found targets
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (10 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 12/18] bpf: Follow integer casts " Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 14/18] bpf: Require the JIT for fmod_ret programs on found 64-bit targets Anton Protopopov
` (4 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
A later patch lets resolve_btfids make the functions found by bpf_scan_ir
fmod_ret targets. Each of them gets a "bpf_fmodret_retval:" decl tag with
the values it returns on paths without side effects, for example
"bpf_fmodret_retval:-22,-12". Returning such a value instead of calling
the function looks to the callers like a real failure.
Read the tag when an fmod_ret program is loaded and check at every exit
that the program returns 0 or one of these values.
A function which the static rules allow keeps today's behaviour. So do
the functions declared in BTF_FMODRET sets, which have no such tag.
Only a kernel function has such a tag, so do not look for it when the
target is a BPF program. Remember the values in the program, and do not
let an extension replace an fmod_ret program with restricted return
values: it could return any value. Check this at load and when an
extension is attached to another program later. Do not allow tail calls
in such a program either: the program it jumps to returns for it, and a
program array does not check the values it may return.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
include/linux/bpf.h | 6 ++
kernel/bpf/syscall.c | 1 +
kernel/bpf/verifier.c | 146 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 153 insertions(+)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index b011acc22c50..69387de119a6 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1967,6 +1967,12 @@ struct bpf_prog_aux {
const struct btf_type *attach_func_proto;
/* function name for valid attach_btf_id */
const char *attach_func_name;
+ /*
+ * The values which an fmod_ret program may return besides 0, from the
+ * "bpf_fmodret_retval:" tag of its target, or NULL
+ */
+ s64 *fmodret_retvals;
+ u32 fmodret_retval_cnt;
struct bpf_prog **func;
struct bpf_prog_aux *main_prog_aux;
/* main prog: there when the prog has timed may_goto */
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 9004f2673ccf..0a78f66a1622 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2490,6 +2490,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
module_put(prog->aux->mod);
kfree(prog->aux->kfunc_tab);
kfree(prog->aux->ctx_arg_info);
+ kfree(prog->aux->fmodret_retvals);
if (prog->aux->attach_btf)
btf_put(prog->aux->attach_btf);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b7329d164771..a9da29a11963 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -13329,6 +13329,14 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
}
if (func_id == BPF_FUNC_tail_call) {
+ /*
+ * The program which a tail call jumps to returns for this one,
+ * and a program array does not check the values it may return.
+ */
+ if (env->prog->aux->fmodret_retvals) {
+ verbose(env, "tail calls are not allowed in fmod_ret programs with restricted return values\n");
+ return -EINVAL;
+ }
if (env->cur_state->curframe) {
struct bpf_verifier_state *branch;
@@ -19887,6 +19895,73 @@ static bool program_returns_void(struct bpf_verifier_env *env)
return false;
}
+/*
+ * A function which the build found by analysing its code has a
+ * "bpf_fmodret_retval:" decl tag with the values it returns on paths without
+ * side effects, for example "bpf_fmodret_retval:-22,-12". See the comment at
+ * the top of tools/bpf/resolve_btfids/main.c. An fmod_ret program attached to
+ * it may return only these values, or 0 to call the function.
+ */
+#define BTF_FMODRET_RETVAL_TAG "bpf_fmodret_retval:"
+
+/* Parse the next value of the list at *LIST and move *LIST past it */
+static int fmodret_retval_next(const char **list, s64 *value)
+{
+ const char *end = strchrnul(*list, ',');
+ size_t len = end - *list;
+ char buf[24];
+
+ if (!len || len >= sizeof(buf) || (*end && !end[1]))
+ return -EINVAL;
+ memcpy(buf, *list, len);
+ buf[len] = 0;
+ *list = *end ? end + 1 : end;
+ return kstrtos64(buf, 10, value);
+}
+
+static bool fmodret_retval_allowed(const struct bpf_prog_aux *aux, u64 value)
+{
+ const struct btf_type *t;
+ u32 i;
+
+ /* "w0 = -22" zero-extends, the caller of an int function sees -22 */
+ t = btf_type_skip_modifiers(aux->attach_btf, aux->attach_func_proto->type, NULL);
+ if (t->size == 4 && !upper_32_bits(value))
+ value = (s32)value;
+ if (!value)
+ return true;
+ for (i = 0; i < aux->fmodret_retval_cnt; i++) {
+ if (aux->fmodret_retvals[i] == (s64)value)
+ return true;
+ }
+ return false;
+}
+
+/* The program must return a constant: 0 or one of the values of the tag */
+static int check_fmodret_retval(struct bpf_verifier_env *env, int regno,
+ const char *exit_ctx, const char *reg_name)
+{
+ const struct bpf_prog_aux *aux = env->prog->aux;
+ struct bpf_reg_state *reg = reg_state(env, regno);
+ int err;
+ u32 i;
+
+ if (reg->type == SCALAR_VALUE) {
+ err = mark_chain_precision(env, regno);
+ if (err)
+ return err;
+ if (tnum_is_const(reg->var_off) &&
+ fmodret_retval_allowed(aux, reg->var_off.value))
+ return 0;
+ }
+
+ verbose(env, "%s the register %s must be a constant, it should be 0 or one of", exit_ctx, reg_name);
+ for (i = 0; i < aux->fmodret_retval_cnt; i++)
+ verbose(env, "%s%lld", i ? "," : " ", aux->fmodret_retvals[i]);
+ verbose(env, "\n");
+ return -EINVAL;
+}
+
static int check_return_code(struct bpf_verifier_env *env, int regno, const char *reg_name)
{
const char *exit_ctx = "At program exit";
@@ -19943,6 +20018,9 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
if (prog_type == BPF_PROG_TYPE_CGROUP_SKB && (env->prog->expected_attach_type == BPF_CGROUP_INET_EGRESS))
enforce_attach_type_range = tnum_range(2, 3);
+ if (env->prog->aux->fmodret_retvals)
+ return check_fmodret_retval(env, regno, exit_ctx, reg_name);
+
if (!return_retval_range(env, &range))
return 0;
@@ -22874,6 +22952,58 @@ static int check_attach_modify_return(unsigned long addr, const char *func_name)
#endif /* CONFIG_FUNCTION_ERROR_INJECTION */
+/*
+ * Programs attached to a function found by the analysis may return only the
+ * values of its "bpf_fmodret_retval:" tag. A function which the static rules
+ * allow stays unrestricted.
+ */
+static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,
+ unsigned long addr, const char *tname)
+{
+ struct bpf_prog_aux *aux = env->prog->aux;
+ const struct btf *btf = aux->attach_btf;
+ const struct btf_type *t = btf_type_by_id(btf, btf_id);
+ s64 *retvals = NULL;
+ const char *values, *list;
+ u32 cnt, size = 0;
+ s64 v;
+
+ if (!check_attach_modify_return(addr, tname))
+ return 0;
+ values = btf_find_decl_tag_value(btf, t, -1, BTF_FMODRET_RETVAL_TAG);
+ if (IS_ERR(values))
+ return PTR_ERR(values) == -ENOENT ? 0 : PTR_ERR(values);
+
+ t = btf_type_by_id(btf, t->type);
+ t = t ? btf_type_skip_modifiers(btf, t->type, NULL) : NULL;
+ if (t && (btf_type_is_int(t) || btf_is_any_enum(t)))
+ size = t->size;
+ if (size != 4 && size != 8)
+ goto invalid;
+
+ for (list = values, cnt = 1; *list; list++)
+ cnt += *list == ',';
+ retvals = kmalloc_array(cnt, sizeof(*retvals), GFP_KERNEL_ACCOUNT);
+ if (!retvals)
+ return -ENOMEM;
+ for (list = values, cnt = 0; *list;) {
+ if (fmodret_retval_next(&list, &v) || v >= 0 || (size == 4 && v < S32_MIN))
+ goto invalid;
+ retvals[cnt++] = v;
+ }
+ if (!cnt)
+ goto invalid;
+
+ aux->fmodret_retvals = retvals;
+ aux->fmodret_retval_cnt = cnt;
+ return 0;
+
+invalid:
+ kfree(retvals);
+ verbose(env, "invalid %s%s tag of %s()\n", BTF_FMODRET_RETVAL_TAG, values, tname);
+ return -EINVAL;
+}
+
static bool is_tracing_multi_id(const struct bpf_prog *prog, u32 btf_id)
{
return is_tracing_multi(prog->expected_attach_type) && bpf_multi_func_btf_id[0] == btf_id;
@@ -23283,6 +23413,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
bpf_log(log, "Cannot extend fentry/fexit/fsession\n");
return -EINVAL;
}
+ /*
+ * An extension could return any value for an fmod_ret program
+ * whose return values are restricted. This is also checked when
+ * an extension is attached to another program than at load.
+ */
+ if (prog_extension && tgt_prog->aux->fmodret_retvals) {
+ bpf_log(log, "Cannot extend an fmod_ret program with restricted return values\n");
+ return -EINVAL;
+ }
} else {
if (prog_extension) {
bpf_log(log, "Cannot replace kernel functions\n");
@@ -23662,6 +23801,13 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)
return -EINVAL;
}
+ /* Only a kernel function has a "bpf_fmodret_retval:" tag */
+ if (prog->expected_attach_type == BPF_MODIFY_RETURN && !tgt_prog) {
+ ret = fmodret_retval_init(env, btf_id, tgt_info.tgt_addr, tgt_info.tgt_name);
+ if (ret)
+ return ret;
+ }
+
/*
* We don't get trampoline for tracing_multi programs at this point,
* it's done when tracing_multi link is created.
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 14/18] bpf: Require the JIT for fmod_ret programs on found 64-bit targets
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (11 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 13/18] bpf: Check the return values of fmod_ret programs on found targets Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 15/18] resolve_btfids: Tag fmod_ret candidates as targets Anton Protopopov
` (3 subsequent siblings)
16 siblings, 0 replies; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
The interpreter returns only the low 32 bits of the return value of a
program, while the trampoline passes all 64 bits to the caller of the
function. If an fmod_ret program on a found target which returns long
is interpreted, then a valid return value like -14 reaches the caller
as 4294967282. This value is not 0, so the function is skipped, and it
is not an error either.
Require the JIT for such programs, as for other features which the
interpreter does not support. The caller of a function which returns int
reads only the low 32 bits, so nothing changes for it.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
kernel/bpf/verifier.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a9da29a11963..0448a33085fb 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -22996,6 +22996,9 @@ static int fmodret_retval_init(struct bpf_verifier_env *env, u32 btf_id,
aux->fmodret_retvals = retvals;
aux->fmodret_retval_cnt = cnt;
+ /* The interpreter returns only 32 bits */
+ if (size == 8)
+ env->prog->jit_required = 1;
return 0;
invalid:
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v1 bpf-next 15/18] resolve_btfids: Tag fmod_ret candidates as targets
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (12 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 14/18] bpf: Require the JIT for fmod_ret programs on found 64-bit targets Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:50 ` bot+bpf-ci
2026-11-06 21:03 ` [PATCH v1 bpf-next 16/18] selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir Anton Protopopov
` (2 subsequent siblings)
16 siblings, 1 reply; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Read the records which bpf_scan_ir writes to .BTF_fmodret_candidates and
choose the functions which can become fmod_ret targets. Map each record
to its function through its relocation in a module or through its
address in vmlinux. A record of a strong definition wins over the
records of the weak definitions it overrides. A strong definition
without values has no record, so use the record of a weak definition
only if the symbol of the function is still weak after the link. Every
weak definition has a record, so a function with two weak definitions is
ambiguous.
Choose a function if the record lists at least one error, the function
has an ftrace call site, is not init, exit or noinstr code, its name
belongs to exactly one ELF function and one BTF FUNC and it returns an
integer of the size the record describes.
Tag each chosen function with "bpf_fmodret" and with
"bpf_fmodret_retval:" followed by the errors it returns on paths without
side effects, for example "bpf_fmodret_retval:-22,-12". The values are
signed decimal numbers, sorted and separated by commas. The verifier
lets programs attached to such a function return only these values or 0.
A function in a BTF_FMODRET set keeps only its own tags.
Mark the range of patchable function entries in the linker script so the
ftrace call sites in vmlinux can be told from the mcount ones.
Take the ftrace call sites of a module from the section the kernel uses:
x86 also has call padding entries in __patchable_function_entries.
With -v, report what was found.
Add the new files to the BPF [COMPILER PLUGINS] entry in MAINTAINERS.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
MAINTAINERS | 1 +
include/asm-generic/vmlinux.lds.h | 2 +
scripts/Makefile.btf | 1 +
tools/bpf/resolve_btfids/Build | 1 +
tools/bpf/resolve_btfids/fmodret.c | 1215 ++++++++++++++++++++++++++++
tools/bpf/resolve_btfids/fmodret.h | 28 +
tools/bpf/resolve_btfids/main.c | 64 ++
7 files changed, 1312 insertions(+)
create mode 100644 tools/bpf/resolve_btfids/fmodret.c
create mode 100644 tools/bpf/resolve_btfids/fmodret.h
diff --git a/MAINTAINERS b/MAINTAINERS
index dfbb45ecc716..29940e8d7205 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5070,6 +5070,7 @@ L: bpf@vger.kernel.org
S: Maintained
F: scripts/bpf/
F: scripts/llvm-plugins/bpf_*
+F: tools/bpf/resolve_btfids/fmodret.*
BPF [CORE]
M: Alexei Starovoitov <ast@kernel.org>
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 4e18758b5ffc..77e86ff2f9a4 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -198,7 +198,9 @@
#define MCOUNT_REC() . = ALIGN(8); \
__start_mcount_loc = .; \
KEEP(*(__mcount_loc)) \
+ __start_patchable_function_entries = .; \
KEEP_PATCHABLE \
+ __stop_patchable_function_entries = .; \
__stop_mcount_loc = .; \
FTRACE_STUB_HACK \
ftrace_ops_list_func = arch_ftrace_ops_list_func;
diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index 98012344a81e..3aa3d084cc6c 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -35,5 +35,6 @@ resolve-btfids-flags-y :=
resolve-btfids-flags-$(CONFIG_WERROR) += --fatal_warnings
resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) += --distill_base
resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) += --verbose
+resolve-btfids-flags-$(CONFIG_FTRACE_MCOUNT_USE_PATCHABLE_FUNCTION_ENTRY) += --patchable_ftrace
export RESOLVE_BTFIDS_FLAGS := $(resolve-btfids-flags-y)
diff --git a/tools/bpf/resolve_btfids/Build b/tools/bpf/resolve_btfids/Build
index 602e1508bf09..af777afa451b 100644
--- a/tools/bpf/resolve_btfids/Build
+++ b/tools/bpf/resolve_btfids/Build
@@ -1,6 +1,7 @@
hostprogs := resolve_btfids
resolve_btfids-y += main.o
+resolve_btfids-y += fmodret.o
resolve_btfids-y += btf_colors.o
resolve_btfids-y += rbtree.o
resolve_btfids-y += zalloc.o
diff --git a/tools/bpf/resolve_btfids/fmodret.c b/tools/bpf/resolve_btfids/fmodret.c
new file mode 100644
index 000000000000..1c29ab9a8616
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.c
@@ -0,0 +1,1215 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Read the records which the bpf_scan_ir compiler plugin writes to
+ * .BTF_fmodret_candidates, see scripts/llvm-plugins/bpf_scan_ir.cpp, and find
+ * the functions which can become fmod_ret targets. A record lists the errors
+ * which clean paths of a function return, that is paths without side effects.
+ *
+ * A function is chosen if
+ * - exactly one record describes its body, or one record of a strong
+ * definition next to records of the weak ones it overrides,
+ * - the record is of a strong definition or the symbol is still weak,
+ * - the record lists at least one error,
+ * - it has an ftrace call site and is not init, exit or noinstr code,
+ * - its name belongs to exactly one ELF function and one BTF FUNC,
+ * - it returns an integer of the size the record describes.
+ *
+ * The plugin records no errors for functions which return pointers yet.
+ */
+
+#include <errno.h>
+#include <gelf.h>
+#include <libelf.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <linux/kernel.h>
+
+#include "fmodret.h"
+#include "../../../scripts/bpf/fmodret_metadata.h"
+
+#define FMODRET_CANDIDATES_SECTION ".BTF_fmodret_candidates"
+#define NO_FUNCTION SIZE_MAX
+
+struct elf_function {
+ const char *name;
+ GElf_Addr value;
+ GElf_Xword size;
+ Elf64_Section shndx;
+ bool traceable;
+ /* The symbol is weak: no strong definition replaced it at the link */
+ bool weak_symbol;
+ /* The records which describe the body at this address */
+ unsigned int nr_strong, nr_weak;
+ size_t strong, weak;
+};
+
+struct elf_function_list {
+ struct elf_function *functions;
+ size_t nr;
+ size_t capacity;
+};
+
+struct fmodret_stats {
+ size_t records, unmapped, with_values;
+ size_t ambiguous_body, not_traceable, ambiguous_name;
+ size_t overridden, no_btf, return_mismatch, added;
+};
+
+static int find_section(Elf *elf, size_t shstrndx, const char *wanted,
+ Elf_Scn **result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (!strcmp(name, wanted)) {
+ *result = scn;
+ return 0;
+ }
+ }
+
+ *result = NULL;
+ return 0;
+}
+
+static int find_symbol(Elf *elf, const char *wanted, GElf_Sym *result,
+ bool *found)
+{
+ Elf_Scn *scn = NULL;
+
+ *found = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ if (strcmp(name, wanted))
+ continue;
+ *result = sym;
+ *found = true;
+ return 0;
+ }
+ }
+ return 0;
+}
+
+static int section_for_address(Elf *elf, GElf_Addr address,
+ Elf64_Section *result)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (!(shdr.sh_flags & SHF_EXECINSTR) || address < shdr.sh_addr ||
+ address - shdr.sh_addr >= shdr.sh_size)
+ continue;
+ *result = elf_ndxscn(scn);
+ return 0;
+ }
+ *result = SHN_UNDEF;
+ return 0;
+}
+
+static bool relocation_offset(const GElf_Shdr *target, GElf_Addr offset,
+ size_t width, size_t *result)
+{
+ if (target->sh_size < width)
+ return false;
+ if (offset >= target->sh_addr &&
+ offset - target->sh_addr <= target->sh_size - width) {
+ *result = offset - target->sh_addr;
+ return true;
+ }
+ /* ET_REL relocation offsets are relative to their target section. */
+ if (offset <= target->sh_size - width) {
+ *result = offset;
+ return true;
+ }
+ return false;
+}
+
+static uint64_t read_elf_value(const unsigned char *p, size_t size,
+ unsigned int encoding)
+{
+ uint64_t value = 0;
+ size_t i;
+
+ if (encoding == ELFDATA2LSB) {
+ for (i = 0; i < size; i++)
+ value |= (uint64_t)p[i] << (8 * i);
+ } else {
+ for (i = 0; i < size; i++)
+ value = (value << 8) | p[i];
+ }
+ return value;
+}
+
+static int copy_section_data(Elf_Scn *scn, const GElf_Shdr *shdr,
+ unsigned char **result)
+{
+ Elf_Data *data = NULL;
+ unsigned char *buf;
+
+ buf = calloc(1, shdr->sh_size ?: 1);
+ if (!buf)
+ return -ENOMEM;
+ while ((data = elf_getdata(scn, data))) {
+ if (data->d_off > shdr->sh_size ||
+ data->d_size > shdr->sh_size - data->d_off) {
+ free(buf);
+ return -EINVAL;
+ }
+ memcpy(buf + data->d_off, data->d_buf, data->d_size);
+ }
+ *result = buf;
+ return 0;
+}
+
+static int add_elf_function(struct elf_function_list *list, const char *name,
+ const GElf_Sym *sym)
+{
+ struct elf_function *functions;
+ size_t capacity;
+
+ if (list->nr == list->capacity) {
+ capacity = list->capacity ? 2 * list->capacity : 1024;
+ if (capacity > SIZE_MAX / sizeof(*functions))
+ return -E2BIG;
+ functions = realloc(list->functions, capacity * sizeof(*functions));
+ if (!functions)
+ return -ENOMEM;
+ list->functions = functions;
+ list->capacity = capacity;
+ }
+
+ list->functions[list->nr++] = (struct elf_function) {
+ .name = name,
+ .value = sym->st_value,
+ .size = sym->st_size,
+ .shndx = sym->st_shndx,
+ .weak_symbol = GELF_ST_BIND(sym->st_info) == STB_WEAK,
+ .strong = NO_FUNCTION,
+ .weak = NO_FUNCTION,
+ };
+ return 0;
+}
+
+static int collect_elf_functions(Elf *elf, struct elf_function_list *list)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_symbols;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_symbols = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_symbols; i++) {
+ const char *name;
+ GElf_Sym sym;
+ int err;
+
+ if (!gelf_getsym(data, i, &sym))
+ return -1;
+ if (GELF_ST_TYPE(sym.st_info) != STT_FUNC ||
+ sym.st_shndx == SHN_UNDEF || !sym.st_name)
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (!name)
+ return -1;
+ err = add_elf_function(list, name, &sym);
+ if (err)
+ return err;
+ }
+ }
+ return 0;
+}
+
+static int cmp_elf_function_location(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ if (a->shndx != b->shndx)
+ return a->shndx > b->shndx ? 1 : -1;
+ if (a->value != b->value)
+ return a->value > b->value ? 1 : -1;
+ return 0;
+}
+
+static int cmp_elf_function_name(const void *va, const void *vb)
+{
+ const struct elf_function *a = va;
+ const struct elf_function *b = vb;
+
+ return strcmp(a->name, b->name) ?: cmp_elf_function_location(va, vb);
+}
+
+/* Return the first function at VALUE in SHNDX. Functions are sorted by location. */
+static size_t function_at(const struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr value)
+{
+ size_t lo = 0, hi = list->nr;
+
+ while (lo < hi) {
+ size_t mid = lo + (hi - lo) / 2;
+ const struct elf_function *f = &list->functions[mid];
+
+ if (f->shndx < shndx || (f->shndx == shndx && f->value < value))
+ lo = mid + 1;
+ else
+ hi = mid;
+ }
+ if (lo == list->nr || list->functions[lo].shndx != shndx ||
+ list->functions[lo].value != value)
+ return NO_FUNCTION;
+ return lo;
+}
+
+static bool function_contains(const struct elf_function *function,
+ GElf_Addr address)
+{
+ if (address < function->value)
+ return false;
+ if (!function->size)
+ return address == function->value;
+ return address - function->value < function->size;
+}
+
+enum ftrace_site_kind {
+ FTRACE_SITE_MCOUNT,
+ FTRACE_SITE_PATCHABLE,
+ FTRACE_SITE_UNKNOWN,
+};
+
+/*
+ * The NOPs which arm64 puts before a function with CALL_OPS. From the entry of
+ * an overridden weak function, whose body has no symbol left, a longer distance
+ * could reach the next function. Other arches may need more.
+ */
+#define MAX_PATCHABLE_FUNCTION_ENTRY_SIZE 8
+
+/* Return true if function I or another function at its address contains ADDRESS */
+static bool alias_contains(const struct elf_function_list *list, size_t i,
+ GElf_Addr address)
+{
+ const struct elf_function *f = &list->functions[i];
+
+ for (;;) {
+ if (function_contains(&list->functions[i], address))
+ return true;
+ if (!i || list->functions[i - 1].shndx != f->shndx ||
+ list->functions[i - 1].value != f->value)
+ return false;
+ i--;
+ }
+}
+
+/*
+ * Mark the function containing an ftrace call site. Functions are sorted by
+ * section and address. Mark all aliases at the same address: at run time BTF
+ * attachment is name based and may resolve to any of them.
+ */
+static bool mark_traceable_function(struct elf_function_list *list,
+ Elf64_Section shndx, GElf_Addr address,
+ enum ftrace_site_kind kind)
+{
+ size_t left = 0;
+ size_t right = list->nr;
+ size_t first;
+ size_t i;
+ bool found = false;
+
+ /* Find the first function after ADDRESS in SHNDX. */
+ while (left < right) {
+ size_t middle = left + (right - left) / 2;
+ struct elf_function *function = &list->functions[middle];
+
+ if (function->shndx < shndx ||
+ (function->shndx == shndx && function->value <= address))
+ left = middle + 1;
+ else
+ right = middle;
+ }
+ /*
+ * A patchable entry is at the start of its function, a few bytes into it
+ * (after an arm64 BTI) or in the NOPs before it
+ */
+ first = SIZE_MAX;
+ if (kind != FTRACE_SITE_MCOUNT) {
+ if (left && list->functions[left - 1].shndx == shndx &&
+ address - list->functions[left - 1].value <
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE &&
+ alias_contains(list, left - 1, address))
+ first = left - 1;
+ else if (left < list->nr && list->functions[left].shndx == shndx &&
+ list->functions[left].value - address <=
+ MAX_PATCHABLE_FUNCTION_ENTRY_SIZE)
+ first = left;
+ }
+ if (first != SIZE_MAX) {
+ while (first && list->functions[first - 1].shndx == shndx &&
+ list->functions[first - 1].value ==
+ list->functions[first].value)
+ first--;
+ for (i = first; i < list->nr &&
+ list->functions[i].shndx == shndx &&
+ list->functions[i].value ==
+ list->functions[first].value; i++) {
+ list->functions[i].traceable = true;
+ found = true;
+ }
+ return found;
+ }
+ if (kind == FTRACE_SITE_PATCHABLE || !left)
+ return false;
+
+ /* The nearest preceding function owns the call site. */
+ first = left - 1;
+ if (list->functions[first].shndx != shndx)
+ return false;
+ while (first && list->functions[first - 1].shndx == shndx &&
+ list->functions[first - 1].value == list->functions[first].value)
+ first--;
+ for (i = first; i < list->nr &&
+ list->functions[i].shndx == shndx &&
+ list->functions[i].value == list->functions[first].value; i++) {
+ if (!function_contains(&list->functions[i], address))
+ continue;
+ list->functions[i].traceable = true;
+ found = true;
+ }
+ return found;
+}
+
+static int mark_relocated_ftrace_sites(Elf *elf, size_t target_ndx,
+ const unsigned char *target_data,
+ const GElf_Shdr *target_shdr,
+ size_t begin, size_t end,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind,
+ bool *relocated)
+{
+ Elf_Scn *scn = NULL;
+
+ *relocated = false;
+ while ((scn = elf_nextscn(elf, scn))) {
+ GElf_Shdr shdr;
+ Elf_Data *data;
+ size_t nr_relocations;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ (shdr.sh_info && shdr.sh_info != target_ndx))
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ if (!data || elf_getdata(scn, data))
+ return -1;
+
+ nr_relocations = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nr_relocations; i++) {
+ Elf_Scn *symtab_scn;
+ Elf_Data *symtab_data;
+ GElf_Shdr symtab_shdr;
+ GElf_Sxword addend;
+ GElf_Addr raw_offset;
+ GElf_Addr address;
+ GElf_Sym sym;
+ size_t sym_ndx;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ GElf_Rela rela;
+
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ sym_ndx = GELF_R_SYM(rela.r_info);
+ raw_offset = rela.r_offset;
+ addend = rela.r_addend;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ sym_ndx = GELF_R_SYM(rel.r_info);
+ raw_offset = rel.r_offset;
+ if (!relocation_offset(target_shdr, raw_offset,
+ ptr_size, &offset))
+ continue;
+ addend = read_elf_value(target_data + offset,
+ ptr_size, encoding);
+ }
+ if (!relocation_offset(target_shdr, raw_offset, ptr_size,
+ &offset))
+ continue;
+ if (offset < begin || offset >= end)
+ continue;
+ *relocated = true;
+
+ /*
+ * PIE vmlinux uses symbol-less RELATIVE relocations. Its
+ * addend is the link-time call-site address, represented as
+ * a signed ELF addend.
+ */
+ if (!sym_ndx) {
+ Elf64_Section target_shndx;
+
+ address = (GElf_Addr)addend;
+ if (section_for_address(elf, address,
+ &target_shndx))
+ return -1;
+ if (target_shndx != SHN_UNDEF)
+ mark_traceable_function(functions,
+ target_shndx,
+ address, kind);
+ continue;
+ }
+
+ symtab_scn = elf_getscn(elf, shdr.sh_link);
+ if (!symtab_scn ||
+ !gelf_getshdr(symtab_scn, &symtab_shdr))
+ return -1;
+ symtab_data = elf_getdata(symtab_scn, NULL);
+ if (!symtab_data || elf_getdata(symtab_scn, symtab_data) ||
+ !gelf_getsym(symtab_data, sym_ndx, &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF)
+ continue;
+ if (sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ if (addend < 0) {
+ GElf_Addr delta = -(addend + 1);
+
+ delta++;
+ if (delta > sym.st_value)
+ continue;
+ address = sym.st_value - delta;
+ } else {
+ if ((GElf_Addr)addend > UINT64_MAX - sym.st_value)
+ continue;
+ address = sym.st_value + addend;
+ }
+ mark_traceable_function(functions, sym.st_shndx, address,
+ kind);
+ }
+ }
+ return 0;
+}
+
+static int read_ftrace_site_range(Elf *elf, Elf_Scn *scn, size_t begin,
+ size_t end, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ unsigned char *buf = NULL;
+ GElf_Shdr shdr;
+ bool relocated;
+ size_t offset;
+ int err;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if (begin > end || end > shdr.sh_size ||
+ (end - begin) % ptr_size)
+ return -EINVAL;
+ if (begin == end)
+ return 0;
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ return err;
+ err = mark_relocated_ftrace_sites(elf, elf_ndxscn(scn), buf, &shdr,
+ begin, end, ptr_size, encoding,
+ functions, kind, &relocated);
+ if (err || relocated)
+ goto out;
+
+ for (offset = begin; offset < end; offset += ptr_size) {
+ GElf_Addr address;
+ Elf64_Section shndx;
+
+ address = read_elf_value(buf + offset, ptr_size, encoding);
+ err = section_for_address(elf, address, &shndx);
+ if (err)
+ goto out;
+ if (shndx != SHN_UNDEF)
+ mark_traceable_function(functions, shndx, address, kind);
+ }
+out:
+ free(buf);
+ return err;
+}
+
+static bool section_prefix(const char *name, const char *prefix)
+{
+ size_t len = strlen(prefix);
+
+ return !strncmp(name, prefix, len) &&
+ (!name[len] || name[len] == '.');
+}
+
+static bool section_cannot_be_attached(const char *name)
+{
+ return section_prefix(name, ".init.text") ||
+ section_prefix(name, ".exit.text") ||
+ section_prefix(name, ".noinstr.text") ||
+ section_prefix(name, ".meminit.text") ||
+ section_prefix(name, ".memexit.text") ||
+ section_prefix(name, ".cpuinit.text") ||
+ section_prefix(name, ".cpuexit.text") ||
+ section_prefix(name, ".devinit.text") ||
+ section_prefix(name, ".devexit.text");
+}
+
+static int remove_unattachable_functions(Elf *elf,
+ struct elf_function_list *functions,
+ size_t shstrndx)
+{
+ GElf_Sym noinstr_start;
+ GElf_Sym noinstr_end;
+ bool found_start;
+ bool found_end;
+ size_t i;
+ int err;
+
+ err = find_symbol(elf, "__noinstr_text_start", &noinstr_start,
+ &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__noinstr_text_end", &noinstr_end, &found_end);
+ if (err)
+ return err;
+
+ for (i = 0; i < functions->nr; i++) {
+ struct elf_function *function = &functions->functions[i];
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ const char *name;
+
+ if (!function->traceable)
+ continue;
+ scn = elf_getscn(elf, function->shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ name = elf_strptr(elf, shstrndx, shdr.sh_name);
+ if (!name)
+ return -1;
+ if (section_cannot_be_attached(name))
+ function->traceable = false;
+ else if (found_start && found_end &&
+ function->shndx == noinstr_start.st_shndx &&
+ noinstr_start.st_shndx == noinstr_end.st_shndx &&
+ function->value >= noinstr_start.st_value &&
+ function->value < noinstr_end.st_value)
+ function->traceable = false;
+ }
+ return 0;
+}
+
+static int read_ftrace_symbol_range(Elf *elf, const GElf_Sym *start,
+ const GElf_Sym *stop, size_t ptr_size,
+ unsigned int encoding,
+ struct elf_function_list *functions,
+ enum ftrace_site_kind kind)
+{
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ size_t begin;
+ size_t end;
+
+ if (start->st_shndx != stop->st_shndx)
+ return -EINVAL;
+ scn = elf_getscn(elf, start->st_shndx);
+ if (!scn || !gelf_getshdr(scn, &shdr))
+ return -1;
+ if (start->st_value < shdr.sh_addr || stop->st_value < start->st_value ||
+ stop->st_value - shdr.sh_addr > shdr.sh_size)
+ return -EINVAL;
+ begin = start->st_value - shdr.sh_addr;
+ end = stop->st_value - shdr.sh_addr;
+ return read_ftrace_site_range(elf, scn, begin, end, ptr_size, encoding,
+ functions, kind);
+}
+
+/*
+ * Mark the functions with an ftrace call site. A module keeps the sites in the
+ * section FTRACE_CALLSITE_SECTION: __patchable_function_entries if PATCHABLE,
+ * else __mcount_loc. x86 also has call padding entries, which are not call
+ * sites, in __patchable_function_entries. vmlinux merges the sites into init
+ * data between the __start_mcount_loc and __stop_mcount_loc symbols, see
+ * MCOUNT_REC().
+ */
+static int read_ftrace_sites(Elf *elf, size_t shstrndx,
+ size_t ptr_size, unsigned int encoding,
+ struct elf_function_list *functions, bool patchable)
+{
+ const char *section = patchable ? "__patchable_function_entries" : "__mcount_loc";
+ Elf_Scn *scn = NULL;
+ GElf_Sym start;
+ GElf_Sym stop;
+ GElf_Sym patchable_start;
+ GElf_Sym patchable_stop;
+ bool found_start;
+ bool found_stop;
+ bool found_patchable_start;
+ bool found_patchable_stop;
+ GElf_Shdr shdr;
+ int err;
+
+ err = find_section(elf, shstrndx, section, &scn);
+ if (err)
+ return err;
+ if (scn) {
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ err = read_ftrace_site_range(elf, scn, 0, shdr.sh_size,
+ ptr_size, encoding, functions,
+ patchable ? FTRACE_SITE_PATCHABLE :
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ }
+
+ err = find_symbol(elf, "__start_mcount_loc", &start, &found_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_mcount_loc", &stop, &found_stop);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__start_patchable_function_entries",
+ &patchable_start, &found_patchable_start);
+ if (err)
+ return err;
+ err = find_symbol(elf, "__stop_patchable_function_entries",
+ &patchable_stop, &found_patchable_stop);
+ if (err)
+ return err;
+ if (found_start && found_stop) {
+ if (found_patchable_start && found_patchable_stop) {
+ err = read_ftrace_symbol_range(elf, &start,
+ &patchable_start,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_MCOUNT);
+ if (err)
+ return err;
+ err = read_ftrace_symbol_range(elf, &patchable_start,
+ &patchable_stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_PATCHABLE);
+ } else {
+ err = read_ftrace_symbol_range(elf, &start, &stop,
+ ptr_size, encoding,
+ functions,
+ FTRACE_SITE_UNKNOWN);
+ }
+ if (err)
+ return err;
+ }
+
+ return remove_unattachable_functions(elf, functions, shstrndx);
+}
+
+/*
+ * Find the function each record points to. In a module the pointer is a
+ * relocation, in a linked image it is the address. REFS maps each pointer
+ * sized slot of the section to a function.
+ */
+static int resolve_record_relocations(Elf *elf, const struct elf_function_list *functions,
+ size_t metadata_ndx, const GElf_Shdr *metadata_shdr,
+ const unsigned char *metadata, size_t *refs,
+ size_t ptr_size, unsigned int encoding)
+{
+ Elf_Scn *scn = NULL;
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ Elf_Data *data, *symbols;
+ Elf_Scn *symtab;
+ GElf_Shdr shdr;
+ size_t i;
+
+ if (!gelf_getshdr(scn, &shdr))
+ return -1;
+ if ((shdr.sh_type != SHT_RELA && shdr.sh_type != SHT_REL) ||
+ shdr.sh_info != metadata_ndx)
+ continue;
+ if (!shdr.sh_entsize || shdr.sh_size % shdr.sh_entsize)
+ return -EINVAL;
+ data = elf_getdata(scn, NULL);
+ symtab = elf_getscn(elf, shdr.sh_link);
+ symbols = symtab ? elf_getdata(symtab, NULL) : NULL;
+ if (!data || !symbols || elf_getdata(scn, data))
+ return -1;
+ for (i = 0; i < shdr.sh_size / shdr.sh_entsize; i++) {
+ GElf_Rela rela;
+ GElf_Sym sym;
+ size_t offset;
+
+ if (shdr.sh_type == SHT_RELA) {
+ if (!gelf_getrela(data, i, &rela))
+ return -1;
+ } else {
+ GElf_Rel rel;
+
+ if (!gelf_getrel(data, i, &rel))
+ return -1;
+ rela = (GElf_Rela){ .r_offset = rel.r_offset,
+ .r_info = rel.r_info };
+ /* REL addends are stored in the pointer itself. */
+ if (!relocation_offset(metadata_shdr, rel.r_offset, ptr_size,
+ &offset))
+ return -EINVAL;
+ rela.r_addend = read_elf_value(metadata + offset, ptr_size,
+ encoding);
+ }
+ if (!relocation_offset(metadata_shdr, rela.r_offset, ptr_size, &offset) ||
+ offset % ptr_size)
+ return -EINVAL;
+ if (!gelf_getsym(symbols, GELF_R_SYM(rela.r_info), &sym))
+ return -1;
+ if (sym.st_shndx == SHN_UNDEF || sym.st_shndx >= SHN_LORESERVE)
+ continue;
+ refs[offset / ptr_size] = function_at(functions, sym.st_shndx,
+ sym.st_value + rela.r_addend);
+ }
+ }
+ return 0;
+}
+
+static size_t record_function(Elf *elf, const struct elf_function_list *functions,
+ const unsigned char *buf, const size_t *refs,
+ size_t offset, size_t ptr_size, unsigned int encoding)
+{
+ Elf64_Section section;
+ GElf_Ehdr ehdr;
+ GElf_Addr value;
+
+ if (refs[offset / ptr_size] != NO_FUNCTION)
+ return refs[offset / ptr_size];
+ if (!gelf_getehdr(elf, &ehdr) || ehdr.e_type == ET_REL)
+ return NO_FUNCTION;
+ value = read_elf_value(buf + offset, ptr_size, encoding);
+ if (!value || section_for_address(elf, value, §ion))
+ return NO_FUNCTION;
+ return function_at(functions, section, value);
+}
+
+/* Remember the records of a body for all the functions at its address */
+static void add_record(struct elf_function_list *functions, size_t first,
+ size_t offset, bool weak)
+{
+ struct elf_function *f = &functions->functions[first];
+ size_t i;
+
+ for (i = first; i < functions->nr &&
+ !cmp_elf_function_location(f, &functions->functions[i]); i++) {
+ if (weak) {
+ functions->functions[i].nr_weak++;
+ functions->functions[i].weak = offset;
+ } else {
+ functions->functions[i].nr_strong++;
+ functions->functions[i].strong = offset;
+ }
+ }
+}
+
+static int parse_records(Elf *elf, struct elf_function_list *functions,
+ const unsigned char *buf, size_t size, const size_t *refs,
+ size_t ptr_size, unsigned int encoding,
+ struct fmodret_stats *stats)
+{
+ size_t offset = 0;
+
+ while (offset < size) {
+ unsigned int flags, length, nr_values;
+ size_t function;
+
+ if (size - offset < FMODRET_META_HEADER_SIZE(ptr_size) ||
+ read_elf_value(buf + offset, 4, encoding) != FMODRET_META_MAGIC ||
+ read_elf_value(buf + offset + 4, 4, encoding) != FMODRET_META_VERSION)
+ goto invalid;
+ length = read_elf_value(buf + offset + 8, 4, encoding);
+ flags = read_elf_value(buf + offset + 12, 4, encoding);
+ nr_values = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ if (length < FMODRET_META_HEADER_SIZE(ptr_size) || length > size - offset ||
+ length % ptr_size ||
+ nr_values != (length - FMODRET_META_HEADER_SIZE(ptr_size)) /
+ FMODRET_META_VALUE_SIZE ||
+ (length - FMODRET_META_HEADER_SIZE(ptr_size)) % FMODRET_META_VALUE_SIZE)
+ goto invalid;
+ stats->records++;
+ function = record_function(elf, functions, buf, refs, offset + 16,
+ ptr_size, encoding);
+ /* A function in a discarded section has no ELF function */
+ if (function == NO_FUNCTION)
+ stats->unmapped++;
+ else
+ add_record(functions, function, offset, flags & FMODRET_FN_WEAK);
+ offset += length;
+ }
+ return 0;
+invalid:
+ fprintf(stderr,
+ "invalid or old fmodret metadata at offset %#zx; rebuild all plugin objects\n",
+ offset);
+ return -EINVAL;
+}
+
+/* Return the record which describes the body of F, or NO_FUNCTION */
+static size_t chosen_record(const struct elf_function *f)
+{
+ if (f->nr_strong == 1)
+ return f->strong;
+ if (!f->nr_strong && f->nr_weak == 1)
+ return f->weak;
+ return NO_FUNCTION;
+}
+
+struct btf_func {
+ const char *name;
+ __u32 id;
+};
+
+static int cmp_btf_func(const void *a, const void *b)
+{
+ const struct btf_func *x = a, *y = b;
+
+ return strcmp(x->name, y->name);
+}
+
+/*
+ * Return the FUNCs of BTF, without those of its base, sorted by name. The BTF
+ * is sorted only after the tags are added, and searching all of it for each
+ * candidate is slow for vmlinux.
+ */
+static struct btf_func *collect_btf_funcs(const struct btf *btf, size_t *cnt)
+{
+ const struct btf *base = btf__base_btf(btf);
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct btf_func *funcs;
+ size_t n = 0;
+
+ funcs = malloc(nr_types * sizeof(*funcs));
+ if (!funcs)
+ return NULL;
+ for (id = base ? btf__type_cnt(base) : 1; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t))
+ continue;
+ funcs[n].name = btf__name_by_offset(btf, t->name_off);
+ funcs[n++].id = id;
+ }
+ qsort(funcs, n, sizeof(*funcs), cmp_btf_func);
+ *cnt = n;
+ return funcs;
+}
+
+/* Return the BTF ID of the only FUNC named NAME, or 0 */
+static int find_btf_func(const struct btf_func *funcs, size_t cnt, const char *name)
+{
+ const struct btf_func key = { .name = name }, *f;
+
+ f = bsearch(&key, funcs, cnt, sizeof(*funcs), cmp_btf_func);
+ if (!f)
+ return 0;
+ if ((f > funcs && !strcmp(f[-1].name, name)) ||
+ (f + 1 < funcs + cnt && !strcmp(f[1].name, name)))
+ return 0;
+ return f->id;
+}
+
+/* Return the size of the integer which FUNC returns, or 0 */
+static __u32 btf_func_int_return_size(const struct btf *btf, int func)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func);
+ int ret;
+
+ t = btf__type_by_id(btf, t->type);
+ if (!t || !btf_is_func_proto(t) || !t->type)
+ return 0;
+ ret = btf__resolve_type(btf, t->type);
+ if (ret < 0)
+ return 0;
+ t = btf__type_by_id(btf, ret);
+ if (!btf_is_int(t) && !btf_is_enum(t) && !btf_is_enum64(t))
+ return 0;
+ return t->size;
+}
+
+/* Format the values of the record at OFFSET as "-22,-12" */
+static char *format_values(const unsigned char *buf, size_t offset, size_t ptr_size,
+ unsigned int encoding)
+{
+ unsigned int nr = read_elf_value(buf + offset + 16 + ptr_size, 4, encoding);
+ const unsigned char *p = buf + offset + FMODRET_META_HEADER_SIZE(ptr_size);
+ size_t size = (size_t)nr * 21 + 1, len = 0;
+ char *s = malloc(size);
+ unsigned int i;
+
+ if (!s)
+ return NULL;
+ for (i = 0; i < nr; i++) {
+ int64_t value = read_elf_value(p + i * FMODRET_META_VALUE_SIZE,
+ FMODRET_META_VALUE_SIZE, encoding);
+
+ len += snprintf(s + len, size - len, "%s%lld", i ? "," : "",
+ (long long)value);
+ }
+ s[len] = 0;
+ return s;
+}
+
+static int cmp_retval(const void *va, const void *vb)
+{
+ const struct fmodret_retval *a = va;
+ const struct fmodret_retval *b = vb;
+
+ return a->btf_id > b->btf_id ? 1 : a->btf_id < b->btf_id ? -1 : 0;
+}
+
+/* Choose the targets among the functions and describe them in RESULT */
+static int choose_functions(struct elf_function_list *functions, const struct btf *btf,
+ const struct btf_func *btf_funcs, size_t nr_btf_funcs,
+ const unsigned char *buf, size_t ptr_size,
+ unsigned int encoding, struct fmodret_retvals *result,
+ struct fmodret_stats *stats, int verbose)
+{
+ size_t i = 0, j;
+
+ qsort(functions->functions, functions->nr, sizeof(*functions->functions),
+ cmp_elf_function_name);
+ for (i = 0; i < functions->nr; i = j) {
+ const struct elf_function *f = &functions->functions[i];
+ size_t record = chosen_record(f);
+ const char *reason = NULL;
+ unsigned int flags = 0;
+ struct fmodret_retval *r;
+ __u32 size;
+ int id = 0;
+
+ for (j = i + 1; j < functions->nr; j++)
+ if (strcmp(f->name, functions->functions[j].name))
+ break;
+
+ if (!f->nr_strong && !f->nr_weak)
+ continue;
+ if (record == NO_FUNCTION) {
+ stats->ambiguous_body++;
+ reason = "ambiguous-body";
+ goto rejected;
+ }
+ /*
+ * A strong definition without values has no record. If it
+ * replaced a weak one, only the record of the weak one points
+ * to its body.
+ */
+ if (!f->nr_strong && !f->weak_symbol) {
+ stats->overridden++;
+ reason = "overridden-weak";
+ goto rejected;
+ }
+ if (!read_elf_value(buf + record + 16 + ptr_size, 4, encoding))
+ continue;
+ stats->with_values++;
+ flags = read_elf_value(buf + record + 12, 4, encoding);
+ if (j - i != 1) {
+ stats->ambiguous_name++;
+ reason = "ambiguous-name";
+ goto rejected;
+ }
+ if (!f->traceable) {
+ stats->not_traceable++;
+ reason = "not-traceable";
+ goto rejected;
+ }
+ id = find_btf_func(btf_funcs, nr_btf_funcs, f->name);
+ if (id <= 0) {
+ stats->no_btf++;
+ reason = "no-unique-BTF-FUNC";
+ goto rejected;
+ }
+ size = btf_func_int_return_size(btf, id);
+ if (size != (flags & FMODRET_FN_RET64 ? 8 : 4)) {
+ stats->return_mismatch++;
+ reason = "BTF-return-type";
+ }
+rejected:
+ if (reason) {
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: rejected %s: %s\n", f->name, reason);
+ continue;
+ }
+
+ if (result->cnt == result->cap) {
+ size_t cap = result->cap ? 2 * result->cap : 256;
+
+ r = realloc(result->retvals, cap * sizeof(*r));
+ if (!r)
+ return -ENOMEM;
+ result->retvals = r;
+ result->cap = cap;
+ }
+ r = &result->retvals[result->cnt];
+ r->btf_id = id;
+ r->values = format_values(buf, record, ptr_size, encoding);
+ if (!r->values)
+ return -ENOMEM;
+ result->cnt++;
+ stats->added++;
+ if (verbose > 1)
+ fprintf(stderr, "fmodret: %s: BTF ID %d, retval %s\n",
+ f->name, id, r->values);
+ }
+ if (result->cnt)
+ qsort(result->retvals, result->cnt, sizeof(*result->retvals), cmp_retval);
+ return 0;
+}
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose)
+{
+ struct elf_function_list functions = {};
+ struct btf_func *btf_funcs = NULL;
+ struct fmodret_stats stats = {};
+ unsigned char *buf = NULL;
+ size_t nr_btf_funcs;
+ size_t *refs = NULL;
+ size_t shstrndx, ptr_size, i;
+ unsigned int encoding;
+ Elf_Scn *scn;
+ GElf_Shdr shdr;
+ int err;
+
+ if (elf_getshdrstrndx(elf, &shstrndx))
+ return -EINVAL;
+ err = find_section(elf, shstrndx, FMODRET_CANDIDATES_SECTION, &scn);
+ if (err || !scn)
+ return err;
+ if (!gelf_getshdr(scn, &shdr))
+ return -EINVAL;
+ ptr_size = gelf_getclass(elf) == ELFCLASS64 ? 8 : 4;
+ encoding = elf_getident(elf, NULL)[EI_DATA];
+ if (shdr.sh_size % ptr_size)
+ return -EINVAL;
+
+ err = collect_elf_functions(elf, &functions);
+ if (err)
+ goto out;
+ qsort(functions.functions, functions.nr, sizeof(*functions.functions),
+ cmp_elf_function_location);
+
+ err = copy_section_data(scn, &shdr, &buf);
+ if (err)
+ goto out;
+ refs = malloc((shdr.sh_size / ptr_size ?: 1) * sizeof(*refs));
+ if (!refs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ for (i = 0; i < shdr.sh_size / ptr_size; i++)
+ refs[i] = NO_FUNCTION;
+ err = resolve_record_relocations(elf, &functions, elf_ndxscn(scn), &shdr, buf,
+ refs, ptr_size, encoding);
+ if (err)
+ goto out;
+ err = parse_records(elf, &functions, buf, shdr.sh_size, refs, ptr_size,
+ encoding, &stats);
+ if (err)
+ goto out;
+ err = read_ftrace_sites(elf, shstrndx, ptr_size, encoding, &functions,
+ patchable_ftrace);
+ if (err)
+ goto out;
+ btf_funcs = collect_btf_funcs(btf, &nr_btf_funcs);
+ if (!btf_funcs) {
+ err = -ENOMEM;
+ goto out;
+ }
+ err = choose_functions(&functions, btf, btf_funcs, nr_btf_funcs, buf, ptr_size,
+ encoding, result, &stats, verbose);
+ if (err)
+ goto out;
+
+ if (verbose)
+ fprintf(stderr,
+ "fmodret: %zu records (%zu without a function), %zu functions with values, %zu added; rejected: %zu ambiguous body, %zu ambiguous name, %zu not traceable, %zu overridden weak, %zu without unique BTF FUNC, %zu BTF return type\n",
+ stats.records, stats.unmapped, stats.with_values, stats.added,
+ stats.ambiguous_body,
+ stats.ambiguous_name, stats.not_traceable, stats.overridden,
+ stats.no_btf, stats.return_mismatch);
+out:
+ if (err == -1)
+ fprintf(stderr, "failed to read %s: %s\n", FMODRET_CANDIDATES_SECTION,
+ elf_errmsg(-1));
+ free(btf_funcs);
+ free(refs);
+ free(buf);
+ free(functions.functions);
+ return err;
+}
+
+void fmodret_free(struct fmodret_retvals *retvals)
+{
+ size_t i;
+
+ for (i = 0; i < retvals->cnt; i++)
+ free(retvals->retvals[i].values);
+ free(retvals->retvals);
+ retvals->retvals = NULL;
+ retvals->cnt = retvals->cap = 0;
+}
diff --git a/tools/bpf/resolve_btfids/fmodret.h b/tools/bpf/resolve_btfids/fmodret.h
new file mode 100644
index 000000000000..c1d5db64970b
--- /dev/null
+++ b/tools/bpf/resolve_btfids/fmodret.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __RESOLVE_BTFIDS_FMODRET_H
+#define __RESOLVE_BTFIDS_FMODRET_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <libelf.h>
+#include <linux/types.h>
+
+struct btf;
+
+/* A function found by the analysis, and the values it may return */
+struct fmodret_retval {
+ __u32 btf_id;
+ char *values;
+};
+
+/* Sorted by BTF ID */
+struct fmodret_retvals {
+ struct fmodret_retval *retvals;
+ size_t cnt, cap;
+};
+
+int fmodret_collect(Elf *elf, const struct btf *btf, struct fmodret_retvals *result,
+ bool patchable_ftrace, int verbose);
+void fmodret_free(struct fmodret_retvals *retvals);
+
+#endif
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index ed1a7fb80ea6..5865c9276983 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -75,6 +75,12 @@
* and also "bpf_fmodret_sleepable" when KF_SLEEPABLE is set, so the sleepable
* tag never comes alone. The kernel builds the set of fmod_ret targets from
* these tags when it loads BTF.
+ *
+ * Functions which the analysis of the compiler plugin records, see fmodret.c,
+ * get "bpf_fmodret" too, and "bpf_fmodret_retval:" with the values they
+ * return on paths without side effects, such as "bpf_fmodret_retval:-22,-12".
+ * The verifier lets programs attached to them return only these values or 0.
+ * A function in a BTF_FMODRET set keeps only its own tags.
*/
#define _GNU_SOURCE
@@ -96,6 +102,7 @@
#include <bpf/btf.h>
#include <bpf/libbpf.h>
#include "btf_colors.h"
+#include "fmodret.h"
#include <subcmd/parse-options.h>
#define BTF_IDS_SECTION ".BTF_ids"
@@ -154,6 +161,11 @@ struct object {
struct btf *orig_btf;
bool distill_base;
bool extract_inline;
+ /* The ftrace call sites are in __patchable_function_entries */
+ bool patchable_ftrace;
+
+ /* Functions found by the analysis of the compiler plugin, see fmodret.c */
+ struct fmodret_retvals fmodret_retvals;
struct {
int fd;
@@ -205,6 +217,7 @@ struct btf_links {
#define DECL_TAG_KFUNC "bpf_kfunc"
#define DECL_TAG_FMODRET "bpf_fmodret"
#define DECL_TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define DECL_TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
#define KF_SLEEPABLE (1 << 5)
#define KF_FASTCALL (1 << 12)
@@ -1791,6 +1804,40 @@ static int add_decl_tag(struct btf2btf_context *ctx, const char *tag_name,
return push_decl_tag_id(ctx, new_id);
}
+static bool has_set_func(const struct rb_root *root, u32 btf_id)
+{
+ const struct rb_node *node = root->rb_node;
+
+ while (node) {
+ const struct kfunc *k = rb_entry(node, struct kfunc, rb_node);
+
+ if (btf_id < k->btf_id)
+ node = node->rb_left;
+ else if (btf_id > k->btf_id)
+ node = node->rb_right;
+ else
+ return true;
+ }
+ return false;
+}
+
+static int add_fmodret_retval_tags(struct btf2btf_context *ctx,
+ const struct fmodret_retval *retval)
+{
+ char *tag;
+ int err;
+
+ err = add_decl_tag(ctx, DECL_TAG_FMODRET, retval->btf_id, -1);
+ if (err)
+ return err;
+
+ if (asprintf(&tag, "%s%s", DECL_TAG_FMODRET_RETVAL, retval->values) < 0)
+ return -ENOMEM;
+ err = add_decl_tag(ctx, tag, retval->btf_id, -1);
+ free(tag);
+ return err;
+}
+
static int btf2btf(struct object *obj)
{
struct btf2btf_context ctx = {};
@@ -1840,6 +1887,16 @@ static int btf2btf(struct object *obj)
}
}
+ for (size_t i = 0; i < obj->fmodret_retvals.cnt; i++) {
+ const struct fmodret_retval *retval = &obj->fmodret_retvals.retvals[i];
+
+ if (has_set_func(&ctx.fmodrets, retval->btf_id))
+ continue;
+ err = add_fmodret_retval_tags(&ctx, retval);
+ if (err)
+ goto out;
+ }
+
err = 0;
out:
free(ctx.decl_tags);
@@ -2193,6 +2250,8 @@ int main(int argc, const char **argv)
"distill --btf_base and emit .BTF.base section data"),
OPT_BOOLEAN(0, "inline", &obj.extract_inline,
"extract location BTF into a .BTF.inline file"),
+ OPT_BOOLEAN(0, "patchable_ftrace", &obj.patchable_ftrace,
+ "ftrace call sites are in __patchable_function_entries"),
OPT_CALLBACK(0, "btf_link", &btf_links, "section:module:btf-file",
"patch a BTF link with --patch_btfids", parse_btf_link),
OPT_STRING(0, "patch_btfids", &btfids_path, "file",
@@ -2251,6 +2310,10 @@ int main(int argc, const char **argv)
if (load_btf(&obj))
goto out;
+ if (fmodret_collect(obj.efile.elf, obj.btf, &obj.fmodret_retvals,
+ obj.patchable_ftrace, verbose))
+ goto out;
+
if (btf2btf(&obj))
goto out;
@@ -2308,6 +2371,7 @@ int main(int argc, const char **argv)
btf_id__free_all(&obj.funcs);
btf_id__free_all(&obj.sets);
free(obj.addr_syms);
+ fmodret_free(&obj.fmodret_retvals);
free(obj.efile.func_symbols);
if (obj.efile.elf) {
elf_end(obj.efile.elf);
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v1 bpf-next 15/18] resolve_btfids: Tag fmod_ret candidates as targets
2026-11-06 21:03 ` [PATCH v1 bpf-next 15/18] resolve_btfids: Tag fmod_ret candidates as targets Anton Protopopov
@ 2026-11-06 21:50 ` bot+bpf-ci
0 siblings, 0 replies; 22+ messages in thread
From: bot+bpf-ci @ 2026-11-06 21:50 UTC (permalink / raw)
To: a.s.protopopov, ast, daniel, andrii, eddyz87, memxor, almagui,
john.fastabend, qmo, brauner, kpsingh, yonghong.song, jolsa,
emil, ihor.solodrai, sashal, bpf, linux-kernel
Cc: a.s.protopopov, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 3921 bytes --]
> resolve_btfids: Tag fmod_ret candidates as targets
TL;DR: Nits only in resolve_btfids fmodret.c: duplicated alias-walk code in
mark_traceable_function(), its unused bool return, and possibly obsolete
section names (.meminit, .cpuinit, .devinit) in section_cannot_be_attached().
> diff --git a/tools/bpf/resolve_btfids/fmodret.c b/tools/bpf/resolve_btfids/fmodret.c
> new file mode 100644
> --- /dev/null
> +++ b/tools/bpf/resolve_btfids/fmodret.c
[ ... ]
> +static bool mark_traceable_function(struct elf_function_list *list,
> + Elf64_Section shndx, GElf_Addr address,
> + enum ftrace_site_kind kind)
> +{
> + size_t left = 0;
> + size_t right = list->nr;
> + size_t first;
> + size_t i;
> + bool found = false;
[ ... ]
> + if (first != SIZE_MAX) {
> + while (first && list->functions[first - 1].shndx == shndx &&
> + list->functions[first - 1].value ==
> + list->functions[first].value)
> + first--;
> + for (i = first; i < list->nr &&
> + list->functions[i].shndx == shndx &&
> + list->functions[i].value ==
> + list->functions[first].value; i++) {
> + list->functions[i].traceable = true;
> + found = true;
> + }
> + return found;
> + }
> + if (kind == FTRACE_SITE_PATCHABLE || !left)
> + return false;
> +
> + /* The nearest preceding function owns the call site. */
> + first = left - 1;
> + if (list->functions[first].shndx != shndx)
> + return false;
> + while (first && list->functions[first - 1].shndx == shndx &&
> + list->functions[first - 1].value == list->functions[first].value)
> + first--;
> + for (i = first; i < list->nr &&
> + list->functions[i].shndx == shndx &&
> + list->functions[i].value == list->functions[first].value; i++) {
> + if (!function_contains(&list->functions[i], address))
> + continue;
> + list->functions[i].traceable = true;
> + found = true;
> + }
> + return found;
> +}
This isn't a bug, but mark_traceable_function() has two near-identical blocks
that walk back to the first alias at a (shndx, value) pair and then loop
forward over the aliases. The second differs only by the
function_contains() test.
Could the walk back use function_at(list, shndx, list->functions[first].value),
with one forward loop that takes the containment test as a condition? Then
the two copies would not need to be kept in sync.
Also, this isn't a bug, but is the bool return of mark_traceable_function()
meant for something, such as a -v count of unmatched ftrace sites? None of
its callers in mark_relocated_ftrace_sites() and read_ftrace_site_range()
read the value, so could it return void and drop 'found'?
[ ... ]
> +static bool section_cannot_be_attached(const char *name)
> +{
> + return section_prefix(name, ".init.text") ||
> + section_prefix(name, ".exit.text") ||
> + section_prefix(name, ".noinstr.text") ||
> + section_prefix(name, ".meminit.text") ||
> + section_prefix(name, ".memexit.text") ||
> + section_prefix(name, ".cpuinit.text") ||
> + section_prefix(name, ".cpuexit.text") ||
> + section_prefix(name, ".devinit.text") ||
> + section_prefix(name, ".devexit.text");
> +}
This isn't a bug, but are .meminit.text, .memexit.text, .cpuinit.text,
.cpuexit.text, .devinit.text and .devexit.text still emitted anywhere?
In include/linux/init.h, __meminit is now either empty or __init, and the
__cpuinit/__devinit annotations were removed. Neither vmlinux.lds.h nor
scripts/module.lds.S places these sections, and modpost lists only
.init.text and .exit.text as init/exit text.
If they are gone, could section_cannot_be_attached() be reduced to
.init.text, .exit.text and .noinstr.text?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42294987431
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v1 bpf-next 16/18] selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (13 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 15/18] resolve_btfids: Tag fmod_ret candidates as targets Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:50 ` bot+bpf-ci
2026-11-06 21:03 ` [PATCH v1 bpf-next 17/18] selftests/bpf: Enable bpf_scan_ir in the config Anton Protopopov
2026-11-06 21:03 ` [PATCH v1 bpf-next 18/18] bpftool: List and show fmod_ret targets Anton Protopopov
16 siblings, 1 reply; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Add bpf_testmod_retval.c to bpf_testmod with functions for bpf_scan_ir.
The "test_retval:" decl tag of each lists the values which bpf_scan_ir
must find. An empty list means that the function must not become an
fmod_ret target. Cover constant errors of both return widths,
assumptions, private stack writes, PHIs, sign extensions, PTR_ERR() of
an ERR_PTR() from an inlined helper, which is not recognized yet, errors
which are not known exactly, observable effects, a notrace function, a
function on the error injection list, a declared target, a weak
definition with an error which a strong one without errors in
bpf_testmod_retval_strong.c replaces and two weak definitions, of which
the one with an error in bpf_testmod_retval_strong.c loses the link.
Rename bpf_testmod.c to bpf_testmod_main.c so the module can consist of
several objects.
Build both files only with the plugin: their functions mean nothing
without it and bpf_testmod_retval.c uses clang extensions such as
__builtin_assume().
Check that the tags in the module BTF match. Check that the verifier
lets fmod_ret programs return only 0 or the listed values on every path,
in the low 32 bits for an int function and in all 64 bits for an s64
one. Declared targets and the static rules stay unrestricted. Check that
an extension cannot replace a function of an fmod_ret program with
restricted return values, neither at load nor when it is attached later,
and that such a program cannot use tail calls.
Skip the test when bpf_testmod has no "test_retval:" tags, that is when
it is built without the plugin.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
.../selftests/bpf/prog_tests/fmodret_retval.c | 205 ++++++++++++
.../selftests/bpf/progs/fmodret_retval.c | 196 +++++++++++
.../selftests/bpf/progs/fmodret_retval_ext.c | 44 +++
.../testing/selftests/bpf/test_kmods/Makefile | 11 +-
.../{bpf_testmod.c => bpf_testmod_main.c} | 0
.../bpf/test_kmods/bpf_testmod_retval.c | 314 ++++++++++++++++++
.../test_kmods/bpf_testmod_retval_strong.c | 28 ++
7 files changed, 797 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
create mode 100644 tools/testing/selftests/bpf/progs/fmodret_retval.c
create mode 100644 tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
rename tools/testing/selftests/bpf/test_kmods/{bpf_testmod.c => bpf_testmod_main.c} (100%)
create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
create mode 100644 tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
diff --git a/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
new file mode 100644
index 000000000000..abb0a9e331b3
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/fmodret_retval.c
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include "fmodret_retval.skel.h"
+#include "fmodret_retval_ext.skel.h"
+
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_RETVAL "bpf_fmodret_retval:"
+#define TAG_TEST_RETVAL "test_retval:"
+
+/* The decl tags of a function */
+struct func_tags {
+ const char *test_retval;
+ const char *retval;
+ bool target;
+};
+
+static bool tag_value(const char *name, const char *key, const char **value)
+{
+ size_t len = strlen(key);
+
+ if (strncmp(name, key, len))
+ return false;
+ *value = name + len;
+ return true;
+}
+
+/*
+ * Return the decl tags of the functions of module BTF, indexed by BTF ID. Only
+ * the types of the module itself are read, not those of its vmlinux base.
+ */
+static struct func_tags *collect_tags(const struct btf *btf, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf);
+ struct func_tags *tags;
+
+ tags = calloc(nr_types, sizeof(*tags));
+ if (!tags)
+ return NULL;
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct func_tags *f;
+ const char *name;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1 ||
+ t->type >= nr_types)
+ continue;
+ f = &tags[t->type];
+ name = btf__name_by_offset(btf, t->name_off);
+ if (!tag_value(name, TAG_TEST_RETVAL, &f->test_retval) &&
+ !tag_value(name, TAG_RETVAL, &f->retval) && !strcmp(name, TAG_FMODRET))
+ f->target = true;
+ }
+
+ return tags;
+}
+
+/* Compare the tags of every fixture with the values it expects */
+static void test_tags(const struct btf *btf, const struct func_tags *tags, int start_id)
+{
+ int id, nr_types = btf__type_cnt(btf), nr_fixtures = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ const char *name;
+
+ if (!btf_is_func(t) || !tags[id].test_retval)
+ continue;
+ name = btf__name_by_offset(btf, t->name_off);
+ ASSERT_STREQ(tags[id].retval ?: "", tags[id].test_retval, name);
+ if (*tags[id].test_retval)
+ ASSERT_TRUE(tags[id].target, name);
+ else if (strcmp(name, "bpf_testmod_retval_declared"))
+ ASSERT_FALSE(tags[id].target, name);
+ nr_fixtures++;
+ }
+ ASSERT_GT(nr_fixtures, 0, "fixtures");
+
+ id = btf__find_by_name_kind(btf, "bpf_testmod_retval_declared", BTF_KIND_FUNC);
+ if (ASSERT_GE(id, start_id, "declared"))
+ ASSERT_TRUE(tags[id].target, "declared_target");
+}
+
+/* Open the programs again with only the extension, to replace FUNC of TARGET */
+static struct fmodret_retval_ext *open_extension(struct bpf_program *target,
+ const char *func, char *log, size_t log_size)
+{
+ struct fmodret_retval_ext *skel;
+
+ skel = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(skel, "fmodret_retval_ext__open"))
+ return NULL;
+ bpf_program__set_autoload(skel->progs.restricted, false);
+ bpf_program__set_autoload(skel->progs.unrestricted, false);
+ if (!ASSERT_OK(bpf_program__set_attach_target(skel->progs.extension,
+ bpf_program__fd(target), func),
+ "set_attach_target")) {
+ fmodret_retval_ext__destroy(skel);
+ return NULL;
+ }
+ bpf_program__set_log_buf(skel->progs.extension, log, log_size);
+ return skel;
+}
+
+/*
+ * An extension could return any value for an fmod_ret program, so it must not
+ * replace one whose return values are restricted, neither at load nor when it
+ * is attached later to another program.
+ */
+static void test_extension(void)
+{
+ struct fmodret_retval_ext *targets, *skel = NULL;
+ struct bpf_link *link;
+ char log[4096] = {};
+
+ if (!is_jit_enabled()) {
+ printf("%s:SKIP:extensions need the JIT\n", __func__);
+ test__skip();
+ return;
+ }
+
+ targets = fmodret_retval_ext__open();
+ if (!ASSERT_OK_PTR(targets, "fmodret_retval_ext__open"))
+ return;
+ bpf_program__set_autoload(targets->progs.extension, false);
+ if (!ASSERT_OK(fmodret_retval_ext__load(targets), "load_targets"))
+ goto out;
+
+ skel = open_extension(targets->progs.restricted, "restricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ ASSERT_ERR(fmodret_retval_ext__load(skel), "load_restricted");
+ ASSERT_HAS_SUBSTR(log, "Cannot extend an fmod_ret program with restricted return values",
+ "log_restricted");
+ fmodret_retval_ext__destroy(skel);
+
+ skel = open_extension(targets->progs.unrestricted, "unrestricted_value", log, sizeof(log));
+ if (!skel)
+ goto out;
+ if (!ASSERT_OK(fmodret_retval_ext__load(skel), "load_unrestricted"))
+ goto out;
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.restricted),
+ "restricted_value");
+ if (!ASSERT_ERR_PTR(link, "attach_restricted"))
+ bpf_link__destroy(link);
+ link = bpf_program__attach_freplace(skel->progs.extension,
+ bpf_program__fd(targets->progs.unrestricted),
+ "unrestricted_value");
+ if (ASSERT_OK_PTR(link, "attach_unrestricted"))
+ bpf_link__destroy(link);
+out:
+ fmodret_retval_ext__destroy(skel);
+ fmodret_retval_ext__destroy(targets);
+}
+
+void test_fmodret_retval(void)
+{
+ struct btf *vmlinux_btf, *btf = NULL;
+ struct func_tags *tags = NULL;
+ int id, start_id, nr_types;
+ bool has_fixtures = false;
+
+ if (!env.has_testmod) {
+ test__skip();
+ return;
+ }
+
+ vmlinux_btf = btf__load_vmlinux_btf();
+ if (!ASSERT_OK_PTR(vmlinux_btf, "btf__load_vmlinux_btf"))
+ return;
+ btf = btf__load_module_btf("bpf_testmod", vmlinux_btf);
+ if (!btf && errno == ENOENT) {
+ printf("%s:SKIP:bpf_testmod has no BTF\n", __func__);
+ test__skip();
+ goto out;
+ }
+ if (!ASSERT_OK_PTR(btf, "btf__load_module_btf"))
+ goto out;
+ start_id = btf__type_cnt(vmlinux_btf);
+ nr_types = btf__type_cnt(btf);
+ tags = collect_tags(btf, start_id);
+ if (!ASSERT_OK_PTR(tags, "collect_tags"))
+ goto out;
+
+ /* The fixtures are built only with the compiler plugin */
+ for (id = start_id; id < nr_types && !has_fixtures; id++)
+ has_fixtures = tags[id].test_retval;
+ if (!has_fixtures) {
+ printf("%s:SKIP:bpf_testmod has no %s tags\n", __func__, TAG_TEST_RETVAL);
+ test__skip();
+ goto out;
+ }
+
+ if (test__start_subtest("tags"))
+ test_tags(btf, tags, start_id);
+ if (test__start_subtest("extension"))
+ test_extension();
+ RUN_TESTS(fmodret_retval);
+out:
+ free(tags);
+ btf__free(btf);
+ btf__free(vmlinux_btf);
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval.c b/tools/testing/selftests/bpf/progs/fmodret_retval.c
new file mode 100644
index 000000000000..30fe15b97484
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval.c
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * fmod_ret programs on the targets which the LLVM IR scanner finds in
+ * bpf_testmod, see test_kmods/bpf_testmod_retval.c. They may return only 0 or
+ * the values of the "bpf_fmodret_retval:" tag of their target.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+/* bpf_testmod_retval_constant() returns -EINVAL */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_einval, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+int BPF_PROG(constant_zero, int ret)
+{
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+int BPF_PROG(constant_other, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_unknown(void)
+{
+ asm volatile (
+ "call %[bpf_get_prandom_u32];"
+ "exit;"
+ :
+ : __imm(bpf_get_prandom_u32)
+ : __clobber_all);
+}
+
+/* The function returns int, so only the low 32 bits count */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__success
+__naked void constant_32bit(void)
+{
+ asm volatile (
+ "w0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Not 0, so the function is not called, but the caller would see 0 */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("it should be 0 or one of -22")
+__naked void constant_high_bits(void)
+{
+ asm volatile (
+ "r0 = 1;"
+ "r0 <<= 32;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_join() returns -EINVAL or -ERANGE */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_einval, int *result, int value, int ret)
+{
+ return -22;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+int BPF_PROG(join_erange, int *result, int value, int ret)
+{
+ return -34;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+int BPF_PROG(join_other, int *result, int value, int ret)
+{
+ return -1;
+}
+
+/* Each path returns an allowed value */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__success
+__naked void join_both_paths(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "exit;"
+"1:"
+ "r0 = -22;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/*
+ * One of two paths to the same exit returns a forbidden value. The verifier
+ * checks the allowed one first and must not prune the other.
+ */
+SEC("fmod_ret/bpf_testmod_retval_join")
+__failure __msg("it should be 0 or one of -34,-22")
+__naked void join_one_forbidden(void)
+{
+ asm volatile (
+ "r2 = *(u64 *)(r1 + 8);"
+ "if r2 s< 0 goto 1f;"
+ "r0 = -34;"
+ "goto 2f;"
+"1:"
+ "r0 = -1;"
+"2:"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* bpf_testmod_retval_wide() returns s64, so all 64 bits count */
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__success
+__load_if_JITed()
+__naked void wide_64bit(void)
+{
+ asm volatile (
+ "r0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+SEC("fmod_ret/bpf_testmod_retval_wide")
+__failure __msg("it should be 0 or one of -14")
+__naked void wide_32bit(void)
+{
+ asm volatile (
+ "w0 = -14;"
+ "exit;"
+ ::: __clobber_all);
+}
+
+/* Declared targets and the static rules are not restricted */
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_any, int ret)
+{
+ return -1;
+}
+
+SEC("fmod_ret/bpf_modify_return_test")
+__success
+int BPF_PROG(static_rule_any, int a, int *b, int ret)
+{
+ return -1;
+}
+
+/* The error injection list allows any value although the target has a tag */
+SEC("fmod_ret/bpf_testmod_retval_injectable")
+__success
+int BPF_PROG(static_rule_tagged, int value, int ret)
+{
+ return -1;
+}
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __uint(max_entries, 1);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(__u32));
+} jmp_table SEC(".maps");
+
+/* A program in a program array may return any value */
+SEC("fmod_ret/bpf_testmod_retval_constant")
+__failure __msg("tail calls are not allowed in fmod_ret programs with restricted return values")
+int BPF_PROG(constant_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+__success
+int BPF_PROG(declared_tail_call, int ret)
+{
+ bpf_tail_call(ctx, &jmp_table, 0);
+ return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
new file mode 100644
index 000000000000..96cef5595842
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/fmodret_retval_ext.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * An extension of fmod_ret programs, see prog_tests/fmodret_retval.c. The
+ * return values of the program on bpf_testmod_retval_constant() are restricted
+ * by its "bpf_fmodret_retval:" tag, those of the program on the declared target
+ * are not. An extension cannot replace the main function of a tracing program,
+ * so each of them calls a global function which the extension replaces.
+ */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+/* Keeps clang from folding the calls */
+int value_bias;
+
+__noinline int restricted_value(int value)
+{
+ return value + value_bias;
+}
+
+__noinline int unrestricted_value(int value)
+{
+ return value + value_bias;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_constant")
+int BPF_PROG(restricted, int ret)
+{
+ return restricted_value(ret) ? -22 : 0;
+}
+
+SEC("fmod_ret/bpf_testmod_retval_declared")
+int BPF_PROG(unrestricted, int ret)
+{
+ return unrestricted_value(ret);
+}
+
+SEC("freplace")
+int extension(int value)
+{
+ return -1;
+}
diff --git a/tools/testing/selftests/bpf/test_kmods/Makefile b/tools/testing/selftests/bpf/test_kmods/Makefile
index 031c7454ce65..475df9a7680e 100644
--- a/tools/testing/selftests/bpf/test_kmods/Makefile
+++ b/tools/testing/selftests/bpf/test_kmods/Makefile
@@ -23,7 +23,16 @@ MODULES = bpf_testmod.ko bpf_test_no_cfi.ko bpf_test_modorder_x.ko \
$(foreach m,$(MODULES),$(eval obj-m += $(m:.ko=.o)))
-CFLAGS_bpf_testmod.o = -I$(src)
+bpf_testmod-y := bpf_testmod_main.o
+# The functions for bpf_scan_ir use clang extensions and need the plugin
+bpf_testmod-$(CONFIG_CLANG_PLUGIN_BPF_SCAN_IR) += bpf_testmod_retval.o \
+ bpf_testmod_retval_strong.o
+# Coverage instrumentation makes every path of the fixtures dirty
+GCOV_PROFILE_bpf_testmod_retval.o := n
+KCOV_INSTRUMENT_bpf_testmod_retval.o := n
+KMSAN_SANITIZE_bpf_testmod_retval.o := n
+
+CFLAGS_bpf_testmod_main.o = -I$(src)
# When BPF_STRICT_BUILD != 0, a missing KDIR is fatal (the default).
# When permissive, skip silently.
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_main.c
similarity index 100%
rename from tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
rename to tools/testing/selftests/bpf/test_kmods/bpf_testmod_main.c
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
new file mode 100644
index 000000000000..926c4a667b78
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
@@ -0,0 +1,314 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Functions for the fmod_ret targets which the LLVM IR scanner finds, see
+ * prog_tests/fmodret_retval.c. The "test_retval:" decl tag of each lists the
+ * values the scanner must find on paths without side effects. An empty list
+ * means that the function must not become such a target.
+ */
+#include <linux/atomic.h>
+#include <linux/btf.h>
+#include <linux/btf_ids.h>
+#include <linux/compiler.h>
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/error-injection.h>
+#include <linux/mutex.h>
+#include <linux/rcupdate.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#define RETVAL(values) __attribute__((btf_decl_tag("test_retval:" values)))
+
+static __used int bpf_testmod_retval_state;
+static __used int *bpf_testmod_retval_captured;
+
+static noinline void bpf_testmod_retval_effect(void)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 1);
+}
+
+static noinline void bpf_testmod_retval_capture(int *ptr)
+{
+ WRITE_ONCE(bpf_testmod_retval_captured, ptr);
+}
+
+static noinline int bpf_testmod_retval_store(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, value);
+ return value;
+}
+
+static noinline int bpf_testmod_retval_clear(int value)
+{
+ WRITE_ONCE(bpf_testmod_retval_state, 0);
+ return value - 1;
+}
+
+/* Constant errors and both return widths */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_constant(void)
+{
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-14") s64 bpf_testmod_retval_wide(void)
+{
+ return -EFAULT;
+}
+
+static __used noinline RETVAL("-1") int bpf_testmod_retval_first_errno(void)
+{
+ return -1;
+}
+
+static __used noinline RETVAL("-4095") int bpf_testmod_retval_last_errno(void)
+{
+ return -MAX_ERRNO;
+}
+
+/*
+ * PTR_ERR() of the ERR_PTR() of an inlined helper. Not recognized yet: the
+ * error reaches the return through a PHI in an earlier block.
+ */
+static __always_inline int *bpf_testmod_retval_slot(int *buf, int idx)
+{
+ if (idx < 0)
+ return ERR_PTR(-EINVAL);
+ if (idx > 8)
+ return ERR_PTR(-EFAULT);
+ return &buf[idx];
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_err_ptr(int *buf, int idx)
+{
+ int *slot = bpf_testmod_retval_slot(buf, idx);
+
+ if (IS_ERR(slot))
+ return PTR_ERR(slot);
+ *slot = 1;
+ return 0;
+}
+
+/* A clean error path may join a dirty success path */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_before_store(int *result, int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ *result = value;
+ return 0;
+}
+
+static __used noinline RETVAL("-34,-22") int bpf_testmod_retval_join(int *result, int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = -ERANGE;
+ else {
+ *result = value;
+ ret = 0;
+ }
+ return ret;
+}
+
+/*
+ * An int error which a long function returns: the PHI of the two calls keeps
+ * the sign extension in the block of the return
+ */
+static __used noinline RETVAL("-22") long bpf_testmod_retval_sign_extend(int value)
+{
+ int ret;
+
+ if (value < 0)
+ ret = -EINVAL;
+ else if (value > 32)
+ ret = bpf_testmod_retval_store(value);
+ else
+ ret = bpf_testmod_retval_clear(value);
+ return ret;
+}
+
+/* An assumption does not make a path dirty */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_assume(int value)
+{
+ __builtin_assume(value > 0);
+ return -EINVAL;
+}
+
+/* Writes to private stack objects are clean, optnone keeps them in the IR */
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_store(int value)
+{
+ int scratch[2];
+
+ scratch[0] = value;
+ scratch[1] = 0;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("-22") __attribute__((optnone))
+int bpf_testmod_retval_private_memset(void)
+{
+ char scratch[32];
+
+ __builtin_memset(scratch, 0, sizeof(scratch));
+ return -EINVAL;
+}
+
+/* An error which is not known exactly is not recorded */
+static __used noinline RETVAL("") int bpf_testmod_retval_masked(unsigned int value)
+{
+ return -1 - (value & 31);
+}
+
+static __used noinline RETVAL("") long bpf_testmod_retval_ptr_err(void *ptr, int *state)
+{
+ if (IS_ERR(ptr))
+ return PTR_ERR(ptr);
+ *state = 1;
+ return 0;
+}
+
+/* Pointer functions are not targets yet */
+static __used noinline RETVAL("") void *bpf_testmod_retval_error_pointer(void)
+{
+ return ERR_PTR(-ENOMEM);
+}
+
+/* Every error path has an observable effect */
+static __used noinline RETVAL("") int bpf_testmod_retval_argument_store(int *state)
+{
+ *state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_global_store(void)
+{
+ bpf_testmod_retval_state = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_volatile_read(int *state)
+{
+ return READ_ONCE(*state) ? -EINVAL : -ERANGE;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_atomic(atomic_t *state)
+{
+ atomic_inc(state);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_call(void)
+{
+ bpf_testmod_retval_effect();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_barrier(void)
+{
+ barrier();
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_only_error_is_dirty(int *state, bool reject)
+{
+ if (reject) {
+ *state = 1;
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static __used noinline RETVAL("")
+int bpf_testmod_retval_lock_held(spinlock_t *lock) __acquires(lock)
+{
+ spin_lock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_stack_escape(void)
+{
+ int scratch;
+
+ bpf_testmod_retval_capture(&scratch);
+ scratch = 1;
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_external_memset(void *dest)
+{
+ __builtin_memset(dest, 0, 32);
+ return -EINVAL;
+}
+
+/* Not recognized yet, although they have no side effects */
+static __used noinline RETVAL("") int bpf_testmod_retval_balanced_spinlock(spinlock_t *lock)
+{
+ spin_lock(lock);
+ spin_unlock(lock);
+ return -EINVAL;
+}
+
+static __used noinline RETVAL("") int bpf_testmod_retval_input_errno(int error)
+{
+ if (error >= -MAX_ERRNO && error < 0)
+ return error;
+ return 0;
+}
+
+/* No ftrace call site, but call padding in __patchable_function_entries on x86 */
+static __used noinline RETVAL("") notrace int bpf_testmod_retval_notrace(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * A weak definition with an error, replaced by a strong one without errors in
+ * bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_overridden(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return -EINVAL;
+}
+
+/*
+ * Two weak definitions: this one with a side effect is linked first and wins
+ * over the one with an error in bpf_testmod_retval_strong.c
+ */
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ bpf_testmod_retval_effect();
+ return 0;
+}
+
+/*
+ * A function on the error injection list gets its tag, but the static rules let
+ * programs return any value
+ */
+static __used noinline RETVAL("-22") int bpf_testmod_retval_injectable(int value)
+{
+ if (value < 0)
+ return -EINVAL;
+ bpf_testmod_retval_effect();
+ return 0;
+}
+ALLOW_ERROR_INJECTION(bpf_testmod_retval_injectable, ERRNO);
+
+/* A declared target keeps only its own tags */
+static __used noinline RETVAL("") int bpf_testmod_retval_declared(void)
+{
+ return -EINVAL;
+}
+
+BTF_FMODRET_START(bpf_testmod_retval_declared_ids)
+BTF_ID_FLAGS(func, bpf_testmod_retval_declared)
+BTF_FMODRET_END(bpf_testmod_retval_declared_ids)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
new file mode 100644
index 000000000000..a246d7f619bc
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval_strong.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Replaces the weak bpf_testmod_retval_overridden() of bpf_testmod_retval.c.
+ * This definition returns no errors, so the function must not become an
+ * fmod_ret target, see prog_tests/fmodret_retval.c.
+ *
+ * The weak bpf_testmod_retval_two_weak() here loses the link to the one of
+ * bpf_testmod_retval.c, so its error must not make the function a target
+ * either.
+ */
+#include <linux/compiler.h>
+#include <linux/errno.h>
+
+int bpf_testmod_retval_overridden(void);
+
+noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_overridden(void)
+{
+ return 0;
+}
+
+int bpf_testmod_retval_two_weak(void);
+
+__weak noinline __attribute__((btf_decl_tag("test_retval:")))
+int bpf_testmod_retval_two_weak(void)
+{
+ return -EINVAL;
+}
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v1 bpf-next 16/18] selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir
2026-11-06 21:03 ` [PATCH v1 bpf-next 16/18] selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir Anton Protopopov
@ 2026-11-06 21:50 ` bot+bpf-ci
0 siblings, 0 replies; 22+ messages in thread
From: bot+bpf-ci @ 2026-11-06 21:50 UTC (permalink / raw)
To: a.s.protopopov, ast, daniel, andrii, eddyz87, memxor, almagui,
john.fastabend, qmo, brauner, kpsingh, yonghong.song, jolsa,
emil, ihor.solodrai, sashal, bpf, linux-kernel
Cc: a.s.protopopov, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2064 bytes --]
> selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir
TL;DR: Nits only in the selftests: a fmod_ret test program's name may
misdescribe its target, and some includes in bpf_testmod_retval.c may be
unused.
> diff --git a/tools/testing/selftests/bpf/progs/fmodret_retval.c b/tools/testing/selftests/bpf/progs/fmodret_retval.c
> new file mode 100644
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/fmodret_retval.c
[ ... ]
> +/* Declared targets and the static rules are not restricted */
> +SEC("fmod_ret/bpf_testmod_retval_declared")
> +__success
> +int BPF_PROG(declared_any, int ret)
> +{
> + return -1;
> +}
> +
> +SEC("fmod_ret/bpf_modify_return_test")
> +__success
> +int BPF_PROG(static_rule_any, int a, int *b, int ret)
> +{
> + return -1;
> +}
This isn't a bug, but bpf_modify_return_test() is in the
bpf_test_modify_return_ids BTF_FMODRET set, so it looks like a declared
target rather than one allowed by check_attach_modify_return().
Would a name like declared_vmlinux_any fit better, or was this meant to
attach to something the static rules allow?
[ ... ]
> diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
> new file mode 100644
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_retval.c
[ ... ]
> +#include <linux/atomic.h>
> +#include <linux/btf.h>
> +#include <linux/btf_ids.h>
> +#include <linux/compiler.h>
> +#include <linux/err.h>
> +#include <linux/errno.h>
> +#include <linux/error-injection.h>
> +#include <linux/mutex.h>
> +#include <linux/rcupdate.h>
> +#include <linux/spinlock.h>
> +#include <linux/types.h>
This isn't a bug, but are linux/mutex.h, linux/rcupdate.h and linux/btf.h
still needed here, or are they left over from earlier fixtures?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42294987431
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v1 bpf-next 17/18] selftests/bpf: Enable bpf_scan_ir in the config
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (14 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 16/18] selftests/bpf: Test fmod_ret candidates found by bpf_scan_ir Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:35 ` bot+bpf-ci
2026-11-06 21:03 ` [PATCH v1 bpf-next 18/18] bpftool: List and show fmod_ret targets Anton Protopopov
16 siblings, 1 reply; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
The bpf_scan_ir plugin is off by default. Without it bpf_testmod has no
fixtures for the plugin and fmodret_retval skips. Enable the plugin in
the selftests config, so CI builds and tests it. Kconfig ignores the
options with GCC and with clang older than 19.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
tools/testing/selftests/bpf/config | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index 49dfe939467c..5461c32cd3cd 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -11,6 +11,7 @@ CONFIG_BPF_STREAM_PARSER=y
CONFIG_BPF_SYSCALL=y
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
CONFIG_CGROUP_BPF=y
+CONFIG_CLANG_PLUGIN_BPF_SCAN_IR=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_MLDSA=y
CONFIG_CRYPTO_SHA256=y
@@ -56,6 +57,7 @@ CONFIG_IPV6_TUNNEL=y
CONFIG_KEYS=y
CONFIG_LIRC=y
CONFIG_LIVEPATCH=y
+CONFIG_LLVM_PLUGINS=y
CONFIG_LWTUNNEL=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_MEMCG=y
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v1 bpf-next 17/18] selftests/bpf: Enable bpf_scan_ir in the config
2026-11-06 21:03 ` [PATCH v1 bpf-next 17/18] selftests/bpf: Enable bpf_scan_ir in the config Anton Protopopov
@ 2026-11-06 21:35 ` bot+bpf-ci
0 siblings, 0 replies; 22+ messages in thread
From: bot+bpf-ci @ 2026-11-06 21:35 UTC (permalink / raw)
To: a.s.protopopov, ast, daniel, andrii, eddyz87, memxor, almagui,
john.fastabend, qmo, brauner, kpsingh, yonghong.song, jolsa,
emil, ihor.solodrai, sashal, bpf, linux-kernel
Cc: a.s.protopopov, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]
> selftests/bpf: Enable bpf_scan_ir in the config
TL;DR: Commit message only: the list of cases where Kconfig ignores the plugin
options looks incomplete (other arches, missing matching llvm-config), so "CI
builds and tests it" may not hold and fmodret_retval may silently skip.
> selftests/bpf: Enable bpf_scan_ir in the config
>
> The bpf_scan_ir plugin is off by default. Without it bpf_testmod has no
> fixtures for the plugin and fmodret_retval skips. Enable the plugin in
> the selftests config, so CI builds and tests it. Kconfig ignores the
> options with GCC and with clang older than 19.
> Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
Is the list of conditions in the last sentence complete? The commit message
says Kconfig ignores the options "with GCC and with clang older than 19", and
that CI "builds and tests it".
scripts/llvm-plugins/Kconfig has more conditions. LLVM_PLUGINS needs
CC_IS_CLANG, a working $(LLVM_CONFIG), an llvm-config with the same major
version as $(CC), and a $(HOSTCXX) that accepts the LLVM headers.
CLANG_PLUGIN_BPF_SCAN_IR needs X86_64 || ARM64, DEBUG_INFO_BTF, !LTO_CLANG and
CLANG_VERSION >= 190000.
So config.s390x, config.ppc64el and config.riscv64 never get the plugin, and a
clang >= 19 host without a matching llvm-dev drops both options silently. In
those cases the build succeeds, bpf_testmod is built without the
bpf_testmod_retval*.o objects, and fmodret_retval reports SKIP.
Could the message list the full set of conditions, or say that "CI builds and
tests it" only holds when a matching llvm-config is available?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42294987431
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v1 bpf-next 18/18] bpftool: List and show fmod_ret targets
2026-11-06 21:02 [PATCH v1 bpf-next 01/18] bpf: Add BTF ID sets for fmod_ret targets Anton Protopopov
` (15 preceding siblings ...)
2026-11-06 21:03 ` [PATCH v1 bpf-next 17/18] selftests/bpf: Enable bpf_scan_ir in the config Anton Protopopov
@ 2026-11-06 21:03 ` Anton Protopopov
2026-11-06 21:35 ` bot+bpf-ci
16 siblings, 1 reply; 22+ messages in thread
From: Anton Protopopov @ 2026-11-06 21:03 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Eduard Zingerman, Kumar Kartikeya Dwivedi, Alan Maguire,
John Fastabend, Quentin Monnet, Christian Brauner, KP Singh,
Yonghong Song, Jiri Olsa, Emil Tsalapatis, Ihor Solodrai,
Sasha Levin, bpf, linux-kernel
Cc: Anton Protopopov
Most fmod_ret targets are now found by the kernel build and only exist
as BTF decl tags. Add a bpftool fmodret command to see them.
The `bpftool fmodret list` command prints the functions tagged with
"bpf_fmodret" in vmlinux and in the loaded modules, and marks the ones
which sleepable programs may attach to.
The `bpftool fmodret show` command prints a function in detail: its
prototype, if it was declared as a target or found by the build, and the
values which fmod_ret programs may return. A function is given by name
or BTF ID.
Both commands read BTF from /sys/kernel/btf or from a file and support
JSON output.
Functions which the verifier allows by its static rules, such as the
security hooks, may also have the tags and be listed. Even if they are
listed, the restrictions of the tags do not apply to them. This may
change in the future by either not listing them, or listing them without
restrictions.
Signed-off-by: Anton Protopopov <a.s.protopopov@gmail.com>
---
.../bpftool/Documentation/bpftool-fmodret.rst | 111 +++
tools/bpf/bpftool/Documentation/bpftool.rst | 2 +-
tools/bpf/bpftool/bash-completion/bpftool | 31 +
tools/bpf/bpftool/fmodret.c | 906 ++++++++++++++++++
tools/bpf/bpftool/main.c | 3 +-
tools/bpf/bpftool/main.h | 1 +
6 files changed, 1052 insertions(+), 2 deletions(-)
create mode 100644 tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
create mode 100644 tools/bpf/bpftool/fmodret.c
diff --git a/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
new file mode 100644
index 000000000000..eef34c18de1b
--- /dev/null
+++ b/tools/bpf/bpftool/Documentation/bpftool-fmodret.rst
@@ -0,0 +1,111 @@
+.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+===============
+bpftool-fmodret
+===============
+-------------------------------------------------------------------------------
+tool to inspect BPF fmod_ret targets
+-------------------------------------------------------------------------------
+
+:Manual section: 8
+
+.. include:: substitutions.rst
+
+SYNOPSIS
+========
+
+**bpftool** [*OPTIONS*] **fmodret** *COMMAND*
+
+*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } *FILE* }
+
+*COMMANDS* := { **list** | **show** | **help** }
+
+FMODRET COMMANDS
+================
+
+| **bpftool** **fmodret list** [*OBJECT*]
+| **bpftool** **fmodret show** *FUNC* [*OBJECT*]
+| **bpftool** **fmodret help**
+|
+| *FUNC* := { **name** *NAME* | **id** *BTF_ID* }
+| *OBJECT* := { **file** *FILE* | **vmlinux** | **module** *NAME* }
+
+DESCRIPTION
+===========
+
+bpftool fmodret list [*OBJECT*]
+ List the fmod_ret targets, the functions which the kernel build tags with
+ the **bpf_fmodret** BTF decl tag, in vmlinux and in loaded modules. Each
+ entry contains the BTF ID of the function, its name and the name of the
+ object it belongs to. Targets which sleepable programs may attach to are
+ marked **sleepable**.
+
+ With **file**, read the BTF from *FILE* (for example a vmlinux image or a
+ **BTF** file) instead of the running kernel. With **vmlinux** or
+ **module**, list only the targets of vmlinux or of the module *NAME*.
+
+ With **--json**, each entry contains **btf_id**, **name**, **owner** and
+ **sleepable** fields.
+
+ The verifier also accepts functions by its static rules, such as the error
+ injection list or **security_** hooks. Such functions are listed only if
+ the kernel build tagged them. Even if they are listed, the restrictions of
+ their **bpf_fmodret_retval:** tags do not apply to them: programs may
+ return any value, and sleepable programs may attach to some of them without
+ the **sleepable** mark. This may change in the future by either not listing
+ them, or listing them without restrictions. Normal verifier and tracing
+ checks apply to all of them.
+
+bpftool fmodret show *FUNC* [*OBJECT*]
+ Show the function named *NAME* or with BTF ID *BTF_ID* in detail. It does
+ not have to be a target. A name may belong to several functions, for
+ example to functions of different modules. A BTF ID is looked up in
+ vmlinux, unless **module** or **file** is given. *OBJECT* is as for
+ **list**.
+
+ Besides the BTF ID, the name and the object, show the prototype of the
+ function and if it is a target: **found by the kernel build**, **declared**
+ or **not tagged**. For a target show if sleepable programs may attach to it
+ and the values which programs may return: 0 and the values of its
+ **bpf_fmodret_retval:** tag with the names of the errors on the host, or
+ **any** for a declared target. A tag which the verifier would reject,
+ malformed, not the only one or with values which the return type cannot
+ hold, is shown as invalid. The values do not apply to a function which the
+ static rules allow, see **list**. For the running kernel, also show the
+ type of the entry of the function in the error injection list, if the list
+ is available in debugfs.
+
+ With **--json**, each function contains **btf_id**, **name**, **owner**,
+ **prototype**, **target** (**found**, **declared** or **none**) and
+ **sleepable** fields, a **retval** array for a target found by the build
+ with the values programs may return, 0 included, or a **retval_error**
+ string if its tag is invalid, and an **error_injection** field if the
+ function is in the error injection list. A BTF ID gives one function, a
+ name an array of them.
+
+bpftool fmodret help
+ Print short help message.
+
+OPTIONS
+========
+
+.. include:: common_options.rst
+
+-B, --base-btf *FILE*
+ Pass the base BTF of a split BTF *FILE*, usually the BTF of vmlinux. Raw
+ BTF of a module needs it. A module image may carry the base it was built
+ against in its **.BTF.base** section instead, but its BTF IDs then differ
+ from those of the running kernel.
+
+EXAMPLES
+========
+
+**# bpftool fmodret show name bpf_testmod_retval_join module bpf_testmod**
+
+::
+
+ 123456 bpf_testmod_retval_join (bpf_testmod)
+ int bpf_testmod_retval_join(int *result, int value)
+ target: found by the kernel build
+ sleepable: no
+ retval: 0, -34 (ERANGE), -22 (EINVAL)
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index f38ae5c40439..2c64c22a5026 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -21,7 +21,7 @@ SYNOPSIS
**bpftool** **version**
*OBJECT* := { **map** | **prog** | **link** | **cgroup** | **perf** | **net** | **feature** |
-**btf** | **gen** | **struct_ops** | **iter** }
+**btf** | **gen** | **struct_ops** | **iter** | **fmodret** }
*OPTIONS* := { { **-V** | **--version** } | |COMMON_OPTIONS| }
diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index a28af31937ad..83288c22b3d3 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -675,6 +675,37 @@ _bpftool()
;;
esac
;;
+ fmodret)
+ case $command in
+ list|show)
+ case $prev in
+ file)
+ _filedir
+ ;;
+ module)
+ COMPREPLY=( $( compgen -W "$( command ls \
+ /sys/kernel/btf 2>/dev/null | \
+ command grep -vx -e vmlinux -e '.*\.inline' )" \
+ -- "$cur" ) )
+ ;;
+ name|id)
+ ;;
+ *)
+ if [[ $prev == show ]]; then
+ COMPREPLY=( $( compgen -W 'name id' -- "$cur" ) )
+ else
+ _bpftool_one_of_list 'file vmlinux module'
+ fi
+ ;;
+ esac
+ return 0
+ ;;
+ *)
+ [[ $prev == $object ]] && \
+ COMPREPLY=( $( compgen -W 'help list show' -- "$cur" ) )
+ ;;
+ esac
+ ;;
map)
case $command in
show|list|dump|peek|pop|dequeue|freeze)
diff --git a/tools/bpf/bpftool/fmodret.c b/tools/bpf/bpftool/fmodret.c
new file mode 100644
index 000000000000..0d2714ef861e
--- /dev/null
+++ b/tools/bpf/bpftool/fmodret.c
@@ -0,0 +1,906 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <dirent.h>
+#include <errno.h>
+#include <limits.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+
+#include "main.h"
+
+/*
+ * Decl tags added by resolve_btfids to functions in BTF_FMODRET sets. A target
+ * which sleepable programs may attach to has both tags. A target which the
+ * build found by analysing its code has a "bpf_fmodret_retval:" tag with the
+ * values programs may return, such as "bpf_fmodret_retval:-22,-12".
+ */
+#define TAG_FMODRET "bpf_fmodret"
+#define TAG_FMODRET_SLEEPABLE "bpf_fmodret_sleepable"
+#define TAG_FMODRET_RETVAL "bpf_fmodret_retval:"
+
+#define SYS_BTF_DIR "/sys/kernel/btf"
+#define ERROR_INJECTION_LIST "/sys/kernel/debug/error_injection/list"
+
+struct fmodret_entry {
+ __u32 btf_id;
+ bool target;
+ bool sleepable;
+ const char *retval;
+ /* More than one "bpf_fmodret_retval:" tag, which the verifier rejects */
+ bool retval_dup;
+};
+
+struct fmodret_filter {
+ /* Only vmlinux, or only the module with this name */
+ bool vmlinux;
+ const char *module;
+ /* The function to show, by name or by BTF ID */
+ const char *name;
+ __u32 btf_id;
+ unsigned int found;
+ /* The BTF is the one of the running kernel */
+ bool live;
+};
+
+typedef int (*btf_cb)(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter);
+
+#define ERRNO_NAME(e) { e, #e }
+
+/* The errors of the uapi and the kernel internal ones, which libc lacks */
+static const struct {
+ int value;
+ const char *name;
+} errno_names[] = {
+ ERRNO_NAME(EPERM),
+ ERRNO_NAME(ENOENT),
+ ERRNO_NAME(ESRCH),
+ ERRNO_NAME(EINTR),
+ ERRNO_NAME(EIO),
+ ERRNO_NAME(ENXIO),
+ ERRNO_NAME(E2BIG),
+ ERRNO_NAME(ENOEXEC),
+ ERRNO_NAME(EBADF),
+ ERRNO_NAME(ECHILD),
+ ERRNO_NAME(EAGAIN),
+ ERRNO_NAME(ENOMEM),
+ ERRNO_NAME(EACCES),
+ ERRNO_NAME(EFAULT),
+ ERRNO_NAME(ENOTBLK),
+ ERRNO_NAME(EBUSY),
+ ERRNO_NAME(EEXIST),
+ ERRNO_NAME(EXDEV),
+ ERRNO_NAME(ENODEV),
+ ERRNO_NAME(ENOTDIR),
+ ERRNO_NAME(EISDIR),
+ ERRNO_NAME(EINVAL),
+ ERRNO_NAME(ENFILE),
+ ERRNO_NAME(EMFILE),
+ ERRNO_NAME(ENOTTY),
+ ERRNO_NAME(ETXTBSY),
+ ERRNO_NAME(EFBIG),
+ ERRNO_NAME(ENOSPC),
+ ERRNO_NAME(ESPIPE),
+ ERRNO_NAME(EROFS),
+ ERRNO_NAME(EMLINK),
+ ERRNO_NAME(EPIPE),
+ ERRNO_NAME(EDOM),
+ ERRNO_NAME(ERANGE),
+ ERRNO_NAME(EDEADLK),
+#if defined(EDEADLOCK) && EDEADLOCK != EDEADLK
+ ERRNO_NAME(EDEADLOCK),
+#endif
+ ERRNO_NAME(ENAMETOOLONG),
+ ERRNO_NAME(ENOLCK),
+ ERRNO_NAME(ENOSYS),
+ ERRNO_NAME(ENOTEMPTY),
+ ERRNO_NAME(ELOOP),
+ ERRNO_NAME(ENOMSG),
+ ERRNO_NAME(EIDRM),
+ ERRNO_NAME(ECHRNG),
+ ERRNO_NAME(EL2NSYNC),
+ ERRNO_NAME(EL3HLT),
+ ERRNO_NAME(EL3RST),
+ ERRNO_NAME(ELNRNG),
+ ERRNO_NAME(EUNATCH),
+ ERRNO_NAME(ENOCSI),
+ ERRNO_NAME(EL2HLT),
+ ERRNO_NAME(EBADE),
+ ERRNO_NAME(EBADR),
+ ERRNO_NAME(EXFULL),
+ ERRNO_NAME(ENOANO),
+ ERRNO_NAME(EBADRQC),
+ ERRNO_NAME(EBADSLT),
+ ERRNO_NAME(EBFONT),
+ ERRNO_NAME(ENOSTR),
+ ERRNO_NAME(ENODATA),
+ ERRNO_NAME(ETIME),
+ ERRNO_NAME(ENOSR),
+ ERRNO_NAME(ENONET),
+ ERRNO_NAME(ENOPKG),
+ ERRNO_NAME(EREMOTE),
+ ERRNO_NAME(ENOLINK),
+ ERRNO_NAME(EADV),
+ ERRNO_NAME(ESRMNT),
+ ERRNO_NAME(ECOMM),
+ ERRNO_NAME(EPROTO),
+ ERRNO_NAME(EMULTIHOP),
+ ERRNO_NAME(EDOTDOT),
+ ERRNO_NAME(EBADMSG),
+ ERRNO_NAME(EOVERFLOW),
+ ERRNO_NAME(ENOTUNIQ),
+ ERRNO_NAME(EBADFD),
+ ERRNO_NAME(EREMCHG),
+ ERRNO_NAME(ELIBACC),
+ ERRNO_NAME(ELIBBAD),
+ ERRNO_NAME(ELIBSCN),
+ ERRNO_NAME(ELIBMAX),
+ ERRNO_NAME(ELIBEXEC),
+ ERRNO_NAME(EILSEQ),
+ ERRNO_NAME(ERESTART),
+ ERRNO_NAME(ESTRPIPE),
+ ERRNO_NAME(EUSERS),
+ ERRNO_NAME(ENOTSOCK),
+ ERRNO_NAME(EDESTADDRREQ),
+ ERRNO_NAME(EMSGSIZE),
+ ERRNO_NAME(EPROTOTYPE),
+ ERRNO_NAME(ENOPROTOOPT),
+ ERRNO_NAME(EPROTONOSUPPORT),
+ ERRNO_NAME(ESOCKTNOSUPPORT),
+ ERRNO_NAME(EOPNOTSUPP),
+ ERRNO_NAME(EPFNOSUPPORT),
+ ERRNO_NAME(EAFNOSUPPORT),
+ ERRNO_NAME(EADDRINUSE),
+ ERRNO_NAME(EADDRNOTAVAIL),
+ ERRNO_NAME(ENETDOWN),
+ ERRNO_NAME(ENETUNREACH),
+ ERRNO_NAME(ENETRESET),
+ ERRNO_NAME(ECONNABORTED),
+ ERRNO_NAME(ECONNRESET),
+ ERRNO_NAME(ENOBUFS),
+ ERRNO_NAME(EISCONN),
+ ERRNO_NAME(ENOTCONN),
+ ERRNO_NAME(ESHUTDOWN),
+ ERRNO_NAME(ETOOMANYREFS),
+ ERRNO_NAME(ETIMEDOUT),
+ ERRNO_NAME(ECONNREFUSED),
+ ERRNO_NAME(EHOSTDOWN),
+ ERRNO_NAME(EHOSTUNREACH),
+ ERRNO_NAME(EALREADY),
+ ERRNO_NAME(EINPROGRESS),
+ ERRNO_NAME(ESTALE),
+ ERRNO_NAME(EUCLEAN),
+ ERRNO_NAME(ENOTNAM),
+ ERRNO_NAME(ENAVAIL),
+ ERRNO_NAME(EISNAM),
+ ERRNO_NAME(EREMOTEIO),
+ ERRNO_NAME(EDQUOT),
+ ERRNO_NAME(ENOMEDIUM),
+ ERRNO_NAME(EMEDIUMTYPE),
+ ERRNO_NAME(ECANCELED),
+ ERRNO_NAME(ENOKEY),
+ ERRNO_NAME(EKEYEXPIRED),
+ ERRNO_NAME(EKEYREVOKED),
+ ERRNO_NAME(EKEYREJECTED),
+ ERRNO_NAME(EOWNERDEAD),
+ ERRNO_NAME(ENOTRECOVERABLE),
+ ERRNO_NAME(ERFKILL),
+ ERRNO_NAME(EHWPOISON),
+#ifdef EFTYPE
+ ERRNO_NAME(EFTYPE),
+#endif
+ { 512, "ERESTARTSYS" },
+ { 513, "ERESTARTNOINTR" },
+ { 514, "ERESTARTNOHAND" },
+ { 515, "ENOIOCTLCMD" },
+ { 516, "ERESTART_RESTARTBLOCK" },
+ { 517, "EPROBE_DEFER" },
+ { 518, "EOPENSTALE" },
+ { 519, "ENOPARAM" },
+ { 521, "EBADHANDLE" },
+ { 522, "ENOTSYNC" },
+ { 523, "EBADCOOKIE" },
+ { 524, "ENOTSUPP" },
+ { 525, "ETOOSMALL" },
+ { 526, "ESERVERFAULT" },
+ { 527, "EBADTYPE" },
+ { 528, "EJUKEBOX" },
+ { 529, "EIOCBQUEUED" },
+ { 530, "ERECALLCONFLICT" },
+ { 531, "ENOGRACE" },
+};
+
+static const char *errno_name(long long value)
+{
+ size_t i;
+
+ for (i = 0; i < ARRAY_SIZE(errno_names); i++)
+ if (-(long long)errno_names[i].value == value)
+ return errno_names[i].name;
+ return NULL;
+}
+
+static int cmp_entry(const void *a, const void *b)
+{
+ const struct fmodret_entry *x = a, *y = b;
+
+ return (x->btf_id > y->btf_id) - (x->btf_id < y->btf_id);
+}
+
+/* The first type ID which belongs to BTF and not to its base */
+static __u32 first_own_type(const struct btf *btf)
+{
+ const struct btf *base = btf__base_btf(btf);
+
+ return base ? btf__type_cnt(base) : 1;
+}
+
+/*
+ * Return the function which T tags with decl tag NAME, or 0. A NAME which ends
+ * with ':' matches the tags which start with it.
+ */
+static __u32 tag_func(const struct btf *btf, const struct btf_type *t, const char *name)
+{
+ size_t len = strlen(name);
+ const char *tag;
+
+ if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1)
+ return 0;
+ tag = btf__name_by_offset(btf, t->name_off);
+ if (strncmp(tag, name, len) || (name[len - 1] != ':' && tag[len]))
+ return 0;
+ if (!btf_is_func(btf__type_by_id(btf, t->type)))
+ return 0;
+ return t->type;
+}
+
+/* Return the fmod_ret targets of BTF, sorted by BTF ID */
+static int collect_targets(const struct btf *btf, struct fmodret_entry **targets,
+ size_t *target_cnt)
+{
+ __u32 id, func_id, start_id = first_own_type(btf);
+ __u32 nr_types = btf__type_cnt(btf);
+ struct fmodret_entry *entries = NULL, *entry, *tmp;
+ size_t cnt = 0, capacity = 0;
+
+ for (id = start_id; id < nr_types; id++) {
+ func_id = tag_func(btf, btf__type_by_id(btf, id), TAG_FMODRET);
+ if (!func_id)
+ continue;
+
+ if (cnt == capacity) {
+ capacity = capacity ? 2 * capacity : 64;
+ tmp = realloc(entries, capacity * sizeof(*entries));
+ if (!tmp) {
+ free(entries);
+ return -ENOMEM;
+ }
+ entries = tmp;
+ }
+ entries[cnt].btf_id = func_id;
+ entries[cnt].target = true;
+ entries[cnt].sleepable = false;
+ entries[cnt].retval = NULL;
+ cnt++;
+ }
+ if (cnt)
+ qsort(entries, cnt, sizeof(*entries), cmp_entry);
+
+ /* The other tags mark functions which are already targets */
+ for (id = start_id; cnt && id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+ struct fmodret_entry key = {};
+
+ key.btf_id = tag_func(btf, t, TAG_FMODRET_SLEEPABLE);
+ if (key.btf_id) {
+ entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
+ if (entry)
+ entry->sleepable = true;
+ continue;
+ }
+ key.btf_id = tag_func(btf, t, TAG_FMODRET_RETVAL);
+ if (key.btf_id) {
+ entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
+ if (entry)
+ entry->retval = btf__name_by_offset(btf, t->name_off) +
+ strlen(TAG_FMODRET_RETVAL);
+ }
+ }
+
+ *targets = entries;
+ *target_cnt = cnt;
+ return 0;
+}
+
+/* Fill ENTRY from the decl tags of function FUNC_ID */
+static void func_entry(const struct btf *btf, __u32 func_id, struct fmodret_entry *entry)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+
+ memset(entry, 0, sizeof(*entry));
+ entry->btf_id = func_id;
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (tag_func(btf, t, TAG_FMODRET) == func_id)
+ entry->target = true;
+ else if (tag_func(btf, t, TAG_FMODRET_SLEEPABLE) == func_id)
+ entry->sleepable = true;
+ else if (tag_func(btf, t, TAG_FMODRET_RETVAL) == func_id) {
+ entry->retval_dup = entry->retval;
+ entry->retval = btf__name_by_offset(btf, t->name_off) +
+ strlen(TAG_FMODRET_RETVAL);
+ }
+ }
+}
+
+static void list_target(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_bool_field(json_wtr, "sleepable", entry->sleepable);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)%s\n", entry->btf_id, name, owner,
+ entry->sleepable ? " sleepable" : "");
+ }
+}
+
+static int list_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ struct fmodret_entry *targets = NULL;
+ size_t cnt = 0, i;
+ int err;
+
+ err = collect_targets(btf, &targets, &cnt);
+ if (err) {
+ p_err("failed to collect fmod_ret targets of %s: %s", owner,
+ strerror(-err));
+ return err;
+ }
+ for (i = 0; i < cnt; i++)
+ list_target(owner, btf, &targets[i]);
+ free(targets);
+ return 0;
+}
+
+struct error_injection {
+ char *name;
+ char *module;
+ char *type;
+};
+
+static struct error_injection *error_injections;
+static size_t error_injection_cnt;
+
+/*
+ * Read the error injection list of the running kernel. Its lines are
+ * "FUNC\tTYPE" and "FUNC [MODULE]\tTYPE". Without debugfs the list is empty.
+ */
+static void read_error_injection_list(void)
+{
+ struct error_injection *tmp;
+ char *line = NULL;
+ size_t len = 0;
+ FILE *f;
+
+ f = fopen(ERROR_INJECTION_LIST, "r");
+ if (!f)
+ return;
+ while (getline(&line, &len, f) > 0) {
+ char *name, *module = NULL, *type, *save;
+
+ /* "NAME [MODULE]\tTYPE" or "NAME\tTYPE" */
+ name = strtok_r(line, " \t\n", &save);
+ type = strtok_r(NULL, " \t\n", &save);
+ if (type && type[0] == '[') {
+ module = type + 1;
+ module[strcspn(module, "]")] = '\0';
+ type = strtok_r(NULL, " \t\n", &save);
+ }
+ if (!name || !type)
+ continue;
+ tmp = realloc(error_injections,
+ (error_injection_cnt + 1) * sizeof(*error_injections));
+ if (!tmp)
+ break;
+ error_injections = tmp;
+ tmp = &error_injections[error_injection_cnt];
+ tmp->name = strdup(name);
+ tmp->module = strdup(module ?: "");
+ tmp->type = strdup(type);
+ if (!tmp->name || !tmp->module || !tmp->type) {
+ free(tmp->name);
+ free(tmp->module);
+ free(tmp->type);
+ break;
+ }
+ error_injection_cnt++;
+ }
+ free(line);
+ fclose(f);
+}
+
+static void free_error_injection_list(void)
+{
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++) {
+ free(error_injections[i].name);
+ free(error_injections[i].module);
+ free(error_injections[i].type);
+ }
+ free(error_injections);
+ error_injections = NULL;
+ error_injection_cnt = 0;
+}
+
+/* Return the error injection type of function NAME of OWNER, or NULL */
+static const char *error_injection_type(const char *name, const char *owner)
+{
+ const char *module = strcmp(owner, "vmlinux") ? owner : "";
+ size_t i;
+
+ for (i = 0; i < error_injection_cnt; i++)
+ if (!strcmp(error_injections[i].name, name) &&
+ !strcmp(error_injections[i].module, module))
+ return error_injections[i].type;
+ return NULL;
+}
+
+static void __printf(2, 0) prototype_printf(void *ctx, const char *fmt, va_list args)
+{
+ vfprintf(ctx, fmt, args);
+}
+
+/* Return the declaration of function FUNC_ID, or NULL */
+static char *func_prototype(const struct btf *btf, __u32 func_id)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ LIBBPF_OPTS(btf_dump_emit_type_decl_opts, opts,
+ .field_name = btf__name_by_offset(btf, t->name_off),
+ );
+ struct btf_dump *d;
+ char *buf = NULL;
+ size_t size = 0;
+ FILE *f;
+ int err;
+
+ f = open_memstream(&buf, &size);
+ if (!f)
+ return NULL;
+ d = btf_dump__new(btf, prototype_printf, f, NULL);
+ err = libbpf_get_error(d);
+ if (!err) {
+ err = btf_dump__emit_type_decl(d, t->type, &opts);
+ btf_dump__free(d);
+ }
+ fclose(f);
+ if (err) {
+ free(buf);
+ return NULL;
+ }
+ return buf;
+}
+
+/* Show 0 and the values of a "bpf_fmodret_retval:" tag as a JSON array */
+static void show_retval_json(const char *retval)
+{
+ const char *p = retval;
+ char *end;
+
+ jsonw_name(json_wtr, "retval");
+ jsonw_start_array(json_wtr);
+ jsonw_int(json_wtr, 0);
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ jsonw_int(json_wtr, value);
+ p = *end == ',' ? end + 1 : end;
+ }
+ jsonw_end_array(json_wtr);
+}
+
+/*
+ * Check the "bpf_fmodret_retval:" tag of FUNC_ID as the verifier does: negative
+ * decimal numbers separated by commas, which the integer or enum of 4 or 8 bytes
+ * that the function returns can hold. Return why it is invalid, or NULL.
+ */
+static const char *retval_invalid(const struct btf *btf, __u32 func_id,
+ const char *retval)
+{
+ const struct btf_type *t = btf__type_by_id(btf, func_id);
+ const char *p = retval;
+ int ret_id;
+ char *end;
+
+ t = btf__type_by_id(btf, t->type);
+ ret_id = t ? btf__resolve_type(btf, t->type) : -EINVAL;
+ t = ret_id > 0 ? btf__type_by_id(btf, ret_id) : NULL;
+ if (!t || !(btf_is_int(t) || btf_is_any_enum(t)) || (t->size != 4 && t->size != 8))
+ return "the function does not return an integer of 4 or 8 bytes";
+ for (;;) {
+ long long value;
+
+ if (*p != '-')
+ return "malformed tag";
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno || value >= 0)
+ return "malformed tag";
+ if (t->size == 4 && value < INT_MIN)
+ return "a value does not fit the return type";
+ if (!*end)
+ return NULL;
+ if (*end != ',')
+ return "malformed tag";
+ p = end + 1;
+ }
+}
+
+/* Show the values of a "bpf_fmodret_retval:" tag with the names of errors */
+static void show_retval_plain(const char *retval)
+{
+ const char *p = retval, *name;
+ char *end;
+
+ printf("\tretval: 0");
+ while (*p) {
+ long long value;
+
+ errno = 0;
+ value = strtoll(p, &end, 10);
+ if (end == p || errno)
+ break;
+ printf(", %lld", value);
+ name = errno_name(value);
+ if (name)
+ printf(" (%s)", name);
+ p = *end == ',' ? end + 1 : end;
+ }
+ printf("\n");
+}
+
+static int show_function(const char *owner, const struct btf *btf,
+ const struct fmodret_entry *entry,
+ struct fmodret_filter *filter)
+{
+ const struct btf_type *t = btf__type_by_id(btf, entry->btf_id);
+ const char *name = btf__name_by_offset(btf, t->name_off);
+ const char *injection = NULL, *invalid = NULL;
+ char *prototype;
+
+ /* The verifier rejects programs on a function with an invalid tag */
+ if (entry->target && entry->retval_dup)
+ invalid = "more than one tag";
+ else if (entry->target && entry->retval)
+ invalid = retval_invalid(btf, entry->btf_id, entry->retval);
+ prototype = func_prototype(btf, entry->btf_id);
+ if (!prototype) {
+ p_err("failed to print the prototype of %s", name);
+ return -EINVAL;
+ }
+ if (filter->live)
+ injection = error_injection_type(name, owner);
+
+ if (json_output) {
+ jsonw_start_object(json_wtr);
+ jsonw_uint_field(json_wtr, "btf_id", entry->btf_id);
+ jsonw_string_field(json_wtr, "name", name);
+ jsonw_string_field(json_wtr, "owner", owner);
+ jsonw_string_field(json_wtr, "prototype", prototype);
+ jsonw_string_field(json_wtr, "target", !entry->target ? "none" :
+ entry->retval ? "found" : "declared");
+ jsonw_bool_field(json_wtr, "sleepable", entry->target && entry->sleepable);
+ if (invalid)
+ jsonw_string_field(json_wtr, "retval_error", invalid);
+ else if (entry->target && entry->retval)
+ show_retval_json(entry->retval);
+ if (injection)
+ jsonw_string_field(json_wtr, "error_injection", injection);
+ jsonw_end_object(json_wtr);
+ } else {
+ printf("%u %s (%s)\n", entry->btf_id, name, owner);
+ printf("\t%s\n", prototype);
+ if (entry->target) {
+ printf("\ttarget: %s\n", entry->retval ? "found by the kernel build" :
+ "declared");
+ printf("\tsleepable: %s\n", entry->sleepable ? "yes" : "no");
+ if (invalid)
+ printf("\tretval: invalid, %s\n", invalid);
+ else if (entry->retval)
+ show_retval_plain(entry->retval);
+ else
+ printf("\tretval: any\n");
+ } else {
+ printf("\ttarget: not tagged, the verifier may accept it by other rules\n");
+ }
+ if (injection)
+ printf("\terror injection: %s\n", injection);
+ }
+ free(prototype);
+ filter->found++;
+ return 0;
+}
+
+static int show_btf(const char *owner, const struct btf *btf,
+ struct fmodret_filter *filter)
+{
+ __u32 id, nr_types = btf__type_cnt(btf);
+ struct fmodret_entry entry;
+ int err;
+
+ if (filter->btf_id) {
+ if (filter->btf_id < first_own_type(btf) || filter->btf_id >= nr_types ||
+ !btf_is_func(btf__type_by_id(btf, filter->btf_id)))
+ return 0;
+ func_entry(btf, filter->btf_id, &entry);
+ return show_function(owner, btf, &entry, filter);
+ }
+
+ for (id = first_own_type(btf); id < nr_types; id++) {
+ const struct btf_type *t = btf__type_by_id(btf, id);
+
+ if (!btf_is_func(t) ||
+ strcmp(btf__name_by_offset(btf, t->name_off), filter->name))
+ continue;
+ func_entry(btf, id, &entry);
+ err = show_function(owner, btf, &entry, filter);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+static int filter_module(const struct dirent *entry)
+{
+ size_t len = strlen(entry->d_name);
+
+ /* Skip ".", "..", vmlinux and the ".inline" companions of modules */
+ return entry->d_name[0] != '.' && strcmp(entry->d_name, "vmlinux") &&
+ !(len > 7 && !strcmp(entry->d_name + len - 7, ".inline"));
+}
+
+static int walk_modules(struct btf *vmlinux, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct dirent **modules = NULL;
+ int i, nr_modules, err = 0;
+
+ if (filter->module) {
+ struct btf *btf;
+
+ btf = btf__load_module_btf(filter->module, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", filter->module,
+ strerror(-err));
+ return err;
+ }
+ err = cb(filter->module, btf, filter);
+ btf__free(btf);
+ return err;
+ }
+
+ nr_modules = scandir(SYS_BTF_DIR, &modules, filter_module, alphasort);
+ if (nr_modules < 0) {
+ /* There is no module directory without module BTF, which is fine */
+ if (errno == ENOENT)
+ return 0;
+ err = -errno;
+ p_err("failed to read %s: %s", SYS_BTF_DIR, strerror(errno));
+ return err;
+ }
+ for (i = 0; i < nr_modules; i++) {
+ const char *name = modules[i]->d_name;
+ struct btf *btf;
+
+ btf = btf__load_module_btf(name, vmlinux);
+ err = libbpf_get_error(btf);
+ if (err == -ENOENT) {
+ /* The module was unloaded meanwhile */
+ err = 0;
+ continue;
+ }
+ if (err) {
+ p_err("failed to load BTF of module %s: %s", name, strerror(-err));
+ break;
+ }
+ err = cb(name, btf, filter);
+ btf__free(btf);
+ if (err)
+ break;
+ }
+ for (i = 0; i < nr_modules; i++)
+ free(modules[i]);
+ free(modules);
+ return err;
+}
+
+static int walk_live_btf(struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *vmlinux;
+ int err = 0;
+
+ vmlinux = btf__load_vmlinux_btf();
+ err = libbpf_get_error(vmlinux);
+ if (err) {
+ p_err("failed to load vmlinux BTF: %s", strerror(-err));
+ return err;
+ }
+ if (!filter->module)
+ err = cb("vmlinux", vmlinux, filter);
+ if (!err && !filter->vmlinux)
+ err = walk_modules(vmlinux, filter, cb);
+ btf__free(vmlinux);
+ return err;
+}
+
+static int walk_file_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ struct btf *btf;
+ int err;
+
+ btf = base_btf ? btf__parse_split(path, base_btf) : btf__parse(path, NULL);
+ err = libbpf_get_error(btf);
+ if (err) {
+ p_err("failed to parse BTF from %s: %s", path, strerror(-err));
+ return err;
+ }
+ err = cb(path, btf, filter);
+ btf__free(btf);
+ return err;
+}
+
+/* Parse the optional OBJECT := { file FILE | vmlinux | module NAME } */
+static int parse_object(int argc, char **argv, const char **path,
+ struct fmodret_filter *filter)
+{
+ if (!argc)
+ return 0;
+ if (is_prefix(*argv, "vmlinux")) {
+ filter->vmlinux = true;
+ NEXT_ARG();
+ } else if (is_prefix(*argv, "file")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ *path = GET_ARG();
+ } else if (is_prefix(*argv, "module")) {
+ NEXT_ARG();
+ if (!REQ_ARGS(1))
+ return -1;
+ filter->module = GET_ARG();
+ }
+ if (argc)
+ return BAD_ARG();
+ return 0;
+}
+
+static int walk_btf(const char *path, struct fmodret_filter *filter, btf_cb cb)
+{
+ return path ? walk_file_btf(path, filter, cb) : walk_live_btf(filter, cb);
+}
+
+static int do_list(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ int err;
+
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+
+ if (json_output)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, list_btf);
+ if (json_output)
+ jsonw_end_array(json_wtr);
+ return err;
+}
+
+static int do_show(int argc, char **argv)
+{
+ struct fmodret_filter filter = {};
+ const char *path = NULL;
+ unsigned long id;
+ char *endptr;
+ int err;
+
+ if (!REQ_ARGS(2))
+ return -1;
+ if (is_prefix(*argv, "name")) {
+ NEXT_ARG();
+ filter.name = GET_ARG();
+ } else if (is_prefix(*argv, "id")) {
+ NEXT_ARG();
+ errno = 0;
+ id = strtoul(*argv, &endptr, 0);
+ if (errno || *endptr || !id || id > UINT_MAX || **argv == '-') {
+ p_err("can't parse %s as BTF ID", *argv);
+ return -1;
+ }
+ filter.btf_id = id;
+ NEXT_ARG();
+ } else {
+ p_err("expected 'name' or 'id', got: '%s'", *argv);
+ return -1;
+ }
+ if (parse_object(argc, argv, &path, &filter))
+ return -1;
+ /* The BTF IDs of different modules overlap */
+ if (filter.btf_id && !path && !filter.module)
+ filter.vmlinux = true;
+
+ filter.live = !path;
+ if (filter.live)
+ read_error_injection_list();
+ /* A BTF ID gives one function, a name an array of them */
+ if (json_output && !filter.btf_id)
+ jsonw_start_array(json_wtr);
+ err = walk_btf(path, &filter, show_btf);
+ if (!err && !filter.found) {
+ if (filter.name)
+ p_err("no function named %s", filter.name);
+ else
+ p_err("no function with BTF ID %u", filter.btf_id);
+ err = -ENOENT;
+ }
+ if (json_output && !filter.btf_id)
+ jsonw_end_array(json_wtr);
+ free_error_injection_list();
+ return err;
+}
+
+static int do_help(int argc, char **argv)
+{
+ if (json_output) {
+ jsonw_null(json_wtr);
+ return 0;
+ }
+
+ fprintf(stderr,
+ "Usage: %1$s %2$s list [OBJECT]\n"
+ " %1$s %2$s show FUNC [OBJECT]\n"
+ " %1$s %2$s help\n"
+ "\n"
+ " FUNC := { name NAME | id BTF_ID }\n"
+ " OBJECT := { file FILE | vmlinux | module NAME }\n"
+ " " HELP_SPEC_OPTIONS " |\n"
+ " {-B|--base-btf} FILE }\n"
+ "\n",
+ bin_name, "fmodret");
+ return 0;
+}
+
+static const struct cmd cmds[] = {
+ { "list", do_list },
+ { "show", do_show },
+ { "help", do_help },
+ { 0 }
+};
+
+int do_fmodret(int argc, char **argv)
+{
+ return cmd_select(cmds, argc, argv, do_help);
+}
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index d15fec71b64e..e2a7571b097c 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -65,7 +65,7 @@ static int do_help(int argc, char **argv)
" %s batch file FILE\n"
" %s version\n"
"\n"
- " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | token }\n"
+ " OBJECT := { prog | map | link | cgroup | perf | net | feature | btf | gen | struct_ops | iter | fmodret | token }\n"
" " HELP_SPEC_OPTIONS " |\n"
" {-V|--version} }\n"
"",
@@ -91,6 +91,7 @@ static const struct cmd commands[] = {
{ "gen", do_gen },
{ "struct_ops", do_struct_ops },
{ "iter", do_iter },
+ { "fmodret", do_fmodret },
{ "token", do_token },
{ "version", do_version },
{ 0 }
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index b1eb4b5d991b..4d877b803b47 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -178,6 +178,7 @@ int do_tracelog(int argc, char **arg) __weak;
int do_feature(int argc, char **argv) __weak;
int do_struct_ops(int argc, char **argv) __weak;
int do_iter(int argc, char **argv) __weak;
+int do_fmodret(int argc, char **argv) __weak;
int do_token(int argc, char **argv) __weak;
int parse_u32_arg(int *argc, char ***argv, __u32 *val, const char *what);
--
2.43.0
^ permalink raw reply [flat|nested] 22+ messages in thread* Re: [PATCH v1 bpf-next 18/18] bpftool: List and show fmod_ret targets
2026-11-06 21:03 ` [PATCH v1 bpf-next 18/18] bpftool: List and show fmod_ret targets Anton Protopopov
@ 2026-11-06 21:35 ` bot+bpf-ci
0 siblings, 0 replies; 22+ messages in thread
From: bot+bpf-ci @ 2026-11-06 21:35 UTC (permalink / raw)
To: a.s.protopopov, ast, daniel, andrii, eddyz87, memxor, almagui,
john.fastabend, qmo, brauner, kpsingh, yonghong.song, jolsa,
emil, ihor.solodrai, sashal, bpf, linux-kernel
Cc: a.s.protopopov, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]
> bpftool: List and show fmod_ret targets
TL;DR: Nit only: in bpftool fmodret.c collect_targets(), the
TAG_FMODRET_RETVAL lookup may be unneeded, since the only caller's list path
doesn't appear to read ->retval.
> diff --git a/tools/bpf/bpftool/fmodret.c b/tools/bpf/bpftool/fmodret.c
> new file mode 100644
> index 000000000000..0d2714ef861e
> --- /dev/null
> +++ b/tools/bpf/bpftool/fmodret.c
[ ... ]
> +/* Return the fmod_ret targets of BTF, sorted by BTF ID */
> +static int collect_targets(const struct btf *btf, struct fmodret_entry **targets,
> + size_t *target_cnt)
> +{
[ ... ]
> + /* The other tags mark functions which are already targets */
> + for (id = start_id; cnt && id < nr_types; id++) {
> + const struct btf_type *t = btf__type_by_id(btf, id);
> + struct fmodret_entry key = {};
> +
> + key.btf_id = tag_func(btf, t, TAG_FMODRET_SLEEPABLE);
> + if (key.btf_id) {
> + entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
> + if (entry)
> + entry->sleepable = true;
> + continue;
> + }
> + key.btf_id = tag_func(btf, t, TAG_FMODRET_RETVAL);
> + if (key.btf_id) {
> + entry = bsearch(&key, entries, cnt, sizeof(*entries), cmp_entry);
> + if (entry)
> + entry->retval = btf__name_by_offset(btf, t->name_off) +
> + strlen(TAG_FMODRET_RETVAL);
> + }
> + }
This isn't a bug, but is the TAG_FMODRET_RETVAL lookup in collect_targets()
needed?
collect_targets() has a single caller, list_btf(), and list_target() does
not seem to read ->retval or ->target from the entries. It looks like the
second loop would only need to handle the TAG_FMODRET_SLEEPABLE tag.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/42294987431
^ permalink raw reply [flat|nested] 22+ messages in thread