* [PATCH bpf-next v5 0/2] bpf, x86: inline bpf_get_current_task() for x86_64
@ 2026-01-19 7:02 Menglong Dong
2026-01-19 7:02 ` [PATCH bpf-next v5 1/2] " Menglong Dong
2026-01-19 7:02 ` [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong
0 siblings, 2 replies; 6+ messages in thread
From: Menglong Dong @ 2026-01-19 7:02 UTC (permalink / raw)
To: ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64
to obtain better performance, and add the testcase for it.
Changes since v4:
* don't support the !CONFIG_SMP case
* v4: https://lore.kernel.org/bpf/20260112104529.224645-1-dongml2@chinatelecom.cn/
Changes since v3:
* handle the !CONFIG_SMP case
* ignore the !CONFIG_SMP case in the testcase, as we enable CONFIG_SMP
for x86_64 in the selftests
Changes since v2:
* implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in
x86_64 JIT (Alexei).
Changes since v1:
* add the testcase
* remove the usage of const_current_task
Menglong Dong (2):
bpf, x86: inline bpf_get_current_task() for x86_64
selftests/bpf: test the jited inline of bpf_get_current_task
kernel/bpf/verifier.c | 22 ++++++++++++
.../selftests/bpf/prog_tests/verifier.c | 2 ++
.../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++
3 files changed, 59 insertions(+)
create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c
--
2.52.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH bpf-next v5 1/2] bpf, x86: inline bpf_get_current_task() for x86_64
2026-01-19 7:02 [PATCH bpf-next v5 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong
@ 2026-01-19 7:02 ` Menglong Dong
2026-01-19 20:43 ` Eduard Zingerman
2026-01-19 7:02 ` [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong
1 sibling, 1 reply; 6+ messages in thread
From: Menglong Dong @ 2026-01-19 7:02 UTC (permalink / raw)
To: ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64
to obtain better performance.
In !CONFIG_SMP case, the percpu variable is just a normal variable, and
we can read the current_task directly.
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
---
v5:
- don't support the !CONFIG_SMP case
v4:
- handle the !CONFIG_SMP case
v3:
- implement it in the verifier with BPF_MOV64_PERCPU_REG() instead of in
x86_64 JIT.
---
kernel/bpf/verifier.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 9de0ec0c3ed9..c4e2ffadfb1f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -17739,6 +17739,10 @@ static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm)
switch (imm) {
#ifdef CONFIG_X86_64
case BPF_FUNC_get_smp_processor_id:
+#ifdef CONFIG_SMP
+ case BPF_FUNC_get_current_task_btf:
+ case BPF_FUNC_get_current_task:
+#endif
return env->prog->jit_requested && bpf_jit_supports_percpu_insn();
#endif
default:
@@ -23319,6 +23323,24 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
insn = new_prog->insnsi + i + delta;
goto next_insn;
}
+
+ /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */
+ if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) &&
+ verifier_inlines_helper_call(env, insn->imm)) {
+ insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task);
+ insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0);
+ insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0);
+ cnt = 3;
+
+ new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt);
+ if (!new_prog)
+ return -ENOMEM;
+
+ delta += cnt - 1;
+ env->prog = prog = new_prog;
+ insn = new_prog->insnsi + i + delta;
+ goto next_insn;
+ }
#endif
/* Implement bpf_get_func_arg inline. */
if (prog_type == BPF_PROG_TYPE_TRACING &&
--
2.52.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next v5 1/2] bpf, x86: inline bpf_get_current_task() for x86_64
2026-01-19 7:02 ` [PATCH bpf-next v5 1/2] " Menglong Dong
@ 2026-01-19 20:43 ` Eduard Zingerman
0 siblings, 0 replies; 6+ messages in thread
From: Eduard Zingerman @ 2026-01-19 20:43 UTC (permalink / raw)
To: Menglong Dong, ast
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
On Mon, 2026-01-19 at 15:02 +0800, Menglong Dong wrote:
> Inline bpf_get_current_task() and bpf_get_current_task_btf() for x86_64
^^^^^^
Nit: this change is no longer x86 specific.
> to obtain better performance.
>
> In !CONFIG_SMP case, the percpu variable is just a normal variable, and
> we can read the current_task directly.
>
> Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
> @@ -23319,6 +23323,24 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
> insn = new_prog->insnsi + i + delta;
> goto next_insn;
> }
> +
> + /* Implement bpf_get_current_task() and bpf_get_current_task_btf() inline. */
> + if ((insn->imm == BPF_FUNC_get_current_task || insn->imm == BPF_FUNC_get_current_task_btf) &&
> + verifier_inlines_helper_call(env, insn->imm)) {
> + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_0, (u32)(unsigned long)¤t_task);
> + insn_buf[1] = BPF_MOV64_PERCPU_REG(BPF_REG_0, BPF_REG_0);
> + insn_buf[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0);
^^^^^^^
Silly question:
This assumes pointer size of 8 bytes.
Which is true for all archs where bpf_jit_supports_percpu_insn()
returns true at the moment. Do we want an additional safety check
here?
> + cnt = 3;
> +
> + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt);
> + if (!new_prog)
> + return -ENOMEM;
> +
> + delta += cnt - 1;
> + env->prog = prog = new_prog;
> + insn = new_prog->insnsi + i + delta;
> + goto next_insn;
> + }
> #endif
> /* Implement bpf_get_func_arg inline. */
> if (prog_type == BPF_PROG_TYPE_TRACING &&
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task
2026-01-19 7:02 [PATCH bpf-next v5 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong
2026-01-19 7:02 ` [PATCH bpf-next v5 1/2] " Menglong Dong
@ 2026-01-19 7:02 ` Menglong Dong
2026-01-19 20:37 ` Eduard Zingerman
1 sibling, 1 reply; 6+ messages in thread
From: Menglong Dong @ 2026-01-19 7:02 UTC (permalink / raw)
To: ast, eddyz87
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
Add the testcase for the jited inline of bpf_get_current_task().
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
---
.../selftests/bpf/prog_tests/verifier.c | 2 ++
.../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++
2 files changed, 37 insertions(+)
create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c
diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
index 38c5ba70100c..2ae7b096bd64 100644
--- a/tools/testing/selftests/bpf/prog_tests/verifier.c
+++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
@@ -111,6 +111,7 @@
#include "verifier_xdp_direct_packet_access.skel.h"
#include "verifier_bits_iter.skel.h"
#include "verifier_lsm.skel.h"
+#include "verifier_jit_inline.skel.h"
#include "irq.skel.h"
#define MAX_ENTRIES 11
@@ -253,6 +254,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); }
void test_verifier_lsm(void) { RUN(verifier_lsm); }
void test_irq(void) { RUN(irq); }
void test_verifier_mtu(void) { RUN(verifier_mtu); }
+void test_verifier_jit_inline(void) { RUN(verifier_jit_inline); }
static int init_test_val_map(struct bpf_object *obj, char *map_name)
{
diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
new file mode 100644
index 000000000000..0938ca1dac87
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
+
+SEC("fentry/bpf_fentry_test1")
+__description("Jit inline, bpf_get_current_task")
+__success __retval(0)
+__arch_x86_64
+__jited(" addq %gs:{{.*}}, %rax")
+__arch_arm64
+__jited(" mrs x7, SP_EL0")
+int inline_bpf_get_current_task(void)
+{
+ bpf_get_current_task();
+
+ return 0;
+}
+
+#else
+
+SEC("kprobe")
+__description("Jit inline is not supported, use a dummy test")
+__success
+int dummy_test(void)
+{
+ return 0;
+}
+
+#endif
+
+char _license[] SEC("license") = "GPL";
--
2.52.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task
2026-01-19 7:02 ` [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong
@ 2026-01-19 20:37 ` Eduard Zingerman
2026-01-20 1:27 ` Menglong Dong
0 siblings, 1 reply; 6+ messages in thread
From: Eduard Zingerman @ 2026-01-19 20:37 UTC (permalink / raw)
To: Menglong Dong, ast
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
On Mon, 2026-01-19 at 15:02 +0800, Menglong Dong wrote:
> Add the testcase for the jited inline of bpf_get_current_task().
>
> Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
> ---
> .../selftests/bpf/prog_tests/verifier.c | 2 ++
> .../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++
> 2 files changed, 37 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
> index 38c5ba70100c..2ae7b096bd64 100644
> --- a/tools/testing/selftests/bpf/prog_tests/verifier.c
> +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
> @@ -111,6 +111,7 @@
> #include "verifier_xdp_direct_packet_access.skel.h"
> #include "verifier_bits_iter.skel.h"
> #include "verifier_lsm.skel.h"
> +#include "verifier_jit_inline.skel.h"
> #include "irq.skel.h"
>
> #define MAX_ENTRIES 11
> @@ -253,6 +254,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); }
> void test_verifier_lsm(void) { RUN(verifier_lsm); }
> void test_irq(void) { RUN(irq); }
> void test_verifier_mtu(void) { RUN(verifier_mtu); }
> +void test_verifier_jit_inline(void) { RUN(verifier_jit_inline); }
>
> static int init_test_val_map(struct bpf_object *obj, char *map_name)
> {
> diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
> new file mode 100644
> index 000000000000..0938ca1dac87
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
> @@ -0,0 +1,35 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include <vmlinux.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_misc.h"
> +
> +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
I don't think this #if is necessary as well as 'dummy_test'.
test_loader.c:run_subtest() checks current architecture against a mask
supplied by __arch_* annotations and skips the test for unsupported
archs.
> +
> +SEC("fentry/bpf_fentry_test1")
> +__description("Jit inline, bpf_get_current_task")
Nit: please don't use __description() for new tests,
it makes "./test_progs -t" tests selection harder.
> +__success __retval(0)
> +__arch_x86_64
> +__jited(" addq %gs:{{.*}}, %rax")
> +__arch_arm64
> +__jited(" mrs x7, SP_EL0")
> +int inline_bpf_get_current_task(void)
> +{
> + bpf_get_current_task();
> +
> + return 0;
> +}
> +
> +#else
> +
> +SEC("kprobe")
> +__description("Jit inline is not supported, use a dummy test")
> +__success
> +int dummy_test(void)
> +{
> + return 0;
> +}
> +
> +#endif
> +
> +char _license[] SEC("license") = "GPL";
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task
2026-01-19 20:37 ` Eduard Zingerman
@ 2026-01-20 1:27 ` Menglong Dong
0 siblings, 0 replies; 6+ messages in thread
From: Menglong Dong @ 2026-01-20 1:27 UTC (permalink / raw)
To: Menglong Dong, ast, Eduard Zingerman
Cc: daniel, john.fastabend, andrii, martin.lau, song, yonghong.song,
kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel
On 2026/1/20 04:37, Eduard Zingerman wrote:
> On Mon, 2026-01-19 at 15:02 +0800, Menglong Dong wrote:
> > Add the testcase for the jited inline of bpf_get_current_task().
> >
> > Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
> > ---
> > .../selftests/bpf/prog_tests/verifier.c | 2 ++
> > .../selftests/bpf/progs/verifier_jit_inline.c | 35 +++++++++++++++++++
> > 2 files changed, 37 insertions(+)
> > create mode 100644 tools/testing/selftests/bpf/progs/verifier_jit_inline.c
> >
> > diff --git a/tools/testing/selftests/bpf/prog_tests/verifier.c b/tools/testing/selftests/bpf/prog_tests/verifier.c
> > index 38c5ba70100c..2ae7b096bd64 100644
> > --- a/tools/testing/selftests/bpf/prog_tests/verifier.c
> > +++ b/tools/testing/selftests/bpf/prog_tests/verifier.c
> > @@ -111,6 +111,7 @@
> > #include "verifier_xdp_direct_packet_access.skel.h"
> > #include "verifier_bits_iter.skel.h"
> > #include "verifier_lsm.skel.h"
> > +#include "verifier_jit_inline.skel.h"
> > #include "irq.skel.h"
> >
> > #define MAX_ENTRIES 11
> > @@ -253,6 +254,7 @@ void test_verifier_bits_iter(void) { RUN(verifier_bits_iter); }
> > void test_verifier_lsm(void) { RUN(verifier_lsm); }
> > void test_irq(void) { RUN(irq); }
> > void test_verifier_mtu(void) { RUN(verifier_mtu); }
> > +void test_verifier_jit_inline(void) { RUN(verifier_jit_inline); }
> >
> > static int init_test_val_map(struct bpf_object *obj, char *map_name)
> > {
> > diff --git a/tools/testing/selftests/bpf/progs/verifier_jit_inline.c b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
> > new file mode 100644
> > index 000000000000..0938ca1dac87
> > --- /dev/null
> > +++ b/tools/testing/selftests/bpf/progs/verifier_jit_inline.c
> > @@ -0,0 +1,35 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include <vmlinux.h>
> > +#include <bpf/bpf_helpers.h>
> > +#include "bpf_misc.h"
> > +
> > +#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64)
>
> I don't think this #if is necessary as well as 'dummy_test'.
> test_loader.c:run_subtest() checks current architecture against a mask
> supplied by __arch_* annotations and skips the test for unsupported
> archs.
Yeah, you are right, they are unnecessary.
>
> > +
> > +SEC("fentry/bpf_fentry_test1")
> > +__description("Jit inline, bpf_get_current_task")
>
> Nit: please don't use __description() for new tests,
> it makes "./test_progs -t" tests selection harder.
OK, I'll remove them.
Thanks!
Menglong Dong
>
> > +__success __retval(0)
> > +__arch_x86_64
> > +__jited(" addq %gs:{{.*}}, %rax")
> > +__arch_arm64
> > +__jited(" mrs x7, SP_EL0")
> > +int inline_bpf_get_current_task(void)
> > +{
> > + bpf_get_current_task();
> > +
> > + return 0;
> > +}
> > +
> > +#else
> > +
> > +SEC("kprobe")
> > +__description("Jit inline is not supported, use a dummy test")
> > +__success
> > +int dummy_test(void)
> > +{
> > + return 0;
> > +}
> > +
> > +#endif
> > +
> > +char _license[] SEC("license") = "GPL";
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-01-20 1:36 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-01-19 7:02 [PATCH bpf-next v5 0/2] bpf, x86: inline bpf_get_current_task() for x86_64 Menglong Dong
2026-01-19 7:02 ` [PATCH bpf-next v5 1/2] " Menglong Dong
2026-01-19 20:43 ` Eduard Zingerman
2026-01-19 7:02 ` [PATCH bpf-next v5 2/2] selftests/bpf: test the jited inline of bpf_get_current_task Menglong Dong
2026-01-19 20:37 ` Eduard Zingerman
2026-01-20 1:27 ` Menglong Dong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®