[PATCH bpf-next v2 0/2] bpf, riscv: Add support for indirect jumps

[PATCH bpf-next v2 0/2] bpf, riscv: Add support for indirect jumps

[Chen Pei]

The indirect jump instruction (BPF_JMP | BPF_JA | BPF_X, "gotox") and the
BPF_MAP_TYPE_INSN_ARRAY jump tables it consumes are core features: the
verifier accepts them on every architecture, and the x86-64, arm64 and
powerpc JITs implement them. On riscv64 a program using gotox passes
verification and then fails to load, because the JIT does not know the
opcode and CONFIG_BPF_JIT_ALWAYS_ON leaves no interpreter to fall back
onto.

This series adds the riscv64 support and turns the existing selftests on
for that architecture.

Patch 1 emits "jalr zero, rd, 0" for gotox and publishes the xlated to
jitted offsets through bpf_prog_update_insn_ptrs(), which is what fills
in the jump table addresses. Patch 2 widens the arch guard in
verifier_gotox.c to riscv64, the same way arm64 and powerpc were enabled.

Changes since v1:
- Rebased onto bpf-next.
- Patch 1: reword the code comment to say "shift ctx->offset[] by one".
- Patch 2: collapse the multi-line #endif marker to a single line.
- Added Reviewed-by/Acked-by from Björn Töpel.

Testing
=======

Environment: QEMU virt rv64, with CONFIG_BPF_JIT=y,
CONFIG_BPF_JIT_ALWAYS_ON=y, CONFIG_DEBUG_INFO_BTF=y; selftests
cross-built with clang 22.

- test_progs -t verifier_gotox: 27/27 subtests pass on bpf-next, 13 of
  them executed through BPF_PROG_TEST_RUN.
- test_progs-cpuv4 -t bpf_gotox: 14/14 subtests pass, none skipped.
  The cpuv4 flavor is needed here because bpf_gotox gates its subtests
  on __BPF_FEATURE_GOTOX, which clang only defines for -mcpu=v4.

Chen Pei (2):
  bpf, riscv: Add support for indirect jumps
  selftests/bpf: Enable gotox tests for riscv64

 arch/riscv/net/bpf_jit_comp64.c                  |  5 +++++
 arch/riscv/net/bpf_jit_core.c                    | 16 ++++++++++++++--
 .../testing/selftests/bpf/progs/verifier_gotox.c |  6 ++++--
 3 files changed, 23 insertions(+), 4 deletions(-)

-- 
2.50.1

[PATCH bpf-next v2 1/2] bpf, riscv: Add support for indirect jumps

[Chen Pei]

Implement JIT support for the indirect jump instruction (BPF_JMP |
BPF_JA | BPF_X), a.k.a. gotox, which lets a BPF program jump through a
BPF_MAP_TYPE_INSN_ARRAY jump table.

Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to
bpf_prog_update_insn_ptrs(), which is what fills in the jump table
entries; without that call the load fails with -EFAULT in
bpf_insn_array_ready(). ctx->offset[] holds the offset of the insn
*following* insn i, as bpf_prog_fill_jited_linfo() expects, so it is
shifted by one and offset[0] comes from the prologue length. build_body()
now records both halves of a multi-insn record, so no slot keeps a
fabricated offset.

Only the RV64 JIT is covered; RV32 keeps failing to load as before.

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
---

Changes since v1:
- Reword the code comment to say "shift ctx->offset[] by one".

 arch/riscv/net/bpf_jit_comp64.c |  5 +++++
 arch/riscv/net/bpf_jit_core.c   | 16 ++++++++++++++--
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index ed0a6f871dea..9de3749fb268 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
 			emit_zextw(rd, rd, ctx);
 		break;
 
+	/* JUMP reg */
+	case BPF_JMP | BPF_JA | BPF_X:
+		emit_jalr(RV_REG_ZERO, rd, 0, ctx);
+		break;
+
 	/* JUMP off */
 	case BPF_JMP | BPF_JA:
 	case BPF_JMP32 | BPF_JA:
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 470a6ace5662..5265dd5bd39e 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)
 		int ret;
 
 		ret = bpf_jit_emit_insn(insn, ctx, extra_pass);
-		if (ret > 0)
-			i++; /* skip the next instruction */
 		if (offset)
 			offset[i] = ctx->ninsns;
+		if (ret > 0) {
+			i++; /* skip the next instruction */
+			if (offset)
+				offset[i] = ctx->ninsns;
+		}
 		if (ret < 0)
 			return ret;
 	}
@@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
 		for (i = 0; i < prog->len; i++)
 			ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
 		bpf_prog_fill_jited_linfo(prog, ctx->offset);
+
+		/*
+		 * bpf_prog_update_insn_ptrs() wants the start of each insn, so
+		 * shift ctx->offset[] by one and get insn 0 from the prologue.
+		 */
+		for (i = prog->len - 1; i > 0; i--)
+			ctx->offset[i] = ctx->offset[i - 1];
+		ctx->offset[0] = ninsns_rvoff(ctx->prologue_len);
+		bpf_prog_update_insn_ptrs(prog, ctx->offset, jit_data->ro_image);
 out_offset:
 		kvfree(ctx->offset);
 		kfree(jit_data);
-- 
2.50.1

Re: [PATCH bpf-next v2 1/2] bpf, riscv: Add support for indirect jumps

[Pu Lehui]

On 2026/9/28 10:21, Chen Pei wrote:
> Implement JIT support for the indirect jump instruction (BPF_JMP |
> BPF_JA | BPF_X), a.k.a. gotox, which lets a BPF program jump through a
> BPF_MAP_TYPE_INSN_ARRAY jump table.
> 
> Emit "jalr zero, rd, 0" and hand the xlated to jitted offsets to
> bpf_prog_update_insn_ptrs(), which is what fills in the jump table
> entries; without that call the load fails with -EFAULT in
> bpf_insn_array_ready(). ctx->offset[] holds the offset of the insn
> *following* insn i, as bpf_prog_fill_jited_linfo() expects, so it is
> shifted by one and offset[0] comes from the prologue length. build_body()
> now records both halves of a multi-insn record, so no slot keeps a
> fabricated offset.
> 
> Only the RV64 JIT is covered; RV32 keeps failing to load as before.
> 
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> Reviewed-by: Björn Töpel <bjorn@kernel.org>
> Acked-by: Björn Töpel <bjorn@kernel.org>
> ---
> 
> Changes since v1:
> - Reword the code comment to say "shift ctx->offset[] by one".
> 
>   arch/riscv/net/bpf_jit_comp64.c |  5 +++++
>   arch/riscv/net/bpf_jit_core.c   | 16 ++++++++++++++--
>   2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
> index ed0a6f871dea..9de3749fb268 100644
> --- a/arch/riscv/net/bpf_jit_comp64.c
> +++ b/arch/riscv/net/bpf_jit_comp64.c
> @@ -1691,6 +1691,11 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
>   			emit_zextw(rd, rd, ctx);
>   		break;
>   
> +	/* JUMP reg */
> +	case BPF_JMP | BPF_JA | BPF_X:
> +		emit_jalr(RV_REG_ZERO, rd, 0, ctx);
> +		break;
> +
>   	/* JUMP off */
>   	case BPF_JMP | BPF_JA:
>   	case BPF_JMP32 | BPF_JA:
> diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
> index 470a6ace5662..5265dd5bd39e 100644
> --- a/arch/riscv/net/bpf_jit_core.c
> +++ b/arch/riscv/net/bpf_jit_core.c
> @@ -26,10 +26,13 @@ static int build_body(struct rv_jit_context *ctx, bool extra_pass, int *offset)
>   		int ret;
>   
>   		ret = bpf_jit_emit_insn(insn, ctx, extra_pass);
> -		if (ret > 0)
> -			i++; /* skip the next instruction */
>   		if (offset)
>   			offset[i] = ctx->ninsns;
> +		if (ret > 0) {
> +			i++; /* skip the next instruction */
> +			if (offset)
> +				offset[i] = ctx->ninsns;
> +		}
>   		if (ret < 0)
>   			return ret;
>   	}
> @@ -176,6 +179,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
>   		for (i = 0; i < prog->len; i++)
>   			ctx->offset[i] = ninsns_rvoff(ctx->offset[i]);
>   		bpf_prog_fill_jited_linfo(prog, ctx->offset);
> +
> +		/*
> +		 * bpf_prog_update_insn_ptrs() wants the start of each insn, so
> +		 * shift ctx->offset[] by one and get insn 0 from the prologue.
> +		 */
> +		for (i = prog->len - 1; i > 0; i--)
> +			ctx->offset[i] = ctx->offset[i - 1];
> +		ctx->offset[0] = ninsns_rvoff(ctx->prologue_len);
> +		bpf_prog_update_insn_ptrs(prog, ctx->offset, jit_data->ro_image);
>   out_offset:
>   		kvfree(ctx->offset);
>   		kfree(jit_data);

Reviewed-by: Pu Lehui <pulehui@huawei.com>

[PATCH bpf-next v2 2/2] selftests/bpf: Enable gotox tests for riscv64

[Chen Pei]

The riscv64 JIT now supports the gotox instruction and jump tables, so
run the tests in verifier_gotox.c on riscv64 too, mirroring what was done
for arm64 and powerpc.

The guard is 64-bit only because the RV32 JIT does not implement gotox.

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
---

Changes since v1:
- Collapse the multi-line #endif marker to a single line.

 tools/testing/selftests/bpf/progs/verifier_gotox.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/verifier_gotox.c b/tools/testing/selftests/bpf/progs/verifier_gotox.c
index f5a9878c7b8d..96a9d914930e 100644
--- a/tools/testing/selftests/bpf/progs/verifier_gotox.c
+++ b/tools/testing/selftests/bpf/progs/verifier_gotox.c
@@ -6,7 +6,9 @@
 #include "bpf_misc.h"
 #include "../../../include/linux/filter.h"
 
-#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || defined(__TARGET_ARCH_powerpc)
+#if defined(__TARGET_ARCH_x86) || defined(__TARGET_ARCH_arm64) || \
+	defined(__TARGET_ARCH_powerpc) || \
+	(defined(__TARGET_ARCH_riscv) && __riscv_xlen == 64)
 
 #define DEFINE_SIMPLE_JUMP_TABLE_PROG(NAME, SRC_REG, OFF, IMM, OUTCOME)	\
 									\
@@ -580,6 +582,6 @@ nop_%=:								\
 	: __clobber_all);
 }
 
-#endif /* __TARGET_ARCH_x86 || __TARGET_ARCH_arm64 || __TARGET_ARCH_powerpc*/
+#endif /* gotox: x86, arm64, powerpc, riscv64 */
 
 char _license[] SEC("license") = "GPL";
-- 
2.50.1