From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0A4B1DED40 for ; Tue, 14 Jul 2026 00:26:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783988814; cv=none; b=PgFhiyrF8/4c8GJIMOhYGoBZksru3JpIqboA2aPQsD4+Pclrinkp++t874w1BMlrvLfR0sG2DXwxP3Pj0JK3niHrj0MAdwIyPTAhAZfAlypW38j4ossS+YamOfPvZx4tn+dmp8RQ78+W23lqzv4KPIDocxVWFYHIv/UiFJuVhLI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783988814; c=relaxed/simple; bh=kqjs8i5Xs6LCh3vScraC6SOCCs8ALahddMkZsW0/mCU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ic/u31rIdf6ZgQ4WFzpjNEmgehD3upwWNt2zm2+VMir8u+WizgmbfnxtFoulKfqyTiLJ+Ku6vZmXXa7NRGSzXcYzZvCA22wBSCbj2z2rnoQzTIDJG1oU8LeC1ncuuhKpfzYhK5tgyGu7ouVgPiMaW48ejWbkStTKery8cMsV7hw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AAwzevdg; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AAwzevdg" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2caced6038eso5608535ad.0 for ; Mon, 13 Jul 2026 17:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1783988812; x=1784593612; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=KgXQYuVybZT9KNEa8c+0sS3pMAKReFnk7Wi8Pjluu4g=; b=AAwzevdgRIqEwGeiPVX1n59jZ6SShppEH2SUllMKYMEWOyviOjVNELRt9Djc6BGAkV vG0my49tdqahsmnu20/qKh8Giy2kBLt+gftga6dcoozVeupGa4kWtyusqSfg7qaQmEci JgYKKVnhmZNpTzrlrmQEZQVmlOLEcGljACMMzYdlTYCfF4W+xVhR/vfyJfUL0ZX5hHur iLgR507XBLiFnNNkbz7kBGgMwkz7895J5R7OTyHE11MocQIktrcb3DLmUEzSzRwQCcmn Uh/nZa5mzZYreFqNnoqqonF7emn8u1FRSLRzpiYjZWg+tvtq8wc2AngDoFybpmsDKEEk Krdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783988812; x=1784593612; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=KgXQYuVybZT9KNEa8c+0sS3pMAKReFnk7Wi8Pjluu4g=; b=Zv+N3GvqC71Br92wj2y8gkJIQEZcGK6Oy/gQh9PuLpdg1ZkfAHEI5KXW2hldtW0qMs n6MamXzeja6xmXu05nyWYxQI3OULyVrn7yEyVI+sb8WNlNwfe2jag0lFzGXfL7QKqNma QsRMydHs6f/3U5Y/0c6tJaWaFgSN/QKY4DXNl4dAC0obx7l0plmt9WedrJAV3eB9clnb uB8IedClMC6J6E91ljbz5BcZ0jl3U50WcIkSBFqK5BQXQVMgCnnqgedBeI1qNptrChW5 JIGaPLrjiaTggAXh2zBMKkQk8da64pnkFtXffEejdcjP7ZKufMbDYB0kytQFpk86ltKF 06rA== X-Forwarded-Encrypted: i=1; AHgh+Ro3sw9y7GSQN5QemNiVcAlWzjvrxOpIxizImX9xPcltQnFr3PMPNoXuEYhTIDmoh5gogKV7ioyZ9epmXtY=@vger.kernel.org X-Gm-Message-State: AOJu0YxWBCvSfQBsl52Lct/gMSPl6djPe3vDWwHhDlJ4IhPYUlfy5HRV wXPsAL3tKwVYy4U4Zu/e0fR/H9V+06zykDsGDbHY+2enou6eIgQK80E+ X-Gm-Gg: AfdE7ckBxjPoynQaVBZFb2x/bIHJckCJjqRO6/ij+jnaz8Nl//u+HsSkfG9vqR4/vLm 45xT3jxAD2F9wXVwqXvXr+lNum/xXeLEr+SNtmmdNJZ3a+e/NBwVoOGX5tGxiCVrlW4bhcAqPIY A9aCv4ZqvzakndrPs0KIp0iMhpWNTGcnS06Ttdqx9kIIUUmYmd9OVJNe/D9YV4YrZnFAIbxfRry ehGlJsxbf4KSIBczzAWF6AFmtjYb5zyk3oJ4QEVj5MqTynua0OfI35Wbqa3vfcaF74vrttMyjVs UJOoo3UKH0/liTIIYnnF0ZnjsIm0sE+rvWHx68JaRHia1PNko1HFFEnhbAGrq1M931Tugpymz4G 4mreme1kfk80U64+Cdb9wBAfsa0oRUSSsIiHy5JdjLo57zaaRcoWYD2S34t6usdD/JMLAPB/x4Q xOfEyrSRDqVlG5BWkRrVQXo9H4j/2D/Vv3iqcdEvUtdjsA6ZvprtEJu3umPgT19kE36jzLSykdA 14i1giKRZNeDB8= X-Received: by 2002:a17:902:d511:b0:2ca:d4f:adab with SMTP id d9443c01a7336-2ce8292cb2cmr149647585ad.19.1783988812034; Mon, 13 Jul 2026 17:26:52 -0700 (PDT) Received: from visitorckw-work01.c.googlers.com.com (131.197.81.34.bc.googleusercontent.com. [34.81.197.131]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2ccc9d1edb0sm105796545ad.53.2026.07.13.17.26.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2026 17:26:51 -0700 (PDT) From: Kuan-Wei Chiu To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, eddyz87@gmail.com, memxor@gmail.com, luke.r.nels@gmail.com, xi.wang@gmail.com, pjw@kernel.org, palmer@dabbelt.com, aou@eecs.berkeley.edu, pulehui@huawei.com Cc: martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, jolsa@kernel.org, emil@etsalapatis.com, alex@ghiti.fr, jserv@ccns.ncku.edu.tw, eleanor15x@gmail.com, marscheng@google.com, bpf@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Kuan-Wei Chiu Subject: [PATCH bpf-next v4 3/3] riscv, bpf: Add 32 bit atomic operations to RV32 JIT Date: Tue, 14 Jul 2026 00:24:51 +0000 Message-ID: <20260714002451.4091139-4-visitorckw@gmail.com> X-Mailer: git-send-email 2.55.0.795.g602f6c329a-goog In-Reply-To: <20260714002451.4091139-1-visitorckw@gmail.com> References: <20260714002451.4091139-1-visitorckw@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The RV32 BPF JIT compiler currently only supports the BPF_ADD atomic operation. Other 32 bit atomic operations (and, or, xor, xchg) and their BPF_FETCH variants are not supported and gracefully fall back to the interpreter. Since the RISC-V A extension is required for Linux on RV32, we can natively support these 32-bit BPF atomic operations by mapping them directly to the corresponding RISC-V amo*.w instructions. Implement BPF_ADD, BPF_AND, BPF_OR, BPF_XOR, and BPF_XCHG with and without BPF_FETCH. BPF_CMPXCHG requires a more complex lr.w/sc.w loop and is left to fall back to the interpreter. Before this patch: [ 138.862161] test_bpf: Summary: 1054 PASSED, 0 FAILED, [843/1042 JIT'ed] After this patch: [ 157.024124] test_bpf: Summary: 1054 PASSED, 0 FAILED, [902/1042 JIT'ed] Signed-off-by: Kuan-Wei Chiu --- arch/riscv/net/bpf_jit_comp32.c | 64 +++++++++++++++++++++++++++------ 1 file changed, 53 insertions(+), 11 deletions(-) diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c index 39e2b0b907dc..a9e0bd5cc81d 100644 --- a/arch/riscv/net/bpf_jit_comp32.c +++ b/arch/riscv/net/bpf_jit_comp32.c @@ -874,14 +874,58 @@ static int emit_load_r64(const s8 *dst, const s8 *src, s16 off, return 0; } -static int emit_store_r64(const s8 *dst, const s8 *src, s16 off, - struct rv_jit_context *ctx, const u8 size, - const u8 mode) +static int emit_bpf_atomic(s8 dst, const s8 *src, const s8 *rs, + struct rv_jit_context *ctx, + const struct bpf_insn *insn) +{ + s32 imm = insn->imm; + bool is_fetch = (imm & BPF_FETCH) || (imm == BPF_XCHG); + s8 fetch_reg = is_fetch ? lo(rs) : RV_REG_ZERO; + int aq = is_fetch ? 1 : 0; + int rl = is_fetch ? 1 : 0; + + switch (imm) { + case BPF_ADD: + case BPF_ADD | BPF_FETCH: + emit(rv_amoadd_w(fetch_reg, lo(rs), dst, aq, rl), ctx); + break; + case BPF_AND: + case BPF_AND | BPF_FETCH: + emit(rv_amoand_w(fetch_reg, lo(rs), dst, aq, rl), ctx); + break; + case BPF_OR: + case BPF_OR | BPF_FETCH: + emit(rv_amoor_w(fetch_reg, lo(rs), dst, aq, rl), ctx); + break; + case BPF_XOR: + case BPF_XOR | BPF_FETCH: + emit(rv_amoxor_w(fetch_reg, lo(rs), dst, aq, rl), ctx); + break; + case BPF_XCHG: + emit(rv_amoswap_w(fetch_reg, lo(rs), dst, aq, rl), ctx); + break; + default: + return -1; + } + + if (is_fetch) { + emit(rv_addi(hi(rs), RV_REG_ZERO, 0), ctx); + bpf_put_reg64(src, rs, ctx); + } + return 0; +} + +static int emit_store_r64(const s8 *dst, const s8 *src, + struct rv_jit_context *ctx, + const struct bpf_insn *insn) { const s8 *tmp1 = bpf2rv32[TMP_REG_1]; const s8 *tmp2 = bpf2rv32[TMP_REG_2]; const s8 *rd = bpf_get_reg64(dst, tmp1, ctx); const s8 *rs = bpf_get_reg64(src, tmp2, ctx); + u8 size = BPF_SIZE(insn->code); + u8 mode = BPF_MODE(insn->code); + s16 off = insn->off; if (mode == BPF_ATOMIC && size != BPF_W) return -1; @@ -901,9 +945,9 @@ static int emit_store_r64(const s8 *dst, const s8 *src, s16 off, case BPF_MEM: emit(rv_sw(RV_REG_T0, 0, lo(rs)), ctx); break; - case BPF_ATOMIC: /* Only BPF_ADD supported */ - emit(rv_amoadd_w(RV_REG_ZERO, lo(rs), RV_REG_T0, 0, 0), - ctx); + case BPF_ATOMIC: + if (emit_bpf_atomic(RV_REG_T0, src, rs, ctx, insn)) + return -1; break; } break; @@ -1303,21 +1347,19 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx, src = tmp2; } - if (emit_store_r64(dst, src, off, ctx, BPF_SIZE(code), - BPF_MODE(code))) + if (emit_store_r64(dst, src, ctx, insn)) return -1; break; case BPF_STX | BPF_ATOMIC | BPF_W: - if (insn->imm != BPF_ADD) { + if (insn->imm == BPF_CMPXCHG) { pr_info_once( "bpf-jit: not supported: atomic operation %02x ***\n", insn->imm); return -EFAULT; } - if (emit_store_r64(dst, src, off, ctx, BPF_SIZE(code), - BPF_MODE(code))) + if (emit_store_r64(dst, src, ctx, insn)) return -1; break; -- 2.55.0.795.g602f6c329a-goog