From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90156EB64DA for ; Wed, 19 Jul 2023 15:18:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231560AbjGSPS5 (ORCPT ); Wed, 19 Jul 2023 11:18:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230057AbjGSPSy (ORCPT ); Wed, 19 Jul 2023 11:18:54 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E57819B9 for ; Wed, 19 Jul 2023 08:18:30 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id AF60561757 for ; Wed, 19 Jul 2023 15:18:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86C11C433C8; Wed, 19 Jul 2023 15:18:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689779890; bh=yRNGM7k1m44UGiSOAxGAbNASoCiFYC+UXo3gbDSSuDI=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=QwLsB+j6V0YIEGRkP2tox2mqqdk3mE2QCURMU/CVETBBJwN3oyIYtmxOUetSAkF40 /RS+ApBqn8DYnYau+OosPfhhGaQMbCAqoV5muR0AMrG9m+67v0S5YqjDvIP571n7Is S92IT+ZnwZqdluFH+xPI4Wr2hvcqWHvhKgaiZUKny/P+2/eLfZk1Mzh8CRyxAK7EYT fyLyR+rYYBa/ZAa1yU3cVuD70VfIUN5JeWYJbpaThX0BbafGKTmiTbyEf+MqjwwtiR k0moGCANuw6dLL87/X8pt6mSGmBYUtp11ymCqAwOr2KXeND2V4L25mEuAKblIZUGeY a9IrYQfBI7DsQ== From: =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= To: Pu Lehui , Pu Lehui , bpf@vger.kernel.org, linux-riscv@lists.infradead.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Palmer Dabbelt , Guo Ren , Song Shuai Subject: Re: [PATCH bpf] riscv, bpf: Adapt bpf trampoline to optimized riscv ftrace framework In-Reply-To: <63986ef9-10a4-bcef-369d-0bad28b192d1@huawei.com> References: <20230715090137.2141358-1-pulehui@huaweicloud.com> <87lefdougi.fsf@all.your.base.are.belong.to.us> <63986ef9-10a4-bcef-369d-0bad28b192d1@huawei.com> Date: Wed, 19 Jul 2023 17:18:08 +0200 Message-ID: <87o7k8udzj.fsf@all.your.base.are.belong.to.us> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Pu Lehui writes: > On 2023/7/19 4:06, Bj=C3=B6rn T=C3=B6pel wrote: >> Pu Lehui writes: >>=20 >>> From: Pu Lehui >>> >>> Commit 6724a76cff85 ("riscv: ftrace: Reduce the detour code size to >>> half") optimizes the detour code size of kernel functions to half with >>> T0 register and the upcoming DYNAMIC_FTRACE_WITH_DIRECT_CALLS of riscv >>> is based on this optimization, we need to adapt riscv bpf trampoline >>> based on this. One thing to do is to reduce detour code size of bpf >>> programs, and the second is to deal with the return address after the >>> execution of bpf trampoline. Meanwhile, add more comments and rename >>> some variables to make more sense. The related tests have passed. >>> >>> This adaptation needs to be merged before the upcoming >>> DYNAMIC_FTRACE_WITH_DIRECT_CALLS of riscv, otherwise it will crash due >>> to a mismatch in the return address. So we target this modification to >>> bpf tree and add fixes tag for locating. >>=20 >> Thank you for working on this! >>=20 >>> Fixes: 6724a76cff85 ("riscv: ftrace: Reduce the detour code size to hal= f") >>=20 >> This is not a fix. Nothing is broken. Only that this patch much come >> before or as part of the ftrace series. > > Yep, it's really not a fix. I have no idea whether this patch target to=20 > bpf-next tree can be ahead of the ftrace series of riscv tree? For this patch, I'd say it's easier to take it via the RISC-V tree, IFF the ftrace series is in for-next. [...] >>> +#define DETOUR_NINSNS 2 >>=20 >> Better name? Maybe call this patchable function entry something? Also, > > How about RV_FENTRY_NINSNS? Sure. And more importantly that it's actually used in the places where nops/skips are done. >> to catch future breaks like this -- would it make sense to have a >> static_assert() combined with something tied to >> -fpatchable-function-entry=3D from arch/riscv/Makefile? > > It is very necessary, but it doesn't seem to be easy. I try to find GCC=20 > related functions, something like __builtin_xxx, but I can't find it so=20 > far. Also try to make it as a CONFIG_PATCHABLE_FUNCTION_ENTRY=3D4 in=20 > Makefile and then static_assert, but obviously it shouldn't be done.=20 > Maybe we can deal with this later when we have a solution? Ok! [...] >>> @@ -787,20 +762,19 @@ static int __arch_prepare_bpf_trampoline(struct b= pf_tramp_image *im, >>> int i, ret, offset; >>> int *branches_off =3D NULL; >>> int stack_size =3D 0, nregs =3D m->nr_args; >>> - int retaddr_off, fp_off, retval_off, args_off; >>> - int nregs_off, ip_off, run_ctx_off, sreg_off; >>> + int fp_off, retval_off, args_off, nregs_off, ip_off, run_ctx_off, sre= g_off; >>> struct bpf_tramp_links *fentry =3D &tlinks[BPF_TRAMP_FENTRY]; >>> struct bpf_tramp_links *fexit =3D &tlinks[BPF_TRAMP_FEXIT]; >>> struct bpf_tramp_links *fmod_ret =3D &tlinks[BPF_TRAMP_MODIFY_RETURN= ]; >>> void *orig_call =3D func_addr; >>> - bool save_ret; >>> + bool save_retval, traced_ret; >>> u32 insn; >>>=20=20=20 >>> /* Generated trampoline stack layout: >>> * >>> * FP - 8 [ RA of parent func ] return address of parent >>> * function >>> - * FP - retaddr_off [ RA of traced func ] return address of traced >>> + * FP - 16 [ RA of traced func ] return address of >>> traced >>=20 >> BPF code uses frame pointers. Shouldn't the trampoline frame look like a >> regular frame [1], i.e. start with return address followed by previous >> frame pointer? >>=20 > > oops, will fix it. Also we need to consider two types of trampoline=20 > stack layout, that is: > > * 1. trampoline called from function entry > * -------------------------------------- > * FP + 8 [ RA of parent func ] return address of parent > * function > * FP + 0 [ FP ] > * > * FP - 8 [ RA of traced func ] return address of traced > * function > * FP - 16 [ FP ] > * -------------------------------------- > * > * 2. trampoline called directly > * -------------------------------------- > * FP - 8 [ RA of caller func ] return address of caller > * function > * FP - 16 [ FP ] > * -------------------------------------- Hmm, could you expand a bit on this? The stack frame top 16B (8+8) should follow what the psabi suggests, regardless of the call site? Maybe it's me that's not following -- please explain a bit more! Bj=C3=B6rn