From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75CB82A1C7; Mon, 22 Sep 2025 05:19:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758518351; cv=none; b=nBI/FgMYbr+I6ydhevfsaFvDYjRVWujqvc7rTLz/sP72Y8WILGb6ix7XEHT3v1YJQlnXZ97q9pQ90B3qpuZZpu/E/SKOqBTzaitIgXl9hxxl2YVHx1I0ngqRS8oaCS/Pb0YqrVhMMHE9y7czl1X1+cKJzeL/IHG7Df5e5HcXNkM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758518351; c=relaxed/simple; bh=s4I9Cmu/ZenfNo5dF9uaj7Nmkog1Gi3BFNvkkZ5A+qA=; h=Date:From:To:Cc:Subject:Message-Id:In-Reply-To:References: Mime-Version:Content-Type; b=Z3zJPOj/ILgT/z2AXcbH7H6edxyEETYXfL5r1ihdCBhiHKgDTRWrcn4FfRKzEGyX2pt7c8gE6AbgZ49XgsoywjGCsrFkHH4KaeR4wcXkPKq7zwxL5/Tv74cQrCgVO9RheOB9iRfpOD+nTEsh9IE3XbKLR+LDv2GZZt1Nh6XuCmY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b481WTcq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b481WTcq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7FCD6C4CEF0; Mon, 22 Sep 2025 05:19:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1758518350; bh=s4I9Cmu/ZenfNo5dF9uaj7Nmkog1Gi3BFNvkkZ5A+qA=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=b481WTcqyg75dl4MRy5qIoLlm4Q62dctkdPxkVmK+q4lcmHWn7hIu9Sk0RbtL4qTp LDS1kESBtpf9TGFH9bu+GMmjcEUIvWL6acqnGiZmXsejn/5jgy05YDuifSulLfB+o6 YGnipdAam/Be2s7fCKWuWVoil0APZASxS1mU32/FHnMA3/Rpsto1uSjYXOz4oSefYO mFJPqqlmyApivZnPimzpa/HKzql7wFh3+e8HN3vsgckLCaXYG3ZF1H9Ok0bXrZi3ZR 4ssJOrSyS6awpBdFi3MR6WVX8KDd9AkcV0KAdA3w4JZ7di78WwqBUc6txTLTI6/MGj vyiO/ufT1CQ+Q== Date: Mon, 22 Sep 2025 14:19:05 +0900 From: Masami Hiramatsu (Google) To: Menglong Dong Cc: Peter Zijlstra , Steven Rostedt , Menglong Dong , jolsa@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, kees@kernel.org, samitolvanen@google.com, rppt@kernel.org, luto@kernel.org, ast@kernel.org, andrii@kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH] tracing: fgraph: Protect return handler from recursion loop Message-Id: <20250922141905.f099ed89dfd5d7c740060a94@kernel.org> In-Reply-To: <5974303.DvuYhMxLoT@7950hx> References: <20250918120939.1706585-1-dongml2@chinatelecom.cn> <175828305637.117978.4183947592750468265.stgit@devnote2> <5974303.DvuYhMxLoT@7950hx> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Sat, 20 Sep 2025 21:39:25 +0800 Menglong Dong wrote: > On 2025/9/19 19:57, Masami Hiramatsu (Google) wrote: > > From: Masami Hiramatsu (Google) > > > > function_graph_enter_regs() prevents itself from recursion by > > ftrace_test_recursion_trylock(), but __ftrace_return_to_handler(), > > which is called at the exit, does not prevent such recursion. > > Therefore, while it can prevent recursive calls from > > fgraph_ops::entryfunc(), it is not able to prevent recursive calls > > to fgraph from fgraph_ops::retfunc(), resulting in a recursive loop. > > This can lead an unexpected recursion bug reported by Menglong. > > > > is_endbr() is called in __ftrace_return_to_handler -> fprobe_return > > -> kprobe_multi_link_exit_handler -> is_endbr. > > > > To fix this issue, acquire ftrace_test_recursion_trylock() in the > > __ftrace_return_to_handler() after unwind the shadow stack to mark > > this section must prevent recursive call of fgraph inside user-defined > > fgraph_ops::retfunc(). > > > > This is essentially a fix to commit 4346ba160409 ("fprobe: Rewrite > > fprobe on function-graph tracer"), because before that fgraph was > > only used from the function graph tracer. Fprobe allowed user to run > > any callbacks from fgraph after that commit. > > > > Reported-by: Menglong Dong > > Closes: https://lore.kernel.org/all/20250918120939.1706585-1-dongml2@chinatelecom.cn/ > > Fixes: 4346ba160409 ("fprobe: Rewrite fprobe on function-graph tracer") > > Cc: stable@vger.kernel.org > > Signed-off-by: Masami Hiramatsu (Google) > > --- > > kernel/trace/fgraph.c | 12 ++++++++++++ > > 1 file changed, 12 insertions(+) > > > > diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c > > index 1e3b32b1e82c..08dde420635b 100644 > > --- a/kernel/trace/fgraph.c > > +++ b/kernel/trace/fgraph.c > > @@ -815,6 +815,7 @@ __ftrace_return_to_handler(struct ftrace_regs *fregs, unsigned long frame_pointe > > unsigned long bitmap; > > unsigned long ret; > > int offset; > > + int bit; > > int i; > > > > ret_stack = ftrace_pop_return_trace(&trace, &ret, frame_pointer, &offset); > > @@ -829,6 +830,15 @@ __ftrace_return_to_handler(struct ftrace_regs *fregs, unsigned long frame_pointe > > if (fregs) > > ftrace_regs_set_instruction_pointer(fregs, ret); > > > > + bit = ftrace_test_recursion_trylock(trace.func, ret); > > + /* > > + * This must be succeeded because the entry handler returns before > > + * modifying the return address if it is nested. Anyway, we need to > > + * avoid calling user callbacks if it is nested. > > + */ > > + if (WARN_ON_ONCE(bit < 0)) > > + goto out; > > Hi, the logic seems right, but the warning is triggered when > I try to run the bpf bench testing: Ah, I remember that this ftrace_test_recursion_trylock() allows one-level nesting. OK, I will remove the WARN_ON. Thanks, > > $ ./benchs/run_bench_trigger.sh kretprobe-multi-all > [ 20.619642] NOTICE: Automounting of tracing to debugfs is deprecated and will be removed in 2030 > [ 139.509036] ------------[ cut here ]------------ > [ 139.509180] WARNING: CPU: 2 PID: 522 at kernel/trace/fgraph.c:839 ftrace_return_to_handler+0x2b9/0x2d0 > [ 139.509411] Modules linked in: virtio_net > [ 139.509514] CPU: 2 UID: 0 PID: 522 Comm: bench Not tainted 6.17.0-rc5-g1fe6d652bfa0 #106 NONE > [ 139.509720] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.17.0-1-1 04/01/2014 > [ 139.509948] RIP: 0010:ftrace_return_to_handler+0x2b9/0x2d0 > [ 139.510086] Code: e8 0c 08 0e 00 0f 0b 49 c7 c1 00 73 20 81 e9 d1 fe ff ff 40 f6 c6 10 75 11 49 c7 c3 ef ff ff ff ba 10 00 00 00 e9 57 fe ff ff <0f> 0b e9 a5 fe ff ff e8 1b 72 0d 01 66 66 2e 0f 1f 84 00 00 00 00 > [ 139.510536] RSP: 0018:ffffc9000012cef8 EFLAGS: 00010002 > [ 139.510664] RAX: ffff88810f709800 RBX: ffffc900007c3678 RCX: 0000000000000003 > [ 139.510835] RDX: 0000000000000008 RSI: 0000000000000018 RDI: 0000000000000000 > [ 139.511007] RBP: 0000000000000000 R08: 0000000000000034 R09: ffffffff82550319 > [ 139.511184] R10: ffffc9000012cf50 R11: fffffffffffffff7 R12: 0000000000000000 > [ 139.511357] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 > [ 139.511532] FS: 00007fe58276fb00(0000) GS:ffff8884ab3b8000(0000) knlGS:0000000000000000 > [ 139.511724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 139.511865] CR2: 0000562a28314b67 CR3: 00000001143f9000 CR4: 0000000000750ef0 > [ 139.512038] PKRU: 55555554 > [ 139.512106] Call Trace: > [ 139.512177] > [ 139.512232] ? irq_exit_rcu+0x4/0xb0 > [ 139.512322] return_to_handler+0x1e/0x50 > [ 139.512422] ? idle_cpu+0x9/0x50 > [ 139.512506] ? sysvec_apic_timer_interrupt+0x69/0x80 > [ 139.512638] ? idle_cpu+0x9/0x50 > [ 139.512731] ? irq_exit_rcu+0x3a/0xb0 > [ 139.512833] ? ftrace_stub_direct_tramp+0x10/0x10 > [ 139.512961] ? sysvec_apic_timer_interrupt+0x69/0x80 > [ 139.513101] > [ 139.513168] > > > + > > #ifdef CONFIG_FUNCTION_GRAPH_RETVAL > > trace.retval = ftrace_regs_get_return_value(fregs); > > #endif > > @@ -852,6 +862,8 @@ __ftrace_return_to_handler(struct ftrace_regs *fregs, unsigned long frame_pointe > > } > > } > > > > + ftrace_test_recursion_unlock(bit); > > +out: > > /* > > * The ftrace_graph_return() may still access the current > > * ret_stack structure, we need to make sure the update of > > > > > > > > > > -- Masami Hiramatsu (Google)