From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12E1633A9E1 for ; Fri, 11 Sep 2026 14:10:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789135816; cv=none; b=J4cE8DRPxGj1QvhGIWkZos8sQJcc4FT6DyrG0gYkVAihMr6t5unMWFnW2ElNL38cfEGUdebBq/eJkyo6xOIKYZ43v+94ECgdoY4o2Kk7gSkHsBYJnXv0khXua2TPI9g0Yxrb8zIjyoc7dvms76Emr3Dh7paNni+rW0Pzqs2LobY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789135816; c=relaxed/simple; bh=xqTPn5S4ER1XAZhtIYfJFNG5V6jdP8hXJAwXIfHAJeM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=o31vGFY9rs7ZKEHTvHopqBncU9rirO7hsDX9Gt30wKDKv0K7valVXwBLW2/Xf5OdSLsaKVLUvVDiobCn7tjDwOk855bBsdGjR803/bnBRPTYrwC2KeGyBrG31hR4DzEjrY9bw7Fk5PEuM67pwxfvH6IPvE9HezbJmaAMa3/HwhM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=toxicpanda.com; spf=pass smtp.mailfrom=toxicpanda.com; dkim=pass (2048-bit key) header.d=toxicpanda.com header.i=@toxicpanda.com header.b=RL7TN91v; arc=none smtp.client-ip=209.85.222.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=toxicpanda.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=toxicpanda.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda.com header.i=@toxicpanda.com header.b="RL7TN91v" Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-9399daa3c8eso76786985a.1 for ; Fri, 11 Sep 2026 07:10:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda.com; s=google; t=1789135813; x=1789740613; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=q0Z6kl89L/mcJKSDFUzFWtYRNb/tGGuag8yv3GYWCr4=; b=RL7TN91v33ejtXycFyxg3fUnBjQf8abI+hin2UNGZI2wBeWwf49htfb9Aj8vUw2Muq k9u8JLOpfzYnL+lg0qTqWffW0ltVG7wjneIpkZseOnt26NGyUT2jtGsu3wJxAwhF9TfT Q7tUWoFJCzyfPptfXsbE6n8fgKUbf9pfjLsavhbujWDYGXt48pF1kZryFdJgRiRIX4ge pIeqsVTdFT92F/zIFJZMB18PnPCOKsOH1fvdO3mIZVGLVrTrMhmrC6bHOTAOhAsTrZJJ W95Cqb6b3057kaKanyW0Z8xbp2flwvBJ4Aa1e2hOY5E02BLr7W9PPTVAWk0kBW/CyAoq rIDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789135813; x=1789740613; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=q0Z6kl89L/mcJKSDFUzFWtYRNb/tGGuag8yv3GYWCr4=; b=L7/PdEgQoCXtFo4pyw0xdFVjKNH7rlwejO7Z445bnJji0QsAO+PGzPtoVnBgjbItG3 RS/94ubGsfhX7JpNJSWdW3ZFSLA8mprxhXIhErAK3ynlCPXJvS6F0eJHBvbsKSSLf7zm XF767X/ZRLPh2pi47cNP2BPX+ULLWyvpUteqDGKWbMIjmPUGFm7CIzsU7bn1iUtmfU34 k6HU5ebqTCexK0qV0tE9qN7+hzdWx6BF2Qevvky2+0xLWLiC0igYoGoK7eijfBd9Tx6T JwSwsNtVbYmVObCcyfyEIjOw0oXnk06A6QMdy9tgoYmA5IHaR917bC/csnAilCJwfuwN iPxA== X-Forwarded-Encrypted: i=1; AKwUvByMaPVmYRIGLA5pFVVspFtu+5dT21imPpYzTjcW16ImqxX5XQRBWFKgW2G33tfqw3/jHjeqabxRHTaM3ck=@vger.kernel.org X-Gm-Message-State: AFuF++lEQxWtqrvyBlUlAFh8/fArKnGSzT9murzRAnlK/Ks6sjsSt+Q1 M5gfnoVEqKzOaknFLLRuRTRCzadGn7DaNALTgfJp4I9hLk0YHUB5ftfpgewIYBVvpEY= X-Gm-Gg: AYBFou35AxBenBsVdUZR8Fx2IQjm8MAYaLTETeAxTa5dRAHEeH6FKuKruMiQUT7wzYf TehZG3Vb/VsZB5OroJ3bpLTidawCfX6JLwoM+w6AupWCv9e/DMOiEQRiJ8rmMFy5tLZmpy6hpAG 9VuiUZpg4EA5f25O3M/JN8V3tFrND9tM9i27W+PRKdJbuVJbwafZpPjgJb4Nvojzf0GomTAxWt4 xD1pUud7krJ/sbN47oum2rV2Ad39kATr0TPtoeY+gkjC7bCBaBp9LZSrSqE/9mIIeyBIB6apmqt l0o/6gmp+njNGBmBcJuBYunMNa5gPVyHPGMlfCcHL+zMxiYDkP447Zr8SVQKoqvGBkHoOAHft8i 9kEnxCvWevKsmy8vou+0Xr/Zu/hFZPI1Ua+4KHSsSwNK6U9TYs1OyGeZUMzHjibclr/sOqR0zFI oQankuHO5+0b1uWvK03o+geQNUArwHiHXkBvIMbJdX/qWY37ycURoOeiGpjt/ZP+51X6xWAVPQC C7y6M62unw807sVxeOm89Q0KjDipANfuSPXc7+DoqZsR7vr839aIb4s X-Received: by 2002:a05:620a:6287:b0:939:6dea:374b with SMTP id af79cd13be357-939ea1af385mr553869585a.47.1789135812492; Fri, 11 Sep 2026 07:10:12 -0700 (PDT) Received: from toxicpanda.com (ec2-34-228-114-98.compute-1.amazonaws.com. [34.228.114.98]) by smtp.gmail.com with ESMTPSA id af79cd13be357-939e80bbab7sm247620785a.34.2026.09.11.07.10.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Sep 2026 07:10:10 -0700 (PDT) From: Josef Bacik Date: Fri, 11 Sep 2026 14:08:47 +0000 Subject: [PATCH RFC v2 09/15] arm64: ftrace: Maintain Tasks RCU trampoline nesting in ftrace_caller Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260911-b4-rcu-tasks-preempt-qs-v2-9-eaaa61ed2da4@toxicpanda.com> References: <20260911-b4-rcu-tasks-preempt-qs-v2-0-eaaa61ed2da4@toxicpanda.com> In-Reply-To: <20260911-b4-rcu-tasks-preempt-qs-v2-0-eaaa61ed2da4@toxicpanda.com> To: "Paul E. McKenney" , Frederic Weisbecker , Neeraj Upadhyay , Joel Fernandes , Boqun Feng , Thomas Gleixner , Peter Zijlstra , Steven Rostedt , Masami Hiramatsu , Mark Rutland , Jiri Olsa , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , x86@kernel.org, Catalin Marinas , Will Deacon , Puranjay Mohan , Xu Kuohai Cc: Andy Lutomirski , Josh Triplett , Uladzislau Rezki , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Juergen Gross , Luis Chamberlain , Ihor Solodrai , linux-kernel@vger.kernel.org, rcu@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, xen-devel@lists.xenproject.org, Josef Bacik X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openssh-sha256; t=1789135736; l=5217; i=josef@toxicpanda.com; h=from:subject:message-id; bh=xqTPn5S4ER1XAZhtIYfJFNG5V6jdP8hXJAwXIfHAJeM=; b=U1NIU0lHAAAAAQAAADMAAAALc3NoLWVkMjU1MTkAAAAgUBr36M/n0nWN0DNbnxwzIiCZez6MG JiruuNaSCI/zXsAAAAGcGF0YXR0AAAAAAAAAAZzaGE1MTIAAABTAAAAC3NzaC1lZDI1NTE5AAAA QDdDEdIwM05h7b7JvpgB0Ncpvko1HStiAd9VzLg7h8HjWsQnjthe7VWx1bLRiDUbfVMTbRgUiKe YsgeGrk28wQI= X-Developer-Key: i=josef@toxicpanda.com; a=openssh; fpr=SHA256:C8kOX2QUJCMqnCX+KEeoqRAjLo9L+ELOSH2NSAJHqGA Bracket the call out to the ftrace_ops callback in ftrace_caller with an increment/decrement of current->rcu_tramp_nesting, using x12/w13 which are scratch there. The read-modify-write is not atomic, but only current modifies the count and every interrupting user is balanced, so nothing is lost. ftrace_caller itself, including the early CALL_OPS direct path and the late direct tail that carry a BPF trampoline address in x17 with the count at zero, is static kernel text: add an ftrace_static_tramp_end marker after ftrace_stub_direct_tramp and provide arch_rcu_tasks_ip_in_trampoline() covering [ftrace_caller, ftrace_static_tramp_end) so the irq-exit check treats a task interrupted anywhere in it as inside a trampoline. The hook is built only under CONFIG_RCU_TASKS_PREEMPT_QS, which arm64 does not select until a later patch. Assisted-by: LLM Signed-off-by: Josef Bacik --- arch/arm64/kernel/asm-offsets.c | 3 +++ arch/arm64/kernel/entry-ftrace.S | 35 +++++++++++++++++++++++++++++++++++ arch/arm64/kernel/ftrace.c | 16 ++++++++++++++++ 3 files changed, 54 insertions(+) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 9c853ed3ceab..f6655a284f18 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -39,6 +39,9 @@ int main(void) DEFINE(TSK_STACK, offsetof(struct task_struct, stack)); #ifdef CONFIG_STACKPROTECTOR DEFINE(TSK_STACK_CANARY, offsetof(struct task_struct, stack_canary)); +#endif +#ifdef CONFIG_TASKS_RCU + DEFINE(TSK_RCU_TRAMP_NESTING, offsetof(struct task_struct, rcu_tramp_nesting)); #endif BLANK(); DEFINE(THREAD_CPU_CONTEXT, offsetof(struct task_struct, thread.cpu_context)); diff --git a/arch/arm64/kernel/entry-ftrace.S b/arch/arm64/kernel/entry-ftrace.S index 025140caafe7..46a102e7199a 100644 --- a/arch/arm64/kernel/entry-ftrace.S +++ b/arch/arm64/kernel/entry-ftrace.S @@ -14,6 +14,33 @@ #include #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS +/* + * Tasks RCU trampoline nesting, see rcu_tasks_trampoline_enter(). The whole + * of ftrace_caller is treated as trampoline text by the irq-exit IP check (see + * arch_rcu_tasks_ip_in_trampoline()), so these only need to bracket the call + * out to ops->func; everything before the increment and after the decrement, + * including the direct-call tails that carry a BPF trampoline address in x17, + * is covered by that. The count is only modified by current and every nested + * user (interrupts) is balanced, so a plain ldr/add/str is sufficient. + */ + .macro rcu_tasks_tramp_enter, tsk:req, tmp:req +#ifdef CONFIG_TASKS_RCU + mrs \tsk, sp_el0 + ldr \tmp, [\tsk, #TSK_RCU_TRAMP_NESTING] + add \tmp, \tmp, #1 + str \tmp, [\tsk, #TSK_RCU_TRAMP_NESTING] +#endif + .endm + + .macro rcu_tasks_tramp_exit, tsk:req, tmp:req +#ifdef CONFIG_TASKS_RCU + mrs \tsk, sp_el0 + ldr \tmp, [\tsk, #TSK_RCU_TRAMP_NESTING] + sub \tmp, \tmp, #1 + str \tmp, [\tsk, #TSK_RCU_TRAMP_NESTING] +#endif + .endm + /* * Due to -fpatchable-function-entry=2, the compiler has placed two NOPs before * the regular function prologue. For an enabled callsite, ftrace_init_nop() and @@ -94,6 +121,8 @@ SYM_CODE_START(ftrace_caller) stp x29, x30, [sp, #FREGS_SIZE] add x29, sp, #FREGS_SIZE + rcu_tasks_tramp_enter x12, w13 + /* Prepare arguments for the tracer func */ sub x0, x30, #AARCH64_INSN_SIZE // ip (callsite's BL insn) mov x1, x9 // parent_ip (callsite's LR) @@ -111,6 +140,8 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) bl ftrace_stub // func(ip, parent_ip, op, regs) #endif + rcu_tasks_tramp_exit x12, w13 + /* * At the callsite x0-x8 and x19-x30 were live. Any C code will have preserved * x19-x29 per the AAPCS, and we created frame records upon entry, so we need @@ -178,6 +209,10 @@ SYM_CODE_START(ftrace_stub_direct_tramp) SYM_CODE_END(ftrace_stub_direct_tramp) #endif /* CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS */ +/* End of [ftrace_caller, ...) for arch_rcu_tasks_ip_in_trampoline(). */ +SYM_CODE_START(ftrace_static_tramp_end) +SYM_CODE_END(ftrace_static_tramp_end) + #else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ /* diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index e1a3c0b3a051..1b7ac2afed0d 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -17,6 +17,22 @@ #include #include +#ifdef CONFIG_RCU_TASKS_PREEMPT_QS +extern void ftrace_static_tramp_end(void); + +/* + * See rcu_tasks_ip_in_trampoline(). ftrace_caller and ftrace_stub_direct_tramp + * are core kernel text but must be treated as trampolines: a task preempted in + * them may be carrying an ops pointer (x11) or a direct-call BPF trampoline + * address (x17) whose lifetime is guarded only by Tasks RCU. + */ +bool arch_rcu_tasks_ip_in_trampoline(unsigned long ip) +{ + return ip >= (unsigned long)ftrace_caller && + ip < (unsigned long)ftrace_static_tramp_end; +} +#endif + #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS struct fregs_offset { const char *name; -- 2.55.0