[RFC PATCH] mm: restrict can_spin_trylock() to preemptible context on PREEMPT_RT

[RFC PATCH] mm: restrict can_spin_trylock() to preemptible context on PREEMPT_RT

[Karl Mehltretter]

On PREEMPT_RT kmalloc_nolock() deadlocks the machine when the caller
holds a pi_lock. A BPF program which creates task local storage from
the sched_waking tracepoint is enough. With it mainline hangs in 10 of
10 boots on a Raspberry Pi 500+ and in 7 of 28 boots in a 4 CPU x86-64
QEMU guest. The CPUs spin on raw locks with interrupts disabled and
there is no output.

sched_waking fires in try_to_wake_up() with p->pi_lock held.
can_spin_trylock() refuses only NMI and hard interrupt context on
PREEMPT_RT, so kmalloc_nolock() does spin_trylock_irqsave() on
n->list_lock, which is a sleeping lock. Two things go wrong:

 - rt_spin_trylock() takes the rtmutex wait_lock under the pi_lock. A
   regular kmalloc() on another CPU holds wait_lock and takes a
   pi_lock in try_to_take_rt_mutex().

 - If the trylock succeeded and a waiter arrived, rt_spin_unlock() has
   to wake it. That is a try_to_wake_up() inside try_to_wake_up():

     try_to_wake_up               raw_spin_lock_irqsave(&p->pi_lock)
     rt_mutex_slowunlock
     rt_spin_unlock
     spin_unlock_irqrestore(&n->list_lock)
     get_from_partial_node
     ___slab_alloc
     __kmalloc_nolock_noprof
     bpf_local_storage_alloc
     ...
     bpf_trace_run1
     try_to_wake_up               sched_waking, a pi_lock is held

   The pi_lock held there belongs to the woken task, not to current.

Commit 99a3e3a1cfc9 ("slab: fix kmalloc_nolock() context check for
PREEMPT_RT") closed this for v6.19 with a !preemptible() check.
v7.0-rc1 relaxed it again, see the Fixes tag.

Allow preemptible context only, as v6.19 did. A held raw spinlock
implies !preemptible(), so the locks of the caller do not have to be
known. With this change both machines pass 10 of 10 boots.

The nolock allocations then fail on PREEMPT_RT from every context with
preemption or interrupts disabled, also where no scheduler lock is
held. Creation of BPF local storage from such a context fails, as it
did in v6.19.

Fixes: 073d5f156292 ("slab: simplify kmalloc_nolock()")
Assisted-by: LLM
---

Notes:
    RFC because I do not know if that last paragraph is acceptable for
    BPF. 073d5f156292 relaxed the check on purpose.
    
    Harry wrote that the _nolock() helpers can be called under pi_lock at
    least in theory, that it was tough to reproduce, and proposed a check
    of current's pi_lock:
    https://lore.kernel.org/r/apfwshahN_C6jBSn@nixos
    https://lore.kernel.org/r/apfyeYF8IE03aazT@nixos
    That check would not see the pi_lock in the stack above. Sebastian
    described the wait_lock and pi_lock order here:
    https://lore.kernel.org/r/20260831143500.x-saxdAs@linutronix.de
    
    Same config, program and load for all rows. x86-64 QEMU (TCG), 4 CPUs,
    PREEMPT_RT, no lockdep, 60 seconds of load per boot:
    
      v6.18                                  10 pass,  0 hang
      parent of f484f4a3e058                 10 pass,  0 hang
      f484f4a3e058                            0 pass, 10 hang
      v6.19                                  10 pass,  0 hang
      parent of 073d5f156292                 10 pass,  0 hang
      073d5f156292                            0 pass, 10 hang
      mainline 40288c9206c1 (v7.3-rc3)       21 pass,  7 hang
      mainline with this change              10 pass,  0 hang
    
    f484f4a3e058 ("bpf: Replace bpf memory allocator with kmalloc_nolock()
    in local storage") made the path reachable.
    
    Raspberry Pi 500+, arm64 defconfig plus PREEMPT_RT, same mainline
    commit: the output stops 1 to 2 seconds after the load starts. 4 of the
    10 boots were then reset by the hardware watchdog. The other 6 made no
    more progress but still answered ping.
    
    The stacks are from gdb on the hung QEMU guests, I have none from the
    Pi. gdb does not unwind through the JITed program. The frames below it
    are from a raw dump of the stacks. In that capture three of four CPUs
    wait like this for the pi_lock of the same task. The first case looks
    like this, two CPUs from the BPF program against one regular kmalloc():
    
      rt_mutex_slowtrylock           raw_spin_lock(&lock->wait_lock)
      rt_spin_trylock
      spin_trylock_irqsave(&n->list_lock)
      get_from_partial_node
      ___slab_alloc
      __kmalloc_nolock_noprof
    
      try_to_take_rt_mutex           raw_spin_lock(&task->pi_lock)
      rtlock_slowlock_locked
      rtlock_slowlock
      rt_spin_lock
      spin_lock(&n->list_lock)
      get_from_partial_node
      ___slab_alloc
    
    With lockdep and DEBUG_RT_MUTEXES the same program gives "possible
    circular locking dependency detected" for wait_lock and pi_lock on the
    first event.
    
    The program:
    
      struct {
    	__uint(type, BPF_MAP_TYPE_TASK_STORAGE);
    	__uint(map_flags, BPF_F_NO_PREALLOC);
    	__type(key, int);
    	__type(value, long);
      } per_task SEC(".maps");
    
      SEC("tp_btf/sched_waking")
      int BPF_PROG(on_waking, struct task_struct *p)
      {
    	struct task_struct *cur = bpf_get_current_task_btf();
    	long *v;
    
    	v = bpf_task_storage_get(&per_task, cur, 0,
    				 BPF_LOCAL_STORAGE_GET_F_CREATE);
    	if (v)
    		bpf_task_storage_delete(&per_task, cur);
    	return 0;
      }
    
    The delete only makes every event allocate again. The load is a few
    loopback ping floods, tmpfs and block writes, /proc walks and fork/pipe
    loops, so that list_lock sees contention. The program is more
    aggressive than a real tool and the load is synthetic.
    
    can_spin_trylock() is also used by the page allocator. Its nolock free
    paths fall back to their deferred lists. I did not test those
    separately. Kernels before v7.3-rc1 have no can_spin_trylock() and need
    the check in kmalloc_nolock() instead.

 mm/internal.h | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index 38b1165212c94..29646c4afb419 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1641,10 +1641,14 @@ static inline bool can_spin_trylock(void)
 	 * confuse PI logic, so return immediately if called from hard IRQ or
 	 * NMI.
 	 *
-	 * Note, irqs_disabled() case is ok. spin_trylock() can be called
-	 * from raw_spin_lock_irqsave region.
+	 * Task context with a raw spinlock held is not safe either. The
+	 * caller may hold a pi_lock, like a BPF program on a tracepoint in
+	 * try_to_wake_up(). rt_spin_trylock() takes the rtmutex wait_lock
+	 * and can take a pi_lock under it. rt_spin_unlock() wakes a waiter
+	 * if there is one, which takes a pi_lock again. The locks held by
+	 * the caller are not known here, so allow preemptible context only.
 	 */
-	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
+	if (IS_ENABLED(CONFIG_PREEMPT_RT) && !preemptible())
 		return false;
 
 	/* On UP, spin_trylock() always succeeds even when it is locked */

base-commit: 40288c9206c17eb66a603262e06a58d300d0f279
-- 
2.53.0

Re: [RFC PATCH] mm: restrict can_spin_trylock() to preemptible context on PREEMPT_RT

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] mm: can_spin_trylock() uses !preemptible() causing persistent bailout
--

commit da2bda4aca43320be916d709a95e77705d7bd75f
Author: Karl Mehltretter <kmehltretter@gmail.com>

mm: restrict can_spin_trylock() to preemptible context on PREEMPT_RT

This patch resolves a deadlock on PREEMPT_RT during kmalloc_nolock() by
restricting can_spin_trylock() to preemptible contexts only. It prevents
nested lock acquisition of the pi_lock when called from sched_waking
tracepoints inside try_to_wake_up().

> diff --git a/mm/internal.h b/mm/internal.h
> --- a/mm/internal.h
> +++ b/mm/internal.h
[ ... ]
> @@ -1641,10 +1641,14 @@ static inline bool can_spin_trylock(void)
>  	 * confuse PI logic, so return immediately if called from hard IRQ or
>  	 * NMI.
>  	 *
> -	 * Note, irqs_disabled() case is ok. spin_trylock() can be called
> -	 * from raw_spin_lock_irqsave region.
> +	 * Task context with a raw spinlock held is not safe either. The
> +	 * caller may hold a pi_lock, like a BPF program on a tracepoint in
> +	 * try_to_wake_up(). rt_spin_trylock() takes the rtmutex wait_lock
> +	 * and can take a pi_lock under it. rt_spin_unlock() wakes a waiter
> +	 * if there is one, which takes a pi_lock again. The locks held by
> +	 * the caller are not known here, so allow preemptible context only.
>  	 */
> -	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi() || in_hardirq()))
> +	if (IS_ENABLED(CONFIG_PREEMPT_RT) && !preemptible())

[Severity: Medium]
Does this !preemptible() check create a persistent bailout for ALLOC_TRYLOCK
paths on PREEMPT_RT?

BPF programs attached to non-faultable tracepoints run with preemption
disabled via guard(preempt_notrace). When they attempt to create task local
storage using kmalloc_nolock(), it calls can_spin_trylock().

Because preemption is always disabled inside the tracepoint, this check
acts as a persistent bailout rather than a transient one, unconditionally
failing the allocation on PREEMPT_RT. Will this permanently break BPF task
storage creation in tracepoints on PREEMPT_RT?

>  		return false;
>  
>  	/* On UP, spin_trylock() always succeeds even when it is locked */

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260919171443.90512-1-kmehltretter@gmail.com?part=1

Re: [RFC PATCH] mm: restrict can_spin_trylock() to preemptible context on PREEMPT_RT

[Alexei Starovoitov]

On Sat, Sep 19, 2026 at 07:14 PM Karl Mehltretter <kmehltretter@gmail.com> wrote:
> Allow preemptible context only, as v6.19 did. A held raw spinlock
> implies !preemptible(), so the locks of the caller do not have to be
> known. With this change both machines pass 10 of 10 boots.
>
> The nolock allocations then fail on PREEMPT_RT from every context with
> preemption or interrupts disabled, also where no scheduler lock is
> held. Creation of BPF local storage from such a context fails, as it
> did in v6.19.

6.19 had this check in kmalloc_nolock() only. alloc_pages_nolock() and
free_pages_nolock() allowed irqs disabled since they were introduced,
and arena was sleepable only under a mutex back then.

[...]

> diff --git a/mm/internal.h b/mm/internal.h
> index 38b1165212c94..29646c4afb419 100644
> --- a/mm/internal.h
> +++ b/mm/internal.h
> @@ -1641,10 +1641,14 @@ static inline bool can_spin_trylock(void)

[...]

> -	if (IS_ENABLED(CONFIG_PREEMPT_RT) && (in_nmi()
 || in_hardirq()))
> +	if (IS_ENABLED(CONFIG_PREEMPT_RT) && !preemptible())
>  		return false;

No. This kills bpf arena on RT.
Since 7.0 arena_alloc_pages() and arena_vm_fault() take
raw_res_spin_lock_irqsave(&arena->spinlock) and call
alloc_pages_nolock() and kmalloc_nolock() (from range_tree) under it.
With !preemptible() here bpf_arena_alloc_pages() returns NULL for every
prog, sleepable included, and a user space fault in arena gets SIGSEGV.

As Sebastian said in
https://lore.kernel.org/r/20260831143500.x-saxdAs@linutronix.de
raw_spinlock_t is fine in general. pi_lock is special. rq lock too,
I think, since rt_spin_unlock() can end up in try_to_wake_up().
The check has to be about those and not about every irq/preempt
disabled section.

pw-bot: cr