mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Andi Kleen <andi@firstfloor.org>
To: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
Cc: Pedro Falcato <pfalcato@suse.de>, Andi Kleen <ak@kernel.org>,
	akpm@linux-foundation.org, liam@infradead.org, jannh@google.com,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] RCU safety for vma maple tree walks
Date: Tue, 1 Sep 2026 14:05:27 -0700	[thread overview]
Message-ID: <apc-F_D5mWwWL74i@firstfloor.org> (raw)
In-Reply-To: <apXaTN6o5YaEO8FR@lucifer>

On Mon, Aug 31, 2026 at 08:55:37PM +0100, Lorenzo Stoakes (ARM) wrote:
> I mean what actually modifies the maple tree nodes here?
> 
> __split_vma()
>   -> vma_complete()
>     -> vma_iter_store_new()
>       -> vma_iter_store_overwrite()
> 
> Right?

The crash was on the walk, but yes the modification likely causes it.

> 
> But I see:
> 
> 	if (vmi->mas.status != ma_start &&
> 	    ((vmi->mas.index > vma->vm_start) || (vmi->mas.last < vma->vm_start)))
> 		vma_iter_invalidate(vmi);
> 
> Which calls mas_pause() which sets mas->node = NULL.
> 
> So I mean, presumably you are saying this doesn't work correctly or this
> criteria is wrong, I can't really see how else there could be a problem here,
> could you explain exactly what's up here?

I can't explain it currently, but yes something wrong with that logic
is a good theory.

I have a (somewhat garbled) processor trace log of the failure, but
it doesn't quite have enough information to untangle it completely.

The original bug also happened in a very memory constrained environment
(4GB guest), but it's somewhat hard to reproduce it in a setup
that still has enough memory to do useful debugging.

I'm currently working on the reproducer on the vanilla kernel.

So far I made some progress to get something that looks closer to the
original trace, but I need artificial sleeps at the vma allocation point
and it still didn't fully reproduce the original scenario yet. 

There was actually one crash with a different
signature, but I haven't analyzed it so far [1].

The WIP stresser that forces something that is close to the 
uprobes is here: 
https://firstfloor.org/~andi/madvise-dontfork-stress.c

> Also why is the solution to insert a whole bunch of RCU read locks everywhere so
> we can keep on accessing a node that we've already decided to free?

I may have an old school understanding of RCU, but I was always
thinking that the rcu read sections are needed for any readers with
preemption. The walker is clearly a reader.

But I guess in this particular case it's not true because
the writer lock and the invalidation is enough. It still seems a little
dubious with all the preemption cases, but at least I cannot see a clear
hole.

> 
> I mean surely the solution really ought to be simply invalidating the iterator
> right?

Yes I guess. It would certainly be simpler.


> 
> Also again, could you share the patch you've applied to the kernel you're
> actually seeing this bug in, given you haven't reproduced it even once with an
> upstream kernel?

Sure it's this patchkit on l-k that adds some new functionality to
uprobes: https://lore.kernel.org/lkml/20260831150651.1134594-1-ak@kernel.org/
(or for more information
https://lore.kernel.org/lkml/20260831150651.1134594-16-ak@kernel.org/ )

I don't think it actually changes any core VM locking or really how the uprobes
interact with the VM code, but it changes timing and makes it easier
to have some obscure DONTCOPY VMA setups. The actual VM code is not
changed.

The code that changes the core uprobes behavior is 5/19, but it can't really
be fully exercised without some of the later patches.

Also the posted version has some issues (at least Sashiko pointed out 
some real problems), but I don't believe it affects this.

-Andi


[1]


[   66.689807][  T532] BUG: unable to handle page fault for address: ffffebde00477288
[   66.691116][  T532] #PF: supervisor read access in kernel mode
[   66.692227][  T532] #PF: error_code(0x0000) - not-present page
[   66.693267][  T532] PGD 0 P4D 0 
[   66.693841][  T532] Oops: Oops: 0000 [#1] SMP KASAN NOPTI
[   66.694596][  T532] CPU: 1 UID: 0 PID: 532 Comm: madvise-dontfor Not tainted 7.2.0-1-debug+ #2 PREEMPT(full)  967bbfd33d6729a809cf8db6188ec03a118152c4
[   66.697960][  T532] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
[   66.700523][  T532] RIP: 0010:qlist_free_all+0x93/0x130
[   66.701296][  T532] Code: c2 4c 01 f2 0f 82 a3 00 00 00 48 c7 c1 00 00 00 80 48 2b 0d b7 c9 34 03 48 01 ca 48 c1 ea 0c 48 c1 e2 06 48 03 15 95 c9 34 03 <48> 8b 4a 08 48 89 ce 83 e6 01 48 83 ee 01 48 09 f1 48 21 ca 31 c9
[   66.702949][  T532] RSP: 0018:ffff8881097a7478 EFLAGS: 00010282
[   66.703474][  T532] RAX: 0000000011dca450 RBX: 0000000000000000 RCX: 0000777f80000000
[   66.704728][  T532] RDX: ffffebde00477280 RSI: ffffea000447ad40 RDI: 0000000000200000
[   66.706497][  T532] RBP: 0000000011dca450 R08: 0000000000000001 R09: ffffffff9a58ccfe
[   66.708160][  T532] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
[   66.709799][  T532] R13: ffff8881097a74b0 R14: 0000000080000000 R15: ffff888111eb5d00
[   66.711393][  T532] FS:  00007f2990b12780(0000) GS:ffff888228ea7000(0000) knlGS:0000000000000000
[   66.713158][  T532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   66.714486][  T532] CR2: ffffebde00477288 CR3: 0000000103565005 CR4: 0000000000f70ef0
[   66.715595][  T532] PKRU: 55555554
[   66.715972][  T532] Call Trace:
[   66.716320][  T532]  <TASK>
[   66.716629][  T532]  kasan_quarantine_reduce+0x19a/0x250
[   66.717190][  T532]  __kasan_slab_alloc+0x6a/0x90
[   66.717706][  T532]  kmem_cache_alloc_noprof+0x214/0x6b0
[   66.718285][  T532]  ? vm_area_dup+0x2b/0x970
[   66.718763][  T532]  vm_area_dup+0x2b/0x970
[   66.719211][  T532]  ? msleep+0x1b/0x30
[   66.719625][  T532]  __split_vma+0x32e/0xbb0
[   66.720086][  T532]  ? __pfx___split_vma+0x10/0x10
[   66.720596][  T532]  ? __pfx_mas_prev+0x10/0x10
[   66.721095][  T532]  ? lock_is_held_type+0xfa/0x1c0
[   66.721613][  T532]  vma_modify+0x1a50/0x24e0
[   66.722087][  T532]  ? __pfx_vma_modify+0x10/0x10
[   66.722593][  T532]  ? lock_is_held_type+0xfa/0x1c0
[   66.723110][  T532]  vma_modify_flags+0x2ed/0x4e0
[   66.723594][  T532]  ? __pfx_vma_modify_flags+0x10/0x10
[   66.724144][  T532]  ? mas_prev_slot+0x328/0x1d30
[   66.724655][  T532]  mprotect_fixup+0x226/0xb90
[   66.725142][  T532]  ? __pfx_mprotect_fixup+0x10/0x10
[   66.725696][  T532]  ? mas_prev_slot+0x328/0x1d30
[   66.726195][  T532]  ? lock_is_held_type+0xfa/0x1c0
[   66.726706][  T532]  ? mas_next_slot+0xa7a/0x20a0
[   66.727200][  T532]  ? lock_sequence+0xd7/0x180
[   66.727676][  T532]  do_mprotect_pkey+0x81d/0xb40
[   66.728167][  T532]  ? __pfx_do_mprotect_pkey+0x10/0x10
[   66.728702][  T532]  ? rcu_is_watching+0x16/0xb0
[   66.729193][  T532]  ? do_syscall_64+0x203/0x6a0
[   66.729696][  T532]  ? lockdep_hardirqs_on+0x95/0x140
[   66.730226][  T532]  ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   66.730811][  T532]  ? do_vmi_munmap+0x159/0x2e0
[   66.731226][  T532]  ? __vm_munmap+0x1dc/0x360
[   66.731635][  T532]  ? __do_sys_mincore+0x4b6/0x6c0
[   66.732129][  T532]  __x64_sys_mprotect+0x78/0xe0
[   66.732616][  T532]  ? lockdep_hardirqs_on+0x95/0x140
[   66.733140][  T532]  ? do_syscall_64+0x83/0x6a0
[   66.733611][  T532]  do_syscall_64+0xf6/0x6a0
[   66.734068][  T532]  ? trace_hardirqs_on_prepare+0x13d/0x190
[   66.734650][  T532]  ? lockdep_hardirqs_on+0x95/0x140
[   66.735185][  T532]  ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   66.735786][  T532]  ? do_syscall_64+0x221/0x6a0
[   66.736264][  T532]  ? rcu_is_watching+0x16/0xb0
[   66.736746][  T532]  ? rcu_is_watching+0x16/0xb0
[   66.737233][  T532]  ? do_syscall_64+0x203/0x6a0
[   66.737721][  T532]  ? trace_hardirqs_on_prepare+0x13d/0x190
[   66.738302][  T532]  ? lockdep_hardirqs_on+0x95/0x140
[   66.738820][  T532]  ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   66.739924][  T532]  ? do_syscall_64+0x221/0x6a0
[   66.740457][  T532]  ? do_syscall_64+0x203/0x6a0
[   66.740934][  T532]  ? trace_hardirqs_on_prepare+0x13d/0x190
[   66.741518][  T532]  ? lockdep_hardirqs_on+0x95/0x140
[   66.742038][  T532]  ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   66.742638][  T532]  ? rcu_is_watching+0x16/0xb0
[   66.743115][  T532]  ? do_syscall_64+0x31/0x6a0
[   66.743582][  T532]  ? trace_hardirqs_off_finish+0x13d/0x190
[   66.744172][  T532]  ? lockdep_hardirqs_off+0xb3/0x100
[   66.744717][  T532]  ? do_syscall_64+0x64/0x6a0
[   66.745212][  T532]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[   66.745834][  T532] RIP: 0033:0x7f2990c22667
[   66.746301][  T532] Code: ef e8 dd f9 ff ff 84 c0 75 b9 31 db 48 83 c4 08 48 89 d8 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 1f 44 00 00 b8 0a 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 79 77 0d 00 f7 d8 64 89 01 48
[   66.748156][  T532] RSP: 002b:00007fffb8fba1e8 EFLAGS: 00000202 ORIG_RAX: 000000000000000a
[   66.748885][  T532] RAX: ffffffffffffffda RBX: 0000000000004000 RCX: 00007f2990c22667
[   66.749575][  T532] RDX: 0000000000000001 RSI: 0000000000004000 RDI: 00007f2990af6000
[   66.750260][  T532] RBP: 0000000000000008 R08: 00000000ffffffff R09: 0000000000000000
[   66.750942][  T532] R10: 0000000000000022 R11: 0000000000000202 R12: 00007f2990af2000
[   66.751948][  T532] R13: 0000000000020000 R14: 00007f2990af6000 R15: 0000000000000001
[   66.752777][  T532]  </TASK>
[   66.753093][  T532] Modules linked in:
[   66.753516][  T532] CR2: ffffebde00477288
[   66.753935][  T532] ---[ end trace 0000000000000000 ]---
[   66.754496][  T532] RIP: 0010:qlist_free_all+0x93/0x130
[   66.754503][  T532] Code: c2 4c 01 f2 0f 82 a3 00 00 00 48 c7 c1 00 00 00 80 48 2b 0d b7 c9 34 03 48 01 ca 48 c1 ea 0c 48 c1 e2 06 48 03 15 95 c9 34 03 <48> 8b 4a 08 48 89 ce 83 e6 01 48 83 ee 01 48 09 f1 48 21 ca 31 c9
[   66.754505][  T532] RSP: 0018:ffff8881097a7478 EFLAGS: 00010282
[   66.754508][  T532] RAX: 0000000011dca450 RBX: 0000000000000000 RCX: 0000777f80000000
[   66.754509][  T532] RDX: ffffebde00477280 RSI: ffffea000447ad40 RDI: 0000000000200000
[   66.754511][  T532] RBP: 0000000011dca450 R08: 0000000000000001 R09: ffffffff9a58ccfe
[   66.754512][  T532] R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
[   66.754513][  T532] R13: ffff8881097a74b0 R14: 0000000080000000 R15: ffff888111eb5d00
[   66.754515][  T532] FS:  00007f2990b12780(0000) GS:ffff888228ea7000(0000) knlGS:0000000000000000
[   66.754517][  T532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   66.754518][  T532] CR2: ffffebde00477288 CR3: 0000000103565005 CR4: 0000000000f70ef0
[   66.754524][  T532] PKRU: 55555554
[   66.754526][  T532] note: madvise-dontfor[532] exited with irqs disabled

  reply	other threads:[~2026-09-01 21:05 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-31 14:35 Andi Kleen
2026-08-31 14:58 ` Pedro Falcato
2026-08-31 19:35   ` Andi Kleen
2026-08-31 19:55     ` Lorenzo Stoakes (ARM)
2026-09-01 21:05       ` Andi Kleen [this message]
2026-09-02 16:21         ` Lorenzo Stoakes (ARM)
2026-08-31 15:50 ` Lorenzo Stoakes (ARM)
2026-08-31 16:29 ` Liam R. Howlett
2026-08-31 16:52   ` Andi Kleen
2026-08-31 22:13     ` Liam R. Howlett

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=apc-F_D5mWwWL74i@firstfloor.org \
    --to=andi@firstfloor.org \
    --cc=ak@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=jannh@google.com \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=pfalcato@suse.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®