Re: [RFC PATCH 0/5] rseq: add support for RSEQ operations

[Thomas Gleixner <tglx@kernel.org>]

Olivier!

On Fri, Aug 28 2026 at 11:33, odion@efficios.com wrote:
> This series introduces RSEQ operations: a per-thread list of operations the
> kernel applies, on behalf of a ask, when returning to user space.
>
> The main motivation is to help TCMalloc migrate from RSEQ v1 to RSEQ v2 and use
> the RSEQ area registered by glibc. TCMalloc currently relies on RSEQ v1
> resetting the cpu_id field to invalidate a per-CPU pointer cached in TLS. This
> requires applications to disable glibc's implicit RSEQ registration and prevents
> sharing the glibc RSEQ area.
>
> RSEQ operations provide an opt-in replacement for that invalidation
> mechanism. User space registers operation nodes with prctl; the kernel keeps the
> nodes in a per-thread circular list and applies them on return to user
> space. Only tasks with registered operations incur the additional RSEQ exit-path
> work.

TBH, to me this sounds like a horrible idea. It's yet another
"interpreter" for a very limited use case and a lot of complexity.

I think this can be done in user space with some help of the kernel of
course. If user space has registered an entry point for this
functionality then the kernel can emulate a call to that on return to
user space:

      if (tsk->rseq.needs_fixup) {
                // modifies regs->sp
		create_callframe_on_user stack(regs);
                regs->ip = tsk->rseq.fixup_ip;
      }
      ret_to_user()

The user space fixup does:

fixup_ip()
      ....
      user_rseq.fixup_in_progress = false;
      restore_and_return() // Returns to the original IP

A reasonable limitation for the fixup function should be a strict "no
syscalls and no floating point within the fixup" rule. No floating point
avoids the whole sigframe disaster.

Of course the above is way too simple to be true :) It works except when
it nests. But that's solvable too:

      if (tsk->rseq.needs_fixup) {
                // modifies regs->sp
		create_callframe_on_user stack(regs);
                regs->ip = tsk->rseq.fixup_ip;
                // Save the callframe SP
                tsk->rseq.fixup_sp = regs->sp;
      }

When return to user observes user_rseq.fixup_in_progress then it can
mangle regs before doing anything else:

        regs->ip = tsk->rseq.fixup_abort_ip;
        regs->sp = tsk->rseq.fixup_sp;
        user_rseq.fixup_in_progress = false;
        tsk->rseq.needs_fixup = true;
    
Which rewinds the stack to the callframe and makes the interrupted fixup
continue at the fixup_abort_ip which just restores registers from the
callframe and returns to the original IP.

Then the flow becomes in case of a trivial interrupt which just returns:

    fixup_ip()
      -> interrupt
                                ...
                                if (user_rseq.fixup_in_progress)
                                   rewind_fixup(regs);
                                 ...
                                if (needs_fixup)
                                   setup_fixup(regs);
                                return to user;
       fixup_ip()
          ...
          user_rseq.fixup_in_progress = false
          restore_and_return()
   fixup_abort_ip:
      restore_and_return()      // Returns to orig_ip

In case of a signal delivery:

    fixup_ip()
      -> interrupt
                                ...
                                if (user_rseq.fixup_in_progress)
                                   rewind_fixup(regs);

                                   deliver_signal()
                                     setup_sigframe()
                                  ...
                                if (needs_fixup)
                                   setup_fixup(regs);
                                return to user;
       fixup_ip()
          ...
          user_rseq.fixup_in_progress = false
          restore_and_return()
       signal_handler()
          sys_sigreturn()
                                restore(regs)
                                return to user;

   fixup_abort_ip:
      restore_and_return()      // Returns to orig_ip

There are obviously a ton of details to take care of (/me mumbles shadow
stacks and RSEQ CS interaction), but the general principle should just
work. Emphasis on should and I'm so NOT going to hack that up. :)

I've played around with something similar pre RSEQ, but all I could
still find are the notes I took back then, which I duly transcribed into
todays RSEQ world. The patches themself would be utterly useless anyway
as all parts which need to be touched have been through the mincer at
least once.

The fixup address and the fixup abort address should probably not be
stored in user rseq memory. For an initial POC they just can be
registered through sys_rseq() which also enables the fixup mechanics,
i.e. the unconditional set of tsk->rseq.needs_fixup in schedule(), which
should be completely independent of the rseq.sched_switch and
rseq.ids_changed bits and not change the existing functionality and
semantics like the current POC does. Keep it separate.

Once the POC dust has settled the registration interface should change
for the final implementation because otherwise this would create the
very same problem of who owns it again.

The final solution should provide a trivial dispatch mechanism in the
VDSO and the RSEQ user area should be expanded to provide storage for
registration. The syscall registration would change to registering a lib
specific callback/data pair and the kernel would reorganize the
storage. Whether that's a linked list or a size limited array in the
RSEQ user area does not matter. As this is thread local there is no
concurrency and the syscall can rearange that storage completely
undisturbed. Libraries can manage their own thread local marker which is
checked in the lib specific callback to decide whether they want to run
or not.

Using a VDSO dispatcher and expanding the RSEQ storage for that should
just work out of the box with any libc which supports RSEQ_V2 and would
therefore not create any additional libc dependencies for other
library developers.

The dispatcher entry needs to be trivial ASM for the restore/return

fixup_ip:
        // setup_fixup() in the kernel stored the TLS user RSEQ address in
        // RDI or whatever an architecture uses for the first argument
	CALL	fixup_c
fixup_ip_abort:
	ASM_RESTORE_REGS
        RET

The C part should be trivial too:

fixup_c(struct rseq *rseq) 
{
        for_each_fixup(fn, data, rseq)
        	fn(data);
}

or something along these lines.

Thoughts?

Thanks,

        tglx