Re: [PATCH v2 00/11] unwind, perf: sframe user space unwinding, deferred perf callchains

[Steven Rostedt <rostedt@goodmis.org>]

On Mon, 16 Sep 2024 16:08:56 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> > 
> > I think the biggest tweak we decided on is that the context id (aka
> > "cookie") would be percpu.  Its initial value is (cpuid << 48).  It gets
> > incremented for every entry from user space.  
> 
> Why? What's the purpose of the cookie? This scheme seems unsound, pin
> yourself on CPU0 and trigger 1<<48 unwinds while keeping CPU1 idle.
> 
> > > That is, we should have an interface like:
> > > 
> > > typedef void (unwinder_callback_t)(struct user_space_stack *, u64 cookie);  
> 
> Just make it a void* and let the consumer figure it out.

Let me try to explain the rationale for the "cookie". It's actually a "context_cookie".
The cookie is unique for every time user space enters the kernel.

Basically it's a "tag" (we could call it that too, but "cookie" seemed
more appropriate as it follows what web browsers do). The idea is this:

 * You're in interrupt context and want a user space back trace, and
   request to have a user stack trace when the task goes back to user
   space. Now the unwinder will return the current context cookie" and
   will use that same cookie when it creates the user stack on exit
   back to user space.

* If this happens in a long system call that does many stack traces
  (records the kernel stack and also wants the user stack), it does not
  record the user trace at the time it is requested. But we need a way
  to tag this event with the user stack trace that will happen when the
  user stack is recorded when going back to user space. We can't pin
  the task to the CPU when we want to profile it. The cookie/tag needs
  to be the same for every request that happens with a single entry
  into the kernel. It must be different for another entry into the
  kernel as the user stack will be different then.

  Basically think that each cookie represents a single user stack.

* When going back to user space, the ptrace path is taken and the user
  stack trace is recorded. It will then call the tracers that requested
  it with the stack trace and the cookie that represents it.

Here's an example:

system_call() {

	<interrupt> trigger user stack trace: assign cookie 123
		tracer records kernel stack trace and adds cookie 123.

	<interrupt> trigger user stack trace: assign cookie 123
		tracer records kernel stack trace and adds cookie 123.

	<interrupt> trigger user stack trace: assign cookie 123
		tracer records kernel stack trace and adds cookie 123.


	<return to user space, take ptrace path>

	<record user stack trace>
	call all the tracers with user stack trace with cookie 123.
}

system_call() {

	<interrupt> trigger user stack trace: assign cookie 124
		tracer records kernel stack trace and adds cookie 124.

	<return to user space, take ptrace path>

	<record user stack trace>
	call all the tracers with user stack trace with cookie 124.
}


Then the tracers can post process the events and append the user stack
trace with cookie 123 on top of the kernel stack events that had cookie
123, and append the user stack trace with cookie 124 on top of the
kernel stack events that had cookie 124.

-- Steve