Re: [RFC PATCH 00/15] io_uring: thread identity handoff for blocking inline issue

["Eric W. Biederman" <ebiederm@xmission.com>]

Jens Axboe <axboe@kernel.dk> writes:

> Hi,
>
> io_uring issues requests inline with IO_URING_F_NONBLOCK and punts to
> io-wq when that isn't possible. For a range of opcodes it isn't possible
> at all, as there's no nonblocking path in the kernel for them: fsync,
> statx, openat, the *at family, xattr, fadvise, splice, etc. Those are
> punted unconditionally, and the punt costs a thread wakeup, a context
> switch and a task_work completion round trip per request. io_uring HAS
> to be cautious to prevent accidental blocking in the kernel, even if the
> operations predominantly never block. Sad story. Examples of that are
> things like an fdatasync that doesn't block, statx that hits dcache,
> openat for O_TMPFILE, etc. All of those would've completed inline just
> fine, but io_uring just cannot rely on that.
>
> This series issues those requests inline in blocking mode instead, and
> only pays for the offload if the request actually blocks. But by the
> time it blocks, the submitter is deep in the kernel with the request on
> its stack, so the work can't be moved to another thread. What we can
> move is the identity. If the submitting task blocks, an idle io-wq
> worker takes over its user visible identity (tid, signal state,
> credentials, scheduling attributes, cgroup, user register state),
> finishes the io_uring_enter() call and returns to userspace as the
> submitter. The original task finishes the request as an
> io-wq worker and joins the pool. Userspace is none the wiser, hopefully,
> the same tid came back from the syscall, it's just on a different
> task_struct. Folks that have been around a while may remember earlier
> attempts at this about 20 years ago.

I don't see anything immediately wrong, but I suspect I am just
not looking hard enough.

In my time working with the kernel I have never seen anyone actually get
this kind of thing correct.

The handoff that we do during exec has a bug with posix timers that
I think is 23 years old that we just caught, and still hasn't been
merged to Linus.

There was the old daemonize call that got it wrong so often I added
kthreadd.


Maybe you want something like the old solaris doors, or vfork.
Perform a synchronous task switch to this other thread, and call this
function in the other thread.  Then block waiting on the other thread
until the other thread blocks, or the function you called finishes.

Is there a reason you didn't try and do it that way?
Just a synchronous switch to and from a thread in your thread pool?

You aren't changing the mm so I really doubt changing the stack pointer
and a registers will be that expensive.


Eric