Re: Syslets, Threadlets, generic AIO support, v6

[Linus Torvalds <torvalds@linux-foundation.org>]

On Wed, 30 May 2007, Ingo Molnar wrote:
>
> * Ulrich Drepper <drepper@redhat.com> wrote:
> > 
> > I'm not going to judge your tests but saying there are no significant 
> > advantages is too one-sided.  There is one huge advantage: the 
> > interface.  A memory-based interface is simply the best form.  File 
> > descriptors are a resource the runtime cannot transparently consume.
> 
> yeah - this is a fundamental design question for Linus i guess :-)

Well, quite frankly, to me, the most important part of syslets is that if 
they are done right, they introduce _no_ new interfaces at all that people 
actually use.

Over the years, we've done lots of nice "extended functionality" stuff. 
Nobody ever uses them. The only thing that gets used is the standard stuff 
that everybody else does too.

So when it comes to syslets, the most important interface will be the 
existing aio_read() etc interfaces _without_ any in-memory stuff at all, 
and everything done by the kernel to just make it look exactly like it 
used to look. And the biggest advantage is that it simplifies the internal 
kernel code, and makes us use the same code for aio and non-aio (and I 
think we have a good possibility of improving performance too, if only 
because we will get much more natural and fine-grained scheduling points!)

Any extended "direct syslets" use is technically _interesting_, but 
ultimately almost totally pointless. Which was why I was pushing really 
really hard for a simple interface and not being too clever or exposing 
internal designs too much. An in-memory thing tends to be the absolute 
_worst_ interface when it comes to abstraction layers and future changes.

> glibc (and other infrastructure libraries) have a fundamental problem: 
> they cannot (and do not) presently use persistent file descriptors to 
> make use of kernel functionality, due to ABI side-effects.

glibc has a more fundamental problem: the "fun" stuff is generally not 
worth it. 

For example, any AIO thing that requires glibc to be rewritten is almost 
totally uninteresting. It should work with _existing_ binaries, and 
_existing_ ABI's to be useful - since 99% of all AIO users are binary- 
only and won't recompile for some experimental library.

The whole epoll/kevent flame-wars have ignored a huge issue: almost nobody 
uses either. People still use poll and select, to such an _overwhelming_ 
degree that it almost doesn't even matter if you were to make the 
alternatives orders of magnitude faster - it wouldn't even be visible. 

> we should perhaps enable glibc to have its separate fd namespace (or 
> 'hidden' file descriptors at the upper end of the fd space) so that it 
> can transparently listen to netlink events (or do epoll), without 
> impacting the application fd namespace - instead of ducking to a memory 
> based API as a workaround.

Yeah, I don't think it would be at all wrong to have "private file 
descriptors". I'd prefer that over memory-based (for all the abstraction 
issues, and because a lot of things really *are* about file descriptors!). 

		Linus