Re: [patch 7/8] fdmap v2 - implement sys_socket2

[Linus Torvalds <torvalds@linux-foundation.org>]

On Sun, 10 Jun 2007, Al Viro wrote:
> > 
> > And that means that libraries currently MUST NOT open their own file 
> > descriptors, exactly because they mess with the "application file 
> > descriptor namespace", namely the linear POSIX-defined fd allocation 
> > rules!
> 
> Unless it does so in a thread that has unshared its descriptor table.

Agreed. That was actually part of the reason why I thought clone() was
much better than the pthreads interface.

That said, the Linux !CLONE_FILES does have downsides:

 - it is potentially much slower to do than sharing everything (if you 
   have lots of file descriptors, incrementing the refcounts etc is 
   actually a real overhead)

 - it simply doesn't work, if the library wants to run in the same 
   execution context, and just wants to open one (or more) file 
   descriptors for some helper thing.

IOW, the most common case for libraries is not that they get invoced to do 
one thing, but that they get loaded and then used over and over and over 
again, and the _reason_ for wanting to have a file descriptor open may 
well be that the library wants to cache the file descriptor, rather than 
having to open a file over and over again!

For example, a library routine that does a full

	fd = open();
	.. do something with it ..
	close(fd);

generally doesn't need any private file descriptors at all (although there 
are the threading issues with exec etc) - it will temporarily use a normal 
file descriptor, and the caller won't be any wiser. Lots of current 
library routines do this all the time.

But let's say that you want to do a library that does name resolution, and 
you actually want to create the socket that binds to the DNS server just 
once, and then re-use that socket across library calls. It's not that the 
library is a thread of its own - it's not - but with the normal linear fd 
space it really cannot do this. Sure, it could try to hide it up somewhere 
in high fd space, but that would slow down other operations, and there's 
no way to guarantee it doesn't clash with some _other_ library doing the 
same thing, so it really isn't a good idea.

Now, when you do a DNS query, the setup cost of opening the socket is the 
least of your worries, so the above example is not a very good one. I'm 
really just giving it as a concrete example of a _conceptual_ problem, 
where some other library really had more pressing performance reasons why 
they cannot keep re-opening a file descriptor and closing it each time.

So _that_ is the kind of situation where I think "anonymous file 
descriptors" make sense. 

		Linus