mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Kyle Moffett <mrmacman_g4@mac.com>
To: Paul Mackerras <paulus@samba.org>
Cc: Davide Libenzi <davidel@xmailserver.org>,
	Alan Cox <alan@lxorguk.ukuu.org.uk>, Theodore Tso <tytso@mit.edu>,
	Ulrich Drepper <drepper@redhat.com>,
	Eric Dumazet <dada1@cosmosbay.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Ingo Molnar <mingo@elte.hu>
Subject: Re: [patch 7/8] fdmap v2 - implement sys_socket2
Date: Sat, 9 Jun 2007 10:38:55 -0400	[thread overview]
Message-ID: <463C79D8-88CB-492A-96FD-10C10D16C4B6@mac.com> (raw)
In-Reply-To: <18026.15766.574693.200636@cargo.ozlabs.ibm.com>

On Jun 09, 2007, at 01:41:42, Paul Mackerras wrote:
> Davide Libenzi writes:
>
>> The only reason we use a floating base, is because Uli preferred  
>> to have non-exactly predictable fd allocations. There no reason of  
>> re-doing the same POSIX mistake all over again:
>
> Why must everything that makes things a bit simpler and more  
> predictable for application programmers be called a "mistake"?

1)  Linear FD allocation makes it IMPOSSIBLE for libraries to  
reliably use persistent FDs behind an application's back.  For  
example, this code sequence should almost always be OK: (except when  
calling into a library specifically to open a file or socket)

close(0);
some_library_function();
fd = open("some_file", O_RDWR);
/* fd == 0 here */

2)  Another common code example which causes problems:

int i;
for (i = 0; i < NR_OPEN; i++)
	if (!fd_is_special_to_us(i))
		close(i);

Note that this is conceptually buggy, but occurs in several major C  
programming books, most of the major shells, and a lot of other  
software to boot.

3) In order to allocate new FDs, the kernel has to scan over a  
(potentially very large) bitmap.  A process with 100,000 fds (not  
terribly uncommon) would have 12.5kbyte of FD bitmap and would trash  
the cache every time it tried to allocate an FD.

You can't even hope to use persistent library FDs with these  
problems, and they even cause problems with servers using lots of  
FDs.  Introducing another linear FD space will just make this sort of  
stuff happen ALL OVER AGAIN.

If you want to make things reproducible, you could have an option  
where the "random" algorithm is just pseudo-linearly allocated FDs  
(no bitmap search) XORed with a boot-time constant either randomly- 
generated or set in the boot args.  That way the people interested in  
maximum security or stress testing can use completely randomized  
numbers and the people who need reproducibility can get that too,  
*without* having the same linear FD allocation problems all over  
again.  The fundamental problem is that a series of numbers  
increasing linearly from X (especially where X == 0) are used as  
meaningful data by lazy programmers, instead of just as a cookie.  By  
nonlinearizing the data, even just a little, that becomes unlikely to  
occur.  It also forces programmers to use FDs correctly and prevents  
problems like this in the future.

Cheers,
Kyle Moffett

  reply	other threads:[~2007-06-09 14:40 UTC|newest]

Thread overview: 129+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-06-06 22:30 Davide Libenzi
2007-06-06 22:44 ` David Miller
2007-06-06 22:52   ` Davide Libenzi
2007-06-06 22:57     ` David Miller
2007-06-06 22:57   ` Ulrich Drepper
2007-06-06 23:02     ` David Miller
2007-06-06 22:59 ` Alan Cox
2007-06-06 22:58   ` Ulrich Drepper
2007-06-06 23:04   ` Davide Libenzi
2007-06-06 23:08     ` David Miller
2007-06-06 23:19     ` Alan Cox
2007-06-06 23:22       ` Ulrich Drepper
2007-06-07 10:04         ` Alan Cox
2007-06-07 11:59           ` Kyle Moffett
2007-06-07 13:12             ` Eric Dumazet
2007-06-07 15:51               ` Davide Libenzi
2007-06-07 19:49               ` Davide Libenzi
2007-06-07 20:02                 ` Ulrich Drepper
2007-06-07 20:05                 ` Eric Dumazet
2007-06-07 20:18                   ` Ulrich Drepper
2007-06-07 21:44                     ` Davide Libenzi
2007-06-07 22:03                       ` Ulrich Drepper
2007-06-07 22:40                         ` Davide Libenzi
2007-06-08 12:07                           ` Theodore Tso
2007-06-08 13:01                             ` Alan Cox
2007-06-08 18:11                               ` Davide Libenzi
2007-06-08 18:26                                 ` Alan Cox
2007-06-08 18:43                                   ` Ulrich Drepper
2007-06-08 18:46                                     ` Al Viro
2007-06-08 18:56                                       ` Ulrich Drepper
2007-06-08 19:07                                         ` Linus Torvalds
2007-06-08 19:21                                           ` Davide Libenzi
2007-06-09  0:03                                             ` Linus Torvalds
2007-06-09  0:13                                               ` Davide Libenzi
2007-06-09  0:36                                               ` Al Viro
2007-06-09  1:19                                                 ` Ulrich Drepper
2007-06-09  1:41                                                   ` Al Viro
2007-06-09  2:10                                                     ` Ulrich Drepper
2007-06-09 15:15                                                       ` Al Viro
2007-06-09 16:26                                                         ` Ulrich Drepper
2007-06-09 16:54                                                           ` Al Viro
2007-06-09 17:04                                                             ` Davide Libenzi
2007-06-09 17:08                                                               ` Davide Libenzi
2007-06-09 17:08                                                             ` Ulrich Drepper
2007-06-09 17:24                                                               ` Al Viro
2007-06-09 19:27                                                                 ` Kyle Moffett
2007-06-09 20:06                                                                   ` Al Viro
2007-06-09 20:21                                                                     ` Linus Torvalds
2007-06-09 20:31                                                                       ` Davide Libenzi
2007-06-09 21:41                                                                         ` Matt Mackall
2007-06-09 22:12                                                                           ` Davide Libenzi
2007-06-09 20:49                                                                       ` Al Viro
2007-06-09 21:55                                                                         ` Matt Mackall
2007-06-09 23:33                                                                         ` Linus Torvalds
2007-06-10  3:35                                                                           ` Davide Libenzi
2007-06-10  3:49                                                                             ` Davide Libenzi
2007-06-10  3:19                                                                       ` Al Viro
2007-06-10  3:48                                                                         ` Linus Torvalds
2007-06-10  4:00                                                                           ` Al Viro
2007-06-10  4:03                                                                             ` Linus Torvalds
2007-06-10  4:06                                                                               ` Al Viro
2007-06-10  4:45                                                                           ` dean gaudet
2007-06-10  5:06                                                                             ` Linus Torvalds
2007-06-10  5:46                                                                               ` Al Viro
2007-06-10 17:23                                                                                 ` Linus Torvalds
2007-06-10  6:35                                                                           ` Kari Hurtta
2007-06-10 15:21                                                                             ` Alan Cox
2007-06-10  9:14                                                                       ` Eric Dumazet
2007-06-10 15:16                                                                         ` Alan Cox
2007-06-10 18:19                                                                         ` Linus Torvalds
2007-06-10  2:40                                                                   ` Al Viro
2007-06-08 19:34                                         ` Alan Cox
2007-06-08 19:30                                     ` Alan Cox
2007-06-08 19:37                                       ` Davide Libenzi
2007-06-08 19:48                                         ` Alan Cox
2007-06-08 19:51                                           ` Davide Libenzi
2007-06-08 21:24                                             ` Alan Cox
2007-06-08 21:59                                               ` Davide Libenzi
2007-06-08 22:28                                                 ` Alan Cox
2007-06-08 22:38                                                   ` Davide Libenzi
2007-06-11  8:24                                       ` Xavier Bestel
2007-06-08 19:22                                   ` Davide Libenzi
2007-06-09  5:41                                 ` Paul Mackerras
2007-06-09 14:38                                   ` Kyle Moffett [this message]
2007-06-10  6:48                                     ` Paul Mackerras
2007-06-10 15:56                                       ` Davide Libenzi
2007-06-10 19:16                                       ` Davide Libenzi
2007-06-09 17:00                                   ` Davide Libenzi
2007-06-10  6:26                                     ` Paul Mackerras
2007-06-10  7:10                                       ` William Lee Irwin III
2007-06-10 15:52                                       ` Davide Libenzi
2007-06-08 18:07                             ` Davide Libenzi
2007-06-08 18:35                             ` Linus Torvalds
2007-06-07 21:57                   ` Davide Libenzi
2007-06-08  4:38                     ` Eric Dumazet
2007-06-08  5:20                       ` Davide Libenzi
2007-06-07 14:25           ` Ulrich Drepper
2007-06-07 17:56             ` Eric Dumazet
2007-06-07 18:03               ` Davide Libenzi
2007-06-07 18:57                 ` Eric Dumazet
2007-06-07 18:26               ` Ulrich Drepper
2007-06-07 18:39                 ` Davide Libenzi
2007-06-07 18:56                   ` Ulrich Drepper
2007-06-07 19:12                     ` Davide Libenzi
2007-06-07 20:03                   ` Andrew Morton
2007-06-08  2:55                     ` Ulrich Drepper
2007-06-08  5:16                       ` Davide Libenzi
2007-06-06 23:29       ` Davide Libenzi
2007-06-07 10:06         ` Alan Cox
2007-06-07 10:45           ` Eric Dumazet
2007-06-07 11:27             ` Alan Cox
2007-06-07 15:41           ` Davide Libenzi
2007-06-07 20:10   ` Linus Torvalds
2007-06-07 20:47     ` Eric Dumazet
2007-06-07 21:08       ` Linus Torvalds
2007-06-07 21:41         ` Davide Libenzi
2007-06-07 20:59     ` Guillaume Chazarain
2007-06-07 21:06       ` Guillaume Chazarain
2007-06-07 21:31     ` Ulrich Drepper
2007-06-07 22:22     ` Davide Libenzi
2007-06-07 23:42       ` Linus Torvalds
2007-06-08  0:04         ` Davide Libenzi
2007-06-08  0:59     ` Matt Mackall
2007-06-08  2:25       ` Linus Torvalds
2007-06-08 15:56     ` Jeff Dike
2007-06-07  0:29 ` Arnd Bergmann
2007-06-07  0:33   ` Davide Libenzi
2007-06-06 22:30 [patch 1/8] fdmap v2 - fdmap core Davide Libenzi
2007-06-07  6:54 ` Eric Dumazet
2007-06-07  7:10   ` Davide Libenzi
2007-06-07 10:39     ` [patch 7/8] fdmap v2 - implement sys_socket2 Eric Dumazet
2007-06-07 15:42       ` Davide Libenzi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=463C79D8-88CB-492A-96FD-10C10D16C4B6@mac.com \
    --to=mrmacman_g4@mac.com \
    --cc=akpm@linux-foundation.org \
    --cc=alan@lxorguk.ukuu.org.uk \
    --cc=dada1@cosmosbay.com \
    --cc=davidel@xmailserver.org \
    --cc=drepper@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=paulus@samba.org \
    --cc=torvalds@linux-foundation.org \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®