SO_REUSEADDR redux

SO_REUSEADDR redux

[Paul D. Smith]

So, I have an application that works fine on Solaris, FreeBSD, etc. and
fails on Linux.

This application uses SO_REUSEADDR in conjunction with INADDR_ANY.  What
it does is bind() to INADDR_ANY, then listen().  Then, it proceeds to
bind (but _not_ listen) various other specific addresses.

This works fine on Solaris, but the second, etc. bind fails on Linux.

I found a thread about this from February, 1999, starting here:

  http://www.uwsg.indiana.edu/hypermail/linux/kernel/9902.1/0828.html

In this message, George Pajari (George.Pajari@faximum.com) says:

  The original security threat that lead to SO_REUSERADDR being broken in the
  1.3.60 kernel (incorrect semantics which I think still persist) was the
  fact that if process A bound to port X using INADDR_ANY and SO_REUSERADDR,
  a second process could bind to the specific IP addresses and obtain
  connections intended for process A.

  So the kernel was changed to prevent the second process from completing the
  bind() call.

Maybe I'm missing something, but it seems to me that having two binds is
not a security problem: what's really the problem is having two
_listens_.  As long as you're only listening on the one, I don't see how
connections/packets could be stolen.

Isn't the correct behavior to fail the second listen(), not the second
bind()?  This, anyway, is what Solaris and FreeBSD (and maybe others)
seem to do.

This app does this because it implements peer-to-peer, fully meshed
connections; it first sets up an INADDR_ANY and listens on it so that
any other instance of the app can reach it, then it proceeds to try to
bind to the other instances.  Since we have no idea which one will start
first, or whether they'll all start together, we don't want to have a
hole where we might miss a connection from another instance.  Thus, we
listen first, then proceed to attempt to connect to the other
instances.

Am I missing something here WRT SO_REUSEADDR vs. INADDR_ANY behavior?

Thanks.


PS. CC'ing me is helpful, but I'll follow along either way.  Thx.

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@baynetworks.com>    HASMAT--HA Software Methods & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: SO_REUSEADDR redux

[Alan Cox]

> This application uses SO_REUSEADDR in conjunction with INADDR_ANY.  What
> it does is bind() to INADDR_ANY, then listen().  Then, it proceeds to
> bind (but _not_ listen) various other specific addresses.

That should be ok if its setting SO_REUSEADDR

> not a security problem: what's really the problem is having two
> _listens_.  As long as you're only listening on the one, I don't see how
> connections/packets could be stolen.

UDP.

In fact the classic exploit consisted of binding to port 2049 witha specific
connect address set on UDP and stealing NFS packets..

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: SO_REUSEADDR redux

[David S. Miller]

Paul D. Smith writes:
 > I agree, and so does Solaris/FreeBSD, but Linux doesn't.  See below for
 > a test program.  Maybe I'm doing something screwed up.

I think I'm starting to get a hold on this.  Can you do a test
for me?

Write a small test program, similar to your TCP one, which instead
uses UDP and let me know what FreeBSD and Solaris do in that case.
Of course, skip the listen part.

I think the test for port reuse in tcp_{v4,v6}_get_port is buggy.
I'll go study this and fix it up... but please do the test I asked
anyways.

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: SO_REUSEADDR redux

[David S. Miller]

Paul D. Smith writes:
 > %% Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
 > 
 >   >> This application uses SO_REUSEADDR in conjunction with INADDR_ANY.  What
 >   >> it does is bind() to INADDR_ANY, then listen().  Then, it proceeds to
 >   >> bind (but _not_ listen) various other specific addresses.
 > 
 >   ac> That should be ok if its setting SO_REUSEADDR
 > 
 > I agree, and so does Solaris/FreeBSD, but Linux doesn't.

After reading up some code, FreeBSD does do a bind() check which is
just as restrictive as Linux's except that they allow INADDR_ANY
combinations when the credentials of the user doing the bind() match
the credentials of all other sockets bound to that port.

I don't think we should change our behavior.  Allowing the combination
in question only when the UIDs match between the socket owners is
dubious at best.

I actually went to the FreeBSD code because what Steven's showed
was extremely loose in what it let through.  It allowed the nfs
port override trick Alan mentioned.

Later,
David S. Miller
davem@redhat.com

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: SO_REUSEADDR redux

[Paul D. Smith]

%% Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

  >> This application uses SO_REUSEADDR in conjunction with INADDR_ANY.  What
  >> it does is bind() to INADDR_ANY, then listen().  Then, it proceeds to
  >> bind (but _not_ listen) various other specific addresses.

  ac> That should be ok if its setting SO_REUSEADDR

I agree, and so does Solaris/FreeBSD, but Linux doesn't.  See below for
a test program.  Maybe I'm doing something screwed up.

-------------------------8>< snip ><8-------------------------
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define MY_PORT     10000


int
main(int argc, char *argv[])
{
  int any_fd, this_fd;
  int val = 1;
  struct sockaddr_in addr;

  if ((any_fd = socket(PF_INET, SOCK_STREAM, 0)) < 0
      || fcntl(any_fd, F_SETFL, O_NONBLOCK) < 0
      || setsockopt(any_fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof val) < 0) {
    perror("setup(any)");
    return 1;
  }

  if ((this_fd = socket(PF_INET, SOCK_STREAM, 0)) < 0
      || fcntl(this_fd, F_SETFL, O_NONBLOCK) < 0
      || setsockopt(this_fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof val) < 0) {
    perror("setup(this)");
    return 1;
  }

  addr.sin_family = AF_INET;
  addr.sin_port = MY_PORT;
  memset(addr.sin_zero, 0, sizeof (addr.sin_zero));

  addr.sin_addr.s_addr = INADDR_ANY;

  if (bind(any_fd, (struct sockaddr *)&addr, sizeof (addr)) < 0) {
    perror("bind(any)");
    return 1;
  }

  if (listen(any_fd, 10) < 0) {
    perror("listen(any)");
    return 1;
  }

  addr.sin_addr.s_addr = INADDR_LOOPBACK;

  if (bind(this_fd, (struct sockaddr *)&addr, sizeof (addr)) < 0) {
    perror("bind(this)");
    return 1;
  }

  return 0;
}
-------------------------8>< snip ><8-------------------------

  solaris$ gcc -o reuseaddr{,.c} -lsocket -lnsl
  solaris$ ./reuseaddr

Works.  Now:

  linux$ gcc -o reuseaddr{,.c}
  linux$ ./reuseaddr
  bind(this): Cannot assign requested address

:(  The real code doesn't use LOOPBACK, of course.

This is Linux 2.2.17, but I tried with 2.2.18 too.  I haven't tried
2.4.x.


Thanks...

-- 
-------------------------------------------------------------------------------
 Paul D. Smith <psmith@baynetworks.com>    HASMAT--HA Software Methods & Tools
 "Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
   These are my opinions---Nortel Networks takes no responsibility for them.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/