AF_UNIX sockets: strange behaviour

AF_UNIX sockets: strange behaviour

[Catalin Drula]

Hi,

I have a small application that communicates over Bluetooth. I use
connection-oriented UNIX domain sockets (AF_UNIX, SOCK_STREAM) to
communicate between the applications's threads. When reading from
one of these sockets, I get a strange behaviour: if I read all the
bytes that are available (13, in this case) all at once, it's fine;
however, if I try to read in two smaller batches (say, first time
6, and second time 7), the first read returns (with the 6 bytes), but
the second read never returns.

As far as I know, this is not expected behaviour for SOCK_STREAM sockets.
I tried looking into the problem so I instrumented net/unix/af_unix.c to
see what is going on. More specifically, I was focusing on the function
unix_stream_recvmsg. Here is what I noticed:

  - there is a skb in the sk_receive_queue with a len of 13
  - 6 bytes are read from it
  - a skb with the remaining 7 bytes is requeued in sk_receive_queue
  - on the next call to unix_stream_recvmsg, the sk_receive_queue is
empty (!)

Thus, this confirms the behaviour observed from userspace. Is this a bug?
Who could be removing the skb from the receive_queue?

Thanks for any ideas/suggestions,

Catalin

ps Please cc: your replies to me.

Re: AF_UNIX sockets: strange behaviour

[linux-os]

On Wed, 17 Nov 2004, Catalin Drula wrote:

> Hi,
>
> I have a small application that communicates over Bluetooth. I use
> connection-oriented UNIX domain sockets (AF_UNIX, SOCK_STREAM) to
> communicate between the applications's threads. When reading from
> one of these sockets, I get a strange behaviour: if I read all the
> bytes that are available (13, in this case) all at once, it's fine;
> however, if I try to read in two smaller batches (say, first time
> 6, and second time 7), the first read returns (with the 6 bytes), but
> the second read never returns.
>
> As far as I know, this is not expected behaviour for SOCK_STREAM sockets.
> I tried looking into the problem so I instrumented net/unix/af_unix.c to
> see what is going on. More specifically, I was focusing on the function
> unix_stream_recvmsg. Here is what I noticed:
>
>  - there is a skb in the sk_receive_queue with a len of 13
>  - 6 bytes are read from it
>  - a skb with the remaining 7 bytes is requeued in sk_receive_queue
>  - on the next call to unix_stream_recvmsg, the sk_receive_queue is
> empty (!)
>
> Thus, this confirms the behaviour observed from userspace. Is this a bug?
> Who could be removing the skb from the receive_queue?
>
> Thanks for any ideas/suggestions,
>
> Catalin
>
> ps Please cc: your replies to me.

If you need STREAM behavior I think you need to use recv(), recvfrom(),
or read().

If you use recvmsg(), the "message" will be removed even it you haven't 
read it all. Note in the 'man' page description:
"If the a message is too long to fit in the supplied buffer, excess bytes
may be discarded depending upon the type of socket the message is being
received from..."


Cheers,
Dick Johnson
Penguin : Linux version 2.6.9 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by John Ashcroft.
                  98.36% of all statistics are fiction.

Re: AF_UNIX sockets: strange behaviour

[Stephen Hemminger]

On Wed, 17 Nov 2004 16:29:14 +0100 (CET)
Catalin Drula <Catalin.Drula@imag.fr> wrote:

> Hi,
> 
> I have a small application that communicates over Bluetooth. I use
> connection-oriented UNIX domain sockets (AF_UNIX, SOCK_STREAM) to
> communicate between the applications's threads. When reading from
> one of these sockets, I get a strange behaviour: if I read all the
> bytes that are available (13, in this case) all at once, it's fine;
> however, if I try to read in two smaller batches (say, first time
> 6, and second time 7), the first read returns (with the 6 bytes), but
> the second read never returns.
> 
> As far as I know, this is not expected behaviour for SOCK_STREAM sockets.
> I tried looking into the problem so I instrumented net/unix/af_unix.c to
> see what is going on. More specifically, I was focusing on the function
> unix_stream_recvmsg. Here is what I noticed:
> 
>   - there is a skb in the sk_receive_queue with a len of 13
>   - 6 bytes are read from it
>   - a skb with the remaining 7 bytes is requeued in sk_receive_queue
>   - on the next call to unix_stream_recvmsg, the sk_receive_queue is
> empty (!)
> 
> Thus, this confirms the behaviour observed from userspace. Is this a bug?
> Who could be removing the skb from the receive_queue?
> 
> Thanks for any ideas/suggestions,
> 
> Catalin
>

See the following, it doesn't seem to be a problem (this is on 2.6.10-rc2)

/* Unix domain socket buffer test */
#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>
#include <sys/un.h>

static void client(const char *path, const char *msg) {
	int s, len = strlen(msg);
	static struct sockaddr_un sun = { .sun_family = AF_UNIX };
	
	s = socket(AF_UNIX, SOCK_STREAM, 0);
	if (s < 0) { perror("socket"); return; }

	strncpy(sun.sun_path, path, sizeof(sun.sun_path));
	if (connect(s, (struct sockadr *)&sun, sizeof(sun)) < 0) {
		perror("connect");
		return;
	}
	
	if (write(s, msg, len) != len)
		perror("write");
	close(s);
}


static void server(int s) {
	struct sockaddr_un nsun;
	int len = sizeof(nsun);
	int ns = accept(s, (struct sockaddr *) &nsun, &len);
	int cc;
	char buf[3];

	if (ns < 0) {
		perror("accept");
		return;
	}

	while ((cc = read(ns, buf, sizeof(buf))) > 0) 
		write(1, buf, cc);

	putchar('\n');

	if (cc < 0)
		perror("read request");

	if (write(ns, "done", 4) != 4)
		perror("write response");
}

int main(int argc, char **argv) {
	int s;
	static struct sockaddr_un sun = { .sun_family = AF_UNIX };

	if (argc != 3) { fprintf(stderr, "Usage: udtest name message\n"); return 1; }
	
	s = socket(AF_UNIX, SOCK_STREAM, 0);
	if (s < 0) { perror("socket"); return 1; }

	strncpy(sun.sun_path, argv[1], sizeof(sun.sun_path));
	if (bind(s, (struct sockaddr *) &sun, sizeof(sun)) < 0) {
		perror("bind"); return 1;
	}

	listen(s, 1);

	if (fork() == 0) {
		close(s);
		client(sun.sun_path, argv[2]);
	} else {
		server(s);
		close(s);
		unlink(sun.sun_path);
	}
	return 0;
}

Re: AF_UNIX sockets: strange behaviour

[Alex Riesen]

On Wed, 17 Nov 2004 16:29:14 +0100 (CET), Catalin Drula
<catalin.drula@imag.fr> wrote:
> I have a small application that communicates over Bluetooth. I use
> connection-oriented UNIX domain sockets (AF_UNIX, SOCK_STREAM) to
> communicate between the applications's threads. When reading from
> one of these sockets, I get a strange behaviour: if I read all the
> bytes that are available (13, in this case) all at once, it's fine;
> however, if I try to read in two smaller batches (say, first time
> 6, and second time 7), the first read returns (with the 6 bytes), but
> the second read never returns.

2.6.9, works. Could you post your code?

#include <unistd.h>
#include <stdio.h>
#include <sys/socket.h>

int main(int argc, char **argv)
{
    char buf[13];
    int s[2];
    if ( socketpair(AF_UNIX, SOCK_STREAM, 0, s) < 0 )
    {
	perror("socketpair");
	return 1;
    }
    if ( fork() == 0 )
    {
	close(s[0]);
	write(s[1], "023456789012", 13);
	read(s[1], buf, 1); /* wait for parent */
    }
    else
    {
	close(s[1]);
	if ( read(s[0], buf, 6) != 6 )
	    perror("6");
	if ( read(s[0], buf, 7) != 7 )
	    perror("6");
	close(s[0]);
    }
    return 0;
}

Re: AF_UNIX sockets: strange behaviour

[Catalin Drula]

On Wed, 17 Nov 2004, Alex Riesen wrote:

> On Wed, 17 Nov 2004 16:29:14 +0100 (CET), Catalin Drula
> <catalin.drula@imag.fr> wrote:
> > I have a small application that communicates over Bluetooth. I use
> > connection-oriented UNIX domain sockets (AF_UNIX, SOCK_STREAM) to
> > communicate between the applications's threads. When reading from
> > one of these sockets, I get a strange behaviour: if I read all the
> > bytes that are available (13, in this case) all at once, it's fine;
> > however, if I try to read in two smaller batches (say, first time
> > 6, and second time 7), the first read returns (with the 6 bytes), but
> > the second read never returns.
>
> 2.6.9, works. Could you post your code?

Nevermind. It was actually a bug in my code. I apologize for wasting
your time.

Catalin

Re: AF_UNIX sockets: strange behaviour

[linux-os]

On Wed, 17 Nov 2004, Catalin Drula wrote:

>> Dick Johnson wrote:
>>
>>> Catalin Drula wrote:
>>>
>>> - there is a skb in the sk_receive_queue with a len of 13
>>> - 6 bytes are read from it
>>> - a skb with the remaining 7 bytes is requeued in sk_receive_queue
>>> - on the next call to unix_stream_recvmsg, the sk_receive_queue is
>>>  empty (!)
>>>
>>> Thus, this confirms the behaviour observed from userspace. Is this a
>>> bug? Who could be removing the skb from the receive_queue?
>>
>> If you need STREAM behavior I think you need to use recv(),
>> recvfrom(),or read().
>>
>> If you use recvmsg(), the "message" will be removed even it you
>> haven't read it all. Note in the 'man' page description:
>> "If the a message is too long to fit in the supplied buffer, excess
>> bytes may be discarded depending upon the type of socket the message
>> is being received from...
>
> At first I used read() and then I tried recv() as well.
>
> Catalin

My reading of the specs is that read() damn-well never throw
away anything that hasn't been read yet. If it does, then
it's a bug. That said, the bug may never be fixed because
(1) people may deny it's a bug, or (2) programs may rely
on it.

FYI, it the first condition, I've seen these kinds of bugs
mysteriously fixed later on even though many people denied they
were bugs.

I suggest you supply a big enough buffer to read the hole thing
and then tear it apart at the application level.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.9 on an i686 machine (5537.79 BogoMips).
  Notice : All mail here is now cached for review by John Ashcroft.
                  98.36% of all statistics are fiction.

Re: AF_UNIX sockets: strange behaviour

[Catalin Drula]

> Dick Johnson wrote:
>
> > Catalin Drula wrote:
> >
> > - there is a skb in the sk_receive_queue with a len of 13
> > - 6 bytes are read from it
> > - a skb with the remaining 7 bytes is requeued in sk_receive_queue
> > - on the next call to unix_stream_recvmsg, the sk_receive_queue is
> >  empty (!)
> >
> > Thus, this confirms the behaviour observed from userspace. Is this a
> > bug? Who could be removing the skb from the receive_queue?
>
> If you need STREAM behavior I think you need to use recv(),
> recvfrom(),or read().
>
> If you use recvmsg(), the "message" will be removed even it you
> haven't read it all. Note in the 'man' page description:
> "If the a message is too long to fit in the supplied buffer, excess
> bytes may be discarded depending upon the type of socket the message
> is being received from...

At first I used read() and then I tried recv() as well.

Catalin