send(), sendmsg(), sendto() not thread-safe

send(), sendmsg(), sendto() not thread-safe

[Mark A Smith]

I discovered that in some cases, send(), sendmsg(), and sendto() are not
thread-safe. Although the man page for these functions does not specify
whether these functions are supposed to be thread-safe, my reading of the
POSIX/SUSv3 specification tells me that they should be. I traced the
problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
up a small page to describe in more detail my findings. You can find it at:
http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .

Thanks,
Mark A. Smith

PS. I am using the term "thread" in the general sense, this is a problem
independent of pthreads, etc. The problem occurs when two processes
(whether or not they share an address space) send on the same socket (and
some other low-resource conditions exist).

Re: send(), sendmsg(), sendto() not thread-safe

[H. Peter Anvin]

Followup to:  <OFE8460E54.0C8D85D8-ON8525716F.0074F22F-8825716F.0076D537@us.ibm.com>
By author:    Mark A Smith <mark1smi@us.ibm.com>
In newsgroup: linux.dev.kernel
>
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
> 
> Thanks,
> Mark A. Smith
> 
> PS. I am using the term "thread" in the general sense, this is a problem
> independent of pthreads, etc. The problem occurs when two processes
> (whether or not they share an address space) send on the same socket (and
> some other low-resource conditions exist).
> 

User error.  Writes onto a streaming socket (or a pipe) are
thread-safe, *but not necessarily atomic*, if the size exceeds PIPE_BUF.

If you want atomicity you either have to do your own locking, or use a
DGRAM or SEQPACKET socket.

	-hpa

Re: send(), sendmsg(), sendto() not thread-safe

[David S. Miller]

From: Mark A Smith <mark1smi@us.ibm.com>
Date: Mon, 15 May 2006 14:39:06 -0700

> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .

I don't understand why the desire is so high to ensure that
individual threads get "atomic" writes, you can't even ensure
that in the general case.

Only sloppy programs that don't do their own internal locking hit into
issues in this area.

>From your findings, the vast majority of systems you investigated do
not provide "atomic" thread safe write semantics over TCP sockets.
And frankly, BSD defines BSD socket semantics here not some wording in
the POSIX standards.

Finally, this discussion belongs on the networking development mailing
list, netdev@vger.kernel.org, not linux-kernel.

RE: send(), sendmsg(), sendto() not thread-safe

[David Schwartz]

> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can 
> find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .
> 
> Thanks,
> Mark A. Smith

	You are confusing thread-safety with atomicity.

	DS

Re: send(), sendmsg(), sendto() not thread-safe

[Christoph Hellwig]

On Mon, May 15, 2006 at 02:39:06PM -0700, Mark A Smith wrote:
> 
> I discovered that in some cases, send(), sendmsg(), and sendto() are not
> thread-safe. Although the man page for these functions does not specify
> whether these functions are supposed to be thread-safe, my reading of the
> POSIX/SUSv3 specification tells me that they should be. I traced the
> problem to tcp_sendmsg(). I was very curious about this issue, so I wrote
> up a small page to describe in more detail my findings. You can find it at:
> http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html .

Please don't confuse thread safety with atomicy, thanks.