Request for O_CLOFORK, akin to O_CLOEXEC.

Request for O_CLOFORK, akin to O_CLOEXEC.

[Ralph Corderoy]

Hi,

Please keep me CC'd.

A multi-threaded program's thread A wants to create a new executable
file with open(O_CLOEXEC), write, close, and then fork and exec it.
Meanwhile, another thread B forks before A's close, carrying a reference
to the open file description with it, thus A's execve fails with
ETXTBUSY because the executable is still open for writing.

    open(O_CLOEXEC)
    write()
                        fork()
    close()
    fork()
    execve(foo)
    -1 ETXTBUSY
                        execve(bar)

This is being seen in the wild, repeatedly.  It bit Go five years ago
and was workaround then with sleeps and retries.  Java has a bug report
from 2014 that has never been resolved.  Recently, Go developers
debugged the same problem, this time from testing where many binaries
want to be written and executed;  sleeping would drag down throughput.
Google shows quite a few other ETXTBUSY reports where folks think
they're closing the FD, but don't cotton on other forks are occasionally
happening at just the wrong moment.

Userspace coordination between threads to lock around close(), dup2(),
and dup3(), that close an FD, and then manually closing FD_CLOEXEC ones
before fork can be proposed, e.g.
https://github.com/golang/go/issues/22315#issuecomment-337597575 but
they assume that program has complete control over all FD closing and
forks;  not true with libraries also kicking off threads.

Just as O_CLOEXEC was needed, it would seem O_CLOFORK is required that
closes such FDs on fork, stopping another thread unwittingly carrying
the FD with it.  It has been independently mooted before without
explaining why it was useful.

    [PATCH] fs: add FD_CLOFORK and O_CLOFORK
    https://lists.gt.net/linux/kernel/1375919

Feedback wanted.

-- 
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy