Re: epoll and closed file descriptors

[Bryan Donlan <bdonlan@gmail.com>]

2009/9/16 Gilad Benjamini <gilad@altornetworks.com>:
>> Subject: Re: epoll and closed file descriptors
>>
>> On Wed, Sep 16, 2009 at 8:40 PM, Gilad Benjamini
>> <gilad@altornetworks.com> wrote:
>> > Davide wrote:
>> >> On Wed, 16 Sep 2009, Gilad Benjamini wrote:
>> >>
>> >> > I would, but epoll is preventing me from doing so.
>> >> > Early in sys_epoll_ctl there are these lines
>> >> >
>> >> >   file = fget(epfd);
>> >> >   if (!file)
>> >> >     goto error_return;
>> >> >
>> >> > Leaving me in a kind of dead lock
>> >>
>> >> The 'epfd' in there, is the _epoll fd_, which, if fget() fails,
>> means
>> >> you
>> >> close it.
>> >> You see likely failing the 'tfile = fget(fd)' (of course, you closed
>> >> it),
>> >> so if someone else keeps the socket open and you have no chance in
>> >> telling
>> >> it to drop it (really?), you need to remove the socket from the set
>> >> before
>> >> closing it.
>> >>
>> >>
>> >>
>> >> - Davide
>> >
>> > My bad. I meant to quote the line that you mentioned.
>> > I agree that the right thing to do is to remove the fd from epoll
>> before
>> > closing it.
>> > However, due to the way curl works, I cannot do that. Changing the
>> curl code
>> > doesn't seem trivial.
>> >
>> > Regardless, I still don't see how the kernel got into this situation,
>> and if
>> > this situation is valid, why it doesn't bail out of it.
>>
>> epoll references the underlying file object; the fd is used _only_ to
>> obtain this file object, and then never used again. Determining when
>> the fd goes away then requires iterating over all fds, and since epoll
>> was designed to avoid doing exactly that, it isn't an acceptable
>> solution.
>
> Regarding bailing out of the situation, I see the logic in your answer.
> What about the first part ? Any ideas how the kernel actually got into that
> tight spot ?
> Looking into the code I can't find a path that can lead into this situation.

Userspace passes in a fd that is unused (closed). Kernel can't find
the file object corresponding to the fd (because the fd went away when
it wasn't looking), so it says to userspace, "Sorry, no such file!".
And it's completely correct.

So, the basic problem is userspace becomes unable to refer to the file
object. The kernel's doing just fine; it's not in a "tight spot" at
all. It's perfectly happy to refer to the file object by a direct
pointer to it - it's just userspace is unable to tell the kernel to
remove it from the epoll, because the only name userspace had for it
has been removed.