Re: [take25 1/6] kevent: Description.

["Miquel van Smoorenburg" <miquels@cistron.nl>]

In article <45663298.7000108@redhat.com>,
Ulrich Drepper  <drepper@redhat.com> wrote:
>Jeff Garzik wrote:
>> Considering current designs, it seems more likely that a single thread 
>> polls for socket activity, then dispatches work.  How often do you 
>> really see in userland multiple threads polling the same set of fds, 
>> then fighting to decide who will handle raised events?
>> 
>> More likely, you will see "prefork" (start N threads, each with its own 
>> ring) or a worker pool (single thread receives events, then dispatches 
>> to multiple threads for execution) or even one-thread-per-fd (single 
>> thread receives events, then starts new thread for handling).
>
>No, absolutely not.  This is exactly not what should/is/will happen.
>
>You create worker threads to handle to work for the entire program. 
>Look at something like a web server.  When creating several queues, how 
>do you distribute all the connections to the different queues?  To 
>ensure every connection is handled as quickly as possible you stuff them 
>all in the same queue and then have all threads use this one queue. 
>Whenever an event is posted a thread is woken.  _One_ thread.  If two 
>events are posted, two threads are woken.  In this situation we have a 
>few atomic ops at userlevel to make sure that the two threads don't pick 
>the same event but that's all there is wrt "fighting".

What you really want is if one thread is able to do all the work,
only keep that one thread busy. Only wake up other threads when
the currently running threads cannot handle the load.

Say you have 8 threads blocked in kevent_wait(). One or more events
become available. You wake one thread, and let it run.

If the one thread has done its work and returns to kevent_wait()
before its timeslice has run out, deliver the next event(s) (which
may already be outstanding) to the same thread, don't wake another one.

If the running thread blocks on say disk i/o, or its timeslice
runs out, the scheduler runs and wakes another thread that is
waiting in kevent_wait().

Mike.