Re: [RFC PATCH] poll(): add poll_wait_set_exclusive()

[Steven Rostedt <rostedt@goodmis.org>]

On Thu, 2010-10-07 at 14:07 -0400, Mathieu Desnoyers wrote:
> * Steven Rostedt (rostedt@goodmis.org) wrote:

> > I never mention affinity. As with trace-cmd, it assigns a process per
> > CPU, but those processes can be on any CPU that the scheduler chooses. I
> > could probably do it with a single process reading all the CPU fds too.
> > I might add that as an option.
> 
> Your scheme works fine because you have only one stream (and thus one fd) per
> cpu. How would you map that with many streams per cpu ?
> 
> Also, you might want to consider using threads rather than processes, to save
> the unnecessary VM swaps.

I thought about threads and could go back to them. I'd have to benchmark
to see the performance hit.

> 
> > 
> > > 
> > > > Or do you have an fd per event per CPU, in which case the threads should just
> > > > poll off of their own fds.
> > > 
> > > I have one fd per per-cpu buffer, but there can be many per-cpu buffers, each
> > > transporting a group of events. Therefore, I don't want to associate one thread
> > > per event group, because this would be a resource waste.  Typically, only a few
> > > per-cpu buffers will be very active, and others will be very quiet.
> > 
> > Lets not talk about threads, what about fds? I'm wondering why you have
> > many threads on the same fd?
> 
> That's because I have fewer threads than file descriptors. So I can choose to
> either:
> 
> 1) somehow assign each thread to many fds statically or
> 2) make each thread wait for data on all fds
> 
> Option (2) adapts much better to workloads where a lots of data would come from
> many file descriptors from a single CPU: all threads can collaboratively work to
> extract the data.

OK, this is what I wanted to know. Now as Linus suggested, go and ask
other application developers (web server developers?) if this is
something that could be useful for them? If you get positive feedback,
have them Cc LKML and we can go from there.

Or, if you see that this type of programming is being done in Apache,
and if you can demonstrate that adding this features helps Apache (or
some other popular program), that would also be of benefit.

-- Steve