Re: [patch] drivers: wait for threaded probes between initcall levels

[Arjan van de Ven <arjan@infradead.org>]

> I admit the complexity is a bit high, but since the maximum nesting  
> is bounded by the complexity of the hardware and the number of  
> busses, and the maximum memory-allocation is strictly limited in the  
> single-threaded case this could allow 64-way systems to probe all  
> their hardware an order of magnitude faster than today without  
> noticeably impacting an embedded system even in the absolute worst case.


how much of this complexity goes away if you consider the
scanning/probing as a series of "work elements", and you end up with a
queue of work elements that threads can pull work off one at a time (so
that if one element blocks the others just continue to flow). If you
then find, say, a new PCI bus you just put another work element to
process it at the end of the queue, or you process it synchronously. Etc
etc.

All you need to scale then is the number of worker threads on the
system, which should be relatively easy to size....
(check every X miliseconds if there are more than X outstanding work
elements, if there are, spawn one new worker thread if the total number
of worker threads is less than the system wide max. Worker threads die
if they have nothing to do for more than Y miliseconds)

Oh and... we have a concept for this already, or at least mostly, via
the work queue mechanism :)