Re: [PATCH] stack and rcu interaction bug in smp_call_function_mask()

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Monday 11 August 2008 14:26, Jeremy Fitzhardinge wrote:
> Nick Piggin wrote:
> > Nice debugging work.
> >
> > I'd suggest something like the attached (untested) patch as the simple
> > fix for now.
> >
> > I expect the benefits from the less synchronized, multiple-in-flight-data
> > global queue will still outweigh the costs of dynamic allocations. But
> > if worst comes to worst then we just go back to a globally synchronous
> > one-at-a-time implementation, but that would be pretty sad!
>
> What if we went the other way and strictly used queue-per-cpu?  It means
> multicast would require multiple enqueueing operations, which is a bit
> heavy, but it does make dequeuing and lifetime management very simple...

Well that's implemented with the optimized call-single code of course,
so it could be used to implement the masked calls...

I had wanted to look into finding a good cutoff point and use the
percpu queues for light weight masks, and the single global queue for
larger ones.

Queue per cpu is not going to be perfect, though. In the current
implementation, you would need a lot of data structures. You could
alleviate this problem by using per CPU vectors rather than lists,
but then you get the added problem of resource starvation at the
remote end too.

For heavy weight masks on large systems, the single queue I'd say
will be a win. But I never did detailed measurements, so I'm open
to be proven wrong.