Re: [RFC 07/13] sched: Reduce stack size requirements in kernel/sched.c

[Peter Zijlstra <peterz@infradead.org>]

On Mon, 2008-09-08 at 07:54 -0700, Mike Travis wrote:
> Peter Zijlstra wrote:

> > get_online_cpus() can sleep, but you just disabled preemption with those
> > get_cpumask_var() horribles!
> > 
> > Couldn't be arsed to look through the rest, but I really hate this
> > cpumask_ptr() stuff that relies on disabling preemption.
> > 
> > NAK
> 
> Yeah, I really agree as well.  But I wanted to start playing with using
> cpumask_t pointers in some fairly straight forward manner.  Linus's and
> Ingo's suggestion to just bite the bullet and redefine the cpumask_t 
> would force a lot of changes to be made, but perhaps that's really the
> way to go.

I much prefer that approach!

> As to obtaining temp cpumask_t's (both early and late), perhaps a pool of
> them would be better?  I believe it could be done similar to alloc_bootmem
> (but much simpler), and I don't think there's enough nesting to require a
> very large pool.  (4 was the largest depth I could find in io_apic.c.)  Of
> course, with preemption enabled then other problems arise...
> 
> One other really big use was for the "allbutself" cpumask in the send_IPI
> functions.  I think here, preemption is ok because the ownership of the
> cpumask temp is very short lived.

The thing is, you add serialization requirements (be it preempt_disable,
or a lock for some preemptable form) to code that didn't had any for a
case that hardly anyone will ever encounter in real life - I mean,
really, who has 4096 cpus?

Stuffing the cpumap_t in an already existing structure that has suitable
serialization requirements is of course the preferred situation, but
lacking that a dynamic cpumap_t is best, since it keeps the references
local, and thus doesn't add requirements to the existing code.

You could also consider adding 1 cpumap_t to task_struct and use that as
temporary scratch pad - but seeing you needed at least 4 that might not
be a feasible solution either.