Re: [RFC] per-cpu preempt_count

[Peter Zijlstra <peterz@infradead.org>]

On Mon, Aug 12, 2013 at 11:53:25AM -0700, Linus Torvalds wrote:
> On Mon, Aug 12, 2013 at 10:51 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> >
> > So we would have code looking something like:
> >
> >         decl %fs:preempt_count
> >         jnz 1f
> >         cmpb $0,%fs:need_resched
> >         je 1f
> >         call __preempt_schedule
> > 1:
> >
> > It's a nontrivial amount of code, but would seem a fair bit better than
> > what we have now, at least.
> 
> Well, we currently don't even bother checking the preempt count at
> all, and we just optimistically assume that we don't nest in the
> common case. The preempt count is then re-checked in
> __preempt_schedule, I think.
> 
> Which sounds like a fair approach.
> 
> So the code would be simplified to just
> 
>          decl %fs:preempt_count
>          cmpb $0,%fs:need_resched
>          jne .. unlikely branch that calls __preempt_schedule
> 
> which is not horrible. Not *quite* as nice as just doing a single
> "decl+js", but hey, certainly better than what we have now.

OK, so doing per-cpu need_resched is a trivial patch except for the
mwait side of things. Then again, Arjan already asked for per-cpu
need_resched specifically for mwait so we might as well do that.

The only complication is that IIRC Arjan wants to stagger the mwait
cache-lines and we would very much like our preempt_count and
need_resched (and possible some other __switch_to related things) in the
same cacheline.

Afaict that'll yield a double indirect again :/