Re: [PATCH 2/2] irq: Cleanup context state transitions in irq_exit()

[Frederic Weisbecker <fweisbec@gmail.com>]

On Fri, Feb 22, 2013 at 01:08:17PM -0800, Linus Torvalds wrote:
> On Fri, Feb 22, 2013 at 7:06 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> >>
> >> I prefer to let you guys have the final word on this patch. Whether you
> >> apply it or not, I fear I'll never be entirely happy either way :)
> >> That's the sad fate of dealing with circular dependencies...
> >
> > plus the butt ugly softirq semantics or the lack thereof ...
> 
> The softirq semantics are perfectly fine. Don't blame softirq for the
> fact that irq_exit() has had shit-for-brains for a long time.
> 
> Just move the whole "invoke_softirq()" thing down to *after* the
> tick_nohz_irq_exit() stuff. And that "wakeup_softirqd()" is garbage
> too, since the whole thing should only be used for the
> "force_irqthreads" case (which invoke_softirq()" got right.

The issue here is that softirqs may change some timers state (ie: enqueue
new timer list, dequeue, modify...), so tick_nohz_irq_exit() really need to be
called after irq tail time softirq processing in order to propagate the timer
changes to the hardware.

But tick_nohz_irq_exit() may trigger the timer softirq itself.

Another issue is rcu_irq_exit(). softirqs may make use of RCU read side critical
sections, so rcu_irq_exit() has to be called after softirq processing.
Meanwhile it seems that RCU may raise its softirq from rcu_irq_exit(). Perhaps
Paul could confirm that.

So we have a circular dependency problem here. I guess the sanest solution is to
rework tick_nohz_irq_exit() and rcu_irq_exit() to remove the use of softirqs there.
And even trigger a warning if that happens. Or we need some way to force raise
through ksoftirqd but that doesn't sound like a proper long term solution.
 
> And get rid of that final
> 
>  #ifndef __ARCH_IRQ_EXIT_IRQS_DISABLED
>         local_irq_restore(flags);
>  #endif
> 
> because even if the architecture enters irq_exit() with interrupts
> enabled, we should damn well exit with them disabled so that there are
> no races with new recursive interrupts (other than the ones that
> wakeup_softirqd already handled).

Agreed, we always return from irq_exit() with irqs disabled as long as
we processed pending softirqs anyway.