full no_hz: how works arch_irq_work_raise() with tick off on generic implementation

full no_hz: how works arch_irq_work_raise() with tick off on generic implementation

[Sebastian Andrzej Siewior]

I just stumbled over this in -RT. The code:

 87 bool irq_work_queue(struct irq_work *work)
 88 {
…
 96         /* If the work is "lazy", handle it from next tick if any */
 97         if (work->flags & IRQ_WORK_LAZY) {
 98                 if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
 99                     tick_nohz_tick_stopped())
100                         arch_irq_work_raise();
101         } else {
102                 if (llist_add(&work->llnode, this_cpu_ptr(&raised_list)))
103                         arch_irq_work_raise();
104         }
…
109 }

so what I asked myself: What happens if the CPU is in NO_HZ mode with
the tick off and you invoke arch_irq_work_raise() on an architecture
without a special IRQ_WORK interrupt?
And why you don't this tick_nohz_tick_stopped() check for the
raised_list?
Wouldn't it be a easier for your full NO_HZ code to simply run in the
hardirq exit path instead of using irq_work?

Sebastian

Re: full no_hz: how works arch_irq_work_raise() with tick off on generic implementation

[Frederic Weisbecker]

On Mon, Mar 09, 2015 at 03:09:17PM +0100, Sebastian Andrzej Siewior wrote:
> I just stumbled over this in -RT. The code:
> 
>  87 bool irq_work_queue(struct irq_work *work)
>  88 {
> …
>  96         /* If the work is "lazy", handle it from next tick if any */
>  97         if (work->flags & IRQ_WORK_LAZY) {
>  98                 if (llist_add(&work->llnode, this_cpu_ptr(&lazy_list)) &&
>  99                     tick_nohz_tick_stopped())
> 100                         arch_irq_work_raise();
> 101         } else {
> 102                 if (llist_add(&work->llnode, this_cpu_ptr(&raised_list)))
> 103                         arch_irq_work_raise();
> 104         }
> …
> 109 }
> 
> so what I asked myself: What happens if the CPU is in NO_HZ mode with
> the tick off and you invoke arch_irq_work_raise() on an architecture
> without a special IRQ_WORK interrupt?

NO_HZ full can't run on such architecture. See the arch_irq_work_has_interrupt()
test on tick_nohz_init().

> And why you don't this tick_nohz_tick_stopped() check for the
> raised_list?

Because if the IRQ_WORK_LAZY flag isn't set, we want the interrupt to always
fire quickly. We don't want to wait for the next tick unless we have IRQ_WORK_LAZY.
Lazy works can wait a bit.

Now if the work is IRQ_WORK_LAZY and the tick is stopped, we need to trigger an
IPI because the next tick maybe too far ahead.

> Wouldn't it be a easier for your full NO_HZ code to simply run in the
> hardirq exit path instead of using irq_work?

Note only the tick wakeup happens from the IPI callback. The tick shutdown always
happens from irq exit.

But I'm working on doing the wakeup as well from irq exit.

I plan to piggyback on scheduler events instead of playing with IPIs, although
we'll still need them for perf and posix timers at least when those subsystems need
periodic tick.

> Sebastian