Re: [PATCH RT] Only run softirqs from the irq thread if the irq affinity is set to 1 CPU

[Darren Hart <dvhltc@us.ibm.com>]

On Wednesday 08 August 2007 13:42:32 you wrote:
> Ingo and Thomas,
>
> John and I have been discussing all the "run softirq from IRQ thread"
> lately and discovered something nasty.
>
> Now it is a nice optimization to run softirqs from the IRQ thread, but
> it may not be feasible because of the semantics of the IRQ thread
> compared with the softirq thread. Namely, the softirq thread is bound to
> a single CPU and the IRQ thread is not.
>
> We use to think that it would be fine to simply bind an IRQ thread to a
> single CPU, either at the start of the IRQ thread code, or just while it
> is running the softirq code.  But this has a major flaw as John Stultz
> discovered.
>
> If a RT hog that is of higher priority than the IRQ thread preempts the
> IRQ thread while it is bound to the CPU (more likely with the latest
> code that always binds the IRQ thread to 1 CPU), then that IRQ is, in
> essence, masked.  That means no more actions will be taken place by that
> IRQ while the RT thread is running.  Normally, one would expect, that if
> the IRQ has its affinity set to all CPUS, if a RT thread were to preempt
> the IRQ thread and run for a long time, it would be expected that the
> IRQ thread would migrate to another CPU and finish. Letting more
> interrupts from the IRQ line in (remember that the IRQ line is masked
> until the IRQ finishes its handler).
>
> This patch will only run the softirq functions if the IRQ thread and the
> softirq thread have the same priority **and** the IRQ thread is already
> bound to a single CPU.  If we are running on UP or the IRQ thread is
> bound to a single CPU, we already have the possibility of having a RT
> hog starve the IRQ.  But we should not add that scenario when the IRQ
> thread has its affinity set to run on other CPUS that don't have RT hogs
> on them.

It seems to me that this patch will reduce the frequency of irqd/softirqd 
starvation, but the core problem still exists: softirq tasks can't migrate to 
other CPUs to perform their work if a higher priority task preempts them.  
I'm wondering if we want to keep special casing things to minimize the 
problem or not - seems to me the worst case is still the same - and isn't the 
worst case the only case that matters (for -rt)?

-- 
Darren Hart
IBM Linux Technology Center
Realtime Linux Team