Any problem if softirq are done in a interrupt context (IRQ stack)?

Any problem if softirq are done in a interrupt context (IRQ stack)?

[Zefang.Wang]

Hello all!

Kernel version : 2.6.18
Arch : i386

With the following conditions,  it is possible that softirqs are
executed in a interrupt context rather than process one
1)   CONFIG_4KSTACKS  ----> ON
That means the dedicated IRQ stack is used for hardirq handler

2)   there exist some Hard IRQ which allows interupt enabled when its
handler being executed.
That means a possibility that a HARD IRQ handler is interrupted by
another one.

3)  CONFIG_LOCKDEP  ---> OFF
Instruction sti will be executed by local_irq_enable_in_hardirq()


Let's suppose the following situation.
1)  A process is running without local irq nor bottom half disabled.
2)  A hardware interrupt happened.
3)  After saving context in process kernel stack,   it switch to irq
stack. 
      But notice :  the preempt_count in irq stack will be zero, because
do_irq does not add HARDIRQ_OFFSET to the preept_count. 
      (anyone tell me the reason?)

	if (curctx != irqctx) {
		int arg1, arg2, ebx;

		/* build the stack frame on the IRQ stack */
		isp = (u32*) ((char*)irqctx + sizeof(*irqctx));
		irqctx->tinfo.task = curctx->tinfo.task;
		irqctx->tinfo.previous_esp = current_stack_pointer;

		/*
		 * Copy the softirq bits in preempt_count so that the
		 * softirq checks work in the hardirq context.
		 */
		irqctx->tinfo.preempt_count =
			(irqctx->tinfo.preempt_count & ~SOFTIRQ_MASK) |
			(curctx->tinfo.preempt_count & SOFTIRQ_MASK);


4)  then __do_irq is called, and handle_irq_event is called. Before
that,  local irq is enabled because the interrupt allow it.
5)  during the execution of the hardirq actions,  another hardware
(depth 2 interrurpt) interrupt happened.
6)  SAVE context,  and then hardirq handler,  during the handler,  some
softirq is marked
7)  when depth 2 interrrupt call irq_exit(),  surely do_softirq will be
called because in_interrupt return a FALSE.
     In this point, the stack is still irq stack.  

I don't know whether it cause some problem,  for example, if some
softirq need to make a flag in process control block.
Another problem is that softirq handling should have a lower prioirty
than hard irq, right?

Thanks for your attention and help.

Regards
Zefang

Re: Any problem if softirq are done in a interrupt context (IRQ stack)?

[Björn Steinbrink]

On 2007.01.03 16:23:28 +0800, Zefang.Wang@nokia.com wrote:
> Hello all!
> 
> Kernel version : 2.6.18
> Arch : i386
> 
> With the following conditions,  it is possible that softirqs are
> executed in a interrupt context rather than process one
> 1)   CONFIG_4KSTACKS  ----> ON
> That means the dedicated IRQ stack is used for hardirq handler
> 
> 2)   there exist some Hard IRQ which allows interupt enabled when its
> handler being executed.
> That means a possibility that a HARD IRQ handler is interrupted by
> another one.
> 
> 3)  CONFIG_LOCKDEP  ---> OFF
> Instruction sti will be executed by local_irq_enable_in_hardirq()
> 
> 
> Let's suppose the following situation.
> 1)  A process is running without local irq nor bottom half disabled.
> 2)  A hardware interrupt happened.
> 3)  After saving context in process kernel stack,   it switch to irq
> stack. 
>       But notice :  the preempt_count in irq stack will be zero, because
> do_irq does not add HARDIRQ_OFFSET to the preept_count. 
>       (anyone tell me the reason?)

Because irq_ctx_init() initializes the preempt count to HARDIRQ_OFFSET,
the value is already correct.

> 
> 	if (curctx != irqctx) {
> 		int arg1, arg2, ebx;
> 
> 		/* build the stack frame on the IRQ stack */
> 		isp = (u32*) ((char*)irqctx + sizeof(*irqctx));
> 		irqctx->tinfo.task = curctx->tinfo.task;
> 		irqctx->tinfo.previous_esp = current_stack_pointer;
> 
> 		/*
> 		 * Copy the softirq bits in preempt_count so that the
> 		 * softirq checks work in the hardirq context.
> 		 */
> 		irqctx->tinfo.preempt_count =
> 			(irqctx->tinfo.preempt_count & ~SOFTIRQ_MASK) |
> 			(curctx->tinfo.preempt_count & SOFTIRQ_MASK);
> 
> 
> 4)  then __do_irq is called, and handle_irq_event is called. Before
> that,  local irq is enabled because the interrupt allow it.
> 5)  during the execution of the hardirq actions,  another hardware
> (depth 2 interrurpt) interrupt happened.
> 6)  SAVE context,  and then hardirq handler,  during the handler,  some
> softirq is marked

Note that curctx is equal to irqctx in this case, so we stay with the
hardirq context and the irq_enter() in do_IRQ() does the right thing.
The preempt count is incremented to HARDIRQ_OFFSET+1.

> 7)  when depth 2 interrrupt call irq_exit(),  surely do_softirq will be
> called because in_interrupt return a FALSE.
>      In this point, the stack is still irq stack.

No, irq_exit() will decrement the preempt count back to HARDIRQ_OFFSET,
so in_interrupt() will return true.

And the irq_exit() call for the first irq will actually happen in
process context, so a) the hard irq context's preempt count will stay at
HARDIRQ_OFFSET and b) the hardirq count in the process context will go
back to 0 (it was raised to 1 by the initial irq_enter() call).

HTH
Björn

Re: Any problem if softirq are done in a interrupt context (IRQ stack)?

[Björn Steinbrink]

[Re-added lkml to the CC list, please don't drop anything from CC]

On 2007.01.03 17:39:48 +0800, Zefang.Wang@nokia.com wrote:
> Hi!
> 
> Thanks very much for your clear explanation !
> 
> I have another question about irq_exit(), hope you can help me.
> 
> void irq_exit(void)
> {
> 	account_system_vtime(current);          
> 	trace_hardirq_exit();
> 	sub_preempt_count(IRQ_EXIT_OFFSET);   
>            ====================================================================
> 		Here, IRQ_EXIT_OFFSET is defined as (HARDIRQ_OFFSET-1),
> 		so the purpose seems that it try to avoid the current process
> 		from being switched-out druing do_softirq()?
> 		And,  if the preempt_count is not zero,  then softirq for 
> 		timer interrupt can set  _TIF_NEED_RESCHED flag to current
> 		process?
> 		What happened if the above sentence is changed to
> 		sub_preept_count(HARDIRQ_OFFSET)?
> 
> 	if (!in_interrupt() && local_softirq_pending())
> 		invoke_softirq();
> 	preempt_enable_no_resched();
>              ==============================================
> 	 The remaining 1 is decremented here.
> }

I can't really help you with that one. I'd assume that you need to make
sure that the current process context is kept while the softirq is
running. Could be, that otherwise the process gets preempted and the
stored process context would be assigned to the new process or
something like that, but I'm just guessing wildly here.
Note that IRQ_EXIT_OFFSET is defined as HARDIRQ_OFFSET if preemption is
disabled, so that's probably key here, I just don't know what kind of
havoc preempting would cause here ;)

Björn

Re: Any problem if softirq are done in a interrupt context (IRQ stack)?

[Steven Rostedt]

On Wed, 2007-01-03 at 11:43 +0100, Björn Steinbrink wrote:
> [Re-added lkml to the CC list, please don't drop anything from CC]

yes, since I would not have replied to this ;)

> 
> On 2007.01.03 17:39:48 +0800, Zefang.Wang@nokia.com wrote:
> > Hi!
> > 
> > Thanks very much for your clear explanation !
> > 
> > I have another question about irq_exit(), hope you can help me.
> > 
> > void irq_exit(void)
> > {
> > 	account_system_vtime(current);          
> > 	trace_hardirq_exit();
> > 	sub_preempt_count(IRQ_EXIT_OFFSET);   
> >            ====================================================================
> > 		Here, IRQ_EXIT_OFFSET is defined as (HARDIRQ_OFFSET-1),
> > 		so the purpose seems that it try to avoid the current process
> > 		from being switched-out druing do_softirq()?
> > 		And,  if the preempt_count is not zero,  then softirq for 
> > 		timer interrupt can set  _TIF_NEED_RESCHED flag to current
> > 		process?
> > 		What happened if the above sentence is changed to
> > 		sub_preept_count(HARDIRQ_OFFSET)?
> > 
> > 	if (!in_interrupt() && local_softirq_pending())
> > 		invoke_softirq();
> > 	preempt_enable_no_resched();
> >              ==============================================
> > 	 The remaining 1 is decremented here.
> > }
> 
> I can't really help you with that one. I'd assume that you need to make
> sure that the current process context is kept while the softirq is
> running. Could be, that otherwise the process gets preempted and the
> stored process context would be assigned to the new process or
> something like that, but I'm just guessing wildly here.
> Note that IRQ_EXIT_OFFSET is defined as HARDIRQ_OFFSET if preemption is
> disabled, so that's probably key here, I just don't know what kind of
> havoc preempting would cause here ;)

It would delay the processing of the softirqs for a full tick or more.

Note that changing to sub_preempt_count(HARDIRQ_OFFSET) is not enough,
you would then need to add a preempt_disable() right afterwards, which
would just be an extra subtraction that is avoided in the current
implementation.

But the big thing is that the softirqs should be handled as soon as they
can be.

-- Steve