SMP in 2.4

SMP in 2.4

[Dennis]

Does 2.4 have something similar to spl levels or does it still require the 
ridiculous MS-DOSish spin-locks to protect every bit of code?

DB

2.4.3 - Kernel panic

[Aleksey I Zavilohin]

Oops catced from screen on paper 8-)

ksymoops 2.3.7 on i686 2.4.3.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.3/ (default)
     -m /boot/System.map-2.4.3 (default)

Warning: You did not tell me where to find symbol information.  I will
assume that the log matches the kernel and modules that are running
right now and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc.  ksymoops -h explains the options.

CPU: 0
EIP: 0010: [<00000000>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010246
        eax: b8        ebx: 0 ecx: c122d9dc edx: 0
        esi: 0 edi: 0  ebp: c01f7fa4 esp: c01f7f3c
        ds: 18 es: 18  ss: 18
Stack:  c0119552 0 0 c02265a0 0 c01f7fa4 0 c010ad4c
        c01f7fac v011649f 0 c01163d8 0 1 c02265c0 e
        c01162df c02265c0 0 c0224900 0 c01081b1 c0105120 c01f6000
Call Trace: [<c0119552>] [<c010ad4c>] [<c011649f>] [<c01163d8>] [<c01162df>] [<c01081b1>] [<c0105120>]
        [<c0105120>] [<c0106e14>] [<c0105120>] [<c0105120>] [<c0100018>] [<c0105143>] [<c01051a9>] [<c0105000>]
        [<c0100191>]
Code: Bad EIP value

>>EIP; 00000000 Before first symbol
Trace; c0119552 <timer_bh+222/25c>
Trace; c010ad4c <timer_interrupt+80/e8>
Trace; c011649f <bh_action+1b/60>
Trace; c01163d8 <tasklet_hi_action+3c/60>
Trace; c01162df <do_softirq+3f/64>
Trace; c01081b1 <do_IRQ+a1/b0>
Trace; c0105120 <default_idle+0/28>
Trace; c0105120 <default_idle+0/28>
Trace; c0106e14 <ret_from_intr+0/20>
Trace; c0105120 <default_idle+0/28>
Trace; c0105120 <default_idle+0/28>
Trace; c0100018 <startup_32+18/139>
Trace; c0105143 <default_idle+23/28>
Trace; c01051a9 <cpu_idle+41/54>
Trace; c0105000 <init+0/110>
Trace; c0100191 <L6+0/2>


1 warning issued.  Results may not be reliable.

-- 
You are the only person to ever get this message.

Re: SMP in 2.4

[Mark Hahn]

Dennis is like a pie in the face: messy, unexpected, but trivial.


On Wed, 18 Apr 2001, Dennis wrote:

> Does 2.4 have something similar to spl levels or does it still require the 
> ridiculous MS-DOSish spin-locks to protect every bit of code?

Re: SMP in 2.4

[Richard B. Johnson]

On Wed, 18 Apr 2001, Dennis wrote:

> Does 2.4 have something similar to spl levels or does it still require the 
> ridiculous MS-DOSish spin-locks to protect every bit of code?
> 
> DB

This must be a Troll. MS-DOS didn't have spin-locks and, when you
have multiple CPUs with one interrupt controller, you don't have
much choice. You either use spin-locks or you Blue-Screen.

Since Linux doesn't have a "Blue-screen of death", it needs spin-
locks.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re: SMP in 2.4

[Matti Aarnio]

On Wed, Apr 18, 2001 at 11:08:22AM -0400, Dennis wrote:
> Does 2.4 have something similar to spl levels or does it still require the 
> ridiculous MS-DOSish spin-locks to protect every bit of code?

  Lets see -- (besides of MSDOS not having any sort of spinlocks), the
  spl() is something out of VAX series of machines, and it really works
  by presuming that there is some sort of priority leveling among irq
  sources.

  At i386 there is only one level of interrupt control, either you
  accept interrupts, or you don't.   It just doesn't scale very well.

  At FreeBSD site there is no documentation of how to handle interrupts,
  presumably that is "read the source, luke".

  Ah, found some man-page with its description..

  Essentially the supported function for  spl()  is:

	spin_lock_irqsave(&sp->lock, flags);

  along with its counterpart (  splx()  ):

	spin_unlock_irqrestore(&sp->lock, flags);

  These block interrupts at the LOCAL processor only (no interprocessor
  communication hazzle for it), and the spinlocks are for blocking
  interrupt processing by *other* processors at SMP systems.
  ( Linux does distribute interrupts these days to *all* processors
    when APIC is in use, and while it is fast and easy to block IRQ
    at local processor, getting other processors also to block IRQs
    is major slow thing...  )

  At UP systems the SAME functions are used, but their internal
  implementations are slightly different -- no SMP related spinlock
  is operated.

  For a model, see   drivers/net/eepro100.c

> DB

Re: SMP in 2.4

[Dennis]

At 02:05 PM 04/18/2001, Matti Aarnio wrote:
>On Wed, Apr 18, 2001 at 11:08:22AM -0400, Dennis wrote:
> > Does 2.4 have something similar to spl levels or does it still require the
> > ridiculous MS-DOSish spin-locks to protect every bit of code?
>
>   Lets see -- (besides of MSDOS not having any sort of spinlocks), the
>   spl() is something out of VAX series of machines, and it really works
>   by presuming that there is some sort of priority leveling among irq
>   sources.


I was referring to the infamous CLI/STI combinations that are more 
analogous to spinlocks than anything you are talking about. spl levels are 
clean and transparent and have been doing a very nice job in  helping to 
avoid race conditions in real unix systems for quite some time now.

db

Re: SMP in 2.4

[Mikulas Patocka]

> I was referring to the infamous CLI/STI combinations that are more 
> analogous to spinlocks than anything you are talking about. spl levels are 
> clean and transparent and have been doing a very nice job in  helping to 
> avoid race conditions in real unix systems for quite some time now.

It has nothing to do with smp ;)

spl levels are actually faster, because hardware interrupt locking
routines are poorly optimized in processors.

I looked at P-6 instruction timing table and found:

PUSHF	16 upos
POPF	17 uops
CLI	9 uops
STI	17 uops

I think soft interrupt locks like this would be better (at least on i386):

cli:
	movb $0, intr_lock

sti:
	movb $1, intr_lock
	testb $1, intr_pending
	jnz somewhere_away_to_handle_defered_interrupt

save_flags:
	movb intr_lock, %al

restore_flags:
	movb %al, intr_lock
	testb %al, intr_pending
	jnz somewhere_away_to_handle_defered_interrupt

And - of course - interrupt checks intr_lock in its entry and if it is
zero, sets intr_pending and exits immediatelly.

Mikulas

Re: SMP in 2.4

[Alan Cox]

> 	testb %al, intr_pending
> 	jnz somewhere_away_to_handle_defered_interrupt
> 
> And - of course - interrupt checks intr_lock in its entry and if it is
> zero, sets intr_pending and exits immediatelly.

And immediately gets called again. You have to mask the irq which is non trivial
especially if you want to do it right on the BX. But you can do this and rtlinux
does

Re: SMP in 2.4

[Mikulas Patocka]

> > 	testb %al, intr_pending
> > 	jnz somewhere_away_to_handle_defered_interrupt
> > 
> > And - of course - interrupt checks intr_lock in its entry and if it is
> > zero, sets intr_pending and exits immediatelly.
> 
> And immediately gets called again. You have to mask the irq which is non trivial
> especially if you want to do it right on the BX. But you can do this and rtlinux
> does

There is already desc->handler->ack(irq) in do_IRQ which does that. Is any
more special handling needed? 

Mikulas

Re: SMP in 2.4

[Alan Cox]

> > especially if you want to do it right on the BX. But you can do this and rtlinux
> > does
> 
> There is already desc->handler->ack(irq) in do_IRQ which does that. Is any
> more special handling needed? 

You need to keep the IRQ line masked not just ack it. The code for handling all
the BX and other cases is there but its a bit more than 3 clock cycles long

Re: SMP in 2.4

[Mikulas Patocka]

On Sat, 21 Apr 2001, Alan Cox wrote:

> > > especially if you want to do it right on the BX. But you can do this and rtlinux
> > > does
> > 
> > There is already desc->handler->ack(irq) in do_IRQ which does that. Is any
> > more special handling needed? 
> 
> You need to keep the IRQ line masked not just ack it.

desc->handler->ack could be mask_and_ack_8259A, ack_edge_ioapic_irq,
mask_and_ack_level_ioapic_irq or ack_lapic_irq - it does exactly that -
masking and acking an interrupt. 

> The code for handling all
> the BX and other cases is there but its a bit more than 3 clock cycles long

What magic does it have to do? Where is it in kernel?

Mikulas