Re: emu10k1 - interrupt storm?

Re: emu10k1 - interrupt storm?

[Jakub Travnik]

Hello and sorry for replying late,

I did experienced same problems with emu10k1 on 2.4.8 (as is in Mandrake 8.1).

After modprobing emu10k1, interrupts per second (as reported by vmstat)
start to increase and in few minutes they were as high as 70000 per second.
rmmod-ing emu10k1 caused number of interrupts per second to slowly decrease.

Following setting affected this:

In BIOS setup, PCI options:
Interrupts triggered by
1, edge
2, level

value 'edge' causes problems,
value 'level' makes thing work without problems.

If you are experiencing problems with this, set to 'level'.


Jakub Travnik
jabber://jtra@jabber.com



----Replied message follows----
From: Zlatko Calusic (zlatko.calusic@iskon.hr)
Date: Sun Dec 09 2001 - 16:54:03 EST
Rui Sousa <rui.p.m.sousa@clix.pt> writes:
 >
 > The current emu10k1 driver uses a hardware clock to generate periodic
 > interrupts. These apparently ran at the wrong rate in some Alpha machines.
 > It's possible that the same problem occur now with more recent i386
 > machines.
 >

Hi!

Sorry to quote a really old email. :)

I'm currently investigating why is my emu10k1 doing so much
interrupts. They are so frequent that they usually show on a kernel
profile report on the top, no matter what I've been doing with the
poor machine (interrupts are there even if I'm not using my
Soundblaster live).

Kernel is the most recent 2.5.x. dmesg says:

Creative EMU10K1 PCI Audio Driver, version 0.16, 16:17:32 Dec 9 2001
emu10k1: EMU10K1 rev 6 model 0x8027 found, IO at 0xc400-0xc41f, IRQ 10
ac97_codec: AC97 Audio codec, id: 0x8384:0x7609 (SigmaTel STAC9721/23)

This is my /proc/interrupts:

            CPU0 CPU1
   0: 423348 425806 IO-APIC-edge timer
   1: 13674 13371 IO-APIC-edge keyboard
   2: 0 0 XT-PIC cascade
   8: 2 1 IO-APIC-edge rtc
  10: 5956299 5956064 IO-APIC-level EMU10K1 <===========
  11: 19803 20033 IO-APIC-level ide2
  12: 104203 101822 IO-APIC-edge PS/2 Mouse
  14: 4356 4285 IO-APIC-edge ide0
  15: 7 9 IO-APIC-edge ide1
NMI: 0 0
LOC: 849167 849165
ERR: 2
MIS: 0

Yes, this is a SMP machine, but I don't know why would that make any
difference. As you can see, number of emu10k1 interrupts is enormous
(I also tried noapic, no changes).

procinfo -d quickly shows that emu10k1 is generating ~1412 interrupts
per second (7060/2).

irq 0: 500 timer irq 10: 7060 EMU10K1
irq 1: 1 keyboard irq 11: 10 ide2
irq 2: 0 cascade [4] irq 12: 164 PS/2 Mouse
irq 3: 0 irq 14: 2 ide0
irq 4: 0 irq 15: 0 ide1
irq 8: 0 rtc

Is that the periodic hardware interrupt you're talking about, and why
are there so many interrupts? Is there a way to stop that storm?

Regards,

-- 
Zlatko
-

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

Jakub Travnik <J.Travnik@sh.cvut.cz> writes:

> Hello and sorry for replying late,
>
> I did experienced same problems with emu10k1 on 2.4.8 (as is in Mandrake 8.1).
>
> After modprobing emu10k1, interrupts per second (as reported by vmstat)
> start to increase and in few minutes they were as high as 70000 per second.
> rmmod-ing emu10k1 caused number of interrupts per second to slowly decrease.
>
> Following setting affected this:
>
> In BIOS setup, PCI options:
> Interrupts triggered by
> 1, edge
> 2, level

Unfortunately I don't have that setting in my BIOS.

>
> value 'edge' causes problems,
> value 'level' makes thing work without problems.
>

But anyway, emu10k1 on irq 5 already uses level type:

  5:       5193       5183   IO-APIC-level  EMU10K1


> If you are experiencing problems with this, set to 'level'.
>

Eventually, I forced esd to exit after few seconds of inactivity. That
way I at least don't have interrupt storms when I don't have any sound
output through esd.

Regards,
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Philipp Matthias Hahn]

Hallo Zlatko!

On Wed, Dec 12, 2001 at 10:47:30PM +0100, Zlatko Calusic wrote:
> Unfortunately esd is started by the gnome desktop environment and I
> can disable or enable it, but can't set any parameters (as far as I
> can see). Probably I'll disable it for good, as emu10k1 driver already
> does a great job mixing multiple sound streams.
Say ever no:

$ fuser /dev/dsp 
/dev/dsp:             1555
$ esdctl off
$ fuser /dev/dsp 
$ ps 1555
  PID TTY      STAT   TIME COMMAND
 1555 ?        S      0:00 esd -nobeeps

BYtE
Philipp
-- 
  / /  (_)__  __ ____  __ Philipp Hahn
 / /__/ / _ \/ // /\ \/ /
/____/_/_//_/\_,_/ /_/\_\ pmhahn@titan.lahn.de

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

Morgan Collins <sirmorcant@morcant.org> writes:

> On Wed, 2001-12-12 at 13:47, Zlatko Calusic wrote:
> > "Marcelo ''Mosca'' de Paula Bezerra" <mosca@roadnet.com.br> writes:
> > 
> > > Try running esd with the -as 10 options..
> > > As the help says, it will disconnect the audio device after 10 seconds
> > > of inactivity. It will at least help you with the interrupt load while
> > > not using sound.
> > > 
> > 
> > Yes, nice idea, but easier said than done. :)
> > 
> > Unfortunately esd is started by the gnome desktop environment and I
> > can disable or enable it, but can't set any parameters (as far as I
> > can see). Probably I'll disable it for good, as emu10k1 driver already
> > does a great job mixing multiple sound streams.
> > 
> > Regards,
> 
> in $prefix/etc/esd.conf
> add:
> 
> spawn_options=-as 10
> 

Hm, that is interesting. In my /etc/esound/esd.conf there are these
few lines:

[esd]
auto_spawn=0
spawn_options=-terminate -nobeeps -as 5
spawn_wait_ms=100

So esd already has -as parameter, but if I look in the process list I
see it started only as `esd -nobeeps'. I'll investigate further...

Thanks for the idea.
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

Doug McNaught <doug@wireboard.com> writes:

> Zlatko Calusic <zlatko.calusic@iskon.hr> writes:
> 
> > Yes, nice idea, but easier said than done. :)
> > 
> > Unfortunately esd is started by the gnome desktop environment and I
> > can disable or enable it, but can't set any parameters (as far as I
> > can see). Probably I'll disable it for good, as emu10k1 driver already
> > does a great job mixing multiple sound streams.
> 
> There's always the hack...
> 
> # cd /usr/bin
> # mv esd esd.real
> # cat >>esd
> #!/bin/sh
> exec esd.real -as 10 $*
> ^D
> # chmod 755 esd
> 

Sure, but it's messy. :)

Also, as I do frequent upgrades (Debian unstable) it would get
overwritten quite fast.
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Morgan Collins]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On Wed, 2001-12-12 at 13:47, Zlatko Calusic wrote:
> "Marcelo ''Mosca'' de Paula Bezerra" <mosca@roadnet.com.br> writes:
> 
> > Try running esd with the -as 10 options..
> > As the help says, it will disconnect the audio device after 10 seconds
> > of inactivity. It will at least help you with the interrupt load while
> > not using sound.
> > 
> 
> Yes, nice idea, but easier said than done. :)
> 
> Unfortunately esd is started by the gnome desktop environment and I
> can disable or enable it, but can't set any parameters (as far as I
> can see). Probably I'll disable it for good, as emu10k1 driver already
> does a great job mixing multiple sound streams.
> 
> Regards,

in $prefix/etc/esd.conf
add:

spawn_options=-as 10

-- 
Morgan Collins http://sirmorcant.morcant.org
Crypto Doesn't Kill - People Do.

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: emu10k1 - interrupt storm?

[Doug McNaught]

Zlatko Calusic <zlatko.calusic@iskon.hr> writes:

> Yes, nice idea, but easier said than done. :)
> 
> Unfortunately esd is started by the gnome desktop environment and I
> can disable or enable it, but can't set any parameters (as far as I
> can see). Probably I'll disable it for good, as emu10k1 driver already
> does a great job mixing multiple sound streams.

There's always the hack...

# cd /usr/bin
# mv esd esd.real
# cat >>esd
#!/bin/sh
exec esd.real -as 10 $*
^D
# chmod 755 esd

;)

-Doug
-- 
Let us cross over the river, and rest under the shade of the trees.
   --T. J. Jackson, 1863

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

Rui Sousa <rui.p.m.sousa@clix.pt> writes:
> 
> Yes. Start by checking the audio format used
> (sample bytes, sample rate, channels) and buffering parameters
> (fragment size). The interrupt rate is:
> 
> fragment_size / ( sample_rate * sample_bytes * channels) irq/s
> 
> 
> Also if the timer rate is screwed up you should see a lot of
> (from emu10k1_waveout_bh() in audio.c)
> 
> DPD(3, "Not enough transfer size -> %d\n", bytestocopy);
> 
> going like 64, 128, 192, 64, 128, 192 if for example you
> have a fragment of 256 bytes and the clock rate is 4 times what
> it should.
> 

As I don't know much about the sound drivers, I hope debug output will
be helpful to see what's really going wrong here. Beware, the
compression ratio is very high. :)

The output is from gnome startup till the gnome logout (just a few
seconds, as I expected lots of output). Notice that gnome produces
sound at the startup and also at logout, so the relevant part is
somewhere in the middle of the output. DEBUG_LEVEL was set at 4.


[-- Attachment #2: syslog.gz --]
[-- Type: application/octet-stream, Size: 20238 bytes --]

[-- Attachment #3: Type: text/plain, Size: 12 bytes --]


-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

"Marcelo ''Mosca'' de Paula Bezerra" <mosca@roadnet.com.br> writes:

> Try running esd with the -as 10 options..
> As the help says, it will disconnect the audio device after 10 seconds
> of inactivity. It will at least help you with the interrupt load while
> not using sound.
> 

Yes, nice idea, but easier said than done. :)

Unfortunately esd is started by the gnome desktop environment and I
can disable or enable it, but can't set any parameters (as far as I
can see). Probably I'll disable it for good, as emu10k1 driver already
does a great job mixing multiple sound streams.

Regards,
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Marcelo ''Mosca'' de Paula Bezerra]

Try running esd with the -as 10 options..
As the help says, it will disconnect the audio device after 10 seconds
of inactivity. It will at least help you with the interrupt load while
not using sound.

Rui Sousa wrote:
> 
> 
> If the sound device is open and playback is enabled the driver runs
> at "full speed" even if the output is just silence.
> 
> 
> Both I think... With some machines the hardware timer(used inside
> the emu10k1) seems to be running at wrong rate.
> 
> Rui Sousa
>

Re: emu10k1 - interrupt storm?

[Rui Sousa]

On 11 Dec 2001, Zlatko Calusic wrote:

> Rui Sousa <rui.p.m.sousa@clix.pt> writes:
> 
> > On 9 Dec 2001, Zlatko Calusic wrote:
> > 
> > The emu10k1 only generates interrupts when playing pcm sound.
> > The interrupt rate depends on the "fragment" size and is usually
> > smaller than ~1000interrupts/s (this is 256bytes fragment 48Khz sample 
> > rate, 16bit, stereo).
> > 
> > Is it possible there is another device producing these interrupts?
> > On-board devices (usb, ...) that you don't use?
> > Does this happen with this kernel only?
> 
> Hm, looks like it is really emu10k1. But...
> 
> When I booted into single user mode, the kernel wasn't reporting
> interrupts. Then I started Gnome, and there they were. Finally I found
> that killing/starting esd changes things. Whenever esd was up,
> interrupts were generated, when I killed it interrupts would stop.

Ok. Esd usually uses very small fragment sizes (~256 bytes) which cause 
high interrupt rates.

> 
> Then I straced esd daemon, just to find it was blocked on select().
> 
> {atlas} [~]# strace -p 705
> select(12, [4 6 7 8 9 10 11], NULL, NULL, NULL
> 
> Strange, if esd is not doing any work (just sleeping) who generates
> interrupts then?

If the sound device is open and playback is enabled the driver runs
at "full speed" even if the output is just silence.

> Also, we don't know now is it the driver to blame, or silly esd daemon.

Both I think... With some machines the hardware timer(used inside 
the emu10k1) seems to be running at wrong rate.

> > 
> > Which functions do you see listed? If it's only "emu10k1_interrupt()"
> > then the interrupt was not generated by the emu10k1.
> > 
> 
> Top 5 entries in the kernel profile:
> 
>    475 __rdtsc_delay                             16.9643
>    309 handle_IRQ_event                           2.4919
>    178 do_softirq                                 0.8725
>    148 emu10k1_waveout_bh                         0.7872
>    135 emu10k1_interrupt                          0.6490
> 
> As time goes by, the numbers get bigger (that is only 15min after
> reboot)...
> 
> Maybe it's time to enable debug option in the driver?

Yes. Start by checking the audio format used
(sample bytes, sample rate, channels) and buffering parameters
(fragment size). The interrupt rate is:

fragment_size / ( sample_rate * sample_bytes * channels) irq/s


Also if the timer rate is screwed up you should see a lot of
(from emu10k1_waveout_bh() in audio.c)

DPD(3, "Not enough transfer size -> %d\n", bytestocopy);

going like 64, 128, 192, 64, 128, 192 if for example you
have a fragment of 256 bytes and the clock rate is 4 times what
it should.

> Sorry for late answer, I got busy yesterday...
> 

Rui Sousa

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

Rui Sousa <rui.p.m.sousa@clix.pt> writes:

> On 9 Dec 2001, Zlatko Calusic wrote:
> 
> The emu10k1 only generates interrupts when playing pcm sound.
> The interrupt rate depends on the "fragment" size and is usually
> smaller than ~1000interrupts/s (this is 256bytes fragment 48Khz sample 
> rate, 16bit, stereo).
> 
> Is it possible there is another device producing these interrupts?
> On-board devices (usb, ...) that you don't use?
> Does this happen with this kernel only?

Hm, looks like it is really emu10k1. But...

When I booted into single user mode, the kernel wasn't reporting
interrupts. Then I started Gnome, and there they were. Finally I found
that killing/starting esd changes things. Whenever esd was up,
interrupts were generated, when I killed it interrupts would stop.

Then I straced esd daemon, just to find it was blocked on select().

{atlas} [~]# strace -p 705
select(12, [4 6 7 8 9 10 11], NULL, NULL, NULL

Strange, if esd is not doing any work (just sleeping) who generates
interrupts then?

Also, we don't know now is it the driver to blame, or silly esd daemon.

> 
> Which functions do you see listed? If it's only "emu10k1_interrupt()"
> then the interrupt was not generated by the emu10k1.
> 

Top 5 entries in the kernel profile:

   475 __rdtsc_delay                             16.9643
   309 handle_IRQ_event                           2.4919
   178 do_softirq                                 0.8725
   148 emu10k1_waveout_bh                         0.7872
   135 emu10k1_interrupt                          0.6490

As time goes by, the numbers get bigger (that is only 15min after
reboot)...

Maybe it's time to enable debug option in the driver?

Sorry for late answer, I got busy yesterday...
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Zlatko Calusic]

Hi! Thanks for the reply!

Rui Sousa <rui.p.m.sousa@clix.pt> writes:

> On 9 Dec 2001, Zlatko Calusic wrote:
> 
> The emu10k1 only generates interrupts when playing pcm sound.
> The interrupt rate depends on the "fragment" size and is usually
> smaller than ~1000interrupts/s (this is 256bytes fragment 48Khz sample 
> rate, 16bit, stereo).
>

OK, if that's so, that means I really have a problem.

> Is it possible there is another device producing these interrupts?
> On-board devices (usb, ...) that you don't use?
> Does this happen with this kernel only?
>

I was into investigating such possibilities, but then I decided to
send an email before I dig deeper. So I'll make some more extensive
checking when I return home, later today (I'm at work now) and report
results.

> Which functions do you see listed? If it's only "emu10k1_interrupt()"
> then the interrupt was not generated by the emu10k1.

I don't have profile output at hand now, but I remember well seeing
emu10k1_waveout_bh() also. I even consulted the source for those two
functions, but audio drivers are, sad to say, not really my area of
expertise. :)

I remind, interrupts are occurring with no music playing at all. In
fact, I'm going to check things again in the single user mode and
report exact profiling data. So as to exclude esd/mixers and other
possible nuisances that could spoil measurements. And also I'm going
to compile kernel without sblive driver to see if interrupts are
present then.

Expect more data soon...
-- 
Zlatko

Re: emu10k1 - interrupt storm?

[Rui Sousa]

On 9 Dec 2001, Zlatko Calusic wrote:

The emu10k1 only generates interrupts when playing pcm sound.
The interrupt rate depends on the "fragment" size and is usually
smaller than ~1000interrupts/s (this is 256bytes fragment 48Khz sample 
rate, 16bit, stereo).

Is it possible there is another device producing these interrupts?
On-board devices (usb, ...) that you don't use?
Does this happen with this kernel only?

> Rui Sousa <rui.p.m.sousa@clix.pt> writes:
> > 
> > The current emu10k1 driver uses a hardware clock to generate periodic 
> > interrupts. These apparently ran at the wrong rate in some Alpha machines.
> > It's possible that the same problem occur now with more recent i386 
> > machines.
> > 
> 
> Hi!
> 
> Sorry to quote a really old email. :)
> 
> I'm currently investigating why is my emu10k1 doing so much
> interrupts. They are so frequent that they usually show on a kernel
> profile report on the top, no matter what I've been doing with the

Which functions do you see listed? If it's only "emu10k1_interrupt()"
then the interrupt was not generated by the emu10k1.

> poor machine (interrupts are there even if I'm not using my
> Soundblaster live).
> 
> Kernel is the most recent 2.5.x. dmesg says:
> 
> Creative EMU10K1 PCI Audio Driver, version 0.16, 16:17:32 Dec  9 2001
> emu10k1: EMU10K1 rev 6 model 0x8027 found, IO at 0xc400-0xc41f, IRQ 10
> ac97_codec: AC97 Audio codec, id: 0x8384:0x7609 (SigmaTel STAC9721/23)
> 
> This is my /proc/interrupts:
> 
>            CPU0       CPU1       
>   0:     423348     425806    IO-APIC-edge  timer
>   1:      13674      13371    IO-APIC-edge  keyboard
>   2:          0          0          XT-PIC  cascade
>   8:          2          1    IO-APIC-edge  rtc
>  10:    5956299    5956064   IO-APIC-level  EMU10K1       <===========
>  11:      19803      20033   IO-APIC-level  ide2
>  12:     104203     101822    IO-APIC-edge  PS/2 Mouse
>  14:       4356       4285    IO-APIC-edge  ide0
>  15:          7          9    IO-APIC-edge  ide1
> NMI:          0          0 
> LOC:     849167     849165 
> ERR:          2
> MIS:          0
> 
> Yes, this is a SMP machine, but I don't know why would that make any
> difference. As you can see, number of emu10k1 interrupts is enormous
> (I also tried noapic, no changes).
> 
> procinfo -d quickly shows that emu10k1 is generating ~1412 interrupts
> per second (7060/2).
> 
> irq  0:       500 timer                 irq 10:      7060 EMU10K1
> irq  1:         1 keyboard              irq 11:        10 ide2
> irq  2:         0 cascade [4]           irq 12:       164 PS/2 Mouse
> irq  3:         0                       irq 14:         2 ide0
> irq  4:         0                       irq 15:         0 ide1
> irq  8:         0 rtc                  
> 
> Is that the periodic hardware interrupt you're talking about, and why
> are there so many interrupts? Is there a way to stop that storm?
> 
> Regards,
> 

Rui Sousa

emu10k1 - interrupt storm?

[Zlatko Calusic]

Rui Sousa <rui.p.m.sousa@clix.pt> writes:
> 
> The current emu10k1 driver uses a hardware clock to generate periodic 
> interrupts. These apparently ran at the wrong rate in some Alpha machines.
> It's possible that the same problem occur now with more recent i386 
> machines.
> 

Hi!

Sorry to quote a really old email. :)

I'm currently investigating why is my emu10k1 doing so much
interrupts. They are so frequent that they usually show on a kernel
profile report on the top, no matter what I've been doing with the
poor machine (interrupts are there even if I'm not using my
Soundblaster live).

Kernel is the most recent 2.5.x. dmesg says:

Creative EMU10K1 PCI Audio Driver, version 0.16, 16:17:32 Dec  9 2001
emu10k1: EMU10K1 rev 6 model 0x8027 found, IO at 0xc400-0xc41f, IRQ 10
ac97_codec: AC97 Audio codec, id: 0x8384:0x7609 (SigmaTel STAC9721/23)

This is my /proc/interrupts:

           CPU0       CPU1       
  0:     423348     425806    IO-APIC-edge  timer
  1:      13674      13371    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  8:          2          1    IO-APIC-edge  rtc
 10:    5956299    5956064   IO-APIC-level  EMU10K1       <===========
 11:      19803      20033   IO-APIC-level  ide2
 12:     104203     101822    IO-APIC-edge  PS/2 Mouse
 14:       4356       4285    IO-APIC-edge  ide0
 15:          7          9    IO-APIC-edge  ide1
NMI:          0          0 
LOC:     849167     849165 
ERR:          2
MIS:          0

Yes, this is a SMP machine, but I don't know why would that make any
difference. As you can see, number of emu10k1 interrupts is enormous
(I also tried noapic, no changes).

procinfo -d quickly shows that emu10k1 is generating ~1412 interrupts
per second (7060/2).

irq  0:       500 timer                 irq 10:      7060 EMU10K1
irq  1:         1 keyboard              irq 11:        10 ide2
irq  2:         0 cascade [4]           irq 12:       164 PS/2 Mouse
irq  3:         0                       irq 14:         2 ide0
irq  4:         0                       irq 15:         0 ide1
irq  8:         0 rtc                  

Is that the periodic hardware interrupt you're talking about, and why
are there so many interrupts? Is there a way to stop that storm?

Regards,
-- 
Zlatko