Q: why _less_ performance on machine with SMP then with UP kernel ?

[dth@cistron.nl (Danny ter Haar)]

I've been posting here recently bout our newsgateway.
For short:

If i enable both CPU's i get less performance than enabling both cpu's

Long version:

Here is a description of the setup:
---
It's a tyan server /motherboard
http://www.tyan.com/products/html/ta26b2882.html
with 2 x OPTERON250 cpu's and 4 GIG of ECC ram.
There is 8 x scsi disks for storage
cupper gig-E for internal communication to spool/header servers etc.
acenic FiberOptic Gig-E 64bit PCI card for link to the internet.

Bandwidth use is sampled from the ethernet switch and with mrtg
visualised.

Take today for example:
http://newsgate.newsserver.nl/kernel/2.6.14-rc1-ethernet-bandwidth.png

>From yesterday till 10:30am i ran 2.6.13.1 in UP mode.
As you can see blue (==incoming traffic) is fairly constant.
This morning i compiled/installed 2.6.14-rc1-smp.
I let it ran till 12:15 but it's clear that it can't keep up
with the flow of data. I rebooted to 2.6.14-rc1 (UP) and that 
keeps up with the data just fine.

So what is the difference between UP & SMP ?
shared memory , shared interrupts.
I don't know _why_ it's living up to _my_ expectation.
I hoped that the load would drop (it's between 4 to 5) op UP kernel
because certain processes would be split over the processors.

Anybody want to try and explain to me where i'm making an error ?

Config file & kern.log output  can be found at :
http://newsgate.newsserver.nl/kernel/

A very confused

Danny