Re: PREEMPT_RT vs I-PIPE: the numbers, part 2

[Bill Huey (hui) <bhuey@lnxw.com>]

On Wed, Jun 22, 2005 at 12:00:32PM -0400, Karim Yaghmour wrote:
> To tell you the truth, we've spent a considerable amount of time as
> it is on this and we need to move on to other things. So while we

Understood.

> I don't know what part of PREEMPT_RT causes this, but looking at
> some of the numbers from this output one is tempted to conclude that
> PREEMPT_RT causes a very significant impact on system load. And I
> don't say this lightly. Have a look for example at the local
> communication latencies between vanilla and PREEMPT_RT when the
> target is subject to the HD test. For a pipe, this goes from 9.4us
> to 21.6. For UDP this goes from 22us to 1070us !!! Even on a
> system without any load, the numbers are similar. Ouch.

I'm involved in other things now, but I wouldn't be surprised if it
was some kind of scheduler bug + softirq wacked interaction. softirqs,
from the last time I looked, was pretty raw in RT. Another thing to do
is to subtract the number of irq-thread context switches from the total
system context switch to see if there's any kind of oddities with the
spinlock conversion. I doubt there's going to be a ton of 'overscheduling',
nevertheless it would be a valuable number. This is such a new patch
that weird things like this are likely, but it's going to take an
investigation to see what the real cause is.

FreeBSD went through some slow down when they moved to SMPng, but not
the kind of numbers you show for things surrounding the network stack.
Something clearly bad happened.

bill