Re: bad networking related lag in v2.6.22-rc2

[Ingo Molnar <mingo@elte.hu>]

* Ingo Molnar <mingo@elte.hu> wrote:

> > ouch! a nearly 1 second delay got observed by the scheduler - something 
> > is really killing your system!
>
> ah, you got the latency tracer from Thomas, as part of the -hrt patchset 
> - that makes it quite a bit easier to debug. [...]

and ... you already did a trace for Thomas, for the softirq problem:

   http://cybertek.info/taitai/trace.txt.bz2

this trace shows really bad networking related kernel activities!

gkrellm-5977 does this at timestamp 0:

 gkrellm-5977  0..s.    0us : cond_resched_softirq (established_get_next)

2 milliseconds later it's still in established_get_next() (!):

 gkrellm-5977  0..s. 2001us : cond_resched_softirq (established_get_next)

and the whole thing takes ... 455 msecs:

 gkrellm-5977  0..s. 455443us+: cond_resched_softirq (established_get_next)

i think this suggests that you have tons of open sockets. What does 
"netstat -ts" say on your box?

	Ingo