Re: [ath9k-devel] ath9k: massive unexplained latency in 2.6.27 (rc5, rc6, probably others)

["Luis R. Rodriguez" <lrodriguez@atheros.com>]

On Fri, Sep 19, 2008 at 5:01 PM, Steven Noonan <steven@uplinklabs.net> wrote:
> I'll pull wireless-testing and run that kernel for another 8 hours.
> What's the URL for the repo? Surely you don't mean the
> wireless-testing on git.kernel.org that hasn't been touched for almost
> a week?

Yes, that's the guy. This week was the Kernel Summit and the Linux
Plumbers Conference too so there is understandably some lag in pulls
as developers are getting drunk.

git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-testing.git

> I kind of wish there was a better debugging facility in ath9k.

Patches are welcomed ;)

Also, we have a few interrupt levels you can use. In this case we are
debugging interrupts.

> With
> ATH_DBG_INTERRUPTS, I see several hundred messages per second, which
> fills log files up like crazy (I switched to ftrace_printk as Ingo
> suggested, which has alleviated this somewhat). Is there a way to make
> it less verbose (perhaps so that it prints data at a rate of no more
> than 2 or 3 messages per second)? Or maybe the rate at which it prints
> this stuff is indicative something's wrong?

No, interrupts do happen quite often so that is what you get, perhaps
we can enhance this by not printing on some interrupts but the point
is to debug interrupts after all. To help us debug in userspace in a
nicer way we can use sadc and sar. So you can try this:

On a window leave this running:

/usr/lib/sysstat/sadc -I 1 irqlog.sar

And when you are seeing your latency issues, run this:

sar -I 17 -f irqlog.sar | less

That is capital "i", this will show you the monitor results of number
of interrupts on IRQ 17. You want to see if the rate of change is huge
when you have latency issues. If not then its most likely an interrupt
storm by ath9k (which is my suspect right now) but something else.

  Luis