From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: Keika Kobayashi <kobayashi.kk@ncos.nec.co.jp>
Cc: kosaki.motohiro@jp.fujitsu.com, linux-kernel@vger.kernel.org,
akpm@linux-foundation.org, h-shimamoto@ct.jp.nec.com
Subject: Re: [PATCH 1/3 v2] softirq: Introduce statistics for softirq
Date: Wed, 26 Nov 2008 19:14:58 +0900 (JST) [thread overview]
Message-ID: <20081126191400.3CCF.KOSAKI.MOTOHIRO@jp.fujitsu.com> (raw)
In-Reply-To: <20081124154609.99deaef2.kobayashi.kk@ncos.nec.co.jp>
> On Sat, 22 Nov 2008 14:36:34 +0900 (JST)
> KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:
>
> > > Statistics for softirq doesn't exist.
> > > It will be helpful like statistics for interrupts.
> > > This patch introduces counting the number of softirq,
> > > which will be exported in /proc/softirqs.
> > >
> > > When softirq handler consumes much CPU time,
> > > /proc/stat is like the following.
> > >
> > > $ while :; do cat /proc/stat | head -n1 ; sleep 10 ; done
> > > cpu 88 0 408 739665 583 28 2 0 0
> > > cpu 450 0 1090 740970 594 28 1294 0 0
> > > ^^^^
> > > softirq
> > >
> > > In such a situation,
> > > /proc/softirqs shows us which softirq handler is invoked.
> > > We can see the increase rate of softirqs.
> > >
> > > <before>
> > > $ cat /proc/softirqs
> > > CPU0 CPU1 CPU2 CPU3
> > > HI 0 0 0 0
> > > TIMER 462850 462805 462782 462718
> > > NET_TX 0 0 0 365
> > > NET_RX 2472 2 2 40
> > > BLOCK 0 0 381 1164
> > > TASKLET 0 0 0 224
> > > SCHED 462654 462689 462698 462427
> > > RCU 3046 2423 3367 3173
> > >
> > > <after>
> > > $ cat /proc/softirqs
> > > CPU0 CPU1 CPU2 CPU3
> > > HI 0 0 0 0
> > > TIMER 463361 465077 465056 464991
> > > NET_TX 53 0 1 365
> > > NET_RX 3757 2 2 40
> > > BLOCK 0 0 398 1170
> > > TASKLET 0 0 0 224
> > > SCHED 463074 464318 464612 463330
> > > RCU 3505 2948 3947 3673
> >
> > Maybe, this printing format don't works well on >100 CPUS.
> >
> > Do you have any experience of trouble-shooting by this patch?
> > Actually, I don't understand this patch usuful working situation.
>
> When reported that CPU TIME of softirq went up,
> we had to investigate the cause.
> In that case, kind of network looked suspicious.
> We added such a function to confirm.
>
> Original patch shows ticks of softirqs.
> But /proc/interrups have counts of interrupts.
> So we changed from ticks to counts.
ok. it seems reasonable reason.
and this patch looks good to me.
Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>
> > Honestly, I think HI and TASKLET are not beneficialness.
> > end user can't know what tasklet spent time.
> > TIMER also are not so much useful because it always have the same rate.
> >
> > So.. guessing...
> >
> > ultra band-width networking problem?
next prev parent reply other threads:[~2008-11-26 10:15 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-11-22 4:22 [PATCH 0/3 v2] softirq: Statistics " Keika Kobayashi
2008-11-22 4:25 ` [PATCH 1/3 v2] softirq: Introduce statistics " Keika Kobayashi
2008-11-22 5:36 ` KOSAKI Motohiro
2008-11-24 23:46 ` Keika Kobayashi
2008-11-26 10:14 ` KOSAKI Motohiro [this message]
2008-11-22 4:27 ` [PATCH 2/3 v2] proc: Export statistics for softirq to /proc Keika Kobayashi
2008-11-22 11:36 ` Eric Dumazet
2008-11-26 10:10 ` KOSAKI Motohiro
2009-01-09 22:16 ` Andrew Morton
2009-01-10 18:22 ` kobayashi.kk
2009-01-10 19:44 ` KOSAKI Motohiro
2009-01-12 11:46 ` KOSAKI Motohiro
2009-01-12 23:53 ` kobayashi.kk
2008-11-22 4:28 ` [PATCH 3/3 v2] proc: Update document for /proc/softirqs and /proc/stat Keika Kobayashi
2008-11-26 10:11 ` KOSAKI Motohiro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20081126191400.3CCF.KOSAKI.MOTOHIRO@jp.fujitsu.com \
--to=kosaki.motohiro@jp.fujitsu.com \
--cc=akpm@linux-foundation.org \
--cc=h-shimamoto@ct.jp.nec.com \
--cc=kobayashi.kk@ncos.nec.co.jp \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®