From: Martin Schwidefsky <schwidefsky@de.ibm.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Michael Holzheu <holzheu@linux.vnet.ibm.com>,
Shailabh Nagar <nagar1234@in.ibm.com>,
Andrew Morton <akpm@linux-foundation.org>,
Venkatesh Pallipadi <venki@google.com>,
Suresh Siddha <suresh.b.siddha@intel.com>,
Ingo Molnar <mingo@elte.hu>, Oleg Nesterov <oleg@redhat.com>,
John stultz <johnstul@us.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Balbir Singh <balbir@linux.vnet.ibm.com>,
Heiko Carstens <heiko.carstens@de.ibm.com>,
Roland McGrath <roland@redhat.com>,
linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org,
"jeremy.fitzhardinge" <jeremy.fitzhardinge@citrix.com>,
Avi Kivity <avi@redhat.com>
Subject: Re: [RFC][PATCH v2 4/7] taskstats: Add per task steal time accounting
Date: Tue, 16 Nov 2010 16:33:25 +0100 [thread overview]
Message-ID: <20101116163325.755a709f@mschwide.boeblingen.de.ibm.com> (raw)
In-Reply-To: <1289909768.2109.592.camel@laptop>
On Tue, 16 Nov 2010 13:16:08 +0100
Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> On Tue, 2010-11-16 at 09:51 +0100, Martin Schwidefsky wrote:
> > On Mon, 15 Nov 2010 19:08:44 +0100
> > Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> >
> > > On Mon, 2010-11-15 at 18:59 +0100, Martin Schwidefsky wrote:
> > > > Steal time per task is at least good for performance problem analysis.
> > > > Sometimes knowing what is not the cause of a performance problem can help you
> > > > tremendously. If a task is slow and has no steal time, well then the hypervisor
> > > > is likely not the culprit. On the other hand if you do see lots of steal time
> > > > for a task while the rest of the system doesn't cause any steal time can tell
> > > > you something as well. That task might hit a specific function which causes
> > > > hypervisor overhead. The usefulness depends on the situation, it is another
> > > > data point which may or may not help you.
> > >
> > > If performance analysis is the only reason, why not add a tracepoint on
> > > vcpu enter that reports the duration the vcpu was out for and use perf
> > > to gather said data? It can tell you what process was running and what
> > > instruction it was at when the vcpu went away.
> > >
> > > No need to add 40 bytes per task for that.
> >
> > Which vcpu enter? We usually have z/VM as our hypervisor and want to be able
> > to do performance analysis with the data we gather inside the guest. There
> > is no vcpu enter we could put a tracepoint on. We would have to put tracepoints
> > on every possible interaction point with z/VM to get this data. To me it seems
> > a lot simpler to add the per-task steal time.
>
> Oh, you guys don't have a hypercall wrapper to exploit? Because from
> what I heard from the kvm/xen/lguest people I gathered they could in
> fact do something like I proposed.
We could do something along the lines of a hypercall wrapper (we would call
it a diagnose wrapper, same thing). The diagnoses we have on s390 do vastly
different things, so it is not easy to have a common diagnose wrapper.
Would be easier to add a tracepoint for each diagnose inline assembly.
> In fact, kvm seems to already have these tracepoints: kvm_exit/kvm_entry
> and it has a separate excplicit hypercall tracepoint as well:
> kvm_hypercall.
But the kvm tracepoints are used when Linux is the hypervisor, no? For our
situation that would be a tracepoint in z/VM - or the equivalent. This is
out of scope of this patch.
> Except that the per-task steal time gives you lot less detail, being
> able to profile on vcpu exit/enter gives you a much more powerfull
> performance tool. Aside from being able to measure the steal-time it
> allows you to instantly find hypercalls (both explicit as well as
> implicit), so you can also measure the hypercall induced steal-time as
> well.
Yes and no. The tracepoint idea looks interesting in itself. But that does
not completely replace the per-task steal time. The hypervisor can take
away the cpu anytime, it is still interesting to know which task was hit
hardest by that. You could view the cpu time lost by a hypercall as
"synchronous" steal time for the task, the remaining delta to the total
per-task steal time as "asynchronous" steal time.
> > And if it is really the additional 40 bytes on x86 that bother you so much,
> > we could put them behind #ifdef CONFIG_VIRT_CPU_ACCOUNTING. There already
> > is one in the task_struct for prev_utime and prev_stime.
>
> Making it configurable would definitely help the embedded people, not
> sure about VIRT_CPU_ACCOUNTING though, I bet the x86 virt weird^Wpeople
> would like it too -- if only to strive for feature parity if nothing
> else :/
>
> Its just that I'm not at all convinced its the best approach to solve
> the problem posed, and once its committed we're stuck with it due to
> ABI.
>
> We should be very careful not to die a death of thousand cuts with all
> this accounting madness, there's way too many weird-ass process
> accounting junk that adds ABI constraints as it is.
>
> I think its definitely worth investing extra time to implement these
> tracepoints if at all possible on your architecture before committing
> yourself to something like this.
I don't think it is either per-task steal time or tracepoints. Ideally
we'd have both. But I understand that you want to be careful about
committing an ABI. From my viewpoint per-task steal is a logical
extension, the data it is based on is already there.
--
blue skies,
Martin.
"Reality continues to ruin my life." - Calvin.
next prev parent reply other threads:[~2010-11-16 15:33 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-11 17:03 [RFC][PATCH v2 0/7] taskstats: Enhancements for precise process accounting (version 2) Michael Holzheu
2010-11-11 17:03 ` [RFC][PATCH v2 1/7] taskstats: Add new taskstats command TASKSTATS_CMD_ATTR_PIDS Michael Holzheu
2010-11-13 19:20 ` Peter Zijlstra
2010-11-15 15:53 ` Michael Holzheu
2010-11-15 16:06 ` Peter Zijlstra
2010-11-15 17:09 ` Michael Holzheu
2010-11-15 17:21 ` Peter Zijlstra
2010-11-16 12:16 ` Michael Holzheu
2010-11-16 12:36 ` Peter Zijlstra
2010-11-13 19:39 ` Peter Zijlstra
2010-11-13 20:00 ` Balbir Singh
2010-11-15 14:50 ` Michael Holzheu
2010-11-11 17:03 ` [RFC][PATCH v2 2/7] taskstats: Add "/proc/taskstats" Michael Holzheu
2010-11-11 17:03 ` [RFC][PATCH v2 3/7] taskstats: Add thread group ID to taskstats structure Michael Holzheu
2010-11-11 17:03 ` [RFC][PATCH v2 4/7] taskstats: Add per task steal time accounting Michael Holzheu
2010-11-13 19:38 ` Peter Zijlstra
2010-11-15 14:50 ` Martin Schwidefsky
2010-11-15 15:11 ` Peter Zijlstra
2010-11-15 17:42 ` Martin Schwidefsky
2010-11-15 17:45 ` Peter Zijlstra
2010-11-15 17:47 ` Peter Zijlstra
2010-11-15 17:48 ` Peter Zijlstra
2010-11-15 17:50 ` Peter Zijlstra
2010-11-15 17:59 ` Martin Schwidefsky
2010-11-15 18:08 ` Peter Zijlstra
2010-11-16 8:51 ` Martin Schwidefsky
2010-11-16 12:16 ` Peter Zijlstra
2010-11-16 15:33 ` Martin Schwidefsky [this message]
2010-11-16 15:45 ` Peter Zijlstra
2010-11-16 16:05 ` Martin Schwidefsky
2010-11-16 18:39 ` Jeremy Fitzhardinge
2010-11-16 16:38 ` Avi Kivity
2010-11-16 16:43 ` Peter Zijlstra
2010-11-16 16:56 ` Avi Kivity
2010-11-16 17:06 ` Avi Kivity
2010-11-11 17:03 ` [RFC][PATCH v2 5/7] taskstats: Improve cumulative CPU " Michael Holzheu
2010-11-13 18:38 ` Oleg Nesterov
2010-11-15 15:55 ` Martin Schwidefsky
2010-11-15 16:03 ` Peter Zijlstra
2010-11-15 17:49 ` Martin Schwidefsky
2010-11-15 17:51 ` Peter Zijlstra
2010-11-15 18:00 ` Martin Schwidefsky
2010-11-15 18:10 ` Peter Zijlstra
2010-11-16 8:54 ` Martin Schwidefsky
2010-11-16 16:57 ` Michael Holzheu
2010-11-18 17:10 ` Oleg Nesterov
2010-11-19 19:46 ` Michael Holzheu
2010-11-16 17:34 ` Michael Holzheu
2010-11-16 17:50 ` Oleg Nesterov
2010-11-18 16:34 ` Oleg Nesterov
2010-11-11 17:03 ` [RFC][PATCH v2 6/7] taskstats: Fix accounting for non-leader thread exec Michael Holzheu
2010-11-11 17:11 ` [RFC][PATCH v2 7/7] taskstats: Precise process accounting user space Michael Holzheu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101116163325.755a709f@mschwide.boeblingen.de.ibm.com \
--to=schwidefsky@de.ibm.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=avi@redhat.com \
--cc=balbir@linux.vnet.ibm.com \
--cc=heiko.carstens@de.ibm.com \
--cc=holzheu@linux.vnet.ibm.com \
--cc=jeremy.fitzhardinge@citrix.com \
--cc=johnstul@us.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=nagar1234@in.ibm.com \
--cc=oleg@redhat.com \
--cc=roland@redhat.com \
--cc=suresh.b.siddha@intel.com \
--cc=tglx@linutronix.de \
--cc=venki@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®