From: "Waskiewicz Jr, Peter P" <peter.p.waskiewicz.jr@intel.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>, Thomas Gleixner <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"H. Peter Anvin" <hpa@zytor.com>, Li Zefan <lizefan@huawei.com>,
"containers@lists.linux-foundation.org"
<containers@lists.linux-foundation.org>,
"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/4] x86: Add Cache QoS Monitoring (CQM) support
Date: Mon, 6 Jan 2014 16:47:57 +0000 [thread overview]
Message-ID: <1389026867.32504.16.camel@ppwaskie-mobl.amr.corp.intel.com> (raw)
In-Reply-To: <20140106164150.GQ31570@twins.programming.kicks-ass.net>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 4039 bytes --]
On Mon, 2014-01-06 at 17:41 +0100, Peter Zijlstra wrote:
> On Mon, Jan 06, 2014 at 04:34:04PM +0000, Waskiewicz Jr, Peter P wrote:
> > On Mon, 2014-01-06 at 12:16 +0100, Peter Zijlstra wrote:
> > > On Sun, Jan 05, 2014 at 05:23:07AM +0000, Waskiewicz Jr, Peter P wrote:
> > > > The CPU side is easy and clean. When something in the software wants to
> > > > monitor when a particular task is scheduled and started, write whatever
> > > > RMID that task is assigned to (through some mechanism) to the proper MSR
> > > > in the CPU. When that task is swapped out, clear the MSR to stop
> > > > monitoring of that RMID. When that RMID's statistics are requested by
> > > > the software (through some mechanism), then the CPU's MSRs are written
> > > > with the RMID in question, and the value is read of what has been
> > > > collected so far. In my case, I decided to use a cgroup for this
> > > > "mechanism" since so much of the grouping and task/group association
> > > > already exists and doesn't need to be rebuilt or re-invented.
> > >
> > > This still doesn't explain why you can't use perf-cgroup for this.
> >
> > I'm not completely familiar with perf-cgroup, so I looked for some
> > documentation for it to better understand it. Are you referring to perf
> > -G to monitor an existing cgroup/all cgroups? Or something else? If
> > it's the former, I'm not following you how this would fit.
>
> All the bits under CONFIG_CGROUP_PERF, I've no idea how userspace looks.
Ah ok. Yes, the userspace side of perf really doesn't fit controlling
the CQM bits at all from what I see.
> > > > > In general, I'm quite strongly opposed against using cgroup as
> > > > > arbitrary grouping mechanism for anything other than resource control,
> > > > > especially given that we're moving away from multiple hierarchies.
> > > >
> > > > Just to clarify then, would the mechanism in the cpuacct cgroup to
> > > > create a group off the root subsystem be considered multi-hierarchical?
> > > > If not, then the intent for this new cacheqos subsystem is to be
> > > > identical in that regard to cpuacct in the behavior.
> > > >
> > > > This is a resource controller, it just happens to be tied to a hardware
> > > > resource instead of an OS resource.
> > >
> > > No, cpuacct and perf-cgroup aren't actually controllers at all. They're
> > > resource monitors at best. Same with your Cache QoS Monitor, it doesn't
> > > control anything.
> >
> > I may be using controller in a different way than you are. Yes, the
> > Cache QoS Monitor is monitoring cache data. But it is also controlling
> > the allocation and deallocation of RMIDs to tasks/task groups as
> > monitoring is enabled and disabled for those groups. That's why I
> > called it a controller. If that's not accurate, I apologize.
>
> Yeah that's not accurate, nor desired I think, because you get into
> horrible problems with hierarchies, do child groups belong to your RMID
> or not?
I'd rather not support a child group of a child group. Only groups off
the root, and each group would be assigned an RMID when it's activated
for monitoring.
> As is I don't really see a good use for RMIDs and I would simply not use
> them.
If you want to use CQM in the hardware, then the RMID is how you get the
cache usage data from the CPU. If you don't want to use CQM, then you
can ignore RMIDs.
One of the best use cases for using RMIDs is in virtualization. A VM
may be a heavy cache user, or a light cache user. Tracing different VMs
on different RMIDs can allow an admin to identify which VM may be
causing high levels of eviction, and either migrate it to another host,
or move other tasks/VMs to other hosts. Without CQM, it's much harder
to find which process is eating the cache up.
Cheers,
-PJ
--
PJ Waskiewicz Open Source Technology Center
peter.p.waskiewicz.jr@intel.com Intel Corp.
ÿôèº{.nÇ+·®+%Ëÿ±éݶ\x17¥wÿº{.nÇ+·¥{±þG«éÿ{ayº\x1dÊÚë,j\a¢f£¢·hïêÿêçz_è®\x03(éÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?¨èÚ&£ø§~á¶iOæ¬z·vØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?I¥
next prev parent reply other threads:[~2014-01-06 16:48 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-03 20:34 Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 1/4] x86: Add support for Cache QoS Monitoring (CQM) detection Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 2/4] x86: Add Cache QoS Monitoring support to x86 perf uncore Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 3/4] cgroup: Add new cacheqos cgroup subsys to support Cache QoS Monitoring Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 4/4] Documentation: Add documentation for cacheqos cgroup Peter P Waskiewicz Jr
2014-01-04 16:10 ` [PATCH 0/4] x86: Add Cache QoS Monitoring (CQM) support Tejun Heo
2014-01-04 22:43 ` Waskiewicz Jr, Peter P
2014-01-04 22:50 ` Tejun Heo
2014-01-05 5:23 ` Waskiewicz Jr, Peter P
2014-01-06 11:16 ` Peter Zijlstra
2014-01-06 16:34 ` Waskiewicz Jr, Peter P
2014-01-06 16:41 ` Peter Zijlstra
2014-01-06 16:47 ` Waskiewicz Jr, Peter P [this message]
2014-01-06 17:53 ` Peter Zijlstra
2014-01-06 18:05 ` Waskiewicz Jr, Peter P
2014-01-06 18:06 ` Peter Zijlstra
2014-01-06 20:10 ` Waskiewicz Jr, Peter P
2014-01-06 21:26 ` Peter Zijlstra
2014-01-06 21:48 ` Waskiewicz Jr, Peter P
2014-01-06 22:12 ` Peter Zijlstra
2014-01-06 22:45 ` Waskiewicz Jr, Peter P
2014-01-07 8:34 ` Peter Zijlstra
2014-01-07 15:15 ` Waskiewicz Jr, Peter P
2014-01-07 21:12 ` Peter Zijlstra
2014-01-10 18:55 ` Waskiewicz Jr, Peter P
2014-01-13 7:55 ` Peter Zijlstra
2014-01-14 17:58 ` H. Peter Anvin
2014-01-27 17:34 ` Peter Zijlstra
2014-02-18 17:29 ` Waskiewicz Jr, Peter P
2014-02-18 19:35 ` Peter Zijlstra
2014-02-18 19:54 ` Waskiewicz Jr, Peter P
2014-02-20 16:58 ` Peter Zijlstra
2014-01-14 20:46 ` Waskiewicz Jr, Peter P
2014-01-06 11:08 ` Peter Zijlstra
2014-01-06 16:42 ` Waskiewicz Jr, Peter P
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1389026867.32504.16.camel@ppwaskie-mobl.amr.corp.intel.com \
--to=peter.p.waskiewicz.jr@intel.com \
--cc=cgroups@vger.kernel.org \
--cc=containers@lists.linux-foundation.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®