From: "Waskiewicz Jr, Peter P" <peter.p.waskiewicz.jr@intel.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>, Thomas Gleixner <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"H. Peter Anvin" <hpa@zytor.com>, Li Zefan <lizefan@huawei.com>,
"containers@lists.linux-foundation.org"
<containers@lists.linux-foundation.org>,
"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/4] x86: Add Cache QoS Monitoring (CQM) support
Date: Mon, 6 Jan 2014 20:10:45 +0000 [thread overview]
Message-ID: <1389039035.32504.35.camel@ppwaskie-mobl.amr.corp.intel.com> (raw)
In-Reply-To: <20140106180636.GG30183@twins.programming.kicks-ass.net>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 3025 bytes --]
On Mon, 2014-01-06 at 19:06 +0100, Peter Zijlstra wrote:
> On Mon, Jan 06, 2014 at 04:47:57PM +0000, Waskiewicz Jr, Peter P wrote:
> > > As is I don't really see a good use for RMIDs and I would simply not use
> > > them.
> >
> > If you want to use CQM in the hardware, then the RMID is how you get the
> > cache usage data from the CPU. If you don't want to use CQM, then you
> > can ignore RMIDs.
>
> I think you can make do with a single RMID (per cpu). When you program
> the counter (be it for a task, cpu or cgroup context) you set the 1 RMID
> and EVSEL and read the CTR.
>
> What I'm not entirely clear on is if the EVSEL and CTR MSR are per
> logical CPU or per L3 (package); /me prays they're per logical CPU.
There is one per logical CPU. However, in the current generation, they
report on the usage of the same L3 cache. But the CPU takes care of the
resolution of which MSR write and read comes from the logical CPU, so
software doesn't need to lock access to it from different CPUs.
> > One of the best use cases for using RMIDs is in virtualization.
>
> *groan*.. /me plugs wax in ears and goes la-la-la-la
>
> > A VM
> > may be a heavy cache user, or a light cache user. Tracing different VMs
> > on different RMIDs can allow an admin to identify which VM may be
> > causing high levels of eviction, and either migrate it to another host,
> > or move other tasks/VMs to other hosts. Without CQM, it's much harder
> > to find which process is eating the cache up.
>
> Not necessarily VMs, there's plenty large processes that exhibit similar
> problems.. why must people always do VMs :-(
Completely agreed. It's just the loudest people right now asking for
this capability are using VMs for the most part.
> That said, even with a single RMID you can get that information by
> simply running it against all competing processes one at a time. Since
> there's limited RMID space you need to rotate at some point anyway.
>
> The cgroup interface you propose wouldn't allow for rotation; other than
> manual by creating different cgroups one after another.
I see your points, and I also think that the cgroup approach now isn't
the best way to make this completely flexible. What about this:
Add a new read/write entry to the /proc/<pid> attributes that is the
RMID to assign that process to. Then expose all the available RMIDs
in /sys/devices/system/cpu, say in a new directory platformqos (or
whatever), which then have all the statistics inside those, plus a knob
to enable monitoring or not. Then all the kernel exposes is a way to
assign a PID to an RMID, and a way to turn on monitoring or turn it off,
and get the data. I can then put a simple userspace tool together to
make the management suck less.
Thoughts?
Cheers,
-PJ
--
PJ Waskiewicz Open Source Technology Center
peter.p.waskiewicz.jr@intel.com Intel Corp.
ÿôèº{.nÇ+·®+%Ëÿ±éݶ\x17¥wÿº{.nÇ+·¥{±þG«éÿ{ayº\x1dÊÚë,j\a¢f£¢·hïêÿêçz_è®\x03(éÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?¨èÚ&£ø§~á¶iOæ¬z·vØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?I¥
next prev parent reply other threads:[~2014-01-06 20:10 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-03 20:34 Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 1/4] x86: Add support for Cache QoS Monitoring (CQM) detection Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 2/4] x86: Add Cache QoS Monitoring support to x86 perf uncore Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 3/4] cgroup: Add new cacheqos cgroup subsys to support Cache QoS Monitoring Peter P Waskiewicz Jr
2014-01-03 20:34 ` [PATCH 4/4] Documentation: Add documentation for cacheqos cgroup Peter P Waskiewicz Jr
2014-01-04 16:10 ` [PATCH 0/4] x86: Add Cache QoS Monitoring (CQM) support Tejun Heo
2014-01-04 22:43 ` Waskiewicz Jr, Peter P
2014-01-04 22:50 ` Tejun Heo
2014-01-05 5:23 ` Waskiewicz Jr, Peter P
2014-01-06 11:16 ` Peter Zijlstra
2014-01-06 16:34 ` Waskiewicz Jr, Peter P
2014-01-06 16:41 ` Peter Zijlstra
2014-01-06 16:47 ` Waskiewicz Jr, Peter P
2014-01-06 17:53 ` Peter Zijlstra
2014-01-06 18:05 ` Waskiewicz Jr, Peter P
2014-01-06 18:06 ` Peter Zijlstra
2014-01-06 20:10 ` Waskiewicz Jr, Peter P [this message]
2014-01-06 21:26 ` Peter Zijlstra
2014-01-06 21:48 ` Waskiewicz Jr, Peter P
2014-01-06 22:12 ` Peter Zijlstra
2014-01-06 22:45 ` Waskiewicz Jr, Peter P
2014-01-07 8:34 ` Peter Zijlstra
2014-01-07 15:15 ` Waskiewicz Jr, Peter P
2014-01-07 21:12 ` Peter Zijlstra
2014-01-10 18:55 ` Waskiewicz Jr, Peter P
2014-01-13 7:55 ` Peter Zijlstra
2014-01-14 17:58 ` H. Peter Anvin
2014-01-27 17:34 ` Peter Zijlstra
2014-02-18 17:29 ` Waskiewicz Jr, Peter P
2014-02-18 19:35 ` Peter Zijlstra
2014-02-18 19:54 ` Waskiewicz Jr, Peter P
2014-02-20 16:58 ` Peter Zijlstra
2014-01-14 20:46 ` Waskiewicz Jr, Peter P
2014-01-06 11:08 ` Peter Zijlstra
2014-01-06 16:42 ` Waskiewicz Jr, Peter P
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1389039035.32504.35.camel@ppwaskie-mobl.amr.corp.intel.com \
--to=peter.p.waskiewicz.jr@intel.com \
--cc=cgroups@vger.kernel.org \
--cc=containers@lists.linux-foundation.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®