About perf tool set exclude_guest = 1 on guest environment

About perf tool set exclude_guest = 1 on guest environment

[Eric Lin]

Hi all,

Currently, I run the perf command 'perf record -e branches:u ls'  and
'perf record -e branches ls'. It will set exclude_guest = 1 as below:

# perf record -e branches:u ls
# perf evlist -v
branches:u: .., exclude_kernel: 1, exclude_hv: 1,, exclude_guest: 1, ..

# perf record -e branches ls
# perf evlist -v
branches: ..., exclude_guest: 1, ...

As I understand it, set exclude_guest =1 will not include guest counting.
May I ask if I run the above commands as a guest user, should the
architecture implementation ignore this exclude_guest flags and still
make perf can count guest events in Linux kernel?

Best regards,
Eric Lin

Re: About perf tool set exclude_guest = 1 on guest environment

[Namhyung Kim]

Hello,

On Mon, Jan 17, 2022 at 8:05 PM Eric Lin <eric.lin@sifive.com> wrote:
>
> Hi all,
>
> Currently, I run the perf command 'perf record -e branches:u ls'  and
> 'perf record -e branches ls'. It will set exclude_guest = 1 as below:
>
> # perf record -e branches:u ls
> # perf evlist -v
> branches:u: .., exclude_kernel: 1, exclude_hv: 1,, exclude_guest: 1, ..
>
> # perf record -e branches ls
> # perf evlist -v
> branches: ..., exclude_guest: 1, ...
>
> As I understand it, set exclude_guest =1 will not include guest counting.
> May I ask if I run the above commands as a guest user, should the
> architecture implementation ignore this exclude_guest flags and still
> make perf can count guest events in Linux kernel?

So do you want to run the command in the host?
Then you can use both 'H' and 'G' modifiers like

  # perf record -e branches:HG ls

or (for user-mode only)

  # perf record -e branches:uHG ls

Thanks,
Namhyung

Re: About perf tool set exclude_guest = 1 on guest environment

[Eric Lin]

Hi Namhyung,

On Wed, Jan 19, 2022 at 1:02 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hello,
>
> On Mon, Jan 17, 2022 at 8:05 PM Eric Lin <eric.lin@sifive.com> wrote:
> >
> > Hi all,
> >
> > Currently, I run the perf command 'perf record -e branches:u ls'  and
> > 'perf record -e branches ls'. It will set exclude_guest = 1 as below:
> >
> > # perf record -e branches:u ls
> > # perf evlist -v
> > branches:u: .., exclude_kernel: 1, exclude_hv: 1,, exclude_guest: 1, ..
> >
> > # perf record -e branches ls
> > # perf evlist -v
> > branches: ..., exclude_guest: 1, ...
> >
> > As I understand it, set exclude_guest =1 will not include guest counting.
> > May I ask if I run the above commands as a guest user, should the
> > architecture implementation ignore this exclude_guest flags and still
> > make perf can count guest events in Linux kernel?
>
> So do you want to run the command in the host?
> Then you can use both 'H' and 'G' modifiers like
>
>   # perf record -e branches:HG ls
>
> or (for user-mode only)
>
>   # perf record -e branches:uHG ls
>

Actually, I want to run the commands in the guest.
But when I run the commands "perf record -e branches:u ls" and "perf
record -e branches ls", it will set exclude_guet = 1.
According to the documentation[1] exclude_guest definition, it should
not count guest events.
So, if I run these commands as a guest user can I get the profile data?

[1] https://man7.org/linux/man-pages/man2/perf_event_open.2.html

Thanks,
Eric Lin

> Thanks,
> Namhyung

Re: About perf tool set exclude_guest = 1 on guest environment

[Arnaldo Carvalho de Melo]

Em Thu, Jan 20, 2022 at 09:35:59PM +0800, Eric Lin escreveu:
> On Wed, Jan 19, 2022 at 1:02 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > On Mon, Jan 17, 2022 at 8:05 PM Eric Lin <eric.lin@sifive.com> wrote:
> > > Currently, I run the perf command 'perf record -e branches:u ls'  and
> > > 'perf record -e branches ls'. It will set exclude_guest = 1 as below:
> > >
> > > # perf record -e branches:u ls
> > > # perf evlist -v
> > > branches:u: .., exclude_kernel: 1, exclude_hv: 1,, exclude_guest: 1, ..
> > >
> > > # perf record -e branches ls
> > > # perf evlist -v
> > > branches: ..., exclude_guest: 1, ...
> > >
> > > As I understand it, set exclude_guest =1 will not include guest counting.
> > > May I ask if I run the above commands as a guest user, should the
> > > architecture implementation ignore this exclude_guest flags and still
> > > make perf can count guest events in Linux kernel?
> >
> > So do you want to run the command in the host?
> > Then you can use both 'H' and 'G' modifiers like
> >
> >   # perf record -e branches:HG ls
> >
> > or (for user-mode only)
> >
> >   # perf record -e branches:uHG ls
> >
> 
> Actually, I want to run the commands in the guest.
> But when I run the commands "perf record -e branches:u ls" and "perf
> record -e branches ls", it will set exclude_guet = 1.
> According to the documentation[1] exclude_guest definition, it should
> not count guest events.
> So, if I run these commands as a guest user can I get the profile data?
> 
> [1] https://man7.org/linux/man-pages/man2/perf_event_open.2.html

The tool shouldn't know that it is running in a guest, right?

And the kernel, running on a guest should exclude its guests if
attr.exclude_guest is set, not itself and its non-guest userland.

So, before delving into how the kernel running on guest _actually_
handles attr.exclude_guest=1 behaves, what are the results you are
getting from running perf on a guest?

Lemme see...

On a fedora:35 guest:

[root@fedora ~]# perf record ls
anaconda-ks.cfg  perf.data
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.014 MB perf.data (3 samples) ]
[root@fedora ~]# perf evlist -v
cpu-clock:pppH: type: 1, size: 128, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
[root@fedora ~]#

IIRC there is some extra magic to allow 'cycles' to be seen on the
guest, but apart from that it seems to work.

Eduardo, can you help us here?

- Arnaldo

Re: About perf tool set exclude_guest = 1 on guest environment

[Eric Lin]

Hi Arnaldo,

On Fri, Jan 21, 2022 at 5:57 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Thu, Jan 20, 2022 at 09:35:59PM +0800, Eric Lin escreveu:
> > On Wed, Jan 19, 2022 at 1:02 PM Namhyung Kim <namhyung@kernel.org> wrote:
> > > On Mon, Jan 17, 2022 at 8:05 PM Eric Lin <eric.lin@sifive.com> wrote:
> > > > Currently, I run the perf command 'perf record -e branches:u ls'  and
> > > > 'perf record -e branches ls'. It will set exclude_guest = 1 as below:
> > > >
> > > > # perf record -e branches:u ls
> > > > # perf evlist -v
> > > > branches:u: .., exclude_kernel: 1, exclude_hv: 1,, exclude_guest: 1, ..
> > > >
> > > > # perf record -e branches ls
> > > > # perf evlist -v
> > > > branches: ..., exclude_guest: 1, ...
> > > >
> > > > As I understand it, set exclude_guest =1 will not include guest counting.
> > > > May I ask if I run the above commands as a guest user, should the
> > > > architecture implementation ignore this exclude_guest flags and still
> > > > make perf can count guest events in Linux kernel?
> > >
> > > So do you want to run the command in the host?
> > > Then you can use both 'H' and 'G' modifiers like
> > >
> > >   # perf record -e branches:HG ls
> > >
> > > or (for user-mode only)
> > >
> > >   # perf record -e branches:uHG ls
> > >
> >
> > Actually, I want to run the commands in the guest.
> > But when I run the commands "perf record -e branches:u ls" and "perf
> > record -e branches ls", it will set exclude_guet = 1.
> > According to the documentation[1] exclude_guest definition, it should
> > not count guest events.
> > So, if I run these commands as a guest user can I get the profile data?
> >
> > [1] https://man7.org/linux/man-pages/man2/perf_event_open.2.html
>
> The tool shouldn't know that it is running in a guest, right?
>
> And the kernel, running on a guest should exclude its guests if
> attr.exclude_guest is set, not itself and its non-guest userland.
>
> So, before delving into how the kernel running on guest _actually_
> handles attr.exclude_guest=1 behaves, what are the results you are
> getting from running perf on a guest?
>
> Lemme see...
>
> On a fedora:35 guest:
>
> [root@fedora ~]# perf record ls
> anaconda-ks.cfg  perf.data
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.014 MB perf.data (3 samples) ]
> [root@fedora ~]# perf evlist -v
> cpu-clock:pppH: type: 1, size: 128, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
> [root@fedora ~]#
>
> IIRC there is some extra magic to allow 'cycles' to be seen on the
> guest, but apart from that it seems to work.
>

I did the same experiment on an x86 kvm guest, and I got the same result.

Here is my understanding about the exclude_guest flag and how the
kernel handles it.

Consider there are two situations that the user executes perf
commands, one is on the guest and another is on the host environment:
When a user is in a guest environment, it launches a kvm guest on a
guest environment. Then the user executes a command `perf stat -e
cycle:u ls` or `perf stat -e cycle ls`, the exclude_guest flag is
raised and the kernel will exclude the kvm guest that the user
launched, not the user's environment itself.

When a user is in a host environment, it launches a kvm guest on a
host environment. Then the user executes a command `perf stat -e
cycle:u ls` or `perf stat -e cycle ls`, the exclude_guest flag is
raised and the kernel will exclude the kvm guest that the user
launched.

Is my understanding about the exclude_guest flag and how the kernel
should handle it correct? Thanks.

Best regards,
Eric Lin

> Eduardo, can you help us here?
>
> - Arnaldo