Re: [PATCH v2 2/2] perf mem: Support HITM for when mem_lvl_num is used

[German Gomez <german.gomez@arm.com>]

On 21/02/2022 22:48, Ali Saidi wrote:
> Current code only support HITM statistics for last level cache (LLC)
> when mem_lvl encodes the level. On existing Arm64 machines there are as
> many as four levels cache and this change supports decoding l1, l2, and
> llc hits from the mem_lvl_num data. Given that the mem_lvl namespace is
> being deprecated take this opportunity to encode the neoverse data into
> mem_lvl_num.

Since Neoverse is mentioned in the commit message, I think there should be a comment somewhere in the code as well.

>
> For loads that hit in a the LLC snoop filter and are fullfilled from a
> higher level cache, it's not usually clear what the true level of the
> cache the data came from (i.e. a transfer from a core could come from
> it's L1 or L2). Instead of making an assumption of where the line came
> from, add support for incrementing HITM if the source is CACHE_ANY.
>
> Since other architectures don't seem to populate the mem_lvl_num field
> here there shouldn't be a change in functionality.
>
> Signed-off-by: Ali Saidi <alisaidi@amazon.com>
> ---
>  tools/perf/util/mem-events.c | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/tools/perf/util/mem-events.c b/tools/perf/util/mem-events.c
> index ed0ab838bcc5..6c3fd4aac7ae 100644
> --- a/tools/perf/util/mem-events.c
> +++ b/tools/perf/util/mem-events.c
> @@ -485,6 +485,7 @@ int c2c_decode_stats(struct c2c_stats *stats, struct mem_info *mi)
>  	u64 daddr  = mi->daddr.addr;
>  	u64 op     = data_src->mem_op;
>  	u64 lvl    = data_src->mem_lvl;
> +	u64 lnum   = data_src->mem_lvl_num;
>  	u64 snoop  = data_src->mem_snoop;
>  	u64 lock   = data_src->mem_lock;
>  	u64 blk    = data_src->mem_blk;
> @@ -527,16 +528,18 @@ do {				\
>  			if (lvl & P(LVL, UNC)) stats->ld_uncache++;
>  			if (lvl & P(LVL, IO))  stats->ld_io++;
>  			if (lvl & P(LVL, LFB)) stats->ld_fbhit++;
> -			if (lvl & P(LVL, L1 )) stats->ld_l1hit++;
> -			if (lvl & P(LVL, L2 )) stats->ld_l2hit++;
> -			if (lvl & P(LVL, L3 )) {
> +			if (lvl & P(LVL, L1) || lnum == P(LVLNUM, L1))
> +				stats->ld_l1hit++;
> +			if (lvl & P(LVL, L2) || lnum == P(LVLNUM, L2))
> +				stats->ld_l2hit++;
> +			if (lvl & P(LVL, L3) || lnum == P(LVLNUM, L4)) {

According to a comment in the previous patch, using L4 is specific to Neoverse, right?

Maybe we need to distinguish the Neoverse case from the generic one here as well

if (is_neoverse)
// treat L4 as llc
else
// treat L3 as llc

>  				if (snoop & P(SNOOP, HITM))
>  					HITM_INC(lcl_hitm);
>  				else
>  					stats->ld_llchit++;
>  			}
>  
> -			if (lvl & P(LVL, LOC_RAM)) {
> +			if (lvl & P(LVL, LOC_RAM) || lnum == P(LVLNUM, RAM)) {
>  				stats->lcl_dram++;
>  				if (snoop & P(SNOOP, HIT))
>  					stats->ld_shared++;
> @@ -564,6 +567,9 @@ do {				\
>  				HITM_INC(rmt_hitm);
>  		}
>  
> +		if (lnum == P(LVLNUM, ANY_CACHE) && snoop & P(SNOOP, HITM))
> +			HITM_INC(lcl_hitm);
> +
>  		if ((lvl & P(LVL, MISS)))
>  			stats->ld_miss++;
>