Re: [PATCH v6 09/22] x86/resctrl: Introduce MBM counters bitmap

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 8/6/24 3:00 PM, Babu Moger wrote:
> Hardware provides a set of counters when mbm_cntr_assignable feature is
> supported. These counters are used for assigning the events in resctrl
> group when the feature is enabled.

"in resctrl group" -> "in a resctrl group"?

> 
> Introduce mbm_cntrs_free_map bitmap to track available and free counters

What is the difference between an available and a free counter?

> and set of routines to allocate and free the counters.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---

> ---
>   arch/x86/kernel/cpu/resctrl/internal.h |  2 ++
>   arch/x86/kernel/cpu/resctrl/rdtgroup.c | 33 ++++++++++++++++++++++++++
>   2 files changed, 35 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index 154983a67646..6263362496a3 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -662,6 +662,8 @@ void __check_limbo(struct rdt_mon_domain *d, bool force_free);
>   void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
>   void __init resctrl_file_fflags_init(const char *config,
>   				     unsigned long fflags);
> +int mbm_cntr_alloc(struct rdt_resource *r);
> +void mbm_cntr_free(u32 cntr_id);
>   void rdt_staged_configs_clear(void);
>   bool closid_allocated(unsigned int closid);
>   int resctrl_find_cleanest_closid(void);
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index ab4fab3b7cf1..c818965e36c9 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -185,6 +185,37 @@ bool closid_allocated(unsigned int closid)
>   	return !test_bit(closid, &closid_free_map);
>   }
>   
> +/*
> + * Counter bitmap for tracking the available counters.
> + * ABMC feature provides set of hardware counters for enabling events.

"ABMC feature" -> "mbm_cntr_assign mode"

> + * Each event takes one hardware counter. Kernel needs to keep track

"Each event takes one hardware counter" -> "Each RMID and event pair takes
one hardware counter" ?


> + * of number of available counters.

"of number of available counters" -> "of the number of available counters"?

> + */
> +static DECLARE_BITMAP(mbm_cntrs_free_map, 64);
> +
> +static void mbm_cntrs_init(struct rdt_resource *r)
> +{
> +	bitmap_fill(mbm_cntrs_free_map, r->mon.num_mbm_cntrs);

Apart from what James mentioned about the different sizes, please also
add checking that the resource actually supports monitoring and
assignable counters before proceeding with the bitmap ops.

> +}
> +
> +int mbm_cntr_alloc(struct rdt_resource *r)
> +{
> +	int cntr_id;
> +
> +	cntr_id = find_first_bit(mbm_cntrs_free_map, r->mon.num_mbm_cntrs);
> +	if (cntr_id >= r->mon.num_mbm_cntrs)
> +		return -ENOSPC;
> +
> +	__clear_bit(cntr_id, mbm_cntrs_free_map);
> +
> +	return cntr_id;
> +}
> +
> +void mbm_cntr_free(u32 cntr_id)
> +{
> +	__set_bit(cntr_id, mbm_cntrs_free_map);
> +}
> +
>   /**
>    * rdtgroup_mode_by_closid - Return mode of resource group with closid
>    * @closid: closid if the resource group
> @@ -2748,6 +2779,8 @@ static int rdt_get_tree(struct fs_context *fc)
>   
>   	closid_init();
>   
> +	mbm_cntrs_init(&rdt_resources_all[RDT_RESOURCE_L3].r_resctrl);
> +
>   	if (resctrl_arch_mon_capable())
>   		flags |= RFTYPE_MON;
>   

This is also an example of what James mentioned elsewhere where there is an
assumption that this feature applies to the L3 resource. This has a consequence
that some code is global (like mbm_cntrs_free_map), assuming the L3 resource, while
other code takes the resource as parameter (eg. mbm_cntr_alloc()). This results
in inconsistent interface where, for example, allocating a counter needs resource
as parameter but freeing a counter does not. James already proposed different
treatment of the bitmap and L3 resource parameters, I expect with such guidance
the interfaces will become more intuitive.

Reinette