Re: [PATCH v7 09/24] x86/resctrl: Introduce bitmap mbm_cntr_free_map to track assignable counters

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 9/4/24 3:21 PM, Babu Moger wrote:
> Hardware provides a set of counters when mbm_cntr_assignable feature is
> supported. These counters are used for assigning the events in resctrl
> a group when the feature is enabled. The kernel must manage and track the

The second sentence ("These counters ...") is difficult to parse.

> number of available counters.

"The kernel must manage and track the number of available counters." ->
"The kernel must manage and track the available counters." ?

> 
> Introduce mbm_cntr_free_map bitmap to track available counters and set
> of routines to allocate and free the counters.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---

...

> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index e3e71843401a..f98cc5b9bebc 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -1175,6 +1175,30 @@ static __init int snc_get_config(void)
>  	return ret;
>  }
>  
> +/*
> + * Counter bitmap for tracking the available counters.
> + * 'mbm_cntr_assign' mode provides set of hardware counters for assigning
> + * RMID, event pair. Each RMID and event pair takes one hardware counter.

(soft-ABMC may need to edit this comment)

> + * Kernel needs to keep track of the number of available counters.

Last sentence seems to be duplicate of the first?

> + */
> +static int mbm_cntrs_init(struct rdt_resource *r)

Needs __init?

> +{
> +	if (r->mon.mbm_cntr_assignable) {
> +		r->mon.mbm_cntr_free_map = bitmap_zalloc(r->mon.num_mbm_cntrs,
> +							 GFP_KERNEL);
> +		if (!r->mon.mbm_cntr_free_map)
> +			return -ENOMEM;
> +		bitmap_fill(r->mon.mbm_cntr_free_map, r->mon.num_mbm_cntrs);
> +	}
> +	return 0;
> +}
> +
> +static void __exit mbm_cntrs_exit(struct rdt_resource *r)
> +{
> +	if (r->mon.mbm_cntr_assignable)
> +		bitmap_free(r->mon.mbm_cntr_free_map);
> +}
> +
>  int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  {
>  	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
> @@ -1240,6 +1264,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  		}
>  	}
>  
> +	ret = mbm_cntrs_init(r);
> +	if (ret)
> +		return ret;

Missing cleanup of earlier allocation on error path here. Even so, this does not
seem to integrate with existing dom_data_init() wrt ordering and locking. Could
this be more fitting when merged with dom_data_init() (after moving it)?

> +
>  	l3_mon_evt_init(r);
>  
>  	r->mon_capable = true;
> @@ -1247,9 +1275,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  	return 0;
>  }
>  
> -void __exit rdt_put_mon_l3_config(void)
> +void __exit rdt_put_mon_l3_config(struct rdt_resource *r)
>  {
>  	dom_data_exit();
> +	mbm_cntrs_exit(r);

There is a mismatch wrt locking used in dom_data_exit() and mbm_cntrs_exit() that is
sure to cause confusion and difficulty in the MPAM transition.

>  }
>  
>  void __init intel_rdt_mbm_apply_quirk(void)
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index ba737890d5c2..a51992984832 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -185,6 +185,25 @@ bool closid_allocated(unsigned int closid)
>  	return !test_bit(closid, &closid_free_map);
>  }
>  
> +int mbm_cntr_alloc(struct rdt_resource *r)
> +{
> +	int cntr_id;
> +
> +	cntr_id = find_first_bit(r->mon.mbm_cntr_free_map,
> +				 r->mon.num_mbm_cntrs);
> +	if (cntr_id >= r->mon.num_mbm_cntrs)
> +		return -ENOSPC;
> +
> +	__clear_bit(cntr_id, r->mon.mbm_cntr_free_map);
> +
> +	return cntr_id;
> +}
> +
> +void mbm_cntr_free(struct rdt_resource *r, u32 cntr_id)
> +{
> +	__set_bit(cntr_id, r->mon.mbm_cntr_free_map);
> +}
> +
>  /**
>   * rdtgroup_mode_by_closid - Return mode of resource group with closid
>   * @closid: closid if the resource group
> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
> index f11d6fdfd977..aab22ff8e0c1 100644
> --- a/include/linux/resctrl.h
> +++ b/include/linux/resctrl.h
> @@ -187,12 +187,14 @@ enum resctrl_scope {
>   * @num_rmid:		Number of RMIDs available
>   * @num_mbm_cntrs:	Number of assignable monitoring counters
>   * @mbm_cntr_assignable:Is system capable of supporting monitor assignment?
> + * @mbm_cntr_free_map:	bitmap of number of assignable MBM counters

The "number of" is not clear ... it seems to indicate tracking a count? How about
just "bitmap of free MBM counters"

>   * @evt_list:		List of monitoring events
>   */
>  struct resctrl_mon {
>  	int			num_rmid;
>  	int			num_mbm_cntrs;
>  	bool			mbm_cntr_assignable;
> +	unsigned long		*mbm_cntr_free_map;
>  	struct list_head	evt_list;
>  };
>  

Reinette