Re: [PATCH v7 09/24] x86/resctrl: Introduce bitmap mbm_cntr_free_map to track assignable counters

["Moger, Babu" <babu.moger@amd.com>]

Hi Reinette,

On 9/19/24 11:42, Reinette Chatre wrote:
> Hi Babu,
> 
> On 9/4/24 3:21 PM, Babu Moger wrote:
>> Hardware provides a set of counters when mbm_cntr_assignable feature is
>> supported. These counters are used for assigning the events in resctrl
>> a group when the feature is enabled. The kernel must manage and track the
> 
> The second sentence ("These counters ...") is difficult to parse.

How about?

Counters are used for assigning the events in resctrl group.

> 
>> number of available counters.
> 
> "The kernel must manage and track the number of available counters." ->
> "The kernel must manage and track the available counters." ?

Sure.

> 
>>
>> Introduce mbm_cntr_free_map bitmap to track available counters and set
>> of routines to allocate and free the counters.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
> 
> ...
> 
>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>> index e3e71843401a..f98cc5b9bebc 100644
>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>> @@ -1175,6 +1175,30 @@ static __init int snc_get_config(void)
>>  	return ret;
>>  }
>>  
>> +/*
>> + * Counter bitmap for tracking the available counters.
>> + * 'mbm_cntr_assign' mode provides set of hardware counters for assigning
>> + * RMID, event pair. Each RMID and event pair takes one hardware counter.
> 
> (soft-ABMC may need to edit this comment)

Agree..

> 
>> + * Kernel needs to keep track of the number of available counters.
> 
> Last sentence seems to be duplicate of the first?

Will remove it.

> 
>> + */
>> +static int mbm_cntrs_init(struct rdt_resource *r)
> 
> Needs __init?

Did you mean to merge this with dom_data_init and don't have to have a
separate function. Please clarify.


> 
>> +{
>> +	if (r->mon.mbm_cntr_assignable) {
>> +		r->mon.mbm_cntr_free_map = bitmap_zalloc(r->mon.num_mbm_cntrs,
>> +							 GFP_KERNEL);
>> +		if (!r->mon.mbm_cntr_free_map)
>> +			return -ENOMEM;
>> +		bitmap_fill(r->mon.mbm_cntr_free_map, r->mon.num_mbm_cntrs);
>> +	}
>> +	return 0;
>> +}
>> +
>> +static void __exit mbm_cntrs_exit(struct rdt_resource *r)
>> +{
>> +	if (r->mon.mbm_cntr_assignable)
>> +		bitmap_free(r->mon.mbm_cntr_free_map);
>> +}
>> +
>>  int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>>  {
>>  	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
>> @@ -1240,6 +1264,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>>  		}
>>  	}
>>  
>> +	ret = mbm_cntrs_init(r);
>> +	if (ret)
>> +		return ret;
> 
> Missing cleanup of earlier allocation on error path here. Even so, this does not
> seem to integrate with existing dom_data_init() wrt ordering and locking. Could
> this be more fitting when merged with dom_data_init() (after moving it)?

Sure. Will do.

> 
>> +
>>  	l3_mon_evt_init(r);
>>  
>>  	r->mon_capable = true;
>> @@ -1247,9 +1275,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>>  	return 0;
>>  }
>>  
>> -void __exit rdt_put_mon_l3_config(void)
>> +void __exit rdt_put_mon_l3_config(struct rdt_resource *r)
>>  {
>>  	dom_data_exit();
>> +	mbm_cntrs_exit(r);
> 
> There is a mismatch wrt locking used in dom_data_exit() and mbm_cntrs_exit() that is
> sure to cause confusion and difficulty in the MPAM transition.

Will merge this with dom_data_exit.

> 
>>  }
>>  
>>  void __init intel_rdt_mbm_apply_quirk(void)
>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index ba737890d5c2..a51992984832 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -185,6 +185,25 @@ bool closid_allocated(unsigned int closid)
>>  	return !test_bit(closid, &closid_free_map);
>>  }
>>  
>> +int mbm_cntr_alloc(struct rdt_resource *r)
>> +{
>> +	int cntr_id;
>> +
>> +	cntr_id = find_first_bit(r->mon.mbm_cntr_free_map,
>> +				 r->mon.num_mbm_cntrs);
>> +	if (cntr_id >= r->mon.num_mbm_cntrs)
>> +		return -ENOSPC;
>> +
>> +	__clear_bit(cntr_id, r->mon.mbm_cntr_free_map);
>> +
>> +	return cntr_id;
>> +}
>> +
>> +void mbm_cntr_free(struct rdt_resource *r, u32 cntr_id)
>> +{
>> +	__set_bit(cntr_id, r->mon.mbm_cntr_free_map);
>> +}
>> +
>>  /**
>>   * rdtgroup_mode_by_closid - Return mode of resource group with closid
>>   * @closid: closid if the resource group
>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>> index f11d6fdfd977..aab22ff8e0c1 100644
>> --- a/include/linux/resctrl.h
>> +++ b/include/linux/resctrl.h
>> @@ -187,12 +187,14 @@ enum resctrl_scope {
>>   * @num_rmid:		Number of RMIDs available
>>   * @num_mbm_cntrs:	Number of assignable monitoring counters
>>   * @mbm_cntr_assignable:Is system capable of supporting monitor assignment?
>> + * @mbm_cntr_free_map:	bitmap of number of assignable MBM counters
> 
> The "number of" is not clear ... it seems to indicate tracking a count? How about
> just "bitmap of free MBM counters"

Sure.

> 
>>   * @evt_list:		List of monitoring events
>>   */
>>  struct resctrl_mon {
>>  	int			num_rmid;
>>  	int			num_mbm_cntrs;
>>  	bool			mbm_cntr_assignable;
>> +	unsigned long		*mbm_cntr_free_map;
>>  	struct list_head	evt_list;
>>  };
>>  
> 
> Reinette
> 

-- 
Thanks
Babu Moger