Re: [PATCH v6 09/22] x86/resctrl: Introduce MBM counters bitmap

["Moger, Babu" <babu.moger@amd.com>]

Hi Reinette,

On 8/16/24 16:35, Reinette Chatre wrote:
> Hi Babu,
> 
> On 8/6/24 3:00 PM, Babu Moger wrote:
>> Hardware provides a set of counters when mbm_cntr_assignable feature is
>> supported. These counters are used for assigning the events in resctrl
>> group when the feature is enabled.
> 
> "in resctrl group" -> "in a resctrl group"?
> 

Sure.

>>
>> Introduce mbm_cntrs_free_map bitmap to track available and free counters
> 
> What is the difference between an available and a free counter?

It is the same. Will correct the text here.

> 
>> and set of routines to allocate and free the counters.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
> 
>> ---
>>   arch/x86/kernel/cpu/resctrl/internal.h |  2 ++
>>   arch/x86/kernel/cpu/resctrl/rdtgroup.c | 33 ++++++++++++++++++++++++++
>>   2 files changed, 35 insertions(+)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h
>> b/arch/x86/kernel/cpu/resctrl/internal.h
>> index 154983a67646..6263362496a3 100644
>> --- a/arch/x86/kernel/cpu/resctrl/internal.h
>> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
>> @@ -662,6 +662,8 @@ void __check_limbo(struct rdt_mon_domain *d, bool
>> force_free);
>>   void rdt_domain_reconfigure_cdp(struct rdt_resource *r);
>>   void __init resctrl_file_fflags_init(const char *config,
>>                        unsigned long fflags);
>> +int mbm_cntr_alloc(struct rdt_resource *r);
>> +void mbm_cntr_free(u32 cntr_id);
>>   void rdt_staged_configs_clear(void);
>>   bool closid_allocated(unsigned int closid);
>>   int resctrl_find_cleanest_closid(void);
>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index ab4fab3b7cf1..c818965e36c9 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -185,6 +185,37 @@ bool closid_allocated(unsigned int closid)
>>       return !test_bit(closid, &closid_free_map);
>>   }
>>   +/*
>> + * Counter bitmap for tracking the available counters.
>> + * ABMC feature provides set of hardware counters for enabling events.
> 
> "ABMC feature" -> "mbm_cntr_assign mode"

Sure.

> 
>> + * Each event takes one hardware counter. Kernel needs to keep track
> 
> "Each event takes one hardware counter" -> "Each RMID and event pair takes
> one hardware counter" ?

Sure.

> 
> 
>> + * of number of available counters.
> 
> "of number of available counters" -> "of the number of available counters"?

Sure.
> 
>> + */
>> +static DECLARE_BITMAP(mbm_cntrs_free_map, 64);
>> +
>> +static void mbm_cntrs_init(struct rdt_resource *r)
>> +{
>> +    bitmap_fill(mbm_cntrs_free_map, r->mon.num_mbm_cntrs);
> 
> Apart from what James mentioned about the different sizes, please also
> add checking that the resource actually supports monitoring and
> assignable counters before proceeding with the bitmap ops.

Sure.
> 
>> +}
>> +
>> +int mbm_cntr_alloc(struct rdt_resource *r)
>> +{
>> +    int cntr_id;
>> +
>> +    cntr_id = find_first_bit(mbm_cntrs_free_map, r->mon.num_mbm_cntrs);
>> +    if (cntr_id >= r->mon.num_mbm_cntrs)
>> +        return -ENOSPC;
>> +
>> +    __clear_bit(cntr_id, mbm_cntrs_free_map);
>> +
>> +    return cntr_id;
>> +}
>> +
>> +void mbm_cntr_free(u32 cntr_id)
>> +{
>> +    __set_bit(cntr_id, mbm_cntrs_free_map);
>> +}
>> +
>>   /**
>>    * rdtgroup_mode_by_closid - Return mode of resource group with closid
>>    * @closid: closid if the resource group
>> @@ -2748,6 +2779,8 @@ static int rdt_get_tree(struct fs_context *fc)
>>         closid_init();
>>   +    mbm_cntrs_init(&rdt_resources_all[RDT_RESOURCE_L3].r_resctrl);
>> +
>>       if (resctrl_arch_mon_capable())
>>           flags |= RFTYPE_MON;
>>   
> 
> This is also an example of what James mentioned elsewhere where there is an
> assumption that this feature applies to the L3 resource. This has a
> consequence
> that some code is global (like mbm_cntrs_free_map), assuming the L3
> resource, while
> other code takes the resource as parameter (eg. mbm_cntr_alloc()). This
> results
> in inconsistent interface where, for example, allocating a counter needs

Yes. Will address it.

> resource
> as parameter but freeing a counter does not. James already proposed different
> treatment of the bitmap and L3 resource parameters, I expect with such
> guidance
> the interfaces will become more intuitive.
> 

How about making "mbm_cntrs_free_map" as part  of struct resctrl_mon?
It will be pointer and allocated dynamically based on number of counters.
All the related information (num_mbm_cntrs and mbm_cntr_assignable) is
already part of this data structure.
-- 
Thanks
Babu Moger