Re: [PATCH v14 18/32] fs/resctrl: Add the functionality to assign MBM events

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 6/13/25 2:05 PM, Babu Moger wrote:
> When supported "mbm_event" mode offers "num_mbm_cntrs" number of counters

"When supported, "mbm_event" counter assignment mode offers ..."?

> that can be assigned to RMID, event pairs and monitor bandwidth usage as
> long as it is assigned.
> 
> Add the functionality to allocate and assign a counter ID to an RMID, event
> pair in the domain.
> 
> If all the counters are in use, kernel will log the error message "Unable
> to allocate counter in domain" in /sys/fs/resctrl/info/last_cmd_status
> when a new assignment is requested. Exit on the first failure when
> assigning counters across all the domains.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---

...

> ---
>  fs/resctrl/internal.h |   3 +
>  fs/resctrl/monitor.c  | 134 ++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 137 insertions(+)
> 
> diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
> index 71059c2cda16..0767a1c46f26 100644
> --- a/fs/resctrl/internal.h
> +++ b/fs/resctrl/internal.h
> @@ -386,6 +386,9 @@ bool closid_allocated(unsigned int closid);
>  
>  int resctrl_find_cleanest_closid(void);
>  
> +int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
> +			      struct rdtgroup *rdtgrp, struct mon_evt *mevt);
> +
>  #ifdef CONFIG_RESCTRL_FS_PSEUDO_LOCK
>  int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp);
>  
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index 3e1a8239b0d3..38800fe45931 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -950,3 +950,137 @@ void resctrl_mon_resource_exit(void)
>  
>  	dom_data_exit(r);
>  }
> +
> +/**
> + * resctrl_config_cntr() - Configure the counter ID for the event, RMID pair in
> + * the domain.
> + *
> + * Assign the counter if @assign is true else unassign the counter. Reset the
> + * associated non-architectural state.

A few reports came through about the kernel-doc issues but I did not see a
discussion finalize on how to resolve them. I do not think it is required for these
static functions to have full kernel-doc. Just having useful comments without
kernel-doc style is valuable. Some kernel-doc syntax can still be useful though, like
above when referring to the parameters. It is ok to keep doing so even if section
does not start with /**. 

Where I think kernel-doc is important is include/linux/resctrl.h.

> + */
> +static void resctrl_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
> +				enum resctrl_event_id evtid, u32 rmid, u32 closid,
> +				u32 cntr_id, bool assign)

If resctrl_arch_config_cntr() does not need a struct resource then resctrl_config_cntr()
may not either?

> +{
> +	struct mbm_state *m;
> +
> +	resctrl_arch_config_cntr(r, d, evtid, rmid, closid, cntr_id, assign);
> +
> +	m = get_mbm_state(d, closid, rmid, evtid);
> +	if (m)
> +		memset(m, 0, sizeof(struct mbm_state));

sizeof(*m).

> +}
> +
> +/**
> + * mbm_cntr_get() - Return the counter ID for the matching @evtid and @rdtgrp.
> + *
> + * Return:
> + * Valid counter ID on success, or -ENOENT on failure.
> + */
> +static int mbm_cntr_get(struct rdt_resource *r, struct rdt_mon_domain *d,
> +			struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
> +{
> +	int cntr_id;
> +

Since mbm_cntr_get() is called in regular flows, could you please also
add an explicit check to return -ENOENT if !r->mon.mbm_cntr_assignable?
Otherwise this is quite subtle with the assumption that
r->mon.num_mbm_cntrs is zero in this case.

> +	if (!resctrl_is_mbm_event(evtid))
> +		return -ENOENT;
> +
> +	for (cntr_id = 0; cntr_id < r->mon.num_mbm_cntrs; cntr_id++) {
> +		if (d->cntr_cfg[cntr_id].rdtgrp == rdtgrp &&
> +		    d->cntr_cfg[cntr_id].evtid == evtid)
> +			return cntr_id;
> +	}
> +
> +	return -ENOENT;
> +}
> +
> +/**
> + * mbm_cntr_alloc() - Initilialize and return a new counter ID in the domain @d.

"Initilialize" -> "Initialize"

> + *

mbm_cntr_alloc() will allocate a counter to a RMID/event pair even
if that pair already has a counter assigned. The doc should note that caveat
here with documentation that the caller is responsible for checking that
a counter is not already assigned.

> + * Return:
> + * Valid counter ID on success, or -ENOSPC on failure.
> + */
> +static int mbm_cntr_alloc(struct rdt_resource *r, struct rdt_mon_domain *d,
> +			  struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
> +{
> +	int cntr_id;
> +
> +	for (cntr_id = 0; cntr_id < r->mon.num_mbm_cntrs; cntr_id++) {
> +		if (!d->cntr_cfg[cntr_id].rdtgrp) {
> +			d->cntr_cfg[cntr_id].rdtgrp = rdtgrp;
> +			d->cntr_cfg[cntr_id].evtid = evtid;
> +			return cntr_id;
> +		}
> +	}
> +
> +	return -ENOSPC;
> +}
> +
> +/**
> + * resctrl_alloc_config_cntr() - Allocate a counter ID and configure it for the
> + * event pointed to by @mevt and the resctrl group @rdtgrp within the domain @d.
> + *
> + * Return:
> + * 0 on success, or a non-zero value on failure.

"or a non-zero value on failure." -> "<0 on failure"

> + */
> +static int resctrl_alloc_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
> +				     struct rdtgroup *rdtgrp, struct mon_evt *mevt)
> +{
> +	int cntr_id;
> +
> +	/* No need to allocate a new counter if it is already assigned */
> +	cntr_id = mbm_cntr_get(r, d, rdtgrp, mevt->evtid);
> +	if (cntr_id >= 0)
> +		goto cntr_configure;
> +
> +	cntr_id = mbm_cntr_alloc(r, d, rdtgrp, mevt->evtid);
> +	if (cntr_id <  0) {
> +		rdt_last_cmd_printf("Unable to allocate counter in domain %d\n",
> +				    d->hdr.id);
> +		return cntr_id;
> +	}
> +
> +cntr_configure:
> +	/*
> +	 * Skip reconfiguration if the event setup is current; otherwise,
> +	 * update and apply the new configuration to the domain.

When could "event setup" *not* be current? As mentioned in earlier patch
I do not see why mon_evt::evt_cfg as well as mbm_cntr_cfg::evt_cfg is
needed. There should be no need to keep these two "in sync" with
only mon_evt::evt_cfg as the source of configuration. I seem to be missing
something here, could you please detail this scenario?

> +	 */
> +	if (mevt->evt_cfg != d->cntr_cfg[cntr_id].evt_cfg) {
> +		d->cntr_cfg[cntr_id].evt_cfg = mevt->evt_cfg;
> +		resctrl_config_cntr(r, d, mevt->evtid, rdtgrp->mon.rmid,
> +				    rdtgrp->closid, cntr_id, true);
> +	}
> +
> +	return 0;
> +}
> +
> +/**
> + * resctrl_assign_cntr_event() - Assign a hardware counter for the event in
> + * @mevt to the resctrl group @rdtgrp. Assign counters to all domains if @d is
> + * NULL; otherwise, assign the counter to the specified domain @d.
> + *
> + * If all counters in a domain are already in use, resctrl_alloc_config_cntr()
> + * will fail. The assignment process will abort at the first failure encountered
> + * during domain traversal, which may result in the event being only partially
> + * assigned.
> + *
> + * Return:
> + * 0 on success, or a non-zero value on failure.

"or a non-zero value on failure" -> "<0 on failure"

> + */
> +int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
> +			      struct rdtgroup *rdtgrp, struct mon_evt *mevt)
> +{
> +	int ret = 0;
> +
> +	if (!d) {
> +		list_for_each_entry(d, &r->mon_domains, hdr.list) {
> +			ret = resctrl_alloc_config_cntr(r, d, rdtgrp, mevt);
> +			if (ret)
> +				return ret;
> +		}
> +	} else {
> +		ret = resctrl_alloc_config_cntr(r, d, rdtgrp, mevt);
> +	}
> +
> +	return ret;
> +}

Reinette