mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ben Horgan <ben.horgan@arm.com>
To: Jinjie Ruan <ruanjinjie@huawei.com>,
	tony.luck@intel.com, reinette.chatre@intel.com,
	Dave.Martin@arm.com, james.morse@arm.com, babu.moger@amd.com,
	corbet@lwn.net, skhan@linuxfoundation.org, rdunlap@infradead.org,
	bp@alien8.de, tan.shaopeng@fujitsu.com, gshan@redhat.com,
	jic23@kernel.org, fenghuay@nvidia.com, zengheng4@huawei.com,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	linux-doc@vger.kernel.org
Subject: Re: [PATCH v6 4/6] fs/resctrl: Stop the limbo handler while resctrl is unmounted
Date: Thu, 19 Nov 2026 12:34:18 +0000	[thread overview]
Message-ID: <daa2a116-a767-42bc-a31f-f1da84e1eeae@arm.com> (raw)
In-Reply-To: <20261119094858.813874-5-ruanjinjie@huawei.com>

Hi Jinjie,

On 19/11/2026 09:48, Jinjie Ruan wrote:
> A monitoring group's RMID is not returned to the free list when the
> group is removed. It stays in limbo until the cache occupancy still
> counted against it has dropped low enough for the RMID to be reused,
> which the limbo handler checks once a second.
> 
> The limbo handler keeps running after resctrl is unmounted, where there
> is nothing for it to achieve. An RMID is only allocated when a group is
> created, which needs resctrl to be mounted, so no RMID the limbo handler
> returns to the free list can be used before the next mount, and that
> mount can check them itself. The MBM overflow handler already stops for
> this reason once resctrl_mounted is cleared.
> 
> Reading the RMIDs in limbo while resctrl is unmounted is not safe on
> every architecture. On MPAM the counter of an RMID is selected by the
> CLOSID that owns it, and the CLOSID depends on the CDP state. Unmount
> clears the CDP state, so the limbo handler that keeps running reads the
> RMIDs freed under CDP through the CLOSIDs of the cleared state.
> 
> Stop the limbo handler while resctrl is unmounted, as the overflow
> handler does, and restart it after a mount on every domain that still
> has RMIDs in limbo. The worker can stay queued across the unmount, so
> cancel it before queuing it again.
> 
> Fixes: 6789fb99282c ("arm_mpam: resctrl: Add CDP emulation")
> Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Cc: <stable@vger.kernel.org>
> ---
> v6:
> - Explain first why stopping the limbo handler is fine on every
>   architecture, and describe MPAM by CLOSID only (Reinette).
> - Rename resctrl_limbo_resume() to cqm_resume_limbo_handler() and declare
>   it with the other limbo handler helpers (Reinette).
> - Return early when the L3 resource is not mon_capable (Reinette).
> - Drop the stable version range, the Fixes tag already gives it.
> ---
>  fs/resctrl/internal.h |  2 ++
>  fs/resctrl/monitor.c  | 42 ++++++++++++++++++++++++++++++++++++++++--
>  fs/resctrl/rdtgroup.c |  2 ++
>  3 files changed, 44 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
> index 1c85e06f09bf..f52e0937702a 100644
> --- a/fs/resctrl/internal.h
> +++ b/fs/resctrl/internal.h
> @@ -401,6 +401,8 @@ bool is_mba_sc(struct rdt_resource *r);
>  void cqm_setup_limbo_handler(struct rdt_l3_mon_domain *dom, unsigned long delay_ms,
>  			     int exclude_cpu);
>  
> +void cqm_resume_limbo_handler(void);
> +
>  void cqm_handle_limbo(struct work_struct *work);
>  
>  bool has_busy_rmid(struct rdt_l3_mon_domain *d);
> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index ae9ddd380515..ede1708b920e 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -852,6 +852,13 @@ void cqm_handle_limbo(struct work_struct *work)
>  	 */
>  	mutex_lock(&rdtgroup_mutex);
>  
> +	/*
> +	 * If the filesystem has been unmounted this work no longer needs to
> +	 * run. cqm_resume_limbo_handler() restarts it on the next mount.
> +	 */
> +	if (!resctrl_mounted)
> +		goto out_unlock;
> +
>  	/*
>  	 * Ensure the worker is dedicated to a CPU as intended and not
>  	 * relocated by workqueue subsystem as part of CPU going offline.
> @@ -1001,8 +1008,8 @@ int setup_rmid_lru_list(void)
>  	/*
>  	 * Called on every mount, but the number of RMIDs cannot change
>  	 * after the first mount, so keep using the same set of rmid_ptrs[]
> -	 * until resctrl_exit(). Note that the limbo handler continues to
> -	 * access rmid_ptrs[] after resctrl is unmounted.
> +	 * until resctrl_exit(). Note that the RMIDs in limbo stay in limbo
> +	 * across an unmount and are checked again after the next mount.
>  	 */
>  	if (rmid_ptrs)
>  		return 0;
> @@ -1033,6 +1040,37 @@ int setup_rmid_lru_list(void)
>  	return 0;
>  }
>  
> +/**
> + * cqm_resume_limbo_handler() - Restart the limbo handler after a mount.
> + *
> + * The limbo handler stops while resctrl is unmounted. Restart it on every
> + * domain that still has RMIDs in limbo.
> + */
> +void cqm_resume_limbo_handler(void)
> +{
> +	struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3);
> +	struct rdt_l3_mon_domain *d;
> +
> +	lockdep_assert_cpus_held();
> +	lockdep_assert_held(&rdtgroup_mutex);
> +
> +	if (!r->mon_capable || !resctrl_is_mon_event_enabled(QOS_L3_OCCUP_EVENT_ID))
> +		return;
> +
> +	list_for_each_entry_rcu(d, &r->mon_domains, hdr.list,
> +				lockdep_is_cpus_held()) {

nit: Unnecessary line break.

Reviewed-by: Ben Horgan <ben.horgan@arm.com>

Thanks,

Ben

> +		if (!has_busy_rmid(d))
> +			continue;
> +
> +		/*
> +		 * The unmount may have left the worker queued. Cancel it so
> +		 * it is queued again on a CPU picked now.
> +		 */
> +		cancel_delayed_work(&d->cqm_limbo);
> +		cqm_setup_limbo_handler(d, 0, RESCTRL_PICK_ANY_CPU);
> +	}
> +}
> +
>  void free_rmid_lru_list(void)
>  {
>  	if (!resctrl_arch_mon_capable())
> diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
> index 3312637fb514..c7c4f618208d 100644
> --- a/fs/resctrl/rdtgroup.c
> +++ b/fs/resctrl/rdtgroup.c
> @@ -3267,6 +3267,8 @@ static int rdt_get_tree(struct fs_context *fc)
>  						   RESCTRL_PICK_ANY_CPU);
>  	}
>  
> +	cqm_resume_limbo_handler();
> +
>  	/*
>  	 * Ensure root remains accessible after mutex is unlocked so that
>  	 * kernfs_kill_sb() can run safely if called by kernfs_get_tree()'s


  reply	other threads:[~2026-11-19 12:34 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  9:48 [PATCH v6 0/6] fs/resctrl: Fix some resctl bugs related to mpam Jinjie Ruan
2026-11-19  9:48 ` [PATCH v6 1/6] fs/resctrl: Report a temporary RMID shortage per CLOSID Jinjie Ruan
2026-11-19  9:48 ` [PATCH v6 2/6] fs/resctrl: Do not allocate a monitor context when releasing limbo RMIDs Jinjie Ruan
2026-11-19  9:48 ` [PATCH v6 3/6] fs/resctrl: Clear the CDP state after the cleanup Jinjie Ruan
2026-11-19 11:21   ` Ben Horgan
2026-11-19 12:15     ` Jinjie Ruan
2026-11-19  9:48 ` [PATCH v6 4/6] fs/resctrl: Stop the limbo handler while resctrl is unmounted Jinjie Ruan
2026-11-19 12:34   ` Ben Horgan [this message]
2026-11-19  9:48 ` [PATCH v6 5/6] fs/resctrl: Release the limbo RMIDs when the CDP state changes Jinjie Ruan
2026-11-19  9:48 ` [PATCH v6 6/6] fs/resctrl: Disallow monitor group move when the RMID index changes Jinjie Ruan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=daa2a116-a767-42bc-a31f-f1da84e1eeae@arm.com \
    --to=ben.horgan@arm.com \
    --cc=Dave.Martin@arm.com \
    --cc=babu.moger@amd.com \
    --cc=bp@alien8.de \
    --cc=corbet@lwn.net \
    --cc=fenghuay@nvidia.com \
    --cc=gshan@redhat.com \
    --cc=james.morse@arm.com \
    --cc=jic23@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=reinette.chatre@intel.com \
    --cc=ruanjinjie@huawei.com \
    --cc=skhan@linuxfoundation.org \
    --cc=tan.shaopeng@fujitsu.com \
    --cc=tony.luck@intel.com \
    --cc=x86@kernel.org \
    --cc=zengheng4@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®