mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jonathan Cameron <jic23@kernel.org>
To: Anisa Su <anisa.su887@gmail.com>
Cc: linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev,
	linux-kernel@vger.kernel.org, Dave Jiang <dave.jiang@intel.com>,
	Alison Schofield <alison.schofield@intel.com>,
	Davidlohr Bueso <dave@stgolabs.net>,
	Ira Weiny <iweiny@kernel.org>,
	Vishal Verma <vishal.l.verma@intel.com>,
	Fan Ni <nifan.cxl@gmail.com>, Li Ming <ming.li@zohomail.com>,
	Benjamin Cheatham <benjamin.cheatham@amd.com>,
	Gregory Price <gourry@gourry.net>,
	Richard Cheng <icheng@nvidia.com>,
	Anisa Su <anisa.su@samsung.com>
Subject: Re: [PATCH v1 04/14] cxl/mem: Introduce DC event handlers
Date: Tue, 17 Nov 2026 02:54:27 +0000	[thread overview]
Message-ID: <20261117025427.48bb687d@jic23-hlaptop> (raw)
In-Reply-To: <20261104230445.66625-5-anisa.su@samsung.com>

On Thu,  5 Nov 2026 08:02:28 +0900
Anisa Su <anisa.su887@gmail.com> wrote:

> From: Ira Weiny <iweiny@kernel.org>
> 
> Add the handling skeleton for DC Add and DC Release events. Per CXL
> r4.0 Section 8.2.10.9.9.3, the host is expected to respond to one or
> more Add Capacity Event Record(s) with an Add Dynamic Capacity Response
> (Opcode 4802h) to acknowledge which extents the host accepts for use.
> Similarly, per CXL r4.0 Section 8.2.10.9.9.4, the host is expected to
> respond to Release Capacity Event Record(s) with a Release Dynamic
> Capacity command (Opcode 4803h), or to issue one that is not
> associated with any event to release capacity on its own.
> 
> The records of one operation can be chained by the More flag. Chained
> records are contiguous in the event log and share one event type (CXL
> r4.0 Table 8-229), so a single staging context, mds->dc_chain, holds
> the records of the open chain until the closing record (More=0)
> arrives and the chain is processed and responded to as a whole. A
> record that fails to stage, or one of the other type arriving inside an
> open chain, drops the chain: an add is refused with an empty response,
> a release is left for the device to retry.
> 
> This commit refuses all extents from Add Capacity chains and acks all
> Release Capacity chains. Subsequent commits implement accepting and
> releasing extents.
> 
> Forced Capacity Release and Region Configuration Updated events are
> logged and ignored.
> 
> Based on an original patch by Navneet Singh.
> 
> Signed-off-by: Ira Weiny <iweiny@kernel.org>
> Co-developed-by: Anisa Su <anisa.su@samsung.com>
> Signed-off-by: Anisa Su <anisa.su@samsung.com>
> Tested-by: Wonjae Lee <wj28.lee@samsung.com>
> Tested-by: Junhee Park <jh9934.park@samsung.com>
> Tested-by: Heesoo Kim <habil.kim@samsung.com>
Sashiko does raise an interesting question about trailing events
in a chain after an error.  If it's shared capacity we aren't allowed
a partial accept. Otherwise it is fine (I think) but perhaps not complexity we
want.  So probably need some flag to say problem occured on a record
with more flag set. Keep dropping until we have had a closing record.

For force release, I'm not sure if we should just panic() (maybe
with a sanity check that the range was mapped) or ignore it and
die in a storm of poison (hopefully rather than zeros).

> ---
>  drivers/cxl/core/mbox.c  | 246 ++++++++++++++++++++++++++++++++++++++-
>  drivers/cxl/core/trace.h |   3 +-
>  drivers/cxl/cxl.h        |   9 ++
>  drivers/cxl/cxlmem.h     |  46 ++++++++
>  drivers/cxl/pci.c        |  20 +++-
>  include/cxl/event.h      |  37 ++++++
>  6 files changed, 353 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
> index 13a68f5cef69..3acf5ace48db 100644
> --- a/drivers/cxl/core/mbox.c
> +++ b/drivers/cxl/core/mbox.c

> +static int cxl_send_dc_response(struct cxl_memdev_state *mds, int opcode,
> +				struct list_head *extent_list, int cnt)
> +{
> +	struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
> +	struct cxl_mbox_dc_response *p;
> +	struct cxl_extent_list_node *pos, *tmp;
> +	struct cxl_extent *extent;
> +	u32 pl_index;
> +
> +	size_t pl_size = struct_size(p, extent_list, cnt);
> +	u32 max_extents = cnt;
> +
> +	/* May have to use more bit on response. */
> +	if (pl_size > cxl_mbox->payload_size) {
> +		max_extents = (cxl_mbox->payload_size - sizeof(*p)) /
> +			      sizeof(struct updated_extent_list);
> +		pl_size = struct_size(p, extent_list, max_extents);
> +	}
> +
> +	struct cxl_mbox_dc_response *response __free(kfree) =
> +						kzalloc(pl_size, GFP_KERNEL);
> +	if (!response) {
> +		struct cxl_mbox_dc_response empty = { };
> +
> +		/* An empty Release DC is invalid (CXL r4.0 Table 8-352) */
> +		if (opcode == CXL_MBOX_OP_RELEASE_DC)
> +			return -ENOMEM;
> +		/* Refuse all extents rather than stall the device. */
> +		return send_one_response(cxl_mbox, &empty, opcode, 0, 0);
> +	}
> +
> +	/* Set __counted_by to capacity before populating; reset per response. */
> +	response->extent_list_size = cpu_to_le32(max_extents);
> +
> +	if (cnt == 0)
> +		return send_one_response(cxl_mbox, response, opcode, 0, 0);
> +
> +	pl_index = 0;
> +	list_for_each_entry_safe(pos, tmp, extent_list, list) {
> +		extent = pos->extent;
> +		response->extent_list[pl_index].dpa_start = extent->start_dpa;
> +		response->extent_list[pl_index].length = extent->length;
> +		pl_index++;
> +
> +		if (pl_index == max_extents) {
Maybe flip?
		if (pl_index != max_extents)
			continue;

		flags = 0;

		...

> +			u8 flags = 0;
> +			int rc;
> +
> +			if (pl_index < cnt)
> +				flags |= CXL_DCD_EVENT_MORE;
> +			rc = send_one_response(cxl_mbox, response, opcode,
> +					       pl_index, flags);
> +			if (rc)
> +				return rc;
> +			cnt -= pl_index;
> +			if (cnt < max_extents)
> +				max_extents = cnt;
> +			pl_index = 0;
> +		}
> +	}
> +
> +	if (!pl_index) /* nothing more to do */
> +		return 0;
> +	return send_one_response(cxl_mbox, response, opcode, pl_index, 0);
> +}

> +
> +static int stage_pending_extent(struct list_head *pending,
> +				struct cxl_extent *to_add)
> +{
> +	struct cxl_extent_list_node *node = kzalloc(sizeof(*node), GFP_KERNEL);
> +	struct cxl_extent *extent;
> +
> +	if (!node)
> +		return -ENOMEM;
> +	extent = kmemdup(to_add, sizeof(*extent), GFP_KERNEL);

Maybe scatter some __free() magic in here and no_free_ptr() where
they get handed off..

> +	if (!extent) {
> +		kfree(node);
> +		return -ENOMEM;
> +	}
> +
> +	node->extent = extent;
e.g.
	node->extent = no_free_ptr(extent);

> +	list_add_tail(&node->list, pending);

	list_add_tail(&no_free_ptr(node)->list, pending);

> +	return 0;
> +}

> +
> +static void cxl_handle_dcd_event_records(struct cxl_memdev_state *mds,
> +					 struct cxl_event_record_raw *raw_rec)
> +{
> +	struct cxl_event_dcd *event = &raw_rec->event.dcd;
> +	struct cxl_extent *extent = &event->extent;
> +	struct device *dev = mds->cxlds.dev;
> +	uuid_t *id = &raw_rec->id;
> +	int rc;
> +
> +	if (!uuid_equal(id, &CXL_EVENT_DC_EVENT_UUID))
> +		return;
> +
> +	dev_dbg(dev, "DCD event %s : DPA:%#llx LEN:%#llx\n",
> +		cxl_dcd_evt_type_str(event->event_type),
> +		le64_to_cpu(extent->start_dpa), le64_to_cpu(extent->length));
> +
> +	switch (event->event_type) {
> +	case DCD_ADD_CAPACITY:
> +	case DCD_RELEASE_CAPACITY:
> +		rc = handle_dcd_chain_event(mds, event);
> +		break;
> +	case DCD_FORCED_CAPACITY_RELEASE:
> +		dev_err_ratelimited(dev, "Forced release event ignored.\n");

If that actually happens to us, maybe we should panic?  All bets are off
if we get one of these for capacity that was actually in use.

> +		rc = 0;
> +		break;
> +	case DCD_REGION_CONFIGURATION_UPDATED:
> +		dev_dbg(dev, "Region configuration updated event ignored.\n");
> +		rc = 0;
> +		break;
> +	default:
> +		rc = -EINVAL;
> +		break;
> +	}
> +
> +	if (rc)
> +		dev_err_ratelimited(dev, "dcd event failed: %d\n", rc);
> +}
> +
>  static void cxl_mem_get_records_log(struct cxl_memdev_state *mds,
>  				    enum cxl_event_log_type type)
>  {
> @@ -1136,9 +1368,13 @@ static void cxl_mem_get_records_log(struct cxl_memdev_state *mds,
>  		if (!nr_rec)
>  			break;
>  
> -		for (i = 0; i < nr_rec; i++)
> +		for (i = 0; i < nr_rec; i++) {
>  			__cxl_event_trace_record(cxlmd, type,
>  						 &payload->records[i]);
> +			if (type == CXL_EVENT_TYPE_DCD)
> +				cxl_handle_dcd_event_records(mds,
> +							     &payload->records[i]);

Just go long on that one. I wouldn't do the normal thing of flipping the condition
because this being out of line makes sense as its an oddity in this function.

> +		}
>  
>  		if (payload->flags & CXL_GET_EVENT_FLAG_OVERFLOW)
>  			trace_cxl_overflow(cxlmd, type, payload);

> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index d098f7f50bef..64138e1768a1 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -12,6 +12,7 @@
>  #include <linux/node.h>
>  #include <linux/io.h>
>  #include <linux/range.h>
> +#include <linux/xarray.h>

This looks misplaces as nothing in the header changed
that has anything to do with xarrays.

>  #include <cxl/cxl.h>
>  
>  extern const struct nvdimm_security_ops *cxl_security_ops;
> @@ -557,6 +558,14 @@ struct cxl_pmem_region {
>  	struct cxl_pmem_region_mapping mapping[];
>  };
>  
> +/* See CXL r4.0 8.2.10.2.1.6 */
> +enum dc_event {
> +	DCD_ADD_CAPACITY,
> +	DCD_RELEASE_CAPACITY,
> +	DCD_FORCED_CAPACITY_RELEASE,
> +	DCD_REGION_CONFIGURATION_UPDATED,
> +};
> +
>  struct cxl_dax_region {
>  	struct device dev;
>  	struct cxl_region *cxlr;


  reply	other threads:[~2026-11-17  2:54 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-04 23:02 [PATCH v1 00/14] DCD CXL Series: Dynamic Capacity extent handling Anisa Su
2026-11-04 23:02 ` [PATCH v1 01/14] cxl/core: Return endpoint decoder information from region search Anisa Su
2026-11-04 23:02 ` [PATCH v1 02/14] cxl/port: Add dynamic capacity options to endpoint decoder mode Anisa Su
2026-11-17  2:22   ` Jonathan Cameron
2026-11-04 23:02 ` [PATCH v1 03/14] cxl/region: Add dynamic DAX region support Anisa Su
2026-11-17  2:32   ` Jonathan Cameron
2026-11-04 23:02 ` [PATCH v1 04/14] cxl/mem: Introduce DC event handlers Anisa Su
2026-11-17  2:54   ` Jonathan Cameron [this message]
2026-11-04 23:02 ` [PATCH v1 05/14] cxl/mem: Add 20 second timeout for stalled DC event chains Anisa Su
2026-11-17  3:00   ` Jonathan Cameron
2026-11-04 23:02 ` [PATCH v1 06/14] cxl/extent: Handle DC Add Capacity events Anisa Su
2026-11-12  3:34   ` Richard Cheng
2026-11-16 21:53     ` Anisa Su
2026-11-04 23:02 ` [PATCH v1 07/14] cxl/mem: Drop misaligned DCD extent groups Anisa Su
2026-11-04 23:02 ` [PATCH v1 08/14] cxl/extent: Validate DC extent partition Anisa Su
2026-11-04 23:02 ` [PATCH v1 09/14] cxl/mem: Enforce tag-group semantics Anisa Su
2026-11-04 23:02 ` [PATCH v1 10/14] cxl/extent: Enforce cross-region tag uniqueness Anisa Su
2026-11-04 23:02 ` [PATCH v1 11/14] cxl/extent: Handle DC Release Capacity events Anisa Su
2026-11-12  7:34   ` Richard Cheng
2026-11-04 23:02 ` [PATCH v1 12/14] cxl/region: Read existing extents on region creation Anisa Su
2026-11-04 23:02 ` [PATCH v1 13/14] cxl/mem: Trace Dynamic capacity Event Record Anisa Su
2026-11-04 23:02 ` [PATCH v1 14/14] Documentation/cxl: Document DCD extent handling Anisa Su

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261117025427.48bb687d@jic23-hlaptop \
    --to=jic23@kernel.org \
    --cc=alison.schofield@intel.com \
    --cc=anisa.su887@gmail.com \
    --cc=anisa.su@samsung.com \
    --cc=benjamin.cheatham@amd.com \
    --cc=dave.jiang@intel.com \
    --cc=dave@stgolabs.net \
    --cc=gourry@gourry.net \
    --cc=icheng@nvidia.com \
    --cc=iweiny@kernel.org \
    --cc=linux-cxl@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ming.li@zohomail.com \
    --cc=nifan.cxl@gmail.com \
    --cc=nvdimm@lists.linux.dev \
    --cc=vishal.l.verma@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®