[PATCH v1 00/14] DCD CXL Series: Dynamic Capacity extent handling

[PATCH v1 00/14] DCD CXL Series: Dynamic Capacity extent handling

[Anisa Su]

As discussed previously, the remaining DCD series will be split for
reviewability. I made the following split:

    DCD CXL Core half (current series) enables:
        - pointing an endpoint decoder at a DC partition
        - creating dax_cxl region for DCD (0-sized until resources added)
        - processing Add / Release Capacity events
        - sending DC Add/Release Response back to the device.

DCD stays disabled here too. The follow up:

  DCD DAX Series:
                - create dax_resources from accepted extents
                - build DAX devices from tag groups
                - prevent opening devdax w/write permission on read-only partition
                - expose tag_groups / extents in sysfs,
                - create_dynamic_ram*_region root decoder files for manual region creation
                - tools/testing/cxl additions for DCD
                - enable DCD

	^ NDCTL set goes with this one

Nothing in this series is reachable at runtime as posted: with
dcd_supported false no DC partition exists, the decoder mode store
refuses the dynamic_ram modes, no DC region can be created, and no
DCD event is ever read.  The intent is to get the extent handling
reviewed on its own while the DAX side follows.  Testing below is done
with the full series.

Patch Organization
==================
   1: (helper) region search returns the endpoint decoder for a DPA.
   2: Add dynamic_ram* endpoint decoder modes.
   3: Set up DC DAX region: 0-size seed device, with 0 HPA until extents
      are added as children
 4-5: Set up event handler pipeline with stubbed add/release logic
6-10: Add Capacity: Fill in add logic, introduce cxl_dc_tag_group /
      dc_extent objects, verify extent alignement, shared_extn_seq,
      check tag uniqueness, etc.
  11: Release Capacity: release tag groups as a whole, send Release
      DC response.
  12: Recover existing extents when a DC region's dax_region probes
  13: Rrace event for Dynamic Capacity event records.
  14: Documentation of the extent handling.

It will probably be useful to read over patch 14 (documentation) first
which explains how tags/shared extent sequence #s are handled.


[RFC] Sysfs interface
=====================
Since the sysfs interface spans both series, it's described here as
a whole so the decoder modes in patch 2 can be read in context. The
whole section is RFC, but especially endpoint decoder mode and the
root decoder create region files.

Memdev partition (exposed in follow up series):
        memX/ram/{size,qos_class}
        memX/pmem/{size,qos_class}
        memX/dynamic_ram_1/{size,qos_class,shareable,read_only}

Currently, only 1 DC partition is supported, but multiple would look
like this:
        memX/dynamic_ram_1/{size,qos_class,shareable,read_only}
        memX/dynamic_ram_2/{size,qos_class,shareable,read_only}
        ...
        memX/dynamic_ram_8/{size,qos_class,shareable,read_only}

dynamic_ram_X tells users which index they are looking at.

* Endpoint decoder mode (patch 2):
        decoderX.Y/mode/{dynamic ram |
                         dynamic_ram_shared |
                         dynamic_ram_ro |
                         dynamic_ram_shared_ro}

The code exists in the current series, but is not exposed because
dcd is turned off. It is necessary to keep in the current series
rather deferred for the follow up because extents need a parent
DAX_CXL region that is dc-capable, which requires an endpoint decoder with
dynamic_ram* mode

A decoder's DPA is allocated from a DC partition by writing one
of dynamic_ram, dynamic_ram_shared, dynamic_ram_ro or
dynamic_ram_shared_ro to decoderX.Y/mode. We only support 1 DC
partition today, so it must match the partition's read_only
and shareable attributes exactly. But for multiple, the intent
is to iterate over DC partitions and find the first one that
matches. If none match, it's refused with -ENXIO.

So users can specify "I want to use the shareable partition", or
"I want to use the read_only partition".

But if there were a device with multiple DC partitions that have
the same shareable/read-only attributes, the user will not be
able to select the specific partition.

So the other choice is to have dynamic_ram_{1-8} as the possible
modes (so in this series with only the first partition enabled,
the only choice would be dynamic_ram_1).

* Root decoder create files & region mode (in follow-up series):
        decoder0.0/create_dynamic_ram_region |
                   create_dynamic_ram_shared_region |
                   create_dynamic_ram_ro_region |
                   create_dynamic_ram_shared_ro_region

        regionX/mode/{dynamic ram |
                      dynamic_ram_shared |
                      dynamic_ram_ro |
                      dynamic_ram_shared_ro}

The decoder0.0/create_dynamic_ram_* fixes the region's mode at creation,
and every endpoint decoder attached must match the same mode.

The alternative is to have a single file:
        decoder0.0/create_dynamic_ram

which will create regionN with no mode. The mode is set when endpoint
decoderX.Y is written to target0 by inheriting the decoder's mode.

Tag Groups & Extents (in follow-up series)

  dax_regionX/tag_groupX.Y/{uuid,size}
                          /extentX.Y/{offset,length[,shared_extent_seq]}

  dax_regionX/untagged/{uuid (0000-000....)}/
                        extentX.Y/{offset,length}

Accepted capacity appears under the region's dax_regionX grouped by tag.
Under each tag_groupX.Y directory, is the group's UUID, combined size
of all extents in the group, and each extentX.Y in the group.
Each extentX.Y shows its offset, length, and if shared: shared_extn_seq.

Untagged extents appear under a general "untagged" directory, with
UUID = null uuid and *no size*, since each untagged extent is its
own logical allocation.

Git Info:
=========
Based on v7.4-rc1.

GitHub branches:
  this series:  https://github.com/anisa-su993/anisa-linux-kernel/tree/dcd-cxl-v1
  DAX series:   https://github.com/anisa-su993/anisa-linux-kernel/tree/dcd-dax-v1

NDCTL:
https://github.com/anisa-su993/anisa-ndctl/tree/dcd-v8

Testing
=======
Build: every patch of the series builds drivers/cxl and drivers/dax
with CONFIG_CXL_REGION=y and =n.

NDCTL cxl_test:
- no regressions
- cxl-dcd.sh also passing although ndctl set is not posted with this series and
the tools/testing changes for mock DC regions is in the follow-up series

Hardware: a DCD with one sharable, writable 128 GiB DC partition:
[   26.524021] cxl_core:cxl_get_dc_config:2377: cxl_pci 0000:0b:00.0: Read 1/1 DC partitions
[   26.524024] cxl_core:cxl_dc_check:2327: cxl_pci 0000:0b:00.0: DC partition 0 start 0x0 size 0x2000000000 blk_size: 0x10000000
[   26.524028] cxl_core:cxl_dev_dc_identify:2473: cxl_pci 0000:0b:00.0: Returning partition 0 0x0 size 0x2000000000
[   26.524031] cxl_core:cxl_configure_dcd:778: cxl_pci 0000:0b:00.0: Adding dynamic ram partition; 0x0 size 0x2000000000
[   26.524036] cxl_core:add_dpa_res:468: cxl_pci 0000:0b:00.0: DPA(dynamic_ram_1): [mem 0x00000000-0x1fffffffff flags 0x200]

Test the following driven by a fabric manager over MCTP from the platform BMC (Initiate
Dynamic Capacity Add and Release, prescriptive and tag-based):

- FMAPI Add (one More-chain): Tag_A{extent_1}, Tag_B{extent_2-4}
        - extents 2-4 with shared seq num 0, 1, 2

- Create DAX device from extent 1 and try mmap:
        anisa@cxlqual:~/ndctl$
        sudo python3 - <<'EOF'
        import mmap, os
        dev, size = "/dev/dax0.1", 0x80000000
        fd = os.open(dev, os.O_RDWR)
        m = mmap.mmap(fd, size)
        m[0:8] = b"DCDHEAD!"                # first page
        m[size - 8:size] = b"DCDTAIL!"      # last page
        m.close(); os.close(fd)
        fd = os.open(dev, os.O_RDONLY)
        m = mmap.mmap(fd, size, prot=mmap.PROT_READ)
        print(m[0:8], m[size - 8:size])
        m.close(); os.close(fd)
        EOF
        b'DCDHEAD!' b'DCDTAIL!'
        anisa@cxlqual:~/ndctl$

Try to add a tag that is already committed:
[ 1119.080597] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4801
[ 1119.080646] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6606.397305] cxl_core:cxl_mem_get_event_records:2053: cxl_pci 0000:0b:00.0: Reading event logs: 10
[ 6606.397322] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0100
[ 6606.397359] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6606.397427] cxl_core:cxl_handle_dcd_event_records:1955: cxl_pci 0000:0b:00.0: DCD event add : DPA:0xb0000000 LEN:0x10000000
[ 6606.397439] cxl_pci 0000:0b:00.0: Tag 1b2c3d4e-5f60-4718-9a2b-3c4d5e6f7081: dropping group, tag already committed (firmware/orchestrator bug)
[ 6606.397451] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4802
[ 6606.397484] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6606.397491] cxl_core:cxl_clear_event_record:1092: cxl_pci 0000:0b:00.0: Event log '4': Clearing 27
[ 6606.397497] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0101
[ 6606.397536] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6606.397544] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0100
[ 6606.397569] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms

Reject untagged extent on sharable partition:
[ 6820.356969] cxl_core:cxl_handle_dcd_event_records:1955: cxl_pci 0000:0b:00.0: DCD event add : DPA:0xc0000000 LEN:0x10000000
[ 6820.356981] cxl_core:cxl_validate_extent:341: cxl_pci 0000:0b:00.0: DC extent DPA [range 0x00000000c0000000-0x00000000cfffffff] (DCR:[mem 0x00000000-0x1fffffffff])(00000000-0000-0000-0000-000000000000)
[ 6820.356993] cxl_pci 0000:0b:00.0: DC extent DPA [range 0x00000000c0000000-0x00000000cfffffff]: sharable-partition extent has null tag (firmware bug)
[ 6820.357003] cxl_core:cxl_realize_group:1612: cxl_pci 0000:0b:00.0: Tag 00000000-0000-0000-0000-000000000000: dropping group, failed to add extent DPA:0xc0000000 LEN:0x10000000
[ 6820.357013] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4802
[ 6820.357046] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6820.357053] cxl_core:cxl_clear_event_record:1092: cxl_pci 0000:0b:00.0: Event log '4': Clearing 28
[ 6820.357059] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0101
[ 6820.357099] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 6820.357105] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0100
[ 6820.357130] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms

Unbind dax_cxl driver releases extents:
[ 6922.815346] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4803
[ 6922.815824] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 1ms
[ 6922.815830] cxl_core:unregister_tag_group:737:  extent0.2: DAX region rm extent HPA [range 0x0000000080000000-0x000000008fffffff]
[ 6922.815858] cxl_core:unregister_tag_group:737:  extent0.3: DAX region rm extent HPA [range 0x0000000090000000-0x000000009fffffff]
[ 6922.815866] cxl_core:unregister_tag_group:737:  extent0.4: DAX region rm extent HPA [range 0x00000000a0000000-0x00000000afffffff]
[ 6922.935361] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4803
[ 6922.935542] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 1ms
[ 6922.935547] cxl_core:unregister_tag_group:737:  extent0.1: DAX region rm extent HPA [range 0x0000000000000000-0x000000007fffffff]

Refuse add extents while dax_cxl not bound:
[ 7064.900283] cxl_core:cxl_handle_dcd_event_records:1955: cxl_pci 0000:0b:00.0: DCD event add : DPA:0xc0000000 LEN:0x10000000
[ 7064.900296] cxl_core:cxl_validate_extent:341: cxl_pci 0000:0b:00.0: DC extent DPA [range 0x00000000c0000000-0x00000000cfffffff] (DCR:[mem 0x00000000-0x1fffffffff])(55555555-5555-5555-5555-555555555555)
[ 7064.900313] cxl_core:__cxl_dpa_to_region:3137: cxl decoder9.0: dpa:0xc0000000 mapped in region:region0
[ 7064.900320] cxl_pci 0000:0b:00.0: DC extent DPA [range 0x00000000c0000000-0x00000000cfffffff]: dax driver not ready, refusing
[ 7064.900327] cxl_core:cxl_realize_group:1612: cxl_pci 0000:0b:00.0: Tag 55555555-5555-5555-5555-555555555555: dropping group, failed to add extent DPA:0xc0000000 LEN:0x10000000
[ 7064.900337] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4802
[ 7064.900370] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[ 7064.900377] cxl_core:cxl_clear_event_record:1092: cxl_pci 0000:0b:00.0: Event log '4': Clearing 33
[ 7064.900384] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0101
[ 7064.900453] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 1ms
[ 7064.900461] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x0100
[ 7064.900486] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms

------ Reboot --------

Read previously accepted extents on reboot:
[   26.570494] cxl_pci:cxl_event_config:801: cxl_pci 0000:0b:00.0: Event config : BIOS DCD supported
[   26.872701] cxl_core:cxl_bus_probe:2412: cxl_pmu pmu_mem0.0: probe: 0
[   27.514742] cxl_pci:__cxl_pci_mbox_send_cmd:264: cxl_pci 0000:0b:00.0: Sending command: 0x4801
[   27.514797] cxl_pci:cxl_pci_mbox_wait_for_doorbell:75: cxl_pci 0000:0b:00.0: Doorbell wait took 0ms
[   27.514826] cxl_core:cxl_read_extent_list:2546: cxl_pci 0000:0b:00.0: Got extent list 0-3 of 4 generation Num:10
[   27.514829] cxl_core:cxl_read_extent_list:2570: cxl_pci 0000:0b:00.0: Processing extent 0/4 [range 0x0000000000000000-0x000000007fffffff] (e5407b12-9a68-4c31-8d5e-2f0916bd73a4) seq 0
[   27.514833] cxl_core:cxl_read_extent_list:2570: cxl_pci 0000:0b:00.0: Processing extent 1/4 [range 0x0000000080000000-0x000000008fffffff] (1b2c3d4e-5f60-4718-9a2b-3c4d5e6f7081) seq 0
[   27.514836] cxl_core:cxl_read_extent_list:2570: cxl_pci 0000:0b:00.0: Processing extent 2/4 [range 0x0000000090000000-0x000000009fffffff] (1b2c3d4e-5f60-4718-9a2b-3c4d5e6f7081) seq 0
[   27.514838] cxl_core:cxl_read_extent_list:2570: cxl_pci 0000:0b:00.0: Processing extent 3/4 [range 0x00000000a0000000-0x00000000afffffff] (1b2c3d4e-5f60-4718-9a2b-3c4d5e6f7081) seq 0
[   27.514843] cxl_core:cxl_validate_extent:341: cxl_pci 0000:0b:00.0: DC extent DPA [range 0x0000000000000000-0x000000007fffffff] (DCR:[mem 0x00000000-0x1fffffffff])(e5407b12-9a68-4c31-8d5e-2f0916bd73a4)
[   27.514849] cxl_core:__cxl_dpa_to_region:3137: cxl decoder9.0: dpa:0x0 mapped in region:region0
[   27.514852] cxl_core:cxl_add_extent:570: cxl decoder9.0: Add extent [range 0x0000000000000000-0x000000007fffffff] (e5407b12-9a68-4c31-8d5e-2f0916bd73a4)
[   27.514855] cxl_core:cxlr_add_extent:496: cxl_dax_region dax_region0: Alloc new tag group
[   27.514873] cxl_core:online_tag_group:842:  extent0.1: dc_extent HPA [range 0x0000000000000000-0x000000007fffffff] (e5407b12-9a68-4c31-8d5e-2f0916bd73a4)
[   27.514875] cxl_core:cxlr_notify_extent:470: cxl_dax_region dax_region0: Notify: type 0 tag e5407b12-9a68-4c31-8d5e-2f0916bd73a4


Anisa Su (5):
  cxl/port: Add dynamic capacity options to endpoint decoder mode
  cxl/mem: Add 20 second timeout for stalled DC event chains
  cxl/mem: Enforce tag-group semantics
  cxl/extent: Enforce cross-region tag uniqueness
  Documentation/cxl: Document DCD extent handling

Ira Weiny (9):
  cxl/core: Return endpoint decoder information from region search
  cxl/region: Add dynamic DAX region support
  cxl/mem: Introduce DC event handlers
  cxl/extent: Handle DC Add Capacity events
  cxl/mem: Drop misaligned DCD extent groups
  cxl/extent: Validate DC extent partition
  cxl/extent: Handle DC Release Capacity events
  cxl/region: Read existing extents on region creation
  cxl/mem: Trace Dynamic capacity Event Record

 Documentation/ABI/testing/sysfs-bus-cxl       |   30 +-
 .../driver-api/cxl/linux/cxl-driver.rst       |  181 +++
 Documentation/driver-api/cxl/maturity-map.rst |    2 +-
 drivers/cxl/core/Makefile                     |    2 +-
 drivers/cxl/core/core.h                       |  105 +-
 drivers/cxl/core/extent.c                     |  754 ++++++++++++
 drivers/cxl/core/hdm.c                        |   21 +-
 drivers/cxl/core/mbox.c                       | 1073 ++++++++++++++++-
 drivers/cxl/core/memdev.c                     |    4 +-
 drivers/cxl/core/port.c                       |   55 +-
 drivers/cxl/core/region.c                     |   74 +-
 drivers/cxl/core/region_dax.c                 |  131 +-
 drivers/cxl/core/trace.h                      |   79 +-
 drivers/cxl/cxl.h                             |  121 ++
 drivers/cxl/cxlmem.h                          |   84 ++
 drivers/cxl/pci.c                             |   20 +-
 drivers/dax/bus.c                             |   23 +
 drivers/dax/bus.h                             |    1 +
 drivers/dax/cxl.c                             |   40 +-
 include/cxl/event.h                           |   37 +
 tools/testing/cxl/Kbuild                      |    5 +-
 21 files changed, 2792 insertions(+), 50 deletions(-)
 create mode 100644 drivers/cxl/core/extent.c


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.43.0

[PATCH v1 01/14] cxl/core: Return endpoint decoder information from region search

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

cxl_dpa_to_region() finds the region from a <DPA, device> tuple.
The search involves finding the device endpoint decoder as well.

Dynamic capacity extent processing uses the endpoint decoder HPA
information to calculate the HPA offset.  In addition, well behaved
extents should be contained within an endpoint decoder.

Return the endpoint decoder found to be used in subsequent DCD code.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
Reviewed-by: Jonathan Cameron <jic23@kernel.org>
Reviewed-by: Fan Ni <nifan.cxl@gmail.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Reviewed-by: Li Ming <ming.li@zohomail.com>
Reviewed-by: Alison Schofield <alison.schofield@intel.com>
---
 drivers/cxl/core/core.h   |  8 ++++++--
 drivers/cxl/core/mbox.c   |  2 +-
 drivers/cxl/core/memdev.c |  4 ++--
 drivers/cxl/core/region.c | 12 +++++++++++-
 4 files changed, 20 insertions(+), 6 deletions(-)

diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 79918fde1575..8a81be004d00 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -47,7 +47,8 @@ int cxl_decoder_detach(struct cxl_region *cxlr,
 int cxl_region_init(void);
 void cxl_region_exit(void);
 int cxl_get_poison_by_endpoint(struct cxl_port *port);
-struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa);
+struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa,
+				     struct cxl_endpoint_decoder **cxled);
 u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd,
 		   u64 dpa);
 int devm_cxl_add_dax_region(struct cxl_region *cxlr);
@@ -61,8 +62,11 @@ static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr,
 	return ULLONG_MAX;
 }
 static inline
-struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)
+struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa,
+				     struct cxl_endpoint_decoder **cxled)
 {
+	if (cxled)
+		*cxled = NULL;
 	return NULL;
 }
 static inline int cxl_get_poison_by_endpoint(struct cxl_port *port)
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 39485b6bcb97..13a68f5cef69 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -966,7 +966,7 @@ void cxl_event_trace_record(struct cxl_memdev *cxlmd,
 		guard(rwsem_read)(&cxl_rwsem.dpa);
 
 		dpa = le64_to_cpu(evt->media_hdr.phys_addr) & CXL_DPA_MASK;
-		cxlr = cxl_dpa_to_region(cxlmd, dpa);
+		cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL);
 		if (cxlr) {
 			u64 cache_size = cxlr->params.cache_size;
 
diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c
index 787988e92965..81f1e0441787 100644
--- a/drivers/cxl/core/memdev.c
+++ b/drivers/cxl/core/memdev.c
@@ -309,7 +309,7 @@ int cxl_inject_poison_locked(struct cxl_memdev *cxlmd, u64 dpa)
 	if (rc)
 		return rc;
 
-	cxlr = cxl_dpa_to_region(cxlmd, dpa);
+	cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL);
 	if (cxlr)
 		dev_warn_once(cxl_mbox->host,
 			      "poison inject dpa:%#llx region: %s\n", dpa,
@@ -378,7 +378,7 @@ int cxl_clear_poison_locked(struct cxl_memdev *cxlmd, u64 dpa)
 	if (rc)
 		return rc;
 
-	cxlr = cxl_dpa_to_region(cxlmd, dpa);
+	cxlr = cxl_dpa_to_region(cxlmd, dpa, NULL);
 	if (cxlr)
 		dev_warn_once(cxl_mbox->host,
 			      "poison clear dpa:%#llx region: %s\n", dpa,
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index a048bd5c06cb..f4a9839c73d2 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -3045,6 +3045,7 @@ int cxl_get_poison_by_endpoint(struct cxl_port *port)
 struct cxl_dpa_to_region_context {
 	struct cxl_region *cxlr;
 	u64 dpa;
+	struct cxl_endpoint_decoder *cxled;
 };
 
 static int __cxl_dpa_to_region(struct device *dev, void *arg)
@@ -3078,15 +3079,21 @@ static int __cxl_dpa_to_region(struct device *dev, void *arg)
 			dev_name(dev));
 
 	ctx->cxlr = cxlr;
+	ctx->cxled = cxled;
 
 	return 1;
 }
 
-struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)
+/* @cxled receives the decoder mapping @dpa, if any, even with no region. */
+struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa,
+				     struct cxl_endpoint_decoder **cxled)
 {
 	struct cxl_dpa_to_region_context ctx;
 	struct cxl_port *port = cxlmd->endpoint;
 
+	if (cxled)
+		*cxled = NULL;
+
 	if (!cxlmd->dev.driver)
 		return NULL;
 
@@ -3096,6 +3103,9 @@ struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa)
 	if (cxl_num_decoders_committed(port))
 		device_for_each_child(&port->dev, &ctx, __cxl_dpa_to_region);
 
+	if (cxled)
+		*cxled = ctx.cxled;
+
 	return ctx.cxlr;
 }
 
-- 
2.43.0

[PATCH v1 02/14] cxl/port: Add dynamic capacity options to endpoint decoder mode

[Anisa Su]

Endpoints can now carry a dynamic ram partition following the
persistent memory partition. A DC partition is further described by
two properties reported in its CDAT DSMAS entry: shareable and
read-only.

Expand the decoder mode to accept 'dynamic_ram', 'dynamic_ram_shared',
'dynamic_ram_ro' and 'dynamic_ram_shared_ro'. cxl_dpa_set_part()
selects the first dynamic ram partition whose properties match exactly.

DCD may be disabled during validation after the DC partition(s) were
added to the DPA layout, so a decoder may only be assigned to a DC
partition while DCD is enabled for its memdev. Add a helper,
cxled_dcd_enabled(), to check cxl_dcd_supported().

Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 Documentation/ABI/testing/sysfs-bus-cxl | 25 +++++++----
 drivers/cxl/core/core.h                 | 23 ++++++++++-
 drivers/cxl/core/hdm.c                  | 21 ++++++++--
 drivers/cxl/core/port.c                 | 55 ++++++++++++++++++++++---
 4 files changed, 105 insertions(+), 19 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
index 7352dbd70bc7..bcbc44fea6db 100644
--- a/Documentation/ABI/testing/sysfs-bus-cxl
+++ b/Documentation/ABI/testing/sysfs-bus-cxl
@@ -339,22 +339,29 @@ Description:
 
 
 What:		/sys/bus/cxl/devices/decoderX.Y/mode
-Date:		May, 2022
-KernelVersion:	v6.0
+Date:		May, 2022, November, 2026
+KernelVersion:	v6.0, v7.5 (dynamic_ram)
 Contact:	linux-cxl@vger.kernel.org
 Description:
 		(RW) When a CXL decoder is of devtype "cxl_decoder_endpoint" it
 		translates from a host physical address range, to a device
 		local address range. Device-local address ranges are further
-		split into a 'ram' (volatile memory) range and 'pmem'
-		(persistent memory) range. The 'mode' attribute emits one of
-		'ram', 'pmem', or 'none'. The 'none' indicates the decoder is
-		not actively decoding, or no DPA allocation policy has been
-		set.
+		split into a 'ram' (volatile memory) range, 'pmem' (persistent
+		memory) range, and dynamic capacity ranges. The 'mode'
+		attribute emits one of 'ram', 'pmem', 'none', or a dynamic
+		capacity selector: 'dynamic_ram', 'dynamic_ram_shared',
+		'dynamic_ram_ro' or 'dynamic_ram_shared_ro', composed from the
+		shareable and read-only properties of the partition the decoder
+		points at. The 'none' indicates the decoder is not actively
+		decoding, or no DPA allocation policy has been set.
 
 		'mode' can be written, when the decoder is in the 'disabled'
-		state, with either 'ram' or 'pmem' to set the boundaries for the
-		next allocation.
+		state, with 'ram', 'pmem', or one of the dynamic capacity
+		selectors to set the boundaries for the next allocation. A
+		selector picks the first dynamic capacity partition whose
+		shareable and read-only properties match it exactly, and is
+		accepted only while the driver has dynamic capacity enabled
+		for the device.
 
 
 What:		/sys/bus/cxl/devices/decoderX.Y/dpa_resource
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 8a81be004d00..24fbb5718abc 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -6,6 +6,7 @@
 
 #include <cxl/mailbox.h>
 #include <linux/rwsem.h>
+#include <cxlmem.h>
 
 extern const struct device_type cxl_nvdimm_bridge_type;
 extern const struct device_type cxl_nvdimm_type;
@@ -18,6 +19,25 @@ enum cxl_detach_mode {
 	DETACH_INVALIDATE,
 };
 
+static inline struct cxl_memdev_state *
+cxled_to_mds(struct cxl_endpoint_decoder *cxled)
+{
+	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
+
+	return to_cxl_memdev_state(cxlmd->cxlds);
+}
+
+/* Type-2 devices have no memdev state; nothing else leaves @cxlds NULL here */
+static inline bool cxled_dcd_enabled(struct cxl_endpoint_decoder *cxled)
+{
+	struct cxl_memdev_state *mds = cxled_to_mds(cxled);
+
+	return mds && cxl_dcd_supported(mds);
+}
+
+const char *cxl_dc_mode_name(bool shareable, bool read_only);
+int cxl_dc_mode_parse(const char *buf, bool *shareable, bool *read_only);
+
 #ifdef CONFIG_CXL_REGION
 
 struct cxl_region_context {
@@ -104,7 +124,8 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
 
 struct dentry *cxl_debugfs_create_dir(const char *dir);
 int cxl_dpa_set_part(struct cxl_endpoint_decoder *cxled,
-		     enum cxl_partition_mode mode);
+		     enum cxl_partition_mode mode, bool shareable,
+		     bool read_only);
 struct cxl_memdev_state;
 int cxl_mem_get_partition_info(struct cxl_memdev_state *mds);
 int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, u64 size);
diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
index fdcef20e01ba..36fba4cff493 100644
--- a/drivers/cxl/core/hdm.c
+++ b/drivers/cxl/core/hdm.c
@@ -619,8 +619,14 @@ int cxl_dpa_free(struct cxl_endpoint_decoder *cxled)
 	return 0;
 }
 
+/*
+ * Select the first partition of @mode. For dynamic capacity the shareable
+ * and read-only flags must match exactly as well, so each selector names
+ * one kind of partition.
+ */
 int cxl_dpa_set_part(struct cxl_endpoint_decoder *cxled,
-		     enum cxl_partition_mode mode)
+		     enum cxl_partition_mode mode, bool shareable,
+		     bool read_only)
 {
 	struct cxl_memdev *cxlmd = cxled_to_memdev(cxled);
 	struct cxl_dev_state *cxlds = cxlmd->cxlds;
@@ -631,9 +637,16 @@ int cxl_dpa_set_part(struct cxl_endpoint_decoder *cxled,
 	if (cxled->cxld.flags & CXL_DECODER_F_ENABLE)
 		return -EBUSY;
 
-	for (part = 0; part < cxlds->nr_partitions; part++)
-		if (cxlds->part[part].mode == mode)
-			break;
+	for (part = 0; part < cxlds->nr_partitions; part++) {
+		struct cxl_dpa_partition *p = &cxlds->part[part];
+
+		if (p->mode != mode)
+			continue;
+		if (mode == CXL_PARTMODE_DYNAMIC_RAM &&
+		    (p->shareable != shareable || p->read_only != read_only))
+			continue;
+		break;
+	}
 
 	if (part >= cxlds->nr_partitions) {
 		dev_dbg(dev, "unsupported mode: %d\n", mode);
diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index f3705ea44db7..9b71b58d4e36 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -203,6 +203,40 @@ static ssize_t target_list_show(struct device *dev,
 }
 static DEVICE_ATTR_RO(target_list);
 
+/* Dynamic capacity selectors: the flags name one kind of DC partition */
+static const struct {
+	const char *name;
+	bool shareable;
+	bool read_only;
+} cxl_dc_modes[] = {
+	{ "dynamic_ram",		false,	false },
+	{ "dynamic_ram_shared",		true,	false },
+	{ "dynamic_ram_ro",		false,	true },
+	{ "dynamic_ram_shared_ro",	true,	true },
+};
+
+const char *cxl_dc_mode_name(bool shareable, bool read_only)
+{
+	for (int i = 0; i < ARRAY_SIZE(cxl_dc_modes); i++)
+		if (cxl_dc_modes[i].shareable == shareable &&
+		    cxl_dc_modes[i].read_only == read_only)
+			return cxl_dc_modes[i].name;
+	return "";
+}
+
+/* Map a dynamic_ram selector to its partition flags; -EINVAL if unknown */
+int cxl_dc_mode_parse(const char *buf, bool *shareable, bool *read_only)
+{
+	for (int i = 0; i < ARRAY_SIZE(cxl_dc_modes); i++) {
+		if (!sysfs_streq(buf, cxl_dc_modes[i].name))
+			continue;
+		*shareable = cxl_dc_modes[i].shareable;
+		*read_only = cxl_dc_modes[i].read_only;
+		return 0;
+	}
+	return -EINVAL;
+}
+
 static ssize_t mode_show(struct device *dev, struct device_attribute *attr,
 			 char *buf)
 {
@@ -215,6 +249,9 @@ static ssize_t mode_show(struct device *dev, struct device_attribute *attr,
 
 	if (part < 0)
 		desc = "none";
+	else if (cxlds->part[part].mode == CXL_PARTMODE_DYNAMIC_RAM)
+		desc = cxl_dc_mode_name(cxlds->part[part].shareable,
+					cxlds->part[part].read_only);
 	else
 		desc = cxlds->part[part].res.name;
 
@@ -225,17 +262,25 @@ static ssize_t mode_store(struct device *dev, struct device_attribute *attr,
 			  const char *buf, size_t len)
 {
 	struct cxl_endpoint_decoder *cxled = to_cxl_endpoint_decoder(dev);
+	bool shareable = false, read_only = false;
 	enum cxl_partition_mode mode;
 	ssize_t rc;
 
-	if (sysfs_streq(buf, "pmem"))
+	if (sysfs_streq(buf, "pmem")) {
 		mode = CXL_PARTMODE_PMEM;
-	else if (sysfs_streq(buf, "ram"))
+	} else if (sysfs_streq(buf, "ram")) {
 		mode = CXL_PARTMODE_RAM;
-	else
-		return -EINVAL;
+	} else {
+		if (cxl_dc_mode_parse(buf, &shareable, &read_only))
+			return -EINVAL;
+		mode = CXL_PARTMODE_DYNAMIC_RAM;
+	}
+
+	/* The DC partition may outlive DCD support; refuse it when disabled */
+	if (mode == CXL_PARTMODE_DYNAMIC_RAM && !cxled_dcd_enabled(cxled))
+		return -ENXIO;
 
-	rc = cxl_dpa_set_part(cxled, mode);
+	rc = cxl_dpa_set_part(cxled, mode, shareable, read_only);
 	if (rc)
 		return rc;
 
-- 
2.43.0

[PATCH v1 03/14] cxl/region: Add dynamic DAX region support

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

CXL DAX regions must allow memory to be added or removed dynamically
to support DCD. In addition to the quantity of memory available, the
location of the memory within a DC partition is dynamic, based on the
extents offered by a device.  CXL DAX regions must accommodate the
dynamic movement of this memory in the management of DAX regions and
devices.

Introduce the concept of a dynamic DAX region. A dynamic region
carries the shareable and read-only properties of the partitions it
decodes, fixed when the region is created: every endpoint decoder
attached to it must point at a partition with exactly those
properties, and the region's 'mode' reports the matching selector.

A dynamic region has no capacity when it is created, so the DAX device
that cxl_dax_region_probe() creates for every region gets size 0
instead of the region size: a seed that claims no capacity the device
has not offered yet.  Its dax_region is created with IORESOURCE_DAX_DCD
rather than IORESOURCE_DAX_KMEM, so the DAX devices on it bind to
device_dax and not to kmem.

Indicate 0 byte available capacity until such time that capacity is
added.

While DCD is disabled, auto region creation and the region 'target'
write refuse endpoint decoders on a DC partition with -ENXIO, as the
decoder 'mode' write already does. The sysfs entries for manual region
creation follow in a separate patch.

Additionally, dynamic regions complicate the range mapping of dax
devices. There is no known use case for range mapping on dynamic
regions. Avoid the complication by preventing range mapping of dax
devices on dynamic regions.

Interleaving is deferred for now.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 Documentation/ABI/testing/sysfs-bus-cxl |  5 ++-
 drivers/cxl/core/region.c               | 60 ++++++++++++++++++++++---
 drivers/cxl/core/region_dax.c           |  6 +++
 drivers/cxl/cxl.h                       |  4 ++
 drivers/dax/bus.c                       | 10 +++++
 drivers/dax/bus.h                       |  1 +
 drivers/dax/cxl.c                       | 17 ++++++-
 7 files changed, 93 insertions(+), 10 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
index bcbc44fea6db..69e4f1fa21b7 100644
--- a/Documentation/ABI/testing/sysfs-bus-cxl
+++ b/Documentation/ABI/testing/sysfs-bus-cxl
@@ -543,7 +543,10 @@ Description:
 		(RO) The mode of a region is established at region creation time
 		and dictates the mode of the endpoint decoder that comprise the
 		region. For more details on the possible modes see
-		/sys/bus/cxl/devices/decoderX.Y/mode
+		/sys/bus/cxl/devices/decoderX.Y/mode. A dynamic capacity
+		region reports the selector it was created with: 'dynamic_ram',
+		'dynamic_ram_shared', 'dynamic_ram_ro' or
+		'dynamic_ram_shared_ro'.
 
 
 What:		/sys/bus/cxl/devices/regionZ/resource
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index f4a9839c73d2..fc9139ce358f 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -495,6 +495,11 @@ static int set_interleave_ways(struct cxl_region *cxlr, int val)
 	int save, rc;
 	u8 iw;
 
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM && val != 1) {
+		dev_dbg(&cxlr->dev, "Interleaving and DCD not supported\n");
+		return -EINVAL;
+	}
+
 	rc = ways_to_eiw(val, &iw);
 	if (rc)
 		return rc;
@@ -644,6 +649,8 @@ static ssize_t mode_show(struct device *dev, struct device_attribute *attr,
 		desc = "ram";
 	else if (cxlr->mode == CXL_PARTMODE_PMEM)
 		desc = "pmem";
+	else if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM)
+		desc = cxl_dc_mode_name(cxlr->shareable, cxlr->read_only);
 	else
 		desc = "";
 
@@ -2080,6 +2087,20 @@ static int cxl_region_attach(struct cxl_region *cxlr,
 		return -EINVAL;
 	}
 
+	/* A DC region's flavor is fixed; every member partition must match */
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM) {
+		struct cxl_dpa_partition *part = &cxlds->part[cxled->part];
+
+		if (part->shareable != cxlr->shareable ||
+		    part->read_only != cxlr->read_only) {
+			dev_dbg(&cxlr->dev, "%s: partition is %s, region is %s\n",
+				dev_name(&cxled->cxld.dev),
+				cxl_dc_mode_name(part->shareable, part->read_only),
+				cxl_dc_mode_name(cxlr->shareable, cxlr->read_only));
+			return -EINVAL;
+		}
+	}
+
 	/* all full of members, or interleave config not established? */
 	if (p->state > CXL_CONFIG_INTERLEAVE_ACTIVE) {
 		dev_dbg(&cxlr->dev, "region already active\n");
@@ -2411,6 +2432,7 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos,
 	if (sysfs_streq(buf, "\n"))
 		rc = detach_target(cxlr, pos);
 	else {
+		struct cxl_endpoint_decoder *cxled;
 		struct device *dev;
 
 		dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf);
@@ -2422,8 +2444,14 @@ static size_t store_targetN(struct cxl_region *cxlr, const char *buf, int pos,
 			goto out;
 		}
 
-		rc = attach_target(cxlr, to_cxl_endpoint_decoder(dev), pos,
-				   TASK_INTERRUPTIBLE);
+		cxled = to_cxl_endpoint_decoder(dev);
+		if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM &&
+		    !cxled_dcd_enabled(cxled)) {
+			dev_dbg(dev, "DCD not enabled\n");
+			rc = -ENXIO;
+			goto out;
+		}
+		rc = attach_target(cxlr, cxled, pos, TASK_INTERRUPTIBLE);
 out:
 		put_device(dev);
 	}
@@ -2737,6 +2765,8 @@ void kill_regions(struct cxl_root_decoder *cxlrd)
  * @cxlrd: root decoder
  * @id: memregion id to create, or memregion_free() on failure
  * @mode: mode for the endpoint decoders of this region
+ * @shareable: dynamic region: its partitions must be shareable
+ * @read_only: dynamic region: its partitions must be read-only
  * @type: select whether this is an expander or accelerator (type-2 or type-3)
  *
  * This is the second step of region initialization. Regions exist within an
@@ -2748,7 +2778,8 @@ void kill_regions(struct cxl_root_decoder *cxlrd)
 static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd,
 					      int id,
 					      enum cxl_partition_mode mode,
-					      enum cxl_decoder_type type)
+					      enum cxl_decoder_type type,
+					      bool shareable, bool read_only)
 {
 	struct cxl_port *port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent);
 	struct cxl_region *cxlr;
@@ -2760,6 +2791,8 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd,
 		return cxlr;
 	cxlr->mode = mode;
 	cxlr->type = type;
+	cxlr->shareable = shareable;
+	cxlr->read_only = read_only;
 
 	dev = &cxlr->dev;
 	rc = dev_set_name(dev, "region%d", id);
@@ -2803,7 +2836,8 @@ static ssize_t create_ram_region_show(struct device *dev,
 
 static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
 					  enum cxl_partition_mode mode, int id,
-					  enum cxl_decoder_type target_type)
+					  enum cxl_decoder_type target_type,
+					  bool shareable, bool read_only)
 {
 	int rc;
 
@@ -2813,6 +2847,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
 	switch (mode) {
 	case CXL_PARTMODE_RAM:
 	case CXL_PARTMODE_PMEM:
+	case CXL_PARTMODE_DYNAMIC_RAM:
 		break;
 	default:
 		dev_err(&cxlrd->cxlsd.cxld.dev, "unsupported mode %d\n", mode);
@@ -2828,7 +2863,8 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd,
 		return ERR_PTR(-EBUSY);
 	}
 
-	return devm_cxl_add_region(cxlrd, id, mode, target_type);
+	return devm_cxl_add_region(cxlrd, id, mode, target_type, shareable,
+				   read_only);
 }
 
 static ssize_t create_region_store(struct device *dev, const char *buf,
@@ -2846,7 +2882,8 @@ static ssize_t create_region_store(struct device *dev, const char *buf,
 	if ((rc = ACQUIRE_ERR(mutex_intr, &regions_lock)))
 		return rc;
 
-	cxlr = __create_region(cxlrd, mode, id, CXL_DECODER_HOSTONLYMEM);
+	cxlr = __create_region(cxlrd, mode, id, CXL_DECODER_HOSTONLYMEM,
+			       false, false);
 	if (IS_ERR(cxlr))
 		return PTR_ERR(cxlr);
 
@@ -3808,10 +3845,18 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd,
 	if (part < 0)
 		return ERR_PTR(-EBUSY);
 
+	if (cxlds->part[part].mode == CXL_PARTMODE_DYNAMIC_RAM &&
+	    !cxled_dcd_enabled(cxled)) {
+		dev_dbg(&cxled->cxld.dev, "DCD not enabled\n");
+		return ERR_PTR(-ENXIO);
+	}
+
 	do {
 		cxlr = __create_region(cxlrd, cxlds->part[part].mode,
 				       atomic_read(&cxlrd->region_id),
-				       cxled->cxld.target_type);
+				       cxled->cxld.target_type,
+				       cxlds->part[part].shareable,
+				       cxlds->part[part].read_only);
 	} while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY);
 
 	if (IS_ERR(cxlr)) {
@@ -4262,6 +4307,7 @@ static int cxl_region_probe(struct device *dev)
 
 		return devm_cxl_add_pmem_region(cxlr);
 	case CXL_PARTMODE_RAM:
+	case CXL_PARTMODE_DYNAMIC_RAM:
 		rc = devm_cxl_region_edac_register(cxlr);
 		if (rc)
 			dev_dbg(&cxlr->dev, "CXL EDAC registration for region_id=%d failed\n",
diff --git a/drivers/cxl/core/region_dax.c b/drivers/cxl/core/region_dax.c
index de04f78f6ad8..47c9c52a3902 100644
--- a/drivers/cxl/core/region_dax.c
+++ b/drivers/cxl/core/region_dax.c
@@ -84,6 +84,12 @@ int devm_cxl_add_dax_region(struct cxl_region *cxlr)
 	struct device *dev;
 	int rc;
 
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM &&
+	    cxlr->params.interleave_ways != 1) {
+		dev_dbg(&cxlr->dev, "Interleaving DC not supported\n");
+		return -EINVAL;
+	}
+
 	struct cxl_dax_region *cxlr_dax __free(put_cxl_dax_region) =
 		cxl_dax_region_alloc(cxlr);
 	if (IS_ERR(cxlr_dax))
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index baa6df2d362a..d098f7f50bef 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -490,6 +490,8 @@ struct cxl_region_params {
  * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown
  * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge
  * @flags: Region state flags
+ * @shareable: (for DC regions) backing partitions must be shareable
+ * @read_only: (for DC regions) backing partitions must be read-only
  * @params: active + config params for the region
  * @coord: QoS access coordinates for the region
  * @node_notifier: notifier for setting the access coordinates to node
@@ -506,6 +508,8 @@ struct cxl_region {
 	struct cxl_nvdimm_bridge *cxl_nvb;
 	struct cxl_pmem_region *cxlr_pmem;
 	unsigned long flags;
+	bool shareable;
+	bool read_only;
 	struct cxl_region_params params;
 	struct access_coordinate coord[ACCESS_COORDINATE_MAX];
 	struct notifier_block node_notifier;
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index e40c25401cf0..d8cc1c012f46 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -177,6 +177,11 @@ static bool is_static(struct dax_region *dax_region)
 	return (dax_region->res.flags & IORESOURCE_DAX_STATIC) != 0;
 }
 
+static bool is_dcd(struct dax_region *dax_region)
+{
+	return (dax_region->res.flags & IORESOURCE_DAX_DCD) != 0;
+}
+
 bool static_dev_dax(struct dev_dax *dev_dax)
 {
 	return is_static(dev_dax->region);
@@ -300,6 +305,9 @@ static unsigned long long dax_region_avail_size(struct dax_region *dax_region)
 
 	lockdep_assert_held(&dax_region_rwsem);
 
+	if (is_dcd(dax_region))
+		return 0;
+
 	for_each_dax_region_resource(dax_region, res)
 		size -= resource_size(res);
 	return size;
@@ -1389,6 +1397,8 @@ static umode_t dev_dax_visible(struct kobject *kobj, struct attribute *a, int n)
 		return 0;
 	if (a == &dev_attr_mapping.attr && is_static(dax_region))
 		return 0;
+	if (a == &dev_attr_mapping.attr && is_dcd(dax_region))
+		return 0;
 	if ((a == &dev_attr_align.attr ||
 	     a == &dev_attr_size.attr) && is_static(dax_region))
 		return 0444;
diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h
index bfc31923f3ff..c877daa8df51 100644
--- a/drivers/dax/bus.h
+++ b/drivers/dax/bus.h
@@ -15,6 +15,7 @@ struct dax_region;
 /* dax bus specific ioresource flags */
 #define IORESOURCE_DAX_STATIC BIT(0)
 #define IORESOURCE_DAX_KMEM BIT(1)
+#define IORESOURCE_DAX_DCD BIT(2)
 
 #define DAX_KMEM_UNPLUGGED	(-1) /* Do not create memory blocks */
 
diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c
index 3ab39b77843d..43eb18c8a035 100644
--- a/drivers/dax/cxl.c
+++ b/drivers/dax/cxl.c
@@ -13,19 +13,32 @@ static int cxl_dax_region_probe(struct device *dev)
 	struct cxl_region *cxlr = cxlr_dax->cxlr;
 	struct dax_region *dax_region;
 	struct dev_dax_data data;
+	resource_size_t dev_size;
+	unsigned long flags;
 
 	if (nid == NUMA_NO_NODE)
 		nid = memory_add_physaddr_to_nid(cxlr_dax->hpa_range.start);
 
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM)
+		flags = IORESOURCE_DAX_DCD;
+	else
+		flags = IORESOURCE_DAX_KMEM;
+
 	dax_region = alloc_dax_region(dev, cxlr->id, &cxlr_dax->hpa_range, nid,
-				      PMD_SIZE, IORESOURCE_DAX_KMEM);
+				      PMD_SIZE, flags);
 	if (!dax_region)
 		return -ENOMEM;
 
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM)
+		/* Add empty seed dax device */
+		dev_size = 0;
+	else
+		dev_size = range_len(&cxlr_dax->hpa_range);
+
 	data = (struct dev_dax_data) {
 		.dax_region = dax_region,
 		.id = -1,
-		.size = range_len(&cxlr_dax->hpa_range),
+		.size = dev_size,
 		.memmap_on_memory = true,
 	};
 
-- 
2.43.0

[PATCH v1 04/14] cxl/mem: Introduce DC event handlers

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Add the handling skeleton for DC Add and DC Release events. Per CXL
r4.0 Section 8.2.10.9.9.3, the host is expected to respond to one or
more Add Capacity Event Record(s) with an Add Dynamic Capacity Response
(Opcode 4802h) to acknowledge which extents the host accepts for use.
Similarly, per CXL r4.0 Section 8.2.10.9.9.4, the host is expected to
respond to Release Capacity Event Record(s) with a Release Dynamic
Capacity command (Opcode 4803h), or to issue one that is not
associated with any event to release capacity on its own.

The records of one operation can be chained by the More flag. Chained
records are contiguous in the event log and share one event type (CXL
r4.0 Table 8-229), so a single staging context, mds->dc_chain, holds
the records of the open chain until the closing record (More=0)
arrives and the chain is processed and responded to as a whole. A
record that fails to stage, or one of the other type arriving inside an
open chain, drops the chain: an add is refused with an empty response,
a release is left for the device to retry.

This commit refuses all extents from Add Capacity chains and acks all
Release Capacity chains. Subsequent commits implement accepting and
releasing extents.

Forced Capacity Release and Region Configuration Updated events are
logged and ignored.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/mbox.c  | 246 ++++++++++++++++++++++++++++++++++++++-
 drivers/cxl/core/trace.h |   3 +-
 drivers/cxl/cxl.h        |   9 ++
 drivers/cxl/cxlmem.h     |  46 ++++++++
 drivers/cxl/pci.c        |  20 +++-
 include/cxl/event.h      |  37 ++++++
 6 files changed, 353 insertions(+), 8 deletions(-)

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 13a68f5cef69..3acf5ace48db 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -5,6 +5,7 @@
 #include <linux/ktime.h>
 #include <linux/mutex.h>
 #include <linux/unaligned.h>
+#include <linux/list.h>
 #include <cxlpci.h>
 #include <cxlmem.h>
 #include <cxl.h>
@@ -1100,6 +1101,237 @@ static int cxl_clear_event_record(struct cxl_memdev_state *mds,
 	return rc;
 }
 
+static int send_one_response(struct cxl_mailbox *cxl_mbox,
+			     struct cxl_mbox_dc_response *response,
+			     int opcode, u32 extent_list_size, u8 flags)
+{
+	struct cxl_mbox_cmd mbox_cmd = (struct cxl_mbox_cmd) {
+		.opcode = opcode,
+		.size_in = struct_size(response, extent_list, extent_list_size),
+		.payload_in = response,
+	};
+
+	response->extent_list_size = cpu_to_le32(extent_list_size);
+	response->flags = flags;
+	return cxl_internal_send_cmd(cxl_mbox, &mbox_cmd);
+}
+
+static int cxl_send_dc_response(struct cxl_memdev_state *mds, int opcode,
+				struct list_head *extent_list, int cnt)
+{
+	struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
+	struct cxl_mbox_dc_response *p;
+	struct cxl_extent_list_node *pos, *tmp;
+	struct cxl_extent *extent;
+	u32 pl_index;
+
+	size_t pl_size = struct_size(p, extent_list, cnt);
+	u32 max_extents = cnt;
+
+	/* May have to use more bit on response. */
+	if (pl_size > cxl_mbox->payload_size) {
+		max_extents = (cxl_mbox->payload_size - sizeof(*p)) /
+			      sizeof(struct updated_extent_list);
+		pl_size = struct_size(p, extent_list, max_extents);
+	}
+
+	struct cxl_mbox_dc_response *response __free(kfree) =
+						kzalloc(pl_size, GFP_KERNEL);
+	if (!response) {
+		struct cxl_mbox_dc_response empty = { };
+
+		/* An empty Release DC is invalid (CXL r4.0 Table 8-352) */
+		if (opcode == CXL_MBOX_OP_RELEASE_DC)
+			return -ENOMEM;
+		/* Refuse all extents rather than stall the device. */
+		return send_one_response(cxl_mbox, &empty, opcode, 0, 0);
+	}
+
+	/* Set __counted_by to capacity before populating; reset per response. */
+	response->extent_list_size = cpu_to_le32(max_extents);
+
+	if (cnt == 0)
+		return send_one_response(cxl_mbox, response, opcode, 0, 0);
+
+	pl_index = 0;
+	list_for_each_entry_safe(pos, tmp, extent_list, list) {
+		extent = pos->extent;
+		response->extent_list[pl_index].dpa_start = extent->start_dpa;
+		response->extent_list[pl_index].length = extent->length;
+		pl_index++;
+
+		if (pl_index == max_extents) {
+			u8 flags = 0;
+			int rc;
+
+			if (pl_index < cnt)
+				flags |= CXL_DCD_EVENT_MORE;
+			rc = send_one_response(cxl_mbox, response, opcode,
+					       pl_index, flags);
+			if (rc)
+				return rc;
+			cnt -= pl_index;
+			if (cnt < max_extents)
+				max_extents = cnt;
+			pl_index = 0;
+		}
+	}
+
+	if (!pl_index) /* nothing more to do */
+		return 0;
+	return send_one_response(cxl_mbox, response, opcode, pl_index, 0);
+}
+
+static void delete_extent_node(struct cxl_extent_list_node *node)
+{
+	list_del(&node->list);
+	kfree(node->extent);
+	kfree(node);
+}
+
+static void clear_pending_extents(void *_mds)
+{
+	struct cxl_memdev_state *mds = _mds;
+	struct cxl_extent_list_node *pos, *tmp;
+
+	list_for_each_entry_safe(pos, tmp, &mds->dc_chain.pending_extents, list)
+		delete_extent_node(pos);
+}
+
+static const char *cxl_dcd_evt_type_str(u8 type)
+{
+	switch (type) {
+	case DCD_ADD_CAPACITY:
+		return "add";
+	case DCD_RELEASE_CAPACITY:
+		return "release";
+	case DCD_FORCED_CAPACITY_RELEASE:
+		return "force release";
+	case DCD_REGION_CONFIGURATION_UPDATED:
+		return "region configuration updated";
+	default:
+		break;
+	}
+
+	return "<unknown>";
+}
+
+/*
+ * Drop the open chain.  An add chain is refused with an empty response so
+ * the device can reclaim the offered capacity; a release chain is simply
+ * dropped, the capacity stays with the host and the device may retry.
+ */
+static void cxl_dc_chain_abort(struct cxl_memdev_state *mds)
+{
+	struct cxl_dc_event_chain *chain = &mds->dc_chain;
+	struct device *dev = mds->cxlds.dev;
+
+	if (chain->event_type == DCD_ADD_CAPACITY &&
+	    cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
+				 &chain->pending_extents, 0))
+		dev_dbg(dev, "Failed to refuse add chain\n");
+
+	clear_pending_extents(mds);
+}
+
+static int stage_pending_extent(struct list_head *pending,
+				struct cxl_extent *to_add)
+{
+	struct cxl_extent_list_node *node = kzalloc(sizeof(*node), GFP_KERNEL);
+	struct cxl_extent *extent;
+
+	if (!node)
+		return -ENOMEM;
+	extent = kmemdup(to_add, sizeof(*extent), GFP_KERNEL);
+	if (!extent) {
+		kfree(node);
+		return -ENOMEM;
+	}
+
+	node->extent = extent;
+	list_add_tail(&node->list, pending);
+	return 0;
+}
+
+static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
+				  struct cxl_event_dcd *event)
+{
+	struct cxl_dc_event_chain *chain = &mds->dc_chain;
+	struct device *dev = mds->cxlds.dev;
+	int rc;
+
+	/* Chained records share one event type (CXL r4.0 Table 8-229) */
+	if (list_empty(&chain->pending_extents)) {
+		chain->event_type = event->event_type;
+	} else if (chain->event_type != event->event_type) {
+		dev_warn_ratelimited(dev, "DC %s record inside an open %s chain; dropping the chain\n",
+				     cxl_dcd_evt_type_str(event->event_type),
+				     cxl_dcd_evt_type_str(chain->event_type));
+		cxl_dc_chain_abort(mds);
+		chain->event_type = event->event_type;
+	}
+
+	rc = stage_pending_extent(&chain->pending_extents, &event->extent);
+	if (rc) {
+		cxl_dc_chain_abort(mds);
+		return rc;
+	}
+
+	if (event->flags & CXL_DCD_EVENT_MORE) {
+		dev_dbg(dev, "more bit set; delay processing of extent\n");
+		return 0;
+	}
+
+	/* Nothing is tracked yet: refuse every add, acknowledge every release */
+	if (event->event_type == DCD_ADD_CAPACITY)
+		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
+					  &chain->pending_extents, 0);
+	else
+		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC,
+					  &chain->pending_extents,
+					  list_count_nodes(&chain->pending_extents));
+	clear_pending_extents(mds);
+	return rc;
+}
+
+static void cxl_handle_dcd_event_records(struct cxl_memdev_state *mds,
+					 struct cxl_event_record_raw *raw_rec)
+{
+	struct cxl_event_dcd *event = &raw_rec->event.dcd;
+	struct cxl_extent *extent = &event->extent;
+	struct device *dev = mds->cxlds.dev;
+	uuid_t *id = &raw_rec->id;
+	int rc;
+
+	if (!uuid_equal(id, &CXL_EVENT_DC_EVENT_UUID))
+		return;
+
+	dev_dbg(dev, "DCD event %s : DPA:%#llx LEN:%#llx\n",
+		cxl_dcd_evt_type_str(event->event_type),
+		le64_to_cpu(extent->start_dpa), le64_to_cpu(extent->length));
+
+	switch (event->event_type) {
+	case DCD_ADD_CAPACITY:
+	case DCD_RELEASE_CAPACITY:
+		rc = handle_dcd_chain_event(mds, event);
+		break;
+	case DCD_FORCED_CAPACITY_RELEASE:
+		dev_err_ratelimited(dev, "Forced release event ignored.\n");
+		rc = 0;
+		break;
+	case DCD_REGION_CONFIGURATION_UPDATED:
+		dev_dbg(dev, "Region configuration updated event ignored.\n");
+		rc = 0;
+		break;
+	default:
+		rc = -EINVAL;
+		break;
+	}
+
+	if (rc)
+		dev_err_ratelimited(dev, "dcd event failed: %d\n", rc);
+}
+
 static void cxl_mem_get_records_log(struct cxl_memdev_state *mds,
 				    enum cxl_event_log_type type)
 {
@@ -1136,9 +1368,13 @@ static void cxl_mem_get_records_log(struct cxl_memdev_state *mds,
 		if (!nr_rec)
 			break;
 
-		for (i = 0; i < nr_rec; i++)
+		for (i = 0; i < nr_rec; i++) {
 			__cxl_event_trace_record(cxlmd, type,
 						 &payload->records[i]);
+			if (type == CXL_EVENT_TYPE_DCD)
+				cxl_handle_dcd_event_records(mds,
+							     &payload->records[i]);
+		}
 
 		if (payload->flags & CXL_GET_EVENT_FLAG_OVERFLOW)
 			trace_cxl_overflow(cxlmd, type, payload);
@@ -1170,6 +1406,8 @@ void cxl_mem_get_event_records(struct cxl_memdev_state *mds, u32 status)
 {
 	dev_dbg(mds->cxlds.dev, "Reading event logs: %x\n", status);
 
+	if (cxl_dcd_supported(mds) && (status & CXLDEV_EVENT_STATUS_DCD))
+		cxl_mem_get_records_log(mds, CXL_EVENT_TYPE_DCD);
 	if (status & CXLDEV_EVENT_STATUS_FATAL)
 		cxl_mem_get_records_log(mds, CXL_EVENT_TYPE_FATAL);
 	if (status & CXLDEV_EVENT_STATUS_FAIL)
@@ -1771,6 +2009,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev, u64 serial,
 						 u16 dvsec)
 {
 	struct cxl_memdev_state *mds;
+	int rc;
 
 	mds = devm_cxl_dev_state_create(dev, CXL_DEVTYPE_CLASSMEM, serial,
 					dvsec, struct cxl_memdev_state, cxlds,
@@ -1781,6 +2020,11 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev, u64 serial,
 	}
 
 	mutex_init(&mds->event.log_lock);
+	INIT_LIST_HEAD(&mds->dc_chain.pending_extents);
+
+	rc = devm_add_action_or_reset(dev, clear_pending_extents, mds);
+	if (rc)
+		return ERR_PTR(rc);
 
 	return mds;
 }
diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h
index c379d60047fc..64c6ce0878e2 100644
--- a/drivers/cxl/core/trace.h
+++ b/drivers/cxl/core/trace.h
@@ -177,7 +177,8 @@ TRACE_EVENT(cxl_aer_correctable_error,
 		{ CXL_EVENT_TYPE_INFO, "Informational" },	\
 		{ CXL_EVENT_TYPE_WARN, "Warning" },		\
 		{ CXL_EVENT_TYPE_FAIL, "Failure" },		\
-		{ CXL_EVENT_TYPE_FATAL, "Fatal" })
+		{ CXL_EVENT_TYPE_FATAL, "Fatal" },		\
+		{ CXL_EVENT_TYPE_DCD, "Dynamic Capacity" })
 
 TRACE_EVENT(cxl_overflow,
 
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index d098f7f50bef..64138e1768a1 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -12,6 +12,7 @@
 #include <linux/node.h>
 #include <linux/io.h>
 #include <linux/range.h>
+#include <linux/xarray.h>
 #include <cxl/cxl.h>
 
 extern const struct nvdimm_security_ops *cxl_security_ops;
@@ -557,6 +558,14 @@ struct cxl_pmem_region {
 	struct cxl_pmem_region_mapping mapping[];
 };
 
+/* See CXL r4.0 8.2.10.2.1.6 */
+enum dc_event {
+	DCD_ADD_CAPACITY,
+	DCD_RELEASE_CAPACITY,
+	DCD_FORCED_CAPACITY_RELEASE,
+	DCD_REGION_CONFIGURATION_UPDATED,
+};
+
 struct cxl_dax_region {
 	struct device dev;
 	struct cxl_region *cxlr;
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 35268d1c387c..c88de7cb65d1 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -7,6 +7,7 @@
 #include <linux/cdev.h>
 #include <linux/uuid.h>
 #include <linux/node.h>
+#include <linux/list.h>
 #include <cxl/event.h>
 #include <cxl/mailbox.h>
 #include "cxl.h"
@@ -426,6 +427,24 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
 	return dev_get_drvdata(cxl_mbox->host);
 }
 
+/**
+ * struct cxl_dc_event_chain - staged Dynamic Capacity event chain
+ * @pending_extents: extents received so far; flushed when the chain
+ *		     closes (More=0)
+ * @event_type: DCD event type of the open chain (add or release)
+ *
+ * The records of one Dynamic Capacity operation can be chained by the
+ * CXL_DCD_EVENT_MORE flag.  Chained records are contiguous in the event
+ * log and share one event type (CXL r4.0 Table 8-229), so at most one
+ * chain is open at a time.  Both Add Capacity and Release Capacity chains
+ * are staged here, @event_type says which.  Records are staged until
+ * More=0, then the chain is processed and responded to as a whole.
+ */
+struct cxl_dc_event_chain {
+	struct list_head pending_extents;
+	u8 event_type;
+};
+
 /**
  * struct cxl_memdev_state - Generic Type-3 Memory Device Class driver data
  *
@@ -444,6 +463,8 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
  * @active_volatile_bytes: sum of hard + soft volatile
  * @active_persistent_bytes: sum of hard + soft persistent
  * @dcd_supported: all DCD commands are supported
+ * @dc_chain: the open Dynamic Capacity event chain, if any
+ *	      (see &struct cxl_dc_event_chain)
  * @event: event log driver state
  * @poison: poison driver state info
  * @security: security driver state info
@@ -463,6 +484,7 @@ struct cxl_memdev_state {
 	u64 active_volatile_bytes;
 	u64 active_persistent_bytes;
 	bool dcd_supported;
+	struct cxl_dc_event_chain dc_chain;
 
 	struct cxl_event_state event;
 	struct cxl_poison_state poison;
@@ -538,6 +560,21 @@ enum cxl_opcode {
 	UUID_INIT(0x5e1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19,     \
 		  0x40, 0x3d, 0x86)
 
+/*
+ * Add Dynamic Capacity Response
+ * CXL r4.0 Section 8.2.10.9.9.3; Table 8-350 & Table 8-351
+ */
+struct cxl_mbox_dc_response {
+	__le32 extent_list_size;
+	u8 flags;
+	u8 reserved[3];
+	struct updated_extent_list {
+		__le64 dpa_start;
+		__le64 length;
+		u8 reserved[8];
+	} __packed extent_list[] __counted_by_le(extent_list_size);
+} __packed;
+
 struct cxl_mbox_get_supported_logs {
 	__le16 entries;
 	u8 rsvd[6];
@@ -608,6 +645,14 @@ struct cxl_mbox_identify {
 	UUID_INIT(0xe71f3a40, 0x2d29, 0x4092, 0x8a, 0x39, 0x4d, 0x1c, 0x96, \
 		  0x6c, 0x7c, 0x65)
 
+/*
+ * Dynamic Capacity Event Record
+ * CXL r4.0 section 8.2.10.2.1; Table 8-221
+ */
+#define CXL_EVENT_DC_EVENT_UUID                                             \
+	UUID_INIT(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f, 0x95, 0x26, 0x8e, \
+		  0x10, 0x1a, 0x2a)
+
 /*
  * Get Event Records output payload
  * CXL rev 3.0 section 8.2.9.2.2; Table 8-50
@@ -633,6 +678,7 @@ enum cxl_event_log_type {
 	CXL_EVENT_TYPE_WARN,
 	CXL_EVENT_TYPE_FAIL,
 	CXL_EVENT_TYPE_FATAL,
+	CXL_EVENT_TYPE_DCD,
 	CXL_EVENT_TYPE_MAX
 };
 
diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 3ed7c3e70042..29541b8f79f2 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -510,12 +510,20 @@ static bool cxl_alloc_irq_vectors(struct pci_dev *pdev)
 	return true;
 }
 
-/* Event logs the driver drains: standard logs when native_cxl */
-static u32 cxl_event_drain_mask(struct pci_host_bridge *host_bridge)
+/*
+ * Event logs the driver drains: standard logs when native_cxl, DCD when
+ * supported.
+ */
+static u32 cxl_event_drain_mask(struct pci_host_bridge *host_bridge,
+				struct cxl_memdev_state *mds)
 {
+	u32 mask = 0;
+
 	if (host_bridge->native_cxl_error)
-		return CXLDEV_EVENT_STATUS_STANDARD_LOGS;
-	return 0;
+		mask |= CXLDEV_EVENT_STATUS_STANDARD_LOGS;
+	if (cxl_dcd_supported(mds))
+		mask |= CXLDEV_EVENT_STATUS_DCD;
+	return mask;
 }
 
 static irqreturn_t cxl_event_thread(int irq, void *id)
@@ -525,7 +533,7 @@ static irqreturn_t cxl_event_thread(int irq, void *id)
 	struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlds);
 	struct pci_host_bridge *host_bridge =
 		pci_find_host_bridge(to_pci_dev(cxlds->dev)->bus);
-	u32 mask = cxl_event_drain_mask(host_bridge);
+	u32 mask = cxl_event_drain_mask(host_bridge, mds);
 	bool handled = false;
 	u32 status;
 
@@ -786,7 +794,7 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge,
 	if (rc)
 		return rc;
 
-	mask = cxl_event_drain_mask(host_bridge);
+	mask = cxl_event_drain_mask(host_bridge, mds);
 	if (mask)
 		cxl_mem_get_event_records(mds, mask);
 
diff --git a/include/cxl/event.h b/include/cxl/event.h
index a9f5c2381cc9..f154ca952340 100644
--- a/include/cxl/event.h
+++ b/include/cxl/event.h
@@ -6,6 +6,7 @@
 #include <linux/types.h>
 #include <linux/uuid.h>
 #include <linux/workqueue_types.h>
+#include <linux/list.h>
 
 /*
  * Common Event Record Format
@@ -141,12 +142,48 @@ struct cxl_event_mem_sparing {
 	u8 reserved2[0x25];
 } __packed;
 
+/*
+ * CXL r4.0 section 8.2.10.2.1.6; Table 8-230
+ */
+struct cxl_extent {
+	__le64 start_dpa;
+	__le64 length;
+	u8 uuid[UUID_SIZE];
+	__le16 shared_extn_seq;
+	u8 reserved[0x6];
+} __packed;
+
+struct cxl_extent_list_node {
+	struct cxl_extent *extent;
+	struct list_head list;
+};
+
+/*
+ * Dynamic Capacity Event Record
+ * CXL r4.0 section 8.2.10.2.1.6; Table 8-229
+ */
+#define CXL_DCD_EVENT_MORE			BIT(0)
+struct cxl_event_dcd {
+	struct cxl_event_record_hdr hdr;
+	u8 event_type;
+	u8 validity_flags;
+	__le16 host_id;
+	u8 updated_region_index;
+	u8 flags;
+	u8 reserved1[0x2];
+	struct cxl_extent extent;
+	u8 reserved2[0x18];
+	__le32 num_avail_extents;
+	__le32 num_avail_tags;
+} __packed;
+
 union cxl_event {
 	struct cxl_event_generic generic;
 	struct cxl_event_gen_media gen_media;
 	struct cxl_event_dram dram;
 	struct cxl_event_mem_module mem_module;
 	struct cxl_event_mem_sparing mem_sparing;
+	struct cxl_event_dcd dcd;
 	/* dram & gen_media event header */
 	struct cxl_event_media_hdr media_hdr;
 } __packed;
-- 
2.43.0

[PATCH v1 05/14] cxl/mem: Add 20 second timeout for stalled DC event chains

[Anisa Su]

A Dynamic Capacity event can span multiple event records grouped
together by the CXL_DCD_EVENT_MORE flag. Extents are staged in the
pending list until the last event record ('More'=0) is received, at
which point the chain is processed. If the device opens such a chain
(More=1) but never sends the closing record, the staged list sits
indefinitely.

Add a delayed-work watchdog that drops the chain on timeout: an add
chain is refused with an empty Add Dynamic Capacity Response (4802h),
a release chain is left for the device to retry.

The spec sets no limit; 20s is a generous upper bound that may be
tightened later, and guards against a lost mailbox response or a
crashed fabric manager.

The deadline is fixed when the chain opens rather than extended per
record, so a device cannot keep a chain alive, and the staged list
growing, by streaming More=1 records.

Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/mbox.c | 84 +++++++++++++++++++++++++++++++++++++++++
 drivers/cxl/cxlmem.h    | 15 ++++++++
 2 files changed, 99 insertions(+)

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 3acf5ace48db..fea910e6c510 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -873,6 +873,8 @@ static const uuid_t log_uuid[] = {
 	[VENDOR_DEBUG_UUID] = DEFINE_CXL_VENDOR_DEBUG_UUID,
 };
 
+static void cxl_cancel_dcd_chain_work(void *_mds);
+
 /**
  * cxl_enumerate_cmds() - Enumerate commands for a device.
  * @mds: The driver data for the operation
@@ -891,6 +893,16 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds)
 	struct cxl_mem_command *cmd;
 	int i, rc;
 
+	/*
+	 * The DC chain watchdog sends a mailbox command when it fires.  Register
+	 * its cancellation here, once the mailbox is usable, so devm runs it
+	 * before the mailbox goes away: in cxl_pci the device registers were
+	 * mapped before this point and are unmapped after the cancel.
+	 */
+	rc = devm_add_action_or_reset(dev, cxl_cancel_dcd_chain_work, mds);
+	if (rc)
+		return rc;
+
 	gsl = cxl_get_gsl(mds);
 	if (IS_ERR(gsl))
 		return PTR_ERR(gsl);
@@ -1198,6 +1210,14 @@ static void clear_pending_extents(void *_mds)
 		delete_extent_node(pos);
 }
 
+/*
+ * Bound on how long the host will wait for a device to finish a
+ * multi-record DC event chain (More=1 ... More=0) before refusing the
+ * chain.  The spec sets no limit; 20s is a generous upper bound, the
+ * maximum GPF timeout.
+ */
+#define CXL_DC_CHAIN_TIMEOUT	(20 * HZ)
+
 static const char *cxl_dcd_evt_type_str(u8 type)
 {
 	switch (type) {
@@ -1226,6 +1246,11 @@ static void cxl_dc_chain_abort(struct cxl_memdev_state *mds)
 	struct cxl_dc_event_chain *chain = &mds->dc_chain;
 	struct device *dev = mds->cxlds.dev;
 
+	lockdep_assert_held(&chain->lock);
+
+	chain->armed = false;
+	cancel_delayed_work(&chain->timeout_work);
+
 	if (chain->event_type == DCD_ADD_CAPACITY &&
 	    cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
 				 &chain->pending_extents, 0))
@@ -1234,6 +1259,43 @@ static void cxl_dc_chain_abort(struct cxl_memdev_state *mds)
 	clear_pending_extents(mds);
 }
 
+static void cxl_dc_chain_timeout(struct work_struct *work)
+{
+	struct cxl_dc_event_chain *chain = container_of(to_delayed_work(work),
+							struct cxl_dc_event_chain,
+							timeout_work);
+	struct cxl_memdev_state *mds = container_of(chain,
+						    struct cxl_memdev_state,
+						    dc_chain);
+	struct device *dev = mds->cxlds.dev;
+
+	guard(mutex)(&chain->lock);
+
+	/*
+	 * handle_dcd_chain_event() cancels this work non-synchronously (a sync
+	 * cancel would deadlock on @chain->lock, which the chain-close path
+	 * holds), so a callback that already started running can reach here
+	 * after its chain has moved on.  Abort only if a chain is still armed
+	 * AND the timer has not been re-armed since this expiry fired: a new
+	 * chain's queue_delayed_work() makes delayed_work_pending() true,
+	 * meaning this expiry belongs to a superseded deadline and must not
+	 * abort the current chain.
+	 */
+	if (!chain->armed || delayed_work_pending(&chain->timeout_work))
+		return;
+
+	dev_warn(dev, "DC %s chain timed out\n",
+		 cxl_dcd_evt_type_str(chain->event_type));
+	cxl_dc_chain_abort(mds);
+}
+
+static void cxl_cancel_dcd_chain_work(void *_mds)
+{
+	struct cxl_memdev_state *mds = _mds;
+
+	cancel_delayed_work_sync(&mds->dc_chain.timeout_work);
+}
+
 static int stage_pending_extent(struct list_head *pending,
 				struct cxl_extent *to_add)
 {
@@ -1260,6 +1322,8 @@ static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 	struct device *dev = mds->cxlds.dev;
 	int rc;
 
+	guard(mutex)(&chain->lock);
+
 	/* Chained records share one event type (CXL r4.0 Table 8-229) */
 	if (list_empty(&chain->pending_extents)) {
 		chain->event_type = event->event_type;
@@ -1279,9 +1343,22 @@ static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 
 	if (event->flags & CXL_DCD_EVENT_MORE) {
 		dev_dbg(dev, "more bit set; delay processing of extent\n");
+		/* The deadline is fixed when the chain opens, not per record */
+		if (!chain->armed) {
+			queue_delayed_work(system_wq, &chain->timeout_work,
+					   CXL_DC_CHAIN_TIMEOUT);
+			chain->armed = true;
+		}
 		return 0;
 	}
 
+	/*
+	 * Chain is closing.  Disarm before flushing so a pending watchdog
+	 * (queued but blocked on @chain->lock) sees !armed and bails out.
+	 */
+	chain->armed = false;
+	cancel_delayed_work(&chain->timeout_work);
+
 	/* Nothing is tracked yet: refuse every add, acknowledge every release */
 	if (event->event_type == DCD_ADD_CAPACITY)
 		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
@@ -2021,7 +2098,14 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev, u64 serial,
 
 	mutex_init(&mds->event.log_lock);
 	INIT_LIST_HEAD(&mds->dc_chain.pending_extents);
+	mutex_init(&mds->dc_chain.lock);
+	INIT_DELAYED_WORK(&mds->dc_chain.timeout_work, cxl_dc_chain_timeout);
 
+	/*
+	 * The watchdog cancel is registered later, by cxl_enumerate_cmds(), so
+	 * devm's reverse-order unwind cancels (and waits for) the watchdog
+	 * before this list cleanup runs.
+	 */
 	rc = devm_add_action_or_reset(dev, clear_pending_extents, mds);
 	if (rc)
 		return ERR_PTR(rc);
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index c88de7cb65d1..52c936ec15a9 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -8,6 +8,8 @@
 #include <linux/uuid.h>
 #include <linux/node.h>
 #include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
 #include <cxl/event.h>
 #include <cxl/mailbox.h>
 #include "cxl.h"
@@ -431,6 +433,11 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
  * struct cxl_dc_event_chain - staged Dynamic Capacity event chain
  * @pending_extents: extents received so far; flushed when the chain
  *		     closes (More=0)
+ * @timeout_work: watchdog that fires if a chain is opened with
+ *		  CXL_DCD_EVENT_MORE but the closing record never arrives
+ * @lock: serialises updates to the chain state against the watchdog
+ * @armed: set when a More=1 chain opens; cleared when the chain closes
+ *	   (More=0), is aborted by cxl_dc_chain_abort(), or times out.
  * @event_type: DCD event type of the open chain (add or release)
  *
  * The records of one Dynamic Capacity operation can be chained by the
@@ -439,9 +446,17 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
  * chain is open at a time.  Both Add Capacity and Release Capacity chains
  * are staged here, @event_type says which.  Records are staged until
  * More=0, then the chain is processed and responded to as a whole.
+ *
+ * If a chain is opened (More=1) but the device never sends the closing
+ * record, the staged list would otherwise sit indefinitely.  @timeout_work
+ * is a defensive watchdog that drops such a chain: an add is refused with
+ * an empty response, a release is held for the device to retry.
  */
 struct cxl_dc_event_chain {
 	struct list_head pending_extents;
+	struct delayed_work timeout_work;
+	struct mutex lock;	/* chain state, against the watchdog */
+	bool armed;
 	u8 event_type;
 };
 
-- 
2.43.0

[PATCH v1 06/14] cxl/extent: Handle DC Add Capacity events

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Replace the refuse-all stub for Add Capacity chains in
handle_dcd_chain_event() with the real add pipeline.

DC Event Records can be grouped together with the 'More' flag. The
previous commit completed the set up for holding onto extents in
the pending list until receiving the last event record of the group,
marked by 'More'=0.

This commit fills in the logic for processing the pending list and
adds basic validation for extents before they are added to the
device model as a child of the cxlr_dax region. More complete
checks for tags/sequence numbers/alignment are added in subsequent
commits.

For each tag that appears in the pending list:
1. Extract all extents in the pending list with that tag to a
local list.

2. The spec requires that shareable extents are ordered by
shared extent sequence number, which "instructs each host
on the relative order these extents must be placed in adjacent
virtual address space" (r4.0 Section 9.13.3 Figure 9-23
Shared Extent List Example). Otherwise, retain arrival order.

Thus the tag group is stable-sorted by shared_extn_seq; for non-sharable
extents every key is 0 and the stable sort preserves arrival
order.

Individual extents are checked for the following:
1. The extent's DPA range fully resolves to an endpoint decoder.

2. Doesn't overlap or repeat a previously accepted extent; the device
rejects a DPA range reported twice in an Add response (r4.0 Section
8.2.10.9.9.3, Invalid Physical Address).

A tag group is one allocation, and for sharable capacity the host
must accept all of it or none (r4.0 Section 8.2.10.9.9.3, otherwise
the device returns Invalid Extent List). So an extent failing any of
these checks drops its whole group.

Upon passing these checks, extents are "onlined" together as a
tag group:

online_tag_group() registers a tagged group as a tag_groupX.Y device
under cxlr_dax->dev and a struct device per dc_extent under it.

A single "untagged" device is registered under cxlr_dax when
accepting the first untagged extent. All future untagged extents
will be registered under it.

A group that fails to online is dropped from the Add response.

Once the pending list has been fully processed, send the
DC_ADD_RESPONSE.

Region teardown invalidates the region's cached lines and releases all
extents under it.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: John Groves <John@Groves.net>
Signed-off-by: John Groves <John@Groves.net>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/Makefile     |   2 +-
 drivers/cxl/core/core.h       |  33 +++
 drivers/cxl/core/extent.c     | 513 ++++++++++++++++++++++++++++++++++
 drivers/cxl/core/mbox.c       | 248 +++++++++++++++-
 drivers/cxl/core/region.c     |   2 +-
 drivers/cxl/core/region_dax.c |  78 +++++-
 drivers/cxl/cxl.h             |  87 ++++++
 drivers/cxl/cxlmem.h          |   2 +
 drivers/dax/bus.c             |  13 +
 tools/testing/cxl/Kbuild      |   5 +-
 10 files changed, 974 insertions(+), 9 deletions(-)
 create mode 100644 drivers/cxl/core/extent.c

diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile
index ce7213818d3c..208917ad8aac 100644
--- a/drivers/cxl/core/Makefile
+++ b/drivers/cxl/core/Makefile
@@ -15,7 +15,7 @@ cxl_core-y += hdm.o
 cxl_core-y += pmu.o
 cxl_core-y += cdat.o
 cxl_core-$(CONFIG_TRACING) += trace.o
-cxl_core-$(CONFIG_CXL_REGION) += region.o region_pmem.o region_dax.o
+cxl_core-$(CONFIG_CXL_REGION) += region.o region_pmem.o region_dax.o extent.o
 cxl_core-$(CONFIG_CXL_MCE) += mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o
diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 24fbb5718abc..759f2c47d765 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -40,6 +40,8 @@ int cxl_dc_mode_parse(const char *buf, bool *shareable, bool *read_only);
 
 #ifdef CONFIG_CXL_REGION
 
+int cxl_region_invalidate_memregion(struct cxl_region *cxlr);
+
 struct cxl_region_context {
 	struct cxl_endpoint_decoder *cxled;
 	struct range hpa_range;
@@ -75,12 +77,42 @@ int devm_cxl_add_dax_region(struct cxl_region *cxlr);
 int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
 void kill_regions(struct cxl_root_decoder *cxlrd);
 
+int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
+		   u16 seq_num);
+int online_tag_group(struct cxl_dc_tag_group *group);
+bool rm_tag_group(struct cxl_dc_tag_group *group);
+void discard_tag_group(struct cxl_dc_tag_group *group);
+int cxl_dax_untagged_add(struct cxl_dax_region *cxlr_dax);
 #else
 static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr,
 				 const struct cxl_memdev *cxlmd, u64 dpa)
 {
 	return ULLONG_MAX;
 }
+
+static inline int cxl_add_extent(struct cxl_memdev_state *mds,
+				 struct cxl_extent *extent, u16 seq_num)
+{
+	return -ENXIO;
+}
+
+static inline int online_tag_group(struct cxl_dc_tag_group *group)
+{
+	return 0;
+}
+
+static inline bool rm_tag_group(struct cxl_dc_tag_group *group)
+{
+	return true;
+}
+
+static inline void discard_tag_group(struct cxl_dc_tag_group *group) { }
+
+static inline int cxl_dax_untagged_add(struct cxl_dax_region *cxlr_dax)
+{
+	return 0;
+}
+
 static inline
 struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa,
 				     struct cxl_endpoint_decoder **cxled)
@@ -182,6 +214,7 @@ long cxl_pci_get_latency(struct pci_dev *pdev);
 int cxl_pci_get_bandwidth(struct pci_dev *pdev, struct access_coordinate *c);
 int cxl_port_get_switch_dport_bandwidth(struct cxl_port *port,
 					struct access_coordinate *c);
+void memdev_release_group(struct cxl_dc_tag_group *group);
 
 static inline struct device *port_to_host(struct cxl_port *port)
 {
diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c
new file mode 100644
index 000000000000..8afb18e7ae5a
--- /dev/null
+++ b/drivers/cxl/core/extent.c
@@ -0,0 +1,513 @@
+// SPDX-License-Identifier: GPL-2.0
+/*  Copyright(c) 2024 Intel Corporation. All rights reserved. */
+
+#include <linux/device.h>
+#include <cxl.h>
+
+#include "core.h"
+
+static struct cxl_dc_tag_group *to_dc_tag_group(struct device *dev)
+{
+	return container_of(dev, struct cxl_dc_tag_group, dev);
+}
+
+static void dc_tag_group_release(struct device *dev)
+{
+	struct cxl_dc_tag_group *group = to_dc_tag_group(dev);
+
+	xa_destroy(&group->dc_extents);
+	/* Drop the pin taken in alloc_tag_group(). */
+	put_device(&group->cxlr_dax->dev);
+	kfree(group);
+}
+
+static const struct device_type dc_tag_group_type = {
+	.name = "tag_group",
+	.release = dc_tag_group_release,
+};
+
+/* Last extent gone: drop the group's own reference, freeing it */
+static void free_tag_group(struct cxl_dc_tag_group *group)
+{
+	put_device(&group->dev);
+}
+
+/* A tagged group is a tag_groupX.Y device, parent of its extents */
+static int dc_tag_group_add(struct cxl_dc_tag_group *group)
+{
+	struct cxl_dax_region *cxlr_dax = group->cxlr_dax;
+	struct device *dev = &group->dev;
+	int id, rc;
+
+	id = ida_alloc(&cxlr_dax->tag_group_ida, GFP_KERNEL);
+	if (id < 0)
+		return id;
+	dev->id = id;
+
+	rc = dev_set_name(dev, "tag_group%d.%d", cxlr_dax->cxlr->id, id);
+	if (!rc)
+		rc = device_add(dev);
+	if (rc)
+		ida_free(&cxlr_dax->tag_group_ida, id);
+	return rc;
+}
+
+/* Called once the group's extents are gone; a no-op for untagged groups */
+static void dc_tag_group_del(struct cxl_dc_tag_group *group)
+{
+	if (!device_is_registered(&group->dev))
+		return;
+	device_del(&group->dev);
+	ida_free(&group->cxlr_dax->tag_group_ida, group->dev.id);
+}
+
+static void dc_extent_release(struct device *dev)
+{
+	struct dc_extent *dc_extent = to_dc_extent(dev);
+	struct cxl_dc_tag_group *group;
+
+	if (!dc_extent)
+		return;
+
+	group = dc_extent->group;
+	xa_erase(&group->cxlr_dax->dc_extents, dc_extent->dev.id);
+	xa_erase(&group->dc_extents, dc_extent->seq_num);
+	group->nr_extents--;
+	if (!group->nr_extents)
+		free_tag_group(group);
+	kfree_rcu(dc_extent, rcu);
+}
+
+static const struct device_type dc_extent_type = {
+	.name = "extent",
+	.release = dc_extent_release,
+};
+
+bool is_dc_extent(struct device *dev)
+{
+	return dev->type == &dc_extent_type;
+}
+EXPORT_SYMBOL_NS_GPL(is_dc_extent, "CXL");
+
+static struct cxl_dc_tag_group *
+alloc_tag_group(struct cxl_dax_region *cxlr_dax, uuid_t *uuid)
+{
+	struct cxl_dc_tag_group *group __free(kfree) = kzalloc_obj(*group);
+	if (!group)
+		return ERR_PTR(-ENOMEM);
+
+	group->cxlr_dax = cxlr_dax;
+	uuid_copy(&group->uuid, uuid);
+	xa_init(&group->dc_extents);
+
+	/*
+	 * Pin cxlr_dax: it is used after cxl_rwsem.region is dropped, so a
+	 * refcount must keep it alive until the group is freed.
+	 */
+	get_device(&cxlr_dax->dev);
+
+	/* Added by online_tag_group() for a tagged group, see dc_tag_group_add() */
+	device_initialize(&group->dev);
+	device_set_pm_not_required(&group->dev);
+	group->dev.parent = &cxlr_dax->dev;
+	group->dev.type = &dc_tag_group_type;
+
+	return no_free_ptr(group);
+}
+
+/*
+ * Stage 1 of the add pipeline: pure, no allocation.  Resolve the extent
+ * to its region/endpoint decoder and ext_range, and verify the range
+ * fits in the resolved endpoint decoder's DPA resource.
+ *
+ * Caller must hold cxl_rwsem.region and cxl_rwsem.dpa for read:
+ * cxl_dpa_to_region() walks every decoder's dpa_res.
+ * On success, @out_cxled / @out_cxlr_dax / @out_ext_range carry the
+ * resolved handles consumed by the rest of the pipeline.
+ */
+static int cxl_validate_extent(struct cxl_memdev_state *mds,
+			       struct cxl_extent *extent,
+			       struct cxl_endpoint_decoder **out_cxled,
+			       struct cxl_dax_region **out_cxlr_dax,
+			       struct range *out_ext_range)
+{
+	u64 start_dpa = le64_to_cpu(extent->start_dpa);
+	struct cxl_memdev *cxlmd = mds->cxlds.cxlmd;
+	struct cxl_endpoint_decoder *cxled;
+	struct cxl_region *cxlr;
+	struct range ext_range = (struct range) {
+		.start = start_dpa,
+		.end = start_dpa + le64_to_cpu(extent->length) - 1,
+	};
+	struct range ed_range;
+
+	cxlr = cxl_dpa_to_region(cxlmd, start_dpa, &cxled);
+	if (!cxlr || !cxlr->cxlr_dax)
+		return -ENXIO;
+
+	ed_range = (struct range) {
+		.start = cxled->dpa_res->start,
+		.end = cxled->dpa_res->end,
+	};
+	if (!range_contains(&ed_range, &ext_range)) {
+		dev_err_ratelimited(&cxled->cxld.dev,
+				    "DC extent DPA %pra (%pU) is not fully in ED %pra\n",
+				    &ext_range, extent->uuid, &ed_range);
+		return -ENXIO;
+	}
+
+	*out_cxled = cxled;
+	*out_cxlr_dax = cxlr->cxlr_dax;
+	*out_ext_range = ext_range;
+	return 0;
+}
+
+enum cxl_extent_class {
+	CXL_EXT_NEW,
+	CXL_EXT_DUPLICATE,
+	CXL_EXT_OVERLAP,
+};
+
+/*
+ * Stage 2: classify @ext_range against extents already accepted on this
+ * cxlr_dax+cxled.  Walks cxlr_dax->dc_extents once: a stored extent that
+ * fully contains @ext_range means the device re-offered accepted capacity;
+ * a stored extent that only overlaps means an inconsistent offer.
+ */
+static enum cxl_extent_class
+cxlr_dax_classify_extent(struct cxl_dax_region *cxlr_dax,
+			 struct cxl_endpoint_decoder *cxled,
+			 const struct range *ext_range)
+{
+	struct dc_extent *entry;
+	unsigned long i;
+
+	/* kfree_rcu() defers the free; hold RCU so entries stay live while walked */
+	guard(rcu)();
+	xa_for_each(&cxlr_dax->dc_extents, i, entry) {
+		if (entry->cxled != cxled)
+			continue;
+		if (range_contains(&entry->dpa_range, ext_range))
+			return CXL_EXT_DUPLICATE;
+		if (range_overlaps(&entry->dpa_range, ext_range))
+			return CXL_EXT_OVERLAP;
+	}
+	return CXL_EXT_NEW;
+}
+
+/*
+ * Stage 3: allocate and populate a dc_extent for an already-validated,
+ * already-classified-as-new @ext_range.  Only -ENOMEM can fail here.
+ */
+static struct dc_extent *
+dc_extent_build(struct cxl_endpoint_decoder *cxled,
+		struct cxl_dax_region *cxlr_dax,
+		struct cxl_extent *extent,
+		const struct range *ext_range, u16 seq_num)
+{
+	resource_size_t dpa_offset = ext_range->start - cxled->dpa_res->start;
+	resource_size_t hpa = cxled->cxld.hpa_range.start + dpa_offset;
+	struct dc_extent *dc_extent;
+
+	dc_extent = kzalloc_obj(*dc_extent);
+	if (!dc_extent)
+		return ERR_PTR(-ENOMEM);
+
+	dc_extent->cxled = cxled;
+	dc_extent->dpa_range = *ext_range;
+	dc_extent->hpa_range.start = hpa - cxlr_dax->hpa_range.start;
+	dc_extent->hpa_range.end = dc_extent->hpa_range.start +
+				   range_len(ext_range) - 1;
+	dc_extent->seq_num = seq_num;
+	import_uuid(&dc_extent->uuid, extent->uuid);
+	return dc_extent;
+}
+
+/*
+ * Stage 4: insert @dc_extent into the pending tag group.  All extents in
+ * one More-chain group share a UUID — enforced here as the group is
+ * either being created (first extent) or appended to.  On any failure
+ * the dc_extent is freed.
+ *
+ * Returns 0 on success, <0 on error.
+ */
+static int cxlr_add_extent(struct cxl_memdev_state *mds,
+			   struct cxl_dax_region *cxlr_dax,
+			   struct dc_extent *dc_extent)
+{
+	struct cxl_dc_tag_group **group = &mds->dc_chain.group;
+	int rc;
+
+	if (*group && !uuid_equal(&(*group)->uuid, &dc_extent->uuid)) {
+		kfree(dc_extent);
+		return -EINVAL;
+	}
+
+	if (!*group) {
+		dev_dbg(&cxlr_dax->dev, "Alloc new tag group\n");
+		*group = alloc_tag_group(cxlr_dax, &dc_extent->uuid);
+		if (IS_ERR(*group)) {
+			rc = PTR_ERR(*group);
+			*group = NULL;
+			kfree(dc_extent);
+			return rc;
+		}
+	} else {
+		dev_dbg(&cxlr_dax->dev, "Append dc_extent to tag group\n");
+	}
+
+	dc_extent->group = *group;
+
+	/*
+	 * Key by @seq_num so iteration order equals assembly order.  @seq_num
+	 * is a dense 0..n-1 index (see &struct dc_extent), so a collision
+	 * here signals a cxl-side validation gap.
+	 */
+	rc = xa_insert(&(*group)->dc_extents, dc_extent->seq_num,
+		       dc_extent, GFP_KERNEL);
+	if (rc) {
+		dev_WARN_ONCE(&cxlr_dax->dev, rc == -EBUSY,
+			      "duplicate seq_num %u in tag %pUb\n",
+			      dc_extent->seq_num, &dc_extent->uuid);
+		kfree(dc_extent);
+		return rc;
+	}
+
+	return 0;
+}
+
+/*
+ * Returns 0 for a successfully added extent and <0 on error.
+ */
+int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
+		   u16 seq_num)
+{
+	struct cxl_endpoint_decoder *cxled;
+	struct cxl_dax_region *cxlr_dax;
+	struct dc_extent *dc_extent;
+	struct range ext_range;
+	int rc;
+
+	/* The caller holds both across the whole group, through online */
+	lockdep_assert_held(&cxl_rwsem.region);
+	lockdep_assert_held(&cxl_rwsem.dpa);
+
+	rc = cxl_validate_extent(mds, extent, &cxled, &cxlr_dax, &ext_range);
+	if (rc)
+		return rc;
+
+	switch (cxlr_dax_classify_extent(cxlr_dax, cxled, &ext_range)) {
+	case CXL_EXT_DUPLICATE:
+		/*
+		 * Never re-reported: the device rejects a DPA range already
+		 * added by a previous response (CXL r4.0 8.2.10.9.9.3).
+		 */
+		dev_warn_ratelimited(&cxled->cxld.dev,
+				     "Extent %pra already accepted; dropping group\n",
+				     &ext_range);
+		return -EEXIST;
+	case CXL_EXT_OVERLAP:
+		return -ENXIO;
+	case CXL_EXT_NEW:
+		break;
+	}
+
+	dc_extent = dc_extent_build(cxled, cxlr_dax, extent, &ext_range,
+				    seq_num);
+	if (IS_ERR(dc_extent))
+		return PTR_ERR(dc_extent);
+
+	dev_dbg(&cxled->cxld.dev, "Add extent %pra (%pU)\n",
+		&dc_extent->dpa_range, &dc_extent->uuid);
+
+	return cxlr_add_extent(mds, cxlr_dax, dc_extent);
+}
+
+/* The group's device may already be visible: readers walk the xarray under RCU */
+static void cleanup_pending_dc_extent(struct dc_extent *dc_extent)
+{
+	struct cxl_dc_tag_group *group = dc_extent->group;
+
+	xa_erase(&group->dc_extents, dc_extent->seq_num);
+	group->nr_extents--;
+	if (!group->nr_extents)
+		free_tag_group(group);
+	kfree_rcu(dc_extent, rcu);
+}
+
+/*
+ * Drop a staged group that never went online: no dc_extent device exists
+ * yet, so there is nothing to unregister and nothing to tell the device.
+ */
+void discard_tag_group(struct cxl_dc_tag_group *group)
+{
+	struct dc_extent *dc_extent;
+	unsigned long index;
+
+	/* Seed and pin nr_extents as online_tag_group() would */
+	xa_for_each(&group->dc_extents, index, dc_extent)
+		group->nr_extents++;
+	group->nr_extents++;
+
+	xa_for_each(&group->dc_extents, index, dc_extent)
+		cleanup_pending_dc_extent(dc_extent);
+
+	group->nr_extents--;
+	if (!group->nr_extents)
+		free_tag_group(group);
+}
+
+/* Unregister every member; the group is freed with its last extent */
+static void unregister_tag_group(struct cxl_dc_tag_group *group)
+{
+	struct dc_extent *dc_extent;
+	unsigned long index;
+
+	/*
+	 * Pin @group across the walk: unregistering a dc_extent drops its
+	 * last reference and fires dc_extent_release(), which frees @group
+	 * on the final nr_extents decrement.  Without the pin the next
+	 * iteration's xa_find_after() dereferences a freed xarray.
+	 */
+	group->nr_extents++;
+	xa_for_each(&group->dc_extents, index, dc_extent) {
+		dev_dbg(&dc_extent->dev, "DAX region rm extent HPA %pra\n",
+			&dc_extent->hpa_range);
+		device_unregister(&dc_extent->dev);
+	}
+	dc_tag_group_del(group);
+	group->nr_extents--;
+	if (!group->nr_extents)
+		free_tag_group(group);
+}
+
+/*
+ * Region teardown, a devres action on the dax_region device: the host gives
+ * the group's capacity back to the device.  Flush the region's cached lines
+ * first; once the Release DC goes out the device may reassign the range.
+ * devres order puts the DAX devices created after the group ahead of it.
+ */
+static void dc_tag_group_teardown(void *_group)
+{
+	struct cxl_dc_tag_group *group = _group;
+
+	/* Exclude a release flush, which holds the lock for read */
+	guard(rwsem_write)(&cxl_rwsem.region);
+	cxl_region_invalidate_memregion(group->cxlr_dax->cxlr);
+	memdev_release_group(group);
+	unregister_tag_group(group);
+}
+
+/*
+ * Drop @group without telling the device anything: the caller has already
+ * decided what, if anything, the device is told about this capacity.
+ * Returns false when the dax region's unbind already owns the teardown.
+ */
+bool rm_tag_group(struct cxl_dc_tag_group *group)
+{
+	if (devm_remove_action_nowarn(&group->cxlr_dax->dev,
+				      dc_tag_group_teardown, group))
+		return false;
+	unregister_tag_group(group);
+	return true;
+}
+
+int online_tag_group(struct cxl_dc_tag_group *group)
+{
+	struct cxl_dax_region *cxlr_dax = group->cxlr_dax;
+	struct device *parent = &cxlr_dax->untagged;
+	struct dc_extent *dc_extent;
+	unsigned long index;
+	int rc = 0;
+
+	/*
+	 * Seed nr_extents with the full group size plus a +1 pin held by
+	 * this function.  The size counts every dc_extent that might
+	 * decrement nr_extents on cleanup; the pin keeps @group alive
+	 * across the body even if every dc_extent release fires in the
+	 * unwind (e.g. a devm_add_action() failure with a single extent).
+	 * The pin is dropped at the end of the function.
+	 */
+	xa_for_each(&group->dc_extents, index, dc_extent)
+		group->nr_extents++;
+	group->nr_extents++;
+
+	/* A tagged group parents its extents; untagged ones share @untagged */
+	if (!uuid_is_null(&group->uuid)) {
+		rc = dc_tag_group_add(group);
+		if (rc)
+			goto unwind;
+		parent = &group->dev;
+	} else {
+		rc = cxl_dax_untagged_add(cxlr_dax);
+		if (rc)
+			goto unwind;
+	}
+
+	xa_for_each(&group->dc_extents, index, dc_extent) {
+		struct device *dev = &dc_extent->dev;
+		u32 id;
+
+		device_initialize(dev);
+		device_set_pm_not_required(dev);
+		dev->parent = parent;
+		dev->type = &dc_extent_type;
+
+		rc = xa_alloc(&cxlr_dax->dc_extents, &id, dc_extent,
+			      xa_limit_32b, GFP_KERNEL);
+		/*
+		 * put_device() fires dc_extent_release().  On xa_alloc failure
+		 * dev->id is still its invalid init value (0), but the xarray is
+		 * declared XA_FLAGS_ALLOC1 so 0 is never a valid id and erasing
+		 * it cannot remove another extent.
+		 */
+		if (rc < 0) {
+			put_device(dev);
+			break;
+		}
+		dev->id = id;
+
+		rc = dev_set_name(dev, "extent%d.%d", cxlr_dax->cxlr->id,
+				  dev->id);
+		if (rc) {
+			put_device(dev);
+			break;
+		}
+
+		rc = device_add(dev);
+		if (rc) {
+			put_device(dev);
+			break;
+		}
+
+		dev_dbg(dev, "dc_extent HPA %pra (%pU)\n",
+			&dc_extent->hpa_range, &group->uuid);
+	}
+
+	/* Give the capacity back when the region goes away */
+	if (!rc)
+		rc = devm_add_action(&cxlr_dax->dev, dc_tag_group_teardown, group);
+
+unwind:
+	if (rc) {
+		/*
+		 * Unwind every remaining dc_extent in the group.  The pin
+		 * above keeps @group alive across this walk.  Onlined
+		 * dc_extents are unregistered; pending ones were never
+		 * initialized as devices.
+		 */
+		xa_for_each(&group->dc_extents, index, dc_extent) {
+			if (device_is_registered(&dc_extent->dev))
+				device_unregister(&dc_extent->dev);
+			else
+				cleanup_pending_dc_extent(dc_extent);
+		}
+		dc_tag_group_del(group);
+	}
+
+	/* Drop the pin; if nothing else still references @group, free it. */
+	group->nr_extents--;
+	if (!group->nr_extents)
+		free_tag_group(group);
+	return rc;
+}
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index fea910e6c510..18044f216de5 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -6,6 +6,7 @@
 #include <linux/mutex.h>
 #include <linux/unaligned.h>
 #include <linux/list.h>
+#include <linux/list_sort.h>
 #include <cxlpci.h>
 #include <cxlmem.h>
 #include <cxl.h>
@@ -1201,6 +1202,84 @@ static void delete_extent_node(struct cxl_extent_list_node *node)
 	kfree(node);
 }
 
+/* Drop every node in @group, releasing its extent. */
+static void drop_extent_group(struct list_head *group)
+{
+	struct cxl_extent_list_node *pos, *tmp;
+
+	list_for_each_entry_safe(pos, tmp, group, list)
+		delete_extent_node(pos);
+}
+
+/* Queue @r for a Release DC command */
+static int stage_release_ack(struct list_head *acks, const struct range *r)
+{
+	struct cxl_extent_list_node *node = kzalloc_obj(*node);
+	struct cxl_extent *extent;
+
+	if (!node)
+		return -ENOMEM;
+	extent = kzalloc_obj(*extent);
+	if (!extent) {
+		kfree(node);
+		return -ENOMEM;
+	}
+
+	extent->start_dpa = cpu_to_le64(r->start);
+	extent->length = cpu_to_le64(range_len(r));
+	node->extent = extent;
+	list_add_tail(&node->list, acks);
+	return 0;
+}
+
+/*
+ * Queue every member of @group on @acks.  Returns the number queued, or
+ * -ENOMEM with nothing queued.
+ */
+static int stage_group_release(struct cxl_dc_tag_group *group,
+			       struct list_head *acks)
+{
+	struct dc_extent *dc_extent;
+	LIST_HEAD(members);
+	unsigned long idx;
+	int cnt = 0, rc;
+
+	xa_for_each(&group->dc_extents, idx, dc_extent) {
+		rc = stage_release_ack(&members, &dc_extent->dpa_range);
+		if (rc) {
+			drop_extent_group(&members);
+			return rc;
+		}
+		cnt++;
+	}
+
+	list_splice_tail(&members, acks);
+	return cnt;
+}
+
+/* Give @group's capacity back to the device with a Release DC response */
+void memdev_release_group(struct cxl_dc_tag_group *group)
+{
+	unsigned long idx = 0;
+	struct dc_extent *dc_extent = xa_find(&group->dc_extents, &idx,
+					      ULONG_MAX, XA_PRESENT);
+	struct cxl_memdev_state *mds;
+	LIST_HEAD(acks);
+	int cnt;
+
+	if (!dc_extent)
+		return;
+	/* A DC region is not interleaved: every member shares one memdev */
+	mds = cxled_to_mds(dc_extent->cxled);
+
+	cnt = stage_group_release(group, &acks);
+	if (cnt < 0 ||
+	    cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC, &acks, cnt))
+		dev_dbg(mds->cxlds.dev, "Failed to release tag %pUb\n",
+			&group->uuid);
+	drop_extent_group(&acks);
+}
+
 static void clear_pending_extents(void *_mds)
 {
 	struct cxl_memdev_state *mds = _mds;
@@ -1208,6 +1287,7 @@ static void clear_pending_extents(void *_mds)
 
 	list_for_each_entry_safe(pos, tmp, &mds->dc_chain.pending_extents, list)
 		delete_extent_node(pos);
+	mds->dc_chain.group = NULL;
 }
 
 /*
@@ -1315,6 +1395,168 @@ static int stage_pending_extent(struct list_head *pending,
 	return 0;
 }
 
+/*
+ * Compare two extents by shared_extn_seq (ascending).  list_sort is
+ * stable, so extents with equal keys keep their arrival order from
+ * stage_pending_extent()'s list_add_tail().
+ */
+static int extent_seq_compare(void *priv,
+			      const struct list_head *a,
+			      const struct list_head *b)
+{
+	const struct cxl_extent_list_node *ea =
+		list_entry(a, struct cxl_extent_list_node, list);
+	const struct cxl_extent_list_node *eb =
+		list_entry(b, struct cxl_extent_list_node, list);
+	u16 sa = le16_to_cpu(ea->extent->shared_extn_seq);
+	u16 sb = le16_to_cpu(eb->extent->shared_extn_seq);
+
+	if (sa < sb)
+		return -1;
+	if (sa > sb)
+		return 1;
+	return 0;
+}
+
+/*
+ * Move every pending extent whose tag matches @tag onto @group, preserving
+ * the order they appear in @pending.
+ */
+static void extract_tag_group(struct list_head *pending,
+			      const uuid_t *tag,
+			      struct list_head *group)
+{
+	struct cxl_extent_list_node *pos, *tmp;
+
+	list_for_each_entry_safe(pos, tmp, pending, list) {
+		uuid_t t;
+
+		import_uuid(&t, pos->extent->uuid);
+		if (uuid_equal(&t, tag))
+			list_move_tail(&pos->list, group);
+	}
+}
+
+/*
+ * Realize a tag @group: assign each extent its dax-side @seq_num and add it
+ * via cxl_add_extent(), then online the resulting tag group.  Returns the
+ * number of accepted extents (>= 0) with @group left holding them for the
+ * caller to splice, or a negative errno on failure with @group untouched for
+ * the caller to drop.
+ */
+static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
+			     struct list_head *group)
+{
+	struct device *dev = mds->cxlds.dev;
+	struct cxl_extent_list_node *pos;
+	struct cxl_dc_tag_group *tag_group;
+	int group_cnt = 0;
+	u16 seq_num;
+	int rc;
+
+	lockdep_assert_held(&cxl_rwsem.region);
+	lockdep_assert_held(&cxl_rwsem.dpa);
+
+	/* Each extent gets a dense dax-side seq_num, 0..n-1 */
+	seq_num = -1;
+	list_for_each_entry(pos, group, list) {
+		seq_num++;
+		rc = cxl_add_extent(mds, pos->extent, seq_num);
+		if (rc < 0) {
+			dev_dbg(dev,
+				"Tag %pUb: dropping group, failed to add extent DPA:%#llx LEN:%#llx\n",
+				tag,
+				le64_to_cpu(pos->extent->start_dpa),
+				le64_to_cpu(pos->extent->length));
+			goto err_discard;
+		}
+		group_cnt++;
+	}
+
+	tag_group = mds->dc_chain.group;
+	mds->dc_chain.group = NULL;
+	rc = online_tag_group(tag_group);
+	if (rc) {
+		dev_warn(dev, "Tag %pUb: failed to online tag group (%d)\n",
+			 tag, rc);
+		return rc;
+	}
+
+	return group_cnt;
+
+err_discard:
+	/*
+	 * A tag group is one allocation and sharable capacity must be
+	 * accepted all or none (CXL r4.0 8.2.10.9.9.3): unstage what was
+	 * built before the failing extent.
+	 */
+	tag_group = mds->dc_chain.group;
+	mds->dc_chain.group = NULL;
+	if (tag_group)
+		discard_tag_group(tag_group);
+	return rc;
+}
+
+/*
+ * Drive the pending Add-Capacity records through cxl_realize_group() one
+ * tag group at a time, moving each group that realizes onto the accepted
+ * list.
+ */
+static int cxl_add_pending(struct cxl_memdev_state *mds,
+			   struct list_head *pending)
+{
+	LIST_HEAD(accepted);
+	int total_accepted = 0;
+
+	/*
+	 * Held across validation, add and online so that cxlr_dax_unpublish()
+	 * excludes a whole flush; cxl_dpa_to_region() also walks every dpa_res.
+	 */
+	guard(rwsem_read)(&cxl_rwsem.region);
+	guard(rwsem_read)(&cxl_rwsem.dpa);
+
+	while (!list_empty(pending)) {
+		struct cxl_extent_list_node *first;
+		LIST_HEAD(group);
+		uuid_t tag;
+		int cnt;
+
+		first = list_first_entry(pending, struct cxl_extent_list_node,
+					 list);
+		import_uuid(&tag, first->extent->uuid);
+		/*
+		 * A null tag carries no allocation identity: surface each
+		 * untagged extent as its own single-extent group rather than
+		 * coalescing all untagged extents together.
+		 */
+		if (uuid_is_null(&tag))
+			list_move_tail(&first->list, &group);
+		else
+			extract_tag_group(pending, &tag, &group);
+
+		/*
+		 * Only a sharable allocation carries a meaningful per-extent
+		 * shared_extn_seq; order those by it.  For non-sharable groups,
+		 * the stable sort maintains arrival order.
+		 */
+		list_sort(NULL, &group, extent_seq_compare);
+
+		cnt = cxl_realize_group(mds, &tag, &group);
+		if (cnt < 0) {
+			drop_extent_group(&group);
+			continue;
+		}
+
+		list_splice_tail_init(&group, &accepted);
+		total_accepted += cnt;
+	}
+
+	list_splice(&accepted, pending);
+
+	return cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
+				    pending, total_accepted);
+}
+
 static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 				  struct cxl_event_dcd *event)
 {
@@ -1359,11 +1601,9 @@ static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 	chain->armed = false;
 	cancel_delayed_work(&chain->timeout_work);
 
-	/* Nothing is tracked yet: refuse every add, acknowledge every release */
 	if (event->event_type == DCD_ADD_CAPACITY)
-		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
-					  &chain->pending_extents, 0);
-	else
+		rc = cxl_add_pending(mds, &chain->pending_extents);
+	else	/* Nothing is tracked yet: acknowledge every release */
 		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC,
 					  &chain->pending_extents,
 					  list_count_nodes(&chain->pending_extents));
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index fc9139ce358f..1b7031b2f159 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -224,7 +224,7 @@ static struct cxl_region_ref *cxl_rr_load(struct cxl_port *port,
 	return xa_load(&port->regions, (unsigned long)cxlr);
 }
 
-static int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
+int cxl_region_invalidate_memregion(struct cxl_region *cxlr)
 {
 	if (!cpu_cache_has_invalidate_memregion()) {
 		if (IS_ENABLED(CONFIG_CXL_REGION_INVALIDATION_TEST)) {
diff --git a/drivers/cxl/core/region_dax.c b/drivers/cxl/core/region_dax.c
index 47c9c52a3902..790176a8aadd 100644
--- a/drivers/cxl/core/region_dax.c
+++ b/drivers/cxl/core/region_dax.c
@@ -13,6 +13,8 @@ static void cxl_dax_region_release(struct device *dev)
 {
 	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
 
+	xa_destroy(&cxlr_dax->dc_extents);
+	ida_destroy(&cxlr_dax->tag_group_ida);
 	kfree(cxlr_dax);
 }
 
@@ -27,6 +29,20 @@ const struct device_type cxl_dax_region_type = {
 	.groups = cxl_dax_region_attribute_groups,
 };
 
+static void cxl_dax_untagged_release(struct device *dev)
+{
+	struct cxl_dax_region *cxlr_dax =
+		container_of(dev, struct cxl_dax_region, untagged);
+
+	/* Drop the pin taken in cxl_dax_region_alloc(). */
+	put_device(&cxlr_dax->dev);
+}
+
+static const struct device_type cxl_dax_untagged_type = {
+	.name = "untagged",
+	.release = cxl_dax_untagged_release,
+};
+
 static bool is_cxl_dax_region(struct device *dev)
 {
 	return dev->type == &cxl_dax_region_type;
@@ -49,7 +65,8 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	struct cxl_dax_region *cxlr_dax;
 	struct device *dev;
 
-	guard(rwsem_read)(&cxl_rwsem.region);
+	/* Publishes cxlr->cxlr_dax, which the event paths read under the read lock */
+	guard(rwsem_write)(&cxl_rwsem.region);
 	if (p->state != CXL_CONFIG_COMMIT)
 		return ERR_PTR(-ENXIO);
 
@@ -57,11 +74,14 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	if (!cxlr_dax)
 		return ERR_PTR(-ENOMEM);
 
+	xa_init_flags(&cxlr_dax->dc_extents, XA_FLAGS_ALLOC1);
+	ida_init(&cxlr_dax->tag_group_ida);
 	cxlr_dax->hpa_range.start = p->res->start;
 	cxlr_dax->hpa_range.end = p->res->end;
 
 	dev = &cxlr_dax->dev;
 	cxlr_dax->cxlr = cxlr;
+	cxlr->cxlr_dax = cxlr_dax;
 	device_initialize(dev);
 	lockdep_set_class(&dev->mutex, &cxl_dax_region_key);
 	device_set_pm_not_required(dev);
@@ -69,16 +89,70 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	dev->bus = &cxl_bus_type;
 	dev->type = &cxl_dax_region_type;
 
+	/* Parent of the untagged extents, see cxl_dax_untagged_add() */
+	dev = &cxlr_dax->untagged;
+	device_initialize(dev);
+	device_set_pm_not_required(dev);
+	dev->parent = &cxlr_dax->dev;
+	dev->type = &cxl_dax_untagged_type;
+	get_device(&cxlr_dax->dev);
+
 	return cxlr_dax;
 }
 
+/*
+ * Clearing cxlr->cxlr_dax here cannot race an Add or Release event.
+ * Event paths find cxlr_dax through cxl_dpa_to_region() and use it with
+ * cxl_rwsem.region held for read, from resolving an extent through
+ * onlining its group.  Taking the write lock waits those paths out, and
+ * later lookups find NULL, so the teardown that follows sees no events.
+ */
+static void cxlr_dax_unpublish(struct cxl_dax_region *cxlr_dax)
+{
+	guard(rwsem_write)(&cxl_rwsem.region);
+	cxlr_dax->cxlr->cxlr_dax = NULL;
+}
+
 static void cxlr_dax_unregister(void *_cxlr_dax)
 {
 	struct cxl_dax_region *cxlr_dax = _cxlr_dax;
 
+	cxlr_dax_unpublish(cxlr_dax);
+	/* Unbind first: the driver's teardown empties untagged and the groups */
+	device_release_driver(&cxlr_dax->dev);
+	if (device_is_registered(&cxlr_dax->untagged))
+		device_del(&cxlr_dax->untagged);
+	put_device(&cxlr_dax->untagged);
 	device_unregister(&cxlr_dax->dev);
 }
 
+/*
+ * The dax_region's driver may probe, and its extent scan run, before
+ * devm_cxl_add_dax_region() returns, so the untagged device is registered
+ * on first use instead.  Callers are serialized by the memdev's
+ * dc_chain.lock; a DC region has a single memdev behind it.
+ */
+int cxl_dax_untagged_add(struct cxl_dax_region *cxlr_dax)
+{
+	struct device *dev = &cxlr_dax->untagged;
+	int rc;
+
+	if (device_is_registered(dev))
+		return 0;
+
+	rc = dev_set_name(dev, "untagged");
+	if (rc)
+		return rc;
+	return device_add(dev);
+}
+
+DEFINE_FREE(unpublish_cxl_dax_region, struct cxl_dax_region *,
+	if (!IS_ERR_OR_NULL(_T)) {
+		cxlr_dax_unpublish(_T);
+		put_device(&_T->untagged);
+		put_device(&_T->dev);
+	})
+
 int devm_cxl_add_dax_region(struct cxl_region *cxlr)
 {
 	struct device *dev;
@@ -90,7 +164,7 @@ int devm_cxl_add_dax_region(struct cxl_region *cxlr)
 		return -EINVAL;
 	}
 
-	struct cxl_dax_region *cxlr_dax __free(put_cxl_dax_region) =
+	struct cxl_dax_region *cxlr_dax __free(unpublish_cxl_dax_region) =
 		cxl_dax_region_alloc(cxlr);
 	if (IS_ERR(cxlr_dax))
 		return PTR_ERR(cxlr_dax);
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 64138e1768a1..f7d7749d704c 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -325,6 +325,42 @@ enum cxl_decoder_state {
 	CXL_DECODER_STATE_AUTO_STAGED,
 };
 
+struct cxl_dc_tag_group;
+
+/**
+ * struct dc_extent - A single dynamic-capacity extent surfaced to the host.
+ *
+ * One per device-stamped extent.  Multiple dc_extents that share a tag
+ * (see &struct cxl_dc_tag_group) form a single logical allocation, but
+ * each dc_extent has its own HPA range.
+ *
+ * @dev: device representing this extent; child of its tag_groupX.Y
+ *	device, or of the dax_region's untagged device for a null tag.
+ * @group: tag group that this extent belongs to; shared across siblings.
+ * @cxled: endpoint decoder backing the DPA range.
+ * @dpa_range: DPA range this extent covers within @cxled.
+ * @hpa_range: HPA range that @dpa_range decodes to, relative to
+ *	       cxlr_dax->hpa_range.start.
+ * @uuid: tag uuid (matches @group->uuid).
+ * @seq_num: 0..n-1 assembly-order index within the tag group.  For a
+ *	     sharable partition the order follows the shared_extn_seq given
+ *	     by the device; for a non-sharable partition, where
+ *	     shared_extn_seq is reserved, the order is the order in which
+ *	     extents arrive.  Either way @seq_num forms a dense range 0..n-1.
+ * @rcu: rcu_head for the kfree_rcu() deferred free in dc_extent_release(),
+ *	 so RCU readers walking the tag group's extents finish safely.
+ */
+struct dc_extent {
+	struct device dev;
+	struct cxl_dc_tag_group *group;
+	struct cxl_endpoint_decoder *cxled;
+	struct range dpa_range;
+	struct range hpa_range;
+	uuid_t uuid;
+	u16 seq_num;
+	struct rcu_head rcu;
+};
+
 /**
  * struct cxl_endpoint_decoder - Endpoint  / SPA to DPA decoder
  * @cxld: base cxl_decoder_object
@@ -490,6 +526,7 @@ struct cxl_region_params {
  * @type: Endpoint decoder target type
  * @cxl_nvb: nvdimm bridge for coordinating @cxlr_pmem setup / shutdown
  * @cxlr_pmem: (for pmem regions) cached copy of the nvdimm bridge
+ * @cxlr_dax: the region's cxl_dax_region while it is registered, else NULL
  * @flags: Region state flags
  * @shareable: (for DC regions) backing partitions must be shareable
  * @read_only: (for DC regions) backing partitions must be read-only
@@ -508,6 +545,7 @@ struct cxl_region {
 	enum cxl_decoder_type type;
 	struct cxl_nvdimm_bridge *cxl_nvb;
 	struct cxl_pmem_region *cxlr_pmem;
+	struct cxl_dax_region *cxlr_dax;
 	unsigned long flags;
 	bool shareable;
 	bool read_only;
@@ -570,8 +608,57 @@ struct cxl_dax_region {
 	struct device dev;
 	struct cxl_region *cxlr;
 	struct range hpa_range;
+	/*
+	 * dc_extents is keyed by an allocator-assigned u32 (see
+	 * online_tag_group()).  Tag groups have no first-class identity in
+	 * this xarray; siblings within a tag find each other via
+	 * dc_extent->group.
+	 */
+	struct xarray dc_extents;
+	/* ids for the tag_groupX.Y devices registered under this region */
+	struct ida tag_group_ida;
+	/* parent of the extentX.Y devices of untagged allocations */
+	struct device untagged;
 };
 
+/**
+ * struct cxl_dc_tag_group - A tagged dynamic-capacity allocation.
+ *
+ * Container for the &struct dc_extent siblings that share a tag.  A
+ * tagged group is a tag_groupX.Y device under the dax_region and the
+ * parent of its extentX.Y devices; untagged extents, each a group of
+ * one, sit under the region's untagged device instead.  The group
+ * exists to keep tag-scoped invariants (atomic add, atomic release,
+ * ordered carve by seq_num) in one place.
+ *
+ * @dev: device representing a tagged group; child of cxlr_dax->dev.
+ *	Holds the group's memory: the last dc_extent release drops the
+ *	final reference.  Never registered for an untagged group.
+ * @cxlr_dax: back reference to parent region device.
+ * @uuid: tag identifying this allocation; same across all member dc_extents.
+ * @dc_extents: xarray of &struct dc_extent in this group, indexed by the
+ *		dc_extent's @seq_num (0..n-1, dense).  See &struct dc_extent
+ *		for how seq_num is sourced for sharable vs non-sharable
+ *		allocations.
+ * @nr_extents: live count of dc_extents in the group; the group is freed
+ *		when the last dc_extent device is released.
+ */
+struct cxl_dc_tag_group {
+	struct device dev;
+	struct cxl_dax_region *cxlr_dax;
+	uuid_t uuid;
+	struct xarray dc_extents;
+	unsigned int nr_extents;
+};
+
+bool is_dc_extent(struct device *dev);
+static inline struct dc_extent *to_dc_extent(struct device *dev)
+{
+	if (!is_dc_extent(dev))
+		return NULL;
+	return container_of(dev, struct dc_extent, dev);
+}
+
 /**
  * struct cxl_port - logical collection of upstream port devices and
  *		     downstream port devices to construct a CXL memory
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 52c936ec15a9..73f01efc0172 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -433,6 +433,7 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
  * struct cxl_dc_event_chain - staged Dynamic Capacity event chain
  * @pending_extents: extents received so far; flushed when the chain
  *		     closes (More=0)
+ * @group: tag group being assembled from an Add Capacity chain
  * @timeout_work: watchdog that fires if a chain is opened with
  *		  CXL_DCD_EVENT_MORE but the closing record never arrives
  * @lock: serialises updates to the chain state against the watchdog
@@ -454,6 +455,7 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox)
  */
 struct cxl_dc_event_chain {
 	struct list_head pending_extents;
+	struct cxl_dc_tag_group *group;
 	struct delayed_work timeout_work;
 	struct mutex lock;	/* chain state, against the watchdog */
 	bool armed;
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index d8cc1c012f46..7d28aedfd2f7 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -539,6 +539,8 @@ static int alloc_dev_dax_id(struct dev_dax *dev_dax)
 	return id;
 }
 
+static bool is_dev_dax(struct device *dev);
+
 static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
 		const char *buf, size_t len)
 {
@@ -555,6 +557,12 @@ static ssize_t delete_store(struct device *dev, struct device_attribute *attr,
 	if (!victim)
 		return -ENXIO;
 
+	/* Tag groups and the untagged device are children of the region too */
+	if (!is_dev_dax(victim)) {
+		put_device(victim);
+		return -EINVAL;
+	}
+
 	device_lock(dev);
 	device_lock(victim);
 	dev_dax = to_dev_dax(victim);
@@ -1443,6 +1451,11 @@ static const struct device_type dev_dax_type = {
 	.groups = dax_attribute_groups,
 };
 
+static bool is_dev_dax(struct device *dev)
+{
+	return dev->type == &dev_dax_type;
+}
+
 /* see "strong" declaration in tools/testing/nvdimm/dax-dev.c */
 __weak phys_addr_t dax_pgoff_to_phys(struct dev_dax *dev_dax, pgoff_t pgoff,
 			      unsigned long size)
diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild
index e49ce644b4b1..f1ab25872e76 100644
--- a/tools/testing/cxl/Kbuild
+++ b/tools/testing/cxl/Kbuild
@@ -67,7 +67,10 @@ cxl_core-y += $(CXL_CORE_SRC)/hdm.o
 cxl_core-y += $(CXL_CORE_SRC)/pmu.o
 cxl_core-y += $(CXL_CORE_SRC)/cdat.o
 cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o
-cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o $(CXL_CORE_SRC)/region_pmem.o $(CXL_CORE_SRC)/region_dax.o
+cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o \
+				 $(CXL_CORE_SRC)/region_pmem.o \
+				 $(CXL_CORE_SRC)/region_dax.o \
+				 $(CXL_CORE_SRC)/extent.o
 cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o
 cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o
 cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o
-- 
2.43.0

[PATCH v1 07/14] cxl/mem: Drop misaligned DCD extent groups

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Add an alignment gate, cxl_validate_group(), that cxl_add_pending() runs
before realizing a tag group: every extent in the group must have its
start_dpa and length aligned to the dax region's mapping granularity.
A misaligned extent makes the resulting dax device unusable, so drop
the whole group rather than accept a partial allocation that would
surface a broken dax_resource.

Based on patches by John Groves.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: John Groves <John@Groves.net>
Signed-off-by: John Groves <John@Groves.net>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/mbox.c | 46 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 18044f216de5..a605ffc4c64f 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -7,6 +7,8 @@
 #include <linux/unaligned.h>
 #include <linux/list.h>
 #include <linux/list_sort.h>
+#include <linux/pgtable.h>
+#include <linux/sizes.h>
 #include <cxlpci.h>
 #include <cxlmem.h>
 #include <cxl.h>
@@ -1395,6 +1397,19 @@ static int stage_pending_extent(struct list_head *pending,
 	return 0;
 }
 
+/*
+ * Extents need to be aligned to dax region's mapping granularity.
+ * Use PMD_SIZE, since cxl_dax_region_probe() calls alloc_dax_region with
+ * PMD_SIZE for the 'align' parameter.
+ */
+static bool cxl_extent_dcd_aligned(const struct cxl_extent *extent)
+{
+	u64 start = le64_to_cpu(extent->start_dpa);
+	u64 len = le64_to_cpu(extent->length);
+
+	return IS_ALIGNED(start, PMD_SIZE) && IS_ALIGNED(len, PMD_SIZE);
+}
+
 /*
  * Compare two extents by shared_extn_seq (ascending).  list_sort is
  * stable, so extents with equal keys keep their arrival order from
@@ -1497,6 +1512,32 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 	return rc;
 }
 
+/*
+ * Validate a tag @group before realizing it.  Returns 0 if the group may be
+ * added, or a negative errno if it must be dropped.
+ */
+static int cxl_validate_group(struct cxl_memdev_state *mds, const uuid_t *tag,
+			      struct list_head *group)
+{
+	struct device *dev = mds->cxlds.dev;
+	struct cxl_extent_list_node *pos;
+
+	/* Alignment gate — drop the group if any member fails */
+	list_for_each_entry(pos, group, list) {
+		if (!cxl_extent_dcd_aligned(pos->extent)) {
+			dev_warn_ratelimited(dev,
+					     "Tag %pUb: dropping group, extent DPA:%#llx LEN:%#llx not %#llx-aligned\n",
+					     tag,
+					     le64_to_cpu(pos->extent->start_dpa),
+					     le64_to_cpu(pos->extent->length),
+					     (u64)PMD_SIZE);
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
 /*
  * Drive the pending Add-Capacity records through cxl_realize_group() one
  * tag group at a time, moving each group that realizes onto the accepted
@@ -1541,6 +1582,11 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 		 */
 		list_sort(NULL, &group, extent_seq_compare);
 
+		if (cxl_validate_group(mds, &tag, &group)) {
+			drop_extent_group(&group);
+			continue;
+		}
+
 		cnt = cxl_realize_group(mds, &tag, &group);
 		if (cnt < 0) {
 			drop_extent_group(&group);
-- 
2.43.0

[PATCH v1 08/14] cxl/extent: Validate DC extent partition

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Extend cxl_validate_extent() — the per-extent check of the add pipeline
to check partition membership.

Resolves an extent's DPA to its containing DC partition.  Sharability is
a property of the partition (part->shareable), taken from its CDAT DSMAS
entry.

An extent from a sharable partition must carry a non-null tag, since hosts
sharing the allocation key on that tag.  A null tag there is a device
firmware bug; reject the extent.

A non-sharable extent must carry 0 in shared_extn_seq (CXL r4.0 Table
8-230); a non-zero value is rejected the same way.  The dense 0..n-1
check for sharable groups follows in the next commit, once the whole
tag group is collected.

Based on patches by John Groves.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: John Groves <John@Groves.net>
Signed-off-by: John Groves <John@Groves.net>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/core.h   | 12 +++++++
 drivers/cxl/core/extent.c | 70 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 82 insertions(+)

diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 759f2c47d765..528eec2a1cc3 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -83,6 +83,10 @@ int online_tag_group(struct cxl_dc_tag_group *group);
 bool rm_tag_group(struct cxl_dc_tag_group *group);
 void discard_tag_group(struct cxl_dc_tag_group *group);
 int cxl_dax_untagged_add(struct cxl_dax_region *cxlr_dax);
+const struct cxl_dpa_partition *
+cxl_extent_dc_partition(struct cxl_memdev_state *mds,
+			struct cxl_extent *extent,
+			struct range *ext_range);
 #else
 static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr,
 				 const struct cxl_memdev *cxlmd, u64 dpa)
@@ -144,6 +148,14 @@ static inline void kill_regions(struct cxl_root_decoder *cxlrd) { };
 #define SET_CXL_REGION_ATTR(x)
 #define CXL_PMEM_REGION_TYPE(x) NULL
 #define CXL_DAX_REGION_TYPE(x) NULL
+
+static inline const struct cxl_dpa_partition *
+cxl_extent_dc_partition(struct cxl_memdev_state *mds,
+			struct cxl_extent *extent,
+			struct range *ext_range)
+{
+	return NULL;
+}
 #endif
 
 struct cxl_send_command;
diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c
index 8afb18e7ae5a..334a97ab583d 100644
--- a/drivers/cxl/core/extent.c
+++ b/drivers/cxl/core/extent.c
@@ -115,6 +115,51 @@ alloc_tag_group(struct cxl_dax_region *cxlr_dax, uuid_t *uuid)
 	return no_free_ptr(group);
 }
 
+/*
+ * Find the DC (Dynamic Capacity) partition that fully contains @ext_range,
+ * or NULL if the extent falls outside every DC partition on this memdev.
+ * The returned pointer is owned by mds->cxlds.part[] and lives for the
+ * lifetime of the memdev.
+ */
+const struct cxl_dpa_partition *
+cxl_extent_dc_partition(struct cxl_memdev_state *mds,
+			struct cxl_extent *extent,
+			struct range *ext_range)
+{
+	struct cxl_dev_state *cxlds = &mds->cxlds;
+	struct device *dev = mds->cxlds.dev;
+
+	/*
+	 * A device-side error could cause end < start, which range_contains()
+	 * would treat as contained in any partition.
+	 */
+	if (ext_range->end < ext_range->start) {
+		dev_err_ratelimited(dev,
+				    "DC extent DPA %pra (%pU) has invalid length (firmware bug)\n",
+				    ext_range, extent->uuid);
+		return NULL;
+	}
+
+	for (int i = 0; i < cxlds->nr_partitions; i++) {
+		struct cxl_dpa_partition *part = &cxlds->part[i];
+		struct range partition_range = {
+			.start = part->res.start,
+			.end = part->res.end,
+		};
+
+		if (part->mode != CXL_PARTMODE_DYNAMIC_RAM)
+			continue;
+
+		if (range_contains(&partition_range, ext_range))
+			return part;
+	}
+
+	dev_err_ratelimited(dev,
+			    "DC extent DPA %pra (%pU) is not in a valid DC partition\n",
+			    ext_range, extent->uuid);
+	return NULL;
+}
+
 /*
  * Stage 1 of the add pipeline: pure, no allocation.  Resolve the extent
  * to its region/endpoint decoder and ext_range, and verify the range
@@ -133,6 +178,8 @@ static int cxl_validate_extent(struct cxl_memdev_state *mds,
 {
 	u64 start_dpa = le64_to_cpu(extent->start_dpa);
 	struct cxl_memdev *cxlmd = mds->cxlds.cxlmd;
+	struct device *dev = mds->cxlds.dev;
+	const struct cxl_dpa_partition *part;
 	struct cxl_endpoint_decoder *cxled;
 	struct cxl_region *cxlr;
 	struct range ext_range = (struct range) {
@@ -140,6 +187,29 @@ static int cxl_validate_extent(struct cxl_memdev_state *mds,
 		.end = start_dpa + le64_to_cpu(extent->length) - 1,
 	};
 	struct range ed_range;
+	uuid_t uuid;
+
+	import_uuid(&uuid, extent->uuid);
+
+	part = cxl_extent_dc_partition(mds, extent, &ext_range);
+	if (!part)
+		return -ENXIO;
+	dev_dbg(dev, "DC extent DPA %pra (DCR:%pR)(%pU)\n", &ext_range,
+		&part->res, &uuid);
+
+	if (part->shareable) {
+		if (uuid_is_null(&uuid)) {
+			dev_err_ratelimited(dev,
+					    "DC extent DPA %pra: sharable-partition extent has null tag (firmware bug)\n",
+					    &ext_range);
+			return -ENXIO;
+		}
+	} else if (le16_to_cpu(extent->shared_extn_seq)) {
+		dev_err_ratelimited(dev,
+				    "DC extent DPA %pra (%pU): non-sharable partition but shared_extn_seq=%u (firmware bug)\n",
+				    &ext_range, &uuid, le16_to_cpu(extent->shared_extn_seq));
+		return -ENXIO;
+	}
 
 	cxlr = cxl_dpa_to_region(cxlmd, start_dpa, &cxled);
 	if (!cxlr || !cxlr->cxlr_dax)
-- 
2.43.0

[PATCH v1 09/14] cxl/mem: Enforce tag-group semantics

[Anisa Su]

The previous commits added validation for individual extents.  Add
two group-level checks to cxl_validate_group():

  - Sequence integrity (cxl_check_group_seq): sharability is taken from
    the group's DC partition attribute (cxl_group_is_shareable).

    For a sharable partition, extents have a shared_extn_seq number,
    which must be 0..n-1 for a tag. Any gaps or duplicate values are an
    error.

    The field is reserved for non-sharable extents, and
    cxl_validate_extent() rejects a non-zero value there when the group
    is realized.  Therefore, cxl_check_group_seq only applies to
    sharable groups.

  - cxl_check_group_partition: verify tagged allocations
    don't span DC partitions.

Each check drops the whole group on violation.

Tag uniqueness checks land in a subsequent commit.

Based on patches by John Groves.

Co-developed-by: John Groves <John@Groves.net>
Signed-off-by: John Groves <John@Groves.net>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/mbox.c | 138 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 133 insertions(+), 5 deletions(-)

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index a605ffc4c64f..a1d3e04a1e1b 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -1452,6 +1452,119 @@ static void extract_tag_group(struct list_head *pending,
 	}
 }
 
+/*
+ * Validate shared_extn_seq across a tag group from a sharable partition,
+ * already sorted ascending.  Per CXL r4.0 Table 8-230 the device stamps
+ * each extent of an n-extent sharable allocation with a unique value in
+ * 0..n-1, so the sorted group must be exactly 0, 1, ..., n-1.  A gap,
+ * duplicate, or out-of-range value is a device firmware bug.
+ *
+ * Non-sharable extents must carry 0 here, which cxl_validate_extent()
+ * enforces per extent when the group is realized, so only sharable
+ * groups are checked; sharability comes from the partition, not the
+ * seq value.
+ */
+static int cxl_check_group_seq(struct device *dev,
+			       const uuid_t *tag,
+			       const struct list_head *group,
+			       bool shareable)
+{
+	struct cxl_extent_list_node *pos;
+	u16 expected = 0;
+
+	if (!shareable)
+		return 0;
+
+	list_for_each_entry(pos, group, list) {
+		u16 s = le16_to_cpu(pos->extent->shared_extn_seq);
+
+		if (s != expected) {
+			dev_warn_ratelimited(dev,
+					     "Tag %pUb: sharable shared_extn_seq must be dense 0..n-1: expected %u got %u (firmware bug)\n",
+					     tag, expected, s);
+			return -EINVAL;
+		}
+		expected++;
+	}
+	return 0;
+}
+
+/*
+ * A tag group's sharability is a property of the DC partition holding its
+ * extents (cxl_check_group_partition() separately enforces that the group
+ * does not span partitions).  Resolve it from the first extent; an empty
+ * group or an extent outside any DC partition is treated as non-sharable.
+ */
+static bool cxl_group_is_shareable(struct cxl_memdev_state *mds,
+				   const struct list_head *group)
+{
+	const struct cxl_dpa_partition *part;
+	struct cxl_extent_list_node *first;
+	struct cxl_extent *extent;
+	struct range ext_range;
+
+	if (list_empty(group))
+		return false;
+
+	first = list_first_entry(group, struct cxl_extent_list_node, list);
+	extent = first->extent;
+	ext_range = (struct range) {
+		.start = le64_to_cpu(extent->start_dpa),
+		.end = le64_to_cpu(extent->start_dpa) +
+			le64_to_cpu(extent->length) - 1,
+	};
+	part = cxl_extent_dc_partition(mds, extent, &ext_range);
+	return part && part->shareable;
+}
+
+/*
+ * For tagged groups, reject allocations that span DC partitions.  A tag
+ * is an allocation identity; the partition's CDAT DSMAS entry is what
+ * tells the host which attributes (sharable, writable, coherency)
+ * apply.  Untagged groups are skipped — the spec does not define a
+ * cross-chain identity for them.
+ */
+static int cxl_check_group_partition(struct cxl_memdev_state *mds,
+				     const uuid_t *tag,
+				     const struct list_head *group)
+{
+	struct device *dev = mds->cxlds.dev;
+	const struct cxl_dpa_partition *first_part = NULL;
+	u64 first_dpa = 0;
+	struct cxl_extent_list_node *pos;
+
+	if (uuid_is_null(tag) || list_empty(group))
+		return 0;
+
+	list_for_each_entry(pos, group, list) {
+		struct cxl_extent *extent = pos->extent;
+		struct range ext_range = (struct range) {
+			.start = le64_to_cpu(extent->start_dpa),
+			.end = le64_to_cpu(extent->start_dpa) +
+				le64_to_cpu(extent->length) - 1,
+		};
+		const struct cxl_dpa_partition *part;
+
+		part = cxl_extent_dc_partition(mds, extent, &ext_range);
+		if (!part)
+			return -ENXIO;
+
+		if (!first_part) {
+			first_part = part;
+			first_dpa = ext_range.start;
+			continue;
+		}
+
+		if (part != first_part) {
+			dev_warn_ratelimited(dev,
+					     "Tag %pUb: extents span DC partitions (DPA:%#llx and DPA:%#llx), firmware bug\n",
+					     tag, first_dpa, ext_range.start);
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
 /*
  * Realize a tag @group: assign each extent its dax-side @seq_num and add it
  * via cxl_add_extent(), then online the resulting tag group.  Returns the
@@ -1460,7 +1573,7 @@ static void extract_tag_group(struct list_head *pending,
  * the caller to drop.
  */
 static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
-			     struct list_head *group)
+			     struct list_head *group, bool shareable)
 {
 	struct device *dev = mds->cxlds.dev;
 	struct cxl_extent_list_node *pos;
@@ -1475,7 +1588,12 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 	/* Each extent gets a dense dax-side seq_num, 0..n-1 */
 	seq_num = -1;
 	list_for_each_entry(pos, group, list) {
-		seq_num++;
+		/* Sharable: device-assigned dense 0..n-1; otherwise by arrival */
+		if (shareable)
+			seq_num = le16_to_cpu(pos->extent->shared_extn_seq);
+		else
+			seq_num++;
+
 		rc = cxl_add_extent(mds, pos->extent, seq_num);
 		if (rc < 0) {
 			dev_dbg(dev,
@@ -1517,11 +1635,19 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
  * added, or a negative errno if it must be dropped.
  */
 static int cxl_validate_group(struct cxl_memdev_state *mds, const uuid_t *tag,
-			      struct list_head *group)
+			      struct list_head *group, bool shareable)
 {
 	struct device *dev = mds->cxlds.dev;
 	struct cxl_extent_list_node *pos;
 
+	/* Sequence-number integrity */
+	if (cxl_check_group_seq(dev, tag, group, shareable))
+		return -EINVAL;
+
+	/* Partition equality (skipped for null UUID) */
+	if (cxl_check_group_partition(mds, tag, group))
+		return -EINVAL;
+
 	/* Alignment gate — drop the group if any member fails */
 	list_for_each_entry(pos, group, list) {
 		if (!cxl_extent_dcd_aligned(pos->extent)) {
@@ -1559,6 +1685,7 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 	while (!list_empty(pending)) {
 		struct cxl_extent_list_node *first;
 		LIST_HEAD(group);
+		bool shareable;
 		uuid_t tag;
 		int cnt;
 
@@ -1581,13 +1708,14 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 		 * the stable sort maintains arrival order.
 		 */
 		list_sort(NULL, &group, extent_seq_compare);
+		shareable = cxl_group_is_shareable(mds, &group);
 
-		if (cxl_validate_group(mds, &tag, &group)) {
+		if (cxl_validate_group(mds, &tag, &group, shareable)) {
 			drop_extent_group(&group);
 			continue;
 		}
 
-		cnt = cxl_realize_group(mds, &tag, &group);
+		cnt = cxl_realize_group(mds, &tag, &group, shareable);
 		if (cnt < 0) {
 			drop_extent_group(&group);
 			continue;
-- 
2.43.0

[PATCH v1 10/14] cxl/extent: Enforce cross-region tag uniqueness

[Anisa Su]

A tag uuid identifies one allocation across all regions and memdevs,
but nothing checks a new group's tag against the allocations already
live on the host: cxlr_add_extent() only compares it with the chain
group being assembled.  The orchestrator owns tag allocation and is
responsible for global uniqueness, but if a firmware bug or other issue
causes a repeated tag, the kernel should also reject it.

Add a host-wide list of live tag groups with non-null uuids.  Null uuids
are skipped because untagged extents are not associated with a single
logical allocation.

Check the list during tag group validation and return -EEXIST if the tag
is already live on the host via cxl_tag_already_committed().  New tags
are inserted to the list via cxl_tag_register(), which inserts atomically
under cxl_tag_lock.

Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/core.h   |  6 +++++
 drivers/cxl/core/extent.c | 56 +++++++++++++++++++++++++++++++++++++++
 drivers/cxl/core/mbox.c   | 16 +++++++++++
 drivers/cxl/cxl.h         |  3 +++
 4 files changed, 81 insertions(+)

diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index 528eec2a1cc3..b7f8886db498 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -79,6 +79,7 @@ void kill_regions(struct cxl_root_decoder *cxlrd);
 
 int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
 		   u16 seq_num);
+bool cxl_tag_already_committed(const uuid_t *tag);
 int online_tag_group(struct cxl_dc_tag_group *group);
 bool rm_tag_group(struct cxl_dc_tag_group *group);
 void discard_tag_group(struct cxl_dc_tag_group *group);
@@ -117,6 +118,11 @@ static inline int cxl_dax_untagged_add(struct cxl_dax_region *cxlr_dax)
 	return 0;
 }
 
+static inline bool cxl_tag_already_committed(const uuid_t *tag)
+{
+	return false;
+}
+
 static inline
 struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa,
 				     struct cxl_endpoint_decoder **cxled)
diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c
index 334a97ab583d..4a59db823af4 100644
--- a/drivers/cxl/core/extent.c
+++ b/drivers/cxl/core/extent.c
@@ -6,6 +6,54 @@
 
 #include "core.h"
 
+/*
+ * Host-wide list of live tag groups with non-null uuids: a tag uuid
+ * identifies exactly one allocation across all regions and memdevs.
+ * The orchestrator owns tag allocation; this rejects a repeated tag
+ * from a firmware bug or other issue.  Untagged (null uuid) extents are
+ * not tracked: they are not associated with a single allocation.
+ */
+static DEFINE_MUTEX(cxl_tag_lock);
+static LIST_HEAD(cxl_tag_groups);
+
+static int cxl_tag_register(struct cxl_dc_tag_group *grp)
+{
+	struct cxl_dc_tag_group *g;
+
+	if (uuid_is_null(&grp->uuid))
+		return 0;
+
+	guard(mutex)(&cxl_tag_lock);
+	list_for_each_entry(g, &cxl_tag_groups, registry_node)
+		if (uuid_equal(&g->uuid, &grp->uuid))
+			return -EBUSY;
+	list_add_tail(&grp->registry_node, &cxl_tag_groups);
+	return 0;
+}
+
+static void cxl_tag_unregister(struct cxl_dc_tag_group *grp)
+{
+	if (uuid_is_null(&grp->uuid))
+		return;
+
+	guard(mutex)(&cxl_tag_lock);
+	list_del(&grp->registry_node);
+}
+
+bool cxl_tag_already_committed(const uuid_t *tag)
+{
+	struct cxl_dc_tag_group *g;
+
+	if (uuid_is_null(tag))
+		return false;
+
+	guard(mutex)(&cxl_tag_lock);
+	list_for_each_entry(g, &cxl_tag_groups, registry_node)
+		if (uuid_equal(&g->uuid, tag))
+			return true;
+	return false;
+}
+
 static struct cxl_dc_tag_group *to_dc_tag_group(struct device *dev)
 {
 	return container_of(dev, struct cxl_dc_tag_group, dev);
@@ -15,6 +63,7 @@ static void dc_tag_group_release(struct device *dev)
 {
 	struct cxl_dc_tag_group *group = to_dc_tag_group(dev);
 
+	cxl_tag_unregister(group);
 	xa_destroy(&group->dc_extents);
 	/* Drop the pin taken in alloc_tag_group(). */
 	put_device(&group->cxlr_dax->dev);
@@ -93,12 +142,19 @@ static struct cxl_dc_tag_group *
 alloc_tag_group(struct cxl_dax_region *cxlr_dax, uuid_t *uuid)
 {
 	struct cxl_dc_tag_group *group __free(kfree) = kzalloc_obj(*group);
+	int rc;
+
 	if (!group)
 		return ERR_PTR(-ENOMEM);
 
 	group->cxlr_dax = cxlr_dax;
 	uuid_copy(&group->uuid, uuid);
 	xa_init(&group->dc_extents);
+	INIT_LIST_HEAD(&group->registry_node);
+
+	rc = cxl_tag_register(group);
+	if (rc)
+		return ERR_PTR(rc);
 
 	/*
 	 * Pin cxlr_dax: it is used after cxl_rwsem.region is dropped, so a
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index a1d3e04a1e1b..0fa501f5800c 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -1640,6 +1640,22 @@ static int cxl_validate_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 	struct device *dev = mds->cxlds.dev;
 	struct cxl_extent_list_node *pos;
 
+	/*
+	 * Cross-More-chain uniqueness.  A non-null tag seen in this group must
+	 * not already correspond to a committed tag group anywhere on this
+	 * host.  More=0 was supposed to close that allocation, and tag uuids
+	 * must be unique across all regions and memdevs (the orchestrator owns
+	 * assignment per spec).  Either constraint failing — same chain
+	 * redelivered, or two distinct allocations colliding on the same uuid —
+	 * is a firmware/orchestrator bug; reject the whole group.
+	 */
+	if (cxl_tag_already_committed(tag)) {
+		dev_warn_ratelimited(dev,
+				     "Tag %pUb: dropping group, tag already committed (firmware/orchestrator bug)\n",
+				     tag);
+		return -EEXIST;
+	}
+
 	/* Sequence-number integrity */
 	if (cxl_check_group_seq(dev, tag, group, shareable))
 		return -EINVAL;
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index f7d7749d704c..79f5773f8f81 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -642,6 +642,8 @@ struct cxl_dax_region {
  *		allocations.
  * @nr_extents: live count of dc_extents in the group; the group is freed
  *		when the last dc_extent device is released.
+ * @registry_node: anchor in the host-wide non-null-tag registry that
+ *		enforces tag uuid uniqueness across all regions and memdevs.
  */
 struct cxl_dc_tag_group {
 	struct device dev;
@@ -649,6 +651,7 @@ struct cxl_dc_tag_group {
 	uuid_t uuid;
 	struct xarray dc_extents;
 	unsigned int nr_extents;
+	struct list_head registry_node;
 };
 
 bool is_dc_extent(struct device *dev);
-- 
2.43.0

[PATCH v1 11/14] cxl/extent: Handle DC Release Capacity events

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Replace the ack-all stub for Release Capacity chains with the real
teardown.

Similar to add capacity events, Release Capacity records arrive as a
More-chain and are staged in mds->dc_chain until the chain closes;
a chain whose closing record never arrives is dropped by the 20s
watchdog without a response.

A tag group is one logical allocation, so a well-behaved fabric manager
is expected to release all members of a tag group together.  However,
CXL r4.0 Section 8.2.10.9.9.4 permits the host to issue an "unsolicited
release" of extents not corresponding to any event.  So as a generous
policy, the kernel releases all members of a tag group upon receiving a
Release Capacity record containing any member of the group.

Extents are matched to what the host owns by DPA range and tag.  An
extent not matching any host-owned extent is acknowledged for release
so the device knows the host is not using it.  An extent that overlaps
owned capacity without matching any host-owned extent by tag and DPA is
left out of the response so the device keeps the capacity for the host
and must retry.

Extents are not usable until exposed through DAX.  The DAX series that
adds that capability also adds the check that an extent is not in use
before it is released.

Everything a chain releases is acknowledged in the Release Dynamic
Capacity response (opcode 4803h), split across several commands chained
with the More flag when the extent list outgrows the mailbox payload.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: John Groves <John@Groves.net>
Signed-off-by: John Groves <John@Groves.net>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/core.h   |  21 ++++++++
 drivers/cxl/core/extent.c | 103 +++++++++++++++++++++++++++++++++++
 drivers/cxl/core/mbox.c   | 109 ++++++++++++++++++++++++++++++++++++--
 3 files changed, 229 insertions(+), 4 deletions(-)

diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index b7f8886db498..f0edd0100c72 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -79,6 +79,11 @@ void kill_regions(struct cxl_root_decoder *cxlrd);
 
 int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
 		   u16 seq_num);
+int cxl_resolve_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
+		       struct cxl_region **cxlrp,
+		       struct cxl_dc_tag_group **groupp);
+int cxl_release_tag_group(struct cxl_region *cxlr,
+			  struct cxl_dc_tag_group *group);
 bool cxl_tag_already_committed(const uuid_t *tag);
 int online_tag_group(struct cxl_dc_tag_group *group);
 bool rm_tag_group(struct cxl_dc_tag_group *group);
@@ -101,6 +106,22 @@ static inline int cxl_add_extent(struct cxl_memdev_state *mds,
 	return -ENXIO;
 }
 
+static inline int cxl_resolve_extent(struct cxl_memdev_state *mds,
+				     struct cxl_extent *extent,
+				     struct cxl_region **cxlrp,
+				     struct cxl_dc_tag_group **groupp)
+{
+	*cxlrp = NULL;
+	*groupp = NULL;
+	return -ENXIO;
+}
+
+static inline int cxl_release_tag_group(struct cxl_region *cxlr,
+					struct cxl_dc_tag_group *group)
+{
+	return -ENXIO;
+}
+
 static inline int online_tag_group(struct cxl_dc_tag_group *group)
 {
 	return 0;
diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c
index 4a59db823af4..1873a3dfe8e0 100644
--- a/drivers/cxl/core/extent.c
+++ b/drivers/cxl/core/extent.c
@@ -449,6 +449,109 @@ int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
 	return cxlr_add_extent(mds, cxlr_dax, dc_extent);
 }
 
+/**
+ * cxl_resolve_extent() - locate the tag group a released extent belongs to
+ * @mds: memdev state
+ * @extent: extent from a DC Release-Capacity record
+ * @cxlrp: returns the region owning the group
+ * @groupp: returns the matching tag group
+ *
+ * Pure lookup, no side effects.  A record names the dc_extent that contains
+ * its range: the host cannot split an extent.
+ *
+ * Return: 0 when @extent matches an extent the host owns.  -ENXIO when the
+ * extent doesn't match anything on the host (no region, no dax region, or a
+ * range that has no overlap with host-owned extents), so the release can
+ * simply be acknowledged.  -EINVAL when it overlaps owned capacity without
+ * matching any single host-owned extent: it straddles multiple extents, or
+ * the tag doesn't match.
+ */
+int cxl_resolve_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
+		       struct cxl_region **cxlrp,
+		       struct cxl_dc_tag_group **groupp)
+{
+	u64 start_dpa = le64_to_cpu(extent->start_dpa);
+	struct cxl_memdev *cxlmd = mds->cxlds.cxlmd;
+	struct cxl_endpoint_decoder *cxled;
+	struct cxl_dax_region *cxlr_dax;
+	struct dc_extent *dc_extent;
+	struct cxl_region *cxlr;
+	struct range dpa_range;
+	unsigned long idx;
+	uuid_t tag;
+
+	*cxlrp = NULL;
+	*groupp = NULL;
+
+	dpa_range = (struct range) {
+		.start = start_dpa,
+		.end = start_dpa + le64_to_cpu(extent->length) - 1,
+	};
+
+	cxlr = cxl_dpa_to_region(cxlmd, start_dpa, &cxled);
+	if (!cxlr)
+		return -ENXIO;
+
+	cxlr_dax = cxlr->cxlr_dax;
+	if (!cxlr_dax)
+		return -ENXIO;
+
+	import_uuid(&tag, extent->uuid);
+
+	/* RCU keeps entries live during the walk; teardown happens later. */
+	scoped_guard(rcu) {
+		xa_for_each(&cxlr_dax->dc_extents, idx, dc_extent) {
+			if (dc_extent->cxled != cxled)
+				continue;
+			if (!range_overlaps(&dc_extent->dpa_range, &dpa_range))
+				continue;
+			if (!range_contains(&dc_extent->dpa_range, &dpa_range))
+				return -EINVAL;
+			if (!uuid_equal(&dc_extent->group->uuid, &tag))
+				return -EINVAL;
+			*cxlrp = cxlr;
+			*groupp = dc_extent->group;
+			return 0;
+		}
+	}
+	return -ENXIO;
+}
+
+/**
+ * cxl_release_tag_group() - tear a tag group down for a device release
+ * @cxlr: region owning the group
+ * @group: the group to release
+ *
+ * The caller acknowledges the group to the device as a whole; the group is
+ * dropped without a Release DC of its own.
+ *
+ * Return: 0 when the group is gone, negative when the caller must leave it
+ * intact and let the device retry.
+ */
+int cxl_release_tag_group(struct cxl_region *cxlr,
+			  struct cxl_dc_tag_group *group)
+{
+	int rc;
+
+	/*
+	 * Invalidate CPU caches for the region before releasing the capacity
+	 * back to the device so it cannot reassign the range while stale
+	 * cached data lingers.  On failure do not release: leave the tag
+	 * group intact and let the device retry.
+	 */
+	rc = cxl_region_invalidate_memregion(cxlr);
+	if (rc)
+		return rc;
+
+	/*
+	 * An unbinding dax region has already collected the group's devres
+	 * teardown; let that finish the release rather than race it.
+	 */
+	if (!rm_tag_group(group))
+		return -EBUSY;
+	return 0;
+}
+
 /* The group's device may already be visible: readers walk the xarray under RCU */
 static void cleanup_pending_dc_extent(struct dc_extent *dc_extent)
 {
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 0fa501f5800c..c3236fe9e7c5 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -1747,6 +1747,109 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 				    pending, total_accepted);
 }
 
+/*
+ * Tear @group down and queue every member on @acks.  Members are queued
+ * before rm_tag_group() frees them; on failure nothing is queued and the
+ * group stays intact for the device to retry.  Returns the number queued.
+ */
+static int cxl_release_group(struct cxl_region *cxlr,
+			     struct cxl_dc_tag_group *group,
+			     struct list_head *acks)
+{
+	LIST_HEAD(members);
+	int cnt, rc;
+
+	cnt = stage_group_release(group, &members);
+	if (cnt < 0)
+		return cnt;
+
+	rc = cxl_release_tag_group(cxlr, group);
+	if (rc) {
+		drop_extent_group(&members);
+		return rc;
+	}
+
+	list_splice_tail(&members, acks);
+	return cnt;
+}
+
+/*
+ * Flush a staged Release chain.
+ *
+ * A tag group is one logical allocation: naming any extent of a group
+ * releases the whole group, and members the chain did not name go back as
+ * an unsolicited release (CXL r4.0 9.13.3).  Each extent is decided on its
+ * own, so a group that refuses teardown, or an extent that overlaps owned
+ * capacity without naming an owned extent, is held for the device to retry
+ * while the rest proceed.  Extents the host is not tracking at all (no
+ * region, no dax region, or nothing owned in range) hold no capacity here
+ * and are acknowledged as named.
+ *
+ * Everything released is acknowledged together in the Release DC response
+ * once the chain is flushed (split across More-chained commands when the
+ * list outgrows the mailbox payload), so a sharable tag is always returned
+ * whole (CXL r4.0 8.2.10.9.9.4).
+ */
+static int cxl_rm_pending(struct cxl_memdev_state *mds)
+{
+	struct list_head *pending = &mds->dc_chain.pending_extents;
+	struct device *dev = mds->cxlds.dev;
+	struct cxl_extent_list_node *pos, *tmp;
+	LIST_HEAD(acks);
+	int cnt = 0, rc;
+
+	/* cxl_resolve_extent() -> cxl_dpa_to_region() walks every decoder's dpa_res */
+	guard(rwsem_read)(&cxl_rwsem.region);
+	guard(rwsem_read)(&cxl_rwsem.dpa);
+
+	/* Untracked first, before a teardown below makes a sibling look untracked */
+	list_for_each_entry_safe(pos, tmp, pending, list) {
+		struct cxl_dc_tag_group *group;
+		struct cxl_region *cxlr;
+
+		if (cxl_resolve_extent(mds, pos->extent, &cxlr, &group) != -ENXIO)
+			continue;
+		list_move_tail(&pos->list, &acks);
+		cnt++;
+	}
+
+	list_for_each_entry(pos, pending, list) {
+		struct cxl_dc_tag_group *group;
+		struct cxl_region *cxlr;
+		struct range r;
+
+		rc = cxl_resolve_extent(mds, pos->extent, &cxlr, &group);
+		if (rc == -ENXIO)	/* sibling of a group released above */
+			continue;
+		if (rc) {
+			r.start = le64_to_cpu(pos->extent->start_dpa);
+			r.end = r.start + le64_to_cpu(pos->extent->length) - 1;
+			dev_warn_ratelimited(dev,
+					     "DC release %pra (%pUb) names no owned extent; holding for retry\n",
+					     &r, pos->extent->uuid);
+			continue;
+		}
+
+		rc = cxl_release_group(cxlr, group, &acks);
+		if (rc < 0) {
+			dev_dbg(dev, "DC release of tag %pUb deferred: %d\n",
+				&group->uuid, rc);
+			continue;
+		}
+		cnt += rc;
+	}
+
+	/* Updated Extent List Size shall be > 0 (CXL r4.0 Table 8-352) */
+	rc = 0;
+	if (cnt)
+		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC, &acks,
+					  cnt);
+
+	/* The caller frees @pending; hand the acknowledged entries back */
+	list_splice(&acks, pending);
+	return rc;
+}
+
 static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 				  struct cxl_event_dcd *event)
 {
@@ -1793,10 +1896,8 @@ static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 
 	if (event->event_type == DCD_ADD_CAPACITY)
 		rc = cxl_add_pending(mds, &chain->pending_extents);
-	else	/* Nothing is tracked yet: acknowledge every release */
-		rc = cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC,
-					  &chain->pending_extents,
-					  list_count_nodes(&chain->pending_extents));
+	else
+		rc = cxl_rm_pending(mds);
 	clear_pending_extents(mds);
 	return rc;
 }
-- 
2.43.0

[PATCH v1 12/14] cxl/region: Read existing extents on region creation

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

Dynamic capacity device extents may be left in an accepted state on a
device due to an unexpected host crash.  In this case it is expected
that the creation of a new region on top of a DC partition can read
those extents and surface them for continued use.

Once all endpoint decoders are part of a region and the region is being
created, a read of the device's extent list can reveal these
previously accepted extents.

CXL r4.0 Section 8.2.10.9.9.2 specifies the mailbox call Get Dynamic
Capacity Extent List for this purpose.  The call returns all the extents
for all dynamic capacity partitions, so only the extents inside the
region's decoder DPA range are accepted.  The rest belong to other
regions, or to partitions Linux does not map, so are not tracked on the
current region.  If the fabric manager is adding extents to any DCD
partition, the extent list for the recovered region may change; the
generation number changing between queries is detected and the read is
retried.

The dax_region device probes asynchronously, so the extent list is
read from the cxl_dax_region probe, not at region creation.  Onlining
a recovered group attaches its devres teardown to the dax_region
device, and devres present before a probe starts makes the driver
core refuse to bind ("Resources present before probing").

The same asynchrony means the region can detach its decoder before or
while the probe runs, and the scan needs that decoder throughout: its
DPA range filters the extent list, and every recovered extent is
resolved through it back to the region.  So cxl_dax_region_alloc()
records the decoder in cxlr_dax->cxled with a reference, which keeps
it valid until the dax_region is released.  A detach mid-scan then
just makes the remaining extents resolve to no region, and the group
is released back to the device.

New add events for a region are refused until that region's pre-existing
extents have been read, so a tag already in use is never registered twice.
Unbinding the dax driver returns every extent to the device.

Recovered extents are already accepted on the device, so no Add
response follows the scan.  cxl_add_extent() and cxl_add_pending()
take an 'existing' flag for this, and a recovered group that fails
validation or online is released back to the device with Release DC
instead of being left out of a response.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/core.h       |   6 +-
 drivers/cxl/core/extent.c     |  18 ++-
 drivers/cxl/core/mbox.c       | 218 +++++++++++++++++++++++++++++++++-
 drivers/cxl/core/region_dax.c |  47 ++++++++
 drivers/cxl/cxl.h             |  18 +++
 drivers/cxl/cxlmem.h          |  21 ++++
 drivers/dax/cxl.c             |  27 ++++-
 7 files changed, 341 insertions(+), 14 deletions(-)

diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h
index f0edd0100c72..e69167818cb5 100644
--- a/drivers/cxl/core/core.h
+++ b/drivers/cxl/core/core.h
@@ -37,6 +37,7 @@ static inline bool cxled_dcd_enabled(struct cxl_endpoint_decoder *cxled)
 
 const char *cxl_dc_mode_name(bool shareable, bool read_only);
 int cxl_dc_mode_parse(const char *buf, bool *shareable, bool *read_only);
+int cxl_process_extent_list(struct cxl_endpoint_decoder *cxled);
 
 #ifdef CONFIG_CXL_REGION
 
@@ -78,7 +79,7 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr);
 void kill_regions(struct cxl_root_decoder *cxlrd);
 
 int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
-		   u16 seq_num);
+		   u16 seq_num, bool existing);
 int cxl_resolve_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
 		       struct cxl_region **cxlrp,
 		       struct cxl_dc_tag_group **groupp);
@@ -101,7 +102,8 @@ static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr,
 }
 
 static inline int cxl_add_extent(struct cxl_memdev_state *mds,
-				 struct cxl_extent *extent, u16 seq_num)
+				 struct cxl_extent *extent, u16 seq_num,
+				 bool existing)
 {
 	return -ENXIO;
 }
diff --git a/drivers/cxl/core/extent.c b/drivers/cxl/core/extent.c
index 1873a3dfe8e0..2abc00f7f55d 100644
--- a/drivers/cxl/core/extent.c
+++ b/drivers/cxl/core/extent.c
@@ -230,7 +230,7 @@ static int cxl_validate_extent(struct cxl_memdev_state *mds,
 			       struct cxl_extent *extent,
 			       struct cxl_endpoint_decoder **out_cxled,
 			       struct cxl_dax_region **out_cxlr_dax,
-			       struct range *out_ext_range)
+			       struct range *out_ext_range, bool existing)
 {
 	u64 start_dpa = le64_to_cpu(extent->start_dpa);
 	struct cxl_memdev *cxlmd = mds->cxlds.cxlmd;
@@ -271,6 +271,17 @@ static int cxl_validate_extent(struct cxl_memdev_state *mds,
 	if (!cxlr || !cxlr->cxlr_dax)
 		return -ENXIO;
 
+	/*
+	 * New adds wait for a bound dax driver that has read the device's
+	 * pre-existing extents, so a tag already in use is never added twice.
+	 */
+	if (!existing && !smp_load_acquire(&cxlr->cxlr_dax->extents_scanned)) {
+		dev_warn_ratelimited(dev,
+				     "DC extent DPA %pra: dax driver not ready, refusing\n",
+				     &ext_range);
+		return -EBUSY;
+	}
+
 	ed_range = (struct range) {
 		.start = cxled->dpa_res->start,
 		.end = cxled->dpa_res->end,
@@ -406,7 +417,7 @@ static int cxlr_add_extent(struct cxl_memdev_state *mds,
  * Returns 0 for a successfully added extent and <0 on error.
  */
 int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
-		   u16 seq_num)
+		   u16 seq_num, bool existing)
 {
 	struct cxl_endpoint_decoder *cxled;
 	struct cxl_dax_region *cxlr_dax;
@@ -418,7 +429,8 @@ int cxl_add_extent(struct cxl_memdev_state *mds, struct cxl_extent *extent,
 	lockdep_assert_held(&cxl_rwsem.region);
 	lockdep_assert_held(&cxl_rwsem.dpa);
 
-	rc = cxl_validate_extent(mds, extent, &cxled, &cxlr_dax, &ext_range);
+	rc = cxl_validate_extent(mds, extent, &cxled, &cxlr_dax, &ext_range,
+				 existing);
 	if (rc)
 		return rc;
 
diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index c3236fe9e7c5..a18b41ff5d23 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -1452,6 +1452,19 @@ static void extract_tag_group(struct list_head *pending,
 	}
 }
 
+/*
+ * Recovered extents are already the host's: a group the host will not keep
+ * goes back to the device rather than leaking there.
+ */
+static void cxl_release_recovered(struct cxl_memdev_state *mds,
+				  const uuid_t *tag, struct list_head *group)
+{
+	if (cxl_send_dc_response(mds, CXL_MBOX_OP_RELEASE_DC, group,
+				 list_count_nodes(group)))
+		dev_dbg(mds->cxlds.dev,
+			"Tag %pUb: failed to release recovered extents\n", tag);
+}
+
 /*
  * Validate shared_extn_seq across a tag group from a sharable partition,
  * already sorted ascending.  Per CXL r4.0 Table 8-230 the device stamps
@@ -1573,7 +1586,8 @@ static int cxl_check_group_partition(struct cxl_memdev_state *mds,
  * the caller to drop.
  */
 static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
-			     struct list_head *group, bool shareable)
+			     struct list_head *group, bool shareable,
+			     bool existing)
 {
 	struct device *dev = mds->cxlds.dev;
 	struct cxl_extent_list_node *pos;
@@ -1594,7 +1608,7 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 		else
 			seq_num++;
 
-		rc = cxl_add_extent(mds, pos->extent, seq_num);
+		rc = cxl_add_extent(mds, pos->extent, seq_num, existing);
 		if (rc < 0) {
 			dev_dbg(dev,
 				"Tag %pUb: dropping group, failed to add extent DPA:%#llx LEN:%#llx\n",
@@ -1612,7 +1626,7 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 	if (rc) {
 		dev_warn(dev, "Tag %pUb: failed to online tag group (%d)\n",
 			 tag, rc);
-		return rc;
+		goto err_release;
 	}
 
 	return group_cnt;
@@ -1627,6 +1641,10 @@ static int cxl_realize_group(struct cxl_memdev_state *mds, const uuid_t *tag,
 	mds->dc_chain.group = NULL;
 	if (tag_group)
 		discard_tag_group(tag_group);
+err_release:
+	/* A fresh group is simply left out of the Add response */
+	if (existing)
+		cxl_release_recovered(mds, tag, group);
 	return rc;
 }
 
@@ -1686,11 +1704,14 @@ static int cxl_validate_group(struct cxl_memdev_state *mds, const uuid_t *tag,
  * list.
  */
 static int cxl_add_pending(struct cxl_memdev_state *mds,
-			   struct list_head *pending)
+			   struct list_head *pending, bool existing)
 {
 	LIST_HEAD(accepted);
 	int total_accepted = 0;
 
+	/* Realizing a group builds through mds->dc_chain.group */
+	lockdep_assert_held(&mds->dc_chain.lock);
+
 	/*
 	 * Held across validation, add and online so that cxlr_dax_unpublish()
 	 * excludes a whole flush; cxl_dpa_to_region() also walks every dpa_res.
@@ -1727,11 +1748,13 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 		shareable = cxl_group_is_shareable(mds, &group);
 
 		if (cxl_validate_group(mds, &tag, &group, shareable)) {
+			if (existing)
+				cxl_release_recovered(mds, &tag, &group);
 			drop_extent_group(&group);
 			continue;
 		}
 
-		cnt = cxl_realize_group(mds, &tag, &group, shareable);
+		cnt = cxl_realize_group(mds, &tag, &group, shareable, existing);
 		if (cnt < 0) {
 			drop_extent_group(&group);
 			continue;
@@ -1743,6 +1766,14 @@ static int cxl_add_pending(struct cxl_memdev_state *mds,
 
 	list_splice(&accepted, pending);
 
+	/*
+	 * Recovered (already-accepted) extents must not be re-reported in an
+	 * Add-DC-Response: the device rejects a DPA range already added by a
+	 * previous response (CXL r4.0 8.2.10.9.9.3, Invalid Physical Address).
+	 */
+	if (existing)
+		return 0;
+
 	return cxl_send_dc_response(mds, CXL_MBOX_OP_ADD_DC_RESPONSE,
 				    pending, total_accepted);
 }
@@ -1895,7 +1926,8 @@ static int handle_dcd_chain_event(struct cxl_memdev_state *mds,
 	cancel_delayed_work(&chain->timeout_work);
 
 	if (event->event_type == DCD_ADD_CAPACITY)
-		rc = cxl_add_pending(mds, &chain->pending_extents);
+		/* Fresh add events: extents are not yet accepted (not existing). */
+		rc = cxl_add_pending(mds, &chain->pending_extents, false);
 	else
 		rc = cxl_rm_pending(mds);
 	clear_pending_extents(mds);
@@ -2439,6 +2471,180 @@ int cxl_dev_dc_identify(struct cxl_mailbox *mbox,
 }
 EXPORT_SYMBOL_NS_GPL(cxl_dev_dc_identify, "CXL");
 
+/*
+ * Stage the device's extents inside @dpa_range on @existing.  Returns
+ * -EAGAIN if the list changes while it is being read.
+ */
+static int cxl_read_extent_list(struct cxl_memdev_state *mds,
+				const struct range *dpa_range,
+				struct list_head *existing)
+{
+	u32 current_index, total_read, total_expected, initial_gen_num;
+	struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
+	struct device *dev = mds->cxlds.dev;
+	struct cxl_mbox_cmd mbox_cmd;
+	u32 max_extent_count;
+	bool first = true;
+	int rc;
+
+	struct cxl_mbox_get_extent_out *extents __free(kvfree) =
+				kvmalloc(cxl_mbox->payload_size, GFP_KERNEL);
+	if (!extents)
+		return -ENOMEM;
+
+	total_read = 0;
+	current_index = 0;
+	total_expected = 0;
+	max_extent_count = (cxl_mbox->payload_size - sizeof(*extents)) /
+			    sizeof(struct cxl_extent);
+	do {
+		u32 nr_returned, current_total, current_gen_num;
+		struct cxl_mbox_get_extent_in get_extent;
+
+		get_extent = (struct cxl_mbox_get_extent_in) {
+			.extent_cnt = cpu_to_le32(max_extent_count),
+			.start_extent_index = cpu_to_le32(current_index),
+		};
+
+		mbox_cmd = (struct cxl_mbox_cmd) {
+			.opcode = CXL_MBOX_OP_GET_DC_EXTENT_LIST,
+			.payload_in = &get_extent,
+			.size_in = sizeof(get_extent),
+			.size_out = cxl_mbox->payload_size,
+			.payload_out = extents,
+			.min_out = sizeof(*extents),
+		};
+
+		rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd);
+		if (rc < 0)
+			return rc;
+
+		/* Save initial data */
+		if (first) {
+			total_expected = le32_to_cpu(extents->total_extent_count);
+			initial_gen_num = le32_to_cpu(extents->generation_num);
+			first = false;
+		}
+
+		nr_returned = le32_to_cpu(extents->returned_extent_count);
+		if (nr_returned > max_extent_count ||
+		    struct_size(extents, extent, nr_returned) > mbox_cmd.size_out) {
+			dev_err(dev, "Extent list returned %u extents, asked for %u in %zu bytes\n",
+				nr_returned, max_extent_count, mbox_cmd.size_out);
+			return -EIO;
+		}
+		total_read += nr_returned;
+		current_total = le32_to_cpu(extents->total_extent_count);
+		current_gen_num = le32_to_cpu(extents->generation_num);
+
+		dev_dbg(dev, "Got extent list %d-%d of %d generation Num:%d\n",
+			current_index, total_read ? total_read - 1 : 0,
+			current_total, current_gen_num);
+
+		if (current_gen_num != initial_gen_num || total_expected != current_total) {
+			dev_warn(dev, "Extent list change detected; gen %u != %u : cnt %u != %u\n",
+				 current_gen_num, initial_gen_num,
+				 total_expected, current_total);
+			return -EAGAIN;
+		}
+
+		/* No progress with more expected: a buggy device would loop forever. */
+		if (!nr_returned && total_expected > total_read) {
+			dev_warn(dev, "Device returned 0 of %u remaining extents\n",
+				 total_expected - total_read);
+			return -EIO;
+		}
+
+		for (int i = 0; i < nr_returned ; i++) {
+			struct cxl_extent *extent = &extents->extent[i];
+			struct range ext_range;
+
+			ext_range.start = le64_to_cpu(extent->start_dpa);
+			ext_range.end = ext_range.start +
+					le64_to_cpu(extent->length) - 1;
+			dev_dbg(dev, "Processing extent %d/%d %pra (%pUb) seq %u\n",
+				current_index + i, total_expected, &ext_range,
+				extent->uuid,
+				le16_to_cpu(extent->shared_extn_seq));
+			/* Another region's, or an unmapped partition's: not ours */
+			if (!range_contains(dpa_range, &ext_range))
+				continue;
+			rc = stage_pending_extent(existing, extent);
+			if (rc)
+				return rc;
+		}
+
+		current_index += nr_returned;
+	} while (total_expected > total_read);
+
+	return 0;
+}
+
+static int __cxl_process_extent_list(struct cxl_endpoint_decoder *cxled)
+{
+	struct cxl_memdev_state *mds = cxled_to_mds(cxled);
+	struct range dpa_range;
+	LIST_HEAD(existing);
+	int rc;
+
+	/*
+	 * The list covers every DC partition and region of the device; only
+	 * the extents inside this decoder's DPA range are this region's.
+	 */
+	scoped_guard(rwsem_read, &cxl_rwsem.dpa) {
+		if (!cxled->dpa_res)
+			return -ENXIO;
+		dpa_range = (struct range) {
+			.start = cxled->dpa_res->start,
+			.end = cxled->dpa_res->end,
+		};
+	}
+
+	/*
+	 * Realizing a group goes through mds->dc_chain.group, shared with the
+	 * event path, so hold its lock.  Stage into a private list so an event
+	 * chain the device left open is not consumed as existing extents.
+	 */
+	guard(mutex)(&mds->dc_chain.lock);
+
+	rc = cxl_read_extent_list(mds, &dpa_range, &existing);
+
+	/*
+	 * Already accepted on the device: no Add response is sent for existing
+	 * extents, and one that fails validation or online is released back to
+	 * the device rather than dropped.
+	 */
+	if (!rc && !list_empty(&existing))
+		rc = cxl_add_pending(mds, &existing, true);
+	drop_extent_group(&existing);
+
+	return rc;
+}
+
+#define CXL_READ_EXTENT_LIST_RETRY 10
+
+/**
+ * cxl_process_extent_list() - Read existing extents
+ * @cxled: Endpoint decoder which is part of a region
+ *
+ * Issue the Get Dynamic Capacity Extent List command to the device
+ * and add existing extents if found.
+ *
+ * A retry of 10 is somewhat arbitrary, however, extent changes should be
+ * relatively rare while bringing up a region.  So 10 should be plenty.
+ */
+int cxl_process_extent_list(struct cxl_endpoint_decoder *cxled)
+{
+	int retry = CXL_READ_EXTENT_LIST_RETRY;
+	int rc;
+
+	do {
+		rc = __cxl_process_extent_list(cxled);
+	} while (rc == -EAGAIN && --retry);
+
+	return rc;
+}
+
 int cxl_get_dirty_count(struct cxl_memdev_state *mds, u32 *count)
 {
 	struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox;
diff --git a/drivers/cxl/core/region_dax.c b/drivers/cxl/core/region_dax.c
index 790176a8aadd..816386301b80 100644
--- a/drivers/cxl/core/region_dax.c
+++ b/drivers/cxl/core/region_dax.c
@@ -13,6 +13,8 @@ static void cxl_dax_region_release(struct device *dev)
 {
 	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
 
+	if (cxlr_dax->cxled)
+		put_device(&cxlr_dax->cxled->cxld.dev);
 	xa_destroy(&cxlr_dax->dc_extents);
 	ida_destroy(&cxlr_dax->tag_group_ida);
 	kfree(cxlr_dax);
@@ -79,6 +81,15 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr)
 	cxlr_dax->hpa_range.start = p->res->start;
 	cxlr_dax->hpa_range.end = p->res->end;
 
+	/*
+	 * DC regions are not interleaved; the probe-time scan uses this decoder
+	 * and may outlive its attachment to the region, so hold a reference.
+	 */
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM)
+		cxlr_dax->cxled = p->targets[0];
+	if (cxlr_dax->cxled)
+		get_device(&cxlr_dax->cxled->cxld.dev);
+
 	dev = &cxlr_dax->dev;
 	cxlr_dax->cxlr = cxlr;
 	cxlr->cxlr_dax = cxlr_dax;
@@ -153,6 +164,42 @@ DEFINE_FREE(unpublish_cxl_dax_region, struct cxl_dax_region *,
 		put_device(&_T->dev);
 	})
 
+/*
+ * Runs from cxl_dax_region_probe().  The region's parameters may not be
+ * read here: a concurrent detach clears p->targets[] under cxl_rwsem.region
+ * before it waits for this probe.  Use the decoder recorded at allocation
+ * instead; cxlr_dax holds a reference on it.
+ */
+int cxl_region_add_existing_extents(struct cxl_dax_region *cxlr_dax)
+{
+	struct cxl_endpoint_decoder *cxled = cxlr_dax->cxled;
+	int rc;
+
+	rc = cxl_process_extent_list(cxled);
+	if (rc) {
+		dev_err(&cxled->cxld.dev, "Existing extent processing failed %d\n",
+			rc);
+		return rc;
+	}
+
+	/* Pairs with the acquire in cxl_validate_extent(); adds may proceed */
+	smp_store_release(&cxlr_dax->extents_scanned, true);
+	return 0;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_region_add_existing_extents, "CXL");
+
+/*
+ * Driver unbind returns the held extents to the device (devres), so refuse
+ * new adds until the next probe has read the device's list again.
+ */
+void cxl_region_reset_extent_scan(struct cxl_dax_region *cxlr_dax)
+{
+	/* Adds test the gate under cxl_rwsem.region; none is mid-flight after this */
+	guard(rwsem_write)(&cxl_rwsem.region);
+	cxlr_dax->extents_scanned = false;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_region_reset_extent_scan, "CXL");
+
 int devm_cxl_add_dax_region(struct cxl_region *cxlr)
 {
 	struct device *dev;
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 79f5773f8f81..cf109f119bc4 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -619,6 +619,10 @@ struct cxl_dax_region {
 	struct ida tag_group_ida;
 	/* parent of the extentX.Y devices of untagged allocations */
 	struct device untagged;
+	/* The region's single endpoint decoder, for the probe-time extent scan */
+	struct cxl_endpoint_decoder *cxled;
+	/* Set once the probe has read the device's pre-existing extents. */
+	bool extents_scanned;
 };
 
 /**
@@ -1006,6 +1010,8 @@ bool is_cxl_pmem_region(struct device *dev);
 struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev);
 int cxl_add_to_region(struct cxl_endpoint_decoder *cxled);
 struct cxl_dax_region *to_cxl_dax_region(struct device *dev);
+int cxl_region_add_existing_extents(struct cxl_dax_region *cxlr_dax);
+void cxl_region_reset_extent_scan(struct cxl_dax_region *cxlr_dax);
 bool cxl_region_contains_resource(const struct resource *res);
 #else
 static inline bool is_cxl_pmem_region(struct device *dev)
@@ -1024,6 +1030,18 @@ static inline struct cxl_dax_region *to_cxl_dax_region(struct device *dev)
 {
 	return NULL;
 }
+
+static inline int
+cxl_region_add_existing_extents(struct cxl_dax_region *cxlr_dax)
+{
+	return 0;
+}
+
+static inline void
+cxl_region_reset_extent_scan(struct cxl_dax_region *cxlr_dax)
+{
+}
+
 static inline bool cxl_region_contains_resource(const struct resource *res)
 {
 	return false;
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 73f01efc0172..3acd4e0e1725 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -592,6 +592,27 @@ struct cxl_mbox_dc_response {
 	} __packed extent_list[] __counted_by_le(extent_list_size);
 } __packed;
 
+/*
+ * Get Dynamic Capacity Extent List; Input Payload
+ * CXL r4.0 section 8.2.10.9.9.2; Table 8-348
+ */
+struct cxl_mbox_get_extent_in {
+	__le32 extent_cnt;
+	__le32 start_extent_index;
+} __packed;
+
+/*
+ * Get Dynamic Capacity Extent List; Output Payload
+ * CXL r4.0 section 8.2.10.9.9.2; Table 8-349
+ */
+struct cxl_mbox_get_extent_out {
+	__le32 returned_extent_count;
+	__le32 total_extent_count;
+	__le32 generation_num;
+	u8 rsvd[4];
+	struct cxl_extent extent[];
+} __packed;
+
 struct cxl_mbox_get_supported_logs {
 	__le16 entries;
 	u8 rsvd[6];
diff --git a/drivers/dax/cxl.c b/drivers/dax/cxl.c
index 43eb18c8a035..7cc512e4b3c6 100644
--- a/drivers/dax/cxl.c
+++ b/drivers/dax/cxl.c
@@ -6,6 +6,12 @@
 #include "../cxl/cxl.h"
 #include "bus.h"
 
+static void cxl_dax_region_remove(struct device *dev)
+{
+	/* Refuse new adds; devres returns the held extents */
+	cxl_region_reset_extent_scan(to_cxl_dax_region(dev));
+}
+
 static int cxl_dax_region_probe(struct device *dev)
 {
 	struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev);
@@ -15,6 +21,7 @@ static int cxl_dax_region_probe(struct device *dev)
 	struct dev_dax_data data;
 	resource_size_t dev_size;
 	unsigned long flags;
+	int rc;
 
 	if (nid == NUMA_NO_NODE)
 		nid = memory_add_physaddr_to_nid(cxlr_dax->hpa_range.start);
@@ -29,11 +36,17 @@ static int cxl_dax_region_probe(struct device *dev)
 	if (!dax_region)
 		return -ENOMEM;
 
-	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM)
+	if (cxlr->mode == CXL_PARTMODE_DYNAMIC_RAM) {
+		/* Recover extents the device still holds for this region */
+		rc = cxl_region_add_existing_extents(cxlr_dax);
+		if (rc)
+			goto err_remove;
+
 		/* Add empty seed dax device */
 		dev_size = 0;
-	else
+	} else {
 		dev_size = range_len(&cxlr_dax->hpa_range);
+	}
 
 	data = (struct dev_dax_data) {
 		.dax_region = dax_region,
@@ -42,12 +55,20 @@ static int cxl_dax_region_probe(struct device *dev)
 		.memmap_on_memory = true,
 	};
 
-	return PTR_ERR_OR_ZERO(devm_create_dev_dax(&data));
+	rc = PTR_ERR_OR_ZERO(devm_create_dev_dax(&data));
+	if (rc)
+		goto err_remove;
+	return 0;
+
+err_remove:
+	cxl_dax_region_remove(dev);
+	return rc;
 }
 
 static struct cxl_driver cxl_dax_region_driver = {
 	.name = "cxl_dax_region",
 	.probe = cxl_dax_region_probe,
+	.remove = cxl_dax_region_remove,
 	.id = CXL_DEVICE_DAX_REGION,
 	.drv = {
 		.suppress_bind_attrs = true,
-- 
2.43.0

[PATCH v1 13/14] cxl/mem: Trace Dynamic capacity Event Record

[Anisa Su]

From: Ira Weiny <iweiny@kernel.org>

CXL r4.0 section 8.2.10.2.1.6 defines the Dynamic Capacity Event Record.
User space can use trace events for debugging of DC capacity changes.

Add DC trace points to the trace log.

Based on an original patch by Navneet Singh.

Signed-off-by: Ira Weiny <iweiny@kernel.org>
Co-developed-by: Anisa Su <anisa.su@samsung.com>
Signed-off-by: Anisa Su <anisa.su@samsung.com>
Tested-by: Wonjae Lee <wj28.lee@samsung.com>
Tested-by: Junhee Park <jh9934.park@samsung.com>
Tested-by: Heesoo Kim <habil.kim@samsung.com>
---
 drivers/cxl/core/mbox.c  |  4 +++
 drivers/cxl/core/trace.h | 76 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 80 insertions(+)

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index a18b41ff5d23..c60c10d7bd31 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -1041,6 +1041,10 @@ static void __cxl_event_trace_record(struct cxl_memdev *cxlmd,
 		ev_type = CXL_CPER_EVENT_MEM_MODULE;
 	else if (uuid_equal(uuid, &CXL_EVENT_MEM_SPARING_UUID))
 		ev_type = CXL_CPER_EVENT_MEM_SPARING;
+	else if (uuid_equal(uuid, &CXL_EVENT_DC_EVENT_UUID)) {
+		trace_cxl_dynamic_capacity(cxlmd, type, &record->event.dcd);
+		return;
+	}
 
 	cxl_event_trace_record(cxlmd, type, ev_type, uuid, &record->event);
 }
diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h
index 64c6ce0878e2..d7b237203bb5 100644
--- a/drivers/cxl/core/trace.h
+++ b/drivers/cxl/core/trace.h
@@ -1108,6 +1108,82 @@ TRACE_EVENT(cxl_poison,
 	)
 );
 
+/*
+ * Dynamic Capacity Event Record - DER
+ *
+ * CXL r4.0 section 8.2.10.2.1.6 Table 8-229
+ */
+
+TRACE_DEFINE_ENUM(DCD_ADD_CAPACITY);
+TRACE_DEFINE_ENUM(DCD_RELEASE_CAPACITY);
+TRACE_DEFINE_ENUM(DCD_FORCED_CAPACITY_RELEASE);
+TRACE_DEFINE_ENUM(DCD_REGION_CONFIGURATION_UPDATED);
+
+#define show_dc_evt_type(type)	__print_symbolic(type,				\
+	{ DCD_ADD_CAPACITY,			"Add capacity" },		\
+	{ DCD_RELEASE_CAPACITY,			"Release capacity" },		\
+	{ DCD_FORCED_CAPACITY_RELEASE,		"Forced capacity release" },	\
+	{ DCD_REGION_CONFIGURATION_UPDATED,	"Region Configuration Updated" } \
+)
+
+TRACE_EVENT(cxl_dynamic_capacity,
+
+	TP_PROTO(const struct cxl_memdev *cxlmd, enum cxl_event_log_type log,
+		 struct cxl_event_dcd *rec),
+
+	TP_ARGS(cxlmd, log, rec),
+
+	TP_STRUCT__entry(
+		CXL_EVT_TP_entry
+
+		/* Dynamic capacity Event */
+		__field(u8, event_type)
+		__field(u16, hostid)
+		__field(u8, updated_region_index)
+		__field(u64, dpa_start)
+		__field(u64, length)
+		__array(u8, uuid, UUID_SIZE)
+		__field(u16, sh_extent_seq)
+	),
+
+	TP_fast_assign(
+		CXL_EVT_TP_fast_assign(cxlmd, log, rec->hdr);
+		__entry->hdr_uuid = CXL_EVENT_DC_EVENT_UUID;
+
+		/* Dynamic_capacity Event */
+		__entry->event_type = rec->event_type;
+
+		/* DCD event record data */
+		__entry->hostid = le16_to_cpu(rec->host_id);
+		/*
+		 * The Updated Region Index is only defined for Region
+		 * Configuration Updated events (Table 8-229); report U8_MAX
+		 * (not a valid index) for other event types where the field
+		 * is reserved.
+		 */
+		if (rec->event_type == DCD_REGION_CONFIGURATION_UPDATED)
+			__entry->updated_region_index = rec->updated_region_index;
+		else
+			__entry->updated_region_index = U8_MAX;
+		__entry->dpa_start = le64_to_cpu(rec->extent.start_dpa);
+		__entry->length = le64_to_cpu(rec->extent.length);
+		memcpy(__entry->uuid, &rec->extent.uuid, UUID_SIZE);
+		__entry->sh_extent_seq = le16_to_cpu(rec->extent.shared_extn_seq);
+	),
+
+	CXL_EVT_TP_printk("event_type='%s' host_id='%d' updated_region_index='%d' " \
+		"starting_dpa=%llx length=%llx tag=%pU " \
+		"shared_extent_sequence=%d",
+		show_dc_evt_type(__entry->event_type),
+		__entry->hostid,
+		__entry->updated_region_index,
+		__entry->dpa_start,
+		__entry->length,
+		__entry->uuid,
+		__entry->sh_extent_seq
+	)
+);
+
 #endif /* _CXL_EVENTS_H */
 
 #define TRACE_INCLUDE_FILE trace
-- 
2.43.0

[PATCH v1 14/14] Documentation/cxl: Document DCD extent handling

[Anisa Su]

Add a "Dynamic Capacity Extents" section to cxl-driver.rst, describing
when CXL core accepts an offered extent and when a release request
is acknowledged.

Update maturity-map.rst to reflect initial support for "Extent add /
release event handling".  Overall DCD Support stays at [0] until the
DAX layer plumbing lands and extents become usable.

Signed-off-by: Anisa Su <anisa.su@samsung.com>
---
 .../driver-api/cxl/linux/cxl-driver.rst       | 181 ++++++++++++++++++
 Documentation/driver-api/cxl/maturity-map.rst |   2 +-
 2 files changed, 182 insertions(+), 1 deletion(-)

diff --git a/Documentation/driver-api/cxl/linux/cxl-driver.rst b/Documentation/driver-api/cxl/linux/cxl-driver.rst
index 212df4fb8d63..28f6c0530c8a 100644
--- a/Documentation/driver-api/cxl/linux/cxl-driver.rst
+++ b/Documentation/driver-api/cxl/linux/cxl-driver.rst
@@ -654,6 +654,187 @@ from HPA to DPA.  This is why they must be aware of the entire interleave set.
 Linux does not support unbalanced interleave configurations.  As a result, all
 endpoints in an interleave set must have the same ways and granularity.
 
+Dynamic Capacity Extents
+========================
+
+A `Dynamic Capacity Device (DCD)` advertises capacity in `DC Regions`
+and surfaces individual chunks of that capacity to the host as
+`extents` (CXL r4.0 Section 9.13.3 "Dynamic Capacity Device (DCD)").
+Linux calls a DC Region a `DC partition` (see "DPA Partitions" above)
+so that "region" keeps its software meaning: a :code:`cxl_region` is
+the host-side object that maps DPA into a host physical address range,
+whereas a DC Region is a device-defined slice of DPA space.
+
+Accepting Extents
+-----------------
+Extents are made available to the host through DC Add events: the
+device sends one or more Dynamic Capacity Event Records (CXL r4.0
+Section 8.2.10.2.1.6, Table 8-229) to the host.  Each record carries
+one extent, which the host may accept or reject.  A device offering
+several extents sends one record per extent, chained with `More`: each
+record has the `More` flag set to announce another incoming record,
+until the last record, which carries `More=0` to mark the end of the
+chain.
+
+For each record received, the host stages the offered extent on a
+pending list.  When the chain closes the host knows that every extent
+of this add event has arrived, and the whole pending list is validated
+before anything is accepted.  A chain whose closing record does not
+arrive within 20 seconds is refused as a whole: the watchdog answers
+with an empty Add Dynamic Capacity Response so the device can reclaim
+the offer, and the staged extents are discarded.
+
+From the pending list, extents that share a (non-null) tag are
+extracted into a tag group so that each group can be validated as a
+unit; a group that is accepted becomes a :code:`struct
+cxl_dc_tag_group`.  A tag group is one logical allocation.  An untagged
+extent is an allocation of its own and becomes a single-member group.
+
+Group-level validation (:code:`cxl_validate_group()`,
+:code:`drivers/cxl/core/mbox.c`) drops the whole group if any check
+fails:
+
+* `Host-wide tag uniqueness`: a non-null tag must not already
+  correspond to a live :code:`cxl_dc_tag_group` anywhere on this host.
+  The orchestrator / Fabric Manager owns tag assignment (CXL r4.0
+  Section 9.13.3.1 "DCD Management by FM" and Section 7.6.7.6 "DCD
+  Management Command Set for LD-FAM"); the list in
+  :code:`drivers/cxl/core/extent.c` (:code:`cxl_tag_register()` /
+  :code:`cxl_tag_already_committed()`) catches firmware bugs and
+  orchestrator misbehavior across every region and memdev.  Skipped
+  for the null UUID because there may be multiple untagged extents.
+* `Sequence-number integrity`: extents may be shared across hosts if
+  the device supports memory sharing.  Since a tag group is one
+  logical allocation, every host needs to see the extents of a shared
+  group in the same order; that order is the Shared Extent Sequence
+  Number of each extent (:code:`shared_extn_seq`, Table 8-230).  A
+  sharable group's sequence numbers must form exactly
+  :code:`0, 1, ..., n-1`: sequences with gaps or duplicates, or that
+  do not start at 0, are rejected.  A non-sharable extent must carry
+  0 in the field; that is rejected per extent by
+  :code:`cxl_validate_extent()`, so the group check covers only
+  sharable partitions and the host orders non-sharable extents by
+  arrival.
+* `Partition equality`: every tagged extent in the group must resolve
+  to the same DC partition.  A single allocation cannot span
+  partitions because CDAT describes sharable / writable / coherency
+  attributes per partition.  Skipped for the null UUID.
+* `Alignment`: every extent's :code:`start_dpa` and :code:`length`
+  must be :code:`PMD_SIZE`-aligned, the alignment the DAX region is
+  created with.  If any extent is misaligned the whole group is
+  dropped: accepting only the aligned subset would leave an unusable
+  allocation.
+
+After group-level validation each extent is validated individually
+(:code:`cxl_add_extent()`, :code:`drivers/cxl/core/extent.c`).  A tag
+group is accepted whole or not at all: for sharable capacity the spec
+requires it (Section 8.2.10.9.9.3, otherwise the device returns Invalid
+Extent List), and an allocation missing an extent would be unusable
+either way.  So an extent that fails any of these checks drops its
+whole group:
+
+* The extent's DPA range must lie within a DC partition of the memdev
+  (:code:`cxl_extent_dc_partition()`).  On a sharable partition the
+  extent must carry a tag; on a non-sharable one its shared extent
+  sequence must be 0.
+* The extent's DPA range must resolve to a CXL region via
+  :code:`cxl_dpa_to_region()`.  An extent with no owning region is
+  dropped, as is one whose :code:`dax_region` has not finished its
+  probe-time scan or has no driver bound (see Recovery below).
+* The extent's DPA range must be fully contained in the endpoint
+  decoder's DPA range.  An extent that straddles the decoder boundary
+  is dropped; the driver never clips an extent to fit.
+* The extent must not overlap or repeat an extent already present in
+  the same region (:code:`cxlr_dax_classify_extent()`).  A range the
+  host already accepted drops its group: the device rejects a DPA
+  range reported twice in an Add response (CXL r4.0 8.2.10.9.9.3,
+  Invalid Physical Address).
+
+Surviving tag groups are registered with the device model by
+:code:`online_tag_group()`: a tagged group becomes a
+:code:`tag_groupX.Y` child of the :code:`dax_region` device and every
+member :code:`dc_extent` an :code:`extentX.Y` child of it, while an
+untagged extent goes under the region's :code:`untagged` device.  The
+host then tells the device which extents it accepted with an Add
+Dynamic Capacity Response (CXL r4.0 Section 8.2.10.9.9.3, opcode 4802h)
+per chain, split across several commands chained with the More flag
+when the extent list outgrows the mailbox payload; an extent left out
+of the response is rejected.
+
+Extents the device already reserves for a region, for instance after a
+host crash, are recovered when the region's :code:`dax_region` probes:
+:code:`cxl_region_add_existing_extents()` reads the device's extent
+list (Get Dynamic Capacity Extent List, Section 8.2.10.9.9.2, opcode
+4801h), keeps the extents inside the region's decoder (the list covers
+every DC partition of the device) and runs them through the same
+validation with :code:`existing` set.  No Add Dynamic Capacity Response
+is sent for them, and a recovered group that fails validation or does
+not online is released back to the device.  Add events for the region
+are refused until that scan completes, and again once the driver
+unbinds: unbinding returns every held extent to the device, so the next
+probe scans afresh.
+
+Releasing Extents
+-----------------
+
+A release may be initiated by the device (a Release Capacity event) or
+by the host, for example when tearing down a region.  A host may also
+issue a Release Dynamic Capacity Response not associated with any
+event, an "unsolicited release", at any time (CXL r4.0 Section 9.13.3).
+The host-initiated path runs :code:`dc_tag_group_teardown()` for each
+tag group of the region: it invalidates the region's cached lines,
+returns the group with its own Release Dynamic Capacity Response and
+unregisters its extents.  The device-initiated path goes through the
+release chain in :code:`drivers/cxl/core/mbox.c`.
+
+A device-initiated release may span several event records chained with
+the More flag, exactly like an add event.  Each record is staged in
+:code:`mds->dc_chain` by :code:`handle_dcd_chain_event()`, the same
+chain an add event uses, until the chain closes (More flag cleared).
+Chained records shall be contiguous and share one event type (CXL r4.0
+Table 8-229), so add and release records cannot coincide; should a
+record of the other type arrive inside an open chain, the open chain is
+aborted (an add chain with an empty Add Dynamic Capacity Response, a
+release chain silently) and the record starts a new chain.  A chain
+whose closing record never arrives is dropped by the 20 second watchdog
+the same way, so the capacity stays provisioned for the host and the
+device may retry.
+
+When the chain closes, :code:`cxl_rm_pending()` processes the extents
+staged in the pending list.  As mentioned above, a tag group is one
+logical allocation, so requesting the release of any extent in a group
+releases the whole group.  Members of the group missing from the event
+record chain (an FM mistake or firmware bug) are still included in the
+Release Dynamic Capacity Response as a "generous" policy.
+
+Each extent pending release is processed:
+
+* The extent is resolved by :code:`cxl_resolve_extent()` to its CXL
+  region and :code:`cxl_dc_tag_group`.
+* An extent the host does not own is acknowledged so the device knows
+  the host is not using it: its DPA range resolves to no region, to a
+  region whose :code:`cxlr_dax` is not set up, or to a range that
+  overlaps nothing the host owns (for example, an extent left over
+  from a host crash that has not yet been re-claimed, or a duplicate
+  release racing region teardown).
+* An extent that overlaps capacity the host owns without matching any
+  single accepted extent (it straddles two extents, or the tag
+  disagrees) is held: it is left out of the response so the device
+  retains the capacity for the host and may retry.
+* An extent that names an owned extent releases its group through
+  :code:`cxl_release_tag_group()`: the backing memregion is
+  invalidated once for the group and :code:`rm_tag_group()`
+  unregisters every member :code:`dc_extent` device.  If the
+  invalidation fails, or the :code:`dax_region`'s unbind already owns
+  the group's teardown, the group is left out of the response, so the
+  device retains the capacity and may retry.
+
+After processing the pending list, every extent the host is giving
+back is sent to the device in the Release Dynamic Capacity Response
+(Section 8.2.10.9.9.4, opcode 4803h), mirroring the Add response.  A
+list that does not fit in one mailbox payload is sent as several
+commands chained with the More flag.
+
 Example Configurations
 ======================
 .. toctree::
diff --git a/Documentation/driver-api/cxl/maturity-map.rst b/Documentation/driver-api/cxl/maturity-map.rst
index fefd31899229..13df87038c88 100644
--- a/Documentation/driver-api/cxl/maturity-map.rst
+++ b/Documentation/driver-api/cxl/maturity-map.rst
@@ -151,7 +151,7 @@ Memory-pooling
 * [0] Dynamic Capacity Device (DCD) Support
 
   * [1] DC partition enumeration / configuration (Get DC Config, CDAT DSMAS, event interrupts)
-  * [0] Extent add / release event handling
+  * [1] Extent add / release event handling
   * [0] DC-backed DAX regions
 
 Multi-host sharing
-- 
2.43.0