* [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem [not found] <CGME20260123113121epcas5p125bc64b4714525d7bbd489cd9be9ba91@epcas5p1.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar [not found] ` <CGME20260123113123epcas5p1cd55b845a358fccd305e655664b4f042@epcas5p1.samsung.com> ` (20 more replies) 0 siblings, 21 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Introduction: ============= CXL Persistent Memory (Pmem) devices region, namespace and content must be persistent across system reboot. In order to achieve this persistency, it uses Label Storage Area (LSA) to store respective metadata. During system reboot, stored metadata in LSA is used to bring back the region, namespace and content of CXL device in its previous state. CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox commands to access the LSA area. nvdimm driver is using same commands to get/set LSA data. There are three types of LSA format and these are part of following specifications: - v1.1: https://pmem.io/documents/NVDIMM_Namespace_Spec.pdf - v1.2: https://uefi.org/sites/default/files/resources/UEFI_Spec_2_7.pdf - v2.1: https://computeexpresslink.org/wp-content/uploads/2024/02/CXL-2.0-Specification.pdf Basic differences between these LSA formats: - v1.1: Support Namespace persistency. Size of Namespace Label is 128 bytes - v1.2: Support Namespace persistency. Size of Namespace Label is 256 bytes - v2.1: Support Namespace and Region persistency. Size of Namespace and Region Label is 256 bytes. Linux nvdimm driver supports only v1.1 and v1.2 LSA format. CXL pmem device require support of LSA v2.1 format for region and namespace persistency. Initial support of LSA 2.1 was add in [1]. This patchset adds support of LSA 2.1 in nvdimm and cxl pmem driver. Patch 1: Introduce NDD_CXL_LABEL flag and Update cxl label index as per v2.1 Patch 2: Skip the need for 'interleave-set cookie' for LSA 2.1 support Patch 3-9: Introduce region label and update namespace label as per LSA 2.1 Patch 10: Refactor cxl pmem region auto assembly using Dan's Infra Patch 11-13: Save cxl region info in LSA and region recreation during reboot Patch 14:15: Segregate out cxl pmem region code from region.c to pmem_region.c Patch 16: Introduce cxl region addition/deletion attributes Patch 17-18: Add support of cxl pmem region re-creation from CXL as per LSA 2.1 Testing: ======== In order to test this patchset, I also added the support of LSA v2.1 format in ndctl. ndctl changes are available at [2]. After review, I’ll push in ndctl repo for community review. 1. Used Qemu using following CXL topology M2="-object memory-backend-file,id=cxl-mem11,share=on,mem-path=$TMP_DIR/cxltest11.raw,size=4G \ -object memory-backend-file,id=cxl-lsa11,share=on,mem-path=$TMP_DIR/lsa.raw11,size=1M \ -object memory-backend-file,id=cxl-mem21,share=on,mem-path=$TMP_DIR/cxltest21.raw,size=4G \ -object memory-backend-file,id=cxl-lsa21,share=on,mem-path=$TMP_DIR/lsa21.raw,size=1M \ -device pxb-cxl,bus=pcie.0,bus_nr=10,id=cxl.1 \ -device cxl-rp,port=110,bus=cxl.1,id=root_port11,chassis=0,slot=11 \ -device cxl-type3,bus=root_port11,memdev=cxl-mem11,lsa=cxl-lsa11,id=cxl-pmem11,sn=11 \ -device pxb-cxl,bus=pcie.0,bus_nr=20,id=cxl.2 \ -device cxl-rp,port=210,bus=cxl.2,id=root_port21,chassis=0,slot=21 \ -device cxl-type3,bus=root_port21,memdev=cxl-mem21,lsa=cxl-lsa21,id=cxl-pmem21,sn=21 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=8G,cxl-fmw.0.interleave-granularity=8k,cxl-fmw.1.targets.0=cxl.2,cxl-fmw.1.size=8G" 2. Create cxl region on both devices cxl create-region -d decoder0.1 -m mem0 cxl create-region -d decoder0.0 -m mem1 root@QEMUCXL6060pmem:~# cxl list [ { "memdevs":[ { "memdev":"mem0", "pmem_size":4294967296, "serial":21, "host":"0000:15:00.0", "firmware_version":"BWFW VERSION 00" }, { "memdev":"mem1", "pmem_size":4294967296, "serial":11, "host":"0000:0b:00.0", "firmware_version":"BWFW VERSION 00" } ] }, { "regions":[ { "region":"region0", "resource":45365592064, "size":4294967296, "type":"pmem", "interleave_ways":1, "interleave_granularity":256, "decode_state":"commit", "qos_class_mismatch":true }, { "region":"region1", "resource":53955526656, "size":4294967296, "type":"pmem", "interleave_ways":1, "interleave_granularity":256, "decode_state":"commit", "qos_class_mismatch":true } ] } ] 3. Re-Start Qemu and we could see cxl region persistency using "cxl list" 4. Create namespace for both regions root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=256M { "dev":"namespace0.1", "mode":"fsdax", "map":"dev", "size":"250.00 MiB (262.14 MB)", "uuid":"5f9e28b4-f403-4e72-bb06-307ead53dec4", "sector_size":512, "align":2097152, "blockdev":"pmem0.1" } real 1m8.795s user 0m0.098s sys 0m48.699s root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=128M { "dev":"namespace1.1", "mode":"fsdax", "map":"dev", "size":"124.00 MiB (130.02 MB)", "uuid":"33682096-7364-412d-bfbc-fa7568939abd", "sector_size":512, "align":2097152, "blockdev":"pmem1.1" } root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=128M { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":"124.00 MiB (130.02 MB)", "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", "sector_size":512, "align":2097152, "blockdev":"pmem0" } real 0m39.805s user 0m0.138s sys 0m21.485s root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=256M { "dev":"namespace1.0", "mode":"fsdax", "map":"dev", "size":"250.00 MiB (262.14 MB)", "uuid":"15f1c8d5-416f-4d2e-9e39-d17ce8f73a42", "sector_size":512, "align":2097152, "blockdev":"pmem1" } real 0m43.899s user 0m0.113s sys 0m31.167s root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=128M { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":"124.00 MiB (130.02 MB)", "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", "sector_size":512, "align":2097152, "blockdev":"pmem0" } real 0m39.805s user 0m0.138s sys 0m21.485s root@QEMUCXL6060pmem:~# ndctl list [ { "dev":"namespace1.0", "mode":"fsdax", "map":"dev", "size":262144000, "uuid":"15f1c8d5-416f-4d2e-9e39-d17ce8f73a42", "sector_size":512, "align":2097152, "blockdev":"pmem1" }, { "dev":"namespace1.1", "mode":"fsdax", "map":"dev", "size":130023424, "uuid":"33682096-7364-412d-bfbc-fa7568939abd", "sector_size":512, "align":2097152, "blockdev":"pmem1.1" }, { "dev":"namespace0.1", "mode":"fsdax", "map":"dev", "size":262144000, "uuid":"5f9e28b4-f403-4e72-bb06-307ead53dec4", "sector_size":512, "align":2097152, "blockdev":"pmem0.1" }, { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":130023424, "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", "sector_size":512, "align":2097152, "blockdev":"pmem0" } ] 5. Re-Start Qemu and we could see - Region persistency using "cxl list" - Namespace persistency using "ndctl list" and cat /proc/iomem root@QEMUCXL6060pmem:~# cat /proc/iomem fed1c000-fed1ffff : Reserved feffc000-feffffff : Reserved fffc0000-ffffffff : Reserved 100000000-27fffffff : System RAM a90000000-c8fffffff : CXL Window 0 a90000000-b8fffffff : Persistent Memory a90000000-b8fffffff : region0 a90000000-a97ffffff : namespace0.0 a98000000-aa7ffffff : namespace0.1 c90000000-e8fffffff : CXL Window 1 c90000000-d8fffffff : Persistent Memory c90000000-d8fffffff : region1 c90000000-c9fffffff : namespace1.0 ca0000000-ca7ffffff : namespace1.1 380000000000-38000000ffff : PCI Bus 0000:0a 380000000000-38000000ffff : 0000:0a:00.0 - NOTE: We can see some lag in restart. Look at Observesation below 6. Also verify LSA version using "ndctl read-labels -j nmem0" root@QEMUCXL6060pmem:~# time ndctl read-labels -j nmem0 { "dev":"nmem0", "index":[ { "signature":"NAMESPACE_INDEX", "major":2, "minor":1, "labelsize":256, "seq":2, "nslot":4090 }, { "signature":"NAMESPACE_INDEX", "major":2, "minor":1, "labelsize":256, "seq":1, "nslot":4090 } ], "label":[ { "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", "uuid":"5b33b940-6767-4951-b923-952455482f52", "name":"", "flags":8, "nrange":1, "position":0, "dpa":134217728, "rawsize":268435456, "slot":0, "align":0, "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", "lbasize":512 }, { "type":"529d7c61-da07-47c4-a93f-ecdf2c06f444", "uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", "flags":0, "nlabel":1, "position":0, "dpa":0, "rawsize":4294967296, "hpa":45365592064, "slot":1, "interleave granularity":256, "align":0 }, { "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", "uuid":"2c1c33c0-909b-4c6a-8de4-dfc6c13ae82c", "name":"", "flags":0, "nrange":1, "position":0, "dpa":0, "rawsize":134217728, "slot":2, "align":0, "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", "lbasize":512 }, { "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", "uuid":"5b33b940-6767-4951-b923-952455482f52", "name":"", "flags":0, "nrange":1, "position":0, "dpa":134217728, "rawsize":268435456, "slot":3, "align":0, "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", "lbasize":512 } ] } read 1 nmem - NOTE: We have following UUID types as per CXL Spec "type":"529d7c61-da07-47c4-a93f-ecdf2c06f444" is region label "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c" is namespace label Limitation (WIP): ================ Current changes only support interleave way == 1 Observation: ============ First time namespace creation using ndctl takes around 10 to 20 second time while executing "devm_memremap_pages" at [3] As using this patchset, after auto region creation, namespace creation is happening in boot sequence (if nvdimm and cxl drivers are static), It is therefore boot sequence is increased by around 10 to 20 sec. Changes ======= Changes in v5->v6 ----------------- - Find v5 link at [4] - Find v4 link at [5] - Find v3 link at [6] - Find v2 link at [7] - v1 patch-set was broken. Find the v1 links at [8] and [9] [Misc] - Rebase with for-7.0/cxl-init [PATCH 01/18] - Add Ira RB tag [PATCH 03/18] - Use export_uuid() instead of uuid_copy() [Jonathan] - Add Jonathan RB tag - Add Ira RB tag [PATCH 04/18] - Move the assignments at relevant place [Jonathan] - Rename lslot to label_slot [Jonathan] - Add Ira RB tag [PATCH 05/18] - Add Jonathan RB tag - Add Ira RB tag [PATCH 06/18] - Add Jonathan RB tag - Add Ira RB tag [PATCH 07/18] - Add Ira RB tag [PATCH 08/18] - use export_uuid() to avoid casts to uuid_t * [Jonathan] - Add Jonathan RB tag - Add Ira RB tag [PATCH 09/18] - Add Jonathan RB tag - Add Ira RB tag [PATCH 10/18] - Initialize struct cxl_memdev_attach variable [Jonathan] - Add Jonathan RB tag [PATCH 10/18] - Its a new patch addition in this series - Seperate out renaming of __create_region() to cxl_create_region() [Jonathan] [PATCH 11/18] - Remove usage of cxl_dpa_free() from alloc_region_dpa() [Jonathan] [PATCH 13/18] - Add Jonathan RB tag [PATCH 15/18] - Add Jonathan RB tag [PATCH 16/18] - Fix wrong name state with region_label_state [Jonathan] - Add Jonathan RB tag [PATCH 17/18] - Optimize loop matching [Jonathan] - Use reverse xmas tree stype [Jonathan] [PATCH 18/18] - Add Jonathan RB tag [1]: https://lore.kernel.org/all/163116432405.2460985.5547867384570123403.stgit@dwillia2-desk3.amr.corp.intel.com/ [2]: https://github.com/neerajoss/ndctl/commits/linux-cxl/V1_CXL_LSA_2.1_Support/ [3]: https://elixir.bootlin.com/linux/v6.13.7/source/drivers/nvdimm/pmem.c#L520 [4]: https://lore.kernel.org/linux-cxl/20260109124437.4025893-1-s.neeraj@samsung.com/ [5]: https://lore.kernel.org/linux-cxl/20250917134116.1623730-1-s.neeraj@samsung.com/ [6]: https://lore.kernel.org/linux-cxl/20251119075255.2637388-1-s.neeraj@samsung.com/ [7]: https://lore.kernel.org/linux-cxl/20250730121209.303202-1-s.neeraj@samsung.com/ [8] v1 Cover Letter: https://lore.kernel.org/linux-cxl/1931444790.41750165203442.JavaMail.epsvc@epcpadp1new/ [9] v1 Rest Thread: https://lore.kernel.org/linux-cxl/158453976.61750165203630.JavaMail.epsvc@epcpadp1new/ Neeraj Kumar (18): nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label nvdimm/label: CXL labels skip the need for 'interleave-set cookie' nvdimm/label: Add namespace/region label support as per LSA 2.1 nvdimm/label: Include region label in slot validation nvdimm/label: Skip region label during ns label DPA reservation nvdimm/label: Preserve region label during namespace creation nvdimm/label: Add region label delete support nvdimm/label: Preserve cxl region information from region label nvdimm/label: Export routine to fetch region information cxl/mem: Refactor cxl pmem region auto-assembling cxl/region: Rename __create_region() to cxl_create_region() cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation cxl/pmem: Preserve region information into nd_set cxl/pmem_region: Prep patch to accommodate pmem_region attributes cxl/pmem_region: Introduce CONFIG_CXL_PMEM_REGION for core/pmem_region.c cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion cxl/pmem_region: Create pmem region using information parsed from LSA cxl/pmem: Add CXL LSA 2.1 support in cxl pmem Documentation/ABI/testing/sysfs-bus-cxl | 22 + drivers/cxl/Kconfig | 15 + drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 35 +- drivers/cxl/core/pmem_region.c | 422 +++++++++++++++++ drivers/cxl/core/region.c | 377 ++++++++------- drivers/cxl/cxl.h | 41 +- drivers/cxl/mem.c | 18 +- drivers/cxl/pci.c | 4 +- drivers/cxl/pmem.c | 15 +- drivers/cxl/port.c | 39 +- drivers/nvdimm/dimm.c | 5 + drivers/nvdimm/dimm_devs.c | 19 + drivers/nvdimm/label.c | 580 ++++++++++++++++++++---- drivers/nvdimm/label.h | 18 +- drivers/nvdimm/namespace_devs.c | 82 +++- drivers/nvdimm/nd-core.h | 2 + drivers/nvdimm/nd.h | 64 +++ drivers/nvdimm/region_devs.c | 10 + include/linux/libnvdimm.h | 28 ++ tools/testing/cxl/Kbuild | 1 + 21 files changed, 1424 insertions(+), 374 deletions(-) create mode 100644 drivers/cxl/core/pmem_region.c base-commit: bc62f5b308cbdedf29132fe96e9d591e526527e1 -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113123epcas5p1cd55b845a358fccd305e655664b4f042@epcas5p1.samsung.com>]
* [PATCH V6 01/18] nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label [not found] ` <CGME20260123113123epcas5p1cd55b845a358fccd305e655664b4f042@epcas5p1.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Prior to LSA 2.1 version, LSA contain only namespace labels. LSA 2.1 introduced in CXL 2.0 Spec, which contain region label along with namespace label. NDD_LABELING flag is used for namespace. Introduced NDD_REGION_LABELING flag for region label. Based on these flags nvdimm driver performs operation on namespace label or region label. NDD_REGION_LABELING will be utilized by cxl driver to enable LSA 2.1 region label support Accordingly updated label index version Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/dimm.c | 1 + drivers/nvdimm/dimm_devs.c | 7 +++++++ drivers/nvdimm/label.c | 21 +++++++++++++++++---- drivers/nvdimm/nd.h | 1 + include/linux/libnvdimm.h | 3 +++ 5 files changed, 29 insertions(+), 4 deletions(-) diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c index 2f6c26cc6a3e..07f5c5d5e537 100644 --- a/drivers/nvdimm/dimm.c +++ b/drivers/nvdimm/dimm.c @@ -62,6 +62,7 @@ static int nvdimm_probe(struct device *dev) if (rc < 0) dev_dbg(dev, "failed to unlock dimm: %d\n", rc); + ndd->cxl = nvdimm_region_label_supported(ndd->dev); /* * EACCES failures reading the namespace label-area-properties diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c index e1349ef5f8fd..3363a97cc5b5 100644 --- a/drivers/nvdimm/dimm_devs.c +++ b/drivers/nvdimm/dimm_devs.c @@ -18,6 +18,13 @@ static DEFINE_IDA(dimm_ida); +bool nvdimm_region_label_supported(struct device *dev) +{ + struct nvdimm *nvdimm = to_nvdimm(dev); + + return test_bit(NDD_REGION_LABELING, &nvdimm->flags); +} + /* * Retrieve bus and dimm handle and return if this bus supports * get_config_data commands diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index 04f4a049599a..0a9b6c5cb2c3 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -688,11 +688,24 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq, - (unsigned long) to_namespace_index(ndd, 0); nsindex->labeloff = __cpu_to_le64(offset); nsindex->nslot = __cpu_to_le32(nslot); - nsindex->major = __cpu_to_le16(1); - if (sizeof_namespace_label(ndd) < 256) + + /* Set LSA Label Index Version */ + if (ndd->cxl) { + /* CXL r3.2: Table 9-9 Label Index Block Layout */ + nsindex->major = __cpu_to_le16(2); nsindex->minor = __cpu_to_le16(1); - else - nsindex->minor = __cpu_to_le16(2); + } else { + nsindex->major = __cpu_to_le16(1); + /* + * NVDIMM Namespace Specification + * Table 2: Namespace Label Index Block Fields + */ + if (sizeof_namespace_label(ndd) < 256) + nsindex->minor = __cpu_to_le16(1); + else /* UEFI 2.7: Label Index Block Definitions */ + nsindex->minor = __cpu_to_le16(2); + } + nsindex->checksum = __cpu_to_le64(0); if (flags & ND_NSINDEX_INIT) { unsigned long *free = (unsigned long *) nsindex->free; diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index b199eea3260e..f631bd84d6f0 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -522,6 +522,7 @@ void nvdimm_set_labeling(struct device *dev); void nvdimm_set_locked(struct device *dev); void nvdimm_clear_locked(struct device *dev); int nvdimm_security_setup_events(struct device *dev); +bool nvdimm_region_label_supported(struct device *dev); #if IS_ENABLED(CONFIG_NVDIMM_KEYS) int nvdimm_security_unlock(struct device *dev); #else diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h index 28f086c4a187..5696715c33bb 100644 --- a/include/linux/libnvdimm.h +++ b/include/linux/libnvdimm.h @@ -44,6 +44,9 @@ enum { /* dimm provider wants synchronous registration by __nvdimm_create() */ NDD_REGISTER_SYNC = 8, + /* dimm supports region labels (LSA Format 2.1) */ + NDD_REGION_LABELING = 9, + /* need to set a limit somewhere, but yes, this is likely overkill */ ND_IOCTL_MAX_BUFLEN = SZ_4M, ND_CMD_MAX_ELEM = 5, -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113125epcas5p2331d76ce8aee6d594dbdd5b3e75e7d6f@epcas5p2.samsung.com>]
* [PATCH V6 02/18] nvdimm/label: CXL labels skip the need for 'interleave-set cookie' [not found] ` <CGME20260123113125epcas5p2331d76ce8aee6d594dbdd5b3e75e7d6f@epcas5p2.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar CXL LSA v2.1 utilizes the region labels stored in the LSA for interleave set configuration instead of interleave-set cookie used in previous LSA versions. As interleave-set cookie is not required for CXL LSA v2.1 format so skip its usage for CXL LSA 2.1 format Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Acked-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/namespace_devs.c | 8 +++++++- drivers/nvdimm/region_devs.c | 10 ++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c index a5edcacfe46d..43fdb806532e 100644 --- a/drivers/nvdimm/namespace_devs.c +++ b/drivers/nvdimm/namespace_devs.c @@ -1678,7 +1678,13 @@ static struct device *create_namespace_pmem(struct nd_region *nd_region, int rc = 0; u16 i; - if (cookie == 0) { + /* + * CXL LSA v2.1 utilizes the region label stored in the LSA for + * interleave set configuration. Whereas EFI LSA v1.1 & v1.2 + * utilizes interleave-set cookie. i.e, CXL labels skip the + * need for 'interleave-set cookie' + */ + if (!ndd->cxl && cookie == 0) { dev_dbg(&nd_region->dev, "invalid interleave-set-cookie\n"); return ERR_PTR(-ENXIO); } diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c index 1220530a23b6..77f36a585f13 100644 --- a/drivers/nvdimm/region_devs.c +++ b/drivers/nvdimm/region_devs.c @@ -841,6 +841,16 @@ u64 nd_region_interleave_set_cookie(struct nd_region *nd_region, if (!nd_set) return 0; + /* + * CXL LSA v2.1 utilizes the region label stored in the LSA for + * interleave set configuration. Whereas EFI LSA v1.1 & v1.2 + * utilizes interleave-set cookie. i.e, CXL labels skip the + * need for 'interleave-set cookie' + */ + if (nsindex && __le16_to_cpu(nsindex->major) == 2 + && __le16_to_cpu(nsindex->minor) == 1) + return 0; + if (nsindex && __le16_to_cpu(nsindex->major) == 1 && __le16_to_cpu(nsindex->minor) == 1) return nd_set->cookie1; -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113127epcas5p2c03d8e1749dc7e266870cc55eda31822@epcas5p2.samsung.com>]
* [PATCH V6 03/18] nvdimm/label: Add namespace/region label support as per LSA 2.1 [not found] ` <CGME20260123113127epcas5p2c03d8e1749dc7e266870cc55eda31822@epcas5p2.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Modify __pmem_label_update() to update region labels into LSA CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5 Modified __pmem_label_update() using setter functions to update namespace label as per CXL LSA 2.1 Create export routine nd_region_label_update() used for creating region label into LSA. It will be used later from CXL subsystem Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/label.c | 360 ++++++++++++++++++++++++++------ drivers/nvdimm/label.h | 17 +- drivers/nvdimm/namespace_devs.c | 20 +- drivers/nvdimm/nd.h | 56 +++++ include/linux/libnvdimm.h | 8 + 5 files changed, 391 insertions(+), 70 deletions(-) diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index 0a9b6c5cb2c3..06c6e23abf4e 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -22,8 +22,8 @@ static uuid_t nvdimm_btt2_uuid; static uuid_t nvdimm_pfn_uuid; static uuid_t nvdimm_dax_uuid; -static uuid_t cxl_region_uuid; -static uuid_t cxl_namespace_uuid; +uuid_t cxl_region_uuid; +uuid_t cxl_namespace_uuid; static const char NSINDEX_SIGNATURE[] = "NAMESPACE_INDEX\0"; @@ -278,15 +278,38 @@ static struct nd_namespace_label *nd_label_base(struct nvdimm_drvdata *ndd) return base + 2 * sizeof_namespace_index(ndd); } +static unsigned long find_slot(struct nvdimm_drvdata *ndd, + unsigned long label) +{ + unsigned long base; + + base = (unsigned long)nd_label_base(ndd); + return (label - base) / sizeof_namespace_label(ndd); +} + +static int to_lsa_slot(struct nvdimm_drvdata *ndd, + union nd_lsa_label *lsa_label) +{ + return find_slot(ndd, (unsigned long) lsa_label); +} + static int to_slot(struct nvdimm_drvdata *ndd, - struct nd_namespace_label *nd_label) + struct nd_label_ent *label_ent) +{ + if (uuid_equal(&cxl_region_uuid, &label_ent->label_uuid)) + return find_slot(ndd, (unsigned long) label_ent->region_label); + else + return find_slot(ndd, (unsigned long) label_ent->label); +} + +static union nd_lsa_label *to_lsa_label(struct nvdimm_drvdata *ndd, int slot) { unsigned long label, base; - label = (unsigned long) nd_label; base = (unsigned long) nd_label_base(ndd); + label = base + sizeof_namespace_label(ndd) * slot; - return (label - base) / sizeof_namespace_label(ndd); + return (union nd_lsa_label *) label; } static struct nd_namespace_label *to_label(struct nvdimm_drvdata *ndd, int slot) @@ -381,6 +404,16 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd, nsl_set_checksum(ndd, nd_label, sum); } +static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd, + struct cxl_region_label *region_label) +{ + u64 sum; + + region_label->checksum = __cpu_to_le64(0); + sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1); + region_label->checksum = __cpu_to_le64(sum); +} + static bool slot_valid(struct nvdimm_drvdata *ndd, struct nd_namespace_label *nd_label, u32 slot) { @@ -584,7 +617,7 @@ int nd_label_active_count(struct nvdimm_drvdata *ndd) return count; } -struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n) +union nd_lsa_label *nd_label_active(struct nvdimm_drvdata *ndd, int n) { struct nd_namespace_index *nsindex; unsigned long *free; @@ -601,7 +634,7 @@ struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n) continue; if (n-- == 0) - return to_label(ndd, slot); + return to_lsa_label(ndd, slot); } return NULL; @@ -737,9 +770,9 @@ static int nd_label_write_index(struct nvdimm_drvdata *ndd, int index, u32 seq, } static unsigned long nd_label_offset(struct nvdimm_drvdata *ndd, - struct nd_namespace_label *nd_label) + union nd_lsa_label *lsa_label) { - return (unsigned long) nd_label + return (unsigned long) lsa_label - (unsigned long) to_namespace_index(ndd, 0); } @@ -823,11 +856,15 @@ static void reap_victim(struct nd_mapping *nd_mapping, struct nd_label_ent *victim) { struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); - u32 slot = to_slot(ndd, victim->label); + u32 slot = to_slot(ndd, victim); dev_dbg(ndd->dev, "free: %d\n", slot); nd_label_free_slot(ndd, slot); - victim->label = NULL; + + if (uuid_equal(&cxl_region_uuid, &victim->label_uuid)) + victim->region_label = NULL; + else + victim->label = NULL; } static void nsl_set_type_guid(struct nvdimm_drvdata *ndd, @@ -884,26 +921,20 @@ enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd, return guid_to_nvdimm_cclass(&nd_label->efi.abstraction_guid); } -static int __pmem_label_update(struct nd_region *nd_region, - struct nd_mapping *nd_mapping, struct nd_namespace_pmem *nspm, - int pos, unsigned long flags) +static int namespace_label_update(struct nd_region *nd_region, + struct nd_mapping *nd_mapping, + struct nd_namespace_pmem *nspm, + int pos, u64 flags, + struct nd_namespace_label *ns_label, + struct nd_namespace_index *nsindex, + u32 slot) { struct nd_namespace_common *ndns = &nspm->nsio.common; struct nd_interleave_set *nd_set = nd_region->nd_set; struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); - struct nd_namespace_label *nd_label; - struct nd_namespace_index *nsindex; - struct nd_label_ent *label_ent; struct nd_label_id label_id; struct resource *res; - unsigned long *free; - u32 nslot, slot; - size_t offset; u64 cookie; - int rc; - - if (!preamble_next(ndd, &nsindex, &free, &nslot)) - return -ENXIO; cookie = nd_region_interleave_set_cookie(nd_region, nsindex); nd_label_gen_id(&label_id, nspm->uuid, 0); @@ -916,33 +947,150 @@ static int __pmem_label_update(struct nd_region *nd_region, return -ENXIO; } + nsl_set_type(ndd, ns_label); + nsl_set_uuid(ndd, ns_label, nspm->uuid); + nsl_set_name(ndd, ns_label, nspm->alt_name); + nsl_set_flags(ndd, ns_label, flags); + nsl_set_nlabel(ndd, ns_label, nd_region->ndr_mappings); + nsl_set_nrange(ndd, ns_label, 1); + nsl_set_position(ndd, ns_label, pos); + nsl_set_isetcookie(ndd, ns_label, cookie); + nsl_set_rawsize(ndd, ns_label, resource_size(res)); + nsl_set_lbasize(ndd, ns_label, nspm->lbasize); + nsl_set_dpa(ndd, ns_label, res->start); + nsl_set_slot(ndd, ns_label, slot); + nsl_set_alignment(ndd, ns_label, 0); + nsl_set_type_guid(ndd, ns_label, &nd_set->type_guid); + nsl_set_region_uuid(ndd, ns_label, &nd_set->uuid); + nsl_set_claim_class(ndd, ns_label, ndns->claim_class); + nsl_calculate_checksum(ndd, ns_label); + nd_dbg_dpa(nd_region, ndd, res, "\n"); + + return 0; +} + +static void region_label_update(struct nd_region *nd_region, + struct cxl_region_label *region_label, + struct nd_mapping *nd_mapping, + int pos, u64 flags, u32 slot) +{ + struct nd_interleave_set *nd_set = nd_region->nd_set; + struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + + /* Set Region Label Format identification UUID */ + export_uuid(region_label->type, &cxl_region_uuid); + + /* Set Current Region Label UUID */ + export_uuid(region_label->uuid, &nd_set->uuid); + + region_label->flags = __cpu_to_le32(flags); + region_label->nlabel = __cpu_to_le16(nd_region->ndr_mappings); + region_label->position = __cpu_to_le16(pos); + region_label->dpa = __cpu_to_le64(nd_mapping->start); + region_label->rawsize = __cpu_to_le64(nd_mapping->size); + region_label->hpa = __cpu_to_le64(nd_set->res->start); + region_label->slot = __cpu_to_le32(slot); + region_label->ig = __cpu_to_le32(nd_set->interleave_granularity); + region_label->align = __cpu_to_le32(0); + + /* Update fletcher64 Checksum */ + region_label_calculate_checksum(ndd, region_label); +} + +static bool is_label_reapable(struct nd_interleave_set *nd_set, + struct nd_namespace_pmem *nspm, + struct nvdimm_drvdata *ndd, + struct nd_label_ent *label_ent, + enum label_type ltype, + unsigned long *flags) +{ + switch (ltype) { + case NS_LABEL_TYPE: + if (test_and_clear_bit(ND_LABEL_REAP, flags) || + nsl_uuid_equal(ndd, label_ent->label, nspm->uuid)) + return true; + + break; + case RG_LABEL_TYPE: + if (region_label_uuid_equal(label_ent->region_label, + &nd_set->uuid)) + return true; + + break; + } + + return false; +} + +static bool is_label_present(struct nd_label_ent *label_ent, + enum label_type ltype) +{ + switch (ltype) { + case NS_LABEL_TYPE: + if (label_ent->label) + return true; + + break; + case RG_LABEL_TYPE: + if (label_ent->region_label) + return true; + + break; + } + + return false; +} + +static int __pmem_label_update(struct nd_region *nd_region, + struct nd_mapping *nd_mapping, + struct nd_namespace_pmem *nspm, + int pos, unsigned long flags, + enum label_type ltype) +{ + struct nd_interleave_set *nd_set = nd_region->nd_set; + struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + struct nd_namespace_index *nsindex; + struct nd_label_ent *label_ent; + union nd_lsa_label *lsa_label; + unsigned long *free; + struct device *dev; + u32 nslot, slot; + size_t offset; + int rc; + + if (!preamble_next(ndd, &nsindex, &free, &nslot)) + return -ENXIO; + /* allocate and write the label to the staging (next) index */ slot = nd_label_alloc_slot(ndd); if (slot == UINT_MAX) return -ENXIO; dev_dbg(ndd->dev, "allocated: %d\n", slot); - nd_label = to_label(ndd, slot); - memset(nd_label, 0, sizeof_namespace_label(ndd)); - nsl_set_uuid(ndd, nd_label, nspm->uuid); - nsl_set_name(ndd, nd_label, nspm->alt_name); - nsl_set_flags(ndd, nd_label, flags); - nsl_set_nlabel(ndd, nd_label, nd_region->ndr_mappings); - nsl_set_nrange(ndd, nd_label, 1); - nsl_set_position(ndd, nd_label, pos); - nsl_set_isetcookie(ndd, nd_label, cookie); - nsl_set_rawsize(ndd, nd_label, resource_size(res)); - nsl_set_lbasize(ndd, nd_label, nspm->lbasize); - nsl_set_dpa(ndd, nd_label, res->start); - nsl_set_slot(ndd, nd_label, slot); - nsl_set_type_guid(ndd, nd_label, &nd_set->type_guid); - nsl_set_claim_class(ndd, nd_label, ndns->claim_class); - nsl_calculate_checksum(ndd, nd_label); - nd_dbg_dpa(nd_region, ndd, res, "\n"); + lsa_label = to_lsa_label(ndd, slot); + memset(lsa_label, 0, sizeof_namespace_label(ndd)); + + switch (ltype) { + case NS_LABEL_TYPE: + dev = &nspm->nsio.common.dev; + rc = namespace_label_update(nd_region, nd_mapping, + nspm, pos, flags, &lsa_label->ns_label, + nsindex, slot); + if (rc) + return rc; + + break; + case RG_LABEL_TYPE: + dev = &nd_region->dev; + region_label_update(nd_region, &lsa_label->region_label, + nd_mapping, pos, flags, slot); + + break; + } /* update label */ - offset = nd_label_offset(ndd, nd_label); - rc = nvdimm_set_config_data(ndd, offset, nd_label, + offset = nd_label_offset(ndd, lsa_label); + rc = nvdimm_set_config_data(ndd, offset, lsa_label, sizeof_namespace_label(ndd)); if (rc < 0) return rc; @@ -950,10 +1098,11 @@ static int __pmem_label_update(struct nd_region *nd_region, /* Garbage collect the previous label */ mutex_lock(&nd_mapping->lock); list_for_each_entry(label_ent, &nd_mapping->labels, list) { - if (!label_ent->label) + if (!is_label_present(label_ent, ltype)) continue; - if (test_and_clear_bit(ND_LABEL_REAP, &label_ent->flags) || - nsl_uuid_equal(ndd, label_ent->label, nspm->uuid)) + + if (is_label_reapable(nd_set, nspm, ndd, label_ent, ltype, + &label_ent->flags)) reap_victim(nd_mapping, label_ent); } @@ -961,16 +1110,21 @@ static int __pmem_label_update(struct nd_region *nd_region, rc = nd_label_write_index(ndd, ndd->ns_next, nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0); if (rc == 0) { - list_for_each_entry(label_ent, &nd_mapping->labels, list) - if (!label_ent->label) { - label_ent->label = nd_label; - nd_label = NULL; - break; - } - dev_WARN_ONCE(&nspm->nsio.common.dev, nd_label, - "failed to track label: %d\n", - to_slot(ndd, nd_label)); - if (nd_label) + list_for_each_entry(label_ent, &nd_mapping->labels, list) { + if (is_label_present(label_ent, ltype)) + continue; + + if (ltype == NS_LABEL_TYPE) + label_ent->label = &lsa_label->ns_label; + else + label_ent->region_label = &lsa_label->region_label; + + lsa_label = NULL; + break; + } + dev_WARN_ONCE(dev, lsa_label, "failed to track label: %d\n", + to_lsa_slot(ndd, lsa_label)); + if (lsa_label) rc = -ENXIO; } mutex_unlock(&nd_mapping->lock); @@ -978,7 +1132,8 @@ static int __pmem_label_update(struct nd_region *nd_region, return rc; } -static int init_labels(struct nd_mapping *nd_mapping, int num_labels) +static int init_labels(struct nd_mapping *nd_mapping, int num_labels, + enum label_type ltype) { int i, old_num_labels = 0; struct nd_label_ent *label_ent; @@ -998,6 +1153,16 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels) label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL); if (!label_ent) return -ENOMEM; + + switch (ltype) { + case NS_LABEL_TYPE: + uuid_copy(&label_ent->label_uuid, &cxl_namespace_uuid); + break; + case RG_LABEL_TYPE: + uuid_copy(&label_ent->label_uuid, &cxl_region_uuid); + break; + } + mutex_lock(&nd_mapping->lock); list_add_tail(&label_ent->list, &nd_mapping->labels); mutex_unlock(&nd_mapping->lock); @@ -1041,19 +1206,19 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid) mutex_lock(&nd_mapping->lock); list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) { - struct nd_namespace_label *nd_label = label_ent->label; - - if (!nd_label) + if (label_ent->label) continue; active++; - if (!nsl_uuid_equal(ndd, nd_label, uuid)) + if (!nsl_uuid_equal(ndd, label_ent->label, uuid)) continue; active--; - slot = to_slot(ndd, nd_label); + slot = to_slot(ndd, label_ent); nd_label_free_slot(ndd, slot); dev_dbg(ndd->dev, "free: %d\n", slot); list_move_tail(&label_ent->list, &list); - label_ent->label = NULL; + + if (uuid_equal(&cxl_namespace_uuid, &label_ent->label_uuid)) + label_ent->label = NULL; } list_splice_tail_init(&list, &nd_mapping->labels); @@ -1067,6 +1232,19 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid) nd_inc_seq(__le32_to_cpu(nsindex->seq)), 0); } +static int find_region_label_count(struct nd_mapping *nd_mapping) +{ + struct nd_label_ent *label_ent; + int region_label_cnt = 0; + + guard(mutex)(&nd_mapping->lock); + list_for_each_entry(label_ent, &nd_mapping->labels, list) + if (uuid_equal(&cxl_region_uuid, &label_ent->label_uuid)) + region_label_cnt++; + + return region_label_cnt; +} + int nd_pmem_namespace_label_update(struct nd_region *nd_region, struct nd_namespace_pmem *nspm, resource_size_t size) { @@ -1075,6 +1253,7 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region, for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + int region_label_cnt; struct resource *res; int count = 0; @@ -1090,12 +1269,20 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region, count++; WARN_ON_ONCE(!count); - rc = init_labels(nd_mapping, count); + region_label_cnt = find_region_label_count(nd_mapping); + /* + * init_labels() scan labels and allocate new label based + * on its second parameter (num_labels). Therefore to + * allocate new namespace label also include previously + * added region label + */ + rc = init_labels(nd_mapping, count + region_label_cnt, + NS_LABEL_TYPE); if (rc < 0) return rc; rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, - NSLABEL_FLAG_UPDATING); + NSLABEL_FLAG_UPDATING, NS_LABEL_TYPE); if (rc) return rc; } @@ -1107,7 +1294,48 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region, for (i = 0; i < nd_region->ndr_mappings; i++) { struct nd_mapping *nd_mapping = &nd_region->mapping[i]; - rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0); + rc = __pmem_label_update(nd_region, nd_mapping, nspm, i, 0, + NS_LABEL_TYPE); + if (rc) + return rc; + } + + return 0; +} + +int nd_pmem_region_label_update(struct nd_region *nd_region) +{ + int i, rc; + + for (i = 0; i < nd_region->ndr_mappings; i++) { + struct nd_mapping *nd_mapping = &nd_region->mapping[i]; + struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + int region_label_cnt; + + /* No need to update region label for non cxl format */ + if (!ndd->cxl) + return 0; + + region_label_cnt = find_region_label_count(nd_mapping); + rc = init_labels(nd_mapping, region_label_cnt + 1, + RG_LABEL_TYPE); + if (rc < 0) + return rc; + + rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, + NSLABEL_FLAG_UPDATING, RG_LABEL_TYPE); + if (rc) + return rc; + } + + /* Clear the UPDATING flag per UEFI 2.7 expectations */ + for (i = 0; i < nd_region->ndr_mappings; i++) { + struct nd_mapping *nd_mapping = &nd_region->mapping[i]; + struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + + WARN_ON_ONCE(!ndd->cxl); + rc = __pmem_label_update(nd_region, nd_mapping, NULL, i, 0, + RG_LABEL_TYPE); if (rc) return rc; } diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h index 0650fb4b9821..f11f54056353 100644 --- a/drivers/nvdimm/label.h +++ b/drivers/nvdimm/label.h @@ -30,6 +30,11 @@ enum { ND_NSINDEX_INIT = 0x1, }; +enum label_type { + RG_LABEL_TYPE, + NS_LABEL_TYPE, +}; + /** * struct nd_namespace_index - label set superblock * @sig: NAMESPACE_INDEX\0 @@ -183,6 +188,15 @@ struct nd_namespace_label { }; }; +/* + * LSA 2.1 format introduces region label, which can also reside + * into LSA along with only namespace label as per v1.1 and v1.2 + */ +union nd_lsa_label { + struct nd_namespace_label ns_label; + struct cxl_region_label region_label; +}; + #define NVDIMM_BTT_GUID "8aed63a2-29a2-4c66-8b12-f05d15d3922a" #define NVDIMM_BTT2_GUID "18633bfc-1735-4217-8ac9-17239282d3f8" #define NVDIMM_PFN_GUID "266400ba-fb9f-4677-bcb0-968f11d0d225" @@ -215,7 +229,7 @@ struct nvdimm_drvdata; int nd_label_data_init(struct nvdimm_drvdata *ndd); size_t sizeof_namespace_index(struct nvdimm_drvdata *ndd); int nd_label_active_count(struct nvdimm_drvdata *ndd); -struct nd_namespace_label *nd_label_active(struct nvdimm_drvdata *ndd, int n); +union nd_lsa_label *nd_label_active(struct nvdimm_drvdata *ndd, int n); u32 nd_label_alloc_slot(struct nvdimm_drvdata *ndd); bool nd_label_free_slot(struct nvdimm_drvdata *ndd, u32 slot); u32 nd_label_nfree(struct nvdimm_drvdata *ndd); @@ -223,4 +237,5 @@ struct nd_region; struct nd_namespace_pmem; int nd_pmem_namespace_label_update(struct nd_region *nd_region, struct nd_namespace_pmem *nspm, resource_size_t size); +int nd_pmem_region_label_update(struct nd_region *nd_region); #endif /* __LABEL_H__ */ diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c index 43fdb806532e..657004021c95 100644 --- a/drivers/nvdimm/namespace_devs.c +++ b/drivers/nvdimm/namespace_devs.c @@ -232,6 +232,13 @@ static ssize_t __alt_name_store(struct device *dev, const char *buf, return rc; } +int nd_region_label_update(struct nd_region *nd_region) +{ + guard(nvdimm_bus)(&nd_region->dev); + return nd_pmem_region_label_update(nd_region); +} +EXPORT_SYMBOL_GPL(nd_region_label_update); + static int nd_namespace_label_update(struct nd_region *nd_region, struct device *dev) { @@ -2122,13 +2129,20 @@ static int init_active_labels(struct nd_region *nd_region) if (!count) continue; for (j = 0; j < count; j++) { - struct nd_namespace_label *label; + union nd_lsa_label *lsa_label; label_ent = kzalloc(sizeof(*label_ent), GFP_KERNEL); if (!label_ent) break; - label = nd_label_active(ndd, j); - label_ent->label = label; + + lsa_label = nd_label_active(ndd, j); + if (is_region_label(ndd, lsa_label)) { + label_ent->region_label = &lsa_label->region_label; + uuid_copy(&label_ent->label_uuid, &cxl_region_uuid); + } else { + label_ent->label = &lsa_label->ns_label; + uuid_copy(&label_ent->label_uuid, &cxl_namespace_uuid); + } mutex_lock(&nd_mapping->lock); list_add_tail(&label_ent->list, &nd_mapping->labels); diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index f631bd84d6f0..14cdc3e927b4 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -14,6 +14,9 @@ #include <linux/nd.h> #include "label.h" +extern uuid_t cxl_namespace_uuid; +extern uuid_t cxl_region_uuid; + enum { /* * Limits the maximum number of block apertures a dimm can @@ -295,6 +298,57 @@ static inline const u8 *nsl_uuid_raw(struct nvdimm_drvdata *ndd, return nd_label->efi.uuid; } +static inline void nsl_set_type(struct nvdimm_drvdata *ndd, + struct nd_namespace_label *ns_label) +{ + if (!(ndd->cxl && ns_label)) + return; + + export_uuid(ns_label->cxl.type, &cxl_namespace_uuid); +} + +static inline void nsl_set_alignment(struct nvdimm_drvdata *ndd, + struct nd_namespace_label *ns_label, + u32 align) +{ + if (!ndd->cxl) + return; + + ns_label->cxl.align = __cpu_to_le32(align); +} + +static inline void nsl_set_region_uuid(struct nvdimm_drvdata *ndd, + struct nd_namespace_label *ns_label, + const uuid_t *uuid) +{ + if (!(ndd->cxl && uuid)) + return; + + export_uuid(ns_label->cxl.region_uuid, uuid); +} + +static inline bool is_region_label(struct nvdimm_drvdata *ndd, + union nd_lsa_label *lsa_label) +{ + uuid_t tmp; + + if (!ndd->cxl) + return false; + + import_uuid(&tmp, lsa_label->region_label.type); + return uuid_equal(&tmp, &cxl_region_uuid); +} + +static inline bool +region_label_uuid_equal(struct cxl_region_label *region_label, + const uuid_t *uuid) +{ + uuid_t tmp; + + import_uuid(&tmp, region_label->uuid); + return uuid_equal(&tmp, uuid); +} + bool nsl_validate_type_guid(struct nvdimm_drvdata *ndd, struct nd_namespace_label *nd_label, guid_t *guid); enum nvdimm_claim_class nsl_get_claim_class(struct nvdimm_drvdata *ndd, @@ -376,7 +430,9 @@ enum nd_label_flags { struct nd_label_ent { struct list_head list; unsigned long flags; + uuid_t label_uuid; struct nd_namespace_label *label; + struct cxl_region_label *region_label; }; enum nd_mapping_lock_class { diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h index 5696715c33bb..2c213b9dac66 100644 --- a/include/linux/libnvdimm.h +++ b/include/linux/libnvdimm.h @@ -117,6 +117,13 @@ struct nd_interleave_set { u64 altcookie; guid_t type_guid; + + /* v2.1 region label info */ + uuid_t uuid; + int interleave_ways; + int interleave_granularity; + struct resource *res; + int nr_targets; }; struct nd_mapping_desc { @@ -307,6 +314,7 @@ int nvdimm_has_flush(struct nd_region *nd_region); int nvdimm_has_cache(struct nd_region *nd_region); int nvdimm_in_overwrite(struct nvdimm *nvdimm); bool is_nvdimm_sync(struct nd_region *nd_region); +int nd_region_label_update(struct nd_region *nd_region); static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113129epcas5p26bb834be2b841296ca1b3fff1a44ab76@epcas5p2.samsung.com>]
* [PATCH V6 04/18] nvdimm/label: Include region label in slot validation [not found] ` <CGME20260123113129epcas5p26bb834be2b841296ca1b3fff1a44ab76@epcas5p2.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Prior to LSA 2.1 Support, label in slot means only namespace label. But with LSA 2.1 a label can be either namespace or region label. Slot validation routine validates label slot by calculating label checksum. It was only validating namespace label. This changeset also validates region label if present. In previous patch to_lsa_label() was introduced along with to_label(). to_label() returns only namespace label whereas to_lsa_label() returns union nd_lsa_label* Convert all usage of to_label() to to_lsa_label() Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/label.c | 94 +++++++++++++++++++++++++++++------------- 1 file changed, 65 insertions(+), 29 deletions(-) diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index 06c6e23abf4e..5a4599c5e5a8 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -312,16 +312,6 @@ static union nd_lsa_label *to_lsa_label(struct nvdimm_drvdata *ndd, int slot) return (union nd_lsa_label *) label; } -static struct nd_namespace_label *to_label(struct nvdimm_drvdata *ndd, int slot) -{ - unsigned long label, base; - - base = (unsigned long) nd_label_base(ndd); - label = base + sizeof_namespace_label(ndd) * slot; - - return (struct nd_namespace_label *) label; -} - #define for_each_clear_bit_le(bit, addr, size) \ for ((bit) = find_next_zero_bit_le((addr), (size), 0); \ (bit) < (size); \ @@ -382,7 +372,7 @@ static bool nsl_validate_checksum(struct nvdimm_drvdata *ndd, { u64 sum, sum_save; - if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum)) + if (!efi_namespace_label_has(ndd, checksum)) return true; sum_save = nsl_get_checksum(ndd, nd_label); @@ -397,13 +387,25 @@ static void nsl_calculate_checksum(struct nvdimm_drvdata *ndd, { u64 sum; - if (!ndd->cxl && !efi_namespace_label_has(ndd, checksum)) + if (!efi_namespace_label_has(ndd, checksum)) return; nsl_set_checksum(ndd, nd_label, 0); sum = nd_fletcher64(nd_label, sizeof_namespace_label(ndd), 1); nsl_set_checksum(ndd, nd_label, sum); } +static bool region_label_validate_checksum(struct nvdimm_drvdata *ndd, + struct cxl_region_label *region_label) +{ + u64 sum, sum_save; + + sum_save = __le64_to_cpu(region_label->checksum); + region_label->checksum = __cpu_to_le64(0); + sum = nd_fletcher64(region_label, sizeof_namespace_label(ndd), 1); + region_label->checksum = __cpu_to_le64(sum_save); + return sum == sum_save; +} + static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd, struct cxl_region_label *region_label) { @@ -415,16 +417,36 @@ static void region_label_calculate_checksum(struct nvdimm_drvdata *ndd, } static bool slot_valid(struct nvdimm_drvdata *ndd, - struct nd_namespace_label *nd_label, u32 slot) + union nd_lsa_label *lsa_label, u32 slot) { + struct cxl_region_label *region_label; + struct nd_namespace_label *nd_label; + enum label_type type; bool valid; + static const char * const label_name[] = { + [RG_LABEL_TYPE] = "region", + [NS_LABEL_TYPE] = "namespace", + }; /* check that we are written where we expect to be written */ - if (slot != nsl_get_slot(ndd, nd_label)) - return false; - valid = nsl_validate_checksum(ndd, nd_label); + if (is_region_label(ndd, lsa_label)) { + region_label = &lsa_label->region_label; + type = RG_LABEL_TYPE; + if (slot != __le32_to_cpu(region_label->slot)) + return false; + valid = region_label_validate_checksum(ndd, region_label); + } else { + nd_label = &lsa_label->ns_label; + type = NS_LABEL_TYPE; + if (slot != nsl_get_slot(ndd, nd_label)) + return false; + valid = nsl_validate_checksum(ndd, nd_label); + } + if (!valid) - dev_dbg(ndd->dev, "fail checksum. slot: %d\n", slot); + dev_dbg(ndd->dev, "%s label checksum fail. slot: %d\n", + label_name[type], slot); + return valid; } @@ -440,14 +462,16 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd) for_each_clear_bit_le(slot, free, nslot) { struct nd_namespace_label *nd_label; struct nd_region *nd_region = NULL; + union nd_lsa_label *lsa_label; struct nd_label_id label_id; struct resource *res; uuid_t label_uuid; u32 flags; - nd_label = to_label(ndd, slot); + lsa_label = to_lsa_label(ndd, slot); + nd_label = &lsa_label->ns_label; - if (!slot_valid(ndd, nd_label, slot)) + if (!slot_valid(ndd, lsa_label, slot)) continue; nsl_get_uuid(ndd, nd_label, &label_uuid); @@ -598,14 +622,26 @@ int nd_label_active_count(struct nvdimm_drvdata *ndd) return 0; for_each_clear_bit_le(slot, free, nslot) { + struct cxl_region_label *region_label; struct nd_namespace_label *nd_label; - - nd_label = to_label(ndd, slot); - - if (!slot_valid(ndd, nd_label, slot)) { - u32 label_slot = nsl_get_slot(ndd, nd_label); - u64 size = nsl_get_rawsize(ndd, nd_label); - u64 dpa = nsl_get_dpa(ndd, nd_label); + union nd_lsa_label *lsa_label; + u32 label_slot; + u64 size, dpa; + + lsa_label = to_lsa_label(ndd, slot); + + if (!slot_valid(ndd, lsa_label, slot)) { + if (is_region_label(ndd, lsa_label)) { + region_label = &lsa_label->region_label; + label_slot = __le32_to_cpu(region_label->slot); + size = __le64_to_cpu(region_label->rawsize); + dpa = __le64_to_cpu(region_label->dpa); + } else { + nd_label = &lsa_label->ns_label; + label_slot = nsl_get_slot(ndd, nd_label); + size = nsl_get_rawsize(ndd, nd_label); + dpa = nsl_get_dpa(ndd, nd_label); + } dev_dbg(ndd->dev, "slot%d invalid slot: %d dpa: %llx size: %llx\n", @@ -627,10 +663,10 @@ union nd_lsa_label *nd_label_active(struct nvdimm_drvdata *ndd, int n) return NULL; for_each_clear_bit_le(slot, free, nslot) { - struct nd_namespace_label *nd_label; + union nd_lsa_label *lsa_label; - nd_label = to_label(ndd, slot); - if (!slot_valid(ndd, nd_label, slot)) + lsa_label = to_lsa_label(ndd, slot); + if (!slot_valid(ndd, lsa_label, slot)) continue; if (n-- == 0) -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113130epcas5p4af26a172d155a1c8b78d33945f495452@epcas5p4.samsung.com>]
* [PATCH V6 05/18] nvdimm/label: Skip region label during ns label DPA reservation [not found] ` <CGME20260123113130epcas5p4af26a172d155a1c8b78d33945f495452@epcas5p4.samsung.com> @ 2026-01-23 11:30 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:30 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar CXL 3.2 Spec mentions CXL LSA 2.1 Namespace Labels at section 9.13.2.5. If Namespace label is present in LSA during nvdimm_probe() then dimm-physical-address(DPA) reservation is required. But this reservation is not required by cxl region label. Therefore if LSA scanning finds any region label, skip it. Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/label.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index 5a4599c5e5a8..f18b04f63dcc 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -471,6 +471,13 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd) lsa_label = to_lsa_label(ndd, slot); nd_label = &lsa_label->ns_label; + /* + * If the LSA label is a region label then it doesn't + * require a dimm-physical-address(DPA) reservation. + */ + if (is_region_label(ndd, lsa_label)) + continue; + if (!slot_valid(ndd, lsa_label, slot)) continue; -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113132epcas5p368496769470a9163183a1d5a967442e2@epcas5p3.samsung.com>]
* [PATCH V6 06/18] nvdimm/label: Preserve region label during namespace creation [not found] ` <CGME20260123113132epcas5p368496769470a9163183a1d5a967442e2@epcas5p3.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar During namespace creation we scan labels present in LSA using scan_labels(). Currently scan_labels() is only preserving namespace labels into label_ent list. In this patch we also preserve region label into label_ent list Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/namespace_devs.c | 47 +++++++++++++++++++++++++++++---- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c index 657004021c95..8411f4152319 100644 --- a/drivers/nvdimm/namespace_devs.c +++ b/drivers/nvdimm/namespace_devs.c @@ -1994,9 +1994,32 @@ static struct device **scan_labels(struct nd_region *nd_region) if (count == 0) { struct nd_namespace_pmem *nspm; + for (i = 0; i < nd_region->ndr_mappings; i++) { + struct cxl_region_label *region_label; + struct nd_label_ent *le, *e; + LIST_HEAD(list); - /* Publish a zero-sized namespace for userspace to configure. */ - nd_mapping_free_labels(nd_mapping); + nd_mapping = &nd_region->mapping[i]; + if (list_empty(&nd_mapping->labels)) + continue; + + list_for_each_entry_safe(le, e, &nd_mapping->labels, + list) { + region_label = le->region_label; + if (!region_label) + continue; + + /* Preserve region labels if present */ + list_move_tail(&le->list, &list); + } + + /* + * Publish a zero-sized namespace for userspace + * to configure. + */ + nd_mapping_free_labels(nd_mapping); + list_splice_init(&list, &nd_mapping->labels); + } nspm = kzalloc(sizeof(*nspm), GFP_KERNEL); if (!nspm) goto err; @@ -2008,7 +2031,7 @@ static struct device **scan_labels(struct nd_region *nd_region) } else if (is_memory(&nd_region->dev)) { /* clean unselected labels */ for (i = 0; i < nd_region->ndr_mappings; i++) { - struct list_head *l, *e; + struct nd_label_ent *le, *e; LIST_HEAD(list); int j; @@ -2019,10 +2042,24 @@ static struct device **scan_labels(struct nd_region *nd_region) } j = count; - list_for_each_safe(l, e, &nd_mapping->labels) { + list_for_each_entry_safe(le, e, &nd_mapping->labels, + list) { + /* Preserve region labels */ + if (uuid_equal(&le->label_uuid, + &cxl_region_uuid)) { + list_move_tail(&le->list, &list); + continue; + } + + /* + * Once preserving selected ns label done + * break out of loop + */ if (!j--) break; - list_move_tail(l, &list); + + /* Preserve selected ns label */ + list_move_tail(&le->list, &list); } nd_mapping_free_labels(nd_mapping); list_splice_init(&list, &nd_mapping->labels); -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113133epcas5p4989af8ca6a75569b2b47d80514e62c57@epcas5p4.samsung.com>]
* [PATCH V6 07/18] nvdimm/label: Add region label delete support [not found] ` <CGME20260123113133epcas5p4989af8ca6a75569b2b47d80514e62c57@epcas5p4.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Create export routine nd_region_label_delete() used for deleting region label from LSA. It will be used later from CXL subsystem Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/label.c | 74 ++++++++++++++++++++++++++++++--- drivers/nvdimm/label.h | 1 + drivers/nvdimm/namespace_devs.c | 7 ++++ drivers/nvdimm/nd.h | 6 +++ include/linux/libnvdimm.h | 1 + 5 files changed, 83 insertions(+), 6 deletions(-) diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index f18b04f63dcc..014af60d68a1 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -1230,7 +1230,8 @@ static int init_labels(struct nd_mapping *nd_mapping, int num_labels, return max(num_labels, old_num_labels); } -static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid) +static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid, + enum label_type ltype) { struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); struct nd_label_ent *label_ent, *e; @@ -1249,11 +1250,24 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid) mutex_lock(&nd_mapping->lock); list_for_each_entry_safe(label_ent, e, &nd_mapping->labels, list) { - if (label_ent->label) + if ((ltype == NS_LABEL_TYPE && !label_ent->label) || + (ltype == RG_LABEL_TYPE && !label_ent->region_label)) continue; active++; - if (!nsl_uuid_equal(ndd, label_ent->label, uuid)) - continue; + + switch (ltype) { + case NS_LABEL_TYPE: + if (!nsl_uuid_equal(ndd, label_ent->label, uuid)) + continue; + + break; + case RG_LABEL_TYPE: + if (!region_label_uuid_equal(label_ent->region_label, uuid)) + continue; + + break; + } + active--; slot = to_slot(ndd, label_ent); nd_label_free_slot(ndd, slot); @@ -1262,10 +1276,12 @@ static int del_labels(struct nd_mapping *nd_mapping, uuid_t *uuid) if (uuid_equal(&cxl_namespace_uuid, &label_ent->label_uuid)) label_ent->label = NULL; + else + label_ent->region_label = NULL; } list_splice_tail_init(&list, &nd_mapping->labels); - if (active == 0) { + if ((ltype == NS_LABEL_TYPE) && (active == 0)) { nd_mapping_free_labels(nd_mapping); dev_dbg(ndd->dev, "no more active labels\n"); } @@ -1301,7 +1317,8 @@ int nd_pmem_namespace_label_update(struct nd_region *nd_region, int count = 0; if (size == 0) { - rc = del_labels(nd_mapping, nspm->uuid); + rc = del_labels(nd_mapping, nspm->uuid, + NS_LABEL_TYPE); if (rc) return rc; continue; @@ -1386,6 +1403,51 @@ int nd_pmem_region_label_update(struct nd_region *nd_region) return 0; } +int nd_pmem_region_label_delete(struct nd_region *nd_region) +{ + struct nd_interleave_set *nd_set = nd_region->nd_set; + struct nd_label_ent *label_ent; + int i, rc; + + for (i = 0; i < nd_region->ndr_mappings; i++) { + struct nd_mapping *nd_mapping = &nd_region->mapping[i]; + struct nvdimm_drvdata *ndd = to_ndd(nd_mapping); + + /* Find non cxl format supported ndr_mappings */ + if (!ndd->cxl) { + dev_info(&nd_region->dev, "Unsupported region label\n"); + return -EINVAL; + } + + /* Find if any NS label using this region */ + guard(mutex)(&nd_mapping->lock); + list_for_each_entry(label_ent, &nd_mapping->labels, list) { + if (!label_ent->label) + continue; + + /* + * Check if any available NS labels has same + * region_uuid in LSA + */ + if (nsl_region_uuid_equal(label_ent->label, + &nd_set->uuid)) { + dev_dbg(&nd_region->dev, + "Region/Namespace label in use\n"); + return -EBUSY; + } + } + } + + for (i = 0; i < nd_region->ndr_mappings; i++) { + rc = del_labels(&nd_region->mapping[i], &nd_set->uuid, + RG_LABEL_TYPE); + if (rc) + return rc; + } + + return 0; +} + int __init nd_label_init(void) { WARN_ON(guid_parse(NVDIMM_BTT_GUID, &nvdimm_btt_guid)); diff --git a/drivers/nvdimm/label.h b/drivers/nvdimm/label.h index f11f54056353..80a7f7dd8ba7 100644 --- a/drivers/nvdimm/label.h +++ b/drivers/nvdimm/label.h @@ -238,4 +238,5 @@ struct nd_namespace_pmem; int nd_pmem_namespace_label_update(struct nd_region *nd_region, struct nd_namespace_pmem *nspm, resource_size_t size); int nd_pmem_region_label_update(struct nd_region *nd_region); +int nd_pmem_region_label_delete(struct nd_region *nd_region); #endif /* __LABEL_H__ */ diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c index 8411f4152319..9047826138be 100644 --- a/drivers/nvdimm/namespace_devs.c +++ b/drivers/nvdimm/namespace_devs.c @@ -239,6 +239,13 @@ int nd_region_label_update(struct nd_region *nd_region) } EXPORT_SYMBOL_GPL(nd_region_label_update); +int nd_region_label_delete(struct nd_region *nd_region) +{ + guard(nvdimm_bus)(&nd_region->dev); + return nd_pmem_region_label_delete(nd_region); +} +EXPORT_SYMBOL_GPL(nd_region_label_delete); + static int nd_namespace_label_update(struct nd_region *nd_region, struct device *dev) { diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index 14cdc3e927b4..ec856601fda0 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -339,6 +339,12 @@ static inline bool is_region_label(struct nvdimm_drvdata *ndd, return uuid_equal(&tmp, &cxl_region_uuid); } +static inline bool nsl_region_uuid_equal(struct nd_namespace_label *ns_label, + const uuid_t *uuid) +{ + return uuid_equal((uuid_t *)ns_label->cxl.region_uuid, uuid); +} + static inline bool region_label_uuid_equal(struct cxl_region_label *region_label, const uuid_t *uuid) diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h index 2c213b9dac66..bbf14a260c93 100644 --- a/include/linux/libnvdimm.h +++ b/include/linux/libnvdimm.h @@ -315,6 +315,7 @@ int nvdimm_has_cache(struct nd_region *nd_region); int nvdimm_in_overwrite(struct nvdimm *nvdimm); bool is_nvdimm_sync(struct nd_region *nd_region); int nd_region_label_update(struct nd_region *nd_region); +int nd_region_label_delete(struct nd_region *nd_region); static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113135epcas5p2b0e7d9651a5a78b5451daeb6b1e018d8@epcas5p2.samsung.com>]
* [PATCH V6 08/18] nvdimm/label: Preserve cxl region information from region label [not found] ` <CGME20260123113135epcas5p2b0e7d9651a5a78b5451daeb6b1e018d8@epcas5p2.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Preserve region information from region label during nvdimm_probe. This preserved region information is used for creating cxl region to achieve region persistency across reboot. This patch supports interleave way == 1, it is therefore it preserves only one region into LSA Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/dimm.c | 4 ++++ drivers/nvdimm/label.c | 38 ++++++++++++++++++++++++++++++++++++++ drivers/nvdimm/nd-core.h | 2 ++ drivers/nvdimm/nd.h | 1 + include/linux/libnvdimm.h | 14 ++++++++++++++ 5 files changed, 59 insertions(+) diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c index 07f5c5d5e537..590ec883903d 100644 --- a/drivers/nvdimm/dimm.c +++ b/drivers/nvdimm/dimm.c @@ -107,6 +107,10 @@ static int nvdimm_probe(struct device *dev) if (rc) goto err; + /* Preserve cxl region info if available */ + if (ndd->cxl) + nvdimm_cxl_region_preserve(ndd); + return 0; err: diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c index 014af60d68a1..054dd4e47ab4 100644 --- a/drivers/nvdimm/label.c +++ b/drivers/nvdimm/label.c @@ -495,6 +495,44 @@ int nd_label_reserve_dpa(struct nvdimm_drvdata *ndd) return 0; } +int nvdimm_cxl_region_preserve(struct nvdimm_drvdata *ndd) +{ + struct nvdimm *nvdimm = to_nvdimm(ndd->dev); + struct cxl_pmem_region_params *p = &nvdimm->cxl_region_params; + struct nd_namespace_index *nsindex; + unsigned long *free; + u32 nslot, slot; + + if (!preamble_current(ndd, &nsindex, &free, &nslot)) + return 0; /* no label, nothing to preserve */ + + for_each_clear_bit_le(slot, free, nslot) { + union nd_lsa_label *lsa_label = to_lsa_label(ndd, slot); + struct cxl_region_label *region_label = &lsa_label->region_label; + uuid_t region_type_uuid; + + import_uuid(®ion_type_uuid, region_label->type); + + /* TODO: Currently preserving only one region */ + if (uuid_equal(&cxl_region_uuid, ®ion_type_uuid)) { + nvdimm->is_region_label = true; + import_uuid(&p->uuid, region_label->uuid); + p->flags = __le32_to_cpu(region_label->flags); + p->nlabel = __le16_to_cpu(region_label->nlabel); + p->position = __le16_to_cpu(region_label->position); + p->dpa = __le64_to_cpu(region_label->dpa); + p->rawsize = __le64_to_cpu(region_label->rawsize); + p->hpa = __le64_to_cpu(region_label->hpa); + p->slot = __le32_to_cpu(region_label->slot); + p->ig = __le32_to_cpu(region_label->ig); + p->align = __le32_to_cpu(region_label->align); + break; + } + } + + return 0; +} + int nd_label_data_init(struct nvdimm_drvdata *ndd) { size_t config_size, read_size, max_xfer, offset; diff --git a/drivers/nvdimm/nd-core.h b/drivers/nvdimm/nd-core.h index bfc6bfeb6e24..a73fac81531e 100644 --- a/drivers/nvdimm/nd-core.h +++ b/drivers/nvdimm/nd-core.h @@ -46,6 +46,8 @@ struct nvdimm { } sec; struct delayed_work dwork; const struct nvdimm_fw_ops *fw_ops; + bool is_region_label; + struct cxl_pmem_region_params cxl_region_params; }; static inline unsigned long nvdimm_security_flags( diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h index ec856601fda0..578a828be918 100644 --- a/drivers/nvdimm/nd.h +++ b/drivers/nvdimm/nd.h @@ -585,6 +585,7 @@ void nvdimm_set_locked(struct device *dev); void nvdimm_clear_locked(struct device *dev); int nvdimm_security_setup_events(struct device *dev); bool nvdimm_region_label_supported(struct device *dev); +int nvdimm_cxl_region_preserve(struct nvdimm_drvdata *ndd); #if IS_ENABLED(CONFIG_NVDIMM_KEYS) int nvdimm_security_unlock(struct device *dev); #else diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h index bbf14a260c93..07ea2e3f821a 100644 --- a/include/linux/libnvdimm.h +++ b/include/linux/libnvdimm.h @@ -108,6 +108,20 @@ struct nd_cmd_desc { int out_sizes[ND_CMD_MAX_ELEM]; }; +struct cxl_pmem_region_params { + uuid_t uuid; + u32 flags; + u16 nlabel; + u16 position; + u64 dpa; + u64 rawsize; + u64 hpa; + u32 slot; + u32 ig; + u32 align; + int nr_targets; +}; + struct nd_interleave_set { /* v1.1 definition of the interleave-set-cookie algorithm */ u64 cookie1; -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113136epcas5p22683ce5ef04a3da50fbb319a43e331a6@epcas5p2.samsung.com>]
* [PATCH V6 09/18] nvdimm/label: Export routine to fetch region information [not found] ` <CGME20260123113136epcas5p22683ce5ef04a3da50fbb319a43e331a6@epcas5p2.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar CXL region information preserved from the LSA needs to be exported for use by the CXL driver for CXL region re-creation. Reviewed-by: Ira Weiny <ira.weiny@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/nvdimm/dimm_devs.c | 12 ++++++++++++ include/linux/libnvdimm.h | 2 ++ 2 files changed, 14 insertions(+) diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c index 3363a97cc5b5..e1c95da92fbf 100644 --- a/drivers/nvdimm/dimm_devs.c +++ b/drivers/nvdimm/dimm_devs.c @@ -280,6 +280,18 @@ void *nvdimm_provider_data(struct nvdimm *nvdimm) } EXPORT_SYMBOL_GPL(nvdimm_provider_data); +bool nvdimm_has_cxl_region(struct nvdimm *nvdimm) +{ + return nvdimm->is_region_label; +} +EXPORT_SYMBOL_GPL(nvdimm_has_cxl_region); + +void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm) +{ + return &nvdimm->cxl_region_params; +} +EXPORT_SYMBOL_GPL(nvdimm_get_cxl_region_param); + static ssize_t commands_show(struct device *dev, struct device_attribute *attr, char *buf) { diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h index 07ea2e3f821a..3ffd50ab6ac4 100644 --- a/include/linux/libnvdimm.h +++ b/include/linux/libnvdimm.h @@ -330,6 +330,8 @@ int nvdimm_in_overwrite(struct nvdimm *nvdimm); bool is_nvdimm_sync(struct nd_region *nd_region); int nd_region_label_update(struct nd_region *nd_region); int nd_region_label_delete(struct nd_region *nd_region); +bool nvdimm_has_cxl_region(struct nvdimm *nvdimm); +void *nvdimm_get_cxl_region_param(struct nvdimm *nvdimm); static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc) -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113138epcas5p4bc54ae41192dbc9c958c43ad8a80b5c6@epcas5p4.samsung.com>]
* [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling [not found] ` <CGME20260123113138epcas5p4bc54ae41192dbc9c958c43ad8a80b5c6@epcas5p4.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 2026-01-26 22:48 ` Dave Jiang 2026-01-28 0:02 ` dan.j.williams 0 siblings, 2 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly used to get called at last in cxl_endpoint_port_probe(), which requires cxl_nvd presence. For cxl region persistency, region creation happens during nvdimm_probe which need the completion of endpoint probe. In order to accommodate both cxl pmem region auto-assembly and cxl region persistency, refactored following 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This will be called only after successful completion of endpoint probe. 2. Create cxl_region_discovery() which performs pmem region auto-assembly and remove cxl pmem region auto-assembly from cxl_endpoint_port_probe() 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets called during cxl_pci_probe() in context of cxl_mem_probe() 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees both the completion of endpoint probe and cxl_nvd presence before calling cxlmd->attach->probe(). Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/core/region.c | 37 +++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 5 +++++ drivers/cxl/mem.c | 18 +++++++++--------- drivers/cxl/pci.c | 4 +++- drivers/cxl/port.c | 39 +-------------------------------------- 5 files changed, 55 insertions(+), 48 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index ae899f68551f..26238fb5e8cf 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3727,6 +3727,43 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled) } EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL"); +static int discover_region(struct device *dev, void *unused) +{ + struct cxl_endpoint_decoder *cxled; + int rc; + + if (!is_endpoint_decoder(dev)) + return 0; + + cxled = to_cxl_endpoint_decoder(dev); + if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) + return 0; + + if (cxled->state != CXL_DECODER_STATE_AUTO) + return 0; + + /* + * Region enumeration is opportunistic, if this add-event fails, + * continue to the next endpoint decoder. + */ + rc = cxl_add_to_region(cxled); + if (rc) + dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", + cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); + + return 0; +} + +int cxl_region_discovery(struct cxl_memdev *cxlmd) +{ + struct cxl_port *port = cxlmd->endpoint; + + device_for_each_child(&port->dev, NULL, discover_region); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_region_discovery, "CXL"); + u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa) { struct cxl_region_ref *iter; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index c796c3db36e0..86efcc4fb963 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -906,6 +906,7 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); int cxl_add_to_region(struct cxl_endpoint_decoder *cxled); struct cxl_dax_region *to_cxl_dax_region(struct device *dev); u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa); +int cxl_region_discovery(struct cxl_memdev *cxlmd); #else static inline bool is_cxl_pmem_region(struct device *dev) { @@ -928,6 +929,10 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, { return 0; } +static inline int cxl_region_discovery(struct cxl_memdev *cxlmd) +{ + return 0; +} #endif void cxl_endpoint_parse_cdat(struct cxl_port *port); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 333c366b69e7..7d19528d9b55 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -114,15 +114,6 @@ static int cxl_mem_probe(struct device *dev) return -ENXIO; } - if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { - rc = devm_cxl_add_nvdimm(parent_port, cxlmd); - if (rc) { - if (rc == -ENODEV) - dev_info(dev, "PMEM disabled by platform\n"); - return rc; - } - } - if (dport->rch) endpoint_parent = parent_port->uport_dev; else @@ -142,6 +133,15 @@ static int cxl_mem_probe(struct device *dev) return rc; } + if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { + rc = devm_cxl_add_nvdimm(parent_port, cxlmd); + if (rc) { + if (rc == -ENODEV) + dev_info(dev, "PMEM disabled by platform\n"); + return rc; + } + } + if (cxlmd->attach) { rc = cxlmd->attach->probe(cxlmd); if (rc) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 549368a9c868..70b40a10be7a 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -903,6 +903,7 @@ __ATTRIBUTE_GROUPS(cxl_rcd); static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); + struct cxl_memdev_attach memdev_attach = { 0 }; struct cxl_dpa_info range_info = { 0 }; struct cxl_memdev_state *mds; struct cxl_dev_state *cxlds; @@ -1006,7 +1007,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) dev_dbg(&pdev->dev, "No CXL Features discovered\n"); - cxlmd = devm_cxl_add_memdev(cxlds, NULL); + memdev_attach.probe = cxl_region_discovery; + cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 7937e7e53797..fbeff1978bfb 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,33 +30,6 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } -static int discover_region(struct device *dev, void *unused) -{ - struct cxl_endpoint_decoder *cxled; - int rc; - - if (!is_endpoint_decoder(dev)) - return 0; - - cxled = to_cxl_endpoint_decoder(dev); - if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) - return 0; - - if (cxled->state != CXL_DECODER_STATE_AUTO) - return 0; - - /* - * Region enumeration is opportunistic, if this add-event fails, - * continue to the next endpoint decoder. - */ - rc = cxl_add_to_region(cxled); - if (rc) - dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", - cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); - - return 0; -} - static int cxl_switch_port_probe(struct cxl_port *port) { /* Reset nr_dports for rebind of driver */ @@ -82,17 +55,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) if (rc) return rc; - rc = devm_cxl_endpoint_decoders_setup(port); - if (rc) - return rc; - - /* - * Now that all endpoint decoders are successfully enumerated, try to - * assemble regions from committed decoders - */ - device_for_each_child(&port->dev, NULL, discover_region); - - return 0; + return devm_cxl_endpoint_decoders_setup(port); } static int cxl_port_probe(struct device *dev) -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling 2026-01-23 11:31 ` [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling Neeraj Kumar @ 2026-01-26 22:48 ` Dave Jiang 2026-01-27 23:45 ` Dave Jiang 2026-01-28 0:02 ` dan.j.williams 1 sibling, 1 reply; 35+ messages in thread From: Dave Jiang @ 2026-01-26 22:48 UTC (permalink / raw) To: Neeraj Kumar, linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel On 1/23/26 4:31 AM, Neeraj Kumar wrote: > In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called > before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly > used to get called at last in cxl_endpoint_port_probe(), which requires > cxl_nvd presence. > > For cxl region persistency, region creation happens during nvdimm_probe > which need the completion of endpoint probe. > > In order to accommodate both cxl pmem region auto-assembly and cxl region > persistency, refactored following > > 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This > will be called only after successful completion of endpoint probe. > > 2. Create cxl_region_discovery() which performs pmem region > auto-assembly and remove cxl pmem region auto-assembly from > cxl_endpoint_port_probe() > > 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets > called during cxl_pci_probe() in context of cxl_mem_probe() > > 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so > move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees > both the completion of endpoint probe and cxl_nvd presence before > calling cxlmd->attach->probe(). > > Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> > Reviewed-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> Hi Neeraj, Just FYI this is the patch that breaks the auto-region assemble in cxl_test. DJ > --- > drivers/cxl/core/region.c | 37 +++++++++++++++++++++++++++++++++++++ > drivers/cxl/cxl.h | 5 +++++ > drivers/cxl/mem.c | 18 +++++++++--------- > drivers/cxl/pci.c | 4 +++- > drivers/cxl/port.c | 39 +-------------------------------------- > 5 files changed, 55 insertions(+), 48 deletions(-) > > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c > index ae899f68551f..26238fb5e8cf 100644 > --- a/drivers/cxl/core/region.c > +++ b/drivers/cxl/core/region.c > @@ -3727,6 +3727,43 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled) > } > EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL"); > > +static int discover_region(struct device *dev, void *unused) > +{ > + struct cxl_endpoint_decoder *cxled; > + int rc; > + > + if (!is_endpoint_decoder(dev)) > + return 0; > + > + cxled = to_cxl_endpoint_decoder(dev); > + if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) > + return 0; > + > + if (cxled->state != CXL_DECODER_STATE_AUTO) > + return 0; > + > + /* > + * Region enumeration is opportunistic, if this add-event fails, > + * continue to the next endpoint decoder. > + */ > + rc = cxl_add_to_region(cxled); > + if (rc) > + dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", > + cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); > + > + return 0; > +} > + > +int cxl_region_discovery(struct cxl_memdev *cxlmd) > +{ > + struct cxl_port *port = cxlmd->endpoint; > + > + device_for_each_child(&port->dev, NULL, discover_region); > + > + return 0; > +} > +EXPORT_SYMBOL_NS_GPL(cxl_region_discovery, "CXL"); > + > u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa) > { > struct cxl_region_ref *iter; > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h > index c796c3db36e0..86efcc4fb963 100644 > --- a/drivers/cxl/cxl.h > +++ b/drivers/cxl/cxl.h > @@ -906,6 +906,7 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); > int cxl_add_to_region(struct cxl_endpoint_decoder *cxled); > struct cxl_dax_region *to_cxl_dax_region(struct device *dev); > u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa); > +int cxl_region_discovery(struct cxl_memdev *cxlmd); > #else > static inline bool is_cxl_pmem_region(struct device *dev) > { > @@ -928,6 +929,10 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, > { > return 0; > } > +static inline int cxl_region_discovery(struct cxl_memdev *cxlmd) > +{ > + return 0; > +} > #endif > > void cxl_endpoint_parse_cdat(struct cxl_port *port); > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c > index 333c366b69e7..7d19528d9b55 100644 > --- a/drivers/cxl/mem.c > +++ b/drivers/cxl/mem.c > @@ -114,15 +114,6 @@ static int cxl_mem_probe(struct device *dev) > return -ENXIO; > } > > - if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { > - rc = devm_cxl_add_nvdimm(parent_port, cxlmd); > - if (rc) { > - if (rc == -ENODEV) > - dev_info(dev, "PMEM disabled by platform\n"); > - return rc; > - } > - } > - > if (dport->rch) > endpoint_parent = parent_port->uport_dev; > else > @@ -142,6 +133,15 @@ static int cxl_mem_probe(struct device *dev) > return rc; > } > > + if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { > + rc = devm_cxl_add_nvdimm(parent_port, cxlmd); > + if (rc) { > + if (rc == -ENODEV) > + dev_info(dev, "PMEM disabled by platform\n"); > + return rc; > + } > + } > + > if (cxlmd->attach) { > rc = cxlmd->attach->probe(cxlmd); > if (rc) > diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c > index 549368a9c868..70b40a10be7a 100644 > --- a/drivers/cxl/pci.c > +++ b/drivers/cxl/pci.c > @@ -903,6 +903,7 @@ __ATTRIBUTE_GROUPS(cxl_rcd); > static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) > { > struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); > + struct cxl_memdev_attach memdev_attach = { 0 }; > struct cxl_dpa_info range_info = { 0 }; > struct cxl_memdev_state *mds; > struct cxl_dev_state *cxlds; > @@ -1006,7 +1007,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) > if (rc) > dev_dbg(&pdev->dev, "No CXL Features discovered\n"); > > - cxlmd = devm_cxl_add_memdev(cxlds, NULL); > + memdev_attach.probe = cxl_region_discovery; > + cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); > if (IS_ERR(cxlmd)) > return PTR_ERR(cxlmd); > > diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c > index 7937e7e53797..fbeff1978bfb 100644 > --- a/drivers/cxl/port.c > +++ b/drivers/cxl/port.c > @@ -30,33 +30,6 @@ static void schedule_detach(void *cxlmd) > schedule_cxl_memdev_detach(cxlmd); > } > > -static int discover_region(struct device *dev, void *unused) > -{ > - struct cxl_endpoint_decoder *cxled; > - int rc; > - > - if (!is_endpoint_decoder(dev)) > - return 0; > - > - cxled = to_cxl_endpoint_decoder(dev); > - if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) > - return 0; > - > - if (cxled->state != CXL_DECODER_STATE_AUTO) > - return 0; > - > - /* > - * Region enumeration is opportunistic, if this add-event fails, > - * continue to the next endpoint decoder. > - */ > - rc = cxl_add_to_region(cxled); > - if (rc) > - dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", > - cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); > - > - return 0; > -} > - > static int cxl_switch_port_probe(struct cxl_port *port) > { > /* Reset nr_dports for rebind of driver */ > @@ -82,17 +55,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) > if (rc) > return rc; > > - rc = devm_cxl_endpoint_decoders_setup(port); > - if (rc) > - return rc; > - > - /* > - * Now that all endpoint decoders are successfully enumerated, try to > - * assemble regions from committed decoders > - */ > - device_for_each_child(&port->dev, NULL, discover_region); > - > - return 0; > + return devm_cxl_endpoint_decoders_setup(port); > } > > static int cxl_port_probe(struct device *dev) ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling 2026-01-26 22:48 ` Dave Jiang @ 2026-01-27 23:45 ` Dave Jiang 2026-03-06 10:22 ` Neeraj Kumar 0 siblings, 1 reply; 35+ messages in thread From: Dave Jiang @ 2026-01-27 23:45 UTC (permalink / raw) To: Neeraj Kumar, linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel On 1/26/26 3:48 PM, Dave Jiang wrote: > > > On 1/23/26 4:31 AM, Neeraj Kumar wrote: >> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called >> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly >> used to get called at last in cxl_endpoint_port_probe(), which requires >> cxl_nvd presence. >> >> For cxl region persistency, region creation happens during nvdimm_probe >> which need the completion of endpoint probe. >> >> In order to accommodate both cxl pmem region auto-assembly and cxl region >> persistency, refactored following >> >> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This >> will be called only after successful completion of endpoint probe. >> >> 2. Create cxl_region_discovery() which performs pmem region >> auto-assembly and remove cxl pmem region auto-assembly from >> cxl_endpoint_port_probe() >> >> 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets >> called during cxl_pci_probe() in context of cxl_mem_probe() >> >> 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so >> move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees >> both the completion of endpoint probe and cxl_nvd presence before >> calling cxlmd->attach->probe(). >> >> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> >> Reviewed-by: Dave Jiang <dave.jiang@intel.com> >> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> > > Hi Neeraj, > Just FYI this is the patch that breaks the auto-region assemble in cxl_test. So it's missing this part in cxl_test. cxl_test auto region is now showing up. But now I'm hitting some lockdep issues. --- diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index cb87e8c0e63c..03af15edd988 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1686,6 +1686,7 @@ static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) static int cxl_mock_mem_probe(struct platform_device *pdev) { + struct cxl_memdev_attach memdev_attach = { 0 }; struct device *dev = &pdev->dev; struct cxl_memdev *cxlmd; struct cxl_memdev_state *mds; @@ -1767,7 +1768,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) cxl_mock_add_event_logs(&mdata->mes); - cxlmd = devm_cxl_add_memdev(cxlds, NULL); + memdev_attach.probe = cxl_region_discovery; + cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); --- > > DJ > >> --- >> drivers/cxl/core/region.c | 37 +++++++++++++++++++++++++++++++++++++ >> drivers/cxl/cxl.h | 5 +++++ >> drivers/cxl/mem.c | 18 +++++++++--------- >> drivers/cxl/pci.c | 4 +++- >> drivers/cxl/port.c | 39 +-------------------------------------- >> 5 files changed, 55 insertions(+), 48 deletions(-) >> >> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c >> index ae899f68551f..26238fb5e8cf 100644 >> --- a/drivers/cxl/core/region.c >> +++ b/drivers/cxl/core/region.c >> @@ -3727,6 +3727,43 @@ int cxl_add_to_region(struct cxl_endpoint_decoder *cxled) >> } >> EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL"); >> >> +static int discover_region(struct device *dev, void *unused) >> +{ >> + struct cxl_endpoint_decoder *cxled; >> + int rc; >> + >> + if (!is_endpoint_decoder(dev)) >> + return 0; >> + >> + cxled = to_cxl_endpoint_decoder(dev); >> + if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) >> + return 0; >> + >> + if (cxled->state != CXL_DECODER_STATE_AUTO) >> + return 0; >> + >> + /* >> + * Region enumeration is opportunistic, if this add-event fails, >> + * continue to the next endpoint decoder. >> + */ >> + rc = cxl_add_to_region(cxled); >> + if (rc) >> + dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", >> + cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); >> + >> + return 0; >> +} >> + >> +int cxl_region_discovery(struct cxl_memdev *cxlmd) >> +{ >> + struct cxl_port *port = cxlmd->endpoint; >> + >> + device_for_each_child(&port->dev, NULL, discover_region); >> + >> + return 0; >> +} >> +EXPORT_SYMBOL_NS_GPL(cxl_region_discovery, "CXL"); >> + >> u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa) >> { >> struct cxl_region_ref *iter; >> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h >> index c796c3db36e0..86efcc4fb963 100644 >> --- a/drivers/cxl/cxl.h >> +++ b/drivers/cxl/cxl.h >> @@ -906,6 +906,7 @@ struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); >> int cxl_add_to_region(struct cxl_endpoint_decoder *cxled); >> struct cxl_dax_region *to_cxl_dax_region(struct device *dev); >> u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa); >> +int cxl_region_discovery(struct cxl_memdev *cxlmd); >> #else >> static inline bool is_cxl_pmem_region(struct device *dev) >> { >> @@ -928,6 +929,10 @@ static inline u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, >> { >> return 0; >> } >> +static inline int cxl_region_discovery(struct cxl_memdev *cxlmd) >> +{ >> + return 0; >> +} >> #endif >> >> void cxl_endpoint_parse_cdat(struct cxl_port *port); >> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c >> index 333c366b69e7..7d19528d9b55 100644 >> --- a/drivers/cxl/mem.c >> +++ b/drivers/cxl/mem.c >> @@ -114,15 +114,6 @@ static int cxl_mem_probe(struct device *dev) >> return -ENXIO; >> } >> >> - if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { >> - rc = devm_cxl_add_nvdimm(parent_port, cxlmd); >> - if (rc) { >> - if (rc == -ENODEV) >> - dev_info(dev, "PMEM disabled by platform\n"); >> - return rc; >> - } >> - } >> - >> if (dport->rch) >> endpoint_parent = parent_port->uport_dev; >> else >> @@ -142,6 +133,15 @@ static int cxl_mem_probe(struct device *dev) >> return rc; >> } >> >> + if (cxl_pmem_size(cxlds) && IS_ENABLED(CONFIG_CXL_PMEM)) { >> + rc = devm_cxl_add_nvdimm(parent_port, cxlmd); >> + if (rc) { >> + if (rc == -ENODEV) >> + dev_info(dev, "PMEM disabled by platform\n"); >> + return rc; >> + } >> + } >> + >> if (cxlmd->attach) { >> rc = cxlmd->attach->probe(cxlmd); >> if (rc) >> diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c >> index 549368a9c868..70b40a10be7a 100644 >> --- a/drivers/cxl/pci.c >> +++ b/drivers/cxl/pci.c >> @@ -903,6 +903,7 @@ __ATTRIBUTE_GROUPS(cxl_rcd); >> static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) >> { >> struct pci_host_bridge *host_bridge = pci_find_host_bridge(pdev->bus); >> + struct cxl_memdev_attach memdev_attach = { 0 }; >> struct cxl_dpa_info range_info = { 0 }; >> struct cxl_memdev_state *mds; >> struct cxl_dev_state *cxlds; >> @@ -1006,7 +1007,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) >> if (rc) >> dev_dbg(&pdev->dev, "No CXL Features discovered\n"); >> >> - cxlmd = devm_cxl_add_memdev(cxlds, NULL); >> + memdev_attach.probe = cxl_region_discovery; >> + cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); >> if (IS_ERR(cxlmd)) >> return PTR_ERR(cxlmd); >> >> diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c >> index 7937e7e53797..fbeff1978bfb 100644 >> --- a/drivers/cxl/port.c >> +++ b/drivers/cxl/port.c >> @@ -30,33 +30,6 @@ static void schedule_detach(void *cxlmd) >> schedule_cxl_memdev_detach(cxlmd); >> } >> >> -static int discover_region(struct device *dev, void *unused) >> -{ >> - struct cxl_endpoint_decoder *cxled; >> - int rc; >> - >> - if (!is_endpoint_decoder(dev)) >> - return 0; >> - >> - cxled = to_cxl_endpoint_decoder(dev); >> - if ((cxled->cxld.flags & CXL_DECODER_F_ENABLE) == 0) >> - return 0; >> - >> - if (cxled->state != CXL_DECODER_STATE_AUTO) >> - return 0; >> - >> - /* >> - * Region enumeration is opportunistic, if this add-event fails, >> - * continue to the next endpoint decoder. >> - */ >> - rc = cxl_add_to_region(cxled); >> - if (rc) >> - dev_dbg(dev, "failed to add to region: %#llx-%#llx\n", >> - cxled->cxld.hpa_range.start, cxled->cxld.hpa_range.end); >> - >> - return 0; >> -} >> - >> static int cxl_switch_port_probe(struct cxl_port *port) >> { >> /* Reset nr_dports for rebind of driver */ >> @@ -82,17 +55,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) >> if (rc) >> return rc; >> >> - rc = devm_cxl_endpoint_decoders_setup(port); >> - if (rc) >> - return rc; >> - >> - /* >> - * Now that all endpoint decoders are successfully enumerated, try to >> - * assemble regions from committed decoders >> - */ >> - device_for_each_child(&port->dev, NULL, discover_region); >> - >> - return 0; >> + return devm_cxl_endpoint_decoders_setup(port); >> } >> >> static int cxl_port_probe(struct device *dev) > > ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling 2026-01-27 23:45 ` Dave Jiang @ 2026-03-06 10:22 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-03-06 10:22 UTC (permalink / raw) To: Dave Jiang Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs [-- Attachment #1: Type: text/plain, Size: 3099 bytes --] On 27/01/26 04:45PM, Dave Jiang wrote: > > >On 1/26/26 3:48 PM, Dave Jiang wrote: >> >> >> On 1/23/26 4:31 AM, Neeraj Kumar wrote: >>> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called >>> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly >>> used to get called at last in cxl_endpoint_port_probe(), which requires >>> cxl_nvd presence. >>> >>> For cxl region persistency, region creation happens during nvdimm_probe >>> which need the completion of endpoint probe. >>> >>> In order to accommodate both cxl pmem region auto-assembly and cxl region >>> persistency, refactored following >>> >>> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This >>> will be called only after successful completion of endpoint probe. >>> >>> 2. Create cxl_region_discovery() which performs pmem region >>> auto-assembly and remove cxl pmem region auto-assembly from >>> cxl_endpoint_port_probe() >>> >>> 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets >>> called during cxl_pci_probe() in context of cxl_mem_probe() >>> >>> 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so >>> move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees >>> both the completion of endpoint probe and cxl_nvd presence before >>> calling cxlmd->attach->probe(). >>> >>> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> >>> Reviewed-by: Dave Jiang <dave.jiang@intel.com> >>> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> >> >> Hi Neeraj, >> Just FYI this is the patch that breaks the auto-region assemble in cxl_test. > >So it's missing this part in cxl_test. cxl_test auto region is now showing up. But now I'm hitting some lockdep issues. > >--- >diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c >index cb87e8c0e63c..03af15edd988 100644 >--- a/tools/testing/cxl/test/mem.c >+++ b/tools/testing/cxl/test/mem.c >@@ -1686,6 +1686,7 @@ static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) > > static int cxl_mock_mem_probe(struct platform_device *pdev) > { >+ struct cxl_memdev_attach memdev_attach = { 0 }; > struct device *dev = &pdev->dev; > struct cxl_memdev *cxlmd; > struct cxl_memdev_state *mds; >@@ -1767,7 +1768,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) > > cxl_mock_add_event_logs(&mdata->mes); > >- cxlmd = devm_cxl_add_memdev(cxlds, NULL); >+ memdev_attach.probe = cxl_region_discovery; >+ cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); > if (IS_ERR(cxlmd)) > return PTR_ERR(cxlmd); > >--- > > >> >> DJ Hi Dave, Yes using the above snippet auto region failure is getting fixed. But still with above fix cxl-topology.sh is failing. Actually we have a test case in cxl-topology.sh where we disable mem device and check its availability after enabling it. And this test case is failing in this case. I am working on multi interleave support. I will test this with next series and fix it accordingly. Regards, Neeraj [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling 2026-01-23 11:31 ` [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling Neeraj Kumar 2026-01-26 22:48 ` Dave Jiang @ 2026-01-28 0:02 ` dan.j.williams 2026-03-06 10:18 ` Neeraj Kumar 1 sibling, 1 reply; 35+ messages in thread From: dan.j.williams @ 2026-01-28 0:02 UTC (permalink / raw) To: Neeraj Kumar, linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Neeraj Kumar wrote: > In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called > before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly > used to get called at last in cxl_endpoint_port_probe(), which requires > cxl_nvd presence. What? > For cxl region persistency, region creation happens during nvdimm_probe > which need the completion of endpoint probe. > > In order to accommodate both cxl pmem region auto-assembly and cxl region > persistency, refactored following > > 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This > will be called only after successful completion of endpoint probe. > > 2. Create cxl_region_discovery() which performs pmem region > auto-assembly and remove cxl pmem region auto-assembly from > cxl_endpoint_port_probe() > > 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets > called during cxl_pci_probe() in context of cxl_mem_probe() > > 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so > move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees > both the completion of endpoint probe and cxl_nvd presence before > calling cxlmd->attach->probe(). This does not make sense. The whole point of having devm_cxl_add_nvdimm() before devm_cxl_add_endpoint() is so that the typical region discovery path can consider pre-existing decoder settings *or* nvdimm labels in its assembly decisions. I would be surprised if this passes existing region assembly and ordering tests. This reads like "do not understand current ordering, change it for thin reasons". ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling 2026-01-28 0:02 ` dan.j.williams @ 2026-03-06 10:18 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-03-06 10:18 UTC (permalink / raw) To: dan.j.williams Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs [-- Attachment #1: Type: text/plain, Size: 4018 bytes --] On 27/01/26 04:02PM, dan.j.williams@intel.com wrote: >Neeraj Kumar wrote: >> In 84ec985944ef3, devm_cxl_add_nvdimm() sequence was changed and called >> before devm_cxl_add_endpoint(). It's because cxl pmem region auto-assembly >> used to get called at last in cxl_endpoint_port_probe(), which requires >> cxl_nvd presence. > >What? Hi Dan, Auto-region assembly was added by a32320b71f085 [1] and during that time devm_cxl_add_nvdimm() was called after devm_cxl_add_endpoint(). Later Li Ming found issue (kernel panic) in pmem region auto-assembly and fixed it using 84ec985944ef3 [2]. During this fix devm_cxl_add_nvdimm() sequence was changed and called before devm_cxl_add_endpoint() In this patch-set I have tried to bring back the original sequence in order to create pmem region (single interleave) based of parsed information from LSA during nvdimm_probe() [1] https://lore.kernel.org/r/167601999958.1924368.9366954455835735048.stgit@dwillia2-xfh.jf.intel.com [2] https://lore.kernel.org/all/20240612064423.2567625-1-ming4.li@intel.com > >> For cxl region persistency, region creation happens during nvdimm_probe >> which need the completion of endpoint probe. >> >> In order to accommodate both cxl pmem region auto-assembly and cxl region >> persistency, refactored following >> >> 1. Re-Sequence devm_cxl_add_nvdimm() after devm_cxl_add_endpoint(). This >> will be called only after successful completion of endpoint probe. >> >> 2. Create cxl_region_discovery() which performs pmem region >> auto-assembly and remove cxl pmem region auto-assembly from >> cxl_endpoint_port_probe() >> >> 3. Register cxl_region_discovery() with devm_cxl_add_memdev() which gets >> called during cxl_pci_probe() in context of cxl_mem_probe() >> >> 4. As cxlmd->attach->probe() calls registered cxl_region_discovery(), so >> move devm_cxl_add_nvdimm() before cxlmd->attach->probe(). It guarantees >> both the completion of endpoint probe and cxl_nvd presence before >> calling cxlmd->attach->probe(). > >This does not make sense. The whole point of having >devm_cxl_add_nvdimm() before devm_cxl_add_endpoint() is so that the >typical region discovery path can consider pre-existing decoder settings >*or* nvdimm labels in its assembly decisions. > >I would be surprised if this passes existing region assembly and >ordering tests. Actually using following change (as Dave has also checked) with the current patch-set is making existing region assembly ordering test pass. But still some other testcase in cxl_test cxl-topology.sh is failing. --- diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index cb87e8c0e63c..03af15edd988 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1686,6 +1686,7 @@ static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) static int cxl_mock_mem_probe(struct platform_device *pdev) { + struct cxl_memdev_attach memdev_attach = { 0 }; struct device *dev = &pdev->dev; struct cxl_memdev *cxlmd; struct cxl_memdev_state *mds; @@ -1767,7 +1768,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) cxl_mock_add_event_logs(&mdata->mes); - cxlmd = devm_cxl_add_memdev(cxlds, NULL); + memdev_attach.probe = cxl_region_discovery; + cxlmd = devm_cxl_add_memdev(cxlds, &memdev_attach); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); --- > >This reads like "do not understand current ordering, change it for thin >reasons". Yes I got your concern that we should reuse the existing infra of region auto assembly instead of doing this refactoring and creating new infra for region creation based on information parsed from LSA. Even the same is discussed in last CXL Collab Sync [1]. I will fix this refactoring along with multi interleave support using existing auto region assembly infra in next series. [1] https://pmem.io/ndctl/collab/#:~:text=LSA%202.1%20support,with%20first%20merge%3F Regards, Neeraj [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113140epcas5p4bc7a864d4a717b96afcb863e3200b3a2@epcas5p4.samsung.com>]
* [PATCH V6 11/18] cxl/region: Rename __create_region() to cxl_create_region() [not found] ` <CGME20260123113140epcas5p4bc7a864d4a717b96afcb863e3200b3a2@epcas5p4.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 2026-01-23 12:37 ` Jonathan Cameron 0 siblings, 1 reply; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Currently __create_region() is a static routine used within region.c So to use it from another file rename it to cxl_create_region(). Later patch will create cxl region after fetching region information from LSA using cxl_create_region() along with two extra function parameters. Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/core/core.h | 12 ++++++++++++ drivers/cxl/core/region.c | 13 ++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 1fb66132b777..268f6d19ab9d 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -42,6 +42,10 @@ int cxl_get_poison_by_endpoint(struct cxl_port *port); struct cxl_region *cxl_dpa_to_region(const struct cxl_memdev *cxlmd, u64 dpa); u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, const struct cxl_memdev *cxlmd, u64 dpa); +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, + enum cxl_partition_mode mode, int id, + struct cxl_pmem_region_params *pmem_params, + struct cxl_endpoint_decoder *cxled); #else static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, @@ -71,6 +75,14 @@ static inline int cxl_region_init(void) static inline void cxl_region_exit(void) { } +static inline struct cxl_region * +cxl_create_region(struct cxl_root_decoder *cxlrd, + enum cxl_partition_mode mode, int id, + struct cxl_pmem_region_params *params, + struct cxl_endpoint_decoder *cxled) +{ + return ERR_PTR(-EOPNOTSUPP); +} #define CXL_REGION_ATTR(x) NULL #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 26238fb5e8cf..2e60e5e72551 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2638,8 +2638,10 @@ static ssize_t create_ram_region_show(struct device *dev, return __create_region_show(to_cxl_root_decoder(dev), buf); } -static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, - enum cxl_partition_mode mode, int id) +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, + enum cxl_partition_mode mode, int id, + struct cxl_pmem_region_params *pmem_params, + struct cxl_endpoint_decoder *cxled) { int rc; @@ -2675,7 +2677,7 @@ static ssize_t create_region_store(struct device *dev, const char *buf, if (rc != 1) return -EINVAL; - cxlr = __create_region(cxlrd, mode, id); + cxlr = cxl_create_region(cxlrd, mode, id, NULL, NULL); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); @@ -3644,8 +3646,9 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, struct cxl_region *cxlr; do { - cxlr = __create_region(cxlrd, cxlds->part[part].mode, - atomic_read(&cxlrd->region_id)); + cxlr = cxl_create_region(cxlrd, cxlds->part[part].mode, + atomic_read(&cxlrd->region_id), + NULL, NULL); } while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY); if (IS_ERR(cxlr)) { -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 11/18] cxl/region: Rename __create_region() to cxl_create_region() 2026-01-23 11:31 ` [PATCH V6 11/18] cxl/region: Rename __create_region() to cxl_create_region() Neeraj Kumar @ 2026-01-23 12:37 ` Jonathan Cameron 0 siblings, 0 replies; 35+ messages in thread From: Jonathan Cameron @ 2026-01-23 12:37 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel On Fri, 23 Jan 2026 17:01:05 +0530 Neeraj Kumar <s.neeraj@samsung.com> wrote: > Currently __create_region() is a static routine used within region.c > So to use it from another file rename it to cxl_create_region(). > > Later patch will create cxl region after fetching region information > from LSA using cxl_create_region() along with two extra function > parameters. > > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113141epcas5p49a1eebff4401a7fc98381358162fde2b@epcas5p4.samsung.com>]
* [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation [not found] ` <CGME20260123113141epcas5p49a1eebff4401a7fc98381358162fde2b@epcas5p4.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 2026-01-23 12:39 ` Jonathan Cameron 2026-02-24 18:13 ` Alejandro Lucero Palau 0 siblings, 2 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar devm_cxl_pmem_add_region() is used to create cxl region based on region information scanned from LSA. devm_cxl_add_region() is used to just allocate cxlr and its fields are filled later by userspace tool using device attributes (*_store()). Inspiration for devm_cxl_pmem_add_region() is taken from these device attributes (_store*) calls. It allocates cxlr and fills information parsed from LSA and calls device_add(&cxlr->dev) to initiate further region creation porbes Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/core/region.c | 118 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 2e60e5e72551..e384eacc46ae 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2621,6 +2621,121 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(rc); } +static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size) +{ + int rc; + + if (!size) + return -EINVAL; + + ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region); + if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))) + return rc; + + return alloc_hpa(cxlr, size); +} + +static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size) +{ + if (!size) + return -EINVAL; + + if (!IS_ALIGNED(size, SZ_256M)) + return -EINVAL; + + return cxl_dpa_alloc(cxled, size); +} + +static struct cxl_region * +cxl_pmem_region_prep(struct cxl_root_decoder *cxlrd, int id, + struct cxl_pmem_region_params *params, + struct cxl_endpoint_decoder *cxled, + enum cxl_decoder_type type) +{ + struct cxl_region_params *p; + struct device *dev; + int rc; + + struct cxl_region *cxlr __free(put_cxl_region) = + cxl_region_alloc(cxlrd, id); + if (IS_ERR(cxlr)) + return cxlr; + + dev = &cxlr->dev; + rc = dev_set_name(dev, "region%d", id); + if (rc) + return ERR_PTR(rc); + + cxlr->mode = CXL_PARTMODE_PMEM; + cxlr->type = type; + + p = &cxlr->params; + p->uuid = params->uuid; + p->interleave_ways = params->nlabel; + p->interleave_granularity = params->ig; + + rc = alloc_region_hpa(cxlr, params->rawsize); + if (rc) + return ERR_PTR(rc); + + rc = cxl_dpa_set_part(cxled, CXL_PARTMODE_PMEM); + if (rc) + return ERR_PTR(rc); + + rc = alloc_region_dpa(cxled, params->rawsize); + if (rc) + return ERR_PTR(rc); + + /* + * TODO: Currently we have support of interleave_way == 1, where + * we can only have one region per mem device. It means mem device + * position (params->position) will always be 0. It is therefore + * attaching only one target at params->position + */ + if (params->position) + return ERR_PTR(-EOPNOTSUPP); + + rc = attach_target(cxlr, cxled, params->position, TASK_INTERRUPTIBLE); + if (rc) + return ERR_PTR(rc); + + rc = __commit(cxlr); + if (rc) + return ERR_PTR(rc); + + rc = device_add(dev); + if (rc) + return ERR_PTR(rc); + + return no_free_ptr(cxlr); +} + +static struct cxl_region * +devm_cxl_pmem_add_region(struct cxl_root_decoder *cxlrd, int id, + struct cxl_pmem_region_params *params, + struct cxl_endpoint_decoder *cxled, + enum cxl_decoder_type type) +{ + struct cxl_port *root_port; + struct cxl_region *cxlr; + int rc; + + cxlr = cxl_pmem_region_prep(cxlrd, id, params, cxled, type); + if (IS_ERR(cxlr)) + return cxlr; + + root_port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent); + rc = devm_add_action_or_reset(root_port->uport_dev, + unregister_region, cxlr); + if (rc) + return ERR_PTR(rc); + + dev_dbg(root_port->uport_dev, "%s: created %s\n", + dev_name(&cxlrd->cxlsd.cxld.dev), dev_name(&cxlr->dev)); + + return cxlr; +} + static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) { return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); @@ -2663,6 +2778,9 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } + if (pmem_params) + return devm_cxl_pmem_add_region(cxlrd, id, pmem_params, cxled, + CXL_DECODER_HOSTONLYMEM); return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM); } -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation 2026-01-23 11:31 ` [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation Neeraj Kumar @ 2026-01-23 12:39 ` Jonathan Cameron 2026-02-24 18:13 ` Alejandro Lucero Palau 1 sibling, 0 replies; 35+ messages in thread From: Jonathan Cameron @ 2026-01-23 12:39 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel On Fri, 23 Jan 2026 17:01:06 +0530 Neeraj Kumar <s.neeraj@samsung.com> wrote: > devm_cxl_pmem_add_region() is used to create cxl region based on region > information scanned from LSA. > > devm_cxl_add_region() is used to just allocate cxlr and its fields are > filled later by userspace tool using device attributes (*_store()). > > Inspiration for devm_cxl_pmem_add_region() is taken from these device > attributes (_store*) calls. It allocates cxlr and fills information > parsed from LSA and calls device_add(&cxlr->dev) to initiate further > region creation porbes > > Reviewed-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation 2026-01-23 11:31 ` [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation Neeraj Kumar 2026-01-23 12:39 ` Jonathan Cameron @ 2026-02-24 18:13 ` Alejandro Lucero Palau 2026-03-06 10:24 ` Neeraj Kumar 1 sibling, 1 reply; 35+ messages in thread From: Alejandro Lucero Palau @ 2026-02-24 18:13 UTC (permalink / raw) To: Neeraj Kumar, linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel Hi Neeraj, I could get some free time for reviewing this series. Dan pointed out to potential conflicts with my Type2 series, so I'm focused on those patches modifying the cxl region creation and how it is being used. I do not know if you are aware of the Type2 work, although I dare to say you are not since some of the functionality is duplicated ... and in your case skipping some steps. Below my comments. On 1/23/26 11:31, Neeraj Kumar wrote: > devm_cxl_pmem_add_region() is used to create cxl region based on region > information scanned from LSA. > > devm_cxl_add_region() is used to just allocate cxlr and its fields are > filled later by userspace tool using device attributes (*_store()). > > Inspiration for devm_cxl_pmem_add_region() is taken from these device > attributes (_store*) calls. It allocates cxlr and fills information > parsed from LSA and calls device_add(&cxlr->dev) to initiate further > region creation porbes > > Reviewed-by: Dave Jiang <dave.jiang@intel.com> > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> > --- > drivers/cxl/core/region.c | 118 ++++++++++++++++++++++++++++++++++++++ > 1 file changed, 118 insertions(+) > > diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c > index 2e60e5e72551..e384eacc46ae 100644 > --- a/drivers/cxl/core/region.c > +++ b/drivers/cxl/core/region.c > @@ -2621,6 +2621,121 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, > return ERR_PTR(rc); > } > > +static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size) > +{ > + int rc; > + > + if (!size) > + return -EINVAL; > + > + ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region); > + if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))) > + return rc; > + > + return alloc_hpa(cxlr, size); > +} Type2 has a more elaborated function preceding the final hpa allocation. First of all, the root decoder needs to match the endpoint type or to have support for it. Then interleave ways taken into account. I know you are only supporting 1 way, what I guess makes the initial support easier, but although some check for not supporting > 1 is fine (I guess this takes a lot of work for matching at least LSA labels and maybe something harder), the core code should support that, mainly because there is ongoing work adding it. In my case I did not need interleaving and it is unlikely other Type2 devices will need it in the near future, but the support is there: https://lore.kernel.org/linux-cxl/20260201155438.2664640-1-alejandro.lucero-palau@amd.com/T/#m56bf70b58bb995082680bf363fd3f6d6f2b074d2 > + > +static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size) > +{ > + if (!size) > + return -EINVAL; > + > + if (!IS_ALIGNED(size, SZ_256M)) > + return -EINVAL; > + > + return cxl_dpa_alloc(cxled, size); > +} I'm not sure if the same applies here as you are passing an endpoint decoder already, but FWIW: https://lore.kernel.org/linux-cxl/20260201155438.2664640-1-alejandro.lucero-palau@amd.com/T/#m38ff266e931fd9c932bc54d000b7ea72186493be I'm sending type2 individual patches for facilitating the integration, and I'll focus next on these two referenced ones so you could use them asap. Thank you, Alejandro > + > +static struct cxl_region * > +cxl_pmem_region_prep(struct cxl_root_decoder *cxlrd, int id, > + struct cxl_pmem_region_params *params, > + struct cxl_endpoint_decoder *cxled, > + enum cxl_decoder_type type) > +{ > + struct cxl_region_params *p; > + struct device *dev; > + int rc; > + > + struct cxl_region *cxlr __free(put_cxl_region) = > + cxl_region_alloc(cxlrd, id); > + if (IS_ERR(cxlr)) > + return cxlr; > + > + dev = &cxlr->dev; > + rc = dev_set_name(dev, "region%d", id); > + if (rc) > + return ERR_PTR(rc); > + > + cxlr->mode = CXL_PARTMODE_PMEM; > + cxlr->type = type; > + > + p = &cxlr->params; > + p->uuid = params->uuid; > + p->interleave_ways = params->nlabel; > + p->interleave_granularity = params->ig; > + > + rc = alloc_region_hpa(cxlr, params->rawsize); > + if (rc) > + return ERR_PTR(rc); > + > + rc = cxl_dpa_set_part(cxled, CXL_PARTMODE_PMEM); > + if (rc) > + return ERR_PTR(rc); > + > + rc = alloc_region_dpa(cxled, params->rawsize); > + if (rc) > + return ERR_PTR(rc); > + > + /* > + * TODO: Currently we have support of interleave_way == 1, where > + * we can only have one region per mem device. It means mem device > + * position (params->position) will always be 0. It is therefore > + * attaching only one target at params->position > + */ > + if (params->position) > + return ERR_PTR(-EOPNOTSUPP); > + > + rc = attach_target(cxlr, cxled, params->position, TASK_INTERRUPTIBLE); > + if (rc) > + return ERR_PTR(rc); > + > + rc = __commit(cxlr); > + if (rc) > + return ERR_PTR(rc); > + > + rc = device_add(dev); > + if (rc) > + return ERR_PTR(rc); > + > + return no_free_ptr(cxlr); > +} > + > +static struct cxl_region * > +devm_cxl_pmem_add_region(struct cxl_root_decoder *cxlrd, int id, > + struct cxl_pmem_region_params *params, > + struct cxl_endpoint_decoder *cxled, > + enum cxl_decoder_type type) > +{ > + struct cxl_port *root_port; > + struct cxl_region *cxlr; > + int rc; > + > + cxlr = cxl_pmem_region_prep(cxlrd, id, params, cxled, type); > + if (IS_ERR(cxlr)) > + return cxlr; > + > + root_port = to_cxl_port(cxlrd->cxlsd.cxld.dev.parent); > + rc = devm_add_action_or_reset(root_port->uport_dev, > + unregister_region, cxlr); > + if (rc) > + return ERR_PTR(rc); > + > + dev_dbg(root_port->uport_dev, "%s: created %s\n", > + dev_name(&cxlrd->cxlsd.cxld.dev), dev_name(&cxlr->dev)); > + > + return cxlr; > +} > + > static ssize_t __create_region_show(struct cxl_root_decoder *cxlrd, char *buf) > { > return sysfs_emit(buf, "region%u\n", atomic_read(&cxlrd->region_id)); > @@ -2663,6 +2778,9 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, > return ERR_PTR(-EBUSY); > } > > + if (pmem_params) > + return devm_cxl_pmem_add_region(cxlrd, id, pmem_params, cxled, > + CXL_DECODER_HOSTONLYMEM); > return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM); > } > ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation 2026-02-24 18:13 ` Alejandro Lucero Palau @ 2026-03-06 10:24 ` Neeraj Kumar 2026-03-06 16:28 ` Alejandro Lucero Palau 0 siblings, 1 reply; 35+ messages in thread From: Neeraj Kumar @ 2026-03-06 10:24 UTC (permalink / raw) To: Alejandro Lucero Palau Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs [-- Attachment #1: Type: text/plain, Size: 3929 bytes --] On 24/02/26 06:13PM, Alejandro Lucero Palau wrote: >Hi Neeraj, > > >I could get some free time for reviewing this series. Dan pointed out >to potential conflicts with my Type2 series, so I'm focused on those >patches modifying the cxl region creation and how it is being used. > > >I do not know if you are aware of the Type2 work, although I dare to >say you are not since some of the functionality is duplicated ... and >in your case skipping some steps. > Hi Alejandro, Thanks for your review. Yes I have not looked at Type2 work in detail. I will have a look and specially the conflicting part with my changes. > >Below my comments. > > >On 1/23/26 11:31, Neeraj Kumar wrote: >>devm_cxl_pmem_add_region() is used to create cxl region based on region >>information scanned from LSA. >> >>devm_cxl_add_region() is used to just allocate cxlr and its fields are >>filled later by userspace tool using device attributes (*_store()). >> >>Inspiration for devm_cxl_pmem_add_region() is taken from these device >>attributes (_store*) calls. It allocates cxlr and fills information >>parsed from LSA and calls device_add(&cxlr->dev) to initiate further >>region creation porbes >> >>Reviewed-by: Dave Jiang <dave.jiang@intel.com> >>Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> >>--- >> drivers/cxl/core/region.c | 118 ++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 118 insertions(+) >> >>diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c >>index 2e60e5e72551..e384eacc46ae 100644 >>--- a/drivers/cxl/core/region.c >>+++ b/drivers/cxl/core/region.c >>@@ -2621,6 +2621,121 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, >> return ERR_PTR(rc); >> } >>+static ssize_t alloc_region_hpa(struct cxl_region *cxlr, u64 size) >>+{ >>+ int rc; >>+ >>+ if (!size) >>+ return -EINVAL; >>+ >>+ ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region); >>+ if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))) >>+ return rc; >>+ >>+ return alloc_hpa(cxlr, size); >>+} > > >Type2 has a more elaborated function preceding the final hpa >allocation. First of all, the root decoder needs to match the >endpoint type or to have support for it. Then interleave ways taken >into account. I know you are only supporting 1 way, what I guess makes >the initial support easier, but although some check for not supporting >> 1 is fine (I guess this takes a lot of work for matching at least >LSA labels and maybe something harder), the core code should support >that, mainly because there is ongoing work adding it. In my case I did >not need interleaving and it is unlikely other Type2 devices will need >it in the near future, but the support is there: > >https://lore.kernel.org/linux-cxl/20260201155438.2664640-1-alejandro.lucero-palau@amd.com/T/#m56bf70b58bb995082680bf363fd3f6d6f2b074d2 > > >>+ >>+static ssize_t alloc_region_dpa(struct cxl_endpoint_decoder *cxled, u64 size) >>+{ >>+ if (!size) >>+ return -EINVAL; >>+ >>+ if (!IS_ALIGNED(size, SZ_256M)) >>+ return -EINVAL; >>+ >>+ return cxl_dpa_alloc(cxled, size); >>+} > > >I'm not sure if the same applies here as you are passing an endpoint >decoder already, but FWIW: > > >https://lore.kernel.org/linux-cxl/20260201155438.2664640-1-alejandro.lucero-palau@amd.com/T/#m38ff266e931fd9c932bc54d000b7ea72186493be > > >I'm sending type2 individual patches for facilitating the integration, >and I'll focus next on these two referenced ones so you could use them >asap. > > >Thank you, > >Alejandro Actually I have created alloc_region_hpa() and alloc_region_dpa() taking inspiration from device attributes (_store*) calls used to create cxl region using cxl userspace tool. I am adding the support of multi interleave, there these routines will not be required as I would be re-using the existing auto region assembly infra. Even though I will re-check any conflicts with Type2 changes. Regards, Neeraj [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation 2026-03-06 10:24 ` Neeraj Kumar @ 2026-03-06 16:28 ` Alejandro Lucero Palau 0 siblings, 0 replies; 35+ messages in thread From: Alejandro Lucero Palau @ 2026-03-06 16:28 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs <snip> On 3/6/26 10:24, Neeraj Kumar wrote: > > Actually I have created alloc_region_hpa() and alloc_region_dpa() taking > inspiration from device attributes (_store*) calls used to create cxl > region using cxl userspace tool. > > I am adding the support of multi interleave, there these routines will > not be required as I would be re-using the existing auto region > assembly infra. > > Even though I will re-check any conflicts with Type2 changes. > > I'm changing the current hpa_get_freespace based on this current need from pmem and some issues reported by Gregory regarding concurrency when several drivers try to get things done. The current implementation reports max free space and the allocation is done later on what with those concurrency issues seems to not be the right solution. I will post a new version early next week. ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113143epcas5p1f056205feaa11092be91be6f74ccc0be@epcas5p1.samsung.com>]
* [PATCH V6 13/18] cxl/pmem: Preserve region information into nd_set [not found] ` <CGME20260123113143epcas5p1f056205feaa11092be91be6f74ccc0be@epcas5p1.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Save region information stored in cxlr to nd_set during cxl_pmem_region_probe in nd_set. This saved region information is being stored into LSA, which will be used for cxl region persistence Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/pmem.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c index e197883690ef..a6eba3572090 100644 --- a/drivers/cxl/pmem.c +++ b/drivers/cxl/pmem.c @@ -377,6 +377,7 @@ static int cxl_pmem_region_probe(struct device *dev) struct nd_mapping_desc mappings[CXL_DECODER_MAX_INTERLEAVE]; struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); struct cxl_region *cxlr = cxlr_pmem->cxlr; + struct cxl_region_params *p = &cxlr->params; struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb; struct cxl_pmem_region_info *info = NULL; struct nd_interleave_set *nd_set; @@ -465,12 +466,12 @@ static int cxl_pmem_region_probe(struct device *dev) ndr_desc.num_mappings = cxlr_pmem->nr_mappings; ndr_desc.mapping = mappings; - /* - * TODO enable CXL labels which skip the need for 'interleave-set cookie' - */ - nd_set->cookie1 = - nd_fletcher64(info, sizeof(*info) * cxlr_pmem->nr_mappings, 0); - nd_set->cookie2 = nd_set->cookie1; + nd_set->uuid = p->uuid; + nd_set->interleave_ways = p->interleave_ways; + nd_set->interleave_granularity = p->interleave_granularity; + nd_set->res = p->res; + nd_set->nr_targets = p->nr_targets; + ndr_desc.nd_set = nd_set; cxlr_pmem->nd_region = -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113144epcas5p18477c68ede07c17a6c8461cf034eb3d1@epcas5p1.samsung.com>]
* [PATCH V6 14/18] cxl/pmem_region: Prep patch to accommodate pmem_region attributes [not found] ` <CGME20260123113144epcas5p18477c68ede07c17a6c8461cf034eb3d1@epcas5p1.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar For region label update, need to create device attribute, which calls nvdimm exported routine thus making pmem_region dependent on libnvdimm. Because of this dependency of pmem region on libnvdimm, segregate pmem region related code from core/region.c to core/pmem_region.c This patch has no functionality change. Its just code movement from core/region.c to core/pmem_region.c Reviewed-by: Dave Jiang <dave.jiang@intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/core.h | 10 ++ drivers/cxl/core/pmem_region.c | 201 +++++++++++++++++++++++++++++++++ drivers/cxl/core/region.c | 188 +----------------------------- tools/testing/cxl/Kbuild | 2 +- 5 files changed, 214 insertions(+), 189 deletions(-) create mode 100644 drivers/cxl/core/pmem_region.c diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 5ad8fef210b5..fe0fcab6d730 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -16,7 +16,7 @@ cxl_core-y += pmu.o cxl_core-y += cdat.o cxl_core-y += ras.o cxl_core-$(CONFIG_TRACING) += trace.o -cxl_core-$(CONFIG_CXL_REGION) += region.o +cxl_core-$(CONFIG_CXL_REGION) += region.o pmem_region.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 268f6d19ab9d..4eed243c0d7d 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -46,6 +46,8 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, enum cxl_partition_mode mode, int id, struct cxl_pmem_region_params *pmem_params, struct cxl_endpoint_decoder *cxled); +struct cxl_region *to_cxl_region(struct device *dev); +int devm_cxl_add_pmem_region(struct cxl_region *cxlr); #else static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, @@ -83,6 +85,14 @@ cxl_create_region(struct cxl_root_decoder *cxlrd, { return ERR_PTR(-EOPNOTSUPP); } +static inline struct cxl_region *to_cxl_region(struct device *dev) +{ + return NULL; +} +static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr) +{ + return 0; +} #define CXL_REGION_ATTR(x) NULL #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c new file mode 100644 index 000000000000..dcaab59108fd --- /dev/null +++ b/drivers/cxl/core/pmem_region.c @@ -0,0 +1,201 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2020 Intel Corporation. */ +#include <linux/device.h> +#include <linux/memregion.h> +#include <cxlmem.h> +#include <cxl.h> +#include "core.h" + +/** + * DOC: cxl pmem region + * + * The core CXL PMEM region infrastructure supports persistent memory + * region creation using LIBNVDIMM subsystem. It has dependency on + * LIBNVDIMM, pmem region needs to update the cxl region information + * in the LSA. + */ + +static void cxl_pmem_region_release(struct device *dev) +{ + struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); + int i; + + for (i = 0; i < cxlr_pmem->nr_mappings; i++) { + struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd; + + put_device(&cxlmd->dev); + } + + kfree(cxlr_pmem); +} + +static const struct attribute_group *cxl_pmem_region_attribute_groups[] = { + &cxl_base_attribute_group, + NULL +}; + +const struct device_type cxl_pmem_region_type = { + .name = "cxl_pmem_region", + .release = cxl_pmem_region_release, + .groups = cxl_pmem_region_attribute_groups, +}; + +bool is_cxl_pmem_region(struct device *dev) +{ + return dev->type == &cxl_pmem_region_type; +} +EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL"); + +struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) +{ + if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev), + "not a cxl_pmem_region device\n")) + return NULL; + return container_of(dev, struct cxl_pmem_region, dev); +} +EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL"); + +static struct lock_class_key cxl_pmem_region_key; + +static int cxl_pmem_region_alloc(struct cxl_region *cxlr) +{ + struct cxl_region_params *p = &cxlr->params; + struct cxl_nvdimm_bridge *cxl_nvb; + struct device *dev; + int i; + + guard(rwsem_read)(&cxl_rwsem.region); + if (p->state != CXL_CONFIG_COMMIT) + return -ENXIO; + + struct cxl_pmem_region *cxlr_pmem __free(kfree) = + kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets), + GFP_KERNEL); + if (!cxlr_pmem) + return -ENOMEM; + + cxlr_pmem->hpa_range.start = p->res->start; + cxlr_pmem->hpa_range.end = p->res->end; + + /* Snapshot the region configuration underneath the cxl_region_rwsem */ + cxlr_pmem->nr_mappings = p->nr_targets; + for (i = 0; i < p->nr_targets; i++) { + struct cxl_endpoint_decoder *cxled = p->targets[i]; + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i]; + + /* + * Regions never span CXL root devices, so by definition the + * bridge for one device is the same for all. + */ + if (i == 0) { + cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint); + if (!cxl_nvb) + return -ENODEV; + cxlr->cxl_nvb = cxl_nvb; + } + m->cxlmd = cxlmd; + get_device(&cxlmd->dev); + m->start = cxled->dpa_res->start; + m->size = resource_size(cxled->dpa_res); + m->position = i; + } + + dev = &cxlr_pmem->dev; + device_initialize(dev); + lockdep_set_class(&dev->mutex, &cxl_pmem_region_key); + device_set_pm_not_required(dev); + dev->parent = &cxlr->dev; + dev->bus = &cxl_bus_type; + dev->type = &cxl_pmem_region_type; + cxlr_pmem->cxlr = cxlr; + cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem); + + return 0; +} + +static void cxlr_pmem_unregister(void *_cxlr_pmem) +{ + struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem; + struct cxl_region *cxlr = cxlr_pmem->cxlr; + struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb; + + /* + * Either the bridge is in ->remove() context under the device_lock(), + * or cxlr_release_nvdimm() is cancelling the bridge's release action + * for @cxlr_pmem and doing it itself (while manually holding the bridge + * lock). + */ + device_lock_assert(&cxl_nvb->dev); + cxlr->cxlr_pmem = NULL; + cxlr_pmem->cxlr = NULL; + device_unregister(&cxlr_pmem->dev); +} + +static void cxlr_release_nvdimm(void *_cxlr) +{ + struct cxl_region *cxlr = _cxlr; + struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb; + + scoped_guard(device, &cxl_nvb->dev) { + if (cxlr->cxlr_pmem) + devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister, + cxlr->cxlr_pmem); + } + cxlr->cxl_nvb = NULL; + put_device(&cxl_nvb->dev); +} + +/** + * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge + * @cxlr: parent CXL region for this pmem region bridge device + * + * Return: 0 on success negative error code on failure. + */ +int devm_cxl_add_pmem_region(struct cxl_region *cxlr) +{ + struct cxl_pmem_region *cxlr_pmem; + struct cxl_nvdimm_bridge *cxl_nvb; + struct device *dev; + int rc; + + rc = cxl_pmem_region_alloc(cxlr); + if (rc) + return rc; + cxlr_pmem = cxlr->cxlr_pmem; + cxl_nvb = cxlr->cxl_nvb; + + dev = &cxlr_pmem->dev; + rc = dev_set_name(dev, "pmem_region%d", cxlr->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent), + dev_name(dev)); + + scoped_guard(device, &cxl_nvb->dev) { + if (cxl_nvb->dev.driver) + rc = devm_add_action_or_reset(&cxl_nvb->dev, + cxlr_pmem_unregister, + cxlr_pmem); + else + rc = -ENXIO; + } + + if (rc) + goto err_bridge; + + /* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */ + return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr); + +err: + put_device(dev); +err_bridge: + put_device(&cxl_nvb->dev); + cxlr->cxl_nvb = NULL; + return rc; +} diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e384eacc46ae..f9b3dd6cee50 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -38,8 +38,6 @@ */ static nodemask_t nodemask_region_seen = NODE_MASK_NONE; -static struct cxl_region *to_cxl_region(struct device *dev); - #define __ACCESS_ATTR_RO(_level, _name) { \ .attr = { .name = __stringify(_name), .mode = 0444 }, \ .show = _name##_access##_level##_show, \ @@ -2429,7 +2427,7 @@ bool is_cxl_region(struct device *dev) } EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL"); -static struct cxl_region *to_cxl_region(struct device *dev) +struct cxl_region *to_cxl_region(struct device *dev) { if (dev_WARN_ONCE(dev, dev->type != &cxl_region_type, "not a cxl_region device\n")) @@ -2866,46 +2864,6 @@ static ssize_t delete_region_store(struct device *dev, } DEVICE_ATTR_WO(delete_region); -static void cxl_pmem_region_release(struct device *dev) -{ - struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); - int i; - - for (i = 0; i < cxlr_pmem->nr_mappings; i++) { - struct cxl_memdev *cxlmd = cxlr_pmem->mapping[i].cxlmd; - - put_device(&cxlmd->dev); - } - - kfree(cxlr_pmem); -} - -static const struct attribute_group *cxl_pmem_region_attribute_groups[] = { - &cxl_base_attribute_group, - NULL, -}; - -const struct device_type cxl_pmem_region_type = { - .name = "cxl_pmem_region", - .release = cxl_pmem_region_release, - .groups = cxl_pmem_region_attribute_groups, -}; - -bool is_cxl_pmem_region(struct device *dev) -{ - return dev->type == &cxl_pmem_region_type; -} -EXPORT_SYMBOL_NS_GPL(is_cxl_pmem_region, "CXL"); - -struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) -{ - if (dev_WARN_ONCE(dev, !is_cxl_pmem_region(dev), - "not a cxl_pmem_region device\n")) - return NULL; - return container_of(dev, struct cxl_pmem_region, dev); -} -EXPORT_SYMBOL_NS_GPL(to_cxl_pmem_region, "CXL"); - struct cxl_poison_context { struct cxl_port *port; int part; @@ -3337,64 +3295,6 @@ static int region_offset_to_dpa_result(struct cxl_region *cxlr, u64 offset, return -ENXIO; } -static struct lock_class_key cxl_pmem_region_key; - -static int cxl_pmem_region_alloc(struct cxl_region *cxlr) -{ - struct cxl_region_params *p = &cxlr->params; - struct cxl_nvdimm_bridge *cxl_nvb; - struct device *dev; - int i; - - guard(rwsem_read)(&cxl_rwsem.region); - if (p->state != CXL_CONFIG_COMMIT) - return -ENXIO; - - struct cxl_pmem_region *cxlr_pmem __free(kfree) = - kzalloc(struct_size(cxlr_pmem, mapping, p->nr_targets), GFP_KERNEL); - if (!cxlr_pmem) - return -ENOMEM; - - cxlr_pmem->hpa_range.start = p->res->start; - cxlr_pmem->hpa_range.end = p->res->end; - - /* Snapshot the region configuration underneath the cxl_rwsem.region */ - cxlr_pmem->nr_mappings = p->nr_targets; - for (i = 0; i < p->nr_targets; i++) { - struct cxl_endpoint_decoder *cxled = p->targets[i]; - struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); - struct cxl_pmem_region_mapping *m = &cxlr_pmem->mapping[i]; - - /* - * Regions never span CXL root devices, so by definition the - * bridge for one device is the same for all. - */ - if (i == 0) { - cxl_nvb = cxl_find_nvdimm_bridge(cxlmd->endpoint); - if (!cxl_nvb) - return -ENODEV; - cxlr->cxl_nvb = cxl_nvb; - } - m->cxlmd = cxlmd; - get_device(&cxlmd->dev); - m->start = cxled->dpa_res->start; - m->size = resource_size(cxled->dpa_res); - m->position = i; - } - - dev = &cxlr_pmem->dev; - device_initialize(dev); - lockdep_set_class(&dev->mutex, &cxl_pmem_region_key); - device_set_pm_not_required(dev); - dev->parent = &cxlr->dev; - dev->bus = &cxl_bus_type; - dev->type = &cxl_pmem_region_type; - cxlr_pmem->cxlr = cxlr; - cxlr->cxlr_pmem = no_free_ptr(cxlr_pmem); - - return 0; -} - static void cxl_dax_region_release(struct device *dev) { struct cxl_dax_region *cxlr_dax = to_cxl_dax_region(dev); @@ -3458,92 +3358,6 @@ static struct cxl_dax_region *cxl_dax_region_alloc(struct cxl_region *cxlr) return cxlr_dax; } -static void cxlr_pmem_unregister(void *_cxlr_pmem) -{ - struct cxl_pmem_region *cxlr_pmem = _cxlr_pmem; - struct cxl_region *cxlr = cxlr_pmem->cxlr; - struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb; - - /* - * Either the bridge is in ->remove() context under the device_lock(), - * or cxlr_release_nvdimm() is cancelling the bridge's release action - * for @cxlr_pmem and doing it itself (while manually holding the bridge - * lock). - */ - device_lock_assert(&cxl_nvb->dev); - cxlr->cxlr_pmem = NULL; - cxlr_pmem->cxlr = NULL; - device_unregister(&cxlr_pmem->dev); -} - -static void cxlr_release_nvdimm(void *_cxlr) -{ - struct cxl_region *cxlr = _cxlr; - struct cxl_nvdimm_bridge *cxl_nvb = cxlr->cxl_nvb; - - scoped_guard(device, &cxl_nvb->dev) { - if (cxlr->cxlr_pmem) - devm_release_action(&cxl_nvb->dev, cxlr_pmem_unregister, - cxlr->cxlr_pmem); - } - cxlr->cxl_nvb = NULL; - put_device(&cxl_nvb->dev); -} - -/** - * devm_cxl_add_pmem_region() - add a cxl_region-to-nd_region bridge - * @cxlr: parent CXL region for this pmem region bridge device - * - * Return: 0 on success negative error code on failure. - */ -static int devm_cxl_add_pmem_region(struct cxl_region *cxlr) -{ - struct cxl_pmem_region *cxlr_pmem; - struct cxl_nvdimm_bridge *cxl_nvb; - struct device *dev; - int rc; - - rc = cxl_pmem_region_alloc(cxlr); - if (rc) - return rc; - cxlr_pmem = cxlr->cxlr_pmem; - cxl_nvb = cxlr->cxl_nvb; - - dev = &cxlr_pmem->dev; - rc = dev_set_name(dev, "pmem_region%d", cxlr->id); - if (rc) - goto err; - - rc = device_add(dev); - if (rc) - goto err; - - dev_dbg(&cxlr->dev, "%s: register %s\n", dev_name(dev->parent), - dev_name(dev)); - - scoped_guard(device, &cxl_nvb->dev) { - if (cxl_nvb->dev.driver) - rc = devm_add_action_or_reset(&cxl_nvb->dev, - cxlr_pmem_unregister, - cxlr_pmem); - else - rc = -ENXIO; - } - - if (rc) - goto err_bridge; - - /* @cxlr carries a reference on @cxl_nvb until cxlr_release_nvdimm */ - return devm_add_action_or_reset(&cxlr->dev, cxlr_release_nvdimm, cxlr); - -err: - put_device(dev); -err_bridge: - put_device(&cxl_nvb->dev); - cxlr->cxl_nvb = NULL; - return rc; -} - static void cxlr_dax_unregister(void *_cxlr_dax) { struct cxl_dax_region *cxlr_dax = _cxlr_dax; diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index 0e151d0572d1..ad2496b38fdd 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -59,7 +59,7 @@ cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o -cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o +cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o $(CXL_CORE_SRC)/pmem_region.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113146epcas5p3d267b8d1aed388debac761ebf5f143d4@epcas5p3.samsung.com>]
* [PATCH V6 15/18] cxl/pmem_region: Introduce CONFIG_CXL_PMEM_REGION for core/pmem_region.c [not found] ` <CGME20260123113146epcas5p3d267b8d1aed388debac761ebf5f143d4@epcas5p3.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar As pmem region label update/delete has hard dependency on libnvdimm. It is therefore put core/pmem_region.c under CONFIG_CXL_PMEM_REGION control. It handles the dependency by selecting CONFIG_LIBNVDIMM if not enabled. Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/Kconfig | 15 +++++++++++++++ drivers/cxl/core/Makefile | 3 ++- drivers/cxl/core/core.h | 18 +++++++++++------- drivers/cxl/cxl.h | 24 ++++++++++++++---------- tools/testing/cxl/Kbuild | 3 ++- 5 files changed, 44 insertions(+), 19 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index f1361ed6a0d4..307fed8f1f56 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -211,6 +211,21 @@ config CXL_REGION If unsure say 'y' +config CXL_PMEM_REGION + bool "CXL: Pmem Region Support" + default CXL_BUS + depends on CXL_REGION + depends on ARCH_HAS_PMEM_API + depends on PHYS_ADDR_T_64BIT + depends on BLK_DEV + select LIBNVDIMM + help + Enable the CXL core to enumerate and provision CXL pmem regions. + A CXL pmem region need to update region label into LSA. For LSA + update/delete libnvdimm is required. + + If unsure say 'y' + config CXL_REGION_INVALIDATION_TEST bool "CXL: Region Cache Management Bypass (TEST)" depends on CXL_REGION diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index fe0fcab6d730..399157beb917 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -16,7 +16,8 @@ cxl_core-y += pmu.o cxl_core-y += cdat.o cxl_core-y += ras.o cxl_core-$(CONFIG_TRACING) += trace.o -cxl_core-$(CONFIG_CXL_REGION) += region.o pmem_region.o +cxl_core-$(CONFIG_CXL_REGION) += region.o +cxl_core-$(CONFIG_CXL_PMEM_REGION) += pmem_region.o cxl_core-$(CONFIG_CXL_MCE) += mce.o cxl_core-$(CONFIG_CXL_FEATURES) += features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += edac.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 4eed243c0d7d..5ae693269771 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -34,7 +34,6 @@ int cxl_decoder_detach(struct cxl_region *cxlr, #define CXL_REGION_ATTR(x) (&dev_attr_##x.attr) #define CXL_REGION_TYPE(x) (&cxl_region_type) #define SET_CXL_REGION_ATTR(x) (&dev_attr_##x.attr), -#define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) #define CXL_DAX_REGION_TYPE(x) (&cxl_dax_region_type) int cxl_region_init(void); void cxl_region_exit(void); @@ -47,7 +46,6 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, struct cxl_pmem_region_params *pmem_params, struct cxl_endpoint_decoder *cxled); struct cxl_region *to_cxl_region(struct device *dev); -int devm_cxl_add_pmem_region(struct cxl_region *cxlr); #else static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, @@ -89,17 +87,23 @@ static inline struct cxl_region *to_cxl_region(struct device *dev) { return NULL; } -static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr) -{ - return 0; -} #define CXL_REGION_ATTR(x) NULL #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) -#define CXL_PMEM_REGION_TYPE(x) NULL #define CXL_DAX_REGION_TYPE(x) NULL #endif +#ifdef CONFIG_CXL_PMEM_REGION +#define CXL_PMEM_REGION_TYPE(x) (&cxl_pmem_region_type) +int devm_cxl_add_pmem_region(struct cxl_region *cxlr); +#else +#define CXL_PMEM_REGION_TYPE(x) NULL +static inline int devm_cxl_add_pmem_region(struct cxl_region *cxlr) +{ + return -EINVAL; +} +#endif + struct cxl_send_command; struct cxl_mem_query_commands; int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 86efcc4fb963..296411be1c36 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -901,21 +901,11 @@ int devm_cxl_add_nvdimm(struct cxl_port *parent_port, struct cxl_memdev *cxlmd); struct cxl_nvdimm_bridge *cxl_find_nvdimm_bridge(struct cxl_port *port); #ifdef CONFIG_CXL_REGION -bool is_cxl_pmem_region(struct device *dev); -struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); int cxl_add_to_region(struct cxl_endpoint_decoder *cxled); struct cxl_dax_region *to_cxl_dax_region(struct device *dev); u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa); int cxl_region_discovery(struct cxl_memdev *cxlmd); #else -static inline bool is_cxl_pmem_region(struct device *dev) -{ - return false; -} -static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) -{ - return NULL; -} static inline int cxl_add_to_region(struct cxl_endpoint_decoder *cxled) { return 0; @@ -935,6 +925,20 @@ static inline int cxl_region_discovery(struct cxl_memdev *cxlmd) } #endif +#ifdef CONFIG_CXL_PMEM_REGION +bool is_cxl_pmem_region(struct device *dev); +struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); +#else +static inline bool is_cxl_pmem_region(struct device *dev) +{ + return false; +} +static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) +{ + return NULL; +} +#endif + void cxl_endpoint_parse_cdat(struct cxl_port *port); void cxl_switch_parse_cdat(struct cxl_dport *dport); diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index ad2496b38fdd..024922326a6b 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -59,7 +59,8 @@ cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o cxl_core-y += $(CXL_CORE_SRC)/ras.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o -cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o $(CXL_CORE_SRC)/pmem_region.o +cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o +cxl_core-$(CONFIG_CXL_PMEM_REGION) += $(CXL_CORE_SRC)/pmem_region.o cxl_core-$(CONFIG_CXL_MCE) += $(CXL_CORE_SRC)/mce.o cxl_core-$(CONFIG_CXL_FEATURES) += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_CXL_EDAC_MEM_FEATURES) += $(CXL_CORE_SRC)/edac.o -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113147epcas5p39e1e5e148faaf7e47b40516cfef3de61@epcas5p3.samsung.com>]
* [PATCH V6 16/18] cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion [not found] ` <CGME20260123113147epcas5p39e1e5e148faaf7e47b40516cfef3de61@epcas5p3.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Using these attributes region label is added/deleted into LSA. These attributes are called from userspace (ndctl) after region creation. Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- Documentation/ABI/testing/sysfs-bus-cxl | 22 +++++++ drivers/cxl/core/pmem_region.c | 88 +++++++++++++++++++++++++ drivers/cxl/cxl.h | 7 ++ 3 files changed, 117 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index c80a1b5a03db..e42213c9b030 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -624,3 +624,25 @@ Description: The count is persistent across power loss and wraps back to 0 upon overflow. If this file is not present, the device does not have the necessary support for dirty tracking. + + +What: /sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_update +Date: Jan, 2026 +KernelVersion: v7.0 +Contact: linux-cxl@vger.kernel.org +Description: + (RW) Write a boolean 'true' string value to this attribute to + update cxl region information into LSA as region label. It is + used to update cxl region information saved during cxl region + creation into LSA. This attribute must be written last during + cxl region creation. Reading this attribute indicates whether + the region label is active or not. + + +What: /sys/bus/cxl/devices/regionZ/pmem_regionZ/region_label_delete +Date: Jan, 2026 +KernelVersion: v7.0 +Contact: linux-cxl@vger.kernel.org +Description: + (WO) When a boolean 'true' is written to this attribute then + pmem_region driver deletes cxl region label from LSA. diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c index dcaab59108fd..53d3d81e9676 100644 --- a/drivers/cxl/core/pmem_region.c +++ b/drivers/cxl/core/pmem_region.c @@ -29,8 +29,96 @@ static void cxl_pmem_region_release(struct device *dev) kfree(cxlr_pmem); } +static ssize_t region_label_update_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); + struct cxl_region *cxlr = cxlr_pmem->cxlr; + ssize_t rc; + bool update; + + rc = kstrtobool(buf, &update); + if (rc) + return rc; + + ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region); + if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))) + return rc; + + /* Region not yet committed */ + if (update && cxlr && cxlr->params.state != CXL_CONFIG_COMMIT) { + dev_dbg(dev, "region not committed, can't update into LSA\n"); + return -ENXIO; + } + + if (!cxlr || !cxlr->cxlr_pmem || !cxlr->cxlr_pmem->nd_region) + return 0; + + rc = nd_region_label_update(cxlr->cxlr_pmem->nd_region); + if (rc) + return rc; + + cxlr->params.state_region_label = CXL_REGION_LABEL_ACTIVE; + + return len; +} + +static ssize_t region_label_update_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); + struct cxl_region *cxlr = cxlr_pmem->cxlr; + struct cxl_region_params *p = &cxlr->params; + ssize_t rc; + + ACQUIRE(rwsem_read_intr, rwsem)(&cxl_rwsem.region); + if ((rc = ACQUIRE_ERR(rwsem_read_intr, &rwsem))) + return rc; + + return sysfs_emit(buf, "%d\n", p->state_region_label); +} +static DEVICE_ATTR_RW(region_label_update); + +static ssize_t region_label_delete_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_pmem_region *cxlr_pmem = to_cxl_pmem_region(dev); + struct cxl_region *cxlr = cxlr_pmem->cxlr; + ssize_t rc; + + ACQUIRE(rwsem_write_kill, rwsem)(&cxl_rwsem.region); + if ((rc = ACQUIRE_ERR(rwsem_write_kill, &rwsem))) + return rc; + + if (!cxlr && !cxlr->cxlr_pmem && !cxlr->cxlr_pmem->nd_region) + return 0; + + rc = nd_region_label_delete(cxlr->cxlr_pmem->nd_region); + if (rc) + return rc; + + cxlr->params.state_region_label = CXL_REGION_LABEL_INACTIVE; + + return len; +} +static DEVICE_ATTR_WO(region_label_delete); + +static struct attribute *cxl_pmem_region_attrs[] = { + &dev_attr_region_label_update.attr, + &dev_attr_region_label_delete.attr, + NULL +}; + +static struct attribute_group cxl_pmem_region_group = { + .attrs = cxl_pmem_region_attrs, +}; + static const struct attribute_group *cxl_pmem_region_attribute_groups[] = { &cxl_base_attribute_group, + &cxl_pmem_region_group, NULL }; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 296411be1c36..32a8296a833a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -473,9 +473,15 @@ enum cxl_config_state { CXL_CONFIG_COMMIT, }; +enum region_label_state { + CXL_REGION_LABEL_INACTIVE, + CXL_REGION_LABEL_ACTIVE, +}; + /** * struct cxl_region_params - region settings * @state: allow the driver to lockdown further parameter changes + * @state_region_label: region label state information * @uuid: unique id for persistent regions * @interleave_ways: number of endpoints in the region * @interleave_granularity: capacity each endpoint contributes to a stripe @@ -488,6 +494,7 @@ enum cxl_config_state { */ struct cxl_region_params { enum cxl_config_state state; + enum region_label_state state_region_label; uuid_t uuid; int interleave_ways; int interleave_granularity; -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113149epcas5p1884531875d9676391e7ccc66a0f314d4@epcas5p1.samsung.com>]
* [PATCH V6 17/18] cxl/pmem_region: Create pmem region using information parsed from LSA [not found] ` <CGME20260123113149epcas5p1884531875d9676391e7ccc66a0f314d4@epcas5p1.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 2026-01-23 12:49 ` Jonathan Cameron 0 siblings, 1 reply; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar create_pmem_region() creates CXL region based on region information parsed from the Label Storage Area (LSA). This routine requires cxl endpoint decoder and root decoder. Add cxl_find_root_decoder_by_port() and cxl_find_free_ep_decoder() to find the root decoder and a free endpoint decoder respectively. Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/core/core.h | 5 ++ drivers/cxl/core/pmem_region.c | 133 +++++++++++++++++++++++++++++++++ drivers/cxl/core/region.c | 21 ++++-- drivers/cxl/cxl.h | 5 ++ 4 files changed, 156 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 5ae693269771..8421ea0ef834 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -46,6 +46,7 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, struct cxl_pmem_region_params *pmem_params, struct cxl_endpoint_decoder *cxled); struct cxl_region *to_cxl_region(struct device *dev); +bool is_free_decoder(struct device *dev); #else static inline u64 cxl_dpa_to_hpa(struct cxl_region *cxlr, @@ -87,6 +88,10 @@ static inline struct cxl_region *to_cxl_region(struct device *dev) { return NULL; } +static inline bool is_free_decoder(struct device *dev) +{ + return false; +} #define CXL_REGION_ATTR(x) NULL #define CXL_REGION_TYPE(x) NULL #define SET_CXL_REGION_ATTR(x) diff --git a/drivers/cxl/core/pmem_region.c b/drivers/cxl/core/pmem_region.c index 53d3d81e9676..cdee2788fad8 100644 --- a/drivers/cxl/core/pmem_region.c +++ b/drivers/cxl/core/pmem_region.c @@ -287,3 +287,136 @@ int devm_cxl_add_pmem_region(struct cxl_region *cxlr) cxlr->cxl_nvb = NULL; return rc; } + +static int match_root_decoder_by_dport(struct device *dev, const void *data) +{ + const struct cxl_port *ep_port = data; + struct cxl_root_decoder *cxlrd; + struct cxl_port *root_port; + struct cxl_decoder *cxld; + struct cxl_dport *dport; + int i; + + if (!is_root_decoder(dev)) + return 0; + + cxld = to_cxl_decoder(dev); + if (!(cxld->flags & CXL_DECODER_F_PMEM)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + + root_port = cxlrd_to_port(cxlrd); + dport = cxl_find_dport_by_dev(root_port, ep_port->host_bridge); + if (!dport) + return 0; + + for (i = 0; i < cxlrd->cxlsd.nr_targets; i++) + if (dport == cxlrd->cxlsd.target[i]) + break; + + if (i == cxlrd->cxlsd.nr_targets) + return 0; + + return is_root_decoder(dev); +} + +/** + * cxl_find_root_decoder_by_port() - find a cxl root decoder on cxl bus + * @port: any descendant port in CXL port topology + * @cxled: endpoint decoder + * + * Caller of this function must call put_device() when done as a device ref + * is taken via device_find_child() + */ +static struct cxl_root_decoder * +cxl_find_root_decoder_by_port(struct cxl_port *port, + struct cxl_endpoint_decoder *cxled) +{ + struct cxl_root *cxl_root __free(put_cxl_root) = find_cxl_root(port); + struct cxl_port *ep_port = cxled_to_port(cxled); + struct device *dev; + + if (!cxl_root) + return NULL; + + dev = device_find_child(&cxl_root->port.dev, ep_port, + match_root_decoder_by_dport); + if (!dev) + return NULL; + + return to_cxl_root_decoder(dev); +} + +static int match_free_ep_decoder(struct device *dev, const void *data) +{ + if (!is_endpoint_decoder(dev)) + return 0; + + return is_free_decoder(dev); +} + +/** + * cxl_find_free_ep_decoder() - find a cxl endpoint decoder using cxl port + * @port: any descendant port in CXL port topology + * + * Caller of this function must call put_device() when done as a device ref + * is taken via device_find_child() + */ +static struct cxl_decoder *cxl_find_free_ep_decoder(struct cxl_port *port) +{ + struct device *dev; + + dev = device_find_child(&port->dev, NULL, match_free_ep_decoder); + if (!dev) + return NULL; + + return to_cxl_decoder(dev); +} + +void create_pmem_region(struct nvdimm *nvdimm) +{ + struct cxl_pmem_region_params *params; + struct cxl_endpoint_decoder *cxled; + struct cxl_nvdimm *cxl_nvd; + struct cxl_memdev *cxlmd; + struct cxl_region *cxlr; + + if (!nvdimm_has_cxl_region(nvdimm)) + return; + + lockdep_assert_held(&cxl_rwsem.region); + cxl_nvd = nvdimm_provider_data(nvdimm); + params = nvdimm_get_cxl_region_param(nvdimm); + cxlmd = cxl_nvd->cxlmd; + + /* TODO: Region creation support only for interleave way == 1 */ + if (!(params->nlabel == 1)) { + dev_dbg(&cxlmd->dev, + "Region Creation is not supported with iw > 1\n"); + return; + } + + struct cxl_decoder *cxld __free(put_cxl_decoder) = + cxl_find_free_ep_decoder(cxlmd->endpoint); + if (!cxld) { + dev_err(&cxlmd->dev, "CXL endpoint decoder not found\n"); + return; + } + + cxled = to_cxl_endpoint_decoder(&cxld->dev); + + struct cxl_root_decoder *cxlrd __free(put_cxl_root_decoder) = + cxl_find_root_decoder_by_port(cxlmd->endpoint, cxled); + if (!cxlrd) { + dev_err(&cxlmd->dev, "CXL root decoder not found\n"); + return; + } + + cxlr = cxl_create_region(cxlrd, CXL_PARTMODE_PMEM, + atomic_read(&cxlrd->region_id), + params, cxled); + if (IS_ERR(cxlr)) + dev_warn(&cxlmd->dev, "Region Creation failed\n"); +} +EXPORT_SYMBOL_NS_GPL(create_pmem_region, "CXL"); diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index f9b3dd6cee50..37b88641b818 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -838,25 +838,22 @@ static int check_commit_order(struct device *dev, void *data) return 0; } -static int match_free_decoder(struct device *dev, const void *data) +bool is_free_decoder(struct device *dev) { struct cxl_port *port = to_cxl_port(dev->parent); struct cxl_decoder *cxld; int rc; - if (!is_switch_decoder(dev)) - return 0; - cxld = to_cxl_decoder(dev); if (cxld->id != port->commit_end + 1) - return 0; + return false; if (cxld->region) { dev_dbg(dev->parent, "next decoder to commit (%s) is already reserved (%s)\n", dev_name(dev), dev_name(&cxld->region->dev)); - return 0; + return false; } rc = device_for_each_child_reverse_from(dev->parent, dev, NULL, @@ -865,9 +862,17 @@ static int match_free_decoder(struct device *dev, const void *data) dev_dbg(dev->parent, "unable to allocate %s due to out of order shutdown\n", dev_name(dev)); - return 0; + return false; } - return 1; + return true; +} + +static int match_free_decoder(struct device *dev, const void *data) +{ + if (!is_switch_decoder(dev)) + return 0; + + return is_free_decoder(dev); } static bool spa_maps_hpa(const struct cxl_region_params *p, diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 32a8296a833a..1fc3e6260a00 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -794,6 +794,7 @@ struct cxl_root *find_cxl_root(struct cxl_port *port); DEFINE_FREE(put_cxl_root, struct cxl_root *, if (_T) put_device(&_T->port.dev)) DEFINE_FREE(put_cxl_port, struct cxl_port *, if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) DEFINE_FREE(put_cxl_root_decoder, struct cxl_root_decoder *, if (!IS_ERR_OR_NULL(_T)) put_device(&_T->cxlsd.cxld.dev)) +DEFINE_FREE(put_cxl_decoder, struct cxl_decoder *, if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) DEFINE_FREE(put_cxl_region, struct cxl_region *, if (!IS_ERR_OR_NULL(_T)) put_device(&_T->dev)) int devm_cxl_enumerate_ports(struct cxl_memdev *cxlmd); @@ -935,6 +936,7 @@ static inline int cxl_region_discovery(struct cxl_memdev *cxlmd) #ifdef CONFIG_CXL_PMEM_REGION bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); +void create_pmem_region(struct nvdimm *nvdimm); #else static inline bool is_cxl_pmem_region(struct device *dev) { @@ -944,6 +946,9 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) { return NULL; } +static inline void create_pmem_region(struct nvdimm *nvdimm) +{ +} #endif void cxl_endpoint_parse_cdat(struct cxl_port *port); -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 17/18] cxl/pmem_region: Create pmem region using information parsed from LSA 2026-01-23 11:31 ` [PATCH V6 17/18] cxl/pmem_region: Create pmem region using information parsed from LSA Neeraj Kumar @ 2026-01-23 12:49 ` Jonathan Cameron 0 siblings, 0 replies; 35+ messages in thread From: Jonathan Cameron @ 2026-01-23 12:49 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel On Fri, 23 Jan 2026 17:01:11 +0530 Neeraj Kumar <s.neeraj@samsung.com> wrote: > create_pmem_region() creates CXL region based on region information > parsed from the Label Storage Area (LSA). This routine requires cxl > endpoint decoder and root decoder. Add cxl_find_root_decoder_by_port() > and cxl_find_free_ep_decoder() to find the root decoder and a free > endpoint decoder respectively. > > Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> ^ permalink raw reply [flat|nested] 35+ messages in thread
[parent not found: <CGME20260123113150epcas5p14974b694807b93b27319c33f3003dcba@epcas5p1.samsung.com>]
* [PATCH V6 18/18] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem [not found] ` <CGME20260123113150epcas5p14974b694807b93b27319c33f3003dcba@epcas5p1.samsung.com> @ 2026-01-23 11:31 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-01-23 11:31 UTC (permalink / raw) To: linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel, Neeraj Kumar Add support of CXL LSA 2.1 using NDD_REGION_LABELING flag. It creates cxl region based on region information parsed from LSA Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Neeraj Kumar <s.neeraj@samsung.com> --- drivers/cxl/pmem.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/cxl/pmem.c b/drivers/cxl/pmem.c index a6eba3572090..5970d1792be8 100644 --- a/drivers/cxl/pmem.c +++ b/drivers/cxl/pmem.c @@ -135,6 +135,7 @@ static int cxl_nvdimm_probe(struct device *dev) return rc; set_bit(NDD_LABELING, &flags); + set_bit(NDD_REGION_LABELING, &flags); set_bit(NDD_REGISTER_SYNC, &flags); set_bit(ND_CMD_GET_CONFIG_SIZE, &cmd_mask); set_bit(ND_CMD_GET_CONFIG_DATA, &cmd_mask); @@ -155,6 +156,7 @@ static int cxl_nvdimm_probe(struct device *dev) return -ENOMEM; dev_set_drvdata(dev, nvdimm); + create_pmem_region(nvdimm); return devm_add_action_or_reset(dev, unregister_nvdimm, nvdimm); } -- 2.34.1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem 2026-01-23 11:30 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem Neeraj Kumar ` (17 preceding siblings ...) [not found] ` <CGME20260123113150epcas5p14974b694807b93b27319c33f3003dcba@epcas5p1.samsung.com> @ 2026-01-23 12:54 ` Jonathan Cameron 2026-01-23 17:52 ` Dave Jiang 2026-01-23 21:16 ` Alison Schofield 20 siblings, 0 replies; 35+ messages in thread From: Jonathan Cameron @ 2026-01-23 12:54 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel Trivial comment on the change log. To me this series looks good now. However, there are some other series touching some of the functions you modify. E.g cxl_endpoint_port_probe() I'm not suggesting you rebase on those (Dave might ask for it however!), but it is probably useful to try a rebase and reply to this cover letter if there are any significant issues. There is a lot of code movement going on so it might just be fuzz. From the cxl tree, try it on top of the next branch. https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=next Thanks, Jonathan > Changes in v5->v6 > ----------------- > - Find v5 link at [4] > - Find v4 link at [5] > - Find v3 link at [6] > - Find v2 link at [7] > - v1 patch-set was broken. Find the v1 links at [8] and [9] > > [Misc] > - Rebase with for-7.0/cxl-init > > [PATCH 01/18] > - Add Ira RB tag > > [PATCH 03/18] > - Use export_uuid() instead of uuid_copy() [Jonathan] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 04/18] > - Move the assignments at relevant place [Jonathan] > - Rename lslot to label_slot [Jonathan] > - Add Ira RB tag > > [PATCH 05/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 06/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 07/18] > - Add Ira RB tag > > [PATCH 08/18] > - use export_uuid() to avoid casts to uuid_t * [Jonathan] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 09/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 10/18] > - Initialize struct cxl_memdev_attach variable [Jonathan] > - Add Jonathan RB tag > > [PATCH 10/18] I briefly got confused when using this to see what chagned! This one is patch 11 > - Its a new patch addition in this series > - Seperate out renaming of __create_region() to cxl_create_region() [Jonathan] > > [PATCH 11/18] This patch 12. > - Remove usage of cxl_dpa_free() from alloc_region_dpa() [Jonathan] > > [PATCH 13/18] > - Add Jonathan RB tag > > [PATCH 15/18] > - Add Jonathan RB tag > > [PATCH 16/18] > - Fix wrong name state with region_label_state [Jonathan] > - Add Jonathan RB tag > > [PATCH 17/18] > - Optimize loop matching [Jonathan] > - Use reverse xmas tree stype [Jonathan] > > [PATCH 18/18] > - Add Jonathan RB tag > ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem 2026-01-23 11:30 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem Neeraj Kumar ` (18 preceding siblings ...) 2026-01-23 12:54 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and " Jonathan Cameron @ 2026-01-23 17:52 ` Dave Jiang 2026-03-06 10:26 ` Neeraj Kumar 2026-01-23 21:16 ` Alison Schofield 20 siblings, 1 reply; 35+ messages in thread From: Dave Jiang @ 2026-01-23 17:52 UTC (permalink / raw) To: Neeraj Kumar, linux-cxl, nvdimm, linux-kernel, gost.dev Cc: a.manzanares, vishak.g, neeraj.kernel On 1/23/26 4:30 AM, Neeraj Kumar wrote: > Introduction: > ============= > CXL Persistent Memory (Pmem) devices region, namespace and content must be > persistent across system reboot. In order to achieve this persistency, it > uses Label Storage Area (LSA) to store respective metadata. During system > reboot, stored metadata in LSA is used to bring back the region, namespace > and content of CXL device in its previous state. > CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox > commands to access the LSA area. nvdimm driver is using same commands to > get/set LSA data. Hi Neeraj, I'm attempting to vet the series before applying to cxl/next. But it seems to be failing multiple unit regression tests in cxl_test. We need to figure out why before the series can be merged. Most likely the changes in this series also require cxl_test updates. Can you please take a look? I will also take a look and try to help you address the issues. Thanks Ok: 10 Expected Fail: 0 Fail: 4 Unexpected Pass: 0 Skipped: 1 Timeout: 0 The script below [1] can get you up and running with a qemu environment where you can run cxl_test. .../run_qemu/run_qemu.sh --cxl --cxl-test --cxl-debug --hmat -r wipe cd ndctl meson test -C build --suite=cxl [1] https://github.com/pmem/run_qemu DJ > > There are three types of LSA format and these are part of following > specifications: > - v1.1: https://pmem.io/documents/NVDIMM_Namespace_Spec.pdf > - v1.2: https://uefi.org/sites/default/files/resources/UEFI_Spec_2_7.pdf > - v2.1: https://computeexpresslink.org/wp-content/uploads/2024/02/CXL-2.0-Specification.pdf > > Basic differences between these LSA formats: > - v1.1: Support Namespace persistency. Size of Namespace Label is 128 bytes > - v1.2: Support Namespace persistency. Size of Namespace Label is 256 bytes > - v2.1: Support Namespace and Region persistency. Size of Namespace and > Region Label is 256 bytes. > > Linux nvdimm driver supports only v1.1 and v1.2 LSA format. CXL pmem device > require support of LSA v2.1 format for region and namespace persistency. > Initial support of LSA 2.1 was add in [1]. > > This patchset adds support of LSA 2.1 in nvdimm and cxl pmem driver. > > Patch 1: Introduce NDD_CXL_LABEL flag and Update cxl label index as per v2.1 > Patch 2: Skip the need for 'interleave-set cookie' for LSA 2.1 support > Patch 3-9: Introduce region label and update namespace label as per LSA 2.1 > Patch 10: Refactor cxl pmem region auto assembly using Dan's Infra > Patch 11-13: Save cxl region info in LSA and region recreation during reboot > Patch 14:15: Segregate out cxl pmem region code from region.c to pmem_region.c > Patch 16: Introduce cxl region addition/deletion attributes > Patch 17-18: Add support of cxl pmem region re-creation from CXL as per LSA 2.1 > > Testing: > ======== > In order to test this patchset, I also added the support of LSA v2.1 format > in ndctl. ndctl changes are available at [2]. After review, I’ll push in > ndctl repo for community review. > > 1. Used Qemu using following CXL topology > M2="-object memory-backend-file,id=cxl-mem11,share=on,mem-path=$TMP_DIR/cxltest11.raw,size=4G \ > -object memory-backend-file,id=cxl-lsa11,share=on,mem-path=$TMP_DIR/lsa.raw11,size=1M \ > -object memory-backend-file,id=cxl-mem21,share=on,mem-path=$TMP_DIR/cxltest21.raw,size=4G \ > -object memory-backend-file,id=cxl-lsa21,share=on,mem-path=$TMP_DIR/lsa21.raw,size=1M \ > -device pxb-cxl,bus=pcie.0,bus_nr=10,id=cxl.1 \ > -device cxl-rp,port=110,bus=cxl.1,id=root_port11,chassis=0,slot=11 \ > -device cxl-type3,bus=root_port11,memdev=cxl-mem11,lsa=cxl-lsa11,id=cxl-pmem11,sn=11 \ > -device pxb-cxl,bus=pcie.0,bus_nr=20,id=cxl.2 \ > -device cxl-rp,port=210,bus=cxl.2,id=root_port21,chassis=0,slot=21 \ > -device cxl-type3,bus=root_port21,memdev=cxl-mem21,lsa=cxl-lsa21,id=cxl-pmem21,sn=21 \ > -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=8G,cxl-fmw.0.interleave-granularity=8k,cxl-fmw.1.targets.0=cxl.2,cxl-fmw.1.size=8G" > > 2. Create cxl region on both devices > cxl create-region -d decoder0.1 -m mem0 > cxl create-region -d decoder0.0 -m mem1 > > root@QEMUCXL6060pmem:~# cxl list > [ > { > "memdevs":[ > { > "memdev":"mem0", > "pmem_size":4294967296, > "serial":21, > "host":"0000:15:00.0", > "firmware_version":"BWFW VERSION 00" > }, > { > "memdev":"mem1", > "pmem_size":4294967296, > "serial":11, > "host":"0000:0b:00.0", > "firmware_version":"BWFW VERSION 00" > } > ] > }, > { > "regions":[ > { > "region":"region0", > "resource":45365592064, > "size":4294967296, > "type":"pmem", > "interleave_ways":1, > "interleave_granularity":256, > "decode_state":"commit", > "qos_class_mismatch":true > }, > { > "region":"region1", > "resource":53955526656, > "size":4294967296, > "type":"pmem", > "interleave_ways":1, > "interleave_granularity":256, > "decode_state":"commit", > "qos_class_mismatch":true > } > ] > } > ] > > 3. Re-Start Qemu and we could see cxl region persistency using "cxl list" > > 4. Create namespace for both regions > root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=256M > { > "dev":"namespace0.1", > "mode":"fsdax", > "map":"dev", > "size":"250.00 MiB (262.14 MB)", > "uuid":"5f9e28b4-f403-4e72-bb06-307ead53dec4", > "sector_size":512, > "align":2097152, > "blockdev":"pmem0.1" > } > > real 1m8.795s > user 0m0.098s > sys 0m48.699s > > > root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=128M > { > "dev":"namespace1.1", > "mode":"fsdax", > "map":"dev", > "size":"124.00 MiB (130.02 MB)", > "uuid":"33682096-7364-412d-bfbc-fa7568939abd", > "sector_size":512, > "align":2097152, > "blockdev":"pmem1.1" > } > > > root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=128M > { > "dev":"namespace0.0", > "mode":"fsdax", > "map":"dev", > "size":"124.00 MiB (130.02 MB)", > "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", > "sector_size":512, > "align":2097152, > "blockdev":"pmem0" > } > > real 0m39.805s > user 0m0.138s > sys 0m21.485s > > > root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region1 --size=256M > { > "dev":"namespace1.0", > "mode":"fsdax", > "map":"dev", > "size":"250.00 MiB (262.14 MB)", > "uuid":"15f1c8d5-416f-4d2e-9e39-d17ce8f73a42", > "sector_size":512, > "align":2097152, > "blockdev":"pmem1" > } > > real 0m43.899s > user 0m0.113s > sys 0m31.167s > > > root@QEMUCXL6060pmem:~# time ndctl create-namespace --mode=fsdax --region=region0 --size=128M > { > "dev":"namespace0.0", > "mode":"fsdax", > "map":"dev", > "size":"124.00 MiB (130.02 MB)", > "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", > "sector_size":512, > "align":2097152, > "blockdev":"pmem0" > } > > real 0m39.805s > user 0m0.138s > sys 0m21.485s > > > root@QEMUCXL6060pmem:~# ndctl list > [ > { > "dev":"namespace1.0", > "mode":"fsdax", > "map":"dev", > "size":262144000, > "uuid":"15f1c8d5-416f-4d2e-9e39-d17ce8f73a42", > "sector_size":512, > "align":2097152, > "blockdev":"pmem1" > }, > { > "dev":"namespace1.1", > "mode":"fsdax", > "map":"dev", > "size":130023424, > "uuid":"33682096-7364-412d-bfbc-fa7568939abd", > "sector_size":512, > "align":2097152, > "blockdev":"pmem1.1" > }, > { > "dev":"namespace0.1", > "mode":"fsdax", > "map":"dev", > "size":262144000, > "uuid":"5f9e28b4-f403-4e72-bb06-307ead53dec4", > "sector_size":512, > "align":2097152, > "blockdev":"pmem0.1" > }, > { > "dev":"namespace0.0", > "mode":"fsdax", > "map":"dev", > "size":130023424, > "uuid":"9681af0e-349a-401e-9833-1cf8903d58fa", > "sector_size":512, > "align":2097152, > "blockdev":"pmem0" > } > ] > > 5. Re-Start Qemu and we could see > - Region persistency using "cxl list" > - Namespace persistency using "ndctl list" and cat /proc/iomem > > root@QEMUCXL6060pmem:~# cat /proc/iomem > fed1c000-fed1ffff : Reserved > feffc000-feffffff : Reserved > fffc0000-ffffffff : Reserved > 100000000-27fffffff : System RAM > a90000000-c8fffffff : CXL Window 0 > a90000000-b8fffffff : Persistent Memory > a90000000-b8fffffff : region0 > a90000000-a97ffffff : namespace0.0 > a98000000-aa7ffffff : namespace0.1 > c90000000-e8fffffff : CXL Window 1 > c90000000-d8fffffff : Persistent Memory > c90000000-d8fffffff : region1 > c90000000-c9fffffff : namespace1.0 > ca0000000-ca7ffffff : namespace1.1 > 380000000000-38000000ffff : PCI Bus 0000:0a > 380000000000-38000000ffff : 0000:0a:00.0 > > - NOTE: We can see some lag in restart. Look at Observesation below > > 6. Also verify LSA version using "ndctl read-labels -j nmem0" > root@QEMUCXL6060pmem:~# time ndctl read-labels -j nmem0 > { > "dev":"nmem0", > "index":[ > { > "signature":"NAMESPACE_INDEX", > "major":2, > "minor":1, > "labelsize":256, > "seq":2, > "nslot":4090 > }, > { > "signature":"NAMESPACE_INDEX", > "major":2, > "minor":1, > "labelsize":256, > "seq":1, > "nslot":4090 > } > ], > "label":[ > { > "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", > "uuid":"5b33b940-6767-4951-b923-952455482f52", > "name":"", > "flags":8, > "nrange":1, > "position":0, > "dpa":134217728, > "rawsize":268435456, > "slot":0, > "align":0, > "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", > "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", > "lbasize":512 > }, > { > "type":"529d7c61-da07-47c4-a93f-ecdf2c06f444", > "uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", > "flags":0, > "nlabel":1, > "position":0, > "dpa":0, > "rawsize":4294967296, > "hpa":45365592064, > "slot":1, > "interleave granularity":256, > "align":0 > }, > { > "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", > "uuid":"2c1c33c0-909b-4c6a-8de4-dfc6c13ae82c", > "name":"", > "flags":0, > "nrange":1, > "position":0, > "dpa":0, > "rawsize":134217728, > "slot":2, > "align":0, > "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", > "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", > "lbasize":512 > }, > { > "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c", > "uuid":"5b33b940-6767-4951-b923-952455482f52", > "name":"", > "flags":0, > "nrange":1, > "position":0, > "dpa":134217728, > "rawsize":268435456, > "slot":3, > "align":0, > "region_uuid":"a1482e6b-968e-4297-b2f5-bf1733966e55", > "abstraction_uuid":"266400ba-fb9f-4677-bcb0-968f11d0d225", > "lbasize":512 > } > ] > } > read 1 nmem > > - NOTE: We have following UUID types as per CXL Spec > "type":"529d7c61-da07-47c4-a93f-ecdf2c06f444" is region label > "type":"68bb2c0a-5a77-4937-9f85-3caf41a0f93c" is namespace label > > > Limitation (WIP): > ================ > Current changes only support interleave way == 1 > > > Observation: > ============ > First time namespace creation using ndctl takes around 10 to 20 second time > while executing "devm_memremap_pages" at [3] > > As using this patchset, after auto region creation, namespace creation is > happening in boot sequence (if nvdimm and cxl drivers are static), It is > therefore boot sequence is increased by around 10 to 20 sec. > > Changes > ======= > Changes in v5->v6 > ----------------- > - Find v5 link at [4] > - Find v4 link at [5] > - Find v3 link at [6] > - Find v2 link at [7] > - v1 patch-set was broken. Find the v1 links at [8] and [9] > > [Misc] > - Rebase with for-7.0/cxl-init > > [PATCH 01/18] > - Add Ira RB tag > > [PATCH 03/18] > - Use export_uuid() instead of uuid_copy() [Jonathan] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 04/18] > - Move the assignments at relevant place [Jonathan] > - Rename lslot to label_slot [Jonathan] > - Add Ira RB tag > > [PATCH 05/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 06/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 07/18] > - Add Ira RB tag > > [PATCH 08/18] > - use export_uuid() to avoid casts to uuid_t * [Jonathan] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 09/18] > - Add Jonathan RB tag > - Add Ira RB tag > > [PATCH 10/18] > - Initialize struct cxl_memdev_attach variable [Jonathan] > - Add Jonathan RB tag > > [PATCH 10/18] > - Its a new patch addition in this series > - Seperate out renaming of __create_region() to cxl_create_region() [Jonathan] > > [PATCH 11/18] > - Remove usage of cxl_dpa_free() from alloc_region_dpa() [Jonathan] > > [PATCH 13/18] > - Add Jonathan RB tag > > [PATCH 15/18] > - Add Jonathan RB tag > > [PATCH 16/18] > - Fix wrong name state with region_label_state [Jonathan] > - Add Jonathan RB tag > > [PATCH 17/18] > - Optimize loop matching [Jonathan] > - Use reverse xmas tree stype [Jonathan] > > [PATCH 18/18] > - Add Jonathan RB tag > > > [1]: https://lore.kernel.org/all/163116432405.2460985.5547867384570123403.stgit@dwillia2-desk3.amr.corp.intel.com/ > [2]: https://github.com/neerajoss/ndctl/commits/linux-cxl/V1_CXL_LSA_2.1_Support/ > [3]: https://elixir.bootlin.com/linux/v6.13.7/source/drivers/nvdimm/pmem.c#L520 > [4]: https://lore.kernel.org/linux-cxl/20260109124437.4025893-1-s.neeraj@samsung.com/ > [5]: https://lore.kernel.org/linux-cxl/20250917134116.1623730-1-s.neeraj@samsung.com/ > [6]: https://lore.kernel.org/linux-cxl/20251119075255.2637388-1-s.neeraj@samsung.com/ > [7]: https://lore.kernel.org/linux-cxl/20250730121209.303202-1-s.neeraj@samsung.com/ > [8] v1 Cover Letter: https://lore.kernel.org/linux-cxl/1931444790.41750165203442.JavaMail.epsvc@epcpadp1new/ > [9] v1 Rest Thread: https://lore.kernel.org/linux-cxl/158453976.61750165203630.JavaMail.epsvc@epcpadp1new/ > > > Neeraj Kumar (18): > nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label > nvdimm/label: CXL labels skip the need for 'interleave-set cookie' > nvdimm/label: Add namespace/region label support as per LSA 2.1 > nvdimm/label: Include region label in slot validation > nvdimm/label: Skip region label during ns label DPA reservation > nvdimm/label: Preserve region label during namespace creation > nvdimm/label: Add region label delete support > nvdimm/label: Preserve cxl region information from region label > nvdimm/label: Export routine to fetch region information > cxl/mem: Refactor cxl pmem region auto-assembling > cxl/region: Rename __create_region() to cxl_create_region() > cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation > cxl/pmem: Preserve region information into nd_set > cxl/pmem_region: Prep patch to accommodate pmem_region attributes > cxl/pmem_region: Introduce CONFIG_CXL_PMEM_REGION for core/pmem_region.c > cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion > cxl/pmem_region: Create pmem region using information parsed from LSA > cxl/pmem: Add CXL LSA 2.1 support in cxl pmem > > Documentation/ABI/testing/sysfs-bus-cxl | 22 + > drivers/cxl/Kconfig | 15 + > drivers/cxl/core/Makefile | 1 + > drivers/cxl/core/core.h | 35 +- > drivers/cxl/core/pmem_region.c | 422 +++++++++++++++++ > drivers/cxl/core/region.c | 377 ++++++++------- > drivers/cxl/cxl.h | 41 +- > drivers/cxl/mem.c | 18 +- > drivers/cxl/pci.c | 4 +- > drivers/cxl/pmem.c | 15 +- > drivers/cxl/port.c | 39 +- > drivers/nvdimm/dimm.c | 5 + > drivers/nvdimm/dimm_devs.c | 19 + > drivers/nvdimm/label.c | 580 ++++++++++++++++++++---- > drivers/nvdimm/label.h | 18 +- > drivers/nvdimm/namespace_devs.c | 82 +++- > drivers/nvdimm/nd-core.h | 2 + > drivers/nvdimm/nd.h | 64 +++ > drivers/nvdimm/region_devs.c | 10 + > include/linux/libnvdimm.h | 28 ++ > tools/testing/cxl/Kbuild | 1 + > 21 files changed, 1424 insertions(+), 374 deletions(-) > create mode 100644 drivers/cxl/core/pmem_region.c > > > base-commit: bc62f5b308cbdedf29132fe96e9d591e526527e1 ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem 2026-01-23 17:52 ` Dave Jiang @ 2026-03-06 10:26 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-03-06 10:26 UTC (permalink / raw) To: Dave Jiang Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs [-- Attachment #1: Type: text/plain, Size: 1548 bytes --] On 23/01/26 10:52AM, Dave Jiang wrote: > > >On 1/23/26 4:30 AM, Neeraj Kumar wrote: >> Introduction: >> ============= >> CXL Persistent Memory (Pmem) devices region, namespace and content must be >> persistent across system reboot. In order to achieve this persistency, it >> uses Label Storage Area (LSA) to store respective metadata. During system >> reboot, stored metadata in LSA is used to bring back the region, namespace >> and content of CXL device in its previous state. >> CXL specification provides Get_LSA (4102h) and Set_LSA (4103h) mailbox >> commands to access the LSA area. nvdimm driver is using same commands to >> get/set LSA data. > >Hi Neeraj, >I'm attempting to vet the series before applying to cxl/next. But it seems to be failing multiple unit regression tests in cxl_test. We need to figure out why before the series can be merged. Most likely the changes in this series also require cxl_test updates. Can you please take a look? I will also take a look and try to help you address the issues. Thanks > >Ok: 10 >Expected Fail: 0 >Fail: 4 >Unexpected Pass: 0 >Skipped: 1 >Timeout: 0 > >The script below [1] can get you up and running with a qemu environment where you can run cxl_test. > >.../run_qemu/run_qemu.sh --cxl --cxl-test --cxl-debug --hmat -r wipe > >cd ndctl >meson test -C build --suite=cxl > >[1] https://github.com/pmem/run_qemu > >DJ Sure Dave, I am able to reproduce the same, I will analyze and fix them in next patch series. Regards, Neeraj [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem 2026-01-23 11:30 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem Neeraj Kumar ` (19 preceding siblings ...) 2026-01-23 17:52 ` Dave Jiang @ 2026-01-23 21:16 ` Alison Schofield 2026-03-06 10:27 ` Neeraj Kumar 20 siblings, 1 reply; 35+ messages in thread From: Alison Schofield @ 2026-01-23 21:16 UTC (permalink / raw) To: Neeraj Kumar Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel On Fri, Jan 23, 2026 at 05:00:54PM +0530, Neeraj Kumar wrote: snip > > Testing: > ======== > In order to test this patchset, I also added the support of LSA v2.1 format > in ndctl. ndctl changes are available at [2]. After review, I’ll push in > ndctl repo for community review. Please post the ndctl changes for review. Thanks! ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem 2026-01-23 21:16 ` Alison Schofield @ 2026-03-06 10:27 ` Neeraj Kumar 0 siblings, 0 replies; 35+ messages in thread From: Neeraj Kumar @ 2026-03-06 10:27 UTC (permalink / raw) To: Alison Schofield Cc: linux-cxl, nvdimm, linux-kernel, gost.dev, a.manzanares, vishak.g, neeraj.kernel, cpgs [-- Attachment #1: Type: text/plain, Size: 561 bytes --] On 23/01/26 01:16PM, Alison Schofield wrote: >On Fri, Jan 23, 2026 at 05:00:54PM +0530, Neeraj Kumar wrote: > >snip > >> >> Testing: >> ======== >> In order to test this patchset, I also added the support of LSA v2.1 format >> in ndctl. ndctl changes are available at [2]. After review, I’ll push in >> ndctl repo for community review. > >Please post the ndctl changes for review. Thanks! Sure Alison, I am working on multi interleave support, So it may require some change in ndctl also. I will post ndctl changes along with next series. Regards, Neeraj [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2026-03-08 4:30 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CGME20260123113121epcas5p125bc64b4714525d7bbd489cd9be9ba91@epcas5p1.samsung.com>
2026-01-23 11:30 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and cxl pmem Neeraj Kumar
[not found] ` <CGME20260123113123epcas5p1cd55b845a358fccd305e655664b4f042@epcas5p1.samsung.com>
2026-01-23 11:30 ` [PATCH V6 01/18] nvdimm/label: Introduce NDD_REGION_LABELING flag to set region label Neeraj Kumar
[not found] ` <CGME20260123113125epcas5p2331d76ce8aee6d594dbdd5b3e75e7d6f@epcas5p2.samsung.com>
2026-01-23 11:30 ` [PATCH V6 02/18] nvdimm/label: CXL labels skip the need for 'interleave-set cookie' Neeraj Kumar
[not found] ` <CGME20260123113127epcas5p2c03d8e1749dc7e266870cc55eda31822@epcas5p2.samsung.com>
2026-01-23 11:30 ` [PATCH V6 03/18] nvdimm/label: Add namespace/region label support as per LSA 2.1 Neeraj Kumar
[not found] ` <CGME20260123113129epcas5p26bb834be2b841296ca1b3fff1a44ab76@epcas5p2.samsung.com>
2026-01-23 11:30 ` [PATCH V6 04/18] nvdimm/label: Include region label in slot validation Neeraj Kumar
[not found] ` <CGME20260123113130epcas5p4af26a172d155a1c8b78d33945f495452@epcas5p4.samsung.com>
2026-01-23 11:30 ` [PATCH V6 05/18] nvdimm/label: Skip region label during ns label DPA reservation Neeraj Kumar
[not found] ` <CGME20260123113132epcas5p368496769470a9163183a1d5a967442e2@epcas5p3.samsung.com>
2026-01-23 11:31 ` [PATCH V6 06/18] nvdimm/label: Preserve region label during namespace creation Neeraj Kumar
[not found] ` <CGME20260123113133epcas5p4989af8ca6a75569b2b47d80514e62c57@epcas5p4.samsung.com>
2026-01-23 11:31 ` [PATCH V6 07/18] nvdimm/label: Add region label delete support Neeraj Kumar
[not found] ` <CGME20260123113135epcas5p2b0e7d9651a5a78b5451daeb6b1e018d8@epcas5p2.samsung.com>
2026-01-23 11:31 ` [PATCH V6 08/18] nvdimm/label: Preserve cxl region information from region label Neeraj Kumar
[not found] ` <CGME20260123113136epcas5p22683ce5ef04a3da50fbb319a43e331a6@epcas5p2.samsung.com>
2026-01-23 11:31 ` [PATCH V6 09/18] nvdimm/label: Export routine to fetch region information Neeraj Kumar
[not found] ` <CGME20260123113138epcas5p4bc54ae41192dbc9c958c43ad8a80b5c6@epcas5p4.samsung.com>
2026-01-23 11:31 ` [PATCH V6 10/18] cxl/mem: Refactor cxl pmem region auto-assembling Neeraj Kumar
2026-01-26 22:48 ` Dave Jiang
2026-01-27 23:45 ` Dave Jiang
2026-03-06 10:22 ` Neeraj Kumar
2026-01-28 0:02 ` dan.j.williams
2026-03-06 10:18 ` Neeraj Kumar
[not found] ` <CGME20260123113140epcas5p4bc7a864d4a717b96afcb863e3200b3a2@epcas5p4.samsung.com>
2026-01-23 11:31 ` [PATCH V6 11/18] cxl/region: Rename __create_region() to cxl_create_region() Neeraj Kumar
2026-01-23 12:37 ` Jonathan Cameron
[not found] ` <CGME20260123113141epcas5p49a1eebff4401a7fc98381358162fde2b@epcas5p4.samsung.com>
2026-01-23 11:31 ` [PATCH V6 12/18] cxl/region: Add devm_cxl_pmem_add_region() for pmem region creation Neeraj Kumar
2026-01-23 12:39 ` Jonathan Cameron
2026-02-24 18:13 ` Alejandro Lucero Palau
2026-03-06 10:24 ` Neeraj Kumar
2026-03-06 16:28 ` Alejandro Lucero Palau
[not found] ` <CGME20260123113143epcas5p1f056205feaa11092be91be6f74ccc0be@epcas5p1.samsung.com>
2026-01-23 11:31 ` [PATCH V6 13/18] cxl/pmem: Preserve region information into nd_set Neeraj Kumar
[not found] ` <CGME20260123113144epcas5p18477c68ede07c17a6c8461cf034eb3d1@epcas5p1.samsung.com>
2026-01-23 11:31 ` [PATCH V6 14/18] cxl/pmem_region: Prep patch to accommodate pmem_region attributes Neeraj Kumar
[not found] ` <CGME20260123113146epcas5p3d267b8d1aed388debac761ebf5f143d4@epcas5p3.samsung.com>
2026-01-23 11:31 ` [PATCH V6 15/18] cxl/pmem_region: Introduce CONFIG_CXL_PMEM_REGION for core/pmem_region.c Neeraj Kumar
[not found] ` <CGME20260123113147epcas5p39e1e5e148faaf7e47b40516cfef3de61@epcas5p3.samsung.com>
2026-01-23 11:31 ` [PATCH V6 16/18] cxl/pmem_region: Add sysfs attribute cxl region label updation/deletion Neeraj Kumar
[not found] ` <CGME20260123113149epcas5p1884531875d9676391e7ccc66a0f314d4@epcas5p1.samsung.com>
2026-01-23 11:31 ` [PATCH V6 17/18] cxl/pmem_region: Create pmem region using information parsed from LSA Neeraj Kumar
2026-01-23 12:49 ` Jonathan Cameron
[not found] ` <CGME20260123113150epcas5p14974b694807b93b27319c33f3003dcba@epcas5p1.samsung.com>
2026-01-23 11:31 ` [PATCH V6 18/18] cxl/pmem: Add CXL LSA 2.1 support in cxl pmem Neeraj Kumar
2026-01-23 12:54 ` [PATCH V6 00/18] Add CXL LSA 2.1 format support in nvdimm and " Jonathan Cameron
2026-01-23 17:52 ` Dave Jiang
2026-03-06 10:26 ` Neeraj Kumar
2026-01-23 21:16 ` Alison Schofield
2026-03-06 10:27 ` Neeraj Kumar
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
Powered by JetHome