[PATCH v17 00/19] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks,

This series adds CXL Reset for Type 2 devices through the existing PCI
reset_method and reset sysfs attributes. It disables CXL.cache, performs
supported cache writeback, initiates CXL Reset with Memory Clear, and
restores cached HDM decoder and CXL Device DVSEC state before PCI use
resumes. Active HDM ranges are reserved and CPU caches invalidated before
and after reset. The method remains limited to eligible single-function
devices with both CXL.cache and CXL.mem enabled. CXL bus reset restores
the same cached state and flushes the host and device caches around the
reset.

The reset sequence follows the CXL 4.0 specification [4], including
CXL.cache disable, optional cache writeback, CXL Reset Memory Clear,
ResetComplete polling, and ResetError reporting. Sections 9.7.2 and 9.7.3
define the completion timeout encodings and reset access restrictions.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace selects "cxl_reset" in reset_method and triggers the reset
through reset. Following Dan's v6 feedback, this uses the PCI interface
instead of adding a memdev-specific sysfs ABI.

For sysfs reset, the generic PCI path invokes the driver's reset_prepare()
callback when provided and disables PCI I/O, memory decoding, and bus
mastering. The CXL path reserves affected HDM ranges exclusively and
blocks IOMMU translations where applicable. It does not offline memory
or revoke mappings: userspace is expected to offline CXL.mem users before
reset. The reservation enforces this for claimed ranges by failing the
reset while a conflicting resource is claimed [9].

The global region lock is held from snapshot through restoration. Memory
Clear completion is awaited for the timeout the device advertises, up to
100 seconds, so the lock can be held for that long plus cache flushes.

Changes from v16 [1]:
- Publish the PCI HDM cache under the PCI device lock instead of
  cxl_rwsem.dpa, and drop the cxl_hdm_cache_present() and
  cxl_hdm_cache_publish() helpers (Li Ming) [6][7]
- Wait for the reset timeout the device advertises, up to 100 seconds,
  instead of clamping it to 20 seconds, so Control2 is not accessed while
  the device may still be resetting (Richard) [8]
- State that userspace is expected to offline CXL.mem users before reset,
  and what the HPA reservation does and does not cover (Richard) [9]
- Enable bus mastering after the IOMMU is restored on CXL bus reset
  (Richard) [10]
- Explain why cxl_set_mem_enable() refreshes the cached DVSEC Control when
  MEM_ENABLE already matches (Richard) [11]
- Initialize the zero-size HDM decoder end to base - 1 and document the
  zero-size case (Richard) [12]
- Set Lock On Commit after programming the decoder registers and before
  Commit when recommitting a locked decoder (Alison) [13]
- Record Normalized Addressing in the cached decoder when region discovery
  sets it, and note the short window before that update (Sashiko) [15]
- Reject an all-ones DVSEC Control read in cxl_set_mem_enable() (Sashiko)
  [15]
- Add the missing context-analysis annotations on the region quiesce
  lock and unlock and on __cxl_reset_execute() (Sashiko) [15]
- Use an explicit pci_dev_put() in read_cdat_data() instead of mixing
  __free() with goto cleanup (Sashiko) [15]
- Check for VFs and multifunction devices before the CXL DVSEC lookups in
  the two reset methods, to avoid repeated capability walks while probing
  VFs (Joshua) [18]
- Fold the CXL bus reset cache flush patch into this series (Dave); the
  series is now 19 patches
- Added Richard's Reviewed-by tags to patches 6-10, 14 and 16
- Specified the reviewer behind each change in the changelogs, per Alison [14]

Earlier revision history is in the v16 [1] and v15 [2] cover letters.

Attribution for Dave Jiang's reference branch is still open.
Since I am unsure about [16] I have not yet added the right tags.
The patches link to his branch below the separator for now.

The v15 discussion on the scope check is still pending [17].

HDM caching requires CXL.cache, CXL.mem, CXL Reset and Reset Memory
Clear capabilities. CXL bus reset state restoration uses that cache.

CXL Reset requires a registered CPU cache clean/invalidate provider
and matching firmware, such as the separate SMCCC provider series [3].
Without that provider, cxl_reset can be advertised while execution returns
-ENOTTY, unless the test-only CXL_REGION_INVALIDATION_TEST bypass is
enabled.

The KUnit suite checks HDM range boundaries, zero-size and uncommitted
state, lock and target-type decoding, and 12-way interleave.

Testing: v17 was tested on an ARM64 target with v5 of the SMCCC
cache provider series [3] applied. Reset loops passed for both cxl_reset and
cxl_bus after running bidirectional host-device traffic. The
cxl_hdm_unpack_kunit suite passes.

[1] https://lore.kernel.org/linux-cxl/20261029145703.2185231-1-smadhavan@nvidia.com/
[2] https://lore.kernel.org/linux-cxl/20261011021422.3428136-1-smadhavan@nvidia.com/
[3] https://lore.kernel.org/linux-arm-kernel/20261021224557.1827361-1-smadhavan@nvidia.com/
[4] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[5] https://lore.kernel.org/all/cover.1791949910.git.alison.schofield@intel.com/
[6] https://lore.kernel.org/linux-cxl/d581f2a0-dd9e-4d0a-bd19-639c1bd9b832@zohomail.com/
[7] https://lore.kernel.org/linux-cxl/20261011021422.3428136-1-smadhavan@nvidia.com/T/#m7f8d0e51d4cfe09fefcebd93d49dcb916be9198a
[8] https://lore.kernel.org/linux-cxl/auQC2UECQ8vaelPz@MWDK4CY14F/
[9] https://lore.kernel.org/linux-cxl/auQRSOGWEDzB_qjF@MWDK4CY14F/
[10] https://lore.kernel.org/linux-cxl/auRhJa-nzR0CnKf3@MWDK4CY14F/
[11] https://lore.kernel.org/linux-cxl/auRQN6a3QVtqjd5a@MWDK4CY14F/
[12] https://lore.kernel.org/linux-cxl/auQ27s1rSCcbdRJ3@MWDK4CY14F/
[13] https://lore.kernel.org/linux-cxl/auAg6MFDGyVQvQxM@aschofie-mobl2.lan/
[14] https://lore.kernel.org/linux-cxl/auOXW3BTb8CYIh4m@aschofie-mobl2.lan/
[15] https://sashiko.dev/#/patchset/20261029145703.2185231-1-smadhavan@nvidia.com
[16] https://lore.kernel.org/linux-cxl/20261011021422.3428136-1-smadhavan@nvidia.com/T/#m2f61467417f422316cb15b976773944dcdcb48c7
[17] https://lore.kernel.org/linux-cxl/20261011021422.3428136-1-smadhavan@nvidia.com/T/#m4f62ed0f038f2d136158c74c1c6c338129266131
[18] https://lore.kernel.org/linux-cxl/20261105143611.31563-1-machaujo@amazon.de/

Srirangan Madhavan (19):
  cxl: Clarify decoder interleave limit
  cxl: Share CXL port upstream PCI device lookup
  cxl: Move decoder declarations to shared header
  cxl: Embed decoder configuration in a standalone structure
  cxl: Introduce endpoint HDM decoder settings
  cxl: Move HDM decoder helpers to built-in code
  cxl: Share HDM decoder register unpacking
  cxl: Move cxl_rwsem into built-in code
  cxl: Refresh cached PCI HDM decoder settings
  cxl: Build register block helpers into the kernel
  PCI/CXL: Cache endpoint HDM state during PCI enumeration
  PCI/CXL: Add CXL Device Reset sequencing
  cxl: Validate and synchronize HDM ranges around reset
  PCI/CXL: Reject reset with unsafe function scope
  PCI/CXL: Restore CXL state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  PCI/CXL: Restore CXL state after CXL bus reset
  PCI/CXL: Flush host and device caches for CXL bus reset
  cxl: Add KUnit checks for HDM decoder unpacking

 MAINTAINERS                          |   1 +
 drivers/cxl/.kunitconfig             |   4 +
 drivers/cxl/Kconfig                  |  18 +-
 drivers/cxl/Makefile                 |   1 +
 drivers/cxl/acpi.c                   |  26 +-
 drivers/cxl/core/Makefile            |   2 +-
 drivers/cxl/core/atl.c               |   4 +-
 drivers/cxl/core/core.h              |  15 +
 drivers/cxl/core/hdm.c               | 273 +++-----
 drivers/cxl/core/hdm_regs.c          | 437 +++++++++++++
 drivers/cxl/core/hdm_state.c         | 402 ++++++++++++
 drivers/cxl/core/pci.c               |  85 ++-
 drivers/cxl/core/port.c              |  68 +-
 drivers/cxl/core/region.c            | 116 ++--
 drivers/cxl/core/regs.c              |   9 +
 drivers/cxl/cxl.h                    |  46 +-
 drivers/cxl/pci.c                    |   2 +-
 drivers/cxl/port.c                   |   4 +-
 drivers/cxl/tests/Makefile           |   5 +
 drivers/cxl/tests/hdm_unpack_kunit.c |  91 +++
 drivers/pci/Makefile                 |   1 +
 drivers/pci/bus.c                    |   1 +
 drivers/pci/cxl.c                    | 907 +++++++++++++++++++++++++++
 drivers/pci/pci.c                    |  27 +-
 drivers/pci/pci.h                    |  44 ++
 drivers/pci/probe.c                  |   1 +
 include/cxl/cxl.h                    |  59 +-
 include/cxl/hdm.h                    | 109 ++++
 include/cxl/pci.h                    |  13 +
 include/linux/pci.h                  |   8 +-
 include/uapi/linux/pci_regs.h        |  14 +
 tools/testing/cxl/Kbuild             |   1 -
 tools/testing/cxl/test/cxl.c         | 158 +++--
 33 files changed, 2526 insertions(+), 426 deletions(-)
 create mode 100644 drivers/cxl/.kunitconfig
 create mode 100644 drivers/cxl/core/hdm_regs.c
 create mode 100644 drivers/cxl/core/hdm_state.c
 create mode 100644 drivers/cxl/tests/Makefile
 create mode 100644 drivers/cxl/tests/hdm_unpack_kunit.c
 create mode 100644 drivers/pci/cxl.c
 create mode 100644 include/cxl/hdm.h


base-commit: fef22d37d47043b63e4007f2f5fe762fa21fae0c
-- 
2.43.0