[PATCH v14 00/16] PCI/CXL: Add CXL reset support for Type 2 devices

[Srirangan Madhavan <smadhavan@nvidia.com>]

Hi folks!

This series adds CXL Reset support for CXL Type 2 devices through the
existing PCI reset_method ABI. The reset sequence follows the CXL 4.0
specification [1], including CXL.cache disable, optional cache
writeback, CXL Reset Memory Clear, ResetComplete polling, and ResetError
reporting.

The userspace ABI is the existing PCI reset interface:

    /sys/bus/pci/devices/.../reset_method
    /sys/bus/pci/devices/.../reset

Userspace can select "cxl_reset" in reset_method and then trigger reset
through the existing reset attribute.

CXL Reset is exposed conservatively. The target must be a
single-function CXL Type 2 device with no enabled VFs, CXL.cache and
CXL.mem must be enabled, and CXL Reset Memory Clear must be supported.
The kernel must also have cached HDM state sufficient to validate
affected ranges and restore decoder programming.

For reset through the sysfs reset attribute, the generic PCI path calls
the driver's reset_prepare() callback when provided and disables PCI I/O,
memory decoding, and bus mastering. The CXL reset path then temporarily
reserves each affected HDM range exclusively for the duration of the
reset, synchronizes CPU and device caches, and blocks IOMMU translations
where applicable. It restores HDM decoder state and the CXL Device DVSEC
Control register before PCI operation resumes. The path does not offline
memory or revoke mappings; it relies on CXL.mem users retaining
conflicting iomem resources while access remains active.

Following Dan's v6 feedback, this uses the existing PCI reset_method
interface instead of adding a memdev-specific sysfs ABI.

v14 changes from v13 [2]:
- Rework the decoder preparation into smaller reviewable patches. Move
  shared declarations first, embed the programming fields in a standalone
  configuration structure, introduce reusable settings, and then move the
  resulting helpers to built-in resource code.
- Simplify the referenced upstream PCI-device lookup and document the
  changed decoder field path.
- Add a separate check for overflowing HDM decoder ranges.
- Keep the early PCI HDM cache synchronized as decoders are enumerated,
  committed, or reset, and keep cache initialization failures from
  blocking PCI enumeration.
- Build and export only the helpers needed when CXL_BUS is modular, and
  preserve the tools/testing/cxl module build.
- Split the reset sequence into cache handling, Memory Clear, completion
  polling, range synchronization, and state restoration steps. Preserve
  the original cache-disable policy across reset.
- Keep range reservations, IOMMU exclusion, and the post-restore CPU-cache
  invalidation in the reset critical section.
- Preserve the pre-reset PCI snapshot on failures before PCI state has
  been restored, so the outer PCI restore cannot replay post-reset state.
- Carry Reviewed-by tags for patches unchanged after review.

Patch overview:

Patch 1 removes the stale interleave-limit comment in isolation.

Patch 2 factors the referenced upstream PCI-device lookup out of the
existing CDAT path before the HDM cache becomes its second user.

Patch 3 moves shared decoder declarations to include/cxl/.

Patch 4 embeds decoder programming fields in a standalone configuration
structure.

Patch 5 introduces reusable decoder settings and separates programming
from commit completion polling.

Patch 6 moves the HDM programming helpers to built-in resource code for
use when cxl_core is modular.

Patch 7 shares HDM register unpacking between normal CXL enumeration and
early PCI discovery.

Patch 8 rejects HDM ranges that overflow the address space.

Patch 9 refreshes cached PCI HDM settings as decoder state changes.

Patch 10 caches endpoint HDM and DVSEC state during PCI enumeration using
direct BAR-relative Register Locator metadata.

Patch 11 implements the CXL Device Reset sequence and its Control2
cleanup and timeout handling.

Patch 12 reserves active HDM ranges and performs mandatory pre-reset and
post-restore CPU-cache invalidation.

Patch 13 rejects reset where function scope is unsafe, including
multifunction devices and PFs with enabled VFs.

Patch 14 restores cached global and per-decoder HDM state, decoder locks,
and the complete CXL Device DVSEC Control register with fail-closed error
handling.

Patch 15 exposes cxl_reset through the PCI reset-method table ahead of
FLR.

Patch 16 restores cached CXL state after cxl_bus reset while its existing
IOMMU exclusion remains active.

The CXL.cache/CXL.mem eligibility requirements remain intentionally
unchanged. Whether CXL.mem-only devices should use this reset method is
left for follow-up discussion rather than expanding this series.

Testing:
- Built the base and every patch incrementally with `W=1 vmlinux modules`
  on the for-7.4/cxl-zero-size-decoder topic tip.
- Built the tools/testing/cxl modules at every step with
  CONFIG_CXL_REGION_INVALIDATION_TEST enabled for the build check.
- The reset and cxl_bus restoration paths from the previous revision were
  boot-tested on arm64 hardware.

References:
[1] https://computeexpresslink.org/wp-content/uploads/2026/02/CXL-Specification_rev4p0_ver1p0_2026February26_clean_evalcopy_v2.pdf
[2] https://lore.kernel.org/linux-cxl/20260922083924.2451158-1-smadhavan@nvidia.com/

Srirangan Madhavan (16):
  cxl: Drop stale decoder interleave limit comment
  cxl: Share CXL port upstream PCI device lookup
  cxl: Move decoder declarations to shared header
  cxl: Embed decoder configuration in a standalone structure
  cxl: Introduce reusable HDM decoder settings
  cxl: Move HDM decoder helpers to built-in resource code
  cxl: Share HDM decoder register unpacking
  cxl: Reject overflowing HDM decoder ranges
  cxl: Refresh cached PCI HDM decoder settings
  cxl: Cache endpoint HDM state during PCI enumeration
  cxl: Add CXL Device Reset sequencing
  cxl: Validate and synchronize HDM ranges around reset
  PCI/CXL: Reject reset with unsafe function scope
  cxl: Restore CXL state after PCI reset
  PCI/CXL: Expose CXL Reset as a PCI reset method
  PCI/CXL: Restore CXL state after CXL bus reset

 drivers/cxl/Kconfig           |    4 +
 drivers/cxl/acpi.c            |   26 +-
 drivers/cxl/core/Makefile     |    2 +-
 drivers/cxl/core/atl.c        |    4 +-
 drivers/cxl/core/core.h       |    8 +
 drivers/cxl/core/hdm.c        |  301 ++++----
 drivers/cxl/core/pci.c        |   42 +-
 drivers/cxl/core/port.c       |   68 +-
 drivers/cxl/core/region.c     |  102 +--
 drivers/cxl/core/regs.c       |    9 +
 drivers/cxl/core/resource.c   | 1320 +++++++++++++++++++++++++++++++++
 drivers/cxl/cxl.h             |   43 +-
 drivers/cxl/pci.c             |    2 +-
 drivers/cxl/port.c            |    4 +-
 drivers/pci/bus.c             |    2 +
 drivers/pci/pci.c             |   13 +
 drivers/pci/probe.c           |    2 +
 include/cxl/cxl.h             |  126 +++-
 include/linux/pci.h           |    8 +-
 include/uapi/linux/pci_regs.h |   14 +
 tools/testing/cxl/Kbuild      |    1 -
 tools/testing/cxl/test/cxl.c  |  158 ++--
 22 files changed, 1853 insertions(+), 406 deletions(-)
 create mode 100644 drivers/cxl/core/resource.c


base-commit: fef22d37d47043b63e4007f2f5fe762fa21fae0c
-- 
2.43.0