[PATCH v5 00/27] vfio/pci: Add CXL Type-2 device passthrough support

[PATCH v5 00/27] vfio/pci: Add CXL Type-2 device passthrough support

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

This series adds VFIO passthrough for CXL Type-2 accelerators. The guest
drives its own virtual HDM decoder and can reset the device. The host
owns the physical decoder and the host physical address the memory lands
at. The guest only picks a guest physical address. This series targets a
single, non-interleaved, firmware-committed endpoint decoder.


Base and dependencies
---------------------

  Base: Linux 7.3-rc1 (cee9395acd80), plus the cxl-reset dependency:
  - CXL reset core from Srirangan's cxl_reset series (v12) [1], which
    caches the endpoint decoder settings in pdev->hdm and keeps the HDM
    decoder and reset helpers in drivers/cxl/core.
  - This series adds a function-scoped reset entry
    (cxl_reset_dvsec_sequence) on top and drives it at the vfio reset
    points.
  - Prerequisite commits from Alejandro's type-2 device support and
    Dan's devm_cxl_probe_mem are already upstreamed in this kernel
    release and those series are not listed as dependencies now.


Changes since v4
----------------

v4 [2] introduced the vfio-cxl provider: the register emulation lives in
a separate module that vfio-pci-core loads on demand, and cxl-core keeps
only the reset entry and a few enabling helpers. v5 keeps that model and
folds the v4 review, reshaping the region and reset handling. The patch
count is unchanged at 27; several patches were split, a few were dropped,
and the composition changed.

Structural
  - The v4 "expose the HDM memory and trap the decoder registers" patch is
    split into three: the mmap-able HDM memory region, memory-failure
    containment for that struct-page-less range, and the read-only
    decoder-register region.
  - "Guard HDM access on device state" is split into two: excluding the
    decoder block from the direct BAR, and clearing the HDM access gate
    after a secondary bus reset.
  - The decoder register block is read live; the static decoder snapshot
    and its refresh-after-reset patch are gone.
  - The CXL DVSEC is virtualized through the config-space permission hooks
    in vfio_pci_config.c; the separate DVSEC-shadow patch is gone.
  - The cxl-core media-readiness patch is dropped; a mailbox-less Type-2
    memdev sets media ready in the provider at bind (mirrors efx_cxl).
  - The cxl-core register-map rework is split into request/ioremap helpers
    plus an owned-resource record, replacing the v4 bar_owned flag.
  - The v4 resource.c include/export patch is dropped; the include ships in
    the cxl_reset base and the disputed re-export no longer exists.

Behavioral
  - The HDM decoder registers are served by live reads with guest writes
    absorbed; the host committed and locked the physical decoder, so the
    guest never drives it.
  - fd read/write of the HDM memory region is serviced by a device-state
    guarded write-back copy (Memory-Space enabled, decoder known-good, and
    media ready) and refused only while that gate is closed, rather than
    unconditionally.
  - The coherent CXL.mem range is exported as a dma-buf and mapped into the
    guest IOAS with IOMMU_IOAS_MAP_FILE, replacing the out-of-tree PFNMAP
    workaround v4 needed for device-side ATS.
  - Reset is wired through the pci_error_handlers .reset_prepare and
    .reset_done callbacks and the CXL reset method rather than open-coded;
    a bus hot reset no longer rejects sibling functions.
  - The excluded-range facility is generic (a list of bar/start/size/flags)
    and the existing MSI-X exclusion is migrated onto it.


Reviewer feedback addressed
---------------------------

The v4 [2] thread has the full discussion; this is where each objection
landed.

Alex Williamson
  - A generic excluded-range list replaces the CXL one-off; MSI-X is
    migrated onto it; an intersecting host read fills -1 and a write is
    dropped rather than erroring.
  - The component-register block is exposed as a live read-only view, not a
    static shadow; the refresh-after-reset patch is eliminated.
  - The HDM memory fd read/write is device-state guarded rather than a bare
    -EIO.
  - Error containment and the decoder-register region are their own patches.
  - Reset uses .reset_prepare/.reset_done; the multifunction/sibling
    hot-reset rejection is dropped; the VM-centric framing is removed.
  - CXL init failure is non-fatal and falls back to plain vfio-pci;
    disable_cxl is a convenience opt-out, not the failure path.
  - The DVSEC virtualization lives in vfio_pci_config.c as an ecap perm map;
    the config path takes no CXL module dependency.
  - Both regions use VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id.
  - Kconfig/Makefile ordering fixed; the .open_device/.close_device hook
    names are kept to mirror vfio_device_ops; the bind-time rejects log
    consistently.

Dave Jiang
  - Component-register ownership uses the owned-resource API, not a
    bar_owned flag.
  - Media readiness is set in the provider at bind (efx_cxl precedent); the
    cxl-core media-readiness change is dropped.
  - The component register defines live in uapi/cxl/cxl_regs.h and the
    message names the selftest as the consumer.
  - The cheap topology rejects are grouped before the range computation.

Jonathan Cameron
  - The v4 resource.c include/export patch is gone: the include arrives with
    the cxl_reset base and the re-export it objected to no longer exists.

Shuai Xue
  - The selftest maps the HDM range through the dma-buf + IOMMU_IOAS_MAP_FILE
    path instead of failing a VA-based IOAS map.
  - The guest cache write-back-invalidate doorbell completes in the shadow;
    the host runs the real WBI inside the reset sequence.


Patch order
-----------

The patches are ordered so the tree builds at every commit and each change
sits next to the code it depends on. Five groups:

  Part 1  CXL core (patches 1-4)
    - Split the BAR block request and ioremap helpers, and let a BAR-owning
      driver own the component register block, so the HDM/RAS sub-blocks are
      left unclaimed for vfio-cxl.
    - Move the component register defines to include/uapi/cxl/cxl_regs.h so a
      VMM (and the selftest) can consume them.
    - Add the function-scoped cxl_reset_dvsec_sequence() for vfio-pci.

  Part 2  vfio-pci-core enabling (patches 5-14)
    - The CXL provider ops registration interface, on-demand provider load,
      -EPROBE_DEFER handling for the built-in case, and the non-fatal
      fallback to plain vfio-pci when CXL init fails.
    - A generic excluded-range list and the MSI-X migration onto it.
    - The CXL DVSEC virtualization in config space, the open/close hooks, the
      reset brackets, and the disable_cxl opt-out.

  Part 3  vfio-cxl provider and HDM regions (patches 15-24)
    - The vfio-cxl module, the CXL memdev created at bind, and ownership of
      the whole component BAR.
    - The mmap-able HDM memory region, memory-failure containment for it, and
      the read-only decoder-register region.
    - Excluding the decoder block from the direct BAR, the hot-reset access
      gate, the device/decoder geometry cap, and the dma-buf export.

  Part 4  vfio-cxl reset (patch 25)
    - Run the CXL DVSEC reset at every vfio reset point.

  Part 5  Documentation and selftests (patches 26-27)


Subsystem boundary
------------------

vfio-pci-core does not implement CXL registers. vfio-cxl is a separate
module that registers a struct vfio_cxl_ops at init. vfio-pci-core loads it
on demand for a CXL device (request_module plus pcie_is_cxl) and pins it
per bound device. If a modular provider is missing or fails to load, the
device is driven as plain vfio-pci. The bind only defers for the built-in
initcall-order case, where request_module cannot help.

The core exposes only the primitives that have to live in core (memory_lock,
mapping revoke, dma-buf quiesce, and the BAR sub-range exclusion). The
CXL-specific work stays behind the ops.


Memory ownership
----------------

The host resolves the host physical address once, at bind, through
devm_cxl_probe_mem(). The memdev is owned for the bind lifetime and torn
down at unbind. The HPA range is claimed IORESOURCE_EXCLUSIVE so no
mismatched cacheable alias can form, including one mapped through /dev/mem.

The guest programs a guest physical address into a trapped virtual decoder
and polls a shadow for commit. It never reaches the physical decoder
registers: those are served only through the live-read trap and are
excluded from the direct BAR mapping, so the guest cannot move the host
physical window.


HDM region access
-----------------

The HDM region carries the coherent device memory. A VMM mmaps it and maps
it into the guest through stage-2; that is the primary access path. The
region advertises READ and WRITE so a VMM can derive an accessible (non
PROT_NONE) mmap protection. fd read/write is serviced by a write-back copy
gated on the same device state as the fault path, and refused while that
gate is closed.

The fault path inserts the pfn only while the decoder is in a known-good
restored state, the device has PCI Memory-Space enabled, and the media is
ready, so a host access cannot reach a revoked or disabled decoder. The
fault inserts the HPA pfn at the largest aligned order, including 2 MB PMDs.
The struct-page-less range is registered with the memory-failure machinery
so a memory error is contained to unmapping the range and a SIGBUS rather
than a host SError.


DMA and iommufd
---------------

A Type-2 accelerator issues ATS-translated DMA to addresses inside its own
HDM window, so that range must be present in the guest IOAS. The HDM range
is struct-page-less coherent memory that a userspace-VA IOMMU_IOAS_MAP
cannot pin, so the HDM memory region is exportable as a dma-buf:
VFIO_DEVICE_FEATURE_DMA_BUF returns an fd that iommufd maps with
IOMMU_IOAS_MAP_FILE, without a VA or a page pin. The dma-buf is revoked
whenever the mapping is torn down, so a stale stage-2 mapping cannot outlive
the HDM window.


Reset
-----

A CXL Type-2 device is reset through its DVSEC sequence at every path that
can reset it, not through an FLR (an FLR would corrupt CXL.mem). The reset
runs at every vfio reset point through one shared helper: VM enable and
close, the VFIO_DEVICE_RESET ioctl, a config-space FLR, and a guest write of
Initiate_CXL_Reset in the CXL DVSEC. Under memory_lock, with the HDM mapping
and the dma-buf revoked, the host runs cxl_reset_dvsec_sequence(), which
always clears device memory on the v12 base and restores and re-samples the
firmware-committed decoder. The guest's cache write-back-invalidate doorbell
completes in the shadow and the real invalidate runs inside that sequence;
the reset outcome comes back through DVSEC STATUS2 for the guest to poll.

A CXL port masks Secondary Bus Reset by default, so a
VFIO_DEVICE_PCI_HOT_RESET does not reach the endpoint and the HDM state is
untouched; a hot reset no longer rejects sibling functions. If the port has
SBR unmasked the reset can decommit the decoder without restoring it, so the
reset_done handler gates HDM access; a later VFIO_DEVICE_RESET runs the CXL
reset sequence and restores it.

A bound CXL Type-2 device is kept out of idle D3. Powering a Type-2 function
down and back up reinitializes it and discards its coherent memory, which
only the accelerator's own driver re-initializes, so vfio-cxl sets
disable_idle_d3 to keep the device in D0 while it is bound.


Validation
----------

  - Each patch builds (drivers/cxl and drivers/vfio) and passes
    scripts/checkpatch.pl --codespell --strict with no errors, warnings, or
    checks.
  - The series applies in sequence on the stated base and dependencies.
  - The coherent CXL.mem range is mapped into the guest IOAS through the
    in-series dma-buf export (IOMMU_IOAS_MAP_FILE); the out-of-tree PFNMAP
    workaround v4 required for device-side ATS is no longer needed.
  - A selftest, tools/testing/selftests/vfio/vfio_cxl_type2_test.c,
    exercises region discovery, the sparse mmap, the huge-page fault, the
    dma-buf IOAS map, the live decoder reads, aligned/range access rejects,
    the commit/lock FSM, the DVSEC virtualization, and a real device reset.
  - Built and functionally tested against a CXL Type-2 device: guest boot,
    decoder commit and mapping, and guest-triggered CXL reset.


Follow-on UAPI
--------------

- The trapped component-register region spans the whole HDM decoder block,
  so a multi-decoder device needs no new region or cap: a VMM reads the
  decoder count and each committed base from every decoder in the block.
- The COMP_REGS geometry cap keeps a reserved field as a versioning anchor.
- Further trapped surfaces such as CXL RAS are planned as new CXL region
  subtypes rather than by extending this cap.
- The single committed, non-interleaved decoder is a bind-time policy in one
  place, not an ABI assumption, so multi-decoder and switched topologies
  relax only there.


Deferred
--------

  - Topology reach. Switched, multi-decoder, and interleaved decoders stay
    rejected at bind.
  - Non firmware-committed decoder support.


AI assistance disclosure
------------------------

This series was developed with substantial AI assistance (Anthropic Claude,
via the Claude Code CLI), used for design exploration and trade-off
analysis, multi-agent regression and adversarial review against the v4
feedback, and drafting the selftests and the documentation. Following
Documentation/process/coding-assistants.rst, each patch carries an
Assisted-by: LLM tag. Build, checkpatch, and on-hardware functional
testing were done by me, not the assistant. I have reviewed every change
and take full DCO responsibility for the series.


References
----------

[1] [PATCH v12 00/12] PCI/CXL: Add CXL reset support for Type 2 devices
https://lore.kernel.org/linux-cxl/20260910070808.1444264-1-smadhavan@nvidia.com/

[2] [PATCH v4 00/27] vfio/pci: Add CXL Type-2 device passthrough support
https://lore.kernel.org/linux-cxl/20260813093631.2288172-1-mhonap@nvidia.com


Manish Honap (27):
  cxl/regs: Split the BAR block request and ioremap helpers
  cxl/regs: Let a BAR-owning driver own the component register block
  cxl: Move component register defines to uapi/cxl/cxl_regs.h
  cxl: Add cxl_reset_dvsec_sequence() for vfio-pci
  vfio/pci: Add the CXL provider ops registration interface
  vfio/pci: Detect CXL devices and load the CXL provider on demand
  vfio/pci: Honor -EPROBE_DEFER from CXL provider probe
  vfio/pci: Fall back to plain vfio-pci when CXL init fails
  vfio/pci: Add a generic excluded-range list
  vfio/pci: Migrate MSI-X exclusion onto the generic excluded-range list
  vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c
  vfio/pci: Call the CXL open and close hooks around device use
  vfio/pci: Bracket PCI resets with the CXL reset hooks
  vfio/pci: Provide an opt-out for the CXL Type-2 extensions
  vfio/cxl: Add the vfio-cxl provider module skeleton
  vfio/cxl: Create the CXL memdev and set media ready at bind
  vfio/cxl: Own the whole component register BAR
  vfio/cxl: Expose the HDM memory region to the guest
  vfio/cxl: Contain HDM memory errors with memory_failure()
  vfio/cxl: Expose the HDM decoder registers read-only to the guest
  vfio/cxl: Exclude the HDM decoder registers from direct BAR access
  vfio/cxl: Clear the HDM access gate after a hot reset
  vfio/cxl: Describe the CXL device and decoder geometry to userspace
  vfio/cxl: Export the HDM memory region as a dma-buf
  vfio/cxl: Run the CXL reset at the vfio reset points
  Documentation: vfio-pci: Document CXL Type-2 device passthrough
  selftests/vfio: Add CXL Type-2 passthrough tests

 Documentation/driver-api/index.rst            |   1 +
 Documentation/driver-api/vfio-pci-cxl.rst     | 188 +++++
 MAINTAINERS                                   |  10 +
 drivers/cxl/core/regs.c                       |  35 +-
 drivers/cxl/core/resource.c                   |  53 ++
 drivers/cxl/cxl.h                             |  47 +-
 drivers/vfio/pci/Kconfig                      |   2 +
 drivers/vfio/pci/Makefile                     |   2 +
 drivers/vfio/pci/cxl/Kconfig                  |  11 +
 drivers/vfio/pci/cxl/Makefile                 |   3 +
 drivers/vfio/pci/cxl/vfio_cxl_core.c          | 704 ++++++++++++++++
 drivers/vfio/pci/vfio_pci.c                   |  13 +
 drivers/vfio/pci/vfio_pci_config.c            | 177 +++-
 drivers/vfio/pci/vfio_pci_core.c              | 544 +++++++++++-
 drivers/vfio/pci/vfio_pci_dmabuf.c            |  27 +-
 drivers/vfio/pci/vfio_pci_priv.h              |  10 +
 drivers/vfio/pci/vfio_pci_rdwr.c              |  50 +-
 include/cxl/cxl.h                             |  14 +
 include/cxl/pci.h                             |   3 +
 include/linux/vfio_pci_core.h                 |  40 +
 include/uapi/cxl/cxl_regs.h                   |  53 ++
 include/uapi/linux/vfio.h                     |  24 +
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../vfio/lib/include/libvfio/iommu.h          |   3 +
 tools/testing/selftests/vfio/lib/iommu.c      |  29 +
 .../selftests/vfio/lib/vfio_pci_device.c      |  57 +-
 .../selftests/vfio/vfio_cxl_type2_test.c      | 780 ++++++++++++++++++
 27 files changed, 2789 insertions(+), 92 deletions(-)
 create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst
 create mode 100644 drivers/vfio/pci/cxl/Kconfig
 create mode 100644 drivers/vfio/pci/cxl/Makefile
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c
 create mode 100644 include/uapi/cxl/cxl_regs.h
 create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c

-- 
2.25.1

[PATCH v5 01/27] cxl/regs: Split the BAR block request and ioremap helpers

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

devm_cxl_iomap_block() claims a register block with
devm_request_mem_region() and then maps it with devm_ioremap().

A driver that already owns the parent BAR needs to skip the sub-block
request while still mapping it, so the request and the map must be
callable independently.

Split them into devm_cxl_request_block() and devm_cxl_ioremap_block().
devm_cxl_iomap_block() keeps calling both in turn, so there is no
functional change.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/cxl/core/regs.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
index 989e79383b99..d79550dbb484 100644
--- a/drivers/cxl/core/regs.c
+++ b/drivers/cxl/core/regs.c
@@ -177,6 +177,20 @@ void cxl_probe_device_regs(struct device *dev, void __iomem *base,
 }
 EXPORT_SYMBOL_NS_GPL(cxl_probe_device_regs, "CXL");
 
+static struct resource *devm_cxl_request_block(struct device *dev,
+					       resource_size_t addr,
+					       resource_size_t length)
+{
+	return devm_request_mem_region(dev, addr, length, dev_name(dev));
+}
+
+static void __iomem *devm_cxl_ioremap_block(struct device *dev,
+					    resource_size_t addr,
+					    resource_size_t length)
+{
+	return devm_ioremap(dev, addr, length);
+}
+
 void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
 				   resource_size_t length)
 {
@@ -186,7 +200,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
 	if (WARN_ON_ONCE(addr == CXL_RESOURCE_NONE))
 		return NULL;
 
-	res = devm_request_mem_region(dev, addr, length, dev_name(dev));
+	res = devm_cxl_request_block(dev, addr, length);
 	if (!res) {
 		resource_size_t end = addr + length - 1;
 
@@ -194,7 +208,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
 		return NULL;
 	}
 
-	ret_val = devm_ioremap(dev, addr, length);
+	ret_val = devm_cxl_ioremap_block(dev, addr, length);
 	if (!ret_val)
 		dev_err(dev, "Failed to map region %pr\n", res);
 
-- 
2.25.1

[PATCH v5 02/27] cxl/regs: Let a BAR-owning driver own the component register block

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

cxl_map_component_regs() claims each mapped sub-block with
devm_request_mem_region(). A driver that already requested the whole
component register BAR, such as vfio-cxl, would then collide with that
claim and fail to map the HDM decoder and RAS blocks.

Add cxl_reg_map_add_owned_resource() so such a driver records the
resource it already owns on the register map, and skip the sub-block
request when the block falls within an owned resource.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/cxl/core/regs.c | 17 +++++++++++++++--
 include/cxl/cxl.h       |  2 ++
 include/cxl/pci.h       |  3 +++
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
index d79550dbb484..58a7c5cafb45 100644
--- a/drivers/cxl/core/regs.c
+++ b/drivers/cxl/core/regs.c
@@ -216,6 +216,13 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
 }
 EXPORT_SYMBOL_NS_GPL(devm_cxl_iomap_block, "CXL");
 
+void cxl_reg_map_add_owned_resource(struct cxl_register_map *map,
+				    struct resource *res)
+{
+	map->owned = res;
+}
+EXPORT_SYMBOL_NS_GPL(cxl_reg_map_add_owned_resource, "CXL");
+
 int cxl_map_component_regs(const struct cxl_register_map *map,
 			   struct cxl_component_regs *regs,
 			   unsigned long map_mask)
@@ -234,6 +241,7 @@ int cxl_map_component_regs(const struct cxl_register_map *map,
 		struct mapinfo *mi = &mapinfo[i];
 		resource_size_t addr;
 		resource_size_t length;
+		struct resource res;
 
 		if (!mi->rmap->valid)
 			continue;
@@ -241,8 +249,13 @@ int cxl_map_component_regs(const struct cxl_register_map *map,
 			continue;
 		addr = map->resource + mi->rmap->offset;
 		length = mi->rmap->size;
-		*(mi->addr) = devm_cxl_iomap_block(host, addr, length);
-		if (!*(mi->addr))
+		res = DEFINE_RES_MEM(addr, length);
+
+		if (map->owned && resource_contains(map->owned, &res))
+			*mi->addr = devm_cxl_ioremap_block(host, addr, length);
+		else
+			*mi->addr = devm_cxl_iomap_block(host, addr, length);
+		if (!*mi->addr)
 			return -ENOMEM;
 	}
 
diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
index a02792969e0a..92314f67b702 100644
--- a/include/cxl/cxl.h
+++ b/include/cxl/cxl.h
@@ -217,6 +217,7 @@ struct cxl_pmu_reg_map {
  * @resource: physical resource base of the register block
  * @max_size: maximum mapping size to perform register search
  * @reg_type: see enum cxl_regloc_type
+ * @owned: driver-owned resource whose sub-blocks are mapped without re-requesting
  * @component_map: cxl_reg_map for component registers
  * @device_map: cxl_reg_maps for device registers
  * @pmu_map: cxl_reg_maps for CXL Performance Monitoring Units
@@ -227,6 +228,7 @@ struct cxl_register_map {
 	resource_size_t resource;
 	resource_size_t max_size;
 	u8 reg_type;
+	struct resource *owned;
 	union {
 		struct cxl_component_reg_map component_map;
 		struct cxl_device_reg_map device_map;
diff --git a/include/cxl/pci.h b/include/cxl/pci.h
index 3e0000015871..8e0b3bd0b3c1 100644
--- a/include/cxl/pci.h
+++ b/include/cxl/pci.h
@@ -16,7 +16,10 @@ enum cxl_regloc_type {
 
 struct cxl_register_map;
 struct pci_dev;
+struct resource;
 
 int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type,
 		       struct cxl_register_map *map);
+void cxl_reg_map_add_owned_resource(struct cxl_register_map *map,
+				    struct resource *res);
 #endif
-- 
2.25.1

[PATCH v5 03/27] cxl: Move component register defines to uapi/cxl/cxl_regs.h

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The CXL component and HDM decoder register layout is spec-defined and is
needed by a userspace consumer that cannot include the private
drivers/cxl/cxl.h

The vfio-cxl selftest
(tools/testing/selftests/vfio/vfio_cxl_type2_test.c) builds against
installed uapi headers only. Move those defines to a new uapi header, the
way PCI register offsets live in uapi pci_regs.h, and include it from
drivers/cxl/cxl.h so existing in-kernel users are unchanged.

No functional change; the GENMASK and BIT forms are written as plain
values as a uapi header requirement.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 MAINTAINERS                 |  1 +
 drivers/cxl/cxl.h           | 47 +-------------------------------
 include/uapi/cxl/cxl_regs.h | 53 +++++++++++++++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 46 deletions(-)
 create mode 100644 include/uapi/cxl/cxl_regs.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 3a19da74d00c..2037501cd621 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6573,6 +6573,7 @@ F:	Documentation/driver-api/cxl
 F:	Documentation/userspace-api/fwctl/fwctl-cxl.rst
 F:	drivers/cxl/
 F:	include/cxl/
+F:	include/uapi/cxl/
 F:	include/uapi/linux/cxl_mem.h
 F:	tools/testing/cxl/
 
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 03ce50a6a06e..43c30e08df86 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -13,6 +13,7 @@
 #include <linux/io.h>
 #include <linux/range.h>
 #include <cxl/cxl.h>
+#include <cxl/cxl_regs.h>
 
 extern const struct nvdimm_security_ops *cxl_security_ops;
 
@@ -24,52 +25,6 @@ extern const struct nvdimm_security_ops *cxl_security_ops;
  * (port-driver, region-driver, nvdimm object-drivers... etc).
  */
 
-/* CXL 2.0 8.2.4 CXL Component Register Layout and Definition */
-#define CXL_COMPONENT_REG_BLOCK_SIZE SZ_64K
-
-/* CXL 2.0 8.2.5 CXL.cache and CXL.mem Registers*/
-#define CXL_CM_OFFSET 0x1000
-#define CXL_CM_CAP_HDR_OFFSET 0x0
-#define   CXL_CM_CAP_HDR_ID_MASK GENMASK(15, 0)
-#define     CM_CAP_HDR_CAP_ID 1
-#define   CXL_CM_CAP_HDR_VERSION_MASK GENMASK(19, 16)
-#define     CM_CAP_HDR_CAP_VERSION 1
-#define   CXL_CM_CAP_HDR_CACHE_MEM_VERSION_MASK GENMASK(23, 20)
-#define     CM_CAP_HDR_CACHE_MEM_VERSION 1
-#define   CXL_CM_CAP_HDR_ARRAY_SIZE_MASK GENMASK(31, 24)
-#define CXL_CM_CAP_PTR_MASK GENMASK(31, 20)
-
-#define   CXL_CM_CAP_CAP_ID_RAS 0x2
-#define   CXL_CM_CAP_CAP_ID_HDM 0x5
-#define   CXL_CM_CAP_CAP_HDM_VERSION 1
-
-/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */
-#define CXL_HDM_DECODER_CAP_OFFSET 0x0
-#define   CXL_HDM_DECODER_COUNT_MASK GENMASK(3, 0)
-#define   CXL_HDM_DECODER_TARGET_COUNT_MASK GENMASK(7, 4)
-#define   CXL_HDM_DECODER_INTERLEAVE_11_8 BIT(8)
-#define   CXL_HDM_DECODER_INTERLEAVE_14_12 BIT(9)
-#define   CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY BIT(11)
-#define   CXL_HDM_DECODER_INTERLEAVE_16_WAY BIT(12)
-#define CXL_HDM_DECODER_CTRL_OFFSET 0x4
-#define   CXL_HDM_DECODER_ENABLE BIT(1)
-#define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10)
-#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET(i) (0x20 * (i) + 0x14)
-#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET(i) (0x20 * (i) + 0x18)
-#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(i) (0x20 * (i) + 0x1c)
-#define CXL_HDM_DECODER0_CTRL_OFFSET(i) (0x20 * (i) + 0x20)
-#define   CXL_HDM_DECODER0_CTRL_IG_MASK GENMASK(3, 0)
-#define   CXL_HDM_DECODER0_CTRL_IW_MASK GENMASK(7, 4)
-#define   CXL_HDM_DECODER0_CTRL_LOCK BIT(8)
-#define   CXL_HDM_DECODER0_CTRL_COMMIT BIT(9)
-#define   CXL_HDM_DECODER0_CTRL_COMMITTED BIT(10)
-#define   CXL_HDM_DECODER0_CTRL_COMMIT_ERROR BIT(11)
-#define   CXL_HDM_DECODER0_CTRL_HOSTONLY BIT(12)
-#define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24)
-#define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28)
-#define CXL_HDM_DECODER0_SKIP_LOW(i) CXL_HDM_DECODER0_TL_LOW(i)
-#define CXL_HDM_DECODER0_SKIP_HIGH(i) CXL_HDM_DECODER0_TL_HIGH(i)
-
 /* HDM decoder control register constants CXL 3.0 8.2.5.19.7 */
 #define CXL_DECODER_MIN_GRANULARITY 256
 #define CXL_DECODER_MAX_ENCODED_IG 6
diff --git a/include/uapi/cxl/cxl_regs.h b/include/uapi/cxl/cxl_regs.h
new file mode 100644
index 000000000000..15a5a1930c75
--- /dev/null
+++ b/include/uapi/cxl/cxl_regs.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* Copyright (c) 2026 NVIDIA Corporation & Affiliates */
+
+#ifndef _UAPI_CXL_REGS_H_
+#define _UAPI_CXL_REGS_H_
+
+/* CXL 2.0 8.2.4 CXL Component Register Layout and Definition */
+#define CXL_COMPONENT_REG_BLOCK_SIZE 0x10000
+
+/* CXL 2.0 8.2.5 CXL.cache and CXL.mem Registers */
+#define CXL_CM_OFFSET 0x1000
+#define CXL_CM_CAP_HDR_OFFSET 0x0
+#define   CXL_CM_CAP_HDR_ID_MASK 0xffff
+#define     CM_CAP_HDR_CAP_ID 1
+#define   CXL_CM_CAP_HDR_VERSION_MASK 0xf0000
+#define     CM_CAP_HDR_CAP_VERSION 1
+#define   CXL_CM_CAP_HDR_CACHE_MEM_VERSION_MASK 0xf00000
+#define     CM_CAP_HDR_CACHE_MEM_VERSION 1
+#define   CXL_CM_CAP_HDR_ARRAY_SIZE_MASK 0xff000000
+#define CXL_CM_CAP_PTR_MASK 0xfff00000
+
+#define   CXL_CM_CAP_CAP_ID_RAS 0x2
+#define   CXL_CM_CAP_CAP_ID_HDM 0x5
+#define   CXL_CM_CAP_CAP_HDM_VERSION 1
+
+/* HDM decoders CXL 2.0 8.2.5.12 CXL HDM Decoder Capability Structure */
+#define CXL_HDM_DECODER_CAP_OFFSET 0x0
+#define   CXL_HDM_DECODER_COUNT_MASK 0xf
+#define   CXL_HDM_DECODER_TARGET_COUNT_MASK 0xf0
+#define   CXL_HDM_DECODER_INTERLEAVE_11_8 0x100
+#define   CXL_HDM_DECODER_INTERLEAVE_14_12 0x200
+#define   CXL_HDM_DECODER_INTERLEAVE_3_6_12_WAY 0x800
+#define   CXL_HDM_DECODER_INTERLEAVE_16_WAY 0x1000
+#define CXL_HDM_DECODER_CTRL_OFFSET 0x4
+#define   CXL_HDM_DECODER_ENABLE 0x2
+#define CXL_HDM_DECODER0_BASE_LOW_OFFSET(i) (0x20 * (i) + 0x10)
+#define CXL_HDM_DECODER0_BASE_HIGH_OFFSET(i) (0x20 * (i) + 0x14)
+#define CXL_HDM_DECODER0_SIZE_LOW_OFFSET(i) (0x20 * (i) + 0x18)
+#define CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(i) (0x20 * (i) + 0x1c)
+#define CXL_HDM_DECODER0_CTRL_OFFSET(i) (0x20 * (i) + 0x20)
+#define   CXL_HDM_DECODER0_CTRL_IG_MASK 0xf
+#define   CXL_HDM_DECODER0_CTRL_IW_MASK 0xf0
+#define   CXL_HDM_DECODER0_CTRL_LOCK 0x100
+#define   CXL_HDM_DECODER0_CTRL_COMMIT 0x200
+#define   CXL_HDM_DECODER0_CTRL_COMMITTED 0x400
+#define   CXL_HDM_DECODER0_CTRL_COMMIT_ERROR 0x800
+#define   CXL_HDM_DECODER0_CTRL_HOSTONLY 0x1000
+#define CXL_HDM_DECODER0_TL_LOW(i) (0x20 * (i) + 0x24)
+#define CXL_HDM_DECODER0_TL_HIGH(i) (0x20 * (i) + 0x28)
+#define CXL_HDM_DECODER0_SKIP_LOW(i) CXL_HDM_DECODER0_TL_LOW(i)
+#define CXL_HDM_DECODER0_SKIP_HIGH(i) CXL_HDM_DECODER0_TL_HIGH(i)
+
+#endif /* _UAPI_CXL_REGS_H_ */
-- 
2.25.1

[PATCH v5 04/27] cxl: Add cxl_reset_dvsec_sequence() for vfio-pci

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

vfio-pci needs to run a CXL reset on a passed-through Type-2 device
without the host-memory handling in cxl_reset_function(): the memory
behind the decoder belongs to the guest, so the HDM range collection and
host CPU cache flush that the host path performs must be skipped.

Add cxl_reset_dvsec_sequence(), which runs the DVSEC reset under the
device lock and restores HDM state through
cxl_restore_state_after_pci_reset(), and cxl_reset_capable() to gate it on
a function-scoped reset. Both are exported to the vfio-cxl module so
vfio-pci core does not import the CXL namespace.

CXL Reset always clears memory, so there is no caller Memory Clear choice.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/cxl/core/resource.c | 53 +++++++++++++++++++++++++++++++++++++
 include/cxl/cxl.h           | 12 +++++++++
 2 files changed, 65 insertions(+)

diff --git a/drivers/cxl/core/resource.c b/drivers/cxl/core/resource.c
index d3491e20c1ca..d373df9aa127 100644
--- a/drivers/cxl/core/resource.c
+++ b/drivers/cxl/core/resource.c
@@ -1398,3 +1398,56 @@ int cxl_reset_function(struct pci_dev *pdev, bool probe)
 	cxl_pci_target_reset_done(pdev, &target_prepared);
 	return rc;
 }
+
+/* True when a function-scoped CXL reset is available for @pdev. */
+bool cxl_reset_capable(struct pci_dev *pdev)
+{
+	u16 cap;
+
+	if (cxl_reset_get_dvsec(pdev, &cap) < 0)
+		return false;
+
+	if (pdev->multifunction || pci_num_vf(pdev))
+		return false;
+
+	return cxl_reset_hdm_available(pdev);
+}
+EXPORT_SYMBOL_FOR_MODULES(cxl_reset_capable, "vfio-cxl");
+
+/*
+ * Run the DVSEC reset sequence and restore HDM state for a caller that owns
+ * device quiesce and PCI config save/restore, such as vfio-pci.
+ * The HDM range collection and host CPU cache flush that cxl_reset_function()
+ * performs for host-owned memory are skipped as that memory belongs to the
+ * guest.
+ */
+int cxl_reset_dvsec_sequence(struct pci_dev *pdev)
+{
+	bool target_prepared = false;
+	bool reset_initiated = false;
+	int dvsec;
+	int rc;
+	u16 cap;
+
+	dvsec = cxl_reset_get_dvsec(pdev, &cap);
+	if (dvsec < 0)
+		return dvsec;
+
+	if (pdev->multifunction || pci_num_vf(pdev))
+		return -ENOTTY;
+
+	if (!pci_dev_trylock(pdev))
+		return -EBUSY;
+
+	rc = cxl_reset_execute(pdev, &target_prepared, &reset_initiated, dvsec,
+			       cap);
+	if (!rc)
+		rc = cxl_restore_state_after_pci_reset(pdev);
+	else if (reset_initiated)
+		cxl_reset_save_disabled_state(pdev);
+
+	cxl_pci_target_reset_done(pdev, &target_prepared);
+	pci_dev_unlock(pdev);
+	return rc;
+}
+EXPORT_SYMBOL_FOR_MODULES(cxl_reset_dvsec_sequence, "vfio-cxl");
diff --git a/include/cxl/cxl.h b/include/cxl/cxl.h
index 92314f67b702..a28903fc2960 100644
--- a/include/cxl/cxl.h
+++ b/include/cxl/cxl.h
@@ -168,6 +168,8 @@ void pci_cxl_hdm_init(struct pci_dev *pdev);
 void pci_cxl_hdm_release(struct pci_dev *pdev);
 int cxl_restore_state_after_pci_reset(struct pci_dev *pdev);
 int cxl_reset_function(struct pci_dev *pdev, bool probe);
+bool cxl_reset_capable(struct pci_dev *pdev);
+int cxl_reset_dvsec_sequence(struct pci_dev *pdev);
 #else
 static inline void pci_cxl_hdm_init(struct pci_dev *pdev)
 {
@@ -186,6 +188,16 @@ static inline int cxl_reset_function(struct pci_dev *pdev, bool probe)
 {
 	return -ENOTTY;
 }
+
+static inline bool cxl_reset_capable(struct pci_dev *pdev)
+{
+	return false;
+}
+
+static inline int cxl_reset_dvsec_sequence(struct pci_dev *pdev)
+{
+	return -ENOTTY;
+}
 #endif
 
 struct cxl_reg_map {
-- 
2.25.1

[PATCH v5 05/27] vfio/pci: Add the CXL provider ops registration interface

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

vfio-pci-core must stay free of any CXL header dependency, so CXL Type-2
handling lives in a separate vfio-cxl module that plugs in a set of
callbacks.

Add the registration interface:
- vfio-cxl registers a single struct vfio_cxl_ops at module_init
- vfio-pci-core stores it under an rwsem.

A second registration is refused with -EBUSY.

The owner field will be used to pin vfio-cxl for the lifetime of each
bound CXL device.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 24 ++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    | 10 ++++++++++
 2 files changed, 34 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 6757054e9d87..2cc5dd20396c 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2178,6 +2178,30 @@ static void vfio_pci_vga_uninit(struct vfio_pci_core_device *vdev)
 					      VGA_RSRC_LEGACY_MEM);
 }
 
+static const struct vfio_cxl_ops *vfio_pci_cxl_ops;
+static DECLARE_RWSEM(vfio_pci_cxl_ops_rwsem);
+
+int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops)
+{
+	guard(rwsem_write)(&vfio_pci_cxl_ops_rwsem);
+
+	if (vfio_pci_cxl_ops)
+		return -EBUSY;
+
+	vfio_pci_cxl_ops = ops;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_register_cxl_ops);
+
+void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops)
+{
+	guard(rwsem_write)(&vfio_pci_cxl_ops_rwsem);
+
+	if (vfio_pci_cxl_ops == ops)
+		vfio_pci_cxl_ops = NULL;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_cxl_ops);
+
 int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 {
 	struct vfio_pci_core_device *vdev =
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9a1674c152aa..9fe0d1a3a370 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -66,6 +66,16 @@ struct vfio_pci_device_ops {
 			       size_t nr_ranges);
 };
 
+struct vfio_cxl_ops {
+	int	(*init)(struct vfio_pci_core_device *vdev);
+	void	(*release)(struct vfio_pci_core_device *vdev);
+	/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
+	struct module *owner;
+};
+
+int vfio_pci_core_register_cxl_ops(const struct vfio_cxl_ops *ops);
+void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops);
+
 #if IS_ENABLED(CONFIG_VFIO_PCI_DMABUF)
 int vfio_pci_core_fill_phys_vec(struct phys_vec *phys_vec,
 				struct vfio_region_dma_range *dma_ranges,
-- 
2.25.1

[PATCH v5 06/27] vfio/pci: Detect CXL devices and load the CXL provider on demand

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the
CXL core in unconditionally would bloat every vfio-pci setup. Detect a
CXL Type-2 device at bind and request_module("vfio-cxl") only then, then
hand the device to the registered ops.

Each bound CXL device pins the provider through
vfio_pci_get_cxl_ops() (try_module_get) and drops it with
vfio_pci_put_cxl_ops() at release, so vfio-cxl can unload once no CXL
device is bound.

If the provider is absent the device is driven as plain vfio-pci. A
built-in provider whose initcall has not run yet is waited for with
-EPROBE_DEFER; the deferred-probe machinery will retry the bind once the
provider registers.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 84 ++++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h    |  3 ++
 2 files changed, 87 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 2cc5dd20396c..9eede1e56ab5 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2202,6 +2202,84 @@ void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops)
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_cxl_ops);
 
+static const struct vfio_cxl_ops *vfio_pci_get_cxl_ops(void)
+{
+	guard(rwsem_read)(&vfio_pci_cxl_ops_rwsem);
+
+	if (vfio_pci_cxl_ops && try_module_get(vfio_pci_cxl_ops->owner))
+		return vfio_pci_cxl_ops;
+
+	return NULL;
+}
+
+static void vfio_pci_put_cxl_ops(const struct vfio_cxl_ops *ops)
+{
+	module_put(ops->owner);
+}
+
+/*
+ * A CXL Type-2 device advertises both CXL.cache and CXL.mem in its CXL DVSEC.
+ * pcie_is_cxl() is also true for Type-1 (cache only) and Type-3 (mem only)
+ * devices, which the vfio-cxl provider does not handle, so confirm the Type-2
+ * identity before engaging it.
+ */
+static bool vfio_pci_is_cxl_type2(struct pci_dev *pdev)
+{
+	u16 dvsec, cap;
+
+	if (!pcie_is_cxl(pdev))
+		return false;
+
+	dvsec = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
+					  PCI_DVSEC_CXL_DEVICE);
+	if (!dvsec)
+		return false;
+
+	if (pci_read_config_word(pdev, dvsec + PCI_DVSEC_CXL_CAP, &cap))
+		return false;
+
+	return (cap & PCI_DVSEC_CXL_CACHE_CAPABLE) &&
+		(cap & PCI_DVSEC_CXL_MEM_CAPABLE);
+}
+
+/*
+ * Load vfio-cxl on demand for a CXL Type-2 device and hand the device to its
+ * ops. If the provider is absent the device is driven as plain vfio-pci; a
+ * built-in provider whose initcall has not run yet is waited for with
+ * -EPROBE_DEFER.
+ */
+static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
+{
+	const struct vfio_cxl_ops *ops;
+	int ret;
+
+	if (!vfio_pci_is_cxl_type2(vdev->pdev))
+		return 0;
+
+	request_module("vfio-cxl");
+	ops = vfio_pci_get_cxl_ops();
+	if (!ops)
+		return IS_BUILTIN(CONFIG_VFIO_CXL) ? -EPROBE_DEFER : 0;
+
+	ret = ops->init(vdev);
+	if (ret) {
+		vfio_pci_put_cxl_ops(ops);
+		return ret;
+	}
+
+	vdev->cxl_ops = ops;
+	return 0;
+}
+
+static void vfio_pci_core_cxl_release(struct vfio_pci_core_device *vdev)
+{
+	if (!vdev->cxl_ops)
+		return;
+
+	vdev->cxl_ops->release(vdev);
+	vfio_pci_put_cxl_ops(vdev->cxl_ops);
+}
+
 int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 {
 	struct vfio_pci_core_device *vdev =
@@ -2223,6 +2301,10 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 	init_rwsem(&vdev->memory_lock);
 	xa_init(&vdev->ctx);
 
+	ret = vfio_pci_core_cxl_init(vdev);
+	if (ret)
+		return ret;
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_init_dev);
@@ -2232,6 +2314,8 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev)
 	struct vfio_pci_core_device *vdev =
 		container_of(core_vdev, struct vfio_pci_core_device, vdev);
 
+	vfio_pci_core_cxl_release(vdev);
+
 	mutex_destroy(&vdev->igate);
 	mutex_destroy(&vdev->ioeventfds_lock);
 	kfree(vdev->region);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 9fe0d1a3a370..7f3a2bcb5830 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -29,6 +29,7 @@ struct vfio_pci_core_device;
 struct vfio_pci_region;
 struct p2pdma_provider;
 struct dma_buf_attachment;
+struct vfio_cxl_state;
 
 struct vfio_pci_eventfd {
 	struct eventfd_ctx	*ctx;
@@ -109,6 +110,8 @@ struct vfio_pci_core_device {
 	struct vfio_device	vdev;
 	struct pci_dev		*pdev;
 	const struct vfio_pci_device_ops *pci_ops;
+	const struct vfio_cxl_ops *cxl_ops;
+	struct vfio_cxl_state   *cxl;
 	void __iomem		*barmap[PCI_STD_NUM_BARS];
 	bool			bar_mmap_supported[PCI_STD_NUM_BARS];
 	/* Flags modified at runtime - dedicated storage unit */
-- 
2.25.1

[PATCH v5 07/27] vfio/pci: Honor -EPROBE_DEFER from CXL provider probe

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The CXL provider .init() can return -EPROBE_DEFER when the device is not
ready yet, for example when its CXL port has not enumerated. Handle that
return separately from other init failures and propagate it, so the bind
is retried later.

This keeps the deferred path distinct from the general failure path that
a following patch makes non-fatal: an -EPROBE_DEFER must retry, not fall
back to plain vfio-pci.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 9eede1e56ab5..a63a4f5228b8 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2262,6 +2262,15 @@ static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
 		return IS_BUILTIN(CONFIG_VFIO_CXL) ? -EPROBE_DEFER : 0;
 
 	ret = ops->init(vdev);
+	/*
+	 * A provider that is not ready yet (for example its CXL port has not
+	 * enumerated) returns -EPROBE_DEFER. Propagate it so the bind retries
+	 * rather than falling back to plain vfio-pci.
+	 */
+	if (ret == -EPROBE_DEFER) {
+		vfio_pci_put_cxl_ops(ops);
+		return ret;
+	}
 	if (ret) {
 		vfio_pci_put_cxl_ops(ops);
 		return ret;
-- 
2.25.1

[PATCH v5 08/27] vfio/pci: Fall back to plain vfio-pci when CXL init fails

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 device is also a valid plain vfio-pci device. If the
vfio-cxl provider cannot bring it up as Type-2, failing the bind would
regress a device that previously worked as plain vfio-pci.

Make a non-deferred CXL init failure non-fatal: log it, drop the provider
reference, leave cxl_ops NULL, and continue driving the device as plain
vfio-pci. The -EPROBE_DEFER path added earlier still retries rather than
falling back.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index a63a4f5228b8..9a75c30b67e2 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2271,9 +2271,17 @@ static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
 		vfio_pci_put_cxl_ops(ops);
 		return ret;
 	}
+	/*
+	 * Any other failure is non-fatal: a CXL device that cannot be brought
+	 * up as Type-2 still works as plain vfio-pci, so log and continue
+	 * rather than failing the bind.
+	 */
 	if (ret) {
+		pci_warn(vdev->pdev,
+			 "CXL init failed (%d), continuing as plain vfio-pci\n",
+			 ret);
 		vfio_pci_put_cxl_ops(ops);
-		return ret;
+		return 0;
 	}
 
 	vdev->cxl_ops = ops;
-- 
2.25.1

[PATCH v5 09/27] vfio/pci: Add a generic excluded-range list

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The MSI-X table is virtualized in vfio_pci_bar_rw() by an open-coded
x_start/x_end window that fills reads with -1 and drops writes. Now that
a generic excluded-range list expresses the same fill/drop behavior,
register the MSI-X table as a read and write excluded range instead of
special-casing it in the read/write path.

Add the range when the MSI-X capability is parsed in
vfio_pci_core_enable() and clear the list in vfio_pci_core_disable()
alongside the config teardown. The read/write path now relies solely on
vfio_pci_bar_find_exclusion(), so MSI-X and a provider's (e.g. vfio-cxl)
trapped registers share one mechanism.

No functional change.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 209 +++++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h |   9 ++
 drivers/vfio/pci/vfio_pci_rdwr.c |   8 ++
 include/linux/vfio_pci_core.h    |  14 +++
 4 files changed, 240 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 9a75c30b67e2..9e4fa5d088a4 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -24,6 +24,7 @@
 #include <linux/pci.h>
 #include <linux/pm_runtime.h>
 #include <linux/slab.h>
+#include <linux/sort.h>
 #include <linux/types.h>
 #include <linux/uaccess.h>
 #include <linux/vgaarb.h>
@@ -1012,6 +1013,204 @@ static int msix_mmappable_cap(struct vfio_pci_core_device *vdev,
 	return vfio_info_add_capability(caps, &header, sizeof(header));
 }
 
+struct vfio_pci_excluded_range {
+	struct list_head	entry;
+	int			bar;
+	u64			start;
+	u64			size;
+	u32			flags;
+};
+
+int vfio_pci_core_add_excluded_range(struct vfio_pci_core_device *vdev, int bar,
+				     u64 start, u64 size, u32 flags)
+{
+	struct vfio_pci_excluded_range *range;
+
+	range = kzalloc_obj(*range);
+	if (!range)
+		return -ENOMEM;
+
+	range->bar = bar;
+	range->start = start;
+	range->size = size;
+	range->flags = flags;
+	list_add_tail(&range->entry, &vdev->excluded_ranges);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_add_excluded_range);
+
+static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_excluded_range *range, *tmp;
+
+	list_for_each_entry_safe(range, tmp, &vdev->excluded_ranges, entry) {
+		list_del(&range->entry);
+		kfree(range);
+	}
+}
+
+bool vfio_pci_bar_find_exclusion(struct vfio_pci_core_device *vdev, int bar,
+				 loff_t pos, size_t count, bool iswrite,
+				 size_t *x_start, size_t *x_end)
+{
+	u32 want = iswrite ? VFIO_PCI_EXCLUDE_WRITE : VFIO_PCI_EXCLUDE_READ;
+	struct vfio_pci_excluded_range *range;
+	bool found = false;
+
+	list_for_each_entry(range, &vdev->excluded_ranges, entry) {
+		if (range->bar != bar || !(range->flags & want))
+			continue;
+		if (pos < range->start + range->size &&
+		    pos + count > range->start) {
+			/*
+			 * A BAR can carry more than one excluded window (e.g.
+			 * the MSI-X table and a CXL HDM decoder block). Return
+			 * the overlapping window with the lowest start so the
+			 * caller can walk them in order.
+			 */
+			if (!found || range->start < *x_start) {
+				*x_start = range->start;
+				*x_end = range->start + range->size;
+				found = true;
+			}
+		}
+	}
+
+	return found;
+}
+
+/* True when [start, start + len) on @bar overlaps an mmap-excluded range. */
+static bool vfio_pci_bar_mmap_excluded(struct vfio_pci_core_device *vdev,
+				       int bar, u64 start, u64 len)
+{
+	struct vfio_pci_excluded_range *range;
+
+	list_for_each_entry(range, &vdev->excluded_ranges, entry) {
+		if (range->bar != bar ||
+		    !(range->flags & VFIO_PCI_EXCLUDE_MMAP))
+			continue;
+		if (start < range->start + range->size &&
+		    start + len > range->start)
+			return true;
+	}
+
+	return false;
+}
+
+/* A page-aligned mmap hole, derived from an mmap-excluded range. */
+struct vfio_pci_mmap_hole {
+	u64 start;
+	u64 end;
+};
+
+static int vfio_pci_mmap_hole_cmp(const void *a, const void *b)
+{
+	const struct vfio_pci_mmap_hole *x = a, *y = b;
+
+	if (x->start < y->start)
+		return -1;
+	return x->start > y->start;
+}
+
+/*
+ * Advertise the BAR as mmappable minus every page-aligned mmap-excluded hole.
+ * A BAR can carry several holes at unrelated offsets (for example an MSI-X
+ * table and one or more trapped CXL component sub-blocks, which the CXL spec
+ * locates by pointer, not at fixed offsets).
+ * Collect the holes, page-align and sort them, coalesce any that overlap or
+ * touch, and advertise the gaps.
+ */
+static int vfio_pci_excluded_sparse_cap(struct vfio_pci_core_device *vdev,
+					int index, struct vfio_info_cap *caps)
+{
+	u64 bar_len = pci_resource_len(vdev->pdev, index);
+	struct vfio_region_info_cap_sparse_mmap *sparse;
+	struct vfio_pci_excluded_range *range;
+	struct vfio_pci_mmap_hole *holes;
+	int nr_holes = 0, nr_areas = 0, i, j;
+	size_t size;
+	u64 pos;
+	int ret;
+
+	list_for_each_entry(range, &vdev->excluded_ranges, entry)
+		if (range->bar == index &&
+		    (range->flags & VFIO_PCI_EXCLUDE_MMAP))
+			nr_holes++;
+
+	if (!nr_holes)
+		return 0;
+
+	holes = kmalloc_array(nr_holes, sizeof(*holes), GFP_KERNEL);
+	if (!holes)
+		return -ENOMEM;
+
+	/*
+	 * mmap is page granular, so each hole rounds out to the page boundaries
+	 * enclosing its excluded sub-range. The byte-granular exclusion still
+	 * governs the fault and read/write paths; only the advertised mmap areas
+	 * round to whole pages.
+	 */
+	i = 0;
+	list_for_each_entry(range, &vdev->excluded_ranges, entry) {
+		if (range->bar != index ||
+		    !(range->flags & VFIO_PCI_EXCLUDE_MMAP))
+			continue;
+		holes[i].start = ALIGN_DOWN(range->start, PAGE_SIZE);
+		holes[i].end = ALIGN(range->start + range->size, PAGE_SIZE);
+		i++;
+	}
+
+	sort(holes, nr_holes, sizeof(*holes), vfio_pci_mmap_hole_cmp, NULL);
+
+	/* Coalesce holes that overlap or touch after page alignment. */
+	for (i = 0, j = 0; i < nr_holes; i++) {
+		if (j && holes[i].start <= holes[j - 1].end)
+			holes[j - 1].end = max(holes[j - 1].end, holes[i].end);
+		else
+			holes[j++] = holes[i];
+	}
+	nr_holes = j;
+
+	/* One mmappable area per gap: before, between, and after the holes. */
+	for (i = 0, pos = 0; i < nr_holes; i++) {
+		if (holes[i].start > pos)
+			nr_areas++;
+		pos = holes[i].end;
+	}
+	if (pos < bar_len)
+		nr_areas++;
+
+	size = struct_size(sparse, areas, nr_areas);
+	sparse = kzalloc(size, GFP_KERNEL);
+	if (!sparse) {
+		kfree(holes);
+		return -ENOMEM;
+	}
+
+	sparse->header.id = VFIO_REGION_INFO_CAP_SPARSE_MMAP;
+	sparse->header.version = 1;
+	sparse->nr_areas = nr_areas;
+
+	for (i = 0, j = 0, pos = 0; i < nr_holes; i++) {
+		if (holes[i].start > pos) {
+			sparse->areas[j].offset = pos;
+			sparse->areas[j].size = holes[i].start - pos;
+			j++;
+		}
+		pos = holes[i].end;
+	}
+	if (pos < bar_len) {
+		sparse->areas[j].offset = pos;
+		sparse->areas[j].size = bar_len - pos;
+	}
+
+	kfree(holes);
+	ret = vfio_info_add_capability(caps, &sparse->header, size);
+	kfree(sparse);
+	return ret;
+}
+
 int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev,
 				      unsigned int type, unsigned int subtype,
 				      const struct vfio_pci_regops *ops,
@@ -1160,6 +1359,10 @@ int vfio_pci_ioctl_get_region_info(struct vfio_device *core_vdev,
 				if (ret)
 					return ret;
 			}
+			ret = vfio_pci_excluded_sparse_cap(vdev, info->index,
+							   caps);
+			if (ret)
+				return ret;
 		}
 
 		break;
@@ -1853,6 +2056,10 @@ int vfio_pci_core_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
 	if (req_start + req_len > phys_len)
 		return -EINVAL;
 
+	/* An excluded sub-range is reachable only through its trap, not mmap. */
+	if (vfio_pci_bar_mmap_excluded(vdev, index, req_start, req_len))
+		return -EINVAL;
+
 	/*
 	 * Ensure the BAR resource region is reserved for use.
 	 */
@@ -2317,6 +2524,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
 	INIT_LIST_HEAD(&vdev->dmabufs);
 	init_rwsem(&vdev->memory_lock);
 	xa_init(&vdev->ctx);
+	INIT_LIST_HEAD(&vdev->excluded_ranges);
 
 	ret = vfio_pci_core_cxl_init(vdev);
 	if (ret)
@@ -2332,6 +2540,7 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev)
 		container_of(core_vdev, struct vfio_pci_core_device, vdev);
 
 	vfio_pci_core_cxl_release(vdev);
+	vfio_pci_free_excluded_ranges(vdev);
 
 	mutex_destroy(&vdev->igate);
 	mutex_destroy(&vdev->ioeventfds_lock);
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index 4e7162234a2e..c268c99aea82 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -44,6 +44,15 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,
 ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			size_t count, loff_t *ppos, bool iswrite);
 
+/*
+ * If a read (or write) to [pos, pos + count) on @bar overlaps an excluded
+ * range, report the byte window do_io_rw() should fill with -1 (or drop) and
+ * return true. A single access spans at most one such window.
+ */
+bool vfio_pci_bar_find_exclusion(struct vfio_pci_core_device *vdev, int bar,
+				 loff_t pos, size_t count, bool iswrite,
+				 size_t *x_start, size_t *x_end);
+
 #ifdef CONFIG_VFIO_PCI_VGA
 ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 			size_t count, loff_t *ppos, bool iswrite);
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index 7f14dd46de17..48da1cb08296 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -261,6 +261,14 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 		x_end = vdev->msix_offset + vdev->msix_size;
 	}
 
+	/*
+	 * A provider-excluded sub-range is filled with -1 on read and dropped on
+	 * write for the same reason: the guest reaches it only through the trap.
+	 * An access spans at most one exclusion window.
+	 */
+	vfio_pci_bar_find_exclusion(vdev, bar, pos, count, iswrite,
+				    &x_start, &x_end);
+
 	done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM, io, buf, pos,
 				      count, x_start, x_end, iswrite, max_width);
 
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 7f3a2bcb5830..92e3db116068 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -162,6 +162,7 @@ struct vfio_pci_core_device {
 	struct notifier_block	nb;
 	struct rw_semaphore	memory_lock;
 	struct list_head	dmabufs;
+	struct list_head	excluded_ranges;
 };
 
 enum vfio_pci_io_width {
@@ -172,6 +173,19 @@ enum vfio_pci_io_width {
 };
 
 /* Will be exported for vfio pci drivers usage */
+/*
+ * A provider can keep a BAR sub-range off the direct guest path, reached only
+ * through its own trap. The flags select which paths are excluded: mmap, and
+ * region read and write (an excluded read fills -1, an excluded write is
+ * dropped).
+ */
+#define VFIO_PCI_EXCLUDE_MMAP	BIT(0)
+#define VFIO_PCI_EXCLUDE_READ	BIT(1)
+#define VFIO_PCI_EXCLUDE_WRITE	BIT(2)
+
+int vfio_pci_core_add_excluded_range(struct vfio_pci_core_device *vdev, int bar,
+				     u64 start, u64 size, u32 flags);
+
 int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev,
 				      unsigned int type, unsigned int subtype,
 				      const struct vfio_pci_regops *ops,
-- 
2.25.1

[PATCH v5 10/27] vfio/pci: Migrate MSI-X exclusion onto the generic excluded-range list

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The MSI-X table is virtualized in vfio_pci_bar_rw() by an open-coded
x_start/x_end window that fills reads with -1 and drops writes. Now that
a generic excluded-range list expresses the same fill/drop behavior,
register the MSI-X table as a read and write excluded range instead of
special-casing it in the read/write path.

Add the range when the MSI-X capability is parsed in
vfio_pci_core_enable() and clear the list in vfio_pci_core_disable()
alongside the config teardown. The read/write path now relies solely on
vfio_pci_bar_find_exclusion(), so MSI-X and a provider's (e.g. vfio-cxl)
trapped registers share one mechanism.

No behavioral change: an access to the MSI-X table still reads -1 and
drops writes.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 19 +++++++++++
 drivers/vfio/pci/vfio_pci_rdwr.c | 56 ++++++++++++++++++++++++--------
 2 files changed, 62 insertions(+), 13 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 9e4fa5d088a4..c5b7a59a4548 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -589,6 +589,8 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = {
 			   NULL)
 };
 
+static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev);
+
 int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
@@ -655,6 +657,22 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 		vdev->msix_offset = table & PCI_MSIX_TABLE_OFFSET;
 		vdev->msix_size = ((flags & PCI_MSIX_FLAGS_QSIZE) + 1) * 16;
 		vdev->has_dyn_msix = pci_msix_can_alloc_dyn(pdev);
+
+		/*
+		 * Virtualize the MSI-X table through the excluded-range list:
+		 * reads fill -1 and writes are dropped so the guest never
+		 * reaches the hardware table directly.
+		 */
+		ret = vfio_pci_core_add_excluded_range(vdev, vdev->msix_bar,
+						       vdev->msix_offset,
+						       vdev->msix_size,
+						       VFIO_PCI_EXCLUDE_READ |
+						       VFIO_PCI_EXCLUDE_WRITE);
+		if (ret) {
+			vfio_pci_free_excluded_ranges(vdev);
+			vfio_config_free(vdev);
+			goto out_free_zdev;
+		}
 	} else {
 		vdev->msix_bar = 0xFF;
 		vdev->has_dyn_msix = false;
@@ -741,6 +759,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 	vdev->region = NULL; /* don't krealloc a freed pointer */
 
 	vfio_config_free(vdev);
+	vfio_pci_free_excluded_ranges(vdev);
 
 	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
 		bar = i + PCI_STD_RESOURCES;
diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c
index 48da1cb08296..f8e5f94a2e8a 100644
--- a/drivers/vfio/pci/vfio_pci_rdwr.c
+++ b/drivers/vfio/pci/vfio_pci_rdwr.c
@@ -256,21 +256,51 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf,
 		}
 	}
 
-	if (bar == vdev->msix_bar) {
-		x_start = vdev->msix_offset;
-		x_end = vdev->msix_offset + vdev->msix_size;
-	}
-
 	/*
-	 * A provider-excluded sub-range is filled with -1 on read and dropped on
-	 * write for the same reason: the guest reaches it only through the trap.
-	 * An access spans at most one exclusion window.
+	 * The MSI-X table and any provider-excluded sub-ranges (such as a CXL
+	 * HDM decoder block) are filled with -1 on read and dropped on write:
+	 * the guest reaches them only through the virtualized path, never the
+	 * hardware directly. A BAR can hold several such windows and a single
+	 * access may span more than one, so walk the access one window at a
+	 * time. The ROM BAR uses the single trailing window set above.
 	 */
-	vfio_pci_bar_find_exclusion(vdev, bar, pos, count, iswrite,
-				    &x_start, &x_end);
-
-	done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM, io, buf, pos,
-				      count, x_start, x_end, iswrite, max_width);
+	if (bar == PCI_ROM_RESOURCE) {
+		done = vfio_pci_core_do_io_rw(vdev, res->flags & IORESOURCE_MEM,
+					      io, buf, pos, count, x_start, x_end,
+					      iswrite, max_width);
+	} else {
+		done = 0;
+		while (count) {
+			size_t chunk;
+			ssize_t ret;
+
+			x_start = 0;
+			x_end = 0;
+			if (vfio_pci_bar_find_exclusion(vdev, bar, pos, count,
+							iswrite, &x_start,
+							&x_end))
+				chunk = min(count, (size_t)(x_end - pos));
+			else
+				chunk = count;
+
+			ret = vfio_pci_core_do_io_rw(vdev,
+						     res->flags & IORESOURCE_MEM,
+						     io, buf, pos, chunk,
+						     x_start, x_end, iswrite,
+						     max_width);
+			if (ret < 0) {
+				if (!done)
+					done = ret;
+				break;
+			}
+			done += ret;
+			pos += ret;
+			buf += ret;
+			count -= ret;
+			if ((size_t)ret < chunk)
+				break;
+		}
+	}
 
 	if (done >= 0)
 		*ppos += done;
-- 
2.25.1

[PATCH v5 11/27] vfio/pci: Virtualize the CXL DVSEC in vfio_pci_config.c

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 device is reprogrammable through its CXL DVSEC: a guest could
set Config Lock, toggle CXL.cache and CXL.mem enable, or rewrite the HDM
range registers that govern host memory decode. Virtualize the DVSEC so
the guest sees a shadow it cannot use to reprogram the hardware.

Build a per-device cxl_perm permission map, modeled on msi_perm, when the
device is bound through the CXL provider (e.g. vfio-cxl). The whole CXL
DVSEC is served from the vconfig shadow; only Control and Control2 are
guest programmable, while Capability, Status, Lock and the Range registers
keep their firmware snapshot. A vendor DVSEC on the same device is
unaffected: the map is selected only for the CXL DVSEC offset.

Control2 carries the CXL reset and cache write-back-invalidate initiate
bits as self-clearing doorbells. vfio never forwards them to hardware, so
a custom writefn synthesizes their completion in the shadow: the initiate
bit self-clears and the matching Status2 bit (Cache Invalid or Reset Done)
is set, so a guest following the spec reset sequence
(INIT_CACHE_WBI, poll Cache Invalid, INIT_CXL_RST, poll Reset Done)
progresses instead of timing out. The host performs the real cache
write-back and reset at the vfio reset points.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_config.c | 132 ++++++++++++++++++++++++++++-
 include/linux/vfio_pci_core.h      |   3 +
 2 files changed, 133 insertions(+), 2 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 9914f3ac69ae..9a020a768055 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -1747,6 +1747,119 @@ static const struct pci_device_id known_bogus_vf_intx_pin[] = {
  * for each area requiring emulated bits, but the array of pointers
  * would be comparable in size (at least for standard config space).
  */
+static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,
+				int count, struct perm_bits *perm,
+				int offset, __le32 val)
+{
+	__le16 *pctrl2, *pstatus2;
+	u16 ctrl2, status2;
+	int start;
+
+	count = vfio_default_config_write(vdev, pos, count, perm, offset, val);
+	if (count < 0)
+		return count;
+
+	/* Only Control2 carries the self-clearing doorbells. */
+	if (offset > PCI_DVSEC_CXL_CTRL2 ||
+	    offset + count <= PCI_DVSEC_CXL_CTRL2)
+		return count;
+
+	start = vfio_find_cap_start(vdev, pos);
+	pctrl2 = (__le16 *)&vdev->vconfig[start + PCI_DVSEC_CXL_CTRL2];
+	pstatus2 = (__le16 *)&vdev->vconfig[start + PCI_DVSEC_CXL_STATUS2];
+	ctrl2 = le16_to_cpu(*pctrl2);
+	status2 = le16_to_cpu(*pstatus2);
+
+	/*
+	 * INIT_CACHE_WBI and INIT_CXL_RST are self-clearing doorbells that vfio
+	 * never forwards to hardware. Synthesize their completion in the shadow
+	 * so the guest poll finishes: clear the initiate bit and set the
+	 * matching Status2 completion.
+	 */
+	if (ctrl2 & PCI_DVSEC_CXL_INIT_CACHE_WBI) {
+		ctrl2 &= ~PCI_DVSEC_CXL_INIT_CACHE_WBI;
+		status2 |= PCI_DVSEC_CXL_CACHE_INV;
+	}
+	if (ctrl2 & PCI_DVSEC_CXL_INIT_CXL_RST) {
+		ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST;
+		status2 &= ~PCI_DVSEC_CXL_RST_ERR;
+		status2 |= PCI_DVSEC_CXL_RST_DONE;
+	}
+
+	*pctrl2 = cpu_to_le16(ctrl2);
+	*pstatus2 = cpu_to_le16(status2);
+
+	return count;
+}
+
+static int init_cxl_dvsec_perm(struct perm_bits *perm, int len)
+{
+	int i;
+
+	if (alloc_perm_bits(perm, len))
+		return -ENOMEM;
+
+	perm->writefn = vfio_cxl_dvsec_write;
+
+	/* Serve the whole CXL DVSEC from the shadow. */
+	for (i = 0; i < len; i++)
+		p_setb(perm, i, (u8)ALL_VIRT, NO_WRITE);
+
+	/*
+	 * Control and Control2 are guest programmable; Capability, Status,
+	 * Lock and the Range registers keep their firmware snapshot, so the
+	 * guest cannot set Config Lock or rewrite the capability and ranges.
+	 */
+	p_setw(perm, PCI_DVSEC_CXL_CTRL, (u16)ALL_VIRT, (u16)ALL_WRITE);
+	p_setw(perm, PCI_DVSEC_CXL_CTRL2, (u16)ALL_VIRT, (u16)ALL_WRITE);
+
+	return 0;
+}
+
+/* Virtualize the CXL DVSEC so a guest cannot reprogram the device through it. */
+static int vfio_cxl_dvsec_init(struct vfio_pci_core_device *vdev)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	u32 dword;
+	u16 dvsec;
+	int len, ret;
+
+	dvsec = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
+					  PCI_DVSEC_CXL_DEVICE);
+	if (!dvsec)
+		return 0;
+
+	ret = pci_read_config_dword(pdev, dvsec + PCI_DVSEC_HEADER1, &dword);
+	if (ret)
+		return pcibios_err_to_errno(ret);
+	len = PCI_DVSEC_HEADER1_LEN(dword);
+
+	/*
+	 * The virtualization writes fixed DVSEC offsets up to Status2 (the reset
+	 * doorbell stamps it). A device that reports a shorter DVSEC is not a
+	 * usable Type-2 function; leave it as plain vfio-pci rather than index the
+	 * device-length-sized perm allocation past its end.
+	 */
+	if (len < PCI_DVSEC_CXL_STATUS2 + 2)
+		return 0;
+
+	vdev->cxl_perm = kmalloc_obj(struct perm_bits, GFP_KERNEL_ACCOUNT);
+	if (!vdev->cxl_perm)
+		return -ENOMEM;
+
+	ret = init_cxl_dvsec_perm(vdev->cxl_perm, len);
+	if (ret) {
+		kfree(vdev->cxl_perm);
+		vdev->cxl_perm = NULL;
+		return ret;
+	}
+
+	vdev->cxl_dvsec = dvsec;
+	vdev->cxl_dvsec_len = len;
+
+	return 0;
+}
+
 int vfio_config_init(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
@@ -1842,6 +1955,12 @@ int vfio_config_init(struct vfio_pci_core_device *vdev)
 	if (ret)
 		goto out;
 
+	if (vdev->cxl_ops) {
+		ret = vfio_cxl_dvsec_init(vdev);
+		if (ret)
+			goto out;
+	}
+
 	return 0;
 
 out:
@@ -1863,6 +1982,12 @@ void vfio_config_free(struct vfio_pci_core_device *vdev)
 		kfree(vdev->msi_perm);
 		vdev->msi_perm = NULL;
 	}
+	if (vdev->cxl_perm) {
+		free_perm_bits(vdev->cxl_perm);
+		kfree(vdev->cxl_perm);
+		vdev->cxl_perm = NULL;
+		vdev->cxl_dvsec = 0;
+	}
 }
 
 /*
@@ -1926,12 +2051,15 @@ ssize_t vfio_pci_config_rw_single(struct vfio_pci_core_device *vdev,
 			 * of the extended capability list.  Use default, ro
 			 * access, which will virtualize the id and next values.
 			 */
+			cap_start = vfio_find_cap_start(vdev, *ppos);
+
 			if (cap_id > PCI_EXT_CAP_ID_MAX)
 				perm = &direct_ro_perms;
+			else if (cap_id == PCI_EXT_CAP_ID_DVSEC && vdev->cxl_perm &&
+				 cap_start == vdev->cxl_dvsec)
+				perm = vdev->cxl_perm;
 			else
 				perm = &ecap_perms[cap_id];
-
-			cap_start = vfio_find_cap_start(vdev, *ppos);
 		} else {
 			WARN_ON(cap_id > PCI_CAP_ID_MAX);
 
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 92e3db116068..e8a1fc398571 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -120,6 +120,9 @@ struct vfio_pci_core_device {
 	u8			*pci_config_map;
 	u8			*vconfig;
 	struct perm_bits	*msi_perm;
+	struct perm_bits	*cxl_perm;
+	u16			cxl_dvsec;
+	u16			cxl_dvsec_len;
 	spinlock_t		irqlock;
 	struct mutex		igate;
 	struct xarray		ctx;
-- 
2.25.1

[PATCH v5 12/27] vfio/pci: Call the CXL open and close hooks around device use

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Volatile CXL register state must be sampled from live hardware when the
guest opens the device, not at bind, because a low-power transition
between bind and open can leave the bind-time values stale.

Add open and close to the CXL ops and call them from the common enable
and close paths. A failed open unwinds the enable in reverse order:
unmap the BARs, drop the excluded ranges, then free the virtual config.

Factor the BAR unmap out of vfio_pci_core_disable() into a helper so the
enable failure path and the disable path share it.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 40 +++++++++++++++++++++++++-------
 include/linux/vfio_pci_core.h    |  2 ++
 2 files changed, 33 insertions(+), 9 deletions(-)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index c5b7a59a4548..2f593b2721a1 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -591,6 +591,21 @@ static const struct dev_pm_ops vfio_pci_core_pm_ops = {
 
 static void vfio_pci_free_excluded_ranges(struct vfio_pci_core_device *vdev);
 
+static void vfio_pci_core_unmap_bars(struct vfio_pci_core_device *vdev)
+{
+	struct pci_dev *pdev = vdev->pdev;
+	int i, bar;
+
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		bar = i + PCI_STD_RESOURCES;
+		if (IS_ERR_OR_NULL(vdev->barmap[bar]))
+			continue;
+		pci_iounmap(pdev, vdev->barmap[bar]);
+		pci_release_selected_regions(pdev, 1 << bar);
+		vdev->barmap[bar] = NULL;
+	}
+}
+
 int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
@@ -683,8 +698,19 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 
 	vfio_pci_core_map_bars(vdev);
 
+	if (vdev->cxl_ops) {
+		ret = vdev->cxl_ops->open_device(vdev);
+		if (ret)
+			goto out_free_config;
+	}
+
 	return 0;
 
+out_free_config:
+	/* A failed first open never reaches vfio_pci_core_disable(). */
+	vfio_pci_core_unmap_bars(vdev);
+	vfio_pci_free_excluded_ranges(vdev);
+	vfio_config_free(vdev);
 out_free_zdev:
 	vfio_pci_zdev_close_device(vdev);
 out_free_state:
@@ -705,7 +731,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 	struct pci_dev *pdev = vdev->pdev;
 	struct vfio_pci_dummy_resource *dummy_res, *tmp;
 	struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp;
-	int i, bar;
+	int i;
 
 	/* For needs_reset */
 	lockdep_assert_held(&vdev->vdev.dev_set->lock);
@@ -761,14 +787,7 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 	vfio_config_free(vdev);
 	vfio_pci_free_excluded_ranges(vdev);
 
-	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
-		bar = i + PCI_STD_RESOURCES;
-		if (IS_ERR_OR_NULL(vdev->barmap[bar]))
-			continue;
-		pci_iounmap(pdev, vdev->barmap[bar]);
-		pci_release_selected_regions(pdev, 1 << bar);
-		vdev->barmap[bar] = NULL;
-	}
+	vfio_pci_core_unmap_bars(vdev);
 
 	list_for_each_entry_safe(dummy_res, tmp,
 				 &vdev->dummy_resources_list, res_next) {
@@ -851,6 +870,9 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev)
 #endif
 	vfio_pci_dma_buf_cleanup(vdev);
 
+	if (vdev->cxl_ops)
+		vdev->cxl_ops->close_device(vdev);
+
 	vfio_pci_core_disable(vdev);
 
 	mutex_lock(&vdev->igate);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index e8a1fc398571..643f32c2141f 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -70,6 +70,8 @@ struct vfio_pci_device_ops {
 struct vfio_cxl_ops {
 	int	(*init)(struct vfio_pci_core_device *vdev);
 	void	(*release)(struct vfio_pci_core_device *vdev);
+	int	(*open_device)(struct vfio_pci_core_device *vdev);
+	void	(*close_device)(struct vfio_pci_core_device *vdev);
 	/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
 	struct module *owner;
 };
-- 
2.25.1

[PATCH v5 13/27] vfio/pci: Bracket PCI resets with the CXL reset hooks

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 device must quiesce its coherent link before any reset and
restore its HDM decoder state afterward, or a reset can leave the device
mastering the bus over decoders that are no longer valid. This has to
happen around every reset the PCI core drives for the device, not only
the ones VFIO issues directly.

Add reset_prepare and reset_done to the CXL ops and register the matching
pci_error_handlers callbacks.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_core.c | 18 ++++++++++++++++++
 include/linux/vfio_pci_core.h    |  2 ++
 2 files changed, 20 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 2f593b2721a1..a91e07181847 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -2794,8 +2794,26 @@ int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);
 
+static void vfio_pci_core_reset_prepare(struct pci_dev *pdev)
+{
+	struct vfio_pci_core_device *vdev = dev_get_drvdata(&pdev->dev);
+
+	if (vdev->cxl_ops)
+		vdev->cxl_ops->reset_prepare(vdev);
+}
+
+static void vfio_pci_core_reset_done(struct pci_dev *pdev)
+{
+	struct vfio_pci_core_device *vdev = dev_get_drvdata(&pdev->dev);
+
+	if (vdev->cxl_ops)
+		vdev->cxl_ops->reset_done(vdev);
+}
+
 const struct pci_error_handlers vfio_pci_core_err_handlers = {
 	.error_detected = vfio_pci_core_aer_err_detected,
+	.reset_prepare = vfio_pci_core_reset_prepare,
+	.reset_done = vfio_pci_core_reset_done,
 };
 EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers);
 
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 643f32c2141f..6ebf9a26275e 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -72,6 +72,8 @@ struct vfio_cxl_ops {
 	void	(*release)(struct vfio_pci_core_device *vdev);
 	int	(*open_device)(struct vfio_pci_core_device *vdev);
 	void	(*close_device)(struct vfio_pci_core_device *vdev);
+	void	(*reset_prepare)(struct vfio_pci_core_device *vdev);
+	void	(*reset_done)(struct vfio_pci_core_device *vdev);
 	/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
 	struct module *owner;
 };
-- 
2.25.1

[PATCH v5 14/27] vfio/pci: Provide an opt-out for the CXL Type-2 extensions

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Let an admin or a variant driver keep the CXL Type-2 extensions out of
the path and drive the device as plain vfio-pci. Add a
vfio_pci.disable_cxl module parameter, and let a variant driver set
vdev->disable_cxl in its own init callback for per-device control. The
parameter is built only when the CXL provider is configured.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/vfio_pci.c   | 13 +++++++++++++
 include/linux/vfio_pci_core.h |  1 +
 2 files changed, 14 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 830369ff878d..66f2e318d8e5 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -60,6 +60,16 @@ static bool disable_denylist;
 module_param(disable_denylist, bool, 0444);
 MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users.");
 
+#if IS_ENABLED(CONFIG_VFIO_CXL)
+/*
+ * A variant driver may instead set vdev->disable_cxl in its own init callback
+ * for per-device control without this module-wide parameter.
+ */
+static bool disable_cxl;
+module_param(disable_cxl, bool, 0644);
+MODULE_PARM_DESC(disable_cxl, "Disable CXL Type-2 extensions for all devices bound to vfio-pci.");
+#endif
+
 static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev)
 {
 	switch (pdev->vendor) {
@@ -142,6 +152,9 @@ static int vfio_pci_init_dev(struct vfio_device *core_vdev)
 #ifdef CONFIG_VFIO_PCI_VGA
 	vdev->disable_vga = disable_vga;
 #endif
+#if IS_ENABLED(CONFIG_VFIO_CXL)
+	vdev->disable_cxl = disable_cxl;
+#endif
 
 	return vfio_pci_core_init_dev(core_vdev);
 }
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 6ebf9a26275e..475a0ecf9e4f 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -149,6 +149,7 @@ struct vfio_pci_core_device {
 	bool			disable_idle_d3:1;
 	bool			nointxmask:1;
 	bool			disable_vga:1;
+	bool			disable_cxl:1;
 	/* Flags modified at runtime - dedicated storage unit */
 	bool			needs_reset;
 	bool			pm_intx_masked;
-- 
2.25.1

[PATCH v5 15/27] vfio/cxl: Add the vfio-cxl provider module skeleton

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Add the vfio-cxl module that vfio-pci-core loads on demand for a CXL
Type-2 device, registering its callbacks at module_init.

vfio-cxl is a feature of vfio-pci-core rather than a variant driver, so
its Kconfig sources above the variant drivers and its objects link right
after vfio-pci-core while still being loaded on demand through the
vfio-cxl MODULE_ALIAS.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 MAINTAINERS                          |  7 ++++
 drivers/vfio/pci/Kconfig             |  2 +
 drivers/vfio/pci/Makefile            |  2 +
 drivers/vfio/pci/cxl/Kconfig         | 11 +++++
 drivers/vfio/pci/cxl/Makefile        |  3 ++
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 61 ++++++++++++++++++++++++++++
 6 files changed, 86 insertions(+)
 create mode 100644 drivers/vfio/pci/cxl/Kconfig
 create mode 100644 drivers/vfio/pci/cxl/Makefile
 create mode 100644 drivers/vfio/pci/cxl/vfio_cxl_core.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 2037501cd621..75d472d7ca07 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28639,6 +28639,13 @@ L:	kvm@vger.kernel.org
 S:	Maintained
 F:	drivers/vfio/cdx/*
 
+VFIO CXL DRIVER
+M:	Manish Honap <mhonap@nvidia.com>
+L:	kvm@vger.kernel.org
+L:	linux-cxl@vger.kernel.org
+S:	Supported
+F:	drivers/vfio/pci/cxl/
+
 VFIO DRIVER
 M:	Alex Williamson <alex@shazbot.org>
 L:	kvm@vger.kernel.org
diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig
index 296bf01e185e..4cd6acd36053 100644
--- a/drivers/vfio/pci/Kconfig
+++ b/drivers/vfio/pci/Kconfig
@@ -58,6 +58,8 @@ config VFIO_PCI_ZDEV_KVM
 config VFIO_PCI_DMABUF
 	def_bool y if VFIO_PCI_CORE && PCI_P2PDMA && DMA_SHARED_BUFFER
 
+source "drivers/vfio/pci/cxl/Kconfig"
+
 source "drivers/vfio/pci/mlx5/Kconfig"
 
 source "drivers/vfio/pci/ism/Kconfig"
diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile
index 6138f1bf241d..b65117d8d463 100644
--- a/drivers/vfio/pci/Makefile
+++ b/drivers/vfio/pci/Makefile
@@ -5,6 +5,8 @@ vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o
 vfio-pci-core-$(CONFIG_VFIO_PCI_DMABUF) += vfio_pci_dmabuf.o
 obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o
 
+obj-$(CONFIG_VFIO_CXL) += cxl/
+
 vfio-pci-y := vfio_pci.o
 vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o
 obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
diff --git a/drivers/vfio/pci/cxl/Kconfig b/drivers/vfio/pci/cxl/Kconfig
new file mode 100644
index 000000000000..1392fa4ed15d
--- /dev/null
+++ b/drivers/vfio/pci/cxl/Kconfig
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VFIO_CXL
+	tristate "VFIO support for CXL Type-2 devices"
+	depends on CXL_BUS && CXL_MEM
+	select VFIO_PCI_CORE
+	help
+	  Add CXL HDM handling to vfio-pci so a CXL Type-2 device, such as a
+	  CXL-attached accelerator, can be assigned to a virtual machine.
+	  vfio-pci loads this module on demand when it binds a CXL device.
+
+	  If you don't know what to do here, say N.
diff --git a/drivers/vfio/pci/cxl/Makefile b/drivers/vfio/pci/cxl/Makefile
new file mode 100644
index 000000000000..8501bf626108
--- /dev/null
+++ b/drivers/vfio/pci/cxl/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_VFIO_CXL) += vfio-cxl.o
+vfio-cxl-y := vfio_cxl_core.o
diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
new file mode 100644
index 000000000000..cd5d41856404
--- /dev/null
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * VFIO support for CXL Type-2 devices.
+ *
+ * Copyright (c) 2026 NVIDIA Corporation & Affiliates
+ */
+
+#include <linux/module.h>
+#include <linux/vfio_pci_core.h>
+
+static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
+{
+	return 0;
+}
+
+static void vfio_cxl_release_device(struct vfio_pci_core_device *vdev)
+{
+}
+
+static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
+{
+	return 0;
+}
+
+static void vfio_cxl_close_device(struct vfio_pci_core_device *vdev)
+{
+}
+
+static void vfio_cxl_reset_prepare(struct vfio_pci_core_device *vdev)
+{
+}
+
+static void vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)
+{
+}
+
+static const struct vfio_cxl_ops vfio_cxl_ops = {
+	.init		= vfio_cxl_init_device,
+	.release	= vfio_cxl_release_device,
+	.open_device	= vfio_cxl_open_device,
+	.close_device	= vfio_cxl_close_device,
+	.reset_prepare	= vfio_cxl_reset_prepare,
+	.reset_done	= vfio_cxl_reset_done,
+	.owner		= THIS_MODULE,
+};
+
+static int __init vfio_cxl_init(void)
+{
+	return vfio_pci_core_register_cxl_ops(&vfio_cxl_ops);
+}
+
+static void __exit vfio_cxl_exit(void)
+{
+	vfio_pci_core_unregister_cxl_ops(&vfio_cxl_ops);
+}
+
+module_init(vfio_cxl_init);
+module_exit(vfio_cxl_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("VFIO support for CXL Type-2 devices");
-- 
2.25.1

[PATCH v5 16/27] vfio/cxl: Create the CXL memdev and set media ready at bind

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

At bind, build the CXL memory device for the passed-through Type-2
accelerator so it joins the CXL topology and its HDM region resolves to a
host physical range. A Type-2 device has no mailbox, so there is no
media-ready register to poll: set media ready directly once the component
registers validate (mirroring drivers/net/ethernet/sfc/efx_cxl.c)

As per current vfio-cxl support, reject a device with:
- more than one HDM decoder
- interleaving enabled
- whose reset the host cannot service

The CXL-core allocations are grouped with devres so a failed bind unwinds
them: init failure falls back to plain vfio-pci with the device still
bound, so devm would otherwise hold them until unbind.

A low-power transition would reset the CXL Type-2 function and lose
its CXL.mem contents, so keep it in D0 while it is assigned.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 118 +++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c     |  16 ++++
 2 files changed, 134 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index cd5d41856404..0d92e6a409c1 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -6,15 +6,132 @@
  */
 
 #include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/range.h>
 #include <linux/vfio_pci_core.h>
+#include <cxl/cxl.h>
+#include <cxl/pci.h>
+
+/**
+ * struct vfio_cxl_state - per-device state for a vfio-cxl device
+ * @cxlds: CXL device state; kept first for devm_cxl_dev_state_create()
+ * @cxlmd: memory device joined to the CXL topology at bind
+ * @hpa_range: host physical range of the HDM region
+ */
+struct vfio_cxl_state {
+	struct cxl_dev_state cxlds;
+	struct cxl_memdev *cxlmd;
+	struct range hpa_range;
+};
 
 static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 {
+	struct pci_dev *pdev = vdev->pdev;
+	struct vfio_cxl_state *cxl;
+	struct cxl_memdev *cxlmd;
+	u64 hdm_size, serial;
+	u16 dvsec;
+	int ret;
+
+	/*
+	 * pdev->hdm is cached at PCI enumeration, before any driver binds, so a
+	 * device without it has no usable HDM decoder. Fall back to plain
+	 * vfio-pci rather than deferring the bind forever.
+	 */
+	if (!pdev->hdm)
+		return -ENODEV;
+
+	/* The guest drives one virtual decoder; multiple are unsupported. */
+	if (pdev->hdm->decoder_count != 1)
+		return -EOPNOTSUPP;
+
+	/* An interleaved decoder cannot be mapped 1:1 to the guest. */
+	if (pdev->hdm->settings[0].interleave_ways != 1)
+		return -EOPNOTSUPP;
+
+	/*
+	 * The guest drives resets through the CXL Device DVSEC and polls the
+	 * shadow for completion. If the host cannot service a function-scoped
+	 * CXL reset, that request could never complete, so refuse the device
+	 * rather than advertise a reset the guest would poll on forever.
+	 */
+	if (!cxl_reset_capable(pdev))
+		return -EOPNOTSUPP;
+
+	hdm_size = range_len(&pdev->hdm->settings[0].hpa_range);
+	if (!hdm_size)
+		return -ENXIO;
+
+	dvsec = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
+					  PCI_DVSEC_CXL_DEVICE);
+	serial = pci_get_dsn(pdev);
+
+	/*
+	 * Group the CXL-core allocations so a later failure unwinds them here.
+	 * A failed init falls back to plain vfio-pci with the device still
+	 * bound, so devm would otherwise hold them until unbind.
+	 */
+	if (!devres_open_group(&pdev->dev, NULL, GFP_KERNEL))
+		return -ENOMEM;
+
+	cxl = devm_cxl_dev_state_create(&pdev->dev, CXL_DEVTYPE_DEVMEM, serial,
+					dvsec, struct vfio_cxl_state, cxlds,
+					false);
+	if (!cxl) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
+	ret = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT,
+				 &cxl->cxlds.reg_map);
+	if (ret) {
+		pci_err(pdev, "vfio-cxl: no component registers\n");
+		goto err;
+	}
+
+	if (!cxl->cxlds.reg_map.component_map.hdm_decoder.valid) {
+		pci_err(pdev, "vfio-cxl: HDM decoder registers not found\n");
+		ret = -ENODEV;
+		goto err;
+	}
+
+	/*
+	 * A Type-2 accelerator has no mailbox and no media-ready register, so
+	 * set media ready directly.
+	 */
+	cxl->cxlds.media_ready = true;
+
+	ret = cxl_set_capacity(&cxl->cxlds, hdm_size);
+	if (ret)
+		goto err;
+
+	cxlmd = devm_cxl_probe_mem(&cxl->cxlds, &cxl->hpa_range);
+	if (IS_ERR(cxlmd)) {
+		ret = PTR_ERR(cxlmd);
+		goto err;
+	}
+
+	cxl->cxlmd = cxlmd;
+	devres_close_group(&pdev->dev, NULL);
+
+	/*
+	 * Powering a CXL Type-2 function down and back up reinitializes its
+	 * device state and discards the contents of its coherent memory. Pin
+	 * it in D0 for as long as it is assigned so CXL.mem stays intact.
+	 */
+	vdev->disable_idle_d3 = true;
+	vdev->cxl = cxl;
+
 	return 0;
+
+err:
+	devres_release_group(&pdev->dev, NULL);
+	return ret;
 }
 
 static void vfio_cxl_release_device(struct vfio_pci_core_device *vdev)
 {
+	vdev->cxl = NULL;
 }
 
 static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
@@ -59,3 +176,4 @@ module_exit(vfio_cxl_exit);
 
 MODULE_LICENSE("GPL");
 MODULE_DESCRIPTION("VFIO support for CXL Type-2 devices");
+MODULE_IMPORT_NS("CXL");
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index a91e07181847..ddd6807893fd 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -325,6 +325,14 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t stat
 	bool needs_restore = false, needs_save = false;
 	int ret;
 
+	/*
+	 * A low-power transition would reset the CXL Type-2 function and lose
+	 * its CXL.mem contents, so keep it in D0 regardless of the guest or
+	 * VMM request.
+	 */
+	if (vdev->cxl_ops && state > PCI_D0)
+		state = PCI_D0;
+
 	/* Prevent changing power state for PFs with VFs enabled */
 	if (state > PCI_D0) {
 		lockdep_assert_held_write(&vdev->memory_lock);
@@ -373,6 +381,14 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t stat
 static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev,
 				     struct eventfd_ctx *efdctx)
 {
+	/*
+	 * Low power entry lets the PCI core autosuspend the device to D3hot,
+	 * which would soft-reset a CXL Type-2 device and lose its coherent HDM
+	 * memory. Refuse it for CXL; the device stays in D0.
+	 */
+	if (vdev->cxl_ops)
+		return -EINVAL;
+
 	/*
 	 * The vdev power related flags are protected with 'memory_lock'
 	 * semaphore.
-- 
2.25.1

[PATCH v5 17/27] vfio/cxl: Own the whole component register BAR

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

devm_cxl_probe_mem() maps the component registers through the CXL core,
which by default claims each HDM and RAS sub-block. That collides with
the full-BAR request vfio-pci-core makes when the guest opens the device.

After the component registers are discovered, declare the containing BAR
owned with cxl_reg_map_add_owned_resource(). The core then ioremaps the
sub-blocks without requesting them, so vfio-pci-core owns the whole BAR
and the requests do not collide.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 0d92e6a409c1..5c8a63833a43 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -89,6 +89,15 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 		goto err;
 	}
 
+	/*
+	 * vfio-pci-core requests the whole component-register BAR when the
+	 * guest opens the device. Declare that BAR owned so the CXL core
+	 * ioremaps the HDM and RAS sub-blocks without claiming them, and the
+	 * full-BAR request does not collide.
+	 */
+	cxl_reg_map_add_owned_resource(&cxl->cxlds.reg_map,
+				       pci_resource_n(pdev, pdev->hdm->hdm_bar));
+
 	if (!cxl->cxlds.reg_map.component_map.hdm_decoder.valid) {
 		pci_err(pdev, "vfio-cxl: HDM decoder registers not found\n");
 		ret = -ENODEV;
-- 
2.25.1

[PATCH v5 18/27] vfio/cxl: Expose the HDM memory region to the guest

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 guest maps the device HDM memory to use its coherent
CXL.mem. The HDM memory is a host physical range with no struct page,
so the guest and KVM need a write-back mapping of it.

Register the range as an mmap-able region under
VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id rather than a
bespoke region type, per open because vfio_pci_core_disable() tears
down all dynamic regions on close.

Own the resolved host physical range exclusively (IORESOURCE_EXCLUSIVE)
so nothing, /dev/mem included, can map a conflicting cacheable alias
that would fault the host once the range is mapped write-back.  There
is no devm form of the exclusive request, so pair it with a devm
release action.

vfio_pci_zap_bars() only unmaps the fixed PCI BAR range, so revoke
mmap-capable device-specific regions there too.  Otherwise a
Memory-Space disable, a D3 transition, or a reset would leave the guest
a live mapping into quiesced device memory; the fault handler re-gates
on device state before it inserts a pfn again.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 186 +++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c     |  37 ++++++
 include/linux/vfio_pci_core.h        |   1 +
 include/uapi/linux/vfio.h            |   4 +
 4 files changed, 228 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 5c8a63833a43..5b65cac30aba 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -5,9 +5,13 @@
  * Copyright (c) 2026 NVIDIA Corporation & Affiliates
  */
 
+#include <linux/cleanup.h>
+#include <linux/io.h>
+#include <linux/mm.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 #include <linux/range.h>
+#include <linux/uaccess.h>
 #include <linux/vfio_pci_core.h>
 #include <cxl/cxl.h>
 #include <cxl/pci.h>
@@ -17,13 +21,144 @@
  * @cxlds: CXL device state; kept first for devm_cxl_dev_state_create()
  * @cxlmd: memory device joined to the CXL topology at bind
  * @hpa_range: host physical range of the HDM region
+ * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock
  */
 struct vfio_cxl_state {
 	struct cxl_dev_state cxlds;
 	struct cxl_memdev *cxlmd;
 	struct range hpa_range;
+	bool hdm_valid;
 };
 
+static unsigned long vfio_cxl_mem_pgoff(struct vm_area_struct *vma,
+					unsigned long addr)
+{
+	unsigned long mask = (1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1;
+
+	return (vma->vm_pgoff & mask) + ((addr - vma->vm_start) >> PAGE_SHIFT);
+}
+
+static vm_fault_t vfio_cxl_mem_huge_fault(struct vm_fault *vmf,
+					  unsigned int order)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct vfio_pci_core_device *vdev = vma->vm_private_data;
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	unsigned long addr = ALIGN_DOWN(vmf->address, PAGE_SIZE << order);
+	unsigned long pfn = PHYS_PFN(cxl->hpa_range.start) +
+			    vfio_cxl_mem_pgoff(vma, addr);
+	vm_fault_t ret = VM_FAULT_FALLBACK;
+
+	if (is_aligned_for_order(vma, addr, pfn, order)) {
+		scoped_guard(rwsem_read, &vdev->memory_lock) {
+			/*
+			 * Insert a PFN only for a known-good decoder whose
+			 * media is ready and whose Memory-Space is enabled.
+			 */
+			if (__vfio_pci_memory_enabled(vdev) &&
+			    cxl->hdm_valid && cxl->cxlds.media_ready)
+				ret = vfio_pci_vmf_insert_pfn(vdev, vmf, pfn,
+							      order);
+			else
+				ret = VM_FAULT_SIGBUS;
+		}
+	}
+
+	return ret;
+}
+
+static vm_fault_t vfio_cxl_mem_fault(struct vm_fault *vmf)
+{
+	return vfio_cxl_mem_huge_fault(vmf, 0);
+}
+
+static const struct vm_operations_struct vfio_cxl_mem_vm_ops = {
+	.fault = vfio_cxl_mem_fault,
+#ifdef CONFIG_ARCH_SUPPORTS_HUGE_PFNMAP
+	.huge_fault = vfio_cxl_mem_huge_fault,
+#endif
+};
+
+static int vfio_cxl_mem_mmap(struct vfio_pci_core_device *vdev,
+			     struct vfio_pci_region *region,
+			     struct vm_area_struct *vma)
+{
+	unsigned long mask = (1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1;
+	u64 req_start = (vma->vm_pgoff & mask) << PAGE_SHIFT;
+	u64 req_len = vma->vm_end - vma->vm_start;
+
+	if (req_start + req_len > region->size)
+		return -EINVAL;
+
+	/*
+	 * CXL.mem is coherent memory, so leave the mapping write-back cacheable.
+	 */
+	vm_flags_set(vma, VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
+	vma->vm_ops = &vfio_cxl_mem_vm_ops;
+	vma->vm_private_data = vdev;
+
+	return 0;
+}
+
+static ssize_t vfio_cxl_mem_rw(struct vfio_pci_core_device *vdev,
+			       char __user *buf, size_t count, loff_t *ppos,
+			       bool iswrite)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	u64 pos = *ppos & VFIO_PCI_OFFSET_MASK;
+	void *mem;
+	ssize_t done;
+
+	if (pos >= range_len(&cxl->hpa_range))
+		return -EINVAL;
+	count = min_t(size_t, count, range_len(&cxl->hpa_range) - pos);
+
+	scoped_guard(rwsem_read, &vdev->memory_lock) {
+		/*
+		 * Same gate as the fault path: only touch the HDM range with
+		 * the decoder in a known-good state AND Memory-Space enabled,
+		 * or a host CPU access aborts as a fatal SError.
+		 */
+		if (!cxl->hdm_valid || !__vfio_pci_memory_enabled(vdev))
+			return -EIO;
+
+		mem = memremap(cxl->hpa_range.start + pos, count, MEMREMAP_WB);
+		if (!mem)
+			return -ENOMEM;
+		if (iswrite)
+			done = copy_from_user(mem, buf, count) ? -EFAULT : count;
+		else
+			done = copy_to_user(buf, mem, count) ? -EFAULT : count;
+		memunmap(mem);
+	}
+	if (done > 0)
+		*ppos += done;
+
+	return done;
+}
+
+/*
+ * The CXL regions carry no per-region state (region->data is the shared,
+ * devm-managed vfio_cxl_state), so releasing a region is a no-op.
+ */
+static void vfio_cxl_region_release(struct vfio_pci_core_device *vdev,
+				    struct vfio_pci_region *region)
+{
+}
+
+static const struct vfio_pci_regops vfio_cxl_mem_regops = {
+	.rw = vfio_cxl_mem_rw,
+	.mmap = vfio_cxl_mem_mmap,
+	.release = vfio_cxl_region_release,
+};
+
+static void vfio_cxl_release_hpa(void *data)
+{
+	struct vfio_cxl_state *cxl = data;
+
+	release_mem_region(cxl->hpa_range.start, range_len(&cxl->hpa_range));
+}
+
 static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
@@ -120,6 +255,21 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 		goto err;
 	}
 
+	/*
+	 * Claim the range IORESOURCE_EXCLUSIVE so no conflicting cacheable
+	 * alias can fault the host once it is mapped write-back; there is no
+	 * devm form, so pair it with a devm release action.
+	 */
+	if (!request_mem_region_exclusive(cxl->hpa_range.start,
+					  range_len(&cxl->hpa_range),
+					  "vfio-cxl-hdm")) {
+		ret = -EBUSY;
+		goto err;
+	}
+	ret = devm_add_action_or_reset(&pdev->dev, vfio_cxl_release_hpa, cxl);
+	if (ret)
+		goto err;
+
 	cxl->cxlmd = cxlmd;
 	devres_close_group(&pdev->dev, NULL);
 
@@ -143,13 +293,49 @@ static void vfio_cxl_release_device(struct vfio_pci_core_device *vdev)
 	vdev->cxl = NULL;
 }
 
+static int vfio_cxl_add_region(struct vfio_pci_core_device *vdev, u32 subtype,
+			       const struct vfio_pci_regops *ops, size_t size,
+			       u32 flags)
+{
+	u32 type = VFIO_REGION_TYPE_PCI_VENDOR_TYPE | PCI_VENDOR_ID_CXL;
+
+	return vfio_pci_core_register_dev_region(vdev, type, subtype, ops,
+						 size, flags, vdev->cxl);
+}
+
 static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 {
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	int ret;
+
+	/*
+	 * vfio_pci_core_disable() frees all dynamic regions on close, so register
+	 * them here per open rather than at bind. A failed first open never
+	 * reaches close_device(), so unwind on error.
+	 */
+	ret = vfio_cxl_add_region(vdev, VFIO_REGION_SUBTYPE_CXL_MEM,
+				  &vfio_cxl_mem_regops, range_len(&cxl->hpa_range),
+				  VFIO_REGION_INFO_FLAG_READ |
+				  VFIO_REGION_INFO_FLAG_WRITE |
+				  VFIO_REGION_INFO_FLAG_MMAP);
+	if (ret)
+		return ret;
+
+	/*
+	 * The decoder is firmware-committed, so host access to the HDM range is
+	 * safe. Open the access gate; reset and power transitions clear it until
+	 * the decoder is restored.
+	 */
+	cxl->hdm_valid = true;
+
 	return 0;
 }
 
 static void vfio_cxl_close_device(struct vfio_pci_core_device *vdev)
 {
+	struct vfio_cxl_state *cxl = vdev->cxl;
+
+	cxl->hdm_valid = false;
 }
 
 static void vfio_cxl_reset_prepare(struct vfio_pci_core_device *vdev)
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index ddd6807893fd..8913a9e24302 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1295,6 +1295,23 @@ int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev,
 }
 EXPORT_SYMBOL_GPL(vfio_pci_core_register_dev_region);
 
+/*
+ * Unregister the most recently registered dynamic region. Used to unwind a
+ * partially built region set on an open-time error; regions are otherwise
+ * released together in vfio_pci_core_disable().
+ */
+void vfio_pci_core_unregister_dev_region(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_pci_region *region;
+
+	if (WARN_ON(!vdev->num_regions))
+		return;
+
+	region = &vdev->region[--vdev->num_regions];
+	region->ops->release(vdev, region);
+}
+EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_dev_region);
+
 static int vfio_pci_info_atomic_cap(struct vfio_pci_core_device *vdev,
 				    struct vfio_info_cap *caps)
 {
@@ -1972,8 +1989,28 @@ static void vfio_pci_zap_bars(struct vfio_pci_core_device *vdev)
 	loff_t start = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_BAR0_REGION_INDEX);
 	loff_t end = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_ROM_REGION_INDEX);
 	loff_t len = end - start;
+	unsigned int i;
 
 	unmap_mapping_range(core_vdev->inode->i_mapping, start, len, true);
+
+	/*
+	 * The unmap above covers the PCI BARs; mmap-capable device-specific
+	 * regions (e.g. a vfio-cxl HDM window) sit above that range, so revoke
+	 * them here too, or a Memory-Space disable, D3/PM transition, or reset
+	 * would leave the guest a live mapping into quiesced device memory.
+	 * Callers hold memory_lock, so the region array is stable.
+	 */
+	for (i = 0; i < vdev->num_regions; i++) {
+		struct vfio_pci_region *region = &vdev->region[i];
+		loff_t roff;
+
+		if (!(region->flags & VFIO_REGION_INFO_FLAG_MMAP))
+			continue;
+
+		roff = VFIO_PCI_INDEX_TO_OFFSET(VFIO_PCI_NUM_REGIONS + i);
+		unmap_mapping_range(core_vdev->inode->i_mapping, roff,
+				    region->size, true);
+	}
 }
 
 void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev)
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 475a0ecf9e4f..39a28cc6ae8c 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -198,6 +198,7 @@ int vfio_pci_core_register_dev_region(struct vfio_pci_core_device *vdev,
 				      unsigned int type, unsigned int subtype,
 				      const struct vfio_pci_regops *ops,
 				      size_t size, u32 flags, void *data);
+void vfio_pci_core_unregister_dev_region(struct vfio_pci_core_device *vdev);
 void vfio_pci_core_close_device(struct vfio_device *core_vdev);
 int vfio_pci_core_init_dev(struct vfio_device *core_vdev);
 void vfio_pci_core_release_dev(struct vfio_device *core_vdev);
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index e41437fa17ad..1bf86763c0f7 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -370,6 +370,10 @@ struct vfio_region_info_cap_type {
  */
 #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD	(1)
 
+/* CXL Type-2 device (0x1e98) sub-types for VFIO_REGION_TYPE_PCI_VENDOR_TYPE */
+/* CXL.mem HDM region of a Type-2 device, mmap-able */
+#define VFIO_REGION_SUBTYPE_CXL_MEM		(1)
+
 /* sub-types for VFIO_REGION_TYPE_GFX */
 #define VFIO_REGION_SUBTYPE_GFX_EDID            (1)
 
-- 
2.25.1

[PATCH v5 19/27] vfio/cxl: Contain HDM memory errors with memory_failure()

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The HDM memory region is mmap-able, so an fd holder can fault its
struct-page-less device memory in from the host CPU. A memory error on
that range cannot be routed through the normal struct-page path and
would otherwise escalate to a fatal host SError.

Register the range with memory_failure() via
register_pfn_address_space() so such an error is instead contained to
unmapping the range and delivering a SIGBUS to the fd holder. Provide a
pfn-to-vma-offset callback so memory_failure() can map a poisoned pfn
back to the file offset of each user mapping and unmap it; the region is
a single linear range at hpa_range.start, so recover the offset the same
way the fault handler derived the pfn.

register_pfn_address_space() returns -EOPNOTSUPP when
CONFIG_MEMORY_FAILURE is off, in which case expose the region without
containment.  Unregister the range on close.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 76 ++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 5b65cac30aba..e099e9a70a5a 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -7,10 +7,12 @@
 
 #include <linux/cleanup.h>
 #include <linux/io.h>
+#include <linux/memory-failure.h>
 #include <linux/mm.h>
 #include <linux/module.h>
 #include <linux/pci.h>
 #include <linux/range.h>
+#include <linux/slab.h>
 #include <linux/uaccess.h>
 #include <linux/vfio_pci_core.h>
 #include <cxl/cxl.h>
@@ -21,12 +23,14 @@
  * @cxlds: CXL device state; kept first for devm_cxl_dev_state_create()
  * @cxlmd: memory device joined to the CXL topology at bind
  * @hpa_range: host physical range of the HDM region
+ * @hdm_pfn_space: HDM-region pfn range registered with memory_failure()
  * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock
  */
 struct vfio_cxl_state {
 	struct cxl_dev_state cxlds;
 	struct cxl_memdev *cxlmd;
 	struct range hpa_range;
+	struct pfn_address_space hdm_pfn_space;
 	bool hdm_valid;
 };
 
@@ -152,6 +156,62 @@ static const struct vfio_pci_regops vfio_cxl_mem_regops = {
 	.release = vfio_cxl_region_release,
 };
 
+/*
+ * Map a poisoned HDM-region pfn back to the file offset of each user mapping so
+ * memory_failure() can unmap it and signal the fd holder. The region is a
+ * single linear range at hpa_range.start; recover the per-vma file offset the
+ * same way the fault handler derived the pfn.
+ */
+static int vfio_cxl_pfn_to_vma_pgoff(struct vm_area_struct *vma,
+				     unsigned long pfn, pgoff_t *pgoff)
+{
+	struct vfio_pci_core_device *vdev;
+	struct vfio_cxl_state *cxl;
+	pgoff_t vma_off, pfn_off;
+	unsigned long start_pfn;
+
+	if (vma->vm_ops != &vfio_cxl_mem_vm_ops)
+		return -ENOENT;
+
+	vdev = vma->vm_private_data;
+	cxl = vdev->cxl;
+
+	start_pfn = PHYS_PFN(cxl->hpa_range.start);
+	if (pfn < start_pfn ||
+	    pfn >= start_pfn + (range_len(&cxl->hpa_range) >> PAGE_SHIFT))
+		return -EFAULT;
+
+	pfn_off = pfn - start_pfn;
+	vma_off = vma->vm_pgoff &
+		  ((1UL << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
+	/* Skip VMAs that do not map the pfn, e.g. a partial mmap of the region. */
+	if (pfn_off < vma_off || pfn_off - vma_off >= vma_pages(vma))
+		return -EFAULT;
+
+	*pgoff = vma->vm_pgoff + (pfn_off - vma_off);
+	return 0;
+}
+
+/*
+ * The HDM region is struct-page-less device memory, so a memory error on it
+ * cannot be routed through the normal page path. Register the range with
+ * memory_failure() so such an error is contained to unmapping the range and a
+ * SIGBUS to the fd holder instead of escalating to a host SError.
+ */
+static int vfio_cxl_register_pfn_space(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	unsigned long start_pfn = PHYS_PFN(cxl->hpa_range.start);
+
+	cxl->hdm_pfn_space.node.start = start_pfn;
+	cxl->hdm_pfn_space.node.last =
+		start_pfn + (range_len(&cxl->hpa_range) >> PAGE_SHIFT) - 1;
+	cxl->hdm_pfn_space.mapping = vdev->vdev.inode->i_mapping;
+	cxl->hdm_pfn_space.pfn_to_vma_pgoff = vfio_cxl_pfn_to_vma_pgoff;
+
+	return register_pfn_address_space(&cxl->hdm_pfn_space);
+}
+
 static void vfio_cxl_release_hpa(void *data)
 {
 	struct vfio_cxl_state *cxl = data;
@@ -321,6 +381,16 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 	if (ret)
 		return ret;
 
+	/*
+	 * The HDM region is advertised mmap-able, so a fd holder can fault its
+	 * struct-page-less device memory in from the host CPU. Register it with
+	 * memory_failure() to contain a memory error. -EOPNOTSUPP means
+	 * CONFIG_MEMORY_FAILURE is off, so run without containment.
+	 */
+	ret = vfio_cxl_register_pfn_space(vdev);
+	if (ret && ret != -EOPNOTSUPP)
+		goto err_unregister_mem;
+
 	/*
 	 * The decoder is firmware-committed, so host access to the HDM range is
 	 * safe. Open the access gate; reset and power transitions clear it until
@@ -329,6 +399,11 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 	cxl->hdm_valid = true;
 
 	return 0;
+
+err_unregister_mem:
+	vfio_pci_core_unregister_dev_region(vdev);
+
+	return ret;
 }
 
 static void vfio_cxl_close_device(struct vfio_pci_core_device *vdev)
@@ -336,6 +411,7 @@ static void vfio_cxl_close_device(struct vfio_pci_core_device *vdev)
 	struct vfio_cxl_state *cxl = vdev->cxl;
 
 	cxl->hdm_valid = false;
+	unregister_pfn_address_space(&cxl->hdm_pfn_space);
 }
 
 static void vfio_cxl_reset_prepare(struct vfio_pci_core_device *vdev)
-- 
2.25.1

[PATCH v5 20/27] vfio/cxl: Expose the HDM decoder registers read-only to the guest

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 guest reads the HDM decoder registers to learn the HDM
region it was handed. Those registers live in the component BAR that
vfio-pci owns.

Map the decoder block at bind: its location comes from the pdev->hdm
enumeration cache, and vfio-pci owns the BAR, so map it without
claiming the block and expose it as a second, read-only region under
VFIO_REGION_TYPE_PCI_VENDOR_TYPE with the CXL vendor id, registered per
open like the HDM memory region.

Serve reads live from the mapped block and absorb writes without
forwarding them to hardware.

Registering a second region is the first point at which an open-time
failure must unwind an already-registered region, so add
vfio_pci_core_unregister_dev_region() to drop the most recently
registered region, and use it to unwind the HDM memory region if the
decoder region fails to register.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 81 +++++++++++++++++++++++++++-
 include/uapi/linux/vfio.h            |  2 +
 2 files changed, 82 insertions(+), 1 deletion(-)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index e099e9a70a5a..da04776356e4 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -24,6 +24,8 @@
  * @cxlmd: memory device joined to the CXL topology at bind
  * @hpa_range: host physical range of the HDM region
  * @hdm_pfn_space: HDM-region pfn range registered with memory_failure()
+ * @hdm_regs: mapped HDM decoder registers, read live by the decoder region
+ * @hdm_len: length of the HDM decoder register block
  * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock
  */
 struct vfio_cxl_state {
@@ -31,6 +33,8 @@ struct vfio_cxl_state {
 	struct cxl_memdev *cxlmd;
 	struct range hpa_range;
 	struct pfn_address_space hdm_pfn_space;
+	void __iomem *hdm_regs;
+	u32 hdm_len;
 	bool hdm_valid;
 };
 
@@ -212,6 +216,56 @@ static int vfio_cxl_register_pfn_space(struct vfio_pci_core_device *vdev)
 	return register_pfn_address_space(&cxl->hdm_pfn_space);
 }
 
+static ssize_t vfio_cxl_comp_rw(struct vfio_pci_core_device *vdev,
+				char __user *buf, size_t count, loff_t *ppos,
+				bool iswrite)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
+	void *tmp;
+
+	if (pos >= cxl->hdm_len)
+		return -EINVAL;
+
+	/* The decoder registers take only aligned dword accesses. */
+	if (pos % sizeof(u32) || count % sizeof(u32))
+		return -EINVAL;
+
+	count = min_t(size_t, count, cxl->hdm_len - pos);
+
+	/*
+	 * The host committed and locked the physical decoder before the guest
+	 * saw the device, so the guest never drives it: absorb writes without
+	 * forwarding them to hardware. The guest programs a GPA that the VMM
+	 * virtualizes; reads return the live registers, which already report the
+	 * decoder committed. BASE_LOW and BASE_HIGH carry the host HPA, visible
+	 * only to the trusted VMM that virtualizes it away from the guest.
+	 */
+	if (iswrite) {
+		*ppos += count;
+		return count;
+	}
+
+	tmp = kmalloc(count, GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	memcpy_fromio(tmp, cxl->hdm_regs + pos, count);
+	if (copy_to_user(buf, tmp, count)) {
+		kfree(tmp);
+		return -EFAULT;
+	}
+	kfree(tmp);
+
+	*ppos += count;
+	return count;
+}
+
+static const struct vfio_pci_regops vfio_cxl_comp_regops = {
+	.rw = vfio_cxl_comp_rw,
+	.release = vfio_cxl_region_release,
+};
+
 static void vfio_cxl_release_hpa(void *data)
 {
 	struct vfio_cxl_state *cxl = data;
@@ -299,6 +353,21 @@ static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 		goto err;
 	}
 
+	/*
+	 * Map the HDM decoder registers so the decoder region can read them
+	 * live. The block location comes from the enumeration cache in
+	 * pdev->hdm; vfio-pci owns the BAR, so map without claiming the block.
+	 */
+	cxl->hdm_regs = devm_ioremap(&pdev->dev,
+				     pci_resource_start(pdev, pdev->hdm->hdm_bar) +
+				     pdev->hdm->hdm_offset, pdev->hdm->hdm_size);
+	if (!cxl->hdm_regs) {
+		ret = -ENOMEM;
+		goto err;
+	}
+
+	cxl->hdm_len = pdev->hdm->hdm_size;
+
 	/*
 	 * A Type-2 accelerator has no mailbox and no media-ready register, so
 	 * set media ready directly.
@@ -381,6 +450,13 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 	if (ret)
 		return ret;
 
+	ret = vfio_cxl_add_region(vdev, VFIO_REGION_SUBTYPE_CXL_COMP_REGS,
+				  &vfio_cxl_comp_regops, cxl->hdm_len,
+				  VFIO_REGION_INFO_FLAG_READ |
+				  VFIO_REGION_INFO_FLAG_WRITE);
+	if (ret)
+		goto err_unregister_mem;
+
 	/*
 	 * The HDM region is advertised mmap-able, so a fd holder can fault its
 	 * struct-page-less device memory in from the host CPU. Register it with
@@ -389,7 +465,7 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 	 */
 	ret = vfio_cxl_register_pfn_space(vdev);
 	if (ret && ret != -EOPNOTSUPP)
-		goto err_unregister_mem;
+		goto err_unregister_comp;
 
 	/*
 	 * The decoder is firmware-committed, so host access to the HDM range is
@@ -400,6 +476,9 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 
 	return 0;
 
+err_unregister_comp:
+	vfio_pci_core_unregister_dev_region(vdev);
+
 err_unregister_mem:
 	vfio_pci_core_unregister_dev_region(vdev);
 
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 1bf86763c0f7..8927a7a4e8e4 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -373,6 +373,8 @@ struct vfio_region_info_cap_type {
 /* CXL Type-2 device (0x1e98) sub-types for VFIO_REGION_TYPE_PCI_VENDOR_TYPE */
 /* CXL.mem HDM region of a Type-2 device, mmap-able */
 #define VFIO_REGION_SUBTYPE_CXL_MEM		(1)
+/* CXL HDM decoder registers: read live, guest writes are absorbed */
+#define VFIO_REGION_SUBTYPE_CXL_COMP_REGS	(2)
 
 /* sub-types for VFIO_REGION_TYPE_GFX */
 #define VFIO_REGION_SUBTYPE_GFX_EDID            (1)
-- 
2.25.1

[PATCH v5 21/27] vfio/cxl: Exclude the HDM decoder registers from direct BAR access

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The HDM decoder register block lives in the component BAR that vfio-pci
owns. The guest reads the decoder only through the trapped, read-only
decoder region; it must not reach the physical registers through the BAR
directly, and a host read of the range through a kernel mapping could
abort as a fatal host SError.

Add the block to the generic excluded-range list with
VFIO_PCI_EXCLUDE_MMAP | VFIO_PCI_EXCLUDE_READ | VFIO_PCI_EXCLUDE_WRITE
similar to the MSI-X exclusion.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index da04776356e4..37e8a3b54cfb 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -435,8 +435,25 @@ static int vfio_cxl_add_region(struct vfio_pci_core_device *vdev, u32 subtype,
 static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 {
 	struct vfio_cxl_state *cxl = vdev->cxl;
+	struct pci_dev *pdev = vdev->pdev;
 	int ret;
 
+	/*
+	 * Keep the HDM decoder block out of the guest's direct BAR access: the
+	 * guest reaches it only through the trapped decoder region, and a host
+	 * read of the range through a kernel mapping could abort as an SError.
+	 * Exclude it from mmap, fill reads with -1 and drop writes. The list is
+	 * cleared on close, so re-add it per open.
+	 */
+	ret = vfio_pci_core_add_excluded_range(vdev, pdev->hdm->hdm_bar,
+					       pdev->hdm->hdm_offset,
+					       cxl->hdm_len,
+					       VFIO_PCI_EXCLUDE_MMAP |
+					       VFIO_PCI_EXCLUDE_READ |
+					       VFIO_PCI_EXCLUDE_WRITE);
+	if (ret)
+		return ret;
+
 	/*
 	 * vfio_pci_core_disable() frees all dynamic regions on close, so register
 	 * them here per open rather than at bind. A failed first open never
-- 
2.25.1

[PATCH v5 22/27] vfio/cxl: Clear the HDM access gate after a hot reset

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

The HDM memory fault inserts a PFN only when the decoder is in a
known-good state (cxl->hdm_valid), so the host CPU never faults the
coherent range into a disabled decoder.

VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset rather than
the CXL reset sequence, so nothing restores the decoder in the
reset_done handler. A secondary bus reset only reaches a CXL endpoint
when the upstream port has SBR unmasked (CXL r3.1 sec 8.1.5.2); in that
case the reset decommits the decoder, so clear cxl->hdm_valid to keep a
later fault from inserting a PFN into a dead decoder.

A subsequent VFIO_DEVICE_RESET runs the CXL reset sequence and restores
it, and a masked SBR is a no-op that leaves the decoder intact.

vfio_cxl_sbr_unmasked() mirrors the cxl_sbr_masked() check the PCI core
uses (drivers/pci/pci.c) for its own CXL bus reset.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 38 ++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 37e8a3b54cfb..395b4a5b0956 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -514,8 +514,46 @@ static void vfio_cxl_reset_prepare(struct vfio_pci_core_device *vdev)
 {
 }
 
+/*
+ * A secondary bus reset only reaches a CXL endpoint when the upstream port
+ * has SBR unmasked (CXL r3.1 sec 8.1.5.2); otherwise the Bridge Control SBR
+ * bit is ignored and the decoder is left intact. Mirrors the cxl_sbr_masked()
+ * check the PCI core uses for its own CXL bus reset.
+ */
+static bool vfio_cxl_sbr_unmasked(struct pci_dev *pdev)
+{
+	struct pci_dev *bridge = pci_upstream_bridge(pdev);
+	u16 dvsec, ctl;
+
+	if (!bridge)
+		return false;
+
+	dvsec = pci_find_dvsec_capability(bridge, PCI_VENDOR_ID_CXL,
+					  PCI_DVSEC_CXL_PORT);
+	if (!dvsec)
+		return false;
+
+	if (pci_read_config_word(bridge, dvsec + PCI_DVSEC_CXL_PORT_CTL, &ctl))
+		return false;
+
+	return ctl & PCI_DVSEC_CXL_PORT_CTL_UNMASK_SBR;
+}
+
 static void vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)
 {
+	struct vfio_cxl_state *cxl = vdev->cxl;
+
+	/*
+	 * VFIO_DEVICE_PCI_HOT_RESET drives a plain secondary bus reset, not the
+	 * CXL-aware cxl_reset_bus_function(), so nothing restores the HDM
+	 * decoder here. When the upstream port has SBR unmasked the reset
+	 * decommits the decoder; gate host access to the HDM range so a later
+	 * fault cannot insert a PFN into a dead decoder. A VFIO_DEVICE_RESET
+	 * then runs the CXL reset sequence and restores it. A masked SBR is a
+	 * no-op and leaves the decoder intact.
+	 */
+	if (vfio_cxl_sbr_unmasked(vdev->pdev))
+		cxl->hdm_valid = false;
 }
 
 static const struct vfio_cxl_ops vfio_cxl_ops = {
-- 
2.25.1

[PATCH v5 23/27] vfio/cxl: Describe the CXL device and decoder geometry to userspace

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Advertise the assignment with a device-info flag so a VMM can tell a CXL
Type-2 device apart from a plain vfio-pci one. On the trapped decoder
register region, add a capability carrying the BAR and offset of the HDM
decoder block so the VMM can place the trapped window where the guest
expects to find it and size the guest's fixed memory window.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 16 ++++++++++++++++
 drivers/vfio/pci/vfio_pci_core.c     |  3 +++
 include/uapi/linux/vfio.h            | 18 ++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 395b4a5b0956..5fe8e35c63c4 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -261,9 +261,25 @@ static ssize_t vfio_cxl_comp_rw(struct vfio_pci_core_device *vdev,
 	return count;
 }
 
+static int vfio_cxl_comp_add_capability(struct vfio_pci_core_device *vdev,
+					struct vfio_pci_region *region,
+					struct vfio_info_cap *caps)
+{
+	struct cxl_hdm_info *hdm = vdev->pdev->hdm;
+	struct vfio_region_info_cap_cxl_comp_regs cap = {
+		.header.id = VFIO_REGION_INFO_CAP_CXL_COMP_REGS,
+		.header.version = 1,
+		.bar = hdm->hdm_bar,
+		.offset = hdm->hdm_offset,
+	};
+
+	return vfio_info_add_capability(caps, &cap.header, sizeof(cap));
+}
+
 static const struct vfio_pci_regops vfio_cxl_comp_regops = {
 	.rw = vfio_cxl_comp_rw,
 	.release = vfio_cxl_region_release,
+	.add_capability = vfio_cxl_comp_add_capability,
 };
 
 static void vfio_cxl_release_hpa(void *data)
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index 8913a9e24302..f02a5240aa71 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -1364,6 +1364,9 @@ static int vfio_pci_ioctl_get_info(struct vfio_pci_core_device *vdev,
 	if (vdev->reset_works)
 		info.flags |= VFIO_DEVICE_FLAGS_RESET;
 
+	if (vdev->cxl_ops)
+		info.flags |= VFIO_DEVICE_FLAGS_CXL;
+
 	info.num_regions = VFIO_PCI_NUM_REGIONS + vdev->num_regions;
 	info.num_irqs = VFIO_PCI_NUM_IRQS;
 
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 8927a7a4e8e4..8f9518f09bd0 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -215,6 +215,7 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_FSL_MC (1 << 6)	/* vfio-fsl-mc device */
 #define VFIO_DEVICE_FLAGS_CAPS	(1 << 7)	/* Info supports caps */
 #define VFIO_DEVICE_FLAGS_CDX	(1 << 8)	/* vfio-cdx device */
+#define VFIO_DEVICE_FLAGS_CXL	(1 << 9)	/* vfio-cxl device */
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
 	__u32   cap_offset;	/* Offset within info struct of first cap */
@@ -503,6 +504,23 @@ struct vfio_region_info_cap_nvlink2_lnkspd {
 	__u32 __pad;
 };
 
+/*
+ * Geometry of a CXL Type-2 device's HDM decoder registers, so a VMM can place
+ * the trapped component register window where the guest expects it. The trapped
+ * region spans the whole HDM decoder block (every decoder), not just decoder 0:
+ * a VMM reads the decoder count and each decoder's committed base from the block
+ * itself. Additional trapped component capabilities, such as CXL RAS, are
+ * exposed as their own region subtypes rather than by extending this cap.
+ */
+#define VFIO_REGION_INFO_CAP_CXL_COMP_REGS	6
+
+struct vfio_region_info_cap_cxl_comp_regs {
+	struct vfio_info_cap_header header;
+	__u32 bar;
+	__u32 __resv;
+	__aligned_u64 offset;
+};
+
 /**
  * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9,
  *				    struct vfio_irq_info)
-- 
2.25.1

[PATCH v5 24/27] vfio/cxl: Export the HDM memory region as a dma-buf

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A Type-2 accelerator issues ATS-translated DMA to addresses inside its
own HDM window, so that coherent host range must be present in the
guest's IOAS (the iommufd IOAS backing the nested SMMU stage-2). iommufd
maps a struct-page-less range only by fd, via IOMMU_IOAS_MAP_FILE over a
dma-buf; a userspace-VA IOMMU_IOAS_MAP of the HDM mmap is rejected
because the VMA is VM_IO | VM_PFNMAP. Without a dma-buf the range could
only be mapped through an out-of-tree PFNMAP work-around.

vfio-pci already exports BAR memory as a P2P dma-buf, but the exporter
is BAR-only: vfio_pci_core_feature_dma_buf() rejects any region index at
or above the ROM index, and vfio_pci_core_get_dmabuf_phys() resolves the
physical range from a PCI BAR. The HDM memory region is a dynamic
device-specific region, not a BAR.

Let a device-specific region reach the device's get_dmabuf_phys(): a
region index at or above VFIO_PCI_NUM_REGIONS skips the BAR-resource
check and is validated by the driver instead, bounded to the regions
that exist. Install a CXL-aware get_dmabuf_phys() in the vfio-cxl
provider that returns cxl->hpa_range for the HDM memory region and
delegates real BARs to the core, keeping the BAR path unchanged and the
core free of CXL knowledge.

The HDM window is coherent host memory with no p2pdma provider of its
own, so borrow BAR 0's, matching nvgrace-gpu's handling of its non-BAR
device memory. The iommufd importer does not consume the provider; the
scatterlist map path (real peer DMA) is left to a follow-up once
upstream grows a negotiated interconnect for coherent CXL memory.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 60 ++++++++++++++++++++++++++++
 drivers/vfio/pci/vfio_pci_dmabuf.c   | 27 +++++++++++--
 2 files changed, 83 insertions(+), 4 deletions(-)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 5fe8e35c63c4..55fa1f86850d 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -11,6 +11,7 @@
 #include <linux/mm.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-p2pdma.h>
 #include <linux/range.h>
 #include <linux/slab.h>
 #include <linux/uaccess.h>
@@ -27,6 +28,7 @@
  * @hdm_regs: mapped HDM decoder registers, read live by the decoder region
  * @hdm_len: length of the HDM decoder register block
  * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock
+ * @mem_region_index: vfio region index of the mmap-able HDM memory region
  */
 struct vfio_cxl_state {
 	struct cxl_dev_state cxlds;
@@ -36,6 +38,7 @@ struct vfio_cxl_state {
 	void __iomem *hdm_regs;
 	u32 hdm_len;
 	bool hdm_valid;
+	unsigned int mem_region_index;
 };
 
 static unsigned long vfio_cxl_mem_pgoff(struct vm_area_struct *vma,
@@ -289,6 +292,50 @@ static void vfio_cxl_release_hpa(void *data)
 	release_mem_region(cxl->hpa_range.start, range_len(&cxl->hpa_range));
 }
 
+/*
+ * Resolve the physical range that backs a dma-buf export. The core exporter
+ * only knows BARs; teach it the HDM memory region so a guest IOAS can map the
+ * coherent window by fd (IOMMU_IOAS_MAP_FILE) instead of the removed PFNMAP
+ * work-around. Real BARs stay on the byte-identical core path.
+ */
+static int vfio_cxl_get_dmabuf_phys(struct vfio_pci_core_device *vdev,
+				    struct p2pdma_provider **provider,
+				    unsigned int region_index,
+				    struct phys_vec *phys_vec,
+				    struct vfio_region_dma_range *dma_ranges,
+				    size_t nr_ranges)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+
+	/* Real BARs go through the core P2P exporter unchanged. */
+	if (region_index < VFIO_PCI_NUM_REGIONS)
+		return vfio_pci_core_get_dmabuf_phys(vdev, provider,
+						     region_index, phys_vec,
+						     dma_ranges, nr_ranges);
+
+	/* Of the device regions, only the HDM memory window is exportable. */
+	if (region_index != cxl->mem_region_index)
+		return -EINVAL;
+
+	/*
+	 * The HDM window is coherent host memory, not BAR MMIO, so it has no
+	 * p2pdma provider of its own. Borrow BAR 0's: the P2P properties match
+	 * and the iommufd importer does not consume the provider. The sgt map
+	 * path (real peer DMA) is not supported for the HDM window.
+	 */
+	*provider = pcim_p2pdma_provider(vdev->pdev, 0);
+	if (!*provider)
+		return -EINVAL;
+
+	return vfio_pci_core_fill_phys_vec(phys_vec, dma_ranges, nr_ranges,
+					  cxl->hpa_range.start,
+					  range_len(&cxl->hpa_range));
+}
+
+static const struct vfio_pci_device_ops vfio_cxl_pci_dev_ops = {
+	.get_dmabuf_phys = vfio_cxl_get_dmabuf_phys,
+};
+
 static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
 {
 	struct pci_dev *pdev = vdev->pdev;
@@ -483,6 +530,19 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
 	if (ret)
 		return ret;
 
+	/* Record where the HDM memory region landed for the dma-buf export. */
+	cxl->mem_region_index = VFIO_PCI_NUM_REGIONS + vdev->num_regions - 1;
+
+	/*
+	 * Override the device ops so a dma-buf export of the HDM memory region
+	 * resolves to the coherent host range. This is done at open, not init:
+	 * vfio_pci_probe() resets pci_ops after vfio_alloc_device() returns, so
+	 * an override installed during init would be clobbered. Only a CXL device
+	 * reaches this hook (cxl_ops is set on init success), so a fallback to
+	 * plain vfio-pci keeps the core ops.
+	 */
+	vdev->pci_ops = &vfio_cxl_pci_dev_ops;
+
 	ret = vfio_cxl_add_region(vdev, VFIO_REGION_SUBTYPE_CXL_COMP_REGS,
 				  &vfio_cxl_comp_regops, cxl->hdm_len,
 				  VFIO_REGION_INFO_FLAG_READ |
diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
index c16f460c01d6..436c616d5b66 100644
--- a/drivers/vfio/pci/vfio_pci_dmabuf.c
+++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
@@ -178,6 +178,15 @@ int vfio_pci_core_get_dmabuf_phys(struct vfio_pci_core_device *vdev,
 {
 	struct pci_dev *pdev = vdev->pdev;
 
+	/*
+	 * This resolver only handles PCI BARs. A device-specific region index
+	 * (>= PCI_STD_NUM_BARS) would index pdev->resource[] out of bounds via
+	 * pcim_p2pdma_provider(), so reject it; a driver that exports such a
+	 * region installs its own get_dmabuf_phys.
+	 */
+	if (region_index >= PCI_STD_NUM_BARS)
+		return -EINVAL;
+
 	*provider = pcim_p2pdma_provider(pdev, region_index);
 	if (!*provider)
 		return -EINVAL;
@@ -227,6 +236,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
 	struct vfio_pci_dma_buf *priv;
 	size_t length;
+	u32 index;
 	int ret;
 
 	if (!vdev->pci_ops || !vdev->pci_ops->get_dmabuf_phys)
@@ -243,13 +253,22 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
 	if (!get_dma_buf.nr_ranges || get_dma_buf.flags)
 		return -EINVAL;
 
+	index = get_dma_buf.region_index;
+
 	/*
-	 * For PCI the region_index is the BAR number like everything
-	 * else.  Check that PCI resources have been claimed for it.
+	 * A fixed region index is the BAR number; only a BAR can be exported
+	 * and its PCI resource must be claimed. A device-specific region (index
+	 * >= VFIO_PCI_NUM_REGIONS) has no BAR resource and is validated by the
+	 * device's get_dmabuf_phys instead, but the index must name a region
+	 * that exists.
 	 */
-	if (get_dma_buf.region_index >= VFIO_PCI_ROM_REGION_INDEX ||
-	    IS_ERR(vfio_pci_core_get_iomap(vdev, get_dma_buf.region_index)))
+	if (index < VFIO_PCI_NUM_REGIONS) {
+		if (index >= VFIO_PCI_ROM_REGION_INDEX ||
+		    IS_ERR(vfio_pci_core_get_iomap(vdev, index)))
+			return -ENODEV;
+	} else if (index - VFIO_PCI_NUM_REGIONS >= vdev->num_regions) {
 		return -ENODEV;
+	}
 
 	dma_ranges = memdup_array_user(&arg->dma_ranges, get_dma_buf.nr_ranges,
 				       sizeof(*dma_ranges));
-- 
2.25.1

[PATCH v5 25/27] vfio/cxl: Run the CXL reset at the vfio reset points

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

A CXL Type-2 function must not take an FLR: it resets the coherent
CXL.mem state and corrupts the HDM decoder. The PCI core already reflects
this, ordering cxl_reset ahead of flr in pci_reset_fn_methods[], so a
function reset of a CXL device runs the DVSEC reset sequence rather than
FLR.

Route the vfio function-reset points (VFIO_DEVICE_RESET and the
virtualized PCIe/AF FLR writes) through a CXL reset op that runs
cxl_reset_dvsec_sequence(). The sequence resets the function, always
clearing device memory, and restores the HDM decoder and PCI config
state, so it is a complete replacement for pci_try_reset_function() on a
CXL device. The op runs under memory_lock and not the PCI device lock, so
cxl_reset_dvsec_sequence() can take the device lock itself.

Clear hdm_valid for the duration of the reset so a fault cannot insert a
PFN into a decoder that is being torn down, and restore it once the
sequence has put the decoder back.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 drivers/vfio/pci/cxl/vfio_cxl_core.c | 41 +++++++++++++++
 drivers/vfio/pci/vfio_pci_config.c   | 49 +++++++++++++++---
 drivers/vfio/pci/vfio_pci_core.c     | 77 +++++++++++++++++++++++-----
 drivers/vfio/pci/vfio_pci_priv.h     |  1 +
 include/linux/vfio_pci_core.h        |  4 ++
 5 files changed, 152 insertions(+), 20 deletions(-)

diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
index 55fa1f86850d..795362aea344 100644
--- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
+++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
@@ -632,6 +632,45 @@ static void vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)
 		cxl->hdm_valid = false;
 }
 
+/*
+ * Run the CXL DVSEC reset sequence in place of a PCI function reset. A CXL
+ * Type-2 function must not take an FLR (it would corrupt CXL.mem), so the vfio
+ * reset points route here. The sequence resets the function, always clearing
+ * device memory, and restores the HDM decoder. The caller holds memory_lock,
+ * and this path does not hold the PCI device lock, so cxl_reset_dvsec_sequence()
+ * can take it.
+ */
+static int vfio_cxl_reset(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+	int ret;
+
+	lockdep_assert_held_write(&vdev->memory_lock);
+
+	/* Host CPU access to the HDM range is unsafe until the decoder is back. */
+	cxl->hdm_valid = false;
+
+	ret = cxl_reset_dvsec_sequence(vdev->pdev);
+	if (!ret)
+		cxl->hdm_valid = true;
+
+	return ret;
+}
+
+/*
+ * The HDM dma-buf may be armed only while the decoder is valid. After a failed
+ * reset hdm_valid is clear, so the generic memory-enable re-arm must skip the
+ * dma-buf rather than map DMA onto an unrestored decoder.
+ */
+static bool vfio_cxl_hdm_active(struct vfio_pci_core_device *vdev)
+{
+	struct vfio_cxl_state *cxl = vdev->cxl;
+
+	lockdep_assert_held_write(&vdev->memory_lock);
+
+	return cxl->hdm_valid;
+}
+
 static const struct vfio_cxl_ops vfio_cxl_ops = {
 	.init		= vfio_cxl_init_device,
 	.release	= vfio_cxl_release_device,
@@ -639,6 +678,8 @@ static const struct vfio_cxl_ops vfio_cxl_ops = {
 	.close_device	= vfio_cxl_close_device,
 	.reset_prepare	= vfio_cxl_reset_prepare,
 	.reset_done	= vfio_cxl_reset_done,
+	.reset		= vfio_cxl_reset,
+	.hdm_active	= vfio_cxl_hdm_active,
 	.owner		= THIS_MODULE,
 };
 
diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
index 9a020a768055..8a5a737efa31 100644
--- a/drivers/vfio/pci/vfio_pci_config.c
+++ b/drivers/vfio/pci/vfio_pci_config.c
@@ -630,7 +630,14 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
 		*virt_cmd &= cpu_to_le16(~mask);
 		*virt_cmd |= cpu_to_le16(new_cmd & mask);
 
-		if (__vfio_pci_memory_enabled(vdev))
+		/*
+		 * Re-arm the dma-bufs on memory-enable, but keep a CXL device's
+		 * HDM dma-buf revoked while the decoder is unrestored (a failed
+		 * reset leaves hdm_valid clear); re-arming would map DMA onto a
+		 * decoder the fault path still gates. Plain vfio-pci is unchanged.
+		 */
+		if (__vfio_pci_memory_enabled(vdev) &&
+		    (!vdev->cxl_ops || vdev->cxl_ops->hdm_active(vdev)))
 			vfio_pci_dma_buf_move(vdev, false);
 		up_write(&vdev->memory_lock);
 	}
@@ -720,7 +727,8 @@ static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev,
 	}
 
 	vfio_pci_set_power_state(vdev, state);
-	if (__vfio_pci_memory_enabled(vdev))
+	if (__vfio_pci_memory_enabled(vdev) &&
+	    (!vdev->cxl_ops || vdev->cxl_ops->hdm_active(vdev)))
 		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
 }
@@ -910,8 +918,14 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,
 		if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) {
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
 			vfio_pci_dma_buf_move(vdev, true);
-			pci_try_reset_function(vdev->pdev);
-			if (__vfio_pci_memory_enabled(vdev))
+			ret = vfio_pci_reset_function(vdev);
+			/*
+			 * Keep the HDM dma-buf revoked if a CXL reset
+			 * failed; re-arming would map DMA onto an
+			 * unrestored decoder. Mirrors the reset ioctl.
+			 */
+			if (__vfio_pci_memory_enabled(vdev) &&
+			    (!vdev->cxl_ops || !ret))
 				vfio_pci_dma_buf_move(vdev, false);
 			up_write(&vdev->memory_lock);
 		}
@@ -995,8 +1009,14 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,
 		if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) {
 			vfio_pci_zap_and_down_write_memory_lock(vdev);
 			vfio_pci_dma_buf_move(vdev, true);
-			pci_try_reset_function(vdev->pdev);
-			if (__vfio_pci_memory_enabled(vdev))
+			ret = vfio_pci_reset_function(vdev);
+			/*
+			 * Keep the HDM dma-buf revoked if a CXL reset
+			 * failed; re-arming would map DMA onto an
+			 * unrestored decoder. Mirrors the reset ioctl.
+			 */
+			if (__vfio_pci_memory_enabled(vdev) &&
+			    (!vdev->cxl_ops || !ret))
 				vfio_pci_dma_buf_move(vdev, false);
 			up_write(&vdev->memory_lock);
 		}
@@ -1781,9 +1801,22 @@ static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,
 		status2 |= PCI_DVSEC_CXL_CACHE_INV;
 	}
 	if (ctrl2 & PCI_DVSEC_CXL_INIT_CXL_RST) {
+		int ret = 0;
+
 		ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST;
-		status2 &= ~PCI_DVSEC_CXL_RST_ERR;
-		status2 |= PCI_DVSEC_CXL_RST_DONE;
+
+		if (vdev->cxl_ops && vdev->cxl_ops->reset) {
+			vfio_pci_zap_and_down_write_memory_lock(vdev);
+			vfio_pci_dma_buf_move(vdev, true);
+			ret = vfio_pci_reset_function(vdev);
+			if (__vfio_pci_memory_enabled(vdev) &&
+			    (!vdev->cxl_ops || !ret))
+				vfio_pci_dma_buf_move(vdev, false);
+			up_write(&vdev->memory_lock);
+		}
+
+		status2 &= ~(PCI_DVSEC_CXL_RST_DONE | PCI_DVSEC_CXL_RST_ERR);
+		status2 |= ret ? PCI_DVSEC_CXL_RST_ERR : PCI_DVSEC_CXL_RST_DONE;
 	}
 
 	*pctrl2 = cpu_to_le16(ctrl2);
diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
index f02a5240aa71..8bd4db7afefe 100644
--- a/drivers/vfio/pci/vfio_pci_core.c
+++ b/drivers/vfio/pci/vfio_pci_core.c
@@ -643,8 +643,27 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
 		goto out_power;
 
 	/* If reset fails because of the device lock, fail this path entirely */
-	ret = pci_try_reset_function(pdev);
-	if (ret == -EAGAIN)
+	if (vdev->cxl_ops && vdev->cxl_ops->reset) {
+		/*
+		 * VM power-on resets a CXL Type-2 device through its DVSEC
+		 * sequence. vconfig is not built yet here, so take memory_lock
+		 * and call the op directly rather than the wrapper.
+		 */
+		down_write(&vdev->memory_lock);
+		ret = vdev->cxl_ops->reset(vdev);
+		up_write(&vdev->memory_lock);
+	} else {
+		ret = pci_try_reset_function(pdev);
+	}
+
+	/*
+	 * -EAGAIN means the reset could not run. For a CXL device any reset
+	 * error must also fail the open: a failed DVSEC reset can leave the HDM
+	 * decoder cleared or unrestored, and continuing would expose the HDM
+	 * region for host access through a decoder in an unknown state.
+	 */
+	if (ret == -EAGAIN ||
+	    (vdev->cxl_ops && vdev->cxl_ops->reset && ret))
 		goto out_disable_device;
 
 	vdev->reset_works = !ret;
@@ -845,16 +864,30 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
 	 * overwrite the previously restored configuration information.
 	 */
 	if (vdev->reset_works) {
-		bridge = pci_upstream_bridge(pdev);
-		if (bridge && !pci_dev_trylock(bridge))
-			goto out_restore_state;
-		if (pci_dev_trylock(pdev)) {
-			if (!__pci_reset_function_locked(pdev))
+		if (vdev->cxl_ops && vdev->cxl_ops->reset) {
+			/*
+			 * VM power-off resets a CXL Type-2 device through its
+			 * DVSEC sequence. The sequence takes its own device lock,
+			 * so run it outside the lock below.
+			 * vconfig is already freed here, so call the op directly
+			 * under memory_lock rather than the wrapper.
+			 */
+			down_write(&vdev->memory_lock);
+			if (!vdev->cxl_ops->reset(vdev))
 				vdev->needs_reset = false;
-			pci_dev_unlock(pdev);
+			up_write(&vdev->memory_lock);
+		} else {
+			bridge = pci_upstream_bridge(pdev);
+			if (bridge && !pci_dev_trylock(bridge))
+				goto out_restore_state;
+			if (pci_dev_trylock(pdev)) {
+				if (!__pci_reset_function_locked(pdev))
+					vdev->needs_reset = false;
+				pci_dev_unlock(pdev);
+			}
+			if (bridge)
+				pci_dev_unlock(bridge);
 		}
-		if (bridge)
-			pci_dev_unlock(bridge);
 	}
 
 out_restore_state:
@@ -1592,6 +1625,20 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev,
 	return ret;
 }
 
+/*
+ * Reset the function. A CXL device runs the CXL DVSEC reset sequence in place
+ * of a PCI function reset: it replaces FLR (which would corrupt CXL.mem),
+ * always clears device memory, and restores the HDM decoder. Callers hold
+ * memory_lock for write.
+ */
+int vfio_pci_reset_function(struct vfio_pci_core_device *vdev)
+{
+	if (!vdev->cxl_ops || !vdev->cxl_ops->reset)
+		return pci_try_reset_function(vdev->pdev);
+
+	return vdev->cxl_ops->reset(vdev);
+}
+
 static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 				void __user *arg)
 {
@@ -1614,8 +1661,14 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
 	vfio_pci_set_power_state(vdev, PCI_D0);
 
 	vfio_pci_dma_buf_move(vdev, true);
-	ret = pci_try_reset_function(vdev->pdev);
-	if (__vfio_pci_memory_enabled(vdev))
+	ret = vfio_pci_reset_function(vdev);
+	/*
+	 * Re-arm the dma-bufs on success. A CXL device whose reset failed leaves
+	 * the HDM decoder unrestored and hdm_valid clear, so re-arming its HDM
+	 * dma-buf would map device DMA onto a decoder the fault path still gates;
+	 * keep it revoked until a reset succeeds. Plain vfio-pci is unchanged.
+	 */
+	if (__vfio_pci_memory_enabled(vdev) && (!vdev->cxl_ops || !ret))
 		vfio_pci_dma_buf_move(vdev, false);
 	up_write(&vdev->memory_lock);
 
diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
index c268c99aea82..e1ef21806a2f 100644
--- a/drivers/vfio/pci/vfio_pci_priv.h
+++ b/drivers/vfio/pci/vfio_pci_priv.h
@@ -78,6 +78,7 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev,
 			     pci_power_t state);
 
 void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev);
+int vfio_pci_reset_function(struct vfio_pci_core_device *vdev);
 u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev);
 void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev,
 					u16 cmd);
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index 39a28cc6ae8c..231679dead45 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -74,6 +74,10 @@ struct vfio_cxl_ops {
 	void	(*close_device)(struct vfio_pci_core_device *vdev);
 	void	(*reset_prepare)(struct vfio_pci_core_device *vdev);
 	void	(*reset_done)(struct vfio_pci_core_device *vdev);
+	/* Run the CXL reset (always clears CXL.mem) in place of FLR */
+	int	(*reset)(struct vfio_pci_core_device *vdev);
+	/* True while the HDM range is valid and its dma-buf may be armed */
+	bool	(*hdm_active)(struct vfio_pci_core_device *vdev);
 	/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
 	struct module *owner;
 };
-- 
2.25.1

[PATCH v5 26/27] Documentation: vfio-pci: Document CXL Type-2 device passthrough

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Describe the vfio-cxl provider module:
- Address model,
- Two regions,
- Live decoder region and the guest commit,
- DVSEC virtualization,
- The dma-buf export for iommufd stage-2 mapping,
- The reset contract

so the supported topology and the kernel/VMM split are on record.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 Documentation/driver-api/index.rst        |   1 +
 Documentation/driver-api/vfio-pci-cxl.rst | 188 ++++++++++++++++++++++
 MAINTAINERS                               |   1 +
 3 files changed, 190 insertions(+)
 create mode 100644 Documentation/driver-api/vfio-pci-cxl.rst

diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
index 6601a258690f..733ea1b64c72 100644
--- a/Documentation/driver-api/index.rst
+++ b/Documentation/driver-api/index.rst
@@ -46,6 +46,7 @@ of interest to most developers working on device drivers.
    uio-howto
    vfio-mediated-device
    vfio
+   vfio-pci-cxl
    vfio-pci-device-specific-driver-acceptance
 
 Bus-level documentation
diff --git a/Documentation/driver-api/vfio-pci-cxl.rst b/Documentation/driver-api/vfio-pci-cxl.rst
new file mode 100644
index 000000000000..d33163df33b4
--- /dev/null
+++ b/Documentation/driver-api/vfio-pci-cxl.rst
@@ -0,0 +1,188 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+VFIO-PCI: CXL Type-2 device passthrough
+=======================================
+
+Overview
+========
+
+A CXL Type-2 device is an accelerator (for example a GPU) that exposes
+host-managed device memory through an HDM decoder. vfio-pci alone does
+not expose the HDM decoder registers or the CXL Device DVSEC, and it does
+not place the device memory at a guest-chosen address.
+
+The optional ``vfio-cxl`` module provides that. It is a provider for
+vfio-pci-core, not a separate PCI driver. vfio-pci-core stays free of CXL
+knowledge and loads ``vfio-cxl`` when it binds a CXL device.
+
+Address model
+=============
+
+The HDM memory is a coherent host physical range (HPA). The host kernel
+resolves that range before the guest sees the device, and owns it for the
+bind lifetime. The guest only chooses where the memory appears in its own
+physical address space (GPA), by programming a virtual endpoint HDM
+decoder. The guest never reprograms the physical decoder.
+
+The kernel holds the HPA and does not see the GPA. The guest programs a
+GPA and does not see the HPA. The VMM holds the device fd, reads the
+committed base from the decoder-register region described below, and maps
+the HPA-backed HDM region at the GPA the guest committed. The base the
+guest reads back is the GPA, not the HPA.
+
+Driver model
+============
+
+There is no separate PCI driver. vfio-pci binds the device. During bind,
+vfio-pci-core detects a CXL device (``pcie_is_cxl()``), loads ``vfio-cxl``
+with ``request_module()``, and calls the registered ``struct
+vfio_cxl_ops``. The module reference is pinned for the bind lifetime so
+``vfio-cxl`` cannot unload while a device is bound.
+
+At bind the provider creates the CXL memory device, takes ownership of the
+whole component-register BAR, and (a Type-2 function has no mailbox) marks
+the media ready directly. A non-CXL device, or a CXL device whose CXL
+setup fails, falls back to the ordinary vfio-pci paths; the failure is not
+fatal to the bind.
+
+Regions
+=======
+
+``vfio-cxl`` adds two regions under the PCI vendor-type region
+``VFIO_REGION_TYPE_PCI_VENDOR_TYPE`` for the CXL vendor (0x1e98):
+
+``VFIO_REGION_SUBTYPE_CXL_MEM``
+    The HDM memory region, backed by the fixed host physical range. It can
+    be mapped with mmap. The fault handler inserts the host PFNs, including
+    2 MB PMDs when the mapping is aligned, but only while the device is in
+    a state where a host CPU access to the range is safe (Memory Space
+    enabled, media ready, and the decoder not mid-reset); otherwise the
+    fault takes ``SIGBUS``. The struct-page-less range is registered with
+    the memory-failure machinery so a memory error can be contained. The
+    VMM maps this region into guest memory at the committed GPA, and can
+    also export it as a dma-buf (see below).
+
+``VFIO_REGION_SUBTYPE_CXL_COMP_REGS``
+    The HDM decoder registers. Access is read/write only (no mmap) and
+    must be dword aligned; a misaligned or out-of-range access returns
+    ``-EINVAL``. Reads are served live from the committed decoder. Guest
+    writes are absorbed: the host already programmed and locked the
+    physical decoder, so the register block is read-only to the guest and
+    a write is dropped rather than forwarded. The region carries a
+    ``VFIO_REGION_INFO_CAP_CXL_COMP_REGS`` capability that reports the
+    component BAR and the offset of the decoder block within it, so the
+    VMM can place the trapped window where the guest expects it.
+
+The decoder register range is also excluded from the direct component-BAR
+mmap and from host-side reads and writes: a kernel read of that range
+through a mapping could abort on the fabric as a host SError, so reads
+return ones and writes are dropped. The rest of the component BAR is a
+normal vfio-pci BAR.
+
+Guest decoder and commit
+========================
+
+The guest programs its virtual endpoint decoder through the trapped
+region: it writes a base (a GPA), a size, and then the COMMIT bit. The
+host already resolved and committed the physical placement before the
+guest ran, so a live read of the decoder always shows COMMITTED and the
+guest's commit poll completes. The physical decoder is never rewritten;
+the guest's writes are absorbed.
+
+The VMM observes the commit, reads the committed base, and maps the HDM
+region at that GPA.
+
+CXL Device DVSEC
+================
+
+The kernel virtualizes the CXL Device DVSEC body through the config-space
+permission hooks. Reads and writes inside the DVSEC body use a per-open
+shadow; a guest write stays in the shadow and does not reach hardware.
+Accesses outside the DVSEC body go to the device as usual.
+
+The self-clearing Control2 doorbells (Initiate CXL Reset and Initiate
+Cache Write-Back and Invalidate) are never forwarded to hardware. The
+kernel synthesizes their completion in the shadow so the guest poll
+finishes, and runs the real operation at the vfio reset points (see
+below).
+
+DMA and iommufd
+===============
+
+A Type-2 accelerator issues ATS-translated DMA to addresses inside its own
+HDM window, so that range must be present in the guest IOAS that backs the
+nested stage-2 translation. The HDM range is struct-page-less coherent
+memory, which a userspace-VA ``IOMMU_IOAS_MAP`` cannot pin.
+
+The HDM memory region is therefore exportable as a dma-buf:
+``VFIO_DEVICE_FEATURE_DMA_BUF`` on that region returns an fd that iommufd
+maps with ``IOMMU_IOAS_MAP_FILE``, mapping the physical range without a VA
+or a page pin. The dma-buf is revoked whenever the mapping is torn down
+(reset, power transition, teardown), so a stale stage-2 mapping cannot
+outlive the HDM window.
+
+Reset
+=====
+
+A CXL Type-2 function must not take a Function Level Reset: an FLR resets
+the coherent CXL.mem state and the HDM decoder. The PCI core reflects this
+by preferring the CXL reset over FLR, so a function reset of a CXL device
+runs the CXL DVSEC reset sequence, which resets the function and then
+restores the HDM decoder and the PCI config state.
+
+A guest requests a reset by writing Initiate CXL Reset in the DVSEC. That
+write only stamps completion in the shadow. The real reset runs at the vfio
+reset points (the reset ioctl and a virtualized FLR through config space):
+the kernel zaps the HDM mapping and revokes the dma-buf, then runs the CXL
+reset, which always clears the device memory, and restores and re-samples
+the decoder afterwards. A CXL port masks Secondary Bus Reset by default, so a
+``VFIO_DEVICE_PCI_HOT_RESET`` does not reach the endpoint and the HDM
+state is untouched. If the port has SBR unmasked the reset can decommit
+the decoder without restoring it, so the reset_done handler gates HDM
+access; a ``VFIO_DEVICE_RESET`` then runs the CXL reset sequence and
+restores it.
+
+The decoder register region is served by live reads of the hardware
+decoder with guest writes absorbed: the decoder is committed and locked by
+the host, so a guest can neither decommit nor reprogram it, and the kernel
+keeps no shadow of the decoder state. After a reset the kernel restores and
+re-samples the firmware-committed decoder, so the geometry the guest reads
+back is unchanged. A VMM that dropped its HDM mapping, for example across a
+reset or a D3hot->D0 transition, must rescan the decoder and rebuild its
+stage-2 mapping before it resumes HDM access.
+
+UAPI
+====
+
+``VFIO_DEVICE_FLAGS_CXL``
+    Set in ``VFIO_DEVICE_GET_INFO`` flags for a CXL Type-2 device.
+
+``VFIO_REGION_TYPE_PCI_VENDOR_TYPE | 0x1e98`` with
+``VFIO_REGION_SUBTYPE_CXL_MEM`` / ``VFIO_REGION_SUBTYPE_CXL_COMP_REGS``
+    Reported through the region-info ``VFIO_REGION_INFO_CAP_TYPE``
+    capability. Userspace finds each region by scanning for the type and
+    subtype.
+
+``VFIO_REGION_INFO_CAP_CXL_COMP_REGS``
+    On the component-register region, reports the component BAR index and
+    the decoder-block offset within it.
+
+``VFIO_DEVICE_FEATURE_DMA_BUF``
+    On the HDM memory region, returns a dma-buf fd for
+    ``IOMMU_IOAS_MAP_FILE``.
+
+The HDM decoder register layout is available to a VMM without a private
+kernel header via ``uapi/cxl/cxl_regs.h``.
+
+Scope
+=====
+
+This support covers a single, non-interleaved endpoint decoder on a
+directly attached device. Multi-decoder devices, interleave, and
+switch-attached topologies are not supported. The interfaces are
+structured so those cases can be added later without changing the UAPI
+described here.
+
+A selftest, ``tools/testing/selftests/vfio/vfio_cxl_type2_test.c``,
+exercises the interfaces above on a bound device.
diff --git a/MAINTAINERS b/MAINTAINERS
index 75d472d7ca07..ab099b523054 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28644,6 +28644,7 @@ M:	Manish Honap <mhonap@nvidia.com>
 L:	kvm@vger.kernel.org
 L:	linux-cxl@vger.kernel.org
 S:	Supported
+F:	Documentation/driver-api/vfio-pci-cxl.rst
 F:	drivers/vfio/pci/cxl/
 
 VFIO DRIVER
-- 
2.25.1

[PATCH v5 27/27] selftests/vfio: Add CXL Type-2 passthrough tests

[mhonap]

From: Manish Honap <mhonap@nvidia.com>

Exercise the vfio-cxl contract on a bound Type-2 device: the CXL
device-info flag and the two vendor-type regions, the HDM memory mmap
(including a 2 MB huge fault) and its survival across a Memory Space
toggle, the read-only trapped decoder region against the live
host-committed decoder, and the guest's DVSEC reset doorbell and manual
HDM discovery.

Map the HDM memory the way a VMM does: export it as a dma-buf with
VFIO_DEVICE_FEATURE_DMA_BUF and map the fd into the IOAS with
IOMMU_IOAS_MAP_FILE. The struct-page-less range cannot be pinned through a
userspace VA, so the by-fd path is the one that works. Skip when the
kernel has no CXL dma-buf exporter, so the test becomes a real pass once
that support is present.

Add a partial-mmap path to the vfio selftest library so the trapped
component BAR maps its sparse ranges, and an __iommu_map_file() helper for
the dma-buf mapping.

Assisted-by: LLM
Signed-off-by: Manish Honap <mhonap@nvidia.com>
---
 MAINTAINERS                                   |   1 +
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../vfio/lib/include/libvfio/iommu.h          |   3 +
 tools/testing/selftests/vfio/lib/iommu.c      |  29 +
 .../selftests/vfio/lib/vfio_pci_device.c      |  57 +-
 .../selftests/vfio/vfio_cxl_type2_test.c      | 780 ++++++++++++++++++
 6 files changed, 868 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/vfio/vfio_cxl_type2_test.c

diff --git a/MAINTAINERS b/MAINTAINERS
index ab099b523054..a65dc71ada5a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -28646,6 +28646,7 @@ L:	linux-cxl@vger.kernel.org
 S:	Supported
 F:	Documentation/driver-api/vfio-pci-cxl.rst
 F:	drivers/vfio/pci/cxl/
+F:	tools/testing/selftests/vfio/vfio_cxl_type2_test.c
 
 VFIO DRIVER
 M:	Alex Williamson <alex@shazbot.org>
diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 2c32c48db509..08f88e88cb4d 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -13,6 +13,7 @@ TEST_GEN_PROGS += vfio_pci_device_test
 TEST_GEN_PROGS += vfio_pci_device_init_perf_test
 TEST_GEN_PROGS += vfio_pci_driver_test
 TEST_GEN_PROGS += vfio_pci_sriov_uapi_test
+TEST_GEN_PROGS += vfio_cxl_type2_test
 
 TEST_FILES += scripts/cleanup.sh
 TEST_FILES += scripts/lib.sh
diff --git a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
index e9a3386a4719..77040fdb8848 100644
--- a/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
+++ b/tools/testing/selftests/vfio/lib/include/libvfio/iommu.h
@@ -42,6 +42,9 @@ static inline void iommu_map(struct iommu *iommu, struct dma_region *region)
 	VFIO_ASSERT_EQ(__iommu_map(iommu, region), 0);
 }
 
+int __iommu_map_file(struct iommu *iommu, int fd, u64 start, u64 length,
+		     iova_t iova);
+
 int __iommu_unmap(struct iommu *iommu, struct dma_region *region, u64 *unmapped);
 
 static inline void iommu_unmap(struct iommu *iommu, struct dma_region *region)
diff --git a/tools/testing/selftests/vfio/lib/iommu.c b/tools/testing/selftests/vfio/lib/iommu.c
index b6f3c5c84e01..9109cedd734c 100644
--- a/tools/testing/selftests/vfio/lib/iommu.c
+++ b/tools/testing/selftests/vfio/lib/iommu.c
@@ -149,6 +149,35 @@ int __iommu_map(struct iommu *iommu, struct dma_region *region)
 	return 0;
 }
 
+/*
+ * Map a range of a file (a memfd or a supported dma-buf, such as a VFIO PCI
+ * dma-buf from VFIO_DEVICE_FEATURE_DMA_BUF) into the IOAS by fd. This is an
+ * iommufd-only ioctl; the legacy VFIO container has no equivalent.
+ */
+int __iommu_map_file(struct iommu *iommu, int fd, u64 start, u64 length,
+		     iova_t iova)
+{
+	struct iommu_ioas_map_file args = {
+		.size = sizeof(args),
+		.flags = IOMMU_IOAS_MAP_READABLE |
+			 IOMMU_IOAS_MAP_WRITEABLE |
+			 IOMMU_IOAS_MAP_FIXED_IOVA,
+		.ioas_id = iommu->ioas_id,
+		.fd = fd,
+		.start = start,
+		.length = length,
+		.iova = iova,
+	};
+
+	if (!iommu->iommufd)
+		return -EINVAL;
+
+	if (ioctl(iommu->iommufd, IOMMU_IOAS_MAP_FILE, &args))
+		return -errno;
+
+	return 0;
+}
+
 static int __vfio_iommu_unmap(int fd, u64 iova, u64 size, u32 flags, u64 *unmapped)
 {
 	struct vfio_iommu_type1_dma_unmap args = {
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 4063a0e2b3df..5f5ec3a70bd8 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -187,9 +187,31 @@ static void vfio_pci_region_get(struct vfio_pci_device *device, int index,
 	ioctl_assert(device->fd, VFIO_DEVICE_GET_REGION_INFO, info);
 }
 
+/* Return the sparse-mmap capability in @info, or NULL if the region has none. */
+static struct vfio_region_info_cap_sparse_mmap *
+vfio_pci_sparse_mmap_cap(struct vfio_region_info *info)
+{
+	struct vfio_info_cap_header *hdr;
+	u32 offset;
+
+	if (!(info->flags & VFIO_REGION_INFO_FLAG_CAPS))
+		return NULL;
+
+	for (offset = info->cap_offset; offset; offset = hdr->next) {
+		hdr = (void *)info + offset;
+		if (hdr->id == VFIO_REGION_INFO_CAP_SPARSE_MMAP)
+			return (struct vfio_region_info_cap_sparse_mmap *)hdr;
+	}
+
+	return NULL;
+}
+
 static void vfio_pci_bar_map(struct vfio_pci_device *device, int index)
 {
 	struct vfio_pci_bar *bar = &device->bars[index];
+	struct vfio_region_info_cap_sparse_mmap *sparse;
+	u8 infobuf[1024] = {};
+	struct vfio_region_info *info = (void *)infobuf;
 	size_t align, size;
 	int prot = 0;
 	void *vaddr;
@@ -217,9 +239,38 @@ static void vfio_pci_bar_map(struct vfio_pci_device *device, int index)
 	align = min_t(size_t, size, SZ_1G);
 
 	vaddr = mmap_reserve(size, align, 0);
-	bar->vaddr = mmap(vaddr, size, prot, MAP_SHARED | MAP_FIXED,
-			  device->fd, bar->info.offset);
-	VFIO_ASSERT_NE(bar->vaddr, MAP_FAILED);
+
+	/*
+	 * A BAR that is only partially mmappable, such as a CXL Type-2 component
+	 * BAR with the HDM decoder block trapped, advertises the mmappable
+	 * ranges through a sparse-mmap capability. Map each area within the
+	 * reservation and leave the excluded ranges unmapped; mapping the whole
+	 * BAR would be rejected.
+	 */
+	info->argsz = sizeof(infobuf);
+	info->index = index;
+	ioctl_assert(device->fd, VFIO_DEVICE_GET_REGION_INFO, info);
+	sparse = vfio_pci_sparse_mmap_cap(info);
+	if (sparse) {
+		u32 i;
+
+		bar->vaddr = vaddr;
+		for (i = 0; i < sparse->nr_areas; i++) {
+			void *p;
+
+			if (!sparse->areas[i].size)
+				continue;
+			p = mmap(vaddr + sparse->areas[i].offset,
+				 sparse->areas[i].size, prot,
+				 MAP_SHARED | MAP_FIXED, device->fd,
+				 bar->info.offset + sparse->areas[i].offset);
+			VFIO_ASSERT_NE(p, MAP_FAILED);
+		}
+	} else {
+		bar->vaddr = mmap(vaddr, size, prot, MAP_SHARED | MAP_FIXED,
+				  device->fd, bar->info.offset);
+		VFIO_ASSERT_NE(bar->vaddr, MAP_FAILED);
+	}
 
 	madvise(bar->vaddr, size, MADV_HUGEPAGE);
 }
diff --git a/tools/testing/selftests/vfio/vfio_cxl_type2_test.c b/tools/testing/selftests/vfio/vfio_cxl_type2_test.c
new file mode 100644
index 000000000000..78468abd2201
--- /dev/null
+++ b/tools/testing/selftests/vfio/vfio_cxl_type2_test.c
@@ -0,0 +1,780 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * vfio_cxl_type2_test - corner-case tests for the vfio-cxl kernel contract.
+ *
+ * Exercises the user-visible surface the vfio-cxl module adds to a CXL Type-2
+ * device: the two VFIO regions (HDM memory and the trapped HDM decoder block),
+ * the component-register geometry capability, the dma-buf export of the HDM
+ * memory, and the guest decoder view.
+ *
+ * The host commits and locks the physical decoder before the guest sees the
+ * device, so the trapped block is served by a live read and a guest write is
+ * absorbed.
+ *
+ * Usage: ./vfio_cxl_type2_test <BDF>  (or export VFIO_SELFTESTS_BDF=<BDF>).
+ * The device must be bound to vfio-pci with the vfio-cxl module available.
+ *
+ * Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.
+ */
+
+#include <errno.h>
+#include <fcntl.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/pci_regs.h>
+#include <linux/sizes.h>
+#include <linux/vfio.h>
+
+#include <cxl/cxl_regs.h>
+
+#include <libvfio.h>
+
+#include "kselftest_harness.h"
+
+#define PCI_DVSEC_VENDOR_ID_CXL		0x1e98
+#define PCI_DVSEC_ID_CXL_DEVICE		0x0000
+
+/* CXL r3.1 8.1.9.1: Register Block Identifier for the component registers. */
+#define CXL_REGLOC_RBI_COMPONENT	1
+
+/* Register Locator DVSEC block-1 field masks. */
+#define REG_LOCATOR_BIR_MASK		0x00000007
+#define REG_LOCATOR_BLOCK_ID_MASK	0x0000ff00
+#define REG_LOCATOR_BLOCK_OFF_LOW_MASK	0xffff0000
+
+/* vfio-pci region-offset packing is kernel-internal, not UAPI; define locally. */
+#ifndef VFIO_PCI_OFFSET_SHIFT
+#define VFIO_PCI_OFFSET_SHIFT		40
+#endif
+#ifndef VFIO_PCI_INDEX_TO_OFFSET
+#define VFIO_PCI_INDEX_TO_OFFSET(i)	((uint64_t)(i) << VFIO_PCI_OFFSET_SHIFT)
+#endif
+
+static const char *device_bdf;
+
+/* Locate a region-info capability by id inside a GET_REGION_INFO buffer. */
+static const struct vfio_info_cap_header *
+find_region_cap(const void *buf, size_t bufsz, uint16_t id)
+{
+	const struct vfio_region_info *ri = buf;
+	const struct vfio_info_cap_header *cap;
+	size_t off = ri->cap_offset;
+
+	while (off && off + sizeof(*cap) <= bufsz) {
+		cap = (const void *)((const char *)buf + off);
+		if (cap->id == id)
+			return cap;
+		off = cap->next;
+	}
+	return NULL;
+}
+
+/* Find a CXL region by subtype; returns the region index or -1, @buf left holding its info. */
+static int find_cxl_region(int fd, uint32_t nregions, uint32_t subtype,
+			   void *buf, size_t bufsz)
+{
+	uint32_t i;
+
+	for (i = 0; i < nregions; i++) {
+		struct vfio_region_info *ri = buf;
+		const struct vfio_region_info_cap_type *t;
+		const struct vfio_info_cap_header *hdr;
+
+		memset(buf, 0, bufsz);
+		ri->argsz = bufsz;
+		ri->index = i;
+		if (ioctl(fd, VFIO_DEVICE_GET_REGION_INFO, ri))
+			continue;
+		if (!(ri->flags & VFIO_REGION_INFO_FLAG_CAPS))
+			continue;
+
+		hdr = find_region_cap(buf, bufsz, VFIO_REGION_INFO_CAP_TYPE);
+		if (!hdr)
+			continue;
+		t = (const void *)hdr;
+		if (t->type == (VFIO_REGION_TYPE_PCI_VENDOR_TYPE |
+				PCI_DVSEC_VENDOR_ID_CXL) &&
+		    t->subtype == subtype)
+			return i;
+	}
+	return -1;
+}
+
+/* Walk the PCI extended capability list for the CXL Device DVSEC. */
+static uint16_t find_cxl_dvsec(struct vfio_pci_device *dev)
+{
+	uint16_t pos = PCI_CFG_SPACE_SIZE;
+	int iter = 0;
+
+	while (pos && iter++ < 64) {
+		uint32_t hdr = vfio_pci_config_readl(dev, pos);
+		uint16_t cap_id = hdr & 0xffff;
+		uint16_t next = (hdr >> 20) & 0xffc;
+		uint32_t h1, h2;
+
+		if (cap_id == PCI_EXT_CAP_ID_DVSEC) {
+			h1 = vfio_pci_config_readl(dev, pos + 4);
+			h2 = vfio_pci_config_readl(dev, pos + 8);
+			if ((h1 & 0xffff) == PCI_DVSEC_VENDOR_ID_CXL &&
+			    (h2 & 0xffff) == PCI_DVSEC_ID_CXL_DEVICE)
+				return pos;
+		}
+		pos = next;
+	}
+	return 0;
+}
+
+FIXTURE(vfio_cxl) {
+	struct iommu *iommu;
+	struct vfio_pci_device *dev;
+
+	int mem_idx;
+	uint64_t mem_size;
+	uint32_t mem_flags;
+	int comp_idx;
+	uint64_t comp_size;
+	uint32_t comp_bar;
+	uint64_t comp_offset;	/* HDM block offset within comp_bar */
+	uint64_t comp_off;	/* mmap/rw base offset of the comp region */
+	uint16_t dvsec;
+};
+
+FIXTURE_SETUP(vfio_cxl)
+{
+	uint8_t infobuf[512] = {};
+	struct vfio_device_info *info = (void *)infobuf;
+	const struct vfio_region_info_cap_cxl_comp_regs *geo;
+	const struct vfio_info_cap_header *hdr;
+	uint8_t rbuf[1024];
+	uint16_t cmd;
+
+	self->iommu = iommu_init(default_iommu_mode);
+	self->dev = vfio_pci_device_init(device_bdf, self->iommu);
+
+	info->argsz = sizeof(infobuf);
+	ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_INFO, info));
+
+	if (!(info->flags & VFIO_DEVICE_FLAGS_CXL))
+		SKIP(return, "not a CXL Type-2 device");
+
+	self->mem_idx = find_cxl_region(self->dev->fd, info->num_regions,
+					VFIO_REGION_SUBTYPE_CXL_MEM,
+					rbuf, sizeof(rbuf));
+	ASSERT_GE(self->mem_idx, 0);
+	self->mem_size = ((struct vfio_region_info *)rbuf)->size;
+	self->mem_flags = ((struct vfio_region_info *)rbuf)->flags;
+
+	self->comp_idx = find_cxl_region(self->dev->fd, info->num_regions,
+					 VFIO_REGION_SUBTYPE_CXL_COMP_REGS,
+					 rbuf, sizeof(rbuf));
+	ASSERT_GE(self->comp_idx, 0);
+	self->comp_size = ((struct vfio_region_info *)rbuf)->size;
+
+	/* The geometry cap rides on the component-register region. */
+	hdr = find_region_cap(rbuf, sizeof(rbuf),
+			      VFIO_REGION_INFO_CAP_CXL_COMP_REGS);
+	ASSERT_NE(NULL, hdr);
+	geo = (const void *)hdr;
+	self->comp_bar = geo->bar;
+	self->comp_offset = geo->offset;
+
+	self->comp_off = VFIO_PCI_INDEX_TO_OFFSET(self->comp_idx);
+	self->dvsec = find_cxl_dvsec(self->dev);
+
+	/* Enable PCI Memory-Space so the HDM mmap tests can touch the mapping. */
+	cmd = vfio_pci_config_readw(self->dev, PCI_COMMAND);
+	vfio_pci_config_writew(self->dev, PCI_COMMAND,
+			       cmd | PCI_COMMAND_MEMORY);
+}
+
+FIXTURE_TEARDOWN(vfio_cxl)
+{
+	vfio_pci_device_cleanup(self->dev);
+	iommu_cleanup(self->iommu);
+}
+
+/* GET_INFO advertises the flag and both CXL regions with a sane geometry cap. */
+TEST_F(vfio_cxl, device_is_cxl)
+{
+	ASSERT_NE(self->mem_idx, self->comp_idx);
+	ASSERT_GT(self->mem_size, 0);
+	ASSERT_GT(self->comp_size, 0);
+	ASSERT_LT(self->comp_bar, PCI_STD_NUM_BARS);
+	/* The HDM memory must advertise mmap; a VMM needs it for stage-2. */
+	ASSERT_NE(0, self->mem_flags & VFIO_REGION_INFO_FLAG_MMAP);
+}
+
+/*
+ * The component BAR carries the physical HDM decoder block, which vfio traps
+ * and excludes from mmap so the guest cannot reprogram it. The whole-BAR map
+ * must fail; the ranges around the excluded block must map.
+ */
+TEST_F(vfio_cxl, comp_bar_sparse_mmap)
+{
+	size_t page_size = getpagesize();
+	uint8_t rbuf[1024] = {};
+	struct vfio_region_info *ri = (void *)rbuf;
+	const struct vfio_region_info_cap_sparse_mmap *sm;
+	const struct vfio_info_cap_header *hdr;
+	uint64_t bar_off, decoder_page;
+	void *map;
+	uint32_t i;
+
+	ri->argsz = sizeof(rbuf);
+	ri->index = self->comp_bar;
+	ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ri));
+	ASSERT_NE(0, ri->flags & VFIO_REGION_INFO_FLAG_MMAP);
+	bar_off = ri->offset;
+
+	/* The trapped decoder block splits the BAR, so it must be sparse. */
+	hdr = find_region_cap(rbuf, sizeof(rbuf),
+			      VFIO_REGION_INFO_CAP_SPARSE_MMAP);
+	ASSERT_NE(NULL, hdr);
+	sm = (const void *)hdr;
+	ASSERT_GT(sm->nr_areas, 0);
+
+	/* Mapping the whole BAR must fail: it covers the excluded block. */
+	map = mmap(NULL, ri->size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		   self->dev->fd, bar_off);
+	ASSERT_EQ(MAP_FAILED, map);
+
+	/* Every advertised area is page aligned and must map. */
+	for (i = 0; i < sm->nr_areas; i++) {
+		uint64_t ao = sm->areas[i].offset;
+		uint64_t as = sm->areas[i].size;
+
+		if (!as)
+			continue;
+		ASSERT_EQ(0, ao & (page_size - 1));
+		ASSERT_EQ(0, as & (page_size - 1));
+
+		map = mmap(NULL, as, PROT_READ | PROT_WRITE, MAP_SHARED,
+			   self->dev->fd, bar_off + ao);
+		ASSERT_NE(MAP_FAILED, map);
+		ASSERT_EQ(0, munmap(map, as));
+	}
+
+	/* The page holding the decoder block must never be mmappable. */
+	decoder_page = self->comp_offset & ~(uint64_t)(page_size - 1);
+	map = mmap(NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED,
+		   self->dev->fd, bar_off + decoder_page);
+	ASSERT_EQ(MAP_FAILED, map);
+}
+
+/* Basic HDM memory mmap read/write round-trip. */
+TEST_F(vfio_cxl, hdm_mem_mmap_rw)
+{
+	uint64_t off = VFIO_PCI_INDEX_TO_OFFSET(self->mem_idx);
+	uint32_t pattern = 0xdeadbeefU, readback = 0;
+	void *map;
+
+	if (self->mem_size < SZ_4K)
+		SKIP(return, "HDM memory < 4K");
+
+	map = mmap(NULL, SZ_4K, PROT_READ | PROT_WRITE, MAP_SHARED,
+		   self->dev->fd, off);
+	ASSERT_NE(MAP_FAILED, map);
+
+	memcpy(map, &pattern, sizeof(pattern));
+	memcpy(&readback, map, sizeof(readback));
+	ASSERT_EQ(pattern, readback);
+
+	ASSERT_EQ(0, munmap(map, SZ_4K));
+}
+
+/* A 2 MB-aligned HDM window mapped as a huge (PMD) fault. */
+TEST_F(vfio_cxl, hdm_mem_huge_mmap)
+{
+	uint64_t off = VFIO_PCI_INDEX_TO_OFFSET(self->mem_idx);
+	uint32_t pattern = 0x5a5a5a5aU, readback = 0;
+	void *map, *last;
+
+	if (self->mem_size < SZ_2M)
+		SKIP(return, "HDM memory < 2M");
+
+	map = mmap(NULL, SZ_2M, PROT_READ | PROT_WRITE, MAP_SHARED,
+		   self->dev->fd, off);
+	ASSERT_NE(MAP_FAILED, map);
+
+	last = (char *)map + SZ_2M - sizeof(pattern);
+	memcpy(last, &pattern, sizeof(pattern));
+	memcpy(&readback, last, sizeof(readback));
+	ASSERT_EQ(pattern, readback);
+
+	ASSERT_EQ(0, munmap(map, SZ_2M));
+}
+
+/*
+ * A VMM maps the HDM range into the guest IOAS by fd (the struct-page-less
+ * range cannot be pinned through a VA): export it as a dma-buf and map that fd
+ * with IOMMU_IOAS_MAP_FILE.
+ */
+TEST_F(vfio_cxl, hdm_mem_ioas_map)
+{
+	uint8_t buf[sizeof(struct vfio_device_feature) +
+		    sizeof(struct vfio_device_feature_dma_buf) +
+		    sizeof(struct vfio_region_dma_range)] = {};
+	struct vfio_device_feature *feat = (void *)buf;
+	struct vfio_device_feature_dma_buf *db = (void *)feat->data;
+	struct iova_allocator *iova_alloc;
+	int dmabuf_fd;
+	iova_t iova;
+	int ret;
+
+	if (!self->iommu->iommufd)
+		SKIP(return, "IOMMU_IOAS_MAP_FILE needs the iommufd backend");
+	if (self->mem_size < SZ_2M)
+		SKIP(return, "HDM memory < 2M");
+
+	feat->argsz = sizeof(buf);
+	feat->flags = VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_DMA_BUF;
+	db->region_index = self->mem_idx;
+	db->nr_ranges = 1;
+	db->dma_ranges[0].offset = 0;
+	db->dma_ranges[0].length = SZ_2M;
+
+	ret = ioctl(self->dev->fd, VFIO_DEVICE_FEATURE, feat);
+	if (ret < 0 && (errno == EINVAL || errno == EOPNOTSUPP))
+		SKIP(return, "kernel has no CXL dma-buf exporter");
+	ASSERT_GE(ret, 0);
+	dmabuf_fd = ret;
+
+	iova_alloc = iova_allocator_init(self->iommu);
+	iova = iova_allocator_alloc(iova_alloc, SZ_2M);
+
+	ASSERT_EQ(0, __iommu_map_file(self->iommu, dmabuf_fd, 0, SZ_2M, iova));
+	iommu_unmap_all(self->iommu);
+
+	iova_allocator_cleanup(iova_alloc);
+	ASSERT_EQ(0, close(dmabuf_fd));
+}
+
+/* The trapped block starts at the HDM decoder registers; CTRL 0 reads back. */
+TEST_F(vfio_cxl, comp_regs_hdm_read)
+{
+	uint64_t ctrl = self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0);
+	uint32_t val = 0;
+
+	ASSERT_GE(self->comp_size, CXL_HDM_DECODER0_CTRL_OFFSET(0) + 4);
+	ASSERT_EQ((ssize_t)sizeof(val),
+		  pread(self->dev->fd, &val, sizeof(val), ctrl));
+}
+
+/* The decoder registers only take aligned dword accesses. */
+TEST_F(vfio_cxl, comp_regs_reject_unaligned)
+{
+	uint32_t val = 0;
+	uint16_t half = 0;
+
+	ASSERT_EQ(-1, pread(self->dev->fd, &val, sizeof(val),
+			    self->comp_off + 1));
+	ASSERT_EQ(-1, pread(self->dev->fd, &half, sizeof(half),
+			    self->comp_off));
+}
+
+/* Accesses past the region end are rejected. */
+TEST_F(vfio_cxl, comp_regs_reject_out_of_range)
+{
+	uint32_t val = 0;
+
+	ASSERT_EQ(-1, pread(self->dev->fd, &val, sizeof(val),
+			    self->comp_off + self->comp_size));
+}
+
+/*
+ * Commit handshake: the host committed and locked the physical decoder, so the
+ * trapped block reads back COMMITTED and a guest decommit request is absorbed.
+ */
+TEST_F(vfio_cxl, hdm_commit_fsm)
+{
+	uint64_t ctrl = self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0);
+	uint32_t v, orig;
+
+	ASSERT_EQ((ssize_t)sizeof(orig),
+		  pread(self->dev->fd, &orig, sizeof(orig), ctrl));
+
+	if (!(orig & CXL_HDM_DECODER0_CTRL_COMMITTED))
+		SKIP(return, "HDM decoder 0 not committed by firmware");
+
+	/* A decommit request is absorbed; the decoder stays committed. */
+	v = orig & ~CXL_HDM_DECODER0_CTRL_COMMIT;
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pwrite(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pread(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_TRUE(v & CXL_HDM_DECODER0_CTRL_COMMITTED);
+}
+
+/*
+ * Lock on commit: the host committed the decoder with LOCK set, so a guest
+ * write is absorbed and it stays committed and locked.
+ */
+TEST_F(vfio_cxl, hdm_lock_on_commit)
+{
+	uint64_t ctrl = self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0);
+	uint32_t v;
+
+	v = CXL_HDM_DECODER0_CTRL_COMMIT | CXL_HDM_DECODER0_CTRL_LOCK;
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pwrite(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pread(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_TRUE(v & CXL_HDM_DECODER0_CTRL_COMMITTED);
+	ASSERT_TRUE(v & CXL_HDM_DECODER0_CTRL_LOCK);
+
+	/* Attempt to decommit the locked decoder; it must stay committed. */
+	v = 0;
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pwrite(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pread(self->dev->fd, &v, sizeof(v), ctrl));
+	ASSERT_TRUE(v & CXL_HDM_DECODER0_CTRL_COMMITTED);
+	ASSERT_TRUE(v & CXL_HDM_DECODER0_CTRL_LOCK);
+}
+
+/*
+ * The committed decoder holds its base read-only: a guest write to the base is
+ * absorbed and reads back the host-committed value.
+ */
+TEST_F(vfio_cxl, hdm_base_write_absorbed)
+{
+	uint64_t lo_off = self->comp_off + CXL_HDM_DECODER0_BASE_LOW_OFFSET(0);
+	uint64_t ctrl_off = self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0);
+	uint32_t v = 0x30000000U;	/* 256 MB-aligned low base bits */
+	uint32_t rb = 0, orig = 0, ctrl = 0;
+
+	ASSERT_GE(self->comp_size, CXL_HDM_DECODER0_BASE_LOW_OFFSET(0) + 4);
+	ASSERT_EQ((ssize_t)sizeof(ctrl),
+		  pread(self->dev->fd, &ctrl, sizeof(ctrl), ctrl_off));
+	if (!(ctrl & CXL_HDM_DECODER0_CTRL_COMMITTED))
+		SKIP(return, "HDM decoder 0 not committed by firmware");
+
+	ASSERT_EQ((ssize_t)sizeof(orig),
+		  pread(self->dev->fd, &orig, sizeof(orig), lo_off));
+
+	/* The write is absorbed; the base reads back unchanged. */
+	ASSERT_EQ((ssize_t)sizeof(v),
+		  pwrite(self->dev->fd, &v, sizeof(v), lo_off));
+	ASSERT_EQ((ssize_t)sizeof(rb),
+		  pread(self->dev->fd, &rb, sizeof(rb), lo_off));
+	ASSERT_EQ(orig, rb);
+}
+
+/*
+ * The CXL Device DVSEC body is virtualized by the kernel; a config read is
+ * served from the shadow.
+ */
+TEST_F(vfio_cxl, dvsec_body_read)
+{
+	uint32_t v;
+
+	if (!self->dvsec)
+		SKIP(return, "CXL Device DVSEC not found");
+
+	v = vfio_pci_config_readl(self->dev, self->dvsec + PCI_DVSEC_HEADER1);
+	ASSERT_NE(0xffffffffU, v);
+}
+
+/*
+ * Guest-initiated CXL reset: the DVSEC Initiate_CXL_Reset write self-clears and
+ * STATUS2 reports completion, synthesized in the shadow; the real reset runs at
+ * the vfio reset points.
+ */
+TEST_F(vfio_cxl, guest_cxl_reset)
+{
+	uint16_t cap, ctrl2, status2;
+
+	if (!self->dvsec)
+		SKIP(return, "CXL Device DVSEC not found");
+
+	cap = vfio_pci_config_readw(self->dev, self->dvsec + PCI_DVSEC_CXL_CAP);
+	if (!(cap & PCI_DVSEC_CXL_RST_CAPABLE))
+		SKIP(return, "device does not support CXL reset");
+
+	ctrl2 = vfio_pci_config_readw(self->dev,
+				      self->dvsec + PCI_DVSEC_CXL_CTRL2);
+	vfio_pci_config_writew(self->dev, self->dvsec + PCI_DVSEC_CXL_CTRL2,
+			       ctrl2 | PCI_DVSEC_CXL_INIT_CXL_RST);
+
+	/* Initiate_CXL_Reset self-clears once the sequence has run. */
+	ctrl2 = vfio_pci_config_readw(self->dev,
+				      self->dvsec + PCI_DVSEC_CXL_CTRL2);
+	ASSERT_FALSE(ctrl2 & PCI_DVSEC_CXL_INIT_CXL_RST);
+
+	/* STATUS2 reports the outcome; a completed reset sets RESET_COMPLETE. */
+	status2 = vfio_pci_config_readw(self->dev,
+					self->dvsec + PCI_DVSEC_CXL_STATUS2);
+	ASSERT_TRUE(status2 & PCI_DVSEC_CXL_RST_DONE);
+	ASSERT_FALSE(status2 & PCI_DVSEC_CXL_RST_ERR);
+}
+
+/*
+ * A real reset must run end to end, not just the shadow DVSEC path
+ * guest_cxl_reset covers: a CXL Type-2 device must advertise
+ * VFIO_DEVICE_FLAGS_RESET and complete a real VFIO_DEVICE_RESET.
+ */
+TEST_F(vfio_cxl, device_reset)
+{
+	uint8_t infobuf[512] = {};
+	struct vfio_device_info *info = (void *)infobuf;
+	int ret, retries = 20;
+
+	info->argsz = sizeof(infobuf);
+	ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_INFO, info));
+
+	ASSERT_NE(0, info->flags & VFIO_DEVICE_FLAGS_RESET);
+
+	/* Drive the real reset, retrying the transient device-lock contention. */
+	do {
+		ret = __vfio_pci_device_reset(self->dev);
+		if (ret == -EAGAIN)
+			usleep(10000);
+	} while (ret == -EAGAIN && retries-- > 0);
+	ASSERT_EQ(0, ret);
+}
+
+/*
+ * The component BAR is reachable by fd read except the trapped decoder block,
+ * which is served only through the comp-regs region; a VMM relies on this split.
+ */
+TEST_F(vfio_cxl, comp_bar_rdwr_split)
+{
+	uint64_t bar_off = VFIO_PCI_INDEX_TO_OFFSET(self->comp_bar);
+	uint32_t val;
+
+	/* A non-decoder dword of the BAR reads back through the fd. */
+	ASSERT_EQ((ssize_t)sizeof(val),
+		  pread(self->dev->fd, &val, sizeof(val), bar_off));
+
+	/*
+	 * The trapped decoder block is off-limits to raw BAR fd access: the read
+	 * returns all-ones, not the real decoder contents.
+	 */
+	val = 0;
+	ASSERT_EQ((ssize_t)sizeof(val),
+		  pread(self->dev->fd, &val, sizeof(val),
+			bar_off + self->comp_offset));
+	ASSERT_EQ(0xffffffffU, val);
+}
+
+/*
+ * Mirror the guest's HDM discovery, which does not use VFIO's geometry cap: it
+ * finds the range by walking the config-space DVSECs and the component-register
+ * array to the HDM decoder itself.
+ */
+TEST_F(vfio_cxl, guest_hdm_discovery)
+{
+	uint64_t bar_off = VFIO_PCI_INDEX_TO_OFFSET(self->comp_bar);
+	uint32_t reg_lo, reg_hi, cap_array, cap_count, hdr;
+	uint32_t bl, bh, sl, sh, ctrl;
+	uint64_t block_off, cm, hdm_off = 0, base, size;
+	uint16_t pos = PCI_CFG_SPACE_SIZE, regloc = 0, block1;
+	int iter = 0, bar, i;
+
+	/* 1. Find the CXL Register Locator DVSEC in config space. */
+	while (pos && iter++ < 64) {
+		uint32_t h = vfio_pci_config_readl(self->dev, pos);
+
+		if ((h & 0xffff) == PCI_EXT_CAP_ID_DVSEC) {
+			uint32_t h1 = vfio_pci_config_readl(self->dev, pos + 4);
+			uint32_t h2 = vfio_pci_config_readl(self->dev, pos + 8);
+
+			if ((h1 & 0xffff) == PCI_DVSEC_VENDOR_ID_CXL &&
+			    (h2 & 0xffff) == PCI_DVSEC_CXL_REG_LOCATOR) {
+				regloc = pos;
+				break;
+			}
+		}
+		pos = (h >> 20) & 0xffc;
+	}
+	ASSERT_NE(0, regloc);
+
+	/* 2. Take the component register block BAR and offset from block 1. */
+	block1 = regloc + PCI_DVSEC_CXL_REG_LOCATOR_BLOCK1;
+	reg_lo = vfio_pci_config_readl(self->dev, block1);
+	reg_hi = vfio_pci_config_readl(self->dev, block1 + 4);
+
+	ASSERT_EQ(CXL_REGLOC_RBI_COMPONENT,
+		  (reg_lo & REG_LOCATOR_BLOCK_ID_MASK) >> 8);
+	bar = reg_lo & REG_LOCATOR_BIR_MASK;
+	block_off = ((uint64_t)reg_hi << 32) |
+		    (reg_lo & REG_LOCATOR_BLOCK_OFF_LOW_MASK);
+
+	/* The DVSEC must name the same BAR the geometry cap reported. */
+	ASSERT_EQ(self->comp_bar, bar);
+
+	/* 3. Walk the CM capability array over the BAR to find the HDM cap. */
+	cm = block_off + CXL_CM_OFFSET;
+	ASSERT_EQ((ssize_t)sizeof(cap_array),
+		  pread(self->dev->fd, &cap_array, sizeof(cap_array),
+			bar_off + cm + CXL_CM_CAP_HDR_OFFSET));
+	ASSERT_EQ(CM_CAP_HDR_CAP_ID, cap_array & CXL_CM_CAP_HDR_ID_MASK);
+
+	cap_count = (cap_array & CXL_CM_CAP_HDR_ARRAY_SIZE_MASK) >> 24;
+	for (i = 1; i <= (int)cap_count; i++) {
+		ASSERT_EQ((ssize_t)sizeof(hdr),
+			  pread(self->dev->fd, &hdr, sizeof(hdr),
+				bar_off + cm + i * 4));
+		if ((hdr & CXL_CM_CAP_HDR_ID_MASK) == CXL_CM_CAP_CAP_ID_HDM) {
+			hdm_off = cm + ((hdr & CXL_CM_CAP_PTR_MASK) >> 20);
+			break;
+		}
+	}
+	ASSERT_NE(0, hdm_off);
+
+	/* The guest's manual walk must land on the decoder the kernel traps. */
+	ASSERT_EQ(self->comp_offset, hdm_off);
+
+	/* 4. Read decoder 0 through the trapped region and derive base/size. */
+	ASSERT_EQ((ssize_t)sizeof(bl),
+		  pread(self->dev->fd, &bl, sizeof(bl),
+			self->comp_off + CXL_HDM_DECODER0_BASE_LOW_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(bh),
+		  pread(self->dev->fd, &bh, sizeof(bh),
+			self->comp_off + CXL_HDM_DECODER0_BASE_HIGH_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(sl),
+		  pread(self->dev->fd, &sl, sizeof(sl),
+			self->comp_off + CXL_HDM_DECODER0_SIZE_LOW_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(sh),
+		  pread(self->dev->fd, &sh, sizeof(sh),
+			self->comp_off + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(ctrl),
+		  pread(self->dev->fd, &ctrl, sizeof(ctrl),
+			self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0)));
+
+	base = ((uint64_t)bh << 32) | bl;
+	size = ((uint64_t)sh << 32) | sl;
+
+	/* A guest only accepts a committed decoder; the derived range must be non-empty. */
+	if (!(ctrl & CXL_HDM_DECODER0_CTRL_COMMITTED))
+		SKIP(return, "HDM decoder 0 not committed by firmware");
+
+	ASSERT_GT(size, 0);
+	ASSERT_LT(base, base + size);
+}
+
+/*
+ * Tie the committed decoder's geometry to the HDM memory region a VMM hands the
+ * guest: confirm the mmap-able region covers exactly the decoder's range, then
+ * map the advertised base and touch it.
+ */
+TEST_F(vfio_cxl, hdm_mem_touch_committed_base)
+{
+	uint64_t mem_off = VFIO_PCI_INDEX_TO_OFFSET(self->mem_idx);
+	uint32_t pattern = 0xc0ffee11U, readback = 0;
+	uint32_t bl, bh, sl, sh, ctrl;
+	uint64_t base, size;
+	void *map;
+
+	ASSERT_EQ((ssize_t)sizeof(ctrl),
+		  pread(self->dev->fd, &ctrl, sizeof(ctrl),
+			self->comp_off + CXL_HDM_DECODER0_CTRL_OFFSET(0)));
+	if (!(ctrl & CXL_HDM_DECODER0_CTRL_COMMITTED))
+		SKIP(return, "HDM decoder 0 not committed by firmware");
+
+	ASSERT_EQ((ssize_t)sizeof(bl),
+		  pread(self->dev->fd, &bl, sizeof(bl),
+			self->comp_off + CXL_HDM_DECODER0_BASE_LOW_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(bh),
+		  pread(self->dev->fd, &bh, sizeof(bh),
+			self->comp_off + CXL_HDM_DECODER0_BASE_HIGH_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(sl),
+		  pread(self->dev->fd, &sl, sizeof(sl),
+			self->comp_off + CXL_HDM_DECODER0_SIZE_LOW_OFFSET(0)));
+	ASSERT_EQ((ssize_t)sizeof(sh),
+		  pread(self->dev->fd, &sh, sizeof(sh),
+			self->comp_off + CXL_HDM_DECODER0_SIZE_HIGH_OFFSET(0)));
+
+	base = ((uint64_t)bh << 32) | bl;
+	size = ((uint64_t)sh << 32) | sl;
+
+	ASSERT_GT(size, 0);
+	ASSERT_LT(base, base + size);
+	/* The mmap-able HDM region must cover exactly the committed decoder. */
+	ASSERT_EQ(size, self->mem_size);
+
+	if (self->mem_size < SZ_4K)
+		SKIP(return, "HDM memory < 4K");
+
+	/* Region offset 0 is the decoder's advertised base; map it and touch it. */
+	map = mmap(NULL, SZ_4K, PROT_READ | PROT_WRITE, MAP_SHARED,
+		   self->dev->fd, mem_off);
+	ASSERT_NE(MAP_FAILED, map);
+
+	memcpy(map, &pattern, sizeof(pattern));
+	memcpy(&readback, map, sizeof(readback));
+	ASSERT_EQ(pattern, readback);
+
+	ASSERT_EQ(0, munmap(map, SZ_4K));
+}
+
+/*
+ * The CXL Device DVSEC is served from a shadow: Control is guest-programmable
+ * but Capability keeps its firmware snapshot, so a guest cannot reprogram the
+ * device through the DVSEC.
+ */
+TEST_F(vfio_cxl, dvsec_write_virtualized)
+{
+	uint16_t cap, ctrl, v;
+
+	if (!self->dvsec)
+		SKIP(return, "CXL Device DVSEC not found");
+
+	/* A write to the read-only Capability register is absorbed. */
+	cap = vfio_pci_config_readw(self->dev, self->dvsec + PCI_DVSEC_CXL_CAP);
+	vfio_pci_config_writew(self->dev, self->dvsec + PCI_DVSEC_CXL_CAP,
+			       cap ^ 0xffff);
+	v = vfio_pci_config_readw(self->dev, self->dvsec + PCI_DVSEC_CXL_CAP);
+	ASSERT_EQ(cap, v);
+
+	/* Control is guest-programmable; a write reads back from the shadow. */
+	ctrl = vfio_pci_config_readw(self->dev, self->dvsec + PCI_DVSEC_CXL_CTRL);
+	vfio_pci_config_writew(self->dev, self->dvsec + PCI_DVSEC_CXL_CTRL,
+			       ctrl ^ 0x1);
+	v = vfio_pci_config_readw(self->dev, self->dvsec + PCI_DVSEC_CXL_CTRL);
+	ASSERT_EQ((uint16_t)(ctrl ^ 0x1), v);
+	vfio_pci_config_writew(self->dev, self->dvsec + PCI_DVSEC_CXL_CTRL, ctrl);
+}
+
+/*
+ * Region-info contract: the HDM memory region is readable, writable and
+ * mmappable; the trapped comp-regs region is fd-only and carries the geometry
+ * capability.
+ */
+TEST_F(vfio_cxl, region_flags)
+{
+	const uint32_t rw = VFIO_REGION_INFO_FLAG_READ |
+			    VFIO_REGION_INFO_FLAG_WRITE;
+	const struct vfio_info_cap_header *geo;
+	uint8_t rbuf[1024] = {};
+	struct vfio_region_info *ri = (void *)rbuf;
+
+	ri->argsz = sizeof(rbuf);
+	ri->index = self->mem_idx;
+	ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ri));
+	ASSERT_EQ(rw | VFIO_REGION_INFO_FLAG_MMAP,
+		  ri->flags & (rw | VFIO_REGION_INFO_FLAG_MMAP));
+
+	memset(rbuf, 0, sizeof(rbuf));
+	ri->argsz = sizeof(rbuf);
+	ri->index = self->comp_idx;
+	ASSERT_EQ(0, ioctl(self->dev->fd, VFIO_DEVICE_GET_REGION_INFO, ri));
+	ASSERT_EQ(rw, ri->flags & rw);
+	ASSERT_EQ(0, ri->flags & VFIO_REGION_INFO_FLAG_MMAP);
+	geo = find_region_cap(rbuf, sizeof(rbuf),
+			      VFIO_REGION_INFO_CAP_CXL_COMP_REGS);
+	ASSERT_NE(NULL, geo);
+}
+
+int main(int argc, char *argv[])
+{
+	device_bdf = vfio_selftests_get_bdf(&argc, argv);
+	return test_harness_run(argc, argv);
+}
-- 
2.25.1

Re: [PATCH v5 26/27] Documentation: vfio-pci: Document CXL Type-2 device passthrough

[Gregory Price]

On Thu, Sep 17, 2026 at 12:05:39AM +0530, mhonap@nvidia.com wrote:
> From: Manish Honap <mhonap@nvidia.com>
> 

1) Thank you so much for writing documentation, i truly appreciate this.

2) I apologize in advance for my terseness, I know writing is hard,
   please do not interpret this as disliking your writing or series.

> +Address model
> +=============
> +
> +The HDM memory is a coherent host physical range (HPA). The host kernel
> +resolves that range before the guest sees the device, and owns it for the
> +bind lifetime. The guest only chooses where the memory appears in its own
> +physical address space (GPA), by programming a virtual endpoint HDM
> +decoder. The guest never reprograms the physical decoder.
> +
> +The kernel holds the HPA and does not see the GPA. The guest programs a
> +GPA and does not see the HPA. The VMM holds the device fd, reads the
> +committed base from the decoder-register region described below, and maps
> +the HPA-backed HDM region at the GPA the guest committed. The base the
> +guest reads back is the GPA, not the HPA.
> +

I think this must be slightly inaccurate / imprecise wording.

The host *must* provide some form of physical memory window to the guest
at initialization time, otherwise the guest has no way to know - at boot
time - that there's even a window of memory it can use.

That's what the CFMWS is.  This is initialized by the hypervisor - which
is controlled by the host.

So the host (at least the VMM) must know, for the region the entire
device *could* inhabit, what that GPA is - because it's the one that makes
the CFMWS for the guest.

If this is not the case, then something is missing from this
documentation to explain why.


If you're actually trying to say is that the GPA's programmed into the
virtual decoders are largely symbolic - this at best feels a bit
inaccurate and simply an implementation detail.

The host's virtio device could enforce ....:

Host Range:
   CFMWS HPA - [0x10000, 0x20000]
                   |         |
Guest Range:       |         |
   CFMWS GPA - [0x50000, 0x60000]

In that case, you'd get the following translation...
   vdecoder0.0 - [0x58000, 0x60000]
   CFMWS GPA   - [0x58000, 0x60000]
   CFMWS HPA   - [0x18000, 0x20000]

Or the virtio device could not enforce that and let the host page-fault
just hand it a random page from the actual CXL device.

   vdecoder0.0 - [0x58000, 0x60000]
   CFMWS GPA   - [0x58000, 0x60000]
                         |
              No discrete host mapping


The former makes sense if the device (accelerator) requires exact
physical placement to do its accelerator nonsense.

The latter makes sense if the device (accelerator) doesn't care
about placement (compressed memory).

This is not saying we need support both out of the box, but we shouldn't
lock ourselves into the former unless there's some reason why the latter
is not reasonable.

Can you please help document what the actual expected behavior is with
examples in the Address model section so it's easier to understand the
intent?  That will help quite a bit.

> +Guest decoder and commit
> +========================
> +
> +The guest programs its virtual endpoint decoder through the trapped
> +region: it writes a base (a GPA), a size, and then the COMMIT bit. The
> +host already resolved and committed the physical placement before the
> +guest ran,

So the host does know GPA, just not exact placement.

> so a live read of the decoder always shows COMMITTED and the
> +guest's commit poll completes. The physical decoder is never rewritten;
> +the guest's writes are absorbed.
> +

Rather clunky, round-about way to say "The guest decoders are
virtualized".   If possible, it would be nice to formalize this concept
into "Virtual Decoders" - since that's what this is.

With that concept i think you can probably generate some nice diagrams
that show how the guest vdecoder's interact with the host drivers.

> +The VMM observes the commit, reads the committed base, and maps the HDM
> +region at that GPA.
> +

So the host does know the GPA.

> +CXL Device DVSEC
> +================
> +
> +The kernel virtualizes the CXL Device DVSEC body through the config-space
> +permission hooks. Reads and writes inside the DVSEC body use a per-open
> +shadow; a guest write stays in the shadow and does not reach hardware.
> +Accesses outside the DVSEC body go to the device as usual.
> +

"The kernel" - what part? vfio-pci ? the vmm ?


> +DMA and iommufd
> +===============
> +
> +A Type-2 accelerator issues ATS-translated DMA to addresses inside its own
> +HDM window, so that range must be present in the guest IOAS that backs the
> +nested stage-2 translation.

Type-2, ATS, DMA, HDM window, guest IOAS, stage-2 translation

I think the only thing i don't know in this sentence is "guest IOAS" and
it's still hurting my brain to read.

Are all accelerators expect to have this particular interaction, or just
yours?

> The HDM range is struct-page-less coherent
> +memory, which a userspace-VA ``IOMMU_IOAS_MAP`` cannot pin.
> +

The hardest part about writing about virtualization is keeping a
consistent mental model from section to section.

which userspace? guest? host?  (i presume guest here)

`struct-page-less coherent memory`
   e.g. the host never hotplugs this, it hands the entire region
   directly to the VFIO device, right?

   I think this would be nice to spell out somewhere.

> +The HDM memory region is therefore exportable as a dma-buf:
> +``VFIO_DEVICE_FEATURE_DMA_BUF`` on that region returns an fd that iommufd
> +maps with ``IOMMU_IOAS_MAP_FILE``, mapping the physical range without a VA
> +or a page pin. The dma-buf is revoked whenever the mapping is torn down
> +(reset, power transition, teardown), so a stale stage-2 mapping cannot
> +outlive the HDM window.
> +

For the sake of readers, I think either a little bit more information
on this "stage-2 mapping" concept is needed to make sense of what's
going on here and why it mustn't outlive the HDM window.

> +Reset
> +=====
> +
> +A CXL Type-2 function must not take a Function Level Reset: an FLR resets
> +the coherent CXL.mem state and the HDM decoder. The PCI core reflects this
> +by preferring the CXL reset over FLR, so a function reset of a CXL device
> +runs the CXL DVSEC reset sequence, which resets the function and then
> +restores the HDM decoder and the PCI config state.
> +

I think what you're trying to say is that FLRs are never passed to the
device because it can cause physical device effects that defeat the
purpose of the virtualization, yes?

So we virtualize FLRs...

> +A guest requests a reset by writing Initiate CXL Reset in the DVSEC. That
> +write only stamps completion in the shadow. The real reset runs at the vfio
> +reset points (the reset ioctl and a virtualized FLR through config space):

As you describe here.

So it's not that an accelerator "must not take an FLR" - it's that FLRs
are virtualized to prevent deleterious effects on the host / hardware.

Am I misunderstanding this?

> +the kernel zaps the HDM mapping and revokes the dma-buf, then runs the CXL
> +reset, which always clears the device memory, and restores and re-samples
> +the decoder afterwards. A CXL port masks Secondary Bus Reset by default, so a
> +``VFIO_DEVICE_PCI_HOT_RESET`` does not reach the endpoint and the HDM
> +state is untouched. If the port has SBR unmasked the reset can decommit
> +the decoder without restoring it, so the reset_done handler gates HDM
> +access; a ``VFIO_DEVICE_RESET`` then runs the CXL reset sequence and
> +restores it.
> +
> +The decoder register region is served by live reads of the hardware
> +decoder with guest writes absorbed: the decoder is committed and locked by
> +the host, so a guest can neither decommit nor reprogram it, and the kernel
> +keeps no shadow of the decoder state.

This is basically what I said at the beginning - it must either be that
the host provides locked auto-decoders at boot, or it must provide
proper virtualization so that the decoders settings are fully
virtualized.

Seems it's the former, and that makes sense.  Please correct me if i'm
misunderstanding.

> After a reset the kernel restores and
> +re-samples the firmware-committed decoder, so the geometry the guest reads
> +back is unchanged. A VMM that dropped its HDM mapping, for example across a
> +reset or a D3hot->D0 transition, must rescan the decoder and rebuild its
> +stage-2 mapping before it resumes HDM access.
> +

Yeah i think we need a bit more information about this stage-2 mapping
rebuild to make sense of this.  Maybe I'm just not read-up enough on
this particular setup - is there another part of the docs you can link
to that talk about this, or are you able to share some details as to
what this rebuild process looks like?

~Gregory

Re: [PATCH v5 21/27] vfio/cxl: Exclude the HDM decoder registers from direct BAR access

[Richard Cheng]

On Thu, Sep 17, 2026 at 12:05:34AM +0800, mhonap@nvidia.com wrote:
> From: Manish Honap <mhonap@nvidia.com>
> 
> The HDM decoder register block lives in the component BAR that vfio-pci
> owns. The guest reads the decoder only through the trapped, read-only
> decoder region; it must not reach the physical registers through the BAR
> directly, and a host read of the range through a kernel mapping could
> abort as a fatal host SError.
> 
> Add the block to the generic excluded-range list with
> VFIO_PCI_EXCLUDE_MMAP | VFIO_PCI_EXCLUDE_READ | VFIO_PCI_EXCLUDE_WRITE
> similar to the MSI-X exclusion.
>

Hi Manish,

I think we need to block eventfd write and BAR DMA as well ?

Have some comments below.

 
> Assisted-by: LLM
> Signed-off-by: Manish Honap <mhonap@nvidia.com>
> ---
>  drivers/vfio/pci/cxl/vfio_cxl_core.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> index da04776356e4..37e8a3b54cfb 100644
> --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
> +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> @@ -435,8 +435,25 @@ static int vfio_cxl_add_region(struct vfio_pci_core_device *vdev, u32 subtype,
>  static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
>  {
>  	struct vfio_cxl_state *cxl = vdev->cxl;
> +	struct pci_dev *pdev = vdev->pdev;
>  	int ret;
>  
> +	/*
> +	 * Keep the HDM decoder block out of the guest's direct BAR access: the
> +	 * guest reaches it only through the trapped decoder region, and a host
> +	 * read of the range through a kernel mapping could abort as an SError.
> +	 * Exclude it from mmap, fill reads with -1 and drop writes. The list is
> +	 * cleared on close, so re-add it per open.
> +	 */
> +	ret = vfio_pci_core_add_excluded_range(vdev, pdev->hdm->hdm_bar,
> +					       pdev->hdm->hdm_offset,
> +					       cxl->hdm_len,
> +					       VFIO_PCI_EXCLUDE_MMAP |
> +					       VFIO_PCI_EXCLUDE_READ |
> +					       VFIO_PCI_EXCLUDE_WRITE);

This adds the HDM reg to the exclusion list, but the eventfd and BAR DMA path doesn't
check that.

Maybe we need to block those 2 path as well ?

Best regards,
Richard Cheng.


> +	if (ret)
> +		return ret;
> +
>  	/*
>  	 * vfio_pci_core_disable() frees all dynamic regions on close, so register
>  	 * them here per open rather than at bind. A failed first open never
> -- 
> 2.25.1
> 
> 

Re: [PATCH v5 24/27] vfio/cxl: Export the HDM memory region as a dma-buf

[Richard Cheng]

On Thu, Sep 17, 2026 at 12:05:37AM +0800, mhonap@nvidia.com wrote:
> From: Manish Honap <mhonap@nvidia.com>
> 
> A Type-2 accelerator issues ATS-translated DMA to addresses inside its
> own HDM window, so that coherent host range must be present in the
> guest's IOAS (the iommufd IOAS backing the nested SMMU stage-2). iommufd
> maps a struct-page-less range only by fd, via IOMMU_IOAS_MAP_FILE over a
> dma-buf; a userspace-VA IOMMU_IOAS_MAP of the HDM mmap is rejected
> because the VMA is VM_IO | VM_PFNMAP. Without a dma-buf the range could
> only be mapped through an out-of-tree PFNMAP work-around.
> 
> vfio-pci already exports BAR memory as a P2P dma-buf, but the exporter
> is BAR-only: vfio_pci_core_feature_dma_buf() rejects any region index at
> or above the ROM index, and vfio_pci_core_get_dmabuf_phys() resolves the
> physical range from a PCI BAR. The HDM memory region is a dynamic
> device-specific region, not a BAR.
> 
> Let a device-specific region reach the device's get_dmabuf_phys(): a
> region index at or above VFIO_PCI_NUM_REGIONS skips the BAR-resource
> check and is validated by the driver instead, bounded to the regions
> that exist. Install a CXL-aware get_dmabuf_phys() in the vfio-cxl
> provider that returns cxl->hpa_range for the HDM memory region and
> delegates real BARs to the core, keeping the BAR path unchanged and the
> core free of CXL knowledge.
> 
> The HDM window is coherent host memory with no p2pdma provider of its
> own, so borrow BAR 0's, matching nvgrace-gpu's handling of its non-BAR
> device memory. The iommufd importer does not consume the provider; the
> scatterlist map path (real peer DMA) is left to a follow-up once
> upstream grows a negotiated interconnect for coherent CXL memory.
> 
> Assisted-by: LLM
> Signed-off-by: Manish Honap <mhonap@nvidia.com>
> ---
>  drivers/vfio/pci/cxl/vfio_cxl_core.c | 60 ++++++++++++++++++++++++++++
>  drivers/vfio/pci/vfio_pci_dmabuf.c   | 27 +++++++++++--
>  2 files changed, 83 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> index 5fe8e35c63c4..55fa1f86850d 100644
> --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
> +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> @@ -11,6 +11,7 @@
>  #include <linux/mm.h>
>  #include <linux/module.h>
>  #include <linux/pci.h>
> +#include <linux/pci-p2pdma.h>
>  #include <linux/range.h>
>  #include <linux/slab.h>
>  #include <linux/uaccess.h>
> @@ -27,6 +28,7 @@
>   * @hdm_regs: mapped HDM decoder registers, read live by the decoder region
>   * @hdm_len: length of the HDM decoder register block
>   * @hdm_valid: true when host CPU access to the HDM range is safe; under memory_lock
> + * @mem_region_index: vfio region index of the mmap-able HDM memory region
>   */
>  struct vfio_cxl_state {
>  	struct cxl_dev_state cxlds;
> @@ -36,6 +38,7 @@ struct vfio_cxl_state {
>  	void __iomem *hdm_regs;
>  	u32 hdm_len;
>  	bool hdm_valid;
> +	unsigned int mem_region_index;
>  };
>  
>  static unsigned long vfio_cxl_mem_pgoff(struct vm_area_struct *vma,
> @@ -289,6 +292,50 @@ static void vfio_cxl_release_hpa(void *data)
>  	release_mem_region(cxl->hpa_range.start, range_len(&cxl->hpa_range));
>  }
>  
> +/*
> + * Resolve the physical range that backs a dma-buf export. The core exporter
> + * only knows BARs; teach it the HDM memory region so a guest IOAS can map the
> + * coherent window by fd (IOMMU_IOAS_MAP_FILE) instead of the removed PFNMAP
> + * work-around. Real BARs stay on the byte-identical core path.
> + */
> +static int vfio_cxl_get_dmabuf_phys(struct vfio_pci_core_device *vdev,
> +				    struct p2pdma_provider **provider,
> +				    unsigned int region_index,
> +				    struct phys_vec *phys_vec,
> +				    struct vfio_region_dma_range *dma_ranges,
> +				    size_t nr_ranges)
> +{
> +	struct vfio_cxl_state *cxl = vdev->cxl;
> +
> +	/* Real BARs go through the core P2P exporter unchanged. */
> +	if (region_index < VFIO_PCI_NUM_REGIONS)
> +		return vfio_pci_core_get_dmabuf_phys(vdev, provider,
> +						     region_index, phys_vec,
> +						     dma_ranges, nr_ranges);
> +
> +	/* Of the device regions, only the HDM memory window is exportable. */
> +	if (region_index != cxl->mem_region_index)
> +		return -EINVAL;
> +
> +	/*
> +	 * The HDM window is coherent host memory, not BAR MMIO, so it has no
> +	 * p2pdma provider of its own. Borrow BAR 0's: the P2P properties match
> +	 * and the iommufd importer does not consume the provider. The sgt map
> +	 * path (real peer DMA) is not supported for the HDM window.
> +	 */
> +	*provider = pcim_p2pdma_provider(vdev->pdev, 0);

I think of a weird scenario which might make peer DMA work, not sure if that's possible.

userspace can give this dma-buf fd to an NIC driver or so to do RDMA ? use HDM memory as
network buffer ?

If that's the case and direct P2P addressing is selected, it applies BAR 0's bus offset to
HDM physical address.

Maybe explicitly reject peer-DMA mapping would be safer ?

Best regards,
Richard Cheng.


> +	if (!*provider)
> +		return -EINVAL;
> +
> +	return vfio_pci_core_fill_phys_vec(phys_vec, dma_ranges, nr_ranges,
> +					  cxl->hpa_range.start,
> +					  range_len(&cxl->hpa_range));
> +}
> +
> +static const struct vfio_pci_device_ops vfio_cxl_pci_dev_ops = {
> +	.get_dmabuf_phys = vfio_cxl_get_dmabuf_phys,
> +};
> +
>  static int vfio_cxl_init_device(struct vfio_pci_core_device *vdev)
>  {
>  	struct pci_dev *pdev = vdev->pdev;
> @@ -483,6 +530,19 @@ static int vfio_cxl_open_device(struct vfio_pci_core_device *vdev)
>  	if (ret)
>  		return ret;
>  
> +	/* Record where the HDM memory region landed for the dma-buf export. */
> +	cxl->mem_region_index = VFIO_PCI_NUM_REGIONS + vdev->num_regions - 1;
> +
> +	/*
> +	 * Override the device ops so a dma-buf export of the HDM memory region
> +	 * resolves to the coherent host range. This is done at open, not init:
> +	 * vfio_pci_probe() resets pci_ops after vfio_alloc_device() returns, so
> +	 * an override installed during init would be clobbered. Only a CXL device
> +	 * reaches this hook (cxl_ops is set on init success), so a fallback to
> +	 * plain vfio-pci keeps the core ops.
> +	 */
> +	vdev->pci_ops = &vfio_cxl_pci_dev_ops;
> +
>  	ret = vfio_cxl_add_region(vdev, VFIO_REGION_SUBTYPE_CXL_COMP_REGS,
>  				  &vfio_cxl_comp_regops, cxl->hdm_len,
>  				  VFIO_REGION_INFO_FLAG_READ |
> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
> index c16f460c01d6..436c616d5b66 100644
> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c
> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
> @@ -178,6 +178,15 @@ int vfio_pci_core_get_dmabuf_phys(struct vfio_pci_core_device *vdev,
>  {
>  	struct pci_dev *pdev = vdev->pdev;
>  
> +	/*
> +	 * This resolver only handles PCI BARs. A device-specific region index
> +	 * (>= PCI_STD_NUM_BARS) would index pdev->resource[] out of bounds via
> +	 * pcim_p2pdma_provider(), so reject it; a driver that exports such a
> +	 * region installs its own get_dmabuf_phys.
> +	 */
> +	if (region_index >= PCI_STD_NUM_BARS)
> +		return -EINVAL;
> +
>  	*provider = pcim_p2pdma_provider(pdev, region_index);
>  	if (!*provider)
>  		return -EINVAL;
> @@ -227,6 +236,7 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
>  	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
>  	struct vfio_pci_dma_buf *priv;
>  	size_t length;
> +	u32 index;
>  	int ret;
>  
>  	if (!vdev->pci_ops || !vdev->pci_ops->get_dmabuf_phys)
> @@ -243,13 +253,22 @@ int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags,
>  	if (!get_dma_buf.nr_ranges || get_dma_buf.flags)
>  		return -EINVAL;
>  
> +	index = get_dma_buf.region_index;
> +
>  	/*
> -	 * For PCI the region_index is the BAR number like everything
> -	 * else.  Check that PCI resources have been claimed for it.
> +	 * A fixed region index is the BAR number; only a BAR can be exported
> +	 * and its PCI resource must be claimed. A device-specific region (index
> +	 * >= VFIO_PCI_NUM_REGIONS) has no BAR resource and is validated by the
> +	 * device's get_dmabuf_phys instead, but the index must name a region
> +	 * that exists.
>  	 */
> -	if (get_dma_buf.region_index >= VFIO_PCI_ROM_REGION_INDEX ||
> -	    IS_ERR(vfio_pci_core_get_iomap(vdev, get_dma_buf.region_index)))
> +	if (index < VFIO_PCI_NUM_REGIONS) {
> +		if (index >= VFIO_PCI_ROM_REGION_INDEX ||
> +		    IS_ERR(vfio_pci_core_get_iomap(vdev, index)))
> +			return -ENODEV;
> +	} else if (index - VFIO_PCI_NUM_REGIONS >= vdev->num_regions) {
>  		return -ENODEV;
> +	}
>  
>  	dma_ranges = memdup_array_user(&arg->dma_ranges, get_dma_buf.nr_ranges,
>  				       sizeof(*dma_ranges));
> -- 
> 2.25.1
> 
> 

Re: [PATCH v5 25/27] vfio/cxl: Run the CXL reset at the vfio reset points

[Richard Cheng]

On Thu, Sep 17, 2026 at 12:05:38AM +0800, mhonap@nvidia.com wrote:
> From: Manish Honap <mhonap@nvidia.com>
> 
> A CXL Type-2 function must not take an FLR: it resets the coherent
> CXL.mem state and corrupts the HDM decoder. The PCI core already reflects
> this, ordering cxl_reset ahead of flr in pci_reset_fn_methods[], so a
> function reset of a CXL device runs the DVSEC reset sequence rather than
> FLR.
> 
> Route the vfio function-reset points (VFIO_DEVICE_RESET and the
> virtualized PCIe/AF FLR writes) through a CXL reset op that runs
> cxl_reset_dvsec_sequence(). The sequence resets the function, always
> clearing device memory, and restores the HDM decoder and PCI config
> state, so it is a complete replacement for pci_try_reset_function() on a
> CXL device. The op runs under memory_lock and not the PCI device lock, so
> cxl_reset_dvsec_sequence() can take the device lock itself.
> 
> Clear hdm_valid for the duration of the reset so a fault cannot insert a
> PFN into a decoder that is being torn down, and restore it once the
> sequence has put the decoder back.
> 
> Assisted-by: LLM
> Signed-off-by: Manish Honap <mhonap@nvidia.com>
> ---
>  drivers/vfio/pci/cxl/vfio_cxl_core.c | 41 +++++++++++++++
>  drivers/vfio/pci/vfio_pci_config.c   | 49 +++++++++++++++---
>  drivers/vfio/pci/vfio_pci_core.c     | 77 +++++++++++++++++++++++-----
>  drivers/vfio/pci/vfio_pci_priv.h     |  1 +
>  include/linux/vfio_pci_core.h        |  4 ++
>  5 files changed, 152 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/vfio/pci/cxl/vfio_cxl_core.c b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> index 55fa1f86850d..795362aea344 100644
> --- a/drivers/vfio/pci/cxl/vfio_cxl_core.c
> +++ b/drivers/vfio/pci/cxl/vfio_cxl_core.c
> @@ -632,6 +632,45 @@ static void vfio_cxl_reset_done(struct vfio_pci_core_device *vdev)
>  		cxl->hdm_valid = false;
>  }
>  
> +/*
> + * Run the CXL DVSEC reset sequence in place of a PCI function reset. A CXL
> + * Type-2 function must not take an FLR (it would corrupt CXL.mem), so the vfio
> + * reset points route here. The sequence resets the function, always clearing
> + * device memory, and restores the HDM decoder. The caller holds memory_lock,
> + * and this path does not hold the PCI device lock, so cxl_reset_dvsec_sequence()
> + * can take it.
> + */
> +static int vfio_cxl_reset(struct vfio_pci_core_device *vdev)
> +{
> +	struct vfio_cxl_state *cxl = vdev->cxl;
> +	int ret;
> +
> +	lockdep_assert_held_write(&vdev->memory_lock);
> +
> +	/* Host CPU access to the HDM range is unsafe until the decoder is back. */
> +	cxl->hdm_valid = false;
> +
> +	ret = cxl_reset_dvsec_sequence(vdev->pdev);
> +	if (!ret)
> +		cxl->hdm_valid = true;
> +
> +	return ret;
> +}
> +
> +/*
> + * The HDM dma-buf may be armed only while the decoder is valid. After a failed
> + * reset hdm_valid is clear, so the generic memory-enable re-arm must skip the
> + * dma-buf rather than map DMA onto an unrestored decoder.
> + */
> +static bool vfio_cxl_hdm_active(struct vfio_pci_core_device *vdev)
> +{
> +	struct vfio_cxl_state *cxl = vdev->cxl;
> +
> +	lockdep_assert_held_write(&vdev->memory_lock);
> +
> +	return cxl->hdm_valid;
> +}
> +
>  static const struct vfio_cxl_ops vfio_cxl_ops = {
>  	.init		= vfio_cxl_init_device,
>  	.release	= vfio_cxl_release_device,
> @@ -639,6 +678,8 @@ static const struct vfio_cxl_ops vfio_cxl_ops = {
>  	.close_device	= vfio_cxl_close_device,
>  	.reset_prepare	= vfio_cxl_reset_prepare,
>  	.reset_done	= vfio_cxl_reset_done,
> +	.reset		= vfio_cxl_reset,
> +	.hdm_active	= vfio_cxl_hdm_active,
>  	.owner		= THIS_MODULE,
>  };
>  
> diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c
> index 9a020a768055..8a5a737efa31 100644
> --- a/drivers/vfio/pci/vfio_pci_config.c
> +++ b/drivers/vfio/pci/vfio_pci_config.c
> @@ -630,7 +630,14 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos,
>  		*virt_cmd &= cpu_to_le16(~mask);
>  		*virt_cmd |= cpu_to_le16(new_cmd & mask);
>  
> -		if (__vfio_pci_memory_enabled(vdev))
> +		/*
> +		 * Re-arm the dma-bufs on memory-enable, but keep a CXL device's
> +		 * HDM dma-buf revoked while the decoder is unrestored (a failed
> +		 * reset leaves hdm_valid clear); re-arming would map DMA onto a
> +		 * decoder the fault path still gates. Plain vfio-pci is unchanged.
> +		 */
> +		if (__vfio_pci_memory_enabled(vdev) &&
> +		    (!vdev->cxl_ops || vdev->cxl_ops->hdm_active(vdev)))
>  			vfio_pci_dma_buf_move(vdev, false);
>  		up_write(&vdev->memory_lock);
>  	}
> @@ -720,7 +727,8 @@ static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev,
>  	}
>  
>  	vfio_pci_set_power_state(vdev, state);
> -	if (__vfio_pci_memory_enabled(vdev))
> +	if (__vfio_pci_memory_enabled(vdev) &&
> +	    (!vdev->cxl_ops || vdev->cxl_ops->hdm_active(vdev)))
>  		vfio_pci_dma_buf_move(vdev, false);
>  	up_write(&vdev->memory_lock);
>  }
> @@ -910,8 +918,14 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos,
>  		if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) {
>  			vfio_pci_zap_and_down_write_memory_lock(vdev);
>  			vfio_pci_dma_buf_move(vdev, true);
> -			pci_try_reset_function(vdev->pdev);
> -			if (__vfio_pci_memory_enabled(vdev))
> +			ret = vfio_pci_reset_function(vdev);
> +			/*
> +			 * Keep the HDM dma-buf revoked if a CXL reset
> +			 * failed; re-arming would map DMA onto an
> +			 * unrestored decoder. Mirrors the reset ioctl.
> +			 */
> +			if (__vfio_pci_memory_enabled(vdev) &&
> +			    (!vdev->cxl_ops || !ret))
>  				vfio_pci_dma_buf_move(vdev, false);
>  			up_write(&vdev->memory_lock);
>  		}
> @@ -995,8 +1009,14 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos,
>  		if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) {
>  			vfio_pci_zap_and_down_write_memory_lock(vdev);
>  			vfio_pci_dma_buf_move(vdev, true);
> -			pci_try_reset_function(vdev->pdev);
> -			if (__vfio_pci_memory_enabled(vdev))
> +			ret = vfio_pci_reset_function(vdev);
> +			/*
> +			 * Keep the HDM dma-buf revoked if a CXL reset
> +			 * failed; re-arming would map DMA onto an
> +			 * unrestored decoder. Mirrors the reset ioctl.
> +			 */
> +			if (__vfio_pci_memory_enabled(vdev) &&
> +			    (!vdev->cxl_ops || !ret))
>  				vfio_pci_dma_buf_move(vdev, false);
>  			up_write(&vdev->memory_lock);
>  		}
> @@ -1781,9 +1801,22 @@ static int vfio_cxl_dvsec_write(struct vfio_pci_core_device *vdev, int pos,
>  		status2 |= PCI_DVSEC_CXL_CACHE_INV;
>  	}
>  	if (ctrl2 & PCI_DVSEC_CXL_INIT_CXL_RST) {
> +		int ret = 0;
> +
>  		ctrl2 &= ~PCI_DVSEC_CXL_INIT_CXL_RST;
> -		status2 &= ~PCI_DVSEC_CXL_RST_ERR;
> -		status2 |= PCI_DVSEC_CXL_RST_DONE;
> +
> +		if (vdev->cxl_ops && vdev->cxl_ops->reset) {
> +			vfio_pci_zap_and_down_write_memory_lock(vdev);
> +			vfio_pci_dma_buf_move(vdev, true);
> +			ret = vfio_pci_reset_function(vdev);
> +			if (__vfio_pci_memory_enabled(vdev) &&
> +			    (!vdev->cxl_ops || !ret))
> +				vfio_pci_dma_buf_move(vdev, false);
> +			up_write(&vdev->memory_lock);
> +		}
> +
> +		status2 &= ~(PCI_DVSEC_CXL_RST_DONE | PCI_DVSEC_CXL_RST_ERR);
> +		status2 |= ret ? PCI_DVSEC_CXL_RST_ERR : PCI_DVSEC_CXL_RST_DONE;
>  	}
>  
>  	*pctrl2 = cpu_to_le16(ctrl2);
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index f02a5240aa71..8bd4db7afefe 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -643,8 +643,27 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
>  		goto out_power;
>  
>  	/* If reset fails because of the device lock, fail this path entirely */
> -	ret = pci_try_reset_function(pdev);
> -	if (ret == -EAGAIN)
> +	if (vdev->cxl_ops && vdev->cxl_ops->reset) {
> +		/*
> +		 * VM power-on resets a CXL Type-2 device through its DVSEC
> +		 * sequence. vconfig is not built yet here, so take memory_lock
> +		 * and call the op directly rather than the wrapper.
> +		 */
> +		down_write(&vdev->memory_lock);
> +		ret = vdev->cxl_ops->reset(vdev);

CXL reset restores PCI config from pdev's saved state. Where is that state refreshed before this call ?

Best regards,
Richard Cheng.


> +		up_write(&vdev->memory_lock);
> +	} else {
> +		ret = pci_try_reset_function(pdev);
> +	}
> +
> +	/*
> +	 * -EAGAIN means the reset could not run. For a CXL device any reset
> +	 * error must also fail the open: a failed DVSEC reset can leave the HDM
> +	 * decoder cleared or unrestored, and continuing would expose the HDM
> +	 * region for host access through a decoder in an unknown state.
> +	 */
> +	if (ret == -EAGAIN ||
> +	    (vdev->cxl_ops && vdev->cxl_ops->reset && ret))
>  		goto out_disable_device;
>  
>  	vdev->reset_works = !ret;
> @@ -845,16 +864,30 @@ void vfio_pci_core_disable(struct vfio_pci_core_device *vdev)
>  	 * overwrite the previously restored configuration information.
>  	 */
>  	if (vdev->reset_works) {
> -		bridge = pci_upstream_bridge(pdev);
> -		if (bridge && !pci_dev_trylock(bridge))
> -			goto out_restore_state;
> -		if (pci_dev_trylock(pdev)) {
> -			if (!__pci_reset_function_locked(pdev))
> +		if (vdev->cxl_ops && vdev->cxl_ops->reset) {
> +			/*
> +			 * VM power-off resets a CXL Type-2 device through its
> +			 * DVSEC sequence. The sequence takes its own device lock,
> +			 * so run it outside the lock below.
> +			 * vconfig is already freed here, so call the op directly
> +			 * under memory_lock rather than the wrapper.
> +			 */
> +			down_write(&vdev->memory_lock);
> +			if (!vdev->cxl_ops->reset(vdev))
>  				vdev->needs_reset = false;
> -			pci_dev_unlock(pdev);
> +			up_write(&vdev->memory_lock);
> +		} else {
> +			bridge = pci_upstream_bridge(pdev);
> +			if (bridge && !pci_dev_trylock(bridge))
> +				goto out_restore_state;
> +			if (pci_dev_trylock(pdev)) {
> +				if (!__pci_reset_function_locked(pdev))
> +					vdev->needs_reset = false;
> +				pci_dev_unlock(pdev);
> +			}
> +			if (bridge)
> +				pci_dev_unlock(bridge);
>  		}
> -		if (bridge)
> -			pci_dev_unlock(bridge);
>  	}
>  
>  out_restore_state:
> @@ -1592,6 +1625,20 @@ static int vfio_pci_ioctl_set_irqs(struct vfio_pci_core_device *vdev,
>  	return ret;
>  }
>  
> +/*
> + * Reset the function. A CXL device runs the CXL DVSEC reset sequence in place
> + * of a PCI function reset: it replaces FLR (which would corrupt CXL.mem),
> + * always clears device memory, and restores the HDM decoder. Callers hold
> + * memory_lock for write.
> + */
> +int vfio_pci_reset_function(struct vfio_pci_core_device *vdev)
> +{
> +	if (!vdev->cxl_ops || !vdev->cxl_ops->reset)
> +		return pci_try_reset_function(vdev->pdev);
> +
> +	return vdev->cxl_ops->reset(vdev);
> +}
> +
>  static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
>  				void __user *arg)
>  {
> @@ -1614,8 +1661,14 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev,
>  	vfio_pci_set_power_state(vdev, PCI_D0);
>  
>  	vfio_pci_dma_buf_move(vdev, true);
> -	ret = pci_try_reset_function(vdev->pdev);
> -	if (__vfio_pci_memory_enabled(vdev))
> +	ret = vfio_pci_reset_function(vdev);
> +	/*
> +	 * Re-arm the dma-bufs on success. A CXL device whose reset failed leaves
> +	 * the HDM decoder unrestored and hdm_valid clear, so re-arming its HDM
> +	 * dma-buf would map device DMA onto a decoder the fault path still gates;
> +	 * keep it revoked until a reset succeeds. Plain vfio-pci is unchanged.
> +	 */
> +	if (__vfio_pci_memory_enabled(vdev) && (!vdev->cxl_ops || !ret))
>  		vfio_pci_dma_buf_move(vdev, false);
>  	up_write(&vdev->memory_lock);
>  
> diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h
> index c268c99aea82..e1ef21806a2f 100644
> --- a/drivers/vfio/pci/vfio_pci_priv.h
> +++ b/drivers/vfio/pci/vfio_pci_priv.h
> @@ -78,6 +78,7 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev,
>  			     pci_power_t state);
>  
>  void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev);
> +int vfio_pci_reset_function(struct vfio_pci_core_device *vdev);
>  u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev);
>  void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev,
>  					u16 cmd);
> diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
> index 39a28cc6ae8c..231679dead45 100644
> --- a/include/linux/vfio_pci_core.h
> +++ b/include/linux/vfio_pci_core.h
> @@ -74,6 +74,10 @@ struct vfio_cxl_ops {
>  	void	(*close_device)(struct vfio_pci_core_device *vdev);
>  	void	(*reset_prepare)(struct vfio_pci_core_device *vdev);
>  	void	(*reset_done)(struct vfio_pci_core_device *vdev);
> +	/* Run the CXL reset (always clears CXL.mem) in place of FLR */
> +	int	(*reset)(struct vfio_pci_core_device *vdev);
> +	/* True while the HDM range is valid and its dma-buf may be armed */
> +	bool	(*hdm_active)(struct vfio_pci_core_device *vdev);
>  	/* Pinned per bound CXL device so vfio-cxl cannot unload under usage */
>  	struct module *owner;
>  };
> -- 
> 2.25.1
> 
> 

Re: [PATCH v5 06/27] vfio/pci: Detect CXL devices and load the CXL provider on demand

[Richard Cheng]

On Thu, Sep 17, 2026 at 12:05:19AM +0800, mhonap@nvidia.com wrote:
> From: Manish Honap <mhonap@nvidia.com>
> 
> A CXL device needs the vfio-cxl callbacks, but pulling vfio-cxl and the
> CXL core in unconditionally would bloat every vfio-pci setup. Detect a
> CXL Type-2 device at bind and request_module("vfio-cxl") only then, then
> hand the device to the registered ops.
> 
> Each bound CXL device pins the provider through
> vfio_pci_get_cxl_ops() (try_module_get) and drops it with
> vfio_pci_put_cxl_ops() at release, so vfio-cxl can unload once no CXL
> device is bound.
> 
> If the provider is absent the device is driven as plain vfio-pci. A
> built-in provider whose initcall has not run yet is waited for with
> -EPROBE_DEFER; the deferred-probe machinery will retry the bind once the
> provider registers.
> 
> Assisted-by: LLM
> Signed-off-by: Manish Honap <mhonap@nvidia.com>
> ---
>  drivers/vfio/pci/vfio_pci_core.c | 84 ++++++++++++++++++++++++++++++++
>  include/linux/vfio_pci_core.h    |  3 ++
>  2 files changed, 87 insertions(+)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index 2cc5dd20396c..9eede1e56ab5 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -2202,6 +2202,84 @@ void vfio_pci_core_unregister_cxl_ops(const struct vfio_cxl_ops *ops)
>  }
>  EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_cxl_ops);
>  
> +static const struct vfio_cxl_ops *vfio_pci_get_cxl_ops(void)
> +{
> +	guard(rwsem_read)(&vfio_pci_cxl_ops_rwsem);
> +
> +	if (vfio_pci_cxl_ops && try_module_get(vfio_pci_cxl_ops->owner))
> +		return vfio_pci_cxl_ops;
> +
> +	return NULL;
> +}
> +
> +static void vfio_pci_put_cxl_ops(const struct vfio_cxl_ops *ops)
> +{
> +	module_put(ops->owner);
> +}
> +

Anything guarantees that we can finish devres cleanup before dropping module ref ?

This function drops the module ref during vfio_pci_remove(), while devres might still pending.

I see in patch 18 it register vfio_cxl_release_hpa() as a devres callback, and it lives in vfio-cxl.
If this drops the last module ref, another process could unload vfio-cxl before devres invokes that callback.

Best regards,
Richard Cheng.




> +/*
> + * A CXL Type-2 device advertises both CXL.cache and CXL.mem in its CXL DVSEC.
> + * pcie_is_cxl() is also true for Type-1 (cache only) and Type-3 (mem only)
> + * devices, which the vfio-cxl provider does not handle, so confirm the Type-2
> + * identity before engaging it.
> + */
> +static bool vfio_pci_is_cxl_type2(struct pci_dev *pdev)
> +{
> +	u16 dvsec, cap;
> +
> +	if (!pcie_is_cxl(pdev))
> +		return false;
> +
> +	dvsec = pci_find_dvsec_capability(pdev, PCI_VENDOR_ID_CXL,
> +					  PCI_DVSEC_CXL_DEVICE);
> +	if (!dvsec)
> +		return false;
> +
> +	if (pci_read_config_word(pdev, dvsec + PCI_DVSEC_CXL_CAP, &cap))
> +		return false;
> +
> +	return (cap & PCI_DVSEC_CXL_CACHE_CAPABLE) &&
> +		(cap & PCI_DVSEC_CXL_MEM_CAPABLE);
> +}
> +
> +/*
> + * Load vfio-cxl on demand for a CXL Type-2 device and hand the device to its
> + * ops. If the provider is absent the device is driven as plain vfio-pci; a
> + * built-in provider whose initcall has not run yet is waited for with
> + * -EPROBE_DEFER.
> + */
> +static int vfio_pci_core_cxl_init(struct vfio_pci_core_device *vdev)
> +{
> +	const struct vfio_cxl_ops *ops;
> +	int ret;
> +
> +	if (!vfio_pci_is_cxl_type2(vdev->pdev))
> +		return 0;
> +
> +	request_module("vfio-cxl");
> +	ops = vfio_pci_get_cxl_ops();
> +	if (!ops)
> +		return IS_BUILTIN(CONFIG_VFIO_CXL) ? -EPROBE_DEFER : 0;
> +
> +	ret = ops->init(vdev);
> +	if (ret) {
> +		vfio_pci_put_cxl_ops(ops);
> +		return ret;
> +	}
> +
> +	vdev->cxl_ops = ops;
> +	return 0;
> +}
> +
> +static void vfio_pci_core_cxl_release(struct vfio_pci_core_device *vdev)
> +{
> +	if (!vdev->cxl_ops)
> +		return;
> +
> +	vdev->cxl_ops->release(vdev);
> +	vfio_pci_put_cxl_ops(vdev->cxl_ops);
> +}
> +
>  int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
>  {
>  	struct vfio_pci_core_device *vdev =
> @@ -2223,6 +2301,10 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev)
>  	init_rwsem(&vdev->memory_lock);
>  	xa_init(&vdev->ctx);
>  
> +	ret = vfio_pci_core_cxl_init(vdev);
> +	if (ret)
> +		return ret;
> +
>  	return 0;
>  }
>  EXPORT_SYMBOL_GPL(vfio_pci_core_init_dev);
> @@ -2232,6 +2314,8 @@ void vfio_pci_core_release_dev(struct vfio_device *core_vdev)
>  	struct vfio_pci_core_device *vdev =
>  		container_of(core_vdev, struct vfio_pci_core_device, vdev);
>  
> +	vfio_pci_core_cxl_release(vdev);
> +
>  	mutex_destroy(&vdev->igate);
>  	mutex_destroy(&vdev->ioeventfds_lock);
>  	kfree(vdev->region);
> diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
> index 9fe0d1a3a370..7f3a2bcb5830 100644
> --- a/include/linux/vfio_pci_core.h
> +++ b/include/linux/vfio_pci_core.h
> @@ -29,6 +29,7 @@ struct vfio_pci_core_device;
>  struct vfio_pci_region;
>  struct p2pdma_provider;
>  struct dma_buf_attachment;
> +struct vfio_cxl_state;
>  
>  struct vfio_pci_eventfd {
>  	struct eventfd_ctx	*ctx;
> @@ -109,6 +110,8 @@ struct vfio_pci_core_device {
>  	struct vfio_device	vdev;
>  	struct pci_dev		*pdev;
>  	const struct vfio_pci_device_ops *pci_ops;
> +	const struct vfio_cxl_ops *cxl_ops;
> +	struct vfio_cxl_state   *cxl;
>  	void __iomem		*barmap[PCI_STD_NUM_BARS];
>  	bool			bar_mmap_supported[PCI_STD_NUM_BARS];
>  	/* Flags modified at runtime - dedicated storage unit */
> -- 
> 2.25.1
> 
>