[RFC PATCH 0/12] drm/fabric: vendor-neutral topology infrastructure for scale-up accelerator interconnects

[Konstantin Sinyuk <ksinyuk@kernel.org>]

Modern GPUs and AI accelerators are increasingly connected through scale-up
fabrics such as AMD xGMI today and emerging UALink systems [4]. Linux lacks
common, vendor-neutral infrastructure for reporting which accelerators are
directly connected, through which ports, and in what state.

Vendors expose fragments through driver-specific interfaces such as amdgpu's
xGMI sysfs [3] and AMD's UALink pod-configuration sysfs [6], while prior
per-driver proposals such as XeLink never became shared infrastructure, so
topology semantics remain per-driver.

Building on the LPC 2025 "Toward Mainline Linux Support for UALink" BoF
[1], this RFC proposes DRM Fabric: vendor-neutral, protocol-agnostic DRM
topology infrastructure for scale-up interconnects, using the model

    fabric -> endpoint -> port -> peer

A fabric groups the endpoints of one provider-defined interconnect
instance. An endpoint is one accelerator attachment holding its physical
ports. A port reports lane capability, operational state and optional
counters. A peer is a typed value naming the directly adjacent accelerator
or switch port, not a reference to a live kernel object; it may identify an
accelerator managed by another OS or an opaque switch in another trust
domain.

The core records direct adjacency only, not end-to-end reachability or
switch forwarding, which remain with the fabric controller. Vendor drivers
retain hardware discovery, firmware interaction, memory semantics and the
hardware-carried data path. DRM Fabric represents topology and control
state of DRM-managed accelerators; it does not create a network device or
own route computation, switch forwarding, transport, or congestion control.

Generic Netlink fits this multi-object, event-driven model better than
sysfs, which maps poorly onto dump enumeration and asynchronous
notifications, and provides the YAML-described uAPI discipline introduced
to DRM by DRM RAS [2]. Devlink was considered, but models devices and
subordinate objects rather than a fabric spanning DRM devices.

Patches 1-6 form a complete, separately tested read-only milestone that can
be merged independently while provisioning remains under review. Providers
publish objects, adjacency, operational state and optional port statistics
through a small in-kernel API; userspace queries the live graph but cannot
modify it.

Patches 7-12 add privileged provisioning for software-defined fabrics.
Userspace can create and delete empty fabrics, attach orphan endpoints
registered without a fabric, request administrative state and manage peer
adjacency, while the provider performs the hardware programming.
Administrative state records control-plane intent, operational state
remains provider-reported, and each port is managed either by its provider
or by userspace. The mutation operations are fabric-new, fabric-del,
endpoint-set, port-set, port-peer-new and port-peer-del.

Provider callbacks may sleep or report state back into the core, so
drm_fabric_lock is not held across them. Provisioning instead uses an outer
mutation lock across validation, callback and commit. The complete locking,
lifetime and failure contracts are documented in
Documentation/gpu/drm-fabric.rst.

Every operation is confined to the initial network namespace, and mutation
additionally requires CAP_NET_ADMIN.

drm_fabric_sim is a software-only provider modeled on netdevsim [5]. It
registers linear, mesh and switch-facing topologies without accelerator
hardware and gains provisioning callbacks and fault injection. Its debugfs
interface is not ABI; the reviewed interface is the Generic Netlink family.

The series defines no accelerator memory sharing, MMU programming, data
transport, route computation, switch policy, key management, live
migration, required userspace daemon or protocol-specific commands.

No production provider is included. Shipping only the synthetic provider is
deliberate, allowing the object model and uAPI to be reviewed before being
tied to hardware. We used the public amdgpu xGMI implementation to shape
the provider API, but that mapping has not been validated by AMD.

Example queries using the in-tree YNL tool are:

    $ ./tools/net/ynl/pyynl/cli.py \
          --spec Documentation/netlink/specs/drm_fabric.yaml \
          --dump fabric-get
    $ ./tools/net/ynl/pyynl/cli.py \
          --spec Documentation/netlink/specs/drm_fabric.yaml \
          --dump endpoint-get --json '{"fabric-id": <id>}'
    $ ./tools/net/ynl/pyynl/cli.py \
          --spec Documentation/netlink/specs/drm_fabric.yaml \
          --do port-get --json '{"endpoint-id": <id>, "port-index": 0}'

Testing includes 12/12 per-commit W=1 builds, 43 passing KUnit tests and
172 passing kselftest results across 14 programs. KASAN, UBSAN, lockdep,
atomic-sleep and kmemleak validation completed without reports.

The series is also available from the public review tree:

  https://git.kernel.org/pub/scm/linux/kernel/git/ksinyuk/linux.git/tag/?h=drm-fabric-v1

We are specifically requesting feedback on:

  1. Is fabric -> endpoint -> port -> peer the right minimum common DRM
     representation for accelerator interconnects?

  2. Are the Generic Netlink semantics, including dump consistency,
     notifications and optional port statistics, aligned with YNL
     expectations?

  3. Do the object model and provider API cover xGMI topology-management
     requirements, or would any xGMI requirement force protocol-specific
     uAPI or provider semantics? AMD's assessment would be especially
     valuable.

[1] LPC 2025 BoF, "Toward Mainline Linux Support for UALink":
    https://lpc.events/event/19/contributions/2308/

[2] Riana Tauro, "[PATCH v10 0/5] Introduce DRM_RAS using generic netlink
    for RAS", 2026-03-04:
    https://lore.kernel.org/dri-devel/20260304074412.464435-7-riana.tauro@intel.com

[3] amdgpu xGMI topology:
    drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c

[4] UALink specification:
    https://ualinkconsortium.org/specification/

[5] netdevsim:
    Documentation/networking/devlink/netdevsim.rst

[6] Alex Deucher, "[PATCH 00/95] Add UALink instrastructure series 1",
    2026-08-21:
    https://lore.kernel.org/amd-gfx/20260821193458.808626-1-alexander.deucher@amd.com/

Ilia Levi (4):
  drm/fabric: add core object model and provider API
  drm/fabric: add query uAPI and generated headers
  drm/fabric: add topology-provisioning core
  drm/fabric: add provisioning netlink uAPI

Konstantin Sinyuk (8):
  drm/fabric: implement query netlink operations
  drm/fabric: add read-only synthetic provider
  drm/fabric: add object-model KUnit tests
  drm/fabric: add YNL query and policy selftests
  drm/fabric: implement mutation netlink operations
  drm/fabric: make the synthetic provider writable
  drm/fabric: add mutation KUnit tests
  drm/fabric: add mutation netlink selftests

 Documentation/gpu/drm-fabric.rst              |  479 +++
 Documentation/gpu/index.rst                   |    1 +
 Documentation/netlink/specs/drm_fabric.yaml   |  580 ++++
 MAINTAINERS                                   |   14 +
 drivers/gpu/drm/Kconfig                       |    1 +
 drivers/gpu/drm/Makefile                      |    1 +
 drivers/gpu/drm/fabric/.kunitconfig           |    5 +
 drivers/gpu/drm/fabric/.kunitconfig.debug     |   16 +
 drivers/gpu/drm/fabric/Kconfig                |   41 +
 drivers/gpu/drm/fabric/Makefile               |    9 +
 drivers/gpu/drm/fabric/drm_fabric.c           | 1045 +++++++
 drivers/gpu/drm/fabric/drm_fabric_internal.h  |   66 +
 drivers/gpu/drm/fabric/drm_fabric_netlink.c   | 1310 ++++++++
 drivers/gpu/drm/fabric/drm_fabric_nl.c        |  231 ++
 drivers/gpu/drm/fabric/drm_fabric_nl.h        |   66 +
 drivers/gpu/drm/fabric/drm_fabric_sim.c       | 1044 +++++++
 drivers/gpu/drm/fabric/drm_fabric_test.c      | 2635 +++++++++++++++++
 include/drm/drm_fabric.h                      |  333 +++
 include/uapi/drm/drm_fabric.h                 |  162 +
 tools/testing/selftests/Makefile              |    1 +
 .../selftests/drivers/gpu/drm_fabric/Makefile |   44 +
 .../drivers/gpu/drm_fabric/README.rst         |  146 +
 .../drivers/gpu/drm_fabric/cap_netadmin.py    |  314 ++
 .../gpu/drm_fabric/check-spec-regen.sh        |  114 +
 .../selftests/drivers/gpu/drm_fabric/config   |   13 +
 .../drivers/gpu/drm_fabric/dump_intr_abi.py   |  361 +++
 .../drivers/gpu/drm_fabric/dump_scale_abi.py  |  178 ++
 .../drivers/gpu/drm_fabric/fabric_abi.py      |  553 ++++
 .../drivers/gpu/drm_fabric/fault_abi.py       |  302 ++
 .../gpu/drm_fabric/harness_reset_abi.py       |  113 +
 .../drivers/gpu/drm_fabric/hotplug_abi.py     |  331 +++
 .../drivers/gpu/drm_fabric/lib_drm_fabric.py  |  498 ++++
 .../drivers/gpu/drm_fabric/netns_abi.py       |  294 ++
 .../drivers/gpu/drm_fabric/nl_policy_probe.py |  651 ++++
 .../drivers/gpu/drm_fabric/port_cursor_abi.py |  401 +++
 .../gpu/drm_fabric/port_stats_cap_abi.py      |  374 +++
 .../drm_fabric/provisioning_scenarios_abi.py  |  324 ++
 .../selftests/drivers/gpu/drm_fabric/settings |    1 +
 .../drivers/gpu/drm_fabric/switch_abi.py      |  137 +
 39 files changed, 13189 insertions(+)
 create mode 100644 Documentation/gpu/drm-fabric.rst
 create mode 100644 Documentation/netlink/specs/drm_fabric.yaml
 create mode 100644 drivers/gpu/drm/fabric/.kunitconfig
 create mode 100644 drivers/gpu/drm/fabric/.kunitconfig.debug
 create mode 100644 drivers/gpu/drm/fabric/Kconfig
 create mode 100644 drivers/gpu/drm/fabric/Makefile
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric.c
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_internal.h
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_netlink.c
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_nl.c
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_nl.h
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_sim.c
 create mode 100644 drivers/gpu/drm/fabric/drm_fabric_test.c
 create mode 100644 include/drm/drm_fabric.h
 create mode 100644 include/uapi/drm/drm_fabric.h
 create mode 100644 tools/testing/selftests/drivers/gpu/drm_fabric/Makefile
 create mode 100644 tools/testing/selftests/drivers/gpu/drm_fabric/README.rst
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/cap_netadmin.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/check-spec-regen.sh
 create mode 100644 tools/testing/selftests/drivers/gpu/drm_fabric/config
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/dump_intr_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/dump_scale_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/fabric_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/fault_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/harness_reset_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/hotplug_abi.py
 create mode 100644 tools/testing/selftests/drivers/gpu/drm_fabric/lib_drm_fabric.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/netns_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/nl_policy_probe.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/port_cursor_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/port_stats_cap_abi.py
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/provisioning_scenarios_abi.py
 create mode 100644 tools/testing/selftests/drivers/gpu/drm_fabric/settings
 create mode 100755 tools/testing/selftests/drivers/gpu/drm_fabric/switch_abi.py


base-commit: e3e05bddbe9fea67639d2a3551e7cf2d473b804c
-- 
2.43.0