[PATCH v3 00/16] New DRM accel driver for Texas Instruments' C7x DSPs

[PATCH v3 00/16] New DRM accel driver for Texas Instruments' C7x DSPs

[Tomeu Vizoso]

This series adds a new DRM accel driver that supports the C7x DSPs
inside some Texas Instruments SoCs such as the J722S (the only SoC
currently supported). These can be used as accelerators for various
workloads, including machine learning inference.

This driver attaches to the DSPs through remoteproc and talks to the
firmware running on them through shared memory and a mailbox. The
firmware is the accelerator's command processor: the driver pushes Jobs
onto a per-core Ring in shared memory and rings a mailbox doorbell; the
firmware runs each Job's Kernel in the DSP's user mode on the client's
own page table, moves a Fence when the Job is done and rings back.

The kernel driver itself allocates buffers, manages contexts and submits
jobs to the DSP firmware. It also populates the per-client page tables
with io-pgtable-arm.

No carveout holds buffers any more: every buffer is the kernel driver's,
page-backed and mapped through the DSP's MMU in the client's own page
table, which answers Andrew F. Davis's question on v2. The one reserved
region the driver uses is the first 256 KiB of the IPC pool, where the
handshake, the Ring and the Context table live. A buffer that a DMA
transfer names must be physically contiguous, since the DRU does not
go through the MMU; the client asks for that at creation.

If there is more than one C7x DSP core, the driver will load balance
jobs between them with drm-gpu-scheduler.

The firmware isolates clients by using its MMU to protect their memory
and by running their kernels at a lower privilege level.

Besides executing user-generated DSP kernels, the firmware can also
execute DMA transfers at the request of userspace. These don't go via
the MMU, so are validated by the kernel driver.

The source code for the firmware running on the DSP is available at:
https://gitlab.freedesktop.org/tomeu/thames-firmware/ mostly under the MIT
license. The ABI with the firmware is defined in a shared header with
versioning.

Known limitations of this version:

- Runtime power management is not implemented (system suspend and
  resume are).
- The J722S EVM does not resume from s2idle with TI's shipped
  system-controller firmware, so the resume path ran only in the
  driver's own tests; the suspend path ran on the board.
- The submit IOCTL waits on no sync object, which a Vulkan driver will
  need.
- The 4D block move is the only DMA transfer type accepted.
- One firmware image serves both cores; a core found running another
  firmware is left out after the handshake's five seconds.
- The DRU channel stride is stated for the J722S instead of queried.
- The J722S is the only SoC this version runs on.

Three additions were needed in the device tree bindings:

- power domain references for the two SoC timers the DSP firmware
  drives, which the remoteproc driver does not hold;
- a second mailbox doorbell, since remoteproc's own serves rpmsg and
  this firmware rings the Ring's; and
- reg entries for the AUX SRAM and the DMA engine's registers, which
  the driver maps into every client's page table.

Everything else is done in userspace, as a Gallium driver (also called
thames) that is part of the Mesa3D project: https://docs.mesa3d.org/teflon.html

Userspace portion of the driver: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39298

Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
Changes in v3:
- Shared-memory ring and mailbox doorbell replace rpmsg
- The kernel driver writes the client page tables
- Buffers are bound at userspace-chosen DSP addresses
- Kernels run in the DSP's user mode; faults reported per job
- DRU transfers submitted by userspace, checked and translated by the kernel driver
- System suspend and resume
- Versioned firmware ABI; hardware ID reported and checked against the SoC
- Firmware rewritten bare-metal, MIT
- Link to v2: https://lore.kernel.org/r/20260114-thames-v2-0-e94a6636e050@tomeuvizoso.net

Changes in v2:
- Add thames_accel.h UAPI header (Robert Nelson).
- Link to v1: https://lore.kernel.org/r/20260113-thames-v1-0-99390026937c@tomeuvizoso.net

To: Bjorn Andersson <andersson@kernel.org>
To: Mathieu Poirier <mathieu.poirier@linaro.org>
To: Rob Herring <robh@kernel.org>
To: Krzysztof Kozlowski <krzk+dt@kernel.org>
To: Conor Dooley <conor+dt@kernel.org>
To: Suman Anna <s-anna@ti.com>
To: Tomeu Vizoso <tomeu@tomeuvizoso.net>
To: Oded Gabbay <ogabbay@kernel.org>
To: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
To: Maxime Ripard <mripard@kernel.org>
To: Thomas Zimmermann <tzimmermann@suse.de>
To: David Airlie <airlied@gmail.com>
To: Simona Vetter <simona@ffwll.ch>
To: Kees Cook <kees@kernel.org>
To: "Gustavo A. R. Silva" <gustavoars@kernel.org>
To: Sumit Semwal <sumit.semwal@linaro.org>
To: Christian König <christian.koenig@amd.com>
To: Jonathan Corbet <corbet@lwn.net>
To: Shuah Khan <skhan@linuxfoundation.org>
To: Randy Dunlap <rdunlap@infradead.org>
To: Nishanth Menon <nm@ti.com>
To: Vignesh Raghavendra <vigneshr@ti.com>
To: Tero Kristo <kristo@kernel.org>
Cc: linux-remoteproc@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: dri-devel@lists.freedesktop.org
Cc: linux-hardening@vger.kernel.org
Cc: linux-media@vger.kernel.org
Cc: linaro-mm-sig@lists.linaro.org
Cc: linux-doc@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org

---
Tomeu Vizoso (16):
      dt-bindings: remoteproc: k3-dsp: Add the L2 auxiliary and DRU regions
      dt-bindings: remoteproc: k3-dsp: Allow a second sub-mailbox
      dt-bindings: remoteproc: k3-dsp: Allow referencing the timers' power domains
      accel/thames: Add header with IPC definitions
      accel/thames: Add driver for the C7x DSPs in TI SoCs
      accel/thames: Create a context per client on device open
      accel/thames: Add IOCTLs for BO creation and mapping
      accel/thames: Add support for submitting DMA operations
      accel/thames: Add IOCTLs for job creation and submission
      accel/thames: Recover from faults and timeouts
      accel/thames: Add IOCTL for memory synchronization
      accel/thames: Install memory mappings populated and dirty
      Documentation: accel/thames: Add initial docs
      arm64: dts: ti: k3-j722s: Add the AUX L2 SRAM and DRU regions
      arm64: dts: ti: k3-j722s: Add second submailbox to the C7x cores
      arm64: dts: ti: k3-j722s: Reference the power domains of the C7x timers

 Documentation/accel/index.rst                      |    1 +
 Documentation/accel/thames/index.rst               |   47 +
 .../bindings/remoteproc/ti,k3-dsp-rproc.yaml       |   33 +-
 MAINTAINERS                                        |    9 +
 arch/arm64/boot/dts/ti/k3-j722s-main.dtsi          |   16 +-
 .../boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi      |   27 +-
 arch/arm64/boot/dts/ti/k3-j722s.dtsi               |    4 +
 drivers/accel/Kconfig                              |    1 +
 drivers/accel/Makefile                             |    3 +-
 drivers/accel/thames/Kconfig                       |   27 +
 drivers/accel/thames/Makefile                      |   12 +
 drivers/accel/thames/thames_abi.h                  |  238 ++++
 drivers/accel/thames/thames_core.c                 |  867 +++++++++++++
 drivers/accel/thames/thames_core.h                 |  120 ++
 drivers/accel/thames/thames_device.h               |   97 ++
 drivers/accel/thames/thames_drv.c                  |  554 ++++++++
 drivers/accel/thames/thames_drv.h                  |   67 +
 drivers/accel/thames/thames_gem.c                  | 1077 ++++++++++++++++
 drivers/accel/thames/thames_gem.h                  |   77 ++
 drivers/accel/thames/thames_job.c                  | 1319 ++++++++++++++++++++
 drivers/accel/thames/thames_job.h                  |   81 ++
 drivers/accel/thames/thames_trace.h                |   77 ++
 drivers/accel/thames/thames_transfers.c            |  405 ++++++
 drivers/accel/thames/thames_transfers.h            |   52 +
 include/uapi/drm/thames_accel.h                    |  843 +++++++++++++
 25 files changed, 6043 insertions(+), 11 deletions(-)
---
base-commit: 1ecab8b0bad393750c60cc3819ce9c256595463c
change-id: 20260113-thames-334127a2d91d

Best regards,
--  
Tomeu Vizoso <tomeu@tomeuvizoso.net>

[PATCH v3 01/16] dt-bindings: remoteproc: k3-dsp: Add the L2 auxiliary and DRU regions

[Tomeu Vizoso]

Besides the existing L2 SRAM region, at least some C7x DSPs in K3 SoCs
have an additional region called in the documentation "AUX L2 SRAM".
This is just some more SRAM that is accessed separately.

Also add the memory region for the control registers of the DRU (Data
Routing Unit), which allow starting DMA transactions and polling for
their completion.

The two entries are optional, so existing device trees still validate,
and are allowed only for the ti,am62a-c7xv-dsp compatible, the C7x
generation that has them.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Initial version
---
 Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
index 8b1ed384ef22..278b88b7f906 100644
--- a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
@@ -122,11 +122,17 @@ allOf:
     then:
       properties:
         reg:
+          minItems: 1
           items:
             - description: Address and Size of the L2 SRAM internal memory region
+            - description: Address and Size of the auxiliary L2 SRAM aperture
+            - description: Address and Size of the DRU real-time register page
         reg-names:
+          minItems: 1
           items:
             - const: l2sram
+            - const: l2aux
+            - const: dru
 
   - $ref: /schemas/arm/keystone/ti,k3-sci-common.yaml#
 

-- 
2.54.0

[PATCH v3 02/16] dt-bindings: remoteproc: k3-dsp: Allow a second sub-mailbox

[Tomeu Vizoso]

Allow specifying a second sub-mailbox that could be used by the firmware
in addition to the first one, which is taken by remoteproc.

This is required by firmware that doesn't use rpmsg for their
communication with the Linux kernel.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Initial version
---
 .../devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml        | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
index 278b88b7f906..d6319c8e96f8 100644
--- a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
@@ -51,10 +51,12 @@ properties:
 
   mboxes:
     description: |
-      OMAP Mailbox specifier denoting the sub-mailbox, to be used for
-      communication with the remote processor. This property should match
-      with the sub-mailbox node used in the firmware image.
-    maxItems: 1
+      OMAP Mailbox specifiers denoting the sub-mailboxes allotted to
+      communication with the remote processor. The first is the one the
+      remoteproc core uses and should match the sub-mailbox node used in the
+      firmware image. The second is optional.
+    minItems: 1
+    maxItems: 2
 
   memory-region:
     minItems: 2

-- 
2.54.0

[PATCH v3 03/16] dt-bindings: remoteproc: k3-dsp: Allow referencing the timers' power domains

[Tomeu Vizoso]

A firmware may drive the core's timers, which need their power domains
held; the remoteproc driver does not hold them, so let the DSP node
reference them for a driver that attaches to the running core to hold.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Initial version
---
 .../devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
index d6319c8e96f8..80eaef0a19b8 100644
--- a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
@@ -76,6 +76,23 @@ properties:
 # Optional properties:
 # --------------------
 
+  power-domains:
+    description: |
+      Power domains of the SoC timers the running firmware ticks on and
+      times its work out with. The firmware programs these timers itself
+      and makes no power request for them, and the timers' own nodes are
+      reserved for it, so nothing else requests their power; a client
+      driver of the firmware holds them on while it drives the processor.
+      The processor's own power is managed through ti,sci-dev-id, not here.
+    items:
+      - description: power domain of the first timer
+      - description: power domain of the second timer
+
+  power-domain-names:
+    items:
+      - const: timer1
+      - const: timer2
+
   sram:
     minItems: 1
     maxItems: 4

-- 
2.54.0

[PATCH v3 04/16] accel/thames: Add header with IPC definitions

[Tomeu Vizoso]

This header defines constants and structs used by both the kernel driver
and the firmware for communication.

It is versioned following Sound Open Firmware's conventions with some
measures to avoid having to increase the major (incompatible) version.

An identical copy is in the firmware's code.

Every structure is laid out on 128-byte lines, the coherence line of
the C7x, with the words the host writes and the words the firmware
writes on different lines. The Ring has 64 entries and a fault record
per entry; the Context table has 64 entries, the ASID width the
firmware reports permitting. The Kernels' stack and save area sit at a
fixed address in every Context, above 1 TiB, so the firmware's user-mode
entry needs no per-Context setup. The host writes a random challenge
into the Ring and the firmware echoes it in the handshake, which tells
this boot's handshake from one an earlier firmware left in the memory.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Initial version.
---
 drivers/accel/thames/thames_abi.h | 238 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 238 insertions(+)

diff --git a/drivers/accel/thames/thames_abi.h b/drivers/accel/thames/thames_abi.h
new file mode 100644
index 000000000000..8f8b8323f747
--- /dev/null
+++ b/drivers/accel/thames/thames_abi.h
@@ -0,0 +1,238 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/* Copyright (C) 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+#ifndef __THAMES_ABI_H__
+#define __THAMES_ABI_H__
+
+/* Shared with the firmware, whose copy is byte-equal. */
+#ifdef __KERNEL__
+#include <linux/types.h>
+#include <linux/stddef.h>
+#include <linux/build_bug.h>
+#else
+#include <stdint.h>
+#include <stddef.h>
+typedef uint8_t __u8;
+typedef uint16_t __u16;
+typedef uint32_t __u32;
+typedef uint64_t __u64;
+#endif
+
+#ifndef THAMES_ABI_ASSERT
+#ifdef __KERNEL__
+#define THAMES_ABI_ASSERT(expr, msg) static_assert(expr, msg)
+#else
+#define THAMES_ABI_ASSERT(expr, msg) _Static_assert(expr, msg)
+#endif
+#endif
+
+/*
+ * The kernel driver refuses a firmware whose major differs; a newer minor on
+ * either side is accepted.  A field appended under a minor reads zero from an
+ * older firmware, which means "not reported".
+ */
+#define THAMES_ABI_MAJOR	1
+#define THAMES_ABI_MINOR	0
+#define THAMES_ABI_PATCH	0
+
+#define THAMES_ABI_VERSION \
+	((THAMES_ABI_MAJOR << 24) | (THAMES_ABI_MINOR << 12) | THAMES_ABI_PATCH)
+
+#define THAMES_ABI_VERSION_MAJOR(version)	(((version) >> 24) & 0xff)
+#define THAMES_ABI_VERSION_MINOR(version)	(((version) >> 12) & 0xfff)
+#define THAMES_ABI_VERSION_PATCH(version)	((version) & 0xfff)
+
+/*
+ * No coherence line holds both a field the kernel driver writes and one the
+ * firmware writes.
+ */
+#define THAMES_ABI_LINE_SIZE	128
+
+#define THAMES_RING_DEPTH	64
+#define THAMES_MAX_CONTEXTS	64
+
+/*
+ * The save page sits on an 8 KiB boundary: TCSP keeps a nesting count in bits
+ * 16:13 and reserves bits 12:0 (SPRUIP0 3.7.49.2, Table 3-76).
+ */
+#define THAMES_USER_REGION_BASE		0x10000000000ull
+#define THAMES_USER_REGION_SIZE		0x200000ull
+#define THAMES_USER_SAVE_AREA_BASE	(THAMES_USER_REGION_BASE + 0x2000ull)
+#define THAMES_USER_SAVE_AREA_SIZE	0x2000ull
+#define THAMES_USER_STACK_BASE		(THAMES_USER_REGION_BASE + 0x5000ull)
+#define THAMES_USER_STACK_SIZE		0x1000ull
+
+/*
+ * The uAPI's fault kinds, register indexes and hardware IDs, for the firmware,
+ * which cannot include the uAPI header.
+ */
+#define THAMES_FAULT_KIND_KERNEL	1u
+#define THAMES_FAULT_KIND_FIRMWARE	2u
+#define THAMES_FAULT_KIND_TIMEOUT	3u
+
+#define THAMES_FAULT_REG_IERR		0
+#define THAMES_FAULT_REG_IEAR		1
+#define THAMES_FAULT_REG_IESR		2
+#define THAMES_FAULT_REG_NRP		3
+#define THAMES_FAULT_REG_NTSR		4
+#define THAMES_FAULT_REG_SE0_FAR	5
+#define THAMES_FAULT_REG_SE0_FSR	6
+#define THAMES_FAULT_REG_SE1_FAR	7
+#define THAMES_FAULT_REG_SE1_FSR	8
+#define THAMES_FAULT_REGISTERS		9
+
+#define THAMES_HW_J722S_C7524		1u
+
+/*
+ * Written by the firmware at boot.  Its loop then copies the Ring's challenge
+ * into challenge_echo, which tells this boot's handshake from one an earlier
+ * firmware left in the IPC memory.
+ *
+ * @ipc_address: the address the firmware was linked for.
+ * @dru_kernel_channel_first: the first of the dru_channel_count channels a
+ *	Job's transfers name, all on one page of the DRU's real-time registers;
+ *	the firmware's own channels are on another.
+ */
+struct thames_handshake {
+	__u32 abi_version;
+	__u32 l2_size;
+	__u64 ipc_address;
+	__u32 dru_channel_count;
+	__u32 dru_queue_depth;
+	__u32 asid_bits;
+	__u32 hardware_id;
+	__u32 dru_channel_stride;
+	__u32 dru_kernel_channel_first;
+	__u32 timestamp_hz;
+	__u32 pad;
+	__u64 challenge_echo;
+	__u64 dru_capabilities;
+	__u32 transfer_table_pool_size;
+	__u8 reserved[2 * THAMES_ABI_LINE_SIZE - 68];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_handshake) == 2 * THAMES_ABI_LINE_SIZE,
+		  "handshake must be two coherence lines");
+
+/*
+ * The firmware invalidates the core's TLB before a Job whose entry's
+ * mapping_generation differs from the one it last honoured.  The page tables
+ * take io-pgtable-arm's attribute indexes for ARM_64_LPAE_S1: 0 non-cacheable,
+ * 1 cacheable, 2 device.
+ */
+struct thames_context_entry {
+	__u64 page_table_base;
+	__u64 mapping_generation;
+	__u32 asid;
+	__u8 reserved[THAMES_ABI_LINE_SIZE - 20];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_context_entry) == THAMES_ABI_LINE_SIZE,
+		  "context entry must be one coherence line");
+
+/*
+ * SPRUIL1D 10.4.3.3; physical addresses: the DRU does not translate through
+ * the core's MMU.
+ */
+struct thames_dru_record {
+	__u32 words[16];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_dru_record) == 64, "a DRU record is 64 bytes");
+
+/*
+ * The groups are an array of __u16, each entry the number of transfers in its
+ * group, the groups consecutive in the table's order.
+ */
+#define THAMES_TRANSFER_TABLE_CHANNELS_OFFSET(transfer_count)	((transfer_count) * 64u)
+#define THAMES_TRANSFER_TABLE_GROUPS_OFFSET(transfer_count) \
+	((transfer_count) * 64u + (((transfer_count) + 7u) & ~7u))
+#define THAMES_TRANSFER_TABLE_SIZE(transfer_count, group_count) \
+	((THAMES_TRANSFER_TABLE_GROUPS_OFFSET(transfer_count) + (group_count) * 2u + \
+	  THAMES_ABI_LINE_SIZE - 1u) & ~(THAMES_ABI_LINE_SIZE - 1u))
+
+/*
+ * @context_serial: a change makes the firmware clear what the last Context left
+ *	in the core.
+ * @timeout_ms: 0 is no device timeout.
+ */
+struct thames_ring_host {
+	__u32 context_id;
+	__u32 transfer_count;
+	__u64 seqno;
+	__u64 program_dsp_va;
+	__u64 params_dsp_va;
+	__u64 context_serial;
+	__u32 transfer_table_offset;
+	__u32 transfer_group_count;
+	__u32 timeout_ms;
+	__u8 reserved[THAMES_ABI_LINE_SIZE - 52];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_ring_host) == THAMES_ABI_LINE_SIZE,
+		  "ring host half must be one coherence line");
+
+/*
+ * Stamps in ticks of the handshake's timestamp_hz; zero when no Kernel ran, and
+ * the Kernel's end stamp also when it faulted or timed out.
+ */
+struct thames_ring_fw {
+	__u64 doorbell_seen_stamp;
+	__u64 run_start_stamp;
+	__u64 run_end_stamp;
+	__u64 fence_value;
+	__u64 kernel_entry_stamp;
+	__u64 kernel_exit_stamp;
+	__u8 reserved[THAMES_ABI_LINE_SIZE - 48];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_ring_fw) == THAMES_ABI_LINE_SIZE,
+		  "ring firmware half must be one coherence line");
+
+struct thames_ring_entry {
+	struct thames_ring_host host;
+	struct thames_ring_fw fw;
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_ring_entry) ==
+		  sizeof(struct thames_ring_host) + THAMES_ABI_LINE_SIZE, "");
+
+struct thames_ring {
+	__u32 tail;
+	__u32 pad;
+	__u64 challenge;
+	__u8 tail_pad[THAMES_ABI_LINE_SIZE - 16];
+	__u32 head;
+	__u8 head_pad[THAMES_ABI_LINE_SIZE - sizeof(__u32)];
+	struct thames_ring_entry entries[THAMES_RING_DEPTH];
+};
+
+struct thames_fence {
+	__u64 seqno;
+	__u8 reserved[THAMES_ABI_LINE_SIZE - 8];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_fence) == THAMES_ABI_LINE_SIZE,
+		  "fence must be one coherence line");
+
+/* The fence_value of a failed or refused Job; the Fence still advances past it. */
+#define THAMES_FENCE_ERROR	(~(__u64)0)
+
+/* @seqno tells a record from one an earlier Job left in the same Ring slot. */
+struct thames_fault_record {
+	__u32 kind;
+	__u32 pad;
+	__u64 seqno;
+	__u64 registers[THAMES_FAULT_REGISTERS];
+	__u8 reserved[THAMES_ABI_LINE_SIZE - 88];
+};
+THAMES_ABI_ASSERT(sizeof(struct thames_fault_record) == THAMES_ABI_LINE_SIZE,
+		  "fault record must be one coherence line");
+
+/*
+ * The handshake's transfer_table_pool_size bytes follow the fixed part.  The
+ * IPC memory is mapped in no Context, so a Kernel cannot change a transfer
+ * record after the kernel driver checked it.
+ */
+struct thames_ipc {
+	struct thames_handshake handshake;
+	struct thames_context_entry context_table[THAMES_MAX_CONTEXTS];
+	struct thames_ring ring;
+	struct thames_fence fence;
+	struct thames_fault_record fault_records[THAMES_RING_DEPTH];
+	__u8 transfer_table_pool[];
+};
+
+#endif /* __THAMES_ABI_H__ */

-- 
2.54.0

[PATCH v3 05/16] accel/thames: Add driver for the C7x DSPs in TI SoCs

[Tomeu Vizoso]

Some SoCs from Texas Instruments contain DSPs that can be used for
general compute tasks. This driver initially supports the C7x cores in
the J722S SoC.

It provides a DRM accel uAPI to userspace for submitting jobs to the DSP
cores and managing the input, output and intermediate memory.

This commit attaches to each DSP core that remoteproc has booted, boots
a core left offline, performs a handshake to make sure that the
firmware is recognized and reads its capabilities.

Communication with the firmware is made via shared memory and a mailbox
doorbell, and an unrecognized firmware at handshake is left alone in
case the user wants to run a different firmware on that core.

Suspend and resume is supported, by powering each core off and on resp.
and redoing the handshake.

The DSP nodes are bound by the K3 DSP remoteproc driver, so this driver
cannot bind to them. It registers one platform device of its own for
the accelerator, finds the DSP nodes by their compatible, attaches to
each core's remoteproc and links its device to each core's, so the
accelerator suspends before the cores and unbinds when they do. One
DRM device serves both cores because a Job may run on either.

The handshake is a block the firmware writes at boot: its ABI version,
the IPC address it was linked for, the L2 size, the DRU's channel count,
queue depth, channel stride, first Kernel channel and capabilities, the
ASID width, the hardware ID it was built for, its timestamp rate and
the transfer table pool's size. The driver refuses a core whose ABI
major differs, whose IPC address is not the device tree's, or whose
hardware ID is not the SoC k3-socinfo identified, and leaves it running.
To tell this boot's handshake from one a previous firmware left in the
memory, the kernel writes a random challenge into the Ring and waits up
to 5 s for the firmware to echo it.

The driver holds the power domains of each core's two timers, which the
firmware drives with no power request of its own, and links its device
to each core's so that it suspends before the DSP and unbinds first.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Move to shared memory IPC.
- Attach to a core remoteproc has booted instead of booting it.
- Discover caps and let userspace query them.
- Add the uAPI header here, moved from the BO patch.
- Group and sort the includes, end the Makefile with a newline (Jani
  Nikula); the driver's own headers stay quoted, as every accel driver
  does.
- Every /** block is kernel-doc or a plain comment (Randy Dunlap).
- Depend on TI_K3_DSP_REMOTEPROC with no COMPILE_TEST (Andrew Davis).
---
 MAINTAINERS                          |   8 +
 drivers/accel/Kconfig                |   1 +
 drivers/accel/Makefile               |   3 +-
 drivers/accel/thames/Kconfig         |  23 ++
 drivers/accel/thames/Makefile        |   7 +
 drivers/accel/thames/thames_core.c   | 624 +++++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_core.h   |  87 +++++
 drivers/accel/thames/thames_device.h |  58 ++++
 drivers/accel/thames/thames_drv.c    | 360 ++++++++++++++++++++
 drivers/accel/thames/thames_drv.h    |  32 ++
 include/uapi/drm/thames_accel.h      | 200 +++++++++++
 11 files changed, 1402 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index a08087fb6e67..c63ab2d77b01 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8019,6 +8019,14 @@ F:	Documentation/devicetree/bindings/npu/rockchip,rk3588-rknn-core.yaml
 F:	drivers/accel/rocket/
 F:	include/uapi/drm/rocket_accel.h
 
+DRM ACCEL DRIVER FOR TI C7x DSPS
+M:	Tomeu Vizoso <tomeu@tomeuvizoso.net>
+L:	dri-devel@lists.freedesktop.org
+S:	Supported
+T:	git https://gitlab.freedesktop.org/drm/misc/kernel.git
+F:	drivers/accel/thames/
+F:	include/uapi/drm/thames_accel.h
+
 DRM COMPUTE ACCELERATORS DRIVERS AND FRAMEWORK
 M:	Oded Gabbay <ogabbay@kernel.org>
 L:	dri-devel@lists.freedesktop.org
diff --git a/drivers/accel/Kconfig b/drivers/accel/Kconfig
index bdf48ccafcf2..cb49c71cd4e4 100644
--- a/drivers/accel/Kconfig
+++ b/drivers/accel/Kconfig
@@ -30,5 +30,6 @@ source "drivers/accel/habanalabs/Kconfig"
 source "drivers/accel/ivpu/Kconfig"
 source "drivers/accel/qaic/Kconfig"
 source "drivers/accel/rocket/Kconfig"
+source "drivers/accel/thames/Kconfig"
 
 endif
diff --git a/drivers/accel/Makefile b/drivers/accel/Makefile
index 1d3a7251b950..6e1db1b381f5 100644
--- a/drivers/accel/Makefile
+++ b/drivers/accel/Makefile
@@ -5,4 +5,5 @@ obj-$(CONFIG_DRM_ACCEL_ARM_ETHOSU)	+= ethosu/
 obj-$(CONFIG_DRM_ACCEL_HABANALABS)	+= habanalabs/
 obj-$(CONFIG_DRM_ACCEL_IVPU)		+= ivpu/
 obj-$(CONFIG_DRM_ACCEL_QAIC)		+= qaic/
-obj-$(CONFIG_DRM_ACCEL_ROCKET)		+= rocket/
\ No newline at end of file
+obj-$(CONFIG_DRM_ACCEL_ROCKET)		+= rocket/
+obj-$(CONFIG_DRM_ACCEL_THAMES)		+= thames/
diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
new file mode 100644
index 000000000000..c621002d8efa
--- /dev/null
+++ b/drivers/accel/thames/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config DRM_ACCEL_THAMES
+	tristate "Thames (support for TI C7x DSP accelerators)"
+	depends on DRM_ACCEL
+	depends on TI_K3_DSP_REMOTEPROC
+	depends on REMOTEPROC
+	depends on MAILBOX
+	help
+	  Choose this option if you have a Texas Instruments J722S and want to
+	  use its C7x DSP cores as compute accelerators.
+
+	  The C7x DSP cores can be used for general-purpose compute acceleration
+	  and are exposed through the DRM accel subsystem.
+
+	  The interface exposed to userspace is described in
+	  include/uapi/drm/thames_accel.h and is used by the Thames userspace
+	  driver in Mesa3D.
+
+	  If unsure, say N.
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called thames.
diff --git a/drivers/accel/thames/Makefile b/drivers/accel/thames/Makefile
new file mode 100644
index 000000000000..175247b31cb2
--- /dev/null
+++ b/drivers/accel/thames/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+obj-$(CONFIG_DRM_ACCEL_THAMES) := thames.o
+
+thames-y := \
+	thames_core.o \
+	thames_drv.o
diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
new file mode 100644
index 000000000000..d74c74b8f8eb
--- /dev/null
+++ b/drivers/accel/thames/thames_core.c
@@ -0,0 +1,624 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#include <linux/dev_printk.h>
+#include <linux/device.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/ktime.h>
+#include <linux/mailbox_client.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_runtime.h>
+#include <linux/random.h>
+#include <linux/remoteproc.h>
+#include <linux/sizes.h>
+#include <linux/string.h>
+#include <linux/sys_soc.h>
+
+#include <drm/thames_accel.h>
+
+#include "thames_abi.h"
+#include "thames_core.h"
+#include "thames_device.h"
+
+#define THAMES_RESTART_TIMEOUT_MS	5000
+
+/*
+ * The IPC memory is the core's first memory-region, which remoteproc allocates
+ * from only for virtio rings and carveouts this firmware does not have.
+ */
+#define THAMES_IPC_MEMORY_REGION	0
+
+static_assert(THAMES_HW_J722S_C7524 == DRM_THAMES_HW_J722S_C7524);
+
+static const struct soc_device_attribute thames_socs[] = {
+	{ .family = "J722S", .data = (void *)(uintptr_t)THAMES_HW_J722S_C7524 },
+	{ }
+};
+
+static int thames_core_read_window(struct thames_core *core, const char *name,
+				   struct resource *window)
+{
+	int index, err;
+
+	index = of_property_match_string(core->of_node, "reg-names", name);
+	if (index < 0) {
+		dev_err(core->dev, "Missing '%s' reg entry on the DSP node\n", name);
+		return index;
+	}
+
+	err = of_address_to_resource(core->of_node, index, window);
+	if (err) {
+		dev_err(core->dev, "Failed to read the '%s' window: %d\n", name, err);
+		return err;
+	}
+
+	return 0;
+}
+
+static int thames_core_read_windows(struct thames_core *core)
+{
+	int err;
+
+	err = thames_core_read_window(core, "l2sram", &core->l2sram);
+	if (err)
+		return err;
+
+	err = thames_core_read_window(core, "l2aux", &core->l2aux);
+	if (err)
+		return err;
+
+	return thames_core_read_window(core, "dru", &core->dru);
+}
+
+/*
+ * Copied once, so a firmware rewriting its handshake cannot change a value
+ * after the check.
+ */
+static int thames_core_read_caps(struct thames_core *core, struct thames_handshake *handshake,
+				 struct thames_caps *caps)
+{
+	struct thames_device *tdev = core->tdev;
+	u64 l2_window = resource_size(&core->l2sram);
+	u64 dru_window = resource_size(&core->dru);
+	u64 dru_kernel_offset, dru_kernel_size;
+	u32 dru_kernel_channel_first;
+	size_t pool_offset = offsetof(struct thames_ipc, transfer_table_pool);
+	int i;
+
+	caps->l2_size = READ_ONCE(handshake->l2_size);
+	caps->dru_channel_count = READ_ONCE(handshake->dru_channel_count);
+	caps->dru_queue_depth = READ_ONCE(handshake->dru_queue_depth);
+	caps->asid_bits = READ_ONCE(handshake->asid_bits);
+	caps->dru_channel_stride = READ_ONCE(handshake->dru_channel_stride);
+	caps->timestamp_hz = READ_ONCE(handshake->timestamp_hz);
+	caps->transfer_table_pool_size = READ_ONCE(handshake->transfer_table_pool_size);
+	caps->dru_capabilities = READ_ONCE(handshake->dru_capabilities);
+	dru_kernel_channel_first = READ_ONCE(handshake->dru_kernel_channel_first);
+	for (i = 0; i < THAMES_NUM_WINDOWS; i++)
+		caps->window_sizes[i] = resource_size(thames_core_window(core, i));
+
+	if (!caps->l2_size || caps->l2_size > l2_window) {
+		dev_err(core->dev, "Firmware L2 size 0x%x does not fit the 0x%llx-byte L2 window\n",
+			caps->l2_size, l2_window);
+		return -EINVAL;
+	}
+
+	if (!caps->dru_channel_count || !caps->dru_queue_depth ||
+	    (u64)caps->dru_channel_count * caps->dru_queue_depth > U16_MAX) {
+		dev_err(core->dev,
+			"Firmware DRU of %u channels of depth %u is empty or exceeds a group's count of %u transfers\n",
+			caps->dru_channel_count, caps->dru_queue_depth, U16_MAX);
+		return -EINVAL;
+	}
+
+	if (!caps->dru_channel_stride || !caps->timestamp_hz) {
+		dev_err(core->dev, "Firmware DRU channel stride %u or timestamp clock %u Hz is zero\n",
+			caps->dru_channel_stride, caps->timestamp_hz);
+		return -EINVAL;
+	}
+
+	if (!caps->transfer_table_pool_size ||
+	    !IS_ALIGNED(caps->transfer_table_pool_size, THAMES_ABI_LINE_SIZE) ||
+	    pool_offset + caps->transfer_table_pool_size > core->ipc_size) {
+		dev_err(core->dev,
+			"Firmware transfer table pool of 0x%x bytes at 0x%zx is empty, not whole lines, or past the 0x%zx-byte IPC memory\n",
+			caps->transfer_table_pool_size, pool_offset, core->ipc_size);
+		return -EINVAL;
+	}
+
+	dru_kernel_offset = (u64)dru_kernel_channel_first * caps->dru_channel_stride;
+	dru_kernel_size = (u64)caps->dru_channel_count * caps->dru_channel_stride;
+	if (caps->dru_channel_count > THAMES_MAX_DRU_CHANNELS ||
+	    !IS_ALIGNED(dru_kernel_offset, SZ_4K) || dru_kernel_size > SZ_4K ||
+	    dru_kernel_offset + SZ_4K > dru_window) {
+		dev_err(core->dev,
+			"Firmware DRU of %u channels from channel %u, 0x%x bytes apart, is not one page of its own in the 0x%llx-byte DRU window, or exceeds the driver's %d channels\n",
+			caps->dru_channel_count, dru_kernel_channel_first,
+			caps->dru_channel_stride, dru_window, THAMES_MAX_DRU_CHANNELS);
+		return -EINVAL;
+	}
+	caps->window_sizes[THAMES_WINDOW_DRU] = round_up(dru_kernel_size, SZ_4K);
+
+	if (!caps->asid_bits || caps->asid_bits > 16) {
+		dev_err(core->dev, "Firmware ASID width %u is not 1 to 16 bits\n",
+			caps->asid_bits);
+		return -EINVAL;
+	}
+
+	if (tdev->num_cores && memcmp(caps, &tdev->caps, sizeof(*caps))) {
+		dev_err(core->dev, "Capabilities or window sizes differ from the first core's\n");
+		return -EINVAL;
+	}
+
+	core->dru_kernel_channel_first = dru_kernel_channel_first;
+
+	return 0;
+}
+
+/*
+ * remoteproc never frees this memory, so it outlives every stop and start of
+ * the core and every reload of the driver.
+ */
+static int thames_core_map_ipc(struct thames_core *core)
+{
+	struct resource res;
+	int err;
+
+	err = of_reserved_mem_region_to_resource(core->of_node, THAMES_IPC_MEMORY_REGION, &res);
+	if (err) {
+		dev_err(core->dev, "No shared block as memory-region %d of %pOF: %d\n",
+			THAMES_IPC_MEMORY_REGION, core->of_node, err);
+		return err;
+	}
+
+	if (resource_size(&res) < sizeof(struct thames_ipc)) {
+		dev_err(core->dev, "The shared block %pR is smaller than the layout's 0x%zx bytes\n",
+			&res, sizeof(struct thames_ipc));
+		return -EINVAL;
+	}
+
+	core->ipc = memremap(res.start, resource_size(&res), MEMREMAP_WC);
+	if (!core->ipc) {
+		dev_err(core->dev, "Failed to map the shared block %pR\n", &res);
+		return -ENOMEM;
+	}
+
+	core->ipc_address = res.start;
+	core->ipc_size = resource_size(&res);
+
+	return 0;
+}
+
+static void thames_core_unmap_ipc(struct thames_core *core)
+{
+	memunmap(core->ipc);
+	core->ipc = NULL;
+}
+
+static int thames_core_check_abi(struct thames_core *core, u32 abi)
+{
+	if (THAMES_ABI_VERSION_MAJOR(abi) == THAMES_ABI_MAJOR)
+		return 0;
+
+	dev_err(core->dev, THAMES_CORE_FMT
+		" runs firmware of ABI %u.%u.%u, the driver's is %u.%u.%u: majors differ; refusing the core, left running\n",
+		THAMES_CORE_ARG(core), THAMES_ABI_VERSION_MAJOR(abi),
+		THAMES_ABI_VERSION_MINOR(abi), THAMES_ABI_VERSION_PATCH(abi),
+		THAMES_ABI_MAJOR, THAMES_ABI_MINOR, THAMES_ABI_PATCH);
+	return -ENODEV;
+}
+
+static u64 thames_core_challenge_echo(struct thames_core *core)
+{
+	return READ_ONCE(core->ipc->handshake.challenge_echo);
+}
+
+/* The IPC memory may still hold the handshake of a firmware before this boot. */
+static int thames_core_wait_handshake(struct thames_core *core)
+{
+	u64 challenge, found, echo;
+	ktime_t start;
+	u32 abi;
+	int err;
+
+	do {
+		challenge = get_random_u64();
+	} while (!challenge);
+
+	found = thames_core_challenge_echo(core);
+	WRITE_ONCE(core->ipc->ring.challenge, challenge);
+	/* Order the challenge before the doorbell. */
+	wmb();
+
+	/*
+	 * A doorbell rung before the firmware listens is lost; its loop reads
+	 * the challenge on its first pass all the same.
+	 */
+	mbox_send_message(core->mbox_chan, (void *)(uintptr_t)THAMES_DOORBELL);
+
+	start = ktime_get();
+	err = read_poll_timeout(thames_core_challenge_echo, echo, echo == challenge,
+				10 * USEC_PER_MSEC, THAMES_RESTART_TIMEOUT_MS * USEC_PER_MSEC,
+				false, core);
+	dev_dbg(core->dev, THAMES_CORE_FMT
+		" challenge %#llx, echo %#llx before the wait, %#llx after %lld ms\n",
+		THAMES_CORE_ARG(core), challenge, found, echo, ktime_ms_delta(ktime_get(), start));
+	if (err) {
+		abi = READ_ONCE(core->ipc->handshake.abi_version);
+		if (abi && thames_core_check_abi(core, abi))
+			return -ENODEV;
+
+		dev_err(core->dev, THAMES_CORE_FMT
+			" did not answer the challenge within %u ms; refusing the core, left running\n",
+			THAMES_CORE_ARG(core), THAMES_RESTART_TIMEOUT_MS);
+		return err;
+	}
+
+	/* Read the handshake after the echo that says it is written. */
+	rmb();
+	return 0;
+}
+
+static int thames_core_read_handshake(struct thames_core *core, struct thames_caps *caps)
+{
+	struct thames_handshake *handshake = &core->ipc->handshake;
+	const struct soc_device_attribute *soc;
+	u64 ipc_address;
+	int err;
+
+	err = thames_core_check_abi(core, READ_ONCE(handshake->abi_version));
+	if (err)
+		return err;
+
+	ipc_address = READ_ONCE(handshake->ipc_address);
+	if (ipc_address != core->ipc_address) {
+		dev_err(core->dev, THAMES_CORE_FMT
+			" runs firmware linked for a shared block at %#llx, the device tree reserves it at %pa; refusing the core, left running\n",
+			THAMES_CORE_ARG(core), ipc_address, &core->ipc_address);
+		return -ENODEV;
+	}
+
+	caps->hardware_id = READ_ONCE(handshake->hardware_id);
+	soc = soc_device_match(thames_socs);
+	if (!soc) {
+		dev_err(core->dev, THAMES_CORE_FMT
+			" runs firmware for hardware ID %u, and k3-socinfo identified no SoC the driver knows; refusing the core, left running\n",
+			THAMES_CORE_ARG(core), caps->hardware_id);
+		return -ENODEV;
+	}
+
+	if (caps->hardware_id != (uintptr_t)soc->data) {
+		dev_err(core->dev, THAMES_CORE_FMT
+			" runs firmware for hardware ID %u, the %s SoC's is %u; refusing the core, left running\n",
+			THAMES_CORE_ARG(core), caps->hardware_id, soc->family,
+			(u32)(uintptr_t)soc->data);
+		return -ENODEV;
+	}
+
+	return thames_core_read_caps(core, handshake, caps);
+}
+
+static void thames_mbox_rx(struct mbox_client *cl, void *mssg)
+{
+	struct thames_core *core = container_of(cl, struct thames_core, mbox_client);
+
+	dev_dbg(core->dev, "Doorbell on " THAMES_CORE_FMT ": 0x%x\n", THAMES_CORE_ARG(core),
+		(u32)(uintptr_t)mssg);
+}
+
+/* Embedded in struct thames_core: nothing to free. */
+static void thames_core_mbox_dev_release(struct device *dev)
+{
+}
+
+/*
+ * A device of the driver's own carrying the DSP node, so the mailbox core finds
+ * the node's mboxes without borrowing the K3 driver's device.
+ */
+static int thames_core_request_doorbell(struct thames_core *core)
+{
+	struct mbox_client *cl = &core->mbox_client;
+	struct device *mbox_dev = &core->mbox_dev;
+	int err;
+
+	device_initialize(mbox_dev);
+	mbox_dev->parent = core->tdev->ddev.dev;
+	mbox_dev->release = thames_core_mbox_dev_release;
+	device_set_node(mbox_dev, of_fwnode_handle(core->of_node));
+	dev_set_name(mbox_dev, "thames-mbox.%u", core->index);
+
+	err = device_add(mbox_dev);
+	if (err) {
+		put_device(mbox_dev);
+		return err;
+	}
+
+	cl->dev = mbox_dev;
+	cl->rx_callback = thames_mbox_rx;
+	cl->tx_block = false;
+	cl->knows_txdone = false;
+
+	core->mbox_chan = mbox_request_channel(cl, 1);
+	if (IS_ERR(core->mbox_chan)) {
+		err = PTR_ERR(core->mbox_chan);
+		core->mbox_chan = NULL;
+		device_del(mbox_dev);
+		put_device(mbox_dev);
+		return err;
+	}
+
+	return 0;
+}
+
+static void thames_core_free_doorbell(struct thames_core *core)
+{
+	mbox_free_channel(core->mbox_chan);
+	core->mbox_chan = NULL;
+	device_del(&core->mbox_dev);
+	put_device(&core->mbox_dev);
+}
+
+static int thames_core_rproc_boot(struct thames_core *core)
+{
+	int err;
+
+	WRITE_ONCE(core->rproc_caller, current);
+	err = rproc_boot(core->rproc);
+	WRITE_ONCE(core->rproc_caller, NULL);
+
+	return err;
+}
+
+static int thames_core_rproc_shutdown(struct thames_core *core)
+{
+	int err;
+
+	WRITE_ONCE(core->rproc_caller, current);
+	err = rproc_shutdown(core->rproc);
+	WRITE_ONCE(core->rproc_caller, NULL);
+
+	return err;
+}
+
+/*
+ * The firmware drives these timers with no power request and the device tree
+ * reserves them for it, so the driver holds their power domains on.
+ */
+static int thames_core_get_timers(struct thames_core *core)
+{
+	const struct dev_pm_domain_attach_data data = {
+		.pd_flags = PD_FLAG_NO_DEV_LINK,
+	};
+	struct dev_pm_domain_list *pds = NULL;
+	int err, i;
+
+	err = dev_pm_domain_attach_list(core->dev, &data, &pds);
+	if (err < 0) {
+		if (err != -EPROBE_DEFER)
+			dev_err(core->dev, THAMES_CORE_FMT ": failed to attach the timers' power domains: %d\n",
+				THAMES_CORE_ARG(core), err);
+		return err;
+	}
+	if (!pds)
+		return 0;
+
+	for (i = 0; i < pds->num_pds; i++) {
+		err = pm_runtime_resume_and_get(pds->pd_devs[i]);
+		if (err) {
+			dev_err(core->dev, THAMES_CORE_FMT ": failed to power the timers' domain %s: %d\n",
+				THAMES_CORE_ARG(core), dev_name(pds->pd_devs[i]), err);
+			goto err_put;
+		}
+	}
+
+	core->timer_pds = pds;
+	return 0;
+
+err_put:
+	while (i--)
+		pm_runtime_put(pds->pd_devs[i]);
+	dev_pm_domain_detach_list(pds);
+	return err;
+}
+
+static void thames_core_put_timers(struct thames_core *core)
+{
+	struct dev_pm_domain_list *pds = core->timer_pds;
+	int i;
+
+	if (!pds)
+		return;
+
+	for (i = 0; i < pds->num_pds; i++)
+		pm_runtime_put(pds->pd_devs[i]);
+	dev_pm_domain_detach_list(pds);
+	core->timer_pds = NULL;
+}
+
+int thames_core_init(struct thames_core *core)
+{
+	struct thames_caps caps = {};
+	struct rproc *rproc;
+	int err;
+
+	/* Without a phandle the lookup would defer forever. */
+	if (!core->of_node->phandle) {
+		dev_err(core->tdev->ddev.dev, "%pOF has no phandle to find its remoteproc by\n",
+			core->of_node);
+		return -EINVAL;
+	}
+
+	rproc = rproc_get_by_phandle(core->of_node->phandle);
+	if (!rproc)
+		return dev_err_probe(core->tdev->ddev.dev, -EPROBE_DEFER,
+				     "%pOF: no remoteproc registered for it yet\n", core->of_node);
+
+	/*
+	 * remoteproc's own boot is still to come: booting the core here as well
+	 * would leave two power counts, and a reset or a suspend could not stop
+	 * it.
+	 */
+	if (rproc->state == RPROC_OFFLINE && rproc->auto_boot) {
+		rproc_put(rproc);
+		return dev_err_probe(core->tdev->ddev.dev, -EPROBE_DEFER,
+				     "%pOF: remoteproc has not booted it yet (auto-boot pending)\n",
+				     core->of_node);
+	}
+
+	core->rproc = rproc;
+	core->dev = rproc->dev.parent;
+	snprintf(core->name, sizeof(core->name), "DSP core %s (index %u)", dev_name(core->dev),
+		 core->index);
+
+	/*
+	 * The PM core then suspends this driver before the DSP, and an unbind
+	 * of the K3 driver unbinds this one first.
+	 */
+	if (!device_link_add(core->tdev->ddev.dev, core->dev, DL_FLAG_AUTOPROBE_CONSUMER)) {
+		dev_err(core->tdev->ddev.dev, "Failed to link to %s\n", dev_name(core->dev));
+		err = -EINVAL;
+		goto err_put;
+	}
+
+	err = thames_core_read_windows(core);
+	if (err)
+		goto err_put;
+
+	err = thames_core_get_timers(core);
+	if (err)
+		goto err_put;
+
+	err = thames_core_map_ipc(core);
+	if (err)
+		goto err_timers;
+
+	err = thames_core_request_doorbell(core);
+	if (err)
+		goto err_unmap;
+
+	if (rproc->state == RPROC_OFFLINE) {
+		err = rproc_boot(rproc);
+		if (err)
+			goto err_doorbell;
+	}
+
+	err = thames_core_wait_handshake(core);
+	if (err)
+		goto err_doorbell;
+
+	err = thames_core_read_handshake(core, &caps);
+	if (err)
+		goto err_doorbell;
+
+	if (!core->tdev->num_cores)
+		core->tdev->caps = caps;
+
+	return 0;
+
+err_doorbell:
+	thames_core_free_doorbell(core);
+err_unmap:
+	thames_core_unmap_ipc(core);
+err_timers:
+	thames_core_put_timers(core);
+err_put:
+	rproc_put(rproc);
+	core->rproc = NULL;
+	return err;
+}
+
+void thames_core_fini(struct thames_core *core)
+{
+	if (core->mbox_chan)
+		thames_core_free_doorbell(core);
+	/*
+	 * Drop only our reference and keep the boot count: the core runs on
+	 * for the next load to reattach to; only a reset or a suspend, which
+	 * idle it first, stop it.
+	 */
+	rproc_put(core->rproc);
+	core->rproc = NULL;
+	thames_core_unmap_ipc(core);
+	thames_core_put_timers(core);
+	of_node_put(core->of_node);
+	core->of_node = NULL;
+}
+
+/*
+ * rproc_shutdown() reaches the system firmware because the core has one power
+ * count, remoteproc's auto-boot or this driver's boot: the probe defers while
+ * an auto-boot is pending.
+ */
+int thames_core_suspend(struct thames_core *core)
+{
+	struct rproc *rproc = core->rproc;
+	int err;
+
+	if (core->suspended)
+		return 0;
+
+	err = thames_core_rproc_shutdown(core);
+	if (!err && rproc->state != RPROC_OFFLINE) {
+		dev_err(core->dev, THAMES_CORE_FMT
+			" is kept running by another power count; refusing the suspend\n",
+			THAMES_CORE_ARG(core));
+		thames_core_rproc_boot(core);
+		err = -EBUSY;
+	}
+	if (err) {
+		return err;
+	}
+
+	core->suspended = true;
+	return 0;
+}
+
+/*
+ * The firmware empties the Ring at boot and moves its head to the tail it
+ * finds.
+ */
+int thames_core_resume(struct thames_core *core)
+{
+	struct thames_caps caps = {};
+	u32 head, tail;
+	int err = 0;
+
+	if (!core->suspended)
+		goto restart;
+
+	err = thames_core_rproc_boot(core);
+	if (err)
+		goto offline;
+
+	core->suspended = false;
+
+	err = thames_core_wait_handshake(core);
+	if (err)
+		goto offline;
+
+	err = thames_core_read_handshake(core, &caps);
+	if (err)
+		goto offline;
+
+	head = READ_ONCE(core->ipc->ring.head);
+	tail = READ_ONCE(core->ipc->ring.tail);
+	if (head != tail) {
+		dev_err(core->dev, "Ring on " THAMES_CORE_FMT
+			" is not empty after its boot (tail %u, head %u)\n",
+			THAMES_CORE_ARG(core), tail, head);
+		err = -EIO;
+		goto offline;
+	}
+	core->ring_tail = head;
+
+	goto restart;
+
+offline:
+restart:
+	return err;
+}
diff --git a/drivers/accel/thames/thames_core.h b/drivers/accel/thames/thames_core.h
new file mode 100644
index 000000000000..590c5716f7f5
--- /dev/null
+++ b/drivers/accel/thames/thames_core.h
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_CORE_H__
+#define __THAMES_CORE_H__
+
+#include <linux/ioport.h>
+#include <linux/list.h>
+#include <linux/mailbox_client.h>
+#include <linux/mutex_types.h>
+#include <linux/remoteproc.h>
+
+
+#include "thames_abi.h"
+
+enum thames_window {
+	THAMES_WINDOW_L2SRAM,
+	THAMES_WINDOW_L2AUX,
+	THAMES_WINDOW_DRU,
+	THAMES_NUM_WINDOWS,
+};
+
+#define THAMES_MAX_DRU_CHANNELS	16
+
+#define THAMES_DOORBELL		1u
+
+struct thames_core {
+	struct device_node *of_node;
+	struct rproc *rproc;
+	struct device *dev;
+	struct thames_device *tdev;
+	unsigned int index;
+	char name[64];
+
+	struct thames_ipc *ipc;
+	phys_addr_t ipc_address;
+	size_t ipc_size;
+	u32 dru_kernel_channel_first;
+
+	struct dev_pm_domain_list *timer_pds;
+
+	struct device mbox_dev;
+	struct mbox_client mbox_client;
+	struct mbox_chan *mbox_chan;
+
+	u32 ring_tail;
+	/*
+	 * @rproc_caller is the task inside the driver's own rproc_boot() or
+	 * rproc_shutdown(), whose subdevice callbacks then do nothing.
+	 */
+	struct task_struct *rproc_caller;
+
+	bool suspended;
+
+	struct resource l2sram;
+	struct resource l2aux;
+	struct resource dru;
+};
+
+static inline const struct resource *thames_core_window(const struct thames_core *core,
+							enum thames_window window)
+{
+	switch (window) {
+	case THAMES_WINDOW_L2SRAM:
+		return &core->l2sram;
+	case THAMES_WINDOW_L2AUX:
+		return &core->l2aux;
+	case THAMES_WINDOW_DRU:
+		return &core->dru;
+	default:
+		return NULL;
+	}
+}
+
+/*
+ * The index counts attached cores only, so a core left out shifts the ones
+ * after it.
+ */
+#define THAMES_CORE_FMT "%s"
+#define THAMES_CORE_ARG(core) ((core)->name)
+
+int thames_core_init(struct thames_core *core);
+void thames_core_fini(struct thames_core *core);
+int thames_core_suspend(struct thames_core *core);
+int thames_core_resume(struct thames_core *core);
+
+#endif
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
new file mode 100644
index 000000000000..f1cf79f2c2e4
--- /dev/null
+++ b/drivers/accel/thames/thames_device.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_DEVICE_H__
+#define __THAMES_DEVICE_H__
+
+#include <linux/atomic.h>
+#include <linux/bitmap.h>
+#include <linux/clk.h>
+#include <linux/container_of.h>
+#include <linux/idr.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_device.h>
+
+#include "thames_core.h"
+
+/* No holes: two cores' caps compare with memcmp(). */
+struct thames_caps {
+	u32 l2_size;
+	u32 dru_channel_count;
+	u32 dru_queue_depth;
+	u32 asid_bits;
+	u32 hardware_id;
+	u32 dru_channel_stride;
+	u32 timestamp_hz;
+	u32 transfer_table_pool_size;
+	u64 dru_capabilities;
+	u64 window_sizes[THAMES_NUM_WINDOWS];
+};
+
+#define THAMES_MAX_CORES	2
+
+struct thames_device {
+	struct drm_device ddev;
+
+	struct thames_core cores[THAMES_MAX_CORES];
+	unsigned int num_cores;
+
+	struct thames_caps caps;
+
+};
+
+#define to_thames_device(drm_dev) \
+	((struct thames_device *)(container_of((drm_dev), struct thames_device, ddev)))
+
+/*
+ * Each core's page tables map that core's window at the first core's physical
+ * address, so a Kernel names one address on either core.
+ */
+static inline u64 thames_window_va(const struct thames_device *tdev, enum thames_window i)
+{
+	return thames_core_window(&tdev->cores[0], i)->start;
+}
+
+#endif /* __THAMES_DEVICE_H__ */
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
new file mode 100644
index 000000000000..de93bc87756c
--- /dev/null
+++ b/drivers/accel/thames/thames_drv.c
@@ -0,0 +1,360 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/idr.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/timekeeping.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+
+#include <drm/drm_accel.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_managed.h>
+#include <drm/thames_accel.h>
+
+#include "thames_core.h"
+#include "thames_drv.h"
+
+static struct platform_device *drm_dev;
+static struct thames_device *tdev;
+
+static const struct of_device_id thames_dsp_of_match[] = {
+	{ .compatible = "ti,am62a-c7xv-dsp" },
+	{}
+};
+
+static int thames_open(struct drm_device *dev, struct drm_file *file)
+{
+	struct thames_device *tdev = to_thames_device(dev);
+	struct thames_file_priv *thames_priv;
+	int ret, idx;
+
+	if (!drm_dev_enter(dev, &idx))
+		return -ENODEV;
+
+	thames_priv = kzalloc_obj(*thames_priv, GFP_KERNEL);
+	if (!thames_priv) {
+		ret = -ENOMEM;
+		goto err_exit;
+	}
+
+	thames_priv->tdev = tdev;
+	kref_init(&thames_priv->refcount);
+
+	file->driver_priv = thames_priv;
+
+	drm_dev_exit(idx);
+	return 0;
+
+err_exit:
+	drm_dev_exit(idx);
+	return ret;
+}
+
+static void thames_file_priv_release(struct kref *kref)
+{
+	struct thames_file_priv *priv = container_of(kref, struct thames_file_priv, refcount);
+
+	kfree(priv);
+}
+
+void thames_file_priv_put(struct thames_file_priv *priv)
+{
+	kref_put(&priv->refcount, thames_file_priv_release);
+}
+
+/* Jobs already on a Ring keep the Context until the Fence passes them. */
+static void thames_postclose(struct drm_device *dev, struct drm_file *file)
+{
+	struct thames_file_priv *thames_priv = file->driver_priv;
+
+	thames_file_priv_put(thames_priv);
+}
+
+#define THAMES_DRIVER_MAJOR 1
+#define THAMES_DRIVER_MINOR 0
+
+static int thames_query_copy_out(struct drm_thames_dev_query *args, const void *data,
+				 size_t size)
+{
+	u32 copied = min_t(size_t, args->size, size);
+
+	if (copy_to_user(u64_to_user_ptr(args->pointer), data, copied))
+		return -EFAULT;
+
+	args->size = copied;
+
+	return 0;
+}
+
+static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query *args)
+{
+	struct thames_device *tdev = to_thames_device(dev);
+	const struct thames_caps *reported = &tdev->caps;
+	struct drm_thames_caps caps = {
+		.l2_size = reported->l2_size,
+		.dru_channel_count = reported->dru_channel_count,
+		.dru_queue_depth = reported->dru_queue_depth,
+		.l2_main_dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2SRAM),
+		.l2_aux.dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2AUX),
+		.l2_aux.size = reported->window_sizes[THAMES_WINDOW_L2AUX],
+		.dru_realtime_dsp_va = thames_window_va(tdev, THAMES_WINDOW_DRU),
+		.user_region.dsp_va = THAMES_USER_REGION_BASE,
+		.user_region.size = THAMES_USER_REGION_SIZE,
+		.hardware_id = reported->hardware_id,
+		.dru_capabilities = reported->dru_capabilities,
+	};
+
+	return thames_query_copy_out(args, &caps, sizeof(caps));
+}
+
+static int thames_ioctl_dev_query(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_thames_dev_query *args = data;
+
+	switch (args->type) {
+	case DRM_THAMES_DEV_QUERY_CAPS:
+		if (!args->pointer) {
+			args->size = sizeof(struct drm_thames_caps);
+			return 0;
+		}
+		return thames_query_caps(dev, args);
+	default:
+		return -EINVAL;
+	}
+}
+
+/*
+ * BO_PREP is not entered: it may wait as long as the client asks and touches
+ * only the CPU caches.
+ */
+#define THAMES_IOCTL_ENTERED(func)						\
+static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,	\
+					 struct drm_file *file)			\
+{										\
+	int ret, idx;								\
+										\
+	if (!drm_dev_enter(dev, &idx))						\
+		return -ENODEV;							\
+										\
+	ret = thames_ioctl_##func(dev, data, file);				\
+	drm_dev_exit(idx);							\
+										\
+	return ret;								\
+}
+
+THAMES_IOCTL_ENTERED(dev_query)
+
+static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
+#define THAMES_IOCTL(n, func) DRM_IOCTL_DEF_DRV(THAMES_##n, thames_ioctl_##func, 0)
+	THAMES_IOCTL(DEV_QUERY, dev_query_entered),
+};
+
+DEFINE_DRM_ACCEL_FOPS(thames_accel_driver_fops);
+
+static const struct drm_driver thames_drm_driver = {
+	.driver_features = DRIVER_COMPUTE_ACCEL | DRIVER_GEM | DRIVER_SYNCOBJ |
+			   DRIVER_SYNCOBJ_TIMELINE,
+	.open = thames_open,
+	.postclose = thames_postclose,
+	.ioctls = thames_drm_driver_ioctls,
+	.num_ioctls = ARRAY_SIZE(thames_drm_driver_ioctls),
+	.fops = &thames_accel_driver_fops,
+	.name = "thames",
+	.desc = "thames DRM",
+	.major = THAMES_DRIVER_MAJOR,
+	.minor = THAMES_DRIVER_MINOR,
+};
+
+static struct thames_device *thames_device_init(struct platform_device *pdev,
+						const struct drm_driver *thames_drm_driver)
+{
+	struct device *dev = &pdev->dev;
+	struct thames_device *tdev;
+	int err;
+
+	tdev = devm_drm_dev_alloc(dev, thames_drm_driver, struct thames_device, ddev);
+	if (IS_ERR(tdev))
+		return tdev;
+
+	dev_set_drvdata(dev, tdev);
+
+	dma_set_max_seg_size(dev, UINT_MAX);
+
+	err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(40));
+	if (err)
+		return ERR_PTR(err);
+
+
+	return tdev;
+}
+
+static void thames_device_fini(struct thames_device *tdev)
+{
+	WARN_ON(tdev->num_cores > 0);
+}
+
+static int thames_platform_probe(struct platform_device *pdev)
+{
+	struct device_node *np;
+	unsigned int core = 0;
+	int err;
+
+	tdev = thames_device_init(pdev, &thames_drm_driver);
+	if (IS_ERR(tdev)) {
+		err = PTR_ERR(tdev);
+		tdev = NULL;
+		return err;
+	}
+
+	for_each_matching_node(np, thames_dsp_of_match) {
+		if (!of_device_is_available(np))
+			continue;
+
+		if (core >= THAMES_MAX_CORES) {
+			of_node_put(np);
+			break;
+		}
+
+		tdev->cores[core].tdev = tdev;
+		tdev->cores[core].index = core;
+		tdev->cores[core].of_node = of_node_get(np);
+
+		err = thames_core_init(&tdev->cores[core]);
+		if (err == -EPROBE_DEFER) {
+			of_node_put(np);
+			of_node_put(np);
+			goto err_fini_cores;
+		}
+		if (err) {
+			dev_warn(&pdev->dev, "Leaving out the core at %pOF: %d\n", np, err);
+			of_node_put(np);
+			memset(&tdev->cores[core], 0, sizeof(tdev->cores[core]));
+			continue;
+		}
+
+		tdev->num_cores++;
+		core++;
+	}
+
+	if (!tdev->num_cores) {
+		dev_err(&pdev->dev, "No DSP core attached\n");
+		err = -ENODEV;
+		goto err_fini_dev;
+	}
+
+	err = drm_dev_register(&tdev->ddev, 0);
+	if (err)
+		goto err_fini_cores;
+
+	return 0;
+
+err_fini_cores:
+	while (core--)
+		thames_core_fini(&tdev->cores[core]);
+	tdev->num_cores = 0;
+err_fini_dev:
+	thames_device_fini(tdev);
+	tdev = NULL;
+	return err;
+}
+
+static void thames_platform_remove(struct platform_device *pdev)
+{
+	unsigned int core;
+
+	if (!tdev)
+		return;
+
+	drm_dev_unplug(&tdev->ddev);
+
+	for (core = 0; core < tdev->num_cores; core++)
+		thames_core_fini(&tdev->cores[core]);
+	tdev->num_cores = 0;
+
+	thames_device_fini(tdev);
+	tdev = NULL;
+}
+
+static int thames_pm_suspend(struct device *dev)
+{
+	struct thames_device *tdev = dev_get_drvdata(dev);
+	unsigned int core;
+	int err;
+
+	for (core = 0; core < tdev->num_cores; core++) {
+		err = thames_core_suspend(&tdev->cores[core]);
+		if (err)
+			goto err_resume;
+	}
+
+	return 0;
+
+err_resume:
+	while (core--)
+		thames_core_resume(&tdev->cores[core]);
+	return err;
+}
+
+static int thames_pm_resume(struct device *dev)
+{
+	struct thames_device *tdev = dev_get_drvdata(dev);
+	unsigned int core;
+	int err, ret = 0;
+
+	for (core = 0; core < tdev->num_cores; core++) {
+		err = thames_core_resume(&tdev->cores[core]);
+		if (err && !ret)
+			ret = err;
+	}
+
+	return ret;
+}
+
+static DEFINE_SIMPLE_DEV_PM_OPS(thames_pm_ops, thames_pm_suspend, thames_pm_resume);
+
+static struct platform_driver thames_platform_driver = {
+	.probe = thames_platform_probe,
+	.remove = thames_platform_remove,
+	.driver = {
+		.name = "thames",
+		.pm = pm_sleep_ptr(&thames_pm_ops),
+	},
+};
+
+static int __init thames_register(void)
+{
+	int ret;
+
+	drm_dev = platform_device_register_simple("thames", -1, NULL, 0);
+	if (IS_ERR(drm_dev))
+		return PTR_ERR(drm_dev);
+
+	ret = platform_driver_register(&thames_platform_driver);
+	if (ret)
+		platform_device_unregister(drm_dev);
+
+	return ret;
+}
+
+static void __exit thames_unregister(void)
+{
+	platform_driver_unregister(&thames_platform_driver);
+
+	platform_device_unregister(drm_dev);
+}
+
+module_init(thames_register);
+module_exit(thames_unregister);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("DRM driver for Texas Instruments' C7x accelerator cores");
+MODULE_AUTHOR("Tomeu Vizoso");
diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
new file mode 100644
index 000000000000..2678abc2e2de
--- /dev/null
+++ b/drivers/accel/thames/thames_drv.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_DRV_H__
+#define __THAMES_DRV_H__
+
+#include <linux/io-pgtable.h>
+#include <linux/kref.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+#include <linux/xarray.h>
+
+#include <drm/drm_mm.h>
+
+#include "thames_device.h"
+
+struct drm_gem_dma_object;
+
+struct thames_file_priv {
+	struct thames_device *tdev;
+
+	/*
+	 * A binding holds no reference: a buffer can outlive its file through
+	 * an mmap or an export.
+	 */
+	struct kref refcount;
+
+};
+
+void thames_file_priv_put(struct thames_file_priv *priv);
+
+#endif
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
new file mode 100644
index 000000000000..af427a29b585
--- /dev/null
+++ b/include/uapi/drm/thames_accel.h
@@ -0,0 +1,200 @@
+/* SPDX-License-Identifier: MIT */
+/* Copyright (C) 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+#ifndef _THAMES_DRM_H_
+#define _THAMES_DRM_H_
+
+#include "drm.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/**
+ * DOC: Terms
+ *
+ * Context: one client's address space on the DSP, a page table and the
+ * buffers bound in it; a fault belongs to a Context, isolation is between
+ * Contexts.
+ *
+ * Kernel: the compiled program a Job runs on a DSP core; never the Linux
+ * kernel, which is "the kernel driver" here.
+ *
+ * Job: an object a client creates once and submits any number of times:
+ * one Kernel with its parameters and its transfers.
+ *
+ * Ring: a core's queue of submitted Jobs, one entry each, run in order.
+ *
+ * Fence: a submitted Job's completion as the host sees it, a sequence
+ * number the firmware writes and a sync object on the host.
+ *
+ * DRU: the Data Routing Unit, the DMA engine a Job's transfers program;
+ * it addresses physical memory, not a Context.
+ *
+ * Dispatch: the Job type that runs a Kernel with its transfers; the only
+ * type today.
+ */
+
+/**
+ * DOC: IOCTL IDs
+ *
+ * enum drm_thames_ioctl_id - IOCTL IDs
+ *
+ * Place new ioctls at the end, don't re-order, don't replace or remove entries.
+ *
+ * These IDs are not meant to be used directly. Use the DRM_IOCTL_THAMES_xxx
+ * definitions instead.
+ */
+enum drm_thames_ioctl_id {
+	/** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
+	DRM_THAMES_DEV_QUERY,
+
+};
+
+/**
+ * DOC: IOCTL arguments
+ */
+
+/**
+ * struct drm_thames_range - A range of the Context's address space.
+ */
+struct drm_thames_range {
+	/** @dsp_va: Start of the range. */
+	__u64 dsp_va;
+
+	/** @size: Size of the range in bytes. */
+	__u64 size;
+};
+
+/**
+ * enum drm_thames_dev_query_type - Query type
+ *
+ * Place new types at the end, don't re-order, don't remove or replace.
+ */
+enum drm_thames_dev_query_type {
+	/** @DRM_THAMES_DEV_QUERY_CAPS: The capability query, &drm_thames_caps. */
+	DRM_THAMES_DEV_QUERY_CAPS = 0,
+
+};
+
+/**
+ * enum drm_thames_hardware_id - &drm_thames_caps.hardware_id
+ */
+enum drm_thames_hardware_id {
+	/** @DRM_THAMES_HW_J722S_C7524: The C7524 cores of the J722S. */
+	DRM_THAMES_HW_J722S_C7524 = 1,
+};
+
+/**
+ * struct drm_thames_caps - The capability query
+ *
+ * Each window sits at one address whatever core runs the Job and reaches that
+ * core's own memory.
+ */
+struct drm_thames_caps {
+	/** @l2_size: Size in bytes of each core's L2 SRAM. */
+	__u32 l2_size;
+
+	/** @dru_channel_count: DRU channels a Job's transfers may name, from 0. */
+	__u32 dru_channel_count;
+
+	/**
+	 * @dru_queue_depth: Transfers one channel queues; no channel's
+	 * transfers in one transfer group may exceed it.
+	 */
+	__u32 dru_queue_depth;
+
+	/**
+	 * @max_transfers: The most transfers one Job may carry; 0 when the
+	 * kernel driver accepts no transfers.
+	 */
+	__u32 max_transfers;
+
+	/**
+	 * @l2_main_dsp_va: Where the L2 main aperture starts in a Context's
+	 * address space.  A Kernel is compiled against this address.
+	 */
+	__u64 l2_main_dsp_va;
+
+	/** @l2_aux: Where the L2 auxiliary aperture sits in a Context. */
+	struct drm_thames_range l2_aux;
+
+	/**
+	 * @dru_realtime_dsp_va: Where the DRU's real-time page starts in a
+	 * Context: the software triggers and the status counts of the Kernels'
+	 * channels, the only DRU page a Kernel reaches.  Channel n's registers
+	 * start at n times the channel stride into it, 0x100 on the J722S.
+	 */
+	__u64 dru_realtime_dsp_va;
+
+	/**
+	 * @user_region: Where the kernel driver maps the Context's user stack
+	 * and save page.
+	 */
+	struct drm_thames_range user_region;
+
+	/**
+	 * @hardware_id: The SoC and core the firmware was built for, one of
+	 * &enum drm_thames_hardware_id.  Userspace keys the facts of the
+	 * hardware the query does not carry on it.
+	 */
+	__u32 hardware_id;
+
+	/**
+	 * @transfer_types: The transfer record types the kernel driver accepts,
+	 * bit n for type n; 0 when it accepts none.
+	 */
+	__u32 transfer_types;
+
+	/**
+	 * @dru_capabilities: The DRU's DRU_CAPABILITIES register as the
+	 * firmware read it: the record types and formats the hardware supports.
+	 * Zero when the firmware does not report it.
+	 */
+	__u64 dru_capabilities;
+};
+
+/**
+ * struct drm_thames_dev_query - Arguments passed to DRM_IOCTL_THAMES_DEV_QUERY
+ */
+struct drm_thames_dev_query {
+	/** @type: One of &enum drm_thames_dev_query_type. */
+	__u32 type;
+
+	/**
+	 * @size: Size of the structure at @pointer.
+	 *
+	 * With @pointer NULL the driver writes the structure's size here and
+	 * copies nothing.  Otherwise it copies min(@size, its own size) bytes
+	 * and writes the count copied.
+	 */
+	__u32 size;
+
+	/** @pointer: User pointer to the query type's structure. */
+	__u64 pointer;
+};
+
+/**
+ * DRM_IOCTL_THAMES() - Build a thames IOCTL number
+ * @__access: Access type. Must be R, W or RW.
+ * @__id: One of the DRM_THAMES_xxx id.
+ * @__type: Suffix of the type being passed to the IOCTL.
+ *
+ * Don't use this macro directly, use the DRM_IOCTL_THAMES_xxx
+ * values instead.
+ *
+ * Return: An IOCTL number to be passed to ioctl() from userspace.
+ */
+#define DRM_IOCTL_THAMES(__access, __id, __type) \
+	DRM_IO ## __access(DRM_COMMAND_BASE + DRM_THAMES_ ## __id, \
+			   struct drm_thames_ ## __type)
+
+enum {
+	DRM_IOCTL_THAMES_DEV_QUERY =
+		DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif /* _THAMES_DRM_H_ */

-- 
2.54.0

[PATCH v3 06/16] accel/thames: Create a context per client on device open

[Tomeu Vizoso]

And manage a page table for each, using IOMMU_IO_PGTABLE_LPAE.

The firmware will switch to this page table when executing a user DSP
kernel, in the DSP core's MMU.

Each Context gets an ASID, allocated cyclically so a freed one is not
handed out again at once, and an entry in the IPC Context table with
its page table base and a mapping generation, which the firmware
compares before a Job to invalidate the core's TLB when the mappings
changed. Context serials and generations start from the boot clock,
above every number a previous load of the module handed out, so the
firmware never takes a stale entry for the current one. Every Context
maps the three windows (the L2 main and auxiliary SRAM, and the DRU's
Kernel channels as device memory) and the Kernels' stack and save page
at the fixed user-region addresses. Page tables and the per-Context
pages come from a driver allocator of physically contiguous memory,
2 MiB aligned from 2 MiB up so they can be mapped as blocks. There are
63 Contexts, the table's 64 entries less entry 0.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 drivers/accel/thames/Kconfig         |   1 +
 drivers/accel/thames/Makefile        |   3 +-
 drivers/accel/thames/thames_core.c   |   2 +
 drivers/accel/thames/thames_device.h |  17 ++
 drivers/accel/thames/thames_drv.c    |  51 ++++
 drivers/accel/thames/thames_drv.h    |  18 ++
 drivers/accel/thames/thames_gem.c    | 546 +++++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_gem.h    |  52 ++++
 8 files changed, 689 insertions(+), 1 deletion(-)

diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
index c621002d8efa..9362b9274075 100644
--- a/drivers/accel/thames/Kconfig
+++ b/drivers/accel/thames/Kconfig
@@ -6,6 +6,7 @@ config DRM_ACCEL_THAMES
 	depends on TI_K3_DSP_REMOTEPROC
 	depends on REMOTEPROC
 	depends on MAILBOX
+	select IOMMU_IO_PGTABLE_LPAE
 	help
 	  Choose this option if you have a Texas Instruments J722S and want to
 	  use its C7x DSP cores as compute accelerators.
diff --git a/drivers/accel/thames/Makefile b/drivers/accel/thames/Makefile
index 175247b31cb2..bb81fcd58927 100644
--- a/drivers/accel/thames/Makefile
+++ b/drivers/accel/thames/Makefile
@@ -4,4 +4,5 @@ obj-$(CONFIG_DRM_ACCEL_THAMES) := thames.o
 
 thames-y := \
 	thames_core.o \
-	thames_drv.o
+	thames_drv.o \
+	thames_gem.o
diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
index d74c74b8f8eb..1c983580b62a 100644
--- a/drivers/accel/thames/thames_core.c
+++ b/drivers/accel/thames/thames_core.c
@@ -24,6 +24,7 @@
 #include "thames_abi.h"
 #include "thames_core.h"
 #include "thames_device.h"
+#include "thames_gem.h"
 
 #define THAMES_RESTART_TIMEOUT_MS	5000
 
@@ -616,6 +617,7 @@ int thames_core_resume(struct thames_core *core)
 	}
 	core->ring_tail = head;
 
+	thames_context_rewrite_entries(core);
 	goto restart;
 
 offline:
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
index f1cf79f2c2e4..e89e0dfd89b8 100644
--- a/drivers/accel/thames/thames_device.h
+++ b/drivers/accel/thames/thames_device.h
@@ -41,6 +41,23 @@ struct thames_device {
 
 	struct thames_caps caps;
 
+	struct ida ctx_ida;
+
+	/*
+	 * Seeded from the boot clock at load, so no serial repeats an earlier
+	 * load's.
+	 */
+	atomic64_t last_context_serial;
+
+	atomic64_t mapping_generation;
+
+	struct ida asid_ida;
+	u32 asid_next;
+	struct mutex asid_lock;
+
+	struct list_head contexts;
+	struct mutex contexts_lock;
+
 };
 
 #define to_thames_device(drm_dev) \
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index de93bc87756c..cb0287430d25 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -22,6 +22,7 @@
 
 #include "thames_core.h"
 #include "thames_drv.h"
+#include "thames_gem.h"
 
 static struct platform_device *drm_dev;
 static struct thames_device *tdev;
@@ -48,12 +49,19 @@ static int thames_open(struct drm_device *dev, struct drm_file *file)
 
 	thames_priv->tdev = tdev;
 	kref_init(&thames_priv->refcount);
+	mutex_init(&thames_priv->mm_lock);
 
 	file->driver_priv = thames_priv;
 
+	ret = thames_context_create(thames_priv);
+	if (ret)
+		goto err_close_job;
+
 	drm_dev_exit(idx);
 	return 0;
 
+err_close_job:
+	kfree(thames_priv);
 err_exit:
 	drm_dev_exit(idx);
 	return ret;
@@ -63,6 +71,9 @@ static void thames_file_priv_release(struct kref *kref)
 {
 	struct thames_file_priv *priv = container_of(kref, struct thames_file_priv, refcount);
 
+	thames_context_destroy(priv);
+	thames_context_va_fini(priv);
+	mutex_destroy(&priv->mm_lock);
 	kfree(priv);
 }
 
@@ -76,6 +87,9 @@ static void thames_postclose(struct drm_device *dev, struct drm_file *file)
 {
 	struct thames_file_priv *thames_priv = file->driver_priv;
 
+	dev_dbg(dev->dev, "Context %u: file closed, %u references left\n",
+		thames_priv->context_id, kref_read(&thames_priv->refcount) - 1);
+
 	thames_file_priv_put(thames_priv);
 }
 
@@ -174,17 +188,36 @@ static const struct drm_driver thames_drm_driver = {
 	.minor = THAMES_DRIVER_MINOR,
 };
 
+static void thames_device_release(struct drm_device *ddev, void *data)
+{
+	struct thames_device *tdev = to_thames_device(ddev);
+
+	ida_destroy(&tdev->ctx_ida);
+	ida_destroy(&tdev->asid_ida);
+}
+
 static struct thames_device *thames_device_init(struct platform_device *pdev,
 						const struct drm_driver *thames_drm_driver)
 {
 	struct device *dev = &pdev->dev;
 	struct thames_device *tdev;
+	u64 seed;
 	int err;
 
 	tdev = devm_drm_dev_alloc(dev, thames_drm_driver, struct thames_device, ddev);
 	if (IS_ERR(tdev))
 		return tdev;
 
+	/*
+	 * Live as long as the DRM device: a client's buffers and last Jobs
+	 * reach them after an unbind.
+	 */
+	ida_init(&tdev->ctx_ida);
+	ida_init(&tdev->asid_ida);
+	err = drmm_add_action_or_reset(&tdev->ddev, thames_device_release, NULL);
+	if (err)
+		return ERR_PTR(err);
+
 	dev_set_drvdata(dev, tdev);
 
 	dma_set_max_seg_size(dev, UINT_MAX);
@@ -193,6 +226,24 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	err = drmm_mutex_init(&tdev->ddev, &tdev->asid_lock);
+	if (err)
+		return ERR_PTR(err);
+
+	err = drmm_mutex_init(&tdev->ddev, &tdev->contexts_lock);
+	if (err)
+		return ERR_PTR(err);
+
+	INIT_LIST_HEAD(&tdev->contexts);
+
+	tdev->asid_next = 1;
+	/*
+	 * Above every number the previous load handed out, which counted up by
+	 * one from an earlier reading of the same clock.
+	 */
+	seed = ktime_get_boottime_ns();
+	atomic64_set(&tdev->last_context_serial, seed);
+	atomic64_set(&tdev->mapping_generation, seed);
 
 	return tdev;
 }
diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
index 2678abc2e2de..f4536078e02f 100644
--- a/drivers/accel/thames/thames_drv.h
+++ b/drivers/accel/thames/thames_drv.h
@@ -25,6 +25,24 @@ struct thames_file_priv {
 	 */
 	struct kref refcount;
 
+	struct io_pgtable_ops *pgtbl_ops[THAMES_MAX_CORES];
+	u64 page_table_base[THAMES_MAX_CORES];
+	u32 asid;
+
+	struct list_head context_node;
+
+	struct drm_mm mm;
+	struct mutex mm_lock;
+	struct drm_mm_node first_page;
+	struct drm_mm_node user_region;
+	struct drm_mm_node window_nodes[THAMES_NUM_WINDOWS];
+
+	u32 context_id;
+
+	u64 context_serial;
+
+	struct drm_gem_dma_object *user_stack;
+	struct drm_gem_dma_object *save_area;
 };
 
 void thames_file_priv_put(struct thames_file_priv *priv);
diff --git a/drivers/accel/thames/thames_gem.c b/drivers/accel/thames/thames_gem.c
new file mode 100644
index 000000000000..b98e0fe203b0
--- /dev/null
+++ b/drivers/accel/thames/thames_gem.c
@@ -0,0 +1,546 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#include <linux/bits.h>
+#include <linux/dma-mapping.h>
+#include <linux/dma-resv.h>
+#include <linux/idr.h>
+#include <linux/io-pgtable.h>
+#include <linux/iommu.h>
+#include <linux/minmax.h>
+#include <linux/scatterlist.h>
+#include <linux/sizes.h>
+
+#include <drm/drm_device.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_gem_dma_helper.h>
+#include <drm/drm_print.h>
+#include <drm/drm_timeout.h>
+#include <drm/thames_accel.h>
+
+#include "thames_abi.h"
+#include "thames_device.h"
+#include "thames_drv.h"
+#include "thames_gem.h"
+
+static size_t thames_pgsize(u64 addr, size_t size, size_t *count)
+{
+	size_t blk_offset = -addr % SZ_2M;
+
+	if (blk_offset || size < SZ_2M) {
+		*count = min_not_zero(blk_offset, size) / SZ_4K;
+		return SZ_4K;
+	}
+
+	*count = size / SZ_2M;
+	return SZ_2M;
+}
+
+static void thames_pgtable_unmap(struct io_pgtable_ops *ops, u64 dsp_va, u64 size)
+{
+	u64 offset = 0;
+
+	while (offset < size) {
+		size_t pgcount, unmapped;
+		size_t pgsize = thames_pgsize(dsp_va + offset, size - offset, &pgcount);
+
+		unmapped = ops->unmap_pages(ops, dsp_va + offset, pgsize, pgcount, NULL);
+		if (WARN_ON(!unmapped))
+			break;
+
+		offset += unmapped;
+	}
+}
+
+static int thames_pgtable_map(struct io_pgtable_ops *ops, u64 dsp_va, int prot,
+			      phys_addr_t paddr, size_t size)
+{
+	size_t offset = 0;
+
+	while (offset < size) {
+		size_t pgcount, mapped = 0;
+		size_t pgsize = thames_pgsize((dsp_va + offset) | (paddr + offset),
+					      size - offset, &pgcount);
+		int ret = ops->map_pages(ops, dsp_va + offset, paddr + offset, pgsize, pgcount,
+					 prot, GFP_KERNEL, &mapped);
+
+		if (!ret && !mapped)
+			ret = -ENOMEM;
+		if (ret) {
+			thames_pgtable_unmap(ops, dsp_va, offset);
+			return ret;
+		}
+
+		offset += mapped;
+	}
+
+	return 0;
+}
+
+static int thames_context_map(struct thames_file_priv *priv, u64 dsp_va, int prot,
+			      phys_addr_t paddr, size_t size)
+{
+	unsigned int c;
+	int ret;
+
+	for (c = 0; c < priv->tdev->num_cores; c++) {
+		ret = thames_pgtable_map(priv->pgtbl_ops[c], dsp_va, prot, paddr, size);
+		if (ret) {
+			while (c--)
+				thames_pgtable_unmap(priv->pgtbl_ops[c], dsp_va, size);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+/* Cyclic, so a freed ASID is not handed out again at once. */
+static int thames_asid_alloc(struct thames_device *tdev)
+{
+	u32 max = BIT(tdev->caps.asid_bits) - 1;
+	int asid = -ENOSPC;
+
+	guard(mutex)(&tdev->asid_lock);
+
+	if (tdev->asid_next <= max)
+		asid = ida_alloc_range(&tdev->asid_ida, tdev->asid_next, max, GFP_KERNEL);
+	if (asid == -ENOSPC)
+		asid = ida_alloc_range(&tdev->asid_ida, 1, max, GFP_KERNEL);
+	if (asid < 0)
+		return asid;
+
+	tdev->asid_next = asid + 1;
+
+	return asid;
+}
+
+/*
+ * The device has no IOMMU, so a DMA address is the physical address the page
+ * tables hold.  The firmware programs a MAIR with these attribute indexes,
+ * which the page table entries must use; this word is what io-pgtable-arm
+ * expects, its cacheable attribute 0xff where the firmware's is 0x7d.
+ */
+#define THAMES_PGTABLE_MAIR_LOW	0xf404ff44u
+
+static void thames_context_pgtable_fini(struct thames_file_priv *priv)
+{
+	unsigned int c;
+
+	for (c = 0; c < THAMES_MAX_CORES; c++) {
+		if (priv->pgtbl_ops[c])
+			free_io_pgtable_ops(priv->pgtbl_ops[c]);
+		priv->pgtbl_ops[c] = NULL;
+	}
+}
+
+/*
+ * Also for offline cores, so a core a reset brings back finds its page table.
+ */
+static int thames_context_pgtable_init(struct thames_file_priv *priv)
+{
+	struct thames_device *tdev = priv->tdev;
+	unsigned int c;
+
+	for (c = 0; c < tdev->num_cores; c++) {
+		struct io_pgtable_cfg cfg = {
+			.pgsize_bitmap = SZ_4K | SZ_2M,
+			.ias = 48,
+			.oas = 40,
+			.coherent_walk = false,
+			.iommu_dev = tdev->ddev.dev,
+		};
+		struct io_pgtable *iop;
+
+		priv->pgtbl_ops[c] = alloc_io_pgtable_ops(ARM_64_LPAE_S1, &cfg, priv);
+		if (!priv->pgtbl_ops[c])
+			return -ENOMEM;
+
+		iop = io_pgtable_ops_to_pgtable(priv->pgtbl_ops[c]);
+		if (WARN_ON((iop->cfg.arm_lpae_s1_cfg.mair & 0xffffffffU) !=
+			    THAMES_PGTABLE_MAIR_LOW))
+			return -EINVAL;
+
+		priv->page_table_base[c] = iop->cfg.arm_lpae_s1_cfg.ttbr;
+	}
+
+	return 0;
+}
+
+static struct thames_context_entry *thames_context_entry(struct thames_core *core,
+							 struct thames_file_priv *priv)
+{
+	return &core->ipc->context_table[priv->context_id];
+}
+
+/*
+ * Offline cores included: a reset brings a core back to the entries in its IPC
+ * memory.
+ */
+#define for_each_core(tdev, core) \
+	for ((core) = (tdev)->cores; (core) < (tdev)->cores + (tdev)->num_cores; (core)++)
+
+static int thames_context_map_core_regions(struct thames_file_priv *priv)
+{
+	struct thames_device *tdev = priv->tdev;
+	unsigned int c, i;
+	int ret;
+
+	for (c = 0; c < tdev->num_cores; c++) {
+		const struct thames_core *core = &tdev->cores[c];
+
+		for (i = 0; i < THAMES_NUM_WINDOWS; i++) {
+			const struct resource *window = thames_core_window(core, i);
+			int prot = IOMMU_READ | IOMMU_WRITE | IOMMU_NOEXEC;
+			phys_addr_t paddr = window->start;
+			u64 size = resource_size(window);
+
+			if (i == THAMES_WINDOW_DRU) {
+				prot |= IOMMU_MMIO;
+				paddr += (u64)core->dru_kernel_channel_first *
+					 tdev->caps.dru_channel_stride;
+				size = tdev->caps.window_sizes[THAMES_WINDOW_DRU];
+			} else {
+				prot |= IOMMU_CACHE;
+			}
+
+			ret = thames_pgtable_map(priv->pgtbl_ops[c], thames_window_va(tdev, i),
+						 prot, paddr, size);
+			if (ret)
+				return ret;
+		}
+	}
+
+	return 0;
+}
+
+static const struct drm_gem_object_funcs thames_gem_funcs;
+
+/*
+ * The allocator zeroes through the cache: clean it before the DSP can reach the
+ * memory, so no dirty line is written back over the DSP's data later.
+ */
+static struct drm_gem_dma_object *thames_gem_create(struct drm_device *ddev, size_t size)
+{
+	struct device *dev = drm_dev_dma_dev(ddev);
+	struct thames_gem_object *bo;
+	struct drm_gem_object *obj;
+	size_t offset = 0;
+	int ret;
+
+	size = PAGE_ALIGN(size);
+
+	bo = kzalloc_obj(*bo, GFP_KERNEL);
+	if (!bo)
+		return ERR_PTR(-ENOMEM);
+
+	obj = &bo->base.base;
+	obj->funcs = &thames_gem_funcs;
+	bo->base.map_noncoherent = true;
+
+	ret = drm_gem_object_init(ddev, obj, size);
+	if (ret) {
+		kfree(bo);
+		return ERR_PTR(ret);
+	}
+
+	ret = drm_gem_create_mmap_offset(obj);
+	if (ret)
+		goto err_put;
+
+	bo->alloc_size = size >= SZ_2M ? size + SZ_2M - PAGE_SIZE : size;
+	bo->alloc_vaddr = dma_alloc_noncoherent(dev, bo->alloc_size, &bo->alloc_dma,
+						DMA_TO_DEVICE, GFP_KERNEL | __GFP_NOWARN);
+	if (!bo->alloc_vaddr) {
+		ret = -ENOMEM;
+		goto err_put;
+	}
+
+	if (size >= SZ_2M)
+		offset = ALIGN(bo->alloc_dma, SZ_2M) - bo->alloc_dma;
+
+	bo->base.vaddr = bo->alloc_vaddr + offset;
+	bo->base.dma_addr = bo->alloc_dma + offset;
+
+	dma_sync_single_for_device(dev, bo->alloc_dma, bo->alloc_size, DMA_TO_DEVICE);
+
+	return &bo->base;
+
+err_put:
+	drm_gem_object_put(obj);
+	return ERR_PTR(ret);
+}
+
+static struct drm_gem_dma_object *thames_context_alloc_user_pages(struct thames_file_priv *priv,
+								  u64 dsp_va, size_t size)
+{
+	int prot = IOMMU_READ | IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_CACHE;
+	struct drm_gem_dma_object *dma_obj;
+	int ret;
+
+	dma_obj = thames_gem_create(&priv->tdev->ddev, size);
+	if (IS_ERR(dma_obj))
+		return dma_obj;
+
+	ret = thames_context_map(priv, dsp_va, prot, dma_obj->dma_addr, dma_obj->base.size);
+	if (ret) {
+		drm_gem_object_put(&dma_obj->base);
+		return ERR_PTR(ret);
+	}
+
+	return dma_obj;
+}
+
+static int thames_context_map_user_pages(struct thames_file_priv *priv)
+{
+	struct drm_gem_dma_object *dma_obj;
+
+	dma_obj = thames_context_alloc_user_pages(priv, THAMES_USER_STACK_BASE,
+						  THAMES_USER_STACK_SIZE);
+	if (IS_ERR(dma_obj))
+		return PTR_ERR(dma_obj);
+	priv->user_stack = dma_obj;
+
+	dma_obj = thames_context_alloc_user_pages(priv, THAMES_USER_SAVE_AREA_BASE,
+						  THAMES_USER_SAVE_AREA_SIZE);
+	if (IS_ERR(dma_obj))
+		return PTR_ERR(dma_obj);
+	priv->save_area = dma_obj;
+
+	return 0;
+}
+
+/* Only after the page tables are freed, so none maps a page given back. */
+static void thames_context_free_user_pages(struct thames_file_priv *priv)
+{
+	if (priv->save_area)
+		drm_gem_object_put(&priv->save_area->base);
+	if (priv->user_stack)
+		drm_gem_object_put(&priv->user_stack->base);
+	priv->save_area = NULL;
+	priv->user_stack = NULL;
+}
+
+/* Every core's entry: a removed mapping may still sit in either core's TLB. */
+static void thames_context_bump_generation(struct thames_file_priv *priv)
+{
+	u64 generation = atomic64_inc_return(&priv->tdev->mapping_generation);
+	struct thames_core *core;
+	int idx;
+
+	if (!drm_dev_enter(&priv->tdev->ddev, &idx))
+		return;
+
+	for_each_core(priv->tdev, core)
+		WRITE_ONCE(thames_context_entry(core, priv)->mapping_generation, generation);
+	/* Order the generation before the next submission. */
+	wmb();
+
+	drm_dev_exit(idx);
+}
+
+static void thames_context_write_entry(struct thames_file_priv *priv)
+{
+	struct thames_core *core;
+
+	for_each_core(priv->tdev, core) {
+		struct thames_context_entry *entry = thames_context_entry(core, priv);
+
+		entry->page_table_base = priv->page_table_base[core->index];
+		entry->asid = priv->asid;
+	}
+	thames_context_bump_generation(priv);
+}
+
+/*
+ * A new generation, so the restarted firmware invalidates its TLB whatever it
+ * remembers.
+ */
+void thames_context_rewrite_entries(struct thames_core *core)
+{
+	struct thames_device *tdev = core->tdev;
+	struct thames_file_priv *priv;
+
+	guard(mutex)(&tdev->contexts_lock);
+
+	list_for_each_entry(priv, &tdev->contexts, context_node) {
+		struct thames_context_entry *entry = thames_context_entry(core, priv);
+
+		entry->page_table_base = priv->page_table_base[core->index];
+		entry->asid = priv->asid;
+		thames_context_bump_generation(priv);
+	}
+}
+
+/* Offline cores too: their entry would otherwise name a freed page table. */
+static void thames_context_clear_entry(struct thames_file_priv *priv)
+{
+	struct thames_core *core;
+	int idx;
+
+	if (!drm_dev_enter(&priv->tdev->ddev, &idx))
+		return;
+
+	for_each_core(priv->tdev, core)
+		memset(thames_context_entry(core, priv), 0, sizeof(struct thames_context_entry));
+	/* Order the cleared entry before the id is freed. */
+	wmb();
+
+	drm_dev_exit(idx);
+}
+
+#define THAMES_DSP_VA_SIZE (1ULL << 47)
+
+static int thames_context_reserve(struct thames_file_priv *priv, struct drm_mm_node *node,
+				  u64 start, u64 size)
+{
+	node->start = start;
+	node->size = size;
+
+	return drm_mm_reserve_node(&priv->mm, node);
+}
+
+void thames_context_va_fini(struct thames_file_priv *priv)
+{
+	unsigned int i;
+
+	for (i = 0; i < THAMES_NUM_WINDOWS; i++) {
+		if (drm_mm_node_allocated(&priv->window_nodes[i]))
+			drm_mm_remove_node(&priv->window_nodes[i]);
+	}
+	if (drm_mm_node_allocated(&priv->user_region))
+		drm_mm_remove_node(&priv->user_region);
+	if (drm_mm_node_allocated(&priv->first_page))
+		drm_mm_remove_node(&priv->first_page);
+
+	drm_mm_takedown(&priv->mm);
+}
+
+static int thames_context_va_init(struct thames_file_priv *priv)
+{
+	struct thames_device *tdev = priv->tdev;
+	unsigned int i;
+	int ret;
+
+	drm_mm_init(&priv->mm, 0, THAMES_DSP_VA_SIZE);
+
+	ret = thames_context_reserve(priv, &priv->first_page, 0, PAGE_SIZE);
+	if (ret)
+		goto err_fini;
+
+	ret = thames_context_reserve(priv, &priv->user_region, THAMES_USER_REGION_BASE,
+				     THAMES_USER_REGION_SIZE);
+	if (ret)
+		goto err_fini;
+
+	for (i = 0; i < THAMES_NUM_WINDOWS; i++) {
+		ret = thames_context_reserve(priv, &priv->window_nodes[i],
+					     thames_window_va(tdev, i),
+					     resource_size(thames_core_window(&tdev->cores[0], i)));
+		if (ret)
+			goto err_fini;
+	}
+
+	return 0;
+
+err_fini:
+	thames_context_va_fini(priv);
+	return ret;
+}
+
+int thames_context_create(struct thames_file_priv *priv)
+{
+	struct thames_device *tdev = priv->tdev;
+	int ret, asid;
+
+	ret = ida_alloc_range(&tdev->ctx_ida, 1, THAMES_MAX_CONTEXTS - 1, GFP_KERNEL);
+	if (ret < 0)
+		return ret;
+
+	priv->context_id = ret;
+	priv->context_serial = atomic64_inc_return(&tdev->last_context_serial);
+
+	ret = thames_context_va_init(priv);
+	if (ret)
+		goto err_free_id;
+
+	ret = thames_context_pgtable_init(priv);
+	if (ret)
+		goto err_free_pgtable;
+
+	ret = thames_context_map_core_regions(priv);
+	if (ret)
+		goto err_free_pgtable;
+
+	ret = thames_context_map_user_pages(priv);
+	if (ret)
+		goto err_free_pgtable;
+
+	asid = thames_asid_alloc(tdev);
+	if (asid < 0) {
+		ret = asid;
+		goto err_free_pgtable;
+	}
+	priv->asid = asid;
+	dev_dbg(tdev->ddev.dev, "Context %u: ASID %d\n", priv->context_id, asid);
+
+	thames_context_write_entry(priv);
+
+	scoped_guard(mutex, &tdev->contexts_lock)
+		list_add_tail(&priv->context_node, &tdev->contexts);
+
+	return 0;
+
+err_free_pgtable:
+	thames_context_pgtable_fini(priv);
+	thames_context_free_user_pages(priv);
+	thames_context_va_fini(priv);
+err_free_id:
+	ida_free(&tdev->ctx_ida, priv->context_id);
+	return ret;
+}
+
+void thames_context_destroy(struct thames_file_priv *priv)
+{
+	struct thames_device *tdev = priv->tdev;
+
+	/*
+	 * The Context goes after its last pushed Job completed, so no Job runs
+	 * on mappings removed here.
+	 */
+	scoped_guard(mutex, &tdev->contexts_lock)
+		list_del(&priv->context_node);
+
+	thames_context_clear_entry(priv);
+
+	thames_context_pgtable_fini(priv);
+	thames_context_free_user_pages(priv);
+	ida_free(&tdev->asid_ida, priv->asid);
+	ida_free(&tdev->ctx_ida, priv->context_id);
+
+	dev_dbg(tdev->ddev.dev, "Context %u: destroyed, ASID %u back in the pool\n",
+		priv->context_id, priv->asid);
+}
+
+static void thames_gem_bo_free(struct drm_gem_object *obj)
+{
+	struct thames_gem_object *bo = to_thames_bo(obj);
+
+	if (bo->alloc_vaddr)
+		dma_free_noncoherent(drm_dev_dma_dev(obj->dev), bo->alloc_size, bo->alloc_vaddr,
+				     bo->alloc_dma, DMA_TO_DEVICE);
+
+	drm_gem_free_mmap_offset(obj);
+	drm_gem_object_release(obj);
+	kfree(bo);
+}
+
+static const struct drm_gem_object_funcs thames_gem_funcs = {
+	.free = thames_gem_bo_free,
+	.print_info = drm_gem_dma_object_print_info,
+	.get_sg_table = drm_gem_dma_object_get_sg_table,
+	.vmap = drm_gem_dma_object_vmap,
+	.mmap = drm_gem_dma_object_mmap,
+	.vm_ops = &drm_gem_dma_vm_ops,
+};
+
diff --git a/drivers/accel/thames/thames_gem.h b/drivers/accel/thames/thames_gem.h
new file mode 100644
index 000000000000..09ac1b50d21a
--- /dev/null
+++ b/drivers/accel/thames/thames_gem.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_GEM_H__
+#define __THAMES_GEM_H__
+
+#include <linux/sizes.h>
+
+#include <drm/drm_gem_dma_helper.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/drm_mm.h>
+
+#include "thames_abi.h"
+
+struct thames_core;
+struct thames_device;
+struct thames_file_priv;
+
+struct thames_gem_object {
+	/*
+	 * Both start with their drm_gem_object, so base.base is the GEM object
+	 * either way.
+	 */
+	union {
+		struct drm_gem_dma_object base;
+		struct drm_gem_shmem_object shmem;
+	};
+
+	/*
+	 * base.vaddr and base.dma_addr are the 2 MiB-aligned part of this
+	 * allocation when the buffer is 2 MiB or more.
+	 */
+	void *alloc_vaddr;
+	dma_addr_t alloc_dma;
+	size_t alloc_size;
+
+};
+
+
+int thames_context_create(struct thames_file_priv *priv);
+
+void thames_context_destroy(struct thames_file_priv *priv);
+void thames_context_rewrite_entries(struct thames_core *core);
+
+void thames_context_va_fini(struct thames_file_priv *priv);
+
+static inline struct thames_gem_object *to_thames_bo(struct drm_gem_object *obj)
+{
+	return container_of(obj, struct thames_gem_object, base.base);
+}
+
+#endif

-- 
2.54.0

[PATCH v3 07/16] accel/thames: Add IOCTLs for BO creation and mapping

[Tomeu Vizoso]

Creation returns a handle and maps nothing. The client maps a buffer
with DRM_IOCTL_THAMES_BO_BIND at a DSP address of its choice, as a
Vulkan driver would; the kernel enters the mapping in the page table
of every core, because we cannot know upfront which core will run a
given job.

We use the SHMEM helpers for non-contiguous memory. Contiguous memory
comes from the driver's own allocator, which aligns a buffer of 2 MiB or
more to 2 MiB so it can be mapped as blocks; only the GEM DMA object
and its mmap and vmap helpers are used for it.

Contiguous memory is needed for buffers that are going to be read from
or written into by the DRU, as it doesn't resolve addresses via the MMU.

Binding at an address the client chooses lets a program be compiled,
with its addresses fixed, on a machine without the hardware, and is what
a Vulkan driver needs.

A buffer is bound anywhere from 4 GiB up to the user region at 1 TiB; a
buffer shared through a dma-buf may be bound in several Contexts, so
the bindings are kept under one device-wide lock. The IOCTLs take their
arrays as versioned objects whose stride is the size userspace was
built against, so a structure can grow under a minor bump.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Added read-only, executable and contiguous flags.
- Added BO_BIND IOCTL.
- Copy the IOCTLs' arrays with vmemdup_array_user(), through one helper
  for versioned arrays (Tvrtko Ursulin).

v2:
- Add thames_accel.h UAPI header (Robert Nelson).
---
 drivers/accel/thames/Kconfig         |   2 +
 drivers/accel/thames/thames_device.h |   6 +
 drivers/accel/thames/thames_drv.c    |  47 +++++
 drivers/accel/thames/thames_drv.h    |   5 +
 drivers/accel/thames/thames_gem.c    | 398 +++++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_gem.h    |  21 ++
 include/uapi/drm/thames_accel.h      | 183 +++++++++++++++-
 7 files changed, 661 insertions(+), 1 deletion(-)

diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
index 9362b9274075..47a2986fcabd 100644
--- a/drivers/accel/thames/Kconfig
+++ b/drivers/accel/thames/Kconfig
@@ -6,6 +6,8 @@ config DRM_ACCEL_THAMES
 	depends on TI_K3_DSP_REMOTEPROC
 	depends on REMOTEPROC
 	depends on MAILBOX
+	select DRM_GEM_DMA_HELPER
+	select DRM_GEM_SHMEM_HELPER
 	select IOMMU_IO_PGTABLE_LPAE
 	help
 	  Choose this option if you have a Texas Instruments J722S and want to
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
index e89e0dfd89b8..a1a54d974359 100644
--- a/drivers/accel/thames/thames_device.h
+++ b/drivers/accel/thames/thames_device.h
@@ -58,6 +58,12 @@ struct thames_device {
 	struct list_head contexts;
 	struct mutex contexts_lock;
 
+	/*
+	 * One lock for the device: a dma-buf shared buffer is bound from
+	 * several files.
+	 */
+	struct mutex bindings_lock;
+
 };
 
 #define to_thames_device(drm_dev) \
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index cb0287430d25..a17e7a90f7c1 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -93,6 +93,41 @@ static void thames_postclose(struct drm_device *dev, struct drm_file *file)
 	thames_file_priv_put(thames_priv);
 }
 
+void *thames_get_uobj_array(const struct drm_thames_obj_array *in, u32 min_stride, u32 obj_size)
+{
+	void __user *in_ptr = u64_to_user_ptr(in->array);
+	void *out, *out_ptr;
+	int ret;
+	u32 i;
+
+	if (!in->count)
+		return NULL;
+
+	if (in->stride < min_stride)
+		return ERR_PTR(-EINVAL);
+
+	if (in->stride == obj_size)
+		return vmemdup_array_user(in_ptr, in->count, obj_size);
+
+	out = kvmalloc_array(in->count, obj_size, GFP_KERNEL);
+	if (!out)
+		return ERR_PTR(-ENOMEM);
+
+	out_ptr = out;
+	for (i = 0; i < in->count; i++) {
+		ret = copy_struct_from_user(out_ptr, obj_size, in_ptr, in->stride);
+		if (ret) {
+			kvfree(out);
+			return ERR_PTR(ret);
+		}
+
+		out_ptr += obj_size;
+		in_ptr += in->stride;
+	}
+
+	return out;
+}
+
 #define THAMES_DRIVER_MAJOR 1
 #define THAMES_DRIVER_MINOR 0
 
@@ -121,6 +156,8 @@ static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query
 		.l2_aux.dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2AUX),
 		.l2_aux.size = reported->window_sizes[THAMES_WINDOW_L2AUX],
 		.dru_realtime_dsp_va = thames_window_va(tdev, THAMES_WINDOW_DRU),
+		.dsp_va_range.dsp_va = THAMES_BIND_VA_START,
+		.dsp_va_range.size = THAMES_BIND_VA_END - THAMES_BIND_VA_START,
 		.user_region.dsp_va = THAMES_USER_REGION_BASE,
 		.user_region.size = THAMES_USER_REGION_SIZE,
 		.hardware_id = reported->hardware_id,
@@ -165,11 +202,17 @@ static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,	\
 	return ret;								\
 }
 
+THAMES_IOCTL_ENTERED(bo_create)
+THAMES_IOCTL_ENTERED(bo_mmap_offset)
 THAMES_IOCTL_ENTERED(dev_query)
+THAMES_IOCTL_ENTERED(bo_bind)
 
 static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
 #define THAMES_IOCTL(n, func) DRM_IOCTL_DEF_DRV(THAMES_##n, thames_ioctl_##func, 0)
+	THAMES_IOCTL(BO_CREATE, bo_create_entered),
+	THAMES_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset_entered),
 	THAMES_IOCTL(DEV_QUERY, dev_query_entered),
+	THAMES_IOCTL(BO_BIND, bo_bind_entered),
 };
 
 DEFINE_DRM_ACCEL_FOPS(thames_accel_driver_fops);
@@ -234,6 +277,10 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	err = drmm_mutex_init(&tdev->ddev, &tdev->bindings_lock);
+	if (err)
+		return ERR_PTR(err);
+
 	INIT_LIST_HEAD(&tdev->contexts);
 
 	tdev->asid_next = 1;
diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
index f4536078e02f..7f9031b63780 100644
--- a/drivers/accel/thames/thames_drv.h
+++ b/drivers/accel/thames/thames_drv.h
@@ -15,6 +15,7 @@
 #include "thames_device.h"
 
 struct drm_gem_dma_object;
+struct drm_thames_obj_array;
 
 struct thames_file_priv {
 	struct thames_device *tdev;
@@ -37,6 +38,8 @@ struct thames_file_priv {
 	struct drm_mm_node user_region;
 	struct drm_mm_node window_nodes[THAMES_NUM_WINDOWS];
 
+	struct list_head bindings;
+
 	u32 context_id;
 
 	u64 context_serial;
@@ -47,4 +50,6 @@ struct thames_file_priv {
 
 void thames_file_priv_put(struct thames_file_priv *priv);
 
+void *thames_get_uobj_array(const struct drm_thames_obj_array *in, u32 min_stride, u32 obj_size);
+
 #endif
diff --git a/drivers/accel/thames/thames_gem.c b/drivers/accel/thames/thames_gem.c
index b98e0fe203b0..c9fd08c7aaf4 100644
--- a/drivers/accel/thames/thames_gem.c
+++ b/drivers/accel/thames/thames_gem.c
@@ -14,6 +14,7 @@
 #include <drm/drm_device.h>
 #include <drm/drm_drv.h>
 #include <drm/drm_gem_dma_helper.h>
+#include <drm/drm_gem_shmem_helper.h>
 #include <drm/drm_print.h>
 #include <drm/drm_timeout.h>
 #include <drm/thames_accel.h>
@@ -23,6 +24,30 @@
 #include "thames_drv.h"
 #include "thames_gem.h"
 
+/*
+ * References neither the buffer nor the Context: whichever goes first removes
+ * it.
+ */
+struct thames_binding {
+	struct thames_gem_object *bo;
+	struct thames_file_priv *priv;
+	struct drm_mm_node mm;
+	struct list_head bo_link;
+	struct list_head context_link;
+};
+
+static int thames_bo_flags_to_prot(u32 flags)
+{
+	int prot = IOMMU_READ | IOMMU_CACHE;
+
+	if (!(flags & DRM_THAMES_BO_EXECUTABLE))
+		prot |= IOMMU_NOEXEC;
+	if (!(flags & (DRM_THAMES_BO_READ_ONLY | DRM_THAMES_BO_EXECUTABLE)))
+		prot |= IOMMU_WRITE;
+
+	return prot;
+}
+
 static size_t thames_pgsize(u64 addr, size_t size, size_t *count)
 {
 	size_t blk_offset = -addr % SZ_2M;
@@ -95,6 +120,72 @@ static int thames_context_map(struct thames_file_priv *priv, u64 dsp_va, int pro
 	return 0;
 }
 
+static void thames_context_unmap(struct thames_file_priv *priv, u64 dsp_va, u64 size)
+{
+	unsigned int c;
+
+	for (c = 0; c < priv->tdev->num_cores; c++)
+		thames_pgtable_unmap(priv->pgtbl_ops[c], dsp_va, size);
+}
+
+struct thames_gem_object *thames_context_find_bo(struct thames_file_priv *priv, u64 start,
+						 u64 end, u64 *offset)
+{
+	struct thames_binding *binding;
+	struct thames_gem_object *bo;
+	struct drm_mm_node *node;
+	unsigned int i;
+
+	guard(mutex)(&priv->mm_lock);
+
+	node = __drm_mm_interval_first(&priv->mm, start, start);
+	if (node == &priv->mm.head_node || node == &priv->first_page ||
+	    node == &priv->user_region)
+		return NULL;
+
+	for (i = 0; i < THAMES_NUM_WINDOWS; i++) {
+		if (node == &priv->window_nodes[i])
+			return NULL;
+	}
+
+	binding = container_of(node, struct thames_binding, mm);
+	bo = binding->bo;
+	if (end > node->start + bo->base.base.size)
+		return NULL;
+
+	if (!kref_get_unless_zero(&bo->base.base.refcount))
+		return NULL;
+
+	*offset = start - node->start;
+
+	return bo;
+}
+
+static struct thames_binding *thames_bo_find_binding(struct thames_gem_object *bo,
+						     struct thames_file_priv *priv)
+{
+	struct thames_binding *binding;
+
+	lockdep_assert_held(&priv->tdev->bindings_lock);
+
+	list_for_each_entry(binding, &bo->bindings, bo_link) {
+		if (binding->priv == priv)
+			return binding;
+	}
+
+	return NULL;
+}
+
+u64 thames_bo_dsp_va(struct thames_gem_object *bo, struct thames_file_priv *priv)
+{
+	struct thames_binding *binding;
+
+	guard(mutex)(&priv->tdev->bindings_lock);
+	binding = thames_bo_find_binding(bo, priv);
+
+	return binding ? binding->mm.start : 0;
+}
+
 /* Cyclic, so a freed ASID is not handed out again at once. */
 static int thames_asid_alloc(struct thames_device *tdev)
 {
@@ -215,6 +306,7 @@ static int thames_context_map_core_regions(struct thames_file_priv *priv)
 }
 
 static const struct drm_gem_object_funcs thames_gem_funcs;
+static const struct drm_gem_object_funcs thames_gem_shmem_funcs;
 
 /*
  * The allocator zeroes through the cache: clean it before the DSP can reach the
@@ -234,6 +326,7 @@ static struct drm_gem_dma_object *thames_gem_create(struct drm_device *ddev, siz
 	if (!bo)
 		return ERR_PTR(-ENOMEM);
 
+	INIT_LIST_HEAD(&bo->bindings);
 	obj = &bo->base.base;
 	obj->funcs = &thames_gem_funcs;
 	bo->base.map_noncoherent = true;
@@ -271,6 +364,38 @@ static struct drm_gem_dma_object *thames_gem_create(struct drm_device *ddev, siz
 	return ERR_PTR(ret);
 }
 
+/*
+ * The DSP reaches these pages only through the page tables, never by a DRU
+ * transfer.
+ */
+static struct thames_gem_object *thames_gem_create_shmem(struct drm_device *ddev, size_t size)
+{
+	struct thames_gem_object *bo;
+	struct sg_table *sgt;
+	int ret;
+
+	bo = kzalloc_obj(*bo, GFP_KERNEL);
+	if (!bo)
+		return ERR_PTR(-ENOMEM);
+
+	INIT_LIST_HEAD(&bo->bindings);
+	bo->shmem.base.funcs = &thames_gem_shmem_funcs;
+
+	ret = drm_gem_shmem_init(ddev, &bo->shmem, PAGE_ALIGN(size));
+	if (ret) {
+		kfree(bo);
+		return ERR_PTR(ret);
+	}
+
+	sgt = drm_gem_shmem_get_pages_sgt(&bo->shmem);
+	if (IS_ERR(sgt)) {
+		drm_gem_object_put(&bo->shmem.base);
+		return ERR_CAST(sgt);
+	}
+
+	return bo;
+}
+
 static struct drm_gem_dma_object *thames_context_alloc_user_pages(struct thames_file_priv *priv,
 								  u64 dsp_va, size_t size)
 {
@@ -459,6 +584,7 @@ int thames_context_create(struct thames_file_priv *priv)
 
 	priv->context_id = ret;
 	priv->context_serial = atomic64_inc_return(&tdev->last_context_serial);
+	INIT_LIST_HEAD(&priv->bindings);
 
 	ret = thames_context_va_init(priv);
 	if (ret)
@@ -500,14 +626,37 @@ int thames_context_create(struct thames_file_priv *priv)
 	return ret;
 }
 
+static void thames_binding_remove(struct thames_binding *binding)
+{
+	struct thames_file_priv *priv = binding->priv;
+
+	lockdep_assert_held(&priv->tdev->bindings_lock);
+
+	thames_context_unmap(priv, binding->mm.start, binding->mm.size);
+	thames_context_bump_generation(priv);
+
+	scoped_guard(mutex, &priv->mm_lock)
+		drm_mm_remove_node(&binding->mm);
+
+	list_del(&binding->bo_link);
+	list_del(&binding->context_link);
+	kfree(binding);
+}
+
 void thames_context_destroy(struct thames_file_priv *priv)
 {
 	struct thames_device *tdev = priv->tdev;
+	struct thames_binding *binding, *tmp;
 
 	/*
 	 * The Context goes after its last pushed Job completed, so no Job runs
 	 * on mappings removed here.
 	 */
+	scoped_guard(mutex, &tdev->bindings_lock) {
+		list_for_each_entry_safe(binding, tmp, &priv->bindings, context_link)
+			thames_binding_remove(binding);
+	}
+
 	scoped_guard(mutex, &tdev->contexts_lock)
 		list_del(&priv->context_node);
 
@@ -522,10 +671,124 @@ void thames_context_destroy(struct thames_file_priv *priv)
 		priv->context_id, priv->asid);
 }
 
+static int thames_context_map_sgt(struct thames_file_priv *priv, u64 dsp_va, int prot,
+				  struct sg_table *sgt)
+{
+	struct scatterlist *sg;
+	u64 offset = 0;
+	unsigned int i;
+	int ret;
+
+	for_each_sgtable_dma_sg(sgt, sg, i) {
+		ret = thames_context_map(priv, dsp_va + offset, prot, sg_dma_address(sg),
+					 sg_dma_len(sg));
+		if (ret) {
+			thames_context_unmap(priv, dsp_va, offset);
+			return ret;
+		}
+
+		offset += sg_dma_len(sg);
+	}
+
+	return 0;
+}
+
+static int thames_bo_bind(struct thames_file_priv *priv, struct thames_gem_object *bo,
+			  u64 dsp_va)
+{
+	size_t size = bo->base.base.size;
+	int prot = thames_bo_flags_to_prot(bo->flags);
+	struct thames_binding *binding;
+	int ret;
+
+	if (!IS_ALIGNED(dsp_va, PAGE_SIZE))
+		return -EINVAL;
+
+	if (dsp_va < THAMES_BIND_VA_START || dsp_va > THAMES_BIND_VA_END ||
+	    size > THAMES_BIND_VA_END - dsp_va)
+		return -EINVAL;
+
+	binding = kzalloc_obj(*binding);
+	if (!binding)
+		return -ENOMEM;
+
+	binding->bo = bo;
+	binding->priv = priv;
+	binding->mm.start = dsp_va;
+	binding->mm.size = size;
+
+	ret = mutex_lock_interruptible(&priv->tdev->bindings_lock);
+	if (ret)
+		goto err_free;
+
+	if (thames_bo_find_binding(bo, priv)) {
+		ret = -EBUSY;
+		goto err_unlock;
+	}
+
+	scoped_guard(mutex, &priv->mm_lock)
+		ret = drm_mm_reserve_node(&priv->mm, &binding->mm);
+	if (ret)
+		goto err_unlock;
+
+	if (bo->flags & DRM_THAMES_BO_CONTIGUOUS)
+		ret = thames_context_map(priv, dsp_va, prot, bo->base.dma_addr, size);
+	else
+		ret = thames_context_map_sgt(priv, dsp_va, prot, bo->shmem.sgt);
+	if (ret) {
+		scoped_guard(mutex, &priv->mm_lock)
+			drm_mm_remove_node(&binding->mm);
+		goto err_unlock;
+	}
+
+	list_add_tail(&binding->bo_link, &bo->bindings);
+	list_add_tail(&binding->context_link, &priv->bindings);
+	mutex_unlock(&priv->tdev->bindings_lock);
+
+	return 0;
+
+err_unlock:
+	mutex_unlock(&priv->tdev->bindings_lock);
+err_free:
+	kfree(binding);
+	return ret;
+}
+
+static int thames_bo_unbind(struct thames_file_priv *priv, struct thames_gem_object *bo)
+{
+	struct thames_binding *binding;
+	int ret;
+
+	ret = mutex_lock_interruptible(&priv->tdev->bindings_lock);
+	if (ret)
+		return ret;
+
+	binding = thames_bo_find_binding(bo, priv);
+	if (binding)
+		thames_binding_remove(binding);
+	else
+		ret = -EINVAL;
+
+	mutex_unlock(&priv->tdev->bindings_lock);
+	return ret;
+}
+
+static void thames_bo_unmap_from_device(struct thames_gem_object *bo)
+{
+	struct thames_device *tdev = to_thames_device(bo->base.base.dev);
+	struct thames_binding *binding, *tmp;
+
+	guard(mutex)(&tdev->bindings_lock);
+	list_for_each_entry_safe(binding, tmp, &bo->bindings, bo_link)
+		thames_binding_remove(binding);
+}
+
 static void thames_gem_bo_free(struct drm_gem_object *obj)
 {
 	struct thames_gem_object *bo = to_thames_bo(obj);
 
+	thames_bo_unmap_from_device(bo);
+
 	if (bo->alloc_vaddr)
 		dma_free_noncoherent(drm_dev_dma_dev(obj->dev), bo->alloc_size, bo->alloc_vaddr,
 				     bo->alloc_dma, DMA_TO_DEVICE);
@@ -535,6 +798,16 @@ static void thames_gem_bo_free(struct drm_gem_object *obj)
 	kfree(bo);
 }
 
+static void thames_gem_shmem_bo_free(struct drm_gem_object *obj)
+{
+	struct thames_gem_object *bo = to_thames_bo(obj);
+
+	thames_bo_unmap_from_device(bo);
+
+	drm_gem_free_mmap_offset(obj);
+	drm_gem_shmem_free(&bo->shmem);
+}
+
 static const struct drm_gem_object_funcs thames_gem_funcs = {
 	.free = thames_gem_bo_free,
 	.print_info = drm_gem_dma_object_print_info,
@@ -544,3 +817,128 @@ static const struct drm_gem_object_funcs thames_gem_funcs = {
 	.vm_ops = &drm_gem_dma_vm_ops,
 };
 
+static const struct drm_gem_object_funcs thames_gem_shmem_funcs = {
+	.free = thames_gem_shmem_bo_free,
+	.print_info = drm_gem_shmem_object_print_info,
+	.pin = drm_gem_shmem_object_pin,
+	.unpin = drm_gem_shmem_object_unpin,
+	.get_sg_table = drm_gem_shmem_object_get_sg_table,
+	.vmap = drm_gem_shmem_object_vmap,
+	.vunmap = drm_gem_shmem_object_vunmap,
+	.mmap = drm_gem_shmem_object_mmap,
+	.vm_ops = &drm_gem_shmem_vm_ops,
+};
+
+int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file *file)
+{
+	struct drm_thames_bo_create *args = data;
+	struct thames_gem_object *bo;
+	struct drm_gem_dma_object *mem;
+	int ret;
+
+	if (args->handle)
+		return -EINVAL;
+
+	if (!args->size)
+		return -EINVAL;
+
+	if (args->flags & ~(DRM_THAMES_BO_READ_ONLY | DRM_THAMES_BO_EXECUTABLE |
+			    DRM_THAMES_BO_CONTIGUOUS))
+		return -EINVAL;
+
+	if (args->flags & DRM_THAMES_BO_CONTIGUOUS) {
+		mem = thames_gem_create(ddev, args->size);
+		if (IS_ERR(mem))
+			return PTR_ERR(mem);
+		bo = to_thames_bo(&mem->base);
+	} else {
+		bo = thames_gem_create_shmem(ddev, args->size);
+		if (IS_ERR(bo))
+			return PTR_ERR(bo);
+	}
+
+	bo->flags = args->flags;
+
+	ret = drm_gem_handle_create(file, &bo->base.base, &args->handle);
+	drm_gem_object_put(&bo->base.base);
+	if (ret) {
+		dev_err(ddev->dev, "Failed to create gem handle: %d", ret);
+		return ret;
+	}
+
+	args->size = bo->base.base.size;
+
+	return 0;
+}
+
+static int thames_bind_exec_op(struct drm_file *file, const struct drm_thames_bind_op *op)
+{
+	struct thames_file_priv *priv = file->driver_priv;
+	struct drm_gem_object *obj;
+	int ret;
+
+	if (op->flags & ~DRM_THAMES_BIND_OP_UNBIND)
+		return -EINVAL;
+
+	if ((op->flags & DRM_THAMES_BIND_OP_UNBIND) && op->dsp_va)
+		return -EINVAL;
+
+	obj = drm_gem_object_lookup(file, op->handle);
+	if (!obj)
+		return -ENOENT;
+
+	if (op->flags & DRM_THAMES_BIND_OP_UNBIND)
+		ret = thames_bo_unbind(priv, to_thames_bo(obj));
+	else
+		ret = thames_bo_bind(priv, to_thames_bo(obj), op->dsp_va);
+
+	drm_gem_object_put(obj);
+
+	return ret;
+}
+
+int thames_ioctl_bo_bind(struct drm_device *ddev, void *data, struct drm_file *file)
+{
+	struct drm_thames_bo_bind *args = data;
+	struct drm_thames_bind_op *ops;
+	int ret = 0;
+	u32 i;
+
+	if (args->flags || args->pad)
+		return -EINVAL;
+
+	ops = thames_get_uobj_array(&args->ops, sizeof(*ops), sizeof(*ops));
+	if (IS_ERR(ops))
+		return PTR_ERR(ops);
+
+	for (i = 0; i < args->ops.count; i++) {
+		ret = thames_bind_exec_op(file, &ops[i]);
+		if (ret) {
+			args->ops.count = i;
+			break;
+		}
+	}
+
+	kvfree(ops);
+
+	return ret;
+}
+
+int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_file *file)
+{
+	struct drm_thames_bo_mmap_offset *args = data;
+	struct drm_gem_object *obj;
+
+	if (args->pad)
+		return -EINVAL;
+
+	obj = drm_gem_object_lookup(file, args->handle);
+	if (!obj)
+		return -ENOENT;
+
+	args->offset = drm_vma_node_offset_addr(&obj->vma_node);
+	drm_gem_object_put(obj);
+
+	return 0;
+}
+
diff --git a/drivers/accel/thames/thames_gem.h b/drivers/accel/thames/thames_gem.h
index 09ac1b50d21a..c9d0e170dad3 100644
--- a/drivers/accel/thames/thames_gem.h
+++ b/drivers/accel/thames/thames_gem.h
@@ -12,6 +12,13 @@
 
 #include "thames_abi.h"
 
+/*
+ * The windows sit below 4 GiB because Kernels load their addresses with 32-bit
+ * moves.
+ */
+#define THAMES_BIND_VA_START	SZ_4G
+#define THAMES_BIND_VA_END	THAMES_USER_REGION_BASE
+
 struct thames_core;
 struct thames_device;
 struct thames_file_priv;
@@ -34,9 +41,18 @@ struct thames_gem_object {
 	dma_addr_t alloc_dma;
 	size_t alloc_size;
 
+	struct list_head bindings;
+
+	u32 flags;
 };
 
 
+int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file *file);
+
+int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_file *file);
+
+int thames_ioctl_bo_bind(struct drm_device *ddev, void *data, struct drm_file *file);
+
 int thames_context_create(struct thames_file_priv *priv);
 
 void thames_context_destroy(struct thames_file_priv *priv);
@@ -44,6 +60,11 @@ void thames_context_rewrite_entries(struct thames_core *core);
 
 void thames_context_va_fini(struct thames_file_priv *priv);
 
+struct thames_gem_object *thames_context_find_bo(struct thames_file_priv *priv, u64 start,
+						 u64 end, u64 *offset);
+
+u64 thames_bo_dsp_va(struct thames_gem_object *bo, struct thames_file_priv *priv);
+
 static inline struct thames_gem_object *to_thames_bo(struct drm_gem_object *obj)
 {
 	return container_of(obj, struct thames_gem_object, base.base);
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
index af427a29b585..ea556b48f1d7 100644
--- a/include/uapi/drm/thames_accel.h
+++ b/include/uapi/drm/thames_accel.h
@@ -45,15 +45,116 @@ extern "C" {
  * definitions instead.
  */
 enum drm_thames_ioctl_id {
+	/** @DRM_THAMES_BO_CREATE: Create a buffer object. */
+	DRM_THAMES_BO_CREATE,
+
+	/**
+	 * @DRM_THAMES_BO_MMAP_OFFSET: Get the file offset to pass to
+	 * mmap to map a GEM object.
+	 */
+	DRM_THAMES_BO_MMAP_OFFSET,
+
 	/** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
 	DRM_THAMES_DEV_QUERY,
 
+	/**
+	 * @DRM_THAMES_BO_BIND: Bind buffers into the file's Context at the
+	 * addresses the client chooses, or unbind them.
+	 */
+	DRM_THAMES_BO_BIND,
 };
 
 /**
  * DOC: IOCTL arguments
  */
 
+/**
+ * enum drm_thames_bo_flags - Buffer object flags, passed at creation time.
+ *
+ * A buffer with neither permission flag is user read-write and execute-never;
+ * DRM_THAMES_BO_CONTIGUOUS chooses the backing, not the permissions.
+ */
+enum drm_thames_bo_flags {
+	/**
+	 * @DRM_THAMES_BO_READ_ONLY: Mapped user read-only in the Context.  For
+	 * the params, the coefficients and the constants, so that a Kernel
+	 * cannot alter the inputs of a Job's next run.
+	 */
+	DRM_THAMES_BO_READ_ONLY = 1 << 0,
+
+	/**
+	 * @DRM_THAMES_BO_EXECUTABLE: Mapped user read-execute and not
+	 * writable.  Required of a Job's program.
+	 */
+	DRM_THAMES_BO_EXECUTABLE = 1 << 1,
+
+	/**
+	 * @DRM_THAMES_BO_CONTIGUOUS: Backed by one physically contiguous
+	 * allocation, so that a transfer of a Job can name it.  For the buffers
+	 * a DRU transfer reads or writes; any other buffer is backed page by
+	 * page.
+	 */
+	DRM_THAMES_BO_CONTIGUOUS = 1 << 2,
+};
+
+/**
+ * struct drm_thames_bo_create - Arguments passed to DRM_IOCTL_THAMES_BO_CREATE.
+ *
+ * Returns a handle and no address; the buffer is bound in no Context until
+ * DRM_IOCTL_THAMES_BO_BIND.
+ */
+struct drm_thames_bo_create {
+	/**
+	 * @size: Requested size for the object
+	 *
+	 * The (page-aligned) allocated size for the object will be returned.
+	 */
+	__u64 size;
+
+	/**
+	 * @handle: MBZ on input; returns the object's handle, which is nonzero.
+	 */
+	__u32 handle;
+
+	/** @flags: Combination of &enum drm_thames_bo_flags. */
+	__u32 flags;
+};
+
+/**
+ * struct drm_thames_bo_mmap_offset - Arguments passed to DRM_IOCTL_THAMES_BO_MMAP_OFFSET.
+ */
+struct drm_thames_bo_mmap_offset {
+	/** @handle: Handle of the object we want an mmap offset for. */
+	__u32 handle;
+
+	/** @pad: MBZ. */
+	__u32 pad;
+
+	/** @offset: The fake offset to use for subsequent mmap calls. */
+	__u64 offset;
+};
+
+/**
+ * struct drm_thames_obj_array - An array of versioned objects passed by pointer.
+ *
+ * The stride is the size of the object userspace was built against, so a
+ * structure can grow at its end under a minor bump.
+ * The kernel reads each element with copy_struct_from_user(): a stride smaller
+ * than its own size has the rest read as zero, down to the size the structure
+ * had when it entered the uAPI and no smaller; a larger stride is accepted only
+ * if the bytes the kernel does not know are zero.
+ */
+struct drm_thames_obj_array {
+	/** @stride: Size of one element in bytes. */
+	__u32 stride;
+
+	/** @count: Number of elements. */
+	__u32 count;
+
+	/** @array: User pointer to the first element. */
+	__u64 array;
+};
+
 /**
  * struct drm_thames_range - A range of the Context's address space.
  */
@@ -65,6 +166,73 @@ struct drm_thames_range {
 	__u64 size;
 };
 
+/**
+ * DOC: Bind operation flags
+ *
+ * - %DRM_THAMES_BIND_OP_UNBIND: Unbind the buffer from the Context instead
+ *   of binding it; &drm_thames_bind_op.dsp_va must be 0.
+ */
+#define DRM_THAMES_BIND_OP_UNBIND	(1 << 0)
+
+/**
+ * struct drm_thames_bind_op - One operation of DRM_IOCTL_THAMES_BO_BIND
+ *
+ * A bind maps the whole buffer at @dsp_va in every page table of the file's
+ * Context, with the permissions of its creation flags; an unbind removes the
+ * mapping.  A buffer is bound at most once in a Context, at one address, over
+ * no range another buffer is bound in.  A buffer shared through a dma-buf may
+ * be bound in each importing file's Context, each at its own address.  The
+ * kernel maps 2 MiB blocks where both the address and the buffer's pages are
+ * 2 MiB aligned, so a client aligns a buffer of 2 MiB or more to 2 MiB.
+ *
+ * Unbinding a buffer a submitted Job still reaches through the MMU is the
+ * client's error: the mapping goes at once, and the Job faults in the client's
+ * own Context as on any other fault.  A Job's transfers keep the buffers they
+ * reach, whether bound or not, until the Job is destroyed.  Freeing a bound
+ * buffer unbinds it.
+ */
+struct drm_thames_bind_op {
+	/** @handle: The buffer. */
+	__u32 handle;
+
+	/**
+	 * @flags: 0 to bind the buffer at @dsp_va,
+	 * %DRM_THAMES_BIND_OP_UNBIND to unbind it.
+	 */
+	__u32 flags;
+
+	/**
+	 * @dsp_va: Where the buffer is bound in the DSP's address space,
+	 * page-aligned, the buffer wholly inside
+	 * &drm_thames_caps.dsp_va_range; 0 for an unbind.
+	 *
+	 * The address the core's MMU translates through the Context's page
+	 * table, the same on every core.  It is the number userspace embeds
+	 * in params and Kernel code.  It is neither the bus address the page
+	 * table entries hold nor the CPU address mmap returns.
+	 */
+	__u64 dsp_va;
+};
+
+/**
+ * struct drm_thames_bo_bind - Arguments passed to DRM_IOCTL_THAMES_BO_BIND
+ *
+ * The operations are applied in order and each is done when the call
+ * returns.  The call stops at the first operation that fails and returns its
+ * error; the operations before it hold, and @ops.count is set to their
+ * number.
+ */
+struct drm_thames_bo_bind {
+	/** @flags: MBZ. */
+	__u32 flags;
+
+	/** @pad: MBZ. */
+	__u32 pad;
+
+	/** @ops: Array of &struct drm_thames_bind_op, applied in order. */
+	struct drm_thames_obj_array ops;
+};
+
 /**
  * enum drm_thames_dev_query_type - Query type
  *
@@ -126,9 +294,16 @@ struct drm_thames_caps {
 	 */
 	__u64 dru_realtime_dsp_va;
 
+	/**
+	 * @dsp_va_range: The span of the Context's address space the client
+	 * binds its buffers in with DRM_IOCTL_THAMES_BO_BIND.  The windows
+	 * above sit below it and @user_region above it.
+	 */
+	struct drm_thames_range dsp_va_range;
+
 	/**
 	 * @user_region: Where the kernel driver maps the Context's user stack
-	 * and save page.
+	 * and save page, outside @dsp_va_range.
 	 */
 	struct drm_thames_range user_region;
 
@@ -189,8 +364,14 @@ struct drm_thames_dev_query {
 			   struct drm_thames_ ## __type)
 
 enum {
+	DRM_IOCTL_THAMES_BO_CREATE =
+		DRM_IOCTL_THAMES(WR, BO_CREATE, bo_create),
+	DRM_IOCTL_THAMES_BO_MMAP_OFFSET =
+		DRM_IOCTL_THAMES(WR, BO_MMAP_OFFSET, bo_mmap_offset),
 	DRM_IOCTL_THAMES_DEV_QUERY =
 		DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
+	DRM_IOCTL_THAMES_BO_BIND =
+		DRM_IOCTL_THAMES(WR, BO_BIND, bo_bind),
 };
 
 #if defined(__cplusplus)

-- 
2.54.0

[PATCH v3 08/16] accel/thames: Add support for submitting DMA operations

[Tomeu Vizoso]

The C7x has a DMA engine (DRU, Data Routing Unit) that can be used to
asynchronously copy data around, while compute happens.

The DRU does not go through the MMU, so its records hold physical
addresses. To preserve isolation among clients, a record names the
Context's own DSP addresses; the kernel bounds each side of the
transfer, checks it against the Context's mappings, translates the two
addresses to physical once and keeps the translated records in memory
no Context reaches. The next commit's job creation IOCTL is where the
records enter.

The kernel driver also verifies that a malicious or defective client
cannot read or write memory other than its own.

A Job's translated records form a table in a pool inside the IPC
memory, where no Context reaches them, carved by a bitmap in 128-byte
lines. A Job carries up to 512 transfers, and no group may hold more
transfers on one channel than the DRU's queue depth, which the firmware
reports. An L2 side is kept as an offset into its aperture and rebased
for the core that runs the Job when the table is written, since the
DRU walks physical addresses and each core's L2 has its own. At attach
the driver checks that the pool holds a 512-transfer table and that
the firmware's DRU capabilities include type 8, and rewrites the tables
of the Jobs it keeps when it attaches to a core again.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 drivers/accel/thames/Makefile           |   3 +-
 drivers/accel/thames/thames_core.c      |  18 ++
 drivers/accel/thames/thames_device.h    |   8 +
 drivers/accel/thames/thames_drv.c       |  17 ++
 drivers/accel/thames/thames_transfers.c | 405 ++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_transfers.h |  52 ++++
 include/uapi/drm/thames_accel.h         |  61 +++++
 7 files changed, 563 insertions(+), 1 deletion(-)

diff --git a/drivers/accel/thames/Makefile b/drivers/accel/thames/Makefile
index bb81fcd58927..143a829f24cf 100644
--- a/drivers/accel/thames/Makefile
+++ b/drivers/accel/thames/Makefile
@@ -5,4 +5,5 @@ obj-$(CONFIG_DRM_ACCEL_THAMES) := thames.o
 thames-y := \
 	thames_core.o \
 	thames_drv.o \
-	thames_gem.o
+	thames_gem.o \
+	thames_transfers.o
diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
index 1c983580b62a..dff76abd8951 100644
--- a/drivers/accel/thames/thames_core.c
+++ b/drivers/accel/thames/thames_core.c
@@ -25,6 +25,7 @@
 #include "thames_core.h"
 #include "thames_device.h"
 #include "thames_gem.h"
+#include "thames_transfers.h"
 
 #define THAMES_RESTART_TIMEOUT_MS	5000
 
@@ -132,6 +133,22 @@ static int thames_core_read_caps(struct thames_core *core, struct thames_handsha
 		return -EINVAL;
 	}
 
+	if (THAMES_TRANSFER_TABLE_SIZE(THAMES_JOB_MAX_TRANSFERS, THAMES_JOB_MAX_TRANSFERS) >
+	    caps->transfer_table_pool_size) {
+		dev_err(core->dev,
+			"Firmware transfer table pool of 0x%x bytes is smaller than a %u-transfer table\n",
+			caps->transfer_table_pool_size, THAMES_JOB_MAX_TRANSFERS);
+		return -EINVAL;
+	}
+
+	if (caps->dru_capabilities &&
+	    (caps->dru_capabilities & THAMES_TRANSFER_TYPES) != THAMES_TRANSFER_TYPES) {
+		dev_err(core->dev,
+			"Firmware DRU capabilities 0x%llx lack the transfer types 0x%lx the driver accepts\n",
+			caps->dru_capabilities, THAMES_TRANSFER_TYPES);
+		return -EINVAL;
+	}
+
 	dru_kernel_offset = (u64)dru_kernel_channel_first * caps->dru_channel_stride;
 	dru_kernel_size = (u64)caps->dru_channel_count * caps->dru_channel_stride;
 	if (caps->dru_channel_count > THAMES_MAX_DRU_CHANNELS ||
@@ -618,6 +635,7 @@ int thames_core_resume(struct thames_core *core)
 	core->ring_tail = head;
 
 	thames_context_rewrite_entries(core);
+	thames_transfers_rewrite(core);
 	goto restart;
 
 offline:
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
index a1a54d974359..e183c78e2acf 100644
--- a/drivers/accel/thames/thames_device.h
+++ b/drivers/accel/thames/thames_device.h
@@ -55,6 +55,14 @@ struct thames_device {
 	u32 asid_next;
 	struct mutex asid_lock;
 
+	/*
+	 * One map for every core: a transfer table sits at the same offset in
+	 * each core's pool.
+	 */
+	unsigned long *transfer_table_pool_map;
+	struct mutex transfer_table_pool_lock;
+
+	struct list_head transfer_tables;
 	struct list_head contexts;
 	struct mutex contexts_lock;
 
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index a17e7a90f7c1..030c60963a0a 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -23,6 +23,7 @@
 #include "thames_core.h"
 #include "thames_drv.h"
 #include "thames_gem.h"
+#include "thames_transfers.h"
 
 static struct platform_device *drm_dev;
 static struct thames_device *tdev;
@@ -152,6 +153,7 @@ static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query
 		.l2_size = reported->l2_size,
 		.dru_channel_count = reported->dru_channel_count,
 		.dru_queue_depth = reported->dru_queue_depth,
+		.max_transfers = THAMES_JOB_MAX_TRANSFERS,
 		.l2_main_dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2SRAM),
 		.l2_aux.dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2AUX),
 		.l2_aux.size = reported->window_sizes[THAMES_WINDOW_L2AUX],
@@ -161,6 +163,7 @@ static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query
 		.user_region.dsp_va = THAMES_USER_REGION_BASE,
 		.user_region.size = THAMES_USER_REGION_SIZE,
 		.hardware_id = reported->hardware_id,
+		.transfer_types = THAMES_TRANSFER_TYPES,
 		.dru_capabilities = reported->dru_capabilities,
 	};
 
@@ -269,6 +272,10 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	err = drmm_mutex_init(&tdev->ddev, &tdev->transfer_table_pool_lock);
+	if (err)
+		return ERR_PTR(err);
+
 	err = drmm_mutex_init(&tdev->ddev, &tdev->asid_lock);
 	if (err)
 		return ERR_PTR(err);
@@ -281,6 +288,7 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	INIT_LIST_HEAD(&tdev->transfer_tables);
 	INIT_LIST_HEAD(&tdev->contexts);
 
 	tdev->asid_next = 1;
@@ -349,6 +357,15 @@ static int thames_platform_probe(struct platform_device *pdev)
 		goto err_fini_dev;
 	}
 
+	tdev->transfer_table_pool_map =
+		devm_bitmap_zalloc(&pdev->dev,
+				   tdev->caps.transfer_table_pool_size / THAMES_ABI_LINE_SIZE,
+				   GFP_KERNEL);
+	if (!tdev->transfer_table_pool_map) {
+		err = -ENOMEM;
+		goto err_fini_cores;
+	}
+
 	err = drm_dev_register(&tdev->ddev, 0);
 	if (err)
 		goto err_fini_cores;
diff --git a/drivers/accel/thames/thames_transfers.c b/drivers/accel/thames/thames_transfers.c
new file mode 100644
index 000000000000..aad4542d5cb9
--- /dev/null
+++ b/drivers/accel/thames/thames_transfers.c
@@ -0,0 +1,405 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#include <linux/bitfield.h>
+#include <linux/bitmap.h>
+#include <linux/bits.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include <drm/drm_file.h>
+#include <drm/drm_gem.h>
+
+#include "thames_core.h"
+#include "thames_device.h"
+#include "thames_drv.h"
+#include "thames_gem.h"
+#include "thames_transfers.h"
+
+/* SPRUIL1D 10.4.3.3. */
+#define TR_FLAGS		0
+#define TR_ICNT01		1
+#define TR_ADDR_LO		2
+#define TR_ADDR_HI		3
+#define TR_DIM1			4
+#define TR_ICNT23		5
+#define TR_DIM2			6
+#define TR_DIM3			7
+#define TR_FMTFLAGS		8
+#define TR_DDIM1		9
+#define TR_DADDR_LO		10
+#define TR_DADDR_HI		11
+#define TR_DDIM2		12
+#define TR_DDIM3		13
+#define TR_DICNT01		14
+#define TR_DICNT23		15
+
+#define TR_FLAGS_TYPE		GENMASK(3, 0)
+#define TR_FLAGS_STATIC		BIT(4)
+#define TR_FLAGS_TRIGGER0	GENMASK(9, 8)
+#define TR_FLAGS_TRIGGER1	GENMASK(13, 12)
+#define TR_FLAGS_TRIGGER1_TYPE	GENMASK(15, 14)
+#define TR_FLAGS_CFG		GENMASK(31, 24)
+#define TR_FLAGS_SUPR_EVT	BIT(26)
+
+#define TR_TRIGGER_NONE		0
+#define TR_TRIGGER_GLOBAL0	1
+
+#define THAMES_TRANSFER_VA_LIMIT	BIT_ULL(48)
+
+/*
+ * Only global trigger 0 of the record's own channel: any other trigger lets a
+ * record wait on another channel's event; indirect addressing would fetch
+ * addresses never checked.
+ */
+static bool thames_transfer_flags_ok(u32 flags)
+{
+	if (!(THAMES_TRANSFER_TYPES & BIT(FIELD_GET(TR_FLAGS_TYPE, flags))))
+		return false;
+
+	if (flags & TR_FLAGS_STATIC)
+		return false;
+
+	if (FIELD_GET(TR_FLAGS_TRIGGER0, flags) != TR_TRIGGER_NONE &&
+	    FIELD_GET(TR_FLAGS_TRIGGER0, flags) != TR_TRIGGER_GLOBAL0)
+		return false;
+
+	if (flags & (TR_FLAGS_TRIGGER1 | TR_FLAGS_TRIGGER1_TYPE))
+		return false;
+
+	return !(flags & TR_FLAGS_CFG & ~TR_FLAGS_SUPR_EVT);
+}
+
+/* SPRUIL1D 10.4.3.3; the footprint includes the strides' gaps. */
+static int thames_transfer_span(u64 base, const u32 icnt[4], const s32 dim[4],
+				u64 *start, u64 *end, u64 *bytes)
+{
+	s64 lo = 0, hi = icnt[0];
+	int i;
+
+	*bytes = 1;
+	for (i = 0; i < 4; i++) {
+		if (!icnt[i])
+			return -EINVAL;
+		*bytes *= icnt[i];
+	}
+
+	for (i = 1; i < 4; i++) {
+		s64 reach = (s64)(icnt[i] - 1) * dim[i];
+
+		if (reach < 0)
+			lo += reach;
+		else
+			hi += reach;
+	}
+
+	if (base >= THAMES_TRANSFER_VA_LIMIT || (s64)base + lo < 0)
+		return -EFAULT;
+
+	*start = base + lo;
+	*end = base + hi;
+	return 0;
+}
+
+static void thames_transfer_hold_bo(struct thames_transfer_table *table,
+				    struct thames_gem_object *bo)
+{
+	struct drm_gem_object *obj = &bo->base.base;
+	u32 i;
+
+	for (i = 0; i < table->bo_count; i++) {
+		if (table->bos[i] == obj) {
+			drm_gem_object_put(obj);
+			return;
+		}
+	}
+
+	table->bos[table->bo_count++] = obj;
+}
+
+/*
+ * An L2 side is kept as an offset into its aperture and rebased per core when
+ * the transfer table is written: the DRU walks physical addresses.
+ */
+static int thames_transfer_side(struct thames_file_priv *file_priv, u64 start, u64 end, bool dest,
+				struct thames_transfer_table *table, u8 *side, u64 *addr)
+{
+	static const enum thames_window l2[] = { THAMES_WINDOW_L2SRAM, THAMES_WINDOW_L2AUX };
+	struct thames_device *tdev = file_priv->tdev;
+	struct thames_gem_object *bo;
+	unsigned int w;
+	u64 offset;
+
+	for (w = 0; w < ARRAY_SIZE(l2); w++) {
+		u64 va = thames_window_va(tdev, l2[w]);
+
+		if (start >= va && end <= va + tdev->caps.window_sizes[l2[w]]) {
+			*side = l2[w];
+			*addr = start - va;
+			return 0;
+		}
+	}
+
+	bo = thames_context_find_bo(file_priv, start, end, &offset);
+	if (!bo)
+		return -EFAULT;
+
+	thames_transfer_hold_bo(table, bo);
+
+	if (dest && (bo->flags & (DRM_THAMES_BO_READ_ONLY | DRM_THAMES_BO_EXECUTABLE)))
+		return -EFAULT;
+
+	/*
+	 * No IOMMU: a contiguous buffer's DMA address is its physical address.
+	 */
+	if (!(bo->flags & DRM_THAMES_BO_CONTIGUOUS))
+		return -EFAULT;
+
+	*side = THAMES_TRANSFER_SIDE_DDR;
+	*addr = bo->base.dma_addr + offset;
+	return 0;
+}
+
+static int thames_transfer_translate(struct thames_file_priv *file_priv,
+				     const struct drm_thames_transfer *t,
+				     struct thames_transfer_table *table,
+				     struct thames_transfer *out)
+{
+	const u32 *w = t->words;
+	u32 icnt[4], dicnt[4];
+	s32 dim[4] = { 0 }, ddim[4] = { 0 };
+	u64 src, dst, start, end, bytes, dbytes, addr;
+	int ret;
+
+	if (!thames_transfer_flags_ok(w[TR_FLAGS]) || w[TR_FMTFLAGS])
+		return -EINVAL;
+
+	icnt[0] = w[TR_ICNT01] & 0xffff;
+	icnt[1] = w[TR_ICNT01] >> 16;
+	icnt[2] = w[TR_ICNT23] & 0xffff;
+	icnt[3] = w[TR_ICNT23] >> 16;
+	dim[1] = w[TR_DIM1];
+	dim[2] = w[TR_DIM2];
+	dim[3] = w[TR_DIM3];
+	src = (u64)w[TR_ADDR_HI] << 32 | w[TR_ADDR_LO];
+
+	dicnt[0] = w[TR_DICNT01] & 0xffff;
+	dicnt[1] = w[TR_DICNT01] >> 16;
+	dicnt[2] = w[TR_DICNT23] & 0xffff;
+	dicnt[3] = w[TR_DICNT23] >> 16;
+	ddim[1] = w[TR_DDIM1];
+	ddim[2] = w[TR_DDIM2];
+	ddim[3] = w[TR_DDIM3];
+	dst = (u64)w[TR_DADDR_HI] << 32 | w[TR_DADDR_LO];
+
+	memcpy(out->record.words, w, sizeof(out->record.words));
+	out->channel = t->channel;
+
+	ret = thames_transfer_span(src, icnt, dim, &start, &end, &bytes);
+	if (ret)
+		return ret;
+	ret = thames_transfer_side(file_priv, start, end, false, table, &out->side[0], &addr);
+	if (ret)
+		return ret;
+	addr += src - start;
+	out->record.words[TR_ADDR_LO] = lower_32_bits(addr);
+	out->record.words[TR_ADDR_HI] = upper_32_bits(addr);
+
+	ret = thames_transfer_span(dst, dicnt, ddim, &start, &end, &dbytes);
+	if (ret)
+		return ret;
+	if (dbytes != bytes)
+		return -EINVAL;
+	ret = thames_transfer_side(file_priv, start, end, true, table, &out->side[1], &addr);
+	if (ret)
+		return ret;
+	addr += dst - start;
+	out->record.words[TR_DADDR_LO] = lower_32_bits(addr);
+	out->record.words[TR_DADDR_HI] = upper_32_bits(addr);
+
+	return 0;
+}
+
+static int thames_transfers_alloc_table(struct thames_device *tdev,
+					struct thames_transfer_table *table)
+{
+	const unsigned long lines = tdev->caps.transfer_table_pool_size / THAMES_ABI_LINE_SIZE;
+	unsigned long start, nr = table->size / THAMES_ABI_LINE_SIZE;
+
+	guard(mutex)(&tdev->transfer_table_pool_lock);
+
+	start = bitmap_find_next_zero_area(tdev->transfer_table_pool_map, lines, 0, nr, 0);
+	if (start + nr > lines)
+		return -ENOSPC;
+
+	bitmap_set(tdev->transfer_table_pool_map, start, nr);
+	table->offset = start * THAMES_ABI_LINE_SIZE;
+	list_add_tail(&table->node, &tdev->transfer_tables);
+	return 0;
+}
+
+void thames_transfers_free(struct thames_device *tdev, struct thames_transfer_table *table)
+{
+	u32 i;
+
+	for (i = 0; i < table->bo_count; i++)
+		drm_gem_object_put(table->bos[i]);
+	kvfree(table->bos);
+	table->bos = NULL;
+	table->bo_count = 0;
+
+	if (!table->size)
+		return;
+
+	scoped_guard(mutex, &tdev->transfer_table_pool_lock) {
+		bitmap_clear(tdev->transfer_table_pool_map, table->offset / THAMES_ABI_LINE_SIZE,
+			     table->size / THAMES_ABI_LINE_SIZE);
+		list_del(&table->node);
+	}
+
+	kvfree(table->records);
+	kvfree(table->group_counts);
+	table->records = NULL;
+	table->group_counts = NULL;
+	table->size = 0;
+}
+
+/*
+ * Written before the Job's handle reaches userspace and never changed while the
+ * Job lives.
+ */
+static void thames_transfers_write_table(struct thames_core *core,
+					 const struct thames_transfer_table *table)
+{
+	static const unsigned int addr_word[2] = { TR_ADDR_LO, TR_DADDR_LO };
+	const struct thames_transfer *records = table->records;
+	const u16 *group_counts = table->group_counts;
+	void *base = core->ipc->transfer_table_pool + table->offset;
+	struct thames_dru_record *out = base;
+	u8 *channels = base + THAMES_TRANSFER_TABLE_CHANNELS_OFFSET(table->count);
+	u32 i, s;
+
+	for (i = 0; i < table->count; i++) {
+		struct thames_dru_record record = records[i].record;
+
+		for (s = 0; s < 2; s++) {
+			u32 *lo = &record.words[addr_word[s]];
+			u64 addr;
+
+			if (records[i].side[s] == THAMES_TRANSFER_SIDE_DDR)
+				continue;
+
+			addr = ((u64)lo[1] << 32 | lo[0]) +
+			       thames_core_window(core, records[i].side[s])->start;
+			lo[0] = lower_32_bits(addr);
+			lo[1] = upper_32_bits(addr);
+		}
+
+		memcpy(&out[i], &record, sizeof(record));
+		channels[i] = records[i].channel;
+	}
+
+	memcpy(base + THAMES_TRANSFER_TABLE_GROUPS_OFFSET(table->count), group_counts,
+	       table->group_count * sizeof(u16));
+}
+
+/* Same offsets, so the Ring entries of the Jobs still queued stay valid. */
+void thames_transfers_rewrite(struct thames_core *core)
+{
+	struct thames_device *tdev = core->tdev;
+	struct thames_transfer_table *table;
+
+	guard(mutex)(&tdev->transfer_table_pool_lock);
+
+	list_for_each_entry(table, &tdev->transfer_tables, node)
+		thames_transfers_write_table(core, table);
+}
+
+int thames_transfers_build(struct drm_file *file, const struct drm_thames_obj_array *transfers,
+			   struct thames_transfer_table *table)
+{
+	struct thames_file_priv *file_priv = file->driver_priv;
+	struct thames_device *tdev = file_priv->tdev;
+	const struct thames_caps *caps = &tdev->caps;
+	u16 per_channel[THAMES_MAX_DRU_CHANNELS] = { 0 };
+	u16 *group_counts = NULL;
+	u16 *group = NULL;
+	struct drm_thames_transfer *in;
+	struct thames_transfer *out;
+	u32 i;
+	int ret;
+
+	memset(table, 0, sizeof(*table));
+
+	if (!transfers->count)
+		return 0;
+
+	if (transfers->count > THAMES_JOB_MAX_TRANSFERS)
+		return -EINVAL;
+
+	in = thames_get_uobj_array(transfers, offsetofend(struct drm_thames_transfer, words),
+				   sizeof(*in));
+	if (IS_ERR(in))
+		return PTR_ERR(in);
+
+	out = kvcalloc(transfers->count, sizeof(*out), GFP_KERNEL);
+	group_counts = kvcalloc(transfers->count, sizeof(u16), GFP_KERNEL);
+	table->bos = kvcalloc(2 * transfers->count, sizeof(*table->bos), GFP_KERNEL);
+	if (!out || !group_counts || !table->bos) {
+		ret = -ENOMEM;
+		goto err_free;
+	}
+
+	for (i = 0; i < transfers->count; i++) {
+		if (in[i].flags & DRM_THAMES_TRANSFER_GROUP) {
+			if (!group) {
+				ret = -EINVAL;
+				goto err_free;
+			}
+			memset(per_channel, 0, sizeof(per_channel));
+			group = NULL;
+		}
+
+		if ((in[i].flags & ~DRM_THAMES_TRANSFER_GROUP) ||
+		    in[i].channel >= caps->dru_channel_count ||
+		    ++per_channel[in[i].channel] > caps->dru_queue_depth) {
+			ret = -EINVAL;
+			goto err_free;
+		}
+
+		ret = thames_transfer_translate(file_priv, &in[i], table, &out[i]);
+		if (ret)
+			goto err_free;
+
+		if (!group)
+			group = &group_counts[table->group_count++];
+		(*group)++;
+	}
+
+	table->count = transfers->count;
+	table->size = THAMES_TRANSFER_TABLE_SIZE(table->count, table->group_count);
+
+	ret = thames_transfers_alloc_table(tdev, table);
+	if (ret) {
+		table->size = 0;
+		goto err_free;
+	}
+
+	table->records = out;
+	table->group_counts = group_counts;
+
+	for (i = 0; i < tdev->num_cores; i++)
+		thames_transfers_write_table(&tdev->cores[i], table);
+
+	kvfree(in);
+	return 0;
+
+err_free:
+	thames_transfers_free(tdev, table);
+	memset(table, 0, sizeof(*table));
+	kvfree(group_counts);
+	kvfree(out);
+	kvfree(in);
+	return ret;
+}
diff --git a/drivers/accel/thames/thames_transfers.h b/drivers/accel/thames/thames_transfers.h
new file mode 100644
index 000000000000..67963cbcb621
--- /dev/null
+++ b/drivers/accel/thames/thames_transfers.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_TRANSFERS_H__
+#define __THAMES_TRANSFERS_H__
+
+#include <linux/bits.h>
+
+#include <drm/drm_gem.h>
+#include <drm/thames_accel.h>
+
+#include "thames_abi.h"
+
+struct drm_file;
+struct thames_core;
+struct thames_device;
+
+#define THAMES_TRANSFER_SIDE_DDR	0xff
+
+/*
+ * SPRUIL1D 10.4.3.3: type 8, the 4D block move, the one type whose footprint
+ * the transfer checks bound.
+ */
+#define THAMES_TRANSFER_TYPES		BIT(8)
+#define THAMES_JOB_MAX_TRANSFERS	512
+
+struct thames_transfer {
+	u8 channel;
+	u8 side[2];
+	struct thames_dru_record record;
+};
+
+struct thames_transfer_table {
+	u32 offset;
+	u32 size;
+	u32 count;
+	u32 group_count;
+	struct drm_gem_object **bos;
+	u32 bo_count;
+	struct thames_transfer *records;
+	u16 *group_counts;
+	struct list_head node;
+};
+
+int thames_transfers_build(struct drm_file *file, const struct drm_thames_obj_array *transfers,
+			   struct thames_transfer_table *table);
+
+void thames_transfers_free(struct thames_device *tdev, struct thames_transfer_table *table);
+
+void thames_transfers_rewrite(struct thames_core *core);
+
+#endif
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
index ea556b48f1d7..5671cffaf8f9 100644
--- a/include/uapi/drm/thames_accel.h
+++ b/include/uapi/drm/thames_accel.h
@@ -348,6 +348,67 @@ struct drm_thames_dev_query {
 	__u64 pointer;
 };
 
+/*
+ * Starts a new transfer group.  The transfers are cut into groups, in array
+ * order: the first group starts at the first transfer, each later group at a
+ * transfer carrying this flag.  The firmware submits the first group before it
+ * enters the Kernel, and each later group, whole, when the Kernel asks for it
+ * with system call 2.  No group may hold more transfers on one channel than
+ * &drm_thames_caps.dru_queue_depth.  Never on the first transfer.  A Job
+ * without the flag is one group.
+ */
+#define DRM_THAMES_TRANSFER_GROUP	(1 << 0)
+
+/**
+ * struct drm_thames_transfer - One DRU transfer of a Job
+ *
+ * @words is a K3 DMA transfer record as the DRU takes it (SPRUIL1D 10.4.3.3,
+ * Table 10-63 for type 15), with its source and destination address fields
+ * holding a dsp_va of the Context.  The kernel reads the type from the flags
+ * word, computes the byte footprint of each side from the counts, strides and
+ * addressing mode of every type and flag it knows how to bound, checks the
+ * source against readable mappings and the destination against writable ones,
+ * and refuses any type, flag or mode it cannot bound; the indirect-address
+ * flags are always refused, since they would let a record fetch addresses the
+ * kernel never checked.  The kernel translates the two addresses to physical
+ * once, at Job creation, and keeps the translated records where no Context
+ * reaches them; the firmware submits the first group before it enters the
+ * Kernel and each later group when the Kernel asks for it (see
+ * DRM_THAMES_TRANSFER_GROUP), and waits for the channels to drain after the
+ * Kernel returns, before the Job's fence signals.  This kernel and firmware
+ * accept type 8, the 4D block move.  The kernel refuses a trigger other than
+ * global trigger 0 (the one the firmware raises by software), a nonzero format
+ * word, a zero count and a side whose byte count differs from the other's.  The
+ * record's own flags say which loop level a software trigger advances; a record
+ * with no trigger runs to completion once submitted.
+ * A Kernel triggers and polls its channels through the page at
+ * &drm_thames_caps.dru_realtime_dsp_va, mapped user read-write in every
+ * Context, which reaches the channels of the core the Job runs on.
+ *
+ * A side addresses either a span of one buffer bound in the Context, found by
+ * its dsp_va among all the Context's buffers, the buffer created
+ * DRM_THAMES_BO_CONTIGUOUS (the DRU walks physical addresses linearly, and only
+ * such a buffer is one physical run), the destination in a buffer not created
+ * DRM_THAMES_BO_READ_ONLY or DRM_THAMES_BO_EXECUTABLE; or the L2 main or
+ * auxiliary aperture, at &drm_thames_caps.l2_main_dsp_va and l2_aux, which the
+ * kernel translates to the physical aperture of the core the Job runs on.  The
+ * Job holds a reference on each buffer a transfer resolves to until it is
+ * destroyed and its last submission has completed.
+ */
+struct drm_thames_transfer {
+	/**
+	 * @channel: The DRU channel this transfer queues on, below
+	 * &drm_thames_caps.dru_channel_count.
+	 */
+	__u32 channel;
+
+	/** @flags: DRM_THAMES_TRANSFER_* bits; others MBZ. */
+	__u32 flags;
+
+	/** @words: The transfer record, sixteen 32-bit words, flags first. */
+	__u32 words[16];
+};
+
 /**
  * DRM_IOCTL_THAMES() - Build a thames IOCTL number
  * @__access: Access type. Must be R, W or RW.

-- 
2.54.0

[PATCH v3 09/16] accel/thames: Add IOCTLs for job creation and submission

[Tomeu Vizoso]

A Job is created once from a program, its parameters and its transfers,
checked then, and submitted any number of times; the params buffer's
contents may change between submissions. A submission pushes the Job
onto a core's Ring and rings the doorbell; the firmware moves the Fence
when the Job is done. Each core has a drm-gpu-scheduler instance, with
one credit per Ring entry. A submit call takes an array of Jobs, the
buffers they use for implicit synchronisation, and sync objects to
signal; it waits on none.

The job submission code was initially based on Panfrost.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v3:
- Add Job create/destroy, for validating transfers.
- Add sync objects to the submission IOCTL.
- Allocate the done fence with the Job; run_job allocates nothing
  (Christian König).
v2:
- Add thames_accel.h UAPI header (Robert Nelson).
---
 drivers/accel/thames/Kconfig         |   1 +
 drivers/accel/thames/Makefile        |   3 +
 drivers/accel/thames/thames_core.c   |  36 +-
 drivers/accel/thames/thames_core.h   |  14 +
 drivers/accel/thames/thames_device.h |   6 +
 drivers/accel/thames/thames_drv.c    |  43 ++
 drivers/accel/thames/thames_drv.h    |   7 +
 drivers/accel/thames/thames_job.c    | 869 +++++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_job.h    |  73 +++
 drivers/accel/thames/thames_trace.h  |  77 ++++
 include/uapi/drm/thames_accel.h      | 259 ++++++++++-
 11 files changed, 1381 insertions(+), 7 deletions(-)

diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
index 47a2986fcabd..f2c32ce8d228 100644
--- a/drivers/accel/thames/Kconfig
+++ b/drivers/accel/thames/Kconfig
@@ -6,6 +6,7 @@ config DRM_ACCEL_THAMES
 	depends on TI_K3_DSP_REMOTEPROC
 	depends on REMOTEPROC
 	depends on MAILBOX
+	select DRM_SCHED
 	select DRM_GEM_DMA_HELPER
 	select DRM_GEM_SHMEM_HELPER
 	select IOMMU_IO_PGTABLE_LPAE
diff --git a/drivers/accel/thames/Makefile b/drivers/accel/thames/Makefile
index 143a829f24cf..7bd652795cf5 100644
--- a/drivers/accel/thames/Makefile
+++ b/drivers/accel/thames/Makefile
@@ -6,4 +6,7 @@ thames-y := \
 	thames_core.o \
 	thames_drv.o \
 	thames_gem.o \
+	thames_job.o \
 	thames_transfers.o
+
+CFLAGS_thames_job.o := -I$(src)
diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
index dff76abd8951..85e8f48aec60 100644
--- a/drivers/accel/thames/thames_core.c
+++ b/drivers/accel/thames/thames_core.c
@@ -25,6 +25,7 @@
 #include "thames_core.h"
 #include "thames_device.h"
 #include "thames_gem.h"
+#include "thames_job.h"
 #include "thames_transfers.h"
 
 #define THAMES_RESTART_TIMEOUT_MS	5000
@@ -327,6 +328,7 @@ static void thames_mbox_rx(struct mbox_client *cl, void *mssg)
 
 	dev_dbg(core->dev, "Doorbell on " THAMES_CORE_FMT ": 0x%x\n", THAMES_CORE_ARG(core),
 		(u32)(uintptr_t)mssg);
+	thames_job_complete(core);
 }
 
 /* Embedded in struct thames_core: nothing to free. */
@@ -515,10 +517,18 @@ int thames_core_init(struct thames_core *core)
 	if (err)
 		goto err_timers;
 
-	err = thames_core_request_doorbell(core);
+	/*
+	 * Before the doorbell: requesting it delivers at once a word the core
+	 * wrote while no driver was loaded.
+	 */
+	err = thames_job_init(core);
 	if (err)
 		goto err_unmap;
 
+	err = thames_core_request_doorbell(core);
+	if (err)
+		goto err_job;
+
 	if (rproc->state == RPROC_OFFLINE) {
 		err = rproc_boot(rproc);
 		if (err)
@@ -540,6 +550,8 @@ int thames_core_init(struct thames_core *core)
 
 err_doorbell:
 	thames_core_free_doorbell(core);
+err_job:
+	thames_job_fini(core);
 err_unmap:
 	thames_core_unmap_ipc(core);
 err_timers:
@@ -554,6 +566,7 @@ void thames_core_fini(struct thames_core *core)
 {
 	if (core->mbox_chan)
 		thames_core_free_doorbell(core);
+	thames_job_fini(core);
 	/*
 	 * Drop only our reference and keep the boot count: the core runs on
 	 * for the next load to reattach to; only a reset or a suspend, which
@@ -567,6 +580,21 @@ void thames_core_fini(struct thames_core *core)
 	core->of_node = NULL;
 }
 
+/* The system firmware accepts a stop only of a core in IDLE. */
+bool thames_core_idles(struct thames_core *core)
+{
+	u32 head = READ_ONCE(core->ipc->ring.head);
+	u32 slot = head % THAMES_RING_DEPTH;
+	struct thames_fault_record *record = &core->ipc->fault_records[slot];
+
+	if (head == core->ring_tail)
+		return true;
+
+	/* Read the fault record after the head that says it is there. */
+	rmb();
+	return READ_ONCE(record->kind) == DRM_THAMES_FAULT_FIRMWARE;
+}
+
 /*
  * rproc_shutdown() reaches the system firmware because the core has one power
  * count, remoteproc's auto-boot or this driver's boot: the probe defers while
@@ -577,6 +605,10 @@ int thames_core_suspend(struct thames_core *core)
 	struct rproc *rproc = core->rproc;
 	int err;
 
+	err = thames_job_suspend(core);
+	if (err)
+		return err;
+
 	if (core->suspended)
 		return 0;
 
@@ -589,6 +621,7 @@ int thames_core_suspend(struct thames_core *core)
 		err = -EBUSY;
 	}
 	if (err) {
+		thames_job_resume(core);
 		return err;
 	}
 
@@ -640,5 +673,6 @@ int thames_core_resume(struct thames_core *core)
 
 offline:
 restart:
+	thames_job_resume(core);
 	return err;
 }
diff --git a/drivers/accel/thames/thames_core.h b/drivers/accel/thames/thames_core.h
index 590c5716f7f5..a78414d18ad3 100644
--- a/drivers/accel/thames/thames_core.h
+++ b/drivers/accel/thames/thames_core.h
@@ -10,6 +10,7 @@
 #include <linux/mutex_types.h>
 #include <linux/remoteproc.h>
 
+#include <drm/gpu_scheduler.h>
 
 #include "thames_abi.h"
 
@@ -44,13 +45,25 @@ struct thames_core {
 	struct mbox_chan *mbox_chan;
 
 	u32 ring_tail;
+	struct list_head submitted;
+	struct list_head held;
+
+	struct mutex job_lock;
+
+	spinlock_t fence_lock;
+
 	/*
 	 * @rproc_caller is the task inside the driver's own rproc_boot() or
 	 * rproc_shutdown(), whose subdevice callbacks then do nothing.
 	 */
 	struct task_struct *rproc_caller;
 
+	struct drm_gpu_scheduler sched;
+	u64 fence_context;
+	u64 emit_seqno;
+
 	bool suspended;
+	unsigned long timeout_left;
 
 	struct resource l2sram;
 	struct resource l2aux;
@@ -81,6 +94,7 @@ static inline const struct resource *thames_core_window(const struct thames_core
 
 int thames_core_init(struct thames_core *core);
 void thames_core_fini(struct thames_core *core);
+bool thames_core_idles(struct thames_core *core);
 int thames_core_suspend(struct thames_core *core);
 int thames_core_resume(struct thames_core *core);
 
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
index e183c78e2acf..0a34764885df 100644
--- a/drivers/accel/thames/thames_device.h
+++ b/drivers/accel/thames/thames_device.h
@@ -36,7 +36,10 @@ struct thames_caps {
 struct thames_device {
 	struct drm_device ddev;
 
+	struct mutex sched_lock;
+
 	struct thames_core cores[THAMES_MAX_CORES];
+	struct drm_gpu_scheduler *sched_list[THAMES_MAX_CORES];
 	unsigned int num_cores;
 
 	struct thames_caps caps;
@@ -72,6 +75,9 @@ struct thames_device {
 	 */
 	struct mutex bindings_lock;
 
+	/* Whoever takes a file off the list closes its Jobs. */
+	struct mutex files_lock;
+	struct list_head files;
 };
 
 #define to_thames_device(drm_dev) \
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index 030c60963a0a..fc3328834162 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -23,6 +23,7 @@
 #include "thames_core.h"
 #include "thames_drv.h"
 #include "thames_gem.h"
+#include "thames_job.h"
 #include "thames_transfers.h"
 
 static struct platform_device *drm_dev;
@@ -54,14 +55,23 @@ static int thames_open(struct drm_device *dev, struct drm_file *file)
 
 	file->driver_priv = thames_priv;
 
+	ret = thames_job_open(thames_priv);
+	if (ret)
+		goto err_free;
+
 	ret = thames_context_create(thames_priv);
 	if (ret)
 		goto err_close_job;
 
+	scoped_guard(mutex, &tdev->files_lock)
+		list_add_tail(&thames_priv->file_node, &tdev->files);
+
 	drm_dev_exit(idx);
 	return 0;
 
 err_close_job:
+	thames_job_close(thames_priv);
+err_free:
 	kfree(thames_priv);
 err_exit:
 	drm_dev_exit(idx);
@@ -87,6 +97,14 @@ void thames_file_priv_put(struct thames_file_priv *priv)
 static void thames_postclose(struct drm_device *dev, struct drm_file *file)
 {
 	struct thames_file_priv *thames_priv = file->driver_priv;
+	struct thames_device *tdev = thames_priv->tdev;
+
+	scoped_guard(mutex, &tdev->files_lock) {
+		if (!list_empty(&thames_priv->file_node)) {
+			list_del_init(&thames_priv->file_node);
+			thames_job_close(thames_priv);
+		}
+	}
 
 	dev_dbg(dev->dev, "Context %u: file closed, %u references left\n",
 		thames_priv->context_id, kref_read(&thames_priv->refcount) - 1);
@@ -208,6 +226,9 @@ static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,	\
 THAMES_IOCTL_ENTERED(bo_create)
 THAMES_IOCTL_ENTERED(bo_mmap_offset)
 THAMES_IOCTL_ENTERED(dev_query)
+THAMES_IOCTL_ENTERED(job_create)
+THAMES_IOCTL_ENTERED(job_submit)
+THAMES_IOCTL_ENTERED(job_destroy)
 THAMES_IOCTL_ENTERED(bo_bind)
 
 static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
@@ -215,6 +236,9 @@ static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
 	THAMES_IOCTL(BO_CREATE, bo_create_entered),
 	THAMES_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset_entered),
 	THAMES_IOCTL(DEV_QUERY, dev_query_entered),
+	THAMES_IOCTL(JOB_CREATE, job_create_entered),
+	THAMES_IOCTL(JOB_SUBMIT, job_submit_entered),
+	THAMES_IOCTL(JOB_DESTROY, job_destroy_entered),
 	THAMES_IOCTL(BO_BIND, bo_bind_entered),
 };
 
@@ -272,6 +296,10 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	err = drmm_mutex_init(&tdev->ddev, &tdev->sched_lock);
+	if (err)
+		return ERR_PTR(err);
+
 	err = drmm_mutex_init(&tdev->ddev, &tdev->transfer_table_pool_lock);
 	if (err)
 		return ERR_PTR(err);
@@ -288,8 +316,13 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	if (err)
 		return ERR_PTR(err);
 
+	err = drmm_mutex_init(&tdev->ddev, &tdev->files_lock);
+	if (err)
+		return ERR_PTR(err);
+
 	INIT_LIST_HEAD(&tdev->transfer_tables);
 	INIT_LIST_HEAD(&tdev->contexts);
+	INIT_LIST_HEAD(&tdev->files);
 
 	tdev->asid_next = 1;
 	/*
@@ -347,6 +380,7 @@ static int thames_platform_probe(struct platform_device *pdev)
 			continue;
 		}
 
+		tdev->sched_list[core] = &tdev->cores[core].sched;
 		tdev->num_cores++;
 		core++;
 	}
@@ -391,6 +425,15 @@ static void thames_platform_remove(struct platform_device *pdev)
 
 	drm_dev_unplug(&tdev->ddev);
 
+	scoped_guard(mutex, &tdev->files_lock) {
+		struct thames_file_priv *priv, *tmp;
+
+		list_for_each_entry_safe(priv, tmp, &tdev->files, file_node) {
+			list_del_init(&priv->file_node);
+			thames_job_close(priv);
+		}
+	}
+
 	for (core = 0; core < tdev->num_cores; core++)
 		thames_core_fini(&tdev->cores[core]);
 	tdev->num_cores = 0;
diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
index 7f9031b63780..2115f7eae589 100644
--- a/drivers/accel/thames/thames_drv.h
+++ b/drivers/accel/thames/thames_drv.h
@@ -11,6 +11,7 @@
 #include <linux/xarray.h>
 
 #include <drm/drm_mm.h>
+#include <drm/gpu_scheduler.h>
 
 #include "thames_device.h"
 
@@ -26,6 +27,12 @@ struct thames_file_priv {
 	 */
 	struct kref refcount;
 
+	struct list_head file_node;
+
+	struct drm_sched_entity sched_entity;
+
+	struct xarray jobs;
+
 	struct io_pgtable_ops *pgtbl_ops[THAMES_MAX_CORES];
 	u64 page_table_base[THAMES_MAX_CORES];
 	u32 asid;
diff --git a/drivers/accel/thames/thames_job.c b/drivers/accel/thames/thames_job.c
new file mode 100644
index 000000000000..6560106d47a8
--- /dev/null
+++ b/drivers/accel/thames/thames_job.c
@@ -0,0 +1,869 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
+/* Copyright 2019 Collabora ltd. */
+/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#include <clocksource/arm_arch_timer.h>
+#include <linux/dev_printk.h>
+#include <linux/dma-fence-chain.h>
+#include <linux/iopoll.h>
+#include <linux/mailbox_client.h>
+#include <linux/platform_device.h>
+#include <linux/xarray.h>
+
+#include <drm/drm_drv.h>
+#include <drm/drm_file.h>
+#include <drm/drm_gem.h>
+#include <drm/drm_print.h>
+#include <drm/drm_syncobj.h>
+#include <drm/thames_accel.h>
+
+#include "thames_core.h"
+#include "thames_device.h"
+#include "thames_drv.h"
+#include "thames_gem.h"
+#include "thames_job.h"
+
+#define CREATE_TRACE_POINTS
+#include "thames_trace.h"
+
+static struct thames_sched_job *to_thames_sched_job(struct drm_sched_job *sched_job)
+{
+	return container_of(sched_job, struct thames_sched_job, base);
+}
+
+static void thames_sched_job_put(struct thames_sched_job *job);
+static void thames_job_put(struct thames_job *job);
+
+#define THAMES_MAX_JOBS_PER_SUBMIT 256
+#define THAMES_MAX_SUBMIT_BOS 4096
+#define THAMES_MAX_SUBMIT_SYNCS 64
+
+/*
+ * The host's timeout is longer than the device's: a Job still on the Ring when
+ * it fires is on a stalled core.
+ */
+#define THAMES_DEVICE_TIMEOUT_MS 500
+
+struct thames_submit_bos {
+	struct kref refcount;
+	u32 read_count;
+	u32 count;
+	struct drm_gem_object *bos[] __counted_by(count);
+};
+
+static void thames_submit_bos_free(struct kref *ref)
+{
+	struct thames_submit_bos *bos = container_of(ref, struct thames_submit_bos, refcount);
+	u32 i;
+
+	for (i = 0; i < bos->count; i++) {
+		if (bos->bos[i])
+			drm_gem_object_put(bos->bos[i]);
+	}
+
+	kvfree(bos);
+}
+
+static void thames_submit_bos_put(struct thames_submit_bos *bos)
+{
+	if (bos)
+		kref_put(&bos->refcount, thames_submit_bos_free);
+}
+
+static const char *thames_fence_get_driver_name(struct dma_fence *fence)
+{
+	return "thames";
+}
+
+static const char *thames_fence_get_timeline_name(struct dma_fence *fence)
+{
+	return "thames";
+}
+
+static const struct dma_fence_ops thames_fence_ops = {
+	.get_driver_name = thames_fence_get_driver_name,
+	.get_timeline_name = thames_fence_get_timeline_name,
+};
+
+static void thames_job_fail_done_fence(struct thames_sched_job *job, int error)
+{
+	if (!dma_fence_was_initialized(job->done_fence) || dma_fence_is_signaled(job->done_fence))
+		return;
+
+	dma_fence_set_error(job->done_fence, error);
+	dma_fence_signal(job->done_fence);
+}
+
+static int thames_job_hw_submit(struct thames_core *core, struct thames_sched_job *sched_job)
+{
+	struct thames_ring *ring = &core->ipc->ring;
+	struct thames_ring_host *host;
+	u32 tail = core->ring_tail;
+	u32 index = tail % THAMES_RING_DEPTH;
+	int ret;
+
+	if (tail - READ_ONCE(ring->head) >= THAMES_RING_DEPTH) {
+		dev_err(core->dev, "Ring full on " THAMES_CORE_FMT "\n", THAMES_CORE_ARG(core));
+		return -ENOSPC;
+	}
+
+	sched_job->stamps[0] = arch_timer_read_counter();
+	sched_job->ring_index = index;
+
+	host = &ring->entries[index].host;
+	host->context_id = sched_job->file_priv->context_id;
+	host->transfer_count = sched_job->job->transfers.count;
+	host->transfer_group_count = sched_job->job->transfers.group_count;
+	host->transfer_table_offset = sched_job->job->transfers.offset;
+	host->seqno = sched_job->seqno;
+	host->program_dsp_va = sched_job->job->program_dsp_va;
+	host->params_dsp_va = sched_job->job->params_dsp_va;
+	host->context_serial = sched_job->file_priv->context_serial;
+	host->timeout_ms = THAMES_DEVICE_TIMEOUT_MS;
+
+	scoped_guard(mutex, &core->job_lock) {
+		kref_get(&sched_job->refcount); /* Put by thames_job_complete(). */
+		list_add_tail(&sched_job->node, &core->submitted);
+
+		/* Order the host half before the tail. */
+		wmb();
+		WRITE_ONCE(ring->tail, tail + 1);
+		core->ring_tail = tail + 1;
+		/* Order the tail before the doorbell. */
+		wmb();
+	}
+
+	/*
+	 * The Job is on the Ring whatever the send returns; it completes
+	 * through the Fence or the timeout.
+	 */
+	ret = mbox_send_message(core->mbox_chan, (void *)(uintptr_t)THAMES_DOORBELL);
+	if (ret < 0)
+		dev_warn_ratelimited(core->dev, "Doorbell not sent: %d\n", ret);
+
+	return 0;
+}
+
+void thames_job_complete(struct thames_core *core)
+{
+	struct thames_sched_job *job, *tmp;
+	LIST_HEAD(done);
+	u64 completed, signalled;
+
+	completed = READ_ONCE(core->ipc->fence.seqno);
+	/* Read the entries after the Fence that covers them. */
+	rmb();
+
+	scoped_guard(mutex, &core->job_lock) {
+		list_for_each_entry_safe(job, tmp, &core->submitted, node) {
+			if (job->seqno > completed)
+				break;
+			list_move_tail(&job->node, &done);
+		}
+	}
+
+	list_for_each_entry_safe(job, tmp, &done, node) {
+		struct thames_ring_fw *fw = &core->ipc->ring.entries[job->ring_index].fw;
+
+		job->stamps[1] = fw->doorbell_seen_stamp;
+		job->stamps[2] = fw->run_start_stamp;
+		job->stamps[3] = fw->run_end_stamp;
+		job->stamps[4] = fw->kernel_entry_stamp;
+		job->stamps[5] = fw->kernel_exit_stamp;
+
+		if (fw->fence_value == THAMES_FENCE_ERROR) {
+			dma_fence_set_error(job->done_fence, -EIO);
+		}
+
+		list_del_init(&job->node);
+		signalled = arch_timer_read_counter();
+		dma_fence_signal(job->done_fence);
+		trace_thames_job_done(core, job, signalled);
+		thames_sched_job_put(job);
+	}
+}
+
+static int thames_acquire_object_fences(struct drm_gem_object **bos, int bo_count,
+					struct drm_sched_job *job, bool is_write)
+{
+	int i, ret;
+
+	for (i = 0; i < bo_count; i++) {
+		ret = dma_resv_reserve_fences(bos[i]->resv, 1);
+		if (ret)
+			return ret;
+
+		ret = drm_sched_job_add_implicit_dependencies(job, bos[i], is_write);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static void thames_attach_object_fences(struct drm_gem_object **bos, int bo_count,
+					struct dma_fence *fence, enum dma_resv_usage usage)
+{
+	int i;
+
+	for (i = 0; i < bo_count; i++)
+		dma_resv_add_fence(bos[i]->resv, fence, usage);
+}
+
+/* Nothing here fails, so a submit call pushes every Job or none. */
+static void thames_job_push(struct thames_sched_job *sched_job)
+{
+	scoped_guard(mutex, &sched_job->tdev->sched_lock) {
+		drm_sched_job_arm(&sched_job->base);
+
+		sched_job->inference_done_fence = dma_fence_get(&sched_job->base.s_fence->finished);
+
+		kref_get(&sched_job->refcount); /* put by scheduler job completion */
+
+		drm_sched_entity_push_job(&sched_job->base);
+	}
+}
+
+static void thames_sched_job_cleanup(struct kref *ref)
+{
+	struct thames_sched_job *sched_job = container_of(ref, struct thames_sched_job, refcount);
+
+	if (dma_fence_was_initialized(sched_job->done_fence))
+		dma_fence_put(sched_job->done_fence);
+	else
+		dma_fence_free(sched_job->done_fence);
+	dma_fence_put(sched_job->inference_done_fence);
+
+	thames_submit_bos_put(sched_job->bos);
+
+	if (sched_job->file_priv)
+		thames_file_priv_put(sched_job->file_priv);
+
+	if (sched_job->job)
+		thames_job_put(sched_job->job);
+
+	kfree(sched_job);
+}
+
+static void thames_sched_job_put(struct thames_sched_job *job)
+{
+	kref_put(&job->refcount, thames_sched_job_cleanup);
+}
+
+static void thames_sched_job_free(struct drm_sched_job *sched_job)
+{
+	struct thames_sched_job *job = to_thames_sched_job(sched_job);
+
+	drm_sched_job_cleanup(sched_job);
+
+	thames_sched_job_put(job);
+}
+
+static struct dma_fence *thames_job_run(struct drm_sched_job *sched_job)
+{
+	struct thames_sched_job *job = to_thames_sched_job(sched_job);
+	struct thames_core *core = container_of(sched_job->sched, struct thames_core, sched);
+	int ret;
+
+	if (unlikely(job->base.s_fence->finished.error))
+		return NULL;
+
+	/* The done fence keeps the seqno of the Job's first push. */
+	job->seqno = ++core->emit_seqno;
+	if (!dma_fence_was_initialized(job->done_fence))
+		dma_fence_init(job->done_fence, &thames_fence_ops, &core->fence_lock,
+			       core->fence_context, job->seqno);
+
+	ret = thames_job_hw_submit(core, job);
+	if (ret) {
+		thames_job_fail_done_fence(job, ret);
+		return ERR_PTR(ret);
+	}
+
+	return dma_fence_get(job->done_fence);
+}
+
+/*
+ * A done fence not signalled here is a Job on a live core's Ring that may still
+ * run.
+ */
+static void thames_job_cancel(struct drm_sched_job *sched_job)
+{
+	struct thames_sched_job *job = to_thames_sched_job(sched_job);
+	struct thames_core *core = container_of(sched_job->sched, struct thames_core, sched);
+
+	if (!dma_fence_was_initialized(job->done_fence) || dma_fence_is_signaled(job->done_fence))
+		return;
+
+	scoped_guard(mutex, &core->job_lock)
+		list_move_tail(&job->node, &core->held);
+
+	thames_job_fail_done_fence(job, -ECANCELED);
+}
+
+static const struct drm_sched_backend_ops thames_sched_ops = {
+	.run_job = thames_job_run,
+	.free_job = thames_sched_job_free,
+	.cancel_job = thames_job_cancel,
+};
+
+int thames_job_init(struct thames_core *core)
+{
+	/* One credit per Ring entry, so the tail never overruns the head. */
+	struct drm_sched_init_args args = {
+		.ops = &thames_sched_ops,
+		.num_rqs = 1,
+		.credit_limit = THAMES_RING_DEPTH,
+		.timeout = MAX_SCHEDULE_TIMEOUT,
+		.name = dev_name(core->dev),
+		.dev = core->dev,
+	};
+	u32 head, tail, before;
+	u64 fence, before_seqno, last;
+	int ret;
+
+	spin_lock_init(&core->fence_lock);
+	mutex_init(&core->job_lock);
+
+	INIT_LIST_HEAD(&core->submitted);
+	INIT_LIST_HEAD(&core->held);
+
+	/*
+	 * Both indices run free, so only a tail not at the head is wrong: the
+	 * previous instance's entries are still pending.  The Fence may trail
+	 * the head by one entry, the firmware moving the head first.
+	 */
+	head = READ_ONCE(core->ipc->ring.head);
+	tail = READ_ONCE(core->ipc->ring.tail);
+	before = (head - 1) % THAMES_RING_DEPTH;
+
+	if (tail != head) {
+		dev_err(core->dev, "Ring on " THAMES_CORE_FMT
+			" has %u pending entries (tail %u, head %u); not attaching\n",
+			THAMES_CORE_ARG(core), tail - head, tail, head);
+		return -EBUSY;
+	}
+
+	/* Read the Fence and the entry after the head and the tail. */
+	rmb();
+	fence = READ_ONCE(core->ipc->fence.seqno);
+	before_seqno = core->ipc->ring.entries[before].host.seqno;
+	last = max(fence, before_seqno);
+
+	core->ring_tail = head;
+	core->emit_seqno = last;
+
+	core->fence_context = dma_fence_context_alloc(1);
+
+	ret = drm_sched_init(&core->sched, &args);
+	if (ret) {
+		dev_err(core->dev, "Failed to create scheduler: %d.", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+/*
+ * These Jobs' core may still run their Kernels: the Jobs, Contexts and buffers
+ * stay allocated.
+ */
+void thames_job_fini(struct thames_core *core)
+{
+	struct thames_sched_job *job;
+	u64 first = U64_MAX, last = 0;
+	size_t kept;
+
+	drm_sched_fini(&core->sched);
+
+	guard(mutex)(&core->job_lock);
+
+	kept = list_count_nodes(&core->held);
+	if (!kept)
+		return;
+
+	list_for_each_entry(job, &core->held, node) {
+		first = min(first, job->seqno);
+		last = max(last, job->seqno);
+	}
+
+	drm_dev_get(&core->tdev->ddev);
+	dev_warn(core->dev, THAMES_CORE_FMT
+		 " may still run %zu Jobs (seqno %llu to %llu); keeping them and their buffers until reboot\n",
+		 THAMES_CORE_ARG(core), kept, first, last);
+}
+
+/*
+ * Above the device's timeout, so the firmware ends a runaway Kernel within the
+ * wait.
+ */
+#define THAMES_SUSPEND_TIMEOUT_MS (THAMES_DEVICE_TIMEOUT_MS + 100)
+
+int thames_job_suspend(struct thames_core *core)
+{
+	struct drm_gpu_scheduler *sched = &core->sched;
+	unsigned int wait_ms;
+	size_t pending;
+	bool idle;
+	int err;
+
+	drm_sched_wqueue_stop(sched);
+	core->timeout_left = drm_sched_suspend_timeout(sched);
+	cancel_delayed_work_sync(&sched->work_tdr);
+	drm_sched_wqueue_stop(sched);
+
+	scoped_guard(mutex, &core->job_lock)
+		pending = list_count_nodes(&core->submitted);
+	wait_ms = max_t(unsigned int, pending, 1) * THAMES_SUSPEND_TIMEOUT_MS;
+
+	err = read_poll_timeout(thames_core_idles, idle, idle, USEC_PER_MSEC,
+				wait_ms * USEC_PER_MSEC, false, core);
+
+	if (!err) {
+		thames_job_complete(core);
+		scoped_guard(mutex, &core->job_lock) {
+			if (!list_empty(&core->submitted))
+				err = -EBUSY;
+		}
+	}
+
+	if (err) {
+		dev_err(core->dev, THAMES_CORE_FMT
+			" is not idle with its Jobs completed after %u ms; refusing the suspend\n",
+			THAMES_CORE_ARG(core), wait_ms);
+		thames_job_resume(core);
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
+void thames_job_resume(struct thames_core *core)
+{
+	drm_sched_resume_timeout(&core->sched, core->timeout_left);
+	drm_sched_wqueue_start(&core->sched);
+}
+
+int thames_job_open(struct thames_file_priv *thames_priv)
+{
+	struct thames_device *tdev = thames_priv->tdev;
+	int ret;
+
+	ret = drm_sched_entity_init(&thames_priv->sched_entity, 0, tdev->sched_list,
+				    tdev->num_cores, NULL);
+	if (WARN_ON(ret))
+		return ret;
+
+	xa_init_flags(&thames_priv->jobs, XA_FLAGS_ALLOC1);
+
+	return 0;
+}
+
+void thames_job_close(struct thames_file_priv *thames_priv)
+{
+	struct thames_job *job;
+	unsigned long handle;
+
+	drm_sched_entity_destroy(&thames_priv->sched_entity);
+
+	xa_for_each(&thames_priv->jobs, handle, job)
+		thames_job_put(job);
+	xa_destroy(&thames_priv->jobs);
+}
+
+static void thames_job_free(struct kref *ref)
+{
+	struct thames_job *job = container_of(ref, struct thames_job, refcount);
+
+	if (job->program)
+		drm_gem_object_put(job->program);
+	if (job->params)
+		drm_gem_object_put(job->params);
+
+	thames_transfers_free(job->tdev, &job->transfers);
+	mutex_destroy(&job->lock);
+	kfree(job);
+}
+
+static void thames_job_put(struct thames_job *job)
+{
+	kref_put(&job->refcount, thames_job_free);
+}
+
+int thames_ioctl_job_create(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_thames_job_create *args = data;
+	struct thames_file_priv *file_priv = file->driver_priv;
+	struct thames_device *tdev = to_thames_device(dev);
+	struct thames_job *job;
+	u32 handle;
+	int ret;
+
+	if (args->flags || args->pad)
+		return -EINVAL;
+
+	if (args->type != DRM_THAMES_JOB_DISPATCH)
+		return -EINVAL;
+
+	if (!args->program || !args->params)
+		return -EINVAL;
+
+	job = kzalloc_obj(*job, GFP_KERNEL);
+	if (!job)
+		return -ENOMEM;
+
+	kref_init(&job->refcount);
+	mutex_init(&job->lock);
+	job->tdev = tdev;
+
+	job->program = drm_gem_object_lookup(file, args->program);
+	if (!job->program) {
+		ret = -ENOENT;
+		goto err_put;
+	}
+
+	if (!(to_thames_bo(job->program)->flags & DRM_THAMES_BO_EXECUTABLE)) {
+		ret = -EINVAL;
+		goto err_put;
+	}
+
+	job->program_dsp_va = thames_bo_dsp_va(to_thames_bo(job->program), file_priv);
+	if (!job->program_dsp_va) {
+		ret = -EINVAL;
+		goto err_put;
+	}
+
+	job->params = drm_gem_object_lookup(file, args->params);
+	if (!job->params) {
+		ret = -ENOENT;
+		goto err_put;
+	}
+
+	job->params_dsp_va = thames_bo_dsp_va(to_thames_bo(job->params), file_priv);
+	if (!job->params_dsp_va) {
+		ret = -EINVAL;
+		goto err_put;
+	}
+
+	ret = thames_transfers_build(file, &args->transfers, &job->transfers);
+	if (ret)
+		goto err_put;
+
+	ret = xa_alloc(&file_priv->jobs, &handle, job, xa_limit_32b, GFP_KERNEL);
+	if (ret)
+		goto err_put;
+
+	job->handle = handle;
+	args->handle = handle;
+
+	return 0;
+
+err_put:
+	thames_job_put(job);
+	return ret;
+}
+
+static struct thames_sched_job *thames_sched_job_new(struct drm_file *file, u32 handle,
+						     struct thames_submit_bos *bos)
+{
+	struct thames_file_priv *file_priv = file->driver_priv;
+	struct thames_job *job;
+	struct thames_sched_job *sched_job;
+	int ret;
+
+	xa_lock(&file_priv->jobs);
+	job = xa_load(&file_priv->jobs, handle);
+	if (job)
+		kref_get(&job->refcount);
+	xa_unlock(&file_priv->jobs);
+
+	if (!job)
+		return ERR_PTR(-ENOENT);
+
+	sched_job = kzalloc_obj(*sched_job, GFP_KERNEL);
+	if (!sched_job) {
+		thames_job_put(job);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	kref_init(&sched_job->refcount);
+	INIT_LIST_HEAD(&sched_job->node);
+	sched_job->tdev = job->tdev;
+	sched_job->job = job; /* The lookup's reference. */
+	/* Put by the completion path once it has counted the Job for the ban. */
+	kref_get(&file_priv->refcount);
+	sched_job->file_priv = file_priv;
+	kref_get(&bos->refcount);
+	sched_job->bos = bos;
+
+	sched_job->done_fence = kzalloc_obj(*sched_job->done_fence);
+	if (!sched_job->done_fence) {
+		thames_sched_job_put(sched_job);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	ret = drm_sched_job_init(&sched_job->base, &file_priv->sched_entity, 1, NULL,
+				 file->client_id);
+	if (ret) {
+		thames_sched_job_put(sched_job);
+		return ERR_PTR(ret);
+	}
+
+	return sched_job;
+}
+
+static int thames_submit_bos_lookup(struct drm_file *file,
+				    const struct drm_thames_job_submit *args,
+				    struct thames_submit_bos **out)
+{
+	const struct drm_thames_obj_array *lists[2] = { &args->read_handles,
+							&args->write_handles };
+	struct thames_submit_bos *bos;
+	u32 total = 0, n = 0, l, i;
+
+	for (l = 0; l < ARRAY_SIZE(lists); l++) {
+		if (lists[l]->count > THAMES_MAX_SUBMIT_BOS - total)
+			return -EINVAL;
+		total += lists[l]->count;
+	}
+
+	bos = kvzalloc(struct_size(bos, bos, total), GFP_KERNEL);
+	if (!bos)
+		return -ENOMEM;
+
+	kref_init(&bos->refcount);
+	bos->count = total;
+	bos->read_count = args->read_handles.count;
+
+	for (l = 0; l < ARRAY_SIZE(lists); l++) {
+		u32 *handles;
+
+		if (!lists[l]->count)
+			continue;
+
+		handles = thames_get_uobj_array(lists[l], sizeof(*handles), sizeof(*handles));
+		if (IS_ERR(handles)) {
+			thames_submit_bos_put(bos);
+			return PTR_ERR(handles);
+		}
+
+		for (i = 0; i < lists[l]->count; i++, n++) {
+			bos->bos[n] = drm_gem_object_lookup(file, handles[i]);
+			if (!bos->bos[n]) {
+				kvfree(handles);
+				thames_submit_bos_put(bos);
+				return -ENOENT;
+			}
+		}
+
+		kvfree(handles);
+	}
+
+	*out = bos;
+	return 0;
+}
+
+/*
+ * Allocated before the first push, so signalling cannot fail once the Jobs run.
+ */
+struct thames_submit_syncs {
+	u32 count;
+	struct drm_thames_sync_op *ops;
+	struct thames_sync_signal {
+		struct drm_syncobj *syncobj;
+		struct dma_fence_chain *chain;
+	} *signals;
+};
+
+static int thames_sync_op_check(const struct drm_thames_sync_op *op)
+{
+	if (op->flags != (DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
+			  DRM_THAMES_SYNC_OP_SIGNAL))
+		return -EINVAL;
+
+	return op->timeline_value ? 0 : -EINVAL;
+}
+
+static void thames_submit_syncs_free(struct thames_submit_syncs *syncs)
+{
+	u32 i;
+
+	for (i = 0; i < syncs->count; i++) {
+		if (syncs->signals[i].syncobj)
+			drm_syncobj_put(syncs->signals[i].syncobj);
+		dma_fence_chain_free(syncs->signals[i].chain);
+	}
+
+	kvfree(syncs->signals);
+	kvfree(syncs->ops);
+}
+
+static int thames_submit_syncs_get(struct drm_file *file, const struct drm_thames_obj_array *in,
+				   struct thames_submit_syncs *syncs)
+{
+	struct drm_thames_sync_op *ops;
+	u32 i;
+	int ret;
+
+	if (!in->count)
+		return 0;
+
+	if (in->count > THAMES_MAX_SUBMIT_SYNCS)
+		return -EINVAL;
+
+	ops = thames_get_uobj_array(in, offsetofend(struct drm_thames_sync_op, timeline_value),
+				    sizeof(*ops));
+	if (IS_ERR(ops))
+		return PTR_ERR(ops);
+	syncs->ops = ops;
+
+	syncs->signals = kvcalloc(in->count, sizeof(*syncs->signals), GFP_KERNEL);
+	if (!syncs->signals)
+		return -ENOMEM;
+	syncs->count = in->count;
+
+	for (i = 0; i < syncs->count; i++) {
+		struct thames_sync_signal *signal = &syncs->signals[i];
+
+		ret = thames_sync_op_check(&ops[i]);
+		if (ret)
+			return ret;
+
+		signal->syncobj = drm_syncobj_find(file, ops[i].handle);
+		if (!signal->syncobj)
+			return -ENOENT;
+
+		signal->chain = dma_fence_chain_alloc();
+		if (!signal->chain)
+			return -ENOMEM;
+	}
+
+	return 0;
+}
+
+static void thames_submit_syncs_signal(struct thames_submit_syncs *syncs, struct dma_fence *fence)
+{
+	u32 i;
+
+	for (i = 0; i < syncs->count; i++) {
+		struct thames_sync_signal *signal = &syncs->signals[i];
+
+		drm_syncobj_add_point(signal->syncobj, signal->chain, fence,
+				      syncs->ops[i].timeline_value);
+		signal->chain = NULL;
+	}
+}
+
+int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_thames_job_submit *args = data;
+	struct thames_submit_syncs syncs = {};
+	struct thames_submit_bos *bos = NULL;
+	struct ww_acquire_ctx acquire_ctx;
+	struct thames_sched_job **subs = NULL;
+	struct dma_fence *last;
+	u32 count = args->jobs.count;
+	u32 pushed = 0;
+	u32 *handles;
+	unsigned int i;
+	int ret = 0;
+
+	if (args->flags || args->pad)
+		return -EINVAL;
+
+	if (!count || count > THAMES_MAX_JOBS_PER_SUBMIT)
+		return -EINVAL;
+
+	handles = thames_get_uobj_array(&args->jobs, sizeof(u32), sizeof(u32));
+	if (IS_ERR(handles))
+		return PTR_ERR(handles);
+
+	ret = thames_submit_syncs_get(file, &args->syncs, &syncs);
+	if (ret)
+		goto out_syncs;
+
+	ret = thames_submit_bos_lookup(file, args, &bos);
+	if (ret)
+		goto out_syncs;
+
+	subs = kvcalloc(count, sizeof(*subs), GFP_KERNEL);
+	if (!subs) {
+		ret = -ENOMEM;
+		goto out_put;
+	}
+
+	for (i = 0; i < count; i++) {
+		subs[i] = thames_sched_job_new(file, handles[i], bos);
+		if (IS_ERR(subs[i])) {
+			ret = PTR_ERR(subs[i]);
+			subs[i] = NULL;
+			goto out_subs;
+		}
+	}
+
+	ret = drm_gem_lock_reservations(bos->bos, bos->count, &acquire_ctx);
+	if (ret)
+		goto out_subs;
+
+	/*
+	 * Only the first Job waits: the entity runs the call's Jobs in order.
+	 */
+	ret = thames_acquire_object_fences(bos->bos, bos->read_count, &subs[0]->base, false);
+	if (ret)
+		goto out_unlock;
+
+	ret = thames_acquire_object_fences(&bos->bos[bos->read_count], bos->count - bos->read_count,
+					   &subs[0]->base, true);
+	if (ret)
+		goto out_unlock;
+
+	for (pushed = 0; pushed < count; pushed++)
+		thames_job_push(subs[pushed]);
+
+	last = subs[count - 1]->inference_done_fence;
+	thames_attach_object_fences(bos->bos, bos->read_count, last, DMA_RESV_USAGE_READ);
+	thames_attach_object_fences(&bos->bos[bos->read_count], bos->count - bos->read_count, last,
+				    DMA_RESV_USAGE_WRITE);
+	thames_submit_syncs_signal(&syncs, last);
+
+out_unlock:
+	drm_gem_unlock_reservations(bos->bos, bos->count, &acquire_ctx);
+out_subs:
+	for (i = 0; i < count; i++) {
+		if (!subs[i])
+			continue;
+		if (i >= pushed)
+			drm_sched_job_cleanup(&subs[i]->base);
+		/* The creation reference; the scheduler holds its own. */
+		thames_sched_job_put(subs[i]);
+	}
+	kvfree(subs);
+out_put:
+	thames_submit_bos_put(bos);
+out_syncs:
+	thames_submit_syncs_free(&syncs);
+	kvfree(handles);
+	return ret;
+}
+
+int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file *file)
+{
+	struct drm_thames_job_destroy *args = data;
+	struct thames_file_priv *file_priv = file->driver_priv;
+	struct thames_job *job;
+
+	if (args->pad)
+		return -EINVAL;
+
+	job = xa_erase(&file_priv->jobs, args->handle);
+	if (!job)
+		return -ENOENT;
+
+	/* A submission still running holds its own reference. */
+	thames_job_put(job);
+
+	return 0;
+}
+
diff --git a/drivers/accel/thames/thames_job.h b/drivers/accel/thames/thames_job.h
new file mode 100644
index 000000000000..a851febe8f98
--- /dev/null
+++ b/drivers/accel/thames/thames_job.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#ifndef __THAMES_JOB_H__
+#define __THAMES_JOB_H__
+
+#include <drm/drm_drv.h>
+#include <drm/gpu_scheduler.h>
+#include <drm/thames_accel.h>
+
+#include "thames_core.h"
+#include "thames_drv.h"
+#include "thames_transfers.h"
+
+struct thames_submit_bos;
+
+struct thames_sched_job {
+	struct drm_sched_job base;
+
+	struct thames_device *tdev;
+	struct thames_file_priv *file_priv;
+
+	struct dma_fence *inference_done_fence;
+	struct dma_fence *done_fence;
+
+	/* @seqno moves at every push; the done fence keeps the first push's. */
+	u64 seqno;
+	u32 ring_index;
+	u64 stamps[6];
+	struct list_head node;
+
+	struct thames_job *job;
+
+	struct thames_submit_bos *bos;
+
+	struct kref refcount;
+};
+
+struct thames_job {
+	struct kref refcount;
+	struct thames_device *tdev;
+	u32 handle;
+
+	struct drm_gem_object *program;
+	struct drm_gem_object *params;
+
+	/*
+	 * Taken at create: after an unbind the Job faults in its Context rather
+	 * than read a binding that is gone.
+	 */
+	u64 program_dsp_va;
+	u64 params_dsp_va;
+
+	struct thames_transfer_table transfers;
+
+	struct mutex lock;
+};
+
+int thames_ioctl_job_create(struct drm_device *dev, void *data, struct drm_file *file);
+int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file);
+int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file *file);
+
+void thames_job_complete(struct thames_core *core);
+
+int thames_job_init(struct thames_core *core);
+int thames_job_suspend(struct thames_core *core);
+void thames_job_resume(struct thames_core *core);
+void thames_job_fini(struct thames_core *core);
+int thames_job_open(struct thames_file_priv *thames_priv);
+void thames_job_close(struct thames_file_priv *thames_priv);
+
+#endif
diff --git a/drivers/accel/thames/thames_trace.h b/drivers/accel/thames/thames_trace.h
new file mode 100644
index 000000000000..3d1e59708869
--- /dev/null
+++ b/drivers/accel/thames/thames_trace.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM thames
+
+#if !defined(__THAMES_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
+#define __THAMES_TRACE_H__
+
+#include <linux/math64.h>
+#include <linux/time64.h>
+#include <linux/tracepoint.h>
+#include <linux/types.h>
+
+#include "thames_core.h"
+#include "thames_device.h"
+#include "thames_drv.h"
+#include "thames_job.h"
+
+/* Every stamp is the SoC's system counter in microseconds, so any two subtract. */
+TRACE_EVENT(thames_job_done,
+	    TP_PROTO(const struct thames_core *core, const struct thames_sched_job *job,
+		     u64 signalled),
+
+	    TP_ARGS(core, job, signalled),
+
+	    TP_STRUCT__entry(__string(dev_name, dev_name(core->dev))
+			     __field(u32, core)
+			     __field(u32, context)
+			     __field(u64, seqno)
+			     __field(int, error)
+			     __field(u64, submit_us)
+			     __field(u64, doorbell_seen_us)
+			     __field(u64, run_start_us)
+			     __field(u64, kernel_entry_us)
+			     __field(u64, kernel_exit_us)
+			     __field(u64, run_end_us)
+			     __field(u64, signalled_us)),
+
+	    TP_fast_assign(u32 hz = core->tdev->caps.timestamp_hz;
+
+			   __assign_str(dev_name);
+			   __entry->core = core->index;
+			   __entry->context = job->file_priv->context_id;
+			   __entry->seqno = job->seqno;
+			   __entry->error = job->done_fence->error;
+			   __entry->submit_us =
+				mul_u64_u32_div(job->stamps[0], USEC_PER_SEC, hz);
+			   __entry->doorbell_seen_us =
+				mul_u64_u32_div(job->stamps[1], USEC_PER_SEC, hz);
+			   __entry->run_start_us =
+				mul_u64_u32_div(job->stamps[2], USEC_PER_SEC, hz);
+			   __entry->run_end_us =
+				mul_u64_u32_div(job->stamps[3], USEC_PER_SEC, hz);
+			   __entry->kernel_entry_us =
+				mul_u64_u32_div(job->stamps[4], USEC_PER_SEC, hz);
+			   __entry->kernel_exit_us =
+				mul_u64_u32_div(job->stamps[5], USEC_PER_SEC, hz);
+			   __entry->signalled_us =
+				mul_u64_u32_div(signalled, USEC_PER_SEC, hz);),
+
+	    TP_printk("%s: core=%u context=%u seqno=%llu error=%d submit_us=%llu doorbell_seen_us=%llu run_start_us=%llu kernel_entry_us=%llu kernel_exit_us=%llu run_end_us=%llu signalled_us=%llu",
+		      __get_str(dev_name), __entry->core, __entry->context,
+		      __entry->seqno, __entry->error, __entry->submit_us,
+		      __entry->doorbell_seen_us, __entry->run_start_us,
+		      __entry->kernel_entry_us, __entry->kernel_exit_us,
+		      __entry->run_end_us, __entry->signalled_us)
+);
+
+#endif /* __THAMES_TRACE_H__ */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE thames_trace
+
+#include <trace/define_trace.h>
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
index 5671cffaf8f9..bd2b4271b896 100644
--- a/include/uapi/drm/thames_accel.h
+++ b/include/uapi/drm/thames_accel.h
@@ -57,6 +57,15 @@ enum drm_thames_ioctl_id {
 	/** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
 	DRM_THAMES_DEV_QUERY,
 
+	/** @DRM_THAMES_JOB_CREATE: Validate a Dispatch once into a Job. */
+	DRM_THAMES_JOB_CREATE,
+
+	/** @DRM_THAMES_JOB_SUBMIT: Run created Jobs once, in order. */
+	DRM_THAMES_JOB_SUBMIT,
+
+	/** @DRM_THAMES_JOB_DESTROY: Release a Job and the buffers it holds. */
+	DRM_THAMES_JOB_DESTROY,
+
 	/**
 	 * @DRM_THAMES_BO_BIND: Bind buffers into the file's Context at the
 	 * addresses the client chooses, or unbind them.
@@ -244,6 +253,21 @@ enum drm_thames_dev_query_type {
 
 };
 
+/**
+ * enum drm_thames_job_type - What a Job is
+ *
+ * A Job is one Dispatch, and a submission is an ordered array of Jobs.  A Job
+ * whose transfers are not K3 DMA records would be a new type, not a new
+ * operation, and so are the copies, fills, timestamps and the list-carrying
+ * command buffer a Vulkan compute driver will add.  The query grows a field
+ * naming the types beyond the Dispatch when the first one comes; an absent type
+ * is refused at Job creation.
+ */
+enum drm_thames_job_type {
+	/** @DRM_THAMES_JOB_DISPATCH: Run a Kernel with its transfers. */
+	DRM_THAMES_JOB_DISPATCH = 0,
+};
+
 /**
  * enum drm_thames_hardware_id - &drm_thames_caps.hardware_id
  */
@@ -255,8 +279,9 @@ enum drm_thames_hardware_id {
 /**
  * struct drm_thames_caps - The capability query
  *
- * Each window sits at one address whatever core runs the Job and reaches that
- * core's own memory.
+ * The submit's limits are stated on &struct drm_thames_job_submit.  Each window
+ * sits at one address whatever core runs the Job and reaches that core's own
+ * memory.
  */
 struct drm_thames_caps {
 	/** @l2_size: Size in bytes of each core's L2 SRAM. */
@@ -328,6 +353,35 @@ struct drm_thames_caps {
 	__u64 dru_capabilities;
 };
 
+/**
+ * enum drm_thames_fault_kind - The kind of a Job's fault
+ *
+ * The kind is what the firmware knows without decoding a register: what stopped
+ * the Job and whose mode it stopped in.
+ */
+enum drm_thames_fault_kind {
+	/**
+	 * @DRM_THAMES_FAULT_KERNEL: A Kernel took an exception in user mode.
+	 * The Context continues; the registers say why.
+	 */
+	DRM_THAMES_FAULT_KERNEL = 1,
+
+	/**
+	 * @DRM_THAMES_FAULT_FIRMWARE: The firmware took an exception in
+	 * supervisor mode while running this Job.  The core was reset.
+	 */
+	DRM_THAMES_FAULT_FIRMWARE,
+
+	/**
+	 * @DRM_THAMES_FAULT_TIMEOUT: The device's Job timeout fired: the Kernel
+	 * ran longer than the driver lets a Kernel run and the device ended it,
+	 * or the Kernel returned with transfers still running and they did not
+	 * drain. The Context continues; NRP and NTSR say where the Kernel was,
+	 * and are 0 for transfers that did not drain.
+	 */
+	DRM_THAMES_FAULT_TIMEOUT,
+};
+
 /**
  * struct drm_thames_dev_query - Arguments passed to DRM_IOCTL_THAMES_DEV_QUERY
  */
@@ -353,9 +407,9 @@ struct drm_thames_dev_query {
  * order: the first group starts at the first transfer, each later group at a
  * transfer carrying this flag.  The firmware submits the first group before it
  * enters the Kernel, and each later group, whole, when the Kernel asks for it
- * with system call 2.  No group may hold more transfers on one channel than
- * &drm_thames_caps.dru_queue_depth.  Never on the first transfer.  A Job
- * without the flag is one group.
+ * with system call 2 (see &struct drm_thames_job_create).  No group may hold
+ * more transfers on one channel than &drm_thames_caps.dru_queue_depth.  Never
+ * on the first transfer.  A Job without the flag is one group.
  */
 #define DRM_THAMES_TRANSFER_GROUP	(1 << 0)
 
@@ -393,7 +447,9 @@ struct drm_thames_dev_query {
  * auxiliary aperture, at &drm_thames_caps.l2_main_dsp_va and l2_aux, which the
  * kernel translates to the physical aperture of the core the Job runs on.  The
  * Job holds a reference on each buffer a transfer resolves to until it is
- * destroyed and its last submission has completed.
+ * destroyed and its last submission has completed.  No list names these
+ * buffers: the read and write lists of &drm_thames_job_submit are for
+ * synchronisation alone.
  */
 struct drm_thames_transfer {
 	/**
@@ -409,6 +465,191 @@ struct drm_thames_transfer {
 	__u32 words[16];
 };
 
+/**
+ * struct drm_thames_job_create - Arguments passed to DRM_IOCTL_THAMES_JOB_CREATE
+ *
+ * Once per Dispatch: the transfers are copied, each checked and translated, a
+ * reference taken on the program, the params and every buffer a transfer
+ * resolves to so the translation holds for the Job's life, and a handle
+ * returned.  The buffers the Kernel reads and writes are named at each
+ * submit, in &drm_thames_job_submit.
+ *
+ * The firmware enters the Kernel at the program's start as a C function
+ * taking one argument, the params' address, and returning a status word:
+ *
+ *	int kernel(void *params);
+ *
+ * That is the whole of what the kernel and the firmware know.  The Kernel finds
+ * everything else, its inputs, outputs, constants, grid and status buffer,
+ * through params, whose layout is private to the userspace driver that emitted
+ * the Kernel.  The program must have been created DRM_THAMES_BO_EXECUTABLE.
+ * The program and the params must be bound in the Context.
+ * The Job keeps the two addresses they are bound at when it is created, and
+ * every submission runs the Kernel at them.
+ *
+ * A Kernel asks for its Job's next transfer group with system call 2 (the
+ * SYSCALL instruction with 2, no argument).  The call returns to the
+ * instruction after it with every register kept but A0-A3 and RP.  The firmware
+ * submits the group only if the Job has one left and every channel the group
+ * uses has no transfer queued or running; otherwise it fails the Job, writing
+ * none of the group's transfers.  A Kernel waits for the tile event of its
+ * channels' DRU records with system call 3.
+ */
+struct drm_thames_job_create {
+	/** @type: One of &enum drm_thames_job_type. */
+	__u32 type;
+
+	/** @flags: MBZ. */
+	__u32 flags;
+
+	/** @program: Handle of the buffer holding the Kernel. */
+	__u32 program;
+
+	/** @params: Handle of the params buffer. */
+	__u32 params;
+
+	/**
+	 * @transfers: Array of &struct drm_thames_transfer, the DRU records
+	 * the compiler scheduled for this Kernel; may be empty.
+	 */
+	struct drm_thames_obj_array transfers;
+
+	/** @handle: Returned handle of the Job.  Job handles are nonzero. */
+	__u32 handle;
+
+	/** @pad: MBZ. */
+	__u32 pad;
+};
+
+/**
+ * DOC: Sync operation flags
+ *
+ * The low byte of &drm_thames_sync_op.flags is the handle type, bit 31 the
+ * direction; every other bit is MBZ.
+ *
+ * The one op accepted is a timeline sync object signalled,
+ * %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
+ * %DRM_THAMES_SYNC_OP_SIGNAL.  Handle type 0 and a clear bit 31, a binary sync
+ * object and a wait, are left for the client that will use them.
+ *
+ * - %DRM_THAMES_SYNC_OP_HANDLE_TYPE_MASK: The bits of the handle type.
+ * - %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ: A timeline sync
+ *   object, at the point &drm_thames_sync_op.timeline_value.
+ * - %DRM_THAMES_SYNC_OP_SIGNAL: The call's last Job's finished fence is
+ *   added to the timeline at the point.
+ */
+#define DRM_THAMES_SYNC_OP_HANDLE_TYPE_MASK		0xff
+#define DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ	1
+#define DRM_THAMES_SYNC_OP_SIGNAL			(1u << 31)
+
+/**
+ * struct drm_thames_sync_op - One signal of a submit call
+ */
+struct drm_thames_sync_op {
+	/**
+	 * @flags: %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
+	 * %DRM_THAMES_SYNC_OP_SIGNAL.
+	 */
+	__u32 flags;
+
+	/** @handle: A timeline sync object of the client. */
+	__u32 handle;
+
+	/**
+	 * @timeline_value: The point on the timeline sync object, nonzero.
+	 */
+	__u64 timeline_value;
+};
+
+/**
+ * struct drm_thames_job_submit - Arguments passed to DRM_IOCTL_THAMES_JOB_SUBMIT
+ *
+ * A submission is an ordered array of Jobs, one Ring entry each, so a sequence
+ * of Dispatches is one call.  Per run the kernel re-checks only the Context's
+ * current state, waits on the implicit fences of the buffers the call names and
+ * pushes the Ring entries.  A submission of a Job while an earlier
+ * submission of the same Job is in flight waits for it, which the Ring's order
+ * gives on its own.  One Ring completes its Jobs in submission order; there is
+ * no order between the two cores' Rings, but the Jobs of one call run in the
+ * order named.
+ *
+ * The buffers the call's Jobs read and write are named here, one pair of lists
+ * per call.  They are for synchronisation alone: the first Job waits on the
+ * fences already on them, and the last Job's fence is set on them once the Jobs
+ * are pushed, for read usage on @read_handles and write usage on
+ * @write_handles, so DRM_IOCTL_THAMES_BO_PREP on a written buffer waits for the
+ * call's Jobs.  No flag turns either side off.  The call holds a reference on
+ * each buffer named until its last Job completes.  A buffer may be named once
+ * in the two lists together.
+ *
+ * The sync objects the call signals are named in @syncs.  Every op receives the
+ * call's last Job's finished fence, the same fence the listed buffers get; the
+ * call waits on no sync object.  The call is all or nothing: every Job handle
+ * is looked up and every sync op resolved before the first Job is pushed, so a
+ * call that fails has submitted nothing and signals nothing.  An opt-out of the
+ * implicit synchronisation would be a bit of @flags, appended under a minor
+ * bump.
+ *
+ * Every Job the call submits has its fault record cleared; a refused call
+ * clears none.
+ *
+ * A Job of the call that fails does not stop the ones after it: each later Job
+ * runs, and completes or fails on its own merits.  Only a Context banned after
+ * its faults has its Jobs not yet run cancelled, their fences signalled with
+ * -ECANCELED, and its later submit calls refused with -ECANCELED.
+ */
+struct drm_thames_job_submit {
+	/** @flags: MBZ. */
+	__u32 flags;
+
+	/** @pad: MBZ. */
+	__u32 pad;
+
+	/**
+	 * @jobs: Array of __u32 Job handles, run in order.
+	 *
+	 * At least 1 and at most 256 Jobs.
+	 */
+	struct drm_thames_obj_array jobs;
+
+	/**
+	 * @read_handles: Array of __u32 handles of the buffers the Jobs read;
+	 * may be empty.
+	 *
+	 * At most 4096 handles in @read_handles and @write_handles together.
+	 */
+	struct drm_thames_obj_array read_handles;
+
+	/**
+	 * @write_handles: Array of __u32 handles of the buffers the Jobs write;
+	 * may be empty.  Counted with @read_handles.
+	 */
+	struct drm_thames_obj_array write_handles;
+
+	/**
+	 * @syncs: Array of &struct drm_thames_sync_op; may be empty.
+	 *
+	 * At most 64 ops.
+	 */
+	struct drm_thames_obj_array syncs;
+};
+
+/**
+ * struct drm_thames_job_destroy - Arguments passed to DRM_IOCTL_THAMES_JOB_DESTROY
+ *
+ * Releases the handle and returns at once; it does not wait for the Job's
+ * submissions.  A submission still in flight keeps the Job, its validated copy
+ * and the references on its buffers until it completes, and its fence signals
+ * as it would have.  The handle names nothing afterwards.
+ */
+struct drm_thames_job_destroy {
+	/** @handle: The Job. */
+	__u32 handle;
+
+	/** @pad: MBZ. */
+	__u32 pad;
+};
+
 /**
  * DRM_IOCTL_THAMES() - Build a thames IOCTL number
  * @__access: Access type. Must be R, W or RW.
@@ -431,6 +672,12 @@ enum {
 		DRM_IOCTL_THAMES(WR, BO_MMAP_OFFSET, bo_mmap_offset),
 	DRM_IOCTL_THAMES_DEV_QUERY =
 		DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
+	DRM_IOCTL_THAMES_JOB_CREATE =
+		DRM_IOCTL_THAMES(WR, JOB_CREATE, job_create),
+	DRM_IOCTL_THAMES_JOB_SUBMIT =
+		DRM_IOCTL_THAMES(WR, JOB_SUBMIT, job_submit),
+	DRM_IOCTL_THAMES_JOB_DESTROY =
+		DRM_IOCTL_THAMES(WR, JOB_DESTROY, job_destroy),
 	DRM_IOCTL_THAMES_BO_BIND =
 		DRM_IOCTL_THAMES(WR, BO_BIND, bo_bind),
 };

-- 
2.54.0

[PATCH v3 10/16] accel/thames: Recover from faults and timeouts

[Tomeu Vizoso]

A Kernel that faults, or runs past the 500 ms device timeout the kernel
writes in every Ring entry, ends in its own fault record, which the
firmware fills and userspace reads through the existing query IOCTL; the
Context continues.

The host's timeout, 2000 ms, is longer than the device's: a Job still on
the Ring when it fires is on a core that stalled. The kernel moves the
Ring's head past the failed Job, stops and starts the core through
remoteproc, reads the handshake again and pushes the Ring's other Jobs
again on their same fences, so a Kernel may run twice; the uAPI says so.
A core someone else stops through remoteproc goes offline, and a
remoteproc subdevice attaches to it again when remoteproc starts it.

A Context with 3 faults among its last 10 Jobs is banned: its Jobs not
yet run are cancelled with -ECANCELED and its later submits refused.
The thresholds let a client survive a stray fault and stop one that
faults repeatedly, each fault costing the core a fault path and, on a
timeout, a reset.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 drivers/accel/thames/thames_core.c   | 191 ++++++++++++++-
 drivers/accel/thames/thames_core.h   |  19 ++
 drivers/accel/thames/thames_device.h |   2 +
 drivers/accel/thames/thames_drv.c    |  33 +++
 drivers/accel/thames/thames_drv.h    |   5 +
 drivers/accel/thames/thames_job.c    | 452 ++++++++++++++++++++++++++++++++++-
 drivers/accel/thames/thames_job.h    |   8 +
 include/uapi/drm/thames_accel.h      |  98 ++++++++
 8 files changed, 806 insertions(+), 2 deletions(-)

diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
index 85e8f48aec60..2a50fb62fe1e 100644
--- a/drivers/accel/thames/thames_core.c
+++ b/drivers/accel/thames/thames_core.c
@@ -30,6 +30,8 @@
 
 #define THAMES_RESTART_TIMEOUT_MS	5000
 
+#define THAMES_IDLE_TIMEOUT_MS		2000
+
 /*
  * The IPC memory is the core's first memory-region, which remoteproc allocates
  * from only for virtio rings and carveouts this firmware does not have.
@@ -383,6 +385,38 @@ static void thames_core_free_doorbell(struct thames_core *core)
 	put_device(&core->mbox_dev);
 }
 
+/* With every core offline all stay ready, so an entity keeps a run queue. */
+static void thames_core_update_sched_ready(struct thames_device *tdev)
+{
+	bool any_online = false;
+	unsigned int i;
+
+	guard(mutex)(&tdev->sched_lock);
+
+	for (i = 0; i < tdev->num_cores; i++)
+		any_online |= !thames_core_is_dead(&tdev->cores[i]);
+
+	for (i = 0; i < tdev->num_cores; i++)
+		tdev->cores[i].sched.ready = !any_online ||
+					     !thames_core_is_dead(&tdev->cores[i]);
+}
+
+bool thames_core_test_and_set_offline(struct thames_core *core)
+{
+	bool was_online = !atomic_xchg(&core->dead, 1);
+
+	thames_core_update_sched_ready(core->tdev);
+
+	return was_online;
+}
+
+void thames_core_set_online(struct thames_core *core)
+{
+	atomic_set(&core->dead, 0);
+
+	thames_core_update_sched_ready(core->tdev);
+}
+
 static int thames_core_rproc_boot(struct thames_core *core)
 {
 	int err;
@@ -405,6 +439,91 @@ static int thames_core_rproc_shutdown(struct thames_core *core)
 	return err;
 }
 
+static bool thames_core_rproc_is_ours(struct thames_core *core)
+{
+	return READ_ONCE(core->rproc_caller) == current;
+}
+
+/* The stop cannot be refused from here; the core is taken offline before it. */
+static void thames_core_subdev_stop(struct rproc_subdev *subdev, bool crashed)
+{
+	struct thames_core *core = container_of(subdev, struct thames_core, subdev);
+	bool idle;
+
+	if (thames_core_rproc_is_ours(core))
+		return;
+
+	if (thames_core_test_and_set_offline(core))
+		dev_warn(core->dev, THAMES_CORE_FMT
+			 " is stopped through remoteproc; it is offline until remoteproc starts it again\n",
+			 THAMES_CORE_ARG(core));
+
+	/* Its fence lock may be held by a thread that never returns. */
+	if (core->completion_stuck)
+		return;
+
+	if (!crashed &&
+	    read_poll_timeout(thames_core_idles, idle, idle, USEC_PER_MSEC,
+			      THAMES_IDLE_TIMEOUT_MS * USEC_PER_MSEC, false, core))
+		dev_crit(core->dev, THAMES_CORE_FMT " is not known to idle; remoteproc stops it all the same\n",
+			 THAMES_CORE_ARG(core));
+
+	thames_job_complete(core);
+	thames_job_fail_submitted(core);
+}
+
+static int thames_core_subdev_start(struct rproc_subdev *subdev)
+{
+	struct thames_core *core = container_of(subdev, struct thames_core, subdev);
+	struct thames_caps caps = {};
+	u32 head, tail;
+	int err;
+
+	if (thames_core_rproc_is_ours(core))
+		return 0;
+
+	if (core->completion_stuck) {
+		dev_crit(core->dev, THAMES_CORE_FMT
+			 " stays offline: a completion of its never finished, a reboot is needed to use it again\n",
+			 THAMES_CORE_ARG(core));
+		return 0;
+	}
+
+	err = thames_core_wait_handshake(core);
+	if (!err)
+		err = thames_core_read_handshake(core, &caps);
+	if (!err) {
+		head = READ_ONCE(core->ipc->ring.head);
+		tail = READ_ONCE(core->ipc->ring.tail);
+		if (head != tail) {
+			dev_err(core->dev, "Ring on " THAMES_CORE_FMT
+				" is not empty after its boot (tail %u, head %u)\n",
+				THAMES_CORE_ARG(core), tail, head);
+			err = -EIO;
+		}
+	}
+	if (err) {
+		dev_crit(core->dev, THAMES_CORE_FMT
+			 " started through remoteproc did not attach (%d); it stays offline, left running\n",
+			 THAMES_CORE_ARG(core), err);
+		return 0;
+	}
+
+	core->ring_tail = head;
+	thames_context_rewrite_entries(core);
+	thames_transfers_rewrite(core);
+	thames_job_release_held(core);
+
+	core->suspended = false;
+	core->unfinished_seqno = 0;
+	thames_core_set_online(core);
+
+	dev_info(core->dev, THAMES_CORE_FMT " started through remoteproc; it is online again\n",
+		 THAMES_CORE_ARG(core));
+
+	return 0;
+}
+
 /*
  * The firmware drives these timers with no power request and the device tree
  * reserves them for it, so the driver holds their power domains on.
@@ -546,6 +665,15 @@ int thames_core_init(struct thames_core *core)
 	if (!core->tdev->num_cores)
 		core->tdev->caps = caps;
 
+	/*
+	 * Under the rproc lock: unlike rpmsg's, this subdevice joins an rproc
+	 * already registered and maybe running.
+	 */
+	core->subdev.start = thames_core_subdev_start;
+	core->subdev.stop = thames_core_subdev_stop;
+	scoped_guard(mutex, &rproc->lock)
+		rproc_add_subdev(rproc, &core->subdev);
+
 	return 0;
 
 err_doorbell:
@@ -564,6 +692,10 @@ int thames_core_init(struct thames_core *core)
 
 void thames_core_fini(struct thames_core *core)
 {
+	scoped_guard(mutex, &core->rproc->lock)
+		rproc_remove_subdev(core->rproc, &core->subdev);
+
+	thames_job_drain(core);
 	if (core->mbox_chan)
 		thames_core_free_doorbell(core);
 	thames_job_fini(core);
@@ -595,6 +727,53 @@ bool thames_core_idles(struct thames_core *core)
 	return READ_ONCE(record->kind) == DRM_THAMES_FAULT_FIRMWARE;
 }
 
+/* Runs on reset_wq, the ordered queue the timeout handler runs on. */
+int thames_core_reset(struct thames_core *core)
+{
+	struct thames_caps caps = {};
+	struct rproc *rproc = core->rproc;
+	bool idle;
+	int err;
+
+	err = read_poll_timeout(thames_core_idles, idle, idle, USEC_PER_MSEC,
+				THAMES_IDLE_TIMEOUT_MS * USEC_PER_MSEC, false, core);
+	if (err) {
+		dev_crit(core->dev, THAMES_CORE_FMT " is not known to idle; not stopping it\n",
+			 THAMES_CORE_ARG(core));
+		return -EBUSY;
+	}
+
+	dev_warn(core->dev, "Resetting " THAMES_CORE_FMT "\n", THAMES_CORE_ARG(core));
+
+	/*
+	 * So the restarted firmware does not run the entries left on the Ring
+	 * again.
+	 */
+	WRITE_ONCE(core->ipc->ring.head, core->ring_tail);
+	/* Order the head before the stop. */
+	wmb();
+
+	err = thames_core_rproc_shutdown(core);
+	if (!err && rproc->state != RPROC_OFFLINE) {
+		dev_err(core->dev, THAMES_CORE_FMT " is kept running by another power count; not resetting it\n",
+			THAMES_CORE_ARG(core));
+		thames_core_rproc_boot(core);
+		return -EBUSY;
+	}
+	if (err)
+		return err;
+
+	err = thames_core_rproc_boot(core);
+	if (err)
+		return err;
+
+	err = thames_core_wait_handshake(core);
+	if (err)
+		return err;
+
+	return thames_core_read_handshake(core, &caps);
+}
+
 /*
  * rproc_shutdown() reaches the system firmware because the core has one power
  * count, remoteproc's auto-boot or this driver's boot: the probe defers while
@@ -612,6 +791,12 @@ int thames_core_suspend(struct thames_core *core)
 	if (core->suspended)
 		return 0;
 
+	if (thames_core_is_dead(core)) {
+		dev_warn(core->dev, THAMES_CORE_FMT " is offline; leaving it as it is across the suspend\n",
+			 THAMES_CORE_ARG(core));
+		return 0;
+	}
+
 	err = thames_core_rproc_shutdown(core);
 	if (!err && rproc->state != RPROC_OFFLINE) {
 		dev_err(core->dev, THAMES_CORE_FMT
@@ -639,7 +824,7 @@ int thames_core_resume(struct thames_core *core)
 	u32 head, tail;
 	int err = 0;
 
-	if (!core->suspended)
+	if (!core->suspended || thames_core_is_dead(core))
 		goto restart;
 
 	err = thames_core_rproc_boot(core);
@@ -672,6 +857,10 @@ int thames_core_resume(struct thames_core *core)
 	goto restart;
 
 offline:
+	thames_core_test_and_set_offline(core);
+	dev_crit(core->dev, THAMES_CORE_FMT
+		 " did not come back from the suspend (%d); it is offline until it is stopped and started again through remoteproc\n",
+		 THAMES_CORE_ARG(core), err);
 restart:
 	thames_job_resume(core);
 	return err;
diff --git a/drivers/accel/thames/thames_core.h b/drivers/accel/thames/thames_core.h
index a78414d18ad3..aa607d22064c 100644
--- a/drivers/accel/thames/thames_core.h
+++ b/drivers/accel/thames/thames_core.h
@@ -52,12 +52,23 @@ struct thames_core {
 
 	spinlock_t fence_lock;
 
+	/* Cleared only when remoteproc boots the core again. */
+	atomic_t dead;
+
 	/*
 	 * @rproc_caller is the task inside the driver's own rproc_boot() or
 	 * rproc_shutdown(), whose subdevice callbacks then do nothing.
 	 */
+	struct rproc_subdev subdev;
 	struct task_struct *rproc_caller;
 
+	/*
+	 * @completion_stuck: the same Job found unfinished at two timeouts in a
+	 * row.
+	 */
+	u64 unfinished_seqno;
+	bool completion_stuck;
+
 	struct drm_gpu_scheduler sched;
 	u64 fence_context;
 	u64 emit_seqno;
@@ -92,8 +103,16 @@ static inline const struct resource *thames_core_window(const struct thames_core
 #define THAMES_CORE_FMT "%s"
 #define THAMES_CORE_ARG(core) ((core)->name)
 
+static inline bool thames_core_is_dead(struct thames_core *core)
+{
+	return atomic_read(&core->dead);
+}
+
+bool thames_core_test_and_set_offline(struct thames_core *core);
+void thames_core_set_online(struct thames_core *core);
 int thames_core_init(struct thames_core *core);
 void thames_core_fini(struct thames_core *core);
+int thames_core_reset(struct thames_core *core);
 bool thames_core_idles(struct thames_core *core);
 int thames_core_suspend(struct thames_core *core);
 int thames_core_resume(struct thames_core *core);
diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
index 0a34764885df..f87c78a5f512 100644
--- a/drivers/accel/thames/thames_device.h
+++ b/drivers/accel/thames/thames_device.h
@@ -46,6 +46,8 @@ struct thames_device {
 
 	struct ida ctx_ida;
 
+	struct workqueue_struct *reset_wq;
+
 	/*
 	 * Seeded from the boot clock at load, so no serial repeats an earlier
 	 * load's.
diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index fc3328834162..5f704cbfe1d0 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -52,6 +52,7 @@ static int thames_open(struct drm_device *dev, struct drm_file *file)
 	thames_priv->tdev = tdev;
 	kref_init(&thames_priv->refcount);
 	mutex_init(&thames_priv->mm_lock);
+	spin_lock_init(&thames_priv->fault_lock);
 
 	file->driver_priv = thames_priv;
 
@@ -163,6 +164,25 @@ static int thames_query_copy_out(struct drm_thames_dev_query *args, const void *
 	return 0;
 }
 
+static int thames_query_job_fault(struct thames_file_priv *file_priv,
+				  struct drm_thames_dev_query *args)
+{
+	struct drm_thames_job_fault fault = {};
+	int ret;
+
+	if (args->size < offsetofend(struct drm_thames_job_fault, job))
+		return -EINVAL;
+
+	if (get_user(fault.job, (u32 __user *)u64_to_user_ptr(args->pointer)))
+		return -EFAULT;
+
+	ret = thames_job_query_fault(file_priv, &fault);
+	if (ret)
+		return ret;
+
+	return thames_query_copy_out(args, &fault, sizeof(fault));
+}
+
 static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query *args)
 {
 	struct thames_device *tdev = to_thames_device(dev);
@@ -190,6 +210,7 @@ static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query
 
 static int thames_ioctl_dev_query(struct drm_device *dev, void *data, struct drm_file *file)
 {
+	struct thames_file_priv *file_priv = file->driver_priv;
 	struct drm_thames_dev_query *args = data;
 
 	switch (args->type) {
@@ -199,6 +220,12 @@ static int thames_ioctl_dev_query(struct drm_device *dev, void *data, struct drm
 			return 0;
 		}
 		return thames_query_caps(dev, args);
+	case DRM_THAMES_DEV_QUERY_JOB_FAULT:
+		if (!args->pointer) {
+			args->size = sizeof(struct drm_thames_job_fault);
+			return 0;
+		}
+		return thames_query_job_fault(file_priv, args);
 	default:
 		return -EINVAL;
 	}
@@ -333,12 +360,18 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
 	atomic64_set(&tdev->last_context_serial, seed);
 	atomic64_set(&tdev->mapping_generation, seed);
 
+	tdev->reset_wq = alloc_ordered_workqueue("thames-reset", 0);
+	if (!tdev->reset_wq)
+		return ERR_PTR(-ENOMEM);
+
 	return tdev;
 }
 
 static void thames_device_fini(struct thames_device *tdev)
 {
 	WARN_ON(tdev->num_cores > 0);
+
+	destroy_workqueue(tdev->reset_wq);
 }
 
 static int thames_platform_probe(struct platform_device *pdev)
diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
index 2115f7eae589..65e5c900313d 100644
--- a/drivers/accel/thames/thames_drv.h
+++ b/drivers/accel/thames/thames_drv.h
@@ -53,6 +53,11 @@ struct thames_file_priv {
 
 	struct drm_gem_dma_object *user_stack;
 	struct drm_gem_dma_object *save_area;
+
+	/* Bit 0 is the newest completed Job, set when it faulted. */
+	spinlock_t fault_lock;
+	u16 fault_history;
+	bool banned;
 };
 
 void thames_file_priv_put(struct thames_file_priv *priv);
diff --git a/drivers/accel/thames/thames_job.c b/drivers/accel/thames/thames_job.c
index 6560106d47a8..1f1a00eb8c63 100644
--- a/drivers/accel/thames/thames_job.c
+++ b/drivers/accel/thames/thames_job.c
@@ -9,7 +9,9 @@
 #include <linux/dma-fence-chain.h>
 #include <linux/iopoll.h>
 #include <linux/mailbox_client.h>
+#include <linux/math64.h>
 #include <linux/platform_device.h>
+#include <linux/time64.h>
 #include <linux/xarray.h>
 
 #include <drm/drm_drv.h>
@@ -45,6 +47,8 @@ static void thames_job_put(struct thames_job *job);
  * it fires is on a stalled core.
  */
 #define THAMES_DEVICE_TIMEOUT_MS 500
+#define THAMES_HOST_TIMEOUT_MS 2000
+static_assert(THAMES_HOST_TIMEOUT_MS > THAMES_DEVICE_TIMEOUT_MS);
 
 struct thames_submit_bos {
 	struct kref refcount;
@@ -72,6 +76,24 @@ static void thames_submit_bos_put(struct thames_submit_bos *bos)
 		kref_put(&bos->refcount, thames_submit_bos_free);
 }
 
+#define THAMES_BAN_FAULTS 3
+#define THAMES_BAN_SPAN 10
+static_assert(THAMES_BAN_SPAN <= 16);
+
+static_assert(THAMES_FAULT_KIND_KERNEL == DRM_THAMES_FAULT_KERNEL);
+static_assert(THAMES_FAULT_KIND_FIRMWARE == DRM_THAMES_FAULT_FIRMWARE);
+static_assert(THAMES_FAULT_KIND_TIMEOUT == DRM_THAMES_FAULT_TIMEOUT);
+static_assert(THAMES_FAULT_REG_IERR == DRM_THAMES_REG_IERR);
+static_assert(THAMES_FAULT_REG_IEAR == DRM_THAMES_REG_IEAR);
+static_assert(THAMES_FAULT_REG_IESR == DRM_THAMES_REG_IESR);
+static_assert(THAMES_FAULT_REG_NRP == DRM_THAMES_REG_NRP);
+static_assert(THAMES_FAULT_REG_NTSR == DRM_THAMES_REG_NTSR);
+static_assert(THAMES_FAULT_REG_SE0_FAR == DRM_THAMES_REG_SE0_FAR);
+static_assert(THAMES_FAULT_REG_SE0_FSR == DRM_THAMES_REG_SE0_FSR);
+static_assert(THAMES_FAULT_REG_SE1_FAR == DRM_THAMES_REG_SE1_FAR);
+static_assert(THAMES_FAULT_REG_SE1_FSR == DRM_THAMES_REG_SE1_FSR);
+static_assert(THAMES_FAULT_REGISTERS == DRM_THAMES_FAULT_REGISTERS);
+
 static const char *thames_fence_get_driver_name(struct dma_fence *fence)
 {
 	return "thames";
@@ -146,11 +168,96 @@ static int thames_job_hw_submit(struct thames_core *core, struct thames_sched_jo
 	return 0;
 }
 
+/*
+ * Copied once: the record is the Job's only if its seqno matches, since a later
+ * Job's slot keeps an earlier record.
+ */
+static bool thames_job_read_fault(struct thames_core *core, struct thames_sched_job *job,
+				  struct thames_fault_record *record)
+{
+	memcpy(record, &core->ipc->fault_records[job->ring_index], sizeof(*record));
+
+	if (record->seqno != job->seqno)
+		return false;
+
+	if (record->kind < DRM_THAMES_FAULT_KERNEL || record->kind > DRM_THAMES_FAULT_TIMEOUT) {
+		dev_warn_ratelimited(core->dev,
+				     "Job seqno %llu: fault record of unknown kind %u refused\n",
+				     job->seqno, record->kind);
+		return false;
+	}
+
+	return true;
+}
+
+/*
+ * The firmware faulted with its Ring empty: it writes the record at its head's
+ * slot under that slot's stale seqno and takes no doorbell after.
+ */
+static bool thames_job_read_empty_ring_fault(struct thames_core *core, struct thames_sched_job *job,
+					     struct thames_fault_record *record)
+{
+	u32 head = READ_ONCE(core->ipc->ring.head);
+
+	if (head % THAMES_RING_DEPTH != job->ring_index)
+		return false;
+
+	/* Read the fault record after the head that says it is there. */
+	rmb();
+	memcpy(record, &core->ipc->fault_records[job->ring_index], sizeof(*record));
+
+	return record->kind == DRM_THAMES_FAULT_FIRMWARE;
+}
+
+static void thames_job_set_fault(struct thames_sched_job *sched_job,
+				 const struct thames_fault_record *record)
+{
+	struct thames_job *job = sched_job->job;
+
+	guard(mutex)(&job->lock);
+
+	job->fault_kind = record->kind;
+	memcpy(job->fault_registers, record->registers, sizeof(job->fault_registers));
+	job->fault_valid = true;
+}
+
+static void thames_context_count_job(struct thames_core *core, struct thames_sched_job *job,
+				     bool faulted)
+{
+	struct thames_file_priv *priv = job->file_priv;
+	bool ban = false;
+	u16 history;
+
+	scoped_guard(spinlock, &priv->fault_lock) {
+		history = (priv->fault_history << 1) | faulted;
+		history &= GENMASK(THAMES_BAN_SPAN - 1, 0);
+		priv->fault_history = history;
+
+		if (!priv->banned && hweight16(history) >= THAMES_BAN_FAULTS) {
+			WRITE_ONCE(priv->banned, true);
+			ban = true;
+		}
+	}
+
+	if (ban)
+		dev_warn(core->dev,
+			 "Context %u banned: %u of its last %u Jobs faulted (Job seqno %llu)\n",
+			 priv->context_id, hweight16(history), THAMES_BAN_SPAN, job->seqno);
+}
+
+static bool thames_context_banned(struct thames_file_priv *priv)
+{
+	return READ_ONCE(priv->banned);
+}
+
 void thames_job_complete(struct thames_core *core)
 {
 	struct thames_sched_job *job, *tmp;
+	struct thames_fault_record record;
 	LIST_HEAD(done);
+	u32 hz = core->tdev->caps.timestamp_hz;
 	u64 completed, signalled;
+	bool faulted, firmware_fault = false;
 
 	completed = READ_ONCE(core->ipc->fence.seqno);
 	/* Read the entries after the Fence that covers them. */
@@ -173,16 +280,52 @@ void thames_job_complete(struct thames_core *core)
 		job->stamps[4] = fw->kernel_entry_stamp;
 		job->stamps[5] = fw->kernel_exit_stamp;
 
+		faulted = false;
 		if (fw->fence_value == THAMES_FENCE_ERROR) {
+			if (thames_job_read_fault(core, job, &record)) {
+				thames_job_set_fault(job, &record);
+				faulted = record.kind == DRM_THAMES_FAULT_KERNEL ||
+					  record.kind == DRM_THAMES_FAULT_TIMEOUT;
+			}
+			if (faulted && record.kind == DRM_THAMES_FAULT_TIMEOUT)
+				dev_warn_ratelimited(core->dev,
+						     "Job seqno %llu of Context %u: its Kernel ran past the device's timeout (entry to exit %llu us)\n",
+						     job->seqno, job->file_priv->context_id,
+						     mul_u64_u32_div(job->stamps[3] -
+								     job->stamps[2],
+								     USEC_PER_SEC, hz));
+			else if (faulted)
+				dev_warn_ratelimited(core->dev,
+						     "Job seqno %llu of Context %u: its Kernel faulted\n",
+						     job->seqno, job->file_priv->context_id);
+			else
+				dev_warn_ratelimited(core->dev,
+						     "Job seqno %llu: firmware could not honour the entry\n",
+						     job->seqno);
 			dma_fence_set_error(job->done_fence, -EIO);
 		}
 
+		thames_context_count_job(core, job, faulted);
+
 		list_del_init(&job->node);
 		signalled = arch_timer_read_counter();
 		dma_fence_signal(job->done_fence);
 		trace_thames_job_done(core, job, signalled);
 		thames_sched_job_put(job);
 	}
+
+	/*
+	 * The firmware's own fault moves neither the Fence nor the head: it
+	 * rings the doorbell and parks in IDLE.
+	 */
+	scoped_guard(mutex, &core->job_lock) {
+		job = list_first_entry_or_null(&core->submitted, struct thames_sched_job, node);
+		if (job && !thames_core_is_dead(core) && thames_job_read_fault(core, job, &record))
+			firmware_fault = record.kind == DRM_THAMES_FAULT_FIRMWARE;
+	}
+
+	if (firmware_fault)
+		drm_sched_fault(&core->sched);
 }
 
 static int thames_acquire_object_fences(struct drm_gem_object **bos, int bo_count,
@@ -215,6 +358,12 @@ static void thames_attach_object_fences(struct drm_gem_object **bos, int bo_coun
 /* Nothing here fails, so a submit call pushes every Job or none. */
 static void thames_job_push(struct thames_sched_job *sched_job)
 {
+	struct thames_job *job = sched_job->job;
+
+	/* Before the push, so the submission's own record cannot be lost. */
+	scoped_guard(mutex, &job->lock)
+		job->fault_valid = false;
+
 	scoped_guard(mutex, &sched_job->tdev->sched_lock) {
 		drm_sched_job_arm(&sched_job->base);
 
@@ -270,6 +419,12 @@ static struct dma_fence *thames_job_run(struct drm_sched_job *sched_job)
 	if (unlikely(job->base.s_fence->finished.error))
 		return NULL;
 
+	if (thames_core_is_dead(core))
+		return ERR_PTR(-ENODEV);
+
+	if (thames_context_banned(job->file_priv))
+		return ERR_PTR(-ECANCELED);
+
 	/* The done fence keeps the seqno of the Job's first push. */
 	job->seqno = ++core->emit_seqno;
 	if (!dma_fence_was_initialized(job->done_fence))
@@ -285,6 +440,223 @@ static struct dma_fence *thames_job_run(struct drm_sched_job *sched_job)
 	return dma_fence_get(job->done_fence);
 }
 
+void thames_job_release_held(struct thames_core *core)
+{
+	struct thames_sched_job *job, *tmp;
+	LIST_HEAD(held);
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_init(&core->held, &held);
+
+	list_for_each_entry_safe(job, tmp, &held, node) {
+		list_del_init(&job->node);
+		thames_sched_job_put(job);
+	}
+}
+
+/*
+ * Held, not freed: the system firmware may refuse the stop and the core run on.
+ */
+void thames_job_fail_submitted(struct thames_core *core)
+{
+	struct thames_sched_job *job;
+	LIST_HEAD(lost);
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_init(&core->submitted, &lost);
+
+	list_for_each_entry(job, &lost, node)
+		thames_job_fail_done_fence(job, -ENODEV);
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_tail_init(&lost, &core->held);
+}
+
+static void thames_job_recover(struct thames_core *core, struct drm_sched_job *sched_job,
+			       struct thames_sched_job *bad,
+			       const struct thames_fault_record *record)
+{
+	struct drm_gpu_scheduler *sched = &core->sched;
+	struct thames_sched_job *job;
+	struct drm_sched_job *s_job;
+	unsigned int resubmitted = 0;
+	LIST_HEAD(lost);
+	int err;
+
+	drm_sched_wqueue_stop(sched);
+
+	/* Failed before drm_sched_stop(), so the stop finds it complete. */
+	if (bad) {
+		scoped_guard(mutex, &core->job_lock)
+			list_del_init(&bad->node);
+
+		dev_warn(core->dev, "Job seqno %llu of Context %u: the firmware faulted\n",
+			 bad->seqno, bad->file_priv->context_id);
+		thames_job_set_fault(bad, record);
+		thames_context_count_job(core, bad, true);
+		dma_fence_set_error(bad->done_fence, -EIO);
+		dma_fence_signal(bad->done_fence);
+		thames_sched_job_put(bad);
+	} else {
+		dev_warn(core->dev, THAMES_CORE_FMT
+			 ": the firmware faulted with its Ring empty (IERR 0x%llx, NRP 0x%llx)\n",
+			 THAMES_CORE_ARG(core), record->registers[THAMES_FAULT_REG_IERR],
+			 record->registers[THAMES_FAULT_REG_NRP]);
+	}
+
+	drm_sched_stop(sched, sched_job);
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_init(&core->submitted, &lost);
+
+	/*
+	 * Kept until the reset has stopped the core, which may still run one of
+	 * their Kernels.
+	 */
+	err = thames_core_reset(core);
+	if (err) {
+		thames_core_test_and_set_offline(core);
+		dev_crit(core->dev, THAMES_CORE_FMT
+			 " could not be reset (%d); it is offline until it is stopped and started again through remoteproc\n",
+			 THAMES_CORE_ARG(core), err);
+
+		list_for_each_entry(job, &lost, node)
+			thames_job_fail_done_fence(job, -ENODEV);
+
+		scoped_guard(mutex, &core->job_lock)
+			list_splice_tail_init(&lost, &core->held);
+
+		drm_sched_start(sched, -ENODEV);
+		return;
+	}
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_tail_init(&lost, &core->held);
+
+	thames_job_release_held(core);
+
+	drm_sched_for_each_pending_job(s_job, sched, NULL) {
+		struct dma_fence *fence = thames_job_run(s_job);
+
+		if (IS_ERR_OR_NULL(fence)) {
+			thames_job_fail_done_fence(to_thames_sched_job(s_job),
+						   fence ? PTR_ERR(fence) : -ECANCELED);
+			continue;
+		}
+
+		s_job->s_fence->parent = fence;
+		resubmitted++;
+	}
+
+	dev_info(core->dev, THAMES_CORE_FMT " restarted, %u Jobs resubmitted\n",
+		 THAMES_CORE_ARG(core), resubmitted);
+
+	drm_sched_start(sched, 0);
+}
+
+/*
+ * Only job_lock: the core's fence lock may be held by a completion thread that
+ * died.
+ */
+static bool thames_job_in_completion(struct thames_core *core, struct thames_sched_job *job)
+{
+	struct thames_sched_job *pending;
+
+	if (dma_fence_is_signaled(&job->base.s_fence->finished))
+		return false;
+
+	guard(mutex)(&core->job_lock);
+	list_for_each_entry(pending, &core->submitted, node) {
+		if (pending == job)
+			return false;
+	}
+
+	return true;
+}
+
+static enum drm_gpu_sched_stat thames_job_timedout(struct drm_sched_job *sched_job)
+{
+	struct thames_sched_job *job = to_thames_sched_job(sched_job);
+	struct thames_sched_job *pending, *oldest = NULL;
+	struct thames_core *core = container_of(sched_job->sched, struct thames_core, sched);
+	struct thames_fault_record record;
+	LIST_HEAD(dead_jobs);
+	bool found = false;
+
+	if (core->completion_stuck)
+		return DRM_GPU_SCHED_STAT_ENODEV;
+
+	if (thames_job_in_completion(core, job)) {
+		if (core->unfinished_seqno != job->seqno) {
+			core->unfinished_seqno = job->seqno;
+			return DRM_GPU_SCHED_STAT_NO_HANG;
+		}
+
+		core->completion_stuck = true;
+		thames_core_test_and_set_offline(core);
+		dev_crit(core->dev, "Job seqno %llu on " THAMES_CORE_FMT
+			 ": its completion never finished; taking the core offline, a reboot is needed to use it again\n",
+			 job->seqno, THAMES_CORE_ARG(core));
+		return DRM_GPU_SCHED_STAT_ENODEV;
+	}
+
+	thames_job_complete(core);
+
+	scoped_guard(mutex, &core->job_lock) {
+		oldest = list_first_entry_or_null(&core->submitted, struct thames_sched_job, node);
+		list_for_each_entry(pending, &core->submitted, node) {
+			if (pending == job) {
+				found = true;
+				break;
+			}
+		}
+	}
+
+	if (!found)
+		return DRM_GPU_SCHED_STAT_NO_HANG;
+
+	/*
+	 * Any other timeout is a core that may still run: it is taken offline,
+	 * never stopped.
+	 */
+	if (!thames_core_is_dead(core) && thames_job_read_fault(core, oldest, &record) &&
+	    record.kind == DRM_THAMES_FAULT_FIRMWARE) {
+		thames_job_recover(core, sched_job, oldest, &record);
+		return DRM_GPU_SCHED_STAT_RESET;
+	}
+
+	if (!thames_core_is_dead(core) && thames_job_read_empty_ring_fault(core, oldest, &record)) {
+		thames_job_recover(core, sched_job, NULL, &record);
+		return DRM_GPU_SCHED_STAT_RESET;
+	}
+
+	if (thames_core_test_and_set_offline(core))
+		dev_crit(core->dev, "Job seqno %llu timed out on " THAMES_CORE_FMT
+			 "; taking the core offline until it is stopped and started again through remoteproc\n",
+			 job->seqno, THAMES_CORE_ARG(core));
+
+	drm_sched_stop(&core->sched, sched_job);
+
+	/*
+	 * The core may still run their Kernels: it keeps the push's reference,
+	 * and with it the Context and the buffers.
+	 */
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_init(&core->submitted, &dead_jobs);
+
+	list_for_each_entry(pending, &dead_jobs, node) {
+		dma_fence_set_error(pending->done_fence, -ENODEV);
+		dma_fence_signal(pending->done_fence);
+	}
+
+	scoped_guard(mutex, &core->job_lock)
+		list_splice_tail_init(&dead_jobs, &core->held);
+
+	drm_sched_start(&core->sched, -ENODEV);
+
+	return DRM_GPU_SCHED_STAT_RESET;
+}
+
 /*
  * A done fence not signalled here is a Job on a live core's Ring that may still
  * run.
@@ -305,6 +677,7 @@ static void thames_job_cancel(struct drm_sched_job *sched_job)
 
 static const struct drm_sched_backend_ops thames_sched_ops = {
 	.run_job = thames_job_run,
+	.timedout_job = thames_job_timedout,
 	.free_job = thames_sched_job_free,
 	.cancel_job = thames_job_cancel,
 };
@@ -316,7 +689,8 @@ int thames_job_init(struct thames_core *core)
 		.ops = &thames_sched_ops,
 		.num_rqs = 1,
 		.credit_limit = THAMES_RING_DEPTH,
-		.timeout = MAX_SCHEDULE_TIMEOUT,
+		.timeout = msecs_to_jiffies(THAMES_HOST_TIMEOUT_MS),
+		.timeout_wq = core->tdev->reset_wq,
 		.name = dev_name(core->dev),
 		.dev = core->dev,
 	};
@@ -326,6 +700,9 @@ int thames_job_init(struct thames_core *core)
 
 	spin_lock_init(&core->fence_lock);
 	mutex_init(&core->job_lock);
+	atomic_set(&core->dead, 0);
+	core->unfinished_seqno = 0;
+	core->completion_stuck = false;
 
 	INIT_LIST_HEAD(&core->submitted);
 	INIT_LIST_HEAD(&core->held);
@@ -414,6 +791,9 @@ int thames_job_suspend(struct thames_core *core)
 	cancel_delayed_work_sync(&sched->work_tdr);
 	drm_sched_wqueue_stop(sched);
 
+	if (thames_core_is_dead(core))
+		return 0;
+
 	scoped_guard(mutex, &core->job_lock)
 		pending = list_count_nodes(&core->submitted);
 	wait_ms = max_t(unsigned int, pending, 1) * THAMES_SUSPEND_TIMEOUT_MS;
@@ -446,6 +826,40 @@ void thames_job_resume(struct thames_core *core)
 	drm_sched_wqueue_start(&core->sched);
 }
 
+static bool thames_job_ring_drained(struct thames_core *core)
+{
+	thames_job_complete(core);
+
+	guard(mutex)(&core->job_lock);
+	return list_empty(&core->submitted);
+}
+
+void thames_job_drain(struct thames_core *core)
+{
+	struct drm_gpu_scheduler *sched = &core->sched;
+	size_t pending;
+	bool drained;
+
+	drm_sched_wqueue_stop(sched);
+	cancel_delayed_work_sync(&sched->work_tdr);
+	drm_sched_wqueue_stop(sched);
+
+	if (thames_core_is_dead(core))
+		return;
+
+	scoped_guard(mutex, &core->job_lock)
+		pending = list_count_nodes(&core->submitted);
+
+	if (!pending)
+		return;
+
+	read_poll_timeout(thames_job_ring_drained, drained, drained, USEC_PER_MSEC,
+			  pending * THAMES_SUSPEND_TIMEOUT_MS * USEC_PER_MSEC, false, core);
+
+	cancel_delayed_work_sync(&sched->work_tdr);
+	drm_sched_wqueue_stop(sched);
+}
+
 int thames_job_open(struct thames_file_priv *thames_priv)
 {
 	struct thames_device *tdev = thames_priv->tdev;
@@ -759,6 +1173,7 @@ static void thames_submit_syncs_signal(struct thames_submit_syncs *syncs, struct
 int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file)
 {
 	struct drm_thames_job_submit *args = data;
+	struct thames_file_priv *file_priv = file->driver_priv;
 	struct thames_submit_syncs syncs = {};
 	struct thames_submit_bos *bos = NULL;
 	struct ww_acquire_ctx acquire_ctx;
@@ -780,6 +1195,13 @@ int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file
 	if (IS_ERR(handles))
 		return PTR_ERR(handles);
 
+	if (thames_context_banned(file_priv)) {
+		dev_warn_ratelimited(dev->dev, "Context %u is banned; submission refused\n",
+				     file_priv->context_id);
+		ret = -ECANCELED;
+		goto out_free;
+	}
+
 	ret = thames_submit_syncs_get(file, &args->syncs, &syncs);
 	if (ret)
 		goto out_syncs;
@@ -844,6 +1266,7 @@ int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file
 	thames_submit_bos_put(bos);
 out_syncs:
 	thames_submit_syncs_free(&syncs);
+out_free:
 	kvfree(handles);
 	return ret;
 }
@@ -867,3 +1290,30 @@ int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file
 	return 0;
 }
 
+int thames_job_query_fault(struct thames_file_priv *file_priv, struct drm_thames_job_fault *fault)
+{
+	struct thames_job *job;
+	int ret = 0;
+
+	xa_lock(&file_priv->jobs);
+	job = xa_load(&file_priv->jobs, fault->job);
+	if (job)
+		kref_get(&job->refcount);
+	xa_unlock(&file_priv->jobs);
+
+	if (!job)
+		return -ENOENT;
+
+	scoped_guard(mutex, &job->lock) {
+		if (job->fault_valid) {
+			fault->kind = job->fault_kind;
+			memcpy(fault->registers, job->fault_registers, sizeof(fault->registers));
+		} else {
+			ret = -ENOENT;
+		}
+	}
+
+	thames_job_put(job);
+
+	return ret;
+}
diff --git a/drivers/accel/thames/thames_job.h b/drivers/accel/thames/thames_job.h
index a851febe8f98..c9aaed79fe82 100644
--- a/drivers/accel/thames/thames_job.h
+++ b/drivers/accel/thames/thames_job.h
@@ -55,17 +55,25 @@ struct thames_job {
 	struct thames_transfer_table transfers;
 
 	struct mutex lock;
+
+	bool fault_valid;
+	u32 fault_kind;
+	u64 fault_registers[DRM_THAMES_FAULT_REGISTERS];
 };
 
 int thames_ioctl_job_create(struct drm_device *dev, void *data, struct drm_file *file);
 int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file);
 int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file *file);
+int thames_job_query_fault(struct thames_file_priv *file_priv, struct drm_thames_job_fault *fault);
 
 void thames_job_complete(struct thames_core *core);
+void thames_job_fail_submitted(struct thames_core *core);
+void thames_job_release_held(struct thames_core *core);
 
 int thames_job_init(struct thames_core *core);
 int thames_job_suspend(struct thames_core *core);
 void thames_job_resume(struct thames_core *core);
+void thames_job_drain(struct thames_core *core);
 void thames_job_fini(struct thames_core *core);
 int thames_job_open(struct thames_file_priv *thames_priv);
 void thames_job_close(struct thames_file_priv *thames_priv);
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
index bd2b4271b896..d193967485cb 100644
--- a/include/uapi/drm/thames_accel.h
+++ b/include/uapi/drm/thames_accel.h
@@ -251,6 +251,11 @@ enum drm_thames_dev_query_type {
 	/** @DRM_THAMES_DEV_QUERY_CAPS: The capability query, &drm_thames_caps. */
 	DRM_THAMES_DEV_QUERY_CAPS = 0,
 
+	/**
+	 * @DRM_THAMES_DEV_QUERY_JOB_FAULT: The fault record of a failed Job,
+	 * &drm_thames_job_fault.
+	 */
+	DRM_THAMES_DEV_QUERY_JOB_FAULT,
 };
 
 /**
@@ -382,6 +387,99 @@ enum drm_thames_fault_kind {
 	DRM_THAMES_FAULT_TIMEOUT,
 };
 
+/**
+ * enum drm_thames_fault_register - Index into &drm_thames_job_fault.registers
+ *
+ * The control registers the fault handler saves, by the names SPRUIP0 gives
+ * them.  A register another core generation reports is appended here and at the
+ * end of &drm_thames_job_fault.registers, which the query's size convention
+ * lets grow; a register the firmware did not save reads 0.
+ */
+enum drm_thames_fault_register {
+	/** @DRM_THAMES_REG_IERR: Internal exception report register. */
+	DRM_THAMES_REG_IERR = 0,
+
+	/** @DRM_THAMES_REG_IEAR: Internal exception address register. */
+	DRM_THAMES_REG_IEAR,
+
+	/** @DRM_THAMES_REG_IESR: Internal exception status register. */
+	DRM_THAMES_REG_IESR,
+
+	/** @DRM_THAMES_REG_NRP: The return pointer saved at the exception, the faulting PC. */
+	DRM_THAMES_REG_NRP,
+
+	/** @DRM_THAMES_REG_NTSR: The task state register saved at the exception. */
+	DRM_THAMES_REG_NTSR,
+
+	/**
+	 * @DRM_THAMES_REG_SE0_FAR: Streaming engine 0's fault address register:
+	 * the virtual address of the earliest fault on stream 0, 128-byte
+	 * aligned; 0 when no stream has faulted since one last closed, or the
+	 * engine could not capture the address.
+	 */
+	DRM_THAMES_REG_SE0_FAR,
+
+	/**
+	 * @DRM_THAMES_REG_SE0_FSR: Streaming engine 0's fault status register:
+	 * bit 12 ERROR and a 12-bit syndrome below it, SPRUIP0 figures 3-139 to
+	 * 3-144.
+	 */
+	DRM_THAMES_REG_SE0_FSR,
+
+	/**
+	 * @DRM_THAMES_REG_SE1_FAR: Streaming engine 1's fault address register:
+	 * the virtual address of the earliest fault on stream 1, 128-byte
+	 * aligned; 0 when no stream has faulted since one last closed, or the
+	 * engine could not capture the address.
+	 */
+	DRM_THAMES_REG_SE1_FAR,
+
+	/**
+	 * @DRM_THAMES_REG_SE1_FSR: Streaming engine 1's fault status register:
+	 * bit 12 ERROR and a 12-bit syndrome below it, SPRUIP0 figures 3-139 to
+	 * 3-144.
+	 */
+	DRM_THAMES_REG_SE1_FSR,
+};
+
+#define DRM_THAMES_FAULT_REGISTERS	9
+
+/**
+ * struct drm_thames_job_fault - Argument of DRM_THAMES_DEV_QUERY_JOB_FAULT
+ *
+ * What the device reported for the Job's last failed submission, filled by the
+ * firmware before it signalled the Fence, read by the host on bounded fields
+ * and never trusted beyond them; the registers raw, decoded by userspace.
+ * Every kind counts towards the Context's ban; the kernel resets the core for
+ * %DRM_THAMES_FAULT_FIRMWARE.  The record is cleared whenever the Job is
+ * submitted, a refused submit call not counting, and set when a submission of
+ * it fails with one; the query returns -ENOENT while it is clear.  A client
+ * that submits a Job again before its last submission completed may read the
+ * earlier one's record.
+ *
+ * When the kernel resets a core, the Jobs on its Ring other than the one that
+ * failed are pushed again after the reset, on the same fences.  Such a Job may
+ * have started before the reset, so its Kernel may run twice, the second time
+ * from its start: a Kernel whose outputs depend on running once, one that reads
+ * what it wrote in place, must not rely on a single run.
+ */
+struct drm_thames_job_fault {
+	/** @job: Handle of the Job, input. */
+	__u32 job;
+
+	/** @kind: One of &enum drm_thames_fault_kind. */
+	__u32 kind;
+
+	/**
+	 * @registers: The saved control registers, indexed by
+	 * &enum drm_thames_fault_register.  For a timeout only NRP, NTSR and the
+	 * four stream registers, the Kernel's PC and TSR and the streams' state
+	 * when the timer interrupted it; every register is 0 when the timeout
+	 * is transfers that did not drain after the Kernel returned.
+	 */
+	__u64 registers[DRM_THAMES_FAULT_REGISTERS];
+};
+
 /**
  * struct drm_thames_dev_query - Arguments passed to DRM_IOCTL_THAMES_DEV_QUERY
  */

-- 
2.54.0

[PATCH v3 11/16] accel/thames: Add IOCTL for memory synchronization

[Tomeu Vizoso]

These DSP cores have their own access to the memory bus, and it isn't
cache coherent with the CPUs.

Add IOCTLs so userspace can mark when the caches need to be flushed, and
wait for every Job still using a buffer, readers included, before the
CPU accesses it.

Initially based on the same IOCTLs from the Etnaviv driver.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

---

v2:
- Add thames_accel.h UAPI header (Robert Nelson).
---
 drivers/accel/thames/thames_drv.c |  3 ++
 drivers/accel/thames/thames_gem.c | 63 +++++++++++++++++++++++++++++++++++++++
 drivers/accel/thames/thames_gem.h |  4 +++
 include/uapi/drm/thames_accel.h   | 56 ++++++++++++++++++++++++++++++++++
 4 files changed, 126 insertions(+)

diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
index 5f704cbfe1d0..798b4743f336 100644
--- a/drivers/accel/thames/thames_drv.c
+++ b/drivers/accel/thames/thames_drv.c
@@ -252,6 +252,7 @@ static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,	\
 
 THAMES_IOCTL_ENTERED(bo_create)
 THAMES_IOCTL_ENTERED(bo_mmap_offset)
+THAMES_IOCTL_ENTERED(bo_fini)
 THAMES_IOCTL_ENTERED(dev_query)
 THAMES_IOCTL_ENTERED(job_create)
 THAMES_IOCTL_ENTERED(job_submit)
@@ -262,6 +263,8 @@ static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
 #define THAMES_IOCTL(n, func) DRM_IOCTL_DEF_DRV(THAMES_##n, thames_ioctl_##func, 0)
 	THAMES_IOCTL(BO_CREATE, bo_create_entered),
 	THAMES_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset_entered),
+	THAMES_IOCTL(BO_PREP, bo_prep),
+	THAMES_IOCTL(BO_FINI, bo_fini_entered),
 	THAMES_IOCTL(DEV_QUERY, dev_query_entered),
 	THAMES_IOCTL(JOB_CREATE, job_create_entered),
 	THAMES_IOCTL(JOB_SUBMIT, job_submit_entered),
diff --git a/drivers/accel/thames/thames_gem.c b/drivers/accel/thames/thames_gem.c
index c9fd08c7aaf4..5a4dc497768f 100644
--- a/drivers/accel/thames/thames_gem.c
+++ b/drivers/accel/thames/thames_gem.c
@@ -942,3 +942,66 @@ int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_
 	return 0;
 }
 
+int thames_ioctl_bo_prep(struct drm_device *ddev, void *data, struct drm_file *file)
+{
+	struct drm_thames_bo_prep *args = data;
+	struct thames_gem_object *bo;
+	struct drm_gem_object *gem_obj;
+	unsigned long timeout = drm_timeout_abs_to_jiffies(args->timeout_ns);
+	long ret = 0;
+
+	if (args->reserved != 0)
+		return -EINVAL;
+
+	gem_obj = drm_gem_object_lookup(file, args->handle);
+	if (!gem_obj)
+		return -ENOENT;
+
+	ret = dma_resv_wait_timeout(gem_obj->resv, DMA_RESV_USAGE_READ, true, timeout);
+	if (ret == 0)
+		ret = timeout ? -ETIMEDOUT : -EBUSY;
+	else if (ret > 0)
+		ret = 0;
+
+	bo = to_thames_bo(gem_obj);
+	if (bo->flags & DRM_THAMES_BO_CONTIGUOUS)
+		dma_sync_single_for_cpu(ddev->dev, bo->base.dma_addr, gem_obj->size,
+					DMA_BIDIRECTIONAL);
+	else
+		dma_sync_sgtable_for_cpu(ddev->dev, bo->shmem.sgt, DMA_BIDIRECTIONAL);
+	/* Order the cache maintenance before the CPU reads. */
+	mb();
+
+	drm_gem_object_put(gem_obj);
+
+	return ret;
+}
+
+int thames_ioctl_bo_fini(struct drm_device *ddev, void *data, struct drm_file *file)
+{
+	struct drm_thames_bo_fini *args = data;
+	struct thames_gem_object *bo;
+	struct drm_gem_object *gem_obj;
+
+	if (args->reserved != 0)
+		return -EINVAL;
+
+	gem_obj = drm_gem_object_lookup(file, args->handle);
+	if (!gem_obj)
+		return -ENOENT;
+
+	bo = to_thames_bo(gem_obj);
+	if (bo->flags & DRM_THAMES_BO_CONTIGUOUS) {
+		dma_sync_single_for_device(ddev->dev, bo->base.dma_addr, gem_obj->size,
+					   DMA_BIDIRECTIONAL);
+	} else {
+		dma_sync_sgtable_for_device(ddev->dev, bo->shmem.sgt, DMA_BIDIRECTIONAL);
+	}
+
+	/* Order the cache maintenance before the return. */
+	mb();
+
+	drm_gem_object_put(gem_obj);
+
+	return 0;
+}
diff --git a/drivers/accel/thames/thames_gem.h b/drivers/accel/thames/thames_gem.h
index c9d0e170dad3..7c9d375d1ebd 100644
--- a/drivers/accel/thames/thames_gem.h
+++ b/drivers/accel/thames/thames_gem.h
@@ -51,6 +51,10 @@ int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file
 
 int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_file *file);
 
+int thames_ioctl_bo_prep(struct drm_device *ddev, void *data, struct drm_file *file);
+
+int thames_ioctl_bo_fini(struct drm_device *ddev, void *data, struct drm_file *file);
+
 int thames_ioctl_bo_bind(struct drm_device *ddev, void *data, struct drm_file *file);
 
 int thames_context_create(struct thames_file_priv *priv);
diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
index d193967485cb..ae43885bdcb5 100644
--- a/include/uapi/drm/thames_accel.h
+++ b/include/uapi/drm/thames_accel.h
@@ -54,6 +54,12 @@ enum drm_thames_ioctl_id {
 	 */
 	DRM_THAMES_BO_MMAP_OFFSET,
 
+	/** @DRM_THAMES_BO_PREP: Prepare a BO for CPU access after DSP writes. */
+	DRM_THAMES_BO_PREP,
+
+	/** @DRM_THAMES_BO_FINI: Finish CPU access and prepare BO for DSP access. */
+	DRM_THAMES_BO_FINI,
+
 	/** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
 	DRM_THAMES_DEV_QUERY,
 
@@ -143,6 +149,52 @@ struct drm_thames_bo_mmap_offset {
 	__u64 offset;
 };
 
+/**
+ * struct drm_thames_bo_prep - ioctl argument for preparing a BO for CPU access.
+ *
+ * Waits for the DSP to be done with the buffer and invalidates the CPU caches,
+ * so that what the DSP wrote is visible to the CPU.  The DSP is not coherent
+ * with the CPU, and the wait and the cache invalidation are deliberately the
+ * same operation: userspace has no way of skipping the wait and keeping the
+ * invalidation, so a mapping is never handed out with stale cache lines.
+ *
+ * Returns 0 once the buffer is ready for CPU access, -ETIMEDOUT if
+ * &drm_thames_bo_prep.timeout_ns passed before the DSP was done with it, or
+ * -EBUSY if that timestamp was not in the future at the call, 0 included, and
+ * the buffer is still in use.  It must never return -EAGAIN: userspace reaches
+ * this ioctl through drmIoctl(), which restarts on both -EINTR and -EAGAIN and
+ * would spin forever.
+ */
+struct drm_thames_bo_prep {
+	/** @handle: Handle of the buffer to prepare for CPU access. */
+	__u32 handle;
+
+	/** @reserved: MBZ. */
+	__u32 reserved;
+
+	/**
+	 * @timeout_ns: Absolute CLOCK_MONOTONIC timestamp at which the wait
+	 * gives up.  A timestamp in the past, 0 included, makes the ioctl
+	 * return without blocking.
+	 */
+	__s64 timeout_ns;
+};
+
+/**
+ * struct drm_thames_bo_fini - ioctl argument for finishing CPU access to a BO.
+ *
+ * Flushes the CPU caches, so that what the CPU wrote is visible to the DSP.
+ * Userspace has to call this after writing to a mapping and before the buffer
+ * is used by a job, as the DSP is not coherent with the CPU.
+ */
+struct drm_thames_bo_fini {
+	/** @handle: Handle of the buffer the CPU is done writing to. */
+	__u32 handle;
+
+	/** @reserved: MBZ. */
+	__u32 reserved;
+};
+
 /**
  * struct drm_thames_obj_array - An array of versioned objects passed by pointer.
  *
@@ -768,6 +820,10 @@ enum {
 		DRM_IOCTL_THAMES(WR, BO_CREATE, bo_create),
 	DRM_IOCTL_THAMES_BO_MMAP_OFFSET =
 		DRM_IOCTL_THAMES(WR, BO_MMAP_OFFSET, bo_mmap_offset),
+	DRM_IOCTL_THAMES_BO_PREP =
+		DRM_IOCTL_THAMES(WR, BO_PREP, bo_prep),
+	DRM_IOCTL_THAMES_BO_FINI =
+		DRM_IOCTL_THAMES(WR, BO_FINI, bo_fini),
 	DRM_IOCTL_THAMES_DEV_QUERY =
 		DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
 	DRM_IOCTL_THAMES_JOB_CREATE =

-- 
2.54.0

[PATCH v3 12/16] accel/thames: Install memory mappings populated and dirty

[Tomeu Vizoso]

The GEM DMA mmap helper maps the whole buffer up front, but with pages
that fault on their first write, since the A53 in the J722S has no
hardware dirty-bit management. The GEM shmem helper maps by page fault,
one at each page's first touch and another at its first write, which
its write-notify needs. Neither helper can install the pages writable
and dirty, so the driver sets the protection itself: for contiguous
buffers it calls dma_mmap_pages() with it, and for the others it
installs the pages with remap_pfn_range().

This saves 20% of the total time in the first invoke of a model on the
J722S.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 drivers/accel/thames/thames_gem.c | 76 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 73 insertions(+), 3 deletions(-)

diff --git a/drivers/accel/thames/thames_gem.c b/drivers/accel/thames/thames_gem.c
index 5a4dc497768f..9f55169c6370 100644
--- a/drivers/accel/thames/thames_gem.c
+++ b/drivers/accel/thames/thames_gem.c
@@ -808,12 +808,82 @@ static void thames_gem_shmem_bo_free(struct drm_gem_object *obj)
 	drm_gem_shmem_free(&bo->shmem);
 }
 
+/*
+ * Installed dirty and writable: the A53s have no hardware dirty-bit management,
+ * so every first write would fault.
+ */
+static int thames_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+	struct drm_gem_dma_object *dma_obj = to_drm_gem_dma_obj(obj);
+	pgprot_t prot;
+	int ret;
+
+	vma->vm_pgoff -= drm_vma_node_start(&obj->vma_node);
+	vm_flags_mod(vma, VM_DONTDUMP | VM_DONTEXPAND, VM_PFNMAP);
+
+	prot = vm_get_page_prot(vma->vm_flags);
+	if (vma->vm_flags & VM_WRITE)
+		prot = pte_pgprot(pte_mkdirty(pte_mkwrite_novma(pfn_pte(0, prot))));
+	vma->vm_page_prot = prot;
+
+	ret = dma_mmap_pages(drm_dev_dma_dev(obj->dev), vma, vma->vm_end - vma->vm_start,
+			     virt_to_page(dma_obj->vaddr));
+	if (ret)
+		drm_gem_vm_close(vma);
+
+	return ret;
+}
+
+/*
+ * Every page installed now, dirty and writable: they are pinned for the
+ * object's life, and the helper would take two faults on each.
+ */
+static int thames_gem_shmem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
+{
+	struct thames_gem_object *bo = to_thames_bo(obj);
+	unsigned long addr = vma->vm_start;
+	struct scatterlist *sg;
+	pgprot_t prot;
+	unsigned int i;
+	int ret;
+
+	if (vma_is_cow_mapping(vma))
+		return -EINVAL;
+
+	vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP);
+
+	prot = vm_get_page_prot(vma->vm_flags);
+	if (vma->vm_flags & VM_WRITE)
+		prot = pte_pgprot(pte_mkdirty(pte_mkwrite_novma(pfn_pte(0, prot))));
+	vma->vm_page_prot = prot;
+
+	for_each_sgtable_sg(bo->shmem.sgt, sg, i) {
+		unsigned long len = min_t(unsigned long, sg->length, vma->vm_end - addr);
+
+		ret = remap_pfn_range(vma, addr, page_to_pfn(sg_page(sg)), len, prot);
+		if (ret)
+			return ret;
+
+		addr += len;
+		if (addr == vma->vm_end)
+			break;
+	}
+
+	return 0;
+}
+
+/* Only the GEM reference: the pages are pinned for the object's life. */
+static const struct vm_operations_struct thames_gem_shmem_vm_ops = {
+	.open = drm_gem_vm_open,
+	.close = drm_gem_vm_close,
+};
+
 static const struct drm_gem_object_funcs thames_gem_funcs = {
 	.free = thames_gem_bo_free,
 	.print_info = drm_gem_dma_object_print_info,
 	.get_sg_table = drm_gem_dma_object_get_sg_table,
 	.vmap = drm_gem_dma_object_vmap,
-	.mmap = drm_gem_dma_object_mmap,
+	.mmap = thames_gem_mmap,
 	.vm_ops = &drm_gem_dma_vm_ops,
 };
 
@@ -825,8 +895,8 @@ static const struct drm_gem_object_funcs thames_gem_shmem_funcs = {
 	.get_sg_table = drm_gem_shmem_object_get_sg_table,
 	.vmap = drm_gem_shmem_object_vmap,
 	.vunmap = drm_gem_shmem_object_vunmap,
-	.mmap = drm_gem_shmem_object_mmap,
-	.vm_ops = &drm_gem_shmem_vm_ops,
+	.mmap = thames_gem_shmem_mmap,
+	.vm_ops = &thames_gem_shmem_vm_ops,
 };
 
 int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file *file)

-- 
2.54.0

[PATCH v3 13/16] Documentation: accel/thames: Add initial docs

[Tomeu Vizoso]

With pointers to the other components and how users can make use of it.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 Documentation/accel/index.rst        |  1 +
 Documentation/accel/thames/index.rst | 47 ++++++++++++++++++++++++++++++++++++
 MAINTAINERS                          |  1 +
 3 files changed, 49 insertions(+)

diff --git a/Documentation/accel/index.rst b/Documentation/accel/index.rst
index cbc7d4c3876a..eb115413db07 100644
--- a/Documentation/accel/index.rst
+++ b/Documentation/accel/index.rst
@@ -11,3 +11,4 @@ Compute Accelerators
    amdxdna/index
    qaic/index
    rocket/index
+   thames/index
diff --git a/Documentation/accel/thames/index.rst b/Documentation/accel/thames/index.rst
new file mode 100644
index 000000000000..6382ff2369be
--- /dev/null
+++ b/Documentation/accel/thames/index.rst
@@ -0,0 +1,47 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+============================================================
+ accel/thames Driver for the C7x DSPs from Texas Instruments
+============================================================
+
+The accel/thames driver supports the C7x DSP cores of the Texas Instruments
+J722S as compute accelerators, for workloads such as machine learning
+inference.
+
+Each core runs a firmware that remoteproc (Documentation/staging/remoteproc.rst)
+loads;
+the driver attaches to a core remoteproc has booted, boots one left offline, and
+resets a core through it after a fault or a timeout.  The firmware's source is
+at https://gitlab.freedesktop.org/tomeu/thames-firmware/.
+
+The driver and the firmware of a core share a block of memory reserved in the
+device tree.  It holds the core's Ring, the queue of submitted Jobs in the
+order the core runs them, and its Fence, the sequence number the firmware
+writes as each Job completes.  The driver adds an entry to the Ring and rings a
+mailbox doorbell; the firmware rings one back when the Fence moves.
+
+Each file that opens the device gets a Context: a page table the core switches
+to while it runs the Context's Jobs, so the buffers of one client are isolated
+from the others'.  A buffer is created without a mapping and bound into the
+Context at a DSP address the client chooses.
+
+A Job is created once from a program, its parameters and the transfers the
+core's DMA unit performs around it, and is checked then.  It can then be
+submitted any number of times, in arrays of Jobs that run in order and signal
+sync objects when they finish; the call waits on no sync object, the fences of
+the listed buffers order the work.  The drm-gpu-scheduler spreads the
+submissions over the cores.  A Job that faults leaves a fault record, which
+userspace reads with a query; the capability query returns the sizes and
+addresses userspace compiles against.
+
+Everything else is done in userspace, as a Gallium driver (also called thames)
+that is part of the Mesa3D project: https://docs.mesa3d.org/teflon.html
+
+The interface is described in ``include/uapi/drm/thames_accel.h``.
+
+.. kernel-doc:: include/uapi/drm/thames_accel.h
+   :doc: Terms
+
+Hardware currently supported:
+
+* J722S
diff --git a/MAINTAINERS b/MAINTAINERS
index c63ab2d77b01..dce817531184 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8024,6 +8024,7 @@ M:	Tomeu Vizoso <tomeu@tomeuvizoso.net>
 L:	dri-devel@lists.freedesktop.org
 S:	Supported
 T:	git https://gitlab.freedesktop.org/drm/misc/kernel.git
+F:	Documentation/accel/thames/
 F:	drivers/accel/thames/
 F:	include/uapi/drm/thames_accel.h
 

-- 
2.54.0

[PATCH v3 14/16] arm64: dts: ti: k3-j722s: Add the AUX L2 SRAM and DRU regions

[Tomeu Vizoso]

The DRM accel Thames driver maps each core's auxiliary L2 SRAM and the
DRU's channel registers into the Contexts it creates, so the DSP nodes
carry these two regions besides the L2 SRAM, and the bus gains ranges
for them. The auxiliary L2 is 256 KiB of SRAM in a 512 KiB aperture; a
read past it faults, so the reg entry covers the SRAM only.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 arch/arm64/boot/dts/ti/k3-j722s-main.dtsi | 16 ++++++++++++----
 arch/arm64/boot/dts/ti/k3-j722s.dtsi      |  4 ++++
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/boot/dts/ti/k3-j722s-main.dtsi b/arch/arm64/boot/dts/ti/k3-j722s-main.dtsi
index 5a5b9a1bfd9e..b6c951a1dd74 100644
--- a/arch/arm64/boot/dts/ti/k3-j722s-main.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j722s-main.dtsi
@@ -385,10 +385,16 @@ main_r5fss0_core0: r5f@78400000 {
 		};
 	};
 
+	/*
+	 * The auxiliary L2 is 256 KiB of SRAM in a 512 KiB aperture; a read
+	 * past it faults.
+	 */
 	c7x_0: dsp@7e000000 {
 		compatible = "ti,am62a-c7xv-dsp";
-		reg = <0x00 0x7e000000 0x00 0x00200000>;
-		reg-names = "l2sram";
+		reg = <0x00 0x7e000000 0x00 0x00200000>,
+		      <0x00 0x7f000000 0x00 0x00040000>,
+		      <0x00 0x7c460000 0x00 0x00020000>;
+		reg-names = "l2sram", "l2aux", "dru";
 		resets = <&k3_reset 208 1>;
 		firmware-name = "j722s-c71_0-fw";
 		ti,sci = <&dmsc>;
@@ -399,8 +405,10 @@ c7x_0: dsp@7e000000 {
 
 	c7x_1: dsp@7e200000 {
 		compatible = "ti,am62a-c7xv-dsp";
-		reg = <0x00 0x7e200000 0x00 0x00200000>;
-		reg-names = "l2sram";
+		reg = <0x00 0x7e200000 0x00 0x00200000>,
+		      <0x00 0x7f800000 0x00 0x00040000>,
+		      <0x00 0x7d460000 0x00 0x00020000>;
+		reg-names = "l2sram", "l2aux", "dru";
 		resets = <&k3_reset 268 1>;
 		firmware-name = "j722s-c71_1-fw";
 		ti,sci = <&dmsc>;
diff --git a/arch/arm64/boot/dts/ti/k3-j722s.dtsi b/arch/arm64/boot/dts/ti/k3-j722s.dtsi
index ca10c6904b69..6c0d2ee94290 100644
--- a/arch/arm64/boot/dts/ti/k3-j722s.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j722s.dtsi
@@ -185,6 +185,10 @@ cbass_main: bus@f0000 {
 			 <0x00 0x78500000 0x00 0x78500000 0x00 0x00008000>, /* MAIN R5FSS0 BTCM */
 			 <0x00 0x7e000000 0x00 0x7e000000 0x00 0x00200000>, /* C7X_0 L2SRAM */
 			 <0x00 0x7e200000 0x00 0x7e200000 0x00 0x00200000>, /* C7X_1 L2SRAM */
+			 <0x00 0x7f000000 0x00 0x7f000000 0x00 0x00080000>, /* C7X_0 L2 aux */
+			 <0x00 0x7f800000 0x00 0x7f800000 0x00 0x00080000>, /* C7X_1 L2 aux */
+			 <0x00 0x7c460000 0x00 0x7c460000 0x00 0x00020000>, /* C7X_0 DRU */
+			 <0x00 0x7d460000 0x00 0x7d460000 0x00 0x00020000>, /* C7X_1 DRU */
 			 <0x01 0x00000000 0x01 0x00000000 0x00 0x00310000>, /* A53 PERIPHBASE */
 			 <0x05 0x00000000 0x05 0x00000000 0x01 0x00000000>, /* FSS0 DAT3 */
 			 <0x06 0x00000000 0x06 0x00000000 0x01 0x00000000>, /* PCIe0 DAT1 */

-- 
2.54.0

[PATCH v3 15/16] arm64: dts: ti: k3-j722s: Add second submailbox to the C7x cores

[Tomeu Vizoso]

For firmware that uses something other than RPMsg for communication with
the Linux kernel.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi b/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
index a59c3648d805..611220ff2655 100644
--- a/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
@@ -86,6 +86,11 @@ mbox_c7x_0: mbox-c7x-0 {
 		ti,mbox-rx = <0 0 0>;
 		ti,mbox-tx = <1 0 0>;
 	};
+
+	mbox_c7x_0_client: mbox-c7x-0-client {
+		ti,mbox-rx = <3 0 0>;
+		ti,mbox-tx = <2 0 0>;
+	};
 };
 
 &mailbox0_cluster3 {
@@ -100,6 +105,11 @@ mbox_c7x_1: mbox-c7x-1 {
 		ti,mbox-rx = <2 0 0>;
 		ti,mbox-tx = <3 0 0>;
 	};
+
+	mbox_c7x_1_client: mbox-c7x-1-client {
+		ti,mbox-rx = <5 0 0>;
+		ti,mbox-tx = <4 0 0>;
+	};
 };
 
 /* Timers are used by Remoteproc firmware */
@@ -152,14 +162,16 @@ &main_r5fss0_core0 {
 };
 
 &c7x_0 {
-	mboxes = <&mailbox0_cluster2 &mbox_c7x_0>;
+	mboxes = <&mailbox0_cluster2 &mbox_c7x_0>,
+		 <&mailbox0_cluster2 &mbox_c7x_0_client>;
 	memory-region = <&c7x_0_dma_memory_region>,
 			<&c7x_0_memory_region>;
 	status = "okay";
 };
 
 &c7x_1 {
-	mboxes = <&mailbox0_cluster3 &mbox_c7x_1>;
+	mboxes = <&mailbox0_cluster3 &mbox_c7x_1>,
+		 <&mailbox0_cluster3 &mbox_c7x_1_client>;
 	memory-region = <&c7x_1_dma_memory_region>,
 			<&c7x_1_memory_region>;
 	status = "okay";

-- 
2.54.0

[PATCH v3 16/16] arm64: dts: ti: k3-j722s: Reference the power domains of the C7x timers

[Tomeu Vizoso]

Make sure that the timers are powered up so the firmware can use them.

Assisted-by: LLM
Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
---
 arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi b/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
index 611220ff2655..7639081dc913 100644
--- a/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
+++ b/arch/arm64/boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi
@@ -166,6 +166,14 @@ &c7x_0 {
 		 <&mailbox0_cluster2 &mbox_c7x_0_client>;
 	memory-region = <&c7x_0_dma_memory_region>,
 			<&c7x_0_memory_region>;
+	/*
+	 * Both timers on each core: core 0's firmware drives TIMER1 and core
+	 * 1's TIMER2; two entries also keep the platform bus from attaching the
+	 * domain to the K3 driver's device.
+	 */
+	power-domains = <&k3_pds 37 TI_SCI_PD_EXCLUSIVE>,
+			<&k3_pds 38 TI_SCI_PD_EXCLUSIVE>;
+	power-domain-names = "timer1", "timer2";
 	status = "okay";
 };
 
@@ -174,5 +182,8 @@ &c7x_1 {
 		 <&mailbox0_cluster3 &mbox_c7x_1_client>;
 	memory-region = <&c7x_1_dma_memory_region>,
 			<&c7x_1_memory_region>;
+	power-domains = <&k3_pds 37 TI_SCI_PD_EXCLUSIVE>,
+			<&k3_pds 38 TI_SCI_PD_EXCLUSIVE>;
+	power-domain-names = "timer1", "timer2";
 	status = "okay";
 };

-- 
2.54.0

Re: [PATCH v3 02/16] dt-bindings: remoteproc: k3-dsp: Allow a second sub-mailbox

[Krzysztof Kozlowski]

On Fri, Oct 23, 2026 at 08:39:27AM +0200, Tomeu Vizoso wrote:
> Allow specifying a second sub-mailbox that could be used by the firmware
> in addition to the first one, which is taken by remoteproc.
> 
> This is required by firmware that doesn't use rpmsg for their
> communication with the Linux kernel.
> 
> Assisted-by: LLM

Drop

> Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>

DCO mismatch.

> 
> ---
> 
> v3:
> - Initial version
> ---
>  .../devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml        | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> index 278b88b7f906..d6319c8e96f8 100644
> --- a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> +++ b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> @@ -51,10 +51,12 @@ properties:
>  
>    mboxes:
>      description: |
> -      OMAP Mailbox specifier denoting the sub-mailbox, to be used for
> -      communication with the remote processor. This property should match
> -      with the sub-mailbox node used in the firmware image.
> -    maxItems: 1
> +      OMAP Mailbox specifiers denoting the sub-mailboxes allotted to
> +      communication with the remote processor. The first is the one the
> +      remoteproc core uses and should match the sub-mailbox node used in the
> +      firmware image. The second is optional.

Instead list the items with description and minItems: 1. Don't repeat
constraints in free form text.

> +    minItems: 1

Best regards,
Krzysztof

Re: [PATCH v3 03/16] dt-bindings: remoteproc: k3-dsp: Allow referencing the timers' power domains

[Krzysztof Kozlowski]

On Fri, Oct 23, 2026 at 08:39:28AM +0200, Tomeu Vizoso wrote:
> A firmware may drive the core's timers, which need their power domains
> held; the remoteproc driver does not hold them, so let the DSP node
> reference them for a driver that attaches to the running core to hold.
> 
> Assisted-by: LLM
> Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
> 
> ---
> 
> v3:
> - Initial version
> ---
>  .../devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> index d6319c8e96f8..80eaef0a19b8 100644
> --- a/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> +++ b/Documentation/devicetree/bindings/remoteproc/ti,k3-dsp-rproc.yaml
> @@ -76,6 +76,23 @@ properties:
>  # Optional properties:
>  # --------------------
>  
> +  power-domains:
> +    description: |
> +      Power domains of the SoC timers the running firmware ticks on and
> +      times its work out with. The firmware programs these timers itself
> +      and makes no power request for them, and the timers' own nodes are
> +      reserved for it, so nothing else requests their power; a client
> +      driver of the firmware holds them on while it drives the processor.
> +      The processor's own power is managed through ti,sci-dev-id, not here.
> +    items:
> +      - description: power domain of the first timer

first timer

> +      - description: power domain of the second timer

second timer

No need to say that power domain is a power domain.

Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>

Best regards,
Krzysztof

Re: [PATCH v3 07/16] accel/thames: Add IOCTLs for BO creation and mapping

[Christian König]

On 10/23/26 08:39, Tomeu Vizoso wrote:
> Creation returns a handle and maps nothing. The client maps a buffer
> with DRM_IOCTL_THAMES_BO_BIND at a DSP address of its choice, as a
> Vulkan driver would; the kernel enters the mapping in the page table
> of every core, because we cannot know upfront which core will run a
> given job.
> 
> We use the SHMEM helpers for non-contiguous memory. Contiguous memory
> comes from the driver's own allocator, which aligns a buffer of 2 MiB or
> more to 2 MiB so it can be mapped as blocks; only the GEM DMA object
> and its mmap and vmap helpers are used for it.

How exactly do you do that? IIRC the SHMEM helpers can only do 2MiB allocations on a best effort basis.

In other words you don't get a guarantee that this always works and when memory is fragmented the 4k fallback must work as well.

Regards,
Christian.

> 
> Contiguous memory is needed for buffers that are going to be read from
> or written into by the DRU, as it doesn't resolve addresses via the MMU.
> 
> Binding at an address the client chooses lets a program be compiled,
> with its addresses fixed, on a machine without the hardware, and is what
> a Vulkan driver needs.
> 
> A buffer is bound anywhere from 4 GiB up to the user region at 1 TiB; a
> buffer shared through a dma-buf may be bound in several Contexts, so
> the bindings are kept under one device-wide lock. The IOCTLs take their
> arrays as versioned objects whose stride is the size userspace was
> built against, so a structure can grow under a minor bump.
> 
> Assisted-by: LLM
> Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
> 
> ---
> 
> v3:
> - Added read-only, executable and contiguous flags.
> - Added BO_BIND IOCTL.
> - Copy the IOCTLs' arrays with vmemdup_array_user(), through one helper
>   for versioned arrays (Tvrtko Ursulin).
> 
> v2:
> - Add thames_accel.h UAPI header (Robert Nelson).
> ---
>  drivers/accel/thames/Kconfig         |   2 +
>  drivers/accel/thames/thames_device.h |   6 +
>  drivers/accel/thames/thames_drv.c    |  47 +++++
>  drivers/accel/thames/thames_drv.h    |   5 +
>  drivers/accel/thames/thames_gem.c    | 398 +++++++++++++++++++++++++++++++++++
>  drivers/accel/thames/thames_gem.h    |  21 ++
>  include/uapi/drm/thames_accel.h      | 183 +++++++++++++++-
>  7 files changed, 661 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
> index 9362b9274075..47a2986fcabd 100644
> --- a/drivers/accel/thames/Kconfig
> +++ b/drivers/accel/thames/Kconfig
> @@ -6,6 +6,8 @@ config DRM_ACCEL_THAMES
>         depends on TI_K3_DSP_REMOTEPROC
>         depends on REMOTEPROC
>         depends on MAILBOX
> +       select DRM_GEM_DMA_HELPER
> +       select DRM_GEM_SHMEM_HELPER
>         select IOMMU_IO_PGTABLE_LPAE
>         help
>           Choose this option if you have a Texas Instruments J722S and want to
> diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
> index e89e0dfd89b8..a1a54d974359 100644
> --- a/drivers/accel/thames/thames_device.h
> +++ b/drivers/accel/thames/thames_device.h
> @@ -58,6 +58,12 @@ struct thames_device {
>         struct list_head contexts;
>         struct mutex contexts_lock;
> 
> +       /*
> +        * One lock for the device: a dma-buf shared buffer is bound from
> +        * several files.
> +        */
> +       struct mutex bindings_lock;
> +
>  };
> 
>  #define to_thames_device(drm_dev) \
> diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
> index cb0287430d25..a17e7a90f7c1 100644
> --- a/drivers/accel/thames/thames_drv.c
> +++ b/drivers/accel/thames/thames_drv.c
> @@ -93,6 +93,41 @@ static void thames_postclose(struct drm_device *dev, struct drm_file *file)
>         thames_file_priv_put(thames_priv);
>  }
> 
> +void *thames_get_uobj_array(const struct drm_thames_obj_array *in, u32 min_stride, u32 obj_size)
> +{
> +       void __user *in_ptr = u64_to_user_ptr(in->array);
> +       void *out, *out_ptr;
> +       int ret;
> +       u32 i;
> +
> +       if (!in->count)
> +               return NULL;
> +
> +       if (in->stride < min_stride)
> +               return ERR_PTR(-EINVAL);
> +
> +       if (in->stride == obj_size)
> +               return vmemdup_array_user(in_ptr, in->count, obj_size);
> +
> +       out = kvmalloc_array(in->count, obj_size, GFP_KERNEL);
> +       if (!out)
> +               return ERR_PTR(-ENOMEM);
> +
> +       out_ptr = out;
> +       for (i = 0; i < in->count; i++) {
> +               ret = copy_struct_from_user(out_ptr, obj_size, in_ptr, in->stride);
> +               if (ret) {
> +                       kvfree(out);
> +                       return ERR_PTR(ret);
> +               }
> +
> +               out_ptr += obj_size;
> +               in_ptr += in->stride;
> +       }
> +
> +       return out;
> +}
> +
>  #define THAMES_DRIVER_MAJOR 1
>  #define THAMES_DRIVER_MINOR 0
> 
> @@ -121,6 +156,8 @@ static int thames_query_caps(struct drm_device *dev, struct drm_thames_dev_query
>                 .l2_aux.dsp_va = thames_window_va(tdev, THAMES_WINDOW_L2AUX),
>                 .l2_aux.size = reported->window_sizes[THAMES_WINDOW_L2AUX],
>                 .dru_realtime_dsp_va = thames_window_va(tdev, THAMES_WINDOW_DRU),
> +               .dsp_va_range.dsp_va = THAMES_BIND_VA_START,
> +               .dsp_va_range.size = THAMES_BIND_VA_END - THAMES_BIND_VA_START,
>                 .user_region.dsp_va = THAMES_USER_REGION_BASE,
>                 .user_region.size = THAMES_USER_REGION_SIZE,
>                 .hardware_id = reported->hardware_id,
> @@ -165,11 +202,17 @@ static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,      \
>         return ret;                                                             \
>  }
> 
> +THAMES_IOCTL_ENTERED(bo_create)
> +THAMES_IOCTL_ENTERED(bo_mmap_offset)
>  THAMES_IOCTL_ENTERED(dev_query)
> +THAMES_IOCTL_ENTERED(bo_bind)
> 
>  static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
>  #define THAMES_IOCTL(n, func) DRM_IOCTL_DEF_DRV(THAMES_##n, thames_ioctl_##func, 0)
> +       THAMES_IOCTL(BO_CREATE, bo_create_entered),
> +       THAMES_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset_entered),
>         THAMES_IOCTL(DEV_QUERY, dev_query_entered),
> +       THAMES_IOCTL(BO_BIND, bo_bind_entered),
>  };
> 
>  DEFINE_DRM_ACCEL_FOPS(thames_accel_driver_fops);
> @@ -234,6 +277,10 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
>         if (err)
>                 return ERR_PTR(err);
> 
> +       err = drmm_mutex_init(&tdev->ddev, &tdev->bindings_lock);
> +       if (err)
> +               return ERR_PTR(err);
> +
>         INIT_LIST_HEAD(&tdev->contexts);
> 
>         tdev->asid_next = 1;
> diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
> index f4536078e02f..7f9031b63780 100644
> --- a/drivers/accel/thames/thames_drv.h
> +++ b/drivers/accel/thames/thames_drv.h
> @@ -15,6 +15,7 @@
>  #include "thames_device.h"
> 
>  struct drm_gem_dma_object;
> +struct drm_thames_obj_array;
> 
>  struct thames_file_priv {
>         struct thames_device *tdev;
> @@ -37,6 +38,8 @@ struct thames_file_priv {
>         struct drm_mm_node user_region;
>         struct drm_mm_node window_nodes[THAMES_NUM_WINDOWS];
> 
> +       struct list_head bindings;
> +
>         u32 context_id;
> 
>         u64 context_serial;
> @@ -47,4 +50,6 @@ struct thames_file_priv {
> 
>  void thames_file_priv_put(struct thames_file_priv *priv);
> 
> +void *thames_get_uobj_array(const struct drm_thames_obj_array *in, u32 min_stride, u32 obj_size);
> +
>  #endif
> diff --git a/drivers/accel/thames/thames_gem.c b/drivers/accel/thames/thames_gem.c
> index b98e0fe203b0..c9fd08c7aaf4 100644
> --- a/drivers/accel/thames/thames_gem.c
> +++ b/drivers/accel/thames/thames_gem.c
> @@ -14,6 +14,7 @@
>  #include <drm/drm_device.h>
>  #include <drm/drm_drv.h>
>  #include <drm/drm_gem_dma_helper.h>
> +#include <drm/drm_gem_shmem_helper.h>
>  #include <drm/drm_print.h>
>  #include <drm/drm_timeout.h>
>  #include <drm/thames_accel.h>
> @@ -23,6 +24,30 @@
>  #include "thames_drv.h"
>  #include "thames_gem.h"
> 
> +/*
> + * References neither the buffer nor the Context: whichever goes first removes
> + * it.
> + */
> +struct thames_binding {
> +       struct thames_gem_object *bo;
> +       struct thames_file_priv *priv;
> +       struct drm_mm_node mm;
> +       struct list_head bo_link;
> +       struct list_head context_link;
> +};
> +
> +static int thames_bo_flags_to_prot(u32 flags)
> +{
> +       int prot = IOMMU_READ | IOMMU_CACHE;
> +
> +       if (!(flags & DRM_THAMES_BO_EXECUTABLE))
> +               prot |= IOMMU_NOEXEC;
> +       if (!(flags & (DRM_THAMES_BO_READ_ONLY | DRM_THAMES_BO_EXECUTABLE)))
> +               prot |= IOMMU_WRITE;
> +
> +       return prot;
> +}
> +
>  static size_t thames_pgsize(u64 addr, size_t size, size_t *count)
>  {
>         size_t blk_offset = -addr % SZ_2M;
> @@ -95,6 +120,72 @@ static int thames_context_map(struct thames_file_priv *priv, u64 dsp_va, int pro
>         return 0;
>  }
> 
> +static void thames_context_unmap(struct thames_file_priv *priv, u64 dsp_va, u64 size)
> +{
> +       unsigned int c;
> +
> +       for (c = 0; c < priv->tdev->num_cores; c++)
> +               thames_pgtable_unmap(priv->pgtbl_ops[c], dsp_va, size);
> +}
> +
> +struct thames_gem_object *thames_context_find_bo(struct thames_file_priv *priv, u64 start,
> +                                                u64 end, u64 *offset)
> +{
> +       struct thames_binding *binding;
> +       struct thames_gem_object *bo;
> +       struct drm_mm_node *node;
> +       unsigned int i;
> +
> +       guard(mutex)(&priv->mm_lock);
> +
> +       node = __drm_mm_interval_first(&priv->mm, start, start);
> +       if (node == &priv->mm.head_node || node == &priv->first_page ||
> +           node == &priv->user_region)
> +               return NULL;
> +
> +       for (i = 0; i < THAMES_NUM_WINDOWS; i++) {
> +               if (node == &priv->window_nodes[i])
> +                       return NULL;
> +       }
> +
> +       binding = container_of(node, struct thames_binding, mm);
> +       bo = binding->bo;
> +       if (end > node->start + bo->base.base.size)
> +               return NULL;
> +
> +       if (!kref_get_unless_zero(&bo->base.base.refcount))
> +               return NULL;
> +
> +       *offset = start - node->start;
> +
> +       return bo;
> +}
> +
> +static struct thames_binding *thames_bo_find_binding(struct thames_gem_object *bo,
> +                                                    struct thames_file_priv *priv)
> +{
> +       struct thames_binding *binding;
> +
> +       lockdep_assert_held(&priv->tdev->bindings_lock);
> +
> +       list_for_each_entry(binding, &bo->bindings, bo_link) {
> +               if (binding->priv == priv)
> +                       return binding;
> +       }
> +
> +       return NULL;
> +}
> +
> +u64 thames_bo_dsp_va(struct thames_gem_object *bo, struct thames_file_priv *priv)
> +{
> +       struct thames_binding *binding;
> +
> +       guard(mutex)(&priv->tdev->bindings_lock);
> +       binding = thames_bo_find_binding(bo, priv);
> +
> +       return binding ? binding->mm.start : 0;
> +}
> +
>  /* Cyclic, so a freed ASID is not handed out again at once. */
>  static int thames_asid_alloc(struct thames_device *tdev)
>  {
> @@ -215,6 +306,7 @@ static int thames_context_map_core_regions(struct thames_file_priv *priv)
>  }
> 
>  static const struct drm_gem_object_funcs thames_gem_funcs;
> +static const struct drm_gem_object_funcs thames_gem_shmem_funcs;
> 
>  /*
>   * The allocator zeroes through the cache: clean it before the DSP can reach the
> @@ -234,6 +326,7 @@ static struct drm_gem_dma_object *thames_gem_create(struct drm_device *ddev, siz
>         if (!bo)
>                 return ERR_PTR(-ENOMEM);
> 
> +       INIT_LIST_HEAD(&bo->bindings);
>         obj = &bo->base.base;
>         obj->funcs = &thames_gem_funcs;
>         bo->base.map_noncoherent = true;
> @@ -271,6 +364,38 @@ static struct drm_gem_dma_object *thames_gem_create(struct drm_device *ddev, siz
>         return ERR_PTR(ret);
>  }
> 
> +/*
> + * The DSP reaches these pages only through the page tables, never by a DRU
> + * transfer.
> + */
> +static struct thames_gem_object *thames_gem_create_shmem(struct drm_device *ddev, size_t size)
> +{
> +       struct thames_gem_object *bo;
> +       struct sg_table *sgt;
> +       int ret;
> +
> +       bo = kzalloc_obj(*bo, GFP_KERNEL);
> +       if (!bo)
> +               return ERR_PTR(-ENOMEM);
> +
> +       INIT_LIST_HEAD(&bo->bindings);
> +       bo->shmem.base.funcs = &thames_gem_shmem_funcs;
> +
> +       ret = drm_gem_shmem_init(ddev, &bo->shmem, PAGE_ALIGN(size));
> +       if (ret) {
> +               kfree(bo);
> +               return ERR_PTR(ret);
> +       }
> +
> +       sgt = drm_gem_shmem_get_pages_sgt(&bo->shmem);
> +       if (IS_ERR(sgt)) {
> +               drm_gem_object_put(&bo->shmem.base);
> +               return ERR_CAST(sgt);
> +       }
> +
> +       return bo;
> +}
> +
>  static struct drm_gem_dma_object *thames_context_alloc_user_pages(struct thames_file_priv *priv,
>                                                                   u64 dsp_va, size_t size)
>  {
> @@ -459,6 +584,7 @@ int thames_context_create(struct thames_file_priv *priv)
> 
>         priv->context_id = ret;
>         priv->context_serial = atomic64_inc_return(&tdev->last_context_serial);
> +       INIT_LIST_HEAD(&priv->bindings);
> 
>         ret = thames_context_va_init(priv);
>         if (ret)
> @@ -500,14 +626,37 @@ int thames_context_create(struct thames_file_priv *priv)
>         return ret;
>  }
> 
> +static void thames_binding_remove(struct thames_binding *binding)
> +{
> +       struct thames_file_priv *priv = binding->priv;
> +
> +       lockdep_assert_held(&priv->tdev->bindings_lock);
> +
> +       thames_context_unmap(priv, binding->mm.start, binding->mm.size);
> +       thames_context_bump_generation(priv);
> +
> +       scoped_guard(mutex, &priv->mm_lock)
> +               drm_mm_remove_node(&binding->mm);
> +
> +       list_del(&binding->bo_link);
> +       list_del(&binding->context_link);
> +       kfree(binding);
> +}
> +
>  void thames_context_destroy(struct thames_file_priv *priv)
>  {
>         struct thames_device *tdev = priv->tdev;
> +       struct thames_binding *binding, *tmp;
> 
>         /*
>          * The Context goes after its last pushed Job completed, so no Job runs
>          * on mappings removed here.
>          */
> +       scoped_guard(mutex, &tdev->bindings_lock) {
> +               list_for_each_entry_safe(binding, tmp, &priv->bindings, context_link)
> +                       thames_binding_remove(binding);
> +       }
> +
>         scoped_guard(mutex, &tdev->contexts_lock)
>                 list_del(&priv->context_node);
> 
> @@ -522,10 +671,124 @@ void thames_context_destroy(struct thames_file_priv *priv)
>                 priv->context_id, priv->asid);
>  }
> 
> +static int thames_context_map_sgt(struct thames_file_priv *priv, u64 dsp_va, int prot,
> +                                 struct sg_table *sgt)
> +{
> +       struct scatterlist *sg;
> +       u64 offset = 0;
> +       unsigned int i;
> +       int ret;
> +
> +       for_each_sgtable_dma_sg(sgt, sg, i) {
> +               ret = thames_context_map(priv, dsp_va + offset, prot, sg_dma_address(sg),
> +                                        sg_dma_len(sg));
> +               if (ret) {
> +                       thames_context_unmap(priv, dsp_va, offset);
> +                       return ret;
> +               }
> +
> +               offset += sg_dma_len(sg);
> +       }
> +
> +       return 0;
> +}
> +
> +static int thames_bo_bind(struct thames_file_priv *priv, struct thames_gem_object *bo,
> +                         u64 dsp_va)
> +{
> +       size_t size = bo->base.base.size;
> +       int prot = thames_bo_flags_to_prot(bo->flags);
> +       struct thames_binding *binding;
> +       int ret;
> +
> +       if (!IS_ALIGNED(dsp_va, PAGE_SIZE))
> +               return -EINVAL;
> +
> +       if (dsp_va < THAMES_BIND_VA_START || dsp_va > THAMES_BIND_VA_END ||
> +           size > THAMES_BIND_VA_END - dsp_va)
> +               return -EINVAL;
> +
> +       binding = kzalloc_obj(*binding);
> +       if (!binding)
> +               return -ENOMEM;
> +
> +       binding->bo = bo;
> +       binding->priv = priv;
> +       binding->mm.start = dsp_va;
> +       binding->mm.size = size;
> +
> +       ret = mutex_lock_interruptible(&priv->tdev->bindings_lock);
> +       if (ret)
> +               goto err_free;
> +
> +       if (thames_bo_find_binding(bo, priv)) {
> +               ret = -EBUSY;
> +               goto err_unlock;
> +       }
> +
> +       scoped_guard(mutex, &priv->mm_lock)
> +               ret = drm_mm_reserve_node(&priv->mm, &binding->mm);
> +       if (ret)
> +               goto err_unlock;
> +
> +       if (bo->flags & DRM_THAMES_BO_CONTIGUOUS)
> +               ret = thames_context_map(priv, dsp_va, prot, bo->base.dma_addr, size);
> +       else
> +               ret = thames_context_map_sgt(priv, dsp_va, prot, bo->shmem.sgt);
> +       if (ret) {
> +               scoped_guard(mutex, &priv->mm_lock)
> +                       drm_mm_remove_node(&binding->mm);
> +               goto err_unlock;
> +       }
> +
> +       list_add_tail(&binding->bo_link, &bo->bindings);
> +       list_add_tail(&binding->context_link, &priv->bindings);
> +       mutex_unlock(&priv->tdev->bindings_lock);
> +
> +       return 0;
> +
> +err_unlock:
> +       mutex_unlock(&priv->tdev->bindings_lock);
> +err_free:
> +       kfree(binding);
> +       return ret;
> +}
> +
> +static int thames_bo_unbind(struct thames_file_priv *priv, struct thames_gem_object *bo)
> +{
> +       struct thames_binding *binding;
> +       int ret;
> +
> +       ret = mutex_lock_interruptible(&priv->tdev->bindings_lock);
> +       if (ret)
> +               return ret;
> +
> +       binding = thames_bo_find_binding(bo, priv);
> +       if (binding)
> +               thames_binding_remove(binding);
> +       else
> +               ret = -EINVAL;
> +
> +       mutex_unlock(&priv->tdev->bindings_lock);
> +       return ret;
> +}
> +
> +static void thames_bo_unmap_from_device(struct thames_gem_object *bo)
> +{
> +       struct thames_device *tdev = to_thames_device(bo->base.base.dev);
> +       struct thames_binding *binding, *tmp;
> +
> +       guard(mutex)(&tdev->bindings_lock);
> +       list_for_each_entry_safe(binding, tmp, &bo->bindings, bo_link)
> +               thames_binding_remove(binding);
> +}
> +
>  static void thames_gem_bo_free(struct drm_gem_object *obj)
>  {
>         struct thames_gem_object *bo = to_thames_bo(obj);
> 
> +       thames_bo_unmap_from_device(bo);
> +
>         if (bo->alloc_vaddr)
>                 dma_free_noncoherent(drm_dev_dma_dev(obj->dev), bo->alloc_size, bo->alloc_vaddr,
>                                      bo->alloc_dma, DMA_TO_DEVICE);
> @@ -535,6 +798,16 @@ static void thames_gem_bo_free(struct drm_gem_object *obj)
>         kfree(bo);
>  }
> 
> +static void thames_gem_shmem_bo_free(struct drm_gem_object *obj)
> +{
> +       struct thames_gem_object *bo = to_thames_bo(obj);
> +
> +       thames_bo_unmap_from_device(bo);
> +
> +       drm_gem_free_mmap_offset(obj);
> +       drm_gem_shmem_free(&bo->shmem);
> +}
> +
>  static const struct drm_gem_object_funcs thames_gem_funcs = {
>         .free = thames_gem_bo_free,
>         .print_info = drm_gem_dma_object_print_info,
> @@ -544,3 +817,128 @@ static const struct drm_gem_object_funcs thames_gem_funcs = {
>         .vm_ops = &drm_gem_dma_vm_ops,
>  };
> 
> +static const struct drm_gem_object_funcs thames_gem_shmem_funcs = {
> +       .free = thames_gem_shmem_bo_free,
> +       .print_info = drm_gem_shmem_object_print_info,
> +       .pin = drm_gem_shmem_object_pin,
> +       .unpin = drm_gem_shmem_object_unpin,
> +       .get_sg_table = drm_gem_shmem_object_get_sg_table,
> +       .vmap = drm_gem_shmem_object_vmap,
> +       .vunmap = drm_gem_shmem_object_vunmap,
> +       .mmap = drm_gem_shmem_object_mmap,
> +       .vm_ops = &drm_gem_shmem_vm_ops,
> +};
> +
> +int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_bo_create *args = data;
> +       struct thames_gem_object *bo;
> +       struct drm_gem_dma_object *mem;
> +       int ret;
> +
> +       if (args->handle)
> +               return -EINVAL;
> +
> +       if (!args->size)
> +               return -EINVAL;
> +
> +       if (args->flags & ~(DRM_THAMES_BO_READ_ONLY | DRM_THAMES_BO_EXECUTABLE |
> +                           DRM_THAMES_BO_CONTIGUOUS))
> +               return -EINVAL;
> +
> +       if (args->flags & DRM_THAMES_BO_CONTIGUOUS) {
> +               mem = thames_gem_create(ddev, args->size);
> +               if (IS_ERR(mem))
> +                       return PTR_ERR(mem);
> +               bo = to_thames_bo(&mem->base);
> +       } else {
> +               bo = thames_gem_create_shmem(ddev, args->size);
> +               if (IS_ERR(bo))
> +                       return PTR_ERR(bo);
> +       }
> +
> +       bo->flags = args->flags;
> +
> +       ret = drm_gem_handle_create(file, &bo->base.base, &args->handle);
> +       drm_gem_object_put(&bo->base.base);
> +       if (ret) {
> +               dev_err(ddev->dev, "Failed to create gem handle: %d", ret);
> +               return ret;
> +       }
> +
> +       args->size = bo->base.base.size;
> +
> +       return 0;
> +}
> +
> +static int thames_bind_exec_op(struct drm_file *file, const struct drm_thames_bind_op *op)
> +{
> +       struct thames_file_priv *priv = file->driver_priv;
> +       struct drm_gem_object *obj;
> +       int ret;
> +
> +       if (op->flags & ~DRM_THAMES_BIND_OP_UNBIND)
> +               return -EINVAL;
> +
> +       if ((op->flags & DRM_THAMES_BIND_OP_UNBIND) && op->dsp_va)
> +               return -EINVAL;
> +
> +       obj = drm_gem_object_lookup(file, op->handle);
> +       if (!obj)
> +               return -ENOENT;
> +
> +       if (op->flags & DRM_THAMES_BIND_OP_UNBIND)
> +               ret = thames_bo_unbind(priv, to_thames_bo(obj));
> +       else
> +               ret = thames_bo_bind(priv, to_thames_bo(obj), op->dsp_va);
> +
> +       drm_gem_object_put(obj);
> +
> +       return ret;
> +}
> +
> +int thames_ioctl_bo_bind(struct drm_device *ddev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_bo_bind *args = data;
> +       struct drm_thames_bind_op *ops;
> +       int ret = 0;
> +       u32 i;
> +
> +       if (args->flags || args->pad)
> +               return -EINVAL;
> +
> +       ops = thames_get_uobj_array(&args->ops, sizeof(*ops), sizeof(*ops));
> +       if (IS_ERR(ops))
> +               return PTR_ERR(ops);
> +
> +       for (i = 0; i < args->ops.count; i++) {
> +               ret = thames_bind_exec_op(file, &ops[i]);
> +               if (ret) {
> +                       args->ops.count = i;
> +                       break;
> +               }
> +       }
> +
> +       kvfree(ops);
> +
> +       return ret;
> +}
> +
> +int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_bo_mmap_offset *args = data;
> +       struct drm_gem_object *obj;
> +
> +       if (args->pad)
> +               return -EINVAL;
> +
> +       obj = drm_gem_object_lookup(file, args->handle);
> +       if (!obj)
> +               return -ENOENT;
> +
> +       args->offset = drm_vma_node_offset_addr(&obj->vma_node);
> +       drm_gem_object_put(obj);
> +
> +       return 0;
> +}
> +
> diff --git a/drivers/accel/thames/thames_gem.h b/drivers/accel/thames/thames_gem.h
> index 09ac1b50d21a..c9d0e170dad3 100644
> --- a/drivers/accel/thames/thames_gem.h
> +++ b/drivers/accel/thames/thames_gem.h
> @@ -12,6 +12,13 @@
> 
>  #include "thames_abi.h"
> 
> +/*
> + * The windows sit below 4 GiB because Kernels load their addresses with 32-bit
> + * moves.
> + */
> +#define THAMES_BIND_VA_START   SZ_4G
> +#define THAMES_BIND_VA_END     THAMES_USER_REGION_BASE
> +
>  struct thames_core;
>  struct thames_device;
>  struct thames_file_priv;
> @@ -34,9 +41,18 @@ struct thames_gem_object {
>         dma_addr_t alloc_dma;
>         size_t alloc_size;
> 
> +       struct list_head bindings;
> +
> +       u32 flags;
>  };
> 
> 
> +int thames_ioctl_bo_create(struct drm_device *ddev, void *data, struct drm_file *file);
> +
> +int thames_ioctl_bo_mmap_offset(struct drm_device *ddev, void *data, struct drm_file *file);
> +
> +int thames_ioctl_bo_bind(struct drm_device *ddev, void *data, struct drm_file *file);
> +
>  int thames_context_create(struct thames_file_priv *priv);
> 
>  void thames_context_destroy(struct thames_file_priv *priv);
> @@ -44,6 +60,11 @@ void thames_context_rewrite_entries(struct thames_core *core);
> 
>  void thames_context_va_fini(struct thames_file_priv *priv);
> 
> +struct thames_gem_object *thames_context_find_bo(struct thames_file_priv *priv, u64 start,
> +                                                u64 end, u64 *offset);
> +
> +u64 thames_bo_dsp_va(struct thames_gem_object *bo, struct thames_file_priv *priv);
> +
>  static inline struct thames_gem_object *to_thames_bo(struct drm_gem_object *obj)
>  {
>         return container_of(obj, struct thames_gem_object, base.base);
> diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
> index af427a29b585..ea556b48f1d7 100644
> --- a/include/uapi/drm/thames_accel.h
> +++ b/include/uapi/drm/thames_accel.h
> @@ -45,15 +45,116 @@ extern "C" {
>   * definitions instead.
>   */
>  enum drm_thames_ioctl_id {
> +       /** @DRM_THAMES_BO_CREATE: Create a buffer object. */
> +       DRM_THAMES_BO_CREATE,
> +
> +       /**
> +        * @DRM_THAMES_BO_MMAP_OFFSET: Get the file offset to pass to
> +        * mmap to map a GEM object.
> +        */
> +       DRM_THAMES_BO_MMAP_OFFSET,
> +
>         /** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
>         DRM_THAMES_DEV_QUERY,
> 
> +       /**
> +        * @DRM_THAMES_BO_BIND: Bind buffers into the file's Context at the
> +        * addresses the client chooses, or unbind them.
> +        */
> +       DRM_THAMES_BO_BIND,
>  };
> 
>  /**
>   * DOC: IOCTL arguments
>   */
> 
> +/**
> + * enum drm_thames_bo_flags - Buffer object flags, passed at creation time.
> + *
> + * A buffer with neither permission flag is user read-write and execute-never;
> + * DRM_THAMES_BO_CONTIGUOUS chooses the backing, not the permissions.
> + */
> +enum drm_thames_bo_flags {
> +       /**
> +        * @DRM_THAMES_BO_READ_ONLY: Mapped user read-only in the Context.  For
> +        * the params, the coefficients and the constants, so that a Kernel
> +        * cannot alter the inputs of a Job's next run.
> +        */
> +       DRM_THAMES_BO_READ_ONLY = 1 << 0,
> +
> +       /**
> +        * @DRM_THAMES_BO_EXECUTABLE: Mapped user read-execute and not
> +        * writable.  Required of a Job's program.
> +        */
> +       DRM_THAMES_BO_EXECUTABLE = 1 << 1,
> +
> +       /**
> +        * @DRM_THAMES_BO_CONTIGUOUS: Backed by one physically contiguous
> +        * allocation, so that a transfer of a Job can name it.  For the buffers
> +        * a DRU transfer reads or writes; any other buffer is backed page by
> +        * page.
> +        */
> +       DRM_THAMES_BO_CONTIGUOUS = 1 << 2,
> +};
> +
> +/**
> + * struct drm_thames_bo_create - Arguments passed to DRM_IOCTL_THAMES_BO_CREATE.
> + *
> + * Returns a handle and no address; the buffer is bound in no Context until
> + * DRM_IOCTL_THAMES_BO_BIND.
> + */
> +struct drm_thames_bo_create {
> +       /**
> +        * @size: Requested size for the object
> +        *
> +        * The (page-aligned) allocated size for the object will be returned.
> +        */
> +       __u64 size;
> +
> +       /**
> +        * @handle: MBZ on input; returns the object's handle, which is nonzero.
> +        */
> +       __u32 handle;
> +
> +       /** @flags: Combination of &enum drm_thames_bo_flags. */
> +       __u32 flags;
> +};
> +
> +/**
> + * struct drm_thames_bo_mmap_offset - Arguments passed to DRM_IOCTL_THAMES_BO_MMAP_OFFSET.
> + */
> +struct drm_thames_bo_mmap_offset {
> +       /** @handle: Handle of the object we want an mmap offset for. */
> +       __u32 handle;
> +
> +       /** @pad: MBZ. */
> +       __u32 pad;
> +
> +       /** @offset: The fake offset to use for subsequent mmap calls. */
> +       __u64 offset;
> +};
> +
> +/**
> + * struct drm_thames_obj_array - An array of versioned objects passed by pointer.
> + *
> + * The stride is the size of the object userspace was built against, so a
> + * structure can grow at its end under a minor bump.
> + * The kernel reads each element with copy_struct_from_user(): a stride smaller
> + * than its own size has the rest read as zero, down to the size the structure
> + * had when it entered the uAPI and no smaller; a larger stride is accepted only
> + * if the bytes the kernel does not know are zero.
> + */
> +struct drm_thames_obj_array {
> +       /** @stride: Size of one element in bytes. */
> +       __u32 stride;
> +
> +       /** @count: Number of elements. */
> +       __u32 count;
> +
> +       /** @array: User pointer to the first element. */
> +       __u64 array;
> +};
> +
>  /**
>   * struct drm_thames_range - A range of the Context's address space.
>   */
> @@ -65,6 +166,73 @@ struct drm_thames_range {
>         __u64 size;
>  };
> 
> +/**
> + * DOC: Bind operation flags
> + *
> + * - %DRM_THAMES_BIND_OP_UNBIND: Unbind the buffer from the Context instead
> + *   of binding it; &drm_thames_bind_op.dsp_va must be 0.
> + */
> +#define DRM_THAMES_BIND_OP_UNBIND      (1 << 0)
> +
> +/**
> + * struct drm_thames_bind_op - One operation of DRM_IOCTL_THAMES_BO_BIND
> + *
> + * A bind maps the whole buffer at @dsp_va in every page table of the file's
> + * Context, with the permissions of its creation flags; an unbind removes the
> + * mapping.  A buffer is bound at most once in a Context, at one address, over
> + * no range another buffer is bound in.  A buffer shared through a dma-buf may
> + * be bound in each importing file's Context, each at its own address.  The
> + * kernel maps 2 MiB blocks where both the address and the buffer's pages are
> + * 2 MiB aligned, so a client aligns a buffer of 2 MiB or more to 2 MiB.
> + *
> + * Unbinding a buffer a submitted Job still reaches through the MMU is the
> + * client's error: the mapping goes at once, and the Job faults in the client's
> + * own Context as on any other fault.  A Job's transfers keep the buffers they
> + * reach, whether bound or not, until the Job is destroyed.  Freeing a bound
> + * buffer unbinds it.
> + */
> +struct drm_thames_bind_op {
> +       /** @handle: The buffer. */
> +       __u32 handle;
> +
> +       /**
> +        * @flags: 0 to bind the buffer at @dsp_va,
> +        * %DRM_THAMES_BIND_OP_UNBIND to unbind it.
> +        */
> +       __u32 flags;
> +
> +       /**
> +        * @dsp_va: Where the buffer is bound in the DSP's address space,
> +        * page-aligned, the buffer wholly inside
> +        * &drm_thames_caps.dsp_va_range; 0 for an unbind.
> +        *
> +        * The address the core's MMU translates through the Context's page
> +        * table, the same on every core.  It is the number userspace embeds
> +        * in params and Kernel code.  It is neither the bus address the page
> +        * table entries hold nor the CPU address mmap returns.
> +        */
> +       __u64 dsp_va;
> +};
> +
> +/**
> + * struct drm_thames_bo_bind - Arguments passed to DRM_IOCTL_THAMES_BO_BIND
> + *
> + * The operations are applied in order and each is done when the call
> + * returns.  The call stops at the first operation that fails and returns its
> + * error; the operations before it hold, and @ops.count is set to their
> + * number.
> + */
> +struct drm_thames_bo_bind {
> +       /** @flags: MBZ. */
> +       __u32 flags;
> +
> +       /** @pad: MBZ. */
> +       __u32 pad;
> +
> +       /** @ops: Array of &struct drm_thames_bind_op, applied in order. */
> +       struct drm_thames_obj_array ops;
> +};
> +
>  /**
>   * enum drm_thames_dev_query_type - Query type
>   *
> @@ -126,9 +294,16 @@ struct drm_thames_caps {
>          */
>         __u64 dru_realtime_dsp_va;
> 
> +       /**
> +        * @dsp_va_range: The span of the Context's address space the client
> +        * binds its buffers in with DRM_IOCTL_THAMES_BO_BIND.  The windows
> +        * above sit below it and @user_region above it.
> +        */
> +       struct drm_thames_range dsp_va_range;
> +
>         /**
>          * @user_region: Where the kernel driver maps the Context's user stack
> -        * and save page.
> +        * and save page, outside @dsp_va_range.
>          */
>         struct drm_thames_range user_region;
> 
> @@ -189,8 +364,14 @@ struct drm_thames_dev_query {
>                            struct drm_thames_ ## __type)
> 
>  enum {
> +       DRM_IOCTL_THAMES_BO_CREATE =
> +               DRM_IOCTL_THAMES(WR, BO_CREATE, bo_create),
> +       DRM_IOCTL_THAMES_BO_MMAP_OFFSET =
> +               DRM_IOCTL_THAMES(WR, BO_MMAP_OFFSET, bo_mmap_offset),
>         DRM_IOCTL_THAMES_DEV_QUERY =
>                 DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
> +       DRM_IOCTL_THAMES_BO_BIND =
> +               DRM_IOCTL_THAMES(WR, BO_BIND, bo_bind),
>  };
> 
>  #if defined(__cplusplus)
> 
> --
> 2.54.0
> 

Re: [PATCH v3 09/16] accel/thames: Add IOCTLs for job creation and submission

[Christian König]

On 10/23/26 08:39, Tomeu Vizoso wrote:
> A Job is created once from a program, its parameters and its transfers,
> checked then, and submitted any number of times; the params buffer's
> contents may change between submissions. A submission pushes the Job
> onto a core's Ring and rings the doorbell; the firmware moves the Fence
> when the Job is done. Each core has a drm-gpu-scheduler instance, with
> one credit per Ring entry. A submit call takes an array of Jobs, the
> buffers they use for implicit synchronisation, and sync objects to
> signal; it waits on none.
> 
> The job submission code was initially based on Panfrost.

I only skimmed over this, but what I saw looks sane to me from the dma_fence requirement side.

Regards,
Christian.

> Assisted-by: LLM
> Signed-off-by: Tomeu Vizoso <tomeu@tomeuvizoso.net>
> 
> ---
> 
> v3:
> - Add Job create/destroy, for validating transfers.
> - Add sync objects to the submission IOCTL.
> - Allocate the done fence with the Job; run_job allocates nothing
>   (Christian König).
> v2:
> - Add thames_accel.h UAPI header (Robert Nelson).
> ---
>  drivers/accel/thames/Kconfig         |   1 +
>  drivers/accel/thames/Makefile        |   3 +
>  drivers/accel/thames/thames_core.c   |  36 +-
>  drivers/accel/thames/thames_core.h   |  14 +
>  drivers/accel/thames/thames_device.h |   6 +
>  drivers/accel/thames/thames_drv.c    |  43 ++
>  drivers/accel/thames/thames_drv.h    |   7 +
>  drivers/accel/thames/thames_job.c    | 869 +++++++++++++++++++++++++++++++++++
>  drivers/accel/thames/thames_job.h    |  73 +++
>  drivers/accel/thames/thames_trace.h  |  77 ++++
>  include/uapi/drm/thames_accel.h      | 259 ++++++++++-
>  11 files changed, 1381 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/accel/thames/Kconfig b/drivers/accel/thames/Kconfig
> index 47a2986fcabd..f2c32ce8d228 100644
> --- a/drivers/accel/thames/Kconfig
> +++ b/drivers/accel/thames/Kconfig
> @@ -6,6 +6,7 @@ config DRM_ACCEL_THAMES
>         depends on TI_K3_DSP_REMOTEPROC
>         depends on REMOTEPROC
>         depends on MAILBOX
> +       select DRM_SCHED
>         select DRM_GEM_DMA_HELPER
>         select DRM_GEM_SHMEM_HELPER
>         select IOMMU_IO_PGTABLE_LPAE
> diff --git a/drivers/accel/thames/Makefile b/drivers/accel/thames/Makefile
> index 143a829f24cf..7bd652795cf5 100644
> --- a/drivers/accel/thames/Makefile
> +++ b/drivers/accel/thames/Makefile
> @@ -6,4 +6,7 @@ thames-y := \
>         thames_core.o \
>         thames_drv.o \
>         thames_gem.o \
> +       thames_job.o \
>         thames_transfers.o
> +
> +CFLAGS_thames_job.o := -I$(src)
> diff --git a/drivers/accel/thames/thames_core.c b/drivers/accel/thames/thames_core.c
> index dff76abd8951..85e8f48aec60 100644
> --- a/drivers/accel/thames/thames_core.c
> +++ b/drivers/accel/thames/thames_core.c
> @@ -25,6 +25,7 @@
>  #include "thames_core.h"
>  #include "thames_device.h"
>  #include "thames_gem.h"
> +#include "thames_job.h"
>  #include "thames_transfers.h"
> 
>  #define THAMES_RESTART_TIMEOUT_MS      5000
> @@ -327,6 +328,7 @@ static void thames_mbox_rx(struct mbox_client *cl, void *mssg)
> 
>         dev_dbg(core->dev, "Doorbell on " THAMES_CORE_FMT ": 0x%x\n", THAMES_CORE_ARG(core),
>                 (u32)(uintptr_t)mssg);
> +       thames_job_complete(core);
>  }
> 
>  /* Embedded in struct thames_core: nothing to free. */
> @@ -515,10 +517,18 @@ int thames_core_init(struct thames_core *core)
>         if (err)
>                 goto err_timers;
> 
> -       err = thames_core_request_doorbell(core);
> +       /*
> +        * Before the doorbell: requesting it delivers at once a word the core
> +        * wrote while no driver was loaded.
> +        */
> +       err = thames_job_init(core);
>         if (err)
>                 goto err_unmap;
> 
> +       err = thames_core_request_doorbell(core);
> +       if (err)
> +               goto err_job;
> +
>         if (rproc->state == RPROC_OFFLINE) {
>                 err = rproc_boot(rproc);
>                 if (err)
> @@ -540,6 +550,8 @@ int thames_core_init(struct thames_core *core)
> 
>  err_doorbell:
>         thames_core_free_doorbell(core);
> +err_job:
> +       thames_job_fini(core);
>  err_unmap:
>         thames_core_unmap_ipc(core);
>  err_timers:
> @@ -554,6 +566,7 @@ void thames_core_fini(struct thames_core *core)
>  {
>         if (core->mbox_chan)
>                 thames_core_free_doorbell(core);
> +       thames_job_fini(core);
>         /*
>          * Drop only our reference and keep the boot count: the core runs on
>          * for the next load to reattach to; only a reset or a suspend, which
> @@ -567,6 +580,21 @@ void thames_core_fini(struct thames_core *core)
>         core->of_node = NULL;
>  }
> 
> +/* The system firmware accepts a stop only of a core in IDLE. */
> +bool thames_core_idles(struct thames_core *core)
> +{
> +       u32 head = READ_ONCE(core->ipc->ring.head);
> +       u32 slot = head % THAMES_RING_DEPTH;
> +       struct thames_fault_record *record = &core->ipc->fault_records[slot];
> +
> +       if (head == core->ring_tail)
> +               return true;
> +
> +       /* Read the fault record after the head that says it is there. */
> +       rmb();
> +       return READ_ONCE(record->kind) == DRM_THAMES_FAULT_FIRMWARE;
> +}
> +
>  /*
>   * rproc_shutdown() reaches the system firmware because the core has one power
>   * count, remoteproc's auto-boot or this driver's boot: the probe defers while
> @@ -577,6 +605,10 @@ int thames_core_suspend(struct thames_core *core)
>         struct rproc *rproc = core->rproc;
>         int err;
> 
> +       err = thames_job_suspend(core);
> +       if (err)
> +               return err;
> +
>         if (core->suspended)
>                 return 0;
> 
> @@ -589,6 +621,7 @@ int thames_core_suspend(struct thames_core *core)
>                 err = -EBUSY;
>         }
>         if (err) {
> +               thames_job_resume(core);
>                 return err;
>         }
> 
> @@ -640,5 +673,6 @@ int thames_core_resume(struct thames_core *core)
> 
>  offline:
>  restart:
> +       thames_job_resume(core);
>         return err;
>  }
> diff --git a/drivers/accel/thames/thames_core.h b/drivers/accel/thames/thames_core.h
> index 590c5716f7f5..a78414d18ad3 100644
> --- a/drivers/accel/thames/thames_core.h
> +++ b/drivers/accel/thames/thames_core.h
> @@ -10,6 +10,7 @@
>  #include <linux/mutex_types.h>
>  #include <linux/remoteproc.h>
> 
> +#include <drm/gpu_scheduler.h>
> 
>  #include "thames_abi.h"
> 
> @@ -44,13 +45,25 @@ struct thames_core {
>         struct mbox_chan *mbox_chan;
> 
>         u32 ring_tail;
> +       struct list_head submitted;
> +       struct list_head held;
> +
> +       struct mutex job_lock;
> +
> +       spinlock_t fence_lock;
> +
>         /*
>          * @rproc_caller is the task inside the driver's own rproc_boot() or
>          * rproc_shutdown(), whose subdevice callbacks then do nothing.
>          */
>         struct task_struct *rproc_caller;
> 
> +       struct drm_gpu_scheduler sched;
> +       u64 fence_context;
> +       u64 emit_seqno;
> +
>         bool suspended;
> +       unsigned long timeout_left;
> 
>         struct resource l2sram;
>         struct resource l2aux;
> @@ -81,6 +94,7 @@ static inline const struct resource *thames_core_window(const struct thames_core
> 
>  int thames_core_init(struct thames_core *core);
>  void thames_core_fini(struct thames_core *core);
> +bool thames_core_idles(struct thames_core *core);
>  int thames_core_suspend(struct thames_core *core);
>  int thames_core_resume(struct thames_core *core);
> 
> diff --git a/drivers/accel/thames/thames_device.h b/drivers/accel/thames/thames_device.h
> index e183c78e2acf..0a34764885df 100644
> --- a/drivers/accel/thames/thames_device.h
> +++ b/drivers/accel/thames/thames_device.h
> @@ -36,7 +36,10 @@ struct thames_caps {
>  struct thames_device {
>         struct drm_device ddev;
> 
> +       struct mutex sched_lock;
> +
>         struct thames_core cores[THAMES_MAX_CORES];
> +       struct drm_gpu_scheduler *sched_list[THAMES_MAX_CORES];
>         unsigned int num_cores;
> 
>         struct thames_caps caps;
> @@ -72,6 +75,9 @@ struct thames_device {
>          */
>         struct mutex bindings_lock;
> 
> +       /* Whoever takes a file off the list closes its Jobs. */
> +       struct mutex files_lock;
> +       struct list_head files;
>  };
> 
>  #define to_thames_device(drm_dev) \
> diff --git a/drivers/accel/thames/thames_drv.c b/drivers/accel/thames/thames_drv.c
> index 030c60963a0a..fc3328834162 100644
> --- a/drivers/accel/thames/thames_drv.c
> +++ b/drivers/accel/thames/thames_drv.c
> @@ -23,6 +23,7 @@
>  #include "thames_core.h"
>  #include "thames_drv.h"
>  #include "thames_gem.h"
> +#include "thames_job.h"
>  #include "thames_transfers.h"
> 
>  static struct platform_device *drm_dev;
> @@ -54,14 +55,23 @@ static int thames_open(struct drm_device *dev, struct drm_file *file)
> 
>         file->driver_priv = thames_priv;
> 
> +       ret = thames_job_open(thames_priv);
> +       if (ret)
> +               goto err_free;
> +
>         ret = thames_context_create(thames_priv);
>         if (ret)
>                 goto err_close_job;
> 
> +       scoped_guard(mutex, &tdev->files_lock)
> +               list_add_tail(&thames_priv->file_node, &tdev->files);
> +
>         drm_dev_exit(idx);
>         return 0;
> 
>  err_close_job:
> +       thames_job_close(thames_priv);
> +err_free:
>         kfree(thames_priv);
>  err_exit:
>         drm_dev_exit(idx);
> @@ -87,6 +97,14 @@ void thames_file_priv_put(struct thames_file_priv *priv)
>  static void thames_postclose(struct drm_device *dev, struct drm_file *file)
>  {
>         struct thames_file_priv *thames_priv = file->driver_priv;
> +       struct thames_device *tdev = thames_priv->tdev;
> +
> +       scoped_guard(mutex, &tdev->files_lock) {
> +               if (!list_empty(&thames_priv->file_node)) {
> +                       list_del_init(&thames_priv->file_node);
> +                       thames_job_close(thames_priv);
> +               }
> +       }
> 
>         dev_dbg(dev->dev, "Context %u: file closed, %u references left\n",
>                 thames_priv->context_id, kref_read(&thames_priv->refcount) - 1);
> @@ -208,6 +226,9 @@ static int thames_ioctl_##func##_entered(struct drm_device *dev, void *data,        \
>  THAMES_IOCTL_ENTERED(bo_create)
>  THAMES_IOCTL_ENTERED(bo_mmap_offset)
>  THAMES_IOCTL_ENTERED(dev_query)
> +THAMES_IOCTL_ENTERED(job_create)
> +THAMES_IOCTL_ENTERED(job_submit)
> +THAMES_IOCTL_ENTERED(job_destroy)
>  THAMES_IOCTL_ENTERED(bo_bind)
> 
>  static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
> @@ -215,6 +236,9 @@ static const struct drm_ioctl_desc thames_drm_driver_ioctls[] = {
>         THAMES_IOCTL(BO_CREATE, bo_create_entered),
>         THAMES_IOCTL(BO_MMAP_OFFSET, bo_mmap_offset_entered),
>         THAMES_IOCTL(DEV_QUERY, dev_query_entered),
> +       THAMES_IOCTL(JOB_CREATE, job_create_entered),
> +       THAMES_IOCTL(JOB_SUBMIT, job_submit_entered),
> +       THAMES_IOCTL(JOB_DESTROY, job_destroy_entered),
>         THAMES_IOCTL(BO_BIND, bo_bind_entered),
>  };
> 
> @@ -272,6 +296,10 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
>         if (err)
>                 return ERR_PTR(err);
> 
> +       err = drmm_mutex_init(&tdev->ddev, &tdev->sched_lock);
> +       if (err)
> +               return ERR_PTR(err);
> +
>         err = drmm_mutex_init(&tdev->ddev, &tdev->transfer_table_pool_lock);
>         if (err)
>                 return ERR_PTR(err);
> @@ -288,8 +316,13 @@ static struct thames_device *thames_device_init(struct platform_device *pdev,
>         if (err)
>                 return ERR_PTR(err);
> 
> +       err = drmm_mutex_init(&tdev->ddev, &tdev->files_lock);
> +       if (err)
> +               return ERR_PTR(err);
> +
>         INIT_LIST_HEAD(&tdev->transfer_tables);
>         INIT_LIST_HEAD(&tdev->contexts);
> +       INIT_LIST_HEAD(&tdev->files);
> 
>         tdev->asid_next = 1;
>         /*
> @@ -347,6 +380,7 @@ static int thames_platform_probe(struct platform_device *pdev)
>                         continue;
>                 }
> 
> +               tdev->sched_list[core] = &tdev->cores[core].sched;
>                 tdev->num_cores++;
>                 core++;
>         }
> @@ -391,6 +425,15 @@ static void thames_platform_remove(struct platform_device *pdev)
> 
>         drm_dev_unplug(&tdev->ddev);
> 
> +       scoped_guard(mutex, &tdev->files_lock) {
> +               struct thames_file_priv *priv, *tmp;
> +
> +               list_for_each_entry_safe(priv, tmp, &tdev->files, file_node) {
> +                       list_del_init(&priv->file_node);
> +                       thames_job_close(priv);
> +               }
> +       }
> +
>         for (core = 0; core < tdev->num_cores; core++)
>                 thames_core_fini(&tdev->cores[core]);
>         tdev->num_cores = 0;
> diff --git a/drivers/accel/thames/thames_drv.h b/drivers/accel/thames/thames_drv.h
> index 7f9031b63780..2115f7eae589 100644
> --- a/drivers/accel/thames/thames_drv.h
> +++ b/drivers/accel/thames/thames_drv.h
> @@ -11,6 +11,7 @@
>  #include <linux/xarray.h>
> 
>  #include <drm/drm_mm.h>
> +#include <drm/gpu_scheduler.h>
> 
>  #include "thames_device.h"
> 
> @@ -26,6 +27,12 @@ struct thames_file_priv {
>          */
>         struct kref refcount;
> 
> +       struct list_head file_node;
> +
> +       struct drm_sched_entity sched_entity;
> +
> +       struct xarray jobs;
> +
>         struct io_pgtable_ops *pgtbl_ops[THAMES_MAX_CORES];
>         u64 page_table_base[THAMES_MAX_CORES];
>         u32 asid;
> diff --git a/drivers/accel/thames/thames_job.c b/drivers/accel/thames/thames_job.c
> new file mode 100644
> index 000000000000..6560106d47a8
> --- /dev/null
> +++ b/drivers/accel/thames/thames_job.c
> @@ -0,0 +1,869 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright 2019 Linaro, Ltd, Rob Herring <robh@kernel.org> */
> +/* Copyright 2019 Collabora ltd. */
> +/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
> +/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
> +
> +#include <clocksource/arm_arch_timer.h>
> +#include <linux/dev_printk.h>
> +#include <linux/dma-fence-chain.h>
> +#include <linux/iopoll.h>
> +#include <linux/mailbox_client.h>
> +#include <linux/platform_device.h>
> +#include <linux/xarray.h>
> +
> +#include <drm/drm_drv.h>
> +#include <drm/drm_file.h>
> +#include <drm/drm_gem.h>
> +#include <drm/drm_print.h>
> +#include <drm/drm_syncobj.h>
> +#include <drm/thames_accel.h>
> +
> +#include "thames_core.h"
> +#include "thames_device.h"
> +#include "thames_drv.h"
> +#include "thames_gem.h"
> +#include "thames_job.h"
> +
> +#define CREATE_TRACE_POINTS
> +#include "thames_trace.h"
> +
> +static struct thames_sched_job *to_thames_sched_job(struct drm_sched_job *sched_job)
> +{
> +       return container_of(sched_job, struct thames_sched_job, base);
> +}
> +
> +static void thames_sched_job_put(struct thames_sched_job *job);
> +static void thames_job_put(struct thames_job *job);
> +
> +#define THAMES_MAX_JOBS_PER_SUBMIT 256
> +#define THAMES_MAX_SUBMIT_BOS 4096
> +#define THAMES_MAX_SUBMIT_SYNCS 64
> +
> +/*
> + * The host's timeout is longer than the device's: a Job still on the Ring when
> + * it fires is on a stalled core.
> + */
> +#define THAMES_DEVICE_TIMEOUT_MS 500
> +
> +struct thames_submit_bos {
> +       struct kref refcount;
> +       u32 read_count;
> +       u32 count;
> +       struct drm_gem_object *bos[] __counted_by(count);
> +};
> +
> +static void thames_submit_bos_free(struct kref *ref)
> +{
> +       struct thames_submit_bos *bos = container_of(ref, struct thames_submit_bos, refcount);
> +       u32 i;
> +
> +       for (i = 0; i < bos->count; i++) {
> +               if (bos->bos[i])
> +                       drm_gem_object_put(bos->bos[i]);
> +       }
> +
> +       kvfree(bos);
> +}
> +
> +static void thames_submit_bos_put(struct thames_submit_bos *bos)
> +{
> +       if (bos)
> +               kref_put(&bos->refcount, thames_submit_bos_free);
> +}
> +
> +static const char *thames_fence_get_driver_name(struct dma_fence *fence)
> +{
> +       return "thames";
> +}
> +
> +static const char *thames_fence_get_timeline_name(struct dma_fence *fence)
> +{
> +       return "thames";
> +}
> +
> +static const struct dma_fence_ops thames_fence_ops = {
> +       .get_driver_name = thames_fence_get_driver_name,
> +       .get_timeline_name = thames_fence_get_timeline_name,
> +};
> +
> +static void thames_job_fail_done_fence(struct thames_sched_job *job, int error)
> +{
> +       if (!dma_fence_was_initialized(job->done_fence) || dma_fence_is_signaled(job->done_fence))
> +               return;
> +
> +       dma_fence_set_error(job->done_fence, error);
> +       dma_fence_signal(job->done_fence);
> +}
> +
> +static int thames_job_hw_submit(struct thames_core *core, struct thames_sched_job *sched_job)
> +{
> +       struct thames_ring *ring = &core->ipc->ring;
> +       struct thames_ring_host *host;
> +       u32 tail = core->ring_tail;
> +       u32 index = tail % THAMES_RING_DEPTH;
> +       int ret;
> +
> +       if (tail - READ_ONCE(ring->head) >= THAMES_RING_DEPTH) {
> +               dev_err(core->dev, "Ring full on " THAMES_CORE_FMT "\n", THAMES_CORE_ARG(core));
> +               return -ENOSPC;
> +       }
> +
> +       sched_job->stamps[0] = arch_timer_read_counter();
> +       sched_job->ring_index = index;
> +
> +       host = &ring->entries[index].host;
> +       host->context_id = sched_job->file_priv->context_id;
> +       host->transfer_count = sched_job->job->transfers.count;
> +       host->transfer_group_count = sched_job->job->transfers.group_count;
> +       host->transfer_table_offset = sched_job->job->transfers.offset;
> +       host->seqno = sched_job->seqno;
> +       host->program_dsp_va = sched_job->job->program_dsp_va;
> +       host->params_dsp_va = sched_job->job->params_dsp_va;
> +       host->context_serial = sched_job->file_priv->context_serial;
> +       host->timeout_ms = THAMES_DEVICE_TIMEOUT_MS;
> +
> +       scoped_guard(mutex, &core->job_lock) {
> +               kref_get(&sched_job->refcount); /* Put by thames_job_complete(). */
> +               list_add_tail(&sched_job->node, &core->submitted);
> +
> +               /* Order the host half before the tail. */
> +               wmb();
> +               WRITE_ONCE(ring->tail, tail + 1);
> +               core->ring_tail = tail + 1;
> +               /* Order the tail before the doorbell. */
> +               wmb();
> +       }
> +
> +       /*
> +        * The Job is on the Ring whatever the send returns; it completes
> +        * through the Fence or the timeout.
> +        */
> +       ret = mbox_send_message(core->mbox_chan, (void *)(uintptr_t)THAMES_DOORBELL);
> +       if (ret < 0)
> +               dev_warn_ratelimited(core->dev, "Doorbell not sent: %d\n", ret);
> +
> +       return 0;
> +}
> +
> +void thames_job_complete(struct thames_core *core)
> +{
> +       struct thames_sched_job *job, *tmp;
> +       LIST_HEAD(done);
> +       u64 completed, signalled;
> +
> +       completed = READ_ONCE(core->ipc->fence.seqno);
> +       /* Read the entries after the Fence that covers them. */
> +       rmb();
> +
> +       scoped_guard(mutex, &core->job_lock) {
> +               list_for_each_entry_safe(job, tmp, &core->submitted, node) {
> +                       if (job->seqno > completed)
> +                               break;
> +                       list_move_tail(&job->node, &done);
> +               }
> +       }
> +
> +       list_for_each_entry_safe(job, tmp, &done, node) {
> +               struct thames_ring_fw *fw = &core->ipc->ring.entries[job->ring_index].fw;
> +
> +               job->stamps[1] = fw->doorbell_seen_stamp;
> +               job->stamps[2] = fw->run_start_stamp;
> +               job->stamps[3] = fw->run_end_stamp;
> +               job->stamps[4] = fw->kernel_entry_stamp;
> +               job->stamps[5] = fw->kernel_exit_stamp;
> +
> +               if (fw->fence_value == THAMES_FENCE_ERROR) {
> +                       dma_fence_set_error(job->done_fence, -EIO);
> +               }
> +
> +               list_del_init(&job->node);
> +               signalled = arch_timer_read_counter();
> +               dma_fence_signal(job->done_fence);
> +               trace_thames_job_done(core, job, signalled);
> +               thames_sched_job_put(job);
> +       }
> +}
> +
> +static int thames_acquire_object_fences(struct drm_gem_object **bos, int bo_count,
> +                                       struct drm_sched_job *job, bool is_write)
> +{
> +       int i, ret;
> +
> +       for (i = 0; i < bo_count; i++) {
> +               ret = dma_resv_reserve_fences(bos[i]->resv, 1);
> +               if (ret)
> +                       return ret;
> +
> +               ret = drm_sched_job_add_implicit_dependencies(job, bos[i], is_write);
> +               if (ret)
> +                       return ret;
> +       }
> +
> +       return 0;
> +}
> +
> +static void thames_attach_object_fences(struct drm_gem_object **bos, int bo_count,
> +                                       struct dma_fence *fence, enum dma_resv_usage usage)
> +{
> +       int i;
> +
> +       for (i = 0; i < bo_count; i++)
> +               dma_resv_add_fence(bos[i]->resv, fence, usage);
> +}
> +
> +/* Nothing here fails, so a submit call pushes every Job or none. */
> +static void thames_job_push(struct thames_sched_job *sched_job)
> +{
> +       scoped_guard(mutex, &sched_job->tdev->sched_lock) {
> +               drm_sched_job_arm(&sched_job->base);
> +
> +               sched_job->inference_done_fence = dma_fence_get(&sched_job->base.s_fence->finished);
> +
> +               kref_get(&sched_job->refcount); /* put by scheduler job completion */
> +
> +               drm_sched_entity_push_job(&sched_job->base);
> +       }
> +}
> +
> +static void thames_sched_job_cleanup(struct kref *ref)
> +{
> +       struct thames_sched_job *sched_job = container_of(ref, struct thames_sched_job, refcount);
> +
> +       if (dma_fence_was_initialized(sched_job->done_fence))
> +               dma_fence_put(sched_job->done_fence);
> +       else
> +               dma_fence_free(sched_job->done_fence);
> +       dma_fence_put(sched_job->inference_done_fence);
> +
> +       thames_submit_bos_put(sched_job->bos);
> +
> +       if (sched_job->file_priv)
> +               thames_file_priv_put(sched_job->file_priv);
> +
> +       if (sched_job->job)
> +               thames_job_put(sched_job->job);
> +
> +       kfree(sched_job);
> +}
> +
> +static void thames_sched_job_put(struct thames_sched_job *job)
> +{
> +       kref_put(&job->refcount, thames_sched_job_cleanup);
> +}
> +
> +static void thames_sched_job_free(struct drm_sched_job *sched_job)
> +{
> +       struct thames_sched_job *job = to_thames_sched_job(sched_job);
> +
> +       drm_sched_job_cleanup(sched_job);
> +
> +       thames_sched_job_put(job);
> +}
> +
> +static struct dma_fence *thames_job_run(struct drm_sched_job *sched_job)
> +{
> +       struct thames_sched_job *job = to_thames_sched_job(sched_job);
> +       struct thames_core *core = container_of(sched_job->sched, struct thames_core, sched);
> +       int ret;
> +
> +       if (unlikely(job->base.s_fence->finished.error))
> +               return NULL;
> +
> +       /* The done fence keeps the seqno of the Job's first push. */
> +       job->seqno = ++core->emit_seqno;
> +       if (!dma_fence_was_initialized(job->done_fence))
> +               dma_fence_init(job->done_fence, &thames_fence_ops, &core->fence_lock,
> +                              core->fence_context, job->seqno);
> +
> +       ret = thames_job_hw_submit(core, job);
> +       if (ret) {
> +               thames_job_fail_done_fence(job, ret);
> +               return ERR_PTR(ret);
> +       }
> +
> +       return dma_fence_get(job->done_fence);
> +}
> +
> +/*
> + * A done fence not signalled here is a Job on a live core's Ring that may still
> + * run.
> + */
> +static void thames_job_cancel(struct drm_sched_job *sched_job)
> +{
> +       struct thames_sched_job *job = to_thames_sched_job(sched_job);
> +       struct thames_core *core = container_of(sched_job->sched, struct thames_core, sched);
> +
> +       if (!dma_fence_was_initialized(job->done_fence) || dma_fence_is_signaled(job->done_fence))
> +               return;
> +
> +       scoped_guard(mutex, &core->job_lock)
> +               list_move_tail(&job->node, &core->held);
> +
> +       thames_job_fail_done_fence(job, -ECANCELED);
> +}
> +
> +static const struct drm_sched_backend_ops thames_sched_ops = {
> +       .run_job = thames_job_run,
> +       .free_job = thames_sched_job_free,
> +       .cancel_job = thames_job_cancel,
> +};
> +
> +int thames_job_init(struct thames_core *core)
> +{
> +       /* One credit per Ring entry, so the tail never overruns the head. */
> +       struct drm_sched_init_args args = {
> +               .ops = &thames_sched_ops,
> +               .num_rqs = 1,
> +               .credit_limit = THAMES_RING_DEPTH,
> +               .timeout = MAX_SCHEDULE_TIMEOUT,
> +               .name = dev_name(core->dev),
> +               .dev = core->dev,
> +       };
> +       u32 head, tail, before;
> +       u64 fence, before_seqno, last;
> +       int ret;
> +
> +       spin_lock_init(&core->fence_lock);
> +       mutex_init(&core->job_lock);
> +
> +       INIT_LIST_HEAD(&core->submitted);
> +       INIT_LIST_HEAD(&core->held);
> +
> +       /*
> +        * Both indices run free, so only a tail not at the head is wrong: the
> +        * previous instance's entries are still pending.  The Fence may trail
> +        * the head by one entry, the firmware moving the head first.
> +        */
> +       head = READ_ONCE(core->ipc->ring.head);
> +       tail = READ_ONCE(core->ipc->ring.tail);
> +       before = (head - 1) % THAMES_RING_DEPTH;
> +
> +       if (tail != head) {
> +               dev_err(core->dev, "Ring on " THAMES_CORE_FMT
> +                       " has %u pending entries (tail %u, head %u); not attaching\n",
> +                       THAMES_CORE_ARG(core), tail - head, tail, head);
> +               return -EBUSY;
> +       }
> +
> +       /* Read the Fence and the entry after the head and the tail. */
> +       rmb();
> +       fence = READ_ONCE(core->ipc->fence.seqno);
> +       before_seqno = core->ipc->ring.entries[before].host.seqno;
> +       last = max(fence, before_seqno);
> +
> +       core->ring_tail = head;
> +       core->emit_seqno = last;
> +
> +       core->fence_context = dma_fence_context_alloc(1);
> +
> +       ret = drm_sched_init(&core->sched, &args);
> +       if (ret) {
> +               dev_err(core->dev, "Failed to create scheduler: %d.", ret);
> +               return ret;
> +       }
> +
> +       return 0;
> +}
> +
> +/*
> + * These Jobs' core may still run their Kernels: the Jobs, Contexts and buffers
> + * stay allocated.
> + */
> +void thames_job_fini(struct thames_core *core)
> +{
> +       struct thames_sched_job *job;
> +       u64 first = U64_MAX, last = 0;
> +       size_t kept;
> +
> +       drm_sched_fini(&core->sched);
> +
> +       guard(mutex)(&core->job_lock);
> +
> +       kept = list_count_nodes(&core->held);
> +       if (!kept)
> +               return;
> +
> +       list_for_each_entry(job, &core->held, node) {
> +               first = min(first, job->seqno);
> +               last = max(last, job->seqno);
> +       }
> +
> +       drm_dev_get(&core->tdev->ddev);
> +       dev_warn(core->dev, THAMES_CORE_FMT
> +                " may still run %zu Jobs (seqno %llu to %llu); keeping them and their buffers until reboot\n",
> +                THAMES_CORE_ARG(core), kept, first, last);
> +}
> +
> +/*
> + * Above the device's timeout, so the firmware ends a runaway Kernel within the
> + * wait.
> + */
> +#define THAMES_SUSPEND_TIMEOUT_MS (THAMES_DEVICE_TIMEOUT_MS + 100)
> +
> +int thames_job_suspend(struct thames_core *core)
> +{
> +       struct drm_gpu_scheduler *sched = &core->sched;
> +       unsigned int wait_ms;
> +       size_t pending;
> +       bool idle;
> +       int err;
> +
> +       drm_sched_wqueue_stop(sched);
> +       core->timeout_left = drm_sched_suspend_timeout(sched);
> +       cancel_delayed_work_sync(&sched->work_tdr);
> +       drm_sched_wqueue_stop(sched);
> +
> +       scoped_guard(mutex, &core->job_lock)
> +               pending = list_count_nodes(&core->submitted);
> +       wait_ms = max_t(unsigned int, pending, 1) * THAMES_SUSPEND_TIMEOUT_MS;
> +
> +       err = read_poll_timeout(thames_core_idles, idle, idle, USEC_PER_MSEC,
> +                               wait_ms * USEC_PER_MSEC, false, core);
> +
> +       if (!err) {
> +               thames_job_complete(core);
> +               scoped_guard(mutex, &core->job_lock) {
> +                       if (!list_empty(&core->submitted))
> +                               err = -EBUSY;
> +               }
> +       }
> +
> +       if (err) {
> +               dev_err(core->dev, THAMES_CORE_FMT
> +                       " is not idle with its Jobs completed after %u ms; refusing the suspend\n",
> +                       THAMES_CORE_ARG(core), wait_ms);
> +               thames_job_resume(core);
> +               return -EBUSY;
> +       }
> +
> +       return 0;
> +}
> +
> +void thames_job_resume(struct thames_core *core)
> +{
> +       drm_sched_resume_timeout(&core->sched, core->timeout_left);
> +       drm_sched_wqueue_start(&core->sched);
> +}
> +
> +int thames_job_open(struct thames_file_priv *thames_priv)
> +{
> +       struct thames_device *tdev = thames_priv->tdev;
> +       int ret;
> +
> +       ret = drm_sched_entity_init(&thames_priv->sched_entity, 0, tdev->sched_list,
> +                                   tdev->num_cores, NULL);
> +       if (WARN_ON(ret))
> +               return ret;
> +
> +       xa_init_flags(&thames_priv->jobs, XA_FLAGS_ALLOC1);
> +
> +       return 0;
> +}
> +
> +void thames_job_close(struct thames_file_priv *thames_priv)
> +{
> +       struct thames_job *job;
> +       unsigned long handle;
> +
> +       drm_sched_entity_destroy(&thames_priv->sched_entity);
> +
> +       xa_for_each(&thames_priv->jobs, handle, job)
> +               thames_job_put(job);
> +       xa_destroy(&thames_priv->jobs);
> +}
> +
> +static void thames_job_free(struct kref *ref)
> +{
> +       struct thames_job *job = container_of(ref, struct thames_job, refcount);
> +
> +       if (job->program)
> +               drm_gem_object_put(job->program);
> +       if (job->params)
> +               drm_gem_object_put(job->params);
> +
> +       thames_transfers_free(job->tdev, &job->transfers);
> +       mutex_destroy(&job->lock);
> +       kfree(job);
> +}
> +
> +static void thames_job_put(struct thames_job *job)
> +{
> +       kref_put(&job->refcount, thames_job_free);
> +}
> +
> +int thames_ioctl_job_create(struct drm_device *dev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_job_create *args = data;
> +       struct thames_file_priv *file_priv = file->driver_priv;
> +       struct thames_device *tdev = to_thames_device(dev);
> +       struct thames_job *job;
> +       u32 handle;
> +       int ret;
> +
> +       if (args->flags || args->pad)
> +               return -EINVAL;
> +
> +       if (args->type != DRM_THAMES_JOB_DISPATCH)
> +               return -EINVAL;
> +
> +       if (!args->program || !args->params)
> +               return -EINVAL;
> +
> +       job = kzalloc_obj(*job, GFP_KERNEL);
> +       if (!job)
> +               return -ENOMEM;
> +
> +       kref_init(&job->refcount);
> +       mutex_init(&job->lock);
> +       job->tdev = tdev;
> +
> +       job->program = drm_gem_object_lookup(file, args->program);
> +       if (!job->program) {
> +               ret = -ENOENT;
> +               goto err_put;
> +       }
> +
> +       if (!(to_thames_bo(job->program)->flags & DRM_THAMES_BO_EXECUTABLE)) {
> +               ret = -EINVAL;
> +               goto err_put;
> +       }
> +
> +       job->program_dsp_va = thames_bo_dsp_va(to_thames_bo(job->program), file_priv);
> +       if (!job->program_dsp_va) {
> +               ret = -EINVAL;
> +               goto err_put;
> +       }
> +
> +       job->params = drm_gem_object_lookup(file, args->params);
> +       if (!job->params) {
> +               ret = -ENOENT;
> +               goto err_put;
> +       }
> +
> +       job->params_dsp_va = thames_bo_dsp_va(to_thames_bo(job->params), file_priv);
> +       if (!job->params_dsp_va) {
> +               ret = -EINVAL;
> +               goto err_put;
> +       }
> +
> +       ret = thames_transfers_build(file, &args->transfers, &job->transfers);
> +       if (ret)
> +               goto err_put;
> +
> +       ret = xa_alloc(&file_priv->jobs, &handle, job, xa_limit_32b, GFP_KERNEL);
> +       if (ret)
> +               goto err_put;
> +
> +       job->handle = handle;
> +       args->handle = handle;
> +
> +       return 0;
> +
> +err_put:
> +       thames_job_put(job);
> +       return ret;
> +}
> +
> +static struct thames_sched_job *thames_sched_job_new(struct drm_file *file, u32 handle,
> +                                                    struct thames_submit_bos *bos)
> +{
> +       struct thames_file_priv *file_priv = file->driver_priv;
> +       struct thames_job *job;
> +       struct thames_sched_job *sched_job;
> +       int ret;
> +
> +       xa_lock(&file_priv->jobs);
> +       job = xa_load(&file_priv->jobs, handle);
> +       if (job)
> +               kref_get(&job->refcount);
> +       xa_unlock(&file_priv->jobs);
> +
> +       if (!job)
> +               return ERR_PTR(-ENOENT);
> +
> +       sched_job = kzalloc_obj(*sched_job, GFP_KERNEL);
> +       if (!sched_job) {
> +               thames_job_put(job);
> +               return ERR_PTR(-ENOMEM);
> +       }
> +
> +       kref_init(&sched_job->refcount);
> +       INIT_LIST_HEAD(&sched_job->node);
> +       sched_job->tdev = job->tdev;
> +       sched_job->job = job; /* The lookup's reference. */
> +       /* Put by the completion path once it has counted the Job for the ban. */
> +       kref_get(&file_priv->refcount);
> +       sched_job->file_priv = file_priv;
> +       kref_get(&bos->refcount);
> +       sched_job->bos = bos;
> +
> +       sched_job->done_fence = kzalloc_obj(*sched_job->done_fence);
> +       if (!sched_job->done_fence) {
> +               thames_sched_job_put(sched_job);
> +               return ERR_PTR(-ENOMEM);
> +       }
> +
> +       ret = drm_sched_job_init(&sched_job->base, &file_priv->sched_entity, 1, NULL,
> +                                file->client_id);
> +       if (ret) {
> +               thames_sched_job_put(sched_job);
> +               return ERR_PTR(ret);
> +       }
> +
> +       return sched_job;
> +}
> +
> +static int thames_submit_bos_lookup(struct drm_file *file,
> +                                   const struct drm_thames_job_submit *args,
> +                                   struct thames_submit_bos **out)
> +{
> +       const struct drm_thames_obj_array *lists[2] = { &args->read_handles,
> +                                                       &args->write_handles };
> +       struct thames_submit_bos *bos;
> +       u32 total = 0, n = 0, l, i;
> +
> +       for (l = 0; l < ARRAY_SIZE(lists); l++) {
> +               if (lists[l]->count > THAMES_MAX_SUBMIT_BOS - total)
> +                       return -EINVAL;
> +               total += lists[l]->count;
> +       }
> +
> +       bos = kvzalloc(struct_size(bos, bos, total), GFP_KERNEL);
> +       if (!bos)
> +               return -ENOMEM;
> +
> +       kref_init(&bos->refcount);
> +       bos->count = total;
> +       bos->read_count = args->read_handles.count;
> +
> +       for (l = 0; l < ARRAY_SIZE(lists); l++) {
> +               u32 *handles;
> +
> +               if (!lists[l]->count)
> +                       continue;
> +
> +               handles = thames_get_uobj_array(lists[l], sizeof(*handles), sizeof(*handles));
> +               if (IS_ERR(handles)) {
> +                       thames_submit_bos_put(bos);
> +                       return PTR_ERR(handles);
> +               }
> +
> +               for (i = 0; i < lists[l]->count; i++, n++) {
> +                       bos->bos[n] = drm_gem_object_lookup(file, handles[i]);
> +                       if (!bos->bos[n]) {
> +                               kvfree(handles);
> +                               thames_submit_bos_put(bos);
> +                               return -ENOENT;
> +                       }
> +               }
> +
> +               kvfree(handles);
> +       }
> +
> +       *out = bos;
> +       return 0;
> +}
> +
> +/*
> + * Allocated before the first push, so signalling cannot fail once the Jobs run.
> + */
> +struct thames_submit_syncs {
> +       u32 count;
> +       struct drm_thames_sync_op *ops;
> +       struct thames_sync_signal {
> +               struct drm_syncobj *syncobj;
> +               struct dma_fence_chain *chain;
> +       } *signals;
> +};
> +
> +static int thames_sync_op_check(const struct drm_thames_sync_op *op)
> +{
> +       if (op->flags != (DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
> +                         DRM_THAMES_SYNC_OP_SIGNAL))
> +               return -EINVAL;
> +
> +       return op->timeline_value ? 0 : -EINVAL;
> +}
> +
> +static void thames_submit_syncs_free(struct thames_submit_syncs *syncs)
> +{
> +       u32 i;
> +
> +       for (i = 0; i < syncs->count; i++) {
> +               if (syncs->signals[i].syncobj)
> +                       drm_syncobj_put(syncs->signals[i].syncobj);
> +               dma_fence_chain_free(syncs->signals[i].chain);
> +       }
> +
> +       kvfree(syncs->signals);
> +       kvfree(syncs->ops);
> +}
> +
> +static int thames_submit_syncs_get(struct drm_file *file, const struct drm_thames_obj_array *in,
> +                                  struct thames_submit_syncs *syncs)
> +{
> +       struct drm_thames_sync_op *ops;
> +       u32 i;
> +       int ret;
> +
> +       if (!in->count)
> +               return 0;
> +
> +       if (in->count > THAMES_MAX_SUBMIT_SYNCS)
> +               return -EINVAL;
> +
> +       ops = thames_get_uobj_array(in, offsetofend(struct drm_thames_sync_op, timeline_value),
> +                                   sizeof(*ops));
> +       if (IS_ERR(ops))
> +               return PTR_ERR(ops);
> +       syncs->ops = ops;
> +
> +       syncs->signals = kvcalloc(in->count, sizeof(*syncs->signals), GFP_KERNEL);
> +       if (!syncs->signals)
> +               return -ENOMEM;
> +       syncs->count = in->count;
> +
> +       for (i = 0; i < syncs->count; i++) {
> +               struct thames_sync_signal *signal = &syncs->signals[i];
> +
> +               ret = thames_sync_op_check(&ops[i]);
> +               if (ret)
> +                       return ret;
> +
> +               signal->syncobj = drm_syncobj_find(file, ops[i].handle);
> +               if (!signal->syncobj)
> +                       return -ENOENT;
> +
> +               signal->chain = dma_fence_chain_alloc();
> +               if (!signal->chain)
> +                       return -ENOMEM;
> +       }
> +
> +       return 0;
> +}
> +
> +static void thames_submit_syncs_signal(struct thames_submit_syncs *syncs, struct dma_fence *fence)
> +{
> +       u32 i;
> +
> +       for (i = 0; i < syncs->count; i++) {
> +               struct thames_sync_signal *signal = &syncs->signals[i];
> +
> +               drm_syncobj_add_point(signal->syncobj, signal->chain, fence,
> +                                     syncs->ops[i].timeline_value);
> +               signal->chain = NULL;
> +       }
> +}
> +
> +int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_job_submit *args = data;
> +       struct thames_submit_syncs syncs = {};
> +       struct thames_submit_bos *bos = NULL;
> +       struct ww_acquire_ctx acquire_ctx;
> +       struct thames_sched_job **subs = NULL;
> +       struct dma_fence *last;
> +       u32 count = args->jobs.count;
> +       u32 pushed = 0;
> +       u32 *handles;
> +       unsigned int i;
> +       int ret = 0;
> +
> +       if (args->flags || args->pad)
> +               return -EINVAL;
> +
> +       if (!count || count > THAMES_MAX_JOBS_PER_SUBMIT)
> +               return -EINVAL;
> +
> +       handles = thames_get_uobj_array(&args->jobs, sizeof(u32), sizeof(u32));
> +       if (IS_ERR(handles))
> +               return PTR_ERR(handles);
> +
> +       ret = thames_submit_syncs_get(file, &args->syncs, &syncs);
> +       if (ret)
> +               goto out_syncs;
> +
> +       ret = thames_submit_bos_lookup(file, args, &bos);
> +       if (ret)
> +               goto out_syncs;
> +
> +       subs = kvcalloc(count, sizeof(*subs), GFP_KERNEL);
> +       if (!subs) {
> +               ret = -ENOMEM;
> +               goto out_put;
> +       }
> +
> +       for (i = 0; i < count; i++) {
> +               subs[i] = thames_sched_job_new(file, handles[i], bos);
> +               if (IS_ERR(subs[i])) {
> +                       ret = PTR_ERR(subs[i]);
> +                       subs[i] = NULL;
> +                       goto out_subs;
> +               }
> +       }
> +
> +       ret = drm_gem_lock_reservations(bos->bos, bos->count, &acquire_ctx);
> +       if (ret)
> +               goto out_subs;
> +
> +       /*
> +        * Only the first Job waits: the entity runs the call's Jobs in order.
> +        */
> +       ret = thames_acquire_object_fences(bos->bos, bos->read_count, &subs[0]->base, false);
> +       if (ret)
> +               goto out_unlock;
> +
> +       ret = thames_acquire_object_fences(&bos->bos[bos->read_count], bos->count - bos->read_count,
> +                                          &subs[0]->base, true);
> +       if (ret)
> +               goto out_unlock;
> +
> +       for (pushed = 0; pushed < count; pushed++)
> +               thames_job_push(subs[pushed]);
> +
> +       last = subs[count - 1]->inference_done_fence;
> +       thames_attach_object_fences(bos->bos, bos->read_count, last, DMA_RESV_USAGE_READ);
> +       thames_attach_object_fences(&bos->bos[bos->read_count], bos->count - bos->read_count, last,
> +                                   DMA_RESV_USAGE_WRITE);
> +       thames_submit_syncs_signal(&syncs, last);
> +
> +out_unlock:
> +       drm_gem_unlock_reservations(bos->bos, bos->count, &acquire_ctx);
> +out_subs:
> +       for (i = 0; i < count; i++) {
> +               if (!subs[i])
> +                       continue;
> +               if (i >= pushed)
> +                       drm_sched_job_cleanup(&subs[i]->base);
> +               /* The creation reference; the scheduler holds its own. */
> +               thames_sched_job_put(subs[i]);
> +       }
> +       kvfree(subs);
> +out_put:
> +       thames_submit_bos_put(bos);
> +out_syncs:
> +       thames_submit_syncs_free(&syncs);
> +       kvfree(handles);
> +       return ret;
> +}
> +
> +int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file *file)
> +{
> +       struct drm_thames_job_destroy *args = data;
> +       struct thames_file_priv *file_priv = file->driver_priv;
> +       struct thames_job *job;
> +
> +       if (args->pad)
> +               return -EINVAL;
> +
> +       job = xa_erase(&file_priv->jobs, args->handle);
> +       if (!job)
> +               return -ENOENT;
> +
> +       /* A submission still running holds its own reference. */
> +       thames_job_put(job);
> +
> +       return 0;
> +}
> +
> diff --git a/drivers/accel/thames/thames_job.h b/drivers/accel/thames/thames_job.h
> new file mode 100644
> index 000000000000..a851febe8f98
> --- /dev/null
> +++ b/drivers/accel/thames/thames_job.h
> @@ -0,0 +1,73 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright 2024-2025 Tomeu Vizoso <tomeu@tomeuvizoso.net> */
> +/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
> +
> +#ifndef __THAMES_JOB_H__
> +#define __THAMES_JOB_H__
> +
> +#include <drm/drm_drv.h>
> +#include <drm/gpu_scheduler.h>
> +#include <drm/thames_accel.h>
> +
> +#include "thames_core.h"
> +#include "thames_drv.h"
> +#include "thames_transfers.h"
> +
> +struct thames_submit_bos;
> +
> +struct thames_sched_job {
> +       struct drm_sched_job base;
> +
> +       struct thames_device *tdev;
> +       struct thames_file_priv *file_priv;
> +
> +       struct dma_fence *inference_done_fence;
> +       struct dma_fence *done_fence;
> +
> +       /* @seqno moves at every push; the done fence keeps the first push's. */
> +       u64 seqno;
> +       u32 ring_index;
> +       u64 stamps[6];
> +       struct list_head node;
> +
> +       struct thames_job *job;
> +
> +       struct thames_submit_bos *bos;
> +
> +       struct kref refcount;
> +};
> +
> +struct thames_job {
> +       struct kref refcount;
> +       struct thames_device *tdev;
> +       u32 handle;
> +
> +       struct drm_gem_object *program;
> +       struct drm_gem_object *params;
> +
> +       /*
> +        * Taken at create: after an unbind the Job faults in its Context rather
> +        * than read a binding that is gone.
> +        */
> +       u64 program_dsp_va;
> +       u64 params_dsp_va;
> +
> +       struct thames_transfer_table transfers;
> +
> +       struct mutex lock;
> +};
> +
> +int thames_ioctl_job_create(struct drm_device *dev, void *data, struct drm_file *file);
> +int thames_ioctl_job_submit(struct drm_device *dev, void *data, struct drm_file *file);
> +int thames_ioctl_job_destroy(struct drm_device *dev, void *data, struct drm_file *file);
> +
> +void thames_job_complete(struct thames_core *core);
> +
> +int thames_job_init(struct thames_core *core);
> +int thames_job_suspend(struct thames_core *core);
> +void thames_job_resume(struct thames_core *core);
> +void thames_job_fini(struct thames_core *core);
> +int thames_job_open(struct thames_file_priv *thames_priv);
> +void thames_job_close(struct thames_file_priv *thames_priv);
> +
> +#endif
> diff --git a/drivers/accel/thames/thames_trace.h b/drivers/accel/thames/thames_trace.h
> new file mode 100644
> index 000000000000..3d1e59708869
> --- /dev/null
> +++ b/drivers/accel/thames/thames_trace.h
> @@ -0,0 +1,77 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright 2026 Texas Instruments Incorporated - https://www.ti.com/ */
> +
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM thames
> +
> +#if !defined(__THAMES_TRACE_H__) || defined(TRACE_HEADER_MULTI_READ)
> +#define __THAMES_TRACE_H__
> +
> +#include <linux/math64.h>
> +#include <linux/time64.h>
> +#include <linux/tracepoint.h>
> +#include <linux/types.h>
> +
> +#include "thames_core.h"
> +#include "thames_device.h"
> +#include "thames_drv.h"
> +#include "thames_job.h"
> +
> +/* Every stamp is the SoC's system counter in microseconds, so any two subtract. */
> +TRACE_EVENT(thames_job_done,
> +           TP_PROTO(const struct thames_core *core, const struct thames_sched_job *job,
> +                    u64 signalled),
> +
> +           TP_ARGS(core, job, signalled),
> +
> +           TP_STRUCT__entry(__string(dev_name, dev_name(core->dev))
> +                            __field(u32, core)
> +                            __field(u32, context)
> +                            __field(u64, seqno)
> +                            __field(int, error)
> +                            __field(u64, submit_us)
> +                            __field(u64, doorbell_seen_us)
> +                            __field(u64, run_start_us)
> +                            __field(u64, kernel_entry_us)
> +                            __field(u64, kernel_exit_us)
> +                            __field(u64, run_end_us)
> +                            __field(u64, signalled_us)),
> +
> +           TP_fast_assign(u32 hz = core->tdev->caps.timestamp_hz;
> +
> +                          __assign_str(dev_name);
> +                          __entry->core = core->index;
> +                          __entry->context = job->file_priv->context_id;
> +                          __entry->seqno = job->seqno;
> +                          __entry->error = job->done_fence->error;
> +                          __entry->submit_us =
> +                               mul_u64_u32_div(job->stamps[0], USEC_PER_SEC, hz);
> +                          __entry->doorbell_seen_us =
> +                               mul_u64_u32_div(job->stamps[1], USEC_PER_SEC, hz);
> +                          __entry->run_start_us =
> +                               mul_u64_u32_div(job->stamps[2], USEC_PER_SEC, hz);
> +                          __entry->run_end_us =
> +                               mul_u64_u32_div(job->stamps[3], USEC_PER_SEC, hz);
> +                          __entry->kernel_entry_us =
> +                               mul_u64_u32_div(job->stamps[4], USEC_PER_SEC, hz);
> +                          __entry->kernel_exit_us =
> +                               mul_u64_u32_div(job->stamps[5], USEC_PER_SEC, hz);
> +                          __entry->signalled_us =
> +                               mul_u64_u32_div(signalled, USEC_PER_SEC, hz);),
> +
> +           TP_printk("%s: core=%u context=%u seqno=%llu error=%d submit_us=%llu doorbell_seen_us=%llu run_start_us=%llu kernel_entry_us=%llu kernel_exit_us=%llu run_end_us=%llu signalled_us=%llu",
> +                     __get_str(dev_name), __entry->core, __entry->context,
> +                     __entry->seqno, __entry->error, __entry->submit_us,
> +                     __entry->doorbell_seen_us, __entry->run_start_us,
> +                     __entry->kernel_entry_us, __entry->kernel_exit_us,
> +                     __entry->run_end_us, __entry->signalled_us)
> +);
> +
> +#endif /* __THAMES_TRACE_H__ */
> +
> +#undef TRACE_INCLUDE_PATH
> +#define TRACE_INCLUDE_PATH .
> +#undef TRACE_INCLUDE_FILE
> +#define TRACE_INCLUDE_FILE thames_trace
> +
> +#include <trace/define_trace.h>
> diff --git a/include/uapi/drm/thames_accel.h b/include/uapi/drm/thames_accel.h
> index 5671cffaf8f9..bd2b4271b896 100644
> --- a/include/uapi/drm/thames_accel.h
> +++ b/include/uapi/drm/thames_accel.h
> @@ -57,6 +57,15 @@ enum drm_thames_ioctl_id {
>         /** @DRM_THAMES_DEV_QUERY: Query the device's capabilities. */
>         DRM_THAMES_DEV_QUERY,
> 
> +       /** @DRM_THAMES_JOB_CREATE: Validate a Dispatch once into a Job. */
> +       DRM_THAMES_JOB_CREATE,
> +
> +       /** @DRM_THAMES_JOB_SUBMIT: Run created Jobs once, in order. */
> +       DRM_THAMES_JOB_SUBMIT,
> +
> +       /** @DRM_THAMES_JOB_DESTROY: Release a Job and the buffers it holds. */
> +       DRM_THAMES_JOB_DESTROY,
> +
>         /**
>          * @DRM_THAMES_BO_BIND: Bind buffers into the file's Context at the
>          * addresses the client chooses, or unbind them.
> @@ -244,6 +253,21 @@ enum drm_thames_dev_query_type {
> 
>  };
> 
> +/**
> + * enum drm_thames_job_type - What a Job is
> + *
> + * A Job is one Dispatch, and a submission is an ordered array of Jobs.  A Job
> + * whose transfers are not K3 DMA records would be a new type, not a new
> + * operation, and so are the copies, fills, timestamps and the list-carrying
> + * command buffer a Vulkan compute driver will add.  The query grows a field
> + * naming the types beyond the Dispatch when the first one comes; an absent type
> + * is refused at Job creation.
> + */
> +enum drm_thames_job_type {
> +       /** @DRM_THAMES_JOB_DISPATCH: Run a Kernel with its transfers. */
> +       DRM_THAMES_JOB_DISPATCH = 0,
> +};
> +
>  /**
>   * enum drm_thames_hardware_id - &drm_thames_caps.hardware_id
>   */
> @@ -255,8 +279,9 @@ enum drm_thames_hardware_id {
>  /**
>   * struct drm_thames_caps - The capability query
>   *
> - * Each window sits at one address whatever core runs the Job and reaches that
> - * core's own memory.
> + * The submit's limits are stated on &struct drm_thames_job_submit.  Each window
> + * sits at one address whatever core runs the Job and reaches that core's own
> + * memory.
>   */
>  struct drm_thames_caps {
>         /** @l2_size: Size in bytes of each core's L2 SRAM. */
> @@ -328,6 +353,35 @@ struct drm_thames_caps {
>         __u64 dru_capabilities;
>  };
> 
> +/**
> + * enum drm_thames_fault_kind - The kind of a Job's fault
> + *
> + * The kind is what the firmware knows without decoding a register: what stopped
> + * the Job and whose mode it stopped in.
> + */
> +enum drm_thames_fault_kind {
> +       /**
> +        * @DRM_THAMES_FAULT_KERNEL: A Kernel took an exception in user mode.
> +        * The Context continues; the registers say why.
> +        */
> +       DRM_THAMES_FAULT_KERNEL = 1,
> +
> +       /**
> +        * @DRM_THAMES_FAULT_FIRMWARE: The firmware took an exception in
> +        * supervisor mode while running this Job.  The core was reset.
> +        */
> +       DRM_THAMES_FAULT_FIRMWARE,
> +
> +       /**
> +        * @DRM_THAMES_FAULT_TIMEOUT: The device's Job timeout fired: the Kernel
> +        * ran longer than the driver lets a Kernel run and the device ended it,
> +        * or the Kernel returned with transfers still running and they did not
> +        * drain. The Context continues; NRP and NTSR say where the Kernel was,
> +        * and are 0 for transfers that did not drain.
> +        */
> +       DRM_THAMES_FAULT_TIMEOUT,
> +};
> +
>  /**
>   * struct drm_thames_dev_query - Arguments passed to DRM_IOCTL_THAMES_DEV_QUERY
>   */
> @@ -353,9 +407,9 @@ struct drm_thames_dev_query {
>   * order: the first group starts at the first transfer, each later group at a
>   * transfer carrying this flag.  The firmware submits the first group before it
>   * enters the Kernel, and each later group, whole, when the Kernel asks for it
> - * with system call 2.  No group may hold more transfers on one channel than
> - * &drm_thames_caps.dru_queue_depth.  Never on the first transfer.  A Job
> - * without the flag is one group.
> + * with system call 2 (see &struct drm_thames_job_create).  No group may hold
> + * more transfers on one channel than &drm_thames_caps.dru_queue_depth.  Never
> + * on the first transfer.  A Job without the flag is one group.
>   */
>  #define DRM_THAMES_TRANSFER_GROUP      (1 << 0)
> 
> @@ -393,7 +447,9 @@ struct drm_thames_dev_query {
>   * auxiliary aperture, at &drm_thames_caps.l2_main_dsp_va and l2_aux, which the
>   * kernel translates to the physical aperture of the core the Job runs on.  The
>   * Job holds a reference on each buffer a transfer resolves to until it is
> - * destroyed and its last submission has completed.
> + * destroyed and its last submission has completed.  No list names these
> + * buffers: the read and write lists of &drm_thames_job_submit are for
> + * synchronisation alone.
>   */
>  struct drm_thames_transfer {
>         /**
> @@ -409,6 +465,191 @@ struct drm_thames_transfer {
>         __u32 words[16];
>  };
> 
> +/**
> + * struct drm_thames_job_create - Arguments passed to DRM_IOCTL_THAMES_JOB_CREATE
> + *
> + * Once per Dispatch: the transfers are copied, each checked and translated, a
> + * reference taken on the program, the params and every buffer a transfer
> + * resolves to so the translation holds for the Job's life, and a handle
> + * returned.  The buffers the Kernel reads and writes are named at each
> + * submit, in &drm_thames_job_submit.
> + *
> + * The firmware enters the Kernel at the program's start as a C function
> + * taking one argument, the params' address, and returning a status word:
> + *
> + *     int kernel(void *params);
> + *
> + * That is the whole of what the kernel and the firmware know.  The Kernel finds
> + * everything else, its inputs, outputs, constants, grid and status buffer,
> + * through params, whose layout is private to the userspace driver that emitted
> + * the Kernel.  The program must have been created DRM_THAMES_BO_EXECUTABLE.
> + * The program and the params must be bound in the Context.
> + * The Job keeps the two addresses they are bound at when it is created, and
> + * every submission runs the Kernel at them.
> + *
> + * A Kernel asks for its Job's next transfer group with system call 2 (the
> + * SYSCALL instruction with 2, no argument).  The call returns to the
> + * instruction after it with every register kept but A0-A3 and RP.  The firmware
> + * submits the group only if the Job has one left and every channel the group
> + * uses has no transfer queued or running; otherwise it fails the Job, writing
> + * none of the group's transfers.  A Kernel waits for the tile event of its
> + * channels' DRU records with system call 3.
> + */
> +struct drm_thames_job_create {
> +       /** @type: One of &enum drm_thames_job_type. */
> +       __u32 type;
> +
> +       /** @flags: MBZ. */
> +       __u32 flags;
> +
> +       /** @program: Handle of the buffer holding the Kernel. */
> +       __u32 program;
> +
> +       /** @params: Handle of the params buffer. */
> +       __u32 params;
> +
> +       /**
> +        * @transfers: Array of &struct drm_thames_transfer, the DRU records
> +        * the compiler scheduled for this Kernel; may be empty.
> +        */
> +       struct drm_thames_obj_array transfers;
> +
> +       /** @handle: Returned handle of the Job.  Job handles are nonzero. */
> +       __u32 handle;
> +
> +       /** @pad: MBZ. */
> +       __u32 pad;
> +};
> +
> +/**
> + * DOC: Sync operation flags
> + *
> + * The low byte of &drm_thames_sync_op.flags is the handle type, bit 31 the
> + * direction; every other bit is MBZ.
> + *
> + * The one op accepted is a timeline sync object signalled,
> + * %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
> + * %DRM_THAMES_SYNC_OP_SIGNAL.  Handle type 0 and a clear bit 31, a binary sync
> + * object and a wait, are left for the client that will use them.
> + *
> + * - %DRM_THAMES_SYNC_OP_HANDLE_TYPE_MASK: The bits of the handle type.
> + * - %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ: A timeline sync
> + *   object, at the point &drm_thames_sync_op.timeline_value.
> + * - %DRM_THAMES_SYNC_OP_SIGNAL: The call's last Job's finished fence is
> + *   added to the timeline at the point.
> + */
> +#define DRM_THAMES_SYNC_OP_HANDLE_TYPE_MASK            0xff
> +#define DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ        1
> +#define DRM_THAMES_SYNC_OP_SIGNAL                      (1u << 31)
> +
> +/**
> + * struct drm_thames_sync_op - One signal of a submit call
> + */
> +struct drm_thames_sync_op {
> +       /**
> +        * @flags: %DRM_THAMES_SYNC_OP_HANDLE_TYPE_TIMELINE_SYNCOBJ |
> +        * %DRM_THAMES_SYNC_OP_SIGNAL.
> +        */
> +       __u32 flags;
> +
> +       /** @handle: A timeline sync object of the client. */
> +       __u32 handle;
> +
> +       /**
> +        * @timeline_value: The point on the timeline sync object, nonzero.
> +        */
> +       __u64 timeline_value;
> +};
> +
> +/**
> + * struct drm_thames_job_submit - Arguments passed to DRM_IOCTL_THAMES_JOB_SUBMIT
> + *
> + * A submission is an ordered array of Jobs, one Ring entry each, so a sequence
> + * of Dispatches is one call.  Per run the kernel re-checks only the Context's
> + * current state, waits on the implicit fences of the buffers the call names and
> + * pushes the Ring entries.  A submission of a Job while an earlier
> + * submission of the same Job is in flight waits for it, which the Ring's order
> + * gives on its own.  One Ring completes its Jobs in submission order; there is
> + * no order between the two cores' Rings, but the Jobs of one call run in the
> + * order named.
> + *
> + * The buffers the call's Jobs read and write are named here, one pair of lists
> + * per call.  They are for synchronisation alone: the first Job waits on the
> + * fences already on them, and the last Job's fence is set on them once the Jobs
> + * are pushed, for read usage on @read_handles and write usage on
> + * @write_handles, so DRM_IOCTL_THAMES_BO_PREP on a written buffer waits for the
> + * call's Jobs.  No flag turns either side off.  The call holds a reference on
> + * each buffer named until its last Job completes.  A buffer may be named once
> + * in the two lists together.
> + *
> + * The sync objects the call signals are named in @syncs.  Every op receives the
> + * call's last Job's finished fence, the same fence the listed buffers get; the
> + * call waits on no sync object.  The call is all or nothing: every Job handle
> + * is looked up and every sync op resolved before the first Job is pushed, so a
> + * call that fails has submitted nothing and signals nothing.  An opt-out of the
> + * implicit synchronisation would be a bit of @flags, appended under a minor
> + * bump.
> + *
> + * Every Job the call submits has its fault record cleared; a refused call
> + * clears none.
> + *
> + * A Job of the call that fails does not stop the ones after it: each later Job
> + * runs, and completes or fails on its own merits.  Only a Context banned after
> + * its faults has its Jobs not yet run cancelled, their fences signalled with
> + * -ECANCELED, and its later submit calls refused with -ECANCELED.
> + */
> +struct drm_thames_job_submit {
> +       /** @flags: MBZ. */
> +       __u32 flags;
> +
> +       /** @pad: MBZ. */
> +       __u32 pad;
> +
> +       /**
> +        * @jobs: Array of __u32 Job handles, run in order.
> +        *
> +        * At least 1 and at most 256 Jobs.
> +        */
> +       struct drm_thames_obj_array jobs;
> +
> +       /**
> +        * @read_handles: Array of __u32 handles of the buffers the Jobs read;
> +        * may be empty.
> +        *
> +        * At most 4096 handles in @read_handles and @write_handles together.
> +        */
> +       struct drm_thames_obj_array read_handles;
> +
> +       /**
> +        * @write_handles: Array of __u32 handles of the buffers the Jobs write;
> +        * may be empty.  Counted with @read_handles.
> +        */
> +       struct drm_thames_obj_array write_handles;
> +
> +       /**
> +        * @syncs: Array of &struct drm_thames_sync_op; may be empty.
> +        *
> +        * At most 64 ops.
> +        */
> +       struct drm_thames_obj_array syncs;
> +};
> +
> +/**
> + * struct drm_thames_job_destroy - Arguments passed to DRM_IOCTL_THAMES_JOB_DESTROY
> + *
> + * Releases the handle and returns at once; it does not wait for the Job's
> + * submissions.  A submission still in flight keeps the Job, its validated copy
> + * and the references on its buffers until it completes, and its fence signals
> + * as it would have.  The handle names nothing afterwards.
> + */
> +struct drm_thames_job_destroy {
> +       /** @handle: The Job. */
> +       __u32 handle;
> +
> +       /** @pad: MBZ. */
> +       __u32 pad;
> +};
> +
>  /**
>   * DRM_IOCTL_THAMES() - Build a thames IOCTL number
>   * @__access: Access type. Must be R, W or RW.
> @@ -431,6 +672,12 @@ enum {
>                 DRM_IOCTL_THAMES(WR, BO_MMAP_OFFSET, bo_mmap_offset),
>         DRM_IOCTL_THAMES_DEV_QUERY =
>                 DRM_IOCTL_THAMES(WR, DEV_QUERY, dev_query),
> +       DRM_IOCTL_THAMES_JOB_CREATE =
> +               DRM_IOCTL_THAMES(WR, JOB_CREATE, job_create),
> +       DRM_IOCTL_THAMES_JOB_SUBMIT =
> +               DRM_IOCTL_THAMES(WR, JOB_SUBMIT, job_submit),
> +       DRM_IOCTL_THAMES_JOB_DESTROY =
> +               DRM_IOCTL_THAMES(WR, JOB_DESTROY, job_destroy),
>         DRM_IOCTL_THAMES_BO_BIND =
>                 DRM_IOCTL_THAMES(WR, BO_BIND, bo_bind),
>  };
> 
> --
> 2.54.0
>